VL - Node API

vvvv beta question
,
1

Ok so couple of quick questions, just the basic bits that comes to my mind about node writing:

  • Can we have connection handlers (so we can have nodes that respond to connections).
  • Is it possible to have connection only nodes (not part of any dataflow/execution model).
  • On a more advanced way, can nodes themselves decide on their connections?
  • Can we have a way to allow to connect multiple “outputs” to a single “input” (some form of implicit cons in some ways)
  • For now it’s pretty easy to create a node, compile and have it imported in the nodelist. Is it possible to dynamically register nodes to the nodelist (bit like factories are doing at runtime at the moment).
  • Can a node update it’s own registration/definition information (like a dynamic plugin works in some ways).
  • Will we be able to define our own regions (guess that’s not a early feature, but I can see huge possibilities into be able to do this).
2

spread of spreads was not enough so let’s implement a region of regions

3

spread of spread is just the beginning antokhio: 4D spreads: Spread<Spread<Spread<Spread>>> ;)
regions are just simulating scopes like
if { foreach { stuff { } } }

4

alright vux,

let me roll out the answers by asking questions first :P And let’s try to be as slow as possible so that we all can follow and get the best outof it. ;)

Can we have connection handlers (so we can have nodes that respond to connections).

What is a connection to you? Or better: what would you need a connection to be like?

  • Is it the physical link that you see in the patch? And would you like feedback on those short distance connections only? Let’s call them SourceCodeLinks. They only change when a user connects or disconnects. E.g. they model the connection TO and FROM a switch, not the connection the switch is doing for you. They exist in the patch (never from patch to patch) and have 1:1 relationship to that visual link. They don’t model the underlying connection that the switch is supposed to manage (when run). They don’t model any other compplex connection, that’s established via combining subpatches, only always one link. (no iobox magic, no switch nor getslice magic like known from nodepins in vvvv). They also only change once independant of how many instances of your patch may exist. They model the ONE source code link that the user connected.

  • Or are connections something under the hood, that might get switched without anybody patching - like via a Switch node? Would you want a feedback on that? Let’s call them RuntimeLinks. They would also always tranpsport YOUR model of a data source to the data sink. So on a connection feedback you get a grip onto what source your sink get’s connected to, not a random stupid pin, but the actual data source (no matter if this comes from within another patch or via getslice, switch or whatever). You get informed for every instance on runtime.

I just had a try on the latter one. Which is basically something that you probably did several times already when using the vvvv plugin interface: Building an own graph, that manages nodes and connections.

Anyway for reference for everybody and just as a sketch. I put that under girlpower\ProcessGraph in latest download. it doesn’t do anything yet, but it shows (together with the attached source code) how to react on connection changes when you model your data sources and compare them via reference equality.
It sounds brute force but has the nice side effect of working in all situations not only when source code is changed. It would also work in an executable later on.
It is build on top of the process node idea, that is living nodes. By that it doesn’t work inside helper patches, but really is all about managing a persistent graph instance and getting feedback on the connection changes of that living thing…

Fist tell me which of the two ways of thinking connections is the more appealing one. For you. Or does it depend?

Let’s see for the other questions after we get an idea of what direction we need to take here.

ProcessGraph.zip (2.3 kB)

5

Both aspects are useful, and each fits a use case scenario.

Source code links, as you call them, are vital in case you want to define you own region (s), since you need to know that user action in order to build whatever magic you want under the hood (totally agree that a link is just a visual representation, and does not reflect what’s behind the scenes, which is actually even more true in VL , a bit less in vvvv45).
Those are just a hint to the runtime in some ways.

Runtime links are useful in other ways, likely in VL it can be modelled a bit differently (using properties or constructor injection), In some ways you can indeed model this in your main system (when a property is assigned, your node does dispatch an event to some listener), after I’m just wondering if we can have a way to have this done outside of the system, or if we give that responsibility to the current datatype, investigation needed.

I should have split the question tho, since actually the 2 more important features for me are dynamic registration/definition update.

Since not everything is easy to do via patching, code editor will still be a really important feature to have, so the idea is :

Build a simple type/node via integrated text editor (same as per dynamic plugins), can be prototyped with a stupid textbox for now, when we save, we do a compile task (as before), but need to register new assembly, so we also need to notify you runtime that those particular elements have changed(or that a property doesn’t exist anymore…)
I guess this is already possible in some ways, since when you patch you do pretty much just that (recompile and update type definition), so I guess it’s just a fact of grabbing the right calls to your runtime to allow update.

Also for now it’s pretty hard (not possible?) to have dynamic input count on some nodes (think Concat node that has 4 inputs), maybe grabbing params keyword or some extra attribute to allow dynamic inputs?

6

right, i was guessing that you wouldn’t be satisfied ;) let’s step back and have a look at the whole system before getting into detail:

We have

  • an IDE with support for VL. Its main component is a patch editor to edit the VL Source Code.
  • a dynamic incremental compiler that emits IL Code for changed VL source code.
  • a runtime that swaps the currently running program and tries to get as much state as possible from the old pogram into the new program. (happening after each source code change)

The patch editor tries to hide the fact that it is just displaying source code. It has the ability to pick up data of a running instance and display it additionally.
But that only blurs the separation between source code and the running program that is very central to the new system.

But the most basic idea we maybe forgot to mention: VL is a language very much like other programming languages: it is of no importance in which order certain source code get’s built by the user. There is nothing like a semantics for a source code change. There is only semantics for source code itself, no matter how it evolved. Like with other languages really. We don’t treat VL source code as a living thing but more like something static. When it changes we try to be clever and benefit from the unchanged parts, but still we try to do it in a way that semantically is the same as recompiling everything. This means a change is basically treated as completely new static source code.

Because of this we didn’t have a need for callbacks on connect yet. It just triggers recompilation like any other source code change.

Also: the source code is a document model like structure. The running programm (e.g. your node) is totally detached from that and only IL Code, which might get called, or a data type that might get instantiated or not, dynamically or statically (like with a process node). But the main point here is: it is very much separated from the source code and therefore can hardly react on source code level events.

Now: i’d like to split the answer to your questions into:

  • VL Language related
  • IDE related

VL Language related

  • Runtime links can be modelled as described above. It would be interesting to see for which problems they are just the right tool.
  • Source code links and events are not planned for now. We don’t have an API for VL source code yet.
  • Nodes deciding on their connections: You’d need to think and tell us if this can be decided within static source code or not. If yes it might be possible to do this once we have such an API, but you’d need to analyze source code, which gets very hard as soon as you want to analyze bigger source code graphs, especially when leaving patch boundaries. This sounds really hard to do. So the question would be: is there really no way to describe connectability with some sort of composed data type, so that our type inference can jump in? I fear the separation of source code and running code is not really beneficial for your nifty connection logic…
  • Dynamic pin count on cons and others. Of course on the short term list.
  • Multiple sources into one sink: Is not well defined. I would rather want to see such a feature outside the central VL language or just for some special collection type. Or maybe in a special patch type that is not all about data flow but more about relations. In VL itself i would hope that the dynamic pin count feature can jump in and by that visualize order.
  • Own region: this would open up the language and the compiler. It is thinkable, but it needs time and is not realistic now.

IDE related

Since some things you ask for might be hard to do within VL itself, it might be more beneficial to focus on the way we enhance the IDE: On a very high level allowing different patch types and editors.

A VL patch could then benefit from nodes created with other editors. The basic building blocks that we can import into VL are operations nodes, process nodes and data types. Your main job would be to create such nodes and tell us about them or remove others. This is not possible for now but worth having a deeper look.

We will need more time here, but will see that a potential c# editor gets integrated in the same way that enables us to integrate other editors as well. All of them contributing dynamically to the pool of available nodes and types.