Yop, had quite a small ever growing collection of ideas on how to improve vvvv, and thought I would share it.
As every software there’s stuff I like/dislike, so let’s get rocking!
Things I like:
Quick/Minimalist/Simple but easy to use interface. Just all connections/shortcuts bring your productivity to the max.
Spread/Bin size, once you get used to it, is just great, the autofill makes life so easy.
Animation filters are second to node, look at any software which already has built-in damper/oscillator/deniro…
Plugin Interface (ah no never used that one, so can’t really comment on it ;). Bringing any of your code for this exotic device, and interface is really striaghtforward.
Cool devvvs which bring new features ;)
New dynamics is great too, i dont use it so much myself apart for quick tests, still enjoy my good old visual studio, but that brings some really nice ne possibilities.
Great community : people who chatted with me on some subjects at node will know what about i’m speaking of.
Ok know let’s stop praise the devvvs (couldn’t praise them enough but that’s not the point of the post) and give them some stuff to think/work on ;)
Gonna split it in different parts:
Graph level
UI tied to mainloop: having my gui dropping at 1 fps when i set my mainloop/or on the other hand, having my patch going down 10fps when i move an iobox is one of the things which could be a major improvement.
No ability to change the evaluation mode for a node (the AutoEvaluate in a plugin or any native node). Means it become very cumbersome to use a lot of switch/s+h to make sure the “whole” subgraph is not evaluated (specially when using LFOs, Dampers…)
Disable a node: Option to just fully disable a node would rock (specially for a decently big subpatch). Just in that case do not call evaluate as all, while still having the advantage of resources preloaded in memory.
Singleton nodes+ alias: Whereas S+H is quite handy, it doesnt cope very well with multiple variables type. So we could declare a node as singleton, and then create an alias in a subpatch (inputs would then be disabled and another color). Internally we could just evaluate the singleton only once, then create some internal links. When you have some config files and xpath queries that you need in several places can come in very handy.
Threaded node: Any node which is “heavy” could be marked to run in a thread as soon as all it’s own dependencies are satisfied, so other “smaller nodes” could still be run in the mean time. That would need to bring in some priority system (you want your heavy node to be executed asap, not last otherwise it’s useless), also you need some node evaluation statuses, so threaded node can notified when it’s done. I would not think that doing auto threading is a good idea. but more give the option to the user to decide (non threaded by default), as a node evaluation speed can change drastically upon some parameters.
Threaded node (again): node which executes outside of the mainloop, with Input/Output pins synced in the evaluation.
Node pooling: Some kind of advanced CreateNode. You create a list of nodes which have the same inputs outputs,and register them to a pool, with a preload/switch option.
More information about graph evaluation process: Some kind of inmproved timing node. Also ability to “watch” a node (evaluation/error) and get the data into a patch would be very useful for profiling.
Node level:
Threading+SIMD: lots of nodes could benefit from new pc architecture, we are not gonna have faster cores for a bit, so using cores+new possibilities is definitely something not to neglect. You can obviously do some cool stuff with GPU, but cpu still stands in between, I class them at friends as they both need each other. Ideally the user could choose if he want to enable threading/sse optimizations, as they useless (and make things slower) for low spread count.
Other optimizations: There’s a lot of little tricks which can be worked on depending on usage, thinking about Select for example, could have a High Spread mode/Low Spread+high select mode
More control on evaluation: For example we could decide if a node executes blindly every frame, only on pin change, or have a bang to specifically mention we want it to evaluate.
Common model for devices nodes: Midi can have the same device on multiple nodes (so somewhere a singleton is kept), UDP server can’t (so the node itself reserves the port). Ideally all of them should behave identically.
Any node which can’t work/fail should have a error pin: Reader (File) should for example be able to tell me if it can’t read a file because it doesn’t exist, or i don’t have read access to it, already in use…
Pin Level:
Ok let’s start by the big one. The biggest problem I see in vvvv now is: non consistent datatypes. Color is double, Value is double, Matrix is float. That would mean somehow we could say “change matrix as double and that’s fine”. But problem is most of the render engines/gpus are optimized for float, meshes are float, textures are byte/float, and same is color. so having a proper consistent datatype would avoid loop/cast and enable direct copy, which is a hige performance improvement, and also helps a lot to make code cleaner. I know it’s a VERY drastic change, but I definitely believe the reward is worth the effort (did i mentionned box2d/bullet/physx/any good geom library also use floats? ;)
Float is cool, but Float16 is better :) Cpu read memory by block of 4, so a float could be 16 bits aligned by default. Apart from the improved read/write operations (which also work on gpu, copy is around 20-30% faster with aligned data), it also enables the use of simd as mentionned above (you can still do sse with unaligned data, but a loadu call is so slow compared to load it’s very often not worth it).
Color/Transforms: Would of course also benefit from alignment (even more as color = 4 floats, matrix = 16 floats = 4*4 floats), so a lot of operations could be heavily streamlined for it. Not going too much into details as it would be 10 pages on it’s own ;)
GPU types + conversion: Having data being stored in gpu instead of cpu and options to bring it back and forth. Nodes could have an option to switch between cpu/gpu types. Also some operations like mesh/texture building could have a huge improvement using gpu types, so having a streamlined system would bring vvvv to the top ;)
Categories for configuration pins: Would be a nice thing to have so when you have a lot of those, so they would be sorted somehow.
Polymorphic types: A bit like the node type, but we could have more inference (like inheritance). Also being able on the first connection what we want to accept (like is i connect a Box, which is also a Shape, if could choose if the node wants to accept Box only or any Shape).
Some device related pins as types. Thinking UDP/Midi for example. Would be great to have a UDP stream type which sends an event when data is received instead of just a string. This kind of data should not be tied to mainloop.
Thanks for reading, got lots of other ideas but I’m tired typing now ;) If you want me to explain some stuff more in details feel free to ask.
PS: In thread type you have statement,bug,feature,question, maybe you should add “Long post of random stuff to keep devvvvs busy” :)
yeah disable a node would be amazing! My club visuals doesn’t allow for loading new patches in live so I often face performance overheads from lots of nodes. Now we have node browser this had got to be possible? view all disabled node and alike. What’s the reason for not being able to disable nodes? can’t see why it hasn’t been done before and guessing their must be a reason.
Hi
great ideas there vux, my 2c is a grid and snap to grid function for the UI, that would make faster the tidying up of a patch.
As mentioned already somewhere else (Dottore?), colored connections would be great, in an automatic or manual fashion.
I would like to spend a comment on the export feature.
Since last version we have crack.exe so any user can have an up and running vvvv box with few effort, just some clicks.
I remember (with my broken ram memory) that Elias at meso said me that deploy is still a problem.
I’m thinking about a packager, the idea is not to have a stand alone executable, but an installer.exe that extracts the root folder, with all the plugins modules effects resources ecc folders, put some shortcut in the menu and an uninstaller somewhere. vvvv would be a prerequisite, as you have to install java to run a .jar, so our german friends are not bypassed and if it is a commercial solution it would need the vvvv license as well.
I am a mathematician but I don’t think I’m crazy when I think that vvvv could be used in a lot of areas, I would like to use it for Business Intelligence where I work in www.virgilio.it . I’m sure that marketing people would really like to explore data using a device like kinect in a presentation, specially if they are in the top of the company. I started showing the svn log patch I picked up from last node and there was a good reaction.
Of course it is not easy to use a tool like vvvv, since nobody knows it in Business Intelligence, probably the boss of my boss would say something really dangerous like “I don’t know what are you doing …”, but I know that some day I will have time to show some cool stuff, connected to a device, and then people would say:
can I use it?
then it will be necessary to have an installer, cause I can’t explain la rava e la fava to users, I mean something that you just click and works and stop: with hidden patches cause vvvv nodes are too trippy to show them to a user, maybe only the renderer should be visible in that kind of “production mode”.
the on off thing
turning nodes or patches off… yes yes yes
the compiler
to be able to deliver ready to go applications is an old dream, i dont dare to ask for this, probably brings a whole new bunch of problems and questions, stopping devs from more important stuff like stability ;)
the packager
probably something kind of impossible to do, vvvv is not after effects and can catch content in various ways, build up filenames dynamically, get them via additional xml files, load patches dynamically. how should a packager analyze all this stuff, collect it and put it in the right place. i guess this would only work with simple projects. i gave up on this idea.
one idea concerning threading/multiple (main)loops that popped up at node after gregsns talk i still like very much:
basicly like boygrouping, just on another thread instead of another pc
one would just mark a bunch of nodes or a subpatch, hit some keyboardshortcut and shwoops, the marked bunch is calculated in another thread. the user has to care about sync (if needed) him/herself. for connections between those branches the always the last ready value is transmitted.
different thread could have different mainloops (subframecalculations, a dedicated very fast audio-stuff-thread,…
branches/subpatches could be more easily disabled, they would then just output the state of their last calculation (as they would do anyways)
the same thing could be done with the gpu calculations :)
one would have hell of possibility to optimize performance of a patch
and much more cool points about it i guess
concerning the packager:
had a semiworking plugin for that, before all this dynamics stuff etc came. now it would need a major rework to be usable again.
the problem, as stated, are all the dynamic resources (files loaded from som crazy string operation, or through the output of dir and some filters below selecting only a few filenames)
had it working parsing for anything like Filename and all the Dir nodes, but still it would only get half the things you really need
disabling is indeed one major advance, which i also believe would not be extremely hard to implement (compared to threaded graph evaluation which is much much more complicated), for quite a very high reward.
About packaging it’s about right, for most of the cases where i need deployement i generally:
Start with a clean beta (no plugins/modules/shaders as all)
Add only what i need
Make sure any dynamic resource folder can be changed in a config file (xml/ini)
Then innosetup can do wonders (And vvvv.exe myroot.v4p /shutup is pretty cool too to hide patches ;)
For more heavy needs packaging (string parsing directories and such) i generally don’t bother, just makes things so complex it’s not worth the time.
Error pins can be done as custom plugin, but sure wouldn’t be too hard to add a little error pin to reader (File) and his friends, saying what happended, avoids node duplication, and streamlining (pin names and such) is always good.
On a similar note to @kalle’s patchlet idea, i’d like to be able to set default options for different types of nodes, like having IOBox (Value Advanced) show up with the SliceOffset pin hidden and a Minimum Value of 324 or something like that when i create them.
You could store the definitions in a patch file somewhere, and vvvv could search through that patch for an instance of whatever type of node you’re creating, and copy that into the current patch.
Also, more control over the file search paths. I’ve been doing a lot of plugin dev, and Visual Studio generates duplicate dll files in “obj” directories in each project. So when I launch vvvv from VS to debug a plugin, vvvv finds all the duplicates and then each node shows up multiple times in the node browser, and i have to figure out which one is pointing to the right dll. If there was a way to exclude files that match a pattern (such as “/obj/.dll”), it would prevent vvvv from loading the duplicates.
ok, since my initial answer to the very insightful original post was rather short i thought i’d give some more detailed comments on some of the mentioned points…
decoupling of ui / execution thread
this an obvious win for several reasons e.g. no more lags on iobox interaction, better use of multicore cpus, headless patch execution…
more control over evaluation
is certainly for more advanced users, but smart default settings would save the day for newbies too.
globals / singleton nodes
has the potential of saving hundreds of more or less redundant nodes/connections from complex patch networks.
having float16 as standard pin datatype
i cannot stress the huuuuge win of this enough! there’d be possibilities for massive performance gains to be made with this, multithreading and SIMD optimisations. think about ten thousands of spreaded transforms… (hello eric :P)
gpu types and conversion
would be heaven for everyone who tried to get some gpu generated content back to vvvv.
i’m totally aware that some of these points would require a rather painful transition process and also probably a way of dealing with legacy stuff, but i think it would be basically really worth it and would further future-proof our beloved vvvv. (i think most users would not have a real problem with deprecation of certain nodes/techniques as there’ll always be the older vvvv releases for dealing with that)
anyway, that’s my 2 cents on this…
the patchlet-thing of course also is meant to store/access “preconfigured” single nodes.
the fun thing about that is:
if you copy a selection of node(s) the complete xml description is already as plain text in the windows clipboard.
ever tried copy/pasting a selection via skype? pure fun :)
so basically it would just need a kind of clipboard manager…
i know it would be insanely difficult, but it would be nice to be able to extend the application at itself via plugins (or some sort of scripting language)…
i’ve probably been spending too much time recently writing plugins for http://ckeditor.com/, but their is something totally amazing about their API, and the way that plugins can dynamically modify the functionality of basically any component of it.
it would be nice to be able to add editing macros (e.g. horizontally distribute the selected nodes to a specified width, or connect the first output of a set of nodes to the series of inputs on another node). It’s been done for Max/MSP (http://code.google.com/p/maxtoolbox/). It might be possible to do a bunch of that sort of thing using SetNode. The Finder plugin is a good example of that general level of control, so it’s possible that the capabilities are there in the API and i just haven’t found them yet. More extensive documentation (beyond just a class reference) would be really useful for that sort of thing (though i totally understand the amount of effort it takes to put together that kind of documentation).
I remember the talk we had about this boythreading (eg: subpatch with it’s own mainloop), and that would definitely be a great feature, specially for any device related code (like you could have a udp server graph which runs at full speed with no need for tricks for missed packets, or you could have a slow camera processing patch not slowing down your global mainloop).
@kalle:
patchlet clipboard is very cool, i’m sure i can do a small gui plugin to handle that for you ;)
@m4d: float is definitely one thing i would crave for, over any other feature to be honest. (they float32 by the way, 16 bits is the alignment ;). There’s nothing much painful than having to convert data from doubles to float. it’s a fast operation nowadays, but nowhere near as easy and fast as just using copymemory/copy/memcopy. You could also easily decide to enable/disable aligmnent (at the end they just floats, it’s just a way to organize them in memory). So yes, a lot of the standard, general usage nodes (operators, transforms…) could be much improved performance wise.
