Forum

Schéma - Streamlined visual programming for physical devices

Schéma is an extensible multi purpose visual scripting toolkit implemented in VL + Skia.

It’s aimed at atmosphere control. Eventually it can be your lighting desk, your VJ tool, your laser source, pixel driver, synth or whatever you can think of as it aims to fully leverage the powers of VL and vvvv and be open and expandable.

Currently it’s installed permanently running headless in one club, also tried it out in few small light installations and it was pretty fun to use.

But there is still a lot to implement, refactor and clean up so it’s not being released just yet, thus the WIP section.

If you want to read more in depth, there’s a WIP docu:
https://schemadocs.domj.net/

For a general overview go here
http://schema.domj.net

I will provide this thread with new information (and hopefully some videos of it in action) as it becomes available.

Also I have a workshop planned for the weekend of 25th and 26th of May where I’ll share the program and go through some basics. It’s in Prague so it might be a bit far for most of you to join but I’ll try to set up a stream as well.

More info on the workshop here

8 Likes

Well alone what I gather from your website and documentation: WOW, BRAVO!

Wow this looks great! Exciting days to work with VL :))

@Elias Thanks a lot for the kind words, it means a lot to me coming from you and thanks again for helping especially with the memory leak issues and big thanks to the rest of the vvvv team for dealing with my silly questions and requests all the time.

At the moment I can hardly imagine doing a project of this scale at this speed in anything else but VL/vvvv, the runtime editing really speeds things up and supports a lot of experimentation and little tweaks while still delivering a proper .NET application. Sure I keep running into bugs here and there and some unfinishidness (such as missing recursion, instantiation by name and needing to use each class in a runtime patch if I want to deserialize it) and sometimes a runtime restart is needed but more often than not it’s smooth sailing. Using vvvv modules as fillin for features yet to be implemented natively is also brilliant and maybe by the time I have everything in VL, gamma might even support exe/assembly? output :)

Speaking of which, I’m still a bit unsure how to design a proper plugin architecture using VL. I can reference various documents, string them together and each injects its own stuff into the runtime (new Blocks, extra routines) but it still requires some extra manual patching effort on the side of the user. Ideally this would be just dynamic loading of assemblies from the patch itself and a common plugin entry point would be used to add functions. Will this be possible after gamma export or is it too far out? Or maybe it’s already quite possible and I’m just missing something?

Might be best to just talk about this stuff personally next time I join a meetup :)

@systray27 Thanks a lot! I’m keeping a close look at your VL.NewAudio library, the CV demo sounds pretty damn sweet already. I plan to eventually have Audio Blocks as well and a big part of the program is audio analysis which as of now runs from vvvv nodes but I do want to fully support gamma eventually. I was looking into NAudio as well and it looked promising so I’m really glad someone took up to make a proper VL library on top of it.

Also a little update might be in order, don’t have time to make a big video right now but here’s one with a few things that I’ve added very recently and think might be worth sharing. On top of some much needed cleanup of my convoluted experimental patches (of which I still have plenty to do) I’ve added:

  • A grid in the Schematic view that helps with orientation (is isometric view you even get little fake shadows :) )
  • A new Repeat block that can record and play back whatever you throw at it. The recorded states also get serialized and can be loaded back (automatically provided by VL). This does bloat up the XML file considerably though so I’ll need to come up with a custom solution. Might prefer to just ignore the pad for saving now but don’t see any way to do that without implementing my own serializer for this particular Block (which is low pri now). Bumping those serialization attributes discussed somewhere I guess. And maybe some On[De]Serialiaze methods could be nice as well.
  • Extended Drivers to support Boolean properties as well. Now any Boolean property can be driven by Value generating Block Stacks as well. Boolean = Value > 0.5 but the threshold can be adjusted by moving the Driver mapping diamonds (it will always end up in the middle of them).
  • Added a new progress feedback Block UI Element, which can be seen running on the new Repeat black. Also put it on Pick.Alternate.
  • Visual feedback for a given aspect is provided on any Block that contains it (eg. you can see Color feedback even on generic non color specific blocks - picks and operators for example)

The video shows 4 light Fixtures all loaded with identical Block Stacks with Repeat and Constant blocks controlled by an MPD218 Midi Controller. The Record Driver is set up in such a way that it takes different pads depending on the Fixture’s channel.

Not seen in the video but also of note is the reworked Browser, now using the same Title drawing routines as Blocks for visual consistency and also now it understands categories, which work as folders. Pressing Esc goes back a level. The placement and styling of the textual prompt has also been improved.

And now it’s also finally possible to define your own groups via the UI just by typing the new name in the Browser and pressing Enter (it wasn’t before).

2 Likes

It’s easy to record the screen so here’s some more but still without commentary.

This just shows the general workflow and also the Browser.

WOW @domj impressive stuff, loving the website!

@domj You’re crediting the wrong person here, have got nothing to do with new audiopack. But cool to hear what you are going for here!

@systray27 my bad, it’s synopia. Guess I got confused by the av synthwork and ADSR questions you were dealing with. The synth’s visuals are really interesting btw, great sync with the MIDIs.

@ravazquez Thanks a lot! Keep your eye on the blog.

Yesterday I went to Berlin to join the lovely meetup #9, got to do a presentation on Schéma and managed to record it, so here it is. Probably most in depth video available to date.

Just put up a new blogpost detailing some of the recent additions to Schéma. It’s kind of long so I think it’s better not to copy it over here, but feel free to reply here.

http://schema.domj.net/last-week-recap/

@domj thanks for the compliments, I will try to check Schema soon. Who knows it can help in my work :)

@domj: this is great work!!!

@sinus Thanks a lot!

Here is some recent development from today, I’ve added Blocks to support “Painting”. That combined with OpenVR integration and other tools in Schéma allowed the creation of this interactive prototype:

Behind the scenes development livestream recording:

1 Like