i recently had a look at the Newtek NDI sdk, which allows for easy, high quality video streaming with low latency over GBit ethernet networks and seems to be supported by lots of hardware and software vendors (and is royalty free!). NDI is really fun and quite easy to work with…
NDI support was requested several times and it is already possible to use it from VVVV via Spout and Spout2NDI & NDI2Spout. but using extra software always adds risks or is flaky for some reasons. having a native solution would therefore be preferable.
here is what i’ve got working until now:
a basic NDI video receiver for VL and VVVV (via UploadImage)
a (even more) basic NDI video sender for VL
what’s missing:
getting rid of occasional hiccups that at times crash everything
clarity about licensing from Newtek. as NDI is available in many open source projects (VLC, OBS, …) this is probably not a big deal but i’d like to be sure before releasing.
audio: having NDI available as a VAudio source is probably the nicest thing to have. but code-wise i have to start learning from scratch here. maybe @tonfilm might give me a hand here.
time: i’m at a state where NDI is basically running but i’m quite busy at the moment and cannot say, when i can continue working on it. but as soon as it is stable and i’m clear about the licensing i’ll give you something to play with.
i wanted to wait with audio patching in VL a bit until we have a better understanding of how to build object graphs with VL, which is what audio rendering is mostly about. recent experiments with the xenko scene graph are promising, but will probably take some months… so maybe just go without audio for now. or have a look at the VL audio template in the VVVV.Audio pack, it explains most of the audio rendering in the patch.
@motzi !!! Great!!! Im looking foward to test the NDI sender
can we expect a similar performance like vvvv->spout ? or its a more intensive CPU ussage?
regarding the NDI sender: for now there is only a sender for IImages in VL, so you can e.g. use the Skia offscreen renderers output to send it via NDI.
sending the output from a DX renderer is a different thing as you would need a readback to get the pixel buffer from the graphics card (e.g. using the PixelData (DX11.Texture2D) node. reading back data is supposed to be quite expensive itself and afterwards the video compression has to be performed as well. i did not perform any tests though but would not expect to be the performance much better than with spout.
low latency, high quality video video compression is a demanding task and we totally rely on the libraries provided by NewTek here. in code i just pass a memory reference to the pixel data and the rest is handled by the library itself - we don’t have much influence on what’s happening here.
i did some tests for the perfomance of the NDI receiver and i’d say it is comparable to the code-example provided by newtek. regarding the sender i’ll have a look at performance the next time i’ll sit down with the code.
At the moment many people is using vvvv->spout-> Resolume, and Resolume to NDI
or spout to NDI sender ( in my personal experience this is way more unstable )
I have a few use cases were NDI out ( and also in! ) directly from vvvv will be super useful.
If you are planing any support campaign I am happy to contribute
ah, how nice is that!
even nicer - Renderer->AsImage->NDI sender works out of the box!
the downside: it’s quite cpu heavy. the problem is that everything (readback with asimage and NDI encoding) now happens on the same thread (opposed to having Spout2NDI doing readback/encoding it in a separate process). a quick test with putting the NDI encoder in an AsyncLoop resulted in image tearing :(
for simple scenarios it might be enough but more demanding patches this might be a problem…
hiccups are mostly under control and i’m getting closer to calling it stable
i did some tests, comparing my plugins with Spout2NDI and watching the outputs simultaneously in the “NDI Studio Monitor” application in overlay mode.
the NDI plugin sender works smoother (less stuttering)
NDI plugin has slightly better latency
the NDI sender is really ressource intensive. 1920x1080 @ 50fps does not leave much overhead for VVVV. lower resolutions are better (use less ressources).
also, you need to set a frame rate and NDI’s internal clock will block the thread for the next frame (i.e. if mainloop is set to 50fps but NDI sender is at 25 fps, the mainloop will still drop to around 25 fps). setting mainloop and sender to the same frame rate worked best for me for now.
audio is still missing. @tonfilm - do you have a code example for how to create audio buffers (AudioSignals?) for VAudio? basically i get a plain pointer to a byte buffer (planar or interleaved) that probably just has to be copied…
Amazing work!
Its working exelent here, I need to test it on a second machine with 3 monitors ( 1 4k ) to compare a few things
But for the moment, using the old screenshot node on fullhd was way to slow, now im using NDI ScanConverter and your receiver node and its way better. Amazing work @motzi
this is vvvv plugin version build with vvvv plugininterface. few years ago i tried this approach and recently updated. it not support audio but sending dx11 texture through network works well.
i think motzi’s VL.NDI should be official plugin for vvvv/VL in the future, but you can try both now;)
here is one more little update. i changed the Guid of the dotnet NDI assembly so VVVV does not get confused any more when you wanted to test mino’s VVVV.NDI and VL.NDI side by side without restarting VVVV (we both based our work on the same dlls but they differ now internally).
also i fixed a minor thing when creating frames to send.
just download the vl.ndi zip from above and open vvvv/girlpower/NDI Overview.v4p (no install necessary).
then just open the helppatch from the receiver. for receiving something you need a NDI source on the network or on your PC. i’d recommend to download the NDITools by newtek. they include e.g. a program called scan converter, that does a screencapture of your PC and announces it as an NDI stream on your network (link to the NDITools is in the helppatch).
Hi @motzi, this is great stuff, thanks for the hard work!
I am trying to use Vl.ndi to send an R16 or R32 float texture across instances of vvvv. The problem I face is that when I use the Receiver node combined with UploadImage the resulting texture has a B8G8R8_UNorm format and my data is corrupted.
So I guess I have two questions:
Does Vl.ndi apply any sort of compression or conversion to the texture?
Is Vl.ndi suited to send data textures across multiple instances of vvvv? (Say 1 channel, 16 bit 1920x1080 float texture @30FPS)
I know it could also be the UploadImage node choosing an incorrect format but I wanted to run it by you first.
