VVVV.Value.Texture_UDP.vl (12 KB)
So I previously used spout to stream a depth texture, from a kinect2 so wonder if I could roll my own spout-esque patch. There is nuget of snappy.net, and it seems to work in VL, compress(codec) takes an array of bytes and compresses it, but I’m lost at the start, of course, with how to create a mutable array to store them in. Any quick pointers or help, with notes, would be lovely, thanks…
Are you sure you want to go the CPU way? Spout is only sharing a texture pointer, no real data.
So if you want to have a byte array from a GPU texture you need to do a readback, which will hurt you fps a lot.
If you have CPU data as IImage there is get data to retrieve the bytes.
If I want to send it over a network it has to be brought to cpu, obviously I’d like shared texture received inside vl but that not possible I think?
Tcpspout is discontinued and you have to select the receiver manually, if I could hack something together it might be useful, ndi won’t work as it’s compressed in very lossy way.
Although as there is a Kinect node it’s doesnt need shared I’d guess
great example to show the convenience of vl. just nuget the lib in and with just a few clicks you can get snappy to work without ever stopping vvvv. this particular library also has good code-documentation so all the input and output pins of the nodes come with explanations that helped me figure out how it works.
VVVV.Value.Texture_UDP.vl (15.8 KB)
note the Create [MutableArray]: this is probably inefficient as it creates the array every frame. this could be optimized by creating it only once (on create, with a max size that works for all images you wanna compress) and then reuse the same array every frame.
not sure i understand correctly, but yes, no shared gpu-textures yet for vl, since we don’t have any 3d rendering there yet.
also not sure what you mean by that?!
not sure i understand correctly, but yes, no shared gpu-textures yet for vl, since we don’t have any 3d rendering there yet.
From/to shared memory to CV and/or Skia would be useful though? I’m not a fan of heavy CPU nodes in vvvv especially when your trying to offload some of the processing into VL.
By kinect, I mean that the thinking being this was to send a 16bit kinect depth image via a network, and VL has a kinect node, but now I see CV doesnt like 16bit single channel images either, so the the greyscale node complains :(
But snappy works, which is cool :)
For the why I was trying this, MLF have a multiple kinect installation that I used TCPspout to send the depth from nuc’s with kinects attached to the main PC, which works but you have to manually set the recievers on the main pc, this was thinking of a workaround using VL.
huh, i didn’t see at first, they offer a much simpler version of the compress operation:
so it should really just be:
KinectDepthToSnappy.vl (23.5 KB)
note the FromImage is customly defined in this document. will add this to corelib soon.
hope this works for you?
Crashes when I load the patch…vvvv.exe-exception-2019-07-19.log (33.1 KB)
I’ll try updating the alpha…
Clean VL install hang on load, so reinstall kinect and opencv this time it opened, to 1 second of depth image, then crash :(
Doesnt load in latest gamma preview, missing some core stuff…
indeed, i see the problem. i’ll get back to you on this…
if you want to visualize the image you can use a Normalize (OpenCV Filter) to get human understandable RGB/BGR from a 16 bit single channel image. You can also use the advanced ConvertTo OpenCV node to change between mat formats altough its easier to mess that one up.
@ravazquez I dont want to look at it, I want to lossless compress the image and send it over a network ;)
KinectDepthToSnappy.vl (40.2 KB)
I got the patch working by dissconnecting the through put to the uncompressor. Tried to get a UDP node in there, but packet size is too big, and I’m lost as always with how to make it work.
In order to look at help patches I opened gamma as my helop viewer, which works, and if your not going to get muliple windows open can I suggest dirty hacking by opening a 2nd instance of gamma for help patches, it really is unworkable looking at help patches and not being able to see what you are looking for help for at the same time, computers are meant to expand the concept of books, not revert to them, and in fact if I had a print manual, I’d still be able to see what I am doing and what the help is! This is one of the things that make VL unpleasent to work in, or certainly learn, if there isnt a dev there to ask…
might be better to use the higher level networking library like NetMQ for this, in 2019 you shouldn’t have to deal with raw sockets anymore:
so the problem with the patch i posted above was caused by the ToImage after the Decompress, which was only there to show a proof of concept. you found it to work as you disconnected that part.
now:
indeed the data is still to much for udp even after the snappy. the attempt you made in your patch (slicing up the data into smaller junks) doesn’t make sense for udp either. you’d not have it guaranteed to get all the junks and even if they all arrive they could be out of order. that is how UDP works.
it would work with TCP, where you’d not have to care about slicing things up. only need to provide a framing that you can then tokenize for on the other end. but TCP in vl is experimental. not sure about its status. so for that i’m afraid you’d have to sort back to beta. tokenizers have help-patches there. the “lengthprefix” one should do.
but indeed, the best option in vl would be using ZMQ.
let us know how you progress…
VL.IO.NetMQ hangs when I try and open the vvvv with the package directory as in the args text
/package-repositories “D:\vvvv\vvvv_50alpha38.2_x64\VL.IO.NetMQ”
And no nuget yet I guess?
indeed i’m afraid no nuget yet.
1>------ Build started: Project: VL.IO.NetMQ, Configuration: Debug Any CPU ------
1>D:\vvvv\vl-libs\VL.IO.NetMQ\src\DummyClass.cs(6,7,6,12): error CS0246: The type or namespace name ‘NetMQ’ could not be found (are you missing a using directive or an assembly reference?)
1>D:\vvvv\vl-libs\VL.IO.NetMQ\src\DummyClass.cs(12,9,12,19): error CS0246: The type or namespace name ‘PushSocket’ could not be found (are you missing a using directive or an assembly reference?)
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========
sounds like the referenced NetMQ nuget is missing. typically visualstudio would just restore that silently on build. not sure why it doesn’t for you. try: rightclick solution in visualstudio solution explorer and run “restore nuget packages”
vvvv.exe-exception-2019-07-27.log (24.8 KB)
Throws an exception ‘The given path’s format is not supported.’
so what is the exact commandline you’re using to run vvvv.exe?