Realsense depth stream z16

Answering my own problem, and maybe @schlonzo too. I was using 50beta35.8; 50beta36 seems to have the missing DX11 nodes.

And just to make it explicit, you need to install the RealSense SDK, and copy two files from the SDK install folder C:\Program Files (x86)\Intel RealSense SDK 2.0\bin\x64, Intel.Realsense.dll and realsense2.dll, into the rs_assembly folder createdby the zip file.

It then works for me, but the VVVV.d430.vl node is still red; inside I see the following red lines, but I do not know enough (yet!) about VL to diagnose it:

Capture-1

Capture-2

Edit: And yes I am seeing the same halts in display without Perfmeter2. Something to do with those error lines in the wait for frames area?

Edit2: If I open the Root_d430 patch, and move my mouse around in it, the display updates smoothly as long as I move my mouse! Whaddupwiddat?

Hi mediadog,

did you download all nugets and add the according dependencies within VL? (click on VVVV.d430.vl -> Dependencies -> Manage Nugets -> Commandline

“nuget install VL.OpenCV -prerelease”

After that I have to manualla add the realsense.dll unter Dependencies -> Files.
voila

Hi, as a vl beginner I’m not sure but dark red/brown as on @mediadog screenshot seems to stand up for nodes assign to “destroy”, not for errors. The vl node may stay red while the patch is working because of “pin is marked as hidden” on “TobufferDescription” node, if so you can right click on the node->open->middle click on Output pin (text not box)->check visible. But this error doesn’t seem to affect the patch anyways.
No ideas about vl patch messing when open it. I have similar issue when I use two cameras. Once it’s setup I restart it and let it close.
@schlonzo is it running smooth in realsense viewer? the camera is sometimes recognize as usb2 instead of usb3. Also the patch is set for d430 module, stream’s index, available resolution/fps might change with other models

@david_sh

its running super smooth within the viewer and vvvv.

@david_sh it runs smooth in the RealSense viewer. @schlonzo, it runs smooth in vvvv if you remove the Perfmeter2 node?

Open the VL patch, there is an async Loop with a framedelay (5000ms). Lower that to increase framerate.

@robotanton
@david_sh

could you give me some pointers on how to stream out the position data PosW as a dynamic buffer, so that I can directly feed it onto a pointcloud emitter? Or what would be the most efficient way of emitting these within a dx11.particles system?

hey, maybe not a big help, but I’m streaming out PosW data like this with an Orbbec Astra:

image

The key here was to copy the data in the async loop, so they are still available when the buffer is uploaded in vvvv.

I know it’s an old topic but did you guys managed to get it working properly? I’m trying to work with a D435 and it only works for a few seconds than freezes. I’m guessing i’ts some kind of memory leak but can’t find it so far.

tested it for ~48h on an udoo. no crashes

With the last patch david_sh posted? I’m trying to get it work for days but no luck.

The root node flashes red every 5 seconds but other than that there are no red nodes or missing dependencies and the log only shows this:

00:03:30 ERR : Exception occured in TMPluginWrapperNode.Evaluate
00:03:30 ERR : One or more errors occurred.

I would really appreciate some help to solve this.

I didn’t go further than the last patch. At the opening I often need to unplug/plug usb/reopen patch but once it run it not crash. Those 5 seconds may be related with waitforframe node, try to tweak “timeout_ms” to 0 and back to 5000 or remove/recreate it (don’t know what it exaclty do but I remember did it once). Also make sure to use perfmeter2 from Microdee’s pack. There is definitely something missing to make the patch stable.

Thanks for the help! I got it working by using an older DLL, the most recent one caused the freezes with the patch. Also got it to work with DX11 particles by casting the depth array to Float32 with marshal and linq which is probably not the most efficient way but works quite well.

older DLL? shall we know the version? is it included in an older SDK version (current ver: 2.18.0)? THANKS

2.10.4

1 Like

Thanks for this!

Is there a way to turn the point cloud from the realsense into blobs for openCV blob detection?

I suggest using Blobs(particles.selection) from the dx11 particles pack.
Drawing the particle system and then using open CV is kind of dirty and has more computational overhead - but should also work: render the particle system from one side with an orthogonal perspective and feed this texture into openCV.

1 Like

Hey, thanks for this!

Yeah that’s what I have been trying to use but I am having trouble creating the particle system to connect to the blobs node. I am trying to use to use the X,Y from the point cloud to create particles but I am relatively new to V4 and am finding it tricky. Could anyone see if they know where i’m going wrong?

Realsense_d430_pointcloud.7z (1.6 MB)

Thanks guys!

Hi,
Just received the RealSense D435, the Patch is working good !
Would anyone be interested in pooling resources for the integration of Realsense cameras? Calibration, Color integration, filters and multiple devices, Rec&Play…

Could this speed up integration and increase the number of users and contributions?

2 Likes

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.