New Orbbec depth cameras & OpenNI 2 in vvvv

I have two nice new Orbbec Astra Pro depth cameras, but they need OpenNI 2 to work, so do not work with vvvv. I’m a sad panda!

Has anyone tried or is trying to port OpenNI 2 into vvvv? These look like very nice cameras with better depth fidelity than the Kinect v1, Primesense, and XTion, but are about the same size as the latter and USB powered. And they have the huge plus of being available!

https://orbbec3d.com/product-astra-pro/

I’ll give it a go when I have some time, but not sure when that will be. If anyone else is interested in trying, happy to help out.

Hey, Just saw these cameras, look really nice. did yo find any workaround ? ?

@manuel I tested them out, but never got the spare time to port OpenNI 2, and everything I’ve done the last year I either had enough Xtions for or used KinectV2s. BUT I’ve got a new Xtion on the way for testing, and so the impetus to port OpenNI 2 into vvvv is getting stronger. And there is also the spiffy ZED stereo camera that works under OpenNI2 that works in full sunlight and has long range (20M) capability.

Porting probably won’t be too bad, as I have ended up dramatically improving my home-brew OpenNI 1.5 plugin with things like multi-camera handling and optimized async operation, so I’m a lot more familiar with that stuff now. Stay tuned…

1 Like

@mediadog thanks for reply
I may use it for a project now, but we’ll se if its easier to just do an OF as I only need depth data in low resolution… but If it’s not so hard to port it, will try to do it

Elliot did some test I guess, since he’s quoted on the site of Orbbec, but maybe not for vvvv

Any news on this? Would be great to connect this devices to vvvv

I’ve had a little time and tried to get to get a plugin going with NIWrapper but have yet to get all the libraries to load. Will be looking at it again in a week or so…

1 Like

Awreet! I have a very rough dynamic plugin working with OpenNI 2! And x64 and DX11 no less.

Stay tuned!

2 Likes

perfect! just ordered an orbec astra shortrange mini. looking forward to try your plugin!

Just tested with an Astra, works fine. So that’s the original XTion, Primesense and the Astra that work, but right now the Xtion2 crashes vvvv. I think that is an XTion2 problem with Win7 as the OpenNI viewer also crashes in Win7 with it. Will test it on Win8 and 10.

The new XTion 2 does work under Win8, which is happy-happy joy-joy. It looks very promising because it seems to basically be a Kinect2 in a USB-powered package smaller than a Primesense or Orbbec Astra.

ill get my orbbec astra mini (which is quite small as well btw) this thursday… any chance you share the plugin?

I am busy cleaning it up into something not TOO embarassing… and I am still fighting some library path resolution issues needed to make it portable.

Note this will not be a node-for-node replacement for the previous OpenNI nodes, as I took a different approach way back when the Kinect first came out and my stuff depends on that legacy. So right now it is a single node that outputs a DX11 depth texture, and optionally will output spreads of Z data and/or world 3D points.

Now though I have DX11 compute shader for generating the pointcloud for doing the world conversion, which besides being much faster takes as inputs a camera transform and both include and exclude area transforms. Again this was done before things like the pointcloud pack came out, so it’s a bit idiosyncratic.

Also, with my legacy OpenNI 1 plugin, it output an OpenNI context, so it could be used with things like the existing OpenNI RGB node, but with this one those nodes don’t exist so I need to make one of those which I have not done yet.

On the plus side, I am working on making this plugin handle spreads for multiple cameras.

But maybe at least someone can take this code and put it into something more familiar; gimme a few more days. If you get really desperate I can email you a zip to test with, I could use the feedback!

At the moment i only need depth out. I would be happy to test it and give you feedback! my email is: elektromeier@swissonline.ch

Was able to get back to this for a bit, and great progress! Major news is I have run FOUR Kinect v2s on one PC with it using libfreenect2 - under Win7! But I am now fighting the libfreenect2 OpenNI driver which does not set the depth and RGB capture properly, so the K2 depth will go down to 15fps if there is not enough light for the RGB camera even if you are not using RGB.

Sadly the Xtion 2s seem to be doing some kind of strange thing on the depth channel, so if something gets close, things next to it in the depth image that are farther away change their depth. Sort of like there is an edge filter on the depth image. Bogus.

Hi guys,

we did a contribution for the Orbbec Astra device! check it out and leave some feedback :)

1 Like

Hey @schlonzo, cool, will try it. It looks like you integrated the Orbbec via OpenNI2, which if so, means other cameras should work as well by putting their drivers in the OpenNI driver folder. Including the KinectV2 via libfreenect.

I have not had enough time to figure out how to get around all that file copying you list and more that needs to be done for a dynamic plugin, which is why I had not submitted this as a contrib.

Have you seen the issue with the Orbbec driver being ill-behaved about recognizing Xtion/Primesense cameras as an Orbbec? Just in case here’s the code I had to use to get around it:

    	DeviceInfo[] devices = OpenNI.EnumerateDevices();

	FLogger.Log(LogType.Debug, "OpenNI2: Reported devices count = " + devices.Length );
    	
	// Work around Orbbec driver bug that sees a PS1080 as an Orbbec

	if ( devices.Length > 0 )
	{
		for ( int i = 0; i < devices.Length; i++ )
		{
			bool bogus = false;
					
			for ( int j = i + 1; j < devices.Length; j++ )
			{
				if ( devices[i].Uri == devices[j].Uri )
				{
					bogus = true;;
				}
			}
					
			if ( ! bogus )
			{
				devices[depthcnt++] = devices[i];
			}
		}
	}

	FLogger.Log(LogType.Debug, "OpenNI2: Correct devices count = " + depthcnt );

I reported this in their forum, and then after months of it being ignored, complained about the lack of response, and then they apparently deleted the thread! Uncool. So I’m curious if their more recent driver fixes this.