VL.Devices.Nuitrack

Hi ravaquez

tried it with:
Orbbec Astra Model:Astra
Microsoft Kinect v2
vvvv gamma 2019.1 preview 411

Overview.vl is loading and I see VL.renderwindows and VL.nodes appearing - then sudden death with Orbbec Astra, with Kinect I even see something in the renderwindows but frozen and then also sudden death - no error message appears

how can I have a look at a kind of a logFile so I can maybe tell more

Greetings
Daniel

Hi @CeeYaa, I updated the installation instructions yesterday, please make sure you follow all the steps here: https://github.com/vvvv/VL.Devices.NuiTrack/blob/master/README.md

Basically your Nuitrack SDK install should be working before you attempt to use the VL nuget.

If you can see data coming in from the steps in section “Activate Nuitrack” then VL should also work.

Please report back with your findings.

Cheers.

Hi @ravazquez

Stupid me - NuitrackSDK not activated - NuiTrackTrial license is linked to one PC, after I got a new key - NuitrackSDK works and also you VLpatche Out Of the Box - as usual, looks good - after 3minutes quit but this is the TRIAL - now I have some testtime, if I should go for a PRO

THX again ravazquez
Great contribution

1 Like

Glad to hear you got it working, and remember it is still very much WIP.

As of version 0.1.6-alpha Skeleton tracking nodes have been added.

Please test and report.

1 Like

Hi @ravazquez

Thanks for the Nuitrack lib,
I tried to run your help patches in beta 39, older rc version(38.2-4278) latest Gamma and the latest alpha as well, but somehow always failed and vvvv just chrashed.
My nuitrack is an activated pro licence and the nuitrack_sample.exe and the VL.Devices.RealSense are working fine with my Realsense D435.
Can you help me how can I fix it (tty render didn’t showed anything)?

Thanks
Zsolt

Hi @KX, did you follow the steps detailed here: https://github.com/vvvv/VL.Devices.NuiTrack/blob/master/README.md?

Specifically:

  • Setting environment variables
  • Activate Nuitrack
  • Getting an activation key

You should not need to build the C# project.

Hi @ravazquez,

Thanks for your answer, I did, I uninstalled than reinstalled the latest nuitrack version, than I generate lic. key on dashboard and activated with Nuitrack activation tool… (one year windows licence

), plus I checked the environment variables and it was correct.

Maybe some version difference between vl and nuitrack.net, they made the Git version at 6th of november.
net

many thanks
Zsolt

Hello again @KX, I will need to have a look at it then but am afraid I will need some time since my schedule is a bit tight these days and I have not really touched Nuitrack in a bit.

How much of a hurry are you in?

Hi @ravazquez,
Sorry to not reply, it would be nice to try out in the next few weeks how is it working, because we need to decide which tracking we gonna use.

Many thanks,
Zsolt

Hi @KX, there seem to be some backend issues with this lib.

@azeno is already aware of them and will have a look when he has some time.

We will update this thread when we know more about what is causing all this.

Thanks for reporting.

1 Like

Just a quick update, I pushed a new version of the library (0.1.17) making use of newly introduced resource operators as well as fixes to point cloud. It requires a very recent VL >= 2019.2-0300 (available through teamcity.vvvv.org)

2 Likes

Pushed another update (0.1.18) which fixes point cloud using a Kinect2 (or any other device which has different resolutions in color and depth or different frame rates on those). Note however that the assigned colors to the points where never tested and are probably wrong when the color and depth images don’t align. Need to have a look at that with Xenko at some point as I had no idea how to render > 50k points in different colors with Skia.

1 Like

@KX could you please test against latest gamma (>= 2019.2-0300) and report back please?

@ravazquez @Elias Hihi,

Thanks for that, It works for me, but I gonna test again tomorrow.

1 Like

Minor update: previously hidden Overview help patch is now visible under Learn/Explanations in the Help Browser.

thanks for your work. having proper depth cam support should prove handy in the future!

I tested the realsense today, it works quite well so far. What is the Min Z/Max Z unit? mm or something?

next would be to have a long-term test (to make sure it is viable for installations) and a 3d test, if the pointcloud can be used to reconstruct the world similar how the late kinect2 did.
for that I would need a way to use your gamma advances in beta.
would it work to simply update some of the nugets and dlls?

@velcrome glad to hear it is working for you. Min and Max Z are in mm (at least for a Kinect2).

Regarding stability I believe @Elias had it running for days with no major issues.

As for the pointcloud, the current solution is far from production ready specially regarding performance. A GPU (Xenko/Stride) approach will be the way to go when that finally lands in VL. In the meantime some improvements could be made but don’t get your expectations too high. (Kinect2 is performing significantly better already if that helps).

As far as beta goes I have no knowledge on what is needed to get this playing along with it but I suspect it involves .netcore and other lower level plattform bits which are beyond the library implementation itself.

I should find time soon to update the VL integration layer of beta to incooperate the latest VL bits. Just updating the packages won’t work because of bigger changes brought by new roslyn backend.

But in theory once we have VL back in its newest form in beta, and beta having DX11 it should be possible to upload the pointcloud to the GPU in a more performant way - I think the basic infrastructure should already be there.

Hi

VL.Devices.Nuitrack works here fine with vvvv_gamma_2019.2.0-0321
but not with the actual vvvv_gamma_2020.1.0-0032

Greets