OpenCV/Open Pose Human Pose Recognition

[mishamatrix]

Hello Everybody!

I am relatively new to VVVV and gamma and I have a question about the
optimization regarding OpenCV/Open Pose Human Pose Recognition.

Is there a way to make it faster and more real-time? Because currently
it is very laggy and loses frames.

I know that this should work because I have had experience with PoseNet in P5.js
and there the performance was much faster so I was really surprised why
VVVV was so slow with Human Pose recognition.

I am working on a university project, which should put static, stylized human clothes
on humans in real-time.

So basically I was wondering how to make the performance better. and then later
to put low poly meshes onto the recognized human.

Could someone please help? :) Is there any path I can follow, documentation, meet up, etc? :)

[ravazquez]

Hi @mishamatrix, I am afraid the library we are using for VL.OpenCV works on the CPU only which for tasks like openPose is not fast enough for realtime.

What you have previously tested is likely GPU powered.

I am currently working on bringing GPU support to VL.OpenCV but this will take some time.

For now I would suggest to look for other alternatives that make use of the GPU such as https://github.com/takuya-takeuchi/OpenPoseDotNet.

Cheers.

[mishamatrix]

@ravazquez Thank you very much!
I am very excited about the VL.OpenCV GPU support!

Thank you for the inforimation, I will check it out!

All the best to you!

Cheers

[tgd]

I have spent some hours trying to get https://github.com/takuya-takeuchi/OpenPoseDotNet to work.
Have tried the officail nuget NuGet Gallery | OpenPoseDotNet 1.4.0.20190117
Also compiled the project from the git repo to get the dlls and tried manually.

Always getting errors about the OpenPoseDotNetNative.dll, which is needed by the OpenPoseDotNet.dll i suppose. Not sure why it is not found/loaded because it is provided either by the nuget or freshly compiled. Obviously i am doing something wrong :(

[sebescudie]

hey, regarding native dependencies, does this help?

Referencing Libraries | vvvv gamma documentation

see the “Unmanaged/Native dependencies” section

[tgd]

Unfortunately this does not help. Also suspecting that dll can be found but gamma has problems loading it. The error message might be misleading. Just guessing, not having great knowledge in .net :(

[tonfilm]

try two things, place the native dll next to the vvvv.exe and also check the native dll with this tool to see whether your machine can execute it: https://www.dependencywalker.com/

[joreg]

the “unable to load dll” error can also mean that not this dll directly, but some of its dependencies cannot be loaded. you can check this using a dependency walker, which for me shows that these files are missing:

and you’d have to ask the openpose developers where/how to get those.

[tgd]

Thanks for the tips! Yes there is a ton of missing dependencies in the OpenPoseDotNetNative.dll
Turns out i need all of those:

Actually they were all in the Release folder after compiling, silly me ignored them.
Still not sure how to use the lib but at least the nodes seem to load with no errors.