Forum

Interactive motion capture (markerless) live performance

vvvv beta general
#1

Hello,
I’m looking for a low cost method of motion capture (markerless) live.
What I’m trying to get is a LIVE performance with a video output like this:
https://youtu.be/mAKveiQ--ys?t=2m48s (2min: 48)
which a 3D avatar can simulate in real time (or almost) the movement of the performer.
The performer will perform in a space of approximately 4x4 meters
For the mocap I’d like to use PSeyes or kinect for their cheapness.
I’m looking for an efficient way to do it… not i’m not a user with great experience in vvvv.

Thank you so much in advance

Donald

#2

Hi Donald,

That in fact is not a thing that can be done very easily. I guess Kinect 2 is the way to go. But there will still be some latency. I’ve achieved something similar, but It was not a pure vvvv solution. I used Unity for the tracking and 3d part, spout to share texture with vvvv and then some processing and mapping with vvvv.

Greetings,

Alex

#3

on the other hand i’d argue we’re very close to see this working, check: Skinned Skeleton and Kinect

only it seems to be extremely complicated to create a suitable mesh. the other option of course is still to take any mesh and adapt the orientations for each joint which would still start from the patch linked in the above thread.

#4

i dont think kinect can track fast movement like blades or flips, but i can be mistaken, i did used brekel pro body few times and kinect seems to me still veeery limited in terms of freedom of movement, but for some basic acting its allright

i think you can also make your own tracking of face features, put a gopro on top of someones face with markers and then do some software magic with recording

#5

The performer in this case will dance a piece of contemporary dance…

#6

from my experience the kinect only does solid motion capturing with the skeleton node when the person is facing the kinect frontally. with more complex movements like turning sideways around (let alone dancing) the tracking algorithm gives in.

what i’d love to try is a perceptron neuron but this one is not neither markerless nor cheap (but waaay cheaper than stuff like VICON mocap systems).

#7

Hey Motzi,

That thing looks cool. But the guy at Schmiede, who used it for his VR-Game, said it wasn’t very reliable…

Greetings,

Alex

#8

oh damn, i missed that someone was there using it :(

#9

Our studioneighbours klingklangklong have a prioVR suit. It looks really similar to the perceptron neutron thingy and is quite cheap. There’s an SDK available which looks good.

by the way are the used YEI sensors also cool for other tasks…

#10

There is also a cheap new motion capture system suit called Salto3d and currently find pledges on kickstarter:
https://www.kickstarter.com/projects/2132274632/salto-captures-your-every-move-vr-ar-film-vfx-gami

© vvvv group