Forum

Using bone animation to drive character nodes

vvvv beta question
#1

Hey you all!

I’m trying to drive a collada - rigged character via its skeleton. Using setJoint works perfect.

But I want to use a vvvv bone skeleton (which has a different scale than the characterSkeleton) to drive the character nodes. I want to use rotations to do so, but the matrixoperations and mathstuff I tried didn’t work at all.

Basically i want to use a kinect to drive the colladaSkeleton and the boneLookAt seemed to get me the rotations between each limb.

Any ideas on how I can achieve that? Will my solution work at least in theory?

#2

the math is:

alpha = acos( (vec2-vec1) cross (vec3-vec2) ) / (length(vec2-vec1) * length(vec3-vec2))

upvec = (vec2-vec1) dot (vec3-vec2)

convert alpha from rad to cycles and feed it into AxisAngle(Quaternionset Vector) , normalize the upvec and feed that into AxisAngle as well and voilá: a rotate(Quaternion Vector) gives you the rotation.

i just have to find a way to feed it to the character nodes.

sadly this calculation is VERY cpu expensiv and I need to do many of these calculations for every frame. is there a cheaper way around or is it possible to do these calculations multithreaded?

#3

Hello unti,

I have been trying to map the Kinect’s ‘absolute’ rotations to ‘relative’ rotations that could be used in skeletal animation too, for a while. It took me quite some trial and error to get there, but I think I am close to a usable version.

I attached a patch that first creates matrix transforms from the 3 Kinect’s rotation vectors, and then tries to get the ‘relative’ rotations out of it.

I hope this helps (i don’t know if this is more efficient), but I am also curious about your approach, so I’d be glad if you posted a patch of yours, so I can see how you managed to do it.

kinect skeleton.zip (251.9 kB)

Kinect retarget animation question
#4

Hey, thanks a lot, I will try to understand your approach :)
I’m not at my vvvv-PC right now, but I will upload the patch in a few hours.

#5

new sdk outputs ready made rotations, but ready made sounds still quite huge remapping work…

#6

well i will test that as well… tomorrow ;)
I hope we’ll get that working so that everyone can have his 3d avatar in vvvv :D

#7

shame it’s quite hard… even with rotations. You still need to do character retargeting wich is quite heavy…

#8

btw: what does the kinect(Microsoft) Clipping XYZW ?? I wasn’t quite able to find out what that data represents.

#9

if part of the skeleton is outside the screen, order is Left Right Top Bottom

#10

@antokhio: what do you mean by ‘character retargeting’ (I don’t know that much about skeletal animation).

#11

@ft: it’s when you have your character in one animation software and pivots aligned to that particular software so you may have something like XZY instead of XYZ, also if your bones are not matching to the kinect’s it’s called retargeting… There is a nice example in SDK 1.5 showing how to do it with XNA.

to be more focused the XZY problem can be fixed by global mesh transform, but there is a problem that skeleton nodes are not acepting this transform,
so your character can be whatever in 3d space like facing floor etc, so with skeleton nodes it will work if you match your local orientation etc.

it’s quite hard also to export char in collada format… Maya 2007 your friend

still need to tweak my kinect nodes to release but first two are base
and SetRotate wrong old version, don’t use it

SelectJointByName (Kinect SDK).v4p (11.3 kB)
GetJointByName (Kinect SDK).v4p (7.6 kB)
SetRotate (Transform).v4p (10.9 kB)
KinectSkeletonCylinder (Kinect SDK).v4p (23.3 kB)

#12

finally, I hope I commented it well enough :)

the nerd way :D (17.8 kB)

#13

I solved your Problem a few days ago antokhio, used the look at as well :)
I’ll try to find the patches if you want me to.
Usig this approach with the character nodes does not work unfortuatly…
Where can I find the Kinect Node that outputs readymade rotations? The one in the 27.2 addon pack doesn’t output rotations…

#14

sadly new driver release will go next addonpack
but we prob put it to contrib in the eve
well that one exactly like my old one…
now i just use lookat and two normals

p.s. since there is nodelist now we will try to put to contrib new driver with with instruction at night i guess

#15

would be awesome!! is it ms then or open ni?

#16

my approach breaks when you use 2 90Degrees Angles in different directions (i.e. vectors like (0,0,0), (1,0,0), (1,0,1) and (1,1,1) ) any ideas why?

#17

i’m quite sure you are not using normal proper
easy as that
you got two points and a normal

Solver.v4p (4.2 kB)

#18

you were right, I forgot to normalize lol. Stupid me. Strangely I get the same results when I normalize it right. The first bone works perfectly fine. The second one doesn’t… there might me some hierarchy stuff that I have jet to find out… we will see what time brings

#19

my bad, select joint binsize must be 3 everywhere in order to that stuff work, it^s vectorsize actually i used for 2d and forgot

#20

continued here: https://discourse.vvvv.org/ for Kinect2

© vvvv group