Hey it has not been solved from my side at least.
good point with the joint count.
but should it not be also possible to ignore the joints that are too many and apply the rotation with setslice just to the ones that fit best to the positions?
model wise i know some models that could be tested
there are many available for free here
http://tf3dm.com/3d-models/characters
also microdees emeshe contains droidgirl which has already a working joint aniation module in the example
try this
proletariat_wz.zip (255.9 kB)
Hey I bought Cinema Mocap for Unity so i am motion capturing at home now.
It records in Collada with skeleton for all joints the Kinect 1 ( sorry dont have V2 yet) has to offer . The recording aint perfect but works for me to play around.
I am uploading a quick test
CinemaMocap (390.5 kB)
hei @antokhio,
thanks for the mesh, that will be a good start. i’ve to say i’ve not come very far yet other than seeing that the msdn-forums are full with similar questions and no answer that helped me…
here is a first guess though: if we want to have it easy (which would be good for a start) the mesh should be rigged with a skeleton that has its joints aligned like you can see in this picture: https://social.msdn.microsoft.com/Forums/en-US/f2e6a544-705c-43ed-a0e1-731ad907b776/meaning-of-rotation-data-of-k4w-v2
see? so Y is always pointing in bone direction and Z is always the joints axis. what do you think? can you update your mesh accordingly?
EDIT: reading further someone suggests that image is not correct and points to here: https://social.msdn.microsoft.com/Forums/en-US/31c9aff6-7dab-433d-9af9-59942dfd3d69/kinect-v20-preview-sdk-jointorientation-vs-boneorientation?forum=kinectv2sdk where it says that Y is always pointing from a joint to its parent (ie. in bone direction) and Z is the joints roll (which i don’t quite understand from the provided picture)… also i couldn’t yet concise info on the kinects “resting pose” (ie. the pose in which orientations are all 0) which i think we’ll need to know in order to go further…
hi joreg, there is not obvious how spine and hip left right shud work since they kinda branch from root joint’s by offsetting. I’m hoping i got this picture in your second link more or less correct…
not sure this place:
proletariat_wz_2 (2).zip (256.1 kB)
ouright, thanks, but i’d argue we are still not there. please compare the two images:
here i’ve loaded and displayed the joint-orientations of proletariat2: proletariat2
here i am standing in front of the kinect: kinect2-0
so here is what i see:
so not to do too many changes at once i think it would be great if you could do those 2 and then we have a look at it again.
could u might save the matrices of kinect2-0 and post it as patch so i can study it closely it’s a script that aligns joints, so i need some proper reference not to mess it up. relaxed T pose proly the best. think i got the idea, but reference patch would be the best
edit: fixing by pic seems enough data
edit2: hating microsoft, who would use negative Z as positive Z i wonder if i can even set that
edit3: ok seems that general problem with maya .-/|- (brain clinking…)
edit4: wtf i thought at list i can change that in max… no way anymore lol
ok this is still most wierd retarded shit i ever done… i used some -Z to fix that up hope it’s gonna work
proletariat_wz_3.zip (255.9 kB)
not sure about details but concerning point 1 (“proletariat is looking towards positive Z…”) i thought it would really be as simple as: select all -> rotate 180° around Y
no?
no, u can’t rotate rig easy also it’s all local transforms so no matter witch way it pointing, only the axis of scene witch might be a problem in our case also
anyways i prolly need a test patch for further debuggin
ok, so i hope attached patch can help to get this sorted.
it allows you to view your mesh next to live kinect data and compare joint names, orientations and degrees.
i’d say the first goal should be to get all the arrows of your mesh’s skeleton point in the same directions as the kinect ones. that should help with further steps…
EDIT:fixed attachment
skeleton2.zip (10.6 kB)
quick update…
found extra an issue… inverting axis in maya results skin twists in dx11
seems assimp dx11 way more perky then dx9 collada. Need some extra test on dx9 here no clue if that prolitoriat_3 is working on dx9…
Anyways pain and suffering only way here…
ai, can you upload proletariat3?
already fixed some aligments however i don’t think raw rotatins will work here…
try this one…
gonna try to use plane T pose for riggin
pro_4.zip (278.4 kB)
ok, i’m afraid the first thing we’d still have to get right is that 180° rotation around Y of the whole model.
so when loading the model in vvvv it has to look towards negative Z. note: it doesn’t help to change the Coordinate configuration settings in ColladaFile (EX9.Geometry) because those somehow don’t affect the joints.
only when that rotation is fixed it makes sense to try and get the individual joints line up with the orientations coming from the kinect.
also for simplicity i’d concentrate on DX9/collada for a start, once we have that it should be easier to make a version for DX11.
mm in 11 version it actually properly oriented i’m afraid this the negative Z vvvv uses by default… witch actually can be fixed by multiplying root joint with scale 1,1,-1 however rotating the rig won’t be a quick job, since to make it that way i need to redoo whole skeleton thing otherwise it will be just rotated from root transform
here is quickly rotated version
pro_4_rot.zip (274.6 kB)
btw from my tests on dx11 single joint works pretty ok not on whole thing tho
and since i had some sucess on a 2d version, i’m pretty sure i know how to fix that thing…
here is also an 11 version
Kinect2_Char_dx11.7z (291.1 kB)
ouright, i think this is getting somewhere, see here: kinect2skinner-directx-renderer and attached. the guy is moving all his limbs according to my movements. i am still not sure how to exactly deal with the extremities (feet, handtips, thumbs) correctly but i think the rest is at least correct concerning the SkinningTransforms.
you see that i had to rotate the mesh by 180° around Y and you see some of the joints seem twisted, but i think it would be best to fix those in the mesh-skeleton. you can do this anto!
Kinect2Skinner.zip (253.1 kB)
Just a quick question … is there going to be similar node/contribution for Kinect 1 ?
I ask because the one we have ( AnimatedCharacter (AssetsDB) ) now is for OpenNI and at least i didn’t manage (don’t have the skills) to make it work properly with MS SDK. OR am I doing something wrong ?
Kinect1 one still quite usefull i think , as it can work on low end /old machines …
i haven’t looked at the orientation-data we get for kinect1 yet. from what i read it is different, so probably the method used here cannot directly be applied. anyway first step: we’d need a skinned mesh with bones according to the kinect1 bones. scroll down here to see the requirements: kinect#kinect v1
I have skinned mesh with animation already :)
the file that i attached few posts up complies with the Kinect1 naming in the document you linked to. Just checked it in Blender and the names are a match.
I am asking because i spend a week or so trying to make it work and i have no success whatsoever .
EDIT: Ok may be not, my bones seems very similar to the scheme but i am not sure yet. Searching fro reference from MS