I’m new to vvvv (Started a couple of days ago) and am already really enjoying it. I’m a 3d animator by trade, and currently putting together a realtime marker based facial motion capture system.
So far, I’m using contour with a webcam altered to work in IR, and tracking reflective markers on my face from an IR led illuminator. Things seem to be working pretty well (still a few issues to iron out) however as the camera is not head-mounted, any head motion results in strange skewing.
My idea for a solution was to track the head motion (position and rotation) and use this to drive a master head bone, with other facial bones following along, still being driven by the facial markers. However, I’m at a loss as to how to do the head tracking.
I noticed the fiducial tracking examples seem to do what I’m after (tracking a rigid object in 3d space) however as I’m working with an IR setup the best I could do is use a rigid box, with the four corners illuminated. This would leave me with 2d tracking data for the 4 corners of the box, however I have no idea how to convert this data to a 3d transform.
Is something like this feasible in vvvv? I’m hoping there’s something obvious I’m missing.
On a related note, is contour what I should be using for the marker tracking? I’m effectively just tracking points, so the contour data itself is of no use. (Just using the centre co-ordinates.)
Woould greatly appreciate any advice on the topic. Thanks in advance.
OK, scratch that. For some reason I had the idea of dots in my head, and didn’t even think of making one of these markers by cutting up some of the reflective material I’m using for the spots into the required pattern.
So, I’m pretty sure that’s solved the initial question! Sometimes you just wind up overlooking the obvious. :-)
i am very curious which kind of reflective marker and IR led illuminator you are using cause i also work with ir tracking like the famous johnny lee did with the wiimote.
i am using a self-made array of 30 VISHAY TSAL7600 IR-Led’s with an extra Led-Reflector on each. on the other side i have some reflective tape i ordered at e-bay.
the thing ist i am not very happy with that solution. the room must be pretty dark and the distance to the illuminator can’t be more than 2 metres until the reflection gets too weak for the wiimote ir-camera.
do you have any hints or ideas? what is your hardware set up?
i would be pretty happy about a feedback.
i have a trackir (the older version 4) at home and it works brillantly. for gaming. there’s also mouse emulation and a few tools to visualize what the camera is seeing. but that’s it, no further software.
i too wanted to ask them if the sdk could be made available for vvvv-purposes, but haven’t done so yet.
there’s a project called freetrack which formerly supported trackir and offers tracking with webcams.
while trackir is a good camera-system with very low latency it’s also made for the user to sit right in front of it. the standard package has a limited range in normal conditions, but i suppose could be tuned by using brighter LEDs than those on the camera. they work fine and with longer range with the wiimote btw.
I may be barking up the wrong tree, but have you thought of using something like a wiimote for its accelerometers and using that data to do head tracking? I’m sure there must be a Win equivalent of Osculator which connects to the wiimote and outputs OSC which can be fed into vvvv. Some smoothing of the data may be necessary as I think the output is pretty noisy but usable none the less. Just a thought…
i did some experimenting with freetrack during the weekend, starting off with a webcam and tuning it according to the instructions (remove ir-filter, put in visible-light-filter). tried tracking a wii-bar i had lying around. it might be the cheap&old camera, but tracking seems only possible if the geometry of the tracked markers is known to the program and stays stable. tracking of markers, e.g. on a hand, that also move their respective positions seems at least hard if not impossible.
it also needs a strong ir-input, reflection like with the trackir is simply not possible - maybe with the better equipment kalle suggested.