6dof head tracking?

Hi all,

I’m new to vvvv (Started a couple of days ago) and am already really enjoying it. I’m a 3d animator by trade, and currently putting together a realtime marker based facial motion capture system.

So far, I’m using contour with a webcam altered to work in IR, and tracking reflective markers on my face from an IR led illuminator. Things seem to be working pretty well (still a few issues to iron out) however as the camera is not head-mounted, any head motion results in strange skewing.

My idea for a solution was to track the head motion (position and rotation) and use this to drive a master head bone, with other facial bones following along, still being driven by the facial markers. However, I’m at a loss as to how to do the head tracking.

I noticed the fiducial tracking examples seem to do what I’m after (tracking a rigid object in 3d space) however as I’m working with an IR setup the best I could do is use a rigid box, with the four corners illuminated. This would leave me with 2d tracking data for the 4 corners of the box, however I have no idea how to convert this data to a 3d transform.

Is something like this feasible in vvvv? I’m hoping there’s something obvious I’m missing.

On a related note, is contour what I should be using for the marker tracking? I’m effectively just tracking points, so the contour data itself is of no use. (Just using the centre co-ordinates.)

Woould greatly appreciate any advice on the topic. Thanks in advance.

Jeremy

OK, scratch that. For some reason I had the idea of dots in my head, and didn’t even think of making one of these markers by cutting up some of the reflective material I’m using for the spots into the required pattern.

So, I’m pretty sure that’s solved the initial question! Sometimes you just wind up overlooking the obvious. :-)

Jeremy

hi jeremy,

i am very curious which kind of reflective marker and IR led illuminator you are using cause i also work with ir tracking like the famous johnny lee did with the wiimote.

i am using a self-made array of 30 VISHAY TSAL7600 IR-Led’s with an extra Led-Reflector on each. on the other side i have some reflective tape i ordered at e-bay.

the thing ist i am not very happy with that solution. the room must be pretty dark and the distance to the illuminator can’t be more than 2 metres until the reflection gets too weak for the wiimote ir-camera.

do you have any hints or ideas? what is your hardware set up?
i would be pretty happy about a feedback.

thanks in advance.

daniel

these guys have some interesting IR headtracking systems

they’re priced for gamers so i think you can get a kit for around £100

there’s sdk for getting the data out

ah bolx
their sdk is only available to commercial game developers

it seems
you might be able to ‘hack’ it by doing some image recognition on the calibration tool using Screenshot node (once worked with that to try and get data out of serato)

the kit’s got a good spec (120fps, 9ms response time), but needs opening up! serious shame on that one

if you’re not a games developer they want to sell you $5k kits with 6 cameras.

maybe can talk them into collaborating on a closed source vvvv plugin so they can show off their technology to more users worldwide

@dandsign:

assuming that you are in a theater environment:

you could try to use any PAR or Fresnel with a combination of LEE 181 “Congo Blue” and LEE 019 “Fire” (or similar filters) in front of it.

it is indeed recommended to use HTfilters and at least two lamps on each position. lamp#1 on 100% for a few minutes, meanwhile lamp#2 cools down. otherwise your filtergels won’t live long…

i have a trackir (the older version 4) at home and it works brillantly. for gaming. there’s also mouse emulation and a few tools to visualize what the camera is seeing. but that’s it, no further software.
i too wanted to ask them if the sdk could be made available for vvvv-purposes, but haven’t done so yet.

there’s a project called freetrack which formerly supported trackir and offers tracking with webcams.

while trackir is a good camera-system with very low latency it’s also made for the user to sit right in front of it. the standard package has a limited range in normal conditions, but i suppose could be tuned by using brighter LEDs than those on the camera. they work fine and with longer range with the wiimote btw.

I may be barking up the wrong tree, but have you thought of using something like a wiimote for its accelerometers and using that data to do head tracking? I’m sure there must be a Win equivalent of Osculator which connects to the wiimote and outputs OSC which can be fed into vvvv. Some smoothing of the data may be necessary as I think the output is pretty noisy but usable none the less. Just a thought…

@sugoku AND @frostpfote
i will check this out but i am not quite sure if this solution is useable for far range applications.

@kalle
this suggestion sounds interesting. i will check this out. are you sure these filters will include the infrared wavelength? i am not very familiar with this stuff.

@dcrow
i work with the wiimote =)

thx all

i did some experimenting with freetrack during the weekend, starting off with a webcam and tuning it according to the instructions (remove ir-filter, put in visible-light-filter). tried tracking a wii-bar i had lying around. it might be the cheap&old camera, but tracking seems only possible if the geometry of the tracked markers is known to the program and stays stable. tracking of markers, e.g. on a hand, that also move their respective positions seems at least hard if not impossible.
it also needs a strong ir-input, reflection like with the trackir is simply not possible - maybe with the better equipment kalle suggested.