Forum

Motion tracking with EyesWeb, application in vvvv

vvvv beta general
#1

hey out there!

we are working on a installation where the user can move in front of a projection wall and his silhouette is tracked by a camera. his tracked silhoutte should be transferred on the projection and with his movements the user should be able to interact with objects in this projection through his silhoutte.

we want to use EyesWeb for the motion tracking and VVVV for the animation and application stuff because they work good together via OSC and already used VVVV for another project.

at the moment we just dont know how to start with the data that is transferred from EyesWeb to VVVV. maybe it is possible to put a matrix (with just 0/1 values) over the tracked silhoutte and transfer this matrix in vvvv to process the collision detection.

if anyone has a hint or knows some patches that could help us for inspiration we would be very happy! I already searched the meso site but did not really found something useful for us…or maybe I am just too dumb at this time!

have a nice evening!
naturetweak

#2

his tracked silhoutte should be transferred on the projection and with his movements the user should be able to interact with objects in this projection through his silhoutte. we want to use EyesWeb for the motion tracking and VVVV for the animation and application stuff because they work good together via OSC and already used VVVV for another project.

i don’t know about the eyesweb-part at all.
but i can imagine a “pure” vvvv-solution:
with an InfraRed-camera you should be able to separate the persons silhouette from the projected content.

IRcam–>VideoIn–>VideoTexture–>(WhatEverPatchesInBetween)–>
RenderObject(e.g.:Quad)–>FullscreenRenderer–>VideoBeamer

should work in case your background is not as hot as your performer.
otherwise you might be the first who needs a cooled projection screen…

(i wonder if heating the screen to >50°C would also be a solution?!? ;-) )

regards and success
@kalle

#3

kalle is right in terms of a full vvvv solution. u could extract the silhouette with the contour freeframe plugin, coming with vvvv.

but i have to say a IR-cam is not always a thermographic cam. usually surveilance cams which are sensible to IR light only detect IR spectrum close to visible light but not thermic radiation. thermographic IR cams are damn expensive and well above 10k $.

unless u get one of these a hot or cold projectionscreen doesnt matter at all.

Termography

to seperate the person from the background u could use background subtraction. to enhance the contrast between human and background u could light up the screen with infrared light but dont forget to put a daylight filter in front of the IR-camera lense to be able to ignore the projection.

#4

Hi naturetweak,

I did this for my diploma last months ;D
I tried it with Eyesweb before, like you want to. But finally I made all stuff with Vvvv because of performance reasons. In Eyesweb all is Cpu rendered, in Vvvv I wrote some shaders and the most stuff then is on Gpu, what is incredible faster.

The most important thing for using Vvvv or Eyesweb is:
->What kind of interaction do you need?
For more advanced data about movement I recommend Eyesweb. If you just need something like “Buttons” or pixelcollisonbased interaction Vvvv is the choice. I just needed pixelcollision.

At moment I postprepare my diploma and later I will post some interesting stuff for community. But this needs some time - comments for all shaders, help for modules, …

Some tipps for you:
-Most Beamers dont have Infrared-emissions. So best you take an IR-Cam for tracking.
-CCD-Webcams can see IR. They have a little filter between optic and chip. Take it out and put a Lee Infrared-Passfilter (let only IR pass through) to get a cheap IR-Cam.
-Buy an IR-Lamp for healing. Around 5-8 euros for lighting. Put a blue or deepblue Lee-filter in front to filter the rest of visible red light. A very cheap and powerful 100W IR-Lamp. Better than Leds. See pic at top.
-Or second way. Take a normal bulb or PAR-Fixture and put deepblue and deepred Lee-Filter in front. This filters all visible light. Only IR will pass - its around 60% of normal bulbs - they produce more heat than light! I had done little IR-spots with Halogenpins like this way.


Just in short: Diploma is an exhibition design about digital filmproducing - a filmmuseum. The idea behind is to have some interactive installations where the visitor can replay the technologies. One of several themes we had build.
This is Composting-Installation. The visitor can walk in the room and seeing hiself doubled in a mediaroom. With his mediabody he shifts some layers (the tech behind compositing) in the image by pixelcollision. The visitor plays a little (very straight) game “help the dancing girl”.
Techs: 2 PCs, one is colortracking from above, 2 Webcams, other is showing projection and pixelcollision from behind, 1 IR-Cam, 1 Dv-Cam

https://vvvv.org/sites/default/files/imagecache/large/images/snapshot20061116123330.bmp
Same in Infrared

I think best point to start is to think about your interaction.
If you have some questions, feel free to ask.
Caio Frank

#5

Hi frank,

do you have any catalogue numbers of these lee filters ready?
and is it possible to use the deep-red and deepblue-filters instead of the ir-passfilter for the camera?

#6

i am pretty sure that
Congo Blue

Lee 181
Rosco #382

and
Primary Red
Lee 106
or
Medium Red
Rosco #27

will do it.

also those colors should be on stock at your local dealer.

When using in front of a 1kW PAR64 or sth like this i’d better go with RoscoSuper Gel for heat resistance reasons.
your fun may last a little bit longer than 15 minutes then.

#7

thx for that information, kalle. im asking me why frank needed that special ir pass filter for the cam. isnt the deep red/deep blue filtercombination an ir passfilter itself?

and what is roscosuper gel exactly?

#8

hey guys, thanks for the help! I am just experimenting around with the freeframe patches. I think we cannot efford a IR cam, so we just have a webcam or a dv-cam for tracking. our way if interaction should be done with the tracked silhouette pixel collides with the other objects in our application. will try around, if anyone has some other usefull patches in this direction I would be glad to have a look at!

seb naturetweak

#9

naturetweak: its really easy to convert a webcam to ir:
have a look here: http://www.hoagieshouse.com/IR/

did that myself to a cheap logitech webcam. it was done in 1 hour and it works perfectly.

#10

hey frank,

I could not contact you via the messaging page, dont now why some error stuff. therefore I am writing inside here. you can answer me via pigalert@gmx.net

is your diploma finished now ? well I am just trying around with the freeframe patches at the moment. I think we cannot organize a IR cam now, cause there are only about 2 months left for our project.

we want to use just a webcam or a dvcam. our installation should not be so complex. we just want to let some objects rain from the top and with the tracked silhouette the user can interact with those objects like catching some, or throwing some objects back again.

we already did a installation last year, where we included a motion sensor and a behringer midi controller and three beamers. so animation and stuff is no problem at all, the main thing is how to seperate the silhouette and than how to use it in the interaction application. the silhouette should be used like sprite of a computergame…

if you have any tips just tell me that would be dope! I will try around with the freeframe plugins to check out how they work!

thanks,
seb

#11

http://www.maxmax.com seems to have some nice products. anyway i didnt order anything yet.

#12

some practical application examples of their products:
http://www.maxmax.com/aX-RayExamples.htm

;)

#13

@elemeier:

and what is roscosuper gel exactly?

Sorry, i misspelled this.

Rosco is another manufacturer of color gels
and in case of Rosco’s SuperGel the color is laminated between to transparent carriers.

normally colors are laminated onto one carrier.

well, i want some of that smutty xray-filters.
for sure nice to use when vj-ing with vvvv and livecam/Beamer…

#14

Hi guys,

sorry for the late answer. =(

@elektromeier
do you have any catalogue numbers of these lee filters ready?
-Best filters are Lee 132 (best resultrs), Lee 141, Lee 721. They filter all visible and most of red light. In fact I hadnt these, so I switched to the deep-blue-red-combination.

and is it possible to use the deep-red and deepblue-filters instead of the ir-passfilter for the camera?
-I didnt tried this because I had a IR-Pass-Filter at home. I guess this will lower the possible IR-lightness.

@kalle
When using in front of a 1kW PAR64 or sth like this i’d better go with RoscoSuper Gel for heat resistance reasons.
your fun may last a little bit longer than 15 minutes then.
-Yes the heat is overwhelming. I have used this Philips IR-lamp. Costs around 7eu in mediastore near you. The bulb has E27, so you can put it in a pro fixture with better cooling. Together with blue-red-filter I had no heat problems for ~8h.
http://geizhals.at/img/pix/64715.jpg

@naturetweak
I ll prepare my userpage with some shaders that could be interesting for you. Its inspired by Eyesweb: a background substraction shader, some noise filters and a collision detection by pixels. I ll contact you via email. My Userpage
If your DvCam has got a Nightshotmode, then you better use that. the difference to Normalmode is minimal. Because in Normalmode the cam puts a little plastic IR-Filter in front of the CCD. But you still need a way to cut off visible light (IR-Pass) if you dont want interference with beamerprojection.

#15

I put my tests with Lee filter onto gallery. Shot with hacked IR-webcam.

Upper left no filter, upper right Lee Filter 132 (best:brigthest), Lee Filter 141, Lee Filter 721. It still looks blue but this is because of wrong white balance. See the cable, in fact this is white. Your eyes wouldnt see any blue.

#16

Nice thread guys!

Here’s a little open source idea for you. Put IR Pass filter in front of hardcopy fiducials. then stick fiducials (now black to the eye) on every side of any complex 3D object (say, a black chair). Then look at the object via IR cam. - a poor man’s 3D object tracking? New node: Chair (devices 3D)? Anyone willing to give it a try?

edit: And it’s a legal type of LSD, too:

After I became accustomed to the darkness inside the filters, I could see through them. Going outside on a sunny day was stunning. The sky was almost black, while the trees and shrubs where all frosty pink. The grass looked like fluorescent red cherry Koolaid powder. Different colors of human skin were always the same light grey. People’s eyes looked very black, and certain dark clothing looked white.
( link )

#17

OT:

max,
again i wonder how you enrich the vvvvorums by changing a technical topic into poetry.

interesting qoute from there:

If you lay three layers of Congo Blue on one layer of Primary Red, you form an opaque black filter which is very transparent to IR!!!

:-)

#18

Cool link 8vo I still have some Pass-filter. I ll try that.

Okay here is another little lowtech gadget:
Take a pingpong ball and divide it into two similar halfs. Be aware that the two halfs are seamless and without any prints - clear white halfs. Put a little string to that, to make it like glasses. If you wear the pingpong glasses and lighten it with colored light - you can see pure colors! You see that colors without any materials, like fog. Its a Ganzfeld-Experiment.

May be after that infrared googles.

#19

and a very interesting hint for hiding the fiducial markers of maxens OpenSourceIdea:

covering the fiducial markers with the congoblue and primary red will make them only visible for the IRcam:
http://amasci.com/amateur/irgogg2.html#shirt

#20

Hi,

Just read through this very interesting thread! You, among other things, discussed different IR setups.

I’m planning an installation for septembember and want to keep the space as dark as possible and guesed that IR camera tracking would be the only way? I’ve never used IR cams and have no idea about where to get one or how to connect it to my Mac.

What cameras do you use and reccomend? Is it easy to connect a IR camera to a computer? I’m on Intel Mac and planning to run vvvv on that.

Any help would be much appreciated. :)

BR’s

/ marcus

© vvvv group