IR Tracking Moving Flat Set Pieces


Hi all,

This is my first post, and I would really love to get some insight on a project I’m undertaking. I tried to do searches, but didn’t come up with exactly what I am looking for. I am a technical director, and for our show this year, we would like to project images onto four 4’x8’ moving panels. They do not necessarily have to be projecting while moving, it is more so to combat the imprecise nature of moving the panels and having the projections be off. I am thinking of setting up an IR filtered camera with IR LEDs on the pieces themselves. What is the process of tracking these panels in VVVV, and then outputting a static image or video to a projector. The panels do not really need to be differentiated between, as they will never overlap. The next issue would be creating a series of “cues” for different scene changes.

Sorry if this has already been answered, I see a lot of posts about which equipment to use, and less about what the actual process is in the software.



hi stephen,

we are currently building sth similar… but we use markers to differentiate different elements on an interactive tracking table.
the biggest challenge would be to detect the objects that you want to project on.
if you already go the route of using a infrared based system, maybe you can separate the panels from the background in order to find the corners of your panels - i assume they are rectangular - and use those corners to map your projections on. the homography node is your friend for this. it is basically a 4-corner-pin that you can use to map your 4 corners of a texture to 4 corners of your camera image - or real world scene. then you would “only” have to map your projector with your scene so they overlay each other.

what kind of a show, or cues are you talking about? and what kind of a content do you intend to show on your panels? i assume it’s static and only needs to be triggered somehow, or is it interactive?



Hello Clemens,

Thanks for the reply! We are going to be doing this as a staged musical, so I figured that IR would be the best way to go with the changing lighting levels. They are going to be rectangular, painted white, with static images on them. No sort of interaction, they are just going to be set pieces. Would that node work for this?

As for cues, they are just going to be different scenic elements that hopefully change at the push of a button. Kind of a playlist of images.

Thanks again!


You would have to undistort your camera, or use a camera with an undistorted lens in the first place, but then i dont see why you would not be able to solve it with the homography node…
The hard part will be the identification of your corner points from the image.