Need urgent help with tracking

Hi people,

Sorry if I posted on the wrong forum, but I need help ASAP. I’ve been trying to develop something to track an area from above, but with no success so far. The area will be 8m long, 3m wide. I know this is a big area. I tried various IR illuminators, various cameras, with and withour visible light filters, and various approaches for tracking (Tratner + Contour, Difference of Textures, Shaders, Trautner with a grid mask, and others…

Can anyone help me? Please? This is an urgent project, we have a time issue now. You can drop me a note, a personal message or any contact, and I’ll be glad to send you more details on the project.

Thanks a lot!

You need to give us a little more info than that!
What are the problems your having, what have you tried exactly, what kind of tracking info do you need, what your trying to do exactly…

Sorry about that, Cat. I was in a hurry.

The greatest problem I am having is the lack of contrast in the image. The camera will be above, like the projector. The project is a floor projection of water-like graphics. And it must be interactive. Okay, the visuals part, the interactivity and these problems are all solved, but the tracking part is driving me nuts.

I tried to use visible light filters on the cameras so that the images projected on the floor do not interfere with the image the camera is capturing, as usual. But this makes the image appear with a very low contrast, messing up with the Trautner output (people appear in parts, and some parts of the body are blended in the background, generating multiple blobs of one person in the Contour). This also messes up the Trautner with a 16 x 16 grid mask. Also, tried using the Difference of Textures shader that comes into the Shaders for Newbies tutorial, but it is suffering the same problems as the Trautner with Hold Background set (obviously). Haven’t tried the ColorTracker or CamShiftTracker, because I don’t know in advance which color to track, and also, the floor will be somewhat blue, so people in blue clothes would not be tracked. But I didn’t tested it for real, so…

Is there a way to improve contrast between the floor and the people that walks on it? Repositioning the IR illuminators, or even using some other tracking technique?


Oh, and also: the only info I need is the position of the people on the floor (xy coordinates). That’s why I thought in Contour first.

OK, there can be problems with infra red because fabrics don’t always reflect IR as well as each other or skin, so getting more than 1 blob per person is quite common.
Low contrast with IR is also quite common.
If the lighting in the room is constant ie no windows you could try background subtraction theres a shader in the bazaar for that. I often use more than 1 stage of image processing to get tracking working, contrast and blur can help.
Also you can never have too much light! Lots of IR thats diffused to stop hotspots is very useful!
In some ways people being broken up into hands and heads is useful as when you move your hand you have reaction in the fx if there is only 1 blob you just get the centre point with contour so you have to move the centre to get reactions.

these shaders may help you, then use contour after them…
Lighting is always fun and will be different for a lot of different variables, shininess of the floor colour etc

i don’t have that much experience with tracking, put from what I’ve heard fiducialtracking is still the most precise one, and is pretty much independent from the whole color and background issue. I once went to an expo where they secretely just placed the markers on every visitor…

What colour or material is the floor?

If you get a low contrast picture of your projected image, but people walking upon it, you could go for dark regions for foreground/people. You need no fine silhouette, dont you?
Brightness of projector and its reflected rays are known and hopefully not disturbed by daylight or beams. So try to seperate your projection by luminance value:
Try to blur your image to get projected floor image more contrastless, close to just a same brightness (may be 100-150 out of 255). Then take a threshold for lowcut (lower than 100) and highcut (up 150) too cut out projection. Rest should be foreground. May be a lowcut threshold is enough (dark regions).

For better results you can use Open/Close/Dilate/Erode filters. One of them (dont know which) is surpressing white areas and keeps big dark. Put it in front of the blur.

Shader attached is this erode/dilate. Open is first erode then dilate and close is other way - I mean you need a double pass of them.
If I find bare time I will upload it with helppatch and more…
Hope it helps.
edit: better take the lower of both. its going for green channel, which is more brightness than blue.

ClearEdges.fx (5.2 kB)

Hi all,

Thanks for the input. Turns out that we did with the illuminators we had and two DIFFERENT cameras. I run into the problem of using two cameras of the same model in one computer, and VideoIn just sees one.

The tracking went fairly good, with some blind areas on the floor (because the illuminators had not a wide angle). It was a good distance to the floor to not generat hotspots.

I’ll show you some pictures if I have the chance. Thanks all!