Forum

Impoving latency/performance of tracking. Freerame wrapper efficency

vvvv beta general
#1

hi devvvvs…

while working/improving on some tracking patches for interactive tables i discovered that there are severall sources for a bad latency. to test latency of the vvvv-freeframewrapper i tried to chain up a freefframedummie that does nothing, it just passes the videosignal. result: latency keeps low as long you just use one or two chained plugins. with the third an forth plugin the latency becomes worse noticeable.

this is the node chain i played around with:

videoin
v
ffdummie
v
ffdummie
v
ffdummie
v
contour tracker
v
videoout

the final tracking patch(or engine) should be able to work on diffused illumination so stuff like blur, dillate/errode, highpass and contour tracker is needed.

my questions:

-what is the best way to keep latency low?

-where does that latency come from? is the ffwrapper itself somehow inneffecient? or is it something in the plugincode?

-is it a good idea to keep all the imageprocessing in one freeframeplugin? i mean a combined plugin which does the filtering plus the tracking i one go…

-is it possible to bypass the freeframewrapper by writing directshow filters (like the undistortion one) for the tracking. how would that compare in performance/latency to freeframe?
other things/problems to consider?

-would it be possible to do the tracking as a csharp plug? i know that it isnt possible to pass textures or videostreams to v4, but that isnt necessary right? just pass the values of the tracker over…

lots of questions, hope that someone can help to clear up things

cheers ele

#2

helo ele,

*where does that latency come from? is the ffwrapper itself somehow inneffecient? or is it something in the plugincode?
the freeframewrapper indeed does at least copy the image from input to output even if the freeframe-code does nothing. this is probably not needed and could be improved by converting the current FreeFrameWrapper from a directshow transform-filter to a transform-inplace-filter.

*is it a good idea to keep all the imageprocessing in one freeframeplugin? i mean a combined plugin which does the filtering plus the tracking i one go…
while this is against a modular approach it could probably allow you to optimize code for your specific purpose.

*is it possible to bypass the freeframewrapper by writing directshow filters (like the undistortion one) for the tracking. how would that compare in performance/latency to freeframe?
i’d consider a directshowfilter not faster per se, apart from the above mentioned transform vs. transform-inplace story. the interface between directshow and freeframe is only a pointer to the image.
*other things/problems to consider?
every directshow-filters needs to be registered manually in vvvv. you’re welcome to send us your directshow-filters but then they would only work with the next release.

*would it be possible to do the tracking as a csharp plug? i know that it isnt possible to pass textures or videostreams to v4, but that isnt necessary right? just pass the values of the tracker over…
jep, why not. if e.g. you have access to the camera-driver from c# you could write a combined VideoIn-Tracking node that only outputs the coordinates. while this is also against the modular approach it would be interesting to see if you gain any performance like this.

#3

ah and which resolution are you working with?
i suggest you to try below 320x240 and compare your results.

#4

Ive seen the same - the latency.

This sounds interesting for me:

could be improved by converting the current FreeFrameWrapper from a directshow transform-filter to a transform-inplace-filter.

-Can it be done in C++/Freeframe code or is it V4 internal?

@ele:You can get video in Freeframe as well with OpenCV code. I didnt try it yet, but wait for one or two days and I can give you my test results. And btw a working Freeframe/OpenCV template is coming then too.

#5

the freeframewrapper is opensource and written in delphi.

#6

For the undistort you could try this method.

I haven’t had the time do it yet. I think it would at least remove one heavy process from the list.

#7

ha, how funny, as I typed the post above, tonfilm posted a undistort based on co-ords. Check it out in this post.

© vvvv group