Forum

Undistort Performance Issues

vvvv beta general
#1

Hi all,

I have a project that requires undistorting a really extreme fisheye lens. I have managed to get this undistorted using the undistort free frame node but I get a very small delay between the video in and video out. I understand this is because it is CPU based processing but for my project I can’t have any delay. I have a really good processor 2.66Mhz 1333Mhz FSB, the best memory, all windows service that can be turned off, and I still get a delay.

Does any one have any advice on performance increase?

So far I’ve tried, using the undistort FX file but the VideoTexture node and then AsVideo to go into the tracker freeframe slows down more.

Does anyone know about GraphEdit? Would this help me?

Does anyone know if this would run faster in Vista?

Thanks, Tom.

#2

ai tom,

no idea, what happened with your approach here? i’d say this is the fastest way to go.

#3

hi joreg,

I halted that approach because future version of my project will require me to use the area output from the tracker. ummm… thinking about it… I guess I could do a maths distort transform on the area too. Would just need to know what it XY co-ords are and then adjust area depending on XY.

I will have a go and work on it, time is not in abundance for me. I prob should have combined these two posts as they are down similar paths.

thanks

#4

a module or plugin to undistort coordinates would be great! if i had the time…

#5

ok, copying the shader code was straight forward. the plugin is on sourceforge. you should replace the _Utils.dll in the plugin folder to use it.

Undistort.zip (19.0 kB)

#6

and a little help patch…

Undistort (2d) help.v4p (10.4 kB)

#7

Wow, thanks tonfilm! I will give this ago and let you know how it goes. Really appreciate the effort. thanks again.

I’m sure this can help out others too. :)

#8

I’ve just put together my tracking based project to test this new undistort node out for a performance comparison and I can’t get it working. It just outputs the correct size spread of 0s.

I’ve tried:

  • changing the help patch you’ve supplied so it just has a single vector input.
  • messing around with all the parameters for the node
  • the lastest _Utils.dll from the resposity and the one you’ve included in the ZIP file
  • the lastest _PluginInterfaces.dll from the resposity and the one you’ve included in the ZIP file
  • checking all this on another computer
  • checking other plugins such as timeliner and they work
  • changing the location of the undistort.dll, and
  • thinking of other ways to test this.

Any ideas please?

#9

hello, it should work like:
replace _PluginInterfaces.dll and _Ultils.dll in vvvv plugin directory (with the ones from the .zip). place the help patch beside the Undistort.dll and start it. make sure its beta17.

#10

Just tried it again. Still doesn’t work. Confirmed I am using beta17. Tried it on a different computer, beta17, didn’t work.

Can somebody else try this plugin please, it does not require a video source, it only maths and supplied with a help file.

Thanks.

#11

Okay, I Have Downloaded Tonfilms Undistort.zip, extracted it, and moved the contents to vvvv_40beta17\plugins, overwrite the old ones.

Next I downloaded the Undistort (2d) help.v4p and also moved it to vvvv_40beta17\plugins, when I open it, all looks like it should. (cool plugin BTW :) )

Now, perhaps you have more than 1 instance off Beta17 on your computer and are just using the wrong folder? Or even worse, you missed the fact that you don’t have Net2.0? (read THIS POST) Do other plugins work for you?

#12

thanks West. that suggested it defiantly me, so downloaded most recent vvvv from site to make sure and tried it on my nextdoor neighbours vista laptop and it works fine!

Just uninstalled vvvv and my machines. Just reinstalled Beta17 from new download, copied the files, and wait for it… it worked!

So who knows. very odd but one of the computing joys we all know and love.

Thanks guys.

I’ll get back with a performance comparison. I’m sure it will make a difference thought.

#13

Briliant. Working great with my project, well done for pushing on with that tonfilm, I know you said you were busy.

I did start trying to make it as a module just for a single x and y pos but got a little hung up with it and had more pressing issues with my project.

It has made a really decent performance gain for my patch. Difficult to measure a value for it but it is significant. Not only does the CPU not need to process my source now before the tracker, I think using unprocessed video may be more efficient for the tracker because it is not dealing with the loss of video quality experienced with using Undistort (DShow9 OpenCV).

Thanks.

#14

thx, for testing, i will need it sooner or later too. good luck with your project.

#15

ummm… it seems my first testing some how didn’t succesfully work. I don’t think I was using the numbers from my undistort(openCV) text file and because I was just testing performance I didn’t do the calibration for my project (not a easy process at the moment, needs work) and assumed it needed doing. The performance was great but now I have calibrated everything there seems to be a problem with you wonderful node.

I’ve attached a patch that should show the issue. I think the patch does prove it is not undistorting properly, possibly I’ve got my logic wrong, but I’m fairly confident.

The numbers for the distortion, principal point etc are copied from undistort(openCV) when I have loaded my saved setting after the auto undistort process. When used with the undistort.fx or undistort(openCV) they %100 undistort my 2.1 lens when used with my cam at 320x240. You can see this to be true from the pattern shown in the left of the renderers.

Have a look.

Thanks.

undistortTest.zip (3.2 kB)

#16

any joy progressing on this issue?

Can anybody else confirm my issue with the great new node?

Thanks loads,

#17

yes, sure, i’ll have a look at it, thx for debugging!

#18

the problem is that both the opencv and the effect code snippets are meant to work on texture coordinates and from the perspective of the output image.

you can imagine a loop going over all output pixels, row by row, pixel by pixel.
the opencv undistort code (and its effect copycat) get the following job done:

  • we have the destination texel already
  • what texel of the source image should be displayed here?

so we have a function like:

sourceTexCd = undistort( destinationTexCd )

outputcolor = sample source image at sourceTexCd

now when we want to take arbitrary positions (not fixed to a known output grid) we need the opposite of that function.

  • we have the source position within the source image.
  • we need to know the destination position.

so we need something like:

sourceTexCd = asTexCd( sourcePos )
destinationTexCd = inverse_undistort( sourceTexCd )
destinationPos = fromTexCd( destinationTexCd )

the problem will then be to write that inverse_undistort function. maybe we need a math program for that. get the formula in and solve for “p”.

for now we maybe should check if the original behaviour of the opencv / effect could be patched somehow with the new plugin (and get the same result).

therefore i used a pipet which samples the original texture. (so we again need to undistort destination texcoords to get source texcoords; and sample the color in the source image - like in the original code)

i found some small error in the plugin code, changed it and checked that in again, but somehow didn’t get the dll running. so if someone could compile it. pleeeaase?

in the following patch the middle iobox color should look like the renderer on the left. (with a freshly compiled undistort plugin)

can somebody aprove that?

testDistort.v4p (23.3 kB)

#19

ha! i got it working…
didn’t reflect the new plugin structure on my hard disk yet…

so here you have the new undistort behaving like the effect when used in conjunction with a pipet.

when used on positions (like you would probably want to) it behaves inverse.

so we would probably want to have two plugin nodes.

  • one working on texcoords (like this attached one)
  • one working on positions (source pos -> dest. renderer pos)

the funny thing is: in fact texcoords are distorted the same way like the camera distorts the outside world. so in fact the image undistort is based on a texture coordinate distort. (in opencv and in the effect)

so the naming should reflect that maybe?

hmmm.

Undistort.dll (20.5 kB)

#20

thx gregsn, i got stuck on exact that point, as i realized, that the undistort of the coords should be the inverse function of the shader code. using negative distort values did not make the exact inverse… do you know how to invert it?

© vvvv group