The real optical flow

You are absolutly right @sanch …standard was definitly the wrong term. What i meant is that dense is standard in OpenCV…Our DX11 Version is below standard and could be seen more as a super fast trick to get something like optical flow…its actually a ultra cheap Port i wrote - copied from a tool that should run in a browser 5 years ago…lol

The problem is that LK definitly increases the amount of texture lookups - as the values of the least square computation must be red from a kernel …and as i said i want to make it efficient and as lightweight as possible…

here is my current version of the derivates which is just difference of t and x, y with framedelay…its full HD and 120 FPS…now lets see how velocity and fps will look once i compute least square…


@princemio :

Main reason about that ported code is that you have a single iteration, and do no use levels.

You need to perform flow for several pyramid levels and propagate them back from higher mip level to lower mip levels.

Also the code from
Is pretty easy to port but not totally right, and can easily be optimized.

If you need a proper GPU implementation documentation, check cuda or OpenCL versions in OpenCV, they not really hard to implement in a mix of compute/pixel shaders, but you’ll likely need to wrap that lot into a custom plugin (If you want to implement multi level, for single version it’s rather straightforward, but then you lose quality)

hey @vux - thx for jumping in - and spreading some light!

the pyramidal levels is indeed an issue. I havent tried to do something like that myself, because i also dont know how to create several pyramids and blur them and then propagate them again

also from my simple understanding the execution time must tremendously increase with each pyramid - for blur , lookups and the propagation - please correct me if im wrong.

My other problem is way simpler - i now ported the simple example with one iteration - but i dont even know how to check if the frame updates.
Hence the optical flow is often executing on frames which are the same - which makes the result flicker on and off.
The current solution in DX.11 has the same issue.

Does anyone know an equivalent of “change-> sample and hold” for textures?

yea u can render to array slice in gs and do smaple hold on array

just wanna reanimate the thread
Is it possible to code it for real time?
approach info:

Super interesting post! Amy news on this?