OPTICAL FLOW to GPU DX11.
some of you might have seen my humble port - but if you tried it… you might came across its limitations.
Optical Flow is actual a quite robust feature to detect and analyse motion in images and since opencv is dead - it might be our best!
http://cs.brown.edu/courses/csci1290/2011/results/final/psastras/
I think it would be a major impact on vvvv if we would have that ish!!!
Anyone Ideas???
mio
opencv is not that dead it’s just a pain in the ass to compile :(
You r right microdee…its not dead - i wanted to be a little provocative…
I was also not referring to opencv for vvvv - but in general / and yes i dont believe
In the future of its algorithms for basics like contour/ backgroundSub or optical flow…
Most of those mechanizms will be translated to gpu specific custom solutions like you see in
Dx11 pointcloud ect. IMHO
there is another nice code base using realtime optical flow shader.
maybe worth checking it out but takes as a lot of time to translate into dx11 shader
Interesting post
+1
Just a quick note:
flowabs is a still image rendering technique. As far as I understood the flow used there is “edge tangent flow (ETF)” as to be read here: http://hpi.de/en/doellner/rendering/flowabs/year/2012/102089/WKO12.html
That’s not the same as optical flow. For OF you need two successive images of a sequence which is not the case for the mentioned NPR technique.
yes i also dont think that the algo from HPI has something to do with optical flow.
i do believe that the algo described here is a nice starting point
http://cs.brown.edu/courses/csci1290/2011/results/final/psastras/
now im not sure how to solve LK (Lucas Kanade?) Least square…
or how to create the image pyramids…how should this be interpolated…anyway maybe we can put our heads together here:
anyone?
ah sorry makes sense.
We should be able to pull this off
http://www.clicktorelease.com/code/optical-flow-webrtc/
no we must!!! lol
ok im so really close now…choose GPU and you see real vectors with variant size in a browser
http://wwwcip.cs.fau.de/~simakamp/opticalflow/opticalflow.html
the corresponding description is ja quite easy
well it doesn’t correspond to reality at all if i need to watch the direction of the red lines. however look at maybe gpu accelerated video encoding papers, i’m sure you’ll find some great examples of optical flow implementation.
This is the most impressive stuff i saw so far, and it’s 6 years old, not sure if color encode the 3d velocity direction:
I dont really care if its color coded or arrows. However arrows is standard since 34 years - and i think it might make sense - but who cares - its like inventing a depth buffer and discussing the red look to it. I think its not the point.
@sanch- what u see there is the cuda implementation of standard optical flow IMHO - its not a 3d vectorfield - its 2D - however still impressive - and most definitly what im looking for too.
if you are interested in 3d Flow from 2 1/2D images then please search for “SCENE FLOW”
To me this is one of the best examples i saw so far. But i dont know someone that can pull this off.
anyone?
I always liked Optical Flow and would be great to implement it in vvvv… I’m sure you’ve seen this one too, about 5 years old:
he post the processing code here: http://www.openprocessing.org/sketch/10435 but you need to download it and run from your machine.
And another processing sketch based on the last one: http://www.openprocessing.org/sketch/84287
scene flow looks promising to me http://epubs.surrey.ac.uk/380512/1/Kinecting_the_dots_Particle_Based_Scene_Flow_From_Depth_Sensors.pdf
http://personal.ee.surrey.ac.uk/Personal/S.Hadfield/papers/Scene%20particles.pdf
hmm ok im on it…
im bad at reading math papers - but quite good at p5…
@metrowave - i have to redesign the p5 code as its architecture is driving me crazy…its unusual style to break down arrays of vectors into three array of floats is so confusing that i cant see the wood because all of the trees - lol - anyway the results are quite nice …and im nearly finished…and now it looks quite easy actually.
Scene Flow might be a complete new topic and would be really interesting. My only fear is that this stuff might be a little more intense on the gpu - and quite a heavy approach for “only” image analyse
btw i ported this code:
http://glip-lib.sourceforge.net/page__lucas_kanade.html
and the results dont look better then the optical flow in dx11 pack…and nope - i dont know why ;)
@princemio the Nvidia video is not standart optical flow, it’s DENSE optical flow, and the dense make all the difference.
The Dx11 version of optical flow is working fine, but is limited by the fact that if there is no color difference between 2 frames the algorithm is unable to compute velocity, the dense optical flow don’t have that limitation but is much more heavy to compute.