Hi,
is it possible, to write FreeframeGL plugins (which use OpenGL GLSL shaders) and use them in VVVV? Is it possible to combine the advantage of spreaded input/output of the “extendend” Freeframe spec by VVVV group with the advantage of using GLSL shader in the freeframe plugin code?
Another question: is it possible to use NVIDIAs CUDA technology in freeframe code? Do you expect any problems due to using DirectX resources already managed by the VVVV application?
helo,
as indicated elsewhere FFGL plugins will not run in vvvv and there are no plans at the moment to support them, as vvvvs HLSL effects should have quite similar possibilities.
*is it possible to use NVIDIAs CUDA technology in freeframe code?
hm…isn’t cuda thought for general purpose gpu usage? and not specially for image manipulation? thought one would rather like to write a vvvv plugin using cude.net, instead of a freeframe plugin. but i might be wrong there.
please… if anyone’s up to this… don’t use CUDA but OpenCL
(«…Nvidia announced on December 9, 2008 to add full support for the OpenCL 1.0 specification to its GPU Computing Toolkit…» they’ll do this through CUDA somehow)
out of interest: why openCl instead of cuda?
'cause CUDA ist nvidia only and this is the OpenCL working group:
ic :-)… no text …
hm…isn’t cuda thought for general purpose gpu usage? and not specially for image manipulation? thought one would rather like to write a vvvv plugin using cude.net, instead of a freeframe plugin. but i might be wrong there.
ok. But is it possible to write a vvvv plugin instead of a freeframe plugin, where the resulting node has a video input and output pin (like the freeframe node would have)? I didn’t find a suitable node type in the specification of vvvv plugins (PluginInterface1.cs)… I’d like to do video manipulation using Cuda from within a plugin.
hi dr jones,
unfortunately the current plugin interface has no support for video in/out pins. but the plugin utils support shared memory: http://vvvv.org/pluginspecs.
with vvvv’s shared memory nodes you can write and read video into shared memory.
but never tried it, i would be curious if it works…
I’ll give it a try, when I’ve got some spare time…
Meanwhile I experiment with CUDA. I wrote a freeframe plugin for background subtraction which calls cuda code. When I use this freeframe in a VVVV patch, I see the background being subtracted, but there are heavy tearing artifacts (even if I switch VSync on in the NVidia settings). I don’t know yet why this happens. What does VVVV do, when processFrame is called? Does it wait until processFrame returns or is the scene being drawn meanwhile?
Has anyone a suggestion what might cause the tearing?
Is it possible to get the DirectX/D3D device or render context used by VVVV from within a freeframe plugin?
helo drj,
are you using
EnterCriticalSection(…)
LeaveCriticalSection(…)
as shown in the TemplateExt example? if not, try it, this could be the reason for the tearing.
in vvvv all video processing is done via directshow which runs in a separate thread. using the critical sections as shown in the template ensures that vvvv is not drawing its scene while freeframe writes an image.
Thanks for your reply, joreg. I use Enter/LeaveCriticalSection. More precisely I call EnterCriticalSection before the first access to pFrame and call LeaveCriticalSection after the last access to pFrame.
So I can be sure, that the problem is not because VVVV renders/uses backbuffers or whatever on the graphics board while freeframe uses the grahics board for CUDA computations?
“So I can be sure, that the problem is not because VVVV renders/uses backbuffers or whatever on the graphics board while freeframe uses the grahics board for CUDA computations?”
hm…now thinking about it again, what i said above may be wrong. the critical sections are only syncing the setting and getting of parameters so that freeframe isn’t computing new output parameters while vvvv is reading them.
graphic card access is not synced between vvvvs drawScene and freeframes processFrame since this was not necessary so far. now i am not very optimistic about this…
And pFrame might be read by VVVV while freeframe writes to it? Hm, then I’ve got a real problem… But if that’s true, everyone would get strange artifacts when using freeframes/plugins and manipulating pFrame’s data, won’t they?But since I didn’t get any artifacts before using CUDA, I guess that VVVV does already somehow assure that pFrame is only rendered, when noone else writes to it…
Do you know if - in case I can get the render context of VVVV - I can sync somehow the rendering of VVVV and the freeframe processing?
If it’s not possible yet, it would be great to allow syncing in the next version of VVVV.
ouright, indeed the whole truth is as follows:
but still it may happen that while freeframe is in processframe vvvv is accessing the GPU. since freeframes didn’t use the GPU so far, this wasn’t a problem. i mean if this is really actually the source of the problem…the dual access to the GPU causing that tearing.
Do you know if - in case I can get the render context of VVVV - I can sync somehow the rendering of VVVV and the freeframe processing?
it is not possible now. in theory of course we should be able to do this, but it doesn’t sound good to me to lock vvvvs drawing mainloop to the dshow thread…would need quite some hakcing…i don’t see this happening anytime soon.
do you actually see performance increases with using CUDA for what you are doing?
do you also get the tearing if you have no Renderer (EX9) and no VideoTexture in your patch? i mean only using VideoOut?
have you tried both VideoTextures, the default and the YUVMixing?
I didn’t do any performance tests so far, but I expect an increase, if things like “memory coalescence” are kept in mind when coding CUDA (some sort of memory prefetch on the GPU - could be very useful especially for stuff like highpass filters accessing neighbouring pixels).
Yes, even if I use VideoOut instead of VideoTexture together with Renderer(EX9) and Quad. It doesn’t matter if the default or the YUVMixing VideoTexture node is used.
If I got it right, the problem may be due to VVVV accessing GPU memory while my CUDA code accesses the same memory…?
Could you please tell me how to do this?
I’ve downloaded the graphedit tool and executed register.bat. I clicked “File->Render Media File” for my freeframe dll and the input video file. See attached image file. How to proceed?
graphedit_test2.JPG (25.4 kB)
huh. how does the freeframe.dll open without the wrapper?
anyway in graphedit you can also open your camera. in a new graph do:
like this you should have a running graph with your freeframe plugin.
I have to use the video file. So instead of the camera filter I used the “AVI/WAV File source” (DirectShow filters).
When I open the wrapper properties and browse for the folder where the freeframe is, it doesn’t show up in the ListBox. Only FiducialTracker is visible (although with VVVV I can add all freeframes in the folder to a patch). I tried two different builds of GraphEdt - same problem. When I try to save the graph, the application shows an error message: “An error occured when attempting to open the file. Schnittstelle nicht unterstützt [[unsupported interface]([unsupported interface)]. Return code: 0x80004002”. Strange, that it says that I attempted to “open” the file though I clicked “save as”…
i am not sure about the “AVI/WAV File source”. i always use the “FileSource (Async.)” which is also the one vvvv uses internally.
When I open the wrapper properties and browse for the folder where the freeframe is, it doesn’t show up in the ListBox. Only FiducialTracker is visible
make sure you have included all paths of dependencies in the system environment variable PATH. so for example for the other vvvv freeframes to show up you need to include a path to vvvvXY\bin which includes the openCV .dlls. not sure whats neede for the CUDAs…
When I try to save the graph, the application shows an error message
ja, that was due to the missing IPersist interface in the DSFreeFrameWrapper.ax. i fixed that for beta>20 and in the attached file.
DSFreeFrameWrapper.ax (568.8 kB)
