Play video smoothly in DX11

vvvv beta question
1

Hi guys! Help me here

What’s the best way to play video in DX11?
Actually I’m using the FileStream (DX11.Texture Vlc) using different codec for render the video, like h.264 with different keyframe/sec and bitrates, or PICvideo m-jpeg (always 25 FPS, it is right?)

and video run always smoothly ENOUGH but never perfectly! Always there are small jumping happening here and there. What are the best setting in your opinion?

I’m also playing with MainLoop (VVVV) , setting FPS (foreground and background) to 25 or 50, or even more (but not sure if is better to set it to Raw, Filtered or Increment)

2

the video should have a framerate that is a divisor of the screen refresh rate. most computers have 60fps, so a video with 30fps will run smoothly.
if you play 25fps videos you need to set your windows refresh rate to 50. either in the windows settings or in your graphics card driver.

3

ok I check it, projector refresh is 60 hz and I created some 30 fps video
looks better, but still some jumping here and there. like, every 3 sec, then 4-5 little jump, then smooth for 7 sec, then again every 3 sec, etc etc

4

maybe that helps: https://vvvv.org/forum/micro-stutters-how-to-diagnose

5

yea i noticed that stutter too, i think weather vux should check it out or we desperately need an DxVA supported node.
On the other hand i think the problem might be in vvvv somewhere…

6

this is my “dpc latency checker”, looks all good here

my computer:
Intel® Xeon® CPU E3-1276 v3 @ 3.60GHz
16Mb ram
AMD FirePro™ W600 Professional Graphics
with windows7

I’m working with a monitor (1920x1080) and 4 projectors (1280x1200)

7

this is the perfmeter normally:

this is perfemeter with MainLoop in “Filtered” mode when the drop happen:

this is the drop with MainLoop in “Raw” mode:

and finally this is the “Increment” mode

even if the last case show fps=30, the drop is really visible
any idea?

8

Increment does not show timing issues as its ment for offline rendering.

So, why does it lag? If its not V4 - which nobody could seriously believe… - something might take the resources of your machine away from V4 every few seconds. (Either HDD or CPU in this case, GPU seems unlikely).
So for some reason V4 can not read the frames in time. This could be windows indexing around, or any other program or service running in the background. So if its not V4 itself - which as stated above is extremly unlikly… - I am afraid most likely there is no simple answer, you will need to dig into your machine and find the cause. If possible install a clean Windows on an external HDD and boot this. Disable all power saving options you can find, lock CPU & GPU freq to max.

If the problem remains its most likely a default windows service causing the problem, if not its something you installed to the system. Here you find out everything you did (not) want to know about windows services:

In audio world where everybody is running smallest possible latency even tiny peaks in CPU usage can lead to crackle. So its quite common to have a seperate extremly stripped OS for audio only. WLAN is often a cause, windows shadow copy sucks as well from time to time…

Last but not least: If your footage is not encoded in a way that every frame is a keyframe this might also be the cause.

+11 for DX11 video player with DVXA support. If it could even read AVI files with alpha using the system wide codecs and allow to scratch trought footage frame accurate without delay, oh my! Maybe even add a writer node? VVhat a vvonderful vvorld this could be!!!

9

there was some teasing on gpu accelerated playback going on one and a half years ago…
https://vvvv.org/blog/hapteaser

10

have you tried to increase the thread priority of vvvv?

it might also be the case that VLC plays in it’s own thread which sometimes hangs… to diagnose this you can move a thin line over the screen and check whether it stutters in the same moment as the video.

11

for all of you here are what i used recently:
if you need frame-by-frame scrubbing and fast playing convert video to image sequence with ffmpeg shellexecuted from vvvv save it in a folder next to video and use Player EX9 texture to read that folder, theres a module doing it in mcropack (called videoplayer), you can share the results from dx9 to dx11 safely with a hacky borderless 0*0 sized ex9 renderer which also is a module in mcropack.
OR
use CV.Image VideoFile node which has absolutely 0 control over the video but it starts when you enable it and pauses when you disable it. if you enable it again it will restart the video. It loops the video infinitely. thank God it outputs current position and the duration of the video immediately in real-time not like vlc so you can patch your logic with it. no seeking though

12

–UPDATE–
Instead of the VLC player I try a generated a 1920x1080 texture Lava (DX11.TextureFX) and it run very smoothly! So looks like the problem is in the VLC node or in the video encoding… but I have already tried every possible encoding option, unless I forgot something

13

make’s me think this is garbage collection problem
when u use lava effect it generates all the stuff for it on GPU and never leave GPU
with vlc it has to read video from HDD decode on CPU then transfer to GPU

anyhow i know everyone would love to have something that can play video without stutters and a don’t need any special treatment

14

There is also @woei’s Player (DX11.Texture) .

15

https://msdn.microsoft.com/en-us/library/windows/desktop/hh162912(v=vs.85).aspx

16

@bjoern
Discovered woei’s player have some ram issues with preloaded
Frames!
Each time the player flush the used memory ( you can see that in the windows performance monitor)
A frame drop occurs in the playback
Tested with a DDS sequence @ 4096x2160@25fps

17

First and foremost, everyone needs to keep in mind that vvvv is not a “real-time” system, because windoze, and all other general-purposes operating systems, are not real-time systems. This means there can ALWAYS be interruptions to program execution, at any time and of any duration. Sure, OS designers try to make things work as smoothly as possible, and there are things you can do to try to minimize the interruptions, but they are a fact of life.

Things CAN run pretty smoothly if your PC is running just one “foreground” process that does not need the whole CPU (which is why your Lava example works fine), but things get messy when you have multiple processes and/or resources running asynchronously that then have to synchronize to get s frame kicked out. I’m pretty sure the VLC node runs as a thread, which means that when vvvv is ready to render a frame, VLC may or may not have a new one ready to show at the right time. Depending on how the handshake is done, that can just mean VLC gives vvvv the same frame as last time, or worse, says gee I got nuttin’ right now and vvvv uses it’s last good frame from VLC.

The latter case, where there is a time period each frame of no good data, means that no matter how tiny a fraction of the overall frame time that may be, as vvvv and VLC rates drift past each other there can be periods when vvvv is talking to VLC at just the wrong time (known as a race condition). Until they drift past that bad timing alignment, vvvv may be repeatedly showing the last good frame it got, making for a very visible hiccup. The reverse can also happen, where asking VLC for something causes vvvv to miss it’s next frame update repeatedly.

Again, I have not looked at the VLC code to see if that is in fact what is going on, but it sure looks and smells like it from long experience of working with asynchronous systems. I ran into exactly this problem with the OpenNI Kinect driver, and had to write better synchronization code to address it - I would see exactly the hiccups in the incoming Kinect data as we see with VLC at times.

So my guess is that the VLC nodes need similar sync code.

And don’t even get me started about boygroup video sync! I don’t know of anywhere else that “within one second” is considered “sync”! Uh oh, I feel a forum post coming up…

18

Hey @mediadog really interesting!
There is some way to check if this is really a “race condition” problem? I mean, some empirical check without go digging in to the VLC code?

And in case, how’s complicated to prepare some sync code for the VLC node?

19

@Luper I don’t know off the top of my head how one could know that’s the case without looking at and instrumenting the code. It may also be the case of a poor “catch-up” process, where if video playback falls behind, it does not try to jump far enough ahead, resulting in repeated missed frames. Hard to say without some code archaeology.

If I can get some time and/or can’t find a good workaround to the boygroup sync issues I’m seeing, I’ll try diving into the VLC code. But at least in my initial glance at it that could be rather involved.

20

I also faced the same problem.I have been searching for the same information. I tried many possible processes for solving that converting problem but could not be solved. Then I using the clipgrab tool for youtube to mp3 converting purpose.