Forum

Boygroup Video Sync (and the lack thereof)

vvvv beta bug
#1

Correct me if I’m wrong, but it appears to me that when playing video in a boygroup setting, that in effect “sync” means “within one second of each other”. I’ve worked in video for decades, and this is the first time I’ve heard “video sync” to refer to anything other than “frame”.

Am I missing something? I find it hard to believe people are using this in places with multiple systems playing a video where parts of the composite display can be up to a second out of sync with each other. And yes I know you can set the tolerance to below a second, but trying to get it within any reasonable concept of “sync” results in the systems thrashing doing constant seeks.

The problem can be seen quite clearly here:


where the first video (coral reef) is synced pretty well, but the jet fighter one is painfully out of sync (the whole display of 21 monitors is running on four boygrouped PCs, in this case using the VLC player node). As the videos cycle, it is pretty random how well they sync up, and I have tried all sorts of tricks to make sure starts were sync’d, pre-searching was done and waited for on all boxes, etc. etc. etc.

HELP!

#2

hei mediadog,

did you check the documentation on video-synchronization? you can definitely do better than “within one second”. not with the VLC nodes though as they won’t work with the available sync-mechanism in vvvv.

you can probably not get 100% always-reliable framesync with vvvv-on-board means but people have developed their own custom ways to realize that with vvvv for high-demand commercial applications.

#3

and there is player (ex9.texture-timebased) and player (dx11.texture)

#4

It’s possible. Of course there is a lot of experimentation and know-how behind it, but it can be done with vvvv as it is – except for hi-res 4k uncompressed playback @ 50 FPS (with more than one layer) . Have a look at this video (starting min 9:35) for example. It was 4 LED walls each split into two or three 4K Signals, 10 playback machines nsynk + some more for backup and satellite feed. The show itself is rather boring though :)

#5

@bjoern but was that really vanilla vvvv or some custom nsynk syncing-technology?

#6

@joreg, yup read all the docs and initially tried the DX9 version. It is more stable/consistent than the VLC version, but even it commonly can get out of sync from the start and just stay that way - always by a noticeable fraction of a second. Interestingly when I have seen this happen, all the clients will be in sync but out of sync with the server PC. I have seen the exact same behavior on different setups.

With the DirectX version it does seem sync is much better after the first play when it was being looped or played repeatedly. I tried the VLC version because the videos I was given were .mp4s and I did not want to degrade them by transcoding (the 360 videos are 4K, but they are blown up tremendously in mapping to the screen window).

@sebl, I have used those before, but in this case where I have five 4K videos of 2-5 minutes each at 30 and 60fps, my guess is that’s just too much image data! And the video is playing across two graphics cards.

@bjoern, I’m curious what this know-how is; I believe that it is possible to up the repeatability by tuning the video file encoding and program sequencing for a specific file, but that’s not a really usable tool. Throw in the requirement for being able to seek to a point and start playback and be in sync, and the underlying system needs to just work.

So what is the sync mechanism in the DirectX node? It looks like the VLC node just does a seek if the two get more than one second (or the amount specified) apart. Does the DirectX node drop frames if needed to catch up? Or stop to be caught up to? From the docs, I suspect there is some fudging of clocks being done in the hopes that the OS/DirectX will automagically fix it. Or is it something more precise? Thanks folks!

#7

@joreg

Well, the queuing (starting) was done using IRIS. But such a starting trigger can also be patched. Syncing was done using vaudio - so not really vvvvanilla but based on stuff that’s available to the public. Exactly how it’s done I am not at liberty to tell.

All professional media sever systems (D3, DVS, IRIS, Wings Platinum …) rely on specific formats/codecs. You can’t just throw any video at a system and expect it to magically play it frame-sync across several machines.

#8

This simply depends on your hardware, but doesn’t sound impossible at all to me. A quick test should definitely be helpful…

#9

@bjoern I would be pretty happy if it was just a matter of using a certain codec, but it’s not - I have different clips using the same codec with the DirectX node and one will always be in sync and another may have problems.

Back in the early 80’s I started an interactive video company doing seamless real-time interactive video using multiple videodisk players controlled by Z-80s over 9600 baud serial lines. We did scene switching in the vertical interval over multiple video streams with random seek etc. and never dropped a frame. 35 years later, with processor and network speeds what they are, how is this still a thing?! We are talking about 1/60th or even 1/25th of a second, a virtual eternity in compute time, and I’d be happy with +/-1 frame - reliably.

I expect it is not because it is a hard problem (it isn’t), but that it was not considered in the design of the windows/whatever playback stack and it is having to be dealt with as an afterthought at an inappropriate higher layer that just does not have the right handles to grab it with.

So I’d rather spend my time addressing the root issue (if I can come to understand what it is) than spending time repeatedly converting and transcoding and testing and re-testing each individual clip forever, or figuring out some black magic kludge that could stop working if anything underneath changes. But before diving into the VLC node code (since it’s available), I was hoping there was a simpler reliable solution.

And @sebl, I’m looking at 11,000 or more 4K frames for just one clip - have you used those nodes for that kind of data? And what about audio sync? I’ll give it a shot.

Thanks, ever onward!

#10

An uncompressed 4K image, TIF or TGA for example, has about 26,5 MB (no alpha).
At 60 FPS you’ll get about 1600MB/sec which is achievable using a 4 SSD Raid0 / stripe volume. But this is just one layer, so no blending.

Even when extending the Raid by more SSDs and thus increasing the read-speed, you won’t be able to play two uncompressed 4K streams at 60 FPS in parallel with vvvv as is, the available texture players just don’t cut it.

Nice thing about uncompressed video is though: no artifacts & performance needed for reading/decoding a frame will always be the same regardless of the image content.

Using DDS / DXT1 you can get a compression ratio of approx. 6:1, so a Raid0 of two SSDs should be able to cope with two parallel streams. But there definitely will be artifacts. Decoding is done on the GPU which can be an advantage performance-wise.

Another possibility is using JPGs, but compression ratio and size depend heavily on quality settings and image content and so does the performance for decoding a frame. Decoding is done on the CPU and an heavily compressed image (let’s say a black frame) can really have a big impact on your framerate.

The same is true for video codecs that do compression. Performance for encoding/decoding is always dependent on the actual content, so it seems to make perfect sense that some videos play flawlessly while others don’t.

I wonder why media-servers addressing this issue cost that much money.

Maybe it’s time to fetch that stuff out of storage and get rich.

#11

OK, so with JPG compression I do get good enough 4K performance with Player on a high-end box. Works great and leaves enough CPU to do other stuff, so I’ll look into DDS and PNG.

Sooooo, why can’t a new Player plugin just use VLC or DirectX to retrieve frames from a video file instead of a folder of files? Seems like there would be minimal additional overhead if any as video files are designed for fast sequential access, far more so than discrete image files. FileStream (DX9 Boygroup) could run on the server node for providing the audio, and sending the frame number to the boygrouped player nodes. Still a kludge, but not nearly so as having to convert video files to images first.

(@bjoern, Yeah right! I gave those videodisk players to a hackerspace about eight years ago. I did keep one just for nostalgia. The central project tanked when the Japanese companies we were working with decided that their MSX systems would not make it in the US because of the IBM PC getting so popular.)

#12

depending on your graphical content png compression can really kill realtime playback since cpu decoding is quite heavy for e.g. really well compressed solid color areas.

and it’s not just minimal additional overhead to retrieve a frame from a video file instead of reading single images. think of I/P/B-Frames, interlacing etc. ever tried to get an exact frame via vlc?
but if you happen to know the easy way, industry will pay good money for that.

sure, having to convert to an imagefile sequence is an overhead. but if you think of playing a seekable videostream you’d have to convert anyways, since most of the videocontent will not be delivered in the right codec anyways (h264 and similars).

#13

If you really want to stay with “monolitic video files” try pic video mjpeg or an other codec that does intraframe-only compression as opposed to interframe prediction or make the export I-frames only. Use FileStream (DShow9) instead of vlc for better sync. Of course it won’t be spreadable and changing videos will also bring framerate hiccups.

#14

Yeah, I thought PNG might be a pain, but will test it anyway.

I understand the problems with seeking to inter-keyframe compressed video, believe me. Just to date myself again, I dealt with all this back with Video for Windows under Windows 3.1 for CD-ROM based interactive video applications. That was the last time I worked in this area, but I can’t imagine it’s any harder (or slower) now.

The problem I see is that going forward, I think most of the big media applications are not going to be big one-offs for shows where it’s justifiable to hand-craft content, but persistent public media installations, where the owners will frequently want to quickly and easily update content and have it just work around the clock. And these installations are going to be interactive, and all those expensive linear playback systems just won’t work.

So instead of saying this looks hard, let’s not do it, I see this as a big opportunity for vvvv (and those who wield it!).

#15

I’ve done this with the player nodes, and also with filestream dx9. WIth dx9 I use the system time pin to micro speed up/slow down the video and send the timecode via udp, pretty much the same with player nodes. You can just use the speed pin and little increments, o I’m a little fast so play at .999 for example.
Vlc will never do it as you cant get a reliable frame number out of it, and it pauses on loop.

#16

how about these fellas for 4k video playback with the texture player: http://www.intel.com/content/www/us/en/solid-state-drives/intel-ssd-dc-family-for-pcie.html ? the pricy one with 4TB can output 5 GB/s read speed for a mere $9000$ bucks, yumm yumm.

© vvvv group