Forum

Getting audio in sync with video?

#1

Hej,

I have (a probably common) setup where I have one server which is playing an audio file and four clients which are playing a video file. The server sends the time of the audio playback via UDP and the videos are synching accordingly. Everything works fine and I have used this method (but not the same patches) in a previous video installation without a flaw BUT …

The video is constantly running slower than the audio which leads to constant resynching and therefore jumping of frames. (In my previous installation this was not happening and I have no clue why.)

At first I measured it and the video is running about 0,6 seconds slower every 60 seconds of audio playback. After this I changed the mainloop to filted and patched arround and had a setup where the speed of the video is set to 1.006 and played back ALMOST as fast as the audio … but I am scared of this strange workaround.

I use videoout because videotexture->quad ueses more cpu-power. Currently I am playing back 2580*720 with about 18% usage of the cpu using the picvideo mjpeg codec which is very fast, also with synching … There are no dropped frames or anything else that indicate that the PC has a performance problem with playing back the video.

Does anyone has any idea why the video is played back slower? Would it make a difference if I would add an audio track to the video or add a video to the audio so that server and clients are playing back the stuff at the same speed?

Thanks for reading/thinking/helping.

#2

hello serik, if its easy, i would exchange the patches and content of the computers and see what happens. if then the audio is slower than the video, its a matter of the computer clock. if again the video is slower it has something to do with the files you playing back.

#3

hej

thanks for the answer. its a good idea but i think i checked the computer clock idea already with this:

i ran a test with two timing nodes (client/server) parallel to the video and audio and the timing nodes were in sync with each other, also the audio and timing but the video and timing node not. i guess the timing node is getting its time from the computer clock, right?

i do not have the pcs here at my place but tomorrow i will make a test playing not just audio but video and audio on all pcs. maybe this will work. but still the audio/video behaviour is strange.

#4

hello,

we made a few tests.

first we changed it that the server also plays video and the players also play audio. it seemed to work but then again not. we restarted vvvv because we mest around with the audioout a lot (create … delete … create … delete) and after the restart everything worked fine. we tested the whole thing about 60 minutes and everything stayed in snyc.

one day later we came back meanwhile the computer where off and nothing has changed but the patches did not work! we checked everything and this time the server was playing the video with audio with slightly under 25 fps and because of that the sync problem was there again. we double/triple/quadruple checked everything and nearly jumped out of the window. then we restarted the server loaded the patch again and it was working again and everything stayed the same.

on both day we checked the timing with the timing node all the time and it was ticking correctly on all PCs with or without the syncproblems. we have absolutly no idea and we are a bit scared because soon we have to show our installation infront of people and fear that the sync problems appear again out of nowhere …

any thoughts on this would be apppreciated.

#5

is your videos fps a multiple of your monitors hz on all pcs? eg. 25fps -> 50hz

do all pcs have the same soundcard? if it fits with the kxproject you could try their renderer on all pcs.

#6

the fps is 25 on all videos. the displays are tfts so probably running with 60 hz. would it matter if e.g. the server is running with 75 hz and the players with 60 hz? but even though it would be odd if it changes by restarting the computer and do this only on one computer.

the PCs are all the same (some modell from acer - i think veriton) so they all have the same onboard sound.

what exactly is the benefit of the kxproject in our case? what do you mean by renderer? … i suppose not video since it is an audio driver only, or not? i am just sacred to install drivers when everything is basicly running … but just sometimes not. maybe after i install the drivers it stops working at all … and people would throw old fruits and eggs at me. wah!

#7

for a video with sound, the video-timing is typically based on the soundcard´s clock. for a video without audio windows might resort to the graphic cards clock.

vvvv´s other nodes use the main cpu´s clock.

these clocks are physically different devices which are not connected at all.

all clocks vary with temperature. i would not expect them to go off by factors like 1.006 even with harsh temperature changes, but who knows what manufacturers do these days to save a few cents.

so it might be a heat issue.

buggy overclocking / energysaving features of your machine might change the clock speed of your cpu without telling windows. i actually have never experienced this anywhere - but i see this as a plausible issue. so update all drivers; closely examine the bios (the computers might be configured differently and deactivate all strange features.

from what i have read above i am not sure if you exactly did tonfilms test above - his procedure absolutely make sense.

#8

thanks for your answer.

I forgot to mention that i already did tonfilms test today. on one PC the server patch was running two slow and on the other where the player was running correct the server patch was running correct too. so I suppose it is not directly or not only a problem with the patch itself.

the heat issues you mention are very interesting and i didnt know about the different clocks either. i dont believe it is a heating issue because we ran the server one hour wiht a constant different timing than the other PCs and did not change the load of the cpu and after the restart everything was the same (load of the cpu) and the sever was running exactly like the others and did this several horus without a prob. after serveral hours we restarted it and it was still running correctly.

the clock thing is interesting though. is it somehow possible that the random god decides to choose not the same clock each time the server patch is started?

tomorrow i will see if some energysaving things are enabled or if I find any other strange setting in the OS or bios. but they should be the same on all PCs since they were deliver together and are all the same. noone ever worked with them. only we used it. we installed vvvv and copied some video and audio and patch files on it. thats all aside from doing some obvious things like turning of the screensaver and power off things and changing the desktop resolution to horizontal span.

#9
  • the fps is 25 on all videos. the displays are tfts so probably running with 60 hz.
    to get smooth running video at all you need the monitors rate to be a multiple of the video framerate. i would ensure this first. if 50hz is not possible then 75hz very likely are. else i would suspect videoplayback to trick around with timing to get the 25fps squeezed into 60hz…

as oschatz explained having all videos playing with sound on server and clients makes sense so that they all use the onboard soundcards clock.

the kxproject renderer is a kernelstreaming low-latency audio renderer. after install, selectable as device on the AudioOut node. i’d suspect this thing to probably be more optimized in general than the default directshow audio renderer. although this suspicion could also come from superstition, as i have never really tried it.

and then: how do you do the syncing? like in tonfilms video-sync patch?

#10

as oschatz explained having all videos playing with sound on server and clients makes sense so that they all use the onboard soundcards clock.

it is already playing video and audio on every PC.

i briefly took a look at tonfilms patch and it seems like it is very similar. but the syncing part works just fine with my patch only that it has to sync too often because of the different timings of the video files playback.

as I mentioned earlier in this thread we restarted the server two days ago and the error was gone. since then it has not reappeared. we tested the setup two more times the last two days but the error is not coming back … hopefully it will never come back cause since it is not there at the moment there is no way of telling how it got there last time (only if we try to bring it up intentionally).

since we will use the videoinstallation in the next two weeks on different locations we will gain more informations about this error. if it occurs we keep your suggestions in mind with the kxproject and the refresh rate. hopefully everything works. i will report how it worked out in two or three weeks or maybe scream for help before then.

thanks for your answers/suggestions.

#11

hej,

we showed our work over the last two weeks a lot of times and the error (that one computer played video/audio faster than the others) did not occur anymore.

from three one hour playbacks i recorded logs and 4 computers togehter synchronised around 60 times. this makes one synchronisation on one computer every 4 minutes. this is more than i hoped for. the patch syncs when the video is more than 1,5 frames off and syncs at max every 0,25 seconds. i wonder if the many syncs came from cpu-usage-peaks from the video playback (but many syncs where in black video parts ?!) or something windows did in the background (cpu usage / hdd usage - although we tried to shut down unnescessary services). strange thing though that it seemed that there were a lot more syncs in the last half of the one hour piece …

well as you see i am not completly satisfied with how my patch worked out … but overall i was very happy that it worked everytime and no other problems occured.

if somebody is interested i could post my patches here. with a brief look at tonfilms patch i suppose the major difference is that i didnt use boygrouping.

© vvvv group