Multichannel Audio in Video

Hello dear vvvvellows,

i have a show next week where i need to play a video with stereo audio in it together with an additional audiostream in sync. The video has soundeffects on it supporting the visuals and the additional audiotrack would be just a clicktrack as a guidance for the musicians.

So my question would be which of these ways would be best suited?

  • having a videofile with all audiostreams in it:
    this would obviously have rocksolid sync, but would filestream let me choose outputs on my soundcard for the different streams?

  • having a videofile with stereo sound in it and playing an additional audiofile with another filestream node:
    would it stay in sync for lets say ~4 minutes of show?

i am open to all sorts of suggestions if anybody has experience with multichannel audio in vvvv.


i don’t think you can multiplex a multichannel audio into a video and have vvvv play it back correctly.

what should work though is playing back the video (with stereo audio) and use its time to sync a second FileStream (DShow9) (playing the clicktrack) using the new boygroup-time-and-filestream-synchronization options.

i disagree, playing back a video with multiplexed multichannel audio can work fine in vvvv.

I am right now playing back a uncompressed avi with 4 mono audiochannels by simply connection audio out to filestream.
The tricky part is the multichannel audio interface and the driver.
The interface needs a multichannel driver, not a driver which only lets you select single channels or pairs of channels. And the internal channel mapping of the interface/driver has to follow the windows standard mapping. I tried several interfaces and ended up with a m-audio firewire 410, that is working perfect.

It is even possible to address other channels (in my case 5 and 6) with another audio out / filestream at the same time without any problems.

To make this more comfortable and more compatible with several audio-interfaces it would be good to have something like a “audio-channel-mapper” within vvvv … no idea though how this could be implemented.

This is the format of the video-file i am using (and is working) here.
I think it should work the same with compressed codecs that allow multichannel audio.

Complete name : xxxx.avi
Format : AVI
Format/Info : Audio Video Interleave
Format profile : OpenDML
File size : 340 GiB
Duration : 24mn 4s
Overall bit rate : 2 020 Mbps
Writing application : TMPGEnc Video Mastering Works 5 Version.
Writing library : TMPGEnc Video Mastering Works 5 Version.

ID : 0
Format : RGB
Codec ID : 0x00000000
Codec ID/Info : Basic Windows bitmap format. 1, 4 and 8 bpp versions are palettised. 16, 24 and 32bpp contain raw RGB samples
Duration : 24mn 4s
Bit rate : 2 014 Mbps
Width : 1 440 pixels
Height : 972 pixels
Display aspect ratio : 3:2
Frame rate : 59.940 fps
Bit depth : 8 bits
Bits/(Pixel*Frame) : 24.000
Stream size : 339 GiB (100%)

ID : 1
Format : PCM
Format profile : Float
Format settings, Endianness : Float
Codec ID : 00001000-0000-0300-8000-00AA00389B71
Codec ID/Hint : IEEE
Duration : 24mn 4s
Bit rate mode : Constant
Bit rate : 6 144 Kbps
Channel(s) : 4 channels
Channel positions : Front: L R, Back: L R
Sampling rate : 48.0 KHz
Bit depth : 32 bits
Stream size : 1.03 GiB (0%)
Interleave, duration : 17 ms (1.00 video frame)
Interleave, preload duration : 341 ms


thanks a bunch for the invaluable feedback, guys!

while i originally wanted to play my h264 encoded video (3072*1024) with filestream via coreavc codec i had to change plans as the coreavc shop site seems down so i can’t aquire the codec and other codecs did not bring sufficient performance with filestream. That’s a bummer, but Markus wise comments led me in the right direction and now i have it playing flawless on ft’s vlc plugin.

will research a bit further and report my findings back…

also an obvious +1 for an “audio-channel-mapper” functionality right in vvvv from me ;)

huge thanks again. love this forum! :)

is’t this iirKlang thingy for that?

irrklang is more for dynamic triggering of sounds i’d say.

(and ofcourse positioning audio in pseudo 3d space and lot of other cool things)

so, can it play multichannel audio?
guess some update on bass nodes will do job too

i think it might also internally work with vlc, but i’m not sure you prob need some arg to change speakers layout to surround

have not tested with irrklang as it would result in higher patch complexity, but vlc plugin just plays the audio according to current speaker layout. (filestream seems to do the same as markus hinted above) so if you play e.g. a 5.1 clip in vlc or filestream the audio channels will just be mapped to the appropriate output channels. so as this show is already way beyond it’s budget the client will get fantastic sounding el-cheapo onboard 5.1 sound… (don’t worry it’s just some effect sounds and the clicktrack for the musicians)

I agree with MSBERGER. Having audio embed in a video stream might not work that great but a good .avi stream output as suggested by the MSBERGER will definitely be a good choice.

[lidar mapping services](lidar mapping services)