Asking here since I got no answer on the forum.
Is there a possibility of getting video-audio into VAudio, without using HAP?
For many projects, it is perfectly fine to use mp4 video. It is not unheard of to have an audio track alongside a video, and I think getting that audio into the audio engine of the “multipurpose toolkit” is not an unreasonable thing to expect from vanilla vvvv gamma.
I am quite sure that I am not the only one who miss better audio support in gamma in general and proper handling of audio from video in particular.
I noticed in the NDI audio receiver there is this ‘synchronizer’ that enables a IVdieoSource and IAudioSource. Given that the VideoPlayers in Gamma 6 are separated from the output sink (Stride, Skia) is there a chance a similar approach to this could be possible with the MediaFoundation?
Indeed some ground work has already been done when we introduced the NDI nodes. To my knowledge the current high level MediaFoundation API we use to grab the video frames doesn’t give us access to the audio data. I could be wrong here, would need to check.
However there would also be a different approach with a lower level API provided by MediaFoundation where one has full control over the playback by manually grabbing the samples (video frame / audio buffers). Some tests were done already with it just with video (video player, source reader). It was simlpy playing back videos as fast as it could. Changing that sample to fetch audio samples should be doable. The hard part however would be to get the synchronization right. Would be fun to get into that topic once more, but as always other parts in our universe need attention as well ;)
