Video to sound

vvvv beta question
1

Hi

i would like to be able to transform videos to noise.

i want to achieve this by

  • first distort a video frame to a 1 px wide, but full height strip
    (this way the pixels contain averaged color data of the full width)
  • after this i want to strech out this 1 px wide strip to full width again
  • and timewarp it - rendering each frames strip next to the previous one, as in the basic timewarp example in the girlpower folder.

the image saved this way can be played by an imagesynth, and synchronised with the raw footage.

the problem i don’t get, is the double transform trick, can anyone help please?

thx (i attached a test which wasnt done realtime)


the sound of this image:
http://picosong.com/ksRc/

2

i think the color averaging is not work by simply scale down the width of the video, because its not aliased.

probably what i did in this patch is doing the exact same thing as an unscaled video would produce with slitscan…

video_to_sound.v4p (15.9 kB)

3

this one is not easy and took me some time to get it right. it needs a good understanding of the rendering pipeline.
there are some tricks you can do with color blending to add all pixels of the width on top of each other. but you have to use a float texture format and a custom shader because of the small values. i have written some explanations in the patch:

video_to_sound.zip (5.9 kB)

4

thank you