I have now actually added it as another extension inside VL.Audio.HDE.vl, is that considered good practice? And one question regarding that - are VL libraries automatically pulling in an HDE file if it is placed in the folder? I could not find out, where it is referenced. In any case, please let me know if you have suggestions for improvements!
And by the way: Really enjoyed the extension patching experience and working with ImGui :)
Well, the input count is usually 2 (or more), so what am I seeing here in the spectrum? A downmix of all channels? Only the first channel? I’d expect a channel selector or (maybe even checkboxes, one for each channel, to make a sum from it) and the display would be showing the selected ones.
I’m really happy you started this, it makes me dream a little :)
My audio interface (MOTU) comes with a set of really helpful audio meters like FFT and spectrogram, Oscilloscope, X-Y Plot, Phase analysis, … Those can be nifty to have in the right situation (and some of them should not be too hard to add with existing nodes…
What I really like in my interface is a combined view for FFT (line) and spectrogram that looks like this (ignore the EQ curve). .
So, if you are ambitious… ;D
Ahhh, yes, I see what you mean now! I thought you were referring to the new channel concept. Yes, indeed, it has to have a channel selector, will add it :)
A spectrogram would be best achieved with a DynamicTexture, right?
yes, that would be one option.
there is even one already (implemented as compute shader) in the FFT 3D Waterfall of an audio signal example patch (coming with VL.Audio).
this one displays linear frequencies. a logarithmic view would be even better…
I wanted to do a line view anyway but was asking myself half way, what additional info it would bring. Combining it with the spectrogram, and providing a switch in the extension to toggle between both views seems like a cool idea!
When adding VL.Stride to VL.Audio as a package (to include the spectrogram), I am getting this error inside the the Record BufferWidget of VL.Audio.HDE
Can I ignore this? Or is there a way how I can deal with this? The type of the pad is set to Buffer, can I somehow specify that the Buffer of VL.Audio is meant in this case?
It also accepts the notation of Audio.Buffers.Buffer I think.
But on another note, is it sane to have VL.Audio depend on Stride? Just thinking of export size etc, forcing a package as a dependency on users which they might not need / only in the IDE … or does the compiler somehow get rid of it if the dependency is not used in the exported code?
I really don’t know and thanks for bringing it up! I actually just needed it for displaying the spectrogram shader… I guess that could also be done in Skia? But I really don’t know how and would need some help here on how to proceed. Otherwise I will simply remove the view for now and do it sometime in the future.
the VLAudio package itself and VL.Audio.vl should not depend on Stride or Skia, but the .HDE file is allowed to reference Skia or Stride, it won’t be in an exported executable.
And the Buffer ambiguity needs to be resolved by specifying which buffer exactly, as mentioned by others above.
I would like to ask if somebody can help with the shader, that displays the frequencies in the Spectrogram. Somehow the queued frequencies glitch over time and are not properly stored in the texture… do you see what I mean?
what I did in order to resolve this issue, in a custom tool for audio analysis and visualization, was to create a slit scanner texture fx instead of queuing spreads.
bellow, all the widgets are basically textures and the spectrum has a resolution of 3000 x 256 px (height is equal to my bands count and width to the time span of my excerpt).
So if I would like to have 1 sec of data in 60fps then my resolution would be 60x256, in order to face this as an issue you can always add a speed factor to determin how fast your slit scanner will accumulate data.
.
