MIDI Polyphonic Expression. Anyone?

[robe]

Hello, The fabulous community of VVVV users and devs!

While I’m trying to follow the promising direction the new gamma version is taking(Stride PBR, Fuse, ML, OpenCV, Kairos), I’m also experimenting with my silly midi stuff…

I’m Roli Seaboard’s owner, and I enjoy playing MPE. I think they’re more expressive than regular midi, and I’m building some skills on it.

Yes, but what’s midi MPE? You may ask.

from Wikipedia:
“MIDI Polyphonic Expression (MPE) is a method of using MIDI that enables pitch bend and other dimensions of expressive control to be adjusted continuously for individual notes. MPE works by assigning each note to its own MIDI channel so that particular messages can be applied to each note individually.”

Actually, Every note could carry three more real-time streams of data with it. The marketing of Roli “invented” “the five dimensions” of touch…

Every NOTE ON has

1. Strike = which is the Velocity (they changed the name)
2. Glide = is the Pitch Wheel, but for every note and is set to +24 -24 range
3. Slide = the associated behaviour is something like the Modulation Wheel but for every note but is sent to CC 74
4. Press is the pressure, so you can also add more control over the initial Velocity or other cutoff or any other control. I can receive this data in a foreach[reactive] from ChannelMessageFilter through the Data1 node
5. Lift is the release velocity or speed of liftoff from a keystroke Lift is transmitted as MIDI release velocity (not interesting at the moment)

All this is possible thanks to a really tricky data stream management of the MIDI standard. All channels 16 are used to stream those five dimensions of multiple notes. Channel 1 is used as a master channel which… `I don’t know what I’m talking about… you can read detailed info on the actual bits used here

I managed to receive

• Channel of the Note (I’m ignoring it, for now, working a solo instrument…)
• Number of the Note

1. Velocity of the Note

through a Split MidiNote in a ForEach[primitive] from NoteState node

I can receive the Press from Data1 from cahnnelMassageFIlter set to ChannelPressure as I said…

and the Slide from Data2 of another ChannelMessageFilter setted to Controller

Unfortunately, I didn’t find a way to access all the other dimensions in a non-reactive way and get a lot of trouble merging the data (Should I use the Channel info to associate the data to the note?)

HoldLatest, Sampler and S+H are not helping out of the reactive region in this case. The output of Sampler is not consistent. I expected to have all active note out of it, but they shuffle even on a stable chord.

My ambition was to recreate the visualizer of the Roli Dashboard to begin, but I’m a bit stuck.

So I created a LinearSpread of cubes and scaling them in the attached patch based on the note number velocity. Any attempt to merge the data from the reactive regions has been a failure.

What would be the best approach?
Would it be possible to do this in a non-reactive way?
If the reactive is necessary, how to manage the rendering stuff?

If anyone is interested in this MPE topic, you can also DM me in element.

Cheers and Bye!

Thanks to everybody in the VVVV group and the community for creating and contributing to this amazing and unmatched software.

MidiMpeTest.vl (68.8 KB)

[robe]

Hi again!
After some time, I’m responding to my old self and all mpe people.
In the meantime, ROLI went bankrupt and the midimpe struggles to find its users… :)

Investigating on Vl.IO.Midi I found an interesting pattern used in ChannelMessageFilter:

If I understand it correctly, this Where region is filtering Observable using ChannelCommand.

Here’s is what I get today on a second try :

Filtering NoteOn
Data1: Note Number
Data2: Velocity

Filtering PitchWheel
GetPitchWheel

Filtering Controller
Data2: Slide

Filtering Channel Pressure
Data1: Pressure

The Lift part (MIDI release velocity ?) remains a mystery to me, but I don’t care now.

Correct me if I’m wrong, but I cannot do something like this:

If I do it, one of the two takes priority; nonetheless, I need Data1 or 2 in different situations…

So:

Is there a way to join data from different Foreach Regions to fill in my MpeNote Record abstraction?

Is the Concurrent Mutable Dictionary a plausible way?

Thanks in advance for reading.
Bye!

[tonfilm]

@robe

Correct me if I’m wrong, but I cannot do something like this:

you can do whatever query you want there, as complex as you can think of.
just make sure it is in an If as in the ChannelMessageFilter or a Cache region, because the where region will create new observable and queries every time it is called. it looks like you placed them directly on update, which will re-create them every frame. you only need to call that once to setup the query, then it can run for as many messages as come in.

if you want to learn more about that, watch the “Introduction to Reactive Patching” here: https://thenodeinstitute.org/courses/node20-vvvv-workshop-bundle/

[joreg]

@robe

Is the Concurrent Mutable Dictionary a plausible way?

either Mutable/ConcurrentDictionary that you access from within the reactive region, or using a Sampler that returns all messages that arrived since the last frame, and iterating over all those and updating a normal Dictionary.

but ja, generally my understanding is that you want to build a state for each note by accumulating all incoming messages. so you’ll have a dictionary with key=notenumber and value being your MprNote record.

you’ll be filtering for all the desired messages in the same Where region which would have one If region per case, like: for every message you find the corresponding dictionary entry via its key (assuming the NoteNumber is always clear with each incoming message?!) then if the message is NoteOn, modify the Velocity of the MpeNote, if the message is PitchWheel, modify the Glide and so on…

[robe]

Thanks, @tonfilm and @joreg!

I need more time to understand reactive patching for sure… I’ll try the dictionary way as soon as I’ve my mpe keyboard with me at my office. Maybe is way overkilled for this case or actually not doable…

In the attached mega-patch I’m using(wrongly) the pattern from ChannelMessageFilter as a workaround to get all the messages I need.

MidiMpeTest3.vl (154.6 KB)

But I don’t think in every case I can get the NoteNumber to build the key of the dictionary.

The VL.IO.midi is conceived and built upon the idea of standard midi, also the abstractions for a note and controller are divided in MidiNote and MidiController for example, which is not the case in midimpe where the idea of note and controller are merged…

mpespec.pdf

The best approach would be to start from scratch, maybe with the same ChannelMessageFIlter approach, but filtering all the messages needed at one time together…

I’ll tinker again with my workarounds, then I’ll try to do it the right way
Thanks