Inconsistency detecting keyboard events with MouseKeyHook in Gamma

[TremensS]

Hey Elias,

Thanks for the extensive tests and reply!
Before your answer I had gone for the safe route and made a unique instance of MouseKeyHook’s GlobalEvents that forwards all the keys in send/receives across my patch.
I’ll definitely give a go to your suggestion when I have a minute!

In parallel, do you guys have plans to implement a keyboard handler that is independant from Skia/Stride and thus a little more universal?

[Elias]

Well yes, it would be very beneficial to have a unified input event handling across the different rendering APIs. We did a little research on that topic in the beginning of this year. Thinking was to use a cross platform windowing API with which we create the render windows and then leave it up to Skia or Stride to draw into those. So far we looked into Stride itself, as well as Silk .NET. If I remember correctly the APIs provided by Stride weren’t flexible enough (it’s also not meant for that) and Silk had bugs with multiple windows. In that time Microsoft also announced MAUI to be released end of the year, so we wanted to wait that one out as well. With 2021.4 finally at our doorsteps we will re-iterate over this topic in the coming weeks and hopefully get some clarity in our roadmap about it.

[tonfilm]

here is a global key hook that can be used in Stride renderers, replace the Notifications output of the keyboard device with the output of the GlobalKeybard node in this patch:

GlobalKeyboardHook.vl (75.5 KB)

For VL.Elementa, one has to modify the keyboard event source somewhere in the Elementa root node.

[bjoern]

@tonfilm

For VL.Elementa, one has to modify the keyboard event source somewhere in the Elementa root node.

any hint where that might be? @sebescudie maybe?

[bjoern]

Don’t know if this is the right place. But here is a hotfix for using Textfields with Stride’s SkiaRenderer based on MouseKeyHook:
TextFieldHotFix.vl (62.9 KB)

[sebescudie]

@bjoern

any hint where that might be? @sebescudie maybe?

don’t know on top of my head, but I can have a look.

edit: is the NotifyContext patch what you’re looking for @bjoern? It’s inside ToSkiaLayerInternal, which is itself in the Elementa root node.

[bjoern]

Thanks, already “found it”. It’s in the patch I posted.

[verstecke]

Hi! I’m trying to patch so that keyboard presses are presented as keycodes (eg. A = 65, B = 66) including when VL is not in focus. Here, key presses would toggle the boolean for that position in the array (eg. A pressed = 65 on, B not pressed = 66 off).

Ideally, this would also have per keyboard selection, so eg. I could plug in an external or bluetooth keyboard, and use that as a controller, while keeping the laptop key presses separate. This also wouldn’t receive key repeats, so the boolean would only change when the physical key is pressed and unpressed.

The next thing is to have each of those key presses send a midi note, to trigger in other programs.

I can’t figure out the receiving a key code part, let alone making an array from the KeyEventArgs which return a datatype I don’t know how to do much with.

I could do this in vvvv, but can’t in VL. Would be super useful to be figure this out, any assistance much appreciated.

[tonfilm]

@verstecke

I can’t figure out the receiving a key code part, let alone making an array from the KeyEventArgs which return a datatype I don’t know how to do much with.

I could do this in vvvv, but can’t in VL. Would be super useful to be figure this out, any assistance much appreciated.

Better put this question in a separate thread, it’s not really related to the original problem…

[catweasel]

I’ve just been looking at this as I want interaction in a render window, I can’t seem to get any mouse buttons working, is that a known thing, or me not doing it correctly?

[bjoern]

@catweasel
Cobbled together a little example:

GlobalHook.7z (24.7 KB)

[bjoern]

Since this is asked for quite often, I thought about turning it into a nuget. Not sure about the naming though,
VL.IO.Global?

[catweasel]

How to use it with a Stride RenderWindow rather than Skia? I would have thought that my patch above would have worked, as the positions are fine, but all mouse buttons seem to be nonfunctional. Definatly could do with a nuget

[bjoern]

@catweasel

How to use it with a Stride RenderWindow rather than Skia?

How To use Mouse (IO.Global) with Stride.vl (71.1 KB)

You have to reference GlobalHook.vl I posted above.

[catweasel]

Thanks!

[sebescudie]

@bjoern

VL.IO.Global?

Sounds a bit ugly but VL.GlobalMouseKeys to be more specific?

[offtopic]
Also, feels weird to me to have a category like “IO” in the package name, that should only be in the nodebrowser I’d say.
[/offtopic]

But appart from that yes, great initiative, this is needed quite often!

[bjoern]

@sebescudie

Also, feels weird to me to have a category like “IO” in the package name, that should only be in the nodebrowser I’d say.

Mh, there are a lot of “official” nugets carrying the category in their names also ones that are somewhat related like VL.IO.Multitouch and the (Skia) mouse stuff is in IO. Currently favoring VL.IO.MouseKeyGlobal.

[io]

Would it be possible to have the HID id as context ?

[bjoern]

Here you go:

VL.IO.MouseKeyGlobal

A global mouse and keyboard hook for vvvv gamma. In contrast to the GlobalHook stuff posted in the forum and the chat beforehand this package is not based on MouseKeyHook but on SharpHook by Tolik Pylypchuk. After some deliberation I chose Sharphook because: it doesn’t depend on Windows.Forms it is cross plattform the hook runs its event handlers on other threads and should behave nicer when vvvv runs at lower framerates for some reason For more info please check: https://sharphook.toli…