I have been patching a couple of utility nodes for detecting key presses, independently from Skia/Stride windows.
Based on @sebescudie’s recommendation, I have been using MouseKeyHook.
It was looking fine so far, but when I started implementing the detection of multiple key combinations in parallel, I’ve been hitting some inconsistencies (like the up/downs of a specific key would stay frozen in a state, until I stop/resume).
Maybe I’m missing some subtleties playing with events:
is there something wrong having multiple instances of GlobalEvents existing inside several instances of my abstractions?
for easier readability when I patch key-combinations, I usually have several abstractions monitoring the same key multiple times in parallel (first capture below). Is there something wrong with that? I found to hit less of these problems when I’m monitoring a specific key only once and sharing the state across my patch (second capture)
Also for some reasons I hit less problems when testing out these key combination detections like captured but when nothing is connected downstream; still I haven’t manage to identify what is specifically making things worse down the line.
I’m actually using all these key combinations to control the window assignation of several Stride renderers, as well as the windowed/fullscreen toggles.
For my tests, I play around with 4 windows like in a 4 output setup but only have 2 screens here, which means that sometimes when I go fullscreen, several renderers “fight” for the fullscreen (like renderers would dock together in Beta); and when I toggle Fullscreen off, the loser renderer has vanished for ever.
It seems that when I bring this situation up, that kind of messes up my keyboard events freezing as described above.
Until I fool around with this superimposed-fullscreen windows thing, I cannot seem to reproduce the keyevents problem otherwise now.
Still noodling to see if this really narrows down the problem to this.
Would be interested in knowing if this could be a relevant hypothesis!
Thx :)
Hi, most of the stuff that hooks to input devices uses pinvoke and some dlls from the windows/system32 if that’s the case it’s really bad idea to call for input in different places, since if it blocks the resource you can have it blocked for other parts of graph… This stuff is really doggie and usually provided buy hosting framework…Since frameworks mostly really on some input implementation in their own, making input framework independent may lead to some issues
So what’s your recommendation then? Rather rely on Gamma’s own implementation (but it’s only tied to the context of a Stride/Skia window for now if I’m correct)
And until then only have one single observer for MouseKeyHook and distribute the key presses to the whole patch via some S/R for instance?
Then I wonder, how could we tackle the situation where multiple nugets making use of MouseKeyHook are running at the same time? Shall we wait for gamma to have native Global nodes that we’d then use?
In their example code they call Hook.GlobalEvents only once and they also clean the hook up with a call to Dispose. Just played shortly with an example as posted by @bjoernhere and after some F8/F5 the mouse started to stutter and the whole program crashed after a few seconds later. So cleaning that hook up seems to be mandatory. Installing it only once is probably also a good idea.
Attached you’ll find @bjoern’s patch with a proper lifetime management of that global hook by making use of two nodes in the Resources category. They will take care that only one hook exists per application and that hook also gets uninstalled when the application shuts down. The NewPooled (PerApp NoKey) region sets up a resource provider per application - when asked for its resource (the hook in this case) by multiple sinks it will hand out the same one to all of them. The Using (Process) node fetches and stores a handle to that resource - it will release that handle when the process node itself dies (for example through F8 or by deleting the node), and once all those sinks released their handles, the original resource which was handed by the resource provider will be disposed of - the hook gets uninstalled.
Here is a screenshot of that logic wrapped in an easy to use process called GlobalEvents
Thanks for the extensive tests and reply!
Before your answer I had gone for the safe route and made a unique instance of MouseKeyHook’s GlobalEvents that forwards all the keys in send/receives across my patch.
I’ll definitely give a go to your suggestion when I have a minute!
In parallel, do you guys have plans to implement a keyboard handler that is independant from Skia/Stride and thus a little more universal?
Well yes, it would be very beneficial to have a unified input event handling across the different rendering APIs. We did a little research on that topic in the beginning of this year. Thinking was to use a cross platform windowing API with which we create the render windows and then leave it up to Skia or Stride to draw into those. So far we looked into Stride itself, as well as Silk .NET. If I remember correctly the APIs provided by Stride weren’t flexible enough (it’s also not meant for that) and Silk had bugs with multiple windows. In that time Microsoft also announced MAUI to be released end of the year, so we wanted to wait that one out as well. With 2021.4 finally at our doorsteps we will re-iterate over this topic in the coming weeks and hopefully get some clarity in our roadmap about it.
here is a global key hook that can be used in Stride renderers, replace the Notifications output of the keyboard device with the output of the GlobalKeybard node in this patch:
Don’t know if this is the right place. But here is a hotfix for using Textfields with Stride’s SkiaRenderer based on MouseKeyHook: TextFieldHotFix.vl (62.9 KB)
Hi! I’m trying to patch so that keyboard presses are presented as keycodes (eg. A = 65, B = 66) including when VL is not in focus. Here, key presses would toggle the boolean for that position in the array (eg. A pressed = 65 on, B not pressed = 66 off).
Ideally, this would also have per keyboard selection, so eg. I could plug in an external or bluetooth keyboard, and use that as a controller, while keeping the laptop key presses separate. This also wouldn’t receive key repeats, so the boolean would only change when the physical key is pressed and unpressed.
The next thing is to have each of those key presses send a midi note, to trigger in other programs.
I can’t figure out the receiving a key code part, let alone making an array from the KeyEventArgs which return a datatype I don’t know how to do much with.
I could do this in vvvv, but can’t in VL. Would be super useful to be figure this out, any assistance much appreciated.
I’ve just been looking at this as I want interaction in a render window, I can’t seem to get any mouse buttons working, is that a known thing, or me not doing it correctly?
How to use it with a Stride RenderWindow rather than Skia? I would have thought that my patch above would have worked, as the positions are fine, but all mouse buttons seem to be nonfunctional. Definatly could do with a nuget
