TheEyeTribe Eyetracking VL plugin

vvvv beta question
,
1

heyhey!

As promised, here goes the EyeTribe VL plugin.

Unzip and start with the helppatches in /vl
/vl/Eyetracker (Devices TheEyeTribe) help.v4p
/vl/CalibrationRunner (TheEyeTribe) help.v4p

You will need the EyeTribe Server that comes with the SDK:

http://dev.theeyetribe.com
Device needs USB3.

As for the device itself, when the lighting is good it will give you quite accurate results. I found it really interesting to record how people look at pictures, especially portraits, so I have included a girlpower that does just that.

Tested in Win10 on my Laptop and a Windows Surface.
Please use latest Alpha.

The Eyetracker client was imported with the (not yet released) VL .net importer. Everything else is patched in VL, with minor exceptions that needed a C# wrapper to be used in VL. C# code included.

There are still some things to finetune, then I’ll make a contribution for you.
Everything else should be explained in the helppatches.
Any feedback really appreciated!

…yours faithfully,
vvvvpraxisintern dominikkoller

TheEyeTribe EyeTracking VL (1.5 MB)

2

Heyhey!

I’ll give you the latest version.

Also I’ve made the repo public:
https://github.com/DominikKoller/EyeTrackerTET-VL

of course, again: this is a call for feedback. Any feedback, +/-, on VL, the patching patterns I chose, or on the plugin specifically, is really appreciated.

vvvv >= 50alpha34.100

…yours faithfully,
vvvvpraxisintern dominikkoller

TheEyeTribe VL Alpha2 (1.6 MB)

3

really nice, though i can’t try it without the device.

can you please elaborate a bit more on that line?

4

and that one :)

5

Great stuff, thanks!

6

@sebl:

sure can.
with the imported EyeTribe library comes a mutable Class called Gazemanager(that is also a Singleton - get_Instance() will return its single instance).

a outputpin of a mutable datatype (like output pin of get_Instance(), returning GazeManager in this example) can have only one connection per output. Otherwise, when calling two operations, it would be unclear which of them should be called first:


as both Activate and CalibrationStart are called on the same instance.
(Should it be decided just visually, from top to down? Left - right? this is too vague.)
In the screenshot, you see
Activate(GazeManager gazemanager);
CalibrationStart(GazeManager gazemanager, …);
and we want to call them in this order. Note that Activate and CalibrationStart are from our C# Wrapper, taking _GazeManager_as an input. The nodes you see here do not have any ouput, so they would be defined by:

public static void Activate(GazeManager gazemanager)
        {
            gazemanager.Activate(GazeManager.ApiVersion.VERSION_1_0, GazeManager.ClientMode.Push);
        }

Note: as we’re dealing with a Singleton, we could avoid the red links by simply doing this:

https://vvvv.org/sites/default/files/imagecache/large/images/two__GetInstance.jpg

However, this is still ambiguous, same as explained above. Only difference is that VL will not show red links, as it is up to the external library what get_Instance returns.

What we need is a link from Activate to CalibrationStart, to define their order. We therefore need to output the Instance of GazeManager at Activate, and use this as the input for the CalibrationStart node, indicating that the GazeManager ‘goes through’ Activate first, and then ‘goes through’ CalibrationStart:

https://vvvv.org/sites/default/files/imagecache/large/images/one__GetInstance_withOutput.jpg

This is why we need the additional output, even though we are dealing with a Singleton.

Does this make things clear enough?

7

super, thanks.

may this outline a possible general way to handle mutable data in VL? (because i still have the feeling that this “mutable/only one Link” way is not the smoothest solution.)

8

Note that our whole C# wrapper is a workaround for things that are not yet available in VL. When a function in the library we imported takes as parameters types that are not available in VL, the function will not be available as a node. Example:

GazeManager.Activate(GazeManager.ApiVersion apiVersion, GazeManager.ClientMode mode);

uses the Nested Enum GazeManager.ApiVersion and GazeManager.ClientMode. Nested Enums are not yet available in VL, therefore GazeManager.Activate will not be imported into VL. We therefore need to write a C# wrapper that does not take these types as a parameter, as done in

https://github.com/DominikKoller/EyeTrackerTET-VL/blob/master/c%23/TETWrapper/TETWrapper/Helper.cs#L16

9

@ sebl
Well, not really…

There is a node called ShareInstance (KillTheSystemBehaviour) that lets you ‘duplicate’ a reference, if you are really sure the order of the two executions does not matter and especially that what you’re doing is thread-safe (which it will be most of the time, sure).

From what I’ve patched so far though, the strict one-connection-rule does not seem to be such a big limitation. After all, when your data is mutable, the order of execution does matter most of the time, and it’s nice to see your data flow from top to bottom.

Anyhow: Any suggestions / usecases / patching experiences are very much appreciated!

10

@sebl

to your second question:
that one’s a bit more tricky to explain, but also doesn’t go as deep into language features of VL. I’ll try to keep it brief.

This is about how to construct an Observable. We basically need two delegates, one that describes what will happen OnSubscribe and one that describes what will happen OnUnsubscribe. We also need to share Data between them. (In our case, this would be the GazeListener).

For anyone not familiar with Observables: we have two Delegates, describing an action that can be executed at any time. We need to share mutable Data between them. The problem is, again, the can be executed at any time.

Therefore, links of mutable data into delegates are not allowed in VL (right now):

What we would have needed, though, is a link from the OnSubscribe region to OnUnsubscribe. Would look like this:
https://vvvv.org/sites/default/files/imagecache/large/images/RegionLink.JPG

The second region OnUnsubscribe is in the first one OnSubscribe because it is defined when the first one is executed. This makes it possible for links to go from the first to the second - but not links of mutable data. That’s why we needed the help of that CreateWorkaround, which uses a Tuple to make the Data available in OnUnsubscribe…

this workaround is a quite bulky solution, though. I will update soon with a more elegant version.

11

Alright, here’s the latest version.

-CreateWorkaround is now called CreateObservable
it now takes two delegates as input parameters, OnSubscribe and OnUnsubscribe, while making it possible to route data from the the former to the latter.

TheEyeTribe VL Alpha3 (1.6 MB)

12

thanks dominik!

yes, the limitation to a single link is more a convention as a barrier. the ‘problem’ remains - simply spoken - in the form of sink-nodes, like Activate and Calibrationstart before their wrapper existed.

so i ask myself, what would be the most convenient way to handle such topics- leaving vvvv and writing a helper by hand isn’t, imho.
but, as the wrapping code you wrote looks rather easy (~ boilerplate), i could imagine a generic way to edit or even generate it. interface-like, it would be perfect, if an imported node (like the original Activate ) could be kind of opened and edited via patching.

basically i’m just interested in a roadmap or similar :)

13

Ah, in our case that wouldn’t have been a Problem! For class methods, VL routes the instance through the node, like so:

It’s quite neatly visualized with the connection of input and output pin.

We just needed the wrapper because of the unsupported Nested Enum in the functions parameters. Our functions though are not a methods of the class, therefore we need to make the outputs of the instance ourselves.

However, you’re right - any function that takes mutable data as an input needs to give it back to you as an output, otherwise you’ve lost that link.

I agree it would be nice to have a visual editor for nodes at import.

14

heyhey

made some design changes.

Now the Datatype available in vvvv is an Observable. We’re looking for a general design pattern for device integration in VL, and this gives the user maximum freedom to handle the data coming from a device.

Also did some cleanups in the patches, renamings and better categorizing.

cheers!

TheEyeTribe VL Alpha4 (1.6 MB)

15

note that this topic is obsolete. the node is now shipping already with alpha builds and will be included in beta35