I’m trying to create a Kinect Studio Tools plugin, however I’ve stubled upon a strange issue.
When I initialize the object in the class constructor(or anywhere else in the same thread), the MainLoop stops until the user interface is interacted.
public class ValueKinectStudioClientReproNode1 : IPluginEvaluate
{
private KStudioClient client;
public ValueKinectStudioClientReproNode1()
{
client = KStudio.CreateClient();
}
public void Evaluate(int SpreadMax)
{
}
}
There is a repro package with the simplified plugin and a repro patch.
1 - Unpack KinectStudioClient_MainLoop_repro.zip
2 - Open Repro.v4p with vvvv_45beta34.2_x64 or vvvv_50alpha34.106_x64
→ Manloop framecount and Renderer animation stops until you interact with a keyboard or a mouse
since vvvv has only one thread, the constructor is also called in that. but you can start a thread to construct the kinect client and work with it in the main loop once it is assigned.
bu i have seen other kinect studio nodes fly around… was it @motzi or @ravazquez?
I’ve tried to construct the client outside of the mainloop thread, but then properties of object, especially recording (Duration, FileSizeBytes, BufferInUseSizeMegabytes) do not update while the recording is in progress.
it was my colleague @timpernagel who tinkered with kinect-studio as a node back in the days. iirc he produced a node that worked in general but the playback was really unreliable (timing-wise) due to concepts of playing back kinect-recordings. at the end we simplay used kinect studio and the usual kinect nodes instead of a dedicated node that does the same job.
i just used the origina kinectstudio as a playback engine for recordings. no special nodes though.
OT: one thing i noticed was that the quality of the skeleton tracking from recorded xef files was worse than from the live stream. but maybe that’s just me seeing things…
@sebl@motzi : yes, at the moment i’m using Kinect Studio as well and i modified to receive osc commands /record, stop record, play, stop play, step once/. but it would be cool to have nodes for kinect studio tools. it’s either an internal bug of kinect studio tools or my lack of c# knowledge that prevents me to create them. :)
@motzi i guess the skeleton is recognized on the go, it is not recorded, perhaps the playback of the stream takes more cpu resources compared to reading stream from an actual device? from what i’ve seen so far, it definitely takes some for the skeleton tracking to catch up when there is a discontinuity in the depth stream (beginning of the playback, beginning of the loop)
@id144: indeed, the skeleton ist not recorded but recreated on every playback (which can be noticed when starting the playback of a file, because it needs a few seconds to recognize it). since i needed to record the playback-data in vvvv from the beginning i first thought i had a problem with my recordings, because i knew that the skeleton was definetely there when recording:
my trick to get skeleton tracking from the beginning of the clip was to set playback to loop, set a breakpoint at the veryfirst frame, playback the file right to the end until the loop makes it restart and pause at the beginning (with the skeleton already tracked from the playback before) and then start recording the skeleton data in VVVV. quite cumbersome, but it worked…
anyway, in future recordings i will record the skeleton (and image) data right away in VVVV (writing a csv-file for the skeleton data and DDS files for image data), mainly because my impression was that live tracking had better quality compared with the studio-playback.
From my experience the MainLoop freeze effect happens mostly when there is no enabled Kinect (2) node in the patch. If you add a Kinect node while a device is connected, enable the node, and leave it in the patch, you are even capable of disconnecting the device and keep on playing and changing .xef files.
@elias and myself spent a good couple of days looking into why this freeze happens and it seems to be related to the system wide event handling mechanism, and the way in which vvvv uses it, and the fact that KinectStudio seems to be tampering with it as well. It was low level Windows stuff that was over my head, perhaps @elias could shed some light.
As for the patch I provide here, it is an alpha version at most, so please excuse all the bugs and quirks it may come with. You need to have KinectStudio installed on the default location for the references to work. If the freeze starts to happen, reload the Kinect node and make sure it is enabled before trying again. If that does not work, close and try again.
If you feel like it, the VS code includes Pause functionality and some other stuff which seemed to always cause the freeze, so it was commented out for now.
@ravazquez Thanks for sharing!
I’ve managed to make few successful recordings but after a while, things went wrong and I was unable to make any further recordings until computer restart. I’ve also noticed the Mainloop issue popping at some point.
Second and maybe related issue is that the recording node is disposing the client before the recording buffer is empty and the resulting file has strange playback issues. But that is an easy fix I suppose.
if (recording.State == KStudioRecordingState.Recording)
{
recording.Stop();
FOutputBufferUsage[0] = 0;
status = "Idle";
client.DisconnectFromService();
client = null;
}
Here is my version of the plugin, the client object is created in a new thread and it waits till the buffer is empty, Kinect node does not need to be present. The recording object triggers the PropertyChanged event handler only when the recording starts and stops. As a result, there is no update on duration, buffer use etc. while the recording is in progress. I would call it pre-alpha. VVVV-MicrosoftKinectTools2.0.zip (214.1 KB)
I’ve published the Recording and Playback as a contribution. I hope this will be a vital tool in many scenarios.
Playback works well, including StepOnce, Pause, Seek etc. The implementation is rather naive, some more advance threading, locking and thread congestion avoiding techniques should be used. It seem the StepOnce shall not be called while previous operations are still in progress.
Recording still does not update the recording information - I do recover this information indirectly by checking available disk space and available memory. It is important to do that since even with a good SSD I’ve experienced situations when 16GB buffer is fully occupied after less than two minutes.
