Cannot persist an output pin's value on an emitted node

[sebescudie]

hellobonjour,

I’m doing tests with node factories and ML.NET. The idea is that you can train an ML model with Visual Studio’s Model Builder and automagically turn it into a VL node with the appropriate inputs and outputs, kinda like VL.RunwayML does.

Visual Studio gives you this black-box zip file that’s been trained on your dataset, you drop it inside a folder next to your patch and you get a node that runs this model.

So far so good, I managed to get a node with the appropriate inputs and outputs and successfully run the model against some arbitrary input data. Here, you can see a model predicting taxi fare prices in New York City.

There’s one issue though : each time I change an input pin’s value or create a new node, or do any action that would trigger the GC (which shows up as this yellow marker in VS’ debugger), the output value of the node is reset to zero :

(For some reason, the orange/red bar under the quad menu does not show up when doing a gif recording…)

For the reference, here what’s happening inside the node’s Update() :

public void Update()
{
    if (!Inputs.Any())
        return;

    if((bool)Inputs.Last().Value)
    {
        // Create an input object that will hold our pin's data
        var inputObject = Activator.CreateInstance(inputType);

        // Stuff our input object with input pin's data
        foreach (var input in Inputs.Cast<MyPin>().SkipLast(1))
        {
            inputType.InvokeMember(input.Name, BindingFlags.SetProperty, null, inputObject, new object[] { input.Value });
        }

        // Invoke the predict method
        var predictMethod = predictionEngine.GetType().GetMethod("Predict", new[] { inputType });
        var result = predictMethod.Invoke(predictionEngine, new[] { inputObject });

        // Look for the "Score" output pin and assign it the value return by the prediction
        var outputPin = Outputs.Cast<MyPin>().FirstOrDefault(o => o.Name == "Score");
        outputPin.Value = outputType.InvokeMember("Score", BindingFlags.GetProperty, null, result, new object[] { });
    }
}

If you wanna have a look at the code, the source is there.

Any idea what’s going on here? I guess the GC is getting rid of inputObject, but how could I persist this?

Thanks!

[tonfilm]

you have to hold a reference as a field of the class that lives as long as the node, or represents the node, because as soon as there is no reference to an object anywhere, the GC will collect and delete it.

also your node doesn’t seem to have fields for the model or so: https://github.com/sebescudie/VL.ML/blob/seb/model-loader/src/VL.ML/ModelFactory.cs#L315

you are using:

    // Prediction engine
    dynamic predictionEngine;

does it change if you use object as type? i think the dynamic typing doesn’t do much here, since you don’t call methods that use a concrete type.

[sebescudie]

Thanks for the pointer @tonfilm, I tried adding a Score property to ModelDescription and set it from inside MyNode.Update(), unfortunately this also resets to its default value after the GC shows up.

I’ll have to take some time here to refine the overall structure of the node, I guess things are not where they’re supposed to be :)

[Elias]

Just to make sure, please set a breakpoint here - I fear this method might get called more often than you’d expect? If it is called multiple times you’ll need to introduce some sort of caching that you don’t end up creating new node (model in your case) descriptions all the time.

[sebescudie]

@Elias uh, indeed, this gets called over and over again. Thanks for the hint!

[sebescudie]

@Elias Applied the latest changes you made to VL.RunwayML, this works as expected now, thanks!

[Elias]

Good to hear!