here is again a question on how properly patch c# async.
The code:
using (var csvFile = new FileStream("mfccs.csv", FileMode.Create))
{
var serializer = new CsvFeatureSerializer(mfccs, timeMarkers, header);
await serializer.SerializeAsync(csvFile);
}
Is it right to translate it to the code like that?
Will be FileStream disposed when an observable will be pushed?
Or I get it wrong? If I’m wrong, how can I tell to the Provider when I’m done and it can be disposed?
shouldn’t there be a bind or bindNew somehere?
is this patch running asynchronously? i guess it blocks mainloop…
@velcrome:
it’s System.Resources.New
a great learning resource for those is the implementation of kinect2 or Astra devices. though the usecase is different here, i guess. for the devices, the things get pooled, so that only one resource exists and every subscriber connects to the same thing. here, the holdlatest is quite immediately after the using region (why?) so there will always only be one subscriber.
instead of converting the task to an observable and trying to mimic the await functionality, i would just wait for it with a blocking method. for example Task.Result. then the Using region will know when the operation is finished and when it is time to release the resource.
if this blocks the mainloop, wrap an AsyncTask around it.
that was my first intent.
But there is no Task.Result for the Task (NonGeneric), right?
I must say it feels a bit awkward to replace 2 lines of code with so many regions and concepts.
Aren’t there a more elegant way as to make a blocking method out of the hot running task and then wrap it into AsyncTask?
it has two sides, async/await is c# syntax sugar that compiles to rather complex code. it creates some kind of state machine and passes data around (similar to observables). this is nice, because in C# it is quite hard to work with tasks manually.
in VL, on the other hand, AsyncTask and observables are nicely readable and there is no that much need for all the async/await stuff as in C#.
so it is a bit of a conflict between textual libraries using async/await and that VL doesn’t really need it that much. so the easiest way to work with it at the moment, is to make the async calls blocking and place AsyncTasks wherever you need it.
of course, we want to support that better in the language in future, because more and more libraries are only using this paradigm. it would involve things like creating anonymous/implicit operations in the patch whenever there is an await. and when the awaited task is finished, this anonymous operation gets called. then the question is, how do you assign nodes to this anonymous operation and so on…
cool. Thank you for your explanations!
Yes, I understand everything you say and I was aware of the C# compiler unwrapping the await into rather huge statemachine. So it was unfair from my side to say, that C# has only 2 lines.
But still, of course, it would be cool if that particular elegance of C#'s await could be matched by the beautiful patches of VL.
And yes, thank you for the Task.Wait and Task.WaitAll hints. Found it.
@robotanton
Back to the initial question, yes, it is correct to patch it like that and you can be sure that the FileStream will be disposed of. Managing the lifetime of resources (like a file stream) was the main reason we came up with the idea of a resource provider. Your example shows nicely howto tie the lifetime of such a resource to an observable with the Using (Provider) [Observable] region. I should however mention that you could’ve skipped the resource provider entirely in this example with the Using [Observable] region.
Regarding async/await in VL. Yes, please open a quest in the VL-Language repository. For example introducing Await as a node shouldn’t be too hard, we can indeed rely on the C# compiler doing the heavy lifting, but the implications of such a feature are quite vast.
@sebl no, it does not block. Even though the patches inside the New and Using regions are executed on the main thread (the moment the HoldLatest node subscribes), the resulting serialization task wrapped in an observable will complete at a later time. The serialization will probably (not necessarily, depends on the library) run on a different thread. Once it completes, the used file stream will be disposed of.
@tonfilm
I’d not recommend using the AsyncTask region and waiting on the result inside as a replacement:
It creates two tasks, one doing the actual work, the other just waiting on it and blocking the thread executing it.
It could be that the async method being called is thread affine (for example expects to be called on the UI thread having a message pump) - in that case the whole thing would not work at all and probably crash.
@velcrome
The Provider was indeed our invention, tackling the problem of managing the lifetime of a resource, especially when resources depend on each other and/or many consumers need it.
In the grander scheme of things like tasks and observables - well from a theoretical perspective they could all be seen as monads
Task<T> with (return = FromResult, bind = ContinueWith)
Observable<T> with (return = Return, bind = SelectMany)
Provider<T> with (return = Return, bind = Bind)
allowing you to chain computations. Equipped with some operators (like Using (Provider) or ToObservable [Task]) you can even combine them as seen in this example. Note however that not all directions necessarily make sense. For example I’d have no clue what an operator going from Task<T> to Provider<U> would accomplish or even how to implement it without blocking the calling thread.
thank you very much for the explanations and clarifying the matter!
I’m happy to hear, that blocking the thread just for waiting for the task is not the way to go.
The interplay of these and some particular regions
On a more theoretical level Monads came up, as these 3 types actually can be classified in that way
But also on another level new ways of expression got asked for
in particular, a way to express a certain kind of Coroutine:
the Coroutine that handles Tasks via the async/await pattern in C# and how that could look like in VL
Regarding that Coroutine issue and the particular async/await:
I will try to summarize the ideas we had in the past to this way of expression. @tonfilm and @Elias already addressed this already briefly. When done right we’d get yield return in the same go.
Give me a moment to write this down.
In another explanatory issue, I’d like to address different evaluation strategies that vvvv users had the pleasure to use secretly:
Call by need: vvvv beta. Only compute a node if its output data is needed by other nodes (or if AutoEvaluate is on). Aka lazy evaluation
Call by value: VL. aka strict evaluation. Always evaluate everything inside a patch. If you want to optimize that: use constructs like Cache to avoid unnecessary computations.
Call by future: Kick off a computation that at some point returns and hands over a result. Define how to react then.
Regarding futures, it is worth noting that
both Tasks and Observables have the ability to express asynchronous computations:
Task will return one value when done
Observables are able to push us several values in the future
we currently have better support for Observables which can express more
we currently work with Regions (like Foreach [Reactive]) that define the continuation patches (regions make my patch look deep)
we’d like to support Coroutines in order to flatten that out
