Many of you might have experienced it. The more modern plugins you use, the more a framerate stutter slips into your patch. Since the the migration to c# started, more of that is occurring.
This happens because of the way memory is managed in .net and hence in c#. You can easily create objects of all kinds and never worry about removing them. This is done by quiet helper in the background: the garbage collector.
The existence of the gc is certainly a great feat of computer science and it helps development a tremendous amount. However, his workings are hard to control, and he is doing his dirty work whenever, and usually quite intense. This can really mess with the smooth real time experience we ask of vvvv.
So this thread is about exchanging ideas on what to do about it.
The first clues I found are:
as a plugin dev use structs instead of objects, because then the gc is circumvened (unless of course your structs have inner objects with a lot of change).
setting your MainLoop (VVVV) to Filter sometimes seems to make the problem even more visible.
But I also have questions:
Is there any best practise I can do as a plugin developer to make it more stable?
Is there any good source to learn more about c# realtime development regarding this issue?
What is the devvvvs stance on the topic, do you have any ideas how to contain the stutter?
And not least, how could a patcher go on finding the culprit plugin that feeds to the heap of the gc?
There’s no rule of thumb, since a lot will depend on your plugin types/what they need to do.
Structs and Class have different purposes, since one is a value type and the other is a reference type. so Structs are also allocated (and GCed). Classes are more suited for complex behaviours, Structs are better for small elements.
Reuse objects when possible. Recreating an object in a plugin every frame will obviously add more garbage collection work.
Structs are value types, so use ref parameter to modify them in place within a function
Example:
MyStruct DoSomething(MyStruct s) { //do something }
Parameter will be copied (so input is not modified, and so will be return parameter)
DoSomething(ref MyStruct s) { s.P = 1.0f; }
Will modify the struct directly, hence saving allocations.
Use classes instead of structs for custom IO. With current implementation of generic pin reference types cost less than value types (as side note, need to check how to optimize it for structs ;)
Use the buffer property from ISpread, to access array directly when possible. Iterate through an array is faster, and pushing data Directly to it will also save you some copies.
Be careful using/registering events. Always unregister events. Class that registers event should implement IDisposable and unregister them on Dispose(). This is one of the most common sources of memory leaks in .net, since such binding might prevent garbage collection.
If you start to have a lot of stutter and suspect it to be GC issue, use a memory profiler, there’s many around, and they can easily help you to spot memory leaks/unnecessary allocations.
That’s it for now, will likely find other ones around.
In my experience value types will not help you that much when it comes to reducing the amount of garbage collections. Yes a local value type variable will get allocated on the stack and will therefor not be subject to garbage collection as variables on the stack get cleaned up as soon as you leave the function. But you’ll usually deal with arrays/lists/spreads of value types, which do get allocated on the heap and will therefor be subject to garbage collection.
Whether to use a value type or not in such cases is a completely different question and will only make a difference in the performance characteristics of your program/plugin, because an array of reference types looks completely different in memory than an array of value types - but it will not reduce the amout of necessary garbage collections.
Like vux already pointed out keep the number of object allocations as low as possible. Especially when the object is a container type like a list, array or spread. You can achieve that by re-using objects, either with a local field or with some kind of object pool.
The problem with container types newly allocated each frame and probably the reason for most of the frame rate drops we experience in vvvv is, that an object exceeding a certain size will get allocated on the large object heap (LOH), which gets treated differently than the small object heap (SOH) by the garbage collector (GC).
The SOH gets compacted - memory gets moved around to keep the fragmentation of the heap as low as possible, which is basically important to avoid the number of CPU cache misses. Further, objects allocated on the SOH can get collected in a very fast Gen 0 and Gen 1 garbage collection.
The LOH does not get compacted - the cost of moving large amount of memory would be higher than one gains of a defragmented heap. And because the designers of the CLR decided that allocations of large objects should not occur that frequently objects on the LOH only get collected during a very expensive Gen 2 collection.
What all this means is that if a plugin allocates a large object each frame, it basically forces the GC doing the costly Gen 2 garbage collections.
I understand that writing code in such a fashion can be hard or nearly impossible in some cases. You’ll certainly find lot’s of code in our repository where we violated those guidelines ourselfs. Very often it is just too tempting to simply write a .ToArray or .ToList - and in many cases it won’t even matter much as the slice counts involved are not very high. But if they are and you start to experience frame rate drops look at those simply looking lines first - especially when you see them in a function called every frame.
@u7angel:
You’re right, frame rate drops caused by the garbage collector will certainly stay with us in the next generation of vvvv. Like now those lines of code doing costly allocations will need to get spotted and dealt with. But the new system will contain more auto-generated and less user-written code so my hope is that there’ll be fewer lines of code to go through when dealing with such an issue.
