Need some HINTS for research - Stride - SystemOutOfMemoryException

[bjoern]

@Elias

Did I dream this or did you post a VL.Stride patch the other day which was similar to the old Player node?

I tried to patch a stackplayer for node, but all my attempts were somehow flawed, either performance was bad or there were memory leaks. In the end it wasn’t really needed and I stopped my pursuit. Maybe I’ll give it another go.
The “player” I posted in the chat was just loading everything at once, playing it back from v-ram, basically filetexture + getslice. During my (very limited) testing with this approach I didn’t encounter any memory leaks though.

@CeeYaa
I don’t think TextureArray is what you are looking for - it just copies all files/textures into one texture. The textures all have to have the same size, format and mipmap count. Also memory consumption should be about the same as loading all textures separately and keeping them in a spread. Additionally something like GetSlice (DX11.TextureArray) to conveniently access the different slices in a TextureArray is still missing in stride. I guess one could do something like this in a TextureFX (haven’t tried):

shader GetTexture_TextureFX : TextureFX
{
    int Index = 0;

    stage override float4 Shading()
    {
        float2 uv = streams.TexCoord;
        return Texture0.Sample(LinearSampler, float3(uv, (float)Index));
    }
}

[Elias]

Indeed the texture loading function in Stride causes a lot of memory allocations. I’ve added a workaround/overload to VL.Stride which I used in a patch I’ll upload here as soon as it is available. Had good results, was able to playback a bmp image stack with 60 fps and memory was completely stable.

[Elias]

Here we go, inside you find a little demo patch showing howto load (and unload) a set of textures. I was testing it with 1280p *.bmp files I generated with ffmpeg before. It should also allow to do what you need, having 25 of out 3000 loaded at a time. You’ll need at least version 2021.3.0-21 for it to run.
TextureLoader.vl (57.2 KB)

[bjoern]

@Elias this is pretty impressive and should definitely go into core lib / VL.Stride.
I tried with 4096x2160 dds/bc7 files (~8 MB on disk) and reached stable 120 FPS on a 3 year old really small form factor desktop pc.
Specs:
Intel i7-8700 @ 3,20 GHz
NVidia Geforce GTX 1070-ti / 8GB VRam
Samsung 960 pro NVME SSD (1TB)

Once I went to higher FPS and a bottleneck was reached (did not figure out what exactly), the textures didn’t get properly disposed anymore, VRam was filling up and eventually vvvv crashed.

[tonfilm]

can we merge that back and improve the stride texture loading itself? which seems to be the reason for this post. having our own solution for a fundamental functionality of the main library doesn’t seem to be the best overall solution to me. also, it would benefit the stride community as well.

[Elias]

@bjoern Thanks! Regarding the crash when reaching too high with the FPS - the culprit could be the quickly hacked together AsyncTexture node in there. Could be that the texture never arrives on the main thread and therefor won’t get disposed. Good to know that a potential future node provided by VL.Stride should need to deal with that case.

@tebjan Indeed, we should provide a fix for Stride itself. I first tried adding an overload there taking ReadOnlyMemory<byte> so the memory needed here (https://github.com/stride3d/stride/blob/9219cbbb9de7a238060d97cf1cfbbf759120aa23/sources/engine/Stride/Graphics/Image.cs#L541) could be allocated on a pool (using ArrayPool from System.Memory). However that reference caused the C++ projects in the Stride solution to complain and I had no idea what to do. So I went a different route and added an overload to our project taking a string instead of a Stream and using the unmanaged heap to allocate the memory for the image. That solution should be easy to backport as it doesn’t need System.Memory.

[CeeYaa]

@elias WOW again amazing stuff THANK YOU - I had the same issue like bjoern said: when it’s to fast or when there are to many (the white flash of unloaded textures is a good marker) memory goes up again, but I can manage this by delay loading of different stacks - so I can use this in my case so far, after some hours it went up just a little bit but far away from before the “AsyncTexture”

[bjoern]

@Elias made a comparison with woei’s player. It can go to up to 180 FPS without hiccups.
Maybe take a look at the way he does the texture loading / creation?

GitHub - woeishi/VVVV.DX11.Player: VVVV node for high performance playback of...

[readme]

Thank you for this thorough benchmarking, Björn!

[Elias]

The major difference I can spot is that the node from @woei uses D3DX11CreateTextureFromFile function (D3DX11.h) - Win32 apps | Microsoft Docs to do the actual dds load, while in Stride this is done by these lines https://github.com/stride3d/stride/blob/9219cbbb9de7a238060d97cf1cfbbf759120aa23/sources/engine/Stride/Graphics/DDSHelper.cs#L988

As you can see the original function has been deprecated by Microsoft and is also no longer available in the SharpDX bindings (the DX11 pack uses the SlimDX bindings where this function still exists). So doing a quick comparison by moving over to that function doesn’t seem to be straight forward. We’d need to pipe the COM pointers from Stride → SharpDX → SlimDX, call the function and then all the way back.

In any case, I think I tackled the disposal topic mentioned earlier. So memory should be stable even if going over the hardware limits. Here’s the updated patch:
TextureLoader.vl (71.2 KB)

[bjoern]

@Elias

We’d need to pipe the COM pointers from Stride → SharpDX → SlimDX, call the function and then all the way back.

Like so?

It seems to work, isn’t leaking, but looks a bit hacky. Couldn’t compare performance yet because I currently haven’t got access to the desktop and my laptop isn’t up to the task.
Tried to use the VVVV.SlimDX nuget but had to reference the dll manually because gamma didn’t pick it up.

TextureLoader_SlimDX.vl (156.7 KB)

[Elias]

Hehe yes, like so. Regarding sRGB https://www.gamedev.net/forums/topic/666668-shaderresourceview-srgb-format-having-no-effect-on-sampler-reads/ - there seem to be known issues with that function. Maybe a reason why it was deprecated? But yeah, let’s wait until you can do some tests.

[bjoern]

TextureLoader_SlimDX.vl (130.7 KB)

[bjoern]

So some more testing.
I tried to make it as comparable as possible. I started the patches let them sit for a minute. Started the playback let it run for 1:30 min an then made the screenshots. The patches without the “benchmarking” seem to run a bit smoother.

First of all, the SlimDX version I added performs abysmal, it reaches max 12 FPS :) Comparing the CPU load to the other approaches I assume some of the “copying” back and forth happens on the CPU. Also there is some error at startup.

Beta using woei’s player again 180 FPS but this time I had some dropouts here and there.

Gamma using @Elias’ improved version I also got to 180FPS with some dropouts. After I made the screenshots I realized that I had set GraphicsResourceUsage to Default but trying with Immutable yielded the same results. So this seems the way to go.

Here again the patches with the settings I used in case someone else wants to give it a go:
Player.DX11Texture.7z (15.4 KB)
TextureLoader.vl (137.0 KB)

[CeeYaa]

This is all amazing cool stuff but far away from this Thread - is it ok when I CHANGE THIS THREAD to

WIP “VL.Stride.TextureLoader”

or does you want to open a new FORUM ENTRY IN WIP?

I mean, not that this nice development got lost or forgotten or visible enough just because it’s wrong labeled

[tonfilm]

it’s maybe better to make a new, dedicated thread… @Elias @bjoern do you want to create one?

[bjoern]

@tonfilm

it’s maybe better to make a new, dedicated thread

I made a WIP post:

VL.Stride.TexturePlayer formerly known as TextureLoader

Hey, I took the texture loader @Elias posted here and wrapped it into a player for image/texture stacks. Put the parts of the player into several documents, not quite sure about the structure though. For more background info and some performance tests please refer to the initial thread. VL.Stride.TexturePlayer_V0.1.7z (23.4 KB)

[schlonzo]

hey guys I have problems implementing @elias “async texture”. which @bjoern uses in his textureplayer.

I add the file dependencies “corelib.extensions” and “stride.extensions”, net nugets “sharp dx” and “direct3d11” after creating the async texture node things like the DIR node stop working, stride closes its scene window and won’t open it again.

callmenames-2021-04-16_02.vl (11.4 KB)

after removing the “stride.extensions” it starts working again.
thing is, bjoerns textureplayer works out of the box, so what’s the difference?
What does this “manage processes” node do? I had the async texture working in 2021.3.1 without it perfectly fine.

[joreg]

@schlonzo not exactly sure where you went wrong, but i think the following comes even closer to what you’re looking for:

A node that loads (async) and keeps in VRAM textures that you hand it as filepaths. ie. the input of this node is a spread of filepaths that you may add to and remove at any time. the output is a corresponding spread of textures. all textures no longer used are removed from VRAM.
TextureSpread.zip (18.5 KB)
(posted this because it is using the exact same innards as discussed above. same as with the TexturePlayer we’re planning to add this to VL.Stride in a coming realease)