TextureFX 3.0 Preview

[joreg]

One of the big updates for vvvv gamma 2021.4 will be a solid collection of TextureFX as you were used to in vvvv beta. That is:

• Sources (noise generators, …)
• Filters (blur, edge, …)
• Mixers (blend, …)
• Utilities (convert colors, …)

Latest 2021.4 previews now have those ready for testing. Status:

• We still have a few open issues, but the main things are done
• Currently all nodes are still marked with the “Experimental” aspect. For the final release, many of them should have this aspect removed and be stable
• All help docs still need a review
• Documentation: Writing your own TextureFX

And this is where we’re hereby asking for your help: Please give the latest preview a spin, go through the texturefx nodes, which you’ll find in Stride -> Textures -> Source/Filter/Mixer and Utils. Then please post here if you have thoughts on the following:

Nodes:

• naming
• help text
• functionality
• any missing?

Pins:

• naming
• help text
• functionality
• order
• type/defaults
• any missing? too much?

Massive thanks to @bj-rn for porting many of the vvvv beta textureFX over!

[schlonzo]

“input 2” is imo a bit misleading for a control texture on the blur nodes. ​And a toggle for overriding them would be convenient. Since there is no dynamic texture available it’s cumbersome to load a white texture just to make the blur effects work.

When creating a node its parameters just seem to take effect after the second time you manipulate them.

[bjoern]

In HowTo convert sRGB to linear space and vice versa the texture path is wrong. But I think it might be better to use Palette as input here anyway. It also shows that LinearTosRGB is still not exactly inverse to sRGBToLinear.
HowTo Convert sRGB color to linear space and vice versa.vl (13.1 KB)

[tonfilm]

@bjoern

It also shows that LinearTosRGB is still not exactly inverse to sRGBToLinear .

i think this doesn’t work yet, see the linked issue, there are two open points for sRGB, the main thing here is that the format must reflect whether the data is sRGB or not. because the format of the texture influences how the sampler works, since sRGB conversion is built into the hardware nowadays.

EDIT:
checked the help patch again, and as the help text of the node states, there is no need to convert to Linear in that case because the loaded data is sRGB and the format already reflects that.

[Rayment]

Hi guys, I found a problem with TextureFX in exported project. I’ve described it here.
It is easy to reproduce, but for some reason you need a second pc to run the compiled project.
It will throw an exception like this one:

System.InvalidCastException: Unable to cast object of type 'System.Single' to type 'Stride.Graphics.Texture'.
   at VL.Core.NodeBuilding.PinInstance`1.VL.Core.IVLPin.set_Value(Object value)
   at _BasicComponents_.Mixer.BasicComponents.InvertComponent_C.Update_()
   at _Core_.Mixer.Core.TwoDScene_C.Update_()
   at _Core_.Mixer.Core.RenderSlot_HjibzFwtiGPPSR7O9tIS7Y.Update(CompositionSlot_C Input_In, Texture& Texture_Out)
   at _Core_.Mixer.Core.Composition_C.Render(Texture& Texture_Out)
   at __MIXER_tester_.Main._MIXER_testerApplication_Ba6sY4xaoelOwKB6LBWbFe.Update()
   at VL.App.AppHost.<RunCore>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at VL.App.WindowsForms.WindowsFormsAppHost.<Application_Idle>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

As far I could find it appears when you use FilterBase Seem to me it is something with pin creation order.
preview-64

[gegenlicht]

For Scaler - i think it would be easier to just enter the desired resolution instead of a multiplier “Rescale” here.

[tonfilm]

yes, good point, both should be there…

[lecloneur]

Hi,

Some testing on textureFX :

Blur (not working)
Dissolve (not working)
Pixelate (not working)
BlurGlow (not doing anything, gets full black when Shape parameters is above 10.40, brightness params is kind or working)
Anaglyph (working except that MapBlur parameter isn’t doing anything)
BackgroundSubstraction (not working)
BlurPerfector (ok)
BoxBlur (ok)
BrightFilter (ok but what is its purpose ? just curious)
Metallica (ok but everytime I change a color vvvv freeze, also I guess default value aren’t correct for a direct use of it)
BubbleNoise (ok)
CrossStitch (ok)
Dither (ok)
Glow (ok)
Edge (ok but full white texture for some time before seeing result)
ScanLine (ok)
HSCB (ok)
Mosaic (ok)
Muffy (ok)
Dots(ok)
ConvertColor(ok)
Lomograph(ok but VignetteDodge parameter is behaving strange)
Liquish (ok)
Tunnel(ok)
Blood (not doing anything)
Gradient (ok)
BlendMixer (ok but is it supposed to be greyish with fader reaching 0.5 ?)
SimpleBloom (ok but effect is a little strange, not blurred enough it seems, looks like boxblur)
Perlin (missing ?)
DirectionalBlur (ok but samplecenter should be of type Vector2 I guess)
Sharpen (not working)

Some other problems I noticed :

Renderer get stuck sometimes, happened with edge filter, while the output pin texture preview is showing effect correctly.

Everytime I modify a value on input pin, renderer get stuck for a moment. If I modify the same parameters I see changes smooth, if I modify a different parameter then rendering get stuck again.

Some default value are probably not appropriated in some case as the effect doesn’t show up correctly right away.

I tried those fx in between a sceneTexture and a quadRenderer.

Also I didn’t understand what the GPU vector4 input pin is suppose to be and how to use it.

[joreg]

@lecloneur thanks a lot for this detailed inspection!

regarding mixers (Dissolve, Pixelate, Blood) not working: did you confirm this with their helppatches? if they work in their helppatches but not in a custom patch you made, could you please share that patch?!

@lecloneur

Everytime I modify a value on input pin, renderer get stuck for a moment. If I modify the same parameters I see changes smooth, if I modify a different parameter then rendering get stuck again.

this is an unfortunate known issue. not related to only texturefx but many other nodes that are created similar (“fragmented”). essentially what’s happening is that every input of a texturefx node is on its own operation that is not executed unless you set a value at least once. thats why only from the second time on you change a value, it updates directly. we’re not happy with this either, let’s see…

@lecloneur

Also I didn’t understand what the GPU vector4 input pin is suppose to be and how to use it.

this is what used to be the “Control” input texture in beta texturefx, ie a grascale value per pixel to control the amount of the effect per pixel. only now you not only use a texture but also values to control this input. use the ValueIn/Map, ColorIn/Map nodes to connect to that input.

i’ll get back to you regarding the other issues as we have more…

[lecloneur]

ok thanks, I will have another test session then.

[joreg]

moments later… we made quite some progress with textureFX, now would be a good moment to give them another spin with latest 2021.4 preview.

@schlonzo

“input 2” is imo a bit misleading for a control texture on the blur nodes.

blur nodes have been updated. this is now fixed.

@Rayment

Hi guys, I found a problem with TextureFX in exported project.

this is now fixed.

@gegenlicht

For Scaler - i think it would be easier to just enter the desired resolution instead of a multiplier “Rescale” here.

There are now dedicated Crop and Resize nodes, in two versions: absolut and relative

@lecloneur

Some testing on textureFX

most of the issues you found where due to effects relying on mipmaps on their inputs, when the incoming texture didn’t have any (like the SceneTexture you tested with). we now added a WantsMips attribute to texture inputs which tells the system to create mipmaps for inputs that don’t have any.

@lecloneur

BackgroundSubstraction (not working)

seems to work now, see helppatch.

@lecloneur

Metallica (ok but everytime I change a color vvvv freeze, also I guess default value aren’t correct for a direct use of it)

seems to work now, see helppatch.

@lecloneur

Edge (ok but full white texture for some time before seeing result)

seems to work now, see helppatch.

@lecloneur

DirectionalBlur (ok but samplecenter should be of type Vector2 I guess)

done, thanks.

@lecloneur

Sharpen (not working)

seems to work now, see helppatch.

@lecloneur

Some default value are probably not appropriated in some case as the effect doesn’t show up correctly right away.

please report any such you find and suggest a better default.

[lecloneur]

Hi,

Normal is missing as NormalGlow isn’t the same.
Also Gradient doesn’t have the choice it had before :

And Echo is missing, but it is an extremely useful TFX. I know the Feedback help patch is suppose to do the same but not exactly. It would be great to have Echo back in the node browser.

Blend mode are a pain in the a… to work with, see post here :

Blend (Mixer) problems

Hi, thanks for the resources, I’ll have a look. The problem is I would say is the behavior I’m looking for and I believe people using graphic tools should be the same as GIMP, Photoshop, Blender etc… Also when using blend mode like this you need to switch between them quickly. Now if Multiply works with a Black background with Alpha 0 then other like Normal won’t. It is in my opinion better if the alpha works the same for all of them. If not then, people will be confuse that in VL things wo…

Invert use to have more options easily accessible from enum :

Can’t find Pillow :

For some reason (as the shade code looks identical) it is not possible to have a thin and contrasted Edge effect :

EDIT–

Here is Echo TFX :
Echo.vl (24.0 KB)

[lecloneur]

Hi, I’m updating the SetAlpha node to work correctly.

The idea is to set the alpha of an original image using a channel from another one.

For the channel there are : Red, Green, Blue, Alpha, Hue, Saturation, Value and Luma. Also the current SetAlpha node as an Average but is it really necessary or make sense to average RGBA and use that value ?

Also the Control(GPU) input doesn’t work. See attached :
SetAlpha.zip (5.4 KB)

---

Here is the Invert shader which now can invert :
RGB, LinearRGB those two sit together in GIMP, maybe they should be both in vvvv as well ?

Red, Green, Blue, Value, Luma (the classics)

and Hue (which is already in the current InvertShader but without proper naming).

Invert.zip (8.2 KB)

---

Here is Pixelate Filter with control over X and Y axis which can create some cool banding :

Pixelate.zip (3.4 KB)

---

Feedback Compute ported from this contribution :

FeedbackCS (DX11.Texture2d)

FeedbackCompute.zip (9.5 KB)

---

ASCII

ASCII.zip (9.2 KB)
Added PointSampler by default for the ASCII texture so it is sharp.
Quantized the grayscale so there isn’t any sliding effect.

ASCII.zip (5.8 KB)

---

Pulse ported from InstanceNoodle (added channel for RGB, HSV and Alpha as well as Invert :

Pulse.zip (4.0 KB)

---

Datamosh ported from : Datamosh (DX11) | vvvv
Added Color Discard as a GPU input so the feedback effect can go through another Texture via ShaderFX.
The threshold parameter is now a GPU for the same reason.

Datamosh.zip (4.8 KB)

---

TransformCMYK

Different approach than ShiftRGB, each channel here has its own Transform2D. More possibilities but need a little more time to setup. What do you think ?
ShiftRGB could work this way as well.

TransformCMYK.zip (243.7 KB)

---

ColorKeyOBS ported by @blacktronics

ColorKeyOBS.zip (6.7 KB)

---

Vibrance

port from mp.dx by Micro D double E aka @microdee. Original by Christian Cann Schuldt Jensen ~ CeeJay.dk
Vibrance intelligently boosts the saturation of pixels so pixels that had little color get a larger boost than pixels that had a lot. This avoids oversaturation of pixels that were already very saturated.

Vibrance.zip (3.3 KB)

---

Technicolor
Ported from mp.dx pack by @microdee

Technicolor.zip (3.8 KB)

---

Adaptive Sharpen

ported from bacondither

AdaptiveSharpen.zip (10.5 KB)

---

Lift Gamma Gain Exposure (same as LightRoom or DavinciResolve)

LiftGammaGainExposure.zip (5.0 KB)

---

ColorMatrix

have fun with color in 4 dimensions

ColorMatrix.zip (4.2 KB)

---

Shadow Midtone Highlight adjustement (basic port)

ShadowsMidtonesHighLights.zip (11.9 KB)

[tonfilm]

Hey @lecloneur, thanks for all these goodies. I’ve made an issue here to potentially include them in future releases:

New TextureFX nodes

[lecloneur]

Hi, glad you guys like it.

There is still something I’m wondering about the TFX so I will add those questions here.

Should every input be of type GPU < T > ? I think it would be great so any of them could be controlled by a ColorMap, ValueMap or a anything else coming from a ShaderFX.

Shouldn’t we port this using FUSE instead ?

I started to port ColorGrading shaders and I believe things could be much more flexible to use within the Gamma environment. Indeed as mention above potentially any inputs can be texture controlled which is a great improvement compare to the textureFX in Beta but also if their different passes where separated then we could do a lot of fun stuff :

GetShadows => Invert => Contrast => SetShadows
or
GetLuma => Contrast => Saturation = > Displace => Vibrance => Blur => SetLuma

Even with some texture feedback in the middle :)

VVVV Gamma would have something like DaVinciResolve for color grading but in realtime … and we could call it TextureFX 3.0.

[andresc4]

Thanks for this clone!!!

[tonfilm]

@lecloneur

Should every input be of type GPU < T > ? I think it would be great so any of them could be controlled by a ColorMap, ValueMap or a anything else coming from a ShaderFX.

In theory, this is possible but not really necessary for all of them. It is up to the developer to decide which ones make sense to get it from a texture or from another shader. this isn’t always easy to decide and we did fewer of them because we didn’t have the automatic conversion from normal value to GPU value when we created them. today, I would also create some more as a GPU value. it might also not always be performat to make it a GPU value, but again, this has to be decided on what the value actually controls in the shader. so just do it if you think a parameter would benefit from controlling it differently for each pixel.

@lecloneur

Shouldn’t we port this using FUSE instead ?

no, the reason FUSE had their own gpu value at the moment is to be able to develop quicker and more independently, but this year, everything will be converted to the “official” shader fx types.

and also today, you can convert between the two types. have a look at the ToShaderFX node in FUSE.

@lecloneur

I started to port ColorGrading shaders

be careful there with the color spaces, Stride uses the more professional linear color space by default. try to read some documentation on that and how to implement color grading in linear color space. examples on the internet often use gamma color space, which will result in “false” results when chained together.

here are some links: What every coder should know about gamma | John Novak or https://medium.com/@Jacob_Bell/programmers-guide-to-gamma-correction-4c1d3a1724fb or https://developer.nvidia.com/gpugems/gpugems3/part-iv-image-effects/chapter-24-importance-being-linear and many more…

the film industry also has their own ideas of color space, if you copy examples from there: Gamma, Gamut, Levels, and Best Practices for Approaching Color Management With DaVinci Resolve and DaVinci Resolve Studio | Fstoppers or The Essential Guide to Color Spaces - Frame.io Insider

the step of color correction is called tone mapping in stride, also look at these shaders, there are already quite some implemented. everyone should use these more to make their output pop on the screens.

looking forward to the new contributions. ideally as a pull request on VL.Stride, as explained in the readme: GitHub - vvvv/VL.Stride: ARCHIVED! VL library with tools and nodes to ease the patch workflow with Stride (fka Xenko).

[lecloneur]

Hi,

if you aren’t on the vvvv chat then here is the current state of TextureFX 3.0 which represents 11 years of community work.

And this is some screenshot during testing :

Uploading: image (20).png…

Still work in progress…

[redredred]

Looks great!

[andresc4]

This post need a donate button!
amazing work clone!