Pure image treatment question: wich evolution for vvvv?

vvvv beta question
1

hi everybody, and hi dear devvvvs.

i m actually testing touchdesigner, just playing around. and i m completey stuck / stumbed / astonished by the easyness of having mix in all senses with its filters.

i dont like at all the interface but i think really that for visual treatment or effects on videos or sources, woaw… that is really powerfull, and very quick to patch, without fighting with ressources.

BUT… i m in love with VVVV, and its inner, quick and elegant wild freedom.

I would like to know if you, or users, could propose a newer way for live images treatment than the fx ( hard coding), or freeframes , renderer based structure.

something like basic elements ( operators add, diff, blur, mathematic, anything) that could be added and daisy chained to create at the end only one filter treatment to one renderer:

( dummy sample:)

videoIN > videotexture > ADD > MIN > EXPR > CONTRAST > SATURATION > MIN > BLUR > quad > renderer

could that be a sort of code agregator, pipeling elements ? or should that be a hidden renderer new structure in VVVV ?

is that relaying on the dynamic plug in missing structure ( i mean texture IN - texture OUT) ?

voilà, yours
christoph

2

yea karistouf check HSL Transform>RGB Transform…

3

hi anthokio, thanks for this quote…
didn t know, it, this really nice…

here i m just affecting the quad where am im putting the image. like if i was putting a color… hum…

i m talikng about compositing coding

4

you know the rules of speed is quite annoying
and touchdesigner quite suck on that one as i know…
eventually in dx you can’t do a shit without shaders and it’s going further and further in that direction…

5

well what i have tested in touch designer is far speed and less consuming resources than shaders in vvvv…
and mroe versatile

6

Even in the visual programming environment, certain things are just better implemented with code. I highly recommend learning some basic c# and shader c0ding if you plan on creating complex patches.

7

I mok with you gilbi. Done also some painfull learning with shaders. But when i m looking at tdesigner easyness to compose effects on a simple texture, i m saying here there is something missing in vvvv

We are all happy with vvvv but dont forget despite the dynamic plug in way that vvvv is not for programmers only ! ;-)

8

helo karistouf,

not sure if i understand your needs correctly so forgive me if i am stating something obvious, but please see attached for a simple/extensible proof of concept to realize a texture manipulation/fx chain.

think of each of the fx-modules as a freeframe/gl plugin (just not coded in cpp but patched in vvvv) that you can chain together and mix arbitrarily. see? all you have to do is work on a set of such modules which you can then easily reuse at will. those modules can include .fx or plugins or can be completely patched.

all you’d need to do is specify a standard for such fx-modules (specify inputs and outputs), call it freeframeV4 and everyone can contribute/share their fx.

now it seems to me a few people out there are already building their setups on such private spec:

so please makers of the above step forward, explain your personal spex, discuss and find common ground on a freeframeV4 specification. maybe there are even parts in the freeframe1.5 spec that can be ripped…

like this everyone could still have their own GUIs (which of course makes sense) but fx could be shared, horray. or karistouf says this is not what he was asking for in which case everyone please keep reinventing the wheel.

fxchain.zip (13.3 kB)

9

hi joreg, thanks to have taken the time of this long answer and demonstration . ;-)
once again learned a lot with the constant one.

svvvitcher, etc … : i m not at all talking about GUI, but about pure compositing approach and facilities.

What i want to say, and maybe its not really clear, is that after my last project ( http://vimeo.com/21553814 ) i would like really to put more energy in artistical field rather than trying to write poor not really good shaders . I m doing with what i can do, and only contributions are helping nicely where my skills are not there.

SO im talking about something that would help poor boys that we are, and avoid to make them write or transform shader when they dont know really what they are doing ;-) yes i know, make the effort to learn they say ! but not anybody is good in this field.

it would be really great to have directly into VVVV a more advanced texture effect base.

something where we could connect directly, without building subpatches and renderers, the texture to, and outputing the texture as a result, in a fluent manner.

I mean something like (Texture) family , that would let have access on a calling a simple node to the following things:

-mixing operators like:
difference, burn, add, xor, divide, etc… mul, div, all operators etc
-texture modifiers:
saturation, thresholds, color reduction, set alpha channel, chroma key ( yes we need), contrast, edge ( differents edges type), emboss, noise, bumpers, blurs with several algorythm, solarization, all effects basis…
-opencv or others functions but not in freeframe format (*)

I know that globally we could say that all of this is somehow present in contributions.
But really it would be interresting to have high quality treatment built-in and written by shaders specialist.

Basics are present with contributions, but if i just open my gimp, for edges i have 5 or 6 different algorythms and more parameters.
About Blur, i have spoken about TDesigner, but if i refer to Gimp again, there are many way to blur, and many parameters, enabling to obtain different results.

What i mean is that actually shaders are a bit auster and something more versatil and rich would be great. Yes i know this is contributions, but really it would be great to be less dependant of contributions when you don t have shaders skills !

  • AND at least, a real getpixelvalue/setpixelvalue function would be nice… pipet is great, but creating quads to transform pixels by data manipulation is really ressources consumering.

So to make short: i would like to have little nodes, transforming the texture with all basic operators you can find in a shader, but without having to create shaders.
i would like to have acess to high quality image filtering, built in VVVV.

Voilà, joreg, i hope i was clear and not wacky ! ;-)
what do you think ( and others) about it ?

(*): Freeframes are less quick than pixelshaders, and a real difference of speed is present when you work on a complex patch. I m not sure freeframe is a good solution, GPU based should be the standard ( no conflict with PS ).

10

I’m not really sure what you’re saying karistouf. I think most/everything you want is in vvvv already. It’s just finding it.

for mixing operators try this >blend-those-pixels

I pulled this from the old legacy list of shaders, you seen this page? there may be some other shaders that help you do what you want to do.

you reference wanting different edges type. You’re only going to get different edge types using shaders. If you want this in vvvv you or someone is going to have to program it. I think you’re basically asking the devs to spend some time on developing some more shaders. I’d love to see this too but I’m not sure this is how they do things. If you’re after a certain effect posting on the forum normally starts getting you places and you might get a dev that will put some time in and sort it for you. Maybe start a thread and try and pull some people to help develop some of the effects you’re talking about.

I understand you want all these effects in a node but I’m not sure how that is any different than packaging up a shader FX in a module, and then every time you want to use that effect just drop that module in place, connect a line and you have chained that effect in. The only way you’re going to get a fast effect is FX. There will be some more effects that are possible when they sort texture inputs for your own nodes. Is that on the cards soon devs?

you have this when you make modules. You’d only need to build the module/subpatch once and that’s it, use in loads of projects. Module with a texture in and a texture out pin. easy.

11

Hi xd nice to read your reply.

Thats the point. I would like to have a strong effect pack with many pins and quality at the end, that enables to construct easely new effects. Despite the beauty of contributions , i m pointing lack of evoluated parameters. When im doing editing or visuals with Gimp vegas premiere i have basics sets of filters in those softwares that give me very good results. This i dont have enough evoluated in vvvv and in contribs.

About nodes or modules, its more aquestion of ergonomy. I agree with you about module approach and i think that what i m trying to point could be understood by doing the get started tutorial of touch designer. Its easy, versatile , and dont need to bang the head of structures and subpatches to just filter in an evoluated way . I m talking about finding a way to have pleasure and easyness like while patching a spread: something easy, quick, powerfull.

I m fondamentally talking about pleasure while patching.

If im also talking about nodes on its own, its perhaps because i feeled very limited in my last work by the graphic card hability. I was stuck to 17 effects with 6 to 12running together. Maybe another way could be achieved to gain in performances than ex9 renderers in modules ? I mean if i want to compose a visual effect made of 12 or 16 components ?

I m doing things for dance and theatre and until now subpatches may not be loaded during performance : structure of a show is in a continuity and effects are overlapping continuosly

Voila!!!

12

Btw karistouf, FreeframeGL (aka Freeframe 1.5) is GPU-based, not CPU like the 1.0 version! I don’t see why this should be much slower than shaders.

13

I don’t think vvvv supports freeframeGl effects, am I right?

And I totally understand what Karistouf is saying.

You have something that outputs a texture (for example a video file), and you want to simply add 1 single node to the chain in order to process this whole texture with some effect, again outputting a texture. If you want another effect on top of that, you simply add this effect to the chain, and vvvv would take care of everything that needs to be done.

That would be the intuitive way of doing it. Texture in => Modified Texture Out

In the end, non-programmers aren’t interested if it’s a texture, if it’s tied to direct3d or opengl, if it happens on GPU or not (as long as it works and it’s fast). It’s an image that should be altered in some way.

You should look at this post from a user perspective.

Karistouf doesn’t say ‘I can’t do what I want’, he says ‘it takes too much patching and adding too much nodes (or alternatively too much messing with shader code) for something that should be intuitive’.

14

sounds a bit like shaderfx to me. only without the 3ds max part…

15

i am with karistouf on this one as well.

having a family of texture-nodes, thats works with textures in a fast and safe way on a general pixellevel with plugins (and hence cpu or opencl) would make vvvv more accessible.

there are a lot of good examples of what is possible, from aviary peacock to eyesweb. I understand your reluctance to add this feature as it is bound to fk up the cpu, though.

However, I’d love to see the old freeframe and maybe even dshow leave vvvv in the future and be replaced by a more flexible texture-based approach. if freeframeGL is a candidate for that, even better.

*v

16

me neither talking about GUIs. i only mentioned those because i think they already do (under the hood) what you also want to do.

so, i feel a bit missunderstood here.
has anybody looked at my demo? (ya xd_nitro seems to be with me).

&

i’d argue that this is exactly what i was demoing there. only the fx-nodes are not native nodes or plugins but modules, right? which i’d see as an advantage rather than a pain. imagine we all agree on a specification to build such modules someone who is not interested in their inner workings could just use them, others could build their own easily and still be compatible.

i was speaking of freeframe only as a standard, not that vvvv should implement freeframeGL but that you (those interested in texture-fx) could easily manage to create such a standard for texture manipulation modules.

so again i’d argue it is actually all there only someone would need to take the first step an prepare a basic package of texture-fx modules and propose a specification for such patches. if i understand your needs right, this has nothing to do with freeframe, openCL, openCV, dshow, CPU vs. GPU, the missing texture input pin for plugins or any other excuses to not like the module-based approach.

or am i still missing a point here?

17

Totally agree with karistouf. I’ve been played with TD for some time and I must say it’s more relaxing for the people without highlevel informatic BG.

18

hi everybody!
Just noticed this forumpost,
I think this is exactly what I was missing in vvvv - some kind of texture generation/manipulation/filtering stuff (got tired of coding shaders every time:) )
so, lets come up with a “specification” for texturing modules, and I’m ready to code loads of filters!
I mean, what we need here? Basically texture in+out, yes, and effect-specific parameters on input too
post your wish-lists for texturefilters!

P.S: is there a way to get textureformat from texture? These texturemodules probably need to output texture with the same resolution and type as input texture… I know that “info” node gives texture width and height, but no type

19

Hi joreg hi all.
About modules: why not. The daisy chain demonstration would be sufficient. Sufficient but perhaps not the solution .

i m affraid of two things on this approach:

  • shaders will stay dependant from community skills, where we need basics and powerfull ( not things just doing the job) effects and a taste, an artist of code that will give them a constant quality and taste. I mean that means that someone rush inside of it. I mean someone who is deeply in charge of in a constant effort of quality.
    -i have the maybe wrong idea that having those textures manipulation tools in the core of vvvv will be be less ressources consuming and avoid bugs and strange consumptions. This thinking also about subpatches and modules that are slowering the main patch
    -i m sure that you will find way to code powerfull texture manipulators in the core, as brillant as stallone concept, giving to vvvv another specific power

And about ergonomy i think there are some here patching but not coding I mean end users and not programmers.
I came to code or patch for artistic reasons
I m very happy of vvvv but there are things where i will stay on high level patching, not low levels.
Shaders are great but they are restrictions for those who are not coding
Despite its awesome plugin structure vvvv should stay open to non coders

Litlle nodes with basic operators for texture manipulation would be really a must for users.
They could construct easely what they want to, in an easy friendly creation processp

Maybe you should try or being demoed touch designer to understand what i m trying yo say about ergonomy and keeping mind concentrated on more important things than tool structuring

Voila ! Sincerely, christoph

Sincerely christoph

20

@unc: grande! Info (EX9.Texture) should return everything you need to know, please see my demo above for a simple specification i suggested to start from. i think some thought should be put in the naming of parameters for GUIs to be able to read and interpret those nicely…

@karistouf: so you’re afraid of the communities skills…ic, but there you have @unc who just offered himself to do some work on this and judging from his contributions so far you should be rather lucky to have someone like him contributing to such an endeavor.

you should realize that once the basic work has been done (which involves no rocketscience at all, so it is fairly realistic to get to a good point real quick) you and the end-users (not programmers) will have exactly what you wanted, i.e. texture-fx-nodes (just forget for a while that they are modules) for patching, no further coding needed.

i am afraid, this does not make sense. compare: “rockets are great but they are a restriction for those who can’t fly them.” see?

absolutely. thats why i am arguing to have texture-fx modules which are much more open to patchers (rightclick) than plugins. also once there is a texture-input for plugins they can still work side-by-side, only that non-coders will not be able to peak at them and improve them or adapt them to their special needs.

i’d say such modules would be fairly lightweight CPU-wise and as such not contribute too much to your resource problems. still the patches and spreadcounts you build around those light modules would probably be the main reason if you encounter low performance.

now post your wishlist for texture-fx as unc asked you to. try to be specific, defining not only the effect but also parameters and possibly their value-range in order to help unc get an overview of what you’re thinking of…