Save transform matrix in texture - read in vertex shader

Hi,
Having a hard nut here.
Its about instanced animation based on a animation-texture in vvvv.js, with the animation being prebaked from vvvv.
the concept is similar to instanced skinning described here http://developer.download.nvidia.com/SDK/10/direct3d/Source/SkinnedInstancing/doc/SkinnedInstancingWhitePaper.pdf
Just instaed of bone matrices i use transform matrices for particles
The idea is to have a typical transform spread in vvvv and sample an animation with it using queue,
then use get matrix,
convert the matrix into glsl row-column style (basicaly just swap dimensions for every matrix),
save the values into a texture,
in glsl rebuild the transform matrices from this data-texture in the vertex shader,
and finaly drive a complicated particle system with it in webGL.

Basically a prebaked particle system with every animation frame stored as a spread of transforms.

So I have attempt to do that and it works, almost.

TThe problem is that the texture coordinates together with the frame, animation is a bit hard to solve.
Additionaly I dont know if the PNG format is realy suitable for transfering a transform matrix.

For transforming the matrix I use 4 pixels with rgba of each pixel being the corresponding 4 rows of the matrix.

I am kind of sure that my input format is correct. I just wanted to post the vertex shader code here and maybe someone is interested in looking at this code and maybe reveal a thinking error in it.
I get the first frame of the animation kind of correctly, just with a wired behaviour when i want to use translation.
maybe i also dont use the transform matrix in the correct hirarchy?
The other problem is the frame-based sampling. When I change to another then the first frame, everything gets messed up. I wonder if there is a logic error

//the texture coordinates for sampling the matrix componetns using instanceID of the particles and frame * stride (transform count per frame) for the frame-based animation. 
// from the resulting index the row and column in the texture 2D array is calculated and its scaled to texturecoordinate space.
  float Coord1x = (mod((id * 4.0 + 0.0) + frame *stride       ,textureRes)/textureRes-0.5 ) * (1.0 - 1.0/textureRes) + 0.5;
  float Coord1y = (floor(((id * 4.0 + 0.0) + frame *stride   )/textureRes)/textureRes-0.5 ) * (1.0 - 1.0/textureRes) + 0.5;
  
  float Coord2x = (mod((id * 4.0 + 1.0) + frame * stride   ,textureRes)/textureRes-0.5 ) * (1.0 - 1.0/textureRes) + 0.5;
  float Coord2y = (floor(((id * 4.0 + 1.0) + frame *stride  )/textureRes)/textureRes-0.5 ) * (1.0 - 1.0/textureRes) + 0.5;
  
  float Coord3x = (mod((id * 4.0 + 2.0) + frame * stride,textureRes)/textureRes-0.5 ) * (1.0 - 1.0/textureRes) + 0.5;
  float Coord3y = ((floor((id * 4.0 + 2.0) + frame * stride)/textureRes)/textureRes-0.5 ) * (1.0 - 1.0/textureRes) + 0.5;
  
  float Coord4x = (mod((id * 4.0 + 3.0) + frame * stride,textureRes)/textureRes-0.5 ) * (1.0 - 1.0/textureRes) + 0.5;
  float Coord4y = ((floor((id * 4.0 + 3.0) + frame * stride)/textureRes)/textureRes-0.5 ) * (1.0 - 1.0/textureRes) + 0.5;
 //the matrix components are sampled and scaled into correct space
  vec4 m1 =  (texture2D(AnimationTex, vec2(Coord1x,Coord1y)  ) - 0.5 )* 2.0;
  vec4 m2 =  (texture2D(AnimationTex, vec2(Coord2x,Coord2y)  ) - 0.5 )* 2.0;
  vec4 m3 =  (texture2D(AnimationTex, vec2(Coord3x,Coord3y)  ) - 0.5 )* 2.0;
  vec4 m4 =  (texture2D(AnimationTex, vec2(Coord4x,Coord4y)  ) - 0.5 )* 2.0;
  // the matrix is constructed from the components
  mat4 transform = mat4(
        m1,
        m2,
		m3,
        m4
   );
  mat4 tWV = tV * tW;
  mat4 tVP = tP * tV;
  mat4 tWVP = tP * tWV;
  
  LightDirV = normalize(-1.0*(tV*vec4(Light_Direction_XYZ,1))).xyz;
  NormV = normalize(tWV * vec4(NormO, 0)).xyz;
  instancedColor = color;
  vec4 PosV = tWV * vec4(PosO, 1);
  ViewDirV = normalize(-PosV).xyz;
  
  
  vec4 pP = vec4(PosO * 4.0, 1.0) *transform ;
  //pP.xyz += offset;
  gl_Position = tVP * pP;
  vs2psTexCd = (Texture_Transform * vec4(TexCd, 0, 1)).xy;

and how do you store it on the vvvv side into the texture?

I am saving the data by using dottores constantPoint2.0 shader from the dx9 particle lib

The row column swap is done with vector split, 4 unzips, 4 vector joins and another vector.

Its relatively straight forward

what is the texture format? and are you sure the matrix values are between 0…1?

I was doing it once, planning to replicate it in FBX4V but with rawbuffers instead of textures, so different instances can access different animations too

Thanks thats very helpful microdee.

How is the data formated exactely in your example regarding rows columns?

Eventually it is always a good idea not to wrap the data “around the corner”, as i was doing it.

But thats not the reason why it was not working here. I think the problem is more the way how PNGs are compressed and if you use 4 channels per pixel you loose data.
Or it could also be the problem of texture filtering and the inability to do texelfetch in WebGL.
Whatever it is, it causes the data to arrive in a distorted way making it so far unusable.

So I switched to CPU variant and using vertex attributes for the animation.
By far not as optimal as I wanted it to be but enough to drive 30-50 characters

why can’t you use uncompressed float textures? with a width of 4 (= 16 float values) and a height = animation frame length.

then the sampling is dead simple: frame number = texture row

The problem is that WebGL supports only squared texture with multiple of 2 (512, 1024) at the moment.
Also support for DDS is very limited and BMP is too big in file size.

But I had a new idea how to improve performance:

Save the data in a .JSON format.
At Loading populate a framebuffer texture with the data
use the texture for animation.

Lets see if this works…

“format” of that was
one column is one frame,
pixels are a row of a matrix (4 pixel column is one matrix)
RGBA values are the 4 components of a row of a matrix

1 Like

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.