Converting OpenVR matrix to VL/Stride

[bjoern]

Hey,

I am currently in the process of making a nuget that enables the use of vive trackers without connected HMD. It is heavily borrowing from @tonfilm 's beta plugin.

Almost everything is working nicely but I just can’t wrap my head around the matrix / coordinate conversion from OpenVR to VL(Stride).

According to open.h OpenVR is a

right-handed system
+y is up
+x is to the right
-z is forward

Which should be the same as in Stride?

Looking at the Stride OpenVR code, the conversion is basically done like this:

Debug.Assert(Unsafe.SizeOf<Matrix>() > Unsafe.SizeOf<HmdMatrix34_t>());
Unsafe.As<Matrix, HmdMatrix34_t>(ref pose) = devicePose.mDeviceToAbsoluteTracking;

But when I do it that way the X and the Z axis are inverted.

Same happens when I do the conversion like that:

public static Matrix ToMatrix(this HmdMatrix34_t m)
        {
            
            return new Matrix()
            {
                M11 = m.m0,     M12 = m.m4,     M13 = m.m8,     M14 = 0,

                M21 = m.m1,     M22 = m.m5,     M23 = m.m9,     M24 = 0,

                M31 = m.m2,     M32 = m.m6,     M33 = m.m10,    M34 = 0,

                M41 = m.m3,     M42 = m.m7,     M43 = m.m11,    M44 = 1
            };
        }

I can of course negate X & Z (M41/M43). But then rotations seem to be off / inverted and I really don’t get how to counter that.

Any help much appreciated.

[motzi]

just a guess: have you tried switching rows and columns? (row-major vs. column-major)

see e.g. here for the memory layout vs. array indices

[bjoern]

@motzi

have you tried switching rows and columns?

No I did not, from what I’ve gathered OpenVR is row major.
Just tried transposing the matrix anyways but that completely breaks things.

[tonfilm]

The method you posted looks correct, but I think the row should be m0123 not the column.

[bjoern]

@tonfilm

but I think the row should be m0123 not the column.

like that?:

M11 = m.m0,     M12 = m.m1,     M13 = m.m2,     M14 = m.m3,

M21 = m.m4,     M22 = m.m5,     M23 = m.m6,     M24 = m.m7,

M31 = m.m8,     M32 = m.m9,     M33 = m.m10,    M34 = m.m11,

M41 = 0,        M42 = 0,        M43 = 0,        M44 = 1

I tried it but the result is even more “broken”.

I took the method from you beta plugin where it looks like that:

public static Matrix ToMatrix(this HmdMatrix34_t m)
   {
        return new Matrix()
        {
            M11 = m.m0,     M12 = m.m4,     M13 = -m.m8,    M14 = 0,

            M21 = m.m1,     M22 = m.m5,     M23 = -m.m9,    M24 = 0,

            M31 = -m.m2,    M32 = -m.m6,    M33 = m.m10,    M34 = 0,

            M41 = m.m3,     M42 = m.m7,     M43 = -m.m11,   M44 = 1
   };

[tonfilm]

@bjoern

like that?:

Oh, my bad. Only 3 per row…

[bjoern]

Sorry but I don’t get it.

[tonfilm]

m012
m345
m678
m91011

just a guess. but the first one you posted seems to be correct, if they actually use the same coordinate system as Stride. i think the matrix might be correct in your first method, but something else might be wrong.

[bjoern]

@tonfilm

m012
m345
m678
m91011

and last column?

0
0
0
1

[bjoern]

Unfortunately that also doesn’t work.
Funky squashed result:

[tonfilm]

@bjoern

But when I do it that way the X and the Z axis are inverted.

how do you know that? is the calibration setup correct? does the headset move in the opposite direction as the tracker?

Unsafe.As<Matrix, HmdMatrix34_t>(ref pose) = devicePose.mDeviceToAbsoluteTracking;

this should indeed be correct. Also, have a look at the controllers in Stride OpenVR, they behave the same as the trackers.

[bjoern]

@tonfilm

how do you know that?

I am not quite sure anymore tbh.

The result looks like this:

But I assumed it should be like that (except for the wrong rotations ofc):

[bjoern]

After some back and forth with @motzi he concluded that everything was correct from the get-go.
So no need to flip or rotate anything at all …

[bjoern]

This is now available as VL.IO.OpenVR.