Hot damn! On a system where I already had the the Mixed Reality headset working (in my case an Acer), I just installed this in Steam:
started up SteamVR, and it recognized the headset and controllers. I then fired up a vvvv patch, and boom there it was in the Acer headset!
There is a problem though, the two eye orientations are not correct; one is looking up at an angle a bit from the other, which makes for some wacky depth perception. This can probably be kludged, but I’m guessing that this plugin may be behind the current rev of SteamVR that enables the MR headsets (it is in early access) and is getting the headset data incorrectly. That and the controller 3D model is wrong of course, but that’s just an external file.
I’ll poke around on it. Anyone with pointers on updating the plugin with new libs, lemme know! The MR headsets are way lighter than the Vive, and use inside-out tracking so don’t need lighthouses, so better for quick setup demos, etc. - although I still like the Vive better for really doing things.
you can recompile the source from the github repo, just replace all OpenVR files with the latest ones.
a new release is on my to-do list… hopefully be the end of this week, if you want to wait.
I may give it a shot if I can find some more time; I’m on the fence on buying a Samsung Odyssey, but will if this can be made to work right.
Found another problem when trying this with a real scene: when you move it’s like there is this time stutter where it jumps back and forth along the movement, as if the time warping index is wrong or frames are getting asynchronously queued in the wrong order.
No joy. Recompiled with latest OpenVR and still have the bizarre eye shift and time discontinuity.
I decomposed the View and Projection transforms, and not only are they shifted in odd ways (X, Y and Z!), but they change as you move the headset! I can think of no reason why things like eye separation are changing. I took the first slice of each and then did a manual shift for the second eye as in the old Rift plugin, and then everything looks correct.
But there then is still the the jumping around in time that becomes obvious when you move. I doubt I will have time to look into it further, hope someone has some luck with it.
A bit more background - neither of these things happen with Steam programs like the Night Cafe, so both problems seem to be related to this plugin, not the MR integration in OpenVR.
we recently used it in a workshop and it was plug&play and looked perfect. so there might be some specialties to your setup… what card and which headset are you using?
First, about the eye offset: I have delved into this a bit more, and verified there is a problem with the eye positions with my Acer MR headset. Here are the eye offsets for a Vive; the top numbers are the individual eye offsets from the head center, and the bottom numbers the difference between the eye Views:
Note the symmetric X translations, and no rotations. The differential does not change with headset orientation changes.
Here is the Acer:
Note the left eye is at the center of the head, and the right eye is shifted the entire inter-ocular distance, plus has an X rotation. Way bogus. I suspect that since they are not symmetrical, that explains why I see the eye View differentials change with head orientation - does that make sense?
I looked in the VVVV.plugin code, and do not think the fault lies there, rather that OpenVR is reporting bad data for the Acer MR headset. I’ll report this as a bug at OpenVR/Steam.
Concerning the time glitches on movement, turns out that happens when the scene is too complex and the FPS drops below about 40-50, getting quite severe if it gets down to 30fps. It does not happen at all if the FPS is above 50; this is on a notebook with an i7-7700HQ and GTX1060. For some reason the MR compositor degrades much less gracefully than the Vive at lower framerates. I’ll poke around more on it.
thanks for investigating, this is very helpful information!
there is hope that the offsets get corrected once the OpenVR pack is compiled against the latest openVR dlls. but not very likely that it brings improvements on such a central functionality… but you never know.
Just got a Samsung Odyssey HMD. It shows the same left eye at head zero problem, but it at least does not have the right eye pitch/X-rotation that the Acer has which is so disconcerting:
But the Samsung is missing the Z offset, just like the Acer; the eyes should not be at the center of the head, but in front if it slightly as with the Vive (-.015). But maybe MR reports the left eye as HMD position instead of head center; not a big difference but can make close objects look wrong.
So it looks like the incorrect eye positions may be a Microsoft Mixed Reality and OpenVR issue, while the right eye pitch is an Acer-specific (at least) issue.
A temporary fix would be to test for a zero X offset of the left eye, and in that case split the right eye offset between the two, shift the Z, and always zero the right eye rotations (or make them the same as the left). I’ll do some more tests to see if I can figure out if MR is in fact reporting the left eye as head center; I haven’t looked at the MR API but maybe they deal with things eye-relative, and since OpenVR deals with things head-relative, it might just be getting lost in “translation” so to speak. (Sorry, couldn’t resist.)
I opened a bug on this on the OpenVR github, and sure enough MR does not report an HMD pose, and apparently OpenVR just uses the left eye pose in that case.
More thoughts about the jumpiness: the OpenVR call GetDeviceToAbsoluteTrackingPose has an input argument fPredictedSecondsToPhotonsFromNow that it uses to predict the pose it returns. If this argument is defaulted, it simply looks at the last synced frame time and uses that to look forward; so if the last frame had a long delay, it looks a long time in the future, if the last was quick, less time in the future.
So in an environment where the framerate can be dramatically varying (sadly vvvv), the returned pose can appear to jump around in time. I think it might be worth a shot to have the option to not take the default behavior, and pass in a target time, or a smoothed actual time. I think it could make for a better and more consistent user experience. Any ideas where this can be done in the code?
