Kinect x Projector Calibration

wip
1

I’ve been trying to find a way to calibrate the kinect and a projector in vvvv gamma.

My first attempt was implementing the KinectProjectorToolkit in vvvv.
However, I can’t get it to work properly and the original code doesn’t work either because of SimpleOpenNI not working. (I have a hunch it can’t be that difficult to fix it but my attempts have failed so far).

This is what I got: https://github.com/chaupow/vvvv-gamma-kinect-proj-calibration

Now I’m not sure what to do next

  • either I’ll keep staring at the code,
  • install ofx and dive into elliotwood’s Camera-and-projector-calibration or
  • try to use kinect pointcloud points in vvvv beta’s vl.opencv CalibrateAndReproject demo.

I’ll keep poking around but happy for any hints and suggestions :)

2 Likes
2

Hi Chaupow

your third option seems a doable one try to use kinect pointcloud points in vvvv beta’s vl.opencv CalibrateAndReproject demo.

create a pointcloud or mesh from the kinect data and use that mesh points as calibration mesh for the calibration routine . there was a rulr old kinect calibration but result was good in accuracy but very slow.

good luck.

1 Like
3

The best way is to decide on what you are doing and implement that… Instead of trying to hack someone else code, pretty sure it’s not gonna work out of the box, and most likely it uses OpenCV node set in both cases…

1 Like
4

You can also have a look at the Calibration help patch in my particles pack -> https://vvvv.org/contribution/dx11.particles

1 Like
5

Thanks for all these inputs! I’ve done a lot of procrastination and a bit of progress. My current understanding is

  • my endgoal is convert(3D-point-in-kinect-space) = 2D-point-in-projector-space
  • with Kinect + Coordinate Mapper I can get a point cloud with 3D coords (=kinect space)
  • with this formula I can project any 3D point onto my projector plane if I know the projector position in kinect space [1]

So, naively I thought I can just do manual guesswork on the projector position but I couldn’t quite guess a projector position to get the corresponding points to match:

E5414319-56E9-4573-A568-1A22ABF1E9C3

So now, I’m trying to figure out how to determine the projector position in the 3D kinect space. It seems like solvePnP can be used to determine the projector position.

Would love to hear if anyone thinks I’m heading in the wrong direction or if I’m on the right track.

Also, I’ve tried to use solvePnP from VL.OpenCV and it keeps crashing on me (nvm vvvv crashes if intrisic param is null otherwise works). I’m also confused on how to determine the intrinsics. If anyone has some experience with that, that’d be great.

[1] (honestly I’m also confused why this formula just requires the position and can disregard the projector’s direction)

1 Like
6

Hi @chaupow, solvePnP is what you need yes, to get the Intrinsics you need to run a camera calibration first and save the results. Have a look at the Camera Calibration help patch in VL.OpenCV. Make sure you calibrate using the actual camera at the same resolution you intend to use.

Also, if you have beta, you can see a full vvvv beta working demo of calibrate and reproject that ships with VL.OpenCV.

Hope that helps.

7

Taken from the Calibrate Camera help patch in gamma:

image

8

Got it to work!

In case someone would ever need the same, here it is https://github.com/chaupow/VL.KinectProjCalibration

Thanks for all the help

3 Likes