Hi drehwurm, yes your task is a bit tricky and advance.
With Kinect is posible but yes as antokhio said is very slow for that.
with ir is also tricky but if your actors remain in same z ,you could calibrate camera projector and maybe with voronoid deluany try to get the polygon using some opencv contour routine … but also hard to get accurate.
for ir camera tracking i used ids https://en.ids-imaging.com/home.html ethernet or usb 3 with fiber both works well or cheap https://corzotech.com/en/kits/184-infrared-ps3-camera-kit-5500010000000.html kit
the projector is one thing than can create lot of latency if not a good one in this quite old testing video https://www.dropbox.com/s/id10fh8qma892s4/MappingSurfaces%20_colorsound.avi?dl=0 the main latency was from projector , i tried kinect , ir , rulr tool from elliotwoods , markers ,ar patterns, and a few more tricks and techniques to see which was best solution for those sort of scenarios , the conclusion each specific thing required specific trick and its harder than oneself think. ;D
About ir light for big spaces installation i used a combination of old theather lamps with the pure color filters rgb ,those lamps emit a lot of infrared and are great for big spaces but filters burn quickly you,ll have to change every night and then for pro option or enforce those https://www.rayteccctv.com/ lamps .
This light combination was used to light up this big theather then track and mask those big moving panels xyz.
hope it helps. cheers.