Forum

Kinect Player Texture to Polygon

Hello everyone, for my thesis which concerns kinect programming for interactive environments and dancing performance, I’ve started to explore the VVVV world for its great real-time efficiency.

I totally fell in love of @dottore’s ParticlesGPU approach and I want to use it for my project.
Referencing to his ParticlesGPU_2d_Dynamic_FieldTexture patch, I need a player-shaped texture to control particles’ behaviour, in which I need to set pixels’ color manually.
My idea was to use Player (Kinect Microsoft), searching for contours using Contour (FreeFrame DShow9) and transforming the obtained points into a Polygon (EX9.Geometry 2d). That’s obviously returned a mesh where all contour points were connected to one center. Speaking about a human body shape that can’t work at all, a lot of lines were out of contours.
Is it possible producing a mesh with more than one central point, like skeleton’s points produced by Kinect for example? In that case I can connect contour points with the nearest skeleton’s point and the mesh should be fine.
Probably I just complicate the whole thing, but I can’t figure out how to solve my problem.

hmm maybe u can continue to work with the texture instead of creating polygons… i think there are multiple possibilities…for example particle avoidance or gpu optical flow…however doin it on gpu seems to be quite nice approach…maybe ull find some inspirations on my page…princemio.net…good luck for your project !!! hope to seee it!!!

Actually your video Kinect Flow n. 1 was my first inspiration! :D it’s a pleasure to meet you! Did you work just on texture obtained from player node?

if you need polys, use the imagepack.
player texture > asImage > contour (mode set to poly) > perimeter

After a lot of work (not just for this issue) i’ve obtained a decent mesh by resampling points from contour and using Delaunay method to create it. Unfortunately Delaunay works just with convex shapes. I’m trying to compile a node for filtering triangles that have a side longer than a threshold (imho it’s just the simpliest and fastest to implement solution), but i can’t make it work (no errors on compiling).
Can someone help me?

Here’s the code:

- region usings
using System;
using System.ComponentModel.Composition;
using System.Collection.Generic;
using VVVV.PluginInterfaces.V1;
using VVVV.PluginInterfaces.V2;
using VVVV.Utils.VColor;
using VVVV.Utils.VMath;

using VVVV.Core.Logging;
- endregion usings

namespace VVVV.Nodes
{
	#region PluginInfo
	[PluginInfo(Name = "DelaunayFix", Category = "Value", Help = "Basic template with one value in/out", Tags = "")](PluginInfo(Name = "DelaunayFix", Category = "Value", Help = "Basic template with one value in/out", Tags = ""))
	#endregion PluginInfo
	public class ValueDelaunayFixNode : IPluginEvaluate
	{
		#region fields & pins
		[Input("Indices", Visibility = PinVisibility.True)](Input("Indices", Visibility = PinVisibility.True))
		public ISpread<Vector3D> Indices;
		[Input ("Vertices", Visibility = PinVisibility.True)](Input ("Vertices", Visibility = PinVisibility.True))
		public ISpread<Vector3D> Vertices;
		[Input ("Threshold", DefaultValue = 0.5, Visibility = PinVisibility.True)](Input ("Threshold", DefaultValue = 0.5, Visibility = PinVisibility.True))
		public double Threshold;
		[Output("Output", Visibility = PinVisibility.True)](Output("Output", Visibility = PinVisibility.True))
		public ISpread<Vector3D> FixedIndices;
		#endregion fields & pins

		//called when data for any output pin is requested
		public void Evaluate(int SpreadMax)
		{
			List<Vector3D> newIndices = new List<Vector3D>();
			//It scrolls all the indices
			//If it finds a triangle's side longer than threshold
			//then it will be discarded
			for (int i = 0; i < SpreadMax; i++)
			{
				double distanceAB = VMath.Dist(Vertices[(int)Indices[i]((int)Indices[i).x], Vertices[(int)Indices[i]((int)Indices[i).y]);
				double distanceBC = VMath.Dist(Vertices[(int)Indices[i]((int)Indices[i).y], Vertices[(int)Indices[i]((int)Indices[i).z]);
				double distanceCA = VMath.Dist(Vertices[(int)Indices[i]((int)Indices[i).z], Vertices[(int)Indices[i]((int)Indices[i).x]);
				if ( (distanceAB < Threshold) && (distanceBC < Threshold) && (distanceCA < Threshold))
				{
					newIndices.Add(Indices[i](i));
				}
			}
			FixedIndices.SliceCount = newIndices.Count;
			FixedIndices.AssignFrom(newIndices);
		}
	}
}

hey hiroj,

really glad u liked flow. Flow is done in Open Frameworks which is kinda what i mostly worked with. However im now nearly 90 % of my time in vvvv for interactive and scenographic works - since i do believe that it drastically enhances the overall artistic process.

flow is just a thresholded image that is used for calculation of optical flow. Once i had this optical flow i used it in order to manipulate fluid using navier stokes equations…There is plenty of solutions for navier stokes and optical flow also in vvvv…

for mapping of tracked data and projector u should check Elliots kinect projector calibration tutorial…it gives u a camera matrix which can be used to find a 2d point in the projected image for each pixel tracked by the cam in 3d.

Good luck and let me know how its goin

About this part of the project i worked with vector fields. Here’s a video: https://mega.co.nz/#!PExD1C4K!wzuGSbYR55yf1YgqngO5uL1Hrd49ycoAjzkpyi5x9GQ
This video refers to a previous version of my work, recently i added some particle behaviours using equations. I’ll surely take a look to navier stokes and optical flow :)

I think i used optical flow when i look for a method to move particles around my silhouette, assigning the colors of that silhouette (i used Metallica (EX9.Texture Filter)to a grid of vectors. Am i right?

Fixed… but (as expected) it’s a bugged solution. WIP for a better version.

- region usings
using System;
using System.ComponentModel.Composition;
using System.Collections.Generic;
using VVVV.PluginInterfaces.V1;
using VVVV.PluginInterfaces.V2;
using VVVV.Utils.VColor;
using VVVV.Utils.VMath;

using VVVV.Core.Logging;
- endregion usings

namespace VVVV.Nodes
{
	#region PluginInfo
	[PluginInfo(Name = "DelaunayFix", Category = "Value", Help = "Basic template with one value in/out", Tags = "")](PluginInfo(Name = "DelaunayFix", Category = "Value", Help = "Basic template with one value in/out", Tags = ""))
	#endregion PluginInfo
	public class ValueDelaunayFixNode : IPluginEvaluate
	{
		#region fields & pins
		[Input("Indices", Visibility = PinVisibility.True)](Input("Indices", Visibility = PinVisibility.True))
		public ISpread<Vector3D> Indices;
		[Input ("Vertices", Visibility = PinVisibility.True)](Input ("Vertices", Visibility = PinVisibility.True))
		public ISpread<Vector3D> Vertices;
		[Input ("Threshold", DefaultValue = 0.5, IsSingle = true, Visibility = PinVisibility.True)](Input ("Threshold", DefaultValue = 0.5, IsSingle = true, Visibility = PinVisibility.True))
		public ISpread<double> Threshold;
		[Output("Output")](Output("Output"))
		public ISpread<Vector3D> FixedIndices;
		#endregion fields & pins

		//called when data for any output pin is requested
		public void Evaluate(int SpreadMax)
		{
			List<Vector3D> newIndices = new List<Vector3D>();
			//It scrolls all the indices
			//If it finds a triangle's side longer than threshold
			//then it will be discarded
			for (int i = 0; i < SpreadMax; i++)
			{
				//double thresh = Threshold;
				double distanceAB = VMath.Dist(Vertices[(int)Indices[i]((int)Indices[i).x], Vertices[(int)Indices[i]((int)Indices[i).y]);
				double distanceBC = VMath.Dist(Vertices[(int)Indices[i]((int)Indices[i).y], Vertices[(int)Indices[i]((int)Indices[i).z]);
				double distanceCA = VMath.Dist(Vertices[(int)Indices[i]((int)Indices[i).z], Vertices[(int)Indices[i]((int)Indices[i).x]);
				if ( (distanceAB < Threshold[0](0)) && (distanceBC < Threshold[0](0)) && (distanceCA < Threshold[0](0)))
				{
					newIndices.Add(Indices[i](i));
				}
			}
			FixedIndices.SliceCount = newIndices.Count;
			FixedIndices.AssignFrom(newIndices);
		}
	}
}

I cannot figure out how to change the texture coordinates so that the texture is mapped within the mesh. In this version it acts like a mask.
Can someone help me?

Delaunay_Silhouette.rar (42.5 kB)