Kinect Color Tracking

In this posting, I would like to show how to track any click color using Kinect as webcam and I am using:

Kinect for Windows (not for Xbox), it may or not work for Xbox

Processing 1.5.1 (stable version), not Processing 2.0 has been released but in beta version.
I use Processing to process image and its interaction because I found it is the easiest to be understood and begun by beginner programmer or you do not even have background about it, like I am. I also found there has been so many programmer interested  to create open source library for it, it means you can have more access to many devices using this light and easy to use software.

Simple-OpenNI version 0.27, before it can function, it is required to install OpenNI, PrimeSense, and NITE software. I wish you may not face too many trouble when installing this 3 softwares.

The PC specification I use is: Windows 7 Home Premium, i7-3.4 GHz, 4GB RAM, and 2GB VGA.

There are not super book which could contain all we need for our project, this topic is inspired from some books, which I also recommend you to buy or email me if you could not find one:

1. Learning Processing: A Beginner’s Guide to Programming Image, Animation, and Interaction

2. Making Things See by Greg Borenstein

In the first book, it only explain how to track any color using your normal computer webcam, all about processing thoroughly details.

In the second book, it only explain how to use Kinect as common camera.

Using  normal webcam to track any object results in very bad resolution and it has not depth sensor for further development. So I think how if I use Kinect as my webcam so I could get better resolution using very good library from Simple OpenNI. After several days, it shows positive results, like this!

Well, this is my room view. You could see a small circle filled with orange color and dark stroke, it tracks orange color as I clicked it. It works!

it shows the version of my Simple OpenNI, then it shows the RGB data.
r for RED, g for GREEN, b for BLUE. Also it contains the pixel location of your clicked. Nice!

And finally here is the code:

import SimpleOpenNI.*;
SimpleOpenNI kinect;
// Frame
PImage currentFrame;
color trackColor;

void setup()
{
 size(640, 480);
 kinect = new SimpleOpenNI(this);
 kinect.enableRGB();

 trackColor = color (255,0,0);
 smooth ();

 currentFrame = createImage (640,480, RGB);

}

void draw()
{
 kinect.update();

 currentFrame = kinect.rgbImage ();
 image(currentFrame,0,0);

 currentFrame.loadPixels();

 // Before we begin searching, the "world record" for closest color is set to a high number that is easy for the first pixel to beat.
 float worldRecord = 500;

 // XY coordinate of closest color
 int closestX = 0;
 int closestY = 0;

 // Begin loop to walk through every pixel
 for (int x = 0; x < currentFrame.width; x ++ ) {
 for (int y = 0; y < currentFrame.height; y ++ ) {
 int loc = x + y*currentFrame.width;
 // What is current color
 color currentColor = currentFrame.pixels[loc];
 float r1 = red(currentColor);
 float g1 = green(currentColor);
 float b1 = blue(currentColor);
 float r2 = red(trackColor);
 float g2 = green(trackColor);
 float b2 = blue(trackColor);

// Using euclidean distance to compare colors
 float d = dist(r1,g1,b1,r2,g2,b2); // We are using the dist( ) function to compare the current color with the color we are tracking.

// If current color is more similar to tracked color than
 // closest color, save current location and current difference
 if (d < worldRecord) {
 worldRecord = d;
 closestX = x;
 closestY = y;
 }
 }
 }

// We only consider the color found if its color distance is less than 10.
 // This threshold of 10 is arbitrary and you can adjust this number depending on how accurate you require the tracking to be.
 if (worldRecord < 10) {
 // Draw a circle at the tracked pixel
 fill(trackColor);
 strokeWeight(4.0);
 stroke(0);
 ellipse(closestX,closestY,16,16);
 }
}
void mousePressed(){
 color c = get(mouseX, mouseY);
 println("r: " + red(c) + " g: " + green(c) + " b: " + blue(c));

// Save color where the mouse is clicked in trackColor variable
 int loc = mouseX + mouseY*(currentFrame.width);
 println (loc);

 trackColor = currentFrame.pixels[loc];
}