cs247

A KinectJS "Hello World": The Joint Tracker

By: Arvind Satyanarayan, Winter 2013.

This short tutorial will help you get a little more comfortable with KinectJS. Our goal is to build an app that tracks a person's skeleton as they interact with the Kinect. Their joints will be drawn as little red circles, which will be connected together to form a skeleton. Here's some starter code, because no one wants to start with a blank page!

Ready? Let's go!

P.S. You may also find the KinectJS Cheat Sheet helpful.

Step One: Set Up KinectJS

To be able to use KinectJS, the first thing we need to do is set it up by identifying how many players we want to track and, for each player, which joints and gestures we care about. Here's a snippet of code to get you started. Make sure you understand it!

// Joints you want to track
var JOINTS = ['HAND_RIGHT', 'HEAD'];

// A helper function we'll want to use later with coordinates
var jnt = function(j) { return JOINTS.indexOf(j); }

kinect.setUp({
    players:  1,                    // # of players, max = 2
    joints:   JOINTS,               // array of joints to track
    gestures: ['ESCAPE', 'JUMP']    // array of gestures to track
})
.sessionPersist()
.modal.make('css/knctModal.css')    // Green modal connection bar
.notif.make();

While we aren't really concerned with tracking any gestures in this app, does this snippet contain all the joints you'd want to track? Double check the cheat sheet to make sure.

Step Two: Register a joints event listener

Sweet, now that we've connected to KinectJS, we can begin to listen for joints events. Any time one of the joints move, an event is "fired" and this function listens for that event.

kinect.onMessage(function() {

    // As this function is called on every joint update, clear the canvas first.
    ctx.clearRect(0, 0, canvas.width, canvas.height);

    this.sk_len;    // # of people in the frame

    this.coords;    // A 2D array[m][n] of (x, y, z) coordinates of joints.
                    // m = the ID of the person (either 0 or 1)
                    // n = the ID of the joint, which is an index into JOINTS above
});

Step Three: Draw a circle for each joint

Great, we've got all our scaffolding set up. Time to get our hands dirty!

In our joint event listener, this.coords gives us (x, y, z) coordinates for all the joints we've asked to track in Step One. Let's use this data to draw circles to represent the position of each joint. Here's a snippet of code that you can use to draw a circle in canvas.

ctx.beginPath();
ctx.arc(centerX, centerY, radius, 0, 2 * Math.PI, false);
ctx.fillStyle = 'red';
ctx.fill();

Remember, you'll want to set centerX and centerY as appropriate from this.coords. It can get a little confusing to remember the indices of joints for the second dimension of coords. Here's where our jnts helper function comes in handy. Check it out!

this.coords[0][jnt('HAND_RIGHT')];  // (x, y, z) for player 1's right hand

Feel free to play around with the value of radius to get the size of the circles exactly right. For more information on the parameters of ctx.arc, check out the Mozilla Developers Network

A simple way of drawing all our joints is to copy and paste this code the required number of times. But, is there another way to do it that results in less code duplication?

Step Four: Connect your circles to form a skeleton

Excellent, now we've got to connect our circles to give us a proper skeleton. Here's a snippet of code to draw line segments in canvas. Again, we encourage you to read this tutorial that does a great job of explaining how to draw shapes in canvas.

ctx.beginPath();     // Enter canvas' "draw mode"
ctx.moveTo(x1, y1);  // Moves the "cursor" to begin drawing at (x1, y1)
ctx.lineTo(x2,y2);   // Draws a line to (x2, y2)
ctx.stroke();        // Paints the line to canvas and exits "draw mode"

Bonus Step Five: Track multiple people

Woo hoo! So at this point, we're successfully tracking one person's skeleton. However, KinectJS can track up to two players, with the second player's joint coordinate information available under this.coords[1]. Is there an easy way for you to adapt your existing code to support tracking multiple people's skeletons?