Download the Project Files

This tutorial will guide you through all the steps necessary to install and use Quartz Composer and Synapse with the Xbox Kinect on Mac. If you have any issues when you reach Step 4, then feel free to download my project files to experiment with.

Download Project Files

Step 1: Setup the Xbox Kinect

First things first, before we can move forward, you’ll need to make sure you have your Xbox Kinect setup. Follow this guide to get you setup and ready to go.

Step 2: Install Quartz Composer

Quartz Composer is an amazing app created by Apple, and distributed via the Apple Developer network.

1. Open Xcode on your Mac, and navigate to: Xcode / Open Developer Tool / More Developer Tools…

Menubar navigation to ‘More Developer Tools…’

2. This will take you to the Apple Developer website.  Once logged in you should see the Apple Developer Downloads page. Scroll until you see the Graphics Tools for Xcode package. Click the disk image icon to download.

3. Once downloaded, double click the disk image, and then copy the contents to the Applications folder on your hard drive.

Step 3: Install & Open Synapse

Synapse is an awesome application by Ryan Challinor that will supercharge your Kinect.  It’ll recognise your body’s joint positions and movements, and then send all that data to Quartz Composer for us to manipulate. So to get started, let’s install this little wonder.

1. Firstly head on over to Ryan’s site and download Synapse for Mac, and Quartz Composer example project + plugins. Unzip and move the contents to the Kinect folder we created in the previous tutorial (or anywhere you like – this is really just to keep things organised).

2. Next navigate to the following folder on your computer: Library / Graphics / Quartz Composer Plug-Ins. If the Grahics or Quartz Composer Plug-Ins folders don’t exist, then feel free to create them.

3. From your Synapse folder, move the two files called qcOSC.plugin and Synapse Kinect.plugin to the Quartz Composer Plug-Ins folder.

4. Awesome, now let’s get skeletal! Back in your freshly unzipped Synapse folder, you will find an application called (funnily enough) Synapse.  Open it up, and you should see a window appear.  Now stand up, bend your elbows, and point your hands to the roof and you’ll see a red 3D skeleton lock onto your body. Wave your hands around and you will also notice the blue swipe marks appear from your movement.

4. Remember that Quartz Composer example project + plugins zip you downloaded? When you unzipped it, you would have created a QuartzKinect folder. Inside here you will find 4 Quartz Passthrough files, these allow Synapse to send your movement data to other applications (very important). To start, lets open the one called quartz_passthrough, and leave it running in the background with Synapse.

Step 4: Animating with Synapse and Quartz Composer

And now for the part you’ve been waiting for! This next section will show you how to use Synapse in conjunction with Quartz Composer to create a motion-controlled animation.

1. At this point, you should have Synapse open with your skeleton detected, and quartz_passthrough running to send through your movement data.  Now open the Quartz Composer application. When you do, you will see the template chooser.  Select Basic Composition and click the Choose button.

When your project opens, you’ll see 3 windows.  The biggest one is your Editor – in Quartz, you’ll use this visual editor to code up your application. Then you have a Library (aka Patch Library) that contains a bunch of cool patches (aka modules) you can drag and drop into the Editor.  And lastly you have the Viewer, which will allow you to preview your application as you build it.

2. In the Library window, type the word Synapse in the filter search box, and drag the Synapse patch over to the Editor.

3. Next search the Library for Sprite, and again drag this patch to the Editor. You’ll notice that when the Sprite is added, your Viewer  will display a white box like so:

When you add a patch in Quartz Composer, you’ll notice they often have a list of parameters.  To edit these easily, click the Parameters button on your Editor window.

4. To get our sprite to stretch and fill our Viewer completely, click the Sprite patch, and then open the Parameters pane. Change the Width to 2, and Height to 1.5.

5. Next we are going to fill our sprite image with the feed from our Xbox Kinect.  To do this, we click on the circle beside Depth Image in the Synapse patch, and drag it to the Image parameter on our Sprite patch.  Once connected, it should look like so:

And your Viewer should look something like this:

6. Okay, your feed’s working, so how about we add some animation! Search your Library for the Particle System patch, and drag it to your Editor. Then search for the Lenticular Halo patch and drag it over too.

7. Next join the two together by dragging the circle next to Image on Lenticular Halo to the circle next to Image on Particle System like so:

You’ll now see something like this in your Viewer:

8. Now let’s make it move. Search your library for the following patches, and drag them to the editor: qcOSC,  Structure Index Member (drag over two of these), Math (and drag six of these).

9. Now click on the qcOSC patch in your Editor, and click the Patch Inspector button.

10. When the Patch Inspector opens, you’ll notice a drop-down menu. Click it and select Settings. Then change the port number to 12348. You’ll notice that the qCOSC patch starts gaining additional parameters.

11. Now let’s get our right hand to control the horizontal x-position of our particle system. To do this, grab 3 of the Math patches, and change their Parameters to the following: Math patch #1: 0, Divide, 640; Math patch #2: 0, Minus, 0.5; Math patch #3: 0, Multiply, 2

12. Now we can link it all up!

• Drag the circle from the qcOSC / righthand_pos_screen parameter to the Structure Index Member / Structure parameter
• Drag the Structure Index Member / Member parameter to connect with the top left circle on your Math (Division) patch
• Drag the top right circle from Math (Division) to the top left circle in Math (Minus)
• Drag the top right circle from Math (Minus) to the top left circle in Math (Multiply)
• And finally, drag the top right Math (Multiply) to the Particle System / X Position parameter.

Now take a look at your Viewer, and move your right hand around. If you did everything right, you should see the particle system following your hand movement.

Now, how about we get the vertical y-position working too! This is pretty similar to what we just did, but with some parameter differences on the Math patches.

13. Select the remaining three Math patches, and set their parameters to the following: Math patch #4: 0, Divide, 480; Math patch #5: 0, Minus, 0.5; Math patch #6: 0, Multiply, -2.

14. Now select the remaining unlinked Structure Index Member patch, and set the Index parameter to 1.

15. Now link them all up like you did for the x-position, except for the final Math (Multiply) object, which you will connect to the Particle System / Y Position parameter like so:

Now wave your right hand around again, and you should see the particle system following your movement on both the x and y axis.

Extra Credit

Okay okay, I hear you.  “The lenticular halo effect is nice, but what else can I do?”  Lucky for you, Quartz composer comes with a raft of patches for you to experiment with.

In one of my previous posts, reader Luis commented that he was looking forward to a Quartz Composer motion tutorial so he could put together something for his wedding.  With this inspiration in mind, let’s alter our project to become all lovey dovey!

1. Click the Lenticular Halo in your Editor and press Delete to remove it.

2. Now search the Library for the Gradient patch.  Now what we want is to render this gradient above our Kinect feed, but below our particle system.  Click the top right corner of the Gradient patch to reveal the layer menu, and select a number that’s lower than that of our Particle System layer, but higher than that of our Sprite.  See these images for reference:

3. Now open the Parameters for the Gradient patch. Select 3 colours of your choice, and change the Blending option to Add to overlay our Kinect feed.

4. Now we have our filter in place, we’re going to import an image to use with our particle system. Search the Library for Image Importer and drag it to the Editor.

5. Now open the Parameters for Image Importer and you’ll see a field called Image Location. The easiest way to fill this is to simply drag an image from your desktop onto the empty field.  Just remember to make sure that the Quartz Composer application is in the foreground when you do this or it won’t register. Alternatively, if your image is in the same directory as your Quartz Composer project file, then you can just write the name of your image here. I’m going with a heart image to match our theme.

6. Lastly, we just need to connect our image to our particle effect. Click the circle next to Image Importer / Image, and connect it to Particle System / Image and watch the magic happen!

And if everything went well, you should have a fun new animation playing in your Viewer:

Troubleshooting

We’ve just been through a lot of really detailed instructions. If your project is not tracking your hand movements, don’t worry, there are a few things we can do to try and fix the issue.

1. Firstly save your Quartz project, then close Synapse, quartz_passthrough & Quartz Composer.  Then re-open them one at a time. Synapse first (then hold your hands to track your skeleton), quartz_passthrough second, and then open your saved Quartz Composer project last.

2. If your qcOSC patch is showing too many parameters (like this), then you’ve probably closed and re-opened Synapse or quartz_passthrough during the tutorial. To fix this, try quitting Synapse & quartz_passthrough, then deleting the qcOSC patch. Then re-add the qcOSC patch to your project, and link it back up. Save your project, and then follow the instructions above to re-open the applications in sequence.

Phew, you made it! Hopefully you were able to get everything up and running.  Remember, if you’re still having trouble, you can download the Quartz project files I created to play around with.  If you have any other issues, then please shoot them through to me in the comments and I’ll try to add them to the troubleshooting section above.