More Gesture Recognition

Okay, so calling the gesture recognition function took five minutes...learning, recording and creating the gestures took a lot longer.

to not bamboozle you, once again look at the mouse direction diagram on this page here.

Essentially, as each direction is assigned a number, you can use this to see what beat pattern the user is trying to make...

for a simple 2/4 beat, the user has to move the stick down then up...so the gesture code is 26.

to call this in action script 3 it was simply...

mg.addGesture("2beat","26");

great, well that's great if you want to limit the user to going only directly up and down... so i added some curves and flicks...

essentially the final 2/4 beat gestures looked like this...

mg.addGesture("2beat","26");
mg.addGesture("2beat","206");
mg.addGesture("2beat","2106");
mg.addGesture("2beat","21076");

that allowed for the user to make a u-shape in the air, add flicks to the movement, and it would still register as a 2/4 beat gesture.

now, 4 entries looks nice... by the time i got to the 6/4 gestures I was going insane trying to think of extra flicks and movements someone might make...

mg.addGesture("6beat","246246246205");
mg.addGesture("6beat","2462462462405");
mg.addGesture("6beat","24624624624605");
mg.addGesture("6beat","24624624624675");
mg.addGesture("6beat","24535353475"); mg.addGesture("6beat","245353543475");
mg.addGesture("6beat","2546346346065");
mg.addGesture("6beat","2546346346205");
mg.addGesture("6beat","2546346346305");
mg.addGesture("6beat","253463463405");
mg.addGesture("6beat","253453462405");
mg.addGesture("6beat","253453453405");
mg.addGesture("6beat","263463463405");
mg.addGesture("6beat","26346346305");
mg.addGesture("6beat","253535305);");
mg.addGesture("6beat","25353505);");

It looks like a lot of work, and it did take a couple of hours of testing and recording to get to this point. But i wanted to give whoever uses this as much freedom as possible, as long as they stick to the basic beat patterns(which will be in the next post), they will be fine.

Next was to change the tempo of the music...
As i previously posted, i have the 6 tracks pre-recorded and ready to play.

Essentially, one sound channel is playing the tracks, but only one at a time, if the user conducts a 5/4 beat pattern, the faster track will play, then if they change to say a 3/4 pattern, the channel stops playing the faster track, plays the 3/4 speed track from the same point that the music was up to, only slower.

After getting the above working, i found a little problem, the gesture recognition was far far too sensitive. If someone had a tremor, or unsteady arm, they would not find it very easy to stick to the patterns. Luckily, within the mouse gesture class there was a pixel-level sensitivity setting which i turned way way down to reduce adverse effects...after a bit of testing it seems to be working well.

Obviously what I have made is not perfect, but I think it's working damn well considering gesture recognition is a tedious process.

my next step is to apply the above to the IR conducting stick

Cheers

Changing Tempo on the fly: Gesture Recognition

Well, this is the bit i was dreading, gesture recognition... recognizing the beat patterns the user is making and changing the music accordingly...*shudders*.

Well actually it turned out to take five minutes thanks to this little doozy of a recognition class. Gesture Recognition

Thank you Didier Brun aka Foxy, for making that open source, you just saved me a weeks work :D

Okay, so to understand how it works have a read through that blog link above...but essentially... it tracks the starting point, direction and ending point of a gesture.

In my case, where the user is moving the conductors baton...

I have not got it functioning the way I want yet as I am still designing the Conductors Baton, but essentially I am using three basic patterns and when using the mouse it worked fine.

To switch it over to the IR position simply put I made the mouse cursor hidden and matched it's coordinates to the IR position...it worked so I plan to have a more stable version running in the next couple of days :D

New Panorama

Because using a cube panorama was getting rather annoying, what with having to change each individual faces image each time the user concentrates on a different part, I decided to revert back to using a spherical panorama.

The only problem I have with the spherical panorama is that it can warp a little bit depending on screen resolutions. So a little more testing, but more stable than the cube.

To make the panorama i just used 3D studio max 9's inbuilt panorama exporter...

first I made the scene using some free google sketch-up models i found at the warehouse.

Then positioned a starting point for the camera and exported.

Essentially I got this...



As you can see, it looks quite distance and weird, but when you chuck it on a sphere in papervision you get this...




And then with the tracking, as you look to different spots the camera changes accordingly, you get feedback etc...

So all in all, not a bad choice to use the sphere, I will just have to watch the warping.

New head tracking

Thanks to the old formula for head-tracking crashing and burning... heres my new one, which may or may not be included in the final prototype...

essentially, trigonometry...

I get the distance between the two points, work out the angle of rotation and hey presto you have got head-tracking (subject to tweaking of course).

Tempo tantrums :D

sorry for that terrible post title...

So, starting to work with the tempo of the audio for the game...

Originally i planned to do it real-time using the one audio track, but this was not too great as Actionscript 3 can't control playback speed directly, and to convert it into a byte array then alter it and then convert it back was lagging up the entire app.

In comes audacity...

using Audacity, I will just get the tracks, and make individual versions of them, all with different tempos...


"It's actually really easy to change the tempo...also I recommend the latest version of Audacity :D"

Then after importing these into the app, i can transition them depending on where the user is pointing and what movements they are making etc.

A little bit of extra work, what with importing the files, but all in all I believe it to be a better choice as the alternative could take months :P.

will have some screen shots soon...

New C# server

been away for a bit...turns out iinet really suck at activating internet accounts :D

but now that it's connected....

Due to IR point management, and some serious laaaagggg :D... I changed the initial app that sends over the IR info.

Now i'm using the default wiimote monitor app that comes with the wiimote library

The server sends out the raw IR data rather than the manipulated string the other app spewed out, it got rid of the lag and made it easier to work with.

heres what the interface is like... as you can see it has everything related to the wiimote, including tracking of up to 4 IR points...



more updates coming soon...