New gesture recognition......ITS A KEEPER

deep breath....


After a brainstorming session....a new gesture recognition idea was developed...with inbuilt volume and tempo control.

How you ask...it can't be done you say?....pffft...with some coffee, no social life and no sleep anything can be done...

Okies...so here how the new one works...

When the app detects the IR point, it begins to track it... and then the fun begins

While its tracking, a beat timer runs in the background...counting in seconds up to 8...and simultaneously, between each beat the distance covered is measured with a specific tolerance of pixels so that the recognition is not too sensitive.

next....the current and previous coordinates of the IR position is constantly being updated... therefore, I can test whether the user is conducting up or down or left or right.

Using this i did the following...

from the initial Ir point I ignored the x values and concentrated in the y...as soon as the IR point was less than the previous coordinate, I know that the direction has changed, so I trigger that as a beat.

next I had to determine which direction the user was conducting in now...
to check if it was left or right do this ignored the Y value and concentrated on the X value.
including the tolerance, I checked the current X value against the previous x value and could then tell the direction.

if the user conducts straight back upwards, the recorder goes back to ignoring the X value....

Next...essentially I mimicked the x checking for determining when the hit the 3rd and 4th beats.

Then, when the user releases the button and the IR is lost...the final beat is recorded and the whole thing resets.

here is a code snippet...

tracking...
public function track(e:TimerEvent):void{

newX = p1x;
newY = p1y;

if(i == 0){
prevX = newX;
prevY2 = prevY;
prevX2 = prevX
prevY = newY;
i = 1;
}
else if(i == 1){
prevX = newX;
prevY = newY;
i = 0;
}

}

and here is the bit that checks from beat 2 to what the next beat is...


public function leftorright():void{

beat2 = true;
if(p1x < prevX2 - tolerance){
directionfound = true;
//trace("Heading to beat 4");
goingleft = true;
beat1to2.stop();
beat2to4.start();
goingup = false;
goingdown = false;
squarecount = 0;
numofbeats +=1;
}

else if (p1x > prevX2 + tolerance){
directionfound = true;
//trace("Heading to beat 3");
beat1to2.stop();
beat2to3.start();
goingup = false;
goingdown = false;
numofbeats +=1;
}

else if(p1x > prevX2 - tolerance && p1x < prevX2 + tolerance && p1y < prevY2 - (3 * tolerance)){
directionfound = true;
//trace("Heads back to beat 1");
beat1to2.stop();
backto1.start();
numofbeats +=1;
}
// beat2to4.start();

}


A little bit hard to digest...however...essentially...

this allows the user to freely conduct and control tempo plus time signatures...

so is really good

:D

Gesture Recognition Rethink...

WOW!....what a couple of days...

okies, so essentially I did switch the gesture recognition class previously talked about over to the IR input. Boy oh boy was I in over my head a little bit. I forgot to take into account that not many people can hold their hand completely still, and even if the human eye could not detect the one pixel movements in the IR...the wiimote sure could. So essentially...every-time the IR moved a pixel off the users desired course...it would register as a direction change and the gesture would be unrecognizable...

Oh well, i did learn a hell of a lot more about class structures and Actionscript 3 language during the last couple of days...now for a different approach...

I want the user to be able to see where they can conduct, but at the same time i still want freedom....so after a bit of brainstorming i came up with this idea...



The final implementation changed a bit from the original design...but here is a screen shot of what the game looks like at the moment...

(Obviously needs polishing) but the set up is there and the gestures work :D...


Okay, here is how it works...

The tempo timer counts through 4 beats before resetting itself. Your standard 4 beat tempo.

as you can see there is an array of conducting points...
During this time, the user can conduct however they want, but when the tempo counter hits 4 beats it checks to see what points have been hit and in what order. It then check against the possible beat patterns and changes the tempo accordingly...

If the user does not conduct, or makes an unrecognisable pattern(or if the cheat and just wave the IR manically :P), the tempo will randomly decrease at different parts of the orchestra...thus creating chaos and the music will sound bad.

So, to conclude, great progress these last couple of days. Although I do miss sleep, it was worth it.

here is a screen capture video of the app in action....a little laggy and no audio...but the IR dot moving around is controlled by the IR conducting baton...and the tempo changes accordingly...as does the music...



My next step is to polish up the orchestra, separate the orchestra into different elements and finally set up my projection surface.

Cheers For Reading...