Algorithm for detecting a knob-turning gesture?

Question

What is the math to identify a gesture for the motion made by two fingers turning a knob (tracking two sets for xycoords)?

Is there a library/api anyone is familiar with that contains a good library of gestures for multitouch applications?

Answer 1

The math with two fingers (assuming that some API always gives you the centers of both fingers) is rather simple, but you get more than just turning. You get scaling and translating motion as well. You star with P1(0) (x1(0), Y1(0)) and P2(0) (x2(0), Y2(0)). Then compute Pc = (P1 + P2) / 2. After that adjust all of the subsequent coordinates such that Pc (center) is at (0, 0) by subtracting Pc from every point. After that you should be able to calculate how each of the fingers rotated around the center - how much it scaled and how much it turned with each step. Note that the user may have slided both fingers as well. Think watching a moving hurricane on a radar screen: it turns, it moves, it disperses and gets wider, or it gets smaller. If that sliding motion is significant, then you might want to compute the center dynamically.

Now, with N measurements of pairs of points P1 and P2, you should have data on how the center slided, how it turned, and how the radius changed. You are looking for little sliding, little radius expansion, and a linear increase/decrease of the angle with time. I would filter out big noise and then try to see how you can fit a straight line in your angle graph, and how well a line approximates it. Well ... the user can change the speed of rotation. Something that no generic library can do for you is decide on the rules for when to ignore a gesture. You need to provide more info on what the turning of the knob is like. I hope there are only 2 fingers involved. Are you trying to measure this turning or merely detect the motion?

EDIT: Thinking of the hurricane a bit more, I think it makes sense to compute the new center with every step. However, if one finger does not move, but the other does trace a circle, then it is a different type of motion than turning the knob. If you still model it as a knob, it will be interpreted as displacement with slow rotation as opposed to faster rotation around a static point.

Again, the math is not necessarily complicated. It is just a 2d transformation matrix + offset vector. But, when more than one finger is involved, the question becomes: what it the 'center' about which the transformation is being performed? It could be either 4 corners of the screen, the center of finger 1, the center of finger 2, the half-way point between two fingers, or other. How you interpret the motion will depend on the choice of the center. It might not matter too much if the users can easily learn and adjust their motion to get it working right. But you still want to take the best guess at how exactly they are planning on using the software, and what other gestures are there.

Answer 2

This is easy to over complicate.

You need to capture and indicate 4 simple ideas

1. they've grasped the knob

2. they started the grasp at angle a

3. they've now got the grasp at angle b

4. they've released the knob

Show 1 by highlighting the knob

Show 2 and 3 by rotating the knob a-b degrees.

Show 4 by removing the highlight

(1) should only happen when a line drawn from finger to finger intersects the knob. Once this happens...

(2) angle a is set

(3) b is continually reset as they move their fingers. This continues until they either remove them or the line no longer intersects the knob.

(4) leaves the knob where it's been rotated.

This is different then manipulating a picture with two fingers only in that no stretching or translation is allowed. Only rotation.