Robert Wang Releases Smart Hand-Gesture Interface

Robert Y. Wang, co-founder of San Francisco-based 3Gear Systems, has developed a gesture-recognition system sophisticated enough to challenge the keyboard and mouse as an interface for productivity applications.

Wang’s system, which was released free of charge to developers Wednesday, uses two 3-D cameras mounted above the area in front of the user to track finger and hand motion to within a fraction of a milimeter. That’s more than enough accuracy to match the motion-detection capabilities of keyboard typing or mouse motion.

The success of Wang’s system depends on luring developers to create applications to take full advantage of 3Gear’s sophisticated hand-tracking algorithms. Among applications envisioned by Wang are engineering tools for prototyping components, manipulation of endoscopic surgical tools and even precision gaming. Unlike current systems that can only detect gross motion of hands held aloft, leading to “gorilla arm” aches, 3Gear’s system can detect subtle motion while the hands and fingers rest in more comfortable positions.

The system places 3-D cameras above the hands, for example, by clipping them to the top of the monitor. That allows the hands to rest on a keyboard or a desktop rather than being held suspended in space. Earlier efforts at creating comfortable gesture-tracking systems involved gloves or markers on the hands and fingers.

At the heart of 3Gear’s system is a program for matching the position information transmitted by the cameras at 30 frames per second to a database of 30,000 hand and finger configurations. The matching process takes a mere 33 miliseconds, creating the impression that the computer is responding instantaneously.

Wang’s system will face rival approaches. One is Microsoft’s update to the gaming Kinect system that allows tracking of the head, eyebrows and the mouth. Single-camera systems developed by Israeli startup Omek, Belgian startup SoftKinetic and another San Francisco startup from San Francisco called Leap Motion claims to track motion to one-hundredth of a millimeter.

Wang’s system represents an advance on an earlier system he co-developed during his graduate studies at MIT. That system used a basic computer video camera and a $1 pair of lycra gloves marked with patches of various shapes and colors for keying to a gesture database.