Low-Cost Pliable Materials Transform Glove Into Sign-to-Text Machine
Researchers have made a low-cost smart glove that can translate the American Sign Language alphabet into text and send the messages via Bluetooth to a smartphone or computer. The glove can also be used to control a virtual hand.
While it could aid the deaf community, its developers say the smart glove could prove really valuable for virtual and augmented reality, remote surgery, and defense uses like controlling bomb-diffusing robots.
This isn’t the first gesture-tracking glove. There are companies pursuing similar devices that recognize gestures for computer control, à la the 2002 film Minority Report. Some researchers have also specifically developed gloves that convert sign language into text or audible speech.
What’s different about the new glove is its use of extremely low-cost, pliable materials, says developer Darren Lipomi, a nanoengineering professor at the University of California, San Diego. The total cost of the components in the system reported in the journal PLOS ONE cost less than US $100, Lipomi says. And unlike other gesture-recognizing gloves, which use MEMS sensors made of brittle materials, the soft stretchable materials in Lipomi’s glove should make it more robust.
The key components of the new glove are flexible strain sensors made of a rubbery polymer. Lipomi and his team make the sensors by cutting narrow strips from a super-thin film of the polymer and coating them with conductive carbon paint.
Then they use a stretchy glue to attach nine sensors on the knuckles of an athletic leather glove, two on each finger and one on the thumb. Thin, stainless steel threads connect each sensor to a circuit board attached at the wrist. The board also has an accelerometer and a Bluetooth transmitter.
When the wearer bends their fingers, the sensors stretch and the electrical resistance across them goes up. Based on these resistance signals, the circuit assigns a digital bit to each knuckle, 0 for relaxed and 1 for bent. This creates a nine-bit code for each hand gesture of the ASL alphabet. So if all fingers are straight, the code reads 000000000; for a fist it would be 111111111.
To distinguish between ASL letters that generate the same code, the researchers incorporated an accelerometer and pressure sensors on the glove. The letters D and Z, for instance, have the same gesture but the hand zigzags for Z while it remains still for D. In U and V, meanwhile, two fingers are held together and apart respectively, which the pressure sensor detects.
In tests, the glove could translate all 26 letters of the American Sign Language alphabet into text. The research team also used the glove to control a virtual hand to sign the ASL letters.
The next version of the glove will incorporate new materials that generate a tactile response so that wearers can feel what they’re touching in virtual reality. Today’s haptic devices simulate the sense of touch by applying forces and vibrations to the user. Lipomi and his students plan to convey a much broader range of signals. “We’re synthesizing materials that can be used to stimulate everything from pressure and temperature to stickiness and sliminess,” he says.
