A smart glove that translates American Sign Language (ASL) into digital text has been developed by scientists at the University of California, San Diego. Timothy O’Connor, Darren Lipomi and colleagues reckon that their device can be produced for less than $100 and could also find use in virtual-reality and remote-control systems.
Most systems for monitoring body movement involve using a camera or infrared emitters and sensors to capture motion. While such systems are effective, they can be bulky, inflexible and require large amounts of energy. As a result, researchers are keen to develop wearable motion sensors – and gloves offer a natural way of tracking hand motion.
Strain sensors
The new device is based on strain sensors that are made of a piezoresistive composite material comprising carbon particles embedded in a flexible material. To make a sensor, the team begins with a strip of silicone 3 cm long, 0.5 cm wide and 340 μm thick. This is coated with a special paint that contains carbon particles in a fluoroelastomer resin to create a piezoresitive film that is about 50 μm thick. Copper contacts are added at either end and the sensor is then encapsulated in polyurethane.
Nine sensors are placed on the back of a leather athletic glove – two on each finger and one on the thumb. The sensors detect the bending of the knuckles, and the sensor signals are digitized and then sent to an on-board microcontroller that is attached to the back of the glove at the wrist. The glove also contains an accelerometer and a pressure sensor on the thumb. Information from these two devices is used by the microcontroller to differentiate between ASL gestures for different letters that produce similar signals in the strain sensors.
Bluetooth link
The team has shown that the microcontroller can translate the hand gestures associated with all 26 letters of ASL into digital text. These data can then be exchanged with other electronic devices via a wireless Bluetooth link.
What’s more, signals from the glove were also used to control a “virtual hand”. This, the researchers say, could have a wide range of applications including virtual reality, telesurgery and controlling aerial drones or bomb-disarming robots.
The process for making the glove does not involve chemical synthesis or access to a clean room. This, say the researchers, suggests that the system could be used as a test bed for evaluating other stretchable electronic components for human–machine interfaces. “We’ve innovated a low-cost and straightforward design for smart wearable devices using off-the-shelf components,” says Limpomi, adding: “Our work could enable other researchers to develop similar technologies without requiring costly materials or complex fabrication methods.”
Sense of touch
The team is now incorporating a “sense of touch” into the glove to allow a user to better control either a virtual or robotic hand by sending tactile sensations back to the user. O’Connor explains: “Our ultimate goal is to make this a smart glove that in the future will allow people to use their hands in virtual reality, which is much more intuitive than using a joystick and other existing controllers.”
Jim Kyle, emeritus professor of deaf studies at the University of Bristol in the UK says that while the technology looks interesting, the glove’s “application to sign communication is probably limited”. “People (deaf or hearing) who are able to use the manual [ASL] alphabet are also likely to be literate to the extent of being able to write. Use of a glove for communication is not necessarily the highest priority.” Kyle also points out that an individual’s implementation of ASL can be highly idiosyncratic and will often not involve spelling out entire words – making text capture difficult.
The glove is described in PLOS ONE.
