BOULDER, Colo. — Artificial intelligence and smart technology could make it easier for people dealing with vision loss to navigate their everyday environment. Engineers at the University of Colorado-Boulder are working designing a “smart” walking stick that would help guide the visually impaired through everyday tasks, from finding a seat to shopping at a grocery store. The study authors describe their device as “assistive technology meets Silicon Valley.”
“I really enjoy grocery shopping and spend a significant amount of time in the store,” says study co-author Shivendra Agrawal, a doctoral student in the department of computer science, in a university release. “A lot of people can’t do that, however, and it can be really restrictive. We think this is a solvable problem.”
At first glance, the walking stick looks like any regular cane you can buy online. When you take a closer look, however, you’ll find several enhancements utilizing camera and computer technology. The walking stick uses the camera and computer to actively map out the surrounding area and then uses vibrations in the handle to guide users to their chosen destination. The cane also comes equipped with a voice assistant that provides directions such as “reach a little bit to your right.”
While the smart cane is not meant to replace the responsibility of making public places handicap accessible, it may help millions of Americans become more independent.
“AI and computer vision are improving, and people are using them to build self-driving cars and similar inventions,” Agrawal adds. “But these technologies also have the potential to improve quality of life for many people.”
The study authors came up with the idea after asking an age-old question: where do I sit?
“Imagine you’re in a café,” explains Agrawal. “You don’t want to sit just anywhere. You usually take a seat close to the walls to preserve your privacy, and you usually don’t like to sit face-to-face with a stranger.”
Previous research suggests that these simple decisions create a lot of pressure for a person who is blind or visually impaired. The study authors redesigned their lab into a makeshift cafe to see if the smart walking stick could simplify the answer to this sitting problem. This included adding several chairs, patrons, and a few obstacles normally found in cafes, like signs.
Each participant received a backpack with a laptop and a smart walking stick. They turned the stick around several times to survey the room with a camera attached near the handle. Like a self-driving car, the laptop created algorithms based on the features in the room and calculated the route to the best seat. Participants found the “right chair” in 10 out of 12 trials with varying difficulty levels.
Study authors say their smart walking cane is meant for the blind and visually impaired. However, the study used sighted people wearing blindfolds during the experiment. The researchers plan to expand the study to include visually impaired people once the cane is more reliable to use.
The cane can help people pick out groceries
Another direction the study authors are moving towards is expanding the type of scenarios people might use the cane in. In research that is not yet published, Agrawal and his co-authors had people go through grocery stores with the cane to try and find a specific item. The lab was once again made into a makeshift grocery store with shelves stocked with different cereals. The authors created a database of product photos, including boxes of Honey Nut Cheerios or Apple Jacks, into the software. The walking stick scanned the shelves to look for the specific item the user wanted.
“It assigns a score to the objects present, selecting what is the most likely product,” Agrawal says. “Then the system issues commands like ‘move a little bit to your left.”
While the invention sounds exciting, it will be a while until it becomes available to the public. The study authors hope to make some modifications, like making it sleeker and more compact. The compactness would help users attach a standard smartphone to the cane, rather than a bulky camera.
“Our aim is to make this technology mature but also attract other researchers into this field of assistive robotics,” Agrawal concludes. “We think assistive robotics has the potential to change the world.”
The study was presented at the 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems in Kyoto, Japan.
