Mindful movement: New mobility tech target the brain — not our limbs

Johnson’s device, called MyHand, looks a bit like one of those coin-operated binoculars you find at tourist sites — it’s a trapezoidal box mounted on a frame. You place your hand inside the box, inserting each finger into a thimble-like cup.

On top of the device, there’s a tablet, which is loaded with games, each requiring precise finger movements. The thimble-sized cups are the game controllers: they measure the user’s finger movements and send the data back to the tablet. They also provide resistance. As users regain mobility, the cups push back, making the tasks more challenging. “If you work out with a multigym, you can strengthen your pecs, shoulder muscles or hamstrings,” says Johnson. “There’s nothing like this available for fingers.”

During the first few weeks, Leblanc didn’t think he was moving his fingers at all. Only the machine could detect his progress — measured, at the time, in millimetres. After the initial sessions, Leblanc felt incredibly fatigued. But the tiredness wasn’t in his fingers; it was in his brain. “I was so exhausted,” Leblanc recalls. “I needed a nap.”

This all makes sense. The MyHand device isn’t a prosthetic. It’s a neurological workout routine in a box. And it’s one of a new cohort of leading-edge mobility technologies focused on the most important organ in the human motor system — the one beneath our skulls.

Where movement happens

We tend to assume that movement resides in our limbs and extremities: We walk with our legs, lift with our arms and grip with our fingers. But it’s probably better to say that we walk, lift and grip with our brains. That’s where the complex cognitive processes (the real work!) of movement occurs. That’s why a stroke, a type of brain injury, has such profound effects on mobility. A limb without a brain is like an airplane without a pilot.

For most of industrial history, prosthetics were just replica body parts, like the iron arm worn by 16th-century sea captain François de la Noue or the steel-and-leather leg braces belonging to U.S. president Franklin D. Roosevelt. The typical prosthetic on the market today is a bit more complex — most arm prosthetics, for instance, use a system of pullies, operated by harnesses on the user’s back — but they’re still basically inert. They don’t engage with the cognitive work that is human movement. 

Prosthetics that think

In the future, the science of prosthetics and assisted mobility will likely be inseparable from the science of cognition. Brokoslaw Laschowski is a scientist and principal investigator at the Neural Robotics Lab, at the KITE Research Institute in Toronto. His business is mobility recovery, but he doesn’t make prosthetics; he makes software. “We want to enable robots, like prosthetic legs, to think, see and move like humans,” he says.

There are two possible ways to do this. You can create prosthetics that figure out what their wearers are thinking. Or you can create prosthetics that think for themselves. Laschowski is building the algorithms necessary for both.

Devices in the former category need to somehow link up with the user’s brain. Surgical brain implants are one way to achieve this feat. (The company most invested in such research is California’s Neuralink, founded by Elon Musk.) But there might be less invasive methods, Laschowski says. An EEG cap, for instance, can pick up on brain signals — albeit fuzzily — through a user’s skull, and “myo-electrical sensors,” attached elsewhere on the body, can decode information about what the brain is trying to do.

When you open and close your hands, you also flex the muscles in your forearm. Those flexes are signs of brain intention. A user without a hand could install an electrode near their wrist, which would pick up the muscle movement and send a “flex” command to a prosthetic. Such myo-electrical sensors already exist today for arms, although there’s nothing yet available for legs. Refining the technology for walking is one of Laschowski’s key priorities. He is creating “neuro-decoding algorithms” — software programs that interpret a user’s mobility intentions based on muscle signals elsewhere on the body.

He’s also creating autonomous prosthetics. When you go for a stroll, he says, the whole point is to let your mind wander; you can’t be repeatedly issuing “walk” commands to your leg. That’s why people might want legs that walk themselves. “Robotic prosthetic legs can use cameras and machine learning to control themselves, similar to self-driving cars,” he says. “We’re designing the artificial brains, the algorithms to understand visual data from the real world.”

This technology, he says, will extend the benefits of mobility to people who would otherwise need a wheelchair to get around. “Giving people the ability to walk can have profound impacts on their quality of life,” Laschowski says. “We often don’t appreciate the independence afforded by autonomous movement.”

His lab isn’t the only player in this game. Inspired by his studies at Ontario Tech University in Oshawa, Hamayal Choudhry founded smartARM in 2018 with the aim of creating the world’s first fully autonomous arm. The startup recently unveiled a smart prosthetic that’s outfitted with a digital camera located in the palm, which reads its surroundings. It can distinguish an item you’re likely to pick up, like a pen or toothbrush, from one you’re likely to leave alone, like a floor lamp. It also has biosensors, which glean a user’s intentions based on their movements. If you move toward a TV remote, you’re probably hoping to use it; if you move past it, you probably aren’t.

The smartARM is powered by AI, which means it has the capacity to learn. By clicking a button on the side of the arm, you can engage in “imitation mode.” Then, using your opposing arm, you can demonstrate how to perform a task, like watering plants or setting a table. “The camera picks up what you’re doing with your other hand and mimics that motion,” says Choudhry. “If you don’t like the way the smartARM holds a pen, you can teach it the proper angles.”

The device will also feed data back to a central smartARM server, enabling the company to refine its algorithms — and, ultimately, to get better at intuiting natural human movement. “Instead of having the user conform to the device,” says Choudhry, “we want the device to conform to the user.”

Learning by doing

Unlike motor injuries, neurological ailments such as strokes, Parkinson’s or multiple sclerosis don’t cause limb damage — the extremities are still intact, despite limited function. And if it’s possible to train a robot prosthetic to think like a human, it’s surely possible to train a human brain to regain capacities it once had. This is what Johnson aims to do. “MyHand is a therapeutic device,” he says.

The workout itself is varied — the device has 11 different games with multiple settings and levels — because our hands, and the jobs they perform, are complex, too. From typing to fastening your watch strap to cutting your food, every action requires a suite of highly differentiated yet co-ordinated movements. “If you think about holding a mug versus a Timmy’s coffee cup,” says Johnson, “the way you squeeze it is different. Those intricacies are the things we’re trying to train. We take the physical and cognitive actions associated with finger movements and split them into countless bite-sized steps.”

After nearly half a year with the device, Leblanc’s work started to pay off. One day, he found he could retrieve knives and forks from the cutlery drawer. A few months later, he threaded a button on his shirt for the first time in almost a decade. Recently, he’s gotten back into fishing. He can tie knots and bait a hook himself. He’s tempted to say that he’s got a working set of fingers again, but he knows the truth is more complicated. “I’ve forged new neural pathways in my brain,” he says.

Simon Lewsen writes about technology for MaRS. Torstar, the parent company of the Toronto Star, has partnered with MaRS to highlight innovation in Canadian companies.