A robotic arm with a sense of touch has enabled a paralyzed man to perform tasks such as pouring water from one cup into another. A team reports in the journal Science that the robotic arm provides tactile feedback directly to the man’s brain while he controls the device with his thoughts.
Most able-bodied people take for granted their ability to perform simple daily tasks—when they reach for a warm mug of coffee, they can feel its weight and temperature and adjust their grip accordingly so that no liquid spills. People who have complete sensory and motor control of their arms and hands can feel that they have made contact with an object the moment they touch or grasp it, allowing them to move or lift it with confidence. However, when a person operates a prosthetic arm, let alone a mind-controlled one, those tasks become much more difficult.
A team of bioengineers from the University of Pittsburgh Rehab Neural Engineering Labs describe how adding brain stimulation that evokes tactile sensations makes it easier for the operator to manipulate a brain-controlled robotic arm in a paper published in Science. In the experiment, augmenting vision with artificial tactile perception cut the time spent grasping and transferring objects in half, from 20.9 to 10.2 seconds on average.
The robotic arm provides tactile feedback directly to the man’s brain as he uses his thoughts to control the device. Adding a BCI that evokes tactile sensations makes it easier for users to manipulate and use robotic arm prosthetics.
“In a sense, this is what we hoped would happen—but perhaps not to the extent that we observed,” said Jennifer Collinger, Ph.D., associate professor in Pitt’s Department of Physical Medicine and Rehabilitation. “Sensory feedback from limbs and hands is crucial for doing normal things in our daily lives, and when that feedback is missing, people’s performance suffers.”
Nathan Copeland, whose progress is described in the paper, is the first person in the world to have tiny electrode arrays implanted not only in his motor cortex but also in his somatosensory cortex—a region of the brain that processes sensory information from the body. Arrays enable him to not only control the robotic arm with his mind but also receive tactile sensory feedback, which is similar to how neural circuits function when a person’s spinal cord is intact.
“I was already very familiar with the sensations produced by stimulation as well as performing the task without stimulation. Even though the sensation isn’t ‘natural,’ it feels like pressure and a gentle tingle, which didn’t bother Copeland. “There was never a point where I felt like I needed to get used to the stimulation. Working on the task while receiving stimulation just seemed to go together like PB&J.”
Copeland enrolled in a clinical trial testing the sensorimotor microelectrode brain-computer interface (BCI) after a car accident left him with limited use of his arms. He was implanted with four microelectrode arrays developed by Blackrock Microsystems (also commonly referred to as Utah arrays).
This paper builds on a previous study that described for the first time how stimulating sensory regions of the brain with tiny electrical pulses can evoke sensation in distinct regions of a person’s hand, even if they have lost feeling in their limbs due to spinal cord injury. The researchers combined reading information from the brain to control the robotic arm’s movement with writing information back in to provide sensory feedback in this new study.
In a series of tests in which the BCI operator was asked to pick up and transfer various objects from a table to a raised platform, providing tactile feedback via electrical stimulation enabled the participant to complete tasks twice as quickly as tests without stimulation. The researchers wanted to test the effect of sensory feedback in conditions that were as close to the real world as possible in their new paper.
“We didn’t want to limit the task by removing the visual component of perception,” said co-senior author Robert Gaunt, Ph.D., associate professor in Pitt’s Department of Physical Medicine and Rehabilitation. “When even limited and imperfect sensation is restored, the person’s performance improves noticeably. We still have a long way to go in terms of making the sensations more realistic and bringing this technology into people’s homes, but the closer we can get to recreating normal brain inputs, the better off we will be.”
