The Paralysed Man Who Can Control a Robotic Arm With His Thoughts
Charles Liu with Erik Sorto. Image: Lance Hayashida, Caltech


This story is over 5 years old.


The Paralysed Man Who Can Control a Robotic Arm With His Thoughts

Erik Sorto is trialling out a pioneering neural implant that lets him control a robotic arm with his brain.

When Erik Sorto was 21, a gunshot wound left him paralysed from the neck down. Over ten years later, he was preparing smoothies for a room full of kids with brain injuries. For the past two years, Sorto's been trialling a neural implant that lets him move a robotic arm with only his thoughts.

He recently even used the arm to accomplish his own personal goal: taking a sip of beer by himself.

Sorto, who is now 34, is the first person in the world to have a neural prosthetic device implanted in an area of the brain called the posterior parietal cortex (PPC), where our movements are planned.


In a nutshell, the two electrode arrays implanted in Sorto's brain read his neural signals. When Sorto's implant is connected to a computer, these signals are decoded by algorithms, then translated into movement in the robot arm.

"I wanted to be part of the project and the solution," Sorto told me. "I was happy to have a small part in a project that has the potential to help so many people with disabilities gain independence."

The clinical collaboration, which is led by US-based researchers from Caltech, Keck Medicine of the University of Southern California, and Rancho Los Amigos National Rehabilitation Center, advances previous studies conducted with prosthetic devices placed in the motor cortex—an area of the brain that controls actual muscle movement. By placing the neural implant in the posterior parietal cortex, researchers hope to produce smoother motions with the robotic arm. The group's research aims to test both the safety and effectiveness of placing a neural implant in the parietal cortex to control a robotic device.

L-R Dr. Christianne Heck, Dr. Richard Andersen, Dr. Mindy Aisen, Dr. Charles Liu, Erik Sorto. Image: Lance Hayashida, Caltech

The motor cortex, said Charles Liu, the lead surgeon in the study and director of the USC Neurorestoration Center, breaks down the individual movements of any action. So if you were reaching for a glass of water, the motor cortex would tell you first to extend your elbow, open your hand, push forward your hand, and close your hand around the glass. The posterior parietal cortex, on the other hand, allows you to move more intuitively by treating the action as one combined movement.


"The advantage about this area of the brain [the parietal cortex] is that there is a much higher level of command. If you think about it in terms of the military, you have the commander-in-chief going all the way down to the soldier in the field," said Liu in a phone interview. "The motor cortex is sort of like a mid-level officer, but the parietal cortex is much higher up on that hierarchy."

Placing a neural implant into a new area of the brain, however, is no easy feat. The decision to place the device in the parietal cortex was based on previous studies, MRI imaging, and roughly a decade-worth of research from Professor of Neuroscience Richard Andersen's laboratory at Caltech.

"This hadn't been explored in use before, and Dr. Andersen described how it's a bit like a moonshot. You plan and then you put it there," said Liu.

"When I drank my own beer for the first time, I was ecstatic. For a few minutes, I forgot I was disabled."

Sorto's condition as a quadriplegic places him at higher risks when undergoing surgery. However, Liu said that for this trial the challenges were different: Firstly, the implants have never been placed in the parietal cortex before; and secondly, the implants themselves are very fragile.

"The entire array measures four by four millimetres, and has about 100 very fine microwires that are roughly 1.5 millimetres in length. If the tips of the microwires are damaged, then even if the arrays are in the right place, you might not get your recordings and that would compromise the study," explained Liu.


Sorto told me that he had no qualms before the operation. "My mum, being a mother, was nervous before the surgery, and so were my son, daughter, and cousin," he said. "Once I explained the benefits to them, and the fact that it would help out a lot of other people with disabilities, they were all for it."

Sorto is prepped for surgery. Image: Spencer Kellis, Caltech

In April 2013, Sorto had the neural prosthetic surgically implanted at the Keck Hospital of USC. "At the beginning it was challenging to get used to it. Two days after the surgery, it was painful. But to be honest, I don't even know they are there. They don't hurt or bother me at all," said Sorto. "Mostly I am reminded of them when people stare at the pedestals [the two protrusions where the neural implants come out of Sorto's skull]."

A month after surgery, Sorto started work with researchers from Caltech and staff from Rancho Los Amigos to control a computer cursor and a robotic arm with his mind. First up was a training stage, where several post-doc students from Caltech had Sorto either observe or imagine the robotic arm move. The aim was to record Sorto's neural activity and match them up with the movements of the robotic arm. This allowed the researchers to transform Sorto's neural output into a command that controls the robotic arm.

The behind-the-scenes work was also pretty extensive. "We have to design the experiments, the software that runs the experiments, and work on the algorithms that control the robotic arm," Spencer Kellis, a neural prosthetics post-doc from Andersen's Lab, told me. "That chain includes reading the neural data, and translating the neural data to having software that actually moves the arm."


The computer shows the activity of a single neuron being modulated by the subject's thoughts. Image: Caltech

But all the hard work paid off as Kellis described the sight of seeing Sorto control the robotic arm by himself for the first time as one of the coolest things he'd seen in his life.

"Erik was just able to zip the robotic arm to exactly the right orientation […] We were all just kind of giddy. It was a really really cool day," said Kellis.

Sorto told me the process was actually pretty straight forward. "The first thing I did was twist the robot's wrist. That took very little effort," he said. Sorto watched as a researcher moved their hand, and used his thoughts to replicate the motion with the robotic arm. "I was just imitating what [the researcher] was doing. I was so excited. I had this out-of-body experience that I just wanted to step out of my chair and high-five and hug everyone when we knew it worked."

So far, Sorto has learned how to point around the room, pick something up and put it elsewhere, make smoothies, do some artwork, play rock paper scissors, and his favorite: drink a beer.

"When I drank my own beer for the first time, I was ecstatic. For a few minutes, I forgot I was disabled," said Sorto. "We have challenges and difficult days. The days that I have more stress in my life, it doesn't work as well as other days."

Sorto uses the robot arm to shake hands for the first time. Image: Caltech, Keck Medicine of USC, Rancho Los Amigos National Rehabilitation Center

For their next trial, Andersen's Lab will be working on introducing a sense of touch to the neural prosthetic. "If someone is controlling a robotic arm, they can see it with their eyes so they have some feedback as to what's going on, but it's nothing like the kind of feedback that you might have from the natural sensation of touching that might come directly into the brain through the nerves if they were still functioning," said Kellis.


To reproduce the sense of touch, Kellis explained how they intended to write signals directly back into the brain in the same way as they read signals out of it. The idea is to deliver small amounts of electrical currents to the somatosensory cortex, which is the part of the brain that handles the sensory inputs that you'd receive from your skin.

"In the ideal case, the robotic arm that you control would be sensorized so that when it touches something, the touch signal would be sent back," said Kellis. "It's a dramatic change in how we approach brain-machine interfaces. Most work at this point has been about control, but we are now trying to move into both control and sensation."

The researchers also intend to explore other areas of the brain, and investigate the possibility of combining neural output from both the motor cortex and the parietal cortex. The durability of the implants is also an issue. "Erik is going on to his third year now, but what about in ten or 20 years?" asked Liu. He noted that fully implantable or wireless technologies could become a possibility in the future.

Over the phone, Sorto told me that he still missed the finer things in life. "As I'm disabled, I really miss grooming myself. I want to try and do that next. I want to try and brush my teeth, and shave," he said.

Given that this is a clinical trial, Sorto is taking part with the knowledge that though he can't take this tech home, his participation could potentially help a whole bunch of other people benefit from neural implants in the future. "Erik's generous in a way that many of us will never understand," said Kellis. "He was willing to have these electrodes implanted simply to help people who might be able to benefit from this technology somewhere down the road."

Dr Mindy Aisen, the lead rehabilitation officer at Rancho Los Amigos said that this technology could potentially help people with ALS, or who have suffered strokes. "There hasn't been very much for spinal injury that has come about in the last decade, despite all this work in stem cells. I felt it would be very good for our patients overall to see that we're never giving up in terms of new approaches to treating them," said Aisen, referring to Erik's on-site trial at Rancho Los Amigos.

The ultimate aim is also to make the technology cheaper and more widely available to people.

"My hopes for this technology for the future is for people to be more independent," said Sorto. "Personally, I would like to groom and feed myself. I hope that people can use brain computer interface technology to operate devices around their house and in their communities."

Jacked In is a series about brains and technology. Follow along here.