A high-performance speech neuroprosthesis

  • Nature. 2023 Aug;620(7976):1031-1036. doi: 10.1038/s41586-023-06377-x.
Francis R Willett  #  1 ,  Erin M Kunz  #  2  3 ,  Chaofei Fan  #  4 ,  Donald T Avansino  5 ,  Guy H Wilson  6 ,  Eun Young Choi  7 ,  Foram Kamdar  7 ,  Matthew F Glasser  8  9 ,  Leigh R Hochberg  10  11  12 ,  Shaul Druckmann  13 ,  Krishna V Shenoy  5  2  3  13  14  15 ,  Jaimie M Henderson  3  7
Affiliations
  • 1. Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA. [email protected].
  • 2. Department of Electrical Engineering, Stanford University, Stanford, CA, USA.
  • 3. Wu Tsai Neurosciences Institute, Stanford University, Stanford, CA, USA.
  • 4. Department of Computer Science, Stanford University, Stanford, CA, USA.
  • 5. Howard Hughes Medical Institute at Stanford University, Stanford, CA, USA.
  • 6. Department of Neuroscience, Stanford University, Stanford, CA, USA.
  • 7. Department of Neurosurgery, Stanford University, Stanford, CA, USA.
  • 8. Department of Neuroscience, Washington University in St. Louis, St. Louis, MO, USA.
  • 9. Department of Radiology, Washington University in St. Louis, St. Louis, MO, USA.
  • 10. VA RR&D Center for Neurorestoration and Neurotechnology, Rehabilitation R&D Service, Providence VA Medical Center, Providence, RI, USA.
  • 11. School of Engineering and Carney Institute for Brain Science, Brown University, Providence, RI, USA.
  • 12. Center for Neurotechnology and Neurorecovery, Department of Neurology, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA.
  • 13. Department of Neurobiology, Stanford University, Stanford, CA, USA.
  • 14. Department of Bioengineering, Stanford University, Stanford, CA, USA.
  • 15. Bio-X Program, Stanford University, Stanford, CA, USA.
  • # Contributed equally.
PMID: 37612500 DOI: 10.1038/s41586-023-06377-x
Abstract

Speech brain-computer interfaces (BCIs) have the potential to restore rapid communication to people with paralysis by decoding neural activity evoked by attempted speech into text1,2 or sound3,4. Early demonstrations, although promising, have not yet achieved accuracies sufficiently high for communication of unconstrained sentences from a large vocabulary1-7. Here we demonstrate a speech-to-text BCI that records spiking activity from intracortical microelectrode arrays. Enabled by these high-resolution recordings, our study participant-who can no longer speak intelligibly owing to amyotrophic lateral sclerosis-achieved a 9.1% word error rate on a 50-word vocabulary (2.7 times fewer errors than the previous state-of-the-art speech BCI2) and a 23.8% word error rate on a 125,000-word vocabulary (the first successful demonstration, to our knowledge, of large-vocabulary decoding). Our participant's attempted speech was decoded at 62 words per minute, which is 3.4 times as fast as the previous record8 and begins to approach the speed of natural conversation (160 words per minute9). Finally, we highlight two aspects of the neural code for speech that are encouraging for speech BCIs: spatially intermixed tuning to speech articulators that makes accurate decoding possible from only a small region of cortex, and a detailed articulatory representation of phonemes that persists years after paralysis. These results show a feasible path forward for restoring rapid communication to people with paralysis who can no longer speak.