Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Neural population partitioning and a concurrent brain-machine interface for sequential motor function

Nature Neuroscience volume 15, pages 1715–1722 (2012)Cite this article

Subjects

Abstract

Although brain-machine interfaces (BMIs) have focused largely on performing single-targeted movements, many natural tasks involve planning a complete sequence of such movements before execution. For these tasks, a BMI that can concurrently decode the full planned sequence before its execution may also consider the higher-level goal of the task to reformulate and perform it more effectively. Using population-wide modeling, we discovered two distinct subpopulations of neurons in the rhesus monkey premotor cortex that allow two planned targets of a sequential movement to be simultaneously held in working memory without degradation. Such marked stability occurred because each subpopulation encoded either only currently held or only newly added target information irrespective of the exact sequence. On the basis of these findings, we developed a BMI that concurrently decodes a full motor sequence in advance of movement and can then accurately execute it as desired.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Task design and experimental setup.
Figure 2: Population decoding accuracy for a selected session.
Figure 3: Decoding accuracies in BMI trials.
Figure 4: Decoding and behavioral performance times.
Figure 5: Example of a second (added) target selective neuron.
Figure 6: Distribution of first and second target information across the population.
Figure 7: The effect of adding information to working memory.

Similar content being viewed by others

References

  1. Chapin, J.K., Moxon, K.A., Markowitz, R.S. & Nicolelis, M.A.L. Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex. Nat. Neurosci. 2, 664–670 (1999).

    Article  CAS  Google Scholar 

  2. Wessberg, J. et al. Real-time prediction of hand trajectory by ensembles of cortical neurons in primates. Nature 408, 361–365 (2000).

    Article  CAS  Google Scholar 

  3. Serruya, M.D., Hatsopoulos, N.G., Paninski, L., Fellows, M.R. & Donoghue, J.P. Instant neural control of a movement signal. Nature 416, 141–142 (2002).

    Article  CAS  Google Scholar 

  4. Hochberg, L.R. et al. Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442, 164–171 (2006).

    Article  CAS  Google Scholar 

  5. Carmena, J.M. et al. Learning to control a brain-machine interface for reaching and grasping by primates. PLoS Biol. 1, e42 (2003).

    Article  Google Scholar 

  6. Taylor, D.M., Tillery, S.I.H. & Schwartz, A.B. Direct cortical control of 3D neuroprosthetic devices. Science 296, 1829–1832 (2002).

    Article  CAS  Google Scholar 

  7. Ganguly, K. & Carmena, J.M. Emergence of a stable cortical map for neuroprosthetic control. PLoS Biol. 7, e1000153 (2009).

    Article  Google Scholar 

  8. Wolpaw, J.R. & McFarland, D.J. Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. Proc. Natl. Acad. Sci. USA 101, 17849–17854 (2004).

    Article  CAS  Google Scholar 

  9. Velliste, M., Perel, S., Spalding, M.C., Whitford, A.S. & Schwartz, A.B. Cortical control of a prosthetic arm for self-feeding. Nature 453, 1098–1101 (2008).

    Article  CAS  Google Scholar 

  10. Moritz, C.T., Perlmutter, S.I. & Fetz, E.E. Direct control of paralysed muscles by cortical neurons. Nature 456, 639–642 (2008).

    Article  CAS  Google Scholar 

  11. Mulliken, G.H., Musallam, S. & Andersen, R.A. Decoding trajectories from posterior parietal cortex ensembles. J. Neurosci. 28, 12913–12926 (2008).

    Article  CAS  Google Scholar 

  12. Kim, S.-P., Simeral, J.D., Hochberg, L.R., Donoghue, J.P. & Black, M.J. Neural control of computer cursor velocity by decoding motor cortical spiking activity in humans with tetraplegia. J. Neural Eng. 5, 455–476 (2008).

    Article  Google Scholar 

  13. Li, Z. et al. Unscented Kalman filter for brain-machine interfaces. PLoS ONE 4, e6243 (2009).

    Article  Google Scholar 

  14. Chase, S.M., Schwartz, A.B. & Kass, R.E. Bias, optimal linear estimation, and the differences between open-loop simulation and closed-loop performance of spiking-based brain-computer interface algorithms. Neural Netw. 22, 1203–1213 (2009).

    Article  Google Scholar 

  15. Musallam, S., Corneil, B.D., Greger, B., Scherberger, H. & Andersen, R.A. Cognitive control signals for neural prosthetics. Science 305, 258–262 (2004).

    Article  CAS  Google Scholar 

  16. Santhanam, G., Ryu, S.I., Yu, B.M., Afshar, A. & Shenoy, K.V. A high-performance brain-computer interface. Nature 442, 195–198 (2006).

    Article  CAS  Google Scholar 

  17. Shanechi, M.M., Wornell, G.W., Williams, Z.M. & Brown, E.N. Feedback-controlled parallel point process filter for estimation of goal-directed movements from neural signals. IEEE Trans. Neural Syst. Rehabil. Eng. published online, doi:10.1109/TNSRE.2012.2221743 (2 October 2012).

  18. Shanechi, M.M., Williams, Z.M., Wornell, G.W. & Brown, E.N. A brain-machine interface combining target and trajectory information using optimal feedback control. in Computational and Systems Neuroscience (COSYNE) Meeting (Salt Lake City, USA, 2011).

  19. Kurata, K. Premotor cortex of monkeys: set- and movement-related activity reflecting amplitude and direction of wrist movements. J. Neurophysiol. 69, 187–200 (1993).

    Article  CAS  Google Scholar 

  20. Messier, J. & Kalaska, J.F. Covariation of primate dorsal premotor cell activity with direction and amplitude during a memorized-delay reaching task. J. Neurophysiol. 84, 152–165 (2000).

    Article  CAS  Google Scholar 

  21. Crammond, D.J. & Kalaska, J.F. Modulation of preparatory neuronal activity in dorsal premotor cortex due to stimulus-response compatibility. J. Neurophysiol. 71, 1281–1284 (1994).

    Article  CAS  Google Scholar 

  22. Boussaoud, D. & Bremmer, F. Gaze effects in the cerebral cortex: reference frames for space coding and action. Exp. Brain Res. 128, 170–180 (1999).

    Article  CAS  Google Scholar 

  23. Crammond, D.J. & Kalaska, J.F. Differential relation of discharge in primary motor cortex and premotor cortex to movements versus actively maintained postures during a reaching task. Exp. Brain Res. 108, 45–61 (1996).

    Article  CAS  Google Scholar 

  24. Crammond, D.J. & Kalaska, J.F. Prior information in motor and premotor cortex: activity during the delay period and effect on pre-movement activity. J. Neurophysiol. 84, 986–1005 (2000).

    Article  CAS  Google Scholar 

  25. Crutcher, M.D., Russo, G.S., Ye, S. & Backus, D.A. Target-, limb-, and context-dependent neural activity in the cingulate and supplementary motor areas of the monkey. Exp. Brain Res. 158, 278–288 (2004).

    Article  CAS  Google Scholar 

  26. Hocherman, S. & Wise, S.P. Effects of hand movement path on motor cortical activity in awake, behaving rhesus monkeys. Exp. Brain Res. 83, 285–302 (1991).

    Article  CAS  Google Scholar 

  27. Batista, A.P. & Andersen, R.A. The parietal reach region codes the next planned movement in a sequential reach task. J. Neurophysiol. 85, 539–544 (2001).

    Article  CAS  Google Scholar 

  28. Ninokura, Y., Mushiake, H. & Tanji, J. Representation of the temporal order of visual objects in the primate lateral prefrontal cortex. J. Neurophysiol. 89, 2868–2873 (2003).

    Article  Google Scholar 

  29. Tanji, J. & Shima, K. Role for supplementary motor area cells in planning several movements ahead. Nature 371, 413–416 (1994).

    Article  CAS  Google Scholar 

  30. Shima, K., Isoda, M., Mushiake, H. & Tanji, J. Categorization of behavioural sequences in the prefrontal cortex. Nature 445, 315–318 (2007).

    Article  CAS  Google Scholar 

  31. Shima, K. & Tanji, J. Neuronal activity in the supplementary and presupplementary motor areas for temporal organization of multiple movements. J. Neurophysiol. 84, 2148–2160 (2000).

    Article  CAS  Google Scholar 

  32. Baldauf, D., Cui, H. & Andersen, R.A. The posterior parietal cortex encodes in parallel both goals for double-reach sequences. J. Neurosci. 28, 10081–10089 (2008).

    Article  CAS  Google Scholar 

  33. Averbeck, B.B., Sohn, J.-W. & Lee, D. Activity in prefrontal cortex during dynamic selection of action sequences. Nat. Neurosci. 9, 276–282 (2006).

    Article  CAS  Google Scholar 

  34. Mushiake, H., Saito, N., Sakamoto, K., Itoyama, Y. & Tanji, J. Activity in the lateral prefrontal cortex reflects multiple steps of future events in action plans. Neuron 50, 631–641 (2006).

    Article  CAS  Google Scholar 

  35. Ohbayashi, M., Ohki, K. & Miyashita, Y. Conversion of working memory to motor sequence in the monkey premotor cortex. Science 301, 233–236 (2003).

    Article  CAS  Google Scholar 

  36. Kettner, R.E., Marcario, J.K. & Port, N.L. Control of remembered reaching sequences in monkey. II. Storage and preparation before movement in motor and premotor cortex. Exp. Brain Res. 112, 347–358 (1996).

    Article  CAS  Google Scholar 

  37. Lu, X. & Ashe, J. Anticipatory activity in primary motor cortex codes memorized movement sequences. Neuron 45, 967–973 (2005).

    Article  CAS  Google Scholar 

  38. Nakajima, T., Hosaka, R., Mushiake, H. & Tanji, J. Covert representation of second-next movement in the pre-supplementary motor area of monkeys. J. Neurophysiol. 101, 1883–1889 (2009).

    Article  Google Scholar 

  39. Averbeck, B.B., Chafee, M.V., Crowe, D.A. & Georgopoulos, A.P. Parallel processing of serial movements in prefrontal cortex. Proc. Natl. Acad. Sci. USA 99, 13172–13177 (2002).

    Article  CAS  Google Scholar 

  40. Saito, N., Mushiake, H., Sakamoto, K., Itoyama, Y. & Tanji, J. Representation of immediate and final behavioral goals in the monkey prefrontal cortex during an instructed delay period. Cereb. Cortex 15, 1535–1546 (2005).

    Article  Google Scholar 

  41. Mushiake, H., Inase, M. & Tanji, J. Selective coding of motor sequence in the supplementary motor area of the monkey cerebral cortex. Exp. Brain Res. 82, 208–210 (1990).

    Article  CAS  Google Scholar 

  42. Smith, A.C. et al. State-space algorithms for estimating spike rate functions. Comput. Intell. Neurosci. published online, doi:10.1155/2010/426539 (5 November 2009).

  43. Tanji, J. Sequential organization of multiple movements: involvement of cortical motor areas. Annu. Rev. Neurosci. 24, 631–651 (2001).

    Article  CAS  Google Scholar 

  44. Cisek, P. & Kalaska, J.F. Neural correlates of reaching decisions in dorsal premotor cortex: specification of multiple direction choices and final selection of action. Neuron 45, 801–814 (2005).

    Article  CAS  Google Scholar 

  45. Hoshi, E. & Tanji, J. Integration of target and body-part information in the premotor cortex when planning action. Nature 408, 466–470 (2000).

    Article  CAS  Google Scholar 

  46. Hoshi, E. & Tanji, J. Contrasting neuronal activity in the dorsal and ventral premotor areas during preparation to reach. J. Neurophysiol. 87, 1123–1128 (2002).

    Article  Google Scholar 

  47. Hoshi, E. & Tanji, J. Differential involvement of neurons in the dorsal and ventral premotor cortex during processing of visual signals for action planning. J. Neurophysiol. 95, 3596–3616 (2006).

    Article  Google Scholar 

  48. Shima, K. & Tanji, J. Both supplementary and presupplementary motor areas are crucial for the temporal organization of multiple movements. J. Neurophysiol. 80, 3247–3260 (1998).

    Article  CAS  Google Scholar 

  49. Brown, E.N., Barbieri, R., Eden, U.T. & Frank, L.M. Likelihood methods for neural data analysis. in Computational Neuroscience: A Comprehensive Approach (ed. Feng, J.) 253–286 (CRC Press, 2003).

  50. Truccolo, W., Eden, U.T., Fellows, M.R., Donoghue, J.P. & Brown, E.N. A point process framework for relating neural spiking activity to spiking history, neural ensemble, and extrinsic covariate effects. J. Neurophysiol. 93, 1074–1089 (2005).

    Article  Google Scholar 

  51. Smith, A.C. & Brown, E.N. Estimating a state-space model from point process observations. Neural Comput. 15, 965–991 (2003).

    Article  Google Scholar 

  52. Dempster, A.P., Laird, N.M. & Rubin, D.B. Maximum likelihood from incomplete data via the EM algorithm. J. Royal Stat. Soc., Series B 39, 1–38 (1977).

    Google Scholar 

  53. Goodman, L.A. On the exact variance of products. J. Am. Stat. Assoc. 55, 708–713 (1960).

    Article  Google Scholar 

Download references

Acknowledgements

R.C.H. is funded by the Neuroscience Research and Education Foundation, E.N.B. is funded by US National Institutes of Health (NIH) DP1 OD003646, and Z.M.W. is funded by NIH 5R01-HD059852, a Presidential Early Career Award for Scientists and Engineers and the Whitehall Foundation.

Author information

Authors and Affiliations

  1. Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA

    Maryam M Shanechi & Gregory W Wornell

  2. Department of Neurosurgery, Massachusetts General Hospital, Boston, Massachusetts, USA

    Maryam M Shanechi, Rollin C Hu, Marissa Powers & Ziv M Williams

  3. Harvard Medical School, Boston, Massachusetts, USA

    Maryam M Shanechi, Rollin C Hu, Emery N Brown & Ziv M Williams

  4. Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA

    Emery N Brown

  5. Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Boston, Massachusetts, USA

    Emery N Brown

  6. Institute for Medical Engineering and Science, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA

    Emery N Brown

  7. Harvard-MIT Health Sciences and Technology Program, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA

    Emery N Brown

Authors
  1. Maryam M Shanechi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rollin C Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marissa Powers
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Gregory W Wornell
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Emery N Brown
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ziv M Williams
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

M.M.S. developed the BMI real-time decoder, conceived and performed the computational analysis, assisted with animal recordings and wrote the manuscript. R.C.H. and M.P. assisted with animal training and recordings. G.W.W. and E.N.B. were involved in the computational methodological development and writing of the manuscript. Z.M.W. conceived and designed the study, developed the BMI system, performed the animal training and recordings, helped implement the computational models and wrote the manuscript.

Corresponding author

Correspondence to Ziv M Williams.

Ethics declarations

Competing interests

The authors declare no competing financial interests.

Supplementary information

Supplementary Text and Figures

Supplementary Figures 1–10 and Supplementary Modeling (PDF 1033 kb)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Shanechi, M., Hu, R., Powers, M. et al. Neural population partitioning and a concurrent brain-machine interface for sequential motor function. Nat Neurosci 15, 1715–1722 (2012). https://doi.org/10.1038/nn.3250

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nn.3250

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing