Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Article
  • Published:

Neurons detect cognitive boundaries to structure episodic memories in humans

Nature Neuroscience volume 25, pages 358–368 (2022)Cite this article

Subjects

Abstract

While experience is continuous, memories are organized as discrete events. Cognitive boundaries are thought to segment experience and structure memory, but how this process is implemented remains unclear. We recorded the activity of single neurons in the human medial temporal lobe (MTL) during the formation and retrieval of memories with complex narratives. Here, we show that neurons responded to abstract cognitive boundaries between different episodes. Boundary-induced neural state changes during encoding predicted subsequent recognition accuracy but impaired event order memory, mirroring a fundamental behavioral tradeoff between content and time memory. Furthermore, the neural state following boundaries was reinstated during both successful retrieval and false memories. These findings reveal a neuronal substrate for detecting cognitive boundaries that transform experience into mnemonic episodes and structure mental time travel during retrieval.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Experiment and recording locations.
Fig. 2: Behavior.
Fig. 3: Boundary cells and event cells demarcate different types of episodic transitions.
Fig. 4: Responses of boundary cells and event cells during encoding correlate with later retrieval success.
Fig. 5: Population neural state shift magnitude following episodic transitions reflects participants’ subsequent memory performance.
Fig. 6: Reinstatement of neural context after boundaries during recognition.

Similar content being viewed by others

Data availability

The data (in NWB format) that supports the key findings of this study are publicly available on the DANDI archive (https://doi.org/10.48324/dandi.000207/0.220216.0323).

Code availability

Codes that support the key findings of this study are publicly available on GitHub (https://github.com/rutishauserlab/cogboundary-zheng).

References

  1. Ezzyat, Y. & Davachi, L. What constitutes an episode in episodic memory? Psychol. Sci. 22, 243–252 (2011).

    Article  PubMed  Google Scholar 

  2. Tulving, E. Episodic memory: from mind to brain. Annu. Rev. Psychol. 53, 1–25 (2002).

    Article  PubMed  Google Scholar 

  3. Radvansky, G. A. & Zacks, J. M. Event Cognition (Oxford University Press, 2014).

  4. Desimone, R. & Duncan, J. Neural mechanisms of selective visual attention. Annu. Rev. Neurosci. 18, 193–222 (1995).

    Article  CAS  PubMed  Google Scholar 

  5. Zacks, J. M. et al. Human brain activity time-locked to perceptual event boundaries. Nat. Neurosci. 4, 651–655 (2001).

    Article  CAS  PubMed  Google Scholar 

  6. Avrahami, J. & Kareev, Y. The emergence of events. Cognition 53, 239–261 (1994).

    Article  CAS  PubMed  Google Scholar 

  7. DuBrow, S. & Davachi, L. The influence of context boundaries on memory for the sequential order of events. J. Exp. Psychol. Gen. 142, 1277–1286 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  8. Chen, J. et al. Shared memories reveal shared structure in neural activity across individuals. Nat. Neurosci. 20, 115–125 (2017).

    Article  CAS  PubMed  Google Scholar 

  9. Kurby, C. A. & Zacks, J. M. Segmentation in the perception and memory of events. Trends Cogn. Sci. 12, 72–79 (2008).

    Article  PubMed  PubMed Central  Google Scholar 

  10. Zacks, J. M., Speer, N. K., Swallow, K. M., Braver, T. S. & Reynolds, J. R. Event perception: a mind–brain perspective. Psychol. Bull. 133, 273–293 (2007).

    Article  PubMed  PubMed Central  Google Scholar 

  11. Jafarpour, A., Griffin, S., Lin, J. J. & Knight, R. T. Medial orbitofrontal cortex, dorsolateral prefrontal cortex, and hippocampus differentially represent the event saliency. J. Cogn. Neurosci. 31, 874–884 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  12. Ben-Yakov, A. & Henson, R. N. The hippocampal film editor: sensitivity and specificity to event boundaries in continuous experience. J. Neurosci. 38, 10057–10068 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Baldassano, C. et al. Discovering event structure in continuous narrative perception and memory. Neuron 95, 709–721 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Spiers, H. J., Hayman, R. M., Jovalekic, A., Marozzi, E. & Jeffery, K. J. Place field repetition and purely local remapping in a multicompartment environment. Cereb. Cortex 25, 10–25 (2015).

    Article  PubMed  Google Scholar 

  15. Derdikman, D. et al. Fragmentation of grid cell maps in a multicompartment environment. Nat. Neurosci. 12, 1325–1332 (2009).

    Article  CAS  PubMed  Google Scholar 

  16. Lever, C., Burton, S., Jeewajee, A., O’Keefe, J. & Burgess, N. Boundary vector cells in the subiculum of the hippocampal formation. J. Neurosci. 29, 9771–9777 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  17. O’Keefe, J. & Burgess, N. Geometric determinants of the place fields of hippocampal neurons. Nature 381, 425–428 (1996).

    Article  PubMed  Google Scholar 

  18. Alme, C. B. et al. Place cells in the hippocampus: eleven maps for eleven rooms. Proc. Natl Acad. Sci. USA 111, 18428–18435 (2014).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Colgin, L. L., Moser, E. I. & Moser, M. B. Understanding memory through hippocampal remapping. Trends Neurosci. 31, 469–477 (2008).

    Article  CAS  PubMed  Google Scholar 

  20. Grieves, R. M., Jenkins, B. W., Harland, B. C., Wood, E. R. & Dudchenko, P. A. Place field repetition and spatial learning in a multicompartment environment. Hippocampus 26, 118–134 (2016).

    Article  PubMed  Google Scholar 

  21. Sun, C., Yang, W., Martin, J. & Tonegawa, S. Hippocampal neurons represent events as transferable units of experience. Nat. Neurosci. 23, 651–663 (2020).

    Article  CAS  PubMed  Google Scholar 

  22. Levine, B. et al. Episodic memory and the self in a case of isolated retrograde amnesia. Brain 121, 1951–1973 (1998).

    Article  PubMed  Google Scholar 

  23. Rutishauser, U. Testing models of human declarative memory at the single-neuron level. Trends Cogn. Sci. 23, 510–524 (2019).

    Article  PubMed  PubMed Central  Google Scholar 

  24. Rutishauser, U., Ross, I. B., Mamelak, A. N. & Schuman, E. M. Human memory strength is predicted by theta-frequency phase-locking of single neurons. Nature 464, 903–907 (2010).

    Article  CAS  PubMed  Google Scholar 

  25. Pacheco Estefan, D. et al. Coordinated representational reinstatement in the human hippocampus and lateral temporal cortex during episodic memory retrieval. Nat. Commun. 10, 2255 (2019).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Manning, J. R., Polyn, S. M., Baltuch, G. H., Litt, B. & Kahana, M. J. Oscillatory patterns in temporal lobe reveal context reinstatement during memory search. Proc. Natl Acad. Sci. USA 108, 12893–12897 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Folkerts, S., Rutishauser, U. & Howard, M. W. Human episodic memory retrieval is accompanied by a neural contiguity effect. J. Neurosci. 38, 4200–4211 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  28. Howard, M. W., Fotedar, M. S., Datey, A. V. & Hasselmo, M. E. The temporal context model in spatial navigation and relational learning: toward a common explanation of medial temporal lobe function across domains. Psychol. Rev. 112, 75–116 (2005).

    Article  PubMed  PubMed Central  Google Scholar 

  29. Swallow, K. M., Zacks, J. M. & Abrams, R. A. Event boundaries in perception affect memory encoding and updating. J. Exp. Psychol. Gen. 138, 236–257 (2009).

    Article  PubMed  PubMed Central  Google Scholar 

  30. Richmond, L. L., Gold, D. A. & Zacks, J. M. Event perception: translations and applications. J. Appl. Res. Mem. Cogn. 6, 111–120 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  31. Ben-Yakov, A. & Dudai, Y. Constructing realistic engrams: poststimulus activity of hippocampus and dorsal striatum predicts subsequent episodic memory. J. Neurosci. 31, 9032–9042 (2011).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Ben-Yakov, A., Eshel, N. & Dudai, Y. Hippocampal immediate poststimulus activity in the encoding of consecutive naturalistic episodes. J. Exp. Psychol. Gen. 142, 1255–1263 (2013).

    Article  PubMed  Google Scholar 

  33. Isik, L., Singer, J., Madsen, J. R., Kanwisher, N. & Kreiman, G. What is changing when: decoding visual information in movies from human intracranial recordings. Neuroimage 180, 147–159 (2018).

    Article  PubMed  Google Scholar 

  34. Aminoff, E. M., Kveraga, K. & Bar, M. The role of the parahippocampal cortex in cognition. Trends Cogn. Sci. 17, 379–390 (2013).

    Article  PubMed  PubMed Central  Google Scholar 

  35. Lisman, J. E. & Grace, A. A. The hippocampal-VTA loop: controlling the entry of information into long-term memory. Neuron 46, 703–713 (2005).

    Article  CAS  PubMed  Google Scholar 

  36. Vinogradova, O. S. Hippocampus as comparator: role of the two input and two output systems of the hippocampus in selection and registration of information. Hippocampus 11, 578–598 (2001).

    Article  CAS  PubMed  Google Scholar 

  37. Solstad, T., Boccara, C. N., Kropff, E., Moser, M. B. & Moser, E. I. Representation of geometric borders in the entorhinal cortex. Science 322, 1865–1868 (2008).

    Article  CAS  PubMed  Google Scholar 

  38. Xiao, X. et al. Transformed neural pattern reinstatement during episodic memory retrieval. J. Neurosci. 37, 2986–2998 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Favila, S. E., Samide, R., Sweigart, S. C. & Kuhl, B. A. Parietal representations of stimulus features are amplified during memory retrieval and flexibly aligned with top–down goals. J. Neurosci. 38, 7809–7821 (2018).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Jang, A. I., Wittig, J. H. Jr., Inati, S. K. & Zaghloul, K. A. Human cortical neurons in the anterior temporal lobe reinstate spiking activity during verbal memory retrieval. Curr. Biol. 27, 1700–1705 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  41. Howard, M. W. & Natu, V. S. Place from time: reconstructing position from a distributed representation of temporal context. Neural Netw. 18, 1150–1162 (2005).

    Article  PubMed  PubMed Central  Google Scholar 

  42. Polyn, S. M. & Kahana, M. J. Memory search and the neural representation of context. Trends Cogn. Sci. 12, 24–30 (2008).

    Article  PubMed  Google Scholar 

  43. Krizhevsky, A., Sutskever, I. & Hinton, G. E. in NIPS’12: Proceedings of the 25th International Conference on Neural Information Processing Systems, Vol. 1 (eds Bartlett, P., Pereira, F. C. N., Burges, C. J. C., Bottoue, L., & Weinberger K. Q.) 1097–1105 (Morgan Kaufmann, 2012).

  44. Rutishauser, U., Schuman, E. M. & Mamelak, A. N. Online detection and sorting of extracellularly recorded action potentials in human medial temporal lobe recordings, in vivo. J. Neurosci. Methods 154, 204–224 (2006).

    Article  PubMed  Google Scholar 

  45. Fried, I., Rutishauser, U., Cerf, M. & Kreiman, G. Single Neuron Studies of the Human Brain: Probing Cognition (The MIT Press, 2014).

  46. Kaminski, J. et al. Persistently active neurons in human medial frontal and medial temporal lobe support working memory. Nat. Neurosci. 20, 590–601 (2017).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  47. Pouzat, C., Mazor, O. & Laurent, G. Using noise signature to optimize spike-sorting and to assess neuronal classification quality. J. Neurosci. Methods 122, 43–57 (2002).

    Article  PubMed  Google Scholar 

  48. Harris, K. D., Henze, D. A., Csicsvari, J., Hirase, H. & Buzsaki, G. Accuracy of tetrode spike separation as determined by simultaneous intracellular and extracellular measurements. J. Neurophysiol. 84, 401–414 (2000).

    Article  CAS  PubMed  Google Scholar 

  49. Reuter, M., Rosas, H. D. & Fischl, B. Highly accurate inverse consistent registration: a robust approach. Neuroimage 53, 1181–1196 (2010).

    Article  PubMed  Google Scholar 

  50. Pauli, W. M., Nili, A. N. & Tyszka, J. M. A high-resolution probabilistic in vivo atlas of human subcortical brain nuclei. Sci. Data 5, 180063 (2018).

    Article  PubMed  PubMed Central  Google Scholar 

  51. Avants, B. et al. Multivariate analysis of structural and diffusion imaging in traumatic brain injury. Acad. Radiol. 15, 1360–1375 (2008).

    Article  PubMed  PubMed Central  Google Scholar 

  52. Banaie Boroujeni, K., Tiesinga, P. & Womelsdorf, T. Adaptive spike-artifact removal from local field potentials uncovers prominent beta and gamma band neuronal synchronization. J. Neurosci. Methods 330, 108485 (2020).

    Article  PubMed  Google Scholar 

  53. Oostenveld, R., Fries, P., Maris, E. & Schoffelen, J. M. FieldTrip: open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data. Comput Intell. Neurosci. 2011, 156869 (2011).

    Article  PubMed  Google Scholar 

  54. Delorme, A. & Makeig, S. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 134, 9–21 (2004).

    Article  PubMed  Google Scholar 

  55. Kobak, D. et al. Demixed principal component analysis of neural population data. eLife 5, e10989 (2016)

  56. Bausch, M. et al. Concept neurons in the human medial temporal lobe flexibly represent abstract relations between concepts. Nat. Commun. 12, 6164 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank C. Katz and K. Patel for helping set up the recording system for single-unit recordings at Toronto Western Hospital, N. Chandravadia and V. Barkely for data transferring and organization, C. Reed, J. Chung and the clinical teams at both Cedars-Sinai Medical Center and Toronto Western Hospital and M. Zhang, J. Kaminski and other members of the Rutishauser and Kreiman labs for discussion. We are especially indebted to the volunteers who participated in this study. This work was supported by NIH U01NS103792 and U01NS117839 (to U.R.), NSF 1231216 (G.K.) and Brain Canada (to T.A.V.). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.

Author information

Authors and Affiliations

  1. Department of Ophthalmology, Children’s Hospital, Harvard Medical School, Boston, MA, USA

    Jie Zheng & Gabriel Kreiman

  2. Krembil Brain Institute and Division of Neurosurgery, University Health Network (UHN), University of Toronto, Toronto, Ontario, Canada

    Andrea G. P. Schjetnan, Suneil K. Kalia & Taufik A. Valiante

  3. Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA

    Mar Yebra, Bernard A. Gomes, Clayton P. Mosher, Adam N. Mamelak & Ueli Rutishauser

  4. Department of Surgery (Neurosurgery), Institute of Biomedical Engineering, and Electrical and Computer Engineering, University of Toronto, Toronto, Ontario, Canada

    Taufik A. Valiante

  5. Max Planck-University of Toronto Center for Neural Science and Technology, University of Toronto, Toronto, Ontario, Canada

    Taufik A. Valiante

  6. Center for Advancing Neurotechnological Innovation to Application, University Health Network, University of Toronto, Toronto, Ontario, Canada

    Taufik A. Valiante

  7. Center for Brains, Minds and Machines, Cambridge, MA, USA

    Gabriel Kreiman

  8. Department of Neurology, Cedars-Sinai Medical Center, Los Angeles, CA, USA

    Ueli Rutishauser

  9. Center for Neural Science and Medicine, Department of Biomedical Sciences, Cedars-Sinai Medical Center, Los Angeles, CA, USA

    Ueli Rutishauser

  10. Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA

    Ueli Rutishauser

Authors
  1. Jie Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrea G. P. Schjetnan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mar Yebra
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bernard A. Gomes
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Clayton P. Mosher
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Suneil K. Kalia
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Taufik A. Valiante
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Adam N. Mamelak
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Gabriel Kreiman
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Ueli Rutishauser
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

J.Z. conceived the project. J.Z., G.K. and U.R. contributed ideas for experiments and analysis. S.K.K., T.A.V. and A.N.M. managed participants and surgeries. J.Z., A.G.P.S., M. Y. and C.P.M. collected data. J.Z. performed the analyses. B.A.G. performed electrode localization. B.A.G., J.Z. and U.R. and produced the Neurodata Without Borders (NWB) formatted dataset for public release. J.Z., G.K. and U.R. wrote the manuscript with input from all authors.

Corresponding authors

Correspondence to Gabriel Kreiman or Ueli Rutishauser.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Neuroscience thanks Christopher Baldassano and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Extended data

Extended Data Fig. 1 Electrode locations in MNI coordinates, Related to Fig. 1.

a-c, Each dot is the location of a microwire bundle in either the amygdala (cyan), hippocampus (yellow) or parahippocampus (red) on which at least one event or boundary cell was recorded, also presented in a template brain in Fig. 1e. Coordinates are in Montreal Neurological Institute (MNI) 152 space, here plotted on top of the CIT168 brain template for axial (a), coronal (b), and sagittal (c) view (see Methods).

Extended Data Fig. 2 Participants’ performance in the scene recognition task did not differ significantly across different boundary types, Related to Fig. 2.

a-c, Behavior quantified by accuracy (a), reaction time (b), and confidence level (c) across all trials. Results are shown for boundary type NB (green), SB (blue), and HB (red) during the scene recognition task. The horizontal dashed lines in (a) show chance levels (0.5) and in (c) show the maximum possible confidence value (3 = high confidence). Each dot represents one recording session. Black lines in (a-c) denote the mean results averaged across all recording sessions. One-way ANOVA between NB/SB/HB, degrees of freedom = (2, 57).

Extended Data Fig. 3 Boundary cells and event cells do not respond to clip onsets and clip offsets during encoding, Related to Fig. 3.

a, Responses during the encoding stage from the same example boundary cells shown in Fig. 3a,b aligned to the clip onsets. b, Firing rates of all 42 boundary cells (solid and dashed arrows denote the examples in a) during the encoding stage aligned to the clip onsets, averaged over trials within each boundary type and normalized to each neuron’s maximum firing rate throughout the entire task (see color scale on bottom). c, Responses during the encoding stage from the same example boundary cells shown in (a) aligned to the clip offsets. d, Firing rates of all 42 boundary cells during the encoding stage aligned to the clip offsets using the same format as (b). e, Responses during the encoding stage from the same example event cells shown in Fig. 3e,f aligned to the clip onsets. f, Firing rates of all 36 event cells (solid and dashed arrows denote the examples in e) during the encoding stage aligned to clip onsets, using the same format as (b). g, Responses during the encoding stage from the same example event cells shown in € aligned to the clip offsets. h, Firing rates of all 36 event cells during the encoding stage aligned to the clip offsets using the same format as (b). For (a), (c), (e), (f), Top: raster plot color coded for different boundary types (green: NB; blue: SB; red: HB). Bottom: Post-stimulus time histogram (bin size = 200 ms, step size = 2 ms, shaded areas represented ± s.e.m. across trials). (b and f) are copied from Fig. 3d,h for comparison purposes.

Extended Data Fig. 4 Boundary cells and event cells do not respond to image onsets and offsets during scene recognition and time discrimination, Related to Fig. 3.

a-b, Responses during scene recognition from the same example boundary cells shown in Fig. 3a,b aligned to stimulus onset. c, Firing rates of all 42 boundary cells (solid and dashed arrows denote the examples in a and b) during scene recognition aligned to the stimulus onsets, averaged over trials within each boundary type and normalized to each neuron’s maximum firing rate throughout the entire task (see color scale on bottom). d-e, Responses during time discrimination from the same example boundary cells shown in (a and b) aligned to stimulus onset. f, Firing rates of all 42 boundary cells during time discrimination aligned to the stimulus onset using the same format as in c. g-h, Responses during scene recognition from the same example event cells shown in Fig. 3e,f aligned to stimulus onsets. i, Firing rates of all 36 event cells (solid and dashed arrows denote the examples in g and h) during scene recognition aligned to the stimulus onset, using the same format as in a and b. j, Responses during time discrimination from the same example event cells shown in g and h aligned to stimulus onset. k, Firing rates of all 36 event cells during time discrimination aligned to the stimulus onsets using the same format as in f. For (a), (b), (d), (e), (g), (h), (j), (k), Top: raster plot color coded for different boundary types (green: NB; blue: SB; red: HB). Bottom: Post-stimulus time histogram (bin size = 200 ms, step size = 2 ms, shaded areas represented ± s.e.m. across trials).

Extended Data Fig. 5 Neurons that respond to clip onsets and clip offsets do not overlap with boundary and event cells, Related to Fig. 3.

a-b, Responses during the encoding stage from an example clip onset-responsive cell located in the amygdala aligned to clip onsets (a), and boundaries (b). Top: raster plots. Bottom: Post-stimulus time histogram (bin size = 200 ms, step size = 2 ms, shaded areas represented ± s.e.m. across trials). A cell was considered as a clip onset cell if its firing rate differed significantly between a 1 s window immediate before and after clip onset (p < 0.05, one-tailed permutation t-test). c-d, Responses during the encoding stage from an example clip offset-responsive cell located in the hippocampus aligned to clip offsets (c), and boundaries (d). A cell was considered as a clip offset cell if its firing rate differed significantly between a 1 s window immediate before and after clip offsets (p < 0.05, one-tailed permutation t-test). Same format as (a and b). e, Seventy six out of 580 cells in the MTL qualified as clip onset-responsive cells and four out of 580 cells in the MTL qualified as clip offset-responsive cells. None of these were also selected as either boundary or event cells.

Extended Data Fig. 6 Responses of boundary cells during encoding grouped by memory outcomes from the time discrimination task, Related to Fig. 4.

a1-a2, Response of the same example boundary cell in Fig. 4a and Fig. 4b. During encoding, this cell responded to SB and HB transitions regardless of whether the temporal order of the clip was later correctly (a1) or incorrectly (a2) retrieved in the time discrimination test. Shaded areas represented ± s.e.m. across trials. b1- b2, Left: timing of spikes from the same boundary cell shown in (a1 and a2) relative to theta phase calculated from the local field potentials, for clips whose temporal order were later correctly (b1) or incorrectly (b2) retrieved. Right: phase distribution of spike times within [0, 1] seconds time windows following the middle of the clip (NB) or boundary (SB, HB) for clips whose temporal order were later correctly (b1) or incorrectly (b2) retrieved. c-d, Population summary for all 42 boundary cells. c, Z-scored firing rate (0–1 s after boundaries during encoding) for each boundary type did not differ between clips whose temporal orders were later correctly (color filled) vs. incorrectly (empty) retrieved. d, Mean resultant length (MRL) of spike times (relative to theta phases, 0–1 s after boundaries during encoding) across all boundary cells for each boundary type did not differ between clips whose temporal orders were later correctly (color filled) vs. incorrectly (empty) retrieved. Each dot represents one boundary cell. Black lines in c and d denote the mean results averaged across all boundary cells. One-tailed permutation t-test, degrees of freedom = (1, 82).

Extended Data Fig. 7 Responses of event cells during encoding grouped by memory outcomes from the scene recognition stage, Related to Fig. 4.

a1-a2, Response of the same example event cell in Fig. 4e,f. During encoding, this cell responded to HB transitions regardless of whether frames were later correctly (a1) or incorrectly (a2) recognized in the scene recognition task. Shaded areas represented ± s.e.m. across trials. b1-b2, Left: timing of spikes from the same event cell shown in a1-a2 relative to theta phase calculated from the local field potentials, for frames that were later correctly (b1) or incorrectly (b2) recognized. Right: phase distribution of spike times within [0, 1] seconds time windows following the middle of the clip (NB) or boundary (SB, HB) for frames that were later correctly (b1) or incorrectly (b2) recognized. c-d, Population summary for all 36 event cells. c, Z-scored firing rate (0–1 s after boundaries during encoding) for each boundary type did not differ between frames that were later correctly (color filled) vs. incorrectly (empty) recognized. d, Mean resultant length (MRL) of spike times (relative to theta phases, 0–1 s after boundaries during encoding) across all event cells for each boundary type did not differ between frames that were later correctly (color filled) vs. incorrectly (empty) recognized. Each dot represents one event cell. Black lines in c and d denote the mean results averaged across all event cells (c, d). One-tailed permutation t-test, degree of freedom = (1, 70).

Extended Data Fig. 8 Neural state changes following soft and hard boundaries shown for individual participants, Related to Fig. 5.

Multidimensional distance (MDD, see Fig. 5d–g for definition) as a function of time aligned to the middle of the clip (green: NB) and boundaries (blue: SB, red: HB). MDD is shown for all MTL cells within each participant (for example, ‘Sub1 in B1 E2 O32’ denotes MDD computed by 1 boundary cell, 2 event cells and 32 other MTL cells in participant 1). Shaded areas represent ± s.e.m. across trials.

Extended Data Fig. 9 Clip-onsets responsive neurons respond to both correct and incorrect targets during scene recognition, Related to Fig. 6.

a-b, Responses during scene recognition from an example clip onset-responsive cell (see definition in Extended Data Fig. 12) located in the amygdala aligned to image onsets in correctly recognized target (a) and forgotten target (b) trials. Top: raster plots. Bottom: Post-stimulus time histogram (bin size = 200 ms, step size = 2 ms, shaded areas represented ± s.e.m. across trials). c, Comparison (across all 76 identified clip-onsets responsive neurons) between mean firing rates averaged within [0 1.5]s after image onsets for remembered vs forgotten targets. On each box, the central mark indicates the mean results averaged across all clip-onsets responsive neurons, and the bottom and top edges of the box indicate the 25th and 75th percentiles, respectively. The whiskers extend to the most extreme data points not considered outliers, and the outliers are plotted individually using the ‘+‘ marker symbol. One-way ANOVA, degrees of freedom = (1, 150).

Supplementary information

Supplementary Information

Supplementary Figs. 1–13 and Tables 1–5.

Reporting Summary

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zheng, J., Schjetnan, A.G.P., Yebra, M. et al. Neurons detect cognitive boundaries to structure episodic memories in humans. Nat Neurosci 25, 358–368 (2022). https://doi.org/10.1038/s41593-022-01020-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41593-022-01020-w

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing