Forgetting across a hierarchy of episodic representations

Rich episodic experiences are represented in a hierarchical manner across a diverse network of brain regions, and as such, the way in which episodes are forgotten is likely to be similarly diverse. Using novel experimental approaches and statistical modelling, recent research has suggested that item-based representations, such as ones related to the colour and shape of an object, fragment over time, whereas higher-order event-based representations may be forgotten in a more ‘holistic’ uniform manner. We propose a framework that reconciles these ﬁndings, where complex episodes are represented in a hierarchical manner, from individual items, to small-scale events, to large-scale episodic narratives. Each level in the hierarchy is represented in distinct brain regions, from the perirhinal cortex, to posterior hippocampus, to anterior hippocampus and ventromedial prefrontal cortex. Critically, forgetting may be underpinned by different mechanisms at each level in the hierarchy, leading to different patterns of behaviour.

Episodic memory supports our ability to vividly recollect past experiences.These experiences can be highly rich and detailed in nature, containing both low-level perceptual and higher-order contextual and narrative details.Given this complexity, it is unlikely that forgetting in episodic memory is uniform in nature -different aspects of an episode may be forgotten via different mechanisms and at different rates.Research into forgetting has historically focussed on the rate at which forgetting occurs (see Ref. [1] for a review) or whether forgetting occurs via interference or decay [2,3].Recent research has begun to tackle the key question of how episodic representations change as a function of forgetting [4 ,5 ,6 ].Building on previous theoretical accounts [7-14], we propose that episodic memories are represented in a hierarchical manner across distinct brain regions and that forgetting at each level in the hierarchy might be underpinned by different mechanisms.

Holistic versus fragmented forgetting
Do episodic representations that support long-term memory fragment over time, such that some aspects of an event are forgotten, whereas others are remembered, or are they forgotten in a more 'holistic' manner?Imagine yourself at your birthday party and your friend gives you a present.Over time, your memory of this event will inevitably change.One critical question is whether the elements of this event (i.e.people, locations, and objects) are forgotten independently (e.g.you may forget the present you received, but still remember the person and location), suggesting that the representation 'fragments' over time, or the elements are forgotten in a dependent manner (e.g. if you forget the object, you are also more likely to forget the person and location), suggesting that the representation is forgotten in a 'holistic' manner.
There is evidence that event-based representations involving multiple elements (i.e.people, locations, and objects) tend to be retrieved and forgotten holistically [15,16,17 ,5 ,18].In these studies, 'events' containing three elements (e.g.kitchen, Barack Obama, and hammer) are encoded as three separate pairwise associations (e.g.kitchen-Barack Obama, kitchen-hammer, and Barack Obamahammer).Participants' memory for these events is then tested by cueing one event element (e.g.kitchen) and asking them to select the associated target (e.g.Barack Obama) among foils of the same category.Across multiple studies, statistical dependency was observed between the retrieval of elements within an event.If you are cued with the location and successfully retrieve the person, you are also more likely to successfully retrieve the object when cued with the location.This retrieval dependency is similar to that observed when all three elements are encoded on a single trial [15,16, cf.19], suggesting that encoding three separate, but overlapping, pairwise associations can form an episodic representation similar to that for events encoded on a single trial (see Ref. [17 ] for fMRI evidence).
Importantly, this measure of retrieval dependency can also be used to infer whether mnemonic representations fragment over time, or are forgotten holistically.If representations fragment over time, such that some elements are forgotten but not others, retrieval dependency should decrease (Figure 1).However, if the whole representation is forgotten, then dependency across events should remain stable -either you remember the whole event or you do not.Joensen et al.
[5 ] recently provided evidence in favour of holistic forgetting -although people remembered fewer events following a delay, retrieval dependency did not decrease.This result suggests that even with the forgetting of some events over time, those events that remain accessible are still retrieved holistically.
These more recent results would seem to be at odds with one previous study.Brady et al.
[20 ] asked whether the features of real-world objects, such as their colour, exemplar, or state (e.g. a closed or an open wardrobe) are forgotten independently over time.Participants viewed objects and at test selected the seen object among foils of the same colour, state or exemplar after short and longer delays.The authors observed independent forgetting for object colour and state; accuracy for these two properties was similar immediately after encoding but decreased more rapidly over time for colour relative to state.Critically, retrieval dependency for the object state and exemplar decreased over time.Utochkin and Brady [21 ] further showed that although retrieval accuracy for a single object feature (e.g.exemplar or state) may be high, people find it difficult to correctly match two features (e.g. which object exemplar was in which state) after seeing objects with different feature conjunctions.These findings are in line with research by Cooper and Ritchey [22 ] who asked participants to reconstruct the colour Forgetting of episodic events Andermane, Joensen and Horner 51  Holistic versus fragmented forgetting.

Holistic forgetting Fragmented forgetting
Representations with multiple elements or features can be forgotten in a holistic manner, with all elements being forgotten, or in a fragmented manner, with some elements being forgotten while others are remembered.Both forms of forgetting result in decreases in retrieval accuracy (Accuracy) between an immediate test (Immediate) and a delayed test (Delay).However, whereas retrieval dependency (Dependency) should remain stable over time in the presence of holistic forgetting, it should decrease in the presence of fragmented forgetting.Evidence suggests that event-based representations are forgotten in a holistic manner [5 ], whereas item-based representations may be forgotten in a more fragmented manner [20 ].
and location of objects previously seen within panoramic scenes.The authors found that the gist of the features was retrieved in a dependent manner, whereas the precision of retrieval for each feature (i.e.resolution) was independent.Thus, research suggests that precise perceptual features of individual event elements may not be bound within the same episodic representation and are therefore more likely to fragment over time.
More recently, the fragmentation of object-based representations has been challenged.Balaban et al. [4 ] investigated several object properties (e.g.exemplar, material, colour, state, and orientation) across experiments that manipulated the stimuli, encoding time, and learning task (incidental or explicit).Participants encoded objects and then selected the seen object among foils with combinations of correct and incorrect features, immediately after encoding and after a delay.Participants consistently remembered and forgot object features in a holistic manner; retrieval of one feature was dependent on that of another at both time points.Importantly, their results also suggested a hierarchical dependence in object representations, as retrieving the object exemplar was possible without retrieving a lower-level feature (i.e.state or colour), but it was not likely that a low-level feature could be remembered without also retrieving the exemplar.Thus, the recent item-based findings [4 ] might be reconciled with earlier work [20 ] if we consider that independently represented object features fragment over time, whereas hierarchically related object features may be forgotten more holistically.

Item-based versus event-based forgetting
Behavioural studies have therefore provided evidence for both holistic and fragmented forms of forgetting.Focussing on the studies by Joensen et al.
[20 ], the results point to a possible dissociation between event-based and item-based representations, respectively.We propose that the way in which information is originally encoded has a direct bearing on how it is forgotten.In the case of object-based representations, object features can be encoded in an independent manner and therefore can be forgotten independently.Eventbased representations are encoded in a more dependent manner and are therefore likely forgotten holistically.Importantly, there is a large body of evidence suggesting that item-based (e.g.object) and event-based representations are supported by different brain regions, with differing neural circuitry.It is possible that these anatomical differences between event-based and item-based representations underpin the behavioural differences in retrieval dependency seen across these studies.
Whereas the perirhinal cortex (PRC), and the ventral visual regions that project to PRC, are thought to support the encoding and retrieval of item-based representations [9, [23][24][25][26][27], the hippocampus (HPC) is thought to support event-based representations [28][29][30][31].Critically, the process of forgetting may be determined by the nature of the neural representation probed [32,33].For example, research suggests that familiarity -a process supported by PRC item-based representations -decreases as a function of interference, whereas recollection -a process supported by HPC event-based representationsdecreases via decay [34 ].
Importantly, the dissociation between item-based and event-based representations is further supported by differences in the underlying neurophysiology of the PRC and HPC.Neocortical representations, such as those in the PRC, are thought to be coded in a distributed manner [35,36], and the overlapping nature of such representations may make them particularly susceptible to interference from related feature-specific information (e.g. a similarly shaped object to the one in memory).HPC representations are thought to be sparse and non-overlapping in nature -due to the ability of the dentate gyrus (DG) to pattern separate input from the entorhinal cortex into non-overlapping orthogonal neural codes [37][38][39].These sparse representations may be less susceptible to interference, relative to neocortical representations.Instead, forgetting may be more likely to occur via decay [3,40].For example, ongoing neurogenesis, where new granule cells form and integrate in the DG and CA3 [40], may alter existing HPC circuitry such that more remote memory representations become less accessible over time, relative to more recent ones.
Returning to fragmented versus holistic forgetting, can the distinction between item-based and event-based representations explain the divergent behavioural findings discussed above?If items and their features are represented in a distributed manner, their forgetting will be dependent on subsequent feature-specific interference.For example, if objects in many shades of blue, but of distinct shapes, are encountered after seeing a blue umbrella, this may induce greater interference in relation to the umbrella's colour, relative to shape.Conversely, if similarly shaped objects (e.g.other umbrellas) of distinct colours are seen, this is more likely to induce interference in relation to the umbrella's shape.Furthermore, these intervening items may differentially affect the accessibility and precision of an item's perceptual features, with precision being more negatively affected by similar information [6 ,41 ].This interference could result from encoding newly encountered items [41 ], or possibly via internally generated reactivation of previously encoded representations during the process of systems consolidation [11,42].Thus, the fragmentation of memory for perceptual features of items, inferred from decreases in retrieval dependency, may be driven by the nature of the interfering material encountered (or re-activated) after the initial encoding of the item.
Hippocampal event-based representations, on the other hand, may be more likely to show a holistic form of forgetting.This is because the HPC is thought to bind multiple elements of a given event into a coherent eventbased representation and retrieve these elements via the process of pattern completion.Recent fMRI findings have provided evidence for this hippocampal pattern completion process in the retrieval of event 'triplets' consisting of locations, people and objects [17 ].Subfield CA3, with its recurrent connections, is thought to support the pattern completion process [43,38,44], with recent high-resolution fMRI evidence supporting this prediction specifically in relation to episodic memory [45].Pattern completion allows for the retrieval of a complete memory trace (pattern) given partial or ambiguous input.It supports the holistic retrieval that is thought to underpin recollection -where a single cue can elicit the retrieval of an entire previous event.Given the coherent nature of HPC representations, it is possible that they are forgotten relatively uniformly.Mnemonic decay may vary across event-based representations, but be uniform within a representation, such that some events are completely forgotten, whereas others are remembered in their entirety.Alternatively, although decay may not be uniform within an event, the process of pattern completion at retrieval may continue to induce dependency at the behavioural level -that is, remembering specific aspects of an event may allow for the retrieval of its more weakly associated elements.
What predictions does this item-based versus eventbased distinction make about forgetting?The first is that item-based representations should predominantly show fragmentation over time, whereas event-based representations should continue to show dependency in the presence of forgetting.In relation to item-based representations, fragmentation may be greater if interfering material for one object property is experimentally manipulated (e.g.interfering with colour but not shape).For example, Sun et al. [41 ] varied the similarity between the colour of working-memory items and intervening items, and observed that presenting dissimilar colours led to reduced memory accessibility for a particular colour whereas colours of intermediate similarity lead to decreases in memory precision.Note that there may be specific situations where item-based representations do not show fragmentation -such as when object-features are hierarchically related to one another [4 ].The second prediction is that event-based representations should continue to show dependency, even when overlapping events are encoded (e.g.events in the same location) [46].Note that it is possible that encoding new overlapping events may induce forgetting of previously learnt events via interference; however, the prediction is that retrieval should continue to be all-or-none due to hippocampal pattern completion.In other words, even if hippocampal event-based representations are susceptible to interference from overlapping events, it will not result in fragmented forgetting.

Forgetting across a hierarchy of episodic representations
We have focussed on a distinction between items and events, but episodic memories are more complex than this.Returning to our earlier example of your birthday party, it is likely that the entire episode contains multiple smaller events in different locations with differing objects and people.Thus, episodes typically consist of an overarching narrative linking together multiple related events.We may play party games such as pass-the-parcel in the living room, then play outside in the garden, then sing happy birthday and eat cake in the kitchen.All three 'events' here are part of the same episodic narrative.
Previous experimental work has shown that these narrative 'core' aspects of episodes are forgotten more slowly than peripheral (e.g.perceptual) details [47 ].Related work on the forgetting of prose passages has also shown differential rates of forgetting dependent on the nature of the information tested (i.e. the exact phrasing of a sentence versus the situation described) [48 ].This is consistent with multiple theoretical accounts [7,8,10,12] that propose semanticised or gist-like representations, likely supported by the ventromedial prefrontal cortex (vmPFC) [13,49,50], are more robust to forgetting than contextually rich and detailed HPC-based memories.Episodic memories are highly complex and hierarchical in nature, with levels of representation ranging from perceptual details of individual items to overarching narratives, and as such, the nature (i.e.rate and coherence) of forgetting may be dependent on which level of the hierarchy we are examining.
Inspired by the research on episodic and autobiographical memory [13,51,52] and event models and narrative structure [48 , [53][54][55] we propose (at least) three distinct representational levels: items, events, and episodic narratives (Figure 2).Our hierarchical proposal is consistent with recent models of episodic and autobiographical memory [13, 51,52]; however, here we focus on how the different levels of representation change as a function of forgetting.The lowest item-based level is likely supported by the PRC and the inputting regions in the ventral visual stream coding feature representations such as colour and shape [23,26,27].The event-based level is likely supported by the HPC [17 , [28][29][30][31]45]. Research has suggested that the longitudinal axis of the HPC may support hierarchical representations, with posterior regions representing fine-grained local detail and anterior regions representing more coarse, global information [56].Recent multivariate fMRI evidence supports this hierarchical prediction in relation to episodic narratives inferred from video-based episodic events [57 ].Thus, our episodic event and narrative levels may map onto this posterior-anterior distinction, and the broader posterior-medial anterior-temporal (PMAT) networks [58].Smaller-scale memory networks involving events from a single spatiotemporal context may be represented in posterior portions of HPC, whereas larger-scale memory networks comprising narratives across several events may be represented more anteriorly in the HPC, as well as the vmPFC [57 ,50,51].
Whether this proposed hierarchy has important implications for how forgetting occurs has not been explored.

Episodic narrative
Episodic event

Items & features
The narrative of an episode, including multiple sequential events across different spatiotemporal contexts, is represented in the anterior HPC and vmPFC.
The constituent elements (e.g.people, locations, and objects) of events with a single spatiotemporal context are bound into coherent event representations in the posterior HPC.
Individual items, such as objects, are represented in the PRC and their individual features (e.g.shape) are represented in feature-specific regions in the ventral visual stream (e.g.LOC).

Current Opinion in Neurobiology
The hierarchical nature of episodic memory.An episodic representation comprises multiple levels within a hierarchy across multiple cortical regions.Individual items (e.g.objects) are represented in the perirhinal cortex (PRC), with individual features represented in feature-specific regions of the ventral visual stream (e.g.object shape in the Lateral Occipital Complex (LOC)).Multiple items, such as objects and people, as well as the spatiotemporal context, are bound into a coherent event representation in the posterior hippocampus (pHPC).Multiple events that take place within a broader episodic narrative are associated in the anterior HPC (aHPC) and ventromedial prefrontal cortex (vmPFC).Neurophysiological differences between these regions, in relation to whether representations are coded in a distributed or sparse manner (PRC versus HPC), and the extent of neurogenesis and relative size of subfields (in the posterior versus anterior HPC) are likely to drive distinct behavioural patterns of forgetting (e.g.rate and coherence) across this hierarchy.real-world objects are represented in a dependent manner in long-term memory.J Exp Psychol Gen 2020, 149:1275-1293 http://dx.doi.org/10.1037/xge0000716.This study investigated whether object representations remain coherent over time.They found that when objects are forgotten, they are forgotten completely and translate into random guesses rather than partial memory for one object feature but not the other.They also showed that for the small proportion of partial feature reports that were observed, retrieval dependency differed as a function of object-feature.That is, retrieval of lower-level features such as material or orientation was dependent on retrieving the higher-order property of object exemplar (but not vice versa).This suggests dependency may differ when object features are represented in a hierarchy (as opposed to independently).

5.
Joensen BH, Gaskell MG, Horner AJ: United we fall: all-or-none forgetting of complex episodic events.J Exp Psychol Gen 2020, 149:230-248 http://dx.doi.org/10.1037/xge0000648.The authors tested retrieval dependency for episodic events comprising three elements (a person, an object, and a location) over time.Despite decreases in accuracy, no evidence for decreases in dependency were seen.This suggests that event-based representations are coherent and tend to be forgotten in a uniform, holistic manner; an event is either retrieved in its entirety or its elements are forgotten completely.

6.
Berens SC, Richards BA, Horner AJ: Dissociating memory accessibility and precision in forgetting.Nat Hum Behav 2020:1-12 http://dx.doi.org/10.1038/s41562-020-0888-8.This registered report tested participants' memory in terms of accessibility and precision for word-location associations over time.They showed that forgetting principally involved decreases in accessibility, with no evidence for decreases in precision.This suggests that if a wordlocation is retrieved successfully, it is remembered with as much precision as when it was first encoded.They also showed that clustering semantically related words in a specific location led to increases in accessibility, but decreases in precision, compared to when no clustering was present. .This fMRI study demonstrated evidence for hippocampal pattern completion in humans in relation to the retrieval of complex episodic events.They showed that location, person, object 'events' were reinstated in neocortex in the presence of a partial cue, and that this reinstatement correlated with the BOLD response in the hippocampus.This supports the proposal that event representations are retrieved in the hippocampus via pattern completion, and this retrieval drives subsequent reinstatement of event elements in the neocortex.Brady TF, Konkle T, Alvarez GA, Oliva A: Real-world objects are not represented as bound units: independent forgetting of different object details from visual memory.J Exp Psychol Gen 2013, 142:791 http://dx.doi.org/10.1037/a0029649.This study was the first to investigate whether representations of objects remain coherent over time.They found that different object features were forgotten at different rates and retrieval dependency between different object features decreased with delay, consistent with the fragmentation of object representations.This suggests that objects may not be stored in long-term memory as fully bound, integrated representations and that different perceptual features (i.e.object colour and state) may be forgotten independently from one another.

21.
Utochkin IS, Brady TF: Independent storage of different features of real-world objects in long-term memory.J Exp Psychol Gen 2019, 149:530-549 http://dx.doi.org/10.1037/xge0000664.This study tested long-term recognition memory for specific feature conjunctions of objects (e.g. two exemplars of a coffee mug in two different states, empty or full).Participants remembered individual object-features; however, they performed poorly when asked to match two features (e.g. which mug was empty?).The finding that object features can be retrieved independently but not together suggests that object features may not be stored in a conjunctive object representation.

22.
Cooper RA, Ritchey M: Cortico-hippocampal network connections support the multidimensional quality of episodic memory.eLife 2019, 8:e45591 http://dx.doi.org/10.7554/eLife.45591.001.This fMRI study investigated the interactions between the hippocampus and surrounding cortical networks as participants retrieved features (e.g.colour, spatial context, and emotional tone) of episodic scenes using continuous response options (e.g.adjusting the colour wheel for object colour and the spatial context for the object location within the scene).They observed retrieval dependency in relation to retrieving the gist of features (i.e.accessibility) but the resolution with which each feature was retrieved (i.e.precision) was independent.This study suggests that a hippocampal event-based representation may not include precise information about each perceptual feature, but instead only represent its gistlike category.

48.
Fisher JS, Radvansky GA: Patterns of forgetting.J Mem Lang 2018, 102:130-141 http://dx.doi.org/10.1016/j.jml.2018.05.008.This study investigated forgetting for written narratives across three levels: (1) the verbatim words and syntax of a sentence (i.e.surface form); (2) the propositional meaning of a sentence (i.e.textbase); (3) the inferred meaning of the situation described in the narrative (i.e.event model).They showed that event model memory decreased initially but then plateaued over time, whereas memory for the textbase decreased gradually with a sharp drop at 7 days.Surface form memory showed the most rapid decline but did not reach chance level at any point.This study highlights the hierarchical structure of memories for narratives and suggests a distinct pattern of forgetting for each level of the representation.This study investigated the role of the HPC long axis in representing episodes of different complexity using video clips of animated life-like events and fMRI multivoxel pattern analysis (i.e.representational similarity analysis).The authors found that simple pairwise event associations were represented in posterior regions, whereas more complex inferred associations between several events were represented in anterior HPC.Their results suggest episodes vary in the resolution in which they are represented, from single spatiotemporal contexts to broader narratives linking several events.Critically, the long axis of HPC appears to track this episodic resolution, with posterior regions representing smaller event networks, middle regions representing medium-scale networks comprising several event associations, and anterior regions tracking complex narratives involving several events including inferences about how the events may be linked.

Forgetting
of episodic events Andermane, Joensen and Horner 55 www.sciencedirect.comCurrent Opinion in Neurobiology 2021, 67:50-57 BL, O'Reilly RC: Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory.