The impact of early and late blindness on language and verbal working memory: A brain-constrained neural model

Neural circuits related to language exhibit a remarkable ability to reorganize and adapt in response to visual deprivation. Particularly, early and late blindness induce distinct neuroplastic changes in the visual cortex, repurposing it for language and semantic processing. Interestingly, these functional changes provoke a unique cognitive advantage – enhanced verbal working memory, particularly in early blindness. Yet, the underlying neuromechanisms and the impact on language and memory-related circuits remain not fully understood. Here, we applied a brain-constrained neural network mimicking the structural and functional features of the frontotemporal-occipital cortices, to model conceptual acquisition in early and late blindness. The results revealed differential expansion of conceptual-related neural circuits into deprived visual areas depending on the timing of visual loss, which is most prominent in early blindness. This neural recruitment is fundamentally governed by the biological principles of neural circuit expansion and the absence of uncorrelated sensory input. Critically, the degree of these changes is constrained by the availability of neural matter previously allocated to visual experiences, as in the case of late blindness. Moreover, we shed light on the implication of visual deprivation on the neural underpinnings of verbal working memory, revealing longer reverberatory neural activity in ‘blind models ’ as compared to the sighted ones. These findings provide a better understanding of the interplay between visual deprivations, neuroplasticity, language processing and verbal working memory.


Introduction
The ability of the human brain to reorganize itself is particularly evident in individuals with visual deprivation.Specifically, visual loss enables the visual system to self-adapt and become functionally involved in language processing, word retrieval, and semantic understanding (Burton, 2003;Lane et al., 2015Lane et al., , 2017;;Occelli et al., 2017;Röder et al., 2002;Watkins et al., 2012).Brain imaging studies have shed light on this adaptive process by revealing the recruitment of both primary and secondary visual regions in congenitally blind individuals for tasks like generating verbs upon hearing nouns or during sentence processing, as compared to sighted individuals (Amedi et al., 2004(Amedi et al., , 2003;;Bedny et al., 2011;Burton, 2002;Raz et al., 2005;Struiksma et al., 2011, see also Burton, 2003).Intriguingly, the activated areas in the visual cortex operate similarly to traditional language-processing regions (Bedny et al., 2011).Functional relevance for language and semantic processing is demonstrated by changes in behaviour in blind individuals, but not in sighted ones, following Transcranial Magnetic Stimulation (TMS) to primary visual cortex (V1, Amedi et al., 2004).
Intriguingly, neuronal reorganization has been associated with cognitive advantages in blindness.Those born blind demonstrate an impressive ability to recall longer lists of both concrete and abstract words, as well as sequences of letters and numbers, which by far outperforms undeprived individuals (Amedi et al., 2003;Arcos et al., 2022;Occelli et al., 2017;Pasqualotto et al., 2013;Withagen et al., 2013).This superior performance appears to be confined to verbal tasks and is not mirrored in spatial ones like recalling object reconfiguration and paths (Gudi-Mindermann et al., 2018;Occelli et al., 2017;Raz et al., 2007) as well as being resilient to interference, such as solving complex math problems during verbal recall (Arcos et al., 2022).This extraordinary capacity has been partly attributed to the necessity for blind individuals to lean more heavily on working memory processes as they navigate daily life without visual information (Raz et al., 2007).Nevertheless, most notably, the intensity of V1 activation in blind individuals has been found to correlate with their verbal working memory performance and their ability to recall long word lists (Amedi et al., 2003;Park et al., 2011;Raz et al., 2005; see for further discussion Sabourin et al., 2022), thus pointing to a unique cognitive advantage stemming from the visual cortex's recruitment for language processing in blindness.
Furthermore, neural recruitment in the visual cortices seems not to be confined to individuals who lose sight at birth or in early childhood.Observations of neural changes have also been reported in late blind individuals, who initially developed language skills with visual experiences, but later lost their sight.Previous research indicates that late blind individuals generally exhibit neural reorganization in the visual cortices, however to a lesser extent as early blind and primarily confined to the primary striate regions (V1) during tasks like generating verbs or semantic judgment tasks (Burton, 2002;Burton and McLaren, 2006;Kujala et al., 1997a;Voss et al., 2008).This is further supported by a study showing correlated activity during resting state between language cortical regions (i.e., Broca) and visual cortices in late blind individuals, arguing for language functional changes being possible also in case of late blindness (Sabbah et al., 2016).Possible compensatory mechanisms, such as enhanced working memory due to late visual deprivation, have also been documented (Voss et al., 2004), which showed superior spatial abilities during auditory perception.However, the interplay between late visual deprivation and enhanced cognitive functions is not well documented and further research is needed in this regard.In contrast, however, other research has posited that significant neuroplasticity involving the left V1 can mainly take place during a critical developmental window in early childhood, when the language cortex is still maturing, as no such activation was found in late blind individuals (Bedny et al., 2011(Bedny et al., , 2012)).Specifically, Bedny et al. (2012) suggest that neural differences reported in visual cortices in case of late blindness are possibly associated with task-specific processes, such as attention, but not with language processing per se.This view argues that the brain may not undergo adaptive reorganizations to the same extent when vision is lost later in life, as compared to early visual loss.Consequently, neural recruitment of the visual cortices in the late blind are mixed in this respect and further work is necessary to fully disentangle the specific differences at the cellular level between early and late blindness.
To close this gap in understanding, the application of artificial neural models has emerged as a powerful tool to examine the neural mechanisms underlying plastic changes caused by deprivation or lesions (e.g., Ralph et al., 2017;Shtyrov et al., 2023;Tomasello et al., 2019).However, models need to be designed to mimic well-known biological principles of the human brain at the cellular and cortical neural levels to offer an explanation of higher cognitive functions that are consistent with neuroscience knowledge and accepted by the neuroscience community (Breakspear, 2017;Dwivedi et al., 2021;Pezzulo et al., 2013;Pulvermüller, 2023;Pulvermüller et al., 2021;van Albada et al., 2021).Adopting this approach, a study employing brain-constrained models successfully simulated the specific anatomy and neurophysiology of the frontotemporal and occipital lobes, shedding light on the neural mechanisms underlying changes resulting from early visual deprivation (Tomasello et al., 2019).This study used Hebbian learning principles to simulate associative word learning in visually deprived and sighted models.The outcome revealed that the visually deprived model, as compared to the sighted ones, displayed the emergence of neural circuits extending into the primary visual area (V1), specifically for action-related words.These findings are in line with neuroimaging studies demonstrating visual cortex activation during verb generation tasks like saying "bake" when hearing "cake" (Amedi et al., 2003;Arcos et al., 2022;Occelli et al., 2017;Pasqualotto et al., 2013;Withagen et al., 2013), which implies the processing of action-related knowledge and hence engagement of action-related neural circuits (Moseley and Pulvermüller, 2014;Pulvermüller, 2012Pulvermüller, , 2018)).This recruitment of neural matter in the visual cortex was reported to be a consequence of two key factors.The first involves Hebbian correlation learning, which triggers the spontaneous strengthening and expansion of frequently activated neural circuits (Doursat and Bienenstock, 2006), a process that leads to cell assembly spreading into adjacent areas.The second factor relates to the imbalance of activity in the visual areas in undeprived individuals caused by the absence of uncorrelated variable noise, typically present during word learning.Consider, for example, the variable sensory input from the environment that changes every time when hearing a word such as "running" while performing the action of running during associative learning.This sensory noise in sighted individuals serves to stop the excessive expansion of neural circuits, and being critical for the semantic formation of category-specific cortical activation, as documented in previous studies (Binder and Desai, 2011;Grisoni et al., 2021;Pulvermüller, 2013;Pulvermüller and Fadiga, 2010).Specifically, in undeprived brains action-related neural circuits extend more in the motor system rather than the visual system, due to the presence of the variable sensory noise and vice versa for symbols grounded into the visual system, such as object related words referring to animals (Tomasello et al., 2019, see also Constant et al., 2023;Garagnani and Pulvermüller, 2016;Tomasello et al., 2018Tomasello et al., , 2017)).
Although a potential explanatory mechanistic of the neural changes in the early blind brain has been put forth through precise artificial cortex models, further investigation is still required to elucidate the biological mechanisms underlying the different neural plastic changes observed in early vs late blind individuals.A key unresolved issue is whether neural changes in the visual system for language processing are at all possible in late blind people (Bedny et al., 2011(Bedny et al., , 2012) ) or if the visual cortex involvement is restricted to the primary visual cortex, as opposed to the involvement of additional visual regions in early blind individuals (Amedi et al., 2004(Amedi et al., , 2003;;Bedny et al., 2011;Burton, 2002;Raz et al., 2005;Struiksma et al., 2011, see also Burton, 2003).Furthermore, the effects of these neural changes on the spatiotemporal dynamics of neural activation during speech understanding are still largely unexplored.This is particularly relevant in relation to the superior verbal working memory observed in blind individuals (Amedi et al., 2003;Arcos et al., 2022;Occelli et al., 2017;Pasqualotto et al., 2013;Withagen et al., 2013).At the neural level, the reverberation period is of special interest, i.e., the duration during which neural circuits tend to maintain activation for a period of time after stimulus presentation for ongoing cognitive processes.This neural correlate of activity maintenance within activated neural circuits has been linked to the active retention of information in verbal working memory (Baddeley, 2003;Fuster, 1995Fuster, , 2009Fuster, , 2000;;Pulvermüller and Garagnani, 2014;Schomers et al., 2017;Zipser et al., 1993;Zylberberg and Strowbridge, 2017).
To shed light on the intricate interplay between visual deprivation at different stages and its implication for verbal working memory, our current investigation draws on the methodologies and comparative approaches used in previous research (Tomasello et al., 2019).We employed brain-constrained neural networks to examine the effects of neuroplastic alterations stemming from visual deprivation in both early and late blindness.Specifically, our study focuses on how these conditions influence symbol learning and processing and compares them to sighted individuals within the sensorimotor systems.Additionally, we examined the spatio-temporal activations (mimicking EEG/MEG activation patterns) of the neural circuits and their reverberatory neural activity across the modelled brain regions during auditory word recognition.This analysis allowed us to evaluate the impact of neural changes that are implicated by visual deprivation, offering a comprehensive insight into the relationship between visual loss, neuroplasticity and verbal working memory.R. Tomasello et al.

General model architecture
The brain-constrained neural network used in this study is built upon well-established biological principles defined as crucial for simulating higher cognitive functions (Pulvermüller et al., 2021).At the micro level, the network employs physiologically realistic spiking neurons, while at the macro level, it incorporates the structural properties of twelve key cortical areas situated in the frontal, temporal, and occipital lobes responsible for language and semantic processing (see Fig. 1A).The connectivity structure linking these different regions (represented by different colored arrows in Fig. 1A&B) is guided by robust neuroscientific evidence (e.g., ′Rilling, 2014; Rilling et al., 2011;Sepulcre et al., 2012;Thiebaut de Schotten et al., 2012).
In brief, the model replicates a range of critical anatomical and physiological features at the micro and macro level of the cerebral cortex, including: (i) Neurophysiological dynamics of spiking pyramidal cells: The behavior of individual neurons in the network was modelled to resemble real spiking pyramidal cells.This involved implementing temporal summation, threshold-based spiking, and adaptation (Connors et al., 1982;Matthews, 2001).These mechanisms are crucial for capturing the firing patterns and excitability changes observed in actual biological neurons.(ii) Synaptic modification through Hebbian learning: Synaptic plasticity, specifically Hebbian learning, was incorporated into the network to mimic the mechanisms of long-term potentiation (LTP) and depression (LTD) (Artola and Singer, 1993) ().These processes allow for the strengthening and weakening of connections between neurons based on their correlated activity, a process for the formation of neural circuits and memory formation.(iii) Local lateral and area-specific inhibition mechanisms: The model incorporated local and area-specific regulation (Braitenberg, 1978;Yuille and Geiger, 2003) critical for the modulation of neural activity within and between different cortical regions, influencing the dynamics of information processing and competition between neural populations.(iv) Within-area connectivity: The network implemented sparse, random, and initially weak connectivity locally within each cortical area.Additionally, there was a neighborhood bias, promoting stronger connections between neighboring neurons.These connectivity principles are critical to shape the local circuitry of cortical regions (Braitenberg and Schüz, 1998;Kaas, 1997).(v) Between-area connectivity based on neurophysiological principles: To establish connections between different cortical regions, the network employed connectivity principles based on neurophysiological evidence (e.g., Rilling et al., 2011;Thiebaut de Schotten et al., 2012, see Table 1).These connections allow for communication and information transfer between distinct brain regions, facilitating the integration of sensory and cognitive processes.(vi) Presence of ongoing uniform uncorrelated white noise: Throughout all phases of learning and retrieval, the model included ongoing uniform uncorrelated white noise in all neurons (Rolls and Deco, 2010).This noise, introduces variability and randomness in neural activity, reflecting the stochastic nature of neural firing observed in real biological systems.Details of single-neuron properties and synaptic plasticity rules are provided in Appendix A and previous publications (Constant et al., 2023;Henningsen-Schomers and Pulvermüller, 2022;Tomasello et al., 2018Tomasello et al., , 2019)).Further details on the single-area model structure and connectivity are specified in the following sections.

Modelled cortical regions and connectivity structure
In the neural network model, each of the 12 cortical areas consisted of two layers: one layer comprised 625 excitatory cells (e-cells), simulating spiking pyramidal neurons, and another layer consisted of 625 inhibitory cells (i-cells), representing local pools of interneurons within the same cortical columns (Wilson and Cowan, 1972;Eggert and van Hemmen, 2000).
The connectivity structure between these regions is derived from neuroanatomical evidence using diffusion tensor and diffusion-weighted imaging (DTI/DWI), as summarized in Table 1.It includes neighboring areas (represented by black arrows), second-next-neighbor areas, also known as "jumping links" (blue arrows), and long-distance corticocortical links (purple arrows in Fig. 1B).

Word learning
To simulate word learning under normal-sighted and visual deprivations, we created a total of 39 neural network models, subdivided into 13 instances for each model type: "sighted," "early blind," and "late blind".All models were designed with the same architecture as described above and were trained with the same input patterns and parameters.
The different models were trained to simulate key features of associative word learning between a phonological word form and the object or manual actions we are used to speak about (Tomasello and Kruger, 1992;Vouloumanos and Werker, 2009).Drawing upon methodologies employed in prior simulation studies (Constant et al., 2023;Garagnani and Pulvermüller, 2016;Tomasello et al., 2017Tomasello et al., , 2018Tomasello et al., , 2019)), we exposed the neural network's primary areas to repeated sensorimotor pattern presentations.Each network instance utilized 12 distinct sets of sensorimotor word patterns, comprising six object-related and six action-related words.These patterns were created by selecting a fixed set of 22 cells at random within the 25 × 25 cell area of each cortical region, amounting to approximately 3.5 % of the total cells.This approach enabled us to investigate the potential neural plastic changes during word-meaning acquisition within the sighted, early blind, and late blind models across the various simulated regions.
Sighted model: The sensorimotor patterns in the sighted model were grounded in sensorimotor systems presented to specific primary areas of the neural network, as follow.Object-related word acquisition was driven by activity in perisylvian regions A1 and M1 i , alongside correlated visual activity patterns from V1.Similarly, for action-related word learning, semantic activity in the lateral motor area (M1 L ) was combined with perisylvian activity (A1 and M1 i ).To introduce variability and emulate real-world learning scenarios, we stimulated simultaneously the fourth non-relevant area (V1 for action words and M1 L for object words) with uncorrelated variable input pattern that changed during each learning step.This input aimed to simulate the diverse ways objects can be manipulated or grasped or the different visual variable inputs encountered during action execution, mirroring the learning process for novel concrete words.Additionally, the primary areas of the network received continuous presentation of extra white noise, referred to as "contextual" noise, which was superimposed at the interstimulus interval.This introduction of contextual noise contributed to a certain level of variability during the word meaning acquisition process for both word types.
Early blind model: For the early congenitally blind scenario, we employed the same network architecture, parameters, and sensorimotor inputs, as in the sighted model but excluded visual experience (i.e., V1 sensorimotor input) throughout the entire learning process.This approach allowed us to explore the unique dynamics of language acquisition in individuals who have never had access to visual information from birth (i.e., early blindness).
Late blind model: In the late blind models, we utilized neural networks previously trained under normal vision (sighted model) and subjected it to retraining, this time without visual experience.This procedure enabled us to delve into the network's adaptive capabilities in Table 1 Connectivity Structure.This table presents the references used to establish the connectivity structure of the network model, categorized by connection type and the regions involved.Taken from Tomasello et al. (2018).
The word-related sensorimotor patterns were repetitively presented a total of 1000 times in each model type, following the approach detailed earlier.Note that previous simulations employing a six area model demonstrated no significant alterations in the relevant primary areas when varying the number of learning steps between 1000 and 10000 (Garagnani et al., 2009;Schomers et al., 2017).Each trial commenced with the presentation of a word and referent patterns, which persisted for 16 simulation time steps.Subsequently, an interstimulus interval (ISI) followed during which no input was provided to the network.The subsequent word pattern, corresponding to the next learning step, was introduced to the network only under the condition that the global inhibition of the A1 and PF i areas declined below a predetermined fixed threshold (0.75 and 0.65 respectively).This criterion ensured that the neural activity returned to a baseline level, thereby minimizing the potential influence of one trial on subsequent ones.Throughout each ISI, the neural network contained only inherent baseline noise, mimicking spontaneous neuronal firing, and "contextual" noise.

Auditory word recognitionsimulated EEG/MEG activation
Following the completion of the training phase, the neural network was employed to simulate the intricate processes of perceiving, recognizing, and comprehending object-and action-related words.This allowed us to explore the underlying neurophysiological mechanisms governing these processes, mimicking the experience of hearing learned words while simultaneously recording brain responses (such as EEG/ MEG activity).Each testing trial was initiated by stimulating the primary auditory area (A1) with only the learned auditory 'word form' pattern.This stimulation persisted for 2 time-steps, succeeded by a 30 time-steps period during which no further input was administered, and an additional 10 time-steps were used as a baseline for subsequent trials.During the testing phase, each auditory pattern was presented three times; however, we excluded the first presentation from the averaging process.This exclusion was due to spurious neural activity stemming from very low global inhibition, leading us to consider only the last two consistent presentations for averaging, thereby ensuring a more accurate depiction of the neural responses.
During the word recognition process, we meticulously recorded the area-specific "within-cell assemblies (CA) activity" for every simulation time-step, encompassing the 10 time-steps preceding stimulus onset and the 30 time-steps following stimulus offset.This within-CA activity was computed by summing the output values (spikes) generated by the emergent CA cells in each area as a function of time.These data were then averaged over the 12 networks for each model type.
Subsequently, we identified the "reverberation time (Rtime)" to investigate sustained neural activity following stimulus presentation.The Rtime was calculated as the time duration between the peak activation, representing the maximum value of CA's action potential (spikes) averages, and the point at which the spiking activation returned to baseline (0 spikes) during the 30 post-stimulus time-steps.To ensure meaningful calculations and avoid analyzing areas with very weak responses, we set a threshold of the minimum maximum activation as > 1.
For each of the 12 learned networks (sighted, early blind, and late blind models), we computed the Rtime values across both word types and all network areas.These values were then averaged to obtain comprehensive insights into the neural response patterns.To assess the significance of the observed differences, statistical analysis was performed, which is detailed in the following sections.

Data processing and statistical analysis
In the process of simulating word learning, we observed the spon-taneous emergence of cell assemblies, representing strongly interconnected networks of neurons.To investigate and quantify the neuronal components forming the 12 distributed cell assembly (CA) circuits that had developed across different network areas, we specifically activated these word form neurons for 16 simulation time-steps in the primary perisylvian auditory-articulatory areas (A1, M1 i ) that simulated 'word production'.During this activation period, we carefully computed and visualized the average firing rate of each excitatory cell, totaling 7500 e-cells.In estimating a cell's average firing rate, we utilized the value ω E (e, t) derived from Eq. (3.2 in Appendix A), incorporating a time-constant τ Favg = 5.A specific excitatory cell (e-cell) was considered part of a given cell assembly (CA) circuit only if its timeaveraged rate (referred to as "firing rate") surpassed a threshold θ, which was both area-and input-pattern dependent.This threshold was defined as γ times the maximal time-averaged response of a single cell within that area to pattern w.More formally, where O(x, t) w is the estimated time-averaged response of cell x to word pattern w (see Eq. 3.3 in Appendix A) and γ ∈ [0,1] is a constant (we used γ = 0.5 on the basis of previous simulation results, (Garagnani et al., 2009(Garagnani et al., , 2008;;Tomasello et al., 2017).This was computed for each of the 13 trained network instances for each model types, averaging the number of CA cells per area over the 6 object-and 6 action-related words.
Cell assembly analysis: To explore potential significant statistical differences on the cell assembly sizes and distribution that emerged from word learning between the sighted, early, and late blind models, we conducted a comprehensive analysis using several ANOVAs.Firstly a 3way ANOVA was run with factors Models (three levels: Sighted, early Blind and late Blind), WordType (two levels: Object and Action) and Areas (12 levels: Perisylvian = {A1, AB, PB, M1 i , PM i , PF i }, Extrasylvian cortex = {V1, TO, AT, M1 L , PM L , PF L }).Additionally, to further investigate differences across the modelled cortical regions between the three models a 5-way ANOVA was run with factors Models (three levels: Sighted, early Blind and late Blind), WordType (two levels: Object and Action), PeriExtra (two levels: Perisylvian = {A1, AB, PB, M1 i , PM i , PF i }, Extrasylvian cortex = {V1, TO, AT, M1 L , PM L , PF L }), TemporalFrontal (TempFront)" (2 levels: temporal areas = {A1, AB, PB, V1, TO, AT}, frontal areas = {M1 L , PM L , PF L , M1 i , PM i , PF i }) and Areas (three levels: Primary = {A1, V1, M1 L , M1 i }, Secondary = {TO, AB, PM L , PM i } and Central = {PB, AT, PF L , PF i } areas).Subsequently, each system, 6 periand 6 extrasylvian areas, were investigated separately with factors "Models", "WordType", "TempFront" and "Areas".The same statistical analysis, but this time omitting "WordType" as a factor was additionally performed to disentangle the different CA distribution of action-and object-related words between the three models.A second level of analysis was run on each Model (sighted, early blind and late blind) separately, first with a 4-way ANOVA with factors "WordType", "PeriExtra", "TempFront" and "Areas"and subsequently, with 3-way ANOVA on each peri-and extrasylvian system within each model, separately.

Neural reverberation analysis
The statistical analysis on the reverberation time (Rtime) was only carried out for the action-related words, as the object words results are difficult to interpret since visually related words are usually acquired by blind people via other modalities, such as tactile ones (more details on this below).To this end, we run first a 2-way ANOVA with factors Models (three levels: Sighted, Early blind and Late blind) and PeriExtra (two levels: Peri-and Extrasylvian system), to examine any global differences in Rtime on both phonological and semantic systems.This was followed by a one-way ANOVA with the factor "Models" on each Extraand perisylvian system separately.To further investigate the differences between the modelled cortical regions between the three models, a 3-R.Tomasello et al. way ANOVA was performed only on the system that showed a significant difference in the one-way ANOVA with the factors "Models", "Temp-Front" & "Areas".The corrected p-values are reported together with the epsilon (ε) values.

Word learning results
In both the sighted and early blind models, the process of associating word forms with their corresponding semantic information resulted in the formation of interconnected neuronal ensembles known as "cell assemblies" (CAs), as conceptualized by Hebb (1949) distributed across the modelled regions.Similarly, when retraining the sighted model without any visual input, to simulate late blindness, significant changes were observed in size and topography of these distributed cell assemblies across the different brain regions.Fig. 2A illustrates an example of the distribution of CAs (each pixel representing one cell) for action-related words and Fig. 3A shows the distribution of object-related words after the learning for the sighted, early blind, and late blind models within a brain schematic.
In the perisylvian language cortex the distribution of cell assemblies (CAs) showed similar sizes and topographies for both word types in sighted, early blind, and late blind models.However, noticeable differences emerged in the extrasylvian cortical regions, where the two word types manifested clear differences across and in each model type.
Sighted model: In the sighted model, a clear topographic CAs distribution between word types was observed: Action-related CAs predominantly extended more into the motor regions (M1 L , PM L ) but not or to a lesser degree into the visual (V1, TO) regions.On the other hand, object-related CAs showed the reverse pattern, by extending more into the visual (V1, TO) and not or less into the motor (M1 L , PM L ) regions.
Early blind model: In the early blind model, the distribution of cell assemblies (CAs) for action words was similar to that in the sighted model, with the key difference of an extension of CAs into the visual system.Specifically, these action-related CAs recruited significant amounts of neural material from both the primary (V1) and secondary (TO) visual regions, a phenomenon not observed in the sighted model.For object-related CAs, the extension into the extrasylvian system was present but to a lesser degree, likely due to the absence of correlated visual input during learning.
Late blind model: In the late blind model, action-related CAs also extended into the visual system, however only within the primary visual region (V1), and to a lesser degree compared to the early blind model results.The distribution of object-related CA cells within the visual system (encompassing V1, TO, AT) was markedly reduced in comparison to the sighted model.However, it's important to note that for 'visually-related words' in both early and late blind models, the results cannot interpreted in the same way as for sighted individuals, as blind individuals do not have access to semantically relevant visual information.Instead, visually related words such as "look" are semantically linked with hand-motor activity patterns in blind children, as, for example, the suggestion to "look up" addressed to blind children is typically followed by manual exploration of (upper) space close to the body (Gleitman, 1990;Landau and Gleitman, 1985).Similarly, blind individuals, including those who become blind later in life, cannot access visual information about object-related words and must rely on other modalities for semantic understanding, such as action-related information.
These observations are supported by the bar plots illustrating the number of CAs cells for action (in Fig. 2B) and object (Fig. 3B) words in the different cortical regions across sighted (turquoise), early blind (magenta), and late blind (rose) models as well as by the statistical analyses described below.Note that only the most pertinent and relevant significant main effects and interactions are reported below underscoring the differences in CA distributions between and with the model types.
Further analysis was carried out in the extrasylvian areas to compare the distribution of cell assemblies (CAs) between the three models for each word type.In this additional statistical analysis, a significant threeway interaction was identified, involving the factors Models, Tempo-ralFrontal, and Areas for both action (F 2,48 = 7.34, p = .005,ε = 0.43) (F 4,48 = 7.99, p = .0004,ε = .69)words.
Bonferroni-corrected planned comparison tests (with 12 comparisons and a corrected critical p-value <.0041) revealed the following: For action-related CA circuits, the Early Blind vs. Sighted Models showed a higher density that was evident in the primary visual (V1, p < .0001)and secondary temporo occipital (TO, p = .0027)areas.The Late Blind vs. Sighted Models revealed differences specifically noticeable in V1 (p < .0001)and for the Early vs. Late Blind Models distinctions were observed in V1 (p = .00023)and lateral prefrontal (PF L , p = .00029)areas (see Fig. 2B).For object-related CA circuits, the Early Blind vs. Sighted Model showed a lower density in the primary visual (V1), temporo occipital (TO), anterior temporo (AT), lateral prefrontal (PF L ), and lateral premotor (PM L ) areas (with p-values all less than <0.0001).The Late Blind vs. Sighted Model showed differences in V1, TO, AT, and PF L (with p-values all less than <0.0027) and the Early vs. Late Blind Model differences were present in V1, AT, PM L , and PF L (with p-values all less than <0.00031, see Fig. 3B).
Subsequently, we performed a 4-way ANOVA separately for each model to analyze the distinct topographical word-related CA distributions within each model type.The ANOVA included the factors Word-Type, PeriExtra, TemporalFrontal, and Areas.
Sighted model: For the sighted model, the analysis displayed a significant 4-way interaction between WordType, PeriExtra, Tempora-lFrontal and Areas (F 2,24 = 5.42, p = .014,ε = 0.89), confirming distinctive CA distributions between the two word types.Furthermore, a main effect of Areas (F 4,48 = 370, p < .0001,ε = 0.87) indicated varying CA cell densities regardless of word type distributed across the multiarea network, with higher densities observed in hubs than in secondary areas (p < .0001),and in secondary than in primary areas (p < .0001).To differentiate the CA distributions between the peri-and extrasylvian systems, we conducted a 3-way ANOVAs on the data from these two systems separately.The extrasylvian system exhibited a highly significant interaction of the factors WordType, TemporalFrontal, and Areas (F 2,24 = 28.63,p < .0001,ε = 0.97).In contrast, the perisylvian regions did not show any significant differences between the two word types (F 2,24 = 0.42, p = .59,ε = 0.69).To explore the word-related CA distribution within the sighted extrasylvian regions, we conducted a Bonferroni-corrected planned comparison test (with 6 comparisons and a corrected critical p-value of < .0083).This analysis revealed higher neuron-density for action-related words compared to object words in the three regions of the dorsal motor stream (M1 L , PM L , PF L , p < .0001)and higher neuron-density for object compared to action words in the three regions of the ventral visual stream (V1, TO, AT, p < .0001).
Early blind model: For the early blind model, the 4-way interaction with WordType, PeriExtra, TemporalFrontal, and Areas (F 2,24 = 2.43, p = .12,ε = 0.73) was not significant, however, a significant interaction was found with WordType, PeriExtra, TemporalFrontal (F 1,12 = 28.42,p = .0001,ε = 1).The additional statistical analysis performed separately on the two systems (peri-and extrasylvian) in the blind model yielded comparable results to the sighted model, with no significant differences in the perisylvian areas (F 2,24 = 0.65, p = .50,ε = 0.82) and significant differences in the extrasylvian areas (F 2,24 = 8.29, p = .004,ε = 0.79).To explore the word-related CA distribution within the blind model's extrasylvian regions, we conducted a Bonferroni-corrected planned comparison test (with 6 comparisons and a corrected critical p-value of < .0083).This analysis revealed higher neuron-density for actionrelated words compared to object words in both the three regions of the dorsal motor stream (M1 L , PM L and PF L , p < .0001)and two regions in the ventral visual stream (V1 and AT p < .003).
Late blind model: For the late blind model, similar significant interactions were observed as in the early blind model: a 3-way interaction with WordType, PeriExtra, TemporalFrontal (F 1,12 = 65.12,p < .0001,ε = 1).The additional statistical analysis performed separately on the two systems in the blind model yielded comparable results to the sighted model, with no significant differences in the perisylvian areas (F 2,24 = 0.79, p = .43,ε = 0.77) and significant differences in the extrasylvian areas (F 2,24 = 11.75, p = .0003,ε = 0.94).Also here, to explore the wordrelated CA distribution within the extrasylvian regions in the late blind model, we conducted a Bonferroni-corrected planned comparison test (with 6 comparisons and a corrected critical p-value of < .0083).This analysis revealed higher neuron-density for action-related words compared to object words in both the three regions of the dorsal motor stream (M1 L , PM L and PF L , p < .0001)and only in V1 in the ventral visual stream (p = .00089).

Auditory word recognition: spatio-temporal activation & reverberation results
During the simulated auditory word recognition process, the word related cell assemblies (CAs) were reactivated by presenting the learned auditory word form pattern exclusively to the primary auditory area (A1), while recording the network responses (spiking activity) over time.This allowed us to observe the dynamic activation patterns of the CAs during the simulated word recognition processes (see method section for more details).
The CAs activation time course during auditory word recognition yielded similar results as the structural distribution of CAs across the different regions, as shown in Figs. 2 and 3.In the perisylvian cortex, all three models (sighted, early blind, and late blind) exhibited similar spiking acitivity distributions for both word types, indicating consistent activation patterns in these regions across the different models.However, clear differences were observed in the extrasylvian system.For action words, the early blind models displayed higher spiking activation in the primary visual cortex (V1) and the temporo-occipital region (TO) R. Tomasello et al. compared to the sighted model and the late blind model.On the other hand, the late blind model showed differences in activation only in V1 compared to the sighted model, while the other regions exhibited similar activation patterns between the late blind and sighted models (Fig. 4A&B).For object-related words, the early blind model shows an overall reduction of activity in all the extrasylvian regions compared to both the late blind and sighted models.On the other hand, the late blind model exhibits a reduction of activation only in the visual areas (V1, TO, AT) compared to the sighted model (Fig. S1 supplementary material).As these findings reflect the static CA topographical distribution across the modelled regions described and statistically tested above, we did not conduct statistical analysis on the spiking activity itself.
Instead, our analysis concentrated on the differences in sustained spiking activity, known as the reverberation time, between the three models following stimulation with the learnt auditory word forms in the auditory cortex.The reverberation time (Rtime) was defined as the time between Tmax (maximum spiking activity) and when the activity returned to baseline (0).This duration represents the time during which the cell assemblies (CAs) remained activate after its full ignition during the auditory word recognition processes (for more details see Methods section).For the Rtime, we focused specifically on the action-related words, as the data of the object-related ones are difficult to interpret for the reasons mentioned above.However, find the plots for object words in the supplementary material (see Fig. S1).
The analysis of reverberation time (Rtime) for action words across the six perisylvian language cortices revealed consistent timing among the three models-sighted, early and late blind.However, more significant differences emerged in the extrasylvian cortex, where both the early and late blind models exhibited longer sustained neural activity following action word recognition compared to the sighted model.These differences in the extrasylvian cortex were primarily due to a longer reverberation time in the early-blind model within the V1 region, as compared to both the sighted and late-blind models.The late-blind model also showed extended reverberation time in the V1 region, however to a lesser extent than the early-blind model, as shown in Fig. 4C.
The described observations were confirmed by the statistical analysis conducted between the model types.The 2-way repeated measures ANOVA on reverberation times of action word related CAs indicated a significant interaction among both factors: Models, and PeriExtra (F 2,24 = 7.27, p = .008,ε = 0.74).The one-way ANOVA run on the peri-and extrasylvian systems separately, showed only a significant main effect of Models in the extrasylvian system data (F 2,24 = 4.5, p = .002,ε = 0.98), and no differences between the three models in the perisylvian system (F 2,24 = 0.05, p = .94).Bonferroni-corrected planned comparison tests (2 comparisons with a corrected critical p-value of < .025)showed that both early and blind models in the extrasylvian system showed longer reverberation time as compared to the sighted model (p < .018)with no differences between early and late blind models (p = .88).
The more fine-grained analysis across the areas of the extrasylvian system run with a 3-way ANOVA (Models x TemporalFrontal xAreas) revealed a main effect of Models (F 2,24 = 4.5, p = .022,ε = 0.98), confirming global differences in reverberation times, and a significant 3way interaction was identified involving the factors Models, Tempora-lFrontal, and Areas (F 4,48 = 51.89,p < .0001,ε = 0.82).Bonferronicorrected planned comparison tests (12 comparisons with a corrected critical p-value of < .0041)provided further support for the findings, described above.Specifically, we observed a prolonged reverberation phase for action-related CA circuits in the early blind model compared to both the sighted model and the late blind model, particularly in the primary visual area (V1, p < .0001).Additionally, the late blind model showed longer reverberation in V1 compared to the sighted model (p < .0001),though to a lesser extent than the early blind model.Furthermore, the late blind model exhibited longer reverberation in the temporo-occipital (TO) area compared to both the sighted and early blind models (p < .0005).

Discussion
We examined neuroplasticity of language processing arising from early and late visual deprivation within a brain-constrained neural network designed to simulate associative semantic-referential learning of action and object words by grounding them in sensorimotor information and related cortical systems.The models revealed distinct patterns of neural circuit (or cell assembly) formation across the modelled cortical areas, depending on the timing of visual deprivation.Specifically, the early-blind model compared to the sighted model exhibited more extensive neural recruitment in both the primary visual cortex (V1) and secondary and higher visual cortices (TO) when learning action words.In contrast, the late-blind model displayed neural recruitment confined to V1 and in general to a lesser extent than that observed in the early-blind model (see Fig. 2).These findings align well with existing neuroimaging studies comparing the two populations, which show stronger activation in both the striate and extra-striate visual regions for early-blind individuals, whereas only the primary striate region was implicated for late-blind individuals, particularly in tasks related to generating verbs associated with actions (Burton, 2002;Kujala et al., 1997a;Voss et al., 2008).
Finally, upon conducting a detailed analysis of the activation time course for the neural circuits associated with action words, following stimulation of the primary auditory (A1) area, the early-blind model displayed a notably extended duration of sustained neural activity in the primary visual cortex (V1).This prolonged reverberation time contributed to an increase in the total maximal reverberation time of the extrasylvian system (Fig. 4C).These results suggest that the early-blind model may exhibits enhanced working memory capabilities, specifically attributable to neural changes observed in the visual system.Notably, the findings imply that semantic information, rather than merely phonological word form information, is retained longer in working memory, as no differences were identified in the perisylvian language system.Overall, this aligns well with previous research highlighting cognitive advantages and neural adaptations in early-blind individuals (Amedi et al., 2003;Arcos et al., 2022;Occelli et al., 2017;Pasqualotto et al., 2013;Withagen et al., 2013).Intriguingly, late-blind models also showed prolonged neural reverberation in the V1 region compared to sighted models, however to a lesser extent than the early-blind ones.Below, we discuss the potential neuromechanisms underlying the observed neural changes in the models and their implications for lexical and verbal working memory processes in early-and late-blind individuals.

Which biological mechanisms underlie neuroplasticity in early and late blind individuals?
Our simulations indicate differential neuroplastic changes in the visual cortex for both early-and late-blind models, particularly when processing action-related words.Notably, the early-blind model shows a stronger engagement of both primary (V1) and secondary (TO) visual cortices compared to the sighted models when processing action-related words.These findings are in alignment with a substantial body of neuroimaging research, which has previously explored neural activations during tasks like verb generation and sentence processing across both sighted and early visually impaired groups (Amedi et al., 2004(Amedi et al., , 2003;;Bedny et al., 2011;Burton, 2002;Raz et al., 2005;Struiksma et al., 2011, see also Burton, 2003).Tasks that necessitate verb generation-such as responding with "bake" upon hearing the noun "cake"-activate neural circuits associated with action information processing (Moseley and Pulvermüller, 2014).Likewise, sentence processing frequently involves action-related lexical elements, such as action verbs (Bedny et al., 2011;Röder et al., 2002), making our simulation results particularly relevant to existing empirical studies.Specifically, we mimicked typical learning scenarios for action words, where a word is spoken while the associated action is performed (Tomasello and Kruger, 1992; Vouloumanos and R. Tomasello et al. Werker, 2009) by varying the timing of visual deprivation (i.e., early or late) and compared these scenarios to those with visual input.We observed changes in the neural networks related to these action words, specifically in the visual system, with differential neural material recruitment depending on the time of deprivation.In particular, it shows a larger density of neural material recruitment for early blind model as compared to sighted and late blind models.In contrast, late blind models, which were initially trained with visual input during the acquisition of action words and later experienced visual deprivation, showed more restricted neural matter recruitment.This was confined solely to the primary visual cortex (V1) and constituted only about 50% of the neural recruitment observed in early-blind models.These differences in neuroplastic adaptations between early and late blind models are congruent with prior neurocognitive research that used verb generation tasks documenting similar activation patterns in the visual cortices between these populations (Burton, 2002;Kujala et al., 1997ab;Voss et al., 2008).We believe that the interpretation of simulation results on action-related words is particularly telling about processes in visually deprived individuals, because these often show regular motor performance in context of symbols that would normally be semantically grounded in visual information in undeprived individuals (see the "look" example discussed above).
As previously outlined in the introduction, the mechanisms underpinning the recruitment of the visual cortex in the early-blind model have been delineated by a prior simulation study (Tomasello et al., 2019).It posits two pivotal biological principles.Firstly, neural circuits naturally evolve through a process termed 'Doursat-Bienenstock expansion', in which regular firing of a neuron results in stronger synaptic connections to its neighbours, thus entailing the formation of neuronal circuits that expand into neighboring regions (Doursat and Bienenstock, 2006).Secondly, the absence of variable, uncorrelated sensory inputs in the visual system facilitates this neural growth.The variable uncorrelated inputs refer to the diverse sensory experiences encountered during the learning of action words, such as sensing different objects while learning the word 'grasp' and simultaneously performing the corresponding action (Tomasello and Kruger, 1992).For sighted networks, this variable input serves as 'noise' and therefore as a crucial barrier against neural circuits expansion into specific visual pathways for action processing, a concept further supported by earlier research (Constant et al., 2023;Tomasello et al., 2017Tomasello et al., , 2018)).Note that Tomasello et al. (2019) only observed significant differences between blind and sighted models in primary visual (V1) areas; the difference in the adjacent visual area failed the significance threshold.However, in the current simulation study, we also identified significant differences in the secondary visual (TO) region as well.The reason for these differences lies in relatively stronger activations; the input pattern that had more active cellsincreasing from 19 to 22 in the current simulations.This increase was optimized for recent simulations on fast word-meaning mapping using the same architecture, so that clear cell assemblies could rise when referential information is trained without language input (Constant et al., 2023).However, this small increase in activation due to 3 additional cells was sufficient to observe significance of the changes in the secondary visual region as well.However, this raises the question of whether larger input patterns would lead to even more pronounced changes, an issue that could be explored in future research systematically investigating its effects on neuroplasticity.
However, importantly, the foundational understanding of visual neural recruitment reported by the previous study (Tomasello et al., 2019), sets the stage for how these principles manifest differently depending on the developmental timing of blindness.According to the prevailing hypothesis, significant neuroplasticity may only occur during early childhood, a critical period when both linguistic and visual cortical areas are undergoing maturation (Bedny et al., 2012).In the context of the early-blind simulations, the observed plastic changes may indeed be attributed to the unaltered synaptic modifications between cells in the neural network, thereby making the neural substrates more malleable and available for recruitment.In contrast, networks initially trained with visual inputs undergo substantial synaptic pruning and strengthening through Hebbian learning principles (Artola and Singer, 1993;Hebb, 1949).This process involves the elimination of redundant neural connections via synaptic pruning (long-term depression, LTD) while conserving functionally relevant connections through strengthening (long-term potentiation, LTP).Consequently, subsequent retraining of the network without visual inputs, mimicking the conditions associated with late-onset blindness, the available neural substrate in the visual cortex for further changes becomes limited.This puts a limit to neural reorganization of language, which in the case of our late blind model, restricted the expansion of language circuits only to significant changes in V1, importantly to lesser extent than in the early blin model.These results are congruent with prior research showing similar neural activation between the two blinds groups (Burton, 2002;Kujala et al., 1997a;Voss et al., 2008), however here we go one step forward and offer a mechanistic explanation thereof.Overall, the output of the model is on one side in agreement with the idea that neural plastic changes comparable to those in early visual deprivation are not as pronounced in cases of late blindness, given that a maturated brain is in general less plastic (Bedny et al., 2011(Bedny et al., , 2012)).However, on the other side, the model suggests that neural changes can occur in the case of late blindness, however to a lesser extent, due to the limited availability of neural resources in the visual cortex, therefore leading to diminished but still present Doursat-Bienenstock expansionof neural circuits.However, it's crucial to emphasize once more that such expansion and associated changes are only possible in the absence of uncorrelated neuronal noise, which plays a key role in enabling these changes.Furthermore, a point not highlighted in previous simulation work (Tomasello et al., 2019) is that the neural extension may be facilitated by weakened area-specific inhibition in the deprived visual regions, resulting from the absence of visual neural activity, which could in turn facilitate neural recruitment.Further simulations could tackle the specific role of local and global inhibition in deprived areas and its role in inducing neuroplastic changes.
Furthermore, the simulation results provide testable predictions on the neural changes due to late visual deprivation for object-related words initially grounded in visual experience.Interestingly, upon the onset of simulated late blindness, we observed a reduction in neural activation for these words, especially in the three visual regions: V1, TO, and AT.This reduction may be attributable to the weakening of synaptic links, resulting in the pruning of weakly connected neurons within these object-related circuits during reverberation without visual input.Despite this decrease, neural activation remained more pronounced in late-blind models compared to early-blind ones, which may suggest that recovery of vision through therapeutic intervention may become more feasible in late-blind individuals.In contrast, in early blind models, object-related neural circuits had minimally extended into the visual and motor systems (see Fig. 3 & S1).This observation aligns with the idea that, for early-blind individuals, the acquisition of object words may be more dependent on or grounding in tactile and manual experiences rather than in visual ones, thus the lower level of activation in these models raises questions that require further investigation, particularly concerning the learning of object words through alternative sensory modalities.To some extent this may apply to late blind individuals too, as other modalities might be more strongly involved when it comes to visual information processing once sight is lost.
Regarding the sighted model, associative learning grounded in the sensorimotor system leads to the spontaneous emergence of distinct neural circuit topographies, each reflecting different semantic functions.Consistent with previous simulations (Constant et al., 2023;Garagnani and Pulvermüller, 2016;Tomasello et al., 2017Tomasello et al., , 2018)), the sighted model exhibited category-specific circuit topographies with action-related circuits predominantly extended into motor regions, with minimal or no presence in the visual system (TO and V1) and vice versa for object words.A recent simulation study examining fast word meaning mapping even demonstrated that such distinct neural representations emerge very quickly after only 3 learning episodes or input presentations, with category-specific processing to emerge first in primary regions, followed by secondary and later in hub regions (Constant et al., 2023).These findings aligns with existing neurocognitive research that points to differential cortical activations based on semantic categories (Chao et al., 1999;Damasio et al., 1996;Dreyer et al., 2015;Grisoni et al., 2021;Hauk et al., 2004;Kemmerer, 2015;Moseley et al., 2013;Vukovic et al., 2017).A pivotal factor in the emergence of category-specificity is the presence of uncorrelated variable noise in the primary regions, which stops the excessive spread of neural circuits into neighboring brain areas.Our findings demonstrate that in the context of both early and late blindness, the lack of uncorrelated noise, in turn resulting in weak inhibition, coupled with available neural material, leads neural circuits associated with action words to engage visual cortical regions to different degrees, consistent with earlier studies (Burton, 2002;Kujala et al., 1997a;Voss et al., 2008).

What are the implications of neuroplasticity in the visual system for verbal working memory?
A range of studies have shown that early blind individuals outperform sighted individuals in verbal working memory tasks in recalling longer lists of words, as well as sequences of letters and numbers (Amedi et al., 2003;Arcos et al., 2022;Occelli et al., 2017;Pasqualotto et al., 2013;Withagen et al., 2013).Although this cognitive advantage has been related to the neural recruitment of the visual cortex (Amedi et al., 2003;Park et al., 2011;Raz et al., 2005;Sabourin et al., 2022), the specific implications at the cellular and neural circuits level are still not fully understood.To close this gap, in the current study we examined the activation time course (simulating EEG/MEG activation) in early and late blind models, comparing them to the sighted model.The strength of activation for action words in each area corresponded to the neural circuit distribution described above (Figs. 2 and 3) with the early blind model showing stronger activation in the primary (V1) and secondary visual (TO) regions, while the late blind model exhibited reduced activation confined only to the primary V1 region (see Fig. 4B and S1B).However, intriguing differences were found in the "reverberation period" following the full ignition of the neural circuits, a phenomenon identified as a correlate of working memory (Baddeley, 2003;Fuster, 1995Fuster, , 2009;;Fuster and Alexander, 1971;Leavitt et al., 2017).
Working memory at the cognitive level is the retention of information during complex cognitive tasks, such as speech perception.At the physiological level, this process has been linked with sustained activity of individual cells, so-called "memory cells", which can be explained by reverberation of neural activity in strongly connected neural circuits, which tend to maintain their activation even after the stimulus that triggered them has ceased (Fuster, 1995(Fuster, , 2009;;Kamiński et al., 2017;Kornblith et al., 2017;Leavitt et al., 2017).This mechanism reflects putative brain mechanisms underlying our ability to temporarily hold and manipulate information.Our data reveal that the early blind model exhibited prolonged neural activation for action-related words in the primary visual (V1) area, a response that persists longer compared to both late blind and sighted models (see Fig. 4C).This prolonged activation can be directly linked to the enhanced working memory observed in early blind individuals compared to sighted ones (Amedi et al., 2003;Arcos et al., 2022;Occelli et al., 2017;Pasqualotto et al., 2013;Withagen et al., 2013).Specifically, this aligns with previous research reporting a correlation between the strength of activation in V1 and the number of words recalled during verbal working memory tasks, an effect not observed in sighted individuals (Amedi et al., 2003;Park et al., 2011;Raz et al., 2005;Sabourin et al., 2022).Importantly, these extended periods of activity in V1 led to longer sustained overall activity in the extrasylvian system compared to the sighted model.This finding suggests that what is retained in memory is the semantic referential information to which words refer to, rather than just the auditory-acoustic features of word forms.However, we should add that, if the simulations had targeted the processing of meaningless pseudowords, the perisylvian circuits forming as a consequence would very likely have shown Doursat-Bienenstock expansion towards the visual cortex too.In this sense, the present work does not identify semantics as a unique factor essential for visual cortex reorganization.Overall, our results provide a cellular and synaptic-level explanation for how the recruitment of the visual cortex in early-blind individuals might contribute to the neural underpinnings of the observed cognitive advantages in verbal working memory following visual deprivation.
Interestingly, the late-blind model also showed a longer reverberation time in the primary visual area (V1) when processing action-related words compared to the sighted model, which also resulted in a longer activation time in the extrasylvian system (see Fig. 4C).Note, however, that while the late-blind model showed less sustained activation in area V1 compared to the early-blind model, it showed unexpectedly longer activation in area TO compared to the sighted model -an effect not observed in the early-blind model.Nevertheless, the longer sustained activation indicates that late-blind individuals may also possess better working memory skills compared to sighted individuals, as indicated by the overall longer activation time in the extrasylvian system.Although these results align with some studies suggesting enhanced auditory abilities in late-blind individuals-similar to those observed in earlyblind individuals-when using acoustic cues to perform spatial tasks (Voss et al., 2004), these studies are not directly related to verbal working memory, such as the retention of longer lists of words as observed in early blind individuals.Therefore, more research directly comparing the two populations in this context is desirable.Our findings call for further investigation into the model's predictions concerning verbal working memory in late-blind individuals.Specifically, future research should examine the ability of late-blind individuals to recall longer lists of action-related words or sentences.The feasibility of validating predictions made by brain-constrained neural network models through neurocognitive studies has been demonstrated in prior work.For example, a recent study focused on identifying the cortical locus of working memory for action words (Shebani et al., 2022), which confirmed predictions made by similar simulation studies (Tomasello et al., 2018).Therefore, this establishes a strong basis for further empirical testing of our model's predictions.
For object-related words, our findings revealed no significant differences in reverberation time between the late-blind and sighted models, although there was a reduction in neural material in the lateblind model compared to the sighted one.The observed differences were primarily between the early-blind models and the late-blind/ sighted models (see Fig. S1, supplemental material).It's important to note that interpreting these data is challenging.As mentioned above, this is because visually deprived individuals, such as those who are early or late blind, cannot acquire semantically relevant visual information in the same way that sighted individuals can, but may, instead, rely on other information sources (e.g., tactile, olfactory, auditory).
Moreover, it is important to consider that factors not covered by the present study may contribute to the enhanced verbal working memory observed in cases of blindness.An argument put forward posits that early blind individuals may outperform sighted individuals simply due to specific learning strategies in response to their missing modality or that blind individuals might rely more heavily on working memory processes to compensate for the absence of visual information (Raz et al., 2007).However, our results provide evidence that changes at the neural level contribute significantly to the enhancement of verbal working memory abilities.It may well be that a combination of both factors, including neural changes and potential compensatory strategies, may collectively contribute to it.Overall, our findings elucidate the complex relationship between visual deprivation, neural plasticity and the neural correlate of verbal working memory in the context of blindness.

Limitations and future perspectives
A potential limitation of our study is the relatively modest number of words acquired by the model, totaling only 12 (previous simulations using the same architecture reached up to 30, Henningsen -Schomers et al., 2023).However, the vocabulary of children at a young age is already vast, reaching up to a thousand words by the age of 2-4.Although deep neural networks have shown the capacity to learn a large number of words (e.g., Arisoy et al., 2012;Sainath et al., 2015), they typically employ supervised learning algorithms, such as backpropagation, which have been defined as biologically implausible (e.g., Mazzoni et al., 1991;O'Reilly, 1998) or for not attempting to model realistic properties of the human brain at different levels (see below for more about this).Thus, future research with the current brain-constrained model with realistic neurobiological learning mechanisms could aim to replicate these results with hundreds or thousands of words.This would involve expanding the amount of neural matter in the models by increasing the size of the regions and their connections within the architecture.
Further simulations could also extend to other forms of symbolic acquisition in deprived and undeprived brains.This includes learning from text or variable contexts (Harnad, 2011;Stramandinoli et al., 2012), or by acquiring the meaning of words through the use of various communicative functions, such as requests, questions, or naming (for a review see Tomasello, 2023) or simulating combinatorial grammatical processes to form meaningful sentences (e.g., Brennan and Pylkkänen, 2012;Moro et al., 2001;Pulvermüller, 2012).A further limitation is with respect to the realism of the neural stimulus patterns that induced the formation of neural representations in the model.The patterns were supposed to code for grounded features of objects and actions of the words' referents and to articulatory-and auditory-phonological features of word forms, but the coding did not make explicit the particular features of specific words and referents.Enhancing the model by incorporating neurons coding for real auditory, visual, and actual action motion feature sets as input to the model would be a significant improvement.Although this refinement might not substantially alter the results of the present model, it would notably increase the realism of the model in terms of perceptual, action-related and linguistic processing and representation.
We would also like to mention that several features of the brain model applied are in need of further refinement.For example, the complexity of the visual system is not fully modelled in the present computational model, because, for example, only three visual areas are implemented, and the area-specific differences in local topology representations of different layers are omitted (van Albada et al., 2021) and the full range of between-area connectivity of human cortex is not captured.Thus, further work could examine much more closely the specific neural recruitment of the visual cortex at the micro, cellular level.Still, already in its current form, the present brain-constrained model makes critical predictions and offers an account of neuroplastic changes under visual deprivation.We note that, in spite of the above limitations, the model is solely based on wider biological principles argued to be critical for simulating cognitive functions (see for discussion Pulvermüller et al., 2021), such as Hebbian learning, cortical area structure and at least some aspects of cortical connectivity revealed by neuroanatomical evidence, and regulatory mechanisms.Unlike other deep neural model approaches to language and beyond (e.g., Dell et al., 1999;LeCun et al., 2015;Ueno et al., 2011) which have used either unplausible learning rules (e.g., backpropagation) or been partly inspired but not constrained by multiple neurophysiological and anatomical brain properties, our model offers a mechanistic explanation of the different patterns of neuroplasticity observed in early and late blind individuals in language and symbolic processing.This distinction is important because it grounds the model in a more realistic biological framework, allowing for more accurate simulations of actual neural processes and functions.In this context, it allows a detailed investigation of neuroplasticity associated with visual deprivation at both cellular and cortical levels and offers a novel explanation for the neural changes observed in people with early and late blindness.

Conclusion
The present simulation study was designed to examine the neural plastic changes resulting from early and late visual deprivation and their impact on neural activation for semantic and language processing using brain-constrained neural networks.The findings revealed neural changes confined to the visual system, with the early blind model recruiting a higher density of neural material in both primary visual (V1) and adjacent temporo-occipital (TO) visual regions for action-related words.In contrast, the late blind model showed plasticity to a lesser extent only in the primary V1 area, in line with previous neuroimaging studies described above.The biological mechanisms allowing for such differential recruitment were linked to the general neural mechanism of neural circuit expansion, referred to as the 'Doursat-Bienenstock expansion,' and an imbalance of neural activity in the visual cortex due to the absence of uncorrelated variable noise present during action word acquisition.However, the impact of neural reorganization and thus neural circuit expansion in the deprived visual regions is constrained by the neural material available that is taken upon by prior language processing with vision.Furthermore, we demonstrate that the neural plastic changes observed in the visual cortex for language processing of action words directly impact reverberatory neural activation, a correlate underlying working memory.This is specifically manifested in the sustained neural activation lasting longer in blind models, providing a partial explanation for the superior working memory abilities observed in blind individuals.Here, we bridge the gap between neural mechanisms, conceptual brain functions, and neural activation, by offering a biological explanation for the reorganization of the visual cortex that occurs after sensory loss, either from birth or later in life, and its functional impact on language, semantic processing, and verbal working memory.

Appendix A. Full model specification
Structure and function of the spiking model.Each of the 12 simulated areas is implemented as two layers of artificial neuron-like elements ("cells"), 625 excitatory and 625 inhibitory, thus resulting in 15,000 cells in total.Each excitatory cell "e" consists of a leaky integrate-and-fire neuron with adaptation and simulates a single pyramidal cell representative of excitatory spiking activity in a cortical microcolumn, while its twin inhibitory cell "i" (see Fig. 1C) is a graded-response cell simulating the average inhibitory response of the cluster of interneurons situated in a local neighborhood (Eggert and van Hemmen, 2000;Wilson and Cowan, 1972).The state of each cell x is uniquely defined by its membrane potential V(x,t), specified by the following equation: τ ⋅ dV(x, t) dt = − V(x, t) + k 1 (V In (x, t) + k 2 η(x, t)) (1.1) where V In (x,t) (defined by equation (1.2)) is the net input acting upon cell x at time t (sum of all inhibitory and excitatory postsynaptic potentials -I/ EPSPs; inhibitory synapses are given a negative sign), τ is the membrane's time constant, k 1 , k 2 are scaling values (see Table 2 for the specific parameter values used in the simulations) and η(e,t) is a white noise process with uniform distribution over [-0.5,0.5].Note that noise is an inherent property of each model cell, intended to mimic the spontaneous activity (baseline firing) of real neurons.Therefore, noise was constantly present in all areas, in equal amounts (inhibitory cells have k 2 = 0, i.e., the noise is generated by the excitatory cells in the model for convenience).
V In (x, t) = − k G ω G (A x , t) + ∑ ∀y w x,y ⋅ φ(y, t) (1.2) In Equation (1.2), y varies over all cells in the network, w x,y is the weight of the link from y to x, and φ (y,t) is y's current output (1 or 0), as defined below (2); ω G (A x ,t) is the area-specific (or "global") inhibition for area A where cell x is located (see explanation below and Eq.(3.3)): this term is identical for all excitatory cells x in A and absent for inhibitory cells (k G is as scaling constant).The weights of inhibitory synapses are assigned a negative sign.The output (or transformation function) φ of an excitatory cell e is defined as follows: φ(e, t) { 1 if (V(e, t) − αω(e, t)) > thresh 0 otherwise (2) Thus, an excitatory cell e spikes (=1) whenever its membrane potential V(e,t) overcomes a fixed threshold thresh by the quantity αω.(e,t) (where α is a constant and ω is defined below).Inhibitory cells are graded response neurons as they intend to represent the average impact of a cluster of local interneurons; the output φ(i,t) of an inhibitory neuron i is 0 if V(i,t) < 0 and V(i,t) otherwise.
To simulate neuronal adaptation (Kandel et al., 2012), function ω(e,t) is defined so as to track the cell's most recent firing rate activity.More precisely, the amount of adaptation ω(e,t) of cell e at time t is defined by: Local (lateral) inhibitory connections (see Fig. 1C) and area-specific inhibition are also implemented, realising, respectively, local and global competition mechanisms (Duncan, 2006;Duncan et al., 1997).More precisely, in Eq. (1.2) the input V In (x,t) to each excitatory cell of the same area includes an area-specific ("global") inhibition term k G .ω G (e,t) (with k G a constant and ω G (e,t) defined below) subtracted from the total I/EPSPs postsynaptic potentials V In in input to the cell; this regulatory mechanism ensures that area (and network) activity is maintained within physiological levels (Braitenberg and Schüz, 1998): Excitatory links within and between (possibly non-adjacent) model areas are established at random and limited to a local (topographic) neighborhood; weights are initialized at random, in the range [0, 0.1].The probability of a synapse to be created between any two cells falls off with their distance (Braitenberg and Schüz, 1998) according to a Gaussian function clipped to 0 outside the chosen neighborhood (a square of size n = 19 for excitatory and n = 5 for inhibitory cell projections).This produces a sparse, patchy and topographic connectivity, as typically found in the mammalian cortex (Amir et al., 1993;Braitenberg and Schüz, 1998;Douglas and Martin, 2004;Kaas, 1997).
The Hebbian learning mechanism implemented simulates well-documented synaptic plasticity phenomena of long-term potentiation (LTP) and depression (LTD), as implemented by Artola, Bröcher and Singer (Artola et al., 1990;(Artola and Singer, 1993) ).In the model, we discretize the continuous range of possible synaptic efficacy changes into two possible levels, +Δ and − Δ (with Δ≪1 and fixed).Following Artola et al., we defined as "active" any (axonal) projection of excitatory cell e such that the estimated firing rate ω E (e,t) of cell e at time t (see Eq. (3.2)) is above ϑ pre , where ϑ pre ∈ [0,1] is an arbitrary threshold representing the minimum level of presynaptic activity required for LTP to occur.Thus, given a pre-synaptic cell i making contact onto a post-synaptic cell j, the change Δw(i,j) inefficacy of the (excitatory-to-excitatory) link from i to j is defined as follows: R. Tomasello et al.

Fig. 1 .
Fig. 1. (A) illustrates the structural organization and connectivity of the 12 frontal, temporal, and occipital cortical areas relevant for word meaning acquisition.The perisylvian cortex encompasses the inferior-frontal articulatory-phonological system, indicated by the red colors, while the extrasylvian areas consist of the lateral dorsal hand-motor system (ranging from yellow to brown) and the visual "what" stream responsible for object processing (depicted in green).The numbers indicate the Brodmann Areas (BAs), and the arrows (in black, purple, and blue) represent the long-distance cortico-cortical connections as established in neuroanatomical studies.(B) presents the overall model structure with corresponding colors.(C) displays the micro-connectivity structure of an excitatory neural element ("e") with local excitatory links (grey) and lateral inhibition from neighboring elements (dark purple).The underlying gray cells represent an inhibitory cell ("i"), which inhibits neighbors proportional to the total input it recieves from the neighborhood shaded in the darker purple.Figure taken from Tomasello et al., (2018).

Fig. 2 .
Fig. 2. Distributions of cell assemblies (CAs) during simulation of action word learning in the 12-area network under sighted (turquoise), early blind (magenta), and late blind (rose) conditions.(A) Each set of 12 squares (in black) corresponds to one specific network area, with colored pixels denoting the distribution of CA neurons across the 12 network areas in a schematic brain following sensorimotor pattern presentations.(B) Mean numbers of cell assembly (CA) neurons in different cortical areas of the sighted (turquoise bars) and early blind (magenta bars) and late blind (rose bars) after simulating the learning of action-related words during word production.Error bars represent standard errors across networks.Grey points display the outputs of each of the 13 networks per model type.Asterisks (*) indicate significant differences in the number of CA cells between the sighted early and late blind models within a given area (Bonferroni-corrected planned comparison tests, 12 comparisons; critical threshold p < .0041).

Fig. 3 .
Fig. 3. Distributions of cell-assemblies (CAs) during simulation of object word learning in the 12-area network under sighted (turquoise), early blind (magenta), and late blind (rose) conditions.(A) Each set of 12 squares (in black) corresponds to one specific network area, with colored pixels denoting the distribution of CA neurons across the 12 network areas in a schematic brain following sensorimotor pattern presentations.(B) Mean numbers of cell assembly (CA) neurons in different cortical areas of the sighted (turquoise bars) and early blind (magenta bars) and late blind (rose bars) after simulating the learning of object-related words during word production.Error bars represent standard errors across networks.Grey points display the outputs of each of the 13 networks per model type.Asterisks (*) indicate significant differences in the number of CA cells between the sighted early and late blind models within a given area (Bonferroni-corrected planned comparison tests, 12 comparisons; critical threshold p < .0041).

Fig. 4 .
Fig. 4. (A) Spatio-temporal activation patterns of the six extrasylvian model areas for action-related words.Each curve represents area-specific spiking activation dynamics over time for the sighted, late-blind, and early-blind models.(B) Differences in spiking activation between the three models within the primary visual (V1) and temporo-occipital (TO) cortical regions.(C) Reverberation time (sustained activity) after full ignition over time for action-related words for each extrasylvian areas and averaged across all regions.
τ ADAPT is the "adaptation" time constant.The solution ω(e,t) of Eq. (A3.1) is the low-pass-filtered output φ of cell e, which provides an estimate of the cell's most recent firing-rate history.A cell's average firing activity is also used to specify the network's Hebbian plasticity rule (see Eq. (4) below); in this context, the (estimated) instantaneous mean firing rate ω E (e,t) of an excitatory neuron e is defined as: τ Favg ⋅ dω E (e, t) dt = − ω E (e, t) + φ(e, t) (3.2)