Concurrent mapping of brain activation from multiple subjects during social interaction by hyperscanning: a mini-review

Social interaction plays an essential role in acquiring knowledge and developing our own personalities in our daily life. Meanwhile, functional magnetic resonance imaging (fMRI)-, electroencephalograph (EEG)-, and functional near inferred spectroscopy (fNIRS)-hyperscanning, enables us to concurrently map brain activation from two or more participants who are engaged in social interaction simultaneously. In this review, we first highlight the recent technologies advances and the most significant findings towards social interaction by using the hyperscanning method. In addition, we also illustrate several well-designed hyperscanning tasks that have been extensively adopted for the study of social interaction. Basically, hyperscanning contains six categories of experimental paradigms that can track the interactive neural process of interest. Furthermore, it contains two main elucidated neural systems which are involved in social interaction, including the mirror neuron system (MNS) and mentalizing system (MS). Finally, future research directions and clinical implications that are associated with hyperscanning are also highlighted and


Introduction
Our communication or social interactions with one another is essential for us to acquire knowledge and to develop our own personalities. Although the forms of social interactions are very broad including imitations, exchanges, completions and cooperation, as well as making decisions (1), we basically exchange our thoughts and ideas in two different manners. The dominant manner is through our sophisticated languages, which is also a characteristic that distinguishing ourselves from other creatures (1,2). The other way is by using non-verbal signs, such as our gestures and facial expressions, which can provide us with additional auxiliary information for social interactions (1,2).
Interestingly enough, although our social nature has been shaped for hundreds and thousands of years, neuroscience studies only just began to shed light on social interaction in the recent years (1,3,4). More importantly, previous neuroimaging studies have exhibited two basic limitations in elucidating neural correlates of social interactions. The first restriction is from the low ecological validity, since most of the previous experiments were performed in an enclosed room, in which individuals were instructed by computer programs, to complete the test tasks (2). However, this is not the case for social interactions in real life, in which individuals need to talk and act with each other simultaneously in a more natural way. Therefore, further neuroscience or neuroimaging studies should be performed by using a more realistic experimental paradigm which can duplicate a reallife situation. The other limitation is that previous studies can only acquire brain data from a single participant each time (5). However, as two or more individuals are engaged in social Review Article Concurrent mapping of brain activation from multiple subjects during social interaction by hyperscanning: a mini-review interactions, it is essential to conduct a concurrent recording from multiple subjects with multiple setups rather than to perform it in isolation (1,2,5).
In this review, fMRI, EEG and fNIRS hyperscanning neuroimaging technologies which have engaged in social interaction are first introduced. Then, the representative experimental paradigms that were extensively adopted in hyperscanning, are also summarized in detail. Subsequently, two core neural systems involved in social interactions are carefully demonstrated. One is MNS, which consists of the primary motor, sensory cortex and parietal cortex, and is responsible for the imitation process; the second one is MS comprising the TPJ (temporal-parietal cortex) and PFC (prefrontal cortex), which is in charge of a more complex cognitive process. More importantly, the future of research perspectives and clinical implications of hyperscanning, are stated clearly in the final section.

Hyperscanning neuroimaging techniques fMRI hyperscanning
As it is hard to place two or more participants into one fMRI tube, two or more fMRI machines should be utilized for an fMRI hyperscanning method to simultaneously record multiple participants' brain signals. In that circumstance, two or more remote fMRI apparatus can be connected by an intranet, while the data sets are stored in a host client ( Figure 1A) (6). To date, several fMRI hyperscanning studies (49)(50)(51)(52)(53)(54)(55) have been conducted to inspect the inter-brain synchrony ( Table 1). For example, neural correlates of trust between two individuals, had been examined by fMRI hyperscanning. They had discovered that trust is an essential social process, involved in all human interaction (54). Inarguably, fMRI hyperscanning has exhibited its advantages in mapping the coherence of brain regions which were associated with social interaction with high structural accuracy and excellent imaging depth. However, it is not accessible and available for everyone, because of the high cost of multiple fMRI setups. More importantly, the ecological validity is also relatively low, since the lab was under the controlled circumstances for fMRI, and is significantly different from real life.

EEG hyperscanning
Since the electrical activity of human brain was firstly recorded by Hans Berger in 1924, EEG has become a core neuroimaging tool in the study of cognition and diseases (59,60). More importantly, EEG is also one of the most powerful techniques for noninvasively exploring neural oscillations (61), in which the EEG signals are originated from the synchronized synaptic activity in populations of cortical neurons (62). Although EEG has been extensively utilized for mapping single individual's brain dynamics underlying specific cognitive tasks, the potential of EEG in exploring the inter-brain interactions or inter-brain connections has not been fully exploited.
Recently, a number of EEG hyperscanning studies ( Table 2) were conducted , aiming to reveal the complex brain interactions between two or multiple participants, as illustrated in Figure 1B (26). These studies exhibited that EEG hyperscanning can map the momentto-moment interactions between two or more individuals simultaneously, which can elucidate how co-variations of the tested individuals' brain activations are correlated with their social interactions. However, despite EEG being suitable to inspect inter-brain synchronization due to its high time resolution, it is still very challenging for EEG to capture the neural activity from deep brain structures.

fNIRS hyperscanning
fNIRS is also a noninvasive and affordable neuroimaging technique, which utilizes the near-infrared light to image brain activation, by measuring the concentration changes of oxyhemoglobin (HbO) and deoxyhemoglobin (HbR) (63)(64)(65). In addition, fNIRS has exhibited its unbeatable advantages in inspecting infants or children's brain activation (66) since it is relatively more tolerant with movement artifacts. More importantly, fNIRS hyperscanning ( Table 3) can ideally be applied to a natural scenario (33,35,36,39), as illustrated in Figure 1C. Although fNIRS has a better temporal resolution when compared to fMRI, it has the low spatial resolution and limited capability to detect deep brain structures.

Hyperscanning paradigms adopted in social interaction
Altogether, there were about six categories of experimental paradigms that were routinely used by the hyperscanning method in the investigation of social interaction.

Imitation tasks
The first category is the imitation tasks, during which one participant imitates the others' movements or behaviors. Although we cannot request one participant to teach the others how to perform specific tasks in the laboratory, we can still instruct one participant to simulate the other individual's actions or behaviors ( Figure 2A). For example, in one EEG hyperscanning study (17), the participant was instructed to imitate the counterpart's meaningless hand movements. The results showed that inter-brain synchronization of right centroparietal regions at alpha-mu band was strongly correlated with the interactional synchrony ( Figure 2B).

Coordination tasks
The second category is the coordination tasks, in which two or more participants need to try their best to act in a synchronized manner. Interestingly, behavioral synchronization in our daily life is one mechanism through which we coordinate our behaviors during social interaction. For example, when we are walking together, our footsteps might be unconsciously synchronized with one another even though our foot lengths and our intrinsic cycles are totally different (22). In addition, coordination/ joint movements can also be synchronized, such as selfpaced rhythmic finger movements (15,16). In particular, a number of EEG or fNIRS hyperscanning studies have   Brain signals implicated in social decision making are modulated by the estimates of expected utility and become correlated more strongly between interacting players who reciprocate one another Main discoveries were directly extracted or adapted from the article's abstracts, which is also applied to the Tables 2,3. The situation of subjects in fMRI hyperscanning is two subjects lying in fMRI tubes separately. PNAS, Proceedings of the National Academy of Sciences of the United States of America; BBR, Behavioural Brain Research; N/A, not available means the authors did not explicitly depict their subjects' relationship or the exact numbers of gender pairs even though some studies addressed the overall numbers of genders. The power of the alpha frequency band (8-13 Hz) in the right temporoparietal region immediately after seeing a round outcome significantly differed between face-to-face and faceblocked conditions and predicted whether an individual would adopt a 'cooperation' or 'defection' strategy  PLV interpersonal synchronization is mediated in part by a lower-level sensory mechanism of speech-tobrain synchronization, but also by the interactive process that takes place in the situation per se  Hand-holding during pain administration increases brainto-brain coupling in a network that mainly involves the central regions of the pain target and the right hemisphere of the pain observer. Moreover, brain-tobrain coupling in this network was found to correlate with analgesia magnitude and observer's empathic accuracy Main discoveries/contributions were directly extracted or adapted from the articles' abstracts. PNAS, Proceedings of the National Academy of Sciences of the United States of America; SCAN, Social Cognitive and Affective Neuroscience; N/A, not available means the authors did not explicitly depict their subjects' relationship or the exact number of gender pairs even though some studies addressed the overall numbers of genders.
been performed to examine the neural synchronizations in coordination/joint movements (9,15,16,20,22,24,29). One example was illustrated in the previous reports (20,24), in which dyads were instructed to synchronize with each other by counting in their mind rhythmically. This study also examined how the social context such as threats, or oxytocin, affected the coordinated movements (20,24), which showed that oxytocin can enhance inter-brain synchronization to facilitate social coordination (20).

Eye contact/gaze tasks
The third category was the eye contact or gaze tasks, in which dyads are instructed to look in each other's eyes, or look towards the third object. Interestingly, the mutual gaze or eye-to-eye contact has the functions that offer pivotal social cues in social interaction and communication. In particular, a universally recognized social link or a pipeline can be established during a non-verbal communication through eye contact or a mutual gaze (35). Importantly, we can infer the others' intentions as well using eye-to-eye contact (50). Further, eye-to-eye contact, through which reciprocal information between individuals are dynamically exchanged, provides a great opportunity to model the neural mechanisms of human interpersonal communication by hyperscanning neuroimaging techniques (35,50). For example, an interesting study was performed, in which the dyads were instructed to look at each other's eyes or eyes in portraits (35). And they discovered that the interbrain coherence of the left superior temporal, middle temporal and supramarginal gyri as well as the pre-and supplementary motor cortices were significantly increased in the eye-to-eye contact case when compared to the data from the eye-to-picture gaze case (35).     Main discoveries/contributions were directly extracted or adapted from the articles' abstracts. PNAS, Proceedings of the National Academy of Sciences of the United States of America; SCAN, Social Cognitive and Affective Neuroscience; N/A, not available, meaning the authors did not explicitly depict their subjects' relationship or the exact numbers of gender pairs even though some studies addressed the overall numbers of genders; WTC, wavelet transform coherence; GC, granger causality; PPI, psychophysiological interaction.

Economic games involving game theory/exchange tasks
The fourth category is playing economic games/exchange tasks, in which one participant provided an economic offer while the counterpart need to make a decision on whether they wanted to take it or not. Game theory can offer a rich collection of both behavioral tasks and well-specified models aiming to articulate social interactions where decision-makers have to interact with one another (58). By contrast, exchange is the most basic type of social interaction, which involves a social process whereby social behavior is exchanged for some type of reward for equal or greater value. One instantiation of game theory/exchange is the trust game, in which one participant need decide how much money should be returned to your opponent (52), as illustrated in Figure 3A. One hyperscanning study illustrated that the paracingulate cortex is critically involved in building a trustworthy relationship (54). In addition, the prisoner's dilemma game was also utilized as a task for the design of hyperscanning. This required the two participants © Quantitative Imaging in Medicine and Surgery. All rights reserved. Quant Imaging Med Surg 2018;8(8):819-837 qims.amegroups.com to make their own decisions simultaneously ( Figure 3B). The prisoner's dilemma game usually consists of three experimental conditions: win-win, lose-lose, and a tit for tat case (14,21,27). Interestingly, previous reports have demonstrated that the decision to defect can be decoded in advance by monitoring the changes of connectivity patterns, as shown in Figure 3C (21). Further, the ultimatum game is also applied to the paradigm design for hyperscanning ( Figure 3D), in which one participant need decide to take your opponent's offer or not (34,55).

Cooperation and competition tasks
The fifth category is cooperation and competition tasks, in which participants need to achieve a goal cooperatively or competitively. Cooperation and competition tasks are ubiquitous, in which goals should be obtained efficiently. One representative paradigm used in the hyperscanning studies was to explore the brain synchronization's underlying cooperation or competition, as plotted in Figure 4A, which consisted of three conditions: the cooperate, competitive, and control conditions (31). Interestingly, this paradigm was first initiated in 2012, and later was utilized to examine the brain coherence differences between groups of the same sex and of mixed sexes ( Figure 4B) (40,44), groups with lovers and strangers ( Figure 4C) (42), or groups with parents verse the child and the stranger verse the child (38). In addition, other paradigm designs were also formulated to inspect the inter-brain synchronization engaged in cooperation and competition (13,32,41,46).

Natural scenario
T h e p a r a d i g m s m e n t i o n e d a b o v e d o o f f e r g r e a t opportunities in inspecting the inter-brain dynamics during social interaction. However, only social interaction through a natural scenario can reflect the real situations in our daily life, which is also the dominated way of communication and thought exchange. An interesting test has been performed, in which two participants were instructed to have conversations with each other while their neural data was concurrently recorded (25,26,28,33,36,39,51). Intriguingly, their findings showed that inter-brain synchronization was higher for a face-to-face talk case, as compared to that of the backto-back talk case (39). In addition, neural synchronization under other circumstances was also explored, such as music playing (18,19), singing together (43), playing games (47,48) or taking a class (10). For example, one study showed that during class time, students' brainwaves are more in sync with each other while they are highly engaged in the teaching (10).

Neural systems involved in hyperscanning during social interaction
Two main neural systems are involved in inter-brain connections (1,2,5). One is the MNS, which includes the primary motor cortex and posterior parietal cortex. The second one is the MS, which consists of the temporal-parietal junction (TPJ), precuneus and prefrontal cortex (PFC).

MNS
When we imitate or even just see the others' actions or movements, neurons in the MNS are fired. This phenomenon was discovered in both monkey and human brains (56). In human brains, the MNS ( Figure 5) consists of the inferior frontal gyrus (IFG) and inferior parietal lobule    A B C (IPL), which is related to language, motor and sensory detection. In addition, the superior temporal gyrus (STG) also plays an essential role in imitations, which can provide additional visual information inputs (56), in which the encoded information of imitated actions is first transformed into a more sophisticated visual representation through STG and then is delivered to the IPL. Once the IPL is activated, potential movements are able to be executed. In addition, the IFG is also activated to manipulate the potential action, which can provide additional supplemental information, such as the goal of the action. The present hyperscanning studies associated with imitation show empirical evidence that MNS is involved in dual participant imitation (8,17). For example, one study demonstrated that when two participants were synchronized in behaviors, their brains were also tuned to the same frequency. Consequently, an inter-brain sychornizing network in the alpha-mu band between the right centroparietal regions was produced ( Figure 2B).

MS
Besides imitating others' actions, we might as well try to understand others' intentions or emotions by their gestures, behaviors and facial expressions, which is termed as mentalizing (57,68). The TPJ and PFC particularly, and the dorsomedial PFC (DMPFC) are the two main brain regions associated with the mentalizing process (68).
The TPJ is the boundary brain region between the temporal and parietal cortex, which is labelled in a red circled area in Figure 5 (67). As depicted in a previous study (69), the mentalizing process contains two steps. In the first step, the static social images are coded as a neural representation from the extrastriate body area. For step two, the encoded representations are constructed to generate moving social entities, and are then incorporated into a context for interpreting the intention. Interestingly, several fNIRS hyperscanning studies have highlighted the TPJ as their region of interest (33,34,39). For example, in an adapted version of the ultimatum game, the interpersonal brain coherence for the right TPJ was higher for underlying the face-to-face condition, than that of the face blocked condition. This indicated the functions of right TPJ, is collaborative in social interactions (34).
The PFC, likes a commander, is also involved in the mentalizing processes. It is responsible for the planning, regulation, integrating of information, and other high cognitive functions. Accumulated neuroimaging evidences  have shown that the PFC was related to interpersonal brain synchronization (31)(32)(33)(34)39,41). For example, the inter-brain coherence in left inferior frontal cortex was significantly higher in face-to-face dialogues, than those from back-toback dialogues, face-to-face monologue, or back-to-back monologues (39).
In summary, both MNS and MS play vital roles in social interactions, although the relationship between them is still unclear. A few studies demonstrated that they are collaborated (70), whereas additional reports also stated that MNS is inferior to MS (57).

Multimodality hyperscanning
Further investigation should be performed by using EEG-fNIRS, fNIRS-fMRI or EEG-fMRI hyperscanning techniques, since the multimodality neuroimaging methods can take advantage of the high temporal resolution of the EEG/fNIRS and the high spatial resolutions of an fMRI. To date, hyperscanning studies that utilize two or three neuroimaging modalities (e.g., EEG & fNIRS fusion) have not been extensively examined. Interestingly, multimodality can provide us new perspectives that a single modality cannot offer, because each neuroimaging method possesses its own advantages. For example, our group recently discovered that a combed EEG and fNIRS can enhance the sensitivity of lie detections (71). Although this is not a hyperscanning study, it enlightens us to more intriguing results or findings about the inter-brain dynamics which can be identified by applying a multimodality hyperscanning method for testing social interaction. In particular, more linked neural information can be revealed, based on the fused measures from neurovascular and neuroelectrical signals, which enable us to gain a more full understanding of the inter-brain effects during social interactions in our daily life.

Applications of hyperscanning in education and interrelationships
For most of the hyperscanning studies, neural data were recorded with two participants simultaneously, although several studies were also conducted by acquiring the brain signals from three or multiple participants (10,33,36). However, inspecting multiple individuals' brain dynamics is crucial in some circumstances such as for the teaching and education settings. For example, one study demonstrated that students' brain-to-brain group synchrony can track not only classroom engagement but also classroom social dynamics (10). But they did not explore the neural dynamics between teachers and students. In addition, the teaching style that can stimulate students' inter-brain synchronization by inspecting the neural dynamics between teachers and students should be further investigated.
Interestingly, hyperscanning can also be applied to examining the interactions between an adult and a child (11,38) and interpersonal relationships, such as lovers (12,25,42). For example, lovers who held their hands together exhibited their capability in alleviating their pain perception (12).

Clinical implications
The hyperscanning method has exhibited a potential for the study of inter-brain synchronization of normal individuals during social interaction. In contrast, hyperscanning of abnormal individuals might manifest an aberrant, or null interpersonal dynamics for disorder detection, particularly those in social deficiencies such as autism and schizophrenia. For example, a previous hyperscanning study showed that autism patients have the ability in recognizing their counterparty's intentions, but they cannot convey this information (72). To date, inspecting the interpersonal neural synchronizations among aberrant populations is still lacking. As a result, it is urgent for us to elucidate the neural mechanisms underlying those social deficits disorders by hyperscanning (1), which can pave a new avenue for improving the detection and treatment of neurological or psychiatric disorders.