Modal Interaction in Interfaith Dialogue: a semiotic study

Linguistic studies, accounting for the communication of meaning within texts, faced challenges arising from the consideration of semiotic modes other than language, in interaction with each other and with language – such as gesture, gaze, dress, visual and aural art, image-text relation, page-layout, etc. This increased the need for studying language in interaction with other modes. Therefore, there has been a focus on multimodality in an attempt to make the point that investigating representational modes other than language is essential and central to actual forms of communication everywhere, not simply some kind of marginal concern . Accordingly, in this endeavor, the interaction among modes, specifically language and gestures, is investigated to present an answer to the main questions underlying this research concerning whether meaning is construable through the analysis of one semiotic mode only or there is a need to investigate other accompanying modes. Another question is related to the distribution of semiotic labor among the co-deployed modes within the same multimodal text, whether it is equal or not . . key word : model , Methodology , interaction, Reverend


Introduction
It is very important to investigate the way different modes interact in multimodal texts and communicative events to gain an insight of meaning production and perception. Changes to the semiotic modes available to people in communication and the way they combine and interact can result in a different interpretation of meaning. As such, the way meaning is configured and construed is crucial. Kress & Leeuwen (1996:34) clarify that through the transcoding between semiotic modes, a better and a more adequate understanding of the communicative process is gained. They take all semiosis as multimodal. Lemke (2002:302) takes the same position and confirms that no meaning is construable through analyzing one semiotic system only (see also Fei,2004;Baldry & Thaibault,2009). Based on that, language is seen as a single mode participating, in addition to other modes, to the multimodality of texts, that is, language is viewed as a part of a much larger multimodal system.
Based on that, this study aims to identify the intersemiotic mechanisms and concepts operating in the co-instantiation of language and gestures. This will shed light on how the semiotic labor (creating meaning) is divided among the semiotic modes of language and gestures. Through the pursuit of fulfilling the aims of the study, an answer to the main question underlying this effort, whether meaning is construable through the analysis of one semiotic mode only or not, will be introduced.
This effort is interwoven based on the assumption that meaning is not fully construable based on the analysis of only one mod, and that the semiotic labor of meaning-making is not allocated equally among language and gestures in interfaith dialogue. This leads to the second assumption, that is, in interfaith dialogue the verbal mode takes the leading role in the interactional process with gestures.
To perform the analysis of the data, a theoretical framework will be built for this purpose. The work of Lim (2004), Liu andO'Halloran (2009), andUnsworth (2006) constitute the blueprint for the framework underlying the analytical process: the convergence or divergence of the modal meaning is investigated based on Lim (2004); the intersemiotic mechanisms operative in the modal marriage are tackled drawing on Liu and O'Halloran (2009); the intersemiotic semantic concepts resulting in the emergent meanings are investigated based on Unsworth's (2006) work. To make the selected gestures clear, a shot of each gesture is taken using a software program and presented in a number of figures presented in the analysis of the data.
This study can hold significance to many other fields other than multimodal studies. It can be of benefit for those analyzing speeches of any kind such as political ones, and pedagogical discourse in lectures.

Multimodality and Intersemiosis
This research coincides with a rapid increase in the stream of research into multimodality. Both linguists and social semioticians have long recognized the need for linguistic and semiotic works to move beyond verbal language (Halliday & Hasan 1976;Hodge & Kress 1988;O'Halloran 1999;Martin 2001). Since the introduction of the seminal work on visual analysis by Kress & van Leeuwen (1990/1996 focusing on photographic image and O'Toole's work (1994) in which the approach is more focused on paintings, architecture, and sculpture, significant advances have taken place in theoretical description and methods of analysis and enormous amount of studies have begun to look beyond the verbal mode by investigating other modes that also contribute to the meaning-making process both individually and when combined with other modes.
In investigating intersemiotic relations among modes, Royce (1998Royce ( , 2002 offers a theory of intersemiotic complementarity, while Martinec & Salway (2005) offer a systemic approach to the analysis of image-text relations. More recently, developments have been made in the area of multimodal document transcription (O'Halloran 2004;Bateman 2008). Also, Unsworth (2001) and Kress (2003), among others, have investigated multiliteracies in the school curriculum.
In much of the work reviewed thus far, two significant approaches to multimodal research can be identified. The first approach is found in the early works of Kress andvan Leeuwen (1996) andO'Toole (1994), where various meaning-making systems are analyzed separately and then the way they interrelate is investigated. The second approach is found in more recent works where an attempt is made to directly analyze the interrelations among different modes of meaning to identify common semiotic principles (Kress & van Leeuwen, 2001;Lemke, 1998bLemke, , 2002bO'Halloran, 2005;Royce, 1998;Thibault, 2000). The challenge in the second approach is to take the total contribution of the combined systems of meaning into consideration, rather than adding up the singular contribution of each of the semiotic systems involved.
In line with the recent works, this study attempts to analyze the interrelations between the modes of language and gestures in interfaith dialogue to reveal the way the emergent meaning is tailored via the interaction of those two modes.

Methodology
This work is performed following a qualitative approach where the main aspects within each adopted theory are presented and applied to selected examples taken from the data of the study. But, before going through the theoretical framework followed in this study, it is necessary to shed light on the concept of gestures and their classification.

Gestures: concept and classification
Gestures represent one of the modes for materializing meaning. Kendon (1996:8) suggests that gestures are constituted through phases of bodily movements with characteristics allowing them to be recognized as meaning bearers and part of a willful communicative action. The coinstantiation of language and gestures can be viewed as a process of resemiotization, a concept presented by Iedema(2001:41) who defines it as "meaning-making shifts from context to context, from practice to practice, or from one stage of a practice to the next". That is, the reformulation of information as moving from one mode to another.
Drawing on Kendon's (2004) study, Lim (2011) classifies gestures into communicative and non-communicative (or performative) gestures. According to Martinec's (2000:243) view, performatives do not serve any semiotic function and are not semantically loaded or willfully conducted such as picking up a pen, or writing on a whiteboard, etc. Lim (2011:156) sub classifies communicative gestures, in terms of their relation to language, into language-corresponding gestures, those that co-occur with language but their interpretation does not demand an access to the meanings realized through language. The languagecorresponding gestures could represent the realization of three aspects: processes, participants, and circumstances. The language-independent ones are gestures that occur with the absence of language and they make meaning on their own. While the language-dependent gestures hold meanings that demand an access to the meanings realized through the accompanying language to be interpreted.
Adopting a systemic functional approach, Martinec (2000:243) classifies gestures into presenting actions, representing actions, and indexical actions. Presenting actions are similar to Lim's performative gestures. Representing actions are, according to Martinec (2000:244), a means of representation with a conventional signifying function that is recognized either universally or within a semiotic community. Compared to Lim's (2011) types of gestures, they can be considered as communicative and sub-classified as either language-corresponding or language-independent gestures based on their relationship with the accompanying language; indexical actions co-occur with language and can be interpreted only by access into the meanings realized by the accompanying language. They are classified as communicative and described as language-dependent gesture. In this endeavor, the classifications of Martinec (2000) and Lim (2011) are adopted.

Intersemiosis: verbal-gestural interaction
Lim (2011) has investigated the process of intersemiosis between language and gestures from the perspective of contextualizing relations. He views modal cohesion as the ongoing process of contextualization. Lim (2004) defines contextualization as "the meaningful relation present between two modalities (p.239). He classifies contextualizing relations into two types, co-contextualization and re-contextualization based on the extent of approximation in the meaning realized by each of the interacting modes. Lim explains that in the case of co-contextualizing relations, the meaning conveyed through one mode reflects or is close to the meaning realized by another mode through convergence, while in the re-contextualizing relations the meanings of the two modes are at odd and not related in any way, in other words, divergent.
Moreover, Liu and O'Halloran (2009:372) propose two mechanisms to account for meanings resulting from semiotic cohesion, these are intersemiotic parallelism, drawing on Hasan (1985), and intersemiotic polysemy. Liu and O'Halloran define intersemiotic parallelism as a cohesive relation which interconnects modes when they share a similar form, while intersemiotic polysemy is based on sharing multiple related meanings which results in co-contextualization relations. An example of parallelism, from Lim's (2011:332) investigation of gestures in pedagogic discourse, is that the teacher whose pedagogic discourse is analyzed makes the action of flipping with her right hand when she says, "You flip through a magazine".
These intersemiotic mechanisms can be related to Unsworth's (2006:60) description of ideational meanings resulting from intersemiosis in his study of language and images integration. Unsworth identifies three ideational meanings, concurrence, complementarity, and connection. In relation to the intersemiosis between language and gestures, only the first two are relevant. Ideational concurrence refers to ideational equivalence between modes, while ideational complementarity is an indication of the state where the meanings of integrated modes complement each other in spite of their difference. According to Lim (2011), intersemiotic polysemy results in complementarity in the emergent meaning, while intersemiotic Parallelism leads to concurrence in the emergent meaning. O'Halloran (2007:373) presented the semiotic concept "semiotic metaphor" that is "an intersemiotic process whereby a shift in the functional status of an element arises through a shift between semiotic modes". semiotic metaphor is one of the emergent meanings that can result from modal interaction. An example from Lim (2011) may further illustrate the concept when the teacher, whose lesson is being investigated, introduces her lesson by saying: "In the course of today's lesson, we will go through step one to six". She co-instantiates her linguistic message with a language-correspondent gesture of oscillating both hands held at the chest level as she describes "the course of today's lesson". The intersemiotic parallelism, in operation in the cocontextualising relations, indicates a gestural reformulation of the linguistic selection of "course" in ideational concurrence. The resemiotization of "course" which is a linguistic entity in the verbal mode into a gestural process realized by the oscillating movement is a Semiotic Metaphor.

Data and Analysis 4.1 Data Description
The main topic of the data in concern represents "Interfaith Dialogue" with an aim of expanding the space of knowledge about "the other". The data is in the form of a dialogue in which a number of panelists, embracing different faiths, discuss matters related to each other's faith in the form of questions posed by the moderator and answered by the panelists successively.
The main method of data collection adopted in this study is a stratified random method, where a large number of video recordings were previewed, and then classified according to several variables. Those variables are related to the micro-genre of the communicative event, and the number of faiths being discussed in each video recording, as well as the way the whole event is organized. Afterwards, and in consistency with the aims of the study, one type was chosen for analysis from which a number of random selections were made. The data was transcribed manually by the researcher through repeatedly going through the recordings, listening, and annotating the speech of each panelist.
The selected communicative event is organized and directed by the moderator, Reverent Amy Heller. The event is held at St. Michael and All Angels Foundation. The whole dialogue goes on for about an hour and a half in which the panelists present aspects related to their faiths in the form of answers to questions posed mostly by the moderator, and sometimes by the other panelists. The panelists are: Rabbi David Stern, chief rabbi at Temple Emmanuel in Dallas (representative of Judaism); Imam Omar Suleiman, Director of the Islamic Learning Foundation in Texas and resident scholar at Valley Ranch Islamic center (representative of the Islamic faith); Reverend Dr. Chris Girata, Rector at Saint Michael All Angels foundation (representative of the Christian faith). The event is held at St. Michael All Angels Foundation in Dallas on 29 th Jan., 2018.
The description of the gestures is based on three criteria: the form of the gesture, its orientation, and the movement made. These three criteria are the variables based on which the type of gesture, the meaning interpretation, and its relation to language is decided.

Analysis
Due to the length of the dialogue which extends to more than twenty pages of transcribed data, and due to the limited time and space, only a number of selected excerpts are analyzed. The focus is on the interaction of modes, language and gestures in specific, in the selected interfaith dialogues. Two excerpts from the selected dialogue "Islam, Judaism, and Christianity-A Conversation" are introduced and analyzed. Moderator: what is a commonly held misconception of your tradition and what would you like us to know? Reverend: "I would never presume to answer first". Rabbi David: I would say, a commonly held misconception might be that the Jewish community is monolithic in either its political or spiritual attitudes.

Imam Omar:
The misconception is that we oppress women, and I think it plays into a large narrative that Islam is a cause of destruction and regression . Do you know which is the largest Muslim country in the world? He continues by answering his own question: its Indonesia where two women were elected prime ministers. So, associating Islam with regression, particularly, the oppression of women and holding them back is, dishonest. Reverend Girata: I think the biggest misconception would be that the Christians are judgmental and we seek to convert or change anyone we meet. I had a conversation with people from other faiths, they are afraid to show up at a church, because they assume that we will kind of get our claws into them to make sure they are changed to be the kind of person God wants them to be. Moderator: It's a wonderful question that came ahead of time from some of the audience, "do we pray to the same God?" Imam Omar: I guess I'll start first, the name "Allah" is mentioned in the Arabic Bible, it is what the Arab Christians believe in, So we have an understanding in our faith that we are calling upon the same God that Jews and Christians call upon. There was an interfaith initiative a few years ago called the "common word", started between Muslims and Catholics and spread which is based on a verse in the Quran that says "Oh people of the book, come to a common word that our God and your God is one". Reverend Girata: I would just echo to say that we all understand that our Ibrahamic root is the same. In every tradition not just the three faiths there are different branches and the way we understand the creator may be different, but that the creator is the same. Rabbi David: I would also echo what my colleagues and friends have said that it is a kind of violation of God's sovereignty to treat Him narrowly. There is one God that we have different paths to and have different expressions of, even within the same faith. As mentioned earlier, verbal and gestural modes co-instantiation could lead to either co-contextualizing relations when the meaning of the two modes is similar or closely related, or re-contextualizing relations when the meaning of the co-instantiated modes is different.
An example would do the illustration. When Imam Omar answered the first question by saying "our God and Your God is the same", the linguistic entity "our" was co-instantiated with an indexical gesture pointing at himself as a member of the Muslim community. His linguistic entity and gestural selection came into a co-contextualizing relation based on the convergence in meaning as both modes denote the same entity which is the Muslim community, as illustrated below: Figure4.1: Co-contextualizing/ Indexical Gesture.
Also, he associated his linguistic entity "your God", with the same movement but with a different orientation:

Figure4.2: Co-contextualizing/ Indexical-Relation.
This time his gesture is oriented towards the Christians and Jewish among the audience as representatives of the two other faiths. Equally, in this example, the linguistic mode comes into a co-contextualizing relation arising from the convergence in meaning. Based on Liu and O'Halloran (2009), intersemiotic polysemy is operative in cocontextualizing relations resulting in ideational complementarity in the emergent meaning. The emergent meaning created here is the meaning of relation.
As for re-contextualizing relation, the matter is different. The meanings realized by the co-instantiated modes are at odds with each other. The divergence in meaning leads to a re-contextualizing relation. An example would be when the Reverend said "I would never presume to answer first", here, he co-instantiated a gesture that realizes the representation of the emergent meaning of receptivity with his verbal message when he pointed with an open palm towards the Muslim. This kind of gesture connotes the meaning of openness and invitation to present an answer to a question. But here, it is used with a statement, therefore, his gesture comes into a re-contextualizing relation with the co-deployed verbal mode. But this divergence is reconciled and assumed to be related activating intersemiotic polysemy which results in ideational complementarity in the emergent meaning.
In relation to language-corresponding gestures, which are a visual reformulation of the meanings realized in the language mode, they mostly come into a co-contextualizing relation resulting from the convergence in meaning. For instance, when the panelists mimic a meaning realized in the mode of language through gestural movements like when the Reverend said "get our claws into them" he made a clawlike gesture representing the process in the verbal mode, or when the moderator in the second question used the verbal unit "ahead of time" with a backward movement representing the meaning of circumstance. This triggers the intersemiotic mechanism of parallelism that serves to create ideational concurrence in the emergent meaning. The added meaning here is the meaning of accentuation and emphasis.
Language-corresponding gestures could also come into recontextualizing relations based on a divergence in the meaning of the codeployed modes, but, intersemiotic polysemy becomes operative by a process of reconciliation resulting in a complementarity in the emergent meaning. For example, in the Reverend's answer to the second question, he said "the way we understand the creator may be different, but the creator is the same" with the first "creator" he joined the fingers of his right hand together and pointed repeatedly upwards as illustrated in figure 4.3 below, and with the second "creator" he used the same gesture but this time with a downward orientation and in both times it does not mimic the verbal information in anyway, the gesture is illustrated in the figure below:  Yet again, this does not mean that re-contextualization leads to a breakdown in the process of communication, the divergence in meaning is reconciled through an intersemiotic mechanism of polysemy resulting in ideational complementarity in the emergent meaning.
Language-independent gestures, can also come into either cocontextualizing or re-contextualizing relations with the accompanying language based on the convergence or divergence in meaning. Intersemiotic polysemy operates when language and gesture come into a co-contextualizing relation resulting in complementarity in the emergent meaning, and when they come into re-contextualizing relations, intersemiotic mechanism of polysemy is also in operation through reconciliation of meaning leading to complementarity in meaning.
Language-corresponding gestures can lead to the production of what is called semiotic metaphor which is produced when the verbal item is resemiotized in the gestural mode with a shift in the function of that entity. For example, the Reverend, in his answer to the second question, resemiotized the item "root" which is an "entity" in the verbal mode into a "process" in the gestural mode, this functional shift leads to the creation of semiotic metaphor. To illustrate, the following figure is presented:

Concluding Remarks
The combination of language and gestures in interfaith dialogue is synergistic since it generates meaning that is larger than the sum of individual meanings produced by each mode where an emergent meaning is brought about. The co-instantiation of the verbal mode with indexical gestures results in several emergent meanings including relation, specificity, and receptivity. While when the verbal mode is associated with a representing gesture, the resulting emergent meanings are redundancy, accentuation, attention and emphasis, even semiotic metaphor can be considered as an emergent meaning. This presents an answer to the main question underlying this effort related to whether meaning is construable through the analysis of one semiotic mode or not. The mode of gesture contributes to the meaning-making process leading to several emergent meanings that exceed the individual meaning realized by the two modes. It is concluded from this that, in interfaith