The Co-effect and the Relation of Image and Words on Visualizing Multimodal Metaphor : The Case of Cell ’ s Covers

This paper is about the analysis of the function in constructing metaphors by both image and words in scientific multimodal texts, involving the topmost biological journal Cell’s covers. The study of this paper departs from the aspect of representational meanings of Kress & van Leeuwen’s visual grammar to analyze the co-effect and the relation of image and words on visualizing the metaphor “CIRCADIAN IS A CLOCK”. Meanwhile, the division of the relation between image and words—complementary and non-complementary—by Zhang & Mu will be utilized also. The findings of this paper not only ascertain the parallel role of image and words in visualizing the metaphor, but also show their complementary relations with each other.


Introduction
In this high-tech dominated society, science unites art into a new pattern-"artistic technology".Visualization is a process of visualizing some abstract ideas into images or products (Lankow et al., 2012).Therefore, we can define the visualization of the scientific ideas as scientific visualization.In previous study, the construction of a visual image in the mind was called visualization (Weber et al., 1972).Acting as the inner structure of the mind, most visualizations are combinations of both pictures and words.However, according to Ware (2004) graphical representation of data or concepts are more likely to be the specialties of this modern world.With high efficiency, visualizations inform something new.In order to ease our cognitive work, visualization transforms data into a more efficient visual format.And the application of scientific knowledge to visual format is the process of science-visualization, which is also applicable to the design of some scientific journals' cover.The present study cites Cell's covers as the corpus which can also be taken as the instances of "sciencevisualization".In visualizing the scientific concepts, cover image, together with the explanatory words and cover story, simplifies the intricate or abstract ones into concrete ones.
But most of the theories about multimodal metaphors remain lingering in the simple and superficial depiction of the metaphorical phenomena, rather than putting forward some universal but theoretical modal to rule the construction and the classification of the multimodal metaphors (El Refaie, 2003, p. 78).Up to now, Forceville makes the relatively complete classification of multimodal metaphors in his ads study: supplement, fusion and juxtaposition.However, this one only concerns the spatial relation, neglecting their interior relations and structures.
However, in Systemic Functional Linguistics (SFL), Halliday (1978) points out that language is a socially-based semiotic system with a meaning-making potential.Therefore, besides language, other modes can also create meanings in a multimodal discourse.In 1990s, the multimodal discourse analysis adopting the general theory of Systemic Functional Linguistics (SFL) developed by Halliday (1978Halliday ( , 1985) ) flourished, attempting to apply linguistic views to explain non-linguistic modes.Two main representatives in this Hallidayan interpretation of meaning-making in visual semiotic modes are O'Toole and Kress & van Leeuwen, who have adopted different linguistically-focused paradigms and applied different materials to examine the way visual information is projected.
O'Toole (1994,1999) applies Halliday's three language metafunctions to examining the displayed art in painting, sculpture and architecture, trying to find how visual modes in displayed art project their meanings.Similarly, he reinterprets "ideational" as "Representational", "interpersonal" to be "Modal", and "textual" as "Compositional".In particular, he emphasizes the importance of realization and rank scale of SFL in the process of interpretation (Halliday, 1994, p. 15).However, O'Toole holds that the framework presented for the semiotic system of displayed art is not a set of rules for constraining the construction of meaning, but integrates "the semiotic space created by the work within which our perceptions and conceptions are negotiated" (O'Toole, 1999, p. 165).In addition, he attempts to apply register to his visual text analysis in terms of contexts of situation.
Kress & van Leeuwen's visual grammar-the grammar of visual design-presents a socially semiotic visual representation (1996, p. 1, 5).The visual images, taken as "text", is regulated by "grammar" which is not a set of correct rules but a way of linking form with meaning, or meaning representation, in the visual discourse.They (1996,2006) claim that visual semiotics represent Representational meanings (ideational), Interactional meanings (interpersonal), and Compositional meanings (textual).Various images organize and represent their meanings representationally (ideational) in two major processes-Conceptual and Narrative processes-which are identified as carrying representational meanings in images.Within Narrative processes in images Kress & van Leeuwen contend there are two major kinds: Actional and Reactional processes; while within Conceptual processes in images they recognize three kinds: Classificational, Analytical, and Symbolic Processes.A number of ways of describing the interactive meanings (interpersonal) are established and maintained between the image and the viewer, with aspects like, Contact, Social Distance, Attitude, and Modality.The Compositional meanings (textual) in the visual grammar, proposed by Kress & van Leeuwen, concern the codes that functionally operate in the layout of an image to create meaning and textual coherence.The codes include Informative Value, Salience, and Framing.

The Relation between Words and Image
Taking advertisements as example, Forceville (1996, pp. 109-145), conducting a relatively overall classification, divides the mapping processes between the source domain and the target domain in a multimodal metaphor into three kinds: 1) replacement, the source domain replaces the target domain; 2) juxtaposition, the source domain coordinates with the target domain; and 3) fusion, the source domain integrates with the target domain.Besides, Phillips & McQuarrie (2004), Teng & Sun (2002) and Schilperoord & Maes (2008) also make the same kind of division as Forceville.However, evidence show that their division cannot involve all the relationships between the words and the image in multimodal metaphors.Feng (2011) studies the words-image relationship from the aspect of metaphorical mapping.Based on Forceville & Urios-Aparisi's (2009) definition of multimodal metaphor as the occurrence of the mapping process within two or more modes, he proposes three types of mapping: cross-modal mapping, mono-modal mapping and multimodal mapping.Cross-modal mapping refers to the mapping process between different modes, both the words and the image, including verbal source domain and pictorial target domain, and pictorial source domain and verbal target domain.The former type refers to the process in which we identify the value of the image through the words.According to Halliday (1994, p. 124), image is a token, and the words attribute values to the token, the relation between the image and words is relational process identification.And the latter refers to the one that the image acts as the target domain, and the words act as the source domain.Mono-modal mapping includes verbal mapping and pictorial mapping.In verbal mapping, metaphors are constructed within words, affiliated by images acting as source domain or target domain.Similarly, in pictorial metaphors, images form metaphors independently, with words illustrating the contents of the source domain and the target domain.
In SFG, the relation between words and images can be correlated (Martinec & Salway, 2005, pp. 337-371); words supply the images, a one-way relation (Barthes, 1977, pp. 33-41), or, in turn, images make complement for words (Marsh & White, 2003, pp. 647-672).Systemic Functional grammar involves the relationships between clauses as conjunction and subordination, besides, logical-systemic relations are also included.Since Kress & van Leeuwen bring in Halliday's three metafunctions of language to extend visual communication.Therefore, Halliday's logical-semantic relations between clauses within sentences can also be applied to the multimodal discourse analysis.According to Halliday (1994, p. 219), there are two logical-semantic relations between clauses within a clause complex: 1) expansion, and 2) projection.Expansion indicates one clause expands the meaning of another by means of elaborating, extending or enhancing.Projection refers to the relation where one clause projects another, that is to say, the other clause is in a secondary place in the language.And it can be classified into two kinds: a locution and an idea.Barthes (1977) sets foot in the relationship between the words and the image from logical-semantic relation which is correspondent with Halliday's (1985Halliday's ( /1994)).He contends that the extensive meanings of images are endless, polysemous and floating.Therefore, the indication of the accurate meanings relies on the words which can narrow down the dispersed meaning potentials in the multimodal discourse (Machin & van Leeuwen, 2007, pp. 157-158).He makes a distinction between two word-image relationships: extension and elaboration.Extension refers to one mode relays another, creating new meanings.Elaboration refers to two modes elaborate the same meaning.Barthes limits the relation as a kind of words-to-image one-way relation, words acting as the subordianate role to images.However, images and words in multimodal metaphors are correlated with each other.Therefore, Kress & van Leeuwen (1996, p. 17), from the aspect of representative relationship between the image and words, points out that image contains its own structure and regulation, and the relationship between words and images are related rather than reliable.Zhang (2009aZhang ( , 2009b) ) proposes that words and images are equal in status, with relationships like, complementary, dependence, reinforcement, embodiment, elaboration, extension and enhancement, summarizing the relation between images and words as: 1) complementary, and 2) non-complementary.He starts from the stylistics to study the stylistic features of multimodal metaphors, concerning how different modes cooperate with each other to express the styles of the same kind of meaning involved in those modes.In multimodal discourse constructed with both words and images, the two modes exists the same kind of features: 1) consistency in context which includes consistent culture, situation and communicative goals; 2) complementary in meanings; and 3) independence in modes.Amongst the consistency in context, Zhang points out that "consistency in culture" refers to the co-occurrence of both the images and the words in the same cultural context.That is to say, the western content should be attached to western pictures rather than eastern style pictures; otherwise the mode conveys something devious to the original intention.This paper will utilize Zhang & Mu's (2012) analysis and division of the relation between image and wordscomplementary and non-complementary, as well as the representational structures of Kress & van Leeuwen's visual grammar, and take Cell's cover, a topmost scientific journal, as the material to study the relation between the image and the words.This kind of multimodal metaphor belongs to a cross-modal mapping process with a complementary relationship with both the image and the words.But the author is not in favor of the "consistency in culture" by Zhang & Mu, for scientific subject concerns little about nation and culture, bearing the common essence of the nature.However, the sameness in science and technology can be depicted by different images conveying different cultures, which refers to ways the authors used to strengthen their nation's scientific identity in the world.The following case will analyze two covers of Cell with the same topic but depicted by different images.And this paper will compare the relation between the image and words in the two covers with the help of visual grammar.

The Corpus
The corpus of this paper originates from Cell, one of the journals in Cell Press.Before going deeper into the corpus, a few facts about Cell are in good order.Cell Press publishes 9 refereed scientific journals including Cell, Neuron, Immunity, Molecular Cell, Cancer Cell, Current Biology, Structure, and Chemistry & Biology.
(Note 1) Among them, Cell Press takes Cell as the publishing criterion for the quality and impact of other subjournals.Cell, one of the topmost international journals in life science research, founded by Benjamin Lewin in January 1974, publishes biweekly the cutting-edge developments in biology, and maintains the continuity of redefining the important areas of science.The findings of great significance include but not limited to areas of experimental biology, molecular biology, developmental biology, genetics, immunology, microbiology, neurobiology, plant biology, structural biology and virology.(Note 2) The submitted papers are admitted, given that they provide advanced biological concepts, or raise provocative biological questions and hypothesis.In addition, all authors of accepted manuscripts are given the chance to submit ideas for the cover of the journal, which should be based on or resemble the figures in the article, artistically and informatively.Only those interesting and creative ones can be selected, based on both the aesthetic quality of the image and the scientific quality of the study.The cover images as well as the relative cover stories of Cell on February 28, 2013 and February 17, 2012 constitute the corpus of the current paper.
Cell gives distinct requirements for their selection of the cover in their official website.The would-be cover should be a single-page PDF document containing: author list, article tile, and cover legend (a brief paragraph that describes the cover image and relates it to the major findings of the article).In the "Subject" line, there should include (a) the surnames of the first and corresponding author(s) in the upper-right corner and (b) the manuscript number (e.g., Smith Jones-13-00663).And the page should include any additional information in the "Message" field.(Note 3) (c) readers should be able to grasp what the image conveys within 2 to 3 seconds.(d) the upper-left quadrant of images displayed will be partially obscured by the title of the article.
This study includes 2 covers, but the research in life science is still ongoing.Besides Volume 156 No. 1 and 2 January 16, 2014, which is a celebration of Cell's 40th anniversary, almost all the covers of Cell are designed in the same fashion.On the upper-left side, there is a bold font "Cell" in different colors, depending on the main color of the page.On the top right hand side of each cover is placed, from above down, with the volume number, date and website.At the bottom of each cover, with indeterminate location, features a caption of the related article(s).An additional text on the following page is a brief explanation of the cover story, which is called linguistic material (LM) in this study, superimposed on the design created to capture the cover story, which is called the pictorial material (PM) here.
The current paper is grounded in qualitative research, and offers the author's interpretation and classification of the words-image relation in constructing metaphors in cover pictures in biology.

Case Study
This section will discuss and evaluate Kress & van Leeuwen's description of the representational meanings of the image involved in the multimodal discourse, as well as how metaphors are constructed between the image and the words in the cover.The materials will be divided into pictorial material (pm) and linguistic material (lm), utilizing two covers of Cell: Volume 152(2013), Issue 5 and Volume 148(2012), Issue 4.

Analysis of Image
In the representational structures discussed by Kress & van Leeuwen (1996, p. 56), two major processes occur: Conceptual and Narrative processes.Conceptual processes serve to represent participants in terms of class, structure or meaning.Narrative processes deal with unfolding actions and events, changes as well as spatial arrangements.

Narrative Processes
There are two kinds of Narrative processes in visuals: Actional and Reactional processes.The former refers to the physical action relating to the represented participants, transitional or non-transitional.In the process, this action is realized by a vector, with a line projected from the actor reaches to the goal.Non-transitional processes have only one participant but no goal.However, transitional ones have two or more participants-an actor, a goal and a beneficiary.Reactional processes are realized by the glance of the reactor.And the vector directs the reactor to look at the phenomenon.Secondary participants in images can be identified as Circumstances, referring to the participants that can be neglected without destroying the meaning of the whole sentence.Based on Halliday (1994, p. 149), Kress & van Leeuwen argue that there are three kinds of Circumstances: Locative Circumstances, Circumstances of Accompaniment, and Circumstances of Means.They are represented by the location, the co-occurrence in the same image, or being used by a participant in the action.
Using Kress & van Leeuwen's definition, Figure 1 reveals the relative processes like: 1) A narrative transactional process, that is of the competition between two Japanese girls, with different clothes and hairstyles, who compete with each other in terms of their "pushes" towards the minute hand in a clock.The transactional relation in the image is realized by the vectors which can be seen from the two figures' action.Metaphorically, the two proteins, FBXL3 and FBXL21, symbolize the two Japanese girls.And the distinct hairstyles and clothes represent the differences of the two proteins, resulting in the "push" competition in circadian rhythm.
2) Another minor narrative transactional process is that of the eyesight of the girls.The two girls are not staring at each other, but in a direction towards the readers.This displays that the gazes of the participants are in a way saying "look at this" to the reader.
Using Kress & van Leeuwen's definition, Figure 2 reveals the relative processes like: 1) A major narrative transactional process is that of the "Jack" on the deck of the cards.This transactional relation is realized by vectors of the eye of "Jack".When the sun turns up, "Jack" opens his eyes.While when the moon arises, "Jack" takes a rest.Here "Jack" metaphorically and phonemically symbolizes the JAK/STAT signaling which plays an important role in the regulation of rest: activity behaviors.
2) A minor narrative transactional process enacted by the hands of the clock decides the circadian behavior of "Jack".When it is the day time, "Jack" opens his eyes; when it is night, "Jack" closes eyes.The movements of the clock as well as its gears refer to the operation of human inner biological system.And the sun on the upside of the clock means the day time, whereas the moon downside represents the night metaphorically.
3) Participants acting as circumstances of means: the "weapon" in the hand of "Jack" is a "sword" used to control the circadian behavior of him, or the rest activity of him.The "sword" metaphorically represents the microRNA, miR-279 which is used to modulate JAK/STAT signaling.

Conceptual Processes
There are three kinds of Conceptual processes: Classificational, Analytical, and Symbolic Processes.Classificational processes relate participants together in terms of taxonomy of types of things.Each of the participants has a superordinate category, which can be a Covert Taxonomy or an Overt Taxonomy.The former one refers to the spatial arrangement of the participants; the latter includes superordinate participant in the frame.
Analytical processes analyze the relation between the participants in terms of part/whole relations, where the Carrier represents the "whole", the Possessive Attributes represent the "parts".Symbolic processes refer to what does the participant means, the symbolism or messages conveyed by the participant relations.The participant establishing its own meaning in the relation is the Carrier, and the participant that can be on behalf of its own meaning or identity is the Symbolic Attribute.The Symbolic Attribute process is very common on the interpretation of the Cell cover, which portrays various meanings.
In Figure 1, the clock on the background represents the circadian clock, or biological clock.This forms a metaphor "CIRCADIAN IS A CLOCK".The two women wearing kimonos with distinct hairstyles, which are the Symbolic Attributes that represent the CRY protein modifications with distinguishable polyubiquitination chain structures, protein FBXL3 and protein FBXL21.The minute hand pasts XII symbolizes how two related F box proteins regulate levels of the circadian regulators CRY 1 and 2 through a balance of protein degradation and stabilization, with protein FBXL3 promoting its degradation and speeding up the oscillation of the circadian clock (represented by the action of the woman on the left) and protein FBXL21, thereby, slowing down the oscillation speed (action of woman on the right).
In Figure 2, the clock on the foregrounding also represents the circadian clock, which metaphorically represents "CIRCADIAN IS A CLOCK".While poker "Jack" holding a "sword", the Symbolic Attribute, represents a microRNA, miR-279, that modulates the JAK/STAT signaling.JAK/STAT signaling is a component of the pathway that functions downstream of the clock in the regulation of rest: activity behaviors, represented by the sun and the moon.The sun takes control of the activity on the day of "Jack" and the moon controls that of the night or rest.In the image, there is no action except the "sword" and the symbol of sun and moon, getting the symbolic meanings.

Analysis of the Words
According to Zhang (2009aZhang ( , 2009b) ) the "consistency in culture" refers to the co-occurrence of both the images and the words in the same cultural context.That is to say, the western content should be attached to western pictures rather than eastern style pictures; otherwise the mode conveys something devious to the original intention.However, this cannot be true in scientific images.
From the linguistic materials of both images we can find that the two covers are concerned about the same topic, the circadian clock, a neutral scientific field.The sameness of the topic in Cell covers belongs to neither western nor eastern culture.But images used to depict the content decorated by different participants, conveying their national features of the authors or designers.
Figure 1 overtly puts two women with strong Japanese features as the participants, which shows clearly the identity and nationality of the researchers.The image and the words are complementary to each other, for their parallel status in depicting the contents involved in the multimodal discourse."Competition Paces Circadian Oscillations" marks the action taken by the two Japanese girls, who are scrambling for the movement of the minute hand on the clock.Its cover story, though, is not printed together on the same page with the cover, affiliating itself to the cover, with concrete depiction for the content and the relation between the participants conveyed in the cover.
Figure 2 employs poker "Jack" as the participant which belongs to western culture.However, the choice of homophones "Jack" and "JAK" is a clever trick for the construction of the picture.The relation between the upside "Jack" and downside "Jack" is a relation of "wiring", for the gears beneath the clock represent the day: night-circulation of "Jack"."Wiring Circadian Behavior" vividly depicts the behavior of "Jack".And his weapon has been changed into a stem-loop shaped precursor microRNA, which modulates the circadian effects of JAK/STAT.The relation in the participants and the circumstances in the image are all depicted in the cover story, which also indicates that the image and the words are parallel in conveying meanings.
Therefore, the author makes a conclusion about the relation between the words and the image of Cell cover.Both the words and the image represent the same metaphors of the subject conveyed underneath the multimodal texts.

Conclusion
The current article contributes to the study of the relation between the image and the words, or the pictorial materials and the linguistic materials, by discussing the representational meaning of the multimodal texts of Kress & van Leeuwen.The corpora, two Cell covers as well as the words used to describe them, dominantly describe the same topic-the circadian rhythm, or the biological clock, but realized by different participants and their relative relations.The images bestow visual concreteness and enhance the national identity of the authors.And the words parallel the images depict the same metaphor "CIRCADIAN IS A CLOCK".And the other participants construct the supplements for the circulation of the clock and their relations convey differences between the two covers.Therefore, in multimodal texts, the words and the images complement one another to transmit the same kind of meanings metaphorically.And both the image and the words used to represent the metaphorical meanings result from the visualization of the scientific ideas.The image visualizes the ideas and word-depictions into concrete and simple ones that can get quick understanding from the readers; and the words depict the processes and scientific results as well as the content of the image in detail.
This paper concerns about just one kind of image-words relation about scientific contents, and some further studies about other kinds as well as the systemic classification of the relations are needed.

Figure 1 .
Figure 1.Cell, Volume 152(2013), Issue 5 Figure 2. Cell, Volume 148(2012), Issue 4 On the cover: In this issue,Hirano et al. (pp.1106Hirano et al. (pp.-1118) )andYoo et al. (pp.1091Yoo et al. (pp.-1105) )demonstrate how two related F box proteins regulate levels of the circadian regulators CRY 1 and 2 through a balance of protein degradation and stabilization.The cover depicts two women with distinct hairstyles representing CRY protein modifications with distinguishable polyubiquitination chain structures.Polyubiquitination of CRYs by the F box protein FBXL3 promotes its degradation and speeds up the oscillation of the circadian clock (represented by the action of the woman on the left).A second related F box protein, FBXL21, forms a different type of ubiquitin chain that stabilizes rather than degrades the CRYs, thereby slowing down the oscillation speed (action of woman on the right).Hence, a balance between the antagonistic functions of FBXL21 and FBXL3 on CRY protein levels ultimately regulates stable and robust oscillation of the circadian clock.Artwork by Yuki Takahashi, inspired by Kentaro Hirose.(Note4)4.2.2CoverStory of Figure2On the cover: Circadian rest: activity rhythms in animals are regulated by endogenous clocks that are typically synchronized to the day: night cycle.In this issue, Luo and Sehgal (pp.765-779) identify JAK/STAT signaling as a component if the pathway that functions downstream of the clock in the regulation of rest: activity behaviors in Drosophila.They show also that circadian effects of JAK/STAT are modulated by a microRNA, miR-279.The cover image depicts a day: night adapted clock that turns on, or downregulates, JAK/STAT signaling, represented by a "Jack" from a deck of cards.The "Jack" is shown to hold a "sword"-a stem-loop shaped precursor microRNA.Artwork by Lili Guo.(Note 5)