Rethinking Literacy Education in New Times: Multimodality, Multiliteracies, & New Literacies

The article presents a theoretical overview of new fields of research, pedagogy, and practice in literacy education. In a digital, media-driven, globalized world, educators are faced with the challenge of mediating traditional notions of what it means to be literate (e.g., read and writing print-based texts) with new and ever-emerging skills and interests in technology and digital media. Focusing on a pilot study in Oakville, ON and a longitudinal research study in Sydney, Australia, we compel readers to think about literacy in a new light. Without a push to redefine literacy, educators run the risk of teaching and learning language and literacy skills in anachronistic paradigms and frameworks. While research has not been able to fully establish the impact of multimodal communication, it is essential that educators learn to use these different modes of communication to teach literacy.

A group of boys sit around an iPad in a grade three classroom in an elementary school in a western suburb outside of Toronto. They are crowded around the iPad playing "A Monster Ate my Homework". We ask them why they like the game and one boy claims that it helps his spatial skills ("it is also fun"). As we move around the room, we encounter two girls playing "Whirly Word" with an iPad close to them. Maureen and I look over their shoulder as they play and they look up at us shyly. We then realize that they would prefer us not to watch as they choose words from the same sound families. Then we move to yet another group with two boys and a girl and they are struggling to find combinations of words in the allotted time and we work with them to make as many words as possible out of five letters. This classroom is part of a pilot study using iPads in the classroom. The teacher self-describes as "non-techie" yet she has embraced the iPad project and she has found "great success with it" over the six week pilot project.
This brief window, an hour to be exact, observing eight year olds using new technologies for word study gave us a window into how quite traditional language skills such as word study and spelling translate into '21 st century learning' and multimodal forms of learning and thinking. By tapping and sliding and problem-solving, learners struggling with reading and spelling are successfully working through levels in a spelling game. Multimodality as in comprehension and competence with language through a variety of modes such as image, sound, touch, multi-dimensions, is the principle upon which digital environments work. This principle of multimodality needs to be understood for educators to apply and assess new modes of learning as a part of everyday classroom practice. (May 25, 2011) The vignette that begins our article describes a moment in time in a classroom in Oakville, Ontario. The moment encapsulates the way that children are able to respond quickly and effectively to the digital technologies that permeate their world. While education policy makers and curriculum designers struggle to find ways of incorporating new modes of communication, many researchers and teachers worldwide are finding ways of using new technologies for literacy and learning.
It is undeniable that students right now require a repertoire of both print and digital literacy practices for their future workplace and life. Terms such as 'new literacies' (Lankshear & Knobel, 2003), 'multiliteracies' (Cope & Kalantzis, 2000) and 'multimodality' (Kress & van Leeuwen, 2001) have been used for some time to conceptualize the way new communication practices are impacting on literacy and learning. The use of terminology such as 'new' or 'multi' in descriptions of changes that have occurred with digital communication are attempts to describe the multiple devices and media texts that are ubiquitous in our world. Multiple modes (e.g., image, sound, gesture, movement and text) are processed during communication and multiple aspects of literacy, or multiliteracies, are needed in our networked, global society. Even more, the 'multiplicative' (Lemke, 1998) effect of the processing of modes for reading and writing, often simultaneously, need to be considered.
In this article, we provide an explanation of the new terms that have developed to theorize changes in literacy and communication in society. We demonstrate the potential of new technologies for classroom literacy learning by discussing the differences between literacy with digital texts compared with print-based texts, and provide some examples of ways in which teachers are using multiple modes in digital texts to enhance literacy learning.

Situating 'new' literacies
The word 'literacies' in new literacies signaled a shift in thinking about the ways that people make meaning with language. Assigning plurality to literacy to privilege 'literacies' opened up what had traditionally been seen as a standardized model of literacy education, to one that acknowledges difference based on situations, subjectivities, and multiple text genres. Making literacy plural signals that there is more than simply one model of literacy, there are many different literacies that shift with contexts, texts, and the identities of people using literacy. Thinking about literacy as a universalized, autonomous entity undermines its diversity and multiple uses and understandings. Yet, what truly differentiated the work of researchers who incorporated such fields as anthropology, sociology, and semiotics in the late 20 th Century was the inclusion of the adjective 'new.' New signaled new approaches, new epistemologies, new methods, new theories, new contexts, and new identities for meaning-makers. New studies in the 1980s and 1990s were new because literacy had not been analyzed in the same way and this radical social and semiotic turn offered a new language of description for literacy, viewing literacy as nested within social context (Street, 1994) and redressing an over-emphasis on language and the written word (Halliday, 1984).
What such work identified are a series of renewed beliefs about literacy education: • More work in other contexts such as homes and communities (Gonzales, Moll, & Amanti, 2005) • Less emphasis on cognitive development (Gee, 1996) as in what happens in our brains or minds and more emphasis on cultural practices • More research examining the interface between identity and literacy development (Gee, 1996;1999) • Less of a divide between oral and written cultures (Ong, 1982) • An acknowledgement of the screen as our dominant text structure (Cope & Kalantzis, 2000;Kress, 2003) • An expansion of definitions from print logic, reading and writing, to screen logic, designing, redesigning, remixing (Cope & Kalantzis, 2000) These epiphanies in research, scholarship, and practice show not only what new studies can demonstrate (though clearly that is there as well), but also, the need for new 21 st Century definitions.

Multiliteracies, Multimodalities and New Literacies
A fundamental part of 'new' literacies in literacy education considers not only that literacies are multiple, but also that they demand different modes. Modes are regularized sets of resources for meaning making. A visual, a sound, a word, a movement, animation, spatial dimensions are resources brought together or in isolation to achieve an effect in texts. Such effects are read and composed in different ways compared with linguistic text features. Semiotic resources are things, artifacts, practices used during meaning making that complete the task in a competent, apt way (Kress, 1997). Multimodality is the field that takes account of how individuals make meaning with different kinds of modes. Multiliteracies is a pedagogy developed by the New London Group and their manifesto developed in the midnineties (New London Group, 1996). The New London Group (1996) argue that the notions of design, available designs and redesign are fundamental to how we make meaning with modern texts. Designing on-screen has not only transformed how we make meaning, but also, transformed ways of reconstructing and renegotiating our identities. Multimodality comes first in that it informs how we make meaning and multiliteracies, as a possible pedagogy, gives us tools for doing so. Multiliteracies scholars claim that the screen governs our understanding of the world and curricula needs to reflect this dramatic shift in our ideological and interpretative frame. Situating teaching based on student needs and competencies, teaching students overtly based on the skills that they have when they enter our classrooms, and most importantly and what students do not necessarily possess, are ways of critically framing their learning to think about multiple modes, issues of power, ruling passions, communities of practices, home and community literacy, the role of their race, culture, religion, and social class in their literacy learning. Multiliteracies as a pedagogy simultaneously accounts for linguistic diversity and the use of multimodalities in communication.

Digital Literacies
Digital literacies is yet another field of research and theory that branches off from the new. In 1995, Lanham maintained that "digital literacy enables us to match the medium we use to the kind of information that we are presenting and to the audience we are presenting it to" (Lanham, 1995: 3). Lankshear and Knobel (2007) complement this perspective with a sociocultural perspective based on the work of Gee, and in so doing, they help us to broaden our definition of a reified notion of digital literacy to digital literacies, taking account of becoming digitally literate as the mastering of multiple Discourses (Gee 1996;1999). Talking about "socially mediated ways of generating meaningful content through multiple modes of representation", Alvermann (2008) adds to the conversation, pointing to explicit skills that arise from digital texts such as wikis, blogs, and webpages that are mediums for social interaction. Alvermann highlights that digital readers and writers need to make many decisions online and as such, they need to have a critical eye on different genres of texts and a meta-awareness of these texts as promoting or silencing particular views (2008: 16). An example of such a study is Julia Davies' research looking at different affinity groups, from Wiccan girls to transnational youth (Davies, 2006) on webpages to show how individuals find solace in online communities and how online communities foster identities and communities.

New Literacies in New Times
There are four characteristics to new literacies research: 1) new technologies offer a way to envision new literacy practices; 2) new literacies are essential to economic, civic and personal participation in a world community; 3) new literacies change, remix, converge as defining technologies change; 4) new literacies are multimodal and multifaceted. Other researchers have spotlighted the role that new literacies play in online conversations, videogaming, and writing fan fiction. Many recent research studies have been designed to understand the ways in which teachers are using blogs and wikis and other interactive, online writing technologies such as threaded discussion groups and ePortfolios in the classroom to show that the time has come when we not only incorporate new literacies into our classrooms, but also that we understand better what happens in digital spaces. Although there is some concern that without an active teacher presence in the dialogue, students' learning is not pushed forward very effectively, most of the studies found that the increased collaborative learning opportunities helped students to refine their thinking and engage in deeper analysis. Research in the area of adolescents' out-of-school digital literacy practices (Davies, 2006) is examining a wide array of texts such as personal blogs, social networking pages, wikis, fan fictions, etc. which demonstrate an intertextual and hybridized quality of students' personal digital writing. Participants in the research blended print-based knowledge emphasized in school with innovative new forms of multimodal composition to create compelling new texts that reflect the authors' socially situated identities and discourses.
One of the most illustrative studies in this area is an often-cited article by Lewis and Fabos (2005) that explored the ways in which students manipulate and play with vernacular conventions, Standard English grammar, and electronic typography in complicated new ways. They interviewed teenagers about their private instant messaging practices and observed the teens while they were engaged in this practice. "The young people we interviewed were conscious of choosing different tones and language styles depending on whom they were IMing" (p. 484). Students involved in the research were creative in their word play and demonstrated sophisticated skills during their texting engagement. Lewis and Fabos' research offers an example of the ways in which adolescent writers mediate their identities and engage in creative compositional practices, as they note: "Andy and his friends experimented with color, font size, and icons such as smiley faces to express their creativity, and Abby tried a variety of combinations of fonts and colors for the same purpose" (p. 482). Drawing from this vast knowledge base we share examples of how new literacies can be remixed with the best instructional practice.

The processes of reading and writing 'on screen'
There are three main aspects that need to be considered when contemplating the differences between traditional practices of reading and writing in classrooms and those that are possible with digital communications. These are: • the actual processes of reading and writing 'on screen'; • the integrative and interactive nature of reading and writing with new texts; and, • changes in patterns of communication as a result of social networking.
The whole nature of digital communications is so integrative that it is difficult to separate each of the above aspects, but these are discussed separately to demonstrate key features that we need to consider for educational implications. Reading and writing are both about making meaning. When we read we have a purpose, such as enjoying a literary text or gaining new knowledge from an information text. We gain meaning as we decode and interact with a text and link our background experiences to new experiences or knowledge. When we write, we have a purpose as we write out our thoughts, communicate information, create a story or present our ideas to a reader or audience.
Meaning making occurs whether we use traditional, paper-based texts or digital, multimodal texts and the level of meaning will vary according to our purpose and the text genre. There are several differences that occur, however, with digital communication and the differences lie in the processing of modes on and from a screen: whether a computer screen, touch pad; game console or a mobile device such as a phone or e-book. As Kress has shown (2003) written language on a page is primarily a linear, sequential process. In contrast, reading and writing with screen-based, digital texts entails the reading and writing of text with images that are usually not presented in a left-to-right, linear format. Other modes that may occur along with written text are image, sound, movement and gesture. Thus the term 'multimodal' (Kress & van Leeuwen, 2001) has been used in recent years to describe the non-linear processing of texts that occur primarily on a screen.
Whatever the medium, there are more modes that need to be responded to when reading so that the distinction between reading and viewing are hard to determine. Reading on screen involves various aspects of online processing that includes responding to animated icons, hypertext, sound effects; and navigating pathways between and within screens. For example, if students are asked in class to research a specific topic such as the scientific phenomenon of "light" they need to learn how to use a search engine with the best use of key words; how to narrow the search to find specific information about aspects of light; how to choose the best url and not the first one on the list; how to judge the authenticity of specific sites and avoid aspects of advertising and pop-ups that distract. When on a specific site students will need to navigate menus, icons and hyperlinks to find the most relevant information. This processing of information will involve viewing of images, maps and graphics and may often include videos, including a link to YouTube, with sound and movement as the student is looking, reading, listening, choosing and navigating with either a mouse or touch feature. All these aspects need to be taught as part of teaching reading with 21 st century texts. For while students may quickly learn the technical skills of touching, scrolling or clicking they need to be shown how to choose the most appropriate information and discriminate between non-relevant information as they are processing information through senses of sight, sound and touch. They also need to be taught about how different modes of image, sound and movement may or may not be influencing how meaning is constructed.
Writing on screen has existed for a long time with word-processing facilities. 'Writing' now very often entails assembling a product that may contain written text as well as quite sophisticated layout, graphics, photographs and images. Similarly a 'writer' can become a 'producer' (Sheridan & Rowsell, 2011) now by designing and producing a text that combines images and graphics with written text as well as sound and movement on screen. For students to produce multimodal texts they need to consider and understand features of design such as layout, composition, use of text and image or graphics -including aspects such as colour, size, medium, angles -and the way these are appropriate for a specific audience. It is significant that other researchers have been investigating design as integral to literacy pedagogy (e.g. Kalantzis & Cope, 2005;Healy, 2008). A brief example from recent research (Walsh, 2011) illustrates some changes that have occurred.
In a Grade 4 classroom students studying the topic of light, referred to previously, had to report what they had learnt from their research. Instead of writing a report and even talking to the class about it, as may have occurred in previous years, they designed a "claymation" to present an aspect about light. This involved the use of photography and stop-motion animation that became a video with the clay figures, sound and voice-over dialogue used to demonstrate how light and shadows vary. A script was written at first then a storyboard developed. Not only did students have to understand the content, they had to plan to convey the information succinctly through the design of the clay figures, and the sequencing of events and movement for the audience to understand. Figure 1 shows students engaged in design and planning their final product.

Picture 1. Students developing a claymation on "Light" (from Walsh, 2011).
The whole process involved reading, writing, designing, photographing, filming, editing and producing -quite a different approach from writing a scientific report.

The integrative and interactive nature of reading and writing with new texts
As the above example shows and in our scenario at the start of this paper, digital communication technologies along with the facilities of touch pad devices ensure the interrelatedness of reading and writing. With Web 2.0 these often occur along with the viewing and posting of images, the blending of sound and the constant interchange and connecting of messages. Over a decade ago, emails revolutionized communication, taking over from letter writing, particularly for adults in the workplace. Now communication through social networking sites such as Facebook and various versions of blogs, wikis or twitters have taken over from emails for many younger people and are being used more by commercial firms. In such communication we are reading, writing and responding asynchronously, although features of new adaptations of technologies such as with web cams, skype, face time and virtual gaming allow for synchronous communication.
The processes of communicating in these new ways incorporate a merging and synchronising of text, images, sound and movement. Whether using a blog, wiki or 'Facebook' type of communication, the 'text' is produced with appropriate layout for screen and can combine text, images, graphics, photos or video with sound and music. Design is important and the design will be carefully developed to reflect the author/producer and to engage the audience who can respond with text and images.
We do not know how such processing of messages and texts is affecting the way children learn, or if the processes involved in activities such as texting, blogging, or communicating online are developing different cognitive abilities than those required for reading and writing traditional print-based texts. Gee's research (2003) on video gaming suggests that the procedures involved offer cognitive advantages with intricate literacy and learning opportunities. The touch features of recent products rely much more on gestural, spatial and kinaesthetic movements that need further investigation in the way this processing is affecting cognitive processes.
These unknowns are challenges for education. A UK study (Bearne et al 2007) on reading screens showed that while students were able to apply aspects of comprehension to obtain screen-based information it was "orchestrating the different modes to make meaning" (p.20) that was seen as a different process that could not be assessed in the same way as the assessment of reading print-based texts. A further study by Bearne & Wolstencroft (2008) demonstrated ways of teachers programming and assessing writing through students' multimodal texts. Bearne and Wolstencroft emphasized the interrelationship between reading and writing in producing texts and explained how students need to understand the meaningmaking potential of different modes, particularly the relationship between words and images, in reading, writing and producing multimodal texts. For educational purposes, as shown in a recent research study by Walsh and Simpson (2010), we need to distinguish between the technical skills of using digital technologies and the cognitive processes of interpreting and communicating meaning. To offset an over-emphasis on technical skills, educators and researchers need to focus on both using technologies and meta-understanding of technology.

The impact of social networking
While considering the differences in both reading and writing on screen compared with printbased texts, it is not realistic to see reading and writing as occurring separately. Nor is it sensible to separate the technical, functional processes of reading or writing on screen from the social practices that accompany these processes. These social practices of literacy have changed and expanded exponentially with the development of Web 2.0 technology and have many implications for classroom practice. If students are using these outside of school, it follows that these modes of communication could be used inside school to engage students in learning. Many teachers have begun to use blogs, nings, wikis, twitter, features of mobile phones or Facebook applications within classroom programs. These have made learning more participatory. In our Sydney research teachers found that when they applied aspects of social networking students became more collaborative. There was more problem-solving occurring as students investigated a topic and then negotiated the way they would create and construct a product to demonstrate their learning.

Conclusion
In this short article we have provided glimpses into different approaches to literacy practices in new times. These have been enabled by the accessibility of new technological tools, many of which were not available even a decade ago. We are constantly reminded in education that the mastery of the tool itself is not the outcome but how we use it. However the nature of digital communications technology has so permeated the way we communicate, informally and formally, that it has become more than a tool in many ways. While research has not been able to fully establish the impact of multimodal communication, it is essential that educators learn to use these different modes of communication for classroom learning.