Abstract
This paper explores intersemiotic construction and digital interactivity through a multimodal social semiotic lens examining the semiotic instantiations of interactive signs on the homepage of “Fighting COVID-19 the Chinese Way” (covid-19.chinadaily.com.cn), a website created to disseminate information on news and development of COVID prevention and control practices in China to contribute to global efforts to fight the pandemic. The study focuses on how digital interactivity is afforded by the COVID China website, where the interactive signs are ideationally and compositionally constructed for meaning representation and interpersonally for text-viewer relation construal. From the multimodal social semiotic perspective, systemic functional model-based cluster and intersemiotic analyses are applied to explore the visual, spatial, and linguistic features that contribute to the design and construction of interactive semiotic signs on the COVID China website and afford digital interactivity for viewers’ action potentials. This study extends the analytical focus to the semiotic instantiations of interactive signs and their intersemiotic construction process that stimulate the enabling of interactivity, instead of the interactivity per se, and demonstrates how different semiotic instantiations of interactive signs are featured and interact to afford digital interactivity. It argues for an integrated lens in analysis to look at the interactive signs not only as signs of action with action-enabling forms but also as signs of meanings that afford user-page interactivity.
Acknowledgments
The COVID China homepage was retrieved with screenshots captured in late December 2020.
References
Adami, E. (2015). What’s in a click? A social semiotic framework for the multimodal analysis of website interactivity. Vis. Commun. 14: 133–153, https://doi.org/10.1177/1470357214565583.Search in Google Scholar
Baldry, A.P. (2004). Phase and transition, type and instance: patterns in media texts as seen through a multimodal concordancer. In: O’Halloran, K.L. (Ed.). Multimodal discourse analysis: systemic-functional perspectives. London: Continuum, pp. 83–107.Search in Google Scholar
Baldry, A. and Thibault, P.J. (2005). Multimodal transcription and text analysis: a multimedia toolkit and coursebook with associated on-line course. Equinox, London.Search in Google Scholar
Bezemer, J. and Mavers, D. (2011). Multimodal transcription as academic practice: a social semiotic perspective. Int. J. Soc. Res. Methodol. 14: 191–206, https://doi.org/10.1080/13645579.2011.563616.Search in Google Scholar
Chen, Z.T. and Cheung, M. (2020). Consumption as extended carnival on Tmall in contemporary China: a social semiotic multimodal analysis of interactive banner ads. Soc. Semiotic., https://doi.org/10.1080/10350330.2020.1720992.Search in Google Scholar
Chiew, A.K.K. (2004). Multisemiotic mediation in hypertext. In: O’Halloran, K.L. (Ed.). Multimodal discourse analysis: systemic-functional perspectives. London: Continuum, pp. 131–159.Search in Google Scholar
Gibson, J.J. (1986). The ecological approach to visual perception. Hillsdale, New Jersey: Lawrence Erlbaum Associates, Publishers. (Original work published 1979).Search in Google Scholar
Halliday, M.A.K. (1994). An introduction to functional grammar, 2nd ed. London: Arnold.Search in Google Scholar
Harvey, K. (2013). Medicalisation, pharmaceutical promotion and the internet: a critical multimodal discourse analysis of hair loss websites. Soc. Semiotic. 23: 691–714, https://doi.org/10.1080/10350330.2013.777596.Search in Google Scholar
Hyland, K. (2012). Individuality or conformity? Identity in personal and university academic homepages. Comput. Compos. 29: 309–322, https://doi.org/10.1016/j.compcom.2012.10.002.Search in Google Scholar
Jakobson, R. (1959). On linguistic aspects of translation. In: Brower, R.A. (Ed.). On translation. Cambridge, MA: Harvard University Press, pp. 232–239.Search in Google Scholar
Kim, S.Y. (2021). College disability support offices as advertisements: a multimodal discourse analysis. Discourse Stud. 23: 166–190, https://doi.org/10.1177/1461445620966921.Search in Google Scholar
Kress, G. (2003). Literacy in the new media age. London: Routledge.Search in Google Scholar
Kress, G. (2010). Multimodality: a social semiotic approach to contemporary communication. London: Routledge.Search in Google Scholar
Kress, G. and Van Leeuwen, T. (2006). Reading images: the grammar of visual design, 2nd ed. London: Routledge.Search in Google Scholar
Krisjanous, J. (2016). An exploratory multimodal discourse analysis of dark tourism websites: communicating issues around contested sites. J. Dest. Mark. Manage. 5: 341–350, https://doi.org/10.1016/j.jdmm.2016.07.005.Search in Google Scholar
Langmann, S. and Gardner, P. (2020). The intersemiotic affordances of photography and poetry. Semiotica 236–237: 85–102, https://doi.org/10.1515/sem-2018-0050.Search in Google Scholar
Lemke, J.L. (2002). Travels in hypermodality. Vis. Commun. 1: 299–325, https://doi.org/10.1177/147035720200100303.Search in Google Scholar
Lewin-Jones, J. (2019). Discourses of ‘internationalisation’: a multimodal critical discourse analysis of university marketing webpages. Res. Post-Compulsory Educ. 24: 208–230, https://doi.org/10.1080/13596748.2019.1596418.Search in Google Scholar
Lim, F.V. (2019). Investigating intersemiosis: a systemic functional multimodal discourse analysis of the relationship between language and gesture in classroom discourse. Vis. Commun. 20: 34–58, https://doi.org/10.1177/1470357218820695.Search in Google Scholar
Mafofo, L. and Banda, F. (2014). Accentuating institutional brands: a multimodal analysis of the homepages of selected South African universities. South. Afr. Ling. Appl. Lang. Stud. 32: 417–432, https://doi.org/10.2989/16073614.2014.997068.Search in Google Scholar
Martin, J.L. (2012). Language verbalizing notation: an intersemiotic analysis of musical notation in student texts. Multimodal Commun. 1: 147–161, https://doi.org/10.1515/mc-2012-0010.Search in Google Scholar
McLachlan, M. (2021). Neoliberal rules: a critical multimodal analysis of metonymy on high school webpages. Ling. Educ. 65: 1–11, https://doi.org/10.1016/j.linged.2021.100957.Search in Google Scholar
Michelson, K. and Valencia, J.A.A. (2016). Study abroad: tourism or education? A multimodal social semiotic analysis of institutional discourses of a promotional website. Discourse Commun. 10: 235–256, https://doi.org/10.1177/1750481315623893.Search in Google Scholar
Moran, C. and Lee, C. (2013). Selling genital cosmetic surgery to healthy women: a multimodal discourse analysis of Australian surgical websites. Crit. Discourse Stud. 10: 373–391, https://doi.org/10.1080/17405904.2013.813772.Search in Google Scholar
O’Halloran, K.L. (2008). Systemic functional-multimodal discourse analysis (SF-MDA): constructing ideational meaning using language and visual imagery. Vis. Commun. 7: 443–475.Search in Google Scholar
Reed, E.S. (1996). Encountering the world: toward an ecological psychology. Oxford: Oxford University Press.Search in Google Scholar
Royce, T. (2002). Multimodality in the TESOL classroom: exploring visual-verbal synergy. Tesol Q. 36: 191–205, https://doi.org/10.2307/3588330.Search in Google Scholar
Steffensen, S.V. (2011). Beyond mind: an extended ecology of languaging. In: Cowley, S.J. (Ed.). Distributed language. Amsterdam/Philadelphia: John Benjamins, pp. 185–210.Search in Google Scholar
Steffensen, S.V. (2013). Human interactivity: problem-solving, solution-probing and verbal patterns in the wild. In: Cowley, S.J. and Frederic, V.T. (Eds.), Cognition beyond the brain: interactivity, computation and human artifice. Springer, London, pp. 195–221.Search in Google Scholar
Steffensen, S.V. (2015). Distributed language and dialogism: notes on non-locality, sense-making and interactivity. Lang. Sci. 50: 105–119, https://doi.org/10.1016/j.langsci.2015.01.004.Search in Google Scholar
Stromer-Galley, J. (2004). Interactivity-as-product and interactivity-as-process. Inf. Soc. 20: 391–394, https://doi.org/10.1080/01972240490508081.Search in Google Scholar
Thibault, P.J. (2017). Language dynamics, values, and interactivity. In: Peeregaard, B. and Duncker, D. (Eds.), Creativity and continuity: perspectives on the dynamics of language conventionalization. U Press, Copenhagen, pp. 353–378.Search in Google Scholar
Turra, E. (2020). Culture, communication and persuasion on gastronomic tourism websites: a multimodal analysis. J. Vis. Literacy 39: 256–272, https://doi.org/10.1080/1051144x.2020.1828676.Search in Google Scholar
Van Leeuwen, T. (2015). Introducing social semiotics. London and New York: Routledge.Search in Google Scholar
Van Lier, L. (2004). The ecology and semiotics of language learning: a sociocultural perspective. Boston: Kluwer Academic Publishers.Search in Google Scholar
Wong, M. (2019). Multimodal communication: a social semiotic approach to text and image in print and digital media. Switzerland: Palgrave Macmillan.Search in Google Scholar
Zhang, Y. and O’Halloran, K. (2012). The gate of the gateway: a hypermodal approach to university homepages. Semiotica 190: 87–109, https://doi.org/10.1515/sem-2012-0041.Search in Google Scholar
Zhang, Z., Tan, S., Wignell, P., and O’Halloran, K. (2020). Addressing international students on Australian and Chinese university webpages: a comparative study. Discourse Context & Media 36: 1–12, https://doi.org/10.1016/j.dcm.2020.100403.Search in Google Scholar
© 2021 Walter de Gruyter GmbH, Berlin/Boston
