Skip to main content
SpringerLink
Log in

Semiotic Structuring in Movie Narrative Generation

  • Conference paper
  • First Online:
Entertainment Computing – ICEC 2023 (ICEC 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14455))

Included in the following conference series:

  • 417 Accesses

Abstract

In this paper we apply, in a novel way, our ongoing research work on the interactive composition of narratives based on semiotic relations. To the two basic components of interactive systems, namely, a software tool and a user interface, we add a third component – AI agents, understood as an upgraded rendition of software agents. Our semiotic relations approach considers four ways of composing new narratives from existing narratives. Along the horizontal syntagmatic axis one can form the new narrative by combining two or more previous narratives. Along the vertical paradigmatic axis, the new narrative may emerge as a similar version, which imitates the previous one in a different context. Along the depth meronymic axis, the hierarchic narrative levels, such as event and scene, are explored, allowing to zoom in and out in the composition process. Lastly, the antithetic consideration, rather than adding a dimension, aims at some form of reversal, through the adoption of opposite values. A fully operational prototype is described, with ChatGPT operating as the main AI agent component. To run the experiments, we concentrated on movie narratives.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://www.icad.puc-rio.br/~logtell/

  2. 2.

    https://openai.com/blog/chatgpt

  3. 3.

    Eco [11] and Metz [28] treat the syntagmatic and paradigmatic elements within a more general approach to the semiology of cinema, which lies beyond the scope of the present paper.

  4. 4.

    https://editorial.rottentomatoes.com/guide/marvel-movies-in-order/

  5. 5.

    https://www.latimes.com/entertainment/movies/la-et-mn-guillermo-del-toro-telluride-20170905-htmlstory.html

  6. 6.

    https://platform.openai.com/docs/api-reference.

  7. 7.

    https://github.com/Stability-AI/stablediffusion.

References

  1. Aarne, A., Thompson, S.: The Types of the Folktale. Acad. Scientiarum Fennica (1961)

    Google Scholar 

  2. Bal, M.: Narratology: Introduction to the Theory of Narrative. University of Toronto Press, Toronto (2017)

    Google Scholar 

  3. Barthes, R.: Theory of the text. In: Young, J.C. (ed.) Untying the Text: A Post-Structuralist Reader, pp. 31–47. Routledge & Kegan Paul, Oxfordshire (1981)

    Google Scholar 

  4. Brody, R.: “Batman v Superman” Is Democrats vs. Republicans, The New Yorker, March 29, 2016. https://www.newyorker.com/culture/richard-brody/batman-v-superman-is-democrats-vs-republicans. Accessed 16 May 2023

  5. Burke, K.: A Grammar of Motives. University of California Press, Oakland (1969)

    Book  Google Scholar 

  6. Carter, E. M.: ChatGPT for Screenwriters: An Easy-To-Follow Guide On How To Create A Screenplay Using Artificial Intelligence, Ethan Michael Carter (2023)

    Google Scholar 

  7. Chandler, D.: Semiotics: The Basics. Routledge, London (2002)

    Google Scholar 

  8. Ciarlini, A., Casanova, M.A., Furtado, A.L., Veloso, P.: Modeling interactive storytelling genres as application domains. J. Intell. Inf. Syst. 35(3), 347–381 (2010). https://doi.org/10.1007/s10844-009-0108-5

    Article  Google Scholar 

  9. Ciarlini, A.E.M., Pozzer, C.T., Furtado, A.L., Feijo, B.: A logic-based tool for interactive generation and dramatization of stories. In: Proceedings of the 2005 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, pp. 133–140. ACM Press, New York (2005). https://doi.org/10.1145/1178477.1178495

  10. Culler, J.: The Pursuit of Signs: Semiotics, Literature, Deconstruction. Routledge, Oxfordshire (1981)

    Google Scholar 

  11. Eco, U.: Articulations of the cinematic code. In: Nichols, B. (ed.) Movies and Methods, pp. 590–607. University of California Press, Oakland (1976)

    Google Scholar 

  12. Fauconnier, G., Turner, M.: Conceptual projection and middle spaces. Technical report 9401, University of California, San Diego (1994)

    Google Scholar 

  13. Furtado, A.L., Casanova, M.A., Barbosa, S.D.J.: A semiotic approach to conceptual modelling. In: Yu, E., Dobbie, G., Jarke, M., Purao, S. (eds.) ER 2014. LNCS, vol. 8824, pp. 1–12. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-319-12206-9_1

  14. Furtado, A.L., Casanova, M.A., Lima, E.S.: Some Preliminary Steps Towards Metaverse Logic, arXiv:2307.05574 [cs.LO] (2023). https://doi.org/10.48550/arXiv.2307.05574

  15. Furtado, A.L.: Semiotic Relations and Proof Methods. Monografias em Ciência da Computação n° 18/11. PUC-Rio, Rio de Janeiro (2011)

    Google Scholar 

  16. Gervás, P., Díaz-Agudo, B., Peinado, F., Hervás, R.: Story plot generation based on CBR. In: Macintosh, A., Ellis, R., Allen, T. (eds) Applications and Innovations in Intelligent Systems XII. SGAI 2004. Springer, London (2004). https://doi.org/10.1007/1-84628-103-2_3

  17. Gonsalves, R. A.: Using ChatGPT as a creative writing partner, towards data science. https://towardsdatascience.com/using-chatgpt-as-a-creative-writing-partner-part-1-prose-dc9a9994d41f. Accessed 16 May 2023

  18. Harty, K.J.: Looking for Arthur in all the wrong places: a note on M. night Shyamalan’s “The Sixth Sense”. Arthuriana 10(4), 57–62 (2000)

    Google Scholar 

  19. Jakobson, R.: Two aspects of language and two types of aphasic disturbances. In: Jakobson, R., Halle, M. (eds.) Fundamentals of Language. Mouton, The Hague (1956)

    Google Scholar 

  20. Jurafsky, D., Martin, J.H.: Speech and Language Processing - An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. Third Edition Draft (2023)

    Google Scholar 

  21. Karsdorp, F., van Kranenburg, P., Meder, T., Trieschnigg, D., van den Bosch, A.: In search of an appropriate abstraction level for motif annotations. In: Proceedings of the Third Workshop on Computational Models of Narrative (2012)

    Google Scholar 

  22. Kolodner, J.: Case-Based Reasoning. Morgan Kaufmann, Burlington (2014)

    Google Scholar 

  23. Lima, E.S., Feijó, B., Casanova, M.A., Furtado, A.L.: Storytelling variants based on semiotic relations. Entertainment Comput. 17, 31–44 (2016). https://doi.org/10.1016/j.entcom.2016.08.003

    Article  Google Scholar 

  24. Lima, E.S., Feijó, B., Furtado, A.L.: Adaptive storytelling based on personality and preference modeling. Entertainment Comput. 34, 100342 (2020). https://doi.org/10.1016/j.entcom.2020.100342

    Article  Google Scholar 

  25. Lima, E.S., Feijó, B., Furtado, A.L., Barbosa, S.D.J., Pozzer, C.T., Ciarlini, A.E.M.: Non-branching interactive comics. In: Reidsma, D., Katayose, H., Nijholt, A. (eds.) ACE 2013. LNCS, vol. 8253, pp. 230–245. Springer, Cham (2013). https://doi.org/10.1007/978-3-319-03161-3_16

  26. Lima, E.S., Feijó, B., Furtado, A.L.: Computational narrative blending based on planning. In: Baalsrud Hauge, J., C. S. Cardoso, J., Roque, L., Gonzalez-Calero, P.A. (eds.) ICEC 2021. LNCS, vol. 13056, pp. 289–303. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-89394-1_22

  27. Lima, E.S., Feijó, B., Furtado, A.L.: Hierarchical generation of dynamic and nondeterministic quests in games. In: Proceedings of the 11th Conference on Advances in Computer Entertainment Technology (ACE 2014), Article 24. ACM Press, New York (2014). https://doi.org/10.1145/2663806.2663833

  28. Lima, E.S., Feijó, B., Furtado, A.L.: Managing the plot structure of character-based interactive narratives in games. Entertainment Comput. 47, 100590 (2023). https://doi.org/10.1016/j.entcom.2023.100590

    Article  Google Scholar 

  29. Lima, E.S., Feijó, B., Furtado, A.L.: Procedural generation of branching quests for games. Entertainment Comput. 43, 100491 (2022). https://doi.org/10.1016/j.entcom.2022.100491

    Article  Google Scholar 

  30. Lima, E.S., Feijó, B., Furtado, A.L.: Storytelling variants: the case of little red riding hood. In: Chorianopoulos, K., Divitini, M., Baalsrud Hauge, J., Jaccheri, L., Malaka, R. (eds.) ICEC 2015. LNCS, vol. 9353, pp. 286–300. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24589-8_22

  31. Lima, E.S., Feijó, B., Furtado, A.L.: Video-based interactive storytelling using real-time video compositing techniques. Multimedia Tools Appl. 77(2), 2333–2357 (2018). https://doi.org/10.1007/s11042-017-4423-5

    Article  Google Scholar 

  32. Lima, E.S., Pozzer, C.T., d’Ornellas, M.C., Ciarlini, A.E.M., Feijó, B., Furtado, A.L.: Virtual Cinematography Director for Interactive Storytelling. In: Proceedings of the International Conference on Advances in Computer Entertainment Technology, pp. 263–270. ACM Press, New York (2009). https://doi.org/10.1145/1690388.1690432

  33. Metz, C.: Language and Cinema. De Gruyter Mouton, Boston (1974). https://doi.org/10.1515/9783110816044

  34. OpenAI: GPT-4 Technical Report. arXiv:2303.08774 (2023). [cs.CL] https://doi.org/10.48550/arXiv.2303.08774

  35. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B.: High-resolution image synthesis with latent diffusion models. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10674–10685. IEEE Press, New York (2022). https://doi.org/10.1109/CVPR52688.2022.01042

  36. Rōnin, Encyclopædia Britannica. https://www.britannica.com/topic/ronin. Accessed 16 May 2023

  37. Saussure, F.: Cours de Linguistique Générale. In: Bally, C., et al. (eds.). Payot, Paris (1995)

    Google Scholar 

  38. Samurai, Encyclopædia Britannica. https://www.britannica.com/topic/samurai. Accessed 16 May 2023

  39. Silva, F.A.G., Furtado, A.L., Ciarlini, A.E.M., Pozzer, C.T., Feijó, B., Lima, E.S.: Information-gathering events in story plots. In: Herrlich, M., Malaka, R., Masuch, M. (eds.) ICEC 2012. LNCS, vol. 7522, pp. 30–44. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33542-6_3

  40. Tehrani, J.J.: The phylogeny of little red riding hood. PLoS ONE 8(11), e78871 (2013). https://doi.org/10.1371/journal.pone.0078871

    Article  Google Scholar 

  41. University of New Hampshire: Getting the Most from ChatGPT, Teaching & Learning Resource Hub (2023). https://www.unh.edu/teaching-learning-resource-hub/resource/getting-most-chatgpt-march-2023. Accessed 10 May 2023

  42. White, H.: Tropics in Discourse: Essays in Cultural Criticism. Johns Hopkins University Press, Baltimore (1978)

    Book  Google Scholar 

  43. Winston, M.E., Chaffin, R., Herrmann, D.: A taxonomy of part-whole relations. Cogn. Sci. 11(4), 417–444 (1987). https://doi.org/10.1207/s15516709cog1104_2

    Article  Google Scholar 

Download references

Acknowledgements

We want to thank CNPq (National Council for Scientific and Technological Development) and FINEP (Funding Agency for Studies and Projects), which belong to the Ministry of Science, Technology, and Innovation of Brazil, for the financial support.

Author information

Authors and Affiliations

  1. Academy for Games and Media, Breda University of Applied Sciences, Breda, The Netherlands

    Edirlei Soares de Lima

  2. Department of Informatics, PUC-Rio, R. Marquês de São Vicente 225, Rio de Janeiro, Brazil

    Marco A. Casanova, Bruno Feijó & Antonio L. Furtado

Authors
  1. Edirlei Soares de Lima
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marco A. Casanova
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bruno Feijó
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Antonio L. Furtado
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Edirlei Soares de Lima .

Editor information

Editors and Affiliations

  1. University of Bologna, Bologna, Italy

    Paolo Ciancarini

  2. University of Bologna, Bologna, Italy

    Angelo Di Iorio

  3. University of Vienna, Vienna, Austria

    Helmut Hlavacs

  4. Italian National Research Council (CNR), Rome, Italy

    Francesco Poggi

Rights and permissions

Reprints and permissions

Copyright information

© 2023 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

de Lima, E.S., Casanova, M.A., Feijó, B., Furtado, A.L. (2023). Semiotic Structuring in Movie Narrative Generation. In: Ciancarini, P., Di Iorio, A., Hlavacs, H., Poggi, F. (eds) Entertainment Computing – ICEC 2023. ICEC 2023. Lecture Notes in Computer Science, vol 14455. Springer, Singapore. https://doi.org/10.1007/978-981-99-8248-6_13

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-8248-6_13

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-8247-9

  • Online ISBN: 978-981-99-8248-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation