Skip to main content
SpringerLink
Log in

Writing in water: dense responsive media in place of relational interfaces

  • Original Article
  • Published:
AI & SOCIETY Aims and scope Submit manuscript

Abstract

In this essay we explore extensive modes of enactive engagement among humans, physical and computational media richer than the modes represented by classical notions of interaction and relation. We make use of a radically material and a potential-theoretic account of event to re-conceive ad hoc, non-pre-schematized activity in responsive environments. We can regard such activity as sense-making via dehomogenization of material that co-articulates subjects and objects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. One can indeed attempt to fuse logical relation with material cause, but this is a much more profound project engaging C.S. Peirce, J. Petitot, M. Epperson and E. Zafiris, beyond the scope of both this essay and Newell and Simon’s thought.

  2. Newell and Simon reduce intelligence to a very restricted version of problem-solving: "To state a problem is to designate (1) a test for a class of symbol structures (solutions of the problem), and (2) a generator of symbol structures (potential solutions). To solve a problem is to generate a structure, using (2), that satisfies the test of (1).”

  3. For a more fully elaborated discussion, see Sha, Poiesis and Enchantment in Topological Matter, MIT, 2013.

  4. I thank Giuseppe Longo for this observation in his lectures at the School of Arts, Media + Engineering, Arizona State University, March 2017: vimeo.com/album/4500874.

  5. Suffice it to say that the class of Turing-computable procedures, though powerful, is in a strict sense quite a bit smaller than the class of all activities. Undecidability and indeterminacy are just the tips of the iceberg.

  6. The sea is not water, smoke is not particle system, wind is not air, and living tissue is not meat.

  7. More precisely, a field is an assignment F: M → V where M is a spatial or material manifold, and V is a manifold of features. For example M could be canonical ℝ3. More usefully it could be the composite of all surfaces in a room that can bear light from a given set of light-emitting devices, or more abstractly, a set of illuminatable objects. V could be the photometric values measured off the surfaces in a standard way. Another example would be the plenoptic function mapping locations and rays to qualities of a lightfield. P: (x, y, z, θ, φ) → (luminosity, color, polarization angle). A timebased field is an assignment F: M x I → V, where I is an interval in ℝ1, a region of what is conventionally called time. Canonical examples of timebased media are video and sound, but this definition accommodates animate etextiles, soft robotics, HVAC air conditioning, logic-controlled public lighting in sensor-infused cities, and so forth.

  8. In 1920, Russian physicist Léon Theremin invented this electronic musical instrument played by waving hands near two radio antenna without touching them. The “Theremin” instrument has developed a legendary status in research and musical performance of gesturally controlled electronic and computational media. http://www.electrotheremin.com/claramethod.html, Carolina Eyck, The Art of Playing the Theremin, Berlin: SERVI Verlag, 2006. In 2015, Google’s Advanced Technology and Projects (ATAP) announced Project Soli for building small, mobile, inexpensive radar devices allowing touchless gestural control. https://atap.google.com/soli/.

  9. To take an analog example, a profound reason that the abacus is so fast to operate is because the very act of setting the beads into the positions corresponding to the “input” already constitutes the resultant state; there are no unidimensional sequence of steps staging input, staging operator, inserting input into the machine, parsing the input into tokens and then interpreting the tokens according to a Turing-equivalent program.

  10. By a priori I mean that which pre-exists the event.

  11. Susan Stepney, "The Neglected Pillar of Material Computation,” Physica D 237 (2008): 1157–64. She argues that it may be too much work to richly exploit the unique power of biological organisms’ processes that have evolved over billions of years to solve very specific problems:

    [T]he biological substrate is extremely complex and complicated, having evolved over billions of years to exploit specific properties. In some sense, biological substrate is as far (or further!) removed from a primitive substrate as are our own designed abstract digital computational media. This makes it extremely difficult to develop any abstract models of biological material computation, or any concepts of how to exploit (program) such material.

    Hence, in order to understand, develop and exploit computation in materio, we need to move to simple (that is, unevolved) materials: move out of the domain of biology, and into that of chemistry and physics. (Stepney, 1159)

  12. Sha Xin Wei, Poiesis and Enchantment in Topological Matter, Cambridge, MA: MIT Press, 2013. Sha established Synthesis at Arizona State Univeristy in 2014 to extend the research creation work of the Topological Media Lab and its affiliates to a broader range of applications in the social and technocultural and to more deeply explore philosophical, scientific implications: http://topologicalmedialab.net/, http://synthesiscenter.net/.

  13. Topological Media Lab Meteor Shower experiment with live tracking of motion and mapping via optical flow to particles to sound: vimeo.com/tml/meteorshower (2006).

  14. See TGarden video: vimeo.com/tml/hopskip (2001).

  15. Synthesiscenter.net and topologicalmedialab.net.

  16. Teoma Naccarato and John MacCallum, hearbeat-based rhythm residency at Synthesis iStage ASU, January 2015. vimeo.com/synthesiscenter/heartbeat, Choreography and Composition of Internal Time, ccinternaltime.wordpress.com, (Naccarato and MacCallum 2017).

  17. For an argument why this is so, see (Kauffman 2016).

  18. Brender 2013, 266.

  19. Brender 2013, 270.

  20. Quoted by Harry Smoak in p. 35–36 of his topical and inventive dissertation, Meaning as Response: Experience, Behavior, and Interactive Environment Design, 2015.

References

  • Brender NM (2013) Sense-making and symmetry-breaking: merleau-ponty, cognitive science, and dynamic systems theory. Can J ContinPhilos 17(2):247–273

    Google Scholar 

  • James W (2003) “Does 'consciousness' exist?,” In: Essays in radical empiricism, Dover Publications (2003/1912).

  • Kauffman SA (2016) Humanity in a creative universe. Oxford University Press, New York

    Google Scholar 

  • Naccarato TJ, John M (2017) Critical appropriations of biosensors in artistic practice. In: Proceedings of the 4th International Conference on Movement Computing (MOCO '17), New York, ACM, 2017, 3:1–3:7. https://doi.org/10.1145/3077981.3078053

  • Newell A, Simon HA (1976) Computer science as empirical inquiry: symbols and search. Commun ACM 19(3):113–126

    Article  MathSciNet  Google Scholar 

  • Nilsson N (2007) The physical symbol system hypothesis: status and prospects. In: Lungarella M, et al. (eds) 50 Years of AI, Festschrift, LNAI 4850 (2007) 9–17.

  • Nirenberg RL, Nirenberg D (2011) Badiou’s number: a critique of mathematics as ontology. CritInq 37(4):583–613

    Google Scholar 

  • Peckham M (1970) The triumph of romanticism. University of South Carolina Press, Columbia

    Google Scholar 

  • Sha XW (2002) Resistance Is Fertile: Gesture and Agency in the Field of Responsive Media. Configurations 10(3):439–472

    Article  Google Scholar 

  • Sha XW (2013) Poiesis and enchantment in topological matter. MIT Press, Cambridge

    Book  Google Scholar 

  • Sha XW, Michael F, Navid N, Timothy S (2010) Ozone: a continuous state-based approach to media choreography. ACM Multiimedia

  • Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27 379–423, 623–656.

  • Smoak H (2015) Meaning as response: experience, behavior, and interactive environment design, PhD Dissertation, Special Individualized Programs, Concordia University, 2015.

  • Stepney S (2008) The neglected pillar of material computation. Physica D 237:1157–1164

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Arts, Media and Engineering; Synthesis, Arizona State University (ASU), Arizona, USA

    Sha Xin Wei

Authors
  1. Sha Xin Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sha Xin Wei.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sha, X. Writing in water: dense responsive media in place of relational interfaces. AI & Soc 38, 1915–1923 (2023). https://doi.org/10.1007/s00146-021-01185-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00146-021-01185-1

Keywords

Search

Navigation