Embodied Temporalities in Cinematic Interactive Experiences

En aquest article es presenten i discuteixen diverses obres interactives que mostren diferents representacions «d'imatges-temps» en que passat i present es fusionen en un mateix espai de representacio. Les experiencies cinematiques d'aquests treballs ofereixen relacions de percepcio i consciencia temporals singulars en relacio amb el cinema, vinculades a la interaccio fisica dels participants amb la imatge digital. A partir d'una aproximacio fenomenologica i cognitiva del paper del cos en la construccio de la percepcio, s’estudien les temporalitats vinculades a la manipulacio de la imatge digital generada en temps real. Estudiar des d'aquesta perspectiva el temps ens apropa a noves comprensions de la nostra relacio amb les imatges contemporanies. Aquestes representacions digitals s'allunyen de les estructures classiques sequencials en que passat, present i futur son elements que no poden coexistir i oferint estructures mes complexes de representacio i vivencia temporal.


Time and the media
Ever since the invention of the first telegraphs and phonographs, information and communication technologies have been bringing about a series of changes to the space-time relationship that underpins our mechanised societies, in which our perception of time and space is strongly mediated by technology (Carey, 1992). This increasing technification has gradually affected our capacity to perceive space and time in line with technological progress, conceiving a compressed space-time in which our rhythms adjust to the impositions of technology: "As users of technology, we adapt our actions to the pace of the machines we use" (Lee and Liebenau, 2000, p. 49). Communication-related technologies (the telegraph, the typewriter, photography, film, television, analogue and digital video, communication networks) have produced new representations and experiences of time, due to their capacity to record, save, transmit and process content. These technologies have made it possible to manipulate time in different ways: they allow us to record and play back sounds and moving images, to repeat, pause and freeze them, and to play them back in reverse or in slow motion, for example. The consistency between time and space shifted from pre-electronic time to electronic and digital time. First, time (when) and place (where) in the pre-modern era were bound together; second, time and place in the modern era began to separate; and finally, time and space are separated in the post-modern era. (Kweon and Jo, 2011, p. 28) Two approaches to time and digital media prevail in academic literature in the field of media studies. The first considers time as a subjective experience mediated by technology (Hansen, 2004;Barker 2012), while the second studies time from the point of view of the way content is structured and processed (Hassan and Purser, 2007;Kittler, 2010). This article focuses on the first approach -the subjective experience of time, based on the way in which digital media alter or affect our perception of the self, in the present in relation to the past.
The study of new temporal practices in image technologies has tended to focus on film and communication networks, two areas in which researchers have developed concepts such as cinematic time (Doane, 2002), global-time (Virilio, 1995), timelessness (Castells, 2011), and absolute time (Lee and Liebenau, 2000). But there is much less research on the subject of temporalities in digital practices that use interactive images (Hansen, 2004;Barker, 2012). It is our belief that it is precisely these practices that can provide experiences that are highly relevant to our field of study.

The digital cinematic present
Some tangible user interfaces and interactive installations offer cinematic experiences that offer new kinds of temporal relationships with images. We believe that these types of projects require new theoretical approaches in regard to temporal perception, and that they have been largely overlooked by researchers so far (Sora, 2010). In order to study these temporal experiences, this paper will look at two case studies of interactive installations that experiment with time.
Although it is true that some works that pre-date installation formats already sought to experiment with time through interactive images, we believe that installation formats more explicitly express new cinematic temporalities in which interaction takes place through the movements and gestures of participants. One of these prior works is The Sub-Division of the Electronic Light (Hoberman, 1996): "I want to make something where time never stops completely (but not where you're trapped in an automated clockwork) where the user can play with time, where time is something malleable (however not something where the user controls time)" (Murray, 2008, p. 202).
Firstly, we will consider Transverser (Kessler, 1998), an interactive installation based on strip or slit-scan photography, in which the presence of spectators generates an image from the left to the right of the space, as their trail accumulates on the horizontal plane. The camera records the images revealed by a column of light that takes half a minute to move from the left to the right-hand side, so that the frozen present-moment is displayed for thirty seconds. Once this real-time construction reaches the right-hand edge, the mechanism begins to create a new image from the left, erasing the previous one (see Figure 1).
Transverser seems to explicitly represent the sense of the present moment as a continuous present in which the past and the future flow constantly like waves in the ocean, as theorised by Bergson: "The interactive computer-video installation acts as an apparatus rearranging the temporal and spatial structures of vision [...] The result are time-space-distorted reflections of the viewers themselves". (Description of Transverser in the exhibition catalogue Machine Times. DEAF00/V2_Organisation, 2000).
Transverser creates a time-image that is the sum of several instants in a single image. The image is constructed over a short span of time, but it is long enough to allow users to become aware of their presence in a multi-temporal image that is transformed in real time. Transverser changes (expands) the traditional time unit of film (the frame) into an image that continuously displays an extension of the previous time; an image in which time is, in a sense, subordinated to space.
In these images that are constantly constructed but, at the same time, show the wake of the past, we see a reflection of the 'virtual image' that Deleuze (2000) discusses in relation to Bergson and cinema. The two images (past and present) merge into a single object (mirror-image), creating a present that must pass in order for the new present to arrive: "We can always say that it (the present) becomes past when it no longer is, when a new present replaces it. But this is meaningless. It is clearly necessary for it to pass on for the new present to arrive, and it is clearly necessary for it to pass at the same time as it is present, at the moment that it is the present." (Deleuze, 2000, p. 79).
The second interactive installation that we will examine is L'escala (Sora and Queralt, 2013), a site-specific work created for Arts Santa Mònica in Barcelona ( Figure 2). In this work, users (unknowingly) enter the installation as they climb the stairs to the second floor of the museum. Once they are in the space, they can see a panoramic view of the four sections of the staircase recorded by four cameras.
The movement of visitors when they enter the space triggers the two different temporal modes of the images, both of which mix the images filmed in real time with stock images recorded earlier at the same place. The first mode displays a trail of the image of the visitors with a two-second delay. Visitors see themselves projected in the space as if it were a kind of mirror, but also a shadow of themselves that follows close behind them, with a short time lapse. If visitors stop, the shadow-image also stops at their actual position and is absorbed by their real image. The second temporal mode is constructed by merging images recorded earlier in the same sections of the staircase. These pre-recorded images show scenes that seem implausible in the context of a museum, such as children playing hopscotch or a girl throwing papers down from the second floor to the first. These stock images also merge with the real-time images of the visitors, usually leading visitors to think that the impossible scene is taking place in the upper parts of the staircase, even though they Transverser, Christian Kessler. 'Querläufer' DEAF_00, Rotterdam -15.11.2000 Image from the artist's website: http://lab911.net/quer/quer_rotterdam/15-11-2000/ frame_en.html L'escala, Carles Sora. Arts Santa Mònica, Barcelona, 2013. Image from the artist's website: http://www.csora.org/escala/ can see that it has been merged into the section they are currently in. The result is an experience reminiscent of the installation Telematic Dreaming (Sermon, 1992), but in this case using past and present images. It is important to note that in L'escala, unlike Transverser, the movement of the body through space makes it possible to perceive the temporalities of the image.
Visitors approaching these types of interactive works go through a first stage of figuring out the artefact (through actions that make it possible to understand the interaction) and a second stage in which they explore the piece in more detail. In phenomenology, these two stages are known as implicit time (when the body functions automatically and we're not aware of it) and explicit time (when we are conscious of the body). While we are in the first stage, the act of adjusting our actions almost prevents us from fully entering the interactive experience. Once we have generated the initial mental map of required actions, we assimilate them and automate them as ourselves (implicit time) and focus our attention on the work, allowing us to become fully conscious of the work presented to us.

The role of the body in temporal consciousness
In the interactive projects mentioned above, the body becomes a key actor in the construction of temporal consciousness. In Transverser, participants have to move through the space in order to explore the image that is being constructed in real time. If they stop moving, the resulting image is abstract and difficult for spectators to understand. The turning of the participant's body triggers this extension of the present moment, and makes the image understandable. And in L'escala, the perception of the temporality of the image is triggered as participants move through the space. In both cases, there is no representation of time in the digital time-images unless there is movement. Physical action -the constant participation of the spectator-makes it possible to create these time-images: "what they present us is not simply a 'neurocinema' rooted in the isomorphism of brain and cinematic apparatus, but a richly embodied interface with the time-image." (Hansen, 2004, p. 239).
Recently, some researchers (Dourish, 2001;Svanaes, 2013) have started to apply early phenomenological theories on the perception of time (Heidegger, 1962;Merleau-Ponty, 1993) to the field of humancomputer interaction (HCI). They argue that perception mediated by technology is an active process of the body, a process involving different parts and senses of our bodies, in interaction with our environment, in which physical action becomes a significant element in the construction of meaning. This approach to the perception of time also gives rise to a cognitive construction of subjects in interaction with their environment and the surrounding objects (Heidegger, 1962).
We perceive the world according to the possibilities for action that it offers us (Merleau-Ponty, 1993). When we interact with a digital interface for the first time, for example, our hands, along with our movement and our observation, allow us to construe the meaning of the object. Svanaes argues that there is no perception without action. He uses Stanley Kubrick's film 2001: A Space Odyssey as a metaphor, because it reminds us that when the monkey realises the bone in his hands can be used to hit things it becomes an artefact, a tool, with everything that this has meant for the human race. Or in other words, the tool becomes present for us when we discover its use and its limits, after we have explored it physically (Svanaes, 2013). Time is therefore a dimension that also emerges through the individual's action upon the world: "Time is understood as a flowing phenomenon that is ever nascent. It is never fully constituted, it always emerges from the relation of subject and object" (Förster-Beuthan, 2014).
At a more internal level, this idea of active action in the construction of perception is a very significant notion in neuroscience. Our brain works by integrating different parts and complex processes that interact with each other in a synchronised manner. This combination of signals that generate an awareness of the moment requires a small interval of time, a third of a second, according to the neurophenomenologist Francisco Varela. We are living in a present which has the duration of a short interval of time in which the past and the present coexist: "Time happens as in the rising of a wave that then subsides, like a wave that comes from the sea. Time is a dynamical process. In every trajectory you have a point that moves, and this moving point has a trail, which is the immediate past states of the trajectory". (From the exhibition catalogue Machine Times [DEAF00/V2_ORGANISATION, 2000:9]).
This trajectory of the present time which leaves a trail is represented in the images from the cinematic works mentioned above, and, more explicitly, in the work Khronos Projector (Cassinelli, 2005). In this installation, spectators can make the images in a time-lapse video move forward in time by pressing the fabric of the screen on which the image is projected. By pressing down on the screen and manipulating it with their hands, spectators can make time move forward in different parts of the image, leaving a trail of the past trajectory, in the sense that Varela refers to. And the more they press down on the screen, the further ahead the images will move in the sense of the arrow of time. But time does not move forward in the entire image frame, only in the part that spectators press and manipulate. This means that as a result of the physical action of spectators, the image is made up of past time, but also of the present.

Real-time in (of) cinematic experiences
Husserl used the phenomenological concept of longitudinal intentionality to suggest that even if the present is always different, there is always some continuity between past and present, which he calls the flow of absolute consciousness (Husserl, 1991 artnodes is a link between the interactive installations discussed above and this temporal flow that Husserl writes about, given that the images in Transverser and L'escala are created from a technological temporal flow that is linked to the flow of our temporal consciousness. Unlike cinema, in which movement-images have a duration that is predetermined by the cinematic apparatus, in these interactive timeimages, the subject manipulates computerised images by means of movement and gestures, and these images have no pre-established limit or duration. And at that precise present moment (which has a certain duration) the two temporalities (digital temporality and the temporality of the subject) become intertwined. We suggest that this link between the temporality of the digital image and our temporal consciousness is enabled by the technical capacity of numerical images to change in response to our actions or, in other words, the fact that these images are, by nature, generated in real time. Real-time images are generated in a time-continuum of interaction (Jordà, 2008), based on the numerical calculations of computers, and they are constantly constructed and modified by data input from user interfaces (in the cases discussed here, the camera's field of vision). The fact that the images are generated immediately and change in response to our actions (the temporal threshold condition that Hansen (2006) discusses) is what enables the connection between our motor system and our perceptual system. This interactivity gives rise to the total integration between the technological elements, the real-time images and our perception (Husserl's bodily schema). It is an extension of our perceptual system that can be compared to a case that Merleau-Ponty writes about, in which the blind man's cane becomes a mediator of external experience, an appendage of the body, an extension of the synthèse du corps (Merleau-Ponty, 1993). But as Hansen points out, real-time interaction, unlike a blind man's cane, does not only generate a spatial extension (physical extension) of our perception, but also enables new kinds of experiences of temporal perception (Hansen, 2006).

Conclusions
The widespread tendency in Western culture to approach the issue of time in terms of a distinction between past, present and future becomes blurred in the abovementioned interactive digital cinematic experiences, in which the temporal consciousness of images is linked to our physical actions. In linear film and video, the arrow of time was usually identifiable from left to right, as a standard cultural metaphor, but this metaphor is no longer effective for perceiving the temporalities generated by images in the works presented here. In these works, time and space are no longer a compact binomial, nor do they flow in a single direction. Metaphors such as waves in the sea or the path of a flock of birds, are more effective than the arrow of time for reflecting the temporal perceptions in these cases. In these interactive experiences, participants become aware of cinematic temporalities as a result of their actions in the duration of a present moment that makes it possible to configure a digital image generated in real time.  (2005), and completed postgraduate studies in Digital Humanities (School of Arts, Oporto, and IRCAM, Paris). He coordinated Artfutura festival, was founder of an interactive exhibition studio, and directed a postgraduate programme in Digital Theatre. He has directed interactive museum and theatre projects that he has presented at international conferences, festivals and academic forums. He has participated in several R+D founded projects as well as in a Spanish government research project.