LeviSense: a platform for the multisensory integration in levitating food and insights into its effect on flavour perception

Eating is one of the most multisensory experiences in everyday life. All of our five senses (i.e. taste, smell, vision, hearing and touch) are involved, even if we are not aware of it. However, while multisensory integration has been well studied in psychology, there is not a single platform for testing systematically the effects of different stimuli. This lack of platform results in unresolved design challenges for the design of taste-based immersive experiences. Here, we present LeviSense: the first system designed for multisensory integration in gustatory experiences based on levitated food. Our system enables the systematic exploration of different sensory effects on eating experiences. It also opens up new opportunities for other professionals (e.g., molecular gastronomy chefs) looking for innovative taste-delivery platforms. We describe the design process behind LeviSense and conduct two experiments to test a subset of the crossmodal combinations (i.e., taste and vision, taste and smell). Our results show how different lighting and smell conditions affect the perceived taste intensity, pleasantness, and satisfaction. We discuss how LeviSense creates a new technical, creative, and expressive possibilities in a series of emerging design spaces within Human-Food Interaction.


Introduction
"Even before start eating a dish, several senses come into play: the smell of the dish and the look of it. 1 " -said Gaggan Anand, a molecular gastronomy chef owning a two Michelin-starred Indian restaurant in Bankok (Thailand). His view on multisensory eating experience is supported by research in Gastrophysics, which has demonstrated that the sensory impression of a dish during the process of eating relies on the integration of cues from all of the human senses (Spence, 2015b), forming the "flavour" of the consumed food. This multisensory aspect of eating leads to an emerging and promising research field in crossmodal correspondences, which investigates the augmentation and modulation of flavour perception through the change of not just the taste but other sensory modalities such as smell, sound, vision, or touch.
Findings in the research field of olfaction (sense of smell) have suggested that it contributes as much as 80% -90% to the food taste (Chartier, 2012;Spence, 2015a). Similarly, Oberfeld et al. (2009) found a strong relationship between the ambient lighting and people's perception of a glass of wine, showing that it tasted 50% sweeter when tried with red ambient light than under other colours. Similar to smell and vision, it has been shown that what we listen to can change our perception of what we are eating or drinking (Crisinel et al., 2012;North, 2012). What we see, touch, or interact with during the process of eating (e.g., the colour, size, shape, and colour of the cutlery or dishes) influences people's perception of flavour (Harrar and Spence, 2013;Piqueras-Fiszman et al., 2013). These examples illustrate a strong interaction between taste and other senses such as smell, sound, vision or haptics to create a flavourful eating experience.
As a consequence, when designing novel gustatory interfaces, it is crucial to consider the multisensory and perceptual mechanisms involved in the act of eating and tasting. To this aim, we can draw upon two pillars: (1) crossmodal research (Crisinel et al., 2012;Harrar and Spence, 2013;North, 2012;Oberfeld et al., 2009;Piqueras-Fiszman et al., 2013) and (2) inspirations from chefs' creations of novel https://doi.org/10.1016/j.ijhcs.2020.102428 Received 1 February 2019; Received in revised form 17 February 2020; Accepted 10 March 2020 multisensory eating experiences (e.g., El Bulli 2 , El Celler de Can Roca's Tocaplats 3 , Alinea's Balloon 4 , Sublimotion 5 , Etxanobe 6 , Morimoto 7 and Tru 8 ). Recently, several interfaces have been created to deliver sensory experiences to the user. Some of them include the stimulation of the sense of taste: LoLLio (Murer et al., 2013) is a small handheld device in a lollipop shape; EdiPulse, is a device that prints a message made of chocolate based on the user's heart rate (Khot et al., 2015); Bean-Counter (Maynes-Aminzade, 2005) maps different types of data with different colours of jelly beans. Other examples move beyond the sense of taste, such as Meta Cookies (Narumi et al., 2010) or Virtual Lemonade (Ranasinghe et al., 2017) that integrate the sense of taste, vision, and smell. Another example is a magnetic dining table and magnetic foods that manipulate the weight of cutlery and foods using magnetic fields (Abd Rahman et al., 2016a;2016b;2016c). However, so far there is no single platform designed to systematically control all the five senses and investigate their influence on the overall flavour perception. This is the design aim of our LeviSense system.
Recent advances in acoustic levitation have demonstrated that ultrasonic waves can be used to levitate food morsels (small pieces of solid food) or droplets (liquid drops) in mid-air and deliver them to the user's tongue (e.g., TastyFloats -Vi et al., 2017c). This type of gustatory interface is interesting because it offers a novel way of eating without cutlery, therefore leaving the user's hands available, and changes the taste perception of the levitated food (i.e., sweet, bitter, and umami tastes). However, TastyFloats presents two problems. First, TastyFloats can only levitate food morsels and in one direction (i.e., left to right), while the authors hinted the need of delivering multiple morsels in different delivery trajactories to achieve a specific experience. Second, the focus of TastyFloat is on the influence of acoustic levitation on the sense of taste: TastyFloats did not account for other sensory factors. In other words, it lacks the essential multisensory aspect of the tasting and dining experiences.
To address the above issues, we present LeviSense, the first integrated platform to investigate multisensory experiences with levitating food. LeviSense can control multiple morsels simultaneously in 3D enabling the manipulation of food's trajectories. The system supports a synchronised integration of levitated food with visual, olfactory, auditory, and tactile stimuli. Consequently, the system is capable of systematically investigating multisensory aspects around levitated food and eating experiences.
In this paper, we first discuss how to expand the work of TastyFloats (Vi et al., 2017c) from the technical and multisensory point of view. In particular, we will define the requirements for an upgraded system that controls the trajectory of multiple levitated morsels, and an augmented experience using vision, smell, directional audio and tactile feedback. We then use this upgraded system to investigate, through two user studies, the effects of other senses (i.e., vision and smell) on the user's taste perception in terms of perceived intensity, pleasantness, and satisfaction. Based on these results, we discuss how LeviSense can be used as a completed multisensorial platform to design multisensory food experiences and investigate flavour perception around levitated food. This work provides chefs and human-food interaction designers with a design tool that let them explore the combination and interaction of different sensory modalities with acoustic levitation in a flexible and interactive manner.
The contributions of this paper are: • Identifying the technical challenges in controlling multiple food morsels using acoustic levitation, and integrating/synchronizing an acoustic levitation food transportation unit with visual, auditory, tactile and olfactory stimuli.
• Demonstrating the multisensory effects that LeviSense can create through the investigation of how different lighting conditions (red, green, no lighting) and smells (vanilla, lemon, and air) influence taste perception of sweet tastes delivered using the system.
• Discuss and demonstrate how LeviSense can be used to inspire multisensory interaction designers and human-food designers.

Multisensory eating experiences
This work introduces a multisensorial platform that enables HCI researchers to investigate how different senses affect eating experiences in levitating food, and compare the results with previous non-levitating works. From this, other users can start exploring more aspects of human-food interaction (HFI). For example, exploring its four phases: growing, cooking, eating, and disposal (Khot and Mueller, 2019). However, to better contrast the differences between the two conditions, we first need to understand the influences of other senses on the sense of taste, this has only been previously explored in non-levitating food.
Our sense of taste starts in the tongue where the taste receptors capture the information about the molecules that constitute the food and drink that we eat. These signals are then transmitted to the brain which interprets the taste of the food (Trivedi, 2012). Experts in taste perception generally agree on the five basic tastes: sweet, bitter, sour, salty, and umami (Chandrashekar et al., 2006) and potentially others (such as starch (Lapis et al., 2016), metallic (Riera et al., 2007), and fat tastes (Besnard et al., 2015)).
It has been argued that what people often perceive as a 'taste' is actually a 'flavour', which combines the sensory inputs from various senses such as visual, smell, and touch (Spence, 2015a). Although it is being debated if some human senses, such as audition and vision, just modulate or actually constitute human flavour perception. Regardless of this debate, it is agreed that the flavour experience of eating is influenced by these senses (Auvray and Spence, 2008;O'Callaghan, 2015;Stokes et al., 2017). Consequently, a multisensory gustatory interface should take into account the interaction of different senses to create a satisfactory eating experience.

Impact of vision on flavor perception
It has been suggested that our flavour perception is partially established prior to the tasting moment (Piqueras-Fiszman and , through visual cues such as branding, labelling, and packaging. However, it is most often the colour that helps our brain to identify the type of food, consequently generating the expectations about its taste and flavour (Hutchings, 2003). N. DuBose et al. (1980) reported a significant influence of colour on flavour response through four experiments that assessed the effect of food colour on flavour identification, perceived intensity, and hedonic quality of beverages and cake. Clydesdale et al. (1992) demonstrated that the addition of a food red coloring increased the perceived sweetness by as much as 10%. Furthermore, O'Mahony (1983) reported a consistency in the participants' mapping of colour to taste: the colour red was matched to sweet, yellow to sour, white to salty, and green & black to bitter.
In addition, the ambient light of the environment also influences the perceived flavour of food and drinks. Oberfeld et al. (2009), for example, reported wine (100 mL of dry Riesling, Rheingau, Germany) in a red ambient light environment tasted about 50% sweeter than in either the blue or the white background colour.

Influence of smell
Previous research has suggested that as much as 80-90% of what people often describe as taste of a consumed food actually comes from the sensory inputs of smell (Spence, 2015a;Stuckey, 2012). There are two main ways in which smell can influence our taste perception. The first way is the orthonasal smell (e.g., when we are smelling through the nostrils), which modulates the expectation and the hedonic dimension of food evaluation (e.g., pleasantness) (Spence, 2016). The second way is the retronasal smell, which arises from inside the mouth during food consumption, into the nose and stimulating the olfactory epithelium (Blankenship et al., 2019). The combination of basic tastes and retronasal sensations modulates the flavour perception (e.g., sweetness or saltiness) (Spence, 2016;Stevenson et al., 1995). Additionally, the order of delivering gustatory and olfactory stimuli is an important factor and needs to be considered carefully as this is a determining factor for taste-odour integration (Kakutani et al., 2017;Spence, 2016).

Influence of sound
The physical interactions with food and drink in the mouth, such as biting, chewing, and slurping, potentially generate informative auditory cues that may influence our perception of the textural properties of the food. For example, Zampini and Spence (2004) demonstrated that potato chips tasted crisper and significantly fresher when the sound of the participants' biting action, played in real-time over closed-ear headphones, had its high-frequency components boosted. The opposite effect was observed when the high-frequency sounds were reduced. In a similar study, these authors showed that a drink was perceived as more carbonated when the sound of bubbles was amplified, or when they made the bubbles pop more frequently (Zampini and Spence, 2005).
Background sounds also contribute significantly to the taste experiences. In a work titled "as bitter as a trombone", Crisinel and Spence (2010) found that high-pitched notes were associated to sweet and sour, while low-pitched notes to umami and bitter. This allowed them to change the perceived taste of a beverage when the specific tone was played. More recently, other researchers have used more complex soundscapes to affect the perceived strength of beers (Reinoso Carvalho et al., 2016) and the sweetness of chocolate (Reinoso Carvalho et al., 2017).

Influence of haptics
"Feel it, feel its temperature, feel its sensuality, whether it's fragile or it's hard or it's wet or it's cold or it's hot, which can only be felt when you touch it" -molecular gastronomy chef Gaggan Anand asking customers to eat with their hands in his restaurant 9 . Similarly, it has been shown in psychology and sensory research that haptic perception plays an important role in flavour perception (Spence, 2015a;Stevenson et al., 2011). In particular, the intensity of sweet (i.e., glucose and sucrose) increased when the solution's temperature was increased between 20 ∘ C and 30 ∘ C (Bajec et al., 2012;Green and Frankmann, 1988). Similarly, the viscosity and texture are linked to the perception of creaminess of the consumed food (Reinoso Carvalho et al., 2017). In fact, it has been shown that one reason why people reject a certain food is because they do not like the food's texture (Nederkoorn et al., 2019).
Outside of the mouth, the haptic perception of surrounding objects (e.g. dishes and cutlery hold in the hands) can impact the perception of food and flavour (Biggs et al., 2016;Harrar and Spence, 2013;Hirose et al., 2015;Piqueras-Fiszman and Spence, 2011). Suzuki et al. (2014) suggested an improved flavor richness and aftertaste strength with a thermal stimulation on the skin around the nose. Additionally, the perception of the mechanical characteristics of foods and beverages can take advantage of the Weber illusion (Stevens and Green, 1978), where cool objects are perceived as heavier than warm objects. This consequently affects how the perceived properties of a food and drink portion differs when consumed hot or cold. However, how the changes in tactile (i.e., on the hand) lead to the changes in the perceived flavour perception (i.e., liking, pleasantness, and satisfaction) has not been studied directly.

Gustatory interfaces and acoustic levitation in HCI
There are many emerging technologies in HCI aimed at delivering novel experiences of taste. Some of them, however, focus only on taste. An example is the BeanCounter (Maynes-Aminzade, 2005), which maps specific information to the colour of jelly beans. Another example is the LOLLio (Murer et al., 2013), a small spherical device that integrate the actual taste of a candy and the sour taste that is pumped from the grip to the outlet of the candy. Other gustatory interfaces deliver a multisensory stimulation of taste, such as the Meta Cookies (Narumi et al., 2010), which simulates the taste of a plain cookie by dispensing its scent into the user's nose; or the Virtual Lemonade (Ranasinghe et al., 2017), which induces sour taste through electrical stimulation and the colour projected on to the drink. Vi et al. (2018) introduces TasteBud, a plug-and-play device that can deliver individual tastes to users with a standardised protocol. A more detailed review of gustatory interfaces in HCI can be found in Vi et al. (2017a).
In this context, acoustic levitation has shown great potential for the design of novel gustatory interfaces (Vi et al., 2017c). In TastyFloats, acoustic levitation allowed to transport and deliver food morsels in mid-air, directly from a preparation area to the user's tongue. This interface, however, only focused on the stimulation of taste. Consequently, although levitating food has been explored, as well as nonlevitating multisensory experiences, there is no previous research into the sensory combinations of levitating food. This leaves an unexplored area in creating levitation-mediated eating experiences using acoustic levitation, putting the burden on molecular gastronomy chefs and HCI designers to integrate levitating foods in their multisensorial design (e.g., in a real-life dining). To overcome this challenge, we first need a platform to explore the multisensory aspects of levitated food, as an experience of its own. The findings of this step set a foundation to further investigate eating experiences surrounding levitated food using acoustic levitation.
We imagine this multisensorial platform located at the junction between three components ( Fig. 1): the advances in acoustic levitation to levitate foods and the two growing communities in human-food interaction as identified by Altarriba . The two communities are food interaction design , and Multisensory Human-Food Interaction that explores multisensory interfaces Obrist et al., 2017; the impact of multisensory interaction on people's eating behaviour (Spence, 2017).

Design rationale
In this section, we describe the technical and perceptual challenges underpinning the design of each sensory input unit and how it is integrated in the LeviSense system. We first present the design rationale behind LeviSense, from the core levitation unit and then we tailored for dining experiences. Here, we imagine how different senses can be stimulated separately in the form of single units or parts. Then, we describe how different units are integrated and work as a single multisensorial platform for delivering levitating food.

Acoustic levitation in HCI
Acoustic levitation uses the momentum carried by sound waves to trap particles in mid-air (Brandt, 2001). In one of its simplest configurations, a standing wave is generated by two opposed emitters (or an emitter and a reflector), this standing wave traps particles at its nodes 9 https://www.bbc.co.uk/news/av/business-46840914/food-porn-starindian-chef-gives-fine-dining-a-twist, last visited 26/12/2019 C.T. Vi, et al. International Journal of Human-Computer Studies 139 (2020) 102428 Trinh, 1985). More recent techniques use singlebeams to trap particles, removing the necessity of using two opposed emitters (Andrade et al., 2018;Marzo et al., 2015). With the development of levitation techniques (Marzo and Drinkwater, 2019;Marzo et al., 2015) and open platforms for levitation (Marzo et al., 2018b), studies of user-centered interactions have started -e.g. selection techniques of the levitated particle (Freeman et al., 2018).

The core levitation unit
The core of LeviSense is the levitation unit ( Fig. 2a), where food morsels are levitated and delivered to the user's tongue. It is composed of two opposed phased-arrays and designed to generate multiple standing waves, capable of trapping and moving various levitated particles at the same time. Each array has a 16x16 ultrasonic transducers (40 kHz, 1 cm diameter -Murata MA40S4S). On each array, a single PCB (Printed Circuit Board) holds both the transducers and amplifiers. A FPGA (Field-Programmable Gate Array -Altera Cyclone IV EP4CE6) receives the phases to be emitted using UART (Universal Asynchronous Receiver-Transmitter) protocol operating at 250 kbauds, with the phases being calculated by software running on a standard PC. Serial to parallel shift registers (74HC595 8-Bit IC -Texas Instruments) multiplex 32 outputs of the FPGA into 256 independent digital channels. Mosfet Drivers (Microchip MIC4127) amplify the signals up to 15 Vpp half-square waves, that are fed into the transducers. Even with a logical input, the output pressure was sinusoidal due to the transducers being narrow-band (Marzo et al., 2018b). This hardware supports a phase resolution of π/16 radians and an update rate of 90 frames per second.
The software running on the computer needs to calculate the phases that will generate standing waves that trap the particles at the target positions. The phases can be recalculated to move the trap positions and thus the trapped particles. To calculate the phases, we use the Ultraino (Marzo et al., 2018b) framework. The employed algorithm is Iterative Backpropagation (1 iteration) (Marzo and Drinkwater, 2019). In Fig. 2(b-e), the amplitude field generated to trap two particles inside our experimental setup is shown. In Fig. 2(f, g), the force exerted on 1mm diameter spherical morsels is shown. The force is proportional to the volume of the particle, and so is the weight, thus density is the only relevant parameter to determine if levitation is possible as long as the particles are smaller than one third of the wavelength (3 mm in our case) . In theory, densities of 7.2, 3.6, 2.4 and 1.8 g/ cm 3 can be levitated for 1, 2, 3 and 4 simultaneous samples respectively. It must be noted that most of the food morsels have densities below 1.5 g/cm 3 . However, increased forces are needed to damp the oscillations on moving samples due to the low drag coefficient of air (Fushimi et al., 2018). Techniques for trapping particles larger than the wavelength have been developed (Andrade et al., 2016;Marzo et al., 2018a), but are still at an early stage and can only levitated very light

Fig. 2.
The core levitation unit: a) technical drawing (dimensions in millimeters); b) amplitude field generated by the systems in order to trap the 2 particles represented as green dots. c) zoom-in the particles area. d) x-force acting on the left particle. e) z-force acting on the left particle. The ultrasonic transducers are driven with a 15 Vpp square signal and the particles are 1 mm in diameter (size increased in the graphical representation to facilitate visualization). f) and g) are the trapping stiffness in the x-asis (f) and z-axis (g) depending on the number of simultaneous traps. The y-axis has similar forces to those in the x-axis. The values shown are the average of 1000 instances of traps generated randomly inside a cube of side 13 cm, located at the center of the two opposed arrays (minimum distance between points of 2 cm). (For interpretation of the references to colour in this figure legend, refer to the web version of this article.) C.T. Vi, et al. International Journal of Human-Computer Studies 139 (2020) 102428 materials.

Upgrading the core unit for a full dining experience
Using the levitation unit described above, food and drink particles can be levitated and moved within the space of the two phased-arrays (labelled A and B in Fig. 2a). As discussed earlier, this system already goes beyond TastyFloat, but in this section we will describe how this LeviSense can be augmented to cover multisensory experiences.
To this purpose, we have incorporated the core unit from Fig. 2a into a larger frame, designed to accommodate the same height (H = 240 mm) between the two phase-arrays, but a wider lateral access (W = 370 mm) and a narrower gap at the edge of the phased-arrays (Fig. 3a). To enable the structural integration of the system and to better support the weight of the two phased-arrays, four acrylic 'legs' (as shown in Fig. 3a) are used to attach each phased-array on the top and bottom sides. All parts of the LeviSense system are screwed together and can be 'flat-packed' to increase its portability. The design of LeviSense also allows for modular integration of the other stimuli (shown in Fig. 4), as detailed below.

Technical details
To deliver directional olfactory stimuli, we used a custom-built smell delivery device, inspired by the work of Dmitrenko et al. (2017a). The device is electrically controlled by an Arduino board and composed of 3 electro-valves (SMC VDW10EA, Solenoid/ Spring pneumatic valve) that regulate the air passage (i.e. on-off) from an ultra-low noise oil-free device compressor (8 Bar max capacity, 24 Ltrs, 93-78 L/Min at 1-2 Bar, Bambi Air UK).
The compressor supplies a regulated air-flow (max 70 l/s) through a 4 mm plastic pipe (2 mm inner diameter), purified by three carbon filters (3-stage breathing air Filter Set). The air gets split into a number of channels (e.g., 3 channels for the use of 3 smells), each is controlled by an electro-valve and arrives at a small glass bottle. The bottles contain either commercially available natural essential oils or just water (to have an odourless smell). The air supply pressure can be controlled to manipulate the delivery speed of the smell. Similarly, the duration and direction of the release determines the lingering period of the smell (Dmitrenko et al., 2017a;2017b).

Spatial and temporal design of the smell integration unit
Different smells can be released individually or in combination, by sending On/Off signals from the control unit to the Arduino (as in Fig. 4, it will be described in more details in a later section). Smells are delivered through the releasing holes on the left and right sides (on layer L1 and R1). As shown in this figure, the smell delivering pipes are hidden inside the middle layers (L2 & L3, R2 & R3), and are concealed by the most outer layer (L4, R4). With this design (in Figs. 3b and 4), smells are projected in the horizontal direction across the front side of the unit.
The delivery of scented air can be activated by a sensor that determines the user's distance and the approaching speed (as shown in Fig. 3b). The smell's release duration can be customized to fit the design purpose, for instance creating a 'scent-filled bubble' by releasing smell for 1.0 s at 20 cm distance (assuming that is the user's speed of  C.T. Vi, et al. International Journal of Human-Computer Studies 139 (2020) 102428 approaching), just before the participants arrive to have the levitating morsels delivered to the mouth.

Technical details
To emit light in the LeviSense system, four LEDs strips (WS2812B LEDs) are attached to the top and bottom sides of unit (see Figs. 4 and 3 b). The LEDs can be controlled individually to create a visual pattern (each LED emits a single colour), or a single ambient colour inside the unit (i.e., when all LEDs emit the same colour). A distance sensor (IRbased proximity sensor) is mounted at the back of the unit to continuously determine user's distance and velocity of approaching. Each LED strip is positioned on a 40 degrees angle toward the centre of the device, where the levitation of the liquid particles took place. The LEDs were controlled by an Arduino that can receive lighting instructions (ON with RGB values or OFF) from the controlling module running on a PC.

Spatial and temporal design of the visual integration unit
As presented earlier, in the Related Work section, visual cues impact on flavour perception mostly before eating. Our set-up allows for illumination to either stay continuously ON or to be activated by user's relative position through the distance sensor, in a similar way to smell activation. With this setup, more LED strips can be installed to aid the design of more complex visual animated cues. Furthermore, individual LEDs on each LED strips can be controlled separately to display different colours, a visual pattern can be a complex visual cue (e.g., to match the movement of the levitating morsels).

Design of audio integration
While sound is all around us, acoustic cues are strongly directional. Immersive content relies on the presence of binaural sound, both in real and virtual reality. In our system, two solutions for spatial acoustic stimuli were considered: a directional speaker and a wireless noisecancellation headphone.
There are two types of commercial directional speakers, both exploiting the nonlinear effects of air to produce audible sounds (Berktay, 1965;Pompei, 1999;Zabolotskaya and Khokhlov, 1969). The first and more common type (e.g. SoundLazer) exploits an array of ultrasonic transducers to produce a highly directional carrier wave, which is then modulated with audible signals (Gan et al., 2012). In this type of devices, the ultrasonic beam can be focused and steered electronically toward a specific region of space or target individual (Bourland et al., 2017;Ochiai et al., 2017). The second type (e.g. Holosonics) produces ultrasound through a vibrating plate. Systems based on both methods have been used to create audio spotlights (Yoneyama et al., 1983), of variable spatial performance (Reis, 2016). However, many of such systems are on the high-cost end of the price range. On the other hand, noise cancellation headphones come nowadays in all price ranges. Spatialized-oriented methods can learn user's positions and movements and adjust the sound between two speakers, giving an illusion of 3Dsound. They are commonly found in packages like Unity TM or Unreal TM .

Technical details
Figs. 3 b and 4 show a small directional speaker (SoundLazer, operating at 40 kHz), mounted at the front of the unit. As for light and smell, this could be kept ON during the whole taste experience or activated by the motion sensor. The speakers can emit either tonal cues or complex sound compositions.
The directional speaker is mounted so that its acoustic beam does not meet the area where the levitated morsels move. For example, it cannot be mounted at the back of the device and pointed toward the user with levitating food particles in between. Since both the directional speaker and the levitation system use the same frequency, the interference can lead to potential instabilities in the trapping force. Our preliminary tests showed that, when pointed towards a levitated droplet, our directional speaker caused it to oscillate and eventually to fall (depending on the volume of the speaker and on the droplet's position in the levitator).
In this configuration, we measured the maximum acoustic pressure level during operation, in the area accessible to the users. The measurement (147 dB at 40 kHz) was beyond the suggested safety limit for ultrasound at these frequencies (117 dB in Gan et al. (2012)), but mainly due to the levitation unit.
To minimise users' exposure, we recommended therefore the use hearing protection during the taste experiences: industrial ear defenders (3M Peltor III earmuffs) reduced the level at the ear to levels below 90 dB. It seemed therefore natural to use the headphone also for transmitting the auditory stimuli. Testing with commercial over-ear headphones (Mixcder E7) showed that, at 40 kHz, these were sufficient to bring the sound pressure levels to below 90 dB.

Design of thermal/touch integration
The sensation of touch and temperature from the food are mostly related to the perception on our tongue. However, since our hands are free from holding cutlery, we can take advantage of the availability of the hands to enrich the human-food interactions (i.e., interact and influence the food's movement, using a Kinect or LeapMotion to detect hand's movement).
Additionally, our hands are an essential part in food interaction and eating experiences, hence they are the second-best location for delivering thermal stimulation. Here, we integrate the thermal stimulation unit in the chair's armrest, users can choose whether to receive or not stimuli and when to receive it. A design example is to map the temperature of the food morsels into the thermal stimulation on the hands so that users are aware of the food temperature. This can also be used to temperature as ambient stimulus, akin to light conditions that can influence a person's tasting experience .

Technical details:
Thermal stimulation is provided to both hands of the user, by two thermoelectric cooler peltier (12V TEC1-12710). Their temperature can be precisely controlled within its operating range of −30 ∘ C to 70 ∘ C. Each peltier is mounted on a CPU water-cooling system (Cooler Master MasterLiquid 120). The temperature of each peltier is continuously monitored by a temperature sensor (DS18B20+T&R -Farnell), signalling the heating up/cooling down of the peltier to keep the set temperature, through a close-loop feedback mechanism (Proportional-Integral-Derivative controller). These components are controlled by an Arduino (Mega2560 microcontroller) and a H-bridge (2A Dual L298) allowing a smooth change in the device temperature.

Design of the thermal unit:
The components of each thermal unit (for the left and right hands) are embedded in a customised extension box, mounted on each armrest of an armchair (as shown in Fig. 3d). The mounted boxes are designed so that the users can comfortably rest their hands with their palms touching the peltiers.
Thermal stimulation on the hands can be related to the type of food being levitated, to enrich taste perception (e.g., by providing thermal stimulation that is congruent or incongruent with the taste). An example of congruent thermal stimulation would be tasting levitated icecream and having a cold stimulation on the palms. It should be considered that Peltier elements take some time to reach the desired temperature, hence this should be planned ahead (e.g., send the heating up/cooling down signals earlier). For example, the Peltier element (12V TEC1-12710) takes approximately 1 s to heat up from 25 ∘ C to 30 ∘ C at its maximum heat pump capacity (89 W). HFI designers should take this technical limitation into consideration in the design, so that it is C.T. Vi, et al. International Journal of Human-Computer Studies 139 (2020) 102428 synchronised with the movement paths of the morsels to maximise the experiences around the moment of eating (food morsels are delivered on the tongue).

The integration of the sensory modules
Based on the above technical parameters and design space of each sensory unit, a LeviSense system can deliver a multisensory eating experience to the user. As visualised in Fig. 5, a user could now choose a type of food that they would deliver to a specific position (i.e., on the mouth). The control software will: • Determine the optimized combination of different sensory input (i.e., smell, visual, thermal, auditory).
• Based on the user input, obtain the density of the food (retrieved from an internal database) and calculate the acoustic pressure needed to apply the appropriate voltage to the transducers.
• Calculate the number of food morsels to be delivered and their order of presentation (i.e., the spatial representation as well as the temporal aspect such as speed of the delivery).
Taste and drink morsels are placed and levitated at the back of the unit. The control software calculates the paths for the morsels, and the operations of each sensory unit. The activation and the duration of each sensory module can be adaptively controlled either spatially or temporally (the Activation Unit (C#) in Fig. 5

):
Spatial & temporal activation: a sensory stimulation could be activated according to the relative distance between the user and the unit or the levitating food morsels. To do this, a distance sensor (Sharp GP2Y0A60SZLF Analog Distance Sensor) is placed at the back of the LeviSense system and provides the distance between itself and the user. This example sensor has an update rate of 60 Hz and a distance measuring range of 10 cm to 150 cm. Multiple distance sensors with different measuring ranges can be mounted together to offer a continuous range (i.e., from 0 cm to beyond 150 cm).
As the food morsels' positions are controlled by the software, the distance between the user and the food morsels is easily calculated. An example of spatial activation is to release a smell when the distance between the user and the food morsels is close to zero (i.e. the eating moment -or user opens the mouth), creating a scent-bubble around the food morsels and stimulate retronasal olfaction (i.e., smell molecules travelling up the nasal passages as one is chewing). Similarly, a smell can be released way before the eating moment to stimulate orthonasal olfaction (i.e., sniffing by the nose).
Consequently, knowing where the food morsels are (i.e., indicated by the focal points, simulated by the control software), HCI designers can control when to release the smell and by how much to simulate the actual experience. For example, each sensory unit can be relative to the presentation duration and movement of the food morsels. For example, the smell intensity could be reduced accordingly to the food's exposure time, to mimic a real-world scenario.

Experiment 1: Effects of vision and acoustic levitation on sweet
Although previous research (Vi et al., 2017c) has shown that taste perception (i.e., intensity and pleasantness) is influenced by the morsel being levitated, there is no investigation on how different sensory modalities (i.e., smell and vision), individually or together, influence taste perception of levitated food. Here, we started this line of investigation by establishing the influence of a single sensory input of vision on the perceived perception of sweet taste.

Study design
We conducted a 3 × 3 within subject experiment in a counterbalanced order, comparing: three visual lighting conditions: Red, Green, and None (no additional lighting, as a control condition -see Fig. 6); and three levels of sweet concentration (Low, Medium, and High).
Each participant completed a total of 36 trials (3 lighting conditions × 3 sweet concentrations × 4 repetitions). Participants were asked not to eat, drink (apart from water), or smoke one hour before taking part in the experiment to avoid any bias of strong flavours on the taste perception (Obrist et al., 2014). The experiment lasted about 45 minutes in total and was approved by the local ethics committee.
Sweet taste was sucrose (obtained from Sigma-Aldrich) dissolved in Evian mineral water, with three levels of concentration: Low (33.47 g/ L), Medium (86.14 g/L), and High (138.80 g/L) . The taste concentrations and lighting conditions were randomised using a Latin square to avoid any order bias (Wakeling and MacFie, 1995).

Procedure
Ten participants (8 males, 2 females, mean age 30.5 years ± 5.5) volunteered for this experiment. Participants read the information sheet and signed the consent form before taking part. They were first presented with three 25 mL cups containing a 2 mL solution of three sweet concentrations (weak, medium, strong) to rinse and swallow. Another identical set of these three cups were presented to participant to rinse and swallow at the end of the experiment. This was to establish if there was a perceptual change of sweet before and after the experiment. The order was randomised between participants.
After ingesting each cup, participants were asked to answer four questions and then rinse their mouth with water: • (Q1) In your own words, what taste did you perceive? -chosen from the options of five basic tastes (sweet, bitter, sour, salty, umami), notaste, and others.
• (Q4) How satisfying was the stimulus? -using a continuous 100-point scale from very unsatisfying to very satisfying).
Participants waited 10 s before having the next cup of solution. Once the baseline measurements were done, participants began with the block of 36 trials.
Each trial started with a 10 µL droplet being placed at the centre of the device, using a micro-pipette. Participants were asked to turned away so that they did not observe this procedure. After this, participants turned back and could see that the droplet was moving toward them with a constant speed of 1 cm/s. The droplet stopped at the edge of the device, 6cm from the centre, and 0.5 cm from the device's front edge, where participants could comfortably take it with their tongue. Participants were instructed to either take the droplet whenever they feel comfortable doing so (while the droplet was moving or after it stopped). After taking the droplet, they turned around to answer the four questions (as above), then rinsed their mouth using mineral water. Participants had a countdown of 15 s on the screen before they could start the next trial. This was to prevent the habituation effect of the ingested taste (Kunka et al., 1981). Participants were given three practice trials with water in the None condition (no additional lighting) to familiarise themselves with the procedure.

Results
To determine an adequate number of participants for this experiment design, we performed a priori statistical power analysis for sample size estimation in G*Power (Faul et al., 2009;. Using a repeated measures ANOVA with three lighting conditions, three sweet concentrations, four repetitions, a power of 0.95, an alpha level of 0.05, and a large effect size ( = f 1.46, = p 0.5 2 ) (Faul et al., 2007;Lakens, 2013), the required sample size is approximately 10 participants. Thus, our number of 10 participants was adequate for the main goal of this study. Partial eta squared (η 2 ) is reported as a measure of effect size, according to Wassertheil and Cohen (1970), with a value of 0.01 as a small effect, 0.06 a medium effect, and 0.14 or greater as a large effect size.

Taste recognition
Given the small size of the levitating droplets (10 µL), we wanted to investigate if participants still recognize the sweet taste and at what concentrations. On average, participants recognised the sweet taste in 82.5% of the trials ( = SE 4.45%). Using an Univariate ANOVA analysis, we found significant differences in the sweet recognition rate between taste concentration ( = < = F p 6.32, .01, 0.86, 2,27 2 Low concentration: M = 63.33 SE = 10.26, Medium: M = 90.00 SE = 3.89, High: M = 94.17 SE = 3.52). Post-hoc tests with Bonferroni correction showed significant difference between concentrations of Low vs. Medium (p < .05) and Low vs. High (p < .01). No significant difference was found between concentrations of Medium vs. High ( = p 1.0). No significant difference was found, in terms of sweet taste recognition, between lighting conditions (p > .05). With this result, the medium concentration of sweet taste is intense enough for participants to recognise the taste clearly. Fig. 7 and Table 1 illustrates the perceived taste intensity, categorized by taste concentration (Low, Medium, High) and lighting condition (Red, Greeen, None). We performed repeated measure ANOVA on taste intensity as a dependent variable. Mauchly's test of sphericity yielded no significance ( = p 0.99) hence the collected data had sphericity correctly assumed. Significant differences within individual groups of the taste concentrations and lighting conditions were found and reported below. We found no interaction effect between the taste concentrations lighting conditions ( ). C.T. Vi, et al. International Journal of Human-Computer Studies 139 (2020)  Post-hoc tests with Bonferroni correction showed significant differences between Red vs. None (p < 0.01) and Green vs. None (p < 0.05).

Taste pleasantness and satisfaction
Similar to taste intensity, we performed repeated measure ANOVA on the taste pleasantness and satisfaction (see Table 1). The findings are similar to the taste intensity and are described in more details below.
Pleasantness: Our results show that red and green lighting condition yielded significantly more pleasantness than having no light. Similarly, medium and high concentration of sweet produced higher pleasantness than low concentration. Specifically, repeated measured ANOVA found significant differences in the taste concentration group

Intermediate discussion
From this experiment, we found that even with a small amount of liquid in the droplet (10 µL), participants still recognized the sweet taste correctly in most of the trials (82.5%). The presence of lighting or the type of lighting did not influence the taste recognition rate, as we could not find any significant differences between them. Furthermore, we found that turning the light (Red or Green) ON significantly increased the intensity, pleasantness, and satisfaction of the perceived sweet taste. However, although the perceived taste intensity with the red light was higher than with the green light, we could not find significant differences between them.

Table 1
Mean scores ( ± SE) of perceived taste intensity, pleasantness, and satisfaction in different lighting conditions (red, green, none), divided by taste concentrations (low, medium, high  C.T. Vi, et al. International Journal of Human-Computer Studies 139 (2020) 102428 5. Experiment 2: The effect of smell and acoustic levitation on sweet

Study design
In this experiment we aimed to investigate the influence of smell on the perception of levitating sweet taste. Similar to Experiment 1, we conducted a 3x3 within subject experiment comparing: three smells that were congruent (Vanilla), incongruent (Lemon), and Neutral (clean air) with sweet taste; three concentration of sweet taste (Low, Medium, and High).
The smells used were from the lemon and vanilla essential oils of Holland and Barrett 10 . They were selected based on previous crossmodal associations knowledge (Spence, 2011), suggesting Lemon is associated with sour taste and vanilla with sweet taste (Kay, 2011). The delivery of scented air was activated by a distance sensor when the user was 20 cm away from the front edge of LeviSense where the levitating morsels stop. The smell was released for a duration of 1.0 s creating a "scent-filled bubble" just before the participants took the levitating morsels.
Eleven participants (8 males, 3 females, mean age 31.00 years ± 6.13) volunteered for this experiment. Participant read the information sheet and signed the consent form before taking part. Identical sweet solutions, apparatus, and procedure as in experiment 1 was used in this experiment.

Results
Similar to Experiment 1, we performed a priori statistical power analysis for sample size estimation in G*Power (Faul et al., 2009; to determine an adequate number of participants for the presented experiment design. Using a repeated measures ANOVA with three lighting conditions, three sweet concentrations, four repetitions, a power of 0.95, an alpha level of 0.05, and a large effect size ( = f 1.46, ηp 2 = 0.5) (Lakens, 2013), the required sample size is approximately 10 participants. Thus, our number of 11 participants was adequate for the main goal of this study. Partial eta squared (η 2 ) is reported as a measure of effect size, according to Wassertheil and Cohen (1970), with a value of 0.01 as a small effect, 0.06 as a medium effect, and 0.14 or greater as a large effect size.

Taste recognition
On average, participants recognised the sweet taste in 78.89% of the trials (SE = 1.81%), as illustrated in Fig. 8a. Using multivariate ANOVA analysis, we found significant differences in the sweet recognition rate between taste concentration (p < 0.001). Post-hoc tests with Bonferroni correction showed significant difference between Low vs. Medium (p < 0.01) and Low vs. High (p < 0.001). No significant difference was found between Medium vs. High ( = p 0.50). Multivariate ANOVA with Bonferroni correction found no significant differences of sweet taste recognition between different scents (Air, Lemon, and Vanilla; p > 0.05). Table 2 & Fig. 8b shows mean values and standard error of the mean for perceived taste intensity, pleasantness, and satisfaction. We performed a repeated measure ANOVA with taste intensity, pleasantness, and satisfaction as dependent variables, concentration levels and smell conditions as independent variables. Mauchly's test of sphericity yielded no significance ( = p 0.99) for within-subject effect of concentration ( = p 0.11) and smell ( = p 0.08) hence the collected data had sphericity correctly assumed. Below we report the results of each taste perception parameter individually. Table 2 shows the detail values of mean and standard error of the mean (SE) of each taste concentration in three smell conditions.

Perceived taste intensity
Regarding perceived taste intensity, we found significant differences between taste concentrations ( ). Overall, our results illustrate that the perceived taste intensity increased accordingly to the taste concentration (Low: M = 16.27, SE = 2.28; Medium: M = 25.16, SE = 2.70; High: M = 34.60, SE = 2.64). Significant differences were found between taste concentrations ( = < F p 23.11, 0.001 (2,20) ). Post-hoc tests with Bonferroni correction showed significant difference between all pairs of Low vs. Medium, (p < 0.05), Low vs. High (p < 0.001), and Medium vs. High (p < 0.05). However, looking deeper into each categories, we found that this increase of intensity, as a result of increased concentration, only applied to Vanilla (congruent with sweet taste). Interestingly, with other smell conditions such as Lemon (incongruent) and Air (neutral), the overall intensity of the stimuli decreased with stronger taste concentrations.
Smell conditions: We found significant differences in perceived taste intensity when the levitating droplets were being eaten with different scents ( = < F p 75.84, 0.001 (2,25.88) ). Specifically, we found that having vanilla smell on sweet taste increased significantly perceived sweet intensity, compared to air (p < 0.001) and lemon (p < 0.001). Interestingly, lemon enhanced slightly perceived sweet intensity than air, but not significantly (p > 0.05).

Intermediate discussion
Experiment 2 results show that having vanilla smell increases the perceived taste intensity significantly, compared to the air and lemon scents. This result is coherent with previous work on non-levitating food where vanilla enhance taste intensity of sweetness (Risso et al., 2018;Stevenson et al., 2011). Surprisingly, having the lemon scent did not suppress the sweetness intensity but slightly enhanced it, given that lemon smell is incongruent with the sweet taste. However, despite the differences in taste intensity, all three scents did not result in different levels of taste pleasantness or satisfaction.

Discussion
In this paper, we introduced the first platform of levitating food incorporating the stimulation of all five human senses. The presented system offers more capabilities than the existing food levitating system (i.e., TastyFloats), which can only move food morsels in a single direction (1D) and focused just on the influence of acoustic levitation on taste. LeviSense, on the other hand, can control multiple morsels simultaneously in 3D between its two significantly larger levitation boards, allowing a more intuitive and flexible manipulation of food's trajectories. While the possibility of levitating multiple morsels simultaneously has not been explored here, as the scope of this work is introducing the design framework, this will allow in the future to explore the mixing of different food types or the mixing of different senses to create a novel multisensory tasting experience.
With this system, we built a solid foundation, opening a new space for exploring the multisensory aspect of levitated food. LeviSense can 10 https://www.hollandandbarrett.com/ C.T. Vi, et al. International Journal of Human-Computer Studies 139 (2020) 102428 be used as an innovative tool for chefs to display their presentation of foods in a novel way (e.g., imagine tasting the menu in front of the restaurant before ordering, to provide a tasting experience of their (bigger) dish). In this context, tasting experiences expand towards a multisensory combination of various senses, in a systematic controllable manner. Chefs, however, would need information on the effects of levitation on users taste perception. Below we discuss the effects of multisensory acoustic levitation on sweet, which we found in our two experiments and then discuss future work directions.

Multisensory acoustic levitation on sweet
In this paper, the multisensory capabilities of LeviSense were demonstrated by two experiments to examine the effects of vision (Red, Green, and No lighting -Experiment 1) and smell (Vanilla, Lemon, and Air -Experiment 2) on taste identity (recognition rates) and perception (intensity, pleasantness, and satisfaction).

On control conditions
Our results show that the control condition in Experiment 1 ("No lighting") produced a taste intensity (M = 24.13 SE = 1.73) comparable to the neutral condition in Experiment 2 ("Air": M = 20.86 SE = 2.87). Given that the same volume of droplets was administered (i.e., 10 µL), we will consider these two control conditions as the same (the small difference between the two conditions is within the SE). In addition, both values of the perceived intensity of sweet droplets are in line with the previous experiment with static levitating droplets (TastyFloats, perceived intensity of sweet droplets in Levitation condition: M = 21.27 SE = 1.56, and in Pipette condition: M = 17.39 SE = 1.38). Although it must be noted that three volumes of droplets (5, 10, and 20 µL) were used in the TastyFloats experiment, it can be inferred from our results that the perception of intensity Levitation condition is consistent. Since in TastyFloats the levitated sweet droplet produced higher perceived intensity compared to the non-levitation condition (i.e., pipette), we assume this starting condition to be the same as in this work.

Influence of light
We found that having a single lighting condition, either Red or Green, increased the perceived taste intensity, pleasantness, and satisfaction. In other words, levitated sweet becomes sweeter and more satisfactory with the light switched ON. While the increased sweetness with Red is in-line with previous findings (Clydesdale et al., 1992;Demattè et al., 2006;Huisman et al., 2016), it is not the same with the Green light. A possible explanation is that the change in taste perception might be the effect of saturation since the saturation was increased with the lighting, as previously investigated by Nishizawa et al. (2016), who showed the saturation of food colour affects perceived sweetness in a projection-based AR system. Another possible explanation for this comes from the shared common attention channel idea (Gibson, 1966), in which the individual senses are described as a functional system sharing a single and common attention channel. Hence, projecting light on the moving droplets would highlight it, making it more aesthetic and attracting more attention from participants .
We found no difference between the Red and Green light conditions although we expected one according to the Related Work, this might be due to the short duration of the light exposure; that is, the time spent interacting with LeviSense is not sufficient to establish an "ambient lighting" condition. The time of exposure to light and the possibility of using changing lights (e.g., synchronised with the movement of the levitated morsel) is an interesting direction for future studies.
In TastyFloats, the authors suggest the innovative use of levitating foods in the context of cinema, such as the Edible Cinema 11 , where users would be able to enjoy little tasty bits during the narrative of a movie. Our proposed system, LeviSense, offers even more immersive experience than TastyFloats, given its multisensory capabilities.  However, our findings show that the designer of such interaction will need to consider the lighting of the scene on the screen, as the latter can also have an effect on the perceived taste perception of the delivered food morsels.

Influence of smell
Experiment 2 shows that Vanilla enhances the perceived intensity of sweet, but not its pleasantness or satisfaction. In terms of intensity, the recorded effect is similar to the one registered with the presence of light (Red or Green), but with a larger effect (14% more than Red). The use of smell and lighting as two design dimensions can be employed by HCI designers to create interesting effect in specific scenarios. For example, Vanilla could be used in provocative scenes (i.e., in a horror movies) to intensify the perceived taste without removing the intended (unpleasant) experience, whereas lighting conditions (Red or Green) could be used in the opposite situations, if a suitable smell decreasing the perceived intensity could be found. This is not the case of lemon. We found in fact that even a strongly perceived lemon scent, which is incongruent with sweet, was not sufficient to suppress the increase in perceived intensity due to the levitation condition. This is despite the fact that the perceived intensity of lemon scent was rated as strong, compared to the moderately rated Vanilla. In our opinion, this is because the smell was perceived mostly orthonasally (through the nose), while the retronasal route of the smell was not controlled by the LeviSense system. Future studies can explore this retronasal route, using LeviSense to project smell into the participant's mouth when it is open. An alternative approach would be to embed smell in a levitated soap bubble (Zang et al., 2017) created with edible material and a specific taste (i.e., similar to the Lick-a-Bubble Edible Bubbles 12 ).
Additionally, the results of Experiment 2 showed that participants had relatively high pleasantness and satisfaction across all taste concentrations and smell conditions. However, we found that increased taste concentration did not result in increased pleasantness and satisfaction. This result is different from Experiment 1 which found increased pleasantness and satisfaction with higher sweet concentrations. This is an interesting finding and may reflect the different influences of lighting and smell on levitating sweet taste.

Future works
Initial explorations in harnessing acoustic levitation have enabled the creation of systems capable of levitating food. From the multisensorial perspective, the questions of how eating experiences change remained under-investigated, even if we are capable to do so. This work is the first attempt to build a multisensorial platform, enabling the investigation of influences of the other senses (i.e., vision and olfaction) on levitating taste. Our initial findings set out the context and learnt lessons for designers and innovators interested in multisensory experience design. Further investigations using the provided platform can provide more insights into tasting experiences of levitated foods for specific application contexts (e.g., dining experience, entertainment, VR, gaming, and education). Additionally, LeviSense provides the foundations for creating applications with levitated food in VR, multimedia (e.g., Edible Cinema), art (e.g., to manipulate the spatial presentation of different sensory stimuli and their temporal interaction as a form of art), and wellbeing (e.g., encourage children to try new foods). A VR example could be to create a StarWars-like effect, as when Anakin Skywalker used the Force to levitate a piece of fruit 13 . Such implementations would have a strong effect on existing interaction paradigm in HCI and push the boundaries in taste-based interaction design.
Our work presents an integrated platform for a computational and multisensorial approach for novel Human-Food Interaction design and research with acoustic levitation. This enables HCI designers to create playful human-food-interaction experiences. For example, food moving along paths in LeviSense can be directly controlled by customers, using an input device such as Leap Motion, enabling them to create a purposeful presentation of the foods or to create their own dish by adjusting the delivery order of the food particles. Additionally, LeviSense can be used as open-ended, experimental, social and playful venues for gastronomy & food design, e.g. the desire for more playful forms of eating (Altarriba Chisik et al., 2018;Mueller et al., 2018;Wilde and Altarriba Bertran, 2019). Future implementation can be done in a field-based method, in places such as museums (similar to the Tate Sensorium project in Vi et al., 2017b). Given the system's capabilities to provide a magical experience of levitating food, it can be used to engage children with unfamiliar flavours and healthy foods. This can be done in a similar fashion as in Vi et al. (2020) that investigated if children are ready to accept this delivery method at the dining table, in the form of a science workshop.
While there are multiple research and design directions emerging from this work, we also need to acknowledge that the current implementation of LeviSense needs more work on a user-friendly input method for designers who are interested in the mapping between the morsels' movement patterns and the user experience. We imagine that potential input devices such as Kinect and Leap Motion can be used to generate these patterns from the designer's hand or body movements. Similarly, these patterns are fed into the software controller to precisely move the levitating morsels accordingly.
LeviSense is the first system to combine all five sensory modalities into a single platform based on levitating foods. The system opens an unexplored area for HCI designers to examine various combinations of sensory input in different real-life scenarios (e.g., at a dining table, in a cinema, or as an educational tool, etc.). Since this is the first of a kind, we focused on developing the complete novel platform and demonstrate its capabilities with two cross-modalities studies: Visual + Taste (Experiment 1) and Smell + Taste (Experiment 2). The presented platform leaves a wide unexplored area for future studies to investigate the influence of thermal or audio modalities on taste, or how different combinations of multiple senses affect the levitated eating experiences.
The LeviSense system can systematically control individual senses and synchronise them, based on the spatial (user's or food's position) or temporal (how long does it take to transport the food or to get it delivered to the tongue) dimensions. Future studies can build a computational map of the sensory and multisensory effects. This could be further integrated with more complex combinations of foods, and recipes (e.g., building on RecipeScape - Chang et al., 2018).

Conclusion
We presented LeviSense, a novel system designed for multisensory integration in gustatory experiences based on levitated food. We systematically described the design process beyond LeviSense and demonstrated how different combination of lights and smells impact the users' perception of taste qualities (i.e., intensity, pleasantness, and satisfaction). We discussed the future applications of LeviSense, opening up new avenues for other audiences (e.g., molecular gastronomy chefs) which are looking for innovative new taste-delivery platforms. LeviSense aims to inspire a new design space in the context of eating and novel taste-based interactions.

Declaration of Competing Interest
Authors declare that they have no conflict of interest.