Damien Ablart
Carlos Velasco
Marianna Obrist
ABSTRACT
"Seeing is believing, but feeling is the truth". This idiom from the seventieth century English clergyman Thomas Fuller gains new momentum in light of an increased proliferation of haptic technologies that allow people to have various kinds of `touch' and `touchless' interactions. Here, we report on the process of creating and integrating touchless feedback (i.e. mid-air haptic stimuli) into short movie experiences (i.e. one-minute movie format). Based on a systematic evaluation of user's experiences of those haptically enhanced movies, we show evidence for the positive effect of haptic feedback during the first viewing experience, but also for a repeated viewing after two weeks. This opens up a promising design space for content creators and researchers interested in sensory augmentation of audiovisual content. We discuss our findings and the use of mid-air haptics technologies with respect to its effect on users' emotions, changes in the viewing experience over time, and the effects of synchronisation.
- S. Ando and H. Shinoda. Ultrasonic emission tactile sensing. Control Systems, IEEE, 15(1):61--69, Feb 1995.Google Scholar
Cross Ref
- A. M. Baranowski and H. Hecht. The big picture: Effects of surround on immersion and size perception. Perception, 43(10):1061, 2014.Google ScholarCross Ref
- 3. W. Boucsein. Electrodermal activity. Springer Science & Business Media, 2012.Google ScholarCross Ref
- M. M. Bradley and P. J. Lang. Measuring emotion: The self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry, 25(1):49--59, 1994.Google ScholarCross Ref
- T. Carter, S. A. Seah, B. Long, B. Drinkwater, and S. Subramanian. Ultrahaptics: Multi-point mid-air haptic feedback for touch surfaces. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology, UIST '13, pages 505--514, New York, NY, USA, 2013. ACM. Google ScholarDigital Library
- F. Danieau, J. Fleureau, A. Cabec, P. Kerbiriou, P. Guillotel, N. Mollet, M. Christie, and A. Lecuyer. Framework for enhancing video viewing experience with haptic effects of motion. In Haptics Symposium (HAPTICS), 2012 IEEE, pages 541--546, March 2012.Google ScholarCross Ref
- M. Faust and Y.-H. Yoo. Haptic feedback in pervasive games. In Third International Workshop on Pervasive Gaming Applications, volume 7, pages 1--8, 2006.Google Scholar
- G. Ghinea, C. Timmerer, W. Lin, and S. R. Gulliver. Mulsemedia: State of the art, perspectives, and challenges. ACM Trans. Multimedia Comput. Commun. Appl., 11(1s):17:1--17:23, Oct. 2014. Google ScholarDigital Library
- D. Hecht and M. Reiner. Sensory dominance in combinations of audio, visual and haptic stimuli. Experimental brain research, 193(2):307--314, 2009.Google Scholar
- M. J. Hertenstein, R. Holmes, M. McCullough, and D. Keltner. The communication of emotion via touch. Emotion, 9(4):566--573, 2009.Google ScholarCross Ref
- S. Hwang and J. hee Ryu. The haptic steering wheel: Vibro-tactile based navigation for the driving environment. In 2010 8th IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops), pages 660--665, March 2010.Google ScholarCross Ref
- Information technology -- Media context and control. Standard, International Organization for Standardization, Geneva, CH, Jan. 2014.Google Scholar
- A. Israr, S.-C. Kim, J. Stec, and I. Poupyrev. Surround haptics: Tactile feedback for immersive gaming experiences. In CHI '12 Extended Abstracts on Human Factors in Computing Systems, CHI EA '12, pages 1087--1090, New York, NY, USA, 2012. ACM. Google ScholarDigital Library
- A. Israr, S. Zhao, K. Schwalje, R. Klatzky, and J. Lehman. Feel effects: Enriching storytelling with haptic feedback. ACM Trans. Appl. Percept., 11(3):11:1--11:17, Sept. 2014. Google ScholarDigital Library
- Y. Kim, J. Cha, I. Oakley, and J. Ryu. Exploring tactile movies: An initial tactile glove design and concept evaluation. IEEE MultiMedia, PP(99):1--1, 2015.Google Scholar
- P. Lemmens, F. Crompvoets, D. Brokken, J. van den Eerenbeemd, and G.-J. de Vries. A body-conforming tactile jacket to enrich movie viewing. In EuroHaptics conference, 2009 and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics 2009. Third Joint, pages 7--12, New York, NY, USA, March 2009. IEEE. Google ScholarDigital Library
- J. Lim, A. Wood, and B. G. Green. Derivation and evaluation of a labeled hedonic scale. Chemical senses, 34(9):739--751, 2009.Google ScholarCross Ref
- B. Long, S. A. Seah, T. Carter, and S. Subramanian. Rendering volumetric haptic shapes in mid-air using ultrasound. ACM Trans. Graph., 33(6):181:1--181:10, Nov. 2014. Google ScholarDigital Library
- R. B. Lull and B. J. Bushman. Immersed in violence: Presence mediates the effect of 3d violent video gameplay on angry feelings. 2014.Google Scholar
- N. Murray, B. Lee, Y. Qiao, and G.-M. Muntean. Olfaction-enhanced multimedia: A survey of application domains, displays, and research challenges. ACM Comput. Surv., 48(4):56:1--56:34, May 2016. Google ScholarDigital Library
- H. Nakamura and H. Miyashita. Augmented gustation using electricity. In Proceedings of the 2Nd Augmented Human International Conference, AH '11, pages 34:1--34:2, New York, NY, USA, 2011. ACM. Google ScholarDigital Library
- M. Obrist, S. A. Seah, and S. Subramanian. Talking about tactile experiences. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '13, pages 1659--1668, New York, NY, USA, 2013. ACM. Google ScholarDigital Library
- M. Obrist, S. Subramanian, E. Gatti, B. Long, and T. Carter. Emotions mediated through mid-air haptics. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI '15, pages 2053--2062, New York, NY, USA, 2015. ACM. Google ScholarDigital Library
- 24. M. Obrist, C. Velasco, C. Vi, N. Ranasinghe, A. Israr, A. Cheok, C. Spence, and P. Gopalakrishnakone. Sensing the future of hci: Touch, taste, and smell user interfaces. interactions, 23(5):40--49, Aug. 2016. Google ScholarDigital Library
- M. Obrist, C. Velasco, C. T. Vi, N. Ranasinghe, A. Israr, A. D. Cheok, C. Spence, and P. Gopalakrishnakone. Touch, taste, & smell user interfaces: The future of multisensory hci. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, CHI EA '16, pages 3285--3292, New York, NY, USA, 2016. ACM. Google ScholarDigital Library
- N. Ranasinghe and E. Y.-L. Do. Digital lollipop: Studying electrical stimulation on the human tongue to simulate taste sensations. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 13(1):5, 2016. Google ScholarDigital Library
- O. S. Schneider, A. Israr, and K. E. MacLean. Tactile animation by direct manipulation of grid displays. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, UIST '15, pages 21--30, New York, NY, USA, 2015. ACM. Google ScholarDigital Library
- R. Sodhi, I. Poupyrev, M. Glisson, and A. Israr. Aireal: Interactive tactile experiences in free air. ACM Trans. Graph., 32(4):134:1--134:10, July 2013. Google ScholarDigital Library
- C. Timmerer, M. Waltl, B. Rainer, and N. Murray. Sensory Experience: Quality of Experience Beyond Audio-Visual, pages 351--365. Springer International Publishing, Cham, 2014.Google Scholar
- S. u. Rehman, J. Sun, L. Liu, and H. Li. Turn your mobile into the ball: Rendering live football game using vibration. IEEE Transactions on Multimedia, 10(6):1022--1033, Oct 2008. Google ScholarDigital Library
- G. Wilson, T. Carter, S. Subramanian, and S. A. Brewster. Perception of ultrasonic haptic feedback on the hand: Localisation and apparent motion. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '14, pages 1133--1142, New York, NY, USA, 2014. ACM. Google ScholarDigital Library
- Y.-H. Yang, Y.-C. Lin, Y.-F. Su, and H. H. Chen. A regression approach to music emotion recognition. IEEE Transactions on audio, speech, and language processing, 16(2):448--457, 2008. Google ScholarDigital Library
- Z. Yuan, G. Ghinea, and G. M. Muntean. Quality of experience study for multiple sensorial media delivery. In 2014 International Wireless Communications and Mobile Computing Conference (IWCMC), pages 1142--1146, Aug 2014.Google ScholarCross Ref
- S. Zhao, O. Schneider, R. Klatzky, J. Lehman, and A. Israr. Feelcraft: Crafting tactile experiences for media using a feel effect library. In Proceedings of the Adjunct Publication of the 27th Annual ACM Symposium on User Interface Software and Technology, UIST'14 Adjunct, pages 51--52, New York, NY, USA, 2014. ACM. Google ScholarDigital Library
Index Terms
- Integrating Mid-Air Haptics into Movie Experiences
Recommendations
Emotions Mediated Through Mid-Air Haptics
CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing SystemsTouch is a powerful vehicle for communication between humans. The way we touch (how) embraces and mediates certain emotions such as anger, joy, fear, or love. While this phenomenon is well explored for human interaction, HCI research is only starting to ...
Movies in Mid-Air: One-Minute Movies Enhanced through Mid-Air Haptic Feedback
TVX '17 Adjunct: Adjunct Publication of the 2017 ACM International Conference on Interactive Experiences for TV and Online VideoWe present a novel movie experience which involves users' sense of touch. In our demo, we showcase this multisensory experience concept whereby a mid-air haptic technology, which creates tactile sensations in mid-air without direct contact, is ...
Understanding and Designing Embodied Experiences Through Mid-air Tactile Stimulation
TEI '18: Proceedings of the Twelfth International Conference on Tangible, Embedded, and Embodied InteractionCurrent attempts to render touch in multimedia technology still represents a challenge. Touch is indeed a complex system and there are many aspects to take into account when trying to rendering it (e.g. the compliance of an object, its weight, ...
Comments