Abstract
Based on the concept of data physicalization, we developed Vital + Morph, an interactive surface for remote connection and awareness of clinical data. It enables users located in remote places to monitor and feel the vital signs measured from a hospitalized person through shape-change. We propose shape-changing interfaces as a way of making data physicalization a richer, intriguing and memorable experience that communicates complex information and insights about data. To demonstrate and validate our proposed concept, we developed an exploratory study about the design and its implications. For evaluating the social impact of shape-changing interfaces in the context of remote monitoring, we presented Vital + Morph in several Media Art festivals. We collected and analyzed the feedback from the visitors during the exhibitions, and discussed the possibilities of the proposed system. A preliminary evaluation shows how shape-changing displays are perceived by users, which establishes not only the potential benefits but also highlights the concerns that several users have raised. Through this study, we aim to contribute to the design of remote monitoring systems by providing a novel approach for displaying clinical data that consider the richness of the physical world. In today’s information-driven society, we should not just focus on how abstract data are collected and analyzed, but also on how it can be presented and incorporated into our daily lives.
Similar content being viewed by others
References
Barrass S (2016) Diagnosing blood pressure with acoustic sonification singing bowls. Int J Hum Comput Stud 85(C):68–71. doi:10.1016/j.ijhcs.2015.08.007
Bitterman N (2006) Technologies and solutions for data display in the operating room. J Clin Monit Comput 20(3):165–173. doi:10.1007/s10877-006-9017-0
Brown C, Hurst A (2012) VizTouch: automatically generated tactile visualizations of coordinate spaces. In: Proceedings of of TEI’12. doi:10.1145/2148131.2148160
Cermack M (2006) Monitoring and telemedicine support in remote environments and in human space flight. Br J Anaesth 97(1):107–114. doi:10.1093/bja/ael132
Coelho M, Zigelbaum J (2011) Shape-changing interfaces. Pers Ubi Comput 15(2):161–173. doi:10.1007/s00779-010-0311-y
Drews FA, Westenskow DR (2016) The right picture is worth a thousand numbers: data displays in anesthesia. Hum Factors 48(1):59–71. doi:10.1518/001872006776412270
Dubus G, Bresin R (2014) A systematic review of mapping strategies for the sonification of physical quantities. PLoS One 9:4. doi:10.1371/journal.pone.0096018
Follmer S, Leithinger D, Olwal A, Hogge A, Ishii H (2013) inFORM: dynamic physical affordances and constraints through shape and object actuation. In: Proceedings of UIST’13. doi:10.1145/2501988.2502032
Garnæs K, Grünberger O, Kjeldskov J, Skov MB (2007) Designing technologies for presence-in-absence: illustrating the cube and the picture frame. Pers Ubi Comput 11(5):403–408. doi:10.1007/s00779-006-0072-9
Görges M, Staggers N (2008) Evaluations of physiological monitoring displays: a systematic review. J Clin Monit Comput. doi:10.1007/s10877-007-9106-8
Greenberg S, Kuzuoka H (1999) Using digital but physical surrogates to mediate awareness, communication and privacy in media spaces. Pers Technol 3(4):182–198. doi:10.1007/BF01540552
Hardy J, Weichel C, Taher F, Vidler J, Alexander J (2015) ShapeClip: towards rapid prototyping with shape-changing displays for designers. In: Proceedings of CHI’15. doi:10.1145/2702123.2702599
Heiner JM, Hudson SE, Tanaka K (1999) The information percolator: ambient information display in a decorative object. In: Proceedings of UIST’99. doi:10.1145/320719.322595
Hemmert F, Löwe M, Wohlauf A, Joost G (2013) Animate mobiles: proxemically reactive posture actuation as a means of relational interaction with mobile phones. In: Proceedings of TEI’12. doi:10.1145/2460625.2460669
Iaconesi S, Persico O (2016) La cura. Codice Edizioni
Ishii H, Ullmer B (1997) Tangible bits: towards seamless interfaces between people, bits and atoms. In: Proceedings of CHI’97. doi:10.1145/258549.258715
Ishii H, Lakatos D, Bonanni L, Labrune JB (2012) Radical atoms: beyond tangible bits, toward transformable materials. Interactions 19(1):38–51. doi:10.1145/2065327.2065337
Iwata H (2005) Art and technology in interface devices. In: Proceedings of VRST’05. doi:10.1145/1101616.1101617
Iwata H, Yano H, Nakaizumi F, Kawamura R (2001) Project FEELEX: adding haptic surface to graphics. In: Proceedings of SIGGRAPH’01. doi:10.1145/383259.383314
Iwata H, Yano H, Ono N (2005) Volflex. In: Emerging technologies SIGGRAPH’05. doi:10.1145/1187297.1187329
Jansen Y, Dragicevic P, Fekete JD (2013) Evaluating the efficiency of physical visualizations. In: Proceedings of CHI’13. doi:10.1145/2470654.2481359
Jansen Y et al (2015) Opportunities and challenges for data physicalization. In: Proceedings of CHI’15. doi:10.1145/2702123.2702180
Janssen JH, Bailenson JN, Ijsselsteijn WA, Westerink JHDM (2010) Intimate heartbeats: opportunities for affective communication technology. IEEE Trans Affect Comput 1(2):72–80. doi:10.1109/T-AFFC.2010.13
Johnson GJ, Ambrose PJ (2006) Neo-tribes: the power and potential of online communities in health care. Commun ACM 49(1):107–113. doi:10.1145/1107458.1107463
Jovanon E, Starcevic D, Radivojevic V (2001) Perceptualization of biomedical data. In: Akay M, Marsh A (eds) Information technologies in medicine, vol I. Medical simulation and education. Wiley, New York, pp 189–204
Kaltenbrunner M, Jorda S, Geiger G, Alonso M (2006) The reacTable*: a collaborative musical instrument. In: Proceedings of IEEE WETICE’06. doi:10.1109/WETICE.2006.68
Korhonen I, Parkka J, Van Gils M (2003) Health monitoring in the home of the future. IEEE Eng Med Biol 22(3):66–73. doi:10.1109/MEMB.2003.1213628
Kusunoki DS, Aleksandra S, Zhang Z, Burd RS (2013) Understanding visual attention of teams in dynamic medical settings through vital signs monitor use. In: Proceedings of CSCW’13. doi:10.1145/2441776.2441836
Leithinger D, Follmer S, Olwal A, Ishii H (2014) Physical telepresence: shape capture and display for embodied, computer-mediated remote collaboration. In: Proceedings of UIST’14. doi:10.1145/2642918.2647377
Lindlbauer D, Grønbæk JE, Birk M, Halskov K, Alexa M, Müller J (2016) Combining shape-changing interfaces and spatial augmented reality enables extended object appearance. In: Proceedings of CHI’16. doi:10.1145/2858036.2858457
Lynn G (1999) Animated form. Princeton Architectural Press, New York
Maciejewski R, Choi S, Ebert DS, Tan HZ (2005) Multi-modal perceptualization of volumetric data and its application to molecular docking. In: Proceedings of world haptics conference. doi:10.1109/WHC.2005.97
Mankoff J, Dey AK, Hsieh G, Kientz J, Lederer S, Ames M (2003) Heuristic evaluation of ambient displays. In: Proceedings of CHI’03. doi:10.1145/642611.642642
Manovich L (2008) Introduction to info-aesthetics. http://www.mariabuszek.com/kcai/PoMoSeminar/Readings/ManovichInfoAesthetics.pdf. Accessed 15 June 2017
McLanders M, Santomauro C, Tran J, Sanderson P (2014) Tactile displays of pulse oximetry in integrated and separated configurations. In: Proceedings of HFES’14. doi:10.1177/1541931214581158
Nafus D (2016) The domestication of data: why embracing digital data means embracing bigger questions. In: Proceedings of EPIC’16. doi:10.1111/1559-8918.2016.01097
Nakamichi D, Nishio S, Ishiguro H (2014) Training of telecommunication through teleoperated android “Telenoid” and its effect. In: Proceedings of IEEE RO-MAN. doi:10.1109/ROMAN.2014.6926396
Nangalia V, Prytherch DR, Smith GB (2010) Health technology assessment review: remote monitoring of vital signs—current status and future challenges. Crit Care 14(5):233. doi:10.1186/cc9208
Pangaro G, Maynes-Aminzade D, Ishii H (2002) The actuated workbench: computer-controlled actuation in tabletop tangible interfaces. In: Proceedings of UIST’02. doi:10.1145/571985.572011
Park YW, Park J, Nam TJ (2015) The trial of Bendi in a coffeehouse: use of a shape-changing device for a tactile-visual phone conversation. In: Proceedings of CHI’15. doi:10.1145/2702123.2702326
Paul C (2015) From immateriality to neomateriality: art and the conditions of digital materiality. In: Proceedings of ISEA 2015. http://isea2015.org/proceeding/submissions/ISEA2015_submission_154.pdf. Accessed 15 June 2017
Poupyrev I, Nashida T, Okabe M (2007) Actuation and tangible user interfaces: the Vaucanson duck, robots, and shape displays. In: Proceedings of TEI’07. doi:10.1145/1226969.1227012
Rasmussen MK, Pedersen EW, Petersen MG, Hornbæk K (2012) Shape-changing interfaces: a review of the design space and open research questions. In: Proceedings of CHI’12. doi:10.1145/2207676.2207781
Schmitz M (2010) Concepts for life-like interactive objects. In: Proceedings of TEI’11. doi:10.1145/1935701.1935732
Sekiguchi D, Inami M, Tachi S (2001) RobotPHONE: RUI for interpersonal communication. In: Proceedings of CHI EA’01. doi:10.1145/634067.634231
Slovák P, Janssen J, Fitzpatrick G (2012) Understanding heart rate sharing: towards unpacking physiological space. In: Proceedings of CHI’12. doi:10.1145/2207676.2208526
Sommerer C, Mignonneau L (2004) Mobile feelings-wireless communication of heartbeat and breath for mobile art. In: Proceedings of ICAT’04
Sugiyama Y, Hirai S (2006) Crawling and jumping by a deformable robot. Int J Robot Res 25(5–6):603–620. doi:10.1177/0278364906065386
Taher F et al (2017) Investigating the use of a dynamic physical bar chart for data exploration and presentation. IEEE Trans Vis Comput Gr 23(1):451–460. doi:10.1109/TVCG.2016.2598498
Thompson D (1992) On growth and form. Cambridge University Press, Cambridge
Van de Moere A (2008) Beyond the tyranny of the pixel: exploring the physicality of information visualization. In: Proceedings of IV’08. doi:10.1109/IV.2008.84
Vertegaal R, Poupyrev I (eds) (2008) Special issue: organic user interfaces. Commun ACM 51(6)
Wakita A, Nakano A, Kobayashi N (2010) Programmable blobs: a rheologic interface for organic shape design. In: Proceedings of TEI’11. doi:10.1145/1935701.1935760
Weiser M (1991) The computer for the 21st century. Sci Am 265(3):94–104
Willett W, Jansen Y, Dragicevic P (2017) Embedded data representations. IEEE Trans Vis Comput Gr 23(1):461–470. doi:10.1109/TVCG.2016.2598608
Wisneski C et al (1998) Ambient displays: turning architectural space into an interface between people and digital information. In: Streitz NA, Konomi S, Burkhardt HJ (eds) Cooperative buildings: integrating information, organization, and architecture. Cobuild 1998, Lecture Notes in Computer Science, vol 1370. Springer, New York, pp 22–32
Yao L et al (2015) bioLogic: natto cells as nanoactuators for shape changing interfaces. In: Proceedings of CHI’15. doi:10.1145/2702123.2702611
Acknowledgements
We would like to thank Kai Sasaki and Shori Kano for the help provided during the development of Vital + Morph. In addition, we want to thank Prof. Hideaki Kuzuoka, Masakazu Hirokawa, and Alessio Chierico for the fruitful discussions and comments provided during the development of this manuscript.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Boem, A., Iwata, H. “It’s like holding a human heart”: the design of Vital + Morph, a shape-changing interface for remote monitoring. AI & Soc 33, 599–619 (2018). https://doi.org/10.1007/s00146-017-0752-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00146-017-0752-1