Skip to main content
Log in

It’s like holding a human heart”: the design of Vital + Morph, a shape-changing interface for remote monitoring

  • Open Forum
  • Published:
AI & SOCIETY Aims and scope Submit manuscript

Abstract

Based on the concept of data physicalization, we developed Vital + Morph, an interactive surface for remote connection and awareness of clinical data. It enables users located in remote places to monitor and feel the vital signs measured from a hospitalized person through shape-change. We propose shape-changing interfaces as a way of making data physicalization a richer, intriguing and memorable experience that communicates complex information and insights about data. To demonstrate and validate our proposed concept, we developed an exploratory study about the design and its implications. For evaluating the social impact of shape-changing interfaces in the context of remote monitoring, we presented Vital + Morph in several Media Art festivals. We collected and analyzed the feedback from the visitors during the exhibitions, and discussed the possibilities of the proposed system. A preliminary evaluation shows how shape-changing displays are perceived by users, which establishes not only the potential benefits but also highlights the concerns that several users have raised. Through this study, we aim to contribute to the design of remote monitoring systems by providing a novel approach for displaying clinical data that consider the richness of the physical world. In today’s information-driven society, we should not just focus on how abstract data are collected and analyzed, but also on how it can be presented and incorporated into our daily lives.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Notes

  1. https://artcom.de/en/project/kinetic-sculpture/.

  2. https://www.microsoft.com/en-us/research/project/physical-charts/.

  3. https://physionet.org/physiobank/.

References

  • Barrass S (2016) Diagnosing blood pressure with acoustic sonification singing bowls. Int J Hum Comput Stud 85(C):68–71. doi:10.1016/j.ijhcs.2015.08.007

    Article  Google Scholar 

  • Bitterman N (2006) Technologies and solutions for data display in the operating room. J Clin Monit Comput 20(3):165–173. doi:10.1007/s10877-006-9017-0

    Article  Google Scholar 

  • Brown C, Hurst A (2012) VizTouch: automatically generated tactile visualizations of coordinate spaces. In: Proceedings of of TEI’12. doi:10.1145/2148131.2148160

  • Cermack M (2006) Monitoring and telemedicine support in remote environments and in human space flight. Br J Anaesth 97(1):107–114. doi:10.1093/bja/ael132

    Article  Google Scholar 

  • Coelho M, Zigelbaum J (2011) Shape-changing interfaces. Pers Ubi Comput 15(2):161–173. doi:10.1007/s00779-010-0311-y

    Article  Google Scholar 

  • Drews FA, Westenskow DR (2016) The right picture is worth a thousand numbers: data displays in anesthesia. Hum Factors 48(1):59–71. doi:10.1518/001872006776412270

    Article  Google Scholar 

  • Dubus G, Bresin R (2014) A systematic review of mapping strategies for the sonification of physical quantities. PLoS One 9:4. doi:10.1371/journal.pone.0096018

    Article  Google Scholar 

  • Follmer S, Leithinger D, Olwal A, Hogge A, Ishii H (2013) inFORM: dynamic physical affordances and constraints through shape and object actuation. In: Proceedings of UIST’13. doi:10.1145/2501988.2502032

  • Garnæs K, Grünberger O, Kjeldskov J, Skov MB (2007) Designing technologies for presence-in-absence: illustrating the cube and the picture frame. Pers Ubi Comput 11(5):403–408. doi:10.1007/s00779-006-0072-9

    Article  Google Scholar 

  • Görges M, Staggers N (2008) Evaluations of physiological monitoring displays: a systematic review. J Clin Monit Comput. doi:10.1007/s10877-007-9106-8

    Article  Google Scholar 

  • Greenberg S, Kuzuoka H (1999) Using digital but physical surrogates to mediate awareness, communication and privacy in media spaces. Pers Technol 3(4):182–198. doi:10.1007/BF01540552

    Article  Google Scholar 

  • Hardy J, Weichel C, Taher F, Vidler J, Alexander J (2015) ShapeClip: towards rapid prototyping with shape-changing displays for designers. In: Proceedings of CHI’15. doi:10.1145/2702123.2702599

  • Heiner JM, Hudson SE, Tanaka K (1999) The information percolator: ambient information display in a decorative object. In: Proceedings of UIST’99. doi:10.1145/320719.322595

  • Hemmert F, Löwe M, Wohlauf A, Joost G (2013) Animate mobiles: proxemically reactive posture actuation as a means of relational interaction with mobile phones. In: Proceedings of TEI’12. doi:10.1145/2460625.2460669

  • Iaconesi S, Persico O (2016) La cura. Codice Edizioni

  • Ishii H, Ullmer B (1997) Tangible bits: towards seamless interfaces between people, bits and atoms. In: Proceedings of CHI’97. doi:10.1145/258549.258715

  • Ishii H, Lakatos D, Bonanni L, Labrune JB (2012) Radical atoms: beyond tangible bits, toward transformable materials. Interactions 19(1):38–51. doi:10.1145/2065327.2065337

    Article  Google Scholar 

  • Iwata H (2005) Art and technology in interface devices. In: Proceedings of VRST’05. doi:10.1145/1101616.1101617

  • Iwata H, Yano H, Nakaizumi F, Kawamura R (2001) Project FEELEX: adding haptic surface to graphics. In: Proceedings of SIGGRAPH’01. doi:10.1145/383259.383314

  • Iwata H, Yano H, Ono N (2005) Volflex. In: Emerging technologies SIGGRAPH’05. doi:10.1145/1187297.1187329

  • Jansen Y, Dragicevic P, Fekete JD (2013) Evaluating the efficiency of physical visualizations. In: Proceedings of CHI’13. doi:10.1145/2470654.2481359

  • Jansen Y et al (2015) Opportunities and challenges for data physicalization. In: Proceedings of CHI’15. doi:10.1145/2702123.2702180

  • Janssen JH, Bailenson JN, Ijsselsteijn WA, Westerink JHDM (2010) Intimate heartbeats: opportunities for affective communication technology. IEEE Trans Affect Comput 1(2):72–80. doi:10.1109/T-AFFC.2010.13

    Article  Google Scholar 

  • Johnson GJ, Ambrose PJ (2006) Neo-tribes: the power and potential of online communities in health care. Commun ACM 49(1):107–113. doi:10.1145/1107458.1107463

    Article  Google Scholar 

  • Jovanon E, Starcevic D, Radivojevic V (2001) Perceptualization of biomedical data. In: Akay M, Marsh A (eds) Information technologies in medicine, vol I. Medical simulation and education. Wiley, New York, pp 189–204

    Google Scholar 

  • Kaltenbrunner M, Jorda S, Geiger G, Alonso M (2006) The reacTable*: a collaborative musical instrument. In: Proceedings of IEEE WETICE’06. doi:10.1109/WETICE.2006.68

  • Korhonen I, Parkka J, Van Gils M (2003) Health monitoring in the home of the future. IEEE Eng Med Biol 22(3):66–73. doi:10.1109/MEMB.2003.1213628

    Article  Google Scholar 

  • Kusunoki DS, Aleksandra S, Zhang Z, Burd RS (2013) Understanding visual attention of teams in dynamic medical settings through vital signs monitor use. In: Proceedings of CSCW’13. doi:10.1145/2441776.2441836

  • Leithinger D, Follmer S, Olwal A, Ishii H (2014) Physical telepresence: shape capture and display for embodied, computer-mediated remote collaboration. In: Proceedings of UIST’14. doi:10.1145/2642918.2647377

  • Lindlbauer D, Grønbæk JE, Birk M, Halskov K, Alexa M, Müller J (2016) Combining shape-changing interfaces and spatial augmented reality enables extended object appearance. In: Proceedings of CHI’16. doi:10.1145/2858036.2858457

  • Lynn G (1999) Animated form. Princeton Architectural Press, New York

    Google Scholar 

  • Maciejewski R, Choi S, Ebert DS, Tan HZ (2005) Multi-modal perceptualization of volumetric data and its application to molecular docking. In: Proceedings of world haptics conference. doi:10.1109/WHC.2005.97

  • Mankoff J, Dey AK, Hsieh G, Kientz J, Lederer S, Ames M (2003) Heuristic evaluation of ambient displays. In: Proceedings of CHI’03. doi:10.1145/642611.642642

  • Manovich L (2008) Introduction to info-aesthetics. http://www.mariabuszek.com/kcai/PoMoSeminar/Readings/ManovichInfoAesthetics.pdf. Accessed 15 June 2017

    Chapter  Google Scholar 

  • McLanders M, Santomauro C, Tran J, Sanderson P (2014) Tactile displays of pulse oximetry in integrated and separated configurations. In: Proceedings of HFES’14. doi:10.1177/1541931214581158

    Article  Google Scholar 

  • Nafus D (2016) The domestication of data: why embracing digital data means embracing bigger questions. In: Proceedings of EPIC’16. doi:10.1111/1559-8918.2016.01097

    Article  Google Scholar 

  • Nakamichi D, Nishio S, Ishiguro H (2014) Training of telecommunication through teleoperated android “Telenoid” and its effect. In: Proceedings of IEEE RO-MAN. doi:10.1109/ROMAN.2014.6926396

  • Nangalia V, Prytherch DR, Smith GB (2010) Health technology assessment review: remote monitoring of vital signs—current status and future challenges. Crit Care 14(5):233. doi:10.1186/cc9208

    Article  Google Scholar 

  • Pangaro G, Maynes-Aminzade D, Ishii H (2002) The actuated workbench: computer-controlled actuation in tabletop tangible interfaces. In: Proceedings of UIST’02. doi:10.1145/571985.572011

  • Park YW, Park J, Nam TJ (2015) The trial of Bendi in a coffeehouse: use of a shape-changing device for a tactile-visual phone conversation. In: Proceedings of CHI’15. doi:10.1145/2702123.2702326

  • Paul C (2015) From immateriality to neomateriality: art and the conditions of digital materiality. In: Proceedings of ISEA 2015. http://isea2015.org/proceeding/submissions/ISEA2015_submission_154.pdf. Accessed 15 June 2017

  • Poupyrev I, Nashida T, Okabe M (2007) Actuation and tangible user interfaces: the Vaucanson duck, robots, and shape displays. In: Proceedings of TEI’07. doi:10.1145/1226969.1227012

  • Rasmussen MK, Pedersen EW, Petersen MG, Hornbæk K (2012) Shape-changing interfaces: a review of the design space and open research questions. In: Proceedings of CHI’12. doi:10.1145/2207676.2207781

  • Schmitz M (2010) Concepts for life-like interactive objects. In: Proceedings of TEI’11. doi:10.1145/1935701.1935732

  • Sekiguchi D, Inami M, Tachi S (2001) RobotPHONE: RUI for interpersonal communication. In: Proceedings of CHI EA’01. doi:10.1145/634067.634231

  • Slovák P, Janssen J, Fitzpatrick G (2012) Understanding heart rate sharing: towards unpacking physiological space. In: Proceedings of CHI’12. doi:10.1145/2207676.2208526

  • Sommerer C, Mignonneau L (2004) Mobile feelings-wireless communication of heartbeat and breath for mobile art. In: Proceedings of ICAT’04

  • Sugiyama Y, Hirai S (2006) Crawling and jumping by a deformable robot. Int J Robot Res 25(5–6):603–620. doi:10.1177/0278364906065386

    Article  Google Scholar 

  • Taher F et al (2017) Investigating the use of a dynamic physical bar chart for data exploration and presentation. IEEE Trans Vis Comput Gr 23(1):451–460. doi:10.1109/TVCG.2016.2598498

    Article  Google Scholar 

  • Thompson D (1992) On growth and form. Cambridge University Press, Cambridge

    Book  Google Scholar 

  • Van de Moere A (2008) Beyond the tyranny of the pixel: exploring the physicality of information visualization. In: Proceedings of IV’08. doi:10.1109/IV.2008.84

  • Vertegaal R, Poupyrev I (eds) (2008) Special issue: organic user interfaces. Commun ACM 51(6)

  • Wakita A, Nakano A, Kobayashi N (2010) Programmable blobs: a rheologic interface for organic shape design. In: Proceedings of TEI’11. doi:10.1145/1935701.1935760

  • Weiser M (1991) The computer for the 21st century. Sci Am 265(3):94–104

    Article  Google Scholar 

  • Willett W, Jansen Y, Dragicevic P (2017) Embedded data representations. IEEE Trans Vis Comput Gr 23(1):461–470. doi:10.1109/TVCG.2016.2598608

    Article  Google Scholar 

  • Wisneski C et al (1998) Ambient displays: turning architectural space into an interface between people and digital information. In: Streitz NA, Konomi S, Burkhardt HJ (eds) Cooperative buildings: integrating information, organization, and architecture. Cobuild 1998, Lecture Notes in Computer Science, vol 1370. Springer, New York, pp 22–32

    Google Scholar 

  • Yao L et al (2015) bioLogic: natto cells as nanoactuators for shape changing interfaces. In: Proceedings of CHI’15. doi:10.1145/2702123.2702611

Download references

Acknowledgements

We would like to thank Kai Sasaki and Shori Kano for the help provided during the development of Vital + Morph. In addition, we want to thank Prof. Hideaki Kuzuoka, Masakazu Hirokawa, and Alessio Chierico for the fruitful discussions and comments provided during the development of this manuscript.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Alberto Boem.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Boem, A., Iwata, H. “It’s like holding a human heart”: the design of Vital + Morph, a shape-changing interface for remote monitoring. AI & Soc 33, 599–619 (2018). https://doi.org/10.1007/s00146-017-0752-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00146-017-0752-1

Keywords

Navigation