Abstract
This paper describes an approach for improving the current systems supporting the exploration and research of scientific literature, which generally adopt a query-based information-seeking paradigm. Our approach is to use a symbiotic system paradigm, exploiting central and peripheral physiological data along with eye-tracking data to adapt to users’ ongoing subjective relevance and satisfaction with search results. The system described, along with the interdisciplinary theoretical work underpinning it, could serve as a stepping stone for the development and diffusion of next-generation symbiotic systems, enabling a productive interdependence between humans and machines. After introducing the concept and evidence informing the development of symbiotic systems over a wide range of application domains, we describe the rationale of the MindSee project, emphasizing its BCI component and pinpointing the criteria around which users’ evaluations can gravitate. We conclude by summarizing the main contribution that MindSee is expected to make.
Keywords
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Licklider, J.C.R.: Man-computer symbiosis. IRE Trans. on Hum. Factors Electron. 1, 4–11 (1960)
Jacucci, G., Spagnolli, A., Freeman, J., Gamberini, L.: Symbiotic interaction: a critical definition and comparison to other human-computer paradigms. In: Jacucci, G., Gamberini, L., Freeman, J., Spagnolli, A. (eds.) Symbiotic 2014. LNCS, vol. 8820, pp. 3–20. Springer, Heidelberg (2014)
Shute, V.J., Zapata-Rivera, D.: Adaptive technologies. In: Handbook of Research on Educational Communications and Technology, pp. 277–294 (2008)
Åman, P., Liikkanen, L.A., Jacucci, G., Hinkka, A.: OUTMedia – symbiotic service for music discovery in urban augmented reality. In: Jacucci, G., Gamberini, L., Freeman, J., Spagnolli, A. (eds.) Symbiotic 2014. LNCS, vol. 8820, pp. 61–71. Springer, Heidelberg (2014)
Moreno, J.C., et al.: Symbiotic wearable robotic exoskeletons: the concept of the biomot project. In: Jacucci, G., Gamberini, L., Freeman, J., Spagnolli, A. (eds.) Symbiotic 2014. LNCS, vol. 8820, pp. 72–83. Springer, Heidelberg (2014)
Pluchino, P., Gamberini, L., Barral, O., Minelle, F.: How semantic processing of words evokes changes in pupil. In: Jacucci, G., Gamberini, L., Freeman, J., Spagnolli, A. (eds.) Symbiotic 2014. LNCS, vol. 8820, pp. 99–112. Springer, Heidelberg (2014)
Serim, B.: Querying and display of information: symbiosis in exploratory search interaction scenarios. In: Jacucci, G., Gamberini, L., Freeman, J., Spagnolli, A. (eds.) Symbiotic 2014. LNCS, vol. 8820, pp. 115–120. Springer, Heidelberg (2014)
Hore, S., Tyrvainen, L., Pyykko, J., Glowacka, D.: A reinforcement learning approach to query-less image retrieval. In: Jacucci, G., Gamberini, L., Freeman, J., Spagnolli, A. (eds.) Symbiotic 2014. LNCS, vol. 8820, pp. 121–126. Springer, Heidelberg (2014)
Barral, O., Jacucci, G.: Applying physiological computing methods to study psychological, affective and motivational relevance. In: Jacucci, G., Gamberini, L., Freeman, J., Spagnolli, A. (eds.) Symbiotic 2014. LNCS, vol. 8820, pp. 35–46. Springer, Heidelberg (2014)
Aranyi, G., Kouider, S., Lindsay, A., Prins, H., Ahmed, I., Jacucci, G., Negri, P., Gamberini, L., Pizzi, D., Cavazza, M.: Subliminal cueing of selection behavior in a virtual environment. Presence: Teleoperators Virtual Environ. 23(1), 33–50 (2014)
Barral, O., et al.: Covert persuasive technologies: bringing subliminal cues to human-computer interaction. In: Spagnolli, A., Chittaro, L., Gamberini, L. (eds.) PERSUASIVE 2014. LNCS, vol. 8462, pp. 1–12. Springer, Heidelberg (2014)
Cetnarski, R., Betella, A., Prins, H., Kouider, S., Verschure, P.F.: Subliminal Response Priming in mixed reality : The ecological validity of a classic paradigm of perception. Presence: Teleoperators Virtual Environ. 23(1), 1–17 (2014)
Negri, P., Gamberini, L., Cutini, S.: A review of the research on subliminal techniques for implicit interaction in symbiotic systems. In: Jacucci, G., Gamberini, L., Freeman, J., Spagnolli, A. (eds.) Symbiotic 2014. LNCS, vol. 8820, pp. 47–58. Springer, Heidelberg (2014)
Vidal, J.J.: Toward direct brain-computer communication. Annu. Rev. Biophys. Bioeng. 2(1), 157–180 (1973)
Wolpaw, J.R., Wolpaw, E.W.: Brain-computer interfaces: something new under the sun. In: Wolpaw, J.R., Wolpaw, E.W. (eds.) Brain-Computer Interfaces: Principles and Practice, pp. 3–12. Oxford University Press, Oxford (2012)
Birbaumer, N., Ghanayim, N., Hinterberger, T., Iversen, I., Kotchoubey, B., Kübler, A., Perelmouter, J., Taub, E., Flor, H.: A spelling device for the paralysed. Nature 398(6725), 297–298 (1999)
Sellers, E.W., Vaughan, T.M., Wolpaw, J.R.: A brain-computer interface for long-term independent home use. Amyotroph. Lateral Scler. 11, 449–455 (2010)
Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., Vaughan, T.M.: Brain-computer interfaces for communication and control. Clin. Neurophysiol. 113(6), 767–791 (2002)
Pfurtscheller, G., Allison, B.Z., Brunner, C., Bauernfeind, G., Solis-Escalante, T., Scherer, R., Zander, T.O., Mueller-Putz, G., Neuper, C., Birbaumer, N.: The hybrid BCI. Front. Neurosci. 4 (2010)
Blankertz, B., Tangermann, M., Vidaurre, C., Fazli, S., Sannelli, C., Haufe, S., Maeder, C., Ramsey, L., Sturm, I., Curio, G., Müller, K.R.: The Berlin brain-computer interface: non-medical uses of BCI technology. Front. Neurosci. 4 (2010)
Brunner, C., Birbaumer, N., Blankertz, B., Guger, C., Kübler, A., Mattia, D., Millán, J.D.R., Miralles, F., Nijholt, A., Opisso, E., Müller-Putz, G.R.: BNCI Horizon 2020: towards a roadmap for the BCI community. Brain-Computer Interfaces (ahead-of-print), 1–10 (2015)
Acqualagna, L., Bosse, S., Porbadnigk, A.K., Curio, G., Müller, K.R., Wiegand, T., Blankertz, B.: EEG-based classification of video quality perception using steady state visual evoked potentials (SSVEPs). J. Neural Eng. 12(2), 026012 (2015)
Porbadnigk, A.K., Treder, M.S., Blankertz, B., Antons, J.N., Schleicher, R., Möller, S., Curio, G., Müller, K.R.: Single-trial analysis of the neural correlates of speech quality perception. J. Neural Eng. 10(5), 056003 (2013)
Venthur, B., Blankertz, B., Gugler, M.F., Curio, G.: Novel applications of BCI technology: psychophysiological optimization of working conditions in industry. In: Systems, Man and Cybernetics (SMC 2010), pp. 417–421. IEEE Press, New York (2010)
Schultze-Kraft, M., Birman, D., Rusconi, M., Allefeld, C., Görgen, K., Dähne, S., Blankertz, B., Haynes, J.D.: Predicting and interrupting movement intentions with a closed loop. In: Presented at the 21st Annual Meeting of the Organization for Human Brain Mapping, Honolulu, USA (2015)
Di Flumeri, G., Borghini, G., Aric, P., Colosimo, A., Pozzi, S., Bonelli, S., Golfetti, A., Kong, W., Babiloni, F.: On the use of cognitive neurometric indexes in aeronautic and air traffic management environments. In: Blankertz, B., Jacucci, G., Gamberini, L., Spagnolli, A., Freeman, J. (eds.) Symbiotic 2015. LNCS, vol. 9359. Springer International Publishing, Cham (2015)
Bos, D.P.O., Reuderink, B., van de Laar, B., Gürkök, H., Mühl, C., Poel, M., Nijholt, A., Heylen, D.: Brain-computer interfacing and games. In: Tan, D., Nijholt, A. (eds.) Brain-Computer Interfaces: Applying our Minds to Human-Computer Interaction, pp. 149–178. Springer, London (2010)
Haufe, S., Treder, M.S., Gugler, M.F., Sagebaum, M., Curio, G., Blankertz, B.: EEG potentials predict upcoming emergency brakings during simulated driving. J. Neural Eng. 8(5), 056001 (2011)
Haufe, S., Kim, J.W., Kim, I.H., Sonnleitner, A., Schrauf, M., Curio, G., Blankertz, B.: Electrophysiology-based detection of emergency braking intention in real-world driving. J. Neural Eng. 11(5), 056011 (2014)
Brunner, C., et al.: BNCI horizon 2020 – towards a roadmap for brain/neural computer interaction. In: Stephanidis, C., Antona, M. (eds.) UAHCI 2014, Part I. LNCS, vol. 8513, pp. 475–486. Springer, Heidelberg (2014)
White, R.W., Roth, R.A.: Exploratory search: Beyond the Query-response Paradigm. Synthesis Lectures on Information Concepts, Retrieval and Services. Morgan and Claypool, San Francisco (2009)
Hinrichs, U., Schmidt, H., Carpendale, S.: EMDialog: Bringing information visualization into the museum. IEEE Trans. Vis. Comput. Graph. 14(6), 1181–1188 (2008)
Jacucci, G., Morrison, A., Richard, G.T., Kleimola, J., Peltonen, P., Parisi, L., Laitinen, T.: Worlds of information: designing for engagement at a public multi-touch display. In: Proceedings of the 28th International Conference on Human Factors in Computing Systems, pp. 2267–2276. ACM (2010)
Coutrix, C., Kuikkaniemi, K., Kurvinen, E., Jacucci, G., Avdouevski, I., Mäkelä, R.: Fizzyvis: designing for playful information browsing on a multitouch public display. In: Proceedings of the 2011 Conference on Designing Pleasurable Products and Interfaces, vol. 27. ACM (2011)
Hearst, M.: Search user interfaces. Cambridge University Press, Cambridge (2009)
Bates, M.J.: Where should the person stop and the information search interface start? Inf. Process. Manag. 26(5), 575–591 (1990)
Marchionini, G.: Exploratory search: from finding to understanding. Communications of the ACM 49(4), 41–46 (2006)
Teevan, J., Alvarado, C., Ackerman, M.S., Karger, D.R.: The perfect search engine is not enough: a study of orienteering behavior in directed search. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 415–422. ACM (2004)
Tombros, A., Sanderson, M.: Advantages of query biased summaries in information retrieval. In: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2–10. ACM (1998)
Manning, C.D., Raghavan, P., Schütze, H.: Introduction to Information Retrieval, pp. 1–6. Cambridge University Press, Cambridge (2008)
Limbu, D.K., Connor, A., Pears, R., MacDonell, S.: Contextual relevance feedback in web information retrieval. In: Proceedings of the 1st International Conference on Information Interaction in Context, pp. 138–143. ACM (2006)
Puolamäki, K., Salojärvi, J., Savia, E., Simola, J., Kaski, S.: Combining eye movements and collaborative filtering for proactive information retrieval. In: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 146–153. ACM (2005)
Kelly, D., Teevan, J.: Implicit feedback for inferring user preference: a bibliography. ACM SIGIR Forum 37(2), 18–28 (2003). ACM
Arapakis, I., Konstas, I., Jose, J.M.: Using facial expressions and peripheral physiological signals as implicit indicators of topical relevance. In: Proceedings of the 17th ACM International Conference on Multimedia, pp. 461–470. ACM (2009)
Damasio, A.: Descartes’ error: Emotion, reason and the human brain. Random House, New York (2008)
Pfister, H.R., Bhm, G.: The multiplicity of emotions: A framework of emotional functions in decision making. Judgm. Decis. Mak. 3(1), 5–17 (2008)
Ajanki, A., Hardoon, D.R., Kaski, S., Puolamki, K., Shawe-Taylor, J.: Can eyes reveal interest? Implicit queries from gaze patterns. User Model User-Adapt Interact. 19(4), 307–339 (2009)
Gwizdka, J.: Distribution of cognitive load in web search. J. Am. Soc. Inform. Sci. Technol. 61(11), 2167–2187 (2010)
Gwizdka, J., Cole, M.J.: Inferring Cognitive States from Multimodal Measures in Information Science (2011)
Eugster, M.J., Ruotsalo, T., Spapé, M.M., Kosunen, I., Barral, O., Ravaja, N., Jacucci, G., Kaski, S.: Predicting term-relevance from brain signals. In: Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 425–434. ACM (2014)
Barral, O., Eugster, M.J., Ruotsalo, T., Spapé, M.M., Kosunen, I., Ravaja, N., Kaski, S., Jacucci, G.: Exploring peripheral physiology as a predictor of perceived relevance in information retrieval. In: Proceedings of the 20th International Conference on Intelligent User Interfaces, pp. 389–399. ACM (2015)
Treder, M.S., Schmidt, N.M., Blankertz, B.: Gaze independent brain computer interfaces based on covert attention and feature attention. J. Neural Eng. 8(6) (2011)
Schmidt, N.M., Blankertz, B., Treder, M.S.: Online detection of error-related potentials boosts the performance of mental typewriters. BMC Neurosci. 13(1), 19 (2012)
Pomplun, M., Sunkara, S.: Pupil dilation as an indicator of cognitive workload in human-computer interaction. In: Proceedings of HCI International 2003, vol. 3, pp. 542–546. Erlbaum, Mahwah (2003)
Iqbal, S.T., Zheng, X.S., Bailey, B.P.: Task-evoked pupillary response to mental workload in human-computer interaction. In: CHI 2004 Extended Abstracts on Human Factors in Computing Systems, pp. 1477–1480. ACM, April 2004
Chen, S., Epps, J.: Using task-induced pupil diameter and blink rate to infer cognitive load. Hum-Comput. Int. 29(4), 390–413 (2014)
Oliveira, F.T., Aula, A., Russell, D.M.: Discriminating the relevance of web search results with measures of pupil size. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2209–2212. ACM (2009)
Laeng, B., Sirois, S., Gredebck, G.: Pupillometry a window to the preconscious? Perspect. Psychol. Sci. 7(1), 18–27 (2012)
Lessiter, J., Freeman, J., Miotto, A., Ferrari, E.: Ghosts in the machines: towards a taxonomy of human computer interaction. In: Jacucci, G., Gamberini, L., Freeman, J., Spagnolli, A. (eds.) Symbiotic 2014. LNCS, vol. 8820, pp. 21–31. Springer, Heidelberg (2014)
Spagnolli, A., Guardigl, E., Orso, V., Varotto, A., Gamberini, L.: Measuring user acceptance of wearable symbiotic devices: validation study across application scenarios. In: Jacucci, G., Gamberini, L., Freeman, J., Spagnolli, A. (eds.) Symbiotic 2014. LNCS, vol. 8820, pp. 87–98. Springer, Heidelberg (2014)
Buenaflor, C.B., Kim, H.C.K.: Six Human Factors to Acceptability of Wearable Computers. IJMUE 8, 103–114 (2013)
Barfield, W., Mann, S., Baird, K., Gemperle, F., Kasabach, C., Stivoric, J., Bauer, M., Martin, R., Cho, G.: Computational clothing and accessories. In: Fundamentals of Wearable Computers and Augmented Reality, pp. 471–509. Erlbaum, Mahwah (2001)
Duval, S., Hoareau, C., Hashizume, H.: 7 humanistic needs as seeds. In: Wong, W.K., Guo, Z.X. (eds.) Smart Clothing, pp. 153–187. Taylor and Francis Group, Boca Raton (2014)
Baber, C.: Wearable computers: a human factors review. Int. J. Hum-Comput. Int. 13(2), 123–145 (2001)
Rubin, J., Chisnell, D.: Handbook of usability testing: how to plan, design and conduct effective tests. John Wiley and Sons, New York (2008)
Gamberini, L., Spagnolli, A., Prontu, L., Furlan, S., Martino, F., Solaz, B.R., Alcañiz, M., Lozano, J.A.: How natural is a natural interface? An evaluation procedure based on action breakdowns. Pers. Ubiquit. Comput. 17(1), 69–79 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Gamberini, L. et al. (2015). Developing a Symbiotic System for Scientific Information Seeking: The MindSee Project. In: Blankertz, B., Jacucci, G., Gamberini, L., Spagnolli, A., Freeman, J. (eds) Symbiotic Interaction. Symbiotic 2015. Lecture Notes in Computer Science(), vol 9359. Springer, Cham. https://doi.org/10.1007/978-3-319-24917-9_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-24917-9_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-24916-2
Online ISBN: 978-3-319-24917-9
eBook Packages: Computer ScienceComputer Science (R0)