Skip to main content
SpringerLink
Log in

Developing a Symbiotic System for Scientific Information Seeking: The MindSee Project

  • Conference paper
  • First Online:
Symbiotic Interaction (Symbiotic 2015)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 9359))

Included in the following conference series:

Abstract

This paper describes an approach for improving the current systems supporting the exploration and research of scientific literature, which generally adopt a query-based information-seeking paradigm. Our approach is to use a symbiotic system paradigm, exploiting central and peripheral physiological data along with eye-tracking data to adapt to users’ ongoing subjective relevance and satisfaction with search results. The system described, along with the interdisciplinary theoretical work underpinning it, could serve as a stepping stone for the development and diffusion of next-generation symbiotic systems, enabling a productive interdependence between humans and machines. After introducing the concept and evidence informing the development of symbiotic systems over a wide range of application domains, we describe the rationale of the MindSee project, emphasizing its BCI component and pinpointing the criteria around which users’ evaluations can gravitate. We conclude by summarizing the main contribution that MindSee is expected to make.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 34.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 44.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Licklider, J.C.R.: Man-computer symbiosis. IRE Trans. on Hum. Factors Electron. 1, 4–11 (1960)

    Article  Google Scholar 

  2. Jacucci, G., Spagnolli, A., Freeman, J., Gamberini, L.: Symbiotic interaction: a critical definition and comparison to other human-computer paradigms. In: Jacucci, G., Gamberini, L., Freeman, J., Spagnolli, A. (eds.) Symbiotic 2014. LNCS, vol. 8820, pp. 3–20. Springer, Heidelberg (2014)

    Google Scholar 

  3. Shute, V.J., Zapata-Rivera, D.: Adaptive technologies. In: Handbook of Research on Educational Communications and Technology, pp. 277–294 (2008)

    Google Scholar 

  4. Åman, P., Liikkanen, L.A., Jacucci, G., Hinkka, A.: OUTMedia – symbiotic service for music discovery in urban augmented reality. In: Jacucci, G., Gamberini, L., Freeman, J., Spagnolli, A. (eds.) Symbiotic 2014. LNCS, vol. 8820, pp. 61–71. Springer, Heidelberg (2014)

    Google Scholar 

  5. Moreno, J.C., et al.: Symbiotic wearable robotic exoskeletons: the concept of the biomot project. In: Jacucci, G., Gamberini, L., Freeman, J., Spagnolli, A. (eds.) Symbiotic 2014. LNCS, vol. 8820, pp. 72–83. Springer, Heidelberg (2014)

    Google Scholar 

  6. Pluchino, P., Gamberini, L., Barral, O., Minelle, F.: How semantic processing of words evokes changes in pupil. In: Jacucci, G., Gamberini, L., Freeman, J., Spagnolli, A. (eds.) Symbiotic 2014. LNCS, vol. 8820, pp. 99–112. Springer, Heidelberg (2014)

    Google Scholar 

  7. Serim, B.: Querying and display of information: symbiosis in exploratory search interaction scenarios. In: Jacucci, G., Gamberini, L., Freeman, J., Spagnolli, A. (eds.) Symbiotic 2014. LNCS, vol. 8820, pp. 115–120. Springer, Heidelberg (2014)

    Google Scholar 

  8. Hore, S., Tyrvainen, L., Pyykko, J., Glowacka, D.: A reinforcement learning approach to query-less image retrieval. In: Jacucci, G., Gamberini, L., Freeman, J., Spagnolli, A. (eds.) Symbiotic 2014. LNCS, vol. 8820, pp. 121–126. Springer, Heidelberg (2014)

    Google Scholar 

  9. Barral, O., Jacucci, G.: Applying physiological computing methods to study psychological, affective and motivational relevance. In: Jacucci, G., Gamberini, L., Freeman, J., Spagnolli, A. (eds.) Symbiotic 2014. LNCS, vol. 8820, pp. 35–46. Springer, Heidelberg (2014)

    Google Scholar 

  10. Aranyi, G., Kouider, S., Lindsay, A., Prins, H., Ahmed, I., Jacucci, G., Negri, P., Gamberini, L., Pizzi, D., Cavazza, M.: Subliminal cueing of selection behavior in a virtual environment. Presence: Teleoperators Virtual Environ. 23(1), 33–50 (2014)

    Article  Google Scholar 

  11. Barral, O., et al.: Covert persuasive technologies: bringing subliminal cues to human-computer interaction. In: Spagnolli, A., Chittaro, L., Gamberini, L. (eds.) PERSUASIVE 2014. LNCS, vol. 8462, pp. 1–12. Springer, Heidelberg (2014)

    Chapter  Google Scholar 

  12. Cetnarski, R., Betella, A., Prins, H., Kouider, S., Verschure, P.F.: Subliminal Response Priming in mixed reality : The ecological validity of a classic paradigm of perception. Presence: Teleoperators Virtual Environ. 23(1), 1–17 (2014)

    Article  Google Scholar 

  13. Negri, P., Gamberini, L., Cutini, S.: A review of the research on subliminal techniques for implicit interaction in symbiotic systems. In: Jacucci, G., Gamberini, L., Freeman, J., Spagnolli, A. (eds.) Symbiotic 2014. LNCS, vol. 8820, pp. 47–58. Springer, Heidelberg (2014)

    Google Scholar 

  14. Vidal, J.J.: Toward direct brain-computer communication. Annu. Rev. Biophys. Bioeng. 2(1), 157–180 (1973)

    Article  MathSciNet  Google Scholar 

  15. Wolpaw, J.R., Wolpaw, E.W.: Brain-computer interfaces: something new under the sun. In: Wolpaw, J.R., Wolpaw, E.W. (eds.) Brain-Computer Interfaces: Principles and Practice, pp. 3–12. Oxford University Press, Oxford (2012)

    Chapter  Google Scholar 

  16. Birbaumer, N., Ghanayim, N., Hinterberger, T., Iversen, I., Kotchoubey, B., Kübler, A., Perelmouter, J., Taub, E., Flor, H.: A spelling device for the paralysed. Nature 398(6725), 297–298 (1999)

    Article  Google Scholar 

  17. Sellers, E.W., Vaughan, T.M., Wolpaw, J.R.: A brain-computer interface for long-term independent home use. Amyotroph. Lateral Scler. 11, 449–455 (2010)

    Article  Google Scholar 

  18. Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., Vaughan, T.M.: Brain-computer interfaces for communication and control. Clin. Neurophysiol. 113(6), 767–791 (2002)

    Article  Google Scholar 

  19. Pfurtscheller, G., Allison, B.Z., Brunner, C., Bauernfeind, G., Solis-Escalante, T., Scherer, R., Zander, T.O., Mueller-Putz, G., Neuper, C., Birbaumer, N.: The hybrid BCI. Front. Neurosci. 4 (2010)

    Google Scholar 

  20. Blankertz, B., Tangermann, M., Vidaurre, C., Fazli, S., Sannelli, C., Haufe, S., Maeder, C., Ramsey, L., Sturm, I., Curio, G., Müller, K.R.: The Berlin brain-computer interface: non-medical uses of BCI technology. Front. Neurosci. 4 (2010)

    Google Scholar 

  21. Brunner, C., Birbaumer, N., Blankertz, B., Guger, C., Kübler, A., Mattia, D., Millán, J.D.R., Miralles, F., Nijholt, A., Opisso, E., Müller-Putz, G.R.: BNCI Horizon 2020: towards a roadmap for the BCI community. Brain-Computer Interfaces (ahead-of-print), 1–10 (2015)

    Google Scholar 

  22. Acqualagna, L., Bosse, S., Porbadnigk, A.K., Curio, G., Müller, K.R., Wiegand, T., Blankertz, B.: EEG-based classification of video quality perception using steady state visual evoked potentials (SSVEPs). J. Neural Eng. 12(2), 026012 (2015)

    Article  Google Scholar 

  23. Porbadnigk, A.K., Treder, M.S., Blankertz, B., Antons, J.N., Schleicher, R., Möller, S., Curio, G., Müller, K.R.: Single-trial analysis of the neural correlates of speech quality perception. J. Neural Eng. 10(5), 056003 (2013)

    Article  Google Scholar 

  24. Venthur, B., Blankertz, B., Gugler, M.F., Curio, G.: Novel applications of BCI technology: psychophysiological optimization of working conditions in industry. In: Systems, Man and Cybernetics (SMC 2010), pp. 417–421. IEEE Press, New York (2010)

    Google Scholar 

  25. Schultze-Kraft, M., Birman, D., Rusconi, M., Allefeld, C., Görgen, K., Dähne, S., Blankertz, B., Haynes, J.D.: Predicting and interrupting movement intentions with a closed loop. In: Presented at the 21st Annual Meeting of the Organization for Human Brain Mapping, Honolulu, USA (2015)

    Google Scholar 

  26. Di Flumeri, G., Borghini, G., Aric, P., Colosimo, A., Pozzi, S., Bonelli, S., Golfetti, A., Kong, W., Babiloni, F.: On the use of cognitive neurometric indexes in aeronautic and air traffic management environments. In: Blankertz, B., Jacucci, G., Gamberini, L., Spagnolli, A., Freeman, J. (eds.) Symbiotic 2015. LNCS, vol. 9359. Springer International Publishing, Cham (2015)

    Chapter  Google Scholar 

  27. Bos, D.P.O., Reuderink, B., van de Laar, B., Gürkök, H., Mühl, C., Poel, M., Nijholt, A., Heylen, D.: Brain-computer interfacing and games. In: Tan, D., Nijholt, A. (eds.) Brain-Computer Interfaces: Applying our Minds to Human-Computer Interaction, pp. 149–178. Springer, London (2010)

    Google Scholar 

  28. Haufe, S., Treder, M.S., Gugler, M.F., Sagebaum, M., Curio, G., Blankertz, B.: EEG potentials predict upcoming emergency brakings during simulated driving. J. Neural Eng. 8(5), 056001 (2011)

    Article  Google Scholar 

  29. Haufe, S., Kim, J.W., Kim, I.H., Sonnleitner, A., Schrauf, M., Curio, G., Blankertz, B.: Electrophysiology-based detection of emergency braking intention in real-world driving. J. Neural Eng. 11(5), 056011 (2014)

    Article  Google Scholar 

  30. Brunner, C., et al.: BNCI horizon 2020 – towards a roadmap for brain/neural computer interaction. In: Stephanidis, C., Antona, M. (eds.) UAHCI 2014, Part I. LNCS, vol. 8513, pp. 475–486. Springer, Heidelberg (2014)

    Google Scholar 

  31. White, R.W., Roth, R.A.: Exploratory search: Beyond the Query-response Paradigm. Synthesis Lectures on Information Concepts, Retrieval and Services. Morgan and Claypool, San Francisco (2009)

    Google Scholar 

  32. Hinrichs, U., Schmidt, H., Carpendale, S.: EMDialog: Bringing information visualization into the museum. IEEE Trans. Vis. Comput. Graph. 14(6), 1181–1188 (2008)

    Article  Google Scholar 

  33. Jacucci, G., Morrison, A., Richard, G.T., Kleimola, J., Peltonen, P., Parisi, L., Laitinen, T.: Worlds of information: designing for engagement at a public multi-touch display. In: Proceedings of the 28th International Conference on Human Factors in Computing Systems, pp. 2267–2276. ACM (2010)

    Google Scholar 

  34. Coutrix, C., Kuikkaniemi, K., Kurvinen, E., Jacucci, G., Avdouevski, I., Mäkelä, R.: Fizzyvis: designing for playful information browsing on a multitouch public display. In: Proceedings of the 2011 Conference on Designing Pleasurable Products and Interfaces, vol. 27. ACM (2011)

    Google Scholar 

  35. Hearst, M.: Search user interfaces. Cambridge University Press, Cambridge (2009)

    Book  Google Scholar 

  36. Bates, M.J.: Where should the person stop and the information search interface start? Inf. Process. Manag. 26(5), 575–591 (1990)

    Article  Google Scholar 

  37. Marchionini, G.: Exploratory search: from finding to understanding. Communications of the ACM 49(4), 41–46 (2006)

    Article  Google Scholar 

  38. Teevan, J., Alvarado, C., Ackerman, M.S., Karger, D.R.: The perfect search engine is not enough: a study of orienteering behavior in directed search. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 415–422. ACM (2004)

    Google Scholar 

  39. Tombros, A., Sanderson, M.: Advantages of query biased summaries in information retrieval. In: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2–10. ACM (1998)

    Google Scholar 

  40. Manning, C.D., Raghavan, P., Schütze, H.: Introduction to Information Retrieval, pp. 1–6. Cambridge University Press, Cambridge (2008)

    Book  MATH  Google Scholar 

  41. Limbu, D.K., Connor, A., Pears, R., MacDonell, S.: Contextual relevance feedback in web information retrieval. In: Proceedings of the 1st International Conference on Information Interaction in Context, pp. 138–143. ACM (2006)

    Google Scholar 

  42. Puolamäki, K., Salojärvi, J., Savia, E., Simola, J., Kaski, S.: Combining eye movements and collaborative filtering for proactive information retrieval. In: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 146–153. ACM (2005)

    Google Scholar 

  43. Kelly, D., Teevan, J.: Implicit feedback for inferring user preference: a bibliography. ACM SIGIR Forum 37(2), 18–28 (2003). ACM

    Article  Google Scholar 

  44. Arapakis, I., Konstas, I., Jose, J.M.: Using facial expressions and peripheral physiological signals as implicit indicators of topical relevance. In: Proceedings of the 17th ACM International Conference on Multimedia, pp. 461–470. ACM (2009)

    Google Scholar 

  45. Damasio, A.: Descartes’ error: Emotion, reason and the human brain. Random House, New York (2008)

    Google Scholar 

  46. Pfister, H.R., Bhm, G.: The multiplicity of emotions: A framework of emotional functions in decision making. Judgm. Decis. Mak. 3(1), 5–17 (2008)

    Google Scholar 

  47. Ajanki, A., Hardoon, D.R., Kaski, S., Puolamki, K., Shawe-Taylor, J.: Can eyes reveal interest? Implicit queries from gaze patterns. User Model User-Adapt Interact. 19(4), 307–339 (2009)

    Article  Google Scholar 

  48. Gwizdka, J.: Distribution of cognitive load in web search. J. Am. Soc. Inform. Sci. Technol. 61(11), 2167–2187 (2010)

    Article  Google Scholar 

  49. Gwizdka, J., Cole, M.J.: Inferring Cognitive States from Multimodal Measures in Information Science (2011)

    Google Scholar 

  50. Eugster, M.J., Ruotsalo, T., Spapé, M.M., Kosunen, I., Barral, O., Ravaja, N., Jacucci, G., Kaski, S.: Predicting term-relevance from brain signals. In: Proceedings of the 37th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 425–434. ACM (2014)

    Google Scholar 

  51. Barral, O., Eugster, M.J., Ruotsalo, T., Spapé, M.M., Kosunen, I., Ravaja, N., Kaski, S., Jacucci, G.: Exploring peripheral physiology as a predictor of perceived relevance in information retrieval. In: Proceedings of the 20th International Conference on Intelligent User Interfaces, pp. 389–399. ACM (2015)

    Google Scholar 

  52. Treder, M.S., Schmidt, N.M., Blankertz, B.: Gaze independent brain computer interfaces based on covert attention and feature attention. J. Neural Eng. 8(6) (2011)

    Google Scholar 

  53. Schmidt, N.M., Blankertz, B., Treder, M.S.: Online detection of error-related potentials boosts the performance of mental typewriters. BMC Neurosci. 13(1), 19 (2012)

    Article  Google Scholar 

  54. Pomplun, M., Sunkara, S.: Pupil dilation as an indicator of cognitive workload in human-computer interaction. In: Proceedings of HCI International 2003, vol. 3, pp. 542–546. Erlbaum, Mahwah (2003)

    Google Scholar 

  55. Iqbal, S.T., Zheng, X.S., Bailey, B.P.: Task-evoked pupillary response to mental workload in human-computer interaction. In: CHI 2004 Extended Abstracts on Human Factors in Computing Systems, pp. 1477–1480. ACM, April 2004

    Google Scholar 

  56. Chen, S., Epps, J.: Using task-induced pupil diameter and blink rate to infer cognitive load. Hum-Comput. Int. 29(4), 390–413 (2014)

    Article  Google Scholar 

  57. Oliveira, F.T., Aula, A., Russell, D.M.: Discriminating the relevance of web search results with measures of pupil size. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 2209–2212. ACM (2009)

    Google Scholar 

  58. Laeng, B., Sirois, S., Gredebck, G.: Pupillometry a window to the preconscious? Perspect. Psychol. Sci. 7(1), 18–27 (2012)

    Article  Google Scholar 

  59. Lessiter, J., Freeman, J., Miotto, A., Ferrari, E.: Ghosts in the machines: towards a taxonomy of human computer interaction. In: Jacucci, G., Gamberini, L., Freeman, J., Spagnolli, A. (eds.) Symbiotic 2014. LNCS, vol. 8820, pp. 21–31. Springer, Heidelberg (2014)

    Google Scholar 

  60. Spagnolli, A., Guardigl, E., Orso, V., Varotto, A., Gamberini, L.: Measuring user acceptance of wearable symbiotic devices: validation study across application scenarios. In: Jacucci, G., Gamberini, L., Freeman, J., Spagnolli, A. (eds.) Symbiotic 2014. LNCS, vol. 8820, pp. 87–98. Springer, Heidelberg (2014)

    Google Scholar 

  61. Buenaflor, C.B., Kim, H.C.K.: Six Human Factors to Acceptability of Wearable Computers. IJMUE 8, 103–114 (2013)

    Google Scholar 

  62. Barfield, W., Mann, S., Baird, K., Gemperle, F., Kasabach, C., Stivoric, J., Bauer, M., Martin, R., Cho, G.: Computational clothing and accessories. In: Fundamentals of Wearable Computers and Augmented Reality, pp. 471–509. Erlbaum, Mahwah (2001)

    Google Scholar 

  63. Duval, S., Hoareau, C., Hashizume, H.: 7 humanistic needs as seeds. In: Wong, W.K., Guo, Z.X. (eds.) Smart Clothing, pp. 153–187. Taylor and Francis Group, Boca Raton (2014)

    Google Scholar 

  64. Baber, C.: Wearable computers: a human factors review. Int. J. Hum-Comput. Int. 13(2), 123–145 (2001)

    Article  Google Scholar 

  65. Rubin, J., Chisnell, D.: Handbook of usability testing: how to plan, design and conduct effective tests. John Wiley and Sons, New York (2008)

    Google Scholar 

  66. Gamberini, L., Spagnolli, A., Prontu, L., Furlan, S., Martino, F., Solaz, B.R., Alcañiz, M., Lozano, J.A.: How natural is a natural interface? An evaluation procedure based on action breakdowns. Pers. Ubiquit. Comput. 17(1), 69–79 (2013)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of General Psychology, University of Padua, Padua, Italy

    Luciano Gamberini, Anna Spagnolli, Luca Chech, Paolo Negri & Valeria Orso

  2. Neurotechnology Group, Technische Universität Berlin, Berlin, Germany

    Benjamin Blankertz, Laura Acqualagna & Markus Wenzel

  3. Helsinki Institute for Information Technology (HIIT), Aalto University, Espoo, Finland

    Samuel Kaski, Manuel Eugster & Giulio Jacucci

  4. Department of Psychology, I2 Media Research, Goldsmiths University of London, London, UK

    Jonathan Freeman & Eva Ferrari

  5. Human Inspired Technology Research Centre (HIT), University of Padua, Padua, Italy

    Luciano Gamberini, Anna Spagnolli, Maura Bellio, Patrik Pluchino & Filippo Minelle

  6. Department of Computer Science, Helsinki Institute for Information Technology (HIIT), University of Helsinki, Helsinki, Finland

    Oswald Barral, Bariş Serim & Giulio Jacucci

Authors
  1. Luciano Gamberini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anna Spagnolli
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Benjamin Blankertz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Samuel Kaski
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jonathan Freeman
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Laura Acqualagna
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Oswald Barral
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Maura Bellio
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Luca Chech
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Manuel Eugster
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Eva Ferrari
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Paolo Negri
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Valeria Orso
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Patrik Pluchino
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Filippo Minelle
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Bariş Serim
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Markus Wenzel
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Giulio Jacucci
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Giulio Jacucci .

Editor information

Editors and Affiliations

  1. Sekr. MAR 4-3, Rm 4.041, TU Berlin, Berlin, Berlin

    Benjamin Blankertz

  2. Dept of Computer Science, University of Helsinki, Helsinki

    Giulio Jacucci

  3. Università di Padova, Padua

    Luciano Gamberini

  4. Università di Padova, Padua

    Anna Spagnolli

  5. Department of Psychology, Goldsmiths University, London

    Jonathan Freeman

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Gamberini, L. et al. (2015). Developing a Symbiotic System for Scientific Information Seeking: The MindSee Project. In: Blankertz, B., Jacucci, G., Gamberini, L., Spagnolli, A., Freeman, J. (eds) Symbiotic Interaction. Symbiotic 2015. Lecture Notes in Computer Science(), vol 9359. Springer, Cham. https://doi.org/10.1007/978-3-319-24917-9_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-24917-9_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-24916-2

  • Online ISBN: 978-3-319-24917-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation