Abstract
One goal of research in the area of human–machine interaction is to improve the ways a human user interacts with a computer through a multimedia interface. This interaction comprises of not only text, graphical animation, stereo sounds, and live video images, but also force and haptic feedback, which can provide more “real” feeling to the user. The force feedback joystick, a human interface device is an input–output device. It not only tracks user's physical manipulation input, but also provides realistic physical sensations of force coordinated with system output. As part of our research, we have developed a multimedia computer game that assimilates images, sounds, and force feedback. We focused on the issues of how to combine these media to allow the user feel the compliance, damping, and vibration effects through the force feedback joystick. We conducted series of human subject experiments that incorporated different combinations of media, including the comparative study of the different performances of 60 human users, aiming to answer the question: What are the effects of force feedback (and associated time delays) when used in combination with visual and auditory information as part of a multi-modal interface? It is hoped that these results can be utilized in the design of enhanced multimedia systems that incorporate force feedback.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Barnett: 1999, http://www.barnett.sk/software/bbook/directx.
Bergamasco, M.: 1997, Haptic interfaces: the study of force and tactile feedback systems, in: Proc. of IEEE Internat. Workshop on Robot and Human Communication.
Browsebooks: 1999, http://www.browsebooks.com/Kovach.
Buttolo, P., Oboe, R., Hannaford, B., and McNeely, B.: 1996, Force feedback in shared virtual simulations, Proceedings MICAD, Paris.
Chor, H. R., Choi, B. H., and Ryew, S. M.: 1997, Haptic display in the virtual collaborative workspace shared by multiple users, in: Proc. of IEEE Internat. Workshop on Robot and Human Communication.
Clark, J.: 1998, May the force feedback be with you: grappling with DirectX and DirectInput, Microsoft Systems Journal.
DirectX: 1999, http://www.directxfaq.com/general.htm.
Force feedback: 1998, http://www.force-feedback.com.
Funkhouser T. A.: 1996, Network topologies for scalable multi-user virtual environments, in: IEEE VRAIS'96, San Jose, CA.
Geocities: 1998, http://www.geocities.com/SiliconValley/Way/3390.
Hannaford, B., Wood, L., McAffee, D., and Zak, H.: 1991, Performance evaluation of a six axis generalized force reflection teleoperator, IEEE Trans. Systems Man Cybernet. 21, 620-633.
He, F.: 1999, Enhanced multimedia human-computer interaction using force feedback, M.S. Thesis, Department of Electrical Engineering and Computer Science, The University of Kansas.
He, F. and Agah, A.: 2000, Effectiveness of incorporating force feedback into multimedia interfaces, in: Proc. of The Second Internat. Forum on Multimedia & Image Processing (IFMIP'00), World Automation Congress (WAC'00), Maui, Hawaii.
Henmi, H. and Yoshikawa, T.: 1997, Virtual lesson and its application to virtual calligraphy system, in: Proc. of IEEE Internat. Workshop on Robot and Human Communication.
Immersion: 1997, A force feedback programming primer, Immersion Corporation.
Ishii, M., Nakata, M., and Sato, M.: 1994, Networked APIDAR: A networked virtual environment with virtual, auditory, and haptic interactions, Presence 3, 351-359.
Iwata, H.: 1999, Haptic communication through Internet, Institute of Engineering Mechanism, University of Tsukuba.
Kavach, P. J.: 1997, The Awesome Power of Direct3D/DirectX, Manning Publications Co.
Lamothe, A.: 1997, Windows Game Programming for Dummies, IDG Books.
Microsoft: 1999, http://www.microsoft.com.
Minsky, M., Ouh-Young, M., Steels, O., Brooks, F. P., and Behensky, M.: 1990, Feeling and seeing: issues in force display, ACM 1990, 235-243.
Miyasato, T. and Nakatsu, R.: 1997, Allowable delay between images and tactile information in a haptic interface, in: Proc. of IEEE Internat. Workshop on Robot and Human Communication.
Ouchi, K. and Hashimoto, S.: 1997, Handshake telephone system to communicate with voice and force, in: Proc. of IEEE Internat. Workshop on Robot and Human Communication.
SigChi: 1999, http://www.acm.org/sigchi/cdg/cdg.html.
Tosenberg, L. B. and Adelstein, B. D.: 1993, Perceptual decomposition of virtual haptic surfaces, in: Proc. of IEEE Symposium on Research Frontiers in Virtual Reality, San Jose, CA, pp. 46-53.
UC Davis: 1999, http://trc.ucdavis.edu/trcsum1/binder.htm#Multimedia.
feedback.html.
Wksoftware: 1999, http://www.wksoftware.com.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
He, F., Agah, A. Multi-Modal Human Interactions with an Intelligent Interface Utilizing Images, Sounds, and Force Feedback. Journal of Intelligent and Robotic Systems 32, 171–190 (2001). https://doi.org/10.1023/A:1013969524676
Issue Date:
DOI: https://doi.org/10.1023/A:1013969524676