Skip to main content
SpringerLink
Log in

Control Problems in Multi-Modal Telepresence Systems

  • Conference paper
Advances in Control

Summary

In this chapter we discuss challenging control problems in multi-modal telepresence and teleaction systems. The typical system structure consists of a human operator, the possibly distant (and differently scaled) teleoperator and various types of barriers between them. Fundamental control modes to organize and distribute control between human and system are discussed. Potential applications and enabling technologies are sampled. A subjective listing of central control issues is given including multi-modal control, impact of time-delays, local control measures, real world, virtual environment (VE) and augmented environment interaction, sensory substitution, etc. An experimental prototype system for multi-modal telepresence and teleaction developed in our laboratory is presented and some experiments further illustrating typical control challenges are highlighted.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  • Anderson, R. (1996). Autonomous, Teleoperated, and Shared Control of Robot Systems. In Proceedings of the IEEE International Conference on Robotics and Automation, Minneapolis, Minnesota, pp. 2025–2032.

    Google Scholar 

  • Anderson, R. and M. Spong (1989). Bilateral Control of Teleoperators with Time Delay. IEEE Transaction on Automatic Control 34, 494–501.

    Article  MathSciNet  Google Scholar 

  • Arimoto, S. (1991). Learning for Skill Refinement. In Proceedings of the International Workshop on Intelligent Robots and Systems IROS, pp. 10–13.

    Chapter  Google Scholar 

  • Baier, H., M. Buss, F. Freyberger, J. Hoogen, P. Kammermeier, and G. Schmidt (1999). Distributed PC-Based Haptic, Visual and Acoustic Telepresence System—Experiments in Virtual and Remote Environments. In Proceedings of the IEEE Virtual Reality Conference VR’99, Houston, TX, pp. 118–125.

    Google Scholar 

  • Baier, H., M. Buss, and G. Schmidt (1997). Control Mode Switching for Teledrilling Based on a Hybrid System Model. In Proceedings of the IEEE/ASME International Conference on Advanced Intelligent Mechatronics AIM’97, Tokyo, Japan, Paper No. 50.

    Google Scholar 

  • Baier, H. and G. Schmidt (1998). Control Aspects of Closed-Loop Haptic Tele-operation. In Proceedings of the 7th International Workshop on Robotics in Alpe-Adria-Danube Region (RAAD’98), Smolenice, Slovakia, pp. 151–156.

    Google Scholar 

  • Barfield, W., C. Hendrix, O. Bjorneseth, K. Kaczmarek, and W. Lotens (1995, Fall). Comparison of Human Sensory Capabilities with Technical Specifications of Virtual Environment Equipment. Presence 4(4), 329–356.

    Google Scholar 

  • Basdogan, C., C. Ho, and M. Srinivasan (1997). Haptic Rendering: Point- and Ray-Based Interactions. In Proceedings of the Second PHANToM Users Group Workshop, Dedham, MA.

    Google Scholar 

  • Bejczy, A., W. Kim, and S. Venema (1990). The phantom robot: Predictive displays for teleoperation with time delay. In Proceedings of the IEEE International Conference on Robotics and Automation, Cincinnati, Ohio, pp. 546–551.

    Chapter  Google Scholar 

  • Brooks, T. (1990). Telerobotic response requirements. In Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, pp. 113–120.

    Chapter  Google Scholar 

  • Burdea, G. (1996). Force and Touch Feedback for Virtual Reality. New York: John Wiley & Sons.

    Google Scholar 

  • Burdea, G. and J. Zhuang (1991). Dexterous telerobotics with force feedback — an overview, part 1: Human factors, part 2: Control and implementation. Robotica 9, 171–178.

    Article  Google Scholar 

  • Burdea, G. and J. Zhuang (1991). Dexterous telerobotics with force feedback — an overview, part 1: Human factors, part 2: Control and implementation. Robotica 9, 291–298.

    Article  Google Scholar 

  • Buss, M. (1994, June). Study on Intelligent Cooperative Manipulation. Ph. D. thesis, University of Tokyo, Tokyo.

    Google Scholar 

  • Buss, M. and H. Hashimoto (1993a). Human manipulative skill analysis and acquisition for the intelligent assisting system. In Proceedings 12th IFAC World Congress, International Federation of Automatic Control, Volume 9, Sydney, Australia, pp. 321–326.

    Google Scholar 

  • Buss, M. and H. Hashimoto (1993b). Skill acquisition and transfer system as approach to the intelligent assisting system—ias. In Proceedings of the 2nd IEEE Conference on Control Applications, Vancouver, British Columbia, Canada, pp. 451–456.

    Chapter  Google Scholar 

  • Buss, M. and H. Hashimoto (1994). Manipulation skill modeling for dexterous hands. In Proceedings of the IEEE International Conference on Robotics and Automation, San Diego, California, pp. 3403–3408.

    Google Scholar 

  • Buss, M. and H. Hashimoto (1995). Motion scheme for dextrous manipulation in the intelligent cooperative manipulation system—icms. In V. Graefe (Ed.), Intelligent Robots and Systems, pp. 279–294. Amsterdam: Elsevier Science.

    Chapter  Google Scholar 

  • Caldwell, D., S. Lawther, and A. Wardle (1996). Tactile Perception and its Application to the Design of Multi-modal Cutaneous Feedback Systems. In Proceedings of the IEEE International Conference on Robotics and Automation, Minneapolis, Minnesota, pp. 3215–3221.

    Google Scholar 

  • Card, S., T. Moran, and A. Newell (1983). The Psychology of Human-Computer Interaction. Lawrence Erlbaum Associates.

    Google Scholar 

  • Colgate, J., P. Grafing, M. Stanley, and G. Schenkel (1993). Implementation of stiff virtual walls in force-reflecting interfaces. In Proceedings of the IEEE Virtual Reality Annual International Symposium VRAIS, pp. 20–26.

    Google Scholar 

  • Delson, N. and H. West (1993). Robot programming by human demonstration: Subtask compliance controller identification. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems IROS, pp. 33–41.

    Chapter  Google Scholar 

  • Denk, J. (1997). Teleoperation of the ROMAN-Gripper over the Internet. Internal Report, Institute of Automatic Control Engineering, Technical University of Munich.

    Google Scholar 

  • Deussen, O. and Kuhn T (1995). Echtzeitsimulation deformierbarer Objekte über nodale Modelle. In Proc. Integration von Bild, Modell und Text, ASIM Mitteilungen No. 46, Universität Magdeburg.

    Google Scholar 

  • Doel, K. and D. Pai (1998, August). The Sounds of Physical Shapes. Presence 7(4), 382–395.

    Article  Google Scholar 

  • Fischer, C., M. Buss, and G. Schmidt (1996). HuMan-Robot-Interface for Intelligent Service Robot Assistance. In Proceedings of the IEEE International Workshop on Robot and Human Communication (ROMAN), Tsukuba, Japan, pp. 177–182.

    Google Scholar 

  • Friedrich, H., J. Holle, and R. Dillmann (1998). Interactive Generation of Flexible Robot Programs. In Proceedings of the IEEE International Conference on Robotics and Automation, Leuven, Belgium, pp. 538–543.

    Google Scholar 

  • Hall, A., S. Kolssouzidis, and T. Schickinger (1998, June). Modeling of Deformable Objects in Virtual Worlds. Internal Report, Institute of Automatic Control Engineering, Technical University of Munich.

    Google Scholar 

  • Handwerker, H. (1989). General Sensory Physiology. In R. Schmidt and G. Thews (Eds.), Human Physiology, pp. 176–195. Berlin Heidelberg New York London Paris Tokyo Hong Kong: Springer-Verlag.

    Google Scholar 

  • Hannaford, B. (1989, August). A design framework for teleoperators with kinesthetic feedback. IEEE Transactions on Robotics and Automation 5(4), 426–434.

    Article  Google Scholar 

  • Hendrix, C. and W. Barfield (1996, Summer). The Sense of Presence within Auditory Virtual Environments. Presence 5(3), 290–301.

    Google Scholar 

  • Hirzinger, G. (1993). Rotex-the first robot in space. In Proceedings of the ICAR International Conference on Advanced Robotics, pp. 9–33.

    Google Scholar 

  • Hollander, A. (1994). An exploration of virtual auditory shape perception. Master’s thesis, Human Interface Technology Laboratory, University of Washington, Seattle, WA.

    Google Scholar 

  • Hoogen, J. (1998). Development of an Experimental System for Kinesthetic Feedback in Telepresence Systems. Internal Report, Institute of Automatic Control Engineering, Technical University of Munich.

    Google Scholar 

  • Johannsen, G. (1993). Mensch-Maschine-Systeme. Springer.

    Google Scholar 

  • Kaczmarek, K., M. Tyler, and P. Bach-y-Rita (1994). Electrotactile Haptic Displays on the Fingertips. In Proceedings of the IEEE International Conference on Engineering in Medicine and Biology, New York.

    Google Scholar 

  • Kammermeier, P. (1997). Development of a Haptic Display for Tactile Feedback in Telepresence Systems. Internal Report, Institute of Automatic Control Engineering, Technical University of Munich.

    Google Scholar 

  • Kazerooni, H., T.-I. Tasy, and K. Hollerbach (1993). A Controller Design Framework for Telerobotic Systems. IEEE Transaction Control Systems Technology 11, 105–116.

    Google Scholar 

  • Kim, W. (1996, Spring). Virtual Reality Calibration and Preview/Predictive Displays for Telerobotics. Presence 5(2), 173–190.

    Google Scholar 

  • Kontarinis, D. and R. Howe (1995, Fall). Tactile Display of Vibratory Information in Teleoperation and Virtual Environments. Presence 4(4), 387–402.

    Google Scholar 

  • Kosuge, K., Y. Fujisawa, and T. Fukuda (1993). Control of robot directly maneuvered by operator. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems IROS, pp. 49–54.

    Chapter  Google Scholar 

  • Kosuge, K., T. Itoh, T. Fukuda, and M. Otsuka (1995). Scaled Telemanipulation System Using Semi-Autonomous Task-Oriented Virtual Tool. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems IROS, Pittsburgh, PA, pp. 124–129.

    Google Scholar 

  • Kosuge, K., H. Murayama, and K. Takeo (1996). Bilateral Feedback Control of Telemanipulators via Computer Network. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems IROS, Osaka, Japan, pp. 1380–1385.

    Chapter  Google Scholar 

  • Kunii, Y., M. Buss, and H. Hashimoto (1995). Force feedback using dynamic force simulator. In Video Proceedings of the IEEE International Conference on Robotics and Automation, Nagoya, Japan, pp. 15.

    Google Scholar 

  • Kunii, Y. and H. Hashimoto (1996). Dynamic force simulator for haptic interface. In J. Gertler, J. Cruz, and M. Peshkin (Eds.), Preprints of the 13th World Congress, Volume L, pp. 403–408. San Francisco: International Federation of Automatic Control—IFAC.

    Google Scholar 

  • Kunii, Y. and H. Hashimoto (1997). Tele-teaching by human demonstration in virtual environment for robotic network system. In Proceedings of the IEEE International Conference on Robotics and Automation, Albuquerque, New Mexiko, pp. 405–410.

    Google Scholar 

  • Lawrence, D. (1993, October). Stability and Transparency in Bilateral Teleoperation. IEEE Transactions on Robotics and Automation 9(5), 624–637.

    Article  MathSciNet  Google Scholar 

  • Leung, G., B. Francis, and A. Apkarian (1993). Bilateral controller for teleoperators with time delay via μ-synthesis. IEEE Transactions on Robotics and Automation 10, 105–116.

    Google Scholar 

  • Mann, S. (1998, November). Humanistic Computing: “WearComp” as a New Framework and Application for Intelligent Signal Processing. Proceedings of the IEEE 86(11), 2123–2151.

    Article  Google Scholar 

  • Massie, T. and J. Salisbury (1994). The PHANToM Haptic Interface: A Device for Probing Virtual Objects. In Proc. of the ASME Winter Annual Meeting, Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, Chicago.

    Google Scholar 

  • Massimino, M. and T. Sheridan (1994). Sensory Substitution for Force Feedback in Teleoperation. Presence 2(4).

    Google Scholar 

    Google Scholar 

  • Mitsuishi, M., T. Hori, and T. Nagao (1994). Predictive Information Display for Tele-Handling/Machining System. In Proceedings of the IEEE/RSJ/GI International Conference on Intelligent Robots and Systems IROS, Munich, Germany, pp. 260–267.

    Chapter  Google Scholar 

  • Nahvi, A., D. Nelson, J. Hollerbach, and D. Johnson (1998). Haptic Manipulation of Virtual Mechanisms from Mechanical CAD Designs. In Proceedings of the IEEE International Conference on Robotics and Automation, Leuven, Belgium, pp. 375–380.

    Google Scholar 

  • Niemeyer, G. and J. E. Slotine (1998). Towards Force-Reflecting Teleoperation Over the Internet. In Proceedings of the IEEE International Conference on Robotics and Automation, Leuven, Belgium, pp. 1909–1915.

    Google Scholar 

  • Rest, E. L., J.-L. Fleureau, and L. Marcé (1997). Semantics and implementation of a language for telerobotics. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems IROS, Grenoble, France, pp. 1807–1812.

    Google Scholar 

  • Rosenberg, L. and B. Adelstein (1993). Perceptual decomposition of virtual haptic surfaces. In Proceedings of the IEEE Symposium on Research Frontiers in Virtual Reality, San Jose, CA, pp. 46–53.

    Chapter  Google Scholar 

  • Salcudean, S. (1998). Control for Teleoperation and Haptic Interfaces. In B. Siciliano and K. Valavanis (Eds.), Lecture Notes in Control and Information Sciences 230: Control Problems in Robotics and Automation, pp. 51–66. London: Springer.

    Google Scholar 

  • Salcudean, S. and T. Vlaar (1994). On the emulation of stiff walls and static friction with a magnetically levitated input/output device. In Proceedings of ASME WAM, New York, pp. 303–309.

    Google Scholar 

  • Satava, R. and S. Jones (1997). Virtual environments for medical training and education. Presence 6, 139–146.

    Google Scholar 

  • Schloerb, D. (1995, Winter). A Quantitative Measure of Telepresence. Presence 4(1), 64–80.

    Google Scholar 

  • Sheridan, T. (1992). Telerobotics, Automation, and Human Supervisory Control. Cambridge, Massachusetts: MIT Press.

    Google Scholar 

  • Sheridan, T. (1996, Spring). Further Musings on the Psychophysics of Presence. Presence 5(2), 241–246.

    Google Scholar 

  • Shimoga, K. (1993a). A survey of perceptual feedback issues in dexterous telemanipulation: Part i. In Proceedings of the IEEE Virtual Reality Annual International Symposium VRAIS, pp. 263–270.

    Chapter  Google Scholar 

  • Shimoga, K. (1993b). A survey of perceptual feedback issues in dexterous telemanipulation: Part ii. In Proceedings of the IEEE Virtual Reality Annual International Symposium VRAIS, pp. 271–279.

    Chapter  Google Scholar 

  • Sitti, M. and H. Hashimoto (1998). Tele-Nanorobotics using Atomic Force Microscope. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems IROS, Vancouver, Canada, pp. 1739–1746.

    Google Scholar 

  • Stein, M. and R. Paul (1994). Operator interaction, for time-delayed teleoperation, with a behavior-based controller. In Proceedings of the IEEE International Conference on Robotics and Automation, San Diego, California, pp. 231–236.

    Google Scholar 

  • Thompson, T., D. Johnson, and E. Cohen (1997). Direct haptic rendering of sculptured models. In Proceedings Symposium on Interactive 3D Graphics, Providence, pp. 167–176.

    Chapter  Google Scholar 

  • Thompson, T., D. Nelson, E. Cohen, and J. Hollerbach (1997). Maneuverable NURBS models within a haptic virtual environment. In Proceedings ASME Dynamic Systems and Control Division, Dallas, pp. 37–44.

    Google Scholar 

  • Welch, R., T. Blackmon, A. Liu, B. Meilers, and L. Stark (1996, Summer). The Effects of Pictorial Realism, Delay of Visual Feedback, and Observer Interactivity on the Subjective Sense of Presence. Presence 5(3), 263–273.

    Google Scholar 

  • Wrenzel, E., F. Wightman, D. Kistler, and S. Foster (1988). The Convolvotron: Realtime synthesis of out-of-head localization. In Second Joint Meeting Acoustic Societies of America and Japan, Honolulu, HI.

    Google Scholar 

  • Wermuth, S. (1998, Mai). Quantitative Investigations of Multi-Modal Telepresence Quality in Visual-Haptic Telepresence Systems. Internal Report, Institute of Automatic Control Engineering, Technical University of Munich.

    Google Scholar 

  • Yan, J. and S. Salcudean (1993). Teleoperation controller design using h ∞ optimization. IEEE Transaction Control Systems Technology 4, 244–258.

    Article  Google Scholar 

  • Yang, B. and H. Asada (1990). Skill acquisition for robot grinding: Learning of high-level feedback laws from teaching data. In Proceedings of the 1990 JAPAN- U.S.A Symposium on Flexible Automation, pp. 649–655.

    Google Scholar 

  • Yang, J., Y. Xu, and C. Chen (1993). Hidden markov model approach to skill learning and its application in telerobotics. In Proceedings of the IEEE International Conference on Robotics and Automation, Atlanta, Georgia, pp. 396–402.

    Google Scholar 

  • Zelek, J. (1997). Human-robot interaction with a minimal spanning natural language template for autonomous & tele-operated control. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems IROS, Grenoble, France, pp. 299–305.

    Google Scholar 

  • Zilles, C. and J. Salisbury (1995). A Constraint-based God-object Method For Haptic Display. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems IROS.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Automatic Control Engineering, Technische Universität München, D-80290, München, Germany

    M. Buss & G. Schmidt

Authors
  1. M. Buss
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. G. Schmidt
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. GH Duisburg, FB9/MRT, Gerhard-Mercator Universität, Bismarckstr. 81, D-47048, Duisburg, Germany

    Paul M. Frank

Rights and permissions

Reprints and permissions

Copyright information

© 1999 Springer-Verlag London Limited

About this paper

Cite this paper

Buss, M., Schmidt, G. (1999). Control Problems in Multi-Modal Telepresence Systems. In: Frank, P.M. (eds) Advances in Control. Springer, London. https://doi.org/10.1007/978-1-4471-0853-5_3

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-0853-5_3

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-1216-7

  • Online ISBN: 978-1-4471-0853-5

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation