Abstract
We describe here the conceptual design ofCicero, an application-independent human-computer interaction manager that performs run-time media coordination and allocation, so as to adapt dynamically to the presentation context; knows what it is presenting, so as to maintain coherent extended human-machine dialogues; and is plug-in compatible with host information resources such as “briefing associate” workstations, expert systems, databases, etc., as well as with multiple media such as natural language, graphics, etc. The system design calls for two linked reactive planners that coordinate the actions of the system's media and information sources. To enable presentational flexibility, the capabilities of each medium and the nature of the contents of each information source are semantically modeled as Virtual Devices — abstract descriptions of device I/O capabilities — and abstract information types respectively in a single uniform knowledge representation framework. These models facilitate extensibility by supporting the specification of new interaction behaviors and the inclusion of new media and information sources.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Agre, P. E. & Chapman, D. (1987). Pengi: An Implementation of a Theory of Activity.Proceedings of The 6th AAAI, 196–201. Seattle, WA.
Arens, Y. & Hovy, E. (1990). How to Describe What? Towards a Theory of Modality Utilization.Proceedings of the 12th Annual Conference of the Cognitive Science Society, 487–494. Cambridge, MA.
Arens, Y., Miller, L., Shapiro, S. C. & Sondheimer, N. K. (1988). Automatic Construction of User-Interface Displays.Proceedings of the 7th AAAI Conference, 808–813. St. Paul, MN.
Arens, Y., Miller, L., & Sondheimer, N. K. (1991). Presentation Design Using an Integrated Knowledge Base. In Sullivan, Joseph, W. and Tyler, Sherman W. (eds.)Intelligent User Interfaces, 241–258. Addison-Wesley: Reading.
Arens, Y., Hovy, E. H. & Trimble, J. (1992). An Automatic Presentation Builder. Unpublished ms, USC Information Sciences Institute.
Arens, Y., Hovy, E. H. & Vossers, M. (1993). On the Knowledge Underlying Multimedia Presentations. In Mark Maybury (ed.)Intelligent Multimedia Interfaces, 280–306. AAAI Press.
Arens, Y., Hovy, E. H. & Van Mulken, S. (1993). Structure and Rules in Automated Multimedia Presentations. In Mark Maybury (ed.)Intelligent Multimedia Interfaces, 280–306. AAAI Press.
Arens, Y., Chee, C. Y., Hsu, C-N. & Knoblock, C. A. (1993). Retrieving and Integrating Data from Multiple Information Sources.International Journal of Intelligent and Cooperative Information Systems 2(2): 127–158.
Burger, J. & Marshall, R. (1991). AIMI: An Intelligent Multimedia Interface.Proceedings of The 9th AAAI Conference, 23–28. Anaheim, CA.
Erman, L. D., Lark, J. S. & Hayes-Roth, F. (1987). ABE: An Environment for Engineering Intelligent Systems. Technical Report TTR-ISE-87-106, Teknowledge Inc., Palo Alto, CA.
Feiner, S. & McKeown, K. R. (1990). Coordinating Text and Graphics in Explanation Generation.Proceedings of the 8th AAAI, 442–449.
Feiner, S. (1991). An Architecture for Knowledge-Based Graphical Interfaces. ACM/SIGCHI Workshop onArchitectures for Intelligent Interfaces: Elements and Prototypes. In Sullivan, Joseph W. & Tyler, Sherman W. (eds.)Intelligent User Interfaces, 259–279. Addison-Wesley: Reading.
Foley, J., Kim, W., Kovacevic, S. & Murray, K. (1991). UIDE — An Intelligent User Interface Design Environment. In Sullivan, J. & Tyler, S. (eds.)Archietectures for Intelligent User Interfaces: Elements and Prototypes, 339–384. Addison-Wesley: Reading.
Hovy, E. H. (1988). Planning Coherent Multisentential Text.Proceedings of The 26th Annual Meeting of the Association for Computational Linguistics, 163–169. Buffalo, N.Y.
Hovy, E. H. & Arens, Y. (1990). When is a Picture Worth a Thousand Words? — Allocation of Modalities in Multimedia Communication. Presented at theAAAI Symposium on Human-Computer Interaction, Stanford University.
Hovy, E. H. & Arens, Y. (1991). Automatic Generation of Formatted Text.Proceedings of The 10th AAAI, 92–97. Anaheim, CA.
Hovy, E. H. & Arens, Y. (1993). The Planning Paradigm Required for Automated Multimedia Presentation Planning. InHuman-Computer Collaboration: Reconciling Theory, Synthesizing Practice. Papers from the 1993 Fall Symposium Series, AAAI Technical Report FS-93-05.
Hovy, E. H., Lavid, J., Maier, E., Mittal, V. & Paris, C. L. (1992). Employing Knowledge Resources in a New Text Planning Architecture. In Dale, R., Hovy, E., Rösner, D. & Stock, O. (eds.)Aspects of Automated Natural Language Generation, 57–72. Heidelberg: Springer Verlag Lecture Notes in AI number 587.
Intelligent User Interfaces. Arens Y., Feiner, S., Foley, J., Hovy, E., John, B., Neches, R., Pausch, R., Schorr, H. & Swartout, W. (1991). USC/ISI Research Report No. ISI/RR-91-288, September 30, 1991.
Luo, P., Szekely, P. & Neches, R. (1993). Management of Interface Design in Humanoid.Proceedings of INTERCHI'93, 107–114. Amsterdam, The Netherlands.
MacGregor, R. (1988). A Deductive Pattern Matcher.Proceedings of AAAI-88, 403–408. St. Paul, MN.
Mackinlay, J. (1986).Automatic Design of Graphical Presentations. Ph.D. dissertation, Stanford University.
Mackinlay, J. (1986). Automating the Design of Graphical Presentation of Relational Information.ACM Transactions on Graphics Special Issue on User Interface Software Part I.5(2): 110–141.
Mann, W. C. & Thompson, S. A. (1988). Rhetorical Structure Theory: Toward a Functional Theory of Text Organization.Text 8(3): 243–281.
Neal, J. G., Shapiro, S. C., Thielman, J. R., Lammens, J. M., Funke, D. J., Byoun, J. S., Paul, R., Dobes, Z., Glanowski, S. & Summers, M. S. (1990). Intelligent Multi-Media Integrated Interface Project. SUNY Buffalo and RADC Technical Report TR-90-128.
Neches, R., Foley, J., Szekely, P., Sukaviriya, P., Luo, P., Kovacevic, S. & Hudson, S. (1993). Knowledgeable Development Environments Using Shared Design Models.Proceedings of the International Workshop on Intelligent User Interfaces, 183–190. Orlando, FL.
Olsen, D. (1986). MIKE: The Menu Interaction Kontrol Environment.ACM Transactions on Graphics.17(3): 43–50.
The Penman Project. (1988).The Penman Primer, User Guide, and Reference Manual. Unpublished USC/ISI documentation.
Schank, R. & Abelson, R. (1977).Scripts, Plans, Goals and Understanding: An Inquiry into Human Knowledge Structures. Hillsdale: Lawrence Erlbaum Associates.
Szekely, P., Luo, P. & Neches, R. (1993). Beyond Interface Builders: Model-Based Interface Design.Proceedings of INTERCHI'93, 383–390. Amsterdam, the Netherlands.
Vossers, M. (1991).Automatic Generation of Formatted Text and Line Drawings. Master's thesis, University of Nijmegen, The Netherlands.
Wahlster, W., André, E., Bandyopadhyay, S., Graf, W. & Rist, T. (1991). WIP: The Coordinated Generation of Multimodal Presentations from a Common Representation. In Ortony, A., Slack, J. & Stock, O. (eds.),Computational Theories of Communication and their Applications, 190–213. Springer Verlag, Berlin.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Arens, Y., Hovy, E. The design of a model-based multimedia interaction manager. Artif Intell Rev 9, 167–188 (1995). https://doi.org/10.1007/BF00849178
Issue Date:
DOI: https://doi.org/10.1007/BF00849178