Skip to main content
SpringerLink
Log in

Determining the value of information for collaborative multi-agent planning

  • Published:
Autonomous Agents and Multi-Agent Systems Aims and scope Submit manuscript

Abstract

This paper addresses the problem of computing the value of information in settings in which the people using an autonomous-agent system have access to information not directly available to the system itself. To know whether to interrupt a user for this information, the agent needs to determine its value. The fact that the agent typically does not know the exact information the user has and so must evaluate several alternative possibilities significantly increases the complexity of the value-of-information calculation. The paper addresses this problem as it arises in multi-agent task planning and scheduling with architectures in which information about the task schedule resides in a separate “scheduler” module. For such systems, calculating the value to overall agent performance of potential new information requires that the system component that interacts with the user query the scheduler. The cost of this querying and inter-module communication itself substantially affects system performance and must be taken into account. The paper provides a decision-theoretic algorithm for determining the value of information the system might acquire, query-reduction methods that decrease the number of queries the algorithm makes to the scheduler, and methods for ordering the queries to enable faster decision-making. These methods were evaluated in the context of a collaborative interface for an automated scheduling agent. Experimental results demonstrate the significant decrease achieved by using the query-reduction methods in the number of queries needed for reasoning about the value of information. They also show the ordering methods substantially increase the rate of value accumulation, enabling faster determination of whether to interrupt the user.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Abbreviations

T :

A scheduling problem—consists of a set of tasks applicable to the problem domain, relationships among those tasks, outcome values for each task, quality accumulation methods and an active schedule

M :

A task—the basic scheduling entity with which ASAs work

o :

An outcome of a task—defined by the values it assigns to a set of outcome characteristics (e.g., duration, cost, performance level, resources consumed), each representing a different task performance quality aspect

P(o):

The a priori probability of outcome o

o.dur :

The value of the duration characteristic of outcome o

O :

The set of possible outcomes of a task

t :

The time when the actual outcome of a task can be obtained from the external source

k :

The number of potential outcomes of a task

k′:

The number of distinct duration outcomes of a task (k′ = |D|)

S t (T, I, Sched):

The schedule that the scheduler produces if it receives at time t a scheduling problem T associated with the active schedule Sched and the new information I

S t (T, I, Sched).quality :

The quality of the schedule S t (T, I, Sched)

I :

New information which gives the actual outcome of task M

Sched :

The active schedule

D :

A vector of the possible duration outcomes of a task

F i :

The value of the difference calculated as part of the summation used in Eq. 1 for the jth query pair

Order scanner, Time−critical scanner, Outcome−space scanner :

Methods for calculating the value of obtaining the actual outcome of a task

Duration scanner, Potential−impact scanner :

Methods for efficient value accumulation as part of calculating the value of obtaining the actual outcome of a task

FM, PAV α, AUTC α :

Value accumulation measures

References

  1. Ai-Chang M., Bresina J., Charest L., Chase A., Hsu J., Jónsson A., Kanefsky B., Morris P., Rajan K., Yglesias J., Chafin B., Dias W., Maldague P. (2004) MAPGEN: Mixed-initiative planning and scheduling for the Mars exploration rover mission. IEEE Intelligent Systems 19(1): 8–12

    Article  Google Scholar 

  2. Alexander, G., Raja, A., & Musliner, D. (2008). Controlling deliberation in a markov decision process-based agent. In: Proceedings of the seventh international joint conference on autonomous agents and multiagent systems (AAMAS-08) (pp. 461–468).

  3. Ang, A., Tang, W. (Eds.) (1984) Probability concepts in engineering planning and design, Volume II—Decision, risk, and reliability. Wiley, New York

    Google Scholar 

  4. Atlas, J., & Decker, K. (2010). Coordination for uncertain outcomes using distributed neighbor exchange. In: Proceedings of the ninth international conference on autonomous agents and multiagent systems (AAMAS-10) (pp. 1047–1054).

  5. Barbulescu, L., Rubinstein, Z., Smith, S., & Zimmerman, T. (2010). Distributed coordination of mobile agent teams: the advantage of planning ahead. In: Proceedings of the ninth international conference on autonomous agents and multiagent systems (AAMAS-10) (pp. 1331–1338).

  6. Bilgic M., Getoor L. (2011) Value of information lattice: Exploiting probabilistic independence for effective feature subset acquisition. Journal of Artificial Intelligence Research 41: 69–95

    MATH  Google Scholar 

  7. Chalupsky, H., Gil, Y., Knoblock, C., Lerman, K., Oh, J., Pynadath, D., Russ, T., & Tambe, M. (2001). Electric elves: Applying agent technology to support human organizations. In: Proceedings of the thirteenth conference on innovative applications of artificial intelligence (pp. 51–58).

  8. Clemen, R. (Eds.) (1991) Making hard decisions: An introduction to decision analysis. Duxbury Press, Belmon, CA

    Google Scholar 

  9. Dittmer, S., & Jensen, F. (1997). Myopic value of information in influence diagrams. In: Proceedings of the thirteenth annual conference on uncertainty in artificial intelligence (UAI-97) (pp. 142–149).

  10. Fleming, M., & Cohen, R. (1999). User modeling in the design of interactive interface agents. In: Proceedings of the seventh international conference on user modeling (pp. 67–76).

  11. Fleming, M., & Cohen, R. (2001). A user modeling approach to determining system initiative in mixed-initiative AI systems. In: Proceedings of the Eighth international conference on user modeling (pp. 54–63).

  12. Fleming, M., & Cohen, R. (2004). A decision procedure for autonomous agents to reason about interaction with humans. In: Proceedings of the AAAI spring symposium on interaction between humans and autonomous systems over extended operation (pp. 81–86).

  13. Gallagher, A. (2009). Embracing conflicts: Exploiting inconsistencies in distributed schedules using simple temporal network representations. PhD thesis, Carnegie Mellon University, School of Computer Science, Pittsburgh, PA, USA.

  14. Heckerman, D., Horvitz, E., & Middleton, B. (1991). An approximate nonmyopic computation for value of information. In: Proceedings of the seventh annual conference on uncertainty in artificial intelligence (UAI-91) (pp. 135–141).

  15. Hiatt, L., Zimmerman, T., Smith, S., & Simmons, R. (2009). Strengthening schedules through uncertainty analysis agents. In Proceedings of the twenty-first international joint conference on artificial intelligence (IJCAI-09) (pp. 175–180).

  16. Horvitz E. (2001) Principles and applications of continual computation. Artificial Intelligence 126(1–2): 159–196

    Article  MathSciNet  MATH  Google Scholar 

  17. Horvitz, E., Breese, J., Heckerman, D., Hovel, D., & Rommelse, K. (1998). The Lumière project: Bayesian user modeling for inferring the goals and needs of software users. In: Proceedings of the fourteenth conference on uncertainty in artificial intelligence (UAI-98) (pp. 256–265).

  18. Horvitz E., Breese J., Henrion M. (1988) Decision theory in expert systems and artificial intelligence. International Journal of Approximate Reasoning 2: 247–302

    Article  Google Scholar 

  19. Horvitz, E., Koch, P., Kadie, C., & Jacobs, A. (2002). Coordinate: Probabilistic forecasting of presence and availability. In: Proceedings of the eighteenth conference on uncertainty in artificial intelligence (UAI-02) (pp. 224–233).

  20. Horvitz E., Kadie C., Paek T., Hovel D. (2003) Models of attention in computing and communication: from principles to applications. Communications of the ACM 46(3): 52–59

    Article  Google Scholar 

  21. Howard, R. (1966). Information value theory. IEEE Transactions on Systems Science and Cybernetics, SSC-2, 22–26.

    Google Scholar 

  22. Hui, B., & Boutilier, C. (2006). Who’s asking for help?: A Bayesian approach to intelligent assistance. In: Proceedings of the eleventh international conference on intelligent user interfaces (IUI-06) (pp. 186–193).

  23. Kapoor, A., Horvitz, E., & Basu, S. (2007). Selective supervision: Guiding supervised learning with decision-theoretic active learning. In: Proceedings of the 20th international joint conference on artificial intelligence (IJCAI-07) (pp. 877–882).

  24. Kis, T., Vancza, J., & Markus, A. (1996). Controlling distributed manufacturing systems by a market mechanism. In: Proceedings of the twelfth European conference on artificial intelligence (ECAI-96) (pp. 534–538).

  25. Krause A., Guestrin C. (2009) Optimal value of information in graphical models. Journal of Artificial Intelligence Research 35: 557–591

    MathSciNet  MATH  Google Scholar 

  26. Krause, A., Horvitz, E., Kansal, A., & Zhao, F. (2008). Toward community sensing. In: Proceedings of information processing in sensor networks (IPSN-08) (pp. 481–492). IEEE Computer Society.

  27. Lesser V., Decker K., Wagner T., Carver N., Garvey A., Horling B., Neiman D., Podorozhny R., NagendraPrasad M., Raja A., Vincent R., Xuan P., Zhang X. (2004) Evolution of the GPGP/TAEMS domain-independent coordination framework. Autonomous Agents and Multi-Agent Systems 9(1): 87–143

    Article  Google Scholar 

  28. Li X., Ji Q. (2005) Active affective state detection and user assistance with dynamic Bayesian networks. IEEE Transactions on Systems, Man, and Cybernetics, Part A 35(1): 93–105

    Article  MathSciNet  Google Scholar 

  29. Liao W., Ji Q. (2008) Efficient non-myopic value-of-information computation for influence diagrams. International Journal of Approximate Reasoning 49: 436–450

    Article  MathSciNet  MATH  Google Scholar 

  30. Maheswaran, R., & Szekely, P. (2008). Criticality metrics for distributed plan and schedule management. In: Proceedings of the eighteenth international conference on automated planning and scheduling (ICAPS 2008) (pp. 14–18).

  31. Maheswaran, R., Tambe, M., Bowring, E., Pearce, J., & Varakantham, P. (2004). Taking DCOP to the real world: Efficient complete solutions for distributed multi-event scheduling. In: Proceedings of the third international joint conference on autonomous agents and multiagent systems (AAMAS-04) (pp. 310–317).

  32. Maheswaran, R., Rogers, C., Sanchez, R., & Szekely, P. (2010). Human-agent collaborative optimization of real-time distributed dynamic multi-agent coordination. In: Proceedings of AAMAS workshop on optimization in multiagent systems (pp. 49–56).

  33. McClure, W. (2000). Technology and command: Implications for military operations in the twenty-first century. Maxwell Air Force Base: Center for Strategy and Technology.

  34. Modi P., Shen W., Tambe M., Yokoo M. (2005) ADOPT: Asynchronous distributed constraint optimization with quality guarantees. Artificial Intelligence Journal 161(1–2): 149–180

    Article  MathSciNet  MATH  Google Scholar 

  35. Musliner, D., Durfee, E., Wu, J., Dolgov, D., Goldman, R., & Boddy, M. (2006). Coordinated plan management using multiagent MDPs. In: Proceedings of the AAAI spring symposium on distributed plan and schedule management (pp. 73–80).

  36. Musliner, D., Goldman, R., Durfee, E., Wu, J., Dolgov, D., & Boddy, M. (2007). Coordination of Highly Contingent Plans. In: Proceedings of the international conference on integration of knowledge intensive multi-agent systems (pp. 418–422).

  37. Myers, K. (1996). Strategic advice for hierarchical planners. In: Proceedings of the fifth international conference on principles of knowledge representation and reasoning (KR-96) (pp. 112–123).

  38. Myers, K., Jarvis, P., Tyson, M., & Wolverton, M. (2003). A mixed-initiative framework for robust plan sketching. In: Proceedings of the thirteenth international conference on automated planning and scheduling (ICAPS-03) (pp. 256–266).

  39. Phelps, J., & Rye, J. (2006). GPGP: A domain-independent implementation. In: Proceedings of the AAAI spring symposium on distributed plan and schedule management (pp. 81–88).

  40. Pollack M. (2005) Intelligent technology for an aging population: The use of AI to assist elders with cognitive impairment. AI Magazine 26(2): 9–24

    Google Scholar 

  41. Raiffa, H., Schlaifer, R. (Eds.) (1961) Applied statistical decision theory. Harvard University Press, Cambridge, MA

    Google Scholar 

  42. Raja, A., Alexander, G., & Mappillai, V. (2006). Leveraging problem classification in online meta- cognition. In: Proceedings of the AAAI spring symposium on distributed plan and schedule management (pp. 97–104).

  43. Rosenfeld A., Kraus S., Ortiz C. (2009) Measuring the expected gain of communicating constraint information. Multiagent and Grid Systems 5(4): 427–449

    MATH  Google Scholar 

  44. Sarne, D., & Grosz, B. J. (2006). Timing interruptions for better human-computer coordinated planning. In: Proceedings of the AAAI spring symposium on distributed plan and schedule management (pp. 161–162).

  45. Sarne, D., & Grosz, B. (2007). Estimating information value in collaborative multi-agent planning systems. In: Proceedings of the sixth international joint conference on autonomous agents and multiagent systems (AAMAS-07) (pp. 1–8).

  46. Sarne, D., & Grosz, B. (2007). Sharing experiences to learn user characteristics in dynamic environments with sparse data. In: Proceeding of the sixth international joint conference on autonomous agents and multiagent systems (AAMAS-07) (pp. 43–50).

  47. Sarne, D., Grosz, B., & Owotoki, P. (2008). Effective information value calculation for interruption management in multi-agent scheduling. In: Proceedings of the eighteenth international conference on automated planning and scheduling (ICAPS-08) (pp. 313–321).

  48. Scerri P., Pynadath D., Tambe M. (2002) Towards adjustable autonomy for the real world. Journal of Artificial Intelligence Research 17: 171–228

    MathSciNet  MATH  Google Scholar 

  49. Scerri, P., Pynadath, D., Johnson, W., Rosenbloom, P., Si, M., Schurr, N., & Tambe, M. (2003). A prototype infrastructure for distributed robot-agent-person teams. In: Proceedings of the second international joint conference on autonomous agents and multiagent systems (AAMAS-03) (pp. 433–440).

  50. Schurr, N., Marecki, J., Lewis, J., Tambe, M., Scerri, P. (2005). The DEFACTO system: Training tool for incident commanders. In: Proceedings of the twentieth national conference on artificial intelligence (AAAI-05) (pp. 1555–1562).

  51. Shahaf, D., & Horvitz, E. (2009). Investigations of continual computation. In: Proceedings of the twenty-first international joint conference on artificial intelligence (IJCAI-09) (pp. 285–291).

  52. Shrot, T., Rosenfeld, A., & Kraus, S. (2009). Leveraging users for efficient interruption management in agent-user systems. In: Proceedings of the 2009 IEEE/WIC/ACM international conference on intelligent agent technology (IAT-09) (pp. 123–130).

  53. Smith, S., Gallagher, A., Zimmerman, T., Barbulescu, L., & Rubinstein, Z. (2007). Distributed management of flexible times schedules. In: Proceedings of the sixth international joint conference on autonomous agents and multiagent systems (AAMAS-07) (pp. 472–479).

  54. Sultanik, E., Modi, P., & Regli, W. (2007). On modeling multiagent task scheduling as a distributed constraint optimization problem. In: Proceedings of the twentieth international joint conference on artificial intelligence (IJCAI-07) (pp. 1531–1536).

  55. Tolpin, D., & Shimony, E. (2010). Rational value of information estimation for measurement selection. In: Proceedings of the twenty-fifth mini-EURO conference on uncertainty and robustness in planning and decision making (URPDM-10).

  56. van Hoeve, W., Gomes, C., Selman, B., & Lombardi, M. (2007). Optimal multi-agent scheduling with constraint programming. In: Proceedings of the twenty-second national conference on artificial intelligence (AAAI-07) (pp. 1813–1818).

  57. Wagner, T., Guralnik, V., & Phelps, J. (2003). A key-based coordination algorithm for dynamic readiness and repair service coordination. In: Proceedings of the second international conference on autonomous agents and multiagent systems (AAMAS-03) (pp. 757–764).

  58. Wagner, T., Phelps, J., Guralnik, V., & VanRiper, R. (2004). An application view of Coordinators: Coordination managers for first responders. In: Proceedings of the nineteenth national conference on artificial intelligence (AAAI-04) (pp. 908–915).

  59. Wagner T., Raja A., Lesser V. (2006) Modeling uncertainty and its implications to sophisticated control in TAEMS agents. Autonomous Agents and Multi-Agent Systems 13(3): 235–292

    Article  Google Scholar 

  60. Wilkins D., Lee T., Berry P. (2003) Interactive execution monitoring of agent teams. Journal of Artificial Intelligence Research 18: 217–261

    MATH  Google Scholar 

  61. Wilkins, D., Smith, S., Kramer, L., Lee, T., & Rauenbusch, T. (2005). Execution monitoring and replanning with incremental and collaborative scheduling. In: Workshop on multiagent planning and scheduling, the fifteenth international conference on automated planning & scheduling (pp. 29–35).

  62. Yakout M., Elmagarmid A., Neville J., Ouzzani M., Ilyas I. (2011) Guided data repair. Proceedings of the VLDB Endowment (PVLDB) 4(5): 279–289

    Google Scholar 

  63. Zhang, Y. (2010). Multi-task active learning with output constraints. In: Proceedings of the twenty-forth national conference on artificial intelligence (AAAI-10).

  64. Zhang, L., Qi, R., & Poole, D. (1993). Incremental computation of the value of perfect information in stepwise-decomposable influence diagrams. In: Proceedings of the ninth international conference on uncertainty in artificial intelligence (UAI-93) (pp. 400–407).

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Bar-Ilan University, Ramat-Gan, 52900, Israel

    David Sarne

  2. School of Engineering and Applied Sciences, Harvard University, Cambridge, MA, 02138, USA

    Barbara J. Grosz

Authors
  1. David Sarne
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Barbara J. Grosz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Sarne.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Sarne, D., Grosz, B.J. Determining the value of information for collaborative multi-agent planning. Auton Agent Multi-Agent Syst 26, 456–496 (2013). https://doi.org/10.1007/s10458-012-9206-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10458-012-9206-9

Keywords

Search

Navigation