Skip to main content

Advertisement

SpringerLink
Log in

Agent-based decentralised coordination for sensor networks using the max-sum algorithm

  • Published:
Autonomous Agents and Multi-Agent Systems Aims and scope Submit manuscript

Abstract

In this paper, we consider the generic problem of how a network of physically distributed, computationally constrained devices can make coordinated decisions to maximise the effectiveness of the whole sensor network. In particular, we propose a new agent-based representation of the problem, based on the factor graph, and use state-of-the-art DCOP heuristics (i.e., DSA and the max-sum algorithm) to generate sub-optimal solutions. In more detail, we formally model a specific real-world problem where energy-harvesting sensors are deployed within an urban environment to detect vehicle movements. The sensors coordinate their sense/sleep schedules, maintaining energy neutral operation while maximising vehicle detection probability. We theoretically analyse the performance of the sensor network for various coordination strategies and show that by appropriately coordinating their schedules the sensors can achieve significantly improved system-wide performance, detecting up to 50 % of the events that a randomly coordinated network fails to detect. Finally, we deploy our coordination approach in a realistic simulation of our wide area surveillance problem, comparing its performance to a number of benchmarking coordination strategies. In this setting, our approach achieves up to a 57 % reduction in the number of missed vehicles (compared to an uncoordinated network). This performance is close to that achieved by a benchmark centralised algorithm (simulated annealing) and to a continuously powered network (which is an unreachable upper bound for any coordination approach).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

Notes

  1. Problems used to benchmark Reinforcement Learning techniques based on MDPs typically involve a few agents with a few actions, see for example the distributed sensor network problem used in [22] where eight sensors must collaborate to track three targets.

  2. A notable exception is the superstabilizing version of DPOP (S-DPOP) proposed in [37], where the authors aim to minimize changes in the optimization protocol when there are low impact failures in the system. Nevertheless, similar to the original DPOP approach, S-DPOP requires agents to exchange large messages (where the size of the messages is exponential in the tree-width of the pseudo-tree arrangement). Hence such an approach would not be feasible in the context of low-power devices that constitute our reference application platform.

  3. see www.robocuprescue.org

  4. Notice that the analysis and empirical evaluation performed in [21, 22] only include pairwise interactions.

  5. Notice that, while the content of the max-sum messages will not change the load balance among the agents will be different depending on the strategy used to allocate the functions. However this issue is outside the scope of the current paper and we refer the interested reader to [44] where computational tasks related to GDL algorithms are allocated to agents explicitly considering their computation and communication capabilities.

  6. As stated in [41] this normalisation will fail in the case of a negative infinity utility that represents a hard constraint on the solution. However, we can replace the negative infinity reward with one whose absolute value is greater than the sum of the maximum values of each function.

  7. This pseudo-code description is based on the procedures for max-sum message computation presented in [7]

  8. This is equivalent to the statement that zero clashes is a strict lower bound for solutions to a graph colouring problem, even though a specific problem instance may not be colourable.

  9. Hence, in the specific setting we consider here, sensors can compute the number of mutually observed events (i.e., \(O_{\{i\}\cup \mathbf k }\)) by sending at regular intervals a message to all neighbours that contains, for each detected vehicles, the time of appearance and the time of disappearance.

  10. Notice that, the empirical setting here is different from Sect. 3.5, but, as discussed below, Simulated Annealing still performs very close to the continuously powered network.

  11. As in version DSA-C of [56] we allow agents to change assignment whenever the utility does not degrade, hence agents are allowed to change the assignment also when the best alternative gives the same value of local utility. However, in [56] authors focus on a graph colouring problem and hence differentiate between situations where there is at least one conflict and the utility does not degrade (both DSA-B and DSA-C can change assignment) and situations where there is no conflict and the utility does not degrade (only DSA-C can change assignment). Since here we do not have hard constraints we do not consider conflicts but only the value of the utility, in this sense our version of DSA is similar to DSA-C.

  12. We set the threshold to \(10^{-3}\).

  13. The performance improvement of a method X over a method Y is computed as (performance of X - performance of Y)/performance of X.

  14. Execution time is heavily dependent on many implementation specific details, which are not relevant to the core ideas of the technique. Simulated annealing is, in this respect, a notable exception, as it requires considerably more time than the other techniques. However, simulated annealing is used here only as a centralised upper bound on system performance.

  15. These results confirm the behaviour observed in [10] where max-sum and DSA were compared in the presence of a lossy communication channel on graph colouring benchmarks.

References

  1. Aji, S. M., & McEliece, R. J. (2000). The generalized distributive law. Information Theory IEEE Transactions, 46(2), 325–343.

    Article  MATH  MathSciNet  Google Scholar 

  2. Ammari, H. M., & Das, S. R. (2009). Fault tolerance measures for large-scale wireless sensor networks. ACM Transactions on Autonomous and Adaptive System, 4(1), 1–28.

    Article  Google Scholar 

  3. Béjar, R., Domshlak, C., Fernández, C., Gomes, C., Krishnamachari, B., Selman, B., et al. (2005). Sensor networks and distributed csp: Communication, computation and complexity. Artificial Intelligence, 161(1–2), 117–147.

    Article  MATH  MathSciNet  Google Scholar 

  4. Bernstein, D. S., Zilberstein, S., Immerman, N. (2000) . The complexity of decentralized control of markov decision processes. In Proceedings of UAI-2000, pp. 32–37.

  5. Bishop, C. M. (2006). Pattern recognition and machine learning. Berlin: Springer.

    MATH  Google Scholar 

  6. Dechter, R. (2003). Constraint processing. San Francisco: Morgan Kaufmann.

    Google Scholar 

  7. Delle Fave, F. M., Farinelli, A., Rogers, A., Jennings, N. R. (2012). A methodology for deploying the max-sum algorithm and a case study on unmanned aerial vehicles. In IAAI 2012: The Twenty-Fourth Innovative Applications of Artificial Intelligence Conference, pp. 2275–2280.

  8. Delle Fave, F. M., Rogers, A., Xu, Z., Sukkarieh, S., Jennings, N. R. (2012). Deploying the max-sum algorithm for coordination and task allocation of unmanned aerial vehicles for live aerial imagery collection. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 469–476.

  9. Farinelli, A., Rogers, A., Jennings, N. R. (2008). Maximising sensor network efficiency through agent-based coordination of sense/sleep schedules. In Proceedings of the Workshop on Energy in Wireless Sensor Networks in conjuction with DCOSS 2008.

  10. Farinelli, A., Rogers, A., Petcu, A., Jennings, N. R. (2008). Decentralised coordination of low-power embedded devices using the max-sum algorithm. In Proceedings of the Seventh International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2008), pp. 639–646.

  11. Fernández, R., Béjar, R., Krishnamachari, B., Gomes, C., & Selman, B. (2003). Distributed sensor networks a multiagent perspective, chapter communication and computation in distributed CSP algorithms (pp. 299–319). Dordrecht: Kluwer Academic.

    Google Scholar 

  12. Fitzpatrick, S., & Meertens, L. (2003). Distributed sensor networks a multiagent perspective, chapter distributed coordination through anarchic optimization (pp. 257–293). Dordrecht: Kluwer Academic.

    Google Scholar 

  13. Frey, B. J., & Dueck, D. (2007). Clustering by passing messages between data points. Science, 315(5814), 972.

    Article  MATH  MathSciNet  Google Scholar 

  14. Giusti, A., Murphy, A. L., & Picco, G. P. (2007). Decentralised scattering of wake-up times in wireless sensor networks. In Proceedings of the Fourth European Conference on Wireless Sensor Networks, pp. 245–260.

  15. Guestrin, C., Koller, D., Parr, R. (2001). Multiagent planning with factored mdps. In Advances in neural information processing systems (NIPS), pp. 1523–1530, Vancouver.

  16. Guestrin, C., Lagoudakis, M., Parr, R. (2002). Coordinated reinforcement learning. In Proceedings of ICML-02, pp. 227–234.

  17. Hsin, C., Liu, M. (2004). Network coverage using low duty-cycled sensors: Random & coordinated sleep algorithm. In Proceedings of the Third International Symposium on Information Processing in Sensor Networks (IPSN 2004), pp. 433–442.

  18. Kansal, A., Hsu, J., Zahedi, S., & Srivastava, M. B. (2007). Power management in energy harvesting sensor networks. ACM Transactions on Embedded Computing Systems, 6(4), 54–61.

    Article  Google Scholar 

  19. Kho, J., Rogers, A., & Jennings, N. R. (2009). Decentralised control of adaptive sampling in wireless sensor networks. ACM Transactions on Sensor Networks, 5(3), 19–35.

    Article  Google Scholar 

  20. Kitano, H. (2000). Robocup rescue: A grand challenge for multi-agent systems. In Proceedings of the Fourth International Conference on Multi-Agent Systems (ICMAS), pp. 5–12.

  21. Kok, J. R., Vlassis, N. (2005). Using the max-plus algorithm for multiagent decision making in coordination graphs. In RoboCup-2005: Robot Soccer World Cup IX, Osaka.

  22. Kok, J. R., & Vlassis, N. (December 2006). Collaborative multiagent reinforcement learning by payoff propagation. Journal of Machine Learning Research, 7, 1789–1828.

  23. Kschischang, F. R., Frey, B. J., & Loeliger, H. A. (2001). Factor graphs and the sum-product algorithm. IEEE Transactions on Information Theory, 42(2), 498–519.

    Article  MathSciNet  Google Scholar 

  24. Kumar, S., Lai, H. T., Balogh, J. (2004). On k-coverage in a mostly sleeping sensor network. In Proceedings of the Tenth Annual International Conference on Mobile Computing and Networking (MobiCom 2004), pp. 144–158.

  25. Lesser, V., Ortiz, C. L., & Tambe, M. (2003). Distributed sensor networks a multiagent perspective. Dordrecht: Kluwer Academic.

    Book  MATH  Google Scholar 

  26. MacKay, D. J. C. (1999). Good error-correcting codes based on very sparse matrices. IEEE Transactions on Information Theory, 45(2), 399–431.

    Article  MATH  MathSciNet  Google Scholar 

  27. MacKay, D. J. C. (2003). Information theory, inference, and learning algorithms. New York: Cambridge University Press.

    MATH  Google Scholar 

  28. Maheswaran, R. T., Pearce, J. P., Tambe, M. (2004). Distributed algorithms for dcop: A graphical-game-based approach. In the 17th International Conference on Parallel and Distributed Computing Systems (PDCS), pp. 432–439.

  29. Mailler, R., Lesser, V. (2004). Solving distributed constraint optimization problems using cooperative mediation. In Proceedings of Third International Joint Conference on Autonomous Agents and MultiAgent Systems (AAMAS 2004), pp. 438–445.

  30. Makarenko, A., Durrant-Whyte, H.F. (2004). Decentralized data fusion and control algorithms in active sensor networks. In Proceedings of Seventh International Conference on Information Fusion (Fusion 2004), pp. 479–486.

  31. Mezard, M., Parisi, G., & Zecchina, R. (2002). Analytic and algorithmic solution of random satisfiability problems. Science, 297(5582), 812–815.

    Article  Google Scholar 

  32. Modi, P. J., Scerri, P., Shen, W. M., & Tambe, M. (2003). Distributed sensor networks a multiagent perspective, chapter distributed resource allocation (pp. 219–256). Dordrecht: Kluwer Academic.

    Google Scholar 

  33. Modi, P. J., Shen, W., Tambe, M., & Yokoo, M. (2005). ADOPT: Asynchronous distributed constraint optimization with quality guarantees. Artificial Intelligence Journal, 161, 149–180.

    Article  MATH  MathSciNet  Google Scholar 

  34. Murphy, K. P., Weiss, Y., Jordan, M. I. (1999). Loopy belief propagation for approximate inference: An empirical study. In Proceedings of the Fifteenth Conference on Uncertainty in, Artificial Intelligence (UAI’99), pp. 467–475.

  35. Oliehoek, Frans A. (2010). Value-based planning for teams of agents in stochastic partially observable environments. PhD thesis, Informatics Institute, University of Amsterdam.

  36. Petcu, A., Faltings, B. (2005). DPOP: A scalable method for multiagent constraint optimization. In Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence, (IJCAI 2005), pp. 266–271.

  37. Petcu, A., Faltings, B. (2005). S-dpop: Superstabilizing, fault-containing multiagent combinatorial optimization. In Proceedings of the National Conference on Artificial Intelligence, AAAI-05, pp. 449–454, Pittsburgh, AAAI.

  38. Ramchurn, S., Farinelli, A., Macarthur, K., Polukarov, M., & Jennings, N. R. (2010). Decentralised coordination in robocup rescue. The Computer Journal, 53(9), 1–15.

    Article  Google Scholar 

  39. Rogers, A., David, E., & Jennings, N. R. (2005). Self-organized routing for wireless microsensor networks. Systems Man and Cybernetics Part A IEEE Transactions, 35(3), 349–359.

    Article  Google Scholar 

  40. Rogers, A., Farinelli, A. and Jennings, N. R. (2010). Self-organising sensors for wide area surveillance using the max-sum algorithm.

  41. Rogers, A., Farinelli, A., Stranders, R., & Jennings, N. R. (February 2011). Bounded approximate decentralised coordination via the max-sum algorithm. Artificial Intelligence, 175(2), 730–759.

  42. Rozanov, Y. A. (1977). Probability theory: A concise course. Dover Publications: Dover Books on Mathematics Series.

  43. Bar Shalom, Y., & Fortmann, T. E. (1988). Tracking and data association. Boston: Academic-Press.

    MATH  Google Scholar 

  44. Stefanovitch, N., Farinelli, A., Rogers, A., Jennings, N. R. (2011). Resource-aware junction trees for efficient multi-agent coordination. In Tenth International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2011), pp. 363–370, Taipei.

  45. Stranders, R., Farinelli, A., Rogers, A., Jennings, N. R. (2009). Decentralised control of continuously valued control parameters using the max-sum algorithm. In 8th International Conference on Autonomous Agents and Multiagent Systems, pp. 601–608.

  46. Stranders, R., Farinelli, A., Rogers, A., & Jennings, N. R. (2009). Decentralised coordination of mobile sensors using the max-sum algorithm. In Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence, pp. 299–304.

  47. Sultanik, E. A., Lass, R. N., Regli, W. C. (2009). Dynamic configuration of agent organizations. In Proceedings of the 21st international jont conference on Artifical, intelligence, IJCAI’09, pp. 305–311.

  48. Teacy,W. T. L., Chalkiadakis, G., Farinelli, A., Rogers, A., Jennings, N. R., McClean, S., Parr, G. (2012). Decentralized bayesian reinforcement learning for online agent collaboration. In 11th International Conference on Autonomous Agents and Multiagent Systems, pp. 417–424.

  49. Teacy W. T. L., Farinelli, A., Grabham, N. J., Padhy, P., Rogers, A., Jennings, N. R. (2008). Max-sum decentralised coordination for sensor systems. In 7th International Conference on Autonomous Agents and Multiagent Systems, pp. 1697–1698.

  50. Velagapudi, P., Varakantham, P., Sycara, K., Scerri, P. (2011). Distributed model shaping for scaling to decentralized pomdps with hundreds of agents. In The 10th International Conference on Autonomous Agents and Multiagent Systems, Vol. 3, AAMAS ’11, pp. 955–962.

  51. Vinyals, M., Cerquides, J., Farinelli, A., Rodrguez-Aguilar, J. A. (2010). Worst-case bounds on the quality of max-product fixed-points. In In Neural Information Processing Systems (NIPS), pp. 2325–2333. Vancouver: MIT Press.

  52. Vinyals, M., Rodriguez-Aguilar, J., & Cerquides, J. (2011). Constructing a unifying theory of dynamic programming dcop algorithms via the generalized distributive law. Autonomous Agents and Multi-Agent Systems, 22, 439–464.

    Article  Google Scholar 

  53. Weddell, A. S., Harris, N. R., White, N. M. (2008). Alternative Energy Sources for Sensor Nodes: Rationalized Design for Long-Term Deployment. In Proceedings of the IEEE International Instrumentation and Measurement Technology Conference (\(I^2MTC\) 2008). (in press).

  54. Weiss, Y., & Freeman, W. T. (2001). On the optimality of solutions of the max-product belief propagation algorithm in arbitrary graphs. IEEE Transactions on Information Theory, 47(2), 723–735.

    Article  MathSciNet  Google Scholar 

  55. Zhang, P., Sadler, C., Lyon, S., Martonosi, M. (2004). Hardware design experiences in zebranet. In Proceedings of the ACM Conference on Embedded Networked Sensor Systems (SenSys).

  56. Zhang, W., Wang, G., Xing, Z., & Wittenburg, L. (January 2005). Distributed stochastic search and distributed breakout: Properties, comparison and applications to constraint optimization problems in sensor networks. Artificial Intelligence, 161(1–2), 55–87.

    Google Scholar 

Download references

Acknowledgments

This work was funded by the ORCHID project (http://www.orchid.ac.uk/). Preliminary versions of some of the material presented in this article have previously appeared in the paper [40] and in the workshop paper [9]. In particular, in [9] we proposed the use of the max-sum approach to coordinate the sense/sleep cycles of energy constrained sensor networks for wide area surveillance, while in [40] we extend that contribution by removing the assumption that agents have a priori knowledge about their deployment. Here we significantly extend both the contributions by providing a more detailed discussion about the max-sum algorithm and new experiments. In particular we give a detailed description of the methodology to model coordination problem using different types of factor graphs and we include an example to clarify max-sum message computation. Moreover, we provide new experiments to evaluate the impact of the calibration phase on network performance, the trade-off between coordination overhead and performance, and finally the performance of coordination mechanisms to lossy communication.

Author information

Authors and Affiliations

  1. Computer Science Department, University of Verona, Verona, Italy

    A. Farinelli

  2. Electronics and Computer Science, University of Southampton, Southampton, SO17 1BJ, UK

    A. Rogers & N. R. Jennings

Authors
  1. A. Farinelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Rogers
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. N. R. Jennings
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. Farinelli.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (wmv 11722 KB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Farinelli, A., Rogers, A. & Jennings, N.R. Agent-based decentralised coordination for sensor networks using the max-sum algorithm. Auton Agent Multi-Agent Syst 28, 337–380 (2014). https://doi.org/10.1007/s10458-013-9225-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10458-013-9225-1

Keywords

Search

Navigation