Skip to main content
SpringerLink
Log in

Exact Combinatorial Optimization with Temporo-Attentional Graph Neural Networks

  • Conference paper
  • First Online:
Machine Learning and Knowledge Discovery in Databases: Research Track (ECML PKDD 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14172))

  • 1032 Accesses

Abstract

Combinatorial optimization finds an optimal solution within a discrete set of variables and constraints. The field has seen tremendous progress both in research and industry. With the success of deep learning in the past decade, a recent trend in combinatorial optimization has been to improve state-of-the-art combinatorial optimization solvers by replacing key heuristic components with machine learning (ML) models. In this paper, we investigate two essential aspects of machine learning algorithms for combinatorial optimization: temporal characteristics and attention. We argue that for the task of variable selection in the branch-and-bound (B&B) algorithm, incorporating the temporal information as well as the bipartite graph attention improves the solver’s performance. We support our claims with intuitions and numerical results over several standard datasets used in the literature and competitions. (Code is available at: https://developer.huaweicloud.com/develop/aigallery/notebook/detail?id=047c6cf2-8463-40d7-b92f-7b2ca998e935.)

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Achterberg, T.: Scip: solving constraint integer programs. Math. Program. Comput. 1(1), 1–41 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  2. Achterberg, T., Koch, T., Martin, A.: Branching rules revisited. Oper. Res. Lett. 33(1), 42–54 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  3. Achterberg, T., Wunderling, R.: Mixed integer programming: Analyzing 12 years of progress. In: Facets of Combinatorial Optimization, pp. 449–481. Springer (2013)

    Google Scholar 

  4. Alvarez, A.M., Louveaux, Q., Wehenkel, L.: A machine learning-based approximation of strong branching. INFORMS J. Comput. 29(1), 185–195 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  5. Banitalebi-Dehkordi, A., Zhang, Y.: Ml4co: Is gcnn all you need? graph convolutional neural networks produce strong baselines for combinatorial optimization problems, if tuned and trained properly, on appropriate data. arXiv preprint arXiv:2112.12251 (2021)

  6. Bennell, J.A., Mesgarpour, M., Potts, C.N.: Dynamic scheduling of aircraft landings. Eur. J. Oper. Res. 258(1), 315–327 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  7. Bergman, D., Cire, A.A., Van Hoeve, W.J., Hooker, J.: Decision diagrams for optimization, vol. 1. Springer (2016)

    Google Scholar 

  8. Bixby, B.: The gurobi optimizer. Transp. Res. Part B 41(2), 159–178 (2007)

    Google Scholar 

  9. Boyd, S., Boyd, S.P., Vandenberghe, L.: Convex optimization. Cambridge University Press (2004)

    Google Scholar 

  10. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  Google Scholar 

  11. Brody, S., Alon, U., Yahav, E.: How attentive are graph attention networks? arXiv preprint arXiv:2105.14491 (2021)

  12. Burges, C.J.: From ranknet to lambdarank to lambdamart: An overview. Learning 11(23–581), 81 (2010)

    Google Scholar 

  13. Cao, Z., Xu, Y., Huang, Z., Zhou, S.: Ml4co-kida: Knowledge inheritance in dataset aggregation. arXiv preprint arXiv:2201.10328 (2022)

  14. Cornuéjols, G., Sridharan, R., Thizy, J.M.: A comparison of heuristics and relaxations for the capacitated plant location problem. Eur. J. Oper. Res. 50(3), 280–297 (1991)

    Article  MATH  Google Scholar 

  15. Ding, J.Y., Zhang, C., Shen, L., Li, S., Wang, B., Xu, Y., Song, L.: Accelerating primal solution findings for mixed integer programs based on solution prediction. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, pp. 1452–1459 (2020)

    Google Scholar 

  16. ecole.ai: Ml4co: 2021 neurips competition on machine learning for combinatorial optimization (2021). www.ecole.ai/2021/ml4co-competition/

  17. Fiala, J., Kočvara, M., Stingl, M.: Penlab: A matlab solver for nonlinear semidefinite optimization. arXiv preprint arXiv:1311.5240 (2013)

  18. Flood, M.M.: The traveling-salesman problem. Oper. Res. 4(1), 61–75 (1956)

    Article  MathSciNet  MATH  Google Scholar 

  19. Gamrath, G., et al.: The SCIP Optimization Suite 7.0. ZIB-Report 20–10, Zuse Institute Berlin, March 2020. www.nbn-resolving.de/urn:nbn:de:0297-zib-78023

  20. Gamrath, G., et al.: The scip optimization suite 7.0 (2020)

    Google Scholar 

  21. Gasse, M., et al.: The machine learning for combinatorial optimization competition (ml4co): Results and insights. arXiv preprint: 2203.02433 (2022)

    Google Scholar 

  22. Gasse, M., Chételat, D., Ferroni, N., Charlin, L., Lodi, A.: Exact combinatorial optimization with graph convolutional neural networks. Advances in Neural Information Processing Systems 32 (2019)

    Google Scholar 

  23. Geurts, P., Ernst, D., Wehenkel, L.: Extremely randomized trees. Mach. Learn. 63(1), 3–42 (2006)

    Google Scholar 

  24. Graham, R.L., Hell, P.: On the history of the minimum spanning tree problem. Ann. History Comput. 7(1), 43–57 (1985)

    Article  MathSciNet  MATH  Google Scholar 

  25. Gupta, P., Gasse, M., Khalil, E., Mudigonda, P., Lodi, A., Bengio, Y.: Hybrid models for learning to branch. Adv. Neural. Inf. Process. Syst. 33, 18087–18097 (2020)

    Google Scholar 

  26. Hamilton, W.L.: Graph representation learning. Synthesis Lectures Artif. Intell. Mach. Learn. 14(3), 1–159 (2020)

    Article  MATH  Google Scholar 

  27. He, H., Daume III, H., Eisner, J.M.: Learning to search in branch and bound algorithms. Advances in neural information processing systems 27 (2014)

    Google Scholar 

  28. Hoffman, K.L., Ralphs, T.K.: Integer and combinatorial optimization. In: Encyclopedia of Operations Research and Management Science, pp. 771–783 (2013)

    Google Scholar 

  29. Howard, R.A.: Dynamic programming and markov processes. John Wiley (1960)

    Google Scholar 

  30. Hussein, A., Gaber, M.M., Elyan, E., Jayne, C.: Imitation learning: a survey of learning methods. ACM Comput. Surv. (CSUR) 50(2), 1–35 (2017)

    Article  Google Scholar 

  31. Hutter, F., Hoos, H.H., Leyton-Brown, K.: Sequential model-based optimization for general algorithm configuration. In: Coello, C.A.C. (ed.) LION 2011. LNCS, vol. 6683, pp. 507–523. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-25566-3_40

    Chapter  Google Scholar 

  32. Khalil, E., Dai, H., Zhang, Y., Dilkina, B., Song, L.: Learning combinatorial optimization algorithms over graphs. Advances in neural information processing systems 30 (2017)

    Google Scholar 

  33. Khalil, E., Le Bodic, P., Song, L., Nemhauser, G., Dilkina, B.: Learning to branch in mixed integer programming. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30 (2016)

    Google Scholar 

  34. Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: International Conference on Learning Representations (ICLR) (2017)

    Google Scholar 

  35. Knueven, B., Ostrowski, J., Watson, J.P.: On mixed-integer programming formulations for the unit commitment problem. INFORMS J. Comput. 32(4), 857–876 (2020)

    MathSciNet  MATH  Google Scholar 

  36. Land, A.H., Doig, A.G.: An automatic method for solving discrete programming problems. In: Jünger, M., Liebling, T.M., Naddef, D., Nemhauser, G.L., Pulleyblank, W.R., Reinelt, G., Rinaldi, G., Wolsey, L.A. (eds.) 50 Years of Integer Programming 1958-2008, pp. 105–132. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-540-68279-0_5

    Chapter  MATH  Google Scholar 

  37. Leyton-Brown, K., Pearson, M., Shoham, Y.: Towards a universal test suite for combinatorial auction algorithms. In: Proceedings of the 2nd ACM Conference on Electronic Commerce, pp. 66–76 (2000)

    Google Scholar 

  38. Manual, C.U.: Ibm ilog cplex optimization studio. Version 12, 1987–2018 (1987)

    Google Scholar 

  39. Morais, H., Kádár, P., Faria, P., Vale, Z.A., Khodr, H.: Optimal scheduling of a renewable micro-grid in an isolated load area using mixed-integer linear programming. Renew. Energy 35(1), 151–156 (2010)

    Article  Google Scholar 

  40. Nair, V., et al.: Solving mixed integer programs using neural networks. arXiv preprint arXiv:2012.13349 (2020)

  41. Papageorgiou, D.J., Nemhauser, G.L., Sokol, J., Cheon, M.S., Keha, A.B.: Mirplib-a library of maritime inventory routing problem instances: survey, core model, and benchmark results. Eur. J. Oper. Res. 235(2), 350–366 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  42. Pomerleau, D.A.: Efficient training of artificial neural networks for autonomous navigation. Neural Comput. 3(1), 88–97 (1991)

    Article  Google Scholar 

  43. Prouvost, A., Dumouchelle, J., Scavuzzo, L., Gasse, M., Chételat, D., Lodi, A.: Ecole: a gym-like library for machine learning in combinatorial optimization solvers. In: Learning Meets Combinatorial Algorithms at NeurIPS2020 (2020). www.openreview.net/forum?id=IVc9hqgibyB

  44. Qu, Q., Li, X., Zhou, Y.: Yordle: an efficient imitation learning for branch and bound. arXiv preprint arXiv:2112.12251 (2022)

  45. Ross, S., Gordon, G., Bagnell, D.: A reduction of imitation learning and structured prediction to no-regret online learning. In: Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pp. 627–635. JMLR Workshop and Conference Proceedings (2011)

    Google Scholar 

  46. Salkin, H.M., De Kluyver, C.A.: The knapsack problem: a survey. Naval Res. Logistics Quarterly 22(1), 127–144 (1975)

    Article  MathSciNet  MATH  Google Scholar 

  47. Shen, Y., Sun, Y., Eberhard, A., Li, X.: Learning primal heuristics for mixed integer programs. In: 2021 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2021)

    Google Scholar 

  48. Song, J., Lanka, R., Zhao, A., Bhatnagar, A., Yue, Y., Ono, M.: Learning to search via retrospective imitation. arXiv preprint arXiv:1804.00846 (2018)

  49. Vaz, A.I.F., Vicente, L.N.: Pswarm: a hybrid solver for linearly constrained global derivative-free optimization. Optimization Methods Softw. 24(4–5), 669–685 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  50. Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. Stat 1050, 20 (2017)

    Google Scholar 

  51. Xu, B., Wang, N., Chen, T., Li, M.: Empirical evaluation of rectified activations in convolutional network. arXiv preprint arXiv:1505.00853 (2015)

  52. Zhang, T., Banitalebi-Dehkordi, A., Zhang, Y.: Deep reinforcement learning for exact combinatorial optimization: learning to branch. In: 26th International Conference on Pattern Recognition, ICPR (2022)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Huawei Technologies Canada Co., Ltd., Shenzhen, China

    Mehdi Seyfi, Amin Banitalebi-Dehkordi, Zirui Zhou & Yong Zhang

Authors
  1. Mehdi Seyfi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amin Banitalebi-Dehkordi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zirui Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Amin Banitalebi-Dehkordi .

Editor information

Editors and Affiliations

  1. University of Michigan, Ann Arbor, MI, USA

    Danai Koutra

  2. University of Vienna, Vienna, Austria

    Claudia Plant

  3. Max Planck Institute for Software Systems, Kaiserslautern, Germany

    Manuel Gomez Rodriguez

  4. Politecnico di Torino, Turin, Italy

    Elena Baralis

  5. CENTAI, Turin, Italy

    Francesco Bonchi

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Seyfi, M., Banitalebi-Dehkordi, A., Zhou, Z., Zhang, Y. (2023). Exact Combinatorial Optimization with Temporo-Attentional Graph Neural Networks. In: Koutra, D., Plant, C., Gomez Rodriguez, M., Baralis, E., Bonchi, F. (eds) Machine Learning and Knowledge Discovery in Databases: Research Track. ECML PKDD 2023. Lecture Notes in Computer Science(), vol 14172. Springer, Cham. https://doi.org/10.1007/978-3-031-43421-1_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43421-1_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43420-4

  • Online ISBN: 978-3-031-43421-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation