Abstract
While mixed integer linear programming (MILP) solvers are routinely used to solve a wide range of important science and engineering problems, it remains a challenging task for end users to write correct and efficient MILP constraints, especially for problems specified using the inherently non-linear Boolean logic operations. To overcome this challenge, we propose a syntax guided synthesis (SyGuS) method capable of generating high-quality MILP constraints from the specifications expressed using arbitrary combinations of Boolean logic operations. At the center of our method is an extensible domain specification language (DSL) whose expressiveness may be improved by adding new integer variables as decision variables, together with an iterative procedure for synthesizing linear constraints from non-linear Boolean logic operations using these integer variables. To make the synthesis method efficient, we also propose an over-approximation technique for soundly proving the correctness of the synthesized linear constraints, and an under-approximation technique for safely pruning away the incorrect constraints. We have implemented and evaluated the method on a wide range of benchmark specifications from statistics, machine learning, and data science applications. The experimental results show that the method is efficient in handling these benchmarks, and the quality of the synthesized MILP constraints is close to, or higher than, that of manually-written constraints in terms of both compactness and solving time.
- Sina Aghaei, Mohammad Javad Azizi, and Phebe Vayanos. 2019. Learning optimal and fair decision trees for non-discriminative decision-making. In Proceedings of the AAAI Conference on Artificial Intelligence. 33, 1418–1426. https://doi.org/10.1609/aaai.v33i01.33011418 Google ScholarDigital Library
- Rajeev Alur, Rastislav Bodik, Garvit Juniwal, Milo MK Martin, Mukund Raghothaman, Sanjit A Seshia, Rishabh Singh, Armando Solar-Lezama, Emina Torlak, and Abhishek Udupa. 2013. Syntax-guided synthesis. In International Conference on Formal Methods in Computer Aided Design. IEEE. https://doi.org/1721.1/90876 Google ScholarCross Ref
- Rajeev Alur, Arjun Radhakrishna, and Abhishek Udupa. 2017. Scaling enumerative program synthesis via divide and conquer. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems. 319–336. https://doi.org/10.1007/978-3-662-54577-5_18 Google ScholarCross Ref
- Pietro Belotti, Leo Liberti, Andrea Lodi, Giacomo Nannicini, and Andrea Tramontani. 2011. Disjunctive inequalities: applications and extensions. Wiley Encyclopedia of Operations Research and Management Science, 2 (2011), 1441–1450. Google Scholar
- Dimitris Bertsimas, Ryan Cory-Wright, and Jean Pauphilet. 2021. A unified approach to mixed-integer optimization problems with logical constraints. SIAM Journal on Optimization, 31, 3 (2021), 2340–2367. https://doi.org/1721.1/144075 Google ScholarDigital Library
- Dimitris Bertsimas, Ryan Cory-Wright, and Jean Pauphilet. 2022. Solving Large-Scale Sparse PCA to Certifiable (Near) Optimality.. J. Mach. Learn. Res., 23 (2022), 13–1. Google Scholar
- Dimitris Bertsimas and Jack Dunn. 2017. Optimal classification trees. Machine Learning, 106, 7 (2017), 1039–1082. https://doi.org/10.1007/s10994-017-5633-9 Google ScholarDigital Library
- Dimitris Bertsimas and Bart Van Parys. 2020. Sparse high-dimensional regression: Exact scalable algorithms and phase transitions. The Annals of Statistics, 48, 1 (2020), 300–323. https://doi.org/10.1214/18-AOS1804 Google ScholarCross Ref
- Jeff Bezanson, Alan Edelman, Stefan Karpinski, and Viral B Shah. 2017. Julia: A fresh approach to numerical computing. SIAM review, 59, 1 (2017), 65–98. https://doi.org/10.1137/141000671 Google ScholarDigital Library
- Bob Bixby. 2007. The gurobi optimizer. Transp. Re-search Part B, 41, 2 (2007), 159–178. Google Scholar
- Tim Blazytko, Moritz Contag, Cornelius Aschermann, and Thorsten Holz. 2017. Syntia: Synthesizing the Semantics of Obfuscated Code.. In USENIX Security Symposium. 643–659. https://doi.org/10.5555/3241189.3241240 Google ScholarDigital Library
- Donald Chai and Andreas Kuehlmann. 2003. A fast pseudo-boolean constraint solver. In Proceedings of the 40th annual Design Automation Conference. 830–835. https://doi.org/10.1145/775832.776041 Google ScholarDigital Library
- Allison Chang, Dimitris Bertsimas, and Cynthia Rudin. 2012. An integer optimization approach to associative classification. In Advances in neural information processing systems. 269–277. Google Scholar
- Allison An Chang. 2012. Integer optimization methods for machine learning. Ph. D. Dissertation. Massachusetts Institute of Technology. https://doi.org/1721.1/72643 Google Scholar
- Marco Cococcioni and Lorenzo Fiaschi. 2021. The Big-M method with the numerical infinite M. Optimization Letters, 15, 7 (2021), 2455–2468. https://doi.org/10.1007/s11590-020-01644-6 Google ScholarCross Ref
- IBM ILOG CPLEX. 2015. V12. 6 User’s Manual for CPLEX 2015. CPLEX division. Google Scholar
- Jo Devriendt, Ambros Gleixner, and Jakob Nordström. 2021. Learn to relax: Integrating 0-1 integer linear programming with pseudo-Boolean conflict-driven search. Constraints, 26, 1 (2021), 26–55. https://doi.org/10.1007/s10601-020-09318-x Google ScholarDigital Library
- Hassan Eldib and Chao Wang. 2014. Synthesis of Masking Countermeasures against Side Channel Attacks. In International Conference on Computer Aided Verification. Springer, 114–130. https://doi.org/10.1007/978-3-319-08867-9_8 Google ScholarDigital Library
- Hassan Eldib, Meng Wu, and Chao Wang. 2016. Synthesis of Fault-Attack Countermeasures for Cryptographic Circuits. In International Conference on Computer Aided Verification. Springer, 343–363. https://doi.org/10.1007/978-3-319-41540-6_19 Google ScholarCross Ref
- Weijie Feng, Binbin Liu, Dongpeng Xu, Qilong Zheng, and Yun Xu. 2020. Neureduce: Reducing mixed boolean-arithmetic expressions by recurrent neural network. In Findings of the Association for Computational Linguistics: EMNLP 2020. 635–644. Google ScholarCross Ref
- Yu Feng, Ruben Martins, Osbert Bastani, and Isil Dillig. 2018. Program synthesis using conflict-driven learning. ACM SIGPLAN Notices, 53, 4 (2018), 420–435. https://doi.org/10.1145/3192366.3192382 Google ScholarDigital Library
- Yu Feng, Ruben Martins, Jacob Van Geffen, Isil Dillig, and Swarat Chaudhuri. 2017. Component-based synthesis of table consolidation and transformation tasks from examples. ACM SIGPLAN Notices, 52, 6 (2017), 422–436. https://doi.org/10.1145/3062341.3062351 Google ScholarDigital Library
- John K Feser, Swarat Chaudhuri, and Isil Dillig. 2015. Synthesizing data structure transformations from input-output examples. ACM SIGPLAN Notices, 50, 6 (2015), 229–239. https://doi.org/10.1145/2813885.2737977 Google ScholarDigital Library
- Richard John Forrester and Harvey J Greenberg. 2008. Quadratic binary programming models in computational biology. Algorithmic Operations Research, 3, 2 (2008). Google Scholar
- Robert Fourer. 2014. Strategies for not linear optimization. In 5th INFORMS Optimization Society Conference. 6–8March. Google Scholar
- Robert Fourer, David M Gay, and Brian W Kernighan. 1987. AMPL: A mathematical programming language. AT & T Bell Laboratories Murray Hill, NJ. Google Scholar
- Jonathan Frankle, Peter-Michael Osera, David Walker, and Steve Zdancewic. 2016. Example-directed synthesis: a type-theoretic interpretation. ACM Sigplan Notices, 51, 1 (2016), 802–815. https://doi.org/10.1145/2914770.2837629 Google ScholarDigital Library
- Fred Glover. 1975. Improved linear integer programming formulations of nonlinear integer problems. Management science, 22, 4 (1975), 455–460. https://doi.org/10.1287/mnsc.22.4.455 Google ScholarDigital Library
- Zheng Guo, Michael James, David Justo, Jiaxiao Zhou, Ziteng Wang, Ranjit Jhala, and Nadia Polikarpova. 2019. Program synthesis by type-guided abstraction refinement. Proceedings of the ACM on Programming Languages, 4, POPL (2019), 1–28. https://doi.org/10.1145/3371080 Google ScholarDigital Library
- LLC Gurobi Optimization. 2018. Gurobi optimizer reference manual. Google Scholar
- Qinheping Hu, Jason Breck, John Cyphert, Loris D’Antoni, and Thomas Reps. 2019. Proving unrealizability for syntax-guided synthesis. In International Conference on Computer Aided Verification. 335–352. https://doi.org/10.1007/978-3-030-25540-4_18 Google ScholarCross Ref
- Qinheping Hu, John Cyphert, Loris D’Antoni, and Thomas Reps. 2020. Exact and approximate methods for proving unrealizability of syntax-guided synthesis problems. In ACM SIGPLAN Conference on Programming Language Design and Implementation. 1128–1142. https://doi.org/10.1145/3385412.3385979 Google ScholarDigital Library
- Tim King, Clark Barrett, and Cesare Tinelli. 2014. Leveraging linear and mixed integer programming for SMT. In 2014 Formal Methods in Computer-Aided Design (FMCAD). 139–146. https://doi.org/10.1109/FMCAD.2014.6987606 Google ScholarCross Ref
- Ron Kohavi. 1996. Scaling up the accuracy of naive-bayes classifiers: A decision-tree hybrid.. In Kdd. 96, 202–207. Google ScholarDigital Library
- Woosuk Lee. 2021. Combining the top-down propagation and bottom-up enumeration for inductive program synthesis. Proceedings of the ACM on Programming Languages, 5, POPL (2021), 1–28. https://doi.org/10.1145/3434335 Google ScholarDigital Library
- Clifford Liem, Yuan Xiang Gu, and Harold Johnson. 2008. A compiler-based infrastructure for software-protection. In ACM SIGPLAN Workshop on Programming Languages and Analysis for Security. 33–44. https://doi.org/10.1145/1375696.1375702 Google ScholarDigital Library
- Binbin Liu, Weijie Feng, Qilong Zheng, Jing Li, and Dongpeng Xu. 2021. Software Obfuscation with Non-Linear Mixed Boolean-Arithmetic Expressions. In International Conference on Information and Communications Security. 276–292. https://doi.org/10.1007/978-3-030-86890-1_16 Google ScholarDigital Library
- Pierre Mahe, Maud Arsac, Sonia Chatellier, Valérie Monnin, Nadine Perrot, Sandrine Mailler, Victoria Girard, Mahendrasingh Ramjeet, Jérémy Surre, and Bruno Lacroix. 2014. Automatic identification of mixed bacterial species fingerprints in a MALDI-TOF mass-spectrum. Bioinformatics, 30, 9 (2014), 1280–1286. Google ScholarCross Ref
- James McDermott and Richard S Forsyth. 2016. Diagnosing a disorder in a classification benchmark. Pattern Recognition Letters, 73 (2016), 41–43. https://doi.org/10.1016/j.patrec.2016.01.004 Google ScholarDigital Library
- Leonardo de Moura and Nikolaj Bjørner. 2008. Z3: An efficient SMT solver. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems. 337–340. https://doi.org/10.1007/978-3-540-78800-3_24 Google ScholarCross Ref
- Pierluigi Nuzzo, Alberto Puggelli, Sanjit A Seshia, and Alberto Sangiovanni-Vincentelli. 2010. CalCS: SMT solving for non-linear convex constraints. In Formal Methods in Computer Aided Design. 71–79. Google Scholar
- Peter-Michael Osera and Steve Zdancewic. 2015. Type-and-example-directed program synthesis. ACM SIGPLAN Notices, 50, 6 (2015), 619–630. https://doi.org/10.1145/2737924.2738007 Google ScholarDigital Library
- Brandon Paulsen and Chao Wang. 2022. Example Guided Synthesis of Linear Approximations for Neural Network Verification. In International Conference on Computer Aided Verification. Springer, 149–170. https://doi.org/10.1007/978-3-031-13185-1_8 Google ScholarDigital Library
- Brandon Paulsen and Chao Wang. 2022. LinSyn: Synthesizing Tight Linear Bounds for Arbitrary Neural Network Activation Functions. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems. Springer, 357–376. https://doi.org/10.1007/978-3-030-99524-9_19 Google ScholarDigital Library
- Nadia Polikarpova, Ivan Kuraj, and Armando Solar-Lezama. 2016. Program synthesis from polymorphic refinement types. ACM SIGPLAN Notices, 51, 6 (2016), 522–538. https://doi.org/10.1145/2980983.2908093 Google ScholarDigital Library
- Andrew Reynolds, Haniel Barbosa, Andres Nötzli, Clark Barrett, and Cesare Tinelli. 2019. cvc 4 sy: smart and fast term enumeration for syntax-guided synthesis. In International Conference on Computer Aided Verification. 74–83. https://doi.org/10.1007/978-3-030-25543-5_5 Google ScholarCross Ref
- Junfu Shen and Jiang Ming. 2021. Mba-blast: unveiling and simplifying mixed boolean-arithmetic obfuscation. Google Scholar
- Yasser Shoukry, Pierluigi Nuzzo, Alberto L Sangiovanni-Vincentelli, Sanjit A Seshia, George J Pappas, and Paulo Tabuada. 2018. SMC: Satisfiability modulo convex programming. Proc. IEEE, 106, 9 (2018), 1655–1679. https://doi.org/10.1109/JPROC.2018.2849003 Google ScholarCross Ref
- Emina Torlak and Rastislav Bodik. 2013. Growing solver-aided languages with Rosette. In Proceedings of the 2013 ACM international Symposium on New Ideas, New Paradigms, and Reflections on Programming & Software. 135–152. https://doi.org/10.1145/2509578.2509586 Google ScholarDigital Library
- Jingbo Wang, Yannan Li, and Chao Wang. 2022. Synthesizing Fair Decision Trees via Iterative Constraint Solving. In International Conference on Computer Aided Verification. 364–385. https://doi.org/10.1007/978-3-031-13188-2_18 Google ScholarDigital Library
- Jingbo Wang, Chungha Sung, Mukund Raghothaman, and Chao Wang. 2021. Data-Driven Synthesis of Provably Sound Side Channel Analyses. In IEEE/ACM International Conference on Software Engineering. IEEE, 810–822. https://doi.org/10.1109/ICSE43902.2021.00079 Google ScholarDigital Library
- Xinyu Wang, Isil Dillig, and Rishabh Singh. 2017. Program synthesis using abstraction refinement. Proceedings of the ACM on Programming Languages, 2, POPL (2017), 1–30. https://doi.org/10.1145/3158151 Google ScholarDigital Library
- H Paul Williams. 2013. Model building in mathematical programming. John Wiley & Sons. Google Scholar
- Yongxin Zhou, Alec Main, Yuan X Gu, and Harold Johnson. 2007. Information hiding in software with mixed boolean-arithmetic transforms. In International Workshop on Information Security Applications. 61–75. https://doi.org/10.1007/978-3-540-77535-5_5 Google ScholarCross Ref
Index Terms
- Synthesizing MILP Constraints for Efficient and Robust Optimization
Recommendations
Efficient robust optimization for robust control with constraints
This paper proposes an efficient computational technique for the optimal control of linear discrete-time systems subject to bounded disturbances with mixed linear constraints on the states and inputs. The problem of computing an optimal state feedback ...
Global optimization of multi-parametric MILP problems
In this paper, we present a novel global optimisation approach for the general solution of multi-parametric mixed integer linear programs (mp-MILPs). We describe an optimisation procedure which iterates between a (master) mixed integer nonlinear program ...
Robust truss topology optimization via semidefinite programming with complementarity constraints: a difference-of-convex programming approach
The robust truss topology optimization against the uncertain static external load can be formulated as mixed-integer semidefinite programming. Although a global optimal solution can be computed with a branch-and-bound method, it is very time-consuming. ...
Comments