Zhihui Yang
Skip Abstract Section
Abstract
We consider accelerating machine learning (ML) inference queries on unstructured datasets. Expensive operators such as feature extractors and classifiers are deployed as user-defined functions (UDFs), which are not penetrable with classic query optimization techniques such as predicate push-down. Recent optimization schemes (e.g., Probabilistic Predicates or PP) assume independence among the query predicates, build a proxy model for each predicate offline, and rewrite a new query by injecting these cheap proxy models in the front of the expensive ML UDFs. In such a manner, unlikely inputs that do not satisfy query predicates are filtered early to bypass the ML UDFs. We show that enforcing the independence assumption in this context may result in sub-optimal plans. In this paper, we propose CORE, a query optimizer that better exploits the predicate correlations and accelerates ML inference queries. Our solution builds the proxy models online for a new query and leverages a branch-and-bound search process to reduce the building costs. Results on three real-world text, image and video datasets show that CORE improves the query throughput by up to 63% compared to PP and up to 80% compared to running the queries as it is.
- Pankaj K Agarwal, Sariel Har-Peled, and Kasturi R Varadarajan. 2005. Geometric approximation via coresets. Combinatorial and computational geometry 52 (2005), 1--30.Google Scholar
- Shivnath Babu, Rajeev Motwani, Kamesh Munagala, Itaru Nishizawa, and Jennifer Widom. 2004. Adaptive Ordering of Pipelined Stream Filters. In Proceedings of the ACM SIGMOD International Conference on Management of Data, June 13-18, 2004. ACM, Paris, France, 407--418.Google ScholarDigital Library
- Shaofeng Cai, Gang Chen, Beng Chin Ooi, and Jinyang Gao. 2019. Model slicing for supporting complex analytics with elastic inference cost and resource constraints. Proceedings of the VLDB Endowment 13, 2 (2019), 86--99.Google ScholarDigital Library
- Zhaowei Cai, Mohammad J. Saberian, and Nuno Vasconcelos. 2015. Learning Complexity-Aware Cascades for Deep Pedestrian Detection. In 2015 IEEE International Conference on Computer Vision, ICCV 2015, December 7-13, 2015. IEEE Computer Society, Santiago, Chile, 3361--3369.Google Scholar
- Surajit Chaudhuri, Bolin Ding, and Srikanth Kandula. 2017. Approximate Query Processing: No Silver Bullet. In Proceedings of the 2017 ACM International Conference on Management of Data, SIGMOD Conference 2017, May 14-19, 2017. ACM, Chicago, IL, USA, 511--519.Google ScholarDigital Library
- Surajit Chaudhuri and Kyuseok Shim. 1999. Optimization of Queries with User-Defined Predicates. ACM Trans. Database Syst. 24, 2 (1999), 177--228.Google ScholarDigital Library
- Xianshun Chen. 2020. Activity Recognition. https://github.com/chen0040/keras-video-classifier. last accessed: 2020-01-22.Google Scholar
- Izrail Solomonovich Gradshteyn and Iosif Moiseevich Ryzhik. 2014. Table of integrals, series, and products. Academic press, Cambridge, MA.Google Scholar
- Sona Hasani, Saravanan Thirumuruganathan, Abolfazl Asudeh, Nick Koudas, and Gautam Das. 2018. Efficient construction of approximate ad-hoc ML models through materialization and reuse. Proceedings of the VLDB Endowment 11, 11 (2018), 1468--1481.Google ScholarDigital Library
- Joseph M. Hellerstein and Michael Stonebraker. 1993. Predicate Migration: Optimizing Queries with Expensive Predicates. In Proceedings of the 1993 ACM SIGMOD International Conference on Management of Data, May 26-28, 1993. ACM Press, Washington, DC, USA, 267--276.Google Scholar
- Benjamin Hilprecht, Andreas Schmidt, Moritz Kulessa, Alejandro Molina, Kristian Kersting, and Carsten Binnig. 2020. DeepDB: Learn from Data, not from Queries! Proc. VLDB Endow. 13, 7 (2020), 992--1005.Google ScholarDigital Library
- Kevin Hsieh, Ganesh Ananthanarayanan, Peter Bodík, Shivaram Venkataraman, Paramvir Bahl, Matthai Philipose, Phillip B. Gibbons, and Onur Mutlu. 2018. Focus: Querying Large Video Datasets with Low Latency and Low Cost. In 13th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2018, October 8-10, 2018. USENIX Association, Carlsbad, CA, USA, 269--286.Google Scholar
- Nacim Ihaddadene and Chabane Djeraba. 2008. Real-time crowd motion analysis. In 19th International Conference on Pattern Recognition (ICPR 2008), December 8-11, 2008. IEEE Computer Society, Tampa, Florida, USA, 1--4.Google ScholarCross Ref
- Ihab F. Ilyas, Volker Markl, Peter J. Haas, Paul Brown, and Ashraf Aboulnaga. 2004. CORDS: Automatic Discovery of Correlations and Soft Functional Dependencies. In Proceedings of the ACM SIGMOD International Conference on Management of Data, June 13-18, 2004. ACM, Paris, France, 647--658.Google ScholarDigital Library
- Thorsten Joachims. 2006. Training linear SVMs in linear time. In Proceedings of the Twelfth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, August 20-23, 2006. ACM, Philadelphia, PA, USA, 217--226.Google ScholarDigital Library
- Daniel Kang, Peter Bailis, and Matei Zaharia. 2019. BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics. Proc. VLDB Endow. 13, 4 (2019), 533--546.Google ScholarDigital Library
- Daniel Kang, John Emmons, Firas Abuzaid, Peter Bailis, and Matei Zaharia. 2017. NoScope: Optimizing Deep CNN-Based Queries over Video Streams at Scale. PVLDB 10, 11 (2017), 1586--1597.Google ScholarDigital Library
- Daniel Kang, Edward Gan, Peter Bailis, Tatsunori Hashimoto, and Matei Zaharia. 2020. Approximate Selection with Guarantees using Proxies. Proc. VLDB Endow. 13, 11 (2020), 1990--2003.Google ScholarDigital Library
- Daniel Kang, John Guibas, Peter Bailis, Tatsunori Hashimoto, and Matei Zaharia. 2020. Task-agnostic Indexes for Deep Learning-based Queries over Unstructured Data. CoRR abs/2009.04540 (2020).Google Scholar
- Walter H Kohler and Kenneth Steiglitz. 1974. Characterization and theoretical comparison of branch-and-bound algorithms for permutation problems. Journal of the ACM (JACM) 21, 1 (1974), 140--156.Google ScholarDigital Library
- Sanjay Krishnan, Adam Dziedzic, and Aaron J. Elmore. 2019. DeepLens: Towards a Visual Data Management System. In 9th Biennial Conference on Innovative Data Systems Research, CIDR 2019, January 13-16, 2019, Online Proceedings. www.cidrdb.org, Asilomar, CA, USA.Google Scholar
- Andreas Kunft, Asterios Katsifodimos, Sebastian Schelter, Sebastian Breß, Tilmann Rabl, and Volker Markl. 2019. An intermediate representation for optimizing machine learning pipelines. Proceedings of the VLDB Endowment 12, 11 (2019), 1553--1567.Google ScholarDigital Library
- Iosif Lazaridis and Sharad Mehrotra. 2007. Optimization of multi-version expensive predicates. In Proceedings of the ACM SIGMOD International Conference on Management of Data, June 12-14, 2007. ACM, Beijing, China, 797--808.Google ScholarDigital Library
- Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature 521, 7553 (2015), 436--444.Google Scholar
- Yann LeCun, Bernhard E. Boser, John S. Denker, Donnie Henderson, Richard E. Howard, Wayne E. Hubbard, and Lawrence D. Jackel. 1989. Handwritten Digit Recognition with a Back-Propagation Network. In Advances in Neural Information Processing Systems 2, [NIPS Conference, November 27-30, 1989]. Morgan Kaufmann, Denver, Colorado, USA, 396--404.Google Scholar
- Haoxiang Li, Zhe Lin, Xiaohui Shen, Jonathan Brandt, and Gang Hua. 2015. A convolutional neural network cascade for face detection. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, June 7-12, 2015. IEEE Computer Society, Boston, MA, USA, 5325--5334.Google ScholarCross Ref
- Xiaoxiao Li, Ziwei Liu, Ping Luo, Chen Change Loy, and Xiaoou Tang. 2017. Not All Pixels Are Equal: Difficulty-Aware Semantic Segmentation via Deep Layer Cascade. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, July 21-26, 2017. IEEE Computer Society, Honolulu, HI, USA, 6459--6468.Google ScholarCross Ref
- Tsung-Yi Lin, Michael Maire, Serge J. Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. 2014. Microsoft COCO: Common Objects in Context. In Computer Vision - ECCV 2014 - 13th European Conference, September 6-12, 2014, Proceedings, Part V (Lecture Notes in Computer Science), Vol. 8693. Springer, Zurich, Switzerland, 740--755.Google Scholar
- John DC Little, Katta G Murty, Dura W Sweeney, and Caroline Karel. 1963. An algorithm for the traveling salesman problem. Operations research 11, 6 (1963), 972--989.Google Scholar
- Yao Lu, Aakanksha Chowdhery, Srikanth Kandula, and Surajit Chaudhuri. 2018. Accelerating Machine Learning Inference with Probabilistic Predicates. In Proceedings of the 2018 International Conference on Management of Data, SIGMOD Conference 2018, June 10-15, 2018. ACM, Houston, TX, USA, 1493--1508.Google ScholarDigital Library
- Christopher D. Manning, Mihai Surdeanu, John Bauer, Jenny Rose Finkel, Steven Bethard, and David McClosky. 2014. The Stanford CoreNLP Natural Language Processing Toolkit. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014, June 22-27, 2014, System Demonstrations. The Association for Computer Linguistics, Baltimore, MD, USA, 55--60.Google ScholarCross Ref
- Venkatesh N. Murthy, Vivek Singh, Terrence Chen, R. Manmatha, and Dorin Comaniciu. 2016. Deep Decision Network for Multi-class Image Classification. In 2016 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2016, June 27-30, 2016. IEEE Computer Society, Las Vegas, NV, USA, 2240--2248.Google Scholar
- Joseph Redmon and Ali Farhadi. 2018. YOLOv3: An Incremental Improvement. CoRR abs/1804.02767 (2018).Google Scholar
- Astrid Rheinländer, Ulf Leser, and Goetz Graefe. 2017. Optimization of Complex Dataflows with User-Defined Functions. ACM Comput. Surv. 50, 3 (2017), 38:1--38:39.Google Scholar
- Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. 2012. UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild. CoRR abs/1212.0402 (2012).Google Scholar
- Alexander Toshev and Christian Szegedy. 2014. DeepPose: Human Pose Estimation via Deep Neural Networks. In 2014 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2014, June 23-28, 2014. IEEE Computer Society, Columbus, OH, USA, 1653--1660.Google ScholarDigital Library
- Twitter API 2019. Twitter API. https://developer.twitter.com/en/docs/twitter-api. last accessed: 2019-01-01.Google Scholar
- Paul A. Viola and Michael J. Jones. 2001. Rapid Object Detection using a Boosted Cascade of Simple Features. In 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2001), with CD-ROM, 8-14 December 2001. IEEE Computer Society, Kauai, HI, USA, 511--518.Google Scholar
- Wei Wang, Jinyang Gao, Meihui Zhang, Sheng Wang, Gang Chen, Teck Khim Ng, Beng Chin Ooi, Jie Shao, and Moaz Reyad. 2018. Rafiki: machine learning as an analytics service system. Proceedings of the VLDB Endowment 12, 2 (2018), 128--140.Google ScholarDigital Library
- Xin Wang, Yujia Luo, Daniel Crankshaw, Alexey Tumanov, Fisher Yu, and Joseph E. Gonzalez. 2018. IDK Cascades: Fast Deep Learning by Learning not to Overthink. In Proceedings of the Thirty-Fourth Conference on Uncertainty in Artificial Intelligence, UAI 2018, August 6-10, 2018. AUAI Press, Monterey, California, USA, 580--590.Google Scholar
- Zhihui Yang, Zuozhi Wang, Yicong Huang, Yao Lu, Chen Li, and X. Sean Wang. 2022. Correlative Proxy Models. https://github.com/ZhihuiYangCS/CorrProxies/wiki/Queries-and-Datasets. last accessed: 2022-02-22.Google Scholar
- Zhihui Yang, Zuozhi Wang, Yicong Huang, Yao Lu, Chen Li, and X. Sean Wang. 2022. Optimizing Machine Learning Inference Queries with Correlative Proxy Models (Technical Report). http://texera.ics.uci.edu/pdf/proxymodel/proxymodel-tech-report.pdf. last accessed: 2022-06-09.Google Scholar
Index Terms
- Optimizing machine learning inference queries with correlative proxy models
Index terms have been assigned to the content through auto-classification.
Recommendations
Demonstration of accelerating machine learning inference queries with correlative proxy models
We will demonstrate a prototype query-processing engine, which utilizes correlations among predicates to accelerate machine learning (ML) inference queries on unstructured data. Expensive operators such as feature extractors and classifiers are deployed ...
Optimizing XML queries: Bitmapped materialized views vs. indexes
Optimizing queries using materialized views has not been addressed adequately in the context of XML due to the many limitations associated with the definition and usability of materialized views in traditional XML query evaluation models. In this paper, ...
Optimizing query rewriting for multiple queries
IIWeb '12: Proceedings of the Ninth International Workshop on Information Integration on the WebWe present an scalable algorithm for answering multiple conjunctive queries using views. This is an important problem in query optimization, data integration and ontology-based data access. Since rewriting one conjunctive query using views is an NP-hard ...
Comments