Dixin Tang
Alan Fekete
Abstract
Many tools empower analysts and data scientists to consume analysis results in a visual interface. When the underlying data changes, these results need to be updated, but this update can take a long time---all while the user continues to explore the results. Tools can either (i) hide away results that haven't been updated, hindering exploration; (ii) make the updated results immediately available to the user (on the same screen as old results), leading to confusion and incorrect insights; or (iii) present old---and therefore stale---results to the user during the update. To help users reason about these options and others, and make appropriate trade-offs, we introduce Transactional Panorama, a formal framework that adopts transactions to jointly model the system refreshing the analysis results and the user interacting with them. We introduce three key properties that are important for user perception in this context: visibility (allowing users to continuously explore results), consistency (ensuring that results presented are from the same version of the data), and monotonicity (making sure that results don't "go back in time"). Within transactional panorama, we characterize all feasible property combinations, design new mechanisms (that we call lenses) for presenting analysis results to the user while preserving a given property combination, formally prove their relative orderings for various performance criteria, and discuss their use cases. We propose novel algorithms to preserve each property combination and efficiently present fresh analysis results. We implement our framework into a popular, open-source BI tool, illustrate the relative performance implications of different lenses, and demonstrate the benefits of the novel lenses and our optimizations.
- Datadog. https://www.datadoghq.com/.Google Scholar
- Google sheets. https://www.google.com/sheets/about/.Google Scholar
- Grafana. https://en.wikipedia.org/wiki/Grafana.Google Scholar
- Kibana. https://en.wikipedia.org/wiki/Kibana.Google Scholar
- Libreoffice calc. https://www.libreoffice.org/discover/calc/.Google Scholar
- Microsoft excel. http://products.office.com/en-us/excel.Google Scholar
- Plotly. https://plotly.com/.Google Scholar
- Power bi. https://powerbi.microsoft.com/en-us/.Google Scholar
- Redash. https://redash.io/.Google Scholar
- Streamlit. https://streamlit.io/.Google Scholar
- Superset. https://superset.apache.org/.Google Scholar
- Tableau. https://www.tableau.com/.Google Scholar
- Transactional panorama: A conceptual framework for user perception in analytical visual interfaces (technical report). https://arxiv.org/abs/2302.05476.Google Scholar
- Y. Ahmad, O. Kennedy, C. Koch, and M. Nikolic. DBToaster: Higher-order delta processing for dynamic, frequently fresh views. Proc. VLDB Endow., 5(10):968--979, 2012.Google ScholarDigital Library
- S. B. Ahsan, R. Yang, S. A. Noghabi, and I. Gupta. Home, safehome: smart home reliability with visibility and atomicity. In A. Barbalace, P. Bhatotia, L. Alvisi, and C. Cadar, editors, EuroSys '21: Sixteenth European Conference on Computer Systems, Online Event, United Kingdom, April 26-28, 2021, pages 590--605. ACM, 2021.Google ScholarDigital Library
- M. Bendre, T. Wattanawaroon, K. Mack, K. Chang, and A. G. Parameswaran. Antifreeze for large and complex spreadsheets: Asynchronous formula computation. In P. A. Boncz, S. Manegold, A. Ailamaki, A. Deshpande, and T. Kraska, editors, Proceedings of the 2019 International Conference on Management of Data, SIGMOD Conference 2019, Amsterdam, The Netherlands, June 30 - July 5, 2019, pages 1277--1294. ACM, 2019.Google Scholar
- P. A. Bernstein, V. Hadzilacos, and N. Goodman. Concurrency Control and Recovery in Database Systems. Addison-Wesley, 1987.Google ScholarDigital Library
- I. Botan, P. M. Fischer, D. Kossmann, and N. Tatbul. Transactional stream processing. In E. A. Rundensteiner, V. Markl, I. Manolescu, S. Amer-Yahia, F. Naumann, and I. Ari, editors, 15th International Conference on Extending Database Technology, EDBT '12, Berlin, Germany, March 27-30, 2012, Proceedings, pages 204--215. ACM, 2012.Google Scholar
- M. J. Cahill, U. Röhm, and A. D. Fekete. Serializable isolation for snapshot databases. ACM Trans. Database Syst., 34(4):20:1--20:42, 2009.Google ScholarDigital Library
- Y. Chen, X. Yu, P. Koutris, A. C. Arpaci-Dusseau, R. H. Arpaci-Dusseau, and J. Shu. Plor: General transactions with predictable, low tail latency. In Z. Ives, A. Bonifati, and A. E. Abbadi, editors, SIGMOD '22: International Conference on Management of Data, Philadelphia, PA, USA, June 12-17, 2022, pages 19--33. ACM, 2022.Google Scholar
- R. Chirkova and J. Yang. Materialized views. Foundations and Trends in Databases, 4(4):295--405, 2012.Google ScholarCross Ref
- L. S. Colby, T. Griffin, L. Libkin, I. S. Mumick, and H. Trickey. Algorithms for deferred view maintenance. In Proceedings of the 1996 ACM SIGMOD International Conference on Management of Data, Montreal, Quebec, Canada, June 4-6, 1996, pages 469--480, 1996.Google ScholarDigital Library
- L. S. Colby, A. Kawaguchi, D. F. Lieuwen, I. S. Mumick, and K. A. Ross. Supporting multiple view maintenance policies. In SIGMOD 1997, Proceedings ACM SIGMOD International Conference on Management of Data, May 13-15, 1997, Tucson, Arizona, USA., pages 405--416, 1997.Google ScholarDigital Library
- L. Golab and T. Johnson. Consistency in a stream warehouse. In Fifth Biennial Conference on Innovative Data Systems Research, CIDR 2011, Asilomar, CA, USA, January 9-12, 2011, Online Proceedings, pages 114--122. www.cidrdb.org, 2011.Google Scholar
- A. Gupta, I. S. Mumick, and V. S. Subrahmanian. Maintaining views incrementally. In Proceedings of the 1993 ACM SIGMOD International Conference on Management of Data, Washington, DC, USA, May 26-28, 1993, pages 157--166, 1993.Google ScholarDigital Library
- H. Lu, C. Hodsdon, K. Ngo, S. Mu, and W. Lloyd. The SNOW theorem and latency-optimal read-only transactions. In K. Keeton and T. Roscoe, editors, 12th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2016, Savannah, GA, USA, November 2-4, 2016, pages 135--150. USENIX Association, 2016.Google Scholar
- J. Meehan, N. Tatbul, S. Zdonik, C. Aslantas, U. Çetintemel, J. Du, T. Kraska, S. Madden, D. Maier, A. Pavlo, M. Stonebraker, K. Tufte, and H. Wang. S-store: Streaming meets transaction processing. Proc. VLDB Endow., 8(13):2134--2145, 2015.Google ScholarDigital Library
- D. Moritz, D. Fisher, B. Ding, and C. Wang. Trust, but verify: Optimistic visualizations of approximate queries for exploring big data. In G. Mark, S. R. Fussell, C. Lampe, m. c. schraefel, J. P. Hourcade, C. Appert, and D. Wigdor, editors, Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA, May 06-11, 2017, pages 2904--2915. ACM, 2017.Google ScholarDigital Library
- T. Neumann, T. Mühlbauer, and A. Kemper. Fast serializable multi-version concurrency control for main-memory database systems. In T. K. Sellis, S. B. Davidson, and Z. G. Ives, editors, Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, Melbourne, Victoria, Australia, May 31 - June 4, 2015, pages 677--689. ACM, 2015.Google ScholarDigital Library
- M. Nikolic, M. Dashti, and C. Koch. How to win a hot dog eating contest: Distributed incremental view maintenance with batch updates. In Proceedings of the 2016 International Conference on Management of Data, SIGMOD Conference 2016, San Francisco, CA, USA, June 26 - July 01, 2016, pages 511--526, 2016.Google ScholarDigital Library
- S. Rahman, M. Aliakbarpour, H. K. Kong, E. Blais, K. Karahalios, A. Parameswaran, and R. Rubinfield. I've seen "enough" incrementally improving visualizations to support rapid decision making. Proceedings of the VLDB Endowment, 10(11):1262--1273, 2017.Google ScholarDigital Library
- V. Raman, B. Raman, and J. M. Hellerstein. Online dynamic reordering for interactive data processing. In VLDB, volume 99, pages 709--720, 1999.Google Scholar
- A. S. Tanenbaum and M. van Steen. Distributed systems - principles and paradigms, 2nd Edition. Pearson Education, 2007.Google Scholar
- D. Tang, H. Jiang, and A. J. Elmore. Adaptive concurrency control: Despite the looking glass, one concurrency control does not fit all. In 8th Biennial Conference on Innovative Data Systems Research, CIDR 2017, Chaminade, CA, USA, January 8-11, 2017, Online Proceedings. www.cidrdb.org, 2017.Google Scholar
- D. Tang, Z. Shang, A. J. Elmore, S. Krishnan, and M. J. Franklin. Thrifty query execution via incrementability. In D. Maier, R. Pottinger, A. Doan, W. Tan, A. Alawini, and H. Q. Ngo, editors, Proceedings of the 2020 International Conference on Management of Data, SIGMOD Conference 2020, online conference [Portland, OR, USA], June 14-19, 2020, pages 1241--1256. ACM, 2020.Google Scholar
- T. Wang, R. Johnson, A. D. Fekete, and I. Pandis. Efficiently making (almost) any concurrency control mechanism serializable. VLDB J., 26(4):537--562, 2017.Google ScholarDigital Library
- Y. Wu, R. Chang, J. M. Hellerstein, and E. Wu. Facilitating exploration with interaction snapshots under high latency. In 31st IEEE Visualization Conference, IEEE VIS 2020 - Short Papers, Virtual Event, USA, October 25-30, 2020, pages 136--140. IEEE, 2020.Google ScholarCross Ref
- X. Yu, G. Bezerra, A. Pavlo, S. Devadas, and M. Stonebraker. Staring into the abyss: An evaluation of concurrency control with one thousand cores. Proc. VLDB Endow., 8(3):209--220, 2014.Google ScholarDigital Library
- E. Zgraggen, A. Galakatos, A. Crotty, J.-D. Fekete, and T. Kraska. How progressive visualizations affect exploratory analysis. IEEE transactions on visualization and computer graphics, 23(8):1977--1987, 2016.Google Scholar
- J. Zhang, K. Huang, T. Wang, and K. Lv. Skeena: Efficient and consistent cross-engine transactions. In Z. Ives, A. Bonifati, and A. E. Abbadi, editors, SIGMOD '22: International Conference on Management of Data, Philadelphia, PA, USA, June 12 - 17, 2022, pages 34--48. ACM, 2022.Google Scholar
- Z. Zhao, F. Li, and Y. Liu. Efficient join synopsis maintenance for data warehouse. In D. Maier, R. Pottinger, A. Doan, W. Tan, A. Alawini, and H. Q. Ngo, editors, Proceedings of the 2020 International Conference on Management of Data, SIGMOD Conference 2020, online conference [Portland, OR, USA], June 14-19, 2020, pages 2027--2042. ACM, 2020.Google Scholar
- J. Zhou, P. Larson, and H. G. Elmongui. Lazy maintenance of materialized views. In Proceedings of the 33rd International Conference on Very Large Data Bases, University of Vienna, Austria, September 23-27, 2007, pages 231--242, 2007.Google ScholarDigital Library
- Y. Zhuge, H. Garcia-Molina, and J. L. Wiener. Multiple view consistency for data warehousing. In W. A. Gray and P. Larson, editors, Proceedings of the Thirteenth International Conference on Data Engineering, April 7-11, 1997, Birmingham, UK, pages 289--300. IEEE Computer Society, 1997.Google Scholar
Recommendations
User preferences for adaptive user interfaces in health information systems
An adaptive user interface requires identification of user requirements. Interface designers and engineers must understand end-user interaction with the system to improve user interface design. A combination of interviews and observations is applied for ...
Interaction analysis for adaptive user interfaces
ICIC'10: Proceedings of the 6th international conference on Advanced intelligent computing theories and applications: intelligent computingAdaptive User Interfaces are able to facilitate the handling of computer systems through the automatic adaptation to users' needs and preferences. For the realization of these systems, information about the individual user is needed. This user ...
A framework for WWW user activity analysis based on user interest
User activity plays an important role in the operation of many kinds of websites. Significant progress has been made on user activity modeling by researchers in many kinds of Web information process fields, such as Web log mining, blogs social network ...
Comments