Abstract
Fault localization, aiming at localizing the root cause of the bug under repair, has been a longstanding research topic. Although many approaches have been proposed in past decades, most of the existing studies work at coarse-grained statement or method levels with very limited insights about how to repair the bug (granularity problem), but few studies target the finer-grained fault localization. In this article, we target the granularity problem and propose a novel finer-grained variable-level fault localization technique. Specifically, the basic idea of our approach is that fault-relevant variables may exhibit different values in failed and passed test runs, and variables that have higher discrimination ability have a larger possibility to be the root causes of the failure. Based on this, we propose a program-dependency-enhanced decision tree model to boost the identification of fault-relevant variables via discriminating failed and passed test cases based on the variable values. To evaluate the effectiveness of our approach, we have implemented it in a tool called VarDT and conducted an extensive study over the Defects4J benchmark. The results show that VarDT outperforms the state-of-the-art fault localization approaches with at least 268.4% improvement in terms of bugs located at Top-1, and the average improvement is 351.3%.
Besides, to investigate whether our finer-grained fault localization result can further improve the effectiveness of downstream APR techniques, we have adapted VarDT to the application of patch filtering, where we use the variables located by VarDT to filter incorrect patches. The results denote that VarDT outperforms the state-of-the-art PATCH-SIM and BATS by filtering 14.8% and 181.8% more incorrect patches, respectively, demonstrating the effectiveness of our approach. It also provides a new way of thinking for improving automatic program repair techniques.
- [1] . 2006. An evaluation of similarity coefficients for software fault localization. In Pacific Rim International Symposium on Dependable Computing. 39–46.
DOI: Google ScholarDigital Library - [2] . 2007. On the accuracy of spectrum-based fault localization. In Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION (TAICPART-MUTATION’07). 89–98.Google ScholarDigital Library
- [3] . 1990. Dynamic program slicing. In Programming Language Design and Implementation. Springer, 246–256.
DOI: Google ScholarDigital Library - [4] . 2007. Statistical debugging using compound boolean predicates. In International Symposium on Software Testing and Analysis. 5–15.
DOI: Google ScholarDigital Library - [5] . 2010. Adaptive bug isolation. In International Conference on Software Engineering. 255–264.
DOI: Google ScholarDigital Library - [6] . 2016. A learning-to-rank based fault localization approach using likely invariants. In International Symposium on Software Testing and Analysis. 177–188.Google Scholar
- [7] . 2010. Causal inference for statistical fault localization. In International Symposium on Software Testing and Analysis. 12 pages.
DOI: Google ScholarDigital Library - [8] . 2015. NUMFL: Localizing faults in numerical software using a value-based causal model. In International Conference on Software Testing, Verification and Validation. 1–10.
DOI: Google ScholarCross Ref - [9] . 1979. An efficient way to find the side effects of procedure calls and the aliases of variables. In ACM SIGPLAN Symposium on Principles of Programming Languages. 29–41.Google Scholar
- [10] . 2009. Pearson correlation coefficient. In Noise Reduction in Speech Processing. Springer, 1–4.Google Scholar
- [11] . 1984. Classification and Regression Trees. Taylor & Francis.Google Scholar
- [12] . 2007. Using machine learning to support debugging with Tarantula. In IEEE International Symposium on Software Reliability Engineering. 10 pages.Google Scholar
- [13] . 2017. Common bug-fix patterns: A large-scale observational study. In ACM/IEEE International Symposium on Empirical Software Engineering and Measurement. 404–413.
DOI: Google ScholarDigital Library - [14] . 2011. Angelic debugging. In International Conference on Software Engineering. 10 pages.
DOI: Google ScholarDigital Library - [15] . 2022. Toward understanding deep learning framework bugs. ACM Trans. Softw. Eng. Methodol. 32, 6 (2022).Google Scholar
- [16] . 2009. HOLMES: Effective statistical debugging via efficient path profiling. In International Conference on Software Engineering. 34–44.
DOI: Google ScholarDigital Library - [17] . 1988. Interprocedural side-effect analysis in linear time. In Programming Language Design and Implementation. Springer, 57–66.
DOI: Google ScholarDigital Library - [18] . 2008. DySy: Dynamic symbolic execution for invariant inference. In International Conference on Software Engineering. 281–290.
DOI: Google ScholarDigital Library - [19] . 2019. Feature selection for text classification: A review. Multimed. Tools Applic. 78, 3 (2019), 3797–3816.Google ScholarDigital Library
- [20] . 1999. Dynamically discovering pointer-based program invariants. In International Conference on Software Engineering, Vol. 373.Google ScholarDigital Library
- [21] . 2020. AFL++: Combining incremental steps of fuzzing research. In USENIX Workshop on Offensive Technologies.Google Scholar
- [22] . 2011. EvoSuite: Automatic test suite generation for object-oriented software. In European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering. 416–419.Google ScholarDigital Library
- [23] . 1993. Dynamic impact analysis: A cost-effective technique to enforce error-propagation. In International Symposium on Software Testing and Analysis. 171–181.
DOI: Google ScholarDigital Library - [24] . 2011. Statistical debugging with elastic predicates. In International Conference on Automated Software Engineering. 492–495.
DOI: Google ScholarDigital Library - [25] . 2020. Novel XGBoost tuned machine learning model for software bug prediction. In International Conference on Intelligent Engineering and Management. 376–380.Google ScholarCross Ref
- [26] . 2002. A precise demand-driven definition-use chaining algorithm. In European Conference on Software Maintenance and Reengineering. 77–86.
DOI: Google ScholarCross Ref - [27] . 2022. A systematic literature review on benchmarks for evaluating debugging approaches. J. Syst. Softw. 192, C (2022), 17 pages.
DOI: Google ScholarDigital Library - [28] . 2008. Fault localization using value replacement. In International Symposium on Software Testing and Analysis. 12 pages.
DOI: Google ScholarDigital Library - [29] . 2019. Combining spectrum-based fault localization and statistical debugging: An empirical study. In International Conference on Automated Software Engineering. 502–514.
DOI: Google ScholarDigital Library - [30] . 2018. Shaping program repair space with existing patches and similar code. In International Symposium on Software Testing and Analysis.Google Scholar
- [31] . 2007. Context-aware statistical debugging: From bug predictors to faulty control flow paths. In International Conference on Automated Software Engineering. 184–193.
DOI: Google ScholarDigital Library - [32] . 2005. Empirical evaluation of the tarantula automatic fault-localization technique. In International Conference on Automated Software Engineering. 273–282.Google ScholarDigital Library
- [33] . 2002. Visualization of test information to assist fault localization. In International Conference on Software Engineering.Google Scholar
- [34] . 2014. Defects4J: A database of existing faults to enable controlled testing studies for Java programs. In International Symposium on Software Testing and Analysis. 437–440.Google Scholar
- [35] . 2019. VFL: Variable-based fault localization. Inf. Softw. Technol. 107 (2019), 179–191.
DOI: Google ScholarCross Ref - [36] . 2016. Practitioners’ expectations on automated fault localization. In International Symposium on Software Testing and Analysis. 165–176.
DOI: Google ScholarDigital Library - [37] . 2021. Improving fault localization by integrating value and predicate based causal inference techniques. In International Conference on Software Engineering. 12 pages.
DOI: Google ScholarDigital Library - [38] . 2017. S3: Syntax-and semantic-guided repair synthesis via programming by examples. In 11th Joint Meeting on Foundations of Software Engineering. 593–604.Google Scholar
- [39] . 2015. The ManyBugs and IntroClass benchmarks for automated repair of C programs. IEEE Trans. Softw. Eng. 41, 12 (2015), 1236–1256.
DOI: Google ScholarDigital Library - [40] . 2021. A large-scale study on API misuses in the wild. In International Conference on Software Testing, Verification and Validation. 241–252.
DOI: Google ScholarCross Ref - [41] . 2017. Transforming programs and tests in tandem for fault localization. ACM Program. Lang. 1, OOPLSA (2017), 92:1–92:30.
DOI: Google ScholarDigital Library - [42] . 2021. Fault localization with code coverage representation learning. In International Conference on Software Engineering. 661–673.
DOI: Google ScholarDigital Library - [43] . 2003. Bug isolation via remote program sampling. In Programming Language Design and Implementation. Springer, 141–154.
DOI: Google ScholarDigital Library - [44] . 2005. Scalable statistical bug isolation. In Programming Language Design and Implementation. Springer, 15–26.
DOI: Google ScholarDigital Library - [45] . 2006. Statistical debugging: A hypothesis testing-based approach. IEEE Trans. Softw. Eng. 32, 10 (2006), 831–848.
DOI: Google ScholarDigital Library - [46] . 2005. SOBER: Statistical model-based bug localization. In European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering. 286–295.
DOI: Google ScholarDigital Library - [47] . 2019. TBar: Revisiting template-based automated program repair. In International Symposium on Software Testing and Analysis. 12 pages. Google Scholar
- [48] . 2021. Boosting coverage-based fault localization via graph-based representation learning. In European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering. 13 pages.
DOI: Google ScholarDigital Library - [49] . 2014. Extended comprehensive study of association measures for fault localization. J. Softw.: Evolut. Process 26, 2 (2014), 172–219.Google ScholarDigital Library
- [50] . 2019. BEARS: An extensible Java bug benchmark for automatic program repair studies. In IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER’19). Retrieved from https://arxiv.org/abs/1901.06024Google ScholarCross Ref
- [51] . 2014. Ask the mutants: Mutating faulty programs for fault localization. In International Conference on Software Testing, Verification and Validation. 153–162.
DOI: Google ScholarDigital Library - [52] . 2015. Metallaxis-FL: Mutation-based fault localization. Softw. Test. Verif. Reliab. 25, 5–7 (
Aug. 2015), 605–628.DOI: Google ScholarDigital Library - [53] . 2017. Evaluating and improving fault localization. In International Conference on Software Engineering. 609–620.Google Scholar
- [54] . 2018. Leveraging qualitative reasoning to improve SFL. In International Joint Conference on Artificial Intelligence. 1935–1941. Google ScholarCross Ref
- [55] . 2015. An analysis of patch plausibility and correctness for generate-and-validate patch generation systems. In International Symposium on Software Testing and Analysis. 257–269.Google Scholar
- [56] . 2014. C4.5: Programs for Machine Learning. Springer.Google Scholar
- [57] . 2018. Bugs.jar: A large-scale, diverse dataset of real-world Java bugs. In International Conference on Mining Software Repositories (MSR’18). 10–13.Google ScholarDigital Library
- [58] . 2017. ELIXIR: Effective object oriented program repair. In International Conference on Automated Software Engineering. Retrieved from http://dl.acm.org/citation.cfm?id=3155562.3155643Google ScholarCross Ref
- [59] . 2021. A comprehensive study of deep learning compiler bugs. In European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering. 968–980.Google ScholarDigital Library
- [60] . 2021. Localizing vulnerabilities statistically from one exploit. In ACM Symposium on Information, Computer and Communications Security (ASIA CCS’21). 537–549.
DOI: Google ScholarDigital Library - [61] . 2015. Is the cure worse than the disease? Overfitting in automated program repair. In ACM SIGSOFT Symposium on the Foundations of Software Engineering. 532–543.Google Scholar
- [62] . 2017. FLUCCS: Using code and change metrics to improve fault localization. International Symposium on Software Testing and Analysis. 273–283.
DOI: Google ScholarDigital Library - [63] . 2016. A deeper look into bug fixes: Patterns, replacements, deletions, and additions. In Working Conference on Mining Software Repositories. 512–515.
DOI: Google ScholarDigital Library - [64] . 1994. Compiler detection of function call side effects. Informatica 18, 2 (1994), 219–227.Google Scholar
- [65] . 2016. Anti-patterns in search-based program repair. In ACM SIGSOFT Symposium on the Foundations of Software Engineering.
DOI: Google ScholarDigital Library - [66] . 2022. Predicting patch correctness based on the similarity of failing test cases. ACM Trans. Softw. Eng. Methodol. 31, 4, Article
77 (Aug. 2022), 30 pages.DOI: Google ScholarDigital Library - [67] . 2018. Differential performance debugging with discriminant regression trees. In AAAI Conference on Artificial Intelligence, Vol. 32.Google ScholarCross Ref
- [68] . 2010. Cohen’s Kappa coefficient as a performance measure for feature selection. In International Conference on Fuzzy Systems. 1–8.Google ScholarCross Ref
- [69] . 2020. Survey of dynamic analysis based program invariant synthesis techniques. J. Softw. 31, 6 (2020), 1681–1702.Google Scholar
- [70] . 2020. Automated patch correctness assessment: How far are we? In International Conference on Automated Software Engineering. 13 pages.
DOI: Google ScholarDigital Library - [71] . 2018. Context-aware patch generation for better automated program repair. In International Conference on Software Engineering.Google ScholarDigital Library
- [72] . 2022. Effective isolation of fault-correlated variables via statistical and mutation analysis. IEEE Trans. Softw. Eng. 49, 4 (2022), 1–16.
DOI: Google ScholarDigital Library - [73] . 2022. XAI4FL: Enhancing spectrum-based fault localization with explainable artificial intelligence. In International Conference on Program Comprehension. 499–510.
DOI: Google ScholarDigital Library - [74] . 2016. Data Mining, Fourth Edition: Practical Machine Learning Tools and Techniques (4th ed.). Google Scholar
- [75] . 2014. The DStar method for effective software fault localization. IEEE Trans. Reliab.1 (
Mar. 2014), 290–308.Google ScholarCross Ref - [76] . 2007. Wilcoxon signed-rank test. Wiley Encyc. Clinic. Trials (2007), 1–3. Google ScholarCross Ref
- [77] . 2017. Identifying test-suite-overfitted patches through test case generation. In International Symposium on Software Testing and Analysis. 11 pages.
DOI: Google ScholarDigital Library - [78] . 2018. Identifying patch correctness in test-based program repair. In International Conference on Software Engineering.Google ScholarDigital Library
- [79] . 2022. L2S: A framework for synthesizing the most probable program under a specification. ACM Trans. Softw. Eng. Methodol. 31, 3, Article
34 (2022), 45 pages.DOI: Google ScholarDigital Library - [80] . 2014. Learning to combine multiple ranking metrics for fault localization. In IEEE International Conference on Software Maintenance and Evolution. 191–200.Google ScholarDigital Library
- [81] . 2017. Better test cases for better automated program repair. In ACM SIGSOFT Symposium on the Foundations of Software Engineering. 831–841.
DOI: Google ScholarDigital Library - [82] . 2021. Automated classification of overfitting patches with statically extracted code features. IEEE Trans. Softw. Eng. 48, 8 (2021).
DOI: Google ScholarDigital Library - [83] . 2002. Isolating cause-effect chains from computer programs. ACM SIGSOFT Symposium on the Foundations of Software Engineering. 1–10.
DOI: Google ScholarDigital Library - [84] . 2002. Simplifying and isolating failure-inducing input. IEEE Trans. Softw. Eng. 28, 2 (2002), 183–200.
DOI: Google ScholarDigital Library - [85] . 2022. Fault localization via efficient probabilistic modeling of program semantics. In International Conference on Software Engineering. 12 pages.
DOI: Google ScholarDigital Library - [86] . 2021. A partial-lifting-based compiling concolic execution approach. In International Conference on Cryptography, Security and Privacy. 123–128.
DOI: Google ScholarCross Ref - [87] . 2006. Locating faults through automated predicate switching. In International Conference on Software Engineering. 272–281.Google Scholar
- [88] . 2006. Pruning dynamic slices with confidence. In Programming Language Design and Implementation. Springer, 169–180.
DOI: Google ScholarDigital Library - [89] . 2022. Program vulnerability repair via inductive inference. In International Symposium on Software Testing and Analysis. 691–702.
DOI: Google ScholarDigital Library - [90] . 2006. Statistical debugging: Simultaneous identification of multiple bugs. In International Conference on Machine Learning. 1105–1112.
DOI: Google ScholarDigital Library - [91] . 2021. A syntax-guided edit decoder for neural program repair. In European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering. 13 pages.Google ScholarDigital Library
- [92] . 2019. An empirical study of fault localization families and their combinations. IEEE Trans. Softw. Eng. 47, 2 (2019).Google Scholar
Index Terms
- Variable-based Fault Localization via Enhanced Decision Tree
Recommendations
Combining mutation and fault localization for automated program debugging
Combining mutation and software fault localization for automated bug-fixing.Using only carefully selected mutant operators.Generating mutants only with respect to the most suspicious statements.Fixing software bugs without human intervention.Examining ...
Effective fault localization via mutation analysis: a selective mutation approach
SAC '14: Proceedings of the 29th Annual ACM Symposium on Applied ComputingWhen programs fail, developers face the problem of identifying the code fragments responsible for this failure. To this end, fault localization techniques try to identify suspicious program places (program statements) by observing the spectrum of the ...
Combining spectrum-based fault localization and statistical debugging: an empirical study
ASE '19: Proceedings of the 34th IEEE/ACM International Conference on Automated Software EngineeringProgram debugging is a time-consuming task, and researchers have proposed different kinds of automatic fault localization techniques to mitigate the burden of manual debugging. Among these techniques, two popular families are spectrum-based fault ...
Comments