skip to main content
research-article

Precise Learn-to-Rank Fault Localization Using Dynamic and Static Features of Target Programs

Published:09 October 2019Publication History
Skip Abstract Section

Abstract

Finding the root cause of a bug requires a significant effort from developers. Automated fault localization techniques seek to reduce this cost by computing the suspiciousness scores (i.e., the likelihood of program entities being faulty). Existing techniques have been developed by utilizing input features of specific types for the computation of suspiciousness scores, such as program spectrum or mutation analysis results. This article presents a novel learn-to-rank fault localization technique called PRecise machINe-learning-based fault loCalization tEchnique (PRINCE). PRINCE uses genetic programming (GP) to combine multiple sets of localization input features that have been studied separately until now. For dynamic features, PRINCE encompasses both Spectrum Based Fault Localization (SBFL) and Mutation Based Fault Localization (MBFL) techniques. It also uses static features, such as dependency information and structural complexity of program entities. All such information is used by GP to train a ranking model for fault localization. The empirical evaluation on 65 real-world faults from CoREBench, 84 artificial faults from SIR, and 310 real-world faults from Defects4J shows that PRINCE outperforms the state-of-the-art SBFL, MBFL, and learn-to-rank techniques significantly. PRINCE localizes a fault after reviewing 2.4% of the executed statements on average (4.2 and 3.0 times more precise than the best of the compared SBFL and MBFL techniques, respectively). Also, PRINCE ranks 52.9% of the target faults within the top ten suspicious statements.

References

  1. [n.d.]. Docker. Retrieved from https://www.docker.com/.Google ScholarGoogle Scholar
  2. Rui Abreu, Peter Zoeteweij, and Arjan J. C. van Gemund. 2006. An evaluation of similarity coefficients for software fault localization. In The Proceedings of the 12th Pacific Rim International Symposium on Dependable Computing (PRDC’06). IEEE, 39--46.Google ScholarGoogle Scholar
  3. Hiralal Agrawal, Richard DeMillo, R. Hathaway, William Hsu, Wynne Hsu, Edward Krauser, Rhonda J. Martin, Aditya Mathur, and Eugene Spafford. 1989. Design of Mutant Operators for the C Programming Language. Technical Report SERC-TR-120-P. Purdue University.Google ScholarGoogle Scholar
  4. Fumio Akiyama. 1971. An example of software system debugging. In IFIP Congress (1), Vol. 71. 353--359.Google ScholarGoogle Scholar
  5. Tien-Duy B. Le, David Lo, Claire Le Goues, and Lars Grunske. 2016. A learning-to-rank based fault localization approach using likely invariants. In Proceedings of the 25th International Symposium on Software Testing and Analysis (ISSTA’16). ACM, New York, NY, 177--188.Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Marcel Böhme and Abhik Roychoudhury. 2014. CoREBench: Studying complexity of regression errors. In Proceedings of the 2014 International Symposium on Software Testing and Analysis (ISSTA’14). ACM, New York, NY, 105--115. DOI:https://doi.org/10.1145/2610384.2628058Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Stefan Bucur, Vlad Ureche, Cristian Zamfir, and George Candea. 2011. Parallel symbolic execution for automated real-world software testing. In Proceedings of the 6th Conference on Computer Systems (EuroSys’11). ACM, New York, NY, 183--198. DOI:https://doi.org/10.1145/1966445.1966463Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Satish Chandra, Emina Torlak, Shaon Barman, and Rastislav Bodik. 2011. Angelic debugging. In Proceedings of the 33rd International Conference on Software Engineering (ICSE’11). ACM, New York, NY, 121--130. DOI:https://doi.org/10.1145/1985793.1985811Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Chih-Chung Chang and Chih-Jen Lin. 2011. LIBSVM: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 3, Article 27 (May 2011), 27 pages. DOI:https://doi.org/10.1145/1961189.1961199Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Shyam R. Chidamber and Chris F. Kemerer. 1994. A metrics suite for object oriented design. IEEE Transactions on Software Engineering 20, 6 (1994), 476--493.Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Holger Cleve and Andreas Zeller. 2005. Locating causes of program failures. In Proceedings of the 27th International Conference on Software Engineering (ICSE’05). ACM, New York, NY, 342--351. DOI:https://doi.org/10.1145/1062455.1062522Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Corinna Cortes and Vladimir Vapnik. 1995. Support-vector networks. Mach. Learn. 20, 3 (1 Sep 1995), 273--297. DOI:https://doi.org/10.1023/A:1022627411411Google ScholarGoogle Scholar
  13. Valentin Dallmeier, Christian Lindig, and Andreas Zeller. 2005. Lightweight bug localization with AMPLE. In Proceedings of the 6th International Symposium on Automated Analysis-driven Debugging (AADEBUG’05). ACM, New York, NY, 99--104. DOI:https://doi.org/10.1145/1085130.1085143Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Marco D’Ambros, Michele Lanza, and Romain Robbes. 2012. Evaluating defect prediction approaches: A benchmark and an extensive comparison. Emp. Softw. Eng. 17, 4 (1 Aug. 2012), 531--577. https://doi.org/10.1007/s10664-011-9173-9Google ScholarGoogle Scholar
  15. Cuthbert Daniel. 1973. One-at-a-time plans. J. Am. Stat. Assoc. 68, 342 (1973), 353--360.Google ScholarGoogle ScholarCross RefCross Ref
  16. Hyunsook Do, Sebastian Elbaum, and Gregg Rothermel. 2005. Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact. Emp. Softw. Eng. 10, 4 (1 Oct. 2005), 405--435. DOI:https://doi.org/10.1007/s10664-005-3861-2Google ScholarGoogle Scholar
  17. Fernando Brito e Abreu and Rogério Carapuça. 1994. Candidate metrics for object-oriented software within a taxonomy framework. J. Syst. Softw. 26, 1 (1994), 87--96.Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Sarah Fakhoury, Devjeet Roy, Sk. Adnan Hassan, and Venera Arnaoudova. 2019. Improving source code readability: Theory and practice. In Proceedings of the 27th International Conference on Program Comprehension (ICPC’19). IEEE Press, Piscataway, NJ, 2--12. DOI:https://doi.org/10.1109/ICPC.2019.00014Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. LIBLINEAR: A library for large linear classification. J. Mach. Learn. Res. 9 (Jun. 2008), 1871--1874. http://dl.acm.org/citation.cfm?id=1390681.1442794.Google ScholarGoogle Scholar
  20. Pei Gong, Ruilian Zhao, and Zheng Li. 2015. Faster mutation-based fault localization with a novel mutation execution strategy. In Proceedings of the 2015 IEEE 8th International Conference on Software Testing, Verification and Validation Workshops (ICSTW’15). 1--10. DOI:https://doi.org/10.1109/ICSTW.2015.7107448Google ScholarGoogle ScholarCross RefCross Ref
  21. Divya Gopinath, Razieh Nokhbeh Zaeem, and Sarfraz Khurshid. 2012. Improving the effectiveness of spectra-based fault localization using specifications. In Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering (ASE’12). ACM, New York, NY, 40--49. DOI:https://doi.org/10.1145/2351676.2351683Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Isabelle Guyon and André Elisseeff. 2003. An introduction to variable and feature selection. J. Mach. Learn. Res. 3 (Mar. 2003), 1157--1182. http://dl.acm.org/citation.cfm?id=944919.944968Google ScholarGoogle Scholar
  23. Maurice H. Halstead. 1977. Elements of Software Science (Operating and Programming Systems Series). Elsevier Science Inc., New York, NY.Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Maurice H. Halstead et al. 1977. Elements of Software Science (Operating and Programming Systems Series). Elsevier Science Inc., New York, NY.Google ScholarGoogle Scholar
  25. Shin Hong, Taehoon Kwak, Byeongcheol Lee, Yiru Jeon, Bongseok Ko, Yunho Kim, and Moonzoo Kim. 2017. MUSEUM: Debugging real-world multilingual programs using mutation analysis. Inf. Softw. Technol. 82 (2017), 80--95. DOI:https://doi.org/10.1016/j.infsof.2016.10.002Google ScholarGoogle ScholarCross RefCross Ref
  26. Shin Hong, Byeongcheol Lee, Taehoon Kwak, Yiru Jeon, Bongsuk Ko, Yunho Kim, and Moonzoo Kim. 2015. Mutation-based fault localization for real-world multilingual programs. In Proceedings of the 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE’15). IEEE Computer Society, Washington, DC, 464--475. DOI:https://doi.org/10.1109/ASE.2015.14Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Paul Jaccard. 1901. Étude comparative de la distribution florale dans une portion des Alpes et des Jura. Bull. Soc. vaud. Sci. nat 37 (1901), 547--579.Google ScholarGoogle Scholar
  28. Tom Janssen, Rui Abreu, and Arjan J. C. van Gemund. 2009. Zoltar: A toolset for automatic fault localization. In Proceedings of the 2009 IEEE/ACM International Conference on Automated Software Engineering (ASE’09). IEEE Computer Society, Washington, DC, 662--664.Google ScholarGoogle Scholar
  29. Dennis Jeffrey, Neelam Gupta, and Rajiv Gupta. 2008. Fault localization using value replacement. In Proceedings of the 2008 International Symposium on Software Testing and Analysis (ISSTA’08). ACM, New York, NY, 167--178. DOI:https://doi.org/10.1145/1390630.1390652Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Yue Jia and Mark Harman. 2008. MILU: A customizable, runtime-optimized higher order mutation testing tool for the full C language. In Proceedings of the 3rd Testing: Academic and Industrial Conference—Practice and Research Techniques (TAIC PART’08). IEEE, 94--98. DOI:https://doi.org/10.1109/TAIC-PART.2008.18Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. James A. Jones and Mary Jean Harrold. 2005. Empirical evaluation of the Tarantula automatic fault-localization technique. In Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering (ASE’05). ACM, New York, NY, 273--282. DOI:https://doi.org/10.1145/1101908.1101949Google ScholarGoogle Scholar
  32. René Just. 2014. The Major mutation framework: Efficient and scalable mutation analysis for Java. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA’14). 433--436.Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. René Just, Darioush Jalali, and Michael D. Ernst. 2014. Defects4J: A database of existing faults to enable controlled testing studies for Java programs. In Proceedings of the 2014 International Symposium on Software Testing and Analysis (ISSTA’14). ACM, New York, NY, 437--440. DOI:https://doi.org/10.1145/2610384.2628055Google ScholarGoogle Scholar
  34. Moonzoo Kim, Yunho Kim, and Gregg Rothermel. 2012. A scalable distributed concolic testing approach: An empirical evaluation. In Proceedings of the 2012 IEEE 5th International Conference on Software Testing, Verification and Validation. 340--349. DOI:https://doi.org/10.1109/ICST.2012.114Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Yunho Kim, Yunja Choi, and Moonzoo Kim. 2018. Precise concolic unit testing of C programs using extended units and symbolic alarm filtering. In Proceedings of the 40th International Conference on Software Engineering (ICSE’18). ACM, New York, NY, 315--326. DOI:https://doi.org/10.1145/3180155.3180253Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. Yunho Kim and Shin Hong. 2019. DeMiner: Test generation for high test coverage through mutant exploration. (submitted).Google ScholarGoogle Scholar
  37. Yunho Kim, Shin Hong, and Moonzoo Kim. 2019. Target-driven compositional concolic testing with function summary refinement for effective bug detection. In Proceedings of the 27th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE’19).Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Yunho Kim, Shin Hong, Bongseok Ko, Duy Loc Phan, and Moonzoo Kim. 2018. Invasive software testing: Mutating target programs to diversify test exploration for high test coverage. In Proceedings of the 2018 IEEE 11th International Conference on Software Testing, Verification and Validation (ICST’18). 239--249. DOI:https://doi.org/10.1109/ICST.2018.00032Google ScholarGoogle ScholarCross RefCross Ref
  39. Yunho Kim, Youil Kim, Taeksu Kim, Gunwoo Lee, Yoonkyu Jang, and Moonzoo Kim. 2013. Automated unit testing of large industrial embedded software using concolic testing. In Proceedings of the 28th IEEE/ACM International Conference on Automated Software Engineering (ASE’13). IEEE Press, Piscataway, NJ, 519--528. DOI:https://doi.org/10.1109/ASE.2013.6693109Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Yunho Kim, Dongju Lee, Junki Baek, and Moonzoo Kim. 2019. Concolic testing for high test coverage and reduced human effort in automotive industry. In Proceedings of the 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP’10). IEEE Press, Piscataway, NJ, 151--160. DOI:https://doi.org/10.1109/ICSE-SEIP.2019.00024Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Yunho Kim, Zihong Xu, Moonzoo Kim, Myra B. Cohen, and Gregg Rothermel. 2014. Hybrid directed test suite augmentation: An interleaving framework. In Proceedings of the 2014 IEEE 7th International Conference on Software Testing, Verification and Validation. 263--272. DOI:https://doi.org/10.1109/ICST.2014.39Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Ching-Pei Lee and Chih-Jen Lin. 2014. Large-scale linear Ranksvm. Neur. Comput. 26, 4 (Apr. 2014), 781--817.Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Jundong Li, Kewei Cheng, Suhang Wang, Fred Morstatter, Robert P. Trevino, Jiliang Tang, and Huan Liu. 2017. Feature selection: A data perspective. ACM Comput. Surv. 50, 6, Article 94 (Dec. 2017), 45 pages. DOI:https://doi.org/10.1145/3136625Google ScholarGoogle Scholar
  44. Bing Liu, Shiva Nejati, Lucia, and Lionel C. Briand. 2019. Effective fault localization of automotive Simulink models: Achieving the trade-off between test oracle effort and fault localization accuracy. Emp. Softw. Eng. 24, 1 (1 Feb. 2019), 444--490. DOI:https://doi.org/10.1007/s10664-018-9611-zGoogle ScholarGoogle Scholar
  45. José Carlos Maldonado, Márcio Eduardo Delamaro, Sandra C. P. F. Fabbri, Adenilso da Silva Simão, Tatiana Sugeta, Auri Marcelo Rizzo Vincenzi, and Paulo Cesar Masiero. 2001. Proteum: A family of tools to support specification and program testing based on mutation. In Mutation Testing for the New Century, W. Eric Wong (Ed.). Springer US, Boston, MA, 113--116. DOI:https://doi.org/10.1007/978-1-4757-5939-6_19Google ScholarGoogle Scholar
  46. Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. 2008. Introduction to Information Retrieval. Cambridge University Press, New York, NY.Google ScholarGoogle Scholar
  47. Radu Marinescu. 2004. Detection strategies: Metrics-based rules for detecting design flaws. In Proceedings of the 20th IEEE International Conference on Software Maintenance 2004. 350--359. DOI:https://doi.org/10.1109/ICSM.2004.1357820Google ScholarGoogle ScholarCross RefCross Ref
  48. Thomas J. McCabe. 1976. A complexity measure. IEEE Transactions on Software Engineering 4 (1976), 308--320.Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Seokhyeon Moon, Yunho Kim, Moonzoo Kim, and Shin Yoo. 2014. Ask the mutants: Mutating faulty programs for fault localization. In Proceedings of the 2014 IEEE International Conference on Software Testing, Verification, and Validation (ICST’14). IEEE Computer Society, Washington, DC, 153--162. DOI:https://doi.org/10.1109/ICST.2014.28Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Lee Naish, Hua Jie Lee, and Kotagiri Ramamohanarao. 2011. A model for spectra-based software diagnosis. ACM Trans. Softw. Eng. Methodol. 20, 3, Article 11 (Aug. 2011), 32 pages. DOI:https://doi.org/10.1145/2000791.2000795Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Akira Ochiai. 1957. Zoogeographic studies on the soleoid fishes found in Japan and its neighbouring regions. Bull. Jpn. Soc. Sci. Fish. 22, 9 (1957), 526--530.Google ScholarGoogle ScholarCross RefCross Ref
  52. Hristina Palikareva, Tomasz Kuchta, and Cristian Cadar. 2016. Shadow of a doubt: Testing for divergences between software versions. In Proceedings of the 38th International Conference on Software Engineering (ICSE’16). ACM, New York, NY, 1181--1192. DOI:https://doi.org/10.1145/2884781.2884845Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Mike Papadakis and Yves Le Traon. 2012. Using mutants to locate ”Unknown” faults. In Proceedings of the 2012 IEEE 5th International Conference on Software Testing, Verification and Validation. 691--700. DOI:https://doi.org/10.1109/ICST.2012.159Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Mike Papadakis and Yves Le Traon. 2015. Metallaxis-FL: Mutation-based fault localization. Softw. Test., Verif. Reliab. 25, 5--7 (2015), 605--628. DOI:https://doi.org/10.1002/stvr.1509Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Chris Parnin and Alessandro Orso. 2011. Are automated debugging techniques actually helping programmers? In Proceedings of the 2011 International Symposium on Software Testing and Analysis (ISSTA’11). ACM, New York, NY, 199--209. DOI:https://doi.org/10.1145/2001420.2001445Google ScholarGoogle ScholarDigital LibraryDigital Library
  56. Karl Pearson. 1895. Note on regression and inheritance in the case of two parents. Proc. Roy. Soc. Lond. 58, 347--352 (1895), 240--242.Google ScholarGoogle Scholar
  57. Spencer Pearson, José Campos, René Just, Gordon Fraser, Rui Abreu, Michael D. Ernst, Deric Pang, and Benjamin Keller. 2017. Evaluating and improving fault localization. In Proceedings of the 39th International Conference on Software Engineering (ICSE’17). IEEE Press, Piscataway, NJ, 609--620. DOI:https://doi.org/10.1109/ICSE.2017.62Google ScholarGoogle ScholarDigital LibraryDigital Library
  58. Alexandre Perez, Rui Abreu, and Marcelo D’Amorim. 2017. Prevalence of single-fault fixes and its impact on fault localization. In Proceedings of the 2017 IEEE International Conference on Software Testing, Verification and Validation (ICST’17). 12--22. DOI:https://doi.org/10.1109/ICST.2017.9Google ScholarGoogle ScholarCross RefCross Ref
  59. Alexandre Perez, Rui Abreu, and Arie van Deursen. 2017. A test-suite diagnosability metric for spectrum-based fault localization approaches. In Proceedings of the 39th International Conference on Software Engineering (ICSE’17). IEEE Press, Piscataway, NJ, 654--664. DOI:https://doi.org/10.1109/ICSE.2017.66Google ScholarGoogle ScholarDigital LibraryDigital Library
  60. Loc Duy Phan, Yunho Kim, and Moonzoo Kim. 2018. MUSIC: Mutation analysis tool with high configurability and extensibility. In Proceedings of the 2018 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW’18). 40--46. DOI:https://doi.org/10.1109/ICSTW.2018.00026Google ScholarGoogle ScholarCross RefCross Ref
  61. Danijel Radjenovic, Marjan Hericko, Richard Torkar, and Ales Zivkovic. 2013. Software fault prediction metrics: A systematic literature review. Inf. Softw. Technol. 55, 8 (2013), 1397--1418. DOI:https://doi.org/10.1016/j.infsof.2013.02.009Google ScholarGoogle ScholarDigital LibraryDigital Library
  62. Santosh S. Rathore and Sandeep Kumar. 2019. A study on software fault prediction techniques. Artif. Intell. Rev. 51, 2 (01 Feb. 2019), 255--327. DOI:https://doi.org/10.1007/s10462-017-9563-5Google ScholarGoogle ScholarDigital LibraryDigital Library
  63. Manos Renieris and Steven P. Reiss. 2003. Fault localization with nearest neighbor queries. In Proceedings of the 18th IEEE International Conference on Automated Software Engineering (ASE’03). IEEE Press, Piscataway, NJ, 30--39. DOI:https://doi.org/10.1109/ASE.2003.1240292Google ScholarGoogle Scholar
  64. Paul F. Russell and T. Ramachandra Rao. 1940. On habitat and association of species of anopheline larvae in South-eastern Madras. J. Malar. Inst. India 3, 1 (1940), 153--178.Google ScholarGoogle Scholar
  65. Raul Santelices, James A. Jones, Yanbing Yu, and Mary Jean Harrold. 2009. Lightweight fault-localization using multiple coverage types. In Proceedings of the 31st International Conference on Software Engineering (ICSE’09). IEEE Computer Society, Washington, DC, 56--66. DOI:https://doi.org/10.1109/ICSE.2009.5070508Google ScholarGoogle ScholarDigital LibraryDigital Library
  66. Tushar Sharma and Diomidis Spinellis. 2018. A survey on software smells. J. Syst. Softw. 138 (2018), 158--173. DOI:https://doi.org/10.1016/j.jss.2017.12.034Google ScholarGoogle ScholarCross RefCross Ref
  67. Nishant Sinha, Nimit Singhania, Satish Chandra, and Manu Sridharan. 2012. Alternate and learn: Finding witnesses without looking all over. In Computer Aided Verification, P. Madhusudan and Sanjit A. Seshia (Eds.). Springer, Berlin, 599--615.Google ScholarGoogle Scholar
  68. Jeongju Sohn and Shin Yoo. 2017. FLUCCS: Using code and change metrics to improve fault localization. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA’17). ACM, New York, NY, 273--283. DOI:https://doi.org/10.1145/3092703.3092717Google ScholarGoogle ScholarDigital LibraryDigital Library
  69. Friedrich Steimann, Marcus Frenkel, and Rui Abreu. 2013. Threats to the validity and value of empirical assessments of the accuracy of coverage-based fault locators. In Proceedings of the 2013 International Symposium on Software Testing and Analysis (ISSTA’13). ACM, New York, NY, 314--324. DOI:https://doi.org/10.1145/2483760.2483767Google ScholarGoogle ScholarDigital LibraryDigital Library
  70. Shin Hwei Tan, Hiroaki Yoshida, Mukul R. Prasad, and Abhik Roychoudhury. 2016. Anti-patterns in search-based program repair. In Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE’16). ACM, New York, NY, 727--738. DOI:https://doi.org/10.1145/2950290.2950295Google ScholarGoogle ScholarDigital LibraryDigital Library
  71. Iris Vessey. 1985. Expertise in debugging computer programs: A process analysis. Int. J. Man-Mach. Stud. 23, 5 (1985), 459--494. DOI:https://doi.org/10.1016/S0020-7373(85)80054-7Google ScholarGoogle ScholarCross RefCross Ref
  72. Frank Wilcoxon. 1945. Individual comparisons by ranking methods. Biometr. Bull. 1, 6 (1945), 80--83. http://www.jstor.org/stable/3001968Google ScholarGoogle ScholarCross RefCross Ref
  73. W. Eric Wong, Vidroha Debroy, and Ruizhi Gao Yihao Li. 2014. The DStar method for effective software fault localization. IEEE Trans. Reliabil. 63, 1 (Mar. 2014), 290--308. DOI:https://doi.org/10.1109/TR.2013.2285319Google ScholarGoogle ScholarCross RefCross Ref
  74. W. Eric Wong, Ruizhi Gao, Yihao Li, Rui Abreu, and Franz Wotawa. 2016. A survey on software fault localization. IEEE Trans. Softw. Eng. 42, 8 (Aug. 2016), 707.Google ScholarGoogle ScholarDigital LibraryDigital Library
  75. W. Eric Wong, Yu Qi, Lei Zhao, and Kai-Yuan Cai. 2007. Effective fault localization using code coverage. In Proceedings of the 31st Annual International Computer Software and Applications Conference, Vol. 01 (COMPSAC’07). IEEE Computer Society, Washington, DC, 449--456. DOI:https://doi.org/10.1109/COMPSAC.2007.109Google ScholarGoogle ScholarDigital LibraryDigital Library
  76. Xin Xia, Lingfeng Bao, David Lo, and Shanping Li. 2016. “Automated debugging considered harmful” considered harmful: A user study revisiting the usefulness of spectra-based fault localization techniques with professionals using real bugs from large systems. In Proceedings of the IEEE International Conference on Software Maintenance and Evolution (ICSME’16). 267--278. DOI:https://doi.org/10.1109/ICSME.2016.67Google ScholarGoogle ScholarCross RefCross Ref
  77. Xiaoyuan Xie, Tsong Yueh Chen, Fei-Ching Kuo, and Baowen Xu. 2013. A theoretical analysis of the risk evaluation formulas for spectrum-based fault localization. ACM Trans. Softw. Eng. Methodol. 22, 4, Article 31 (Oct. 2013), 40 pages. DOI:https://doi.org/10.1145/2522920.2522924Google ScholarGoogle ScholarDigital LibraryDigital Library
  78. Zhihong Xu, Yunho Kim, Moonzoo Kim, Myra B. Cohen, and Gregg Rothermel. 2015. Directed test suite augmentation: An empirical investigation. Softw. Test. Verif. Reliabil. 25, 2 (2015), 77--114. DOI:https://doi.org/10.1002/stvr.1562 arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1002/stvr.1562Google ScholarGoogle ScholarDigital LibraryDigital Library
  79. Zhihong Xu, Yunho Kim, Moonzoo Kim, and Gregg Rothermel. 2011. A hybrid directed test suite augmentation technique. In Proceedings of the 2011 IEEE 22nd International Symposium on Software Reliability Engineering (ISSRE’11). IEEE Computer Society, Washington, DC, 150--159. DOI:https://doi.org/10.1109/ISSRE.2011.21Google ScholarGoogle ScholarDigital LibraryDigital Library
  80. Zhihong Xu, Yunho Kim, Moonzoo Kim, Gregg Rothermel, and Myra B. Cohen. 2010. Directed test suite augmentation: Techniques and tradeoffs. In Proceedings of the 18th ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE’10). ACM, New York, NY, 257--266. DOI:https://doi.org/10.1145/1882291.1882330Google ScholarGoogle Scholar
  81. Jifeng Xuan and M. Monperrus. 2014. Learning to combine multiple ranking metrics for fault localization. In Proceedings of the IEEE International Conference on Software Maintenance and Evolution (ICSME’14). 191--200. DOI:https://doi.org/10.1109/ICSME.2014.41Google ScholarGoogle Scholar
  82. Shin Yoo. 2012. Evolving human competitive spectra-based fault localisation techniques. In Proceedings of the 4th International Conference on Search Based Software Engineering (SSBSE’12). Springer-Verlag, Berlin, 244--258. DOI:https://doi.org/10.1007/978-3-642-33119-0_18Google ScholarGoogle ScholarDigital LibraryDigital Library
  83. Shin Yoo, Xiaoyuan Xie, Fei-Ching Kuo, Tsong Yueh Chen, and Mark Harman. 2017. Human competitiveness of genetic programming in spectrum-based fault localisation: Theoretical and empirical analysis. ACM Trans. Softw. Eng. Methodol. 26, 1, Article 4 (Jun. 2017), 30 pages. DOI:https://doi.org/10.1145/3078840Google ScholarGoogle ScholarDigital LibraryDigital Library
  84. Andreas Zeller. 2002. Isolating cause-effect chains from computer programs. In Proceedings of the 10th ACM SIGSOFT Symposium on Foundations of Software Engineering (SIGSOFT’02/FSE-10). ACM, New York, NY, 1--10. DOI:https://doi.org/10.1145/587051.587053Google ScholarGoogle ScholarDigital LibraryDigital Library
  85. Lingming Zhang, Lu Zhang, and Sarfraz Khurshid. 2013. Injecting mechanical faults to localize developer faults for evolving software. In Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages 8 Applications (OOPSLA’13). ACM, New York, NY, 765--784. DOI:https://doi.org/10.1145/2509136.2509551Google ScholarGoogle ScholarDigital LibraryDigital Library
  86. Xiangyu Zhang, Neelam Gupta, and Rajiv Gupta. 2006. Locating faults through automated predicate switching. In Proceedings of the 28th International Conference on Software Engineering (ICSE’06). ACM, New York, NY, 272--281. DOI:https://doi.org/10.1145/1134285.1134324Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Precise Learn-to-Rank Fault Localization Using Dynamic and Static Features of Target Programs

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Transactions on Software Engineering and Methodology
      ACM Transactions on Software Engineering and Methodology  Volume 28, Issue 4
      October 2019
      231 pages
      ISSN:1049-331X
      EISSN:1557-7392
      DOI:10.1145/3360049
      • Editor:
      • Mauro Pezzè
      Issue’s Table of Contents

      Copyright © 2019 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 9 October 2019
      • Revised: 1 July 2019
      • Accepted: 1 July 2019
      • Received: 1 July 2018
      Published in tosem Volume 28, Issue 4

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article
      • Research
      • Refereed

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    HTML Format

    View this article in HTML Format .

    View HTML Format