Dongjiang You
ABSTRACT
Regression testing is often performed with a time budget and it does not allow executing all test cases. Test case prioritization techniques re-order test cases to increase the rate of fault detection. Several time-aware test case prioritization techniques have been proposed to satisfy a time budget. Since it is difficult to collect the time cost of each test case in some cases, a natural question is whether it is worth using such information when prioritizing test cases. In this paper, two most popular criteria: statement coverage and fault detection are considered for time-aware test case prioritization. We investigate whether the time cost of each test case affects the effectiveness of prioritization techniques, i.e. the rate of statement coverage and the rate of fault detection. Our empirical study shows that: although the techniques considering the time cost of each test case are slightly better than the techniques not considering such information in some cases, they have no significant difference in most cases.
- Gregg Rothermel, Roland H. Untch, Chengyun Chu, and Mary Jean Harrold. Test case prioritization: An empirical study. In ICSM, pages 179--188, 1999. Google Scholar
Digital Library
- Sebastian G. Elbaum, Alexey G. Malishevsky, and Gregg Rothermel. Test case prioritization: A family of empirical studies. IEEE Transactions on Software Engineering, 28(2): 159--182, 2002. Google ScholarDigital Library
- Hyunsook Do, Gregg Rothermel, and Alex Kinneer. Empirical studies of test case prioritization in a junit testing environment. In ISSRE, pages 113--124, 2004. Google ScholarDigital Library
- Kristen R. Walcott, Mary Lou Soffa, Gregory M. Kapfhammer, and Robert S. Roos. Time-aware test suite prioritization. In ISSTA, pages 1--12, 2006. Google ScholarDigital Library
- Lu Zhang, Shan-Shan Hou, Chao Guo, Tao Xie, and Hong Mei. Time-aware test-case prioritization using integer linear programming. In ISSTA, pages 213--224, 2009. Google ScholarDigital Library
- Sara Alspaug, Kristen R. Walcott, Michael Belanich, Gregory M. Kapfhammer, and Mary Lou Soffa. Efficient time-aware prioritization with knapsack solvers. In WEASELTech, pages 17--31, 2007. Google ScholarDigital Library
- Zengkai Ma and Jianjun Zhao. Test case prioritization based on analysis of program structure. In APSEC, pages 471--478, 2008. Google ScholarDigital Library
- Shin Yoo, Mark Harman, Paolo Tonella, and Angelo Susi. Clustering test cases to achieve effective and scalable prioritisation incorporating expert knowledge. In ISSTA, pages 201--212, 2009. Google ScholarDigital Library
- Xiaofang Zhang, Changhai Nie, Baowen Xu, and Bo Qu. Test case prioritization based on varying testing requirement priorities and test case costs. In QSIC, pages 15--24, 2007. Google ScholarDigital Library
- Hyunsook Do, Siavash Mirarab, Ladan Tahvildari, and Gregg Rothermel. An empirical study of the effect of time constraints on the cost-benefits of regression testing. In SIGSOFT FSE, pages 71--82, 2008. Google ScholarDigital Library
- Hyunsook Do, Siavash Mirarab, Ladan Tahvildari, and Gregg Rothermel. The effects of time constraints on test case prioritization: A series of controlled experiments. IEEE Transactions on Software Engineering, 36(5): 593--617, 2010. Google ScholarDigital Library
- Sebastian G. Elbaum, Alexey G. Malishevsky, and Gregg Rothermel. Incorporating varying test costs and fault severities into test case prioritization. In ICSE, pages 329--338, 2001. Google ScholarDigital Library
- Hyunsook Do, Sebastian G. Elbaum, and Gregg Rothermel. Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact. Empirical Software Engineering, 10(4): 405--435, 2005. Google ScholarDigital Library
- Zheng Li, Mark Harman, and Robert M. Hierons. Search algorithms for regression test case prioritization. IEEE Transactions on Software Engineering, 33(4): 225--237, 2007. Google ScholarDigital Library
- Sihan Li, Naiwen Bian, Zhenyu Chen, Dongjiang You, and Yuchen He. A simulation study on some search algorithms for regression test case prioritization. In QSIC, pages 72--81, 2010. Google ScholarDigital Library
- Hao Zhong, Lu Zhang, and Hong Mei. An experimental study of four typical test suite reduction techniques. Information & Software Technology, 50(6): 534--546, 2008. Google ScholarDigital Library
Index Terms
- An empirical study on the effectiveness of time-aware test case prioritization techniques
Recommendations
Time-aware test-case prioritization using integer linear programming
ISSTA '09: Proceedings of the eighteenth international symposium on Software testing and analysisTechniques for test-case prioritization re-order test cases to increase their rate of fault detection. When there is a fixed time budget that does not allow the execution of all the test cases, time-aware techniques for test-case prioritization may ...
TimeAware test suite prioritization
ISSTA '06: Proceedings of the 2006 international symposium on Software testing and analysisRegression test prioritization is often performed in a time constrained execution environment in which testing only occurs for a fixed time period. For example, many organizations rely upon nightly building and regression testing of their applications ...
A Unified Test Case Prioritization Approach
Test case prioritization techniques attempt to reorder test cases in a manner that increases the rate at which faults are detected during regression testing. Coverage-based test case prioritization techniques typically use one of two overall strategies: ...
Comments