Lionel Briand
Abstract
This position paper aims at discussing a number of issues that typically arise when performing empirical studies with software testing techniques. Though some problems are general to all empirical disciplines, software testing studies face a number of specific challenges. Some of the main ones are discussed in sequence below.
- G. Antoniol, L. C. Briand, M. Di Penta and Y. Labiche, "A Case Study Using the Round-Trip Strategy for State-Based Class Testing," Proc. 13th IEEE International Symposium on Software Reliability Engineering (ISSRE'02), Annapolis (MD, USA), IEEE Computer Society, pp. 269--279, November 12-15, 2002. Google Scholar
Digital Library
- L. Briand, Y. Labiche and Y. Wang, "Using Simulation to Empirically Investigate Test Coverage Criteria," Proc. (to appear) IEEE/ACM International Conference on Software Engineering, Edinburgh, May, 2004. Google ScholarDigital Library
- L. C. Briand, Y. Labiche and H. Sun, "Investigating the Use of Analysis Contracts to Improve the Testability of Object-Oriented Code," Software - Practice and Experience, vol. 33 (7), pp. 637--672, 2003. Google ScholarDigital Library
- M. E. Delamaro, J. C. Maldonado and A. P. Mathur, "Interface Mutation: An Approach for Integration Testing," IEEE Transactions of Software Engineering, vol. 27 (3), pp. 228--247, 2001. Google ScholarDigital Library
- R. A. DeMillo, "Hints on Test Data Selection: Help for the Practicing Programmer," in IEEE Computer, vol. 11, 1978, pp. 34--41.Google ScholarDigital Library
- T. L. Graves, M. J. Harrold, J.-M. Kim, A. Porter and G. Rothermel, "An Empirical Study of Regression Test Selection Techniques," ACM Transactions on Software Engineering and Methodology, vol. 10 (2), pp. 184--208, 2001. Google ScholarDigital Library
- M. Hutchins, H. Froster, T. Goradia and T. Ostrand, "Experiments on the Effectiveness of Dataflow- and Controlflow-Based Test Adequacy Criteria," Proc. 16th IEEE International Conference on Software Engineering, Sorrento (Italy), pp. 191--200, May 16-21, 1994. Google ScholarDigital Library
- N. Juristo and A. M. Moreno, Basics of Software Engineering Experimentation, Kluwer, 2001. Google ScholarDigital Library
- S. Kim, J. A. Clark and J. A. McDermid, "Class Mutation: Mutation Testing for Object-Oriented Programs," Proc. Net. ObjectDays, Erfurt, Germany, October 9-12, 2000.Google Scholar
- K. N. King and A. J. Offutt, "A Fortran Language System for Mutation-Based Software Testing," Software - Practice and Experience, vol. 21 (7), pp. 686--718, 1991. Google ScholarDigital Library
- A. J. Offutt, "Investigations of the Software Testing Coupling Effect," ACM Transactions on Software Engineering and Methodology, vol. 1 (1), pp. 3--18, 1992. Google ScholarDigital Library
- E. Weyuker, "More Experience with Data Flow Testing," IEEE Transactions on Software Engineering, vol. 19 (9), pp. 912--919, 1993. Google ScholarDigital Library
- C. Wohlin, P. Runeson, M. Host, M. C. Ohlsson, B. Regnell and A. Wesslen, Experimentation in Software Engineering - An Introduction, Kluwer, 2000. Google ScholarDigital Library
Index Terms
- Empirical studies of software testing techniques: challenges, practical strategies, and future research
Recommendations
Ethical Issues in Empirical Studies of Software Engineering
The popularity of empirical methods in software engineering research is on the rise. Surveys, experiments, metrics, case studies, and field studies are examples of empirical methods used to investigate both software engineering processes and products. ...
A Critical Analysis of Empirical Research in Software Testing
ESEM '07: Proceedings of the First International Symposium on Empirical Software Engineering and MeasurementIn the foreseeable future, software testing will remain one of the best tools we have at our disposal to ensure software dependability. Empirical studies are crucial to software testing research in order to compare and improve software testing ...
Empirical Studies of Test Case Prioritization in a JUnit Testing Environment
ISSRE '04: Proceedings of the 15th International Symposium on Software Reliability EngineeringTest case prioritization provides a way to run test cases with the highest priority earliest. Numerous empirical studies have shown that prioritization can improve a test suite's rate of fault detection, but the extent to which these results generalize ...
Comments