Allen Goldberg
ABSTRACT
For certain structural testing criteria a significant proportion of tests instances are infeasible in the sense the semantics of the program implies that test data cannot be constructed that meet the test requirement. This paper describes the design and prototype implementation of a structural testing system that uses a theorem prover to determine feasibility of testing requirements and to optimize the number of test cases required to achieve test coverage. Using this approach, we were able to accurately and efficiently determine path feasibility for moderately-sized program units of production code written in a subset of Ada. On these problems, the computer solutions were obtained much faster and with greater accuracy than manual analysis. The paper describes how we formalize test criteria as control flow graph path expressions; how the criteria are mapped to logic formulas; and how we control the complexity of the inference task. It describes the limitations of the system and proposals for its improvement as well as other applications of the analysis.
- Andrews81.Andrews, P. B. Theorem Proving via general mating. J. ACM 28 (2), 1981, 193-214. Google Scholar
Digital Library
- Bibel81.Bibel, W. On matrices with connections. J. ACM 28(4) (1981) 633- 645. Google ScholarDigital Library
- Bicevskis79.Bicevskis, J., Borzovs, J., Straujums, U., Zarins, A., and Miller, E. SMOTL -- a system to construct samples for data processing program debugging. IEEE Transactions on Software Engineering SE-5, 8 (August 1990), 60-- 66.Google Scholar
- Bledsoe83.Bledsoe, W. W. The UT interactive prover. Tech. Report ATP- 17B, Department of Mathematics, The University of Texas at Austin (1983).Google Scholar
- Boyer75.Boyer, R., Elspas, B., and Levitt, K. SELECT -- A formal system for testing and debugging programs by symbolic execution. SIGPLAN Notices 10, 6, June 1975, 234--245. Google ScholarDigital Library
- Boyer86.Boyer, R. S. and Moore, J. S. Integrating decision procedures into heuristic theorem provers: a case study with linear arithmetic. In, Machine Intelligence 11, Oxford University Press, 1986. Google ScholarDigital Library
- Clarke76.Clarke, L.A. A system to generate test data and symbolically execute programs. IEEE Transactions on Software Engineering SE-2, 3 (September 1976), 215--222.Google ScholarDigital Library
- Clarke88.Clarke, L. A., Richardson, D.J., and Zeil, S.J. TEAM: A support environment for testing, evaluation, and analysis. InProceedings of the ACM SIGSOFT/SIGPLAN Software Engineering Symposium on Practical Software Development Environments Boston, MA, November 28--30, 1988, ACM, pp. 153-- 162. Google ScholarDigital Library
- Field93.Field, J., A simple rewriting semantics for realistic imperative programs and its application to program analysis, Revision of paper appearing in the 1992 Workshop on Partial Evaluation and Semantics-Based Program Manipulation, San Francisco.Google Scholar
- Frankl88.Frankl, P.G., and Weyuker, E.J. An applicable family of data flow testing criteria. IEEE Transactions on Software Engineering SE-14, 10, October 1988, 1483--1498. Google ScholarDigital Library
- Goff91.Goff, G., Kennedy, K. and Tseng, C., Practical dependence testing. In, ACM SiGPLAN '91 Conference on Programming Language Design and Implementation. ACM. Toronto, Canada, June 26--28, 1991, 15-29. Google ScholarDigital Library
- Hoare69.Hoare, C. An axiomatic basis for computer programming, CACM, 12,10 (1969). Google ScholarDigital Library
- Horwitz88.Horwitz, S., Prins, J., and Reps. T. On the adequacy of program dependence graphs for representing programs. In, Fifteenth A CM Symposium on Principles of Programrning Languages. ACM San Diego, CA, January 13-- 15, 1988, 146--157. Google ScholarDigital Library
- Howden77.Howden, W.E. Symbolic testing and the DISSECT symbolic evaluation techniques. IEEE Transactions on Software Engineering SE-4, 4 (1977), 266--278.Google ScholarDigital Library
- Jasper94.Jasper, R., Brennan, M., Williamson, K., Currier, C., and Zimmerman, D, Test data generation and feasible path analysis. International Symposium on Software Testing and Analysis (Seattle, WA, August 17-- 19, 1994). Google ScholarDigital Library
- Korel90.Korel, B. Automated software test data generation. IEEE Transactions on Software Engineering 16, SE-8 August 1990, 870--879. Google ScholarDigital Library
- Kotik89.Kotik, G.B., and Markosian, L.Z. Automating software analysis and testing using a program transformation system. In Proceedings of the ACM SIGSOFT '89 Third Symposium on Software Testing, Analysis, and Verification (Key West, F1, December 13-- 15, 1989), ACM, 48--54. Google ScholarDigital Library
- Laski90.Laski, Janusz, Data flow testing in STAD, J. Systems and Software, vol. 12 (1990), 3-14. Google ScholarDigital Library
- Letovsky88.Letovsky, S. I. Plan Analysis of Programs. Ph.D. Thesis, Yale University, Department of Computer Science December 1988. Google ScholarDigital Library
- Murray87.Murray, N.V., and Rosenthal, E. Inference with path resolution and semantic graphs, Journal of the ACM 34,2, (April, 1987), 225--254 Google ScholarDigital Library
- Pugh92.Pugh, W. A practical algorithm for exact array dependence analysis. Communications of the A CM 35, 8, August 1992, 102-- 115. Google ScholarDigital Library
- Ramamoorthy76.Ramamoorthy, C., Ho, S., and Chen, W. On the automated generation of program test data. IEEE Transactions on Software Engineering SE- 2, 4 (December 1976), 293--300.Google Scholar
- Reasoning90.RefineTM 3.0 User's Guide, Reasoning Systems Incorporated, Palo Alto, CA. May 25 1990.Google Scholar
- Reasoning92.Refine/Ada User's Guide, Reasoning Systems incorporated, Palo Alto, CA. July 26, 1992.Google Scholar
- Shostak79.Shostak, R. E. A practical decision procedure for arithmetic with functional symbols. Journal of the ACM 26, 2, April 1979, 351--360. Google ScholarDigital Library
- Shostak81.Shostak, R. E. Deciding linear inequalities by computing loop residues. Communications of the ACM 28, 4, October 1981,769--779. Google ScholarDigital Library
- Shostak84.Shostak, R. E. Deciding combinations of theories. Journal of the ACM 31, 1, January 1984, 1-- ! 2. Google ScholarDigital Library
- Smith90.Smith, D.R. KIDS ~- a semiautomatic program development system.IEEE Transactions on Software Engineering Special Issue on Formal Methods in Software Engineering 16, 9 (September 1990), 1024-- 1043. Google ScholarDigital Library
- Tripathy91.Tripathy, P., and Sarikaya, B. Test generation from LOTOS specifications, IEEE Trans. on Computers, 40, 4 (April, 1991), 543-561. Google ScholarDigital Library
- Wang85.Wang, T.C. Designing examples for semantically guided hierarchical deduction. In 9th International Joint Conference on Artificial Intelligence (Los Angeles, California, August 18-23, 1985).Google Scholar
- Wang87.Wang, T.C., and Bledsoe, W.W. Hierarchical deduction. Journal of Automated Reasoning 3, 1 (March 1987), 35--77. Google ScholarDigital Library
- Wang92.Wang, T.C., and Goldberg, A. RVF: an automated formal verification system.in Eleventh Conference on Automated Deduction, D.Kapur, Ed. Springer-Verlag, Berlin, 1992, pp. 131-- 138.Lecture Notes in Computer Science, Vol. 607. Google ScholarDigital Library
- Werner91.Werner, L. and W.E. Howden, An investigation of the applicability of data usage analysis. J. Systems and Software, Vol 15 (1991), 205-215. Google ScholarDigital Library
- Weyuker88.Weyuker, E.J. An empirical study of the complexity of data flow testing. In Proceedings of the Second Workshop on Software Testing, Analysis, and Verification, Banff, Canada, July 19-- 21, 1988, IEEE Computer Society, pp. 188--195.Google ScholarCross Ref
- Weyuker90.The cost of data flow testing: an empirical study, IEEE Trans. on Software Engineering, 16,2 (1990), 121- 128. Google ScholarDigital Library
- Wills87.Wills, L. M. Automated program recognition. Tech. Rep. MIT-AI- 904, MiT AI Laboratory, February 1987. Google ScholarDigital Library
- Woodward80.Woodward, M.R., Hedley, D., and Hennell, M.A. Experience with path analysis and testing. In Tutorial: Software Testing & Validation Techniques, 2Ed, E.Miller and W. Howden, Eds. IEEE Computer Society Press, Los Alamitos, CA, 1981, pp. 194--206.Google Scholar
- Wos68.Wos, L., and Robinson, A, Paramodulation and set of support. Proceedings of the IRIA Symposium on Automatic Demonstration, Versailles, France, France, Spring-Verlag (1968) 276-310.Google Scholar
- Yates89.Yates, D.F., and Malevris, N. Reducing the effects of infeasible paths in branch testing. In Proceedings of the ACM SIGSOFT '89 Third Symposium onSoftware Testing, Analysis, and Verification (Key West, F1, December 13-- 15, 1989), ACM, 48--54. Google ScholarDigital Library
Index Terms
- Applications of feasible path analysis to program testing
Recommendations
Self-Checked Metamorphic Testing of an Image Processing Program
SSIRI '10: Proceedings of the 2010 Fourth International Conference on Secure Software Integration and Reliability ImprovementMetamorphic testing is an effective technique for testing systems that do not have test oracles, for which it is practically impossible to know the correct output of an arbitrary test input. In metamorphic testing, instead of checking the correctness of ...
Path Coverage Information for Adaptive Random Testing
ICIT '17: Proceedings of the 2017 International Conference on Information TechnologyThis paper proposes the application of path coverage information into Adaptive Random Testing (ART). The ART is a distance-based technique. It selects the furthest test cases from the previous executed test cases. Applying Path Coverage is intended to ...
Experience with Path Analysis and Testing of Programs
There are a number of practical difficulties in performing a path testing strategy for computer programs. One problem is in deciding which paths, out of a possible infinity, to use as test cases. A hierarchy of structural test metrics is suggested to ...
Comments