Sylvain Hallé
ABSTRACT
Boolean test input generation is the process of finding sets of values for variables of a logical expression such that a given coverage criterion is achieved. This paper presents a formal framework in which evaluating an expression produces a tree structure, and where a coverage criterion is expressed as equivalence classes induced by a particular transformation over these trees. It then defines many well-known coverage criteria as particular cases of this framework. The paper describes an algorithm to generate test suites by a reduction through a graph problem; this algorithm works in the same way regardless of the criterion considered. An experimental evaluation of this technique shows that it produces test suites that are in many cases smaller than existing tools.
- P. Ammann and J. Offutt. Introduction to Software Testing, 2nd Ed. Cambridge University Press, 2016.Google Scholar
Cross Ref
- Z. Awedikian, K. Ayari, and G. Antoniol. MC/DC automatic test input data generation. In F. Rothlauf, editor, GECCO, pages 1657--1664. ACM, 2009.Google ScholarDigital Library
- R. Bloem, K. Greimel, R. Koenighofer, and F. Roeck. Model-based MCDC testing of complex decisions for the Java Card applet firewall. In VALID, pages 1--6. IARIA, 2013.Google Scholar
- J. Chang and C. Huang. A study of enhanced MC/DC coverage criterion for software testing. In COMPSAC, pages 457--464. IEEE Comp. Soc., 2007.Google ScholarDigital Library
- T. Chen and M. Lau. An empirical study on the effectiveness of the greedy MUTP strategy. In SEEP, pages 338--344. IEEE, 1998.Google Scholar
- T. Chen, M. Lau, and Y. Yu. MUMCUT: a fault-based strategy for testing Boolean specifications. In APSEC, pages 606--613, Takamatsu, Japan, 1999. IEEE Comp. Soc.Google ScholarCross Ref
- T. Y. Chen and M. F. Lau. Dividing strategies for the optimization of a test suite. Inf. Process. Lett., 60(3):135--141, 1996.Google ScholarDigital Library
- J. J. Chilenski. An investigation of three forms of the modified condition decision coverage (MCDC) criterion. Technical Report DOT/FAA/AR-01/18, Office of Aviation Research, Federal Aviation Administration, April 2001.Google Scholar
- J. J. Chilenski and S. P. Miller. Applicability of modified condition/decision coverage to software testing. Softw. Eng. J., 9(5):193--200, 1994.Google ScholarCross Ref
- J. Flum and M. Grohe. Parameterized Complexity Theory. Springer, 2006.Google Scholar
- S. Hallé, E. L. Chance, and S. Gaboury. Graph methods for generating test cases with universal and existential constraints. In K. El-Fakih, G. D. Barlas, and N. Yevtushenko, editors, ICTSS, volume 9447 of LNCS, pages 55--70. Springer, 2015.Google Scholar
- S. Hallé, R. Khoury, and M. Awesso. Streamlining the inclusion of computer experiments in a research paper. Computer, 51(11):78--89, 2018.Google ScholarDigital Library
- S. Hallé. Boolean condition coverage with evaluation trees (LabPal bundle), 2022. Google ScholarCross Ref
- E. Halperin. Improved approximation algorithms for the vertex cover problem in graphs and hypergraphs. SIAM J. Comput., 31(5):1608--1623, 2002.Google ScholarDigital Library
- L. Hu, W. E. Wong, D. R. Kuhn, and R. Kacker. MCDC-Star: A white-box based automated test generation for high MC/DC coverage. In DSA, pages 102--112. IEEE, 2018.Google ScholarCross Ref
- G. K. Kaminski and P. Ammann. Reducing logic test set size while preserving fault detection. Softw. Test. Verification Reliab., 21(3):155--193, 2011.Google ScholarDigital Library
- G. K. Kaminski, G. Williams, and P. Ammann. Reconciling perspectives of software logic testing. Softw. Test. Verification Reliab., 18(3):149--188, 2008.Google ScholarDigital Library
- S. Kangoye, A. Todoskoff, and M. Barreau. Practical methods for automatic MC/DC test case generation of Boolean expressions. In Proc. AUTOTESTCON, pages 203--212. IEEE, 2015.Google ScholarDigital Library
- K. Kapoor and J. P. Bowen. A formal analysis of MCDC and RCDC test criteria. Softw. Test. Verification Reliab., 15(1):21--40, 2005.Google ScholarDigital Library
- G. Karakostas. A better approximation ratio for the vertex cover problem. In L. Caires, G. F. Italiano, L. Monteiro, C. Palamidessi, and M. Yung, editors, ICALP, volume 3580 of LNCS, pages 1043--1050. Springer, 2005.Google Scholar
- T. Kitamura, Q. Maissonneuve, E. Choi, C. Artho, and A. Gargantini. Optimal test suite generation for modified condition decision coverage using SAT solving. In B. Gallina, A. Skavhaug, and F. Bitsch, editors, SAFECOMP, volume 11093 of LNCS, pages 123--138. Springer, 2018.Google Scholar
- D. Kuhn, D. Wallace, and A. Gallo. Software fault interactions and implications for software testing. IEEE Trans. on Softw. Eng., 30(6):418--421, June 2004.Google ScholarDigital Library
- R. Kuhn. Automated combinatorial testing for software. https://csrc.nist.gov/Projects/automated-combinatorial-testing-for-software/downloadable-tools, Accessed February 27th, 2021.Google Scholar
- M. F. Lau and Y. Yu. On comparing testing criteria for logical decisions. In F. Kordon and Y. Kermarrec, editors, Ada-Europe, volume 5570 of LNCS, pages 44--58. Springer, 2009.Google Scholar
- Y. Lei, R. Kacker, D. R. Kuhn, V. Okun, and J. Lawrence. Ipog: A general strategy for t-way software testing. In ECBS, pages 549--556. IEEE, 2007.Google ScholarDigital Library
- D. Lopez-Escogido, J. Torres-Jimenez, E. Rodriguez-Tello, and N. Rangel-Valdez. Strength two covering arrays construction using a SAT representation. In MICAI, pages 44--53. Springer, 2008.Google ScholarDigital Library
- E. J. McCluskey, Jr. Minimization of Boolean functions. Bell System Tech. J., 35(6):1417--1444, November 1956.Google ScholarCross Ref
- K. J. Nurmela. Upper bounds for covering arrays by tabu search. Discrete applied mathematics, 138(1--2):143--152, 2004.Google Scholar
- T. K. Paul and M. F. Lau. A systematic literature review on modified condition and decision coverage. In Y. Cho, S. Y. Shin, S. Kim, C. Hung, and J. Hong, editors, SAC, pages 1301--1308. ACM, 2014.Google Scholar
- R. K. Singh, P. Chandra, and Y. Singh. An evaluation of boolean expression testing techniques. ACM SIGSOFT Softw. Eng. Notes, 31(5):1--6, 2006.Google ScholarDigital Library
- C. Sun, Y. Dong, R. Lai, K. Sim, and T. Chen. Analyzing and extending MUMCUT for fault-based testing of general boolean expressions. In CIT, pages 184--184. IEEE, 2006.Google ScholarDigital Library
- C.-a. Sun, K. Y. Sim, T. Tse, and T. Chen. An Empirical Evaluation and Analysis of the Fault-Detection Capability of MUMCUT for General Boolean Expressions. In ICS, pages 926--932, 2004.Google Scholar
- J. Torres-Jiménez and E. Rodriguez-Tello. New bounds for binary covering arrays using simulated annealing. Inf. Sci., 185(1):137--152, 2012.Google ScholarDigital Library
- S. Vilkomir and D. Anderson. Relationship between pair-wise and MC/DC testing: Initial experimental results. In ICST Workshops. IEEE, 2015.Google ScholarCross Ref
- S. A. Vilkomir and J. P. Bowen. From MC/DC to RC/DC: formalization and analysis of control-flow testing criteria. Formal Aspects Comput., 18(1):42--62, 2006.Google ScholarDigital Library
- E. J. Weyuker, T. Goradia, and A. Singh. Automatically generating test data from a boolean specification. IEEE Trans. Software Eng., 20(5):353--363, 1994.Google ScholarDigital Library
- Y. Yu and M. F. Lau. A comparison of MC/DC, MUMCUT and several other coverage criteria for logical decisions. J. Syst. Softw., 79(5):577--590, 2006.Google ScholarDigital Library
- Y. Yu, M. F. Lau, and T. Y. Chen. Automatic generation of test cases from Boolean specifications using the MUMCUT strategy. J. Syst. Softw., 79(6):820--840, 2006.Google ScholarDigital Library
- Yu Lei and K. C. Tai. In-parameter-order: a test generation strategy for pairwise testing. In HASE, pages 254--261, Nov 1998.Google ScholarDigital Library
Index Terms
- Test suite generation for boolean conditions with equivalence class partitioning
Recommendations
A Novel Approach of Equivalence Class Partitioning for Numerical Input
Software testing is an indispensible phase of software development that supports not only productivity but also quality assurance from small to large scale software products. Software Systems essentially need an effective way to develop a strategy for ...
A detailed investigation of the effectiveness of whole test suite generation
A common application of search-based software testing is to generate test cases for all goals defined by a coverage criterion (e.g., lines, branches, mutants). Rather than generating one test case at a time for each of these goals individually, whole ...
Regression test suite prioritization using system models
During regression testing, a modified system is often retested using an existing test suite. Since the size of the test suite may be very large, testers are interested in detecting faults in the modified system as early as possible during this retesting ...
Comments