Kai Pan
Xintao Wu
Tao Xie
Skip Abstract Section
Abstract
Testing database applications typically requires the generation of tests consisting of both program inputs and database states. Recently, a testing technique called Dynamic Symbolic Execution (DSE) has been proposed to reduce manual effort in test generation for software applications. However, applying DSE to generate tests for database applications faces various technical challenges. For example, the database application under test needs to physically connect to the associated database, which may not be available for various reasons. The program inputs whose values are used to form the executed queries are not treated symbolically, posing difficulties for generating valid database states or appropriate database states for achieving high coverage of query-result-manipulation code. To address these challenges, in this article, we propose an approach called SynDB that synthesizes new database interactions to replace the original ones from the database application under test. In this way, we bridge various constraints within a database application: query-construction constraints, query constraints, database schema constraints, and query-result-manipulation constraints. We then apply a state-of-the-art DSE engine called Pex for .NET from Microsoft Research to generate both program inputs and database states. The evaluation results show that tests generated by our approach can achieve higher code coverage than existing test generation approaches for database applications.
- Rafi Ahmed, Allison W. Lee, Andrew Witkowski, Dinesh Das, Hong Su, Mohamed Zaït, and Thierry Cruanes. 2006. Cost-based query transformation in Oracle. In Proceedings of the International Conference on Very Large Data Bases (VLDB). 1026--1036. Google Scholar
Digital Library
- Carsten Binnig, Donald Kossmann, and Eric Lo. 2007a. Reverse query processing. In Proceedings of the IEEE International Conference on Data Engineering (ICDE). 506--515.Google ScholarCross Ref
- Carsten Binnig, Donald Kossmann, Eric Lo, and M. Tamer Özsu. 2007b. QAGen: Generating query-aware test databases. In Proceedings of the ACM SIGMOD Conference. 341--352. Google ScholarDigital Library
- María José Suárez Cabal and Javier Tuya. 2004. Using an SQL coverage measurement for testing database applications. In Proceedings of the ACM SIGSOFT Sympsium on the Foundations of Software Engineering (SIGSOFT-FSE). 253--262. Google ScholarDigital Library
- David Chays, Yuetang Deng, Phyllis G. Frankl, Saikat Dan, Filippos I. Vokolos, and Elaine J. Weyuker. 2004. An AGENDA for testing relational database applications. Softw. Test. Verif. Reliab. 14, 1, 17--44. Google ScholarDigital Library
- David Chays, John Shahid, and Phyllis G. Frankl. 2008. Query-based test generation for database applications. In Proceedings of the International Workshop on Testing Database Systems (DBTest). 6. Google ScholarDigital Library
- Sarah R. Clark, Jake Cobb, Gregory M. Kapfhammer, James A. Jones, and Mary Jean Harrold. 2011. Localizing SQL faults in database applications. In Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE). 213--222. Google ScholarDigital Library
- L. A. Clarke. 1976. A system to generate test data and symbolically execute programs. IEEE Trans. Softw. Eng. 2, 3, 215--222. Google ScholarDigital Library
- Michael A. Cusumano and Richard W. Selby. 1997. How Microsoft builds software. Commun. ACM 40, 6 (1997), 53--61. Google ScholarDigital Library
- U. Dayal. 1987. Of nests and trees: A unified approach to processing queries that contain nested subqueries, aggregates, and quantifiers. In Proceedings of the International Conference on Very Large Data Bases (VLDB). 197--208. Google ScholarDigital Library
- Claudio de la Riva, María José Suárez Cabal, and Javier Tuya. 2010. Constraint-based test database generation for SQL queries. In Proceedings of the International Workshop on Automation of Software Test (AST). 67--74. Google ScholarDigital Library
- Yuetang Deng and David Chays. 2005. Testing database transactions with AGENDA. In Proceedings of the International Conference on Software Engineering (ICSE). 78--87. Google ScholarDigital Library
- Michael Emmi, Rupak Majumdar, and Koushik Sen. 2007. Dynamic test input generation for database applications. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA). 151--162. Google ScholarDigital Library
- Patrice Godefroid, Nils Klarlund, and Koushik Sen. 2005. DART: Directed automated random testing. In Proceedings of the Programming Language Design and Implementation (PLDI). 213--223. Google ScholarDigital Library
- R. L. Goodstein. 2007. Boolean Algebra. Dover Publications.Google Scholar
- Mark Grechanik, Christoph Csallner, Chen Fu, and Qing Xie. 2010. Is data privacy always good for software testing? In Proceedings of the IEEE International Symposium on Software Reliability Engineering (ISSRE). 368--377. Google ScholarDigital Library
- Bhanu Pratap Gupta, Devang Vira, and S. Sudarshan. 2010. X-data: Generating test data for killing SQL mutants. In Proceedings of the IEEE International Conference on Data Engineering (ICDE). 876--879.Google Scholar
- William G. J. Halfond and Alessandro Orso. 2006. Command-form coverage for testing database applications. In Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE). 69--80. Google ScholarDigital Library
- Shadi Abdul Khalek, Bassem Elkarablieh, Yai O. Laleye, and Sarfraz Khurshid. 2008. Query-aware test generation using a relational constraint solver. In Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE). 238--247. Google ScholarDigital Library
- Won Kim. 1982. On Optimizing an SQL-like nested query. ACM Trans. Datab. Syst. 7, 3 (1982), 443--469. Google ScholarDigital Library
- J. C. King. 1976. Symbolic execution and program testing. Commun. ACM, 19, 7, 385--394. Google ScholarDigital Library
- Chengkai Li and Christoph Csallner. 2010. Dynamic symbolic database application testing. In Proceedings of the International Workshop on Testing Database Systems (DBTest). 1--6. Google ScholarDigital Library
- Microsoft. 2007. Pex: Dynamic analysis and test generation for .NET. Microsoft Research Foundation of Software Engineering Group.Google Scholar
- Microsoft. 2012a. DataTable. Microsoft MSDN. http://msdn.microsoft.com/en-us/library/system.data. datatable.aspx. (Last accessed May 2012).Google Scholar
- Microsoft. 2012b. .NET framework data provider for SQL server. Microsoft MSDN. (May 2012).Google Scholar
- Kai Pan, Xintao Wu, and Tao Xie. 2011a. Database state generation via dynamic symbolic execution for coverage criteria. In Proceedings of the International Workshop on Testing Database Systems (DBTest). 1--6. Google ScholarDigital Library
- Kai Pan, Xintao Wu, and Tao Xie. 2011b. Generating program inputs for database application testing. In Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE). 73--82. Google ScholarDigital Library
- Koushik Sen, Darko Marinov, and Gul Agha. 2005. CUTE: A concolic unit testing engine for C. In Proceedings of the Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE). 263--272. Google ScholarDigital Library
- Kunal Taneja, Mark Grechanik, Rayid Ghani, and Tao Xie. 2011. Testing software in age of data privacy: A balancing act. In Proceedings of the Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE). 201--211. Google ScholarDigital Library
- Kunal Taneja, Yi Zhang, and Tao Xie. 2010. MODA: Automated test generation for database applications via mock objects. In Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE). 289--292. Google ScholarDigital Library
- G. Tassey. 2002. The economic impacts of inadequate infrastructure for software testing. Tech. Repport. NIST Planning. 02-3, National Institute of Standards and Technology.Google Scholar
- Nikolai Tillmann and Jonathan de Halleux. 2008. Pex-White Box test generation for .NET. In Proceedings of the International Conference on Tests and Proofs (TAP). 134--153. Google ScholarDigital Library
- Nikolai Tillmann and Wolfram Schulte. 2006. Mock-object generation with behavior. In Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE). 365--368. Google ScholarDigital Library
- Javier Tuya, María José Suárez Cabal, and Claudio de la Riva. 2010. Full predicate coverage for testing SQL database queries. Softw. Test. Verif. Reliab. 20, 3, 237--288. Google ScholarDigital Library
- Margus Veanes, Pavel Grigorenko, Peli de Halleux, and Nikolai Tillmann. 2009. Symbolic query exploration. In Proceedings of the International Conference on Formal Engineering Methods (ICFEM). 49--68. Google ScholarDigital Library
- David Willmor and Suzanne M. Embury. 2006. An intensional approach to the specification of test cases for database applications. In Proceedings of the International Conference on Software Engineering (ICSE). 102--111. Google ScholarDigital Library
- Xintao Wu, Chintan Sanghvi, Yongge Wang, and Yuliang Zheng. 2005. Privacy aware data generation for testing database applications. In Proceedings of the International Database Engineering and Applications Symposium (IDEAS). 317--326. Google ScholarDigital Library
- Xintao Wu, Yongge Wang, Songtao Guo, and Yuliang Zheng. 2007. Privacy preserving database generation for database application testing. Fundam. Inform. 78, 4, 595--612. Google ScholarDigital Library
- Tao Xie, Nikolai Tillmann, Peli de Halleux, and Wolfram Schulte. 2009. Fitness-guided path exploration in dynamic symbolic execution. In Proceedings of the IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). 359--368.Google ScholarCross Ref
- Linghao Zhang, Xiaoxing Ma, Jian Lu, Nikolai Tillmann, Jonathan de Halleux, and Tao Xie. 2012. Environment modeling for automated testing of cloud applications. IEEE Softw. Special Issue on Software Engineering for Cloud Computing 29, 2, 30--35. Google ScholarDigital Library
- Chixiang Zhou and Phyllis G. Frankl. 2009. Mutation testing for Java database applications. In Proceedings of the IEEE International Conference on Software Testing, Verification and Validation (ICST). 396--405. Google ScholarDigital Library
- Chixiang Zhou and Phyllis G. Frankl. 2011. Inferential checking for mutants modifying database states. In Proceedings of the IEEE International Conference on Software Testing, Verification and Validation (ICST). 259--268. Google ScholarDigital Library
Index Terms
- Guided test generation for database applications via synthesized database interactions
Recommendations
Dynamic test input generation for database applications
ISSTA '07: Proceedings of the 2007 international symposium on Software testing and analysisWe describe an algorithm for automatic test input generation for database applications. Given a program in an imperative language that interacts with a database through API calls, our algorithm generates both input data for the program as well as ...
Constraint-based test database generation for SQL queries
AST '10: Proceedings of the 5th Workshop on Automation of Software TestPopulating test databases with meaningful test data is a difficult task as it involves generating data for many joined tables that must be diverse enough to be able to reveal faults and small enough to make the testing process efficient. This paper ...
Program-input generation for testing database applications using existing database states
Testing is essential for quality assurance of database applications. Achieving high code coverage of the database applications is important in testing. In practice, there may exist a copy of live databases that can be used for database application ...
Comments