Wontae Choi
Abstract
Smartphones and tablets with rich graphical user interfaces (GUI) are becoming increasingly popular. Hundreds of thousands of specialized applications, called apps, are available for such mobile platforms. Manual testing is the most popular technique for testing graphical user interfaces of such apps. Manual testing is often tedious and error-prone. In this paper, we propose an automated technique, called Swift-Hand, for generating sequences of test inputs for Android apps. The technique uses machine learning to learn a model of the app during testing, uses the learned model to generate user inputs that visit unexplored states of the app, and uses the execution of the app on the generated inputs to refine the model. A key feature of the testing algorithm is that it avoids restarting the app, which is a significantly more expensive operation than executing the app on a sequence of inputs. An important insight behind our testing algorithm is that we do not need to learn a precise model of an app, which is often computationally intensive, if our goal is to simply guide test execution into unexplored parts of the state space. We have implemented our testing algorithm in a publicly available tool for Android apps written in Java. Our experimental results show that we can achieve significantly better coverage than traditional random testing and L*-based testing in a given time budget. Our algorithm also reaches peak coverage faster than both random and L*-based testing.
- Managing the Activity Lifecycle. http://developer.android.com/training/basics/activity-lifecycle/index.html.Google Scholar
- MonkeyRunner. http://developer.android.com/tools/help/monkeyrunner_concepts.html.Google Scholar
- UI/Application Exerciser Monkey. http://developer.android.com/tools/help/monkey.html.Google Scholar
- axml, read and write Android binary xml files. http://code.google.com/p/axml/, 2012.Google Scholar
- R. Alur, P. Cerny, P. Madhusudan, and W. Nam. Synthesis of interface specifications for java classes. In POPL, pages 98--109, 2005. Google ScholarDigital Library
- D. Amalfitano, A. R. Fasolino, P. Tramontana, S. D. Carmine, and A. M. Memon. Using GUI ripping for automated testing of Android applications. In ASE, pages 258--261, 2012. Google ScholarDigital Library
- S. Anand, M. Naik, M. J. Harrold, and H. Yang. Automated concolic testing of smartphone apps. In SIGSOFT FSE, page 59, 2012. Google ScholarDigital Library
- A. A. Andrews, J. Offutt, and R. T. Alexander. Testing web applications by modeling with FSMs. Software and System Modeling, 4(3):326--345, 2005.Google ScholarDigital Library
- D. Angluin. Inference of reversible languages. J. ACM, 29(3):741--765, July 1982. Google ScholarDigital Library
- D. Angluin. Learning regular sets from queries and counterex-amples. Inf. Comput., 75(2):87--106, 1987. Google ScholarDigital Library
- F. Belli. Finite-state testing and analysis of graphical user interfaces. In 12th International Symposium on Software Reliability Engineering (ISSRE'01), page 34. IEEE Computer Society, 2001. Google ScholarDigital Library
- T. Berg, O. Grinchtein, B. Jonsson, M. Leucker, H. Raffelt, and B. Steffen. On the correspondence between conformance testing and regular inference. In FASE, pages 175--189, 2005. Google ScholarDigital Library
- J. M. Cobleigh, D. Giannakopoulou, and C. S. Pasareanu. Learning assumptions for compositional verification. In TACAS, pages 331--346, 2003. Google ScholarDigital Library
- F. Coste, D. Fredouille, C. Kermorvant, and C. de la Higuera. Introducing domain and typing bias in automata inference. In ICGI, pages 115--126, 2004.Google Scholar
- J. N. Departarnento and P. Garcia. Identifying regular languages in polynomial. In Advances in Structural and Syntactic Pattern Recognition, volume 5 of Series in Machine Perception and Artificial Intelligence, pages 99--108. World Scientific, 1992.Google Scholar
- P. Godefroid, N. Klarlund, and K. Sen. DART: directed automated random testing. In PLDI, pages 213--223, 2005. Google ScholarDigital Library
- A. Groce, D. Peled, and M. Yannakakis. AMC: An adaptive model checker. In CAV, pages 521--525, 2002. Google ScholarDigital Library
- A. Groce, D. Peled, and M. Yannakakis. Adaptive model checking. Logic Journal of the IGPL, 14(5):729--744, 2006.Google ScholarCross Ref
- R. Groz, M.-N. Irfan, and C. Oriat. Algorithmic improvements on regular inference of software models and perspectives for security testing. In ISoLA (1), pages 444--457, 2012. Google ScholarDigital Library
- C. Hu and I. Neamtiu. A GUI bug finding framework for Android applications. In SAC, pages 1490--1491, 2011. Google ScholarDigital Library
- B. Lambeau, C. Damas, and P. Dupont. State-merging DFA induction algorithms with mandatory merge constraints. In ICGI, pages 139--153, 2008. Google ScholarDigital Library
- K. Lang, B. Pearlmutter, and R. Price. Results of the Abbadingo One DFA learning competition and a new evidence-driven state merging algorithm, 1998.Google ScholarCross Ref
- K. J. Lang. Random DFA's can be approximately learned from sparse uniform examples. In COLT, pages 45--52, 1992. Google ScholarDigital Library
- X. Leroy. Java bytecode verification: algorithms and formalizations. Journal of Automated Reasoning, 30(3--4):235--269, 2003. Google ScholarDigital Library
- A. MacHiry, R. Tahiliani, and M. Naik. Dynodroid: An input generation system for Android apps. In SIGSOFT FSE, pages 224--235, 2013. Google ScholarDigital Library
- K. Meinke and M. A. Sindhu. Incremental learning-based testing for reactive systems. In TAP, pages 134--151, 2011. Google ScholarDigital Library
- K. Meinke and N. Walkinshaw. Model-based testing and model inference. In ISoLA (1), pages 440--443, 2012. Google ScholarDigital Library
- A. M. Memon. An event-flow model of GUI-based applications for testing. Softw. Test., Verif. Reliab., 17(3):137--157, 2007. Google ScholarDigital Library
- A. Mesbah, A. van Deursen, and S. Lenselink. Crawling Ajax-based web applications through dynamic analysis of user interface state changes. TWEB, 6(1):3, 2012. Google ScholarDigital Library
- N. Mirzaei, S. Malek, C. S. Pasareanu, N. Esfahani, and R. Mahmood. Testing Android apps through symbolic execution. ACM SIGSOFT Software Engineering Notes, 37(6):1--5, 2012. Google ScholarDigital Library
- J. Nevo and P. Crégut. ASMDEX. http://asm.ow2.org/asmdex-index.html, 2012.Google Scholar
- W. M. Newman. A system for interactive graphical programming. In Proc. of the spring joint computer conference (AFIPS'68 (Spring)), pages 47--54. ACM, 1968. Google ScholarDigital Library
- C. D. Nguyen, A. Marchetto, and P. Tonella. Combining model-based and combinatorial testing for effective test case generation. In ISSTA, pages 100--110, 2012. Google ScholarDigital Library
- D. Peled, M. Y. Vardi, and M. Yannakakis. Black box checking. In FORTE, pages 225--240, 1999. Google ScholarDigital Library
- H. Raffelt, B. Steffen, and T. Margaria. Dynamic testing via automata learning. In Haifa Verification Conference, pages 136--152, 2007. Google ScholarDigital Library
- V. Rastogi, Y. Chen, and W. Enck. Appsplayground: automatic security analysis of smartphone applications. In CODASPY, pages 209--220, 2013. Google ScholarDigital Library
- H. Reza, S. Endapally, and E. Grant. A model-based approach for testing GUI using hierarchical predicate transition nets. In International Conference on Information Technology (ITNG'07), pages 366--370. IEEE Computer Society, 2007. Google ScholarDigital Library
- R. L. Rivest and R. E. Schapire. Inference of finite automata using homing sequences (extended abstract). In STOC, pages 411--420, 1989. Google ScholarDigital Library
- R. K. Shehady and D. P. Siewiorek. A methodology to automate user interface testing using variable finite state machines. In FTCS, pages 80--88, 1997. Google ScholarDigital Library
- T. Takala, M. Katara, and J. Harty. Experiences of system- level model-based GUI testing of an Android application. In ICST, pages 377--386, 2011. Google ScholarDigital Library
- N. Walkinshaw, K. Bogdanov, J. Derrick, and J. Paris. Increasing functional coverage by inductive testing: A case study. In ICTSS, pages 126--141, 2010. Google ScholarDigital Library
- N. Walkinshaw, J. Derrick, and Q. Guo. Iterative refinement of reverse-engineered models by model-based testing. In FM, pages 305--320, 2009. Google ScholarDigital Library
- L. White and H. Almezen. Generating test cases for GUI responsibilities using complete interaction sequences. In 11th International Symposium on Software Reliability Engineering (ISSRE'00), page 110. IEEE Computer Society, 2000. Google ScholarDigital Library
- T. Xie and D. Notkin. Mutually enhancing test generation and specification inference. In FATES, pages 60--69, 2003.Google Scholar
- W. Yang, M. R. Prasad, and T. Xie. A grey-box approach for automated GUI-model generation of mobile applications. In FASE, pages 250--265, 2013. Google ScholarDigital Library
- X. Yuan, M. Cohen, and A. M. Memon. Covering array sampling of input event sequences for automated GUI testing. In ASE '07: Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering, pages 405--408, New York, NY, USA, 2007. ACM. Google ScholarDigital Library
- X. Yuan and A. M. Memon. Iterative execution-feedback model-directed GUI testing. Information & Software Technology, 52(5):559--575, 2010. Google ScholarDigital Library
Index Terms
- Guided GUI testing of android apps with minimal restart and approximate learning
Recommendations
Targeted and depth-first exploration for systematic testing of android apps
OOPSLA '13: Proceedings of the 2013 ACM SIGPLAN international conference on Object oriented programming systems languages & applicationsSystematic exploration of Android apps is an enabler for a variety of app analysis and testing tasks. Performing the exploration while apps run on actual phones is essential for exploring the full range of app capabilities. However, exploring real-world ...
Guided, stochastic model-based GUI testing of Android apps
ESEC/FSE 2017: Proceedings of the 2017 11th Joint Meeting on Foundations of Software EngineeringMobile apps are ubiquitous, operate in complex environments and are developed under the time-to-market pressure. Ensuring their correctness and reliability thus becomes an important challenge. This paper introduces Stoat, a novel guided approach to ...
Dynodroid: an input generation system for Android apps
ESEC/FSE 2013: Proceedings of the 2013 9th Joint Meeting on Foundations of Software EngineeringWe present a system Dynodroid for generating relevant inputs to unmodified Android apps. Dynodroid views an app as an event-driven program that interacts with its environment by means of a sequence of events through the Android framework. By ...
Comments