Abstract
By the very definition of complex systems, complex behaviour emerges from the interactions between the individual parts. This emergent behaviour may be difficult or impossible to predict by analysing the parts. As a result, systematic and thorough testing of the interactions of complex systems, including multi-agent systems, is an important part of the verification and validation process. This paper defines two sets of test coverage criteria for multi-agent interaction testing. The first uses only the protocol specification, while the second considers also the plans that generate and receive the messages in the protocol. We describe how an existing debugging agent can be used as a test oracle for assessing correctness of a test, and how the Petri Net representation of the debugging agent can be annotated to support test coverage measurements. This work both specifies, and shows how to measure, the degree of thoroughness of a set of test cases. It also provides a basis for the future specification of test case input, designed to provide good coverage.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Braberman, V., Felder, M., Marre, M.: Testing timing behavior of real-time software. In: International Software Quality Week (1997)
Caire, G., Cossentino, M., Negri, A., Poggi, A., Turci, P.: Multi-Agent Systems Implementation and Testing. In: The Fourth International Symposium: From Agent Theory to Agent Implementation, April 14-16 (2004)
Coelho, R., Kulesza, U., von Staa, A., Lucena, C.: Unit testing in multi-agent systems using mock agents and aspects. In: Proc. of the 2006 Intl. Workshop on Software Engineering for Large-Scale Multi-Agent Systems, pp. 83–90 (2006)
DeLoach, S., Padgham, L., Perini, A., Susi, A., Thangarajah, J.: Using three AOSE toolkits to develop a sample design. International Journal of Agent-Oriented Software Engineering 3(4), 416–476 (2009)
Desel, J., Oberweis, A., Zimmer, T., Zimmermann, G.: Validation of information system models: Petri Nets and test case generation. In: IEEE International Conference on Systems, Man, and Cybernetics, vol. 4, pp. 3401–3406. IEEE, Los Alamitos (2002)
FIPA. FIPA ACL message structure specification. Standard SC00061G, Foundation for Intelligent Physical Agents (December 2002)
FIPA. FIPA query interaction protocol specification. Standard SC00027H, Foundation for Intelligent Physical Agents (December 2003)
Hatley, D.J., Pirbhai, I.A.: Strategies for real-time system specification. Dorset House, New York (1988)
Huget, M., Odell, J.: Representing agent interaction protocols with Agent UML. In: Odell, J.J., Giorgini, P., Müller, J.P. (eds.) AOSE 2004. LNCS, vol. 3382, pp. 16–30. Springer, Heidelberg (2005)
IEEE. IEEE standard glossary of software engineering terminology. Technical Report 610.12-1990, Institute of Electrical and Electronic Engineers (1990)
Jensen, K.: Coloured Petri Nets. Springer, Heidelberg (1997)
Kissoum, Y., Sahnoun, Z.: A recursive colored Petri Nets semantics for AUML as base of test case generation. In: IEEE/ACS International Conference on Computer Systems and Applications, pp. 785–792. IEEE, Los Alamitos (2008)
Low, C., Chen, T.Y., Ronnquist, R.: Automated test case generation for BDI agents. Autonomous Agents and Multi-Agent Systems 2(4), 311–332 (1999)
Myers, G.J.: The Art of Software Testing. Wiley, New York (1979)
Myers, G.J., Sandler, C., Badgett, T., Thomas, T.M.: The Art of Software Testing, 2nd edn. Wiley, Chichester (2004)
Nguyen, C., Perini, A., Tonella, P.: Automated continuous testing of multi-agent systems. In: Fifth European Workshop on Multi-Agent Systems, Hammamet, Tunisia (December 2007)
Nguyen, C., Perini, A., Tonella, P.: eCAT: a tool for automating test case generation and execution in testing multi-agent systems (demo paper). In: Proceedings of AAMAS 2008, Estoril, Portugal, pp. 1669–1670 (2008)
Nguyen, C., Perini, A., Tonella, P.: Ontology-based test generation for multi-agent systems. In: Proceedings of AAMAS 2008, pp. 1315–1320 (2008)
Poutakidis, D., Padgham, L., Winikoff, M.: Debugging multi-agent systems using design artifacts: The case of interaction protocols. In: Alonso, E., Kudenko, D., Kazakov, D. (eds.) AAMAS 2000 and AAMAS 2002. LNCS (LNAI), vol. 2636, pp. 960–967. Springer, Heidelberg (2003)
Spillner, A.: Test criteria and coverage measures for software integration testing. Software Quality Journal 4(4), 275–286 (1995)
Tiryaki, A., Öztuna, S., Dikenelli, O., Cenk Erdur, R.: SUNIT: A unit testing framework for test driven development of multi-agent systems. In: Padgham, L., Zambonelli, F. (eds.) AOSE VII / AOSE 2006. LNCS, vol. 4405, pp. 156–173. Springer, Heidelberg (2007)
Tjell, S.: Model-based testing of a reactive system with coloured Petri Nets. Proceedings of INFORMATIK 94, 274–281 (2006)
Utting, M., Legeard, B.: Practical Model-Based Testing: A Tools Approach. Morgan-Kaufmann, San Francisco (2007)
Zhang, Z., Thangarajah, J., Padgham, L.: Automated unit testing for agent systems. In: 2nd International Working Conference on Evaluation of Novel Approaches to Software Engineering, Spain, pp. 10–18 (July 2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Miller, T., Padgham, L., Thangarajah, J. (2011). Test Coverage Criteria for Agent Interaction Testing. In: Weyns, D., Gleizes, MP. (eds) Agent-Oriented Software Engineering XI. AOSE 2010. Lecture Notes in Computer Science, vol 6788. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-22636-6_6
Download citation
DOI: https://doi.org/10.1007/978-3-642-22636-6_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-22635-9
Online ISBN: 978-3-642-22636-6
eBook Packages: Computer ScienceComputer Science (R0)