Aspects of software quality applied to the process of agile software development: a systematic literature review

Abstract

The importance of quality in software engineering is widely recognized today. As a result, agile methods have incorporated quality practices in the software development process to achieve quality software quickly and in harmony with rapidly changing customer requirements. Experts have published a substantial amount of information related to the diverse aspects of software quality and agile development. This work aims to perform a systematic literature review of the following characteristics: critical success factors, metrics to measure quality, quality attributes, agile practices, quality models, and agile principles considered by these models. 773 papers related to quality in the agile development process were identified, of which, for the preparation of this article, 71 papers were selected. This study catalogs 118 critical success factors, 137 agile practices, 102 metrics, and 70 quality attributes that affect the agile software development. In addition, 14 quality models, which do not cover all the aspects of quality related to agile development, were studied to prepare this paper. 28.17% of the studies analyzed consider the agile practices as they affect the quality of software, which is the most studied aspect.

Appendices

Appendix A: Studies included in the review

[R1] Rico, D.F.: Effects of Agile Methods on Website Quality for Electronic Commerce. In: Hawaii International Conference on System Sciences, Proceedings of the 41st Annual. pp. 463–463 (2008).

[R2] Chow, T., Cao, D.-B.: A survey study of critical success factors in agile software projects. J. Syst. Softw. 81, 961–971 (2008).

[R3] Clarke, P., O’Connor, R.V.: The situational factors that affect the software development process: Towards a comprehensive reference framework. Inf. Softw. Technol. 54, 433–447 (2012).

[R4] Bermejo, P.H. de S., Zambalde, A.L., Tonelli, A.O., Souza, S.A., Zuppo, L.A., Rosa, P.L.: Agile Principles and Achievement of Success in Software Development: A Quantitative Study in Brazilian Organizations. Procedia Technol. 16, 718–727 (2014).

[R5] Acuña, S.T., Gómez, M., Juristo, N.: How do personality, team processes and task characteristics relate to job satisfaction and software quality? Inf. Softw. Technol. 51, 627–639 (2009).

[R6] Misra, S.C., Kumar, V., Kumar, U.: Identifying some important success factors in adopting agile software development practices. J. Syst. Softw. 82, 1869–1890 (2009).

[R7] Acuña, S.T., Gómez, M.N., Hannay, J.E., Juristo, N., Pfahl, D.: Are team personality and climate related to satisfaction and software quality? Aggregating results from a twice replicated experiment. Inf. Softw. Technol. 57, 141–156 (2015).

[R8] Acuña, S.T., Gómez, M., Juristo, N.: Towards understanding the relationship between team climate and software quality—a quasi-experimental study. Empir. Softw. Eng. 13, 401–434 (2008).

[R9] Begier, B.: Users’ involvement may help respect social and ethical values and improve software quality. Inf. Syst. Front. 12, 389–397 (2010).

[R10] Fernández-Sanz, L., Misra, S.: Influence of Human Factors in Software Quality and Productivity. In: Murgante, B., Gervasi, O., Iglesias, A., Taniar, D., and Apduhan, B.O. (eds.) Computational Science and Its Applications - ICCSA 2011. pp. 257–269. Springer Berlin Heidelberg (2011).

[R11] Senapathi, M., Srinivasan, A.: An Empirical Investigation of the Factors Affecting Agile Usage. In: Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering. p. 10:1–10:10. ACM, New York, NY, USA (2014).

[R12] De Souza Mariz, L.M.R., França, A.C.C., da Silva, F.Q.B.: An Empirical Study on the Relationship between the Use of Agile Practices and the Success of Software Projects that Use Scrum. In: 2010 Brazilian Symposium on Software Engineering (SBES). pp. 110–117 (2010).

[R13] Jain, P., Sharma, A., Ahuja, L.: ISM based identification of quality attributes for agile development. In: 2016 5th International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO). pp. 615–619 (2016).

[R14] Foerderer, J., Kude, T., Mithas, S., Heinzl, A.: How Temporal Work Styles and Product Modularity Influence Software Quality and Job Satisfaction. Presented at the Proceedings of the 2016 ACM SIGMIS Conference on Computers and People Research February 6 (2016).

[R15] Greening, D.R.: Agile Enterprise Metrics. In: 2015 48th Hawaii International Conference on System Sciences (HICSS). pp. 5038–5044 (2015).

[R16] Tommy, R., Mhaisekar, M., Kallepally, S., Varghese, L., Ahmed, S., Somaraju, M.D.: Dynamic quality control in agile methodology for improving the quality. In: 2015 IEEE International Conference on Computer Graphics, Vision and Information Security (CGVIS). pp. 233–236 (2015).

[R17] Yamada, S., Kii, R.: Software quality analysis for agile development. In: 2015 4th International Conference on Reliability, Infocom Technologies and Optimization (ICRITO) (Trends and Future Directions). pp. 1–5 (2015).

[R18] Olague, H.M., Etzkorn, L.H., Gholston, S., Quattlebaum, S.: Empirical Validation of Three Software Metrics Suites to Predict Fault-Proneness of Object-Oriented Classes Developed Using Highly Iterative or Agile Software Development Processes. IEEE Trans. Softw. Eng. 33, 402–419 (2007).

[R19] Hartmann, D., Dymond, R.: Appropriate Agile Measurement: Using Metrics and Diagnostics to Deliver Business Value. Presented at the Proceedings of the conference on AGILE 2006 July 23 (2006).

[R20] Roden, P.L., Virani, S., Etzkorn, L.H., Messimer, S.: An Empirical Study of the Relationship of Stability Metrics and the QMOOD Quality Models Over Software Developed Using Highly Iterative or Agile Software Processes. In: Seventh IEEE International Working Conference on Source Code Analysis and Manipulation, 2007. SCAM 2007. pp. 171–179 (2007).

[R21] Beckhaus, A., Karg, L.M., Hanselmann, G.: Applicability of Software Reliability Growth Modeling in the Quality Assurance Phase of a Large Business Software Vendor. In: Computer Software and Applications Conference, 2009. COMPSAC’09. 33rd Annual IEEE International. pp. 209–215 (2009).

[R22] Schwanke, R., Xiao, L., Cai, Y.: Measuring architecture quality by structure plus history analysis. In: 2013 35th International Conference on Software Engineering (ICSE). pp. 891–900 (2013).

[R23] Jinzenji, K., Hoshino, T., Williams, L., Takahashi, K.: Metric-Based Quality Evaluations for Iterative Software Development Approaches Like Agile. In: 2012 IEEE 23rd International Symposium on Software Reliability Engineering Workshops (ISSREW). pp. 54–63 (2012).

[R24] Boerman, M.P., Lubsen, Z., Tamburri, D.A., Visser, J.: Measuring and Monitoring Agile Development Status. In: 2015 IEEE/ACM 6th International Workshop on Emerging Trends in Software Metrics. pp. 54–62 (2015).

[R25] Layman, L., Williams, L., Cunningham, L.: Motivations and measurements in an agile case study. J. Syst. Archit. 52, 654–667 (2006).

[R26] Kunz, M., Dumke, R.R., Schmietendorf, A.: How to Measure Agile Software Development. In: Cuadrado-Gallego, J.J., Braungarten, R., Dumke, R.R., and Abran, A. (eds.) Software Process and Product Measurement. pp. 95–101. Springer Berlin Heidelberg (2008).

[R27] Ambu, W., Concas, G., Marchesi, M., Pinna, S.: Studying the Evolution of Quality Metrics in an Agile/Distributed Project. In: Abrahamsson, P., Marchesi, M., and Succi, G. (eds.) Extreme Programming and Agile Processes in Software Engineering. pp. 85–93. Springer Berlin Heidelberg (2006).

[R28] Singh, Y., Kaur, A., Malhotra, R.: Empirical validation of object-oriented metrics for predicting fault proneness models. Softw. Qual. J. 18, 3–35 (2010).

[R29] Destefanis, G., Counsell, S., Concas, G., Tonelli, R.: Software Metrics in Agile Software: An Empirical Study. In: Cantone, G. and Marchesi, M. (eds.) Agile Processes in Software Engineering and Extreme Programming. pp. 157–170. Springer International Publishing (2014).

[R30] Hulkko, H., Abrahamsson, P.: A Multiple Case Study on the Impact of Pair Programming on Product Quality. In: Proceedings of the 27th International Conference on Software Engineering. pp. 495–504. ACM, New York, NY, USA (2005).

[R31] Kupiainen, E., Mäntylä, M.V., Itkonen, J.: Why Are Industrial Agile Teams Using Metrics and How Do They Use Them? In: Proceedings of the 5th International Workshop on Emerging Trends in Software Metrics. pp. 23–29. ACM, New York, NY, USA (2014).

[R32] Kunz, M., Dumke, R.R., Zenker, N.: Software Metrics for Agile Software Development. In: 19th Australian Conference on Software Engineering, 2008. ASWEC 2008. pp. 673–678 (2008).

[R33] Forouzani, S., Chiam, Y.K., Forouzani, S.: Method for Assessing Software Quality Using Source Code Analysis. Presented at the Proceedings of the Fifth International Conference on Network, Communication and Computing December 17 (2016).

[R34] Kovacheva, T., Todorov, N.: Optimizing software development process: A case study for integrated Agile-CMMI process model.

[R35] Bougroun, Z., Zeaaraoui, A., Bouchentouf, T.: The projection of the specific practices of the third level of CMMI model in agile methods: Scrum, XP and Kanban. In: Information Science and Technology (CIST), 2014 Third IEEE International Colloquium in. pp. 174–179 (2014).

[R36] Marrington, A., Hogan, J.M., Thomas, R.: Quality assurance in a student-based agile software engineering process. In: Software Engineering Conference, 2005. Proceedings. 2005 Australian. pp. 324–331 (2005).

[R37] Opelt, K., Beeson, T.: Agile Teams Require Agile QA: How to Make it Work, An Experience Report. In: Agile, 2008. AGILE’08. Conference. pp. 229–232 (2008).

[R38] Sutherland, J., Jakobsen, C.R., Johnson, K.: Scrum and CMMI Level 5: The Magic Potion for Code Warriors. In: Hawaii International Conference on System Sciences, Proceedings of the 41st Annual. pp. 466–466 (2008).

[R39] Xu, B.: Towards High Quality Software Development with Extreme Programming Methodology: Practices from Real Software Projects. In: International Conference on Management and Service Science, 2009. MASS’09. pp. 1–4 (2009).

[R40] Ahmed, A., Ahmad, S., Ehsan, N., Mirza, E., Sarwar, S.Z.: Agile software development: Impact on productivity and quality. In: 2010 IEEE International Conference on Management of Innovation and Technology (ICMIT). pp. 287–291 (2010).

[R41] Mirnalini, K., Raya, V.R.: Agile - A software development approach for quality software. In: 2010 International Conference on Educational and Information Technology (ICEIT). pp. V1-242-V1-244 (2010).

[R42] Scharff, C.: Guiding global software development projects using Scrum and Agile with quality assurance. In: 2011 24th IEEE-CS Conference on Software Engineering Education and Training (CSEE T). pp. 274–283 (2011).

[R43] Mundra, A., Misra, S., Dhawale, C..: Practical Scrum-Scrum Team: Way to Produce Successful and Quality Software. In: 2013 13th International Conference on Computational Science and Its Applications (ICCSA). pp. 119–123 (2013).

[R44] Manjunath, K.N., Jagadeesh, J., Yogeesh, M.: Achieving quality product in a long term software product development in healthcare application using Lean and Agile principles: Software engineering and software development. In: 2013 International Multi-Conference on Automation, Computing, Communication, Control and Compressed Sensing (iMac4 s). pp. 26–34 (2013).

[R45] Marcal, A.S.C., Soares, F.S.F., Belchior, A.D.: Mapping CMMI Project Management Process Areas to SCRUM Practices. In: 31st IEEE Software Engineering Workshop, 2007. SEW 2007. pp. 13–22 (2007).

[R46] Bellomo, S., Nord, R.L., Ozkaya, I.: Elaboration on an integrated architecture and requirement practice: Prototyping with quality attribute focus. In: Twin Peaks of Requirements and Architecture (TwinPeaks), 2013 2nd International Workshop on the. pp. 8–13 (2013).

[R47] Sultana, S., Motla, Y.H., Asghar, S., Jamal, M., Azad, R.: A hybrid model by integrating agile practices for Pakistani software industry. In: 2014 International Conference on Electronics, Communications and Computers (CONIELECOMP). pp. 256–262 (2014).

[R48] Garousi, V., Coşkunçay, A., Betin-Can, A., Demirörs, O.: A survey of software engineering practices in Turkey. J. Syst. Softw. 108, 148–177 (2015).

[R49] Diebold, P., Dahlem, M.: Agile Practices in Practice: A Mapping Study. In: Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering. p. 30:1–30:10. ACM, New York, NY, USA (2014).

[R50] Curcio, K., Malucelli, A., Reinehr, S., Paludo, M.A.: An analysis of the factors determining software product quality: A comparative study. Comput. Stand. Interfaces. 48, 10–18 (2016).

[R51] Zhou, Z.Q., Xiang, S., Chen, T.Y.: Metamorphic Testing for Software Quality Assessment: A Study of Search Engines. IEEE Trans. Softw. Eng. 42, 264–284 (2016).

[R52] Bhasin, S.: Quality Assurance in Agile: A Study towards Achieving Excellence. In: AGILE India (AGILE INDIA), 2012. pp. 64–67 (2012).

[R53] Selleri Silva, F., Santana Furtado Soares, F., Lima Peres, A., Monteiro de Azevedo, I., Pereira Pinto, P., Romero de Lemos Meira, S.: A Reference Model for Agile Quality Assurance: Combining Agile Methodologies and Maturity Models. In: Quality of Information and Communications Technology (QUATIC), 2014 9th International Conference on the. pp. 139–144 (2014).

[R54] Nord, R.L., Tomayko, J.E.: Software architecture-centric methods and agile development. IEEE Softw. 23, 47–53 (2006).

[R55] Hongying, G., Cheng, Y.: A customizable agile software Quality Assurance model. In: Information Science and Service Science (NISS), 2011 5th International Conference on New Trends in. pp. 382–387 (2011).

[R56] Um, T., Kim, N., Lee, D., In, H.P.: A Quality Attributes Evaluation Method for an Agile Approach. In: 2011 First ACIS/JNU International Conference on Computers, Networks, Systems and Industrial Engineering (CNSI). pp. 460–461 (2011).

[R57] Ayed, H., Vanderose, B., Habra, N.: A Metamodel-Based Approach for Customizing and Assessing Agile Methods. In: Quality of Information and Communications Technology (QUATIC), 2012 Eighth International Conference on the. pp. 66–74 (2012).

[R58] Ayed, H., Habra, N., Vanderose, B.: AM-QuICk: A Measurement-Based Framework for Agile Methods Customisation. In: 2013 Joint Conference of the 23rd International Workshop on Software Measurement and the 2013 Eighth International Conference on Software Process and Product Measurement (IWSM-MENSURA). pp. 71–80 (2013).

[R59] Jinzenji, K., Hoshino, T., Williams, L., Takahashi, K.: An experience report for software quality evaluation in highly iterative development methodology using traditional metrics. In: 2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE). pp. 310–319 (2013).

[R60] Janus, A., Dumke, R., Schmietendorf, A., Jager, J.: The 3C approach for Agile Quality Assurance. In: 2012 3rd International Workshop on Emerging Trends in Software Metrics (WETSoM). pp. 9–13 (2012).

[R61] Tarhan, A., Yilmaz, S.G.: Systematic analyses and comparison of development performance and product quality of Incremental Process and Agile Process. Inf. Softw. Technol. 56, 477–494 (2014).

[R62] Sampaio, A., Vasconcelos, A., Sampaio, P.R.F.: Assessing agile methods: An empirical study. J. Braz. Comput. Soc. 10, 21–48 (2004).

[R63] Singh, N.P., Soni, R.: Agile Software: Ensuring Quality Assurance and Processes. In: Mantri, A., Nandi, S., Kumar, G., and Kumar, S. (eds.) High Performance Architecture and Grid Computing. pp. 640–648. Springer Berlin Heidelberg (2011).

[R64] Desharnais, J.-M., Kocaturk, B., Abran, A.: Using the COSMIC Method to Evaluate the Quality of the Documentation of Agile User Stories. In: Software Measurement, 2011 Joint Conference of the 21st Int’l Workshop on and 6th Int’l Conference on Software Process and Product Measurement (IWSM-MENSURA). pp. 269–272 (2011).

[R65] Jeon, S., Han, M., Lee, E., Lee, K.: Quality Attribute Driven Agile Development. In: 2011 9th International Conference on Software Engineering Research, Management and Applications (SERA). pp. 203–210 (2011).

[R66] Mnkandla, E., Dwolatzky, B.: Defining Agile Software Quality Assurance. In: International Conference on Software Engineering Advances. pp. 36–36 (2006).

[R67] Ran, H., Zhuo, W., Jianfeng, X.: Web Quality of Agile Web Development. In: IITA International Conference on Services Science, Management and Engineering, 2009. SSME’09. pp. 426–429 (2009).

[R68] Christensen, H.B., Hansen, K.M., Lindstrøm, B.: Lightweight and Continuous Architectural Software Quality Assurance Using the aSQA Technique. In: Babar, M.A. and Gorton, I. (eds.) Software Architecture. pp. 118–132. Springer Berlin Heidelberg (2010).

[R69] Khomh, F., Guéhéneuc, Y.-G.: DEQUALITE: Building Design-based Software Quality Models. In: Proceedings of the 15th Conference on Pattern Languages of Programs. p. 2:1–2:7. ACM, New York, NY, USA (2008).

[R70] Bansal, A., Pundir, S.: Software quality score board based on SQA framework to improvise software reliability. In: 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom). pp. 2421–2427 (2016).

[R71] Mohan M., Ferrer, R.M., Narayan, J.: Scaling Agile Scrum Software Development: Providing Agility and Quality to Platform Development by Reducing Time to Market. In: 2016 IEEE 11th International Conference on Global Software Engineering (ICGSE). pp. 84–88 (2016).

Appendix B

See Table 15

Table 15 List of success critical factors

Appendix C

See Table 16

Table 16 List of metrics

Appendix D

See Table 17

Table 17 List of agile practices

Appendix E

See Table 18

Table 18 List of quality attributes