Ayşe Tosun
Burak Turhan
ABSTRACT
We have conducted a study in a large telecommunication company in Turkey to employ a software measurement program and to predict pre-release defects. We have previously built such predictors using AI techniques. This project is a transfer of our research experience into a real life setting to solve a specific problem for the company: to improve code quality by predicting pre-release defects and efficiently allocating testing resources. Our results in this project have many practical implications that managers have started benefiting: code analysis, bug tracking, effective use of version management system and defect prediction. Using version history information, developers can find around 88% of the defects with 28% false alarms, compared to same detection rate with 50% false alarms without using historical data. In this paper we also shared in detail our experience in terms of the project steps (i.e. challenges and opportunities).
- Menzies, T., Greenwald, J., Frank, A. 2007. Data Mining Static Code Attributes to Learn Defect Predictors. IEEE Transactions on Software Engineering, vol. 33, no. 1 (January 2007), 2--13. Google Scholar
Digital Library
- Ostrand, T. J., Weyuker E. J., Bell, R. M. 2005. Predicting the Location and Number of Faults in Large Software Systems. IEEE Transactions on Software Engineering, vol. 31, no. 4 (April 2005), 340--355. Google ScholarDigital Library
- Menzies, T., Turhan, B., Bener, A., Gay, G., Cukic, B., and Jiang, Y. 2000. Implications of Ceiling Effects in Defect Predictors. in the Proceedings of PROMISE 2008 Workshop, Germany. Google ScholarDigital Library
- Halstead, H. M. 1977. Elements of Software Science. Elsevier, New York. Google ScholarDigital Library
- McCabe, T. 1976. A Complexity Measure. IEEE Transactions on Software Engineering. vol. 2, no. 4, 308--320. Google ScholarDigital Library
- Turhan, B., Menzies, T., Bener, A., Distefano, J. 2009. On the Relative Value of Cross-company and Within-Company Data for Defect Prediction. Empirical Software Engineering Journal (January 2009), DOI: 10.1007/s10664-008-9103-7. Google ScholarDigital Library
- Boetticher, G., Menzies, T., Ostrand, T. 2007. PROMISE Repository of empirical software engineering data. http://promisedata.org/repository. West Virginia University, Department of Computer Science, 2007.Google Scholar
- Turhan, B., Bener, A., Menzies, T. 2008. Nearest Neighbor Sampling for Cross Company Defect Prediction. In Proceedings of the 1st International Workshop on Defects in Large Software Systems, DEFECTS 2008, 26. Google ScholarDigital Library
- Prest. 2009. Department of Computer Engineering, Bogazici University, http://code.google.com/p/prest/Google Scholar
- Nagappan, N., Ball, T., Murphy, B. 2006. Using Historical In-Process and Product Metrics for Early Estimation of Software Failures. In Proceedings of the International Symposium on Software Reliability Engineering, NC, November 2006. Google ScholarDigital Library
- Nagappan, N., Murphy, B., Basili, V. 2008. The Influence of Organizational Structure on Software Quality. in Proceedings of the International Conference on Software Engineering, Germany, May 2008. Google ScholarDigital Library
- NASA WVU IV & V Facility, Metrics Program. 2004. http://mdp.ivv.nasa.govGoogle Scholar
- Heeger, D. 1998. Signal Detection Theory. http://white.stanford.edu//~heeger/sdt/sdt.htmlGoogle Scholar
- Tosun, A., Turhan, B., Bener, A. 2008. Direct and Indirect Effects of Software Defect Predictors on Development Lifecycle: An Industrial Case Study. in Proceedings of the 19th International Symposium on Software Reliability Engineering, Seattle, USA, November 2008.Google Scholar
- Tosun, A., Turhan, B., Bener, A. 2008. Ensemble of Software Defect Predictors: A Case Study. In Proceedings of the 2nd International Symposium on Empirical Software Engineering and Measurement, Germany, October 2008, 318--320. Google ScholarDigital Library
- Turhan, B., Bener, A. 2009. Analysis of Naive Bayes' Assumptions on Software Fault Data: An Empirical Study. Data and Knowledge Engineering Journal, vol. 68, no. 2, 278--290. Google ScholarDigital Library
- Koru, G., Liu, H. 2007. Building effective defect prediction models in practice. IEEE Software, 23--29. Google ScholarDigital Library
- Lessmann, S., Baesens, B., Mues, C, Pietsch, S. 2008. Benchmarking Classification Models for Software Defect Prediction: A Proposed Framework and Novel Findings. IEEE Transactions on Software Engineering, vol. 34, no. 4, July/August 2008, 1--12. Google ScholarDigital Library
- Chidamber, S. R., Kemerer, C. F. 1994. A Metrics Suite for Object Oriented Design. IEEE Transactions on Software Engineering, vol. 20, no. 6, 476--493. Google ScholarDigital Library
- Fenton, N. E., Neil, M., Marsh, W., Hearty, P., Radlinski, L., and Krause, P. 2008. On the effectiveness of early life cycle defect prediction with Bayesian Nets. Empirical Software Engineering, vol. 13, 2008, 499--537. Google ScholarDigital Library
- The, A. N., Ruhe, G. 2009. Optimized Resource Allocation for Software Release Planning. IEEE Transactions on Software Engineering, vol. 35, no. 1, Jan/Feb 2009, 109--123. Google ScholarDigital Library
- Arisholm, E., Briand, C. L. 2006. Predicting fault prone components in a Java legacy system. In Proceedings of ISESE'06, 1--22. Google ScholarDigital Library
Index Terms
- Practical considerations in deploying AI for defect prediction: a case study within the Turkish telecommunication industry
Recommendations
Practical considerations in deploying statistical methods for defect prediction: A case study within the Turkish telecommunications industry
Context: Building defect prediction models in large organizations has many challenges due to limited resources and tight schedules in the software development lifecycle. It is not easy to collect data, utilize any type of algorithm and build a permanent ...
Software Defect Association Mining and Defect Correction Effort Prediction
Much current software defect prediction work focuses on the number of defects remaining in a software system. In this paper, we present association rule mining based methods to predict defect associations and defect correction effort. This is to help ...
A multi-objective effort-aware defect prediction approach based on NSGA-II
AbstractEffort-Aware Defect Prediction (EADP) technique sorts software modules by the defect density and aims to find more bugs when testing a certain number of Lines of Code (LOC). The existing EADP methods ignore the number of required inspected ...
Highlights- Propose a multi-objective effort-aware defect prediction approach based on NSGA-II.
- The main objective of effort-aware defect prediction is finding more bugs and inspecting as fewer modules as possible, when testing a specific number ...
Comments