John M. Roche
Skip Abstract Section
Abstract
Software measurement is widely advocated as a fundamental constituent of an engineering approach to planning and controlling software development. Unfortunately, there is a dichotomy between the quantity of developed metrics and those used. This paper provides a tutorial review of software engineering measurement indicating the depth and breadth of the field. Individual metrics are not described due to the interest of this paper being on the measurement process and not the products of that process. Generic problems have been identified within existing measurement processes, these provide learning points for the expression of measurement principles. These principles are classified and described according to their position within the formulation, analysis and application stages of measurement. Conclusions are elaborated that suggest that existing measurement frameworks for applying measurement - often called measurement methods - do not provide sufficient support for the principles and their continued use will only serve to replicate the problems. In order to improve the products i.e. metrics, the measurement process requires improvement through inclusion of these principles in a new method.
- Albrecht, A.J. and Gaffney, J.E. (1983) Software Function, Source Lines of Code, and Development Prediction" A Software Science Validation. In: IEEE Trans. on Software Eng. SE 9 (6) p. 639-647 1983.Google Scholar
Digital Library
- Bainbridge, J. and Whitty, R. (1990) Obtaining structural metrics of Z specifications for system development. In: Proc. 5th Ann. Z User Meeting Oxford 17-18 Dec. 1990. Google ScholarDigital Library
- Basili, V.R. and Selby, R.W. (1985) Calculation and use of an environment's characteristic software metric set. In: Proc. 8th Intl. Conf. on Softw. Eng. p. 386-391 1985. Google ScholarDigital Library
- Basili, V.R. and Hutchens, D.H. (1983) An Empirical Study of a Syntactic Complexity Family. In: IEEE Trans. Softw. Eng. SE 9 (6) p. 664-672 1983.Google ScholarDigital Library
- Basili, V.R. and Rombach, H.D. (1988) The TAME project: Towards Improvement-oriented software environments. In: IEEE Trans on Softw. Eng. SE 14 (6) June p. 758- 771 1988.Google Scholar
- Basili, V.R. and Selby, R.W. (1987) Comparing the effectiveness of software testing strategies: In: IEEE Trans. on Softw. Eng. SE 13 (12) p. 1278-1296. 1987. Google ScholarDigital Library
- Basili, V.R. and Selby, R.W. (1991) Paradigms for experimentation and empirical studies in software engineering. In: Reliability Engineering and System Safety 32 p. 171- 191 1991.Google Scholar
- Basili, V.R. and Weiss, D.M. (1984) A methodology for collecting valid software engineering data. In: IEEE Trans. on Softw. Eng. SE 10 p. 728-738 1984.Google ScholarDigital Library
- Belady, L.A. and Evangelisti, C.J. (1981) System partitioning and its measure. In: J. of Syst. and Softw. SE 2 p. 23-29 1981.Google Scholar
- Benyon-Tinker, G. (1979) Complexity measures in an evolving large system. In: Proc. ACM Workshop on Quantitative: Software Models p. 117-127 1979.Google Scholar
- Boehm, B.W. (1981) Software Engineering Economics. Englewood Cliffs NJ Prentice-Hall 1981. Google ScholarDigital Library
- Boehm, B.W. (1984) Software engineering economics. In: IEEE Trans on Softw. Eng. SE 10 (1) p. 4-21 1984.Google ScholarDigital Library
- Born, G. Controlling software quality. In: Softw. Eng. J. 1 (1) Special issue on Controlling software projects January p. 24-28 1986. Google Scholar
- Bourque, P. Cote, V. (1991) An experiment in software sizing with structured analysis metrics. In: J. of Syst. and Softw. 15 (2) p. 159-72 1991. Google Scholar
- Bredero, R. Caracoglia, G. Jaggers, C. Kok, P. Tate, G. Verner, J. (1989) Comparative evaluation of existing cost estimation tools. In: MERMAID Project Report D7.1Y November 1989.Google Scholar
- Bush, M. and Fenton, N.E. (1990) Software measurement: A conceptual framework. In: J. Syst. Softw. 12 p. 223-231 1990. Google ScholarDigital Library
- Card, D.N. and Agresti, W.W. (1988) Measuring software design complexity. In: J. of Syst. and Softw. 1988. Google ScholarDigital Library
- Chapin, N. (1979) A measure of software complexity. In: Proc. AFIPS p. 995-1002 1979.Google Scholar
- Chemiavsky, J. C. (1991) On Weyuker's axioms for software complexity measures. In: IEEE Trans. on Softw. Eng. SE 17 (6) p. 636-638 1991. Google ScholarDigital Library
- Conte, S.D. Dunsmore, H.E. and Shen, V.Y. (1986) Software Engineering Metrics and Models. Benjamin Cummings Memlo Park Ca USA 1986. Google ScholarDigital Library
- Coulter, N.S. (1983) Software Science and Cognitive Psychology. In: IEEE Trans. Softw. Eng. SE 9 (2) p. 166-171 1983.Google ScholarDigital Library
- Cowderoy, A.J.C. (1989) New trends in Cost-Estimation. In: Proc. Centre for Software Reliability Conf. on Measurement for Softw. Control and Assurance (Eds.) Kitchenham, B. and Littlewood, B. Elsevier Bristol U.K. p. 63-88 1989.Google Scholar
- Curtis, B. Sheppard, S. Milliman, P. Borst, M. and Love, T. (1979b) Measuring the psychological complexity of software maintenance tasks with the Halstead and McCabe metrics. In: IEEE Trans. on Softw. Eng. SE 5 (2) p. 96-104 1979.Google ScholarDigital Library
- de Paoli, F. Morasca, S. (1990) Extending software complexity metrics to concurrent programs. In: Proc. 14th Annual International Computer Software and Applications Conference p. 414-19 1990.Google ScholarCross Ref
- deMarco, T. (1982) Controlling Software Project Management, Measurement and Estimation. New York, Yourdon Press, 1982. Google ScholarDigital Library
- Emerson, T.J. (1984) A discriminant metric for module cohesion. In: Proc. 7th Int. Conf. on Softw. Eng. p. 335-343 1984. Google ScholarDigital Library
- Evangelist, W.M. (1983) Software Complexity Metric Sensitivity to Program structuring rules. In: J. Syst. and Softw. 3 (3) p. 231-243 1983. Google Scholar
- Evans, M.W. and Marciniak, J.J. (1987) Software quality assurance and management. John Wiley and Sons, Chichester UK 1987. Google ScholarDigital Library
- Fenton, N.E. and Whitty, R.W. (1986) Axiomatic approach to software metrication through program decomposition. In: Computer J. 29 (4) p. 329-339 1986.Google Scholar
- Fenton N.E. (1991) Software Metrics: A Rigorous Approach. Chapman Hall London 1991. Google ScholarDigital Library
- Frewin, G.D. and Hatton, B.J. (1986) Quality management - procedures and practices. In: Softw. Eng. J. 1 (1) Special issue on Controlling software projects January p. 29-38 1986. Google Scholar
- Fuchs, N. and S. Stainer (1991) Language independent definition of axiomatic metrics In: Proceedings 1st BACS Seminar on Formal Aspects of Measurement University of the South Bank London 1991. Google ScholarDigital Library
- Grady, R.B. and Caswell, D.L. (1987) Software Metrics: Establishing a company wide program. Prentice Hall NJ 1987. Google ScholarDigital Library
- Grady, R.B. (1992) Practical Software Metrics for Project management and Process Improvement. Prentice Hall Englewood cliffs NJ. 1992. Google ScholarDigital Library
- Halstead, M.H. and Zislis, P.M (1973) Experimental Verification of Two Theorems of Software Physics. In: Computer Science Department Technical Report 97 Purdue University 1973.Google Scholar
- Halstead, M.H. (1975) Software Physics: Basic Principles. In: IBM Research Report RJ1582 1975.Google Scholar
- Halstead, M.H. (1977) Elements of Software Science. New York Elsevier North Holland 1977. Google ScholarDigital Library
- Hamer, P.G. and Frewin, G.D. (1982) M.H. Halstead's Software Science - A Critical Examination. In: Proc. 6th Intl. Conf. Softw. Eng. p. 197-206 1982. Google ScholarDigital Library
- Hansen, W.J. (1978) Measurement of program complexity by the pair (Cyclomatic number, operator count) In: ACM SIGPLAN Notices 13 (3) p. 29-33 1978. Google ScholarDigital Library
- Harrison, W. and Magel, K. (1981) A complexity measure based upon nesting level. In: ACM SIGPLAN Notices 16 (3) p. 63-74 1981. Google Scholar
- Harrison, W. Magel, K. Kluczny, R. and DeKock, A. (1982) Applying software complexity metrics to program maintenance. In: Computer 15 (9) Sept. p. 65-78 1982.Google Scholar
- Henry, S. and Kafura, D. (1981) Software structure metrics based on information flow. In: IEEE Trans. on Softw. Eng. SE 7 (5) Sept. p. 510-517 1981.Google Scholar
- Hihn, J. and Habib-Agahi, H. (1991) Cost estimation of software intensive projects: A survey of current practice. In: Proc. 13th Intl. Conf. on Softw. Eng. 13-16 May Austin Texas USA IEEE Computer Society Press p. 276-287 1991. Google ScholarDigital Library
- Ince, D. (1990) An annotated bibliography of software metrics In: ACM SIGPLAN Notices 25(8) p. 15-23 Aug. 1990. Google ScholarDigital Library
- Ince, D.C. and Shepperd, M.J. (1988) System design metrics: A review and perspective. In: Proc. IEE/BCS Conf. on Softw. Eng. p. 23-27 1988.Google Scholar
- Ince, D.C. Shepperd, M.J. (1989) An empirical and theoretical analysis of an information flow-based system design metric. In: Proc. ESEC '89. 2nd European Softw. Eng. Conf. p. 86-99 (Eds.) Ghezzi, C. McDermid, J.A. Springer-Verlag Berlin West Germany 1989. Google ScholarDigital Library
- Johnston, D.B. and Lister, A.M. (1981) A Note on the Software Science Length Equation. In: Softw. Practice and Experience 11 (8) p. 875-879 1981.Google Scholar
- Kafura, D. and Canning, J. (1985) A validation of software metrics using many metrics and two resources. In: Proc. 8th Intl. Conf. on Softw. Eng. p. 378-385 1985. Google ScholarDigital Library
- Kemerer, C.F. (1987) An empirical validation of software cost estimation models. In: CACM 30 (5) May p. 416-429 1987. Google Scholar
- Kemerer, C.F. (1993) Reliability of Function Points Measurement. In: CACM 36 (2) February p. 85-97 1993. Google Scholar
- Kitchenham, B. and Pickard, L (1987a) Towards a constructive quality model Part I: Software quality modelling, measurement and prediction. In: Softw. Eng. J. p. 105-113 July 1987. Google ScholarDigital Library
- Kitchenham, B. and Pickard, L (1987b) Towards a constructive quality model Part II: statistical techniques for modelling software quality in the ESPRIT REQUEST project. In: Softw. Eng., J. p. 114-126 July 1987. Google ScholarDigital Library
- Kitchenham, B.A. and Walker, J.G. (1989a) A quantitative approach to monitoring software development. In: Softw. Eng. J. January p. 2-13 1989. Google Scholar
- Kitchenham, B.A. and Linkman, S.J. (1990) Design metrics in practice. In: Info. and Softw. Tech. 32(4) May p. 304-309 1990. Google Scholar
- Kitchenham, B.A. and McDermid, J.A. (1986) Software metrics and integrated project support environments. In: Softw. Eng. J. 1(1) p. 58-64 1986. Google ScholarDigital Library
- Kitchenham, B.A. Pickard, L.M. and Linkman, S.J. (1990a) An evaluation of some design metrics. In: Softw. Eng. J. 5 (1) Special issue on software reliability and metrics January p. 50-58 1990. Google Scholar
- Kitchenham, B.A. (1991) Empirical studies of the assumptions underlying software cost estimation models. In: Info. and Softw. Tech. 34(4) p. 211-218 1991. Google ScholarDigital Library
- Kitchenham, B.A. (1993) Inter-item correlations among function points. In: Proc. 1st Intl. Software Metrics Symposium Baltimore USA May 21-22 p. 11-15 1993.Google Scholar
- Kok, P.A.M. Kitchenham, B.A. and Kirakowski, J. (1990) The Mermaid approach to software cost estimation. In: Proc. ESPRIT '90 Conf. Kluwer Academic Press 1990.Google ScholarCross Ref
- Lassez, D. Van de Knijff, and Shepherd, (1981) A Critical Examination of Software Science. In: J. Syst. and Softw. 2 p. 105-112 1981.Google Scholar
- Lehman, M.M. (1991) Software engineering - theory and practice. In: Softw. Eng. J. 6 (5) Special issue on Software process and its support Sept. p. 243-258 1991. Google Scholar
- Lewis, J.A. and Henry, S.M. (1990) On the benefits and difficulties of a maintainability via metrics methodology. In: Softw. Maintenance: Research and Practice 2 p. 113-131 1990. Google Scholar
- Li, H.F. Cheung, W.K. (1987) An empirical study of software metrics. In: IEEE Trans. on Softw. Eng. SE 13 (6) 1987. Google ScholarDigital Library
- Londeix, B. (1988) Aspects of estimation practice in software development. In: Proc. IEE/BCS Conf. an Softw. Eng. '88 p. 75-79 1988.Google Scholar
- Low, G.C. and Jeffery, D.R. (1990) Function Points in the estimation and evaluation of the software process. In: IEEE Trans. on Softw. Eng. SE 16(1) p. 64-71 1990. Google ScholarDigital Library
- McCabe, T.J. (1976) A complexity measure. In: IEEE Trans. on Softw. Eng. SE 2 (4) Dec. p. 308-320 1976. Google Scholar
- McClure, C. (1978) A model for program complexity analysis. In: Proc. 3rd Intl. Conf. on Softw, Eng. p. 149-157 1978. Google ScholarDigital Library
- Myrvold, A. (1990) Data analysis for software metrics. In: J. Syst. and Softw. 12 p. 271- 275 1990. Google Scholar
- Oviedo, E. (1980) Control flow, data flow and program complexity. In: Proc. IEEE Computer Software and Applications Conference Chicago USA p. 146-152 1980.Google Scholar
- Pickard, L.M. (1989) Analysis of software metrics. In: Proc. Centre for Software Reliability Conf. on Measurement for Softw. Control and Assurance (Eds) Kitchenham, B. and Littlewood, B. Bristol U.K. Elsevier p. 155-180. 1989.Google Scholar
- Putnam, L.H. (1978) A general empirical solution to the macro software sizing and estimating problem, in: IEEE Trans. on Softw. Eng. SE 4 p. 345-361 1978.Google ScholarDigital Library
- Ramamurthy, B. Melton, A. (1988) A synthesis of software science measures and the cyclomatic number. In: IEEE Trans. on Softw. Eng. SE 14 (8) August p. 1116-1121 1988. Google Scholar
- Ratcliffe, B. and Rollo, A. (1990) Adapting Function Point analysis to Jackson system development. In: Softw. Eng. J. 5 (1) Special issue on software reliability and metrics, p. 79-84 1990. Google ScholarDigital Library
- Reynolds, R.G. (1984) Metrics to measure the complexity of partial programs. In: J. of Syst. and Softw. 4(1) p. 75-92 1984. Google Scholar
- Ross, N. (1989) The collection and use of data for monitoring software projects. In: Proc. Centre for Software Reliability Conf. on Measurement for Softw. Control and Assurance (Eds Kitchenham, B. and Littlewood, B. Bristol U.K. Elsevier p. 125-153 1989.Google Scholar
- Samson, W.B. Nevill, D.G. and Dugard, P.I. (1987) Predictive software metrics based upon a formal specification. In: Info. and Softw. Tech. 29(5) p. 242-248 1987. Google ScholarDigital Library
- Shen, V.Y. Conte, S.D. Dunsmore, H.E. (1983) Software science revisited: A critical analysis of the theory and its empirical support. In: IEEE Trans. on Softw. Eng. SE 9 (2) March p. 155-165 1983.Google Scholar
- Shepperd, M.J. (1988) An Evaluation of Software Product Metrics. In: Info. and Softw. Tech. 30 (3) p. 177-188 1988. Google ScholarDigital Library
- Shepperd, M.J. (1991) System architecture metrics: An evaluation. In: Unpublished PhD thesis March 1991. Google ScholarDigital Library
- Symons, C.R. (1988) Function Point Analysis: Difficulties and Improvements. In: IEEE Trans. on Softw. Eng. SE 14 (1) p. 2-11 1988. Google ScholarDigital Library
- Szulewski, P.A. Whitworth, M.H. Buchan, P. Dewolf, J.B. (1981) The measurement of software science parameters in software designs. In: ACM SIGMETRICS P.E.R. p. 89-94 1981. Google ScholarDigital Library
- Weyuker, E.J. (1988) Evaluating software complexity measures. In: IEEE Trans. on Softw. Eng. SE 14 (9) Sept. p. 1357-1365 1988. Google Scholar
- Whale, G. (1990) Software metrics and plagiarism detection. In: J. Syst. and Softw. 13 (2) 131-138 1990. Google ScholarDigital Library
- Whitty, R. (1989) Structural metrics for Z specifications. In: Proc. 4th Ann. Z Technical and Users Meeting Oxford 14-15 December 1989.Google Scholar
- Wingrove, A. (1986) The problems of managing software projects. In: Softw. Eng. J. 1 (1) Special issue on Controlling software projects January p. 3-6 1986. Google Scholar
- Woodfield, S. (1980) Enhanced effort estimation by extending basic programming models to include modularity factors. In: Ph.D. Dissertation Computer Science Dept. Purdue University, 1980. Google ScholarDigital Library
- Yau, S.S. Collofello, J.S. and MacGregor, T.M. (1978) Ripple effect analysis of software maintenance. In: Proc. COMPSAC '78 p. 60-65 1978.Google ScholarCross Ref
- Yin and Winchester (1978) The establishment and use of measures to evaluate the quality of software designs. In: Proc. of the Softw. Quality Assurance Workshop San Diego Calif November 15-17 p. 45-52 1978. Google ScholarDigital Library
Index Terms
- Software metrics and measurement principles
Recommendations
Comparing software metrics tools
ISSTA '08: Proceedings of the 2008 international symposium on Software testing and analysisThis paper shows that existing software metric tools interpret and implement the definitions of object-oriented software metrics differently. This delivers tool-dependent metrics results and has even implications on the results of analyses based on ...
Software metrics: using measurement theory to describe the properties and scales of static software complexity metrics
Over the last decade many software metrics have been introduced by researchers and many software tools have been developed using software metrics to measure the "quality" of programs. These metrics for measuring productivity, reliability, ...
Software Metrics: Good, Bad and Missing
The software industry is an embarrassment when it comes to measurement and metrics. Many software managers and practitioners, including tenured academics in software engineering and computer science, seem to know little or nothing about these topics. ...
Comments