Skip to main content
SpringerLink
Log in

An Instrument for Measuring the Key Factors of Success in Software Process Improvement

  • Published:
Empirical Software Engineering Aims and scope Submit manuscript

Abstract

Understandinghow to implement SPI successfully is arguably the most challengingissue facing the SPI field today. The SPI literature containsmany case studies of successful companies and descriptions oftheir SPI programs. However, there has been no systematic attemptto synthesize and organize the prescriptions offered. The researchefforts to date are limited and inconclusive and without adequatetheoretical and psychometric justification.

This paper provides a synthesis of prescriptions for successfulquality management and process improvement found from an extensivereview of the quality management, organizational learning, andsoftware process improvement literature. The literature reviewwas confirmed by empirical studies among both researchers andpractitioners. The main result is an instrument for measuringthe key factors of success in SPI based on data collected from120 software organizations. The measures were found to have satisfactorypsychometric properties. Hence, managers can use the instrumentto guide SPI activities in their respective organizations andresearchers can use it to build models to relate the facilitatingfactors to both learning processes and SPI outcomes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Ahire, S. L., Golhar, D. Y. and Waller, M. A. 1996 Development and Validation of TQM Implementation Constructs, Decision Sciences, 27(1), 23-56.

    Google Scholar 

  • Anastasi, A. and Urbina, S. 1997 Psychological Testing, Seventh edition, Upper Saddle River, New Jersey: Prentice-Hall.

    Google Scholar 

  • Argyris, C. and Schön, D. A. 1978 Organizational Learning: A Theory of Action Perspective, Reading, Massachusetts: Addison-Wesley.

    Google Scholar 

  • Argyris, C. and Schön, D. A. 1996 Organizational Learning II: Theory, Method, and Practice, Reading, Massachusetts: Addison-Wesley.

    Google Scholar 

  • Bagozzi, R. P. 1996 Measurement in Marketing Research: Basic Principles of Questionnaire Design, in R. P. Bagozzi (Ed.), Principles of Marketing Research, Cambridge, Massachusetts: Blackwell, pp. 1-49.

    Google Scholar 

  • Baruch, Y. 1999 Response Rate in Academic Studies—A Comparative Analysis, Human Relations, 52(4), 421-438.

    Google Scholar 

  • Basili, V. R. 1996 The Role of Experimentation in Software Engineering: Past, Current, and Future, Proceedings of the 18th International Conference on Software Engineering (ICSE-18), Berlin, Germany, March 25–29, pp. 442-449.

  • Basili, V. R. and Caldiera, G. 1995 Improve Software Quality by Reusing Knowledge and Experience, Sloan Management Review, 37, Autumn, pp. 55-64.

    Google Scholar 

  • Basili, V. R. and Rombach, H. D. 1988 The TAME Project: Towards Improvement-Oriented Software Environments, IEEE Transactions on Software Engineering, 14(6), 758-773.

    Google Scholar 

  • Basili, V. R. and Selby, R. W. 1991 Paradigms for Experimentation and Empirical Studies in Software Engineering, Reliability Engineering and System Safety, 32(1–2), pp. 171-191.

    Google Scholar 

  • Basili, V. R., Selby, R. W. and Hutchens, D. H. 1986 Experimentation in Software Engineering, IEEE Transactions on Software Engineering, 12(7), 733-743.

    Google Scholar 

  • Black, S. A. and Porter, L. J. 1996 Identification of the Critical Factors of TQM, Decision Sciences, 27(1), 1-21.

    Google Scholar 

  • Blalock, H. M. 1969 Theory Construction: From Verbal to Mathematical Formulations, Englewood Cliffs, N.J.: Prentice-Hall.

    Google Scholar 

  • Carmines, E. G. and Zeller, R. A. 1979 Reliability and Validity Assessment, Sage University Paper series on Quantitative Applications in the Social Sciences, series no. 07-017, Newbury Park, CA: Sage.

    Google Scholar 

  • Cattell, R. B. 1966 The Scree Test for the Number of Factors, Multivariate Behavioral Research, 1, 245-276.

    Google Scholar 

  • Choo, C. W. 1995 The Knowing Organization: How Organizations Use Information to Construct Meaning Create Knowledge and Make Decisions, International Journal of Information Management, 16(5), 329-340.

    Google Scholar 

  • Choo, C. W. 1998 The Knowing Organization: How Organizations Use Information to Construct Meaning, Create Knowledge, and Make Decisions, New York: Oxford University Press.

    Google Scholar 

  • Cliff, N. R. 1988 The Eigenvalues-greater-than-one Rule and the Reliability of Components, Psychological Bulletin, 103, 276-279.

    Google Scholar 

  • Cohen, J. 1988 Statistical Power Analysis for the Behavioral Sciences, Second Edition, Hillsdale, New Jersey: Laurence Erlbaum.

    Google Scholar 

  • Comrey, A. 1973 A First Course on Factor Analysis, London: Academic Press.

    Google Scholar 

  • Cronbach, L. J. 1951 Coefficient Alpha and the Internal Consistency of Tests, Psychometrica, 16, 297-334, September.

    Google Scholar 

  • Cronbach, L. J. 1971 Test Validation, in Educational Measurement, 2nd Edition, R. L. Thorndike (ed.), American Council on Education, Washington, D.C, pp. 443-507.

    Google Scholar 

  • Crosby, P. B. 1979 Quality is Free: The Art of Making Quality Certain, New York: McGraw-Hill.

    Google Scholar 

  • Crosby, P. B. 1984 Quality Without Tears, New York: McGraw-Hill.

    Google Scholar 

  • Crosby, P. B. 1996 Quality Is Still Free: Making Quality Certain in Uncertain Times, New York: McGraw-Hill.

    Google Scholar 

  • Davis, D. 1996 Business Research for Decision Making, Fourth Edition, Belmont, California: Duxbury Press.

    Google Scholar 

  • Deming, W. E. 1982 Quality, Productivity, and Competitive Position, Camebridge, Massachusetts: MIT Centre for Advanced Engineering Study.

    Google Scholar 

  • Deming, W. E. 1986 Out of the Crisis, Cambridge, Massachusetts: MIT Center for Advanced Engineering Study.

    Google Scholar 

  • Deephouse, C., Mukhopadhyay, T., Goldenson, D. R. and Kellner, M. I. 1996 Software Processes and Project Performance, Journal of Management Information Systems, 12(3), 187-205.

    Google Scholar 

  • DeVellis, R. F. 1991 Scale Development: Theory and Applications, Newbury Park, CA: Sage.

    Google Scholar 

  • Dybå, T. 2000a Improvisation in Small Software Organizations, IEEE Software, 17(5), Sept.–Oct.

  • Dybå, T. (Ed.) 2000b SPIQ—Software Process Improvement for better Quality: Methodology Handbook, Version 3.0 (Final), IDI Report 2/2000, Norwegian University of Science and Technology, Trondheim, Norway (in Norwegian).

    Google Scholar 

  • EFQM 1999 The EFQM Excellence Model, Brussels: European Foundation for Quality Management.

    Google Scholar 

  • El Emam 1998 The Internal Consistency of the ISO/IEC 15504 Software Process Capability Scale, Proceedings of the Fifth International Symposium on Software Metrics, IEEE Computer Society Press, pp. 72-81.

  • El Emam, K. and Birk, A. 2000 Validating the ISO/IEC 15504 Measure of Software Requirements Analysis Process Capability, IEEE Transactions on Software Engineering, 26(6), 541-566.

    Google Scholar 

  • El Emam, K. and Madhavji, N. H. 1995 The Reliability of Measuring Organizational Maturity, Software Process—Improvement and Practice, 1, 3-25.

    Google Scholar 

  • El Emam, K. and Madhavji, N. H. 1996 An Instrument for Measuring the Success of the Requirements Engineering Process in Information Systems Development, Empirical Software Engineering, 1(3), 201-240.

    Google Scholar 

  • El Emam, K., Fusaro, P. and Smith, B. 1999 Success Factors and Barriers for Software Process Improvement, in R. Messnarz, and C. Tully (Eds.) Better Software Practice for Business Benefit: Principles and Experiences, Los Alamitos, California: IEEE Computer Society Press, pp. 355-371.

    Google Scholar 

  • El Emam, K., Goldenson, D. R., McCurley J. and Herbsleb, J. 1998 Success or Failure? Modeling the Likelihood of Software Process Improvement, Technical Report, ISERN-98-15, International Software Engineering Research Network.

  • Feigenbaum, A. V. 1991 Total Quality Control, Fortieth Anniversary Edition, New York: McGraw-Hill.

    Google Scholar 

  • Fink, A. and Kosecoff, J. 1998 How to Conduct Surveys: A Step-By-Step Guide, Second Edition, Thousand Oaks, California: Sage Publications.

    Google Scholar 

  • French, W. L. and Bell, C. H. Jr. 1999 Organization Development: Behavioral Science Interventions for Organization Improvement, Sixth Edition, Upper Saddle River, New Jersey: Prentice-Hall.

    Google Scholar 

  • Fusaro, P., El Emam, K. and Smith, B. 1998 The Internal Consistency of the 1987 SEI Maturity Questionnaire and the SPICE Capability Dimension, Empirical Software Engineering, 3(2), 179-201.

    Google Scholar 

  • Garvin, D. A. 1983 Quality on the Line, Harvard Business Review, 61(5), 65-75.

    Google Scholar 

  • Garvin, D. A. 1984 Japanese Quality Management, Columbia Journal of World Business, 19(3), 3-19.

    Google Scholar 

  • Garvin, D. A. 1993 Building a Learning Organization, Harvard Business Review, 71(4), 78-91.

    Google Scholar 

  • Goldenson, D. R. and Herbsleb, J. (1995) After the Appraisal: A Systematic Survey of Process Improvement, its Benefits, and Factors that Influence Success, Technical Report, CMU/SEI-95-TR-009, Carnegie Mellon University, Software Engineering Institute.

  • Goldenson, D. R., El Emam, K., Herbsleb, J. and Deephouse, C. 1999 Empirical Studies of Software Process Assessment Methods, in K. El Emam and N. H. Madhavji (Eds.), Elements of Software Process Assessment and Improvement, Los Alamitos, California: IEEE Computer Society Press, pp. 177-218.

    Google Scholar 

  • Guilford, J. P. 1954 Psychometric Methods, 2nd edition, New York: McGraw-Hill.

    Google Scholar 

  • Harrison, R., Badoo, N., Barry, E., Biffl, S., Parra, A., Winter, B. and Wuest, J. 1999 Directions and Methodologies for Empirical Software Engineering Research, Empirical Software Engineering, 4(4), 405-410.

    Google Scholar 

  • Humphrey, W. S. 1989 Managing the Software Process, Reading, Massachusetts: Addison-Wesley.

    Google Scholar 

  • Humphrey, W. S. 1997 Managing Technical People: Innovation, Teamwork, and the Software Process, Reading, Massachusetts: Addison-Wesley.

    Google Scholar 

  • Hunt, R. and Buzan, T. 1999 Creating a Thinking Organization: Groundrules for Success, Hampshire, England: Gower.

    Google Scholar 

  • Hunter, J. E. and Schmidt, F. L. 1990 Methods of Meta-Analysis: Correcting Error and Bias in Research Findings, Newbury Park, California: Sage Publications.

    Google Scholar 

  • Ishikawa, K. 1986 Guide to Quality Control, Second Edition, New York: Quality Resources.

    Google Scholar 

  • Ishikawa, K. 1990 Introduction to Quality Control, London: Chapman & Hall.

    Google Scholar 

  • ISO/DIS 9000 2000 Quality Management Systems—Fundamentals and Vocabulary.

  • ISO/DIS 9004 2000 Quality Management Systems—Guidelines for Performance Improvements.

  • ISO/IEC TR 15504-7 1998 Information Technology—Software Process Assessment—Part 7: Guide for Use in Process Improvement.

  • Jeffery, D. R. and Votta, L. G. 1999 Guest Editor's Special Section Introduction, IEEE Transactions on Software Engineering, 25(4), 435-437.

    Google Scholar 

  • Juran, J. M. 1992 Juran on Quality by Design: The New Steps for Planning Quality into Goods and Services, New York: Free Press.

    Google Scholar 

  • Juran, J. M. and Godfrey, A. B. (Eds.) 1999 Juran's Quality Handbook, Fifth Edition, New York: McGraw-Hill.

    Google Scholar 

  • Kaiser, H. F. 1960 The Application of Electronic Computers to Factor Analysis, Educational and Psychological Measurements, 20, 141-151.

    Google Scholar 

  • Kaiser, H. F. 1970 A Second Generation Little Jiffy, Psychometrika, 35, 401-417.

    Google Scholar 

  • Kanuk, L. and Berenson, C. 1975 Mail Surveys and Response Rates: A Literature Review, Journal of Marketing Research, 12, 440-453.

    Google Scholar 

  • Kerlinger, F. 1986 Foundations of Behavioral Research, Holt, Rinehart and Winston.

  • Larsen, E. A, and Kautz, K. 1997 Quality Assurance and Software Process Improvement in Norway, Software Process—Improvement and Practice, 3(2), 71-86.

    Google Scholar 

  • Lawrence, P. R. and Lorsch, J. W. 1986 Organization and Environment: Managing Differentiation and Integration, 2nd Edition, Boston: Harvard Business School Press.

    Google Scholar 

  • Likert, R. 1932 A Technique for the Measurement of Attitudes, Archives of Psychology, 22(140).

  • Likert, R. 1967 The Human Organization: Its Management and Value, New York: McGraw-Hill.

    Google Scholar 

  • Likert, R. and Roslow, S. 1934 The Effects Upon the Reliability of Attitude Scales of Using Three, Five or Seven Alternatives, Working Paper, New York University.

  • Lissitz, R. W. and Green, S. B. 1975 Effects of the Number of Scale Points on Reliability: A Monte Carlo Approach, Journal of Applied Psychology, 60, 10-13, February.

    Google Scholar 

  • March, J. G. 1991 Exploration and Exploitation in Organizational Learning, Organization Science, 2(1), 71-87.

    Google Scholar 

  • March, J. G. 1999 The Pursuit of Organizational Intelligence, Malden, Massachusetts: Blackwell.

    Google Scholar 

  • Miller, G. A. 1956 The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information, Psychological Review, 63, 81-97.

    Google Scholar 

  • Neuman, W. L. 2000 Social Research Methods: Qualitative and Quantitative Approaches, Fourth Edition, Boston: Allyn and Bacon.

    Google Scholar 

  • Nevis, E. C., DiBella, A. J. and Gould, J. M. 1995 Understanding Organizations as Learning Systems, Sloan Management Review, 37, 73-85.

    Google Scholar 

  • Nonaka, I. 1991 The Knowledge-Creating Company, Harvard Business Review, 69(6), 96-104.

    Google Scholar 

  • Nonaka, I. 1994 A Dynamic Theory of Organizational Knowledge Creation, Organization Science, 5(1), 14-37.

    Google Scholar 

  • Nonaka, I. and Takeuchi, H. 1995. The Knowledge-Creating Company: How Japanese Companies Create the Dynamics of Innovation, New York: Oxford University Press.

    Google Scholar 

  • Novick, M. and Lewis, G. 1967 Coefficient Alpha and the Reliability of Composite Measurements, Psychometrika, 32, 1-13.

    Google Scholar 

  • Nunnally, J. C. 1978 Psychometric Theory, Second edition, New York: McGraw-Hill.

    Google Scholar 

  • Nunnally, J. C. and Bernstein, I. A. 1994 Psychometric Theory, Third edition, New York: McGraw-Hill.

    Google Scholar 

  • Powell, T. C. 1995 Total Quality Management as Competitive Advantage: A Review and Empirical Study, Strategic Management Journal, 16, 15-37.

    Google Scholar 

  • Rogers, E. M. 1995 Diffusion of Innovations, Fourth Edition, New York: The Free Press.

    Google Scholar 

  • Sanders, M. (Ed.) 1998 The SPIRE Handbook: Better, Faster, Cheaper Software Development in Small Organisations, Dublin: Centre for Software Engineering Ltd.

    Google Scholar 

  • Saraph, J. V., Benson, P. G. and Schroeder, R. G. 1989. An Instrument for Measuring the Critical Factors of Quality Management, Decision Sciences, 20(4), 810-829.

    Google Scholar 

  • Schaffer, R. H. and Thomson, H. A. 1992. Successful Change Programs Begin with Results, Harvard Business Review, 70, 80-89.

    Google Scholar 

  • Senge, P. M. 1990 The Fifth Discipline: The Art and Practice of the Learning Organization, New York: Doubleday.

    Google Scholar 

  • Senge, P. M., Kleiner, A., Roberts, C., Ross, R. and Smith, B. 1994 The Fifth Discipline Fieldbook: Strategies and Tools for Building a Learning Organization, New York: Currency/Doubleday.

    Google Scholar 

  • Senge, P. M., Kleiner, A., Roberts, C., Ross, R., Roth, G. and Smith, B. 1999 The Dance of Change: The Challenges of Sustaining Momentum in Learning Organizations, New York: Currency/Doubleday.

    Google Scholar 

  • Spector, P. 1992 Summated Rating Scale Construction: An Introduction, Sage University Paper series on Quantitative Applications in the Social Sciences, series no. 07-082, Newbury Park, CA: Sage.

    Google Scholar 

  • SPSS 1999a SPSS Base 9.0: User's Guide, Chicago, IL: SPSS Inc.

    Google Scholar 

  • SPSS 1999b SPSS Base 9.0: Applications Guide, Chicago, IL: SPSS Inc.

    Google Scholar 

  • Stelzer, D. and Mellis, W. 1998 Success Factors of Organizational Change in Software Process Improvement, Software Process—Improvement and Practice, 4(4), 227-250.

    Google Scholar 

  • Stelzer, D., Mellis, W. and Herzwurm, G. 1996 Software Process Improvement via ISO 9000? Results of Two Surveys among European Software Houses, Proceedings of the 29th Hawaii International Conference on Systems Sciences, January 3–6, Wailea, Hawaii, USA.

  • Stevens, J. 1992 Applied Multivariate Statistics for the Social Sciences, London: Lawrence Erlbaum.

    Google Scholar 

  • Straub, D. W. 1989 Validating Instruments in MIS Research, MIS Quarterly, 13(2), 147-169.

    Google Scholar 

  • Stålhane, T., Borgersen, P. C. and Arnesen, K. 1997 In Search of the Customer's Quality View, Journal of Systems Software, 38, 85-93.

    Google Scholar 

  • Saarinen, T. 1996 An Expanded Instrument for Evaluating Information System Success, Information & Management, 31, 103-118.

    Google Scholar 

  • Taguchi, G. 1986 Introduction to Quality Engineering: Designing Quality into Products and Processes, Tokyo: Asian Productivity Organization.

    Google Scholar 

  • Taguchi, G., Elsayed, E. A. and Hsiang, T. C. 1989 Quality Engineering in Production Systems, New York: McGraw-Hill.

    Google Scholar 

  • Teng, J. T. C., Jeong, S. R. and Grover, V. 1998 Profiling Successful Reengineering Projects, Communications of the ACM, 41(6), 96-102.

    Google Scholar 

  • Van de Ven, A. H. and Ferry, D. L. 1980 Measuring and Assessing Organizations, New York: John Wiley & Sons.

    Google Scholar 

  • Winter, S. G. 1994 Organizing for Continuous Improvement: Evolutionary Theory Meets the Quality Revolution, in J. A. C. Baum and J. Singh (Eds.) The Evolutionary Dynamics of Organizations, Oxford University Press.

  • Yusof, S. M. and Aspinwall, E. 1999 Critical Success Factors for Total Quality Management Implementation in Small and Medium Enterprises, Total Quality Management, 10 (4&5), 803-809.

    Google Scholar 

  • Zahran, S. 1998 Software Process Improvement: Practical Guidelines for Business Success, Harlow, England: Addison-Wesley.

    Google Scholar 

  • Zwick, W. R. and Velicer, W. F. 1986 Comparison of Five Rules for Determining the Number of Components to Retain, Psychological Bulletin, 99, 432-442.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. SINTEF Telecom and Informatics, Norway

    Tore Dyba

Authors
  1. Tore Dyba
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Dyba, T. An Instrument for Measuring the Key Factors of Success in Software Process Improvement. Empirical Software Engineering 5, 357–390 (2000). https://doi.org/10.1023/A:1009800404137

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1009800404137

Search

Navigation