Abstract
This paper proposes a software reliability prediction approach based on software metrics. Metrics measurement results are connected to quantitative reliability predictions through defect information and consideration of the operational environments. An application of the proposed approach to a safety critical software deployed in a nuclear power plant is discussed. Results show that the proposed prediction approach could be applied using a variety of software metrics at different stages of the software development life cycle and could be used as an indicator of software quality. Therefore the approach could also guide the development process and help make design decisions. Experiences and lessons learned from the application are also discussed.
Similar content being viewed by others
References
(2011) Tech Salary Survey Results. [Online] http://marketing.dice.com/pdf/Dice_2010-11_TechSalarySurvey.pdf
Butler RW, Finelli GB (1993) The infeasibility of quantifying the reliability of life-critical real-time software. IEEE Trans Softw Eng 19(1):3–12
Campbell D, Stanley J (1966) Experimental and quasi-experimental designs for research. Chicago, USA
Card DN, Agresti WW (1988) Measuring software design complexity. J Syst Softw 8(3):185–197
Chapman M, Solomon D (2002) Software metrics as error predictors
Dougherty EM, Fragola JR (1988) Human reliability analysis: a system engineering approach with nuclear power plant applications. John Wiley & Sons
Embrey DE (1983) The use of performance shaping factors and quantified expert judgement in the evaluation of human reliability: an initial appraisal. In: U.S. Nuclear Regulatory Commission, NUREG/CR-2986
Eom HS, Kang HG, Park KH et al. (2005) A study on the quantitative assessment method of software requirement documents using software engineering measures and Bayesian belief networks. Proc KNS Autumn Meet
Fenton NE, Pfleeger SL (1997) Software metrics: a rigorous and practical approach, 2nd edn. Int’l Thomson Computer Pres, New York
Fenton N et al (1998) Assessing dependability of safety critical systems using diverse evidence. IEEE Proceedings Software Engineering 145(1):35–39
Gaffney JE (1984) Estimating the number of faults in code. IEEE Trans Softw Eng 10(1):459–464
Garg RK, Sharma K, Nagpal CK, Garg R, Garg R, Kumar R, Sandhya (2013) Ranking of software engineering metrics by fuzzy-based matrix methodology. Software Testing, Verification and Reliability 23(2):149–168
Herrmann D (2000) Software safety and reliability: techniques, approaches, and standards of key industrial sectors, 1st edn. ed. Wiley-IEEE Computer Society
Huang C-Y, Lyu MR (2011) Estimation and analysis of some generalized multiple change-point software reliability models. IEEE Trans Reliab 60(2):498–514
IEEE (1988) IEEE standard dictionary of measures to produce reliable software 982.1. IEEE Computer Society
IEEE (2008) IEEE recommended practice on software reliability. New York, USA
International Function Point Users Group (2004) Function point counting practices manual (Release 4.2)
Ireson WG (1966) Reliability handbook. McGraw Hill
Jones C (1995) Measuring global software quality. Software Productivity Research, Burlington
Jones C (1996) Applied software measurement: assuring productivity and quality, 2nd ed. New York, USA
Kapur PK, Pham H, Anand S, Yadav K (2011) A unified approach for developing software reliability growth models in the presence of imperfect debugging and error generation. IEEE Trans Reliab 60(1):331–340
Kapur PK, Pham H, Aggarwal AG, Kaur G (2012) Two dimensional multi-release software reliability modeling and optimal release planning. IEEE Trans Reliab 61(3):758–768
Kaufman L, Dugan JB, Johnson B et al. (1998) Using statistics of the extremes for software reliability analysis of safety critical systems. Proc Ninth Int Symp Software Reliab Eng 355-363
Kristiansen M, Winther R, Natvig B (2011) A Bayesian hypothesis testing approach for finding upper bounds for probabilities that pairs of software components fail simultaneously. International Journal of Reliability, Quality and Safety Engineeering 18(3):209–236
Kumar C, Yadav DK (2014) Software defects estimation using metrics of early phases of software development life cycle. International Journal of System Assurance Engineering and Management. doi:10.1007/s13198-014-0326-2
Lawrence JD, Sicherman A, Person WL et al. (1998) Assessment of software reliability measurement methods for use in probabilistics risk assessment. Lawrence Livermore Nat’l Laboratory, FESSP
Lee W, Lee JK, Baik J et al. (2011) Software reliability prediction for open source software adoption systems based on early lifecycle measurements. 2011 I.E. 35th Ann Comput Software Applic Conf (COMPSAC) 366-371
Li M, Smidts C (2003) A ranking of software engineering measures based on expert opinion. IEEE Trans Softw Eng 29(9):811–824
Li M et al (2004) Validation of a methodology for assessing software reliability. Proc 15th Int Symp Software Reliab Eng, Saint-Malo, France, pp. 66-76
Lyu M, Ed.(1996) Handbook of software reliability engineering. New York, USA
Lyu M (2007) Software reliability engineering: a roadmap. In: Future of Software Engineering. Minneapolis, USA
Malaiya YK, Li N, Bieman JM et al. (1994) The relationship between test coverage and reliability. Proc 5th Int Symp Software Reliab Eng, Los Alamitos, pp. 186-195
McCabe T (1976) A complexity measure. IEEE Trans Softw Eng SE-2(4):308–320
McCabe T (1982) Structured testing: a software testing methodology using the cyclomatic complexity metric. National Bureau of Standards Report NBS 500-99. Washington, DC
Miller KW et al (1992) Estimating the probability of failure when testing reveals no failures. IEEE Trans Softw Eng 18(1):33–43
Misra RB, Kumar KS (2008) An enhanced model for early software reliability predcition using software engineering metrics. 2nd Int Conf Sec Syst Integrat Reliab Improve 177-178
Munson J, Khoshgoftaar T (1992) The detection of fault-prone program. IEEE Trans Softw Eng 18(5):423–433
Murine GE (1985) On validating software quality metrics. 4th Ann IEEE Conf Software Quality 39-44
Musa J (1990) Software reliability: measurement, prediction, application. New York, USA
Musa J (1992) The operational profile in software reliability engineering: an overview. Third Int Symp Software Reliab Eng 140-154
NASA (2004) Standard for Software Assurance. NASA, STD 8739.8
Okamura H, Dohi T. (2014) A novel framework of software reliability evaluation with software reliability growth models and software metrics. 2014 I.E. 15th Int Symp High-Assurance Syst Eng (HASE 97–104
Pandey AK, Goyal NK (2010) Fault prediction model by fuzzy profile development of reliability relevant software metrics. Int J Comput Applic 11(6):34–41
Pandey AK, Goyal NK (2013) Early fault prediction model by fuzzy profile development of reliabiliy relevant software metrics. Int J Comput Applic 11(6):34–41
Reason J (1990) Human error. Cambridge University Press
RTCA (1992) Software considerations in airborne systems and equipment certification. RTCA, RTCA/DO-178
Sandfoss RV, Meyer SA (1997) Input requirements needed to produce an operational profile for a new telecommunications system. Eighth Int Symp Software Reliab Eng 110
Schneidewind N (1997) Reliability modeling for safety-critical software. IEEE Trans Reliab 46(1):88–98
Shi Y, Kong W, Smidts C et al. (2007) Data collection and analysis for the reliability prediction and estimation of a safety critical system. Reliab Anal Syst Failure Data 1-2
Shi Y, Li M, Smidts C et al (2009) On the use of extended finite state machine models for software fault propagation and software reliability estimation. 6th American Nuclear Society International Topical Meeting on Nuclear Plant Instrumentation, Controls, and Human Machine Interface Technology. Knoxville, Tennessee
Smidts C, Li M (2000) Software engineering measures for predicting software reliability in safety critical digital systems. USNRC, Washington DC, NUREG/GR-0019
Smidts CS, Li M (2004) Preliminary validation of a methodology for assessing software quality. U.S. Nuclear Regulatory Commission, Washington, DC, NUREG/CR-6848
Smidts C, Shi Y (2009) A test coverage-based model for predicting software fault content and location. In: Slezak D, et al. (ed) Advances in software engineering 321-329
Smidts C, Shi Y (2010) Predicting the types and locations of faults introduced during an imperfect repair process and their impact on reliability international journal of system assurance engineering and management. Int J Syst Assurance Eng Manag 1(1):36–43
Smidts CS, Shi Y, Li M, Kong W, Dai J (2011) A Large Scale Validation of A Methodology for Assessing Software Reliability. U.S. Nuclear Regulatory Commission, Washington, DC, NUREG/CR-7042.
Sterritt R, Bustard D (2010) Autonomic computing—a means of achieving dependability?. 10th IEEE Int Conf Workshop Eng Comput-Based Syst 247-251
Stutzke MA, Smidts CS (2001) A stochastic model of fault introduction and removal during software development. IEEE Trans Reliab Eng 50(2):184–193
Thomason MG, Whittaker JA (1999) Properties of rare fail-states and extreme values of TTF in a Markov Chain Model for Software Reliabilit. University of Tennessee, Knoxville
Tukey JW (1953) The Problem of Multiple Comparisons. Technical report, Princeton University
Ullah N, Morisio M (2013) An empirical analysis of open source software defects data through software reliability growth models. 2013 I.E. EUROCON 460–466
van Manen S, Brandt E, van Ekris J, Geurts W (2013) TOPAAS: an alternative approach to software reliability quantification. Int J Quality Reliab Eng 31(2):183–191
Varkoi T (2013) Safety as a process quality characteristic. In: Woronowicz T, Rout T, O’Connor R, Dorling A (ed) Software process improvement and capability determination. Bremen, Germany, pp. 1-12
Voas JM (1992) PIE: a dynamic failure-based technique. IEEE Trans Softw Eng 18(8):717–727
Welker EL, Lipow M (1974) Estimating the exponential failure rate from data with no failure events. 1974 Ann Reliab Maintain Conf, Los Angeles 420-427
Wu H, Zhong Y, Chen Y et al. (2010) A software reliability prediction model based on benchmark measurement. 2nd IEEE Conf Inform Manag Eng (ICIME), 131-134
Yadav HB, Yadav DK (2014) Early software reliability analysis using reliability relevant software metrics. Int J Syst Assurance Eng Manag. doi:10.1007/s13198-014-0325-3
Yadav HB, Yadav DK (2015) A fuzzy logic approach for multistage defects density analysis of software. 4th Int Conf Soft Comput Problem Solving, 122-136
Yadav HB, Yadav DK (2015b) A fuzzy logic based approach for phase-wise software defects prediction using software metrics. Inf Softw Technol 63(1):44–57
Yadav DK, Chaturvedi SK, Misra RB (2012) Early software defects prediction using fuzzy logic. Int J Perform Eng 8(4):399
Zhang X, Pham H (2000) An analysis of factors affecting software reliability. J. Syst Software 50(1): 14
Acknowledgments
We would like to acknowledge the support for portions of this research through a contract with the U.S. Nuclear Regulatory Commission (NRC) (NRC-04-09-146). This paper was prepared as an account of work sponsored by an agency of the U.S. Government. Neither the U.S. Government nor any agency thereof, nor any of their employees, makes any warranty, expressed or implied, or assumes any legal liability or responsibility for any third party’s use, or the results of such use, of any information, apparatus, product, or process disclosed in this report, or represents that its use by such third party would not infringe privately owned rights. The views expressed in this paper are not necessarily those of the U.S. Nuclear Regulatory Commission.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by: Vittorio Cortellessa
Appendices
Appendix A. Thirteen Software Metrics used in this Study
1.1 Bugs per Line of Code (BLOC)
The goal of this measure is to give a crude estimate of the number of faults in a program module per line of code.
1.2 Cause & Effect Graphing (CEG)
A CEG is a formal translation of natural-language specification into its input conditions and expected outputs. The graph depicts a combinatorial logic network. CEG aids in identifying requirements that are incomplete and ambiguous. This measure explores the inputs and expected outputs of a program and identifies the ambiguities. Once these ambiguities are eliminated, the specifications are considered complete and consistent. The measure is computed as follows:
Where:
- A existing :
-
is the number of ambiguities in a program remaining to be eliminated and
- A tot :
-
is the total number of ambiguities identified.
1.3 Software Capability Maturity Model (CMM)
CMM is a framework that describes the key elements of an effective software process. The goal of this measure is to describe the principles and practices underlying software-process maturity and to help software organizations improve the maturity of their software processes (IEEE 1988):
1.4 Completeness (COM)
The COM measure determines the completeness of the software requirements specifications (SRS). This measure provides a systematic guideline to identify the incompleteness defects in SRS. COM is the weighted sum of ten derivatives, D1 through D10 (IEEE 1988; Chapman and Solomon 2002):
Where:
- COM :
-
is the completeness measure
- W i :
-
is the weight of the ith derived measure
- Di :
-
is the i th derived measure calculated from the primitive measures B i (i = 1,…,18).
1.5 Cyclomatic Complexity (CC)
This measure determines the structural complexity of a coded module. The Cyclomatic Complexity (CC) of a module is the number of linearly independent paths through a module. The cyclomatic complexity for the i th module is originally defined by McCabe (Murine 1985; McCabe 1976) as:
Where:
- CC i :
-
is the cyclomatic complexity measure of the ith module
- E i :
-
is the number of edges of the i th module (program flows between nodes)
- N i :
-
is the number of nodes of the i th module (sequential groups of program statements).
1.6 Coverage Factor (CF)
The coverage factor is defined as the probability that the system recovers from a fault given that a fault occurs and can be formalized as follows:
Where:
Pr is the probability of H(g) = 1 when g ∈ G;
H is a variable characterizing the handling of a particular faulty condition
G is the complete input space of a fault tolerance mechanism
g is a faulty condition, or a point in space G.
1.7 Defect Density (DD)
Defect density is defined as the number of defects remaining divided by the number of lines of code in the software. The defects are discovered by independent inspection. Defect Density is given as:
Where:
- D i :
-
is the number of unique defects detected during the ith design or code inspection that still remain in the code.
- KSLOC :
-
is the number of source lines of code (LOC) in thousands.
1.8 Fault Days Number (FD)
This measure represents the number of days that faults remain in the software system from introduction to removal. The fault day measure evaluates the number of days between the time a fault is introduced into a system and until the point the fault is detected and removed (Smidts & Li 2000; McCabe 1982), such that:
Where:
- FD :
-
is the fault-days for the total system
- FD i :
-
is the Fault-days for the ith fault
- fin i :
-
is the date at which the ith fault was introduced into the system
- fout i :
-
is the date at which the ith fault was removed from the system
1.9 Function Point (FP)
Function Point is a measure designed to determine the functional size of the software. The Function Point Counting Practices Manual is the definitive description of the Function Pointing Counting Standard. The latest version is Release 4.2, which was published in 2004 (Herrmann 2000).
1.10 Requirement Specification Change Request (RSCR)
RSCR is defined as the number of change requests that are made to the requirements specification. This measure indicates the stability and/or growth of the functional requirements. The requested changes are counted from the first release of the requirements specification document to the time when the product begins its operational life.
RSCR = Σ(requested changes to the requirements specification)
Where:
The summation is taken all requirements change requests initiated during the software development life cycle.
1.11 Requirements Traceability (RT)
According to IEEE (IEEE 1988), the requirements traceability measure aids in identifying requirements that are either missing from, or in addition to, the original requirements. Requirements traceability is defined as:
Where:
- RT :
-
is the value of the measure requirements traceability
- R 1 :
-
is the number of requirements met by architecture, and
- R 2 :
-
is the number of original requirements.
1.12 System Design Complexity (SDC)
The goal of SDC is to identify the system complexity. System design complexity is a function of the average complexity of the various modules where module design complexity is a combination of structural complexity and data complexity:
Where:
- n :
-
is the total number of the modules in the system
- f(i) :
-
is the fanout of the i th module and
- ϑ(i) :
-
is the number of I/O variables in the i th module
1.13 Test Coverage (TC)
As in IEEE (IEEE 1988), Test coverage (TC) is the percentage of requirement primitives implemented multiplied by the percentage of primitives executed during a set of tests. A simple interpretation of test coverage can be expressed by the following formula:
Where:
- IC :
-
is the implemented capabilities
- RC :
-
is the required capabilities
- PPT :
-
is the tested program primitives and
- TPP :
-
is the total program primitives.
Appendix B.The M-D Models for Each of the Thirteen RePSs
2.1 Bugs per Line of Code (BLOC)
Gaffney (Gaffney 1984) established that the number of defects remaining in the software (N G ) could be expressed empirically as a function of the number of line of codes:
Where:
- i :
-
is the module index
- M :
-
is the number of modules, and
- S i :
-
is the number of lines of code for the ith module.
To obtain the M-D model, the next step is the partitioning of the defects based on their criticality. Jones (Jones 1996) classifies defects into four severity levels, given as: 1) Severity level 1: Critical problem (software does not operate at all); 2) Severity level 2: Significant problem (major feature disabled or incorrect); 3) Severity level 3: Minor problem (some inconvenience for the users); 4) Severity level 4: Cosmetic problem (spelling errors in messages; no effect on operations). Only defects of Severity level 1 and Severity level 2, labeled critical defects and significant defects, should be considered while estimating software reliability. Defects with Severity level 3 and level 4, called minor defects and cosmetic defects do not have an impact on the functional performance of the software system. Thus, they have no effect on reliability quantification. Using Table 13 from (Jones 1996), for “US Average Percentages for Delivered Defects by Severity Level” which provides proportions of delivered defects at different severity levels and various functional sizes and logarithmic interpolation, the percentages of delivered defects by severity levels for a software system of the functional size of APP can be obtained as.
So the number of delivered defects of interest (N) can be obtained as:
Where:
SL(FP) is the percentage of defects introduced at the severity level of interest, i.e. at severity level 1and level 2 for a software system of functional size FP. This value for APP is equal to 0.1391 (0.0185 + 0.1206).
2.2 Cause & Effect Graphing (CEG)
CEG is a measurement of the ratio of resolved ambiguities in the SRS (see Appendix A). To determine the value of CEG, inspections of the SRS are carried out using well defined inspection rules. Detailed inspection rules are provided in (Smidts et al. 2011). As ambiguities (i.e. defects) are uncovered their descriptions, their locations and their types are recorded. Information of value to the reliability prediction process discussed in this paper relates to unresolved defects and includes Nf,CEG C = Aexisting, the number of remaining ambiguities/defects in the current version identified through CEG measurement, Li, CEG (i = 1, Nf,CEG C), location of the remaining defects/ambiguities in the current version identified through CEG measurement and Tyi,CEG (i = 1, Nf,CEG C), defect type.
2.3 Software Capability Maturity Model (CMM)
Historical industry data collected by Software Productivity Research Inc (Jones 1995) links the CMM level to the number of defects per function points. Table 14 presents this data.
Using Table 13 from (Jones 1996) for “US Averages Percentages for Delivered Defects by Severity Level” and logarithmic interpolation, the percentages of delivered defects by severity level can be obtained.
Therefore, the number of defects based on the measurement of CMM can be expressed as:
Where:
- avn d/fp,CMM :
-
is the average number of defects for a certain CMM level
- FP :
-
is the number of function points for the software system under study
- SL(FP) :
-
is the percentage of defects introduced at the severity level of interest (i.e. severity level 1 and severity level 2) for a software system of functional size FP. This value for APP is equal to 0.1391 (0.0185 + 0.1206).
2.4 Completeness (COM)
COM is a measurement of the completeness of the requirements (SRS) (see Appendix A). To determine the value of COM, inspections of the SRS are carried out using well defined inspection rules. Detailed inspection rules are provided in (Smidts et al. 2011).
These inspections uncover defects such as functions which are not satisfactorily defined, data references that have no origin etc. As defects are uncovered their descriptions, their locations and their types are recorded. Information of value to the reliability prediction process discussed in this paper relate to unresolved defects and include Nf,COM C = ∑ i = 7 i = 1 B (2i + 1) + B 16 + B 18, the number of remaining defects in the current version identified through COM measurement, Li, COM C (i = 1, Nf,COM C), location of the remaining defects in the current version identified through COM measurement and Tyi,COM C (i = 1, Nf,COM C), defect type.
2.5 Cyclomatic Complexity (CC)
An empirical correlation was derived to relate cyclomatic complexity and number of defects. The correlation is based on an experimental data set composed of system software applications developed by graduate students. The empirical correlation was established by following the form proposed in (International Function Point Users Group 2004):
Where:
- SLI CC :
-
is the SLI value of the cyclomatic complexity factor
- f i :
-
is the failure likelihood fi used for SLI1 calculations (International Function Point Users Group 2004)
- p i :
-
is the percentage of modules whose cyclomatic complexity belong to the ith level, i = 1, 2, …, 9.
SLI stands for Success Likelihood Index which is used to represent the likelihood of an error occurring in a particular situation depends on the combined effects of a relatively small number of performance influencing factors (PIFs). SLI was used as an index which quantifies whether a particular environment will increase the human error probability or decrease it (with respect to a “normal situation”) (Malaiya et al. 1994). SLICC is related to the likelihood that developers will err (i.e. introduce fault in the software product and/or fail to remove them) because of the cyclomatic complexity of the modules.
The number of defects predicted based on CC measurement results is:
Where:
- SIZE:
-
is the size of the delivered source code in terms of LOC (Line of Code)
2.6 Coverage Factor (CF)
The coverage factor is defined as the probability of system recovery given that a fault exists (see Appendix A). Coverage analysis specifically targets hardware failures that might be experienced by the microprocessor on which the safety critical software runs. The goal is to identify faults in the fault tolerance mechanism. The number of defects used for reliability prediction is the total number of hardware faults injected to the system while the system could not recovery from the faulty state to normal/fail-safe/recoverable state (Smidts et al. 2011), i.e. Nf,CF C = ∑ n i = 1 N i unre cov erable .
2.7 Defect Density (DD)
DD is a measurement of the ratio of defects identified in the software (see Appendix A). To determine the value of DD inspections of the SRS, SDD and code are carried out using well defined inspection rules. Detailed inspection rules are provided in (Smidts et al. 2011). These inspections uncover defects. As defects are uncovered their descriptions, their locations and their types are recorded. Information of value to the reliability prediction process discussed in this paper relate to unresolved defects in the current version and include Nf,DD C = ∑ I i = 1 D i × u C i (where μ C i is the fraction of defects unresolved in the current version), the number of remaining defects in the current version identified through DD measurement, Li, COM (i = 1, Nf,DD C), location of the remaining defects in the current version identified through DD measurement and Tyi,DD (i = 1, Nf,DD C), defect type.
2.8 Fault Days Number (FD)
Based on the cumulative characteristic of the FD metric and by using the concepts introduced in (Stutzke & Smidts 2001), one can show that FD is related to μ U (t) by the following equation:
Where:
- μ U (t):
-
is the fault count at time t
This equation shows the direct relationship between the measured real FD and the corresponding fault count. The number of faults can be obtained using this equation once FD is known (i.e. measured). However, the real FD cannot be obtained experimentally since not all the faults can be discovered during the inspection. One can only obtain the apparent FD, FDA which corresponds to faults identified through the inspection process and removed through repair. One can relate FDA to FD by:
Where:
- γ(t; ϑμ H , z a , μ R ):
-
is a function of ϑμ H , z a , μ R
- ϑμ H :
-
is the estimate of the fault introduction rate
- z a :
-
is the intensity function of per-fault detection
- μ R :
-
is the expected change in fault count due to each repair.
Therefore, one can still obtain the fault count based on the measured apparent FD as shown by:
The fault count at time t, μ U (t) can be related to Ne,FD C by:
Where:
- tC :
-
is the time at which development activities carried out on the current version end.
In addition to this information, defect information related to the previous version of the software is also available and can be used for reliability prediction. Information of value to the reliability prediction process discussed in this paper then includes Nf,FD P, the number of defects found in the previous version of the software, Li,FD P (i = 1, Nf,FD P), location of the defects found through testing of the previous version of the software and Tyi,FD P (i = 1, Nf,FD P), defect type.
2.9 Function Point (FP)
Jones’ empirical industry data (Jones 1996) links the FP to the number of defects per function point for different categories of applications. Table 15 (Table 3.46 in (Jones 1996)) provides the average numbers for delivered defects per function point for different types of software systems. Logarithmic interpolation can then be used for number of defects quantification.
Therefore, the number of defects based on the measurement of FP can be expressed as:
Where:
- add fp,systype :
-
is the average for delivered defects per function point obtained from Table 15 for a software system of type systype. For APP, systype=”systems”
- SL(FP) :
-
is the percentage of defects introduced at the severity level of interest (severity level 1 and 2) for a software system of functional size FP. This value for APP is equal to 0.1391 (0.0185 + 0.1206).
2.10 Requirement Specification Change Request (RSCR)
To link requirements specification change requests to the reliability of a software system, a derived measure called REVL was used. The measure is:
Where:
- REVL :
-
is a measure of requirements evolution and volatility
- SIZE changed due to RSCR :
-
is the size of changed source code corresponding to RSCR, in Kilo Line of Code (KLOC)
- SIZE delivered :
-
is the size of the delivered source code, in KLOC.
- SLI :
-
for the REVL, denoted by SLI RSCR , is estimated using the value of REVL, as shown in Table 16 (Table 17).
The number of defects predicted based on RSCR measurement results is:
2.11 Requirements Traceability (RT)
The requirements traceability measure aids in identifying software implementation defects such as requirements that are either missing from, or in addition to, the original requirements (see Appendix A). To determine the value of RT inspections of the SRS, SDD and code are carried out using well defined inspection rules. Detailed inspection rules are provided in (Smidts et al. 2011). As defects are uncovered their descriptions, their locations and their types are recorded. Information of value to the reliability prediction process discussed in this paper relate to unresolved defects in the current version and include Nf,RT C , the number of remaining defects in the current version identified through RT measurement, Li, RT (i = 1, Nf,RT C), location of the remaining defects in the current version identified through RT measurement and Tyi,RT (i = 1, Nf,RT C), defect type.
2.12 System Design Complexity (SDC)
The empirical relations between SDC and defect rate are investigated in (Card and Agresti 1988) and the expected number of defect can be estimated using the following equation:
Where:
- SDC :
-
is the measured system design complexity
- SIZE developed :
-
is the developed line of code (in KLOC) which includes all newly developed source lines of code plus 20 % of reused source lines of code and
- SL(FP) :
-
is the percentage of defects introduced at the severity level of interest (severity level 1 and 2) for a software system of functional size FP. This value for APP is equal to 0.1391 (0.0185 + 0.1206).
2.13 Test Coverage (TC)
As noted in Appendix A, test coverage (TC) is the percentage of requirement primitives implemented multiplied by the percentage of primitives executed during a set of tests. TC is defined as:
If we assume that all requirements have been implemented and if we further assume that the tested program primitives are represented by the lines of code, the definition of test coverage then becomes:
Where:
- C 1 :
-
is test coverage obtained through testing.
- LOC Tested :
-
is the number of lines of code that are being executed by the test data listed in the test plan.
- LOC Total :
-
is the total number of lines of code.
Malaiya et al. investigated the relationship between defect coverage, C0, and statement coverage, C1. In (Malaiya et al. 1994), the following relationship was proposed:
Where:
- a 0 , a 1 , a 2 :
-
are coefficients
- C 1 :
-
is the statement coverage.
The number of defects remaining in the software then becomes N C e,TC is:
Where:
- N 0 :
-
is the number of defects found by test cases provided in the test plan
- C 0 :
-
is the defect coverage.
In addition to this information, defect information related to the N0 defects identified by the test plan is also available and can be used for reliability prediction. Information of value to the reliability prediction process discussed in this paper then includes Nf,TC P = N0, the number of defects found in the previous version of the software, Li, TC P (i = 1, Nf,TC P), location of the defects found through testing of the previous version of the software and Tyi,TC P (i = 1, Nf,TC P), defect type.
Rights and permissions
About this article
Cite this article
Shi, Y., Li, M., Arndt, S. et al. Metric-based software reliability prediction approach and its application. Empir Software Eng 22, 1579–1633 (2017). https://doi.org/10.1007/s10664-016-9425-9
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10664-016-9425-9