Defining acceptable limits for the metrological traceability of specific measurands

Introduction

Over recent years, metrological traceability of analytical results has become a major focus in laboratory medicine and there is a concerted effort to standardize results produced by different laboratories and on different platforms [1, 2]. These efforts are being driven by a coalition of professional organizations such as IFCC and AACC, metrological organizations such as BIPM, through Joint Committee for Traceability in Laboratory Medicine (JCTLM), and the in-vitro diagnostic (IVD) manufacturers [3]. The aim of these efforts is to ensure equivalence of patient results for the measurement of the same measurand, that is, laboratory results should be equivalent no matter where and on which platform they are generated. One important reason for adopting a traceable, metrological-based approach is that it will allow the use of common reference intervals and clinical decision limits [4].

A number of documents describing traceability in Laboratory Medicine are available but none of these actually define the analytical requirements for a traceability chain to be clinically acceptable [5, 6]. However, establishing traceability of results for a test measurement should be inseparably linked to the definition of acceptable measurement uncertainty to fit the intended clinical application (“fitness for purpose”) [7]. In 2004, Thienpont etal. [8] first pointed out that the absence of clearly defined tolerable deviations derived from clinical needs “might results in a large grey zone with respect to the extent of traceability expected from IVD manufacturers, partially or totally invalidating its theoretical advantages”. Consequently, an objective approach needs to be applied to every measurand determined in the clinical laboratory in order to establish if the current status of the uncertainty budget of its measurement associated with the proposed metrological traceability chain is suitable for the clinical application of the test [9]. As an example, using the measurement of albumin in serum as a model, Infusino etal. [10] recently demonstrated that the reference measurement system (and the associated uncertainty) currently available is probably not enough to guarantee the accuracy needed for the clinical usefulness of this protein.

With these premises, the IFCC WG-AETR has been formed

1. to specifically define clinically acceptable limits for the metrological traceability of specific measurands in order to promote the:

   • harmonization of patient results for their better clinical application;

2. to cooperate with manufacturers, regulatory bodies and end-users to make patient results traceable to higher order reference materials and methods whenever feasible.

With the adoption of the IVD Directive (Directive of the European Parliament on In Vitro Medical Devices Directive 98/79/EC), manufacturers are compelled to have traceability of the values assigned to their calibrators if a suitable reference measurement system is available. They carry out this process by using a calibration hierarchy to ensure metrological traceability. An example of a traceability chain typically used by manufacturers is shown for glucose in Figure 1. The chain commences with the definition of the measurand and then proceeds through a series of related steps in which a hierarchical series of calibration materials are prepared, each related to the previous step in the chain and ultimately back to the measurand SI definition. Due to cost and limited resources, IVD manufacturers, however, do not perform the full traceable series of steps to value assign every new lot of assay calibrator. They often rely on value transfer from their internally stored (“master”) calibrator material. In most cases, this procedure is probably valid, but there have been cases when the stored calibrator has degraded or changed in some way resulting in the final value assignment to laboratory calibrators being incorrect [11].

Figure 1

An example of a typical traceability chain used by manufacturers for the measurement of glucose showing the relationship between the definition of the measurand, glucose to the reference method and then to the various calibrators.

Although all the major IVD manufacturers have developed traceability profiles for many of their products, there is still no equivalence between results for many commonly used measurands even though higher order reference material or reference procedures exist. One reason for this is that uncertainty limits for the different steps in the chain and measurement errors have never been appropriately assessed and estimated. In other words, although a particular calibrator from a manufacturer may be traceable along a chain, there are no allowable performance criteria defined to ensure the obtained final measurement meets acceptable specifications, i.e., fit for purpose.

We have illustrated this in Figure 2 in which there are two traceability paths for a measurand, say from two different manufacturers. The assigned values for the end calibrators from the two pathways would have significantly different values resulting in different laboratory results being measured on a patient. The assigned values for the two calibrators are both derived from valid traceability chains, but produce results that would not be equivalent. For these two systems to produce equivalent laboratory results for a measurand, would require the use of a correction factor determined by a correlation study at the steps where there is divergence. Although this would remove the bias between the two chains, it would impact on the measurement uncertainty of the final product. This scenario is commonly seen in immunoassays when manufacturers use proprietary antibodies to measure products. Obviously, this also means that clinical decision limits or reference intervals would be quite different for the two assays unless the bias is removed.

Figure 2

Two schematic traceability processes which are both metrological acceptable.

In both cases, the final product is traceable to the measurand definition but the divergence would result in significantly different patient results. Note that both processes could have the same allowable measurand limit.

Traceability chain and clinically acceptable limits

Within a traceability chain, three main elements contribute to the uncertainty and bias [12, 13]:

1. the components within the chain – reference materials, reference procedures, manufacturer’s calibrators;

2. the components of the procedures – values transfer procedures, reagents, instruments, staff;

3. the sample – preanalytical effect, biological variation.

It is generally agreed that the uncertainty that is fit for purpose should be defined for the entire traceability chain, from the reference materials, through the manufacturing processes for calibrator value assignment to the final result reported to the end-users [4, 14, 15]. It is not always possible to determine the individual uncertainties for all the specific components in a traceability chain and thus, an approach can be used in which the chain is divided into parts so that the data generated can be used to calculate the combined uncertainty [6].

There are two important limits that should be defined for the clinical application of the test once a traceability chain has been defined:

1. the allowable limits for the uncertainty of manufacturer’s calibrators;

2. the allowable error for measurements done by individual clinical laboratories.

In doing this, one should clearly distinguish between the bias issue and the uncertainty at the level of manufacturer’s calibrator. At the calibrator level if a bias is known, the Guide to the Expression of Uncertainty in Measurement [16] asks manufacturers for its compensation by adjusting the value of the calibrators. After this realignment, only uncertainty of calibrator remains and this will therefore be added to the uncertainty of previous chain levels to obtain the combined uncertainty of the commercial product calibrator [7, 17]. However, also under perfectly traceable conditions, it is impossible for individual laboratories to avoid a bias in their daily activity. This requires the introduction of a goal for tolerable bias when discussing the error of the individual measurements on biological samples. The definition of this error can be made according to total error theory, which requires the definition of a bias goal in addition to the goal for standard uncertainty that, at this lowest measurement level, includes the accumulated uncertainty of the corresponding traceability chain and the uncertainty due to “random“ effects in the laboratory.

Approaches for defining allowable limits

The IFCC-IUPAC Stockholm Conference for setting quality specification in Laboratory Medicine [18] agreed on a hierarchical approach to defining analytical performance based on:

1. the evaluation of the effect of analytical performance on clinical outcome in the specific clinical setting;

2. data based on biologic variation information or on clinician survey; and

3. data based on clinical and laboratory experts’ opinion and published recommendations.

As the outcome-based information is missing for the majority of measurands, the approach using biological variation data has generated more attention, even though recent publications have questioned the reliability and robustness of available information [19, 20].

Biological variation approach

The concept that for desirable performance, analytical variation (CV_a) should be less than half the within-subject biological variation (CV_i) was developed over 30 years ago, i.e.,

By adopting 0.5 CV_ias the goal for expanded uncertainty of measurement [obtained by multiplying the relative combined standard uncertainty by a coverage factor of 2 (95% level of confidence)], it can be calculated that the true result variability due to biological variation would not be increased by more than 12% due to the analytical uncertainty.

As described above, bias in individual measurements can also have a significant effect on a patient result [7]. For desirable assay performance, it has been accepted that the analytical bias (B_a) should be less than one-quarter of the total biological variation, where total biological variation is made up of within-subject (CV_i) and between-subject biological variation (CV_g), i.e.,

These two equations can then be used to define general desirable quality specifications for total allowable error (TE_a) as

where 1.65 is the Z-score at 95% probability [21].

Different levels of analytical quality can be defined by modulating the described theory [22] so that, minimum analytical performance goals can be defined as

expanded uncertainty <0.75 CV_i

B_a<0.375 (CV_I²+CV_g²)^1/2

TE_a<1.65 (0.75 CV_i)+0.375 (CV_I²+CV_g²)^1/2

and optimum desirable performance as:

expanded uncertainty <0.25 CV_i

B_a<0.125 (CV_I²+CV_g²)^1/2

TE_a<1.65 (0.25 CV_i)+0.125 (CV_I²+CV_g²)^1/2

If for being acceptable, the degree of uncertainty (expanded) of a measurand in the clinical laboratory (including the accumulated uncertainty of the corresponding traceability chain) using unbiased assays should stay within ±0.25 CV_i, ±0.50 CV_i or ±0.75 CV_i (optimum, desirable or minimum quality level, respectively), decisions need to be made on what proportion of this budget can be used up in the traceability chain to ensure enough budget is left for use in routine analysis. In the past, only the contribution of the uncertainty related to the reference procedures has been considered. In particular, Stöckl etal. [23, 24] proposed that this uncertainty should be <0.2 times the maximal tolerated limit, i.e., the clinically allowable uncertainty of measurements.

We have selected three measurands to analyze the current situation and evaluate if traceability is compatible with the application of the biological variation approach.

Alternate statistical approach

As demonstrated by the example of sodium, the approach based on biologic variability may be too rigid to be able to be adopted at present for all measurands. An alternative approach is to use probability theory. The question then becomes “In a traceability chain, what is the probability that the final sample result has the desired value?” Obviously, this can never be absolutely certain, but the more times the measurements at the individual steps of a traceability chain are repeated, the more certain the final result. This approach requires a statistical goal to be set for the probability of the final step being correct, e.g., 95% certainty.

In the glucose example shown in Figure 1, there are four measurement steps, weighing of the primary calibrator, measurement of the secondary calibrator using a reference procedure (IDMS) by the manufacturer, measurement of the manufacturer’s master calibrator by the manufacturer’s selected measurement procedure and finally measurement of the product calibrator by the manufacturer’s standing procedure. Statistically independent repeated observations are determined at each step and this chain of repeated measurement procedures would give mean±SD for each step [16]. Assuming the repeated measurements are normally distributed, then the confidence interval (CI) for each step can be calculated and it would be reasonable to have 95% CI as the level of acceptability.

However, for this approach to be valid, there needs to be defined limits for the 95% CI because this interval is dependent on the dispersion of the results so that an imprecise method would have a much wider interval. We suggest the 95% CI be based on desirable TE_a. For serum glucose the TE_a is 6.9% and if the glucose value was 5.05 mmol/L would give a range of 5.07±3.45% (± half the TE_a of 6.9%), that is 4.88–5.22 mmol/L.

In a theoretical experiment in which glucose was measured 10 times so that the final result and uncertainty fitted with the 95% CI of 4.88–5.22 mmol/L, the results were 5, 5.05, 5.1, 4.95, 5.0, 5.1, 4.95, 5.1, 5.15, 4.95 mmol/L. These results give a mean±SD of 5.035±0.0747 mmol/L and 95% CI of 4.889–5.171. These results would be acceptable but it means that the measuring procedure would be required to generate a standard deviation of 0.0747 which is 1.47% or less at the mean concentration of 5.035 mmol/L. In one of our laboratory, the CV for glucose is 1.7% at 4.7 mmol/L so it should be possible for the manufacturer to meet the 1.5% SD requirement using their very precise, unbiased and standardized procedures although it may also require a significant number of repeat determinations at each step.

Conclusions

Patients often have clinical results from different laboratories and without traceability to a reference measurement system, physicians have little or no information about the equivalence of results. To ensure better healthcare for patients, the harmonization of laboratory results is a major initiative involving health professionals, metrological organizations and IVD manufacturers. Metrological traceability means that all measurements made with a calibrated instrument or device are directly traceable back to a stable reference such as the SI. Currently, there are no defined criteria for acceptable limits for the traceability chain of specific measurands and the WG-AETR has been identifying approaches to define these limits.

In particular, two limits should be defined for the clinical application of the test once a traceability chain has been defined: allowable limits for the uncertainty of manufacturer’s calibrators and the allowable error for measurements done by individual clinical laboratories. Basically, it is considered appropriate that any defined limits should be based on biological variation in the absence of agreed analytical goals based on clinical outcomes. For the former, desiderable (<0.50 CV_i) or minimum (<0.75 CV_i) goals can be applied for defining the total expanded uncertainty and a concomitant evaluation of the three major sources of uncertainty in the traceability chain (reference materials, manufacturer’s calibrators, and value transfer procedures and uncertainty related to the assay) should be performed to understand the contribution of each source and the state-of-the-art of the measurement [9, 10]. The allowable error for measurements done by individual clinical laboratories can actually refer to the classical TE_a concept.

We have used the TE_a for creatinine and glucose to show that it is possible for manufacturers correctly applying the traceability approach to provide commercial calibrators with an uncertainty that leaves enough uncertainty budget to enable clinical laboratories to measure these measurands with an accuracy suitable for clinical application of the test. However, there are other measurands, such as sodium or albumin in serum, in which available reference materials display too high uncertainties and/or current methods are too inaccurate to allow the fulfilment of goals derived from biologic variation. These are measurands for which, depending on their biology and strict homeostatic control, the requested analytical quality is high and the performance of field methods should be extremely good to permit their application in clinical setting.

The next step for the Working Group is to apply this approach to every measurand measured in the clinical laboratory for which a reference material exists in the JCTLM database in order to establish if the current status of the uncertainty budget of its measurement associated with the proposed metrological traceability chain is suitable for clinical application of the test. As reported by others [27, 28], we strongly believe that the specifications of certified reference materials and calibration materials should be defined by the performance needs of the clinical assays; therefore, stakeholders should be prepared to provide new suitable reference materials together with improved assay methods whenever the clinical application of the test is made questionable. The required quality of these materials should be based on data showing the effect of assay performance on clinical outcome.