Operator variability in scan positioning is a major component of HR-pQCT precision error and is reduced by standardized training

Abstract

Summary

In this study, we determined that operator positioning precision contributes significant measurement error in high-resolution peripheral quantitative computed tomography (HR-pQCT). Moreover, we developed software to quantify intra- and inter-operator variability and demonstrated that standard positioning training (now available as a web-based application) can significantly reduce inter-operator variability.

Introduction

HR-pQCT is increasingly used to assess bone quality, fracture risk, and anti-fracture interventions. The contribution of the operator has not been adequately accounted in measurement precision. Operators acquire a 2D projection (“scout view image”) and define the region to be scanned by positioning a “reference line” on a standard anatomical landmark. In this study, we (i) evaluated the contribution of positioning variability to in vivo measurement precision, (ii) measured intra- and inter-operator positioning variability, and (iii) tested if custom training software led to superior reproducibility in new operators compared to experienced operators.

Methods

To evaluate the operator in vivo measurement precision, we compared precision errors calculated in 64 co-registered and non-co-registered scan-rescan images. To quantify operator variability, we developed software that simulates the positioning process of the scanner’s software. Eight experienced operators positioned reference lines on scout view images designed to test intra- and inter-operator reproducibility. Finally, we developed modules for training and evaluation of reference line positioning. We enrolled six new operators to participate in a common training, followed by the same reproducibility experiments performed by the experienced group.

Results

In vivo precision errors were up to threefold greater (Tt.BMD and Ct.Th) when variability in scan positioning was included. The inter-operator precision errors were significantly greater than the short-term intra-operator precision (p < 0.001). New trained operators achieved comparable intra-operator reproducibility to experienced operators and lower inter-operator reproducibility (p < 0.001). Precision errors were significantly greater for the radius than for the tibia.

Conclusion

Operator reference line positioning contributes significantly to in vivo measurement precision and is significantly greater for multi-operator datasets. Inter-operator variability can be significantly reduced using a systematic training platform, now available online (http://webapps.radiology.ucsf.edu/refline/).

Appendix: HR-pQCT operator training

To train new operators, we developed custom software and a detailed scan positioning protocol. The software comprised two modules, one to train reference line positioning and one to evaluate positioning performance for the purposes of operator certification. The graphical user interface reproduces the original acquisition software (Fig. 6(c)). Graphical control elements on the left side of the interface allowed the user to select training or evaluation modules, and from a series of scout view collections (eight scout images each) according to anatomic site, laterality, and difficulty. In training mode, the software provides operators both immediate and summary feedback on their positioning accuracy. The immediate feedback consists of a traffic light that is illuminated after each reference line is positioned. A green, yellow, or red light is shown, corresponding to “exact”, “good”, or “out of range” positions, respectively. The final feedback consists of a separate window displaying a summary report of operator performance after completing the selected collection (Fig. 4d). Positioning feedback was determined by comparing the user’s reference line position to a gold standard reference position. The gold standard positions were determined through independent consensus of three experienced operators at UCSF. Exact indicates that the operator and reference positions coincided; good indicates that the operator position was within 2 pixels from the reference position; and out of range indicates that the operator position was greater than 2 pixels from the reference position. Two pixels corresponded to 0.34 mm, which was the short-term standard deviation among experienced operators.

We created training and evaluation collections for the radius and tibia from the double-length scans described in “Methods”. A horizontal mirror image of each scout view image was created to have an equal number of “left” and “right” limb images, for a total of 112 scout view images per anatomic site. We divided the scout view images into two groups depending on the degree of difficulty in positioning the reference line. Difficulty was assigned based on the presence or absence of a clear anatomic landmark, as assessed by an experienced UCSF operator. From each of these two groups, 49 images were randomly selected to create the training collections, and we used the remaining seven images to create the evaluation datasets. Both for training and evaluation modules, we created 15 collections of eight images randomly selected within their visibility group. Before accessing the reproducibility experiments, the six new operators had to complete all the datasets of the training module at least once, and had to succeed on at least three sets for the radius and three sets for the tibia. A set was considered successful when the sum of exact and good positions was seven out of eight positions.

Together with the training and evaluation software, we provided specific guidelines to identify the anatomic landmark and to position the reference line, as follow (Fig. 6). For the radius, we defined the anatomic landmark as the peak of the radiocarpal articular surface of the radius, which separates the radiopaque subchondral bone (bright signal) from the radiolucent joint space (dark signal) (Fig. 6 (a, b)). When the peak is clearly visible, the reference line must intersect the peak (Fig. 6 (a)), whereas when the peak is not visible, the reference line must intersect the mid-point of the joint surface (Fig. 5b). The hiker analogy in Fig. 6 (e–h) illustrates the concept. Similarly, for the tibia, the anatomic landmark is the peak of the edge of the tibial plafond, which separates the radiopaque subchondral bone (bright signal) from the radiolucent joint space (dark signal) (Fig. 6 (c, d)). When the peak is clearly visible, the reference line must intersect the center of the peak (Fig. 6 (c)), whereas when the peak is not visible, the reference line must intersect the flat plafond (Fig. 6 (d)).

Fig. 6

Guidelines for reference line positioning recommended for the training and evaluation software. In the radius, the reference line intersects (a) the peak when visible or (b) the mid-articular surface when the peak is not visible. Similarly, in the tibia, the reference line intersects (c) the peak when visible or (d) the flat plafond when the peak is not visible. In the radius, the hiker analogy easily explains the positioning of the reference line. The articular surface of the radius (e) can be considered as a hiking path (f). The hiker walks along the path (g) until he reaches the peak where he plants his flag, which coincides with the location where the reference line intersects the edge that represents the articular surface of the radius (h)