Article focus

• BACH is a new classification system for long bone osteomyelitis. This study serves to validate this classification in terms of interuser assessment and optimize specific variables that are included in the classification.

Key messages

• BACH can be applied with good accuracy and reproducibility to varied cases of long bone osteomyelitis.

• To classify the bone involvement of osteomyelitis, a clinical imaging utility should be used that allows 3D visualization of the pattern of infection.

• The absolute number of resistant susceptibility tests performed for a bacterial isolate has a high correlation with multidrug resistance as defined by the European Society of Clinical Microbiology and Infectious Diseases (ESCMID).

Strengths and limitations

• This study assesses the reproducibility of the BACH classification system with users who have a variety of clinical backgrounds and experience in the management of osteomyelitis.

• This study proposes a classification system that can be used in clinical practice and in subsequent prospective studies.

Introduction

Osteomyelitis is a complex disease that carries a significant burden for patients.¹ Specialist knowledge and treatment are essential for the successful management of patients with bone and joint infections.² A method of stratifying patients is necessary to ensure that referral to specialist teams occurs at an appropriate time during the management process. One tool to address this need is a clinically applicable method of classifying osteomyelitis.

With the aim of establishing a new classification system, we previously performed a systematic review that revealed 13 classification systems for osteomyelitis.³ After an in-depth analysis and using a multidisciplinary approach, we hypothesized that four key variables for classification would guide prognosis and requirement for referral to a specialist treatment centre. These were the following, from which the acronym BACH arises: 1) the bone involvement (B-variable); 2) the antimicrobial options (A-variable); 3) the coverage of the soft tissue (C-variable); and 4) the host status (H-variable).³ The bone involvement was divided into three options based on the presence of cavitary involvement (which includes a cortical, medullary, and non-segmental corticomedullary infection) (B1), segmental involvement (B2), or the concomitant involvement of a joint irrespective of the segmental or cavitary infection (B3). The antimicrobial options used the European Society of Microbiology and Infectious Diseases (ESCMID) criteria for stratifying isolates into multidrug-resistant (MDR), extensively drug-resistant (XDR), and pan-drug-resistant (PDR) isolates,⁴ and in case of a foreign body, whether an antimicrobial compound with activity against adhering (biofilm) bacteria was available. The coverage of the soft tissues was classified based on whether patients required plastic surgical expertise for skin closure after surgical excision of the infection. The host status stratified patients based on the presence or absence of comorbidities such as diabetes mellitus, vascular or immune compromise, or the presence of recurrent osteomyelitis following previous reconstructive surgery. The host status also accounted for patients who did not require an operation, were not fit enough to receive an anaesthetic, or declined surgery.

Based on the classification of these individual variables, the complexity of osteomyelitis can be determined. Each variable in BACH is stratified into either ‘uncomplicated’ or ‘complex’. In two variables, the antimicrobial options and the host status, osteomyelitis can also be stratified as having ‘few or no options available’. The overall complexity of the osteomyelitis is determined by classification of the most severely classified variable.

Several steps are mandatory to validate a classification system. First, we analyzed the applicability to a retrospective cohort. The promising results of this investigation have been reported elsewhere.⁵ The objectives of the present study are: 1) to evaluate BACH using an interuser assessment; 2) to refine these variables according to this assessment; and 3) to adapt BACH in view of the outcomes of these assessments. Based on these, version 3 of the BACH classification system will be presented.

Materials and Methods

The process of modification and optimization of the BACH classification system is illustrated in Figure 1. In a primary analysis, we measured the applicability of the classification system. In a secondary analysis, we assessed the performance of criteria that demonstrated suboptimal performance in the primary analysis.

Fig. 1

Flowchart presenting the development of the BACH classification of osteomyelitis. Here, the development of BACH version 2 to BACH version 3 is presented. The results of the retrospective assessment have been published previously.⁵

Results

Bone involvement

The interuser assessment of the bone involvement variable demonstrated accuracy of 77.0% (95% CI 71.5 to 82.5) among users. It had a ‘fair agreement’ among users with an Fκ of 0.479 (95% CI 0.475 to 0.483). The returned answer correlated with the reference answer in 72.3% of B1 cases, 95.0% of B2 cases, and 82.0% of B3 cases. Closer review of the cases that scored in the lowest quartile for accuracy demonstrated that the 2D nature of the online form used for the data collection was a key limitation in the accuracy of classification. This was therefore explored further in a secondary analysis. Other explanations for lower accuracy scores were due to unclear radiology reports and distorted anatomy.

The accuracy of classification in the 3D bone assessment using webPACS was 95.7% (95% CI 90.8 to 100.0), which was significantly higher when compared with the users’ scores in the 2D assessment (p = 0.028, paired t-test) and with the total population who completed the 2D assessment (p = 0.009, ANOVA with Dunnett’s post hoc test) (Fig. 2). This result was substantiated by an improvement in the Fκ in the 3D group to 0.899 (95% CI 0.869 to 0.929), demonstrating an almost perfect agreement between users. The sample population who took the 3D assessment were deemed to be representative of the total population who took the 2D assessment (p = 0.981, Kolmogrov–Smirnov test).

Fig. 2

Chart showing the results of the classification of the bone involvement variable using a 3D web-based picture archiving and communication system (webPACS) system, as performed in clinical practice, versus using a screenshot of the images alone (2D group). The sample population demonstrated a significant improvement when completing the 3D analysis compared with both the total interuser population who completed the 2D analysis (95.7% (95% confidence interval (CI) 90.8 to 100.0) vs 77.0% (95% CI 71.2 to 82.8); p = 0.009, analysis of variance with Dunnett’s post hoc test and the sample population when completing the 2D analysis (79.3% (95% CI 62.3 to 96.3); p = 0.028, Student’s t-test).

Antimicrobial options

The accuracy of classification of the antimicrobial options was 84.4% (95% CI 81.3 to 87.0) with a Fκ of 0.815 (95% CI 0.811 to 0.819). This indicated that there was an almost perfect agreement among users. Despite this, microorganisms with multiple resistant susceptibility tests did not return an accurate answer with the reference answer. The returned answer correlated with the reference answer in only 46.7% of the A2 group and 10.0% of the A3 group (Fig. 3). Therefore, an easier method than the proposed classification was deemed important.

Fig. 3

Results for the antimicrobial options variable in the interuser assessment. a) Bar chart showing the individual answers in each of the cases for the antimicrobial options variable, organized by the reference score. The dotted line is the mean score for the antimicrobial options category (92.2%). b) Heat map demonstrating the returned antimicrobial options score versus the reference antimicrobial options score.

The baseline characteristics for the cohorts are shown in Table I. For the 2013 to 2017 cohort, 72/222 (32.4%) cases had a culture negative osteomyelitis. There were 247 isolates, with 181 (73.3%) being ESCMID-classifiable and 66 (26.7%) being non-ESCMID-classifiable (Fig. 4a). The ESCMID-classifiable isolates comprised of Staphylococcus aureus (94/181, 51.9%), Enterobacteriaceae (57/181, 31.5%), Pseudomonas aeruginosa (15/181, 8.3%), and Enterococcus spp. (15/181, 8.3%) (Fig. 4b). A total of 40 (22.1%) were MDR, XDR, or PDR (for simplicity, termed MDR+) according to ESCMID classification. A similar number of susceptibility tests were performed for non-MDR+ and MDR+ (mean 16.7 vs 16.1).

Table I.

Summary of the baseline characteristics of the two cohorts included in the further analysis of the microbiology

Characteristic	2013 to 2017 cohort	2001 to 2004 cohort
Patients, n
Total	253	176
Excluded	31	10
Included	222	166
Age
Mean age, yrs (sd; range)	51.3 (16.0; 17 to 88)	45.0 (16.5; 12 to 90)
Sex, n (%)
Male	166 (74.8)	121 (72.9)
Female	56 (25.2)	45 (21.1)
Growth of microorganisms, n (%)
Culture-negative osteomyelitis	72 (32.4)	47 (28.3)
Culture-positive osteomyelitis	150 (67.6)	119 (71.7)

Fig. 4

Summary of investigations into the development of the antimicrobial options classification. a) Flowchart of patients in the 2013 to 2017 cohort and the isolates grown on microbiology culture. b) Chart showing the proportion of European Society of Clinical Microbiology and Infectious Diseases (ESCMID) isolates compared between the early (2001 to 2004) and late (2013 to 2017) cohort. c) Receiver operating characteristic (ROC) curve of number of resistant susceptibility tests for ESCMID classifiable isolates in the 2013 to 2017 cohort (area under the curve 98.1% (95% confidence interval (CI) 96.6 to 99.6)). d) ROC curve demonstrating using the percentage of sensitive susceptibility tests for the ESCMID classifiable isolates in the 2013 to 2017 cohort (area under the curve 98.8% (95% CI 98.1 to 100.0)). e) Scatterplot of each isolate from the 2013 to 2017 cohort with percentage resistant susceptibility tests against the total number of susceptibility tests performed. The dotted line represents four resistant tests. Crosses represent multidrug-resistant (MDR) isolates and coloured dots represent non-MDR isolates, according to ESCMID. This substantiates the results of the ROC analysis where 98.3% of MDR isolates are situated on or above the dotted line (four resistant tests).

The number of resistant susceptibility tests were counted for each ESCMID classifiable isolate. ROC curves were constructed using the number of resistant susceptibility tests and then the percentage of sensitive susceptibility test results, versus criteria positive according to MDR+ (Figs 4c and 4d). The area under the curve (AUC) was calculated for each of these ROC curves. Using the number of resistant susceptibility tests, the AUC was 98.1% (95% CI 96.6 to 99.6); using the percentage of sensitive susceptibility tests, the AUC was 98.8% (95% CI 98.1 to 100.0). The Youden index demonstrated that optimum cut-off was the presence of four or more resistant susceptibility tests (Youden index 0.85) and less than 78.5% sensitive susceptibility tests (Youden index 0.96). Using these thresholds, there was a false negative rate of 1.1% (2/181) and a false positive rate of 7.2% (13/181) in this cohort (Table II). This suggests that the presence of resistance to four or more antibiotic susceptibility tests or a total of less than 80% susceptibility tests sensitive is a good measure of multidrug resistance, as defined by the ESCMID criteria (Fig 4e).

Table II.

Summary of the classification of isolates in the 2013 to 2017 cohort when using the thresholds determined in the further assessment of microbiology. Isolates are classified as either European Society of Microbiology and Infectious Diseases (ESCMID) multidrug-resistant (MDR+; reference test) or ESCMID non-MDR+ and this is compared with using either four or more resistant susceptibility tests or less than 80% susceptibility tests sensitive as a measure of an isolate having either high resistance or low resistance

Resistance	Reference test
	ESCMID non-MDR+	ESCMID MDR+
Low resistance	128 (true negative)	2 (false negative)
High resistance	13 (false positive)	38 (true positive)

To validate this method of classifying multidrug-resistant isolates, a similar analysis was performed using an earlier cohort of patients who received surgery in the same centre between 2001 and 2004 (n = 176). This cohort was comparable in terms of baseline characteristics to the 2013 to 2017 cohort (Table I). Here, the number of resistant susceptibility tests gave an AUC of 91.0% (95% CI 85.2 to 96.8) with an optimum threshold of four or more. The percentage of sensitive susceptibility tests gave an AUC of 91.2% (95% CI 85.4 to 96.9) with an optimum threshold of less than 75.8%.

Discussion

The present study has demonstrated that the BACH classification system is able to be accurately applied to adults with long bone osteomyelitis by users who have a variety of backgrounds and little previous experience of the system. It has identified the need for optimization of the antimicrobial options variable that allows stratification of resistant isolates by non-specialists.

The definition of the bone involvement variable has remained consistent since the first version of BACH. The investigations performed as part of this study demonstrate the importance of using software that can give a 3D representation of the bone involvement, which improves the agreement and accuracy of the classification. Despite this improvement, the combination of clinical information and imaging must be used to make the classification. By stratifying patients into those who have a nonunion (segmental involvement) or adjacent joint involvement early in the disease process, specialist reconstructive options may be offered at an early stage in management. This is supported by previous work that has demonstrated the difficulty in the management of infected nonunion, even in specialist centres.^11,12

The initial antimicrobial options variable was based upon the ESCMID criteria for the definitions of MDR, XDR, and PDR.⁴ The classification of the multidrug-resistant isolates were classified with a disappointing accuracy of 46.7% for the A2 group (MDR isolates) and 10.0% in the A3 group (XDR and PDR isolates). Despite these criteria being detailed, they are not comprehensive and account for only a few species (i.e. Enterobacteriaceae, Enterococcus spp., Pseudomonas aeruginosa, Acinetobacter spp., and Staphylococcus aureus). In an attempt to ensure that all isolates were classifiable, the version of the classification system used in the interuser assessment included a line that stated that the A2 group should include “bacteria that are not part of the ESCMID criteria with resistance to appropriate antimicrobial agent”. Early identification of multidrug-resistant isolates is essential, as delay in appropriate treatment is thought to be the main cause for mortality, rather than the virulence of the bacteria per se.^13,14 Therefore, an alternative and simple method for the identification of these isolates by a non-specialist is important, especially in cases where access to a specialist infectious diseases physician is not available. Using the number of resistant or sensitive susceptibility tests means that the classification is easily applicable by a non-specialist. The limitation to this method is the variation between laboratories for the susceptibility testing that is performed for each isolate. However, guidelines for susceptibility testing, such as the European Committee on Antimicrobial Susceptibility (EUCAST) guidelines¹⁵ and British Society of Antimicrobial Chemotherapy recommendations, are increasingly being applied.¹⁶ This ensures that relevant and appropriate susceptibilities are being tested as part of a standardized operating procedure.¹⁷ In any case, there is evidence to suggest the importance of inclusion of a specialist infectious disease physician or clinical microbiologist in the care of bone and joint infection.¹⁸

The coverage of the soft tissues is based on the requirement on whether a wound can be closed directly or whether specialist soft-tissue coverage is required. The accuracy of this variable was 92.5% in the interuser assessment. This variable does not offer advice on when specialist expertise is required. However, this system may be beneficial because it can account for the individual expertise of the surgeon or treating department. The benefit of including a plastic surgeon for limb reconstruction has been reported in the setting of lower limb trauma¹⁹ and also applies to bone infection.^18,20

The host status can be complex to classify with multiple scoring systems available for general use^21,22 and for use specifically in bone and joint infection.²³²⁵ The distinction between defining how well controlled a comorbidity is and the severity of disease is important in outlining the prognosis to patients. Host status in osteomyelitis can fall into a number of categories: patients without health compromise who wish to have definitive treatment; patients with reversible or irreversible health compromise who wish to have treatment; and patients who decline treatment or who are unfit for definitive treatment. These groups all have differing prognosis. Depending on the progression of their comorbidity, age, and consent for surgery, they may then progress to either H1, H2, or H3 at a future timepoint.

The principle aim of BACH is to guide the need for complex elements of management in long bone osteomyelitis. Version 3 of BACH is shown in Figure 5, with the user key presented as Supplementary Material. Each variable is able to determine the complexity of management and requirement for multidisciplinary input. The overall complexity is governed by the outcome of the most severely classified variable. ‘Uncomplicated’ osteomyelitis may be managed at a non-specialist centre, but either ‘complex’ or ‘limited options available’ osteomyelitis should be referred early to a centre with appropriate expertise. In addition, BACH may also be able to offer prognosis to patients in terms of patient-reported outcomes, surgical complications, and medical complications. However, this has not been assessed by this study. In view of this, a prospective evaluation is planned to assess the classification system for both prognosis and management of bone and joint infection, with potential further refinement in the future.

Fig. 5

The BACH classification system for long bone osteomyelitis. The four key variables are headers with the corresponding criteria for making the classification in each. The green band denotes ‘uncomplicated’ osteomyelitis that can be managed at a non-specialist centre, the amber band denotes ‘complex’ osteomyelitis that should be managed at a centre with specialist expertise, and the red band is ‘limited options available’. The overall complexity of the osteomyelitis is determined by the band of the most severely classified variable.

There were limitations to this study. First, we wanted to ensure that the user key was read in its entirety prior to classification of the cases. In view of this, the password to gain access to the online form was placed at the foot of the user key document. Although this ensured that the document was opened, it did not ensure that it would be studied thoroughly. This is a pragmatic approach to using the classification system, which may reflect uptake in real clinical practice. To ensure that users understood the classification system, users could have been asked to complete a number of practice cases with feedback on performance prior to the full assessment. Second, cases included in the primary interuser assessment were selected to represent the frequency of clinical cases from the retrospective cohort.⁵ This resulted in an unequal representation of each variable (for example, there were 13 cases of B1, two cases of B2, and five cases of B3) but allows the resultant accuracy and Fκ scores to indicate the application of BACH in real clinical practice. Third, the alternative method for classification of MDR isolates by non-specialist clinicians was not reassessed using a second interuser assessment. We hypothesize that counting both the total number of performed susceptibility tests and number of sensitive/resistant test results would be easily applicable by clinicians. In addition, the method depends on relevant and appropriate susceptibility testing of a meaningful number of antibiotics. However, this will be re-evaluated for accuracy and agreement between users in the future. Finally, the improvement observed in the secondary interuser assessment of the bone involvement variable could be attributable the classification of only one variable and the inclusion of fewer cases. In the primary assessment, a total of 80 classifications were required by each user (20 cases each with four variables), which compares with ten classifications (ten cases with only one variable) in the secondary assessment. The higher demand in the number of classifications for the primary assessment could have been overbearing for clinicians not experienced with the classification system, resulting in lower scores.

In summary, BACH is a new classification for bone and joint infection. This study has optimized the variables that are included and has demonstrated the applicability of the classification system by an interuser assessment. The results of this study lead the way for the completion of a prospective assessment of BACH in cases of long bone osteomyelitis to assess the management and prognosis of this complex condition.