Original Article
Two Long-Term Mortality Risk Models For Coronary Artery Bypass Graft Surgery Produced in American Populations Validated in an Australian Population

https://doi.org/10.1016/j.hlc.2017.02.008Get rights and content

Background

Short-term mortality prediction models have an important role in current cardiac surgical practice. There has been much less attention paid to prediction of long-term outcomes which are probably an equal marker both of surgeon performance and appropriateness of surgical treatment. The aim of this study was to assess the performance of the New York State Cardiac Surgery Reporting System (NYSCSRS) risk model and the Northern New England Cardiovascular Disease Study Group (NNECDSG) risk model on the Australian and New Zealand Society of Cardiac and Thoracic Surgeons (ANZSCTS) patient database.

Methods

The NYSCSRS and the NNECDSG risk models were applied to all patients undergoing isolated coronary artery bypass graft (CABG) surgery that had complete data, were over the age of 18 and had a body mass index between 12 and 78 kg/m2. Predicted mortality was calculated using the published risk model formulae and compared with observed mortality, obtained via linkage with the National Death Index, at four time-points (one, three, five and seven years following surgery). Model discrimination and model calibration were tested at all four time points by determining the C-statistics for receiver operator characteristic (ROC) curves, and studying the Hosmer-Lemeshow chi-square tests, respectively.

Results

The NYSCSRS and NNECDSG risk models were applied to 34,961 and 34,998 patients, respectively. The NYSCSRS risk model over-predicted mortality by between 130% and 216% at all four time-points while the NNECDSG risk model under-predicted mortality at one year by 4.3% but over-predicted mortality at three, five and seven years by between 42.5% and 145.7%. The C-statistics obtained fell between 0.779 and 0.741 for the NYSCSRS risk model and between 0.785 and 0.752 for the NNECDSG risk model at all four time-points. Hosmer-Lemeshow chi-square tests returned p-values <0.001 at all four time-points for both risk models.

Conclusion

The NYSCSRS and NNECDSG risk models do not accurately predict long-term mortality following isolated CABG surgery in the ANZSCTS patient population. The use of either of these risk models is not appropriate in Australia.

Introduction

There is an important role for accurate risk prediction models for long-term mortality in current cardiac surgical practice. Risk models aim to identify and weigh patient demographic characteristics and risk factors that influence specific patient outcomes. In cardiac surgery, risk models can be used in the performance profiling of surgeons, hospitals or countries where they can retrospectively adjust for case-mix differences between groups. They can also be used prospectively for patient counselling, informed consent and the identification of low- or high-risk patient subgroups that may require closer follow-up. Most of the interest in such models has centred on those that predict short-term outcomes.

Long-term mortality risk models are a relatively recent addition to cardiac surgery. Historically, short-term mortality risk factors were studied and then developed into short-term mortality risk models [1], [2], [3], [4], [5], [6], [7], [8], [9]. These were subsequently implemented into clinical decision-making and provider profiling worldwide. More recently, however, there has been a greater emphasis on long-term mortality risk factors. These are vastly published throughout the literature and have subsequently been developed into long-term mortality risk models and prediction tools [10], [11], [12]. Two of the currently available long-term risk models for coronary artery bypass graft (CABG) surgery, developed in the United States of America, include:

  • New York State’s Cardiac Surgery Reporting System (NYSCSRS) risk model [10]

  • Northern New England Cardiovascular Disease Study Group (NNECDSG) risk model [12]

Risk scoring systems are most applicable when the preoperative patient characteristics and treatment profiles are comparable with those on which the system was originated. For this reason any risk scoring system can only be used reliably when its validity has been tested in the local patient population [13]. Therefore, without validation within an Australian dataset, the long-term risk models listed above have limited use for cardiac surgery within Australia.

The NYSCSRS risk model is reported to predict mortality up to seven years following surgery whilst the NNECDSG risk model is reported to predict mortality up to eight years following surgery. The formulae from these two risk-models was applied within the Australian and New Zealand Society of Cardiac and Thoracic Surgeons (ANZSCTS) database to study its ability to stratify risk within this population.

Section snippets

Database

The ANZSCTS National Cardiac Surgery Database Program has prospectively collected information on adult patients undergoing cardiac surgery in Australia since 2001. Currently, 28 hospitals across Australia are contributing data to the registry. The registry collects 287 preoperative, intraoperative and postoperative variables using internationally standardised data definitions [7], [14]. The data collection and its audit methods were discussed previously [1], [7]. The registry was used to

NYSCSRS x ANZSCTS

The two populations (ANZSCTS and NYSCSRS) were very similar. The only significant difference was in the prevalence of left main coronary artery stenosis of greater than 50%. The demographic data and the frequency of risk factors are shown in Table 2.

Of the 34,961 patients studied, there were 792, 1402, 2015 and 2458 deaths observed at one, three, five and seven years following surgery, respectively. These figures correlate to observed mortality rates of 2.3%, 4.0%, 5.8% and 7.0%, respectively.

Discussion

Long-term mortality risk prediction tools for CABG surgery can play an important role in cardiac surgery practice. They can be used to meaningfully compare the outcomes between surgeons, hospitals or even countries by adjusting for the variable case-mix that is typically present. Furthermore, they can be used in the decision-making process for surgeons who are deciding whether or not to undertake CABG surgery on a particular patient. In order for surgeons to use an accurate risk prediction

Conclusions

The NYSCSRS and NNECDSG long-term mortality risk models for CABG surgery could not be validated within the ANZSCTS population.

The NYSCSRS risk model showed poor statistical precision at all time points (one, three, five and seven years following surgery), however, showed good statistical accuracy with C-statistics between 0.741 and 0.779 across the four time points. Therefore, the NYSCSRS risk model can stratify risk within the ANZSCTS population quite well, however, if the risk model were to

Acknowledgements

ANZSCTS Data Management Centre, CCRE, Monash University: Prof Chris Reid, Dr Lavinia Tran, Ms Dhenisha Dahya, Mrs Angela Brennan.

ANZSCTS Database Program Steering Committee: Mr Gil Shardey (Chair), Mr Peter Skillington, Mr Julian Smith, Mr Andrew Newcomb, Mr Siven Seevanayagam, Mr Bo Zhang, Mr Hugh Wolfenden, Mr Adrian Pick, Mr Jurgen Passage, A/Prof Rob Baker, Prof Chris Reid, Dr Lavinia Tran, Mrs Angela Brennan, Mr Andrew Clarke.

The Australian and New Zealand Society of Cardiac and Thoracic

References (18)

There are more references available in the full text version of this article.
View full text