Introduction

Gestational diabetes mellitus (GDM) is a common metabolic disorder of pregnancy, affecting up to 5–14% of pregnant women1,2,3. GDM is associated with approximately 20% greater risk of type 2 diabetes in the decade following delivery4,5. Early onset of type 2 diabetes is a risk factor for diabetic complications such as renal disease, circulatory disease, and diabetic acidosis, which pose huge public health concern6,7. Guidelines from national diabetes associations in Canada and the US8,9 recommend screening for type 2 diabetes 6 weeks to 6 months after a pregnancy complicated by GDM. However, this recommendation has been challenging to implement as less than 50% of women with GDM are screened after pregnancy10,11. As the majority of women who have GDM do not develop type 2 diabetes in the short term, targeting high-risk individuals through a risk score may increase screening and interventions to prevent future diabetic complications.

A few models exist to predict the development of type 2 diabetes in women with GDM12,13. These models, however, focus solely on the development of type 2 diabetes, and do not include diabetic complications—arguably the more clinically relevant downstream outcomes accounting for the highest economic burden due to diabetes mellitus6. Furthermore, the small sample sizes of these studies (less than 500 individuals) limit the generalizability of their findings and model application. There are currently no existing models to predict the risk of type 2 diabetic complications after GDM. There is, therefore, a need for a simple model to help with identifying women with GDM who are at the most risk of developing type 2 diabetic complications, and improve targeted surveillance. Our objective was to develop a model using large administrative data to predict the risk of developing type 2 diabetic complications occurring within 10 years of delivery, among women with a history of GDM. We hypothesized that such easily measured factors can reasonably predict these adverse outcomes in women with GDM and be used to guide health policy planning on a population level.

Materials and methods

Ethics

The data used for this study were de-identified and ethics review and participant consent were waived by the institutional review board of the University of Montreal Hospital Centre. All methods were carried out in accordance with the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans.

Study population

We conducted a retrospective cohort study of women who had hospital-based deliveries in Quebec, Canada from April 1989 until March 2016 (cohort entry); women were then followed until 2018 to identify outcomes14. The cohort was constructed from the Maintenance and Use of Data for the Study of Hospital Clientele registry, which comprises > 99% of deliveries in Quebec.

Individuals aged 18 to 45 years who had GDM in at least one pregnancy were included, with the cohort entry point (t0) at the first GDM-affected pregnancy. GDM was defined as abnormal glucose tolerance of the mother, first identified during pregnancy, and identified using diagnostic codes from the 9th and 10th revisions of the International Classification of Diseases (ICD) (Table S1). These codes have been previously validated and adequately capture GDM diagnoses with specificity of > 90% and positive predictive values of > 80%15,16. There are some variations in approaches for identifying GDM in different centres, that is, one-step vs two-step approaches; however, both approaches are endorsed by Diabetes Canada17.

Women who died in their first affected pregnancy and women with pre-existing diabetes, or its complications were excluded (Fig. 1).

Figure 1
figure 1

Development of study cohort.

Full size image

Outcome

The primary outcome was hospitalization for type 2 diabetic complications within 10 years after delivery of the first pregnancy affected by GDM. Type 2 diabetic complications was defined as a diagnosis of type 2 diabetes with the development of one or more of the following complications: diabetic coma, acidosis, kidney, ophthalmic, neurological, circulatory, or other complications resulting from diabetes and identified using ICD-9 and 10 codes, previously validated in studies with specificity of 99% and positive predictive values of > 80% (Table S1).

The secondary outcome was type 2 diabetic complications occurring anytime (up to 29 years) after delivery of the first pregnancy affected by GDM.

Women were followed from cohort entry until any of the outcome occurrence, death, or the end of the study period (March 31, 2018).

Statistical analyses

We developed Cox proportional hazards regression models to predict type 2 diabetic complications, according to the previously outlined steps18,19, and report the process using the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) guidelines (Table S2)20.

Candidate predictors, variable selection and coding

We considered demographic, reproductive, and clinical factors known to be associated with an increased risk of type 2 diabetes as potential predictor variables5,21. These factors included maternal age, substance use, morbid obesity, socioeconomic deprivation (measured using a composite score of neighbourhood income, education, and employment)22, pregnancy factors such as parity, and multifetal pregnancy, and pregnancy complications such as hypertensive disorders of pregnancy (HDP), severe maternal morbidity (SMM)23, stillbirth, preterm delivery, low birth weight, and admission into neonatal intensive care unit (NICU) or adult intensive care unit (ICU). Candidate predictors were measured at the time of the index delivery (cohort entry).

Clinical variables that had low incidence were combined with other similar variables (e.g., previous obstetric complications such as SMM, stillbirth, preterm delivery, low birthweight, NICU admission, or neonatal death were combined). Previous history of obstetric complications was further combined with parity as follows: previous obstetric complication (among multiparous women), no previous obstetric complication (among multiparous women) and no previous obstetric complication (among primiparous women). When collinearity (r > 0.5) existed between variables, the most clinically relevant variable was selected.

Continuous candidate predictor variables (e.g., maternal age) were modelled using restricted cubic splines with three knot locations19. We assessed interaction terms and retained predictors that were statistically significant (alpha = 0.10)18. The final model variables were selected using Least Absolute Selection and Shrinkage Operator (LASSO) regression18.

Model performance and internal validation

Predictive performance of the model was assessed based on discriminatory, calibration, and risk stratification accuracy18. Discrimination was measured by the c-statistic, which is equivalent to the area under the receiver operating characteristic curve (AUROC)19. An AUROC of ≥ 0.7 was interpreted as good discrimination and 0.6 to < 0.7 modest, while 0.5 to 0.6 was considered poor and < 0.5 as not having any discriminative ability. Calibration performance was examined by plotting the mean observed events versus the mean predicted risks by decile. Calibration slopes were interpreted as good (slope > 0.7), poor (0.5 < slope ≤ 0.7) or non-informative (slope ≤ 0.5)24.

Using a risk classification table, we examined the ability of the model to stratify the population into low- and high-risk categories. We divided the population into four risk groups, with the highest calculated risk group corresponding to the overall incidence rate of the outcome in the study population25. Likelihood ratios (LR) were computed to assess the classification accuracy within each group26. For clinical use, positive LRs (LR+) of > 5 or > 10 were interpreted as moderate or good “rule-in” tests, respectively, whereas negative LRs (LR−) of < 0.2 and < 0.1 were considered as moderate or good rule-out tests, respectively24.

The model was assessed for internal validity using the bootstrap method with 200 iterations and the over-optimism (i.e., degree to which a model is overfit) was reported18.

Secondary analyses

Using the same final selected variables, we also developed a prediction model for type 2 diabetic complication up to 29 years after delivery and assessed the model discriminatory performance.

Sample size

We estimated our sample size based on the rule of thumb of 10–20 events per degree of freedom19, to avoid model overfitting. With a total of 1025 events during follow-up, we had sufficient sample size to consider up to 50 degrees of freedom for candidate predictors.

Analyses were conducted using R version 3.5.1 (The R Project for Statistical Computing).

Result

Cohort description

Among 1,229,263 women who delivered between 1989 and 2016 in Quebec, our final cohort included 90,143 (7.3%) individuals who met study inclusion criteria (Fig. 1). Among these individuals, the number of people admitted to hospital within 10 years of delivery with any diagnosis of type 2 diabetes, including both complicated and uncomplicated diabetes, was 1858 (2.06%) corresponding to an incidence rate of 25.0 (23.9–26.2) per 10,000 person-years. Within 10 years of delivery, there were 283 (0.3%) women who were hospitalized with type 2 diabetic complications [incidence rate 3.8 per 10,000 person-years (95% CI 3.4–4.3)], (Table 1). The median follow-up time was 6.2 years. The incidence of type 2 diabetic complications was higher for women who were younger than 25 years, obese, had substance use disorders, or were socioeconomically deprived at their first GDM-affected pregnancy, compared with those who did not have these characteristics (Table 1).

Table 1 Ten-year incidence of type 2 diabetic complications in women with gestational diabetes mellitus, according to characteristics at cohort entry.
Full size table

Diabetic ketoacidosis [0.9 per 10,000 person-years (95% CI 0.7–1.12)], kidney complications [0.8 per 10,000 person-years (95% CI 0.7–1.1)], and neurological complications [0.8 per 10,000 person-years (95% CI 0.7–1.1)] were the specific diabetic complications with the highest incidence rates during the 10-year period (Table 2).

Table 2 Incidence rates of hospitalization for type 2 diabetic complications among women with gestational diabetes mellitus, within 10 years of delivery, Quebec, 1989 to 2018. N = 90,143.
Full size table

Model performance for prediction of type 2 diabetic complications

The final models consisted of seven variables measured at the first GDM-affected pregnancy: maternal age, socioeconomic deprivation, substance use disorder, gestational age at delivery, SMM, previous pregnancy complications, and type of hypertensive disorder of pregnancy (Table 3; prediction model equation is provided in Table S4).

Table 3 Final prediction model with coefficients for prediction of type 2 diabetic complications among women with gestational diabetes mellitus, at 10 years of follow-up,
Full size table

The model had good discriminatory performance for the prediction of type 2 diabetic complications (AUROC 0.72, 95% CI 0.69–0.74) with a mean optimism of 0.0077 upon internal validation, indicating minimal overfitting (Fig. 2). Calibration performance was good with a slope of 0.99 and an intercept of − 0.31 (Fig. 3).

Figure 2
figure 2

Area under the receiver operating characteristic curve [AUROC (95% confidence interval)] showing model discriminatory performance for 10-year prediction of type 2 diabetic complications.

Full size image
Figure 3
figure 3

Calibration plots of observed versus predicted 10-year risks using deciles of predicted probability for type 2 diabetic complications.

Full size image

In the risk stratification table, the proportion of women with the primary outcome increased in each risk category, as the predicted risk of the outcome increased (Table 4). At the highest risk group (women with calculated risk of ≥ 0.03), 2.5% had the primary outcome whereas only 0.2% in the lowest risk group (women with calculated risk of < 0.006) had the outcome. The resulting LR+ in the highest risk group was 8.68 (95% CI 4.14–18.23) suggesting a moderate rule-in of type 2 diabetic complications in the top 0.3% of the women with the highest calculated predicted risk. However, the confidence interval was wide, due to low number of outcomes in this group (n = 7).

Table 4 Risk classification and stratification table for prediction of type 2 diabetic complications among women with gestational diabetes mellitus, at 10 years of follow-up.
Full size table

Secondary analyses (prediction of outcome up to 29 years after delivery)

The incidence of the type 2 diabetic complications up to 29 years of follow-up was 9.0 (8.5–9.6) per 10,000 person-years, respectively (Table S3). Discriminatory performance decreased slightly, with AUROC of 0.63 (0.61–0.64).

Discussion

In this study comprising 90,143 women with GDM, we developed Cox-proportional hazard models to predict the risk of developing type 2 diabetic complications occurring within 10 years of obstetric delivery. The incidence of hospitalization for diabetic complications in this timeframe was low, reflecting the young age of the cohort, and possibly good clinical care. However, our developed models consisting of readily measured variables at the time of birth showed moderate discriminative and calibration abilities for the prediction of type 2 diabetic complications after GDM pregnancy. Although external validation of the model is required, the AUROC of > 0.7 and LR+ of 8.68 in the highest risk category of ≥ 0.03 suggests that the model may eventually be used to reliably identify women at high risk of developing complicated diabetes.

Previous work has established a strong etiologic link between GDM and a future diagnosis of type 2 diabetes mellitus4,5. In a systematic review of 28 studies evaluating the risk of type 2 diabetes risk after GDM, the cumulative incidence of type 2 diabetes varied depending on time since delivery and ethnicity, and ranged from 2.6% to over 70% in included studies2. Our outcome of interest was hospitalization with type 2 diabetic complications within 10 years of delivery, reflecting more serious outcomes related to diabetes. The systematic review by Kim et al. also revealed that the cumulative incidence of type 2 diabetes increased over the first 5 years after delivery and plateaued after 10 years, supporting the chosen timeframe for our model2.

Previous models have focused on predicting type 2 diabetes after GDM, irrespective of the presence of diabetic complications12,13. A genetic risk score developed by Kwak et al. using data from 395 women with GDM reported a C statistic of 0.775 for predicting type 2 diabetes12. Another clinical model which included three types of lipid was developed by Lappas et al. in a cohort of 104 women and had C statistic ranging from 0.756–0.86513. These models did not include diabetic complications, which arguably have a more significant impact on women’s health, and would justify enhanced surveillance and early aggressive risk reduction therapy. Additionally, the small sample sizes used in these prior studies and the inclusion of variables not easily measured (e.g. genetic factors) lower the generalizability to routine clinical settings18.

Despite advances in our understanding of the association between pregnancy complications and long-term conditions, incorporation of pregnancy factors into existing risk scores has not improved net reclassification27,28,29. Possible reasons for the lack of success of previous models include their development in older populations and short length of follow-up, making them less suitable for women of reproductive age30. Developing prediction models among specific subpopulations, including postpartum women who have had a pregnancy complication, may circumvent some of these issues12,13,14. Despite the young age of individuals in our cohort, the predicted probability of type 2 diabetic complications at 10 years is substantial, indicating a robust model.

Late diagnosis of type 2 diabetes is often accompanied by development of diabetic complications, which is associated with healthcare expenditures three times greater than treatment of uncomplicated diabetes6. Clinical practice guidelines in Canada recommend performing a 75-g oral glucose tolerance test between 6 weeks and 6 months of delivery after GDM to identify type 2 diabetes early, with the goal of minimizing downstream sequelae11. Unfortunately, uptake of screening for type 2 diabetes after GDM remains low (< 50%), due to limited appreciation of risk, logistics of appointments and tests, and concerns of new mothers11. A shift in focus on screening strategies based on risk of more severe diabetes may allow for a greater mobilization of resources to a smaller segment of the population with GDM, particularly in lower resource settings. Our study shows that future type 2 diabetic complications can be predicted with moderate accuracy in women with GDM, using readily available clinical variables present in administrative datasets. The advantages of administrative data are that they are population-based records, and are larger than data from clinical settings, therefore our model can forecast future complications for health planning purposes31. The model requires external validation before ultimately being converted to an online risk calculator18,32,33 8–10 to aid in counselling and triaging of resources in postpartum women with GDM. If the postpartum patient has a calculated high risk, the approach and follow-up would differ (e.g., endocrinology referral, early OGTT, intensive lifestyle approach) from that of a lower-risk patient (e.g., glycosylated hemoglobin, community follow-up, usual postpartum lifestyle recommendations). This individualized approach will help with the study, planning, and implementation of targeted preventive strategies at the population level.

Our study has several strengths. First, we used a large population-based cohort of women from Quebec, which represents more than one-quarter of the Canadian population with a diverse multi-ethnic population. We had sufficient sample size for model development and minimal overfitting as shown with internal validation. Second, our data allowed us to test short-term as well as long-term risks of the outcomes. We chose 10 years for the primary prediction period as this represents the time when up to 20% of individuals develop type 2 diabetes2. Furthermore, we followed model development methods recommended by experts, including the use of LASSO for final variable selection, rather than forward and backward selections which are more prone to bias18. Finally, we ensured transparency in modelling and reporting by following the TRIPOD statement20.

As is the case with many large administrative datasets, a downfall is the reliance on ICD codes to define predictors. In particular, the dataset lacked information on clinical parameters such as blood pressure, body mass index, lipid levels or blood glucose measurements which are commonly used in type 2 diabetes clinical prediction models. These continuous variables could potentially improve model discrimination as they increase heterogeneity between women20. Other markers of severity of GDM such as insulin levels are predictive of type 2 diabetes but were lacking in our dataset2. In addition, we lacked information on ethnicity and family history, which are known determinants of diabetes and its complications2. Our prediction models, therefore, need to be improved and externally validated before they can be considered for use in a clinical setting. Nonetheless, our study identified easily measured variables that contribute to the prediction of long-term type 2 diabetic complications after GDM. Our models should therefore be used as a foundation to improve upon risk classification after pregnancy.

Finally, it is important to note that the diagnosis of GDM has evolved over time34. However, in un-shown analyses, restricting our population to deliveries prior to 2010, corresponding with the International Association of Diabetes and Pregnancy Study Groups (IADPSG) diagnostic recommendations publication34, did not change model performance.

Conclusion

Type 2 diabetic complications can be moderately predicted by our clinical model developed from administrative health data among individuals with a history of GDM. Future work is needed to improve the model performance and test the validity of the model in other datasets. After validation in external cohorts, women with GDM who are at high risk of developing type 2 diabetic complications can be earlier identified immediately after delivery. This will enable individualized planning and targeted approaches, such as endocrinology referral, early OGTT, and appropriate lifestyle modifications, to prevent future type 2 diabetic complications. Simple models to predict diabetic complications such as ours have the potential to help to develop a rational risk-based approach to screening and surveillance after GDM.