figure a

Introduction

Therapies targeting pancreatic islet autoimmunity are being tested for their ability to preserve insulin-secreting beta cells and modify the natural history of type 1 diabetes after diagnosis [1]. The widely accepted measure of their efficacy is the average plasma C-peptide concentration over 2 h following a mixed-meal test (CPAVE) [2]. However, the measurement of CPAVE requires ingestion of a liquid meal and collection of at least seven venous blood samples. A more convenient measure would streamline the assessment of beta cell function, particularly when disease-modifying therapies enter routine clinical practice.

In clinical trials, the biological agents rituximab, teplizumab and abatacept have been shown to improve beta cell function for at least 1 year in people with recently diagnosed type 1 diabetes [3,4,5]. Improved CPAVE in these trials was associated with a decrease in insulin requirement and in HbA1c, suggesting that these routine clinical measures may be useful surrogates of beta cell function. Indeed, insulin dosage and HbA1c are used to calculate insulin-dose-adjusted HbA1c (IDAA1c), which identifies type 1 diabetes in children with residual beta cell function [6, 7]. Other studies in children and adults at high risk of developing type 1 diabetes have shown that HbA1c, age and BMI correlate with the C-peptide response to oral glucose [8,9,10], again suggesting that these routine measures could serve as useful surrogates of beta cell function in the clinic.

We aimed to develop a simple and reliable model that could accurately estimate CPAVE, based on a combination of routine clinical measures and fasting C-peptide (FCP) plasma levels. Data from eight trials involving people with recently diagnosed type 1 diabetes [3, 4, 11,12,13,14,15,16] were used to build predictive models to approximate CPAVE and derive estimates of variability for use in future trial design.

Methods

Data collection

Study participants gave informed consent if adult and assent if aged under 18 years. All studies were approved by the responsible ethics committee and were carried out in accordance with the Declaration of Helsinki as revised in 2008. Clinical and biochemical data from the TrialNet TN-02, TN-05, TN-08, TN-09 and TN-14 clinical trials (Table 1) [3, 11,12,13] were extracted from the TrialNet data repository in April 2014. In all of these trials, predominantly white participants were assessed at 0, 3, 6 and 12 months after enrolment and, for TN-08 and TN-14, also at 9 months. Additional data from the Immune Tolerance Network (ITN)-27, ITN-28 and ITN-45 trials [14,15,16] were extracted in February 2016 and comprised clinical and biochemical measures obtained at the 0, 6 and 12 month time points. Data from Australian adults participating in an ongoing clinical trial of empagliflozin in recently diagnosed type 1 diabetes (Australian New Zealand Clinical Trials Registry [www.anzctr.org.au] registration no. ACTRN12617000016336) were obtained in April 2018. Plasma C-peptide concentrations in TrialNet and ITN trials were determined to sensitivities of 0.017 and 0.05 nmol/l with TOSOH 2000 and TOSOH 1800 autoanalysers (TOSOH, South San Francisco, CA, USA), respectively. In Australia, C-peptide and HbA1c were measured by Melbourne Health Pathology (Parkville, VIC, Australia) using ARCHITECT (Abbott, Wiesbaden, Germany) and Ultra2 (Primus Diagnostics, Kansas City, MO, USA) kits, respectively.

Table 1 Baseline characteristics of participants aged less than 21 years according to trial and treatment group
Full size table

After receipt of the archived data, missing weight, height, insulin dose and HbA1c values were imputed where possible by filling backwards or forwards from the nearest time point (if within 1 month) or by averaging values either side of the missing value. Undetectable C-peptide concentrations observed in TrialNet and ITN datasets were assigned values of half of the lower limit of detection. Because daily insulin requirements are ~20% lower with insulin pump therapy than with injection therapy [17], the daily insulin dose of TrialNet participants who reported using insulin pumps was multiplied by 1.25.

Analyses

Correlation and receiver operating characteristic (ROC) curve analyses were performed using Prism software (v6.0g for Mac; GraphPad, San Diego, CA, USA). Data modelling was performed using R software v3.3.2 (www.r-project.org). Half of the participants aged <21 years at baseline were randomly assigned to train the Linear Mixed Models to determine the estimated CPAVE (CPEST); a validation dataset, comprising data from the remaining participants aged <21 years at baseline, was used to identify the best models. CPAVE was loge-transformed after adding 1 [18] and eight covariates were chosen for inclusion in the prediction model: age, sex, BMI, diabetes duration, insulin dose per kg body weight, FCP, fasting plasma glucose (FPG) and HbA1c. Participant identification no. was added as a random effect to account for the repeated measurements from the same individual. The ‘dredge’ function in the MuMIn library (v1.15.6; www.r-project.org) was used to construct 256 models from all possible combinations of variables and these models were ranked by Akaike’s information criterion (AIC), corrected for a finite sample size. To validate the rankings of the models, the ‘lmer’ function in the lme4 library (v1.1-13; www.r-project.org) was used to rebuild the models in the validation dataset based on the relevant inputs, thereby enabling their AIC values to be determined. To compare treatment arms of clinical trials, mixed models were fitted using ‘lmer’ with a random intercept per participant and adjusted for sex, age and baseline loge (CPAVE + 1) or loge (CPEST + 1). The lmer-Test package was used to calculate p values based on F statistics for treatment comparisons.

Power calculations for the comparison of two groups with equal variance were performed using placebo-group data from the validation dataset and Stata (v14.2) software (StataCorp, College Station, TX, USA). They were based on the mean and SD of the loge (CPAVE + 1) values and a conservative approximation of the SD of loge (CPEST + 1) values, calculated by combining the variance of loge (CPAVE + 1) values with an estimated variance of the difference between the loge (CPAVE + 1) and loge (CPEST + 1) values according to the following formula:

$$ {\mathrm{SD}}_{\mathrm{APPROX}}=\surd \left({\sigma^2}_{\log_e\left({\mathrm{CP}}_{\mathrm{AVE}}+1\right)}+{\sigma^2}_{\log_e\left({\mathrm{CP}}_{\mathrm{AVE}}+1\right)-{\log}_e\left({\mathrm{CP}}_{\mathrm{EST}}+1\right)}\right) $$

A standard trial design that assumed a treatment effect of 50% increase in loge (CPAVE + 1) at 12 months, two-tailed α = 0.05, power = 0.8 and 2:1 (active: placebo) randomisation was used to estimate the required number of participants.

Results

Developing and validating equations to estimate beta cell function

The baseline characteristics of participants whose data were used to develop the models are presented according to clinical trial and treatment assignment in Table 1. Initially, we used data from participants aged <21 years to fit and test linear models for three reasons: (1) this age group accounts for over 75% of classic type 1 diabetes presentations [19]; (2) beta cell function declines more slowly in older people [20, 21] and (3) preservation of beta cell function is more characteristic of younger participants in trials of biological agents [22]. Half of the participants were randomly assigned to train linear models to estimate CPAVE using one or more of the eight input variables of age, sex, BMI, diabetes duration, insulin dose/kg body weight, FCP, FPG and HbA1c. Based on one to eight predictor variables, the AIC value was used to identify the most accurate models, hereafter referred to as M1–M8. The coefficients and associated standard errors of the variables included in the eight models are provided in electronic supplementary material (ESM) Table 1. Data from the remaining half of the participants were used to validate the models. M6, which is based on BMI, diabetes duration, insulin dose/kg body weight, FCP, FPG and HbA1c, was chosen for subsequent testing because its AIC value was lowest in the validation dataset (Fig. 1). Within the validation dataset, M6-modelled CPAVE (hereafter called CPEST) and observed CPAVE were strongly correlated (r2 = 0.816, p < 0.001). The equation for M6 is:

Fig. 1
figure 1

Performance characteristics of eight models to estimate loge (CPAVE+1) from single time point data. The components of each model are indicated below the graph of the AIC value against the number of model variables in the context of the validation dataset. M6 was used to calculate CPEST values. T1D, type 1 diabetes

Full size image

loge (CPEST + 1) = 0.317 + 0.00956 × BMI − 0.000159 × duration + 0.710 × FCP − 0.0117 × FPG − 0.0186 × HbA1c − 0.0665 × insulin (ESM Methods), where BMI is in kg/m2, duration is in days, FCP is in nmol/l, FPG is in mmol/l, HbA1c is in % and insulin is in U/kg.

Because M6 did not require age as an input, we determined whether it might also be accurate in the 150 trial participants aged >21 years whose data were not included in either the training dataset or the validation dataset (baseline characteristics are presented in ESM Table 2). Correlation analysis of data from 554 meal tests performed during the first trial year again demonstrated a strong correlation between CPAVE and CPEST (r2 = 0.729, p < 0.001). Strong agreement between CPAVE and CPEST (r2 = 0.869, p < 0.001) was also observed when M6 was applied to data from 31 meal tests from ten participants (three female sex, seven male sex, aged 18–37 years at diagnosis; ESM Table 3) in an ongoing Australian trial of empagliflozin in recently diagnosed type 1 diabetes.

Applying CPEST to clinical practice

ROC curve analysis of the validation dataset was performed to determine how accurately CPEST identified significant loss of beta cell function at 3, 6 and 12 months after clinical trial entry, defined as a decrease of 7.5% or more of the baseline CPAVE [20, 23]. The ROC curves (Fig. 2) show areas under the curve ranging from 0.86 (95% CI 0.81, 0.91) to 0.91 (95% CI 0.87, 0.95). When tested for the ability to identify significant loss of beta cell function at 3, 6 and 12 months compared with baseline, CPEST furnished an area under the ROC (AUROC) of 0.89 (95% CI 0.87, 0.92). The corresponding AUROC for trial participants aged >21 years was 0.88 (95% CI 0.84, 0.91). We also determined how accurately IDAA1c, an extant clinical measure of beta cell function [6], identified trial participants who had lost significant beta cell function. The AUROC of the ratio of baseline to 3, 6 and 12 month IDAA1c was markedly lower at 0.72 (95% CI 0.68, 0.76).

Fig. 2
figure 2

(a–c) ROC curve analysis was used to determine how accurately CPEST identified participants whose CPAVE decreased by more than 7.5% of the baseline value at 3 (a), 6 (b) and 12 (c) months after clinical trial entry. (d) ROC curve analysis for the participants with a 7.5% decrease of CPAVE at 3, 6 and 12 months. The AUROC (95% CI) was 0.86 (0.81, 0.91), 0.88 (0.84, 0.92), 0.91 (0.87, 0.95) and 0.89 (0.87, 0.92) for (a–d), respectively. These analyses used the validation dataset, which was derived from half of the participant population and was fully independent of the dataset used to develop the CPEST model

Full size image

Implications for clinical trial design

The potential suitability of CPEST as an alternative primary outcome measure for clinical trials was then assessed. All available data from participants (children and adults) in the TN-05 rituximab [4], TN-09 abatacept [3] and ITN-27 teplizumab [15] trials were analysed. The major conclusion from each trial, that the active therapy preserved beta cell function over the first year after diagnosis, held regardless of whether CPAVE or CPEST was used to compare treatment groups (Fig. 3). We also applied CPEST to data from the other five negative trials and observed similar treatment effects (ESM Fig. 1).

Fig. 3
figure 3

Outcomes of TN-05 (rituximab), TN-09 (abatacept) and ITN-27 (teplizumab) trials according to CPAVE and CPEST. Outcomes for participants receiving active (black circles) and placebo (white squares) treatment in TN-05 (a, b; 51 active and 29 placebo participants), TN-09 (c, d; 74 active and 31 placebo participants) and ITN-27 (e, f; 54 active and 25 placebo participants) are shown as means±SEM. CPAVE measured by meal test is presented in (a, c, e) and CPEST measured from single time point measures is presented in (b, d, f). Differences between treatment groups across all time points after baseline were determined using a mixed model that corrects for baseline CPAVE, age and sex (a, c, e) or CPEST, age and sex (b, d, f). *p<0.05, **p<0.01 and ***p<0.001 for differences between treatment groups

Full size image

To examine implications for clinical trial design, the SD of loge (CPEST + 1) values was conservatively estimated by combining the variance of loge (CPAVE + 1) values with the variance of the difference between the loge (CPAVE + 1) and loge (CPEST + 1) values, as outlined in Methods. Using 12 month placebo-group data from the validation dataset from participants aged <21 years, the mean±SD of loge (CPAVE + 1) and loge (CPEST + 1) was 0.320 ± 0.218 and 0.331 ± 0.166, respectively. The variance of the difference between these values was 0.0087, resulting in an estimated SD for loge (CPEST + 1) of 0.237. When the loge (CPAVE + 1) mean±SD and the estimated SD for loge (CPEST + 1) were applied to a standard trial design that assumed a treatment effect of 50% increase in loge (CPAVE + 1) at 12 months (i.e. Δ = 0.160), two-tailed α = 0.05 and 2:1 (active: placebo) randomisation, the number of participants required to achieve 80% power was 69 for loge (CPAVE + 1) and 81 for loge (CPEST + 1) (i.e. 17% higher). When the validation data were combined with placebo-group data from adult participants aged >21 years (combined dataset), the mean±SD for loge (CPAVE + 1) and loge (CPEST + 1) increased to 0.370 ± 0.227 and 0.377 ± 0.174, respectively, and the estimated SD for loge (CPEST + 1) increased to 0.247, yielding Δ = 0.185 and a requirement for 57 participants if loge (CPAVE + 1) was the primary outcome measure and 66 (i.e. 16% higher), if loge (CPEST + 1) was used. If geometric means for loge (CPAVE + 1) were instead used as the basis for power calculations, the use of loge (CPEST + 1) as the primary outcome measure required 17% and 13% more participants, respectively, in the context of the validation dataset and combined dataset.

Discussion

Using six, single time point measures, we describe a model (CPEST) for estimating CPAVE that reliably identifies loss of beta cell function in children and adults with recently diagnosed type 1 diabetes. The accuracy of CPEST was comparable with that of CPAVE and superior to that of IDAA1c. When applied to data from the active and placebo treatment arms of three trials of immune modulators that preserved beta cell function, CPEST identified differences in beta cell function over the first year that were similar to those identified using CPAVE. These findings reinforce the strong correlation between FCP and CPAVE in people with recently diagnosed type 1 diabetes [8, 20] and suggest that the relatively simple biochemical measurements of HbA1c, FCP and FPG combined with BMI, insulin dose and disease duration may be sufficient to assess an individual’s response to disease-modifying therapy.

CPEST did not require age as an input despite the known strong association of age with beta cell function and with its rate of decline following diagnosis [20, 24]. Whereas age was an input for M4, it was not used in the optimal models that incorporated five or six inputs and which instead used HbA1c, BMI and insulin dose. Clearly these other clinical measures accounted for the effect of age on beta cell function. During model development with the training dataset, using age as an input did not always increase accuracy. For example, of the eight models based on four inputs that were more accurate than M3, only two (including M4) included age as an input. Similarly, of the six models based on five inputs that were more accurate than M4, only three used age.

Power calculations, based on a conservative estimate of the SD of loge (CPEST + 1), indicated that sample size would need to increase by up to 17% if CPEST was used as a primary outcome measure. However, because the SD of loge (CPEST + 1) was lower than the SD of loge (CPAVE + 1), it is possible that modelled values are inherently less variable and therefore more accurate measures of beta cell function. This might be explained by the fact that a single fasting test eliminates variation attributable to meal ingestion and multiple sampling. Alternatively, incorporation of FPG into the model may account for day-to-day variation in insulin sensitivity [25], which in turn could alter beta cell function [26] and increase CPAVE variability between meal tests. It will be important to establish the power of CPEST relative to CPAVE in future trials because CPEST is simpler and much more convenient. Even if subsequent testing shows that using CPEST would require a modest increase in sample size, this would need to be balanced against its potential to improve participant recruitment and satisfaction. CPEST also enables more frequent assessment of beta cell function during a trial and obviates the need to admit participants to a clinical trials unit for a meal test, thereby reducing trial costs.

In the clinical setting, the ability of CPEST to identify individuals who lose beta cell function commends it for routine use in monitoring an individual’s beta cell function over time and determining their response to disease-modifying therapy. CPEST is also likely to be useful for larger Phase 3 and 4 trials, and for studies of type 1 diabetes cohorts that aim to identify factors associated with disease progression and the relationship between C-peptide preservation and long-term complications such as hypoglycaemia unawareness and rates of micro- and macrovascular disease.

IDAA1c is a measure of beta cell function that has gained acceptance in clinical practice because it reliably identifies children with type 1 diabetes who have substantial beta cell reserve, defined as a peak plasma C-peptide response to a mixed meal of greater than 0.3 nmol/l (0.9 ng/ml) [6, 7]. However, our analysis shows that IDAA1c has relatively poor accuracy for diagnosing significant loss of beta cell function, in accord with an earlier study which showed that IDAA1c was not a reliable surrogate of CPAVE during the first 4 years following the diagnosis of type 1 diabetes [21]. Therefore, compared with modelled CPEST, IDAA1c is not suitable for assessing disease-modifying therapy.

Last, several caveats are in order. Our cohort comprised participants who were mostly of European descent and had type 1 diabetes for no more than 100 days when CPAVE was first measured. Therefore, the accuracy of our model in other ethnic groups or in those with longer-standing type 1 diabetes is uncertain. In addition, despite the model’s accuracy in the two adult populations tested, caution should be exercised in applying it to other adult populations until its accuracy is further confirmed. Finally, because FCP and HbA1c were measured at only three laboratories, the generalisability of CPEST should be determined in the context of other laboratories and assay platforms.

In summary, CPEST modelled from six routine clinical and biochemical variables is an accurate measure of beta cell function in children and young adults with recently diagnosed type 1 diabetes. The simplicity and convenience of CPEST combined with its superior accuracy when compared with IDAA1c argues for its implementation and further validation in assessing beta cell function in clinical trials and during the course of routine clinical care.