Skip to main content
Log in

The impact of missing data on clinical trials: a re-analysis of a placebo controlled trial of Hypericum perforatum (St Johns wort) and sertraline in major depressive disorder

  • Original Investigation
  • Published:
Psychopharmacology Aims and scope Submit manuscript

Abstract

Rationale and objective

Hypericum perforatum (St John's wort) is used to treat depression, but the effectiveness has not been established. Recent guidelines described the analysis of clinical trials with missing data, inspiring the reanalysis of this trial using proper missing data methods. The objective was to determine whether hypericum was superior to placebo in treating major depression.

Methods

A placebo-controlled, randomized clinical trial was conducted for 8 weeks to determine the effectiveness of hypericum or sertraline in reducing depression, measured using the Hamilton depression scale. We performed sensitivity analyses under different assumptions about the missing data process.

Results

Three hundred forty participants were randomized, with 28 % lost to follow-up. The missing data mechanism was not missing completely at random. Under missing at random assumptions, some sensitivity analyses found no difference between either treatment arm and placebo, while some sensitivity analyses found a significant difference from baseline to week 8 between sertraline and placebo (−1.28, 95 % credible interval [−2.48; −0.08]), but not between hypericum and placebo (0.56, [−0.64;1.76]). The results were similar when the missing data process was assumed to be missing not at random.

Conclusions

There is no difference between hypericum and placebo, regardless of the assumption about the missing data process. There is a significant difference between sertraline and placebo with some statistical methods used. It is important to conduct an analysis that takes account of missing data using valid statistically principled methods. The assumptions about the missing data process could influence the results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Carpenter J, Kenward MG (2007) Missing data in randomised controlled trials—a practical guide

  • Carpenter J, Kenward MG, Evans S, White I (2004) Last observation carried forward and last observation analysis. Letter to the editor. Statistics in medicine 23

  • Carpenter J, Pocock S, Lamm CJ (2002) Coping with missing data in clinical trials: a model-based approach applied to asthma trials. Statistics in medicine 21:1043–1066

    Article  PubMed  Google Scholar 

  • Carpenter J, Roger J, Kenward M (2013) Analysis of longitudinal trials with protocol deviation: a framework for relevant, accessible assumptions and inference via multiple imputation. J Biopharm Stat 23:1352–1371

    Article  PubMed  Google Scholar 

  • CHMP (2010) Guideline on missing data in confirmatory clinical trials. European Medicines Agency, London

    Google Scholar 

  • Daniels MJ, Hogan JW (2008) Missing data in longitudinal studies. Strategies for Bayesian modeling and sensitivity analysis, 1st edn. Chapman & Hall/CRC, Boca Raton, FL

    Google Scholar 

  • Fegert J, Kolch M, Zito JM, Glaeske G, Janhsen K (2006) Antidepressant use in children and adolescents in Germany. J Child Adolesc Psychopharmacol 16:197–206

    Article  PubMed  Google Scholar 

  • Geman S, Geman D (1984) Stochastic relaxation, Gibbs distributions, and the Bayesion restoration of images. IEEE Trans on Pattern Anal and Mach Intell 6:721–741

    Article  CAS  Google Scholar 

  • Hamilton M (1960) A rating scale for depression. J Neurol, Neurosurg Psychiatry 23:56–62

    Article  CAS  Google Scholar 

  • Horton NJ, Kleinman KP (2007) Much ado about nothing: a comparison of missing data methods and software to fit incomplete data regression models. Am Stat 61:79–90

    Article  PubMed Central  PubMed  Google Scholar 

  • Hypericum Depression Trial Study Group (2002) Effect of Hypericum perforatum (St John's wort) in major depressive disorder. J Am Med Assoc 287:1807–1814

    Article  Google Scholar 

  • Jansen I, Hens N, Molenberghs G, Aerts M, Verbeke G, Kenward MG (2006) The nature of sensitivity in missing not at random models. Comput Stat and Data Anal 50:830–858

    Article  Google Scholar 

  • Kalb R, Trautmann-Sponsel RD, Kieser M (2001) Efficacy and tolerability of Hypericum extract WS 5572 versus placebo in mildly to moderately depressed patients. Pharmacopsychiatry 34:96,103

    Article  Google Scholar 

  • Kasper S, Gastpar M, Moller HJ, Muller WE, Volz HP, Dienel A, Kieser M (2010) Better tolerability of St. John's wort extract WS 5570 compared to treatment with SSRIs: a reanalysis of data from controlled clinical trials in acute major depression. Int Clin Psychopharmacol 25:204–213

    Article  PubMed  Google Scholar 

  • Linde K, Berner MM, Kriston L (2008) St John's wort for major depression. Cochrane Database of Systematic Reviews 4.

  • Linde K, Ramirez G, Mulrow CD, Pauls A, Weidenhammer W, Melchart D (1996) St John's wort for depression—an overview and meta-analysis of randomised clinical trials. BMJ 313:253–258

    Article  PubMed Central  PubMed  CAS  Google Scholar 

  • Little RJ, D'Agostino R, Cohen ML, Dickersin K, Emerson SS, Farrar JT, Frangakis C, Hogan JW, Molenberghs G, Murphy SA, Neaton JD, Rotnitzky A, Scharfstein D, Shih WJ, Siegel JP, Stern H (2012) The prevention and treatment of missing data in clinical trials. N Engl J Med 367:1355–1360

    Article  PubMed Central  PubMed  CAS  Google Scholar 

  • Mallinckrodt CH, Clark WS, David SR (2001) Type I error rates from mixed-effects model repeated measures versus fixed effects analysis of variance with missing values imputed via last observation carried forward. Drug Inf J 35:1215–1225

    Article  Google Scholar 

  • Mason A, Richardson S, Plewis I, Best N (2012) Strategy for modelling nonrandom missing data mechanisms in observational studies using Bayesian methods. J Offician Stat 28:279–302

    Google Scholar 

  • Michiels B, Molenberghs G (1997) Protective estimation of longitudinal categorical data with nonrandom dropout. Commun in Stat, Theory and Methods 26:65–94

    Article  Google Scholar 

  • Molenberghs G, Kenward MG (2007) Missing data in clinical studies. Wiley, Chichester

    Book  Google Scholar 

  • Molenberghs G, Thijs H, Jansen I, Beunckens C, Kenward MG, Mallinckrodt C, Carroll RJ (2004) Analyzing incomplete longitudinal clinical trial data. Biostatistics 5:445–464

    Article  PubMed  Google Scholar 

  • National Research Council (2010) The prevention and treatment of missing data in clinical trials. The National Academic Press, Washington DC

    Google Scholar 

  • Rahimi R, Nikfar S, Abdollahi M (2009) Efficacy and tolerability of Hypericum perforatum in major depressive disorder in comparison with selective serotonin reuptake inhibitors: a meta-analysis. Prog Neuro-Psychopharmacol Biol Psychiatry 33:118–127

    Article  CAS  Google Scholar 

  • Ratitch B, O'Kelly M, Tosiello R (2013) Missing data in clinical trials: from clinical assumptions to statistical analysis using pattern mixture models. Pharmaceutical Statistics: n/a-n/a.

  • Rubin DB (1976) Inference and missing data. Biometrika 63:581–592

    Article  Google Scholar 

  • Sarris J, Fava M, Schweitzer I, Mischoulon D (2012) St John's Wort (Hypericum perforatum) versus sertraline and placebo in major depressive disorder: continuation data from a 26-week RCT. Pharmacopsychiatry 45:275–278

    Article  PubMed  CAS  Google Scholar 

  • Shelton RC, Keller MB, Gelenberg A, Dunner DL, Hirschfeld R, Thase ME, Russell J, Lydiard B, Crits-Christoph P, Gallop R, Todd L, Hellerstein D, Goodnick P, Keitner G, Stahl SM, Halbreich U (2001) Effectiveness of St John's wort in major depression: a randomized controlled trial. JAMA 285:1978–1986

    Article  PubMed  CAS  Google Scholar 

  • Ware JH, Harrington D, Hunter DJ, D'Agostino RB (2012) Missing Data. N Engl J Med 367:1353–1354

    Article  CAS  Google Scholar 

  • White IR, Royston P, Wood AM (2011) Multiple imputation using chained equations: issues and guidance for practice. Stat Med 30:377–399

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

Data used in the preparation of this article were obtained from the limited access datasets (version 4.1) distributed from the NIH-supported “A Placebo-Controlled Clinical Trial of a Standardized Extract of H. perforatum in Major Depressive Disorder” (Hypericum). This is a multisite, clinical trial of persons with depression comparing the effectiveness of randomly assigned medication treatment. The purpose of this trial is to study the acute efficacy and safety of a standardized extract of the herb H. perforatum (St. John's wort) in the treatment of patients with major depression. The study was supported by NIMH contract # N01MH70007 to the Duke University Medical Center. The ClinicalTrials.gov identifier is NCT00005013. This manuscript reflects the views of the authors and may not reflect the opinions or views of the Hypericum Study Investigators or the NIH. Anneke Grobler had full access to the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Geert Molenberghs gratefully acknowledges financial support from the IAP research Network P7/06 of the Belgian Government (Belgian Science Policy).

Conflict of interest

The authors declare that they have no conflicts of interest.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Anneke C. Grobler.

Additional information

Trial Registration: Clintrials.gov, NCT00005013, http://www.clinicaltrials.gov/ct2/show/NCT00005013?term=Hypericum+perforatum+major+depression

Rights and permissions

Reprints and permissions

About this article

Cite this article

Grobler, A.C., Matthews, G. & Molenberghs, G. The impact of missing data on clinical trials: a re-analysis of a placebo controlled trial of Hypericum perforatum (St Johns wort) and sertraline in major depressive disorder. Psychopharmacology 231, 1987–1999 (2014). https://doi.org/10.1007/s00213-013-3344-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00213-013-3344-x

Keywords

Navigation