Hostname: page-component-8448b6f56d-mp689 Total loading time: 0 Render date: 2024-04-23T06:52:35.474Z Has data issue: false hasContentIssue false
  • English
  • Français

Association between methodological characteristics and outcome in health technology assessments which included case series

Published online by Cambridge University Press:  04 August 2005

Ken Stein ,
Kim Dalziel ,
Ruth Garside ,
Emanuela Castelnuovo  and
Ali Round
Ken Stein
Affiliation:
Peninsula Medical School
Kim Dalziel
Affiliation:
Peninsula Medical School
Ruth Garside
Affiliation:
Peninsula Medical School
Emanuela Castelnuovo
Affiliation:
Peninsula Medical School
Ali Round
Affiliation:
Peninsula Medical School
Get access Rights & Permissions [Opens in a new window]

Abstract

Objectives: Case series constitute a weak form of evidence for effectiveness of health technologies. However, for a variety of reasons, such studies may be included in health technology assessments. There are no clear criteria for assessing the quality of case series. We carried out an empirical investigation of the association between outcome frequency and methodological characteristics in a sample of health technology assessments.

Methods: Systematic reviews of functional endoscopic sinus surgery for nasal polyps, spinal cord stimulation for chronic back pain, and percutaneous transluminal coronary angioplasty and coronary artery bypass grafting for chronic stable angina were identified as containing more than forty case series. Data were extracted by one reviewer and checked by a second on population characteristics, outcomes, and the following methodological features: sample size, prospective/retrospective approach, consecutive recruitment, multi- or single-center organization, length of follow-up, independence of outcome measurement, and date of publication. Association between methodological features and outcome were explored in univariate and multivariate analyses using parametric and nonparametric tests and robust regression or analysis of variance/analysis of covariance, as appropriate.

Results: Included reviews contained between forty-two and seventy-six case series studies, involving 5 to 172,283 participants. Reporting of methodological features was poor and limited the analyses. In general, we found little evidence of any association between methodological characteristics and outcome. Sample size is used as an inclusion criterion in many reviews of case series but was consistently shown to have no relationship to outcome in all analyses. A prospective approach was not associated with outcome. Insufficient data were available to explore consecutive recruitment. Mixed results were shown for length of follow-up, independence of outcome measurement, and publication date.

Conclusion: We found little evidence to support the use of many of the factors included in tools used for quality assessment of case series. Importantly, we found no relationship between study size and outcome across the four examples studied. Isolated examples of a potentially important relationship between other methodological factors and outcome were shown, for example, blinding of outcome measurement, but these examples were not shown consistently across the small number of examples studied. Further research into the determinants of quality in case series studies is required to support health technology assessment.

Keywords

Case seriesUncontrolled studiesMethodsMethodological qualityCritical appraisalStudy design
Type
GENERAL ESSAYS
Information
International Journal of Technology Assessment in Health Care , Volume 21 , Issue 3 , July 2005 , pp. 277 - 287
Copyright
© 2005 Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bryant J, Cave C, Mihaylova B, et al. 2002 Clinical effectiveness and cost-effectiveness of growth hormone in children: A systematic review and economic evaluation. Health Technol Assess. 6: 1– 168.Google Scholar
Clarke M, Oxman A. 2002. Cochrane Reviewers' Handbook 4.1.5 [updated April 2002]. The Cochrane Library. Oxford: Update Software;
Dalziel K, Garside R, Stein K, Round A. 2003. Use of case series evidence in National Institute for Clinical Excellence (NICE) health technology assessments—why, how and did it matter? Canmore, Alberta: International Society for Technology Assessment in Health Care (ISTAHC);
Dalziel K, Stein K, Round A, Garside R. 2003 Systematic review of endoscopic sinus surgery for nasal polyps. Health Technol Assess. 7: 1– 159.Google Scholar
Fitzpatrick R, Shortall E, Sculpher M, et al. 1998 Primary total hip replacement surgery: A systematic review of outcomes and modelling of cost-effectiveness associated with different prostheses. Health Technol Assess. 2: 1– 64.Google Scholar
Grodstein F, Stampfer MJ, Colditz GA. 1997 Postmenopausal hormone therapy and mortality. N Engl J Med. 336: 1769– 1775.Google Scholar
Hulley S, Grady D, Bush T, 1998 for the Heart and Estrogen/Progestin Replacement Study (HERS) Research group. Randomised trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. JAMA. 280: 605– 613.Google Scholar
Hyde C, Wake B, Bryan S, et al. 2002 Fludarabine as second-line therapy for B-cell chronic lymphocytic leukaemia: A technology assessment. Health Technol Assess. 6: 1– 89.Google Scholar
Jobanputra P, Parry D, Fry-Smith A, Burls A. 2001 Effectiveness of autologous chondrocyte transplantation for hyaline cartilage defects in knees: A rapid and systematic review. Health Technol Assess. 5: 1– 57.Google Scholar
Juni P, Altman D, Egger M. 2000 Assessing the quality of randomised controlled trials. In: Egger M, Davey-Smith G, Altman D, eds. Systematic reviews in health care. London: BMJ Publishing Group;
Sculpher M, Petticrew M, Kelland J, et al. 1998 Resource allocation for chronic stable angina: A systematic review of effectiveness, costs and cost-effectiveness of alternative interventions. Health Technol Assess. 2: 1– 176.Google Scholar
Taylor R, Van Buyten J-P, Buchser E, et al. 2004 Spinal cord stimulation for chronic low back pain/failed back surgery syndrome and complex regional pain syndrome–systematic review of the clinical effectiveness and cost-effectiveness literature. J Pain Symptom Manage. 27: 370– 378.Google Scholar
Vale L, Wyness L, McCormack et al. 2002 A systematic review of the effectiveness and cost-effectiveness of metal-on-metal hip resurfacing arthroplasty for treatment of hip disease. Health Technol Assess. 6: 1– 109.Google Scholar
Vardulaki KA, Bennett-Lloyd B, Parfitt J, et al. 2000. A systematic review of the effectiveness and cost-effectiveness of laparoscopic surgery for colorectal cancer. London: National Institute for Clinical Excellence;
Wake B, Hyde C, Bryan S, et al. 2002 Rituximab as third-line treatment for refractory or recurrent stage III or IV follicular non-Hodgkin's lymphoma: A systematic review and economic evaluation. Health Technol Assess. 6: 1– 85.Google Scholar