Skip to main content
SpringerLink
Log in

Impact of acquisition and interpretation on total inter-observer variability in echocardiography: results from the quality assurance program of the STAAB cohort study

  • Original Paper
  • Published:
The International Journal of Cardiovascular Imaging Aims and scope Submit manuscript

Abstract

Variability related to image acquisition and interpretation is an important issue of echocardiography in clinical trials. Nevertheless, there is no broadly accepted standard method for quality assessment of echocardiography in clinical research reports. We present analyses based on the echocardiography quality-assurance program of the ongoing STAAB cohort study (characteristics and course of heart failure stages A–B and determinants of progression). In 43 healthy individuals (mean age 50 ± 14 years; 18 females), duplicate echocardiography scans were acquired and mutually interpreted by one of three trained sonographers and an EACVI certified physician, respectively. Acquisition (AcV), interpretation (InV), and inter-observer variability (IOV; i.e., variability between the acquisition-interpretation sequences of two different observers), were determined for selected M-mode, B-mode, and Doppler parameters. We calculated Bland–Altman upper 95% limits of absolute differences, implying that 95% of measurement differences were smaller/equal to the given value: e.g. LV end-diastolic volume (mL): 25.0, 25.0, 27.9; septal e′ velocity (cm/s): 3.03, 1.25, 3.58. Further, 90, 85, and 80% upper limits of absolute differences were determined for the respective parameters. Both, acquisition and interpretation, independently and sizably contributed to IOV. As such, separate assessment of AcV and InV is likely to aid in echocardiography training and quality-assurance. Our results further suggest to routinely determine IOV in clinical trials as a comprehensive measure of imaging quality. The derived 95, 90, 85, and 80% upper limits of absolute differences are suggested as reproducibility targets of future studies, thus contributing to the international efforts of standardization in quality-assurance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Abbreviations

2D:

Two-dimensional

A:

A-wave velocity

a′:

Peak late- diastolic lengthening velocity

AcV:

Acquisition variability

CW:

Continuous wave

E:

E-wave velocity

e′:

Peak early diastolic lengthening velocity

EACVI:

European Association of Cardiovascular Imaging

EF:

Ejection fraction

IaInV:

Intra-interpretation variability

ICC:

Intraclass correlation coefficient

InV:

Interpretation variability

IOV:

Inter-observer variability

STAAB:

Characteristics and course of heart failure stages A–B and determinants of progression cohort study

LA:

Left atrium

LV:

Left ventricle

LOA:

Limits of agreement

LVDed:

Left ventricular end-diastolic diameter

LVVed:

Left ventricular end-diastolic volume

LVVes:

Left ventricular end-systolic volume

MAPSE:

Mitral annular plane systolic excursion

PW:

Pulsed wave

RA:

Right atrium

RV:

Right ventricle

s′:

Peak systolic shortening velocity

TAPSE:

Tricuspid annular plane systolic excursion

TDI:

Tissue Doppler imaging

References

  1. Crowley AL, Yow E, Barnhart HX, Daubert MA, Bigelow R, Sullivan DC, Pencina M, Douglas PS (2016) Critical review of current approaches for echocardiographic reproducibility and reliability assessment in clinical research. J Am Soc Echocardiogr 29(12):1144–1154.e7. https://doi.org/10.1016/j.echo.2016.08.006

    Article  PubMed  Google Scholar 

  2. Pellikka PA, Douglas PS, Miller JG, Abraham TP, Baumann R, Buxton DB, Byrd BF 3rd, Chen P, Cook NL, Gardin JM, Hansen G, Houle HC, Husson S, Kaul S, Klein AL, Lang RM, Leong-Poi H, Lopez H, Mahmoud TM, Maslak S, McCulloch ML, Metz S, Nagueh SF, Pearlman AS, Pibarot P, Picard MH, Porter TR, Prater D, Rodriguez R, Sarano ME, Scherrer-Crosbie M, Shirali GS, Sinusas A, Slosky JJ, Sugeng L, Tatpati A, Villanueva FS, von Ramm OT, Weissman NJ, Zamani S (2013) American society of echocardiography cardiovascular technology and research summit: a roadmap for 2020. J Am Soc Echocardiogr 26(4):325–338. https://doi.org/10.1016/j.echo.2013.02.003

    Article  PubMed  Google Scholar 

  3. Kardys I, Deckers JW, Stricker BH, Vletter WB, Hofman A, Witteman J (2010) Distribution of echocardiographic parameters and their associations with cardiovascular risk factors in the Rotterdam Study. Eur J Epidemiol 25(7):481–490. https://doi.org/10.1007/s10654-010-9453-5

    Article  PubMed  PubMed Central  Google Scholar 

  4. Kou S, Caballero L, Dulgheru R, Voilliot D, De Sousa C, Kacharava G, Athanassopoulos GD, Barone D, Baroni M, Cardim N, Gomez De Diego JJ, Hagendorff A, Henri C, Hristova K, Lopez T, Magne J, De La Morena G, Popescu BA, Penicka M, Ozyigit T, Rodrigo Carbonero JD, Salustri A, Van De Veire N, Von Bardeleben RS, Vinereanu D, Voigt JU, Zamorano JL, Donal E, Lang RM, Badano LP, Lancellotti P (2014) Echocardiographic reference ranges for normal cardiac chamber size: results from the NORRE study. Eur Heart J Cardiovasc Imaging 15(6):680–690. https://doi.org/10.1093/ehjci/jet284

    Article  PubMed  PubMed Central  Google Scholar 

  5. Jenkins C, Bricknell K, Hanekom L, Marwick TH (2004) Reproducibility and accuracy of echocardiographic measurements of left ventricular parameters using real-time three-dimensional echocardiography. J Am Coll Cardiol 44(4):878–886. https://doi.org/10.1016/j.jacc.2004.05.050

    Article  PubMed  Google Scholar 

  6. Wagner M, Tiffe T, Morbach C, Gelbrich G, Stork S, Heuschmann PU, Consortium S (2016) Characteristics and course of heart failure stages A-B and determinants of progression—design and rationale of the STAAB cohort study. Eur J Prev Cardiol 24(5):468–479. https://doi.org/10.1177/2047487316680693

    Article  PubMed  Google Scholar 

  7. Lang RM, Badano LP, Mor-Avi V, Afilalo J, Armstrong A, Ernande L, Flachskampf FA, Foster E, Goldstein SA, Kuznetsova T, Lancellotti P, Muraru D, Picard MH, Rietzschel ER, Rudski L, Spencer KT, Tsang W, Voigt JU (2015) Recommendations for cardiac chamber quantification by echocardiography in adults: an update from the American Society of Echocardiography and the European Association of Cardiovascular Imaging. Eur Heart J Cardiovasc Imaging 16(3):233–270. https://doi.org/10.1093/ehjci/jev014

    Article  PubMed  Google Scholar 

  8. Bland JM, Altman DG (1999) Measuring agreement in method comparison studies. Stat Methods Med Res 8(2):135–160

    Article  PubMed  CAS  Google Scholar 

  9. Thorstensen A, Dalen H, Amundsen BH, Aase SA, Stoylen A (2010) Reproducibility in echocardiographic assessment of the left ventricular global and regional function, the HUNT study. Eur J Echocardiogr 11(2):149–156. https://doi.org/10.1093/ejechocard/jep188

    Article  PubMed  Google Scholar 

  10. Stritzke J, Linsel-Nitschke P, Markus MR, Mayer B, Lieb W, Luchner A, Doring A, Koenig W, Keil U, Hense HW, Schunkert H, Investigators MK (2009) Association between degenerative aortic valve disease and long-term exposure to cardiovascular risk factors: results of the longitudinal population-based KORA/MONICA survey. Eur Heart J 30(16):2044–2053. https://doi.org/10.1093/eurheartj/ehp287

    Article  PubMed  Google Scholar 

  11. Colan SD, Shirali G, Margossian R, Gallagher D, Altmann K, Canter C, Chen S, Golding F, Radojewski E, Camitta M, Carboni M, Rychik J, Stylianou M, Tani LY, Selamet Tierney ES, Wang Y, Sleeper LA, Pediatric Heart Network I (2012) The ventricular volume variability study of the Pediatric Heart Network: study design and impact of beat averaging and variable type on the reproducibility of echocardiographic measurements in children with chronic dilated cardiomyopathy. J Am Soc Echocardiogr 25(8):842–854.e6. https://doi.org/10.1016/j.echo.2012.05.004

    Article  PubMed  PubMed Central  Google Scholar 

  12. Otterstad JE, Froeland G, St John Sutton M, Holme I (1997) Accuracy and reproducibility of biplane two-dimensional echocardiographic measurements of left ventricular dimensions and function. Eur Heart J 18(3):507–513

    Article  PubMed  CAS  Google Scholar 

  13. Barnhart HX, Yow E, Crowley AL, Daubert MA, Rabineau D, Bigelow R, Pencina M, Douglas PS (2014) Choice of agreement indices for assessing and improving measurement reproducibility in a core laboratory setting. Stat Methods Med Res 25(6):2939–2958. https://doi.org/10.1177/0962280214534651

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We thank Martina Bauer, lead sonographer of the Comprehensive Heart Failure Center (CHFC), for excellent echocardiography teaching, as well as Jasmin Simon, Martin Cramer and Frances Knobeloch, CHFC sonographers, for their valuable contribution to this study.

Funding

The study is supported by the German Ministry of Research and Education within the Comprehensive Heart Failure Centre Würzburg (BMBF 01EO1004 and 01EO1504). This publication was funded by the German Research Foundation (DFG) and the University of Wuerzburg in the funding programme Open Access Publishing.

Author information

Authors and Affiliations

  1. Comprehensive Heart Failure Center, University Hospital and University of Würzburg, Am Schwarzenberg 15, 97078, Würzburg, Germany

    Caroline Morbach, Margret Breunig, Theresa Tiffe, Martin Wagner & Stefan Störk

  2. Department of Medicine I, Cardiology, University Hospital Würzburg, Oberdürrbacher Str. 6, 97080, Würzburg, Germany

    Caroline Morbach, Margret Breunig & Stefan Störk

  3. Institute of Clinical Epidemiology and Biometry, University of Würzburg, Josef-Schneider-Str. 2a, 97080, Würzburg, Germany

    Götz Gelbrich, Theresa Tiffe, Martin Wagner & Peter U. Heuschmann

  4. Clinical Trial Center, University Hospital Würzburg, Josef-Schneider-Str. 2a, 97080, Würzburg, Germany

    Götz Gelbrich & Peter U. Heuschmann

Authors
  1. Caroline Morbach
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Götz Gelbrich
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Margret Breunig
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Theresa Tiffe
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Martin Wagner
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Peter U. Heuschmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Stefan Störk
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stefan Störk.

Ethics declarations

Conflict of interest

CM reports a co-investigatorship in a trial sponsored by NOVARTIS, a travel grant from THERMO FISHER and a speaker honorarium from AMGEN. GG reports grants from University Göttingen within the FIND-AF-randomized trial (FIND-AF-randomized is supported by an unrestricted grant to the University Göttingen from Boehringer-Ingelheim), grants from University Göttingen within DZHK analysis projects (the DZHK is funded by a BMBF grant), personal fees from Charité - Universitätsmedizin Berlin within the TIM-HF-II trial for data safety and monitoring board membership (TIM-HF-II is supported by a BMBF grant to the Charité), grants from University Hospital Würzburg within the MOOD-HF trial for biometry and steering committee membership (MOOD-HF is supported by a BMBF grant and an unrestricted grant to the University Hospital Würzburg from Lundbeck), outside the submitted work. PUH reports research grants from the German Ministry of Research and Education, European Union, Charité, Berlin Chamber of Physicians, German Parkinson Society, University Hospital Würzburg, Robert-Koch-Institute, German Heart Foundation, Charité–Universitätsmedizin Berlin (within MonDAFIS; MonDAFIS is supported by an unrestricted research grant to the Charité from Bayer), University Göttingen (within FIND-AF-randomized; FIND-AF randomized is supported by an unrestricted research grant to the University Göttingen from Boehringer-Ingelheim), and University Hospital Heidelberg (within RASUNOA-prime; RASUNOA-prime is supported by an unrestricted research grant to the University Hospital Heidelberg from Bayer, BMS, Boehringer-Ingelheim, Daiichi Sankyo), outside submitted work. SS reports research grants from the German Ministry of Education and Research, European Union, University Hospital Würzburg; participation in Data Safety Monitoring and Event Adjudication Boards in trials sponsored by ROCHE and MEDTRONIC; principal investigator in trials (co-)sponsored by BOEHRINGER, NOVARTIS, BAYER, LUNDBECK; speaker honaria by BOEHRINGER, SERVIER, NOVARTIS, ASTRA-ZENECA, PFIZER, BAYER. MW declares that he has no conflict of interest. TT declares that she has no conflict of interest. MB declares that she has no conflict of interest.

Ethical approval

The STAAB cohort study protocol and procedures received positive votes from the Ethics Committee of the Medical Faculty (vote 98/13) as well as from the data protection officer of the University of Würzburg (J-117.605-09/13). All procedures performed in this study were in accordance with the ethical standards of the institutional and national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

All participants provided written informed consent prior to any study examination.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Morbach, C., Gelbrich, G., Breunig, M. et al. Impact of acquisition and interpretation on total inter-observer variability in echocardiography: results from the quality assurance program of the STAAB cohort study. Int J Cardiovasc Imaging 34, 1057–1065 (2018). https://doi.org/10.1007/s10554-018-1315-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10554-018-1315-3

Keywords

Search

Navigation