Abstract
Background
Clinical practice guidelines (CPGs) can positively affect the quality of patient care offered by physicians because they decrease variability in clinical practice and may help reduce unnecessary testing, promoting a more responsible use of resources. Building on existing framework for reporting guideline development, including the work of the Enhancing the Quality and Transparency of Health Research Network, the Reporting Items for Practice Guidelines in Healthcare (RIGHT) Working Group created a 2016 checklist of 35 items considered essential for high-quality reporting of CPGs.
Objectives
To evaluate how many previously published CPGs in orthopedic surgery met the RIGHT criteria and assess how improvements can be made in future orthopedic CPGs based on any found deficiencies.
Methods
All 18 CPGs published before January 1, 2018, by the American Academy of Orthopedic Surgeons (AAOS) are publicly available on orthoguidelines.org. Two authors downloaded each file and both of those authors independently scored each CPG using piloted abstraction RIGHT checklist forms.
Results
Of the 35 RIGHT criteria outlined in 22 checklist items, 23 (65.7%) were met across all AAOS guidelines, 6 (17.1%) were not met by any of the AAOS guidelines, and 6 (17.2%) were met by some of the AAOS guidelines.
Conclusion
Overall, the AAOS guidelines addressed many important recommendations within the RIGHT checklist. Assessing adherence to the RIGHT checklist can help ensure that future guidelines are more effectively communicated, hopefully assisting end users in efficient implementation and increasing the level of evidence-based patient care.
Clinical practice guidelines (CPGs) provide recommendations to physicians for the diagnosis and management of disease. This guidance is presumed to be constructed from the most current and methodologically sound research evidence. Recommendations discuss multiple aspects of patient care, from patient decisions made by physicians to financial decisions made by insurers and governments.1 Clinical practice guidelines have great potential to positively affect the quality of patient care offered by physicians because they decrease variability in clinical practice. Furthermore, these guidelines may help reduce unnecessary testing and promote a more responsible use of resources. For example, Adair et al2 demonstrated how CPGs decreased unnecessary echocardiograms before hip fracture surgery.
The ability of CPGs to positively affect patient outcomes and reduce waste depends on many factors. One such factor is the methodologic soundness of the research on which these recommendations are based. For instance, a guideline developed by the American Academy of Orthopedic Surgeons (AAOS) for management of rotator cuff problems generated criticism because the guideline's recommendations were founded almost exclusively on expert opinion rather than systematically compiled literature.3 A second set of interrelated factors associated with the effectiveness of CPGs is how quickly and effectively they are disseminated and how well physicians adhere to them. A 2007 study assessed barriers to implementation and found that a lack of knowledge about guidelines and a lack of time to research them were the most frequently reported obstacles.4 These barriers may be overcome, in part, by producing CPGs that clearly communicate the methods and the strength of the evidence underpinning the recommendations.
The Reporting Items for Practice Guidelines in Healthcare (RIGHT) checklist was developed in 2016 to address the quality of reporting in the development of CPGs. Building on existing framework for reporting guideline development, including the work of the Enhancing the Quality and Transparency of Health Research Network, the RIGHT Working Group created a checklist of 22 items (10 of which contain multiple elements, for a total of 35 criteria) considered essential for high-quality reporting of CPGs.5 By emulating similar efforts in the reporting of clinical trials and systematic reviews, the RIGHT checklist can assist developers in reporting CPGs, support journal editors and peer reviewers when considering guidelines for publication, and help physicians understand and implement a guideline, all of which increase the reach and speed of guideline dissemination.5
The aim of this study was to use the RIGHT checklist to assess the strengths and weaknesses in CPGs published by the American Academy of Orthopedic Surgeons (AAOS).
To our knowledge, this study is the first in orthopedics to apply the RIGHT checklist to evaluate the clarity of reporting; by doing so, we hope to highlight the areas in which previously published orthopedic guidelines have excelled, while also identifying areas of improvement for future guideline development. Addressing these areas in which prior CPGs have deviated from the RIGHT recommendations should support future guidelines being more thoroughly reported and better understood by journal editors, peer reviewers, orthopedic surgeons, and general physicians. Clear guidelines may ultimately lead to better evidence-based patient care.
Methods
Protocol Development and Registration
Search strategies, eligibility criteria, and data abstraction were specified in the research protocol developed and piloted a priori. This study did not meet the regulatory definition of human subject research as defined in 45 CFR 46.102(d) and (f) of the Department of Health and Human Service Code of Federal Regulations and was not subject to institutional review board oversight.
Identification of Eligible CPGs
All 186-23 CPGs published by the AAOS in the field of orthopedic surgery on or before January 1, 2018, were evaluated. All guidelines were obtained from the AAOS website (https://www.aaos.org/guidelines/?ssopc=1) by 2 authors (K.F. and J.X.C.). All AAOS guidelines state that the CPGs went through external peer review by external content experts prior to approval by AAOS for publication. All CPGs were included; however, we did not include consensus statements or appropriate use criteria.
Data Abstraction and Scoring
Two authors (K.F. and J.X.C.) independently abstracted and scored the CPGs using previously tested extraction forms. These forms were created by the senior author for previous studies for the purpose of ensuring as clean and accurate data extraction as possible. Each author's score was verified by the second investigator. Disagreements were to be resolved by consensus between the pair; however, no disagreements occurred during scoring. A third-party adjudication process was established in the protocol to be used if the first 2 authors could not resolve their disagreement by consensus. All authors were trained in using the RIGHT checklist prior to scoring to ensure reliability between investigators.
RIGHT Checklist
The RIGHT checklist (http://right-statement.org/uploads/3ddf27568783e0653096fb31.pdf) focuses on essential components for well-reported CPGs. This checklist was developed by a multidisciplinary team of experts from 12 countries. The RIGHT checklist consists of 22 checklist items (for a total of 35 components) that cover multiple domains, including basic information, background, evidence, recommendations, review and quality assurance, funding declaration, and management of interests.5 The RIGHT checklist suggests that the following safeguards be included within each CPG5:
standardized information within the title/subtitle
■ an executive summary
■ definitions for key terms and abbreviations
■ corresponding developer information
■ a brief description of the health problem
■ a description of the aims and objectives of the guideline
■ information on the target population
■ information on the end users of the guideline
■ a summary of the health care questions that formed the basis of the guideline
■ a description of how the body of evidence was assessed
■ suggestions regarding the structure of the recommendations
■ rationale for the recommendations
■ evidence to the process in reaching the recommendation decision
■ a summary of any external review and quality assurance
■ the funding source and conflicts of interest
■ a description of where the guideline may be accessed
■ suggestions for future research
■ limitations of the guideline
Results
All 18 guidelines were obtained for evaluation by the RIGHT checklist. The CPGs we evaluated were published between 2009 and 2017: 9 of the 18 guidelines (50.0%) were published between 2009 and 2011, 5 (27.8%) were published between 2012 and 2015, and 4 (22.2%) were published between 2015 and 2017. Thus, the median age of the AAOS guidelines available at the time of this study was 4.8 years.
Of the 35 criteria, 23 (65.7%) were met across all AAOS guidelines (eTable) and 6 (17.1%) were not met by any of the AAOS guidelines (eTable and Figure). The other 6 criteria were a combination of met and unmet guidelines between the individual CPGs.
Tornado plot displaying the Reporting Items for Practice Guidelines in Healthcare (RIGHT) checklist items met by the guidelines.
Discussion
Transparent and effective CPGs can positively influence patient outcomes in orthopedic surgery. Adequate reporting of CPG development can lead to more efficient dissemination, improved clinical understanding, and heightened awareness of CPGs5; the effectiveness of a CPG is directly proportional to the quality of evidence on which its recommendations were developed.24 One of the most effective ways to ensure that a guideline reflects the strongest available evidence is timely updating of the guideline. A recent study showed that guidelines quickly become outdated, with roughly 20% becoming obsolete after 3 years.25 The AAOS has stated its commitment to retiring, reviewing, or updating its guidelines every 5 years.26 At the time of this study, the average age of the published guidelines was 4.8 years, with half being 5 or more years old. Taking this information into account, many of these guidelines may no longer reflect the current body of orthopedic evidence. Our findings can thus serve as a starting point for improvements to the reporting of subsequent versions of the AAOS guidelines.
To encourage the implementation of CPGs and adherence to them by physicians, clear communication about the methods used to create them is important. A 199927 study identified a lack of evidence (or the inclusion of insufficient evidence) as a critical factor in the creation of negative attitudinal barriers to guideline implementation. Clearly describing the methodologic approach used to assess the strength of the evidence underpinning the guideline and clearly identifying the roles and affiliations of all those involved in creating the guideline can help to remove these barriers. Incorporation of these factors as criteria in the RIGHT checklist further demonstrates their importance during guideline development.
Of the 18 AAOS guidelines included in this study, all of them met the recommendation of listing all individuals involved in developing the guideline, including title, role, and institutional affiliation. All guidelines also met the recommendation of describing the approach used to assess the certainty of the body of evidence. However, only 1 of the 18 guidelines adequately listed contact information for at least 1 corresponding author or developer that can be contacted about the guideline (Figure). Although some of the guidelines did have contact information for one of the authors, it was not specified whether that author was the corresponding author who could be contacted regarding the guideline. Contacting developers of a guideline would prove useful to physicians and other groups attempting to create new guidelines or adapt existing guidelines.
A generalized, national CPG cannot be expected to be effective in every situation at all times and across all demographics. Customizing a CPG to a specific organization or region may improve implementation and adherence and lead to better patient outcomes. Studies28,29 have indicated that active involvement of physicians in the customization of a guideline can lead to changes in practice. Adaptation of existing high-quality guidelines to different organizational or regional settings can decrease duplication of effort and enhance applicability. By clearly describing the specific setting for which a guideline is intended, changes required in adapting it for different settings can be more readily identified. None of the AAOS guidelines met the RIGHT checklist recommendation of adequately describing setting(s) for which the guideline was intended. It should be noted that every guideline did describe the intended end-user population for the guideline.
Another barrier to implementation of and adherence to a CPG is suspicion about underlying financial motives, with nonimplementers citing beliefs that cost reduction and standardization of care would negatively affect the patient-physician relationship.30 Clearly stating the source of funding and the role of the funder(s) in each stage of guideline development and implementation can help mitigate these concerns. Although all of the AAOS guidelines met criteria for describing the sources of funding, none of them specified the role of the funder(s) in the different stages of development and implementation. Every guideline did describe potential conflicts of interest, both financial and nonfinancial.
The aim of our study was to evaluate the completeness of reporting (reporting quality) of the AAOS CPGs, not their methodologic quality. Reporting quality and methodologic quality are considered unique constructs and should be evaluated differently. With this in mind, it is important to showcase the differences between the RIGHT Checklist (reporting quality) and other guideline evaluation tools such as the Appraisal of Guidelines for Research and Evaluation (AGREE) instrument (methodologic quality). The AGREE instrument initially released in 2003 and the updated version, AGREE II, are evaluations of methodologic quality for CPGs structured around 6 quality domains, which include (1) scope and purpose, (2) stakeholder involvement, (3) rigor of development, (4) clarity of presentation, (5) applicability, and (6) editorial independence.34 In contrast, the RIGHT checklist was developed using the process implemented by other reporting guideline work groups, and its structure reflects the order in which readers would expect to find the information (beginning with the title and then executive summary). It also includes an explanation and elaboration document to showcase examples of well-reported information. Further, the RIGHT checklist contains important items that are currently overlooked by AGREE-II: quality assurance, access, suggestions for future research, and guideline limitations.5 It also emphasizes the importance of using the Population, Intervention, Comparator, Outcome (PICO) clinical question format and deriving recommendations from research evidence.6 Future use of the RIGHT checklist will provide additional information regarding its utility in studying CPGs, and the possibility for studying guidelines using both the RIGHT Checklist and AGREE may be of interest. This study and the RIGHT checklist have limitations. The RIGHT checklist is a new tool, and it has not been validated as leading to the creation of guidelines that are more readily implemented in clinical practice. Some of the criteria comprising the guideline are also generalized, leaving interpretation to the evaluator. Despite these limitations, our hope is that this study can highlight opportunities for improvement in future development of CPGs. Because the RIGHT checklist is new and lacks validation, it should be stressed that the AAOS guidelines (and other guidelines) should not be criticized for not meeting all of the standards within the checklist. Rather, the checklist should serve as a point of discussion to improve guidelines in the future.
Conclusion
Overall, the AAOS guidelines addressed many important recommendations within the RIGHT checklist. Providing clear and precise recommendations within a guideline will assist end users in more efficiently implementing the guidelines in practice. With confirmation of the strength of recommendations and the certainty of the supporting evidence in each guideline, physicians can be confident that they are implementing guidelines that are supported by evidence-based medical practice. Through the identification of the strengths and weaknesses in current guidelines, future guidelines can be more effectively implemented and more easily communicated to end users. These factors will lead to greater adherence, ultimately increasing the level of evidence-based patient care in orthopedic surgery.
References
1. WoolfSH, GrolR, HutchinsonA, EcclesM, Grimshaw J. Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ. 1999;318(7182):527-530. doi:10.1136/bmj.318.7182.527Search in Google Scholar PubMed PubMed Central
2. AdairC, SwartE, SeymourR, PattJ, KarunakarMA. Clinical practice guidelines decrease unnecessary echocardiograms before hip fracture surgery. J Bone Joint Surg Am.2017;99(8):676-680. doi:10.2106/JBJS.16.01108Search in Google Scholar PubMed
3. LubowitzJH, McIntyreLF, ProvencherMT, PoehlingGG. AAOS rotator cuff clinical practice guideline misses the mark. Arthroscopy.2012;28(5):589-592. doi:10.1016/j.arthro.2012.03.021Search in Google Scholar PubMed
4. CochraneLJ, OlsonCA, MurrayS, DupuisM, ToomanT, HayesS. Gaps between knowing and doing: understanding and assessing the barriers to optimal health care. J Contin Educ Health Prof.2007;27(2):94-102.10.1002/chp.106Search in Google Scholar PubMed
5. ChenY, YangK, MarušicA, et al.. A reporting tool for practice guidelines in health care: the RIGHT statement. Ann Intern Med.2017;166(2):128-132. doi:10.7326/M16-1565Search in Google Scholar PubMed
6. Health (nursing, medicine, allied health): search strategies: framing the question (PICO). NYU Libraries website. https://guides.nyu.edu/c.php?g=276561&p=1847897. Accessed December 30, 2019.Search in Google Scholar
7. KouJ. AAOS clinical practice guideline: acute achilles tendon rupture. J Am Acad Orthop Surg. 2010;18(8):511-513. doi:10.5435/00124635-201008000-00008Search in Google Scholar PubMed
8. CareyJL, SheaKG. AAOS clinical practice guideline: management of anterior cruciate ligament injuries: evidence-based guideline. J Am Acad Orthop Surg2015;23(5):e6-e8. doi:10.5435/JAAOS-D-15-00095Search in Google Scholar PubMed
9. AmirfeyzR, LeslieIJ. AAOS (American Academy of Orthopaedic Surgeons) clinical practice guideline. Treatment of carpal tunnel syndrome. Orthop Trauma. 2011;25(1):78. doi:10.1016/j.mporth.2010.02.001Search in Google Scholar
10. LichtmanDM, BindraRR, BoyerMI, et al.. American Academy of Orthopaedic Surgeons clinical practice guideline on: the treatment of distal radius fractures. J Bone Joint Surg Am.2011;93(8):775-778. doi:10.2106/JBJS.938eboSearch in Google Scholar PubMed
11. IzquierdoR, VoloshinI, EdwardsS, et al.. Treatment of glenohumeral osteoarthritis. J Am Acad Orthop Surg. 2010;18(6):375-82. doi:10.5435/00124635-201006000-00010Search in Google Scholar PubMed
12. RobertsKC, BroxWT. Management of hip fractures in the elderly. J Am Acad Orthop Surg. 2015;23(2):131-137. doi:10.5435/JAAOS-D-14-0043224.Search in Google Scholar
13. JevsevarDS, AbtE.The new AAOS-ADA clinical practice guideline on prevention of orthopedic implant infection in patients undergoing dental procedures.J Am Acad Orthop Surg. 2013;21(3):195-197. doi:10.5435/JAAOS-21-03-195Search in Google Scholar PubMed
14. QuinnRH, MurrayJ, PezoldR, HallQ. Management of osteoarthritis of the hip. J Am Acad Orthop Surg. 2018;26(20):e434-e436. doi:10.5435/JAAOS-D-18-00351Search in Google Scholar PubMed
15. JevsevarDS. Treatment of osteoarthritis of the knee: evidence-based guideline, 2nd edition. J Am Acad Orthop Surg. 2013;21(9):571-576. doi:10.5435/JAAOS-21-09-571Search in Google Scholar PubMed
16. ChambersHG, SheaKG, CareyJL. AAOS clinical practice guideline: diagnosis and treatment of osteochondritis dissecans. J Am Acad Orthop Surg. 2011;19(5):307-309. doi:10.5435/00124635-201105000-00008Search in Google Scholar PubMed
17. MulpuriK, SongKM, GrossRH, et al.. The American Academy of Orthopaedic Surgeons evidence-based guideline on detection and nonoperative management of pediatric developmental dysplasia of the hip in infants up to six months of age. J Bone Joint Surg Am. 2015;97(20):1717-1718. doi:10.2106/jbjs.o.00500Search in Google Scholar
18. JevsevarDS, SheaKG, MurrayJN, SevarinoKS. AAOS clinical practice guideline on the treatment of pediatric diaphyseal femur fractures. J Am Acad Orthop Surg. 2015;23(12):e101. doi:10.5435/JAAOS-D-15-00523Search in Google Scholar PubMed
19. MulpuriK, HosalkarH, HowardA. AAOS clinical practice guideline: the treatment of pediatric supracondylar humerus fractures. J Am Acad Orthop Surg. 2012;20(5):328-330. doi:10.5435/JAAOS-20-05-328.Search in Google Scholar PubMed
20. ParviziJ, Della ValleCJ. AAOS Clinical Practice Guideline: diagnosis and treatment of periprosthetic joint infections of the hip and knee. J Am Acad Orthop Surg. 2010;18(12):771-772. doi:10.5435/00124635-201012000-00007Search in Google Scholar PubMed
21. TashjianRZ. AAOS clinical practice guideline: optimizing the management of rotator cuff problems. J Am Acad Orthop Surg. 2011;19(6):380-383. doi:10.5435/00124635-201106000-00008Search in Google Scholar PubMed
22. McGuireR. AAOS clinical practice guideline: the treatment of symptomatic osteoporotic spinal compression fractures. J Am Acad Orthop Surg. 2011;19(3):183-184.10.5435/00124635-201103000-00008Search in Google Scholar PubMed
23. MontMA, JacobsJJ. AAOS clinical practice guideline: preventing venous thromboembolic disease in patients undergoing elective hip and knee arthroplasty. J Am Acad Orthop Surg. 2011;19(12):777-778. doi:10.5435/00124635-201112000-00008Search in Google Scholar PubMed
24. TurnerL, ShamseerL, AltmanDG, et al.. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst Rev. 2012;11:MR000030.10.1002/14651858.MR000030.pub2Search in Google Scholar PubMed PubMed Central
25. MoherD, LiberatiA, TezlaffJ, AltmanDG. The PRISMA Group. Preferred reporting items for systematic reviews and meta analyses. PLoS Med. 2009;6(7):e1000097. doi:10.1371/journal.pmed1000097Search in Google Scholar
26. OrthoGuidelines. American Academy of Orthopedic Surgeons website. http://www.orthoguidelines.org/guidelines. Accessed December 30, 2019.Search in Google Scholar
27. BergmanDA. Evidence-based guidelines and critical pathways for quality improvement. Pediatrics.1999;103(1suppl E):225-232.10.1542/peds.103.SE1.225Search in Google Scholar
28. MartínezGarcía L, SanabriaAJ, García AlvarezE, et al.. The validity of recommendations from clinical guidelines: a survival analysis. CMAJ. 2014;186(16):1211-1219. doi:10.1503/cmaj.140547Search in Google Scholar PubMed PubMed Central
29. Management of Rotator Cuff Injuries Clinical Practice Guideline. Rosemont, IL: American Academy of Orthopaedic Surgeons; 2019.Search in Google Scholar
30. FischerF, LangeK, KloseK, GreinerW, KraemerA. Barriers and strategies in guideline implementation-a scoping review. Healthcare (Basel). 2016;4(3). doi:10.3390/healthcare4030036Search in Google Scholar PubMed PubMed Central
31. HarrisonMB, GrahamID, LorimerK, FriedbergE, PierscianowskiT, BrandysT. Leg-ulcer care in the community, before and after implementation of an evidence-based service. CMAJ. 2005;172(11):1447-1452. doi:10.1503/cmaj.1041441Search in Google Scholar PubMed PubMed Central
32. HarrisonMB, GrahamID, LorimerK, et al.. Nurse clinic versus home delivery of evidence-based community leg ulcer care: a randomized health services trial. BMC Health Serv Res. 2008;8:243. doi:10.1186/1472-6963-8-243Search in Google Scholar PubMed PubMed Central
33. ForsnerT, HanssonJ, BrommelsM, WistedtAA, ForsellY. Implementing clinical guidelines in psychiatry: a qualitative study of perceived facilitators and barriers. BMC Psychiatry. 2010;10:8. doi:10.1186/1471-244X-10-8Search in Google Scholar PubMed PubMed Central
34. BrouwersMC, KhoME, BrowmanGP, et al.. (2010) AGREE II: advancing guideline development, reporting, and evaluation in health care. Prev Med. 2010;51(5):421-424.10.1016/j.ypmed.2010.08.005Search in Google Scholar PubMed
© 2020 American Osteopathic Association
