Free
Special Communication  |   August 2007
Evidence-Based Medicine, Part 6. An Introduction to Critical Appraisal of Clinical Practice Guidelines
Brent W. Sanderlin, DO; Nashila AbdulRahim, OMS IV
Author Notes
  • From the Department of Family Medicine at the University of North Texas Health Science Center—Texas College of Osteopathic Medicine in Fort Worth. 
  • Address correspondence to Brent W. Sanderlin, DO, Department of Family Medicine, Texas College of Osteopathic Medicine, University of North Texas Health Science Center, 3400 Camp Bowie Blvd, Fort Worth, TX 76107-2729.E-mail: bsanderl@hsc.unt.edu 
Article Information
Evidence-Based Medicine
Special Communication   |   August 2007
Evidence-Based Medicine, Part 6. An Introduction to Critical Appraisal of Clinical Practice Guidelines
The Journal of the American Osteopathic Association, August 2007, Vol. 107, 321-324. doi:https://doi.org/10.7556/jaoa.2007.107.8.321
The Journal of the American Osteopathic Association, August 2007, Vol. 107, 321-324. doi:https://doi.org/10.7556/jaoa.2007.107.8.321
Abstract

This article provides an introductory step-by-step process to appraise clinical practice guidelines. The authors introduce these principles using a systematic approach and case-based format. The process of assessing the validity of clinical practice guidelines, determining their importance, and applying them to an individual patient is reviewed. The concepts of study population homogeneity, equal treatment, and study completeness are discussed to help physicians determine the validity of clinical practice guidelines. Finally, information that is learned from the previously mentioned steps is applied to patient care. Study generalizability and the role of patient values, expectations, and concerns are also addressed. The skills learned from appraising clinical practice guidelines in the manner outlined provides a solid basis for life-long learning and improved patient care.

Physicians, who use their clinical skills to treat patients and diagnose diseases on a daily basis, must be informed of current clinical practice guidelines and standards of care in order to provide their patients with the best care possible. However, because such guidelines may have different recommendations depending on who wrote or sponsored the guideline, they may not be applicable to any given patient. Therefore, physicians must remain cognizant of the rationale used to develop such guidelines—in particular the evidence base used—while at the same time assessing the guidelines' validity and applicability. One such tool to develop clinicians' skills in evaluating guidelines is evidence-based medicine (EBM). 
In this article, we introduce a strategy for busy physicians, physician residents, and medical students to critically assess clinical practice guidelines. In-depth details of research methods are beyond the scope of this introductory series on EBM. Readers are encouraged to seek further training on these topics with supplemental learning opportunities and continuing medical education. Finally, the clinical scenario described has been simplified to provide readers with an illustrative example for the general concepts introduced. 
Searching the Evidence
The initial step required for a critical review of clinical practice guidelines involves evaluating the document's stated purpose (eg, diagnosis, prevention, symptom control). Secondarily, physicians practicing EBM are concerned with evaluating how closely the guideline's purpose is aligned with a particular patient's needs (Figure 1). 
Guidelines are valuable insofar as they provide clear and practical recommendations for a clinical challenge. A guideline, to prove relevant, must be supported by evidence that assesses the risks, benefits, and costs of the proposed intervention. A guideline should also explain how its recommendations can be implemented, regularly reviewed, and updated to ensure accurate and up-to-date information is provided. 
Validity of Clinical Practice Guidelines
There are a number of items to take into consideration when appraising a clinical practice guideline. Greenhalgh1 suggests 10 questions to use when reviewing a guideline (Figure 2). For example, Greenhalgh asks, “Did the preparation and publication of this guideline involve a significant conflict of interest?” If the guideline developers received financial support from a pharmaceutical company that manufactures a medication recommended in the guideline, those guideline authors may be unconsciously (or consciously) biased with their specified recommendations. Similar to how authors of articles published in medical journals are required to disclose financial relationships,2,3 members of guideline development teams should divulge any special interests they have related to the topic of the clinical practice guideline. 
Figure 1.
Clinical scenario.
Clinical scenario.
View Original  |  Slide (.ppt)
Figure 1.
Clinical scenario.
Clinical scenario.
View Original  |  Slide (.ppt)
Although conflict-of-interest statements are often absent from guidelines,4 a good guideline will disclose any potential bias so physicians can better determine the strength of the guidelines. If a physician finds that there is a significant conflict of interest in either the preparation or publication of the guideline, he or she should begin a new search for recommendations. 
Figure 2.
Questions practitioners should use when appraising a clinical practice guideline. 4
Questions practitioners should use when appraising a clinical practice guideline. 4
View Original  |  Slide (.ppt)
Figure 2.
Questions practitioners should use when appraising a clinical practice guideline. 4
Questions practitioners should use when appraising a clinical practice guideline. 4
View Original  |  Slide (.ppt)
Additional questions that might be asked by a physician practicing EBM include the following: 
  • How strong is the evidence used to support the clinical practice guidelines?
    The quality or “strength” of clinical practice guidelines can vary significantly as a result of the evidence used in their development. Such sources can range from well-designed randomized controlled trials (RCTs) to expert opinions.5 The strongest evidence, according to the US Preventive Services Task Force,6 comes from well-designed experimental studies, such as RCTs. And yet, guideline authors can undermine the value of their work by including in their evidence base RCTs that fail to account for a small sample size, patients lost to follow-up, possible biases, and results that cannot be replicated in other trials. Where possible, physicians should critically review the individual studies used by guideline authors in making their recommendations for standards of care.
    Avoiding adverse events or high costs by using inexpensive, low-risk treatments can strengthen clinical practice guideline recommendations. Such guidelines provide alternative choices that may be effective yet inexpensive with less risk to the patient for adverse effects. For example, the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure7 recommends the use of diuretics, which tend to be safer and less expensive than many alternatives.
  • Are the guidelines objective?
    Outside involvement of a researcher with appropriate and unbiased analytical skills, in addition to expert validation, provides a more objective methodology for guideline development. Development of such guidelines must also take ethical, cultural, social, and personal values as well as societal views into account. Clinical practice guidelines should be based on all evidence related to the guideline's topic; basing recommendations solely on expert opinion is subjective and possibly misleading. The appraisal processes for specific types of studies have been reviewed in detail in other articles associated with this series.8-11
  • Are the methods of the studies used homogenous?
    Not surprisingly, the validity of studies' results determines the quality of the guideline's recommendations. However, differences in population characteristics, clinical settings, methods of administering interventions, and follow-up rates can lead to somewhat undesirable heterogeneous study results—which may then, in turn, lead to less valid guideline recommendations.12 Clinical practice guidelines that are based on several studies with homogenous methods and complementary results are stronger (Figure 3).
    A summary of the grades of guideline recommendations based on study design is available in Figure 4.5
  • Are the study subjects different in any significant way from your specific patient?
    The findings of a valid, important study are considered generalizable when patients to be treated are similar to those described in the study. Similarly, recommendations made in clinical practice guidelines cannot be generalized if the data or evidence on which they are based is derived from a population that significantly differs from physicians' patients. Such information can be accessed by reviewing the population characteristics of the studies used to create clinical practice guidelines.
  • Are the guideline's recommendations an accurate reflection of the appraised evidence?
    Final recommendations of clinical practice guidelines should be an accurate reflection of the appraised evidence. That is, the recommendations should be weighted on the evidence that has been critically scrutinized and appraised. More importantly, these valid recommendations must be amenable to medical practice variants and the compromises associated with realworld practice (eg, guidelines should be flexible in various clinical settings and relevant to clinical practice).
Figure 3.
Clinical scenario (continued).
Clinical scenario (continued).
View Original  |  Slide (.ppt)
Figure 3.
Clinical scenario (continued).
Clinical scenario (continued).
View Original  |  Slide (.ppt)
Figure 4.
Ratings of guideline recommendations based on the study designs of articles used to support the guideline's findings. Abbreviations: CIs, confidence intervals; NNT, number needed to treat; RCTs, randomized controlled trials. Adapted from Wilson MC et al. JAMA. 1995;274:1630-1632.1 Copyright 1995, American Medical Association. All rights reserved.
Ratings of guideline recommendations based on the study designs of articles used to support the guideline's findings. Abbreviations: CIs, confidence intervals; NNT, number needed to treat; RCTs, randomized controlled trials. Adapted from Wilson MC et al. JAMA. 1995;274:1630-1632.1 Copyright 1995, American Medical Association. All rights reserved.
View Original  |  Slide (.ppt)
Figure 4.
Ratings of guideline recommendations based on the study designs of articles used to support the guideline's findings. Abbreviations: CIs, confidence intervals; NNT, number needed to treat; RCTs, randomized controlled trials. Adapted from Wilson MC et al. JAMA. 1995;274:1630-1632.1 Copyright 1995, American Medical Association. All rights reserved.
Ratings of guideline recommendations based on the study designs of articles used to support the guideline's findings. Abbreviations: CIs, confidence intervals; NNT, number needed to treat; RCTs, randomized controlled trials. Adapted from Wilson MC et al. JAMA. 1995;274:1630-1632.1 Copyright 1995, American Medical Association. All rights reserved.
View Original  |  Slide (.ppt)
Figure 5.
Clinical scenario (continued).
Clinical scenario (continued).
View Original  |  Slide (.ppt)
Figure 5.
Clinical scenario (continued).
Clinical scenario (continued).
View Original  |  Slide (.ppt)
Practical Use
A guideline's generalizability is directly related to its inclusion and exclusion criteria (eg, guidelines based on more stringent criteria are less generalizable to a population).5 In addition, applicability of clinical practice guidelines must consider the burden of disease, the patient's values and unique circumstances, and any practical barriers to guideline implementation. If the burden of disease is very low, any intervention's risk-to-benefit ratio becomes less advantageous to the patient.12 
Patients and physicians must work together to ensure both parties are making informed decisions. Insurance restrictions, availability of interventions, and patient factors and limitations (eg, financial, logistic) are common practical barriers to guideline implementation. All of these factors must be addressed to ensure a guideline is applicable to any given patient (Figure 5). However, these factors do not affect the basic evidence or scientific validity of a guideline.13 
Conclusion
Although most clinicians are already incorporating EBM principles in their practices, often instinctively, some physicians may require a more organized approach to integrating this relatively new model of self-education. Improved comfort levels and true expertise in the practice of EBM are the result of additional education, repetition, and self-assessment. The principles of EBM allow physicians to stay informed while also improving the quality of the information communicated to patients during patient encounters. The systematic approach that is used to appraise clinical practice guidelines is but one step in practicing EBM. Remember, the goal is always to provide the best care possible to patients—using one's clinical expertise to address patient values and expectations for treatment. 
  [Editor's note: This article is part 6 of a six-article series intended to introduce the principles of evidence-based medicine (EBM) to busy clinicians, physician residents, and medical students. Because the application of EBM is a career-long process, further training is needed beyond the information provided within this article and series. A foundation of knowledge about research methods is critical in understanding EBM; however, such details, though introduced, are beyond the scope of this series.]
 
Greenhalgh T. How to Read a Paper: The Basics of Evidence-Based Medicine. 3rd ed. Malden, Mass: Blackwell Publishing Ltd; 2006.
Uniform Requirements for Manuscripts Submitted to Biomedical Journals: Writing and Editing for Biomedical Publication. February 2006. International Committee of Medical Journal Editors Web site. Available at: http://www.icmje.org/. Accessed August 15, 2007.
Ethical and legal considerations. In: Iverson C, ed. AMA Manual of Style: A Guide for Authors and Editors. 10th ed. New York, NY: Oxford University Press, Inc; 2007.
Taylor R, Giles J. Cash interests taint drug advice. Nature. 2005;437:1070-1071.
Wilson MC, Hayward RS, Tunis SR, Bass EB, Guyatt G, for the Evidence-Based Medicine Working Group. Users' guides to the medical literature. VIII. B. How to use clinical guidelines. What are the recommendations and will they help you in caring for your patients? JAMA. 1995;274:1630-1632.
Agency for Healthcare Research and Quality. United States Preventive Services Task Force. Available at: http://www.ahrq.gov/clinic/uspstfix.htm. Accessed August 12, 2007.
Chobanian AV, Bakris GL, Black HR. The Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure: the JNC 7 report [published correction appears in JAMA. 2003;290:197]. JAMA. 2003;289:2560-2573.
Cardarelli R, Virgilio RF, Taylor L. Evidence-based medicine, part 2. An introduction to critical appraisal of articles on therapy. J Am Osteopath Assoc. 2007;107:299-303. Available at: http://www.jaoa.org/cgi/content/full/107/8/299.
Schranz DA, Dunn MA. Evidence-based medicine, part 3. An introduction to critical appraisal of articles on diagnosis. J Am Osteopath Assoc. 2007;107:304-309. Available at: http://www.jaoa.org/cgi/content/full/107/8/304.
Cardarelli R, Seater MM. Evidence-based medicine, part 4. An introduction to critical appraisal of articles on harm. J Am Osteopath Assoc. 2007;107:310-314. Available at: http://www.jaoa.org/cgi/content/full/107/8/310.
Cardarelli R, Oberdorfer JR. Evidence-based medicine, part 5. An introduction to critical appraisal of articles on prognosis. J Am Osteopath Assoc. 2007;107:315-319. Available at: http://www.jaoa.org/cgi/content/full/107/8/315.
Guyatt G, Sackett DL, Sinclair JC, Hayward R, Cook DJ, Cook RJ, for the Evidence-Based Medicine Working Group. Users' guide to the medical literature. IX. A method for grading health care recommendations. JAMA. 1995;274:1800-1804.
Straus SE, Richardson WS, Glasziou P, Haynes RB. Evidence-Based Medicine: How to Practice and Teach EBM. 3rd ed. London: Churchill Livingstone;2005 .
Figure 1.
Clinical scenario.
Clinical scenario.
View Original  |  Slide (.ppt)
Figure 1.
Clinical scenario.
Clinical scenario.
View Original  |  Slide (.ppt)
Figure 2.
Questions practitioners should use when appraising a clinical practice guideline. 4
Questions practitioners should use when appraising a clinical practice guideline. 4
View Original  |  Slide (.ppt)
Figure 2.
Questions practitioners should use when appraising a clinical practice guideline. 4
Questions practitioners should use when appraising a clinical practice guideline. 4
View Original  |  Slide (.ppt)
Figure 3.
Clinical scenario (continued).
Clinical scenario (continued).
View Original  |  Slide (.ppt)
Figure 3.
Clinical scenario (continued).
Clinical scenario (continued).
View Original  |  Slide (.ppt)
Figure 4.
Ratings of guideline recommendations based on the study designs of articles used to support the guideline's findings. Abbreviations: CIs, confidence intervals; NNT, number needed to treat; RCTs, randomized controlled trials. Adapted from Wilson MC et al. JAMA. 1995;274:1630-1632.1 Copyright 1995, American Medical Association. All rights reserved.
Ratings of guideline recommendations based on the study designs of articles used to support the guideline's findings. Abbreviations: CIs, confidence intervals; NNT, number needed to treat; RCTs, randomized controlled trials. Adapted from Wilson MC et al. JAMA. 1995;274:1630-1632.1 Copyright 1995, American Medical Association. All rights reserved.
View Original  |  Slide (.ppt)
Figure 4.
Ratings of guideline recommendations based on the study designs of articles used to support the guideline's findings. Abbreviations: CIs, confidence intervals; NNT, number needed to treat; RCTs, randomized controlled trials. Adapted from Wilson MC et al. JAMA. 1995;274:1630-1632.1 Copyright 1995, American Medical Association. All rights reserved.
Ratings of guideline recommendations based on the study designs of articles used to support the guideline's findings. Abbreviations: CIs, confidence intervals; NNT, number needed to treat; RCTs, randomized controlled trials. Adapted from Wilson MC et al. JAMA. 1995;274:1630-1632.1 Copyright 1995, American Medical Association. All rights reserved.
View Original  |  Slide (.ppt)
Figure 5.
Clinical scenario (continued).
Clinical scenario (continued).
View Original  |  Slide (.ppt)
Figure 5.
Clinical scenario (continued).
Clinical scenario (continued).
View Original  |  Slide (.ppt)