Background articleCurrent methods of the U.S. Preventive Services Task Force: A review of the process
Introduction
The U.S. Preventive Services Task Force (Task Force/USPSTF)represents one of several efforts by governments and national organizations to take a more evidence-based approach to the development of clinical practice guidelines. Guidelines developed by an evidence-based approach tend to be based on conclusions supported more by scientific evidence than by expert opinion.1 Efforts are made to link the strength of recommendations to the quality of evidence; to make that linkage transparent and explicit, and to ensure that the review of evidence is comprehensive, objective, and attentive to quality.2
Methods for reviewing the evidence have matured over the years as groups have gained experience in developing evidence-based guidelines. Systematic searches of multiple bibliographic research databases help ensure thorough and unbiased identification of the relevant literature. Predetermined selection criteria minimize bias and improve the efficiency of reviewing that literature. Quality criteria developed by methodologists guide judgments of weaknesses and strengths of individual research studies. Frameworks and models explicitly define methods for rating and integrating multiple pieces of heterogeneous evidence.3
Methods for linking evidence and recommendations have also matured.4 Initially the recommendations of the USPSTF and other evidence-based groups were strongly correlated with the research design of the most important studies. An A recommendation, for example, usually meant that use of the preventive service was supported by a randomized controlled trial (RCT).5, 6 Guideline developers now understand the need to consider the evidence as a whole, including the trade-offs among benefits, harms, and costs and the net benefit relative to other health care needs for optimal resource allocation.7
In the case of prevention, moreover, special scientific and policy considerations apply in reviewing evidence and setting policy. Preventive services require a distinctive logic in considering, for example, the incremental benefit of early detection or the ability of counselors to motivate behavior change. Because the populations affected by preventive care recommendations are often large and have no recognized symptoms or signs of the target condition, harms incurred by even a small percentage can affect a large number of people. Thus, the potential for doing greater harm than good must be taken seriously.
In the context of these methodologic advances and with an awareness of the many unresolved issues for which sound methods are lacking, the third Task Force formed a methods subcommittee (Methods Work Group). It comprises members of the Task Force, representatives of the Canadian Task Force on Preventive Health Care, staff of the two Evidence-based Practice Centers (EPCs) that support the Task Force, and staff of the Agency for Healthcare Research and Quality (AHRQ). The mission of the Work Group is to revisit methods used by previous U.S. Preventive Services Task Forces, to develop more sophisticated methods to be used in current work, and to understand better the theoretical considerations for problems that lack easy answers.
The discussions of this group and subsequent discussions by the entire Task Force have led to several modifications of Task Force methods and identified areas that need further examination. This article describes the methods in current use by the third USPSTF. As the Task Force identifies better ways to do its work, the Methods Work Group will explore additional revisions and refinements to its methods.
We discuss these changes in the sequence of steps of recommendation development: scope and selection of topics, review of the evidence, assessing the magnitude of net benefit, extrapolation and generalization, translating evidence into recommendations, drafting the report, and external review.
Section snippets
Scope
In defining its scope of interest, the Task Force must consider types of services, populations of patients and providers, and sites for which its recommendations are intended. Clarifying these definitions has both methodologic and practical importance. Resource limitations make it impossible for the Task Force to review evidence for all services that prevent disease; the project must, therefore, set boundaries.
The third Task Force has retained the previous policy of focusing on screening tests,
Intensity
Current methods for conducting systematic reviews emphasize a comprehensive literature search and evaluation and detailed documentation of methods and findings.10 An advantage of this approach is that it avoids the tendency of some guideline panels to cite evidence selectively in support of their recommendations. This approach also enables others outside the process to understand, judge, and replicate the interpretation of the evidence. The disadvantage of this approach is that it produces
Assessing magnitude of net benefit
When the overall quality of the evidence is judged to be good or fair, the Task Force proceeds to consider the magnitude of net benefit to be expected from implementation of the preventive service. Determining net benefit requires assessing both the magnitude of benefits and the magnitude of harms and weighing the two. When the evidence is considered to be poor, the Task Force has no scientific basis for making conjectures about magnitude.
The Task Force classifies benefits, harms, and net
Extrapolation and generalization
As noted in the “Review of the Evidence” section, the Task Force regularly faces the issue of generalization in determining the quality of evidence. The Task Force makes recommendations intended for the general primary care situation; for this purpose, high-quality evidence is evidence that is relevant and valid for this setting. When studies examine different situations and settings, the issue of generalization arises.
Likewise, the magnitude of the effect of interest to the Task Force is that
General principles
Making recommendations for clinical practice involves considerations that extend beyond scientific evidence. Direct scientific evidence is of pre-eminent interest, but such issues as cost effectiveness, resource prioritization, logistical factors, ethical and legal concerns, and patient and societal expectations should also be considered.
Historically, the Task Force has taken a conservative, evidence-based approach to this process, making recommendations that reflect primarily the state of the
Drafting the report
In its earliest days, background papers and recommendations of the Task Force were written by individual panel members assigned to those topics. In later years, they were written by staff with close oversight by the Task Force. In time a sharp demarcation has evolved between descriptions of the evidence and recommendations.
Thus, for the third Task Force, topic teams led by EPC staff write systematic evidence reviews. These reviews define the strengths and limits of the evidence but stop short
External review
Before the Task Force makes its final determinations about recommendations on a given preventive service, the EPC and AHRQ send a draft systematic evidence review to four to six external experts and to federal agencies and professional and disease-based health organizations with interests in the topic. They ask the experts to examine the review critically for accuracy and completeness and to respond to a series of specific questions about the document. After assembling these external review
Conclusion
Methods for making evidence-based practice policies are evolving. At one extreme, guidelines panels could insist on direct evidence or point to any information gaps to justify a negative recommendation for almost any service. Such an approach would result in positive recommendations only for services that had a very narrow confidence interval for net benefit, but many effective services would not be recommended. At the other extreme, guideline groups that accept incomplete data and allow easy
Acknowledgements
This paper was developed by the Research Triangle Institute–University of North Carolina at Chapel Hill (RTI-UNC) and the Oregon Health Sciences University (OHSU) Evidence-Based Practice Centers under contracts from the Agency for Healthcare Research and Quality (contract nos. 290-97-0011 and 290-97-0018, respectively). We acknowledge the assistance of Jacqueline Besteman, JD, MA, EPC Program Officer; the AHRQ staff working with the third Task Force; and the staffs of the EPCs at RTI-UNC and at
References (35)
- et al.
Evidence-based medicineinterpreting studies and setting policy
Hematol Oncol Clin N Amer
(2000) - et al.
Assessing “best evidence”issues in grading the quality of studies for systematic reviews
J Qual Improv
(1999) - et al.
Problems in the “evidence” of “evidence-based medicine
” Am J Med
(1997) - et al.
Differing perspectives on preventive care guidelinesa new look at the mammography controversy
Am J Prev Med
(1999) - Field MJ, Lohr KN, eds. Guidelines for clinical practice: from development to use. Washington, DC: National Academy...
- Mulrow CD, Cook D, eds. Systematic reviews: synthesis of best evidence for health care decisions. Philadelphia:...
- et al.
The trials and tribulations of clinical practice guidelines
JAMA
(1999) - et al.
Report of the U.S. Preventive Services Task Force
JAMA
(1990) - U.S. Preventive Services Task Force. Guide to clinical preventive services: report of the U.S. Preventive Services Task...
- Eddy DM. Clinical decision making: from theory to practice. A collection of essays from JAMA. Boston: Jones and...
Developing evidence-based clinical practice guidelineslessons learned by the U.S. Preventive Services Task Force
Ann Rev Public Health
Cited by (0)
- 1
Other members of the Methods Work Group include: Alfred O. Berg, MD, MPH, University of Washington School of Medicine; Karen B. Eden, PhD, Oregon Health Sciences University; John Feightner, MD, MSc, FCFP, University of Western Ontario–Parkwood Hospital; Susan Mahon, MPH, Oregon Health Sciences University; and Michael Pignone, MD, MPH, University of North Carolina School of Medicine.
- 2
Reprints are available from the AHRQ Web site at www.ahrq.gov/clinic/uspstfix.htm, through the National Guideline Clearinghouse (www.guideline.gov), or in print through the AHRQ Publications Clearinghouse (1-800-358-9295).