ACC/AHA guidelines largely based on weak evidence and expert opinion

February 24, 2009 | Michael O'Riordan (Clinical cardiology)

Durham, NC - The number of recommendations in the American Heart Association/American College of Cardiology (AHA/ACC) clinical-practice guidelines have progressively increased over time, but these recommendations are often based on weak evidence and expert opinion, a new study, published in the February 25, 2009 issue of the Journal of the American Medical Association, has shown [1].

Among guidelines reporting an update in recent years, the number of recommendations has increased 48% from the first to the current version, with the largest increase in recommendations for which there is conflicting evidence and/or a divergence of opinion about the efficacy of a procedure or treatment (class 2). In addition, roughly half of all recommendations are based on expert opinion, case studies, or standard of care rather than on data from multiple clinical trials or meta-analyses.

"It's a sobering result," lead investigator Dr Pierluigi Tricoci (Duke Clinical Research Institute, Durham, NC) told heartwire. "Almost half of the recommendations have a level of evidence C, meaning they have no supporting evidence or just a little supporting evidence. This means that situations where we are sure of what to do are small in number, while the situations where we are not so sure of what to do are the majority."

Guidelines changing over the years

In this study, the investigators wanted to assess how the guidelines changed since the AHA/ACC began publishing clinical-practice guidelines more than 20 years ago. The guidelines currently use a grading system based on the level of evidence and class of recommendation. The level of evidence—A, B, or C—includes a description of the existence and types of studies available supporting the recommendation and expert consensus. The class of recommendation—1, 2, 2a, 2b, and 3—indicates the strength of the recommendation based on an objective judgment about the relative merits of the data.

This means that situations where we are sure of what to do are small in number, while the situations where we are not so sure of what to do are the majority.

From 1984 to 2008, the ACC/AHA issued 53 guidelines on 22 topics for a total of 7196 recommendations. Among these guidelines, 24 were disease-based, 15 were for interventional procedures, and 14 were for diagnostics. The disease-based and interventional-based guidelines were updated approximately every five years, while the diagnostic-procedure-based guidelines were updated eight years after the last publication.

Among guidelines with at least one revision, the number of recommendations increased 48% from the first guidelines to the most recent version. The increase in recommendations was driven primarily by increases in the number of class 2 recommendations—those with conflicting evidence and/or divergent opinions. Of the 16 current guidelines reporting levels of evidence, just 11% of recommendations were classified as evidence A, those based on data from multiple clinical trials or meta-analyses. Just under half of the recommendations, 48%, were based on expert opinion, case studies, or standards of care (level of evidence C).

Among the class 1 recommendations of guidelines reporting level of evidence, just 19% had a level of evidence A. There was also wide variation across the different specialties, with the disease-based and interventional-based guidelines having slightly better levels of evidence than the diagnostic-procedure guidelines. Roughly 70% of the recommendations within the valvular-heart-disease guidelines are evidence level C, with just one recommendation with evidence based on clinical trials. Heart-failure, secondary-prevention, and unstable-angina guidelines had the most recommendations based on clinical trial data or meta-analyses, or evidence level A.

"Some fields are in better shape than others, but this is a general problem afflicting all the fields of cardiology," said Tricoci.

Ignore the guidelines altogether

In an editorial accompanying the published study [2], Drs Terrence Shaneyfelt and Robert Centor (University of Alabama School of Medicine, Birmingham) write that the "overreliance on expert opinion in guidelines is problematic," particularly since the guideline committees come to the table with implicit biases, values, and goals that aren't typically disclosed. Moreover, too many current guidelines are marketing- and opinion-based pieces, "delivering directive rather than assistive statements," they add.  In addition, they argue that guidelines are not patient-specific enough to be useful and do not allow for individualization of care, instead adopting a one-size-fits-all mentality. Moreover, there are simply too many guidelines on the same topic. The editorialists write that if the guidelines continue to exist, they need to undergo major changes, including limiting committee members with financial ties to industry or other potential conflicts of interest. Guideline development should also be centralized under a governing body to reduce bias and redundancy, something that might be achieved by allowing the US Department of Health and Human Services to oversee their drafting.

"However, it seems unlikely that substantial change will occur because many guideline developers seem set in their ways," write Shaneyfelt and Centor. "If all that can be produced are biased, minimally applicable consensus statements, perhaps guidelines should be avoided completely. Unless there is evidence of appropriate changes in the guideline process, clinicians and policy makers must reject calls for adherence to guidelines." Speaking with heartwire, Tricoci, rather than fault the guideline committees, said the findings represent a failure of the current research system. With clinical trials performed predominantly by industry, unaddressed clinical questions are overlooked, as these companies are interested primarily in bringing new products to market.

"While it's certainly great to have newer and newer products, there are a lot of things that we don't know about the products that we currently use or even decisions that do not necessarily involve products, such as treatment strategies or timing issues," said Tricoci. "There is not a lot of funding supporting this type of research."

In addition, clinical trials are costly and complex and require long periods of time to complete, said Tricoci. This limits the funding available for the study, as well as the number of questions that can be answered by the trialists. Streamlining clinical trials that focus on areas with deficient research would partly alleviate the knowledge gaps in the clinical guidelines.