Clinicians Agree Overall with Appropriateness Criteria, with Some Caveats

Download this article's Factoid (PDF & PPT for Gold Subscribers)

A small survey of practicing cardiologists reveals broad agreement with the American College of Cardiology (ACC) technical panel regarding the appropriateness of revascularization—categorized as appropriate, uncertain, or inappropriate—for a wide range of patient scenarios. But disagreement among the clinicians was common, as were differing assessments between individual cardiologists and the technical panel. The findings were published in the April 5, 2011, issue of the Journal of the American College of Cardiology.

In 2008—before publication of the ACC appropriate use criteria (AUC)—investigators led by Paul S. Chan, MD, MSc, of Saint Luke’s Mid America Heart and Vascular Institute (Kansas City, MO), asked practicing cardiologists to assess a representative sample of the 198 scenarios evaluated by the ACC technical panel in developing the AUC, relying on the same data. Eighty-five clinicians from 10 institutions completed the survey.

Overall, there was 84% concordance in appropriateness category assignments between the survey group and the AUC panel. However, within the survey group of practicing cardiologists, the rate of nonagreement was 66%.

Consensus on ‘Appropriate’ Indications

Specifically, for the 36 indications designated as appropriate by the panel, the survey group concurred 94% of the time. Moreover, the cardiologists’ ratings were largely unaffected by whether or not they were interventionalists, how long they had practiced, or how much of their time they devoted to research. However, there was greater rating variability among the survey group than in the panel (table 1).

Most disagreement centered on the uncertain and inappropriate designations. For the 22 uncertain indications, the panel and the survey group were in accord 73% of the time. Within the groups, rates of nonagreement were higher among the survey cardiologists than the panel. Similarly, for the 10 inappropriate indications, the survey group concurred with the panel 70% of the time, although non-interventional cardiologists were in complete agreement with the panel. There was substantially greater disagreement about the inappropriate designations among the survey group than among panel members, although interventionalists and physicians spending ≥ 10% of their time on research showed less disagreement than their counterparts in the survey group (table 1).

Table 1. Within-Group Disagreement Regarding Appropriateness Designations


Technical Panel
(n = 17)

Survey Group
(n = 85)

(n = 44)

(n = 41)





















In a multivariable model, the presence of a proximal LAD stenosis was associated with higher rates of physician nonagreement (RR 1.29; 95% CI 1.1-1.51; P = 0.001). By contrast, several other factors predicted less disagreement:

  • High-risk noninvasive study for ischemia (RR 0.51; 95% CI 0.40-0.65; P < 0.001)
  • Maximal intensity (2 or more agents) of anti-ischemic therapy (RR 0.75; 95% CI 0.69-0.82; P < 0.001)
  • Substantial (Canadian Cardiovascular Society [CCS] class III to IV) symptoms (RR 0.46; 95% CI 0.38-0.57; P < 0.001)

‘Disagreeable’ Clinicians

In addition, there was marked variability in appropriateness assignments between individual cardiologists and the panel, with certain clinicians not agreeing with the panel for any indication.

Remarkably, 1 in 4 physicians rated an indication for coronary revascularization as uncertain or inappropriate for 43% of indications categorized as appropriate by the AUC and rated an indication as uncertain or appropriate for 70% of the inappropriate indications.

Rates of disagreement were much lower for indications involving severe CCS class III to IV angina and high-risk noninvasive studies for ischemia, for which the benefits of revascularization are likely to be greater. On the other hand, significant LAD obstruction evoked more disagreement about the value of treatment.

The authors write, “We believe that it will be important to both measure and provide feedback to clinicians about the appropriateness of patients that they treat, while concurrently educating them about the AUC, if more uniform practice is to be achieved.”

Different Methods, Different Results

In an invited commentary, a group led by Manesh R. Patel, MD, of the Duke Clinical Research Institute (Durham, NC), argues that the variability should be interpreted in light of important differences between the survey method and the “rigorous AUC process.” For example, the latter included a deliberative step that was missing from the survey: after panel members offered their initial ratings, disputed scenarios were discussed, and panelists were encouraged to base decisions on the best available published evidence.

Gregg W. Stone, MD, of Columbia University Medical Center (New York, NY), agreed that the groups’ differing approaches to evaluating scenarios likely contributed to the discrepant results. The panel was more academically based and was tasked to spend more time examining the literature, he told TCTMD in a telephone interview.

On the other hand, he observed, “it’s possible that the practicing physicians see a lot more nuances in the [scenarios] and are not quite as prescriptive as the AUC group, understanding the art and not just the science of medicine.” In addition, he observed, the panel represents a very small sample of the cardiology community. “For something that carries the purported weight of appropriateness criteria, there should be much larger groups that come up with these recommendations,” he suggested. Moreover, “the panel’s methodology is relatively arbitrary. Just one point on an ordinal scale distinguishes an uncertain from inappropriate [designation].”

The Obligation of Self-Regulation

Dr. Patel and colleagues point out that the AUC were developed in a climate of concern over spiraling health-care costs and apparent overuse of costly procedures such as revascularization. “Our profession has the privilege of self-regulation, and this is our struggle,” they write. “Failure to accept this responsibility will only accelerate regulation by those whose motives may be different and who are not at the bedside with the patient.

“How can benchmarking based on AUC be rationally implemented and improved without impeding doctor-patient relationships and innovation?” the commentators ask. For one thing, the AUC must be regularly updated to remain current with the evidence, they say. And databases such as the National Cardiovascular Data Registry could be used to “highlight areas for which differences between real-life practice and the AUC may require modifications in clinical practice, the AUC, or both.

AUC Susceptible to Misuse

In an accompanying editorial, Paul T. Vaitkus, MD, of Midwest Heart Specialists (Rockford, IL), observes that the AUC carry the potential for “administrative applications” such as elimination of reimbursement for supposedly “unproven therapies” and physician profiling. In that light, the current study and other evidence “cast serious doubt that the AUC is ready for ‘prime time,’” he writes.

Certain “intrinsic methodologic limitations [of the AUC process] must be acknowledged,” he says. First, there is the risk that any consensus-seeking committee may engage in ‘groupthink.’ In addition, he notes, ‘appropriateness’ lacks an objective, quantifiable standard—precisely what is needed in the contentious ‘gray areas’ of practice.

Moreover, Dr. Vaitkus notes, the AUC revision is inherently slow, and physician profiling or reimbursement policies could “penalize physicians who either generate pioneering data or are early adapters.”

One prominent example of a potential ‘chilling effect’ is the current AUC ‘inappropriate’ designation for PCI for left main disease. “Do we Americans thus cede all future efforts at ‘pushing the envelope’ to our international colleagues?” Dr. Vaitkus asks.

Dr. Stone concurred that the AUC are vulnerable to misuse. “On the other hand, many studies have shown that hospitals that follow evidence-based medicine guidelines—and appropriateness criteria are a kind of offshoot of that—tend to have better outcomes. So it’s not a misguided effort. In fact, it would be very interesting to see how the appropriateness criteria track with outcomes.”


1. Chan PS, Brindis RG, Cohen DJ, et al. Concordance of physician rating with the appropriate use criteria for coronary revascularization. J Am Coll Cardiol. 2011;57:1546-1553.

2. Patel MR, Wolk MJ, Allen JM, et al. The privilege of self-regulation: The role of appropriate use criteria. J Am Coll Cardiol. 2011;57:1557-1559.

3. Vaitkus PT. Can we appropriately measure appropriateness? J Am Coll Cardiol. 2011;57:1554-1556.



Related Stories:

Clinicians Agree Overall with Appropriateness Criteria, with Some Caveats

A small survey of practicing cardiologists reveals broad agreement with the American College of Cardiology (ACC) technical panel regarding the appropriateness of revascularization—categorized as appropriate, uncertain, or inappropriate—for a wide range of patient scenarios. But disagreement among the clinicians was
  • Dr. Chan reports being supported by a Career Development Grant Award from the National Heart, Lung, and Blood Institute.
  • Dr. Patel reports no relevant conflicts of interest.
  • Dr. Stone reports serving on the advisory board for Abbott Vascular and Boston Scientific and serving as a consultant to The Medicines Company.
  • Dr. Vaitkus reports being an employee of Daiichi-Sankyo.