Download this article's Factoid (PDF & PPT for Gold Subscribers

Compared with standard coronary angiography, computed tomographic (CT) angiography generally delivers good diagnostic accuracy, but real-world results may vary widely from center to center, according to a study published online March 14, 2011, ahead of print in the Archives of Internal Medicine.

To better understand the performance of CT angiography ‘in the field,’ Canadian investigators led by Ron Goeree, MA, of St. Joseph’s Healthcare Hamilton (Hamilton, Canada), evaluated the accuracy of the technology compared with invasive angiography in 169 patients with low or intermediate pretest probability of obstructive CAD at 4 centers between September 2006 and June 2009.

The overall pretest likelihood for obstructive CAD was 46.8%. Patients were divided into 2 groups. Group 1 consisted of 52 patients, of whom 46 had valvular heart disease, 3 cardiomyopathy, 2 congenital heart disease, and 1 aortic disease; group 2 consisted of 117 symptomatic patients with intermediate pretest probability of CAD.

Overall Sensitivity, Specificity Strong

The overall prevalence of angiographic disease (≥ 50% by standard angiography) was 53%—21% in group 1 and 61% in group 2. In measures of accuracy, CT angiography showed:

• Sensitivity of 81.3% (95% CI 71.0%-89.1%)
• Specificity of 93.3% (95% CI 85.9%-97.5%)
• Positive predictive value of 91.6% (95% CI 82.5%-96.8%)
• Negative predictive value of 84.7% (95% CI 76.0%-91.2%)

Overall CT performance was similar in both groups as well as when using 50% and 70% thresholds of pretest probability of CAD.

Factors that increased the likelihood of false results—either false-positive or false-negative—were pretest probability of CAD (OR 1.02; P = 0.005) and the presence of coronary calcification (OR 1.09; P = 0.03). In addition, on multivariate analysis, the participating center was a predictor of false results (OR 0.28; P = 0.005).

Diagnostic accuracy varied widely across the 4 participating centers, with sensitivity ranging from 50.0% to 93.2%, specificity from 92.0% to 100.0%, positive predictive value from 84.6% to 100%, and negative predictive value from 42.9% to 94.7%.

In particular, center 1 clearly outperformed the other 3 (P < 0.001). The greatest discrepancies were seen in sensitivity and negative predictive value (table 1).

Table 1. CT Angiography Accuracy at Detecting ≥ 50% Stenosis

Abbreviations: PPV, positive predictive value; NPV, negative predictive value.

Patient demographics and CT parameters varied among the centers, including pretest likelihood of CAD (P = 0.003), prevalence of CAD (P = 0.005), smoking status (P = 0.03), and contrast infusion rate, a factor influencing image quality (P < 0.001).

In light of potential bias in patient characteristics and the presumed high negative predictive value of CT angiography, sensitivity analyses were performed in patients at 2 pretest probabilities of CAD. Center 1 performed better than the other 3 even when considering these factors, whether using the 50% or 70% threshold for stenosis.

In addition, disagreements between the imaging modalities did not appear to be explained by interobserver variability, since agreement between 2 image readers was similar for CT (0.88; 95% CI 0.81-0.94) and invasive angiography (0.81; 95% CI 0.75-0.88), and third readers were required to resolve differences in similar proportions of cases (40.8% and 38.5%, respectively).

“Enthusiasm for [CT angiography] must be tempered by the reality that centers may have different patient cohorts, acquisition protocols, expertise, and interpretation thresholds. There is a need to develop standardized measures for [CT angiography] acquisition and interpretation to ensure optimal patient diagnosis and care,” the authors write.

Results Distorted by Methodology?

In an invited commentary, George A. Diamond, MD, and Sanjay Kaul, MD, both of Cedars-Sinai Medical Center (Los Angeles, CA), argue that the reported accuracy of CT testing was distorted by the study protocol.

The most important problem, they say, follows from the fact that “test performance is conditioned on the overall prevalence of abnormal responses in the patients undergoing testing: those in whom disease status is verified and those in whom it is not. As a result, the observed sensitivity and specificity are conditioned on the magnitude of bias introduced by the process of verification.”

Drs. Diamond and Kaul explain that not only did the study not correct for such bias but it failed to even assess its magnitude by reporting the proportion of abnormal test results observed in the 425 candidates who were not referred for invasive coronary angiography.

Outdated Technique

In a telephone interview with TCTMD, Matthew J. Budoff, MD, of Harbor-UCLA Medical Center (Torrance, CA), noted that the study was initiated early in the history of cardiac CT. “Almost all of the imaging studies were done in a way we don’t even do anymore,” he said. 

The retrospective scanning used in this study is very outdated, confirmed Michael Poon, MD, of Stony Brook University Medical Center (Stony Brook, NY). Use of the technique “usually reflects centers that are not very experienced,” he told TCTMD in a telephone interview. “[These operators] don’t know how to select patients or prepare them. Just by looking at this protocol, I know they were amateur sites.”

In part for this reason, he took issue with the commentators’ implication that verification bias played a key role in the CT results. “I think they missed the point,” Dr. Poon said. “They think the results were skewed because of the patients excluded. I think the exclusions were reasonable and the results are accurate, but the center selection was skewed.”

Nonetheless, the finding of considerable variability in CT diagnostic accuracy across centers is probably correct, Dr. Poon said, adding, “The specificity and sensitivity are all over the place. I guess that’s what they mean by the ‘real world.’”

“I think this type of field evaluation is valuable because this is what the public [encounters],” Dr. Poon commented. “Unless you go to centers where CT is performed by experts, you’re not going to get the kind of results that you see in the literature.”

Dr. Budoff said one lesson of such variability is that cardiac CT involves a learning curve. “There are good readers and bad readers, and we have to continue to work toward getting all interpreting physicians up to speed,” he said, citing multiple training initiatives by the Society of Cardiovascular Computed Tomography (SCCT).

Although scanners and workstations have improved considerably over the past few years, Dr. Budoff noted, “my guess is that the variability seen here was due to the fact that the interpreting physicians were somewhat limited rather than to the equipment itself.”

However, “we will see improvements over time,” he predicted. “In fact, if you repeated the same study today, patient selection would be a little better because the doctors would have learned from their mistakes, and the [image] reading would be better.” 

Study Details 

Retrospective ECG-gated datasets were acquired with either the GE Volume CT Scanner (GE Healthcare, Milwaukee, WI) or the Aquilion 64 multidetector CT scanner (Toshiba Medical Systems, Tochigi, Japan).

2. Diamond GA, Kaul S. Gone fishing! On the “real-world” accuracy of computed tomographic coronary angiography [commentary]. Arch Intern Med. 2011;Epub ahead of print.

Related Stories:

• Updated Consensus Statement Outlines Appropriate Cardiac CT Use
• CT Angiography Rules Out Stenosis in Patients at Low-Intermediate CAD Risk
• CTA Reliably Rules Out CAD, Specificity Still an Issue