Prudence or Spin? Mulling Embolic Protection After PROTECTED TAVR

It’s easy to see why observers are confused by PROTECTED TAVR. Since the cerebral embolic protection (CEP) results were released at TCT 2022, there have been conflicting messages as to just how much you can take away from a trial that missed its primary endpoint, despite a powerful instinct to “do something” to mitigate stroke risk during TAVI.

The primary investigator himself took a rosy view based on an apparent reduction in disabling strokes, as did Sentinel device maker Boston Scientific. But others saw that as spin, cautioning against reading too much into a secondary endpoint when there was no significant difference in all strokes in the first 72 hours after the procedure—the study’s primary endpoint. The paper in the New England Journal of Medicine muddled things further with a conclusion that left open the possibility of a benefit based on the 95% confidence interval.

Samir Kapadia, MD (Cleveland Clinic, OH), global principal investigator for the trial, acknowledges that a straightforward recounting of PROTECTED TAVR—a miss, pure and simple—would have been much easier and less controversial. But it would have been a disservice to clinicians and patients to state that CEP doesn’t work, he argued, telling TCTMD that he debated how best to balance his interpretation of the results as both a clinician and a scientist.

Considering information from PROTECTED TAVR and other sources like national registries and the Cleveland Clinic’s own data, Kapadia said he is persuaded that use of the Sentinel device reduces major or disabling strokes during TAVI. “That’s why I have interpreted the data with caution on my side as a clinician,” he said. “It is actually a conservative interpretation from the clinician’s side, and it is a little bit of an aggressive interpretation from the academic side.”

It is actually a conservative interpretation from the clinician’s side, and it is a little bit of an aggressive interpretation from the academic side. Samir Kapadia

Many remain unconvinced. Benoy Shah, MBBS, MD (Wessex Cardiac Centre, University Hospital Southampton, England), president of the British Heart Valve Society, said there was “a lot of disappointment” when the trial missed its primary endpoint because of how feared stroke is as a complication of the procedure by both physicians and patients. “People are obviously eager to see technology like this work,” he commented. “We all want it to work.”

But Shah joined the chorus of voices warning that it’s risky to read too much into a secondary endpoint, noting that PROTECTED TAVR was not powered to detect a difference in disabling stroke and the number of such events was small—eight in the Sentinel arm and 20 in the control arm.

Operators see all of the debris captured by the filter at the end of the procedure and think that it must have done some good for the patient, but “the intuitive benefit still has to be proven by data,” Shah said.

It’s “really strange” how some are latching onto the disabling stroke finding, he added.

“I don’t believe these are people that are doing it because they’re financially conflicted,” Shah said. “I think they’re doing it because they’re emotionally conflicted, meaning that they want to prevent stroke. They want this technology to work.”

Spinning the Results?

Shortly after their TCT release, Boston Scientific started touting the results of the neutral trial, emphasizing the 60% relative risk reduction in disabling stroke through 72 hours. That’s the first thing visitors to the company’s PROTECTED TAVR website will see. You have to scroll about halfway down the page before finding out that the primary endpoint, all stroke, was not met and that the reduction in disabling stroke was seen in a secondary analysis.

Davide Capodanno, MD, PhD (University of Catania, Italy), editor-in-chief of EuroIntervention, called the reduction in disabling strokes “interesting.”

“But at the same time we know that a secondary outcome that is positive in a trial that is neutral is at best hypothesis-generating,” he added. Although it makes sense that disabling strokes would be reduced, since they’re caused by large emboli, he agreed, the statistical power of the trial precludes any definitive conclusions for this endpoint.

“If we follow the rules of critical interpretation, we need to wait for a larger trial, which is, by the way, ongoing,” Capodanno said. BHF PROTECT-TAVI has a projected enrollment of 7,000 patients and a meta-analysis combining the results of that trial with those of PROTECTED TAVR is already planned.

If we follow the rules of critical interpretation, we need to wait for a larger trial. Davide Capodanno

Sanjay Kaul, MD (Cedars-Sinai Medical Center, Los Angeles, CA), said he wasn’t surprised by the results of the trial, since prior studies—including Sentinel IDE trial and REFLECT II, which evaluated another CEP device, TriGUARD 3 (Keystone Heart)—have not shown that protection reduces stroke risk.

Yet what was “somewhat surprising,” he commented, was the way the trial was interpreted by the investigators. “I think they went beyond the evidence.”

PROTECTED TAVR, said Kaul, was designed to detect a 2% absolute risk difference or a 50% relative risk reduction. As such, “appropriate interpretation” would be that “the entire 95% confidence interval excluded the effect size that the study was powered to detect, which would put it in the category of being a negative trial,” he explained.

And since the primary endpoint was not met, the secondary endpoint findings could be a false positive. Putting a lot of weight into the disabling stroke finding “doesn’t sit well with me,” Kaul said. “He could have been a little bit more cautious in his interpretation.”

Kapadia, during his TCT presentation, calculated that 125 patients would need to be treated to prevent one disabling stroke, and concluded that use of protection should be considered for all patients undergoing TAVI. “I would call [that] embellishing the data to the extreme,” Kaul said. “You really can’t draw that inference and therefore change clinical practice on the basis of this data.”

For David Cohen, MD (St. Francis Hospital and Heart Center, Roslyn, NY, and Cardiovascular Research Foundation, New York, NY), the discussion around PROTECTED TAVR over the last several weeks has been reasonable, with consideration that the trial was likely underpowered to detect smaller but clinically important differences in the primary endpoint. It was powered to detect a 50% relative reduction in stroke, which “is very optimistic for almost anything,” he said.

Though it’s understandable to want to focus on disabling stroke, a clearly important endpoint, Cohen said, “we have to temper our enthusiasm for that finding and that conclusion, because it was not prespecified as the primary endpoint and therefore it’s possible that we’re substantially overestimating that effect or finding an effect that isn’t actually there.”

I think they went beyond the evidence. Sanjay Kaul

Kapadia, when asked about some of these concerns, defended his interpretation, citing the need to balance the interests of clinicians, patients, statisticians, and trialists. He told TCTMD that the NEJM editors initially didn’t want the difference in disabling stroke mentioned in the abstract at all, but that they relented after Kapadia argued that it was not just a secondary endpoint but also a description of the primary endpoint.

What About That NEJM Conclusion?

Though a positive interpretation by investigators and sponsors may be understandable, some observers have also expressed surprise at the NEJM publication itself, and in particular the paper’s conclusion. Rather than simply stating that the trial failed to demonstrate a reduction in all stroke with embolic protection, the conclusion also states that “on the basis of the 95% confidence interval around this outcome, the results may not rule out a benefit of CEP during TAVR.” The absolute rate of periprocedural stroke was 0.6% lower in the protection arm, with a confidence interval ranging from -1.7% to 0.5%.

Megan Coylewright, MD (Erlanger Health System, Chattanooga, TN), editor of the American College of Cardiology (ACC)’s CardioSmart, said she’s never before seen a conclusion like the one on the PROTECTED TAVR paper.

“The overall trial results did not show a clinical benefit with regards to reduction of stroke and the analyses of some of the secondary endpoints, including disabling stroke, did show a signal of benefit, and in that context we do not have a clear signal that embolic protection is needed for all patients undergoing TAVR,” she said.

For Kaul, the conclusion statement fits a recent pattern of inconsistency among NEJM papers. DISCHARGE, designed as a superiority trial, showed no benefit of CT over invasive coronary angiography for the initial diagnosis of patients with stable chest pain, yet the NEJM conclusion said the approaches were similar. That’s “absolutely inappropriate” after a superiority trial fails to establish superiority, Kaul said.

He pointed out, too, that the PARAGON-HF and DANCAVAS trials both fell shy of meeting their primary endpoints, yet the conclusions in the respective NEJM papers made no mention of a possible benefit on the basis of the 95% confidence intervals.

Journals shouldn’t be too strict about what types of conclusions they allow, for fear of becoming too formulaic, Kaul added, “but you have to try to minimize the inconsistency in how you allow people to conclude.”

It’s important to get this right, Capodanno indicated, because many busy readers focus on the conclusions of papers. “I think that conclusion, in a way, may be a little bit misleading,” he said.

Capodanno speculated that the editors likely “pondered a lot about this” and decided to introduce the qualification into the conclusion to address issues of power and to make clear that a smaller potential benefit could be seen in a larger trial: “For the power of the study, there was no chance to detect an effect [as large as what they powered the trial for], which doesn’t mean that an effect doesn’t exist.”

They designed the study to find an overly optimistic level of benefit and then afterwards they to some degree realized that even more modest levels of benefit are probably worth it. David Cohen

Though the NEJM is usually more dogmatic about how conclusions are written, sticking more strictly to statistical facts, a more-nuanced conclusion was reasonable in this case, according to Cohen. He pointed to a study he was a part of looking at data from the TVT Registry, which suggested a reduction in the risk of stroke by 10% to 20% with cerebral embolic protection, similar to the nonsignificant 21% reduction observed in PROTECTED TAVR.

It's possible, then, that PROTECTED TAVR was “just severely underpowered to find that level of benefit,” Cohen said. “The conclusion they wrote was correct, but the challenge was that they designed the study to find an overly optimistic level of benefit and then afterwards they to some degree realized that even more modest levels of benefit are probably worth it.”

Kapadia discussed the issue during a deep dive session after presentation of the main results, showing attendees what was initially submitted to the journal as the paper’s conclusion: “The PROTECTED TAVR trial did not demonstrate a difference in periprocedural stroke with CEP, however CEP was safe and was associated with fewer disabling strokes.”

The journal didn’t allow that, but did allow the nuanced conclusion leaving open the possibility of a benefit in terms of all strokes.

That was appropriate, Kapadia told TCTMD, because it would have been wrong to leave readers with the impression that the device doesn’t work. “I would feel bad if somebody had a stroke, because I told them not to use it,” he said.

Asked about the thinking behind the published conclusion, an NEJM spokesperson declined to provide details, saying, “The publication process is confidential; because of this, we cannot share the editorial discussion of articles.”

No Major Shifts in Use Predicted

Some centers, like the Cleveland Clinic, use embolic protection in nearly all cases, but wider uptake has not been seen in most places. A study published earlier this year based on data from the TVT Registry showed that protection was used in just 7.1% of TAVIs performed between January 2018 and September 2019. Most physicians interviewed by TCTMD don’t see that changing much one way or another.

Capodanno urged waiting for the results of the BHF PROTECT-TAVI trial before making major changes in the use of embolic protection. The lesson learned from trials of aspiration thrombectomy in STEMI is that it’s better to wait for results from additional trials before making practice decisions, he said.

That’s especially true here, since the PROTECTED TAVR results “do not endorse or authorize the use more broadly of the device,” he said. “Of course, you will continue to do it in selected patients where you are worried about the risk of stroke, but with another trial upcoming I would really wait for the results and then move from that.”

We do not have a clear signal that embolic protection is needed for all patients undergoing TAVR. Megan Coylewright

Cohen predicted that PROTECTED TAVR will be used to bolster the positions of operators on both sides of the debate: “I think the people who were believers even before there was a large trial will point to the secondary endpoint [of disabling stroke] and say, ‘See, it’s beneficial just like I thought.’ And I’ve also heard from a lot of people who have not been practicing using this device in more than a handful of patients saying, ‘This supports what we believe. There isn’t definitive evidence of benefit of this costly therapy and we’re going to be cautious.’”

Ultimately, he said, “I’m not sure this is going to move the needle very much.”

Kaul forecasted a decline in use of embolic protection over the next few years. “This trial is going to discourage the uncritical use of this device because of the additional cost incurred by the hospital,” he said. Although there was previously an add-on payment for this device, which costs roughly $3,000 to $4,000, it expired last year, Kaul noted.

“If I was the hospital administrator and the interventionalist told me that I need this in this patient or every patient, I would have a tough time paying $4,000 dollars out of pocket, so to speak,” he said.

Coylewright noted that after a formal review of the data, the heart team at her center has decided there’s not enough evidence to support using protection during TAVI, even selectively. But she raised the possibility that external forces could come into play on a broader scale.

“I think in competitive markets—that is, in cities that have a number of aortic valve programs—it becomes more complicated, because there is a lot of advertising around this technology that leads the public to believe that there is a reduction in stroke,” said Coylewright “And so I think it makes it challenging if you’re not going to use it to say, ‘Now we’re pulling back,’ versus adopting it.”

Support for Further Research

Neurologists are no doubt watching this space closely. Luciano Sposato, MD, MBA (London Health Sciences Centre, Western University, Canada), chair of World Stroke Organization’s brain and heart task force, told TCTMD that PROTECTED TAVR was very much needed to evaluate the utility of CEP during TAVI, indicating, however, that there’s not “a lot of room for a change in practice based on these results.”

He supported the importance of looking for ways to prevent periprocedural stroke in TAVI, especially since the procedure will continue to be used in a broader range of patients, and made a call for future studies to include more-extensive use of neuroimaging to gain more insights.

“Whatever we can do to reduce the number of strokes should be welcomed,” Sposato said, noting that the stroke rate in PROTECTED TAVR, though much lower than in the early days of TAVI, still represents a “fairly large number of events.”

The stroke rate’s going down and so maybe we won’t need devices as much as we did before. Catherine Otto

Catherine Otto, MD (University of Washington, Seattle), co-chair of the latest US valvular heart disease guidelines, said there’s biological plausibility to support embolic protection, pointing to the debris captured by the device. “It’s hard to understand how you can have such biological plausibility and then not have a benefit,” she commented. “I think that’s probably where people are surprised and trying to understand” the results of PROTECTED TAVR.

Otto agreed that the stroke rate in PROTECTED TAVR is still higher than is desirable, but said its excellent news that the rate appears to be falling: “My takeaway message would be that the stroke rate’s going down and so maybe we won’t need devices as much as we did before.”

Still, Otto said, “we need to reduce the risk of stroke with transcatheter valve implantation and if this isn’t the right approach, what is?”

Nobody was ready to close the door on the concept of preventing stroke during TAVI through use of some type of protection device, and Cohen noted that many companies are still very interested in this area. “If we had a device that we were strongly convinced reduced disabling stroke by 70%, I think it would be very worth using,” he said. “We just aren’t quite there yet in terms of the strength of evidence.”

Coylewright is content to watch and wait.

“My mind remains very open for the promise of technologies to reduce the risk of stroke in TAVR. I’m hopeful that we can continue to do research to find how we’ll get that number to zero,” she said. “The disabling stroke signal lets us know that there is still work to be done in this area and that we have the potential to improve patient care.”