‘Message Unchanged,’ Say Researchers of Criticized COVID-19 CMR Study

(UPDATED) Investigators who led a widely publicized cardiac magnetic resonance (CMR) study showing that COVID-19 may cause lingering cardiac damage have responded to questions raised on social media challenging their findings.

Eike Nagel, MD, and Valentina Puntmann, MD, PhD (both University Hospital Frankfurt, Germany), issued a correction this week in JAMA Cardiology, stating they reviewed and reanalyzed their data after TCTMD alerted to them Twitter discussions about their work. While they identified a few other errors as part of their reanalysis, once all the mistakes were accounted for and corrected, they say, the initial message of their CMR study still stands.

“We are pleased to confirm that reanalysis of the data has not led to a change in the main conclusions of the study,” write Nagel and Puntmann in a letter to the editor. On the group’s website, the researchers go on to say they “are reassured that the validity of the originally published results remains solid and the message unchanged.”

That message, as reported by TCTMD, is that COVID-19 may result in lasting cardiac damage even in nonhospitalized patients. In a study of 100 people who recovered from COVID-19, 78% had abnormal CMR findings 2 to 3 months after initially testing positive for the virus. Compared with healthy and risk factor-matched controls, patients who recovered from COVID-19 had lower LV ejection fractions, higher LV volumes, and elevated values of native T1 and T2 on CMR.

Recovered COVID-19 patients, in that first paper, also had a high LV mass index compared with healthy controls, but that association was no longer statistically significant after the researchers obtained missing data from the original CMR scans, the correction notes.

I look at their interpretation being only correct for the out-of-date data, and I hope they agree that the new data is now fully reassuring. Darrel Francis

JAMA Cardiology editor Robert Bonow, MD, and deputy editor Clyde Yancy, MD (both Northwestern University Feinberg School of Medicine, Chicago, IL), also weighed in, issuing a brief letter accompanying the correction. They proceeded with a repeated statistical review—and requested reanalysis and revision by the original investigators—following questions raised after the study’s publication, they write. “A rigorous review has confirmed that the findings as originally reported remain valid.”   

Twitter Storm

After the study was published on July 27, 2020, Darrel Francis, MD, and Graham Cole, MBBS, PhD (both Imperial College Healthcare NHS Trust, London, England), challenged some of its numbers, questioning whether key values were reported as medians instead of means. In another instance, Cole identified a patient with a high-sensitivity cardiac troponin T (hs-cTnT) of 17.8 pg/mL in Figure 1 but was unable to find that patient in Figure 3 that plotted hs-cTnT levels against the time since diagnosis.

In their letter to the editor, Nagel and Puntmann said some of the metrics for key values were not reported correctly and that they have since updated the paper to include the accurate means (standard deviations) or medians (interquartile ranges). They also acknowledge inconsistencies in key figures, stating that data for the time between COVID-19 diagnosis and the CMR images and hs-cTnT were not reported correctly—this, they say, led to problems with Figure 3. During their review, the researchers discovered other mistakes that they have now corrected. 

“While reviewing the complete data set for any other potential errors, we also benefited from some additional data which had become available, as well as the journal’s statistical advice, and took the opportunity to improve the presentation of certain results,” write Nagel and Puntmann on their website. While some P values may have changed, the main message is consistent.

Speaking with TCTMD about the revisions, as well as the statement that the conclusions are unchanged, Francis had a completely different take, saying that, in his view, this study and specifically these new data do not raise any alarms about the lingering cardiac effects of COVID-19. He praised the researchers for their responsiveness in updating their tables and figures to reflect the most accurate data, but asserted that their reading of the study should be updated as well.

“I look at their interpretation being only correct for the out-of-date data, and I hope they agree that the new data is now fully reassuring,” he said. “I appreciate them for making it available so quickly.”

We strongly refrain from interpreting the data in terms of the permanency and long-term impact of our results, because this information is simply not available. Valentina Puntmann 

As he was on Twitter, Francis continues to remain critical of the group’s statistical analysis. While the researchers included P values showing significant differences in the blood test results and CMR findings between the three study groups, the P value for a three-arm comparison is unhelpful because it includes a healthy control group who’d be expected to have much better lab and CMR results than COVID-19 patients, he said. Instead, the most important comparator arm is the risk factor-matched control group, and when looking at those cohorts, the difference in abnormal CMR findings isn’t nearly so striking, Francis argued.

“The COVID-19 survivors have different heart scans than the healthy controls, but this is because of their preexisting risk factors and not because of COVID-19,” he said. In a two-arm comparison of the abnormal native T1 findings on CMR, Francis said there isn’t any significant difference between the COVID-19 patients and risk factor-matched controls.

Responding to Questions

To TCTMD, Puntmann said they appreciated Francis’s contributions to the discussion and are grateful he is helping to put their changes into perspective. All corrections, she stressed, were made to “improve the clarity of the presentation and not to change or correct the content,” she said in an email. With respect to the statistical analysis, Puntmann said they followed the direction of the journal’s statistician to present their data in a three-way comparison throughout the paper to be consistent. The post hoc differences between the COVID-19 and risk factor-matched control groups were significant as originally reported and indicated in the updated table, she noted.

“We strongly refrain from interpreting the data in terms of the permanency and long-term impact of our results, because this information is simply not available,” said Puntmann. “Similarly, whether the differences are regarded as large or cause of major concern cannot be extracted from the current paper. We merely report our observations.”

In his review of the data, Francis also focused on the blood test results, which included some values that were adjusted as part of the corrected publication. Unlike interpreting CMR images for evidence of cardiac damage, interpreting the laboratory values is rather “straightforward,” he said. When looking only at high-sensitivity C-reactive protein, hs-cTnT, and NT-proBNP values, Francis said there isn’t much evidence that patients with COVID-19 look any worse in follow-up than patients with similar cardiovascular risk factors.

Regarding the magnitude of troponin release, Puntmann acknowledged that it’s smaller than in other disease states, but said there is little known about the association between troponin levels and clinical outcomes in COVID-19 because it’s not as well studied as other disease contexts, such as ACS. “This discussion is good, because it highlights the important gaps in knowledge,” she said, adding that the biomarker data were mainly reported to show parallels with the CMR measurements. That the biomarker values aren’t particularly high in the COVID-19 population makes sense given that they don’t have the type of cardiac damage, such MI or heart failure, that would result in highly abnormal blood markers.

“In our observations we focus more on CMR measures, because these have been more explored and similar changes have been related to worse outcome in other inflammatory disease contexts,” said Puntmann. “We again clearly state that we cannot tell at present whether this can be transferred to this specific group [of COVID-19 patients].”

Puntmann said they are concerned about the evidence of myocardial water, as documented by the number of COVID-19 patients with significantly abnormal T2 findings on CMR, significantly increased fibrosis, and nonischemic scar. “However, again, this ‘concern' is transferred from previous knowledge and not proven in the current cohort,” she specified.