Q&A: Haider Warraich on the Quest for Agentic AI Tools for CVD

Last week, the US Advanced Research Projects Agency for Health (ARPA-H) announced the teams selected for its Agentic AI-Enabled Cardiovascular Care Transformation (ADVOCATE) program, which aims to provide $62.7 million over 4 years to support the creation of “the world’s first, reliable, FDA-authorized clinical agentic AI system that serves 24/7 as a new, digital member of the clinical care team.” Agentic AI tools can operate autonomously without continual oversight from humans.

TCTMD spoke with Haider Warraich, MD, the cardiologist serving as program manager of ADVOCATE, about the effort.

How did the ADVOCATE program come about?

I’m a cardiologist who’s always been focused on how I can do the best for the patient right in front of me and also the really large population of patients with cardiovascular disease that we serve. One of the things that’s always bothered me is how we have been unable to take clinical knowledge—things that we know to be right as cardiologists—and really implement it at scale for populations.

The other thing that’s always troubled me as a cardiologist has been how inaccessible subspecialty care is, even though cardiovascular issues are the most common cause of disease in the US. Our research group has shown that 46% of counties in this country don’t have a single cardiologist. And while that’s a big problem, I now practice in the most cardiology-dense area in the entire world—which is Boston—and it still takes people 6 months to see a cardiologist.

These have been some of the things that I’ve been focused on in my career, and then a few things happened that really led me to this moment.

First, I worked at the FDA as a senior advisor to then FDA Commissioner Robert Califf, MD, who’s also a cardiologist. My charge was chronic disease. But the problem in chronic disease was never that we didn’t have enough good treatments, at least for most common chronic diseases. It was that we were unable to get those treatments to patients who need them the most.

While I was at the FDA, we had the ChatGPT moment, if you may. Generative artificial intelligence (AI) became a reality and instantly transformed things. We all knew what this technology would be able to do. It reset the floor and eliminated the ceiling, and it was clear to me that this was a technology that could potentially help scale this clinical knowledge.

But the problem is that, to date, there is not a single FDA-authorized technology that uses generative AI or agentic AI that can provide safe and effective clinical care directly to the patient.

There’s a lot of regulatory uncertainty, there are technical risks, and there are questions about implementation—all of these issues that couldn’t be addressed in series, which is the way we usually solve problems in medicine, in part because 20 million people use ChatGPT every day for health-related questions. We just didn’t have the time to wait around and think about developing a traditional pathway like we normally would for a medical device.

This was around the time that I’d heard of ARPA-H, a relatively new federal research agency modeled after the Defense Advanced Research Projects Agency (DARPA) that sits within the Department of Health and Human Services, with a mission of making transformative advances in health outcomes through really ambitious investments and programs that push technology forward in a way that would be impossible without this agency’s existence. And that’s when I pitched the ADVOCATE program. This was back in August 2023.

Why did you choose heart failure as the initial focus of the ADVOCATE program?

There’s been a lot of work that shows that less than 2% of patients with heart failure with reduced ejection fraction in this country are on the evidence-based medicine that we know can prolong and improve their lives. This is not because clinicians aren’t trying hard enough. I’ve practiced medicine. I see heart failure patients. I’ve done so in large cities. I’ve done so in rural areas where I was the only cardiologist in a county of hundreds of thousands of people. And it just feels like there is a structural problem, especially on the outpatient side, where the way that it’s designed, we’re not able to do what I think we all wish we could do for not just the patient in front of us, but the entire population of patients.

Focusing on patients with heart failure was an interesting choice when we made it because, at that time, most people were not thinking about high-risk conditions. Most people who were thinking about these technologies were really thinking of lower-risk populations, such as patients with mental health disorders or patients in primary care. Yet if you look at the history of medical devices, especially those in cardiovascular disease, we really often start with patients with high-risk conditions or disease subtypes because that is where the potential benefit may exceed the potential risk.

We’ve been able to make that case to the entire ecosystem, which is aligned with our vision that if you’re going to start anywhere with this technology, heart failure is as good as it gets. This is because the need is high, the return on investment is there, use of the standard of care that we wish patients would receive is very low, and event rates are high, so your studies can be short. In addition, the integration of physiologic signals makes a lot of sense because so many of those signals can be captured through wearables and then used to enhance the actions of these technologies.

What are the main goals of the ADVOCATE program?

One is to support teams that are going to propose the development of a patient-facing technology that can do many of the functions a clinician can, including making a diagnosis, performing triage, and making changes to guideline-directed medical therapy or diuretic therapy autonomously, all with a clinician in the loop.

From a technical perspective, we want these technologies to be fully integrated with the electronic health record and be a real part of clinical practice rather than one that lives outside of the traditional clinical practice environment. And it should not just be reactive. It should have a surveillance layer. It shouldn’t just wait for the patient to get sick. It should be able to anticipate exacerbations before they happen. And we ask teams to do so through integration with wearable technology. We gave these teams an ambitious goal—that we want you to develop a plan in which you can be ready to go to the FDA for market authorization within 2 years. We will work with these teams in close coordination with FDA, which continues to this day to be an amazing partner, to maximize the chances of success.

[The goal] is that you treat autonomous clinical agentic AI as a new member of the clinical team and you have a clear delineation of who’s taking what action, making clear where the risk lies. Haider Warraich

For the second part of this program, we’re asking teams to develop an independent supervisory agent or supervisory layer that essentially is also an AI-based technology that is co-deployed with the primary clinical technology. It helps to determine which outputs can autonomously proceed back to the patient and which need an additional layer of human check, whether that’s passive or active. I think we can all agree that if a system like this relies on human input every single time for every single action, that will overwhelm an already stretched system.

But at the same time, I think we can also agree that we’re just not there yet when it comes to a technology that doesn’t have human oversight. The technology is not mature. We don’t have the type of information needed to be able to allow autonomous functionality. Clinicians and health systems are putting their licenses and their reputations on the line, so the technology needs an additional layer of safety that is going to help with adoption and implementation.

Oftentimes by the time a clinician sees new technologies, especially digital technologies, they’re very late in development. So another aim of ADVOCATE was for clinicians to have essentially a role as co-developing these technologies from day one. We want to ensure that health systems and clinicians and patients are part of the solution from the very origin, rather than simply being helicoptered in at the end. So for the third part, we did an open recruitment for health systems across the country to participate as co-developers of this technology and we will provide clinical resources from the start, provide the resources to perform the actual studies that are going to be needed to validate these technologies, using electronic health record data, shadow-mode deployments, and, ultimately, randomized trials.

Together, these steps will lead to the scalable design, deployment, and implementation of these technologies, keeping pace with an industry that’s moving so fast. And instead of doing all these things one after the other, we’re doing them all together.

What is the role of ARPA-H?

ARPA-H is a funding agency. We put a lot of effort into designing programs, designing metrics and milestones, identifying the right problems, and recruiting the best teams. But all the work is done by the external teams that we select and fund. And for this program, we were delighted to see the level of interest in the community. We had almost 300 teams apply, one of the highest numbers for any ARPA-H program.

We are incredibly excited not only about the teams that we are selecting, but also about the quality of the proposals from the teams that didn’t get selected. The goal of an ARPA-H program is to support teams that we think are most likely to achieve our goals, but we also want to stimulate an ecosystem and develop knowledge that can be transferred across the ecosystem—not just create a walled garden of knowledge, but to really develop standards, precedence, and approaches that everyone can learn from so that we lift all boats.

How are you ensuring that the AI tools that come out of this effort will be deployed in a safe and appropriate manner?

If you ask me today whether agentic AI is ready to do the things that we want these teams to do in the ADVOCATE program, the answer is clearly no. I do not believe that agentic AI is ready today. I do think that it can be, and I do believe that it will be—given appropriate resources, oversight, and metrics that these technologies have to meet. That’s why this program exists, because we believe that today these technologies are not there and will not be there unless ARPA-H makes this investment.

We are very metric- and milestone-based. We have very, very high standards for the type of performance we will accept for these technologies for each of the tiers. And to assess if teams are meeting that, we have brought in one of the most well-accomplished teams in the world, the Applied Physics Lab at Johns Hopkins University. This is a team that specializes in independent verification and validation (IV&V), especially for AI technologies. We have been working with them for several months and that will continue through the course of this program to ensure these technologies are doing what they say they’re doing and doing it in environments that very closely mirror clinical deployment and answer the questions that clinicians care about.

That type of transparency is critical because this is a technology that does come with risks. We’re not just developing a chatbot, even though there are risks associated with chatbots, too. We are developing a technology that is highly capable. And if you’re going to develop a highly capable technology, how you evaluate it has to be equally robust.

The reason we’re doing that is because we think that FDA authorization for these technologies is the floor. FDA authorization for these technologies, even if achieved, is not going to convince clinicians, health systems, and patients. That bar is going to be higher, especially at this early stage. We’re making clear the type of evaluation needed to develop a technology like this and bring it to the bedside that can then be replicated across disease states—not just heart failure.

For this first generation of technologies, having a high bar is important because there are so many unknowns. But also, this is a once-in-a-lifetime technology that can overcome so many of the challenges we as a health community have been facing for such a long time, so we believe that doing it well is more important than doing the bare minimum. We want a maximalist approach to evaluation and assurance, because we understand what the potential of this technology is.

If an agentic AI tool makes a mistake, who owns that?

It’s a great question, and I will say case law in this space is limited but is almost assuredly likely to grow over the next couple of years as more patients use these technologies and as more of these technologies reach clinical settings.

In my mind, it comes down to whether these technologies are assistive. For example, an AI scribe that dictates a note and then presents that note to a clinician to sign off on carries a different level of risk than a technology that is fully autonomously prescribing medications. That level of autonomy will shift who bears more of the risk.

Many companies in this space are taking on the liability risk already, so my guess is that this is a technology that’s going to be regulated similarly to other medical technologies. I don’t foresee that a new set of laws or regulations are going to be made for AI, especially from a liability perspective. It’s really going to come down to the individual use case and how it was designed and what type of decision-making authority or autonomy the technology possesses.

What I do worry about is that a response to liability risk is often developing assistive technologies in which the technology doesn’t make the decision but curates it, with the clinician signing off on it. I find a few risks there as a clinician. One is automation bias. This is a known risk where once a technology starts to work, users—whether clinicians, engineers, or anyone making decisions—will default to accepting the action. And as these technologies proliferate, the number of actions that we have to default to is going to increase a lot.

Pair this with deskilling. So much data out there shows that as people rely more on these technologies, their own skills diminish. What that means is that assistive technologies will put a huge burden of liability on clinicians. For me, an end state that is probably ideal is that you treat autonomous clinical agentic AI as a new member of the clinical team and you have a clear delineation of who’s taking what action, making clear where the risk lies.

My interest as a program manager and as someone who is part of this agency is to create a solution that is going to work for all the folks in our ecosystem—our patients, our health systems, our clinicians, our innovators, and our regulators—because only then can you have a durable marketplace and really unlock what I believe is the defining technology of our generation.

As this process plays out, we’re committed to communicating what we’re doing. We’re going to be engaging with the cardiovascular community in all sorts of ways to make sure that their voice is represented in this program and also with the technology community, because there’s so much interest that teams have in learning from how we are approaching this frontier technology. One of our missions is to make sure that we are always keeping teams up to date, not just at the launch of the program, but importantly, throughout the course of this program.