Healthcare industry advances AI governance amid lack of federal regulation

In the absence of federal standards, the healthcare industry is struggling to adopt artificial intelligence, and the Trump administration is unlikely to help, leaving the burden of implementing AI responsibly entirely on the industry itself. Experts at the HIMSS healthcare conference in Las Vegas said this task is becoming increasingly difficult as the technology grows more complex.

"One thing is clear: this administration will not regulate AI. For better or worse, figure it out on your own," said Tanay Tandon, CEO of Commure, a provider automation company, during a panel discussion.

President Trump's hands-off approach to AI governance means hospitals hoping to use AI to save costs and reduce burdens on overworked clinicians may remain in a regulatory gray area for at least the next four years. The president has said his goal is to allow American developers to innovate freely, but this has adverse effects on technology developers and healthcare companies—which are eager for guardrails because AI is prone to errors, degrades in performance over time, and can exacerbate existing biases.

Additionally, some experts noted that the lack of national standards could actually hinder AI development and adoption. "When there's no federal framework, it absolutely raises all sorts of issues," said Leigh Burchell, president of the Electronic Health Record Association. "We all want to know what the rules are, and then we can comply."

Biden and Trump's differing stances on healthcare AI

To date, a handful of federal agencies, including the Office of the National Coordinator for Health Information Technology at the Department of Health and Human Services, the Centers for Medicare & Medicaid Services, and the Food and Drug Administration, have issued targeted rules on the use and quality of healthcare AI. But neither Congress nor the executive branch has focused on a comprehensive regulatory framework—although some progress was made during the Biden administration, when an HHS working group was working to build a unified regulatory structure.

That working group released a strategic plan in January—just 10 days before Trump's inauguration. However, Trump struck down the blueprint in one of his first executive orders. Meanwhile, federal employees working on AI regulation, including those at the FDA, have been swept up in the Trump administration's government downsizing efforts. Amid this turmoil, the future of the HHS office responsible for AI policy remains unclear.

As a result, the already weak momentum in Washington toward developing a concrete healthcare AI regulatory strategy appears to have stalled, at least for now. Instead, Trump announced the "Stargate Project," a $500 billion investment deal with private companies to prioritize AI development and maintain U.S. leadership in the field—a high-stakes bet that was quickly complicated by the release of China's high-performance, low-cost open-source model DeepSeek.

"This administration—the brakes are off, the pedal is to the metal."

—Brian Spisak, Director of AI and Leadership Programs at Harvard's National Preparedness Leadership Initiative

The Trump administration issued a request for information in early February seeking public input on a potential national AI action plan. However, the plan's wording made clear the administration's priorities: namely, to "maintain and enhance America's AI dominance and ensure that unnecessary burdensome requirements do not hinder private-sector AI innovation."

Revoking the Biden-era AI plan was largely symbolic, as agencies had not yet imposed any requirements on developers or users. But with "this administration—the brakes are off, the pedal is to the metal," Brian Spisak said at HIMSS, "health system leadership bears a significant responsibility to find the best balance between innovation and speed, and safety and tradition."

Technological upheaval

This responsibility—also falling on AI developers creating the models, software vendors integrating AI into health records, and clinicians using them—is not to be underestimated. Currently, the most advanced AI in healthcare organizations is used only for administrative automation, touching patient care only marginally. But this seems to be changing: according to a survey conducted by HIMSS in the fall, healthcare organizations are increasingly interested in more clinical applications of AI, such as customizing treatment plans or assisting clinicians in diagnosis.

Many applications involve generative AI, which can create original text and images. But such models are known to hallucinate, providing factually incorrect or irrelevant answers. AI may miss important information, an error called omission. Models can also drift, meaning the AI's performance changes or degrades over time. Experts say that given AI's growing use in extracting data from electronic health record systems, transcribing doctor-patient conversations, and more, such errors could interfere with clinicians' ability to care for patients.

Meanwhile, the technology is advancing at a staggering pace. Last year, the healthcare industry was just beginning to grapple with governance issues of generative AI. But now, discussions have shifted to AI agents, which can complete complex tasks with little to no human oversight.

Commure's Tanay compared the current moment to the late 19th-century U.S. transition from kerosene to electricity. "The way we did things six months ago is completely irrelevant," he said. Given the rapid pace of development, experts say any federal standards from any government would need to remain flexible.

The Biden administration's HHS working group suggested that the government could develop guidelines around tool testing and piloting, and provide some adoption support. However, it avoided prescriptive approaches. This aligns with the expectations of many stakeholders. Many tech company and hospital system executives say any federal standards should be tiered based on the level of risk posed by the AI model—for example, stricter oversight for algorithms that help doctors diagnose diseases, and looser restrictions for algorithms that help hospital staff allocate beds.

"We must weigh the balance between under-regulation (which may increase risk) and over-regulation (which will stifle innovation)," said Anthony Chang, Chief Intelligence and Innovation Officer at Children's Hospital of Orange County, during a panel discussion. "This administration is more likely to lean toward under-regulation. Therefore, as professionals, we must be careful not to allow that to happen," Chang said.

States and industry organizations fill the gap

Lacking guidance from Washington, hospitals and medical groups are scrambling to establish their own internal controls while piecing together state laws and voluntary standards issued by industry organizations. States including Colorado, Utah, and California have enacted laws setting disclaimer requirements for AI systems. More states are considering similar laws: according to Burchell, the Electronic Health Record Association is tracking 150 state bills related to healthcare AI. "The number of bills has surged," Burchell said.

But inconsistent standards could prevent healthcare AI developers and software companies from launching products in certain states, potentially putting patients at a disadvantage based on where they live. She added that more risk-averse software companies might avoid AI or certain states altogether. "State laws of all shapes and sizes pose a risk to us because it means we have to do all sorts of different development. We'd rather develop a system that can be widely used and accepted across the country," Burchell said.

Healthcare AI standards organizations are also filling the void left by the federal government. These groups are typically composed of leading hospitals, digital health companies, and tech giants, including the Health AI Partnership (an industry-facing AI learning network) and the Coalition for Health AI (CHAI), which recently launched an AI registry for hospitals. "I think we may see more non-governmental organizations like the Health AI Partnership become our North Star today, providing some leadership," said Rachel Wilkes, enterprise lead for generative AI initiatives at EHR vendor Meditech.

But experts say that without federal government support, industry coalition standards carry little weight. Historically, voluntary standards have not been particularly effective. "Without a federal framework, people have room to act in their own best interests, whatever that may be," Wilkes said.

"We still don't quite know how to deal with it"

EHR vendors and hospital operators say they are establishing rigorous internal standards for AI tools, including validation and frequent audits. "Government oversight has its place, but I do think the evolution of clinical practice is often driven more by what happens inside healthcare systems," said Seth Howard, executive vice president of R&D at Epic, the largest U.S. EHR company. In interviews, executives from Epic, Oracle, Meditech, and eClinicalWorks said they are providing AI to physicians with strict oversight, including backend accuracy checks and continuous monitoring. However, tech leaders emphasized that ensuring everything runs as planned is also the responsibility of hospitals and clinicians.

"We are in a business that involves human lives. It cannot be taken lightly. The discussions about guardrails, checks and balances, and what needs to be done cannot be underestimated," said Girish Navani, CEO of eClinicalWorks. Tech giants hold similar views. For example, Google has partnered with for-profit hospital giant HCA to develop an evaluation framework to capture any errors produced by its AI models and ensure their reliability, according to Aashima Gupta, head of healthcare at Google Cloud. "We provide these tools for the evaluation framework, and there are people involved in the feedback loop at every stage, which makes the model more effective," Gupta said. "That gives me peace of mind."

Although some in the private sector say they have governance handled, AI engineers argue that modern AI is extremely difficult to oversee. The primary strength of generative AI—creativity—also introduces subjectivity, complicating the evaluation of its outputs. For example, if two clinicians are asked to summarize a patient's history based on clinical notes, their results could differ significantly while still being accurate. The same applies to generative AI, experts say: when such variability exists, how do you measure quality in a standardized way? "AI governance is still a very mature process," said Harvard's Spisak.

Hospital executives say they are handling oversight cautiously. But some research suggests that governance systems for simpler predictive AI models are already insufficiently rigorous. According to a study published last year in the New England Journal of Medicine, hospitals with clear procedures for AI tool use and evaluation are still struggling to identify and mitigate problems. Even some of the most well-resourced and technologically advanced systems are struggling. Cleveland Clinic has an AI governance body with members from all stakeholders across the academic medical center, according to Rohit Chandra, the institution's chief digital officer. That working group oversees AI's impact on the organization and patients while ensuring clinical safety, and discusses thorny issues like privacy, legality, and bias. But "I don't think we've fully figured it out," Chandra said during a panel. "The word 'hallucination' has only emerged in the past two or three years, and we still don't quite know how to deal with it."

Hospitals should try to hold specific individuals accountable for tool performance, as part of a larger governance body that includes executives, lawyers, physicians, and nurses, said Brenton Hill, operations lead at standards organization CHAI. Hospitals need to decide how to effectively monitor AI and report that information, depending on the products they have. They also need to consider what resources AI will use and establish appropriate data use agreements with AI vendors, Hill said during a panel. But "there is no one-size-fits-all governance structure that solves everything," Hill said.

"A pipe dream"

Although a roadmap from federal regulators would help, stakeholders committed to integrating AI tools into healthcare say they are not holding out much hope. "While self-regulation is good, we don't think it's enough. We think AI is too important to go unregulated," said Google's Gupta. But when asked about her expectations for the Trump administration, Gupta was noncommittal. "It's hard to say right now. We're trying to figure out how to best work with them, share our best practices... It's too early to tell. I think the entire healthcare community is waiting," Gupta said.

Other experts say the Trump administration is a wake-up call for hospital executives who had hoped Washington would take on the responsibility of overseeing AI. Instead, the responsibility should fall on all those who touch the technology to ensure AI algorithms—given their variability and inherent opacity—operate as designed, especially in an industry where any error can affect patient health. "The simple answer is, if a regulator says it's safe, then I can trust it. I think people hoped that would be the case with AI," said Aaron Neinstein, chief medical officer of agentic AI company Notable. "I think that was a pipe dream."