Healthcare AI Is Expanding Fast but Governance Still Lags Behind

Healthcare is putting AI into daily work faster than most teams can govern it. In 2024, 31.5% of surveyed nonfederal U.S. hospitals said they were already using generative AI, and 24.7% said they planned to add it within a year.

If I had to boil the article down to one point, it’s this: AI risk in healthcare is now a people-and-process problem more than a tool problem. The weak spots are plain - missing inventories, weak vendor checks, unclear ownership, thin review of outputs, and poor escalation when something fails.

Here’s the short version of what matters:

• AI is already in core workflows like charting, imaging, claims, prior auth, and analytics.
• Governance is behind adoption, which leaves gaps in privacy, cyber, compliance, and patient care.
• Shadow AI is a direct risk: 20% of healthcare organizations reported a breach tied to it in 2025.
• Ownership is often fuzzy: many teams still can’t say who approves a tool, who checks outputs, or who acts when performance slips.
• Vendor risk is bigger than a security questionnaire. Data-use rights, model updates, PHI handling, subcontractors, and BAAs all need review.
• Rules are tightening through the 2026 HIPAA Security Rule changes, 2026 FDA CDS guidance, NIST AI RMF use, and state-level AI laws.
• The fix is simple in concept: inventory every AI tool, assign an owner, standardize vendor review, monitor for drift, and centralize oversight.

A few numbers make the gap hard to ignore:

• 7 health systems launched Epic’s AI Charting in production on day one in 2026.
• 70% of healthcare organizations have AI governance committees, but only 30% keep an enterprise-wide AI inventory.
• 38% either split AI risk across groups without a clear escalation path or have not defined ownership at all.
• The DOJ reported more than $5.7 billion in healthcare False Claims Act recoveries in FY 2025, which puts weak traceability under more pressure.

Bottom line: if you work in a health system, you don’t need a brand-new governance program from scratch. You need tighter control over inventory, ownership, vendor review, policy mapping, and cross-team oversight before AI use outpaces your ability to control it.

Healthcare AI Governance - Risks, Compliance, and Frameworks Explained

sbb-itb-535baee

Where Healthcare AI Governance Is Falling Short

The weak spots show up most often in vendor controls, workflow review, and clear ownership.

Third-Party AI Vendor and Supply Chain Controls Have Gaps

Standard security questionnaires often miss the stuff that matters most with AI. They may not ask whether a vendor uses customer data to train models, whether AI features are tucked inside the product without clear disclosure, or whether there’s a full chain of BAAs covering every endpoint that handles PHI. So a tool can look approved on paper and still put PHI at risk in day-to-day use.

Contracts have the same problem. They often don’t line up with HIPAA duties or spell out who is responsible for AI-generated outputs. And some standard vendor terms give the vendor broad rights to use customer data to train commercial models. That can clash head-on with the BAA. ^{[6] [7]}

SOC 2 Type II and HITRUST can help, but they’re control attestations. They are not a stand-in for a HIPAA Security Rule risk analysis tied to a specific deployment. There’s also another issue: vendors change models after go-live. When that happens, organizations need a way to check that the new model behavior still matches what was approved in the first place. ^[7]

Clinical AI, Data Use, and Privacy Controls Are Not Keeping Pace

One of the biggest myths in healthcare AI is the idea that a clinician is meaningfully reviewing every output. In high-volume workflows, that often turns into little more than quick sign-off, not a real safeguard. ^[2]

Data use is another sore spot. Secondary use of patient data for vendor model training without consent is a growing risk, and many patients have no clear notice that their data is moving through AI systems at all. State laws like California's AB 3030 are adding disclosure and oversight duties that many organizations still haven’t mapped to their current data-sharing practices. And when data-flow mapping is incomplete, it becomes very hard to tell which AI tools touch PHI or where that data goes. ^{[2] [7]}

When no one owns the process, these privacy failures don’t just sit there - they fall into a black hole.

Model Ownership and Escalation Are Undefined

In many organizations, no one can quickly say who owns a given AI tool. It might be IT. It might be the clinical team that asked for it. It might be the vendor. Too often, AI systems have no named owner and no clear escalation path.

In May 2026, the VA Office of Inspector General found this exact pattern inside the Department of Veterans Affairs. The office cited a critical lack of oversight for generative AI chat tools that were already being used. ^[5]

The root problem is simple: many governance frameworks still treat AI like ordinary software. But ordinary software doesn’t drift. AI does.

As patient populations change or clinical protocols shift, model performance can move away from the baseline that was approved. If that performance slips and no owner is named, no one steps in. No one escalates. The problem just keeps moving through the system. ^[2]

Taken together, these gaps make AI governance an operational risk, not just a policy problem. The result is exposure across clinical care, privacy, compliance, and liability.

What Good Healthcare AI Governance Actually Looks Like

To close the gap, healthcare groups need named owners, lifecycle controls, and policies tied straight to regulatory duties.

Build a Cross-Functional Governance Structure with Clear Ownership

A standing AI governance committee is the starting point. It should include people from Compliance, Clinical Informatics, IT Security, Legal, Procurement, and Clinical Leadership. Each group needs a clear lane.

• Clinical leadership handles output review
• The CISO handles access and logging
• Privacy and compliance handle HIPAA and BAAs
• Procurement and legal handle contracts
• The CIO handles inventory and lifecycle controls

That last item is where many groups get stuck. The problem isn't abstract. It's day-to-day execution: 70% of healthcare organizations have AI governance committees, but only 30% maintain an enterprise-wide AI inventory ^[8].

Each AI tool also needs a named contact for escalation when outputs look questionable. If no one owns that step, the path breaks fast. Right now, 38% of healthcare organizations either share AI risk responsibility across multiple groups without clear escalation paths or have not defined ownership at all ^[8].

Once ownership is set, use the same control model from intake to retirement.

Apply Lifecycle Controls from Intake Through Retirement

Good governance covers the full life of the tool - from intake and assessment to deployment, monitoring, incident response, change management, and retirement.

Set a baseline before go-live. Then review drift, override rates, and vendor updates on a fixed schedule. If a model changes, it should go back into review before it reaches production again. And if a tool keeps missing performance or bias thresholds, it shouldn't stay in use.

Write Policies That Connect NIST AI RMF, HIPAA, and Healthcare Operations

The NIST AI RMF's four functions - Govern, Map, Measure, and Manage - give teams a practical way to turn broad rules into day-to-day work. Policies should translate those controls into standard requirements for cyber, privacy, compliance, and clinical teams.

Vendor documentation should be reviewable, not just sitting in a folder. Model Cards need to show training data, limitations, and validation results. That's not busywork. It's how teams spot gaps before they turn into patient or compliance issues. For example, the Epic Sepsis Model's external validation performance fell well below the developer-reported range ^[10].

Proposed 2026 updates to the HIPAA Security Rule are expected to remove the "addressable" safeguard loophole and bring AI systems handling ePHI into scope. That would require an annual technology asset inventory and written risk analyses ^{[2] [9]}. Teams that line up their program now won't be scrambling later.

How to Put AI Risk Management Into Practice at Scale

Once governance rules are in place, the next job is turning them into day-to-day work across the business.

That’s where things often break down. Governance looks fine on paper, but if teams carry it out in different ways, risk starts to pile up. And for many healthcare organizations, that execution gap is the problem.

Standardize AI Vendor Assessments, Contracts, and Evidence Reviews

Start with a repeatable vendor review process. Use a standard intake questionnaire. Require disclosures for model changes and subcontractors. Review all contracts, even the ones that never mention AI, for data-use rights, PHI training limits, output accountability, update notice, and audit or termination rights.

As AI use grows, repeatable workflows matter more than ad hoc review. A central workflow tool can help distribute questionnaires, collect evidence, and summarize findings, while still keeping people involved in the final review.

Those findings need to feed into one shared review and escalation process. If they stay stuck in separate teams or systems, the same gaps tend to show up again.

Centralize AI Oversight Across Cyber, Privacy, Compliance, and Clinical Teams

Fragmented oversight is one of the most common failure points. 38% of healthcare organizations share AI risk responsibility across multiple groups without clear escalation paths or have not defined ownership at all ^[8]. That’s not just a policy issue. It’s an operations gap.

Use one dashboard, one audit trail, and clear routing for approvals, findings, and escalation. Censinet AI routes findings to the right teams and keeps one audit trail across cyber, privacy, compliance, and clinical review. Automation handles the volume. Reviewers make the final calls.

The contrast below shows how fragmented oversight turns into repeatable governance.

Current vs. Target AI Governance

Conclusion: Close the Governance Gap Before AI Risk Grows Further

By early 2026, AI use was moving faster than governance. 31.5% of surveyed nonfederal U.S. hospitals were using generative AI, yet only 16% of health systems had an enterprise-wide AI governance strategy ^[4]. That mismatch is where cyber, privacy, compliance, and patient safety risk starts to pile up, often in ways that stay hidden until something breaks.

The main issue isn't the tech itself. It's governance. Healthcare already has the bones of an AI governance framework, but many organizations still lack the inventory, monitoring, and clear accountability needed to run it day to day. A policy on paper won't fix that. Governance has to work like an operating process.

The good news is that the fix isn't complicated. It comes down to inventory, ownership, vendor control, policy alignment, and centralized oversight. No full rebuild needed. Just five concrete moves:

• Inventory every AI tool in use, including vendor-embedded features and unsanctioned shadow AI. That's a big one, because shadow AI caused breaches at 20% of organizations in 2025 ^[4].
• Assign a named owner to every AI application, so there's always a clear path for escalation when something goes wrong.
• Standardize vendor assessments and contracts to address data-use rights, model training rights, and liability for AI-generated outputs.
• Align policies with the NIST AI RMF and HIPAA, and disclose AI use in diagnosis or treatment ^[1][3].
• Centralize oversight across cyber, privacy, compliance, and clinical teams, replacing fragmented point-in-time reviews with continuous governance ^[8].

The pressure is only building. The 2026 HIPAA Security Rule overhaul and updated FDA clinical decision support guidance are tightening the rules for AI systems that handle PHI ^[2][3]. Healthcare AI will keep growing. The part that decides whether that growth lowers risk or makes it worse is governance.

FAQs

What counts as AI in a health system?

In a health system, AI means more than generative chatbots. It also includes machine-based systems that analyze data and produce predictions, recommendations, or decisions that shape clinical and operational workflows. In many cases, these tools sit inside EHRs or are built into products from existing vendors.

Examples include ambient documentation tools, claims review models, medical imaging algorithms, patient risk models, and message triage platforms.

Put simply, if a tool uses automated processing to influence diagnosis, treatment, reimbursement, or care management, it counts as AI.

Who should own AI governance internally?

AI governance needs centralized accountability, not scattered ownership.

A multidisciplinary oversight committee should lead the work, with people from legal, compliance, IT, information security, privacy, clinical operations, and ethics at the table.

That group should have clear authority across the full AI lifecycle. And each AI tool should have a named owner, so there’s no confusion about who is responsible.

Accountability should also be shared in a practical way: clinical leaders should own patient safety, while technical leaders should own system integrity and data security.

How can hospitals find shadow AI in use?

Hospitals should treat shadow AI discovery as a visibility problem. The first step is to map every AI application, prompt, model call, and downstream action that touches protected health information.

Legacy data loss prevention tools may miss dynamic, back-and-forth AI exchanges. So before rolling out broader policies, use runtime controls and identity-linked logs to build a clear inventory of AI apps, agents, and conversations.

Related Blog Posts

• Clinical Intelligence: Using AI to Improve Patient Care While Managing Risk
• The AI Governance Revolution: Moving Beyond Compliance to True Risk Control
• AI Governance in Healthcare: Privacy Challenges
• AI Is Moving Faster Than Healthcare Governance and That’s the Problem