The Boardroom Challenge at the Center of Healthcare AI

If AI can affect a diagnosis, a clinical note, or a claim, I see it as a board issue, not just an IT issue.

Here’s the short version: healthcare AI now creates risk across patient safety, privacy, security, compliance, vendors, and accountability. The article’s main point is simple: boards need a formal AI governance model before AI use spreads further across the health system.

A few facts make that clear:

• ECRI listed weak AI governance among the top patient safety concerns for 2025
• One 2025 study found 6% of AI-generated patient portal messages included a hallucination
• That same study found 7.1% posed a severe risk of harm
• A 2025 IHI review said hallucinations can also appear in clinical notes, not just chatbot replies

If I were briefing a board, I’d boil the article down to this:

• AI risk does not fit old oversight models
• Different AI tools create different levels of risk
• The board sets risk limits; management runs the controls
• Every AI tool should face approval before go-live
• Vendor AI needs the same scrutiny as in-house AI
• Boards need dashboards, named owners, and set escalation paths

The article also lays out what boards should ask management to prove:

• What controls are in place
• Whether those controls work
• Who owns failures and fixes

At its core, this is not about model design. It’s about whether a health system can show, in writing, that it knows:

• what AI tools are in use
• where patient data goes
• which use cases are high risk
• how performance is checked over time
• when issues must be escalated

A simple way to read the full piece is this: inventory every AI tool, tier the risk, validate before use, monitor after launch, and report the results to the board in a form directors can act on.

Health Care Corporate Governance: Critical New AI-Related Issues for Health Care Boards

sbb-itb-535baee

The Problem: AI Risk in Healthcare Does Not Fit Existing Oversight by Default

Most health systems didn't roll out AI through one big, coordinated decision. It usually happened piece by piece. Physicians started using ambient documentation scribes. Revenue cycle teams brought in coding automation. Vendors slipped AI into EHR upgrades. IT launched chatbots. Each move went through its own workflow, approval path, and oversight standard.

That creates a fractured setup: no single owner has the full view, no shared inventory, no steady approval standard, and no plain escalation path. And if a board can't see the whole picture, it can't govern it. That's what makes AI risk so hard to spot, track, and control.

This matters because AI doesn't behave like ordinary software. Standard IT governance assumes outputs stay the same unless the code changes. AI doesn't play by that rule. Generative models can give different answers to the same input. Predictive models can drift as patient populations change, even if no one touches the software. A small tweak to input fields can shift a recommendation without setting off normal change control. Old-school control testing won't catch model drift or prompt-driven variation. And when those outputs shape a clinical note, a triage call, or a billing submission, the fallout can hit clinical care, legal exposure, and regulation all at once.

Five Risk Domains the Board Must Oversee Together

AI risk in healthcare doesn't belong to one department. It runs across five connected domains, and boards need to treat them that way. A weak control in one area can spill into another fast.

Patient safety and clinical quality is the most direct point of exposure. Diagnostic algorithms can miss findings or repeat biased patterns. Predictive models for sepsis or readmission can classify risk the wrong way and change care paths.

Cybersecurity and PHI protection is the second domain. Cloud-based scribes and coding tools take in audio, notes, and structured EHR data. That creates new PHI processing points and opens up more attack surface.

Regulatory and accreditation exposure is the third. AI tools that work like a medical device can trigger FDA oversight. Documentation or coding AI that affects billing can create False Claims Act exposure. Boards need to make sure AI governance lines up with CMS, Joint Commission, and state board expectations, not just internal IT policy.

Vendor and fourth-party dependence is the fourth. If a health system relies on an EHR vendor's embedded AI module, the vendor may control model updates, but the health system still carries the risk.

The fifth is organizational accountability. Someone has to own decisions, failures, and fixes. If AI plays a part in a missed diagnosis or uneven care, the institution takes the legal, moral, and reputational hit, no matter which vendor supplied the model.

Why Common AI Use Cases Carry Different Levels of Risk

Not every AI tool creates the same level of board exposure. Treating them all the same is a governance mistake on its own.

Generative AI scribes and documentation tools can place errors straight into the legal medical record. They also bring PHI handling issues tied to audio capture and cloud processing. AI-assisted coding tools bring billing risk at scale: mistakes can spread across many claims before a payer audit finds them, and fixing the damage may mean claim reprocessing and regulatory disclosure.

Diagnostic and predictive models carry the highest clinical stakes. They can show bias against certain demographic groups, lose accuracy over time, and feed automation bias, where clinicians trust the model too much and miss errors their own judgment might have caught.

Vendor-supplied workflow AI for scheduling, throughput, staffing, and revenue cycle adds operational and contract risk. The algorithms may be opaque. Audit rights may be thin. Integration failures can disrupt care delivery with no warning.

Boards need tiered approval standards that fit the level of clinical impact and regulatory exposure. One-size-fits-all approval sounds neat on paper, but it breaks down fast in practice.

Board Oversight vs. Management Execution

The board sets risk tolerance. Management has to show that controls work.

In practice, that means the board defines risk appetite, approves governance policy, asks for structured reporting, and requires remediation plans when controls fail. Management turns that into day-to-day action by validating use cases, running clinical and cybersecurity assessments, monitoring performance, and handling incidents.

The Health Sector Coordinating Council recommends that boards receive regular briefings on AI cyber risk posture, regulatory trends, and incidents and may need to make annual attestations to AI cyber governance policies within corporate compliance statements.^[3][4]

Clear role separation helps close accountability gaps. If the board gets pulled too far into operating detail, it loses independence. If management doesn't escalate material AI issues, directors can't meet their fiduciary duty. A clean split, backed by set escalation thresholds, is what makes AI governance easier to defend when something goes wrong. That's why healthcare AI needs formal governance, named decision rights, and approval standards before adoption spreads across the organization.

The Solution: Build a Formal AI Governance Model Before Scaling Adoption

Fragmented AI rollouts have a direct fix: build a governance model with named owners and one approval standard for every AI tool.

Then turn that model into clear approval rules, monitoring, and escalation paths. That’s how a health system balances patient safety, cybersecurity, regulatory exposure, vendor dependence, and accountability across the organization.

Create an AI Governance Structure With Named Decision Rights

An effective AI governance structure in a U.S. health system has three parts: a committee, written decision rights, and links to current oversight bodies.

The committee should include leaders from clinical care, security, privacy, legal, compliance, analytics, and operations. That way, safety, security, legal, and day-to-day workflow risks get reviewed together. It also helps keep AI decisions from being boxed into IT or data science alone.

Decision rights need to be written down in a board-approved charter or AI governance policy. That charter should spell out who can:

• Approve a new AI use case
• Pause or roll back a deployment when risk thresholds are crossed
• Escalate AI-related safety incidents
• Own revalidation after a model or workflow change

If those answers aren’t in writing, gaps in accountability show up fast.

The committee should connect to current committees, not replace them. AI-related patient safety issues should feed into the Quality and Patient Safety Committee. Cybersecurity concerns should go to the Information Security Council. Material enterprise exposure should move to the Board Risk or Audit Committee. That link keeps AI governance tied to the way the organization already makes decisions.

Set Approval Criteria Before Any AI Tool Goes Live

Every AI tool, including EHR-embedded features, should pass a pre-deployment checklist.^[2][6] This is the control set the board should require before any AI tool enters workflow:

• Documented use case and intended benefit, including the expected effect on quality, efficiency, or staff burden
• Local validation on the health system's own data, with clear performance metrics such as sensitivity, specificity, and calibration
• Bias and safety review with stratified analysis across race, ethnicity, sex, age, and key clinical subgroups
• Human-in-the-loop workflows that keep licensed clinicians responsible for final decisions
• Security and privacy assessment coordinated by the CISO and privacy office against HIPAA Security Rule requirements
• Fallback procedures so care is not disrupted if the AI tool goes down
• Audit logging of inputs, outputs, overrides, and model versions
• Named owners for ongoing performance and safety monitoring

This checklist gives the board something concrete it can review and audit.

Adopt a Risk Framework the Board Can Actually Monitor

The NIST AI Risk Management Framework (AI RMF 1.0) gives boards a practical structure built around four functions: Govern, Map, Measure, and Manage.^[1][5] NIST places Govern and Map before Measure and Manage, which means boards need to define decision rights and build an inventory first.

In healthcare, that translates pretty cleanly. Govern means a board-approved AI policy and defined decision rights. Map means a live inventory of every AI system, sorted by risk level and linked to rules such as HIPAA and FDA device rules. Measure means tracking performance, bias, safety events, and cybersecurity posture over time. Manage means the controls, approvals, pauses, and remediation plans that kick in when thresholds are crossed.

The Joint Commission’s guidance lines up with this structure. It calls for AI policy and governance bodies, ongoing quality monitoring, risk and bias review, and reporting of AI safety events, all of which fit the NIST RMF categories.^[7][8] Once that framework is in place, the board can apply it across vendors, high-impact workflows, and continuous monitoring.

What the Board Must Decide About Vendors, Controls, and Ongoing Monitoring

Vendor AI needs its own board review. Why? Because the vendor controls updates, data handling, and security, while the health system still carries the clinical and legal risk. So contract terms, change notices, and escalation rights aren't just procurement fine print. They're board-level calls.

Approval Standards for Third-Party and Vendor-Supplied AI

A vendor saying it's HIPAA-compliant is not enough. Boards should ask for documented proof of AI-specific security, privacy, and data-flow controls. That means controls for prompt injection, API abuse, data leakage, and cloud dependencies.^[9]

The board should also expect a clear PHI data flow map that shows:

• what patient data goes into the model
• where that data is stored
• whether the data is used for further training
• how long the data is kept

Data handling is only part of the picture. Boards also need model update transparency locked into the deal. Vendors should have to notify the organization when models, training data, or major settings change. That notice should include versioning, release notes, and impact assessments.

BAA terms for AI data use, updates, incidents, and liability should be non-negotiable. The same goes for audit rights, including access to SOC 2 reports and penetration test results. Contracts also need to spell out incident response, liability, and remediation when AI fails.

Controls for High-Impact Use Cases Such as Generative Documentation and Diagnostic AI

Once the board approves a use case, the controls need to fit the workflow. A note-writing tool and a diagnostic model don't create the same kind of risk, so they shouldn't be governed the same way.

For generative documentation tools - AI that drafts progress notes, discharge summaries, or procedure reports - the board should require tightly defined and limited approved use contexts. A licensed clinician must review and attest before any AI-generated text goes into the legal record. Periodic quality audits of a statistically meaningful sample of AI-assisted notes should follow, with results reported to the AI governance committee. That gives the board a direct feedback loop instead of a guess.

For diagnostic and predictive AI - imaging models, sepsis risk scores, readmission predictors - the bar should be higher. Vendor diagnostic models need local revalidation after updates and at least once a year. Clinicians must be free to override AI recommendations without penalty. The organization should also track model drift over time using outcome-based metrics. High-impact models should go through annual reapproval, with updated validation data reviewed by a multidisciplinary committee and then summarized for the board.

Use Centralized Risk Operations to Support Oversight

Centralized risk operations help keep this from turning into a mess of spreadsheets and one-off reviews. Use one inventory, standardize third-party assessments, and route exceptions to the right owners.

At a minimum, keep each AI tool's risk tier, owner, data type, and current posture in one place. Those records become the basis for the board metrics in the next section.

Board Reporting: The Metrics, Dashboards, and Decisions That Prove Accountability

Once approval standards and controls are in place, the board needs a simple reporting rhythm to make sure those controls still do their job. After AI tools go live, directors need proof that controls are holding up, risks are shifting in the right direction, and someone is clearly on the hook when issues show up.

The Core Metrics Every Board Report Should Include

A useful board report should track both adoption and risk.

On the adoption side, directors need to see how many AI systems are in production and which risk tier each system falls into. On the risk side, the report should show open validation gaps, safety incidents, near misses, documentation error rates, privacy or cybersecurity events, model drift signals, vendor-related issues, and time to remediation for corrective actions.

That last metric matters more than it may seem at first glance. If high-risk findings stay open across several reporting cycles, that's a warning sign. It usually means the governance process isn't moving fast enough to match deployment. Trend lines help here. A smaller backlog and shorter closure times show a process that's getting stronger. A growing backlog shows the reverse.

How to Present AI Risk to Directors in a Usable Format

Directors need a concise dashboard tied to a live AI inventory. At a minimum, that inventory should show system name, owner, risk tier, data sources, validation status, approval date, next review date, and unresolved findings. When that information sits in one place, ownership, approvals, and review dates are hard to miss.

The dashboard should group findings into the same three areas the board governs: safety, security, and compliance.

Conclusion: The Board's Role in Responsible Healthcare AI

AI governance in healthcare is a fiduciary issue, not a technology project. When boards treat it like a tech rollout, they can end up answerable for decisions they never formally made. The operating model needs to stay clear: the board sets the oversight structure and risk appetite, management meets defined approval and monitoring standards, and third-party AI tools face the same scrutiny as anything built in-house.

Defensible governance rests on evidence. Directors should be able to see who owns each system, its current risk status, what has been approved, what is still unresolved, and when the next review is set. Named accountability, measurable reporting, and documented escalation paths make AI oversight something the board can stand behind.

FAQs

Why is healthcare AI a board issue?

Healthcare AI is a board-level issue because it touches the risks that matter most to the business: patient safety, legal liability, cybersecurity, and regulatory compliance. When AI goes wrong, the damage isn’t abstract. It can lead to clinical harm, data breaches, and fines or other regulatory action. That’s why boards need to provide fiduciary oversight.

Board governance should center on accountability, vendor oversight, and continuous monitoring. In plain terms, that means every AI tool needs a clear owner, it needs to sit inside formal risk management processes, and it needs to be watched over time for problems like bias, drift, and performance decline.

How should boards prioritize high-risk AI tools?

Boards should use a tiered risk framework built around clinical impact, data sensitivity, and workflow integration. If an AI tool directly shapes diagnosis, treatment, or triage, it should be treated as high-risk and face a deeper review from a multidisciplinary group.

Before approval, those tools need local validation, bias testing across demographic subgroups, and careful vendor due diligence. Once deployed, boards should also require ongoing monitoring for model drift and performance decline, along with clear escalation triggers and executive accountability.

What should boards require from AI vendors?

Boards should ask for more than the usual software terms. AI tools can affect patient care, so the bar needs to be higher.

Vendors should share clear documentation, such as the CHAI Applied Model Card, that explains:

• where the training data came from
• how the model performs across demographic subgroups
• where the model tends to fail

Contracts should also spell out a few non-negotiables: notice of material changes, audit rights, and firm limits on using patient data to train models.

On top of that, boards should require local validation before broad use. They should also keep the power to suspend or roll back a tool without penalty if performance drifts or safety risks start to show.

Related Blog Posts

• Board-Level AI: How C-Suite Leaders Can Master AI Governance
• Boardroom to Bedside: Making AI Governance Everyone's Responsibility
• What ANSI/HSI 2800:2025 Means for Healthcare Executives and Boards
• Why Board-Level Oversight Is Now Central to Healthcare AI Governance