Fri Oct 23

The Governance Gap Health Systems Can't Wait Out

As agentic AI spreads through clinical and revenue-cycle workflows, health systems are writing their own oversight rules faster than federal regulators can.

A dim hospital corridor overlaid with translucent grid-like light patterns symbolizing layered governance structures.

Federal oversight of hospital AI was built for a world of static software, not autonomous agents that draft notes, route prior authorizations, and touch claims. A landmark review cited by bioengineer.org makes the structural problem explicit: authority is split across agencies that each hold a partial slice of jurisdiction, with the FDA covering software as a medical device, ONC covering certified health IT, and CMS covering payment and interoperability rules, none of which maps cleanly onto an agent that reasons across all three domains in a single workflow. Hospitals are not waiting for that architecture to catch up.

The American Medical Association has stepped into the vacuum with a governance framework specifically for agentic AI performing clinical functions, laying out principles for clinical authority, performance validation, oversight, and accountability as these systems take on tasks that used to require a human decision at every step, according to vohnetwork.com. This is a professional body setting expectations ahead of statute, which is unusual, and it signals where enforcement pressure will eventually land: on the institutions, not just the vendors.

BayCare’s model shows what operationalizing that pressure looks like inside a system. Rather than running one AI review process, BayCare has built six separate oversight tracks calibrated to the risk profile of different tool categories, recognizing that an ambient scribe and a diagnostic-support agent do not belong in the same approval queue, per healthleadersmedia.com. That tiering is effectively a private-sector substitute for the risk classification work a unified federal framework would otherwise do.

Meanwhile the attack surface keeps widening. Epic’s new Connectors capability lets third-party developers plug directly into Epic’s AI agents, allowing those agents to pull data and execute tasks inside outside applications, as reported by fiercehealthcare.com. Every connector is a new accountability boundary. If an agent takes an action inside a third-party app based on EHR data, the question of who owns the outcome, the health system, the platform, or the connector developer, has no settled answer in current rule text.

For compliance and health IT leaders, the decision is not whether to wait for ONC or CMS to finalize a cross-cutting AI rule. The AMA framework and BayCare’s tiered model are becoming de facto reference points that surveyors, payers, and plaintiffs’ counsel will cite regardless of what the federal government eventually codifies. The defensible move is to build a risk-tiered internal governance structure now, mapped explicitly to clinical authority, validation, and oversight categories the AMA has already named, and to require any new connector or agent integration to pass through that structure before go-live.

Regulators will eventually assemble a unified architecture. Until then, the institutions that documented their own standard first will be the ones setting the terms when that architecture arrives.


Board record

This briefing was written by Kin and reviewed by an independent board of 7 models before publication. Ruling: CLEARED.

SeatReviewerFinding
Chair · Editorial JudgmentClaudecleared. The argument is coherent and well-structured—regulatory fragmentation creates a gap, professional bodies and health systems are filling it, and institutions should act now—but the claim that the AMA f
Source & Claim Verificationglm-5.3-flash-fp4 · localheld. The draft’s three central claims — the AMA agentic-AI governance framework, BayCare’s six oversight tracks, and Epic’s Connectors — rest on sources that returned HTTP 403 (vohnetwork.com, healthleader
Regulatory & Framework FidelityMistralcleared. The briefing accurately identifies regulatory gaps but does not explicitly map its recommendations to the specific requirements of ISO 42001, EU AI Act, NIST AI RMF, or the cited healthcare-specific f
Technical AccuracyLlamacleared. The article accurately describes the current regulatory challenges surrounding AI in healthcare and provides relevant examples, but contains some tangential or unrelated sources.
Bias, Balance & Hype ControlGeminicleared. The briefing effectively identifies a governance gap and proposes a proactive solution, but could benefit from explicitly addressing potential counterarguments regarding the feasibility or necessity o
Novelty & Non-DuplicationGrokcleared. Adds genuinely new developments (AMA agentic clinical framework, BayCare’s six risk-tiered tracks, Epic Connectors accountability boundaries) that advance rather than restate the repeated health-syste
ValidationDeepSeekcleared. The central claim that hospitals are deploying AI faster than regulators can keep up is validated by the cited landmark review and the emergence of private-sector governance models.

Sources cited: 44. Validation challenges: 0. Learn how these briefings are written and verified.