How Should Enterprises Control AI Agents While They Are Running in 2026?

The Direct Answer to Runtime Governance

Enterprise AI runtime governance is the set of technical and organizational controls applied while an AI model or agent is producing an answer, calling a tool, changing data, or initiating a transaction. It closes the gap between approving a model during development and observing what the deployed system actually does in production. That distinction matters because an agent can select a different data source, generate a harmful action, exceed a token budget, or operate under a different identity every time it runs. A static model card or one-time compliance review cannot reveal all of those events.

Also worth reading: How Can Enterprises Control AI Gateway Costs Without Slowing Agent Development? · What Is an Agentic AI Control Plane, and How Do Enterprises Choose One? · What Is Runtime Agent Security, and How Should Enterprises Defend AI Agents in 2026?

A useful runtime control system must place explicit limits around identity, data access, tools, destinations, spending, latency, autonomy, and evidence retention. It should also define who can approve an exception and how quickly a team can stop an agent. The governing policy should be enforced at execution time rather than trusted to prompt text alone. However, runtime governance is not automatically a new platform category that every enterprise needs: a low-risk internal assistant with read-only access may justify lighter controls than an agent that can transfer money, modify customer records, or execute production code.

The practical objective is controlled autonomy, not unrestricted agent activity. As of September 29, 2026, enterprises are moving in that direction through agent firewalls, runtime data controls, identity-aware proxies, policy engines, and auditable execution layers. Collibra, Fastly, F5, Skyflow, NVIDIA, SAP, and several smaller vendors are all addressing parts of this problem, but their scopes differ. An organization should therefore evaluate the decisions it needs to govern instead of assuming that buying an “agent security” product supplies a complete operating model.

How Runtime Governance Differs from Pre-Deployment Approval

Pre-deployment governance asks whether a model, use case, and proposed architecture are acceptable. Runtime governance asks whether a particular invocation remains within the boundaries assigned to that use case. It examines the effective prompt and context, the authenticated user, the model version, retrieved information, tool calls, outbound network requests, files accessed, actions attempted, and resulting side effects. It can then apply a decision such as allow, redact, require human approval, reduce autonomy, or terminate execution.

This shift is necessary because the same agent may behave differently under different conditions. A support agent approved to look up an order may later attempt to issue a refund because a customer’s phrasing resembles an authorized request. A coding assistant connected to a repository may read one file during testing and deploy a change during production. A retrieval system may receive public documents during acceptance testing and sensitive customer records after launch. These are runtime variations that development-stage review cannot reliably enumerate in advance.

Runtime governance also assigns decision ownership more precisely. Model risk teams may own classification and approval thresholds, data owners may control access, security teams may own containment, and business owners remain accountable for the action. That allocation is more useful than a generic statement that “AI is governed.” It lets an organization answer four operational questions for every consequential decision: who authorized it, which policy applied, what evidence was retained, and who can revoke the authorization?

The industry’s move toward executable policies is therefore meaningful, but it does not eliminate model evaluation or pre-release testing. Those activities estimate baseline behavior and expose unsafe design choices. Runtime governance handles the residual risk created by changing inputs, changing tools, changing data, and changing external systems after approval. Mature programs use both stages together.

The Controls an Enterprise AI Runtime Must Apply

Identity is the first control layer because an agent should never inherit an administrator’s implicit permissions merely because an engineer connected it during development. Each agent, user session, and delegated task should have a distinct identity with least-privilege access. For example, a reporting agent might read approved datasets but not export them, while an invoice agent might prepare a draft but not post it to the general ledger. Temporary credentials should expire when a task ends, and service accounts should not be shared across unrelated agents.

Data controls must consider both the prompt and the agent’s behavior. Organizations need rules for permitted data sources, jurisdictions, retention periods, masking, and prohibited fields. Skyflow’s 2025 announcement around runtime data control for Glean illustrates the commercial attention being directed at data flowing through enterprise AI systems, but technology selection remains secondary to policy design. A company that cannot state which data classifications are permitted in a particular workflow cannot expect a gateway to infer that answer correctly.

Tool and network controls determine what an agent can affect. A governed runtime can maintain an allowlist of approved APIs, validate parameters before invocation, restrict outbound destinations, and block commands or files associated with malicious instructions. It should distinguish read operations from write operations and assign higher approval requirements to irreversible actions. Useful thresholds include a maximum tool-call count, a wall-clock execution limit, a token or spending ceiling, and a maximum number of records that can be changed in one transaction.

Evidence and response controls complete the model. Each important run should record the policy version, model version, prompt or prompt hash, relevant context references, tool arguments, approval events, and action results. Logs should be synchronized to an enterprise security platform while sensitive content remains appropriately protected. If a policy engine is unavailable, a conservative runtime may fail closed for high-impact actions, although the enterprise should test whether that behavior creates an unacceptable business outage.

A Practical Rollout Plan for Governance Teams

Begin with one workflow that has measurable value and bounded permissions. Good candidates include customer-service knowledge retrieval, internal policy search, or preparation of sales reports. Avoid beginning with an open-ended agent that can browse the internet, write to production systems, and communicate externally. Document the business owner, data classes, approved tools, expected latency, maximum cost per run, and actions requiring human approval. The initial objective should be evidence gathering and containment, not proving that the agent can operate independently.

Create a risk-based action matrix with at least four levels: allow, monitor, human approval, and deny. For example, reading a public knowledge article could be allowed, accessing an internal document could require monitoring, sending an external email could require human approval, and changing a production firewall could be denied. Establish numerical thresholds before production, including perhaps 100 tool calls per task, a 60-second response target, or a fixed dollar ceiling. These numbers should come from workload testing and business impact, not an arbitrary industry benchmark.

Next, enforce the controls independently of the agent framework. A prompt instruction such as “never disclose confidential data” is helpful documentation but is not a reliable security boundary. Put enforcement in infrastructure the model does not control, such as API gateways, identity brokers, database permissions, policy decision points, and sandboxed tool services. Pilot the design with red-team tests involving prompt injection, indirect instructions in retrieved documents, credential theft, oversized responses, loop behavior, and unauthorized tool invocation.

A 60- to 90-day pilot can produce useful evidence, although serious regulated deployments may require longer. The team should measure blocked actions, false approvals, policy latency, cost variance, incident response time, and the percentage of runs with complete audit records. If more than 10% of routine executions produce unresolved policy errors, or if the gateway adds more than 500 milliseconds to a latency-sensitive workflow, those figures may justify redesign. Those are operating targets, not universal standards. Production approval should depend on measured residual risk rather than elapsed time alone.

Comparing the Main Architecture Choices

Enterprises generally have four implementation routes, and they are not mutually exclusive. A managed platform can accelerate deployment but may provide less control over execution internals. A governance gateway is well suited to inspecting model and tool traffic but may not replace domain-level authorization. An open-source runtime offers customization at the cost of engineering and support work. A custom internal layer provides maximum integration but creates long-term maintenance obligations.

Hybrid designs are often the sensible result. An enterprise may use a managed agent platform for orchestration, a gateway for network and tool controls, and internal data platforms for authorization and evidence. The comparison should therefore evaluate control placement, not just product labels. A cheaper license can become expensive if every exception requires manual review, while an expensive platform can be poor value if it cannot cover a regulated data system or preserve required records.

Open-source runtimes inspired by modern application frameworks can make policy more programmable. The cited Rust and TypeScript agent-runtime projects show active experimentation with developer experience, while mesh-based control planes and emerging agent contract models point toward decentralized policy coordination. These approaches are promising but still carry integration risk. Enterprises should examine maintenance activity, release stability, identity support, observability, test tooling, and commercial support before assigning a bank workflow to a young codebase.

Common Mistakes That Produce False Assurance

The most frequent mistake is treating the system prompt as a security policy. Language models can misunderstand, ignore, or be induced to disregard textual restrictions, so critical controls belong in deterministic systems. A second mistake is assuming that an LLM-based judge can provide dependable enforcement for actions with legal or financial consequences. Model-based classification can assist triage, but deterministic rules and human authorization should govern the highest-impact paths.

Another error is applying governance only to the model endpoint while leaving connected tools unconstrained. Even a perfectly controlled model can cause damage through an overpowered API credential. Enterprises should use short-lived credentials, scoped tokens, parameterized tools, destination restrictions, and transaction limits. They should also avoid giving an agent a general shell, unrestricted database client, or production administrator account merely to simplify integration.

Teams also underestimate auditability. Recording that an agent ran is not the same as recording the evidence necessary to reconstruct it. Logs can omit raw prompts for privacy reasons, yet authorized investigators still need hashes, references, tool arguments, model settings, policy decisions, and outcome identifiers. The retention design should specify which artifacts are immutable, where they are stored, who may access them, and whether clocks and timestamps can support an investigation across services.

Finally, many organizations buy a gateway and declare governance complete. Technology cannot decide whether a permitted action is appropriate for the customer, jurisdiction, or contract. It cannot assign accountability or resolve conflicting legal requirements. Effective runtime governance combines technical enforcement with named owners, approved procedures, tested incident response, and periodic review of exceptions.

When to Act and How Expensive It Is

An enterprise should act before an agent receives production credentials, writes data, communicates externally, or handles regulated information. Waiting for a visible incident creates little advantage because the important control is preventive. Evaluation can begin during a sandboxed proof of concept, but the same threat model, identity model, and audit plan should be designed before connecting business systems. If an agent will make more than a few low-risk recommendations per day, runtime controls are already a more credible option than informal review.

There is no single market price for enterprise AI runtime governance as of September 29, 2026, and public list prices are often obscured by negotiated contracts. A planning range for an initial commercial gateway or platform deployment can be roughly $25,000 to $250,000 annually, driven by seats, protected model traffic, connected tools, environments, support, and compliance requirements. Implementation may add $50,000 to $500,000 or more, especially where identity, data discovery, and legacy APIs require integration. These are budgeting ranges, not vendor quotations.

Open-source components may reduce direct license expense, but they are not free. A small platform team may still need several engineers for policy integration, testing, observability, upgrades, and 24/7 operations. Regulated enterprises can also incur costs for evidence retention, independent assurance, data residency, and control testing. A sensible first-year allocation may therefore reserve 40% to 60% of the budget for integration and operations rather than assuming that the software subscription is the dominant cost.

A useful business case compares avoided loss and review cost with measurable operating expense. Include the reduction in manual authorization, the number of blocked unsafe actions, audit preparation time, and per-task runtime cost. Do not value prevented incidents solely from an extreme worst-case estimate; use plausible scenarios and document assumptions. If a team cannot estimate either its token expense or its review burden, it is not ready to choose an autonomy threshold.

The Recommended Governance Standard

The appropriate standard for 2026 is not “zero human involvement.” It is bounded, observable, and revocable autonomy. Low-impact decisions can proceed automatically, medium-impact decisions can receive randomized or threshold-based review, and high-impact actions should require explicit authorization or remain prohibited. Thresholds should be tied to concrete factors such as monetary amount, number of affected records, data classification, destination, reversibility, and the agent’s confidence.

A mature program should be able to show a complete chain from policy to execution. For every consequential action, an investigator should identify the responsible business owner, authenticated principal, model and prompt versions, permitted data sources, tool authorization, applied policy, exception, and final outcome. The same controls should work during normal operations, regional outages, model-provider changes, and emergency revisions. That last test is important because governance that works only under ideal conditions may fail precisely when the environment is under pressure.

The strongest near-term pattern is layered enforcement: identity at the foundation, least privilege at each tool, runtime policy at the gateway, human approval at defined boundaries, and immutable evidence across the stack. Enterprises should pilot that pattern against one bounded use case, measure it for 60 to 90 days, and expand only after failures are understood. Runtime governance should earn trust through measurable restrictions and reliable evidence—not through branding, a large feature count, or an unsupported claim that autonomous AI is inherently safe.