AI Agent Security Governance Must Come Before Scaling

AI Agent security governance is lagging behind agent expansion. Deloitte’s 2026 State of AI in the Enterprise (subtitle: The Untapped Edge) gives two sets of figures: 23% of companies currently use agentic AI at least moderately, and that figure is expected to reach 74% within two years. Only 21% say they already have a mature governance model for autonomous agents. The report titles this section “AI agents are scaling faster than the guardrails,” meaning agents are expanding faster than their safeguards.

The report also answers two specific questions: why agent governance cannot use the existing approach, and which specific requirements define “mature.”

The human review step has disappeared from the process

Page 20 of the report says traditional AI systems provide recommendations for people to consider. Agents act directly, for example by placing orders, sending communications, and modifying systems.

The difference is where an error can be stopped, regardless of system capability. With recommendation-based AI, the sequence is: the model makes an error, a person sees it, the person decides against using the recommendation, and no consequences follow. Human review can stop the error even if nobody deliberately designed it into the process. With an agent, the sequence is: the model makes an error, the agent executes the action, consequences occur, and a person learns about them afterward. At the same error rate, the first process causes no loss; in the second, the loss has already occurred.

Governance therefore needs to move from reviewing outputs afterward to defining boundaries in advance, monitoring in real time, and maintaining full traceability afterward. People need to design all three stages because the process no longer has a default human review step to catch errors.

The report defines “mature” through four requirements you can check

Deloitte provides no quantitative scoring rubric. It specifies the capabilities mature governance needs:

1. Clear boundaries: clearly define which decisions an agent can make independently and which require human approval.
2. Real-time monitoring: track agent behavior and flag anomalies.
3. Complete audit trails: capture the full chain of agent actions for accountability and continuous improvement.
4. Cross-functional structure: IT, legal, compliance, and business unit leaders jointly set policies, monitor performance, and handle escalations.

These four requirements can each be assessed by asking, “Do we have this?” They are easier to verify than a phrase such as “strengthen controls.” The fourth is easiest to skip because it is not a system feature and cannot be assigned to a specific ticket. The report’s recommendations say companies whose senior leaders directly help shape AI governance achieve significantly more business value than those that leave it entirely to technical teams. Drawing the boundaries requires business and legal judgment. Technical teams cannot make that decision and should not be solely responsible for it.

The 21% is self-reported

The 21% comes from respondents’ self-assessments, with no third-party audit. Deloitte surveyed 3235 IT and business leaders, from directors to C-suite executives, across 24 countries from August to September 2025. IT and business respondents were roughly evenly split. The survey captured their judgments about their own organizations. The report did not independently verify the actual governance of any organization.

The same report provides two comparison figures: 42% of companies consider themselves highly ready on strategy, and 30% consider themselves highly ready on risk and governance. When respondents decide for themselves what counts as “mature” or “ready,” their answers tend to be optimistic. The 21% is therefore more likely to be an upper bound than a lower bound.

Another set of figures shows that most of the AI risks companies worry about concern governance: data privacy and security, 73%; legal matters, intellectual property, and regulatory compliance, 50%; governance capability and oversight, 46%; model quality, consistency, and explainability, 46%. These concerns show that the issues remain unresolved.

Agent governance does not mean longer approval chains

Requiring approval for every agent action would go to the other extreme. The report does not support that approach. Its recommendations list three points:

• Integrate governance into existing risk and oversight structures. Avoid creating a separate “shadow” function that runs in parallel.
• Establish clear, adjustable guardrails so teams can move quickly and safely, without adding bureaucracy.
• Balance governance between risk management and innovation so oversight allows experiments to proceed without blocking them.

The report finds that the best-performing companies take a measured approach: start with low-risk use cases and build governance capabilities as they expand. They avoid both large-scale rollout followed by retrofitted governance and waiting for a complete governance system before launch. The first leaves risk exposure; the second keeps the company in pilots indefinitely.

A practical test is whether your governance design makes the team more or less willing to put agents into production. If it makes them less willing, governance is probably in the wrong place. Approval chains are being used for work that boundary definitions and audit trails should handle.

Where to start

For most teams today, agents are already running. The rule defining which actions they can take independently and which need human approval has never been written down. It exists only in individual developers’ judgments, and nobody has treated it as a decision that needs to be settled. To check, ask the team that question. If they cannot produce within ten minutes a list that both legal and business leaders approve, the boundaries have not been defined.

The recommended order is to write down the boundaries first. A two-column table is enough: actions that may be executed independently / actions that require human approval. Add audit trails next, because reconstructing an incident is a prerequisite for all follow-up actions. Then add real-time monitoring. Finally, formally bring legal and business leaders into the process. The first two steps cost little and address most concerns about putting agents into production.

First identify which agents are running in the company and who is responsible for each. For the method, see Agent Management Starts With an Inventory: Who Runs Employee-Built Agents?.

This is what the first tier of Baize AI’s services addresses: first connect AI in a stable, controlled, auditable way, then turn it into engineering capability and products. If that order is reversed, faster expansion brings greater risk.