Why AI Agents Should Not Approve Their Own Source Changes

Proposal is not authorization

An AI coding agent can inspect a repository, generate a plausible patch, explain its reasoning, run tools, and continue working. None of those abilities establish that the agent should be the unquestioned authority that decides its own change may alter durable source.

The distinction is easy to miss because a conversational interface collapses several roles into one voice. The same system proposes the change, describes the evidence, predicts the consequence, asks for permission, applies the mutation, and reports success. Fluency makes that loop feel coherent. Architecturally, it is a separation-of-authority failure.

A proposal answers: what change does the model want to make? Authorization answers: is this exact change allowed to cross this exact boundary under the current policy and state? Application answers a third question: did the system mutate only what was authorized? Those questions should not disappear merely because one model can discuss all three.

The naive agent loop

The naive lifecycle is short:

Agent decides → agent edits source → system moves on

It is attractive because it minimizes friction. It is also brittle because the component with the least reliable view of authority becomes the authority.

The model may have stale repository context. It may misunderstand an invariant outside the files it inspected. A tool result may be incomplete. An instruction may have been compromised. A test may be missing, flaky, or unrelated to the real consequence. The agent may be overconfident about a migration, a security boundary, or an external side effect. Even if its reasoning is good, the source state may have changed between proposal and application.

The problem is not that AI always fails. The problem is that the loop provides no independent place to establish what the system was allowed to do when it did fail.

Why self-approval collapses the control boundary

If the agent creates the proposal and decides that the proposal satisfies policy, then policy has become another model output. If it also summarizes the evidence presented to a reviewer, the reviewer may be approving the agent’s story rather than the concrete mutation. If it applies the change through the same tool path, there is no reliable guarantee that the approved proposal and the applied bytes match.

That collapse creates several distinct failure modes.

Hallucinated authority. The model can infer a permission that was never granted.

Stale context. The proposal can be reasonable for a source snapshot that is no longer current.

Instruction compromise. Prompt injection or malicious repository content can influence both the change and the model’s assessment of the change.

Evaluation error. A green test can be irrelevant, incomplete, or produced against the wrong state.

Narrative substitution. A persuasive explanation can take the place of a bound diff, policy, and evidence bundle.

Unintended side effects. The changed code can affect data, permissions, external services, or deployment behavior beyond the reviewed file.

These failures do not require a malicious model. They emerge from combining generation, judgment, authority, and mutation in one probabilistic loop.

Separate the lifecycle

A stronger lifecycle is longer for a reason:

Proposal → evidence → policy eligibility → approval → deterministic application → audit record → regression → rollback or recovery

The proposal is a structured description of the requested mutation. Evidence identifies the relevant source state, diff, tests, dependencies, and recovery preparation. A deterministic policy gate decides whether the proposal is eligible to proceed. Human or system approval occurs according to consequence. A separate application path checks that the authorized artifact still matches the target state, applies only the bounded mutation, and records what happened. Post-change evaluation determines whether the result should remain in place.

Separating the stages does not make the model less capable. It prevents capability from silently becoming authority.

Evidence must bind to the actual change

“The agent ran the tests” is not sufficient evidence. Which source snapshot was tested? Which patch? Which test command? Which configuration and dependency state? Did the target change between evaluation and application? Did the bytes applied match the bytes reviewed?

A useful evidence bundle binds the request, proposal, active policy, relevant source state, decision, and recovery preparation. Hashes or manifests can make later mutation detectable within that bounded set. They do not prove that the evidence was complete, identify who created it, or turn a local record into an immutable audit system.

This distinction matters. Vague logs preserve a story. Bound evidence preserves a relationship between the decision and the thing that was actually evaluated.

Approval should depend on consequence

Not every typo requires a human approval ceremony. A system that interrupts for every low-risk operation creates review fatigue and trains people to click through. The boundary should consider consequence, reversibility, scope, data sensitivity, external side effects, and the confidence of the available evidence.

A low-consequence formatting correction inside a disposable branch may be eligible for automatic application. A permission change, schema migration, production configuration update, security control, external communication, or mutation to shared durable memory may require explicit human authorization. Some operations should remain outside the agent’s available action set entirely.

Risk-tiered autonomy is not a euphemism for trusting the model more. It is a way to allocate friction where the consequence justifies it.

Rollback should exist before mutation

Recovery is weakest when it is invented after the system has already changed. Before approval, the boundary should identify the state needed to recover, the restore or inverse operation, the person or process allowed to initiate it, and the evidence that recovery has been tested.

Some changes are not cleanly reversible. An external email cannot be unsent. A secret may remain exposed after a configuration is restored. A migration may discard information. Distributed systems may observe intermediate state. In those cases “rollback” can be a misleading promise; the honest requirement is a documented recovery or containment plan.

Rollback readiness is evidence of preparation, not a guarantee of success.

Regression testing after governed changes

Approval does not prove correctness. It establishes permission to proceed under the evidence available at that point. The applied result still needs validation: syntax or schema checks, focused tests, broader regression suites, policy checks, behavioral evaluation, and monitoring for effects that appear only after the change.

The governance layer needs its own tests. Unknown operations should fail closed. Missing evidence should not become permission. Stale source state should invalidate the decision. Repeated application should be detectable or idempotent. Concurrent proposals should not race past the same boundary. An interrupted mutation should leave a state the recovery process understands.

A controlled change can still be wrong. The system is stronger because it knows how that wrongness will be detected and what happens next.

What CLU explores

CLU Governance is an open-source, local-first preflight gate that explores this separation for AI-proposed source changes. Its core accepts structured input, evaluates it under a deny-by-default policy, and returns a structured eligibility decision before any separate approval or application step. It binds the request, proposal, policy, source state, decision, and rollback-readiness evidence with hashes.

CLU deliberately does not approve, apply, stage, commit, or push source changes. Its public repository includes an experimental Claude Code adapter for a narrow existing-file edit workflow, tests, architecture and decision records, and explicit claims-and-limitations documentation.

That adapter makes the separation concrete: a CLU policy allow maps to an instruction to ask for permission, never to automatic application. In the gate evidence itself, mutation_authorized and mutation_applied remain false.

CLU is pre-alpha. It is not an authenticated approval system, immutable audit log, signed-provenance service, sandbox, security certification, or guarantee of rollback. Those limits are not a footnote to the architecture; they define what the current evidence can support.

The CLU source repository is public for inspection.

Beyond source code

The same separation applies wherever model output can become durable consequence.

Infrastructure and configuration. An agent can propose a change, while policy and approval remain outside the planning loop.

Persistent AI memory. A model can suggest a memory update without gaining unilateral authority over the context that will influence its future decisions.

Permissions. A recommendation to grant access is not the authorization to alter the access-control system.

Financial actions. A proposed transaction requires controls that are independent of the model’s confidence. This is an architectural principle, not financial advice.

External communications. A draft can be useful without giving the drafting system final authority to speak for a person or organization.

The specific policy changes by domain. The separation of proposal, authority, application, evidence, and recovery remains useful.

The durable boundary

The right question is not whether an AI agent is smart enough to edit source code. Many are. The question is whether the system can establish what was proposed, what policy and state were evaluated, who or what authorized the consequence, whether the applied change matched that authorization, and how the result will be tested or recovered.

An agent may generate the best proposal in the system. It still should not be the only authority that decides its own proposal may become durable reality.

For the commercial engineering perspective, see AI Governance Engineering. For authorship and the wider work behind Arjia, see About Arjia Technologies.