The missing boundary between improvement and permission
The previous article defined a continual loop: Run, Measure, Evidence, Risk, Compliance, Approval and Promotion. This article isolates its most consequential transition. An optimizer can suggest a new prompt, retrieval strategy, model configuration, skill or workflow. None of those suggestions grants permission to change what a production agent is allowed to do.
Promotion answers a narrower question than “is this version better?” It asks whether this identified capability, within these conditions, may replace or supplement the currently qualified one. Its inputs include measured performance, but also constraints, authority, known failures, reversibility and observed operational effects.
The primary architecture anchor is L2 Governed Continual Harness. It has two explicit interfaces: L1 Strategy & Portfolio can approve investment and business commitment, while L7 Evaluation & Observability supplies independently assessable qualification and runtime evidence. Neither investment approval nor an evaluator's score can silently confer L2 promotion authority.
Promotion is a state transition with a controlled object
A workable implementation distinguishes at least six states:
- Candidate: a versioned change proposal exists and its intended effect and changed surfaces are identified.
- Evaluated: bounded tests produced reproducible results, counterexamples and documented gaps.
- Qualified: an independent gate established that the evidence meets the policy for a stated operating envelope.
- Authorized: the accountable authority approved the exact qualified artifact and its limits.
- Activated: the deployment controller verified that the installed artifact, configuration and permissions match the authorization.
- Monitored or revoked: observed effects remain within the approved envelope, or operation is constrained, suspended or rolled back.
These are conceptual control states, not names mandated by a standard. Each transition should have its own actor, predicate, evidence reference and recorded outcome. In particular, model completion is not workflow completion, and passing an evaluation does not automatically authorize deployment.
Define the promotion object before evaluating it
Calling a change “agent version 2” conceals its real risk. A promotion candidate should identify the complete behavior-affecting configuration: model and version, prompts, retrieval policy, knowledge source or index revision, skills, tool interfaces, orchestration logic, evaluation configuration, permitted data, environment and relevant dependencies.
Not every field needs to be embedded in one artifact. The essential property is a reproducible identity: a manifest and digests that bind the components materially relevant to the qualification claim. A seemingly minor change to an MCP tool, data source, routing policy or system prompt can invalidate an earlier measurement even if model weights remain unchanged.
A worked example: a GRC assessment assistant
Consider a GRC assistant that drafts control assessments from policy text, tickets and operational evidence. A proposed retrieval update improves answer completeness in a development evaluation. The candidate is not being authorized to “perform audits”. It is being considered for one bounded task: preparing draft assessments for a specified control set while preserving evidence traceability and explicit human assurance authority.
The promotion package could require the following:
- Proposed benefit: fewer unsupported control-assessment statements against the current baseline, measured on fixed and newly held-out cases.
- Hard boundaries: no fabricated evidence, no unsupported “implemented” verdict, no modification to the source-of-truth policies, and no independent sign-off on risk acceptance or assurance conclusions.
- Evaluation: source grounding, correct citation to specific records, handling of conflicting and stale evidence, missing-data abstention, cross-document reasoning, and regression on previously failed cases.
- Operating envelope: named control families, authorized data sources, organization-specific permissions, evaluator versions, cost and latency budgets, and an accountable reviewer.
- Recovery: restore the last qualified retrieval configuration and preserve the failed candidate's evaluation evidence for analysis.
Suppose the new system achieves a better aggregate assessment score but starts treating missing evidence as evidence that a control is operating. The score improved; the qualification failed. Promotion must be blocked rather than approved with a vague warning.
What the independent gate must inspect
A promotion gate should be capable of rejecting an apparently successful experiment. At minimum, its decision packet should retain:
- Identity and lineage: candidate manifest or digest, parent version, changed surfaces and dependency versions.
- Evaluation contract: test-set identity, task population, acceptance criteria established before testing, evaluator implementation, repeatability and baseline comparison.
- Negative evidence: regressions, uncovered scenarios, uncertainty, known counterexamples and failed attempts, not merely the passing summary.
- Risk and applicability: expected use, data and access scope, material obligations, expected consequences and residual risk owner.
- Decision authority: approver, decision time, scope, conditions, expiry or review trigger, and reasoned exceptions where policy allows them.
- Deployment and verification: approved artifact identity, observed deployed identity, initial health evidence, rollback target and a way to revoke the authorization.
The evaluator need not be a separate company or a large committee, but its acceptance criteria and authority must not be controlled by the same optimizer whose changes it judges. An agent can assemble the packet; it should not be able to rewrite the grading rule, suppress counterexamples or authorize its own promotion.
Three decisions that a green test cannot make
First, whether the organization should invest. That is a portfolio decision. A technically qualified capability can still be economically unjustified.
Second, whether unresolved risk is acceptable. That requires an accountable decision in the applicable governance system, informed by evidence and relevant obligations. A policy engine may enforce an approved risk rule; it cannot invent risk appetite.
Third, whether the deployed effect matches the approval. A successful pipeline or API response establishes neither the actual target configuration nor the resulting business outcome. The deployment record must be reconciled with the target system and monitored in use.
Triggers for requalification and rollback
Qualification is conditional, not permanent. Reassess when model or tool versions change, data populations drift, retrieval sources move, permissions expand, evaluation assumptions fail, a new harmful behavior appears, an applicable obligation changes or an approval expires. Changes that alter the capability's effective execution boundary deserve particular attention.
When the gate fails, the normal outcomes are repair, narrow the scope, defer, reject or escalate. In production, detection of a material regression may require suspending further promotions, revoking a particular capability or restoring the previous qualified version. Rollback must itself be tested; a rollback plan without a working recovery route is only an intention.
Standards support the governance need, not this exact state machine
NIST AI RMF 1.0 calls for documented evaluation methods, monitoring in deployment-relevant conditions, differentiated responsibilities and oversight. It supports the rationale for evidence and independent assessment, but does not prescribe the six-state promotion contract above. NIST has also signaled work on revising the AI RMF; the discussion here refers specifically to version 1.0.
ISO/IEC 42001:2023 specifies requirements for an organizational AI management system, including continual improvement. It does not, by itself, demonstrate that a particular agent build is safe, approved or correctly deployed. The promotion model here is an engineering interpretation designed to make organizational governance operational and testable.
Make the decision auditable
A promotion is complete only when an independent observer can answer five questions: What changed? Against which baseline was it qualified? Who authorized this exact version and scope? What was actually activated? What evidence would cause its authorization to be withdrawn?
That is the central L2 control contract. Continual improvement may produce many candidates. A governed system changes production capability only through evidence-bound, authorized and observable transitions.
Previous in the L2 series: The Governed Continual Harness
Explore the full Harness architecture series
Privacy Notice: With your consent, this website uses Google Analytics to understand aggregate visits and content usage. Analytics is optional and is not used for advertising or cross-site profiling. Some content and images may be generated or modified with the assistance of AI; every AI-assisted item is reviewed and manually edited by a human before publication. Full Privacy Policy ·