The architecture decision
L2 separates the ability to propose change from the authority to qualify and promote it. The governing loop is Run → Measure → Evidence → Risk → Compliance → Approval → Promotion. An observed improvement is a candidate for review, not an implicit authorization to modify production behavior.
Two interfaces matter here. L1 Strategy & Portfolio owns the investment decision and business appetite; Enterprise Risk & Decision Governance provides the accountable risk and approval boundary. Neither is interchangeable with the L2 qualification decision.
Why the loop needs explicit states
Run captures the bounded task, context, configuration and authority. Measure compares behavior with a predeclared baseline and failure criteria. Evidence retains reproducible traces, evaluation conditions and counterexamples. Risk assesses changed exposure and residual uncertainty. Compliance checks applicable obligations without assuming that a standard alone constitutes approval. Approval binds a named authority to a specific change envelope. Promotion deploys only that approved version, with rollback and monitoring conditions.
Any missing evidence, failed test, changed scope or expired approval returns the candidate to review. Promotion must not be an automatic consequence of a favorable score.
Separate experimentation from qualification
A self-improvement experiment may generate a better prompt, policy proposal, workflow or code change. It must remain in a constrained experiment state until the proposed change has been evaluated against intended outcomes, regressions, security and reliability conditions. Evaluation evidence supports a qualification decision; it cannot itself exercise the authority to approve production use.
For each proposed change, preserve the originating observation, exact candidate version, evaluation dataset and conditions, known failures, accountable owner, approval scope, expiration trigger and rollback reference. A reviewer should be able to answer not just whether the candidate worked, but where, under which assumptions, and who accepted the remaining risk.
What the external evidence supports
NIST AI Risk Management Framework 1.0 provides a voluntary framework for governing, mapping, measuring and managing AI risk throughout the lifecycle. It supports ongoing evaluation and documented risk decisions; it does not prescribe this seven-step Harness loop.
ISO/IEC 42001 specifies requirements for an AI management system, including continual improvement and organizational accountability. A management-system requirement is not independent assurance that a particular agent change is safe or effective.
Research on autonomous improvement can motivate controlled experimentation, but results from a preprint, demonstration or implementation repository do not establish general reliability or authorization to promote changes. Such sources must be assessed for experimental conditions, limitations and reproducibility before they affect qualification criteria.
The practical decision boundary
Before promoting a candidate, require: (1) a defined change and intended benefit; (2) evaluation against a fixed baseline; (3) evidence of failures and regressions; (4) a risk and obligation review; (5) approval by an accountable authority; and (6) version-bound deployment, monitoring and rollback. A successful experiment may still be rejected when its operational risk, cost or uncertainty is unacceptable.
The architectural principle is simple: continual learning is permitted; uncontrolled self-authorization is not.
Previous: The AI/Agent Factory — Portfolio Gates
Privacy Notice: With your consent, this website uses Google Analytics to understand aggregate visits and content usage. Analytics is optional and is not used for advertising or cross-site profiling. Some content and images may be generated or modified with the assistance of AI; every AI-assisted item is reviewed and manually edited by a human before publication. Full Privacy Policy ·