What our repair cap can prove, and what it can’t

We noticed a gap between what our bounded repair policy was meant to show and what the records could actually prove.

The policy is intended to expose repeated review demand. Rather than letting an agent keep proposing patches, it limits repair attempts. Once that limit is reached, the change goes back to a person.

That boundary matters because agent-led repair can avoid a visible hand-off while creating a less visible problem. We’re asking the agent that misunderstood a requirement to reinterpret both its own work and the reviewer’s objection. The correction can spread beyond the original change, touch unaffected code or introduce more assumptions. Reviewers then have to understand and check each new implementation pass.

But can we show that the policy reduces review demand, escaped defects or failed acceptance?

Not yet.

We don’t have complete metric definitions, aligned event records or a sourced change trace that would let us quantify those effects.

The objective isn’t to make agents better at looping. It’s to stop ambiguity reaching review.

What first-pass acceptance can tell us

First-pass acceptance asks whether an implementation completes its configured review and evidence path without another implementation pass.

It measures an outcome, rather than activity. Generated code, responses to comments and repeated attempts don’t count as progress towards it. Nor does the measure tell us directly about effort, elapsed time or code quality in the abstract. Review can assess only the requirements, checks and evidence available to it.

That distinction is easy to lose.

A carefully specified change may take substantial reasoning and still pass on its first attempt. A quick change may fail because a requirement remains unsettled. If we used first-pass acceptance as a proxy for effort, we could encourage teams to rush work into review instead of removing uncertainty before coding.

To define the metric properly, we need:

• a numerator covering changes accepted without another implementation pass;
• a denominator covering every eligible change entering the configured path;
• a defined cohort and measurement period;
• explicit treatment of abandoned changes and changes routed to a person;
• documented inclusion and exclusion rules.

We don’t yet have those details. We’ve therefore omitted the previously reported rate until its basis is documented.

Repair is an exception

Our policy treats repair as a bounded exception.

Only blocking or major findings trigger it. Minor observations don’t automatically reopen implementation. Each risk band has a repair budget with a hard cap, and once that budget is exhausted, the change returns to a person for re-specification.

An agent can’t keep iterating simply because it can produce another patch.

The person might then narrow the requirement, resolve a disputed rule, split the change or decide that the implementation approach is wrong. The cap is a safety boundary, not a throughput target. It stops repeated implementation and review cycles from disappearing into hidden delivery churn.

The intended process looks like this:

Implementation
    ↓
Configured review and evidence path
    ├── Accepted → Complete
    └── Rejected
          ├── Not eligible for repair → Human disposition
          └── Eligible for repair
                ├── Budget available → Bounded repair → Review
                └── Budget exhausted → Human re-specification

That’s the policy model. It isn’t yet a verified event schema, and we don’t have a sourced case tracing a change from rejection to its final disposition.

Where the records stop

The measurements available today describe different events.

First-pass acceptance records whether a change was accepted through its configured review and evidence path. Repair-cycle distribution records whether repair was carried out. At present, that distribution contains only the no-repair category.

It’s tempting to combine the two into a single funnel, but the denominators don’t align. The data don’t fully account for rejected changes that were ineligible for repair, abandoned or routed to a person. As a result, we can’t reconcile the zero-repair distribution with first-pass failures.

Several other pieces are missing:

• a verified event schema linking rejection, repair eligibility, repair execution, abandonment, human routing and final disposition;
• counts for intermediate and terminal dispositions;
• published definitions for finding severity, risk bands, repair budgets and configured review paths;
• documented review controls and criteria for reviewer independence;
• a sourced change trace from rejection to final disposition;
• aligned cohort counts for testing review demand, acceptance outcomes and escaped defects.

So bounded repair remains a policy position rather than an observed operational effect.

That’s a meaningful limit on what we can claim.

Testing the ambiguity hypothesis

Our working hypothesis is that pressure for a second pass often begins with ambiguous intent. The ambiguity may come from incomplete acceptance conditions, unresolved business rules, unclear ownership of side effects or plans that leave too much design work to implementation.

The current evidence doesn’t quantify that relationship. It also doesn’t show that approved plans improve first-pass acceptance.

There’s another complication. Planning is selected partly according to risk and complexity, so a direct comparison between planned and unplanned work could confuse selection effects with plan quality.

The policy puts approved plans and settled clarification questions before implementation. A useful plan sets out behavioural boundaries, affected components, validation expectations and explicit exclusions. Clarification answers become implementation constraints, rather than discoveries made during review.

Does that reduce repair pressure? We still need to test it.

We’re now defining the first-pass numerator, denominator, cohort and measurement period, then aligning those definitions with repair-event records. Accepted, rejected, repair-eligible, repaired, abandoned and human-routed outcomes need to be recorded separately. We also need to test that they cover every disposition without overlap.

We’re recording why rejected changes are or aren’t eligible for repair, documenting review controls, risk bands and repair caps, and applying the plan gate consistently. Where the evidence supports it, we’ll trace correctness findings back to unresolved requirements. Clarification questions must be settled before implementation starts.

The repair cap remains hard. The work now is to build the evidence around it, so we can tell whether repeated review demand is genuinely being exposed or merely recorded somewhere else.