Where we draw the line on agent-led repairs

The second repair attempt was where the risk changed.

By then, the agent was no longer interpreting only the original requirement. It also had to work out why its first implementation had been rejected, using much of the same reasoning that produced the rejected code.

Sometimes that preserves momentum. If the problem is clear and the repair is tightly scoped, another pass can make sense. But when the requirement itself is unsettled, each attempt can carry the same misunderstanding into a wider change.

That’s where we draw the line.

Our policy puts a hard cap on agent-led repair. Once that budget is exhausted, the work returns to a person for re-specification.

A repair cap is a safety boundary, not a throughput target.

When a repair compounds the original assumption

Consider a hypothetical sequence. An agent interprets an unsettled business rule and implements it. Review rejects the resulting behaviour, but the agent treats the objection as a local code defect rather than evidence that the rule itself remains unsettled.

The repair changes the rejected path. It may also affect nearby behaviour that depended on the original interpretation. Review now has to assess the first misunderstanding and the broader correction.

We don’t have a representative repair trace in the available material, so this sequence is illustrative. Still, it shows the failure mode the cap is intended to contain: repeated automated work under assumptions that nobody has resolved.

A correction can extend beyond its intended scope, change behaviour elsewhere or introduce fresh assumptions. Without a limit, how many times should the agent be allowed to reinterpret the same unclear requirement?

Our answer is to limit the attempts, then change ownership.

The boundary in practice

Each risk band has a repair budget, although the values assigned to those bands haven’t been published.

Only blocking or major findings can trigger an automated repair, and only while budget remains:

• Blocking: eligible for repair when budget remains.
• Major: eligible for repair when budget remains.
• Minor: doesn’t automatically reopen implementation.
• Correctness: reported as a finding category, although its relationship to blocking and major classifications isn’t defined.

The available material doesn’t include the detailed classification criteria. It also doesn’t say how repair budgets differ between risk bands.

That matters. The cost of a wrong repair depends on the change. A low-risk change and a critical financial change shouldn’t receive identical permission to keep iterating, but the evidence doesn’t show the budget assigned to either.

When the budget runs out, the agent doesn’t get another attempt under the same assumptions. The work goes back to a person, who can narrow the requirement, settle a disputed rule, split the change or reject the implementation approach.

The workflow looks like this:

So the cap does two jobs. It bounds the eligible attempts. Then, when another automated pass would carry too much unresolved uncertainty, it hands the decision back to a person.

Trying to avoid the second pass

Our working hypothesis is that much of the pressure for a second pass starts before implementation, with ambiguous intent. Possible sources include incomplete acceptance conditions, unresolved business rules, unclear ownership of side effects and plans that leave too much design work until coding begins.

It’s still a hypothesis.

We don’t yet have a measured causal result linking those conditions to repair. What we do have is a policy requiring approved plans and settled clarification questions before implementation.

A useful plan sets out:

• behavioural boundaries;
• affected components;
• validation expectations;
• explicit exclusions.

The aim is to turn clarification answers into implementation constraints before they appear as review findings. When planning and clarification haven’t removed enough uncertainty, the repair cap remains the final boundary.

First-pass acceptance describes one part of this process. It records whether a change completes its configured review and evidence path without another implementation pass.

It doesn’t measure effort, elapsed time, code volume or the work needed to answer review comments. It doesn’t prove code quality in the abstract either.

Treating first-pass acceptance as a proxy for effort would create the wrong incentive. Teams could improve the measure by reaching review sooner rather than removing uncertainty before coding.

What we still can’t see

The available material reports two measurements:

They describe different events. First-pass acceptance records passage through review and evidence without another implementation pass. Repair-cycle distribution records whether repair was executed.

Without a shared cohort and linked state data, we can’t treat them as a single funnel.

The source also reports blocking, major and correctness findings, but doesn’t explain how those findings relate to a distribution with no executed repairs. They could cover different periods, populations or workflow stages. The evidence doesn’t tell us which.

Roughly a third of changes are reported as having an approved plan. Planning is selected partly by risk and complexity, so a direct comparison between planned and unplanned work could mistake selection effects for plan quality.

Several gaps still prevent an independent assessment:

• repair-cap values and budget assignments by risk band;
• definitions of the configured review and evidence path;
• cohort dates, sizes, inclusion criteria and denominators;
• linked states from rejection through final outcome;
• classification criteria for reported findings;
• a representative repair sequence;
• evidence that planning causes higher first-pass acceptance.

We need to make the process and its instrumentation more explicit. That means publishing the cap for each risk band, aligning first-pass and repair denominators, linking rejection to final outcome, and recording why rejected changes aren’t eligible for repair. Correctness findings should be tied to unresolved requirements only when the evidence supports that link.

Those gaps don’t remove the reason for the boundary. They do make it harder to judge how the policy behaves in practice.

For now, the cap stays hard. The next step is to publish its values and connect the workflow data around it. Then we can see where repair is helping, and where another attempt would simply repeat the same unresolved assumption.