skip to content
Replays

Judging Whether the Work Was Actually Finished

Review learns to ask if a sprint covered its brief, and a cut-off agent stream stops meaning failure.

Gantry milestones

0 milestones 3 sprints

Review learns to ask if a sprint covered its brief, and a cut-off agent stream stops meaning failure.

1h 1m total 18m 11s per sprint

24.9M tokens in 214.3k tokens out

$24.90 nominal cost

This job changed how Gantry reads two uncertain signals after an agent has worked. A green gate no longer lets review assume the brief was fully covered, and a cut-off agent stream no longer means the agent failed before producing useful code.

The run added a scope verdict to review, a bounded way to discard incomplete work and try the same sprint again, and an inconclusive agent outcome for truncated streams. The current tree has since reorganized the build code, but those behaviours are still present under the split build modules, the harness classifier, and the smoke tests that exercise the recovery paths.

How this walkthrough is structured

Feature

What did the run build and ship?

Gantry now asks review whether the brief was covered, can reset an incomplete attempt without erasing its audit trail, and treats a truncated execute as a question for the worktree, gate, and review to answer.

Build

How did Gantry structure the work?

The run stayed green because the cut put the judgement first, the recovery mechanism second, and the truncation rule last, so each sprint inherited only the contract it needed.

Verdict Completeness Detection

0 agents 9m 28s wall time 31.9k tokens out $3.79 nominal cost

Review then asked for a `SCOPE:` trailer alongside a historical plan-impact trailer, and the parser read scope independently from historical plan-impact notes. In today's tree the prompt has moved to `config/agent-prompts/review.md`, and the parser lives in `src/engine/build/gate.rs`; both still show the scope judgement as review-only, not a gate concern. Later work expanded the incomplete form to name a retry or replan route, but the basic verdict introduced here is still the path review uses.

This sprint carried the smallest useful foundation: make review report whether the diff covered the brief, then surface incompleteness as its own stop. A fresh agent could hold that boundary because the retry behaviour was explicitly out of scope for this step. The run facts show no red gate or repair here, so the notable fact is the clean contract it left for the next sprint rather than any stress at the boundary.

Bounded Rollback and Re-execute

0 agents 21m 16s wall time 87.8k tokens out $8.80 nominal cost

The current worktree surface still exposes reset operations that preserve the run's orchestration directory while reverting project files, and its tests cover staged edits, untracked files, logs, ledgers, and pre-existing stash state. The build drivers route scope-incomplete recovery through that reset surface, and smoke coverage still checks that the first incomplete attempt is discarded while the retry receives the reviewer's note. The original single-retry wording has since become a configured recovery budget, but the same bounded recovery shape remains.

The second sprint could depend on the new verdict and focus on recovery: capture the pre-execute head, reset the project tree, and run the same sprint again within a fixed recovery bound. That is a broad change, but it was still agent-sized because the parser and stop reason already existed. It also landed without a recorded retry or review-driven re-plan, which means the hard safety property was proven inside the sprint's own tests rather than discovered by the outer run.

Truncation as Indeterminate

0 agents 18m 11s wall time 73.7k tokens out $9.61 nominal cost

`AgentOutcome::Indeterminate` still exists as a distinct outcome, and the Claude stream classifier returns it when events arrive without a terminal result unless a throttle signal makes the case transient. Execute stages are the special case: the build code lets an indeterminate execute proceed to the worktree diff, the gate, and scope review, while planning and gate-authoring stages remain conservative. Current harness tests pin the truncated-stream classifications, including the case where throttle text appears only in agent-authored content.

The final sprint was intentionally last because it only becomes safe after review can reject partial work and recovery can rewind it. Its task was to split an inconclusive machine signal from a true agent error, then let execute-stage truncation continue to the normal proof path. The sprint stayed green, so the decomposition did what it needed to do: the risky routing change leaned on already-tested safeguards instead of carrying them in the same context.