skip to content
Replays

A Fast Default Gate and an Opt-In Display Tier

Split the test suite so the everyday gate stops launching the real display stack

Gantry milestones

3 milestones 10 sprints 24 agents

5 plan 11 execute 14 review 2 fix

127k peak context

93k median execute

Speeding up the suite with a dedicated display E2E tier.

7h 1m total 10m 40s per sprint

4m 10s plan 54m 19s execute 26m 4s review

66 edits 609 commands

codex harness

Gantry's root suite is part of the product surface: every autonomous build leans on it, so avoidable test time taxes the whole system. This job split slow display-stack coverage out of the ordinary check, tightened polling waits that were still paid by the default tier, and then reorganized the integration tests so the runner layout matched the new tiers.

The run is useful because the decomposition had to preserve coverage while moving it. Display tests could be marked and selected before waits were shortened; wait work could finish before test binaries were merged; and shared process-state hardening had to precede consolidation so plain cargo execution still remained the required gate.

How this walkthrough is structured

Feature

What did the run build and ship?

The current tree keeps the ordinary gate separate from display-stack coverage, runs consolidated integration modules through the root test entry point, and offers optional readable output without requiring that tool for verification.

Build

How did Gantry structure the work?

The job split a suite-speed problem into tiering, wait reduction, and consolidation, with a re-plan after the first milestone carrying an end-to-end discovery constraint into the later runner work.

Milestone 1

Display E2E Tier

3 sprints 9 agents 43m 18s wall time 126k peak context

The display-stack split is still visible, though later consolidation moved the files it first touched. The opt-in helper lives under test support, the display runner still checks for terminal and media tools before it runs, and display tests remain marked so the default gate does not launch the real stack. The original top-level test files are gone because the later milestone folded them into the tiered integration tree.

This milestone carried the first dependency: make the expensive display-stack tests explicit before any later cleanup relied on a faster default gate. Its scope was wide enough to require auditing tests by behavior rather than filename, but narrow enough for the milestone to end with one new tier, one runner, and matching verification guidance. The ledger records a green boundary, and its note is the important build fact: review found that the runner discovered ignored tests from top-level integration files, so later consolidation had to update discovery and gate coverage together.

Tiered Display Tests

2 agents 12m 38s wall time 126k peak context

The specific files this sprint marked have since been reorganized, but the behavior remains. Display-stack tests now sit under the integration end-to-end module and carry ignored annotations with an opt-in helper. Cheap checks that were near those tests are represented in the default integration modules instead of being swept into the display tier.

This sprint was asked to do the classification work that made the rest of the job possible. A fresh agent could hold it because the boundary was per-test behavior: display recording, terminal automation, media validation, and real-time replay left the default tier, while cheap neighboring checks stayed put. The run facts show no repair on this slice, so the risk was carried by the audit rather than by later recovery.

E2E Runner Command

2 agents 11m 18s wall time 76k peak context

The runner still exists as a committed shell command and now discovers display tests from the consolidated end-to-end directory. It prints that the display tier is being run, verifies required display and media tools, and exits cleanly when a host cannot provide them. Focused integration coverage exercises the script's selection and skip behavior in the current layout.

The tier needed a command boundary so selecting expensive tests did not depend on memory of cargo flags. This sprint was scoped to the runner and its checks: set the opt-in environment, select only the intended ignored tests, and distinguish missing local display tools from a real failure. It stayed green, which left the later historical plan-impact note as a structural warning rather than a repair.

Test Tier Docs

2 agents 10m 54s wall time 56k peak context

The exact documentation touched by this sprint has been revised by later project changes, but the distinction it introduced still appears in the agent-facing verification guide. The docs name the ordinary root gate, describe the heavy display tier separately, and state that optional readable output does not become a required dependency. Some older gate prompt files from the job are no longer present.

This was the milestone's thin closing piece: update the places that tell contributors what verification means. Its boundary was documentation alignment with the tier and runner already built, so the agent did not need to reason about test selection internals. The sprint finished green, and the milestone review then supplied the discovery constraint that affected later work.

Milestone 2

Tighter Test Waits

3 sprints 10 agents 28m 54s wall time 104k peak context

The wait work is less visible as a single structure than the tier work, because it lives inside scripts, fixtures, and engine tests. The current tree still contains faster fake replay defaults, condition polling for output readiness, and shorter receive or abort checks in engine tests. Some old files were later moved into the consolidated integration tree, but the tightened behavior survived the move.

After display coverage stopped running by default, this milestone attacked the remaining fixed waits without changing what the tests asserted. The cut moved from replay fixtures to integration polling and then to unit-level waits, which let each agent work on one kind of timing evidence at a time. The integration-polling sprint went red after review and was repaired, locating the fragile part in script and subprocess readiness rather than in the broader unit sweep.

Replay Fixture Speedup

2 agents 3m 58s wall time 77k peak context

The fake replay fixture still defaults to a short sleep for ordinary overlay tests, with a separate long-running path for failure cases that need the process to remain alive. The surrounding tests continue to assert start, finish, cleanup, and failure propagation rather than just shortening a timer. Later file moves place that coverage under the default integration module.

This sprint isolated the fake replay fixture before broader polling changes could mask it. The brief drew a clean line between tests that merely need a replay process to move through states and tests where elapsed playback time is the assertion. The green result meant later wait cleanup started from a faster fixture rather than compensating around a slow fake process.

Integration Polling Tuning

3 agents 10m 58s wall time 81k peak context 1 fix

The scripts still wait for observable output or process state instead of paying a fixed interval after a replay finishes. Integration coverage checks both successful readiness and capture failure paths, including cases where the recorder never produces bytes. Later consolidation changed paths, but the script behavior and the default-tier tests around it remain.

This was the stress point in the timing milestone. The sprint had to change shell scripts and integration tests from fixed polling to condition-based readiness while preserving bounded failure behavior, and the gate sent it back once after review. That repair is evidence that the hard part was not finding sleeps, but proving the shortened waits still failed correctly when readiness never arrived.

Unit Polling Tuning

2 agents 10m 25s wall time 104k peak context

Engine tests now use short receive windows and abort-aware waits where they are waiting for another thread or process to become observable. Timing-sensitive behavior, such as replay pacing and abort responsiveness, is still tested as elapsed behavior instead of being flattened away. These changes are folded into the engine test code rather than exposed as a named feature.

The final timing sprint applied the same discipline to in-module and unit-test waits. Its scope was broad, so the brief constrained the work to pure polling around channels, files, processes, and state flags, while leaving tests that prove timing semantics alone. The run stayed green here, which fits a conservative sweep over local wait helpers rather than a semantic rewrite.

Milestone 3

Runner UX Consolidation

4 sprints 17 agents 55m 53s wall time 127k peak context

This is the part of the job most plainly standing in the tree. The integration suite has a single root test entry point with default and end-to-end module rosters, shared helpers live under test support, the display runner discovers tests from the consolidated directory, and nextest profiles are committed as optional configuration. Later root-runner work expanded the command, but the tier-organized structure remains.

This milestone took the structural work only after the tier and timing changes were already proven. Its order matters: harden shared process support first, merge default tests second, move the end-to-end tier and runner discovery third, then add optional readable output last. The run facts show manual recovery around default consolidation and failed gates around the final runner sprint; the ledger records that the final red was attributed to a full-suite flake, with the committed code confirmed by hand.

Shared Test Support

2 agents 8m 9s wall time 103k peak context

Shared process ownership is now represented by a test support guard keyed to the workspace, with environment restore and temporary directory helpers nearby. That support is used by tests that touch environment variables, current directory, and other process-wide state. The coverage includes the shared-process behavior itself, so the consolidation did not rely on nextest process isolation.

Before separate integration files could become one threaded test binary, tests that mutate process state needed an explicit guard. This sprint was scoped to helpers and fixtures rather than movement, which kept concurrency risk out of the later relocation work. It stayed green and gave the consolidation agents a support layer to reuse instead of rediscovering isolation issues in moved test bodies.

Default Tier Consolidation

4 agents 27m 1s wall time 127k peak context

The default integration tier now enters through the root integration test file and a default module roster. That roster includes a source-count check so a migrated test module cannot disappear silently. Many original top-level integration files no longer exist because their contents became modules under the default tier.

This sprint carried the main file-layout reduction for the default tier. Its charge was structural but risky: preserve names, assertions, and cargo name filtering while moving default tests into one tiered integration binary. The ledger note says mixed display files had to be split by hand during recovery, so the difficult boundary was separating default tests from display tests without leaving any top-level leftovers.

E2E Tier Consolidation

2 agents 9m 1s wall time 93k peak context

Display tests now live under the integration end-to-end module tree, with a roster check for ignored display annotations. The display runner scans that directory and runs exact test names from the consolidated integration binary. Its gate coverage verifies that discovery remains scoped to display-stack tests rather than every ignored test.

This sprint addressed the review finding from the first milestone directly. Once display tests moved into the consolidated structure, the runner and its gate coverage had to change with them or the command would keep selecting the old shape. The sprint stayed green, which indicates the earlier historical plan-impact note was carried into the right boundary before the file move landed.

Optional Readable Runner

5 agents 6m 40s wall time 109k peak context 1 fix

The optional runner layer is still present as committed nextest profiles and root runner behavior that uses nextest when available and falls back to cargo when it is not. The display profile is scoped to the consolidated end-to-end module path, while the ordinary root command still remains usable without nextest. Later work broadened the root runner into default and heavy tiers, but it kept the optional-tooling contract from this sprint.

The final sprint was intentionally layered over the completed layout: readable output could only reflect the finished default and display tiers. It came under run stress, with failed gates and investigation recorded before the ledger marked the code green by manual confirmation. That boundary matters because the brief required optional tooling only; the ordinary cargo path could not become dependent on nextest.