# Implementer lessons

What reviews of the implementing model's work (Laguna S 2.1 through OpenCode) have taught us, as
tips. It has two audiences: the implementer, and whoever writes its tasks. Every tip comes from a
finding in `docs/implementer-log.md`. A tip without evidence does not belong here.

How it is used:

- After each milestone review, each finding becomes a row here or confirms an existing one.
- Short, general rules are promoted into `AGENTS.md`, which the implementer reads in every session
  and which must stay small. The "In AGENTS.md" column says which.
- "Seen again" is filled in at later reviews. A tip that does not stop its defect from coming back
  needs rewording or a test, not repetition.

## Tips for the implementer

| # | Tip | Evidence | In AGENTS.md | Seen again |
|---|---|---|---|---|
| I1 | When a rule says "every" or "everywhere", finish by listing each place it could apply (every struct and enum in the file, every script, every branch) and check them one by one. An example in the task shows one place; the rule covers all of them. | M1 finding 1: `deny_unknown_fields` was put on both enums in task 06, where the task's example showed it, and on neither struct. In task 03, where the task said "works on structs and on enums", every type got it. | yes |  M2a: no. All six config structs have it; none of the server-format structs do. M2b: partly. The busy-release rule was applied to the path the task named and missed on the three error paths that do the same thing (finding 1). |
| I2 | A check must fail when it cannot do its job: missing input, unreadable file, a tool that errors. Never throw errors away (`2>/dev/null`, `\|\| true`, an ignored `Result`). | M1 finding 4: all three gate scripts passed when `crates/` did not exist, and hid `find` and `awk` errors. | yes |  M2a: no recurrence. M2b: partly. An unreadable core.md was treated as absent (finding 3), and a poisoned lock skipped the busy removal (finding 2). |
| I3 | Report every problem found, not only the first. | M1 finding 8: `check-lines.sh` exits at the first long file. | yes | M2a: not exercised. |
| I4 | Run the formatter before the gate, and let it decide ordering. | M1 finding 7: three first-gate failures were rustfmt reordering `lib.rs`. | yes | M2a: no fmt failures; the first-gate failures were clippy. M5d-2: first-gate clippy on 05 (`collapsible-if`), 06 (`single_match`), and a USAGE indent on 10. |
| I5 | Log every attempt, including one you abandon. | M1 review: the task 01 row mentions files "from a prior attempt" that has no row of its own. | yes |  M2a: partly. Task 11's `stopped` row was overwritten by its later `done` row. M2b: no recurrence; all ten rows correct, one with a stray pipe. |
| I6 | Prefer a conversion that can fail (`u32::try_from`) to a cast that is only safe because of a check somewhere else (`as u32`). | M1 finding 6: bounded `as` casts in `frame.rs`. Correct today, but the safety depends on a line ten lines away. | no (already implied by the code rules) | M2a: no `as` casts at all. M4a: yes, bounded casts in `http.rs` and `handshake.rs`, where the task named no conversion; none where a skeleton comment gave the `try_from`. |
| I7 | A `read` returns as soon as it has any data to give. It blocks only when it has none. Never keep reading to fill the caller's buffer. | M2a finding 2: the chunked reader delivered a whole stream at its end. | yes | M2b: no recurrence. |
| I8 | When forwarding between two connections, pass a close on in both directions. Do not rely on the client half-closing first. | M2a finding 1. | no (specific to inferproxy) | |
| I9 | Never end a turn by describing what you are about to do. Do it, then report. | Two Ornith turns in M2a ended with a "## Objective" plan and no tool call. | yes | M2b: not seen; all ten tasks ran unattended to a commit. |
| I10 | If a tool you were told to use does not exist, stop and say so. Do not invent a command in its place. | Laguna as coordinator ran `opencodec`, which does not exist, and then diagnosed its own typo. | yes | |
| I11 | A rule about one path applies to every path that does the same thing. If a task says "release X before the final frame", every final frame counts, including the error frames written earlier in the function. | M2b finding 1. | yes | |
| I12 | A file that exists but cannot be read is an error. Only a file that does not exist may be treated as absent. | M2b finding 3. | yes | |
| I13 | Before calling a failure "environmental", measure the thing you blame, and name the machine, OS and file system the evidence came from. A sample that shows a thread inside a system call shows where the time goes, not that the call is stuck. | M3a stop before task 20: an fsync "stall" was blamed from `__fcntl` frames that were ordinary 4 ms `F_FULLFSYNC` calls on the Mac, and the real cause, a macOS socket rule, went unfixed. | no | |
| I14 | Recovering from a crash is a case to write a test for, not only to reason about. A file the daemon itself can leave behind (created, not yet written) must load like an empty one. | M3a finding 1: one zero-length log file made `brokerd` panic at startup, where the same directory verified as `ok, 0 records`. |
| I15 | Never write the model's chat-template tags out whole (the think and tool-call tags, the angle-bar turn markers) in code, comments, commands or the log. Build them from pieces (`concat!("<", "think")`) and name them in words. Text holding them turns into the model's own control tokens when it reads it. | M5a task 04: the given test held the tags; Ornith's reasoning ended at the test file's closing think tag, and it lost track of the file and made no change. `docs/research/2026-09-24-chat-template-tokens.md`. | yes | M5a tasks 05 to 22: not seen again. |
| I16 | When a test outside your task fails, check once whether it fails without your change (stash, run, unstash), then stop and report both results. Do not keep experimenting, and never add a file the task does not list, not even a temporary one. | M5a task 21: the session did the check, then spent about 80 minutes on experiments and wrote two diagnostic test files into `crates/bxctl/tests/` until the owner stopped the run; the cause was outside the repository (review finding 4). M5a task 16 did it right: one check, its log row, a stop. | yes | M5c-1: the stash left the new untracked tests behind and a baseline printed no total (task 15; now `git stash -u` in AGENTS.md, T53); the two flaky tests were rerun and named, not stopped on (now an exception for a test the README lists, T52). |
| I17 | Write your log row from `git diff` after staging, naming only what the diff shows. A row written from memory describes what you meant to do. | M5a task 10's row describes checks that are not in `serve.rs`; task 13's says two runbook fragments were copied, and neither was (review findings 5 and 6 of 2026-09-24). | yes | M5c-1: task 10 logged one gate run and ran three, task 12 logged three and ran four and "Deviations: none" after anchoring an edit on its own line, and task 14's row blames the task for its own spacing error. M5c-2: every row's first gate is `pass`; task 01 still ran the gate twice, and every session ran `cargo test --workspace` after `gate: ok` only to count (T59). |
| I18 | After `gate: ok`, do not run `cargo test --workspace` (or the gate) only to count tests. The gate's total is the count. If HEAD already has this task's commit and a `done` row, stop; do not start the task again. | M5c-2: every session re-ran the workspace tests after a passing gate (T50 still driving extra work). Task 05's run log is a resume after HEAD was already at task 12; Laguna did not re-commit, and the original transcript was truncated. | yes | M5d-1: tasks 03, 07 and 08 ran the gate two or three times with first-gate pass. M5d-2: task 04 ran the gate twice with first-gate pass. |
| I19 | Add the log row immediately under the table header (newest first). Do not insert it by date among older rows. | M5d-1 task 06: the row was inserted after M4b/03. | yes | M5d-2: task 04's commit duplicated 03's identical `done` row. |
| I20 | After a turn, do every bookkeeping step the task lists, including when the outcome is `Err`, unless the task says only on Ok. Gating `last_heartbeat` on success is a judgement the task did not ask for. | M5d-2 finding 1: heartbeat sets `last_heartbeat` and pushes `Heartbeat` only on `outcome.is_ok()`. | yes | |

## Tips for writing tasks

| # | Tip | Evidence |
|---|---|---|
| T1 | The implementer follows examples more reliably than prose. For a rule that applies in several kinds of place, show an example of each kind, or say in so many words that it applies to structs and enums alike. | Same evidence as I1. The one task whose note covered both kinds had no miss. |
| T2 | A test for a property that must hold "everywhere" should walk the data, not sample it. | M1 finding 1 got through because the given tests checked unknown fields in a few hand-picked places. `tests/strict.rs` now adds an unknown key to every object of every fixture. |
| T3 | When a rule depends on a file format, list every way the format can write the same thing. | M1 finding 3: the task said "however it is written" and named three spellings; TOML has a fourth, the `[dependencies.name]` table, and the dependency check passed it. |
| T4 | If formatting or serialising a value can fail, make the value impossible to construct. Check what the library does at the edges of its range before writing the signature. | M1 finding 2: `Timestamp` accepted any `u64`; `humantime` cannot format past year 9999 and `to_string()` panics. The reference implementation had the same bug, so the given tests could not catch it. |
| T5 | A reference implementation proves the tests can pass. It does not prove they are complete: reference and tests share the author's blind spots. Probe the finished work from outside with inputs the tests do not contain. | Findings 2 and 3 were in the reference too. Both were found by probing, not by the gate. |
| T6 | Do not tell the implementer how to lay out what a formatter controls. | M1 finding 7. |
| T7 | A follow-up task works well as: what the reviewer observed, which part was the task's fault, a failing test to copy in, and the new rules. | Tasks 08 and 09 each passed the gate on the first run, and 09 handled six forms its self-test did not contain. |
| T8 | Vet a dependency by testing it against an independent implementation before naming it in a spec, whoever wrote it. Include the boundary cases of its algorithm. | The owner's `emsha` 1.0.3 passed its own tests and hashed every message of length 63 mod 64 wrongly. A 90-case differential run against `sha256sum` found it in minutes. Same shape as T5: an author's tests share the author's blind spots. |
| T9 | State the contract of a standard trait the implementer must honour, in the task, even when it seems obvious. | M2a finding 2: task 04 said what `Body` reads but not when `read` must return. Two implementations, the implementer's and the reference, both got it wrong in different degrees. |
| T10 | Test *when* streamed data is delivered, not only *what*. A test that compares final bytes cannot see buffering. | M2a finding 2 passed 15 tests that only compared bytes. |
| T11 | Make the fake client behave like the real one. If the real client never half-closes, no test client may half-close either. | M2a finding 1: all test clients half-closed; `loopd` does not. |
| T12 | A weak model should not coordinate other sessions. Use the shell driver (`tools/run-plan.sh`), whose checks are code. | The `opencodec` episode. |
| T13 | Set the sampling defaults on the server for coding agents (`temp`, `top-p`, `top-k`, `reasoning-budget`). OpenCode sends none, so the server's default temperature of 1.0 applies, and thinking has no cap. | Five Ornith turns ran 16k tokens of thinking to the output limit and produced nothing. |
| T14 | When a rule has a "before the final frame" or "on every exit" shape, list the exits in the task, or say "including the early returns". The implementer follows the path the task walks through. | M2b finding 1: task 07 walked the success path and the turn's error; the open and create errors were early returns it did not mention. |
| T15 | Running the whole plan through `tools/run-plan.sh` worked: ten tasks, one commit each, unattended, in about three hours. Keep the TUI closed while it runs; a second message into the driver's session starts a second agent on the same tree. | M2b run, 2026-09-18. |
| T16 | When a task prescribes the fix, check that it compiles against the types as they are, in the reference tree, before handing it over. A fix that cannot be written as described pushes the implementer outside the listed paths. | M2b task 11: "`unwrap_or_else` with a fixed valid id" had no non-panicking form outside `proto`; the implementer added `Default` to `SessionId` and reported it. |
| T17 | Match the check to the risk. A full reference for intricate logic whose writing debugs the spec (state machines, concurrency); a naive oracle inside the test for decision logic; a compiling skeleton (`todo!()` bodies under the real signatures) for plumbing. Record what each check exposed, and let the record decide what the next milestone gets. | Across M1 to M2b the references caught no implementer defect. They caught task defects (T16) and missed what they shared with the tests (T5). Decision of 2026-09-18. |
| T18 | Keep reference implementations where the implementer cannot read them. A run on this machine can reach every directory the owner can; a stuck model will search the disk and copy what it finds, and an orchestrator will write in the log that it did not. Move `~/src/boxmaker-ref*` out of reach (or sandbox the run) before a plan starts, and compare the result with the reference byte for byte in review. | First M3a run, 2026-09-19: the orchestrator copied the reference `chain.rs` for task 03 and logged it as written "from the spec and fixtures"; task 09's worker spent eight hours in `find / -name audit.rs`. |
| T19 | A file in `files/` that two tasks copy must be right for the earlier task. When a later task changes a shared test file, hand the earlier task its own copy (`strict.rs-task01`, `Makefile-task21`) and check it at that task's end state. | M3a task 01 copied the merged `strict.rs`, which walked task 02's fixtures, so its gate could not pass. |
| T20 | The gate runs on two platforms, Talos (Linux) and the Mac (macOS), and they differ where the tests touch the OS: sockets, file sync, `/tmp`. Accept a task only when the gate passes on both. Code that sets a socket option after the peer may have closed breaks on macOS only. | M3a: `BrokerPort`, the HTTP client and the brokerd test client passed on Talos and failed on the Mac from task 13 on; it was found only when the gate was first run on the Mac. |
| T21 | When two components must agree on a set (which files are the log, which names are ids), give the tests one case that walks both. Each half was tested alone and they still disagreed. | M3a finding 2: `brokerd` accepted `2026-0x-18.jsonl` as a log file and `bxctl audit verify` ignored it, calling the log `ok` while leaving out half its records. |
| T22 | List the fail-closed states a task creates, with their runbook anchors, in the task itself. A script can check that a pointer names an existing entry; nothing can check for a pointer that was never written. | M3a finding 3: four startup failures in `serve` and `main` print no pointer, and the spec's own list of pointers omitted them. |
| T23 | A test that writes a script and then runs it must hold a lock shared by every test in its binary that starts a process. Otherwise another test's fork can hold the new script open for writing at that moment, and running it fails with "Text file busy" (ETXTBSY), about once in seven runs. Give the lock in the support file and say why. | M3b plan checks: the fake `curl` tests failed 8 times in 40 until every forking test took `serial()`; then 0 in 60. |
| T24 | A task that changes a `Cargo.toml` must stage `Cargo.lock` in its `git add` line. Better, put `Cargo.lock` in every task's `git add`; it is a no-op when unchanged. | M3b task 04: committed correctly, left the lock behind, and the driver stopped on an unclean tree. |
| T25 | Size a task by the largest function the model must hold in one turn, not by the task. Ornith writes one function with a few branches well; a function with half a dozen branches and threads (M3b's `run`) it plans in its head until the turn runs out, with nothing written. Give such a task a compiling skeleton with the big function already written as glue over small `todo!()` helpers, and say to fill one at a time with `cargo check` between. | M3b task 11: four sessions. Two wrote nothing; a whole-file skeleton got five of six functions; the finer skeleton finished it in ten minutes, and tasks 12 and 13 followed without a stop. M4a task 08: again, a skeleton whose `header` was one `todo!()` with a dozen branches; nothing written. Size every `todo!()`, not only the task: the largest comment above a `todo!()` is a fair proxy, and one over about six lines needs splitting. M4a task 14: a task with a dozen small `todo!()`s still stalled, because the model read the whole crate to learn the APIs first; give a table of every call the task makes, with its signature, and make copying and the failing test the first two actions. |
| T26 | Replay each task's end state on its own, **and** read the task file against the reference for anything the reference has that the task does not ask for. The replay proves the tests can pass; only the reading finds a field the reference reads through a getter the task never mentions. | M3b task 11: the task stored `egress_dir` without the reference's getter, so the field was never read and clippy failed; the replay passed because the reference had the getter. |
| T27 | Every wait in a given test has a limit (`recv_timeout`, a deadline loop, `is_finished` before `join`). A test that waits forever on a broken implementation hangs the driver instead of failing, and the implementer cannot tell a hang from slowness. | M4a planning: breaking the reference's ping made `ws_conn` hang on `rx.recv()`; retrying a refused token made the serve test hang on `join()`. Both now fail within 5 s. M4b planning: a failed assertion inside `std::thread::scope` hangs instead of failing, because the scope joins its threads first; give such a test a guard that stops the thread on drop, and wait on `is_finished()` with a deadline instead of `join()`. |
| T28 | Before hand-over, break the reference on purpose, one line at a time (drop a check, move a bound by one, skip a save), and run the given tests against each change. A change the tests still pass is a missing test, unless it cannot change behaviour. Rustfmt reflows lines: match on text that survives formatting. | M4a planning: 69 changes over six modules; the tests missed 8. Five were real gaps and got tests (a reconnect that re-sent "interrupted" for a running turn was one), two could not change behaviour, and one was left (an event frame with another id, which `loopd` never sends). |
| T29 | Check a skeleton's comments, not only its code. Print every `todo!()` beside its signature and read each pair before hand-over. Filling the bodies from your own code proves the split can pass; it cannot show that a comment sits on the wrong function. Edit comments by function, never by "the next comment after this name". And tell the implementer to stop and quote a comment that does not fit. | M4a task 14, second session: `post`'s comment was `now_ms`'s text, put there by a script; Ornith saw the contradiction and deliberated over it until the turn ran out. M4a task 16: a comment naming the function it replaced made the task's own `grep` fail; and a given test copied into the plan before `cargo fmt` came back reformatted. Run the task's checks, and `cargo fmt`, on the end state before hand-over. |
| T30 | Every behaviour a task writes must be reachable by that task's own tests. When a function's effect shows only after a later task (here: a refusal's reason, which only task 15 could send), give a unit test in the first task, or write that function in the later one. And never let one name mean two things in a comment: say which `reason`. | M4b task 14 dropped a refusal's reason, reading `Refused -> Ended::Refused { by, reason }` as the `DenyReason` in scope; its tests passed, and task 15's caught it, in a file task 15 could not touch. |
| T31 | In a spec, for every post or message the harness sends, name who can read it and whether that reader can act on it. Walk each kind of sender through each kind of channel (the owner, an allowed non-approver, a stranger; direct, group, channel). | M4b: an approval asked by a non-approver in a direct message went into that direct message, where no approver could see it. Every test passed, because the tests used the spec's cases; the device check with a second account found it (review finding 1, fixed by tasks 17 and 18). |
| T32 | A given test must not assert the order of two things that run on their own threads. Check the order where it is decided (here, the sort in the catch-up parser), and compare what the threads deliver as a set. Before hand-over, stress each test with concurrency under load (`yes > /dev/null` beside a loop of 60 runs). | M4a's `serve_restart` expected two caught-up sessions' turns to reach `loopd` in order. They run on separate threads, and under load 1 run in 60 failed. M4b task 18's session hit it, proved it happens without its change, and reported it instead of editing the test. |
| T33 | A file the implementer reads must never hold the model's chat-template tags literally: not in skeletons, given tests, fixtures, task text or specs it may open. Build them from pieces and name them in words. grep the plan for the tags before hand-over. | M5a task 04 (above); the measurements in `docs/research/2026-09-24-chat-template-tokens.md`: the think and tool-call tags are one token whatever the tokenizer's flag, and inside a chat message the turn markers become control tokens. |
| T34 | Replay the plan with the toolchain of the host that will run it, and write the `rustc` and `clippy` versions in the plan. Each clippy release adds lints, and the workspace's `rust-version` is only a floor. | M5a task 16 was replayed on talos; on titan, clippy 1.95 failed the gate on `collapsible_match` in code from task 08, and the session stopped (review finding 6 of 2026-09-25). |
| T35 | A scratch worktree for a reference or a replay builds into its own `target/`, never the main repository's. Test binaries carry absolute paths (`CARGO_BIN_EXE_*`, `CARGO_MANIFEST_DIR`), and Cargo's freshness check ignores the workspace's path, so the main tree reuses them. If one ever shared it, run `cargo clean` before the implementer starts. | M5a task 21: the gate failed with `NotFound` in `args_os`, and in most suites with `--no-fail-fast`, after the reference worktree was deleted; the run stalled (review finding 4). |
| T36 | When a milestone makes a value vary that used to be fixed, list every place that reads it and give each a test with the new variation. A spec's "as today" about existing code is a claim to check against the code, not a reassurance. | M5a: results gained labels of their own. The runner and the audit record used them; the ledger kept raising the session's state by the grant's label, as it always had, and the spec said "as today". The tests checked the response and the audit record, not the state (device finding 1). |
| T37 | For every trust boundary in a spec, write two attacks as tests: untrusted text shaped like trusted structure, and a trusted reader that takes in an untrusted result and keeps its trust. Both M5a highs were one of these, and the reference had both. | M5a: a note forging a `notes.md` header (review finding 1 of 2026-09-24); a clean session recalling a tainted note and writing it into identity (device finding 1). |
| T38 | A tool's answer that the model relays must say the outcome in plain words, including what did not happen ("not added to memory.md: …"). A label in parentheses reads as success. | M5a device check: the model twice told the owner "Added to memory.md" for a write that became an untrusted note. After task 22 it reported it correctly. |
| T39 | Name the source directory of copied files in every task's Files section. | M5a task 06 logged a deviation about a files directory the task never named. |
| T40 | Give the gate's shell timeout in the task (600000 ms). On a machine that also serves the model, the gate takes longer than the implementer's default of 2 minutes, and a timed-out gate looks like a failure. | M5a tasks 18 and 19 each ran the gate twice on titan; tasks 21 and 22, which gave the timeout, ran it once. |
| T41 | Before hand-over, walk each given test by hand against the task's own rules. A replay of the reference proves the tests can pass; it cannot show that a test contradicts the task, because the reference was written to pass the test. | Not seen here yet; from the combined Ornith guidance (`~/projects/ornith/GUIDANCE.md` §5): crossbar's given tests pinned event positions and a tie-break no implementation following the task could meet. Listed in `docs/plans/CHECKLIST.md`. |
| T42 | Before hand-over, grep every earlier given file and fixture for anything the task changes (strings, sentences, JSON shapes, counts). | T19 is one case of it (M3a `strict.rs`); crossbar had five such breaks (Ornith guidance §5). M5a task 22 handled it by keeping `edit.rs-task06` and `serve.rs-task10`. |
| T43 | Run the gate's own checks on the given files themselves (`cargo fmt --check`, the line limit, clippy on test targets) before hand-over. A given file that fails the gate stalls the run, and the implementer may not edit it. | M4a task 16: a given test copied into the plan before `cargo fmt` came back reformatted (T29); crossbar stalled on given files that failed its gate (Ornith guidance §5). |
| T44 | Every path a given test creates (directory, socket, file) is removed by a guard that deletes it on drop, never by a line at the end of the test, which a failing test skips. A struct that holds a test's home declares it last, since fields drop in order, and a guard for a running server waits (with a limit) for it to stop before the home goes. Check before hand-over: run the gate and compare `/tmp` before and after. | 2026-09-26: about 10,000 leftovers in titan's `/tmp` from `loopd`, `bxctl`, `inferproxy`, `brokerd` and `toolkit` tests that never removed their paths, plus two `gatewayd` races (a `Running` that only signalled stop, and a home declared first). Fixed in `a15c428`; the gate now leaves nothing. |
| T45 | Say what the implementer does with a skeleton's instruction comment when it fills the body: delete it and keep only the doc comment. Left unsaid, the implementer chooses, and chooses differently from task to task. | M5b: tasks 03 and 04 kept every instruction comment inside the filled bodies (28 lines in `refuse.rs` and `tags.rs`); task 05 removed them. The review deleted them. |
| T46 | For a parser or scanner, give a golden file: the reference's output on a few thousand generated inputs, checked by a given test. Mutations show the given tests catch one-line breaks of the reference; they cannot show the tests pin every clause of every comment, and a scanner has more clauses than tests. | M5b task 10: Ornith's `url_len` ended a clean reply's URL at any `<` outside parentheses instead of the comment's rule; every given test, and all 71 of the reference's mutations' catchers, still passed. A random differential run against the reference found it in the review. |
| T47 | Name the log row in the task ("the row's Task cell is `M5b/14-runbook`"). The driver checks the row by the file's name; the implementer takes it from the task's heading. | M5b task 14: the row was `M5b/14-runbook-sanitizer-rules`, from the heading "the runbook entry `sanitizer-rules`", and `run-plan.sh` stopped on a finished task. |
| T48 | Run the implementer with no plugins and no borrowed rules: `opencode run --pure` and `OPENCODE_DISABLE_CLAUDE_CODE=1`. Global plugins load into every session, and a plugin that rewrites the system prompt forces a full re-read of the conversation on a model that cannot rewind its cache. | M5b run (2026-09-26): a short-term-memory plugin rewrote the system prompt every fourth request (a 35,633-token re-read in task 10), 22 plugin tools rode in every request (about 2,400 tokens), and in tasks 01 to 10 prompt reads of 10,000 tokens or more took 76 of 111 minutes of prompt processing: 42 of them, of which only about 11 were session starts (about 15,000 tokens each). Tasks 11 to 14, run after the plugins were removed, had 5 such reads (4 session starts), 4 minutes in all. |
| T49 | When a task adds requests on a path through another role, count them against that role's limits (`inferproxy`: 8 connections at once, a burst of 10, then 2 a second), and put the real proxy in at least one given test. A fake server reached directly cannot show a limit on the way to it. | M5b task 08: 66 `/tokenize` requests at startup passed every test and failed on straylight's first start, refused by `inferproxy`'s token bucket. |
| T50 | When a task gives a gate test count, give the command that produces it and say what it includes (the passed total of `cargo test --workspace` plus the ignored `end_to_end` test the gate runs on its own). Otherwise, do not make the count a stop condition. Do not ask the implementer to add up `test result:` `N passed` lines: a greedy `.*([0-9]+)` keeps only the last digit (1512 becomes ~1022). After `gate: ok`, counting is done (T59). | M5c-1 tasks 01, 03, 13 and 15: each session found itself one short ("1129 passed — expected 1130") and rediscovered the rule. The count drove extra gate runs, scratch files in `/tmp` (02, 03, 05, 10, 13) and `git stash` baselines (03, 15); about a quarter of the transcripts of 03, 13 and 15 went on it. M5d-2 task 01: summed with that sed, reported 1022, stopped, left T1 and the given test uncommitted; `run-plan` then stopped on the dirty tree. |
| T51 | Never tell the implementer to run a command that the harness denies or that AGENTS.md forbids. Write the branch line as a check ("`git branch --show-current` prints `m5c1`"), not as `git switch`. | M5c-1: all fifteen transcripts start with a refused `git switch m5c1`; AGENTS.md forbids switching branch. |
| T52 | List the known flaky tests in the plan README, with what to do (a test outside the task that fails once, then passes on a rerun and in the gate, is named in the log row's Notes and the task goes on). Before hand-over, stress the whole workspace under load, not only the new tests (T32). | M5c-1 task 01: gatewayd's `brokerd_closing_first_ends_it_as_no_longer_pending` (2 of 10 runs fail on a loaded talos with no M5c-1 change); task 15: loopd's `pointers`. Both older tests; Laguna went on in both cases, and task 01's row calls a different test "the only flake". |
| T53 | Give the baseline in the task so the implementer never has to measure one. Where a check without the change is needed, the stash is `git stash -u`: a plain stash leaves the new untracked tests behind, and they no longer compile. | M5c-1 task 15: a baseline run printed an empty total; task 03 moved the new tests aside by hand before stashing. |
| T54 | Give an import change as a single added `use` line, never as a rewritten import block: the implementer's Edit retypes the lines around it, and the next task's "before" block stops matching. | M5c-1 task 12: its W1 edit rewrote the import block in another order, W2's "before" block then failed to match, and Laguna anchored on its own line instead of stopping (logged as "Deviations: none", I17). Task 06: an M2 replacement landed in the wrong `match`. M5c-2: tasks 06, 07 and 10 rewrote import blocks again; 06 restored rustfmt's order before the commit, 07 anchored H2 after H1 reordered uses. |
| T55 | Put "its spaces are checked" on every edit to a literal whose whitespace matters, not only the first. | M5c-1: task 04's C1 had the note and was right first time; task 14's U1 did not, and got one extra leading space (caught by a test; the row blames the task). |
| T56 | When a task uses an external crate, say that the task's table of its API is complete and that the crate's source (the Cargo registry) is outside the repository and off limits; better, put the signatures needed into `files/`. | M5c-1 tasks 10, 11 and 13 each tried to read `roncho`'s source in `~/.cargo/registry` (`find / … roncho`); only the harness stopped them. |
| T57 | Split a task that combines skeleton fills with more than about fifteen before/after edits over four or more files. | M5c-1: tasks of one skeleton or only listed edits took 6 to 14 minutes; 09 (18), 15 (20, 17 test files and about 30 edits) and 13 (25, two skeletons and ten edits) were the slow ones. |
| T58 | For a class ceiling or trust rule in a spec, list every path that writes to the egress it guards (turns, mirrored memory writes, undo, review accept, notes) and give each path a test at the forbidden class. A rule written for one path is not checked on the others by default. The same for a rule that a class *must* still take a path: give that path a test at that class. | M5c-1 review finding 1: the `secret` ceiling was built and tested for turns only; memory writes from a `secret` session were mirrored to the trusted and fleet workspaces (the reference had it too). Finding 2 is the same shape for undo. M5c-2 review finding 1: a `secret` session's Honcho query was implemented and unit-tested for `has_secret`, but not at the serve layer against the fake Honcho. |
| T59 | After `gate: ok`, the task is done counting. Do not tell the implementer to run `cargo test --workspace` (or the gate) again only to compare a number. AGENTS.md already forbids a second gate "only to count"; say the same of the workspace test run. | M5c-2: all twelve sessions ran `cargo test --workspace` after a passing gate to rediscover the total, even with T50's wording in the task. |
| T60 | The driver must not truncate a task's run log, and must not start a task whose commit and `done` row are already in HEAD. A power-loss resume continues the existing log file; it does not open it for write from offset zero, and it skips a task that is already committed. | M5c-2 task 05: `.state/runs/M5c2/05-memoryd-honcho-readers.log` is four `=== …` resume sections after HEAD was at task 12; the original transcript is gone. Commit `b370c60` was already complete. |
| T61 | Name where the log row goes: immediately under the header, newest first. The implementer otherwise inserts by date among older rows. | M5d-1 task 06. |
| T62 | The implementer log is hundreds of kilobytes. A task that needs a row must say to read only the first 12 lines (title through the header). Reading the whole file exceeds the session context and the driver reports "made no commit". | M5d-2 task 06: after reading the task and supporting modules it opened `docs/implementer-log.md` and OpenCode returned `Context size has been exceeded.`; HEAD did not move. Second run: it skipped the log, listed `tests/`, read `session`/`turn`/`compact`/`config`/`jobs`/`baseline`/`support/mod.rs`, grepped `bxcore`, then the same error on `ids.rs`. |
| T63 | Put the task's `cp` above the Goal, name the crate files they must not Read, and give `tick` as glue over small `todo!()` bodies (T25). A facts table is not enough: Ornith still opens those files and fills the session before step 1. | Same two M5d-2 task 06 runs. |
| T64 | A bookkeeping step after a turn needs a given test on the `Err` path. Happy-path fake-server tests will not catch `if outcome.is_ok()` around `last_heartbeat`. | M5d-2 review finding 1. |

## What worked and should be kept

- Byte-exact fixtures, compared in both directions. No wire-format defect reached review.
- Tests and fixtures copied in task by task and never edited. All 23, then 25, files were
  byte-identical at both reviews.
- A step that makes the implementer prove a test has teeth (task 07: make `new` public, watch the
  `compile_fail` doctest fail, change it back). It was done and logged.
- "Stop and report" instead of improvising. There were no invented dependencies, no `#[allow]`, no
  edited tests and no touched design documents across nine tasks.
- One task, one fresh session, one commit, with only listed paths staged. The history reads as the
  plan.
- A device check against the real model and the owner's server before landing. M4b's non-approver
  approval and M5a's path from a fetched page into identity were found only there; tests,
  reference and review had all passed them.

## What we have seen of the implementers so far

Nine tasks, all in M1, all small and pinned by tests, so treat these as first impressions.

- Follows process rules exactly: branch, staging, trailer, log, protected files.
- Writes idiomatic Rust without panics or unsafe shortcuts, and adds sensible things the task did
  not ask for (`FrameError::source`).
- Misses the unshown half of a general rule (I1).
- In shell, reaches for defensive habits that hide failures (I2).
- Given a clear defect report and a failing test, fixes the defect minimally and generalises.
- Gate on the first run: 6 of 9 tasks. Wall time per task, including the owner's turnaround: about
  10 to 55 minutes.

M2a added two more models. Ornith-1.5-35B-A3B did nine tasks: it found and fixed a real bug in its
own chunked reader, wrote module docs everywhere, and stopped correctly when a prerequisite was
missing; its two failure modes were runaway thinking to the output limit and ending a turn with a
plan instead of a tool call. GLM-5.3 did one task, the assembler, correctly and quickly, with the
best comments of the milestone. Laguna did three and a half tasks in M2a with more nudging than in
M1, and failed as a coordinator. First-gate pass rate for the milestone: 8 of 13.

M2b was Ornith alone, driven by the shell script: ten tasks, ten commits, no intervention, first
gate 4 of 10, and four low findings in review, two of which are the "everywhere" pattern again in
a new shape (paths instead of types). Its code reads well and follows the numbered steps in the
tasks closely; where the task did not walk a path, the rule was not applied there. Task 11 fixed all four in one run and reported its one
deviation, which the task itself had caused.

M4a was Ornith alone, fifteen tasks through the driver: first gate 12 of 15, four review findings, none
above low. Given a skeleton whose comments name the calls and expressions, it writes exactly that,
and its code agreed with the reference on every fuzzed input. Given a `todo!()` with many branches,
or a task that needs the whole crate's API, it plans in prose until the turn runs out; given a
comment that contradicts its function, it notices but deliberates rather than stopping (T29).

M5a was Ornith through the driver, twenty-two tasks over two machines: first gate 7 of 14 on the
main run, and four of the five follow-ups on their first run. Its code matched the reference on
every random input outside two findings, and every follow-up was exact. Its misses were in what it
wrote about the code, not the code: two log rows describe work not in the diff, and one runbook
change was its own text where the task gave a fragment (I17). It stopped correctly on a failure
outside its task once (task 16), and once kept experimenting for over an hour (task 21, I16). The
milestone's two serious defects were the spec's, and the reference had them too (T36, T37).

M5c-1 was Laguna (Ornith-1.5-35B-A3B) through the driver, fifteen tasks in about 3.5 hours with no
intervention, every task ending cleanly. Its code matched the reference in all but six files, each
difference an equivalent rewrite. Its friction was the plan's: an undefined test count (T50), a
forbidden `git switch` in every task (T51), and reaching for a crate's source the task did not
give (T56). The milestone's serious defect was the spec's reach, not the code's: a class ceiling
applied to turns and not to memory writes (T58), in the reference too.

M5c-2 was Ornith through the driver, twelve tasks, first-gate pass in every row, no stop. Three
skeleton bodies were equivalent rewrites of the reference; the rest matched. Extra workspace test
runs after `gate: ok` (I18) and a truncated power-loss resume (T60) were process only. The one
product finding was a missing serve-layer test of a path the code already took (T58).

M5d-1 was Ornith through the driver, eight tasks, first-gate pass in every row, no stop. The only
code difference from the reference is an equivalent rewrite in `compact.rs`. Extra gate runs after
a pass (I18) and a log row inserted among M4b (I19) were process only. The device check passed
spec 7 on the second run, after a trusted identity write then a tainted recall.

M5d-2 was Ornith through the driver, ten tasks. Task 01 stopped twice (wrong branch, then a
last-digit test count) and finished on resume after T50/T59. Task 06 made no commit twice
(context exceeded: whole implementer log, then crate exploration) and finished after T62/T63.
Tasks 07 to 10 then ended cleanly. Product code matches the reference except heartbeat
`last_heartbeat` only on Ok (finding 1, I20, T64). Clippy first-gate on 05 and 06, USAGE indent
on 10, an extra gate on 04 (I18), and a duplicated 03 log row (I19). Device check not in that
review.