docs: the M5b review, and tips T45 to T48
The review of tasks 01 to 14 in the implementer log (task 14's row renamed to its file's name), and four tips for writing tasks: what to do with a skeleton's comments, golden files for scanners, naming the log row, and running the implementer without plugins. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -82,6 +82,10 @@ How it is used:
|
||||
| T42 | Before hand-over, grep every earlier given file and fixture for anything the task changes (strings, sentences, JSON shapes, counts). | T19 is one case of it (M3a `strict.rs`); crossbar had five such breaks (Ornith guidance §5). M5a task 22 handled it by keeping `edit.rs-task06` and `serve.rs-task10`. |
|
||||
| T43 | Run the gate's own checks on the given files themselves (`cargo fmt --check`, the line limit, clippy on test targets) before hand-over. A given file that fails the gate stalls the run, and the implementer may not edit it. | M4a task 16: a given test copied into the plan before `cargo fmt` came back reformatted (T29); crossbar stalled on given files that failed its gate (Ornith guidance §5). |
|
||||
| T44 | Every path a given test creates (directory, socket, file) is removed by a guard that deletes it on drop, never by a line at the end of the test, which a failing test skips. A struct that holds a test's home declares it last, since fields drop in order, and a guard for a running server waits (with a limit) for it to stop before the home goes. Check before hand-over: run the gate and compare `/tmp` before and after. | 2026-09-26: about 10,000 leftovers in titan's `/tmp` from `loopd`, `bxctl`, `inferproxy`, `brokerd` and `toolkit` tests that never removed their paths, plus two `gatewayd` races (a `Running` that only signalled stop, and a home declared first). Fixed in `a15c428`; the gate now leaves nothing. |
|
||||
| T45 | Say what the implementer does with a skeleton's instruction comment when it fills the body: delete it and keep only the doc comment. Left unsaid, the implementer chooses, and chooses differently from task to task. | M5b: tasks 03 and 04 kept every instruction comment inside the filled bodies (28 lines in `refuse.rs` and `tags.rs`); task 05 removed them. The review deleted them. |
|
||||
| T46 | For a parser or scanner, give a golden file: the reference's output on a few thousand generated inputs, checked by a given test. Mutations show the given tests catch one-line breaks of the reference; they cannot show the tests pin every clause of every comment, and a scanner has more clauses than tests. | M5b task 10: Ornith's `url_len` ended a clean reply's URL at any `<` outside parentheses instead of the comment's rule; every given test, and all 71 of the reference's mutations' catchers, still passed. A random differential run against the reference found it in the review. |
|
||||
| T47 | Name the log row in the task ("the row's Task cell is `M5b/14-runbook`"). The driver checks the row by the file's name; the implementer takes it from the task's heading. | M5b task 14: the row was `M5b/14-runbook-sanitizer-rules`, from the heading "the runbook entry `sanitizer-rules`", and `run-plan.sh` stopped on a finished task. |
|
||||
| T48 | Run the implementer with no plugins and no borrowed rules: `opencode run --pure` and `OPENCODE_DISABLE_CLAUDE_CODE=1`. Global plugins load into every session, and a plugin that rewrites the system prompt forces a full re-read of the conversation on a model that cannot rewind its cache. | M5b run (2026-09-26): a short-term-memory plugin rewrote the system prompt every fourth request (a 35,633-token re-read in task 10), 22 plugin tools rode in every request (about 2,400 tokens), and in tasks 01 to 10 prompt reads of 10,000 tokens or more took 76 of 111 minutes of prompt processing: 42 of them, of which only about 11 were session starts (about 15,000 tokens each). Tasks 11 to 14, run after the plugins were removed, had 5 such reads (4 session starts), 4 minutes in all. |
|
||||
|
||||
## What worked and should be kept
|
||||
|
||||
|
||||
Reference in New Issue
Block a user