diff --git a/docs/implementer-lessons.md b/docs/implementer-lessons.md index 37cf299..607ab9d 100644 --- a/docs/implementer-lessons.md +++ b/docs/implementer-lessons.md @@ -78,6 +78,9 @@ How it is used: | T38 | A tool's answer that the model relays must say the outcome in plain words, including what did not happen ("not added to memory.md: …"). A label in parentheses reads as success. | M5a device check: the model twice told the owner "Added to memory.md" for a write that became an untrusted note. After task 22 it reported it correctly. | | T39 | Name the source directory of copied files in every task's Files section. | M5a task 06 logged a deviation about a files directory the task never named. | | T40 | Give the gate's shell timeout in the task (600000 ms). On a machine that also serves the model, the gate takes longer than the implementer's default of 2 minutes, and a timed-out gate looks like a failure. | M5a tasks 18 and 19 each ran the gate twice on titan; tasks 21 and 22, which gave the timeout, ran it once. | +| T41 | Before hand-over, walk each given test by hand against the task's own rules. A replay of the reference proves the tests can pass; it cannot show that a test contradicts the task, because the reference was written to pass the test. | Not seen here yet; from the combined Ornith guidance (`~/projects/ornith/GUIDANCE.md` §5): crossbar's given tests pinned event positions and a tie-break no implementation following the task could meet. Listed in `docs/plans/CHECKLIST.md`. | +| T42 | Before hand-over, grep every earlier given file and fixture for anything the task changes (strings, sentences, JSON shapes, counts). | T19 is one case of it (M3a `strict.rs`); crossbar had five such breaks (Ornith guidance §5). M5a task 22 handled it by keeping `edit.rs-task06` and `serve.rs-task10`. | +| T43 | Run the gate's own checks on the given files themselves (`cargo fmt --check`, the line limit, clippy on test targets) before hand-over. A given file that fails the gate stalls the run, and the implementer may not edit it. | M4a task 16: a given test copied into the plan before `cargo fmt` came back reformatted (T29); crossbar stalled on given files that failed its gate (Ornith guidance §5). | ## What worked and should be kept diff --git a/docs/plans/CHECKLIST.md b/docs/plans/CHECKLIST.md new file mode 100644 index 0000000..fbc9f8f --- /dev/null +++ b/docs/plans/CHECKLIST.md @@ -0,0 +1,62 @@ +# Before a plan is handed over + +The design model works through this list for every task before a plan goes to the implementer. +Each item exists because of a recorded failure; the tip numbers point to +`docs/implementer-lessons.md`, and "Ornith guidance" to the combined assessment of three projects +(`~/projects/ornith/GUIDANCE.md`, section 5), whose crossbar findings are not in this repository's +log. + +## The reference + +- [ ] The reference is replayed through `make gate` at each task's end state, on the toolchain of + the host that will run the plan (T34), in a scratch worktree that builds into its own + `target/` (T35). If a worktree ever shared the main `target/`, run `cargo clean` in the main + tree before the run. +- [ ] The reference is broken on purpose one line at a time, and every break that can change + behaviour is caught by the given tests (T28). Record the count in the plan's README. +- [ ] Each task file is read against the reference for anything the reference has that the task + does not ask for (T26). +- [ ] The reference is moved off this machine (a git bundle on straylight) and its local branch + and worktree deleted before the run (T18). + +## The given files + +- [ ] Each given test fails against the tree before the task for exactly the reasons the task + names, and the task lists those tests by name. +- [ ] **Walk each given test by hand against the task's own rules.** A replay cannot catch a test + that no implementation following the task could pass (Ornith guidance: crossbar's pinned + event positions and tie-breaks). +- [ ] **Grep every earlier given file and fixture** for anything this task changes: strings, + JSON shapes, sentences, counts. A later task that changes a shared file gives the earlier + task its own copy (`name-taskNN`, T19; Ornith guidance: five breaks in crossbar). +- [ ] **Run the gate's own checks on the given files**: `cargo fmt --check`, the 500-line limit, + clippy on test targets. A given file that fails the gate stalls the run (Ornith guidance). +- [ ] No test depends on timing: every wait has a deadline on observable state (T27), no test + asserts the order of work done on separate threads (T32), and nothing sleeps instead of + waiting. Stress any test that ever flaked under load. +- [ ] No file the implementer reads holds the model's chat-template tags whole (T33): grep the + plan, the given files and the skeletons for them. + +## The task text + +- [ ] Every quoted "before" block is found exactly once in the tree at that task's start, and every + "after" block is in the reference. +- [ ] New code is a compiling skeleton; no `todo!()` has a comment longer than about six lines + (T25); every comment is read beside its signature (T29); conversions and accessors are + named (`u16::try_from`, `.get(i)`). +- [ ] A call table lists every API the task uses, with signatures (T25). +- [ ] Environment facts the implementer would otherwise go looking for are stated (OS limits, + library quirks, where a file lives). For a client of another program, its contract is + pasted in: a saved response or a schema fragment per call (Ornith guidance). +- [ ] The files section names the source directory of every copied file (T39). +- [ ] The steps give the gate's shell timeout, 600000 ms (T40), and `Cargo.lock` is in the + `git add` line (T24). +- [ ] The commit subject is given exactly. The trailer comes from `tools/run-plan.sh`. +- [ ] "Stop and report if" names the specific conditions: a quoted line not found, a different set + of failing tests, any other test failing, a comment that does not fit its signature. + +## The run + +- [ ] The plan README gives the command, with `OPENCODE_MODEL`, and the expected test count after + each task. +- [ ] The working tree is clean, the reference is off the machine, and the OpenCode TUI is closed.