From d1e5b879e2c0d39e968d2404ae4e1525a968f7cd Mon Sep 17 00:00:00 2001 From: Kyle Isom Date: Sat, 26 Sep 2026 04:49:56 -0700 Subject: [PATCH] ASSESSMENT: sift P5a complete; review with real inputs Co-Authored-By: Claude Opus 5.5 --- ASSESSMENT.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/ASSESSMENT.md b/ASSESSMENT.md index 42350d3..8d03a1c 100644 --- a/ASSESSMENT.md +++ b/ASSESSMENT.md @@ -37,7 +37,7 @@ during the run" sections of each `docs/plans/*/README.md`, and the commits. | v2.1 | 2 | 2/2 | 0 | 2 | 1 refusal-ending | | | v2.2 | 2 | 1/2 (01 failed its first gate after the restart) | 0 | 2 | 0 | 21 and 8 min | | v2.3 | 4 | 4/4 | 1 | 2 | 1 timing hack, 1 refusal-ending, 1 malformed tool call, 1 duplicate call | | -| sift P5a 01 | 1 | (in progress) | 0 | 1 | 1 refusal-ending, 1 hung session (tooling) | 55 min lost to an owner test bug | +| sift P5a | 8 (06 split in two) | every task that reached the gate | 0 | 3 (deadlocked test; oversized task 06, looped 3 h; named destinations left out of the spec) | 1 refusal-ending, 1 malformed call, 3 defects found in review (outline empty on LaTeX/Acrobat PDFs, pool race, slow over-limit) | 2 tooling hangs (OpenCode stdin); merged 2026-09-26 | **Totals:** 28 crossbar tasks, all finished and merged. - **Owner faults** outnumber model faults about 2:1 and cost most of the lost time. @@ -171,9 +171,11 @@ Ornith's time was lost mostly to the owner's tests and task text. By category: - one fresh session per task, with the checks listed in §1; - kill a stuck session **by pid** (`pkill -f` matches the driver too); - add a per-session timeout (see §8). -9. **Review is not optional.** Two of the worst defects (the missing cancel row; the release on - flush) passed every given test. Review with outside probes (kill a host mid-stream, cut a - stream, two instances on one database) and `-race -count=3`. +9. **Review is not optional, and it must use real inputs.** Defects that passed every given test: + the missing cancel row, the release on flush, and in sift an outline that was empty for every + LaTeX and Acrobat PDF. The synthetic fixtures used direct page references, while real + papers use named destinations. Review with outside probes (kill a host mid-stream, cut a + stream, two instances on one database, real documents end to end) and `-race -count=3`. 10. **Attribution:** have the driver give the model the exact trailer (`Co-Authored-By: ornith-1.5-35b-a3b `) and refuse a commit without it (done in sift's `run-plan.sh`; crossbar's older commits carry