kyle/crossbar
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ASSESSMENT: sift P5a complete; review with real inputs

Browse Source 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
kyle
2026-09-26 04:49:56 -07:00
co-authored by Claude Opus 5.5
parent 2683df51bd
commit d1e5b879e2
1 changed files with 6 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ASSESSMENT.md
+6 -4
View File
@@ -37,7 +37,7 @@ during the run" sections of each `docs/plans/*/README.md`, and the commits.
| v2.1 | 2 | 2/2 | 0 | 2 | 1 refusal-ending | | | v2.1 | 2 | 2/2 | 0 | 2 | 1 refusal-ending | |
| v2.2 | 2 | 1/2 (01 failed its first gate after the restart) | 0 | 2 | 0 | 21 and 8 min | | v2.2 | 2 | 1/2 (01 failed its first gate after the restart) | 0 | 2 | 0 | 21 and 8 min |
| v2.3 | 4 | 4/4 | 1 | 2 | 1 timing hack, 1 refusal-ending, 1 malformed tool call, 1 duplicate call | | | v2.3 | 4 | 4/4 | 1 | 2 | 1 timing hack, 1 refusal-ending, 1 malformed tool call, 1 duplicate call | |
| sift P5a 01 | 1 | (in progress) | 0 | 1 | 1 refusal-ending, 1 hung session (tooling) | 55 min lost to an owner test bug | | sift P5a | 8 (06 split in two) | every task that reached the gate | 0 | 3 (deadlocked test; oversized task 06, looped 3 h; named destinations left out of the spec) | 1 refusal-ending, 1 malformed call, 3 defects found in review (outline empty on LaTeX/Acrobat PDFs, pool race, slow over-limit) | 2 tooling hangs (OpenCode stdin); merged 2026-09-26 |
**Totals:** 28 crossbar tasks, all finished and merged. **Totals:** 28 crossbar tasks, all finished and merged.
- **Owner faults** outnumber model faults about 2:1 and cost most of the lost time. - **Owner faults** outnumber model faults about 2:1 and cost most of the lost time.
@@ -171,9 +171,11 @@ Ornith's time was lost mostly to the owner's tests and task text. By category:
- one fresh session per task, with the checks listed in §1; - one fresh session per task, with the checks listed in §1;
- kill a stuck session **by pid** (`pkill -f` matches the driver too); - kill a stuck session **by pid** (`pkill -f` matches the driver too);
- add a per-session timeout (see §8). - add a per-session timeout (see §8).
9. **Review is not optional.** Two of the worst defects (the missing cancel row; the release on 9. **Review is not optional, and it must use real inputs.** Defects that passed every given test:
flush) passed every given test. Review with outside probes (kill a host mid-stream, cut a the missing cancel row, the release on flush, and in sift an outline that was empty for every
stream, two instances on one database) and `-race -count=3`. LaTeX and Acrobat PDF. The synthetic fixtures used direct page references, while real
papers use named destinations. Review with outside probes (kill a host mid-stream, cut a
stream, two instances on one database, real documents end to end) and `-race -count=3`.
10. **Attribution:** have the driver give the model the exact trailer 10. **Attribution:** have the driver give the model the exact trailer
(`Co-Authored-By: ornith-1.5-35b-a3b <ornith-1.5-35b-a3b@llama.cpp.invalid>`) and refuse a (`Co-Authored-By: ornith-1.5-35b-a3b <ornith-1.5-35b-a3b@llama.cpp.invalid>`) and refuse a
commit without it (done in sift's `run-plan.sh`; crossbar's older commits carry commit without it (done in sift's `run-plan.sh`; crossbar's older commits carry