docs: M5b review finding 10, the tag check against inferproxy; tip T49

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-27 19:08:42 -07:00
parent 5a2464d81d
commit 6f243ad779
2 changed files with 2 additions and 0 deletions
@@ -86,6 +86,7 @@ How it is used:
 | T46 | For a parser or scanner, give a golden file: the reference's output on a few thousand generated inputs, checked by a given test. Mutations show the given tests catch one-line breaks of the reference; they cannot show the tests pin every clause of every comment, and a scanner has more clauses than tests. | M5b task 10: Ornith's `url_len` ended a clean reply's URL at any `<` outside parentheses instead of the comment's rule; every given test, and all 71 of the reference's mutations' catchers, still passed. A random differential run against the reference found it in the review. |
 | T47 | Name the log row in the task ("the row's Task cell is `M5b/14-runbook`"). The driver checks the row by the file's name; the implementer takes it from the task's heading. | M5b task 14: the row was `M5b/14-runbook-sanitizer-rules`, from the heading "the runbook entry `sanitizer-rules`", and `run-plan.sh` stopped on a finished task. |
 | T48 | Run the implementer with no plugins and no borrowed rules: `opencode run --pure` and `OPENCODE_DISABLE_CLAUDE_CODE=1`. Global plugins load into every session, and a plugin that rewrites the system prompt forces a full re-read of the conversation on a model that cannot rewind its cache. | M5b run (2026-09-26): a short-term-memory plugin rewrote the system prompt every fourth request (a 35,633-token re-read in task 10), 22 plugin tools rode in every request (about 2,400 tokens), and in tasks 01 to 10 prompt reads of 10,000 tokens or more took 76 of 111 minutes of prompt processing: 42 of them, of which only about 11 were session starts (about 15,000 tokens each). Tasks 11 to 14, run after the plugins were removed, had 5 such reads (4 session starts), 4 minutes in all. |
+| T49 | When a task adds requests on a path through another role, count them against that role's limits (`inferproxy`: 8 connections at once, a burst of 10, then 2 a second), and put the real proxy in at least one given test. A fake server reached directly cannot show a limit on the way to it. | M5b task 08: 66 `/tokenize` requests at startup passed every test and failed on straylight's first start, refused by `inferproxy`'s token bucket. |

 ## What worked and should be kept

@@ -832,6 +832,7 @@ almost only in style. The device check (spec, section 13) is still to do.
 | 6 | nit | Ornith (task 10) | The row gives one gate run and a first pass; its notes say the first run failed on six clippy warnings. | Noted here |
 | 7 | medium | the run, not the code | Task 10 ran past the 90-minute limit, and `RESUME` resumes only a session that ended by itself, so the run stopped; the owner resumed it by hand. Global OpenCode plugins loaded into every session: one rewrote the system prompt every fourth request, forcing full re-reads (tasks 01 to 10: prompt reads of 10,000 tokens or more took 76 of 111 minutes of prompt processing). | Plugins removed (owner, 2026-09-27); tip T48; `run-plan.sh` to pass `--pure` and resume after the limit |
 | 8 | low | tests (M4b) | `brokerd`'s `a_torn_tail_is_recovered_and_it_serves` failed once under full load (task 07's first gate) and passed alone. | Open |
+| 10 | high (for the device) | plan (task 08; the reference too) | The self-test asked `/tokenize` about each tag and each neutralised form, 66 requests, one connection each, and `inferproxy` allows a burst of ten (M2a spec): on straylight the ninth tag request was refused and `loopd` would not start. Every given test talks to a fake server directly, with no `inferproxy`. | Review: two requests, all the tags at once, then all the forms (spec 8, runbook, `selftest_tags.rs`); tip T49 |
 | 9 | low | tests (M2b) | `loopd`'s `serve_refuses_to_start_when_the_self_test_fails` and `a_failed_self_test_names_its_entry` wait on the process with no deadline; a `serve` that skipped the self-test would hang the gate (found while writing task 08). | Open |

 What was good: Ornith copied every given file unchanged and made every exact edit as written, in