From ba3c89791f2706a55836f869103960e73b6bd62e Mon Sep 17 00:00:00 2001 From: Kyle Isom Date: Sat, 26 Sep 2026 17:30:44 -0700 Subject: [PATCH] M5b plan: README brought up to the written tasks The task table follows the files (03 tags, 04 refuse, 05 flag) and gives each task's gate count and mutations; the interfaces are the ones the tasks built (task 02's role argument and SecretSpec, Refusal.at, Flag, TurnDone.untrusted required, disarm, LockError::line); the review focus names the tasks that pin each item; which task adds each runbook entry, and the -taskNN convention for replaced given copies. Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/plans/M5b/README.md | 82 ++++++++++++++++++++++++---------------- 1 file changed, 49 insertions(+), 33 deletions(-) diff --git a/docs/plans/M5b/README.md b/docs/plans/M5b/README.md index eb47c9a..d6938e3 100644 --- a/docs/plans/M5b/README.md +++ b/docs/plans/M5b/README.md @@ -37,14 +37,14 @@ Inputs the spec implies but no single task's tests pin down; each line names the add it. 1. **Tags in other shapes**: a listed tag in upper case, split by a zero-width character, or inside - a longer word. Neutralising matches the listed spellings exactly; the zero-width split is - refused in memory (spec 6.1) but reaches `loopd` in a tool result. Task 04 pins what happens to - each shape, so the behaviour is decided, not accidental. + a longer word. Task 03 measured each shape against the tokenizer and pins it: inside a word the + tag is still one token and is neutralised; upper case, zero-width, spaced or full-width forms + are ordinary tokens. Memory refuses the zero-width split as invisible (task 04). 2. **Neutralising grows text past a cap**: `<` is longer than `<`, so a tool result capped at its - limit and then neutralised is a few bytes over. Task 08 pins the order (cap, then neutralise) and - that the log records the result's true length. + limit and then neutralised is a few bytes over. Task 07 pins the order (cap, then neutralise) and + that `truncated` comes from the cap alone. 3. **Non-Latin text and the homoglyph map**: a note written in Russian or Greek must not be flagged - because its letters fold to Latin. Task 05's code-work corpus includes such notes. + because its letters fold to Latin. Task 05's `sanitize_flag_edges.rs` holds such notes. 4. **Links already in code, and Markdown links**: in an untrusted reply, a URL already inside a code span or fence is not wrapped twice, and `[text](url)` is disarmed as a whole. Task 10 pins both. 5. **A lock file that cannot be created** (an unwritable `run/`, or `inferproxy`'s socket directory @@ -52,23 +52,29 @@ add it. ## Tasks -| # | File | Delivers | Crates | Check | -|---|---|---|---|---| -| 00 | (the design model, before the run) | `proto` → `bxcore`, compiler-driven, with the four layers of spec 3; `AGENTS.md`, `CLAUDE.md`, `docs/dependencies.md` and the brief updated | all | done: `2b3ae08`, `9672cb5`, `6798502`; gate 944 | -| 01 | `01-invalid-arguments-detail.md` | `ToolResponse::Denied.detail`; `brokerd` keeps the `ArgsError`; `loopd` appends it | bxcore, brokerd, loopd | fixture; reference | -| 02 | `02-secrets-move.md` | `bxcore::secrets` behind the `secrets` feature; `gatewayd` uses it | bxcore, gatewayd | the moved tests; reference | -| 03 | `03-sanitize-refuse.md` | `sanitize::refuse`: control, invisible, private use, noncharacters, known-prefix secrets, and tags in memory | bxcore | corpora (refuse part); mutations | -| 04 | `04-sanitize-tags.md` | the tag list (from the tokenizer, built from pieces), `sanitize::neutralise`, and the decided handling of review-focus 1 | bxcore | corpora (neutralise part); mutations | -| 05 | `05-sanitize-flag.md` | `sanitize::normalise` and `sanitize::flag`, Hermes' strict patterns (MIT) and M5a's phrases | bxcore | corpora (flag part, incl. non-Latin notes); mutations | -| 06 | `06-memoryd-sanitize.md` | `memoryd` refuses and flags through `sanitize`; a flag keeps the text as an untrusted note; the block leaves out refused entries by rule name; `check.rs` goes | memoryd | given tests; mutations | -| 07 | `07-loopd-neutralise.md` | `loopd` neutralises user messages, tool results (after the cap) and the memory block when they are logged | loopd | given tests; replay byte-identical | -| 08 | `08-loopd-selftest-tags.md` | the self-test's fourth check, against a fake tokenizer | loopd | given tests | -| 09 | `09-turn-done-untrusted.md` | `TurnDone.untrusted` from the session's labels | bxcore, loopd | fixture; given tests | -| 10 | `10-gatewayd-links.md` | images escaped always; URLs in code spans with the notice when untrusted | gatewayd | given tests incl. review-focus 4 | -| 11 | `11-bxctl-sanitize-check.md` | `bxctl sanitize check` | bxctl | given tests | -| 12 | `12-instance-lock.md` | `bxcore::instance::lock`, and `loopd` and `inferproxy` take it | bxcore, loopd, inferproxy | given tests incl. review-focus 5 | -| 13 | `13-instance-lock-rest.md` | `brokerd`, `memoryd` and `gatewayd` take it | brokerd, memoryd, gatewayd | given tests | -| 14 | `14-runbook.md` | the six runbook entries as given fragments | docs | gate's runbook check | +| # | File | Delivers | Crates | Gate after | Mutations | +|---|---|---|---|---|---| +| 00 | (the design model, before the run) | `proto` → `bxcore`, compiler-driven; `AGENTS.md`, `CLAUDE.md`, `docs/dependencies.md` and the brief updated | all | 944 | done: `2b3ae08`, `9672cb5`, `6798502` | +| 01 | `01-invalid-arguments-detail.md` | `ToolResponse::Denied.detail`; `brokerd` keeps the `ArgsError`; `loopd` appends it | bxcore, brokerd, loopd | 956 | 20 of 21 | +| 02 | `02-secrets-move.md` | `bxcore::secrets` behind the `secrets` feature; `gatewayd` uses it | bxcore, gatewayd, loopd (a guard test) | 963 | 23 | +| 03 | `03-sanitize-tags.md` | the 33 tags with their token ids (from the tokenizer, built from pieces), `sanitize::neutralise`, the measured shapes | bxcore | 981 | 16 | +| 04 | `04-sanitize-refuse.md` | `sanitize::refuse`: control, invisible, private use, noncharacters, known-prefix secrets (PGP private keys too), and tags | bxcore | 997 | 40 | +| 05 | `05-sanitize-flag.md` | `sanitize::normalise` and `sanitize::flag`: seven rules from Hermes (MIT, `d0288be`) and M5a's phrases | bxcore | 1023 | 60 | +| 06 | `06-memoryd-sanitize.md` | `memoryd` refuses and flags through `sanitize`; a flag keeps the text as untrusted notes; the block leaves out refused entries by rule name; `check.rs` goes; three runbook entries | memoryd | 1033 | 26 | +| 07 | `07-loopd-neutralise.md` | `loopd` neutralises user messages, tool results (after the cap) and the memory block when they are first logged | loopd | 1042 | 14 | +| 08 | `08-loopd-selftest-tags.md` | the self-test's check "tags tokenize as recorded" (66 `/tokenize` calls), and its runbook entry | loopd | 1050 | 20 | +| 09 | `09-turn-done-untrusted.md` | `TurnDone.untrusted`, required, from the session log's labels | bxcore, loopd | 1058 | 12 | +| 10 | `10-gatewayd-links.md` | images always as code; URLs as code with the notice when untrusted | gatewayd | 1081 | 71 | +| 11 | `11-bxctl-sanitize-check.md` | `bxctl sanitize check` | bxctl | 1099 | 21 | +| 12 | `12-instance-lock.md` | `bxcore::instance::lock`; `loopd` and `inferproxy` take it; runbook `already-running` | bxcore, loopd, inferproxy | 1113 | 24 of 25 | +| 13 | `13-instance-lock-rest.md` | `brokerd`, `memoryd` and `gatewayd` take it | brokerd, memoryd, gatewayd | 1118 | 14 | +| 14 | `14-runbook.md` | the runbook entry `sanitizer-rules`, and two entries pointed at it | docs | 1118 | none (docs) | + +Each count is the gate's test total on titan (rustc 1.98.1) at the end of that task's reference. +The mutations are one-line breaks of the reference, each caught by a given test unless the row +says otherwise: task 01's missed one and task 12's (`TryLockError::Error` read as `Held`, which no +test can provoke) are known. Tasks 06 and 10 are the largest: 06 is 23 exact edits, 10 is 16 small +`todo!()` helpers in a Markdown scanner; watch them in the run. After task 14, the design model runs the device check of spec 13 on straylight and the owner's Mattermost, as for M5a. @@ -78,15 +84,24 @@ Mattermost, as for M5a. Names later tasks rely on; each task file gives the full signatures. - Task 01: `bxcore::ToolResponse::Denied { reason: DenyReason, detail: Option }`. -- Task 02: `bxcore::secrets::{Secret, SecretSource, load}`, feature `secrets`. +- Task 02: `bxcore::secrets::{Secret, SecretSource, SecretSpec, load, read_checked}`, feature + `secrets`; `load` and `read_checked` take the role's name. - Tasks 03 to 05, in `bxcore::sanitize`: - - `refuse(text: &str) -> Option`, where `Refusal` carries the rule's name and sentence - (only memory writes use it in M5b; M5c's outgoing turns may add a narrower variant); - - `neutralise(text: &str) -> Cow<'_, str>` and `TAGS: &[Tag]` (each with its token id); - - `normalise(text: &str) -> String` and `flag(text: &str) -> Option` (the rule's name). -- Task 09: `bxcore::TurnDone { content, usage, untrusted: bool }`. + - `TAGS: &[Tag]` (each with its text and token id) and `neutralise(text: &str) -> Cow<'_, str>`; + - `refuse(text: &str) -> Option`, `Refusal { rule, sentence, at }` (`at` is a byte + offset; the first rule only); + - `normalise(text: &str) -> String` and `flag(text: &str) -> Option`, `Flag { rule }`. +- Task 09: `bxcore::TurnDone { …, untrusted: bool }`, required on the wire. +- Task 10: `gatewayd::links::disarm(text, untrusted) -> Disarmed { text, links }`. - Task 12: `bxcore::instance::lock(path: &Path) -> Result`, with - `LockError::Held` and `LockError::Io`. + `LockError::Held`, `LockError::Io` and `LockError::line(role, path)`. + +Runbook entries, and the task that adds each (the gate fails on a pointer without its entry): +`memory-write-refused`, `memory-write-flagged` and `memory-entry-excluded` (06), +`loopd-tags-unexpected` (08), `already-running` (12), `sanitizer-rules` (14). + +Given copies replaced by a later task keep the earlier version as `-taskNN`, named for the +task that copies it (`broker.rs-task01`, `serve.rs-task01`). ## Before hand-over @@ -95,9 +110,10 @@ each task's end state on titan's toolchain, in a scratch worktree with its own b mutations; given tests walked by hand and grepped against earlier given files; `/tmp` compared around a gate run; the reference moved to straylight as a bundle before the run. -Facts to fetch while writing the tasks, not guessed: the tag list and token ids from Ornith's -tokenizer configuration on straylight (task 04); Hermes' `tools/threat_patterns.py` and its -licence (task 05). +Facts fetched while writing the tasks: the tag list and token ids from the GGUF on straylight, +each checked with `/tokenize` (task 03); Hermes' `tools/threat_patterns.py` at `d0288be` and its +MIT licence (task 05); llama-server's `/tokenize` fields at b10964 (task 08); Mattermost's +Markdown, from its `marked` fork and server parser (task 10). ## Running it