From aea594a1adb946960477b207c39c2925cdd7b001 Mon Sep 17 00:00:00 2001 From: Kyle Isom Date: Sat, 26 Sep 2026 01:00:26 -0700 Subject: [PATCH] M5b spec (draft): hardening bxcore (compiler-driven rename, with the spike's four layers named); the invalid-arguments reason on the wire; the secret store moved into bxcore behind a feature loopd never enables; bxcore::sanitize with refuse, neutralise and flag, its lists, hand-written normalisation, two corpora, a rule-change procedure and `bxctl sanitize check`; memoryd refusing and flagging (a flag keeps the text as an untrusted note); loopd neutralising tags on append, with a tokenizer self-test and the session's trust on TurnDone; gatewayd escaping images and, in untrusted sessions, links; a table of every link between the roles; runbook entries; attack tests per trust boundary and the device checks. Four points left for the owner in section 14. Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/specs/2026-09-26-m5b-hardening.md | 359 +++++++++++++++++++++++++ 1 file changed, 359 insertions(+) create mode 100644 docs/specs/2026-09-26-m5b-hardening.md diff --git a/docs/specs/2026-09-26-m5b-hardening.md b/docs/specs/2026-09-26-m5b-hardening.md new file mode 100644 index 0000000..8024e78 --- /dev/null +++ b/docs/specs/2026-09-26-m5b-hardening.md @@ -0,0 +1,359 @@ +# M5b design: hardening + +Status: draft for the owner's review, 2026-09-26. M5b is the hardening half of the old M5b (P24): +it needs neither `roncho` nor Honcho, and it closes a gap that has been open since M3. Every +decision below was made with the owner during the design (2026-09-25 and 26) and is recorded in +`docs/decisions.md` or cited here; section 14 lists what is still open. + +This document names the model's chat-template tags in words and never writes one out (tip T33). + +## 1. What M5b proves + +From the milestone table (`docs/milestones.md`): + +- A tag in a fetched page reaches the model escaped. +- A memory write holding an invisible character or a key is refused. +- An identity write shaped like an attack becomes a note. +- A memory about code (a hash, a path, a fence, a command) is written unchanged. +- An image link in a reply is posted as text. +- A role whose secret is missing or readable by others does not start, and says which. + +And, from the design: a malformed tool call is refused with the reason, and a reply from a session +that has read untrusted content carries its links as code. + +## 2. Components, in the order the plan builds them + +1. **`bxcore`**: `proto` becomes `bxcore`, the one crate every role may depend on (section 3). +2. **The invalid-arguments reason** on the wire (section 4). +3. **`bxcore::secrets`**: the secret store, moved out of `gatewayd` (section 5). +4. **`bxcore::sanitize`**: refuse, neutralise, flag, with its lists and two corpora (section 6). +5. **`memoryd`** refuses and flags through it (section 7). +6. **`loopd`** neutralises tags and checks them in its self-test (section 8). +7. **`gatewayd`** escapes images and disarms links (section 9). +8. **`bxctl sanitize check`** (section 6.6). +9. The runbook entries (section 12) and the links table (section 10). + +## 3. `bxcore` + +`crates/proto` becomes `crates/bxcore` (package `bxcore`). It holds everything `proto` holds today, +as modules, plus `secrets` (section 5) and `sanitize` (section 6). Nothing else moves in M5b; the +blocking HTTP client that `loopd` and `gatewayd` each have is a candidate for later. + +**The rule** becomes: every role depends on `bxcore` and on no other workspace crate; `bxcore` +depends on no workspace crate. `scripts/check-crate-deps.sh` and its self-test in +`scripts/test-gate-scripts.sh` change to name `bxcore`. `AGENTS.md` and `CLAUDE.md` say so. + +**The name is not `core`**: a crate named `core` shadows Rust's own (checked 2026-09-26: +`core::mem` stops resolving, and derive macros emit `::core` paths). + +**How it is done** (owner, 2026-09-26): the crate and manifests are renamed, and the build says +what breaks; no search-and-replace of the sources. A throwaway spike found four layers, which the +task names so that none is a surprise: + +1. The compiler: 138 errors in 76 files at first, all "`proto` unresolved" (and two type errors + caused by them); 169 source files in the end, once library failures stop hiding test targets. + `inferproxy` has none. +2. `cargo fmt`: `bxcore` sorts differently from `proto`, so imports move in about 179 files. +3. Six fixture paths built as strings in `brokerd`'s and `bxctl`'s tests + (`"{}/../proto/tests/fixtures/audit/{case}"` and the like), and four comments naming + `crates/proto/`. The compiler cannot see them; two audit tests fail at run time. +4. The crate-dependency check and its self-test. + +With all four done, the spike's gate passed: clippy clean, 944 tests, `cargo-deny` clean. + +## 4. The invalid-arguments reason + +Today `brokerd`'s argument parser explains itself (`ArgsError`), and `policy::decide` drops the +explanation: the model is told only "the arguments are not valid for this tool" and has to guess. +On the device (2026-09-25) it spent two calls finding that `note` takes no `file`. + +- `ToolResponse::Denied` gains `detail: Option`, serialised only when present, so every + existing fixture stays valid. It is set for `invalid_arguments` only, from the `ArgsError`'s + `Display`: one line, control characters removed, at most 300 characters. +- `loopd` appends it: ``Denied: the arguments are not valid for this tool: unknown field `file`, + expected `text`.`` +- The audit record is unchanged: it holds the arguments already, so the reason can be reproduced. + The texts `gatewayd` and `bxctl` show the owner are unchanged. + +Echoing the text is safe: every value in these messages came from the model's own call, which is +already in its context and its log. From M5b on, tool results are neutralised as well (section 8). + +## 5. Secrets for every role + +`gatewayd`'s secret store (M4a spec, section 4; P15) moves to `bxcore::secrets` unchanged in +behaviour: a secret is named in a role's configuration as `[secrets.]` with exactly one of +`credential` (a systemd credential, `$CREDENTIALS_DIRECTORY/`, the default in the brief), +`env` or `file` (owner-only, checked against races, with a warning at startup); it is read once at +startup into `Secret`, wiped on drop, never logged; a secret that cannot be read stops the role, +naming it, with the existing runbook entries (`secret-unavailable`, `secret-in-a-file`). + +- The module sits behind a Cargo feature, `secrets`. `gatewayd` enables it now; `brokerd` and + `memoryd` enable it when they first need a secret (the Honcho keys of M5c). `loopd` never does: + it holds no credentials, and its binary does not contain the code that reads them. +- `gatewayd` switches to the shared module; its own copy goes. Its tests move with the code. + +This is the mechanism the owner asked for "for components to obtain secrets on start up". On the +owner's NixOS hosts (M8) it maps to `LoadCredentialEncrypted=` per unit; on other hosts, to files. + +## 6. The sanitizer: `bxcore::sanitize` + +One module, three actions, used by `memoryd` (refuse, flag), `loopd` (neutralise) and `bxctl`. +It replaces `memoryd`'s check (`check.rs`), whose rules move into it (6.1, 6.3). The agent may +work on code, so nothing is refused that ordinary code or notes about code contain (owner, +2026-09-25). + +### 6.1 Refuse: bytes that are only ever attacks + +A text holding any of these is refused, with the rule's name: + +- Control characters other than line feed and tab (M5a's rule, kept). +- Invisible and format characters: M5a's default-ignorable table and its extra list, kept. They + already include the bidirectional controls (U+061C, U+200E, U+200F, U+202A to U+202E, U+2066 to + U+2069), which is the "Trojan Source" attack on code, the zero-width characters, the variation + selectors, the tag block, the Hangul and Braille fillers, U+2028 and U+2029. +- New in M5b: private-use characters (U+E000 to U+F8FF and planes 15 and 16) and noncharacters + (U+FDD0 to U+FDEF, and the last two code points of every plane). +- Known-prefix secrets, each with a minimum length after the prefix so that prose does not match + (`sk-learn` is not a key): GitHub (`ghp_`, `gho_`, `ghu_`, `ghs_`, `ghr_`, `github_pat_`), + GitLab (`glpat-`), `sk-` followed by at least 20 key characters, Slack (`xox` then one of `b`, + `a`, `p`, `r`, `s`, then `-`), AWS access keys (`AKIA` or `ASIA` then 16 upper-case letters or + digits), and private-key blocks (a line starting `-----BEGIN` and ending `PRIVATE KEY-----`). +- The chat-template tags (6.2's list), in memory writes only: memory refuses them outright, which + is stricter than neutralising (M5a rule, kept). + +**Not refused**: hex or base64 runs, links, code fences, headings, commands, paths. Unassigned +code points are not refused either: that needs Unicode's full tables and ties memory to a Unicode +version, for little gain. + +### 6.2 Neutralise: chat-template tags + +**The list** is the model's template and control tags: the think tags, the tool-call tags, the +tool-response tags if the template uses them, and the angle-bar control tokens (start of turn, end +of turn, end of text, and the vision markers). The plan takes the exact list from the model's +tokenizer configuration on straylight and records it in `bxcore` built from pieces, never whole, +each entry with its token id. + +**The replacement** turns the angle brackets of a listed tag into `<` and `>`, leaving the +rest. The model still reads "the closing think tag", but the tokenizer sees ordinary characters. +Measured (`docs/research/2026-09-24-chat-template-tokens.md`): each tag is one token, and inside a +chat message the angle-bar ones become control tokens; an ampersand-escaped form is several +ordinary tokens. + +### 6.3 Flag: text shaped like an attack + +**Patterns** are ported from Hermes' strict threat patterns (`hermes-agent`, +`tools/threat_patterns.py`, MIT; attribution in the module and in `docs/dependencies.md`), whose +rule is "match attack behaviour, not bossy English": instructions to ignore or replace earlier +instructions, to send data to a URL, to dump the conversation, to persist in SSH or agent +configuration. M5a's six instruction phrases join this list: they flag from M5b on, where M5a +refused. + +**Normalisation** before matching, written by hand with no new crate: lower case; full-width ASCII +folded to ASCII; runs of whitespace and zero-width characters collapsed; common leetspeak folded; +and about forty Cyrillic and Greek letters that look Latin mapped to them. + +**What a flag does**: the text is kept, one tier lower, and marked **untrusted**. That reuses what +M5a has, with no new header or tag: + +- An identity write (`add`, `replace`, `consolidate` into `memory.md` or `user.md`) that matches is + written as a note, marked untrusted, and journalled with the rule's name, like an untrusted + session's write. Its answer says so, in the form task 22 set: `not added to memory.md: it matches + the rule , so it was saved in notes.md as an untrusted note`. +- A note that matches is written marked untrusted. +- `recall` returns such a note as `memory-tainted`, which makes the reading session untrusted. + +A flag never refuses and never loses text: the worst case of a false positive is a fact one tier +lower, which the owner can move back by hand. + +### 6.4 The two corpora + +Given tests, in `bxcore`: + +- **The attack corpus**: each case with the action it must get (refuse with rule X, neutralise, + flag with rule Y), from the research documents and the device findings. +- **The code-work corpus**: text that must pass every action untouched: commit hashes, sha256 + values, paths, shell commands, fenced code, URLs, base64 test data, lines like `system:` inside + code, and prose that names the chat-template tags in words. + +### 6.5 Changing a rule (owner, 2026-09-26) + +1. **Evidence**: the rule's name in the journal (`bxctl memory log`) or in a refusal, a device + check, a review, new research. +2. **The case first**: into the attack corpus or the code-work corpus, with a comment saying where + it was seen. It fails against the current rules. +3. **The rule**: an entry added or narrowed, with a comment naming its source. The gate runs both + corpora, so a rule that catches code fails before it lands. +4. **Recording**: a policy change (a new class of refusal, a rule moved from flag to refuse) gets a + row in `docs/decisions.md`; an ordinary rule change is its commit. +5. **Deployment**: the rules are compiled in, so a change reaches an agent when it is rebuilt. + Stored memory is never rewritten; an identity entry that breaks a new refuse rule leaves the + memory block at once (section 7). + +Rules are compiled in rather than read from a file the owner edits, so that every change passes +the corpora and review, and a typo cannot switch a rule off. + +### 6.6 `bxctl sanitize check` + +`bxctl sanitize check ` (or the text on standard input) runs the three actions and prints +each rule that fires, and what it would do, writing nothing: + +```text +refuse secret-github ghp_… at 12 +flag exfil-url "send … to https://…" +neutralise 1 tag +``` + +It exits 0 when nothing fires, 1 when something does. It needs no daemon: it calls +`bxcore::sanitize` directly. + +## 7. `memoryd` + +- Every new or rewritten entry and note goes through refuse (6.1), then flag (6.3). A refused + write fails with the rule's sentence, and nothing is written or journalled. A flagged one is + written as 6.3 says. +- When the memory block is built, each identity entry is checked against the refuse rules again, + as M5a does. An entry that fails is left out and logged, now with the rule's name. So a new + refuse rule protects at once; nothing waits for a migration. +- `soul.md` is still not checked on write (the owner writes it), but `loopd` neutralises the + block (section 8). + +## 8. `loopd` + +**Neutralising.** Text is neutralised when it is first appended and logged, so the logged bytes +are what every later request replays and the inference contract holds: + +- the owner's and other speakers' messages (`User` records); +- tool results, `recall` included, after they are capped; +- the memory block, when the baseline is assembled. + +The model's own output (reasoning, content, tool calls) is not touched: it is logged and replayed +exactly as the server returned it. + +**The self-test** gains a fourth check, run at startup before `loopd` serves anyone: for each tag in +the list, the server's `/tokenize` must give one token for the tag, and more than one for its +neutralised form. If either fails, `loopd` does not start, with a new runbook entry. This catches a +list entry that no longer works after a model or template change; it cannot find a tag missing +from the list, because the server does not list its special tokens. + +**The session's trust.** `loopd` knows whether a session has read untrusted content: its log holds +every tool result with its label. At the end of a turn it puts that in `TurnDone` (section 9). + +## 9. `gatewayd`: images and links + +The owner's Mattermost (11.11.0, checked 2026-09-26 through the bot's client configuration) has +`EnableLinkPreviews = true`, so the server fetches the first link in a post, a plain URL included, +and `HasImageProxy = false`, so every client fetches a post's images itself. Either can carry data +out in a URL, with no click. The posts API has no per-post switch for previews. + +Owner's choice (2026-09-26, "C"): + +- **Images are always escaped.** Every image in a reply, inline (an exclamation mark, then + `[alt](url)`) or reference-style, is posted inside a code span, so it shows as text and nothing is + fetched. +- **In a reply from a session that has read untrusted content**, every URL is also posted inside a + code span, and the reply ends with one line: `Links in this reply are shown as code because this + conversation has read untrusted content.` +- **In a clean session**, links stay as they are: clickable, and previewed by the server. + +`TurnDone` gains `untrusted: bool` (a new field, so its fixture changes). Trusting `loopd`'s own +labels here is sound: this is a defence in depth on presentation, not a trust gate. + +Separately, the owner may want to review `EnableLinkPreviews` for the whole server: it applies to +every post, including other people's. + +## 10. Links between the roles + +Every link as the tree stands after M5b. Paths are the defaults under the role's home; each can be +set in its configuration. Every Unix-socket link carries framed JSON messages from `bxcore` +(length-prefixed, unknown fields refused); one request per connection unless the row says +otherwise. + +| Link | Serves | Connects | Path or address | Carries | When the server is down | +|---|---|---|---|---|---| +| Inference | `inferproxy` | `loopd` | `[infer] socket` | HTTP bytes, unchanged, to llama-server's TCP port | requests retried, then the turn fails (inference contract) | +| Turns | `loopd` | `gatewayd`, `bxctl chat` | `run/loop/loop.sock` | `Turn`, then `TurnEvent`s and `TurnDone`, or `Error` | `gatewayd` reports the turn as not delivered (`LoopDown`) | +| Tool calls | `brokerd` | `loopd` | `run/loop-broker/broker.sock` | `ToolRequest` → `ToolResponse` | calls fail (`broker-unavailable`) | +| Owner's broker | `brokerd` | `bxctl` | `run/owner-broker/admin.sock` | approvals, `Approve`, `Refuse`, `CheckGrants` and their answers | the command fails | +| Approvals | `gatewayd` | `brokerd` | `run/gateway/gateway.sock` | `ApprovalRequest`, `ApprovalPosted`, `ApprovalReply`, `ApprovalOutcome` | approvals still answerable with `bxctl` (`gateway-unreachable`) | +| Memory for turns | `memoryd` | `loopd` | `run/loop-memory/memory.sock` | `MemoryBlock` → `MemoryBlockReply`; `SessionUpdated` | new sessions cannot start (`memory-unavailable`) | +| Memory tools | `memoryd` | `brokerd` | `run/broker-memory/memory.sock` | `MemoryCall` → `ToolResponse` | memory tool calls fail (`memoryd-unreachable`) | +| Owner's memory | `memoryd` | `bxctl` | `run/owner-memory/admin.sock` | `Reindex` → `Reindexed`; `MemoryAdmin` → `MemoryAdminReply` | the command fails | +| Containers | (Podman) | `brokerd` | the `podman` program; `run/egress/` for each call's proxy socket | a tool run per call, arguments on standard input | the call fails with the runner's sentence and its runbook pointer | +| Mattermost | Mattermost | `gatewayd` | `[mattermost] url` (HTTPS, WebSocket) | REST calls, events | `gatewayd` reconnects | + +M5b changes two messages on these links: `ToolResponse::Denied` gains `detail` (tool calls) and +`TurnDone` gains `untrusted` (turns). M5c adds two links: `memoryd` → `brokerd`, a read-only +question about a session's state on a new socket that `brokerd` serves; and `memoryd` → Honcho. + +The plan checks every row of this table against the code before it is handed over. + +## 11. Security notes + +- Neutralisation closes the forged-turn path the research measured, on every text path into the + prompt except the model's own output. Memory keeps its stricter refusal. +- The sanitizer is defence in depth. Taint is the control: an untrusted session's writes become + untrusted notes whatever the sanitizer says, and a flag only ever lowers a write. +- The refuse list is short on purpose, so the agent can work on code; the code-work corpus + enforces that. +- The echo in the invalid-arguments reason adds nothing to the context that was not there. +- Image escaping and link disarming close the automatic-fetch channels in the owner's Mattermost + as configured; a clean session's links are still fetched by the server's previews, by the owner's + choice. +- `loopd`'s binary does not contain the secret loader. + +## 12. Runbook + +New entries, given in the plan as fragments to copy verbatim (lesson of M5a task 13): + +- `memory-write-refused`: a write refused by a sanitizer rule. What you see, how to confirm with + `bxctl sanitize check`, what to do if it is a false positive (6.5). +- `memory-write-flagged`: a write kept as an untrusted note because it matched a flag rule. Where + the reason shows (`bxctl memory log`), and how to move it into `memory.md` by hand. +- `memory-entry-excluded`: an identity entry left out of the memory block by a refuse rule. +- `sanitizer-rules`: the procedure of 6.5. +- `loopd-tags-unexpected`: the self-test found a tag that does not tokenize as recorded. + +## 13. Testing + +Given tests, first, as in every plan: + +- `bxcore`: both corpora; the refuse, neutralise and flag functions; the wire change of + `ToolResponse::Denied` and `TurnDone` with their fixtures; the secret store's existing tests. +- `brokerd`: an invalid call's `detail`, for each kind of `ArgsError`, capped and on one line. +- `memoryd`: a refused write writes nothing; a flagged identity write becomes an untrusted note + with the rule's name in the journal and the answer; an entry that breaks a refuse rule leaves the + block and is logged. +- `loopd`: user messages, tool results and the memory block are neutralised in the log and in the + request; the model's output is not; a replay sends the logged bytes; the self-test fails on a tag + that tokenizes differently (against a fake server); `TurnDone.untrusted` follows the session's + labels. +- `gatewayd`: images escaped in every reply, URLs disarmed only when `untrusted`, with the notice. +- `bxctl`: `sanitize check`'s output and exit status. + +**Attack tests for every trust boundary (T37)**: a fetched page ending the tool result and forging a +system turn reaches the model with the tags escaped; an identity write that asks to send the +conversation to a URL becomes an untrusted note and never reaches the block; a recalled flagged note +makes the reading session untrusted; a reply that embeds data in an image URL is posted as text. + +**On the device** (straylight and the owner's Mattermost, as for M5a): a fetched page holding a tag +arrives escaped (checked in the session log and the request); an image in a reply shows as text; an +untrusted session's links arrive as code with the notice, a clean session's stay clickable; a +memory write holding a key is refused; an attack-shaped identity write becomes an untrusted note; +`bxctl sanitize check` explains each; `loopd`'s self-test passes against the real tokenizer. + +## 14. For the owner's review + +1. **Emoji.** The variation selectors and the zero-width joiner are default-ignorable, so M5a + already refuses them, and the research advises keeping that. It means an emoji written with a + variation selector, or a joined sequence (a family, a flag), is refused in memory. Accept, or + allow the emoji variation selector (U+FE0F) and the zero-width joiner? +2. **M5a's six phrases** move from refuse to flag (6.3). A trusted session could then store + "ignore previous instructions" as an untrusted note where M5a refused it. That seems right for + an agent that works on security code; say if you would rather keep them refused. +3. **Code spans and previews.** That a URL inside a code span gets neither a link nor a server + preview is Mattermost's documented Markdown behaviour for links, but whether the server's + preview fetch skips code spans is not in its documentation. It is checked on the device before + M5b lands; if it does not hold, links in untrusted replies are defanged instead + (`hxxps://…`). +4. **Hermes' patterns** are ported by the design model when writing the plan, from the file named + in 6.3, fetched then and checked against its licence.