kyle/boxmaker
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

M5b spec (draft): hardening

Browse Source 
bxcore (compiler-driven rename, with the spike's four layers named); the
invalid-arguments reason on the wire; the secret store moved into
bxcore behind a feature loopd never enables; bxcore::sanitize with
refuse, neutralise and flag, its lists, hand-written normalisation, two
corpora, a rule-change procedure and `bxctl sanitize check`; memoryd
refusing and flagging (a flag keeps the text as an untrusted note);
loopd neutralising tags on append, with a tokenizer self-test and the
session's trust on TurnDone; gatewayd escaping images and, in untrusted
sessions, links; a table of every link between the roles; runbook
entries; attack tests per trust boundary and the device checks. Four
points left for the owner in section 14.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
kyle
2026-09-26 01:00:26 -07:00
co-authored by Claude Opus 5.5
parent e2993ad869
commit aea594a1ad
1 changed files with 359 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/specs/2026-09-26-m5b-hardening.md
+359
View File
@@ -0,0 +1,359 @@
# M5b design: hardening
Status: draft for the owner's review, 2026-09-26. M5b is the hardening half of the old M5b (P24):
it needs neither `roncho` nor Honcho, and it closes a gap that has been open since M3. Every
decision below was made with the owner during the design (2026-09-25 and 26) and is recorded in
`docs/decisions.md` or cited here; section 14 lists what is still open.
This document names the model's chat-template tags in words and never writes one out (tip T33).
## 1. What M5b proves
From the milestone table (`docs/milestones.md`):
- A tag in a fetched page reaches the model escaped.
- A memory write holding an invisible character or a key is refused.
- An identity write shaped like an attack becomes a note.
- A memory about code (a hash, a path, a fence, a command) is written unchanged.
- An image link in a reply is posted as text.
- A role whose secret is missing or readable by others does not start, and says which.
And, from the design: a malformed tool call is refused with the reason, and a reply from a session
that has read untrusted content carries its links as code.
## 2. Components, in the order the plan builds them
1. **`bxcore`**: `proto` becomes `bxcore`, the one crate every role may depend on (section 3).
2. **The invalid-arguments reason** on the wire (section 4).
3. **`bxcore::secrets`**: the secret store, moved out of `gatewayd` (section 5).
4. **`bxcore::sanitize`**: refuse, neutralise, flag, with its lists and two corpora (section 6).
5. **`memoryd`** refuses and flags through it (section 7).
6. **`loopd`** neutralises tags and checks them in its self-test (section 8).
7. **`gatewayd`** escapes images and disarms links (section 9).
8. **`bxctl sanitize check`** (section 6.6).
9. The runbook entries (section 12) and the links table (section 10).
## 3. `bxcore`
`crates/proto` becomes `crates/bxcore` (package `bxcore`). It holds everything `proto` holds today,
as modules, plus `secrets` (section 5) and `sanitize` (section 6). Nothing else moves in M5b; the
blocking HTTP client that `loopd` and `gatewayd` each have is a candidate for later.
**The rule** becomes: every role depends on `bxcore` and on no other workspace crate; `bxcore`
depends on no workspace crate. `scripts/check-crate-deps.sh` and its self-test in
`scripts/test-gate-scripts.sh` change to name `bxcore`. `AGENTS.md` and `CLAUDE.md` say so.
**The name is not `core`**: a crate named `core` shadows Rust's own (checked 2026-09-26:
`core::mem` stops resolving, and derive macros emit `::core` paths).
**How it is done** (owner, 2026-09-26): the crate and manifests are renamed, and the build says
what breaks; no search-and-replace of the sources. A throwaway spike found four layers, which the
task names so that none is a surprise:
1. The compiler: 138 errors in 76 files at first, all "`proto` unresolved" (and two type errors
caused by them); 169 source files in the end, once library failures stop hiding test targets.
`inferproxy` has none.
2. `cargo fmt`: `bxcore` sorts differently from `proto`, so imports move in about 179 files.
3. Six fixture paths built as strings in `brokerd`'s and `bxctl`'s tests
(`"{}/../proto/tests/fixtures/audit/{case}"` and the like), and four comments naming
`crates/proto/`. The compiler cannot see them; two audit tests fail at run time.
4. The crate-dependency check and its self-test.
With all four done, the spike's gate passed: clippy clean, 944 tests, `cargo-deny` clean.
## 4. The invalid-arguments reason
Today `brokerd`'s argument parser explains itself (`ArgsError`), and `policy::decide` drops the
explanation: the model is told only "the arguments are not valid for this tool" and has to guess.
On the device (2026-09-25) it spent two calls finding that `note` takes no `file`.
- `ToolResponse::Denied` gains `detail: Option<String>`, serialised only when present, so every
existing fixture stays valid. It is set for `invalid_arguments` only, from the `ArgsError`'s
`Display`: one line, control characters removed, at most 300 characters.
- `loopd` appends it: ``Denied: the arguments are not valid for this tool: unknown field `file`,
expected `text`.``
- The audit record is unchanged: it holds the arguments already, so the reason can be reproduced.
The texts `gatewayd` and `bxctl` show the owner are unchanged.
Echoing the text is safe: every value in these messages came from the model's own call, which is
already in its context and its log. From M5b on, tool results are neutralised as well (section 8).
## 5. Secrets for every role
`gatewayd`'s secret store (M4a spec, section 4; P15) moves to `bxcore::secrets` unchanged in
behaviour: a secret is named in a role's configuration as `[secrets.<name>]` with exactly one of
`credential` (a systemd credential, `$CREDENTIALS_DIRECTORY/<name>`, the default in the brief),
`env` or `file` (owner-only, checked against races, with a warning at startup); it is read once at
startup into `Secret`, wiped on drop, never logged; a secret that cannot be read stops the role,
naming it, with the existing runbook entries (`secret-unavailable`, `secret-in-a-file`).
- The module sits behind a Cargo feature, `secrets`. `gatewayd` enables it now; `brokerd` and
`memoryd` enable it when they first need a secret (the Honcho keys of M5c). `loopd` never does:
it holds no credentials, and its binary does not contain the code that reads them.
- `gatewayd` switches to the shared module; its own copy goes. Its tests move with the code.
This is the mechanism the owner asked for "for components to obtain secrets on start up". On the
owner's NixOS hosts (M8) it maps to `LoadCredentialEncrypted=` per unit; on other hosts, to files.
## 6. The sanitizer: `bxcore::sanitize`
One module, three actions, used by `memoryd` (refuse, flag), `loopd` (neutralise) and `bxctl`.
It replaces `memoryd`'s check (`check.rs`), whose rules move into it (6.1, 6.3). The agent may
work on code, so nothing is refused that ordinary code or notes about code contain (owner,
2026-09-25).
### 6.1 Refuse: bytes that are only ever attacks
A text holding any of these is refused, with the rule's name:
- Control characters other than line feed and tab (M5a's rule, kept).
- Invisible and format characters: M5a's default-ignorable table and its extra list, kept. They
already include the bidirectional controls (U+061C, U+200E, U+200F, U+202A to U+202E, U+2066 to
U+2069), which is the "Trojan Source" attack on code, the zero-width characters, the variation
selectors, the tag block, the Hangul and Braille fillers, U+2028 and U+2029.
- New in M5b: private-use characters (U+E000 to U+F8FF and planes 15 and 16) and noncharacters
(U+FDD0 to U+FDEF, and the last two code points of every plane).
- Known-prefix secrets, each with a minimum length after the prefix so that prose does not match
(`sk-learn` is not a key): GitHub (`ghp_`, `gho_`, `ghu_`, `ghs_`, `ghr_`, `github_pat_`),
GitLab (`glpat-`), `sk-` followed by at least 20 key characters, Slack (`xox` then one of `b`,
`a`, `p`, `r`, `s`, then `-`), AWS access keys (`AKIA` or `ASIA` then 16 upper-case letters or
digits), and private-key blocks (a line starting `-----BEGIN` and ending `PRIVATE KEY-----`).
- The chat-template tags (6.2's list), in memory writes only: memory refuses them outright, which
is stricter than neutralising (M5a rule, kept).
**Not refused**: hex or base64 runs, links, code fences, headings, commands, paths. Unassigned
code points are not refused either: that needs Unicode's full tables and ties memory to a Unicode
version, for little gain.
### 6.2 Neutralise: chat-template tags
**The list** is the model's template and control tags: the think tags, the tool-call tags, the
tool-response tags if the template uses them, and the angle-bar control tokens (start of turn, end
of turn, end of text, and the vision markers). The plan takes the exact list from the model's
tokenizer configuration on straylight and records it in `bxcore` built from pieces, never whole,
each entry with its token id.
**The replacement** turns the angle brackets of a listed tag into `&lt;` and `&gt;`, leaving the
rest. The model still reads "the closing think tag", but the tokenizer sees ordinary characters.
Measured (`docs/research/2026-09-24-chat-template-tokens.md`): each tag is one token, and inside a
chat message the angle-bar ones become control tokens; an ampersand-escaped form is several
ordinary tokens.
### 6.3 Flag: text shaped like an attack
**Patterns** are ported from Hermes' strict threat patterns (`hermes-agent`,
`tools/threat_patterns.py`, MIT; attribution in the module and in `docs/dependencies.md`), whose
rule is "match attack behaviour, not bossy English": instructions to ignore or replace earlier
instructions, to send data to a URL, to dump the conversation, to persist in SSH or agent
configuration. M5a's six instruction phrases join this list: they flag from M5b on, where M5a
refused.
**Normalisation** before matching, written by hand with no new crate: lower case; full-width ASCII
folded to ASCII; runs of whitespace and zero-width characters collapsed; common leetspeak folded;
and about forty Cyrillic and Greek letters that look Latin mapped to them.
**What a flag does**: the text is kept, one tier lower, and marked **untrusted**. That reuses what
M5a has, with no new header or tag:
- An identity write (`add`, `replace`, `consolidate` into `memory.md` or `user.md`) that matches is
written as a note, marked untrusted, and journalled with the rule's name, like an untrusted
session's write. Its answer says so, in the form task 22 set: `not added to memory.md: it matches
the rule <name>, so it was saved in notes.md as an untrusted note`.
- A note that matches is written marked untrusted.
- `recall` returns such a note as `memory-tainted`, which makes the reading session untrusted.
A flag never refuses and never loses text: the worst case of a false positive is a fact one tier
lower, which the owner can move back by hand.
### 6.4 The two corpora
Given tests, in `bxcore`:
- **The attack corpus**: each case with the action it must get (refuse with rule X, neutralise,
flag with rule Y), from the research documents and the device findings.
- **The code-work corpus**: text that must pass every action untouched: commit hashes, sha256
values, paths, shell commands, fenced code, URLs, base64 test data, lines like `system:` inside
code, and prose that names the chat-template tags in words.
### 6.5 Changing a rule (owner, 2026-09-26)
1. **Evidence**: the rule's name in the journal (`bxctl memory log`) or in a refusal, a device
check, a review, new research.
2. **The case first**: into the attack corpus or the code-work corpus, with a comment saying where
it was seen. It fails against the current rules.
3. **The rule**: an entry added or narrowed, with a comment naming its source. The gate runs both
corpora, so a rule that catches code fails before it lands.
4. **Recording**: a policy change (a new class of refusal, a rule moved from flag to refuse) gets a
row in `docs/decisions.md`; an ordinary rule change is its commit.
5. **Deployment**: the rules are compiled in, so a change reaches an agent when it is rebuilt.
Stored memory is never rewritten; an identity entry that breaks a new refuse rule leaves the
memory block at once (section 7).
Rules are compiled in rather than read from a file the owner edits, so that every change passes
the corpora and review, and a typo cannot switch a rule off.
### 6.6 `bxctl sanitize check`
`bxctl sanitize check <text>` (or the text on standard input) runs the three actions and prints
each rule that fires, and what it would do, writing nothing:
```text
refuse secret-github ghp_… at 12
flag exfil-url "send … to https://…"
neutralise 1 tag
```
It exits 0 when nothing fires, 1 when something does. It needs no daemon: it calls
`bxcore::sanitize` directly.
## 7. `memoryd`
- Every new or rewritten entry and note goes through refuse (6.1), then flag (6.3). A refused
write fails with the rule's sentence, and nothing is written or journalled. A flagged one is
written as 6.3 says.
- When the memory block is built, each identity entry is checked against the refuse rules again,
as M5a does. An entry that fails is left out and logged, now with the rule's name. So a new
refuse rule protects at once; nothing waits for a migration.
- `soul.md` is still not checked on write (the owner writes it), but `loopd` neutralises the
block (section 8).
## 8. `loopd`
**Neutralising.** Text is neutralised when it is first appended and logged, so the logged bytes
are what every later request replays and the inference contract holds:
- the owner's and other speakers' messages (`User` records);
- tool results, `recall` included, after they are capped;
- the memory block, when the baseline is assembled.
The model's own output (reasoning, content, tool calls) is not touched: it is logged and replayed
exactly as the server returned it.
**The self-test** gains a fourth check, run at startup before `loopd` serves anyone: for each tag in
the list, the server's `/tokenize` must give one token for the tag, and more than one for its
neutralised form. If either fails, `loopd` does not start, with a new runbook entry. This catches a
list entry that no longer works after a model or template change; it cannot find a tag missing
from the list, because the server does not list its special tokens.
**The session's trust.** `loopd` knows whether a session has read untrusted content: its log holds
every tool result with its label. At the end of a turn it puts that in `TurnDone` (section 9).
## 9. `gatewayd`: images and links
The owner's Mattermost (11.11.0, checked 2026-09-26 through the bot's client configuration) has
`EnableLinkPreviews = true`, so the server fetches the first link in a post, a plain URL included,
and `HasImageProxy = false`, so every client fetches a post's images itself. Either can carry data
out in a URL, with no click. The posts API has no per-post switch for previews.
Owner's choice (2026-09-26, "C"):
- **Images are always escaped.** Every image in a reply, inline (an exclamation mark, then
`[alt](url)`) or reference-style, is posted inside a code span, so it shows as text and nothing is
fetched.
- **In a reply from a session that has read untrusted content**, every URL is also posted inside a
code span, and the reply ends with one line: `Links in this reply are shown as code because this
conversation has read untrusted content.`
- **In a clean session**, links stay as they are: clickable, and previewed by the server.
`TurnDone` gains `untrusted: bool` (a new field, so its fixture changes). Trusting `loopd`'s own
labels here is sound: this is a defence in depth on presentation, not a trust gate.
Separately, the owner may want to review `EnableLinkPreviews` for the whole server: it applies to
every post, including other people's.
## 10. Links between the roles
Every link as the tree stands after M5b. Paths are the defaults under the role's home; each can be
set in its configuration. Every Unix-socket link carries framed JSON messages from `bxcore`
(length-prefixed, unknown fields refused); one request per connection unless the row says
otherwise.
| Link | Serves | Connects | Path or address | Carries | When the server is down |
|---|---|---|---|---|---|
| Inference | `inferproxy` | `loopd` | `[infer] socket` | HTTP bytes, unchanged, to llama-server's TCP port | requests retried, then the turn fails (inference contract) |
| Turns | `loopd` | `gatewayd`, `bxctl chat` | `run/loop/loop.sock` | `Turn`, then `TurnEvent`s and `TurnDone`, or `Error` | `gatewayd` reports the turn as not delivered (`LoopDown`) |
| Tool calls | `brokerd` | `loopd` | `run/loop-broker/broker.sock` | `ToolRequest` → `ToolResponse` | calls fail (`broker-unavailable`) |
| Owner's broker | `brokerd` | `bxctl` | `run/owner-broker/admin.sock` | approvals, `Approve`, `Refuse`, `CheckGrants` and their answers | the command fails |
| Approvals | `gatewayd` | `brokerd` | `run/gateway/gateway.sock` | `ApprovalRequest`, `ApprovalPosted`, `ApprovalReply`, `ApprovalOutcome` | approvals still answerable with `bxctl` (`gateway-unreachable`) |
| Memory for turns | `memoryd` | `loopd` | `run/loop-memory/memory.sock` | `MemoryBlock` → `MemoryBlockReply`; `SessionUpdated` | new sessions cannot start (`memory-unavailable`) |
| Memory tools | `memoryd` | `brokerd` | `run/broker-memory/memory.sock` | `MemoryCall` → `ToolResponse` | memory tool calls fail (`memoryd-unreachable`) |
| Owner's memory | `memoryd` | `bxctl` | `run/owner-memory/admin.sock` | `Reindex` → `Reindexed`; `MemoryAdmin` → `MemoryAdminReply` | the command fails |
| Containers | (Podman) | `brokerd` | the `podman` program; `run/egress/` for each call's proxy socket | a tool run per call, arguments on standard input | the call fails with the runner's sentence and its runbook pointer |
| Mattermost | Mattermost | `gatewayd` | `[mattermost] url` (HTTPS, WebSocket) | REST calls, events | `gatewayd` reconnects |
M5b changes two messages on these links: `ToolResponse::Denied` gains `detail` (tool calls) and
`TurnDone` gains `untrusted` (turns). M5c adds two links: `memoryd` → `brokerd`, a read-only
question about a session's state on a new socket that `brokerd` serves; and `memoryd` → Honcho.
The plan checks every row of this table against the code before it is handed over.
## 11. Security notes
- Neutralisation closes the forged-turn path the research measured, on every text path into the
prompt except the model's own output. Memory keeps its stricter refusal.
- The sanitizer is defence in depth. Taint is the control: an untrusted session's writes become
untrusted notes whatever the sanitizer says, and a flag only ever lowers a write.
- The refuse list is short on purpose, so the agent can work on code; the code-work corpus
enforces that.
- The echo in the invalid-arguments reason adds nothing to the context that was not there.
- Image escaping and link disarming close the automatic-fetch channels in the owner's Mattermost
as configured; a clean session's links are still fetched by the server's previews, by the owner's
choice.
- `loopd`'s binary does not contain the secret loader.
## 12. Runbook
New entries, given in the plan as fragments to copy verbatim (lesson of M5a task 13):
- `memory-write-refused`: a write refused by a sanitizer rule. What you see, how to confirm with
`bxctl sanitize check`, what to do if it is a false positive (6.5).
- `memory-write-flagged`: a write kept as an untrusted note because it matched a flag rule. Where
the reason shows (`bxctl memory log`), and how to move it into `memory.md` by hand.
- `memory-entry-excluded`: an identity entry left out of the memory block by a refuse rule.
- `sanitizer-rules`: the procedure of 6.5.
- `loopd-tags-unexpected`: the self-test found a tag that does not tokenize as recorded.
## 13. Testing
Given tests, first, as in every plan:
- `bxcore`: both corpora; the refuse, neutralise and flag functions; the wire change of
`ToolResponse::Denied` and `TurnDone` with their fixtures; the secret store's existing tests.
- `brokerd`: an invalid call's `detail`, for each kind of `ArgsError`, capped and on one line.
- `memoryd`: a refused write writes nothing; a flagged identity write becomes an untrusted note
with the rule's name in the journal and the answer; an entry that breaks a refuse rule leaves the
block and is logged.
- `loopd`: user messages, tool results and the memory block are neutralised in the log and in the
request; the model's output is not; a replay sends the logged bytes; the self-test fails on a tag
that tokenizes differently (against a fake server); `TurnDone.untrusted` follows the session's
labels.
- `gatewayd`: images escaped in every reply, URLs disarmed only when `untrusted`, with the notice.
- `bxctl`: `sanitize check`'s output and exit status.
**Attack tests for every trust boundary (T37)**: a fetched page ending the tool result and forging a
system turn reaches the model with the tags escaped; an identity write that asks to send the
conversation to a URL becomes an untrusted note and never reaches the block; a recalled flagged note
makes the reading session untrusted; a reply that embeds data in an image URL is posted as text.
**On the device** (straylight and the owner's Mattermost, as for M5a): a fetched page holding a tag
arrives escaped (checked in the session log and the request); an image in a reply shows as text; an
untrusted session's links arrive as code with the notice, a clean session's stay clickable; a
memory write holding a key is refused; an attack-shaped identity write becomes an untrusted note;
`bxctl sanitize check` explains each; `loopd`'s self-test passes against the real tokenizer.
## 14. For the owner's review
1. **Emoji.** The variation selectors and the zero-width joiner are default-ignorable, so M5a
already refuses them, and the research advises keeping that. It means an emoji written with a
variation selector, or a joined sequence (a family, a flag), is refused in memory. Accept, or
allow the emoji variation selector (U+FE0F) and the zero-width joiner?
2. **M5a's six phrases** move from refuse to flag (6.3). A trusted session could then store
"ignore previous instructions" as an untrusted note where M5a refused it. That seems right for
an agent that works on security code; say if you would rather keep them refused.
3. **Code spans and previews.** That a URL inside a code span gets neither a link nor a server
preview is Mattermost's documented Markdown behaviour for links, but whether the server's
preview fetch skips code spans is not in its documentation. It is checked on the device before
M5b lands; if it does not hold, links in untrusted replies are defanged instead
(`hxxps://…`).
4. **Hermes' patterns** are ported by the design model when writing the plan, from the file named
in 6.3, fetched then and checked against its licence.