memoryd: refuse and flag memory writes through bxcore::sanitize

Co-Authored-By: ornith-1.5-35b-a3b <agent@wntrmute.dev>

docs/implementer-log.md

@@ -6,6 +6,7 @@ reviewer adds findings under "Reviews" once per milestone.
 
 | Task | Date | Status | Gate runs | First gate | Deviations | Notes | Model |
 |---|---|---|---|---|---|---|---|
+| M5b/06-memoryd-sanitize | 2026-09-26 | done | 1 | pass | none | Copied the five given tests from `docs/plans/M5b/files/`: `sanitize.rs`, `serve_sanitize.rs`, and new versions of `edit.rs`, `files.rs` and `serve.rs` (did not open them). Step 2 showed exactly the seven named failures (`refusals_change_nothing` in `edit`, `an_entry_that_fails_the_check_is_left_out_and_logged` in `serve`, all five in `serve_sanitize`); `files` did not compile (one E0609, no field `rule` on `&memoryd::check::Refused`) and `sanitize` did not compile (13: E0432 for `memoryd::edit::flagged`/`refused`, E0609 for `flag` on `Outcome` and `&Record` and `rule` on `Refused`, E0615 for `sentence`). Appended the three runbook fragments to `docs/runbook.md`; `check-runbook.sh` printed nothing and exited 0. `git rm`'d `crates/memoryd/src/check.rs` and its test. Made the edits in order: E1 removed `pub mod check;` from `lib.rs`; E2-E10 in `edit.rs` (imports, `Outcome` gains `flag`, five functions replacing `checked`, the refuse check in `apply`, the flag check after `edited`, `flag: None`, `text.trim()` in `edited`, `noted` plus `as_notes`, `text.trim()` in `consolidate`); E11-E13 in `files.rs` (imports, `Block`, the two `block` comments, the `refuse` match); E14-E19 in `journal.rs` (`Record` gains `flag` after `untrusted`, `record` sets `flag: outcome.flag.map(str::to_string)`, `log` appends `  flagged {rule}`, `undo` sets `flag: None`); E20 in `review.rs` (`reject` sets `flag: None`); E21-E23 in `serve.rs` (`block` logs by `refusal.rule` with the runbook pointer, the `memory` comment, and `memory` calls `edit::refused` first then `edit::flagged` to keep flagged writes off review and logs the rule). `cargo fmt --all`; `cargo check -p memoryd` clean; `cargo test -p memoryd` every suite passes (sanitize 11, serve_sanitize 5, edit 8, files 8, serve 8); `make gate` printed `gate: ok` with 1033 tests; `grep -rn 'memoryd::check\|crate::check' crates/` prints nothing. No new dependency, so `Cargo.lock` is unchanged. | ornith-1.5-35b-a3b |
 | M5b/05-sanitize-flag | 2026-09-26 | done | 1 | pass | none | Copied the three given tests from `docs/plans/M5b/files/`: `bxcore/tests/sanitize_flag.rs`, `sanitize_flag_corpus.rs`, `sanitize_flag_edges.rs`. Step 2 showed exactly the named E0432s (`Flag`/`flag`/`normalise` for `sanitize_flag`, `flag` for the other two). Copied the two skeletons, `bxcore/src/sanitize/normalise.rs` (five functions) and `flag.rs` (seven), then made the two edits: E1 replaced `bxcore/src/sanitize/mod.rs` whole (added `mod flag; mod normalise;` and `pub use flag::{Flag, flag};` / `pub use normalise::normalise;`), E2 made `invisible` `pub(super)` in `refuse.rs` (nothing else changed). Filled `normalise.rs` in the given order, each from its comment and `cargo check` after: `leet` (`LEET.iter().find` the digit, `map_or(c, |&(_, letter)| letter)`); `homoglyph` (`HOMOGLYPHS.iter().find` the `from`, `map` the `to`); `full_width` (`u32::from(c)` inside `FULL_WIDTH`, `checked_sub(FULL_WIDTH_OFFSET).and_then(char::from_u32).unwrap_or(c)`); `push_folded` (`full_width` then `homoglyph`, pushing the letter when `Some`, else `c.to_lowercase().map(leet)`); `normalise` (walk chars, `space` bool: whitespace sets it, `invisible` dropped leaving it, any other char pushes one `' '` when `space` and the output is not empty, clears `space`, then `push_folded`). Filled `flag.rs` in the given order: `in_sentence` (`!word.ends_with(['.','!','?',';']) && !["&&","||","|"].contains(&word)`); `filler` (`!word.is_empty() && word.chars().all(|c| c.is_alphanumeric() || c == '_')`); `is` (trim non-word ends with `trim_matches`, compare the word and its stripped form to each keyword); `ends_clause` (`word.ends_with(CLAUSE_ENDS)`, or `after.is_empty()`, or `after` starts with a `CLAUSE_FOLLOWERS` matched word by word as `is`); `one_word` (`split_first`, `test` the first word with the words after it, then `seq_at` the remaining parts on those words); `skip` (counts `0..=max`, `split_at_checked`: a skipped word failing `test` or a count past the end returns `false`, otherwise when all pass check `seq_at`); `seq_at` (`split_first`; `Is`/`Ends`/`Holds` go to `one_word` with `is`, `is && ends_clause`, `contains`, `Filler`/`Gap` to `skip` with `filler`/`in_sentence`). My first `one_word` edit dropped a `]` and `)` from the `impl Fn` signature; caught by diffing against the skeleton and fixed. After the fills `cargo check -p bxcore` is clean. `sanitize_flag` 22 passed, `sanitize_flag_corpus` 2, `sanitize_flag_edges` 2; `sanitize_refuse` 14, `sanitize_refuse_corpus` 2, `sanitize_tags` 15, `sanitize_corpus` 3. `cargo fmt --all`; `make gate` printed `gate: ok`; `cargo test --workspace` reports 1022 passed, plus the gate's separate loopd `end_to_end` ignored test, for 1023 total. No new dependency, so `Cargo.lock` is unchanged. | ornith-1.5-35b-a3b |
 | M5b/04-sanitize-refuse | 2026-09-26 | done | 1 | pass | none | Copied the given files from `docs/plans/M5b/files/`: the skeleton `bxcore/src/sanitize/refuse.rs` and the tests `bxcore/tests/sanitize_refuse.rs`, `bxcore/tests/sanitize_refuse_corpus.rs`, `bxcore/tests/support/keys.rs`. `bxcore/src/sanitize/mod.rs`: replaced the whole file (E1), adding `mod refuse;` and `pub use refuse::{Refusal, refuse};`. `bxcore/src/sanitize/tags.rs`: `next_tag` became `pub(super)` (E2), nothing else changed. Filled the eight functions in `refuse.rs` in the given order, each from its comment: `control` (`c.is_control() && c != '\n' && c != '\t'`), `first_char` (`char_indices().find` the first char the rule accepts, `map` to its byte offset), `private_use` (`u32::from(c)` inside one inclusive `PRIVATE_USE` range, as `invisible` does for `DEFAULT_IGNORABLE`), `noncharacter` (`0xFDD0..=0xFDEF` or `code & 0xFFFE == 0xFFFE`), `listed_tag` (`next_tag(text)` mapped to `before.len()`), `starts_word` (`!before.chars().next_back().is_some_and(|c| c.is_ascii_alphanumeric() || c == '_' || c == '-')`), `key_run` (`take_while` the run, counting chars and noting an upper and a digit, length in range and, if `mixed`, both seen), `private_key` (`match_indices("-----BEGIN")`, the line before trims empty and the BEGIN line trim_end()s onto one of `PRIVATE_KEY_ENDS`). Step 2 showed the three named E0432; step 3 compiled with the unfilled-function dead-code warnings. `key_run` failed once on `RangeInclusive` precedence: `secret.min..=secret.max.contains(&count)` parsed as a range, fixed by `(secret.min..=secret.max).contains(&count)`. After the edits `cargo check -p bxcore` is clean; `sanitize_refuse` 14 passed, `sanitize_refuse_corpus` 2 passed, `sanitize_tags` 15, `sanitize_corpus` 3. `cargo fmt --all`; `make gate` printed `gate: ok`; `cargo test --workspace` reports 996 passed, plus the gate's separate loopd `end_to_end` ignored test, for 997 total. No new dependency, so `Cargo.lock` is unchanged. | ornith-1.5-35b-a3b |
 | M5b/03-sanitize-tags | 2026-09-26 | done | 1 | pass | none | Copy of the skeleton `bxcore/src/sanitize/tags.rs` and the three tests `bxcore/tests/sanitize_tags.rs`, `bxcore/tests/sanitize_corpus.rs`, `bxcore/tests/support/tags.rs`. Created `bxcore/src/sanitize/mod.rs` (E1); added `pub mod sanitize;` to `bxcore/src/lib.rs` between `pub mod memory;` and the `#[cfg(feature = "secrets")]` line (E2). Filled `push_escaped` (walk `tag.text.chars()`, write the first bracket as `&lt;` and the second as `&gt;`, push every other character as itself). Filled `next_tag` (iterate `text.match_indices('<')`, `split_at_checked` the offset, `TAGS.iter().find` the tag the rest starts with, return the text before, the tag, and the text after; otherwise fall through to `None`). Step 2 showed the two named E0432; step 3 compiled with the two `todo!()`s, 3 pass and 12 fail in `sanitize_tags`, 3 fail in `sanitize_corpus`, every failure `not yet implemented`. My first `next_tag` returned `None` on the first `<` that did not start a tag, so `attack_texts_come_back_neutralised` and `the_forged_turns_come_back_exactly_so` each left a tag (the failure named it); changed the `find` to `if let Some` so the loop continues, matching the comment's "After the loop, None". After the edits the step-3 command shows 15 passed in `sanitize_tags` and 3 passed in `sanitize_corpus`; `cargo check -p bxcore` is clean. `cargo fmt --all`; `make gate` printed `gate: ok`; `cargo test --workspace` reports 980 passed, plus the gate's separate loopd `end_to_end` ignored test, for 981 total. No new dependency, so `Cargo.lock` is unchanged. | ornith-1.5-35b-a3b |

docs/runbook.md

@@ -651,3 +651,80 @@ fails too, stop `memoryd`, move `index/memory.sqlite3` aside, and start `memoryd
 it.
 
 **Check.** `bxctl reindex` prints how many logs, records and notes it read.
+
+## memory-write-refused
+
+**What you see.** The agent says memory refused something: its `memory` call was answered
+`memory refused this entry: it contains …` (what looks like a GitHub token, an invisible or
+direction-changing character, and so on), and `memoryd` logged `memoryd: a memory write from
+session <id> was refused by the rule <rule>; see docs/runbook.md#memory-write-refused`. Nothing
+was written to `memory/`, staged for review or journalled.
+
+**Why.** Every text a memory write would store is checked against the sanitizer's refuse rules
+(M5b spec, section 6.1): control characters other than line feed and tab, invisible and
+direction-changing characters, private-use characters, noncharacters, secrets known by their
+prefix (GitHub, GitLab, `sk-` keys, Slack, AWS, private-key blocks), and the model's
+chat-template tags. Such text is only ever an attack or a leaked secret, and identity goes into
+every prompt. The log names the rule, never the text.
+
+**Confirm.** The rule in the log line. `bxctl sanitize check` with the text (as an argument or on
+standard input) names each rule that fires and writes nothing. The session's log holds the call's
+arguments.
+
+**Fix.** A real secret: rotate it, since it has been in the conversation; keep it out of memory.
+Ordinary text refused by mistake: add it to the code-work corpus and narrow the rule, as the M5b
+spec's section 6.5 says; the change reaches the agent when it is rebuilt.
+
+**Check.** `bxctl sanitize check` with the text prints nothing and exits 0, and the agent's next
+write of it is answered `… updated` or `saved in notes.md`.
+
+## memory-write-flagged
+
+**What you see.** A memory write was kept one tier lower. The `memory` call was answered `not
+added to memory.md: it matches the rule <rule>, so it was saved in notes.md as an untrusted note`
+(or `user.md`), or, for a note, `saved in notes.md, marked untrusted: it matches the rule <rule>`.
+`memoryd` logged `memoryd: a memory write from session <id> matches the rule <rule>; it was saved
+in notes.md as an untrusted note (edit <edit id>); see docs/runbook.md#memory-write-flagged`, and
+`bxctl memory log` shows that edit with `flagged <rule>`.
+
+**Why.** The text is shaped like an attack (M5b spec, section 6.3): it tells the reader to ignore
+or replace earlier instructions, to send data to a URL, to dump the conversation, or to write into
+SSH configuration, or it holds one of M5a's six instruction phrases. A flag never refuses and
+never loses text: `memory.md` and `user.md`, which are in every prompt, are not changed, and every
+text of the write is saved at the end of `notes.md` under an `untrusted` header. `recall` returns
+it as `memory-tainted`, so a session that recalls it becomes untrusted. With review on, a flagged
+write is not staged: it is a note at once. The log names the rule, never the text.
+
+**Confirm.** `bxctl memory log`: the edit, `flagged <rule>`. The text is at the end of
+`<home>/memory/notes.md`, under a line `§ noted <time> in session <id> · <class> · untrusted`.
+`bxctl sanitize check` with the text names the rule.
+
+**Fix.** If it is an attack, leave it: the note is the record of what was tried, and it never
+reaches the prompt. If it is a true fact wrongly flagged, move it into identity by hand: add it to
+`<home>/memory/memory.md` (or `user.md`) with an editor, as an entry of its own, with a line
+holding only `§` between it and the entry before, keeping the file within its limit. The note
+stays in `notes.md`, which is append-only. If the rule catches ordinary text, narrow it as the M5b
+spec's section 6.5 says.
+
+**Check.** Start a new conversation and ask the agent what `memory.md` holds; the entry is there.
+
+## memory-entry-excluded
+
+**What you see.** `memoryd` logged `memoryd: an entry of memory.md was left out of the block by
+the rule <rule>; see docs/runbook.md#memory-entry-excluded` (or `user.md`), each time a session
+starts. Sessions start as usual, but the agent does not know that entry.
+
+**Why.** When a session starts, every entry of `memory.md` and `user.md` is checked again against
+the sanitizer's refuse rules (M5b spec, section 7), and an entry that breaks one is left out of
+the prompt. So a rule added later, or an entry an editor put in by hand, protects at once, with no
+migration. The file itself is not changed. Flag rules are not checked here, so an entry you moved
+from `notes.md` by hand stays in, and `soul.md` is the owner's and is not checked at all.
+
+**Confirm.** The rule in the log line names the kind of text (`secret-github`, `invisible`, …).
+Run each entry of the file through `bxctl sanitize check`: the one it reports is the one left out.
+
+**Fix.** Edit the file with an editor: remove the entry, or the part of it that breaks the rule.
+Entries are separated by a line holding only `§`. A secret does not belong in memory: rotate it
+if it was real.
+
+**Check.** Start a new conversation: `memoryd` does not log the line again.