Research: prompt-injection defences for persisted memory; the M5a check refuses all default-ignorable code points

The research feeds M5b's sanitizer. One finding is a bug in the M5a
draft: category Cf misses variation selectors, the combining grapheme
joiner, and the Hangul and Braille fillers.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

docs/research/2026-09-24-prompt-injection-memory.md

@@ -0,0 +1,168 @@
+# Prompt-injection defences for persisted memory
+
+Research of 2026-09-24, for the sanitizer module planned as M5b hardening (`docs/decisions.md`,
+row of that date). Sources were read that day; hermes-agent was read at commit e8acdc5 and tirith
+at c18753a. Every claim below has its source. What could not be checked is listed at the end.
+
+## 1. Attacks on persisted memory
+
+- **Stored instructions.** Injected text gets the agent to save a standing instruction.
+  - SpAIware wrote one into ChatGPT's memory. It leaked every later chat through rendered markdown
+    images ([embracethered, 2024](https://embracethered.com/blog/posts/2024/chatgpt-macos-app-persistent-data-exfiltration/)).
+  - The Gemini attack delays the write: a hidden instruction waits until the user says "yes" or
+    "sure" ([embracethered, 2025](https://embracethered.com/blog/posts/2025/gemini-memory-persistence-prompt-injection/)).
+  - OWASP names the class ASI06, Memory & Context Poisoning
+    ([OWASP agentic top 10](https://genai.owasp.org/2025/12/09/owasp-top-10-for-agentic-applications-the-benchmark-for-agentic-security-in-the-age-of-autonomous-ai/)).
+- **Injection through queries alone.** MINJA reached 98.2% injection success by chatting with the
+  agent and nothing else ([arXiv 2503.03704](https://arxiv.org/abs/2503.03704)). The rate is
+  lower when the agent already holds realistic amounts of legitimate memory
+  ([arXiv 2601.05504](https://arxiv.org/abs/2601.05504)).
+- **Retrieval backdoors.** AgentPoison: over 80% success at under 0.1% poisoning
+  ([arXiv 2407.12784](https://arxiv.org/abs/2407.12784)). This matters less for identity files,
+  which are always in the prompt rather than retrieved. It matters for `recall`.
+- **Poisoned procedures and reasoning.**
+  - MemoryGraft plants fake "successful experiences"
+    ([arXiv 2512.16962](https://arxiv.org/abs/2512.16962)).
+  - FARMA forges reasoning in wording that passes keyword checks, with up to 100% success
+    ([arXiv 2607.05029](https://arxiv.org/abs/2607.05029)).
+- **Plain false facts.** A write-time screen that caught 83.2% of injections rejected none of 360
+  poisoned facts ([arXiv 2608.21230](https://arxiv.org/abs/2608.21230)). Existing injection
+  defences do not cover memory poisoning (MPBench,
+  [arXiv 2606.04329](https://arxiv.org/abs/2606.04329)).
+- **Invisible payloads.**
+  - Unicode tag characters (the U+E0000 block) carry ASCII text that models read
+    ([embracethered](https://embracethered.com/blog/posts/2024/hiding-and-finding-text-with-unicode-tags/)).
+  - Variation selectors can hide bytes behind an emoji
+    ([FireTail](https://www.firetail.ai/blog/peek-a-boo-emoji-smuggling-and-modern-llms)).
+
+Text in the system prompt has the highest privilege under an instruction hierarchy
+([arXiv 2404.13208](https://arxiv.org/abs/2404.13208)), and memory placed there inherits it. That
+is why identity files must be written only by clean sessions.
+
+## 2. Checks on the text itself
+
+- **M5a's category `Cf` rule has gaps.** Checked with Unicode 16 data, none of these are `Cf`:
+  - variation selectors U+FE00–FE0F and U+E0100–E01EF (`Mn`);
+  - the combining grapheme joiner U+034F (`Mn`);
+  - Hangul fillers U+3164 and U+115F (`Lo`);
+  - Braille blank U+2800 (`So`);
+  - U+2028 (`Zl`).
+
+  The fix is to refuse the `Default_Ignorable_Code_Point` property, plus an explicit list: those
+  fillers, `Zl`/`Zp`, private use, noncharacters and unassigned code points.
+- **Character tricks defeat classifiers.** They evaded six classifier guardrails, including
+  Prompt Guard and Azure Prompt Shield, up to 100% of the time
+  ([arXiv 2504.11168](https://arxiv.org/abs/2504.11168)). Refusing those characters before any
+  other check closes that route.
+- **Scan several normalised forms.** tirith scans the raw text and forms derived from it:
+  - invisible characters stripped;
+  - NFKC;
+  - a confusable skeleton (UTS #39);
+  - spaced-out letters collapsed;
+  - a bounded leetspeak fold;
+  - short base64 and hex decoded.
+
+  Source: tirith, `crates/tirith-core/src/deobfuscate.rs`. tirith is AGPL-3.0 or commercial
+  ([repository](https://github.com/sheeki03/tirith)): reimplement the ideas, do not copy the code.
+  NFKC does not fold Cyrillic look-alikes. The skeleton does; it comes from the `unicode-security`
+  crate, MIT/Apache-2.0 ([docs.rs](https://docs.rs/unicode-security/latest/unicode_security/)).
+- **The most reusable pattern set is hermes-agent's `tools/threat_patterns.py` (MIT).** It is
+  applied to NFKC text in three scopes. Its "strict" scope, used for memory, adds:
+  - SSH and agent-config persistence;
+  - sending data to URLs;
+  - dumping the conversation;
+  - hardcoded secrets.
+
+  Its rule is to match attack behaviour, not bossy English, to keep false positives down
+  ([source](https://github.com/NousResearch/hermes-agent/blob/main/tools/threat_patterns.py)).
+- **Phrase lists miss paraphrase (FARMA) and false facts (2608.21230).** They are a tripwire, not
+  a boundary.
+
+## 3. Classifier models
+
+| Detector | Size, licence | Measured | Weakness |
+|---|---|---|---|
+| [Llama Prompt Guard 2](https://huggingface.co/meta-llama/Llama-Prompt-Guard-2-86M) | 22M/86M, Llama 4 Community, 512-token window | 97.5% (86M) and 88.7% (22M) recall at 1% false positives, Meta's own evaluation | adaptive attacks succeed over 90% ([arXiv 2510.09023](https://arxiv.org/html/2510.09023)) |
+| [ProtectAI deberta-v3-base v2](https://huggingface.co/protectai/deberta-v3-base-prompt-injection-v2) | 0.2B, Apache-2.0, archived | 95.25% accuracy | English only; false positives on system-prompt-like text; adaptive over 90% |
+| [PIGuard](https://github.com/leolee99/PIGuard) ([arXiv 2410.22770](https://arxiv.org/abs/2410.22770)) | MIT | 30.8% better than prior guards on NotInject (benign text with trigger words) | adaptive 71% |
+| [DataSentinel](https://arxiv.org/abs/2504.11358) | Mistral-7B fine-tune | false positives near 0 | adaptive about 80% |
+| Known-answer check with a general model | any | high error both ways | false negatives up to 0.93 under adaptive attack |
+
+Memory entries are often imperative preferences ("always reply in UTC"): exactly the benign text
+that trips classifiers. Nothing measures a 9B general model, such as ornith-1.5-9b, as a judge.
+
+## 4. Design-level defences
+
+- **Taint and information-flow control have the strongest evidence.**
+  - FIDES tracks integrity labels dynamically ([arXiv 2505.23643](https://arxiv.org/abs/2505.23643)).
+  - CaMeL solves 77% of AgentDojo tasks with provable security, against 84% undefended
+    ([arXiv 2503.18813](https://arxiv.org/abs/2503.18813)).
+  - A catalogue of design patterns is in [arXiv 2506.08837](https://arxiv.org/abs/2506.08837).
+
+  Boxmaker's session taint is a coarse form of these.
+- **Provenance and lineage.**
+  - MemLineage blocks actions whose justification traces to external sources, reaching 0% attack
+    success on its workloads ([arXiv 2605.14421](https://arxiv.org/abs/2605.14421)).
+  - HMAC-signed writes stop unsigned injection ([arXiv 2606.12703](https://arxiv.org/abs/2606.12703)).
+  - Ranking recall by provenance alone did not help (p=0.80; 2608.21230).
+- **Spotlighting (marking where text came from).** Attack success fell from over 50% to under 2%
+  ([arXiv 2403.14720](https://arxiv.org/abs/2403.14720)), but adaptive attacks exceed 95%. It is
+  cheap framing, not a boundary.
+- **Models trained against injection.** StruQ, SecAlign and Meta SecAlign
+  ([arXiv 2507.02735](https://arxiv.org/abs/2507.02735)) change the model itself, which is outside
+  the harness's control. Meta SecAlign still showed 96% adaptive success.
+- **Consensus voting.** A-MemGuard ([arXiv 2510.02373](https://arxiv.org/abs/2510.02373)) is
+  defeated by FARMA.
+
+## 5. Recommendations for the sanitizer, ranked
+
+1. **Taint as the main control.**
+   - It must stay sticky for the whole session, including across compaction (M5c). A write
+     triggered later in a session that read a document (the Gemini pattern) must still become an
+     untrusted note.
+   - The journal keeps each entry's provenance.
+
+   Boxmaker's taint is already sticky within a session. M5c must carry it across epochs.
+2. **Refuse invisible and unusual code points.** The set is:
+   - `Default_Ignorable_Code_Point`;
+   - control characters other than `\n` and `\t`;
+   - `Zl` and `Zp`, private use, unassigned code points, noncharacters;
+   - all variation selectors;
+   - the Hangul and Braille fillers.
+
+   Refuse rather than strip, so the owner and the model see the same bytes. The cost is emoji
+   joined sequences and flags, which is acceptable for memory.
+3. **Scan normalised forms** (raw text, NFKC with casefold, the UTS #39 skeleton, whitespace
+   collapsed, leet folded) against the patterns. Refuse single words that mix scripts. Store the
+   original text in NFC.
+4. **A wider markup check.**
+   - Take the special tokens from Ornith's own tokenizer at startup, rather than a hand list.
+   - Add the generic forms: `<|…|>`, `[INST]`, `<<SYS>>`, `<start_of_turn>`, and lines starting
+     `system:`, `assistant:` or `user:`.
+   - Also check the skeleton form.
+5. **Adopt hermes-agent's strict patterns** (MIT, with attribution), keeping its rule of matching
+   attack behaviour, not bossy English.
+6. **Limits on structure.**
+   - Refuse markdown images and links, raw HTML, code fences and headings. SpAIware leaked
+     through a markdown image, and Mattermost renders markdown.
+   - Refuse base64 or hex runs of 32 characters or more, and values shaped like secrets.
+7. **Show the owner what was written.** A periodic digest or diff is the only defence in the
+   evidence against false facts and paraphrase. This is at odds with the decision that writes are
+   journalled and not announced; it is a question for the owner, not a build item.
+8. **A classifier only as an advisory flag, later.** The costs:
+   - an ONNX runtime to embed;
+   - Prompt Guard's licence;
+   - false positives on preferences;
+   - over 70% adaptive bypass.
+
+   Do not gate writes on a 9B model as a judge; nothing measures it.
+
+## Not verified
+
+- The full text of OWASP ASI06's mitigations.
+- Per-model NotInject numbers.
+- PIGuard's size.
+- Whether hermes-agent scans memory again when loading it.
+- Ornith's special-token list.
+- The per-defence adaptive figures come from a summary of arXiv 2510.09023's HTML page, not its
+  tables.

docs/specs/2026-09-24-m5a-local-memory.md

@@ -211,8 +211,12 @@ the file fails safely: every `old` must match an entry of the file as it is now,
 if it contains:
 
 1. a control character other than newline and tab (C0, DEL, C1);
-2. an invisible or direction-changing character (Unicode general category `Cf`: zero-width
-   spaces and joiners, bidirectional controls, and the like);
+2. an invisible or direction-changing character: any code point with the Unicode property
+   `Default_Ignorable_Code_Point` (zero-width spaces and joiners, bidirectional controls,
+   variation selectors, tag characters U+E0000 to U+E007F, and the like), and U+034F, U+115F,
+   U+3164, U+2800, U+2028 and U+2029, which are invisible without being in that property or in
+   category `Cf` (`docs/research/2026-09-24-prompt-injection-memory.md`, section 2). The table
+   is fixed in the code, from Unicode 16 data;
 3. chat-template markup: `<|` or `|>`, or `<think`, `</think`, `<tool_call`, `</tool_call`;
 4. one of these phrases, ignoring case and runs of spaces: `ignore previous instructions`,
    `ignore all previous`, `disregard previous instructions`, `disregard all previous`,