Research: chat-template tags in text the model reads; tips I15 and T33; an open decision

M5a task 04 stopped twice on the chat-template tags the check refuses.
The first time, the model read them in a file and took them for its own
template. The second time, it spelled them out in its own reasoning.

Measured on straylight: the think and tool-call tags are one token
whatever the tokenizer's flag. Inside any chat message, the turn
markers become control tokens, so a tool result can forge a system
turn.

Adds the rule to AGENTS.md, and proposes neutralizing the tags in
loopd as an open decision.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-09-25 16:41:44 -07:00
committed by kyle
co-authored by Claude Opus 5.5
parent 6f093fcd8e
commit fb0aa2e576
4 changed files with 141 additions and 0 deletions
+1
View File
@@ -68,6 +68,7 @@ Newest first. A decision that changes `docs/design.md` lands in the same commit
| How an approval whose arguments do not fit one Mattermost post is shown. Settled on 2026-09-18: what is being approved is never truncated, in any channel, because the unseen tail is where injected text would put its payload; `bxctl` prints arguments in full. Leading option for Mattermost: a thread of numbered chunks split on character boundaries of the escaped text, where only a reply or reaction on the last post counts and that post repeats the id, the chunk count and the hash of the whole; above some number of chunks, the post points to `bxctl approvals` and approval is local only. The post size limit and rate limits must be checked against the Mattermost API reference first. | M4 |
| Secret store: settled 2026-09-23 (systemd credentials, environment, owner-only file; P15 proposed). Where `brokerd` reads tool secrets from, and whether `SecretStore` moves out of `gatewayd` into a shared place, is decided when a tool first needs a secret. | when a tool needs a secret |
| Whether cloud-led sessions are ever allowed, and for which data classes. | M6 |
| Neutralizing chat-template tags in everything `loopd` sends (the owner's and other agents' messages, tool results, recalled text): measured 2026-09-24, a tag inside any message becomes Ornith's control token, so a fetched page can end its tool result and forge a system turn (`docs/research/2026-09-24-chat-template-tokens.md`). Proposed: replace each tag in a fixed list with an escaped look-alike when the text is first appended and logged (replay stays byte-identical), the list checked against the tokenizer by the self-test. Memory writes already refuse the tags (M5a). | before M5b; the exposure exists since M3 |
| Session-scoped and standing approvals for tools other than memory: answering an approval with `!approve 42 session` (the same pattern for the rest of the session) or `!approve 42 always` (standing), words after the id as `!deny 42 <reason>` has (owner's syntax, 2026-09-24). To settle: what "session" covers (this call's tool and path or host, or the whole grant it matched); and where "always" writes, since `brokerd` cannot write grants (a proposed grant for the owner, or a separate directory of chat-approved grants). A temporary grant is new authority (the grants are the owner's files, and `brokerd` cannot write them), so it needs its own design. Owner's point, 2026-09-24: a harness should not ask a stream of questions. | after M5 |
| Projects in memory. Honcho can separate projects by a workspace per project (hard walls, shared naturally by every agent on the project, but Honcho's picture of the owner splits per workspace) or by scopes inside one workspace (one picture, recall filtered by project; semantics not yet read). Either way a project is a `memoryd` setting: its channels and threads, its workspace or scope, `write` and `trust`. A shared project workspace is written by several agents, so its trust is the owner's call per project. Test both on rift before choosing. Owner's idea, 2026-09-24. | M5b or a later M5d |