Research: chat-template tags in text the model reads; tips I15 and T33; an open decision

M5a task 04 stopped twice on the chat-template tags the check refuses.
The first time, the model read them in a file and took them for its own
template. The second time, it spelled them out in its own reasoning.

Measured on straylight: the think and tool-call tags are one token
whatever the tokenizer's flag. Inside any chat message, the turn
markers become control tokens, so a tool result can forge a system
turn.

Adds the rule to AGENTS.md, and proposes neutralizing the tags in
loopd as an open decision.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

docs/implementer-lessons.md

@@ -30,6 +30,7 @@ How it is used:
 | I12 | A file that exists but cannot be read is an error. Only a file that does not exist may be treated as absent. | M2b finding 3. | yes | |
 | I13 | Before calling a failure "environmental", measure the thing you blame, and name the machine, OS and file system the evidence came from. A sample that shows a thread inside a system call shows where the time goes, not that the call is stuck. | M3a stop before task 20: an fsync "stall" was blamed from `__fcntl` frames that were ordinary 4 ms `F_FULLFSYNC` calls on the Mac, and the real cause, a macOS socket rule, went unfixed. | no | |
 | I14 | Recovering from a crash is a case to write a test for, not only to reason about. A file the daemon itself can leave behind (created, not yet written) must load like an empty one. | M3a finding 1: one zero-length log file made `brokerd` panic at startup, where the same directory verified as `ok, 0 records`. |
+| I15 | Never write the model's chat-template tags out whole (the think and tool-call tags, the angle-bar turn markers) in code, comments, commands or the log. Build them from pieces (`concat!("<", "think")`) and name them in words. Text holding them turns into the model's own control tokens when it reads it. | M5a task 04: the given test held the tags; Ornith's reasoning ended at the test file's closing think tag, and it lost track of the file and made no change. `docs/research/2026-09-24-chat-template-tokens.md`. | yes | |
 
 ## Tips for writing tasks
 
@@ -67,6 +68,7 @@ How it is used:
 | T30 | Every behaviour a task writes must be reachable by that task's own tests. When a function's effect shows only after a later task (here: a refusal's reason, which only task 15 could send), give a unit test in the first task, or write that function in the later one. And never let one name mean two things in a comment: say which `reason`. | M4b task 14 dropped a refusal's reason, reading `Refused -> Ended::Refused { by, reason }` as the `DenyReason` in scope; its tests passed, and task 15's caught it, in a file task 15 could not touch. |
 | T31 | In a spec, for every post or message the harness sends, name who can read it and whether that reader can act on it. Walk each kind of sender through each kind of channel (the owner, an allowed non-approver, a stranger; direct, group, channel). | M4b: an approval asked by a non-approver in a direct message went into that direct message, where no approver could see it. Every test passed, because the tests used the spec's cases; the device check with a second account found it (review finding 1, fixed by tasks 17 and 18). |
 | T32 | A given test must not assert the order of two things that run on their own threads. Check the order where it is decided (here, the sort in the catch-up parser), and compare what the threads deliver as a set. Before hand-over, stress each test with concurrency under load (`yes > /dev/null` beside a loop of 60 runs). | M4a's `serve_restart` expected two caught-up sessions' turns to reach `loopd` in order. They run on separate threads, and under load 1 run in 60 failed. M4b task 18's session hit it, proved it happens without its change, and reported it instead of editing the test. |
+| T33 | A file the implementer reads must never hold the model's chat-template tags literally: not in skeletons, given tests, fixtures, task text or specs it may open. Build them from pieces and name them in words. grep the plan for the tags before hand-over. | M5a task 04 (above); the measurements in `docs/research/2026-09-24-chat-template-tokens.md`: the think and tool-call tags are one token whatever the tokenizer's flag, and inside a chat message the turn markers become control tokens. |
 
 ## What worked and should be kept