M5a plan, task 04: chat-template tags built from pieces

Task 04's session read the given test and the check.rs skeleton, both
holding the tags the check refuses, and Ornith took them for its own
template. It lost track of the file and made no change. The tags are
now built with concat! and format! in check.rs and in the four given
tests that held them, and the spec describes them in words. The
rewritten tests pass against the reference, are rustfmt-clean, and
still catch the case-folding mutation.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-09-25 16:41:44 -07:00
committed by kyle
co-authored by Claude Opus 5.5
parent b84b9af540
commit 8328e8f607
8 changed files with 45 additions and 15 deletions
@@ -49,7 +49,16 @@ const DEFAULT_IGNORABLE: &[(u32, u32)] = &[
/// separators.
const ALSO_INVISIBLE: &[u32] = &[0x2800, 0x2028, 0x2029];
const MARKUP: &[&str] = &["<|", "|>", "<think", "</think", "<tool_call", "</tool_call"];
// Built with `concat!` so this file never holds the markup itself: a model reading it could take
// the markup for its own.
const MARKUP: &[&str] = &[
concat!("<", "|"),
concat!("|", ">"),
concat!("<", "think"),
concat!("<", "/think"),
concat!("<", "tool_call"),
concat!("<", "/tool_call"),
];
const PHRASES: &[&str] = &[
"ignore previous instructions",