Commit Graph
1 Commits
Author SHA1 Message Date
kyleandClaude Opus 5.5 d4b7f42b56 M5b plan: task 08, the self-test checks the tags against the tokenizer
loopd's self-test gains a check, run second (after "server matches
config", before any prompt): each tag in bxcore::sanitize::TAGS must be
one token from /tokenize, its recorded id, and its neutralised form
more than one. A failure names the tag by its id, never its text, and
points at a new runbook entry, loopd-tags-unexpected, given as a
fragment; SelfTestError::pointer picks the entry per failure. One
request per text (66): /tokenize has no batch form. The fake server
gains respond and a fake tokenizer; selftest.rs, serve.rs, pointers.rs
and support/mod.rs get M5b copies.

Checks, on a reference at ref 08 (titan, rustc 1.98.1, own target/):
- /tokenize fields from llama.cpp tools/server/README.md at b10964
  (b29c606) and its handler; on straylight all 33 tags gave their ids
  and every neutralised form 7 to 11 tokens, 66 requests in ~0.3 s
- given tests before the change: 4 fail as the task lists, selftest_tags
  does not compile (7 E0599); pointers passes
- make gate: gate: ok, 1050 tests (1042 + 8); /tmp and $TMPDIR
  unchanged around it; the four suites 25 runs under load, no failure
- 20 mutations, each caught: id not compared; neutralised check
  dropped; > 1 as >= 1, as == 1 missing 0, as > 2; first or last tag
  skipped; tag asked twice; pointer dropped or wrong (all or Tokenize);
  check skipped in serve; check run last; parse_special false;
  add_special left out; id missing from the message; the tag's text in
  the message; tokenize error as Infer; wrong count; wrong other id
- the task's edits applied to a fresh ref-07 copy give files
  byte-identical to the reference; every before block found once
- no chat-template tag whole in the task or given files; task lines
  within 100 characters except tables and verbatim lines; given files
  under 500 lines

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-26 06:14:58 -07:00