M5b plan: task 07, loopd neutralises text when it is first logged
loopd neutralises the user's message (the User record), every text appended as a tool result (results, recall's, denials, failures and loopd's own answers share the one append in run_turn), after the cap, and the memory block in Baseline::assemble_with. The model's output, system.md and replay are untouched: an old log with a raw tag is replayed byte for byte. Checks, on a reference at ref 07 (titan, rustc 1.98.1, own target/): - given tests before the change: 7 of 9 fail as the task lists; the two that guard what must not change pass before and after - make gate: gate: ok, 1042 tests (1033 + 9); /tmp unchanged around it; the two new suites 20 runs, no failure - 14 mutations, each caught: user not neutralised; neutralised before the cap; recall path missed; only untrusted results; truncated from the final length; block not neutralised; model content, reasoning or tool arguments neutralised; replay re-neutralises user, tool or system; open rewrites records; system.md neutralised too - the task's edits applied to a fresh ref-06 copy give files byte-identical to the reference; every before block found once - no chat-template tag whole in the task or given files; task lines within 100 characters except tables; given files under 500 lines Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,85 @@
|
||||
//! The model's chat-template tags for `loopd`'s neutralising tests, built from their names so that
|
||||
//! no test file holds one whole (tip T33), a naive oracle for a neutralised text, and a streamed
|
||||
//! completion built from deltas. Included with `#[path]`. No assertion prints a tag: a failing
|
||||
//! case is named, never shown. Do not edit.
|
||||
|
||||
#![allow(dead_code)] // each test file uses a different part of this module
|
||||
|
||||
use crate::support::Reply;
|
||||
|
||||
/// An angle-bar tag: a less-than sign and a vertical bar, the name, a vertical bar and a
|
||||
/// greater-than sign.
|
||||
pub fn bar(name: &str) -> String {
|
||||
["<", "|", name, "|", ">"].concat()
|
||||
}
|
||||
|
||||
/// An opening angle-bracket tag: a less-than sign, the name, a greater-than sign.
|
||||
pub fn open(name: &str) -> String {
|
||||
["<", name, ">"].concat()
|
||||
}
|
||||
|
||||
/// A closing angle-bracket tag: a less-than sign and a slash, the name, a greater-than sign.
|
||||
pub fn close(name: &str) -> String {
|
||||
["<", "/", name, ">"].concat()
|
||||
}
|
||||
|
||||
/// `text` with every angle bracket written as its HTML entity. Every angle bracket in these
|
||||
/// tests' texts is part of a listed tag, so this is what neutralising them gives.
|
||||
pub fn escaped(text: &str) -> String {
|
||||
text.replace('<', "<").replace('>', ">")
|
||||
}
|
||||
|
||||
/// Asserts `got == want` without printing either, since they may hold a tag; `what` names the
|
||||
/// case.
|
||||
#[track_caller]
|
||||
pub fn same<T: PartialEq + ?Sized>(got: &T, want: &T, what: &str) {
|
||||
assert!(
|
||||
got == want,
|
||||
"{what}: not as expected (not shown: it may hold a tag)"
|
||||
);
|
||||
}
|
||||
|
||||
/// The string at `value`, or "" when it is not a string.
|
||||
pub fn text(value: &serde_json::Value) -> &str {
|
||||
value.as_str().unwrap_or_default()
|
||||
}
|
||||
|
||||
/// True if `text` holds any of the listed tags whole.
|
||||
pub fn holds_a_tag(text: &str) -> bool {
|
||||
bxcore::sanitize::TAGS.iter().any(|t| text.contains(t.text))
|
||||
}
|
||||
|
||||
/// A streamed completion as the server sends one: a chunk per delta, then a chunk with the finish
|
||||
/// reason and the timings, then the end marker. The body runs to the close.
|
||||
pub fn stream(deltas: &[serde_json::Value], finish: &str) -> Reply {
|
||||
let id = "chatcmpl-neutralise";
|
||||
let mut body = String::new();
|
||||
for delta in deltas {
|
||||
let chunk = serde_json::json!({
|
||||
"id": id,
|
||||
"choices": [{ "index": 0, "finish_reason": null, "delta": delta }]
|
||||
});
|
||||
body.push_str(&format!("data: {chunk}\n\n"));
|
||||
}
|
||||
let last = serde_json::json!({
|
||||
"id": id,
|
||||
"choices": [{ "index": 0, "finish_reason": finish, "delta": {} }],
|
||||
"timings": { "cache_n": 0, "prompt_n": 40, "predicted_n": 8 }
|
||||
});
|
||||
body.push_str(&format!("data: {last}\n\ndata: [DONE]\n\n"));
|
||||
Reply::raw(format!(
|
||||
"HTTP/1.1 200 OK\r\nContent-Type: text/event-stream\r\nConnection: close\r\n\r\n{body}"
|
||||
))
|
||||
}
|
||||
|
||||
/// A delta that calls the tool `name` with `arguments` (JSON text).
|
||||
pub fn call(id: &str, name: &str, arguments: &str) -> serde_json::Value {
|
||||
serde_json::json!({
|
||||
"tool_calls": [{
|
||||
"index": 0,
|
||||
"id": id,
|
||||
"type": "function",
|
||||
"function": { "name": name, "arguments": arguments }
|
||||
}]
|
||||
})
|
||||
}
|
||||
Reference in New Issue
Block a user