M5b plan: task 07, loopd neutralises text when it is first logged
loopd neutralises the user's message (the User record), every text appended as a tool result (results, recall's, denials, failures and loopd's own answers share the one append in run_turn), after the cap, and the memory block in Baseline::assemble_with. The model's output, system.md and replay are untouched: an old log with a raw tag is replayed byte for byte. Checks, on a reference at ref 07 (titan, rustc 1.98.1, own target/): - given tests before the change: 7 of 9 fail as the task lists; the two that guard what must not change pass before and after - make gate: gate: ok, 1042 tests (1033 + 9); /tmp unchanged around it; the two new suites 20 runs, no failure - 14 mutations, each caught: user not neutralised; neutralised before the cap; recall path missed; only untrusted results; truncated from the final length; block not neutralised; model content, reasoning or tool arguments neutralised; replay re-neutralises user, tool or system; open rewrites records; system.md neutralised too - the task's edits applied to a fresh ref-06 copy give files byte-identical to the reference; every before block found once - no chat-template tag whole in the task or given files; task lines within 100 characters except tables; given files under 500 lines Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5.5
@@ -0,0 +1,273 @@
|
||||
# M5b task 07: `loopd` neutralises text from outside when it is first logged
|
||||
|
||||
**Branch:** `m5b` (run `git switch m5b`; `git status --short` must be empty, otherwise stop)
|
||||
**Commit subject:** `loopd: neutralise user messages, tool results and the memory block`
|
||||
|
||||
## Read this first: the attack texts
|
||||
|
||||
The given tests put your model's chat-template tags (the markup strings) into user messages,
|
||||
tool results and the memory block on purpose, and one forges a system turn inside a fetched page.
|
||||
Your model uses these same strings to mark its reasoning and its tool calls. **If you spell one
|
||||
out, even in your thinking, it ends your reasoning or your turn.** So:
|
||||
|
||||
- **Do not open** the three given test files (`neutralise.rs`, `neutralise_replay.rs`,
|
||||
`support/tags.rs`), and do not work through their cases in your reasoning. You copy them; you
|
||||
never need to read them. Their failure messages name a case and never show a text.
|
||||
- Nothing you write needs a tag. The code calls `neutralise`, which task 03 wrote; it never looks
|
||||
at what a tag says. Do not write a tag anywhere, as tasks 03 to 06 said.
|
||||
|
||||
## Goal
|
||||
|
||||
A tag in a fetched page, a user message or the memory block becomes the model's own token when
|
||||
it reaches the model, so the text can end a tool result and forge a system turn. Task 03 wrote
|
||||
`bxcore::sanitize::neutralise`, which writes each listed tag's angle brackets as `<` and
|
||||
`>`. This task calls it in `loopd` (M5b spec, section 8), **once, where the text is first
|
||||
appended and logged**, so the logged bytes are what every later request replays and the
|
||||
inference contract holds (each request extends the one before, byte for byte):
|
||||
|
||||
- **the user's message** (the owner's, or another speaker's): the `User` record, in `run_turn`;
|
||||
- **every text appended as a tool result**, after the cap: a tool's result (`recall`'s among
|
||||
them), a denial, a failure, and `loopd`'s own answers (`find_tool`, `call_tool`, `clock`, the
|
||||
repeated-call sentence). They all reach the model as tool messages through the one
|
||||
`LogRecord::ToolResult` append in `run_turn`, so one call there covers them all. A denial's
|
||||
detail echoes the model's arguments and a failure's message comes from the broker or the tool:
|
||||
in a tool message a tag is a control token wherever it came from. A fixed sentence holds no
|
||||
tag, and `neutralise` gives it back unchanged;
|
||||
- **the memory block**, in `Baseline::assemble_with`. `soul.md` is part of the block, and nothing
|
||||
checks it on write. The baseline is written to `0.baseline.json` when the session is created,
|
||||
and every request of the session sends it from there.
|
||||
|
||||
What is **not** touched:
|
||||
|
||||
- **The model's own output** (`content`, `reasoning_content`, `tool_calls`): logged and replayed
|
||||
exactly as the server returned it (inference contract).
|
||||
- **Replay.** `baseline::messages` and `Session::open` send the logged bytes as they are. A log
|
||||
written before this task can hold a raw tag; it is replayed unchanged, never rewritten and never
|
||||
neutralised again, because changing an earlier byte of the prompt forces the server to read the
|
||||
whole prompt again.
|
||||
- `system.md`, the owner's system prompt around the block.
|
||||
|
||||
**The cap comes first** (M5b plan, review focus 2). `cap_result` cuts a result to the cap and
|
||||
sets `truncated`; the cut text is then neutralised. `<` is longer than `<`, so a logged result
|
||||
can be a few bytes over the cap, and `truncated` says only whether the cap cut it. A tag the cut
|
||||
splits is no longer a tag, and is left as it is.
|
||||
|
||||
| Name | What it is |
|
||||
|---|---|
|
||||
| `bxcore::sanitize::neutralise` | `pub fn neutralise(text: &str) -> std::borrow::Cow<'_, str>` (task 03): `Borrowed` when nothing matches |
|
||||
| `loopd::tools::cap_result` | `pub fn cap_result(text: &str, cap: usize) -> (String, bool)`: unchanged |
|
||||
| `loopd::turn::run_turn` | two appends change (T3, T4) |
|
||||
| `loopd::baseline::Baseline::assemble_with` | the block is neutralised (B3) |
|
||||
|
||||
`loopd` already depends on `bxcore`; `sanitize` needs no feature. No dependency changes, and
|
||||
`loopd` never enables `bxcore`'s feature `secrets` (the test `no_secrets.rs` checks it).
|
||||
|
||||
## Every place `loopd` logs text
|
||||
|
||||
This is the complete list. Each one is covered by an edit below, or stays as it is.
|
||||
|
||||
| Place | What is logged | What happens |
|
||||
|---|---|---|
|
||||
| `crates/loopd/src/turn.rs`, `run_turn`, `LogRecord::User` | the user's message | neutralised (T3) |
|
||||
| `crates/loopd/src/turn.rs`, `run_turn`, `LogRecord::Assistant` | the model's output | unchanged |
|
||||
| `crates/loopd/src/turn.rs`, `run_turn`, `LogRecord::Usage` and `CacheLoss` | numbers | unchanged |
|
||||
| `crates/loopd/src/turn.rs`, `run_turn`, `LogRecord::ToolResult` | every tool message | capped, then neutralised (T4) |
|
||||
| `crates/loopd/src/baseline.rs`, `assemble_with` | the memory block, into the baseline | neutralised (B3) |
|
||||
| `crates/loopd/src/session.rs`, `create` | `SessionStart` and the baseline file | unchanged: the baseline is already neutralised |
|
||||
| `crates/loopd/src/baseline.rs`, `messages`; `crates/loopd/src/session.rs`, `open` | replay | unchanged: never neutralise here |
|
||||
|
||||
## Files
|
||||
|
||||
- Copy, from `docs/plans/M5b/files/` (the same path under it): the new tests
|
||||
`crates/loopd/tests/neutralise.rs`, `crates/loopd/tests/neutralise_replay.rs` and
|
||||
`crates/loopd/tests/support/tags.rs` (included by the two tests with `#[path]`; do not add it
|
||||
to `support/mod.rs`).
|
||||
- Modify: `crates/loopd/src/turn.rs` (T1 to T4), `crates/loopd/src/baseline.rs` (B1 to B3),
|
||||
`docs/implementer-log.md`.
|
||||
- Do not touch: `crates/bxcore`, any `Cargo.toml`, any other file of `loopd`, any other crate.
|
||||
|
||||
## The edits, exactly
|
||||
|
||||
Every "before" block is in its file exactly once. Nothing else in these files changes.
|
||||
|
||||
### `crates/loopd/src/turn.rs`
|
||||
|
||||
**T1.** The module comment: the lines
|
||||
|
||||
```rust
|
||||
//! One turn: ask, run any tool calls under their limits, and return.
|
||||
//! The loop is bounded every way a runaway can go: a cap on tool iterations, a repeated-call
|
||||
//! detector, the two M2a limits (thinking tokens, a full context), and a tool-result cap.
|
||||
```
|
||||
|
||||
become
|
||||
|
||||
```rust
|
||||
//! One turn: ask, run any tool calls under their limits, and return.
|
||||
//! The loop is bounded every way a runaway can go: a cap on tool iterations, a repeated-call
|
||||
//! detector, the two M2a limits (thinking tokens, a full context), and a tool-result cap.
|
||||
//! Text from outside (the user's message, every tool result) is neutralised as it is first
|
||||
//! logged (M5b spec, section 8), so every later request replays the same bytes; the model's own
|
||||
//! output is logged as the server returned it.
|
||||
```
|
||||
|
||||
**T2.** The line
|
||||
|
||||
```rust
|
||||
use crate::tools::{Registry, ToolPort, cap_result, denial_message, dispatch};
|
||||
```
|
||||
|
||||
becomes
|
||||
|
||||
```rust
|
||||
use crate::tools::{Registry, ToolPort, cap_result, denial_message, dispatch};
|
||||
use bxcore::sanitize::neutralise;
|
||||
```
|
||||
|
||||
**T3.** In `run_turn`, the user's message: the lines
|
||||
|
||||
```rust
|
||||
time: Timestamp::now(),
|
||||
content: content.to_string(),
|
||||
```
|
||||
|
||||
become
|
||||
|
||||
```rust
|
||||
time: Timestamp::now(),
|
||||
content: neutralise(content).into_owned(),
|
||||
```
|
||||
|
||||
**T4.** In `run_turn`, the tool result: the line
|
||||
|
||||
```rust
|
||||
let (text, truncated) = cap_result(&text, rt.cfg.r#loop.tool_result_cap);
|
||||
```
|
||||
|
||||
becomes
|
||||
|
||||
```rust
|
||||
// Capped first, then neutralised: the logged text may be a few bytes over the cap,
|
||||
// and `truncated` says only whether the cap cut it. Denials and failures take this
|
||||
// path too, since they reach the model as tool messages as well.
|
||||
let (text, truncated) = cap_result(&text, rt.cfg.r#loop.tool_result_cap);
|
||||
let text = neutralise(&text).into_owned();
|
||||
```
|
||||
|
||||
The `LogRecord::ToolResult` below it takes `content: text` and `truncated` as today, and the
|
||||
`TurnEvent::ToolResult` after it sends the same `truncated`.
|
||||
|
||||
### `crates/loopd/src/baseline.rs`
|
||||
|
||||
**B1.** The line
|
||||
|
||||
```rust
|
||||
use crate::tools::Registry;
|
||||
```
|
||||
|
||||
becomes
|
||||
|
||||
```rust
|
||||
use crate::tools::Registry;
|
||||
use bxcore::sanitize::neutralise;
|
||||
```
|
||||
|
||||
**B2.** The comment of `assemble_with`: the lines
|
||||
|
||||
```rust
|
||||
/// The system prompt trimmed, then a blank line and the memory block `memoryd` rendered (M5a
|
||||
/// spec, section 4), when there is one; the core tool schemas.
|
||||
```
|
||||
|
||||
become
|
||||
|
||||
```rust
|
||||
/// The system prompt trimmed, then a blank line and the memory block `memoryd` rendered (M5a
|
||||
/// spec, section 4), when there is one; the core tool schemas. The block is neutralised here
|
||||
/// (M5b spec, section 8): `soul.md` is not checked on write, and the snapshot of the baseline
|
||||
/// is what every request of the session sends.
|
||||
```
|
||||
|
||||
**B3.** In `assemble_with`: the lines
|
||||
|
||||
```rust
|
||||
let memory = memory.trim();
|
||||
if !memory.is_empty() {
|
||||
system.push_str("\n\n");
|
||||
system.push_str(memory);
|
||||
}
|
||||
```
|
||||
|
||||
become
|
||||
|
||||
```rust
|
||||
let memory = neutralise(memory.trim());
|
||||
if !memory.is_empty() {
|
||||
system.push_str("\n\n");
|
||||
system.push_str(&memory);
|
||||
}
|
||||
```
|
||||
|
||||
`memory` is now a `Cow<str>`: `is_empty()` and `&memory` reach the `str` inside it.
|
||||
|
||||
## Steps
|
||||
|
||||
- [ ] **1. Copy the given tests.** From the repository root, three commands:
|
||||
|
||||
```sh
|
||||
cp docs/plans/M5b/files/crates/loopd/tests/neutralise.rs crates/loopd/tests/
|
||||
cp docs/plans/M5b/files/crates/loopd/tests/neutralise_replay.rs crates/loopd/tests/
|
||||
cp docs/plans/M5b/files/crates/loopd/tests/support/tags.rs crates/loopd/tests/support/
|
||||
```
|
||||
|
||||
- [ ] **2. See them fail.**
|
||||
`cargo test -p loopd --no-fail-fast --test neutralise --test neutralise_replay`: both compile.
|
||||
In `neutralise`, all six tests fail; in `neutralise_replay`, one fails and two pass:
|
||||
|
||||
| Test | Before this task |
|
||||
|---|---|
|
||||
| `a_user_message_is_neutralised_in_the_log_and_the_request` | fails: `the record in memory: not as expected …` |
|
||||
| `a_tool_result_is_neutralised_in_the_log_and_the_next_request` | fails: `the record, label kept: not as expected …` |
|
||||
| `a_recall_result_is_neutralised` | fails: `the record: not as expected …` |
|
||||
| `denied_and_failed_texts_are_neutralised_too` | fails: `denied: not as expected …` |
|
||||
| `the_cap_comes_first_then_neutralising` | fails: `case 0: not as expected …` |
|
||||
| `a_fetched_page_that_forges_a_system_turn_reaches_the_model_escaped` | fails: `the record: not as expected …` |
|
||||
| `the_memory_block_is_neutralised_when_the_baseline_is_assembled` | fails: `the assembled baseline: not as expected …` |
|
||||
| `the_models_output_is_logged_and_replayed_byte_exact` | passes, before and after |
|
||||
| `an_old_log_with_a_raw_tag_is_replayed_as_it_is` | passes, before and after |
|
||||
|
||||
Every message ends `(not shown: it may hold a tag)`.
|
||||
- [ ] **3. `turn.rs`.** Make edits T1 to T4. `cargo check -p loopd`: no error.
|
||||
- [ ] **4. `baseline.rs`.** Make edits B1 to B3. `cargo check -p loopd`: no error.
|
||||
- [ ] **5. See them pass.** The command of step 2: `neutralise` 6 passed, `neutralise_replay` 3
|
||||
passed. Then `cargo test -p loopd`: every suite passes.
|
||||
- [ ] **6. Check.** `grep -rn 'neutralise(' crates/loopd/src` prints exactly three lines: two in
|
||||
`turn.rs` (T3, T4) and one in `baseline.rs` (B3).
|
||||
- [ ] **7. Run the gate.** `cargo fmt --all`, then `make gate`, with a shell timeout of 600000 ms.
|
||||
Expected last line: `gate: ok`, with 1042 tests: 1033 before this task, plus 6 in
|
||||
`neutralise.rs` and 3 in `neutralise_replay.rs`.
|
||||
- [ ] **8. Log and commit.** `git status --short` lists `crates/loopd/src/turn.rs`,
|
||||
`crates/loopd/src/baseline.rs` and your log as modified, and the three new test files. Stage
|
||||
with `git add crates/loopd docs/implementer-log.md Cargo.lock`, then commit as `AGENTS.md`
|
||||
says. `Cargo.lock` does not change in this task; the `git add` is harmless.
|
||||
|
||||
## Done when
|
||||
|
||||
- `make gate` prints `gate: ok` with 1042 tests.
|
||||
- `git diff HEAD~1 --stat` lists only `crates/loopd/src/turn.rs`, `crates/loopd/src/baseline.rs`,
|
||||
the three new files under `crates/loopd/tests/`, and `docs/implementer-log.md`.
|
||||
|
||||
## Stop and report if
|
||||
|
||||
- A "before" block of T1 to T4 or B1 to B3 is not in its file exactly as shown, or is there more
|
||||
than once.
|
||||
- Step 2 shows a different result: a test that does not compile, other failing tests, or other
|
||||
messages.
|
||||
- After step 5 a test fails. Do not open the test to find out why, and do not change a test: stop,
|
||||
and give the failing tests' names and their messages.
|
||||
- The fix seems to need a change anywhere else: in `session.rs`, `baseline::messages`,
|
||||
`channel.rs`, `tools.rs`, `bxcore`, or a `Cargo.toml`. It does not; stop and say why you think
|
||||
so.
|
||||
- A test outside `loopd` fails in the gate: check once whether it fails without your change, then
|
||||
stop and report both results.
|
||||
- The gate passes with a count other than 1042.
|
||||
Reference in New Issue
Block a user