boxmaker/docs/plans/M2a/12-selftest.md

M2a task 12: the startup self-test

Branch: m2a (run git switch m2a; git status --short must be empty, otherwise stop) Commit subject: Add the startup self-test and the loopd selftest command

Goal

Before loopd serves anyone it checks that the server is the one its config describes, that tool calls come back parsed, and that a second turn reuses the first turn's cache. If any check fails, loopd refuses to start. This task writes the checks and a command that runs them.

Context

From the design brief: "On boot loopd checks: tool-call round trip parses, turn-2 prompt processing count shows a cache hit, configured context matches what the server reports. It refuses to start if any check fails."

Files

• Copy: crates/loopd/tests/selftest.rs
• Create: crates/loopd/src/selftest.rs
• Modify: crates/loopd/src/lib.rs, crates/loopd/src/main.rs, docs/implementer-log.md

Interfaces

// crates/loopd/src/selftest.rs
#[derive(Debug)]
pub enum SelfTestError {
    Mismatch { what: &'static str, expected: String, got: String },
    ToolCall(String),
    CacheMiss { expected: u64, got: u64 },
    Infer(InferError),
    Hash,
}   // Display, std::error::Error, and From<InferError>

/// Runs the three checks in order. `on_step` is told the name of each check as it starts.
pub fn run(client: &Client, on_step: &mut dyn FnMut(&str)) -> Result<(), SelfTestError>;

The checks. All chat requests use client.config().slots.main, thinking: false, and chat_with_retry with an event callback that does nothing.

1. on_step("server matches config"). Call props(). Compare, in this order, and return Mismatch with the given what for the first that differs (expected from the config, got from the server, both as text):

what	Config	Server
"chat template sha256"	expect.template_sha256 (as hex)	proto::sha256(chat_template.as_bytes()) (as hex); a hash error is SelfTestError::Hash
"context per slot"	expect.n_ctx	props.n_ctx
"slot count"	expect.slots	props.total_slots

A server that is not the expected one is sent no prompt at all.

2. on_step("tool call round trip"). Messages: System "You are Boxmaker, a careful personal agent." and User "Read /etc/hostname and tell me what it says." One tool:

ToolSchema {
    name: "read_file".to_string(),
    description: "Read a text file and return its contents.".to_string(),
    parameters: serde_json::json!({
        "type": "object",
        "properties": { "path": { "type": "string", "description": "Absolute path" } },
        "required": ["path"],
    }),
}

Return ToolCall(reason) unless: finish_reason is ToolCalls, there is a first tool call, its name is read_file, its arguments parse as JSON, and the JSON has a string path.

3. on_step("turn 2 cache hit"). No tools. Turn 1: the same System message and User "Name one colour. One word." Turn 2: those two messages, then an Assistant message made from turn 1's completion (its content, reasoning_content and tool_calls, unchanged), then User "Name another. One word." If cache_outcome(&turn1.timings, &turn2.timings) is a Loss, return CacheMiss with its two numbers.

main.rs. loopd selftest --config <path> loads the config (Config::load), makes a Client, runs the self-test printing selftest: <step> to stderr for each step, then selftest: ok and exit code 0, or selftest: FAILED: <error> and exit code 1. A config that cannot be loaded prints loopd: <error> and exits 1. Any other arguments print a usage line and exit 2.

Steps

• 1. Copy. git switch m2a, then cp docs/plans/M2a/files/crates/loopd/tests/selftest.rs crates/loopd/tests/
• 2. See the test fail. cargo test -p loopd --test selftest. Expected: it does not compile.
• 3. Write selftest.rs, register it in lib.rs, and rewrite main.rs. Run cargo fmt --all.
• 4. See the test pass. cargo test -p loopd --test selftest. Expected: 6 passed.
• 5. Try it on the real server, if straylight can be reached. In one terminal: cargo run -p inferproxy -- --listen /tmp/ip.sock --upstream straylight:11434. Write /tmp/loopd.toml by copying crates/loopd/tests/fixtures/config/minimal.toml and changing socket to /tmp/ip.sock. Then cargo run -p loopd -- selftest --config /tmp/loopd.toml. Expected: three step lines and selftest: ok, in a few seconds. Then set slots = 3 in the file and run it again. Expected: selftest: FAILED: slot count: … and exit code 1. Put what you saw in your log row. Remove both files in /tmp afterwards.
• 6. Run the gate. make gate. Expected last line: gate: ok.
• 7. Log and commit. git add crates/loopd docs/implementer-log.md && git commit

Done when

• cargo test -p loopd --test selftest reports 6 passed; make gate prints gate: ok.

Stop and report if

• The real server fails check 1 with the unchanged minimal.toml. That means the server's chat template or settings changed, which the owner needs to know; it is not something to fix in code.