Files

T

kyleandClaude Fable 5.1 76ccc251cd Add M2a plan: thirteen tasks, tests, fake server and recordings

The tasks build the inference path: emsha-backed SHA-256, inferproxy,
config, a hand-written HTTP and SSE client, request building, delta
assembly, the chat state machine, the thinking cap, the slot gate with
retry, the startup self-test and on-device verification.

Everything the tasks copy in was checked against a private reference
implementation: the gate passes after each task in order, the timing
tests pass repeatedly under CPU load, and the reference passes the
self-test and all four device checks on straylight. Expected results
for the recorded streams were derived by a separate script.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

2026-09-17 13:34:11 -07:00

2.9 KiB

Raw Permalink Blame History

M2a task 13: verification on the real server

Branch: m2a (run git switch m2a; git status --short must be empty, otherwise stop) Commit subject: Add on-device verification for the inference path

Goal

Run the whole inference path against the real server on straylight and show that it holds: the self-test, a capped thinking block, a request that survives its connection being killed, and a second turn that reuses the cache of a first turn that contained thinking.

You write no library code in this task. You put two given files in place, run the checks, and report what happened. If a check fails, that is a finding. Do not change a test or a limit to make it pass.

Context

make verify-device builds the workspace, then runs the tests in crates/loopd/tests/device.rs that are marked #[ignore], one at a time. Each starts its own inferproxy pointed at straylight:11434 and talks to the model through it, on slot 0. The "survives" test kills and restarts its own inferproxy in the middle of an answer. Nothing here touches the real llama-server process.

Files

Copy: crates/loopd/tests/device.rs, Makefile (replaces the old one; only the verify-device target differs)
Modify: docs/implementer-log.md

Steps

1. Copy.

git switch m2a
cp docs/plans/M2a/files/crates/loopd/tests/device.rs crates/loopd/tests/
cp docs/plans/M2a/files/Makefile Makefile
git diff --stat Makefile

2. Check that the ordinary gate ignores the new tests. make gate. Expected last line: gate: ok, and in the output for device: 0 passed; 0 failed; 4 ignored.
3. Check that the server can be reached. curl -s http://straylight:11434/health. Expected: {"status":"ok"}. If not, stop and report: this task cannot be done without it.
4. Run the device checks. make verify-device. Expected: 4 passed, then verify-device: ok, in well under two minutes. The first run may take longer if the model has to be loaded.
5. If a check failed, run it alone to see its output, for example:

BOXMAKER_INFERPROXY=$PWD/target/debug/inferproxy BOXMAKER_UPSTREAM=straylight:11434 \
  cargo test -p loopd --test device -- --ignored --test-threads=1 --nocapture a_capped_thinking

Put the test's name and its message in your log row with status stopped. A failure here is either a real defect in an earlier task or a change on the server; the reviewer decides which.

6. Log and commit. Put the four results and the total time in your log row.

git add Makefile crates/loopd/tests/device.rs docs/implementer-log.md
git commit

Done when

make gate prints gate: ok and make verify-device prints verify-device: ok.
cmp Makefile docs/plans/M2a/files/Makefile prints nothing.
git log --oneline master..m2a shows one commit per task.

Stop and report if

The server cannot be reached, or any device check fails twice.

2.9 KiB Raw Permalink Blame History