The tasks build the inference path: emsha-backed SHA-256, inferproxy, config, a hand-written HTTP and SSE client, request building, delta assembly, the chat state machine, the thinking cap, the slot gate with retry, the startup self-test and on-device verification. Everything the tasks copy in was checked against a private reference implementation: the gate passes after each task in order, the timing tests pass repeatedly under CPU load, and the reference passes the self-test and all four device checks on straylight. Expected results for the recorded streams were derived by a separate script. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
147 lines
6.0 KiB
Markdown
147 lines
6.0 KiB
Markdown
# M2a task 03: `loopd` configuration
|
|
|
|
**Branch:** `m2a` (run `git switch m2a`; `git status --short` must be empty, otherwise stop)
|
|
**Commit subject:** `Add loopd configuration with every M2a limit`
|
|
|
|
## Goal
|
|
|
|
Read `config.toml` into a typed `Config`. Every limit the inference client uses is a field here
|
|
with a default, so that the owner can change it in one line and tests can make it milliseconds.
|
|
|
|
## Context
|
|
|
|
This is **our own format**, so unknown keys are errors, in every table. The reason is practical:
|
|
a misspelt `livenes_ms` that silently fell back to its default would be a limit the owner believes
|
|
is set and is not. The test checks every table separately.
|
|
|
|
A full file:
|
|
|
|
```toml
|
|
[infer]
|
|
socket = "/run/boxmaker/infer/infer.sock" # where inferproxy listens
|
|
model = "ornith-1.5-35b-a3b" # the id the router knows the model by
|
|
|
|
[slots]
|
|
main = 0
|
|
background = 1
|
|
|
|
[expect] # what the server must report; checked at startup
|
|
template_sha256 = "f55f52930aa8bf44ab5cb85f99370fcc3c56e9a85640b812086d5330bce5d86b"
|
|
n_ctx = 131072
|
|
slots = 2
|
|
|
|
[sampling] # optional; these are the defaults
|
|
temperature = 0.6
|
|
top_p = 0.95
|
|
top_k = 20
|
|
|
|
[limits] # optional; see the table below for the defaults
|
|
liveness_ms = 30000
|
|
```
|
|
|
|
## Files
|
|
|
|
- Copy: `crates/loopd/tests/config.rs`, `crates/loopd/tests/fixtures/config/` (2 files)
|
|
- Create: `crates/loopd/src/config.rs`
|
|
- Modify: `crates/loopd/Cargo.toml`, `crates/loopd/src/lib.rs`, `docs/implementer-log.md`,
|
|
`Cargo.lock` (generated)
|
|
|
|
## Interfaces
|
|
|
|
Produces, in `crates/loopd/src/config.rs`. All fields are `pub`.
|
|
|
|
```rust
|
|
pub struct Config { infer: Infer, slots: Slots, expect: Expect, sampling: Sampling, limits: Limits }
|
|
pub struct Infer { socket: std::path::PathBuf, model: String }
|
|
pub struct Slots { main: u32, background: u32 }
|
|
pub struct Expect { template_sha256: proto::Hash32, n_ctx: u64, slots: u32 }
|
|
pub struct Sampling { temperature: f64, top_p: f64, top_k: u32 }
|
|
pub struct Limits { /* twelve fields, below */ }
|
|
|
|
pub enum ConfigError { Read(PathBuf, std::io::Error), Parse(PathBuf, toml::de::Error) }
|
|
|
|
impl Config {
|
|
pub fn parse(text: &str) -> Result<Config, toml::de::Error>;
|
|
pub fn load(path: &std::path::Path) -> Result<Config, ConfigError>;
|
|
}
|
|
```
|
|
|
|
`[infer]`, `[slots]` and `[expect]` are required. `[sampling]` and `[limits]` are optional, and
|
|
inside them every key is optional: a file that sets one limit keeps the defaults for the others.
|
|
|
|
| `Limits` field | Type | Default | Meaning |
|
|
|---|---|---|---|
|
|
| `poll_ms` | `u64` | 5000 | Read timeout while no byte has arrived; one `/slots` poll per timeout |
|
|
| `busy_wait_ms` | `u64` | 600000 | How long a busy slot is waited for |
|
|
| `load_wait_ms` | `u64` | 180000 | How long an unavailable server is waited for |
|
|
| `idle_grace_ms` | `u64` | 30000 | How long an idle slot may stay silent |
|
|
| `liveness_ms` | `u64` | 30000 | Silence allowed once bytes are arriving |
|
|
| `thinking_cap` | `u64` | 4096 | Reasoning tokens before the cap fires |
|
|
| `thinking_overrun` | `u64` | 256 | Reasoning tokens allowed after the cap fired |
|
|
| `max_tokens` | `u64` | 8192 | Backstop sent with every request |
|
|
| `queue_len` | `usize` | 8 | Requests that may wait for one slot |
|
|
| `retry_attempts` | `u32` | 4 | Attempts in all, the first included |
|
|
| `retry_backoff_ms` | `Vec<u64>` | `[2000, 8000, 30000]` | Wait before each retry |
|
|
| `retry_window_ms` | `u64` | 300000 | No retry starts later than this |
|
|
|
|
Derives: `Debug, Clone, PartialEq, Deserialize` on all five structs, plus `Eq` on the three that
|
|
hold no `f64` (`Infer`, `Expect`, `Limits`), plus `Copy` and `Eq` on `Slots`, plus `Copy` on
|
|
`Sampling`. `Sampling` and `Limits` implement `Default` by hand with the values above.
|
|
`ConfigError` derives `Debug` and implements `Display` (the message names the file) and
|
|
`std::error::Error`.
|
|
|
|
## API notes (serde 1.0, verified)
|
|
|
|
- `#[serde(deny_unknown_fields)]` goes on **all five** structs. The test adds an unknown key to
|
|
each table in turn, and at the top level, and as a whole unknown table.
|
|
- `#[serde(default)]` on a **field** uses the field type's `Default` when the key (here: the whole
|
|
table) is missing. `#[serde(default)]` on a **struct** fills each missing field from the
|
|
struct's own `Default` impl; that is what makes a partial `[limits]` table work. The two combine
|
|
with `deny_unknown_fields` in one attribute: `#[serde(deny_unknown_fields, default)]`.
|
|
- `proto::Hash32` already deserializes from a 64-character lowercase hex string and rejects
|
|
anything else.
|
|
|
|
## Steps
|
|
|
|
- [ ] **1. Copy the test and fixtures, add the dependencies.**
|
|
|
|
```sh
|
|
git switch m2a
|
|
mkdir -p crates/loopd/tests/fixtures
|
|
cp docs/plans/M2a/files/crates/loopd/tests/config.rs crates/loopd/tests/
|
|
cp -r docs/plans/M2a/files/crates/loopd/tests/fixtures/config crates/loopd/tests/fixtures/
|
|
```
|
|
|
|
Add to `[dependencies]` in `crates/loopd/Cargo.toml`, below `proto.workspace = true`:
|
|
|
|
```toml
|
|
serde.workspace = true
|
|
serde_json.workspace = true
|
|
toml.workspace = true
|
|
```
|
|
|
|
All three are already in the workspace table and in `docs/dependencies.md`. `serde_json` is not
|
|
used until the next task; add it now so that the manifest is touched once.
|
|
|
|
- [ ] **2. See the test fail.** `cargo test -p loopd --test config`. Expected: it does not compile.
|
|
- [ ] **3. Write `config.rs`,** and add `pub mod config;` to `crates/loopd/src/lib.rs` below the doc
|
|
comment. Run `cargo fmt --all`.
|
|
- [ ] **4. See the test pass.** `cargo test -p loopd --test config`. Expected: `6 passed`.
|
|
- [ ] **5. Check the "everywhere" rule yourself.** List the structs in `config.rs` and confirm each
|
|
of the five has `deny_unknown_fields`. Write the list in your log row.
|
|
- [ ] **6. Run the gate.** `cargo build`, then `make gate`. Expected last line: `gate: ok`.
|
|
- [ ] **7. Log and commit.**
|
|
|
|
```sh
|
|
git add Cargo.lock crates/loopd docs/implementer-log.md
|
|
git commit
|
|
```
|
|
|
|
## Done when
|
|
|
|
- `cargo test -p loopd --test config` reports 6 passed; `make gate` prints `gate: ok`.
|
|
|
|
## Stop and report if
|
|
|
|
- `deny_unknown_fields` and a struct-level `default` cannot be combined as described.
|