docs: M6b implementation plan

Six closed tasks for consult: registry, CONSULT_WAIT, [consult]/[secrets], ToolName and grants, Decision body, loopback HTTP. Reference is the git bundle ~/boxmaker-refs/m6b-ref.bundle at 5fb05f0. Gate after task 06 is 1611. Co-Authored-By: Grok 4.6 <noreply@x.ai>
2026-10-03 21:39:31 -07:00
parent 09580c705a
commit 50e9cc2037
15 changed files with 2887 additions and 0 deletions
@@ -0,0 +1,426 @@
+# M6b task 03: `[consult]` and `[secrets]`, start fails closed
+
+**Branch:** `m6b`. Check it: `git branch --show-current` prints `m6b` and `git status --short`
+is empty; otherwise stop. Never switch branch.
+**Commit subject:** `brokerd: consult config, secrets, loopback URL`
+**Log row:** the Task cell is `M6b/03-brokerd-consult-config`.
+
+**First action.** Run this before any Read of crate source. Do not `ls`
+`crates/brokerd/tests/`. Do not Read `args.rs`, `broker.rs`, `ledger/`,
+`docs/implementer-log.md` (except the first 12 lines at the log step). Do not edit
+`crates/brokerd/tests/config.rs`. The signatures below are complete.
+
+```sh
+cp docs/plans/M6b/files/crates/brokerd/tests/config_consult.rs crates/brokerd/tests/
+cp docs/plans/M6b/files/crates/brokerd/tests/consult_secret.rs crates/brokerd/tests/
+```
+
+Then append the runbook fragment (step 1 of Steps names the exact command).
+
+## Goal
+
+`brokerd.toml` may have `[consult]` and `[secrets.<name>]`. A present `[consult]` is
+validated at load: loopback `http://` only, non-empty model and secret, `timeout_ms` ≥ 1,
+temperature from 0 to 0.6 inclusive. Temperature is stored as `temperature_milli: u16` so
+`Config` stays `Eq`. No `[consult]` is `None` and an empty `[secrets]` map. Loading the
+named secret at start is `load_secret`; a missing or unreadable secret is
+`consult-secret-unreadable` and `brokerd` must not start. This task does not POST, parse
+tool arguments, or change `ToolName`.
+
+## Every place that changes
+
+| Place | What happens there |
+|---|---|
+| `crates/brokerd/Cargo.toml` | `bxcore` enables feature `secrets` |
+| `crates/brokerd/src/consult.rs` | new: `Consult`, `validate`, `parse_loopback_http`, `load_secret` |
+| `crates/brokerd/src/lib.rs` | `pub mod consult;` |
+| `crates/brokerd/src/config.rs` | `consult`, `secrets`; validate at load |
+| `crates/brokerd/tests/support/rig.rs` | the `Config { }` literal names the new fields |
+| `docs/runbook.md` | append both consult entries |
+| `crates/brokerd/tests/config_consult.rs` | new, given |
+| `crates/brokerd/tests/consult_secret.rs` | new, given |
+| everything else | **no change**: not `ToolName`, not `main.rs`, not `ledger` |
+
+## Files
+
+- Copy, from `docs/plans/M6b/files/` (the same path under it): the new tests
+  `crates/brokerd/tests/config_consult.rs` and
+  `crates/brokerd/tests/consult_secret.rs`.
+- Append, from `docs/plans/M6b/files/runbook-consult.md`, to `docs/runbook.md`. Both
+  headings go in now so `scripts/check-runbook.sh` accepts `SECRET_POINTER`; task 05
+  adds the `consult-not-configured` pointer in code.
+- Create: `crates/brokerd/src/consult.rs` (the module below).
+- Modify: `crates/brokerd/Cargo.toml`, `crates/brokerd/src/lib.rs`,
+  `crates/brokerd/src/config.rs`, `crates/brokerd/tests/support/rig.rs`,
+  `docs/runbook.md`, `docs/implementer-log.md`.
+- Do not touch: any other file. Do not edit `crates/brokerd/tests/config.rs`.
+  After the Cargo.toml change, run `cargo build` once so `Cargo.lock` updates.
+
+## Facts you need
+
+- **No vendor names.** The URL, model and secret name are configuration. The module must
+  not mention a cloud provider.
+- **Loopback HTTP only.** `parse_loopback_http` accepts `http://127.0.0.1` or
+  `http://localhost`, optional port 1–65535 and path. `https://`, a non-loopback host, a
+  userinfo (`@`), or port `0` is an error. Default port is 80. A URL with no path uses
+  `"/"`.
+- **Temperature.** TOML `temperature` is an `f64` in 0..=0.6, or the integer `0`. Any
+  other integer is invalid (`must be from 0 to 0.6`). Store `(v * 1000.0).round()` as
+  `u16` via `format!("{milli:.0}").parse::<u16>()`. Default 600 (0.6). `Config` stays
+  `Eq`; do not store `f64` on `Consult`.
+- **`#[serde(default)]` on the new `Config` fields.** `an_empty_file_gets_every_default`
+  in `config.rs` compares a loaded empty file to `Config::default()`. Missing tables are
+  `consult: None` and `secrets: {}`.
+- **The only `Config { }` literal** is `crates/brokerd/tests/support/rig.rs`. Add
+  `consult: None, secrets: Default::default()`. Other tests load TOML.
+- **Secrets.** Same `SecretSpec` as `gatewayd`: exactly one of `credential` / `env` /
+  `file`. `load` is `bxcore::secrets::load`. Role string `"brokerd"`. Load the consult
+  secret only when `[consult]` is present; absent table is `Ok(None)`.
+- **Do not add `ToolArgs`, `QUESTION_CAP`, `NOT_CONFIGURED`, a transport, or HTTP.**
+  Those are later tasks. An unused constant fails clippy.
+- The items the edits use. This table is complete.
+
+  | Item | Signature |
+  |---|---|
+  | `SecretSpec` | `{ credential: Option<String>, env: Option<String>, file: Option<PathBuf> }` in `bxcore::secrets` |
+  | `SecretSpec::source` | `source(&self) -> Result<SecretSource, String>` |
+  | `bxcore::secrets::load` | `load(role: &str, name: &str, source: &SecretSource, env: &dyn Fn(&str) -> Option<OsString>) -> Result<Loaded, SecretError>` |
+  | `Loaded` | `{ secret: Secret, warning: Option<String> }` |
+  | `SecretError` | `Display` names the secret and `docs/runbook.md#secret-unavailable` |
+  | `Config::load` | `load(path: &Path) -> Result<Config, ConfigError>` |
+  | `ConfigError::Invalid` | `Invalid(PathBuf, String)` |
+  | `ConfigError::Parse` | `Parse(PathBuf, toml::de::Error)` |
+
+## The edits, exactly
+
+### `crates/brokerd/Cargo.toml`
+
+**C1.** The line
+
+```toml
+bxcore.workspace = true
+```
+
+becomes
+
+```toml
+bxcore = { workspace = true, features = ["secrets"] }
+```
+
+Then `cargo build` once (timeout 600000 ms) so `Cargo.lock` records the feature.
+
+### `crates/brokerd/src/lib.rs`
+
+**L1.** After `pub mod config;` add `pub mod consult;`. rustfmt decides the final order.
+
+### `crates/brokerd/src/consult.rs`
+
+Write this module. Do not add `ConsultArgs`, `wire_body`, `ConsultTransport`, or
+`ConsultRuntime`.
+
+```rust
+//! `consult`: an allowlisted chat-completions POST decided by the grant's `max_taint`.
+//! Nothing here names a vendor (M6 spec, section 4).
+
+use std::ffi::OsString;
+use std::time::Duration;
+
+use bxcore::secrets::{Loaded, SecretError, SecretSpec};
+use serde::{Deserialize, Deserializer};
+
+use crate::config::Config;
+
+pub const CONSULT: &str = "consult";
+/// Default wait for the remote completion, milliseconds.
+pub const DEFAULT_TIMEOUT_MS: u64 = 180_000;
+/// Default sampling temperature as thousandths (0.6).
+pub const DEFAULT_TEMPERATURE_MILLI: u16 = 600;
+/// Start failure when the named secret cannot be read.
+pub const SECRET_POINTER: &str = "see docs/runbook.md#consult-secret-unreadable";
+
+/// `[consult]` in `brokerd.toml`. Unknown keys are errors.
+#[derive(Debug, Clone, PartialEq, Eq, Deserialize)]
+#[serde(deny_unknown_fields)]
+pub struct Consult {
+    pub url: String,
+    pub model: String,
+    #[serde(
+        rename = "temperature",
+        default = "default_temperature_milli",
+        deserialize_with = "de_temperature_milli"
+    )]
+    pub temperature_milli: u16,
+    #[serde(default = "default_timeout_ms")]
+    pub timeout_ms: u64,
+    pub secret: String,
+}
+
+fn default_temperature_milli() -> u16 {
+    DEFAULT_TEMPERATURE_MILLI
+}
+
+fn default_timeout_ms() -> u64 {
+    DEFAULT_TIMEOUT_MS
+}
+
+fn de_temperature_milli<'de, D: Deserializer<'de>>(deserializer: D) -> Result<u16, D::Error> {
+    #[derive(Deserialize)]
+    #[serde(untagged)]
+    enum Num {
+        F(f64),
+        I(i64),
+    }
+    let v = match Num::deserialize(deserializer).map_err(serde::de::Error::custom)? {
+        Num::F(f) => f,
+        Num::I(0) => 0.0,
+        Num::I(_) => {
+            return Err(serde::de::Error::custom(
+                "[consult] temperature must be from 0 to 0.6",
+            ));
+        }
+    };
+    if !v.is_finite() || v < 0.0 || v > 0.6 {
+        return Err(serde::de::Error::custom(
+            "[consult] temperature must be from 0 to 0.6",
+        ));
+    }
+    let milli = (v * 1000.0).round();
+    let text = format!("{milli:.0}");
+    text.parse::<u16>().map_err(serde::de::Error::custom)
+}
+
+impl Consult {
+    /// The temperature written in the JSON body.
+    pub fn temperature(&self) -> f64 {
+        f64::from(self.temperature_milli) / 1000.0
+    }
+
+    pub fn timeout(&self) -> Duration {
+        Duration::from_millis(self.timeout_ms)
+    }
+}
+
+/// Why a `[consult]` table cannot be used.
+pub fn validate(consult: &Consult) -> Result<(), String> {
+    parse_loopback_http(&consult.url).map(|_| ())?;
+    if consult.model.is_empty() {
+        return Err("[consult] model must not be empty".to_string());
+    }
+    if consult.timeout_ms == 0 {
+        return Err("[consult] timeout_ms must be at least 1".to_string());
+    }
+    if consult.secret.is_empty() {
+        return Err("[consult] secret must not be empty".to_string());
+    }
+    if consult.temperature_milli > DEFAULT_TEMPERATURE_MILLI {
+        return Err("[consult] temperature must be from 0 to 0.6".to_string());
+    }
+    Ok(())
+}
+
+/// Host, port and path of a loopback `http://` URL. M6 stays on loopback.
+pub fn parse_loopback_http(url: &str) -> Result<(String, u16, String), String> {
+    let rest = match url.strip_prefix("http://") {
+        Some(rest) => rest,
+        None => {
+            return Err("[consult] url must be http://127.0.0.1 or http://localhost".to_string());
+        }
+    };
+    if rest.contains('@') {
+        return Err("[consult] url must not include a userinfo".to_string());
+    }
+    let (hostport, path) = match rest.find('/') {
+        Some(i) => match rest.split_at_checked(i) {
+            Some(pair) => pair,
+            None => {
+                return Err("[consult] url is not a valid http URL".to_string());
+            }
+        },
+        None => (rest, "/"),
+    };
+    let path = if path.is_empty() { "/" } else { path };
+    let (host, port) = match hostport.split_once(':') {
+        Some((host, port)) => {
+            let port: u16 = port
+                .parse()
+                .map_err(|_| "[consult] url port is not a number".to_string())?;
+            if port == 0 {
+                return Err("[consult] url port must be from 1 to 65535".to_string());
+            }
+            (host, port)
+        }
+        None => (hostport, 80),
+    };
+    if host != "127.0.0.1" && host != "localhost" {
+        return Err("[consult] url must be http://127.0.0.1 or http://localhost".to_string());
+    }
+    Ok((host.to_string(), port, path.to_string()))
+}
+
+/// Why `brokerd` will not start with `[consult]` set.
+#[derive(Debug)]
+pub enum SecretStartError {
+    Missing(String),
+    Spec(String, String),
+    Load(SecretError),
+}
+
+impl std::fmt::Display for SecretStartError {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+        match self {
+            SecretStartError::Missing(name) => {
+                write!(f, "secret {name} is not in [secrets]; {SECRET_POINTER}")
+            }
+            SecretStartError::Spec(name, why) => {
+                write!(f, "[secrets.{name}]: {why}; {SECRET_POINTER}")
+            }
+            SecretStartError::Load(err) => write!(f, "{err}\n{SECRET_POINTER}"),
+        }
+    }
+}
+
+impl std::error::Error for SecretStartError {
+    fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
+        match self {
+            SecretStartError::Load(err) => Some(err),
+            SecretStartError::Missing(_) | SecretStartError::Spec(_, _) => None,
+        }
+    }
+}
+
+/// Load the bearer for `[consult]`, or `Ok(None)` when the table is absent.
+pub fn load_secret(
+    cfg: &Config,
+    env: &dyn Fn(&str) -> Option<OsString>,
+) -> Result<Option<Loaded>, SecretStartError> {
+    let Some(consult) = &cfg.consult else {
+        return Ok(None);
+    };
+    let spec: &SecretSpec = match cfg.secrets.get(&consult.secret) {
+        Some(spec) => spec,
+        None => return Err(SecretStartError::Missing(consult.secret.clone())),
+    };
+    let source = spec
+        .source()
+        .map_err(|why| SecretStartError::Spec(consult.secret.clone(), why))?;
+    let loaded = bxcore::secrets::load("brokerd", &consult.secret, &source, env)
+        .map_err(SecretStartError::Load)?;
+    Ok(Some(loaded))
+}
+```
+
+### `crates/brokerd/src/config.rs`
+
+**F1.** The imports
+
+```rust
+use serde::Deserialize;
+use std::path::{Path, PathBuf};
+```
+
+become
+
+```rust
+use serde::Deserialize;
+use std::collections::BTreeMap;
+use std::path::{Path, PathBuf};
+
+use bxcore::secrets::SecretSpec;
+```
+
+**F2.** After `pub memory: Memory,` (the last field of `Config`) add:
+
+```rust
+    #[serde(default)]
+    pub consult: Option<crate::consult::Consult>,
+    #[serde(default)]
+    pub secrets: BTreeMap<String, SecretSpec>,
+```
+
+**F3.** In `Config::load`, immediately before `Ok(config)`, add:
+
+```rust
+        if let Some(consult) = &config.consult
+            && let Err(why) = crate::consult::validate(consult)
+        {
+            return Err(ConfigError::Invalid(path.to_path_buf(), why));
+        }
+```
+
+Do not change `time_limit` (no `ToolName::Consult` yet).
+
+### `crates/brokerd/tests/support/rig.rs`
+
+**R1.** The `Config {` fields ending
+
+```rust
+            gateway: None,
+            memory: Default::default(),
+        };
+```
+
+become
+
+```rust
+            gateway: None,
+            memory: Default::default(),
+            consult: None,
+            secrets: Default::default(),
+        };
+```
+
+### `docs/runbook.md`
+
+**B1.** Append the file `docs/plans/M6b/files/runbook-consult.md` as-is (both `##`
+headings) after the last line of `docs/runbook.md`.
+
+## Steps
+
+- [ ] **1. Copy the given tests and append the runbook.**
+
+  ```sh
+  cp docs/plans/M6b/files/crates/brokerd/tests/config_consult.rs crates/brokerd/tests/
+  cp docs/plans/M6b/files/crates/brokerd/tests/consult_secret.rs crates/brokerd/tests/
+  cat docs/plans/M6b/files/runbook-consult.md >> docs/runbook.md
+  ```
+
+- [ ] **2. See them fail.** `cargo test -p brokerd --test config_consult --test
+  consult_secret`, timeout 600000 ms: both targets do not compile (`E0432` unresolved
+  `brokerd::consult`).
+- [ ] **3. The edits.** C1 (then `cargo build`), L1, write `consult.rs`, F1–F3, R1, B1.
+  `cargo check -p brokerd` prints no error and no warning. `wc -l
+  crates/brokerd/src/consult.rs` and `wc -l crates/brokerd/src/config.rs` each ≤ 500.
+- [ ] **4. The tests.** `cargo test -p brokerd --test config_consult --test consult_secret
+  --test config`: every test passes, 6 in `config_consult`, 3 in `consult_secret`.
+  `an_empty_file_gets_every_default` still passes.
+- [ ] **5. Run the gate.** `cargo fmt --all`, then `make gate`, timeout 600000 ms. Last
+  line `gate: ok`. Test count **1602** (1593 before, plus 9). After `gate: ok`, do not add
+  up `test result:` lines.
+- [ ] **6. Log and commit.** Task cell `M6b/03-brokerd-consult-config`. Read only the first
+  12 lines of `docs/implementer-log.md`; insert the row under the header. Stage with
+  `git add crates/brokerd Cargo.lock docs/runbook.md docs/implementer-log.md`, then
+  commit as `AGENTS.md` says.
+
+## Done when
+
+- `make gate` prints `gate: ok` with 1602 tests.
+- `git diff HEAD~1 --stat` lists only `crates/brokerd/Cargo.toml`,
+  `crates/brokerd/src/consult.rs`, `crates/brokerd/src/lib.rs`,
+  `crates/brokerd/src/config.rs`, `crates/brokerd/tests/support/rig.rs`,
+  `crates/brokerd/tests/config_consult.rs`, `crates/brokerd/tests/consult_secret.rs`,
+  `docs/runbook.md`, the log, and `Cargo.lock` if it changed.
+
+## Stop and report if
+
+- The branch check fails; a "before" block is missing or duplicated; step 2 compiles; you
+  edit `tests/config.rs` or add `ToolName::Consult`; `cargo check` warns; existing
+  `config` tests fail; a test outside the named files fails in the gate (`git stash -u`,
+  rerun, `git stash pop`, report both).
+
+## What the given tests pin
+
+No `[consult]` is `None` with empty secrets. A table loads url/model/secret and defaults
+temperature 0.6 / timeout 180_000. `temperature = 0.7` is parse-invalid. A non-loopback
+URL is `ConfigError::Invalid`. An unknown `[consult]` key is parse-invalid. `[secrets.consult]`
+loads. A missing secrets entry and an unset env secret both name
+`consult-secret-unreadable`; a present env secret loads.