From 0e1e070fc399c6bc501cd011e0b83d1391e3fcfd Mon Sep 17 00:00:00 2001 From: Kyle Isom Date: Mon, 28 Sep 2026 02:48:16 -0700 Subject: [PATCH] M5c-1 plan: task 06, memoryd checks [honcho] and its keys at start memoryd.toml gains an optional [honcho] section, [[honcho.read]] and the Honcho keys in [secrets.], checked at start into memoryd::honcho::config::HonchoConfig (13 reasons, runbook entry honcho-config-invalid given as a fragment), and [sockets] state for brokerd's state.sock. Keys are checked, never read; no [honcho] leaves memoryd as it was. memoryd enables bxcore's feature secrets. Checks: a reference at ref 05 + this task passes make gate with 1188 tests (1171 + 17); /tmp unchanged around the gate; the given tests fail before as the task says (two targets do not compile, 4 start tests fail); 49 one-line mutations of the reference, all caught (one gap, a URL with a user and no password, found and closed); the new tests 40 times under CPU load, no failure; a replay from the task text on a fresh ref-05 worktree with its own target is byte-identical and passes the gate, and every before block was found once; given files pass fmt and the 500-line limit; no chat-template tag in any file. Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/plans/M5c1/06-memoryd-honcho-config.md | 611 ++++++++++++++++++ .../files/crates/memoryd/src/honcho/config.rs | 220 +++++++ .../crates/memoryd/tests/honcho_config.rs | 229 +++++++ .../crates/memoryd/tests/honcho_reasons.rs | 338 ++++++++++ .../crates/memoryd/tests/honcho_start.rs | 238 +++++++ .../files/runbook-honcho-config-invalid.md | 50 ++ 6 files changed, 1686 insertions(+) create mode 100644 docs/plans/M5c1/06-memoryd-honcho-config.md create mode 100644 docs/plans/M5c1/files/crates/memoryd/src/honcho/config.rs create mode 100644 docs/plans/M5c1/files/crates/memoryd/tests/honcho_config.rs create mode 100644 docs/plans/M5c1/files/crates/memoryd/tests/honcho_reasons.rs create mode 100644 docs/plans/M5c1/files/crates/memoryd/tests/honcho_start.rs create mode 100644 docs/plans/M5c1/files/runbook-honcho-config-invalid.md diff --git a/docs/plans/M5c1/06-memoryd-honcho-config.md b/docs/plans/M5c1/06-memoryd-honcho-config.md new file mode 100644 index 0000000..e5562b8 --- /dev/null +++ b/docs/plans/M5c1/06-memoryd-honcho-config.md @@ -0,0 +1,611 @@ +# M5c-1 task 06: `memoryd` checks its `[honcho]` section and its keys at start + +**Branch:** `m5c1` (run `git switch m5c1`; `git status --short` must be empty, otherwise stop) +**Commit subject:** `memoryd: [honcho] and its keys checked at start, and brokerd's state socket` +**Log row:** the Task cell is `M5c1/06-memoryd-honcho-config`. + +## Goal + +From M5c on, `memoryd` sends finished turns and memory writes to Honcho, the memory the agents of +the fleet share (M5c spec, sections 3 and 10). This task adds the configuration for it, read and +checked when `memoryd` starts. Nothing talks to Honcho or to `brokerd` yet: later tasks use what +this task checks. + +- **`[honcho]`**, optional, in `memoryd.toml`: + + ```toml + [honcho] + url = "http://rift.scylla-hammerhead.ts.net:8000" # http(s)://, a host, optional port + agent = "boxmaker0" # this agent's peer + owner = "kyle" # the owner's peer, the same in the fleet + trusted = "boxmaker0-trusted" # this agent's three workspaces + untrusted = "boxmaker0-untrusted" + fleet = "fleet" + + [[honcho.read]] # zero or more: other agents' workspaces + agent = "boxmaker1" + workspace = "boxmaker1-trusted" + trust = "untrusted" # "trusted" or "untrusted"; the default + class = "private" # "public", "private" or "secret"; default + + [secrets.honcho_trusted] # optional: a key per workspace + env = "HONCHO_KEY" + ``` + + The keys' names are `honcho_trusted`, `honcho_untrusted`, `honcho_fleet`, and + `honcho_read_` for an agent in `[[honcho.read]]`. A key's `[secrets.]` table is the + one every role uses (`credential`, `env` or `file`, M5b). Here a key is only **checked** (its + table names one well-formed source); it is **not read**: a later task reads it when it makes the + Honcho client. No key is ever printed. +- **No `[honcho]` section** (and no key): Honcho is off, and `memoryd` behaves exactly as before. + Every existing `memoryd` test passes unchanged. +- **An invalid section** stops `memoryd` at start, before it takes its lock or touches its home, + with one line and the pointer: + + ```text + memoryd: /etc/boxmaker/memoryd.toml: [honcho] agent and owner are both "kyle": the agent's peer must not be the owner's + see docs/runbook.md#honcho-config-invalid + ``` + + The reasons are in the table "The reasons" below. This task appends the runbook entry, given. +- **`[sockets] state`**: the path of `brokerd`'s `state.sock` (task 05), which `memoryd` will + connect to, to ask for a session's state. Like the other three `[sockets]` keys, empty or + missing means the default under `home`: `/run/memory-broker/state.sock`, the path + `brokerd` serves by default. + +## Interfaces + +Later tasks use these exactly as written. They are in the skeleton you copy (step 3); only the +bodies of seven private or small functions are yours to fill. + +In `crates/memoryd/src/honcho/config.rs` (the module `memoryd::honcho::config`): + +```rust +pub const RUNBOOK: &str = "see docs/runbook.md#honcho-config-invalid"; +pub const MAX_NAME: usize = 512; +pub const TRUSTED_KEY: &str = "honcho_trusted"; +pub const UNTRUSTED_KEY: &str = "honcho_untrusted"; +pub const FLEET_KEY: &str = "honcho_fleet"; +pub const READ_KEY: &str = "honcho_read_"; + +/// `[honcho]` as written (serde; unknown keys are errors). +pub struct HonchoSection { + pub url: String, + pub agent: String, + pub owner: String, + pub trusted: String, + pub untrusted: String, + pub fleet: String, + pub read: Vec, // #[serde(default)] +} +/// One `[[honcho.read]]` as written (serde; unknown keys are errors). +pub struct ReadSection { + pub agent: String, + pub workspace: String, + pub trust: Option, // #[serde(default)] + pub class: Option, // #[serde(default)] +} +/// `[honcho]` and its keys, checked. +pub struct HonchoConfig { + pub url: String, + pub agent: String, + pub owner: String, + pub trusted: String, + pub untrusted: String, + pub fleet: String, + pub read: Vec, + pub keys: BTreeMap, // by the name in [secrets.] +} +pub struct ReadWorkspace { + pub agent: String, + pub workspace: String, + pub trust: Trust, + pub class: DataClass, +} +pub enum Trust { + Trusted, + Untrusted, +} + +pub fn read_key(agent: &str) -> String; // "honcho_read_" +pub fn is_name(name: &str) -> bool; // Honcho's rule: 1 to 512 of a-z, A-Z, 0-9, _ and - +pub fn check( + section: Option<&HonchoSection>, + secrets: &BTreeMap, +) -> Result, String>; +``` + +The derives: `HonchoSection` and `ReadSection` `Debug, Clone, PartialEq, Eq, Deserialize`; +`HonchoConfig` and `ReadWorkspace` `Debug, Clone, PartialEq, Eq`; `Trust` also `Copy`. + +In `crates/memoryd/src/config.rs` (edits C1 to C8): `Config` gains `pub honcho: +Option` and `pub secrets: BTreeMap`; `Sockets` gains `pub +state: PathBuf`; `ConfigError` gains `Honcho(PathBuf, String)`; `Config` gains +`pub fn honcho_config(&self) -> Result, String>` and +`pub fn state_socket(&self) -> PathBuf`. + +## The reasons + +`check` returns the first reason it finds. The order is the given `check`'s: R4 (no section), +R1, R2 for the five names of `[honcho]`, R3, R5, then each `[[honcho.read]]` in turn (R2, R6, R7, +R8, R9, R10), then the keys (R11, R12, R13). `{x:?}` is Rust's `Debug` format: the text in +double quotes, with `"` and control characters escaped. R1 to R4 are in the given code already; +R5 to R13 are yours. + +| # | When | The reason (the text in `Err`) | +|---|---|---| +| R1 | `url` is not `http://` or `https://`, a host and an optional port | `[honcho] url must be http:// or https://, a host and an optional port, and nothing else` (the constant `URL_REASON`; it never quotes the URL, which could hold a password) | +| R2 | a name is not a Honcho name | `{table} {key} {name:?} is not a Honcho name: 1 to 512 of a-z, A-Z, 0-9, _ and -`, `table` being `[honcho]` or `[[honcho.read]]` (the given `check_name`) | +| R3 | `agent` and `owner` are equal | `[honcho] agent and owner are both {agent:?}: the agent's peer must not be the owner's` | +| R4 | a `[secrets.]` with no `[honcho]` | `[secrets.{name}] is set, but there is no [honcho] section` (the first name) | +| R5 | two of `trusted`, `untrusted`, `fleet` are equal | `[honcho] workspace {name:?} is named twice: trusted, untrusted and fleet must be three different workspaces` | +| R6 | a read agent is `[honcho] agent` | `[[honcho.read]] agent {agent:?} is this agent ([honcho] agent)` | +| R7 | a read agent is in an earlier entry | `[[honcho.read]] agent {agent:?} is listed twice` | +| R8 | a read workspace is one of the three | `[[honcho.read]] workspace {workspace:?} is one of this agent's own` | +| R9 | `trust` is another text | `[[honcho.read]] trust {trust:?} of agent {agent:?} is not "trusted" or "untrusted"` | +| R10 | `class` is another text | `[[honcho.read]] class {class:?} of agent {agent:?} is not "public", "private" or "secret"` | +| R11 | `honcho_read_` for an agent not in `[[honcho.read]]` | `[secrets.{name}] is the key of agent {agent:?}, which is not in [[honcho.read]]` | +| R12 | a `[secrets.]` with any other name | `[secrets.{name}] is not a Honcho key: memoryd has only honcho_trusted, honcho_untrusted, honcho_fleet and honcho_read_` | +| R13 | `SecretSpec::source` refuses the table | `[secrets.{name}] {why}`, `why` being `source`'s own text | + +In Rust, R9 and R10 contain `\"` for each double quote shown: for R9, +`format!("[[honcho.read]] trust {other:?} of agent {:?} is not \"trusted\" or \"untrusted\"", +entry.agent)`, where `other` is the text written. In R11, `agent` is what follows `honcho_read_` +in the name. `name` in R4, R11, R12 and R13 is shown as it is, without `:?`. + +## Every place that changes + +This is the complete list. + +| Place | What happens there | +|---|---| +| `crates/memoryd/Cargo.toml` | `bxcore` with the feature `secrets` (edit K1) | +| `crates/memoryd/src/lib.rs` | `pub mod honcho;` (edit L1) | +| `crates/memoryd/src/honcho/mod.rs` | new: step 3 gives it whole | +| `crates/memoryd/src/honcho/config.rs` | new: a skeleton you copy, then fill 7 bodies (step 4) | +| `crates/memoryd/src/config.rs` | module comment, imports, `Sockets.state`, `Config.honcho` and `.secrets`, `ConfigError::Honcho`, `load`, `honcho_config`, `state_socket` (edits C1 to C8) | +| `crates/memoryd/src/main.rs` | the config error's pointer by its kind (edits M1, M2) | +| `docs/runbook.md` | the entry `honcho-config-invalid`, appended (step 6) | +| `docs/dependencies.md` | the `zeroize` row names `memoryd` (edit D1) | +| everything else in `memoryd` (`serve.rs`, `socket.rs`, …) | **no change** | + +## Files + +- Copy, from `docs/plans/M5c1/files/` (the same path under it): + - new tests: `crates/memoryd/tests/honcho_config.rs`, `crates/memoryd/tests/honcho_reasons.rs`, + `crates/memoryd/tests/honcho_start.rs` + - the skeleton `crates/memoryd/src/honcho/config.rs` (you fill 7 bodies in it) +- Append, from `docs/plans/M5c1/files/`, to the end of `docs/runbook.md`, with one blank line + before it: `runbook-honcho-config-invalid.md` (step 6 gives the command). +- Create: `crates/memoryd/src/honcho/mod.rs` (step 3). +- Modify: `crates/memoryd/Cargo.toml`, `crates/memoryd/src/lib.rs`, + `crates/memoryd/src/config.rs`, `crates/memoryd/src/main.rs`, `docs/dependencies.md`, + `docs/implementer-log.md`. +- Do not touch: any other source file, any file of another crate, any other `Cargo.toml`, any + other test (`crates/memoryd/tests/config.rs` in particular: it must pass as it is), the test + support files, and in the skeleton anything but the 7 `todo!()` bodies. + +## Facts you need + +- **The feature `secrets`.** `SecretSpec` and `SecretSource` are in `bxcore::secrets`, which is + compiled only when a crate enables `bxcore`'s feature `secrets` (M5b). `gatewayd` does; `loopd` + never does, as it holds no credentials. `memoryd` holds Honcho's keys, so edit K1 enables it. + `make gate` builds the whole workspace at once, and Cargo then turns the feature on for every + crate because `gatewayd` asks for it; so without K1 the gate could pass while + `cargo build -p memoryd` alone fails. `Cargo.lock` does not change: `zeroize`, the feature's one + dependency, is already `bxcore`'s. +- **Two stages.** `Config::parse` (serde) reads the file: a missing key in `[honcho]` or + `[[honcho.read]]`, an unknown key, or a value of the wrong type is a parse error, as in every + other table, and keeps its pointer `memoryd-start-failed`. Then `Config::load` calls + `honcho_config`, which checks the values (`honcho::config::check`): an `Err` is + `ConfigError::Honcho`, and `main` prints the pointer `honcho-config-invalid` for it. `parse` + itself does not check (the serve tests build their `Config` with it); `honcho_config` is pure, + and later tasks call it again to get the `HonchoConfig`. +- **`trust` and `class` stay text** (`Option`) in `ReadSection`, so that a wrong value is + one of the reasons (R9, R10) with this runbook entry, not a serde error. Both are lower case + only: `"Trusted"` and `"Private"` are refused. +- **Honcho names** (M5c spec, section 6): Honcho accepts `^[a-zA-Z0-9_-]+$`, at most 512 + characters, for a workspace, a peer or a session. Checked on bytes: a non-ASCII letter such as + `é` fails, and so does any Unicode digit that is not `0` to `9`. +- **The URL.** `http://` or `https://` (lower case), a host of ASCII letters, digits, `.` and `-`, + and optionally `:` and a port from 1 to 65535 written in digits only (`u16::from_str` alone + would take a leading `+`). Nothing may follow: not even `/`, since `roncho` builds each request + URL from `/v3/...` and would drop a path silently. A user or password (`user:pw@host`) fails, + because the host would contain `@` or the port would not be digits. The reason never quotes the + URL. +- **Keys are checked, not read.** `SecretSpec::source` checks the table (exactly one of + `credential`, `env`, `file`; a well-formed name; an absolute path) without reading anything. + The given tests name environment variables that are not set and files that do not exist, and + expect `Ok`. Do not call `bxcore::secrets::load` or `read_checked`. +- **Where the check runs.** `main` loads the config first, before the instance lock and before + anything under the home is made (as it already does); a given test checks that `/run` and + `/memory` do not exist after an invalid section. +- **The given tests.** + - `honcho_config.rs` (8 tests): what a valid section gives, the defaults, the keys, the key + names and the pointer, parse errors, `state_socket`, and `load`. + - `honcho_reasons.rs` (5 tests): every reason of the table, exactly, one problem per section; + names and URLs at their edges; the first problem is the one reported. + - `honcho_start.rs` (4 tests): starts `memoryd serve` as a process (every wait has a 10-second + deadline; a guard kills and waits for the process before the home is removed). An invalid + section: exit 1 and exactly the two lines; an unknown key: still `memoryd-start-failed`; a + valid section with keys: `memoryd` serves as before. The two key values it passes (an + environment variable, a 0600 file) must appear in nothing `memoryd` prints. +- The calls the bodies and edits use: + + | Call | Signature, and what it gives | + |---|---| + | `str::strip_prefix` | `url.strip_prefix("https://") -> Option<&str>` | + | `str::split_once` | `rest.split_once(':') -> Option<(&str, &str)>`: split at the first `:` | + | `str::bytes` | `.bytes().all(\|b\| b.is_ascii_alphanumeric() \|\| b == b'-')` | + | `u8::is_ascii_alphanumeric`, `u8::is_ascii_digit` | `b.is_ascii_digit() -> bool` | + | `str::parse` | `port.parse::() -> Result` | + | `Option::as_deref` | `entry.trust.as_deref() -> Option<&str>`, to `match` on `Some("trusted")` | + | `slice::iter().any` | `earlier.iter().any(\|r\| r.agent == entry.agent) -> bool` | + | `SecretSpec::source` | `pub fn source(&self) -> Result` (`bxcore::secrets`) | + | `BTreeMap::insert` | `keys.insert(name.clone(), source)` | + | `check_name` (given) | `fn check_name(table: &str, key: &str, name: &str) -> Result<(), String>`: `Err` is R2 | + | `honcho::check` (given) | see Interfaces; `honcho_config` calls it (C7) | + +## The edits, exactly + +Nothing else in these files changes. Each quoted "before" block occurs exactly once in its file. + +### `crates/memoryd/Cargo.toml` + +**K1.** The line + +```toml +bxcore.workspace = true +``` + +becomes + +```toml +bxcore = { workspace = true, features = ["secrets"] } +``` + +### `crates/memoryd/src/lib.rs` + +**L1.** The line + +```rust +pub mod files; +``` + +becomes + +```rust +pub mod files; +pub mod honcho; +``` + +### `crates/memoryd/src/config.rs` + +**C1.** The line + +```rust +//! `memoryd` configuration: read `memoryd.toml` into a typed `Config` (M5a spec, section 11). +``` + +becomes + +```rust +//! `memoryd` configuration: read `memoryd.toml` into a typed `Config` (M5a spec, section 11), and +//! check its `[honcho]` section at start (M5c spec, section 10). +``` + +**C2.** The lines + +```rust +use std::path::{Path, PathBuf}; + +use bxcore::DataClass; +use serde::Deserialize; +``` + +become + +```rust +use std::collections::BTreeMap; +use std::path::{Path, PathBuf}; + +use bxcore::DataClass; +use bxcore::secrets::SecretSpec; +use serde::Deserialize; + +use crate::honcho::config::{self as honcho, HonchoConfig, HonchoSection}; +``` + +**C3.** In `Sockets`, the lines + +```rust + pub broker: PathBuf, + pub admin: PathBuf, +} +``` + +become + +```rust + pub broker: PathBuf, + pub admin: PathBuf, + /// `brokerd`'s `state.sock`, which `memoryd` connects to, to ask for a session's state (M5c + /// spec, section 7). `brokerd.toml` names the same path as its `[sockets] state`. + pub state: PathBuf, +} +``` + +**C4.** In `Config`, the lines + +```rust + pub sockets: Sockets, +} +``` + +become + +```rust + pub sockets: Sockets, + /// `[honcho]` as written; `honcho_config` checks it. Without it, Honcho is off. + pub honcho: Option, + /// `[secrets.]`: the Honcho keys, checked by `honcho_config`, never read here. + pub secrets: BTreeMap, +} +``` + +**C5.** The lines + +```rust + Parse(PathBuf, toml::de::Error), +} +``` + +become + +```rust + Parse(PathBuf, toml::de::Error), + /// `[honcho]` or a key for it is not allowed: the path, and `honcho_config`'s reason. + Honcho(PathBuf, String), +} +``` + +**C6.** In `impl std::fmt::Display for ConfigError`, the line + +```rust + ConfigError::Parse(path, err) => write!(f, "{}: {err}", path.display()), +``` + +becomes + +```rust + ConfigError::Parse(path, err) => write!(f, "{}: {err}", path.display()), + ConfigError::Honcho(path, why) => write!(f, "{}: {why}", path.display()), +``` + +**C7.** The lines + +```rust + /// Read and parse `path`; a relative `soul_starter` is taken to be next to it. + pub fn load(path: &Path) -> Result { + let text = + std::fs::read_to_string(path).map_err(|e| ConfigError::Read(path.to_path_buf(), e))?; + let mut config = + Config::parse(&text).map_err(|e| ConfigError::Parse(path.to_path_buf(), e))?; +``` + +become + +```rust + /// Read, parse and check `path`; a relative `soul_starter` is taken to be next to it. + pub fn load(path: &Path) -> Result { + let text = + std::fs::read_to_string(path).map_err(|e| ConfigError::Read(path.to_path_buf(), e))?; + let mut config = + Config::parse(&text).map_err(|e| ConfigError::Parse(path.to_path_buf(), e))?; + if let Err(why) = config.honcho_config() { + return Err(ConfigError::Honcho(path.to_path_buf(), why)); + } +``` + +and the line + +```rust + fn socket(&self, set: &Path, default: &str) -> PathBuf { +``` + +becomes + +```rust + /// `[honcho]` and its keys, checked: `Ok(None)` when there is no `[honcho]` and no key. + pub fn honcho_config(&self) -> Result, String> { + honcho::check(self.honcho.as_ref(), &self.secrets) + } + + fn socket(&self, set: &Path, default: &str) -> PathBuf { +``` + +**C8.** The lines + +```rust + self.socket(&self.sockets.admin, "run/owner-memory/admin.sock") + } +``` + +become + +```rust + self.socket(&self.sockets.admin, "run/owner-memory/admin.sock") + } + pub fn state_socket(&self) -> PathBuf { + self.socket(&self.sockets.state, "run/memory-broker/state.sock") + } +``` + +### `crates/memoryd/src/main.rs` + +**M1.** The line + +```rust +use memoryd::config::Config; +``` + +becomes + +```rust +use memoryd::config::{Config, ConfigError}; +use memoryd::honcho::config::RUNBOOK as HONCHO_INVALID; +``` + +**M2.** The lines + +```rust + let config = match Config::load(&path) { + Ok(config) => config, + Err(e) => { + eprintln!("memoryd: {e}\n{START_FAILED}"); + return ExitCode::from(1); + } +``` + +become + +```rust + let config = match Config::load(&path) { + Ok(config) => config, + Err(e) => { + let pointer = match &e { + ConfigError::Read(..) | ConfigError::Parse(..) => START_FAILED, + ConfigError::Honcho(..) => HONCHO_INVALID, + }; + eprintln!("memoryd: {e}\n{pointer}"); + return ExitCode::from(1); + } +``` + +### `docs/dependencies.md` + +**D1.** The line (the `zeroize` row) + +```text +| `zeroize` | 1.9.0 | `bxcore`, feature `secrets` (enabled by `gatewayd`; never by `loopd`) | Wipes a secret's memory when it is dropped (`Secret`). Already a dependency of `rustls`. Apache-2.0 OR MIT. | +``` + +becomes + +```text +| `zeroize` | 1.9.0 | `bxcore`, feature `secrets` (enabled by `gatewayd` and `memoryd`; never by `loopd`) | Wipes a secret's memory when it is dropped (`Secret`). Already a dependency of `rustls`. Apache-2.0 OR MIT. | +``` + +## Steps + +- [ ] **1. Copy the given tests.** From the repository root, three commands: + + ```sh + cp docs/plans/M5c1/files/crates/memoryd/tests/honcho_config.rs crates/memoryd/tests/ + cp docs/plans/M5c1/files/crates/memoryd/tests/honcho_reasons.rs crates/memoryd/tests/ + cp docs/plans/M5c1/files/crates/memoryd/tests/honcho_start.rs crates/memoryd/tests/ + ``` + +- [ ] **2. See them fail.** Two commands, each with a shell timeout of 600000 ms: + - `cargo build --tests --keep-going -p memoryd`: two test targets do not compile. + `honcho_config`, with 8 errors: one `E0433` (``cannot find `honcho` in `memoryd` ``), one + `E0432` (``unresolved import `bxcore::secrets` ``: the feature is not on yet), two `E0599` + for `honcho_config`, three `E0599` for `state_socket` (``no method named … found for struct + `Config` ``) and one `E0599` for the variant `Honcho` of `ConfigError`. `honcho_reasons`, with + 2: the same `E0433`, and one `E0599` for `honcho_config`. `honcho_start` compiles. + - `cargo test -p memoryd --test honcho_start`: it compiles, and its 4 tests fail (about 10 s: + one waits out its deadline). Each `memoryd` stopped with + ``unknown field `honcho`, expected one of `paths`, `memory`, `index`, `sockets` `` and the + pointer `memoryd-start-failed`: + + | Test | Before this task | + |---|---| + | `an_invalid_section_stops_memoryd_with_its_reason_and_the_pointer` | `assertion left == right failed`: the parse error where the reason was wanted | + | `a_key_for_an_agent_not_read_stops_memoryd` | the same | + | `an_unknown_key_in_the_section_is_still_a_start_failure` | the stderr, printed as the message: it names `honcho`, not `fleat` | + | `a_valid_section_with_keys_starts_memoryd_as_before` | `memoryd never answered; stderr: …` after 10 s | +- [ ] **3. The module.** Make edits K1 and L1. Create `crates/memoryd/src/honcho/mod.rs` with + exactly this content: + + ```rust + //! Honcho, the memory this agent shares with the fleet, reached through `roncho` (M5c spec). + + pub mod config; + ``` + + Then copy the skeleton: + + ```sh + cp docs/plans/M5c1/files/crates/memoryd/src/honcho/config.rs crates/memoryd/src/honcho/ + ``` + + `cargo check -p memoryd`: no error, and 13 warnings, all from the unfilled bodies: ten + `unused variable` (`name`, `url`, `section` twice, `entry` three times, `earlier`, `secrets`, + `read`), ``constant `URL_REASON` is never used``, and ``function `trust_of` is never used`` + and the same for `class_of`. They go as you fill the bodies. +- [ ] **4. Fill the seven bodies of `honcho/config.rs`, one at a time, in this order**, each from + its comment and the table "The reasons", and run `cargo check -p memoryd` after each: `is_name`, + `check_url`, `check_own_workspaces`, `trust_of`, `class_of`, `read_workspace`, `check_keys`. + When a body is filled, delete its instruction comment (the `//` lines inside the body) and + keep the function's doc comment (`///`) as it is. Change nothing else in the file. After the + seventh, `cargo check -p memoryd` prints no warning. +- [ ] **5. The configuration.** Make edits C1 to C8 in `config.rs`, then M1 and M2 in `main.rs`. + `cargo check -p memoryd`: no error and no warning. Then this command, and every test passes + (8, 5, 4 and 4): + + ```sh + cargo test -p memoryd --test honcho_config --test honcho_reasons --test honcho_start \ + --test config + ``` +- [ ] **6. The runbook and the dependency row.** From the repository root: + + ```sh + { echo; cat docs/plans/M5c1/files/runbook-honcho-config-invalid.md; } >> docs/runbook.md + ``` + + Then `grep -c '^## honcho-config-invalid$' docs/runbook.md` prints `1`. Make edit D1. +- [ ] **7. Check.** Four commands: + - `cargo build -p memoryd`: it builds (it needs K1: see "The feature `secrets`"). + - `grep -rn 'runbook.md#honcho-config-invalid' crates/memoryd/src` prints exactly one line, + the `RUNBOOK` constant in `honcho/config.rs`. + - `grep -n 'secrets::load\|read_checked' crates/memoryd/src -r` prints nothing: no key is read. + - `git diff --stat -- crates/bxcore crates/loopd crates/brokerd crates/gatewayd crates/bxctl` + prints nothing. +- [ ] **8. Run the gate.** `cargo fmt --all`, then `make gate`, with a shell timeout of 600000 ms. + Expected last line: `gate: ok`, with 1188 tests: 1171 before this task, plus 17 new ones (8 in + `honcho_config.rs`, 5 in `honcho_reasons.rs`, 4 in `honcho_start.rs`). +- [ ] **9. Log and commit.** Add your row to `docs/implementer-log.md`; its Task cell is + `M5c1/06-memoryd-honcho-config`. `git status --short` lists `crates/memoryd/Cargo.toml`, + `crates/memoryd/src/lib.rs`, `crates/memoryd/src/config.rs`, `crates/memoryd/src/main.rs`, + `docs/runbook.md`, `docs/dependencies.md` and your log as modified, and + `crates/memoryd/src/honcho/` and the three test files as new. Stage with + `git add crates/memoryd docs/runbook.md docs/dependencies.md docs/implementer-log.md Cargo.lock`, + then commit as `AGENTS.md` says. `Cargo.lock` does not change in this task; the `git add` is + harmless. + +## Done when + +- `make gate` prints `gate: ok` with 1188 tests. +- `git diff HEAD~1 --stat` lists only the four modified `memoryd` files, `honcho/mod.rs`, + `honcho/config.rs`, the three new test files, `docs/runbook.md`, `docs/dependencies.md` and + `docs/implementer-log.md`. + +## Stop and report if + +- A "before" block of K1, L1, C1 to C8, M1, M2 or D1 is not in its file exactly as shown, or is + there more than once. +- Step 2 or 3 shows a different result: other test targets, errors of another kind, other + failing tests, or other warnings. +- `cargo check` reports an error after a body you filled or an edit you made, and the comment or + the edit does not tell you how to fix it. Quote the error, and stop. +- A comment in the skeleton does not fit the code under it, or a reason in the table does not fit + the comment that names it. Quote both and stop. +- After step 4 or 5 a test fails although each body follows its comment and each edit is as + shown. Do not change a test or the skeleton's given code: stop, and give the failing tests' + names and their messages. +- The fix seems to need a change anywhere else: `serve.rs`, another crate, the test support + files, reading a key, or a check the table does not list. It does not; stop and say why you + think so. +- A test outside these three files fails in the gate: check once whether it fails without your + change, then stop and report both results. +- The gate passes with a count other than 1188. diff --git a/docs/plans/M5c1/files/crates/memoryd/src/honcho/config.rs b/docs/plans/M5c1/files/crates/memoryd/src/honcho/config.rs new file mode 100644 index 0000000..fafcb56 --- /dev/null +++ b/docs/plans/M5c1/files/crates/memoryd/src/honcho/config.rs @@ -0,0 +1,220 @@ +//! `[honcho]` in `memoryd.toml` and its keys in `[secrets.]`, checked at start into a +//! `HonchoConfig` (M5c spec, sections 3, 6 and 10). Nothing here reaches Honcho or reads a key: a +//! key's source is checked here and read only when a client is made. + +use std::collections::BTreeMap; + +use bxcore::DataClass; +use bxcore::secrets::{SecretSource, SecretSpec}; +use serde::Deserialize; + +/// The runbook entry for every reason `check` gives. +pub const RUNBOOK: &str = "see docs/runbook.md#honcho-config-invalid"; + +/// The longest name Honcho accepts for a workspace, a peer or a session (M5c spec, section 6). +pub const MAX_NAME: usize = 512; + +/// The names of the keys in `[secrets.]`, one per workspace. A read workspace's key is +/// `READ_KEY` and its agent's name (`read_key`). +pub const TRUSTED_KEY: &str = "honcho_trusted"; +pub const UNTRUSTED_KEY: &str = "honcho_untrusted"; +pub const FLEET_KEY: &str = "honcho_fleet"; +pub const READ_KEY: &str = "honcho_read_"; + +/// `[honcho]` as written. Our own format: unknown keys are errors. +#[derive(Debug, Clone, PartialEq, Eq, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct HonchoSection { + pub url: String, + pub agent: String, + pub owner: String, + pub trusted: String, + pub untrusted: String, + pub fleet: String, + #[serde(default)] + pub read: Vec, +} + +/// One `[[honcho.read]]` as written. `trust` and `class` stay text until `check`, so that a wrong +/// value is one of its reasons. +#[derive(Debug, Clone, PartialEq, Eq, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct ReadSection { + pub agent: String, + pub workspace: String, + #[serde(default)] + pub trust: Option, + #[serde(default)] + pub class: Option, +} + +/// `[honcho]` and its keys, checked: every name is a Honcho name, and the names fit together. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct HonchoConfig { + /// `http://` or `https://`, a host and an optional port. + pub url: String, + /// This agent's peer. + pub agent: String, + /// The owner's peer, the same in every workspace of the fleet. + pub owner: String, + /// This agent's three workspaces. + pub trusted: String, + pub untrusted: String, + pub fleet: String, + /// Other agents' workspaces this agent reads, in the order written. + pub read: Vec, + /// Every `[secrets.]`, by its name: a source that is checked, never read here. + pub keys: BTreeMap, +} + +/// One `[[honcho.read]]`, checked. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ReadWorkspace { + pub agent: String, + pub workspace: String, + pub trust: Trust, + pub class: DataClass, +} + +/// How the owner labelled another agent's workspace. `Untrusted` unless written out. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum Trust { + Trusted, + Untrusted, +} + +/// The name of the key for the workspace of the read agent `agent`. +pub fn read_key(agent: &str) -> String { + format!("{READ_KEY}{agent}") +} + +/// True when `name` is a Honcho name: 1 to `MAX_NAME` characters, each an ASCII letter, a digit, +/// `_` or `-`. +pub fn is_name(name: &str) -> bool { + // Not empty, `name.len() <= MAX_NAME`, and every byte of `name.bytes()` is + // `is_ascii_alphanumeric()`, `b'_'` or `b'-'`. Bytes, not chars: a non-ASCII letter is refused. + todo!() +} + +/// Check `[honcho]` and the keys in `[secrets]`. No section and no key is `Ok(None)`: Honcho is +/// off. `Err` is the first reason found, one line that names the table and never a key's value. +pub fn check( + section: Option<&HonchoSection>, + secrets: &BTreeMap, +) -> Result, String> { + let Some(section) = section else { + if let Some(name) = secrets.keys().next() { + return Err(format!( + "[secrets.{name}] is set, but there is no [honcho] section" + )); + } + return Ok(None); + }; + check_url(§ion.url)?; + for (key, name) in [ + ("agent", §ion.agent), + ("owner", §ion.owner), + ("trusted", §ion.trusted), + ("untrusted", §ion.untrusted), + ("fleet", §ion.fleet), + ] { + check_name("[honcho]", key, name)?; + } + if section.agent == section.owner { + return Err(format!( + "[honcho] agent and owner are both {:?}: the agent's peer must not be the owner's", + section.agent + )); + } + check_own_workspaces(section)?; + let mut read: Vec = Vec::new(); + for entry in §ion.read { + let checked = read_workspace(section, entry, &read)?; + read.push(checked); + } + let keys = check_keys(secrets, &read)?; + Ok(Some(HonchoConfig { + url: section.url.clone(), + agent: section.agent.clone(), + owner: section.owner.clone(), + trusted: section.trusted.clone(), + untrusted: section.untrusted.clone(), + fleet: section.fleet.clone(), + read, + keys, + })) +} + +/// `Err` with the reason when `name`, the value of `key` in `table`, is not a Honcho name. +fn check_name(table: &str, key: &str, name: &str) -> Result<(), String> { + if is_name(name) { + return Ok(()); + } + Err(format!( + "{table} {key} {name:?} is not a Honcho name: 1 to {MAX_NAME} of a-z, A-Z, 0-9, _ and -" + )) +} + +/// `http://` or `https://`, then a host, then optionally `:` and a port; nothing else. +fn check_url(url: &str) -> Result<(), String> { + // 1. `url.strip_prefix("https://")`, else `url.strip_prefix("http://")`, else the error below. + // 2. What is left, `split_once(':')`: the host before the first `:` and the port after it. + // 3. The host: not empty, every byte `is_ascii_alphanumeric()`, `b'.'` or `b'-'`. + // 4. A port, if there is a `:`: not empty, every byte `is_ascii_digit()`, and + // `port.parse::()` is `Ok(n)` with `n > 0`. + // Both good: `Ok(())`. Anything else: `Err(URL_REASON.to_string())`. + todo!() +} + +/// The reason for any `url` that `check_url` refuses. It never quotes the URL, which could hold a +/// password (`http://user:password@host`). +const URL_REASON: &str = + "[honcho] url must be http:// or https://, a host and an optional port, and nothing else"; + +/// `Err` when two of this agent's three workspaces have the same name. +fn check_own_workspaces(section: &HonchoSection) -> Result<(), String> { + // The pairs (trusted, untrusted), (trusted, fleet), (untrusted, fleet), in that order: the first + // pair whose two names are equal is `Err` with reason R5 and that name. None equal: `Ok(())`. + todo!() +} + +/// One `[[honcho.read]]`, checked against `[honcho]` and the entries before it, `earlier`. +fn read_workspace( + section: &HonchoSection, + entry: &ReadSection, + earlier: &[ReadWorkspace], +) -> Result { + // In this order, `Err` at the first that fails: `check_name("[[honcho.read]]", "agent", ..)`, + // then the same for "workspace" (R2); the agent is `section.agent` (R6); an entry of `earlier` + // has the same agent (R7); the workspace is `section.trusted`, `.untrusted` or `.fleet` (R8). + // Then `Ok(ReadWorkspace { .. })`: both names cloned, `trust: trust_of(entry)?` and + // `class: class_of(entry)?`. + todo!() +} + +/// The entry's `trust`: `Untrusted` when it is not written. +fn trust_of(entry: &ReadSection) -> Result { + // `match entry.trust.as_deref()`: `None` or `Some("untrusted")` is `Trust::Untrusted`, + // `Some("trusted")` is `Trust::Trusted`, and any other text (case counts) is reason R9. + todo!() +} + +/// The entry's `class`: `Private` when it is not written. +fn class_of(entry: &ReadSection) -> Result { + // `match entry.class.as_deref()`: `None` or `Some("private")` is `DataClass::Private`, + // `"public"` and `"secret"` are `Public` and `Secret`, any other text (case counts) is R10. + todo!() +} + +/// Every `[secrets.]`: a Honcho key's name, and a source that `SecretSpec::source` accepts. +fn check_keys( + secrets: &BTreeMap, + read: &[ReadWorkspace], +) -> Result, String> { + // For each `(name, spec)` of `secrets`, in the map's order: a name other than `TRUSTED_KEY`, + // `UNTRUSTED_KEY` and `FLEET_KEY` must be `READ_KEY` and the agent of an entry of `read` + // (`name.strip_prefix(READ_KEY)`); with the prefix and any other agent it is R11, without the + // prefix R12. Then `spec.source()`: `Err(why)` is R13; `Ok(source)` goes into a new + // `BTreeMap` under `name.clone()`. Return the map. + todo!() +} diff --git a/docs/plans/M5c1/files/crates/memoryd/tests/honcho_config.rs b/docs/plans/M5c1/files/crates/memoryd/tests/honcho_config.rs new file mode 100644 index 0000000..6030b63 --- /dev/null +++ b/docs/plans/M5c1/files/crates/memoryd/tests/honcho_config.rs @@ -0,0 +1,229 @@ +//! `[honcho]` in `memoryd.toml`, its `[[honcho.read]]` list and its keys, checked at start (M5c +//! spec, section 10), and `[sockets] state`, `brokerd`'s session-state socket: what a valid +//! section gives, and how `load` refuses an invalid one. The reasons are in `honcho_reasons.rs`. +//! Do not edit. + +#[path = "support/tmp.rs"] +mod tmp; + +use std::collections::BTreeMap; +use std::path::PathBuf; + +use bxcore::DataClass; +use bxcore::secrets::SecretSource; +use memoryd::config::{Config, ConfigError}; +use memoryd::honcho::config::{ + FLEET_KEY, HonchoConfig, MAX_NAME, READ_KEY, RUNBOOK, ReadWorkspace, TRUSTED_KEY, Trust, + UNTRUSTED_KEY, read_key, +}; +use tmp::TempDir; + +/// The smallest valid section: the six keys, no read workspace, no key. +const SECTION: &str = "[honcho] +url = \"http://rift.scylla-hammerhead.ts.net:8000\" +agent = \"boxmaker0\" +owner = \"kyle\" +trusted = \"boxmaker0-trusted\" +untrusted = \"boxmaker0-untrusted\" +fleet = \"fleet\" +"; + +/// Two read workspaces: the first with the defaults, the second with both written out. +const READS: &str = "[[honcho.read]] +agent = \"boxmaker1\" +workspace = \"boxmaker1-trusted\" +[[honcho.read]] +agent = \"boxmaker2\" +workspace = \"boxmaker2-trusted\" +trust = \"trusted\" +class = \"secret\" +"; + +/// `SECTION` with `from` (found exactly once) replaced by `to`, then `extra`. +fn with(from: &str, to: &str, extra: &str) -> String { + assert_eq!(SECTION.matches(from).count(), 1, "{from}"); + format!("{}{extra}", SECTION.replace(from, to)) +} + +/// The checked section of `text`, which must parse. +fn checked(text: &str) -> Result, String> { + let config = Config::parse(text).unwrap_or_else(|e| panic!("{text}\n{e}")); + config.honcho_config() +} + +#[test] +fn without_a_honcho_section_it_is_none() { + for text in [ + "", + "[paths]\nhome = \"/srv/bx\"\n", + "[memory]\nreview = true\n[sockets]\nloop = \"/r/l.sock\"\n", + ] { + assert_eq!(checked(text), Ok(None), "{text}"); + } +} + +#[test] +fn the_smallest_section_is_checked_as_written() { + let honcho = checked(SECTION).unwrap().unwrap(); + assert_eq!( + honcho, + HonchoConfig { + url: "http://rift.scylla-hammerhead.ts.net:8000".to_string(), + agent: "boxmaker0".to_string(), + owner: "kyle".to_string(), + trusted: "boxmaker0-trusted".to_string(), + untrusted: "boxmaker0-untrusted".to_string(), + fleet: "fleet".to_string(), + read: Vec::new(), + keys: BTreeMap::new(), + } + ); +} + +#[test] +fn read_workspaces_default_to_untrusted_and_private_and_keep_their_order() { + let honcho = checked(&format!("{SECTION}{READS}")).unwrap().unwrap(); + assert_eq!( + honcho.read, + vec![ + ReadWorkspace { + agent: "boxmaker1".to_string(), + workspace: "boxmaker1-trusted".to_string(), + trust: Trust::Untrusted, + class: DataClass::Private, + }, + ReadWorkspace { + agent: "boxmaker2".to_string(), + workspace: "boxmaker2-trusted".to_string(), + trust: Trust::Trusted, + class: DataClass::Secret, + }, + ] + ); + let written = "[[honcho.read]]\nagent = \"b1\"\nworkspace = \"w1\"\ntrust = \"untrusted\"\nclass = \"public\"\n[[honcho.read]]\nagent = \"b2\"\nworkspace = \"w2\"\nclass = \"private\"\n"; + let honcho = checked(&format!("{SECTION}{written}")).unwrap().unwrap(); + let labels: Vec<(Trust, DataClass)> = honcho.read.iter().map(|r| (r.trust, r.class)).collect(); + assert_eq!( + labels, + vec![ + (Trust::Untrusted, DataClass::Public), + (Trust::Untrusted, DataClass::Private) + ] + ); +} + +#[test] +fn every_key_is_checked_by_its_name_and_never_read() { + // None of these exists: a key is checked here, and read only when a client is made. + let keys = "[secrets.honcho_trusted]\nenv = \"BOXMAKER_TEST_NO_SUCH_VARIABLE\"\n\ + [secrets.honcho_untrusted]\nfile = \"/nonexistent/boxmaker/untrusted.key\"\n\ + [secrets.honcho_fleet]\ncredential = \"honcho-fleet\"\n\ + [secrets.honcho_read_boxmaker2]\nenv = \"HONCHO_READ_2\"\n"; + let honcho = checked(&format!("{SECTION}{READS}{keys}")) + .unwrap() + .unwrap(); + let expected: BTreeMap = [ + ( + "honcho_trusted", + SecretSource::Env("BOXMAKER_TEST_NO_SUCH_VARIABLE".to_string()), + ), + ( + "honcho_untrusted", + SecretSource::File(PathBuf::from("/nonexistent/boxmaker/untrusted.key")), + ), + ( + "honcho_fleet", + SecretSource::Credential("honcho-fleet".to_string()), + ), + ( + "honcho_read_boxmaker2", + SecretSource::Env("HONCHO_READ_2".to_string()), + ), + ] + .into_iter() + .map(|(name, source)| (name.to_string(), source)) + .collect(); + assert_eq!(honcho.keys, expected); + assert_eq!(honcho.read.len(), 2, "the keys change nothing else"); +} + +#[test] +fn the_key_names_and_the_pointer() { + assert_eq!(TRUSTED_KEY, "honcho_trusted"); + assert_eq!(UNTRUSTED_KEY, "honcho_untrusted"); + assert_eq!(FLEET_KEY, "honcho_fleet"); + assert_eq!(READ_KEY, "honcho_read_"); + assert_eq!(read_key("boxmaker1"), "honcho_read_boxmaker1"); + assert_eq!(RUNBOOK, "see docs/runbook.md#honcho-config-invalid"); + assert_eq!(MAX_NAME, 512); +} + +#[test] +fn unknown_keys_and_wrong_types_are_parse_errors() { + for text in [ + with("fleet = \"fleet\"", "fleet = \"fleet\"\nfleat = \"x\"", ""), + format!("{SECTION}[[honcho.read]]\nagent = \"b1\"\nworkspace = \"w1\"\nlabel = \"x\"\n"), + format!("{SECTION}[secrets.honcho_trusted]\nenv = \"K\"\nvalue = \"x\"\n"), + format!("{SECTION}[honcho.read]\nagent = \"b1\"\nworkspace = \"w1\"\n"), + format!("{SECTION}[[honcho.read]]\nagent = \"b1\"\nworkspace = \"w1\"\ntrust = true\n"), + format!("{SECTION}[[honcho.read]]\nagent = \"b1\"\n"), + format!("{SECTION}[sockets]\nstates = \"/x\"\n"), + with("owner = \"kyle\"\n", "", ""), + with("agent = \"boxmaker0\"", "agent = 0", ""), + "[secrets]\nhoncho_trusted = \"HONCHO_KEY\"\n".to_string(), + ] { + assert!(Config::parse(&text).is_err(), "{text}"); + } +} + +#[test] +fn the_state_socket_is_under_home_unless_set() { + let config = Config::parse("[paths]\nhome = \"/srv/bx\"\n").unwrap(); + assert_eq!( + config.state_socket(), + PathBuf::from("/srv/bx/run/memory-broker/state.sock") + ); + let config = + Config::parse("[paths]\nhome = \"/h\"\n[sockets]\nstate = \"/r/s.sock\"\n").unwrap(); + assert_eq!(config.state_socket(), PathBuf::from("/r/s.sock")); + let config = Config::parse("[paths]\nhome = \"/h\"\n[sockets]\nstate = \"\"\n").unwrap(); + assert_eq!( + config.state_socket(), + PathBuf::from("/h/run/memory-broker/state.sock") + ); + // The other three do not move. + assert_eq!( + config.loop_socket(), + PathBuf::from("/h/run/loop-memory/memory.sock") + ); +} + +#[test] +fn load_refuses_an_invalid_section_and_keeps_a_valid_one() { + let dir = TempDir::new("honcho-load"); + let good = dir.write("good.toml", &format!("{SECTION}{READS}")); + let config = Config::load(&good).unwrap(); + assert_eq!(config.honcho_config().unwrap().unwrap().read.len(), 2); + + let bad = dir.write( + "bad.toml", + &with("owner = \"kyle\"", "owner = \"boxmaker0\"", ""), + ); + let err = Config::load(&bad).unwrap_err(); + let why = + "[honcho] agent and owner are both \"boxmaker0\": the agent's peer must not be the owner's"; + assert!( + matches!(&err, ConfigError::Honcho(path, reason) if path == &bad && reason == why), + "{err:?}" + ); + assert_eq!(err.to_string(), format!("{}: {why}", bad.display())); + + let unknown = dir.write( + "unknown.toml", + &with("fleet = \"fleet\"", "fleet = \"fleet\"\nfleat = \"x\"", ""), + ); + assert!( + matches!(Config::load(&unknown), Err(ConfigError::Parse(..))), + "an unknown key stays a parse error" + ); +} diff --git a/docs/plans/M5c1/files/crates/memoryd/tests/honcho_reasons.rs b/docs/plans/M5c1/files/crates/memoryd/tests/honcho_reasons.rs new file mode 100644 index 0000000..96063fd --- /dev/null +++ b/docs/plans/M5c1/files/crates/memoryd/tests/honcho_reasons.rs @@ -0,0 +1,338 @@ +//! Every reason an invalid `[honcho]` section, `[[honcho.read]]` or Honcho key is refused with at +//! start (M5c spec, sections 6 and 10): each invalid section has one problem and one exact reason. +//! Do not edit. + +use memoryd::config::Config; +use memoryd::honcho::config::{HonchoConfig, is_name}; + +/// The smallest valid section: the six keys, no read workspace, no key. +const SECTION: &str = "[honcho] +url = \"http://rift.scylla-hammerhead.ts.net:8000\" +agent = \"boxmaker0\" +owner = \"kyle\" +trusted = \"boxmaker0-trusted\" +untrusted = \"boxmaker0-untrusted\" +fleet = \"fleet\" +"; + +const URL_REASON: &str = + "[honcho] url must be http:// or https://, a host and an optional port, and nothing else"; + +/// `SECTION` with `from` (found exactly once) replaced by `to`, then `extra`. +fn with(from: &str, to: &str, extra: &str) -> String { + assert_eq!(SECTION.matches(from).count(), 1, "{from}"); + format!("{}{extra}", SECTION.replace(from, to)) +} + +/// The checked section of `text`, which must parse. +fn checked(text: &str) -> Result, String> { + let config = Config::parse(text).unwrap_or_else(|e| panic!("{text}\n{e}")); + config.honcho_config() +} + +fn reason(text: &str) -> String { + match checked(text) { + Ok(ok) => panic!("accepted:\n{text}\n{ok:?}"), + Err(why) => why, + } +} + +fn name_reason(table: &str, key: &str, name: &str) -> String { + format!("{table} {key} {name:?} is not a Honcho name: 1 to 512 of a-z, A-Z, 0-9, _ and -") +} + +#[test] +fn honcho_names_at_the_edges() { + let longest = "a".repeat(512); + for name in [ + "a", + "Z", + "0", + "_", + "-", + "boxmaker0-trusted", + "A_b-9", + longest.as_str(), + ] { + assert!(is_name(name), "{name}"); + } + let too_long = "a".repeat(513); + for name in [ + "", + too_long.as_str(), + "boxmaker.0", + "box maker", + "a/b", + "a:b", + "kyle@home", + "caf\u{e9}", + "tab\t", + "new\nline", + "\u{0660}", + ] { + assert!(!is_name(name), "{name:?}"); + } + // The longest name is accepted in every place a name goes. + let text = with("fleet = \"fleet\"", &format!("fleet = \"{longest}\""), ""); + assert_eq!(checked(&text).unwrap().unwrap().fleet, longest); + let text = with("owner = \"kyle\"", &format!("owner = \"{too_long}\""), ""); + assert_eq!(reason(&text), name_reason("[honcho]", "owner", &too_long)); +} + +#[test] +fn urls_that_are_accepted() { + for url in [ + "http://rift.scylla-hammerhead.ts.net:8000", + "https://rift.scylla-hammerhead.ts.net", + "http://127.0.0.1:1", + "http://honcho:65535", + "https://Rift-2.Example.org:443", + "http://h", + ] { + let text = with( + "url = \"http://rift.scylla-hammerhead.ts.net:8000\"", + &format!("url = \"{url}\""), + "", + ); + assert_eq!(checked(&text).unwrap().unwrap().url, url); + } +} + +#[test] +fn urls_that_are_refused_and_the_reason_never_quotes_them() { + for url in [ + "", + "rift:8000", + "ftp://rift:8000", + "HTTP://rift:8000", + "http:/rift", + "http://", + "https://", + "http://:8000", + "http://rift:", + "http://rift:0", + "http://rift:65536", + "http://rift:+80", + "http://rift:80:90", + "http://rift:8000/", + "http://rift:8000/v3", + "http://rift/", + "http://kyle:hunter2@rift:8000", + "http://kyle@rift:8000", + "http://kyle@rift", + "http://rift?x=1", + "http://rift#top", + "http://ri ft", + " http://rift", + "http://rift_1", + "http://[::1]:8000", + ] { + let text = with( + "url = \"http://rift.scylla-hammerhead.ts.net:8000\"", + &format!("url = \"{url}\""), + "", + ); + assert_eq!(reason(&text), URL_REASON, "{url}"); + } +} + +#[test] +fn each_invalid_section_gives_its_reason() { + let read = |agent: &str, workspace: &str| { + format!("[[honcho.read]]\nagent = \"{agent}\"\nworkspace = \"{workspace}\"\n") + }; + let cases: Vec<(String, String)> = vec![ + ( + with("agent = \"boxmaker0\"", "agent = \"boxmaker.0\"", ""), + name_reason("[honcho]", "agent", "boxmaker.0"), + ), + ( + with("owner = \"kyle\"", "owner = \"\"", ""), + name_reason("[honcho]", "owner", ""), + ), + ( + with( + "trusted = \"boxmaker0-trusted\"", + "trusted = \"boxmaker0 trusted\"", + "", + ), + name_reason("[honcho]", "trusted", "boxmaker0 trusted"), + ), + ( + with( + "untrusted = \"boxmaker0-untrusted\"", + "untrusted = \"boxmaker0/untrusted\"", + "", + ), + name_reason("[honcho]", "untrusted", "boxmaker0/untrusted"), + ), + ( + with("fleet = \"fleet\"", "fleet = \"fleet:all\"", ""), + name_reason("[honcho]", "fleet", "fleet:all"), + ), + ( + format!("{SECTION}{}", read("boxmaker.1", "boxmaker1-trusted")), + name_reason("[[honcho.read]]", "agent", "boxmaker.1"), + ), + ( + format!("{SECTION}{}", read("boxmaker1", "boxmaker1.trusted")), + name_reason("[[honcho.read]]", "workspace", "boxmaker1.trusted"), + ), + ( + with("owner = \"kyle\"", "owner = \"boxmaker0\"", ""), + "[honcho] agent and owner are both \"boxmaker0\": the agent's peer must not be the owner's" + .to_string(), + ), + ( + with( + "untrusted = \"boxmaker0-untrusted\"", + "untrusted = \"boxmaker0-trusted\"", + "", + ), + "[honcho] workspace \"boxmaker0-trusted\" is named twice: trusted, untrusted and fleet must be three different workspaces" + .to_string(), + ), + ( + with("fleet = \"fleet\"", "fleet = \"boxmaker0-trusted\"", ""), + "[honcho] workspace \"boxmaker0-trusted\" is named twice: trusted, untrusted and fleet must be three different workspaces" + .to_string(), + ), + ( + with("fleet = \"fleet\"", "fleet = \"boxmaker0-untrusted\"", ""), + "[honcho] workspace \"boxmaker0-untrusted\" is named twice: trusted, untrusted and fleet must be three different workspaces" + .to_string(), + ), + ( + format!("{SECTION}{}", read("boxmaker0", "boxmaker1-trusted")), + "[[honcho.read]] agent \"boxmaker0\" is this agent ([honcho] agent)".to_string(), + ), + ( + format!( + "{SECTION}{}{}", + read("boxmaker1", "boxmaker1-trusted"), + read("boxmaker1", "boxmaker1-other") + ), + "[[honcho.read]] agent \"boxmaker1\" is listed twice".to_string(), + ), + ( + format!("{SECTION}{}", read("boxmaker1", "boxmaker0-trusted")), + "[[honcho.read]] workspace \"boxmaker0-trusted\" is one of this agent's own" + .to_string(), + ), + ( + format!("{SECTION}{}", read("boxmaker1", "boxmaker0-untrusted")), + "[[honcho.read]] workspace \"boxmaker0-untrusted\" is one of this agent's own" + .to_string(), + ), + ( + format!("{SECTION}{}", read("boxmaker1", "fleet")), + "[[honcho.read]] workspace \"fleet\" is one of this agent's own".to_string(), + ), + ( + format!( + "{SECTION}{}trust = \"Trusted\"\n", + read("boxmaker1", "boxmaker1-trusted") + ), + "[[honcho.read]] trust \"Trusted\" of agent \"boxmaker1\" is not \"trusted\" or \"untrusted\"" + .to_string(), + ), + ( + format!( + "{SECTION}{}trust = \"\"\n", + read("boxmaker1", "boxmaker1-trusted") + ), + "[[honcho.read]] trust \"\" of agent \"boxmaker1\" is not \"trusted\" or \"untrusted\"" + .to_string(), + ), + ( + format!( + "{SECTION}{}class = \"top-secret\"\n", + read("boxmaker1", "boxmaker1-trusted") + ), + "[[honcho.read]] class \"top-secret\" of agent \"boxmaker1\" is not \"public\", \"private\" or \"secret\"" + .to_string(), + ), + ( + format!( + "{SECTION}{}class = \"Private\"\n", + read("boxmaker1", "boxmaker1-trusted") + ), + "[[honcho.read]] class \"Private\" of agent \"boxmaker1\" is not \"public\", \"private\" or \"secret\"" + .to_string(), + ), + ( + format!("{SECTION}[secrets.honcho_trust]\nenv = \"HONCHO_KEY\"\n"), + "[secrets.honcho_trust] is not a Honcho key: memoryd has only honcho_trusted, honcho_untrusted, honcho_fleet and honcho_read_" + .to_string(), + ), + ( + format!("{SECTION}[secrets.mattermost_token]\nenv = \"MM_TOKEN\"\n"), + "[secrets.mattermost_token] is not a Honcho key: memoryd has only honcho_trusted, honcho_untrusted, honcho_fleet and honcho_read_" + .to_string(), + ), + ( + format!( + "{SECTION}{}[secrets.honcho_read_boxmaker2]\nenv = \"HONCHO_KEY\"\n", + read("boxmaker1", "boxmaker1-trusted") + ), + "[secrets.honcho_read_boxmaker2] is the key of agent \"boxmaker2\", which is not in [[honcho.read]]" + .to_string(), + ), + ( + format!("{SECTION}[secrets.honcho_read_boxmaker0]\nenv = \"HONCHO_KEY\"\n"), + "[secrets.honcho_read_boxmaker0] is the key of agent \"boxmaker0\", which is not in [[honcho.read]]" + .to_string(), + ), + ( + format!("{SECTION}[secrets.honcho_trusted]\n"), + "[secrets.honcho_trusted] needs exactly one of credential, env and file".to_string(), + ), + ( + format!( + "{SECTION}[secrets.honcho_fleet]\nenv = \"HONCHO_KEY\"\nfile = \"/etc/honcho.key\"\n" + ), + "[secrets.honcho_fleet] needs exactly one of credential, env and file".to_string(), + ), + ( + format!("{SECTION}[secrets.honcho_untrusted]\nenv = \"honcho_key\"\n"), + "[secrets.honcho_untrusted] env \"honcho_key\" is not a variable name (A-Z, 0-9, _)" + .to_string(), + ), + ( + format!( + "{SECTION}{}[secrets.honcho_read_boxmaker1]\nfile = \"keys/boxmaker1\"\n", + read("boxmaker1", "boxmaker1-trusted") + ), + "[secrets.honcho_read_boxmaker1] file \"keys/boxmaker1\" must be an absolute path" + .to_string(), + ), + ( + "[secrets.honcho_trusted]\nenv = \"HONCHO_KEY\"\n".to_string(), + "[secrets.honcho_trusted] is set, but there is no [honcho] section".to_string(), + ), + ]; + for (text, want) in &cases { + assert_eq!(&reason(text), want, "\n{text}"); + } + assert_eq!(cases.len(), 29); +} + +#[test] +fn the_first_problem_is_the_one_reported() { + // A bad url and an owner that is the agent: the url is checked first. + let text = with( + "url = \"http://rift.scylla-hammerhead.ts.net:8000\"", + "url = \"rift\"", + "", + ) + .replace("owner = \"kyle\"", "owner = \"boxmaker0\""); + assert_eq!(reason(&text), URL_REASON); + // A read agent that is this agent, with a bad trust: the agent is checked first. + let text = format!( + "{SECTION}[[honcho.read]]\nagent = \"boxmaker0\"\nworkspace = \"w\"\ntrust = \"no\"\n" + ); + assert_eq!( + reason(&text), + "[[honcho.read]] agent \"boxmaker0\" is this agent ([honcho] agent)" + ); +} diff --git a/docs/plans/M5c1/files/crates/memoryd/tests/honcho_start.rs b/docs/plans/M5c1/files/crates/memoryd/tests/honcho_start.rs new file mode 100644 index 0000000..d68db4b --- /dev/null +++ b/docs/plans/M5c1/files/crates/memoryd/tests/honcho_start.rs @@ -0,0 +1,238 @@ +//! `memoryd serve` with a `[honcho]` section (M5c spec, section 10): an invalid one stops it at +//! start with the reason and the `honcho-config-invalid` pointer, before it takes its lock or +//! touches its home; a valid one starts it as before. A key is never printed. Every wait has a +//! deadline, and every process is killed and every path removed by a guard (tip T44). Do not +//! edit. + +#[path = "support/tmp.rs"] +mod tmp; + +use std::os::unix::fs::PermissionsExt; +use std::os::unix::net::UnixStream; +use std::path::PathBuf; +use std::process::{Child, Command, Stdio}; +use std::time::{Duration, Instant}; + +use bxcore::{Empty, Envelope, Message, PROTOCOL_VERSION, read_frame, write_frame}; +use tmp::TempDir; + +const POINTER: &str = "see docs/runbook.md#honcho-config-invalid"; +const START_FAILED: &str = "see docs/runbook.md#memoryd-start-failed"; + +/// The values of the two keys the configs name. Built from pieces, so that no file holds one +/// whole; neither may appear in anything `memoryd` prints. +fn env_key() -> String { + ["hk", "env", "7f3a9c"].join("-") +} +fn file_key() -> String { + ["hk", "file", "b81d2e"].join("-") +} + +const SECTION: &str = "[honcho] +url = \"http://127.0.0.1:9\" +agent = \"boxmaker0\" +owner = \"kyle\" +trusted = \"boxmaker0-trusted\" +untrusted = \"boxmaker0-untrusted\" +fleet = \"fleet\" +[[honcho.read]] +agent = \"boxmaker1\" +workspace = \"boxmaker1-trusted\" +"; + +struct Home { + dir: TempDir, + config: PathBuf, +} + +impl Home { + /// A home whose `memoryd.toml` is the usual one and `extra`, with a key file holding + /// `file_key()` (mode 0600) at `/keys/honcho.key`. + fn new(tag: &str, extra: &str) -> Home { + let dir = TempDir::new(tag); + let soul = dir.write("etc/soul.md", "You are Boxmaker.\n"); + let key = dir.write("keys/honcho.key", &format!("{}\n", file_key())); + std::fs::set_permissions(&key, std::fs::Permissions::from_mode(0o600)).unwrap(); + let text = format!( + "[paths]\nhome = \"{}\"\n[memory]\nsoul_starter = \"{}\"\n{extra}", + dir.path().display(), + soul.display() + ); + let text = text.replace("KEY_FILE", &key.display().to_string()); + let config = dir.write("memoryd.toml", &text); + Home { dir, config } + } + + fn path(&self, relative: &str) -> PathBuf { + self.dir.path().join(relative) + } + + fn command(&self) -> Command { + let mut command = Command::new(env!("CARGO_BIN_EXE_memoryd")); + command + .args(["serve", "--config"]) + .arg(&self.config) + .env("HONCHO_TEST_KEY", env_key()) + .stdin(Stdio::null()) + .stdout(Stdio::null()); + command + } + + /// Runs `memoryd serve` to its end, at most 10 s (then it is killed), and gives its exit code + /// (`None` if it had to be killed) and its stderr. + fn run_to_end(&self) -> (Option, String) { + let mut child = Daemon(self.command().stderr(Stdio::piped()).spawn().unwrap()); + let deadline = Instant::now() + Duration::from_secs(10); + let mut code = None; + while Instant::now() < deadline { + if let Ok(Some(status)) = child.0.try_wait() { + code = status.code(); + break; + } + std::thread::sleep(Duration::from_millis(20)); + } + let _ = child.0.kill(); + let _ = child.0.wait(); + let mut stderr = String::new(); + if let Some(mut pipe) = child.0.stderr.take() { + let _ = std::io::Read::read_to_string(&mut pipe, &mut stderr); + } + (code, stderr) + } + + /// `memoryd serve` with its stderr in `/memoryd.err`. + fn start(&self) -> Daemon { + let err = std::fs::File::create(self.path("memoryd.err")).unwrap(); + Daemon(self.command().stderr(Stdio::from(err)).spawn().unwrap()) + } + + fn stderr(&self) -> String { + std::fs::read_to_string(self.path("memoryd.err")).unwrap_or_default() + } +} + +/// A running `memoryd serve`, killed and waited for when dropped. Each test declares it after its +/// `Home`, so it drops first. +struct Daemon(Child); + +impl Drop for Daemon { + fn drop(&mut self) { + let _ = self.0.kill(); + let _ = self.0.wait(); + } +} + +/// True once `memoryd` answers `memory_block` on `loopd`'s socket, within 10 s. +fn answers(home: &Home) -> bool { + let socket = home.path("run/loop-memory/memory.sock"); + let deadline = Instant::now() + Duration::from_secs(10); + while Instant::now() < deadline { + if let Ok(mut stream) = UnixStream::connect(&socket) { + let limit = Some(Duration::from_secs(5)); + let ask = Envelope { + v: PROTOCOL_VERSION, + id: 1, + r#final: true, + msg: Message::MemoryBlock(Empty {}), + }; + if stream.set_read_timeout(limit).is_ok() + && write_frame(&mut stream, &ask).is_ok() + && matches!( + read_frame(&mut stream), + Ok(Envelope { + msg: Message::MemoryBlockReply(_), + .. + }) + ) + { + return true; + } + } + std::thread::sleep(Duration::from_millis(20)); + } + false +} + +fn assert_no_key(text: &str) { + assert!( + !text.contains(&env_key()), + "the env key was printed: {text}" + ); + assert!( + !text.contains(&file_key()), + "the file key was printed: {text}" + ); +} + +#[test] +fn an_invalid_section_stops_memoryd_with_its_reason_and_the_pointer() { + let extra = format!( + "{}[secrets.honcho_trusted]\nenv = \"HONCHO_TEST_KEY\"\n[secrets.honcho_fleet]\nfile = \"KEY_FILE\"\n", + SECTION.replace("owner = \"kyle\"", "owner = \"boxmaker0\"") + ); + let home = Home::new("honcho-bad", &extra); + let (code, stderr) = home.run_to_end(); + assert_eq!(code, Some(1), "{stderr}"); + assert_eq!( + stderr, + format!( + "memoryd: {}: [honcho] agent and owner are both \"boxmaker0\": the agent's peer must not be the owner's\n{POINTER}\n", + home.config.display() + ) + ); + assert_no_key(&stderr); + assert!( + !home.path("run").exists(), + "memoryd stopped before its lock and sockets" + ); + assert!(!home.path("memory").exists(), "and before its memory"); +} + +#[test] +fn a_key_for_an_agent_not_read_stops_memoryd() { + let extra = format!("{SECTION}[secrets.honcho_read_boxmaker2]\nenv = \"HONCHO_TEST_KEY\"\n"); + let home = Home::new("honcho-key", &extra); + let (code, stderr) = home.run_to_end(); + assert_eq!(code, Some(1), "{stderr}"); + assert_eq!( + stderr, + format!( + "memoryd: {}: [secrets.honcho_read_boxmaker2] is the key of agent \"boxmaker2\", which is not in [[honcho.read]]\n{POINTER}\n", + home.config.display() + ) + ); + assert_no_key(&stderr); +} + +#[test] +fn an_unknown_key_in_the_section_is_still_a_start_failure() { + let extra = SECTION.replace("fleet = \"fleet\"", "fleet = \"fleet\"\nfleat = \"x\""); + let home = Home::new("honcho-unknown", &extra); + let (code, stderr) = home.run_to_end(); + assert_eq!(code, Some(1), "{stderr}"); + assert!( + stderr.starts_with(&format!("memoryd: {}: ", home.config.display())), + "{stderr}" + ); + assert!(stderr.contains("fleat"), "{stderr}"); + assert!(stderr.ends_with(&format!("\n{START_FAILED}\n")), "{stderr}"); + assert!(!stderr.contains(POINTER), "{stderr}"); +} + +#[test] +fn a_valid_section_with_keys_starts_memoryd_as_before() { + let extra = format!( + "{SECTION}[secrets.honcho_trusted]\nenv = \"HONCHO_TEST_KEY\"\n[secrets.honcho_fleet]\nfile = \"KEY_FILE\"\n[secrets.honcho_read_boxmaker1]\nenv = \"HONCHO_TEST_KEY\"\n" + ); + let home = Home::new("honcho-good", &extra); + let _daemon = home.start(); + assert!( + answers(&home), + "memoryd never answered; stderr: {}", + home.stderr() + ); + let stderr = home.stderr(); + assert!(stderr.contains("memoryd: serving\n"), "{stderr}"); + assert_no_key(&stderr); + assert!(!stderr.contains(POINTER), "{stderr}"); +} diff --git a/docs/plans/M5c1/files/runbook-honcho-config-invalid.md b/docs/plans/M5c1/files/runbook-honcho-config-invalid.md new file mode 100644 index 0000000..e9a26ce --- /dev/null +++ b/docs/plans/M5c1/files/runbook-honcho-config-invalid.md @@ -0,0 +1,50 @@ +## honcho-config-invalid + +**What you see.** `memoryd` exits 1 at start, before it takes its lock or touches its home, with +one line naming its config file and the reason, then this entry: + +```text +memoryd: /etc/boxmaker/memoryd.toml: [honcho] agent and owner are both "kyle": the agent's peer must not be the owner's +see docs/runbook.md#honcho-config-invalid +``` + +**Why.** `[honcho]` in `memoryd.toml` names this agent's peer, the owner's peer and the +workspaces the agent writes to and reads in Honcho, and `[secrets.]` where each workspace's +key comes from (M5c spec, section 10). A wrong name would send turns and memory to a workspace +they do not belong in, so `memoryd` checks the section before it starts. The reasons: + +- `url` is not `http://` or `https://`, a host (letters, digits, `.` and `-`) and an optional + port (1 to 65535), and nothing else: no path, not even a `/`, and no user or password. The line + does not repeat the URL. +- A name (`agent`, `owner`, `trusted`, `untrusted`, `fleet`, and each `[[honcho.read]]`'s `agent` + and `workspace`) is not 1 to 512 of `a-z`, `A-Z`, `0-9`, `_` and `-`, Honcho's own rule: a `.` + or a space is not allowed. +- `agent` and `owner` are the same peer, or two of `trusted`, `untrusted` and `fleet` are the same + workspace. +- A `[[honcho.read]]` names this agent, names an agent twice, or names one of this agent's three + workspaces; or its `trust` is not `trusted` or `untrusted`, or its `class` not `public`, + `private` or `secret` (all lower case). +- A `[secrets.]` is not `honcho_trusted`, `honcho_untrusted`, `honcho_fleet` or + `honcho_read_` for an agent in `[[honcho.read]]`; or it does not name exactly one + well-formed `credential`, `env` or `file` (as in `secret-unavailable`); or there is no + `[honcho]` section for it. + +A key missing from `[honcho]` or `[[honcho.read]]`, an unknown key, or a value of the wrong type +is a parse error instead, and points to `memoryd-start-failed`. A key's value is not read here, +only where it comes from, and it is never printed. While `memoryd` is stopped nothing is synced, +and `bxctl memory sync` cannot reach it. + +**Confirm.** The line quotes the table and the value. Open the file it names and compare the +section with the M5c spec, section 10 (`docs/specs/2026-09-27-m5c-honcho.md`): + +```sh +grep -n -A6 -e '^\[honcho\]' -e '^\[\[honcho.read\]\]' -e '^\[secrets\.' +``` + +**Fix.** Correct the value. Keep a name that is already in use as it is: a new `trusted`, +`untrusted` or `fleet` sends everything from then on to a new, empty workspace, and a new `agent` +is a new peer. `agent` is this agent's own name, different on each agent; `owner` and `fleet` are +the same in every agent's file. To turn Honcho off, remove `[honcho]`, its `[[honcho.read]]` +tables and every `[secrets.honcho_…]`. + +**Check.** `memoryd serve --config ` prints `memoryd: serving`.