M5c-2 plan: task 05, memoryd's readers

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 18:31:51 -07:00
parent c6f43002a8
commit 9a4f0c5875
6 changed files with 1706 additions and 1 deletions
@@ -0,0 +1,410 @@
+# M5c-2 task 05: `memoryd`'s readers, the Honcho workspaces it reads
+
+**Branch:** `m5c2`. Check it: `git branch --show-current` prints `m5c2` and `git status --short`
+is empty; otherwise stop. Never switch branch.
+**Commit subject:** `memoryd: the Honcho workspaces it reads, with their clients and labels`
+**Log row:** the Task cell is `M5c2/05-memoryd-honcho-readers`.
+
+## Goal
+
+`memoryd` writes to three Honcho workspaces through the `Clients` of M5c-1 (the agent's trusted
+and untrusted ones and the fleet's), which move into the sync worker's thread. M5c-2 also reads
+(M5c spec, sections 3, 8 and 9): `recall` searches other agents' workspaces and the fleet's, and
+`ask_memory` asks one or two workspaces by name. Reading needs clients of its own. This task
+builds them at start and gives later tasks the types they use; nothing searches or asks yet
+(tasks 07 and 08).
+
+- **`Readers`** (new module `crates/memoryd/src/honcho/read.rs`, given whole): a `Reader` for
+  - each `[[honcho.read]]` workspace, in the order written, with that entry's `trust` and
+    `class` and the key `honcho_read_<agent>`;
+  - the fleet's workspace (key `honcho_fleet`), always untrusted and `private`;
+  - the agent's own trusted workspace (key `honcho_trusted`), trusted, `private`, and its own
+    untrusted one (key `honcho_untrusted`), untrusted, `private`. These two are for `ask_memory`
+    only: `recall` never searches the agent's own workspaces (spec, section 8).
+- **Two clients per reader.** A `roncho` client has one read timeout. Searches wait 5 s
+  (`SEARCH_READ`) and chat 150 s (`CHAT_READ`, which this task changes from 180 s: spec 9 gives
+  chat 150 s, inside `brokerd`'s 170 s); both connect within 5 s (`CONNECT`).
+- **The label** of each reader's results: the name each hit shows (the agent's name, `fleet`,
+  `own` or `untrusted`), the trust, and the class. `Label::tag()` is `memory` for a trusted
+  workspace and `memory-tainted` otherwise, as a local `recall` hit's tag.
+- **`recall_sources()`**: the read workspaces, then the fleet's. **`resolve(name)`**: the reader
+  `ask_memory` means by `own`, `untrusted`, `fleet` or a read agent's name; any other name is
+  the sentence the model is shown, which lists the names it may use.
+- **Reserved names.** Since `ask_memory` names this agent's own workspaces `own`, `untrusted`
+  and `fleet`, a `[[honcho.read]]` agent with one of those names is now refused at start
+  (`honcho-config-invalid`). Otherwise a read agent called `fleet` would shadow the fleet in
+  `ask_memory`.
+- **At start**, `start::open` builds the readers beside the writers' `Clients` and keeps them in
+  `Opened::readers`. A read key that cannot be read stops `memoryd`, as a written workspace's
+  does. The three written workspaces' keys are read by both; each key file's warning is logged
+  once. `serve.rs` does not change: nothing uses the readers until task 07, which takes
+  `opened.readers` before the worker's thread takes the rest of `Opened`.
+
+## Every place that changes
+
+This is the complete list.
+
+| Place | What happens there |
+|---|---|
+| `crates/memoryd/src/honcho/read.rs` | new, given whole: copy it |
+| `crates/memoryd/src/honcho/mod.rs` | `pub mod read;` (edit H1) |
+| `crates/memoryd/src/honcho/client.rs` | `CHAT_READ` is 150 s; `client` and `key` become `pub(crate)`, for `read.rs` (edits C1 to C3) |
+| `crates/memoryd/src/honcho/config.rs` | `RESERVED`, and the check that uses it (edits G1, G2) |
+| `crates/memoryd/src/honcho/start.rs` | `Opened::readers`, built in `open` (edits O1 to O3) |
+| `docs/runbook.md`, `honcho-config-invalid` | the reserved names among the reasons (edit D1) |
+| `crates/memoryd/tests/honcho_read.rs` | new, given |
+| `crates/memoryd/tests/honcho_client.rs`, `honcho_start.rs` | new versions, given |
+| everything else | **no change**: not `serve.rs`, not `recall.rs`, not `worker.rs`, no `Cargo.toml` |
+
+## Files
+
+- Copy, from `docs/plans/M5c2/files/` (the same path under it): the new test
+  `crates/memoryd/tests/honcho_read.rs`; **new versions** of (they replace the files in the tree)
+  `crates/memoryd/tests/honcho_client.rs` and `crates/memoryd/tests/honcho_start.rs`; and the
+  new source `crates/memoryd/src/honcho/read.rs`, which is complete: it has no `todo!()`, and you
+  do not change it.
+- Modify: `crates/memoryd/src/honcho/mod.rs`, `crates/memoryd/src/honcho/client.rs`,
+  `crates/memoryd/src/honcho/config.rs`, `crates/memoryd/src/honcho/start.rs`,
+  `docs/runbook.md`, `docs/implementer-log.md`.
+- Do not touch: any other file.
+
+## Facts you need
+
+- **`read.rs` needs** from `client.rs`: `CONNECT`, `CHAT_READ`, `Timeouts`, and the two helpers
+  `client(url, workspace, key, timeouts) -> Result<Client, String>` and
+  `key(config, name, env, warnings) -> Result<Option<secrets::Secret>, String>`, which are
+  private today. Edits C2 and C3 make them `pub(crate)`: visible in `memoryd`, not outside it.
+  From `config.rs` it needs `FLEET_KEY`, `TRUSTED_KEY`, `UNTRUSTED_KEY`, `read_key`, `Trust` and
+  `HonchoConfig`, all public already, and from `bxcore` `DataClass` and
+  `bxcore::memory::OWN_WORKSPACE` (`"own"`, task 01).
+- **What `Readers` offers** (for tasks 07 and 08; read `read.rs` for the whole):
+
+  ```rust
+  pub const SEARCH_READ: Duration; // 5 s
+  pub const UNTRUSTED: &str;       // "untrusted"
+  pub const FLEET: &str;           // "fleet"
+  pub struct ReadTimeouts { pub connect: Duration, pub search: Duration, pub chat: Duration }
+  pub enum Source { Agent(String), Fleet, Own, OwnUntrusted }
+  pub struct Label { pub name: String, pub trust: Trust, pub class: DataClass }
+  impl Label { pub fn tag(&self) -> &'static str; pub fn untrusted(&self) -> bool }
+  pub struct Reader { pub source: Source, pub label: Label, /* two clients */ }
+  impl Reader { pub fn search(&self) -> &Client; pub fn chat(&self) -> &Client }
+  impl Readers {
+      pub fn new(config: &HonchoConfig, env: &dyn Fn(&str) -> Option<OsString>,
+                 timeouts: ReadTimeouts) -> Result<(Readers, Vec<String>), String>;
+      pub fn recall_sources(&self) -> Vec<&Reader>;
+      pub fn resolve(&self, name: &str) -> Result<&Reader, String>;
+  }
+  ```
+
+- **`roncho::blocking::Client`** is `Clone`, `Send` and `Sync` (its documentation says so, and
+  a given test checks it at compile time), and its `Debug` shows the timeouts and whether a key
+  is present, never the key. This task's table of `roncho`'s API is complete; `roncho`'s source
+  (the Cargo registry, `~/.cargo/registry`) is outside the repository and off limits.
+- **Why the warnings are filtered.** `Clients::new` and `Readers::new` each read the keys of
+  the three written workspaces, and each returns a key file's plaintext warning. Edit O3 logs
+  the writers' warnings, then only those of the readers' that are not among them.
+- **The given tests.**
+  - `honcho_read.rs` (8 tests), against the fake Honcho: the client and the readers are
+    `Send + Sync`; the timeouts, read from each client's `Debug`; each reader searches its own
+    workspace with its own key (the fake records each request's `Authorization` header), and no
+    header without a key; no key in a reader's `Debug`, an error, or the error of a key that
+    cannot be read; `recall_sources` never holds the agent's own workspaces; each source's
+    label and tag; `resolve` for every name, and the exact sentence for an unknown one; a read
+    agent named `own`, `untrusted` or `fleet` refused by `config::check`, with its sentence.
+  - `honcho_client.rs`: one line changes, `CHAT_READ` is 150 s.
+  - `honcho_start.rs`: a new test, `a_read_key_that_cannot_be_read_stops_memoryd`, and one new
+    assertion in `a_valid_section_with_keys_starts_memoryd_as_before`: the fleet key file's
+    warning is printed once.
+- The items the edits use. This table is complete.
+
+  | Item | Where |
+  |---|---|
+  | `Readers::new`, `ReadTimeouts::DEFAULT` (from `read.rs`) | O3 |
+  | `StartError::Secret(String)` | O3, as `Clients::new`'s error beside it |
+  | `Vec::contains`, `Iterator::filter`, `Iterator::chain` | O3 |
+  | `Arc::new` | O3 |
+  | `[&str; 3]::contains(&&str)`, `String::as_str` | G2: `RESERVED.contains(&entry.agent.as_str())` |
+
+## The edits, exactly
+
+Nothing else in these files changes. Each quoted "before" block occurs exactly once in its file
+when its turn comes (make them in order). Keep the fields in the order shown.
+
+### `crates/memoryd/src/honcho/mod.rs`
+
+**H1.** Add one line. The line
+
+```rust
+pub mod naming;
+```
+
+becomes
+
+```rust
+pub mod naming;
+pub mod read;
+```
+
+### `crates/memoryd/src/honcho/client.rs`
+
+**C1.** The lines
+
+```rust
+/// A chat's read: a non-streamed chat sends nothing until it is done (M5c-2's `ask_memory`).
+pub const CHAT_READ: Duration = Duration::from_secs(180);
+```
+
+become
+
+```rust
+/// A chat's read: a non-streamed chat sends nothing until it is done (M5c spec, section 9).
+pub const CHAT_READ: Duration = Duration::from_secs(150);
+```
+
+**C2.** The lines
+
+```rust
+/// One workspace's client: no `roncho.toml` read, the timeouts given, the key when there is one.
+fn client(
+```
+
+become
+
+```rust
+/// One workspace's client: no `roncho.toml` read, the timeouts given, the key when there is one.
+pub(crate) fn client(
+```
+
+**C3.** The lines
+
+```rust
+/// The key named `name` in `[secrets]`, if one is configured, read now.
+fn key(
+```
+
+become
+
+```rust
+/// The key named `name` in `[secrets]`, if one is configured, read now.
+pub(crate) fn key(
+```
+
+### `crates/memoryd/src/honcho/config.rs`
+
+**G1.** The lines
+
+```rust
+/// The names of the keys in `[secrets.<name>]`, one per workspace. A read workspace's key is
+```
+
+become
+
+```rust
+/// The names `ask_memory` gives this agent's own workspaces and the fleet's (M5c spec, section 9),
+/// which no `[[honcho.read]]` agent may have.
+pub const RESERVED: [&str; 3] = ["own", "untrusted", "fleet"];
+
+/// The names of the keys in `[secrets.<name>]`, one per workspace. A read workspace's key is
+```
+
+**G2.** In `read_workspace`, the lines (its spaces are checked: the sentence is one string split
+across two lines with a `\` at the end of the first, and the given test compares it letter for
+letter)
+
+```rust
+            "[[honcho.read]] agent {:?} is this agent ([honcho] agent)",
+            entry.agent
+        ));
+    }
+```
+
+become
+
+```rust
+            "[[honcho.read]] agent {:?} is this agent ([honcho] agent)",
+            entry.agent
+        ));
+    }
+    if RESERVED.contains(&entry.agent.as_str()) {
+        return Err(format!(
+            "[[honcho.read]] agent {:?} is a name ask_memory gives this agent's own workspaces: \
+             own, untrusted and fleet cannot be agents' names",
+            entry.agent
+        ));
+    }
+```
+
+### `crates/memoryd/src/honcho/start.rs`
+
+**O1.** Add one line. The line
+
+```rust
+use crate::honcho::mirror::Mirror;
+```
+
+becomes
+
+```rust
+use crate::honcho::mirror::Mirror;
+use crate::honcho::read::{ReadTimeouts, Readers};
+```
+
+**O2.** The lines
+
+```rust
+pub struct Opened {
+    pub sync: Arc<Mutex<TurnSync>>,
+    clients: Clients,
+```
+
+become
+
+```rust
+pub struct Opened {
+    pub sync: Arc<Mutex<TurnSync>>,
+    /// The workspaces `recall` and `ask_memory` read (M5c spec, sections 8 and 9).
+    pub readers: Arc<Readers>,
+    clients: Clients,
+```
+
+**O3.** In `open`, the lines
+
+```rust
+    for warning in warnings {
+        (log)(&warning);
+    }
+    Ok(Opened {
+        sync: Arc::new(Mutex::new(sync)),
+        clients,
+```
+
+become
+
+```rust
+    let (readers, read_warnings) =
+        Readers::new(honcho, &env, ReadTimeouts::DEFAULT).map_err(StartError::Secret)?;
+    // The readers load the keys of the three written workspaces again: each warning once.
+    let read_warnings: Vec<String> = read_warnings
+        .into_iter()
+        .filter(|w| !warnings.contains(w))
+        .collect();
+    for warning in warnings.into_iter().chain(read_warnings) {
+        (log)(&warning);
+    }
+    Ok(Opened {
+        sync: Arc::new(Mutex::new(sync)),
+        readers: Arc::new(readers),
+        clients,
+```
+
+### `docs/runbook.md`
+
+**D1.** In the entry `honcho-config-invalid`, the lines (its spaces are checked: the entry is
+read by people, and `scripts/check-runbook.sh` reads the file)
+
+```text
+- A `[[honcho.read]]` names this agent, names an agent twice, or names one of this agent's three
+  workspaces; or its `trust` is not `trusted` or `untrusted`, or its `class` not `public`,
+  `private` or `secret` (all lower case).
+```
+
+become
+
+```text
+- A `[[honcho.read]]` names this agent, names an agent twice, names an agent `own`, `untrusted`
+  or `fleet` (the names `ask_memory` gives this agent's own workspaces), or names one of this
+  agent's three workspaces; or its `trust` is not `trusted` or `untrusted`, or its `class` not
+  `public`, `private` or `secret` (all lower case).
+```
+
+## Steps
+
+- [ ] **1. Copy the given tests.** From the repository root, three commands:
+
+  ```sh
+  cp docs/plans/M5c2/files/crates/memoryd/tests/honcho_read.rs crates/memoryd/tests/
+  cp docs/plans/M5c2/files/crates/memoryd/tests/honcho_client.rs crates/memoryd/tests/
+  cp docs/plans/M5c2/files/crates/memoryd/tests/honcho_start.rs crates/memoryd/tests/
+  ```
+
+  Then `git status --short` lists `honcho_client.rs` and `honcho_start.rs` as modified and
+  `honcho_read.rs` as new, all under `crates/memoryd/tests/`.
+- [ ] **2. See them fail.** Two commands, each with a shell timeout of 600000 ms:
+  - `cargo build --tests --keep-going -p memoryd`: one test target, `honcho_read`, does not
+    compile: `E0432` for ``unresolved import `memoryd::honcho::read` `` and for
+    ``unresolved import `memoryd::honcho::config::RESERVED` ``. Every other target compiles.
+  - `cargo test --no-fail-fast -p memoryd --test honcho_client --test honcho_start`: two tests
+    fail: `the_timeouts` in `honcho_client` (`CHAT_READ` is 180 s) and
+    `a_read_key_that_cannot_be_read_stops_memoryd` in `honcho_start` (`memoryd` does not read
+    that key yet, so it starts, and the test stops it after 10 s).
+- [ ] **3. The new module and the edits.** Copy the module:
+
+  ```sh
+  cp docs/plans/M5c2/files/crates/memoryd/src/honcho/read.rs crates/memoryd/src/honcho/
+  ```
+
+  Then make edits H1, C1 to C3, G1, G2, O1 to O3 and D1, in order. `cargo check -p memoryd`
+  prints no error and no warning.
+- [ ] **4. The tests.** With a shell timeout of 600000 ms, every test passes: 13 in
+  `honcho_client`, 8 in `honcho_config`, 3 in `honcho_keys` (and 1 ignored, as before), 8 in
+  `honcho_read`, 5 in `honcho_start`:
+
+  ```sh
+  cargo test -p memoryd --test honcho_read --test honcho_client --test honcho_start \
+    --test honcho_config --test honcho_keys
+  ```
+- [ ] **5. Check.** Two commands:
+  - `git diff --stat -- crates/memoryd/src` lists exactly `client.rs`, `config.rs`, `mod.rs` and
+    `start.rs` under `crates/memoryd/src/honcho/` (the new `read.rs` is untracked, so `git diff`
+    does not list it), and `git status --short crates/memoryd/src` shows `read.rs` as new.
+  - `git diff --stat -- crates/memoryd/src/serve.rs crates/memoryd/src/recall.rs Cargo.toml Cargo.lock`
+    prints nothing.
+- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`, with a shell timeout of 600000 ms.
+  Expected last line: `gate: ok`. The test count is the passed total of `cargo test --workspace`
+  plus one, for the ignored `end_to_end` test that the gate runs on its own: **1403** after this
+  task, 1394 before it; the 9 new ones are the 8 tests of `honcho_read.rs` and the new test in
+  `honcho_start.rs`. To count, add up the `N passed` of every `test result:` line the gate's
+  `cargo test --workspace` printed, and add 1. Never run the gate again only to count.
+- [ ] **7. Log and commit.** Add your row to `docs/implementer-log.md`; its Task cell is
+  `M5c2/05-memoryd-honcho-readers`. `git status --short` lists the four edited sources, the two
+  replaced tests, `docs/runbook.md` and your log as modified, and `read.rs` and `honcho_read.rs`
+  as new. Stage with `git add crates/memoryd docs/runbook.md docs/implementer-log.md Cargo.lock`,
+  then commit as `AGENTS.md` says. `Cargo.lock` does not change in this task; the `git add` is
+  harmless.
+
+## Done when
+
+- `make gate` prints `gate: ok` with 1403 tests.
+- `git diff HEAD~1 --stat` lists only the four edited sources, `read.rs`, the three test files of
+  step 1, `docs/runbook.md` and `docs/implementer-log.md`.
+
+## Stop and report if
+
+- The branch check at the top fails.
+- A "before" block is not in its file exactly as shown when its turn comes, or is there more
+  than once.
+- Step 2 shows a different result: another target failing, other errors, other tests failing.
+- `cargo check` in step 3 prints an error or a warning. Quote it, and stop. Do not change
+  `read.rs` to make it compile.
+- A test fails in step 4. Do not change a test: stop, and give the failing tests' names and
+  their messages.
+- The fix seems to need a change anywhere else: `serve.rs`, `recall.rs`, `worker.rs`, a change
+  to `read.rs`, a new dependency. It does not; stop and say why you think so.
+- A test outside the given files fails in the gate: check once whether it fails without your
+  change (`git stash -u`, run it, `git stash pop`), then stop and report both results.
+- The gate passes with a count other than 1403.
+- A comment in an edit does not fit the code under it. Quote it and stop.
+
+## What the given tests pin
+
+Each of these changes to the finished code makes at least one given test fail (24 tried, all
+caught): the search clients given chat's timeout, or chat's the search one; either timeout
+constant changed; a read workspace given the fleet's key, the fleet the trusted key, or the own
+untrusted workspace the trusted key; the own reader on the untrusted workspace; a key not sent;
+a read workspace's trust or class ignored; the fleet trusted, or the own workspace untrusted;
+the tags swapped, or `untrusted()` inverted; `recall_sources` holding the own workspace, or not
+the fleet's; `untrusted` resolved to the trusted workspace; read agents not resolved, or
+resolved by workspace instead of name; the sentence without the read agents' names; the
+reserved names allowed; the readers' errors ignored at start; the key file's warning logged
+twice.