boxmaker/docs/plans/M5c2/05-memoryd-honcho-readers.md

# M5c-2 task 05: `memoryd`'s readers, the Honcho workspaces it reads

**Branch:** `m5c2`. Check it: `git branch --show-current` prints `m5c2` and `git status --short`
is empty; otherwise stop. Never switch branch.
**Commit subject:** `memoryd: the Honcho workspaces it reads, with their clients and labels`
**Log row:** the Task cell is `M5c2/05-memoryd-honcho-readers`.

## Goal

`memoryd` writes to three Honcho workspaces through the `Clients` of M5c-1 (the agent's trusted
and untrusted ones and the fleet's), which move into the sync worker's thread. M5c-2 also reads
(M5c spec, sections 3, 8 and 9): `recall` searches other agents' workspaces and the fleet's, and
`ask_memory` asks one or two workspaces by name. Reading needs clients of its own. This task
builds them at start and gives later tasks the types they use; nothing searches or asks yet
(tasks 08 to 10).

- **`Readers`** (new module `crates/memoryd/src/honcho/read.rs`, given whole): a `Reader` for
  - each `[[honcho.read]]` workspace, in the order written, with that entry's `trust` and
    `class` and the key `honcho_read_<agent>`;
  - the fleet's workspace (key `honcho_fleet`), always untrusted and `private`;
  - the agent's own trusted workspace (key `honcho_trusted`), trusted, `private`, and its own
    untrusted one (key `honcho_untrusted`), untrusted, `private`. These two are for `ask_memory`
    only: `recall` never searches the agent's own workspaces (spec, section 8).
- **Two clients per reader.** A `roncho` client has one read timeout. Searches wait 5 s
  (`SEARCH_READ`) and chat 150 s (`CHAT_READ`, which this task changes from 180 s: spec 9 gives
  chat 150 s, inside `brokerd`'s 170 s); both connect within 5 s (`CONNECT`).
- **The label** of each reader's results: the name each hit shows (the agent's name, `fleet`,
  `own` or `untrusted`), the trust, and the class. `Label::tag()` is `memory` for a trusted
  workspace and `memory-tainted` otherwise, as a local `recall` hit's tag.
- **`recall_sources()`**: the read workspaces, then the fleet's. **`resolve(name)`**: the reader
  `ask_memory` means by `own`, `untrusted`, `fleet` or a read agent's name; any other name is
  the sentence the model is shown, which lists the names it may use.
- **Reserved names.** Since `ask_memory` names this agent's own workspaces `own`, `untrusted`
  and `fleet`, a `[[honcho.read]]` agent with one of those names is now refused at start
  (`honcho-config-invalid`). Otherwise a read agent called `fleet` would shadow the fleet in
  `ask_memory`.
- **At start**, `start::open` builds the readers beside the writers' `Clients` and keeps them in
  `Opened::readers`. A read key that cannot be read stops `memoryd`, as a written workspace's
  does. The three written workspaces' keys are read by both; each key file's warning is logged
  once. `serve.rs` does not change: nothing uses the readers until task 09, which takes
  `opened.readers` before the worker's thread takes the rest of `Opened`.

## Every place that changes

This is the complete list.

| Place | What happens there |
|---|---|
| `crates/memoryd/src/honcho/read.rs` | new, given whole: copy it |
| `crates/memoryd/src/honcho/mod.rs` | `pub mod read;` (edit H1) |
| `crates/memoryd/src/honcho/client.rs` | `CHAT_READ` is 150 s; `client` and `key` become `pub(crate)`, for `read.rs` (edits C1 to C3) |
| `crates/memoryd/src/honcho/config.rs` | `RESERVED`, and the check that uses it (edits G1, G2) |
| `crates/memoryd/src/honcho/start.rs` | `Opened::readers`, built in `open` (edits O1 to O3) |
| `docs/runbook.md`, `honcho-config-invalid` | the reserved names among the reasons (edit D1) |
| `crates/memoryd/tests/honcho_read.rs` | new, given |
| `crates/memoryd/tests/honcho_client.rs`, `honcho_start.rs` | new versions, given |
| everything else | **no change**: not `serve.rs`, not `recall.rs`, not `worker.rs`, no `Cargo.toml` |

## Files

- Copy, from `docs/plans/M5c2/files/` (the same path under it): the new test
  `crates/memoryd/tests/honcho_read.rs`; **new versions** of (they replace the files in the tree)
  `crates/memoryd/tests/honcho_client.rs` and `crates/memoryd/tests/honcho_start.rs`; and the
  new source `crates/memoryd/src/honcho/read.rs`, which is complete: it has no `todo!()`, and you
  do not change it.
- Modify: `crates/memoryd/src/honcho/mod.rs`, `crates/memoryd/src/honcho/client.rs`,
  `crates/memoryd/src/honcho/config.rs`, `crates/memoryd/src/honcho/start.rs`,
  `docs/runbook.md`, `docs/implementer-log.md`.
- Do not touch: any other file.

## Facts you need

- **`read.rs` needs** from `client.rs`: `CONNECT`, `CHAT_READ`, `Timeouts`, and the two helpers
  `client(url, workspace, key, timeouts) -> Result<Client, String>` and
  `key(config, name, env, warnings) -> Result<Option<secrets::Secret>, String>`, which are
  private today. Edits C2 and C3 make them `pub(crate)`: visible in `memoryd`, not outside it.
  From `config.rs` it needs `FLEET_KEY`, `TRUSTED_KEY`, `UNTRUSTED_KEY`, `read_key`, `Trust` and
  `HonchoConfig`, all public already, and from `bxcore` `DataClass` and
  `bxcore::memory::OWN_WORKSPACE` (`"own"`, task 01).
- **What `Readers` offers** (for tasks 08 to 10; read `read.rs` for the whole):

  ```rust
  pub const SEARCH_READ: Duration; // 5 s
  pub const UNTRUSTED: &str;       // "untrusted"
  pub const FLEET: &str;           // "fleet"
  pub struct ReadTimeouts { pub connect: Duration, pub search: Duration, pub chat: Duration }
  pub enum Source { Agent(String), Fleet, Own, OwnUntrusted }
  pub struct Label { pub name: String, pub trust: Trust, pub class: DataClass }
  impl Label { pub fn tag(&self) -> &'static str; pub fn untrusted(&self) -> bool }
  pub struct Reader { pub source: Source, pub label: Label, /* two clients */ }
  impl Reader { pub fn search(&self) -> &Client; pub fn chat(&self) -> &Client }
  impl Readers {
      pub fn new(config: &HonchoConfig, env: &dyn Fn(&str) -> Option<OsString>,
                 timeouts: ReadTimeouts) -> Result<(Readers, Vec<String>), String>;
      pub fn recall_sources(&self) -> Vec<&Reader>;
      pub fn resolve(&self, name: &str) -> Result<&Reader, String>;
  }
  ```

- **`roncho::blocking::Client`** is `Clone`, `Send` and `Sync` (its documentation says so, and
  a given test checks it at compile time), and its `Debug` shows the timeouts and whether a key
  is present, never the key. This task's table of `roncho`'s API is complete; `roncho`'s source
  (the Cargo registry, `~/.cargo/registry`) is outside the repository and off limits.
- **Why the warnings are filtered.** `Clients::new` and `Readers::new` each read the keys of
  the three written workspaces, and each returns a key file's plaintext warning. Edit O3 logs
  the writers' warnings, then only those of the readers' that are not among them.
- **The given tests.**
  - `honcho_read.rs` (8 tests), against the fake Honcho: the client and the readers are
    `Send + Sync`; the timeouts, read from each client's `Debug`; each reader searches its own
    workspace with its own key (the fake records each request's `Authorization` header), and no
    header without a key; no key in a reader's `Debug`, an error, or the error of a key that
    cannot be read; `recall_sources` never holds the agent's own workspaces; each source's
    label and tag; `resolve` for every name, and the exact sentence for an unknown one; a read
    agent named `own`, `untrusted` or `fleet` refused by `config::check`, with its sentence.
  - `honcho_client.rs`: one line changes, `CHAT_READ` is 150 s.
  - `honcho_start.rs`: a new test, `a_read_key_that_cannot_be_read_stops_memoryd`, and one new
    assertion in `a_valid_section_with_keys_starts_memoryd_as_before`: the fleet key file's
    warning is printed once.
- The items the edits use. This table is complete.

  | Item | Where |
  |---|---|
  | `Readers::new`, `ReadTimeouts::DEFAULT` (from `read.rs`) | O3 |
  | `StartError::Secret(String)` | O3, as `Clients::new`'s error beside it |
  | `Vec::contains`, `Iterator::filter`, `Iterator::chain` | O3 |
  | `Arc::new` | O3 |
  | `[&str; 3]::contains(&&str)`, `String::as_str` | G2: `RESERVED.contains(&entry.agent.as_str())` |

## The edits, exactly

Nothing else in these files changes. Each quoted "before" block occurs exactly once in its file
when its turn comes (make them in order). Keep the fields in the order shown.

### `crates/memoryd/src/honcho/mod.rs`

**H1.** Add one line. The line

```rust
pub mod naming;
```

becomes

```rust
pub mod naming;
pub mod read;
```

### `crates/memoryd/src/honcho/client.rs`

**C1.** The lines

```rust
/// A chat's read: a non-streamed chat sends nothing until it is done (M5c-2's `ask_memory`).
pub const CHAT_READ: Duration = Duration::from_secs(180);
```

become

```rust
/// A chat's read: a non-streamed chat sends nothing until it is done (M5c spec, section 9).
pub const CHAT_READ: Duration = Duration::from_secs(150);
```

**C2.** The lines

```rust
/// One workspace's client: no `roncho.toml` read, the timeouts given, the key when there is one.
fn client(
```

become

```rust
/// One workspace's client: no `roncho.toml` read, the timeouts given, the key when there is one.
pub(crate) fn client(
```

**C3.** The lines

```rust
/// The key named `name` in `[secrets]`, if one is configured, read now.
fn key(
```

become

```rust
/// The key named `name` in `[secrets]`, if one is configured, read now.
pub(crate) fn key(
```

### `crates/memoryd/src/honcho/config.rs`

**G1.** The lines

```rust
/// The names of the keys in `[secrets.<name>]`, one per workspace. A read workspace's key is
```

become

```rust
/// The names `ask_memory` gives this agent's own workspaces and the fleet's (M5c spec, section 9),
/// which no `[[honcho.read]]` agent may have.
pub const RESERVED: [&str; 3] = ["own", "untrusted", "fleet"];

/// The names of the keys in `[secrets.<name>]`, one per workspace. A read workspace's key is
```

**G2.** In `read_workspace`, the lines (its spaces are checked: the sentence is one string split
across two lines with a `\` at the end of the first, and the given test compares it letter for
letter)

```rust
            "[[honcho.read]] agent {:?} is this agent ([honcho] agent)",
            entry.agent
        ));
    }
```

become

```rust
            "[[honcho.read]] agent {:?} is this agent ([honcho] agent)",
            entry.agent
        ));
    }
    if RESERVED.contains(&entry.agent.as_str()) {
        return Err(format!(
            "[[honcho.read]] agent {:?} is a name ask_memory gives this agent's own workspaces: \
             own, untrusted and fleet cannot be agents' names",
            entry.agent
        ));
    }
```

### `crates/memoryd/src/honcho/start.rs`

**O1.** Add one line. The line

```rust
use crate::honcho::mirror::Mirror;
```

becomes

```rust
use crate::honcho::mirror::Mirror;
use crate::honcho::read::{ReadTimeouts, Readers};
```

**O2.** The lines

```rust
pub struct Opened {
    pub sync: Arc<Mutex<TurnSync>>,
    clients: Clients,
```

become

```rust
pub struct Opened {
    pub sync: Arc<Mutex<TurnSync>>,
    /// The workspaces `recall` and `ask_memory` read (M5c spec, sections 8 and 9).
    pub readers: Arc<Readers>,
    clients: Clients,
```

**O3.** In `open`, the lines

```rust
    for warning in warnings {
        (log)(&warning);
    }
    Ok(Opened {
        sync: Arc::new(Mutex::new(sync)),
        clients,
```

become

```rust
    let (readers, read_warnings) =
        Readers::new(honcho, &env, ReadTimeouts::DEFAULT).map_err(StartError::Secret)?;
    // The readers load the keys of the three written workspaces again: each warning once.
    let read_warnings: Vec<String> = read_warnings
        .into_iter()
        .filter(|w| !warnings.contains(w))
        .collect();
    for warning in warnings.into_iter().chain(read_warnings) {
        (log)(&warning);
    }
    Ok(Opened {
        sync: Arc::new(Mutex::new(sync)),
        readers: Arc::new(readers),
        clients,
```

### `docs/runbook.md`

**D1.** In the entry `honcho-config-invalid`, the lines (its spaces are checked: the entry is
read by people, and `scripts/check-runbook.sh` reads the file)

```text
- A `[[honcho.read]]` names this agent, names an agent twice, or names one of this agent's three
  workspaces; or its `trust` is not `trusted` or `untrusted`, or its `class` not `public`,
  `private` or `secret` (all lower case).
```

become

```text
- A `[[honcho.read]]` names this agent, names an agent twice, names an agent `own`, `untrusted`
  or `fleet` (the names `ask_memory` gives this agent's own workspaces), or names one of this
  agent's three workspaces; or its `trust` is not `trusted` or `untrusted`, or its `class` not
  `public`, `private` or `secret` (all lower case).
```

## Steps

- [ ] **1. Copy the given tests.** From the repository root, three commands:

  ```sh
  cp docs/plans/M5c2/files/crates/memoryd/tests/honcho_read.rs crates/memoryd/tests/
  cp docs/plans/M5c2/files/crates/memoryd/tests/honcho_client.rs crates/memoryd/tests/
  cp docs/plans/M5c2/files/crates/memoryd/tests/honcho_start.rs crates/memoryd/tests/
  ```

  Then `git status --short` lists `honcho_client.rs` and `honcho_start.rs` as modified and
  `honcho_read.rs` as new, all under `crates/memoryd/tests/`.
- [ ] **2. See them fail.** Two commands, each with a shell timeout of 600000 ms:
  - `cargo build --tests --keep-going -p memoryd`: one test target, `honcho_read`, does not
    compile: `E0432` for ``unresolved import `memoryd::honcho::read` `` and for
    ``unresolved import `memoryd::honcho::config::RESERVED` ``. Every other target compiles.
  - `cargo test --no-fail-fast -p memoryd --test honcho_client --test honcho_start`: two tests
    fail: `the_timeouts` in `honcho_client` (`CHAT_READ` is 180 s) and
    `a_read_key_that_cannot_be_read_stops_memoryd` in `honcho_start` (`memoryd` does not read
    that key yet, so it starts, and the test stops it after 10 s).
- [ ] **3. The new module and the edits.** Copy the module:

  ```sh
  cp docs/plans/M5c2/files/crates/memoryd/src/honcho/read.rs crates/memoryd/src/honcho/
  ```

  Then make edits H1, C1 to C3, G1, G2, O1 to O3 and D1, in order. `cargo check -p memoryd`
  prints no error and no warning.
- [ ] **4. The tests.** With a shell timeout of 600000 ms, every test passes: 13 in
  `honcho_client`, 8 in `honcho_config`, 3 in `honcho_keys` (and 1 ignored, as before), 8 in
  `honcho_read`, 5 in `honcho_start`:

  ```sh
  cargo test -p memoryd --test honcho_read --test honcho_client --test honcho_start \
    --test honcho_config --test honcho_keys
  ```
- [ ] **5. Check.** Two commands:
  - `git diff --stat -- crates/memoryd/src` lists exactly `client.rs`, `config.rs`, `mod.rs` and
    `start.rs` under `crates/memoryd/src/honcho/` (the new `read.rs` is untracked, so `git diff`
    does not list it), and `git status --short crates/memoryd/src` shows `read.rs` as new.
  - `git diff --stat -- crates/memoryd/src/serve.rs crates/memoryd/src/recall.rs Cargo.toml Cargo.lock`
    prints nothing.
- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`, with a shell timeout of 600000 ms.
  Expected last line: `gate: ok`. The test count is the passed total of `cargo test --workspace`
  plus one, for the ignored `end_to_end` test that the gate runs on its own: **1403** after this
  task, 1394 before it; the 9 new ones are the 8 tests of `honcho_read.rs` and the new test in
  `honcho_start.rs`. To count, add up the `N passed` of every `test result:` line the gate's
  `cargo test --workspace` printed, and add 1. Never run the gate again only to count.
- [ ] **7. Log and commit.** Add your row to `docs/implementer-log.md`; its Task cell is
  `M5c2/05-memoryd-honcho-readers`. `git status --short` lists the four edited sources, the two
  replaced tests, `docs/runbook.md` and your log as modified, and `read.rs` and `honcho_read.rs`
  as new. Stage with `git add crates/memoryd docs/runbook.md docs/implementer-log.md Cargo.lock`,
  then commit as `AGENTS.md` says. `Cargo.lock` does not change in this task; the `git add` is
  harmless.

## Done when

- `make gate` prints `gate: ok` with 1403 tests.
- `git diff HEAD~1 --stat` lists only the four edited sources, `read.rs`, the three test files of
  step 1, `docs/runbook.md` and `docs/implementer-log.md`.

## Stop and report if

- The branch check at the top fails.
- A "before" block is not in its file exactly as shown when its turn comes, or is there more
  than once.
- Step 2 shows a different result: another target failing, other errors, other tests failing.
- `cargo check` in step 3 prints an error or a warning. Quote it, and stop. Do not change
  `read.rs` to make it compile.
- A test fails in step 4. Do not change a test: stop, and give the failing tests' names and
  their messages.
- The fix seems to need a change anywhere else: `serve.rs`, `recall.rs`, `worker.rs`, a change
  to `read.rs`, a new dependency. It does not; stop and say why you think so.
- A test outside the given files fails in the gate: check once whether it fails without your
  change (`git stash -u`, run it, `git stash pop`), then stop and report both results.
- The gate passes with a count other than 1403.
- A comment in an edit does not fit the code under it. Quote it and stop.

## What the given tests pin

Each of these changes to the finished code makes at least one given test fail (24 tried, all
caught): the search clients given chat's timeout, or chat's the search one; either timeout
constant changed; a read workspace given the fleet's key, the fleet the trusted key, or the own
untrusted workspace the trusted key; the own reader on the untrusted workspace; a key not sent;
a read workspace's trust or class ignored; the fleet trusted, or the own workspace untrusted;
the tags swapped, or `untrusted()` inverted; `recall_sources` holding the own workspace, or not
the fleet's; `untrusted` resolved to the trusted workspace; read agents not resolved, or
resolved by workspace instead of name; the sentence without the read agents' names; the
reserved names allowed; the readers' errors ignored at start; the key file's warning logged
twice.