boxmaker/docs/plans/M5c1/10-memoryd-honcho-client.md

# M5c-1 task 10: `memoryd`'s Honcho clients, and making sure a workspace, its peers and a session exist

**Branch:** `m5c1` (run `git switch m5c1`; `git status --short` must be empty, otherwise stop)
**Commit subject:** `memoryd: a Honcho client per workspace, and its workspace, peers and sessions made sure of`
**Log row:** the Task cell is `M5c1/10-memoryd-honcho-client`.

## Goal

Tasks 11 and 13 send turns, conclusions and fleet messages to Honcho. This task gives them what
they send through (M5c spec, sections 3 and 6):

- **A client per workspace.** One `roncho` blocking client for each of this agent's three
  workspaces (`[honcho] trusted`, `untrusted`, `fleet`), each with its own key when
  `[secrets.honcho_trusted]` (and so on) is configured. Every client is built with
  `.without_config_file()`: without it `roncho` reads `~/.config/roncho/roncho.toml` and takes an
  `api_key` from it. Timeouts: connect 5 s, read ("no byte for") 30 s; chat, which M5c-2 uses, 180
  s (the constant only, here).
- **Making sure things exist** before anything is written: the workspace, the agent's peer, the
  speakers' peers, then the session. Each is Honcho's get-or-create (`POST`: 201 when made, 200
  when it was there; both are success, and so is a 409). A session must be made before a message
  is added to it: Honcho makes a missing one on the fly, with no peers configured (recorded:
  `20-messages-add-missing-session.json`).
  - The fleet workspace is made with `reasoning`, `summary` and `dream` all `enabled: false`; the
    other two with Honcho's defaults (no configuration sent).
  - The agent's peer is made with `observe_me: false`; a speaker's peer with nothing.
  - A session holds the agent's peer (`observe_me: false, observe_others: false`) and each
    speaker's (`observe_me: true, observe_others: false`).
  - What was made sure of is remembered in memory for the run. After a restart everything is made
    sure of again, which costs one request each and changes nothing.
- **Which Honcho session a turn goes to** (spec 3): the Boxmaker session's id; a speaker new to it
  starts the next, `<id>-2`, `<id>-3`, holding every speaker so far (a Honcho session's peers are
  fixed when it is made). The numbering comes from the ledger alone: the speakers of that
  session's decided `Turn` actions, in any workspace, in the order of the decisions, each counted
  once, up to and including the turn. A restart numbers the same.
- **Errors by kind and status, never a body.** A 422 echoes the whole request, memory text
  included, and `roncho`'s `Status` and `Decode` errors carry up to 4 KiB of the response. So a
  `HonchoError` keeps only a kind (a fixed word) and a status.
- **Deleting a conclusion** (for task 13, which uses it): 204 is done, and 404 (deleted before) is
  done too.

Nothing calls this yet: task 11 builds the clients at start and sends turns. No request goes to
a real Honcho in any test: a fake Honcho on `127.0.0.1` answers from the responses the design
model recorded from rift in task 00, `crates/memoryd/tests/fixtures/honcho/`, which this task
copies in.

## Interfaces

In `crates/memoryd/src/honcho/client.rs` (the module `memoryd::honcho::client`):

```rust
pub const CONNECT: Duration = Duration::from_secs(5);
pub const READ: Duration = Duration::from_secs(30);
pub const CHAT_READ: Duration = Duration::from_secs(180);
pub struct Timeouts { pub connect: Duration, pub read: Duration } // Timeouts::WRITES = (CONNECT, READ)
pub enum HonchoError {
    Unreachable { kind: &'static str, status: Option<u16> },
    Refused { kind: &'static str, status: Option<u16> },
    KeyRefused { status: u16 },
}
impl HonchoError { pub fn kind(&self) -> &'static str; pub fn status(&self) -> Option<u16>; }
pub fn classify(error: &roncho::blocking::Error) -> HonchoError;
pub fn fleet_configuration() -> serde_json::Map<String, Value>;
pub fn session_peers(agent: &str, speakers: &[String]) -> HashMap<String, SessionPeerConfig>;
pub struct Clients { /* private */ }
impl Clients {
    pub fn new(config: &HonchoConfig, env: &dyn Fn(&str) -> Option<OsString>, timeouts: Timeouts)
        -> Result<(Clients, Vec<String>), String>;
    pub fn client(&self, workspace: Workspace) -> &roncho::blocking::Client;
    pub fn ensure_workspace(&mut self, workspace: Workspace) -> Result<(), HonchoError>;
    pub fn ensure_peer(&mut self, workspace: Workspace, peer: &str) -> Result<(), HonchoError>;
    pub fn ensure_session(&mut self, workspace: Workspace, session: &HonchoSession)
        -> Result<(), HonchoError>;
    pub fn delete_conclusion(&self, workspace: Workspace, id: &str) -> Result<(), HonchoError>;
}
```

In `crates/memoryd/src/honcho/naming.rs` (the module `memoryd::honcho::naming`):

```rust
pub struct HonchoSession { pub name: String, pub speakers: Vec<String> }
pub fn session_name(session: &SessionId, count: usize) -> String;
pub fn honcho_session<'a, I>(decisions: I, session: &SessionId, turn: u64) -> Option<HonchoSession>
where I: IntoIterator<Item = (&'a Item, &'a Action)>;
```

In `crates/memoryd/src/honcho/ledger.rs`, one new method (edit L1):
`pub fn decisions(&self) -> impl Iterator<Item = (&Item, &Action)> + '_`, every decided item and
its action (`Skip` included) in the order of the `Decided` records.

`Clients::new` returns the key files' plaintext warnings for the caller to log. Its error is one
line naming the key (`bxcore::secrets`' own text, with its pointer) or the workspace; never a
key's value.

## The errors

`classify` is given whole in the skeleton. It maps `roncho`'s error so:

| `roncho::blocking::Error` | `HonchoError` |
|---|---|
| `Connect`, `Timeout`, `Closed`, `Io` | `Unreachable`, kind `connect`, `timeout`, `closed`, `io`; no status |
| `Status` with a status of 500 or more | `Unreachable`, kind `server`, the status |
| `Status` below 500 | `Refused`, kind `status`, the status |
| `Unauthorized` (401 or 403) | `KeyRefused`, the status |
| `Redirect` | `Refused`, kind `redirect`, the status |
| `ActiveSessions` | `Refused`, kind `active-sessions`, 409 |
| `TooLarge`, `Decode`, `Stream`, `Config`, any other | `Refused`, kind `too-large`, `decode`, `stream`, `config`, `other`; no status |

`Display` (given): `Honcho is unreachable (<kind>[, HTTP <status>])`, `Honcho refused the request
(<kind>[, HTTP <status>])`, `Honcho refused the key (key-refused, HTTP <status>)`. For a create,
a `Refused` with status 409 is success (`created`); for a delete, one with 404 is.

## The requests

Exactly what the given tests expect (`Value` bodies, compared as JSON):

| Step | Request | Body |
|---|---|---|
| a workspace | `POST /v3/workspaces` | `{"id":"<name>"}`; the fleet: `{"id":"<name>","configuration":{"reasoning":{"enabled":false},"summary":{"enabled":false},"dream":{"enabled":false}}}` |
| the agent's peer | `POST /v3/workspaces/<name>/peers` | `{"id":"<agent>","configuration":{"observe_me":false}}` |
| a speaker's peer | the same | `{"id":"<speaker>"}` |
| a session | `POST /v3/workspaces/<name>/sessions` | `{"id":"<session>","peers":{"<agent>":{"observe_me":false,"observe_others":false},"<speaker>":{"observe_me":true,"observe_others":false},…}}` |
| delete a conclusion | `DELETE /v3/workspaces/<name>/conclusions/<id>` | none |

`ensure_session` (given whole) makes sure of the workspace, the agent's peer, each speaker's peer,
and then the session, each only if not yet made sure of in this run. A failed step is not
remembered, so the next call tries it again.

## Every place that changes

This is the complete list.

| Place | What happens there |
|---|---|
| `crates/memoryd/src/honcho/mod.rs` | `pub mod client;` and `pub mod naming;` (edit H1) |
| `crates/memoryd/src/honcho/ledger.rs` | `decisions` (edit L1) |
| `crates/memoryd/src/honcho/client.rs` | new: a skeleton you copy, then fill 11 bodies |
| `crates/memoryd/src/honcho/naming.rs` | new: a skeleton you copy, then fill 2 bodies |
| `crates/memoryd/tests/fixtures/honcho/` | new: the recorded responses, copied |
| everything else (`serve.rs`, `turns.rs`, `config.rs`, `Cargo.toml`, other crates) | **no change** |

## Files

(A copy command names `<file>-taskNN` where a later task gives a newer version of that file:
this task's version is the one named.)

- Copy, from `docs/plans/M5c1/files/` (the same path under it):
  - new tests: `crates/memoryd/tests/honcho_client.rs`, `crates/memoryd/tests/honcho_keys.rs`,
    `crates/memoryd/tests/honcho_naming.rs`
  - new test support: `crates/memoryd/tests/support/honcho.rs` (the fake Honcho), from
    `honcho.rs-task10` (this task's version; task 13 gives a later one)
  - the directory `crates/memoryd/tests/fixtures/honcho/` (29 JSON files and a README), whole
  - the skeletons `crates/memoryd/src/honcho/client.rs` and `crates/memoryd/src/honcho/naming.rs`
- Modify: `crates/memoryd/src/honcho/mod.rs`, `crates/memoryd/src/honcho/ledger.rs`,
  `docs/implementer-log.md`.
- Do not touch: any other source file, any other test or support file, the fixtures (never edit
  one, and never record new ones: there is no Honcho to reach from here), any `Cargo.toml`, and in
  the skeletons anything but the 13 `todo!()` bodies.

## Facts you need

- **`roncho` 1.0.0** is already a dependency (`roncho.workspace = true`, the feature `tls`, which
  brings `blocking`). Its blocking client is HTTP/1.1 over `std::net`, one request per
  connection. The calls this task uses, all on `roncho::blocking::Client`:

  | Call | Signature, and what it gives |
  |---|---|
  | `Client::builder()` | `-> ClientBuilder`; then `.without_config_file()`, `.base_url(url)`, `.workspace_id(name)`, `.connect_timeout(d)`, `.read_timeout(d)`, `.api_key(key)` (each `self -> Self`), `.build() -> Result<Client, Error>` |
  | `workspace_id` | `client.workspace_id() -> &str` |
  | `get_or_create_workspace` | `client.get_or_create_workspace(&WorkspaceCreate) -> Result<roncho::Workspace, Error>` |
  | `peer` | `client.peer(&PeerCreate) -> Result<roncho::models::Peer, Error>` |
  | `session` | `client.session(&SessionCreate) -> Result<roncho::models::Session, Error>` |
  | `delete_conclusion` | `client.delete_conclusion(id: &str) -> Result<(), Error>` |
  | `WorkspaceCreate` | `roncho::WorkspaceCreate::new(id)`, `.with_configuration(Map<String, Value>)` |
  | `PeerCreate` | `roncho::PeerCreate::new(id)`, `.with_configuration(Map<String, Value>)` |
  | `SessionCreate` | `roncho::SessionCreate::new(id)`, `.with_peers(HashMap<String, SessionPeerConfig>)` |
  | `SessionPeerConfig` | `roncho::SessionPeerConfig::new().observe_me(bool).observe_others(bool)` |

  `roncho`'s `Client` and `ClientBuilder` `Debug` show `api_key_present`, never the key; the
  client keeps the key in a `Zeroizing<String>`.
- **Keys.** `HonchoConfig.keys` (task 06) maps `honcho_trusted`, `honcho_untrusted`,
  `honcho_fleet` to a checked `SecretSource`. `bxcore::secrets::load("memoryd", name, source,
  env) -> Result<Loaded, SecretError>` reads it (`Loaded { secret: Secret, warning:
  Option<String> }`; a file source always has a warning). `Secret::expose() -> &str` is the one
  place the value is taken, to hand it to `roncho`. `SecretError`'s `Display` names the key and
  ends with `see docs/runbook.md#secret-unavailable`. `env` is `std::env::var_os` in `memoryd`;
  tests pass their own.
- **serde_json.** `serde_json::Map::new()`, `map.insert(String, Value)`, and
  `serde_json::json!({ "enabled": false })` (imported as `json`).
- **The fake Honcho** (`tests/support/honcho.rs`) answers by method and the shape of the path:
  the first create of a thing with its 201 fixture, a repeat with its 200 one; the first delete
  of a conclusion 204, a repeat 404; anything else 404. It can be set to answer every request with
  one status (a 422 echoing the request's body, as Honcho does) or to answer nothing, and it
  records every request with its headers and body.
- **The given tests.**
  - `honcho_client.rs` (13 tests): the timeouts; each client's workspace and timeouts (from
    `roncho`'s `Debug`); the fleet's configuration equal to the recorded one; the four requests of
    a session, in order and exactly; nothing asked twice in a run, a new speaker's peer and
    session only, another workspace on its own; everything asked again after a restart (200 is
    success); the fleet's workspace and a session with the agent alone; 409 on create; errors
    by kind and status, not remembered, and a timeout (the fake answers nothing, the read timeout
    is 300 ms); nothing listening; no response text in an error; `classify` for every error;
    deleting a conclusion twice.
  - `honcho_keys.rs` (3 tests, and 1 ignored that the third runs): each key sent as a bearer
    token to its own workspace only, and shown by no client, error or warning; a key that cannot
    be read; a `roncho.toml` with an `api_key` under `HOME` not read (a child process of the test
    binary, since `HOME` is the process's).
  - `honcho_naming.rs` (5 tests): the names; each new speaker, in any workspace, starting the next
    session; a skipped or undecided turn; the order of decision; a restart; `decisions`.
- **Where the delete lives.** `delete_conclusion` is here, with its two recorded answers; task 13
  calls it for an undo.

## The edits, exactly

### `crates/memoryd/src/honcho/mod.rs`

**H1.** The lines

```rust
pub mod config;
pub mod decide;
pub mod ledger;
```

become

```rust
pub mod client;
pub mod config;
pub mod decide;
pub mod ledger;
pub mod naming;
```

### `crates/memoryd/src/honcho/ledger.rs`

**L1.** The lines

```rust
    /// The items waiting to be sent, with their actions, in the order they were decided.
    pub fn outbox(&self) -> Vec<(Item, Action)> {
```

become

```rust
    /// Every decided item with its action, `Skip` included, in the order they were decided.
    pub fn decisions(&self) -> impl Iterator<Item = (&Item, &Action)> + '_ {
        self.order
            .iter()
            .filter_map(|item| Some((item, &self.entries.get(item)?.action)))
    }

    /// The items waiting to be sent, with their actions, in the order they were decided.
    pub fn outbox(&self) -> Vec<(Item, Action)> {
```

## Steps

- [ ] **1. Copy the given files.** From the repository root:

  ```sh
  mkdir -p crates/memoryd/tests/fixtures
  cp -r docs/plans/M5c1/files/crates/memoryd/tests/fixtures/honcho crates/memoryd/tests/fixtures/
  cp docs/plans/M5c1/files/crates/memoryd/tests/honcho_client.rs crates/memoryd/tests/
  cp docs/plans/M5c1/files/crates/memoryd/tests/honcho_keys.rs crates/memoryd/tests/
  cp docs/plans/M5c1/files/crates/memoryd/tests/honcho_naming.rs-task10 crates/memoryd/tests/honcho_naming.rs
  cp docs/plans/M5c1/files/crates/memoryd/tests/support/honcho.rs-task10 crates/memoryd/tests/support/honcho.rs
  ```

  `ls crates/memoryd/tests/fixtures/honcho | wc -l` prints `30`.
- [ ] **2. See them fail.** `cargo build --tests --keep-going -p memoryd`, with a shell timeout
  of 600000 ms: three test targets do not compile. `honcho_client`, 2 errors: `E0432`
  (``unresolved import `memoryd::honcho::client` ``) and `E0432` (the same for
  `memoryd::honcho::naming`). `honcho_keys`, 1 error: the `E0432` for `client`. `honcho_naming`,
  8 errors: the `E0432` for `naming`, and seven `E0599` (``no method named `decisions` found for
  struct `memoryd::honcho::ledger::Ledger` ``). Every other target compiles.
- [ ] **3. The modules.** Make edits H1 and L1, then copy the two skeletons:

  ```sh
  cp docs/plans/M5c1/files/crates/memoryd/src/honcho/client.rs crates/memoryd/src/honcho/
  cp docs/plans/M5c1/files/crates/memoryd/src/honcho/naming.rs crates/memoryd/src/honcho/
  ```

  `cargo check -p memoryd`: no error, and 26 warnings, all from the unfilled bodies: ``unused
  import: `json` ``; ``unused imports: `PeerCreate` and `WorkspaceCreate` ``; twenty-two `unused
  variable` (`workspace` five times, `session` twice, and `agent`, `config`, `count`,
  `decisions`, `env`, `id`, `key`, `name`, `peer`, `result`, `speakers`, `timeouts`, `turn`,
  `url`, `warnings` once each); ``fields `trusted`, `untrusted`, and `fleet` are never read``;
  and ``variants `Workspace` and `Peer` are never constructed``. They go as you fill the bodies.
- [ ] **4. Fill the thirteen bodies, one at a time, in the order of each file**, each from its
  comment, and run `cargo check -p memoryd` after each. In `client.rs`: `kind`, `status`,
  `fleet_configuration`, `session_peers`, `created`, `client` (the free function), `key`,
  `Clients::client`, `ensure_workspace`, `ensure_peer`, `delete_conclusion`. In `naming.rs`:
  `session_name`, `honcho_session`. When a body is filled, delete its instruction comment (the
  `//` lines inside the body) and keep the function's doc comment (`///`) as it is. Change
  nothing else: `classify`, `Display`, `Clients::new` and `ensure_session` are given. After the
  last, `cargo check -p memoryd` prints no warning.
- [ ] **5. The tests.** With a shell timeout of 600000 ms, and every test passes (13, 3 and 5; one
  test is ignored: the child the third test of `honcho_keys` runs):

  ```sh
  cargo test -p memoryd --test honcho_client --test honcho_keys --test honcho_naming
  ```
- [ ] **6. Check.** Three commands:
  - `grep -c 'without_config_file' crates/memoryd/src/honcho/client.rs` prints `1`.
  - `grep -n 'expose()' crates/memoryd/src -r` prints exactly one line, in `client` (the free
    function): the one place a key's value is taken.
  - `git diff --stat -- crates/bxcore crates/loopd crates/brokerd crates/gatewayd crates/bxctl
    Cargo.toml Cargo.lock crates/memoryd/Cargo.toml crates/memoryd/src/serve.rs` prints nothing.
- [ ] **7. Run the gate.** `cargo fmt --all`, then `make gate`, with a shell timeout of 600000 ms.
  Expected last line: `gate: ok`, with 1278 tests: 1257 before this task, plus 21 new ones (13 in
  `honcho_client.rs`, 3 in `honcho_keys.rs`, 5 in `honcho_naming.rs`).
- [ ] **8. Log and commit.** Add your row to `docs/implementer-log.md`; its Task cell is
  `M5c1/10-memoryd-honcho-client`. `git status --short` lists `honcho/mod.rs`, `honcho/ledger.rs`
  and your log as modified, and `honcho/client.rs`, `honcho/naming.rs`, the three test files,
  `tests/support/honcho.rs` and `tests/fixtures/` as new. Stage with
  `git add crates/memoryd docs/implementer-log.md Cargo.lock`, then commit as `AGENTS.md` says.
  `Cargo.lock` does not change in this task; the `git add` is harmless.

## Done when

- `make gate` prints `gate: ok` with 1278 tests.
- `git diff HEAD~1 --stat` lists only the files of step 8.

## Stop and report if

- A "before" block of H1 or L1 is not in its file exactly as shown, or is there more than once.
- Step 1 does not give 30 files, or step 2 or 3 shows a different result: other test targets,
  errors of another kind, or other warnings.
- `cargo check` reports an error after a body you filled, and the comment does not tell you how
  to fix it. Quote the error, and stop.
- A comment in a skeleton does not fit the code under it, or disagrees with this task's tables.
  Quote both and stop.
- After step 4 a test fails although each body follows its comment. Do not change a test, the
  fake, a fixture or the skeletons' given code: stop, and give the failing tests' names and their
  messages.
- A test seems to need a fixture that is not there, or a request to a real Honcho. It does not;
  stop and say why you think so. Never record a response.
- The fix seems to need a change anywhere else: `serve.rs`, `turns.rs`, a `Cargo.toml`, another
  crate. It does not; stop and say why you think so.
- A test outside the new files fails in the gate: check once whether it fails without your
  change, then stop and report both results.
- The gate passes with a count other than 1278.

## What the given tests pin

Each of these changes to the finished code makes at least one given test fail (44 tried, one
cannot change behaviour): either timeout ignored or changed; `roncho.toml` read; each error
mapped to the other side (a 5xx refused, a 401 refused, a timeout, a connect failure or a closed
socket refused, a bad body unreachable); a 409 on create an error, or any refusal on create
taken as success; a 404 on delete an error; the fleet's settings on another workspace, missing
one, or on; the agent observed, or every peer unobserved; the workspace, a peer or a session not
remembered, or remembered for every workspace, or before it was made; the session made without
its workspace, its agent or its speakers first; the agent observed or left out of a session, the
speakers unobserved or observing others; a key sent to the wrong workspace or dropped; a key
file's warning dropped; a key that cannot be read taken as no key; the wrong client for a
workspace; an error shown without its status; a session named `<id>-1`; a speaker counted
twice; another session's speakers counted; the speakers of later turns counted; `decisions`
without the skipped items or in reverse.

## After this task

Task 11 builds the `Clients` at start (after the ledger, logging the warnings, a key that cannot
be read stopping `memoryd` with `secret-unavailable`), and sends turns with `ensure_session` and
`honcho_session`. Task 13 uses `delete_conclusion`, and makes the fleet's sessions with no
speakers.