kyle/boxmaker
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

M5c-1 plan: task 10, the Honcho clients and the ensure steps in memoryd

Browse Source 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
kyle
2026-09-28 08:05:57 -07:00
co-authored by Claude Opus 5.5
parent 2926031b4d
commit a359f34a73
7 changed files with 1693 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/plans/M5c1/10-memoryd-honcho-client.md
+362
View File
@@ -0,0 +1,362 @@
# M5c-1 task 10: `memoryd`'s Honcho clients, and making sure a workspace, its peers and a session exist
**Branch:** `m5c1` (run `git switch m5c1`; `git status --short` must be empty, otherwise stop)
**Commit subject:** `memoryd: a Honcho client per workspace, and its workspace, peers and sessions made sure of`
**Log row:** the Task cell is `M5c1/10-memoryd-honcho-client`.
## Goal
Tasks 11 and 13 send turns, conclusions and fleet messages to Honcho. This task gives them what
they send through (M5c spec, sections 3 and 6):
- **A client per workspace.** One `roncho` blocking client for each of this agent's three
workspaces (`[honcho] trusted`, `untrusted`, `fleet`), each with its own key when
`[secrets.honcho_trusted]` (and so on) is configured. Every client is built with
`.without_config_file()`: without it `roncho` reads `~/.config/roncho/roncho.toml` and takes an
`api_key` from it. Timeouts: connect 5 s, read ("no byte for") 30 s; chat, which M5c-2 uses, 180
s (the constant only, here).
- **Making sure things exist** before anything is written: the workspace, the agent's peer, the
speakers' peers, then the session. Each is Honcho's get-or-create (`POST`: 201 when made, 200
when it was there; both are success, and so is a 409). A session must be made before a message
is added to it: Honcho makes a missing one on the fly, with no peers configured (recorded:
`20-messages-add-missing-session.json`).
- The fleet workspace is made with `reasoning`, `summary` and `dream` all `enabled: false`; the
other two with Honcho's defaults (no configuration sent).
- The agent's peer is made with `observe_me: false`; a speaker's peer with nothing.
- A session holds the agent's peer (`observe_me: false, observe_others: false`) and each
speaker's (`observe_me: true, observe_others: false`).
- What was made sure of is remembered in memory for the run. After a restart everything is made
sure of again, which costs one request each and changes nothing.
- **Which Honcho session a turn goes to** (spec 3): the Boxmaker session's id; a speaker new to it
starts the next, `<id>-2`, `<id>-3`, holding every speaker so far (a Honcho session's peers are
fixed when it is made). The numbering comes from the ledger alone: the speakers of that
session's decided `Turn` actions, in any workspace, in the order of the decisions, each counted
once, up to and including the turn. A restart numbers the same.
- **Errors by kind and status, never a body.** A 422 echoes the whole request, memory text
included, and `roncho`'s `Status` and `Decode` errors carry up to 4 KiB of the response. So a
`HonchoError` keeps only a kind (a fixed word) and a status.
- **Deleting a conclusion** (for task 13, which uses it): 204 is done, and 404 (deleted before) is
done too.
Nothing calls this yet: task 11 builds the clients at start and sends turns. No request goes to
a real Honcho in any test: a fake Honcho on `127.0.0.1` answers from the responses the design
model recorded from rift in task 00, `crates/memoryd/tests/fixtures/honcho/`, which this task
copies in.
## Interfaces
In `crates/memoryd/src/honcho/client.rs` (the module `memoryd::honcho::client`):
```rust
pub const CONNECT: Duration = Duration::from_secs(5);
pub const READ: Duration = Duration::from_secs(30);
pub const CHAT_READ: Duration = Duration::from_secs(180);
pub struct Timeouts { pub connect: Duration, pub read: Duration } // Timeouts::WRITES = (CONNECT, READ)
pub enum HonchoError {
Unreachable { kind: &'static str, status: Option<u16> },
Refused { kind: &'static str, status: Option<u16> },
KeyRefused { status: u16 },
}
impl HonchoError { pub fn kind(&self) -> &'static str; pub fn status(&self) -> Option<u16>; }
pub fn classify(error: &roncho::blocking::Error) -> HonchoError;
pub fn fleet_configuration() -> serde_json::Map<String, Value>;
pub fn session_peers(agent: &str, speakers: &[String]) -> HashMap<String, SessionPeerConfig>;
pub struct Clients { /* private */ }
impl Clients {
pub fn new(config: &HonchoConfig, env: &dyn Fn(&str) -> Option<OsString>, timeouts: Timeouts)
-> Result<(Clients, Vec<String>), String>;
pub fn client(&self, workspace: Workspace) -> &roncho::blocking::Client;
pub fn ensure_workspace(&mut self, workspace: Workspace) -> Result<(), HonchoError>;
pub fn ensure_peer(&mut self, workspace: Workspace, peer: &str) -> Result<(), HonchoError>;
pub fn ensure_session(&mut self, workspace: Workspace, session: &HonchoSession)
-> Result<(), HonchoError>;
pub fn delete_conclusion(&self, workspace: Workspace, id: &str) -> Result<(), HonchoError>;
}
```
In `crates/memoryd/src/honcho/naming.rs` (the module `memoryd::honcho::naming`):
```rust
pub struct HonchoSession { pub name: String, pub speakers: Vec<String> }
pub fn session_name(session: &SessionId, count: usize) -> String;
pub fn honcho_session<'a, I>(decisions: I, session: &SessionId, turn: u64) -> Option<HonchoSession>
where I: IntoIterator<Item = (&'a Item, &'a Action)>;
```
In `crates/memoryd/src/honcho/ledger.rs`, one new method (edit L1):
`pub fn decisions(&self) -> impl Iterator<Item = (&Item, &Action)> + '_`, every decided item and
its action (`Skip` included) in the order of the `Decided` records.
`Clients::new` returns the key files' plaintext warnings for the caller to log. Its error is one
line naming the key (`bxcore::secrets`' own text, with its pointer) or the workspace; never a
key's value.
## The errors
`classify` is given whole in the skeleton. It maps `roncho`'s error so:
| `roncho::blocking::Error` | `HonchoError` |
|---|---|
| `Connect`, `Timeout`, `Closed`, `Io` | `Unreachable`, kind `connect`, `timeout`, `closed`, `io`; no status |
| `Status` with a status of 500 or more | `Unreachable`, kind `server`, the status |
| `Status` below 500 | `Refused`, kind `status`, the status |
| `Unauthorized` (401 or 403) | `KeyRefused`, the status |
| `Redirect` | `Refused`, kind `redirect`, the status |
| `ActiveSessions` | `Refused`, kind `active-sessions`, 409 |
| `TooLarge`, `Decode`, `Stream`, `Config`, any other | `Refused`, kind `too-large`, `decode`, `stream`, `config`, `other`; no status |
`Display` (given): `Honcho is unreachable (<kind>[, HTTP <status>])`, `Honcho refused the request
(<kind>[, HTTP <status>])`, `Honcho refused the key (key-refused, HTTP <status>)`. For a create,
a `Refused` with status 409 is success (`created`); for a delete, one with 404 is.
## The requests
Exactly what the given tests expect (`Value` bodies, compared as JSON):
| Step | Request | Body |
|---|---|---|
| a workspace | `POST /v3/workspaces` | `{"id":"<name>"}`; the fleet: `{"id":"<name>","configuration":{"reasoning":{"enabled":false},"summary":{"enabled":false},"dream":{"enabled":false}}}` |
| the agent's peer | `POST /v3/workspaces/<name>/peers` | `{"id":"<agent>","configuration":{"observe_me":false}}` |
| a speaker's peer | the same | `{"id":"<speaker>"}` |
| a session | `POST /v3/workspaces/<name>/sessions` | `{"id":"<session>","peers":{"<agent>":{"observe_me":false,"observe_others":false},"<speaker>":{"observe_me":true,"observe_others":false},…}}` |
| delete a conclusion | `DELETE /v3/workspaces/<name>/conclusions/<id>` | none |
`ensure_session` (given whole) makes sure of the workspace, the agent's peer, each speaker's peer,
and then the session, each only if not yet made sure of in this run. A failed step is not
remembered, so the next call tries it again.
## Every place that changes
This is the complete list.
| Place | What happens there |
|---|---|
| `crates/memoryd/src/honcho/mod.rs` | `pub mod client;` and `pub mod naming;` (edit H1) |
| `crates/memoryd/src/honcho/ledger.rs` | `decisions` (edit L1) |
| `crates/memoryd/src/honcho/client.rs` | new: a skeleton you copy, then fill 11 bodies |
| `crates/memoryd/src/honcho/naming.rs` | new: a skeleton you copy, then fill 2 bodies |
| `crates/memoryd/tests/fixtures/honcho/` | new: the recorded responses, copied |
| everything else (`serve.rs`, `turns.rs`, `config.rs`, `Cargo.toml`, other crates) | **no change** |
## Files
- Copy, from `docs/plans/M5c1/files/` (the same path under it):
- new tests: `crates/memoryd/tests/honcho_client.rs`, `crates/memoryd/tests/honcho_keys.rs`,
`crates/memoryd/tests/honcho_naming.rs`
- new test support: `crates/memoryd/tests/support/honcho.rs` (the fake Honcho)
- the directory `crates/memoryd/tests/fixtures/honcho/` (29 JSON files and a README), whole
- the skeletons `crates/memoryd/src/honcho/client.rs` and `crates/memoryd/src/honcho/naming.rs`
- Modify: `crates/memoryd/src/honcho/mod.rs`, `crates/memoryd/src/honcho/ledger.rs`,
`docs/implementer-log.md`.
- Do not touch: any other source file, any other test or support file, the fixtures (never edit
one, and never record new ones: there is no Honcho to reach from here), any `Cargo.toml`, and in
the skeletons anything but the 13 `todo!()` bodies.
## Facts you need
- **`roncho` 1.0.0** is already a dependency (`roncho.workspace = true`, the feature `tls`, which
brings `blocking`). Its blocking client is HTTP/1.1 over `std::net`, one request per
connection. The calls this task uses, all on `roncho::blocking::Client`:
| Call | Signature, and what it gives |
|---|---|
| `Client::builder()` | `-> ClientBuilder`; then `.without_config_file()`, `.base_url(url)`, `.workspace_id(name)`, `.connect_timeout(d)`, `.read_timeout(d)`, `.api_key(key)` (each `self -> Self`), `.build() -> Result<Client, Error>` |
| `workspace_id` | `client.workspace_id() -> &str` |
| `get_or_create_workspace` | `client.get_or_create_workspace(&WorkspaceCreate) -> Result<roncho::Workspace, Error>` |
| `peer` | `client.peer(&PeerCreate) -> Result<roncho::models::Peer, Error>` |
| `session` | `client.session(&SessionCreate) -> Result<roncho::models::Session, Error>` |
| `delete_conclusion` | `client.delete_conclusion(id: &str) -> Result<(), Error>` |
| `WorkspaceCreate` | `roncho::WorkspaceCreate::new(id)`, `.with_configuration(Map<String, Value>)` |
| `PeerCreate` | `roncho::PeerCreate::new(id)`, `.with_configuration(Map<String, Value>)` |
| `SessionCreate` | `roncho::SessionCreate::new(id)`, `.with_peers(HashMap<String, SessionPeerConfig>)` |
| `SessionPeerConfig` | `roncho::SessionPeerConfig::new().observe_me(bool).observe_others(bool)` |
`roncho`'s `Client` and `ClientBuilder` `Debug` show `api_key_present`, never the key; the
client keeps the key in a `Zeroizing<String>`.
- **Keys.** `HonchoConfig.keys` (task 06) maps `honcho_trusted`, `honcho_untrusted`,
`honcho_fleet` to a checked `SecretSource`. `bxcore::secrets::load("memoryd", name, source,
env) -> Result<Loaded, SecretError>` reads it (`Loaded { secret: Secret, warning:
Option<String> }`; a file source always has a warning). `Secret::expose() -> &str` is the one
place the value is taken, to hand it to `roncho`. `SecretError`'s `Display` names the key and
ends with `see docs/runbook.md#secret-unavailable`. `env` is `std::env::var_os` in `memoryd`;
tests pass their own.
- **serde_json.** `serde_json::Map::new()`, `map.insert(String, Value)`, and
`serde_json::json!({ "enabled": false })` (imported as `json`).
- **The fake Honcho** (`tests/support/honcho.rs`) answers by method and the shape of the path:
the first create of a thing with its 201 fixture, a repeat with its 200 one; the first delete
of a conclusion 204, a repeat 404; anything else 404. It can be set to answer every request with
one status (a 422 echoing the request's body, as Honcho does) or to answer nothing, and it
records every request with its headers and body.
- **The given tests.**
- `honcho_client.rs` (13 tests): the timeouts; each client's workspace and timeouts (from
`roncho`'s `Debug`); the fleet's configuration equal to the recorded one; the four requests of
a session, in order and exactly; nothing asked twice in a run, a new speaker's peer and
session only, another workspace on its own; everything asked again after a restart (200 is
success); the fleet's workspace and a session with the agent alone; 409 on create; errors
by kind and status, not remembered, and a timeout (the fake answers nothing, the read timeout
is 300 ms); nothing listening; no response text in an error; `classify` for every error;
deleting a conclusion twice.
- `honcho_keys.rs` (3 tests, and 1 ignored that the third runs): each key sent as a bearer
token to its own workspace only, and shown by no client, error or warning; a key that cannot
be read; a `roncho.toml` with an `api_key` under `HOME` not read (a child process of the test
binary, since `HOME` is the process's).
- `honcho_naming.rs` (5 tests): the names; each new speaker, in any workspace, starting the next
session; a skipped or undecided turn; the order of decision; a restart; `decisions`.
- **Where the delete lives.** `delete_conclusion` is here, with its two recorded answers; task 13
calls it for an undo.
## The edits, exactly
### `crates/memoryd/src/honcho/mod.rs`
**H1.** The lines
```rust
pub mod config;
pub mod decide;
pub mod ledger;
```
become
```rust
pub mod client;
pub mod config;
pub mod decide;
pub mod ledger;
pub mod naming;
```
### `crates/memoryd/src/honcho/ledger.rs`
**L1.** The lines
```rust
/// The items waiting to be sent, with their actions, in the order they were decided.
pub fn outbox(&self) -> Vec<(Item, Action)> {
```
become
```rust
/// Every decided item with its action, `Skip` included, in the order they were decided.
pub fn decisions(&self) -> impl Iterator<Item = (&Item, &Action)> + '_ {
self.order
.iter()
.filter_map(|item| Some((item, &self.entries.get(item)?.action)))
}
/// The items waiting to be sent, with their actions, in the order they were decided.
pub fn outbox(&self) -> Vec<(Item, Action)> {
```
## Steps
- [ ] **1. Copy the given files.** From the repository root:
```sh
mkdir -p crates/memoryd/tests/fixtures
cp -r docs/plans/M5c1/files/crates/memoryd/tests/fixtures/honcho crates/memoryd/tests/fixtures/
cp docs/plans/M5c1/files/crates/memoryd/tests/honcho_client.rs crates/memoryd/tests/
cp docs/plans/M5c1/files/crates/memoryd/tests/honcho_keys.rs crates/memoryd/tests/
cp docs/plans/M5c1/files/crates/memoryd/tests/honcho_naming.rs crates/memoryd/tests/
cp docs/plans/M5c1/files/crates/memoryd/tests/support/honcho.rs crates/memoryd/tests/support/
```
`ls crates/memoryd/tests/fixtures/honcho | wc -l` prints `30`.
- [ ] **2. See them fail.** `cargo build --tests --keep-going -p memoryd`, with a shell timeout
of 600000 ms: three test targets do not compile. `honcho_client`, 2 errors: `E0432`
(``unresolved import `memoryd::honcho::client` ``) and `E0432` (the same for
`memoryd::honcho::naming`). `honcho_keys`, 1 error: the `E0432` for `client`. `honcho_naming`,
8 errors: the `E0432` for `naming`, and seven `E0599` (``no method named `decisions` found for
struct `memoryd::honcho::ledger::Ledger` ``). Every other target compiles.
- [ ] **3. The modules.** Make edits H1 and L1, then copy the two skeletons:
```sh
cp docs/plans/M5c1/files/crates/memoryd/src/honcho/client.rs crates/memoryd/src/honcho/
cp docs/plans/M5c1/files/crates/memoryd/src/honcho/naming.rs crates/memoryd/src/honcho/
```
`cargo check -p memoryd`: no error, and 26 warnings, all from the unfilled bodies: ``unused
import: `json` ``; ``unused imports: `PeerCreate` and `WorkspaceCreate` ``; twenty-two `unused
variable` (`workspace` five times, `session` twice, and `agent`, `config`, `count`,
`decisions`, `env`, `id`, `key`, `name`, `peer`, `result`, `speakers`, `timeouts`, `turn`,
`url`, `warnings` once each); ``fields `trusted`, `untrusted`, and `fleet` are never read``;
and ``variants `Workspace` and `Peer` are never constructed``. They go as you fill the bodies.
- [ ] **4. Fill the thirteen bodies, one at a time, in the order of each file**, each from its
comment, and run `cargo check -p memoryd` after each. In `client.rs`: `kind`, `status`,
`fleet_configuration`, `session_peers`, `created`, `client` (the free function), `key`,
`Clients::client`, `ensure_workspace`, `ensure_peer`, `delete_conclusion`. In `naming.rs`:
`session_name`, `honcho_session`. When a body is filled, delete its instruction comment (the
`//` lines inside the body) and keep the function's doc comment (`///`) as it is. Change
nothing else: `classify`, `Display`, `Clients::new` and `ensure_session` are given. After the
last, `cargo check -p memoryd` prints no warning.
- [ ] **5. The tests.** With a shell timeout of 600000 ms, and every test passes (13, 3 and 5; one
test is ignored: the child the third test of `honcho_keys` runs):
```sh
cargo test -p memoryd --test honcho_client --test honcho_keys --test honcho_naming
```
- [ ] **6. Check.** Three commands:
- `grep -c 'without_config_file' crates/memoryd/src/honcho/client.rs` prints `1`.
- `grep -n 'expose()' crates/memoryd/src -r` prints exactly one line, in `client` (the free
function): the one place a key's value is taken.
- `git diff --stat -- crates/bxcore crates/loopd crates/brokerd crates/gatewayd crates/bxctl
Cargo.toml Cargo.lock crates/memoryd/Cargo.toml crates/memoryd/src/serve.rs` prints nothing.
- [ ] **7. Run the gate.** `cargo fmt --all`, then `make gate`, with a shell timeout of 600000 ms.
Expected last line: `gate: ok`, with 1278 tests: 1257 before this task, plus 21 new ones (13 in
`honcho_client.rs`, 3 in `honcho_keys.rs`, 5 in `honcho_naming.rs`).
- [ ] **8. Log and commit.** Add your row to `docs/implementer-log.md`; its Task cell is
`M5c1/10-memoryd-honcho-client`. `git status --short` lists `honcho/mod.rs`, `honcho/ledger.rs`
and your log as modified, and `honcho/client.rs`, `honcho/naming.rs`, the three test files,
`tests/support/honcho.rs` and `tests/fixtures/` as new. Stage with
`git add crates/memoryd docs/implementer-log.md Cargo.lock`, then commit as `AGENTS.md` says.
`Cargo.lock` does not change in this task; the `git add` is harmless.
## Done when
- `make gate` prints `gate: ok` with 1278 tests.
- `git diff HEAD~1 --stat` lists only the files of step 8.
## Stop and report if
- A "before" block of H1 or L1 is not in its file exactly as shown, or is there more than once.
- Step 1 does not give 30 files, or step 2 or 3 shows a different result: other test targets,
errors of another kind, or other warnings.
- `cargo check` reports an error after a body you filled, and the comment does not tell you how
to fix it. Quote the error, and stop.
- A comment in a skeleton does not fit the code under it, or disagrees with this task's tables.
Quote both and stop.
- After step 4 a test fails although each body follows its comment. Do not change a test, the
fake, a fixture or the skeletons' given code: stop, and give the failing tests' names and their
messages.
- A test seems to need a fixture that is not there, or a request to a real Honcho. It does not;
stop and say why you think so. Never record a response.
- The fix seems to need a change anywhere else: `serve.rs`, `turns.rs`, a `Cargo.toml`, another
crate. It does not; stop and say why you think so.
- A test outside the new files fails in the gate: check once whether it fails without your
change, then stop and report both results.
- The gate passes with a count other than 1278.
## What the given tests pin
Each of these changes to the finished code makes at least one given test fail (44 tried, one
cannot change behaviour): either timeout ignored or changed; `roncho.toml` read; each error
mapped to the other side (a 5xx refused, a 401 refused, a timeout, a connect failure or a closed
socket refused, a bad body unreachable); a 409 on create an error, or any refusal on create
taken as success; a 404 on delete an error; the fleet's settings on another workspace, missing
one, or on; the agent observed, or every peer unobserved; the workspace, a peer or a session not
remembered, or remembered for every workspace, or before it was made; the session made without
its workspace, its agent or its speakers first; the agent observed or left out of a session, the
speakers unobserved or observing others; a key sent to the wrong workspace or dropped; a key
file's warning dropped; a key that cannot be read taken as no key; the wrong client for a
workspace; an error shown without its status; a session named `<id>-1`; a speaker counted
twice; another session's speakers counted; the speakers of later turns counted; `decisions`
without the skipped items or in reverse.
## After this task
Task 11 builds the `Clients` at start (after the ledger, logging the warnings, a key that cannot
be read stopping `memoryd` with `secret-unavailable`), and sends turns with `ensure_session` and
`honcho_session`. Task 13 uses `delete_conclusion`, and makes the fleet's sessions with no
speakers.
docs/plans/M5c1/files/crates/memoryd/src/honcho/client.rs
+272
View File
@@ -0,0 +1,272 @@
//! This agent's three Honcho workspaces through `roncho`'s blocking client (M5c spec, section 6):
//! one client per workspace, with its own key, and the steps that make sure a workspace, a peer
//! and a session exist before anything is written to them. Nothing here keeps or logs a response
//! body or a key: an error is its kind and its status.
use std::collections::{HashMap, HashSet};
use std::ffi::OsString;
use std::time::Duration;
use bxcore::secrets;
use roncho::blocking::{Client, Error};
use roncho::{PeerCreate, SessionCreate, SessionPeerConfig, WorkspaceCreate};
use serde_json::{Map, Value, json};
use crate::honcho::config::{FLEET_KEY, HonchoConfig, TRUSTED_KEY, UNTRUSTED_KEY};
use crate::honcho::ledger::Workspace;
use crate::honcho::naming::HonchoSession;
/// How long a connection may take, and a read may go without a byte.
pub const CONNECT: Duration = Duration::from_secs(5);
pub const READ: Duration = Duration::from_secs(30);
/// A chat's read: a non-streamed chat sends nothing until it is done (M5c-2's `ask_memory`).
pub const CHAT_READ: Duration = Duration::from_secs(180);
/// The timeouts a client is built with. Tests shorten them.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Timeouts {
pub connect: Duration,
pub read: Duration,
}
impl Timeouts {
pub const WRITES: Timeouts = Timeouts {
connect: CONNECT,
read: READ,
};
}
/// Why a call to Honcho failed: its kind and status only, never the response's body.
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum HonchoError {
/// No answer that counts: no connection, a timeout, a closed socket, or a 5xx. Try again later.
Unreachable {
kind: &'static str,
status: Option<u16>,
},
/// Honcho answered, and will answer the same again: a 4xx, a redirect, or a body that is not
/// what `roncho` expects.
Refused {
kind: &'static str,
status: Option<u16>,
},
/// The workspace's key was refused (401 or 403).
KeyRefused { status: u16 },
}
impl HonchoError {
pub fn kind(&self) -> &'static str {
// `Unreachable { kind, .. }` and `Refused { kind, .. }` (one arm, `|`): `kind`;
// `KeyRefused { .. }`: `"key-refused"`.
todo!()
}
pub fn status(&self) -> Option<u16> {
// `Unreachable { status, .. }` and `Refused { status, .. }` (one arm): `*status`;
// `KeyRefused { status }`: `Some(*status)`.
todo!()
}
}
impl std::fmt::Display for HonchoError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
let what = match self {
HonchoError::Unreachable { .. } => "Honcho is unreachable",
HonchoError::Refused { .. } => "Honcho refused the request",
HonchoError::KeyRefused { .. } => "Honcho refused the key",
};
match self.status() {
Some(status) => write!(f, "{what} ({}, HTTP {status})", self.kind()),
None => write!(f, "{what} ({})", self.kind()),
}
}
}
impl std::error::Error for HonchoError {}
/// `roncho`'s error as a `HonchoError`. Its text is dropped: a `Status` or `Decode` error carries
/// up to 4 KiB of the response, which can echo memory text (a 422 echoes the whole input).
pub fn classify(error: &Error) -> HonchoError {
let unreachable = |kind, status| HonchoError::Unreachable { kind, status };
let refused = |kind, status| HonchoError::Refused { kind, status };
match error {
Error::Connect(_) => unreachable("connect", None),
Error::Timeout => unreachable("timeout", None),
Error::Closed => unreachable("closed", None),
Error::Io(_) => unreachable("io", None),
Error::Status { status, .. } if *status >= 500 => unreachable("server", Some(*status)),
Error::Status { status, .. } => refused("status", Some(*status)),
Error::Unauthorized { status } => HonchoError::KeyRefused { status: *status },
Error::Redirect { status } => refused("redirect", Some(*status)),
Error::ActiveSessions { .. } => refused("active-sessions", Some(409)),
Error::TooLarge => refused("too-large", None),
Error::Decode(_) => refused("decode", None),
Error::Stream(_) => refused("stream", None),
Error::Config(_) => refused("config", None),
_ => refused("other", None),
}
}
/// What has been made sure of in one workspace during this run.
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
enum Ensured {
Workspace,
Peer(String),
Session(String),
}
/// The three clients, and what they have made sure of since `memoryd` started. After a restart
/// everything is made sure of again: each step is get-or-create, so that costs a request, no more.
#[derive(Debug)]
pub struct Clients {
trusted: Client,
untrusted: Client,
fleet: Client,
agent: String,
ensured: HashSet<(Workspace, Ensured)>,
}
/// The fleet workspace's configuration: nothing reasons over the fleet's notes (M5c spec,
/// section 6). All three are set, since Honcho checks each on its own.
pub fn fleet_configuration() -> Map<String, Value> {
// A `Map::new()`; for each feature of `["reasoning", "summary", "dream"]`, insert
// `feature.to_string()` -> `json!({ "enabled": false })`. Return the map.
todo!()
}
/// A session's peers: the agent's, unobserved, and each speaker's, observed; no peer forms a
/// picture of the others.
pub fn session_peers(agent: &str, speakers: &[String]) -> HashMap<String, SessionPeerConfig> {
// `HashMap::new()`; insert `agent.to_string()` -> `SessionPeerConfig::new()
// .observe_me(false).observe_others(false)`; then for each speaker, `speaker.clone()` ->
// `SessionPeerConfig::new().observe_me(true).observe_others(false)`. Return the map.
todo!()
}
/// A create that found the thing there (409) did what it was for.
fn created<T>(result: Result<T, Error>) -> Result<(), HonchoError> {
// `Ok(_)` -> `Ok(())`. `Err(e)`: `match classify(&e)`: `HonchoError::Refused { status:
// Some(409), .. }` -> `Ok(())`; any other `other` -> `Err(other)`.
todo!()
}
/// One workspace's client: no `roncho.toml` read, the timeouts given, the key when there is one.
fn client(
url: &str,
workspace: &str,
key: Option<&secrets::Secret>,
timeouts: Timeouts,
) -> Result<Client, String> {
// `let mut builder = Client::builder()` then, chained: `.without_config_file()`,
// `.base_url(url)`, `.workspace_id(workspace)`, `.connect_timeout(timeouts.connect)`,
// `.read_timeout(timeouts.read)`. When `key` is `Some(key)`: `builder = builder.api_key(
// key.expose())`. Then `builder.build()`, its error `e` mapped to
// `format!("the Honcho client for {workspace}: {e}")`.
todo!()
}
/// The key named `name` in `[secrets]`, if one is configured, read now.
fn key(
config: &HonchoConfig,
name: &str,
env: &dyn Fn(&str) -> Option<OsString>,
warnings: &mut Vec<String>,
) -> Result<Option<secrets::Secret>, String> {
// `let Some(source) = config.keys.get(name) else { return Ok(None) };`. Then
// `secrets::load("memoryd", name, source, env)`, its error mapped with `e.to_string()` and `?`,
// into `loaded`. `warnings.extend(loaded.warning);` then `Ok(Some(loaded.secret))`.
todo!()
}
impl Clients {
/// A client for each of the three workspaces, each with its key when one is configured. The
/// warnings are the key files' plaintext warnings, for the caller to log. An error names the
/// key or the workspace, never a key's value.
pub fn new(
config: &HonchoConfig,
env: &dyn Fn(&str) -> Option<OsString>,
timeouts: Timeouts,
) -> Result<(Clients, Vec<String>), String> {
let mut warnings = Vec::new();
let trusted_key = key(config, TRUSTED_KEY, env, &mut warnings)?;
let untrusted_key = key(config, UNTRUSTED_KEY, env, &mut warnings)?;
let fleet_key = key(config, FLEET_KEY, env, &mut warnings)?;
let clients = Clients {
trusted: client(&config.url, &config.trusted, trusted_key.as_ref(), timeouts)?,
untrusted: client(
&config.url,
&config.untrusted,
untrusted_key.as_ref(),
timeouts,
)?,
fleet: client(&config.url, &config.fleet, fleet_key.as_ref(), timeouts)?,
agent: config.agent.clone(),
ensured: HashSet::new(),
};
Ok((clients, warnings))
}
/// The client of `workspace`.
pub fn client(&self, workspace: Workspace) -> &Client {
// `match workspace`: `Trusted` -> `&self.trusted`, `Untrusted` -> `&self.untrusted`,
// `Fleet` -> `&self.fleet`.
todo!()
}
/// Make sure `workspace` exists; the fleet one is made with `fleet_configuration`, the other
/// two with Honcho's defaults.
pub fn ensure_workspace(&mut self, workspace: Workspace) -> Result<(), HonchoError> {
// `let key = (workspace, Ensured::Workspace);`; if `self.ensured.contains(&key)`, return
// `Ok(())`. `let client = self.client(workspace);` and
// `let mut create = WorkspaceCreate::new(client.workspace_id());`;
// `if workspace == Workspace::Fleet`,
// `create = create.with_configuration(fleet_configuration());`. Then
// `created(client.get_or_create_workspace(&create))?;`, `self.ensured.insert(key);`,
// `Ok(())`.
todo!()
}
/// Make sure peer `peer` exists in `workspace`. The agent's own peer is made unobserved
/// (`observe_me` false): Honcho draws no conclusions from the agent's own output.
pub fn ensure_peer(&mut self, workspace: Workspace, peer: &str) -> Result<(), HonchoError> {
// `let key = (workspace, Ensured::Peer(peer.to_string()));`; if already there, return
// `Ok(())`. `let mut create = PeerCreate::new(peer);`. `if peer == self.agent`: `let mut
// configuration = Map::new();`, insert `"observe_me".to_string()` -> `json!(false)`, and
// `create = create.with_configuration(configuration);`. Then
// `created(self.client(workspace).peer(&create))?;`, `self.ensured.insert(key);`, `Ok(())`.
todo!()
}
/// Make sure `session` exists in `workspace`, with the agent's peer and its speakers' peers,
/// after the workspace and every peer. A session must exist before a message is added to
/// it: Honcho makes a missing one on the fly, with no peers configured.
pub fn ensure_session(
&mut self,
workspace: Workspace,
session: &HonchoSession,
) -> Result<(), HonchoError> {
self.ensure_workspace(workspace)?;
let agent = self.agent.clone();
self.ensure_peer(workspace, &agent)?;
for speaker in &session.speakers {
self.ensure_peer(workspace, speaker)?;
}
let key = (workspace, Ensured::Session(session.name.clone()));
if self.ensured.contains(&key) {
return Ok(());
}
let peers = session_peers(&agent, &session.speakers);
let create = SessionCreate::new(session.name.as_str()).with_peers(peers);
created(self.client(workspace).session(&create))?;
self.ensured.insert(key);
Ok(())
}
/// Delete conclusion `id` in `workspace`. Gone already (404) counts as done.
pub fn delete_conclusion(&self, workspace: Workspace, id: &str) -> Result<(), HonchoError> {
// `match self.client(workspace).delete_conclusion(id)`: `Ok(())` -> `Ok(())`; `Err(e)` ->
// `match classify(&e)`: `HonchoError::Refused { status: Some(404), .. }` -> `Ok(())`; any
// other `other` -> `Err(other)`.
todo!()
}
}
docs/plans/M5c1/files/crates/memoryd/src/honcho/naming.rs
+39
View File
@@ -0,0 +1,39 @@
//! Which Honcho session a turn goes to (M5c spec, section 3). A Boxmaker session maps to the Honcho
//! session of its own id; a speaker new to it starts the next one, `<id>-2`, `<id>-3` and so on,
//! holding every speaker so far, since a Honcho session's peers are fixed when it is made. The
//! numbering is read from the ledger alone, so a restart numbers the same way.
use bxcore::SessionId;
use crate::honcho::ledger::{Action, Item};
/// A Honcho session: its name, and the speakers' peers it holds besides the agent's.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct HonchoSession {
pub name: String,
pub speakers: Vec<String>,
}
/// The name of the Honcho session for the `count`-th speaker of `session`: the id itself for the
/// first (and for none), then `<id>-<count>`.
pub fn session_name(session: &SessionId, count: usize) -> String {
// When `count <= 1`: `session.as_str().to_string()`; otherwise
// `format!("{}-{count}", session.as_str())`.
todo!()
}
/// The Honcho session of turn `turn` of `session`, from the ledger's decisions in their order:
/// the speakers of that session's `Turn` actions (in any workspace) up to and including this
/// turn, each counted once, in the order they first spoke. `None` when the turn has no `Turn`
/// action: undecided, or decided to go nowhere.
pub fn honcho_session<'a, I>(decisions: I, session: &SessionId, turn: u64) -> Option<HonchoSession>
where
I: IntoIterator<Item = (&'a Item, &'a Action)>,
{
// `let mut speakers: Vec<String> = Vec::new();`. For each `(item, action)`: `let
// Item::Turn { session: s, turn: t } = item else { continue; };`, then `let Action::Turn {
// speaker, .. } = action else { continue; };`; skip it when `s != session`. Push
// `speaker.clone()` unless `speakers.contains(speaker)`. When `*t == turn`, return `Some(
// HonchoSession { name: session_name(session, speakers.len()), speakers })`. After: `None`.
todo!()
}
docs/plans/M5c1/files/crates/memoryd/tests/honcho_client.rs
+439
View File
@@ -0,0 +1,439 @@
//! `memoryd`'s Honcho clients (M5c spec, sections 3 and 6), against a fake Honcho that answers
//! from the responses recorded from rift: the workspace, the peers and the session made before
//! anything is written, each once per run, with the fleet's settings and the agent unobserved;
//! the errors by kind and status, never a body. The keys are in `honcho_keys.rs`. Do not edit.
#[path = "support/honcho.rs"]
mod honcho;
use std::collections::BTreeMap;
use std::ffi::OsString;
use std::time::Duration;
use honcho::{FakeHoncho, Mode, Request};
use memoryd::honcho::client::{
CHAT_READ, CONNECT, Clients, HonchoError, READ, Timeouts, classify, fleet_configuration,
};
use memoryd::honcho::config::HonchoConfig;
use memoryd::honcho::ledger::Workspace;
use memoryd::honcho::naming::HonchoSession;
use serde_json::{Value, json};
const FAST: Timeouts = Timeouts {
connect: Duration::from_secs(5),
read: Duration::from_millis(300),
};
fn config(url: &str) -> HonchoConfig {
HonchoConfig {
url: url.to_string(),
agent: "boxmaker0".to_string(),
owner: "kyle".to_string(),
trusted: "boxmaker0-trusted".to_string(),
untrusted: "boxmaker0-untrusted".to_string(),
fleet: "fleet".to_string(),
read: Vec::new(),
keys: BTreeMap::new(),
}
}
fn no_env(_: &str) -> Option<OsString> {
None
}
fn clients(fake: &FakeHoncho) -> Clients {
Clients::new(&config(&fake.url), &no_env, FAST).unwrap().0
}
fn session(name: &str, speakers: &[&str]) -> HonchoSession {
HonchoSession {
name: name.to_string(),
speakers: speakers.iter().map(|s| s.to_string()).collect(),
}
}
/// A request as (method, path, body).
fn seen(requests: &[Request]) -> Vec<(String, String, Value)> {
requests
.iter()
.map(|r| {
(
r.method.clone(),
r.path.clone(),
r.body.clone().unwrap_or(Value::Null),
)
})
.collect()
}
fn post(path: &str, body: Value) -> (String, String, Value) {
("POST".to_string(), path.to_string(), body)
}
#[test]
fn the_timeouts() {
assert_eq!(CONNECT, Duration::from_secs(5));
assert_eq!(READ, Duration::from_secs(30));
assert_eq!(CHAT_READ, Duration::from_secs(180));
assert_eq!(
Timeouts::WRITES,
Timeouts {
connect: CONNECT,
read: READ
}
);
}
#[test]
fn each_client_has_its_workspace_and_timeouts() {
let fake = FakeHoncho::start();
let c = clients(&fake);
for (workspace, name) in [
(Workspace::Trusted, "boxmaker0-trusted"),
(Workspace::Untrusted, "boxmaker0-untrusted"),
(Workspace::Fleet, "fleet"),
] {
let client = c.client(workspace);
assert_eq!(client.workspace_id(), name);
// `roncho`'s `Debug` shows the timeouts, and whether there is a key, never the key.
let shown = format!("{client:?}");
assert!(shown.contains("connect_timeout: 5s"), "{shown}");
assert!(shown.contains("read_timeout: 300ms"), "{shown}");
assert!(shown.contains("api_key_present: false"), "{shown}");
}
}
#[test]
fn the_fleets_configuration_is_the_recorded_one() {
let path = concat!(
env!("CARGO_MANIFEST_DIR"),
"/tests/fixtures/honcho/01-workspace-create.json"
);
let recorded: Value = serde_json::from_str(&std::fs::read_to_string(path).unwrap()).unwrap();
assert_eq!(
Value::Object(fleet_configuration()),
recorded["request"]["body"]["configuration"]
);
}
#[test]
fn a_session_is_made_after_its_workspace_and_peers() {
let fake = FakeHoncho::start();
let mut c = clients(&fake);
c.ensure_session(Workspace::Trusted, &session("chat-1", &["kyle"]))
.unwrap();
let w = "/v3/workspaces/boxmaker0-trusted";
assert_eq!(
seen(&fake.requests()),
vec![
post("/v3/workspaces", json!({ "id": "boxmaker0-trusted" })),
post(
&format!("{w}/peers"),
json!({ "id": "boxmaker0", "configuration": { "observe_me": false } })
),
post(&format!("{w}/peers"), json!({ "id": "kyle" })),
post(
&format!("{w}/sessions"),
json!({ "id": "chat-1", "peers": {
"boxmaker0": { "observe_me": false, "observe_others": false },
"kyle": { "observe_me": true, "observe_others": false },
} })
),
]
);
}
#[test]
fn what_was_made_is_not_asked_for_again_in_this_run() {
let fake = FakeHoncho::start();
let mut c = clients(&fake);
c.ensure_session(Workspace::Trusted, &session("chat-1", &["kyle"]))
.unwrap();
fake.clear();
c.ensure_session(Workspace::Trusted, &session("chat-1", &["kyle"]))
.unwrap();
assert!(fake.requests().is_empty(), "{:?}", fake.requests());
// A new speaker: only its peer and the next session.
c.ensure_session(Workspace::Trusted, &session("chat-1-2", &["kyle", "ana_b"]))
.unwrap();
let w = "/v3/workspaces/boxmaker0-trusted";
assert_eq!(
seen(&fake.requests()),
vec![
post(&format!("{w}/peers"), json!({ "id": "ana_b" })),
post(
&format!("{w}/sessions"),
json!({ "id": "chat-1-2", "peers": {
"boxmaker0": { "observe_me": false, "observe_others": false },
"kyle": { "observe_me": true, "observe_others": false },
"ana_b": { "observe_me": true, "observe_others": false },
} })
),
]
);
// Another workspace is made sure of on its own.
fake.clear();
c.ensure_session(Workspace::Untrusted, &session("chat-1", &["kyle"]))
.unwrap();
assert_eq!(fake.requests().len(), 4);
assert!(
fake.requests()
.iter()
.skip(1)
.all(|r| r.path.starts_with("/v3/workspaces/boxmaker0-untrusted/"))
);
}
#[test]
fn after_a_restart_everything_is_made_sure_of_again() {
let fake = FakeHoncho::start();
clients(&fake)
.ensure_session(Workspace::Trusted, &session("chat-1", &["kyle"]))
.unwrap();
fake.clear();
// Honcho answers 200 now: it has them all. That is success too.
clients(&fake)
.ensure_session(Workspace::Trusted, &session("chat-1", &["kyle"]))
.unwrap();
assert_eq!(fake.requests().len(), 4);
}
#[test]
fn the_fleet_is_made_with_its_settings_and_the_agent_alone() {
let fake = FakeHoncho::start();
let mut c = clients(&fake);
c.ensure_session(Workspace::Fleet, &session("chat-1", &[]))
.unwrap();
assert_eq!(
seen(&fake.requests()),
vec![
post(
"/v3/workspaces",
json!({ "id": "fleet", "configuration": {
"reasoning": { "enabled": false },
"summary": { "enabled": false },
"dream": { "enabled": false },
} })
),
post(
"/v3/workspaces/fleet/peers",
json!({ "id": "boxmaker0", "configuration": { "observe_me": false } })
),
post(
"/v3/workspaces/fleet/sessions",
json!({ "id": "chat-1", "peers": {
"boxmaker0": { "observe_me": false, "observe_others": false },
} })
),
]
);
}
#[test]
fn a_409_on_create_means_it_is_there() {
let fake = FakeHoncho::start();
let mut c = clients(&fake);
fake.set(Mode::Status(409));
c.ensure_workspace(Workspace::Trusted).unwrap();
c.ensure_peer(Workspace::Trusted, "kyle").unwrap();
fake.clear();
c.ensure_workspace(Workspace::Trusted).unwrap();
assert!(fake.requests().is_empty());
}
#[test]
fn errors_by_kind_and_status() {
let fake = FakeHoncho::start();
let mut c = clients(&fake);
let cases = [
(
500,
HonchoError::Unreachable {
kind: "server",
status: Some(500),
},
),
(
503,
HonchoError::Unreachable {
kind: "server",
status: Some(503),
},
),
(401, HonchoError::KeyRefused { status: 401 }),
(403, HonchoError::KeyRefused { status: 403 }),
(
400,
HonchoError::Refused {
kind: "status",
status: Some(400),
},
),
(
404,
HonchoError::Refused {
kind: "status",
status: Some(404),
},
),
(
422,
HonchoError::Refused {
kind: "status",
status: Some(422),
},
),
];
for (status, expected) in cases {
fake.set(Mode::Status(status));
assert_eq!(
c.ensure_workspace(Workspace::Trusted),
Err(expected.clone())
);
assert_eq!(c.ensure_peer(Workspace::Fleet, "kyle"), Err(expected));
}
// Nothing failed is remembered: the next call asks again, and succeeds.
fake.set(Mode::Fixtures);
fake.clear();
c.ensure_workspace(Workspace::Trusted).unwrap();
assert_eq!(fake.requests().len(), 1);
fake.set(Mode::Hang);
assert_eq!(
c.ensure_peer(Workspace::Trusted, "ana_b"),
Err(HonchoError::Unreachable {
kind: "timeout",
status: None
})
);
}
#[test]
fn nothing_listening_is_unreachable() {
let port = {
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
listener.local_addr().unwrap().port()
};
let (mut c, _) =
Clients::new(&config(&format!("http://127.0.0.1:{port}")), &no_env, FAST).unwrap();
assert_eq!(
c.ensure_workspace(Workspace::Trusted),
Err(HonchoError::Unreachable {
kind: "connect",
status: None
})
);
}
#[test]
fn an_error_never_holds_the_response() {
let fake = FakeHoncho::start();
let mut c = clients(&fake);
fake.set(Mode::Status(422));
let err = c
.ensure_peer(Workspace::Trusted, "private-note-text")
.unwrap_err();
let body = fake.requests()[0].body.clone().unwrap();
assert_eq!(body["id"], "private-note-text", "the fake did not echo it");
for shown in [err.to_string(), format!("{err:?}")] {
assert!(!shown.contains("private-note-text"), "{shown}");
}
assert_eq!(
err.to_string(),
"Honcho refused the request (status, HTTP 422)"
);
assert_eq!((err.kind(), err.status()), ("status", Some(422)));
let lines = [
HonchoError::Unreachable {
kind: "timeout",
status: None,
}
.to_string(),
HonchoError::KeyRefused { status: 401 }.to_string(),
];
assert_eq!(lines[0], "Honcho is unreachable (timeout)");
assert_eq!(lines[1], "Honcho refused the key (key-refused, HTTP 401)");
}
#[test]
fn every_roncho_error_is_classified() {
use roncho::blocking::Error;
let io = || std::io::Error::other("x");
let cases = [
(Error::Connect(io()), "connect", None, true),
(Error::Timeout, "timeout", None, true),
(Error::Closed, "closed", None, true),
(Error::Io(io()), "io", None, true),
(
Error::Status {
status: 502,
body: "b".into(),
},
"server",
Some(502),
true,
),
(
Error::Status {
status: 499,
body: "b".into(),
},
"status",
Some(499),
false,
),
(
Error::Redirect { status: 301 },
"redirect",
Some(301),
false,
),
(
Error::ActiveSessions { body: "b".into() },
"active-sessions",
Some(409),
false,
),
(Error::TooLarge, "too-large", None, false),
(Error::Decode("b".into()), "decode", None, false),
(Error::Stream("b".into()), "stream", None, false),
(Error::Config("b".into()), "config", None, false),
];
for (error, kind, status, unreachable) in cases {
let got = classify(&error);
assert_eq!((got.kind(), got.status()), (kind, status), "{error:?}");
assert_eq!(
matches!(got, HonchoError::Unreachable { .. }),
unreachable,
"{error:?}"
);
}
let got = classify(&Error::Unauthorized { status: 403 });
assert_eq!(got, HonchoError::KeyRefused { status: 403 });
}
#[test]
fn a_conclusion_deleted_or_gone_is_done() {
let fake = FakeHoncho::start();
let c = clients(&fake);
c.delete_conclusion(Workspace::Trusted, "YUBMqIqHhuJprV4o-inrg")
.unwrap();
c.delete_conclusion(Workspace::Trusted, "YUBMqIqHhuJprV4o-inrg")
.unwrap();
let requests = fake.requests();
assert_eq!(requests.len(), 2);
assert!(requests.iter().all(|r| r.method == "DELETE"
&& r.path == "/v3/workspaces/boxmaker0-trusted/conclusions/YUBMqIqHhuJprV4o-inrg"));
fake.set(Mode::Status(500));
assert_eq!(
c.delete_conclusion(Workspace::Trusted, "x"),
Err(HonchoError::Unreachable {
kind: "server",
status: Some(500)
})
);
fake.set(Mode::Status(403));
assert_eq!(
c.delete_conclusion(Workspace::Trusted, "x"),
Err(HonchoError::KeyRefused { status: 403 })
);
}
docs/plans/M5c1/files/crates/memoryd/tests/honcho_keys.rs
+171
View File
@@ -0,0 +1,171 @@
//! The keys of `memoryd`'s Honcho clients (M5c spec, section 10): each read with
//! `bxcore::secrets`, sent as a bearer token to its own workspace only, and never shown by a
//! client, an error or a warning; a key that cannot be read is an error naming it. Against a fake
//! Honcho. Do not edit.
#[path = "support/honcho.rs"]
mod honcho;
#[path = "support/tmp.rs"]
mod tmp;
use std::collections::BTreeMap;
use std::ffi::OsString;
use std::os::unix::fs::PermissionsExt;
use std::process::{Command, Stdio};
use std::time::{Duration, Instant};
use bxcore::secrets::SecretSource;
use honcho::{FakeHoncho, Mode};
use memoryd::honcho::client::{Clients, Timeouts};
use memoryd::honcho::config::{FLEET_KEY, HonchoConfig, TRUSTED_KEY};
use memoryd::honcho::ledger::Workspace;
use tmp::TempDir;
const FAST: Timeouts = Timeouts {
connect: Duration::from_secs(5),
read: Duration::from_millis(300),
};
fn config(url: &str) -> HonchoConfig {
HonchoConfig {
url: url.to_string(),
agent: "boxmaker0".to_string(),
owner: "kyle".to_string(),
trusted: "boxmaker0-trusted".to_string(),
untrusted: "boxmaker0-untrusted".to_string(),
fleet: "fleet".to_string(),
read: Vec::new(),
keys: BTreeMap::new(),
}
}
fn no_env(_: &str) -> Option<OsString> {
None
}
/// Key values built from pieces, so that no file holds one whole.
fn env_key() -> String {
["hk", "trusted", "5e1f0a"].join("-")
}
fn file_key() -> String {
["hk", "fleet", "c27d94"].join("-")
}
#[test]
fn each_key_goes_to_its_own_workspace_and_is_never_shown() {
let dir = TempDir::new("honcho-keys");
let file = dir.write("fleet.key", &format!("{}\n", file_key()));
std::fs::set_permissions(&file, std::fs::Permissions::from_mode(0o600)).unwrap();
let fake = FakeHoncho::start();
let mut config = config(&fake.url);
config.keys.insert(
TRUSTED_KEY.to_string(),
SecretSource::Env("HONCHO_TRUSTED".to_string()),
);
config
.keys
.insert(FLEET_KEY.to_string(), SecretSource::File(file.clone()));
let env = |name: &str| (name == "HONCHO_TRUSTED").then(|| OsString::from(env_key()));
let (mut c, warnings) = Clients::new(&config, &env, FAST).unwrap();
assert_eq!(warnings.len(), 1, "{warnings:?}");
assert!(warnings[0].contains("honcho_fleet"), "{}", warnings[0]);
c.ensure_workspace(Workspace::Trusted).unwrap();
c.ensure_workspace(Workspace::Untrusted).unwrap();
c.ensure_workspace(Workspace::Fleet).unwrap();
let auth: Vec<Option<String>> = fake
.requests()
.iter()
.map(|r| r.header("authorization").map(str::to_string))
.collect();
assert_eq!(
auth,
vec![
Some(format!("Bearer {}", env_key())),
None,
Some(format!("Bearer {}", file_key())),
]
);
// Nothing shows a key: the clients, their errors, the warnings.
fake.set(Mode::Status(401));
let err = c.ensure_peer(Workspace::Trusted, "kyle").unwrap_err();
let shown = [
format!("{c:?}"),
format!("{err:?}"),
err.to_string(),
warnings.join("\n"),
];
for text in shown {
assert!(!text.contains(&env_key()), "{text}");
assert!(!text.contains(&file_key()), "{text}");
}
}
#[test]
fn a_key_that_cannot_be_read_is_an_error_naming_it() {
let fake = FakeHoncho::start();
let mut config = config(&fake.url);
config.keys.insert(
TRUSTED_KEY.to_string(),
SecretSource::Env("HONCHO_TRUSTED".to_string()),
);
let err = Clients::new(&config, &no_env, FAST).unwrap_err();
assert!(err.contains("honcho_trusted"), "{err}");
assert!(
err.contains("see docs/runbook.md#secret-unavailable"),
"{err}"
);
}
/// Set in the process `a_roncho_toml_in_home_is_not_read` starts, and only there.
const CHILD: &str = "MEMORYD_TEST_RONCHO_HOME";
#[test]
fn a_roncho_toml_in_home_is_not_read() {
// `roncho` reads `$HOME/.config/roncho/roncho.toml` unless told not to, and takes its
// `api_key`. HOME is the process's, so the check runs in a process of its own: this test
// binary, running only the test below.
let dir = TempDir::new("honcho-home");
let key = ["hk", "from", "roncho", "toml"].join("-");
dir.write(
".config/roncho/roncho.toml",
&format!("api_key = \"{key}\"\n"),
);
let mut child = Command::new(std::env::current_exe().unwrap())
.args(["--exact", "in_a_home_with_a_roncho_toml", "--ignored"])
.env("HOME", dir.path())
.env_remove("XDG_CONFIG_HOME")
.env(CHILD, "1")
.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
.unwrap();
let deadline = Instant::now() + Duration::from_secs(20);
let mut status = None;
while Instant::now() < deadline {
if let Ok(Some(done)) = child.try_wait() {
status = Some(done);
break;
}
std::thread::sleep(Duration::from_millis(20));
}
let _ = child.kill();
let output = child.wait_with_output().unwrap();
let stdout = String::from_utf8_lossy(&output.stdout);
assert!(status.is_some_and(|s| s.success()), "{stdout}");
assert!(stdout.contains("1 passed"), "{stdout}");
}
#[test]
#[ignore = "run by a_roncho_toml_in_home_is_not_read, in a process of its own"]
fn in_a_home_with_a_roncho_toml() {
if std::env::var_os(CHILD).is_none() {
return;
}
let fake = FakeHoncho::start();
let (mut c, _) = Clients::new(&config(&fake.url), &no_env, FAST).unwrap();
c.ensure_workspace(Workspace::Trusted).unwrap();
let requests = fake.requests();
assert_eq!(requests.len(), 1);
assert_eq!(requests[0].header("authorization"), None);
}
docs/plans/M5c1/files/crates/memoryd/tests/honcho_naming.rs
+150
View File
@@ -0,0 +1,150 @@
//! Which Honcho session a turn goes to (M5c spec, section 3): the Boxmaker session's id, then
//! `<id>-2`, `<id>-3` for each new speaker, holding every speaker so far; numbered from the
//! ledger's decisions alone, so a restart numbers the same. Do not edit.
#[path = "support/tmp.rs"]
mod tmp;
use bxcore::{SessionId, Timestamp};
use memoryd::honcho::ledger::{Action, Item, Ledger, Record, Workspace};
use memoryd::honcho::naming::{HonchoSession, honcho_session, session_name};
use tmp::TempDir;
fn sid(s: &str) -> SessionId {
SessionId::new(s).unwrap()
}
fn turn(session: &str, turn: u64) -> Item {
Item::Turn {
session: sid(session),
turn,
}
}
fn spoke(workspace: Workspace, speaker: &str) -> Action {
Action::Turn {
workspace,
speaker: speaker.to_string(),
}
}
fn skip() -> Action {
Action::Skip {
why: "the turn has no single speaker".to_string(),
}
}
fn hs(name: &str, speakers: &[&str]) -> Option<HonchoSession> {
Some(HonchoSession {
name: name.to_string(),
speakers: speakers.iter().map(|s| s.to_string()).collect(),
})
}
/// A ledger with these decisions, in this order.
fn ledger(dir: &TempDir, decisions: &[(Item, Action)]) -> Ledger {
let path = dir.path().join("honcho.jsonl");
let mut ledger = Ledger::open(&path).unwrap();
for (item, action) in decisions {
ledger
.append(Record::Decided {
time: Timestamp::parse("2026-09-28T10:00:00.000Z").unwrap(),
item: item.clone(),
action: action.clone(),
})
.unwrap();
}
ledger
}
#[test]
fn session_names() {
assert_eq!(session_name(&sid("chat-1"), 0), "chat-1");
assert_eq!(session_name(&sid("chat-1"), 1), "chat-1");
assert_eq!(session_name(&sid("chat-1"), 2), "chat-1-2");
assert_eq!(session_name(&sid("chat-1"), 12), "chat-1-12");
}
#[test]
fn each_new_speaker_starts_the_next_session() {
let dir = TempDir::new("naming-order");
let decisions = [
(turn("chat-1", 1), spoke(Workspace::Trusted, "kyle")),
(turn("chat-2", 1), spoke(Workspace::Trusted, "zed")),
(turn("chat-1", 2), spoke(Workspace::Trusted, "kyle")),
(turn("chat-1", 3), spoke(Workspace::Untrusted, "ana_b")),
(turn("chat-1", 4), skip()),
(turn("chat-1", 5), spoke(Workspace::Trusted, "kyle")),
(turn("chat-1", 6), spoke(Workspace::Untrusted, "bob")),
(turn("chat-1", 7), spoke(Workspace::Trusted, "ana_b")),
];
let l = ledger(&dir, &decisions);
let at = |t: u64| honcho_session(l.decisions(), &sid("chat-1"), t);
assert_eq!(at(1), hs("chat-1", &["kyle"]));
assert_eq!(at(2), hs("chat-1", &["kyle"]));
// A new speaker in any workspace counts.
assert_eq!(at(3), hs("chat-1-2", &["kyle", "ana_b"]));
assert_eq!(at(4), None, "a skipped turn goes nowhere");
assert_eq!(at(5), hs("chat-1-2", &["kyle", "ana_b"]));
assert_eq!(at(6), hs("chat-1-3", &["kyle", "ana_b", "bob"]));
assert_eq!(at(7), hs("chat-1-3", &["kyle", "ana_b", "bob"]));
assert_eq!(at(8), None, "an undecided turn");
// Another session's speakers do not count.
assert_eq!(
honcho_session(l.decisions(), &sid("chat-2"), 1),
hs("chat-2", &["zed"])
);
}
#[test]
fn the_order_is_the_order_of_decision() {
let dir = TempDir::new("naming-decided");
// Turn 2 was decided first (turn 1 waited for brokerd): its speaker is the first.
let decisions = [
(turn("chat-1", 2), spoke(Workspace::Trusted, "ana_b")),
(turn("chat-1", 1), spoke(Workspace::Trusted, "kyle")),
];
let l = ledger(&dir, &decisions);
assert_eq!(
honcho_session(l.decisions(), &sid("chat-1"), 2),
hs("chat-1", &["ana_b"])
);
assert_eq!(
honcho_session(l.decisions(), &sid("chat-1"), 1),
hs("chat-1-2", &["ana_b", "kyle"])
);
}
#[test]
fn a_restart_numbers_the_same() {
let dir = TempDir::new("naming-restart");
let decisions = [
(turn("chat-1", 1), spoke(Workspace::Trusted, "kyle")),
(turn("chat-1", 2), skip()),
(turn("chat-1", 3), spoke(Workspace::Trusted, "ana_b")),
];
let before: Vec<Option<HonchoSession>> = {
let l = ledger(&dir, &decisions);
(1..=3)
.map(|t| honcho_session(l.decisions(), &sid("chat-1"), t))
.collect()
};
let l = Ledger::open(&dir.path().join("honcho.jsonl")).unwrap();
let after: Vec<Option<HonchoSession>> = (1..=3)
.map(|t| honcho_session(l.decisions(), &sid("chat-1"), t))
.collect();
assert_eq!(before, after);
assert_eq!(after[2], hs("chat-1-2", &["kyle", "ana_b"]));
}
#[test]
fn decisions_are_every_decided_item_in_order() {
let dir = TempDir::new("naming-decisions");
let decisions = [
(turn("chat-1", 2), skip()),
(turn("chat-1", 1), spoke(Workspace::Trusted, "kyle")),
];
let l = ledger(&dir, &decisions);
let got: Vec<(Item, Action)> = l.decisions().map(|(i, a)| (i.clone(), a.clone())).collect();
assert_eq!(got, decisions.to_vec());
}
docs/plans/M5c1/files/crates/memoryd/tests/support/honcho.rs
+260
View File
@@ -0,0 +1,260 @@
//! A fake Honcho for tests, on 127.0.0.1 and a port of its own: it answers each request from the
//! responses recorded from rift (`tests/fixtures/honcho/`), by method and path, and records every
//! request, body included. It can be set to answer every request with a status instead, or to
//! answer nothing. Stopped, and its held connections released, when dropped; every wait has a
//! deadline. Do not edit.
#![allow(dead_code)] // each test file uses a different part of this module
use std::collections::HashSet;
use std::io::{Read, Write};
use std::net::{TcpListener, TcpStream};
use std::path::PathBuf;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::{Arc, Mutex};
use std::thread::JoinHandle;
use std::time::{Duration, Instant};
use serde_json::Value;
/// One request as the fake read it.
#[derive(Debug, Clone, PartialEq)]
pub struct Request {
pub method: String,
pub path: String,
/// Header names in lower case.
pub headers: Vec<(String, String)>,
pub body: Option<Value>,
}
impl Request {
pub fn header(&self, name: &str) -> Option<&str> {
self.headers
.iter()
.find(|(k, _)| k == name)
.map(|(_, v)| v.as_str())
}
}
/// How the fake answers.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Mode {
/// From the fixtures.
Fixtures,
/// Every request with this status. A 422 echoes the request's body, as Honcho does.
Status(u16),
/// Read the request, then answer nothing until the fake is dropped (at most 30 s).
Hang,
}
struct Shared {
mode: Mutex<Mode>,
requests: Mutex<Vec<Request>>,
/// What was made, by method and path and id: a second create answers 200, a second delete
/// 404.
made: Mutex<HashSet<String>>,
stop: AtomicBool,
}
pub struct FakeHoncho {
pub url: String,
port: u16,
shared: Arc<Shared>,
accept: Option<JoinHandle<()>>,
}
/// A recorded fixture: its response's status and body.
fn fixture(name: &str) -> (u16, String) {
let path = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("tests/fixtures/honcho")
.join(name);
let text = std::fs::read_to_string(&path).unwrap();
let value: Value = serde_json::from_str(&text).unwrap();
let status = u16::try_from(value["response"]["status"].as_u64().unwrap()).unwrap();
let body = match &value["response"]["body"] {
Value::String(s) if s.is_empty() => String::new(),
other => other.to_string(),
};
(status, body)
}
impl FakeHoncho {
pub fn start() -> FakeHoncho {
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let port = listener.local_addr().unwrap().port();
let shared = Arc::new(Shared {
mode: Mutex::new(Mode::Fixtures),
requests: Mutex::new(Vec::new()),
made: Mutex::new(HashSet::new()),
stop: AtomicBool::new(false),
});
let for_accept = Arc::clone(&shared);
let accept = std::thread::spawn(move || {
for stream in listener.incoming() {
if for_accept.stop.load(Ordering::SeqCst) {
return;
}
let Ok(stream) = stream else { return };
let shared = Arc::clone(&for_accept);
std::thread::spawn(move || serve(stream, &shared));
}
});
FakeHoncho {
url: format!("http://127.0.0.1:{port}"),
port,
shared,
accept: Some(accept),
}
}
pub fn set(&self, mode: Mode) {
*self.shared.mode.lock().unwrap() = mode;
}
/// Every request read so far, in order.
pub fn requests(&self) -> Vec<Request> {
self.shared.requests.lock().unwrap().clone()
}
/// Forget the requests read so far (not what was made).
pub fn clear(&self) {
self.shared.requests.lock().unwrap().clear();
}
/// Waits up to 10 s until at least `n` requests were read; true if they were.
pub fn wait_for(&self, n: usize) -> bool {
let deadline = Instant::now() + Duration::from_secs(10);
while Instant::now() < deadline {
if self.requests().len() >= n {
return true;
}
std::thread::sleep(Duration::from_millis(10));
}
false
}
}
impl Drop for FakeHoncho {
fn drop(&mut self) {
self.shared.stop.store(true, Ordering::SeqCst);
let _ = TcpStream::connect(("127.0.0.1", self.port));
if let Some(accept) = self.accept.take() {
let _ = accept.join();
}
}
}
/// Reads one request: the head up to its blank line, then `Content-Length` bytes of body.
fn read_request(stream: &mut TcpStream) -> Option<Request> {
let mut buf = Vec::new();
let mut byte = [0u8; 1];
while !buf.ends_with(b"\r\n\r\n") {
if stream.read(&mut byte).ok()? == 0 {
return None;
}
buf.push(byte[0]);
}
let head = String::from_utf8(buf).ok()?;
let mut lines = head.split("\r\n");
let mut first = lines.next()?.split(' ');
let (method, path) = (first.next()?.to_string(), first.next()?.to_string());
let headers: Vec<(String, String)> = lines
.filter_map(|l| l.split_once(": "))
.map(|(k, v)| (k.to_ascii_lowercase(), v.to_string()))
.collect();
let length: usize = headers
.iter()
.find(|(k, _)| k == "content-length")
.map_or(Some(0), |(_, v)| v.parse().ok())?;
let mut body = vec![0u8; length];
stream.read_exact(&mut body).ok()?;
let body = if length == 0 {
None
} else {
serde_json::from_slice(&body).ok()
};
Some(Request {
method,
path,
headers,
body,
})
}
/// The fixture that answers `request`, by its method and the shape of its path; the second of a
/// pair when the same thing was made (or deleted) before.
fn answer(request: &Request, made: &Mutex<HashSet<String>>) -> (u16, String) {
let parts: Vec<&str> = request.path.trim_start_matches('/').split('/').collect();
let id = request
.body
.as_ref()
.and_then(|b| b["id"].as_str())
.unwrap_or_default()
.to_string();
let again = !made
.lock()
.unwrap()
.insert(format!("{} {} {id}", request.method, request.path));
let configured = request
.body
.as_ref()
.is_some_and(|b| b.get("configuration").is_some());
match (request.method.as_str(), parts.as_slice()) {
("POST", ["v3", "workspaces"]) if again => fixture("02-workspace-create-again.json"),
("POST", ["v3", "workspaces"]) => fixture("01-workspace-create.json"),
("POST", ["v3", "workspaces", _, "peers"]) if again => {
fixture("05-peer-create-boxmaker0-again.json")
}
("POST", ["v3", "workspaces", _, "peers"]) if configured => {
fixture("04-peer-create-boxmaker0.json")
}
("POST", ["v3", "workspaces", _, "peers"]) => fixture("03-peer-create-kyle.json"),
("POST", ["v3", "workspaces", _, "sessions"]) if again => {
fixture("07-session-create-again.json")
}
("POST", ["v3", "workspaces", _, "sessions"]) => fixture("06-session-create.json"),
("POST", ["v3", "workspaces", _, "sessions", _, "messages"]) => {
fixture("08-messages-add-batch.json")
}
("DELETE", ["v3", "workspaces", _, "conclusions", _]) if again => {
fixture("16-conclusion-delete-again.json")
}
("DELETE", ["v3", "workspaces", _, "conclusions", _]) => {
fixture("15-conclusion-delete.json")
}
_ => (404, r#"{"detail":"Not Found"}"#.to_string()),
}
}
fn serve(mut stream: TcpStream, shared: &Shared) {
let _ = stream.set_read_timeout(Some(Duration::from_secs(10)));
let Some(request) = read_request(&mut stream) else {
return;
};
shared.requests.lock().unwrap().push(request.clone());
let mode = *shared.mode.lock().unwrap();
let (status, body) = match mode {
Mode::Fixtures => answer(&request, &shared.made),
Mode::Status(422) => {
let input = request.body.clone().unwrap_or(Value::Null);
(
422,
serde_json::json!({ "detail": [{ "input": input }] }).to_string(),
)
}
Mode::Status(status) => (status, r#"{"detail":"refused"}"#.to_string()),
Mode::Hang => {
let deadline = Instant::now() + Duration::from_secs(30);
while !shared.stop.load(Ordering::SeqCst) && Instant::now() < deadline {
std::thread::sleep(Duration::from_millis(10));
}
return;
}
};
let head = format!(
"HTTP/1.1 {status} X\r\nContent-Type: application/json\r\nContent-Length: {}\r\nConnection: close\r\n\r\n",
body.len()
);
let _ = stream.write_all(head.as_bytes());
let _ = stream.write_all(body.as_bytes());
}