M5c-2 plan: task 05, memoryd's readers
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5.5
@@ -0,0 +1,410 @@
|
||||
# M5c-2 task 05: `memoryd`'s readers, the Honcho workspaces it reads
|
||||
|
||||
**Branch:** `m5c2`. Check it: `git branch --show-current` prints `m5c2` and `git status --short`
|
||||
is empty; otherwise stop. Never switch branch.
|
||||
**Commit subject:** `memoryd: the Honcho workspaces it reads, with their clients and labels`
|
||||
**Log row:** the Task cell is `M5c2/05-memoryd-honcho-readers`.
|
||||
|
||||
## Goal
|
||||
|
||||
`memoryd` writes to three Honcho workspaces through the `Clients` of M5c-1 (the agent's trusted
|
||||
and untrusted ones and the fleet's), which move into the sync worker's thread. M5c-2 also reads
|
||||
(M5c spec, sections 3, 8 and 9): `recall` searches other agents' workspaces and the fleet's, and
|
||||
`ask_memory` asks one or two workspaces by name. Reading needs clients of its own. This task
|
||||
builds them at start and gives later tasks the types they use; nothing searches or asks yet
|
||||
(tasks 07 and 08).
|
||||
|
||||
- **`Readers`** (new module `crates/memoryd/src/honcho/read.rs`, given whole): a `Reader` for
|
||||
- each `[[honcho.read]]` workspace, in the order written, with that entry's `trust` and
|
||||
`class` and the key `honcho_read_<agent>`;
|
||||
- the fleet's workspace (key `honcho_fleet`), always untrusted and `private`;
|
||||
- the agent's own trusted workspace (key `honcho_trusted`), trusted, `private`, and its own
|
||||
untrusted one (key `honcho_untrusted`), untrusted, `private`. These two are for `ask_memory`
|
||||
only: `recall` never searches the agent's own workspaces (spec, section 8).
|
||||
- **Two clients per reader.** A `roncho` client has one read timeout. Searches wait 5 s
|
||||
(`SEARCH_READ`) and chat 150 s (`CHAT_READ`, which this task changes from 180 s: spec 9 gives
|
||||
chat 150 s, inside `brokerd`'s 170 s); both connect within 5 s (`CONNECT`).
|
||||
- **The label** of each reader's results: the name each hit shows (the agent's name, `fleet`,
|
||||
`own` or `untrusted`), the trust, and the class. `Label::tag()` is `memory` for a trusted
|
||||
workspace and `memory-tainted` otherwise, as a local `recall` hit's tag.
|
||||
- **`recall_sources()`**: the read workspaces, then the fleet's. **`resolve(name)`**: the reader
|
||||
`ask_memory` means by `own`, `untrusted`, `fleet` or a read agent's name; any other name is
|
||||
the sentence the model is shown, which lists the names it may use.
|
||||
- **Reserved names.** Since `ask_memory` names this agent's own workspaces `own`, `untrusted`
|
||||
and `fleet`, a `[[honcho.read]]` agent with one of those names is now refused at start
|
||||
(`honcho-config-invalid`). Otherwise a read agent called `fleet` would shadow the fleet in
|
||||
`ask_memory`.
|
||||
- **At start**, `start::open` builds the readers beside the writers' `Clients` and keeps them in
|
||||
`Opened::readers`. A read key that cannot be read stops `memoryd`, as a written workspace's
|
||||
does. The three written workspaces' keys are read by both; each key file's warning is logged
|
||||
once. `serve.rs` does not change: nothing uses the readers until task 07, which takes
|
||||
`opened.readers` before the worker's thread takes the rest of `Opened`.
|
||||
|
||||
## Every place that changes
|
||||
|
||||
This is the complete list.
|
||||
|
||||
| Place | What happens there |
|
||||
|---|---|
|
||||
| `crates/memoryd/src/honcho/read.rs` | new, given whole: copy it |
|
||||
| `crates/memoryd/src/honcho/mod.rs` | `pub mod read;` (edit H1) |
|
||||
| `crates/memoryd/src/honcho/client.rs` | `CHAT_READ` is 150 s; `client` and `key` become `pub(crate)`, for `read.rs` (edits C1 to C3) |
|
||||
| `crates/memoryd/src/honcho/config.rs` | `RESERVED`, and the check that uses it (edits G1, G2) |
|
||||
| `crates/memoryd/src/honcho/start.rs` | `Opened::readers`, built in `open` (edits O1 to O3) |
|
||||
| `docs/runbook.md`, `honcho-config-invalid` | the reserved names among the reasons (edit D1) |
|
||||
| `crates/memoryd/tests/honcho_read.rs` | new, given |
|
||||
| `crates/memoryd/tests/honcho_client.rs`, `honcho_start.rs` | new versions, given |
|
||||
| everything else | **no change**: not `serve.rs`, not `recall.rs`, not `worker.rs`, no `Cargo.toml` |
|
||||
|
||||
## Files
|
||||
|
||||
- Copy, from `docs/plans/M5c2/files/` (the same path under it): the new test
|
||||
`crates/memoryd/tests/honcho_read.rs`; **new versions** of (they replace the files in the tree)
|
||||
`crates/memoryd/tests/honcho_client.rs` and `crates/memoryd/tests/honcho_start.rs`; and the
|
||||
new source `crates/memoryd/src/honcho/read.rs`, which is complete: it has no `todo!()`, and you
|
||||
do not change it.
|
||||
- Modify: `crates/memoryd/src/honcho/mod.rs`, `crates/memoryd/src/honcho/client.rs`,
|
||||
`crates/memoryd/src/honcho/config.rs`, `crates/memoryd/src/honcho/start.rs`,
|
||||
`docs/runbook.md`, `docs/implementer-log.md`.
|
||||
- Do not touch: any other file.
|
||||
|
||||
## Facts you need
|
||||
|
||||
- **`read.rs` needs** from `client.rs`: `CONNECT`, `CHAT_READ`, `Timeouts`, and the two helpers
|
||||
`client(url, workspace, key, timeouts) -> Result<Client, String>` and
|
||||
`key(config, name, env, warnings) -> Result<Option<secrets::Secret>, String>`, which are
|
||||
private today. Edits C2 and C3 make them `pub(crate)`: visible in `memoryd`, not outside it.
|
||||
From `config.rs` it needs `FLEET_KEY`, `TRUSTED_KEY`, `UNTRUSTED_KEY`, `read_key`, `Trust` and
|
||||
`HonchoConfig`, all public already, and from `bxcore` `DataClass` and
|
||||
`bxcore::memory::OWN_WORKSPACE` (`"own"`, task 01).
|
||||
- **What `Readers` offers** (for tasks 07 and 08; read `read.rs` for the whole):
|
||||
|
||||
```rust
|
||||
pub const SEARCH_READ: Duration; // 5 s
|
||||
pub const UNTRUSTED: &str; // "untrusted"
|
||||
pub const FLEET: &str; // "fleet"
|
||||
pub struct ReadTimeouts { pub connect: Duration, pub search: Duration, pub chat: Duration }
|
||||
pub enum Source { Agent(String), Fleet, Own, OwnUntrusted }
|
||||
pub struct Label { pub name: String, pub trust: Trust, pub class: DataClass }
|
||||
impl Label { pub fn tag(&self) -> &'static str; pub fn untrusted(&self) -> bool }
|
||||
pub struct Reader { pub source: Source, pub label: Label, /* two clients */ }
|
||||
impl Reader { pub fn search(&self) -> &Client; pub fn chat(&self) -> &Client }
|
||||
impl Readers {
|
||||
pub fn new(config: &HonchoConfig, env: &dyn Fn(&str) -> Option<OsString>,
|
||||
timeouts: ReadTimeouts) -> Result<(Readers, Vec<String>), String>;
|
||||
pub fn recall_sources(&self) -> Vec<&Reader>;
|
||||
pub fn resolve(&self, name: &str) -> Result<&Reader, String>;
|
||||
}
|
||||
```
|
||||
|
||||
- **`roncho::blocking::Client`** is `Clone`, `Send` and `Sync` (its documentation says so, and
|
||||
a given test checks it at compile time), and its `Debug` shows the timeouts and whether a key
|
||||
is present, never the key. This task's table of `roncho`'s API is complete; `roncho`'s source
|
||||
(the Cargo registry, `~/.cargo/registry`) is outside the repository and off limits.
|
||||
- **Why the warnings are filtered.** `Clients::new` and `Readers::new` each read the keys of
|
||||
the three written workspaces, and each returns a key file's plaintext warning. Edit O3 logs
|
||||
the writers' warnings, then only those of the readers' that are not among them.
|
||||
- **The given tests.**
|
||||
- `honcho_read.rs` (8 tests), against the fake Honcho: the client and the readers are
|
||||
`Send + Sync`; the timeouts, read from each client's `Debug`; each reader searches its own
|
||||
workspace with its own key (the fake records each request's `Authorization` header), and no
|
||||
header without a key; no key in a reader's `Debug`, an error, or the error of a key that
|
||||
cannot be read; `recall_sources` never holds the agent's own workspaces; each source's
|
||||
label and tag; `resolve` for every name, and the exact sentence for an unknown one; a read
|
||||
agent named `own`, `untrusted` or `fleet` refused by `config::check`, with its sentence.
|
||||
- `honcho_client.rs`: one line changes, `CHAT_READ` is 150 s.
|
||||
- `honcho_start.rs`: a new test, `a_read_key_that_cannot_be_read_stops_memoryd`, and one new
|
||||
assertion in `a_valid_section_with_keys_starts_memoryd_as_before`: the fleet key file's
|
||||
warning is printed once.
|
||||
- The items the edits use. This table is complete.
|
||||
|
||||
| Item | Where |
|
||||
|---|---|
|
||||
| `Readers::new`, `ReadTimeouts::DEFAULT` (from `read.rs`) | O3 |
|
||||
| `StartError::Secret(String)` | O3, as `Clients::new`'s error beside it |
|
||||
| `Vec::contains`, `Iterator::filter`, `Iterator::chain` | O3 |
|
||||
| `Arc::new` | O3 |
|
||||
| `[&str; 3]::contains(&&str)`, `String::as_str` | G2: `RESERVED.contains(&entry.agent.as_str())` |
|
||||
|
||||
## The edits, exactly
|
||||
|
||||
Nothing else in these files changes. Each quoted "before" block occurs exactly once in its file
|
||||
when its turn comes (make them in order). Keep the fields in the order shown.
|
||||
|
||||
### `crates/memoryd/src/honcho/mod.rs`
|
||||
|
||||
**H1.** Add one line. The line
|
||||
|
||||
```rust
|
||||
pub mod naming;
|
||||
```
|
||||
|
||||
becomes
|
||||
|
||||
```rust
|
||||
pub mod naming;
|
||||
pub mod read;
|
||||
```
|
||||
|
||||
### `crates/memoryd/src/honcho/client.rs`
|
||||
|
||||
**C1.** The lines
|
||||
|
||||
```rust
|
||||
/// A chat's read: a non-streamed chat sends nothing until it is done (M5c-2's `ask_memory`).
|
||||
pub const CHAT_READ: Duration = Duration::from_secs(180);
|
||||
```
|
||||
|
||||
become
|
||||
|
||||
```rust
|
||||
/// A chat's read: a non-streamed chat sends nothing until it is done (M5c spec, section 9).
|
||||
pub const CHAT_READ: Duration = Duration::from_secs(150);
|
||||
```
|
||||
|
||||
**C2.** The lines
|
||||
|
||||
```rust
|
||||
/// One workspace's client: no `roncho.toml` read, the timeouts given, the key when there is one.
|
||||
fn client(
|
||||
```
|
||||
|
||||
become
|
||||
|
||||
```rust
|
||||
/// One workspace's client: no `roncho.toml` read, the timeouts given, the key when there is one.
|
||||
pub(crate) fn client(
|
||||
```
|
||||
|
||||
**C3.** The lines
|
||||
|
||||
```rust
|
||||
/// The key named `name` in `[secrets]`, if one is configured, read now.
|
||||
fn key(
|
||||
```
|
||||
|
||||
become
|
||||
|
||||
```rust
|
||||
/// The key named `name` in `[secrets]`, if one is configured, read now.
|
||||
pub(crate) fn key(
|
||||
```
|
||||
|
||||
### `crates/memoryd/src/honcho/config.rs`
|
||||
|
||||
**G1.** The lines
|
||||
|
||||
```rust
|
||||
/// The names of the keys in `[secrets.<name>]`, one per workspace. A read workspace's key is
|
||||
```
|
||||
|
||||
become
|
||||
|
||||
```rust
|
||||
/// The names `ask_memory` gives this agent's own workspaces and the fleet's (M5c spec, section 9),
|
||||
/// which no `[[honcho.read]]` agent may have.
|
||||
pub const RESERVED: [&str; 3] = ["own", "untrusted", "fleet"];
|
||||
|
||||
/// The names of the keys in `[secrets.<name>]`, one per workspace. A read workspace's key is
|
||||
```
|
||||
|
||||
**G2.** In `read_workspace`, the lines (its spaces are checked: the sentence is one string split
|
||||
across two lines with a `\` at the end of the first, and the given test compares it letter for
|
||||
letter)
|
||||
|
||||
```rust
|
||||
"[[honcho.read]] agent {:?} is this agent ([honcho] agent)",
|
||||
entry.agent
|
||||
));
|
||||
}
|
||||
```
|
||||
|
||||
become
|
||||
|
||||
```rust
|
||||
"[[honcho.read]] agent {:?} is this agent ([honcho] agent)",
|
||||
entry.agent
|
||||
));
|
||||
}
|
||||
if RESERVED.contains(&entry.agent.as_str()) {
|
||||
return Err(format!(
|
||||
"[[honcho.read]] agent {:?} is a name ask_memory gives this agent's own workspaces: \
|
||||
own, untrusted and fleet cannot be agents' names",
|
||||
entry.agent
|
||||
));
|
||||
}
|
||||
```
|
||||
|
||||
### `crates/memoryd/src/honcho/start.rs`
|
||||
|
||||
**O1.** Add one line. The line
|
||||
|
||||
```rust
|
||||
use crate::honcho::mirror::Mirror;
|
||||
```
|
||||
|
||||
becomes
|
||||
|
||||
```rust
|
||||
use crate::honcho::mirror::Mirror;
|
||||
use crate::honcho::read::{ReadTimeouts, Readers};
|
||||
```
|
||||
|
||||
**O2.** The lines
|
||||
|
||||
```rust
|
||||
pub struct Opened {
|
||||
pub sync: Arc<Mutex<TurnSync>>,
|
||||
clients: Clients,
|
||||
```
|
||||
|
||||
become
|
||||
|
||||
```rust
|
||||
pub struct Opened {
|
||||
pub sync: Arc<Mutex<TurnSync>>,
|
||||
/// The workspaces `recall` and `ask_memory` read (M5c spec, sections 8 and 9).
|
||||
pub readers: Arc<Readers>,
|
||||
clients: Clients,
|
||||
```
|
||||
|
||||
**O3.** In `open`, the lines
|
||||
|
||||
```rust
|
||||
for warning in warnings {
|
||||
(log)(&warning);
|
||||
}
|
||||
Ok(Opened {
|
||||
sync: Arc::new(Mutex::new(sync)),
|
||||
clients,
|
||||
```
|
||||
|
||||
become
|
||||
|
||||
```rust
|
||||
let (readers, read_warnings) =
|
||||
Readers::new(honcho, &env, ReadTimeouts::DEFAULT).map_err(StartError::Secret)?;
|
||||
// The readers load the keys of the three written workspaces again: each warning once.
|
||||
let read_warnings: Vec<String> = read_warnings
|
||||
.into_iter()
|
||||
.filter(|w| !warnings.contains(w))
|
||||
.collect();
|
||||
for warning in warnings.into_iter().chain(read_warnings) {
|
||||
(log)(&warning);
|
||||
}
|
||||
Ok(Opened {
|
||||
sync: Arc::new(Mutex::new(sync)),
|
||||
readers: Arc::new(readers),
|
||||
clients,
|
||||
```
|
||||
|
||||
### `docs/runbook.md`
|
||||
|
||||
**D1.** In the entry `honcho-config-invalid`, the lines (its spaces are checked: the entry is
|
||||
read by people, and `scripts/check-runbook.sh` reads the file)
|
||||
|
||||
```text
|
||||
- A `[[honcho.read]]` names this agent, names an agent twice, or names one of this agent's three
|
||||
workspaces; or its `trust` is not `trusted` or `untrusted`, or its `class` not `public`,
|
||||
`private` or `secret` (all lower case).
|
||||
```
|
||||
|
||||
become
|
||||
|
||||
```text
|
||||
- A `[[honcho.read]]` names this agent, names an agent twice, names an agent `own`, `untrusted`
|
||||
or `fleet` (the names `ask_memory` gives this agent's own workspaces), or names one of this
|
||||
agent's three workspaces; or its `trust` is not `trusted` or `untrusted`, or its `class` not
|
||||
`public`, `private` or `secret` (all lower case).
|
||||
```
|
||||
|
||||
## Steps
|
||||
|
||||
- [ ] **1. Copy the given tests.** From the repository root, three commands:
|
||||
|
||||
```sh
|
||||
cp docs/plans/M5c2/files/crates/memoryd/tests/honcho_read.rs crates/memoryd/tests/
|
||||
cp docs/plans/M5c2/files/crates/memoryd/tests/honcho_client.rs crates/memoryd/tests/
|
||||
cp docs/plans/M5c2/files/crates/memoryd/tests/honcho_start.rs crates/memoryd/tests/
|
||||
```
|
||||
|
||||
Then `git status --short` lists `honcho_client.rs` and `honcho_start.rs` as modified and
|
||||
`honcho_read.rs` as new, all under `crates/memoryd/tests/`.
|
||||
- [ ] **2. See them fail.** Two commands, each with a shell timeout of 600000 ms:
|
||||
- `cargo build --tests --keep-going -p memoryd`: one test target, `honcho_read`, does not
|
||||
compile: `E0432` for ``unresolved import `memoryd::honcho::read` `` and for
|
||||
``unresolved import `memoryd::honcho::config::RESERVED` ``. Every other target compiles.
|
||||
- `cargo test --no-fail-fast -p memoryd --test honcho_client --test honcho_start`: two tests
|
||||
fail: `the_timeouts` in `honcho_client` (`CHAT_READ` is 180 s) and
|
||||
`a_read_key_that_cannot_be_read_stops_memoryd` in `honcho_start` (`memoryd` does not read
|
||||
that key yet, so it starts, and the test stops it after 10 s).
|
||||
- [ ] **3. The new module and the edits.** Copy the module:
|
||||
|
||||
```sh
|
||||
cp docs/plans/M5c2/files/crates/memoryd/src/honcho/read.rs crates/memoryd/src/honcho/
|
||||
```
|
||||
|
||||
Then make edits H1, C1 to C3, G1, G2, O1 to O3 and D1, in order. `cargo check -p memoryd`
|
||||
prints no error and no warning.
|
||||
- [ ] **4. The tests.** With a shell timeout of 600000 ms, every test passes: 13 in
|
||||
`honcho_client`, 8 in `honcho_config`, 3 in `honcho_keys` (and 1 ignored, as before), 8 in
|
||||
`honcho_read`, 5 in `honcho_start`:
|
||||
|
||||
```sh
|
||||
cargo test -p memoryd --test honcho_read --test honcho_client --test honcho_start \
|
||||
--test honcho_config --test honcho_keys
|
||||
```
|
||||
- [ ] **5. Check.** Two commands:
|
||||
- `git diff --stat -- crates/memoryd/src` lists exactly `client.rs`, `config.rs`, `mod.rs` and
|
||||
`start.rs` under `crates/memoryd/src/honcho/` (the new `read.rs` is untracked, so `git diff`
|
||||
does not list it), and `git status --short crates/memoryd/src` shows `read.rs` as new.
|
||||
- `git diff --stat -- crates/memoryd/src/serve.rs crates/memoryd/src/recall.rs Cargo.toml Cargo.lock`
|
||||
prints nothing.
|
||||
- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`, with a shell timeout of 600000 ms.
|
||||
Expected last line: `gate: ok`. The test count is the passed total of `cargo test --workspace`
|
||||
plus one, for the ignored `end_to_end` test that the gate runs on its own: **1403** after this
|
||||
task, 1394 before it; the 9 new ones are the 8 tests of `honcho_read.rs` and the new test in
|
||||
`honcho_start.rs`. To count, add up the `N passed` of every `test result:` line the gate's
|
||||
`cargo test --workspace` printed, and add 1. Never run the gate again only to count.
|
||||
- [ ] **7. Log and commit.** Add your row to `docs/implementer-log.md`; its Task cell is
|
||||
`M5c2/05-memoryd-honcho-readers`. `git status --short` lists the four edited sources, the two
|
||||
replaced tests, `docs/runbook.md` and your log as modified, and `read.rs` and `honcho_read.rs`
|
||||
as new. Stage with `git add crates/memoryd docs/runbook.md docs/implementer-log.md Cargo.lock`,
|
||||
then commit as `AGENTS.md` says. `Cargo.lock` does not change in this task; the `git add` is
|
||||
harmless.
|
||||
|
||||
## Done when
|
||||
|
||||
- `make gate` prints `gate: ok` with 1403 tests.
|
||||
- `git diff HEAD~1 --stat` lists only the four edited sources, `read.rs`, the three test files of
|
||||
step 1, `docs/runbook.md` and `docs/implementer-log.md`.
|
||||
|
||||
## Stop and report if
|
||||
|
||||
- The branch check at the top fails.
|
||||
- A "before" block is not in its file exactly as shown when its turn comes, or is there more
|
||||
than once.
|
||||
- Step 2 shows a different result: another target failing, other errors, other tests failing.
|
||||
- `cargo check` in step 3 prints an error or a warning. Quote it, and stop. Do not change
|
||||
`read.rs` to make it compile.
|
||||
- A test fails in step 4. Do not change a test: stop, and give the failing tests' names and
|
||||
their messages.
|
||||
- The fix seems to need a change anywhere else: `serve.rs`, `recall.rs`, `worker.rs`, a change
|
||||
to `read.rs`, a new dependency. It does not; stop and say why you think so.
|
||||
- A test outside the given files fails in the gate: check once whether it fails without your
|
||||
change (`git stash -u`, run it, `git stash pop`), then stop and report both results.
|
||||
- The gate passes with a count other than 1403.
|
||||
- A comment in an edit does not fit the code under it. Quote it and stop.
|
||||
|
||||
## What the given tests pin
|
||||
|
||||
Each of these changes to the finished code makes at least one given test fail (24 tried, all
|
||||
caught): the search clients given chat's timeout, or chat's the search one; either timeout
|
||||
constant changed; a read workspace given the fleet's key, the fleet the trusted key, or the own
|
||||
untrusted workspace the trusted key; the own reader on the untrusted workspace; a key not sent;
|
||||
a read workspace's trust or class ignored; the fleet trusted, or the own workspace untrusted;
|
||||
the tags swapped, or `untrusted()` inverted; `recall_sources` holding the own workspace, or not
|
||||
the fleet's; `untrusted` resolved to the trusted workspace; read agents not resolved, or
|
||||
resolved by workspace instead of name; the sentence without the read agents' names; the
|
||||
reserved names allowed; the readers' errors ignored at start; the key file's warning logged
|
||||
twice.
|
||||
@@ -77,7 +77,7 @@ Honcho sources and chats outside its state lock, and answers `ask_memory`.
|
||||
| 02 | `brokerd`: `ask_memory` as a built-in memory tool (name, parsing, the grant `builtin-ask-memory`, owner grants naming it refused, routing to `memoryd`); its answer's labels raise the session | brokerd |
|
||||
| 03 | `brokerd`: a per-tool wait on `memoryd`, 170 s for `ask_memory` (`ASK_MEMORY_WAIT`) and 30 s for the others, chosen by `MemoryPort::wait_for` | brokerd |
|
||||
| 04 | `loopd`: `ask_memory`'s schema as a core tool in a new registry; a per-tool wait on `brokerd`, 180 s for `ask_memory` and `[broker] timeout_ms` for the others; the system prompt's line on memory tools | loopd |
|
||||
| 05 | `memoryd`: read clients, one per `[[honcho.read]]` workspace, the fleet and the agent's own two, with a 5 s limit for searches and 150 s for chat | memoryd |
|
||||
| 05 | `memoryd`: read clients, one per `[[honcho.read]]` workspace, the fleet and the agent's own two, with a 5 s limit for searches and 150 s for chat; each source's label; `recall_sources` and `resolve`; `own`, `untrusted` and `fleet` reserved as read agents' names | memoryd |
|
||||
| 06 | `memoryd`, pure: Honcho messages and conclusions as `recall` items (tag, source name, class, neutralised, capped), kept only when they share a term with the query, interleaved with the local hits to `k`, the local index first | memoryd |
|
||||
| 07 | `memoryd`: `recall`'s Honcho sources, each on its own thread with its own limit, after the local search and outside the lock; `<source>: unavailable`; a query holding a secret sends nothing | memoryd |
|
||||
| 08 | `memoryd`: `ask_memory`: workspace names checked against `[honcho]`, both chats at once, each answer tagged and capped, labels, `<workspace>: unavailable`; a question holding a secret sends nothing | memoryd |
|
||||
@@ -101,6 +101,7 @@ The gate count is the passed total of `cargo test --workspace` plus one, for the
|
||||
| 02 | 1381 | 17, 17 |
|
||||
| 03 | 1384 | 10, 10 |
|
||||
| 04 | 1394 | 19, 19 |
|
||||
| 05 | 1403 | 24, 24 |
|
||||
|
||||
## Before hand-over
|
||||
|
||||
|
||||
@@ -0,0 +1,226 @@
|
||||
//! The Honcho workspaces this agent reads (M5c spec, sections 3, 8 and 9): each `[[honcho.read]]`
|
||||
//! workspace, the fleet's, and the agent's own two, which only `ask_memory` asks. Each has a
|
||||
//! client for searches and one for chat, whose answers take far longer, and the label its results
|
||||
//! carry. Nothing here writes to Honcho, keeps a response or shows a key.
|
||||
|
||||
use std::ffi::OsString;
|
||||
use std::time::Duration;
|
||||
|
||||
use bxcore::DataClass;
|
||||
use bxcore::memory::OWN_WORKSPACE;
|
||||
use roncho::blocking::Client;
|
||||
|
||||
use crate::honcho::client::{CHAT_READ, CONNECT, Timeouts, client, key};
|
||||
use crate::honcho::config::{FLEET_KEY, HonchoConfig, TRUSTED_KEY, Trust, UNTRUSTED_KEY, read_key};
|
||||
|
||||
/// How long a search may go without a byte (M5c spec, section 8).
|
||||
pub const SEARCH_READ: Duration = Duration::from_secs(5);
|
||||
/// The names `ask_memory` gives the agent's own untrusted workspace and the fleet's. The own
|
||||
/// trusted one is `bxcore::memory::OWN_WORKSPACE`, `own`.
|
||||
pub const UNTRUSTED: &str = "untrusted";
|
||||
pub const FLEET: &str = "fleet";
|
||||
|
||||
/// The timeouts the read clients are built with. Tests shorten them.
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub struct ReadTimeouts {
|
||||
pub connect: Duration,
|
||||
pub search: Duration,
|
||||
pub chat: Duration,
|
||||
}
|
||||
|
||||
impl ReadTimeouts {
|
||||
pub const DEFAULT: ReadTimeouts = ReadTimeouts {
|
||||
connect: CONNECT,
|
||||
search: SEARCH_READ,
|
||||
chat: CHAT_READ,
|
||||
};
|
||||
}
|
||||
|
||||
/// What a reader reads.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub enum Source {
|
||||
/// Another agent's workspace, from `[[honcho.read]]`, by the agent's name.
|
||||
Agent(String),
|
||||
Fleet,
|
||||
/// This agent's own trusted workspace, and its own untrusted one: `ask_memory` only.
|
||||
Own,
|
||||
OwnUntrusted,
|
||||
}
|
||||
|
||||
/// What a reader's results are labelled with: the name each hit shows, whether the owner trusts
|
||||
/// the workspace, and its class.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct Label {
|
||||
pub name: String,
|
||||
pub trust: Trust,
|
||||
pub class: DataClass,
|
||||
}
|
||||
|
||||
impl Label {
|
||||
/// The tag on each hit, as a local `recall` hit has: `memory` when trusted, else
|
||||
/// `memory-tainted`.
|
||||
pub fn tag(&self) -> &'static str {
|
||||
match self.trust {
|
||||
Trust::Trusted => "memory",
|
||||
Trust::Untrusted => "memory-tainted",
|
||||
}
|
||||
}
|
||||
|
||||
/// Whether a result makes the session untrusted.
|
||||
pub fn untrusted(&self) -> bool {
|
||||
self.trust == Trust::Untrusted
|
||||
}
|
||||
}
|
||||
|
||||
/// One workspace this agent reads: what it is, its label, and its two clients.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct Reader {
|
||||
pub source: Source,
|
||||
pub label: Label,
|
||||
search: Client,
|
||||
chat: Client,
|
||||
}
|
||||
|
||||
impl Reader {
|
||||
/// The client for searches and conclusion queries, with the short read timeout.
|
||||
pub fn search(&self) -> &Client {
|
||||
&self.search
|
||||
}
|
||||
|
||||
/// The client for workspace chat, with the long read timeout.
|
||||
pub fn chat(&self) -> &Client {
|
||||
&self.chat
|
||||
}
|
||||
}
|
||||
|
||||
/// Every workspace this agent reads, built at start with `[honcho]`. Shared by the threads that
|
||||
/// search and ask (`roncho`'s blocking client is `Send + Sync`).
|
||||
#[derive(Debug)]
|
||||
pub struct Readers {
|
||||
read: Vec<Reader>,
|
||||
fleet: Reader,
|
||||
own: Reader,
|
||||
own_untrusted: Reader,
|
||||
}
|
||||
|
||||
/// What every reader is built from, and the key files' warnings so far.
|
||||
struct Build<'a> {
|
||||
config: &'a HonchoConfig,
|
||||
env: &'a dyn Fn(&str) -> Option<OsString>,
|
||||
timeouts: ReadTimeouts,
|
||||
warnings: Vec<String>,
|
||||
}
|
||||
|
||||
impl Build<'_> {
|
||||
/// A reader of `workspace`: both clients, with the key named `key_name` when one is
|
||||
/// configured, read once.
|
||||
fn reader(
|
||||
&mut self,
|
||||
workspace: &str,
|
||||
key_name: &str,
|
||||
source: Source,
|
||||
label: Label,
|
||||
) -> Result<Reader, String> {
|
||||
let secret = key(self.config, key_name, self.env, &mut self.warnings)?;
|
||||
let search = Timeouts {
|
||||
connect: self.timeouts.connect,
|
||||
read: self.timeouts.search,
|
||||
};
|
||||
let chat = Timeouts {
|
||||
connect: self.timeouts.connect,
|
||||
read: self.timeouts.chat,
|
||||
};
|
||||
let url = &self.config.url;
|
||||
Ok(Reader {
|
||||
source,
|
||||
label,
|
||||
search: client(url, workspace, secret.as_ref(), search)?,
|
||||
chat: client(url, workspace, secret.as_ref(), chat)?,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// A `private` label.
|
||||
fn private(name: &str, trust: Trust) -> Label {
|
||||
Label {
|
||||
name: name.to_string(),
|
||||
trust,
|
||||
class: DataClass::Private,
|
||||
}
|
||||
}
|
||||
|
||||
impl Readers {
|
||||
/// A reader for each `[[honcho.read]]` workspace, in the order written, then the fleet's and
|
||||
/// the agent's own two. The warnings are the key files' plaintext warnings, for the caller to
|
||||
/// log. An error names the key or the workspace, never a key's value.
|
||||
pub fn new(
|
||||
config: &HonchoConfig,
|
||||
env: &dyn Fn(&str) -> Option<OsString>,
|
||||
timeouts: ReadTimeouts,
|
||||
) -> Result<(Readers, Vec<String>), String> {
|
||||
let mut build = Build {
|
||||
config,
|
||||
env,
|
||||
timeouts,
|
||||
warnings: Vec::new(),
|
||||
};
|
||||
let mut read = Vec::new();
|
||||
for entry in &config.read {
|
||||
let label = Label {
|
||||
name: entry.agent.clone(),
|
||||
trust: entry.trust,
|
||||
class: entry.class,
|
||||
};
|
||||
let source = Source::Agent(entry.agent.clone());
|
||||
read.push(build.reader(&entry.workspace, &read_key(&entry.agent), source, label)?);
|
||||
}
|
||||
let fleet = private(FLEET, Trust::Untrusted);
|
||||
let fleet = build.reader(&config.fleet, FLEET_KEY, Source::Fleet, fleet)?;
|
||||
let own = private(OWN_WORKSPACE, Trust::Trusted);
|
||||
let own = build.reader(&config.trusted, TRUSTED_KEY, Source::Own, own)?;
|
||||
let own_untrusted = private(UNTRUSTED, Trust::Untrusted);
|
||||
let own_untrusted = build.reader(
|
||||
&config.untrusted,
|
||||
UNTRUSTED_KEY,
|
||||
Source::OwnUntrusted,
|
||||
own_untrusted,
|
||||
)?;
|
||||
let readers = Readers {
|
||||
read,
|
||||
fleet,
|
||||
own,
|
||||
own_untrusted,
|
||||
};
|
||||
Ok((readers, build.warnings))
|
||||
}
|
||||
|
||||
/// What `recall` searches: the `[[honcho.read]]` workspaces in the order written, then the
|
||||
/// fleet's. Never the agent's own (M5c spec, section 8): the local index holds their text.
|
||||
pub fn recall_sources(&self) -> Vec<&Reader> {
|
||||
self.read
|
||||
.iter()
|
||||
.chain(std::iter::once(&self.fleet))
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// The reader `ask_memory` names `name`: `own`, `untrusted`, `fleet`, or a `[[honcho.read]]`
|
||||
/// agent. Any other name is the sentence the model is shown, listing the names it may use.
|
||||
pub fn resolve(&self, name: &str) -> Result<&Reader, String> {
|
||||
match name {
|
||||
OWN_WORKSPACE => return Ok(&self.own),
|
||||
UNTRUSTED => return Ok(&self.own_untrusted),
|
||||
FLEET => return Ok(&self.fleet),
|
||||
_ => {}
|
||||
}
|
||||
if let Some(reader) = self.read.iter().find(|r| r.label.name == name) {
|
||||
return Ok(reader);
|
||||
}
|
||||
let mut names = vec![OWN_WORKSPACE, UNTRUSTED, FLEET];
|
||||
names.extend(self.read.iter().map(|r| r.label.name.as_str()));
|
||||
let (last, rest) = names.split_last().unwrap_or((&FLEET, &[]));
|
||||
Err(format!(
|
||||
"no workspace is named {name}; ask {} or {last}",
|
||||
rest.join(", ")
|
||||
))
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,439 @@
|
||||
//! `memoryd`'s Honcho clients (M5c spec, sections 3 and 6), against a fake Honcho that answers
|
||||
//! from the responses recorded from rift: the workspace, the peers and the session made before
|
||||
//! anything is written, each once per run, with the fleet's settings and the agent unobserved;
|
||||
//! the errors by kind and status, never a body. The keys are in `honcho_keys.rs`. Do not edit.
|
||||
|
||||
#[path = "support/honcho.rs"]
|
||||
mod honcho;
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::ffi::OsString;
|
||||
use std::time::Duration;
|
||||
|
||||
use honcho::{FakeHoncho, Mode, Request};
|
||||
use memoryd::honcho::client::{
|
||||
CHAT_READ, CONNECT, Clients, HonchoError, READ, Timeouts, classify, fleet_configuration,
|
||||
};
|
||||
use memoryd::honcho::config::HonchoConfig;
|
||||
use memoryd::honcho::ledger::Workspace;
|
||||
use memoryd::honcho::naming::HonchoSession;
|
||||
use serde_json::{Value, json};
|
||||
|
||||
const FAST: Timeouts = Timeouts {
|
||||
connect: Duration::from_secs(5),
|
||||
read: Duration::from_millis(300),
|
||||
};
|
||||
|
||||
fn config(url: &str) -> HonchoConfig {
|
||||
HonchoConfig {
|
||||
url: url.to_string(),
|
||||
agent: "boxmaker0".to_string(),
|
||||
owner: "kyle".to_string(),
|
||||
trusted: "boxmaker0-trusted".to_string(),
|
||||
untrusted: "boxmaker0-untrusted".to_string(),
|
||||
fleet: "fleet".to_string(),
|
||||
read: Vec::new(),
|
||||
keys: BTreeMap::new(),
|
||||
}
|
||||
}
|
||||
|
||||
fn no_env(_: &str) -> Option<OsString> {
|
||||
None
|
||||
}
|
||||
|
||||
fn clients(fake: &FakeHoncho) -> Clients {
|
||||
Clients::new(&config(&fake.url), &no_env, FAST).unwrap().0
|
||||
}
|
||||
|
||||
fn session(name: &str, speakers: &[&str]) -> HonchoSession {
|
||||
HonchoSession {
|
||||
name: name.to_string(),
|
||||
speakers: speakers.iter().map(|s| s.to_string()).collect(),
|
||||
}
|
||||
}
|
||||
|
||||
/// A request as (method, path, body).
|
||||
fn seen(requests: &[Request]) -> Vec<(String, String, Value)> {
|
||||
requests
|
||||
.iter()
|
||||
.map(|r| {
|
||||
(
|
||||
r.method.clone(),
|
||||
r.path.clone(),
|
||||
r.body.clone().unwrap_or(Value::Null),
|
||||
)
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn post(path: &str, body: Value) -> (String, String, Value) {
|
||||
("POST".to_string(), path.to_string(), body)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_timeouts() {
|
||||
assert_eq!(CONNECT, Duration::from_secs(5));
|
||||
assert_eq!(READ, Duration::from_secs(30));
|
||||
assert_eq!(CHAT_READ, Duration::from_secs(150));
|
||||
assert_eq!(
|
||||
Timeouts::WRITES,
|
||||
Timeouts {
|
||||
connect: CONNECT,
|
||||
read: READ
|
||||
}
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn each_client_has_its_workspace_and_timeouts() {
|
||||
let fake = FakeHoncho::start();
|
||||
let c = clients(&fake);
|
||||
for (workspace, name) in [
|
||||
(Workspace::Trusted, "boxmaker0-trusted"),
|
||||
(Workspace::Untrusted, "boxmaker0-untrusted"),
|
||||
(Workspace::Fleet, "fleet"),
|
||||
] {
|
||||
let client = c.client(workspace);
|
||||
assert_eq!(client.workspace_id(), name);
|
||||
// `roncho`'s `Debug` shows the timeouts, and whether there is a key, never the key.
|
||||
let shown = format!("{client:?}");
|
||||
assert!(shown.contains("connect_timeout: 5s"), "{shown}");
|
||||
assert!(shown.contains("read_timeout: 300ms"), "{shown}");
|
||||
assert!(shown.contains("api_key_present: false"), "{shown}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_fleets_configuration_is_the_recorded_one() {
|
||||
let path = concat!(
|
||||
env!("CARGO_MANIFEST_DIR"),
|
||||
"/tests/fixtures/honcho/01-workspace-create.json"
|
||||
);
|
||||
let recorded: Value = serde_json::from_str(&std::fs::read_to_string(path).unwrap()).unwrap();
|
||||
assert_eq!(
|
||||
Value::Object(fleet_configuration()),
|
||||
recorded["request"]["body"]["configuration"]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_session_is_made_after_its_workspace_and_peers() {
|
||||
let fake = FakeHoncho::start();
|
||||
let mut c = clients(&fake);
|
||||
c.ensure_session(Workspace::Trusted, &session("chat-1", &["kyle"]))
|
||||
.unwrap();
|
||||
let w = "/v3/workspaces/boxmaker0-trusted";
|
||||
assert_eq!(
|
||||
seen(&fake.requests()),
|
||||
vec![
|
||||
post("/v3/workspaces", json!({ "id": "boxmaker0-trusted" })),
|
||||
post(
|
||||
&format!("{w}/peers"),
|
||||
json!({ "id": "boxmaker0", "configuration": { "observe_me": false } })
|
||||
),
|
||||
post(&format!("{w}/peers"), json!({ "id": "kyle" })),
|
||||
post(
|
||||
&format!("{w}/sessions"),
|
||||
json!({ "id": "chat-1", "peers": {
|
||||
"boxmaker0": { "observe_me": false, "observe_others": false },
|
||||
"kyle": { "observe_me": true, "observe_others": false },
|
||||
} })
|
||||
),
|
||||
]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn what_was_made_is_not_asked_for_again_in_this_run() {
|
||||
let fake = FakeHoncho::start();
|
||||
let mut c = clients(&fake);
|
||||
c.ensure_session(Workspace::Trusted, &session("chat-1", &["kyle"]))
|
||||
.unwrap();
|
||||
fake.clear();
|
||||
c.ensure_session(Workspace::Trusted, &session("chat-1", &["kyle"]))
|
||||
.unwrap();
|
||||
assert!(fake.requests().is_empty(), "{:?}", fake.requests());
|
||||
// A new speaker: only its peer and the next session.
|
||||
c.ensure_session(Workspace::Trusted, &session("chat-1-2", &["kyle", "ana_b"]))
|
||||
.unwrap();
|
||||
let w = "/v3/workspaces/boxmaker0-trusted";
|
||||
assert_eq!(
|
||||
seen(&fake.requests()),
|
||||
vec![
|
||||
post(&format!("{w}/peers"), json!({ "id": "ana_b" })),
|
||||
post(
|
||||
&format!("{w}/sessions"),
|
||||
json!({ "id": "chat-1-2", "peers": {
|
||||
"boxmaker0": { "observe_me": false, "observe_others": false },
|
||||
"kyle": { "observe_me": true, "observe_others": false },
|
||||
"ana_b": { "observe_me": true, "observe_others": false },
|
||||
} })
|
||||
),
|
||||
]
|
||||
);
|
||||
// Another workspace is made sure of on its own.
|
||||
fake.clear();
|
||||
c.ensure_session(Workspace::Untrusted, &session("chat-1", &["kyle"]))
|
||||
.unwrap();
|
||||
assert_eq!(fake.requests().len(), 4);
|
||||
assert!(
|
||||
fake.requests()
|
||||
.iter()
|
||||
.skip(1)
|
||||
.all(|r| r.path.starts_with("/v3/workspaces/boxmaker0-untrusted/"))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn after_a_restart_everything_is_made_sure_of_again() {
|
||||
let fake = FakeHoncho::start();
|
||||
clients(&fake)
|
||||
.ensure_session(Workspace::Trusted, &session("chat-1", &["kyle"]))
|
||||
.unwrap();
|
||||
fake.clear();
|
||||
// Honcho answers 200 now: it has them all. That is success too.
|
||||
clients(&fake)
|
||||
.ensure_session(Workspace::Trusted, &session("chat-1", &["kyle"]))
|
||||
.unwrap();
|
||||
assert_eq!(fake.requests().len(), 4);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_fleet_is_made_with_its_settings_and_the_agent_alone() {
|
||||
let fake = FakeHoncho::start();
|
||||
let mut c = clients(&fake);
|
||||
c.ensure_session(Workspace::Fleet, &session("chat-1", &[]))
|
||||
.unwrap();
|
||||
assert_eq!(
|
||||
seen(&fake.requests()),
|
||||
vec![
|
||||
post(
|
||||
"/v3/workspaces",
|
||||
json!({ "id": "fleet", "configuration": {
|
||||
"reasoning": { "enabled": false },
|
||||
"summary": { "enabled": false },
|
||||
"dream": { "enabled": false },
|
||||
} })
|
||||
),
|
||||
post(
|
||||
"/v3/workspaces/fleet/peers",
|
||||
json!({ "id": "boxmaker0", "configuration": { "observe_me": false } })
|
||||
),
|
||||
post(
|
||||
"/v3/workspaces/fleet/sessions",
|
||||
json!({ "id": "chat-1", "peers": {
|
||||
"boxmaker0": { "observe_me": false, "observe_others": false },
|
||||
} })
|
||||
),
|
||||
]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_409_on_create_means_it_is_there() {
|
||||
let fake = FakeHoncho::start();
|
||||
let mut c = clients(&fake);
|
||||
fake.set(Mode::Status(409));
|
||||
c.ensure_workspace(Workspace::Trusted).unwrap();
|
||||
c.ensure_peer(Workspace::Trusted, "kyle").unwrap();
|
||||
fake.clear();
|
||||
c.ensure_workspace(Workspace::Trusted).unwrap();
|
||||
assert!(fake.requests().is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn errors_by_kind_and_status() {
|
||||
let fake = FakeHoncho::start();
|
||||
let mut c = clients(&fake);
|
||||
let cases = [
|
||||
(
|
||||
500,
|
||||
HonchoError::Unreachable {
|
||||
kind: "server",
|
||||
status: Some(500),
|
||||
},
|
||||
),
|
||||
(
|
||||
503,
|
||||
HonchoError::Unreachable {
|
||||
kind: "server",
|
||||
status: Some(503),
|
||||
},
|
||||
),
|
||||
(401, HonchoError::KeyRefused { status: 401 }),
|
||||
(403, HonchoError::KeyRefused { status: 403 }),
|
||||
(
|
||||
400,
|
||||
HonchoError::Refused {
|
||||
kind: "status",
|
||||
status: Some(400),
|
||||
},
|
||||
),
|
||||
(
|
||||
404,
|
||||
HonchoError::Refused {
|
||||
kind: "status",
|
||||
status: Some(404),
|
||||
},
|
||||
),
|
||||
(
|
||||
422,
|
||||
HonchoError::Refused {
|
||||
kind: "status",
|
||||
status: Some(422),
|
||||
},
|
||||
),
|
||||
];
|
||||
for (status, expected) in cases {
|
||||
fake.set(Mode::Status(status));
|
||||
assert_eq!(
|
||||
c.ensure_workspace(Workspace::Trusted),
|
||||
Err(expected.clone())
|
||||
);
|
||||
assert_eq!(c.ensure_peer(Workspace::Fleet, "kyle"), Err(expected));
|
||||
}
|
||||
// Nothing failed is remembered: the next call asks again, and succeeds.
|
||||
fake.set(Mode::Fixtures);
|
||||
fake.clear();
|
||||
c.ensure_workspace(Workspace::Trusted).unwrap();
|
||||
assert_eq!(fake.requests().len(), 1);
|
||||
fake.set(Mode::Hang);
|
||||
assert_eq!(
|
||||
c.ensure_peer(Workspace::Trusted, "ana_b"),
|
||||
Err(HonchoError::Unreachable {
|
||||
kind: "timeout",
|
||||
status: None
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn nothing_listening_is_unreachable() {
|
||||
let port = {
|
||||
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
|
||||
listener.local_addr().unwrap().port()
|
||||
};
|
||||
let (mut c, _) =
|
||||
Clients::new(&config(&format!("http://127.0.0.1:{port}")), &no_env, FAST).unwrap();
|
||||
assert_eq!(
|
||||
c.ensure_workspace(Workspace::Trusted),
|
||||
Err(HonchoError::Unreachable {
|
||||
kind: "connect",
|
||||
status: None
|
||||
})
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_error_never_holds_the_response() {
|
||||
let fake = FakeHoncho::start();
|
||||
let mut c = clients(&fake);
|
||||
fake.set(Mode::Status(422));
|
||||
let err = c
|
||||
.ensure_peer(Workspace::Trusted, "private-note-text")
|
||||
.unwrap_err();
|
||||
let body = fake.requests()[0].body.clone().unwrap();
|
||||
assert_eq!(body["id"], "private-note-text", "the fake did not echo it");
|
||||
for shown in [err.to_string(), format!("{err:?}")] {
|
||||
assert!(!shown.contains("private-note-text"), "{shown}");
|
||||
}
|
||||
assert_eq!(
|
||||
err.to_string(),
|
||||
"Honcho refused the request (status, HTTP 422)"
|
||||
);
|
||||
assert_eq!((err.kind(), err.status()), ("status", Some(422)));
|
||||
let lines = [
|
||||
HonchoError::Unreachable {
|
||||
kind: "timeout",
|
||||
status: None,
|
||||
}
|
||||
.to_string(),
|
||||
HonchoError::KeyRefused { status: 401 }.to_string(),
|
||||
];
|
||||
assert_eq!(lines[0], "Honcho is unreachable (timeout)");
|
||||
assert_eq!(lines[1], "Honcho refused the key (key-refused, HTTP 401)");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_roncho_error_is_classified() {
|
||||
use roncho::blocking::Error;
|
||||
let io = || std::io::Error::other("x");
|
||||
let cases = [
|
||||
(Error::Connect(io()), "connect", None, true),
|
||||
(Error::Timeout, "timeout", None, true),
|
||||
(Error::Closed, "closed", None, true),
|
||||
(Error::Io(io()), "io", None, true),
|
||||
(
|
||||
Error::Status {
|
||||
status: 502,
|
||||
body: "b".into(),
|
||||
},
|
||||
"server",
|
||||
Some(502),
|
||||
true,
|
||||
),
|
||||
(
|
||||
Error::Status {
|
||||
status: 499,
|
||||
body: "b".into(),
|
||||
},
|
||||
"status",
|
||||
Some(499),
|
||||
false,
|
||||
),
|
||||
(
|
||||
Error::Redirect { status: 301 },
|
||||
"redirect",
|
||||
Some(301),
|
||||
false,
|
||||
),
|
||||
(
|
||||
Error::ActiveSessions { body: "b".into() },
|
||||
"active-sessions",
|
||||
Some(409),
|
||||
false,
|
||||
),
|
||||
(Error::TooLarge, "too-large", None, false),
|
||||
(Error::Decode("b".into()), "decode", None, false),
|
||||
(Error::Stream("b".into()), "stream", None, false),
|
||||
(Error::Config("b".into()), "config", None, false),
|
||||
];
|
||||
for (error, kind, status, unreachable) in cases {
|
||||
let got = classify(&error);
|
||||
assert_eq!((got.kind(), got.status()), (kind, status), "{error:?}");
|
||||
assert_eq!(
|
||||
matches!(got, HonchoError::Unreachable { .. }),
|
||||
unreachable,
|
||||
"{error:?}"
|
||||
);
|
||||
}
|
||||
let got = classify(&Error::Unauthorized { status: 403 });
|
||||
assert_eq!(got, HonchoError::KeyRefused { status: 403 });
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_conclusion_deleted_or_gone_is_done() {
|
||||
let fake = FakeHoncho::start();
|
||||
let c = clients(&fake);
|
||||
c.delete_conclusion(Workspace::Trusted, "YUBMqIqHhuJprV4o-inrg")
|
||||
.unwrap();
|
||||
c.delete_conclusion(Workspace::Trusted, "YUBMqIqHhuJprV4o-inrg")
|
||||
.unwrap();
|
||||
let requests = fake.requests();
|
||||
assert_eq!(requests.len(), 2);
|
||||
assert!(requests.iter().all(|r| r.method == "DELETE"
|
||||
&& r.path == "/v3/workspaces/boxmaker0-trusted/conclusions/YUBMqIqHhuJprV4o-inrg"));
|
||||
fake.set(Mode::Status(500));
|
||||
assert_eq!(
|
||||
c.delete_conclusion(Workspace::Trusted, "x"),
|
||||
Err(HonchoError::Unreachable {
|
||||
kind: "server",
|
||||
status: Some(500)
|
||||
})
|
||||
);
|
||||
fake.set(Mode::Status(403));
|
||||
assert_eq!(
|
||||
c.delete_conclusion(Workspace::Trusted, "x"),
|
||||
Err(HonchoError::KeyRefused { status: 403 })
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,367 @@
|
||||
//! The workspaces `memoryd` reads (M5c spec, sections 3, 8 and 9): a reader for each
|
||||
//! `[[honcho.read]]` workspace, the fleet's and the agent's own two, each with its own key, a
|
||||
//! short wait for searches and a long one for chat; the label its results carry; what `recall`
|
||||
//! searches and the names `ask_memory` may use. Against a fake Honcho. Do not edit.
|
||||
|
||||
#[path = "support/honcho.rs"]
|
||||
mod honcho;
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::ffi::OsString;
|
||||
use std::time::Duration;
|
||||
|
||||
use bxcore::DataClass;
|
||||
use bxcore::secrets::SecretSource;
|
||||
use honcho::{FakeHoncho, Mode};
|
||||
use memoryd::honcho::client::{CHAT_READ, CONNECT, classify};
|
||||
use memoryd::honcho::config::{
|
||||
FLEET_KEY, HonchoConfig, HonchoSection, RESERVED, ReadSection, ReadWorkspace, TRUSTED_KEY,
|
||||
Trust, check, read_key,
|
||||
};
|
||||
use memoryd::honcho::read::{
|
||||
FLEET, Label, ReadTimeouts, Reader, Readers, SEARCH_READ, Source, UNTRUSTED,
|
||||
};
|
||||
use roncho::MessageSearch;
|
||||
|
||||
const FAST: ReadTimeouts = ReadTimeouts {
|
||||
connect: Duration::from_secs(5),
|
||||
search: Duration::from_millis(300),
|
||||
chat: Duration::from_millis(900),
|
||||
};
|
||||
|
||||
/// Key values built from pieces, so that no file holds one whole.
|
||||
fn key(which: &str) -> String {
|
||||
["hk", which, "7a31c0"].join("-")
|
||||
}
|
||||
|
||||
fn read(agent: &str, trust: Trust, class: DataClass) -> ReadWorkspace {
|
||||
ReadWorkspace {
|
||||
agent: agent.to_string(),
|
||||
workspace: format!("{agent}-trusted"),
|
||||
trust,
|
||||
class,
|
||||
}
|
||||
}
|
||||
|
||||
/// `boxmaker0`, reading `boxmaker1` (labelled trusted, `secret`) and `boxmaker2` (the defaults).
|
||||
/// Keys for `boxmaker1`, the fleet and the own trusted workspace; none for the others.
|
||||
fn config(url: &str) -> HonchoConfig {
|
||||
let mut keys = BTreeMap::new();
|
||||
for (name, var) in [
|
||||
(read_key("boxmaker1"), "K_READ_1"),
|
||||
(FLEET_KEY.to_string(), "K_FLEET"),
|
||||
(TRUSTED_KEY.to_string(), "K_TRUSTED"),
|
||||
] {
|
||||
keys.insert(name, SecretSource::Env(var.to_string()));
|
||||
}
|
||||
HonchoConfig {
|
||||
url: url.to_string(),
|
||||
agent: "boxmaker0".to_string(),
|
||||
owner: "kyle".to_string(),
|
||||
trusted: "boxmaker0-trusted".to_string(),
|
||||
untrusted: "boxmaker0-untrusted".to_string(),
|
||||
fleet: "fleet".to_string(),
|
||||
read: vec![
|
||||
read("boxmaker1", Trust::Trusted, DataClass::Secret),
|
||||
read("boxmaker2", Trust::Untrusted, DataClass::Public),
|
||||
],
|
||||
keys,
|
||||
}
|
||||
}
|
||||
|
||||
fn env(name: &str) -> Option<OsString> {
|
||||
match name {
|
||||
"K_READ_1" => Some(OsString::from(key("read1"))),
|
||||
"K_FLEET" => Some(OsString::from(key("fleet"))),
|
||||
"K_TRUSTED" => Some(OsString::from(key("trusted"))),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
|
||||
fn readers(url: &str) -> Readers {
|
||||
Readers::new(&config(url), &env, FAST).unwrap().0
|
||||
}
|
||||
|
||||
/// Every reader: `recall`'s, then the own two.
|
||||
fn all(r: &Readers) -> Vec<&Reader> {
|
||||
let mut all = r.recall_sources();
|
||||
all.push(r.resolve("own").unwrap());
|
||||
all.push(r.resolve("untrusted").unwrap());
|
||||
all
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_reader_client_can_be_shared_across_threads() {
|
||||
fn shared<T: Send + Sync>() {}
|
||||
shared::<roncho::blocking::Client>();
|
||||
shared::<Reader>();
|
||||
shared::<Readers>();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn the_timeouts() {
|
||||
assert_eq!(SEARCH_READ, Duration::from_secs(5));
|
||||
assert_eq!(CHAT_READ, Duration::from_secs(150));
|
||||
assert_eq!(
|
||||
ReadTimeouts::DEFAULT,
|
||||
ReadTimeouts {
|
||||
connect: CONNECT,
|
||||
search: SEARCH_READ,
|
||||
chat: CHAT_READ
|
||||
}
|
||||
);
|
||||
let fake = FakeHoncho::start();
|
||||
let r = Readers::new(&config(&fake.url), &env, ReadTimeouts::DEFAULT)
|
||||
.unwrap()
|
||||
.0;
|
||||
for reader in all(&r) {
|
||||
// `roncho`'s `Debug` shows the timeouts, and whether there is a key, never the key.
|
||||
let search = format!("{:?}", reader.search());
|
||||
let chat = format!("{:?}", reader.chat());
|
||||
for shown in [&search, &chat] {
|
||||
assert!(shown.contains("connect_timeout: 5s"), "{shown}");
|
||||
}
|
||||
assert!(search.contains("read_timeout: 5s"), "{search}");
|
||||
assert!(chat.contains("read_timeout: 150s"), "{chat}");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn each_reader_reads_its_own_workspace_with_its_own_key() {
|
||||
let fake = FakeHoncho::start();
|
||||
let r = readers(&fake.url);
|
||||
let want = [
|
||||
("boxmaker1-trusted", Some(key("read1"))),
|
||||
("boxmaker2-trusted", None),
|
||||
("fleet", Some(key("fleet"))),
|
||||
("boxmaker0-trusted", Some(key("trusted"))),
|
||||
("boxmaker0-untrusted", None),
|
||||
];
|
||||
let readers = all(&r);
|
||||
assert_eq!(readers.len(), want.len());
|
||||
for (reader, (workspace, _)) in readers.iter().zip(&want) {
|
||||
assert_eq!(reader.search().workspace_id(), *workspace);
|
||||
assert_eq!(reader.chat().workspace_id(), *workspace);
|
||||
// The fake answers a search it has no recording for with 404: only the request matters.
|
||||
let _ = reader
|
||||
.search()
|
||||
.search_workspace(&MessageSearch::new("launch"));
|
||||
let _ = reader
|
||||
.chat()
|
||||
.search_workspace(&MessageSearch::new("launch"));
|
||||
}
|
||||
let requests = fake.requests();
|
||||
assert_eq!(requests.len(), 2 * want.len());
|
||||
for (n, request) in requests.iter().enumerate() {
|
||||
let (workspace, secret) = &want[n / 2];
|
||||
assert_eq!(request.path, format!("/v3/workspaces/{workspace}/search"));
|
||||
let bearer = secret.as_ref().map(|k| format!("Bearer {k}"));
|
||||
assert_eq!(
|
||||
request.header("authorization").map(str::to_string),
|
||||
bearer,
|
||||
"{workspace}"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn no_key_is_shown_by_a_reader_or_an_error() {
|
||||
let fake = FakeHoncho::start();
|
||||
let r = readers(&fake.url);
|
||||
fake.set(Mode::Status(401));
|
||||
let mut shown = vec![format!("{r:?}")];
|
||||
for reader in all(&r) {
|
||||
let err = reader
|
||||
.search()
|
||||
.search_workspace(&MessageSearch::new("launch"))
|
||||
.unwrap_err();
|
||||
let err = classify(&err);
|
||||
shown.push(format!("{err:?}"));
|
||||
shown.push(err.to_string());
|
||||
}
|
||||
for text in shown {
|
||||
for which in ["read1", "fleet", "trusted"] {
|
||||
assert!(!text.contains(&key(which)), "{text}");
|
||||
}
|
||||
}
|
||||
// A key that cannot be read stops the build, naming the key and never a value.
|
||||
let mut config = config(&fake.url);
|
||||
config.keys.insert(
|
||||
read_key("boxmaker2"),
|
||||
SecretSource::Env("K_MISSING".to_string()),
|
||||
);
|
||||
let err = Readers::new(&config, &env, FAST).unwrap_err();
|
||||
assert!(err.contains("honcho_read_boxmaker2"), "{err}");
|
||||
assert!(
|
||||
err.contains("see docs/runbook.md#secret-unavailable"),
|
||||
"{err}"
|
||||
);
|
||||
for which in ["read1", "fleet", "trusted"] {
|
||||
assert!(!err.contains(&key(which)), "{err}");
|
||||
}
|
||||
}
|
||||
|
||||
/// `recall` searches the read workspaces, in the order written, then the fleet's; never the
|
||||
/// agent's own two (M5c spec, section 8).
|
||||
#[test]
|
||||
fn recall_never_searches_the_agents_own_workspaces() {
|
||||
let fake = FakeHoncho::start();
|
||||
let r = readers(&fake.url);
|
||||
let sources: Vec<Source> = r
|
||||
.recall_sources()
|
||||
.iter()
|
||||
.map(|s| s.source.clone())
|
||||
.collect();
|
||||
assert_eq!(
|
||||
sources,
|
||||
[
|
||||
Source::Agent("boxmaker1".to_string()),
|
||||
Source::Agent("boxmaker2".to_string()),
|
||||
Source::Fleet
|
||||
]
|
||||
);
|
||||
let workspaces: Vec<&str> = r
|
||||
.recall_sources()
|
||||
.iter()
|
||||
.map(|s| s.search().workspace_id())
|
||||
.collect();
|
||||
assert_eq!(
|
||||
workspaces,
|
||||
["boxmaker1-trusted", "boxmaker2-trusted", "fleet"]
|
||||
);
|
||||
let mut none = config(&fake.url);
|
||||
none.read.clear();
|
||||
let r = Readers::new(&none, &env, FAST).unwrap().0;
|
||||
let sources: Vec<Source> = r
|
||||
.recall_sources()
|
||||
.iter()
|
||||
.map(|s| s.source.clone())
|
||||
.collect();
|
||||
assert_eq!(sources, [Source::Fleet]);
|
||||
}
|
||||
|
||||
fn label(name: &str, trust: Trust, class: DataClass) -> Label {
|
||||
Label {
|
||||
name: name.to_string(),
|
||||
trust,
|
||||
class,
|
||||
}
|
||||
}
|
||||
|
||||
/// A result is tagged `memory` only from a workspace the owner labelled trusted, or the agent's
|
||||
/// own trusted one; the fleet and everything else is `memory-tainted`.
|
||||
#[test]
|
||||
fn each_source_has_its_label() {
|
||||
let fake = FakeHoncho::start();
|
||||
let r = readers(&fake.url);
|
||||
let got: Vec<(Label, &str, bool)> = all(&r)
|
||||
.iter()
|
||||
.map(|reader| {
|
||||
(
|
||||
reader.label.clone(),
|
||||
reader.label.tag(),
|
||||
reader.label.untrusted(),
|
||||
)
|
||||
})
|
||||
.collect();
|
||||
assert_eq!(
|
||||
got,
|
||||
[
|
||||
(
|
||||
label("boxmaker1", Trust::Trusted, DataClass::Secret),
|
||||
"memory",
|
||||
false
|
||||
),
|
||||
(
|
||||
label("boxmaker2", Trust::Untrusted, DataClass::Public),
|
||||
"memory-tainted",
|
||||
true
|
||||
),
|
||||
(
|
||||
label("fleet", Trust::Untrusted, DataClass::Private),
|
||||
"memory-tainted",
|
||||
true
|
||||
),
|
||||
(
|
||||
label("own", Trust::Trusted, DataClass::Private),
|
||||
"memory",
|
||||
false
|
||||
),
|
||||
(
|
||||
label("untrusted", Trust::Untrusted, DataClass::Private),
|
||||
"memory-tainted",
|
||||
true
|
||||
),
|
||||
]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ask_memory_names_resolve_to_their_readers() {
|
||||
let fake = FakeHoncho::start();
|
||||
let r = readers(&fake.url);
|
||||
for (name, source, workspace) in [
|
||||
("own", Source::Own, "boxmaker0-trusted"),
|
||||
(UNTRUSTED, Source::OwnUntrusted, "boxmaker0-untrusted"),
|
||||
(FLEET, Source::Fleet, "fleet"),
|
||||
(
|
||||
"boxmaker1",
|
||||
Source::Agent("boxmaker1".to_string()),
|
||||
"boxmaker1-trusted",
|
||||
),
|
||||
(
|
||||
"boxmaker2",
|
||||
Source::Agent("boxmaker2".to_string()),
|
||||
"boxmaker2-trusted",
|
||||
),
|
||||
] {
|
||||
let reader = r.resolve(name).unwrap();
|
||||
assert_eq!(reader.source, source, "{name}");
|
||||
assert_eq!(reader.chat().workspace_id(), workspace, "{name}");
|
||||
}
|
||||
for unknown in ["boxmaker9", "boxmaker0", "Own", "boxmaker1-trusted"] {
|
||||
assert_eq!(
|
||||
r.resolve(unknown).unwrap_err(),
|
||||
format!(
|
||||
"no workspace is named {unknown}; ask own, untrusted, fleet, boxmaker1 or boxmaker2"
|
||||
)
|
||||
);
|
||||
}
|
||||
let mut none = config(&fake.url);
|
||||
none.read.clear();
|
||||
let r = Readers::new(&none, &env, FAST).unwrap().0;
|
||||
assert_eq!(
|
||||
r.resolve("boxmaker1").unwrap_err(),
|
||||
"no workspace is named boxmaker1; ask own, untrusted or fleet"
|
||||
);
|
||||
}
|
||||
|
||||
/// `own`, `untrusted` and `fleet` name this agent's workspaces in `ask_memory`, so no
|
||||
/// `[[honcho.read]]` agent may be called that: the fleet can never be labelled trusted.
|
||||
#[test]
|
||||
fn no_read_agent_takes_a_reserved_name() {
|
||||
assert_eq!(RESERVED, ["own", "untrusted", "fleet"]);
|
||||
for agent in RESERVED {
|
||||
let section = HonchoSection {
|
||||
url: "http://127.0.0.1:9".to_string(),
|
||||
agent: "boxmaker0".to_string(),
|
||||
owner: "kyle".to_string(),
|
||||
trusted: "boxmaker0-trusted".to_string(),
|
||||
untrusted: "boxmaker0-untrusted".to_string(),
|
||||
fleet: "fleet".to_string(),
|
||||
read: vec![ReadSection {
|
||||
agent: agent.to_string(),
|
||||
workspace: "someone-trusted".to_string(),
|
||||
trust: Some("trusted".to_string()),
|
||||
class: None,
|
||||
}],
|
||||
};
|
||||
assert_eq!(
|
||||
check(Some(§ion), &BTreeMap::new()).unwrap_err(),
|
||||
format!(
|
||||
"[[honcho.read]] agent {agent:?} is a name ask_memory gives this agent's own \
|
||||
workspaces: own, untrusted and fleet cannot be agents' names"
|
||||
)
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,262 @@
|
||||
//! `memoryd serve` with a `[honcho]` section (M5c spec, section 10): an invalid one stops it at
|
||||
//! start with the reason and the `honcho-config-invalid` pointer, before it takes its lock or
|
||||
//! touches its home; a valid one starts it as before. A key is never printed. Every wait has a
|
||||
//! deadline, and every process is killed and every path removed by a guard (tip T44). Do not
|
||||
//! edit.
|
||||
|
||||
#[path = "support/tmp.rs"]
|
||||
mod tmp;
|
||||
|
||||
use std::os::unix::fs::PermissionsExt;
|
||||
use std::os::unix::net::UnixStream;
|
||||
use std::path::PathBuf;
|
||||
use std::process::{Child, Command, Stdio};
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
use bxcore::{Empty, Envelope, Message, PROTOCOL_VERSION, read_frame, write_frame};
|
||||
use tmp::TempDir;
|
||||
|
||||
const POINTER: &str = "see docs/runbook.md#honcho-config-invalid";
|
||||
const START_FAILED: &str = "see docs/runbook.md#memoryd-start-failed";
|
||||
|
||||
/// The values of the two keys the configs name. Built from pieces, so that no file holds one
|
||||
/// whole; neither may appear in anything `memoryd` prints.
|
||||
fn env_key() -> String {
|
||||
["hk", "env", "7f3a9c"].join("-")
|
||||
}
|
||||
fn file_key() -> String {
|
||||
["hk", "file", "b81d2e"].join("-")
|
||||
}
|
||||
|
||||
const SECTION: &str = "[honcho]
|
||||
url = \"http://127.0.0.1:9\"
|
||||
agent = \"boxmaker0\"
|
||||
owner = \"kyle\"
|
||||
trusted = \"boxmaker0-trusted\"
|
||||
untrusted = \"boxmaker0-untrusted\"
|
||||
fleet = \"fleet\"
|
||||
[[honcho.read]]
|
||||
agent = \"boxmaker1\"
|
||||
workspace = \"boxmaker1-trusted\"
|
||||
";
|
||||
|
||||
struct Home {
|
||||
dir: TempDir,
|
||||
config: PathBuf,
|
||||
}
|
||||
|
||||
impl Home {
|
||||
/// A home whose `memoryd.toml` is the usual one and `extra`, with a key file holding
|
||||
/// `file_key()` (mode 0600) at `<home>/keys/honcho.key`.
|
||||
fn new(tag: &str, extra: &str) -> Home {
|
||||
let dir = TempDir::new(tag);
|
||||
let soul = dir.write("etc/soul.md", "You are Boxmaker.\n");
|
||||
let key = dir.write("keys/honcho.key", &format!("{}\n", file_key()));
|
||||
std::fs::set_permissions(&key, std::fs::Permissions::from_mode(0o600)).unwrap();
|
||||
let text = format!(
|
||||
"[paths]\nhome = \"{}\"\n[memory]\nsoul_starter = \"{}\"\n{extra}",
|
||||
dir.path().display(),
|
||||
soul.display()
|
||||
);
|
||||
let text = text.replace("KEY_FILE", &key.display().to_string());
|
||||
let config = dir.write("memoryd.toml", &text);
|
||||
Home { dir, config }
|
||||
}
|
||||
|
||||
fn path(&self, relative: &str) -> PathBuf {
|
||||
self.dir.path().join(relative)
|
||||
}
|
||||
|
||||
fn command(&self) -> Command {
|
||||
let mut command = Command::new(env!("CARGO_BIN_EXE_memoryd"));
|
||||
command
|
||||
.args(["serve", "--config"])
|
||||
.arg(&self.config)
|
||||
.env("HONCHO_TEST_KEY", env_key())
|
||||
.stdin(Stdio::null())
|
||||
.stdout(Stdio::null());
|
||||
command
|
||||
}
|
||||
|
||||
/// Runs `memoryd serve` to its end, at most 10 s (then it is killed), and gives its exit code
|
||||
/// (`None` if it had to be killed) and its stderr.
|
||||
fn run_to_end(&self) -> (Option<i32>, String) {
|
||||
let mut child = Daemon(self.command().stderr(Stdio::piped()).spawn().unwrap());
|
||||
let deadline = Instant::now() + Duration::from_secs(10);
|
||||
let mut code = None;
|
||||
while Instant::now() < deadline {
|
||||
if let Ok(Some(status)) = child.0.try_wait() {
|
||||
code = status.code();
|
||||
break;
|
||||
}
|
||||
std::thread::sleep(Duration::from_millis(20));
|
||||
}
|
||||
let _ = child.0.kill();
|
||||
let _ = child.0.wait();
|
||||
let mut stderr = String::new();
|
||||
if let Some(mut pipe) = child.0.stderr.take() {
|
||||
let _ = std::io::Read::read_to_string(&mut pipe, &mut stderr);
|
||||
}
|
||||
(code, stderr)
|
||||
}
|
||||
|
||||
/// `memoryd serve` with its stderr in `<home>/memoryd.err`.
|
||||
fn start(&self) -> Daemon {
|
||||
let err = std::fs::File::create(self.path("memoryd.err")).unwrap();
|
||||
Daemon(self.command().stderr(Stdio::from(err)).spawn().unwrap())
|
||||
}
|
||||
|
||||
fn stderr(&self) -> String {
|
||||
std::fs::read_to_string(self.path("memoryd.err")).unwrap_or_default()
|
||||
}
|
||||
}
|
||||
|
||||
/// A running `memoryd serve`, killed and waited for when dropped. Each test declares it after its
|
||||
/// `Home`, so it drops first.
|
||||
struct Daemon(Child);
|
||||
|
||||
impl Drop for Daemon {
|
||||
fn drop(&mut self) {
|
||||
let _ = self.0.kill();
|
||||
let _ = self.0.wait();
|
||||
}
|
||||
}
|
||||
|
||||
/// True once `memoryd` answers `memory_block` on `loopd`'s socket, within 10 s.
|
||||
fn answers(home: &Home) -> bool {
|
||||
let socket = home.path("run/loop-memory/memory.sock");
|
||||
let deadline = Instant::now() + Duration::from_secs(10);
|
||||
while Instant::now() < deadline {
|
||||
if let Ok(mut stream) = UnixStream::connect(&socket) {
|
||||
let limit = Some(Duration::from_secs(5));
|
||||
let ask = Envelope {
|
||||
v: PROTOCOL_VERSION,
|
||||
id: 1,
|
||||
r#final: true,
|
||||
msg: Message::MemoryBlock(Empty {}),
|
||||
};
|
||||
if stream.set_read_timeout(limit).is_ok()
|
||||
&& write_frame(&mut stream, &ask).is_ok()
|
||||
&& matches!(
|
||||
read_frame(&mut stream),
|
||||
Ok(Envelope {
|
||||
msg: Message::MemoryBlockReply(_),
|
||||
..
|
||||
})
|
||||
)
|
||||
{
|
||||
return true;
|
||||
}
|
||||
}
|
||||
std::thread::sleep(Duration::from_millis(20));
|
||||
}
|
||||
false
|
||||
}
|
||||
|
||||
fn assert_no_key(text: &str) {
|
||||
assert!(
|
||||
!text.contains(&env_key()),
|
||||
"the env key was printed: {text}"
|
||||
);
|
||||
assert!(
|
||||
!text.contains(&file_key()),
|
||||
"the file key was printed: {text}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_invalid_section_stops_memoryd_with_its_reason_and_the_pointer() {
|
||||
let extra = format!(
|
||||
"{}[secrets.honcho_trusted]\nenv = \"HONCHO_TEST_KEY\"\n[secrets.honcho_fleet]\nfile = \"KEY_FILE\"\n",
|
||||
SECTION.replace("owner = \"kyle\"", "owner = \"boxmaker0\"")
|
||||
);
|
||||
let home = Home::new("honcho-bad", &extra);
|
||||
let (code, stderr) = home.run_to_end();
|
||||
assert_eq!(code, Some(1), "{stderr}");
|
||||
assert_eq!(
|
||||
stderr,
|
||||
format!(
|
||||
"memoryd: {}: [honcho] agent and owner are both \"boxmaker0\": the agent's peer must not be the owner's\n{POINTER}\n",
|
||||
home.config.display()
|
||||
)
|
||||
);
|
||||
assert_no_key(&stderr);
|
||||
assert!(
|
||||
!home.path("run").exists(),
|
||||
"memoryd stopped before its lock and sockets"
|
||||
);
|
||||
assert!(!home.path("memory").exists(), "and before its memory");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_key_for_an_agent_not_read_stops_memoryd() {
|
||||
let extra = format!("{SECTION}[secrets.honcho_read_boxmaker2]\nenv = \"HONCHO_TEST_KEY\"\n");
|
||||
let home = Home::new("honcho-key", &extra);
|
||||
let (code, stderr) = home.run_to_end();
|
||||
assert_eq!(code, Some(1), "{stderr}");
|
||||
assert_eq!(
|
||||
stderr,
|
||||
format!(
|
||||
"memoryd: {}: [secrets.honcho_read_boxmaker2] is the key of agent \"boxmaker2\", which is not in [[honcho.read]]\n{POINTER}\n",
|
||||
home.config.display()
|
||||
)
|
||||
);
|
||||
assert_no_key(&stderr);
|
||||
}
|
||||
|
||||
/// The readers are built at start (M5c-2): a read workspace's key that cannot be read stops
|
||||
/// `memoryd`, as a written workspace's does.
|
||||
#[test]
|
||||
fn a_read_key_that_cannot_be_read_stops_memoryd() {
|
||||
let extra = format!("{SECTION}[secrets.honcho_read_boxmaker1]\nenv = \"HONCHO_NOT_SET\"\n");
|
||||
let home = Home::new("honcho-read-key", &extra);
|
||||
let (code, stderr) = home.run_to_end();
|
||||
assert_eq!(code, Some(1), "{stderr}");
|
||||
assert!(stderr.contains("honcho_read_boxmaker1"), "{stderr}");
|
||||
assert!(
|
||||
stderr.contains("see docs/runbook.md#secret-unavailable"),
|
||||
"{stderr}"
|
||||
);
|
||||
assert_no_key(&stderr);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn an_unknown_key_in_the_section_is_still_a_start_failure() {
|
||||
let extra = SECTION.replace("fleet = \"fleet\"", "fleet = \"fleet\"\nfleat = \"x\"");
|
||||
let home = Home::new("honcho-unknown", &extra);
|
||||
let (code, stderr) = home.run_to_end();
|
||||
assert_eq!(code, Some(1), "{stderr}");
|
||||
assert!(
|
||||
stderr.starts_with(&format!("memoryd: {}: ", home.config.display())),
|
||||
"{stderr}"
|
||||
);
|
||||
assert!(stderr.contains("fleat"), "{stderr}");
|
||||
assert!(stderr.ends_with(&format!("\n{START_FAILED}\n")), "{stderr}");
|
||||
assert!(!stderr.contains(POINTER), "{stderr}");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_valid_section_with_keys_starts_memoryd_as_before() {
|
||||
let extra = format!(
|
||||
"{SECTION}[secrets.honcho_trusted]\nenv = \"HONCHO_TEST_KEY\"\n[secrets.honcho_fleet]\nfile = \"KEY_FILE\"\n[secrets.honcho_read_boxmaker1]\nenv = \"HONCHO_TEST_KEY\"\n"
|
||||
);
|
||||
let home = Home::new("honcho-good", &extra);
|
||||
let _daemon = home.start();
|
||||
assert!(
|
||||
answers(&home),
|
||||
"memoryd never answered; stderr: {}",
|
||||
home.stderr()
|
||||
);
|
||||
let stderr = home.stderr();
|
||||
assert!(stderr.contains("memoryd: serving\n"), "{stderr}");
|
||||
assert_no_key(&stderr);
|
||||
// The writers and the readers both load the fleet's key file; its warning is logged once.
|
||||
assert_eq!(
|
||||
stderr
|
||||
.matches("secret honcho_fleet is read in plaintext")
|
||||
.count(),
|
||||
1,
|
||||
"{stderr}"
|
||||
);
|
||||
assert!(!stderr.contains(POINTER), "{stderr}");
|
||||
}
|
||||
Reference in New Issue
Block a user