boxmaker/docs/plans/M5c2/05-memoryd-honcho-readers.md

M5c-2 task 05: memoryd's readers, the Honcho workspaces it reads

Branch: m5c2. Check it: git branch --show-current prints m5c2 and git status --short is empty; otherwise stop. Never switch branch. Commit subject: memoryd: the Honcho workspaces it reads, with their clients and labels Log row: the Task cell is M5c2/05-memoryd-honcho-readers.

Goal

memoryd writes to three Honcho workspaces through the Clients of M5c-1 (the agent's trusted and untrusted ones and the fleet's), which move into the sync worker's thread. M5c-2 also reads (M5c spec, sections 3, 8 and 9): recall searches other agents' workspaces and the fleet's, and ask_memory asks one or two workspaces by name. Reading needs clients of its own. This task builds them at start and gives later tasks the types they use; nothing searches or asks yet (tasks 08 to 10).

• Readers (new module crates/memoryd/src/honcho/read.rs, given whole): a Reader for
  • each [[honcho.read]] workspace, in the order written, with that entry's trust and class and the key honcho_read_<agent>;
  • the fleet's workspace (key honcho_fleet), always untrusted and private;
  • the agent's own trusted workspace (key honcho_trusted), trusted, private, and its own untrusted one (key honcho_untrusted), untrusted, private. These two are for ask_memory only: recall never searches the agent's own workspaces (spec, section 8).
• Two clients per reader. A roncho client has one read timeout. Searches wait 5 s (SEARCH_READ) and chat 150 s (CHAT_READ, which this task changes from 180 s: spec 9 gives chat 150 s, inside brokerd's 170 s); both connect within 5 s (CONNECT).
• The label of each reader's results: the name each hit shows (the agent's name, fleet, own or untrusted), the trust, and the class. Label::tag() is memory for a trusted workspace and memory-tainted otherwise, as a local recall hit's tag.
• recall_sources(): the read workspaces, then the fleet's. resolve(name): the reader ask_memory means by own, untrusted, fleet or a read agent's name; any other name is the sentence the model is shown, which lists the names it may use.
• Reserved names. Since ask_memory names this agent's own workspaces own, untrusted and fleet, a [[honcho.read]] agent with one of those names is now refused at start (honcho-config-invalid). Otherwise a read agent called fleet would shadow the fleet in ask_memory.
• At start, start::open builds the readers beside the writers' Clients and keeps them in Opened::readers. A read key that cannot be read stops memoryd, as a written workspace's does. The three written workspaces' keys are read by both; each key file's warning is logged once. serve.rs does not change: nothing uses the readers until task 09, which takes opened.readers before the worker's thread takes the rest of Opened.

Every place that changes

This is the complete list.

Place	What happens there
crates/memoryd/src/honcho/read.rs	new, given whole: copy it
crates/memoryd/src/honcho/mod.rs	pub mod read; (edit H1)
crates/memoryd/src/honcho/client.rs	CHAT_READ is 150 s; client and key become pub(crate), for read.rs (edits C1 to C3)
crates/memoryd/src/honcho/config.rs	RESERVED, and the check that uses it (edits G1, G2)
crates/memoryd/src/honcho/start.rs	Opened::readers, built in open (edits O1 to O3)
docs/runbook.md, honcho-config-invalid	the reserved names among the reasons (edit D1)
crates/memoryd/tests/honcho_read.rs	new, given
crates/memoryd/tests/honcho_client.rs, honcho_start.rs	new versions, given
everything else	no change: not serve.rs, not recall.rs, not worker.rs, no Cargo.toml

Files

• Copy, from docs/plans/M5c2/files/ (the same path under it): the new test crates/memoryd/tests/honcho_read.rs; new versions of (they replace the files in the tree) crates/memoryd/tests/honcho_client.rs and crates/memoryd/tests/honcho_start.rs; and the new source crates/memoryd/src/honcho/read.rs, which is complete: it has no todo!(), and you do not change it.
• Modify: crates/memoryd/src/honcho/mod.rs, crates/memoryd/src/honcho/client.rs, crates/memoryd/src/honcho/config.rs, crates/memoryd/src/honcho/start.rs, docs/runbook.md, docs/implementer-log.md.
• Do not touch: any other file.

Facts you need

• read.rs needs from client.rs: CONNECT, CHAT_READ, Timeouts, and the two helpers client(url, workspace, key, timeouts) -> Result<Client, String> and key(config, name, env, warnings) -> Result<Option<secrets::Secret>, String>, which are private today. Edits C2 and C3 make them pub(crate): visible in memoryd, not outside it. From config.rs it needs FLEET_KEY, TRUSTED_KEY, UNTRUSTED_KEY, read_key, Trust and HonchoConfig, all public already, and from bxcore DataClass and bxcore::memory::OWN_WORKSPACE ("own", task 01).

• What Readers offers (for tasks 08 to 10; read read.rs for the whole):

  pub const SEARCH_READ: Duration; // 5 s
  pub const UNTRUSTED: &str;       // "untrusted"
  pub const FLEET: &str;           // "fleet"
  pub struct ReadTimeouts { pub connect: Duration, pub search: Duration, pub chat: Duration }
  pub enum Source { Agent(String), Fleet, Own, OwnUntrusted }
  pub struct Label { pub name: String, pub trust: Trust, pub class: DataClass }
  impl Label { pub fn tag(&self) -> &'static str; pub fn untrusted(&self) -> bool }
  pub struct Reader { pub source: Source, pub label: Label, /* two clients */ }
  impl Reader { pub fn search(&self) -> &Client; pub fn chat(&self) -> &Client }
  impl Readers {
      pub fn new(config: &HonchoConfig, env: &dyn Fn(&str) -> Option<OsString>,
                 timeouts: ReadTimeouts) -> Result<(Readers, Vec<String>), String>;
      pub fn recall_sources(&self) -> Vec<&Reader>;
      pub fn resolve(&self, name: &str) -> Result<&Reader, String>;
  }
  

• roncho::blocking::Client is Clone, Send and Sync (its documentation says so, and a given test checks it at compile time), and its Debug shows the timeouts and whether a key is present, never the key. This task's table of roncho's API is complete; roncho's source (the Cargo registry, ~/.cargo/registry) is outside the repository and off limits.

• Why the warnings are filtered. Clients::new and Readers::new each read the keys of the three written workspaces, and each returns a key file's plaintext warning. Edit O3 logs the writers' warnings, then only those of the readers' that are not among them.

• The given tests.

  • honcho_read.rs (8 tests), against the fake Honcho: the client and the readers are Send + Sync; the timeouts, read from each client's Debug; each reader searches its own workspace with its own key (the fake records each request's Authorization header), and no header without a key; no key in a reader's Debug, an error, or the error of a key that cannot be read; recall_sources never holds the agent's own workspaces; each source's label and tag; resolve for every name, and the exact sentence for an unknown one; a read agent named own, untrusted or fleet refused by config::check, with its sentence.
  • honcho_client.rs: one line changes, CHAT_READ is 150 s.
  • honcho_start.rs: a new test, a_read_key_that_cannot_be_read_stops_memoryd, and one new assertion in a_valid_section_with_keys_starts_memoryd_as_before: the fleet key file's warning is printed once.

• The items the edits use. This table is complete.

  Item	Where
  Readers::new, ReadTimeouts::DEFAULT (from read.rs)	O3
  StartError::Secret(String)	O3, as Clients::new's error beside it
  Vec::contains, Iterator::filter, Iterator::chain	O3
  Arc::new	O3
  [&str; 3]::contains(&&str), String::as_str	G2: RESERVED.contains(&entry.agent.as_str())

The edits, exactly

Nothing else in these files changes. Each quoted "before" block occurs exactly once in its file when its turn comes (make them in order). Keep the fields in the order shown.

crates/memoryd/src/honcho/mod.rs

H1. Add one line. The line

pub mod naming;

becomes

pub mod naming;
pub mod read;

crates/memoryd/src/honcho/client.rs

C1. The lines

/// A chat's read: a non-streamed chat sends nothing until it is done (M5c-2's `ask_memory`).
pub const CHAT_READ: Duration = Duration::from_secs(180);

become

/// A chat's read: a non-streamed chat sends nothing until it is done (M5c spec, section 9).
pub const CHAT_READ: Duration = Duration::from_secs(150);

C2. The lines

/// One workspace's client: no `roncho.toml` read, the timeouts given, the key when there is one.
fn client(

become

/// One workspace's client: no `roncho.toml` read, the timeouts given, the key when there is one.
pub(crate) fn client(

C3. The lines

/// The key named `name` in `[secrets]`, if one is configured, read now.
fn key(

become

/// The key named `name` in `[secrets]`, if one is configured, read now.
pub(crate) fn key(

crates/memoryd/src/honcho/config.rs

G1. The lines

/// The names of the keys in `[secrets.<name>]`, one per workspace. A read workspace's key is

become

/// The names `ask_memory` gives this agent's own workspaces and the fleet's (M5c spec, section 9),
/// which no `[[honcho.read]]` agent may have.
pub const RESERVED: [&str; 3] = ["own", "untrusted", "fleet"];

/// The names of the keys in `[secrets.<name>]`, one per workspace. A read workspace's key is

G2. In read_workspace, the lines (its spaces are checked: the sentence is one string split across two lines with a \ at the end of the first, and the given test compares it letter for letter)

            "[[honcho.read]] agent {:?} is this agent ([honcho] agent)",
            entry.agent
        ));
    }

become

            "[[honcho.read]] agent {:?} is this agent ([honcho] agent)",
            entry.agent
        ));
    }
    if RESERVED.contains(&entry.agent.as_str()) {
        return Err(format!(
            "[[honcho.read]] agent {:?} is a name ask_memory gives this agent's own workspaces: \
             own, untrusted and fleet cannot be agents' names",
            entry.agent
        ));
    }

crates/memoryd/src/honcho/start.rs

O1. Add one line. The line

use crate::honcho::mirror::Mirror;

becomes

use crate::honcho::mirror::Mirror;
use crate::honcho::read::{ReadTimeouts, Readers};

O2. The lines

pub struct Opened {
    pub sync: Arc<Mutex<TurnSync>>,
    clients: Clients,

become

pub struct Opened {
    pub sync: Arc<Mutex<TurnSync>>,
    /// The workspaces `recall` and `ask_memory` read (M5c spec, sections 8 and 9).
    pub readers: Arc<Readers>,
    clients: Clients,

O3. In open, the lines

    for warning in warnings {
        (log)(&warning);
    }
    Ok(Opened {
        sync: Arc::new(Mutex::new(sync)),
        clients,

become

    let (readers, read_warnings) =
        Readers::new(honcho, &env, ReadTimeouts::DEFAULT).map_err(StartError::Secret)?;
    // The readers load the keys of the three written workspaces again: each warning once.
    let read_warnings: Vec<String> = read_warnings
        .into_iter()
        .filter(|w| !warnings.contains(w))
        .collect();
    for warning in warnings.into_iter().chain(read_warnings) {
        (log)(&warning);
    }
    Ok(Opened {
        sync: Arc::new(Mutex::new(sync)),
        readers: Arc::new(readers),
        clients,

docs/runbook.md

D1. In the entry honcho-config-invalid, the lines (its spaces are checked: the entry is read by people, and scripts/check-runbook.sh reads the file)

- A `[[honcho.read]]` names this agent, names an agent twice, or names one of this agent's three
  workspaces; or its `trust` is not `trusted` or `untrusted`, or its `class` not `public`,
  `private` or `secret` (all lower case).

become

- A `[[honcho.read]]` names this agent, names an agent twice, names an agent `own`, `untrusted`
  or `fleet` (the names `ask_memory` gives this agent's own workspaces), or names one of this
  agent's three workspaces; or its `trust` is not `trusted` or `untrusted`, or its `class` not
  `public`, `private` or `secret` (all lower case).

Steps

• 1. Copy the given tests. From the repository root, three commands:

  cp docs/plans/M5c2/files/crates/memoryd/tests/honcho_read.rs crates/memoryd/tests/
  cp docs/plans/M5c2/files/crates/memoryd/tests/honcho_client.rs crates/memoryd/tests/
  cp docs/plans/M5c2/files/crates/memoryd/tests/honcho_start.rs crates/memoryd/tests/
  

  Then git status --short lists honcho_client.rs and honcho_start.rs as modified and honcho_read.rs as new, all under crates/memoryd/tests/.

• 2. See them fail. Two commands, each with a shell timeout of 600000 ms:

  • cargo build --tests --keep-going -p memoryd: one test target, honcho_read, does not compile: E0432 for unresolved import `memoryd::honcho::read` and for unresolved import `memoryd::honcho::config::RESERVED` . Every other target compiles.
  • cargo test --no-fail-fast -p memoryd --test honcho_client --test honcho_start: two tests fail: the_timeouts in honcho_client (CHAT_READ is 180 s) and a_read_key_that_cannot_be_read_stops_memoryd in honcho_start (memoryd does not read that key yet, so it starts, and the test stops it after 10 s).

• 3. The new module and the edits. Copy the module:

  cp docs/plans/M5c2/files/crates/memoryd/src/honcho/read.rs crates/memoryd/src/honcho/
  

  Then make edits H1, C1 to C3, G1, G2, O1 to O3 and D1, in order. cargo check -p memoryd prints no error and no warning.

• 4. The tests. With a shell timeout of 600000 ms, every test passes: 13 in honcho_client, 8 in honcho_config, 3 in honcho_keys (and 1 ignored, as before), 8 in honcho_read, 5 in honcho_start:

  cargo test -p memoryd --test honcho_read --test honcho_client --test honcho_start \
    --test honcho_config --test honcho_keys
  

• 5. Check. Two commands:

  • git diff --stat -- crates/memoryd/src lists exactly client.rs, config.rs, mod.rs and start.rs under crates/memoryd/src/honcho/ (the new read.rs is untracked, so git diff does not list it), and git status --short crates/memoryd/src shows read.rs as new.
  • git diff --stat -- crates/memoryd/src/serve.rs crates/memoryd/src/recall.rs Cargo.toml Cargo.lock prints nothing.

• 6. Run the gate. cargo fmt --all, then make gate, with a shell timeout of 600000 ms. Expected last line: gate: ok. The test count is the passed total of cargo test --workspace plus one, for the ignored end_to_end test that the gate runs on its own: 1403 after this task, 1394 before it; the 9 new ones are the 8 tests of honcho_read.rs and the new test in honcho_start.rs. To count, add up the N passed of every test result: line the gate's cargo test --workspace printed, and add 1. Never run the gate again only to count.

• 7. Log and commit. Add your row to docs/implementer-log.md; its Task cell is M5c2/05-memoryd-honcho-readers. git status --short lists the four edited sources, the two replaced tests, docs/runbook.md and your log as modified, and read.rs and honcho_read.rs as new. Stage with git add crates/memoryd docs/runbook.md docs/implementer-log.md Cargo.lock, then commit as AGENTS.md says. Cargo.lock does not change in this task; the git add is harmless.

Done when

• make gate prints gate: ok with 1403 tests.
• git diff HEAD~1 --stat lists only the four edited sources, read.rs, the three test files of step 1, docs/runbook.md and docs/implementer-log.md.

Stop and report if

• The branch check at the top fails.
• A "before" block is not in its file exactly as shown when its turn comes, or is there more than once.
• Step 2 shows a different result: another target failing, other errors, other tests failing.
• cargo check in step 3 prints an error or a warning. Quote it, and stop. Do not change read.rs to make it compile.
• A test fails in step 4. Do not change a test: stop, and give the failing tests' names and their messages.
• The fix seems to need a change anywhere else: serve.rs, recall.rs, worker.rs, a change to read.rs, a new dependency. It does not; stop and say why you think so.
• A test outside the given files fails in the gate: check once whether it fails without your change (git stash -u, run it, git stash pop), then stop and report both results.
• The gate passes with a count other than 1403.
• A comment in an edit does not fit the code under it. Quote it and stop.

What the given tests pin

Each of these changes to the finished code makes at least one given test fail (24 tried, all caught): the search clients given chat's timeout, or chat's the search one; either timeout constant changed; a read workspace given the fleet's key, the fleet the trusted key, or the own untrusted workspace the trusted key; the own reader on the untrusted workspace; a key not sent; a read workspace's trust or class ignored; the fleet trusted, or the own workspace untrusted; the tags swapped, or untrusted() inverted; recall_sources holding the own workspace, or not the fleet's; untrusted resolved to the trusted workspace; read agents not resolved, or resolved by workspace instead of name; the sentence without the read agents' names; the reserved names allowed; the readers' errors ignored at start; the key file's warning logged twice.