boxmaker/docs/plans/M5c2/08-memoryd-honcho-search.md

M5c-2 task 08: recall's Honcho part, every source at once with its own limit

Branch: m5c2. Check it: git branch --show-current prints m5c2 and git status --short is empty; otherwise stop. Never switch branch. Commit subject: memoryd: recall's Honcho part, every source at once with its own limit Log row: the Task cell is M5c2/08-memoryd-honcho-search.

Goal

With [honcho], recall searches other agents' Honcho workspaces and the fleet's beside the local index (M5c spec, section 8). Tasks 05 to 07 gave the readers, the items and the local side. This task writes the Honcho part itself, as a new module crates/memoryd/src/honcho/search.rs, and the answer made from both parts. Task 09 wires it into memoryd's recall, outside the state lock; until then nothing calls it.

• search(readers, owner, args, wait, log) -> Searched:
  • A query holding a secret asks nothing. If decide::has_secret(&args.query) (M5c-1's known-prefix secret test, with its masked and deleted forms), no source is asked: Searched::Secret.
  • Every source at once. Each source of readers.recall_sources() (the [[honcho.read]] workspaces in the order written, then the fleet's; never the agent's own) is asked on a thread of its own.
  • A source's calls, one after another on its thread: search_workspace with limit = k; then, for another agent's workspace only, query_conclusions with top_k = k for that agent's peer, then for the owner's peer (observer and observed the same peer, as memoryd writes conclusions). The fleet's is the one search. A source has answered only when all its calls have; its kept items are task 06's items.
  • Each source has wait (5 s in memoryd, SOURCE_WAIT; the tests give less), counted from when the Honcho part starts, not added up. A source that failed, or has not answered by then, is unavailable. A late thread is left to finish on its own, holding no lock; what it finds is dropped.
  • Logging: each unavailable source is one line, memoryd: recall: {name} unavailable: {error}, where the error is HonchoError's kind and status (Honcho is unreachable (server, HTTP 503)) or no answer in {wait} ms. Never the query, never any text. A Honcho that is down gives one line per source per recall, and the model calls recall a few times a turn at most.
  • The answer, Searched::Asked { found, unavailable }: each source's kept items in recall_sources() order (empty for one that did not answer), and the unavailable sources' names, in the same order.
• answer(local, query, k, searched) -> Recalled: the local items (task 07's recall::Local), then Honcho's, interleaved up to k in all and numbered after the local ones (task 06's interleave and render); (no matches for "…") when there is neither; then the unavailable lines, or, for Secret, the line honcho: not asked; the query looks like it holds a secret. The label is the stricter of the local part's and the shown Honcho items' (task 06's label): the higher class, and untrusted when either is.
• The fake Honcho of the tests (crates/memoryd/tests/support/honcho.rs, a given file) gets a new version: it answers a workspace search from fixture 30 and a conclusion query from fixture 32 (about kyle) or 33 (about anyone else), can answer as a workspace that does not exist (Mode::Missing: fixtures 34, 35 and 36), and can take a mode for one workspace (set_workspace). Everything it did before is unchanged.

Every place that changes

This is the complete list.

Place	What happens there
crates/memoryd/src/honcho/search.rs	new: a skeleton you copy, then fill 2 bodies
crates/memoryd/src/honcho/mod.rs	pub mod search; (edit H1)
crates/memoryd/tests/honcho_search.rs	new, given
crates/memoryd/tests/support/honcho.rs	new version, given
everything else	no change: not serve.rs (task 09), not recall.rs, no Cargo.toml

Files

• Copy, from docs/plans/M5c2/files/ (the same path under it): the new test crates/memoryd/tests/honcho_search.rs; a new version of crates/memoryd/tests/support/honcho.rs (it replaces the file in the tree); and the skeleton crates/memoryd/src/honcho/search.rs.
• Modify: crates/memoryd/src/honcho/mod.rs, docs/implementer-log.md.
• Do not touch: any other file, and in the skeleton anything but its two todo!() bodies.

Facts you need

• The skeleton is written except two bodies: source (one source's calls) and answer. conclusions (one conclusion query), search (the threads, the limit, the log) and the types are written; read them before you fill the bodies. search calls source on each thread.

• roncho's calls and types (this table is complete; roncho's source, in the Cargo registry, is outside the repository and off limits):

  Item	What it does
  Client::search_workspace(&self, &MessageSearch) -> Result<Vec<Message>, roncho::blocking::Error>	POST /v3/workspaces/{w}/search
  MessageSearch { query: String, filters, limit: Option<u32>, scope }, MessageSearch::new(query)	all fields public: MessageSearch { limit: Some(k), ..MessageSearch::new(query) } sets the limit
  Client::query_conclusions, ConclusionQuery	used in conclusions, written
  roncho::Message, roncho::Conclusion	what items (task 06) takes

• From earlier tasks: classify(&roncho::blocking::Error) -> HonchoError (M5c-1, in client.rs); Reader::search() -> &Client, Reader::source, Reader::label (task 05); items, interleave, render, label, unavailable (task 06, recalled.rs); Local { items, class, untrusted }, Recalled { text, class, untrusted } (task 07, recall.rs).

• A name clash in answer: Searched::Asked's field unavailable has the name of task 06's function unavailable. Bind the field under another name (unavailable: missing), as the comment says, or the function is hidden inside that arm.

• The given test honcho_search.rs (12 tests), against the fake Honcho:

  • the limit and the secret line;
  • every source searched with its calls, their paths and bodies (limit, top_k, the peers);
  • the agent's own workspaces never searched;
  • each source's hits in the answer with their tags, numbered after a local item;
  • a fleet hit making the answer untrusted, and a trusted read workspace's hit not (T58);
  • a hanging source: one unavailable line after the limit (1.5 s), while the others answer;
  • every source hanging, answered after one limit, not one per source;
  • a failing source (503, 422): unavailable, logged by kind and status only;
  • a missing workspace: no hits, not unavailable;
  • a query holding a secret: no request at all, and the secret line;
  • nothing found: (no matches for "zebra"), then any unavailable line;
  • the local hits filling k, and the two labels joined.

  A key in it is built from pieces; the waits have limits (the whole file takes about 3 s).

• The calls the bodies use: MessageSearch::new, Client::search_workspace, Result::map_err, classify, conclusions (written), items; usize::try_from, interleave, unavailable, label, render, DataClass::max, slice::join.

The edits, exactly

crates/memoryd/src/honcho/mod.rs

H1. Add one line. The line

pub mod report;

becomes

pub mod report;
pub mod search;

Steps

• 1. Copy the given tests. From the repository root, two commands:

  cp docs/plans/M5c2/files/crates/memoryd/tests/honcho_search.rs crates/memoryd/tests/
  cp docs/plans/M5c2/files/crates/memoryd/tests/support/honcho.rs crates/memoryd/tests/support/
  

  Then git status --short lists crates/memoryd/tests/support/honcho.rs as modified and honcho_search.rs as new.

• 2. See them fail. cargo build --tests --keep-going -p memoryd, with a shell timeout of 600000 ms: one test target, honcho_search, does not compile: E0432, unresolved import `memoryd::honcho::search` . Every other target compiles.

• 3. The skeleton and the edit. Copy the skeleton, then make edit H1:

  cp docs/plans/M5c2/files/crates/memoryd/src/honcho/search.rs crates/memoryd/src/honcho/
  

  cargo check -p memoryd: no error, and 12 warnings, all from the unfilled bodies: unused imports (MessageSearch, Source, and interleave, items, label, render, unavailable), unused variables (reader, owner, query, k, local, searched), and the function conclusions, never used.

• 4. Fill the bodies, one at a time, from their comments, with cargo check -p memoryd after each: source, then answer. Delete each instruction comment and keep the doc comment above the function. After the last, cargo check -p memoryd prints no warning.

• 5. The tests. With a shell timeout of 600000 ms, every test passes: 13 in honcho_client, 3 in honcho_keys (and 1 ignored, as before), 8 in honcho_read, 12 in honcho_search, 12 in honcho_worker:

  cargo test -p memoryd --test honcho_search --test honcho_read --test honcho_client \
    --test honcho_keys --test honcho_worker
  

• 6. Check. git diff --stat lists exactly crates/memoryd/src/honcho/mod.rs and crates/memoryd/tests/support/honcho.rs, and git status --short shows search.rs and honcho_search.rs as new.

• 7. Run the gate. cargo fmt --all, then make gate, with a shell timeout of 600000 ms. Expected last line: gate: ok. The test count is the passed total of cargo test --workspace plus one, for the ignored end_to_end test that the gate runs on its own: 1430 after this task, 1418 before it; the 12 new ones are the tests of honcho_search.rs. To count, add up the N passed of every test result: line the gate's cargo test --workspace printed, and add 1. Never run the gate again only to count.

• 8. Log and commit. Add your row to docs/implementer-log.md; its Task cell is M5c2/08-memoryd-honcho-search. Stage with git add crates/memoryd docs/implementer-log.md Cargo.lock, then commit as AGENTS.md says. Cargo.lock does not change in this task; the git add is harmless.

Done when

• make gate prints gate: ok with 1430 tests.
• git diff HEAD~1 --stat lists only mod.rs, search.rs, honcho_search.rs, support/honcho.rs and docs/implementer-log.md.

Stop and report if

• The branch check at the top fails.
• The "before" line of H1 is not in the file exactly once.
• Step 2 or 3 shows a different result: other targets failing, other errors, or warnings about anything but the unfilled bodies.
• cargo check reports an error after a body, and its comment does not tell you how to fix it. Quote the error, and stop.
• After step 4 a test fails although each body follows its comment. Do not change a test, the fake or the written parts of the skeleton: stop, and give the failing tests' names and their messages.
• The fix seems to need a change anywhere else: serve.rs, recall.rs, read.rs, recalled.rs, a new dependency. It does not; stop and say why you think so.
• A test outside the given files fails in the gate: check once whether it fails without your change (git stash -u, run it, git stash pop), then stop and report both results.
• The gate passes with a count other than 1430.
• A comment in the skeleton does not fit its signature. Quote it and stop.

What the given tests pin

Each of these changes to the finished code makes at least one given test fail (19 tried, all caught): the secret check skipped; the search without its limit, or a conclusion query without top_k or with another observer; the conclusions about the agent's peer or the owner's left out, or asked of the fleet; a failed or a late source not reported unavailable; no limit on the wait; the sources asked one after another; the local items left out, or Honcho's numbered from 1 or interleaved as if there were none; no (no matches …) line; the unavailable lines left out; either part's label ignored; the secret line's words.