boxmaker/docs/plans/M5c2/08-memoryd-honcho-search.md

# M5c-2 task 08: `recall`'s Honcho part, every source at once with its own limit

**Branch:** `m5c2`. Check it: `git branch --show-current` prints `m5c2` and `git status --short`
is empty; otherwise stop. Never switch branch.
**Commit subject:** `memoryd: recall's Honcho part, every source at once with its own limit`
**Log row:** the Task cell is `M5c2/08-memoryd-honcho-search`.

## Goal

With `[honcho]`, `recall` searches other agents' Honcho workspaces and the fleet's beside the
local index (M5c spec, section 8). Tasks 05 to 07 gave the readers, the items and the local
side. This task writes the Honcho part itself, as a new module
`crates/memoryd/src/honcho/search.rs`, and the answer made from both parts. Task 09 wires it into
`memoryd`'s `recall`, outside the state lock; until then nothing calls it.

- **`search(readers, owner, args, wait, log) -> Searched`**:
  - **A query holding a secret asks nothing.** If `decide::has_secret(&args.query)` (M5c-1's
    known-prefix secret test, with its masked and deleted forms), no source is asked:
    `Searched::Secret`.
  - **Every source at once.** Each source of `readers.recall_sources()` (the `[[honcho.read]]`
    workspaces in the order written, then the fleet's; never the agent's own) is asked on a thread
    of its own.
  - **A source's calls**, one after another on its thread: `search_workspace` with `limit = k`;
    then, for another agent's workspace only, `query_conclusions` with `top_k = k` for that
    agent's peer, then for the owner's peer (observer and observed the same peer, as `memoryd`
    writes conclusions). The fleet's is the one search. A source has answered only when all its
    calls have; its kept items are task 06's `items`. A read agent's three calls share the
    source's one limit, so a slow rift makes the whole source unavailable, not only its last
    call.
  - **Each source has `wait`** (5 s in `memoryd`, `SOURCE_WAIT`; the tests give less), counted
    from when the Honcho part starts, not added up. A source that failed, or has not answered by
    then, is unavailable. A late thread is left to finish on its own, holding no lock; what it
    finds is dropped.
  - **Logging**: each unavailable source is one line, `memoryd: recall: {name} unavailable:
    {error}`, where the error is `HonchoError`'s kind and status (`Honcho is unreachable (server,
    HTTP 503)`) or `no answer in {wait} ms`. Never the query, never any text. A Honcho that is
    down gives one line per source per `recall`, and the model calls `recall` a few times a turn
    at most.
  - The answer, `Searched::Asked { found, unavailable }`: each source's kept items in
    `recall_sources()` order (empty for one that did not answer), and the unavailable sources'
    names, in the same order.
- **`answer(local, query, k, searched) -> Recalled`**: the local items (task 07's
  `recall::Local`), then Honcho's, interleaved up to `k` in all and numbered after the local
  ones (task 06's `interleave` and `render`); `(no matches for "…")` when there is neither; then
  the `unavailable` lines, or, for `Secret`, the line `honcho: not asked; the query looks like it
  holds a secret`. The label is the stricter of the local part's and the shown Honcho items'
  (task 06's `label`): the higher class, and untrusted when either is.
- **The fake Honcho** of the tests (`crates/memoryd/tests/support/honcho.rs`, a given file) gets a
  new version: it answers a workspace search from fixture 30 and a conclusion query from fixture
  32 (about `kyle`) or 33 (about anyone else), can answer as a workspace that does not exist
  (`Mode::Missing`: fixtures 34, 35 and 36), and can take a mode for one workspace
  (`set_workspace`). Everything it did before is unchanged.

## Every place that changes

This is the complete list.

| Place | What happens there |
|---|---|
| `crates/memoryd/src/honcho/search.rs` | new: a skeleton you copy, then fill 2 bodies |
| `crates/memoryd/src/honcho/mod.rs` | `pub mod search;` (edit H1) |
| `crates/memoryd/tests/honcho_search.rs` | new, given |
| `crates/memoryd/tests/support/honcho.rs` | new version, given |
| everything else | **no change**: not `serve.rs` (task 09), not `recall.rs`, no `Cargo.toml` |

## Files

- Copy, from `docs/plans/M5c2/files/` (the same path under it): the new test
  `crates/memoryd/tests/honcho_search.rs`; a **new version** of
  `crates/memoryd/tests/support/honcho.rs` (it replaces the file in the tree): yours is the given
  `honcho.rs-task08`, since the given `honcho.rs` is task 10's; and the skeleton
  `crates/memoryd/src/honcho/search.rs`.
- Modify: `crates/memoryd/src/honcho/mod.rs`, `docs/implementer-log.md`.
- Do not touch: any other file, and in the skeleton anything but its two `todo!()` bodies.

## Facts you need

- **The skeleton** is written except two bodies: `source` (one source's calls) and `answer`.
  `conclusions` (one conclusion query), `search` (the threads, the limit, the log) and the types
  are written; read them before you fill the bodies. `search` calls `source` on each thread.
- **`roncho`'s calls and types** (this table is complete; `roncho`'s source, in the Cargo
  registry, is outside the repository and off limits):

  | Item | What it does |
  |---|---|
  | `Client::search_workspace(&self, &MessageSearch) -> Result<Vec<Message>, roncho::blocking::Error>` | `POST /v3/workspaces/{w}/search` |
  | `MessageSearch { query: String, filters, limit: Option<u32>, scope }`, `MessageSearch::new(query)` | all fields public: `MessageSearch { limit: Some(k), ..MessageSearch::new(query) }` sets the limit |
  | `Client::query_conclusions`, `ConclusionQuery` | used in `conclusions`, written |
  | `roncho::Message`, `roncho::Conclusion` | what `items` (task 06) takes |

- **From earlier tasks**: `classify(&roncho::blocking::Error) -> HonchoError` (M5c-1, in
  `client.rs`); `Reader::search() -> &Client`, `Reader::source`, `Reader::label` (task 05);
  `items`, `interleave`, `render`, `label`, `unavailable` (task 06, `recalled.rs`); `Local {
  items, class, untrusted }`, `Recalled { text, class, untrusted }` (task 07, `recall.rs`).
- **A name clash** in `answer`: `Searched::Asked`'s field `unavailable` has the name of task 06's
  function `unavailable`. Bind the field under another name (`unavailable: missing`), as the
  comment says, or the function is hidden inside that arm.
- **The given test** `honcho_search.rs` (12 tests), against the fake Honcho:
  - the limit and the secret line;
  - every source searched with its calls, their paths and bodies (limit, `top_k`, the peers);
  - the agent's own workspaces never searched;
  - each source's hits in the answer with their tags, numbered after a local item;
  - a fleet hit making the answer untrusted, and a trusted read workspace's hit not (T58);
  - a hanging source: one `unavailable` line after the limit (1.5 s), while the others answer;
  - every source hanging, answered after one limit, not one per source;
  - a failing source (503, 422): `unavailable`, logged by kind and status only;
  - a missing workspace: no hits, not unavailable;
  - a query holding a secret: no request at all, and the secret line;
  - nothing found: `(no matches for "zebra")`, then any `unavailable` line;
  - the local hits filling `k`, and the two labels joined.

  A key in it is built from pieces; the waits have limits (the whole file takes about 3 s).
- The calls the bodies use: `MessageSearch::new`, `Client::search_workspace`,
  `Result::map_err`, `classify`, `conclusions` (written), `items`; `usize::try_from`,
  `interleave`, `unavailable`, `label`, `render`, `DataClass::max`, `slice::join`.

## The edits, exactly

### `crates/memoryd/src/honcho/mod.rs`

**H1.** Add one line. The line

```rust
pub mod report;
```

becomes

```rust
pub mod report;
pub mod search;
```

## Steps

- [ ] **1. Copy the given tests.** From the repository root, two commands:

  ```sh
  cp docs/plans/M5c2/files/crates/memoryd/tests/honcho_search.rs crates/memoryd/tests/
  cp docs/plans/M5c2/files/crates/memoryd/tests/support/honcho.rs-task08 \
    crates/memoryd/tests/support/honcho.rs
  ```

  Then `git status --short` lists `crates/memoryd/tests/support/honcho.rs` as modified and
  `honcho_search.rs` as new.
- [ ] **2. See them fail.** `cargo build --tests --keep-going -p memoryd`, with a shell timeout of
  600000 ms: one test target, `honcho_search`, does not compile: `E0432`, ``unresolved import
  `memoryd::honcho::search` ``. Every other target compiles.
- [ ] **3. The skeleton and the edit.** Copy the skeleton, then make edit H1:

  ```sh
  cp docs/plans/M5c2/files/crates/memoryd/src/honcho/search.rs crates/memoryd/src/honcho/
  ```

  `cargo check -p memoryd`: no error, and 12 warnings, all from the unfilled bodies: unused
  imports (`MessageSearch`, `Source`, and `interleave`, `items`, `label`, `render`,
  `unavailable`), unused variables (`reader`, `owner`, `query`, `k`, `local`, `searched`), and
  the function `conclusions`, never used.
- [ ] **4. Fill the bodies**, one at a time, from their comments, with `cargo check -p memoryd`
  after each: `source`, then `answer`. Delete each instruction comment and keep the doc comment
  above the function. After the last, `cargo check -p memoryd` prints no warning.
- [ ] **5. The tests.** With a shell timeout of 600000 ms, every test passes: 13 in
  `honcho_client`, 3 in `honcho_keys` (and 1 ignored, as before), 8 in `honcho_read`, 12 in
  `honcho_search`, 12 in `honcho_worker`:

  ```sh
  cargo test -p memoryd --test honcho_search --test honcho_read --test honcho_client \
    --test honcho_keys --test honcho_worker
  ```
- [ ] **6. Check.** `git diff --stat` lists exactly `crates/memoryd/src/honcho/mod.rs` and
  `crates/memoryd/tests/support/honcho.rs`, and `git status --short` shows `search.rs` and
  `honcho_search.rs` as new.
- [ ] **7. Run the gate.** `cargo fmt --all`, then `make gate`, with a shell timeout of 600000 ms.
  Expected last line: `gate: ok`. The test count is the passed total of `cargo test --workspace`
  plus one, for the ignored `end_to_end` test that the gate runs on its own: **1430** after this
  task, 1418 before it; the 12 new ones are the tests of `honcho_search.rs`. To count, add up
  the `N passed` of every `test result:` line the gate's `cargo test --workspace` printed, and
  add 1. Never run the gate again only to count.
- [ ] **8. Log and commit.** Add your row to `docs/implementer-log.md`; its Task cell is
  `M5c2/08-memoryd-honcho-search`. Stage with
  `git add crates/memoryd docs/implementer-log.md Cargo.lock`, then commit as `AGENTS.md` says.
  `Cargo.lock` does not change in this task; the `git add` is harmless.

## Done when

- `make gate` prints `gate: ok` with 1430 tests.
- `git diff HEAD~1 --stat` lists only `mod.rs`, `search.rs`, `honcho_search.rs`,
  `support/honcho.rs` and `docs/implementer-log.md`.

## Stop and report if

- The branch check at the top fails.
- The "before" line of H1 is not in the file exactly once.
- Step 2 or 3 shows a different result: other targets failing, other errors, or warnings about
  anything but the unfilled bodies.
- `cargo check` reports an error after a body, and its comment does not tell you how to fix it.
  Quote the error, and stop.
- After step 4 a test fails although each body follows its comment. Do not change a test, the
  fake or the written parts of the skeleton: stop, and give the failing tests' names and their
  messages.
- The fix seems to need a change anywhere else: `serve.rs`, `recall.rs`, `read.rs`,
  `recalled.rs`, a new dependency. It does not; stop and say why you think so.
- A test outside the given files fails in the gate: check once whether it fails without your
  change (`git stash -u`, run it, `git stash pop`), then stop and report both results.
- The gate passes with a count other than 1430.
- A comment in the skeleton does not fit its signature. Quote it and stop.

## What the given tests pin

Each of these changes to the finished code makes at least one given test fail (19 tried, all
caught): the secret check skipped; the search without its limit, or a conclusion query without
`top_k` or with another observer; the conclusions about the agent's peer or the owner's left
out, or asked of the fleet; a failed or a late source not reported unavailable; no limit on the
wait; the sources asked one after another; the local items left out, or Honcho's numbered from
1 or interleaved as if there were none; no `(no matches …)` line; the `unavailable` lines left
out; either part's label ignored; the secret line's words.