Apply P19: recall over the local index and Honcho; trust per workspace

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

docs/decisions.md

@@ -65,7 +65,6 @@ Newest first. A decision that changes `docs/design.md` lands in the same commit
 | Secret store: settled 2026-09-23 (systemd credentials, environment, owner-only file; P15 proposed). Where `brokerd` reads tool secrets from, and whether `SecretStore` moves out of `gatewayd` into a shared place, is decided when a tool first needs a secret. | when a tool needs a secret |
 | Whether cloud-led sessions are ever allowed, and for which data classes. | M6 |
 | Projects in memory. Honcho can separate projects by a workspace per project (hard walls, shared naturally by every agent on the project, but Honcho's picture of the owner splits per workspace) or by scopes inside one workspace (one picture, recall filtered by project; semantics not yet read). Either way a project is a `memoryd` setting: its channels and threads, its workspace or scope, `write` and `trust`. A shared project workspace is written by several agents, so its trust is the owner's call per project. Test both on rift before choosing. Owner's idea, 2026-09-24. | M5b or a later M5d |
-| Where the M5b embedding model runs. Superseded by P19 if it is approved: Honcho's embeddings run on dixie (`honcho-embed`). | M5b |
 
 ## Applied to the design brief
 
@@ -97,6 +96,7 @@ Later changes to the brief:
 | 2026-09-24 | P16, Network isolation 3: approvals are commands (`!approve`, `!deny`) replied in the approval's thread; emoji reactions are not used. | Owner's decision and approval, 2026-09-24; M4b spec `docs/specs/2026-09-24-m4b-approvals.md`. |
 | 2026-09-24 | P17, Memory 1, the baseline and State: the core block becomes three files (`soul.md`, owner-written; `memory.md` and `user.md`, agent-edited at `ask` with add, replace, remove and `consolidate`), read once per epoch; moved entries go to `memory/archive.md`. | Owner's approval, 2026-09-24; see the row of that date under Decided. |
 | 2026-09-24 | P18, Roles and Network isolation: a sixth role, `memoryd`, owns `memory/` and `index/`, syncs turns to Honcho and runs memory tool calls; `loopd` has four sockets and no longer holds the memory files. | Owner's approval, 2026-09-24; see the row of that date under Decided. |
+| 2026-09-24 | P19, Memory 2 to 8: `recall` over a local FTS5 index (session logs, archive) and Honcho, `ask_memory`; Honcho writes only from clean sessions; per-workspace trust set by the owner; notes, the flush job and the embedding index go; memory text stays on the owner's hosts. | Owner's approval, 2026-09-24; see the row of that date under Decided. |
 
 ## Proposed changes to the design brief
 
@@ -104,5 +104,4 @@ Proposed 2026-09-24 for M5 (row of that date under Decided). Not yet applied.
 
 | # | Change | Reason |
 |---|---|---|
-| P19 | Memory 2 to 9: notes files, the pre-compaction flush, the embedding index (M5b) and "no vector database service" are replaced. `recall` searches the local full-text index (SQLite FTS5 over the session logs and `memory/archive.md`, rebuildable with `bxctl reindex`) and Honcho; `ask_memory` asks Honcho's dialectic. Both go through `brokerd` at `auto` and run in `memoryd`. Entries found in `archive.md` come back as approved memory. `memoryd` counts `recall` hits per archive entry and when each was last found, for the tiering job (P20, M5c). From M5b, entries moved by `consolidate` are also written to Boxmaker's Honcho workspace as conclusions. `memoryd` writes to Honcho only turns from sessions that saw no untrusted content and are within a configured data class, and it enforces that itself. Each workspace read has an owner-set trust level; results from an untrusted workspace (the default) are marked untrusted and taint the session. Trust is per workspace because its writers differ; the brief names no workspace. Every result carries its source (the file and line, or the workspace) and data class. "No memory text leaves the host" becomes "no memory text leaves the owner's hosts": Honcho runs on rift with its models on dixie. | Honcho's deriver, summaries and dreamer do the extraction the flush job was for, and `honcho-embed` on dixie is the embedding index. Honcho's conclusions carry no per-entry provenance, so the untrusted filter must be applied when writing, and each workspace is labelled as a whole. |
 | P20 | Milestones: M5 becomes M5a (`memoryd` without a network: the three files, the `memory` tool with `consolidate`, the archive, full-text `recall`), M5b (`memoryd` and Honcho) and M5c (heartbeat and cron on their own slot). The old M5b row goes. Acceptance moves with the parts: a fact from epoch 1 is recalled in epoch 2, and an entry moved by `consolidate` is found by `recall` (M5a); a turn from a tainted session never reaches Honcho, and a result from an untrusted workspace taints the session (M5b); a scheduled run leaves the main slot's cache intact, and the tiering job's proposals reach the owner as ordinary approvals and change nothing on their own (M5c). M5c's tiering job: a scheduled session that reads the memory files, the archive with its hit counts, and Boxmaker's Honcho summaries, and proposes a `consolidate` (moving entries down) or an `add` naming its source (bringing one up), when a file passes about 80% full or weekly, at most one per file per run. | As for M2 to M4: the parts are reviewed separately, and M5a needs no network. |

docs/design.md

@@ -188,27 +188,29 @@ the inference contract, and it must not become a way for injected text to persis
    as a diff. An approved edit takes effect at the next epoch, never mid-session. A write over a
    file's limit is refused with a pointer to `consolidate`.
 2. **Everything else is retrieved, not injected.** A `recall(query, k)` tool returns size-capped
-   snippets with file, line and provenance. Results are appended like any tool result, so the
+   snippets with their source (a file and line, or a Honcho workspace). An `ask_memory` tool asks
-   cache prefix is untouched.
+   Honcho's dialectic. Results are appended like any tool result, so the cache prefix is untouched.
-3. **Notes.** `memory/notes/YYYY-MM-DD.md`, append-only, written by a `remember` tool. Each entry
+3. **Local index.** SQLite FTS5 over the session logs and `memory/archive.md`, kept by `memoryd`
-   records its session, time, data-class label and whether the session had seen untrusted content.
+   and rebuildable with `bxctl reindex`. It needs no network and no model, so it still answers
-4. **Flush before compaction.** When a session goes idle and is about to change epoch, a job on
+   when Honcho is down. `memoryd` counts `recall` hits per archive entry and when each was last
-   the scheduled slot extracts durable facts from the log into notes. The main slot is not used.
+   found. Retrieval sits behind a `Retriever` trait.
-5. **Provenance on recall.** Recalled text is presented as data with its provenance, never placed
+4. **Honcho.** The fleet's memory service; it and the models it calls run on the owner's hosts. Each
-   in the system prompt. Entries written from sessions that saw untrusted content are marked as
+   Boxmaker writes finished turns to its own workspace, and only turns from sessions that saw no
-   such when recalled. Recalled entries carry their data-class label and taint the session.
+   untrusted content and are within a configured data class; `memoryd` enforces this itself. Entries
-6. **Both tools go through `brokerd`** under ordinary grants: `recall` and note-writing `auto`,
+   moved by `consolidate` are written there too. Honcho's own extraction, summaries and
-   core and topic edits `ask`.
+   consolidation replace a flush job of ours. What Honcho holds is derived from the session logs and
-7. **Index.** Lexical search first: SQLite FTS5 over the Markdown files, rebuildable with
+   `memory/archive.md`, and can be rebuilt from them.
-   `bxctl reindex`. Retrieval sits behind a `Retriever` trait.
+5. **Workspaces and trust.** `memoryd` can read other workspaces. The owner marks each one
-8. **Embeddings are a later, additive index** (M5b), never a store. Local model only, served by a
+   trusted or untrusted; untrusted is the default. Results from an untrusted workspace are marked
-   separate `llama-server` embedding instance so chat slots are unaffected. Vectors live in a flat
+   untrusted and taint the session. Honcho's conclusions carry no per-entry provenance, so the
-   file or SQLite table keyed by chunk hash and embedding-model hash; a model change triggers a
+   filter is applied when writing and the label is per workspace.
-   rebuild. Combine with lexical results by rank fusion. Add M5b when logged `recall` misses show
+6. **Provenance on recall.** Recalled text is presented as data with its source and data class,
-   lexical search failing, not before.
+   never placed in the system prompt, and taints the session with that class. Entries from
-9. **No vector database service.** Search is exact brute-force cosine, in-process. Revisit only if
+   `memory/archive.md` come back as approved memory.
-   the index passes about one million chunks or measured `recall` latency exceeds one second, and
+7. **The tools go through `brokerd`** under ordinary grants and run in `memoryd`: `recall` and
-   then prefer an embedded, file-backed approximate index over a server.
+   `ask_memory` `auto`, every `memory` edit `ask`.
+8. **No memory text leaves the owner's hosts.** Honcho and the models it calls run on the owner's
+   hosts, and each is listed in `docs/egress.md`.
 
 ## State
 
@@ -216,7 +218,7 @@ Files are the source of truth. SQLite is allowed only for rebuildable indexes an
 
 - `sessions/<id>/<epoch>.jsonl` — append-only session log
 - `memory/soul.md`, `memory/memory.md`, `memory/user.md`, `memory/archive.md` — owner-readable memory
-- `index/` — FTS5 and, later, vectors; disposable and rebuildable from `memory/`
+- `index/` — FTS5, kept by `memoryd`; disposable and rebuildable from `memory/` and `sessions/`
 - `grants/*.toml` — owner-written
 - `audit/*.jsonl` — hash-chained
 - `broker/sessions/<id>.json` — each session's taint and untrusted flag, written only by