Apply P19: recall over the local index and Honcho; trust per workspace

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-09-24 13:12:11 -07:00
co-authored by Claude Opus 5.5
parent 9685289242
commit 774807aa81
2 changed files with 25 additions and 24 deletions
+24 -22
View File
@@ -188,27 +188,29 @@ the inference contract, and it must not become a way for injected text to persis
as a diff. An approved edit takes effect at the next epoch, never mid-session. A write over a
file's limit is refused with a pointer to `consolidate`.
2. **Everything else is retrieved, not injected.** A `recall(query, k)` tool returns size-capped
snippets with file, line and provenance. Results are appended like any tool result, so the
cache prefix is untouched.
3. **Notes.** `memory/notes/YYYY-MM-DD.md`, append-only, written by a `remember` tool. Each entry
records its session, time, data-class label and whether the session had seen untrusted content.
4. **Flush before compaction.** When a session goes idle and is about to change epoch, a job on
the scheduled slot extracts durable facts from the log into notes. The main slot is not used.
5. **Provenance on recall.** Recalled text is presented as data with its provenance, never placed
in the system prompt. Entries written from sessions that saw untrusted content are marked as
such when recalled. Recalled entries carry their data-class label and taint the session.
6. **Both tools go through `brokerd`** under ordinary grants: `recall` and note-writing `auto`,
core and topic edits `ask`.
7. **Index.** Lexical search first: SQLite FTS5 over the Markdown files, rebuildable with
`bxctl reindex`. Retrieval sits behind a `Retriever` trait.
8. **Embeddings are a later, additive index** (M5b), never a store. Local model only, served by a
separate `llama-server` embedding instance so chat slots are unaffected. Vectors live in a flat
file or SQLite table keyed by chunk hash and embedding-model hash; a model change triggers a
rebuild. Combine with lexical results by rank fusion. Add M5b when logged `recall` misses show
lexical search failing, not before.
9. **No vector database service.** Search is exact brute-force cosine, in-process. Revisit only if
the index passes about one million chunks or measured `recall` latency exceeds one second, and
then prefer an embedded, file-backed approximate index over a server.
snippets with their source (a file and line, or a Honcho workspace). An `ask_memory` tool asks
Honcho's dialectic. Results are appended like any tool result, so the cache prefix is untouched.
3. **Local index.** SQLite FTS5 over the session logs and `memory/archive.md`, kept by `memoryd`
and rebuildable with `bxctl reindex`. It needs no network and no model, so it still answers
when Honcho is down. `memoryd` counts `recall` hits per archive entry and when each was last
found. Retrieval sits behind a `Retriever` trait.
4. **Honcho.** The fleet's memory service; it and the models it calls run on the owner's hosts. Each
Boxmaker writes finished turns to its own workspace, and only turns from sessions that saw no
untrusted content and are within a configured data class; `memoryd` enforces this itself. Entries
moved by `consolidate` are written there too. Honcho's own extraction, summaries and
consolidation replace a flush job of ours. What Honcho holds is derived from the session logs and
`memory/archive.md`, and can be rebuilt from them.
5. **Workspaces and trust.** `memoryd` can read other workspaces. The owner marks each one
trusted or untrusted; untrusted is the default. Results from an untrusted workspace are marked
untrusted and taint the session. Honcho's conclusions carry no per-entry provenance, so the
filter is applied when writing and the label is per workspace.
6. **Provenance on recall.** Recalled text is presented as data with its source and data class,
never placed in the system prompt, and taints the session with that class. Entries from
`memory/archive.md` come back as approved memory.
7. **The tools go through `brokerd`** under ordinary grants and run in `memoryd`: `recall` and
`ask_memory` `auto`, every `memory` edit `ask`.
8. **No memory text leaves the owner's hosts.** Honcho and the models it calls run on the owner's
hosts, and each is listed in `docs/egress.md`.
## State
@@ -216,7 +218,7 @@ Files are the source of truth. SQLite is allowed only for rebuildable indexes an
- `sessions/<id>/<epoch>.jsonl` — append-only session log
- `memory/soul.md`, `memory/memory.md`, `memory/user.md`, `memory/archive.md` — owner-readable memory
- `index/` — FTS5 and, later, vectors; disposable and rebuildable from `memory/`
- `index/` — FTS5, kept by `memoryd`; disposable and rebuildable from `memory/` and `sessions/`
- `grants/*.toml` — owner-written
- `audit/*.jsonl` — hash-chained
- `broker/sessions/<id>.json` — each session's taint and untrusted flag, written only by