Apply P18: memoryd, the one owner of memory
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
+1
-1
@@ -96,6 +96,7 @@ Later changes to the brief:
|
||||
| 2026-09-23 | P15, State, the secrets line: three `SecretStore` backends chosen per secret (a systemd credential by default, an environment variable, an owner-only file), the last two in plaintext and for portability, a file secret warning at startup. | Owner's decision and approval, 2026-09-23 (M4a). The brief's "no plaintext secrets on disk" did not allow the file backend. |
|
||||
| 2026-09-24 | P16, Network isolation 3: approvals are commands (`!approve`, `!deny`) replied in the approval's thread; emoji reactions are not used. | Owner's decision and approval, 2026-09-24; M4b spec `docs/specs/2026-09-24-m4b-approvals.md`. |
|
||||
| 2026-09-24 | P17, Memory 1, the baseline and State: the core block becomes three files (`soul.md`, owner-written; `memory.md` and `user.md`, agent-edited at `ask` with add, replace, remove and `consolidate`), read once per epoch; moved entries go to `memory/archive.md`. | Owner's approval, 2026-09-24; see the row of that date under Decided. |
|
||||
| 2026-09-24 | P18, Roles and Network isolation: a sixth role, `memoryd`, owns `memory/` and `index/`, syncs turns to Honcho and runs memory tool calls; `loopd` has four sockets and no longer holds the memory files. | Owner's approval, 2026-09-24; see the row of that date under Decided. |
|
||||
|
||||
## Proposed changes to the design brief
|
||||
|
||||
@@ -103,6 +104,5 @@ Proposed 2026-09-24 for M5 (row of that date under Decided). Not yet applied.
|
||||
|
||||
| # | Change | Reason |
|
||||
|---|---|---|
|
||||
| P18 | Roles: a sixth role, `memoryd`, the one owner of memory. Purpose: keeps the memory files and the index, syncs finished turns to Honcho, and answers memory tool calls. Holds: `memory/` (the three files and `archive.md`), `index/` (SQLite), Honcho's address and workspace list (and a key, if Honcho's authentication is on). Can reach: Honcho only (from M5b); a socket from `loopd` (finished turns in; the three files out, once per epoch) and one from `brokerd` (memory tool calls it has decided). Reads `sessions/` for `bxctl reindex`, never writes it. Never holds: grants, tool secrets, the Mattermost token. Sessions above a configured data class are not indexed. `loopd` loses "memory files" from its Holds column, keeps no network, and gains a fourth socket. | `brokerd` never holds conversation history and `loopd` never has a network; syncing turns needs both conversation text and the network. One owner for every memory tool means `brokerd` decides and `memoryd` executes, with no call from `brokerd` into `loopd`, and SQLite is linked by one crate. The cost: a copy of session text in a process that reaches Honcho; `memoryd` is not steered by a model, its network is Honcho only (enforced in M7), and the most sensitive classes are not indexed. |
|
||||
| P19 | Memory 2 to 9: notes files, the pre-compaction flush, the embedding index (M5b) and "no vector database service" are replaced. `recall` searches the local full-text index (SQLite FTS5 over the session logs and `memory/archive.md`, rebuildable with `bxctl reindex`) and Honcho; `ask_memory` asks Honcho's dialectic. Both go through `brokerd` at `auto` and run in `memoryd`. Entries found in `archive.md` come back as approved memory. `memoryd` counts `recall` hits per archive entry and when each was last found, for the tiering job (P20, M5c). From M5b, entries moved by `consolidate` are also written to Boxmaker's Honcho workspace as conclusions. `memoryd` writes to Honcho only turns from sessions that saw no untrusted content and are within a configured data class, and it enforces that itself. Each workspace read has an owner-set trust level; results from an untrusted workspace (the default) are marked untrusted and taint the session. Trust is per workspace because its writers differ; the brief names no workspace. Every result carries its source (the file and line, or the workspace) and data class. "No memory text leaves the host" becomes "no memory text leaves the owner's hosts": Honcho runs on rift with its models on dixie. | Honcho's deriver, summaries and dreamer do the extraction the flush job was for, and `honcho-embed` on dixie is the embedding index. Honcho's conclusions carry no per-entry provenance, so the untrusted filter must be applied when writing, and each workspace is labelled as a whole. |
|
||||
| P20 | Milestones: M5 becomes M5a (`memoryd` without a network: the three files, the `memory` tool with `consolidate`, the archive, full-text `recall`), M5b (`memoryd` and Honcho) and M5c (heartbeat and cron on their own slot). The old M5b row goes. Acceptance moves with the parts: a fact from epoch 1 is recalled in epoch 2, and an entry moved by `consolidate` is found by `recall` (M5a); a turn from a tainted session never reaches Honcho, and a result from an untrusted workspace taints the session (M5b); a scheduled run leaves the main slot's cache intact, and the tiering job's proposals reach the owner as ordinary approvals and change nothing on their own (M5c). M5c's tiering job: a scheduled session that reads the memory files, the archive with its hit counts, and Boxmaker's Honcho summaries, and proposes a `consolidate` (moving entries down) or an `add` naming its source (bringing one up), when a file passes about 80% full or weekly, at most one per file per run. | As for M2 to M4: the parts are reviewed separately, and M5a needs no network. |
|
||||
|
||||
+9
-6
@@ -47,9 +47,10 @@ cloud-led orchestration as the default, in-process third-party code of any kind.
|
||||
| Role | Purpose | Holds | Can reach | Never holds |
|
||||
|---|---|---|---|---|
|
||||
| `gatewayd` | Carries messages between the owner's Mattermost and the harness. | Mattermost bot token, allowlist, message queue | Mattermost server (outbound only), `loopd` socket | Tool secrets, grants |
|
||||
| `loopd` | Runs the conversation: builds each prompt, calls the model, and asks `brokerd` for every tool call. | Session logs, prompt assembly, memory files | Three Unix sockets: `gatewayd`, `brokerd`, `inferproxy` | Any credential, any network interface |
|
||||
| `loopd` | Runs the conversation: builds each prompt, calls the model, and asks `brokerd` for every tool call. | Session logs, prompt assembly | Four Unix sockets: `gatewayd`, `brokerd`, `inferproxy`, `memoryd` | Any credential, any network interface |
|
||||
| `inferproxy` | Lets `loopd` reach the model without having a network. | The `llama-server` address | `llama-server` only | Anything else; it forwards bytes and logs nothing |
|
||||
| `brokerd` | Decides whether each tool call may run, runs it in a fresh container, and records the decision. | Grants (read-only), secrets, audit log | Container runtime, `gatewayd` socket (for approvals) | Conversation history |
|
||||
| `memoryd` | Keeps the memory files and their index, syncs finished turns to Honcho, and runs every memory tool call `brokerd` has decided. | `memory/`, `index/`, Honcho's address and workspace list (and its key, if any) | Honcho only; sockets from `loopd` (finished turns in, the memory files out once per epoch) and from `brokerd` (memory tool calls); `sessions/` read-only, for reindexing | Grants, tool secrets, the Mattermost token |
|
||||
| tool containers | Do one approved tool call and then cease to exist. | Only what one call was granted | Only what one call was granted | Anything persistent |
|
||||
|
||||
v0 runs all roles on one host. The interfaces must not assume co-location with `llama-server`.
|
||||
@@ -60,17 +61,19 @@ The host's tailnet identity is shared by every container that has a network, so
|
||||
ordinary outbound networking can reach the whole tailnet and the internet as the host.
|
||||
|
||||
1. `loopd` runs with no network namespace interfaces except loopback (`--network=none`). Its only
|
||||
I/O is three Unix sockets on a shared volume.
|
||||
I/O is four Unix sockets on a shared volume.
|
||||
2. `inferproxy` is a forwarder of about 100 lines: one Unix socket in, one fixed upstream out. The
|
||||
upstream is the shared router's TCP listener on the host. It is required: `loopd` has no
|
||||
network, and the router stays on TCP for its other clients.
|
||||
3. `gatewayd` has outbound network only. It opens no listening TCP port. Approvals are commands
|
||||
(`!approve`, `!deny`) replied in the approval's thread, which arrive over the Mattermost
|
||||
WebSocket, so no callback URL is needed.
|
||||
4. Tool containers get no network unless the grant names hosts. Granted network access goes through
|
||||
4. `memoryd` reaches Honcho and nothing else. It does not index sessions above a configured data
|
||||
class.
|
||||
5. Tool containers get no network unless the grant names hosts. Granted network access goes through
|
||||
an allowlisting egress proxy that `brokerd` configures per call.
|
||||
5. Tailnet ACLs (kept in `deploy/`) limit the host's node to `llama-server` and Mattermost, plus
|
||||
whatever tool grants need. This is defence in depth, not the role boundary.
|
||||
6. Tailnet ACLs (kept in `deploy/`) limit the host's node to `llama-server`, Mattermost and Honcho,
|
||||
plus whatever tool grants need. This is defence in depth, not the role boundary.
|
||||
|
||||
## Channel: Mattermost
|
||||
|
||||
@@ -227,7 +230,7 @@ Files are the source of truth. SQLite is allowed only for rebuildable indexes an
|
||||
## Code constraints
|
||||
|
||||
- Rust stable, Cargo workspace. One crate per role in the table above (`loopd`, `brokerd`,
|
||||
`gatewayd`, `inferproxy`), plus three that are not roles:
|
||||
`gatewayd`, `inferproxy`, `memoryd`), plus three that are not roles:
|
||||
|
||||
| Crate | Purpose |
|
||||
|---|---|
|
||||
|
||||
Reference in New Issue
Block a user