From 6d36c54ddd818858a781618d0932094e0f1dd012 Mon Sep 17 00:00:00 2001 From: "K. Isom" Date: Thu, 24 Sep 2026 13:03:07 -0700 Subject: [PATCH] M5 proposals: consolidate moves memories to an archive; memoryd owns the memory files and the index The owner's review of 2026-09-24. P17: the memory tool gains consolidate, which moves entries out of memory.md into memory/archive.md, and from M5b into Honcho, instead of losing them. P18: memoryd owns memory/ and the SQLite index, so every memory tool runs in one place. Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/decisions.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index ad321d9..a51af18 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -6,7 +6,7 @@ Newest first. A decision that changes `docs/design.md` lands in the same commit | Date | Decision | Reason | |---|---|---| -| 2026-09-24 | M5 direction, before a spec. Memory is three local files and Honcho. The files: `memory/soul.md` (owner-written), `memory/memory.md` and `memory/user.md` (agent-edited, `ask`), read once per epoch, as Hermes does. Honcho, the fleet's memory service on rift (v3.2.0; models on dixie), reached by a new role, `memoryd`: each Boxmaker writes to its own workspace, only turns from sessions that saw no untrusted content, and can read other workspaces such as `hermes-fleet`, each with an owner-set trust level (untrusted by default). Local full-text search over the session logs stays. M5 splits into M5a (files and search, no network), M5b (`memoryd`, Honcho through the owner's `roncho` client) and M5c (the scheduler). Brief changes P17 to P20 proposed. | Owner, 2026-09-24: Honcho lets several Boxmakers share memory across the fleet; the Hermes agents do Boxmaker's job today and may be trusted. `brokerd` never holds conversation history and `loopd` has no network, so syncing turns needs a role of its own. Honcho's derivation replaces the brief's notes, pre-compaction flush and embedding index. | +| 2026-09-24 | M5 direction, before a spec. Memory is three local files and Honcho. The files: `memory/soul.md` (owner-written), `memory/memory.md` and `memory/user.md` (agent-edited, `ask`), read once per epoch, as Hermes does. Honcho, the fleet's memory service on rift (v3.2.0; models on dixie), reached by a new role, `memoryd`, which also owns the memory files and the SQLite index (owner, same day): each Boxmaker writes to its own workspace, only turns from sessions that saw no untrusted content, and can read other workspaces such as `hermes-fleet`, each with an owner-set trust level (untrusted by default). Local full-text search over the session logs stays. `consolidate` moves entries out of `memory.md` into `memory/archive.md` (searched by `recall`) and, from M5b, into Honcho. M5 splits into M5a (files and search, no network), M5b (`memoryd`, Honcho through the owner's `roncho` client) and M5c (the scheduler). Brief changes P17 to P20 proposed. | Owner, 2026-09-24: Honcho lets several Boxmakers share memory across the fleet; the Hermes agents do Boxmaker's job today and may be trusted. `brokerd` never holds conversation history and `loopd` has no network, so syncing turns needs a role of its own. Honcho's derivation replaces the brief's notes, pre-compaction flush and embedding index. | | 2026-09-24 | M4b, changing the agreed outline (row of 2026-09-23): no reactions, an approval is answered only by `!approve ` or `!deny [reason]` in its thread; an `[approvals] users` list in `gatewayd.toml`, which falls back to `allow.users` when empty; approvals are posted in the asking thread, channel threads included; on restart, `gatewayd` marks its old approval posts dead and `brokerd`'s courier posts pending approvals again. Spec `docs/specs/2026-09-24-m4b-approvals.md`. Brief change P16 proposed. Also: `gatewayd` strips a leading mention of its own bot from every message before handling it; in a thread shared by several Boxmakers the operator addresses approval commands, and a bare one is taken by every Boxmaker with that id pending there. | Owner: reactions are not used. Approvers are separate because an agent allowed to talk to Boxmaker should not be able to approve for it, but the default keeps one list for one person. Redacting secrets from arguments comes with the tools work. A restart must not leave posts that look answerable and are not. The mention is how the operator chooses the bot; the frontend removes it so neither the commands nor the model see it. | | 2026-09-23 | M4a spec approved with two changes: a secret read from a file prints a warning at startup (and still starts); and in allowed channels and group messages shared with other agents, Boxmaker answers a post only if it names this bot, or it is a reply in one of this Boxmaker's threads that names no other user (`@channel`, `@here`, `@all` name nobody). One Boxmaker per machine, each with its own bot account; nothing is shared between them. | Owner's notes. The owner runs several agents in channels, possibly one Boxmaker per machine. A post's `from_bot` property is user-settable in v11.11.0, so bots are told apart by the allowlist, not by that property. | | 2026-09-23 | M4 is split: M4a is conversations over Mattermost (`gatewayd`, the secret store), M4b is approvals over Mattermost. Spec `docs/specs/2026-09-23-m4a-gateway.md`. | As with M2 and M3: the authority-bearing half is reviewed separately, and the open question about long approvals lands in M4b. | @@ -101,7 +101,7 @@ Proposed 2026-09-24 for M5 (row of that date under Decided). Not yet applied. | # | Change | Reason | |---|---|---| -| P17 | Memory 1 and State: the core block becomes three files, read once per epoch into the baseline, in this order after the system prompt: `memory/soul.md` (persona; owner-written only; at most 4,000 characters), `memory/memory.md` and `memory/user.md` (the agent's notes on its work and on the owner; at most 2,200 and 1,375 characters). The agent edits the last two with a `memory` tool (add, replace, remove an entry) at `ask`; `loopd` applies an approved edit to the file, and it takes effect at the next epoch. The baseline stays at 3,000 tokens or less, measured on the device. `core.md` and its 300-token limit go. | Owner: as Hermes does (`~/.hermes/SOUL.md`, `memories/MEMORY.md`, `USER.md`, frozen at session start for the prompt cache). The owner's current files are 2,870, 2,151 and 790 characters, about 1,500 tokens; the rest of the baseline measured 251 tokens at M2b. | -| P18 | Roles: a sixth role, `memoryd`. Purpose: syncs finished turns to Honcho and answers memory queries. Holds: Honcho's address and workspace list (and a key, if Honcho's authentication is on). Can reach: Honcho only; a socket from `loopd` (writes) and one from `brokerd` (reads). Never holds: grants, tool secrets, the Mattermost token, session files. `loopd` gains a fourth socket. | `brokerd` never holds conversation history and `loopd` never has a network; syncing turns needs both conversation text and the network, so it cannot live in either. Reads stay ordinary tool calls that `brokerd` decides and audits. | -| P19 | Memory 2 to 9: notes files, the pre-compaction flush, the embedding index (M5b) and "no vector database service" are replaced. `recall` searches the local full-text index (SQLite FTS5 over the session logs, rebuildable with `bxctl reindex`) and Honcho; `ask_memory` asks Honcho's dialectic. Both go through `brokerd` at `auto`. `memoryd` writes to Honcho only turns from sessions that saw no untrusted content and are within a configured data class, and it enforces that itself. Each workspace read has an owner-set trust level; results from an untrusted workspace (the default) are marked untrusted and taint the session. Every result carries its source (the file and line, or the workspace) and data class. "No memory text leaves the host" becomes "no memory text leaves the owner's hosts": Honcho runs on rift with its models on dixie. | Honcho's deriver, summaries and dreamer do the extraction the flush job was for, and `honcho-embed` on dixie is the embedding index. Honcho's conclusions carry no per-entry provenance, so the untrusted filter must be applied when writing, and each workspace is labelled as a whole. | -| P20 | Milestones: M5 becomes M5a (the three files, the `memory` tool, full-text `recall` over session logs; no network), M5b (`memoryd` and Honcho) and M5c (heartbeat and cron on their own slot). The old M5b row goes. Acceptance moves with the parts: a fact from epoch 1 is recalled in epoch 2 (M5a); a turn from a tainted session never reaches Honcho, and a result from an untrusted workspace taints the session (M5b); a scheduled run leaves the main slot's cache intact (M5c). | As for M2 to M4: the parts are reviewed separately, and M5a needs no network. | +| P17 | Memory 1 and State: the core block becomes three files, read once per epoch into the baseline, in this order after the system prompt: `memory/soul.md` (persona; owner-written only; at most 4,000 characters), `memory/memory.md` and `memory/user.md` (the agent's core notes on its work and on the owner; at most 2,200 and 1,375 characters; each shown with how full it is). The agent edits the last two with a `memory` tool: `add`, `replace` and `remove` an entry, and `consolidate`, one edit that marks every entry of a file kept, rewritten, moved or dropped. Moved entries are appended to `memory/archive.md` (append-only, each entry with the date and session it was first approved in), where `recall` finds them; nothing leaves `memory.md` silently. Every edit is `ask`, and the approval shows `consolidate` as a diff. An approved edit takes effect at the next epoch. A write over the cap is refused with a pointer to `consolidate`. The baseline stays at 3,000 tokens or less, measured on the device. `core.md` and its 300-token limit go. | Owner: as Hermes does (`~/.hermes/SOUL.md`, `memories/MEMORY.md`, `USER.md`, frozen at session start for the prompt cache); and `consolidate` moves memories out of the fast path rather than losing them (owner, 2026-09-24). The owner's current files are 2,870, 2,151 and 790 characters, about 1,500 tokens; the rest of the baseline measured 251 tokens at M2b. With every edit at `ask`, one consolidation is one approval, not one per entry. | +| P18 | Roles: a sixth role, `memoryd`, the one owner of memory. Purpose: keeps the memory files and the index, syncs finished turns to Honcho, and answers memory tool calls. Holds: `memory/` (the three files and `archive.md`), `index/` (SQLite), Honcho's address and workspace list (and a key, if Honcho's authentication is on). Can reach: Honcho only (from M5b); a socket from `loopd` (finished turns in; the three files out, once per epoch) and one from `brokerd` (memory tool calls it has decided). Reads `sessions/` for `bxctl reindex`, never writes it. Never holds: grants, tool secrets, the Mattermost token. Sessions above a configured data class are not indexed. `loopd` loses "memory files" from its Holds column, keeps no network, and gains a fourth socket. | `brokerd` never holds conversation history and `loopd` never has a network; syncing turns needs both conversation text and the network. One owner for every memory tool means `brokerd` decides and `memoryd` executes, with no call from `brokerd` into `loopd`, and SQLite is linked by one crate. The cost: a copy of session text in a process that reaches Honcho; `memoryd` is not steered by a model, its network is Honcho only (enforced in M7), and the most sensitive classes are not indexed. | +| P19 | Memory 2 to 9: notes files, the pre-compaction flush, the embedding index (M5b) and "no vector database service" are replaced. `recall` searches the local full-text index (SQLite FTS5 over the session logs and `memory/archive.md`, rebuildable with `bxctl reindex`) and Honcho; `ask_memory` asks Honcho's dialectic. Both go through `brokerd` at `auto` and run in `memoryd`. Entries found in `archive.md` come back as approved memory. From M5b, entries moved by `consolidate` are also written to Boxmaker's Honcho workspace as conclusions. `memoryd` writes to Honcho only turns from sessions that saw no untrusted content and are within a configured data class, and it enforces that itself. Each workspace read has an owner-set trust level; results from an untrusted workspace (the default) are marked untrusted and taint the session. Every result carries its source (the file and line, or the workspace) and data class. "No memory text leaves the host" becomes "no memory text leaves the owner's hosts": Honcho runs on rift with its models on dixie. | Honcho's deriver, summaries and dreamer do the extraction the flush job was for, and `honcho-embed` on dixie is the embedding index. Honcho's conclusions carry no per-entry provenance, so the untrusted filter must be applied when writing, and each workspace is labelled as a whole. | +| P20 | Milestones: M5 becomes M5a (`memoryd` without a network: the three files, the `memory` tool with `consolidate`, the archive, full-text `recall`), M5b (`memoryd` and Honcho) and M5c (heartbeat and cron on their own slot). The old M5b row goes. Acceptance moves with the parts: a fact from epoch 1 is recalled in epoch 2, and an entry moved by `consolidate` is found by `recall` (M5a); a turn from a tainted session never reaches Honcho, and a result from an untrusted workspace taints the session (M5b); a scheduled run leaves the main slot's cache intact (M5c). | As for M2 to M4: the parts are reviewed separately, and M5a needs no network. |