kyle/boxmaker
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

M5c spec: Honcho, draft for the owner's review

Browse Source 
Designed with the owner on 2026-09-27: per agent a trusted and an
untrusted workspace plus one fleet workspace for untrusted notes;
private ceiling, secret sessions never synced; no rebuild (contra-facts,
or deleting our own conclusion); the owner and other people observed,
the agent not; every trusted memory write mirrored as a conclusion; a
decision ledger that doubles as the outbox; recall over the local index
and the fleet; ask_memory by workspace chat. Grounded in roncho 1.0.0
and Honcho 3.2.0 (source read at v3.2.0) and rift's live metrics.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
kyle
2026-09-27 22:11:31 -07:00
co-authored by Claude Opus 5.5
parent 4d1120032a
commit 23c9ebec12
1 changed files with 325 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/specs/2026-09-27-m5c-honcho.md
+325
View File
@@ -0,0 +1,325 @@
# M5c design: Honcho
Status: draft for the owner's review, 2026-09-27. Designed with the owner the same day (the
decisions are marked "owner"). Supersedes P23 where they differ (section 14).
## 1. What M5c proves
The agent's experience outlives a session and is shared across the fleet, without anything a
session read from outside reaching memory the fleet treats as trusted. Finished turns go to
Honcho, whose own extraction replaces a flush job of ours; every trusted memory write is mirrored
there at once; `recall` searches the fleet as well as the local index; a new `ask_memory` tool asks
Honcho's dialectic.
The milestone is checked by:
- a turn from a session that read untrusted content never reaches a trusted workspace, and a turn
from a `secret` session reaches no workspace;
- a result from an untrusted workspace taints the session that reads it;
- `recall` still answers from the local index when Honcho is down, and says what it could not
reach;
- a memory write the agent is asked to keep ("remember that …") is in Honcho at once, and undoing
it removes it.
Honcho is not rebuilt from the logs (owner, 2026-09-27). It accumulates; what it holds wrongly is
corrected by writing a contrary fact, which Honcho's dreaming reconciles, or, for a conclusion
`memoryd` wrote itself, by deleting that conclusion.
## 2. Two halves
One spec, two plans, each run and device-checked on its own (owner):
1. **Write path (M5c-1)**: the `brokerd` session-state socket, the speaker on each turn, the
ledger, the sync worker, mirrored memory writes, configuration, `bxctl memory sync`, runbook.
2. **Read path (M5c-2)**: `recall` over the fleet, the `ask_memory` tool, trust labels on other
agents' workspaces.
Projects in memory stay out of M5c (open row in `docs/decisions.md`).
## 3. Workspaces and peers
**Workspaces** (owner, 2026-09-27). Each agent writes to three:
| Workspace | Holds | Read by | Trust | Class |
|---|---|---|---|---|
| its **trusted** one | turns from clean sessions; conclusions mirroring its trusted memory writes | the agent (through `ask_memory`); other agents, as the owner labels it | trusted | `private` |
| its **untrusted** one | turns from sessions that read untrusted content | the agent only (through `ask_memory`) | untrusted, fixed | `private` |
| the **fleet** one, shared by every agent | conclusions from untrusted notes, each agent under its own peer | every agent | untrusted, fixed in code | `private` |
A workspace's class is also its ceiling for writes: nothing above `private` is written to Honcho
(owner). A `public` turn is written as `private`, since `public` records where a result came
from, not a permission. A session that reached `secret` by a turn's end is never synced, to any
workspace (owner).
**Peers** (owner):
- the owner: one peer, the same name in every workspace of the fleet (`[honcho] owner`); observed;
- other people who speak in a channel the agent answers: a peer each, named from their
Mattermost username (a `.` becomes `_`: Honcho allows `[a-zA-Z0-9_-]`); observed;
- the agent: a peer under its own name (`[honcho] agent`), **not observed** (`observe_me = false`).
Its replies are written as messages and serve as context when Honcho reasons about the people
it talks to, but Honcho draws no conclusions about the agent from them: they are the agent's
own output and can restate what the session read. What the agent knows enters Honcho only as
conclusions `memoryd` writes (section 5).
A Honcho session's peers, and who observes whom, are fixed when it is created (`roncho`'s blocking
client cannot change them). A Boxmaker session maps to one Honcho session per workspace, named after
it (in the fleet workspace too, where it holds only the agent's peer and the session's untrusted
notes); when a speaker who is not yet in it speaks, `memoryd` starts the next Honcho session for it,
the same name with `-2`, `-3` and so on, holding every peer so far.
## 4. What goes where
**Turns.** A finished turn is two messages: the speaker's message (the turn's text as `loopd`
logged it) from the speaker's peer, and the agent's final reply from the agent's peer (P23).
Tool calls, tool results and reasoning are not written. A text over 25,000 characters (Honcho's
limit) is written as several messages of at most 25,000, in order.
| The turn's session at its end | Goes to |
|---|---|
| clean, class `public` or `private` | the trusted workspace |
| untrusted (read untrusted content), class `public` or `private` | the untrusted workspace |
| class `secret` | nowhere |
| no single speaker (a burst joined from several people) | nowhere |
| the text holds a known-prefix secret (`sanitize::refuse`'s secret rules) | nowhere |
**Memory writes.** Every write is mirrored as a conclusion the moment it is journalled (owner),
so "remember this" reaches Honcho at once, through the memory tool and its checks, never by
spotting the phrase:
| Journalled edit | In Honcho |
|---|---|
| identity `add` (memory.md or user.md) | a conclusion in the trusted workspace |
| identity `replace` | the old entry's conclusion deleted, the new one created |
| an entry dropped from identity (`remove`, or dropped by `consolidate`) | its conclusion deleted |
| an entry moved out of identity by `consolidate` | nothing: its conclusion is already there |
| a trusted note | a conclusion in the trusted workspace |
| an untrusted note | a conclusion in the fleet workspace |
| a flagged note (a sanitizer flag rule matched) | nothing: it stays local (owner) |
| `bxctl memory undo` of an edit whose conclusions were written | those conclusions deleted; any it had deleted, written again |
A conclusion's observer and observed are the same peer, which is Honcho's own picture of that
peer: a `user.md` entry is the owner's (it is about the owner); a `memory.md` entry or a note is
the agent's. Each carries the Honcho session of the Boxmaker session in which the write was made
(Honcho leaves session-less conclusions out of session-filtered queries, and its automatic
dreaming skips a peer whose newest conclusion has none). `memoryd` never deletes a Honcho session:
that would delete the conclusions stamped with it.
The session logs and `memory/` stay the source of truth for everything local; Honcho is not
rebuilt from them.
## 5. The write path
1. **A turn ends.** `loopd` already sends `SessionUpdated` to `memoryd`. It gains the turn's
number in the session (M5c-1 wire change, section 11).
2. **Decide.** `memoryd` asks `brokerd` for the session's state (section 7) and reads the turn's
records in `loopd`'s log. The turn is untrusted if either says so (`brokerd`'s `untrusted`, or
any `ToolResult` so far labelled untrusted); its class is `brokerd`'s. It decides by the table
of section 4 and appends the decision to the ledger **at once**, so a Honcho outage later
cannot change where the turn goes.
3. **If `brokerd` cannot answer**, nothing is decided. The turn is decided at the next
`SessionUpdated` of any session, or by the worker's retry, from the session's state then; a
late decision can only move a turn towards untrusted or unsynced (P23).
4. **Memory writes** are decided when journalled: the table of section 4, recorded in the same
ledger.
5. **Send.** One worker thread in `memoryd` sends what is decided and not yet sent, oldest first:
for a turn it creates the Honcho workspace, peers and session on first use (section 6), then
adds the two messages as one batch; for a memory write it creates or deletes the conclusion.
Each send is recorded in the ledger with Honcho's ids.
6. **Honcho down** (a connection error, a timeout, a 5xx): the worker waits and tries again,
doubling the wait from 5 s to 10 minutes. It logs the outage once, when it starts, and once
when it ends, not on every try (runbook `honcho-unreachable`).
7. **Honcho refuses** (a 4xx other than 409 on create, which means "exists"): the entry is marked
failed in the ledger, logged with the status and without the text (runbook
`honcho-send-refused`), and the worker moves on. `bxctl memory sync` lists failed entries.
**The ledger**, `memory/honcho.jsonl`, is append-only JSON lines, one per event: `decided` (the
item, the workspace or `none` and why), `sent` (the item and Honcho's ids), `failed` (the item and
the status). What is decided and not sent is the outbox; there is no separate queue. An item is
never sent twice: the worker checks for its `sent` line first, and on start `memoryd` reads the
ledger through. Items are a turn (`session`, `turn`) or an edit (its journal id).
Everything written to Honcho is neutralised (`sanitize::neutralise`); a reply is the model's own
output and was logged as it came, so it is neutralised here, when it leaves.
**The guarantee.** Nothing reaches a trusted workspace unless `brokerd` and `loopd`'s labels both
say the session was clean, and `brokerd` says its class was at most `private`, when the turn
ended.
## 6. Honcho, through `roncho`
`memoryd` uses `roncho` 1.0.0 from crates.io with `default-features = false, features =
["blocking"]` (`["tls"]` when the URL is HTTPS). It adds one crate to the lock file; `cargo deny`
passes (checked 2026-09-27).
- **Every client is built with `.without_config_file()`.** `roncho` otherwise reads
`~/.config/roncho/roncho.toml` and takes an `api_key` from it when none is set.
- One client per workspace (a key is per workspace), shared by the worker and the read path.
- Timeouts: connect 5 s; read ("no bytes") 30 s for writes and searches, 180 s for chat (rift's
dialectic took up to 120 s, and a non-streamed chat sends nothing until it is done).
- `roncho`'s `Status` and `Decode` errors carry up to 4 KiB of the response, which can be memory
text: `memoryd` logs only the error's kind and status, never its `Display`.
- On creation, `memoryd` configures each workspace: the trusted and untrusted ones with reasoning,
summaries and dreaming on (Honcho's defaults); the fleet one holds conclusions only.
- Honcho ids allow `[a-zA-Z0-9_-]`, at most 512 characters. Boxmaker session ids fit; names from
configuration are checked at start.
## 7. `brokerd`'s session state
A new socket, `run/memory-broker/state.sock`, served by `brokerd`, connected to only by
`memoryd` (directory mode 0700, owned by the user both run as, as the other sockets). One message,
`SessionState { session }`, answered with `SessionStateReply { class, untrusted }`. It changes
nothing. A session `brokerd` has not seen answers `private`, trusted: it has made no tool call.
## 8. `recall`
One tool, as today, searching (owner):
| Source | Searched with | Tag on each hit | Class |
|---|---|---|---|
| the local index | FTS5, as today | as today (`identity`, `memory`, `memory-tainted`) | as recorded |
| each other agent's trusted workspace in `[[honcho.read]]` | message search, and a conclusion query for that agent's peer and the owner's | `memory` if the owner labelled it trusted, else `memory-tainted`; the agent's name | the workspace's `class` |
| the fleet workspace | a conclusion query for this agent's peer and each agent's in `[[honcho.read]]` | `memory-tainted`; the writing agent's name | `private` |
The agent's own workspaces are not searched: the local index holds every turn and note of this
agent with exact labels, and would return the same text twice. Each source runs on its own thread
with its own limit (the local index as today; 5 s for each Honcho source); `recall` answers with
what arrived, and a source that did not answer is one line, `<source>: unavailable`. Hits are
ranked within their source and interleaved to `k`, the local index first. Text from Honcho is
neutralised on arrival and size-capped like every recall snippet. Any `memory-tainted` hit marks
the session untrusted, and every hit raises the session's class to its label, as local hits do.
Honcho's search endpoints find messages only, and the fleet workspace holds only conclusions, as
do the mirrored memory writes in another agent's trusted workspace: so `recall` also queries
conclusions (`POST /conclusions/query`, which needs the observer and the observed; for a
conclusion `memoryd` wrote, both are the same peer). `roncho`'s blocking client cannot list a
workspace's peers, so the agents whose fleet notes an agent reads are itself and those in
`[[honcho.read]]`.
## 9. `ask_memory`
A new core tool, `ask_memory { question, workspaces?, reasoning? }`:
- `workspaces`: names from `[honcho]`: `own` (default), `untrusted`, `fleet`, or another agent's
name from `[[honcho.read]]`;
- `reasoning`: `minimal` (default: one search, then the answer) or `low`;
- `question`: at most 2,000 characters.
One workspace chat (`POST /workspaces/{w}/chat`) per named workspace, in parallel, at most two at a
time; workspace chat searches every conclusion and message of the workspace without the model
choosing a peer (it averaged 13 s on rift, over only three calls; peer chat, 32 s over 318). Each
has 150 s. The answer lists each workspace's reply, neutralised and capped, tagged like a `recall`
hit (`memory` for the agent's own trusted workspace and for others the owner labelled trusted,
`memory-tainted` otherwise), or `<workspace>: unavailable`. Labels taint the session as `recall`'s
do. `brokerd`'s wait for this tool is 170 s and `loopd`'s wait on `brokerd` for it 180 s (section
11).
Adding a tool changes the `tools` array: a new epoch for every session when M5c-2 is deployed,
and the baseline is measured again against the 3,000-token budget.
## 10. Configuration and secrets
In `memoryd.toml`; without a `[honcho]` section, `memoryd` behaves exactly as after M5b:
```toml
[honcho]
url = "http://rift.scylla-hammerhead.ts.net:8000"
agent = "boxmaker0"
owner = "kyle"
trusted = "boxmaker0-trusted"
untrusted = "boxmaker0-untrusted"
fleet = "fleet"
[secrets.honcho_trusted] # optional, one per workspace: honcho_trusted, honcho_untrusted,
file = "…" # honcho_fleet, honcho_read_<agent>; sent only when configured
[[honcho.read]]
agent = "boxmaker1"
workspace = "boxmaker1-trusted"
trust = "untrusted" # the default; "trusted" must be written out
class = "private"
```
Keys come from `bxcore::secrets` (M5b). An invalid `[honcho]` section stops `memoryd` at start
with the reason (runbook `honcho-config-invalid`). `docs/egress.md` gains rift: `memoryd` →
the configured URL, for writes, searches and chat.
## 11. Links between the roles
M5c's changes to the table of the M5b spec, section 10:
| Link | Serves | Connects | Path or address | Carries | When the server is down |
|---|---|---|---|---|---|
| Session state (new) | `brokerd` | `memoryd` | `run/memory-broker/state.sock` | `SessionState` → `SessionStateReply` | turns wait undecided (`broker-state-unavailable`) |
| Honcho (new) | Honcho | `memoryd` | `[honcho] url` | `roncho` calls | sends wait (`honcho-unreachable`); `recall` answers from the local index |
| Memory for turns | `memoryd` | `loopd` | as before | `SessionUpdated` gains `turn` | as before |
| Turns | `loopd` | `gatewayd`, `bxctl chat` | as before | `Turn` gains `speaker` | as before |
Wire changes (fixtures change): `Turn` gains `speaker: Option<String>`, which `gatewayd` sets to
the Mattermost username when every post of the burst is from one person, and `bxctl chat` sets
to the owner's name from its configuration; `loopd` logs it on the `User` record (a new optional
field: older logs read as `None`); `SessionUpdated` gains `turn: u64`. A turn whose speaker is
`None` is not synced.
## 12. Runbook
New entries, given as fragments: `honcho-unreachable`, `honcho-send-refused`,
`broker-state-unavailable`, `honcho-config-invalid`. Each names what `bxctl memory sync` shows.
## 13. Testing
Given tests, first, against a fake Honcho server (a small HTTP server in `memoryd`'s test support
that records requests and answers from fixtures taken from rift's OpenAPI document, like
`loopd`'s fake llama server):
- **Decisions**: clean → trusted; untrusted by `brokerd` or by the log → untrusted; `secret` →
none; no single speaker → none; a key in the text → none; `brokerd` down → undecided, then
decided from the later state, never towards trusted.
- **The ledger**: a restart sends nothing twice; an outage sends everything once it ends, in
order; a refused entry is marked failed and the rest go on.
- **Memory writes**: each row of section 4's table, including undo.
- **Peers and sessions**: the agent's peer not observed; a new speaker starts the next Honcho
session; names fit Honcho's pattern.
- **`roncho`**: a `roncho.toml` with an `api_key` in the test's `HOME` is not read.
- **`recall`**: the fleet sources with their tags and classes; a source that times out is one
`unavailable` line and the rest still answer; the agent's own workspaces are not searched.
- **`ask_memory`**: two at a time; the per-workspace limit; labels; unknown workspace names.
- **No `[honcho]` section**: M5b's behaviour, every M5b test unchanged.
- **Attack tests (T37)**: a turn from a session that fetched a page never reaches the trusted
workspace, even when `brokerd` answers late; a fleet hit taints the reading session.
**On the device** (straylight, rift, the owner's Mattermost), for each half:
- M5c-1: a clean turn appears in the trusted workspace, an untrusted one in the untrusted
workspace, a `secret` session's in neither; "remember that …" appears as a conclusion at once,
and `bxctl memory undo` removes it; rift stopped (or unreachable) → the outbox waits and drains
once it is back, sending nothing twice; `bxctl memory sync` shows each state.
- M5c-2: `recall` returns a fleet note written by a second agent (a second `memoryd` with its own
name), tagged `memory-tainted`, and the session turns untrusted; `ask_memory` answers from the
agent's own trusted workspace within its limit; with rift down, `recall` answers from the local
index with `unavailable` lines.
## 14. Changes to the brief and decisions
With this spec, as their own commits:
- `docs/design.md`, Memory items 4 and 5: two workspaces per agent and the fleet workspace; no
rebuild ("corrected by writing a contrary fact"); mirrored memory writes; the peers.
- `docs/decisions.md`: P23 applied as amended here (the fleet workspace for untrusted notes; no
rebuild); rows for the class ceiling, the peers, mirroring, the ledger, and `recall`'s sources.
- `docs/milestones.md`, M5c: its check "Honcho's contents are rebuilt from the logs" goes.
## 15. For the owner's review
1. **The speaker on each turn** (section 11) is new: Honcho needs a peer per message, and today's
turn text carries no name. A burst joined from several people is not synced at all, rather
than guessed at.
2. **Which fleet notes an agent reads** (section 8): its own and those of the agents in
`[[honcho.read]]`, since the client cannot list the fleet's peers. An agent the owner has not
listed contributes to the fleet but is not read.
3. **For `roncho`'s agent** (found 2026-09-27, none blocking): make the config file opt-in (it
supplies an `api_key` today; the README's "reads no environment variables" is wrong); evidence
decoding requires `observer_id` and `observed_id`, which rift's schema lacks; a message's
`configuration` (reasoning per message) is not exposed; the tag `v1.0.0` names `b4e7864` on
`origin` and `1f5ea42` on crates.io.
4. **Rift**: auth is still off; the keys in section 10 are ready for it.