From ab47840285680fa36ef903fcc74f9b576574b064 Mon Sep 17 00:00:00 2001 From: "K. Isom" Date: Fri, 2 Oct 2026 20:51:18 -0700 Subject: [PATCH] docs: compaction at 85% of n_ctx, and ToolSummary in the index Co-Authored-By: Grok 4.6 --- docs/decisions.md | 4 +++- docs/design.md | 11 ++++++++--- docs/milestones.md | 4 ++-- 3 files changed, 13 insertions(+), 6 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 55c8956..6645dc3 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -6,6 +6,7 @@ Newest first. A decision that changes `docs/design.md` lands in the same commit | Date | Decision | Reason | |---|---|---| +| 2026-10-03 | M5d design (spec `docs/specs/2026-10-03-m5d-scheduler.md`, approved): two halves, epochs and compaction then the scheduler; compaction after a turn at 85% of `n_ctx` (configurable), never because the session has been quiet; turn numbers monotonic across epoch files; `untrusted()` reads every epoch; heartbeat and cron on the background slot, answers in the session log only; `http_fetch` summaries indexed as `ToolSummary` with the result's labels; the tiering job is an ordinary cron job that may call `memory`. | Owner's answers, 2026-10-03 (spec section 11). Closes the open row on trust across compaction. | | 2026-09-28 | One Rust toolchain everywhere: 1.98.1, which titan and talos run, pinned in the flake through `rust-overlay` for the Nix build and the dev shell; `rust-version` rises from 1.95 to 1.98. | Owner's decision, 2026-09-27. Plans are checked on titan and run on straylight or talos; a clippy difference between hosts stopped an M5a task (T34). nixos-26.05's own Rust stays 1.95.0, so the toolchain comes from the overlay rather than a newer nixpkgs, which host configs that follow `nixpkgs` would undo. | | 2026-09-27 | M5c design (spec `docs/specs/2026-09-27-m5c-honcho.md`, approved): the owner and other people are observed Honcho peers, the agent's peer is not; every trusted memory write is mirrored as a conclusion at once (a `user.md` entry as the owner's, a `memory.md` entry or note as the agent's), and undo deletes it; a turn without complete speaker information is not synced; `memoryd` records each decision and send in an append-only ledger, `memory/honcho.jsonl`, which is also the outbox; `recall` searches the local index, other agents' trusted workspaces and the fleet, never the agent's own workspaces; `ask_memory` asks each named workspace by workspace chat, `minimal` by default; API keys per workspace from `bxcore::secrets`. M5c is planned and run in two halves, the write path first. | Owner's decisions, 2026-09-27, grounded in `roncho` 1.0.0 (checked against its requirements) and Honcho 3.2.0 (source read at `v3.2.0`, and rift's live metrics). | | 2026-09-27 | M5b settles three open items. An invalid-arguments denial carries its reason (`ToolResponse::Denied.detail`, one line, 300 characters at most). `loopd` neutralises the model's chat-template tags when text from outside is first logged (user messages, tool results after their cap, the memory block), checked against the tokenizer by the self-test in two `/tokenize` requests. The secret store is `bxcore::secrets`, behind the `secrets` feature, for every role but `loopd`. | M5b spec, sections 4, 5 and 8; landed 2026-09-27, device-checked on straylight (review in `docs/implementer-log.md`). The self-test's per-tag requests overran `inferproxy`'s burst on the device and were batched (review finding 10). | @@ -78,7 +79,6 @@ Newest first. A decision that changes `docs/design.md` lands in the same commit | Whether cloud-led sessions are ever allowed, and for which data classes. | M6 | | Session-scoped and standing approvals for tools other than memory: answering an approval with `!approve 42 session` (the same pattern for the rest of the session) or `!approve 42 always` (standing), words after the id as `!deny 42 ` has (owner's syntax, 2026-09-24). To settle: what "session" covers (this call's tool and path or host, or the whole grant it matched); and where "always" writes, since `brokerd` cannot write grants (a proposed grant for the owner, or a separate directory of chat-approved grants). A temporary grant is new authority (the grants are the owner's files, and `brokerd` cannot write them), so it needs its own design. Owner's point, 2026-09-24: a harness should not ask a stream of questions. | after M5 | | Projects in memory. Honcho can separate projects by a workspace per project (hard walls, shared naturally by every agent on the project, but Honcho's picture of the owner splits per workspace) or by scopes inside one workspace (one picture, recall filtered by project; semantics not yet read). Either way a project is a `memoryd` setting: its channels and threads, its workspace or scope, `write` and `trust`. A shared project workspace is written by several agents, so its trust is the owner's call per project. Test both on rift before choosing. Owner's idea, 2026-09-24. | M5c or later | -| A session's trust across compaction. From M5b (plan task 09) `loopd` reads whether a session has read untrusted content from its log: true once any tool result in `sessions//0.jsonl` is labelled untrusted, and it never goes back. Compaction starts a new epoch file, so the state must be carried into the new epoch (a record at its head, or `untrusted()` reading every epoch), or a compacted session would read as clean and `gatewayd` would leave its links live. | the milestone that adds compaction | ## Applied to the design brief @@ -114,6 +114,8 @@ Later changes to the brief: | 2026-09-24 | P20, Milestones: M5 becomes M5a (local memory in `memoryd`), M5b (Honcho) and M5c (the scheduler and the tiering job); the embedding-index M5b goes. | Owner's approval, 2026-09-24; see the row of that date under Decided. | | 2026-09-24 | P21, Memory 1, 2, 6, 7, State and Milestones: memory is identity (`soul.md`, `memory.md`, `user.md`, in the prompt, edited only by clean sessions) and experience (`notes.md` and session logs, recalled with a source tag: `identity`, `memory`, `memory-tainted`); writes need no approval; `note` in any session; an untrusted session's identity `add` becomes a note; a check on every write, a journal and undo; the policy of the memory tools is built in. | Owner's approval, 2026-09-24; see the rows of that date under Decided. | | 2026-09-27 | P23, as amended by the M5c spec (`docs/specs/2026-09-27-m5c-honcho.md`), Memory 4 and 5: per agent a trusted and an untrusted Honcho workspace for turns (the speaker's message and the final reply); untrusted notes, as messages, to one fleet workspace every agent shares and reads, always untrusted; nothing above `private`, and never a `secret` session; the gate is `brokerd`'s session state, asked over a read-only socket when the turn ends, with `loopd`'s labels as a second check; no rebuild from the logs. | Owner's decisions, 2026-09-25 (P23) and 2026-09-27 (the amendments). The fleet workspace keeps one agent's raw untrusted turns out of the others' dialectic (Honcho cannot delete a message, and its dialectic filters only by session); rebuild is dropped because Honcho accumulates and is corrected by contrary facts. | +| 2026-10-03 | P25, Inference contract 4: compaction runs after a turn when `cache_n + prompt_n` is at or above a configured fraction of `n_ctx` (default 0.85), while the session is not in another turn. A quiet session stays on its current epoch. | Owner's approval, 2026-10-03; M5d spec `docs/specs/2026-10-03-m5d-scheduler.md`. An idle timer would compact while the owner slept. | +| 2026-10-03 | P26, Memory 3: the local index also holds `ToolSummary` records for tools that opt in, labelled as the tool result was. Raw tool results stay unindexed. | Owner's approval, 2026-10-03; M5d spec. A fetch's body is otherwise invisible to `recall`. | ## Proposed changes to the design brief diff --git a/docs/design.md b/docs/design.md index e0d07e4..fe33971 100644 --- a/docs/design.md +++ b/docs/design.md @@ -98,8 +98,10 @@ ordinary outbound networking can reach the whole tailnet and the internet as the `tool_calls`) and replays it unchanged. Ornith's template keeps every thinking block, so dropping or editing one changes the prefix. 3. **No in-place pruning.** Tool results are size-capped when first appended, never trimmed later. -4. **Compaction is an epoch change.** It happens only when the session is idle, writes a summary, - and starts a new epoch whose prefix is the baseline plus the summary. The old log is kept. +4. **Compaction is an epoch change.** After a turn, when that turn's `cache_n + prompt_n` is at + or above a configured fraction of `n_ctx` (default 0.85), it writes a summary and starts a new + epoch whose prefix is the baseline plus the summary. The session is not in another turn when + this runs. The old log is kept. A quiet session stays on its current epoch. 5. **Pinned slots, not reserved.** Every request carries `id_slot`, and a session always uses the same slot: moving it costs a full re-read. Slot assignment is configuration. With the two slots deployed today, main sessions (one per Mattermost thread) share one slot, and subagents and @@ -198,7 +200,10 @@ the inference contract, and it must not become a way for injected text to persis Results are appended like any tool result, so the cache prefix is untouched. 3. **Local index.** SQLite FTS5 over the session logs and `memory/notes.md`, kept by `memoryd` and rebuildable with `bxctl reindex`. It needs no network and no model, so it still answers - when Honcho is down. `memoryd` counts `recall` hits per note and when each was last found. Retrieval sits behind a `Retriever` trait. + when Honcho is down. Session logs contribute the owner's messages, the assistant's replies, + tool calls by name and arguments, and `ToolSummary` records for tools that opt in (a web + fetch), labelled as the tool result was; raw tool results are not indexed. `memoryd` counts + `recall` hits per note and when each was last found. Retrieval sits behind a `Retriever` trait. 4. **Honcho.** The fleet's memory service; it and the models it calls run on the owner's hosts. Each Boxmaker writes finished turns (the speaker's message and its own final reply, from their own peers) to two workspaces of its own: a trusted one for turns from sessions that saw no diff --git a/docs/milestones.md b/docs/milestones.md index b532496..7d112ea 100644 --- a/docs/milestones.md +++ b/docs/milestones.md @@ -13,10 +13,10 @@ One session per milestone. The design brief is `docs/design.md`. | M3b | `brokerd` runner | Rootless container runner, four tools (read file, write file, sandboxed shell, HTTP fetch with host allowlist), egress proxy | A container with no network grant cannot reach the network; fetch refused for hosts outside the grant | | M4a | `gatewayd` + Mattermost conversations | Bot connection over TLS, user-ID allowlist, threads as sessions, bursts as one turn, typing while working, catching up after downtime, the secret store, no listening port | A direct message answered on the real server; a non-allowlisted user gets no response | | M4b | Approvals over Mattermost | `gateway.sock`, approval posts, answers by `!approve`/`!deny` in the thread, approvers list, long approvals in parts | An `ask` call approved and refused from Mattermost; the audit log names the post | -| M5a | Local memory | `memoryd` without a network: the three memory files served to `loopd` once per epoch, the `memory` tool (`note`, and add, replace, remove, `consolidate` for identity) without questions and its check, `memory/notes.md`, the journal and `bxctl memory log`/`undo`, FTS5 `recall` over session logs and notes with source tags, `recall` hit counts, `bxctl reindex` | A fact from one session is recalled in another; an entry moved by `consolidate` is found by `recall`; a write from a session that read untrusted content becomes an untrusted note, never identity; a memory edit changes the prompt only in the next session (next epoch once M5c adds epochs); the baseline with the owner's files stays under 3,000 tokens | +| M5a | Local memory | `memoryd` without a network: the three memory files served to `loopd` once per epoch, the `memory` tool (`note`, and add, replace, remove, `consolidate` for identity) without questions and its check, `memory/notes.md`, the journal and `bxctl memory log`/`undo`, FTS5 `recall` over session logs and notes with source tags, `recall` hit counts, `bxctl reindex` | A fact from one session is recalled in another; an entry moved by `consolidate` is found by `recall`; a write from a session that read untrusted content becomes an untrusted note, never identity; a memory edit changes the prompt only in the next session (next epoch once M5d adds epochs); the baseline with the owner's files stays under 3,000 tokens | | M5b | Hardening | The sanitizer for memory writes, from the prompt-injection research of 2026-09-24: refuse only bytes that are never legitimate (invisible and format characters, bidirectional overrides, tag characters, private use, noncharacters, controls other than newline and tab) and known-prefix secrets; flag attack-shaped text (an identity write becomes a note, the reason journalled); chat-template tags neutralised to a visible look-alike wherever text enters the model's context or memory, the list checked against the tokenizer by the self-test; image links escaped in what `gatewayd` posts; a reason in invalid-arguments refusals; the secret store (systemd credential, environment, owner-only file) out of `gatewayd`, so every role can load its secrets at startup, failing closed | A tag in a fetched page reaches the model escaped; a write holding an invisible character or a key is refused; an identity write shaped like an attack becomes a note; a memory about code (a hash, a path, a fence, a command) is written unchanged; an image link in a reply is posted as text; a role whose secret is missing or readable by others does not start, and says which | | M5c | Honcho | `memoryd` syncs finished turns through `roncho` to two workspaces of its own: the speaker's message and the agent's final reply, to the trusted workspace while the session is clean and at most `private` at the turn's end (asked of `brokerd` over a socket), to the untrusted one otherwise, never for a `secret` session; every trusted memory write mirrored as a conclusion at once; untrusted notes to the fleet workspace, as messages; `recall` over the local index, other agents' trusted workspaces and the fleet; `ask_memory` over named workspaces in parallel; keys from the secret store. Planned and run in two halves: the write path, then the read path (spec `docs/specs/2026-09-27-m5c-honcho.md`) | A turn from a tainted session never reaches the trusted workspace, and a `secret` session's reaches none; a result from an untrusted workspace taints the session; `recall` still answers from the local index when Honcho is down; a memory write is in Honcho at once, and undoing it removes it | -| M5d | Scheduler and compaction | Heartbeat and cron on their own slot in isolated sessions; compaction at idle, starting a new epoch from the baseline and a summary; summaries of tool output for the tools that opt in (a web fetch, say), indexed with the output's label; the tiering job, which decides what stays at which tier | A scheduled run leaves the main slot's cache intact; a fact from epoch 1 is recalled in epoch 2, and a memory edit reaches the next epoch; the tiering job promotes only trusted notes into identity, and its writes are journalled like any other | +| M5d | Scheduler and compaction | Heartbeat and cron on their own slot in isolated sessions; compaction after a turn at 85% of `n_ctx` (configurable), starting a new epoch from the baseline and a summary; summaries of tool output for the tools that opt in (a web fetch, say), indexed with the output's label; the tiering job, which decides what stays at which tier | A scheduled run leaves the main slot's cache intact; a fact from epoch 1 is recalled in epoch 2, and a memory edit reaches the next epoch; the tiering job promotes only trusted notes into identity, and its writes are journalled like any other | | M6 | Subagents + cloud consult | Subagents on their slot returning summaries; consult tool gated by data-class grants with full payload log | A tainted session is refused; payload log matches what was sent | | M7 | Split deployment | Each role in its own rootless container; `loopd` with no network; socket volume permissions; egress proxy for tools; host ACL policy in `deploy/` | From inside the `loopd` container, requests to the internet, the tailnet and the host all fail; a tool container without a host grant cannot reach the tailnet |