Grok 4.6
@@ -0,0 +1,344 @@
|
|||||||
|
# M5d design: epochs, compaction, and the scheduler
|
||||||
|
|
||||||
|
Status: approved by the owner, 2026-10-03, with section 11's answers. The implementer is Ornith
|
||||||
|
(through `tools/run-plan.sh`), as for M5c.
|
||||||
|
|
||||||
|
It rests on the brief (`docs/design.md`) as changed by P17 to P24, on M5a's files and index
|
||||||
|
(`docs/specs/2026-09-24-m5a-local-memory.md`), and on M5c's Honcho ledger (`docs/specs/2026-09-27-m5c-honcho.md`).
|
||||||
|
Where this document and the brief disagree, section 10 names the brief change.
|
||||||
|
|
||||||
|
## 1. What M5d proves
|
||||||
|
|
||||||
|
Continuity inside a live session, and work that runs when the owner is not talking, without
|
||||||
|
breaking the inference contract or the main slot's cache.
|
||||||
|
|
||||||
|
The milestone is checked by:
|
||||||
|
|
||||||
|
- a scheduled run leaves the main slot's cache intact;
|
||||||
|
- a fact from epoch 1 is recalled in epoch 2, and a memory edit applied in epoch 1 is in epoch 2's
|
||||||
|
baseline;
|
||||||
|
- the tiering job promotes only trusted notes into identity, and its writes are journalled like
|
||||||
|
any other.
|
||||||
|
|
||||||
|
## 2. Two halves
|
||||||
|
|
||||||
|
One spec, two plans, each run and device-checked on its own (owner):
|
||||||
|
|
||||||
|
1. **Epochs and compaction (M5d-1)**: `loopd` writes `N.jsonl` / `N.baseline.json` for `N > 0`;
|
||||||
|
turn numbers and `untrusted()` span every epoch; compaction after a turn whose last `Usage`
|
||||||
|
is at or above a configured fraction of `expect.n_ctx`; the summary call on the background
|
||||||
|
slot; `memoryd`'s Honcho walker reads every epoch file of a session.
|
||||||
|
2. **The scheduler (M5d-2)**: heartbeat and cron as isolated sessions on the background slot;
|
||||||
|
tool-output summaries for tools that opt in; the tiering job.
|
||||||
|
|
||||||
|
## 3. Epochs and compaction (M5d-1)
|
||||||
|
|
||||||
|
### 3.1 Disk
|
||||||
|
|
||||||
|
A session directory already has `0.baseline.json` and `0.jsonl`. After compaction it also has:
|
||||||
|
|
||||||
|
```
|
||||||
|
sessions/<id>/0.jsonl ends with EpochEnd { next: 1, summary }
|
||||||
|
sessions/<id>/1.baseline.json rebuilt from memoryd at compaction
|
||||||
|
sessions/<id>/1.jsonl starts with SessionStart { epoch: 1, … }
|
||||||
|
```
|
||||||
|
|
||||||
|
`LogRecord::EpochEnd { time, next, summary }` is already in `bxcore` and unused. M5d-1 writes it.
|
||||||
|
The old log is kept. `memoryd`'s indexer already walks every `<epoch>.jsonl` under a session
|
||||||
|
(`memoryd::index::logs`).
|
||||||
|
|
||||||
|
`Session::open` loads the highest epoch that has both a baseline file and a log, and the summary
|
||||||
|
carried from the previous epoch's `EpochEnd` (none for epoch 0). `untrusted()` is true if any
|
||||||
|
`ToolResult` in any epoch of the session is labelled untrusted. `turns()` counts `User` records
|
||||||
|
across every epoch, in epoch order then file order; the first `User` of the session is turn 1 for
|
||||||
|
its life.
|
||||||
|
|
||||||
|
### 3.2 Turn numbers and Honcho
|
||||||
|
|
||||||
|
`memoryd` keys a turn as `Item::Turn { session, turn }` and today reads only `0.jsonl`
|
||||||
|
(`honcho::turns::log_path`). After M5d-1 it concatenates every epoch's records in epoch order
|
||||||
|
and uses the same `turn_records` / `turns` helpers. Because `User` records are never written for
|
||||||
|
the carried summary, those helpers stay correct.
|
||||||
|
|
||||||
|
`loopd` reports `SessionUpdated { turn: session.turns() }` as it does now; that number is
|
||||||
|
monotonic.
|
||||||
|
|
||||||
|
### 3.3 When compaction runs
|
||||||
|
|
||||||
|
After a turn has finished (the session is not in the busy set for a live turn), if the current
|
||||||
|
epoch's last `Usage` has `cache_n + prompt_n` at or above `threshold × expect.n_ctx`, `loopd`
|
||||||
|
compacts that session before it accepts another turn on it.
|
||||||
|
|
||||||
|
Default `threshold` is `0.85`. It is a float in `loopd.toml` (`[compact] threshold`) so it can
|
||||||
|
move once this agent's own `Usage` records exist. There is no idle timer: a session that sits
|
||||||
|
for hours under the threshold stays on its current epoch.
|
||||||
|
|
||||||
|
If a live request returns `exceed_context_size_error` (`TurnError::SessionFull` today), `loopd`
|
||||||
|
compacts and retries that user message once in the new epoch. If the retry is still too large,
|
||||||
|
the turn ends `session_full` as today.
|
||||||
|
|
||||||
|
Compaction holds the session busy. Incoming channel turns wait, as they do for any other turn.
|
||||||
|
|
||||||
|
### 3.4 The summary call
|
||||||
|
|
||||||
|
On the **background** slot, with **no tools**, thinking on. The messages are the current epoch's
|
||||||
|
replay (`messages(baseline, records)`), then one user line whose text is `config/compact.md`
|
||||||
|
(next to `system.md`; a starter ships in the repository). The completion's `content` is the
|
||||||
|
summary: tag-neutralised, capped to `[compact] summary_chars` (default 4,000 characters). It is
|
||||||
|
not stuffed into the baseline, so it does not count against the 3,000-token budget.
|
||||||
|
|
||||||
|
The session's live slot (the one on `SessionStart`, main for a Mattermost thread) is unchanged.
|
||||||
|
The next live turn of the new epoch is a cold start on that slot: new prefix.
|
||||||
|
|
||||||
|
If the summary call itself is refused as context-full, `loopd` writes a fallback summary: the
|
||||||
|
epoch's `User` texts, newest last, cut to `summary_chars`. Compaction still proceeds.
|
||||||
|
|
||||||
|
The large prompt shares the server's unified KV pool. That pressure is accepted; running it on
|
||||||
|
the background slot keeps it off a live main-slot request.
|
||||||
|
|
||||||
|
### 3.5 The new epoch's prefix
|
||||||
|
|
||||||
|
1. Append `EpochEnd` to the old log.
|
||||||
|
2. Ask `memoryd` for a fresh memory block; write `<n+1>.baseline.json` (system.md + block + the
|
||||||
|
same core tool schemas). Identity edits applied during the old epoch land here.
|
||||||
|
3. Create `<n+1>.jsonl` with `SessionStart { epoch: n+1, slot: the session's live slot, baseline:
|
||||||
|
hash of the new file }`.
|
||||||
|
4. The in-memory `Session` switches to the new baseline, empty records (the `SessionStart`), and
|
||||||
|
`carry = Some(summary)`.
|
||||||
|
|
||||||
|
`messages` for epoch `n > 0` is: the new system message, then one `ChatMessage::User` whose
|
||||||
|
content is a fixed heading plus `carry` (the heading is built from pieces in code, named in
|
||||||
|
words), then the new epoch's records. That user message is **not** a `LogRecord::User`, so it is
|
||||||
|
not a turn, not reported to Honcho, and not counted by `turns()`.
|
||||||
|
|
||||||
|
Within an epoch the request is still a strict extension of the previous one. A new epoch is a
|
||||||
|
new prefix by definition.
|
||||||
|
|
||||||
|
### 3.6 Trust across compaction
|
||||||
|
|
||||||
|
`Session::untrusted()` reads every epoch file, so a compacted session that had read untrusted
|
||||||
|
content still does. `TurnDone.untrusted` follows that, and `gatewayd` keeps escaping links.
|
||||||
|
`brokerd`'s `broker/sessions/<id>.json` is already per session, not per epoch, and stays the
|
||||||
|
authority for taint on tool calls.
|
||||||
|
|
||||||
|
## 4. Isolated sessions and the scheduler (M5d-2)
|
||||||
|
|
||||||
|
Scheduled work is a session like any other: same core tools, same `brokerd` path, same memory
|
||||||
|
tools. It runs on the **background** slot (`SessionStart.slot = cfg.slots.background`). Live
|
||||||
|
Mattermost and `bxctl chat` sessions keep `cfg.slots.main`. `run_turn` uses the session's slot,
|
||||||
|
not always `slots.main` (today's `turn.rs`).
|
||||||
|
|
||||||
|
The scheduler is a thread inside `loopd serve` (the only role that can start inference). It
|
||||||
|
shares `Context` and the busy set. It never talks to `gatewayd`. Answers stay in
|
||||||
|
`sessions/<id>/`. The owner reads them with `bxctl chat --session <id>` or by opening the log.
|
||||||
|
|
||||||
|
Session ids:
|
||||||
|
|
||||||
|
| Kind | Id | Lives |
|
||||||
|
|---|---|---|
|
||||||
|
| Heartbeat | `heartbeat` | one standing session, compacted like any other |
|
||||||
|
| A cron run | `job-<name>-<yyyymmdd>t<hhmm>` (`[a-z0-9-]`) | one session per run |
|
||||||
|
| Mattermost | `mm-<root>` as today | unchanged |
|
||||||
|
|
||||||
|
A heartbeat or cron turn has no speaker, so Honcho skips it (M5c: no single speaker). The local
|
||||||
|
index still holds the log.
|
||||||
|
|
||||||
|
Priority, when the background slot is contended: a compaction summary first, then a cron run
|
||||||
|
that is due, then heartbeat, then a tool-output summary. A live turn on the main slot is
|
||||||
|
independent. `SlotGate` already serialises requests on one slot.
|
||||||
|
|
||||||
|
### 4.1 Heartbeat
|
||||||
|
|
||||||
|
`[heartbeat]` in `loopd.toml`: `every_s` (0, the default, disables it) and `prompt` (a path next
|
||||||
|
to the config file, default `heartbeat.md`). A starter `config/heartbeat.md` ships in the
|
||||||
|
repository.
|
||||||
|
|
||||||
|
When `every_s` has elapsed since the last heartbeat turn (or since start, if none), the
|
||||||
|
scheduler runs one turn on session `heartbeat` whose user text is the file's contents. Volatile
|
||||||
|
time belongs in that newest message: the scheduler prepends one line `It is <RFC 3339 UTC>.`
|
||||||
|
ahead of the file.
|
||||||
|
|
||||||
|
### 4.2 Cron
|
||||||
|
|
||||||
|
Owner-written files `{home}/jobs/<name>.toml`. `name` is `[a-z0-9-]+`, short enough that
|
||||||
|
`job-<name>-<yyyymmdd>t<hhmm>` fits in a `SessionId` (64 bytes). Unknown keys are errors.
|
||||||
|
`loopd` reads them and cannot write them.
|
||||||
|
|
||||||
|
```toml
|
||||||
|
schedule = "0 7 * * *" # five-field UTC crontab: minute hour day-of-month month day-of-week
|
||||||
|
prompt = "Check overnight mail and write a note of anything that needs the owner."
|
||||||
|
```
|
||||||
|
|
||||||
|
`prompt` is the user text of that run, with the same time line prepended as heartbeat. A new
|
||||||
|
session each run. If a previous run of the same `name` is still busy, this tick is skipped (the
|
||||||
|
schedule does not queue).
|
||||||
|
|
||||||
|
The matcher is a small module in `loopd` (UTC, no cron crate): `*`, a single number, `n-m`,
|
||||||
|
`*/n`. Day-of-week 0 is Sunday. Tested against a table of (schedule, timestamp, due) cases.
|
||||||
|
`loopd` wakes at least once a minute while any job file exists.
|
||||||
|
|
||||||
|
A run ledger `{home}/jobs/runs.jsonl` records `{time, name, session, status}` (`started`,
|
||||||
|
`finished`, `failed`). It is how "previous run still busy" is answered after a restart: a
|
||||||
|
`started` line with no later `finished`/`failed` for that `name` means skip until the process
|
||||||
|
that owns the busy set is this one and the session is not busy. On start, a leftover `started`
|
||||||
|
from a dead process is closed as `failed`.
|
||||||
|
|
||||||
|
Scheduled jobs re-check current grants on every tool call: that is already `brokerd`'s rule.
|
||||||
|
|
||||||
|
### 4.3 Tool-output summaries
|
||||||
|
|
||||||
|
M5a's index records owner text, assistant replies, and tool **calls**; it does not record tool
|
||||||
|
**results**. A fetch's body is therefore invisible to `recall` except as whatever the assistant
|
||||||
|
restated.
|
||||||
|
|
||||||
|
A tool that opts in gets a second record, for the index only:
|
||||||
|
|
||||||
|
```text
|
||||||
|
LogRecord::ToolSummary { time, call, tool_call_id, content, class, untrusted }
|
||||||
|
```
|
||||||
|
|
||||||
|
`messages()` ignores it, so the epoch's prefix does not change. `class` and `untrusted` are
|
||||||
|
copied from the `ToolResult` they summarise. `memoryd` indexes `content` as kind `summary`, same
|
||||||
|
labels; `recall` tags it `memory` or `memory-tainted` as for any other session hit.
|
||||||
|
|
||||||
|
M5d opts in `http_fetch` only. `[compact] summarise = ["http_fetch"]` is the list; unknown names
|
||||||
|
are a config error.
|
||||||
|
|
||||||
|
After a turn that logged an opted-in `ToolResult` with no matching `ToolSummary` yet, the
|
||||||
|
scheduler runs a no-tools completion on the background slot: the user text is
|
||||||
|
`config/summarise.md` plus the result, capped to the tool-result cap. The completion is
|
||||||
|
tag-neutralised, capped to `summary_chars`, and appended as `ToolSummary`. The model call does
|
||||||
|
not hold the session busy; appending the record does (the same log lock as any append). A live
|
||||||
|
turn on that session is never delayed by summarising. Compaction of a session waits until
|
||||||
|
in-flight summaries for it have appended or failed.
|
||||||
|
|
||||||
|
### 4.4 The tiering job
|
||||||
|
|
||||||
|
An ordinary cron job. Nothing in `loopd` special-cases it. A starter `{repo}/config/jobs/tier.toml.example`
|
||||||
|
documents a weekly schedule and a prompt that says: read the listing, then `consolidate` /
|
||||||
|
`add` / `remove` / `note` as needed; promote only trusted notes; do not fetch.
|
||||||
|
|
||||||
|
The listing is the job's user text after the time line, from a new `memoryd` admin request
|
||||||
|
`MemoryCommand::Tiers`: each identity entry and each note, with how many times `recalls/` named
|
||||||
|
it and when it was last found (M5a already writes that log so counts survive a reindex).
|
||||||
|
Untrusted notes are included and marked; the prompt forbids promoting them, and `memoryd` still
|
||||||
|
refuses identity edits from an untrusted session, so a job that fetches a page cannot promote.
|
||||||
|
|
||||||
|
`bxctl memory tiers` prints the same listing for the owner.
|
||||||
|
|
||||||
|
## 5. Config
|
||||||
|
|
||||||
|
New tables on `loopd.toml`. Unknown keys are errors. Defaults disable scheduled work.
|
||||||
|
|
||||||
|
```toml
|
||||||
|
[compact]
|
||||||
|
threshold = 0.85
|
||||||
|
summary_chars = 4000
|
||||||
|
summarise = ["http_fetch"]
|
||||||
|
|
||||||
|
[heartbeat]
|
||||||
|
every_s = 0
|
||||||
|
prompt = "heartbeat.md" # next to this config file
|
||||||
|
```
|
||||||
|
|
||||||
|
`expect.n_ctx` is already required. Depth is `cache_n + prompt_n` of the current epoch's last
|
||||||
|
`Usage`.
|
||||||
|
|
||||||
|
Starter files in the repository, copied like `soul.md` / `system.md`: `config/compact.md`,
|
||||||
|
`config/heartbeat.md`, `config/summarise.md`, `config/jobs/tier.toml.example`.
|
||||||
|
|
||||||
|
## 6. Components
|
||||||
|
|
||||||
|
| Piece | Change |
|
||||||
|
|---|---|
|
||||||
|
| `bxcore` | `LogRecord::ToolSummary`; `MemoryCommand::Tiers` and its reply. `EpochEnd` stays as it is. |
|
||||||
|
| `loopd` | Current epoch on open; `turns` / `untrusted` over every epoch; `run_turn` uses the session's slot; compaction (section 3); scheduler thread (section 4). |
|
||||||
|
| `memoryd` | Honcho reads every epoch file of a session; indexes `ToolSummary`; `Tiers` from `recalls/`. |
|
||||||
|
| `gatewayd` | None. `TurnDone.untrusted` keeps doing the work if `loopd` is right. |
|
||||||
|
| `brokerd` | None beyond using the session's existing state. |
|
||||||
|
| `bxctl` | `bxctl memory tiers`. Heartbeat and job sessions are ordinary `--session` ids. |
|
||||||
|
|
||||||
|
`loopd` still depends only on `bxcore`. No new crate for cron.
|
||||||
|
|
||||||
|
## 7. Testing
|
||||||
|
|
||||||
|
Given tests, first, against the fake llama server and a fake `memoryd`:
|
||||||
|
|
||||||
|
- **Open / resume**: a directory with `0` and `1` loads epoch 1 and the carry from `0`'s
|
||||||
|
`EpochEnd`; a missing carry on epoch 1 is an error (the file exists but has no `EpochEnd`).
|
||||||
|
- **Turns**: two `User` records in epoch 0 and one in epoch 1 → `turns() == 3`; Honcho
|
||||||
|
`turn_records` of turn 3 is the epoch-1 user.
|
||||||
|
- **Trust**: an untrusted `ToolResult` in epoch 0, none in epoch 1 → `untrusted()` is true
|
||||||
|
(the open item in `docs/decisions.md`).
|
||||||
|
- **Prefix**: property test per epoch; after compaction the first request of epoch 1 is the new
|
||||||
|
baseline plus carry, not an extension of epoch 0.
|
||||||
|
- **Trigger**: last `Usage` at 84% of `n_ctx` leaves the epoch; at 85% the next turn waits on
|
||||||
|
compaction (the fake server records a no-tools request on the background slot) and then runs
|
||||||
|
on epoch 1. An idle gap with no turn does nothing.
|
||||||
|
- **SessionFull**: a context-full error on epoch 0 compacts and retries once.
|
||||||
|
- **Slot**: a `heartbeat` turn's request carries `id_slot = background`; a `mm-*` turn still
|
||||||
|
carries `main`. A heartbeat turn in between two main-slot turns of another session leaves the
|
||||||
|
fake server's recorded cache prefix of the main session intact.
|
||||||
|
- **Heartbeat / cron**: `every_s = 1` fires; `schedule = "* * * * *"` fires; a still-busy run of
|
||||||
|
the same name is skipped; a leftover `started` after a restart is `failed`; unknown keys in a
|
||||||
|
job file refuse start; a due run with no speaker is not a Honcho item (existing skip).
|
||||||
|
- **ToolSummary**: an `http_fetch` result produces a `ToolSummary` with the same labels;
|
||||||
|
`messages()` is unchanged; `recall` finds the summary text and taints when `untrusted`.
|
||||||
|
- **Tiers**: a trusted note with hits is listed; an untrusted note is listed and marked; a
|
||||||
|
tiering session that is untrusted cannot `add` to `memory.md` (existing M5a rule).
|
||||||
|
|
||||||
|
**On the device** (straylight, Ornith slot 0 for the live session, slot 2 for background), with
|
||||||
|
`[compact] threshold` lowered so the check does not need 220k tokens:
|
||||||
|
|
||||||
|
- M5d-1: a `bxctl` session writes a distinctive fact into `memory.md`, continues until
|
||||||
|
compaction, and the next turn's baseline contains the fact; `recall` of a phrase from epoch 0
|
||||||
|
still hits; `TurnDone.untrusted` stays true across the epoch change after a tainted recall.
|
||||||
|
- M5d-2: a heartbeat turn runs on the background slot; a following main-slot turn of a second
|
||||||
|
session shows a cache hit on its own prefix (`cache_n` near its previous total). A
|
||||||
|
`jobs/tier.toml` for the check promotes only a trusted note; the journal has the edit; an
|
||||||
|
untrusted note is still a note.
|
||||||
|
|
||||||
|
Heartbeat and cron answers are not posted to Mattermost. The check reads the session log.
|
||||||
|
|
||||||
|
## 8. Runbook
|
||||||
|
|
||||||
|
New entries, each with a pointer from the message:
|
||||||
|
|
||||||
|
- `session-compact-failed`: the summary call and the fallback both failed; the session stays on
|
||||||
|
the old epoch and the next turn is `session_full` if it would not fit.
|
||||||
|
- `jobs-invalid`: a job file failed to load; `loopd` does not start.
|
||||||
|
- `heartbeat-prompt-unreadable`: the prompt file exists but cannot be read; heartbeat is skipped
|
||||||
|
and the line is logged once until it reads.
|
||||||
|
|
||||||
|
## 9. Out of scope
|
||||||
|
|
||||||
|
- Subagents (M6) share the background slot; the scheduler's queue is the one they will join.
|
||||||
|
- Optional delivery of a cron answer to Mattermost.
|
||||||
|
- Session-scoped approvals (open row in `docs/decisions.md`).
|
||||||
|
- Changing `Item::Turn` to carry an epoch: turn numbers are monotonic instead.
|
||||||
|
- A cron crate; nicknames (`@daily`); seconds fields; non-UTC zones.
|
||||||
|
|
||||||
|
## 10. Changes to the brief and decisions
|
||||||
|
|
||||||
|
With this spec, as their own commits:
|
||||||
|
|
||||||
|
- `docs/design.md`, Inference contract 4: compaction runs after a turn, when
|
||||||
|
`cache_n + prompt_n` is at or above a configured fraction of `n_ctx` (default 0.85), while the
|
||||||
|
session is not in another turn. It does not run because the session has been quiet.
|
||||||
|
- `docs/design.md`, Memory 3 / M5a index: `recall` also indexes `ToolSummary` records of tools
|
||||||
|
that opt in, labelled as the tool result was. Raw tool results stay unindexed.
|
||||||
|
- `docs/decisions.md`: close "A session's trust across compaction" (section 3.6); row for
|
||||||
|
monotonic turn numbers; row for scheduled sessions staying off Mattermost; P20's "M5c
|
||||||
|
scheduler" wording already moved to M5d by P24.
|
||||||
|
|
||||||
|
## 11. The owner's answers (2026-10-03)
|
||||||
|
|
||||||
|
1. **Heartbeat delivery**: answers stay in the session log. They are not posted to Mattermost.
|
||||||
|
2. **Cron**: five-field UTC crontab, `{home}/jobs/<name>.toml`, a new session each run, a tick
|
||||||
|
skipped when that job is still live.
|
||||||
|
3. **Compaction trigger**: 85% of `expect.n_ctx` from the last `Usage`, configurable. Retune
|
||||||
|
from this agent's `Usage` records once they exist. A quiet session is not a reason to
|
||||||
|
compact (the owner may have gone to sleep).
|
||||||
|
4. **Tool summaries**: on the background slot; a `ToolSummary` log record with the result's
|
||||||
|
labels, FTS-indexed; `http_fetch` opts in.
|
||||||
|
5. **Split**: M5d-1 (epochs and compaction), then M5d-2 (the scheduler), two plans.
|
||||||
Reference in New Issue
Block a user