boxmaker/docs/specs/2026-10-04-m7-split-deployment.md

M7 design: split deployment

Status: draft, 2026-10-04, awaiting the owner's answers in section 11. The implementer is Ornith (through tools/run-plan.sh), as for M6. No plan is written until this spec is approved.

It rests on the brief (docs/design.md, Network isolation, Roles, Authority 4, Target environment), on the 2026-09-18 decisions (rootless Podman, separate users per role, broker.sock and admin.sock enforced with mounts in M7), and on what already exists: inferproxy, the M3b tool image and egress proxy, flake.nix packages and deploy/tools-image.nix. P22 / M8 (a fleet module and make install) stays out of this milestone.

Where this document and the brief disagree, section 10 names the brief change. There is none yet.

1. What M7 proves

Each role already has a narrow binary. Until now they all run as the owner's user on the host. M7 puts each role in its own rootless container so a compromise of one process is contained by namespaces, mounts and users, not only by code.

The milestone is checked by:

• from inside the loopd container, requests to the internet, the tailnet and the host all fail
• a tool container without a host grant cannot reach the tailnet (already true of M3b; repeated from the new layout)
• loopd cannot open admin.sock; bxctl on the host still can

2. What already holds

Piece	Today
Tool containers	Fresh rootless Podman per call, image from Nix, named by digest, --network=none unless the grant names hosts, egress proxy for http_fetch
inferproxy	Unix socket in, one TCP upstream to the host llama-server
loopd	No credentials, no network in process terms; it still runs with the host network namespace
Sockets	Pair directories under $HOME/run/; both ends are the same Unix user, so the split is code (forbidden)
flake.nix	Builds every binary and the tool image; no NixOS module (P22 / M8)
Production on straylight	Host processes, owner's user, Mattermost bot untouched by this spec's device check

M7 does not rewrite the IPC, grants, audit, or inference contract. It changes where the binaries run and who can see which sockets and files.

3. Isolation model

One long-lived container per role: loopd, brokerd, gatewayd, memoryd, inferproxy. Tool containers stay as they are (one shot, from brokerd). bxctl stays on the host: it is the owner's CLI, not a role.

Each role container:

• runs as a dedicated user on the host (the 2026-09-18 decision: separate users per role)
• is started rootless Podman (--userns=keep-id so the role user owns its files)
• is --read-only with tmpfs for /tmp and the lock/socket directories it must write
• drops capabilities (--cap-drop=all, no-new-privileges)
• mounts only the directories that role may see, with the mode that role needs

Network per role (brief, Network isolation):

Role	Network
loopd	--network=none. Loopback only. Four Unix sockets.
inferproxy	Outbound to the configured llama-server host:port and nothing else
gatewayd	Outbound to Mattermost. No listening TCP port
memoryd	Outbound to Honcho and nothing else
brokerd	No outbound internet in M7. It talks to the container runtime, to its sockets, and (for consult) to a loopback URL already required to be 127.0.0.1 or localhost. How loopback consult is reached from inside the container is question 4
tool containers	Unchanged: none, or the per-call egress proxy

4. Sockets and mounts

Peer authentication stays directory permissions, now across users.

Each pair of roles shares one directory that only those two users can enter. The listening socket lives there, mode 0600, owned by the listener. The other role's container mounts that directory. A third role does not get the directory.

The pairs that already exist under run/:

Directory	Listener	Client	M7 extra
run/loop/	loopd (loop.sock)	gatewayd, bxctl (host)	bxctl is in the owner's group or ACL for this dir
run/loop-broker/	brokerd (broker.sock)	loopd	loopd does not mount run/owner-broker/
run/owner-broker/	brokerd (admin.sock)	bxctl (host)	not mounted into loopd
run/infer/	inferproxy	loopd
run/loop-memory/	memoryd	loopd
run/broker-memory/	memoryd	brokerd
run/memory-broker/	brokerd (state.sock)	memoryd
run/owner-memory/	memoryd (admin.sock)	bxctl (host)	not mounted into loopd
run/gateway/	brokerd (gateway.sock), when [gateway] is set	gatewayd

Files:

Tree	Who writes	Who reads
sessions/	loopd	memoryd (reindex, Honcho send), bxctl
memory/	memoryd	loopd (baseline copies at epoch start, through the socket, as today)
grants/	the owner, from the host	brokerd (read-only)
audit/	brokerd	bxctl audit verify on the host
broker/sessions/	brokerd	brokerd
config *.toml, system.md, soul.md, heartbeat.md, compact.md	the owner	the role that loads them

loopd never sees grants/, audit/, or admin.sock. That is the mount half of the 2026-09-18 socket split.

5. Images and how they start

flake.nix already builds one package of every binary. M7 adds, under deploy/:

• a rootless image per role or one image holding all role binaries (question 2), still built by Nix, loaded with podman load, named by digest
• a host ACL file for the tailnet node (question 5): deploy/acl.hujson or the owner's existing Tailscale ACL repo, copied or pointed at from here so the gate can see it
• unit files or a small supervisor script that starts the five role containers with the mounts in section 4 (question 3)

No NixOS module in M7 (P22 / M8). No make install for Omarchy in M7.

Secrets stay bxcore::secrets: systemd credential, env, or owner-only file. Inside a container the credential backend needs the host to pass the credential in; the env and file backends already work if the file is mounted mode 0600. Default remains credential.

One instance per daemon still holds: the existing lock files live on the mounted run/ dirs.

6. Components

Piece	Change
loopd	none required in Rust if --network=none plus mounts are enough. A start failure when a required socket dir is missing already has runbook entries; add any new pointer M7 creates
brokerd	Podman is invoked from inside the brokerd container to start tool containers (nested rootless, question 1). consult stays loopback HTTP/1.1
gatewayd	none in Rust
memoryd	none in Rust
inferproxy	none in Rust; its upstream is still host llama-server
bxctl	stays on the host; needs group/ACL on loop.sock and admin.sock dirs
bxcore	none
deploy/	role image(s), start units or script, tailnet ACL policy, example mounts
docs/egress.md	no new run-time destination. consult is still loopback until a later live URL

7. Testing

Given tests first, against a fake podman / a recorded argument list where the start script is code, and against permission checks on a scratch tree of run/ dirs:

• loopd has no net: a helper in the loopd image (or busybox wget) to 1.1.1.1:443, to the tailnet Mattermost host, and to the host's llama-server port, all fail. The four Unix sockets still work.
• admin.sock is not in the loopd mount: opening it from the loopd container is ENOENT or EACCES. bxctl approvals from the host still lists.
• broker.sock is in the loopd mount: a tool request still reaches brokerd.
• Tool without hosts: shell or http_fetch with no host grant cannot reach the tailnet (repeat of M3b from the new brokerd container).
• inferproxy: loopd still completes a self-test through the socket.

On the device (straylight, private $HOME, production daemons untouched, slot 1 left busy):

• start the five role containers from deploy/
• from inside loopd, the three negative probes in the first bullet fail
• bxctl chat --session m7-live --no-thinking completes one turn on slot 0
• a tool call with no host grant cannot reach the tailnet
• production Mattermost bot stays down for this check

8. Runbook

New entries, each with a pointer from the message, for every start failure M7 adds. Candidates:

• a role container cannot bind its socket (directory missing or wrong user)
• nested Podman from brokerd is unavailable
• loopd is missing one of its four sockets at start

Existing entries (brokerd-start-failed, secret-unavailable, consult-not-configured, …) keep their text. If a path in an entry assumes host processes, the task that adds the unit updates the check steps.

9. Out of scope

• P22 / M8: NixOS module, dist/ units for Omarchy, fleet of agents, crossbar
• A live consult URL off-host (own egress row, grant, secret, later)
• HTTPS for consult
• Cloud-led sessions
• Changing the inference contract or slot layout
• Moving bxctl into a container
• Replacing rootless Podman

10. Changes to the brief and decisions

None required if section 11 stays inside the brief. The 2026-09-18 rows already say M7 enforces the socket split with mounts and gives each role its own user.

11. The owner's answers (open)

1. Nested Podman. brokerd today starts tool containers as the owner's user. In M7 it runs inside a container. Is nested rootless Podman (socket from the host, or Podman-in-Podman) acceptable, or does brokerd stay a host process that only the tools and loopd move?
2. Images. One image per role, or one image with every role binary and a different command?
3. How they start on straylight. systemd user units wrapping podman run, Podman quadlets, or a script in deploy/ the device check invokes? (NixOS module is M8.)
4. consult loopback from the brokerd container. slirp4netns/pasta localhost is not the host's 127.0.0.1. Keep consult on host-gateway loopback, put the dummy in the brokerd netns, or leave brokerd on the host until a live URL exists?
5. Tailnet ACL source of truth. A file in deploy/ that the owner's Tailscale ACL is generated from, or a pointer plus a checked-in copy of the policy this host should have?
6. Production cutover. Device check on a private $HOME only in M7, or also move the live Mattermost bot into the new layout as part of the milestone?
7. Split. One plan, or M7a (loopd --network=none + socket mounts, still one user) then M7b (users, images, ACLs)?