boxmaker/docs/plans/M3b/12-brokerd-egress.md

M3b task 12: the egress proxy's lifecycle

Branch: m3b (run git switch m3b; git status --short must be empty, otherwise stop) Commit subject: brokerd: start and remove the egress proxy for http_fetch

Goal

For an http_fetch call, the tool's container still has no network. Before it runs, brokerd makes a directory for this call, starts the egress proxy (task 08) in its own container with a network and that directory mounted, and waits for the proxy's socket. The tool's container gets the same directory. Afterwards, on every path, the proxy's container and the directory are removed. Spec section 6, "http_fetch".

Files

• Copy: crates/brokerd/tests/container_egress.rs
• Modify: crates/brokerd/src/container.rs, docs/implementer-log.md

Interfaces (added to container.rs)

pub const EGRESS_WAIT: Duration = Duration::from_secs(5);

impl Podman {
    /// The same runtime with another wait for the proxy's socket, for tests.
    pub fn with_egress_wait(self, egress_wait: Duration) -> Podman;
}

Podman gains a private field egress_wait, set to EGRESS_WAIT by new.

run, now

After step 2 of task 11 (name, input, limit):

• spec.egress() is None → exactly as before: tool_args(…, None) and run_container.
• spec.egress() is Some(hosts) → start the proxy (below), and on success call run_container with tool_args(spec, &runner, &name, Some(&dir)).

run_container (from task 11) does steps 3 to 7 for any argument list, so the only change to run is this choice. Put the proxy's start in its own function (start_egress), returning the guard or the error.

Starting the proxy: every step and exit

Use a guard: a private struct holding the proxy's container name (<name>-egress) and the directory, whose Drop runs podman rm -f <name>-egress (the helper from task 11) and then remove_dir_all(dir) (log an error other than NotFound). Create the guard first, before anything can fail, so that every return below cleans up by dropping it, and keep it alive until the tool's container has finished.

1. dir = egress_dir.join(&name). Make egress_dir (and its parents) with mode 0700 (DirBuilder::new().recursive(true).mode(0o700)), then set its mode to 0700 anyway (it may have existed). If dir already exists, it was left by a crash: remove_dir_all it (a NotFound is fine). Then create dir with mode 0700, not recursively. Any failure → log brokerd: cannot make {dir}: {e} + "\n" + RUNBOOK, return Err(RunError::Unavailable(CANNOT_START)).
2. Run podman with podman::egress_args(&runner, &name, &dir, hosts), standard input and output null, standard error piped, with .output(). It does not succeed → log brokerd: podman could not start {name}-egress: {stderr} + "\n" + RUNBOOK, return Unavailable(CANNOT_START). It cannot be started at all → the same log as task 11 step 3, and the same answer.
3. Wait until dir.join("egress.sock") exists, checking every 20 ms, for at most egress_wait. It does not appear → log brokerd: {name}-egress did not make its socket within {ms} ms + "\n" + RUNBOOK, return Unavailable(CANNOT_START).
4. Return the guard. Run the tool's container as in task 11. Whatever it answers, the guard is dropped after it, removing the proxy and the directory.

So for a successful call the fake sees exactly three podman calls: run -d … (the proxy), run --rm -i … (the tool), rm -f <name>-egress. For a tool past its time limit it sees the proxy's run, the tool's run, kill <name>, rm -f <name>, then rm -f <name>-egress.

Steps

• 1. Copy. git switch m3b, then cp docs/plans/M3b/files/crates/brokerd/tests/container_egress.rs crates/brokerd/tests/
• 2. See it fail. cargo test -p brokerd --test container_egress. Expected: it does not compile (with_egress_wait does not exist).
• 3. Write the code. Run cargo fmt --all. container.rs must stay under 500 lines.
• 4. See it pass. cargo test -p brokerd --test container_egress --test container. Expected: 6 and 11 passed. Run them ten times; they must pass every time.
• 5. Walk the exits. For each of the four steps, say how the guard cleans up after it.
• 6. Run the gate. make gate. Expected last line: gate: ok.
• 7. Log and commit. git add crates/brokerd docs/implementer-log.md Cargo.lock && git commit

Done when

• Both suites pass ten times running; make gate prints gate: ok.

Stop and report if

• A path leaves the proxy's container or the directory behind, and a guard does not fix it.
• The tool's container would get a network.