A draft spec for the owner's review and 13 offline tasks with their given tests: shared tool arguments and host rules in proto, the sealed fetch target (M3a finding 14), the toolkit tools and SOCKS5 egress proxy, and brokerd's [runner], podman argument lists, runtime and proxy lifecycle. Each task's tests were run against a reference at that task's end state (560 to 638 tests, clippy clean); the reference is not in the repository. Adds the runner-unavailable runbook entry and tip T23 (ETXTBSY in script tests). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
111 lines
5.8 KiB
Markdown
111 lines
5.8 KiB
Markdown
# M3b task 11: the Podman runtime
|
|
|
|
**Branch:** `m3b` (run `git switch m3b`; `git status --short` must be empty, otherwise stop)
|
|
**Commit subject:** `brokerd: the Podman runtime`
|
|
|
|
## Goal
|
|
|
|
`Podman` implements `Runtime`: each call runs in a fresh container, its arguments go on standard
|
|
input, its output is kept up to a cap, and it is stopped at its time limit. Whatever the tool
|
|
prints is the result's content, labelled by the grant; every failure is a **fixed sentence**, and
|
|
what Podman itself said goes only to `brokerd`'s log. Spec section 6, "One call". `http_fetch`'s
|
|
proxy is task 12; in this task every call runs its container with no egress directory.
|
|
|
|
## Files
|
|
|
|
- Copy: `crates/brokerd/tests/support/fake_podman.rs`, `crates/brokerd/tests/container.rs`
|
|
- Create: `crates/brokerd/src/container.rs`
|
|
- Modify: `crates/brokerd/src/lib.rs` (`pub mod container;`), `docs/implementer-log.md`
|
|
|
|
## Interfaces
|
|
|
|
```rust
|
|
pub const RUNBOOK: &str = "see docs/runbook.md#runner-unavailable";
|
|
pub const COULD_NOT_RUN: &str = "the tool could not run";
|
|
pub const CANNOT_START: &str = "the tool runner could not start the container";
|
|
pub const KILLED: &str = "the tool was stopped: it ran out of memory or was killed";
|
|
pub const TIMED_OUT: &str = "the tool ran past its time limit";
|
|
pub const UNEXPECTED: &str = "the tool failed with an unexpected status";
|
|
pub const POLL: Duration = Duration::from_millis(50);
|
|
pub const STDERR_KEPT: usize = 4096;
|
|
|
|
pub type Log = Arc<dyn Fn(&str) + Send + Sync>;
|
|
|
|
pub struct Podman { /* runner: Runner, egress_dir: PathBuf, log: Log, next: AtomicU64 — private */ }
|
|
impl Podman {
|
|
/// `egress_dir` is `Config::egress_dir()`; task 12 uses it.
|
|
pub fn new(runner: Runner, egress_dir: PathBuf, log: Log) -> Podman;
|
|
}
|
|
impl Runtime for Podman {
|
|
fn run(&self, spec: &RunSpec) -> Result<RunOutput, RunError>;
|
|
}
|
|
```
|
|
|
|
## `run`: every step and exit
|
|
|
|
1. `n = next.fetch_add(1, SeqCst)`; `name = podman::container_name(spec.session(), spec.call(), n)`.
|
|
The first call of a `Podman` is number 0.
|
|
2. `args = podman::tool_args(spec, &runner, &name, None)`;
|
|
`input = spec.arguments().canonical_json()`; `limit = runner.time_limit(spec.tool())`.
|
|
3. Spawn `Command::new(&runner.podman).args(args)` with standard input, output and error all
|
|
piped. A failure → log `brokerd: cannot start {podman path}: {e}` + `"\n"` + `RUNBOOK`, and
|
|
return `Err(RunError::Unavailable(CANNOT_START))`.
|
|
4. Three threads, so no pipe can block another: one **writes** `input` to standard input and then
|
|
drops it (closing it); one **reads** standard output, keeping the first `output_cap` bytes and
|
|
**reading on past the cap**, throwing the rest away, and remembering that there was more; one
|
|
reads standard error the same way, keeping `STDERR_KEPT` bytes. (Stopping to read would leave
|
|
the tool blocked on a full pipe; closing the pipe would make Podman fail. Both give the wrong
|
|
answer.) Write the reading loop once, as a private function returning `(Vec<u8>, bool)`.
|
|
5. Wait with a loop: `child.try_wait()`; if it has ended, go to 6. If `limit` has passed since the
|
|
spawn, **stop the container**: run `podman kill <name>`, then `podman rm -f <name>` (each with
|
|
`.status()`, standard streams null; log a line if one does not succeed, and go on), then
|
|
`child.kill()` and `child.wait()`. Otherwise sleep `POLL` and try again.
|
|
6. Join the three threads (a thread that panicked counts as empty output).
|
|
7. The answer:
|
|
|
|
| How it ended | Answer | Log |
|
|
|---|---|---|
|
|
| past the time limit (step 5) | `Err(RunError::Failed(TIMED_OUT))` | a line naming the container |
|
|
| exit 0 or 1 | `Ok(RunOutput { content, truncated })`, content = the kept output decoded with `from_utf8_lossy` | — |
|
|
| exit 2 | `Err(RunError::Failed(COULD_NOT_RUN))` | the kept standard error |
|
|
| exit 125, 126 or 127 | `Err(RunError::Unavailable(CANNOT_START))` | the kept standard error, then `"\n"` + `RUNBOOK` |
|
|
| exit 137 | `Err(RunError::Failed(KILLED))` | — |
|
|
| anything else, including killed by a signal | `Err(RunError::Failed(UNEXPECTED))` | the status and the kept standard error |
|
|
|
|
**The tool's output never goes into a `RunError`**: only the six constants do.
|
|
|
|
Also add `pub(crate) fn podman(&self, args: &[&str])`, the helper step 5 uses to run a short
|
|
`podman` command and log if it fails; task 12 uses it too.
|
|
|
|
## About the given tests
|
|
|
|
The tests run `brokerd`'s real `runner::run` against a fake `podman`: a shell script that records
|
|
every call's arguments in a file and then does what the test says. As in task 06, every test takes
|
|
the `serial()` lock first, because writing a script and running it at once races with other tests'
|
|
forks ("Text file busy"). The time-limit test's fake ends with `exec sleep 30`, so killing the
|
|
process kills the sleep and nothing keeps the pipes open.
|
|
|
|
## Steps
|
|
|
|
- [ ] **1. Copy.** `git switch m3b`, then
|
|
`cp docs/plans/M3b/files/crates/brokerd/tests/support/fake_podman.rs crates/brokerd/tests/support/`
|
|
and `cp docs/plans/M3b/files/crates/brokerd/tests/container.rs crates/brokerd/tests/`
|
|
- [ ] **2. See it fail.** `cargo test -p brokerd --test container`. Expected: it does not compile.
|
|
- [ ] **3. Write `container.rs`.** Run `cargo fmt --all`.
|
|
- [ ] **4. See it pass.** `cargo test -p brokerd --test container`. Expected: 11 passed. Run it ten
|
|
times; it must pass every time.
|
|
- [ ] **5. Walk the table.** Point at the line of your code for each row, and check that no row
|
|
puts output into a `RunError`.
|
|
- [ ] **6. Run the gate.** `make gate`. Expected last line: `gate: ok`.
|
|
- [ ] **7. Log and commit.** `git add crates/brokerd docs/implementer-log.md && git commit`
|
|
|
|
## Done when
|
|
|
|
- `cargo test -p brokerd --test container` reports 11 passed ten times running; `make gate` prints
|
|
`gate: ok`.
|
|
|
|
## Stop and report if
|
|
|
|
- A test hangs, or fails only sometimes: report which, and how often, rather than add sleeps.
|
|
- A test wants tool output inside a failure message.
|