diff --git a/CLAUDE.md b/CLAUDE.md index c6cc897..62daf8f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -15,7 +15,9 @@ merged: tool containers run from the image `deploy/tools-image.nix` builds, chec M4 is split: M4a (`gatewayd`, conversations over Mattermost, spec `docs/specs/2026-09-23-m4a-gateway.md`) is done and merged, reviewed in `docs/implementer-log.md` and checked end to end against the owner's Mattermost; no finding is open. M4b (approvals over -Mattermost) is specified in `docs/specs/2026-09-24-m4b-approvals.md` (approved) and not yet planned. The owner's Mattermost settings and the +Mattermost) is specified in `docs/specs/2026-09-24-m4b-approvals.md` (approved) and planned in +`docs/plans/M4b/` (15 tasks, branch `m4b`); its reference is a bundle on straylight, +`~/boxmaker-refs/m4b-ref.bundle`. The owner's Mattermost settings and the bot's token are in `mattermost.env` at the repository root: ignored by git, never committed, and the token is never printed. Straylight now serves Ornith as four slots over one 262,144-token pool; see diff --git a/docs/implementer-lessons.md b/docs/implementer-lessons.md index 95b23fd..ec84b1c 100644 --- a/docs/implementer-lessons.md +++ b/docs/implementer-lessons.md @@ -62,7 +62,7 @@ How it is used: | T25 | Size a task by the largest function the model must hold in one turn, not by the task. Ornith writes one function with a few branches well; a function with half a dozen branches and threads (M3b's `run`) it plans in its head until the turn runs out, with nothing written. Give such a task a compiling skeleton with the big function already written as glue over small `todo!()` helpers, and say to fill one at a time with `cargo check` between. | M3b task 11: four sessions. Two wrote nothing; a whole-file skeleton got five of six functions; the finer skeleton finished it in ten minutes, and tasks 12 and 13 followed without a stop. M4a task 08: again, a skeleton whose `header` was one `todo!()` with a dozen branches; nothing written. Size every `todo!()`, not only the task: the largest comment above a `todo!()` is a fair proxy, and one over about six lines needs splitting. M4a task 14: a task with a dozen small `todo!()`s still stalled, because the model read the whole crate to learn the APIs first; give a table of every call the task makes, with its signature, and make copying and the failing test the first two actions. | | T26 | Replay each task's end state on its own, **and** read the task file against the reference for anything the reference has that the task does not ask for. The replay proves the tests can pass; only the reading finds a field the reference reads through a getter the task never mentions. | M3b task 11: the task stored `egress_dir` without the reference's getter, so the field was never read and clippy failed; the replay passed because the reference had the getter. | -| T27 | Every wait in a given test has a limit (`recv_timeout`, a deadline loop, `is_finished` before `join`). A test that waits forever on a broken implementation hangs the driver instead of failing, and the implementer cannot tell a hang from slowness. | M4a planning: breaking the reference's ping made `ws_conn` hang on `rx.recv()`; retrying a refused token made the serve test hang on `join()`. Both now fail within 5 s. | +| T27 | Every wait in a given test has a limit (`recv_timeout`, a deadline loop, `is_finished` before `join`). A test that waits forever on a broken implementation hangs the driver instead of failing, and the implementer cannot tell a hang from slowness. | M4a planning: breaking the reference's ping made `ws_conn` hang on `rx.recv()`; retrying a refused token made the serve test hang on `join()`. Both now fail within 5 s. M4b planning: a failed assertion inside `std::thread::scope` hangs instead of failing, because the scope joins its threads first; give such a test a guard that stops the thread on drop, and wait on `is_finished()` with a deadline instead of `join()`. | | T28 | Before hand-over, break the reference on purpose, one line at a time (drop a check, move a bound by one, skip a save), and run the given tests against each change. A change the tests still pass is a missing test, unless it cannot change behaviour. Rustfmt reflows lines: match on text that survives formatting. | M4a planning: 69 changes over six modules; the tests missed 8. Five were real gaps and got tests (a reconnect that re-sent "interrupted" for a running turn was one), two could not change behaviour, and one was left (an event frame with another id, which `loopd` never sends). | | T29 | Check a skeleton's comments, not only its code. Print every `todo!()` beside its signature and read each pair before hand-over. Filling the bodies from your own code proves the split can pass; it cannot show that a comment sits on the wrong function. Edit comments by function, never by "the next comment after this name". And tell the implementer to stop and quote a comment that does not fit. | M4a task 14, second session: `post`'s comment was `now_ms`'s text, put there by a script; Ornith saw the contradiction and deliberated over it until the turn ran out. M4a task 16: a comment naming the function it replaced made the task's own `grep` fail; and a given test copied into the plan before `cargo fmt` came back reformatted. Run the task's checks, and `cargo fmt`, on the end state before hand-over. | diff --git a/docs/plans/M4b/01-proto-escape.md b/docs/plans/M4b/01-proto-escape.md new file mode 100644 index 0000000..b05438a --- /dev/null +++ b/docs/plans/M4b/01-proto-escape.md @@ -0,0 +1,42 @@ +# M4b task 01: move the text escaping into `proto` + +**Branch:** `m4b` (run `git switch m4b`; `git status --short` must be empty, otherwise stop) +**Commit subject:** `proto: escape, moved from bxctl so gatewayd shows the same thing` + +## Goal + +`bxctl` escapes text it did not write (model output, tool arguments, server errors) before it +reaches the terminal: control characters, zero-width and bidirectional marks, line separators and +the BOM become `\uXXXX`. M4b posts tool arguments in Mattermost, and they must be escaped by the +same rules, so the code moves to `proto`, where both can use it. **Nothing changes in what it +does.** `bxctl::escape` stays, as a re-export, so no `bxctl` code or test changes. + +## Files + +- Copy: `crates/proto/src/escape.rs` (complete), `crates/bxctl/src/escape.rs` (complete, replaces + the old one), `crates/proto/tests/escape.rs` +- Modify: `crates/proto/src/lib.rs` (`pub mod escape;`, after `pub mod class;`), + `docs/implementer-log.md` + +`proto/src/escape.rs` is `bxctl`'s old file with a new module comment. `bxctl/src/escape.rs` becomes +one line: `pub use proto::escape::{escape_json_text, escape_model_text};`. The new test is +`bxctl`'s escape test, pointed at `proto::escape`. + +## Steps + +- [ ] **1. Copy.** `git switch m4b`, then + `cp docs/plans/M4b/files/crates/proto/src/escape.rs crates/proto/src/ && cp docs/plans/M4b/files/crates/bxctl/src/escape.rs crates/bxctl/src/ && cp docs/plans/M4b/files/crates/proto/tests/escape.rs crates/proto/tests/`. + Add `pub mod escape;` to `crates/proto/src/lib.rs`. +- [ ] **2. See it pass.** `cargo test -p proto --test escape` (8 passed) and `cargo test -p bxctl` + (every suite passes, unchanged). +- [ ] **3. Run the gate.** `cargo fmt --all`, then `make gate`. Expected last line: `gate: ok`. +- [ ] **4. Log and commit.** + `git add crates/proto crates/bxctl docs/implementer-log.md Cargo.lock && git commit` + +## Done when + +- Both crates' tests pass; `make gate` prints `gate: ok`. + +## Stop and report if + +- Any `bxctl` test changes result: the move must not change behaviour. diff --git a/docs/plans/M4b/02-proto-gateway-wire.md b/docs/plans/M4b/02-proto-gateway-wire.md new file mode 100644 index 0000000..d04159c --- /dev/null +++ b/docs/plans/M4b/02-proto-gateway-wire.md @@ -0,0 +1,86 @@ +# M4b task 02: the `gateway.sock` messages in `proto` + +**Branch:** `m4b` (run `git switch m4b`; `git status --short` must be empty, otherwise stop) +**Commit subject:** `proto: gateway, the four messages between brokerd and gatewayd` + +## Goal + +One `gateway.sock` connection carries one pending approval from `brokerd` to `gatewayd`, and its +answer back (M4b spec, section 4). Every frame is a `proto` `Envelope`: + +| # | From | `kind` | `final` | Body | +|---|---|---|---|---| +| 1 | `brokerd` | `approval_request` | true | `ApprovalRequest { approval: PendingApproval }` | +| 2 | `gatewayd` | `approval_posted` | false | `ApprovalPosted { posts: Vec }` | +| 3 | `gatewayd` | `approval_reply` | true | `ApprovalReply { approval, approved, by, post, reason }` | +| 4 | `brokerd` | `approval_outcome` | true | `ApprovalOutcome { outcome: GatewayOutcome }` | + +These are **our** format: unknown fields are errors everywhere. + +## Files + +- Copy: the skeleton `crates/proto/src/gateway.rs`, `crates/proto/tests/gateway_wire.rs`, and the + eight fixtures `crates/proto/tests/fixtures/wire/gateway_*.json` +- Modify: `crates/proto/src/lib.rs`, `crates/proto/src/wire.rs`, `crates/brokerd/src/broker.rs`, + `docs/implementer-log.md` + +## The skeleton + +Written: `MAX_REASON` (500), the four body types and `GatewayOutcome` with their serde attributes +(they are the wire format: do not change them). To fill: `is_mattermost_id` and +`ApprovalReply::check`, each with its steps above it. + +## The changes, exactly + +**`crates/proto/src/lib.rs`**: `pub mod gateway;` after `pub mod frame;`, and this re-export, +after the `pub use frame::…` line: + +```rust +pub use gateway::{ + ApprovalOutcome, ApprovalPosted, ApprovalReply, ApprovalRequest, GatewayOutcome, MAX_REASON, + is_mattermost_id, +}; +``` + +**`crates/proto/src/wire.rs`**: four variants at the end of `enum Message`, after +`GrantsReport(GrantsReport),`: + +```rust + ApprovalRequest(ApprovalRequest), + ApprovalPosted(ApprovalPosted), + ApprovalReply(ApprovalReply), + ApprovalOutcome(ApprovalOutcome), +``` + +and at the top, `use crate::gateway::{ApprovalOutcome, ApprovalPosted, ApprovalReply, ApprovalRequest};`. + +**`crates/brokerd/src/broker.rs`**: `kind` must name every message. Add these four arms after +`Message::GrantsReport(_) => "grants_report",`, and change "all fourteen kinds" in its doc +comment to "all eighteen kinds": + +```rust + Message::ApprovalRequest(_) => "approval_request", + Message::ApprovalPosted(_) => "approval_posted", + Message::ApprovalReply(_) => "approval_reply", + Message::ApprovalOutcome(_) => "approval_outcome", +``` + +## Steps + +- [ ] **1. Copy.** `git switch m4b`, then + `cp docs/plans/M4b/files/crates/proto/src/gateway.rs crates/proto/src/ && cp docs/plans/M4b/files/crates/proto/tests/gateway_wire.rs crates/proto/tests/ && cp docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_*.json crates/proto/tests/fixtures/wire/`. +- [ ] **2. Make the three edits above.** `cargo build --workspace` must succeed. +- [ ] **3. See it fail.** `cargo test -p proto --test gateway_wire`. Expected: 1 fails + (`an_answer_is_checked`), 2 pass (the fixtures, since the types are given). +- [ ] **4. Fill `is_mattermost_id`, then `check`.** `cargo check -p proto` after each. +- [ ] **5. See it pass.** `cargo test -p proto --test gateway_wire`. Expected: 3 passed. +- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`. Expected last line: `gate: ok`. +- [ ] **7. Log and commit.** `git add crates/proto crates/brokerd docs/implementer-log.md Cargo.lock && git commit` + +## Done when + +- `cargo test -p proto --test gateway_wire` passes; `make gate` prints `gate: ok`. + +## Stop and report if + +- A comment above a `todo!()` does not fit its function, or contradicts a test: quote it. diff --git a/docs/plans/M4b/03-bxctl-deny.md b/docs/plans/M4b/03-bxctl-deny.md new file mode 100644 index 0000000..bf75fda --- /dev/null +++ b/docs/plans/M4b/03-bxctl-deny.md @@ -0,0 +1,42 @@ +# M4b task 03: `bxctl deny`, another name for `refuse` + +**Branch:** `m4b` (run `git switch m4b`; `git status --short` must be empty, otherwise stop) +**Commit subject:** `bxctl: deny, another name for refuse` + +## Goal + +In Mattermost the owner answers `!deny `; in the terminal the word should work too. `bxctl +deny` means exactly what `bxctl refuse` means, argument for argument, errors included. + +## Files + +- Copy: `crates/bxctl/tests/cli_deny.rs` +- Modify: `crates/bxctl/src/cli.rs`, `docs/implementer-log.md` + +## The changes, exactly + +In `crates/bxctl/src/cli.rs`: + +1. In `parse`, `"refuse" => parse_refuse(rest, home),` becomes + `"refuse" | "deny" => parse_refuse(rest, home),`. +2. In `USAGE`, after the two lines about `bxctl refuse`, add a blank line and: + +```text + bxctl deny [--reason TEXT] [--admin-socket PATH] + Another name for refuse. +``` + +(keep the two-space and six-space indents of the lines around it). + +## Steps + +- [ ] **1. Copy.** `git switch m4b`, then `cp docs/plans/M4b/files/crates/bxctl/tests/cli_deny.rs crates/bxctl/tests/` +- [ ] **2. See it fail.** `cargo test -p bxctl --test cli_deny`. Expected: 2 fail. +- [ ] **3. Make the two changes.** +- [ ] **4. See it pass.** `cargo test -p bxctl`. Expected: every suite passes; `cli_deny` 2 passed. +- [ ] **5. Run the gate.** `cargo fmt --all`, then `make gate`. Expected last line: `gate: ok`. +- [ ] **6. Log and commit.** `git add crates/bxctl docs/implementer-log.md Cargo.lock && git commit` + +## Done when + +- `cargo test -p bxctl` passes; `make gate` prints `gate: ok`. diff --git a/docs/plans/M4b/04-brokerd-answer-post.md b/docs/plans/M4b/04-brokerd-answer-post.md new file mode 100644 index 0000000..db2df1a --- /dev/null +++ b/docs/plans/M4b/04-brokerd-answer-post.md @@ -0,0 +1,96 @@ +# M4b task 04: an answer records its post; the table says what is pending + +**Branch:** `m4b` (run `git switch m4b`; `git status --short` must be empty, otherwise stop) +**Commit subject:** `brokerd: answers carry the Mattermost post; Table::contains` + +## Goal + +When an approval is answered from Mattermost, the audit record must name the approver **and the +post** that answered (the `Approval` record has had a `post` field since M3a, always `None` so +far). And the courier (task 06) needs to ask the approval table whether an approval is still +pending. + +`ledger.rs` is at 499 lines, so first `Answer` and `Answered` move to `approvals.rs`, beside +`Verdict` and `Entry`; `ledger` re-exports them, so every `crate::ledger::Answer` still works. + +## Files + +- Copy: `crates/brokerd/tests/ledger_post.rs`, and **new versions** of three given tests, + `crates/brokerd/tests/broker_pending.rs`, `broker_sequence.rs` and `ledger_answer.rs` (the only + change in each: `post: None,` in every `Answer::Approved` and `Answer::Refused`) +- Modify: `crates/brokerd/src/approvals.rs`, `crates/brokerd/src/ledger.rs`, + `crates/brokerd/src/admin.rs`, `docs/implementer-log.md` + +## The changes, exactly + +**1. Move.** Cut `enum Answer` and `struct Answered` from `ledger.rs`, with their doc comments, +and paste them into `approvals.rs`, before `pub struct Entry`. In `ledger.rs`, add +`pub use crate::approvals::{Answer, Answered};` next to `use crate::approvals::Verdict;`. In +`approvals.rs`, `use proto::{DenyReason, PendingApproval, Timestamp};` becomes +`use proto::{DecisionRecord, DenyReason, PendingApproval, Timestamp};`. + +**2. `post`.** In `approvals.rs`, `Answer` becomes: + +```rust +/// An owner's answer to a pending call. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Answer { + /// `by` is "bxctl" or a Mattermost user id; `post` the Mattermost post that answered. + Approved { + by: Option, + post: Option, + }, + Refused { + by: Option, + post: Option, + reason: Option, + }, + Expired, +} +``` + +In `ledger.rs`, `answer`, step 3 becomes: + +```rust + let (answer, by, post, reason) = match &answer { + Answer::Approved { by, post } => { + (ApprovalAnswer::Approved, by.clone(), post.clone(), None) + } + Answer::Refused { by, post, reason } => ( + ApprovalAnswer::Refused, + by.clone(), + post.clone(), + reason.clone(), + ), + Answer::Expired => (ApprovalAnswer::Expired, None, None, None), + }; +``` + +and in step 6, the `AuditEvent::Approval` field `post: None,` becomes `post,`. + +In `admin.rs`, the two answers `bxctl` gives get `post: None,` after their `by: Some(BY.to_string()),`. + +**3. `contains`.** In `approvals.rs`, `impl Table`, before `take`: + +```rust + /// Is approval `id` still pending? Only a hint: it may be taken the moment after. + pub fn contains(&self, id: u64) -> bool { + self.lock().contains_key(&id) + } +``` + +## Steps + +- [ ] **1. Copy.** `git switch m4b`, then + `cp docs/plans/M4b/files/crates/brokerd/tests/ledger_post.rs docs/plans/M4b/files/crates/brokerd/tests/broker_pending.rs docs/plans/M4b/files/crates/brokerd/tests/broker_sequence.rs docs/plans/M4b/files/crates/brokerd/tests/ledger_answer.rs crates/brokerd/tests/` +- [ ] **2. See it fail.** `cargo test -p brokerd --test ledger_post`. Expected: it does not + compile (`post` is not a field of `Answer`; `contains` does not exist). +- [ ] **3. Make the three changes.** Run `cargo fmt --all`. +- [ ] **4. See it pass.** `cargo test -p brokerd`. Expected: every suite passes; `ledger_post` 3 + passed. `wc -l crates/brokerd/src/ledger.rs` is under 490. +- [ ] **5. Run the gate.** `make gate`. Expected last line: `gate: ok`. +- [ ] **6. Log and commit.** `git add crates/brokerd docs/implementer-log.md Cargo.lock && git commit` + +## Done when + +- `cargo test -p brokerd` passes; `make gate` prints `gate: ok`. diff --git a/docs/plans/M4b/05-brokerd-config-gateway.md b/docs/plans/M4b/05-brokerd-config-gateway.md new file mode 100644 index 0000000..cee667e --- /dev/null +++ b/docs/plans/M4b/05-brokerd-config-gateway.md @@ -0,0 +1,73 @@ +# M4b task 05: `[gateway]` in `brokerd.toml` + +**Branch:** `m4b` (run `git switch m4b`; `git status --short` must be empty, otherwise stop) +**Commit subject:** `brokerd: config, [gateway] socket for Mattermost approvals` + +## Goal + +`brokerd` carries pending approvals to `gatewayd` only when `brokerd.toml` says where (M4b spec, +section 3): + +```toml +[gateway] # absent: Mattermost approvals are off, as in M3 +socket = "" # empty: /run/gateway/gateway.sock +``` + +## Files + +- Copy: `crates/brokerd/tests/config_gateway.rs`, and a **new version** of the given + `crates/brokerd/tests/support/rig.rs` (its `Config` gains `gateway: None,`) +- Modify: `crates/brokerd/src/config.rs`, `docs/implementer-log.md` + +## The changes, exactly + +In `crates/brokerd/src/config.rs`: + +1. Before `fn default_podman`, the section: + +```rust +/// `[gateway]`: where `gatewayd` takes approvals to post in Mattermost. Without the section, +/// Mattermost approvals are off and only `bxctl` answers. An empty path means the default. +#[derive(Debug, Clone, PartialEq, Eq, Deserialize, Default)] +#[serde(deny_unknown_fields, default)] +pub struct Gateway { + pub socket: PathBuf, +} +``` + +2. In `struct Config`, after `pub runner: Option,`: + +```rust + #[serde(default)] + pub gateway: Option, +``` + +3. In `impl Config`, before `audit_dir`: + +```rust + /// `gateway.sock`, when `[gateway]` is set. + pub fn gateway_socket(&self) -> Option { + let gateway = self.gateway.as_ref()?; + if gateway.socket.as_os_str().is_empty() { + Some(self.paths.home.join("run/gateway/gateway.sock")) + } else { + Some(gateway.socket.clone()) + } + } +``` + +## Steps + +- [ ] **1. Copy.** `git switch m4b`, then + `cp docs/plans/M4b/files/crates/brokerd/tests/config_gateway.rs crates/brokerd/tests/ && cp docs/plans/M4b/files/crates/brokerd/tests/support/rig.rs crates/brokerd/tests/support/` +- [ ] **2. See it fail.** `cargo test -p brokerd --test config_gateway`. Expected: it does not + compile. +- [ ] **3. Make the three changes.** Run `cargo fmt --all`. +- [ ] **4. See it pass.** `cargo test -p brokerd`. Expected: every suite passes; `config_gateway` + 4 passed. +- [ ] **5. Run the gate.** `make gate`. Expected last line: `gate: ok`. +- [ ] **6. Log and commit.** `git add crates/brokerd docs/implementer-log.md Cargo.lock && git commit` + +## Done when + +- `cargo test -p brokerd` passes; `make gate` prints `gate: ok`. diff --git a/docs/plans/M4b/06-brokerd-courier.md b/docs/plans/M4b/06-brokerd-courier.md new file mode 100644 index 0000000..5cd275e --- /dev/null +++ b/docs/plans/M4b/06-brokerd-courier.md @@ -0,0 +1,133 @@ +# M4b task 06: the courier + +**Branch:** `m4b` (run `git switch m4b`; `git status --short` must be empty, otherwise stop) +**Commit subject:** `brokerd: courier, carrying pending approvals to gatewayd and answers back` + +## Goal + +When an `ask` call enters the pending table and `[gateway]` is set, a **courier** thread carries +it to `gatewayd` over `gateway.sock` (task 02's four frames) and brings the answer back (M4b spec, +section 5): + +- It stops as soon as the table no longer holds the approval (answered in `bxctl`, expired, the + requester gone), closing the connection; `gatewayd` then marks the post "no longer pending". +- If `gatewayd` cannot be reached or cannot post, it logs **once** for this approval, with the + pointer `see docs/runbook.md#gateway-unreachable`, and tries again every 5 seconds; when a later + try posts it, it logs `brokerd: approval posted in Mattermost`. +- If `gatewayd` goes away after posting, it posts it again. +- On an answer, it takes the entry out of the table and answers it with `admin::answer`, exactly + as `bxctl approve` and `refuse` do, with the Mattermost user and post (task 04); then sends what + came of it. If the entry was already gone, the outcome is `Gone` and nothing is written. + +The courier runs **beside** the connection thread that waits for the verdict: a scoped thread +(`std::thread::Builder::spawn_scoped`), stopped by a flag when the wait ends. So `ask`'s wait moves +into a function of its own, called inside `courier::alongside`. + +## Files + +- Copy: the skeleton `crates/brokerd/src/courier.rs`, `crates/brokerd/tests/courier.rs`, + `crates/brokerd/tests/support/fake_gateway.rs` +- Append: `docs/plans/M4b/files/runbook-gateway-unreachable.md` to the end of `docs/runbook.md` +- Modify: `crates/brokerd/src/lib.rs` (`pub mod courier;` after `pub mod config;`), + `crates/brokerd/src/broker.rs`, `docs/implementer-log.md` + +## The skeleton + +Written, because they are the protocol and its glue: `RUNBOOK`, `Timing` (5 s retry, 1 s tick), +`Attempt`, **`alongside`**, **`carry`**, **`attempt`**, **`exchange`** and `lost`. Read them first: +they call everything you write. To fill, each a few lines with the code in its comment: +`finished`, `pause`, `frames`, `answer`. + +`frames` reads `gatewayd`'s frames whole on a thread of its own and hands them over a channel, so +the courier can wait with a timeout without losing half a frame (`UnixStream::peek` is not stable). + +## `broker.rs`, exactly + +1. `use std::sync::mpsc::RecvTimeoutError;` becomes + `use std::sync::mpsc::{Receiver, RecvTimeoutError};`, and add `use crate::courier;` after + `use crate::config::Config;`. +2. In `pending`, keep a copy of what the table lists, just before it is inserted: + `let shown = info.clone();` on the line before `let verdict = broker.table().insert(info, ask);`. +3. Replace step 4 of `pending`, from `// 4. Wait for the verdict, or for the peer to leave.` down + to the end of that `let outcome = loop { … };`, with: + +```rust + // 4. Wait for the verdict, or for the peer to leave, with a courier carrying the approval to + // Mattermost meanwhile when `[gateway]` is set. + let waited = courier::alongside(broker, &shown, || { + wait_for_verdict(broker, stream, &verdict, seq) + }); + let outcome = match waited { + Waited::Verdict(outcome) => outcome, + Waited::PeerLeft => return None, // 4. the peer left while we waited + }; +``` + +4. Add, before `pub fn alive`, the old step 4 as a function (the same branches, with `return`s + instead of `break`s): + +```rust +/// What waiting for the verdict came to. +enum Waited { + Verdict(Result), + PeerLeft, +} + +/// Wait for the verdict, or for the peer to leave. If the peer leaves and the entry is still in +/// the table, taking it ends the approval unanswered; if someone already took it, their verdict +/// is awaited. +fn wait_for_verdict( + broker: &Broker, + stream: &UnixStream, + verdict: &Receiver, + seq: u64, +) -> Waited { + loop { + match verdict.recv_timeout(Duration::from_secs(1)) { + Ok(verdict) => return Waited::Verdict(Ok(verdict)), + Err(RecvTimeoutError::Timeout) => { + if alive(stream) { + continue; + } + match broker.table().take(seq) { + Some(_) => return Waited::PeerLeft, + None => match verdict.recv() { + Ok(verdict) => return Waited::Verdict(Ok(verdict)), + Err(_) => return Waited::Verdict(Err(DenyReason::AuditUnavailable)), + }, + } + } + // The taker dropped it unanswered. + Err(RecvTimeoutError::Disconnected) => { + return Waited::Verdict(Err(DenyReason::AuditUnavailable)); + } + } + } +} +``` + +## Steps + +- [ ] **1. Copy.** `git switch m4b`, then + `cp docs/plans/M4b/files/crates/brokerd/src/courier.rs crates/brokerd/src/ && cp docs/plans/M4b/files/crates/brokerd/tests/courier.rs crates/brokerd/tests/ && cp docs/plans/M4b/files/crates/brokerd/tests/support/fake_gateway.rs crates/brokerd/tests/support/` + and `cat docs/plans/M4b/files/runbook-gateway-unreachable.md >> docs/runbook.md`. Add + `pub mod courier;` to `lib.rs`. +- [ ] **2. Make the four `broker.rs` changes.** `cargo build -p brokerd` must succeed. +- [ ] **3. See it fail.** `cargo test -p brokerd --test courier`. Expected: 9 fail, 1 passes + (`without_a_gateway_nothing_is_carried`), in about 10 s. +- [ ] **4. Fill `finished`, `pause`, `frames`, `answer`**, `cargo check -p brokerd` after each. +- [ ] **5. See it pass.** `cargo test -p brokerd --test courier`, three times: 10 passed each + time, in about 10 s (two tests wait out the 5 s retry on purpose). Then `cargo test -p brokerd`: + every suite passes. +- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`. Expected last line: `gate: ok`. +- [ ] **7. Log and commit.** `git add crates/brokerd docs/runbook.md docs/implementer-log.md Cargo.lock && git commit` + +## Done when + +- The courier suite passes three times running; `make gate` prints `gate: ok`. + +## Stop and report if + +- A comment above a `todo!()` does not fit its function, or contradicts a test: quote it. +- A written function (`alongside`, `carry`, `attempt`, `exchange`) seems to need a change. +- A test hangs or takes much longer than stated. diff --git a/docs/plans/M4b/07-gatewayd-config-approvals.md b/docs/plans/M4b/07-gatewayd-config-approvals.md new file mode 100644 index 0000000..b334123 --- /dev/null +++ b/docs/plans/M4b/07-gatewayd-config-approvals.md @@ -0,0 +1,103 @@ +# M4b task 07: `[approvals]` in `gatewayd.toml` + +**Branch:** `m4b` (run `git switch m4b`; `git status --short` must be empty, otherwise stop) +**Commit subject:** `gatewayd: config, [approvals] users and socket` + +## Goal + +Who may answer approvals, and where `brokerd` brings them (M4b spec, section 3): + +```toml +[approvals] +users = [] # Mattermost user ids who may answer; empty: everyone in [allow] users +socket = "" # empty: /run/gateway/gateway.sock +``` + +An approver must also be in `[allow] users`: one who is not heard could never answer, so that is a +configuration error. The **first approver** (`approvals.users[0]`, or `allow.users[0]` when the +list is empty) gets, in direct messages, the approvals of sessions that are not in Mattermost. + +## Files + +- Copy: `crates/gatewayd/tests/config_approvals.rs` +- Modify: `crates/gatewayd/src/config.rs`, `docs/runbook.md`, `docs/implementer-log.md` + +## The changes, exactly + +In `crates/gatewayd/src/config.rs`: + +1. In `struct Config`, after `pub limits: Limits,`: + +```rust + #[serde(default)] + pub approvals: ApprovalsConfig, +``` + +and after `struct Config`: + +```rust +/// `[approvals]`: who may answer approvals, and where `brokerd` brings them. +#[derive(Debug, Clone, PartialEq, Eq, Deserialize, Default)] +#[serde(deny_unknown_fields, default)] +pub struct ApprovalsConfig { + /// Mattermost user ids; empty means everyone in `[allow] users`. + pub users: Vec, + /// Empty: `/run/gateway/gateway.sock`. + pub socket: PathBuf, +} +``` + +2. In `problem`, right before the `[limits]` checks (`if self.limits.queue == 0`): + +```rust + for id in &self.approvals.users { + if !self.allow.users.contains(id) { + return Some(format!( + "[approvals] {id:?} is not in [allow] users: an approver who is not heard could never answer" + )); + } + } +``` + +3. In `impl Config`, before `state_path`, three methods. Write them from these descriptions: + - `pub fn approvers(&self) -> &[String]`: `&self.approvals.users`, or `&self.allow.users` when + the first is empty. + - `pub fn first_approver(&self) -> Option<&str>`: the first of `approvers()` + (`.first().map(String::as_str)`). + - `pub fn gateway_socket(&self) -> PathBuf`: `self.approvals.socket`, or + `self.paths.home.join("run/gateway/gateway.sock")` when it is empty + (`as_os_str().is_empty()`). + +In `docs/runbook.md`, entry `gatewayd-start-failed`, replace the **Why** and **Confirm** +paragraphs with: + +```markdown +**Why.** `gatewayd` could not read or parse `gatewayd.toml`, or a value in it is not allowed: a +`url` that is not `http://` or `https://` with a host and an optional port, an id that is not 26 +characters of `a-z0-9`, an empty `[allow] users`, an id in `[approvals] users` that is not also in +`[allow] users`, or a secret with not exactly one of `credential`, `env` and `file`. Or it could +not create `/gateway/`, or could not listen on `gateway.sock` (its directory is made 0700 +and the socket 0600, as `brokerd`'s are; a path of 108 bytes or more cannot be a socket). It will +not guess at a configuration that decides who it answers and who approves. + +**Confirm.** The line says which: `: …` for the file (unknown keys are errors; check +it against `docs/specs/2026-09-23-m4a-gateway.md`, section 3, and +`docs/specs/2026-09-24-m4b-approvals.md`, section 3), `cannot prepare : …` for a directory +(`ls -ld "$(dirname )"`), or `cannot listen on : …` for `gateway.sock` +(`Address already in use` means another `gatewayd` runs: `pgrep -x gatewayd`). +``` + +## Steps + +- [ ] **1. Copy.** `git switch m4b`, then `cp docs/plans/M4b/files/crates/gatewayd/tests/config_approvals.rs crates/gatewayd/tests/` +- [ ] **2. See it fail.** `cargo test -p gatewayd --test config_approvals`. Expected: it does not + compile. +- [ ] **3. Make the changes.** Run `cargo fmt --all`. +- [ ] **4. See it pass.** `cargo test -p gatewayd --test config_approvals --test config`. + Expected: 4 and 7 passed. +- [ ] **5. Run the gate.** `make gate`. Expected last line: `gate: ok`. +- [ ] **6. Log and commit.** `git add crates/gatewayd docs/runbook.md docs/implementer-log.md Cargo.lock && git commit` + +## Done when + +- Both suites pass; `make gate` prints `gate: ok`. diff --git a/docs/plans/M4b/08-gatewayd-mention-commands.md b/docs/plans/M4b/08-gatewayd-mention-commands.md new file mode 100644 index 0000000..aed7fbb --- /dev/null +++ b/docs/plans/M4b/08-gatewayd-mention-commands.md @@ -0,0 +1,170 @@ +# M4b task 08: the leading mention, and approval commands + +**Branch:** `m4b` (run `git switch m4b`; `git status --short` must be empty, otherwise stop) +**Commit subject:** `gatewayd: strip a leading mention of this bot; recognise approval commands` + +## Goal + +Two changes to routing (M4b spec, section 7): + +1. **The leading mention.** After M4a has decided a post is for this Boxmaker, a leading + `@` is stripped before anything reads the message: `@boxmaker0 !approve 42` is a + command, and `@boxmaker0 summarise the log` reaches the model as `summarise the log`. Where two + Boxmakers share a thread, that is how the owner addresses one of them. +2. **Approval commands.** `!approve `, `!deny ` and `!deny ` (exactly) become + a new `Route::Approval`, carrying the command, who wrote it, which post, and when. Task 15 hands + it to the approvals module; until then `serve` answers it with M4a's notice, so nothing else + changes yet. Any other `!…` is still answered with `UNKNOWN_COMMAND`. + +## Files + +- Copy: `crates/gatewayd/tests/sessions_mention.rs`, and **new versions** of two given tests: + `crates/gatewayd/tests/sessions.rs` (commands now route to `Approval`; the mention is stripped) + and `crates/gatewayd/tests/serve.rs`, from `serve.rs-task08` (one line: the turn's text is + `start`, no longer `@boxmaker-straylight start`) +- Modify: `crates/gatewayd/src/sessions.rs`, `crates/gatewayd/src/serve/handle.rs`, + `docs/implementer-log.md` + +## The changes, exactly + +In `crates/gatewayd/src/sessions.rs`: + +**1.** In `enum Route`, after `Queue(Message),`: + +```rust + /// `!approve` or `!deny`, for the approvals module to judge. + Approval { + thread: Thread, + command: ApprovalCommand, + /// Who wrote it, which post it is, and when (Mattermost's `create_at`). + user: String, + post: String, + at: i64, + }, +``` + +**2.** After `enum Route`, the command type and `approval_command`, as given: + +```rust +/// An approval command as typed: exactly `!approve `, `!deny ` or `!deny `. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum ApprovalCommand { + Approve(u64), + Deny(u64, Option), + /// `!approve` or `!deny` in any other form. + Malformed, +} + +/// The longest reason a `!deny` may give, in characters. +pub const MAX_REASON: usize = proto::MAX_REASON; + +/// `!approve …` or `!deny …` as an approval command; `None` for any other text. +pub fn approval_command(text: &str) -> Option { + let after = text.strip_prefix('!')?; + let (word, rest) = match after.split_once(' ') { + Some((word, rest)) => (word, Some(rest)), + None => (after, None), + }; + if !matches!(word, "approve" | "deny") { + return None; + } + let id = |s: &str| -> Option { + if s.is_empty() || s.len() > 20 || !s.bytes().all(|b| b.is_ascii_digit()) { + return None; + } + s.parse().ok() + }; + let parsed = match (word, rest) { + ("approve", Some(rest)) => id(rest).map(ApprovalCommand::Approve), + ("deny", Some(rest)) => match rest.split_once(' ') { + None => id(rest).map(|n| ApprovalCommand::Deny(n, None)), + Some((n, reason)) => { + let fits = !reason.is_empty() && reason.chars().count() <= MAX_REASON; + id(n) + .filter(|_| fits) + .map(|n| ApprovalCommand::Deny(n, Some(reason.to_string()))) + } + }, + _ => None, + }; + Some(parsed.unwrap_or(ApprovalCommand::Malformed)) +} +``` + +**3.** Write `strip_mention` yourself, next to them: + +```rust +/// The text after a leading mention of this bot (`me`, lower-case), and the spaces after it. The +/// mention is matched as `named` matches names; nothing else is removed. A message that would be +/// left empty is kept as it was. +pub fn strip_mention<'a>(message: &'a str, me: &str) -> &'a str +``` + + 1. `let Some(after) = message.strip_prefix('@') else { return message; };` + 2. The name's length: `after.find(|c: char| !(c.is_ascii_alphanumeric() || matches!(c, '.' | '-' | '_'))).unwrap_or(after.len())`; + `let raw = after.get(..len).unwrap_or_default();` and + `let name = raw.trim_end_matches('.');` (a trailing dot is not part of a name). + 3. `name` empty, or not `name.eq_ignore_ascii_case(me)`: return `message`. + 4. `let rest = after.get(name.len()..).unwrap_or_default().trim_start_matches(' ');` (spaces + only, not newlines). + 5. `if rest.is_empty() { message } else { rest }`. + +**4.** In `Router::route`: + - `let message = &post.message;` becomes `let message = strip_mention(&post.message, &self.me_name);` + - The block `if let Some(after) = message.strip_prefix('!') { … }` (which answered + `M4B_COMMAND`) becomes: + +```rust + if let Some(command) = approval_command(message) { + return Route::Approval { + thread, + command, + user: post.user_id.clone(), + post: post.id.clone(), + at: post.create_at, + }; + } + if message.starts_with('!') { + return Route::Reply { + thread, + text: UNKNOWN_COMMAND.to_string(), + }; + } +``` + + - In the final `Route::Queue`, `text: message.clone(),` becomes `text: message.to_string(),`. + - In `route`'s step comment, the part about commands now reads: "Then a leading mention of this + bot is stripped (M4b spec, section 7), and: "!!..." drops one "!" and goes on as a message; + `approval_command` -> Approval; any other "!" -> Reply UNKNOWN_COMMAND." + +In `crates/gatewayd/src/serve/handle.rs`, `handle_post`'s `match route` gains, after the +`Route::Reply` arm: + +```rust + // Until the approvals module is wired in (task 15), M4a's notice. + Route::Approval { thread, .. } => { + self.post(&thread.channel, &thread.root, M4B_COMMAND) + } +``` + +and its `use crate::sessions::{…}` gains `M4B_COMMAND`. + +## Steps + +- [ ] **1. Copy.** `git switch m4b`, then + `cp docs/plans/M4b/files/crates/gatewayd/tests/sessions_mention.rs docs/plans/M4b/files/crates/gatewayd/tests/sessions.rs crates/gatewayd/tests/ && cp docs/plans/M4b/files/crates/gatewayd/tests/serve.rs-task08 crates/gatewayd/tests/serve.rs` +- [ ] **2. See it fail.** `cargo test -p gatewayd --test sessions_mention`. Expected: it does not + compile. +- [ ] **3. Make the changes**, `strip_mention` last. Run `cargo fmt --all`. +- [ ] **4. See it pass.** `cargo test -p gatewayd --test sessions_mention --test sessions --test serve`. + Expected: 4, 8 and 6 passed. +- [ ] **5. Run the gate.** `make gate`. Expected last line: `gate: ok`. +- [ ] **6. Log and commit.** `git add crates/gatewayd docs/implementer-log.md Cargo.lock && git commit` + +## Done when + +- The three suites pass; `make gate` prints `gate: ok`. + +## Stop and report if + +- A test wants the mention kept, or stripped somewhere other than at the very start. diff --git a/docs/plans/M4b/09-gatewayd-mm-edit.md b/docs/plans/M4b/09-gatewayd-mm-edit.md new file mode 100644 index 0000000..bbad5f3 --- /dev/null +++ b/docs/plans/M4b/09-gatewayd-mm-edit.md @@ -0,0 +1,67 @@ +# M4b task 09: editing a post, and a username + +**Branch:** `m4b` (run `git switch m4b`; `git status --short` must be empty, otherwise stop) +**Commit subject:** `gatewayd: mm, patch_post and username` + +## Goal + +When an approval ends, `gatewayd` edits its post to say how, naming who answered (M4b spec, +section 6). Two REST calls, checked against Mattermost v11.11.0, and confirmed on the owner's +server (the bot may edit its own posts): + +- `PUT /api/v4/posts/{post_id}/patch` with `{"message": "…"}`: replaces the post's text; the + answer is the post. +- `GET /api/v4/users/{user_id}`: the user, with `username`. + +Every id that goes into a path must be a Mattermost id first, or nothing is sent. + +## Files + +- Copy: `crates/gatewayd/tests/mm_edit.rs` +- Modify: `crates/gatewayd/src/mm/rest.rs`, `docs/implementer-log.md` + +## The changes, exactly + +In `crates/gatewayd/src/mm/rest.rs`, after `struct Channel`: + +```rust +#[derive(Deserialize)] +struct User { + username: String, +} + +/// A Mattermost username: 1 to 64 characters of `a-z`, `0-9`, `.`, `-` and `_`. +fn valid_username(name: &str) -> bool { + (1..=64).contains(&name.len()) + && name.bytes().all(|b| { + b.is_ascii_lowercase() || b.is_ascii_digit() || matches!(b, b'.' | b'-' | b'_') + }) +} +``` + +and two methods at the end of `impl Client`, written like `posts_since` and `create_post`: + +1. `pub fn patch_post(&self, post: &str, message: &str) -> Result<(), MmError>`: + - `post` not `valid_id`: `Err(MmError::Json("a post path must be a Mattermost id".to_string()))`, + nothing sent. + - `PUT` `/api/v4/posts/{post}/patch` with the body + `serde_json::json!({ "message": message }).to_string()`, through `self.call`. + - The answer is a `Post`: `json::(&body)?.check().map(|_| ())`. +2. `pub fn username(&self, user: &str) -> Result`: + - `user` not `valid_id`: `Err(MmError::Json("a user path must be a Mattermost id".to_string()))`. + - `GET` `/api/v4/users/{user}` into `User`; a username that is not `valid_username` is + `Err(MmError::Json("the user has no valid username".to_string()))`. Else `Ok(username)`. + +## Steps + +- [ ] **1. Copy.** `git switch m4b`, then `cp docs/plans/M4b/files/crates/gatewayd/tests/mm_edit.rs crates/gatewayd/tests/` +- [ ] **2. See it fail.** `cargo test -p gatewayd --test mm_edit`. Expected: it does not compile. +- [ ] **3. Make the changes.** Run `cargo fmt --all`. +- [ ] **4. See it pass.** `cargo test -p gatewayd --test mm_edit --test mm_rest`. Expected: 4 and + 10 passed. +- [ ] **5. Run the gate.** `make gate`. Expected last line: `gate: ok`. +- [ ] **6. Log and commit.** `git add crates/gatewayd docs/implementer-log.md Cargo.lock && git commit` + +## Done when + +- Both suites pass; `make gate` prints `gate: ok`. diff --git a/docs/plans/M4b/10-gatewayd-approval-text.md b/docs/plans/M4b/10-gatewayd-approval-text.md new file mode 100644 index 0000000..0a58071 --- /dev/null +++ b/docs/plans/M4b/10-gatewayd-approval-text.md @@ -0,0 +1,58 @@ +# M4b task 10: the text of an approval's posts + +**Branch:** `m4b` (run `git switch m4b`; `git status --short` must be empty, otherwise stop) +**Commit subject:** `gatewayd: approvals::text, the posts that show an approval` + +## Goal + +What an approval looks like in Mattermost (M4b spec, section 6). Pure: text in, text out. + +```text +**Approval 42** · tool `shell` · grant `ops-shell` · taint private · expires 2026-09-24 14:47 UTC +``` +{"command":"ls -la /srv/backups"} +``` +Reply in this thread with `!approve 42` or `!deny 42 `. +``` + +- The arguments are escaped with `proto::escape::escape_json_text` (task 01) and fenced with a run + of backticks longer than any inside them, so nothing in them renders or mentions anyone. +- **Nothing is shortened.** Arguments over 15,000 characters come in parts (at most 4), and only + the last carries the reply line, with the count and the SHA-256 of the whole. More than that, or + a part whose post would be over 15,500 characters, is one post saying it is too long to show, + answerable only in `bxctl`. +- When an approval ends, a line is added to its last post (`Ended`, `ended_line`, `with_ending`). + +## Files + +- Copy: `crates/gatewayd/src/approvals/mod.rs` from `mod.rs-task10` (three lines: the module + comment and `pub mod text;`), the skeleton `crates/gatewayd/src/approvals/text.rs`, and + `crates/gatewayd/tests/approvals_text.rs` +- Modify: `crates/gatewayd/src/lib.rs` (`pub mod approvals;` first, before `pub mod config;`), + `docs/implementer-log.md` + +## The skeleton + +Written: the constants, `Shown`, `Ended`, and **`render`**, which builds the posts from the +helpers below (read it first). To fill, each with its code in its comment: `code`, `fence`, +`longest_run`, `minute`, `name`, `reply_line`, `ended_line`, `with_ending`. + +## Steps + +- [ ] **1. Copy.** `git switch m4b`, then + `mkdir -p crates/gatewayd/src/approvals && cp docs/plans/M4b/files/crates/gatewayd/src/approvals/mod.rs-task10 crates/gatewayd/src/approvals/mod.rs && cp docs/plans/M4b/files/crates/gatewayd/src/approvals/text.rs crates/gatewayd/src/approvals/ && cp docs/plans/M4b/files/crates/gatewayd/tests/approvals_text.rs crates/gatewayd/tests/`. + Add `pub mod approvals;` to `lib.rs`. +- [ ] **2. See it fail.** `cargo test -p gatewayd --test approvals_text`. Expected: it compiles and + 6 tests fail. +- [ ] **3. Fill the eight functions**, `longest_run` first, `cargo check -p gatewayd` after each. +- [ ] **4. See it pass.** `cargo test -p gatewayd --test approvals_text`. Expected: 6 passed. +- [ ] **5. Run the gate.** `cargo fmt --all`, then `make gate`. Expected last line: `gate: ok`. +- [ ] **6. Log and commit.** `git add crates/gatewayd docs/implementer-log.md Cargo.lock && git commit` + +## Done when + +- `cargo test -p gatewayd --test approvals_text` passes; `make gate` prints `gate: ok`. + +## Stop and report if + +- A comment above a `todo!()` does not fit its function, or contradicts a test: quote it. diff --git a/docs/plans/M4b/11-gatewayd-approvals-registry.md b/docs/plans/M4b/11-gatewayd-approvals-registry.md new file mode 100644 index 0000000..2f23e6d --- /dev/null +++ b/docs/plans/M4b/11-gatewayd-approvals-registry.md @@ -0,0 +1,52 @@ +# M4b task 11: the approvals posted here, and the commands that answer them + +**Branch:** `m4b` (run `git switch m4b`; `git status --short` must be empty, otherwise stop) +**Commit subject:** `gatewayd: approvals, the registry and the judging of commands` + +## Goal + +`gatewayd` keeps the approvals it has posted and not yet seen answered. A command answers one only +if **all** of these hold, checked in this order, each refusal answered in the thread with its own +line (M4b spec, section 7): + +1. its author is an approver (`NOT_APPROVER` otherwise); +2. it is well formed (`USAGE` otherwise); +3. that approval is pending here; +4. it was not too long to show (then only `bxctl` answers it); +5. the command is in the approval's own thread; +6. it was posted after the approval's last part. + +An accepted answer is **taken out** of the registry as it is sent to the approval's +`gateway.sock` connection, so a second `!approve` finds nothing. + +## Files + +- Copy: the skeleton `crates/gatewayd/src/approvals/mod.rs` (it replaces task 10's three lines; + `pub mod text;` is still in it), and `crates/gatewayd/tests/approvals_judge.rs` +- Modify: `docs/implementer-log.md` + +## The skeleton + +Written: `NOT_APPROVER`, `USAGE`, `Posted`, `Judged`, `Registry` with `new` and `lock`. To fill: +`add`, `remove`, `holds`, `waiting_sessions` (one line each), and `judge`, whose seven steps are +above its `todo!()`. + +## Steps + +- [ ] **1. Copy.** `git switch m4b`, then + `cp docs/plans/M4b/files/crates/gatewayd/src/approvals/mod.rs crates/gatewayd/src/approvals/ && cp docs/plans/M4b/files/crates/gatewayd/tests/approvals_judge.rs crates/gatewayd/tests/` +- [ ] **2. See it fail.** `cargo test -p gatewayd --test approvals_judge`. Expected: it compiles + and 6 tests fail. +- [ ] **3. Fill the four small functions, then `judge`**, `cargo check -p gatewayd` after each. +- [ ] **4. See it pass.** `cargo test -p gatewayd --test approvals_judge --test approvals_text`. + Expected: 6 and 6 passed. +- [ ] **5. Run the gate.** `cargo fmt --all`, then `make gate`. Expected last line: `gate: ok`. +- [ ] **6. Log and commit.** `git add crates/gatewayd docs/implementer-log.md Cargo.lock && git commit` + +## Done when + +- Both suites pass; `make gate` prints `gate: ok`. + +## Stop and report if + +- A comment above a `todo!()` does not fit its function, or contradicts a test: quote it. diff --git a/docs/plans/M4b/12-gatewayd-state-approvals.md b/docs/plans/M4b/12-gatewayd-state-approvals.md new file mode 100644 index 0000000..07b603d --- /dev/null +++ b/docs/plans/M4b/12-gatewayd-state-approvals.md @@ -0,0 +1,75 @@ +# M4b task 12: approval posts in the state file + +**Branch:** `m4b` (run `git switch m4b`; `git status --short` must be empty, otherwise stop) +**Commit subject:** `gatewayd: state, the posts of open approvals, and the turn of a session` + +## Goal + +Two things the serve loop needs from `state.json` (M4b spec, sections 6 and 8): + +- **Approval posts.** The posts of each approval shown and not yet ended, so that at the next + start the ones left behind can be marked dead. The field is **optional**: an M4a state file + loads unchanged. Like every other field, its ids must be valid, or the file is refused. +- **The turn of a session**: where an approval of that session goes (its thread). + +## Files + +- Copy: `crates/gatewayd/tests/state_approvals.rs` +- Modify: `crates/gatewayd/src/state.rs`, `docs/implementer-log.md` + +## The changes, exactly + +In `crates/gatewayd/src/state.rs`: + +**1.** In `struct StateFile`, after `in_flight: Vec,`: + +```rust + /// Optional, so an M4a state file loads unchanged. + #[serde(default)] + approvals: Vec, +``` + +and after `struct StateFile`: + +```rust +/// The posts of an approval shown in Mattermost and not yet ended (M4b spec, section 8). +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct ApprovalPosts { + pub approval: u64, + pub channel: String, + /// First part first; the last is the one that is edited when it ends. + pub posts: Vec, +} +``` + +**2.** In `StateFile::problem`, after the `in_flight` loop and before `None`: each entry of +`approvals` must have a `valid_id` channel and at least one post, all `valid_id`; otherwise +`Some(format!("approval {}'s posts are not valid", shown.approval))`. + +**3.** Four methods at the end of `impl State`, each saving (`self.save()`) when it changes +something, like the methods already there: + +- `pub fn turn_of(&self, session: &str) -> Option`: the entry of `in_flight` with that + session, cloned. +- `pub fn add_approval(&mut self, shown: ApprovalPosts) -> Result<(), StateError>`: remove any + entry with the same `approval`, push this one, save. +- `pub fn remove_approval(&mut self, approval: u64) -> Result<(), StateError>`: remove the entries + with that `approval`; save only if one was removed. +- `pub fn take_approvals(&mut self) -> Result, StateError>`: take them all out + (`std::mem::take`); save only when there were some; return them. (As `take_in_flight` does.) + +## Steps + +- [ ] **1. Copy.** `git switch m4b`, then `cp docs/plans/M4b/files/crates/gatewayd/tests/state_approvals.rs crates/gatewayd/tests/` +- [ ] **2. See it fail.** `cargo test -p gatewayd --test state_approvals`. Expected: it does not + compile. +- [ ] **3. Make the three changes.** Run `cargo fmt --all`. +- [ ] **4. See it pass.** `cargo test -p gatewayd --test state_approvals --test state --test serve_restart`. + Expected: 4, 4 and 7 passed. +- [ ] **5. Run the gate.** `make gate`. Expected last line: `gate: ok`. +- [ ] **6. Log and commit.** `git add crates/gatewayd docs/implementer-log.md Cargo.lock && git commit` + +## Done when + +- The three suites pass; `make gate` prints `gate: ok`. diff --git a/docs/plans/M4b/13-gatewayd-gateway-sock.md b/docs/plans/M4b/13-gatewayd-gateway-sock.md new file mode 100644 index 0000000..f6b105b --- /dev/null +++ b/docs/plans/M4b/13-gatewayd-gateway-sock.md @@ -0,0 +1,50 @@ +# M4b task 13: `gateway.sock`, `gatewayd`'s side + +**Branch:** `m4b` (run `git switch m4b`; `git status --short` must be empty, otherwise stop) +**Commit subject:** `gatewayd: gateway, the gateway.sock listener and one thread per connection` + +## Goal + +`gatewayd` listens on `gateway.sock`, a Unix socket (not TCP) in its own 0700 directory, the socket +0600, a stale socket from an earlier run replaced. Each `brokerd` connection gets a thread that +speaks the four frames of task 02 (M4b spec, sections 4 and 6). + +A connection thread **only speaks the protocol**. Posting in Mattermost and writing the state file +are the serve loop's, which alone owns them (task 14): the thread asks the loop through a channel, +`ToLoop::Show` (post this; answers come on this channel) and `ToLoop::Ended` (it ended; mark it). + +## Files + +- Copy: the skeleton `crates/gatewayd/src/gateway.rs`, `crates/gatewayd/tests/gateway_sock.rs` +- Modify: `crates/gatewayd/src/lib.rs` (`pub mod gateway;` after `pub mod deliver;`), + `docs/implementer-log.md` + +## The skeleton + +Written: `Log`, `ToLoop`, `Ending`, `Timing`, `frame`, and **`connection`**, the protocol: read +frame 1, ask the loop to post, send frame 2, wait for an approver's answer or for `brokerd` to +close, send frame 3, read frame 4, tell the loop how it ended. Read it first. To fill, each with its +steps in its comment: `listen`, `serve`, `frames` (the same as `brokerd`'s courier's), `end`. + +## Steps + +- [ ] **1. Copy.** `git switch m4b`, then + `cp docs/plans/M4b/files/crates/gatewayd/src/gateway.rs crates/gatewayd/src/ && cp docs/plans/M4b/files/crates/gatewayd/tests/gateway_sock.rs crates/gatewayd/tests/`. + Add `pub mod gateway;` to `lib.rs`. +- [ ] **2. See it fail.** `cargo test -p gatewayd --test gateway_sock`. Expected: it compiles and 6 + tests fail. +- [ ] **3. Fill `end`, `frames`, `listen`, `serve`**, `cargo check -p gatewayd` after each. +- [ ] **4. See it pass.** `cargo test -p gatewayd --test gateway_sock`, three times. Expected: 6 + passed each time, in about a second. +- [ ] **5. Run the gate.** `cargo fmt --all`, then `make gate`. Expected last line: `gate: ok`. +- [ ] **6. Log and commit.** `git add crates/gatewayd docs/implementer-log.md Cargo.lock && git commit` + +## Done when + +- `cargo test -p gatewayd --test gateway_sock` passes three times running; `make gate` prints + `gate: ok`. + +## Stop and report if + +- A comment above a `todo!()` does not fit its function, or contradicts a test: quote it. +- `connection` seems to need a change: it is given. diff --git a/docs/plans/M4b/14-gatewayd-serve-approvals.md b/docs/plans/M4b/14-gatewayd-serve-approvals.md new file mode 100644 index 0000000..5fa4008 --- /dev/null +++ b/docs/plans/M4b/14-gatewayd-serve-approvals.md @@ -0,0 +1,96 @@ +# M4b task 14: the serve loop posts approvals and marks them + +**Branch:** `m4b` (run `git switch m4b`; `git status --short` must be empty, otherwise stop) +**Commit subject:** `gatewayd: serve, posting approvals, marking them, and the listener` + +## Goal + +The serve loop does what `gateway.sock` connections ask (M4b spec, sections 6 and 8): + +- **Show**: place the approval (its session's turn in flight: that thread; otherwise the first + approver's direct messages, as a new thread), post its parts in order, record the posts in the + state file, register it for answers (task 11), and reply with the post ids, or why it could not + be posted. +- **Ended**: forget it, and add how it ended to its last post, naming who answered. +- At the **first connection after a start**, the approval posts the last run left behind are + marked: their last post is replaced by its title and the restart line. +- `run` opens `gateway.sock` before connecting; a failure stops `gatewayd` at start + (`Stop::Start`, pointer `gatewayd-start-failed`). + +## Files + +- Copy: the skeleton `crates/gatewayd/src/serve/approvals.rs`, + `crates/gatewayd/tests/serve_approvals.rs`, `crates/gatewayd/tests/support/brokerd_client.rs`, + and a **new version** of the given `crates/gatewayd/tests/support/fake_mm.rs` (it gains post + edits, user names, and refusing posts) +- Modify: `crates/gatewayd/src/serve/mod.rs`, `docs/implementer-log.md` + +## The skeleton + +Written: `Open`, **`gateway_requests`** (drain the channel, never blocking) and **`show`** (read +both first). To fill, each with its steps in its comment: `place`, `ended`, `mark_left_behind`, +`mark`. + +## `serve/mod.rs`, exactly + +1. `mod approvals;` before `mod handle;`. Imports: `use std::collections::HashMap;`, + `use crate::approvals::Registry;`, `use crate::gateway::{self, ToLoop};`. +2. Three fields at the end of `struct Gateway`: + +```rust + /// The approvals posted here, shared with the turns (for the notice) and judged by commands. + registry: Arc, + /// What `gateway.sock` connections ask of the loop. + gateway_rx: Receiver, + /// The last post of each approval posted here and not yet ended. + open: HashMap, +``` + +3. `Gateway::new` gains a last parameter `gateway_rx: Receiver`, and sets + `registry: Arc::new(Registry::new()), gateway_rx, open: HashMap::new(),`. +4. In `run`, right before `let client = Arc::new(Client::new(…));`: + +```rust + let listener = match gateway::listen(&config.gateway_socket()) { + Ok(listener) => listener, + Err(why) => return Stop::Start(why), + }; + let (gateway_tx, gateway_rx) = mpsc::channel(); + if let Err(e) = gateway::serve( + listener, + gateway_tx, + gateway::Timing::default(), + Arc::clone(&log), + ) { + return Stop::Start(format!("cannot start the gateway.sock thread: {e}")); + } +``` + + and `Gateway::new(config, client, state, log)` becomes + `Gateway::new(config, client, state, log, gateway_rx)`. +5. In `connected`, inside `if !self.restarted { … }`, after the `take_in_flight` loop: + `self.mark_left_behind()?;` +6. In `event_loop`, after `self.finished()?;`: `self.gateway_requests()?;` + +## Steps + +- [ ] **1. Copy.** `git switch m4b`, then + `cp docs/plans/M4b/files/crates/gatewayd/src/serve/approvals.rs crates/gatewayd/src/serve/ && cp docs/plans/M4b/files/crates/gatewayd/tests/serve_approvals.rs crates/gatewayd/tests/ && cp docs/plans/M4b/files/crates/gatewayd/tests/support/brokerd_client.rs docs/plans/M4b/files/crates/gatewayd/tests/support/fake_mm.rs crates/gatewayd/tests/support/` +- [ ] **2. Make the six `serve/mod.rs` changes.** `cargo build -p gatewayd` must succeed. +- [ ] **3. See it fail.** `cargo test -p gatewayd --test serve_approvals`. Expected: 4 fail, 1 + passes (the socket is there while `gatewayd` runs). +- [ ] **4. Fill `mark`, `place`, `ended`, `mark_left_behind`**, `cargo check -p gatewayd` after + each. +- [ ] **5. See it pass.** `cargo test -p gatewayd --test serve_approvals`, three times: 5 passed. + Then `cargo test -p gatewayd`: every suite passes. +- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`. Expected last line: `gate: ok`. +- [ ] **7. Log and commit.** `git add crates/gatewayd docs/implementer-log.md Cargo.lock && git commit` + +## Done when + +- `cargo test -p gatewayd` passes; `make gate` prints `gate: ok`. + +## Stop and report if + +- A comment above a `todo!()` does not fit its function, or contradicts a test: quote it. +- `show` or `gateway_requests` seems to need a change: they are given. diff --git a/docs/plans/M4b/15-gatewayd-answers.md b/docs/plans/M4b/15-gatewayd-answers.md new file mode 100644 index 0000000..9b6ec88 --- /dev/null +++ b/docs/plans/M4b/15-gatewayd-answers.md @@ -0,0 +1,152 @@ +# M4b task 15: commands answer approvals; typing pauses; the notice + +**Branch:** `m4b` (run `git switch m4b`; `git status --short` must be empty, otherwise stop) +**Commit subject:** `gatewayd: approvals answered from Mattermost; typing paused; the notice` + +## Goal + +The last three pieces (M4b spec, section 7): + +1. **Commands.** `Route::Approval` (task 08) goes to the registry's `judge` (task 11): an accepted + answer goes to `brokerd`; anything else is answered in the thread with its reason. M4a's notice + `M4B_COMMAND` goes away. +2. **Typing.** A turn waiting for its approval is not working: no `user_typing` for its thread + while the registry holds an approval of its session. +3. **The notice.** On `approval_pending`, the turn waits up to 10 s for `brokerd` to bring that + approval here (`Registry::was_posted`); only if it never does, it posts `waiting for approval + : brokerd has not sent it here; answer it with bxctl approvals`. + +## Files + +- Copy: `crates/gatewayd/tests/serve_answers.rs`, `crates/gatewayd/tests/deliver_notice.rs`, and + **new versions** of three given files: `crates/gatewayd/tests/deliver.rs` (passes a `Notice`; + the new notice text), `crates/gatewayd/tests/serve.rs` (`!approve 3` is now "approval 3 is not + pending here"), `crates/gatewayd/tests/support/gateway.rs` (`Tuning` has `notice`) +- Modify: `crates/gatewayd/src/approvals/mod.rs`, `crates/gatewayd/src/deliver.rs`, + `crates/gatewayd/src/serve/mod.rs`, `crates/gatewayd/src/serve/handle.rs`, + `crates/gatewayd/src/sessions.rs`, `docs/implementer-log.md` + +## The changes, exactly + +**`approvals/mod.rs`**: every approval ever posted, for the notice. `Registry` gains a field +`ever: Mutex>` (doc comment: "Every approval ever posted here, answered or not: for +the notice."; import `HashSet` beside `HashMap`); `add` also inserts `posted.approval` into it +(`self.ever.lock().unwrap_or_else(|p| p.into_inner()).insert(…)`, before inserting into `posted`); +and a new method: + +```rust + /// Was approval `id` ever posted here? + pub fn was_posted(&self, id: u64) -> bool { + self.ever + .lock() + .unwrap_or_else(|p| p.into_inner()) + .contains(&id) + } +``` + +**`deliver.rs`**: +- `approval_text` returns + `format!("waiting for approval {approval}: brokerd has not sent it here; answer it with bxctl approvals")`. +- After it, and `use std::time::{Duration, Instant};` at the top: + +```rust +/// How long a turn waits for `brokerd` to bring its approval here before posting the notice. +pub const NOTICE_WAIT: Duration = Duration::from_secs(10); + +/// Whether an approval was brought here (and so is posted already), and how long to wait for it. +pub struct Notice<'a> { + pub posted: &'a dyn Fn(u64) -> bool, + pub wait: Duration, +} +``` + + and write `impl Notice<'_> { fn arrives(&self, id: u64) -> bool }`: loop: `(self.posted)(id)` + → true; past `Instant::now() + self.wait` (taken once, before the loop) → false; otherwise sleep + `Duration::from_millis(50).min()` (`saturating_duration_since`). +- `deliver` gains a last parameter `notice: &Notice`, and its `ApprovalPending` branch posts the + notice only when it does not arrive: + +```rust + // The approval post itself comes through `gateway.sock`; the notice only when it does not. + if let TurnEvent::ApprovalPending { approval, .. } = event + && !notice.arrives(*approval) + { + post(poster, &batch.thread, approval_text(*approval), log); + } +``` + +**`serve/mod.rs`**: `Tuning` gains `pub notice: Duration,` (doc: "How long a turn waits for its +approval to come through `gateway.sock` before the notice."), `crate::deliver::NOTICE_WAIT` in its +`Default`; `Gateway` gains a field `notice: Duration` (doc "`Tuning::notice`."), set by a new last +parameter of `Gateway::new`, which `run` passes as `tuning.notice`. + +**`serve/handle.rs`**: +- The `Route::Approval` arm becomes: + +```rust + Route::Approval { + thread, + command, + user, + post: command_post, + at, + } => { + let approvers = self.config.approvers(); + let judged = self.registry.judge( + &command, + &user, + &thread.root, + &command_post, + at, + approvers, + ); + if let Judged::Reply(text) = judged { + self.post(&thread.channel, &thread.root, &text); + } + } +``` + +- In `start`, the thread gets the registry and the wait, and passes a `Notice`: + +```rust + let (registry, wait) = (Arc::clone(&self.registry), self.notice); + let spawned = std::thread::Builder::new().spawn(move || { + let posted = |id: u64| registry.was_posted(id); + let notice = Notice { + posted: &posted, + wait, + }; + deliver(client.as_ref(), &socket, &batch, &|line| log(line), ¬ice); + let _ = done.send(batch.session); + }); +``` + +- In `typing`, before the loop: `let waiting = self.registry.waiting_sessions();`, and at the top + of the loop body: `if waiting.contains(&format!("mm-{}", thread.root)) { continue; }` (with the + comment "A turn waiting for its approval is not working: no typing in its thread."). +- Imports: `use crate::approvals::Judged;`; `Notice` beside `deliver`; `M4B_COMMAND` removed. + +**`sessions.rs`**: delete `pub const M4B_COMMAND`. + +## Steps + +- [ ] **1. Copy.** `git switch m4b`, then + `cp docs/plans/M4b/files/crates/gatewayd/tests/serve_answers.rs docs/plans/M4b/files/crates/gatewayd/tests/deliver_notice.rs docs/plans/M4b/files/crates/gatewayd/tests/deliver.rs docs/plans/M4b/files/crates/gatewayd/tests/serve.rs crates/gatewayd/tests/ && cp docs/plans/M4b/files/crates/gatewayd/tests/support/gateway.rs crates/gatewayd/tests/support/` +- [ ] **2. See it fail.** `cargo test -p gatewayd --test serve_answers --test deliver_notice`. + Expected: they do not compile (`deliver::Notice` and `Tuning::notice` do not exist yet). +- [ ] **3. Make the changes**, one file at a time, `cargo check -p gatewayd` after each. Run + `cargo fmt --all`. +- [ ] **4. See it pass.** `cargo test -p gatewayd --test serve_answers --test deliver_notice --test deliver --test serve`, + three times: 4, 4, 9 and 6 passed. Then `cargo test -p gatewayd`: every suite passes. +- [ ] **5. Run the gate.** `make gate`. Expected last line: `gate: ok`, with about 845 tests. +- [ ] **6. Log and commit.** `git add crates/gatewayd docs/implementer-log.md Cargo.lock && git commit` + +This is the last task of M4b. Stop after the commit. + +## Done when + +- `cargo test -p gatewayd` passes; `make gate` prints `gate: ok`. + +## Stop and report if + +- A test passes only sometimes, or waits much longer than stated. diff --git a/docs/plans/M4b/README.md b/docs/plans/M4b/README.md new file mode 100644 index 0000000..21db49f --- /dev/null +++ b/docs/plans/M4b/README.md @@ -0,0 +1,75 @@ +# M4b implementation plan: approvals over Mattermost + +> **For the implementing model:** do not work from this file. The owner gives you one task file at +> a time (`01-…` to `15-…`). This file is the index for the owner and the reviewer. + +**Goal:** an `ask` call is posted in Mattermost, in the thread that asked for it (or, for a session +not in Mattermost, in the first approver's direct messages), and an approver answers it there with +`!approve ` or `!deny [reason]`. The answer is recorded exactly as `bxctl`'s is, naming +the approver and the post; the post then shows what came of it. Nobody else can answer, and no +post is left looking answerable when it is not. + +**Architecture:** `proto` gains the shared escaping (01) and the four `gateway.sock` messages (02); +`bxctl` gains `deny` (03). `brokerd` records the answering post (04), reads `[gateway]` (05), and +gets the **courier**, which carries each pending approval to `gatewayd` and its answer back (06). +`gatewayd` reads `[approvals]` (07), strips a leading mention of itself and recognises approval +commands (08), can edit a post and look up a name (09), renders an approval's posts (10), keeps +the approvals it posted and judges commands (11), records their posts in the state file (12), +listens on `gateway.sock` (13), posts and marks approvals from its serve loop (14), and wires the +commands, the typing pause and the fallback notice (15). + +**Spec:** `docs/specs/2026-09-24-m4b-approvals.md` (approved; P16 applied to the brief). Runbook: +task 06 adds `gateway-unreachable`, task 07 widens `gatewayd-start-failed`. + +**No task needs Mattermost, Podman or a network.** Everything runs against fakes: a fake `gatewayd` +for `brokerd`, and for `gatewayd` a fake Mattermost, a fake `loopd` and a fake `brokerd` client. + +## Global constraints + +- Everything in `AGENTS.md`, including "Lessons from earlier reviews". +- No new dependency. +- Branch `m4b`. One task, one fresh OpenCode session, one commit. Run `cargo fmt --all` before the + gate. Review happens once, after task 15. +- **New files** come as skeletons: types, constants and the glue written, `todo!()` bodies with the + code to write in the comment above. **Existing files** are changed by the exact edits in the + task, never replaced whole: a whole-file copy would overwrite an earlier task's work. +- Some tasks hand over **new versions of given test files**, because the change is in what the + test must see (a new field, a new text). Copy them over the old ones as the task says; they are + still not to be edited. + +## Tasks + +Every task had a reference implementation, built in this order from master with the gate at every +step (tip T26), its given tests run against it, and mutations of it caught by them (tip T28); then +the reference was moved off this machine (tip T18). Every skeleton was checked to compile at its +task's end state and fail its tests, and every comment above a `todo!()` was read beside its +signature (tip T29). + +| # | File | Delivers | Tests | Check | +|---|---|---|---|---| +| 01 | `01-proto-escape.md` | `proto::escape`, moved from `bxctl` | `proto/tests/escape.rs` | reference; `bxctl` unchanged | +| 02 | `02-proto-gateway-wire.md` | the four `gateway.sock` messages | `gateway_wire.rs`, 8 fixtures | reference | +| 03 | `03-bxctl-deny.md` | `bxctl deny` | `cli_deny.rs` | reference | +| 04 | `04-brokerd-answer-post.md` | answers record the post; `Table::contains` | `ledger_post.rs`; 3 given tests updated | reference | +| 05 | `05-brokerd-config-gateway.md` | `[gateway]` | `config_gateway.rs`; `rig.rs` updated | reference | +| 06 | `06-brokerd-courier.md` | the courier | `courier.rs`, `support/fake_gateway.rs` | reference; 10 mutations caught; no test hangs | +| 07 | `07-gatewayd-config-approvals.md` | `[approvals]` | `config_approvals.rs` | reference | +| 08 | `08-gatewayd-mention-commands.md` | the leading mention; `Route::Approval` | `sessions_mention.rs`; `sessions.rs`, `serve.rs` updated | reference; 7 mutations, 5 caught, 2 that cannot change behaviour | +| 09 | `09-gatewayd-mm-edit.md` | `patch_post`, `username` | `mm_edit.rs` | reference; the bot's edit checked on the owner's server | +| 10 | `10-gatewayd-approval-text.md` | the posts' text | `approvals_text.rs` | reference; 7 mutations caught | +| 11 | `11-gatewayd-approvals-registry.md` | the registry, `judge` | `approvals_judge.rs` | reference; 6 mutations caught | +| 12 | `12-gatewayd-state-approvals.md` | approval posts in `state.json`; `turn_of` | `state_approvals.rs` | reference | +| 13 | `13-gatewayd-gateway-sock.md` | the listener and connection threads | `gateway_sock.rs` | reference; 6 mutations, 5 caught, 1 that cannot change behaviour | +| 14 | `14-gatewayd-serve-approvals.md` | posting, marking, the restart, the listener in `run` | `serve_approvals.rs`, `support/brokerd_client.rs`; `fake_mm.rs` updated | reference; 5 mutations caught | +| 15 | `15-gatewayd-answers.md` | commands judged, typing paused, the notice | `serve_answers.rs`, `deliver_notice.rs`; `deliver.rs`, `serve.rs`, `support/gateway.rs` updated | reference; 4 mutations caught | + +At the end of task 15: about 845 tests (768 before task 01). + +## Running it + +```sh +git switch m4b +BOXMAKER_MODEL=straylight/ornith-1.5-35b-a3b tools/run-plan.sh docs/plans/M4b +``` + +Keep the OpenCode TUI closed while it runs. diff --git a/docs/plans/M4b/files/crates/brokerd/src/courier.rs b/docs/plans/M4b/files/crates/brokerd/src/courier.rs new file mode 100644 index 0000000..04ab5e4 --- /dev/null +++ b/docs/plans/M4b/files/crates/brokerd/src/courier.rs @@ -0,0 +1,235 @@ +//! The courier: one pending approval carried to `gatewayd` over `gateway.sock`, and its answer +//! back (M4b spec, section 5). It runs beside the connection thread that waits for the verdict, +//! and is one more taker of the table's entry, answering it exactly as `bxctl` does. + +use std::os::unix::net::UnixStream; +use std::path::Path; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::mpsc::{self, Receiver, RecvTimeoutError}; +use std::time::{Duration, Instant}; + +use proto::{ + ApprovalOutcome, ApprovalReply, ApprovalRequest, DecisionRecord, DenyReason, Envelope, + FrameError, GatewayOutcome, Message, PendingApproval, Timestamp, read_frame, +}; + +use crate::admin; +use crate::broker::{Broker, send}; +use crate::ledger::Answer; + +pub const RUNBOOK: &str = "see docs/runbook.md#gateway-unreachable"; + +/// How long to wait before trying again, and how often to look at the table meanwhile. +#[derive(Debug, Clone, Copy)] +pub struct Timing { + pub retry: Duration, + pub tick: Duration, +} + +impl Default for Timing { + fn default() -> Self { + Timing { + retry: Duration::from_secs(5), + tick: Duration::from_secs(1), + } + } +} + +/// How one attempt ended. +enum Attempt { + /// The approval is answered or no longer pending: the courier is done. + Done, + /// It was never posted: `gatewayd` could not be reached, or could not post. + Unreachable(String), + /// It was posted, then `gatewayd` went away or broke the protocol. + Lost(String), +} + +/// Run `wait` (the connection thread's wait for the verdict) with a courier beside it, when +/// `[gateway]` is set. The courier stops when `wait` returns. +pub fn alongside(broker: &Broker, info: &PendingApproval, wait: impl FnOnce() -> T) -> T { + let Some(socket) = broker.cfg().gateway_socket() else { + return wait(); + }; + let stop = AtomicBool::new(false); + std::thread::scope(|scope| { + let started = std::thread::Builder::new().spawn_scoped(scope, || { + carry(broker, &socket, info, &stop, Timing::default()) + }); + if let Err(e) = started { + broker.log(&format!( + "brokerd: approval {}: cannot start its courier: {e}; bxctl can still answer it", + info.approval + )); + } + let out = wait(); + stop.store(true, Ordering::SeqCst); + out + }) +} + +/// Carry approval `info` to `gatewayd` until it is answered or no longer pending. +pub fn carry( + broker: &Broker, + socket: &Path, + info: &PendingApproval, + stop: &AtomicBool, + timing: Timing, +) { + let id = info.approval; + let mut warned = false; + loop { + if finished(broker, id, stop) { + return; + } + match attempt(broker, socket, info, stop, timing, warned) { + Attempt::Done => return, + Attempt::Unreachable(why) => { + if !warned { + warned = true; + broker.log(&format!( + "brokerd: approval {id}: gatewayd is not reachable at {}: {why}; bxctl can still answer it\n{RUNBOOK}", + socket.display() + )); + } + pause(broker, id, stop, timing); + } + Attempt::Lost(why) => { + broker.log(&format!("brokerd: approval {id}: {why}; posting it again")); + pause(broker, id, stop, timing); + } + } + } +} + +/// The courier's work is over: stopped, or the entry has left the table. +fn finished(broker: &Broker, id: u64, stop: &AtomicBool) -> bool { + // `stop.load(Ordering::SeqCst) || !broker.table().contains(id)`. + todo!() +} + +/// Wait `timing.retry`, in ticks, unless the courier's work ends first. +fn pause(broker: &Broker, id: u64, stop: &AtomicBool, timing: Timing) { + // `let until = Instant::now() + timing.retry;` then, while `Instant::now() < until` and not + // `finished(broker, id, stop)`, sleep + // `timing.tick.min(until.saturating_duration_since(Instant::now()))`. + todo!() +} + +/// Every frame `gatewayd` sends, read whole on a thread of its own, so the courier can wait with +/// a timeout without losing half a frame. Shutting the stream down ends the thread. +fn frames(stream: &UnixStream) -> std::io::Result>> { + // 1. `let mut reader = stream.try_clone()?;` and `let (tx, rx) = mpsc::channel();`. + // 2. `std::thread::Builder::new().spawn(move || loop { ... })?`, where the loop body is: `let + // frame = read_frame(&mut reader); let last = frame.is_err(); if tx.send(frame).is_err() || + // last { return; }`. + // 3. `Ok(rx)`. + todo!() +} + +/// One connection: post, then wait for the answer, looking at the table every tick. +fn attempt( + broker: &Broker, + socket: &Path, + info: &PendingApproval, + stop: &AtomicBool, + timing: Timing, + warned: bool, +) -> Attempt { + let mut stream = match UnixStream::connect(socket) { + Ok(stream) => stream, + Err(e) => return Attempt::Unreachable(e.to_string()), + }; + let rx = match frames(&stream) { + Ok(rx) => rx, + Err(e) => return Attempt::Unreachable(e.to_string()), + }; + let ended = exchange(broker, &mut stream, &rx, info, stop, timing, warned); + let _ = stream.shutdown(std::net::Shutdown::Both); + ended +} + +/// Frames 1 to 4 of section 4 on an open connection. +fn exchange( + broker: &Broker, + stream: &mut UnixStream, + rx: &Receiver>, + info: &PendingApproval, + stop: &AtomicBool, + timing: Timing, + warned: bool, +) -> Attempt { + let id = info.approval; + let request = Message::ApprovalRequest(ApprovalRequest { + approval: info.clone(), + }); + if !send(stream, 1, true, request) { + return Attempt::Unreachable("the request could not be sent".to_string()); + } + let mut posted = false; + loop { + let frame = match rx.recv_timeout(timing.tick) { + Ok(Ok(frame)) => frame, + Err(RecvTimeoutError::Timeout) => { + if finished(broker, id, stop) { + return Attempt::Done; + } + continue; + } + Ok(Err(_)) | Err(RecvTimeoutError::Disconnected) => { + return lost(posted, "gatewayd went away"); + } + }; + match (frame.id, frame.r#final, frame.msg) { + (1, false, Message::ApprovalPosted(_)) if !posted => { + posted = true; + if warned { + broker.log(&format!("brokerd: approval {id} posted in Mattermost")); + } + } + (1, true, Message::Error(e)) if !posted => { + return Attempt::Unreachable(format!("gatewayd could not post it: {}", e.detail)); + } + (1, true, Message::ApprovalReply(reply)) if posted => { + if reply.approval != id { + return lost(posted, "gatewayd answered another approval"); + } + if let Err(why) = reply.check() { + return lost(posted, &format!("gatewayd sent a bad answer ({why})")); + } + let outcome = answer(broker, id, reply); + let _ = send( + stream, + 2, + true, + Message::ApprovalOutcome(ApprovalOutcome { outcome }), + ); + return Attempt::Done; + } + _ => return lost(posted, "gatewayd broke the protocol"), + } + } +} + +fn lost(posted: bool, why: &str) -> Attempt { + if posted { + Attempt::Lost(why.to_string()) + } else { + Attempt::Unreachable(why.to_string()) + } +} + +/// Take the entry and answer it as `bxctl` would, with who answered and in which post. +fn answer(broker: &Broker, id: u64, reply: ApprovalReply) -> GatewayOutcome { + // 1. `let Some(entry) = broker.table().take(id) else { return GatewayOutcome::Gone {}; };` + // 2. `let (by, post) = (Some(reply.by), Some(reply.post));` and the `Answer`: `Answer::Approved + // { by, post }` when `reply.approved`, else `Answer::Refused { by, post, reason: + // reply.reason }`. + // 3. `match (reply.approved, admin::answer(broker, entry, answer, Timestamp::now()))`: (true, + // DecisionRecord::Denied { reason }) -> `GatewayOutcome::Denied { reason }`; (true, _) -> + // `GatewayOutcome::Runs {}`; (false, DecisionRecord::Denied { reason: + // DenyReason::ApprovalRefused }) -> `GatewayOutcome::Refused {}`; (false, + // DecisionRecord::Denied { reason }) -> `GatewayOutcome::Denied { reason }` (the refusal + // could not be recorded); (false, _) -> `GatewayOutcome::Refused {}`. + todo!() +} diff --git a/docs/plans/M4b/files/crates/brokerd/tests/broker_pending.rs b/docs/plans/M4b/files/crates/brokerd/tests/broker_pending.rs new file mode 100644 index 0000000..89402f2 --- /dev/null +++ b/docs/plans/M4b/files/crates/brokerd/tests/broker_pending.rs @@ -0,0 +1,232 @@ +//! A call an `ask` grant matched: the pending frame, the table entry, the wait, and a requester +//! that goes away. The tests answer entries by hand, as `admin` will. Do not edit. + +#[path = "support/client.rs"] +mod client; +#[path = "support/rig.rs"] +mod rig; +#[path = "support/runtime.rs"] +mod runtime; +#[path = "support/sink.rs"] +mod sink; +#[path = "support/tmp.rs"] +mod tmp; + +use std::os::unix::net::UnixStream; +use std::sync::Arc; +use std::time::{Duration, Instant}; + +use brokerd::broker::{self, Broker, GONE}; +use brokerd::ledger::Answer; +use client::{Serve, next, open}; +use proto::{ + AuditEvent, DataClass, DecisionRecord, DenyReason, Message, ResultStatus, Timestamp, + ToolResponse, +}; +use rig::{Rig, grant_text, request}; +use runtime::Recording; + +fn ask_notes(rig: &Rig, extra: &str) { + rig.grant( + "notes", + &grant_text("read_file", "ask", extra, "paths = [\"/n\"]"), + ); +} + +/// Sends a call and reads its pending frame. Returns the connection and the frame's values. +fn start(broker: &Arc, arguments: &str) -> (UnixStream, u64, Timestamp) { + let req = request("s1", 9, "read_file", arguments); + let mut stream = open(broker, broker::handle, 9, Message::ToolRequest(req)); + let env = next(&mut stream); + assert_eq!((env.id, env.r#final), (9, false), "{env:?}"); + match env.msg { + Message::ToolResponse(ToolResponse::PendingApproval { approval, expires }) => { + (stream, approval, expires) + } + other => panic!("{other:?}"), + } +} + +/// What `admin` does for an entry it took: record the answer, send the verdict. +fn answer_by_hand(broker: &Broker, id: u64, answer: Answer) -> DecisionRecord { + let entry = broker.table().take(id).expect("the entry is pending"); + answer_entry(broker, entry, answer) +} + +fn answer_entry( + broker: &Broker, + entry: brokerd::approvals::Entry, + answer: Answer, +) -> DecisionRecord { + let grants = broker.grants(); + let done = broker.ledger().answer( + entry.ask, + entry.info.approval, + answer, + &grants, + Timestamp::now(), + ); + entry.reply.send(done.verdict).unwrap(); + done.outcome +} + +fn approved() -> Answer { + Answer::Approved { + by: Some("bxctl".to_string()), + post: None, + } +} + +fn final_answer(stream: &mut UnixStream) -> ToolResponse { + let env = next(stream); + assert!(env.r#final, "{env:?}"); + match env.msg { + Message::ToolResponse(r) => r, + other => panic!("{other:?}"), + } +} + +fn eventually(what: &str, mut done: impl FnMut() -> bool) { + let until = Instant::now() + Duration::from_secs(10); + while !done() { + assert!(Instant::now() < until, "never happened: {what}"); + std::thread::sleep(Duration::from_millis(20)); + } +} + +#[test] +fn an_ask_call_waits_is_listed_and_runs_when_approved() { + let rig = Rig::new("pending-approve"); + ask_notes(&rig, ""); + let rt = Recording::answering("the notes"); + let broker = rig.broker(&rt); + let before = Timestamp::now(); + // Spaced JSON: the table shows the arguments as policy parsed them, not as sent. + let (mut stream, approval, expires) = start(&broker, r#"{ "path" : "/n/a" }"#); + assert_eq!(approval, 0, "the approval id is the decision's seq"); + let items = broker.table().list(); + assert_eq!(items.len(), 1); + let item = &items[0]; + assert_eq!(item.approval, 0); + assert_eq!((item.session.as_str(), item.call.0), ("s1", 9)); + assert_eq!(item.tool, "read_file"); + assert_eq!(item.arguments, r#"{"path":"/n/a"}"#); + assert_eq!( + (item.grant.as_str(), item.taint), + ("notes", DataClass::Private) + ); + assert!(item.created >= before); + assert_eq!(item.expires, expires); + assert_eq!( + expires.unix_millis() - item.created.unix_millis(), + 900_000, + "now plus ttl_ms" + ); + assert_eq!(rt.count(), 0, "nothing runs while it waits"); + + assert_eq!( + answer_by_hand(&broker, 0, approved()), + DecisionRecord::Ask {} + ); + assert!(matches!( + final_answer(&mut stream), + ToolResponse::Result { .. } + )); + assert_eq!(rt.count(), 1); + let events = rig.events(); + assert!( + matches!( + events.as_slice(), + [ + AuditEvent::Decision { .. }, + AuditEvent::Approval { decision: 0, .. }, + AuditEvent::Result { decision: 0, .. } + ] + ), + "{events:?}" + ); +} + +#[test] +fn the_expiry_is_the_grants_when_that_is_earlier() { + let rig = Rig::new("pending-grant-expiry"); + let soon = Timestamp::from_unix_millis(Timestamp::now().unix_millis() + 60_000).unwrap(); + ask_notes(&rig, &format!("expires = \"{}\"", soon.to_rfc3339())); + let rt = Recording::answering("x"); + let broker = rig.broker(&rt); + let (_stream, _, expires) = start(&broker, r#"{"path":"/n/a"}"#); + assert_eq!(expires, soon); +} + +#[test] +fn a_denied_verdict_reaches_the_requester_and_nothing_runs() { + let rig = Rig::new("pending-refuse"); + ask_notes(&rig, ""); + let rt = Recording::answering("x"); + let broker = rig.broker(&rt); + let (mut stream, approval, _) = start(&broker, r#"{"path":"/n/a"}"#); + let refused = Answer::Refused { + by: Some("bxctl".to_string()), + post: None, + reason: None, + }; + answer_by_hand(&broker, approval, refused); + assert_eq!( + final_answer(&mut stream), + ToolResponse::Denied { + reason: DenyReason::ApprovalRefused + } + ); + assert_eq!(rt.count(), 0); +} + +#[test] +fn a_requester_that_leaves_takes_its_own_entry_and_nothing_is_written() { + let rig = Rig::new("pending-leave"); + ask_notes(&rig, ""); + let rt = Recording::answering("x"); + let broker = rig.broker(&rt); + let (stream, _, _) = start(&broker, r#"{"path":"/n/a"}"#); + drop(stream); + // The waiting thread looks at its connection every second. + eventually("the entry is removed", || broker.table().list().is_empty()); + std::thread::sleep(Duration::from_millis(200)); + assert_eq!(rig.events().len(), 1, "only the decision"); + let report = brokerd::audit::verify_dir(&rig.cfg.audit_dir()).unwrap(); + assert_eq!(report.abandoned, [0]); + assert!(report.failure.is_none()); + assert_eq!(rt.count(), 0); +} + +#[test] +fn a_requester_gone_at_the_last_look_runs_nothing_and_closes_the_call() { + let rig = Rig::new("pending-lastlook"); + ask_notes(&rig, ""); + let rt = Recording::answering("x"); + let broker = rig.broker(&rt); + let (stream, approval, _) = start(&broker, r#"{"path":"/n/a"}"#); + // Take the entry first, so the waiting thread cannot take it back when it sees the + // connection gone; then approve. + let entry = broker.table().take(approval).unwrap(); + drop(stream); + answer_entry(&broker, entry, approved()); + eventually("the call is closed", || rig.events().len() == 3); + match &rig.events()[2] { + AuditEvent::Result { + status, + decision, + sha256, + bytes, + .. + } => { + assert_eq!((*status, *decision), (ResultStatus::Failed, 0)); + assert_eq!(*sha256, proto::sha256(GONE.as_bytes()).unwrap()); + assert_eq!(*bytes, GONE.len() as u64); + } + other => panic!("{other:?}"), + } + assert_eq!(GONE, "the requester went away"); + assert_eq!(rt.count(), 0); + let report = brokerd::audit::verify_dir(&rig.cfg.audit_dir()).unwrap(); + assert!(report.unfinished.is_empty() && report.abandoned.is_empty()); +} diff --git a/docs/plans/M4b/files/crates/brokerd/tests/broker_sequence.rs b/docs/plans/M4b/files/crates/brokerd/tests/broker_sequence.rs new file mode 100644 index 0000000..021e6f0 --- /dev/null +++ b/docs/plans/M4b/files/crates/brokerd/tests/broker_sequence.rs @@ -0,0 +1,242 @@ +//! Properties over sequences of calls, through `broker::handle` with many threads at once: +//! the log verifies and no `seq` repeats; taint never goes down; every `Result` follows the +//! record that let its call run; the runtime sees a call only after `allowed` or an approval. +//! Do not edit. + +#[path = "support/client.rs"] +mod client; +#[path = "support/rig.rs"] +mod rig; +#[path = "support/runtime.rs"] +mod runtime; +#[path = "support/sink.rs"] +mod sink; +#[path = "support/tmp.rs"] +mod tmp; + +use std::collections::BTreeMap; +use std::sync::Arc; +use std::sync::atomic::{AtomicBool, Ordering}; +use std::time::Duration; + +use brokerd::ledger::Answer; +use client::{Serve, call}; +use proto::{ + AuditEvent, AuditRecord, CallId, DataClass, DecisionRecord, Message, SessionId, Timestamp, + ToolResponse, +}; +use rig::{Rig, grant_text, request}; +use runtime::Recording; + +fn grants(rig: &Rig) { + let with = |class: &str, max: &str| { + grant_text("read_file", "auto", "", "") + .replace( + "result_class = \"private\"", + &format!("result_class = \"{class}\""), + ) + .replace("max_taint = \"secret\"", &format!("max_taint = \"{max}\"")) + }; + rig.grant("notes", &(with("private", "secret") + "paths = [\"/n\"]\n")); + rig.grant("keys", &(with("secret", "secret") + "paths = [\"/k\"]\n")); + // Stops applying once a session has read a secret. + rig.grant( + "public", + &(with("public", "private") + "paths = [\"/p\"]\n"), + ); + rig.grant( + "never", + &grant_text("read_file", "deny", "", "paths = [\"/d\"]"), + ); + rig.grant( + "asked", + &grant_text("read_file", "ask", "", "paths = [\"/a\"]"), + ); +} + +/// Which call ran on what authority: (session, call) of every Decision that allowed and every +/// Approval whose re-decision let the call run, keyed by the decision's seq. +fn check_sequence(records: &[AuditRecord]) -> usize { + let mut may_run: BTreeMap = BTreeMap::new(); + let mut taint: BTreeMap = BTreeMap::new(); + let mut results = 0; + for (i, r) in records.iter().enumerate() { + assert_eq!(r.seq, i as u64, "no seq repeats or skips"); + match &r.event { + AuditEvent::Decision { + session, + call, + outcome: DecisionRecord::Allowed {}, + .. + } => { + may_run.insert(r.seq, (session.clone(), *call)); + } + AuditEvent::Approval { + session, + call, + decision, + outcome: DecisionRecord::Allowed {} | DecisionRecord::Ask {}, + .. + } => { + may_run.insert(*decision, (session.clone(), *call)); + } + AuditEvent::Result { + session, + call, + decision, + taint_after, + .. + } => { + results += 1; + let allowed = may_run.remove(decision); + assert_eq!( + allowed, + Some((session.clone(), *call)), + "seq {}: a Result with no record letting its call run", + r.seq + ); + let before = taint.insert(session.as_str().to_string(), *taint_after); + assert!( + before.is_none_or(|b| b <= *taint_after), + "seq {}: taint went down", + r.seq + ); + } + _ => {} + } + } + assert!( + may_run.is_empty(), + "calls allowed but never finished: {may_run:?}" + ); + results +} + +#[test] +fn eight_threads_two_sessions_fifty_calls_each() { + let rig = Rig::new("sequence-many"); + grants(&rig); + let rt = Recording::answering("content"); + let broker = rig.broker(&rt); + let paths = ["/n/x", "/k/x", "/p/x", "/d/x", "/none/x"]; + let threads: Vec<_> = (0..8u64) + .map(|t| { + let broker = Arc::clone(&broker); + std::thread::spawn(move || { + let session = if t % 2 == 0 { "s-even" } else { "s-odd" }; + for i in 0..50u64 { + let path = paths[((t + i) % 5) as usize]; + let args = format!(r#"{{"path":"{path}"}}"#); + let req = request(session, t * 1000 + i, "read_file", &args); + let frames = call(&broker, req); + assert_eq!(frames.len(), 1); + } + }) + }) + .collect(); + for t in threads { + t.join().unwrap(); + } + let report = brokerd::audit::verify_dir(&rig.cfg.audit_dir()).unwrap(); + assert!(report.failure.is_none(), "{:?}", report.failure); + assert!(report.unfinished.is_empty()); + let records = rig.records(); + let decisions = records + .iter() + .filter(|r| matches!(r.event, AuditEvent::Decision { .. })) + .count(); + assert_eq!(decisions, 400); + let results = check_sequence(&records); + assert_eq!( + rt.count(), + results, + "the runtime saw exactly the allowed calls" + ); + assert!(results > 0 && results < 400); + // Both sessions read a secret. + for s in ["s-even", "s-odd"] { + let state = rig.state().read(&SessionId::new(s).unwrap()).unwrap(); + assert_eq!(state.taint, DataClass::Secret); + } +} + +#[test] +fn approved_and_refused_calls_run_only_after_an_approval() { + let rig = Rig::new("sequence-ask"); + grants(&rig); + let rt = Recording::answering("content"); + let broker = rig.broker(&rt); + // An owner who approves even ids and refuses odd ones, as fast as they appear. + let stop = Arc::new(AtomicBool::new(false)); + let owner = { + let (broker, stop) = (Arc::clone(&broker), Arc::clone(&stop)); + std::thread::spawn(move || { + while !stop.load(Ordering::SeqCst) { + for item in broker.table().list() { + let Some(entry) = broker.table().take(item.approval) else { + continue; + }; + let answer = if item.approval % 2 == 0 { + Answer::Approved { + by: None, + post: None, + } + } else { + Answer::Refused { + by: None, + post: None, + reason: None, + } + }; + let grants = broker.grants(); + let done = broker.ledger().answer( + entry.ask, + item.approval, + answer, + &grants, + Timestamp::now(), + ); + entry.reply.send(done.verdict).unwrap(); + } + std::thread::sleep(Duration::from_millis(2)); + } + }) + }; + let threads: Vec<_> = (0..4u64) + .map(|t| { + let broker = Arc::clone(&broker); + std::thread::spawn(move || { + for i in 0..10u64 { + let path = if i % 2 == 0 { "/a/x" } else { "/n/x" }; + let args = format!(r#"{{"path":"{path}"}}"#); + let frames = call(&broker, request("s1", t * 100 + i, "read_file", &args)); + let last = frames.last().unwrap(); + assert!( + matches!( + &last.msg, + Message::ToolResponse( + ToolResponse::Result { .. } | ToolResponse::Denied { .. } + ) + ), + "{last:?}" + ); + } + }) + }) + .collect(); + for t in threads { + t.join().unwrap(); + } + stop.store(true, Ordering::SeqCst); + owner.join().unwrap(); + let records = rig.records(); + let approvals = records + .iter() + .filter(|r| matches!(r.event, AuditEvent::Approval { .. })) + .count(); + assert_eq!(approvals, 20, "one Approval for each of the twenty asks"); + let results = check_sequence(&records); + assert_eq!(rt.count(), results); + let report = brokerd::audit::verify_dir(&rig.cfg.audit_dir()).unwrap(); + assert!(report.failure.is_none() && report.abandoned.is_empty()); +} diff --git a/docs/plans/M4b/files/crates/brokerd/tests/config_gateway.rs b/docs/plans/M4b/files/crates/brokerd/tests/config_gateway.rs new file mode 100644 index 0000000..e655767 --- /dev/null +++ b/docs/plans/M4b/files/crates/brokerd/tests/config_gateway.rs @@ -0,0 +1,43 @@ +//! `[gateway]` in `brokerd.toml`: absent means Mattermost approvals are off; an empty socket is +//! the default under `home` (M4b spec, section 3). Do not edit. + +use std::path::PathBuf; + +use brokerd::config::Config; + +fn parse(text: &str) -> Config { + Config::parse(&format!("[paths]\nhome = \"/srv/bx\"\n{text}")).unwrap() +} + +#[test] +fn without_the_section_approvals_stay_in_bxctl() { + let config = parse(""); + assert_eq!(config.gateway, None); + assert_eq!(config.gateway_socket(), None); +} + +#[test] +fn an_empty_section_or_socket_is_the_default() { + let default = Some(PathBuf::from("/srv/bx/run/gateway/gateway.sock")); + assert_eq!(parse("[gateway]\n").gateway_socket(), default); + assert_eq!( + parse("[gateway]\nsocket = \"\"\n").gateway_socket(), + default + ); +} + +#[test] +fn a_socket_path_is_used_as_given() { + let config = parse("[gateway]\nsocket = \"/run/bx/gw.sock\"\n"); + assert_eq!( + config.gateway_socket(), + Some(PathBuf::from("/run/bx/gw.sock")) + ); +} + +#[test] +fn unknown_keys_are_errors() { + assert!(Config::parse("[gateway]\nsockets = \"/x\"\n").is_err()); + assert!(Config::parse("[gateway]\nsocket = \"/x\"\nenabled = true\n").is_err()); + assert!(Config::parse("[gatewy]\nsocket = \"/x\"\n").is_err()); +} diff --git a/docs/plans/M4b/files/crates/brokerd/tests/courier.rs b/docs/plans/M4b/files/crates/brokerd/tests/courier.rs new file mode 100644 index 0000000..1b0207b --- /dev/null +++ b/docs/plans/M4b/files/crates/brokerd/tests/courier.rs @@ -0,0 +1,447 @@ +//! The courier carries a pending approval to `gatewayd` and its answer back, against a fake +//! `gatewayd` on a real socket (M4b spec, section 5). Do not edit. + +#[path = "support/client.rs"] +mod client; +#[path = "support/fake_gateway.rs"] +mod fake_gateway; +#[path = "support/rig.rs"] +mod rig; +#[path = "support/runtime.rs"] +mod runtime; +#[path = "support/sink.rs"] +mod sink; +#[path = "support/tmp.rs"] +mod tmp; + +use std::os::unix::net::UnixStream; +use std::sync::Arc; +use std::sync::atomic::AtomicBool; +use std::time::{Duration, Instant}; + +use brokerd::broker::{self, Broker}; +use brokerd::config::Gateway; +use brokerd::courier::{self, Timing}; +use brokerd::ledger::Answer; +use client::{Serve, next, open}; +use fake_gateway::{FakeGateway, KYLE, POST}; +use proto::{ + ApprovalAnswer, ApprovalReply, AuditEvent, DenyReason, GatewayOutcome, Message, Timestamp, + ToolResponse, +}; +use rig::{Rig, grant_text, request}; +use runtime::Recording; + +const WAIT: Duration = Duration::from_secs(5); + +/// A rig whose `[gateway]` is a fake gatewayd in its home, and an `ask` grant for /n. +fn rig_with_gateway(tag: &str) -> (Rig, FakeGateway) { + let mut rig = Rig::new(tag); + let socket = rig.dir.path().join("gw.sock"); + let fake = FakeGateway::start(&socket); + rig.cfg.gateway = Some(Gateway { socket }); + rig.grant( + "notes", + &grant_text("read_file", "ask", "", "paths = [\"/n\"]"), + ); + (rig, fake) +} + +/// A call that waits for approval; its connection and the approval id. +fn ask(broker: &Arc) -> (UnixStream, u64) { + let mut stream = open( + broker, + broker::handle, + 9, + Message::ToolRequest(request("s1", 9, "read_file", r#"{"path":"/n/a"}"#)), + ); + match next(&mut stream).msg { + Message::ToolResponse(ToolResponse::PendingApproval { approval, .. }) => (stream, approval), + other => panic!("{other:?}"), + } +} + +fn final_answer(stream: &mut UnixStream) -> ToolResponse { + let env = next(stream); + assert!(env.r#final, "{env:?}"); + match env.msg { + Message::ToolResponse(r) => r, + other => panic!("{other:?}"), + } +} + +/// The `Approval` record's (answer, by, post, reason). +fn approval_record( + rig: &Rig, +) -> ( + ApprovalAnswer, + Option, + Option, + Option, +) { + let events = rig.events(); + let found = events.iter().find_map(|e| match e { + AuditEvent::Approval { + answer, + by, + post, + reason, + .. + } => Some((*answer, by.clone(), post.clone(), reason.clone())), + _ => None, + }); + found.expect("an Approval record") +} + +fn some(s: &str) -> Option { + Some(s.to_string()) +} + +/// Sets the courier's stop flag when dropped, so a failing assertion ends the courier too. +struct StopOnDrop<'a>(&'a AtomicBool); + +impl Drop for StopOnDrop<'_> { + fn drop(&mut self) { + self.0.store(true, std::sync::atomic::Ordering::SeqCst); + } +} + +/// The courier must end by itself within five seconds. +fn ends_by_itself(courier: &std::thread::ScopedJoinHandle<'_, T>) { + let until = Instant::now() + WAIT; + while !courier.is_finished() { + assert!(Instant::now() < until, "the courier did not stop"); + std::thread::sleep(Duration::from_millis(20)); + } +} + +#[test] +fn approved_in_mattermost_it_runs_and_the_record_names_who_and_where() { + let (rig, fake) = rig_with_gateway("courier-approve"); + let rt = Recording::answering("the notes"); + let broker = rig.broker(&rt); + let (mut stream, id) = ask(&broker); + let mut conn = fake.next(WAIT).expect("the approval reaches gatewayd"); + assert_eq!( + (conn.request.approval, conn.request.tool.as_str()), + (id, "read_file") + ); + assert_eq!(conn.request.arguments, r#"{"path":"/n/a"}"#); + conn.posted(); + conn.approve(); + assert_eq!(conn.outcome(), Some(GatewayOutcome::Runs {})); + assert!(matches!( + final_answer(&mut stream), + ToolResponse::Result { .. } + )); + assert_eq!(rt.count(), 1); + assert_eq!( + approval_record(&rig), + (ApprovalAnswer::Approved, some(KYLE), some(POST), None) + ); +} + +#[test] +fn refused_in_mattermost_it_is_denied_with_the_reason() { + let (rig, fake) = rig_with_gateway("courier-refuse"); + let rt = Recording::answering("x"); + let broker = rig.broker(&rt); + let (mut stream, _) = ask(&broker); + let mut conn = fake.next(WAIT).unwrap(); + conn.posted(); + conn.refuse("not on a Friday"); + assert_eq!(conn.outcome(), Some(GatewayOutcome::Refused {})); + assert_eq!( + final_answer(&mut stream), + ToolResponse::Denied { + reason: DenyReason::ApprovalRefused + } + ); + assert_eq!(rt.count(), 0); + assert_eq!( + approval_record(&rig), + ( + ApprovalAnswer::Refused, + some(KYLE), + some(POST), + some("not on a Friday") + ) + ); +} + +#[test] +fn approved_after_the_grant_is_gone_it_is_denied_on_the_recheck() { + let (rig, fake) = rig_with_gateway("courier-recheck"); + let rt = Recording::answering("x"); + let broker = rig.broker(&rt); + let (mut stream, _) = ask(&broker); + let mut conn = fake.next(WAIT).unwrap(); + conn.posted(); + rig.remove_grant("notes"); + conn.approve(); + assert_eq!( + conn.outcome(), + Some(GatewayOutcome::Denied { + reason: DenyReason::NoGrant + }) + ); + assert_eq!( + final_answer(&mut stream), + ToolResponse::Denied { + reason: DenyReason::NoGrant + } + ); + assert_eq!(rt.count(), 0); +} + +#[test] +fn answered_in_bxctl_the_courier_closes_without_an_outcome() { + let (rig, fake) = rig_with_gateway("courier-bxctl"); + let rt = Recording::answering("x"); + let broker = rig.broker(&rt); + let (mut stream, id) = ask(&broker); + let mut conn = fake.next(WAIT).unwrap(); + conn.posted(); + let entry = broker.table().take(id).unwrap(); + brokerd::admin::answer( + &broker, + entry, + Answer::Approved { + by: some("bxctl"), + post: None, + }, + Timestamp::now(), + ); + assert!( + conn.closed_within(WAIT), + "the courier closes, and sends nothing" + ); + assert!(matches!( + final_answer(&mut stream), + ToolResponse::Result { .. } + )); + assert_eq!(approval_record(&rig).1, some("bxctl")); +} + +#[test] +fn an_answer_after_it_ended_is_gone() { + let rig = Rig::new("courier-gone"); + rig.grant( + "notes", + &grant_text("read_file", "ask", "", "paths = [\"/n\"]"), + ); + let rt = Recording::answering("x"); + let broker = rig.broker(&rt); + let (_stream, id) = ask(&broker); + // The courier is run by hand here, so that the answer can arrive after the entry is taken. + let socket = rig.dir.path().join("gw.sock"); + let fake = FakeGateway::start(&socket); + let info = broker.table().list().remove(0); + let stop = AtomicBool::new(false); + std::thread::scope(|scope| { + let courier = scope.spawn(|| { + courier::carry( + &broker, + &socket, + &info, + &stop, + Timing { + retry: Duration::from_millis(100), + tick: Duration::from_secs(5), + }, + ) + }); + let _guard = StopOnDrop(&stop); + let mut conn = fake.next(WAIT).unwrap(); + conn.posted(); + let entry = broker.table().take(id).unwrap(); + brokerd::admin::answer(&broker, entry, Answer::Expired, Timestamp::now()); + conn.approve(); + assert_eq!(conn.outcome(), Some(GatewayOutcome::Gone {})); + ends_by_itself(&courier); + }); +} + +#[test] +fn if_gatewayd_goes_away_the_approval_is_posted_again() { + let (rig, fake) = rig_with_gateway("courier-again"); + let rt = Recording::answering("x"); + let broker = rig.broker(&rt); + let (mut stream, id) = ask(&broker); + let mut first = fake.next(WAIT).unwrap(); + first.posted(); + first.close(); + let mut second = fake.next(Duration::from_secs(10)).expect("posted again"); + assert_eq!(second.request.approval, id); + second.posted(); + second.approve(); + assert_eq!(second.outcome(), Some(GatewayOutcome::Runs {})); + assert!(matches!( + final_answer(&mut stream), + ToolResponse::Result { .. } + )); + assert!( + rig.lines.all().iter().any(|l| l.contains(&format!( + "approval {id}: gatewayd went away; posting it again" + ))), + "{:?}", + rig.lines.all() + ); +} + +#[test] +fn a_bad_answer_answers_nothing() { + let (rig, fake) = rig_with_gateway("courier-bad"); + let rt = Recording::answering("x"); + let broker = rig.broker(&rt); + let (_stream, id) = ask(&broker); + for bad in [ + ApprovalReply { + approval: id + 1, + approved: true, + by: KYLE.to_string(), + post: POST.to_string(), + reason: None, + }, + ApprovalReply { + approval: id, + approved: true, + by: "kyle".to_string(), + post: POST.to_string(), + reason: None, + }, + ApprovalReply { + approval: id, + approved: false, + by: KYLE.to_string(), + post: POST.to_string(), + reason: Some("x".repeat(501)), + }, + ] { + let mut conn = fake.next(Duration::from_secs(10)).unwrap(); + conn.posted(); + conn.reply(bad); + assert_eq!(conn.outcome(), None, "no outcome for a bad answer"); + assert!(broker.table().contains(id), "still pending"); + } + assert_eq!(rt.count(), 0); +} + +#[test] +fn unreachable_at_first_then_posted_and_said_once() { + let rig = Rig::new("courier-late"); + rig.grant( + "notes", + &grant_text("read_file", "ask", "", "paths = [\"/n\"]"), + ); + let rt = Recording::answering("x"); + let broker = rig.broker(&rt); + let (_stream, id) = ask(&broker); + let socket = rig.dir.path().join("gw.sock"); + let info = broker.table().list().remove(0); + let stop = AtomicBool::new(false); + let timing = Timing { + retry: Duration::from_millis(100), + tick: Duration::from_millis(20), + }; + std::thread::scope(|scope| { + let courier = scope.spawn(|| courier::carry(&broker, &socket, &info, &stop, timing)); + let _guard = StopOnDrop(&stop); + std::thread::sleep(Duration::from_millis(500)); + let fake = FakeGateway::start(&socket); + let mut conn = fake.next(WAIT).unwrap(); + conn.posted(); + let until = Instant::now() + WAIT; + while !rig + .lines + .all() + .iter() + .any(|l| l.contains("posted in Mattermost")) + { + assert!(Instant::now() < until, "{:?}", rig.lines.all()); + std::thread::sleep(Duration::from_millis(20)); + } + broker.table().take(id); + ends_by_itself(&courier); + }); + let lines = rig.lines.all(); + let unreachable: Vec<&String> = lines + .iter() + .filter(|l| l.contains("gatewayd is not reachable")) + .collect(); + assert_eq!( + unreachable.len(), + 1, + "said once, not every retry: {lines:?}" + ); + assert!( + unreachable[0].starts_with(&format!( + "brokerd: approval {id}: gatewayd is not reachable at " + )), + "{lines:?}" + ); + assert!( + unreachable[0] + .ends_with("; bxctl can still answer it\nsee docs/runbook.md#gateway-unreachable"), + "{lines:?}" + ); + assert!( + lines.contains(&format!("brokerd: approval {id} posted in Mattermost")), + "{lines:?}" + ); +} + +#[test] +fn if_gatewayd_cannot_post_it_is_tried_again() { + let (rig, fake) = rig_with_gateway("courier-cannot"); + let rt = Recording::answering("x"); + let broker = rig.broker(&rt); + let (mut stream, _) = ask(&broker); + fake.next(WAIT).unwrap().cannot_post(); + let mut conn = fake.next(Duration::from_secs(10)).expect("tried again"); + conn.posted(); + conn.approve(); + assert_eq!(conn.outcome(), Some(GatewayOutcome::Runs {})); + assert!(matches!( + final_answer(&mut stream), + ToolResponse::Result { .. } + )); + assert!( + rig.lines + .all() + .iter() + .any(|l| l.contains("gatewayd could not post it: Mattermost is down")) + ); +} + +#[test] +fn without_a_gateway_nothing_is_carried() { + let rig = Rig::new("courier-none"); + rig.grant( + "notes", + &grant_text("read_file", "ask", "", "paths = [\"/n\"]"), + ); + let socket = rig.dir.path().join("gw.sock"); + let fake = FakeGateway::start(&socket); + let rt = Recording::answering("x"); + let broker = rig.broker(&rt); + let (mut stream, id) = ask(&broker); + assert!( + fake.next(Duration::from_millis(500)).is_none(), + "no [gateway]: no connection" + ); + let entry = broker.table().take(id).unwrap(); + brokerd::admin::answer( + &broker, + entry, + Answer::Approved { + by: some("bxctl"), + post: None, + }, + Timestamp::now(), + ); + assert!(matches!( + final_answer(&mut stream), + ToolResponse::Result { .. } + )); +} diff --git a/docs/plans/M4b/files/crates/brokerd/tests/ledger_answer.rs b/docs/plans/M4b/files/crates/brokerd/tests/ledger_answer.rs new file mode 100644 index 0000000..89c8913 --- /dev/null +++ b/docs/plans/M4b/files/crates/brokerd/tests/ledger_answer.rs @@ -0,0 +1,264 @@ +//! The ledger's second step: an approval decides again and is recorded; a refusal and an expiry +//! are recorded as denials. Do not edit. + +#[path = "support/build.rs"] +mod build; +#[path = "support/rig.rs"] +mod rig; +#[path = "support/sink.rs"] +mod sink; +#[path = "support/tmp.rs"] +mod tmp; + +use brokerd::approvals::Verdict; +use brokerd::ledger::{Answer, Answered, Decided, Grants, Ledger}; +use brokerd::policy::{Ask, Label}; +use build::{grant, now, read, set}; +use proto::{ + ApprovalAnswer, AuditEvent, CallId, DataClass, DecisionRecord, DenyReason, Mode, SessionId, +}; +use rig::Rig; + +fn asking() -> Grants { + Ok(set(vec![grant("n", "read_file", Mode::Ask).paths(&["/n"])])) +} + +fn pending(ledger: &Ledger) -> Ask { + match ledger.decide(read("/n/deep/a"), &asking(), now()) { + Decided::Ask { ask, seq: 0, .. } => ask, + other => panic!("{other:?}"), + } +} + +fn bxctl() -> Answer { + Answer::Approved { + by: Some("bxctl".to_string()), + post: None, + } +} + +fn denied(reason: DenyReason) -> DecisionRecord { + DecisionRecord::Denied { reason } +} + +fn verdict_reason(answered: &Answered) -> Option { + match &answered.verdict { + Verdict::Run(_) => None, + Verdict::Denied(reason) => Some(*reason), + } +} + +/// The one `Approval` record, as (answer, by, reason, outcome, grant). +type Row = ( + ApprovalAnswer, + Option, + Option, + DecisionRecord, + Option, +); + +fn approval(rig: &Rig) -> Row { + let events = rig.events(); + assert_eq!(events.len(), 2, "a decision and one approval: {events:?}"); + match &events[1] { + AuditEvent::Approval { + session, + call, + decision, + answer, + by, + post, + reason, + outcome, + grant, + .. + } => { + assert_eq!(session, &SessionId::new("s1").unwrap()); + assert_eq!((*call, *decision, post), (CallId(1), 0, &None)); + ( + *answer, + by.clone(), + reason.clone(), + outcome.clone(), + grant.clone(), + ) + } + other => panic!("{other:?}"), + } +} + +#[test] +fn an_approval_that_still_asks_lets_the_call_run() { + let rig = Rig::new("answer-ask"); + let ledger = rig.ledger(); + let ask = pending(&ledger); + let answered = ledger.answer(ask, 0, bxctl(), &asking(), now()); + match &answered.verdict { + Verdict::Run(decision) => assert_eq!(decision.grant(), "n"), + other => panic!("{other:?}"), + } + assert_eq!(answered.outcome, DecisionRecord::Ask {}); + let row = approval(&rig); + assert_eq!( + row, + ( + ApprovalAnswer::Approved, + Some("bxctl".to_string()), + None, + DecisionRecord::Ask {}, + Some("n".to_string()) + ) + ); +} + +#[test] +fn an_approval_under_a_grant_that_is_now_auto_records_allowed() { + let rig = Rig::new("answer-auto"); + let ledger = rig.ledger(); + let ask = pending(&ledger); + let now_auto = Ok(set(vec![ + grant("n", "read_file", Mode::Auto).paths(&["/n"]), + ])); + let answered = ledger.answer(ask, 0, bxctl(), &now_auto, now()); + assert!(matches!(answered.verdict, Verdict::Run(_))); + assert_eq!(answered.outcome, DecisionRecord::Allowed {}); +} + +#[test] +fn the_approval_names_the_grant_matched_now() { + let rig = Rig::new("answer-grant"); + let ledger = rig.ledger(); + let ask = pending(&ledger); + let wider = Ok(set(vec![ + grant("n", "read_file", Mode::Ask).paths(&["/n"]), + grant("z-deep", "read_file", Mode::Ask).paths(&["/n/deep"]), + ])); + let answered = ledger.answer(ask, 0, bxctl(), &wider, now()); + match &answered.verdict { + Verdict::Run(decision) => assert_eq!(decision.grant(), "z-deep"), + other => panic!("{other:?}"), + } + assert_eq!(approval(&rig).4.as_deref(), Some("z-deep")); +} + +#[test] +fn an_approval_after_the_grant_is_gone_is_denied() { + let rig = Rig::new("answer-gone"); + let ledger = rig.ledger(); + let ask = pending(&ledger); + let answered = ledger.answer(ask, 0, bxctl(), &Ok(set(Vec::new())), now()); + assert_eq!(verdict_reason(&answered), Some(DenyReason::NoGrant)); + assert_eq!(answered.outcome, denied(DenyReason::NoGrant)); + let row = approval(&rig); + assert_eq!( + (row.0, row.3, row.4), + (ApprovalAnswer::Approved, denied(DenyReason::NoGrant), None) + ); +} + +#[test] +fn an_approval_after_the_taint_rose_is_denied() { + let rig = Rig::new("answer-taint"); + let ledger = rig.ledger(); + let low = || { + Ok(set(vec![ + grant("n", "read_file", Mode::Ask) + .paths(&["/n"]) + .max_taint(DataClass::Private), + ])) + }; + let ask = match ledger.decide(read("/n/a"), &low(), now()) { + Decided::Ask { ask, .. } => ask, + other => panic!("{other:?}"), + }; + // Another call of the session read a secret while this one waited. + let s1 = SessionId::new("s1").unwrap(); + let label = Label { + class: DataClass::Secret, + untrusted: false, + }; + rig.state() + .raise(&s1, rig.state().read(&s1).unwrap(), label) + .unwrap(); + let answered = ledger.answer(ask, 0, bxctl(), &low(), now()); + assert_eq!(verdict_reason(&answered), Some(DenyReason::TaintTooHigh)); + match &rig.events()[1] { + AuditEvent::Approval { taint, outcome, .. } => { + assert_eq!(*taint, DataClass::Secret, "the state at the re-decision"); + assert_eq!(*outcome, denied(DenyReason::TaintTooHigh)); + } + other => panic!("{other:?}"), + } +} + +#[test] +fn an_approval_with_invalid_grants_is_denied() { + let rig = Rig::new("answer-invalid"); + let ledger = rig.ledger(); + let ask = pending(&ledger); + let answered = ledger.answer(ask, 0, bxctl(), &Err(Vec::new()), now()); + assert_eq!(verdict_reason(&answered), Some(DenyReason::GrantsInvalid)); +} + +#[test] +fn a_refusal_is_recorded_with_the_owners_reason() { + let rig = Rig::new("answer-refuse"); + let ledger = rig.ledger(); + let ask = pending(&ledger); + let refused = Answer::Refused { + by: Some("bxctl".to_string()), + post: None, + reason: Some("not now".to_string()), + }; + let answered = ledger.answer(ask, 0, refused, &asking(), now()); + assert_eq!(verdict_reason(&answered), Some(DenyReason::ApprovalRefused)); + assert_eq!( + approval(&rig), + ( + ApprovalAnswer::Refused, + Some("bxctl".to_string()), + Some("not now".to_string()), + denied(DenyReason::ApprovalRefused), + None + ) + ); +} + +#[test] +fn an_expiry_is_recorded_by_nobody() { + let rig = Rig::new("answer-expire"); + let ledger = rig.ledger(); + let ask = pending(&ledger); + let answered = ledger.answer(ask, 0, Answer::Expired, &asking(), now()); + assert_eq!(verdict_reason(&answered), Some(DenyReason::ApprovalExpired)); + assert_eq!( + approval(&rig), + ( + ApprovalAnswer::Expired, + None, + None, + denied(DenyReason::ApprovalExpired), + None + ) + ); +} + +#[test] +fn an_approval_that_cannot_be_recorded_does_not_run() { + let rig = Rig::new("answer-norecord"); + let ledger = rig.ledger(); + let ask = pending(&ledger); + rig.switch.fail(true); + let answered = ledger.answer(ask, 0, bxctl(), &asking(), now()); + assert_eq!( + verdict_reason(&answered), + Some(DenyReason::AuditUnavailable) + ); + assert_eq!(answered.outcome, denied(DenyReason::AuditUnavailable)); + assert_eq!(rig.events().len(), 1, "only the decision"); + assert!( + !rig.lines + .with("see docs/runbook.md#audit-unavailable") + .is_empty() + ); +} diff --git a/docs/plans/M4b/files/crates/brokerd/tests/ledger_post.rs b/docs/plans/M4b/files/crates/brokerd/tests/ledger_post.rs new file mode 100644 index 0000000..48b2614 --- /dev/null +++ b/docs/plans/M4b/files/crates/brokerd/tests/ledger_post.rs @@ -0,0 +1,118 @@ +//! An answer from Mattermost records who answered and in which post, approved or refused; the +//! table says whether an approval is still pending (M4b spec, section 5). Do not edit. + +#[path = "support/build.rs"] +mod build; +#[path = "support/rig.rs"] +mod rig; +#[path = "support/sink.rs"] +mod sink; +#[path = "support/tmp.rs"] +mod tmp; + +use brokerd::approvals::Table; +use brokerd::ledger::{Answer, Decided, Grants, Ledger}; +use brokerd::policy::Ask; +use build::{grant, now, read, set}; +use proto::{ApprovalAnswer, AuditEvent, CallId, DataClass, Mode, PendingApproval, SessionId}; +use rig::Rig; + +const KYLE: &str = "k0000000000000000000000000"; +const POST: &str = "c0000000000000000000000001"; + +fn asking() -> Grants { + Ok(set(vec![grant("n", "read_file", Mode::Ask).paths(&["/n"])])) +} + +fn pending(ledger: &Ledger) -> Ask { + match ledger.decide(read("/n/deep/a"), &asking(), now()) { + Decided::Ask { ask, seq: 0, .. } => ask, + other => panic!("{other:?}"), + } +} + +/// The one `Approval` record's (answer, by, post, reason). +fn recorded( + rig: &Rig, +) -> ( + ApprovalAnswer, + Option, + Option, + Option, +) { + let events = rig.events(); + match &events[1] { + AuditEvent::Approval { + answer, + by, + post, + reason, + .. + } => (*answer, by.clone(), post.clone(), reason.clone()), + other => panic!("{other:?}"), + } +} + +#[test] +fn an_approval_from_mattermost_names_the_user_and_the_post() { + let rig = Rig::new("post-approve"); + let ledger = rig.ledger(); + let answer = Answer::Approved { + by: Some(KYLE.to_string()), + post: Some(POST.to_string()), + }; + ledger.answer(pending(&ledger), 0, answer, &asking(), now()); + assert_eq!( + recorded(&rig), + ( + ApprovalAnswer::Approved, + Some(KYLE.to_string()), + Some(POST.to_string()), + None + ) + ); +} + +#[test] +fn a_refusal_from_mattermost_names_them_and_keeps_the_reason() { + let rig = Rig::new("post-refuse"); + let ledger = rig.ledger(); + let answer = Answer::Refused { + by: Some(KYLE.to_string()), + post: Some(POST.to_string()), + reason: Some("not on a Friday".to_string()), + }; + ledger.answer(pending(&ledger), 0, answer, &asking(), now()); + assert_eq!( + recorded(&rig), + ( + ApprovalAnswer::Refused, + Some(KYLE.to_string()), + Some(POST.to_string()), + Some("not on a Friday".to_string()) + ) + ); +} + +#[test] +fn the_table_says_what_is_pending() { + let rig = Rig::new("post-table"); + let ledger = rig.ledger(); + let table = Table::new(); + assert!(!table.contains(0)); + let info = PendingApproval { + approval: 0, + session: SessionId::new("s1").unwrap(), + call: CallId(1), + tool: "read_file".to_string(), + arguments: "{}".to_string(), + grant: "n".to_string(), + taint: DataClass::Private, + created: now(), + expires: now(), + }; + let _verdict = table.insert(info, pending(&ledger)); + assert!(table.contains(0) && !table.contains(1)); + assert!(table.take(0).is_some()); + assert!(!table.contains(0)); +} diff --git a/docs/plans/M4b/files/crates/brokerd/tests/support/fake_gateway.rs b/docs/plans/M4b/files/crates/brokerd/tests/support/fake_gateway.rs new file mode 100644 index 0000000..88e3149 --- /dev/null +++ b/docs/plans/M4b/files/crates/brokerd/tests/support/fake_gateway.rs @@ -0,0 +1,150 @@ +//! A fake `gatewayd` on a Unix socket: it accepts each courier's connection, reads the approval, +//! and lets the test post it, answer it, close, or read the outcome. Do not edit. + +#![allow(dead_code)] // each test file uses a different part of this module + +use std::os::unix::net::{UnixListener, UnixStream}; +use std::path::{Path, PathBuf}; +use std::sync::mpsc::{self, Receiver}; +use std::time::Duration; + +use proto::{ + ApprovalPosted, ApprovalReply, Envelope, ErrorCode, GatewayOutcome, Message, PROTOCOL_VERSION, + PendingApproval, WireError, read_frame, write_frame, +}; + +pub const KYLE: &str = "k0000000000000000000000000"; +pub const POST: &str = "c0000000000000000000000001"; + +pub struct FakeGateway { + pub socket: PathBuf, + conns: Receiver, +} + +/// One courier's connection, its request already read. +pub struct Conn { + stream: UnixStream, + pub request: PendingApproval, +} + +fn frame(id: u64, last: bool, msg: Message) -> Envelope { + Envelope { + v: PROTOCOL_VERSION, + id, + r#final: last, + msg, + } +} + +impl FakeGateway { + /// Listen on `socket` (created now). + pub fn start(socket: &Path) -> FakeGateway { + let listener = UnixListener::bind(socket).unwrap(); + let (tx, conns) = mpsc::channel(); + std::thread::spawn(move || { + for stream in listener.incoming() { + let Ok(mut stream) = stream else { continue }; + let Ok(env) = read_frame(&mut stream) else { + continue; + }; + let Message::ApprovalRequest(req) = env.msg else { + panic!("not a request: {env:?}") + }; + assert_eq!((env.id, env.r#final), (1, true)); + if tx + .send(Conn { + stream, + request: req.approval, + }) + .is_err() + { + return; + } + } + }); + FakeGateway { + socket: socket.to_path_buf(), + conns, + } + } + + /// The next courier connection within `wait`, or `None`. + pub fn next(&self, wait: Duration) -> Option { + self.conns.recv_timeout(wait).ok() + } +} + +impl Conn { + pub fn posted(&mut self) { + let posts = vec![POST.to_string()]; + write_frame( + &mut self.stream, + &frame(1, false, Message::ApprovalPosted(ApprovalPosted { posts })), + ) + .unwrap(); + } + + pub fn cannot_post(&mut self) { + let e = WireError { + code: ErrorCode::Internal, + detail: "Mattermost is down".to_string(), + }; + write_frame(&mut self.stream, &frame(1, true, Message::Error(e))).unwrap(); + } + + pub fn reply(&mut self, reply: ApprovalReply) { + let _ = write_frame( + &mut self.stream, + &frame(1, true, Message::ApprovalReply(reply)), + ); + } + + pub fn approve(&mut self) { + let approval = self.request.approval; + self.reply(ApprovalReply { + approval, + approved: true, + by: KYLE.to_string(), + post: POST.to_string(), + reason: None, + }); + } + + pub fn refuse(&mut self, reason: &str) { + let approval = self.request.approval; + self.reply(ApprovalReply { + approval, + approved: false, + by: KYLE.to_string(), + post: POST.to_string(), + reason: Some(reason.to_string()), + }); + } + + /// Frame 4 within five seconds, or `None` if the courier closed without one. + pub fn outcome(&mut self) -> Option { + self.stream + .set_read_timeout(Some(Duration::from_secs(5))) + .unwrap(); + match read_frame(&mut self.stream) { + Ok(env) => { + assert_eq!((env.id, env.r#final), (2, true)); + match env.msg { + Message::ApprovalOutcome(o) => Some(o.outcome), + other => panic!("not an outcome: {other:?}"), + } + } + Err(_) => None, + } + } + + /// Did the courier close the connection within `wait`, without sending anything? + pub fn closed_within(&mut self, wait: Duration) -> bool { + self.stream.set_read_timeout(Some(wait)).unwrap(); + matches!(read_frame(&mut self.stream), Err(proto::FrameError::Closed)) + } + + pub fn close(self) { + let _ = self.stream.shutdown(std::net::Shutdown::Both); + } +} diff --git a/docs/plans/M4b/files/crates/brokerd/tests/support/rig.rs b/docs/plans/M4b/files/crates/brokerd/tests/support/rig.rs new file mode 100644 index 0000000..431094c --- /dev/null +++ b/docs/plans/M4b/files/crates/brokerd/tests/support/rig.rs @@ -0,0 +1,120 @@ +//! A home for ledger and broker tests: grants, audit log, session state, a flaky audit sink and +//! a log to read. Do not edit. +//! +//! Included with `#[path = "support/rig.rs"] mod rig;`, beside `tmp` and `sink`. The broker +//! tests add `client`. + +#![allow(dead_code)] // each test file uses a different part of this module + +use std::path::PathBuf; + +use brokerd::audit::Writer; +use brokerd::config::{Approvals, Config, Paths, Sockets}; +use brokerd::ledger::Ledger; +use brokerd::state::StateStore; +use proto::{AuditEvent, AuditRecord, CallId, SessionId, ToolRequest}; + +use crate::sink::{Flaky, Lines, Switch}; +use crate::tmp::TempDir; + +pub struct Rig { + pub dir: TempDir, + pub cfg: Config, + pub switch: Switch, + pub lines: Lines, +} + +impl Rig { + pub fn new(tag: &str) -> Rig { + Rig::with_ttl(tag, 900_000) + } + + pub fn with_ttl(tag: &str, ttl_ms: u64) -> Rig { + let dir = TempDir::new(tag); + let grants = dir.path().join("grants"); + std::fs::create_dir_all(&grants).unwrap(); + let cfg = Config { + paths: Paths { + home: dir.path().to_path_buf(), + grants, + }, + sockets: Sockets::default(), + approvals: Approvals { ttl_ms }, + runner: None, + gateway: None, + }; + Rig { + dir, + cfg, + switch: Switch::default(), + lines: Lines::default(), + } + } + + pub fn state(&self) -> StateStore { + StateStore::new(&self.cfg.state_dir()) + } + + /// Opens the audit log (once: the writer holds its lock) behind the flaky sink. + pub fn ledger(&self) -> Ledger { + let opened = Writer::open(&self.cfg.audit_dir(), false).unwrap(); + let sink = Flaky { + writer: opened.writer, + switch: self.switch.clone(), + }; + Ledger::new(Box::new(sink), self.state(), self.lines.sink()) + } + + /// Writes `grants/.toml`. + pub fn grant(&self, id: &str, text: &str) { + std::fs::write(self.cfg.paths.grants.join(format!("{id}.toml")), text).unwrap(); + } + + pub fn remove_grant(&self, id: &str) { + std::fs::remove_file(self.cfg.paths.grants.join(format!("{id}.toml"))).unwrap(); + } + + pub fn state_file(&self, session: &str) -> PathBuf { + self.cfg.state_dir().join(format!("{session}.json")) + } + + /// Every record in the audit log, in order. + pub fn records(&self) -> Vec { + let dir = self.cfg.audit_dir(); + let mut names: Vec = std::fs::read_dir(&dir) + .unwrap() + .map(|e| e.unwrap().file_name().into_string().unwrap()) + .filter(|n| n.ends_with(".jsonl")) + .collect(); + names.sort(); + let mut out = Vec::new(); + for name in names { + let text = std::fs::read_to_string(dir.join(name)).unwrap(); + for line in text.lines() { + out.push(serde_json::from_str(line).unwrap()); + } + } + out + } + + pub fn events(&self) -> Vec { + self.records().into_iter().map(|r| r.event).collect() + } +} + +/// A grant file's text. `extra` goes before `[constraints]`, `constraints` after it. +pub fn grant_text(tool: &str, mode: &str, extra: &str, constraints: &str) -> String { + format!( + "tool = \"{tool}\"\nmode = \"{mode}\"\nmax_taint = \"secret\"\nresult_class = \"private\"\n\ + untrusted = false\n{extra}\n[constraints]\n{constraints}\n" + ) +} + +pub fn request(session: &str, call: u64, tool: &str, arguments: &str) -> ToolRequest { + ToolRequest { + session: SessionId::new(session).unwrap(), + call: CallId(call), + tool: tool.to_string(), + arguments: arguments.to_string(), + } +} diff --git a/docs/plans/M4b/files/crates/bxctl/src/escape.rs b/docs/plans/M4b/files/crates/bxctl/src/escape.rs new file mode 100644 index 0000000..a0d4072 --- /dev/null +++ b/docs/plans/M4b/files/crates/bxctl/src/escape.rs @@ -0,0 +1,3 @@ +//! The text-escape helpers, from `proto` (moved there in M4b so `gatewayd` shows the same thing). + +pub use proto::escape::{escape_json_text, escape_model_text}; diff --git a/docs/plans/M4b/files/crates/bxctl/tests/cli_deny.rs b/docs/plans/M4b/files/crates/bxctl/tests/cli_deny.rs new file mode 100644 index 0000000..3d0ccd0 --- /dev/null +++ b/docs/plans/M4b/files/crates/bxctl/tests/cli_deny.rs @@ -0,0 +1,42 @@ +//! `bxctl deny` is another name for `bxctl refuse`: every argument list means the same under both +//! (M4b spec, section 2). Do not edit. + +use bxctl::cli::{USAGE, parse}; +use std::path::Path; + +fn under(command: &str, rest: &[&str]) -> Vec { + std::iter::once(command) + .chain(rest.iter().copied()) + .map(str::to_string) + .collect() +} + +#[test] +fn deny_parses_as_refuse() { + let cases: &[&[&str]] = &[ + &["41"], + &["41", "--reason", "not on a Friday"], + &["--reason", "no", "41"], + &["41", "--admin-socket", "/tmp/a.sock"], + &["41", "--reason", "x", "--admin-socket", "/tmp/a.sock"], + &[], + &["x41"], + &["41", "--reason"], + &["41", "42"], + &["41", "--bogus"], + ]; + for rest in cases { + let home = Path::new("/srv/bx"); + assert_eq!( + parse(&under("deny", rest), home), + parse(&under("refuse", rest), home), + "{rest:?}" + ); + } +} + +#[test] +fn usage_names_deny() { + assert!(USAGE.contains("bxctl deny "), "{USAGE}"); + assert!(USAGE.contains("bxctl refuse "), "{USAGE}"); +} diff --git a/docs/plans/M4b/files/crates/gatewayd/src/approvals/mod.rs b/docs/plans/M4b/files/crates/gatewayd/src/approvals/mod.rs new file mode 100644 index 0000000..0578a6e --- /dev/null +++ b/docs/plans/M4b/files/crates/gatewayd/src/approvals/mod.rs @@ -0,0 +1,105 @@ +//! Approvals in Mattermost (M4b spec, sections 6 and 7): the text of the posts, and the +//! approvals this `gatewayd` has posted and the commands that answer them. + +pub mod text; + +use std::collections::HashMap; +use std::sync::mpsc::Sender; +use std::sync::{Mutex, MutexGuard}; + +use proto::ApprovalReply; + +use crate::sessions::ApprovalCommand; + +pub const NOT_APPROVER: &str = "only an approver can answer approvals"; +pub const USAGE: &str = "use !approve or !deny "; + +/// One approval posted here and not yet answered. +#[derive(Debug)] +pub struct Posted { + pub approval: u64, + /// The session whose turn waits for it. + pub session: String, + /// The thread it is answered in. + pub channel: String, + pub root: String, + /// The last post's `create_at`: only a command after it counts. + pub shown_at: i64, + /// False when the arguments were too long to show: only `bxctl` answers it. + pub answerable: bool, + /// Where an accepted answer goes: the thread of its `gateway.sock` connection. + pub answer: Sender, +} + +/// What a command came to. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Judged { + /// The answer went to `brokerd`'s connection; the approval post will show the outcome. + Sent, + /// Refused: this line goes in the thread. + Reply(String), +} + +/// The approvals posted here, by id. +#[derive(Debug, Default)] +pub struct Registry { + posted: Mutex>, +} + +impl Registry { + pub fn new() -> Registry { + Registry::default() + } + + fn lock(&self) -> MutexGuard<'_, HashMap> { + self.posted.lock().unwrap_or_else(|p| p.into_inner()) + } + + pub fn add(&self, posted: Posted) { + // `self.lock().insert(posted.approval, posted);` + todo!() + } + + /// Take it out: it ended, or its connection is done with it. + pub fn remove(&self, id: u64) -> Option { + // `self.lock().remove(&id)` + todo!() + } + + pub fn holds(&self, id: u64) -> bool { + // `self.lock().contains_key(&id)` + todo!() + } + + /// The sessions with an approval waiting here: their turns are waiting, not working. + pub fn waiting_sessions(&self) -> Vec { + // The `session` of every entry: `self.lock().values().map(|p| + // p.session.clone()).collect()`. + todo!() + } + + /// Judge a command from `user`, posted as `post` at `at` in the thread `root`, in the order of + /// the spec's section 7. An accepted answer is taken out and sent; nothing else changes. + pub fn judge( + &self, + command: &ApprovalCommand, + user: &str, + root: &str, + post: &str, + at: i64, + approvers: &[String], + ) -> Judged { + // In this order; each refusal is `Judged::Reply()` and changes nothing: + // 1. `user` not in `approvers`: NOT_APPROVER. + // 2. The command: Approve(id) is (id, approved true, reason None); Deny(id, reason) is (id, + // false, reason.clone()); Malformed: USAGE. + // 3. `let mut posted = self.lock();` Not in `posted`: "approval {id} is not pending here". + // 4. `!entry.answerable`: "approval {id} can only be answered with bxctl approvals". + // 5. `entry.root != root`: "approval {id} is answered in its own thread". + // 6. `at <= entry.shown_at`: "approval {id} was answered before it was shown in full". + // 7. Take it out (`posted.remove(&id)`) and send `ApprovalReply { approval: id, approved, + // by: user, post, reason }` on its `answer`: Ok -> `Judged::Sent`; Err (the connection + // is gone) -> "approval {id} is not pending here". + todo!() + } +} diff --git a/docs/plans/M4b/files/crates/gatewayd/src/approvals/mod.rs-task10 b/docs/plans/M4b/files/crates/gatewayd/src/approvals/mod.rs-task10 new file mode 100644 index 0000000..6a8cd58 --- /dev/null +++ b/docs/plans/M4b/files/crates/gatewayd/src/approvals/mod.rs-task10 @@ -0,0 +1,4 @@ +//! Approvals in Mattermost (M4b spec, sections 6 and 7): the text of the posts, and the +//! approvals this `gatewayd` has posted and the commands that answer them. + +pub mod text; diff --git a/docs/plans/M4b/files/crates/gatewayd/src/approvals/text.rs b/docs/plans/M4b/files/crates/gatewayd/src/approvals/text.rs new file mode 100644 index 0000000..b03ae54 --- /dev/null +++ b/docs/plans/M4b/files/crates/gatewayd/src/approvals/text.rs @@ -0,0 +1,157 @@ +//! The text of an approval's posts, and of the line added when it ends (M4b spec, section 6). +//! Nothing here is shortened: arguments too long to show whole are not shown at all. + +use proto::escape::escape_json_text; +use proto::{DenyReason, PendingApproval, Timestamp, sha256}; + +/// Arguments are split into parts of at most this many characters. +pub const PART_CHARS: usize = 15_000; +/// At most this many parts; more are answerable only in `bxctl`. +pub const MAX_PARTS: usize = 4; +/// The longest a part's post may be, leaving room for the line added when it ends. +pub const MAX_PART_POST: usize = 15_500; + +/// An approval's posts, in order. Only an answer after the last counts; when `answerable` is +/// false, no command answers it. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Shown { + pub posts: Vec, + pub answerable: bool, +} + +/// How an approval ended, for the line added to its last post. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Ended { + Runs { by: String }, + Denied { by: String, reason: DenyReason }, + Refused { by: String, reason: Option }, + NoLonger, + Restarted, +} + +/// `text` as inline code, with a backtick run longer than any inside it. +pub fn code(text: &str) -> String { + // `let ticks = "`".repeat(longest_run(text) + 1);`, and a pad of one space on both sides when + // `text` starts or ends with a backtick (CommonMark needs it), else none: + // `format!("{ticks}{pad}{text}{pad}{ticks}")`. + todo!() +} + +/// `text` as a fenced block, with a fence longer than any backtick run inside it (at least 3). +pub fn fence(text: &str) -> String { + // `let ticks = "`".repeat((longest_run(text) + 1).max(3));` then + // `format!("{ticks}\n{text}\n{ticks}")`. + todo!() +} + +fn longest_run(text: &str) -> usize { + // The longest run of consecutive backticks in `text`: walk `text.chars()`, counting up on a + // backtick and back to 0 on anything else, keeping the most seen. + todo!() +} + +/// `2026-09-24 14:47 UTC`. +fn minute(at: Timestamp) -> String { + // `let rfc = at.to_rfc3339();` ("2026-09-24T14:47:10.000Z"); then `format!("{} {} UTC", + // rfc.get(..10).unwrap_or_default(), rfc.get(11..16).unwrap_or_default())`. + todo!() +} + +/// A serde name (`private`, `no_grant`), or the debug form if there is none. +fn name(value: &T) -> String { + // `match serde_json::to_value(value)`: `Ok(serde_json::Value::String(s))` -> s; anything else + // -> `format!("{value:?}")`. + todo!() +} + +fn reply_line(id: u64) -> String { + // `format!("Reply in this thread with `!approve {id}` or `!deny {id} `.")`. + todo!() +} + +/// The posts that show `info`. +pub fn render(info: &PendingApproval) -> Shown { + let id = info.approval; + let args = escape_json_text(&info.arguments); + let header = format!( + "**Approval {id}** · tool {} · grant {} · taint {} · expires {}", + code(&escape_json_text(&info.tool)), + code(&escape_json_text(&info.grant)), + name(&info.taint), + minute(info.expires), + ); + let chars: Vec = args.chars().collect(); + let parts: Vec = if chars.is_empty() { + vec![String::new()] + } else { + chars + .chunks(PART_CHARS) + .map(|c| c.iter().collect()) + .collect() + }; + let count = parts.len(); + let posts: Vec = if count == 1 { + vec![format!("{header}\n{}\n{}", fence(&args), reply_line(id))] + } else { + let hash = sha256(args.as_bytes()) + .map(|h| h.to_hex()) + .unwrap_or_default(); + parts + .iter() + .enumerate() + .map(|(i, part)| { + let n = i + 1; + let top = if n == 1 { + format!("{header}\nPart 1 of {count} of the arguments:") + } else { + format!("**Approval {id}**, part {n} of {count} of the arguments:") + }; + let mut post = format!("{top}\n{}", fence(part)); + if n == count { + post.push_str(&format!( + "\nApproval {id}: {count} parts, {} characters, sha256 {hash} (of the arguments as shown).\n{}", + chars.len(), + reply_line(id) + )); + } + post + }) + .collect() + }; + if count > MAX_PARTS || posts.iter().any(|p| p.chars().count() > MAX_PART_POST) { + let hash = sha256(args.as_bytes()) + .map(|h| h.to_hex()) + .unwrap_or_default(); + let post = format!( + "{header}\nThe arguments are {} characters (sha256 {hash}), too long to show here; answer it with `bxctl approvals`.", + chars.len() + ); + return Shown { + posts: vec![post], + answerable: false, + }; + } + Shown { + posts, + answerable: true, + } +} + +/// The line added to the last post when the approval ends. `by` is `@username`, or the id. +pub fn ended_line(ended: &Ended) -> String { + // A `match ended`, one line each, exactly (`by` as given): + // 1. Runs: "✅ Approved by {by}: the call runs." + // 2. Denied: "Approved by {by}, but denied on the re-check: {name(reason)}." + // 3. Refused with Some(reason): "❌ Refused by {by}: {code(&escape_json_text(reason))}." + // 4. Refused with None: "❌ Refused by {by}: no reason given." + // 5. NoLonger: "No longer pending (answered elsewhere, expired, or its turn ended)." + // 6. Restarted: "No longer answerable here: gatewayd restarted. If it is still pending it is + // posted again." + todo!() +} + +/// The last post's text with the ending line added. +pub fn with_ending(last: &str, ended: &Ended) -> String { + // `format!("{last}\n\n{}", ended_line(ended))`. + todo!() +} diff --git a/docs/plans/M4b/files/crates/gatewayd/src/gateway.rs b/docs/plans/M4b/files/crates/gatewayd/src/gateway.rs new file mode 100644 index 0000000..e81c569 --- /dev/null +++ b/docs/plans/M4b/files/crates/gatewayd/src/gateway.rs @@ -0,0 +1,224 @@ +//! `gateway.sock`: the listener, and one thread per `brokerd` connection (M4b spec, sections 4 +//! and 6). A connection thread speaks the protocol only; everything that touches Mattermost or the +//! state file is asked of the serve loop, which alone owns them. + +use std::os::unix::fs::{DirBuilderExt, FileTypeExt, PermissionsExt}; +use std::os::unix::net::{UnixListener, UnixStream}; +use std::path::Path; +use std::sync::Arc; +use std::sync::mpsc::{self, Receiver, RecvTimeoutError, Sender, TryRecvError}; +use std::time::Duration; + +use proto::{ + ApprovalPosted, ApprovalReply, Envelope, ErrorCode, FrameError, GatewayOutcome, Message, + PROTOCOL_VERSION, PendingApproval, WireError, read_frame, write_frame, +}; + +/// A log line printer, shared by the threads. +pub type Log = Arc; + +/// What a connection thread asks of the serve loop. +#[derive(Debug)] +pub enum ToLoop { + /// Post this approval; an accepted answer comes back on `answer`, and the post ids (or why + /// it could not be posted) on `shown`. + Show { + info: PendingApproval, + answer: Sender, + shown: Sender, String>>, + }, + /// It ended: edit its last post, and forget it. + Ended { approval: u64, ending: Ending }, +} + +/// How an approval ended, as far as this `gatewayd` knows. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum Ending { + /// Answered here, by `by`, and `brokerd` said what came of it. + Answered { + by: String, + reason: Option, + outcome: GatewayOutcome, + }, + /// `brokerd` closed first, or said nothing after the answer. + NoLonger, +} + +/// How long to wait: for the first frame, for the loop to post, for `brokerd`'s outcome; and how +/// often to look for an answer meanwhile. +#[derive(Debug, Clone, Copy)] +pub struct Timing { + pub first_frame: Duration, + pub posting: Duration, + pub outcome: Duration, + pub tick: Duration, +} + +impl Default for Timing { + fn default() -> Self { + Timing { + first_frame: Duration::from_secs(10), + posting: Duration::from_secs(120), + outcome: Duration::from_secs(30), + tick: Duration::from_millis(100), + } + } +} + +/// Listen on `socket`: its directory made 0700, a stale socket removed, the socket made 0600. +pub fn listen(socket: &Path) -> Result { + // 1. The directory: `socket.parent()`, which must not be empty, else "cannot prepare : + // a socket needs a directory". + // 2. `std::fs::DirBuilder::new().recursive(true).mode(0o700).create(dir)`, then + // `set_permissions(dir, 0o700)`; an error is "cannot prepare : ". + // 3. `std::fs::symlink_metadata(socket)`: a socket (`file_type().is_socket()`) is removed + // (stale); anything else there is "cannot listen on : something else is there"; + // nothing there is fine. + // 4. `UnixListener::bind(socket)`, then `set_permissions(socket, 0o600)`; an error is "cannot + // listen on : ". + // 5. Ok(listener). Paths are shown with `.display()`. + todo!() +} + +/// Accept connections on a thread of their own, each served on its own thread. +pub fn serve( + listener: UnixListener, + to_loop: Sender, + timing: Timing, + log: Log, +) -> std::io::Result<()> { + // Spawn (with `std::thread::Builder`, `?`) a thread that loops over `listener.incoming()`, + // skipping errors. For each stream: clone `to_loop` and `log`, and spawn a thread running + // `connection(stream, &to_loop, timing, &log)`; if that spawn fails, log "gatewayd: cannot + // start a thread for an approval: ". Then Ok(()). + todo!() +} + +fn frame(id: u64, last: bool, msg: Message) -> Envelope { + Envelope { + v: PROTOCOL_VERSION, + id, + r#final: last, + msg, + } +} + +/// Every frame `brokerd` sends, read whole on a thread of its own; shutting the stream down ends it. +fn frames(stream: &UnixStream) -> std::io::Result>> { + // As in `brokerd`'s courier: `let mut reader = stream.try_clone()?;`, a channel, and a thread + // (`std::thread::Builder`, `?`) looping `let frame = read_frame(&mut reader); let last = + // frame.is_err(); if tx.send(frame).is_err() || last { return; }`. Then `Ok(rx)`. + todo!() +} + +/// One `brokerd` connection, frames 1 to 4 of the spec's section 4. +pub fn connection(mut stream: UnixStream, to_loop: &Sender, timing: Timing, log: &Log) { + // 1. The request, within `first_frame`. + let _ = stream.set_read_timeout(Some(timing.first_frame)); + let info = match read_frame(&mut stream) { + Ok(Envelope { + id: 1, + r#final: true, + msg: Message::ApprovalRequest(req), + .. + }) => req.approval, + _ => return, + }; + let _ = stream.set_read_timeout(None); + let id = info.approval; + // 2. The loop posts it. + let (answer_tx, answer_rx) = mpsc::channel(); + let (shown_tx, shown_rx) = mpsc::channel(); + if to_loop + .send(ToLoop::Show { + info, + answer: answer_tx, + shown: shown_tx, + }) + .is_err() + { + return; + } + let posts = match shown_rx.recv_timeout(timing.posting) { + Ok(Ok(posts)) => posts, + Ok(Err(why)) => { + let e = WireError { + code: ErrorCode::Internal, + detail: why, + }; + let _ = write_frame(&mut stream, &frame(1, true, Message::Error(e))); + return; + } + Err(_) => { + let e = WireError { + code: ErrorCode::Internal, + detail: "gatewayd did not post it in time".to_string(), + }; + let _ = write_frame(&mut stream, &frame(1, true, Message::Error(e))); + return; + } + }; + let rx = match frames(&stream) { + Ok(rx) => rx, + Err(_) => return end(to_loop, id, Ending::NoLonger), + }; + if write_frame( + &mut stream, + &frame(1, false, Message::ApprovalPosted(ApprovalPosted { posts })), + ) + .is_err() + { + return end(to_loop, id, Ending::NoLonger); + } + // 3. An answer from an approver, or brokerd closing first. + let mut answers = Some(answer_rx); + let reply = loop { + if let Some(answers_rx) = &answers { + match answers_rx.recv_timeout(timing.tick) { + Ok(reply) => break reply, + Err(RecvTimeoutError::Timeout) => {} + Err(RecvTimeoutError::Disconnected) => answers = None, + } + } else { + std::thread::sleep(timing.tick); + } + match rx.try_recv() { + Err(TryRecvError::Empty) => {} + // brokerd closed (the approval ended elsewhere), or sent what it may not. + _ => { + let _ = stream.shutdown(std::net::Shutdown::Both); + return end(to_loop, id, Ending::NoLonger); + } + } + }; + let (by, reason) = (reply.by.clone(), reply.reason.clone()); + if write_frame(&mut stream, &frame(1, true, Message::ApprovalReply(reply))).is_err() { + return end(to_loop, id, Ending::NoLonger); + } + // 4. What came of it. + let ending = match rx.recv_timeout(timing.outcome) { + Ok(Ok(Envelope { + id: 2, + r#final: true, + msg: Message::ApprovalOutcome(o), + .. + })) => Ending::Answered { + by, + reason, + outcome: o.outcome, + }, + _ => { + (log)(&format!( + "gatewayd: approval {id}: brokerd said nothing after the answer" + )); + Ending::NoLonger + } + }; + let _ = stream.shutdown(std::net::Shutdown::Both); + end(to_loop, id, ending); +} + +fn end(to_loop: &Sender, approval: u64, ending: Ending) { + // `let _ = to_loop.send(ToLoop::Ended { approval, ending });` + todo!() +} diff --git a/docs/plans/M4b/files/crates/gatewayd/src/serve/approvals.rs b/docs/plans/M4b/files/crates/gatewayd/src/serve/approvals.rs new file mode 100644 index 0000000..8efe3a1 --- /dev/null +++ b/docs/plans/M4b/files/crates/gatewayd/src/serve/approvals.rs @@ -0,0 +1,143 @@ +//! The serve loop's part in approvals (M4b spec, sections 6 and 8): posting what a `gateway.sock` +//! connection brings, marking it when it ends, and marking what a restart left behind. The loop +//! alone touches Mattermost and the state file; connection threads only ask. + +use std::sync::mpsc::Sender; + +use proto::{ApprovalReply, GatewayOutcome, PendingApproval}; + +use crate::approvals::Posted; +use crate::approvals::text::{self, Ended}; +use crate::gateway::{Ending, ToLoop}; +use crate::serve::{Gateway, Stop}; +use crate::state::ApprovalPosts; + +/// An approval's last post, the one that is edited when it ends. +#[derive(Debug)] +pub(super) struct Open { + pub(super) last_post: String, + pub(super) last_text: String, +} + +impl Gateway { + /// Everything the connection threads asked since the last look, never blocking. + pub(super) fn gateway_requests(&mut self) -> Result<(), Stop> { + while let Ok(asked) = self.gateway_rx.try_recv() { + match asked { + ToLoop::Show { + info, + answer, + shown, + } => { + let result = self.show(&info, answer)?; + let _ = shown.send(result); + } + ToLoop::Ended { approval, ending } => self.ended(approval, ending)?, + } + } + Ok(()) + } + + /// Where an approval goes: its session's turn in flight, or the first approver's direct + /// messages as a new thread (an empty root). + fn place(&self, info: &PendingApproval) -> Result<(String, String), String> { + // 1. `self.state.turn_of(info.session.as_str())`: if Some(turn), `Ok((turn.channel, + // turn.root))`. + // 2. Otherwise the first approver's direct messages, as a new thread: + // `self.config.first_approver()` (None is Err("no approver is configured")), then + // `self.client.direct_channel(&self.me.id, first)` (an Err(e) is Err(format!("no direct + // channel with the first approver: {e}"))). `Ok((channel, String::new()))`. + todo!() + } + + /// Post an approval's parts in order, record them, and register it for answers. An `Err` + /// inside is why it could not be posted; the outer one stops `gatewayd` (the state file). + fn show( + &mut self, + info: &PendingApproval, + answer: Sender, + ) -> Result, String>, Stop> { + let id = info.approval; + let (channel, mut root) = match self.place(info) { + Ok(place) => place, + Err(why) => return Ok(Err(why)), + }; + let shown = text::render(info); + let mut posts = Vec::new(); + let mut shown_at = 0; + for body in &shown.posts { + match self.client.create_post(&channel, &root, body) { + Ok(post) => { + if root.is_empty() { + root = post.id.clone(); + } + shown_at = post.create_at; + posts.push(post.id); + } + Err(e) => { + // Parts already posted must not look answerable. + if let (Some(last), Some(text)) = + (posts.last(), shown.posts.get(posts.len().wrapping_sub(1))) + { + self.mark(id, last, &text::with_ending(text, &Ended::NoLonger)); + } + return Ok(Err(format!("cannot post approval {id}: {e}"))); + } + } + } + self.state.add_approval(ApprovalPosts { + approval: id, + channel: channel.clone(), + posts: posts.clone(), + })?; + self.registry.add(Posted { + approval: id, + session: info.session.as_str().to_string(), + channel, + root, + shown_at, + answerable: shown.answerable, + answer, + }); + if let (Some(last_post), Some(last_text)) = (posts.last(), shown.posts.last()) { + self.open.insert( + id, + Open { + last_post: last_post.clone(), + last_text: last_text.clone(), + }, + ); + } + Ok(Ok(posts)) + } + + /// An approval ended: forget it, and add how it ended to its last post. + fn ended(&mut self, id: u64, ending: Ending) -> Result<(), Stop> { + // 1. `self.registry.remove(id);` and `self.state.remove_approval(id)?;`. + // 2. `let Some(open) = self.open.remove(&id) else { return Ok(()); };` + // 3. The `Ended`: Ending::NoLonger -> Ended::NoLonger. Ending::Answered { by, reason, + // outcome } -> `by` becomes "@" from `self.client.username(&by)`, or stays the + // id if that fails; then outcome Runs -> Ended::Runs { by }, Denied { reason } -> + // Ended::Denied { by, reason }, Refused -> Ended::Refused { by, reason }, Gone -> + // Ended::NoLonger. + // 4. `self.mark(id, &open.last_post, &text::with_ending(&open.last_text, &ended));` and + // Ok(()). + todo!() + } + + /// At the first connection after a start: the approval posts the last run left are no longer + /// answerable. Their last post is replaced by its title and the restart line. + pub(super) fn mark_left_behind(&mut self) -> Result<(), Stop> { + // For each `left` in `self.state.take_approvals()?`: if it has a last post, + // `self.mark(left.approval, last, &body)` with `body = format!("**Approval {}**\n\n{}", + // left.approval, text::ended_line(&Ended::Restarted))`. Then Ok(()). + todo!() + } + + /// Edit a post; a failure is logged, not retried. + fn mark(&self, id: u64, post: &str, body: &str) { + // `if let Err(e) = self.client.patch_post(post, body)`: log "gatewayd: cannot mark approval + // {id} as ended: {e}" with `(self.log)(&line)`. + todo!() + } +} diff --git a/docs/plans/M4b/files/crates/gatewayd/tests/approvals_judge.rs b/docs/plans/M4b/files/crates/gatewayd/tests/approvals_judge.rs new file mode 100644 index 0000000..0d225c8 --- /dev/null +++ b/docs/plans/M4b/files/crates/gatewayd/tests/approvals_judge.rs @@ -0,0 +1,166 @@ +//! Which commands answer an approval, in the order of the M4b spec, section 7; everything else is +//! answered with its reason and changes nothing. Do not edit. + +use std::sync::mpsc::{self, Receiver}; + +use gatewayd::approvals::{Judged, NOT_APPROVER, Posted, Registry, USAGE}; +use gatewayd::sessions::ApprovalCommand::{self, Approve, Deny, Malformed}; +use proto::ApprovalReply; + +const KYLE: &str = "k0000000000000000000000000"; +const HERMES: &str = "h0000000000000000000000000"; +const DM: &str = "d0000000000000000000000000"; +const ROOT: &str = "r0000000000000000000000000"; +const ELSEWHERE: &str = "e0000000000000000000000000"; +const CMD: &str = "c0000000000000000000000001"; + +fn approvers() -> Vec { + vec![KYLE.to_string()] +} + +/// A registry with approval 42 posted in ROOT, shown at 100. +fn posted(answerable: bool) -> (Registry, Receiver) { + let registry = Registry::new(); + let (tx, rx) = mpsc::channel(); + registry.add(Posted { + approval: 42, + session: format!("mm-{ROOT}"), + channel: DM.to_string(), + root: ROOT.to_string(), + shown_at: 100, + answerable, + answer: tx, + }); + (registry, rx) +} + +fn judge(r: &Registry, command: &ApprovalCommand, user: &str, root: &str, at: i64) -> Judged { + r.judge(command, user, root, CMD, at, &approvers()) +} + +#[test] +fn an_approver_in_the_thread_after_the_post_answers_it() { + let (r, rx) = posted(true); + assert_eq!(judge(&r, &Approve(42), KYLE, ROOT, 101), Judged::Sent); + assert_eq!( + rx.try_recv().unwrap(), + ApprovalReply { + approval: 42, + approved: true, + by: KYLE.to_string(), + post: CMD.to_string(), + reason: None + } + ); + assert!(!r.holds(42), "answered once"); + assert_eq!( + judge(&r, &Approve(42), KYLE, ROOT, 102), + Judged::Reply("approval 42 is not pending here".to_string()) + ); + assert!(rx.try_recv().is_err(), "a second answer goes nowhere"); +} + +#[test] +fn a_refusal_carries_its_reason() { + let (r, rx) = posted(true); + assert_eq!( + judge( + &r, + &Deny(42, Some("not on a Friday".to_string())), + KYLE, + ROOT, + 101 + ), + Judged::Sent + ); + let got = rx.try_recv().unwrap(); + assert_eq!( + (got.approved, got.reason.as_deref()), + (false, Some("not on a Friday")) + ); + let (r, rx) = posted(true); + assert_eq!(judge(&r, &Deny(42, None), KYLE, ROOT, 101), Judged::Sent); + assert_eq!(rx.try_recv().unwrap().reason, None); +} + +#[test] +fn everything_else_is_refused_with_its_reason_in_order() { + let cases: &[(&ApprovalCommand, &str, &str, i64, &str)] = &[ + (&Approve(42), HERMES, ROOT, 101, NOT_APPROVER), + (&Malformed, HERMES, ROOT, 101, NOT_APPROVER), + (&Malformed, KYLE, ROOT, 101, USAGE), + ( + &Approve(7), + KYLE, + ROOT, + 101, + "approval 7 is not pending here", + ), + ( + &Approve(42), + KYLE, + ELSEWHERE, + 101, + "approval 42 is answered in its own thread", + ), + ( + &Approve(42), + KYLE, + ROOT, + 100, + "approval 42 was answered before it was shown in full", + ), + ( + &Approve(42), + KYLE, + ROOT, + 50, + "approval 42 was answered before it was shown in full", + ), + ( + &Approve(42), + KYLE, + ELSEWHERE, + 50, + "approval 42 is answered in its own thread", + ), + ]; + for (command, user, root, at, want) in cases { + let (r, rx) = posted(true); + assert_eq!( + judge(&r, command, user, root, *at), + Judged::Reply(want.to_string()), + "{command:?} {user} {root} {at}" + ); + assert!(r.holds(42), "still pending after {command:?}"); + assert!(rx.try_recv().is_err(), "nothing sent for {command:?}"); + } +} + +#[test] +fn too_long_to_show_is_answered_only_in_bxctl() { + let (r, rx) = posted(false); + assert_eq!( + judge(&r, &Approve(42), KYLE, ROOT, 101), + Judged::Reply("approval 42 can only be answered with bxctl approvals".to_string()) + ); + assert!(r.holds(42) && rx.try_recv().is_err()); +} + +#[test] +fn an_answer_whose_connection_is_gone_is_not_pending() { + let (r, rx) = posted(true); + drop(rx); + assert_eq!( + judge(&r, &Approve(42), KYLE, ROOT, 101), + Judged::Reply("approval 42 is not pending here".to_string()) + ); +} + +#[test] +fn the_waiting_sessions_and_removal() { + let (r, _rx) = posted(true); + assert_eq!(r.waiting_sessions(), [format!("mm-{ROOT}")]); + assert_eq!(r.remove(42).map(|p| p.approval), Some(42)); + assert!(r.waiting_sessions().is_empty() && !r.holds(42) && r.remove(42).is_none()); +} diff --git a/docs/plans/M4b/files/crates/gatewayd/tests/approvals_text.rs b/docs/plans/M4b/files/crates/gatewayd/tests/approvals_text.rs new file mode 100644 index 0000000..fd6922b --- /dev/null +++ b/docs/plans/M4b/files/crates/gatewayd/tests/approvals_text.rs @@ -0,0 +1,197 @@ +//! The text of an approval's posts and of its ending line (M4b spec, section 6). Do not edit. + +use gatewayd::approvals::text::{ + Ended, MAX_PART_POST, MAX_PARTS, PART_CHARS, code, ended_line, fence, render, with_ending, +}; +use proto::{CallId, DataClass, DenyReason, PendingApproval, SessionId, Timestamp, sha256}; + +/// The escape for one code point, built here so this file holds no such character itself. +fn esc(code: u32) -> String { + format!("{}u{code:04x}", char::from(92)) +} + +fn info(arguments: &str) -> PendingApproval { + PendingApproval { + approval: 42, + session: SessionId::new("mm-r0000000000000000000000001").unwrap(), + call: CallId(3), + tool: "shell".to_string(), + arguments: arguments.to_string(), + grant: "ops-shell".to_string(), + taint: DataClass::Private, + created: Timestamp::parse("2026-09-24T14:32:10.000Z").unwrap(), + expires: Timestamp::parse("2026-09-24T14:47:10.000Z").unwrap(), + } +} + +#[test] +fn one_part_exactly() { + let shown = render(&info( + r#"{"command":"ls -la /srv/backups","timeout_ms":10000}"#, + )); + assert!(shown.answerable); + assert_eq!( + shown.posts, + [ + "**Approval 42** · tool `shell` · grant `ops-shell` · taint private · expires 2026-09-24 14:47 UTC\n```\n{\"command\":\"ls -la /srv/backups\",\"timeout_ms\":10000}\n```\nReply in this thread with `!approve 42` or `!deny 42 `." + ] + ); +} + +#[test] +fn fences_and_code_outrun_backticks() { + assert_eq!(fence("a"), "```\na\n```"); + assert_eq!(fence("a ``` b ```` c"), "`````\na ``` b ```` c\n`````"); + assert_eq!(code("ops"), "`ops`"); + assert_eq!(code("a`b"), "``a`b``"); + assert_eq!(code("`x"), "`` `x ``"); + let shown = render(&info(r#"{"command":"echo ```` done"}"#)); + assert!( + shown.posts[0].contains("\n`````\n{\"command\":\"echo ```` done\"}\n`````\n"), + "{}", + shown.posts[0] + ); +} + +#[test] +fn what_must_not_reach_the_reader_raw_is_escaped() { + let raw = "a\u{7}b\u{202e}c\u{200b}d\u{2028}e\u{feff}f\nnewline"; + let shown = render(&info(&format!("{{\"command\":\"{raw}\"}}"))); + let post = &shown.posts[0]; + for bad in ['\u{7}', '\u{202e}', '\u{200b}', '\u{2028}', '\u{feff}'] { + assert!(!post.contains(bad), "{bad:?} reached the post"); + } + let want = format!( + "a{}b{}c{}d{}e{}f{}newline", + esc(0x7), + esc(0x202e), + esc(0x200b), + esc(0x2028), + esc(0xfeff), + esc(0xa) + ); + assert!(post.contains(&want), "{post}"); + let mut odd = info("{}"); + odd.grant = "g`rant\u{202e}".to_string(); + assert!( + render(&odd).posts[0].contains(&format!("grant ``g`rant{}``", esc(0x202e))), + "{}", + render(&odd).posts[0] + ); +} + +#[test] +fn long_arguments_come_in_parts_with_a_hash() { + let args = format!("{{\"content\":\"{}\"}}", "é".repeat(PART_CHARS * 2)); + let shown = render(&info(&args)); + assert!(shown.answerable); + assert_eq!(shown.posts.len(), 3); + let total = args.chars().count(); + assert!(shown.posts[0].starts_with("**Approval 42** · tool `shell` · grant `ops-shell` · taint private · expires 2026-09-24 14:47 UTC\nPart 1 of 3 of the arguments:\n```\n{\"content\":\"")); + assert!(shown.posts[1].starts_with("**Approval 42**, part 2 of 3 of the arguments:\n```\n")); + let hash = sha256(args.as_bytes()).unwrap().to_hex(); + let last = &shown.posts[2]; + assert!(last.ends_with(&format!("\nApproval 42: 3 parts, {total} characters, sha256 {hash} (of the arguments as shown).\nReply in this thread with `!approve 42` or `!deny 42 `.")), "{last}"); + assert!( + !shown.posts[0].contains("Reply in this thread") + && !shown.posts[1].contains("Reply in this thread") + ); + let shown_again: String = shown + .posts + .iter() + .map(|p| { + let body = p.split_once("\n```\n").unwrap().1; + body.split_once("\n```").unwrap().0.to_string() + }) + .collect(); + assert_eq!( + shown_again, args, + "the parts, joined, are the arguments: nothing lost" + ); + assert!( + shown + .posts + .iter() + .all(|p| p.chars().count() <= MAX_PART_POST) + ); +} + +#[test] +fn too_long_to_show_is_answerable_only_in_bxctl() { + let args = "x".repeat(PART_CHARS * MAX_PARTS + 1); + let shown = render(&info(&args)); + assert!(!shown.answerable); + let hash = sha256(args.as_bytes()).unwrap().to_hex(); + assert_eq!( + shown.posts, + [format!( + "**Approval 42** · tool `shell` · grant `ops-shell` · taint private · expires 2026-09-24 14:47 UTC\nThe arguments are {} characters (sha256 {hash}), too long to show here; answer it with `bxctl approvals`.", + args.len() + )] + ); + let exactly = "x".repeat(PART_CHARS * MAX_PARTS); + assert!( + render(&info(&exactly)).answerable, + "four full parts still show" + ); + let ticks = format!("{}{}", "`".repeat(PART_CHARS - 10), "x".repeat(20)); + assert!( + !render(&info(&ticks)).answerable, + "a fence that would overflow the post is not shown" + ); +} + +#[test] +fn ending_lines() { + let kyle = "@kyle".to_string(); + assert_eq!( + ended_line(&Ended::Runs { by: kyle.clone() }), + "✅ Approved by @kyle: the call runs." + ); + assert_eq!( + ended_line(&Ended::Denied { + by: kyle.clone(), + reason: DenyReason::GrantExpired + }), + "Approved by @kyle, but denied on the re-check: grant_expired." + ); + assert_eq!( + ended_line(&Ended::Refused { + by: kyle.clone(), + reason: None + }), + "❌ Refused by @kyle: no reason given." + ); + assert_eq!( + ended_line(&Ended::Refused { + by: kyle.clone(), + reason: Some("not @channel `now`".to_string()) + }), + "❌ Refused by @kyle: `` not @channel `now` ``." + ); + assert_eq!( + ended_line(&Ended::NoLonger), + "No longer pending (answered elsewhere, expired, or its turn ended)." + ); + assert_eq!( + ended_line(&Ended::Restarted), + "No longer answerable here: gatewayd restarted. If it is still pending it is posted again." + ); + assert_eq!( + with_ending("post", &Ended::NoLonger), + "post\n\nNo longer pending (answered elsewhere, expired, or its turn ended)." + ); + let longest = render(&info(&"y".repeat(PART_CHARS))).posts.remove(0); + let ended = with_ending( + &longest, + &Ended::Refused { + by: format!("@{}", "u".repeat(64)), + reason: Some("r".repeat(500)), + }, + ); + assert!( + ended.chars().count() <= 16_383, + "an ended post still fits: {}", + ended.chars().count() + ); +} diff --git a/docs/plans/M4b/files/crates/gatewayd/tests/config_approvals.rs b/docs/plans/M4b/files/crates/gatewayd/tests/config_approvals.rs new file mode 100644 index 0000000..136345d --- /dev/null +++ b/docs/plans/M4b/files/crates/gatewayd/tests/config_approvals.rs @@ -0,0 +1,68 @@ +//! `[approvals]` in `gatewayd.toml`: who may answer, the first approver, and `gateway.sock` +//! (M4b spec, section 3). Do not edit. + +use std::path::PathBuf; + +use gatewayd::config::Config; + +const KYLE: &str = "k0000000000000000000000000"; +const HERMES: &str = "h0000000000000000000000000"; +const OTHER: &str = "o0000000000000000000000000"; + +fn text(approvals: &str) -> String { + format!( + "[mattermost]\nurl = \"https://chat.example\"\n[secrets.mattermost_token]\nenv = \"T\"\n[allow]\nusers = [\"{KYLE}\", \"{HERMES}\"]\n[paths]\nhome = \"/srv/bx\"\n{approvals}" + ) +} + +fn config(approvals: &str) -> Config { + let c = Config::parse(&text(approvals)).unwrap(); + assert_eq!(c.problem(), None); + c +} + +#[test] +fn without_the_section_everyone_allowed_may_answer() { + let c = config(""); + assert_eq!(c.approvers(), [KYLE, HERMES]); + assert_eq!(c.first_approver(), Some(KYLE)); + assert_eq!( + c.gateway_socket(), + PathBuf::from("/srv/bx/run/gateway/gateway.sock") + ); + assert_eq!( + config("[approvals]\nusers = []\n").approvers(), + [KYLE, HERMES] + ); +} + +#[test] +fn a_list_narrows_who_may_answer() { + let c = config(&format!( + "[approvals]\nusers = [\"{HERMES}\"]\nsocket = \"/run/gw.sock\"\n" + )); + assert_eq!(c.approvers(), [HERMES]); + assert_eq!(c.first_approver(), Some(HERMES)); + assert_eq!(c.gateway_socket(), PathBuf::from("/run/gw.sock")); +} + +#[test] +fn an_approver_must_be_heard() { + let c = Config::parse(&text(&format!( + "[approvals]\nusers = [\"{KYLE}\", \"{OTHER}\"]\n" + ))) + .unwrap(); + let why = c.problem().expect("refused"); + assert_eq!( + why, + format!( + "[approvals] \"{OTHER}\" is not in [allow] users: an approver who is not heard could never answer" + ) + ); +} + +#[test] +fn unknown_keys_are_errors() { + assert!(Config::parse(&text("[approvals]\napprovers = []\n")).is_err()); + assert!(Config::parse(&text("[approvals]\nusers = []\nttl = 5\n")).is_err()); +} diff --git a/docs/plans/M4b/files/crates/gatewayd/tests/deliver.rs b/docs/plans/M4b/files/crates/gatewayd/tests/deliver.rs new file mode 100644 index 0000000..02ffb64 --- /dev/null +++ b/docs/plans/M4b/files/crates/gatewayd/tests/deliver.rs @@ -0,0 +1,313 @@ +//! A turn on `loop.sock` and its answer in the thread, against a fake `loopd`: the answer, long +//! answers, approvals, errors, an unknown session, and a loop that is not there (M4a spec, section +//! 8; M4b spec, section 7, for the notice). Do not edit. + +#[path = "support/fake_loop.rs"] +mod fake_loop; +#[path = "support/tmp.rs"] +mod tmp; + +use std::sync::Mutex; +use std::time::Duration; + +use fake_loop::{Reply, done, error, event, serve_loop}; +use gatewayd::deliver::{EMPTY_ANSWER, LOOP_DOWN, MAX_POST, Notice, Poster, deliver, split_answer}; +use gatewayd::mm::MmError; +use gatewayd::sessions::{Batch, Thread}; +use proto::{DataClass, ErrorCode, SessionId, Timestamp, TurnEvent}; +use tmp::TempDir; + +const DM: &str = "d0000000000000000000000000"; +const ROOT: &str = "r0000000000000000000000000"; + +#[derive(Default)] +struct Record { + posts: Mutex>, + fail: bool, +} + +impl Poster for Record { + fn post(&self, channel: &str, root: &str, text: &str) -> Result<(), MmError> { + if self.fail { + return Err(MmError::Status(500, "down".to_string())); + } + self.posts + .lock() + .unwrap() + .push((channel.to_string(), root.to_string(), text.to_string())); + Ok(()) + } +} + +impl Record { + fn texts(&self) -> Vec { + let posts = self.posts.lock().unwrap(); + assert!( + posts.iter().all(|(c, r, _)| c == DM && r == ROOT), + "every post in the thread" + ); + posts.iter().map(|(_, _, t)| t.clone()).collect() + } +} + +fn batch(resume: bool, text: &str) -> Batch { + Batch { + session: SessionId::new(&format!("mm-{ROOT}")).unwrap(), + thread: Thread { + channel: DM.to_string(), + root: ROOT.to_string(), + }, + resume, + text: text.to_string(), + } +} + +fn run(dir: &TempDir, poster: &Record, batch: &Batch) -> Vec { + let log = Mutex::new(Vec::new()); + let never = |_: u64| false; + let notice = Notice { + posted: &never, + wait: Duration::ZERO, + }; + deliver( + poster, + &dir.path().join("loop.sock"), + batch, + &|line| log.lock().unwrap().push(line.to_string()), + ¬ice, + ); + log.into_inner().unwrap() +} + +#[test] +fn the_answer_is_posted_and_nothing_else() { + let dir = TempDir::new("deliver-answer"); + let turns = serve_loop(&dir.path().join("loop.sock"), |_, _| { + vec![ + event(TurnEvent::Progress { + total: 10, + cache: 0, + processed: 10, + }), + event(TurnEvent::Reasoning { + text: "private thoughts".to_string(), + }), + event(TurnEvent::ToolCallStarted { + name: "read_file".to_string(), + }), + event(TurnEvent::ToolResult { + name: "read_file".to_string(), + class: DataClass::Private, + truncated: false, + }), + event(TurnEvent::Content { + text: "The ans".to_string(), + }), + done("The answer."), + ] + }); + let poster = Record::default(); + let log = run(&dir, &poster, &batch(true, "one\n\ntwo")); + assert_eq!(poster.texts(), ["The answer."]); + assert!(log.is_empty(), "{log:?}"); + let turn = turns.recv_timeout(Duration::from_secs(5)).unwrap(); + assert_eq!( + (turn.session.as_str(), turn.content.as_str(), turn.resume), + (format!("mm-{ROOT}").as_str(), "one\n\ntwo", true) + ); +} + +#[test] +fn an_approval_is_announced_once_before_the_answer() { + let dir = TempDir::new("deliver-approval"); + let expires = Timestamp::from_unix_millis(1_758_650_000_000).unwrap(); + let _turns = serve_loop(&dir.path().join("loop.sock"), move |_, _| { + vec![ + event(TurnEvent::ApprovalPending { + approval: 42, + tool: "shell".to_string(), + expires, + }), + done("done"), + ] + }); + let poster = Record::default(); + run(&dir, &poster, &batch(false, "go")); + assert_eq!( + poster.texts(), + [ + "waiting for approval 42: brokerd has not sent it here; answer it with bxctl approvals", + "done" + ] + ); +} + +#[test] +fn errors_are_posted_with_their_code() { + let dir = TempDir::new("deliver-error"); + let _turns = serve_loop(&dir.path().join("loop.sock"), |_, _| { + vec![error( + ErrorCode::Inference, + "the model server failed\nsee docs/runbook.md#loopd-selftest-failed", + )] + }); + let poster = Record::default(); + run(&dir, &poster, &batch(false, "go")); + assert_eq!( + poster.texts(), + ["Error: inference: the model server failed\nsee docs/runbook.md#loopd-selftest-failed"] + ); +} + +#[test] +fn a_reply_in_a_thread_loopd_does_not_know_creates_the_session() { + let dir = TempDir::new("deliver-unknown"); + let turns = serve_loop(&dir.path().join("loop.sock"), |n, _| { + if n == 0 { + vec![error(ErrorCode::NoSuchSession, "no such session")] + } else { + vec![done("hello")] + } + }); + let poster = Record::default(); + run(&dir, &poster, &batch(true, "hi")); + assert_eq!(poster.texts(), ["hello"]); + let first = turns.recv_timeout(Duration::from_secs(5)).unwrap(); + let second = turns.recv_timeout(Duration::from_secs(5)).unwrap(); + assert_eq!( + (first.resume, second.resume, second.content.as_str()), + (true, false, "hi") + ); +} + +#[test] +fn a_new_session_is_not_retried() { + let dir = TempDir::new("deliver-noretry"); + let turns = serve_loop(&dir.path().join("loop.sock"), |_, _| { + vec![error(ErrorCode::NoSuchSession, "odd")] + }); + let poster = Record::default(); + run(&dir, &poster, &batch(false, "hi")); + assert_eq!(poster.texts(), ["Error: no_such_session: odd"]); + assert!(turns.recv_timeout(Duration::from_secs(5)).is_ok()); + assert!( + turns.recv_timeout(Duration::from_millis(200)).is_err(), + "one turn only" + ); +} + +#[test] +fn a_loop_that_is_not_there_or_goes_away() { + let dir = TempDir::new("deliver-down"); + let poster = Record::default(); + let log = run(&dir, &poster, &batch(false, "hi")); + assert_eq!(poster.texts(), [LOOP_DOWN]); + assert_eq!(log.len(), 1, "{log:?}"); + assert!( + log[0].starts_with(&format!("gatewayd: mm-{ROOT}: cannot connect to ")), + "{log:?}" + ); + + for (n, replies) in [ + vec![ + event(TurnEvent::Content { + text: "x".to_string(), + }), + Reply::Close, + ], + vec![Reply::Bytes(b"\x00\x00\x00\x05{bad}".to_vec())], + vec![ + Reply::Frame(proto::Envelope { + v: 1, + id: 2, + r#final: true, + msg: proto::Message::Ok(proto::Empty {}), + }), + done("late"), + ], + ] + .into_iter() + .enumerate() + { + let dir = TempDir::new(&format!("deliver-early-{n}")); + let replies = Mutex::new(Some(replies)); + let _turns = serve_loop(&dir.path().join("loop.sock"), move |_, _| { + replies.lock().unwrap().take().unwrap_or_default() + }); + let poster = Record::default(); + let log = run(&dir, &poster, &batch(false, "hi")); + assert_eq!(poster.texts(), [LOOP_DOWN], "case {n}"); + assert_eq!(log.len(), 1, "case {n}: {log:?}"); + } +} + +#[test] +fn a_post_that_fails_is_logged() { + let dir = TempDir::new("deliver-postfail"); + let _turns = serve_loop(&dir.path().join("loop.sock"), |_, _| vec![done("lost")]); + let poster = Record { + fail: true, + ..Record::default() + }; + let log = run(&dir, &poster, &batch(false, "hi")); + assert_eq!( + log, + [format!( + "gatewayd: cannot post in {DM} (thread {ROOT}): status 500: \"down\"" + )] + ); +} + +#[test] +fn long_answers_are_split_at_newlines() { + let short = "a".repeat(MAX_POST); + assert_eq!(split_answer(&short), [short.as_str()]); + let over = "a".repeat(MAX_POST + 1); + assert_eq!(split_answer(&over), ["a".repeat(MAX_POST), "a".to_string()]); + let lines = format!( + "{}\n{}\n{}", + "a".repeat(10_000), + "b".repeat(5_000), + "c".repeat(2_000) + ); + assert_eq!( + split_answer(&lines), + [ + format!("{}\n{}", "a".repeat(10_000), "b".repeat(5_000)), + "c".repeat(2_000) + ] + ); + let wide = "é".repeat(MAX_POST + 5); + let parts = split_answer(&wide); + assert_eq!( + parts.iter().map(|p| p.chars().count()).collect::>(), + [MAX_POST, 5], + "characters, not bytes" + ); + let leading = format!("\n{}", "x".repeat(MAX_POST + 1)); + let parts = split_answer(&leading); + assert!( + parts + .iter() + .all(|p| !p.is_empty() && p.chars().count() <= MAX_POST), + "{:?}", + parts.iter().map(|p| p.len()).collect::>() + ); + assert_eq!(parts.concat(), leading, "a hard cut drops nothing"); + assert_eq!(split_answer(""), [EMPTY_ANSWER]); + assert_eq!(split_answer(" \n "), [EMPTY_ANSWER]); +} + +#[test] +fn a_long_answer_is_posted_in_order() { + let dir = TempDir::new("deliver-long"); + let answer = format!("{}\n{}", "a".repeat(MAX_POST - 1), "b".repeat(MAX_POST)); + let sent = answer.clone(); + let _turns = serve_loop(&dir.path().join("loop.sock"), move |_, _| vec![done(&sent)]); + let poster = Record::default(); + run(&dir, &poster, &batch(false, "hi")); + assert_eq!( + poster.texts(), + ["a".repeat(MAX_POST - 1), "b".repeat(MAX_POST)] + ); +} diff --git a/docs/plans/M4b/files/crates/gatewayd/tests/deliver_notice.rs b/docs/plans/M4b/files/crates/gatewayd/tests/deliver_notice.rs new file mode 100644 index 0000000..1fdf0cb --- /dev/null +++ b/docs/plans/M4b/files/crates/gatewayd/tests/deliver_notice.rs @@ -0,0 +1,95 @@ +//! The notice for an approval that `brokerd` did not bring: posted only when it has not arrived +//! within the wait (M4b spec, section 7). Do not edit. + +#[path = "support/fake_loop.rs"] +mod fake_loop; +#[path = "support/tmp.rs"] +mod tmp; + +use std::sync::Mutex; +use std::time::{Duration, Instant}; + +use fake_loop::{done, event, serve_loop}; +use gatewayd::deliver::{Notice, Poster, deliver}; +use gatewayd::mm::MmError; +use gatewayd::sessions::{Batch, Thread}; +use proto::{SessionId, Timestamp, TurnEvent}; +use tmp::TempDir; + +const DM: &str = "d0000000000000000000000000"; +const ROOT: &str = "r0000000000000000000000000"; +const NOTICE: &str = + "waiting for approval 42: brokerd has not sent it here; answer it with bxctl approvals"; + +#[derive(Default)] +struct Record(Mutex>); + +impl Poster for Record { + fn post(&self, _: &str, _: &str, text: &str) -> Result<(), MmError> { + self.0.lock().unwrap().push(text.to_string()); + Ok(()) + } +} + +/// A turn whose loop says approval 42 is pending, then answers; what was posted, and how long it took. +fn turn(tag: &str, posted: &dyn Fn(u64) -> bool, wait: Duration) -> (Vec, Duration) { + let dir = TempDir::new(tag); + let socket = dir.path().join("loop.sock"); + let expires = Timestamp::parse("2026-09-24T08:20:00.000Z").unwrap(); + let _turns = serve_loop(&socket, move |_, _| { + vec![ + event(TurnEvent::ApprovalPending { + approval: 42, + tool: "read_file".to_string(), + expires, + }), + done("done"), + ] + }); + let batch = Batch { + session: SessionId::new(&format!("mm-{ROOT}")).unwrap(), + thread: Thread { + channel: DM.to_string(), + root: ROOT.to_string(), + }, + resume: false, + text: "go".to_string(), + }; + let poster = Record::default(); + let started = Instant::now(); + deliver(&poster, &socket, &batch, &|_| {}, &Notice { posted, wait }); + let took = started.elapsed(); + (poster.0.into_inner().unwrap(), took) +} + +#[test] +fn brought_already_no_notice_and_no_wait() { + let (posts, took) = turn("notice-there", &|id| id == 42, Duration::from_secs(5)); + assert_eq!(posts, ["done"]); + assert!(took < Duration::from_secs(1), "{took:?}"); +} + +#[test] +fn brought_during_the_wait_no_notice() { + let started = Instant::now(); + let later = move |id: u64| id == 42 && started.elapsed() > Duration::from_millis(300); + let (posts, took) = turn("notice-later", &later, Duration::from_secs(5)); + assert_eq!(posts, ["done"]); + assert!(took < Duration::from_secs(2), "{took:?}"); +} + +#[test] +fn never_brought_the_notice_after_the_wait() { + let (posts, took) = turn("notice-never", &|_| false, Duration::from_millis(400)); + assert_eq!(posts, [NOTICE, "done"]); + assert!( + took >= Duration::from_millis(400) && took < Duration::from_secs(3), + "{took:?}" + ); +} + +#[test] +fn another_approval_brought_does_not_count() { + let (posts, _) = turn("notice-other", &|id| id == 7, Duration::from_millis(200)); + assert_eq!(posts, [NOTICE, "done"]); +} diff --git a/docs/plans/M4b/files/crates/gatewayd/tests/gateway_sock.rs b/docs/plans/M4b/files/crates/gatewayd/tests/gateway_sock.rs new file mode 100644 index 0000000..112d308 --- /dev/null +++ b/docs/plans/M4b/files/crates/gatewayd/tests/gateway_sock.rs @@ -0,0 +1,288 @@ +//! `gateway.sock` from `gatewayd`'s side: the socket's permissions, and one connection's frames, +//! with the test playing both the serve loop and `brokerd` (M4b spec, sections 4 and 6). Do not +//! edit. + +#[path = "support/tmp.rs"] +mod tmp; + +use std::os::unix::fs::PermissionsExt; +use std::os::unix::net::UnixStream; +use std::path::Path; +use std::sync::mpsc::{self, Receiver}; +use std::sync::{Arc, Mutex}; +use std::time::Duration; + +use gatewayd::gateway::{Ending, Timing, ToLoop, listen, serve}; +use proto::{ + ApprovalOutcome, ApprovalReply, ApprovalRequest, CallId, DataClass, Envelope, ErrorCode, + GatewayOutcome, Message, PROTOCOL_VERSION, PendingApproval, SessionId, Timestamp, read_frame, + write_frame, +}; +use tmp::TempDir; + +const KYLE: &str = "k0000000000000000000000000"; +const POST: &str = "p0000000000000000000000001"; +const CMD: &str = "c0000000000000000000000001"; +const WAIT: Duration = Duration::from_secs(5); + +fn info(approval: u64) -> PendingApproval { + PendingApproval { + approval, + session: SessionId::new("mm-r0000000000000000000000001").unwrap(), + call: CallId(3), + tool: "shell".to_string(), + arguments: "{}".to_string(), + grant: "g".to_string(), + taint: DataClass::Private, + created: Timestamp::parse("2026-09-24T08:05:00.000Z").unwrap(), + expires: Timestamp::parse("2026-09-24T08:20:00.000Z").unwrap(), + } +} + +fn frame(id: u64, last: bool, msg: Message) -> Envelope { + Envelope { + v: PROTOCOL_VERSION, + id, + r#final: last, + msg, + } +} + +fn timing() -> Timing { + Timing { + first_frame: Duration::from_millis(500), + posting: WAIT, + outcome: Duration::from_millis(500), + tick: Duration::from_millis(20), + } +} + +/// A listening gatewayd side: the socket and what it asks of the loop. +fn start( + dir: &TempDir, +) -> ( + std::path::PathBuf, + Receiver, + Arc>>, +) { + let socket = dir.path().join("run/gateway/gateway.sock"); + let listener = listen(&socket).unwrap(); + let (tx, rx) = mpsc::channel(); + let lines = Arc::new(Mutex::new(Vec::new())); + let sink = Arc::clone(&lines); + serve( + listener, + tx, + timing(), + Arc::new(move |l: &str| sink.lock().unwrap().push(l.to_string())), + ) + .unwrap(); + (socket, rx, lines) +} + +/// brokerd: connect and send the request. +fn request(socket: &Path, approval: u64) -> UnixStream { + let mut s = UnixStream::connect(socket).unwrap(); + s.set_read_timeout(Some(WAIT)).unwrap(); + write_frame( + &mut s, + &frame( + 1, + true, + Message::ApprovalRequest(ApprovalRequest { + approval: info(approval), + }), + ), + ) + .unwrap(); + s +} + +fn reply(approval: u64) -> ApprovalReply { + ApprovalReply { + approval, + approved: true, + by: KYLE.to_string(), + post: CMD.to_string(), + reason: None, + } +} + +#[test] +fn the_socket_is_private_and_a_stale_one_is_replaced() { + let dir = TempDir::new("gw-perms"); + let socket = dir.path().join("run/gateway/gateway.sock"); + drop(listen(&socket).unwrap()); + let listener = listen(&socket).expect("a stale socket is replaced"); + let mode = |p: &Path| std::fs::metadata(p).unwrap().permissions().mode() & 0o777; + assert_eq!( + (mode(&socket), mode(socket.parent().unwrap())), + (0o600, 0o700) + ); + drop(listener); + let file = dir.path().join("run/gateway/not-a-socket"); + std::fs::write(&file, "x").unwrap(); + let why = listen(&file).unwrap_err(); + assert!( + why.starts_with(&format!("cannot listen on {}", file.display())), + "{why}" + ); + assert!( + dir.path().join("run/gateway/not-a-socket").exists(), + "a file that is not a socket is left alone" + ); +} + +#[test] +fn posted_answered_and_ended() { + let dir = TempDir::new("gw-answer"); + let (socket, to_loop, _) = start(&dir); + let mut brokerd = request(&socket, 42); + let (answer, shown) = match to_loop.recv_timeout(WAIT).unwrap() { + ToLoop::Show { + info: got, + answer, + shown, + } => { + assert_eq!(got, info(42)); + (answer, shown) + } + other => panic!("{other:?}"), + }; + shown.send(Ok(vec![POST.to_string()])).unwrap(); + let posted = read_frame(&mut brokerd).unwrap(); + assert_eq!((posted.id, posted.r#final), (1, false)); + assert!( + matches!(posted.msg, Message::ApprovalPosted(ref p) if p.posts == [POST]), + "{posted:?}" + ); + answer.send(reply(42)).unwrap(); + let answered = read_frame(&mut brokerd).unwrap(); + assert_eq!(answered, frame(1, true, Message::ApprovalReply(reply(42)))); + write_frame( + &mut brokerd, + &frame( + 2, + true, + Message::ApprovalOutcome(ApprovalOutcome { + outcome: GatewayOutcome::Runs {}, + }), + ), + ) + .unwrap(); + match to_loop.recv_timeout(WAIT).unwrap() { + ToLoop::Ended { approval, ending } => { + assert_eq!(approval, 42); + assert_eq!( + ending, + Ending::Answered { + by: KYLE.to_string(), + reason: None, + outcome: GatewayOutcome::Runs {} + } + ); + } + other => panic!("{other:?}"), + } +} + +#[test] +fn brokerd_closing_first_ends_it_as_no_longer_pending() { + let dir = TempDir::new("gw-closed"); + let (socket, to_loop, _) = start(&dir); + let mut brokerd = request(&socket, 7); + let ToLoop::Show { shown, answer, .. } = to_loop.recv_timeout(WAIT).unwrap() else { + panic!() + }; + shown.send(Ok(vec![POST.to_string()])).unwrap(); + read_frame(&mut brokerd).unwrap(); + drop(brokerd); + match to_loop.recv_timeout(WAIT).unwrap() { + ToLoop::Ended { approval, ending } => assert_eq!((approval, ending), (7, Ending::NoLonger)), + other => panic!("{other:?}"), + } + assert!( + answer.send(reply(7)).is_err(), + "nobody listens for an answer any more" + ); +} + +#[test] +fn an_approval_that_cannot_be_posted_is_an_error_to_brokerd() { + let dir = TempDir::new("gw-cannot"); + let (socket, to_loop, _) = start(&dir); + let mut brokerd = request(&socket, 8); + let ToLoop::Show { shown, .. } = to_loop.recv_timeout(WAIT).unwrap() else { + panic!() + }; + shown.send(Err("Mattermost is down".to_string())).unwrap(); + let got = read_frame(&mut brokerd).unwrap(); + assert_eq!((got.id, got.r#final), (1, true)); + match got.msg { + Message::Error(e) => assert_eq!( + (e.code, e.detail.as_str()), + (ErrorCode::Internal, "Mattermost is down") + ), + other => panic!("{other:?}"), + } + assert!( + to_loop.recv_timeout(Duration::from_millis(300)).is_err(), + "nothing more for the loop" + ); +} + +#[test] +fn no_outcome_after_the_answer_is_no_longer_pending_and_logged() { + let dir = TempDir::new("gw-silent"); + let (socket, to_loop, lines) = start(&dir); + let mut brokerd = request(&socket, 9); + let ToLoop::Show { shown, answer, .. } = to_loop.recv_timeout(WAIT).unwrap() else { + panic!() + }; + shown.send(Ok(vec![POST.to_string()])).unwrap(); + read_frame(&mut brokerd).unwrap(); + answer.send(reply(9)).unwrap(); + read_frame(&mut brokerd).unwrap(); + match to_loop.recv_timeout(WAIT).unwrap() { + ToLoop::Ended { approval, ending } => assert_eq!((approval, ending), (9, Ending::NoLonger)), + other => panic!("{other:?}"), + } + assert_eq!( + *lines.lock().unwrap(), + ["gatewayd: approval 9: brokerd said nothing after the answer"] + ); +} + +#[test] +fn anything_but_a_request_first_is_dropped() { + let dir = TempDir::new("gw-bad"); + let (socket, to_loop, _) = start(&dir); + for first in [ + frame(1, true, Message::ApprovalReply(reply(1))), + frame( + 2, + true, + Message::ApprovalRequest(ApprovalRequest { approval: info(1) }), + ), + frame( + 1, + false, + Message::ApprovalRequest(ApprovalRequest { approval: info(1) }), + ), + ] { + let mut s = UnixStream::connect(&socket).unwrap(); + s.set_read_timeout(Some(WAIT)).unwrap(); + write_frame(&mut s, &first).unwrap(); + assert!( + read_frame(&mut s).is_err(), + "closed without an answer: {first:?}" + ); + } + let silent = UnixStream::connect(&socket).unwrap(); + std::thread::sleep(Duration::from_millis(700)); + drop(silent); + assert!( + to_loop.recv_timeout(Duration::from_millis(300)).is_err(), + "nothing reached the loop" + ); +} diff --git a/docs/plans/M4b/files/crates/gatewayd/tests/mm_edit.rs b/docs/plans/M4b/files/crates/gatewayd/tests/mm_edit.rs new file mode 100644 index 0000000..47b72cb --- /dev/null +++ b/docs/plans/M4b/files/crates/gatewayd/tests/mm_edit.rs @@ -0,0 +1,129 @@ +//! Editing a post and looking up a username, against a scripted server (M4b spec, sections 6 and +//! 7). Do not edit. + +#[path = "support/http_server.rs"] +mod http_server; +#[path = "support/tls_server.rs"] +mod tls_server; + +use std::net::SocketAddr; +use std::time::Duration; + +use gatewayd::config::ServerUrl; +use gatewayd::mm::MmError; +use gatewayd::mm::rest::Client; +use gatewayd::net::Connector; +use gatewayd::secrets::Secret; +use http_server::{reply, serve_http}; +use serde_json::json; + +const BOT: &str = "b0000000000000000000000000"; +const KYLE: &str = "k0000000000000000000000000"; +const DM: &str = "d0000000000000000000000000"; +const POST: &str = "p0000000000000000000000001"; +const WAIT: Duration = Duration::from_secs(5); + +fn client(addr: SocketAddr) -> Client { + let url = ServerUrl { + tls: false, + host: "localhost".to_string(), + port: addr.port(), + }; + Client::new( + Connector::new(url, None).unwrap(), + Secret::new("TOKEN".to_string()), + WAIT, + ) +} + +#[test] +fn a_post_is_patched_with_its_new_text() { + let (addr, rx) = serve_http(None, |_, r| { + let text = r.json()["message"].as_str().unwrap().to_string(); + let post = json!({"id": POST, "user_id": BOT, "channel_id": DM, "root_id": "", "message": text, "create_at": 5, "is_pinned": false}); + reply(200, "", &post.to_string()) + }); + client(addr) + .patch_post( + POST, + "**Approval 42**\n✅ Approved by @kyle: the call runs.", + ) + .unwrap(); + let r = rx.recv_timeout(WAIT).unwrap(); + assert_eq!( + (r.method.as_str(), r.path.as_str()), + ("PUT", format!("/api/v4/posts/{POST}/patch").as_str()) + ); + assert!( + r.head.contains("Authorization: Bearer TOKEN\r\n"), + "{}", + r.head + ); + assert_eq!( + r.json(), + json!({"message": "**Approval 42**\n✅ Approved by @kyle: the call runs."}) + ); +} + +#[test] +fn a_post_that_is_not_an_id_is_never_sent() { + let (addr, rx) = serve_http(None, |_, _| reply(200, "", "{}")); + for bad in ["../users/me", "", "P0000000000000000000000001"] { + assert!( + matches!(client(addr).patch_post(bad, "x"), Err(MmError::Json(_))), + "{bad:?}" + ); + assert!( + matches!(client(addr).username(bad), Err(MmError::Json(_))), + "{bad:?}" + ); + } + std::thread::sleep(Duration::from_millis(100)); + assert!(rx.try_recv().is_err(), "nothing was sent"); +} + +#[test] +fn a_refused_edit_is_an_error() { + let (addr, _rx) = serve_http(None, |_, _| { + reply( + 403, + "", + r#"{"id":"api.post.update_post.permissions_details.app_error"}"#, + ) + }); + assert!(matches!( + client(addr).patch_post(POST, "x"), + Err(MmError::Auth(403)) + )); +} + +#[test] +fn the_username_of_a_user() { + let (addr, rx) = serve_http(None, |_, _| { + reply( + 200, + "", + &json!({"id": KYLE, "username": "kyle", "first_name": "K", "roles": "system_user"}) + .to_string(), + ) + }); + assert_eq!(client(addr).username(KYLE).unwrap(), "kyle"); + let r = rx.recv_timeout(WAIT).unwrap(); + assert_eq!( + (r.method.as_str(), r.path.as_str()), + ("GET", format!("/api/v4/users/{KYLE}").as_str()) + ); + for bad in [ + json!({"id": KYLE}), + json!({"id": KYLE, "username": ""}), + json!({"id": KYLE, "username": "Kyle\n@channel"}), + json!({"id": KYLE, "username": "x".repeat(65)}), + ] { + let body = bad.to_string(); + let (addr, _rx) = serve_http(None, move |_, _| reply(200, "", &body)); + assert!( + matches!(client(addr).username(KYLE), Err(MmError::Json(_))), + "{bad}" + ); + } +} diff --git a/docs/plans/M4b/files/crates/gatewayd/tests/serve.rs b/docs/plans/M4b/files/crates/gatewayd/tests/serve.rs new file mode 100644 index 0000000..6227145 --- /dev/null +++ b/docs/plans/M4b/files/crates/gatewayd/tests/serve.rs @@ -0,0 +1,214 @@ +//! `gatewayd` end to end, against a fake Mattermost and a fake `loopd`: who is answered, where, +//! and how (M4a spec, sections 7 and 8). Do not edit. + +#[path = "support/fake_loop.rs"] +mod fake_loop; +#[path = "support/fake_mm.rs"] +mod fake_mm; +#[path = "support/gateway.rs"] +mod gateway; +#[path = "support/tmp.rs"] +mod tmp; + +use std::sync::{Mutex, mpsc}; +use std::time::Duration; + +use fake_loop::{done, serve_loop}; +use fake_mm::{BOT, BOT_NAME, DM, EVE, EVE_DM, FakeMm, KYLE, id, post}; +use gateway::{OTHER, SHARED, WAIT, config, loop_dir, read_state, start, up}; +use gatewayd::serve::Stop; +use gatewayd::sessions::BUSY; +use tmp::TempDir; + +#[test] +fn a_direct_message_is_answered_in_its_thread_while_typing() { + let home = TempDir::new("serve-dm"); + let (mm, running, turns, mut ws) = up(&home, Duration::from_millis(600)); + let p1 = id('p', 1); + ws.posted(&post(&p1, KYLE, DM, "", "hello there", 5), "D"); + let turn = turns.recv_timeout(WAIT).unwrap(); + assert_eq!( + (turn.session.as_str(), turn.content.as_str(), turn.resume), + (format!("mm-{p1}").as_str(), "hello there", false) + ); + let typing = ws.typing_within(Duration::from_millis(500)); + assert!(typing.len() >= 2, "{typing:?}"); + assert!( + typing.iter().all(|(c, p)| c == DM && *p == p1), + "{typing:?}" + ); + let posts = mm.wait_posts(1, WAIT); + assert_eq!( + posts, + [( + DM.to_string(), + p1.clone(), + "answer to hello there".to_string() + )] + ); + // Typing sent just before the answer may still be on its way; after that, it stops. + ws.typing_within(Duration::from_millis(300)); + assert!( + ws.typing_within(Duration::from_millis(400)).is_empty(), + "typing stops after the answer" + ); + let log = running.log(); + assert!( + log.iter() + .any(|l| l == &format!("gatewayd: connected to {} as {BOT_NAME}", mm.url())), + "{log:?}" + ); + assert!(matches!(running.finish(), Stop::Asked)); +} + +#[test] +fn anyone_else_gets_nothing_at_all() { + let home = TempDir::new("serve-stranger"); + let (mm, running, turns, mut ws) = up(&home, Duration::ZERO); + let p1 = id('p', 1); + ws.posted(&post(&p1, EVE, EVE_DM, "", "secret words", 5), "D"); + let mut own = post(&id('p', 2), BOT, DM, "", "my own post", 6); + own["user_id"] = serde_json::json!(BOT); + ws.posted(&own, "D"); + let log = running.wait_log("not allowed"); + assert!(turns.recv_timeout(Duration::from_millis(300)).is_err()); + assert!(ws.typing_within(Duration::from_millis(200)).is_empty()); + assert!(mm.posts().is_empty()); + assert!( + log.contains(&format!( + "gatewayd: ignored post {p1} from {EVE}: not allowed" + )), + "{log:?}" + ); + assert!(!log.iter().any(|l| l.contains("secret words")), "{log:?}"); +} + +#[test] +fn messages_during_a_turn_go_together_in_the_next() { + let home = TempDir::new("serve-burst"); + loop_dir(&home); + let (release_tx, release_rx) = mpsc::channel::<()>(); + let release = Mutex::new(release_rx); + let turns = serve_loop(&home.path().join("run/loop/loop.sock"), move |n, turn| { + if n == 0 { + let _ = release.lock().unwrap().recv_timeout(WAIT); + } + vec![done(&format!("answer to {}", turn.content))] + }); + let mm = FakeMm::start(); + let _running = start(config(&home, &mm.url(), "")); + let mut ws = mm.next_ws(WAIT); + let p1 = id('p', 1); + ws.posted(&post(&p1, KYLE, DM, "", "one", 5), "D"); + assert_eq!(turns.recv_timeout(WAIT).unwrap().content, "one"); + let saved = read_state(&home); + assert_eq!( + saved["in_flight"], + serde_json::json!([{"session": format!("mm-{p1}"), "channel": DM, "root": p1}]) + ); + ws.posted(&post(&id('p', 2), KYLE, DM, &p1, "two", 6), "D"); + ws.posted(&post(&id('p', 3), KYLE, DM, &p1, "three", 7), "D"); + std::thread::sleep(Duration::from_millis(200)); + release_tx.send(()).unwrap(); + let second = turns.recv_timeout(WAIT).unwrap(); + assert_eq!( + ( + second.session.as_str(), + second.content.as_str(), + second.resume + ), + (format!("mm-{p1}").as_str(), "two\n\nthree", true) + ); + let posts = mm.wait_posts(2, WAIT); + let texts: Vec<&str> = posts.iter().map(|(_, _, t)| t.as_str()).collect(); + assert_eq!(texts, ["answer to one", "answer to two\n\nthree"]); +} + +#[test] +fn a_full_queue_says_busy() { + let home = TempDir::new("serve-busy"); + loop_dir(&home); + let (release_tx, release_rx) = mpsc::channel::<()>(); + let release = Mutex::new(release_rx); + let _turns = serve_loop(&home.path().join("run/loop/loop.sock"), move |_, _| { + let _ = release.lock().unwrap().recv_timeout(WAIT); + vec![done("ok")] + }); + let mm = FakeMm::start(); + let _running = start(config(&home, &mm.url(), "queue = 1")); + let mut ws = mm.next_ws(WAIT); + let p1 = id('p', 1); + for (n, text) in ["run", "waits", "too many"].iter().enumerate() { + let n = u32::try_from(n).unwrap(); + let root = if n == 0 { String::new() } else { p1.clone() }; + ws.posted( + &post(&id('p', n + 1), KYLE, DM, &root, text, i64::from(n) + 5), + "D", + ); + } + let posts = mm.wait_posts(1, WAIT); + assert_eq!(posts, [(DM.to_string(), p1, BUSY.to_string())]); + release_tx.send(()).unwrap(); + release_tx.send(()).unwrap(); +} + +#[test] +fn channels_are_answered_only_when_named_or_in_our_thread() { + let home = TempDir::new("serve-channel"); + let (mm, _running, turns, mut ws) = up(&home, Duration::ZERO); + let p1 = id('p', 1); + ws.posted( + &post(&id('p', 9), KYLE, SHARED, "", "hello everyone", 4), + "O", + ); + ws.posted( + &post( + &id('p', 8), + KYLE, + OTHER, + "", + "@boxmaker-straylight elsewhere", + 4, + ), + "O", + ); + ws.posted( + &post(&p1, KYLE, SHARED, "", "@boxmaker-straylight start", 5), + "O", + ); + assert_eq!(turns.recv_timeout(WAIT).unwrap().content, "start"); + mm.wait_posts(1, WAIT); + ws.posted( + &post(&id('p', 2), KYLE, SHARED, &p1, "@hermes your turn", 6), + "O", + ); + ws.posted(&post(&id('p', 3), KYLE, SHARED, &p1, "and more", 7), "O"); + let next = turns.recv_timeout(WAIT).unwrap(); + assert_eq!((next.content.as_str(), next.resume), ("and more", true)); + let posts = mm.wait_posts(2, WAIT); + assert!( + posts.iter().all(|(c, r, _)| c == SHARED && *r == p1), + "{posts:?}" + ); + assert!(turns.recv_timeout(Duration::from_millis(200)).is_err()); + let saved = read_state(&home); + assert!(saved["channels"].get(OTHER).is_none(), "{saved}"); + assert_eq!(saved["threads"], serde_json::json!([p1])); +} + +#[test] +fn commands_are_answered_without_a_turn() { + let home = TempDir::new("serve-command"); + let (mm, _running, turns, mut ws) = up(&home, Duration::ZERO); + let p1 = id('p', 1); + ws.posted(&post(&p1, KYLE, DM, "", "!approve 3", 5), "D"); + assert_eq!( + mm.wait_posts(1, WAIT), + [( + DM.to_string(), + p1, + "approval 3 is not pending here".to_string() + )] + ); + assert!(turns.recv_timeout(Duration::from_millis(200)).is_err()); +} diff --git a/docs/plans/M4b/files/crates/gatewayd/tests/serve.rs-task08 b/docs/plans/M4b/files/crates/gatewayd/tests/serve.rs-task08 new file mode 100644 index 0000000..e5e1b1c --- /dev/null +++ b/docs/plans/M4b/files/crates/gatewayd/tests/serve.rs-task08 @@ -0,0 +1,210 @@ +//! `gatewayd` end to end, against a fake Mattermost and a fake `loopd`: who is answered, where, +//! and how (M4a spec, sections 7 and 8). Do not edit. + +#[path = "support/fake_loop.rs"] +mod fake_loop; +#[path = "support/fake_mm.rs"] +mod fake_mm; +#[path = "support/gateway.rs"] +mod gateway; +#[path = "support/tmp.rs"] +mod tmp; + +use std::sync::{Mutex, mpsc}; +use std::time::Duration; + +use fake_loop::{done, serve_loop}; +use fake_mm::{BOT, BOT_NAME, DM, EVE, EVE_DM, FakeMm, KYLE, id, post}; +use gateway::{OTHER, SHARED, WAIT, config, loop_dir, read_state, start, up}; +use gatewayd::serve::Stop; +use gatewayd::sessions::{BUSY, M4B_COMMAND}; +use tmp::TempDir; + +#[test] +fn a_direct_message_is_answered_in_its_thread_while_typing() { + let home = TempDir::new("serve-dm"); + let (mm, running, turns, mut ws) = up(&home, Duration::from_millis(600)); + let p1 = id('p', 1); + ws.posted(&post(&p1, KYLE, DM, "", "hello there", 5), "D"); + let turn = turns.recv_timeout(WAIT).unwrap(); + assert_eq!( + (turn.session.as_str(), turn.content.as_str(), turn.resume), + (format!("mm-{p1}").as_str(), "hello there", false) + ); + let typing = ws.typing_within(Duration::from_millis(500)); + assert!(typing.len() >= 2, "{typing:?}"); + assert!( + typing.iter().all(|(c, p)| c == DM && *p == p1), + "{typing:?}" + ); + let posts = mm.wait_posts(1, WAIT); + assert_eq!( + posts, + [( + DM.to_string(), + p1.clone(), + "answer to hello there".to_string() + )] + ); + // Typing sent just before the answer may still be on its way; after that, it stops. + ws.typing_within(Duration::from_millis(300)); + assert!( + ws.typing_within(Duration::from_millis(400)).is_empty(), + "typing stops after the answer" + ); + let log = running.log(); + assert!( + log.iter() + .any(|l| l == &format!("gatewayd: connected to {} as {BOT_NAME}", mm.url())), + "{log:?}" + ); + assert!(matches!(running.finish(), Stop::Asked)); +} + +#[test] +fn anyone_else_gets_nothing_at_all() { + let home = TempDir::new("serve-stranger"); + let (mm, running, turns, mut ws) = up(&home, Duration::ZERO); + let p1 = id('p', 1); + ws.posted(&post(&p1, EVE, EVE_DM, "", "secret words", 5), "D"); + let mut own = post(&id('p', 2), BOT, DM, "", "my own post", 6); + own["user_id"] = serde_json::json!(BOT); + ws.posted(&own, "D"); + let log = running.wait_log("not allowed"); + assert!(turns.recv_timeout(Duration::from_millis(300)).is_err()); + assert!(ws.typing_within(Duration::from_millis(200)).is_empty()); + assert!(mm.posts().is_empty()); + assert!( + log.contains(&format!( + "gatewayd: ignored post {p1} from {EVE}: not allowed" + )), + "{log:?}" + ); + assert!(!log.iter().any(|l| l.contains("secret words")), "{log:?}"); +} + +#[test] +fn messages_during_a_turn_go_together_in_the_next() { + let home = TempDir::new("serve-burst"); + loop_dir(&home); + let (release_tx, release_rx) = mpsc::channel::<()>(); + let release = Mutex::new(release_rx); + let turns = serve_loop(&home.path().join("run/loop/loop.sock"), move |n, turn| { + if n == 0 { + let _ = release.lock().unwrap().recv_timeout(WAIT); + } + vec![done(&format!("answer to {}", turn.content))] + }); + let mm = FakeMm::start(); + let _running = start(config(&home, &mm.url(), "")); + let mut ws = mm.next_ws(WAIT); + let p1 = id('p', 1); + ws.posted(&post(&p1, KYLE, DM, "", "one", 5), "D"); + assert_eq!(turns.recv_timeout(WAIT).unwrap().content, "one"); + let saved = read_state(&home); + assert_eq!( + saved["in_flight"], + serde_json::json!([{"session": format!("mm-{p1}"), "channel": DM, "root": p1}]) + ); + ws.posted(&post(&id('p', 2), KYLE, DM, &p1, "two", 6), "D"); + ws.posted(&post(&id('p', 3), KYLE, DM, &p1, "three", 7), "D"); + std::thread::sleep(Duration::from_millis(200)); + release_tx.send(()).unwrap(); + let second = turns.recv_timeout(WAIT).unwrap(); + assert_eq!( + ( + second.session.as_str(), + second.content.as_str(), + second.resume + ), + (format!("mm-{p1}").as_str(), "two\n\nthree", true) + ); + let posts = mm.wait_posts(2, WAIT); + let texts: Vec<&str> = posts.iter().map(|(_, _, t)| t.as_str()).collect(); + assert_eq!(texts, ["answer to one", "answer to two\n\nthree"]); +} + +#[test] +fn a_full_queue_says_busy() { + let home = TempDir::new("serve-busy"); + loop_dir(&home); + let (release_tx, release_rx) = mpsc::channel::<()>(); + let release = Mutex::new(release_rx); + let _turns = serve_loop(&home.path().join("run/loop/loop.sock"), move |_, _| { + let _ = release.lock().unwrap().recv_timeout(WAIT); + vec![done("ok")] + }); + let mm = FakeMm::start(); + let _running = start(config(&home, &mm.url(), "queue = 1")); + let mut ws = mm.next_ws(WAIT); + let p1 = id('p', 1); + for (n, text) in ["run", "waits", "too many"].iter().enumerate() { + let n = u32::try_from(n).unwrap(); + let root = if n == 0 { String::new() } else { p1.clone() }; + ws.posted( + &post(&id('p', n + 1), KYLE, DM, &root, text, i64::from(n) + 5), + "D", + ); + } + let posts = mm.wait_posts(1, WAIT); + assert_eq!(posts, [(DM.to_string(), p1, BUSY.to_string())]); + release_tx.send(()).unwrap(); + release_tx.send(()).unwrap(); +} + +#[test] +fn channels_are_answered_only_when_named_or_in_our_thread() { + let home = TempDir::new("serve-channel"); + let (mm, _running, turns, mut ws) = up(&home, Duration::ZERO); + let p1 = id('p', 1); + ws.posted( + &post(&id('p', 9), KYLE, SHARED, "", "hello everyone", 4), + "O", + ); + ws.posted( + &post( + &id('p', 8), + KYLE, + OTHER, + "", + "@boxmaker-straylight elsewhere", + 4, + ), + "O", + ); + ws.posted( + &post(&p1, KYLE, SHARED, "", "@boxmaker-straylight start", 5), + "O", + ); + assert_eq!(turns.recv_timeout(WAIT).unwrap().content, "start"); + mm.wait_posts(1, WAIT); + ws.posted( + &post(&id('p', 2), KYLE, SHARED, &p1, "@hermes your turn", 6), + "O", + ); + ws.posted(&post(&id('p', 3), KYLE, SHARED, &p1, "and more", 7), "O"); + let next = turns.recv_timeout(WAIT).unwrap(); + assert_eq!((next.content.as_str(), next.resume), ("and more", true)); + let posts = mm.wait_posts(2, WAIT); + assert!( + posts.iter().all(|(c, r, _)| c == SHARED && *r == p1), + "{posts:?}" + ); + assert!(turns.recv_timeout(Duration::from_millis(200)).is_err()); + let saved = read_state(&home); + assert!(saved["channels"].get(OTHER).is_none(), "{saved}"); + assert_eq!(saved["threads"], serde_json::json!([p1])); +} + +#[test] +fn commands_are_answered_without_a_turn() { + let home = TempDir::new("serve-command"); + let (mm, _running, turns, mut ws) = up(&home, Duration::ZERO); + let p1 = id('p', 1); + ws.posted(&post(&p1, KYLE, DM, "", "!approve 3", 5), "D"); + assert_eq!( + mm.wait_posts(1, WAIT), + [(DM.to_string(), p1, M4B_COMMAND.to_string())] + ); + assert!(turns.recv_timeout(Duration::from_millis(200)).is_err()); +} diff --git a/docs/plans/M4b/files/crates/gatewayd/tests/serve_answers.rs b/docs/plans/M4b/files/crates/gatewayd/tests/serve_answers.rs new file mode 100644 index 0000000..0a343e7 --- /dev/null +++ b/docs/plans/M4b/files/crates/gatewayd/tests/serve_answers.rs @@ -0,0 +1,248 @@ +//! Approvals answered from Mattermost end to end: `!approve` and `!deny` from an approver in the +//! approval's thread reach `brokerd`, the post shows what came of it, typing pauses while the turn +//! waits, and everything else is answered with its reason; the notice comes only when `brokerd` +//! never brought the approval (M4b spec, section 7). Do not edit. + +#[path = "support/brokerd_client.rs"] +mod brokerd_client; +#[path = "support/fake_loop.rs"] +mod fake_loop; +#[path = "support/fake_mm.rs"] +mod fake_mm; +#[path = "support/gateway.rs"] +mod gateway; +#[path = "support/tmp.rs"] +mod tmp; + +use std::path::PathBuf; +use std::sync::{Mutex, mpsc}; +use std::time::{Duration, Instant}; + +use brokerd_client::{Asking, ask, pending}; +use fake_loop::{done, event, serve_loop}; +use fake_mm::{DM, EVE, FakeMm, KYLE, WsPeer, id, post}; +use gateway::{Running, WAIT, loop_dir, start}; +use gatewayd::approvals::NOT_APPROVER; +use gatewayd::config::Config; +use proto::{GatewayOutcome, Timestamp, Turn, TurnEvent}; +use tmp::TempDir; + +/// KYLE and EVE may talk; only KYLE approves. +fn config(home: &TempDir, url: &str) -> Config { + let text = format!( + "[mattermost]\nurl = \"{url}\"\n[secrets.mattermost_token]\nenv = \"NOT_READ_BY_RUN\"\n[allow]\nusers = [\"{KYLE}\", \"{EVE}\"]\n[approvals]\nusers = [\"{KYLE}\"]\n[paths]\nhome = \"{}\"\n[limits]\ntyping_every_ms = 100\n", + home.path().display() + ); + Config::parse(&text).unwrap() +} + +/// A loop whose first turn waits for `go`, then sends `before` and answers "done". +fn held(home: &TempDir, before: Vec) -> (mpsc::Receiver, mpsc::Sender<()>) { + loop_dir(home); + let (go_tx, go_rx) = mpsc::channel::<()>(); + let go = Mutex::new(go_rx); + let before = Mutex::new(Some(before)); + let turns = serve_loop(&home.path().join("run/loop/loop.sock"), move |_, _| { + let _ = go.lock().unwrap().recv_timeout(Duration::from_secs(20)); + let mut replies: Vec<_> = before + .lock() + .unwrap() + .take() + .unwrap_or_default() + .into_iter() + .map(event) + .collect(); + replies.push(done("done")); + replies + }); + (turns, go_tx) +} + +struct Stage { + _home: TempDir, + mm: FakeMm, + _running: Running, + ws: WsPeer, + go: mpsc::Sender<()>, + socket: PathBuf, + root: String, +} + +/// gatewayd up, a direct message from KYLE whose turn is held, and the approval's socket. +fn stage(tag: &str, before: Vec) -> Stage { + let home = TempDir::new(tag); + let (turns, go) = held(&home, before); + let mm = FakeMm::start(); + let running = start(config(&home, &mm.url())); + let mut ws = mm.next_ws(WAIT); + running.wait_log("gatewayd: connected to "); + let root = id('p', 1); + ws.posted(&post(&root, KYLE, DM, "", "read my notes", 5), "D"); + turns.recv_timeout(WAIT).expect("the turn started"); + let socket = home.path().join("run/gateway/gateway.sock"); + Stage { + _home: home, + mm, + _running: running, + ws, + go, + socket, + root, + } +} + +impl Stage { + fn ask(&self, approval: u64) -> Asking { + ask( + &self.socket, + pending(approval, &format!("mm-{}", self.root), r#"{"path":"/n/a"}"#), + ) + .unwrap() + } + + fn command(&mut self, n: u32, user: &str, root: &str, text: &str, at: i64) { + self.ws + .posted(&post(&id('c', n), user, DM, root, text, at), "D"); + } + + /// The text posted in the thread after the approval, within five seconds. + fn replies(&self, after: usize) -> Vec { + let posts = self.mm.wait_posts(after + 1, WAIT); + posts[after..].iter().map(|(_, _, t)| t.clone()).collect() + } +} + +fn wait_until(what: &str, mut done: impl FnMut() -> bool) { + let until = Instant::now() + WAIT; + while !done() { + assert!(Instant::now() < until, "never happened: {what}"); + std::thread::sleep(Duration::from_millis(20)); + } +} + +#[test] +fn approved_by_the_approver_the_post_says_it_runs_and_typing_pauses_meanwhile() { + let mut s = stage("ans-approve", vec![]); + s.ws.typing_within(Duration::from_millis(300)); + let mut asking = s.ask(42); + s.ws.typing_within(Duration::from_millis(200)); + assert!( + s.ws.typing_within(Duration::from_millis(400)).is_empty(), + "no typing while it waits for approval" + ); + let root = s.root.clone(); + s.command(1, KYLE, &root, "!approve 42", 5); + let reply = asking.answer(WAIT).expect("the answer reaches brokerd"); + assert_eq!( + ( + reply.approval, + reply.approved, + reply.by.as_str(), + reply.post.clone(), + reply.reason.clone() + ), + (42, true, KYLE, id('c', 1), None) + ); + asking.outcome(GatewayOutcome::Runs {}); + let edits = s.mm.wait_edits(1, WAIT); + assert_eq!(edits[0].0, FakeMm::post_id(0)); + assert!( + edits[0] + .1 + .ends_with("\n\n✅ Approved by @kyle: the call runs."), + "{}", + edits[0].1 + ); + assert_eq!(s.mm.posts().len(), 1, "nothing else posted in the thread"); + wait_until("typing again", || { + !s.ws.typing_within(Duration::from_millis(300)).is_empty() + }); + s.go.send(()).unwrap(); +} + +#[test] +fn refused_with_a_reason_the_post_says_so() { + let mut s = stage("ans-deny", vec![]); + let mut asking = s.ask(42); + let root = s.root.clone(); + s.command(1, KYLE, &root, "@boxmaker-straylight !deny 42 not now", 5); + let reply = asking.answer(WAIT).expect("the answer reaches brokerd"); + assert_eq!( + (reply.approved, reply.reason.as_deref()), + (false, Some("not now")) + ); + asking.outcome(GatewayOutcome::Refused {}); + let edits = s.mm.wait_edits(1, WAIT); + assert!( + edits[0].1.ends_with("\n\n❌ Refused by @kyle: `not now`."), + "{}", + edits[0].1 + ); + s.go.send(()).unwrap(); +} + +#[test] +fn nobody_else_and_nothing_else_answers_it() { + let mut s = stage("ans-refused", vec![]); + let mut asking = s.ask(42); + let root = s.root.clone(); + let elsewhere = id('p', 9); + s.command(1, EVE, &root, "!approve 42", 5); + s.command(2, KYLE, &elsewhere, "!approve 42", 6); + s.command(3, KYLE, &root, "!approve 42", 1); + s.command(4, KYLE, &root, "!approve 7", 7); + s.command(5, KYLE, &root, "!approve 42 please", 8); + assert_eq!( + s.replies(1)[..5], + [ + NOT_APPROVER.to_string(), + "approval 42 is answered in its own thread".to_string(), + "approval 42 was answered before it was shown in full".to_string(), + "approval 7 is not pending here".to_string(), + "use !approve or !deny ".to_string(), + ] + ); + assert_eq!( + asking.answer(Duration::from_millis(300)), + None, + "nothing reached brokerd" + ); + s.command(6, KYLE, &root, "!approve 42", 9); + assert!( + asking.answer(WAIT).is_some(), + "the approver, in the thread, after the post" + ); + asking.outcome(GatewayOutcome::Runs {}); + s.go.send(()).unwrap(); +} + +#[test] +fn the_notice_only_when_brokerd_never_brought_it() { + let expires = Timestamp::parse("2026-09-24T08:20:00.000Z").unwrap(); + let pending_event = TurnEvent::ApprovalPending { + approval: 42, + tool: "read_file".to_string(), + expires, + }; + let s = stage("ans-notice", vec![pending_event.clone()]); + s.go.send(()).unwrap(); + let posts = s.mm.wait_posts(2, WAIT); + let texts: Vec<&str> = posts.iter().map(|(_, _, t)| t.as_str()).collect(); + assert_eq!( + texts, + [ + "waiting for approval 42: brokerd has not sent it here; answer it with bxctl approvals", + "done" + ] + ); + + let s = stage("ans-no-notice", vec![pending_event]); + let asking = s.ask(42); + s.go.send(()).unwrap(); + let posts = s.mm.wait_posts(2, WAIT); + assert!(posts[0].2.starts_with("**Approval 42**"), "{posts:?}"); + assert_eq!(posts[1].2, "done"); + std::thread::sleep(Duration::from_millis(500)); + assert_eq!(s.mm.posts().len(), 2, "no notice: {:?}", s.mm.posts()); + asking.close(); +} diff --git a/docs/plans/M4b/files/crates/gatewayd/tests/serve_approvals.rs b/docs/plans/M4b/files/crates/gatewayd/tests/serve_approvals.rs new file mode 100644 index 0000000..277d9a5 --- /dev/null +++ b/docs/plans/M4b/files/crates/gatewayd/tests/serve_approvals.rs @@ -0,0 +1,199 @@ +//! The serve loop posts what `gateway.sock` brings, where it belongs, and marks it when it ends +//! or when a restart left it behind (M4b spec, sections 6 and 8). Do not edit. + +#[path = "support/brokerd_client.rs"] +mod brokerd_client; +#[path = "support/fake_loop.rs"] +mod fake_loop; +#[path = "support/fake_mm.rs"] +mod fake_mm; +#[path = "support/gateway.rs"] +mod gateway; +#[path = "support/tmp.rs"] +mod tmp; + +use std::path::PathBuf; +use std::sync::{Mutex, mpsc}; +use std::time::Duration; + +use brokerd_client::{ask, pending}; +use fake_loop::{done, serve_loop}; +use fake_mm::{DM, FakeMm, KYLE, id, post}; +use gateway::{WAIT, config, loop_dir, read_state, start}; +use tmp::TempDir; + +fn socket(home: &TempDir) -> PathBuf { + home.path().join("run/gateway/gateway.sock") +} + +/// A turn that waits until the returned sender says so. +fn held_loop(home: &TempDir) -> mpsc::Sender<()> { + loop_dir(home); + let (release_tx, release_rx) = mpsc::channel::<()>(); + let release = Mutex::new(release_rx); + let _turns = serve_loop(&home.path().join("run/loop/loop.sock"), move |_, _| { + let _ = release + .lock() + .unwrap() + .recv_timeout(Duration::from_secs(20)); + vec![done("done")] + }); + release_tx +} + +#[test] +fn an_approval_goes_into_the_asking_thread_and_is_marked_when_it_ends() { + let home = TempDir::new("sa-thread"); + let release = held_loop(&home); + let mm = FakeMm::start(); + let _running = start(config(&home, &mm.url(), "")); + let mut ws = mm.next_ws(WAIT); + let p1 = id('p', 1); + ws.posted(&post(&p1, KYLE, DM, "", "read my notes", 5), "D"); + let until = std::time::Instant::now() + WAIT; + let path = home.path().join("gateway/state.json"); + let in_flight = || -> bool { + let Ok(text) = std::fs::read_to_string(&path) else { + return false; + }; + let v: serde_json::Value = serde_json::from_str(&text).unwrap(); + !v["in_flight"].as_array().unwrap().is_empty() + }; + while !in_flight() { + assert!(std::time::Instant::now() < until, "the turn never started"); + std::thread::sleep(Duration::from_millis(20)); + } + let asking = ask( + &socket(&home), + pending(42, &format!("mm-{p1}"), r#"{"path":"/n/a"}"#), + ) + .unwrap(); + let posts = mm.posts(); + assert_eq!(posts.len(), 1, "{posts:?}"); + let (channel, root, text) = &posts[0]; + assert_eq!( + (channel.as_str(), root.as_str()), + (DM, p1.as_str()), + "in the asking thread" + ); + assert!(text.starts_with("**Approval 42** · tool `read_file` · grant `notes` · taint private · expires 2026-09-24 08:20 UTC\n```\n{\"path\":\"/n/a\"}\n```\n"), "{text}"); + assert_eq!(asking.posts, [FakeMm::post_id(0)]); + assert_eq!( + read_state(&home)["approvals"], + serde_json::json!([{"approval": 42, "channel": DM, "posts": [FakeMm::post_id(0)]}]) + ); + asking.close(); + let edits = mm.wait_edits(1, WAIT); + assert_eq!(edits.len(), 1, "{edits:?}"); + assert_eq!(edits[0].0, FakeMm::post_id(0)); + assert_eq!( + edits[0].1, + format!("{text}\n\nNo longer pending (answered elsewhere, expired, or its turn ended).") + ); + let until = std::time::Instant::now() + WAIT; + while !read_state(&home)["approvals"] + .as_array() + .unwrap() + .is_empty() + { + assert!(std::time::Instant::now() < until, "still recorded"); + std::thread::sleep(Duration::from_millis(20)); + } + release.send(()).unwrap(); +} + +#[test] +fn without_a_turn_in_flight_it_goes_to_the_first_approvers_direct_messages_in_parts() { + let home = TempDir::new("sa-dm"); + loop_dir(&home); + let mm = FakeMm::start(); + let _running = start(config(&home, &mm.url(), "")); + let _ws = mm.next_ws(WAIT); + let long = format!("{{\"content\":\"{}\"}}", "x".repeat(20_000)); + let asking = ask(&socket(&home), pending(7, "chat-1789700000-1", &long)).unwrap(); + let posts = mm.posts(); + assert_eq!(posts.len(), 2, "two parts"); + assert_eq!( + (posts[0].0.as_str(), posts[0].1.as_str()), + (DM, ""), + "a new thread in the direct channel" + ); + assert_eq!( + (posts[1].0.as_str(), posts[1].1.clone()), + (DM, FakeMm::post_id(0)), + "the next part in its thread" + ); + assert!( + posts[1] + .2 + .contains("Reply in this thread with `!approve 7`") + ); + assert_eq!(asking.posts, [FakeMm::post_id(0), FakeMm::post_id(1)]); + asking.close(); + let edits = mm.wait_edits(1, WAIT); + assert_eq!( + edits[0].0, + FakeMm::post_id(1), + "only the last part is edited" + ); +} + +#[test] +fn an_approval_that_cannot_be_posted_is_refused_to_brokerd() { + let home = TempDir::new("sa-refused"); + loop_dir(&home); + let mm = FakeMm::start(); + let _running = start(config(&home, &mm.url(), "")); + let _ws = mm.next_ws(WAIT); + mm.refuse_posts(true); + let why = ask(&socket(&home), pending(8, "chat-1789700000-1", "{}")) + .err() + .expect("refused"); + assert!( + why.starts_with("cannot post approval 8: status 400"), + "{why}" + ); + assert!( + read_state(&home)["approvals"] + .as_array() + .unwrap() + .is_empty() + ); +} + +#[test] +fn a_restart_marks_the_posts_it_left_behind() { + let home = TempDir::new("sa-restart"); + let (a, b) = (id('q', 1), id('q', 2)); + let state = serde_json::json!({ + "channels": {}, "recent": [], "threads": [], "in_flight": [], + "approvals": [{"approval": 5, "channel": DM, "posts": [a, b]}] + }); + home.write("gateway/state.json", &state.to_string()); + loop_dir(&home); + let mm = FakeMm::start(); + let _running = start(config(&home, &mm.url(), "")); + let _ws = mm.next_ws(WAIT); + let edits = mm.wait_edits(1, WAIT); + assert_eq!(edits, [(b, "**Approval 5**\n\nNo longer answerable here: gatewayd restarted. If it is still pending it is posted again.".to_string())]); + assert!( + read_state(&home)["approvals"] + .as_array() + .unwrap() + .is_empty() + ); +} + +#[test] +fn the_socket_is_there_while_gatewayd_runs() { + let home = TempDir::new("sa-socket"); + loop_dir(&home); + let mm = FakeMm::start(); + let running = start(config(&home, &mm.url(), "")); + let _ws = mm.next_ws(WAIT); + use std::os::unix::fs::{FileTypeExt, PermissionsExt}; + let meta = std::fs::metadata(socket(&home)).unwrap(); + assert!(meta.file_type().is_socket()); + assert_eq!(meta.permissions().mode() & 0o777, 0o600); + drop(running); +} diff --git a/docs/plans/M4b/files/crates/gatewayd/tests/sessions.rs b/docs/plans/M4b/files/crates/gatewayd/tests/sessions.rs new file mode 100644 index 0000000..6313c06 --- /dev/null +++ b/docs/plans/M4b/files/crates/gatewayd/tests/sessions.rs @@ -0,0 +1,314 @@ +//! Which posts become turns, in which session, and how messages wait for a running turn (M4a spec, +//! section 7, including its table of examples). Do not edit. + +use gatewayd::mm::Post; +use gatewayd::sessions::{ + ApprovalCommand, BUSY, Batch, Ignored, Message, Pushed, Queues, Route, Router, Thread, + UNKNOWN_COMMAND, named, +}; +use proto::SessionId; + +const BOT: &str = "b0000000000000000000000000"; +const KYLE: &str = "k0000000000000000000000000"; +const EVE: &str = "e0000000000000000000000000"; +const DM: &str = "d0000000000000000000000000"; +const SHARED: &str = "c0000000000000000000000000"; +const OTHER: &str = "o0000000000000000000000000"; +const ROOT: &str = "r0000000000000000000000000"; +const POST: &str = "p0000000000000000000000000"; + +fn router() -> Router { + Router::new( + BOT, + "boxmaker-straylight", + &[KYLE.to_string()], + &[SHARED.to_string()], + ) +} + +fn post(channel: &str, root: &str, message: &str) -> Post { + Post { + id: POST.to_string(), + user_id: KYLE.to_string(), + channel_id: channel.to_string(), + root_id: root.to_string(), + message: message.to_string(), + create_at: 5, + delete_at: 0, + kind: String::new(), + } +} + +fn known(root: &str) -> bool { + root == ROOT +} + +fn unknown(_: &str) -> bool { + false +} + +fn queued(route: Route) -> Message { + match route { + Route::Queue(m) => m, + other => panic!("not queued: {other:?}"), + } +} + +#[test] +fn the_examples_in_the_spec() { + let r = router(); + let cases = [ + ("", "@boxmaker-straylight summarise the audit log", true), + (ROOT, "and the older files?", true), + (ROOT, "@hermes what do you think?", false), + ("", "@boxmaker-straylight @hermes compare notes", true), + ("", "@boxmaker-straylightx hello", false), + ("", "@channel standup in five", false), + ]; + for (root, message, yes) in cases { + let got = r.route(&post(SHARED, root, message), "O", &known); + assert_eq!(matches!(got, Route::Queue(_)), yes, "{message}: {got:?}"); + if !yes { + assert_eq!(got, Route::Ignore(Ignored::NotForUs), "{message}"); + } + } +} + +#[test] +fn naming() { + assert_eq!(named("@Boxmaker-Straylight, look"), ["boxmaker-straylight"]); + assert_eq!(named("ask @boxmaker-straylight."), ["boxmaker-straylight"]); + assert_eq!(named("@a.b_c-d... and @e"), ["a.b_c-d", "e"]); + assert_eq!(named("@ alone, @@x, trailing @"), ["x"]); + assert_eq!(named("ünïcødé @ʙob @bob"), ["bob"]); + assert!(named("no names here").is_empty()); + let r = router(); + for message in [ + "hi @BOXMAKER-STRAYLIGHT", + "@boxmaker-straylight.", + "(@boxmaker-straylight)", + ] { + assert!( + matches!( + r.route(&post(SHARED, "", message), "P", &unknown), + Route::Queue(_) + ), + "{message}" + ); + } +} + +#[test] +fn replies_in_our_thread_that_name_everyone_are_still_ours() { + let r = router(); + for message in ["@here any news?", "@all done", "thanks @channel"] { + assert!( + matches!( + r.route(&post(SHARED, ROOT, message), "O", &known), + Route::Queue(_) + ), + "{message}" + ); + } + let got = r.route( + &post( + SHARED, + "q0000000000000000000000000", + "a reply in someone else's thread", + ), + "O", + &known, + ); + assert_eq!(got, Route::Ignore(Ignored::NotForUs)); +} + +#[test] +fn who_and_where() { + let r = router(); + let mut own = post(DM, "", "hello"); + own.user_id = BOT.to_string(); + assert_eq!(r.route(&own, "D", &unknown), Route::Ignore(Ignored::Own)); + let mut system = post(DM, "", "joined"); + system.kind = "system_join_channel".to_string(); + assert_eq!( + r.route(&system, "D", &unknown), + Route::Ignore(Ignored::System) + ); + let mut stranger = post(DM, "", "@boxmaker-straylight hello"); + stranger.user_id = EVE.to_string(); + assert_eq!( + r.route(&stranger, "D", &unknown), + Route::Ignore(Ignored::NotAllowed) + ); + let mut stranger_cmd = post(DM, "", "!approve 1"); + stranger_cmd.user_id = EVE.to_string(); + assert_eq!( + r.route(&stranger_cmd, "D", &unknown), + Route::Ignore(Ignored::NotAllowed) + ); + let naming = "@boxmaker-straylight hello"; + assert_eq!( + r.route(&post(OTHER, "", naming), "O", &unknown), + Route::Ignore(Ignored::NotForUs) + ); + assert_eq!( + r.route(&post(SHARED, "", naming), "X", &unknown), + Route::Ignore(Ignored::NotForUs) + ); + assert!(matches!( + r.route(&post(SHARED, "", naming), "G", &unknown), + Route::Queue(_) + )); + assert!(matches!( + r.route(&post(DM, "", "no name needed"), "D", &unknown), + Route::Queue(_) + )); +} + +#[test] +fn sessions_and_threads() { + let r = router(); + let top = queued(r.route(&post(DM, "", "hello"), "D", &unknown)); + assert_eq!(top.session.as_str(), format!("mm-{POST}")); + assert_eq!( + top.thread, + Thread { + channel: DM.to_string(), + root: POST.to_string() + } + ); + assert!(!top.resume && !top.joins_thread); + assert_eq!(top.text, "hello"); + let reply = queued(r.route(&post(DM, ROOT, "more"), "D", &unknown)); + assert_eq!(reply.session.as_str(), format!("mm-{ROOT}")); + assert_eq!(reply.thread.root, ROOT); + assert!(reply.resume); + let channel = queued(r.route(&post(SHARED, "", "@boxmaker-straylight hi"), "O", &unknown)); + assert!(channel.joins_thread && !channel.resume); + assert_eq!(channel.text, "hi", "the leading mention is stripped (M4b)"); +} + +#[test] +fn commands() { + let r = router(); + let thread = Thread { + channel: DM.to_string(), + root: ROOT.to_string(), + }; + for (message, command) in [ + ("!approve 42", ApprovalCommand::Approve(42)), + ( + "!deny 42 not now", + ApprovalCommand::Deny(42, Some("not now".to_string())), + ), + ("!deny", ApprovalCommand::Malformed), + ( + "@boxmaker-straylight !approve 7", + ApprovalCommand::Approve(7), + ), + ] { + assert_eq!( + r.route(&post(DM, ROOT, message), "D", &unknown), + Route::Approval { + thread: thread.clone(), + command, + user: KYLE.to_string(), + post: POST.to_string(), + at: 5, + }, + "{message}" + ); + } + for message in ["!approved", "!help", "!"] { + assert_eq!( + r.route(&post(DM, ROOT, message), "D", &unknown), + Route::Reply { + thread: thread.clone(), + text: UNKNOWN_COMMAND.to_string(), + }, + "{message}" + ); + } + assert_eq!( + queued(r.route(&post(DM, ROOT, "!!approve is a word"), "D", &unknown)).text, + "!approve is a word" + ); + assert_eq!( + queued(r.route(&post(DM, ROOT, "!!"), "D", &unknown)).text, + "!" + ); + assert_eq!( + queued(r.route(&post(DM, ROOT, " !help"), "D", &unknown)).text, + " !help" + ); +} + +fn message(root: &str, resume: bool, text: &str) -> Message { + Message { + session: SessionId::new(&format!("mm-{root}")).unwrap(), + thread: Thread { + channel: DM.to_string(), + root: root.to_string(), + }, + resume, + text: text.to_string(), + joins_thread: false, + } +} + +#[test] +fn messages_wait_for_the_running_turn_and_go_together() { + let mut q = Queues::new(3); + let first = message(ROOT, false, "one"); + let Pushed::Start(batch) = q.push(first.clone()) else { + panic!("not started") + }; + assert_eq!( + batch, + Batch { + session: first.session.clone(), + thread: first.thread.clone(), + resume: false, + text: "one".to_string() + } + ); + assert_eq!(q.push(message(ROOT, true, "two")), Pushed::Waiting); + assert_eq!(q.push(message(ROOT, true, "three\nlines")), Pushed::Waiting); + let other = message(POST, false, "elsewhere"); + assert!( + matches!(q.push(other.clone()), Pushed::Start(_)), + "another session starts at once" + ); + assert_eq!(q.running(), 2); + let mut roots: Vec = q.threads().into_iter().map(|t| t.root).collect(); + roots.sort(); + assert_eq!(roots, [POST, ROOT]); + let next = q.finish(&first.session).unwrap(); + assert_eq!( + (next.text.as_str(), next.resume), + ("two\n\nthree\nlines", true) + ); + assert_eq!(q.finish(&first.session), None); + assert_eq!(q.finish(&other.session), None); + assert_eq!(q.running(), 0); + assert!( + matches!(q.push(message(ROOT, true, "later")), Pushed::Start(_)), + "idle again" + ); +} + +#[test] +fn a_full_queue_drops_the_message() { + let mut q = Queues::new(2); + let m = message(ROOT, false, "run"); + assert!(matches!(q.push(m.clone()), Pushed::Start(_))); + assert_eq!(q.push(message(ROOT, true, "a")), Pushed::Waiting); + assert_eq!(q.push(message(ROOT, true, "b")), Pushed::Waiting); + assert_eq!( + q.push(message(ROOT, true, "c")), + Pushed::Full(m.thread.clone()) + ); + assert_eq!(q.finish(&m.session).unwrap().text, "a\n\nb"); + assert!(!BUSY.is_empty()); + assert_eq!(q.finish(&SessionId::new("mm-never").unwrap()), None); +} diff --git a/docs/plans/M4b/files/crates/gatewayd/tests/sessions_mention.rs b/docs/plans/M4b/files/crates/gatewayd/tests/sessions_mention.rs new file mode 100644 index 0000000..2a3de4a --- /dev/null +++ b/docs/plans/M4b/files/crates/gatewayd/tests/sessions_mention.rs @@ -0,0 +1,142 @@ +//! A leading mention of this bot is stripped before anything reads a message, and approval +//! commands are recognised exactly (M4b spec, section 7). Do not edit. + +use gatewayd::mm::Post; +use gatewayd::sessions::{ + ApprovalCommand, MAX_REASON, Route, Router, approval_command, strip_mention, +}; + +const ME: &str = "boxmaker0"; +const BOT: &str = "b0000000000000000000000000"; +const KYLE: &str = "k0000000000000000000000000"; +const SHARED: &str = "c0000000000000000000000000"; +const ROOT: &str = "r0000000000000000000000000"; + +#[test] +fn a_leading_mention_is_stripped() { + assert_eq!( + strip_mention("@boxmaker0 summarise the log", ME), + "summarise the log" + ); + assert_eq!(strip_mention("@BoxMaker0 !approve 42", ME), "!approve 42"); + assert_eq!( + strip_mention("@boxmaker0 @hermes compare notes", ME), + "@hermes compare notes" + ); + assert_eq!(strip_mention("@boxmaker0, look", ME), ", look"); + assert_eq!( + strip_mention("@boxmaker0\nsecond line", ME), + "\nsecond line", + "only spaces go with it" + ); +} + +#[test] +fn anything_else_is_kept() { + for message in [ + "@boxmaker0", + "@boxmaker0 ", + "@boxmaker0x hello", + "@hermes @boxmaker0 hello", + "hello @boxmaker0", + " @boxmaker0 hello", + "@ boxmaker0 hello", + "@@boxmaker0 hello", + "", + ] { + assert_eq!(strip_mention(message, ME), message, "{message:?}"); + } + assert_eq!( + strip_mention("@boxmaker0. hi", ME), + ". hi", + "the trailing dot is not part of the name" + ); +} + +#[test] +fn approval_commands_exactly() { + use ApprovalCommand::{Approve, Deny, Malformed}; + let cases: &[(&str, Option)] = &[ + ("!approve 42", Some(Approve(42))), + ("!approve 0", Some(Approve(0))), + ("!approve 18446744073709551615", Some(Approve(u64::MAX))), + ("!deny 42", Some(Deny(42, None))), + ( + "!deny 42 not on a Friday", + Some(Deny(42, Some("not on a Friday".to_string()))), + ), + ( + "!deny 42 two spaces", + Some(Deny(42, Some(" two spaces".to_string()))), + ), + ("!approve", Some(Malformed)), + ("!approve 42", Some(Malformed)), + ("!approve 42 ", Some(Malformed)), + ("!approve 42 now", Some(Malformed)), + ("!approve +42", Some(Malformed)), + ("!approve 4x2", Some(Malformed)), + ("!approve 18446744073709551616", Some(Malformed)), + ("!deny", Some(Malformed)), + ("!deny 42 ", Some(Malformed)), + ("!deny x reason", Some(Malformed)), + ("!approved 42", None), + ("!denied 42", None), + ("!help", None), + ("approve 42", None), + ("!Approve 42", None), + ]; + for (text, want) in cases { + assert_eq!(&approval_command(text), want, "{text:?}"); + } + let long = format!("!deny 42 {}", "é".repeat(MAX_REASON)); + assert!( + matches!(approval_command(&long), Some(Deny(42, Some(_)))), + "characters, not bytes" + ); + let over = format!("!deny 42 {}", "x".repeat(MAX_REASON + 1)); + assert_eq!(approval_command(&over), Some(Malformed)); +} + +fn post(message: &str, root: &str) -> Post { + Post { + id: "p0000000000000000000000001".to_string(), + user_id: KYLE.to_string(), + channel_id: SHARED.to_string(), + root_id: root.to_string(), + message: message.to_string(), + create_at: 9, + delete_at: 0, + kind: String::new(), + } +} + +#[test] +fn in_a_shared_thread_the_mention_picks_the_bot() { + let me = Router::new(BOT, ME, &[KYLE.to_string()], &[SHARED.to_string()]); + let ours = |root: &str| root == ROOT; + let addressed = me.route(&post("@boxmaker0 !approve 42", ROOT), "O", &ours); + assert!( + matches!( + addressed, + Route::Approval { + command: ApprovalCommand::Approve(42), + .. + } + ), + "{addressed:?}" + ); + let other = me.route(&post("@boxmaker1 !approve 42", ROOT), "O", &ours); + assert!( + matches!(other, Route::Ignore(_)), + "addressed to another bot: {other:?}" + ); + let bare = me.route(&post("!approve 42", ROOT), "O", &ours); + assert!( + matches!(bare, Route::Approval { .. }), + "a bare command in our thread is ours too: {bare:?}" + ); + match me.route(&post("@boxmaker0 hello there", ""), "O", &ours) { + Route::Queue(m) => assert_eq!(m.text, "hello there"), + other => panic!("{other:?}"), + } +} diff --git a/docs/plans/M4b/files/crates/gatewayd/tests/state_approvals.rs b/docs/plans/M4b/files/crates/gatewayd/tests/state_approvals.rs new file mode 100644 index 0000000..ee32a94 --- /dev/null +++ b/docs/plans/M4b/files/crates/gatewayd/tests/state_approvals.rs @@ -0,0 +1,95 @@ +//! The approval posts in the state file, for marking them at a restart; an M4a state file still +//! loads; and the turn of a session (M4b spec, sections 6 and 8). Do not edit. + +#[path = "support/tmp.rs"] +mod tmp; + +use gatewayd::state::{ApprovalPosts, InFlight, State, StateError}; +use tmp::TempDir; + +const DM: &str = "d0000000000000000000000000"; +const ROOT: &str = "r0000000000000000000000000"; + +fn id(n: u32) -> String { + format!("p{n:025}") +} + +fn shown(approval: u64, posts: &[u32]) -> ApprovalPosts { + ApprovalPosts { + approval, + channel: DM.to_string(), + posts: posts.iter().map(|n| id(*n)).collect(), + } +} + +#[test] +fn approval_posts_survive_a_restart_and_are_taken_once() { + let dir = TempDir::new("state-approvals"); + let path = dir.path().join("gateway/state.json"); + let mut s = State::load(&path).unwrap(); + s.add_approval(shown(42, &[1, 2])).unwrap(); + s.add_approval(shown(43, &[3])).unwrap(); + s.add_approval(shown(42, &[4])).unwrap(); + s.remove_approval(43).unwrap(); + s.remove_approval(99).unwrap(); + let mut again = State::load(&path).unwrap(); + assert_eq!( + again.take_approvals().unwrap(), + [shown(42, &[4])], + "the later posting of 42 replaced the first" + ); + assert!( + State::load(&path) + .unwrap() + .take_approvals() + .unwrap() + .is_empty(), + "taking is saved" + ); +} + +#[test] +fn an_m4a_state_file_loads() { + let dir = TempDir::new("state-m4a"); + let text = format!(r#"{{"channels":{{"{DM}":5}},"recent":[],"threads":[],"in_flight":[]}}"#); + let path = dir.write("state.json", &text); + let mut s = State::load(&path).unwrap(); + assert_eq!(s.since(DM), Some(5)); + assert!(s.take_approvals().unwrap().is_empty()); +} + +#[test] +fn bad_approval_posts_are_refused() { + let dir = TempDir::new("state-approvals-bad"); + for (n, approvals) in [ + r#"[{"approval":1,"channel":"../x","posts":["p0000000000000000000000001"]}]"#, + r#"[{"approval":1,"channel":"d0000000000000000000000000","posts":[]}]"#, + r#"[{"approval":1,"channel":"d0000000000000000000000000","posts":["short"]}]"#, + r#"[{"approval":1,"channel":"d0000000000000000000000000","posts":["p0000000000000000000000001"],"extra":1}]"#, + r#"[{"approval":-1,"channel":"d0000000000000000000000000","posts":["p0000000000000000000000001"]}]"#, + ] + .iter() + .enumerate() + { + let text = format!(r#"{{"channels":{{}},"recent":[],"threads":[],"in_flight":[],"approvals":{approvals}}}"#); + let path = dir.write(&format!("s{n}.json"), &text); + assert!(matches!(State::load(&path), Err(StateError::Read(..))), "{approvals}"); + } +} + +#[test] +fn the_turn_of_a_session() { + let dir = TempDir::new("state-turn-of"); + let mut s = State::load(&dir.path().join("state.json")).unwrap(); + let turn = InFlight { + session: format!("mm-{ROOT}"), + channel: DM.to_string(), + root: ROOT.to_string(), + }; + assert_eq!(s.turn_of(&turn.session), None); + s.start_turn(turn.clone()).unwrap(); + assert_eq!(s.turn_of(&turn.session), Some(turn.clone())); + assert_eq!(s.turn_of("mm-other"), None); + s.end_turn(&turn.session).unwrap(); + assert_eq!(s.turn_of(&turn.session), None); +} diff --git a/docs/plans/M4b/files/crates/gatewayd/tests/support/brokerd_client.rs b/docs/plans/M4b/files/crates/gatewayd/tests/support/brokerd_client.rs new file mode 100644 index 0000000..0d33661 --- /dev/null +++ b/docs/plans/M4b/files/crates/gatewayd/tests/support/brokerd_client.rs @@ -0,0 +1,97 @@ +//! `brokerd`'s side of one `gateway.sock` connection, for tests: ask, read what comes back, +//! answer with an outcome, or close. Do not edit. + +#![allow(dead_code)] // each test file uses a different part of this module + +use std::os::unix::net::UnixStream; +use std::path::Path; +use std::time::Duration; + +use proto::{ + ApprovalOutcome, ApprovalReply, ApprovalRequest, CallId, DataClass, Envelope, GatewayOutcome, + Message, PROTOCOL_VERSION, PendingApproval, SessionId, Timestamp, read_frame, write_frame, +}; + +pub struct Asking { + stream: UnixStream, + pub posts: Vec, +} + +/// An approval of `session`, with these arguments. +pub fn pending(approval: u64, session: &str, arguments: &str) -> PendingApproval { + PendingApproval { + approval, + session: SessionId::new(session).unwrap(), + call: CallId(3), + tool: "read_file".to_string(), + arguments: arguments.to_string(), + grant: "notes".to_string(), + taint: DataClass::Private, + created: Timestamp::parse("2026-09-24T08:05:00.000Z").unwrap(), + expires: Timestamp::parse("2026-09-24T08:20:00.000Z").unwrap(), + } +} + +fn frame(id: u64, last: bool, msg: Message) -> Envelope { + Envelope { + v: PROTOCOL_VERSION, + id, + r#final: last, + msg, + } +} + +/// Connect and ask; `Ok` with the posts, or `Err` with gatewayd's error detail. +pub fn ask(socket: &Path, info: PendingApproval) -> Result { + let mut stream = UnixStream::connect(socket).map_err(|e| e.to_string())?; + stream + .set_read_timeout(Some(Duration::from_secs(10))) + .unwrap(); + write_frame( + &mut stream, + &frame( + 1, + true, + Message::ApprovalRequest(ApprovalRequest { approval: info }), + ), + ) + .unwrap(); + match read_frame(&mut stream).map_err(|e| e.to_string())?.msg { + Message::ApprovalPosted(p) => Ok(Asking { + stream, + posts: p.posts, + }), + Message::Error(e) => Err(e.detail), + other => panic!("{other:?}"), + } +} + +impl Asking { + /// The answer, within `wait`, or `None`. + pub fn answer(&mut self, wait: Duration) -> Option { + self.stream.set_read_timeout(Some(wait)).unwrap(); + match read_frame(&mut self.stream) { + Ok(Envelope { + msg: Message::ApprovalReply(r), + .. + }) => Some(r), + _ => None, + } + } + + pub fn outcome(&mut self, outcome: GatewayOutcome) { + write_frame( + &mut self.stream, + &frame( + 2, + true, + Message::ApprovalOutcome(ApprovalOutcome { outcome }), + ), + ) + .unwrap(); + } + + pub fn close(self) { + let _ = self.stream.shutdown(std::net::Shutdown::Both); + } +} diff --git a/docs/plans/M4b/files/crates/gatewayd/tests/support/fake_mm.rs b/docs/plans/M4b/files/crates/gatewayd/tests/support/fake_mm.rs new file mode 100644 index 0000000..879ea90 --- /dev/null +++ b/docs/plans/M4b/files/crates/gatewayd/tests/support/fake_mm.rs @@ -0,0 +1,381 @@ +//! A fake Mattermost on 127.0.0.1, plain TCP: the four REST calls `gatewayd` makes, and the +//! WebSocket, whose events the test sends and whose requests it reads. Do not edit. + +#![allow(dead_code)] // each test file uses a different part of this module + +use std::collections::HashMap; +use std::io::{Read, Write}; +use std::net::{Shutdown, SocketAddr, TcpListener, TcpStream}; +use std::sync::{Arc, Mutex, mpsc}; +use std::time::{Duration, Instant}; + +use gatewayd::ws::handshake::accept_for; +use serde_json::{Value, json}; + +pub const BOT: &str = "b0000000000000000000000000"; +pub const BOT_NAME: &str = "boxmaker-straylight"; +pub const KYLE: &str = "k0000000000000000000000000"; +pub const EVE: &str = "e0000000000000000000000000"; +/// The direct channel between the bot and Kyle, and between the bot and anyone else. +pub const DM: &str = "d0000000000000000000000000"; +pub const EVE_DM: &str = "f0000000000000000000000000"; + +#[derive(Default)] +struct Inner { + /// Status for `users/me`: 200 unless a test sets another. + me_status: u16, + /// The body for `channels//posts?since=`, by channel. + since: HashMap, + /// Every post made: channel, root, message. + posts: Vec<(String, String, String)>, + /// Every REST call: method and path. + calls: Vec<(String, String)>, + /// Every edit: the post and its new text. + edits: Vec<(String, String)>, + /// When set, `POST /posts` is refused with 400. + refuse_posts: bool, +} + +pub struct FakeMm { + pub addr: SocketAddr, + inner: Arc>, + sockets: Mutex>, +} + +/// One WebSocket connection from `gatewayd`. +pub struct WsPeer { + writer: TcpStream, + /// The text of every text frame `gatewayd` sends. + pub texts: mpsc::Receiver, +} + +/// A post as Mattermost sends it. +pub fn post( + id: &str, + user: &str, + channel: &str, + root: &str, + message: &str, + create_at: i64, +) -> Value { + json!({ + "id": id, "create_at": create_at, "update_at": create_at, "delete_at": 0, "user_id": user, + "channel_id": channel, "root_id": root, "message": message, "type": "", "props": {} + }) +} + +pub fn id(prefix: char, n: u32) -> String { + format!("{prefix}{n:025}") +} + +fn frame(opcode: u8, payload: &[u8]) -> Vec { + let mut out = vec![0x80 | opcode]; + match payload.len() { + n if n < 126 => out.push(n as u8), + n => { + out.push(126); + out.extend_from_slice(&(n as u16).to_be_bytes()); + } + } + out.extend_from_slice(payload); + out +} + +impl WsPeer { + pub fn event(&mut self, value: &Value) { + let _ = self + .writer + .write_all(&frame(0x1, value.to_string().as_bytes())); + } + + pub fn posted(&mut self, post: &Value, channel_type: &str) { + let data = json!({"post": post.to_string(), "channel_type": channel_type, "team_id": ""}); + self.event(&json!({"event": "posted", "data": data, "broadcast": {}, "seq": 1})); + } + + /// End the connection without a close frame. + pub fn drop_connection(self) { + let _ = self.writer.shutdown(Shutdown::Both); + } + + /// The `user_typing` requests received within `wait`, as (channel, parent). + pub fn typing_within(&self, wait: Duration) -> Vec<(String, String)> { + let until = Instant::now() + wait; + let mut got = Vec::new(); + while let Ok(text) = self + .texts + .recv_timeout(until.saturating_duration_since(Instant::now())) + { + let v: Value = serde_json::from_str(&text).unwrap(); + if v["action"] == "user_typing" { + let data = &v["data"]; + got.push(( + data["channel_id"].as_str().unwrap().to_string(), + data["parent_id"].as_str().unwrap().to_string(), + )); + } + } + got + } +} + +fn read_head(stream: &mut TcpStream) -> Option { + let mut head = Vec::new(); + let mut byte = [0u8; 1]; + while !head.ends_with(b"\r\n\r\n") { + if stream.read(&mut byte).ok()? == 0 { + return None; + } + head.push(byte[0]); + } + String::from_utf8(head).ok() +} + +/// Unmask the client's frames and send each text on `tx`, until the connection ends. +fn read_frames(mut stream: TcpStream, tx: mpsc::Sender) { + let mut exact = |n: usize| -> Option> { + let mut buf = vec![0u8; n]; + stream.read_exact(&mut buf).ok().map(|()| buf) + }; + loop { + let Some(head) = exact(2) else { return }; + let len = match head[1] & 0x7F { + 126 => u16::from_be_bytes(exact(2).unwrap().try_into().unwrap()) as usize, + 127 => return, + n => n as usize, + }; + let Some(mask) = exact(4) else { return }; + let Some(raw) = exact(len) else { return }; + let payload: Vec = raw + .iter() + .zip(mask.iter().cycle()) + .map(|(b, m)| b ^ m) + .collect(); + if head[0] & 0x0F == 0x1 { + let _ = tx.send(String::from_utf8(payload).unwrap()); + } + } +} + +impl FakeMm { + pub fn start() -> FakeMm { + let listener = TcpListener::bind("127.0.0.1:0").unwrap(); + let addr = listener.local_addr().unwrap(); + let inner = Arc::new(Mutex::new(Inner { + me_status: 200, + ..Inner::default() + })); + let (ws_tx, ws_rx) = mpsc::channel(); + let shared = Arc::clone(&inner); + std::thread::spawn(move || { + for stream in listener.incoming() { + let Ok(stream) = stream else { continue }; + let (inner, ws_tx) = (Arc::clone(&shared), ws_tx.clone()); + std::thread::spawn(move || connection(stream, &inner, &ws_tx)); + } + }); + FakeMm { + addr, + inner, + sockets: Mutex::new(ws_rx), + } + } + + pub fn url(&self) -> String { + format!("http://127.0.0.1:{}", self.addr.port()) + } + + pub fn refuse_token(&self) { + self.inner.lock().unwrap().me_status = 401; + } + + pub fn set_since(&self, channel: &str, posts: &[Value]) { + let order: Vec = posts.iter().map(|p| p["id"].clone()).collect(); + let map: serde_json::Map = posts + .iter() + .map(|p| (p["id"].as_str().unwrap().to_string(), p.clone())) + .collect(); + self.inner + .lock() + .unwrap() + .since + .insert(channel.to_string(), json!({"order": order, "posts": map})); + } + + /// The next WebSocket `gatewayd` opens, after its hello. + pub fn next_ws(&self, wait: Duration) -> WsPeer { + self.sockets + .lock() + .unwrap() + .recv_timeout(wait) + .expect("no WebSocket connection") + } + + pub fn posts(&self) -> Vec<(String, String, String)> { + self.inner.lock().unwrap().posts.clone() + } + + /// Wait until at least `n` posts were made, for at most `wait`. + pub fn wait_posts(&self, n: usize, wait: Duration) -> Vec<(String, String, String)> { + let until = Instant::now() + wait; + while self.posts().len() < n && Instant::now() < until { + std::thread::sleep(Duration::from_millis(10)); + } + self.posts() + } + + pub fn calls(&self) -> Vec<(String, String)> { + self.inner.lock().unwrap().calls.clone() + } + + /// The id of the n-th post made (from 0). + pub fn post_id(n: usize) -> String { + id('x', u32::try_from(n).unwrap()) + } + + pub fn edits(&self) -> Vec<(String, String)> { + self.inner.lock().unwrap().edits.clone() + } + + /// Wait until at least `n` edits were made, for at most `wait`. + pub fn wait_edits(&self, n: usize, wait: Duration) -> Vec<(String, String)> { + let until = Instant::now() + wait; + while self.edits().len() < n && Instant::now() < until { + std::thread::sleep(Duration::from_millis(10)); + } + self.edits() + } + + pub fn refuse_posts(&self, refuse: bool) { + self.inner.lock().unwrap().refuse_posts = refuse; + } +} + +fn connection(mut stream: TcpStream, inner: &Mutex, ws_tx: &mpsc::Sender) { + let Some(head) = read_head(&mut stream) else { + return; + }; + let mut words = head.split_whitespace(); + let (method, path) = ( + words.next().unwrap_or("").to_string(), + words.next().unwrap_or("").to_string(), + ); + if path == "/api/v4/websocket" { + let key = head + .lines() + .find_map(|l| l.strip_prefix("Sec-WebSocket-Key: ")) + .unwrap_or("") + .trim() + .to_string(); + let reply = format!( + "HTTP/1.1 101 Switching Protocols\r\nUpgrade: websocket\r\nConnection: Upgrade\r\nSec-WebSocket-Accept: {}\r\n\r\n", + accept_for(&key) + ); + let _ = stream.write_all(reply.as_bytes()); + let _ = stream.write_all(&frame( + 0x1, + br#"{"event":"hello","data":{},"broadcast":{},"seq":0}"#, + )); + let (tx, texts) = mpsc::channel(); + let reader = stream.try_clone().unwrap(); + std::thread::spawn(move || read_frames(reader, tx)); + let _ = ws_tx.send(WsPeer { + writer: stream, + texts, + }); + return; + } + let length = head + .lines() + .find_map(|l| { + l.to_ascii_lowercase() + .strip_prefix("content-length:") + .map(|v| v.trim().parse::().unwrap_or(0)) + }) + .unwrap_or(0); + let mut body = vec![0u8; length]; + let _ = stream.read_exact(&mut body); + let (status, answer) = rest(inner, &method, &path, &body); + let text = answer.to_string(); + let reply = format!( + "HTTP/1.1 {status} X\r\nContent-Type: application/json\r\nContent-Length: {}\r\n\r\n{text}", + text.len() + ); + let _ = stream.write_all(reply.as_bytes()); +} + +fn rest(inner: &Mutex, method: &str, path: &str, body: &[u8]) -> (u16, Value) { + let mut inner = inner.lock().unwrap(); + inner.calls.push((method.to_string(), path.to_string())); + match (method, path) { + ("GET", "/api/v4/users/me") if inner.me_status == 200 => { + (200, json!({"id": BOT, "username": BOT_NAME})) + } + ("GET", "/api/v4/users/me") => ( + inner.me_status, + json!({"id": "api.context.session_expired.app_error"}), + ), + ("POST", "/api/v4/channels/direct") => { + let users: Vec = serde_json::from_slice(body).unwrap(); + let channel = if users.iter().any(|u| u == KYLE) { + DM + } else { + EVE_DM + }; + (201, json!({"id": channel, "type": "D"})) + } + ("POST", "/api/v4/posts") if inner.refuse_posts => { + (400, json!({"id": "api.post.create_post.refused"})) + } + ("POST", "/api/v4/posts") => { + let p: Value = serde_json::from_slice(body).unwrap(); + let n = u32::try_from(inner.posts.len()).unwrap(); + let (channel, root, message) = ( + p["channel_id"].as_str().unwrap(), + p["root_id"].as_str().unwrap(), + p["message"].as_str().unwrap(), + ); + inner + .posts + .push((channel.to_string(), root.to_string(), message.to_string())); + (201, post(&id('x', n), BOT, channel, root, message, 1)) + } + ("GET", p) if p.contains("/posts?since=") => { + let channel = p + .trim_start_matches("/api/v4/channels/") + .split('/') + .next() + .unwrap_or(""); + ( + 200, + inner + .since + .get(channel) + .cloned() + .unwrap_or(json!({"order": [], "posts": {}})), + ) + } + ("PUT", p) if p.starts_with("/api/v4/posts/") && p.ends_with("/patch") => { + let post_id = p + .trim_start_matches("/api/v4/posts/") + .trim_end_matches("/patch") + .to_string(); + let message = serde_json::from_slice::(body).unwrap()["message"] + .as_str() + .unwrap() + .to_string(); + inner.edits.push((post_id.clone(), message.clone())); + (200, post(&post_id, BOT, DM, "", &message, 1)) + } + ("GET", p) if p.starts_with("/api/v4/users/") => { + let user = p.trim_start_matches("/api/v4/users/"); + let name = if user == KYLE { + "kyle".to_string() + } else { + format!("user-{}", &user[..4]) + }; + (200, json!({"id": user, "username": name})) + } + _ => (404, json!({"message": "not found"})), + } +} diff --git a/docs/plans/M4b/files/crates/gatewayd/tests/support/gateway.rs b/docs/plans/M4b/files/crates/gatewayd/tests/support/gateway.rs new file mode 100644 index 0000000..0e2c94e --- /dev/null +++ b/docs/plans/M4b/files/crates/gatewayd/tests/support/gateway.rs @@ -0,0 +1,142 @@ +//! Running `gatewayd`'s serve loop in a test, against the fake Mattermost and the fake `loopd`. +//! Do not edit. + +#![allow(dead_code)] // each test file uses a different part of this module + +use std::sync::atomic::{AtomicBool, Ordering}; +use std::sync::{Arc, Mutex, mpsc}; +use std::thread::JoinHandle; +use std::time::Duration; + +use gatewayd::config::Config; +use gatewayd::secrets::Secret; +use gatewayd::serve::{Stop, Tuning, run}; +use proto::Turn; + +use crate::fake_loop::{done, serve_loop}; +use crate::fake_mm::{FakeMm, KYLE, WsPeer}; +use crate::tmp::TempDir; + +pub const SHARED: &str = "c0000000000000000000000000"; +pub const OTHER: &str = "o0000000000000000000000000"; +pub const WAIT: Duration = Duration::from_secs(5); + +pub struct Running { + pub stop: Arc, + pub log: Arc>>, + pub handle: Option>, +} + +impl Running { + pub fn log(&self) -> Vec { + self.log.lock().unwrap().clone() + } + + pub fn wait_log(&self, part: &str) -> Vec { + let until = std::time::Instant::now() + WAIT; + while !self.log().iter().any(|l| l.contains(part)) { + assert!( + std::time::Instant::now() < until, + "no log line with {part:?}: {:?}", + self.log() + ); + std::thread::sleep(Duration::from_millis(10)); + } + self.log() + } + + /// The `Stop` `run` returns by itself within 5 s; after that it is stopped, and the test fails. + pub fn join_within(mut self) -> Stop { + let handle = self.handle.take().unwrap(); + let until = std::time::Instant::now() + WAIT; + while !handle.is_finished() && std::time::Instant::now() < until { + std::thread::sleep(Duration::from_millis(10)); + } + self.stop.store(true, Ordering::SeqCst); + let stop = handle.join().unwrap(); + assert!(!matches!(stop, Stop::Asked), "run did not stop by itself"); + stop + } + + pub fn finish(mut self) -> Stop { + self.stop.store(true, Ordering::SeqCst); + self.handle.take().unwrap().join().unwrap() + } +} + +impl Drop for Running { + fn drop(&mut self) { + self.stop.store(true, Ordering::SeqCst); + } +} + +pub fn config(home: &TempDir, url: &str, extra: &str) -> Config { + let text = format!( + r#" +[mattermost] +url = "{url}" +[secrets.mattermost_token] +env = "NOT_READ_BY_RUN" +[allow] +users = ["{KYLE}"] +channels = ["{SHARED}"] +[paths] +home = "{}" +[limits] +typing_every_ms = 100 +{extra} +"#, + home.path().display() + ); + Config::parse(&text).unwrap() +} + +pub fn start(config: Config) -> Running { + let stop = Arc::new(AtomicBool::new(false)); + let log = Arc::new(Mutex::new(Vec::new())); + let tuning = Tuning { + backoff: vec![Duration::from_millis(50)], + poll: Duration::from_millis(20), + rest_timeout: WAIT, + notice: Duration::from_millis(300), + }; + let (s, l) = (Arc::clone(&stop), Arc::clone(&log)); + let handle = std::thread::spawn(move || { + let sink: gatewayd::serve::Log = + Arc::new(move |line: &str| l.lock().unwrap().push(line.to_string())); + run(config, Secret::new("TOKEN".to_string()), tuning, sink, &s) + }); + Running { + stop, + log, + handle: Some(handle), + } +} + +/// A fake loop that answers every turn with "answer to ", after `delay`. +pub fn answering(home: &TempDir, delay: Duration) -> mpsc::Receiver { + serve_loop(&home.path().join("run/loop/loop.sock"), move |_, turn| { + std::thread::sleep(delay); + vec![done(&format!("answer to {}", turn.content))] + }) +} + +pub fn read_state(home: &TempDir) -> serde_json::Value { + let text = std::fs::read_to_string(home.path().join("gateway/state.json")).unwrap(); + serde_json::from_str(&text).unwrap() +} + +pub fn loop_dir(home: &TempDir) { + std::fs::create_dir_all(home.path().join("run/loop")).unwrap(); +} + +/// Start with a fake Mattermost and a fake loop; the first WebSocket is returned. +pub fn up(home: &TempDir, delay: Duration) -> (FakeMm, Running, mpsc::Receiver, WsPeer) { + loop_dir(home); + let turns = answering(home, delay); + let mm = FakeMm::start(); + let running = start(config(home, &mm.url(), "")); + let ws = mm.next_ws(WAIT); + running.wait_log("gatewayd: connected to "); + (mm, running, turns, ws) +} diff --git a/docs/plans/M4b/files/crates/proto/src/escape.rs b/docs/plans/M4b/files/crates/proto/src/escape.rs new file mode 100644 index 0000000..6ce3987 --- /dev/null +++ b/docs/plans/M4b/files/crates/proto/src/escape.rs @@ -0,0 +1,45 @@ +//! Text from outside (model output, tool arguments, server errors) made safe to show: control +//! characters, zero-width and bidirectional marks, line separators and the BOM become `\uXXXX`. +//! Used by `bxctl` for the terminal and by `gatewayd` for Mattermost, so both show the same. + +// The control bytes, the zero-width and Unicode line/paragraph separators, and the BOM. A code +// point in any of these ranges must never reach the terminal raw. +const ESCAPE_RANGES: &[(u32, u32)] = &[ + (0x0000, 0x001f), + (0x007f, 0x009f), + (0x200b, 0x200f), + (0x2028, 0x202e), + (0x2060, 0x2069), + (0xfeff, 0xfeff), +]; + +pub fn escape_json_text(s: &str) -> String { + escape(s, false) +} + +pub fn escape_model_text(s: &str) -> String { + escape(s, true) +} + +fn escape(s: &str, model_text: bool) -> String { + let mut out = String::with_capacity(s.len()); + for c in s.chars() { + let cp = u32::from(c); + if should_escape(c, model_text) { + out.push_str(&format!("\\u{:04x}", cp)); + } else { + out.push(c); + } + } + out +} + +// A control point is escaped, except that model text keeps \n and \t. +fn should_escape(c: char, model_text: bool) -> bool { + let cp = u32::from(c); + if ESCAPE_RANGES.iter().any(|(lo, hi)| cp >= *lo && cp <= *hi) { + !(model_text && (cp == 0x09 || cp == 0x0a)) + } else { + false + } +} diff --git a/docs/plans/M4b/files/crates/proto/src/gateway.rs b/docs/plans/M4b/files/crates/proto/src/gateway.rs new file mode 100644 index 0000000..bdab965 --- /dev/null +++ b/docs/plans/M4b/files/crates/proto/src/gateway.rs @@ -0,0 +1,76 @@ +//! The messages on `gateway.sock`, between `brokerd` and `gatewayd`: one connection carries one +//! pending approval to Mattermost and its answer back (M4b spec, section 4). + +use serde::{Deserialize, Serialize}; + +use crate::wire::DenyReason; +use crate::wire::PendingApproval; + +/// The most characters a refusal's reason may have. +pub const MAX_REASON: usize = 500; + +/// `brokerd` → `gatewayd`, first: the approval to post. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct ApprovalRequest { + pub approval: PendingApproval, +} + +/// `gatewayd` → `brokerd`: it is posted, in these posts, first part first. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct ApprovalPosted { + pub posts: Vec, +} + +/// `gatewayd` → `brokerd`: an approver answered, with this post. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct ApprovalReply { + pub approval: u64, + pub approved: bool, + pub by: String, + pub post: String, + pub reason: Option, +} + +/// `brokerd` → `gatewayd`, last: what came of the answer. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct ApprovalOutcome { + pub outcome: GatewayOutcome, +} + +// JSON: {"outcome":"denied","reason":"no_grant"} ; the tag sits beside the fields +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(tag = "outcome", rename_all = "snake_case", deny_unknown_fields)] +pub enum GatewayOutcome { + /// Approved, and the grants as they are now still allow it. + Runs {}, + /// Approved, but denied on the re-check, or the answer could not be recorded. + Denied { reason: DenyReason }, + /// The refusal is recorded. + Refused {}, + /// It was no longer pending when the answer came. + Gone {}, +} + +/// A Mattermost id: 26 characters of `a-z0-9`. +pub fn is_mattermost_id(id: &str) -> bool { + // `id.len() == 26` and every byte `b.is_ascii_lowercase() || b.is_ascii_digit()`. + todo!() +} + +impl ApprovalReply { + /// `by` and `post` are Mattermost ids and the reason is at most `MAX_REASON` characters; the + /// first problem otherwise. + pub fn check(&self) -> Result<(), String> { + // In this order, each an Err with exactly this text: + // 1. `!is_mattermost_id(&self.by)`: "the answer's `by` is not a Mattermost id". + // 2. `!is_mattermost_id(&self.post)`: "the answer's `post` is not a Mattermost id". + // 3. a reason with more than `MAX_REASON` characters (`r.chars().count()`, not bytes): + // `format!("the reason is over {MAX_REASON} characters")`. + // 4. Ok(()). + todo!() + } +} diff --git a/docs/plans/M4b/files/crates/proto/tests/escape.rs b/docs/plans/M4b/files/crates/proto/tests/escape.rs new file mode 100644 index 0000000..74bbb30 --- /dev/null +++ b/docs/plans/M4b/files/crates/proto/tests/escape.rs @@ -0,0 +1,150 @@ +//! Tests for `proto::escape`: text the model wrote is printed as data. Do not edit. +//! +//! The expected escapes are built by `esc`, never spelled out, so that nothing that handles this +//! file can turn one into the character it stands for. + +use proto::escape::{escape_json_text, escape_model_text}; + +const BACKSLASH: char = '\\'; + +/// The escape for one code point: a backslash, `u`, and four lowercase hex digits. +fn esc(code: u32) -> String { + format!("{BACKSLASH}u{code:04x}") +} + +fn ch(code: u32) -> char { + char::from_u32(code).unwrap() +} + +/// Every code point that must be escaped, as inclusive ranges. +const HIDDEN: [(u32, u32); 6] = [ + (0x0000, 0x001f), + (0x007f, 0x009f), + (0x200b, 0x200f), + (0x2028, 0x202e), + (0x2060, 0x2069), + (0xfeff, 0xfeff), +]; + +fn hidden(code: u32) -> bool { + HIDDEN.iter().any(|(lo, hi)| (*lo..=*hi).contains(&code)) +} + +/// Walks every code point below U+11000, not a sample: each is either escaped exactly or left +/// exactly as it is. +#[test] +fn every_listed_code_point_is_escaped_and_no_other() { + let mut escaped = 0; + for code in 0..0x11000u32 { + let Some(c) = char::from_u32(code) else { + continue; // the surrogates are not characters + }; + let text = format!("a{c}b"); + let got = escape_json_text(&text); + if hidden(code) { + escaped += 1; + assert_eq!(got, format!("a{}b", esc(code)), "U+{code:04X}"); + } else { + assert_eq!(got, text, "U+{code:04X} must pass through"); + } + } + assert_eq!(escaped, 32 + 33 + 5 + 7 + 10 + 1, "the six ranges, in full"); +} + +#[test] +fn the_edges_of_each_range() { + for (lo, hi) in HIDDEN { + assert_eq!(escape_json_text(&ch(lo).to_string()), esc(lo)); + assert_eq!(escape_json_text(&ch(hi).to_string()), esc(hi)); + if lo > 0 { + let before = ch(lo - 1).to_string(); + assert_eq!(escape_json_text(&before), before, "U+{:04X}", lo - 1); + } + let after = ch(hi + 1).to_string(); + assert_eq!(escape_json_text(&after), after, "U+{:04X}", hi + 1); + } +} + +#[test] +fn hex_digits_are_lowercase_and_there_are_always_four() { + assert_eq!(escape_json_text("\x1b"), esc(0x1b)); + assert!(escape_json_text("\x1b").ends_with("001b")); + assert!(escape_json_text("\0").ends_with("0000")); + assert!(escape_json_text(&ch(0xfeff).to_string()).ends_with("feff")); + assert!(escape_json_text(&ch(0x202e).to_string()).ends_with("202e")); +} + +#[test] +fn an_escape_sequence_cannot_reach_the_terminal() { + let text = "before\x1b[8mhidden\x1b[0m\x07after"; + let got = escape_json_text(text); + assert!(!got.contains('\x1b') && !got.contains('\x07'), "{got:?}"); + assert_eq!( + got, + format!( + "before{}[8mhidden{}[0m{}after", + esc(0x1b), + esc(0x1b), + esc(0x07) + ) + ); +} + +#[test] +fn a_path_cannot_be_shown_backwards() { + // U+202E makes a terminal draw what follows from right to left. + let text = format!("/home/kyle/notes/{}dm.terces", ch(0x202e)); + assert_eq!( + escape_json_text(&text), + format!("/home/kyle/notes/{}dm.terces", esc(0x202e)) + ); + let text = format!("a{}b{}c", ch(0x200b), ch(0x2066)); + assert_eq!( + escape_json_text(&text), + format!("a{}b{}c", esc(0x200b), esc(0x2066)) + ); +} + +#[test] +fn ordinary_text_is_unchanged() { + for text in [ + "", + "plain", + r#"{"command":"ls -l","cwd":"/home/kyle"}"#, + "naïve café 日本語 🙂", + "a backslash \\ and a quote \" stay as they are", + ] { + assert_eq!(escape_json_text(text), text); + assert_eq!(escape_model_text(text), text); + } +} + +#[test] +fn json_text_escapes_newline_and_tab_but_model_text_keeps_them() { + let text = "one\n\ttwo\r\n"; + assert_eq!( + escape_json_text(text), + format!("one{}{}two{}{}", esc(0x0a), esc(0x09), esc(0x0d), esc(0x0a)) + ); + assert_eq!( + escape_model_text(text), + format!("one\n\ttwo{}\n", esc(0x0d)), + "only newline and tab pass; a carriage return could overwrite the line" + ); +} + +#[test] +fn model_text_escapes_everything_else_the_same_way() { + for (lo, hi) in HIDDEN { + for code in lo..=hi { + if code == 0x0a || code == 0x09 { + continue; + } + assert_eq!( + escape_model_text(&ch(code).to_string()), + esc(code), + "U+{code:04X}" + ); + } + } +} diff --git a/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_outcome_denied.json b/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_outcome_denied.json new file mode 100644 index 0000000..c6fac58 --- /dev/null +++ b/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_outcome_denied.json @@ -0,0 +1 @@ +{"v":1,"id":2,"final":true,"msg":{"kind":"approval_outcome","body":{"outcome":{"outcome":"denied","reason":"no_grant"}}}} diff --git a/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_outcome_gone.json b/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_outcome_gone.json new file mode 100644 index 0000000..3274e15 --- /dev/null +++ b/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_outcome_gone.json @@ -0,0 +1 @@ +{"v":1,"id":2,"final":true,"msg":{"kind":"approval_outcome","body":{"outcome":{"outcome":"gone"}}}} diff --git a/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_outcome_refused.json b/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_outcome_refused.json new file mode 100644 index 0000000..52cab33 --- /dev/null +++ b/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_outcome_refused.json @@ -0,0 +1 @@ +{"v":1,"id":2,"final":true,"msg":{"kind":"approval_outcome","body":{"outcome":{"outcome":"refused"}}}} diff --git a/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_outcome_runs.json b/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_outcome_runs.json new file mode 100644 index 0000000..fc0147a --- /dev/null +++ b/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_outcome_runs.json @@ -0,0 +1 @@ +{"v":1,"id":2,"final":true,"msg":{"kind":"approval_outcome","body":{"outcome":{"outcome":"runs"}}}} diff --git a/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_posted.json b/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_posted.json new file mode 100644 index 0000000..88e66b0 --- /dev/null +++ b/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_posted.json @@ -0,0 +1 @@ +{"v":1,"id":1,"final":false,"msg":{"kind":"approval_posted","body":{"posts":["p0000000000000000000000001","p0000000000000000000000002"]}}} diff --git a/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_reply_approved.json b/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_reply_approved.json new file mode 100644 index 0000000..ff42b64 --- /dev/null +++ b/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_reply_approved.json @@ -0,0 +1 @@ +{"v":1,"id":1,"final":true,"msg":{"kind":"approval_reply","body":{"approval":41,"approved":true,"by":"k0000000000000000000000000","post":"c0000000000000000000000001","reason":null}}} diff --git a/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_reply_refused.json b/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_reply_refused.json new file mode 100644 index 0000000..df3f197 --- /dev/null +++ b/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_reply_refused.json @@ -0,0 +1 @@ +{"v":1,"id":1,"final":true,"msg":{"kind":"approval_reply","body":{"approval":41,"approved":false,"by":"k0000000000000000000000000","post":"c0000000000000000000000001","reason":"not on a Friday"}}} diff --git a/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_request.json b/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_request.json new file mode 100644 index 0000000..e7d69e0 --- /dev/null +++ b/docs/plans/M4b/files/crates/proto/tests/fixtures/wire/gateway_request.json @@ -0,0 +1 @@ +{"v":1,"id":1,"final":true,"msg":{"kind":"approval_request","body":{"approval":{"approval":41,"session":"mm-r0000000000000000000000001","call":3,"tool":"shell","arguments":"{\"command\":\"ls -la /srv\"}","grant":"shell-srv","taint":"private","created":"2026-09-24T08:05:00.000Z","expires":"2026-09-24T08:20:00.000Z"}}}} diff --git a/docs/plans/M4b/files/crates/proto/tests/gateway_wire.rs b/docs/plans/M4b/files/crates/proto/tests/gateway_wire.rs new file mode 100644 index 0000000..ce5886e --- /dev/null +++ b/docs/plans/M4b/files/crates/proto/tests/gateway_wire.rs @@ -0,0 +1,187 @@ +//! The four messages of `gateway.sock` against byte-exact fixtures, every object in them refusing +//! an unknown key, and the checks on an answer (M4b spec, section 4). Do not edit these or the +//! fixtures. + +use proto::{ + ApprovalOutcome, ApprovalPosted, ApprovalReply, ApprovalRequest, CallId, DataClass, DenyReason, + Envelope, GatewayOutcome, MAX_REASON, Message, PendingApproval, SessionId, Timestamp, + is_mattermost_id, +}; +use serde_json::Value; + +const BY: &str = "k0000000000000000000000000"; +const POST: &str = "c0000000000000000000000001"; + +fn fixture(name: &str) -> String { + let path = format!("{}/tests/fixtures/wire/{name}", env!("CARGO_MANIFEST_DIR")); + let text = std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("{path}: {e}")); + text.trim_end_matches('\n').to_string() +} + +/// The fixture must decode to `want`, and `want` must encode to exactly the fixture's bytes. +fn check(name: &str, id: u64, last: bool, msg: Message) { + let want = Envelope { + v: 1, + id, + r#final: last, + msg, + }; + let text = fixture(name); + let got: Envelope = serde_json::from_str(&text).unwrap_or_else(|e| panic!("{name}: {e}")); + assert_eq!(got, want, "{name}: decoded value"); + assert_eq!( + serde_json::to_string(&want).unwrap(), + text, + "{name}: encoded bytes" + ); +} + +fn pending() -> PendingApproval { + PendingApproval { + approval: 41, + session: SessionId::new("mm-r0000000000000000000000001").unwrap(), + call: CallId(3), + tool: "shell".to_string(), + arguments: r#"{"command":"ls -la /srv"}"#.to_string(), + grant: "shell-srv".to_string(), + taint: DataClass::Private, + created: Timestamp::parse("2026-09-24T08:05:00.000Z").unwrap(), + expires: Timestamp::parse("2026-09-24T08:20:00.000Z").unwrap(), + } +} + +fn reply(approved: bool, reason: Option<&str>) -> ApprovalReply { + ApprovalReply { + approval: 41, + approved, + by: BY.to_string(), + post: POST.to_string(), + reason: reason.map(str::to_string), + } +} + +#[test] +fn the_four_messages() { + check( + "gateway_request.json", + 1, + true, + Message::ApprovalRequest(ApprovalRequest { + approval: pending(), + }), + ); + let posts = vec![ + "p0000000000000000000000001".to_string(), + "p0000000000000000000000002".to_string(), + ]; + check( + "gateway_posted.json", + 1, + false, + Message::ApprovalPosted(ApprovalPosted { posts }), + ); + check( + "gateway_reply_approved.json", + 1, + true, + Message::ApprovalReply(reply(true, None)), + ); + check( + "gateway_reply_refused.json", + 1, + true, + Message::ApprovalReply(reply(false, Some("not on a Friday"))), + ); + for (name, outcome) in [ + ("gateway_outcome_runs.json", GatewayOutcome::Runs {}), + ( + "gateway_outcome_denied.json", + GatewayOutcome::Denied { + reason: DenyReason::NoGrant, + }, + ), + ("gateway_outcome_refused.json", GatewayOutcome::Refused {}), + ("gateway_outcome_gone.json", GatewayOutcome::Gone {}), + ] { + check( + name, + 2, + true, + Message::ApprovalOutcome(ApprovalOutcome { outcome }), + ); + } +} + +/// Every copy of `value` with one extra key in one object. +fn with_one_unknown_key(value: &Value) -> Vec { + let mut out = Vec::new(); + if let Value::Object(map) = value { + let mut extended = map.clone(); + extended.insert("zz_unknown".to_string(), Value::Bool(true)); + out.push(Value::Object(extended)); + for (key, child) in map { + for changed in with_one_unknown_key(child) { + let mut copy = map.clone(); + copy.insert(key.clone(), changed); + out.push(Value::Object(copy)); + } + } + } + out +} + +#[test] +fn every_object_refuses_an_unknown_key() { + for (name, objects) in [ + ("gateway_request.json", 4), + ("gateway_posted.json", 3), + ("gateway_reply_approved.json", 3), + ("gateway_reply_refused.json", 3), + ("gateway_outcome_runs.json", 4), + ("gateway_outcome_denied.json", 4), + ("gateway_outcome_refused.json", 4), + ("gateway_outcome_gone.json", 4), + ] { + let value: Value = serde_json::from_str(&fixture(name)).unwrap(); + let variations = with_one_unknown_key(&value); + assert_eq!(variations.len(), objects, "{name}: objects walked"); + for changed in variations { + assert!( + serde_json::from_value::(changed.clone()).is_err(), + "{name}: accepted {changed}" + ); + } + } +} + +#[test] +fn an_answer_is_checked() { + assert_eq!(reply(true, None).check(), Ok(())); + assert_eq!(reply(false, Some(&"x".repeat(MAX_REASON))).check(), Ok(())); + assert_eq!( + reply(false, Some(&"é".repeat(MAX_REASON))).check(), + Ok(()), + "characters, not bytes" + ); + assert!( + reply(false, Some(&"x".repeat(MAX_REASON + 1))) + .check() + .is_err() + ); + for bad in [ + "", + "K0000000000000000000000000", + "k000000000000000000000000", + "k00000000000000000000000000", + "../../../../../../../../etc", + ] { + let mut r = reply(true, None); + r.by = bad.to_string(); + assert!(r.check().is_err(), "by {bad:?}"); + let mut r = reply(true, None); + r.post = bad.to_string(); + assert!(r.check().is_err(), "post {bad:?}"); + assert!(!is_mattermost_id(bad), "{bad:?}"); + } + assert!(is_mattermost_id(BY) && is_mattermost_id(POST)); +} diff --git a/docs/plans/M4b/files/runbook-gateway-unreachable.md b/docs/plans/M4b/files/runbook-gateway-unreachable.md new file mode 100644 index 0000000..8de8814 --- /dev/null +++ b/docs/plans/M4b/files/runbook-gateway-unreachable.md @@ -0,0 +1,34 @@ + +## gateway-unreachable + +**What you see.** `brokerd` prints `brokerd: approval : gatewayd is not reachable at : +; bxctl can still answer it`, then this entry, once per approval. No approval post +appears in Mattermost. `bxctl approvals` lists the approval as usual. + +**Why.** `brokerd.toml` has a `[gateway]` section, so each pending approval is carried to +`gatewayd` to be posted in Mattermost. The courier could not connect to `gateway.sock`, or +`gatewayd` answered that it could not post (Mattermost unreachable). The approval is not lost: it +stays pending, `bxctl` can answer it, and the courier tries again every 5 seconds until it is +posted or answered. When a later try works, `brokerd` prints `brokerd: approval posted in +Mattermost`. + +**Confirm.** + +```sh +ls -l # gatewayd creates it at start; [approvals] socket in gatewayd.toml +pgrep -x gatewayd # is it running? +``` + +and `gatewayd`'s own output: `cannot reach …` there means Mattermost is the problem (see +[mattermost-unreachable](#mattermost-unreachable)). + +**Fix.** Start `gatewayd`, or make `[gateway] socket` in `brokerd.toml` and `[approvals] socket` +in `gatewayd.toml` name the same path (both default to `/run/gateway/gateway.sock`). To turn +Mattermost approvals off, remove `[gateway]` from `brokerd.toml` and restart `brokerd`. + +**Check.** Trigger an `ask` call: its approval is posted in Mattermost, and `!approve ` in +its thread answers it. + +Where two Boxmakers share a channel thread, address approval commands to one of them +(`@ !approve `): a bare `!approve ` is taken by every Boxmaker with that id pending +in that thread.