# M5b task 13: `brokerd`, `memoryd` and `gatewayd` take the instance lock too **Branch:** `m5b` (run `git switch m5b`; `git status --short` must be empty, otherwise stop) **Commit subject:** `brokerd, memoryd, gatewayd: a lock file keeps each to one instance` ## Goal Task 12 wrote `bxcore::instance`, the lock that keeps a daemon to one instance (M5b spec, section 9a), and made `loopd` and `inferproxy` take it. This task makes the other three daemons take it the same way: before they touch their sockets, each locks `/run/.lock` and holds the lock for the life of the process. A second copy against the same home exits 1 with one line and touches nothing else: ```text brokerd: another brokerd is running (it holds /var/lib/boxmaker/run/brokerd.lock); see docs/runbook.md#already-running memoryd: another memoryd is running (it holds /var/lib/boxmaker/run/memoryd.lock); see docs/runbook.md#already-running gatewayd: another gatewayd is running (it holds /var/lib/boxmaker/run/gatewayd.lock); see docs/runbook.md#already-running ``` Any other error on the lock file stops the daemon too, with the line `: cannot take the lock : ; see docs/runbook.md#already-running`. Both lines come from `LockError::line`; the runbook entry `already-running` is there since task 12, and covers all five daemons. ## Files - Copy, from `docs/plans/M5b/files/` (the same path under it): - new tests: `crates/brokerd/tests/instance.rs`, `crates/memoryd/tests/instance.rs`, `crates/gatewayd/tests/instance.rs` - a **new version** of `crates/brokerd/tests/serve.rs` (it replaces the file in the tree). Do not copy `serve.rs-task01`: it is the older version. - Modify: `crates/brokerd/src/main.rs` (B1, B2), `crates/memoryd/src/main.rs` (D1, D2), `crates/gatewayd/src/main.rs` (G1 to G3), `docs/implementer-log.md`. - Do not touch: `crates/bxcore` (`instance` is done), any other source file, any `Cargo.toml`, any other test, `docs/runbook.md`. ## Facts you need - **What `bxcore::instance` gives** (task 12; do not change it): | Name | What it is | |---|---| | `instance::lock` | `pub fn lock(path: &Path) -> Result`: creates the file (0600) and a missing directory (0700), and tries the lock once | | `InstanceLock` | the held lock; the lock is released when it is dropped or the process ends | | `LockError` | `Held` (another instance holds it) or `Io(std::io::Error)` | | `LockError::line` | `e.line("brokerd", &lock_path) -> String`: the whole line to print, pointer included | - **Keep the lock alive.** Each daemon binds the `InstanceLock` to a name, `_instance`, in `main` (or `serve` for `gatewayd`), the function that serves until the process ends. `let _ = …`, or a `match` arm `Ok(_) => {}`, would drop it, and the lock with it, at once. The edits show the form. - **Where each daemon takes it**, and why there: - `brokerd` (`main` in `crates/brokerd/src/main.rs`): right after the config loads, before `serve::start`, which opens the audit log and binds both sockets. The home is `cfg.paths.home`. - `memoryd` (`main` in `crates/memoryd/src/main.rs`): right after the config loads, before `serve::start`, which creates `memory/` and the index and binds three sockets. The home is `config.paths.home`. - `gatewayd` (`serve` in `crates/gatewayd/src/main.rs`): after the secret is loaded, before the state directory is made and before `run`, which binds `gateway.sock` and then connects to Mattermost. Not earlier: the test `a_missing_secret_stops_at_start` in `crates/gatewayd/tests/main.rs` checks that nothing under the home is made before the secret is read, and the lock creates `/run`. The home is `config.paths.home`. - **`brokerd`'s audit lock stays.** `brokerd` has held a lock on `/audit/.lock` since M3a, and a second `brokerd` used to stop there, with `brokerd is already running` and the pointer `brokerd-already-running`. It stays: it guards the log itself, and its own test (`a_second_writer_is_refused` in `crates/brokerd/tests/audit.rs`) still passes. But the instance lock now comes first, so a second `brokerd` stops with the new line instead. That is why `serve.rs` has a new version: its test `a_second_brokerd_on_the_same_home_refuses_to_start` expected the old message, and it is gone; `instance.rs` checks the second `brokerd` now. Nothing else in `serve.rs` changes. - **The given tests start each daemon as a process**, as `brokerd`'s `serve.rs` does. The `gatewayd` one runs it against the fake Mattermost of `tests/support/fake_mm.rs`, with its token in the environment (`GW_TEST_TOKEN`), and checks that it answers on `gateway.sock` by asking it to post an approval, as `brokerd` would. Every wait has a deadline and every process is killed by a guard. Before this task, the `memoryd` and `gatewayd` tests that expect a second instance to be refused wait out their deadline (the second copy starts and serves), so each of those files takes about 10 s to fail. That is this, not a hang. - The calls the edits make: `instance::lock` and `LockError::line` from the table above; `Path::join` for the lock file's path; `eprintln!` and `ExitCode::from(1)` as the lines around them already do. ## The edits, exactly Nothing else in these files changes. Each quoted "before" block occurs exactly once in its file. ### `crates/brokerd/src/main.rs` **B1.** The line ```rust use brokerd::serve::{self, ServeError}; ``` becomes ```rust use brokerd::serve::{self, ServeError}; use bxcore::instance; ``` **B2.** In `main`, the line ```rust // Keep the socket paths for the serving notice, before `cfg` is moved into `start`. ``` becomes ```rust // One brokerd per home (M5b spec, section 9a). The lock comes before anything is made, the // audit log's own lock included, so a second copy never takes the sockets of the one that // runs. Held until the process ends. let lock_path = cfg.paths.home.join("run/brokerd.lock"); let _instance = match instance::lock(&lock_path) { Ok(held) => held, Err(e) => { eprintln!("{}", e.line("brokerd", &lock_path)); return ExitCode::from(1); } }; // Keep the socket paths for the serving notice, before `cfg` is moved into `start`. ``` ### `crates/memoryd/src/main.rs` **D1.** The line ```rust use memoryd::config::Config; ``` becomes ```rust use bxcore::instance; use memoryd::config::Config; ``` **D2.** In `main`, the line ```rust let log: serve::Log = Arc::new(|line: &str| eprintln!("{line}")); ``` becomes ```rust // One memoryd per home (M5b spec, section 9a). The lock comes before anything under the home // is made or bound, so a second copy never takes the sockets of the one that runs. Held until // the process ends. let lock_path = config.paths.home.join("run/memoryd.lock"); let _instance = match instance::lock(&lock_path) { Ok(held) => held, Err(e) => { eprintln!("{}", e.line("memoryd", &lock_path)); return ExitCode::from(1); } }; let log: serve::Log = Arc::new(|line: &str| eprintln!("{line}")); ``` ### `crates/gatewayd/src/main.rs` **G1.** The line ```rust use bxcore::secrets; ``` becomes ```rust use bxcore::{instance, secrets}; ``` **G2.** In the comment at the top of `serve`, the line ```rust // Then `if let Some(warning) = &loaded.warning { eprintln!("{warning}"); }`. ``` becomes ```rust // Then `if let Some(warning) = &loaded.warning { eprintln!("{warning}"); }`. // 3b. The instance lock, `/run/gatewayd.lock` (M5b spec, section 9a): after the secret, // which is read before anything is made, and before the directory and the socket. ``` **G3.** In `serve`, the lines ```rust if let Some(warning) = &loaded.warning { eprintln!("{warning}"); } ``` become ```rust if let Some(warning) = &loaded.warning { eprintln!("{warning}"); } let lock_path = config.paths.home.join("run/gatewayd.lock"); let _instance = match instance::lock(&lock_path) { Ok(held) => held, Err(e) => { eprintln!("{}", e.line("gatewayd", &lock_path)); return ExitCode::from(1); } }; ``` ## Steps - [ ] **1. Copy the given tests.** From the repository root, four commands: ```sh cp docs/plans/M5b/files/crates/brokerd/tests/instance.rs crates/brokerd/tests/ cp docs/plans/M5b/files/crates/brokerd/tests/serve.rs crates/brokerd/tests/ cp docs/plans/M5b/files/crates/memoryd/tests/instance.rs crates/memoryd/tests/ cp docs/plans/M5b/files/crates/gatewayd/tests/instance.rs crates/gatewayd/tests/ ``` - [ ] **2. See them fail.** Three commands, each with a shell timeout of 600000 ms. Each file compiles; `serve.rs` passes. - `cargo test -p brokerd --no-fail-fast --test instance --test serve` (`--no-fail-fast`, or cargo stops at the first failing file): `serve` 8 passed. `instance`: 2 failed. `a_second_brokerd_on_the_same_home_is_refused_and_the_first_keeps_serving` fails on `the lock's line, and nothing else` (the second stopped at the audit lock, with `brokerd is already running`); `after_the_first_is_killed_a_new_brokerd_removes_the_stale_sockets_and_serves` fails on `the lock file, mode 0600`, `left: None`, `right: Some(384)`. - `cargo test -p memoryd --test instance`: 2 failed (about 10 s). `a_second_memoryd_on_the_same_home_is_refused_and_the_first_keeps_serving` fails on `the second memoryd: …` with `left: None`, `right: Some(1)` (it started and served); `after_the_first_is_killed_a_new_memoryd_removes_the_stale_sockets_and_serves` on `the lock file, mode 0600`, `left: None`. - `cargo test -p gatewayd --test instance`: 2 failed (about 10 s), the same two shapes: `the second gatewayd: …` with `left: None`, and `the lock file, mode 0600` with `left: None`. - [ ] **3. `brokerd`.** Make edits B1 and B2. `cargo check -p brokerd`: no error. `cargo test -p brokerd --test instance --test serve`: `instance` 2 passed, `serve` 8 passed. - [ ] **4. `memoryd`.** Make edits D1 and D2. `cargo check -p memoryd`: no error. `cargo test -p memoryd --test instance`: 2 passed. - [ ] **5. `gatewayd`.** Make edits G1 to G3. `cargo check -p gatewayd`: no error. `cargo test -p gatewayd --test instance --test main`: `instance` 2 passed, `main` 5 passed. - [ ] **6. Check.** `grep -rn 'instance::lock(' crates/*/src` prints exactly five lines, one in the `main.rs` of each of `brokerd`, `gatewayd`, `inferproxy`, `loopd` and `memoryd`. `grep -rn 'runbook.md#already-running' crates/*/src` still prints exactly one line, in `crates/bxcore/src/instance.rs`. `git diff --stat -- crates/bxcore '*Cargo.toml'` prints nothing. - [ ] **7. Run the gate.** `cargo fmt --all`, then `make gate`, with a shell timeout of 600000 ms. Expected last line: `gate: ok`, with 1118 tests: 1113 before this task, minus 1 in the new `crates/brokerd/tests/serve.rs` (its old second-instance test), plus 2 in each of the three new `instance.rs` files. - [ ] **8. Log and commit.** `git status --short` lists `crates/brokerd/src/main.rs`, `crates/memoryd/src/main.rs`, `crates/gatewayd/src/main.rs`, `crates/brokerd/tests/serve.rs` and your log as modified, and the three `instance.rs` test files as new. Stage with `git add crates/brokerd crates/memoryd crates/gatewayd docs/implementer-log.md Cargo.lock`, then commit as `AGENTS.md` says. `Cargo.lock` does not change in this task; the `git add` is harmless. ## Done when - `make gate` prints `gate: ok` with 1118 tests. - `git diff HEAD~1 --stat` lists only the three `main.rs` files, `crates/brokerd/tests/serve.rs`, the three new `instance.rs` test files and `docs/implementer-log.md`. ## Stop and report if - A "before" block of B1, B2, D1, D2 or G1 to G3 is not in its file exactly as shown, or is there more than once. - Step 2 shows a different result: other failing tests, other messages, or other counts. - `cargo check` reports an error after an edit made as shown. Quote it and stop. - After step 3, 4 or 5 a test fails although the edits are as shown. Do not change a test or `bxcore`: stop, and give the failing tests' names and their messages. - The fix seems to need a change anywhere else: `bxcore`, another file of a daemon, the test support files, a `Cargo.toml`, the runbook. It does not; stop and say why you think so. - A test outside these files fails in the gate: check once whether it fails without your change, then stop and report both results. - The gate passes with a count other than 1118.