boxmaker/docs/plans/M3a/21-runbook-check.md

# M3a task 21: the runbook gate check

**Branch:** `m3a` (run `git switch m3a`; `git status --short` must be empty, otherwise stop)
**Commit subject:** `Check that every runbook pointer has an entry`

## Goal

Every message for a fail-closed state ends with `see docs/runbook.md#<anchor>`. A new gate script
fails if the code names an entry that `docs/runbook.md` does not have.

## Files

- Copy: `scripts/test-gate-scripts.sh`, and `Makefile` from `docs/plans/M3a/files/Makefile-task21`
  (`files/Makefile` is task 22's, with a line that needs a test this task does not have yet)
- Create: `scripts/check-runbook.sh`
- Modify: `docs/implementer-log.md`

You never edit `docs/runbook.md`. If the script finds a missing entry in the real tree, the
pointer in the code is wrong, or the entry is missing: stop and report which.

## What the script does

`sh scripts/check-runbook.sh [ROOT]`, `ROOT` defaulting to `.`, in POSIX `sh` like the other
scripts in `scripts/`. Read `scripts/check-lines.sh` first for the style.

1. A **pointer** is the text `docs/runbook.md#` followed by zero or more characters of
   `[A-Za-z0-9_-]`, anywhere in a `*.rs` file under `ROOT/crates`. Files under any `target/`
   directory are ignored (`find ... -type d -name target -prune -o -type f -name '*.rs' -print`).
   Test files count. A line can hold more than one pointer, and each counts.
2. The pointer's **anchor** is what follows the `#`. Its **entry** is a line of
   `ROOT/docs/runbook.md` that is exactly `## <anchor>`: the whole line, nothing before or after
   it (`grep -q -x -F -e "## $anchor"`). `### grants-invalid`, `## grants-invalid and more`, a
   mention in prose and `## Grants-Invalid` are not the entry for `grants-invalid`.
3. Exit status 0 if every anchor has its entry. Otherwise 1.

Every way it can fail, each with a message on stderr that starts with `check-runbook:`:

| Condition | Why |
|---|---|
| `ROOT/crates` is not a directory | nothing to check: fail closed |
| `ROOT/docs/runbook.md` is not a file | nothing to check against |
| `find`, `awk`, `sort` or `mktemp` fails, or a file cannot be read | a check that cannot do its job fails |
| `grep` exits with a status other than 0 or 1 | 1 means "no such line"; 2 means `grep` itself failed |
| no pointer is found anywhere | the harness has fail-closed states, so the search is broken |
| an anchor is empty, as in `docs/runbook.md#{anchor}` or `docs/runbook.md#<anchor>` | the script cannot read an anchor that is not written out |
| an anchor has no entry | the point of the script |

Rules from earlier reviews, which the self-test checks:

- **Report every problem, not only the first.** Go through all the anchors, print a message for
  each one that is missing, and exit 1 at the end. Do not `exit` inside the loop over anchors. For
  each missing anchor, name the files that point to it.
- **Never hide an error.** No `2>/dev/null`, no `|| true`. Test each command's exit status.
- A `while read` loop at the end of a pipeline runs in a subshell, and an `exit` or a variable
  set inside it is lost. Write the list to a file made with `mktemp -d` and read it with
  `while IFS= read -r line; do ...; done < "$file"`. Remove the directory with a `trap ... EXIT`.
- `grep -o` is not POSIX. Take the pointers out of a line with `awk`:

```sh
awk '{
    s = $0
    while (match(s, /docs\/runbook\.md#[A-Za-z0-9_-]*/)) {
        printf "%s\t%s\n", substr(s, RSTART + 16, RLENGTH - 16), FILENAME
        s = substr(s, RSTART + RLENGTH)
    }
}' "$f"
```

  `docs/runbook.md#` is 16 characters, so this prints the anchor, a tab, and the file.

## The Makefile

The copied `Makefile` runs `sh scripts/check-runbook.sh` in `gate`, after `check-dep-docs.sh`.

## Steps

- [ ] **1. Copy.** `git switch m3a`, then
  `cp docs/plans/M3a/files/scripts/test-gate-scripts.sh scripts/ && cp docs/plans/M3a/files/Makefile-task21 Makefile`
- [ ] **2. See the self-test fail.** `sh scripts/test-gate-scripts.sh`. Expected: failures that
  name `check-runbook.sh` cases, and exit status 1.
- [ ] **3. Write `scripts/check-runbook.sh`.**
- [ ] **4. See the self-test pass.** `sh scripts/test-gate-scripts.sh`. Expected:
  `test-gate-scripts: ok`.
- [ ] **5. Prove the self-test has teeth.** Change `-x` to nothing in your `grep` line and run the
  self-test: it must fail with "the entry is the whole line, at level two". Change it back. Then
  put `exit 1` where a missing anchor is reported and run it: it must fail with "both missing
  entries and their files are reported". Change it back. Write both results in the log's Notes.
- [ ] **6. Run it on the real tree.** `sh scripts/check-runbook.sh; echo $?`. Expected: no output
  and `0`. If it names a missing entry, stop and report: do not edit the runbook, the pointer or
  a copied test.
- [ ] **7. Run the gate.** `make gate`. Expected last line: `gate: ok`.
- [ ] **8. Log and commit.**
  `git add scripts/check-runbook.sh scripts/test-gate-scripts.sh Makefile docs/implementer-log.md && git commit`

## Done when

- `sh scripts/test-gate-scripts.sh` prints `test-gate-scripts: ok`; `sh scripts/check-runbook.sh`
  exits 0 on the repository; `make gate` prints `gate: ok`.

## Stop and report if

- Step 6 reports a missing entry or an empty anchor.
- `sh` on this machine lacks something the task relies on (`mktemp -d`, `awk`'s `match`).