## already-running

**What you see.** A daemon exits 1 at start, before it touches its sockets, with one of these
lines:

- `<role>: another <role> is running (it holds <lock file>); see docs/runbook.md#already-running`
- `<role>: cannot take the lock <lock file>: <error>; see docs/runbook.md#already-running`

`<role>` is `loopd`, `brokerd`, `memoryd`, `gatewayd` or `inferproxy`. The lock file is
`$BOXMAKER_HOME/run/<role>.lock`; for `inferproxy` it is its `--listen` socket's path with
`.lock` added (`infer.sock.lock`, beside `infer.sock`).

**Why.** Every daemon removes a socket it finds at its path before it binds, so that a socket
left by a daemon that was killed never blocks the next start. A second copy started by mistake
would therefore take the sockets of the one that runs, which would keep running, unreachable,
and nothing would say so. So each daemon first takes an exclusive lock on its lock file, and
holds it for as long as it runs (M5b spec, section 9a). The system releases the lock when the
process ends, however it ends, so a killed daemon never leaves a lock behind. The file stays in
place and holds nothing.

**Confirm.** For `another <role> is running`, find the copy that runs, with its command line
(the `--config` path says which home it serves):

```sh
pgrep -a -x <role>
ls -l "$BOXMAKER_HOME/run/<role>.lock"
```

For `cannot take the lock`, the error says why: `Permission denied` means the daemon's user
cannot create or open the file, or create `run/` (`ls -ld "$BOXMAKER_HOME" "$BOXMAKER_HOME/run"`
and `ls -l` on the lock file show who owns them); `Not a directory` means a path above it is a
file; `Read-only file system` means what it says.

**Fix.** Two copies of a role against the same home are always a mistake: use the one that runs.
To restart it, stop it (`pkill -x <role>`, or Ctrl-C in its terminal), wait until `pgrep -x
<role>` prints nothing, and start it again. Two copies with different homes are fine (each has
its own lock), as long as their configs do not name the same socket path. For a lock that cannot
be taken, give the daemon's user the directory (`chown`), or point its home somewhere it can
write.

Never delete a lock file while its daemon runs: the lock is held on the open file, so a new copy
would create a new file, lock that, and take the sockets. Deleting it while the daemon is stopped
does no harm, and does not help either.

**Check.** `pgrep -x <role>` prints one process id, and the daemon prints its start line
(`loopd: serving on …`, `inferproxy: <socket> -> <upstream>`, `brokerd: serving tools on …`,
`memoryd: serving`, `gatewayd: connected to …`).