Harden deployment and fix PEN-01
- Fix Bearer token extraction to validate prefix (PEN-01) - Add TestExtractBearerFromRequest covering PEN-01 edge cases - Fix flaky TestRenewToken timing (2s → 4s lifetime) - Move default config/install paths to /srv/mcias - Add RUNBOOK.md for operational procedures - Update AUDIT.md with penetration test round 4 Security: extractBearerFromRequest now uses case-insensitive prefix validation instead of fixed-offset slicing, rejecting non-Bearer Authorization schemes that were previously accepted. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
464
RUNBOOK.md
Normal file
464
RUNBOOK.md
Normal file
@@ -0,0 +1,464 @@
|
||||
# MCIAS Runbook
|
||||
|
||||
Operational procedures for running and maintaining the MCIAS authentication
|
||||
server. All required files live under `/srv/mcias`.
|
||||
|
||||
---
|
||||
|
||||
## Directory Layout
|
||||
|
||||
```
|
||||
/srv/mcias/
|
||||
mcias.toml — server configuration (TOML)
|
||||
server.crt — TLS certificate (PEM)
|
||||
server.key — TLS private key (PEM, mode 0640)
|
||||
mcias.db — SQLite database (WAL mode creates .db-wal and .db-shm)
|
||||
env — environment file: MCIAS_MASTER_PASSPHRASE (mode 0640)
|
||||
master.key — optional raw AES-256 key file (mode 0640, alternative to env)
|
||||
```
|
||||
|
||||
All files are owned by the `mcias` system user and group (`mcias:mcias`).
|
||||
The directory itself is mode `0750`.
|
||||
|
||||
---
|
||||
|
||||
## Installation
|
||||
|
||||
Run as root from the repository root after `make build`:
|
||||
|
||||
```sh
|
||||
sh dist/install.sh
|
||||
```
|
||||
|
||||
This script is idempotent. It:
|
||||
1. Creates the `mcias` system user and group if they do not exist.
|
||||
2. Installs binaries to `/usr/local/bin/`.
|
||||
3. Creates `/srv/mcias/` with correct ownership and permissions.
|
||||
4. Installs the systemd service unit to `/etc/systemd/system/mcias.service`.
|
||||
5. Installs example config files to `/srv/mcias/` (will not overwrite existing files).
|
||||
|
||||
After installation, complete the steps below before starting the service.
|
||||
|
||||
---
|
||||
|
||||
## First-Run Setup
|
||||
|
||||
### 1. Generate a TLS certificate
|
||||
|
||||
**Self-signed (personal/development use):**
|
||||
|
||||
```sh
|
||||
openssl req -x509 -newkey ed25519 -days 3650 \
|
||||
-keyout /srv/mcias/server.key \
|
||||
-out /srv/mcias/server.crt \
|
||||
-subj "/CN=auth.example.com" \
|
||||
-nodes
|
||||
chmod 0640 /srv/mcias/server.key
|
||||
chown mcias:mcias /srv/mcias/server.key /srv/mcias/server.crt
|
||||
```
|
||||
|
||||
**Using the `cert` tool:**
|
||||
|
||||
```sh
|
||||
go install github.com/kisom/cert@latest
|
||||
|
||||
cat > /tmp/request.yaml <<EOF
|
||||
subject:
|
||||
common_name: auth.example.com
|
||||
hosts:
|
||||
- auth.example.com
|
||||
key:
|
||||
algo: ecdsa
|
||||
size: 521
|
||||
ca:
|
||||
expiry: 87600h
|
||||
EOF
|
||||
|
||||
cert genkey -a ec -s 521 > /srv/mcias/server.key
|
||||
cert selfsign -p /srv/mcias/server.key -f /tmp/request.yaml > /srv/mcias/server.crt
|
||||
chmod 0640 /srv/mcias/server.key
|
||||
chown mcias:mcias /srv/mcias/server.key /srv/mcias/server.crt
|
||||
rm /tmp/request.yaml
|
||||
```
|
||||
|
||||
### 2. Write the configuration file
|
||||
|
||||
```sh
|
||||
cp /srv/mcias/mcias.conf.example /srv/mcias/mcias.toml
|
||||
$EDITOR /srv/mcias/mcias.toml
|
||||
chmod 0640 /srv/mcias/mcias.toml
|
||||
chown mcias:mcias /srv/mcias/mcias.toml
|
||||
```
|
||||
|
||||
Minimum required settings:
|
||||
|
||||
```toml
|
||||
[server]
|
||||
listen_addr = "0.0.0.0:8443"
|
||||
tls_cert = "/srv/mcias/server.crt"
|
||||
tls_key = "/srv/mcias/server.key"
|
||||
|
||||
[database]
|
||||
path = "/srv/mcias/mcias.db"
|
||||
|
||||
[tokens]
|
||||
issuer = "https://auth.example.com"
|
||||
|
||||
[master_key]
|
||||
passphrase_env = "MCIAS_MASTER_PASSPHRASE"
|
||||
```
|
||||
|
||||
See `dist/mcias.conf.example` for the full annotated reference.
|
||||
|
||||
### 3. Set the master key passphrase
|
||||
|
||||
```sh
|
||||
cp /srv/mcias/mcias.env.example /srv/mcias/env
|
||||
$EDITOR /srv/mcias/env # set MCIAS_MASTER_PASSPHRASE to a long random value
|
||||
chmod 0640 /srv/mcias/env
|
||||
chown mcias:mcias /srv/mcias/env
|
||||
```
|
||||
|
||||
Generate a strong passphrase:
|
||||
|
||||
```sh
|
||||
openssl rand -base64 32
|
||||
```
|
||||
|
||||
> **IMPORTANT:** Back up the passphrase to a secure offline location.
|
||||
> Losing it permanently destroys access to all encrypted data in the database.
|
||||
|
||||
### 4. Create the first admin account
|
||||
|
||||
```sh
|
||||
export MCIAS_MASTER_PASSPHRASE=your-passphrase
|
||||
|
||||
mciasdb --config /srv/mcias/mcias.toml account create \
|
||||
--username admin --type human
|
||||
# note the UUID printed
|
||||
|
||||
mciasdb --config /srv/mcias/mcias.toml account set-password --id <UUID>
|
||||
mciasdb --config /srv/mcias/mcias.toml role grant --id <UUID> --role admin
|
||||
```
|
||||
|
||||
### 5. Enable and start the service
|
||||
|
||||
```sh
|
||||
systemctl enable mcias
|
||||
systemctl start mcias
|
||||
systemctl status mcias
|
||||
```
|
||||
|
||||
### 6. Verify
|
||||
|
||||
```sh
|
||||
curl -k https://auth.example.com:8443/v1/health
|
||||
# {"status":"ok"}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Routine Operations
|
||||
|
||||
### Start / stop / restart
|
||||
|
||||
```sh
|
||||
systemctl start mcias
|
||||
systemctl stop mcias
|
||||
systemctl restart mcias
|
||||
```
|
||||
|
||||
### View logs
|
||||
|
||||
```sh
|
||||
journalctl -u mcias -f
|
||||
journalctl -u mcias --since "1 hour ago"
|
||||
```
|
||||
|
||||
### Check service status
|
||||
|
||||
```sh
|
||||
systemctl status mcias
|
||||
```
|
||||
|
||||
### Reload configuration
|
||||
|
||||
The server reads its configuration at startup only. To apply config changes:
|
||||
|
||||
```sh
|
||||
systemctl restart mcias
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Account Management
|
||||
|
||||
All account management can be done via `mciasctl` (REST API) when the server
|
||||
is running, or `mciasdb` for offline/break-glass operations.
|
||||
|
||||
```sh
|
||||
# Set env for offline tool
|
||||
export MCIAS_MASTER_PASSPHRASE=your-passphrase
|
||||
CONF="--config /srv/mcias/mcias.toml"
|
||||
|
||||
# List accounts
|
||||
mciasdb $CONF account list
|
||||
|
||||
# Create account
|
||||
mciasdb $CONF account create --username alice --type human
|
||||
|
||||
# Set password (prompts interactively)
|
||||
mciasdb $CONF account set-password --id <UUID>
|
||||
|
||||
# Grant or revoke a role
|
||||
mciasdb $CONF role grant --id <UUID> --role admin
|
||||
mciasdb $CONF role revoke --id <UUID> --role admin
|
||||
|
||||
# Disable account
|
||||
mciasdb $CONF account set-status --id <UUID> --status inactive
|
||||
|
||||
# Delete account
|
||||
mciasdb $CONF account set-status --id <UUID> --status deleted
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Token Management
|
||||
|
||||
```sh
|
||||
CONF="--config /srv/mcias/mcias.toml"
|
||||
|
||||
# List active tokens for an account
|
||||
mciasdb $CONF token list --id <UUID>
|
||||
|
||||
# Revoke a specific token by JTI
|
||||
mciasdb $CONF token revoke --jti <JTI>
|
||||
|
||||
# Revoke all tokens for an account (e.g., suspected compromise)
|
||||
mciasdb $CONF token revoke-all --id <UUID>
|
||||
|
||||
# Prune expired tokens from the database
|
||||
mciasdb $CONF prune tokens
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Database Maintenance
|
||||
|
||||
### Verify schema
|
||||
|
||||
```sh
|
||||
mciasdb --config /srv/mcias/mcias.toml schema verify
|
||||
```
|
||||
|
||||
### Run pending migrations
|
||||
|
||||
```sh
|
||||
mciasdb --config /srv/mcias/mcias.toml schema migrate
|
||||
```
|
||||
|
||||
### Force schema version (break-glass)
|
||||
|
||||
```sh
|
||||
mciasdb --config /srv/mcias/mcias.toml schema force --version N
|
||||
```
|
||||
|
||||
Use only when `schema migrate` reports a dirty version after a failed migration.
|
||||
|
||||
### Backup the database
|
||||
|
||||
SQLite WAL mode creates three files. Back up all three atomically using the
|
||||
SQLite backup API or by stopping the server first:
|
||||
|
||||
```sh
|
||||
# Online backup (preferred — no downtime):
|
||||
sqlite3 /srv/mcias/mcias.db ".backup /path/to/backup/mcias-$(date +%F).db"
|
||||
|
||||
# Offline backup:
|
||||
systemctl stop mcias
|
||||
cp /srv/mcias/mcias.db /path/to/backup/mcias-$(date +%F).db
|
||||
systemctl start mcias
|
||||
```
|
||||
|
||||
Store backups alongside a copy of the master key passphrase in a secure
|
||||
offline location. A database backup without the passphrase is unrecoverable.
|
||||
|
||||
---
|
||||
|
||||
## Audit Log
|
||||
|
||||
```sh
|
||||
CONF="--config /srv/mcias/mcias.toml"
|
||||
|
||||
# Show last 50 audit events
|
||||
mciasdb $CONF audit tail --n 50
|
||||
|
||||
# Query by account
|
||||
mciasdb $CONF audit query --account <UUID>
|
||||
|
||||
# Query by event type since a given time
|
||||
mciasdb $CONF audit query --type login_failure --since 2026-01-01T00:00:00Z
|
||||
|
||||
# Output as JSON (for log shipping)
|
||||
mciasdb $CONF audit query --json
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Upgrading
|
||||
|
||||
1. Build the new binaries: `make build`
|
||||
2. Stop the service: `systemctl stop mcias`
|
||||
3. Install new binaries: `sh dist/install.sh`
|
||||
- The script will not overwrite existing config files.
|
||||
- New example files are placed with a `.new` suffix for review.
|
||||
4. Review any `.new` config files in `/srv/mcias/` and merge changes manually.
|
||||
5. Run schema migrations if required:
|
||||
```sh
|
||||
mciasdb --config /srv/mcias/mcias.toml schema migrate
|
||||
```
|
||||
6. Start the service: `systemctl start mcias`
|
||||
7. Verify: `curl -k https://auth.example.com:8443/v1/health`
|
||||
|
||||
---
|
||||
|
||||
## Master Key Rotation
|
||||
|
||||
> This operation is not yet automated. Until a rotation command is
|
||||
> implemented, rotation requires a full re-encryption of the database.
|
||||
> Contact the project maintainer for the current procedure.
|
||||
|
||||
---
|
||||
|
||||
## TLS Certificate Renewal
|
||||
|
||||
Replace the certificate and key files, then restart the server:
|
||||
|
||||
```sh
|
||||
# Generate or obtain new cert/key, then:
|
||||
cp new-server.crt /srv/mcias/server.crt
|
||||
cp new-server.key /srv/mcias/server.key
|
||||
chmod 0640 /srv/mcias/server.key
|
||||
chown mcias:mcias /srv/mcias/server.crt /srv/mcias/server.key
|
||||
systemctl restart mcias
|
||||
```
|
||||
|
||||
For Let's Encrypt with Certbot, add a deploy hook:
|
||||
|
||||
```sh
|
||||
# /etc/letsencrypt/renewal-hooks/deploy/mcias.sh
|
||||
#!/bin/sh
|
||||
cp /etc/letsencrypt/live/auth.example.com/fullchain.pem /srv/mcias/server.crt
|
||||
cp /etc/letsencrypt/live/auth.example.com/privkey.pem /srv/mcias/server.key
|
||||
chmod 0640 /srv/mcias/server.key
|
||||
chown mcias:mcias /srv/mcias/server.crt /srv/mcias/server.key
|
||||
systemctl restart mcias
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Docker Deployment
|
||||
|
||||
```sh
|
||||
make docker
|
||||
|
||||
mkdir -p /srv/mcias
|
||||
cp dist/mcias.conf.docker.example /srv/mcias/mcias.toml
|
||||
$EDITOR /srv/mcias/mcias.toml
|
||||
|
||||
# Place TLS cert and key under /srv/mcias/
|
||||
# Set ownership so uid 10001 (container mcias user) can read them.
|
||||
chown -R 10001:10001 /srv/mcias
|
||||
|
||||
docker run -d \
|
||||
--name mcias \
|
||||
-v /srv/mcias:/srv/mcias \
|
||||
-e MCIAS_MASTER_PASSPHRASE=your-passphrase \
|
||||
-p 8443:8443 \
|
||||
-p 9443:9443 \
|
||||
--restart unless-stopped \
|
||||
mcias:latest
|
||||
```
|
||||
|
||||
See `dist/mcias.conf.docker.example` for the full annotated Docker config.
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Server fails to start: "open database"
|
||||
|
||||
Check that `/srv/mcias/` is writable by the `mcias` user:
|
||||
|
||||
```sh
|
||||
ls -la /srv/mcias/
|
||||
stat /srv/mcias/mcias.db # if it already exists
|
||||
```
|
||||
|
||||
Fix: `chown mcias:mcias /srv/mcias`
|
||||
|
||||
### Server fails to start: "environment variable ... is not set"
|
||||
|
||||
The `MCIAS_MASTER_PASSPHRASE` env var is missing. Ensure `/srv/mcias/env`
|
||||
exists, is readable by the mcias user, and contains the correct variable:
|
||||
|
||||
```sh
|
||||
grep MCIAS_MASTER_PASSPHRASE /srv/mcias/env
|
||||
```
|
||||
|
||||
Also confirm the systemd unit loads it:
|
||||
|
||||
```sh
|
||||
systemctl cat mcias | grep EnvironmentFile
|
||||
```
|
||||
|
||||
### Server fails to start: "decrypt signing key"
|
||||
|
||||
The master key passphrase has changed or is wrong. The passphrase must match
|
||||
the one used when the database was first initialized (the KDF salt is stored
|
||||
in the database). Restore the correct passphrase from your offline backup.
|
||||
|
||||
### TLS errors in client connections
|
||||
|
||||
Verify the certificate is valid and covers the correct hostname:
|
||||
|
||||
```sh
|
||||
openssl x509 -in /srv/mcias/server.crt -noout -text | grep -E "Subject|DNS"
|
||||
openssl x509 -in /srv/mcias/server.crt -noout -dates
|
||||
```
|
||||
|
||||
### Database locked / WAL not cleaning up
|
||||
|
||||
Check for lingering `mcias.db-wal` and `mcias.db-shm` files after an unclean
|
||||
shutdown. These are safe to leave in place — SQLite will recover on next open.
|
||||
Do not delete them while the server is running.
|
||||
|
||||
### Schema dirty after failed migration
|
||||
|
||||
```sh
|
||||
mciasdb --config /srv/mcias/mcias.toml schema verify
|
||||
mciasdb --config /srv/mcias/mcias.toml schema force --version N
|
||||
mciasdb --config /srv/mcias/mcias.toml schema migrate
|
||||
```
|
||||
|
||||
Replace `N` with the last successfully applied version number.
|
||||
|
||||
---
|
||||
|
||||
## File Permissions Reference
|
||||
|
||||
| Path | Mode | Owner |
|
||||
|------|------|-------|
|
||||
| `/srv/mcias/` | `0750` | `mcias:mcias` |
|
||||
| `/srv/mcias/mcias.toml` | `0640` | `mcias:mcias` |
|
||||
| `/srv/mcias/server.crt` | `0644` | `mcias:mcias` |
|
||||
| `/srv/mcias/server.key` | `0640` | `mcias:mcias` |
|
||||
| `/srv/mcias/mcias.db` | `0640` | `mcias:mcias` |
|
||||
| `/srv/mcias/env` | `0640` | `mcias:mcias` |
|
||||
| `/srv/mcias/master.key` | `0640` | `mcias:mcias` |
|
||||
|
||||
Verify permissions:
|
||||
|
||||
```sh
|
||||
ls -la /srv/mcias/
|
||||
```
|
||||
Reference in New Issue
Block a user