mcias/RUNBOOK.md

MCIAS Runbook

Operational procedures for running and maintaining the MCIAS authentication server. All required files live under /srv/mcias.

---

Directory Layout

/srv/mcias/
  mcias.toml      — server configuration (TOML)
  server.crt      — TLS certificate (PEM)
  server.key      — TLS private key (PEM, mode 0640)
  mcias.db        — SQLite database (WAL mode creates .db-wal and .db-shm)
  env             — environment file: MCIAS_MASTER_PASSPHRASE (mode 0640)
  master.key      — optional raw AES-256 key file (mode 0640, alternative to env)

All files are owned by the mcias system user and group (mcias:mcias). The directory itself is mode 0750.

---

Installation

Run as root from the repository root after make build:

sh dist/install.sh

This script is idempotent. It:

1. Creates the mcias system user and group if they do not exist.
2. Installs binaries to /usr/local/bin/.
3. Creates /srv/mcias/ with correct ownership and permissions.
4. Installs the systemd service unit to /etc/systemd/system/mcias.service.
5. Installs example config files to /srv/mcias/ (will not overwrite existing files).

After installation, complete the steps below before starting the service.

---

First-Run Setup

1. Generate a TLS certificate

Self-signed (personal/development use):

openssl req -x509 -newkey ed25519 -days 3650 \
  -keyout /srv/mcias/server.key \
  -out    /srv/mcias/server.crt \
  -subj   "/CN=auth.example.com" \
  -nodes
chmod 0640 /srv/mcias/server.key
chown mcias:mcias /srv/mcias/server.key /srv/mcias/server.crt

Using the cert tool:

go install github.com/kisom/cert@latest

cat > /tmp/request.yaml <<EOF
subject:
  common_name: auth.example.com
hosts:
  - auth.example.com
key:
  algo: ecdsa
  size: 521
ca:
  expiry: 87600h
EOF

cert genkey -a ec -s 521 > /srv/mcias/server.key
cert selfsign -p /srv/mcias/server.key -f /tmp/request.yaml > /srv/mcias/server.crt
chmod 0640 /srv/mcias/server.key
chown mcias:mcias /srv/mcias/server.key /srv/mcias/server.crt
rm /tmp/request.yaml

2. Write the configuration file

cp /srv/mcias/mcias.conf.example /srv/mcias/mcias.toml
$EDITOR /srv/mcias/mcias.toml
chmod 0640 /srv/mcias/mcias.toml
chown mcias:mcias /srv/mcias/mcias.toml

Minimum required settings:

[server]
listen_addr = "0.0.0.0:8443"
tls_cert    = "/srv/mcias/server.crt"
tls_key     = "/srv/mcias/server.key"

[database]
path = "/srv/mcias/mcias.db"

[tokens]
issuer = "https://auth.example.com"

[master_key]
passphrase_env = "MCIAS_MASTER_PASSPHRASE"

See dist/mcias.conf.example for the full annotated reference.

3. Set the master key passphrase

cp /srv/mcias/mcias.env.example /srv/mcias/env
$EDITOR /srv/mcias/env     # set MCIAS_MASTER_PASSPHRASE to a long random value
chmod 0640 /srv/mcias/env
chown mcias:mcias /srv/mcias/env

Generate a strong passphrase:

openssl rand -base64 32

IMPORTANT: Back up the passphrase to a secure offline location. Losing it permanently destroys access to all encrypted data in the database.

4. Create the first admin account

export MCIAS_MASTER_PASSPHRASE=your-passphrase

mciasdb --config /srv/mcias/mcias.toml account create \
  --username admin --type human
# note the UUID printed

mciasdb --config /srv/mcias/mcias.toml account set-password --id <UUID>
mciasdb --config /srv/mcias/mcias.toml role grant --id <UUID> --role admin

5. Enable and start the service

systemctl enable mcias
systemctl start mcias
systemctl status mcias

6. Verify

curl -k https://auth.example.com:8443/v1/health
# {"status":"ok"}

---

Routine Operations

Start / stop / restart

systemctl start mcias
systemctl stop mcias
systemctl restart mcias

View logs

journalctl -u mcias -f
journalctl -u mcias --since "1 hour ago"

Check service status

systemctl status mcias

Reload configuration

The server reads its configuration at startup only. To apply config changes:

systemctl restart mcias

---

Account Management

All account management can be done via mciasctl (REST API) when the server is running, or mciasdb for offline/break-glass operations.

# Set env for offline tool
export MCIAS_MASTER_PASSPHRASE=your-passphrase
CONF="--config /srv/mcias/mcias.toml"

# List accounts
mciasdb $CONF account list

# Create account
mciasdb $CONF account create --username alice --type human

# Set password (prompts interactively)
mciasdb $CONF account set-password --id <UUID>

# Grant or revoke a role
mciasdb $CONF role grant  --id <UUID> --role admin
mciasdb $CONF role revoke --id <UUID> --role admin

# Disable account
mciasdb $CONF account set-status --id <UUID> --status inactive

# Delete account
mciasdb $CONF account set-status --id <UUID> --status deleted

---

Token Management

CONF="--config /srv/mcias/mcias.toml"

# List active tokens for an account
mciasdb $CONF token list --id <UUID>

# Revoke a specific token by JTI
mciasdb $CONF token revoke --jti <JTI>

# Revoke all tokens for an account (e.g., suspected compromise)
mciasdb $CONF token revoke-all --id <UUID>

# Prune expired tokens from the database
mciasdb $CONF prune tokens

---

Database Maintenance

Verify schema

mciasdb --config /srv/mcias/mcias.toml schema verify

Run pending migrations

mciasdb --config /srv/mcias/mcias.toml schema migrate

Force schema version (break-glass)

mciasdb --config /srv/mcias/mcias.toml schema force --version N

Use only when schema migrate reports a dirty version after a failed migration.

Backup the database

SQLite WAL mode creates three files. Back up all three atomically using the SQLite backup API or by stopping the server first:

# Online backup (preferred — no downtime):
sqlite3 /srv/mcias/mcias.db ".backup /path/to/backup/mcias-$(date +%F).db"

# Offline backup:
systemctl stop mcias
cp /srv/mcias/mcias.db /path/to/backup/mcias-$(date +%F).db
systemctl start mcias

Store backups alongside a copy of the master key passphrase in a secure offline location. A database backup without the passphrase is unrecoverable.

---

Audit Log

CONF="--config /srv/mcias/mcias.toml"

# Show last 50 audit events
mciasdb $CONF audit tail --n 50

# Query by account
mciasdb $CONF audit query --account <UUID>

# Query by event type since a given time
mciasdb $CONF audit query --type login_failure --since 2026-01-01T00:00:00Z

# Output as JSON (for log shipping)
mciasdb $CONF audit query --json

---

Upgrading

1. Build the new binaries: make build
2. Stop the service: systemctl stop mcias
3. Install new binaries: sh dist/install.sh
   • The script will not overwrite existing config files.
   • New example files are placed with a .new suffix for review.
4. Review any .new config files in /srv/mcias/ and merge changes manually.
5. Run schema migrations if required:

   mciasdb --config /srv/mcias/mcias.toml schema migrate
   

6. Start the service: systemctl start mcias
7. Verify: curl -k https://auth.example.com:8443/v1/health

---

Master Key Rotation

This operation is not yet automated. Until a rotation command is implemented, rotation requires a full re-encryption of the database. Contact the project maintainer for the current procedure.

---

TLS Certificate Renewal

Replace the certificate and key files, then restart the server:

# Generate or obtain new cert/key, then:
cp new-server.crt /srv/mcias/server.crt
cp new-server.key /srv/mcias/server.key
chmod 0640 /srv/mcias/server.key
chown mcias:mcias /srv/mcias/server.crt /srv/mcias/server.key
systemctl restart mcias

For Let's Encrypt with Certbot, add a deploy hook:

# /etc/letsencrypt/renewal-hooks/deploy/mcias.sh
#!/bin/sh
cp /etc/letsencrypt/live/auth.example.com/fullchain.pem /srv/mcias/server.crt
cp /etc/letsencrypt/live/auth.example.com/privkey.pem   /srv/mcias/server.key
chmod 0640 /srv/mcias/server.key
chown mcias:mcias /srv/mcias/server.crt /srv/mcias/server.key
systemctl restart mcias

---

Docker Deployment

make docker

mkdir -p /srv/mcias
cp dist/mcias.conf.docker.example /srv/mcias/mcias.toml
$EDITOR /srv/mcias/mcias.toml

# Place TLS cert and key under /srv/mcias/
# Set ownership so uid 10001 (container mcias user) can read them.
chown -R 10001:10001 /srv/mcias

docker run -d \
  --name mcias \
  -v /srv/mcias:/srv/mcias \
  -e MCIAS_MASTER_PASSPHRASE=your-passphrase \
  -p 8443:8443 \
  -p 9443:9443 \
  --restart unless-stopped \
  mcias:latest

See dist/mcias.conf.docker.example for the full annotated Docker config.

---

Troubleshooting

Server fails to start: "open database"

Check that /srv/mcias/ is writable by the mcias user:

ls -la /srv/mcias/
stat /srv/mcias/mcias.db   # if it already exists

Fix: chown mcias:mcias /srv/mcias

Server fails to start: "environment variable ... is not set"

The MCIAS_MASTER_PASSPHRASE env var is missing. Ensure /srv/mcias/env exists, is readable by the mcias user, and contains the correct variable:

grep MCIAS_MASTER_PASSPHRASE /srv/mcias/env

Also confirm the systemd unit loads it:

systemctl cat mcias | grep EnvironmentFile

Server fails to start: "decrypt signing key"

The master key passphrase has changed or is wrong. The passphrase must match the one used when the database was first initialized (the KDF salt is stored in the database). Restore the correct passphrase from your offline backup.

TLS errors in client connections

Verify the certificate is valid and covers the correct hostname:

openssl x509 -in /srv/mcias/server.crt -noout -text | grep -E "Subject|DNS"
openssl x509 -in /srv/mcias/server.crt -noout -dates

Database locked / WAL not cleaning up

Check for lingering mcias.db-wal and mcias.db-shm files after an unclean shutdown. These are safe to leave in place — SQLite will recover on next open. Do not delete them while the server is running.

Schema dirty after failed migration

mciasdb --config /srv/mcias/mcias.toml schema verify
mciasdb --config /srv/mcias/mcias.toml schema force --version N
mciasdb --config /srv/mcias/mcias.toml schema migrate

Replace N with the last successfully applied version number.

---

File Permissions Reference

Path	Mode	Owner
/srv/mcias/	0750	mcias:mcias
/srv/mcias/mcias.toml	0640	mcias:mcias
/srv/mcias/server.crt	0644	mcias:mcias
/srv/mcias/server.key	0640	mcias:mcias
/srv/mcias/mcias.db	0640	mcias:mcias
/srv/mcias/env	0640	mcias:mcias
/srv/mcias/master.key	0640	mcias:mcias

Verify permissions:

ls -la /srv/mcias/