mc/metacrypt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add MEK rotation, per-engine DEKs, and v2 ciphertext format (audit #6, #22)

Browse Source 
Implement a two-level key hierarchy: the MEK now wraps per-engine DEKs
stored in a new barrier_keys table, rather than encrypting all barrier
entries directly. A v2 ciphertext format (0x02) embeds the key ID so the
barrier can resolve which DEK to use on decryption. v1 ciphertext remains
supported for backward compatibility.

Key changes:
- crypto: EncryptV2/DecryptV2/ExtractKeyID for v2 ciphertext with key IDs
- barrier: key registry (CreateKey, RotateKey, ListKeys, MigrateToV2, ReWrapKeys)
- seal: RotateMEK re-wraps DEKs without re-encrypting data
- engine: Mount auto-creates per-engine DEK
- REST + gRPC: barrier/keys, barrier/rotate-mek, barrier/rotate-key, barrier/migrate
- proto: BarrierService (v1 + v2) with ListKeys, RotateMEK, RotateKey, Migrate
- db: migration v2 adds barrier_keys table

Also includes: security audit report, CSRF protection, engine design specs
(sshca, transit, user), path-bound AAD migration tool, policy engine
enhancements, and ARCHITECTURE.md updates.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Kyle Isom
2026-03-16 18:27:44 -07:00
parent ac4577f778
commit 64d921827e
44 changed files with 5184 additions and 90 deletions
Download Patch File Download Diff File Expand all files Collapse all files

353
engines/sshca.md Normal file
View File

@@ -0,0 +1,353 @@
# SSH CA Engine Implementation Plan
## Overview
The SSH CA engine signs SSH host and user certificates using Go's
`golang.org/x/crypto/ssh` package. It follows the same architecture as the CA
engine: a single CA key pair signs certificates directly (no intermediate
hierarchy, since SSH certificates are flat).
## Engine Type
`sshca` — registered constant already exists in `internal/engine/engine.go`.
## Mount Configuration
Passed as `config` at mount time:
| Field | Default | Description |
|-----------------|------------------|------------------------------------------|
| `key_algorithm` | `"ed25519"` | CA key type: ed25519, ecdsa, rsa |
| `key_size` | `0` | Key size (ignored for ed25519; 256/384/521 for ECDSA, 2048/4096 for RSA) |
| `max_ttl` | `"87600h"` | Maximum certificate validity |
| `default_ttl` | `"24h"` | Default certificate validity |
## Barrier Storage Layout
```
engine/sshca/{mount}/config.json Engine configuration
engine/sshca/{mount}/ca/key.pem CA private key (PEM, PKCS8)
engine/sshca/{mount}/ca/pubkey.pub CA public key (SSH authorized_keys format)
engine/sshca/{mount}/profiles/{name}.json Signing profiles
engine/sshca/{mount}/certs/{serial}.json Signed cert records
engine/sshca/{mount}/krl.bin Current KRL (OpenSSH format)
```
## In-Memory State
```go
type SSHCAEngine struct {
barrier barrier.Barrier
config *SSHCAConfig
caKey crypto.PrivateKey // CA signing key
caSigner ssh.Signer // ssh.Signer wrapping caKey
mountPath string
mu sync.RWMutex
}
```
Key material (`caKey`, `caSigner`) is zeroized on `Seal()`.
## Lifecycle
### Initialize
1. Parse and store config in barrier as `config.json`.
2. Generate CA key pair using the configured algorithm.
3. Store private key PEM and SSH public key in barrier.
4. Load key into memory as `ssh.Signer`.
### Unseal
1. Load config from barrier.
2. Load CA private key from barrier, parse into `crypto.PrivateKey`.
3. Wrap as `ssh.Signer`.
### Seal
1. Zeroize `caKey` (same `zeroizeKey` helper used by CA engine).
2. Nil out `caSigner`, `config`.
## Operations
| Operation | Auth Required | Description |
|-------------------|---------------|-----------------------------------------------------|
| `get-ca-pubkey` | None | Return CA public key in SSH authorized_keys format |
| `sign-host` | User+Policy | Sign an SSH host certificate |
| `sign-user` | User+Policy | Sign an SSH user certificate |
| `create-profile` | Admin | Create a signing profile |
| `update-profile` | Admin | Update a signing profile |
| `get-profile` | User/Admin | Get signing profile details |
| `list-profiles` | User/Admin | List signing profiles |
| `delete-profile` | Admin | Delete a signing profile |
| `get-cert` | User/Admin | Get cert record by serial |
| `list-certs` | User/Admin | List issued cert summaries |
| `revoke-cert` | Admin | Revoke a certificate (soft flag) |
| `delete-cert` | Admin | Delete a certificate record |
### sign-host
Request data:
| Field | Required | Description |
|---------------|----------|------------------------------------------------|
| `public_key` | Yes | SSH public key to sign (authorized_keys format) |
| `hostnames` | Yes | Valid principals (hostnames) |
| `ttl` | No | Validity duration (default: `default_ttl`) |
| `extensions` | No | Map of extensions to include |
Flow:
1. Authenticate caller (`IsUser()`); admins bypass policy/ownership checks.
2. Parse the supplied SSH public key.
3. Generate a 64-bit serial using `crypto/rand`.
4. Build `ssh.Certificate` with `CertType: ssh.HostCert`, principals, validity, serial.
5. Policy check: `sshca/{mount}/id/{hostname}` for each principal, with ownership
rules (same as CA engine — hostname not held by another user's active cert).
6. Sign with `caSigner`.
7. Store `CertRecord` in barrier (certificate bytes, metadata; **no private key**).
8. Return signed certificate in OpenSSH format.
### sign-user
Request data:
| Field | Required | Description |
|------------------|----------|------------------------------------------------|
| `public_key` | Yes | SSH public key to sign (authorized_keys format) |
| `principals` | Yes | Valid usernames/principals |
| `ttl` | No | Validity duration (default: `default_ttl`) |
| `profile` | No | Signing profile name (see below) |
| `extensions` | No | Map of extensions (e.g. `permit-pty`) |
Critical options are **not accepted directly** in the sign request. They can
only be applied via a signing profile. This prevents unprivileged users from
setting security-sensitive options like `force-command` or `source-address`.
Flow:
1. Authenticate caller (`IsUser()`); admins bypass.
2. Parse the supplied SSH public key.
3. If `profile` is specified, load the signing profile and check policy
(`sshca/{mount}/profile/{profile_name}`, action `read`). Merge the
profile's critical options and extensions into the certificate. Any
extensions in the request are merged with profile extensions; conflicts
are resolved in favor of the profile.
4. Generate a 64-bit serial using `crypto/rand`.
5. Build `ssh.Certificate` with `CertType: ssh.UserCert`, principals, validity, serial.
6. If the profile specifies `max_ttl`, enforce it (cap the requested TTL).
7. Policy check: `sshca/{mount}/id/{principal}` for each principal.
Default rule: a user can only sign certs for their own username as principal,
unless a policy grants access to other principals.
8. Sign with `caSigner`.
9. Store `CertRecord` in barrier (includes profile name if used).
10. Return signed certificate in OpenSSH format.
### Signing Profiles
A signing profile is a named, admin-defined template that controls what goes
into a signed user certificate. Profiles are the only way to set critical
options, and access to each profile is policy-gated.
#### Profile Configuration
```go
type SigningProfile struct {
Name string `json:"name"`
CriticalOptions map[string]string `json:"critical_options"` // e.g. {"force-command": "/usr/bin/rsync", "source-address": "10.0.0.0/8"}
Extensions map[string]string `json:"extensions"` // merged with request extensions
MaxTTL string `json:"max_ttl,omitempty"` // overrides engine max_ttl if shorter
AllowedPrincipals []string `json:"allowed_principals,omitempty"` // if set, restricts principals
}
```
#### Storage
```
engine/sshca/{mount}/profiles/{name}.json
```
#### Operations
| Operation | Auth Required | Description |
|------------------|---------------|------------------------------------------|
| `create-profile` | Admin | Create a signing profile |
| `update-profile` | Admin | Update a signing profile |
| `get-profile` | User/Admin | Get profile details |
| `list-profiles` | User/Admin | List available profiles |
| `delete-profile` | Admin | Delete a signing profile |
#### Policy Gating
Access to a profile is controlled via policy on resource
`sshca/{mount}/profile/{profile_name}`, action `read`. A user must have
policy access to both the profile and the requested principals to sign
a certificate using that profile.
Example use cases:
- **`restricted-sftp`**: `force-command: "internal-sftp"`, `source-address: "10.0.0.0/8"` — grants users SFTP-only access from internal networks.
- **`deploy`**: `force-command: "/usr/local/bin/deploy"`, `source-address: "10.0.1.0/24"` — CI/CD deploy key with restricted command.
- **`unrestricted`**: empty critical options — for trusted users who need full shell access (admin-only policy).
## CertRecord
```go
type CertRecord struct {
Serial uint64 `json:"serial"`
CertType string `json:"cert_type"` // "host" or "user"
Principals []string `json:"principals"`
CertData string `json:"cert_data"` // OpenSSH format
IssuedBy string `json:"issued_by"`
IssuedAt time.Time `json:"issued_at"`
ExpiresAt time.Time `json:"expires_at"`
Revoked bool `json:"revoked,omitempty"`
RevokedAt time.Time `json:"revoked_at,omitempty"`
RevokedBy string `json:"revoked_by,omitempty"`
}
```
## Key Revocation List (KRL)
SSH servers cannot query Metacrypt in real time to check whether a certificate
has been revoked. Instead, the SSH CA engine generates an OpenSSH-format KRL
(Key Revocation List) that SSH servers fetch periodically and reference via
`RevokedKeys` in `sshd_config`.
### KRL Generation
The engine maintains a KRL in memory, rebuilt whenever a certificate is revoked
or deleted. The KRL is a binary blob in OpenSSH KRL format
(`golang.org/x/crypto/ssh` provides marshalling helpers), containing:
- **Serial revocations**: Revoked certificate serial numbers, keyed to the CA
public key. This is the most compact representation.
- **KRL version**: Monotonically increasing counter, incremented on each
rebuild. SSH servers can use this to detect stale KRLs.
- **Generated-at timestamp**: Included in the KRL for freshness checking.
The KRL is stored in the barrier at `engine/sshca/{mount}/krl.bin` and cached
in memory. It is rebuilt on:
- `revoke-cert` — adds the serial to the KRL.
- `delete-cert` — if the cert was revoked, the KRL is regenerated from all
remaining revoked certs.
- Engine unseal — loaded from barrier into memory.
### Distribution
KRL distribution is a pull model. SSH servers fetch the current KRL via an
unauthenticated endpoint (analogous to the public CA key endpoint):
| Method | Path | Description |
|--------|-------------------------------------|--------------------------------|
| GET | `/v1/sshca/{mount}/krl` | Current KRL (binary, OpenSSH format) |
The response includes:
- `Content-Type: application/octet-stream`
- `ETag` header derived from the KRL version, enabling conditional fetches.
- `Cache-Control: max-age=60` to encourage periodic refresh without
overwhelming the server.
SSH servers should be configured to fetch the KRL on a cron schedule (e.g.
every 1–5 minutes) and write it to a local file referenced by `sshd_config`:
```
RevokedKeys /etc/ssh/metacrypt_krl
```
A helper script or systemd timer can fetch the KRL:
```bash
curl -s -o /etc/ssh/metacrypt_krl \
https://metacrypt.example.com:8443/v1/sshca/ssh/krl
```
### Operations
| Operation | Auth Required | Description |
|------------|---------------|----------------------------------------------|
| `get-krl` | None | Return the current KRL in OpenSSH format |
## gRPC Service (proto/metacrypt/v2/sshca.proto)
```protobuf
service SSHCAService {
rpc GetCAPublicKey(GetCAPublicKeyRequest) returns (GetCAPublicKeyResponse);
rpc SignHost(SignHostRequest) returns (SignHostResponse);
rpc SignUser(SignUserRequest) returns (SignUserResponse);
rpc CreateProfile(CreateProfileRequest) returns (CreateProfileResponse);
rpc UpdateProfile(UpdateProfileRequest) returns (UpdateProfileResponse);
rpc GetProfile(GetProfileRequest) returns (GetProfileResponse);
rpc ListProfiles(ListProfilesRequest) returns (ListProfilesResponse);
rpc DeleteProfile(DeleteProfileRequest) returns (DeleteProfileResponse);
rpc GetCert(SSHGetCertRequest) returns (SSHGetCertResponse);
rpc ListCerts(SSHListCertsRequest) returns (SSHListCertsResponse);
rpc RevokeCert(SSHRevokeCertRequest) returns (SSHRevokeCertResponse);
rpc DeleteCert(SSHDeleteCertRequest) returns (SSHDeleteCertResponse);
rpc GetKRL(GetKRLRequest) returns (GetKRLResponse);
}
```
## REST Endpoints
Public (unseal required, no auth):
| Method | Path | Description |
|--------|-------------------------------------|--------------------------------|
| GET | `/v1/sshca/{mount}/ca` | CA public key (SSH format) |
| GET | `/v1/sshca/{mount}/krl` | Current KRL (OpenSSH format) |
Typed endpoints (auth required):
| Method | Path | Description |
|--------|--------------------------------------------|----------------------|
| POST | `/v1/sshca/{mount}/sign-host` | Sign host cert |
| POST | `/v1/sshca/{mount}/sign-user` | Sign user cert |
| POST | `/v1/sshca/{mount}/profiles` | Create profile |
| GET | `/v1/sshca/{mount}/profiles` | List profiles |
| GET | `/v1/sshca/{mount}/profiles/{name}` | Get profile |
| PUT | `/v1/sshca/{mount}/profiles/{name}` | Update profile |
| DELETE | `/v1/sshca/{mount}/profiles/{name}` | Delete profile |
| GET | `/v1/sshca/{mount}/cert/{serial}` | Get cert record |
| POST | `/v1/sshca/{mount}/cert/{serial}/revoke` | Revoke cert |
| DELETE | `/v1/sshca/{mount}/cert/{serial}` | Delete cert record |
All operations are also accessible via the generic `POST /v1/engine/request`.
## Web UI
Add to `/dashboard` the ability to mount an SSH CA engine.
Add an `/sshca` page (or section on the existing PKI page) displaying:
- CA public key (for `TrustedUserCAKeys` / `@cert-authority` lines)
- Sign host/user certificate form
- Certificate list with detail view
## Implementation Steps
1. **`internal/engine/sshca/`** — Implement `SSHCAEngine` (types, lifecycle,
operations). Reuse `zeroizeKey` from `internal/engine/ca/` (move to shared
helper or duplicate).
2. **Register factory** in `cmd/metacrypt/main.go`:
`registry.RegisterFactory(engine.EngineTypeSSHCA, sshca.NewSSHCAEngine)`.
3. **Proto definitions** — `proto/metacrypt/v2/sshca.proto`, run `make proto`.
4. **gRPC handlers** — `internal/grpcserver/sshca.go`.
5. **REST routes** — Add to `internal/server/routes.go`.
6. **Web UI** — Add template + webserver routes.
7. **Tests** — Unit tests with in-memory barrier following the CA test pattern.
## Dependencies
- `golang.org/x/crypto/ssh` (already in `go.mod` via transitive deps)
## Security Considerations
- CA private key encrypted at rest in barrier, zeroized on seal.
- Signed certificates do not contain private keys.
- Serial numbers are always generated server-side using `crypto/rand` (64-bit);
user-provided serials are not accepted.
- `max_ttl` is enforced server-side; the engine rejects TTL values exceeding it.
- User cert signing defaults to allowing only the caller's own username as
principal, preventing privilege escalation.
- Critical options (`force-command`, `source-address`, etc.) are only settable
via admin-defined signing profiles, never directly in the sign request. This
prevents unprivileged users from bypassing `sshd_config` restrictions.
- Profile access is policy-gated: a user must have policy access to
`sshca/{mount}/profile/{name}` to use a profile.

468
engines/transit.md Normal file
View File

@@ -0,0 +1,468 @@
# Transit Engine Implementation Plan
## Overview
The transit engine provides encryption-as-a-service: applications send plaintext
to Metacrypt and receive ciphertext (or vice versa), without ever handling raw
encryption keys. This enables envelope encryption, key rotation, and centralized
key management.
The design is inspired by HashiCorp Vault's transit secrets engine.
## Engine Type
`transit` — registered constant already exists in `internal/engine/engine.go`.
## Mount Configuration
Passed as `config` at mount time:
| Field | Default | Description |
|--------------------|---------|------------------------------------------------|
| `max_key_versions` | `0` | Maximum key versions to retain (0 = unlimited) |
No engine-wide key algorithm is configured; each named key specifies its own.
## Core Concepts
### Named Keys
The transit engine manages **named encryption keys**. Each key has:
- A unique name (e.g. `"payments"`, `"session-tokens"`)
- A key type (symmetric or asymmetric)
- One or more **versions** (for key rotation)
- Policy flags (exportable, allow-deletion)
### Key Types
| Type | Algorithm | Operations |
|-----------------|-------------------|------------------|
| `aes256-gcm` | AES-256-GCM | Encrypt, Decrypt |
| `chacha20-poly` | ChaCha20-Poly1305 | Encrypt, Decrypt |
| `ed25519` | Ed25519 | Sign, Verify |
| `ecdsa-p256` | ECDSA P-256 | Sign, Verify |
| `ecdsa-p384` | ECDSA P-384 | Sign, Verify |
| `hmac-sha256` | HMAC-SHA256 | HMAC |
| `hmac-sha512` | HMAC-SHA512 | HMAC |
RSA key types are intentionally excluded. The transit engine is not the right
place for RSA — asymmetric encryption belongs in the user engine (via ECDH),
and RSA signing offers no advantage over Ed25519/ECDSA for this use case.
### Key Rotation
Each key has a current version and may retain older versions. Encryption always
uses the latest version. Decryption selects the version from the ciphertext
header.
Each key tracks a `min_decryption_version` (default 1). Decryption requests
for ciphertext encrypted with a version below this minimum are rejected. This
lets operators complete a rotation cycle:
1. Rotate the key (creates version N+1).
2. Rewrap all existing ciphertext to the latest version.
3. Set `min_decryption_version` to N+1.
4. Old key versions at or below the minimum can then be pruned via
`max_key_versions` or `trim-key`.
Until `min_decryption_version` is advanced, old versions must be retained.
### Ciphertext Format
Transit ciphertexts use a versioned prefix:
```
metacrypt:v{version}:{base64(nonce + ciphertext + tag)}
```
The `v{version}` identifies which key version to use for decryption.
## Barrier Storage Layout
```
engine/transit/{mount}/config.json Engine configuration
engine/transit/{mount}/keys/{name}/config.json Key configuration + policy
engine/transit/{mount}/keys/{name}/v{N}.key Key material for version N
```
## In-Memory State
```go
type TransitEngine struct {
barrier barrier.Barrier
config *TransitConfig
keys map[string]*keyState // loaded named keys
mountPath string
mu sync.RWMutex
}
type keyState struct {
config *KeyConfig
versions map[int]*keyVersion
minDecryptionVersion int // reject decrypt for versions below this
}
type keyVersion struct {
version int
key []byte // symmetric key material
privKey crypto.PrivateKey // asymmetric private key (nil for symmetric)
pubKey crypto.PublicKey // asymmetric public key (nil for symmetric)
}
```
## Lifecycle
### Initialize
1. Parse and store config in barrier.
2. No keys are created at init time (keys are created on demand).
### Unseal
1. Load config from barrier.
2. Discover and load all named keys and their versions from the barrier.
### Seal
1. Zeroize all key material (symmetric keys overwritten with zeros,
asymmetric keys via `zeroizeKey`).
2. Nil out all maps.
## Operations
| Operation | Auth Required | Description |
|------------------|---------------|-----------------------------------------------|
| `create-key` | Admin | Create a new named key |
| `delete-key` | Admin | Delete a named key (if `allow_deletion` set) |
| `get-key` | User/Admin | Get key metadata (no raw material) |
| `list-keys` | User/Admin | List named keys |
| `rotate-key` | Admin | Create a new version of a named key |
| `update-key-config` | Admin | Update mutable key config (e.g. `min_decryption_version`) |
| `trim-key` | Admin | Delete versions older than `min_decryption_version` |
| `encrypt` | User+Policy | Encrypt plaintext with a named key |
| `decrypt` | User+Policy | Decrypt ciphertext with a named key |
| `rewrap` | User+Policy | Re-encrypt ciphertext with the latest key version |
| `batch-encrypt` | User+Policy | Encrypt multiple plaintexts with a named key |
| `batch-decrypt` | User+Policy | Decrypt multiple ciphertexts with a named key |
| `batch-rewrap` | User+Policy | Re-encrypt multiple ciphertexts with latest version |
| `sign` | User+Policy | Sign data with an asymmetric key (Ed25519, ECDSA) |
| `verify` | User+Policy | Verify an asymmetric signature |
| `hmac` | User+Policy | Compute HMAC with an HMAC key |
| `get-public-key` | User/Admin | Get public key for asymmetric keys |
### create-key
Request data:
| Field | Required | Default | Description |
|-------------------|----------|----------------|----------------------------------|
| `name` | Yes | | Key name |
| `type` | Yes | | Key type (see table above) |
| `exportable` | No | `false` | Whether raw key material can be exported |
| `allow_deletion` | No | `false` | Whether key can be deleted |
The key is created at version 1 with `min_decryption_version` = 1.
### encrypt
Request data:
| Field | Required | Description |
|-------------|----------|---------------------------------------------------|
| `key` | Yes | Named key to use |
| `plaintext` | Yes | Base64-encoded plaintext |
| `context` | No | Base64-encoded context for AEAD additional data |
Response: `{ "ciphertext": "metacrypt:v1:..." }`
### decrypt
Request data:
| Field | Required | Description |
|--------------|----------|---------------------------------------------------|
| `key` | Yes | Named key to use |
| `ciphertext` | Yes | Ciphertext string from encrypt |
| `context` | No | Base64-encoded context (must match encrypt context) |
Response: `{ "plaintext": "<base64>" }`
### sign
Asymmetric keys only (Ed25519, ECDSA). HMAC keys must use the `hmac` operation
instead — HMAC is a MAC, not a digital signature, and does not provide
non-repudiation.
Request data:
| Field | Required | Description |
|-------------|----------|--------------------------------------------|
| `key` | Yes | Named key (Ed25519 or ECDSA type) |
| `input` | Yes | Base64-encoded data to sign |
| `algorithm` | No | Hash algorithm (default varies by key type) |
The engine rejects `sign` requests for HMAC key types with an error.
Response: `{ "signature": "metacrypt:v1:..." }`
### verify
Asymmetric keys only. Rejects HMAC key types (use `hmac` to recompute and
compare instead).
Request data:
| Field | Required | Description |
|-------------|----------|--------------------------------------------|
| `key` | Yes | Named key (Ed25519 or ECDSA type) |
| `input` | Yes | Base64-encoded original data |
| `signature` | Yes | Signature string from sign |
Response: `{ "valid": true }`
### update-key-config
Admin-only. Updates mutable key configuration fields.
Request data:
| Field | Required | Description |
|--------------------------|----------|------------------------------------------|
| `key` | Yes | Named key |
| `min_decryption_version` | No | Minimum version allowed for decryption |
`min_decryption_version` can only be increased, never decreased. It cannot
exceed the current version (you must always be able to decrypt with the latest).
### trim-key
Admin-only. Permanently deletes key versions older than `min_decryption_version`.
This is irreversible — ciphertext encrypted with trimmed versions can never be
decrypted.
Request data:
| Field | Required | Description |
|-------|----------|-------------|
| `key` | Yes | Named key |
Response: `{ "trimmed_versions": [1, 2, ...] }`
## Batch Operations
The transit engine supports batch variants of `encrypt`, `decrypt`, and
`rewrap` for high-throughput use cases (e.g. encrypting many database fields,
re-encrypting after key rotation). Without batch support, callers are pushed
toward caching keys locally, defeating the purpose of transit encryption.
### Design
Each batch request targets a **single named key** with an array of items.
Results are returned in the same order. Errors are **per-item** (partial
success model) — a single bad ciphertext does not fail the entire batch.
Single-key-per-batch simplifies authorization: one policy check per batch
request rather than per item. Callers needing multiple keys issue multiple
batch requests.
### batch-encrypt
Request data:
| Field | Required | Description |
|---------|----------|---------------------------------------------------|
| `key` | Yes | Named key to use |
| `items` | Yes | Array of encrypt items (see below) |
Each item:
| Field | Required | Description |
|-------------|----------|-----------------------------------------------|
| `plaintext` | Yes | Base64-encoded plaintext |
| `context` | No | Base64-encoded context for AEAD additional data |
| `reference` | No | Caller-defined reference string (echoed back) |
Response: `{ "results": [...] }`
Each result:
| Field | Description |
|--------------|------------------------------------------------------|
| `ciphertext` | `"metacrypt:v1:..."` on success, empty on error |
| `reference` | Echoed from the request item (if provided) |
| `error` | Error message on failure, empty on success |
### batch-decrypt
Request data:
| Field | Required | Description |
|---------|----------|---------------------------------------------------|
| `key` | Yes | Named key to use |
| `items` | Yes | Array of decrypt items (see below) |
Each item:
| Field | Required | Description |
|--------------|----------|-----------------------------------------------|
| `ciphertext` | Yes | Ciphertext string from encrypt |
| `context` | No | Base64-encoded context (must match encrypt) |
| `reference` | No | Caller-defined reference string (echoed back) |
Response: `{ "results": [...] }`
Each result:
| Field | Description |
|-------------|------------------------------------------------------|
| `plaintext` | Base64-encoded plaintext on success, empty on error |
| `reference` | Echoed from the request item (if provided) |
| `error` | Error message on failure, empty on success |
### batch-rewrap
Request data:
| Field | Required | Description |
|---------|----------|---------------------------------------------------|
| `key` | Yes | Named key to use |
| `items` | Yes | Array of rewrap items (see below) |
Each item:
| Field | Required | Description |
|--------------|----------|-----------------------------------------------|
| `ciphertext` | Yes | Ciphertext to re-encrypt with latest version |
| `context` | No | Base64-encoded context (must match original) |
| `reference` | No | Caller-defined reference string (echoed back) |
Response: `{ "results": [...] }`
Each result:
| Field | Description |
|--------------|------------------------------------------------------|
| `ciphertext` | Re-encrypted ciphertext on success, empty on error |
| `reference` | Echoed from the request item (if provided) |
| `error` | Error message on failure, empty on success |
### Implementation Notes
Batch operations are handled inside the transit engine's `HandleRequest` as
three additional operation cases (`batch-encrypt`, `batch-decrypt`,
`batch-rewrap`). No changes to the `Engine` interface are needed. The engine
loops over items internally, loading the key once and reusing it for all items
in the batch.
The `reference` field is opaque to the engine — it allows callers to correlate
results with their source records (e.g. a database row ID) without maintaining
positional tracking.
## Authorization
Follows the same model as the CA engine:
- **Admins**: grant-all for all operations.
- **Users**: can encrypt/decrypt/sign/verify/hmac if policy allows.
- **Policy resources**: `transit/{mount}/key/{key_name}` with granular actions:
`encrypt`, `decrypt`, `sign`, `verify`, `hmac` for cryptographic operations;
`read` for metadata (get-key, list-keys, get-public-key); `write` for
management (create-key, delete-key, rotate-key, update-key-config, trim-key).
The `any` action matches all of the above (but never `admin`).
- No ownership concept (transit keys are shared resources); access is purely
policy-based.
## gRPC Service (proto/metacrypt/v2/transit.proto)
```protobuf
service TransitService {
rpc CreateKey(CreateTransitKeyRequest) returns (CreateTransitKeyResponse);
rpc DeleteKey(DeleteTransitKeyRequest) returns (DeleteTransitKeyResponse);
rpc GetKey(GetTransitKeyRequest) returns (GetTransitKeyResponse);
rpc ListKeys(ListTransitKeysRequest) returns (ListTransitKeysResponse);
rpc RotateKey(RotateTransitKeyRequest) returns (RotateTransitKeyResponse);
rpc UpdateKeyConfig(UpdateTransitKeyConfigRequest) returns (UpdateTransitKeyConfigResponse);
rpc TrimKey(TrimTransitKeyRequest) returns (TrimTransitKeyResponse);
rpc Encrypt(TransitEncryptRequest) returns (TransitEncryptResponse);
rpc Decrypt(TransitDecryptRequest) returns (TransitDecryptResponse);
rpc Rewrap(TransitRewrapRequest) returns (TransitRewrapResponse);
rpc BatchEncrypt(BatchTransitEncryptRequest) returns (BatchTransitEncryptResponse);
rpc BatchDecrypt(BatchTransitDecryptRequest) returns (BatchTransitDecryptResponse);
rpc BatchRewrap(BatchTransitRewrapRequest) returns (BatchTransitRewrapResponse);
rpc Sign(TransitSignRequest) returns (TransitSignResponse);
rpc Verify(TransitVerifyRequest) returns (TransitVerifyResponse);
rpc Hmac(TransitHmacRequest) returns (TransitHmacResponse);
rpc GetPublicKey(GetTransitPublicKeyRequest) returns (GetTransitPublicKeyResponse);
}
```
## REST Endpoints
All auth required:
| Method | Path | Description |
|--------|---------------------------------------------|--------------------|
| POST | `/v1/transit/{mount}/keys` | Create key |
| GET | `/v1/transit/{mount}/keys` | List keys |
| GET | `/v1/transit/{mount}/keys/{name}` | Get key metadata |
| DELETE | `/v1/transit/{mount}/keys/{name}` | Delete key |
| POST | `/v1/transit/{mount}/keys/{name}/rotate` | Rotate key |
| PATCH | `/v1/transit/{mount}/keys/{name}/config` | Update key config |
| POST | `/v1/transit/{mount}/keys/{name}/trim` | Trim old versions |
| POST | `/v1/transit/{mount}/encrypt/{key}` | Encrypt |
| POST | `/v1/transit/{mount}/decrypt/{key}` | Decrypt |
| POST | `/v1/transit/{mount}/rewrap/{key}` | Rewrap |
| POST | `/v1/transit/{mount}/batch/encrypt/{key}` | Batch encrypt |
| POST | `/v1/transit/{mount}/batch/decrypt/{key}` | Batch decrypt |
| POST | `/v1/transit/{mount}/batch/rewrap/{key}` | Batch rewrap |
| POST | `/v1/transit/{mount}/sign/{key}` | Sign |
| POST | `/v1/transit/{mount}/verify/{key}` | Verify |
| POST | `/v1/transit/{mount}/hmac/{key}` | HMAC |
All operations are also accessible via the generic `POST /v1/engine/request`.
## Web UI
Add to `/dashboard` the ability to mount a transit engine.
Add a `/transit` page displaying:
- Named key list with metadata (type, version, created, exportable)
- Key detail view with version history
- Encrypt/decrypt form for interactive testing
- Key rotation button (admin)
## Implementation Steps
1. **`internal/engine/transit/`** — Implement `TransitEngine`:
- `types.go` — Config, KeyConfig, key version types.
- `transit.go` — Lifecycle (Initialize, Unseal, Seal, HandleRequest).
- `encrypt.go` — Encrypt/Decrypt/Rewrap operations.
- `sign.go` — Sign/Verify/HMAC operations.
- `keys.go` — Key management (create, delete, rotate, list, get).
2. **Register factory** in `cmd/metacrypt/main.go`.
3. **Proto definitions** — `proto/metacrypt/v2/transit.proto`, run `make proto`.
4. **gRPC handlers** — `internal/grpcserver/transit.go`.
5. **REST routes** — Add to `internal/server/routes.go`.
6. **Web UI** — Add template + webserver routes.
7. **Tests** — Unit tests for each operation, key rotation, rewrap correctness.
## Dependencies
- `golang.org/x/crypto/chacha20poly1305` (for ChaCha20-Poly1305 key type)
- Standard library `crypto/aes`, `crypto/cipher`, `crypto/ecdsa`,
`crypto/ed25519`, `crypto/hmac`, `crypto/sha256`, `crypto/sha512`
## Security Considerations
- All key material encrypted at rest in the barrier, zeroized on seal.
- Symmetric keys generated with `crypto/rand`.
- Ciphertext format includes version to support key rotation without data loss.
- `exportable` flag is immutable after creation — cannot be enabled later.
- `allow_deletion` is immutable after creation.
- `max_key_versions` pruning only removes old versions, never the current one.
- Rewrap operation never exposes plaintext to the caller.
- Context (AAD) binding prevents ciphertext from being used in a different context.
- `min_decryption_version` enforces key rotation completion: once advanced,
old versions are unusable for decryption and can be permanently trimmed.
- RSA key types are excluded to avoid padding scheme vulnerabilities
(Bleichenbacher attacks on PKCS#1 v1.5). Asymmetric encryption belongs in
the user engine; signing uses Ed25519/ECDSA.

302
engines/user.md Normal file
View File

@@ -0,0 +1,302 @@
# User Engine Implementation Plan
## Overview
The user engine provides end-to-end encryption between Metacircular platform
users. Each user has a key pair managed by Metacrypt; the service handles key
exchange, encryption, and decryption so that messages/data are encrypted at
rest and only readable by the intended recipients.
The design uses hybrid encryption: an asymmetric key pair per user for key
exchange, combined with symmetric encryption for message payloads. This enables
multi-recipient encryption without sharing symmetric keys directly.
## Engine Type
`user` — registered constant already exists in `internal/engine/engine.go`.
## Mount Configuration
Passed as `config` at mount time:
| Field | Default | Description |
|-----------------|-------------|------------------------------------------|
| `key_algorithm` | `"x25519"` | Key exchange algorithm: x25519, ecdh-p256, ecdh-p384 |
| `sym_algorithm` | `"aes256-gcm"` | Symmetric algorithm for message encryption |
## Trust Model
Server-trust: Metacrypt holds all private keys in the barrier, same as every
other engine. Access control is enforced at the application layer — no API
surface exports private keys, and only the engine accesses them internally
during encrypt/decrypt operations. An operator with barrier access could
theoretically extract keys, which is accepted and consistent with the barrier
trust model used throughout Metacrypt.
## Core Concepts
### User Provisioning
Any MCIAS user can have a keypair, whether or not they have ever logged in to
Metacrypt. Keypairs are created in three ways:
1. **Self-registration** — an authenticated user calls `register`.
2. **Admin provisioning** — an admin calls `provision` with a username. The
user does not need to have logged in.
3. **Auto-provisioning on encrypt** — when a sender encrypts to a recipient
who has no keypair, the engine generates one automatically. The recipient
can decrypt when they eventually authenticate.
### User Key Pairs
Each provisioned user has a key exchange key pair. The private key is stored
encrypted in the barrier and is only used by the engine on behalf of the owning
user (enforced in `HandleRequest`). The public key is available to any
authenticated user (needed to encrypt messages to that user).
### Encryption Flow (Sender → Recipient)
1. Sender calls `encrypt` with plaintext, recipient username(s), and optional
metadata.
2. Engine generates a random symmetric data encryption key (DEK).
3. Engine encrypts the plaintext with the DEK.
4. For each recipient: engine performs key agreement (sender private key +
recipient public key → shared secret), derives a wrapping key via HKDF,
and wraps the DEK.
5. Returns an envelope containing ciphertext + per-recipient wrapped DEKs.
### Decryption Flow
1. Recipient calls `decrypt` with the envelope.
2. Engine finds the recipient's wrapped DEK entry.
3. Engine performs key agreement (recipient private key + sender public key →
shared secret), derives the wrapping key, unwraps the DEK.
4. Engine decrypts the ciphertext with the DEK.
5. Returns plaintext.
### Envelope Format
```json
{
"version": 1,
"sender": "alice",
"sym_algorithm": "aes256-gcm",
"ciphertext": "<base64(nonce + encrypted_payload + tag)>",
"recipients": {
"bob": "<base64(wrapped_dek)>",
"carol": "<base64(wrapped_dek)>"
}
}
```
The envelope is base64-encoded as a single opaque blob for transport.
## Barrier Storage Layout
```
engine/user/{mount}/config.json Engine configuration
engine/user/{mount}/users/{username}/priv.pem User private key
engine/user/{mount}/users/{username}/pub.pem User public key
engine/user/{mount}/users/{username}/config.json Per-user metadata
```
## In-Memory State
```go
type UserEngine struct {
barrier barrier.Barrier
config *UserConfig
users map[string]*userState
mountPath string
mu sync.RWMutex
}
type userState struct {
privKey crypto.PrivateKey // key exchange private key
pubKey crypto.PublicKey // key exchange public key
config *UserKeyConfig
}
```
## Lifecycle
### Initialize
1. Parse and store config in barrier.
2. No user keys are created at init time (created on demand or via `register`).
### Unseal
1. Load config from barrier.
2. Discover and load all user key pairs from barrier.
### Seal
1. Zeroize all private key material.
2. Nil out all maps.
## Operations
| Operation | Auth Required | Description |
|------------------|---------------|-------------------------------------------------|
| `register` | User (self) | Create a key pair for the authenticated user |
| `provision` | Admin | Create a key pair for any MCIAS user by username |
| `get-public-key` | User/Admin | Get any user's public key |
| `list-users` | User/Admin | List registered users |
| `encrypt` | User+Policy | Encrypt data for one or more recipients |
| `decrypt` | User (self) | Decrypt an envelope addressed to the caller |
| `rotate-key` | User (self) | Rotate the caller's key pair |
| `delete-user` | Admin | Remove a user's key pair |
### register
Creates a key pair for the authenticated caller. No-op if the caller already
has a keypair (returns existing public key).
Request data: none (uses `CallerInfo.Username`).
Response: `{ "public_key": "<base64>" }`
### provision
Admin-only. Creates a key pair for the given username. The user does not need
to have logged in to Metacrypt. No-op if the user already has a keypair.
Request data: `{ "username": "<mcias_username>" }`
Response: `{ "public_key": "<base64>" }`
### encrypt
Request data:
| Field | Required | Description |
|--------------|----------|-------------------------------------------|
| `recipients` | Yes | List of usernames to encrypt for |
| `plaintext` | Yes | Base64-encoded plaintext |
| `metadata` | No | Arbitrary string metadata (authenticated) |
Flow:
1. Caller must be provisioned (has a key pair). Auto-provision if not.
2. For each recipient without a keypair: auto-provision them.
3. Load sender's private key and each recipient's public key.
4. Generate random DEK, encrypt plaintext with DEK.
5. For each recipient: ECDH(sender_priv, recipient_pub) → shared_secret,
HKDF(shared_secret, salt, info) → wrapping_key, AES-KeyWrap(wrapping_key,
DEK) → wrapped_dek.
6. Build and return envelope.
Authorization:
- Admins: grant-all.
- Users: can encrypt to any registered user by default.
- Policy can restrict which users a sender can encrypt to:
resource `user/{mount}/recipient/{username}`, action `write`.
### decrypt
Request data:
| Field | Required | Description |
|------------|----------|--------------------------------|
| `envelope` | Yes | Base64-encoded envelope blob |
Flow:
1. Parse envelope, find the caller's wrapped DEK entry.
2. Load sender's public key and caller's private key.
3. ECDH(caller_priv, sender_pub) → shared_secret → wrapping_key → DEK.
4. Decrypt ciphertext with DEK.
5. Return plaintext.
A user can only decrypt envelopes addressed to themselves.
### rotate-key
Generates a new key pair for the caller. The old private key is zeroized and
deleted. Old envelopes encrypted with the previous key cannot be decrypted
after rotation — callers should re-encrypt any stored data before rotating.
## gRPC Service (proto/metacrypt/v2/user.proto)
```protobuf
service UserService {
rpc Register(UserRegisterRequest) returns (UserRegisterResponse);
rpc Provision(UserProvisionRequest) returns (UserProvisionResponse);
rpc GetPublicKey(UserGetPublicKeyRequest) returns (UserGetPublicKeyResponse);
rpc ListUsers(UserListUsersRequest) returns (UserListUsersResponse);
rpc Encrypt(UserEncryptRequest) returns (UserEncryptResponse);
rpc Decrypt(UserDecryptRequest) returns (UserDecryptResponse);
rpc RotateKey(UserRotateKeyRequest) returns (UserRotateKeyResponse);
rpc DeleteUser(UserDeleteUserRequest) returns (UserDeleteUserResponse);
}
```
## REST Endpoints
All auth required:
| Method | Path | Description |
|--------|-----------------------------------------|------------------------|
| POST | `/v1/user/{mount}/register` | Register caller |
| POST | `/v1/user/{mount}/provision` | Provision user (admin) |
| GET | `/v1/user/{mount}/keys` | List registered users |
| GET | `/v1/user/{mount}/keys/{username}` | Get user's public key |
| DELETE | `/v1/user/{mount}/keys/{username}` | Delete user (admin) |
| POST | `/v1/user/{mount}/encrypt` | Encrypt for recipients |
| POST | `/v1/user/{mount}/decrypt` | Decrypt envelope |
| POST | `/v1/user/{mount}/rotate` | Rotate caller's key |
All operations are also accessible via the generic `POST /v1/engine/request`.
## Web UI
Add to `/dashboard` the ability to mount a user engine.
Add a `/user-crypto` page displaying:
- Registration status / register button
- Public key display
- Encrypt form (select recipients, enter message)
- Decrypt form (paste envelope)
- Key rotation button with warning
## Implementation Steps
1. **`internal/engine/user/`** — Implement `UserEngine`:
- `types.go` — Config types, envelope format.
- `user.go` — Lifecycle (Initialize, Unseal, Seal, HandleRequest).
- `crypto.go` — ECDH key agreement, HKDF derivation, DEK wrap/unwrap,
symmetric encrypt/decrypt.
- `keys.go` — User registration, key rotation, deletion.
2. **Register factory** in `cmd/metacrypt/main.go`.
3. **Proto definitions** — `proto/metacrypt/v2/user.proto`, run `make proto`.
4. **gRPC handlers** — `internal/grpcserver/user.go`.
5. **REST routes** — Add to `internal/server/routes.go`.
6. **Web UI** — Add template + webserver routes.
7. **Tests** — Unit tests: register, encrypt/decrypt roundtrip, multi-recipient,
key rotation invalidates old envelopes, authorization checks.
## Dependencies
- `golang.org/x/crypto/hkdf` (for key derivation from ECDH shared secret)
- `crypto/ecdh` (Go 1.20+, for X25519 and NIST curve key exchange)
- Standard library `crypto/aes`, `crypto/cipher`, `crypto/rand`
## Security Considerations
- Private keys encrypted at rest in the barrier, zeroized on seal.
- DEK is random per-encryption; never reused.
- HKDF derivation includes sender and recipient identities in the info string
to prevent key confusion attacks:
`info = "metacrypt-user-v1:" + sender + ":" + recipient`.
- Envelope includes sender identity so the recipient can derive the correct
shared secret.
- Key rotation is destructive — old data cannot be decrypted. The engine should
warn and require explicit confirmation (admin or self only).
- Server-trust model: the server holds all private keys in the barrier. No API
surface exports private keys. Access control is application-enforced — the
engine only uses a private key on behalf of its owner during encrypt/decrypt.
- Auto-provisioned users have keypairs waiting for them; their private keys are
protected identically to explicitly registered users.
- Metadata in the envelope is authenticated (included as additional data in
AEAD) but not encrypted — it is visible to anyone holding the envelope.
- Post-quantum readiness: the `key_algorithm` config supports future hybrid
schemes (e.g. X25519 + ML-KEM). The envelope version field enables migration.