Implement a two-level key hierarchy: the MEK now wraps per-engine DEKs stored in a new barrier_keys table, rather than encrypting all barrier entries directly. A v2 ciphertext format (0x02) embeds the key ID so the barrier can resolve which DEK to use on decryption. v1 ciphertext remains supported for backward compatibility. Key changes: - crypto: EncryptV2/DecryptV2/ExtractKeyID for v2 ciphertext with key IDs - barrier: key registry (CreateKey, RotateKey, ListKeys, MigrateToV2, ReWrapKeys) - seal: RotateMEK re-wraps DEKs without re-encrypting data - engine: Mount auto-creates per-engine DEK - REST + gRPC: barrier/keys, barrier/rotate-mek, barrier/rotate-key, barrier/migrate - proto: BarrierService (v1 + v2) with ListKeys, RotateMEK, RotateKey, Migrate - db: migration v2 adds barrier_keys table Also includes: security audit report, CSRF protection, engine design specs (sshca, transit, user), path-bound AAD migration tool, policy engine enhancements, and ARCHITECTURE.md updates. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
303 lines
12 KiB
Markdown
303 lines
12 KiB
Markdown
# User Engine Implementation Plan
|
|
|
|
## Overview
|
|
|
|
The user engine provides end-to-end encryption between Metacircular platform
|
|
users. Each user has a key pair managed by Metacrypt; the service handles key
|
|
exchange, encryption, and decryption so that messages/data are encrypted at
|
|
rest and only readable by the intended recipients.
|
|
|
|
The design uses hybrid encryption: an asymmetric key pair per user for key
|
|
exchange, combined with symmetric encryption for message payloads. This enables
|
|
multi-recipient encryption without sharing symmetric keys directly.
|
|
|
|
## Engine Type
|
|
|
|
`user` — registered constant already exists in `internal/engine/engine.go`.
|
|
|
|
## Mount Configuration
|
|
|
|
Passed as `config` at mount time:
|
|
|
|
| Field | Default | Description |
|
|
|-----------------|-------------|------------------------------------------|
|
|
| `key_algorithm` | `"x25519"` | Key exchange algorithm: x25519, ecdh-p256, ecdh-p384 |
|
|
| `sym_algorithm` | `"aes256-gcm"` | Symmetric algorithm for message encryption |
|
|
|
|
## Trust Model
|
|
|
|
Server-trust: Metacrypt holds all private keys in the barrier, same as every
|
|
other engine. Access control is enforced at the application layer — no API
|
|
surface exports private keys, and only the engine accesses them internally
|
|
during encrypt/decrypt operations. An operator with barrier access could
|
|
theoretically extract keys, which is accepted and consistent with the barrier
|
|
trust model used throughout Metacrypt.
|
|
|
|
## Core Concepts
|
|
|
|
### User Provisioning
|
|
|
|
Any MCIAS user can have a keypair, whether or not they have ever logged in to
|
|
Metacrypt. Keypairs are created in three ways:
|
|
|
|
1. **Self-registration** — an authenticated user calls `register`.
|
|
2. **Admin provisioning** — an admin calls `provision` with a username. The
|
|
user does not need to have logged in.
|
|
3. **Auto-provisioning on encrypt** — when a sender encrypts to a recipient
|
|
who has no keypair, the engine generates one automatically. The recipient
|
|
can decrypt when they eventually authenticate.
|
|
|
|
### User Key Pairs
|
|
|
|
Each provisioned user has a key exchange key pair. The private key is stored
|
|
encrypted in the barrier and is only used by the engine on behalf of the owning
|
|
user (enforced in `HandleRequest`). The public key is available to any
|
|
authenticated user (needed to encrypt messages to that user).
|
|
|
|
### Encryption Flow (Sender → Recipient)
|
|
|
|
1. Sender calls `encrypt` with plaintext, recipient username(s), and optional
|
|
metadata.
|
|
2. Engine generates a random symmetric data encryption key (DEK).
|
|
3. Engine encrypts the plaintext with the DEK.
|
|
4. For each recipient: engine performs key agreement (sender private key +
|
|
recipient public key → shared secret), derives a wrapping key via HKDF,
|
|
and wraps the DEK.
|
|
5. Returns an envelope containing ciphertext + per-recipient wrapped DEKs.
|
|
|
|
### Decryption Flow
|
|
|
|
1. Recipient calls `decrypt` with the envelope.
|
|
2. Engine finds the recipient's wrapped DEK entry.
|
|
3. Engine performs key agreement (recipient private key + sender public key →
|
|
shared secret), derives the wrapping key, unwraps the DEK.
|
|
4. Engine decrypts the ciphertext with the DEK.
|
|
5. Returns plaintext.
|
|
|
|
### Envelope Format
|
|
|
|
```json
|
|
{
|
|
"version": 1,
|
|
"sender": "alice",
|
|
"sym_algorithm": "aes256-gcm",
|
|
"ciphertext": "<base64(nonce + encrypted_payload + tag)>",
|
|
"recipients": {
|
|
"bob": "<base64(wrapped_dek)>",
|
|
"carol": "<base64(wrapped_dek)>"
|
|
}
|
|
}
|
|
```
|
|
|
|
The envelope is base64-encoded as a single opaque blob for transport.
|
|
|
|
## Barrier Storage Layout
|
|
|
|
```
|
|
engine/user/{mount}/config.json Engine configuration
|
|
engine/user/{mount}/users/{username}/priv.pem User private key
|
|
engine/user/{mount}/users/{username}/pub.pem User public key
|
|
engine/user/{mount}/users/{username}/config.json Per-user metadata
|
|
```
|
|
|
|
## In-Memory State
|
|
|
|
```go
|
|
type UserEngine struct {
|
|
barrier barrier.Barrier
|
|
config *UserConfig
|
|
users map[string]*userState
|
|
mountPath string
|
|
mu sync.RWMutex
|
|
}
|
|
|
|
type userState struct {
|
|
privKey crypto.PrivateKey // key exchange private key
|
|
pubKey crypto.PublicKey // key exchange public key
|
|
config *UserKeyConfig
|
|
}
|
|
```
|
|
|
|
## Lifecycle
|
|
|
|
### Initialize
|
|
|
|
1. Parse and store config in barrier.
|
|
2. No user keys are created at init time (created on demand or via `register`).
|
|
|
|
### Unseal
|
|
|
|
1. Load config from barrier.
|
|
2. Discover and load all user key pairs from barrier.
|
|
|
|
### Seal
|
|
|
|
1. Zeroize all private key material.
|
|
2. Nil out all maps.
|
|
|
|
## Operations
|
|
|
|
| Operation | Auth Required | Description |
|
|
|------------------|---------------|-------------------------------------------------|
|
|
| `register` | User (self) | Create a key pair for the authenticated user |
|
|
| `provision` | Admin | Create a key pair for any MCIAS user by username |
|
|
| `get-public-key` | User/Admin | Get any user's public key |
|
|
| `list-users` | User/Admin | List registered users |
|
|
| `encrypt` | User+Policy | Encrypt data for one or more recipients |
|
|
| `decrypt` | User (self) | Decrypt an envelope addressed to the caller |
|
|
| `rotate-key` | User (self) | Rotate the caller's key pair |
|
|
| `delete-user` | Admin | Remove a user's key pair |
|
|
|
|
### register
|
|
|
|
Creates a key pair for the authenticated caller. No-op if the caller already
|
|
has a keypair (returns existing public key).
|
|
|
|
Request data: none (uses `CallerInfo.Username`).
|
|
|
|
Response: `{ "public_key": "<base64>" }`
|
|
|
|
### provision
|
|
|
|
Admin-only. Creates a key pair for the given username. The user does not need
|
|
to have logged in to Metacrypt. No-op if the user already has a keypair.
|
|
|
|
Request data: `{ "username": "<mcias_username>" }`
|
|
|
|
Response: `{ "public_key": "<base64>" }`
|
|
|
|
### encrypt
|
|
|
|
Request data:
|
|
|
|
| Field | Required | Description |
|
|
|--------------|----------|-------------------------------------------|
|
|
| `recipients` | Yes | List of usernames to encrypt for |
|
|
| `plaintext` | Yes | Base64-encoded plaintext |
|
|
| `metadata` | No | Arbitrary string metadata (authenticated) |
|
|
|
|
Flow:
|
|
1. Caller must be provisioned (has a key pair). Auto-provision if not.
|
|
2. For each recipient without a keypair: auto-provision them.
|
|
3. Load sender's private key and each recipient's public key.
|
|
4. Generate random DEK, encrypt plaintext with DEK.
|
|
5. For each recipient: ECDH(sender_priv, recipient_pub) → shared_secret,
|
|
HKDF(shared_secret, salt, info) → wrapping_key, AES-KeyWrap(wrapping_key,
|
|
DEK) → wrapped_dek.
|
|
6. Build and return envelope.
|
|
|
|
Authorization:
|
|
- Admins: grant-all.
|
|
- Users: can encrypt to any registered user by default.
|
|
- Policy can restrict which users a sender can encrypt to:
|
|
resource `user/{mount}/recipient/{username}`, action `write`.
|
|
|
|
### decrypt
|
|
|
|
Request data:
|
|
|
|
| Field | Required | Description |
|
|
|------------|----------|--------------------------------|
|
|
| `envelope` | Yes | Base64-encoded envelope blob |
|
|
|
|
Flow:
|
|
1. Parse envelope, find the caller's wrapped DEK entry.
|
|
2. Load sender's public key and caller's private key.
|
|
3. ECDH(caller_priv, sender_pub) → shared_secret → wrapping_key → DEK.
|
|
4. Decrypt ciphertext with DEK.
|
|
5. Return plaintext.
|
|
|
|
A user can only decrypt envelopes addressed to themselves.
|
|
|
|
### rotate-key
|
|
|
|
Generates a new key pair for the caller. The old private key is zeroized and
|
|
deleted. Old envelopes encrypted with the previous key cannot be decrypted
|
|
after rotation — callers should re-encrypt any stored data before rotating.
|
|
|
|
## gRPC Service (proto/metacrypt/v2/user.proto)
|
|
|
|
```protobuf
|
|
service UserService {
|
|
rpc Register(UserRegisterRequest) returns (UserRegisterResponse);
|
|
rpc Provision(UserProvisionRequest) returns (UserProvisionResponse);
|
|
rpc GetPublicKey(UserGetPublicKeyRequest) returns (UserGetPublicKeyResponse);
|
|
rpc ListUsers(UserListUsersRequest) returns (UserListUsersResponse);
|
|
rpc Encrypt(UserEncryptRequest) returns (UserEncryptResponse);
|
|
rpc Decrypt(UserDecryptRequest) returns (UserDecryptResponse);
|
|
rpc RotateKey(UserRotateKeyRequest) returns (UserRotateKeyResponse);
|
|
rpc DeleteUser(UserDeleteUserRequest) returns (UserDeleteUserResponse);
|
|
}
|
|
```
|
|
|
|
## REST Endpoints
|
|
|
|
All auth required:
|
|
|
|
| Method | Path | Description |
|
|
|--------|-----------------------------------------|------------------------|
|
|
| POST | `/v1/user/{mount}/register` | Register caller |
|
|
| POST | `/v1/user/{mount}/provision` | Provision user (admin) |
|
|
| GET | `/v1/user/{mount}/keys` | List registered users |
|
|
| GET | `/v1/user/{mount}/keys/{username}` | Get user's public key |
|
|
| DELETE | `/v1/user/{mount}/keys/{username}` | Delete user (admin) |
|
|
| POST | `/v1/user/{mount}/encrypt` | Encrypt for recipients |
|
|
| POST | `/v1/user/{mount}/decrypt` | Decrypt envelope |
|
|
| POST | `/v1/user/{mount}/rotate` | Rotate caller's key |
|
|
|
|
All operations are also accessible via the generic `POST /v1/engine/request`.
|
|
|
|
## Web UI
|
|
|
|
Add to `/dashboard` the ability to mount a user engine.
|
|
|
|
Add a `/user-crypto` page displaying:
|
|
- Registration status / register button
|
|
- Public key display
|
|
- Encrypt form (select recipients, enter message)
|
|
- Decrypt form (paste envelope)
|
|
- Key rotation button with warning
|
|
|
|
## Implementation Steps
|
|
|
|
1. **`internal/engine/user/`** — Implement `UserEngine`:
|
|
- `types.go` — Config types, envelope format.
|
|
- `user.go` — Lifecycle (Initialize, Unseal, Seal, HandleRequest).
|
|
- `crypto.go` — ECDH key agreement, HKDF derivation, DEK wrap/unwrap,
|
|
symmetric encrypt/decrypt.
|
|
- `keys.go` — User registration, key rotation, deletion.
|
|
2. **Register factory** in `cmd/metacrypt/main.go`.
|
|
3. **Proto definitions** — `proto/metacrypt/v2/user.proto`, run `make proto`.
|
|
4. **gRPC handlers** — `internal/grpcserver/user.go`.
|
|
5. **REST routes** — Add to `internal/server/routes.go`.
|
|
6. **Web UI** — Add template + webserver routes.
|
|
7. **Tests** — Unit tests: register, encrypt/decrypt roundtrip, multi-recipient,
|
|
key rotation invalidates old envelopes, authorization checks.
|
|
|
|
## Dependencies
|
|
|
|
- `golang.org/x/crypto/hkdf` (for key derivation from ECDH shared secret)
|
|
- `crypto/ecdh` (Go 1.20+, for X25519 and NIST curve key exchange)
|
|
- Standard library `crypto/aes`, `crypto/cipher`, `crypto/rand`
|
|
|
|
## Security Considerations
|
|
|
|
- Private keys encrypted at rest in the barrier, zeroized on seal.
|
|
- DEK is random per-encryption; never reused.
|
|
- HKDF derivation includes sender and recipient identities in the info string
|
|
to prevent key confusion attacks:
|
|
`info = "metacrypt-user-v1:" + sender + ":" + recipient`.
|
|
- Envelope includes sender identity so the recipient can derive the correct
|
|
shared secret.
|
|
- Key rotation is destructive — old data cannot be decrypted. The engine should
|
|
warn and require explicit confirmation (admin or self only).
|
|
- Server-trust model: the server holds all private keys in the barrier. No API
|
|
surface exports private keys. Access control is application-enforced — the
|
|
engine only uses a private key on behalf of its owner during encrypt/decrypt.
|
|
- Auto-provisioned users have keypairs waiting for them; their private keys are
|
|
protected identically to explicitly registered users.
|
|
- Metadata in the envelope is authenticated (included as additional data in
|
|
AEAD) but not encrypted — it is visible to anyone holding the envelope.
|
|
- Post-quantum readiness: the `key_algorithm` config supports future hybrid
|
|
schemes (e.g. X25519 + ML-KEM). The envelope version field enables migration.
|