Implement a two-level key hierarchy: the MEK now wraps per-engine DEKs stored in a new barrier_keys table, rather than encrypting all barrier entries directly. A v2 ciphertext format (0x02) embeds the key ID so the barrier can resolve which DEK to use on decryption. v1 ciphertext remains supported for backward compatibility. Key changes: - crypto: EncryptV2/DecryptV2/ExtractKeyID for v2 ciphertext with key IDs - barrier: key registry (CreateKey, RotateKey, ListKeys, MigrateToV2, ReWrapKeys) - seal: RotateMEK re-wraps DEKs without re-encrypting data - engine: Mount auto-creates per-engine DEK - REST + gRPC: barrier/keys, barrier/rotate-mek, barrier/rotate-key, barrier/migrate - proto: BarrierService (v1 + v2) with ListKeys, RotateMEK, RotateKey, Migrate - db: migration v2 adds barrier_keys table Also includes: security audit report, CSRF protection, engine design specs (sshca, transit, user), path-bound AAD migration tool, policy engine enhancements, and ARCHITECTURE.md updates. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
302
engines/user.md
Normal file
302
engines/user.md
Normal file
@@ -0,0 +1,302 @@
|
||||
# User Engine Implementation Plan
|
||||
|
||||
## Overview
|
||||
|
||||
The user engine provides end-to-end encryption between Metacircular platform
|
||||
users. Each user has a key pair managed by Metacrypt; the service handles key
|
||||
exchange, encryption, and decryption so that messages/data are encrypted at
|
||||
rest and only readable by the intended recipients.
|
||||
|
||||
The design uses hybrid encryption: an asymmetric key pair per user for key
|
||||
exchange, combined with symmetric encryption for message payloads. This enables
|
||||
multi-recipient encryption without sharing symmetric keys directly.
|
||||
|
||||
## Engine Type
|
||||
|
||||
`user` — registered constant already exists in `internal/engine/engine.go`.
|
||||
|
||||
## Mount Configuration
|
||||
|
||||
Passed as `config` at mount time:
|
||||
|
||||
| Field | Default | Description |
|
||||
|-----------------|-------------|------------------------------------------|
|
||||
| `key_algorithm` | `"x25519"` | Key exchange algorithm: x25519, ecdh-p256, ecdh-p384 |
|
||||
| `sym_algorithm` | `"aes256-gcm"` | Symmetric algorithm for message encryption |
|
||||
|
||||
## Trust Model
|
||||
|
||||
Server-trust: Metacrypt holds all private keys in the barrier, same as every
|
||||
other engine. Access control is enforced at the application layer — no API
|
||||
surface exports private keys, and only the engine accesses them internally
|
||||
during encrypt/decrypt operations. An operator with barrier access could
|
||||
theoretically extract keys, which is accepted and consistent with the barrier
|
||||
trust model used throughout Metacrypt.
|
||||
|
||||
## Core Concepts
|
||||
|
||||
### User Provisioning
|
||||
|
||||
Any MCIAS user can have a keypair, whether or not they have ever logged in to
|
||||
Metacrypt. Keypairs are created in three ways:
|
||||
|
||||
1. **Self-registration** — an authenticated user calls `register`.
|
||||
2. **Admin provisioning** — an admin calls `provision` with a username. The
|
||||
user does not need to have logged in.
|
||||
3. **Auto-provisioning on encrypt** — when a sender encrypts to a recipient
|
||||
who has no keypair, the engine generates one automatically. The recipient
|
||||
can decrypt when they eventually authenticate.
|
||||
|
||||
### User Key Pairs
|
||||
|
||||
Each provisioned user has a key exchange key pair. The private key is stored
|
||||
encrypted in the barrier and is only used by the engine on behalf of the owning
|
||||
user (enforced in `HandleRequest`). The public key is available to any
|
||||
authenticated user (needed to encrypt messages to that user).
|
||||
|
||||
### Encryption Flow (Sender → Recipient)
|
||||
|
||||
1. Sender calls `encrypt` with plaintext, recipient username(s), and optional
|
||||
metadata.
|
||||
2. Engine generates a random symmetric data encryption key (DEK).
|
||||
3. Engine encrypts the plaintext with the DEK.
|
||||
4. For each recipient: engine performs key agreement (sender private key +
|
||||
recipient public key → shared secret), derives a wrapping key via HKDF,
|
||||
and wraps the DEK.
|
||||
5. Returns an envelope containing ciphertext + per-recipient wrapped DEKs.
|
||||
|
||||
### Decryption Flow
|
||||
|
||||
1. Recipient calls `decrypt` with the envelope.
|
||||
2. Engine finds the recipient's wrapped DEK entry.
|
||||
3. Engine performs key agreement (recipient private key + sender public key →
|
||||
shared secret), derives the wrapping key, unwraps the DEK.
|
||||
4. Engine decrypts the ciphertext with the DEK.
|
||||
5. Returns plaintext.
|
||||
|
||||
### Envelope Format
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"sender": "alice",
|
||||
"sym_algorithm": "aes256-gcm",
|
||||
"ciphertext": "<base64(nonce + encrypted_payload + tag)>",
|
||||
"recipients": {
|
||||
"bob": "<base64(wrapped_dek)>",
|
||||
"carol": "<base64(wrapped_dek)>"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The envelope is base64-encoded as a single opaque blob for transport.
|
||||
|
||||
## Barrier Storage Layout
|
||||
|
||||
```
|
||||
engine/user/{mount}/config.json Engine configuration
|
||||
engine/user/{mount}/users/{username}/priv.pem User private key
|
||||
engine/user/{mount}/users/{username}/pub.pem User public key
|
||||
engine/user/{mount}/users/{username}/config.json Per-user metadata
|
||||
```
|
||||
|
||||
## In-Memory State
|
||||
|
||||
```go
|
||||
type UserEngine struct {
|
||||
barrier barrier.Barrier
|
||||
config *UserConfig
|
||||
users map[string]*userState
|
||||
mountPath string
|
||||
mu sync.RWMutex
|
||||
}
|
||||
|
||||
type userState struct {
|
||||
privKey crypto.PrivateKey // key exchange private key
|
||||
pubKey crypto.PublicKey // key exchange public key
|
||||
config *UserKeyConfig
|
||||
}
|
||||
```
|
||||
|
||||
## Lifecycle
|
||||
|
||||
### Initialize
|
||||
|
||||
1. Parse and store config in barrier.
|
||||
2. No user keys are created at init time (created on demand or via `register`).
|
||||
|
||||
### Unseal
|
||||
|
||||
1. Load config from barrier.
|
||||
2. Discover and load all user key pairs from barrier.
|
||||
|
||||
### Seal
|
||||
|
||||
1. Zeroize all private key material.
|
||||
2. Nil out all maps.
|
||||
|
||||
## Operations
|
||||
|
||||
| Operation | Auth Required | Description |
|
||||
|------------------|---------------|-------------------------------------------------|
|
||||
| `register` | User (self) | Create a key pair for the authenticated user |
|
||||
| `provision` | Admin | Create a key pair for any MCIAS user by username |
|
||||
| `get-public-key` | User/Admin | Get any user's public key |
|
||||
| `list-users` | User/Admin | List registered users |
|
||||
| `encrypt` | User+Policy | Encrypt data for one or more recipients |
|
||||
| `decrypt` | User (self) | Decrypt an envelope addressed to the caller |
|
||||
| `rotate-key` | User (self) | Rotate the caller's key pair |
|
||||
| `delete-user` | Admin | Remove a user's key pair |
|
||||
|
||||
### register
|
||||
|
||||
Creates a key pair for the authenticated caller. No-op if the caller already
|
||||
has a keypair (returns existing public key).
|
||||
|
||||
Request data: none (uses `CallerInfo.Username`).
|
||||
|
||||
Response: `{ "public_key": "<base64>" }`
|
||||
|
||||
### provision
|
||||
|
||||
Admin-only. Creates a key pair for the given username. The user does not need
|
||||
to have logged in to Metacrypt. No-op if the user already has a keypair.
|
||||
|
||||
Request data: `{ "username": "<mcias_username>" }`
|
||||
|
||||
Response: `{ "public_key": "<base64>" }`
|
||||
|
||||
### encrypt
|
||||
|
||||
Request data:
|
||||
|
||||
| Field | Required | Description |
|
||||
|--------------|----------|-------------------------------------------|
|
||||
| `recipients` | Yes | List of usernames to encrypt for |
|
||||
| `plaintext` | Yes | Base64-encoded plaintext |
|
||||
| `metadata` | No | Arbitrary string metadata (authenticated) |
|
||||
|
||||
Flow:
|
||||
1. Caller must be provisioned (has a key pair). Auto-provision if not.
|
||||
2. For each recipient without a keypair: auto-provision them.
|
||||
3. Load sender's private key and each recipient's public key.
|
||||
4. Generate random DEK, encrypt plaintext with DEK.
|
||||
5. For each recipient: ECDH(sender_priv, recipient_pub) → shared_secret,
|
||||
HKDF(shared_secret, salt, info) → wrapping_key, AES-KeyWrap(wrapping_key,
|
||||
DEK) → wrapped_dek.
|
||||
6. Build and return envelope.
|
||||
|
||||
Authorization:
|
||||
- Admins: grant-all.
|
||||
- Users: can encrypt to any registered user by default.
|
||||
- Policy can restrict which users a sender can encrypt to:
|
||||
resource `user/{mount}/recipient/{username}`, action `write`.
|
||||
|
||||
### decrypt
|
||||
|
||||
Request data:
|
||||
|
||||
| Field | Required | Description |
|
||||
|------------|----------|--------------------------------|
|
||||
| `envelope` | Yes | Base64-encoded envelope blob |
|
||||
|
||||
Flow:
|
||||
1. Parse envelope, find the caller's wrapped DEK entry.
|
||||
2. Load sender's public key and caller's private key.
|
||||
3. ECDH(caller_priv, sender_pub) → shared_secret → wrapping_key → DEK.
|
||||
4. Decrypt ciphertext with DEK.
|
||||
5. Return plaintext.
|
||||
|
||||
A user can only decrypt envelopes addressed to themselves.
|
||||
|
||||
### rotate-key
|
||||
|
||||
Generates a new key pair for the caller. The old private key is zeroized and
|
||||
deleted. Old envelopes encrypted with the previous key cannot be decrypted
|
||||
after rotation — callers should re-encrypt any stored data before rotating.
|
||||
|
||||
## gRPC Service (proto/metacrypt/v2/user.proto)
|
||||
|
||||
```protobuf
|
||||
service UserService {
|
||||
rpc Register(UserRegisterRequest) returns (UserRegisterResponse);
|
||||
rpc Provision(UserProvisionRequest) returns (UserProvisionResponse);
|
||||
rpc GetPublicKey(UserGetPublicKeyRequest) returns (UserGetPublicKeyResponse);
|
||||
rpc ListUsers(UserListUsersRequest) returns (UserListUsersResponse);
|
||||
rpc Encrypt(UserEncryptRequest) returns (UserEncryptResponse);
|
||||
rpc Decrypt(UserDecryptRequest) returns (UserDecryptResponse);
|
||||
rpc RotateKey(UserRotateKeyRequest) returns (UserRotateKeyResponse);
|
||||
rpc DeleteUser(UserDeleteUserRequest) returns (UserDeleteUserResponse);
|
||||
}
|
||||
```
|
||||
|
||||
## REST Endpoints
|
||||
|
||||
All auth required:
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|-----------------------------------------|------------------------|
|
||||
| POST | `/v1/user/{mount}/register` | Register caller |
|
||||
| POST | `/v1/user/{mount}/provision` | Provision user (admin) |
|
||||
| GET | `/v1/user/{mount}/keys` | List registered users |
|
||||
| GET | `/v1/user/{mount}/keys/{username}` | Get user's public key |
|
||||
| DELETE | `/v1/user/{mount}/keys/{username}` | Delete user (admin) |
|
||||
| POST | `/v1/user/{mount}/encrypt` | Encrypt for recipients |
|
||||
| POST | `/v1/user/{mount}/decrypt` | Decrypt envelope |
|
||||
| POST | `/v1/user/{mount}/rotate` | Rotate caller's key |
|
||||
|
||||
All operations are also accessible via the generic `POST /v1/engine/request`.
|
||||
|
||||
## Web UI
|
||||
|
||||
Add to `/dashboard` the ability to mount a user engine.
|
||||
|
||||
Add a `/user-crypto` page displaying:
|
||||
- Registration status / register button
|
||||
- Public key display
|
||||
- Encrypt form (select recipients, enter message)
|
||||
- Decrypt form (paste envelope)
|
||||
- Key rotation button with warning
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
1. **`internal/engine/user/`** — Implement `UserEngine`:
|
||||
- `types.go` — Config types, envelope format.
|
||||
- `user.go` — Lifecycle (Initialize, Unseal, Seal, HandleRequest).
|
||||
- `crypto.go` — ECDH key agreement, HKDF derivation, DEK wrap/unwrap,
|
||||
symmetric encrypt/decrypt.
|
||||
- `keys.go` — User registration, key rotation, deletion.
|
||||
2. **Register factory** in `cmd/metacrypt/main.go`.
|
||||
3. **Proto definitions** — `proto/metacrypt/v2/user.proto`, run `make proto`.
|
||||
4. **gRPC handlers** — `internal/grpcserver/user.go`.
|
||||
5. **REST routes** — Add to `internal/server/routes.go`.
|
||||
6. **Web UI** — Add template + webserver routes.
|
||||
7. **Tests** — Unit tests: register, encrypt/decrypt roundtrip, multi-recipient,
|
||||
key rotation invalidates old envelopes, authorization checks.
|
||||
|
||||
## Dependencies
|
||||
|
||||
- `golang.org/x/crypto/hkdf` (for key derivation from ECDH shared secret)
|
||||
- `crypto/ecdh` (Go 1.20+, for X25519 and NIST curve key exchange)
|
||||
- Standard library `crypto/aes`, `crypto/cipher`, `crypto/rand`
|
||||
|
||||
## Security Considerations
|
||||
|
||||
- Private keys encrypted at rest in the barrier, zeroized on seal.
|
||||
- DEK is random per-encryption; never reused.
|
||||
- HKDF derivation includes sender and recipient identities in the info string
|
||||
to prevent key confusion attacks:
|
||||
`info = "metacrypt-user-v1:" + sender + ":" + recipient`.
|
||||
- Envelope includes sender identity so the recipient can derive the correct
|
||||
shared secret.
|
||||
- Key rotation is destructive — old data cannot be decrypted. The engine should
|
||||
warn and require explicit confirmation (admin or self only).
|
||||
- Server-trust model: the server holds all private keys in the barrier. No API
|
||||
surface exports private keys. Access control is application-enforced — the
|
||||
engine only uses a private key on behalf of its owner during encrypt/decrypt.
|
||||
- Auto-provisioned users have keypairs waiting for them; their private keys are
|
||||
protected identically to explicitly registered users.
|
||||
- Metadata in the envelope is authenticated (included as additional data in
|
||||
AEAD) but not encrypted — it is visible to anyone holding the envelope.
|
||||
- Post-quantum readiness: the `key_algorithm` config supports future hybrid
|
||||
schemes (e.g. X25519 + ML-KEM). The envelope version field enables migration.
|
||||
Reference in New Issue
Block a user