Implement a two-level key hierarchy: the MEK now wraps per-engine DEKs stored in a new barrier_keys table, rather than encrypting all barrier entries directly. A v2 ciphertext format (0x02) embeds the key ID so the barrier can resolve which DEK to use on decryption. v1 ciphertext remains supported for backward compatibility. Key changes: - crypto: EncryptV2/DecryptV2/ExtractKeyID for v2 ciphertext with key IDs - barrier: key registry (CreateKey, RotateKey, ListKeys, MigrateToV2, ReWrapKeys) - seal: RotateMEK re-wraps DEKs without re-encrypting data - engine: Mount auto-creates per-engine DEK - REST + gRPC: barrier/keys, barrier/rotate-mek, barrier/rotate-key, barrier/migrate - proto: BarrierService (v1 + v2) with ListKeys, RotateMEK, RotateKey, Migrate - db: migration v2 adds barrier_keys table Also includes: security audit report, CSRF protection, engine design specs (sshca, transit, user), path-bound AAD migration tool, policy engine enhancements, and ARCHITECTURE.md updates. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
12 KiB
User Engine Implementation Plan
Overview
The user engine provides end-to-end encryption between Metacircular platform users. Each user has a key pair managed by Metacrypt; the service handles key exchange, encryption, and decryption so that messages/data are encrypted at rest and only readable by the intended recipients.
The design uses hybrid encryption: an asymmetric key pair per user for key exchange, combined with symmetric encryption for message payloads. This enables multi-recipient encryption without sharing symmetric keys directly.
Engine Type
user — registered constant already exists in internal/engine/engine.go.
Mount Configuration
Passed as config at mount time:
| Field | Default | Description |
|---|---|---|
key_algorithm |
"x25519" |
Key exchange algorithm: x25519, ecdh-p256, ecdh-p384 |
sym_algorithm |
"aes256-gcm" |
Symmetric algorithm for message encryption |
Trust Model
Server-trust: Metacrypt holds all private keys in the barrier, same as every other engine. Access control is enforced at the application layer — no API surface exports private keys, and only the engine accesses them internally during encrypt/decrypt operations. An operator with barrier access could theoretically extract keys, which is accepted and consistent with the barrier trust model used throughout Metacrypt.
Core Concepts
User Provisioning
Any MCIAS user can have a keypair, whether or not they have ever logged in to Metacrypt. Keypairs are created in three ways:
- Self-registration — an authenticated user calls
register. - Admin provisioning — an admin calls
provisionwith a username. The user does not need to have logged in. - Auto-provisioning on encrypt — when a sender encrypts to a recipient who has no keypair, the engine generates one automatically. The recipient can decrypt when they eventually authenticate.
User Key Pairs
Each provisioned user has a key exchange key pair. The private key is stored
encrypted in the barrier and is only used by the engine on behalf of the owning
user (enforced in HandleRequest). The public key is available to any
authenticated user (needed to encrypt messages to that user).
Encryption Flow (Sender → Recipient)
- Sender calls
encryptwith plaintext, recipient username(s), and optional metadata. - Engine generates a random symmetric data encryption key (DEK).
- Engine encrypts the plaintext with the DEK.
- For each recipient: engine performs key agreement (sender private key + recipient public key → shared secret), derives a wrapping key via HKDF, and wraps the DEK.
- Returns an envelope containing ciphertext + per-recipient wrapped DEKs.
Decryption Flow
- Recipient calls
decryptwith the envelope. - Engine finds the recipient's wrapped DEK entry.
- Engine performs key agreement (recipient private key + sender public key → shared secret), derives the wrapping key, unwraps the DEK.
- Engine decrypts the ciphertext with the DEK.
- Returns plaintext.
Envelope Format
{
"version": 1,
"sender": "alice",
"sym_algorithm": "aes256-gcm",
"ciphertext": "<base64(nonce + encrypted_payload + tag)>",
"recipients": {
"bob": "<base64(wrapped_dek)>",
"carol": "<base64(wrapped_dek)>"
}
}
The envelope is base64-encoded as a single opaque blob for transport.
Barrier Storage Layout
engine/user/{mount}/config.json Engine configuration
engine/user/{mount}/users/{username}/priv.pem User private key
engine/user/{mount}/users/{username}/pub.pem User public key
engine/user/{mount}/users/{username}/config.json Per-user metadata
In-Memory State
type UserEngine struct {
barrier barrier.Barrier
config *UserConfig
users map[string]*userState
mountPath string
mu sync.RWMutex
}
type userState struct {
privKey crypto.PrivateKey // key exchange private key
pubKey crypto.PublicKey // key exchange public key
config *UserKeyConfig
}
Lifecycle
Initialize
- Parse and store config in barrier.
- No user keys are created at init time (created on demand or via
register).
Unseal
- Load config from barrier.
- Discover and load all user key pairs from barrier.
Seal
- Zeroize all private key material.
- Nil out all maps.
Operations
| Operation | Auth Required | Description |
|---|---|---|
register |
User (self) | Create a key pair for the authenticated user |
provision |
Admin | Create a key pair for any MCIAS user by username |
get-public-key |
User/Admin | Get any user's public key |
list-users |
User/Admin | List registered users |
encrypt |
User+Policy | Encrypt data for one or more recipients |
decrypt |
User (self) | Decrypt an envelope addressed to the caller |
rotate-key |
User (self) | Rotate the caller's key pair |
delete-user |
Admin | Remove a user's key pair |
register
Creates a key pair for the authenticated caller. No-op if the caller already has a keypair (returns existing public key).
Request data: none (uses CallerInfo.Username).
Response: { "public_key": "<base64>" }
provision
Admin-only. Creates a key pair for the given username. The user does not need to have logged in to Metacrypt. No-op if the user already has a keypair.
Request data: { "username": "<mcias_username>" }
Response: { "public_key": "<base64>" }
encrypt
Request data:
| Field | Required | Description |
|---|---|---|
recipients |
Yes | List of usernames to encrypt for |
plaintext |
Yes | Base64-encoded plaintext |
metadata |
No | Arbitrary string metadata (authenticated) |
Flow:
- Caller must be provisioned (has a key pair). Auto-provision if not.
- For each recipient without a keypair: auto-provision them.
- Load sender's private key and each recipient's public key.
- Generate random DEK, encrypt plaintext with DEK.
- For each recipient: ECDH(sender_priv, recipient_pub) → shared_secret, HKDF(shared_secret, salt, info) → wrapping_key, AES-KeyWrap(wrapping_key, DEK) → wrapped_dek.
- Build and return envelope.
Authorization:
- Admins: grant-all.
- Users: can encrypt to any registered user by default.
- Policy can restrict which users a sender can encrypt to:
resource
user/{mount}/recipient/{username}, actionwrite.
decrypt
Request data:
| Field | Required | Description |
|---|---|---|
envelope |
Yes | Base64-encoded envelope blob |
Flow:
- Parse envelope, find the caller's wrapped DEK entry.
- Load sender's public key and caller's private key.
- ECDH(caller_priv, sender_pub) → shared_secret → wrapping_key → DEK.
- Decrypt ciphertext with DEK.
- Return plaintext.
A user can only decrypt envelopes addressed to themselves.
rotate-key
Generates a new key pair for the caller. The old private key is zeroized and deleted. Old envelopes encrypted with the previous key cannot be decrypted after rotation — callers should re-encrypt any stored data before rotating.
gRPC Service (proto/metacrypt/v2/user.proto)
service UserService {
rpc Register(UserRegisterRequest) returns (UserRegisterResponse);
rpc Provision(UserProvisionRequest) returns (UserProvisionResponse);
rpc GetPublicKey(UserGetPublicKeyRequest) returns (UserGetPublicKeyResponse);
rpc ListUsers(UserListUsersRequest) returns (UserListUsersResponse);
rpc Encrypt(UserEncryptRequest) returns (UserEncryptResponse);
rpc Decrypt(UserDecryptRequest) returns (UserDecryptResponse);
rpc RotateKey(UserRotateKeyRequest) returns (UserRotateKeyResponse);
rpc DeleteUser(UserDeleteUserRequest) returns (UserDeleteUserResponse);
}
REST Endpoints
All auth required:
| Method | Path | Description |
|---|---|---|
| POST | /v1/user/{mount}/register |
Register caller |
| POST | /v1/user/{mount}/provision |
Provision user (admin) |
| GET | /v1/user/{mount}/keys |
List registered users |
| GET | /v1/user/{mount}/keys/{username} |
Get user's public key |
| DELETE | /v1/user/{mount}/keys/{username} |
Delete user (admin) |
| POST | /v1/user/{mount}/encrypt |
Encrypt for recipients |
| POST | /v1/user/{mount}/decrypt |
Decrypt envelope |
| POST | /v1/user/{mount}/rotate |
Rotate caller's key |
All operations are also accessible via the generic POST /v1/engine/request.
Web UI
Add to /dashboard the ability to mount a user engine.
Add a /user-crypto page displaying:
- Registration status / register button
- Public key display
- Encrypt form (select recipients, enter message)
- Decrypt form (paste envelope)
- Key rotation button with warning
Implementation Steps
internal/engine/user/— ImplementUserEngine:types.go— Config types, envelope format.user.go— Lifecycle (Initialize, Unseal, Seal, HandleRequest).crypto.go— ECDH key agreement, HKDF derivation, DEK wrap/unwrap, symmetric encrypt/decrypt.keys.go— User registration, key rotation, deletion.
- Register factory in
cmd/metacrypt/main.go. - Proto definitions —
proto/metacrypt/v2/user.proto, runmake proto. - gRPC handlers —
internal/grpcserver/user.go. - REST routes — Add to
internal/server/routes.go. - Web UI — Add template + webserver routes.
- Tests — Unit tests: register, encrypt/decrypt roundtrip, multi-recipient, key rotation invalidates old envelopes, authorization checks.
Dependencies
golang.org/x/crypto/hkdf(for key derivation from ECDH shared secret)crypto/ecdh(Go 1.20+, for X25519 and NIST curve key exchange)- Standard library
crypto/aes,crypto/cipher,crypto/rand
Security Considerations
- Private keys encrypted at rest in the barrier, zeroized on seal.
- DEK is random per-encryption; never reused.
- HKDF derivation includes sender and recipient identities in the info string
to prevent key confusion attacks:
info = "metacrypt-user-v1:" + sender + ":" + recipient. - Envelope includes sender identity so the recipient can derive the correct shared secret.
- Key rotation is destructive — old data cannot be decrypted. The engine should warn and require explicit confirmation (admin or self only).
- Server-trust model: the server holds all private keys in the barrier. No API surface exports private keys. Access control is application-enforced — the engine only uses a private key on behalf of its owner during encrypt/decrypt.
- Auto-provisioned users have keypairs waiting for them; their private keys are protected identically to explicitly registered users.
- Metadata in the envelope is authenticated (included as additional data in AEAD) but not encrypted — it is visible to anyone holding the envelope.
- Post-quantum readiness: the
key_algorithmconfig supports future hybrid schemes (e.g. X25519 + ML-KEM). The envelope version field enables migration.