# User Engine Implementation Plan ## Overview The user engine provides end-to-end encryption between Metacircular platform users. Each user has a key pair managed by Metacrypt; the service handles key exchange, encryption, and decryption so that messages/data are encrypted at rest and only readable by the intended recipients. The design uses hybrid encryption: an asymmetric key pair per user for key exchange, combined with symmetric encryption for message payloads. This enables multi-recipient encryption without sharing symmetric keys directly. ## Engine Type `user` — registered constant already exists in `internal/engine/engine.go`. ## Mount Configuration Passed as `config` at mount time: | Field | Default | Description | |-----------------|-------------|------------------------------------------| | `key_algorithm` | `"x25519"` | Key exchange algorithm: x25519, ecdh-p256, ecdh-p384 | | `sym_algorithm` | `"aes256-gcm"` | Symmetric algorithm for message encryption | ## Trust Model Server-trust: Metacrypt holds all private keys in the barrier, same as every other engine. Access control is enforced at the application layer — no API surface exports private keys, and only the engine accesses them internally during encrypt/decrypt operations. An operator with barrier access could theoretically extract keys, which is accepted and consistent with the barrier trust model used throughout Metacrypt. ## Core Concepts ### User Provisioning Any MCIAS user can have a keypair, whether or not they have ever logged in to Metacrypt. Keypairs are created in three ways: 1. **Self-registration** — an authenticated user calls `register`. 2. **Admin provisioning** — an admin calls `provision` with a username. The user does not need to have logged in. 3. **Auto-provisioning on encrypt** — when a sender encrypts to a recipient who has no keypair, the engine generates one automatically. The recipient can decrypt when they eventually authenticate. ### User Key Pairs Each provisioned user has a key exchange key pair. The private key is stored encrypted in the barrier and is only used by the engine on behalf of the owning user (enforced in `HandleRequest`). The public key is available to any authenticated user (needed to encrypt messages to that user). **Key generation by algorithm:** - `x25519`: `ecdh.X25519().GenerateKey(rand.Reader)` (Go 1.20+ `crypto/ecdh`). - `ecdh-p256`: `ecdh.P256().GenerateKey(rand.Reader)`. - `ecdh-p384`: `ecdh.P384().GenerateKey(rand.Reader)`. **Key serialization in barrier:** - Private key: `x509.MarshalPKCS8PrivateKey(privKey)` → PEM block with type `"PRIVATE KEY"`. Stored at `{mountPath}users/{username}/priv.pem`. - Public key: `x509.MarshalPKIXPublicKey(pubKey)` → PEM block with type `"PUBLIC KEY"`. Stored at `{mountPath}users/{username}/pub.pem`. Note: `crypto/ecdh` keys implement the interfaces required by `x509.MarshalPKCS8PrivateKey` and `x509.MarshalPKIXPublicKey` as of Go 1.20. ### Cryptographic Details **ECDH key agreement:** ```go sharedSecret, err := senderPrivKey.ECDH(recipientPubKey) ``` The raw shared secret is **never used directly** as a key. It is always fed through HKDF. **HKDF key derivation:** ```go salt := make([]byte, 32) rand.Read(salt) hkdf := hkdf.New(sha256.New, sharedSecret, salt, info) wrappingKey := make([]byte, 32) // 256-bit AES key io.ReadFull(hkdf, wrappingKey) ``` - **Hash:** SHA-256 (sufficient for 256-bit key derivation). - **Salt:** 32 bytes of `crypto/rand` randomness, generated fresh per recipient per encryption. The salt is stored alongside the wrapped DEK in the envelope (see updated envelope format below). - **Info:** `"metacrypt-user-v1:" + sender + ":" + recipient` (UTF-8 encoded). This binds the derived key to the specific sender-recipient pair, preventing key confusion if the same shared secret were somehow reused. **DEK wrapping:** The wrapping key from HKDF encrypts the DEK using AES-256-GCM (not AES Key Wrap / RFC 3394). AES-GCM is used because: - It is already a core primitive in the codebase. - It provides authenticated encryption, same as AES Key Wrap. - The DEK is 32 bytes — well within GCM's plaintext size limits. ```go block, _ := aes.NewCipher(wrappingKey) gcm, _ := cipher.NewGCM(block) nonce := make([]byte, gcm.NonceSize()) // 12 bytes rand.Read(nonce) wrappedDEK := gcm.Seal(nonce, nonce, dek, nil) // nonce || ciphertext || tag ``` **Symmetric encryption (payload):** ```go block, _ := aes.NewCipher(dek) gcm, _ := cipher.NewGCM(block) nonce := make([]byte, gcm.NonceSize()) // 12 bytes rand.Read(nonce) ciphertext := gcm.Seal(nonce, nonce, plaintext, aad) ``` - AAD: if `metadata` is provided, it is used as additional authenticated data. This means metadata is integrity-protected but not encrypted. - Nonce: 12 bytes from `crypto/rand`. ### Encryption Flow (Sender → Recipient) 1. Sender calls `encrypt` with plaintext, recipient username(s), and optional metadata. 2. Engine generates a random symmetric data encryption key (DEK). 3. Engine encrypts the plaintext with the DEK. 4. For each recipient: engine performs key agreement (sender private key + recipient public key → shared secret), derives a wrapping key via HKDF, and wraps the DEK. 5. Returns an envelope containing ciphertext + per-recipient wrapped DEKs. ### Decryption Flow 1. Recipient calls `decrypt` with the envelope. 2. Engine finds the recipient's wrapped DEK entry. 3. Engine performs key agreement (recipient private key + sender public key → shared secret), derives the wrapping key, unwraps the DEK. 4. Engine decrypts the ciphertext with the DEK. 5. Returns plaintext. ### Envelope Format ```json { "version": 1, "sender": "alice", "sym_algorithm": "aes256-gcm", "ciphertext": "", "metadata": "", "recipients": { "bob": { "salt": "", "wrapped_dek": "" }, "carol": { "salt": "", "wrapped_dek": "" } } } ``` Each recipient entry includes: - `salt`: the per-recipient random HKDF salt used during key derivation. - `wrapped_dek`: the AES-256-GCM encryption of the DEK (nonce-prepended). The envelope is base64-encoded as a single opaque blob for transport. ## Barrier Storage Layout ``` engine/user/{mount}/config.json Engine configuration engine/user/{mount}/users/{username}/priv.pem User private key engine/user/{mount}/users/{username}/pub.pem User public key engine/user/{mount}/users/{username}/config.json Per-user metadata ``` ## In-Memory State ```go type UserEngine struct { barrier barrier.Barrier config *UserConfig users map[string]*userState mountPath string mu sync.RWMutex } type userState struct { privKey crypto.PrivateKey // key exchange private key pubKey crypto.PublicKey // key exchange public key config *UserKeyConfig } type UserKeyConfig struct { Algorithm string `json:"algorithm"` // key exchange algorithm (x25519, ecdh-p256, ecdh-p384) CreatedAt time.Time `json:"created_at"` AutoProvisioned bool `json:"auto_provisioned"` // true if created via auto-provisioning } ``` ## Lifecycle ### Initialize 1. Parse and validate config: ensure `key_algorithm` is one of `x25519`, `ecdh-p256`, `ecdh-p384`. Ensure `sym_algorithm` is `aes256-gcm`. 2. Store config in barrier as `{mountPath}config.json`: ```go configJSON, _ := json.Marshal(config) barrier.Put(ctx, mountPath+"config.json", configJSON) ``` 3. No user keys are created at init time (created on demand via `register`, `provision`, or auto-provisioning). ### Unseal 1. Load config JSON from barrier, unmarshal into `*UserConfig`. 2. List all user directories under `{mountPath}users/`. 3. For each user, load `priv.pem` and `pub.pem`: - Parse private key PEM: `pem.Decode` → `x509.ParsePKCS8PrivateKey` → type-assert to `*ecdh.PrivateKey`. - Parse public key PEM: `pem.Decode` → `x509.ParsePKIXPublicKey` → type-assert to `*ecdh.PublicKey`. 4. Populate `users` map with loaded key states. ### Seal 1. Zeroize all private key material using `engine.ZeroizeKey(privKey)`. 2. Nil out `users` map and `config`. ## Operations | Operation | Auth Required | Description | |------------------|---------------|-------------------------------------------------| | `register` | User (self) | Create a key pair for the authenticated user | | `provision` | Admin | Create a key pair for any MCIAS user by username | | `get-public-key` | User/Admin | Get any user's public key | | `list-users` | User/Admin | List registered users | | `encrypt` | User+Policy | Encrypt data for one or more recipients | | `decrypt` | User (self) | Decrypt an envelope addressed to the caller | | `re-encrypt` | User (self) | Re-encrypt an envelope with current key pairs | | `rotate-key` | User (self) | Rotate the caller's key pair | | `delete-user` | Admin | Remove a user's key pair | ### HandleRequest dispatch Follow the CA engine's pattern (`internal/engine/ca/ca.go:284-317`): ```go func (e *UserEngine) HandleRequest(ctx context.Context, req *engine.Request) (*engine.Response, error) { switch req.Operation { case "register": return e.handleRegister(ctx, req) case "provision": return e.handleProvision(ctx, req) case "get-public-key": return e.handleGetPublicKey(ctx, req) case "list-users": return e.handleListUsers(ctx, req) case "encrypt": return e.handleEncrypt(ctx, req) case "decrypt": return e.handleDecrypt(ctx, req) case "re-encrypt": return e.handleReEncrypt(ctx, req) case "rotate-key": return e.handleRotateKey(ctx, req) case "delete-user": return e.handleDeleteUser(ctx, req) default: return nil, fmt.Errorf("user: unknown operation: %s", req.Operation) } } ``` ### register Creates a key pair for the authenticated caller. No-op if the caller already has a keypair (returns existing public key). Request data: none (uses `CallerInfo.Username`). Response: `{ "public_key": "" }` ### provision Admin-only. Creates a key pair for the given username. The user does not need to have logged in to Metacrypt. No-op if the user already has a keypair. Request data: `{ "username": "" }` Response: `{ "public_key": "" }` ### encrypt Request data: | Field | Required | Description | |--------------|----------|-------------------------------------------| | `recipients` | Yes | List of usernames to encrypt for | | `plaintext` | Yes | Base64-encoded plaintext | | `metadata` | No | Arbitrary string metadata (authenticated) | Flow: 1. Validate that `len(recipients) <= maxRecipients` (100). Reject with `400 Bad Request` if exceeded. 2. Caller must be provisioned (has a key pair). If not, auto-provision the caller (generate keypair, store in barrier). This is safe because the caller is already authenticated via MCIAS — their identity is verified. 3. For each recipient without a keypair: validate the username exists in MCIAS via `auth.ValidateUsername(username)`. If the user does not exist, return an error: `"recipient not found: {username}"`. If the user exists, auto-provision them. Auto-provisioning only creates a key pair; it does not grant any MCIAS roles or permissions. The recipient's private key is only accessible when they authenticate. 4. Load sender's private key and each recipient's public key. 5. Generate random 32-byte DEK (`crypto/rand`). Encrypt plaintext with DEK using AES-256-GCM (metadata as AAD if present). 6. For each recipient: - `sharedSecret := senderPrivKey.ECDH(recipientPubKey)` - Generate 32-byte random salt. - `wrappingKey := HKDF(sha256, sharedSecret, salt, info)` - `wrappedDEK := AES-GCM-Encrypt(wrappingKey, DEK)` 7. Build envelope with ciphertext, per-recipient `{salt, wrapped_dek}`. 8. Zeroize DEK, all shared secrets, and all wrapping keys. Authorization: - Admins: grant-all. - Users: can encrypt to any registered user by default. - Policy can restrict which users a sender can encrypt to: resource `user/{mount}/recipient/{username}`, action `write`. ### decrypt Request data: | Field | Required | Description | |------------|----------|--------------------------------| | `envelope` | Yes | Base64-encoded envelope blob | Flow: 1. Parse envelope JSON, find the caller's entry in `recipients`. If the caller is not a recipient, return an error. 2. Load sender's public key (from `envelope.sender`) and caller's private key. 3. `sharedSecret := callerPrivKey.ECDH(senderPubKey)`. 4. `wrappingKey := HKDF(sha256, sharedSecret, recipient.salt, info)`. 5. `dek := AES-GCM-Decrypt(wrappingKey, recipient.wrapped_dek)`. 6. Decrypt ciphertext with DEK (metadata as AAD if present in envelope). 7. Zeroize DEK, shared secret, wrapping key. 8. Return plaintext. A user can only decrypt envelopes addressed to themselves. ### re-encrypt Re-encrypts an envelope with the caller's current key pair. This is the safe way to migrate data before a key rotation. Request data: | Field | Required | Description | |------------|----------|--------------------------------| | `envelope` | Yes | Base64-encoded envelope blob | Flow: 1. Decrypt the envelope (same as `decrypt` flow). 2. Re-encrypt the plaintext for the same recipients using fresh DEKs and current key pairs (same as `encrypt` flow, preserving metadata). 3. Return the new envelope. The caller must be a recipient in the original envelope. The new envelope uses current key pairs for all recipients — if any recipient has rotated their key since the original encryption, the new envelope uses their new public key. ### rotate-key Generates a new key pair for the caller. The old private key is zeroized and deleted. Old envelopes encrypted with the previous key cannot be decrypted after rotation. **Recommended workflow**: Before rotating, re-encrypt all stored envelopes using the `re-encrypt` operation. Then call `rotate-key`. This ensures no data is lost. ## gRPC Service (proto/metacrypt/v2/user.proto) ```protobuf service UserService { rpc Register(UserRegisterRequest) returns (UserRegisterResponse); rpc Provision(UserProvisionRequest) returns (UserProvisionResponse); rpc GetPublicKey(UserGetPublicKeyRequest) returns (UserGetPublicKeyResponse); rpc ListUsers(UserListUsersRequest) returns (UserListUsersResponse); rpc Encrypt(UserEncryptRequest) returns (UserEncryptResponse); rpc Decrypt(UserDecryptRequest) returns (UserDecryptResponse); rpc ReEncrypt(UserReEncryptRequest) returns (UserReEncryptResponse); rpc RotateKey(UserRotateKeyRequest) returns (UserRotateKeyResponse); rpc DeleteUser(UserDeleteUserRequest) returns (UserDeleteUserResponse); } ``` ## REST Endpoints All auth required: | Method | Path | Description | |--------|-----------------------------------------|------------------------| | POST | `/v1/user/{mount}/register` | Register caller | | POST | `/v1/user/{mount}/provision` | Provision user (admin) | | GET | `/v1/user/{mount}/keys` | List registered users | | GET | `/v1/user/{mount}/keys/{username}` | Get user's public key | | DELETE | `/v1/user/{mount}/keys/{username}` | Delete user (admin) | | POST | `/v1/user/{mount}/encrypt` | Encrypt for recipients | | POST | `/v1/user/{mount}/decrypt` | Decrypt envelope | | POST | `/v1/user/{mount}/re-encrypt` | Re-encrypt envelope | | POST | `/v1/user/{mount}/rotate` | Rotate caller's key | All operations are also accessible via the generic `POST /v1/engine/request`. ### REST Route Registration Add to `internal/server/routes.go` in `registerRoutes`, following the CA engine's pattern with `chi.URLParam`: ```go // User engine routes. r.Post("/v1/user/{mount}/register", s.requireAuth(s.handleUserRegister)) r.Post("/v1/user/{mount}/provision", s.requireAdmin(s.handleUserProvision)) r.Get("/v1/user/{mount}/keys", s.requireAuth(s.handleUserListUsers)) r.Get("/v1/user/{mount}/keys/{username}", s.requireAuth(s.handleUserGetPublicKey)) r.Delete("/v1/user/{mount}/keys/{username}", s.requireAdmin(s.handleUserDeleteUser)) r.Post("/v1/user/{mount}/encrypt", s.requireAuth(s.handleUserEncrypt)) r.Post("/v1/user/{mount}/decrypt", s.requireAuth(s.handleUserDecrypt)) r.Post("/v1/user/{mount}/re-encrypt", s.requireAuth(s.handleUserReEncrypt)) r.Post("/v1/user/{mount}/rotate", s.requireAuth(s.handleUserRotateKey)) ``` Each handler extracts `chi.URLParam(r, "mount")` and optionally `chi.URLParam(r, "username")`, builds an `engine.Request`, and calls `s.engines.HandleRequest(...)`. ### gRPC Interceptor Maps Add to `sealRequiredMethods`, `authRequiredMethods`, and `adminRequiredMethods` in `internal/grpcserver/server.go`: ```go // sealRequiredMethods — all user RPCs: "/metacrypt.v2.UserService/Register": true, "/metacrypt.v2.UserService/Provision": true, "/metacrypt.v2.UserService/GetPublicKey": true, "/metacrypt.v2.UserService/ListUsers": true, "/metacrypt.v2.UserService/Encrypt": true, "/metacrypt.v2.UserService/Decrypt": true, "/metacrypt.v2.UserService/ReEncrypt": true, "/metacrypt.v2.UserService/RotateKey": true, "/metacrypt.v2.UserService/DeleteUser": true, // authRequiredMethods — all user RPCs: "/metacrypt.v2.UserService/Register": true, "/metacrypt.v2.UserService/Provision": true, "/metacrypt.v2.UserService/GetPublicKey": true, "/metacrypt.v2.UserService/ListUsers": true, "/metacrypt.v2.UserService/Encrypt": true, "/metacrypt.v2.UserService/Decrypt": true, "/metacrypt.v2.UserService/ReEncrypt": true, "/metacrypt.v2.UserService/RotateKey": true, "/metacrypt.v2.UserService/DeleteUser": true, // adminRequiredMethods — admin-only user RPCs: "/metacrypt.v2.UserService/Provision": true, "/metacrypt.v2.UserService/DeleteUser": true, ``` The `adminOnlyOperations` map in `routes.go` already contains user entries (qualified as `user:provision`, `user:delete-user` — keys are `engineType:operation` to avoid cross-engine name collisions). ## Web UI Add to `/dashboard` the ability to mount a user engine. Add a `/user-crypto` page displaying: - Registration status / register button - Public key display - Encrypt form (select recipients, enter message) - Decrypt form (paste envelope) - Key rotation button with warning ## Implementation Steps 1. **Prerequisite**: `engine.ZeroizeKey` must exist in `internal/engine/helpers.go` (created as part of the SSH CA engine implementation — see `engines/sshca.md` step 1). 2. **`internal/engine/user/`** — Implement `UserEngine`: - `types.go` — Config types, envelope format. - `user.go` — Lifecycle (Initialize, Unseal, Seal, HandleRequest). - `crypto.go` — ECDH key agreement, HKDF derivation, DEK wrap/unwrap, symmetric encrypt/decrypt. - `keys.go` — User registration, key rotation, deletion. 3. **Register factory** in `cmd/metacrypt/main.go`. 4. **Proto definitions** — `proto/metacrypt/v2/user.proto`, run `make proto`. 5. **gRPC handlers** — `internal/grpcserver/user.go`. 6. **REST routes** — Add to `internal/server/routes.go`. 7. **Web UI** — Add template + webserver routes. 8. **Tests** — Unit tests: register, encrypt/decrypt roundtrip, multi-recipient, key rotation invalidates old envelopes, re-encrypt roundtrip, authorization checks. ## Dependencies - `golang.org/x/crypto/hkdf` (for key derivation from ECDH shared secret) - `crypto/ecdh` (Go 1.20+, for X25519 and NIST curve key exchange) - Standard library `crypto/aes`, `crypto/cipher`, `crypto/rand`, `crypto/sha256`, `crypto/x509`, `encoding/pem` ## Security Considerations - Private keys encrypted at rest in the barrier, zeroized on seal. - DEK is random 32 bytes per-encryption; never reused. - HKDF salt is 32 bytes of `crypto/rand` randomness, generated fresh per recipient per encryption. Stored in the envelope alongside the wrapped DEK. A random salt ensures that even if the same sender-recipient pair encrypts multiple messages, the derived wrapping keys are unique. - HKDF info string includes sender and recipient identities to prevent key confusion attacks: `info = "metacrypt-user-v1:" + sender + ":" + recipient`. - DEK wrapping uses AES-256-GCM (not AES Key Wrap / RFC 3394). Both provide authenticated encryption; AES-GCM is preferred for consistency with the rest of the codebase and avoids adding a new primitive. - All intermediate secrets (shared secrets, wrapping keys, DEKs) are zeroized immediately after use using `crypto.Zeroize`. - Envelope includes sender identity so the recipient can derive the correct shared secret. - Key rotation is destructive — old data cannot be decrypted. The engine should warn and require explicit confirmation (admin or self only). - Server-trust model: the server holds all private keys in the barrier. No API surface exports private keys. Access control is application-enforced — the engine only uses a private key on behalf of its owner during encrypt/decrypt. - Auto-provisioning creates key pairs for unregistered recipients. Before creating a key pair, the engine validates that the recipient username exists in MCIAS via `auth.ValidateUsername`. This prevents barrier pollution from non-existent usernames. Auto-provisioning is safe because: (a) the recipient must be a real MCIAS user, (b) no MCIAS permissions are granted, (c) the private key is only usable after MCIAS authentication, (d) key pairs are stored identically to explicitly registered users. Auto-provisioning is only triggered by authenticated users during `encrypt`. - Encrypt requests are limited to 100 recipients to prevent resource exhaustion from ECDH + HKDF computation. - Metadata in the envelope is authenticated (included as additional data in AEAD) but not encrypted — it is visible to anyone holding the envelope. - Post-quantum readiness: the `key_algorithm` config supports future hybrid schemes (e.g. X25519 + ML-KEM). The envelope version field enables migration. - X25519 is the default algorithm because it provides 128-bit security with the smallest key size and fastest operations. NIST curves are offered for compliance contexts. ## Implementation References These existing code patterns should be followed exactly: | Pattern | Reference File | Lines | |---------|---------------|-------| | HandleRequest switch dispatch | `internal/engine/ca/ca.go` | 284–317 | | zeroizeKey helper | `internal/engine/ca/ca.go` | 1481–1498 | | REST route registration with chi | `internal/server/routes.go` | 38–50 | | gRPC handler structure | `internal/grpcserver/ca.go` | full file | | gRPC interceptor maps | `internal/grpcserver/server.go` | 107–205 | | Engine factory registration | `cmd/metacrypt/server.go` | 76 | | adminOnlyOperations map | `internal/server/routes.go` | 265–285 |