Update engine specs, audit doc, and server tests for SSH CA, transit, and user engines

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-16 20:16:23 -07:00
parent 7237b2951e
commit 128f5abc4d
6 changed files with 1309 additions and 182 deletions
--- a/engines/user.md
+++ b/engines/user.md
@@ -54,6 +54,72 @@ encrypted in the barrier and is only used by the engine on behalf of the owning
 user (enforced in `HandleRequest`). The public key is available to any
 authenticated user (needed to encrypt messages to that user).

+**Key generation by algorithm:**
+- `x25519`: `ecdh.X25519().GenerateKey(rand.Reader)` (Go 1.20+ `crypto/ecdh`).
+- `ecdh-p256`: `ecdh.P256().GenerateKey(rand.Reader)`.
+- `ecdh-p384`: `ecdh.P384().GenerateKey(rand.Reader)`.
+
+**Key serialization in barrier:**
+- Private key: `x509.MarshalPKCS8PrivateKey(privKey)` → PEM block with type
+  `"PRIVATE KEY"`. Stored at `{mountPath}users/{username}/priv.pem`.
+- Public key: `x509.MarshalPKIXPublicKey(pubKey)` → PEM block with type
+  `"PUBLIC KEY"`. Stored at `{mountPath}users/{username}/pub.pem`.
+
+Note: `crypto/ecdh` keys implement the interfaces required by
+`x509.MarshalPKCS8PrivateKey` and `x509.MarshalPKIXPublicKey` as of Go 1.20.
+
+### Cryptographic Details
+
+**ECDH key agreement:**
+```go
+sharedSecret, err := senderPrivKey.ECDH(recipientPubKey)
+```
+The raw shared secret is **never used directly** as a key. It is always fed
+through HKDF.
+
+**HKDF key derivation:**
+```go
+salt := make([]byte, 32)
+rand.Read(salt)
+hkdf := hkdf.New(sha256.New, sharedSecret, salt, info)
+wrappingKey := make([]byte, 32) // 256-bit AES key
+io.ReadFull(hkdf, wrappingKey)
+```
+
+- **Hash:** SHA-256 (sufficient for 256-bit key derivation).
+- **Salt:** 32 bytes of `crypto/rand` randomness, generated fresh per
+  recipient per encryption. The salt is stored alongside the wrapped DEK in
+  the envelope (see updated envelope format below).
+- **Info:** `"metacrypt-user-v1:" + sender + ":" + recipient` (UTF-8 encoded).
+  This binds the derived key to the specific sender-recipient pair, preventing
+  key confusion if the same shared secret were somehow reused.
+
+**DEK wrapping:** The wrapping key from HKDF encrypts the DEK using AES-256-GCM
+(not AES Key Wrap / RFC 3394). AES-GCM is used because:
+- It is already a core primitive in the codebase.
+- It provides authenticated encryption, same as AES Key Wrap.
+- The DEK is 32 bytes — well within GCM's plaintext size limits.
+
+```go
+block, _ := aes.NewCipher(wrappingKey)
+gcm, _ := cipher.NewGCM(block)
+nonce := make([]byte, gcm.NonceSize()) // 12 bytes
+rand.Read(nonce)
+wrappedDEK := gcm.Seal(nonce, nonce, dek, nil) // nonce || ciphertext || tag
+```
+
+**Symmetric encryption (payload):**
+```go
+block, _ := aes.NewCipher(dek)
+gcm, _ := cipher.NewGCM(block)
+nonce := make([]byte, gcm.NonceSize()) // 12 bytes
+rand.Read(nonce)
+ciphertext := gcm.Seal(nonce, nonce, plaintext, aad)
+```
+- AAD: if `metadata` is provided, it is used as additional authenticated data.
+  This means metadata is integrity-protected but not encrypted.
+- Nonce: 12 bytes from `crypto/rand`.
+
 ### Encryption Flow (Sender → Recipient)

 1. Sender calls `encrypt` with plaintext, recipient username(s), and optional
@@ -82,13 +148,24 @@ authenticated user (needed to encrypt messages to that user).
    "sender": "alice",
    "sym_algorithm": "aes256-gcm",
    "ciphertext": "<base64(nonce + encrypted_payload + tag)>",
+    "metadata": "<optional plaintext metadata>",
    "recipients": {
-        "bob": "<base64(wrapped_dek)>",
-        "carol": "<base64(wrapped_dek)>"
+        "bob": {
+            "salt": "<base64(32-byte HKDF salt)>",
+            "wrapped_dek": "<base64(nonce + encrypted_dek + tag)>"
+        },
+        "carol": {
+            "salt": "<base64(32-byte HKDF salt)>",
+            "wrapped_dek": "<base64(nonce + encrypted_dek + tag)>"
+        }
    }
 }
 ```

+Each recipient entry includes:
+- `salt`: the per-recipient random HKDF salt used during key derivation.
+- `wrapped_dek`: the AES-256-GCM encryption of the DEK (nonce-prepended).
+
 The envelope is base64-encoded as a single opaque blob for transport.

 ## Barrier Storage Layout
@@ -116,24 +193,43 @@ type userState struct {
    pubKey  crypto.PublicKey   // key exchange public key
    config  *UserKeyConfig
 }
+
+type UserKeyConfig struct {
+    Algorithm       string    `json:"algorithm"`         // key exchange algorithm (x25519, ecdh-p256, ecdh-p384)
+    CreatedAt       time.Time `json:"created_at"`
+    AutoProvisioned bool      `json:"auto_provisioned"`  // true if created via auto-provisioning
+}
 ```

 ## Lifecycle

 ### Initialize

-1. Parse and store config in barrier.
-2. No user keys are created at init time (created on demand or via `register`).
+1. Parse and validate config: ensure `key_algorithm` is one of `x25519`,
+   `ecdh-p256`, `ecdh-p384`. Ensure `sym_algorithm` is `aes256-gcm`.
+2. Store config in barrier as `{mountPath}config.json`:
+   ```go
+   configJSON, _ := json.Marshal(config)
+   barrier.Put(ctx, mountPath+"config.json", configJSON)
+   ```
+3. No user keys are created at init time (created on demand via `register`,
+   `provision`, or auto-provisioning).

 ### Unseal

-1. Load config from barrier.
-2. Discover and load all user key pairs from barrier.
+1. Load config JSON from barrier, unmarshal into `*UserConfig`.
+2. List all user directories under `{mountPath}users/`.
+3. For each user, load `priv.pem` and `pub.pem`:
+   - Parse private key PEM: `pem.Decode` → `x509.ParsePKCS8PrivateKey` →
+     type-assert to `*ecdh.PrivateKey`.
+   - Parse public key PEM: `pem.Decode` → `x509.ParsePKIXPublicKey` →
+     type-assert to `*ecdh.PublicKey`.
+4. Populate `users` map with loaded key states.

 ### Seal

-1. Zeroize all private key material.
-2. Nil out all maps.
+1. Zeroize all private key material using `engine.ZeroizeKey(privKey)`.
+2. Nil out `users` map and `config`.

 ## Operations

@@ -145,9 +241,41 @@ type userState struct {
 | `list-users`     | User/Admin    | List registered users                            |
 | `encrypt`        | User+Policy   | Encrypt data for one or more recipients          |
 | `decrypt`        | User (self)   | Decrypt an envelope addressed to the caller      |
+| `re-encrypt`     | User (self)   | Re-encrypt an envelope with current key pairs    |
 | `rotate-key`     | User (self)   | Rotate the caller's key pair                     |
 | `delete-user`    | Admin         | Remove a user's key pair                         |

+### HandleRequest dispatch
+
+Follow the CA engine's pattern (`internal/engine/ca/ca.go:284-317`):
+
+```go
+func (e *UserEngine) HandleRequest(ctx context.Context, req *engine.Request) (*engine.Response, error) {
+    switch req.Operation {
+    case "register":
+        return e.handleRegister(ctx, req)
+    case "provision":
+        return e.handleProvision(ctx, req)
+    case "get-public-key":
+        return e.handleGetPublicKey(ctx, req)
+    case "list-users":
+        return e.handleListUsers(ctx, req)
+    case "encrypt":
+        return e.handleEncrypt(ctx, req)
+    case "decrypt":
+        return e.handleDecrypt(ctx, req)
+    case "re-encrypt":
+        return e.handleReEncrypt(ctx, req)
+    case "rotate-key":
+        return e.handleRotateKey(ctx, req)
+    case "delete-user":
+        return e.handleDeleteUser(ctx, req)
+    default:
+        return nil, fmt.Errorf("user: unknown operation: %s", req.Operation)
+    }
+}
+```
+
 ### register

 Creates a key pair for the authenticated caller. No-op if the caller already
@@ -177,14 +305,27 @@ Request data:
 | `metadata`   | No       | Arbitrary string metadata (authenticated) |

 Flow:
-1. Caller must be provisioned (has a key pair). Auto-provision if not.
-2. For each recipient without a keypair: auto-provision them.
-3. Load sender's private key and each recipient's public key.
-4. Generate random DEK, encrypt plaintext with DEK.
-5. For each recipient: ECDH(sender_priv, recipient_pub) → shared_secret,
-   HKDF(shared_secret, salt, info) → wrapping_key, AES-KeyWrap(wrapping_key,
-   DEK) → wrapped_dek.
-6. Build and return envelope.
+1. Validate that `len(recipients) <= maxRecipients` (100). Reject with
+   `400 Bad Request` if exceeded.
+2. Caller must be provisioned (has a key pair). If not, auto-provision the
+   caller (generate keypair, store in barrier). This is safe because the
+   caller is already authenticated via MCIAS — their identity is verified.
+3. For each recipient without a keypair: validate the username exists in MCIAS
+   via `auth.ValidateUsername(username)`. If the user does not exist, return an
+   error: `"recipient not found: {username}"`. If the user exists, auto-provision
+   them. Auto-provisioning only creates a key pair; it does not grant any MCIAS
+   roles or permissions. The recipient's private key is only accessible when
+   they authenticate.
+4. Load sender's private key and each recipient's public key.
+5. Generate random 32-byte DEK (`crypto/rand`). Encrypt plaintext with DEK
+   using AES-256-GCM (metadata as AAD if present).
+6. For each recipient:
+   - `sharedSecret := senderPrivKey.ECDH(recipientPubKey)`
+   - Generate 32-byte random salt.
+   - `wrappingKey := HKDF(sha256, sharedSecret, salt, info)`
+   - `wrappedDEK := AES-GCM-Encrypt(wrappingKey, DEK)`
+7. Build envelope with ciphertext, per-recipient `{salt, wrapped_dek}`.
+8. Zeroize DEK, all shared secrets, and all wrapping keys.

 Authorization:
 - Admins: grant-all.
@@ -201,19 +342,48 @@ Request data:
 | `envelope` | Yes      | Base64-encoded envelope blob   |

 Flow:
-1. Parse envelope, find the caller's wrapped DEK entry.
-2. Load sender's public key and caller's private key.
-3. ECDH(caller_priv, sender_pub) → shared_secret → wrapping_key → DEK.
-4. Decrypt ciphertext with DEK.
-5. Return plaintext.
+1. Parse envelope JSON, find the caller's entry in `recipients`.
+   If the caller is not a recipient, return an error.
+2. Load sender's public key (from `envelope.sender`) and caller's private key.
+3. `sharedSecret := callerPrivKey.ECDH(senderPubKey)`.
+4. `wrappingKey := HKDF(sha256, sharedSecret, recipient.salt, info)`.
+5. `dek := AES-GCM-Decrypt(wrappingKey, recipient.wrapped_dek)`.
+6. Decrypt ciphertext with DEK (metadata as AAD if present in envelope).
+7. Zeroize DEK, shared secret, wrapping key.
+8. Return plaintext.

 A user can only decrypt envelopes addressed to themselves.

+### re-encrypt
+
+Re-encrypts an envelope with the caller's current key pair. This is the safe
+way to migrate data before a key rotation.
+
+Request data:
+
+| Field      | Required | Description                    |
+|------------|----------|--------------------------------|
+| `envelope` | Yes      | Base64-encoded envelope blob   |
+
+Flow:
+1. Decrypt the envelope (same as `decrypt` flow).
+2. Re-encrypt the plaintext for the same recipients using fresh DEKs and
+   current key pairs (same as `encrypt` flow, preserving metadata).
+3. Return the new envelope.
+
+The caller must be a recipient in the original envelope. The new envelope uses
+current key pairs for all recipients — if any recipient has rotated their key
+since the original encryption, the new envelope uses their new public key.
+
 ### rotate-key

 Generates a new key pair for the caller. The old private key is zeroized and
 deleted. Old envelopes encrypted with the previous key cannot be decrypted
-after rotation — callers should re-encrypt any stored data before rotating.
+after rotation.
+
+**Recommended workflow**: Before rotating, re-encrypt all stored envelopes
+using the `re-encrypt` operation. Then call `rotate-key`. This ensures no
+data is lost.

 ## gRPC Service (proto/metacrypt/v2/user.proto)

@@ -225,6 +395,7 @@ service UserService {
    rpc ListUsers(UserListUsersRequest)          returns (UserListUsersResponse);
    rpc Encrypt(UserEncryptRequest)              returns (UserEncryptResponse);
    rpc Decrypt(UserDecryptRequest)              returns (UserDecryptResponse);
+    rpc ReEncrypt(UserReEncryptRequest)          returns (UserReEncryptResponse);
    rpc RotateKey(UserRotateKeyRequest)          returns (UserRotateKeyResponse);
    rpc DeleteUser(UserDeleteUserRequest)        returns (UserDeleteUserResponse);
 }
@@ -243,10 +414,70 @@ All auth required:
 | DELETE | `/v1/user/{mount}/keys/{username}`      | Delete user (admin)    |
 | POST   | `/v1/user/{mount}/encrypt`              | Encrypt for recipients |
 | POST   | `/v1/user/{mount}/decrypt`              | Decrypt envelope       |
+| POST   | `/v1/user/{mount}/re-encrypt`           | Re-encrypt envelope    |
 | POST   | `/v1/user/{mount}/rotate`               | Rotate caller's key    |

 All operations are also accessible via the generic `POST /v1/engine/request`.

+### REST Route Registration
+
+Add to `internal/server/routes.go` in `registerRoutes`, following the CA
+engine's pattern with `chi.URLParam`:
+
+```go
+// User engine routes.
+r.Post("/v1/user/{mount}/register", s.requireAuth(s.handleUserRegister))
+r.Post("/v1/user/{mount}/provision", s.requireAdmin(s.handleUserProvision))
+r.Get("/v1/user/{mount}/keys", s.requireAuth(s.handleUserListUsers))
+r.Get("/v1/user/{mount}/keys/{username}", s.requireAuth(s.handleUserGetPublicKey))
+r.Delete("/v1/user/{mount}/keys/{username}", s.requireAdmin(s.handleUserDeleteUser))
+r.Post("/v1/user/{mount}/encrypt", s.requireAuth(s.handleUserEncrypt))
+r.Post("/v1/user/{mount}/decrypt", s.requireAuth(s.handleUserDecrypt))
+r.Post("/v1/user/{mount}/re-encrypt", s.requireAuth(s.handleUserReEncrypt))
+r.Post("/v1/user/{mount}/rotate", s.requireAuth(s.handleUserRotateKey))
+```
+
+Each handler extracts `chi.URLParam(r, "mount")` and optionally
+`chi.URLParam(r, "username")`, builds an `engine.Request`, and calls
+`s.engines.HandleRequest(...)`.
+
+### gRPC Interceptor Maps
+
+Add to `sealRequiredMethods`, `authRequiredMethods`, and `adminRequiredMethods`
+in `internal/grpcserver/server.go`:
+
+```go
+// sealRequiredMethods — all user RPCs:
+"/metacrypt.v2.UserService/Register":     true,
+"/metacrypt.v2.UserService/Provision":    true,
+"/metacrypt.v2.UserService/GetPublicKey": true,
+"/metacrypt.v2.UserService/ListUsers":    true,
+"/metacrypt.v2.UserService/Encrypt":      true,
+"/metacrypt.v2.UserService/Decrypt":      true,
+"/metacrypt.v2.UserService/ReEncrypt":    true,
+"/metacrypt.v2.UserService/RotateKey":    true,
+"/metacrypt.v2.UserService/DeleteUser":   true,
+
+// authRequiredMethods — all user RPCs:
+"/metacrypt.v2.UserService/Register":     true,
+"/metacrypt.v2.UserService/Provision":    true,
+"/metacrypt.v2.UserService/GetPublicKey": true,
+"/metacrypt.v2.UserService/ListUsers":    true,
+"/metacrypt.v2.UserService/Encrypt":      true,
+"/metacrypt.v2.UserService/Decrypt":      true,
+"/metacrypt.v2.UserService/ReEncrypt":    true,
+"/metacrypt.v2.UserService/RotateKey":    true,
+"/metacrypt.v2.UserService/DeleteUser":   true,
+
+// adminRequiredMethods — admin-only user RPCs:
+"/metacrypt.v2.UserService/Provision":    true,
+"/metacrypt.v2.UserService/DeleteUser":   true,
+```
+
+The `adminOnlyOperations` map in `routes.go` already contains user entries
+(qualified as `user:provision`, `user:delete-user` — keys are
+`engineType:operation` to avoid cross-engine name collisions).
+
 ## Web UI

 Add to `/dashboard` the ability to mount a user engine.
@@ -260,33 +491,47 @@ Add a `/user-crypto` page displaying:

 ## Implementation Steps

-1. **`internal/engine/user/`** — Implement `UserEngine`:
+1. **Prerequisite**: `engine.ZeroizeKey` must exist in
+   `internal/engine/helpers.go` (created as part of the SSH CA engine
+   implementation — see `engines/sshca.md` step 1).
+
+2. **`internal/engine/user/`** — Implement `UserEngine`:
   - `types.go` — Config types, envelope format.
   - `user.go` — Lifecycle (Initialize, Unseal, Seal, HandleRequest).
   - `crypto.go` — ECDH key agreement, HKDF derivation, DEK wrap/unwrap,
     symmetric encrypt/decrypt.
   - `keys.go` — User registration, key rotation, deletion.
-2. **Register factory** in `cmd/metacrypt/main.go`.
-3. **Proto definitions** — `proto/metacrypt/v2/user.proto`, run `make proto`.
-4. **gRPC handlers** — `internal/grpcserver/user.go`.
-5. **REST routes** — Add to `internal/server/routes.go`.
-6. **Web UI** — Add template + webserver routes.
-7. **Tests** — Unit tests: register, encrypt/decrypt roundtrip, multi-recipient,
-   key rotation invalidates old envelopes, authorization checks.
+3. **Register factory** in `cmd/metacrypt/main.go`.
+4. **Proto definitions** — `proto/metacrypt/v2/user.proto`, run `make proto`.
+5. **gRPC handlers** — `internal/grpcserver/user.go`.
+6. **REST routes** — Add to `internal/server/routes.go`.
+7. **Web UI** — Add template + webserver routes.
+8. **Tests** — Unit tests: register, encrypt/decrypt roundtrip, multi-recipient,
+   key rotation invalidates old envelopes, re-encrypt roundtrip, authorization
+   checks.

 ## Dependencies

 - `golang.org/x/crypto/hkdf` (for key derivation from ECDH shared secret)
 - `crypto/ecdh` (Go 1.20+, for X25519 and NIST curve key exchange)
- Standard library `crypto/aes`, `crypto/cipher`, `crypto/rand`
+- Standard library `crypto/aes`, `crypto/cipher`, `crypto/rand`, `crypto/sha256`,
+  `crypto/x509`, `encoding/pem`

 ## Security Considerations

 - Private keys encrypted at rest in the barrier, zeroized on seal.
- DEK is random per-encryption; never reused.
- HKDF derivation includes sender and recipient identities in the info string
-  to prevent key confusion attacks:
-  `info = "metacrypt-user-v1:" + sender + ":" + recipient`.
+- DEK is random 32 bytes per-encryption; never reused.
+- HKDF salt is 32 bytes of `crypto/rand` randomness, generated fresh per
+  recipient per encryption. Stored in the envelope alongside the wrapped DEK.
+  A random salt ensures that even if the same sender-recipient pair encrypts
+  multiple messages, the derived wrapping keys are unique.
+- HKDF info string includes sender and recipient identities to prevent key
+  confusion attacks: `info = "metacrypt-user-v1:" + sender + ":" + recipient`.
+- DEK wrapping uses AES-256-GCM (not AES Key Wrap / RFC 3394). Both provide
+  authenticated encryption; AES-GCM is preferred for consistency with the rest
+  of the codebase and avoids adding a new primitive.
+- All intermediate secrets (shared secrets, wrapping keys, DEKs) are zeroized
+  immediately after use using `crypto.Zeroize`.
 - Envelope includes sender identity so the recipient can derive the correct
  shared secret.
 - Key rotation is destructive — old data cannot be decrypted. The engine should
@@ -294,9 +539,34 @@ Add a `/user-crypto` page displaying:
 - Server-trust model: the server holds all private keys in the barrier. No API
  surface exports private keys. Access control is application-enforced — the
  engine only uses a private key on behalf of its owner during encrypt/decrypt.
- Auto-provisioned users have keypairs waiting for them; their private keys are
-  protected identically to explicitly registered users.
+- Auto-provisioning creates key pairs for unregistered recipients. Before
+  creating a key pair, the engine validates that the recipient username exists
+  in MCIAS via `auth.ValidateUsername`. This prevents barrier pollution from
+  non-existent usernames. Auto-provisioning is safe because: (a) the recipient
+  must be a real MCIAS user, (b) no MCIAS permissions are granted, (c) the
+  private key is only usable after MCIAS authentication, (d) key pairs are
+  stored identically to explicitly registered users. Auto-provisioning is only
+  triggered by authenticated users during `encrypt`.
+- Encrypt requests are limited to 100 recipients to prevent resource exhaustion
+  from ECDH + HKDF computation.
 - Metadata in the envelope is authenticated (included as additional data in
  AEAD) but not encrypted — it is visible to anyone holding the envelope.
 - Post-quantum readiness: the `key_algorithm` config supports future hybrid
  schemes (e.g. X25519 + ML-KEM). The envelope version field enables migration.
+- X25519 is the default algorithm because it provides 128-bit security with
+  the smallest key size and fastest operations. NIST curves are offered for
+  compliance contexts.
+
+## Implementation References
+
+These existing code patterns should be followed exactly:
+
+| Pattern | Reference File | Lines |
+|---------|---------------|-------|
+| HandleRequest switch dispatch | `internal/engine/ca/ca.go` | 284–317 |
+| zeroizeKey helper | `internal/engine/ca/ca.go` | 1481–1498 |
+| REST route registration with chi | `internal/server/routes.go` | 38–50 |
+| gRPC handler structure | `internal/grpcserver/ca.go` | full file |
+| gRPC interceptor maps | `internal/grpcserver/server.go` | 107–205 |
+| Engine factory registration | `cmd/metacrypt/server.go` | 76 |
+| adminOnlyOperations map | `internal/server/routes.go` | 265–285 |