mc/metacrypt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update engine specs, audit doc, and server tests for SSH CA, transit, and user engines

Browse Source 
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Kyle Isom
2026-03-16 20:16:23 -07:00
parent 7237b2951e
commit 128f5abc4d
6 changed files with 1309 additions and 182 deletions
Download Patch File Download Diff File Expand all files Collapse all files

368
engines/transit.md
View File

@@ -38,7 +38,7 @@ The transit engine manages **named encryption keys**. Each key has:
| Type | Algorithm | Operations |
|-----------------|-------------------|------------------|
| `aes256-gcm` | AES-256-GCM | Encrypt, Decrypt |
| `chacha20-poly` | ChaCha20-Poly1305 | Encrypt, Decrypt |
| `chacha20-poly` | XChaCha20-Poly1305 | Encrypt, Decrypt |
| `ed25519` | Ed25519 | Sign, Verify |
| `ecdsa-p256` | ECDSA P-256 | Sign, Verify |
| `ecdsa-p384` | ECDSA P-384 | Sign, Verify |
@@ -49,6 +49,71 @@ RSA key types are intentionally excluded. The transit engine is not the right
place for RSA — asymmetric encryption belongs in the user engine (via ECDH),
and RSA signing offers no advantage over Ed25519/ECDSA for this use case.
### Cryptographic Details
**Nonce sizes:**
- `aes256-gcm`: 12-byte nonce via `cipher.AEAD.NonceSize()` (standard GCM).
- `chacha20-poly`: 24-byte nonce via `chacha20poly1305.NewX()` (XChaCha20-
Poly1305). The `X` variant is used specifically because it has a large
enough nonce (192-bit) for safe random generation without birthday-bound
concerns. Use `chacha20poly1305.NonceSizeX` (24).
**Nonce generation:** Always `crypto/rand.Read(nonce)`. Never use a counter —
keys may be used concurrently from multiple goroutines.
**Signing algorithms:**
- `ed25519`: Direct Ed25519 signing (`ed25519.Sign`). The input is the raw
message — Ed25519 performs its own internal SHA-512 hashing. No prehash.
- `ecdsa-p256`: SHA-256 hash of input, then `ecdsa.SignASN1(rand, key,
hash)`. Signature is ASN.1 DER encoded (the standard Go representation).
- `ecdsa-p384`: SHA-384 hash of input, then `ecdsa.SignASN1(rand, key,
hash)`. Signature is ASN.1 DER encoded.
The `algorithm` field in sign requests is currently unused (reserved for
future prehash options). Each key type has exactly one hash algorithm; there
is no caller choice.
**Signature format:**
```
metacrypt:v{version}:{base64(signature_bytes)}
```
The `v{version}` identifies which key version was used for signing. For
Ed25519, `signature_bytes` is the raw 64-byte signature. For ECDSA,
`signature_bytes` is the ASN.1 DER encoding.
**Verification:** `verify` parses the version from the signature string, loads
the corresponding public key version, and calls `ed25519.Verify` or
`ecdsa.VerifyASN1` as appropriate.
**HMAC:** `hmac-sha256` uses `hmac.New(sha256.New, key)`, `hmac-sha512` uses
`hmac.New(sha512.New, key)`. Output uses the same versioned prefix format as
ciphertext and signatures:
```
metacrypt:v{version}:{base64(mac_bytes)}
```
The `v{version}` identifies which HMAC key version produced the MAC. This is
essential for HMAC verification after key rotation — without the version
prefix, the engine would not know which key version to use for recomputation.
HMAC verification parses the version, loads the corresponding key (subject to
`min_decryption_version` enforcement), recomputes the MAC, and compares using
`hmac.Equal` for constant-time comparison.
**Key material sizes:**
- `aes256-gcm`: 32 bytes (`crypto/rand`).
- `chacha20-poly`: 32 bytes (`crypto/rand`).
- `ed25519`: `ed25519.GenerateKey(rand.Reader)` — 64-byte private key.
- `ecdsa-p256`: `ecdsa.GenerateKey(elliptic.P256(), rand.Reader)`.
- `ecdsa-p384`: `ecdsa.GenerateKey(elliptic.P384(), rand.Reader)`.
- `hmac-sha256`: 32 bytes (`crypto/rand`).
- `hmac-sha512`: 64 bytes (`crypto/rand`).
**Key serialization in barrier:**
- Symmetric keys: raw bytes.
- Ed25519: `ed25519.PrivateKey` raw bytes (64 bytes).
- ECDSA: PKCS8 DER via `x509.MarshalPKCS8PrivateKey`.
### Key Rotation
Each key has a current version and may retain older versions. Encryption always
@@ -67,6 +132,26 @@ lets operators complete a rotation cycle:
Until `min_decryption_version` is advanced, old versions must be retained.
### `max_key_versions` Behavior
When `max_key_versions` is set (> 0), the engine enforces a soft limit on the
number of retained versions. Pruning happens automatically during `rotate-key`,
after the new version is created:
1. Count total versions. If `<= max_key_versions`, no pruning needed.
2. Identify candidate versions for pruning: versions **strictly less than**
`min_decryption_version`.
3. Delete candidates (oldest first) until the total count is within the limit
or no more candidates remain.
4. If the total still exceeds `max_key_versions` after pruning all eligible
candidates, include a warning in the response:
`"warning": "max_key_versions exceeded; advance min_decryption_version to enable pruning"`.
This ensures `max_key_versions` **never** deletes a version at or above
`min_decryption_version`. The operator must complete the rotation cycle
(rotate → rewrap → advance min) before old versions become prunable.
`max_key_versions` is a safety net, not a foot-gun.
### Ciphertext Format
Transit ciphertexts use a versioned prefix:
@@ -114,19 +199,32 @@ type keyVersion struct {
### Initialize
1. Parse and store config in barrier.
2. No keys are created at init time (keys are created on demand).
1. Parse and validate config: parse `max_key_versions` as integer (must be ≥ 0).
2. Store config in barrier as `{mountPath}config.json`:
```go
configJSON, _ := json.Marshal(config)
barrier.Put(ctx, mountPath+"config.json", configJSON)
```
3. No keys are created at init time (keys are created on demand via
`create-key`).
### Unseal
1. Load config from barrier.
2. Discover and load all named keys and their versions from the barrier.
1. Load config JSON from barrier, unmarshal into `*TransitConfig`.
2. List all key directories under `{mountPath}keys/`.
3. For each key, load `config.json` and all `v{N}.key` entries:
- Symmetric keys (`aes256-gcm`, `chacha20-poly`, `hmac-*`): raw 32-byte
or 64-byte key material.
- Ed25519: `ed25519.PrivateKey` (64 bytes), derive public key.
- ECDSA: parse PKCS8 DER → `*ecdsa.PrivateKey`, extract `PublicKey`.
4. Populate `keys` map with all loaded key states.
### Seal
1. Zeroize all key material (symmetric keys overwritten with zeros,
asymmetric keys via `zeroizeKey`).
2. Nil out all maps.
1. Zeroize all key material: symmetric keys overwritten with zeros via
`crypto.Zeroize(key)`, asymmetric keys via `engine.ZeroizeKey(privKey)`
(shared helper, see sshca.md Implementation References).
2. Nil out `keys` map and `config`.
## Operations
@@ -150,6 +248,53 @@ type keyVersion struct {
| `hmac` | User+Policy | Compute HMAC with an HMAC key |
| `get-public-key` | User/Admin | Get public key for asymmetric keys |
### HandleRequest dispatch
Follow the CA engine's pattern (`internal/engine/ca/ca.go:284-317`):
```go
func (e *TransitEngine) HandleRequest(ctx context.Context, req *engine.Request) (*engine.Response, error) {
switch req.Operation {
case "create-key":
return e.handleCreateKey(ctx, req)
case "delete-key":
return e.handleDeleteKey(ctx, req)
case "get-key":
return e.handleGetKey(ctx, req)
case "list-keys":
return e.handleListKeys(ctx, req)
case "rotate-key":
return e.handleRotateKey(ctx, req)
case "update-key-config":
return e.handleUpdateKeyConfig(ctx, req)
case "trim-key":
return e.handleTrimKey(ctx, req)
case "encrypt":
return e.handleEncrypt(ctx, req)
case "decrypt":
return e.handleDecrypt(ctx, req)
case "rewrap":
return e.handleRewrap(ctx, req)
case "batch-encrypt":
return e.handleBatchEncrypt(ctx, req)
case "batch-decrypt":
return e.handleBatchDecrypt(ctx, req)
case "batch-rewrap":
return e.handleBatchRewrap(ctx, req)
case "sign":
return e.handleSign(ctx, req)
case "verify":
return e.handleVerify(ctx, req)
case "hmac":
return e.handleHmac(ctx, req)
case "get-public-key":
return e.handleGetPublicKey(ctx, req)
default:
return nil, fmt.Errorf("transit: unknown operation: %s", req.Operation)
}
}
```
### create-key
Request data:
@@ -158,9 +303,14 @@ Request data:
|-------------------|----------|----------------|----------------------------------|
| `name` | Yes | | Key name |
| `type` | Yes | | Key type (see table above) |
| `exportable` | No | `false` | Whether raw key material can be exported |
| `allow_deletion` | No | `false` | Whether key can be deleted |
The `exportable` flag has been intentionally omitted. Transit's value
proposition is that keys never leave the service — all cryptographic operations
happen server-side. If key export is ever needed (e.g., for migration), a
dedicated admin-only export operation can be added with appropriate audit
logging.
The key is created at version 1 with `min_decryption_version` = 1.
### encrypt
@@ -199,11 +349,12 @@ Request data:
|-------------|----------|--------------------------------------------|
| `key` | Yes | Named key (Ed25519 or ECDSA type) |
| `input` | Yes | Base64-encoded data to sign |
| `algorithm` | No | Hash algorithm (default varies by key type) |
| `algorithm` | No | Reserved for future prehash options (currently ignored) |
The engine rejects `sign` requests for HMAC key types with an error.
The engine rejects `sign` requests for HMAC and symmetric key types with an
error. Only Ed25519 and ECDSA keys are accepted.
Response: `{ "signature": "metacrypt:v1:..." }`
Response: `{ "signature": "metacrypt:v{version}:...", "key_version": N }`
### verify
@@ -236,9 +387,9 @@ exceed the current version (you must always be able to decrypt with the latest).
### trim-key
Admin-only. Permanently deletes key versions older than `min_decryption_version`.
This is irreversible — ciphertext encrypted with trimmed versions can never be
decrypted.
Admin-only. Permanently deletes key versions **strictly less than**
`min_decryption_version`. This is irreversible — ciphertext encrypted with
trimmed versions can never be decrypted.
Request data:
@@ -246,6 +397,24 @@ Request data:
|-------|----------|-------------|
| `key` | Yes | Named key |
Deletion logic:
1. Load the key's `min_decryption_version` (must be > 1, otherwise no-op).
2. Enumerate all version files: `{mountPath}keys/{name}/v{N}.key`.
3. For each version `N` where `N < min_decryption_version`:
- Zeroize the in-memory key material (`crypto.Zeroize` for symmetric,
`engine.ZeroizeKey` for asymmetric).
- Delete the version from the barrier: `barrier.Delete(ctx, versionPath)`.
- Remove from the in-memory `versions` map.
4. Return the list of trimmed version numbers.
If `min_decryption_version` is 1 (the default), trim-key is a no-op and
returns an empty list. This ensures you cannot accidentally trim all versions
without first explicitly advancing the minimum.
The current version is **never** trimmable — `min_decryption_version` cannot
exceed the current version (enforced by `update-key-config`), so the latest
version is always retained.
Response: `{ "trimmed_versions": [1, 2, ...] }`
## Batch Operations
@@ -346,13 +515,25 @@ Each result:
| `reference` | Echoed from the request item (if provided) |
| `error` | Error message on failure, empty on success |
### Batch Size Limits
Each batch request is limited to **500 items**. Requests exceeding this limit
are rejected before processing with a `400 Bad Request` / `InvalidArgument`
error. This prevents a single request from monopolizing the engine's lock and
memory.
The limit is a compile-time constant (`maxBatchSize = 500`) in the engine
package. It can be tuned if needed but should not be exposed as user-
configurable — it exists as a safety valve, not a feature.
### Implementation Notes
Batch operations are handled inside the transit engine's `HandleRequest` as
three additional operation cases (`batch-encrypt`, `batch-decrypt`,
`batch-rewrap`). No changes to the `Engine` interface are needed. The engine
loops over items internally, loading the key once and reusing it for all items
in the batch.
acquires a read lock once, loads the key once, and processes all items in the
batch while holding the lock. This ensures atomicity with respect to key
rotation (all items in a batch use the same key version).
The `reference` field is opaque to the engine — it allows callers to correlate
results with their source records (e.g. a database row ID) without maintaining
@@ -367,6 +548,10 @@ Follows the same model as the CA engine:
`encrypt`, `decrypt`, `sign`, `verify`, `hmac` for cryptographic operations;
`read` for metadata (get-key, list-keys, get-public-key); `write` for
management (create-key, delete-key, rotate-key, update-key-config, trim-key).
`rewrap` maps to the `decrypt` action — rewrap internally decrypts with the
old version and re-encrypts with the latest, so the caller must have decrypt
permission. Batch variants (`batch-encrypt`, `batch-decrypt`, `batch-rewrap`)
map to the same action as their single counterparts.
The `any` action matches all of the above (but never `admin`).
- No ownership concept (transit keys are shared resources); access is purely
policy-based.
@@ -417,48 +602,151 @@ All auth required:
| POST | `/v1/transit/{mount}/sign/{key}` | Sign |
| POST | `/v1/transit/{mount}/verify/{key}` | Verify |
| POST | `/v1/transit/{mount}/hmac/{key}` | HMAC |
| GET | `/v1/transit/{mount}/keys/{name}/public-key` | Get public key |
All operations are also accessible via the generic `POST /v1/engine/request`.
### REST Route Registration
Add to `internal/server/routes.go` in `registerRoutes`, following the CA
engine's pattern with `chi.URLParam`:
```go
// Transit key management routes (admin).
r.Post("/v1/transit/{mount}/keys", s.requireAdmin(s.handleTransitCreateKey))
r.Get("/v1/transit/{mount}/keys", s.requireAuth(s.handleTransitListKeys))
r.Get("/v1/transit/{mount}/keys/{name}", s.requireAuth(s.handleTransitGetKey))
r.Delete("/v1/transit/{mount}/keys/{name}", s.requireAdmin(s.handleTransitDeleteKey))
r.Post("/v1/transit/{mount}/keys/{name}/rotate", s.requireAdmin(s.handleTransitRotateKey))
r.Patch("/v1/transit/{mount}/keys/{name}/config", s.requireAdmin(s.handleTransitUpdateKeyConfig))
r.Post("/v1/transit/{mount}/keys/{name}/trim", s.requireAdmin(s.handleTransitTrimKey))
// Transit crypto operations (auth + policy).
r.Post("/v1/transit/{mount}/encrypt/{key}", s.requireAuth(s.handleTransitEncrypt))
r.Post("/v1/transit/{mount}/decrypt/{key}", s.requireAuth(s.handleTransitDecrypt))
r.Post("/v1/transit/{mount}/rewrap/{key}", s.requireAuth(s.handleTransitRewrap))
r.Post("/v1/transit/{mount}/batch/encrypt/{key}", s.requireAuth(s.handleTransitBatchEncrypt))
r.Post("/v1/transit/{mount}/batch/decrypt/{key}", s.requireAuth(s.handleTransitBatchDecrypt))
r.Post("/v1/transit/{mount}/batch/rewrap/{key}", s.requireAuth(s.handleTransitBatchRewrap))
r.Post("/v1/transit/{mount}/sign/{key}", s.requireAuth(s.handleTransitSign))
r.Post("/v1/transit/{mount}/verify/{key}", s.requireAuth(s.handleTransitVerify))
r.Post("/v1/transit/{mount}/hmac/{key}", s.requireAuth(s.handleTransitHmac))
r.Get("/v1/transit/{mount}/keys/{name}/public-key", s.requireAuth(s.handleTransitGetPublicKey))
```
Each handler extracts `chi.URLParam(r, "mount")` and `chi.URLParam(r, "key")`
or `chi.URLParam(r, "name")`, builds an `engine.Request`, and calls
`s.engines.HandleRequest(...)`.
### gRPC Interceptor Maps
Add to `sealRequiredMethods`, `authRequiredMethods`, and `adminRequiredMethods`
in `internal/grpcserver/server.go`:
```go
// sealRequiredMethods — all transit RPCs:
"/metacrypt.v2.TransitService/CreateKey": true,
"/metacrypt.v2.TransitService/DeleteKey": true,
"/metacrypt.v2.TransitService/GetKey": true,
"/metacrypt.v2.TransitService/ListKeys": true,
"/metacrypt.v2.TransitService/RotateKey": true,
"/metacrypt.v2.TransitService/UpdateKeyConfig": true,
"/metacrypt.v2.TransitService/TrimKey": true,
"/metacrypt.v2.TransitService/Encrypt": true,
"/metacrypt.v2.TransitService/Decrypt": true,
"/metacrypt.v2.TransitService/Rewrap": true,
"/metacrypt.v2.TransitService/BatchEncrypt": true,
"/metacrypt.v2.TransitService/BatchDecrypt": true,
"/metacrypt.v2.TransitService/BatchRewrap": true,
"/metacrypt.v2.TransitService/Sign": true,
"/metacrypt.v2.TransitService/Verify": true,
"/metacrypt.v2.TransitService/Hmac": true,
"/metacrypt.v2.TransitService/GetPublicKey": true,
// authRequiredMethods — all transit RPCs:
"/metacrypt.v2.TransitService/CreateKey": true,
"/metacrypt.v2.TransitService/DeleteKey": true,
"/metacrypt.v2.TransitService/GetKey": true,
"/metacrypt.v2.TransitService/ListKeys": true,
"/metacrypt.v2.TransitService/RotateKey": true,
"/metacrypt.v2.TransitService/UpdateKeyConfig": true,
"/metacrypt.v2.TransitService/TrimKey": true,
"/metacrypt.v2.TransitService/Encrypt": true,
"/metacrypt.v2.TransitService/Decrypt": true,
"/metacrypt.v2.TransitService/Rewrap": true,
"/metacrypt.v2.TransitService/BatchEncrypt": true,
"/metacrypt.v2.TransitService/BatchDecrypt": true,
"/metacrypt.v2.TransitService/BatchRewrap": true,
"/metacrypt.v2.TransitService/Sign": true,
"/metacrypt.v2.TransitService/Verify": true,
"/metacrypt.v2.TransitService/Hmac": true,
"/metacrypt.v2.TransitService/GetPublicKey": true,
// adminRequiredMethods — admin-only transit RPCs:
"/metacrypt.v2.TransitService/CreateKey": true,
"/metacrypt.v2.TransitService/DeleteKey": true,
"/metacrypt.v2.TransitService/RotateKey": true,
"/metacrypt.v2.TransitService/UpdateKeyConfig": true,
"/metacrypt.v2.TransitService/TrimKey": true,
```
The `adminOnlyOperations` map in `routes.go` already contains transit entries
(qualified as `transit:create-key`, `transit:delete-key`, etc. — keys are
`engineType:operation` to avoid cross-engine name collisions).
## Web UI
Add to `/dashboard` the ability to mount a transit engine.
Add a `/transit` page displaying:
- Named key list with metadata (type, version, created, exportable)
- Named key list with metadata (type, version, created, allow_deletion)
- Key detail view with version history
- Encrypt/decrypt form for interactive testing
- Key rotation button (admin)
## Implementation Steps
1. **`internal/engine/transit/`** — Implement `TransitEngine`:
1. **Prerequisite**: `engine.ZeroizeKey` must exist in
`internal/engine/helpers.go` (created as part of the SSH CA engine
implementation — see `engines/sshca.md` step 1).
2. **`internal/engine/transit/`** — Implement `TransitEngine`:
- `types.go` — Config, KeyConfig, key version types.
- `transit.go` — Lifecycle (Initialize, Unseal, Seal, HandleRequest).
- `encrypt.go` — Encrypt/Decrypt/Rewrap operations.
- `sign.go` — Sign/Verify/HMAC operations.
- `keys.go` — Key management (create, delete, rotate, list, get).
2. **Register factory** in `cmd/metacrypt/main.go`.
3. **Proto definitions** — `proto/metacrypt/v2/transit.proto`, run `make proto`.
4. **gRPC handlers** — `internal/grpcserver/transit.go`.
5. **REST routes** — Add to `internal/server/routes.go`.
6. **Web UI** — Add template + webserver routes.
7. **Tests** — Unit tests for each operation, key rotation, rewrap correctness.
3. **Register factory** in `cmd/metacrypt/main.go`.
4. **Proto definitions** — `proto/metacrypt/v2/transit.proto`, run `make proto`.
5. **gRPC handlers** — `internal/grpcserver/transit.go`.
6. **REST routes** — Add to `internal/server/routes.go`.
7. **Web UI** — Add template + webserver routes.
8. **Tests** — Unit tests for each operation, key rotation, rewrap correctness.
## Dependencies
- `golang.org/x/crypto/chacha20poly1305` (for ChaCha20-Poly1305 key type)
- `golang.org/x/crypto/chacha20poly1305` (for XChaCha20-Poly1305 key type)
- Standard library `crypto/aes`, `crypto/cipher`, `crypto/ecdsa`,
`crypto/ed25519`, `crypto/hmac`, `crypto/sha256`, `crypto/sha512`
`crypto/ed25519`, `crypto/hmac`, `crypto/sha256`, `crypto/sha512`,
`crypto/elliptic`, `crypto/x509`, `crypto/rand`
## Security Considerations
- All key material encrypted at rest in the barrier, zeroized on seal.
- Symmetric keys generated with `crypto/rand`.
- XChaCha20-Poly1305 used instead of ChaCha20-Poly1305 for its 192-bit nonce,
which is safe for random nonce generation at high volume (birthday bound at
2^96 messages vs 2^48 for 96-bit nonces).
- Nonces are always random (`crypto/rand`), never counter-based, to avoid
nonce-reuse risks from concurrent access or crash recovery.
- Ciphertext format includes version to support key rotation without data loss.
- `exportable` flag is immutable after creation — cannot be enabled later.
- `allow_deletion` is immutable after creation.
- Key export is not supported — transit keys never leave the service.
- `allow_deletion` is immutable after creation; `delete-key` returns an error
if `allow_deletion` is `false`.
- `max_key_versions` pruning only removes old versions, never the current one.
- `trim-key` only deletes versions below `min_decryption_version`, and
`min_decryption_version` cannot exceed the current version. This guarantees
the current version is never trimmable.
- Rewrap operation never exposes plaintext to the caller.
- Context (AAD) binding prevents ciphertext from being used in a different context.
- `min_decryption_version` enforces key rotation completion: once advanced,
@@ -466,3 +754,25 @@ Add a `/transit` page displaying:
- RSA key types are excluded to avoid padding scheme vulnerabilities
(Bleichenbacher attacks on PKCS#1 v1.5). Asymmetric encryption belongs in
the user engine; signing uses Ed25519/ECDSA.
- ECDSA signatures use ASN.1 DER encoding (Go's native format), not raw
concatenated (r,s) — this avoids signature malleability issues.
- Ed25519 signs raw messages (no prehash) — this is the standard Ed25519
mode, not Ed25519ph, avoiding the collision resistance reduction.
- Batch operations enforce a 500-item limit to prevent resource exhaustion.
- Batch operations hold a read lock for the entire batch to ensure all items
use the same key version, preventing TOCTOU between key rotation and
encryption.
## Implementation References
These existing code patterns should be followed exactly:
| Pattern | Reference File | Lines |
|---------|---------------|-------|
| HandleRequest switch dispatch | `internal/engine/ca/ca.go` | 284–317 |
| zeroizeKey helper | `internal/engine/ca/ca.go` | 1481–1498 |
| REST route registration with chi | `internal/server/routes.go` | 38–50 |
| gRPC handler structure | `internal/grpcserver/ca.go` | full file |
| gRPC interceptor maps | `internal/grpcserver/server.go` | 107–205 |
| Engine factory registration | `cmd/metacrypt/server.go` | 76 |
| adminOnlyOperations map | `internal/server/routes.go` | 265–285 |