mc/metacrypt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
b4dbc088cb854a61f64c857e060bf41d91912576
metacrypt/ARCHITECTURE.md

32 KiB
Raw Blame History

Metacrypt Architecture & Specification

Metacrypt is a cryptographic service for the Metacircular platform. It provides cryptographic resources via a modular "engines" architecture, backed by an encrypted storage barrier inspired by HashiCorp Vault. Authentication is delegated to the Metacircular Identity and Access Service (MCIAS).

Table of Contents

  1. System Overview
  2. Key Hierarchy & Cryptographic Design
  3. Seal/Unseal Lifecycle
  4. Encrypted Storage Barrier
  5. Authentication & Authorization
  6. Engine Architecture
  7. API Surface
  8. Web Interface
  9. Database Schema
  10. Configuration
  11. Deployment
  12. Security Model
  13. Future Work

System Overview

┌─────────────────────────────────────────┐
│         Clients (CLI / Web UI)          │
└──────────────────┬──────────────────────┘
                   │ HTTPS (TLS 1.2+)
┌──────────────────▼──────────────────────┐
│  HTTP API & Web Routes (server/)        │
│   ├── Seal/Unseal endpoints             │
│   ├── Auth endpoints (MCIAS delegation) │
│   ├── Engine request routing            │
│   └── Policy management                 │
├─────────────────────────────────────────┤
│  Middleware                             │
│   ├── Structured logging                │
│   ├── Token authentication              │
│   ├── Policy authorization              │
│   └── Unseal-state gating               │
├─────────────────────────────────────────┤
│  Service Layer                          │
│   ├── Seal Manager     (seal/)          │
│   ├── Authenticator    (auth/)          │
│   ├── Policy Engine    (policy/)        │
│   └── Engine Registry  (engine/)        │
├─────────────────────────────────────────┤
│  Encrypted Storage Barrier (barrier/)   │
│   └── AES-256-GCM per-entry encryption  │
├─────────────────────────────────────────┤
│  Database Layer (db/)                   │
│   └── SQLite (WAL mode, foreign keys)   │
└─────────────────────────────────────────┘

Package Layout

cmd/metacrypt/          CLI entry point (server, init, status, snapshot)
internal/
  config/               TOML configuration loading & validation
  crypto/               Low-level cryptographic primitives
  db/                   SQLite setup & schema migrations
  seal/                 Seal/unseal state machine
  barrier/              Encrypted key-value storage abstraction
  auth/                 MCIAS token authentication & caching
  policy/               Priority-based ACL engine
  engine/               Pluggable engine registry & interface
    ca/                 CA (PKI) engine — X.509 certificate issuance
  server/               REST API HTTP server, routes, middleware
  grpcserver/           gRPC server, interceptors, per-service handlers
  webserver/            Web UI HTTP server, routes, HTMX handlers
proto/metacrypt/
  v1/                   Original gRPC proto definitions (generic Execute RPC)
  v2/                   Typed gRPC proto definitions (per-operation RPCs, Timestamp fields)
gen/metacrypt/v1/       Generated Go gRPC/protobuf code (v1)
web/
  templates/            Go HTML templates (layout, init, unseal, login, dashboard, PKI)
  static/               CSS, HTMX
deploy/                 Docker Compose, example configs

Key Hierarchy & Cryptographic Design

Primitives

Purpose Algorithm Parameters
Key derivation Argon2id 3 iterations, 128 MiB, 4 threads (configurable)
Symmetric encryption AES-256-GCM 256-bit keys, 12-byte random nonce
Key size 256 bits All symmetric keys
Salt size 256 bits Argon2id salt
CSPRNG crypto/rand Keys, salts, nonces
Constant-time comparison crypto/subtle Password & token comparison
Zeroization Explicit overwrite MEK, KWK, passwords in memory

Key Hierarchy

User Password (not stored)
      │
      ▼
┌─────────────────────────────────────┐
│ Argon2id(password, salt, params)    │  salt + params stored in seal_config
└──────────────────┬──────────────────┘
                   ▼
          Key Wrap Key (KWK)              256-bit, ephemeral (derived on unseal)
                   │
                   ▼
          ┌────────────────┐
          │ AES-256-GCM    │              encrypted_mek stored in seal_config
          │ Decrypt         │
          └───────┬────────┘
                  ▼
       Master Encryption Key (MEK)        256-bit, held in memory only when unsealed
                  │
                  ▼
       ┌──────────────────────┐
       │ Barrier Entries       │           Each entry encrypted individually
       │  ├── Policy rules     │           with MEK via AES-256-GCM
       │  ├── Engine configs   │
       │  └── Engine DEKs      │           Per-engine data encryption keys
       └──────────────────────┘

Ciphertext Format

All encrypted values use a versioned binary format:

[version: 1 byte][nonce: 12 bytes][ciphertext + GCM tag]

The version byte (currently 0x01) enables future algorithm migration, including post-quantum hybrid schemes.

Seal/Unseal Lifecycle

Metacrypt operates as a state machine with four states:

                  ┌────────────────┐
                  │ Uninitialized  │  No seal_config row in DB
                  └───────┬────────┘
                          │ Initialize(password)
                          │  • generate salt
                          │  • derive KWK = Argon2id(password, salt)
                          │  • generate random MEK
                          │  • store Encrypt(KWK, MEK) + salt + params
                          ▼
 Seal()         ┌──────────────────┐
 ◄──────────────│    Unsealed      │  MEK in memory; barrier open
 │              └──────────────────┘
 │                        ▲
 │                        │ Unseal(password)
 │                        │  • load salt + params + encrypted_mek
 │                        │  • derive KWK = Argon2id(password, salt)
 │                        │  • MEK = Decrypt(KWK, encrypted_mek)
 │                        │  • barrier.Unseal(MEK)
 │              ┌──────────────────┐
 └─────────────►│     Sealed       │  MEK zeroized; barrier locked
                └──────────────────┘

Rate Limiting

Unseal attempts are rate-limited to mitigate online brute-force:

  • 5 attempts within a 1-minute sliding window
  • 60-second lockout after exceeding the limit
  • Counter resets after 1 minute of inactivity

Sealing

Calling Seal() immediately:

  1. Zeroizes the MEK from memory
  2. Seals the storage barrier (all reads/writes return ErrSealed)
  3. Seals all mounted engines
  4. Flushes the authentication token cache

Encrypted Storage Barrier

The barrier provides an encrypted key-value store over the barrier_entries table. Every value is independently encrypted with the MEK using AES-256-GCM.

Interface

type Barrier interface {
    Unseal(mek []byte) error
    Seal() error
    IsSealed() bool
    Get(ctx context.Context, path string) ([]byte, error)
    Put(ctx context.Context, path string, value []byte) error
    Delete(ctx context.Context, path string) error
    List(ctx context.Context, prefix string) ([]string, error)
}

Path Namespace Conventions

Prefix Owner Contents
policy/rules/{id} Policy engine JSON-encoded ACL rules
engine/{type}/{mount}/ Engine Config, keys, engine data
engine/ca/{mount}/root/ CA engine Root CA cert + key
engine/ca/{mount}/issuers/ CA engine Issuer certs, keys, config
engine/ca/{mount}/certs/ CA engine Issued cert records (no private keys)

Properties

  • Encryption at rest: All values encrypted with MEK before database write
  • Fresh nonce per write: Every Put generates a new random nonce
  • Atomic upsert: Uses INSERT ... ON CONFLICT UPDATE for Put
  • Glob listing: List(prefix) returns relative paths matching the prefix
  • Thread-safe: All operations guarded by sync.RWMutex
  • Fail-closed: Returns ErrSealed for any operation when the barrier is sealed

Authentication & Authorization

Authentication (MCIAS Delegation)

Metacrypt does not manage user accounts. All authentication is delegated to MCIAS:

  1. Client sends POST /v1/auth/login with {username, password, totp_code}
  2. Metacrypt forwards credentials to the MCIAS client library
  3. On success, MCIAS returns a bearer token and expiration
  4. Token is returned to the client (also set as metacrypt_token cookie for web UI)
  5. Subsequent requests include Authorization: Bearer <token> or the cookie

Token validation calls MCIAS ValidateToken(), with results cached for 30 seconds (keyed by SHA-256 hash of the token) to reduce MCIAS load.

Admin detection: Users with the admin role in MCIAS are granted admin privileges in Metacrypt.

Authorization (Policy Engine)

The policy engine evaluates access control rules stored in the barrier.

Rule structure:

type Rule struct {
    ID        string    // unique identifier
    Priority  int       // lower number = higher priority
    Effect    Effect    // "allow" or "deny"
    Usernames []string  // match specific users (optional)
    Roles     []string  // match roles (optional)
    Resources []string  // glob patterns, e.g. "engine/transit/*" (optional)
    Actions   []string  // e.g. "read", "write", "admin" (optional)
}

Evaluation algorithm:

  1. If the requester has the admin role, allow immediately (bypass)
  2. Collect all rules where username, role, resource, and action match
  3. Sort matching rules by priority (ascending; lower number = higher priority)
  4. Return the effect of the highest-priority matching rule
  5. Default deny if no rules match

Matching is case-insensitive for usernames and roles. Resources use glob patterns. Empty fields in a rule match everything.

Engine Architecture

Engines are pluggable cryptographic service providers. The CA (PKI) engine is implemented; remaining engine types are planned.

Engine Types

Type Status Purpose
ca Implemented X.509 Certificate Authority (PKI)
sshca Planned SSH Certificate Authority
transit Planned Encrypt/decrypt data in transit (envelope encryption)
user Planned User-to-user end-to-end encryption

Engine Interface

type CallerInfo struct {
    Username string
    Roles    []string
    IsAdmin  bool
}

type Request struct {
    Operation  string
    Path       string
    Data       map[string]interface{}
    CallerInfo *CallerInfo
}

type Engine interface {
    Type() EngineType
    Initialize(ctx context.Context, b barrier.Barrier, mountPath string, config map[string]interface{}) error
    Unseal(ctx context.Context, b barrier.Barrier, mountPath string) error
    Seal() error
    HandleRequest(ctx context.Context, req *Request) (*Response, error)
}

CallerInfo carries authentication context into engine operations, allowing engines to enforce their own auth requirements (e.g. admin-only operations).

Initialize accepts a config map for engine-specific configuration passed at mount time.

Mount Registry

Engines are instantiated through a factory pattern and tracked in a central registry:

registry.RegisterFactory(engine.EngineTypeCA, ca.NewCAEngine)
registry.Mount(ctx, "pki", engine.EngineTypeCA, map[string]interface{}{
    "organization": "Metacircular",
    "key_algorithm": "ecdsa",
    "key_size":      384,
})
// Creates engine at barrier path: engine/ca/pki/

Each mount gets its own namespace in the barrier for isolated data storage (config, keys, operational data). Mounting an engine calls Initialize(), which performs first-time setup (e.g. generating a root CA). On subsequent unseals, Unseal() loads existing state from the barrier.

The registry provides GetEngine(name) and GetMount(name) methods for direct engine access, used by the public PKI routes to serve certificates without authentication.

Request Routing

POST /v1/engine/request {mount: "pki", operation: "issue", path: "infra", data: {...}}
  → Registry.HandleRequest("pki", req)
    → engine.HandleRequest(ctx, req)
      → Response{Data: {...}}

CA (PKI) Engine

The CA engine (internal/engine/ca/) provides X.509 certificate issuance for Metacircular infrastructure. It implements a two-tier PKI: a single root CA issues scoped intermediate CAs ("issuers"), which in turn issue leaf certificates.

Certificate generation uses the certgen package from git.wntrmute.dev/kyle/goutils/certlib/certgen.

Lifecycle

  • Initialize: Generates a self-signed root CA, stores root cert+key and config in the barrier.
  • Unseal: Loads config, root cert+key, and all issuers from the barrier into memory.
  • Seal: Zeroizes all in-memory private key material (root key, all issuer keys), nils out pointers.

Operations

Operation Auth Required Description
get-root None Return root CA cert PEM
get-chain None Return full chain PEM (issuer + root)
get-issuer None Return issuer cert PEM
create-issuer Admin Generate intermediate CA signed by root
delete-issuer Admin Remove issuer and zeroize its key
list-issuers Any auth List issuer names
issue User/Admin Issue leaf cert from named issuer
get-cert Any auth Get cert record by serial
list-certs Any auth List issued cert summaries
renew User/Admin Re-issue cert with same attributes

Certificate Profiles

Three default profiles control leaf certificate key usage and validity:

Profile Key Usage Ext Key Usage Default TTL
server Digital Signature, Key Encipherment Server Auth 90 days
client Digital Signature Client Auth 90 days
peer Digital Signature, Key Encipherment Server Auth, Client Auth 90 days

Users can override TTL, key usages, ext key usages, and key algorithm at issuance time.

Issuance Flow

  1. Look up issuer by name
  2. Start from named profile defaults, apply user overrides
  3. Generate leaf key pair, build CSR, sign with issuer via profile.SignRequest
  4. Store CertRecord in barrier (cert PEM + metadata; no private key)
  5. Return cert PEM, private key PEM, chain PEM, serial, metadata

Barrier Storage Layout

engine/ca/{mount}/config.json                    CA config (org, key algo, root expiry)
engine/ca/{mount}/root/cert.pem                  Root CA certificate
engine/ca/{mount}/root/key.pem                   Root CA private key
engine/ca/{mount}/issuers/{name}/cert.pem        Issuer certificate
engine/ca/{mount}/issuers/{name}/key.pem         Issuer private key
engine/ca/{mount}/issuers/{name}/config.json     Issuer config
engine/ca/{mount}/certs/{serial_hex}.json         Issued cert record (no private key)

CA Configuration

Passed as config at mount time:

Field Default Description
organization "Metacircular" Root/issuer certificate O field
country "" Root/issuer certificate C field
key_algorithm "ecdsa" Key type: ecdsa, rsa, ed25519
key_size 384 Key size (e.g. 256/384 for ECDSA, 2048/4096 for RSA)
root_expiry "87600h" Root CA validity (10 years)

API Surface

Metacrypt exposes two API surfaces: a JSON REST API and a gRPC API. Both are kept in sync — every operation available via REST has a corresponding gRPC RPC.

Seal/Unseal (Unauthenticated)

Method Path Description Precondition
GET /v1/status Service state + version info None
POST /v1/init First-time seal initialization uninitialized
POST /v1/unseal Unseal with password sealed

Seal Control (Admin Only)

Method Path Description
POST /v1/seal Seal service & engines

Authentication

Method Path Description Auth Required
POST /v1/auth/login MCIAS login → token No
POST /v1/auth/logout Invalidate token Yes
GET /v1/auth/tokeninfo Current user info Yes

Engines (Authenticated)

Method Path Description Auth
GET /v1/engine/mounts List mounted engines User
POST /v1/engine/mount Create new engine mount Admin
POST /v1/engine/unmount Remove engine mount Admin
POST /v1/engine/request Route request to engine User

The mount endpoint accepts {name, type, config} where config is an engine-type-specific configuration object. The request endpoint accepts {mount, operation, path, data} and populates CallerInfo from the authenticated user's token.

Public PKI (Unauthenticated, Unsealed Required)

Method Path Description
GET /v1/pki/{mount}/ca Root CA certificate (PEM)
GET /v1/pki/{mount}/ca/chain?issuer= Full chain: issuer + root (PEM)
GET /v1/pki/{mount}/issuer/{name} Issuer certificate (PEM)

These routes serve certificates with Content-Type: application/x-pem-file, allowing systems to bootstrap TLS trust without authentication. The mount must be of type ca; returns 404 otherwise.

Policy (Admin Only)

Method Path Description
GET /v1/policy/rules List all rules
POST /v1/policy/rules Create a rule
GET /v1/policy/rule?id= Get rule by ID
DELETE /v1/policy/rule?id= Delete rule by ID

Error Responses

All API errors return JSON:

{"error": "description of what went wrong"}

HTTP status codes:

  • 401 — missing or invalid token
  • 403 — insufficient privileges
  • 412 — service not initialized
  • 503 — service is sealed

gRPC API

Metacrypt also exposes a gRPC API defined in proto/metacrypt/. Two API versions exist:

v1 (current implementation)

The v1 API uses a generic Execute RPC for all engine operations:

rpc Execute(ExecuteRequest) returns (ExecuteResponse);

message ExecuteRequest {
  string mount     = 1;
  string operation = 2;
  string path      = 3;
  google.protobuf.Struct data = 4;  // JSON-like map
}

message ExecuteResponse {
  google.protobuf.Struct data = 1;  // JSON-like map
}

Timestamps are represented as RFC3339 strings within the Struct payload. The EngineService also provides Mount, Unmount, and ListMounts RPCs for engine lifecycle management.

v2 (defined, not yet implemented)

The v2 API (proto/metacrypt/v2/) replaces the generic Execute RPC with strongly-typed, per-operation RPCs and uses google.protobuf.Timestamp for all time fields. Key changes:

  • CAService: 11 typed RPCs — ImportRoot, GetRoot, CreateIssuer, DeleteIssuer, ListIssuers, GetIssuer, GetChain, IssueCert, GetCert, ListCerts, RenewCert.
  • EngineService: Retains Mount, Unmount, ListMounts; drops the generic Execute RPC. MountRequest.config is map<string, string> instead of google.protobuf.Struct.
  • Timestamps: All issued_at / expires_at fields use google.protobuf.Timestamp instead of RFC3339 strings.
  • Message types: CertRecord (full certificate data) and CertSummary (lightweight, for list responses) replace the generic struct maps.
  • ACMEService and AuthService: String timestamps replaced by google.protobuf.Timestamp.

The v2 proto definitions pass buf lint with no warnings. Server-side implementation of v2 is planned as a future milestone.

gRPC Interceptors (v1)

The gRPC server (internal/grpcserver/) uses three interceptor maps to gate access:

Interceptor map Effect
sealRequiredMethods Returns UNAVAILABLE if the barrier is sealed
authRequiredMethods Validates MCIAS bearer token; populates caller info
adminRequiredMethods Requires IsAdmin == true on the caller

All three maps include the Execute RPC, ensuring engine operations are always authenticated and gated on unseal state.

Web Interface

Metacrypt includes an HTMX-powered web UI for basic operations:

Route Purpose
/ Redirects based on service state
/init Password setup form (first-time only)
/unseal Password entry to unseal
/login MCIAS login form (username, password, TOTP)
/dashboard Engine mounts, service state, admin controls
/pki PKI overview: list issuers, download CA/issuer PEMs
/pki/issuer/{name} Issuer detail: certificates issued by that issuer

The dashboard shows mounted engines, the service state, and (for admins) a seal button. Templates use Go's html/template with a shared layout. HTMX provides form submission without full page reloads.

The PKI pages communicate with the backend via the internal gRPC client (internal/webserver/client.go), which wraps the v1 gRPC Execute RPC. The issuer detail page supports filtering certificates by common name (case-insensitive substring match) and sorting by common name (default) or expiry date.

Database Schema

SQLite with WAL mode, foreign keys enabled, 5-second busy timeout.

Tables

seal_config — Single row storing the encrypted master key material.

Column Type Description
id INTEGER Always 1 (enforced primary key)
encrypted_mek BLOB MEK encrypted with KWK
kdf_salt BLOB 32-byte Argon2id salt
argon2_time INTEGER Argon2id time cost
argon2_memory INTEGER Argon2id memory cost (KiB)
argon2_threads INTEGER Argon2id parallelism
initialized_at DATETIME Timestamp of initialization

barrier_entries — Encrypted key-value store.

Column Type Description
path TEXT Primary key; hierarchical path
value BLOB AES-256-GCM encrypted value
created_at DATETIME Row creation time
updated_at DATETIME Last modification time

schema_migrations — Tracks applied schema versions.

Column Type Description
version INTEGER Migration version number
applied_at DATETIME When applied

Migrations are idempotent and run sequentially at startup.

Configuration

TOML configuration with environment variable overrides (METACRYPT_*).

[server]
listen_addr = ":8443"           # required
tls_cert    = "/path/cert.pem"  # required
tls_key     = "/path/key.pem"   # required

[database]
path = "/path/metacrypt.db"     # required

[mcias]
server_url = "https://mcias.metacircular.net:8443"  # required
ca_cert    = "/path/ca.pem"     # optional, for custom CA

[seal]
argon2_time    = 3              # default: 3
argon2_memory  = 131072         # default: 128 MiB (in KiB)
argon2_threads = 4              # default: 4

[log]
level = "info"                  # default: "info"

Required fields are validated at startup; the server refuses to start if any are missing.

Deployment

Docker

Multi-stage build:

  1. Builder stage: Go compilation with symbols stripped (-s -w)
  2. Runtime stage: Alpine 3.21, non-root metacrypt user
VOLUME /data    # config, certs, database
EXPOSE 8443
ENTRYPOINT ["metacrypt", "server", "--config", "/data/metacrypt.toml"]

TLS Configuration

  • Minimum TLS version: 1.2
  • Cipher suites: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
  • Timeouts: read 30s, write 30s, idle 120s

CLI Commands

Command Purpose
server Start the HTTPS server
init Interactive first-time seal setup
status Query a running server's state
snapshot Create a consistent database backup (VACUUM INTO)

Graceful Shutdown

The server handles SIGINT and SIGTERM signals, sealing all engines and closing connections before exit.

Security Model

Threat Mitigations

Threat Mitigation
Database theft All barrier values encrypted with MEK (AES-256-GCM)
Brute-force unseal Argon2id (128 MiB memory-hard), rate limiting (5/min + lockout)
MEK exposure Held in memory only when unsealed; zeroized on seal
Token theft 30-second cache TTL; MCIAS-backed validation
Privilege escalation Default-deny policy; admin bypass only for MCIAS admin role
Nonce reuse Fresh random nonce per encryption operation
Timing attacks Constant-time comparison for passwords and tokens
Unauthorized access at rest Database file permissions 0600; non-root container user
TLS downgrade Minimum TLS 1.2; only AEAD cipher suites
CA key compromise CA/issuer keys encrypted in barrier; zeroized on seal; two-tier PKI limits blast radius
Leaf key leakage via storage Issued cert private keys never persisted; only returned to requester

Security Invariants

  1. The MEK never leaves process memory and is never logged or serialized in plaintext.
  2. The seal password is never stored; only its Argon2id-derived output is used transiently.
  3. All barrier writes produce fresh ciphertexts (random nonce per encryption).
  4. The service is fail-closed: a sealed barrier rejects all operations.
  5. Admin privileges are determined solely by MCIAS role membership; Metacrypt has no local user database.
  6. Issued certificate private keys are returned to the caller but never stored in the barrier. Only cert metadata is persisted.
  7. CA and issuer private keys are encrypted at rest in the barrier and zeroized from memory on seal (explicit overwrite of ECDSA D, RSA D and primes, Ed25519 key bytes).

Future Work

Remaining Engine Implementations

  • SSH CA Engine — Sign SSH host and user certificates
  • Transit Engine — Encrypt/decrypt payloads on behalf of applications (envelope encryption); key rotation
  • User Engine — Key exchange and encryption between Metacircular users

CA Engine Enhancements

  • CRL management — Certificate revocation lists
  • OCSP responder — Online certificate status checking
  • Certificate templates — Admin-defined custom profiles beyond server/client/peer

Planned Capabilities

  • gRPC v2 server implementation — The v2 typed proto definitions are complete; the server-side handlers and generated Go code remain to be implemented
  • Post-quantum readiness — Hybrid key exchange (ML-KEM + ECDH); the versioned ciphertext format and engine interface are designed for algorithm agility
  • Key rotation — MEK and per-engine DEK rotation without re-sealing
  • Audit logging — Tamper-evident log of all cryptographic operations
  • Engine persistence — Auto-remounting engines from barrier state on unseal

Public Key Algorithms

The CA engine supports ECDSA (P-256, P-384, P-521), RSA, and Ed25519 for certificate key pairs, configurable per-CA and overridable per-issuer or per-issuance. The system is designed for algorithm agility to support future post-quantum algorithms.

Reference in New Issue View Git Blame Copy Permalink