metacrypt/ARCHITECTURE.md

Metacrypt Architecture & Specification

Metacrypt is a cryptographic service for the Metacircular platform. It provides cryptographic resources via a modular "engines" architecture, backed by an encrypted storage barrier inspired by HashiCorp Vault. Authentication is delegated to the Metacircular Identity and Access Service (MCIAS).

Table of Contents

1. System Overview
2. Key Hierarchy & Cryptographic Design
3. Seal/Unseal Lifecycle
4. Encrypted Storage Barrier
5. Authentication & Authorization
6. Engine Architecture
7. API Surface
8. Web Interface
9. Database Schema
10. Configuration
11. Deployment
12. Security Model
13. Future Work

---

System Overview

┌─────────────────────────────────────────┐
│         Clients (CLI / Web UI)          │
└──────────────────┬──────────────────────┘
                   │ HTTPS (TLS 1.2+)
┌──────────────────▼──────────────────────┐
│  HTTP API & Web Routes (server/)        │
│   ├── Seal/Unseal endpoints             │
│   ├── Auth endpoints (MCIAS delegation) │
│   ├── Engine request routing            │
│   └── Policy management                 │
├─────────────────────────────────────────┤
│  Middleware                             │
│   ├── Structured logging                │
│   ├── Token authentication              │
│   ├── Policy authorization              │
│   └── Unseal-state gating               │
├─────────────────────────────────────────┤
│  Service Layer                          │
│   ├── Seal Manager     (seal/)          │
│   ├── Authenticator    (auth/)          │
│   ├── Policy Engine    (policy/)        │
│   └── Engine Registry  (engine/)        │
├─────────────────────────────────────────┤
│  Encrypted Storage Barrier (barrier/)   │
│   └── AES-256-GCM per-entry encryption  │
├─────────────────────────────────────────┤
│  Database Layer (db/)                   │
│   └── SQLite (WAL mode, foreign keys)   │
└─────────────────────────────────────────┘

Package Layout

cmd/metacrypt/          CLI entry point (server, init, status, snapshot)
internal/
  config/               TOML configuration loading & validation
  crypto/               Low-level cryptographic primitives
  db/                   SQLite setup & schema migrations
  seal/                 Seal/unseal state machine
  barrier/              Encrypted key-value storage abstraction
  auth/                 MCIAS token authentication & caching
  policy/               Priority-based ACL engine
  engine/               Pluggable engine registry & interface
    ca/                 CA (PKI) engine — X.509 certificate issuance
  server/               HTTP server, routes, middleware
web/
  templates/            Go HTML templates (layout, init, unseal, login, dashboard)
  static/               CSS, HTMX
deploy/                 Docker Compose, example configs

---

Key Hierarchy & Cryptographic Design

Primitives

Purpose	Algorithm	Parameters
Key derivation	Argon2id	3 iterations, 128 MiB, 4 threads (configurable)
Symmetric encryption	AES-256-GCM	256-bit keys, 12-byte random nonce
Key size	256 bits	All symmetric keys
Salt size	256 bits	Argon2id salt
CSPRNG	crypto/rand	Keys, salts, nonces
Constant-time comparison	crypto/subtle	Password & token comparison
Zeroization	Explicit overwrite	MEK, KWK, passwords in memory

Key Hierarchy

User Password (not stored)
      │
      ▼
┌─────────────────────────────────────┐
│ Argon2id(password, salt, params)    │  salt + params stored in seal_config
└──────────────────┬──────────────────┘
                   ▼
          Key Wrap Key (KWK)              256-bit, ephemeral (derived on unseal)
                   │
                   ▼
          ┌────────────────┐
          │ AES-256-GCM    │              encrypted_mek stored in seal_config
          │ Decrypt         │
          └───────┬────────┘
                  ▼
       Master Encryption Key (MEK)        256-bit, held in memory only when unsealed
                  │
                  ▼
       ┌──────────────────────┐
       │ Barrier Entries       │           Each entry encrypted individually
       │  ├── Policy rules     │           with MEK via AES-256-GCM
       │  ├── Engine configs   │
       │  └── Engine DEKs      │           Per-engine data encryption keys
       └──────────────────────┘

Ciphertext Format

All encrypted values use a versioned binary format:

[version: 1 byte][nonce: 12 bytes][ciphertext + GCM tag]

The version byte (currently 0x01) enables future algorithm migration, including post-quantum hybrid schemes.

---

Seal/Unseal Lifecycle

Metacrypt operates as a state machine with four states:

                  ┌────────────────┐
                  │ Uninitialized  │  No seal_config row in DB
                  └───────┬────────┘
                          │ Initialize(password)
                          │  • generate salt
                          │  • derive KWK = Argon2id(password, salt)
                          │  • generate random MEK
                          │  • store Encrypt(KWK, MEK) + salt + params
                          ▼
 Seal()         ┌──────────────────┐
 ◄──────────────│    Unsealed      │  MEK in memory; barrier open
 │              └──────────────────┘
 │                        ▲
 │                        │ Unseal(password)
 │                        │  • load salt + params + encrypted_mek
 │                        │  • derive KWK = Argon2id(password, salt)
 │                        │  • MEK = Decrypt(KWK, encrypted_mek)
 │                        │  • barrier.Unseal(MEK)
 │              ┌──────────────────┐
 └─────────────►│     Sealed       │  MEK zeroized; barrier locked
                └──────────────────┘

Rate Limiting

Unseal attempts are rate-limited to mitigate online brute-force:

• 5 attempts within a 1-minute sliding window
• 60-second lockout after exceeding the limit
• Counter resets after 1 minute of inactivity

Sealing

Calling Seal() immediately:

1. Zeroizes the MEK from memory
2. Seals the storage barrier (all reads/writes return ErrSealed)
3. Seals all mounted engines
4. Flushes the authentication token cache

---

Encrypted Storage Barrier

The barrier provides an encrypted key-value store over the barrier_entries table. Every value is independently encrypted with the MEK using AES-256-GCM.

Interface

type Barrier interface {
    Unseal(mek []byte) error
    Seal() error
    IsSealed() bool
    Get(ctx context.Context, path string) ([]byte, error)
    Put(ctx context.Context, path string, value []byte) error
    Delete(ctx context.Context, path string) error
    List(ctx context.Context, prefix string) ([]string, error)
}

Path Namespace Conventions

Prefix	Owner	Contents
policy/rules/{id}	Policy engine	JSON-encoded ACL rules
engine/{type}/{mount}/	Engine	Config, keys, engine data
engine/ca/{mount}/root/	CA engine	Root CA cert + key
engine/ca/{mount}/issuers/	CA engine	Issuer certs, keys, config
engine/ca/{mount}/certs/	CA engine	Issued cert records (no private keys)

Properties

• Encryption at rest: All values encrypted with MEK before database write
• Fresh nonce per write: Every Put generates a new random nonce
• Atomic upsert: Uses INSERT ... ON CONFLICT UPDATE for Put
• Glob listing: List(prefix) returns relative paths matching the prefix
• Thread-safe: All operations guarded by sync.RWMutex
• Fail-closed: Returns ErrSealed for any operation when the barrier is sealed

---

Authentication & Authorization

Authentication (MCIAS Delegation)

Metacrypt does not manage user accounts. All authentication is delegated to MCIAS:

1. Client sends POST /v1/auth/login with {username, password, totp_code}
2. Metacrypt forwards credentials to the MCIAS client library
3. On success, MCIAS returns a bearer token and expiration
4. Token is returned to the client (also set as metacrypt_token cookie for web UI)
5. Subsequent requests include Authorization: Bearer <token> or the cookie

Token validation calls MCIAS ValidateToken(), with results cached for 30 seconds (keyed by SHA-256 hash of the token) to reduce MCIAS load.

Admin detection: Users with the admin role in MCIAS are granted admin privileges in Metacrypt.

Authorization (Policy Engine)

The policy engine evaluates access control rules stored in the barrier.

Rule structure:

type Rule struct {
    ID        string    // unique identifier
    Priority  int       // lower number = higher priority
    Effect    Effect    // "allow" or "deny"
    Usernames []string  // match specific users (optional)
    Roles     []string  // match roles (optional)
    Resources []string  // glob patterns, e.g. "engine/transit/*" (optional)
    Actions   []string  // e.g. "read", "write", "admin" (optional)
}

Evaluation algorithm:

1. If the requester has the admin role, allow immediately (bypass)
2. Collect all rules where username, role, resource, and action match
3. Sort matching rules by priority (ascending; lower number = higher priority)
4. Return the effect of the highest-priority matching rule
5. Default deny if no rules match

Matching is case-insensitive for usernames and roles. Resources use glob patterns. Empty fields in a rule match everything.

---

Engine Architecture

Engines are pluggable cryptographic service providers. The CA (PKI) engine is implemented; remaining engine types are planned.

Engine Types

Type	Status	Purpose
ca	Implemented	X.509 Certificate Authority (PKI)
sshca	Planned	SSH Certificate Authority
transit	Planned	Encrypt/decrypt data in transit (envelope encryption)
user	Planned	User-to-user end-to-end encryption

Engine Interface

type CallerInfo struct {
    Username string
    Roles    []string
    IsAdmin  bool
}

type Request struct {
    Operation  string
    Path       string
    Data       map[string]interface{}
    CallerInfo *CallerInfo
}

type Engine interface {
    Type() EngineType
    Initialize(ctx context.Context, b barrier.Barrier, mountPath string, config map[string]interface{}) error
    Unseal(ctx context.Context, b barrier.Barrier, mountPath string) error
    Seal() error
    HandleRequest(ctx context.Context, req *Request) (*Response, error)
}

CallerInfo carries authentication context into engine operations, allowing engines to enforce their own auth requirements (e.g. admin-only operations).

Initialize accepts a config map for engine-specific configuration passed at mount time.

Mount Registry

Engines are instantiated through a factory pattern and tracked in a central registry:

registry.RegisterFactory(engine.EngineTypeCA, ca.NewCAEngine)
registry.Mount(ctx, "pki", engine.EngineTypeCA, map[string]interface{}{
    "organization": "Metacircular",
    "key_algorithm": "ecdsa",
    "key_size":      384,
})
// Creates engine at barrier path: engine/ca/pki/

Each mount gets its own namespace in the barrier for isolated data storage (config, keys, operational data). Mounting an engine calls Initialize(), which performs first-time setup (e.g. generating a root CA). On subsequent unseals, Unseal() loads existing state from the barrier.

The registry provides GetEngine(name) and GetMount(name) methods for direct engine access, used by the public PKI routes to serve certificates without authentication.

Request Routing

POST /v1/engine/request {mount: "pki", operation: "issue", path: "infra", data: {...}}
  → Registry.HandleRequest("pki", req)
    → engine.HandleRequest(ctx, req)
      → Response{Data: {...}}

CA (PKI) Engine

The CA engine (internal/engine/ca/) provides X.509 certificate issuance for Metacircular infrastructure. It implements a two-tier PKI: a single root CA issues scoped intermediate CAs ("issuers"), which in turn issue leaf certificates.

Certificate generation uses the certgen package from git.wntrmute.dev/kyle/goutils/certlib/certgen.

Lifecycle

• Initialize: Generates a self-signed root CA, stores root cert+key and config in the barrier.
• Unseal: Loads config, root cert+key, and all issuers from the barrier into memory.
• Seal: Zeroizes all in-memory private key material (root key, all issuer keys), nils out pointers.

Operations

Operation	Auth Required	Description
get-root	None	Return root CA cert PEM
get-chain	None	Return full chain PEM (issuer + root)
get-issuer	None	Return issuer cert PEM
create-issuer	Admin	Generate intermediate CA signed by root
delete-issuer	Admin	Remove issuer and zeroize its key
list-issuers	Any auth	List issuer names
issue	User/Admin	Issue leaf cert from named issuer
get-cert	Any auth	Get cert record by serial
list-certs	Any auth	List issued cert summaries
renew	User/Admin	Re-issue cert with same attributes

Certificate Profiles

Three default profiles control leaf certificate key usage and validity:

Profile	Key Usage	Ext Key Usage	Default TTL
server	Digital Signature, Key Encipherment	Server Auth	90 days
client	Digital Signature	Client Auth	90 days
peer	Digital Signature, Key Encipherment	Server Auth, Client Auth	90 days

Users can override TTL, key usages, ext key usages, and key algorithm at issuance time.

Issuance Flow

1. Look up issuer by name
2. Start from named profile defaults, apply user overrides
3. Generate leaf key pair, build CSR, sign with issuer via profile.SignRequest
4. Store CertRecord in barrier (cert PEM + metadata; no private key)
5. Return cert PEM, private key PEM, chain PEM, serial, metadata

Barrier Storage Layout

engine/ca/{mount}/config.json                    CA config (org, key algo, root expiry)
engine/ca/{mount}/root/cert.pem                  Root CA certificate
engine/ca/{mount}/root/key.pem                   Root CA private key
engine/ca/{mount}/issuers/{name}/cert.pem        Issuer certificate
engine/ca/{mount}/issuers/{name}/key.pem         Issuer private key
engine/ca/{mount}/issuers/{name}/config.json     Issuer config
engine/ca/{mount}/certs/{serial_hex}.json         Issued cert record (no private key)

CA Configuration

Passed as config at mount time:

Field	Default	Description
organization	"Metacircular"	Root/issuer certificate O field
country	""	Root/issuer certificate C field
key_algorithm	"ecdsa"	Key type: ecdsa, rsa, ed25519
key_size	384	Key size (e.g. 256/384 for ECDSA, 2048/4096 for RSA)
root_expiry	"87600h"	Root CA validity (10 years)

---

API Surface

Seal/Unseal (Unauthenticated)

Method	Path	Description	Precondition
GET	/v1/status	Service state + version info	None
POST	/v1/init	First-time seal initialization	uninitialized
POST	/v1/unseal	Unseal with password	sealed

Seal Control (Admin Only)

Method	Path	Description
POST	/v1/seal	Seal service & engines

Authentication

Method	Path	Description	Auth Required
POST	/v1/auth/login	MCIAS login → token	No
POST	/v1/auth/logout	Invalidate token	Yes
GET	/v1/auth/tokeninfo	Current user info	Yes

Engines (Authenticated)

Method	Path	Description	Auth
GET	/v1/engine/mounts	List mounted engines	User
POST	/v1/engine/mount	Create new engine mount	Admin
POST	/v1/engine/unmount	Remove engine mount	Admin
POST	/v1/engine/request	Route request to engine	User

The mount endpoint accepts {name, type, config} where config is an engine-type-specific configuration object. The request endpoint accepts {mount, operation, path, data} and populates CallerInfo from the authenticated user's token.

Public PKI (Unauthenticated, Unsealed Required)

Method	Path	Description
GET	/v1/pki/{mount}/ca	Root CA certificate (PEM)
GET	/v1/pki/{mount}/ca/chain?issuer=	Full chain: issuer + root (PEM)
GET	/v1/pki/{mount}/issuer/{name}	Issuer certificate (PEM)

These routes serve certificates with Content-Type: application/x-pem-file, allowing systems to bootstrap TLS trust without authentication. The mount must be of type ca; returns 404 otherwise.

Policy (Admin Only)

Method	Path	Description
GET	/v1/policy/rules	List all rules
POST	/v1/policy/rules	Create a rule
GET	/v1/policy/rule?id=	Get rule by ID
DELETE	/v1/policy/rule?id=	Delete rule by ID

Error Responses

All API errors return JSON:

{"error": "description of what went wrong"}

HTTP status codes:

• 401 — missing or invalid token
• 403 — insufficient privileges
• 412 — service not initialized
• 503 — service is sealed

---

Web Interface

Metacrypt includes an HTMX-powered web UI for basic operations:

Route	Purpose
/	Redirects based on service state
/init	Password setup form (first-time only)
/unseal	Password entry to unseal
/login	MCIAS login form (username, password, TOTP)
/dashboard	Engine mounts, service state, admin controls

The dashboard shows mounted engines, the service state, and (for admins) a seal button. Templates use Go's html/template with a shared layout. HTMX provides form submission without full page reloads.

---

Database Schema

SQLite with WAL mode, foreign keys enabled, 5-second busy timeout.

Tables

seal_config — Single row storing the encrypted master key material.

Column	Type	Description
id	INTEGER	Always 1 (enforced primary key)
encrypted_mek	BLOB	MEK encrypted with KWK
kdf_salt	BLOB	32-byte Argon2id salt
argon2_time	INTEGER	Argon2id time cost
argon2_memory	INTEGER	Argon2id memory cost (KiB)
argon2_threads	INTEGER	Argon2id parallelism
initialized_at	DATETIME	Timestamp of initialization

barrier_entries — Encrypted key-value store.

Column	Type	Description
path	TEXT	Primary key; hierarchical path
value	BLOB	AES-256-GCM encrypted value
created_at	DATETIME	Row creation time
updated_at	DATETIME	Last modification time

schema_migrations — Tracks applied schema versions.

Column	Type	Description
version	INTEGER	Migration version number
applied_at	DATETIME	When applied

Migrations are idempotent and run sequentially at startup.

---

Configuration

TOML configuration with environment variable overrides (METACRYPT_*).

[server]
listen_addr = ":8443"           # required
tls_cert    = "/path/cert.pem"  # required
tls_key     = "/path/key.pem"   # required

[database]
path = "/path/metacrypt.db"     # required

[mcias]
server_url = "https://mcias.metacircular.net:8443"  # required
ca_cert    = "/path/ca.pem"     # optional, for custom CA

[seal]
argon2_time    = 3              # default: 3
argon2_memory  = 131072         # default: 128 MiB (in KiB)
argon2_threads = 4              # default: 4

[log]
level = "info"                  # default: "info"

Required fields are validated at startup; the server refuses to start if any are missing.

---

Deployment

Docker

Multi-stage build:

1. Builder stage: Go compilation with symbols stripped (-s -w)
2. Runtime stage: Alpine 3.21, non-root metacrypt user

VOLUME /data    # config, certs, database
EXPOSE 8443
ENTRYPOINT ["metacrypt", "server", "--config", "/data/metacrypt.toml"]

TLS Configuration

• Minimum TLS version: 1.2
• Cipher suites: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
• Timeouts: read 30s, write 30s, idle 120s

CLI Commands

Command	Purpose
server	Start the HTTPS server
init	Interactive first-time seal setup
status	Query a running server's state
snapshot	Create a consistent database backup (VACUUM INTO)

Graceful Shutdown

The server handles SIGINT and SIGTERM signals, sealing all engines and closing connections before exit.

---

Security Model

Threat Mitigations

Threat	Mitigation
Database theft	All barrier values encrypted with MEK (AES-256-GCM)
Brute-force unseal	Argon2id (128 MiB memory-hard), rate limiting (5/min + lockout)
MEK exposure	Held in memory only when unsealed; zeroized on seal
Token theft	30-second cache TTL; MCIAS-backed validation
Privilege escalation	Default-deny policy; admin bypass only for MCIAS admin role
Nonce reuse	Fresh random nonce per encryption operation
Timing attacks	Constant-time comparison for passwords and tokens
Unauthorized access at rest	Database file permissions 0600; non-root container user
TLS downgrade	Minimum TLS 1.2; only AEAD cipher suites
CA key compromise	CA/issuer keys encrypted in barrier; zeroized on seal; two-tier PKI limits blast radius
Leaf key leakage via storage	Issued cert private keys never persisted; only returned to requester

Security Invariants

1. The MEK never leaves process memory and is never logged or serialized in plaintext.
2. The seal password is never stored; only its Argon2id-derived output is used transiently.
3. All barrier writes produce fresh ciphertexts (random nonce per encryption).
4. The service is fail-closed: a sealed barrier rejects all operations.
5. Admin privileges are determined solely by MCIAS role membership; Metacrypt has no local user database.
6. Issued certificate private keys are returned to the caller but never stored in the barrier. Only cert metadata is persisted.
7. CA and issuer private keys are encrypted at rest in the barrier and zeroized from memory on seal (explicit overwrite of ECDSA D, RSA D and primes, Ed25519 key bytes).

---

Future Work

Remaining Engine Implementations

• SSH CA Engine — Sign SSH host and user certificates
• Transit Engine — Encrypt/decrypt payloads on behalf of applications (envelope encryption); key rotation
• User Engine — Key exchange and encryption between Metacircular users

CA Engine Enhancements

• CRL management — Certificate revocation lists
• OCSP responder — Online certificate status checking
• Certificate templates — Admin-defined custom profiles beyond server/client/peer

Planned Capabilities

• gRPC API — In addition to the JSON REST API (config field already reserved)
• Post-quantum readiness — Hybrid key exchange (ML-KEM + ECDH); the versioned ciphertext format and engine interface are designed for algorithm agility
• Key rotation — MEK and per-engine DEK rotation without re-sealing
• Audit logging — Tamper-evident log of all cryptographic operations
• Engine persistence — Auto-remounting engines from barrier state on unseal

Public Key Algorithms

The CA engine supports ECDSA (P-256, P-384, P-521), RSA, and Ed25519 for certificate key pairs, configurable per-CA and overridable per-issuer or per-issuance. The system is designed for algorithm agility to support future post-quantum algorithms.