mcias/PROGRESS.md

# MCIAS Progress

Source of truth for current development state.
---
All phases complete. 137 Go server tests + 25 Go client tests + 23 Rust client
tests + 37 Lisp client tests + 32 Python client tests pass. Zero race
conditions (go test -race ./...).
- [x] Phase 0: Repository bootstrap (go.mod, .gitignore, docs)
- [x] Phase 1: Foundational packages (model, config, crypto, db)
- [x] Phase 2: Auth core (auth, token, middleware)
- [x] Phase 3: HTTP server (server, mciassrv binary)
- [x] Phase 4: Admin CLI (mciasctl binary)
- [x] Phase 5: E2E tests, security hardening, commit
- [x] Phase 6: mciasdb — direct SQLite maintenance tool
- [x] Phase 7: gRPC interface (alternate transport; dual-stack with REST)
- [x] Phase 8: Operational artifacts (Makefile, Dockerfile, systemd, man pages, install script)
- [x] Phase 9: Client libraries (Go, Rust, Common Lisp, Python)
---
### 2026-03-11 — Phase 9: Client libraries

**clients/testdata/** — shared JSON fixtures
- login_response.json, account_response.json, accounts_list_response.json
- validate_token_response.json, public_key_response.json, pgcreds_response.json
- error_response.json, roles_response.json

**clients/go/** — Go client library
- Module: `git.wntrmute.dev/kyle/mcias/clients/go`; package `mciasgoclient`
- Typed errors: `MciasAuthError`, `MciasForbiddenError`, `MciasNotFoundError`,
  `MciasInputError`, `MciasConflictError`, `MciasServerError`
- TLS 1.2+ enforced via `tls.Config{MinVersion: tls.VersionTLS12}`
- Token state guarded by `sync.RWMutex` for concurrent safety
- JSON decoded with `DisallowUnknownFields` on all responses
- 25 tests in `client_test.go`; all pass with `go test -race`

**clients/rust/** — Rust async client library
- Crate: `mcias-client`; tokio async, reqwest + rustls-tls (no OpenSSL dep)
- `MciasError` enum via `thiserror`; `Arc<RwLock<Option<String>>>` for token
- 23 integration tests using `wiremock`; `cargo clippy -- -D warnings` clean

**clients/lisp/** — Common Lisp client library
- ASDF system `mcias-client`; HTTP via dexador, JSON via yason
- CLOS class `mcias-client`; plain functions for all operations
- Conditions: `mcias-error` base + 6 typed subclasses
- Mock server: Hunchentoot `mock-dispatcher` subclass (port 0, random per test)
- 37 fiveam checks; all pass on SBCL 2.6.1
- Fixed: yason decodes JSON `false` as `:false`; `validate-token` normalises
  to `t`/`nil` before returning

**clients/python/** — Python 3.11+ client library
- Package `mcias_client` (setuptools, pyproject.toml); dep: `httpx >= 0.27`
- `Client` context manager; `py.typed` marker; all symbols fully annotated
- Dataclasses: `Account`, `PublicKey`, `PGCreds`
- 32 pytest tests using `respx` mock transport; `mypy --strict` clean; `ruff` clean

**test/mock/mockserver.go** — Go in-memory mock server
- `Server` struct with `sync.RWMutex`; used by Go client integration test
- `NewServer()`, `AddAccount()`, `ServeHTTP()` for httptest.Server use

---
**Makefile**
- Targets: build, test, lint, generate, man, install, clean, dist, docker
- build: compiles all four binaries to bin/ with CGO_ENABLED=1 and
  -trimpath -ldflags="-s -w"
- dist: cross-compiled tarballs for linux/amd64 and linux/arm64
- docker: builds image tagged mcias:$(git describe --tags --always)
- VERSION derived from git describe --tags --always
**Dockerfile** (multi-stage)
- Build stage: golang:1.26-bookworm with CGO_ENABLED=1
- Runtime stage: debian:bookworm-slim with only ca-certificates and libc6;
  no Go toolchain, no source, no build cache in final image
- Non-root user mcias (uid/gid 10001)
- EXPOSE 8443 (REST/TLS) and EXPOSE 9443 (gRPC/TLS)
- VOLUME /data for the SQLite database mount point
- ENTRYPOINT ["mciassrv"] CMD ["-config", "/etc/mcias/mcias.conf"]
**dist/ artifacts**
- dist/mcias.service: hardened systemd unit with ProtectSystem=strict,
  ProtectHome=true, PrivateTmp=true, NoNewPrivileges=true,
  CapabilityBoundingSet= (no capabilities), ReadWritePaths=/var/lib/mcias,
  EnvironmentFile=/etc/mcias/env, Restart=on-failure, LimitNOFILE=65536
- dist/mcias.env.example: passphrase env file template
- dist/mcias.conf.example: fully-commented production TOML config reference
- dist/mcias-dev.conf.example: local dev config (127.0.0.1, short expiry)
- dist/mcias.conf.docker.example: container config template
- dist/install.sh: idempotent POSIX sh installer; creates user/group,
  installs binaries, creates /etc/mcias and /var/lib/mcias, installs
  systemd unit and man pages; existing configs not overwritten (placed .new)
**man/ pages** (mdoc format)
- man/man1/mciassrv.1: synopsis, options, config, REST API, signals, files
- man/man1/mciasctl.1: all subcommands, env vars, examples
- man/man1/mciasdb.1: trust model warnings, all subcommands, examples
- man/man1/mciasgrpcctl.1: gRPC subcommands, grpcurl examples
**Documentation**
- README.md: replaced dev-workflow notes with user-facing docs; quick-start,
  first-run setup, build instructions, CLI references, Docker deployment,
  man page index, security notes
- .gitignore: added /bin/, dist/mcias_*.tar.gz, man/man1/*.gz
### 2026-03-11 — Phase 7: gRPC dual-stack

**proto/mcias/v1/**
- `common.proto` — shared types: Account, TokenInfo, PGCreds, Error
- `admin.proto` — AdminService: Health (public), GetPublicKey (public)
- `auth.proto` — AuthService: Login (public), Logout, RenewToken,
  EnrollTOTP, ConfirmTOTP, RemoveTOTP (admin)
- `token.proto` — TokenService: ValidateToken (public),
  IssueServiceToken (admin), RevokeToken (admin)
- `account.proto` — AccountService (CRUD + roles, all admin) +
  CredentialService (GetPGCreds, SetPGCreds, all admin)
- `proto/generate.go` — go:generate directive for protoc regeneration
- Generated Go stubs in `gen/mcias/v1/` via protoc + protoc-gen-go-grpc

**internal/grpcserver**
- `grpcserver.go` — Server struct, interceptor chain
  (loggingInterceptor → authInterceptor → rateLimitInterceptor),
  GRPCServer() / GRPCServerWithCreds(creds) / buildServer() helpers,
  per-IP token-bucket rate limiter (same parameters as REST: 10 req/s,
  burst 10), extractBearerFromMD, requireAdmin
- `admin.go` — Health, GetPublicKey implementations
- `auth.go` — Login (with dummy-Argon2 timing guard), Logout, RenewToken,
  EnrollTOTP, ConfirmTOTP, RemoveTOTP
- `tokenservice.go` — ValidateToken (returns valid=false on error, never
  an RPC error), IssueServiceToken, RevokeToken
- `accountservice.go` — ListAccounts, CreateAccount, GetAccount,
  UpdateAccount, DeleteAccount, GetRoles, SetRoles
- `credentialservice.go` — GetPGCreds (AES-GCM decrypt), SetPGCreds
  (AES-GCM encrypt)

**Security invariants (same as REST server):**
- Authorization metadata value never logged by any interceptor
- Credential fields (PasswordHash, TOTPSecret*, PGPassword) absent from
  all proto response messages by proto design + grpcserver enforcement
- JWT validation: alg-first, then signature, then revocation table lookup
- Public RPCs bypass auth: Health, GetPublicKey, ValidateToken, Login
- Admin-only RPCs checked in-handler via requireAdmin(ctx)
- Dummy Argon2 in Login for unknown users prevents timing enumeration

**internal/config additions**
- `GRPCAddr string` field in ServerConfig (optional; omit to disable gRPC)

**cmd/mciassrv updates**
- Dual-stack: starts both HTTPS (REST) and gRPC/TLS listeners when
  grpc_addr is configured in [server] section
- gRPC listener uses same TLS cert/key as REST; credentials passed at
  server-construction time via GRPCServerWithCreds
- Graceful shutdown drains both listeners within 15s window

**cmd/mciasgrpcctl**
- New companion CLI for gRPC management
- Global flags: -server (host:port), -token (or MCIAS_TOKEN), -cacert
- Commands: health, pubkey, account (list/create/get/update/delete),
  role (list/set), token (validate/issue/revoke),
  pgcreds (get/set)
- Connects with TLS; custom CA cert support for self-signed certs

**Tests**
- `internal/grpcserver/grpcserver_test.go`: 20 tests using bufconn
  (in-process, no network sockets); covers:
  - Health and GetPublicKey (public RPCs, no auth)
  - Auth interceptor: no token, invalid token, revoked token all → 401
  - Non-admin calling admin RPC → 403
  - Login: success, wrong password, unknown user
  - Logout and RenewToken
  - ValidateToken: good token → valid=true; garbage → valid=false (no error)
  - IssueServiceToken requires admin
  - ListAccounts: non-admin → 403, admin → OK
  - CreateAccount, GetAccount, UpdateAccount, SetRoles, GetRoles lifecycle
  - SetPGCreds + GetPGCreds with AES-GCM round-trip verification
  - PGCreds requires admin
  - Credential fields absent from account responses (structural enforcement)

**Dependencies added**
- `google.golang.org/grpc v1.68.0`
- `google.golang.org/protobuf v1.36.0`
- `google.golang.org/grpc/test/bufconn` (test only, included in grpc module)

Total: 137 tests, all pass, zero race conditions (go test -race ./...)

### 2026-03-11 — Phase 6: mciasdb

**cmd/mciasdb**
- Binary skeleton: config loading, master key derivation (identical to
  mciassrv for key compatibility), DB open + migrate on startup
- `schema verify` / `schema migrate` — reports and applies pending migrations
- `account list/get/create/set-password/set-status/reset-totp` — offline
  account management; set-password prompts interactively (no --password flag)
- `role list/grant/revoke` — direct role management
- `token list/revoke/revoke-all` + `prune tokens` — token maintenance
- `audit tail/query` — audit log inspection with --json output flag
- `pgcreds get/set` — decrypt/encrypt Postgres credentials with master key;
  set prompts interactively; get prints warning before sensitive output
- All write operations emit audit log entries tagged `actor:"mciasdb"`

**internal/db additions**
- `ListTokensForAccount(accountID)` — newest-first token list for an account
- `ListAuditEvents(AuditQueryParams)` — filtered audit query (account, type,
  since, limit)
- `TailAuditEvents(n)` — last n events, returned oldest-first
- `SchemaVersion(db)` / `LatestSchemaVersion` — exported for mciasdb verify

**Dependencies**
- Added `golang.org/x/term v0.29.0` for interactive password prompting
  (no-echo terminal reads); pinned to version compatible with local module cache
- `golang.org/x/crypto` pinned at v0.33.0 (compatible with term@v0.29.0)

**Tests**
- `internal/db/mciasdb_test.go`: 4 tests covering ListTokensForAccount,
  ListAuditEvents filtering, TailAuditEvents ordering, combined filters
- `cmd/mciasdb/mciasdb_test.go`: 20 tests covering all subcommands via
  in-memory SQLite and stdout capture

Total: 117 tests, all pass, zero race conditions (go test -race ./...)

### 2026-03-11 — Initial Full Implementation

#### Phase 0: Bootstrap

- Wrote ARCHITECTURE.md (security model, crypto choices, DB schema, API design)
- Wrote PROJECT_PLAN.md (5 phases, 12 steps with acceptance criteria)
- Created go.mod with dependencies (golang-jwt/jwt/v5, uuid, go-toml/v2,
  golang.org/x/crypto, modernc.org/sqlite)
- Created .gitignore

#### Phase 1: Foundational Packages

**internal/model**
- Account (human/system), Role, TokenRecord, SystemToken, PGCredential,
  AuditEvent structs
- All credential fields tagged `json:"-"` — never serialised to responses
- Audit event type constants

**internal/config**
- TOML config parsing with validation
- Enforces OWASP 2023 Argon2id minimums (time≥2, memory≥64MiB)
- Requires exactly one of passphrase_env or keyfile for master key
- NewTestConfig() for test use

**internal/crypto**
- Ed25519 key generation, PEM marshal/parse
- AES-256-GCM seal/open with random nonces
- Argon2id KDF (DeriveKey) with OWASP-exceeding parameters
- NewSalt(), RandomBytes()

**internal/db**
- SQLite with WAL mode, FK enforcement, busy timeout
- Idempotent migrations (schema_version table)
- Migration 1: full schema (server_config, accounts, account_roles,
  token_revocation, system_tokens, pg_credentials, audit_log)
- Migration 2: master_key_salt column in server_config
- Full CRUD: accounts, roles, tokens, PG credentials, audit log

#### Phase 2: Auth Core

**internal/auth**
- Argon2id password hashing in PHC format
- Constant-time password verification (crypto/subtle)
- TOTP generation and validation (RFC 6238 ±1 window, constant-time)
- HOTP per RFC 4226

**internal/token**
- Ed25519/EdDSA JWT issuance with UUID JTI
- alg header validated BEFORE signature verification (alg confusion defence)
- alg:none explicitly rejected
- ErrWrongAlgorithm, ErrExpiredToken, ErrInvalidSignature, ErrMissingClaim

**internal/middleware**
- RequestLogger — never logs Authorization header
- RequireAuth — validates JWT, checks revocation table
- RequireRole — checks claims for required role
- RateLimit — per-IP token bucket

#### Phase 3: HTTP Server

**internal/server**
- Full REST API wired to middleware
- Handlers: health, public-key, login (dummy Argon2 on unknown user for
  timing uniformity), logout, renew, token validate/issue/revoke,
  account CRUD, roles, TOTP enrol/confirm/remove, PG credentials
- Strict JSON decoding (DisallowUnknownFields)
- Credential fields never appear in any response

**cmd/mciassrv**
- Config loading, master key derivation (passphrase via Argon2id KDF or
  key file), signing key load/generate (AES-256-GCM encrypted in DB),
  HTTPS listener with graceful shutdown
- TLS 1.2+ minimum, X25519+P256 curves
- 30s read/write timeouts, 5s header timeout

#### Phase 4: Admin CLI

**cmd/mciasctl**
- Subcommands: account (list/create/get/update/delete), role (list/set),
  token (issue/revoke), pgcreds (get/set)
- Auth via -token flag or MCIAS_TOKEN env var
- Custom CA cert support for self-signed TLS

#### Phase 5: Tests and Hardening

**Test coverage:**
- internal/model: 5 tests
- internal/config: 8 tests
- internal/crypto: 12 tests
- internal/db: 13 tests
- internal/auth: 13 tests
- internal/token: 9 tests (including alg confusion and alg:none attacks)
- internal/middleware: 12 tests
- internal/server: 14 tests
- test/e2e: 11 tests

Total: 97 tests — all pass, zero race conditions (go test -race ./...)

**Security tests (adversarial):**
- JWT alg:HS256 confusion attack → 401
- JWT alg:none attack → 401
- Revoked token reuse → 401
- Non-admin calling admin endpoint → 403
- Wrong password → 401 (same response as unknown user)
- Credential material absent from all API responses

**Security hardening:**
- go vet ./... — zero issues
- gofmt applied to all files
- golangci-lint v2 config updated (note: v2.6.2 built with go1.25.3
  cannot analyse go1.26 source; go vet used as primary linter for now)

---

## Architecture Decisions

- **SQLite driver**: `modernc.org/sqlite` (pure Go, no CGo)
- **JWT**: `github.com/golang-jwt/jwt/v5`; alg validated manually before
  library dispatch to defeat algorithm confusion
- **No ORM**: `database/sql` with parameterized statements only
- **Master key salt**: stored in server_config table for stable KDF across
  restarts; generated on first run
- **Signing key**: stored AES-256-GCM encrypted in server_config; generated
  on first run, decrypted each startup using master key
- **Timing uniformity**: unknown user login runs dummy Argon2 to match
  timing of wrong-password path; all credential comparisons use
  `crypto/subtle.ConstantTimeCompare`