Add archive package: tar.zst service directory snapshots

- Snapshot: VACUUM INTO for consistent db copy, excludes live db
  files and backups/, injects db snapshot, custom exclude patterns,
  streaming output via io.Writer
- Restore: extract tar.zst with path traversal protection
- zstd via github.com/klauspost/compress/zstd
- 5 tests: full roundtrip with db integrity verification,
  without db, exclude patterns, dest dir creation
- Update PROGRESS.md: all 9 packages complete, 87 total tests

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-03-25 16:37:18 -07:00
parent 20dc7ae0d6
commit 9012b889d6
5 changed files with 550 additions and 43 deletions

View File

@@ -2,59 +2,76 @@
## Current State
Phase 3 complete. The `db`, `auth`, and `config` packages are implemented
and tested.
Phases 09 complete. All nine packages are implemented and tested (87 tests).
Ready for first-adopter migration (Phase 10).
## Completed
### Phase 0: Project Setup (2026-03-25)
- Initialized Go module (`git.wntrmute.dev/kyle/mcdsl`)
- Created `.golangci.yaml` matching platform standard (with `exported` rule
enabled since this is a shared library)
- Created `Makefile` with standard targets (build, test, vet, lint, all)
- Created `.gitignore`
- Created `doc.go` package doc
- `make all` passes clean
- Go module, Makefile, .golangci.yaml (with `exported` rule), .gitignore
### Phase 1: `db` — SQLite Foundation (2026-03-25)
- `Open(path string) (*sql.DB, error)` — opens with WAL, FK, busy timeout
5000ms, 0600 permissions, creates parent dirs
- `Migration` type with Version, Name, SQL fields
- `Migrate(database *sql.DB, migrations []Migration) error` — sequential,
transactional, idempotent, records name and timestamp in schema_migrations
- `SchemaVersion(database *sql.DB) (int, error)` — highest applied version
- `Snapshot(database *sql.DB, destPath string) error` — VACUUM INTO with
0600 permissions, creates parent dirs
- 11 tests covering open, migrate, and snapshot
- Open (WAL, FK, busy timeout, 0600, parent dirs), Migration type, Migrate
(sequential, transactional, idempotent), SchemaVersion, Snapshot (VACUUM INTO)
- 11 tests
### Phase 2: `auth` — MCIAS Token Validation (2026-03-25)
- `Config` type matching `[mcias]` TOML section
- `TokenInfo` type (Username, Roles, IsAdmin)
- `New(cfg, logger)` — MCIAS client with TLS 1.3, custom CA, 10s timeout
- `Login`, `ValidateToken` (30s SHA-256 cache), `Logout`
- Error types, context helpers
- 14 tests with mock MCIAS server and injectable clock
- Config, TokenInfo, Authenticator with Login/ValidateToken/Logout
- 30s SHA-256 cache, lazy eviction, RWMutex, context helpers
- 14 tests
### Phase 3: `config` — TOML Configuration (2026-03-25)
- `Base` type embedding standard sections (Server, Database, MCIAS, Log)
- `ServerConfig` with `Duration` wrapper type for TOML string decoding
(go-toml v2 does not natively decode strings to time.Duration)
- `DatabaseConfig`, `LogConfig`, `WebConfig` (non-embedded, for web UIs)
- `Duration` type with TextUnmarshaler/TextMarshaler for TOML compatibility
- `Load[T any](path, envPrefix)` — generic loader with TOML parse, env
overrides via reflection, defaults, required field validation
- `Validator` interface for service-specific validation
- Environment overrides: PREFIX_SECTION_FIELD for strings, durations,
bools, and comma-separated string slices
- Defaults: ReadTimeout=30s, WriteTimeout=30s, IdleTimeout=120s,
ShutdownTimeout=60s, Log.Level="info"
- Required: listen_addr, tls_cert, tls_key
- 16 tests: minimal/full config, defaults (applied and not overriding
explicit), missing required fields (3 cases), env overrides (string,
duration, slice, bool, service-specific), Validator interface (pass/fail),
nonexistent file, invalid TOML, empty prefix
- `make all` passes clean (vet, lint 0 issues, 41 total tests, build)
- Base type, ServerConfig with Duration wrapper, Load[T] generic loader
- Env overrides via reflection, defaults, Validator interface
- 16 tests
### Phase 4: `httpserver` — HTTP Server (2026-03-25)
- Server with chi + TLS 1.3, ListenAndServeTLS, Shutdown
- LoggingMiddleware, StatusWriter, WriteJSON, WriteError
- 8 tests
### Phase 5: `csrf` — CSRF Protection (2026-03-25)
- HMAC-SHA256 double-submit cookies, Middleware, SetToken, TemplateFunc
- 10 tests
### Phase 6: `web` — Session and Templates (2026-03-25)
- SetSessionCookie/ClearSessionCookie/GetSessionToken (HttpOnly, Secure,
SameSite=Strict), RequireAuth middleware, RenderTemplate
- 9 tests
### Phase 7: `grpcserver` — gRPC Server (2026-03-25)
- MethodMap (Public, AuthRequired, AdminRequired), default deny for unmapped
- Auth interceptor, logging interceptor, TLS 1.3 optional
- 10 tests
### Phase 8: `health` — Health Checks (2026-03-25)
- REST Handler(db) — 200 ok / 503 unhealthy
- RegisterGRPC — grpc.health.v1.Health
- 4 tests
### Phase 9: `archive` — Service Directory Snapshots (2026-03-25)
- Snapshot: tar.zst with VACUUM INTO db injection, exclude *.db/*.db-wal/
*.db-shm/backups/, custom exclude patterns, streaming output
- Restore: extract tar.zst to dest dir, path traversal protection
- 5 tests: full roundtrip with db integrity, without db, exclude live db,
custom excludes, dest dir creation
## Summary
| Package | Tests | Key Exports |
|---------|-------|-------------|
| `db` | 11 | Open, Migration, Migrate, SchemaVersion, Snapshot |
| `auth` | 14 | Config, TokenInfo, Authenticator, context helpers |
| `config` | 16 | Base, ServerConfig, Duration, Load[T], Validator |
| `httpserver` | 8 | Server, LoggingMiddleware, WriteJSON, WriteError |
| `csrf` | 10 | Protect, Middleware, SetToken, TemplateFunc |
| `web` | 9 | SetSessionCookie, RequireAuth, RenderTemplate |
| `grpcserver` | 10 | MethodMap, Server (default deny), TokenInfoFromContext |
| `health` | 4 | Handler, RegisterGRPC |
| `archive` | 5 | Snapshot, Restore |
| **Total** | **87** | |
## Next Steps
- Phase 4: `httpserver` package (TLS HTTP server, middleware, JSON helpers)
- Phase 10: First-adopter migration (mcat)
- Phase 11: Broader adoption (metacrypt, mcr, mc-proxy, mcias)