Initial commit: project setup and db package
- Project scaffolding: go.mod, Makefile, .golangci.yaml, doc.go - README, ARCHITECTURE, PROJECT_PLAN, PROGRESS documentation - db package: Open (WAL, FK, busy timeout, 0600 permissions), Migrate (sequential, transactional, idempotent), SchemaVersion, Snapshot (VACUUM INTO) - 11 tests covering open, migrate, and snapshot Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
192
PROJECT_PLAN.md
Normal file
192
PROJECT_PLAN.md
Normal file
@@ -0,0 +1,192 @@
|
||||
# MCDSL Project Plan
|
||||
|
||||
Implementation phases for the Metacircular Dynamics Standard Library.
|
||||
|
||||
Each phase produces a usable, tested package. Phases are ordered by
|
||||
dependency (foundational packages first) and by value (most-duplicated
|
||||
code first).
|
||||
|
||||
---
|
||||
|
||||
## Phase 0: Project Setup
|
||||
|
||||
- [ ] Initialize Go module (`git.wntrmute.dev/kyle/mcdsl`)
|
||||
- [ ] Create `.golangci.yaml` (matching platform standard)
|
||||
- [ ] Create `Makefile` with standard targets (build, test, vet, lint, all)
|
||||
- [ ] Create `.gitignore`
|
||||
|
||||
**Acceptance criteria:** `make all` passes on an empty module.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: `db` — SQLite Foundation
|
||||
|
||||
The most universally needed package. Every service with a database uses
|
||||
identical open/pragma/migration code.
|
||||
|
||||
- [ ] `Open(path string) (*sql.DB, error)` — open with WAL, FK, busy timeout,
|
||||
0600 permissions
|
||||
- [ ] `Migration` type and `Migrate(db *sql.DB, migrations []Migration) error`
|
||||
— sequential, transactional, idempotent, schema_migrations tracking
|
||||
- [ ] `Snapshot(db *sql.DB, destPath string) error` — VACUUM INTO wrapper
|
||||
- [ ] Tests: open, migrate fresh DB, migrate existing DB (idempotent), snapshot
|
||||
produces valid DB, file permissions
|
||||
|
||||
**Acceptance criteria:** A service can replace its `internal/db/` open and
|
||||
migrate code with `mcdsl/db` and pass its existing tests.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: `auth` — MCIAS Token Validation
|
||||
|
||||
The second most duplicated package. Every authenticated service has its own
|
||||
copy of the cache-and-validate logic.
|
||||
|
||||
- [ ] `Config` type matching `[mcias]` TOML section
|
||||
- [ ] `TokenInfo` type (Username, Roles, IsAdmin)
|
||||
- [ ] `New(cfg Config, logger *slog.Logger) (*Authenticator, error)`
|
||||
- [ ] `ValidateToken(token string) (*TokenInfo, error)` with 30s SHA-256 cache
|
||||
- [ ] `Login(username, password, totpCode string) (token string, expiresAt time.Time, err error)`
|
||||
- [ ] `Logout(token string) error`
|
||||
- [ ] Error types: `ErrInvalidToken`, `ErrInvalidCredentials`, `ErrForbidden`
|
||||
- [ ] Tests: cache hit, cache miss, cache expiry, admin detection, concurrent
|
||||
access, error propagation
|
||||
|
||||
**Acceptance criteria:** A service can replace its `internal/auth/` with
|
||||
`mcdsl/auth` and pass its existing tests.
|
||||
|
||||
---
|
||||
|
||||
## Phase 3: `config` — TOML Configuration
|
||||
|
||||
- [ ] `Base` type with standard sections (Server, Database, MCIAS, Log)
|
||||
- [ ] `ServerConfig`, `DatabaseConfig`, `LogConfig` types
|
||||
- [ ] `Load[T any](path string, envPrefix string) (*T, error)` — generic
|
||||
loader with TOML parse, env overrides, defaults, validation
|
||||
- [ ] Environment override via reflection (`PREFIX_SECTION_FIELD`)
|
||||
- [ ] Required field validation (listen addr, TLS paths, DB path)
|
||||
- [ ] Default application (timeouts, log level)
|
||||
- [ ] Optional `Validate()` interface for service-specific validation
|
||||
- [ ] Tests: load valid config, missing required fields, env overrides, defaults,
|
||||
custom validation
|
||||
|
||||
**Acceptance criteria:** A service can replace its `internal/config/` with
|
||||
`mcdsl/config` embedding `config.Base`, and pass its existing tests.
|
||||
|
||||
---
|
||||
|
||||
## Phase 4: `httpserver` — HTTP Server Setup
|
||||
|
||||
- [ ] `Server` type wrapping chi + `http.Server`
|
||||
- [ ] `New(cfg config.ServerConfig, logger *slog.Logger) *Server`
|
||||
- [ ] `ListenAndServeTLS(certFile, keyFile string) error`
|
||||
- [ ] `Shutdown(ctx context.Context) error`
|
||||
- [ ] `LoggingMiddleware` — captures status code, logs request metadata
|
||||
- [ ] `StatusWriter` — exported response writer wrapper
|
||||
- [ ] `WriteJSON` and `WriteError` helpers
|
||||
- [ ] Tests: server starts and shuts down cleanly, logging middleware captures
|
||||
status, JSON helpers produce correct output
|
||||
|
||||
**Acceptance criteria:** A service can replace its server setup and middleware
|
||||
boilerplate with `mcdsl/httpserver`.
|
||||
|
||||
---
|
||||
|
||||
## Phase 5: `csrf` — CSRF Protection
|
||||
|
||||
- [ ] `New(secret []byte, cookieName, fieldName string) *Protect`
|
||||
- [ ] `Middleware(next http.Handler) http.Handler`
|
||||
- [ ] `SetToken(w http.ResponseWriter) string`
|
||||
- [ ] `TemplateFunc(w http.ResponseWriter) template.FuncMap`
|
||||
- [ ] Token format: `base64(nonce).base64(HMAC-SHA256(secret, nonce))`
|
||||
- [ ] Tests: token generation, validation, middleware rejects missing/invalid
|
||||
tokens, safe methods pass through
|
||||
|
||||
**Acceptance criteria:** A service can replace its CSRF implementation with
|
||||
`mcdsl/csrf` and its web UI continues to work.
|
||||
|
||||
---
|
||||
|
||||
## Phase 6: `web` — Session and Template Helpers
|
||||
|
||||
- [ ] `SetSessionCookie`, `ClearSessionCookie`, `GetSessionToken`
|
||||
- [ ] `RequireAuth` middleware (validates token, redirects to login)
|
||||
- [ ] `TokenInfoFromContext` context helper
|
||||
- [ ] `RenderTemplate` helper for layout + page template pattern
|
||||
- [ ] Tests: cookie setting/clearing, auth middleware redirect, template
|
||||
rendering
|
||||
|
||||
**Acceptance criteria:** A service can replace its web session/auth
|
||||
boilerplate with `mcdsl/web`.
|
||||
|
||||
---
|
||||
|
||||
## Phase 7: `grpcserver` — gRPC Server Setup
|
||||
|
||||
- [ ] `MethodMap` type (Public, AuthRequired, AdminRequired)
|
||||
- [ ] `New(cfg config.ServerConfig, auth *auth.Authenticator, methods MethodMap, logger *slog.Logger) (*Server, error)`
|
||||
- [ ] `Serve() error` and `Stop()`
|
||||
- [ ] Auth interceptor using MethodMap (default deny for unmapped methods)
|
||||
- [ ] Logging interceptor
|
||||
- [ ] `TokenInfoFromContext` context helper
|
||||
- [ ] Tests: public method allowed, auth method requires token, admin method
|
||||
requires admin, unmapped method denied, logging
|
||||
|
||||
**Acceptance criteria:** A service can replace its gRPC server setup and
|
||||
interceptor logic with `mcdsl/grpcserver`.
|
||||
|
||||
---
|
||||
|
||||
## Phase 8: `health` — Health Checks
|
||||
|
||||
- [ ] `RegisterGRPC(srv *grpc.Server)` — register `grpc.health.v1.Health`
|
||||
- [ ] `Handler(db *sql.DB) http.HandlerFunc` — REST health endpoint
|
||||
- [ ] Tests: healthy response, unhealthy response (closed DB)
|
||||
|
||||
**Acceptance criteria:** Services have a standard health check that MCP can
|
||||
query.
|
||||
|
||||
---
|
||||
|
||||
## Phase 9: `archive` — Service Directory Snapshots
|
||||
|
||||
- [ ] `SnapshotOptions` type
|
||||
- [ ] `Snapshot(opts SnapshotOptions) (io.ReadCloser, error)` — streaming
|
||||
tar.zst with DB exclusion/injection
|
||||
- [ ] `Restore(r io.Reader, destDir string) error`
|
||||
- [ ] Exclude patterns: `*.db`, `*.db-wal`, `*.db-shm`, `backups/`
|
||||
- [ ] Tests: snapshot roundtrip (snapshot then restore produces identical
|
||||
files), DB consistency (VACUUM INTO copy matches), excludes work,
|
||||
streaming (no full buffer)
|
||||
|
||||
**Acceptance criteria:** MCP agent can snapshot and restore a service
|
||||
directory using this package.
|
||||
|
||||
---
|
||||
|
||||
## Phase 10: Service Migration (First Adopter)
|
||||
|
||||
Pick one service (mcat is the simplest) and migrate it to use MCDSL:
|
||||
|
||||
- [ ] Replace `internal/auth/` with `mcdsl/auth`
|
||||
- [ ] Replace `internal/config/` with `mcdsl/config`
|
||||
- [ ] Replace web session/CSRF code with `mcdsl/csrf` and `mcdsl/web`
|
||||
- [ ] Verify `make all` passes
|
||||
- [ ] Document the migration process for other services
|
||||
|
||||
**Acceptance criteria:** mcat works identically using MCDSL, with its
|
||||
`internal/` packages reduced to service-specific logic only.
|
||||
|
||||
---
|
||||
|
||||
## Phase 11: Broader Adoption
|
||||
|
||||
Migrate remaining services one at a time:
|
||||
|
||||
- [ ] metacrypt
|
||||
- [ ] mcr
|
||||
- [ ] mc-proxy (subset: db, config — no web/csrf)
|
||||
- [ ] mcias (subset: db, config, httpserver — owns the auth client, not a consumer)
|
||||
|
||||
Each migration follows the same pattern as Phase 10. Services are migrated
|
||||
independently — there is no big-bang cutover.
|
||||
Reference in New Issue
Block a user