6 Commits

Author SHA1 Message Date
ebe2079a83 Migrate module path from kyle/ to mc/ org
All import paths updated from git.wntrmute.dev/kyle/mcdsl to
git.wntrmute.dev/mc/mcdsl to match the Gitea organization.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 02:03:45 -07:00
cef06bdf63 Merge pull request 'Add $PORT env var overrides for MCP agent port assignment' (#1) from port-env-support into master 2026-03-27 08:13:58 +00:00
f94c4b1abf Add $PORT and $PORT_GRPC env var overrides for MCP agent port assignment
After TOML loading and generic env overrides, config.Load now checks
$PORT and $PORT_GRPC and overrides ServerConfig.ListenAddr and
ServerConfig.GRPCAddr respectively. These take precedence over all
other config sources because they represent agent-assigned authoritative
port bindings.

Handles both Base embedding (MCR, MCNS, MCAT) and direct ServerConfig
embedding (Metacrypt) via struct tree walking.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-27 00:49:19 -07:00
021ba3b710 Add CLAUDE.md with library overview, packages, and design decisions
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 22:10:09 -07:00
ee88ebecf2 Add pre/post interceptor hooks to grpcserver.New
New Options parameter with PreInterceptors and PostInterceptors
allows services to inject custom interceptors into the chain:

  [pre-interceptors] → logging → auth → [post-interceptors] → handler

This enables services like metacrypt to add seal-check (pre-auth)
and audit-logging (post-auth) interceptors while using the shared
auth and logging infrastructure.

Pass nil for the default chain (logging + auth only).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-26 14:21:29 -07:00
20d8d8d4b4 Set MaxOpenConns(1) to eliminate SQLite SQLITE_BUSY errors
Go's database/sql opens multiple connections by default, but SQLite
only supports one concurrent writer. Under concurrent load (e.g.
parallel blob uploads to MCR), multiple connections compete for the
write lock and exceed busy_timeout, causing transient 500 errors.

With WAL mode, a single connection still allows concurrent reads
from other processes. Go serializes access through the connection
pool, eliminating busy errors entirely.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-25 23:28:46 -07:00
18 changed files with 327 additions and 29 deletions

View File

@@ -27,7 +27,7 @@ problem, usable independently or together.
### Module Path ### Module Path
``` ```
git.wntrmute.dev/kyle/mcdsl git.wntrmute.dev/mc/mcdsl
``` ```
### Dependencies ### Dependencies
@@ -39,7 +39,7 @@ git.wntrmute.dev/kyle/mcdsl
| `github.com/pelletier/go-toml/v2` | TOML config parsing | | `github.com/pelletier/go-toml/v2` | TOML config parsing |
| `google.golang.org/grpc` | gRPC server | | `google.golang.org/grpc` | gRPC server |
| `github.com/klauspost/compress/zstd` | Zstandard compression for archives | | `github.com/klauspost/compress/zstd` | Zstandard compression for archives |
| `git.wntrmute.dev/kyle/mcias/clients/go` | MCIAS client library | | `git.wntrmute.dev/mc/mcias/clients/go` | MCIAS client library |
All dependencies are already used by existing services. MCDSL adds no new All dependencies are already used by existing services. MCDSL adds no new
dependencies to the platform. dependencies to the platform.

96
CLAUDE.md Normal file
View File

@@ -0,0 +1,96 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
MCDSL (Metacircular Dynamics Standard Library) is the shared Go library for the Metacircular platform. It extracts common patterns from production services into reusable, tested packages. MCDSL is not a deployed service — it is imported by other services.
**Module path:** `git.wntrmute.dev/mc/mcdsl`
## Build Commands
```bash
make all # vet -> lint -> test -> build
make build # go build ./...
make test # go test ./...
make vet # go vet ./...
make lint # golangci-lint run ./...
make clean # go clean ./...
```
Run a single test:
```bash
go test ./auth/ -run TestCacheExpiry
```
## Packages
| Package | Purpose |
|---------|---------|
| `auth` | MCIAS token validation with 30s SHA-256 cache |
| `db` | SQLite setup (WAL, foreign keys, busy timeout), migration runner, VACUUM INTO snapshots |
| `config` | TOML loading with env var overrides, Base config struct with standard sections |
| `httpserver` | TLS 1.3 HTTP server with chi, logging middleware, JSON helpers, graceful shutdown |
| `grpcserver` | gRPC server with TLS, auth/admin interceptors, default-deny for unmapped methods |
| `csrf` | HMAC-SHA256 double-submit cookie CSRF protection |
| `web` | Session cookies, auth middleware, template rendering for htmx UIs |
| `health` | REST and gRPC health check handlers |
| `archive` | tar.zst snapshots with SQLite-aware backup (VACUUM INTO) |
## Import Pattern
Services import individual packages with aliased imports:
```go
import (
mcdslauth "git.wntrmute.dev/mc/mcdsl/auth"
mcdsldb "git.wntrmute.dev/mc/mcdsl/db"
mcdslconfig "git.wntrmute.dev/mc/mcdsl/config"
"git.wntrmute.dev/mc/mcdsl/httpserver"
"git.wntrmute.dev/mc/mcdsl/grpcserver"
)
```
## Inter-Package Dependencies
```
archive --> db
auth --> (mcias client library)
config --> auth (for auth.Config type)
csrf --> (stdlib only)
db --> (modernc.org/sqlite)
grpcserver --> auth, config
health --> db
httpserver --> config
web --> auth, csrf
```
No circular dependencies. Each package can be imported independently except where noted.
## Key Design Decisions
- **No global state, no init().** Explicit construction, explicit errors.
- **Stdlib-compatible types.** Functions accept and return `*sql.DB`, `http.Handler`, `*slog.Logger`, `context.Context` — not custom wrappers.
- **TLS 1.3 minimum is non-configurable.** `httpserver` and `grpcserver` enforce this unconditionally.
- **Default-deny for unmapped gRPC methods.** A method not in the MethodMap is rejected, not allowed. This is the most important safety property.
- **Cookie security flags are non-configurable.** Session cookies are always `HttpOnly`, `Secure`, `SameSite=Strict`.
- **Extract, don't invent.** Every package comes from patterns proven across multiple services. No speculative abstractions.
- **Optional composition.** Services import only the packages they need.
## Critical Rules
- No CGo — all builds use `CGO_ENABLED=0`, SQLite via `modernc.org/sqlite`
- Testing uses stdlib `testing` only, real SQLite in `t.TempDir()`, no mocks for databases
- Token cache keys are SHA-256 of the raw token (never use raw token as map key)
- CSRF secrets must come from `crypto/rand`, unique per service instance
- Database file permissions are always `0600`
- Verify changes with `go build ./...` and `go test ./...` before claiming resolution
## Key Documents
- `ARCHITECTURE.md` — Full library specification with all package APIs
- `README.md` — Usage examples and quick start
- `PROJECT_PLAN.md` — Implementation phases
- `PROGRESS.md` — Development status
- `../engineering-standards.md` — Platform-wide standards (authoritative reference)

View File

@@ -91,11 +91,11 @@ Remaining mcat-specific code:
- `web/` — templates and static assets (unchanged) - `web/` — templates and static assets (unchanged)
Dependencies removed: Dependencies removed:
- `git.wntrmute.dev/kyle/mcias/clients/go` (mcdsl/auth handles MCIAS directly) - `git.wntrmute.dev/mc/mcias/clients/go` (mcdsl/auth handles MCIAS directly)
- `github.com/pelletier/go-toml/v2` (now indirect via mcdsl/config) - `github.com/pelletier/go-toml/v2` (now indirect via mcdsl/config)
Dependencies added: Dependencies added:
- `git.wntrmute.dev/kyle/mcdsl` (local replace directive) - `git.wntrmute.dev/mc/mcdsl` (local replace directive)
Result: vet clean, lint 0 issues, builds successfully. Result: vet clean, lint 0 issues, builds successfully.

View File

@@ -10,7 +10,7 @@ code first).
## Phase 0: Project Setup ## Phase 0: Project Setup
- [ ] Initialize Go module (`git.wntrmute.dev/kyle/mcdsl`) - [ ] Initialize Go module (`git.wntrmute.dev/mc/mcdsl`)
- [ ] Create `.golangci.yaml` (matching platform standard) - [ ] Create `.golangci.yaml` (matching platform standard)
- [ ] Create `Makefile` with standard targets (build, test, vet, lint, all) - [ ] Create `Makefile` with standard targets (build, test, vet, lint, all)
- [ ] Create `.gitignore` - [ ] Create `.gitignore`

View File

@@ -25,7 +25,7 @@ it and provide only their service-specific logic.
## Module Path ## Module Path
``` ```
git.wntrmute.dev/kyle/mcdsl git.wntrmute.dev/mc/mcdsl
``` ```
## Packages ## Packages
@@ -46,10 +46,10 @@ git.wntrmute.dev/kyle/mcdsl
```go ```go
import ( import (
"git.wntrmute.dev/kyle/mcdsl/auth" "git.wntrmute.dev/mc/mcdsl/auth"
"git.wntrmute.dev/kyle/mcdsl/db" "git.wntrmute.dev/mc/mcdsl/db"
"git.wntrmute.dev/kyle/mcdsl/config" "git.wntrmute.dev/mc/mcdsl/config"
"git.wntrmute.dev/kyle/mcdsl/httpserver" "git.wntrmute.dev/mc/mcdsl/httpserver"
) )
// Load config with standard sections + service-specific fields. // Load config with standard sections + service-specific fields.

View File

@@ -20,7 +20,7 @@ import (
"github.com/klauspost/compress/zstd" "github.com/klauspost/compress/zstd"
"git.wntrmute.dev/kyle/mcdsl/db" "git.wntrmute.dev/mc/mcdsl/db"
) )
// defaultExcludePatterns are always excluded from snapshots. // defaultExcludePatterns are always excluded from snapshots.

View File

@@ -7,7 +7,7 @@ import (
"path/filepath" "path/filepath"
"testing" "testing"
"git.wntrmute.dev/kyle/mcdsl/db" "git.wntrmute.dev/mc/mcdsl/db"
) )
// setupServiceDir creates a realistic /srv/<service>/ directory. // setupServiceDir creates a realistic /srv/<service>/ directory.

View File

@@ -34,7 +34,7 @@ import (
"github.com/pelletier/go-toml/v2" "github.com/pelletier/go-toml/v2"
"git.wntrmute.dev/kyle/mcdsl/auth" "git.wntrmute.dev/mc/mcdsl/auth"
) )
// Base contains the configuration sections common to all Metacircular // Base contains the configuration sections common to all Metacircular
@@ -144,6 +144,8 @@ func Load[T any](path string, envPrefix string) (*T, error) {
applyEnvToStruct(reflect.ValueOf(&cfg).Elem(), envPrefix) applyEnvToStruct(reflect.ValueOf(&cfg).Elem(), envPrefix)
} }
applyPortEnv(&cfg)
applyBaseDefaults(&cfg) applyBaseDefaults(&cfg)
if err := validateBase(&cfg); err != nil { if err := validateBase(&cfg); err != nil {
@@ -239,6 +241,70 @@ func findBase(cfg any) *Base {
return nil return nil
} }
// applyPortEnv overrides ServerConfig.ListenAddr and ServerConfig.GRPCAddr
// from $PORT and $PORT_GRPC respectively. These environment variables are
// set by the MCP agent to assign authoritative port bindings, so they take
// precedence over both TOML values and generic env overrides.
func applyPortEnv(cfg any) {
sc := findServerConfig(cfg)
if sc == nil {
return
}
if port, ok := os.LookupEnv("PORT"); ok {
sc.ListenAddr = ":" + port
}
if port, ok := os.LookupEnv("PORT_GRPC"); ok {
sc.GRPCAddr = ":" + port
}
}
// findServerConfig returns a pointer to the ServerConfig in the config
// struct. It first checks for an embedded Base (which contains Server),
// then walks the struct tree via reflection to find any ServerConfig field
// directly (e.g., the Metacrypt pattern where ServerConfig is embedded
// without Base).
func findServerConfig(cfg any) *ServerConfig {
if base := findBase(cfg); base != nil {
return &base.Server
}
return findServerConfigReflect(reflect.ValueOf(cfg))
}
// findServerConfigReflect walks the struct tree to find a ServerConfig field.
func findServerConfigReflect(v reflect.Value) *ServerConfig {
if v.Kind() == reflect.Ptr {
v = v.Elem()
}
if v.Kind() != reflect.Struct {
return nil
}
scType := reflect.TypeOf(ServerConfig{})
t := v.Type()
for i := range t.NumField() {
field := t.Field(i)
fv := v.Field(i)
if field.Type == scType {
sc, ok := fv.Addr().Interface().(*ServerConfig)
if ok {
return sc
}
}
// Recurse into embedded or nested structs.
if fv.Kind() == reflect.Struct && field.Type != scType {
if sc := findServerConfigReflect(fv); sc != nil {
return sc
}
}
}
return nil
}
// applyEnvToStruct recursively walks a struct and overrides field values // applyEnvToStruct recursively walks a struct and overrides field values
// from environment variables. The env variable name is built from the // from environment variables. The env variable name is built from the
// prefix and the toml tag: PREFIX_SECTION_FIELD (uppercased). // prefix and the toml tag: PREFIX_SECTION_FIELD (uppercased).

View File

@@ -401,3 +401,109 @@ func TestEmptyEnvPrefix(t *testing.T) {
t.Fatalf("ListenAddr = %q, want %q", cfg.Server.ListenAddr, ":8443") t.Fatalf("ListenAddr = %q, want %q", cfg.Server.ListenAddr, ":8443")
} }
} }
// directServerConfig embeds ServerConfig without Base (Metacrypt pattern).
type directServerConfig struct {
Server ServerConfig `toml:"server"`
Extra string `toml:"extra"`
}
func TestPortEnvOverridesListenAddr(t *testing.T) {
path := writeTOML(t, minimalTOML)
t.Setenv("PORT", "9999")
cfg, err := Load[testConfig](path, "TEST")
if err != nil {
t.Fatalf("Load: %v", err)
}
if cfg.Server.ListenAddr != ":9999" {
t.Fatalf("ListenAddr = %q, want %q", cfg.Server.ListenAddr, ":9999")
}
}
func TestPortGRPCEnvOverridesGRPCAddr(t *testing.T) {
path := writeTOML(t, minimalTOML)
t.Setenv("PORT_GRPC", "9998")
cfg, err := Load[testConfig](path, "TEST")
if err != nil {
t.Fatalf("Load: %v", err)
}
if cfg.Server.GRPCAddr != ":9998" {
t.Fatalf("GRPCAddr = %q, want %q", cfg.Server.GRPCAddr, ":9998")
}
}
func TestPortEnvOverridesTOMLValue(t *testing.T) {
// fullTOML sets listen_addr = ":8443" and grpc_addr = ":9443".
path := writeTOML(t, fullTOML)
t.Setenv("PORT", "9999")
cfg, err := Load[testConfig](path, "TEST")
if err != nil {
t.Fatalf("Load: %v", err)
}
if cfg.Server.ListenAddr != ":9999" {
t.Fatalf("ListenAddr = %q, want %q ($PORT should override TOML)", cfg.Server.ListenAddr, ":9999")
}
}
func TestPortEnvOverridesGenericEnv(t *testing.T) {
path := writeTOML(t, minimalTOML)
t.Setenv("TEST_SERVER_LISTEN_ADDR", ":7777")
t.Setenv("PORT", "9999")
cfg, err := Load[testConfig](path, "TEST")
if err != nil {
t.Fatalf("Load: %v", err)
}
if cfg.Server.ListenAddr != ":9999" {
t.Fatalf("ListenAddr = %q, want %q ($PORT should override generic env)", cfg.Server.ListenAddr, ":9999")
}
}
func TestNoPortEnvNoChange(t *testing.T) {
path := writeTOML(t, minimalTOML)
cfg, err := Load[testConfig](path, "TEST")
if err != nil {
t.Fatalf("Load: %v", err)
}
// minimalTOML sets listen_addr = ":8443", no $PORT set.
if cfg.Server.ListenAddr != ":8443" {
t.Fatalf("ListenAddr = %q, want %q (TOML value preserved without $PORT)", cfg.Server.ListenAddr, ":8443")
}
}
func TestPortEnvDirectServerConfig(t *testing.T) {
// Test the Metacrypt pattern: ServerConfig embedded without Base.
toml := `
[server]
listen_addr = ":8443"
tls_cert = "/tmp/cert.pem"
tls_key = "/tmp/key.pem"
grpc_addr = ":9443"
extra = "value"
`
path := writeTOML(t, toml)
t.Setenv("PORT", "5555")
t.Setenv("PORT_GRPC", "5556")
cfg, err := Load[directServerConfig](path, "TEST")
if err != nil {
t.Fatalf("Load: %v", err)
}
if cfg.Server.ListenAddr != ":5555" {
t.Fatalf("ListenAddr = %q, want %q ($PORT on direct ServerConfig)", cfg.Server.ListenAddr, ":5555")
}
if cfg.Server.GRPCAddr != ":5556" {
t.Fatalf("GRPCAddr = %q, want %q ($PORT_GRPC on direct ServerConfig)", cfg.Server.GRPCAddr, ":5556")
}
}

View File

@@ -59,6 +59,12 @@ func Open(path string) (*sql.DB, error) {
} }
} }
// SQLite supports concurrent readers but only one writer. With WAL mode,
// reads don't block writes, but multiple Go connections competing for
// the write lock causes SQLITE_BUSY under concurrent load. Limit to one
// connection to serialize all access and eliminate busy errors.
database.SetMaxOpenConns(1)
// Ensure permissions are correct even if the file already existed. // Ensure permissions are correct even if the file already existed.
if err := os.Chmod(path, 0600); err != nil { if err := os.Chmod(path, 0600); err != nil {
_ = database.Close() _ = database.Close()

2
go.mod
View File

@@ -1,4 +1,4 @@
module git.wntrmute.dev/kyle/mcdsl module git.wntrmute.dev/mc/mcdsl
go 1.25.7 go 1.25.7

View File

@@ -21,7 +21,7 @@ import (
"google.golang.org/grpc/metadata" "google.golang.org/grpc/metadata"
"google.golang.org/grpc/status" "google.golang.org/grpc/status"
"git.wntrmute.dev/kyle/mcdsl/auth" "git.wntrmute.dev/mc/mcdsl/auth"
) )
// MethodMap classifies gRPC methods for access control. // MethodMap classifies gRPC methods for access control.
@@ -48,21 +48,45 @@ type Server struct {
listener net.Listener listener net.Listener
} }
// Options configures optional behavior for the gRPC server.
type Options struct {
// PreInterceptors run before the logging and auth interceptors.
// Use for lifecycle gates like seal checks that should reject
// requests before any auth validation occurs.
PreInterceptors []grpc.UnaryServerInterceptor
// PostInterceptors run after auth but before the handler.
// Use for audit logging, rate limiting, or other cross-cutting
// concerns that need access to the authenticated identity.
PostInterceptors []grpc.UnaryServerInterceptor
}
// New creates a gRPC server with TLS (if certFile and keyFile are // New creates a gRPC server with TLS (if certFile and keyFile are
// non-empty) and an interceptor chain: logging → auth → handler. // non-empty) and an interceptor chain:
//
// [pre-interceptors] → logging → auth → [post-interceptors] → handler
// //
// The auth interceptor uses methods to determine the access level for // The auth interceptor uses methods to determine the access level for
// each RPC. Methods not in any map are denied by default. // each RPC. Methods not in any map are denied by default.
// //
// If certFile and keyFile are empty, TLS is skipped (for testing). // If certFile and keyFile are empty, TLS is skipped (for testing).
func New(certFile, keyFile string, authenticator *auth.Authenticator, methods MethodMap, logger *slog.Logger) (*Server, error) { // opts is optional; pass nil for the default chain (logging + auth only).
chain := grpc.ChainUnaryInterceptor( func New(certFile, keyFile string, authenticator *auth.Authenticator, methods MethodMap, logger *slog.Logger, opts *Options) (*Server, error) {
var interceptors []grpc.UnaryServerInterceptor
if opts != nil {
interceptors = append(interceptors, opts.PreInterceptors...)
}
interceptors = append(interceptors,
loggingInterceptor(logger), loggingInterceptor(logger),
authInterceptor(authenticator, methods), authInterceptor(authenticator, methods),
) )
if opts != nil {
interceptors = append(interceptors, opts.PostInterceptors...)
}
chain := grpc.ChainUnaryInterceptor(interceptors...)
var opts []grpc.ServerOption var serverOpts []grpc.ServerOption
opts = append(opts, chain) serverOpts = append(serverOpts, chain)
if certFile != "" && keyFile != "" { if certFile != "" && keyFile != "" {
cert, err := tls.LoadX509KeyPair(certFile, keyFile) cert, err := tls.LoadX509KeyPair(certFile, keyFile)
@@ -73,11 +97,11 @@ func New(certFile, keyFile string, authenticator *auth.Authenticator, methods Me
Certificates: []tls.Certificate{cert}, Certificates: []tls.Certificate{cert},
MinVersion: tls.VersionTLS13, MinVersion: tls.VersionTLS13,
} }
opts = append(opts, grpc.Creds(credentials.NewTLS(tlsCfg))) serverOpts = append(serverOpts, grpc.Creds(credentials.NewTLS(tlsCfg)))
} }
return &Server{ return &Server{
GRPCServer: grpc.NewServer(opts...), GRPCServer: grpc.NewServer(serverOpts...),
Logger: logger, Logger: logger,
}, nil }, nil
} }

View File

@@ -13,7 +13,7 @@ import (
"google.golang.org/grpc/metadata" "google.golang.org/grpc/metadata"
"google.golang.org/grpc/status" "google.golang.org/grpc/status"
"git.wntrmute.dev/kyle/mcdsl/auth" "git.wntrmute.dev/mc/mcdsl/auth"
) )
func mockMCIAS(t *testing.T) *httptest.Server { func mockMCIAS(t *testing.T) *httptest.Server {
@@ -216,7 +216,7 @@ func TestNewWithoutTLS(t *testing.T) {
defer srv.Close() defer srv.Close()
a := testAuth(t, srv.URL) a := testAuth(t, srv.URL)
s, err := New("", "", a, testMethods, slog.Default()) s, err := New("", "", a, testMethods, slog.Default(), nil)
if err != nil { if err != nil {
t.Fatalf("New: %v", err) t.Fatalf("New: %v", err)
} }

View File

@@ -10,7 +10,7 @@ import (
"google.golang.org/grpc" "google.golang.org/grpc"
"git.wntrmute.dev/kyle/mcdsl/db" "git.wntrmute.dev/mc/mcdsl/db"
) )
func TestHandlerHealthy(t *testing.T) { func TestHandlerHealthy(t *testing.T) {

View File

@@ -13,7 +13,7 @@ import (
"github.com/go-chi/chi/v5" "github.com/go-chi/chi/v5"
"git.wntrmute.dev/kyle/mcdsl/config" "git.wntrmute.dev/mc/mcdsl/config"
) )
// Server wraps a chi router and an http.Server with the standard // Server wraps a chi router and an http.Server with the standard

View File

@@ -9,7 +9,7 @@ import (
"testing" "testing"
"time" "time"
"git.wntrmute.dev/kyle/mcdsl/config" "git.wntrmute.dev/mc/mcdsl/config"
) )
func testConfig() config.ServerConfig { func testConfig() config.ServerConfig {

View File

@@ -8,7 +8,7 @@ import (
"io/fs" "io/fs"
"net/http" "net/http"
"git.wntrmute.dev/kyle/mcdsl/auth" "git.wntrmute.dev/mc/mcdsl/auth"
) )
// SetSessionCookie sets a session cookie with the standard Metacircular // SetSessionCookie sets a session cookie with the standard Metacircular

View File

@@ -9,7 +9,7 @@ import (
"testing" "testing"
"testing/fstest" "testing/fstest"
"git.wntrmute.dev/kyle/mcdsl/auth" "git.wntrmute.dev/mc/mcdsl/auth"
) )
func TestSetSessionCookie(t *testing.T) { func TestSetSessionCookie(t *testing.T) {