diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index d66e7e1..6aa37e7 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -167,9 +167,10 @@ on which port the client connected to. ### Route Table Source -Route tables are defined inline under each listener in the TOML -configuration file. The design anticipates future migration to a SQLite -database for dynamic route management via the control plane API. +Route tables are persisted in the SQLite database. On first run, they are +seeded from the TOML configuration. On subsequent runs, the database is +the source of truth. Routes can be added or removed at runtime via the +gRPC admin API. --- @@ -179,7 +180,11 @@ TOML configuration file, loaded at startup. The proxy refuses to start if required fields are missing or invalid. ```toml -# Listeners. Each has its own route table. +# Database. Required. +[database] +path = "/srv/mc-proxy/mc-proxy.db" + +# Listeners. Each has its own route table (seeds DB on first run). [[listeners]] addr = ":443" @@ -205,6 +210,13 @@ addr = ":9443" hostname = "mcias.metacircular.net" backend = "127.0.0.1:28443" +# gRPC admin API. Optional — omit addr to disable. +[grpc] +addr = "127.0.0.1:9090" +tls_cert = "/srv/mc-proxy/certs/cert.pem" +tls_key = "/srv/mc-proxy/certs/key.pem" +client_ca = "/srv/mc-proxy/certs/ca.pem" + # Firewall. Global blocklist, evaluated before routing. Default allow. [firewall] geoip_db = "/srv/mc-proxy/GeoLite2-Country.mmdb" @@ -240,17 +252,64 @@ these are structural and must be in the TOML file. ## Storage -mc-proxy has minimal storage requirements. There is no database in the -initial implementation. +### SQLite Database + +Listeners, routes, and firewall rules are persisted in a SQLite database +(WAL mode, foreign keys enabled, busy timeout 5000ms). The pure-Go driver +`modernc.org/sqlite` is used (no CGo). + +**Startup behavior:** + +1. Open the database at the configured path. Run migrations. +2. If the database is empty (first run): seed from the TOML config. +3. If the database has data: load from it. TOML listener/route/firewall + fields are ignored. + +The TOML config continues to own operational settings: proxy timeouts, +log level, gRPC config, GeoIP database path. + +**Write-through pattern:** The gRPC admin API writes to the database first, +then updates in-memory state. If the database write fails, the in-memory +state is not modified. + +### Schema + +```sql +CREATE TABLE listeners ( + id INTEGER PRIMARY KEY, + addr TEXT NOT NULL UNIQUE +); + +CREATE TABLE routes ( + id INTEGER PRIMARY KEY, + listener_id INTEGER NOT NULL REFERENCES listeners(id) ON DELETE CASCADE, + hostname TEXT NOT NULL, + backend TEXT NOT NULL, + UNIQUE(listener_id, hostname) +); + +CREATE TABLE firewall_rules ( + id INTEGER PRIMARY KEY, + type TEXT NOT NULL CHECK(type IN ('ip', 'cidr', 'country')), + value TEXT NOT NULL, + UNIQUE(type, value) +); +``` + +### Data Directory ``` /srv/mc-proxy/ ├── mc-proxy.toml Configuration +├── mc-proxy.db SQLite database +├── certs/ TLS certificates (for gRPC admin API) ├── GeoLite2-Country.mmdb GeoIP database (if using country blocks) -└── backups/ Reserved for future use +└── backups/ Database snapshots ``` -No TLS certificates are stored — mc-proxy does not terminate TLS. +mc-proxy does not terminate TLS on the proxy listeners, so no proxy +certificates are needed. The `certs/` directory is for the gRPC admin +API's TLS and optional mTLS keypair. --- @@ -334,8 +393,7 @@ Items are listed roughly in priority order: | Item | Description | |------|-------------| -| **gRPC admin API** | Internal-only API for managing routes and firewall rules at runtime, integrated with the Metacircular Control Plane. | -| **SQLite route storage** | Migrate route table from TOML to SQLite for dynamic management via the admin API. | +| **MCP integration** | Wire the gRPC admin API into the Metacircular Control Plane for centralized management. | | **L7 HTTPS support** | TLS-terminating mode for selected routes, enabling HTTP-level features (user-agent blocking, header inspection, request routing). | | **ACME integration** | Automatic certificate provisioning via Let's Encrypt for L7 routes. | | **User-agent blocking** | Block connections based on user-agent string (requires L7 mode). | diff --git a/CLAUDE.md b/CLAUDE.md index cb9de74..ab247d4 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -15,6 +15,8 @@ make build # compile all packages make test # run all tests make vet # go vet make lint # golangci-lint +make proto # regenerate gRPC code from proto definitions +make devserver # build and run locally with srv/mc-proxy.toml ``` Run a single test: @@ -26,9 +28,10 @@ go test ./internal/sni -run TestExtract - **Module path**: `git.wntrmute.dev/kyle/mc-proxy` - **Go with CGO_ENABLED=0**, statically linked, Alpine containers -- **No API surface yet** — config-driven via TOML; gRPC admin API planned for future MCP integration -- **No auth** — this is pre-auth infrastructure; services behind it handle their own MCIAS auth -- **No database** — routes and firewall rules are in the TOML config; SQLite planned for dynamic route management +- **gRPC admin API** — manages routes and firewall rules at runtime; TLS with optional mTLS; optional (disabled if `[grpc]` section omitted from config) +- **No auth on proxy listeners** — this is pre-auth infrastructure; services behind it handle their own MCIAS auth +- **SQLite database** — persists listeners, routes, and firewall rules; pure-Go driver (`modernc.org/sqlite`); seeded from TOML on first run, DB is source of truth thereafter +- **Write-through pattern** — gRPC mutations write to DB first, then update in-memory state - **Config**: TOML via `go-toml/v2`, runtime data in `/srv/mc-proxy/` - **Testing**: stdlib `testing` only, `t.TempDir()` for isolation - **Linting**: golangci-lint v2 with `.golangci.yaml` @@ -36,10 +39,13 @@ go test ./internal/sni -run TestExtract ## Package Structure - `internal/config/` — TOML config loading and validation +- `internal/db/` — SQLite database: migrations, CRUD for listeners/routes/firewall rules, seeding, snapshots - `internal/sni/` — TLS ClientHello parser; extracts SNI hostname without consuming bytes -- `internal/firewall/` — global blocklist evaluation (IP, CIDR, GeoIP via MaxMind GeoLite2); thread-safe GeoIP reload +- `internal/firewall/` — global blocklist evaluation (IP, CIDR, GeoIP via MaxMind GeoLite2); thread-safe mutations and GeoIP reload - `internal/proxy/` — bidirectional TCP relay with half-close propagation and idle timeout -- `internal/server/` — orchestrates listeners → firewall → SNI → route → proxy pipeline; graceful shutdown +- `internal/server/` — orchestrates listeners → firewall → SNI → route → proxy pipeline; per-listener state with connection tracking +- `internal/grpcserver/` — gRPC admin API: route/firewall CRUD, status, write-through to DB +- `proto/mc-proxy/v1/` — protobuf definitions; `gen/mc-proxy/v1/` has generated code ## Signals @@ -52,3 +58,4 @@ go test ./internal/sni -run TestExtract - Firewall rules are always evaluated before any routing decision. - SNI matching is exact and case-insensitive. - Blocked connections get a TCP RST — no error messages, no TLS alerts. +- Database writes must succeed before in-memory state is updated (write-through).