Extend the config, database schema, and server internals to support per-route L4/L7 mode selection and PROXY protocol fields. This is the foundation for L7 HTTP/2 reverse proxying and multi-hop PROXY protocol support described in the updated ARCHITECTURE.md. Config: Listener gains ProxyProtocol; Route gains Mode, TLSCert, TLSKey, BackendTLS, SendProxyProtocol. L7 routes validated at load time (cert/key pair must exist and parse). Mode defaults to "l4". DB: Migration v2 adds columns to listeners and routes tables. CRUD and seeding updated to persist all new fields. Server: RouteInfo replaces bare backend string in route lookup. handleConn dispatches on route.Mode (L7 path stubbed with error). ListenerState and ListenerData carry ProxyProtocol flag. All existing L4 tests pass unchanged. New tests cover migration v2, L7 field persistence, config validation for mode/cert/key, and proxy_protocol flag round-tripping. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
719 lines
31 KiB
Markdown
719 lines
31 KiB
Markdown
# ARCHITECTURE.md
|
|
|
|
mc-proxy is a TLS proxy and router for Metacircular Dynamics services. It
|
|
operates in two modes on a per-route basis:
|
|
|
|
- **L4 (passthrough):** Reads the SNI field from the TLS ClientHello and
|
|
proxies the raw TCP stream to the matched backend without terminating TLS.
|
|
This is the default mode and the original behavior.
|
|
- **L7 (terminating):** Terminates the client TLS connection, then reverse
|
|
proxies HTTP/2 (and HTTP/1.1) traffic to the backend. This enables
|
|
HTTP-level features: header inspection, user-agent blocking, request
|
|
routing, and `X-Forwarded-For` injection.
|
|
|
|
A global firewall evaluates every connection before routing. PROXY protocol
|
|
support enables multi-hop deployments where a public edge mc-proxy forwards
|
|
through a private network (e.g., Tailscale) to an origin mc-proxy while
|
|
preserving the real client IP.
|
|
|
|
## Table of Contents
|
|
|
|
1. [System Overview](#system-overview)
|
|
2. [Connection Lifecycle](#connection-lifecycle)
|
|
3. [PROXY Protocol](#proxy-protocol)
|
|
4. [Firewall](#firewall)
|
|
5. [Routing](#routing)
|
|
6. [L7 Proxying](#l7-proxying)
|
|
7. [gRPC Admin API](#grpc-admin-api)
|
|
8. [Configuration](#configuration)
|
|
9. [Storage](#storage)
|
|
10. [Deployment](#deployment)
|
|
11. [Security Model](#security-model)
|
|
12. [Future Work](#future-work)
|
|
|
|
---
|
|
|
|
## System Overview
|
|
|
|
```
|
|
┌──────────────────────────────────────────────┐
|
|
│ mc-proxy │
|
|
Clients ──────┐ │ │
|
|
│ │ ┌───────┐ ┌──────┐ ┌───┐ ┌─────┐ │
|
|
├────▶│ │ PROXY │─▶│ Fire │─▶│SNI│─▶│Route│──┐ │
|
|
│ │ │ proto │ │ wall │ │ │ │Table│ │ │
|
|
├────▶│ └───────┘ └──────┘ └───┘ └─────┘ │ │
|
|
│ │ │ │ RST │ │ │ │
|
|
Clients ──────┘ │ │ ▼ │ │ │ │
|
|
│ (optional) (blocked) │ │ │ │
|
|
│ │ │ │ │
|
|
│ ┌──────────────────────┘ │ │ │
|
|
│ │ L4 route │ │ │
|
|
│ ▼ │ │ │ ┌────────────┐
|
|
│ ┌──────────┐ │ │ │────▶│ Backend A │
|
|
│ │ Raw TCP │───────────────────│────│─────│ │ :8443 (TLS)│
|
|
│ │ Relay │ │ │ │ └────────────┘
|
|
│ └──────────┘ │ │ │
|
|
│ │ │ │
|
|
│ ┌─────────────────────────┘ │ │
|
|
│ │ L7 route │ │
|
|
│ ▼ │ │ ┌────────────┐
|
|
│ ┌──────────┐ ┌──────────┐ │ │────▶│ Backend B │
|
|
│ │ TLS Term │─▶│ HTTP/2 │──────────│─────│ │ :8080 (h2c)│
|
|
│ │ (certs) │ │ Rev Proxy│ │ │ └────────────┘
|
|
│ └──────────┘ └──────────┘ │ │
|
|
└──────────────────────────────────────────────┘
|
|
|
|
Listener 1 (:443) ─┐
|
|
Listener 2 (:8443) ─┼─ Each runs the same pipeline; L4/L7 is per-route
|
|
Listener N (:9443) ─┘
|
|
```
|
|
|
|
Key properties:
|
|
|
|
- **Dual mode, per-route.** Each route is independently configured as L4
|
|
(passthrough) or L7 (terminating). Both modes coexist on the same listener.
|
|
- **TLS-only on ingest.** Non-TLS connections are not supported. If the first
|
|
bytes of a connection are not a TLS ClientHello (after any PROXY protocol
|
|
header), the connection is reset.
|
|
- **Multiple listeners.** A single mc-proxy instance binds to one or more
|
|
ports. Each listener runs the same pipeline.
|
|
- **Global firewall.** Firewall rules apply to all listeners uniformly.
|
|
There are no per-route firewall rules.
|
|
- **PROXY protocol.** Listeners can accept PROXY protocol v1/v2 headers to
|
|
learn the real client IP from an upstream proxy. Routes can send PROXY
|
|
protocol v2 headers to downstream backends or proxies.
|
|
- **No authentication.** mc-proxy is pre-auth infrastructure. It sits in
|
|
front of services that handle their own authentication via MCIAS.
|
|
|
|
---
|
|
|
|
## Connection Lifecycle
|
|
|
|
Every inbound connection follows this sequence. Steps 1-5 are common to all
|
|
connections. The route's mode determines whether the L4 or L7 path is taken.
|
|
|
|
```
|
|
1. ACCEPT Listener accepts TCP connection.
|
|
2. PROXY PROTOCOL If listener has proxy_protocol enabled:
|
|
Parse PROXY v1/v2 header to extract real client IP.
|
|
If malformed or missing → RST, done.
|
|
Otherwise: use TCP source IP directly.
|
|
3. FIREWALL Check source IP (real, from step 2) against blocklists:
|
|
a. Per-IP rate limit check.
|
|
b. IP/CIDR block check.
|
|
c. GeoIP country block check.
|
|
If blocked → RST, done.
|
|
4. SNI EXTRACT Read the TLS ClientHello (without consuming it).
|
|
Extract the SNI hostname.
|
|
If no valid ClientHello or no SNI → RST, done.
|
|
5. ROUTE LOOKUP Match SNI hostname against the route table.
|
|
If no match → RST, done.
|
|
Route determines mode (L4 or L7).
|
|
|
|
── L4 path (passthrough) ──────────────────────────────────────────────
|
|
|
|
6a. BACKEND DIAL Open TCP connection to the matched backend.
|
|
If dial fails → RST, done.
|
|
7a. PROXY HEADER If route has send_proxy_protocol enabled:
|
|
Send PROXY v2 header with real client IP to backend.
|
|
8a. FORWARD Write the buffered ClientHello to the backend.
|
|
9a. RELAY Bidirectional byte copy: client ↔ backend.
|
|
10a. CLOSE Either side closes → half-close propagation → done.
|
|
|
|
── L7 path (terminating) ──────────────────────────────────────────────
|
|
|
|
6b. TLS HANDSHAKE Complete TLS handshake with the client.
|
|
The buffered ClientHello is replayed into crypto/tls
|
|
via a prefixConn wrapper. The server certificate is
|
|
selected from the route's configured cert/key pair
|
|
using tls.Config.GetCertificate keyed on SNI.
|
|
If handshake fails → close, done.
|
|
7b. HTTP PARSE Read HTTP/2 (or HTTP/1.1) request from the TLS conn.
|
|
8b. L7 POLICY Apply HTTP-level policies:
|
|
- User-agent blocking (future)
|
|
- Header inspection (future)
|
|
If blocked → HTTP error response, done.
|
|
9b. BACKEND DIAL Open connection to the matched backend.
|
|
If backend_tls: TLS connection to backend.
|
|
Otherwise: plaintext (h2c or HTTP/1.1).
|
|
If dial fails → HTTP 502, done.
|
|
10b. PROXY HEADER If route has send_proxy_protocol enabled:
|
|
Send PROXY v2 header with real client IP to backend.
|
|
11b. REVERSE PROXY Forward HTTP request to backend, stream response to
|
|
client. 1:1 connection mapping (no pooling).
|
|
12b. CLOSE Connection closed normally or on error.
|
|
```
|
|
|
|
### SNI Extraction
|
|
|
|
The proxy peeks at the initial bytes of the connection without consuming
|
|
them. It parses just enough of the TLS record layer and ClientHello to
|
|
extract the `server_name` extension.
|
|
|
|
- **L4 routes:** The full ClientHello (including the SNI) is forwarded to
|
|
the backend so the backend's TLS handshake proceeds normally.
|
|
- **L7 routes:** The buffered ClientHello is replayed into `crypto/tls.Server`
|
|
via a `prefixConn` wrapper (a `net.Conn` that returns the buffered bytes
|
|
before reading from the underlying connection). The TLS library sees the
|
|
complete ClientHello and performs a normal handshake.
|
|
|
|
If the ClientHello spans multiple TCP segments, the proxy buffers up to
|
|
16 KiB (the maximum TLS record size) before giving up.
|
|
|
|
### Bidirectional Copy (L4)
|
|
|
|
After the backend connection is established, the proxy runs two concurrent
|
|
copy loops (client->backend and backend->client). When either direction
|
|
encounters an EOF or error:
|
|
|
|
1. The write side of the opposite direction is half-closed.
|
|
2. The remaining direction drains to completion.
|
|
3. Both connections are closed.
|
|
|
|
Timeouts apply to the copy phase to prevent idle connections from
|
|
accumulating indefinitely (see [Configuration](#configuration)).
|
|
|
|
---
|
|
|
|
## PROXY Protocol
|
|
|
|
mc-proxy supports [PROXY protocol](https://www.haproxy.org/download/2.9/doc/proxy-protocol.txt)
|
|
for multi-hop deployments where the real client IP must be preserved across
|
|
proxy boundaries.
|
|
|
|
### Use Case: Edge-to-Origin
|
|
|
|
A common deployment has a public-facing mc-proxy on a VPS forwarding
|
|
through Tailscale to a private-network mc-proxy:
|
|
|
|
```
|
|
Client ──→ [edge mc-proxy] ──PROXY v2 + TLS──→ Tailscale ──→ [origin mc-proxy] ──→ backend
|
|
VPS, :443 private network
|
|
proxy_protocol = false proxy_protocol = true
|
|
send_proxy_protocol = true send_proxy_protocol = false
|
|
```
|
|
|
|
Without PROXY protocol, the origin mc-proxy sees the Tailscale peer IP as
|
|
the client address. With it, the real client IP is preserved for firewall
|
|
evaluation, logging, and `X-Forwarded-For` injection on L7 routes.
|
|
|
|
### Receiving (Listener Side)
|
|
|
|
When `proxy_protocol = true` on a listener:
|
|
|
|
1. After accepting the TCP connection, read the PROXY protocol header
|
|
**before** any TLS data.
|
|
2. Accept both v1 (text) and v2 (binary) formats.
|
|
3. Extract the real client IP and port from the header.
|
|
4. Use the real client IP for all subsequent processing (firewall, logging,
|
|
`X-Forwarded-For`).
|
|
5. If the header is malformed, missing, or the connection times out waiting
|
|
for it, reset the connection.
|
|
|
|
When `proxy_protocol = false` (default), the TCP source IP is used directly.
|
|
No PROXY header is expected or tolerated — a connection that starts with a
|
|
PROXY header will fail SNI extraction (it's not a TLS ClientHello) and be
|
|
reset. This prevents a non-PROXY listener from accidentally trusting
|
|
attacker-supplied headers.
|
|
|
|
### Sending (Route Side)
|
|
|
|
When `send_proxy_protocol = true` on a route:
|
|
|
|
1. After connecting to the backend, send a PROXY protocol v2 header before
|
|
any other data.
|
|
2. The header contains the real client IP (from the PROXY header on the
|
|
listener, or the TCP source IP if the listener is not PROXY-enabled).
|
|
3. For L4 routes: the v2 header is sent, then the buffered ClientHello,
|
|
then raw relay.
|
|
4. For L7 routes: the v2 header is sent on the backend connection before
|
|
HTTP traffic begins.
|
|
|
|
mc-proxy always sends v2 (binary format) when `send_proxy_protocol` is
|
|
enabled. v2 is more compact, supports IPv6 natively, and is the modern
|
|
standard.
|
|
|
|
### Header Timeout
|
|
|
|
A hardcoded 5-second timeout applies to reading the PROXY protocol header.
|
|
If the header is not fully received within this window, the connection is
|
|
reset. This prevents slowloris-style attacks on PROXY-enabled listeners.
|
|
|
|
---
|
|
|
|
## Firewall
|
|
|
|
The firewall is a global, ordered rule set evaluated on every new
|
|
connection before SNI extraction. Rules are evaluated in definition order;
|
|
the first matching rule determines the outcome. If no rule matches, the
|
|
connection is **allowed** (default allow -- the firewall is an explicit
|
|
blocklist, not an allowlist).
|
|
|
|
The source IP used for firewall evaluation is the **real client IP** --
|
|
either from a PROXY protocol header (if the listener is PROXY-enabled) or
|
|
the TCP source address.
|
|
|
|
### Rule Types
|
|
|
|
| Config Field | Match Field | Example |
|
|
|--------------|-------------|---------|
|
|
| `blocked_ips` | Source IP address | `"192.0.2.1"` |
|
|
| `blocked_cidrs` | Source IP prefix | `"198.51.100.0/24"` |
|
|
| `blocked_countries` | Source country (ISO 3166-1 alpha-2) | `"KP"`, `"CN"`, `"IN"`, `"IL"` |
|
|
|
|
### Blocked Connection Handling
|
|
|
|
Blocked connections receive a TCP RST. No TLS alert, no HTTP error page, no
|
|
indication of why the connection was refused. This is intentional -- blocked
|
|
sources should receive minimal information.
|
|
|
|
### GeoIP Database
|
|
|
|
mc-proxy uses the [MaxMind GeoLite2](https://dev.maxmind.com/geoip/geolite2-free-geolocation-data)
|
|
free database for country-level IP geolocation. The database file is
|
|
distributed separately from the binary and referenced by path in the
|
|
configuration.
|
|
|
|
The GeoIP database is loaded into memory at startup and can be reloaded
|
|
via `SIGHUP` without restarting the proxy. If the database file is missing
|
|
or unreadable at startup and GeoIP rules are configured, the proxy refuses
|
|
to start.
|
|
|
|
---
|
|
|
|
## Routing
|
|
|
|
Each listener has its own route table mapping SNI hostnames to backend
|
|
addresses. A route entry consists of:
|
|
|
|
| Field | Type | Default | Description |
|
|
|-------|------|---------|-------------|
|
|
| `hostname` | string | *(required)* | Exact SNI hostname to match (e.g. `metacrypt.metacircular.net`) |
|
|
| `backend` | string | *(required)* | Backend address as `host:port` (e.g. `127.0.0.1:8443`) |
|
|
| `mode` | string | `"l4"` | `"l4"` for passthrough, `"l7"` for TLS-terminating reverse proxy |
|
|
| `tls_cert` | string | `""` | Path to TLS certificate file (PEM). Required for L7 routes. |
|
|
| `tls_key` | string | `""` | Path to TLS private key file (PEM). Required for L7 routes. |
|
|
| `backend_tls` | bool | `false` | If true, connect to backend with TLS (re-encrypt). If false, plaintext h2c/HTTP. |
|
|
| `send_proxy_protocol` | bool | `false` | If true, send PROXY protocol v2 header to backend. |
|
|
|
|
Routes are scoped to the listener that accepted the connection. The same
|
|
hostname can appear on different listeners with different backends and
|
|
different modes.
|
|
|
|
### Match Semantics
|
|
|
|
- Hostname matching is **exact** and **case-insensitive** (per RFC 6066,
|
|
SNI hostnames are DNS names and compared case-insensitively).
|
|
- Wildcard matching is not supported in the initial implementation.
|
|
- If duplicate hostnames appear within the same listener, the proxy refuses
|
|
to start.
|
|
- If no route matches an incoming SNI hostname, the connection is reset.
|
|
|
|
### Route Table Source
|
|
|
|
Route tables are persisted in the SQLite database. On first run, they are
|
|
seeded from the TOML configuration. On subsequent runs, the database is
|
|
the source of truth. Routes can be added or removed at runtime via the
|
|
gRPC admin API.
|
|
|
|
### Validation
|
|
|
|
- L7 routes must have `tls_cert` and `tls_key` set and pointing to
|
|
readable files. The proxy validates certificate/key pairs at startup and
|
|
refuses to start if any are invalid.
|
|
- L4 routes must not have `tls_cert` or `tls_key` set (they are ignored
|
|
and produce a warning if present).
|
|
- `backend_tls` is only meaningful for L7 routes. On L4 routes it is
|
|
ignored (the raw TLS stream passes through regardless).
|
|
|
|
---
|
|
|
|
## L7 Proxying
|
|
|
|
L7 routes terminate TLS at the proxy and reverse proxy HTTP traffic to the
|
|
backend. This enables HTTP-level features that are impossible in L4 mode.
|
|
|
|
### TLS Termination
|
|
|
|
The proxy uses `crypto/tls.Server` with a `GetCertificate` callback that
|
|
selects the certificate based on the SNI hostname. Since SNI has already
|
|
been extracted in step 4 of the connection lifecycle, the route (and its
|
|
cert/key) is known before the handshake begins.
|
|
|
|
The `prefixConn` wrapper replays the buffered ClientHello bytes, then reads
|
|
from the underlying `net.Conn`. From `crypto/tls`'s perspective, it is
|
|
reading a normal TCP connection.
|
|
|
|
TLS configuration:
|
|
|
|
- Minimum version: TLS 1.2 (TLS 1.3 preferred).
|
|
- Cipher suites: Go's default secure set (no manual override).
|
|
- ALPN: `h2` and `http/1.1` advertised. The negotiated protocol determines
|
|
whether the connection uses HTTP/2 or HTTP/1.1.
|
|
|
|
### HTTP/2 Reverse Proxy
|
|
|
|
After the TLS handshake, the proxy serves HTTP using Go's `net/http` server
|
|
with HTTP/2 enabled. Each request is forwarded to the backend using a
|
|
reverse proxy.
|
|
|
|
**Backend transports:**
|
|
|
|
| `backend_tls` | Protocol | Description |
|
|
|----------------|----------|-------------|
|
|
| `false` | h2c | HTTP/2 over plaintext TCP. Used for localhost/trusted-network backends. Supports gRPC and standard HTTP. |
|
|
| `true` | h2 (TLS) | HTTP/2 over TLS to backend. Used when backend is remote or requires encryption. |
|
|
|
|
The reverse proxy handles:
|
|
|
|
- **HTTP/2 and HTTP/1.1**: Protocol negotiated via ALPN on the client side.
|
|
Backend connections use h2c (or h2 if `backend_tls`). If the backend only
|
|
speaks HTTP/1.1, Go's transport handles the protocol translation.
|
|
- **gRPC**: gRPC is HTTP/2 with `content-type: application/grpc`. The
|
|
reverse proxy forwards HTTP/2 frames, including **trailers** (critical
|
|
for gRPC status codes and error details).
|
|
- **Streaming**: Server-side, client-side, and bidirectional gRPC streaming
|
|
work through the reverse proxy. Request and response bodies are streamed,
|
|
not buffered.
|
|
- **Header injection**: `X-Forwarded-For`, `X-Forwarded-Proto`, and
|
|
`X-Real-IP` are set from the real client IP (PROXY protocol or TCP source).
|
|
- **Hop-by-hop headers**: `Connection`, `Transfer-Encoding`, and other
|
|
hop-by-hop headers are stripped per RFC 7230.
|
|
|
|
### Connection Model
|
|
|
|
L7 uses 1:1 connection mapping: each client connection maps to one backend
|
|
connection. There is no connection pooling. This keeps the implementation
|
|
simple and the connection lifecycle predictable. Pooling may be added later
|
|
if backend connection setup becomes a bottleneck.
|
|
|
|
### Error Handling
|
|
|
|
| Condition | Client Response |
|
|
|-----------|----------------|
|
|
| TLS handshake failure | Connection closed (TCP RST) |
|
|
| Backend dial failure | HTTP 502 Bad Gateway |
|
|
| Backend timeout | HTTP 504 Gateway Timeout |
|
|
| Backend returns error | Error response forwarded as-is |
|
|
|
|
---
|
|
|
|
## gRPC Admin API
|
|
|
|
The admin API is optional (disabled if `[grpc]` is omitted from the config).
|
|
It listens on a Unix domain socket for security -- access is controlled via
|
|
filesystem permissions (0600, owner-only). The API provides runtime management
|
|
of routes and firewall rules without restarting the proxy.
|
|
|
|
### RPCs
|
|
|
|
| RPC | Description |
|
|
|-----|-------------|
|
|
| `ListRoutes` | List all routes for a given listener |
|
|
| `AddRoute` | Add a route to a listener (write-through to DB) |
|
|
| `RemoveRoute` | Remove a route from a listener (write-through to DB) |
|
|
| `GetFirewallRules` | List all firewall rules |
|
|
| `AddFirewallRule` | Add a firewall rule (write-through to DB) |
|
|
| `RemoveFirewallRule` | Remove a firewall rule (write-through to DB) |
|
|
| `GetStatus` | Return version, uptime, listener status, connection counts |
|
|
| `grpc.health.v1.Health` | Standard gRPC health check (Check, Watch) |
|
|
|
|
### Input Validation
|
|
|
|
The admin API validates all inputs before persisting:
|
|
|
|
- **Route backends** must be valid `host:port` tuples.
|
|
- **Route mode** must be `"l4"` or `"l7"`.
|
|
- **L7 routes** must include valid `tls_cert` and `tls_key` paths.
|
|
- **IP firewall rules** must be valid IP addresses (`netip.ParseAddr`).
|
|
- **CIDR firewall rules** must be valid prefixes in canonical form.
|
|
- **Country firewall rules** must be exactly 2 uppercase letters (ISO 3166-1 alpha-2).
|
|
|
|
### Security
|
|
|
|
The gRPC admin API has no MCIAS integration -- mc-proxy is pre-auth
|
|
infrastructure. Access control relies on Unix socket filesystem permissions:
|
|
|
|
- Socket is created with mode `0600` (read/write for owner only)
|
|
- Only processes running as the same user can connect
|
|
- No network exposure -- the API is not accessible over TCP
|
|
|
|
---
|
|
|
|
## Configuration
|
|
|
|
TOML configuration file, loaded at startup. The proxy refuses to start if
|
|
required fields are missing or invalid.
|
|
|
|
```toml
|
|
# Database. Required.
|
|
[database]
|
|
path = "/srv/mc-proxy/mc-proxy.db"
|
|
|
|
# Listeners. Each has its own route table (seeds DB on first run).
|
|
[[listeners]]
|
|
addr = ":443"
|
|
proxy_protocol = false # accept PROXY protocol headers (default: false)
|
|
|
|
# L4 route: passthrough, no TLS termination.
|
|
[[listeners.routes]]
|
|
hostname = "metacrypt.metacircular.net"
|
|
backend = "127.0.0.1:18443"
|
|
mode = "l4"
|
|
|
|
# L7 route: terminate TLS, reverse proxy HTTP/2.
|
|
[[listeners.routes]]
|
|
hostname = "api.metacircular.net"
|
|
backend = "127.0.0.1:8080"
|
|
mode = "l7"
|
|
tls_cert = "/srv/mc-proxy/certs/api.crt"
|
|
tls_key = "/srv/mc-proxy/certs/api.key"
|
|
|
|
# L4 route with PROXY protocol to downstream proxy.
|
|
[[listeners.routes]]
|
|
hostname = "mcias.metacircular.net"
|
|
backend = "10.0.0.5:443"
|
|
mode = "l4"
|
|
send_proxy_protocol = true
|
|
|
|
[[listeners]]
|
|
addr = ":8443"
|
|
|
|
[[listeners.routes]]
|
|
hostname = "metacrypt.metacircular.net"
|
|
backend = "127.0.0.1:18443"
|
|
|
|
# gRPC admin API. Optional -- omit or leave addr empty to disable.
|
|
# Listens on a Unix socket; access controlled via filesystem permissions.
|
|
[grpc]
|
|
addr = "/var/run/mc-proxy.sock"
|
|
|
|
# Firewall. Global blocklist, evaluated before routing. Default allow.
|
|
[firewall]
|
|
geoip_db = "/srv/mc-proxy/GeoLite2-Country.mmdb"
|
|
blocked_ips = ["192.0.2.1"]
|
|
blocked_cidrs = ["198.51.100.0/24"]
|
|
blocked_countries = ["KP", "CN", "IN", "IL"]
|
|
rate_limit = 20 # max new connections per IP per rate_window
|
|
rate_window = "10s"
|
|
|
|
# Proxy behavior.
|
|
[proxy]
|
|
connect_timeout = "5s" # Timeout for dialing backend
|
|
idle_timeout = "300s" # Close connections idle longer than this
|
|
shutdown_timeout = "30s" # Graceful shutdown drain period
|
|
|
|
# Logging.
|
|
[log]
|
|
level = "info" # debug, info, warn, error
|
|
```
|
|
|
|
### Environment Variable Overrides
|
|
|
|
Configuration values can be overridden via environment variables using the
|
|
prefix `MCPROXY_` with underscore-separated paths:
|
|
|
|
```
|
|
MCPROXY_LOG_LEVEL=debug
|
|
MCPROXY_PROXY_IDLE_TIMEOUT=600s
|
|
```
|
|
|
|
Environment variables cannot define listeners, routes, or firewall rules --
|
|
these are structural and must be in the TOML file.
|
|
|
|
---
|
|
|
|
## Storage
|
|
|
|
### SQLite Database
|
|
|
|
Listeners, routes, and firewall rules are persisted in a SQLite database
|
|
(WAL mode, foreign keys enabled, busy timeout 5000ms). The pure-Go driver
|
|
`modernc.org/sqlite` is used (no CGo).
|
|
|
|
**Startup behavior:**
|
|
|
|
1. Open the database at the configured path. Run migrations.
|
|
2. If the database is empty (first run): seed from the TOML config.
|
|
3. If the database has data: load from it. TOML listener/route/firewall
|
|
fields are ignored.
|
|
|
|
The TOML config continues to own operational settings: proxy timeouts,
|
|
log level, gRPC config, GeoIP database path.
|
|
|
|
**Write-through pattern:** The gRPC admin API writes to the database first,
|
|
then updates in-memory state. If the database write fails, the in-memory
|
|
state is not modified.
|
|
|
|
### Schema
|
|
|
|
```sql
|
|
CREATE TABLE listeners (
|
|
id INTEGER PRIMARY KEY,
|
|
addr TEXT NOT NULL UNIQUE,
|
|
proxy_protocol INTEGER NOT NULL DEFAULT 0
|
|
);
|
|
|
|
CREATE TABLE routes (
|
|
id INTEGER PRIMARY KEY,
|
|
listener_id INTEGER NOT NULL REFERENCES listeners(id) ON DELETE CASCADE,
|
|
hostname TEXT NOT NULL,
|
|
backend TEXT NOT NULL,
|
|
mode TEXT NOT NULL DEFAULT 'l4' CHECK(mode IN ('l4', 'l7')),
|
|
tls_cert TEXT NOT NULL DEFAULT '',
|
|
tls_key TEXT NOT NULL DEFAULT '',
|
|
backend_tls INTEGER NOT NULL DEFAULT 0,
|
|
send_proxy_protocol INTEGER NOT NULL DEFAULT 0,
|
|
UNIQUE(listener_id, hostname)
|
|
);
|
|
|
|
CREATE TABLE firewall_rules (
|
|
id INTEGER PRIMARY KEY,
|
|
type TEXT NOT NULL CHECK(type IN ('ip', 'cidr', 'country')),
|
|
value TEXT NOT NULL,
|
|
UNIQUE(type, value)
|
|
);
|
|
```
|
|
|
|
### Data Directory
|
|
|
|
```
|
|
/srv/mc-proxy/
|
|
├── mc-proxy.toml Configuration
|
|
├── mc-proxy.db SQLite database
|
|
├── mc-proxy.sock Unix socket for gRPC admin API
|
|
├── certs/ TLS certificates for L7 routes
|
|
│ ├── api.crt
|
|
│ └── api.key
|
|
├── GeoLite2-Country.mmdb GeoIP database (if using country blocks)
|
|
└── backups/ Database snapshots
|
|
```
|
|
|
|
L4 routes do not use any TLS material -- the raw TLS stream passes through.
|
|
L7 routes require certificate/key pairs in the `certs/` directory (or any
|
|
path specified in the route config). The gRPC admin API uses a Unix socket.
|
|
|
|
---
|
|
|
|
## Deployment
|
|
|
|
### Binary
|
|
|
|
Single static binary, built with `CGO_ENABLED=0`. No runtime dependencies
|
|
beyond the configuration file, optional GeoIP database, and TLS
|
|
certificates for any L7 routes.
|
|
|
|
### Container
|
|
|
|
Multi-stage Docker build:
|
|
|
|
1. **Builder**: `golang:<version>-alpine`, static compilation.
|
|
2. **Runtime**: `alpine`, non-root user (`mc-proxy`), port exposure
|
|
determined by configuration.
|
|
|
|
### systemd
|
|
|
|
| File | Purpose |
|
|
|------|---------|
|
|
| `mc-proxy.service` | Main proxy service |
|
|
| `mc-proxy-backup.service` | Oneshot database backup (VACUUM INTO) |
|
|
| `mc-proxy-backup.timer` | Daily backup timer (02:00 UTC, 5-minute jitter) |
|
|
|
|
The proxy binds to privileged ports (443) and should use `AmbientCapabilities=CAP_NET_BIND_SERVICE`
|
|
in the systemd unit rather than running as root.
|
|
|
|
Standard security hardening directives apply per engineering standards
|
|
(`NoNewPrivileges=true`, `ProtectSystem=strict`, etc.).
|
|
|
|
### Multi-Hop Deployment
|
|
|
|
A typical production topology uses two mc-proxy instances:
|
|
|
|
```
|
|
Internet ──→ [edge mc-proxy] ──Tailscale──→ [origin mc-proxy] ──→ backends
|
|
VPS, :443 private network
|
|
```
|
|
|
|
**Edge mc-proxy** (public VPS):
|
|
- Listeners with `proxy_protocol = false` (clients connect directly).
|
|
- All routes are L4 passthrough with `send_proxy_protocol = true`.
|
|
- Firewall rules for GeoIP and IP blocking applied here.
|
|
|
|
**Origin mc-proxy** (private network):
|
|
- Listeners with `proxy_protocol = true` (receives real client IP from edge).
|
|
- Mix of L4 and L7 routes depending on the backend service.
|
|
- L7 routes terminate TLS and reverse proxy HTTP/2 to local backends.
|
|
- Firewall rules applied using the real client IP from the PROXY header.
|
|
|
|
### Graceful Shutdown
|
|
|
|
On `SIGINT` or `SIGTERM`:
|
|
|
|
1. Stop accepting new connections on all listeners.
|
|
2. Wait for in-flight connections to complete (up to `shutdown_timeout`).
|
|
3. Force-close remaining connections.
|
|
4. Exit.
|
|
|
|
On `SIGHUP`:
|
|
|
|
1. Reload the GeoIP database from disk.
|
|
2. Continue serving with the updated database.
|
|
|
|
Routes and firewall rules can be modified at runtime via the gRPC admin API
|
|
(write-through to SQLite). Listener changes (adding/removing ports) require
|
|
a full restart. TOML configuration changes (timeouts, log level, GeoIP path)
|
|
also require a restart.
|
|
|
|
---
|
|
|
|
## Security Model
|
|
|
|
mc-proxy is infrastructure that sits in front of authenticated services.
|
|
It has no authentication or authorization of its own.
|
|
|
|
### Threat Mitigations
|
|
|
|
| Threat | Mitigation |
|
|
|--------|------------|
|
|
| SNI spoofing | L4: Backend performs its own TLS handshake -- spoofed SNI fails certificate validation at the backend. L7: The proxy validates the certificate against the SNI during its own handshake, so a mismatched SNI results in a handshake failure before reaching the backend. |
|
|
| TLS private key compromise (L7) | Keys are stored on the filesystem with restrictive permissions. The proxy process runs as a dedicated non-root user. Keys are loaded into memory at startup; the file can be made read-only after startup. Consider integration with metacrypt for key storage (future). |
|
|
| Client IP spoofing via PROXY protocol | PROXY protocol headers are only trusted on listeners explicitly configured with `proxy_protocol = true`. Non-PROXY listeners reject connections that start with a PROXY header (they fail SNI extraction). Edge listeners facing the public internet should never enable `proxy_protocol`. |
|
|
| Resource exhaustion (connection flood) | Idle timeout closes stale connections. Per-IP rate limiting. Per-listener connection limits (future). |
|
|
| GeoIP evasion via IPv6 | GeoLite2 database includes IPv6 mappings. Both IPv4 and IPv6 source addresses are checked. |
|
|
| GeoIP evasion via VPN/proxy | Accepted risk. GeoIP blocking is a compliance measure, not a security boundary. Determined adversaries will bypass it. |
|
|
| Slowloris / slow ClientHello | Hardcoded 10-second timeout on the SNI extraction phase. 5-second timeout on PROXY protocol header parsing. |
|
|
| Backend unavailability | Connect timeout prevents indefinite hangs. L4: connection reset. L7: HTTP 502 response. |
|
|
| Information leakage (L4) | Blocked connections receive only a TCP RST. No version strings, no error messages, no TLS alerts. |
|
|
| Information leakage (L7) | Error responses are minimal (502/504 with no detail). The proxy does not expose its version or software name in HTTP headers. |
|
|
| HTTP request smuggling (L7) | Go's `net/http` server handles HTTP parsing. The reverse proxy uses `httputil.ReverseProxy` which strips hop-by-hop headers. HTTP/2 framing prevents classic smuggling vectors. |
|
|
|
|
### Security Invariants
|
|
|
|
1. **L4 routes never terminate TLS.** The proxy cannot read application-layer
|
|
traffic on L4 routes. It never modifies the byte stream.
|
|
2. **L7 routes terminate TLS at the proxy.** The proxy can read HTTP headers
|
|
and request metadata on L7 routes. It cannot read encrypted backend
|
|
traffic when `backend_tls = true`.
|
|
3. Firewall rules are always evaluated before any routing decision.
|
|
4. PROXY protocol is only parsed on explicitly enabled listeners.
|
|
5. The proxy never logs request or response bodies -- only metadata (source
|
|
IP, SNI hostname, backend, timestamps, bytes transferred, HTTP status
|
|
codes on L7 routes).
|
|
|
|
---
|
|
|
|
## Future Work
|
|
|
|
Items are listed roughly in priority order:
|
|
|
|
| Item | Description |
|
|
|------|-------------|
|
|
| **ACME integration** | Automatic certificate provisioning via Let's Encrypt for L7 routes, removing the need for manual cert management. |
|
|
| **L7 policies** | User-agent blocking, header-based routing, request rate limiting per endpoint. Requires L7 mode. |
|
|
| **MCP integration** | Wire the gRPC admin API into the Metacircular Control Plane for centralized management. |
|
|
| **Connection pooling** | Pool backend connections for L7 routes to reduce connection setup overhead under high request volume. |
|
|
| **Per-listener connection limits** | Cap maximum concurrent connections per listener. |
|
|
| **Metrics** | Prometheus-compatible metrics: connections per listener, firewall blocks by rule, backend dial latency, active connections, HTTP status code distributions. |
|
|
| **Metacrypt key storage** | Store L7 TLS private keys in metacrypt rather than on the filesystem. |
|