crossbar/docs/plans/v0/03-proxy.md

v0 task 03: the routing reverse proxy

Branch: v0 (run git switch v0; git status --short must be empty, otherwise stop) Commit subject: Add the routing reverse proxy

Goal

Write internal/proxy: an http.Handler that takes /{route}/v1/…, picks a host from the route's list using the health table, forwards the request with httputil.ReverseProxy, streams the answer back as it arrives, and tells the health table when a host fails.

Context

Clients (OpenCode, Hermes) only know a base URL, so the route is the first path segment: http://crossbar:7777/opencode-a/v1/chat/completions. The upstream must see /v1/chat/completions with the query string kept. Answers are often server-sent-event streams of hundreds of small chunks over minutes; a proxy that buffers them makes the client look frozen, so FlushInterval is -1 (flush after every write) and anything wrapping the ResponseWriter must still implement http.Flusher. Request bodies are looked at once, for a top-level "model" field, so that a request for a model only one host has loaded goes there; the body is then handed to the upstream unchanged. Bodies are never logged.

Files

• Copy: internal/proxy/proxy_test.go
• Create: internal/proxy/proxy.go
• Modify: docs/implementer-log.md

Interfaces

Produces, in internal/proxy/proxy.go, package proxy:

const MaxBody = 16 << 20                 // largest request body we look at
const HostHeader = "X-Crossbar-Host"     // set on every proxied response: the host that answered

// Health is what the proxy needs from the health table (internal/health satisfies it).
type Health interface {
    Get(name string) (health.Status, bool)
    MarkDown(name, reason string)
}

type Handler struct { /* private: *config.Config, Health, *slog.Logger */ }

func New(cfg *config.Config, h Health, log *slog.Logger) *Handler     // nil log -> slog.Default()
func (p *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request)

// SplitRoute takes the first path segment as the route.
//   "/a/v1/x" -> ("a", "/v1/x", true)      "/a" and "/a/" -> ("a", "/", true)
//   "/", "//x", "", "noslash/v1" -> ("", "", false)
func SplitRoute(path string) (route, rest string, ok bool)

// Choose: the first host in order that is healthy and lists model in Loaded; failing that, the
// first healthy host; ok == false if none. model may be "".
func Choose(hosts []string, model string, h Health) (string, bool)

Rules the tests check, in the order ServeHTTP applies them. Every error answer is JSON {"error":"<msg>"} with Content-Type: application/json:

1. SplitRoute(r.URL.Path) not ok → 400 missing route.
2. Route not in cfg.Routes → 404 unknown route.
3. rest must start with /v1/ or be exactly /health or /props; else → 404 not found. (/_crossbar/… through a route is therefore 404 too.)
4. Model peek. For requests other than GET/HEAD with a body: read up to MaxBody + 1 bytes (io.LimitReader); more than MaxBody → 413 body too large. Put the bytes back (r.Body = io.NopCloser(bytes.NewReader(body)), r.ContentLength = len(body)). Then try to decode {"model": "…"}; a body that is not JSON, or has no model, simply gives "" — that is not an error. If the model is "", use the route's DefaultModel.
5. Choose(route.Hosts, model, h) not ok → 503 no healthy host. Nothing is marked down by rules 1–5.
6. Forward with a httputil.ReverseProxy:
   • Rewrite: pr.SetURL(target) where target is the host's BaseURL parsed once; pr.Out.URL.Path = target.Path + rest; pr.Out.URL.RawPath = ""; pr.Out.Host = target.Host; pr.SetXForwarded(). The query string is kept (the tests check /v1/models?x=1 arrives as /v1/models?x=1).
   • FlushInterval: -1.
   • ModifyResponse: set HostHeader to the host's name.
   • ErrorHandler: if errors.Is(err, context.Canceled), do nothing (the client left); otherwise h.MarkDown(name, err.Error()) and answer 502 {"error":"upstream failed","host":"<name>"}.
7. One log line per proxied request, after it finishes, through the logger: p.log.Info("request", "route", …, "host", …, "method", …, "path", rest, "status", …, "ms", …). To know the status, wrap the ResponseWriter in a small recorder that implements WriteHeader and Flush (forwarding to the underlying http.Flusher). Without Flush the reverse proxy cannot stream and TestStreamingIsNotBuffered fails.
8. Never log, print or keep a request or response body. Never panic on a request.

Steps

• 1. Copy.

git switch v0
cp docs/plans/v0/files/internal/proxy/proxy_test.go internal/proxy/

Read the test. fakeHealth stands in for the table; newUpstream records what arrived. TestStreamingIsNotBuffered is rule 6/7: the upstream sends one chunk and then waits until the test has read it; a buffering proxy hangs there.

• 2. See the test fail. go test ./internal/proxy/. Expected: it does not compile.
• 3. Write internal/proxy/proxy.go. gofmt -w internal/proxy/.
• 4. See the test pass. go test -race -count=1 ./internal/proxy/. Expected: ok.
• 5. Run the gate. make gate. Expected last line: gate: ok.
• 6. Log and commit. Row v0/03-proxy.

git add internal/proxy docs/implementer-log.md
git commit

Done when

• go test -race -count=1 ./internal/proxy/ is ok; make gate prints gate: ok.
• cmp internal/proxy/proxy_test.go docs/plans/v0/files/internal/proxy/proxy_test.go prints nothing.

Stop and report if

• TestStreamingIsNotBuffered still fails with FlushInterval: -1 and a recorder that implements Flush: stop and describe exactly what you wrote.