From bf7d7126c342f4471f39dc3a7bfee2dde369ca6f Mon Sep 17 00:00:00 2001 From: Kyle Isom Date: Thu, 26 Mar 2026 23:50:20 -0700 Subject: [PATCH] Add ARCHITECTURE.md and CLAUDE.md Design specification for mcdoc: a public documentation server that fetches markdown from Gitea, renders with goldmark, and serves via mc-proxy. Cache-first architecture with webhook refresh. No auth, no database, no local state beyond config. Co-Authored-By: Claude Opus 4.6 (1M context) --- ARCHITECTURE.md | 347 ++++++++++++++++++++++++++++++++++++++++++++++++ CLAUDE.md | 71 ++++++++++ 2 files changed, 418 insertions(+) create mode 100644 ARCHITECTURE.md create mode 100644 CLAUDE.md diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 0000000..f5c8e3c --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,347 @@ +# mcdoc Architecture + +Metacircular Documentation Server — Technical Design Document + +--- + +## 1. Overview + +mcdoc is a public documentation server for the Metacircular platform. It +fetches markdown files from Gitea, renders them to HTML, and serves a +navigable read-only documentation site. No authentication is required. + +mcdoc replaces the need to check out repositories or use a local tool to +read platform documentation. It is the web counterpart to browsing docs +locally — anyone with the URL can read the current documentation for any +Metacircular service. + +### Goals + +- Serve rendered markdown documentation for all Metacircular services +- Public access, no authentication +- Refresh automatically when documentation changes (Gitea webhooks) +- Fit the platform conventions (Go, htmx, MCP-deployed, mc-proxy-routed) + +### Non-Goals + +- Editing or authoring (docs are authored in git) +- Versioned documentation (tagged snapshots, historical diffs) +- Serving non-markdown content (images, PDFs) +- Search (v1 — may be added later) + +--- + +## 2. Architecture + +``` + ┌────────────────────────┐ + │ Gitea (deimos) │ + │ git.wntrmute.dev │ + │ │ + │ mc/mcr │ + │ mc/mcp │ + │ mc/metacrypt │ + │ ... │ + └──────────┬─────────────┘ + │ + fetch (boot + webhook) + │ + ▼ + Reader ──── HTTPS ────► mc-proxy ────► mcdoc (rift) + :443 (L7) ┌──────────────────┐ + docs. │ │ + metacircular. │ Content cache │ + net │ (in-memory) │ + │ │ + │ Rendered HTML │ + │ per repo/file │ + │ │ + └──────────────────┘ +``` + +mcdoc runs on rift as a single container. mc-proxy terminates TLS on +`docs.metacircular.net` and forwards to mcdoc's HTTP listener. + +Internally, mcdoc is also reachable at `mcdoc.svc.mcp.metacircular.net` +for Gitea webhook delivery and internal access. + +--- + +## 3. Content Model + +### Source + +mcdoc fetches markdown files from public repositories in the `mc` +organization on Gitea (`git.wntrmute.dev`). The repos are public, so no +authentication token is needed. + +### Discovery + +On boot, mcdoc fetches the list of repositories in the `mc` org via the +Gitea API, then for each repo: + +1. Fetch the file tree (recursive) from the default branch. +2. Filter to `*.md` files. +3. Exclude files matching configurable patterns (default: `vendor/`, + `.claude/`, `node_modules/`). +4. Fetch each markdown file's raw content. +5. Render to HTML via goldmark (GitHub Flavored Markdown). +6. Store the rendered HTML in the in-memory cache, keyed by + `repo/filepath`. + +### Navigation Structure + +The site is organized by repository, then by file path within the repo: + +``` +/ → index: list of repos with descriptions +/mcr/ → repo index: list of docs in mcr +/mcr/ARCHITECTURE → rendered ARCHITECTURE.md +/mcr/RUNBOOK → rendered RUNBOOK.md +/mcp/ARCHITECTURE → rendered mcp ARCHITECTURE.md +/mcp/docs/bootstrap → rendered mcp docs/bootstrap.md +``` + +The `.md` extension is stripped from URLs. Directory structure within +a repo is preserved. + +### Ordering + +The repo index page lists documents in a fixed priority order for +well-known filenames, followed by alphabetical: + +1. README +2. ARCHITECTURE +3. RUNBOOK +4. CLAUDE +5. (all others, alphabetical) + +The top-level index lists repos alphabetically with the first line of +each repo's Gitea description as a subtitle. + +--- + +## 4. Caching and Refresh + +### Boot Fetch + +On startup, mcdoc fetches all content from Gitea. This is the +cold-start path. With ~8 repos and ~50 markdown files total, the full +fetch takes 2-3 seconds using concurrent requests (bounded to avoid +overwhelming Gitea). Fetches are parallelized per-repo. + +mcdoc serves requests only after the initial fetch completes. During +boot, it returns HTTP 503 with a "loading" page. + +### Webhook Refresh + +Gitea sends a push webhook to `mcdoc.svc.mcp.metacircular.net/webhook` +on every push to a configured repo's default branch. mcdoc re-fetches +that repo's file tree and content, re-renders, and atomically swaps the +cache entries for that repo. In-flight requests to other repos are +unaffected. + +The webhook endpoint validates the Gitea webhook secret (shared secret, +HMAC-SHA256 signature in `X-Gitea-Signature` header). + +### Fallback Poll + +As a safety net, mcdoc polls Gitea for changes every 15 minutes. This +catches missed webhooks (network blips, Gitea restarts). The poll +checks each repo's latest commit SHA against the cached version and +only re-fetches repos that have changed. + +### Resilience + +- **Gitea unreachable at boot**: mcdoc starts, serves a "docs + unavailable, retrying" page, and retries the fetch every 30 seconds + until it succeeds. +- **Gitea unreachable after boot**: stale cache continues serving. + Readers see the last-known-good content. Poll/webhook failures are + logged but do not affect availability. +- **Single file fetch failure**: skip the file, log a warning, serve + the rest of the repo's docs. Retry on next poll cycle. + +--- + +## 5. Rendering + +### Markdown + +goldmark with the following extensions: + +- GitHub Flavored Markdown (tables, strikethrough, autolinks, task lists) +- Syntax highlighting for fenced code blocks (chroma) +- Heading anchors (linkable `#section-name` fragments) +- Table of contents generation (extracted from headings, rendered as a + sidebar or top-of-page nav) + +### HTML Output + +Each markdown file is rendered into a page template with: + +- **Header**: site title, repo name, breadcrumb navigation +- **Sidebar**: document list for the current repo (persistent nav) +- **Content**: rendered markdown +- **Footer**: last-updated timestamp (from Gitea commit metadata) + +The page template uses htmx for navigation — clicking a doc link swaps +the content pane without a full page reload, keeping the sidebar state. + +### Styling + +Clean, readable typography optimized for long-form technical documents. +The design should prioritize readability: + +- Serif or readable sans-serif body text +- Generous line height and margins +- Constrained content width (~70ch) +- Syntax-highlighted code blocks with a muted theme +- Responsive layout (readable on mobile) + +--- + +## 6. Configuration + +TOML configuration at `/srv/mcdoc/mcdoc.toml`: + +```toml +[server] +listen_addr = ":8080" + +[gitea] +url = "https://git.wntrmute.dev" +org = "mc" +webhook_secret = "..." +poll_interval = "15m" +fetch_timeout = "30s" +max_concurrency = 4 + +[gitea.exclude_paths] +patterns = ["vendor/", ".claude/", "node_modules/", ".junie/"] + +[gitea.exclude_repos] +names = [] + +[log] +level = "info" +``` + +Environment variable overrides follow platform convention: `MCDOC_*` +(e.g., `MCDOC_GITEA_WEBHOOK_SECRET`). + +--- + +## 7. Deployment + +### Container + +Single binary, single container. Multi-stage Docker build per platform +convention (`golang:alpine` builder, `alpine` runtime). + +mcdoc listens on a single HTTP port. mc-proxy handles TLS termination +and routes `docs.metacircular.net` to mcdoc's listener. + +### MCP Service Definition + +```toml +name = "mcdoc" +node = "rift" +active = true +path = "mcdoc" + +[build] +uses_mcdsl = false + +[build.images] +mcdoc = "Dockerfile" + +[[components]] +name = "mcdoc" +image = "mcr.svc.mcp.metacircular.net:8443/mcdoc:v0.1.0" +network = "mcpnet" +restart = "unless-stopped" +volumes = ["/srv/mcdoc:/srv/mcdoc"] +cmd = ["server", "--config", "/srv/mcdoc/mcdoc.toml"] +``` + +Port assignment is pending MCP support for automatic port allocation +and mc-proxy route registration. Until then, a manually assigned port +will be used. + +### mc-proxy Routes + +```toml +# On :443 listener +[[listeners.routes]] +hostname = "docs.metacircular.net" +backend = "127.0.0.1:" +mode = "l7" +tls_cert = "/srv/mc-proxy/certs/docs.pem" +tls_key = "/srv/mc-proxy/certs/docs.key" +backend_tls = false + +[[listeners.routes]] +hostname = "mcdoc.svc.mcp.metacircular.net" +backend = "127.0.0.1:" +mode = "l7" +tls_cert = "/srv/mc-proxy/certs/mcdoc-svc.pem" +tls_key = "/srv/mc-proxy/certs/mcdoc-svc.key" +backend_tls = false +``` + +Note: `backend_tls = false` — mcdoc is plain HTTP behind mc-proxy. +This is safe because mc-proxy and mcdoc are on the same host. TLS is +terminated at mc-proxy. + +### DNS + +| Record | Value | +|--------|-------| +| `docs.metacircular.net` | Public DNS → rift's public IP | +| `mcdoc.svc.mcp.metacircular.net` | Internal DNS (MCNS) → rift | + +--- + +## 8. Package Structure + +``` +cmd/mcdoc/ CLI entry point (cobra: server subcommand) +internal/ + config/ TOML config loading and validation + gitea/ Gitea API client (list repos, fetch trees, fetch files) + cache/ In-memory content cache (atomic swap per repo) + render/ goldmark rendering pipeline + server/ HTTP server, chi routes, htmx handlers +web/ + templates/ Go html/template files (index, repo, doc, error) + static/ CSS, favicon +``` + +--- + +## 9. Routes + +| Method | Path | Description | +|--------|------|-------------| +| GET | `/` | Top-level index (repo list) | +| GET | `/{repo}/` | Repo index (doc list) | +| GET | `/{repo}/{path...}` | Rendered document | +| POST | `/webhook` | Gitea push webhook receiver | +| GET | `/health` | Health check (200 if cache is populated, 503 if not) | + +htmx partial responses: when `HX-Request` header is present, return +only the content fragment (no surrounding layout). This enables +client-side navigation without full page reloads. + +--- + +## 10. Future Work + +| Item | Description | +|------|-------------| +| **Search** | Full-text search across all docs (bleve or similar) | +| **Cross-linking** | Resolve relative markdown links across repos | +| **Mermaid/diagrams** | Render mermaid fenced blocks as SVG | +| **Dark mode** | Theme toggle (light/dark) | +| **Pinned versions** | Serve docs at a specific git tag | diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..859a22c --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,71 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Project Overview + +mcdoc is a public documentation server for the Metacircular platform. It +fetches markdown files from Gitea repositories, renders them to HTML, and +serves a navigable read-only site. No authentication required. + +See `ARCHITECTURE.md` for the full specification. + +**Priorities (in order):** correctness, simplicity, readability. + +## Build Commands + +```bash +make all # vet → lint → test → build +make mcdoc # build binary with version injection +make build # compile all packages +make test # run all tests +make vet # go vet +make lint # golangci-lint +make devserver # build and run locally with srv/mcdoc.toml +make clean # remove built binaries +make docker # build container image +``` + +Run a single test: +```bash +go test ./internal/server -run TestDocPage +``` + +## Tech Stack + +- **Language:** Go 1.25+, `CGO_ENABLED=0`, statically linked +- **Module path:** `git.wntrmute.dev/kyle/mcdoc` +- **Config:** TOML via `go-toml/v2`, env overrides via `MCDOC_*` +- **HTTP:** chi router, htmx for navigation +- **Rendering:** goldmark (GFM), chroma (syntax highlighting) +- **Templates:** Go `html/template`, embedded via `//go:embed` +- **Linting:** golangci-lint v2 + +## Architecture + +- Single binary, single container +- Fetches markdown from Gitea API (mc org, public repos) +- Renders to HTML at fetch time, caches in memory +- Refreshes via Gitea push webhooks + 15-minute poll fallback +- mc-proxy terminates TLS, mcdoc serves plain HTTP +- No database, no local state beyond config + +## Package Structure + +- `cmd/mcdoc/` — CLI entry point (cobra: server subcommand) +- `internal/config/` — TOML config loading +- `internal/gitea/` — Gitea API client +- `internal/cache/` — In-memory content cache +- `internal/render/` — goldmark rendering pipeline +- `internal/server/` — HTTP server, chi routes, htmx handlers +- `web/` — Templates and static files, embedded via `//go:embed` + +## Critical Rules + +- No authentication — this is a public site +- Never serve stale-on-error without logging the staleness +- Webhook secret must be validated (HMAC-SHA256) before processing +- All user-supplied path segments must be sanitized (no path traversal) +- Template rendering must use html/template (auto-escaping), never text/template +- No CGo in production +- Generated code is never edited by hand