mcdoc/ARCHITECTURE.md

mcdoc Architecture

Metacircular Documentation Server — Technical Design Document

---

1. Overview

mcdoc is a public documentation server for the Metacircular platform. It fetches markdown files from Gitea, renders them to HTML, and serves a navigable read-only documentation site. No authentication is required.

mcdoc replaces the need to check out repositories or use a local tool to read platform documentation. It is the web counterpart to browsing docs locally — anyone with the URL can read the current documentation for any Metacircular service.

Goals

• Serve rendered markdown documentation for all Metacircular services
• Public access, no authentication
• Refresh automatically when documentation changes (Gitea webhooks)
• Fit the platform conventions (Go, htmx, MCP-deployed, mc-proxy-routed)

Non-Goals

• Editing or authoring (docs are authored in git)
• Versioned documentation (tagged snapshots, historical diffs)
• Serving non-markdown content (images, PDFs)
• Search (v1 — may be added later)

---

2. Architecture

                          ┌────────────────────────┐
                          │   Gitea (deimos)       │
                          │   git.wntrmute.dev     │
                          │                        │
                          │   mc/mcr               │
                          │   mc/mcp               │
                          │   mc/metacrypt          │
                          │   ...                   │
                          └──────────┬─────────────┘
                                     │
                          fetch (boot + webhook)
                                     │
                                     ▼
  Reader ──── HTTPS ────► mc-proxy ────► mcdoc (rift)
              :443        (L7)          ┌──────────────────┐
              docs.                     │                  │
              metacircular.             │  Content cache   │
              net                       │  (in-memory)     │
                                        │                  │
                                        │  Rendered HTML   │
                                        │  per repo/file   │
                                        │                  │
                                        └──────────────────┘

mcdoc runs on rift as a single container. mc-proxy terminates TLS on docs.metacircular.net and forwards to mcdoc's HTTP listener.

Internally, mcdoc is also reachable at mcdoc.svc.mcp.metacircular.net for Gitea webhook delivery and internal access.

---

3. Content Model

Source

mcdoc fetches markdown files from public repositories in the mc organization on Gitea (git.wntrmute.dev). The repos are public, so no authentication token is needed.

Discovery

On boot, mcdoc fetches the list of repositories in the mc org via the Gitea API, then for each repo:

1. Fetch the file tree (recursive) from the default branch.
2. Filter to *.md files.
3. Exclude files matching configurable patterns (default: vendor/, .claude/, node_modules/).
4. Fetch each markdown file's raw content.
5. Render to HTML via goldmark (GitHub Flavored Markdown).
6. Store the rendered HTML in the in-memory cache, keyed by repo/filepath.

Navigation Structure

The site is organized by repository, then by file path within the repo:

/                           → index: list of repos with descriptions
/mcr/                       → repo index: list of docs in mcr
/mcr/ARCHITECTURE           → rendered ARCHITECTURE.md
/mcr/RUNBOOK                → rendered RUNBOOK.md
/mcp/ARCHITECTURE           → rendered mcp ARCHITECTURE.md
/mcp/docs/bootstrap         → rendered mcp docs/bootstrap.md

The .md extension is stripped from URLs. Directory structure within a repo is preserved.

Ordering

The repo index page lists documents in a fixed priority order for well-known filenames, followed by alphabetical:

1. README
2. ARCHITECTURE
3. RUNBOOK
4. CLAUDE
5. (all others, alphabetical)

The top-level index lists repos alphabetically with the first line of each repo's Gitea description as a subtitle.

---

4. Caching and Refresh

Boot Fetch

On startup, mcdoc fetches all content from Gitea. This is the cold-start path. With ~8 repos and ~50 markdown files total, the full fetch takes 2-3 seconds using concurrent requests (bounded to avoid overwhelming Gitea). Fetches are parallelized per-repo.

mcdoc serves requests only after the initial fetch completes. During boot, it returns HTTP 503 with a "loading" page.

Webhook Refresh

Gitea sends a push webhook to mcdoc.svc.mcp.metacircular.net/webhook on every push to a configured repo's default branch. mcdoc re-fetches that repo's file tree and content, re-renders, and atomically swaps the cache entries for that repo. In-flight requests to other repos are unaffected.

The webhook endpoint validates the Gitea webhook secret (shared secret, HMAC-SHA256 signature in X-Gitea-Signature header).

Fallback Poll

As a safety net, mcdoc polls Gitea for changes every 15 minutes. This catches missed webhooks (network blips, Gitea restarts). The poll checks each repo's latest commit SHA against the cached version and only re-fetches repos that have changed.

Resilience

• Gitea unreachable at boot: mcdoc starts, serves a "docs unavailable, retrying" page, and retries the fetch every 30 seconds until it succeeds.
• Gitea unreachable after boot: stale cache continues serving. Readers see the last-known-good content. Poll/webhook failures are logged but do not affect availability.
• Single file fetch failure: skip the file, log a warning, serve the rest of the repo's docs. Retry on next poll cycle.

---

5. Rendering

Markdown

goldmark with the following extensions:

• GitHub Flavored Markdown (tables, strikethrough, autolinks, task lists)
• Syntax highlighting for fenced code blocks (chroma)
• Heading anchors (linkable #section-name fragments)
• Table of contents generation (extracted from headings, rendered as a sidebar or top-of-page nav)

HTML Output

Each markdown file is rendered into a page template with:

• Header: site title, repo name, breadcrumb navigation
• Sidebar: document list for the current repo (persistent nav)
• Content: rendered markdown
• Footer: last-updated timestamp (from Gitea commit metadata)

The page template uses htmx for navigation — clicking a doc link swaps the content pane without a full page reload, keeping the sidebar state.

Styling

Clean, readable typography optimized for long-form technical documents. The design should prioritize readability:

• Serif or readable sans-serif body text
• Generous line height and margins
• Constrained content width (~70ch)
• Syntax-highlighted code blocks with a muted theme
• Responsive layout (readable on mobile)

---

6. Configuration

TOML configuration at /srv/mcdoc/mcdoc.toml:

[server]
listen_addr = ":8080"

[gitea]
url             = "https://git.wntrmute.dev"
org             = "mc"
webhook_secret  = "..."
poll_interval   = "15m"
fetch_timeout   = "30s"
max_concurrency = 4

[gitea.exclude_paths]
patterns = ["vendor/", ".claude/", "node_modules/", ".junie/"]

[gitea.exclude_repos]
names = []

[log]
level = "info"

Environment variable overrides follow platform convention: MCDOC_* (e.g., MCDOC_GITEA_WEBHOOK_SECRET).

---

7. Deployment

Container

Single binary, single container. Multi-stage Docker build per platform convention (golang:alpine builder, alpine runtime).

mcdoc listens on a single HTTP port. mc-proxy handles TLS termination and routes docs.metacircular.net to mcdoc's listener.

MCP Service Definition

name = "mcdoc"
node = "rift"
version = "v0.1.0"

[build.images]
mcdoc = "Dockerfile"

[[components]]
name = "mcdoc"

[[components.routes]]
port = 443
mode = "l7"
hostname = "docs.metacircular.net"

Port assignment is pending MCP support for automatic port allocation and mc-proxy route registration (see PLATFORM_EVOLUTION.md). Until then, a manually assigned port will be used.

mc-proxy Routes

# On :443 listener
[[listeners.routes]]
hostname    = "docs.metacircular.net"
backend     = "127.0.0.1:<port>"
mode        = "l7"
tls_cert    = "/srv/mc-proxy/certs/docs.pem"
tls_key     = "/srv/mc-proxy/certs/docs.key"
backend_tls = false

[[listeners.routes]]
hostname    = "mcdoc.svc.mcp.metacircular.net"
backend     = "127.0.0.1:<port>"
mode        = "l7"
tls_cert    = "/srv/mc-proxy/certs/mcdoc-svc.pem"
tls_key     = "/srv/mc-proxy/certs/mcdoc-svc.key"
backend_tls = false

Note: backend_tls = false — mcdoc is plain HTTP behind mc-proxy. This is safe because mc-proxy and mcdoc are on the same host. TLS is terminated at mc-proxy.

DNS

Record	Value
docs.metacircular.net	Public DNS → rift's public IP
mcdoc.svc.mcp.metacircular.net	Internal DNS (MCNS) → rift

---

8. Package Structure

cmd/mcdoc/           CLI entry point (cobra: server subcommand)
internal/
  config/            TOML config loading and validation
  gitea/             Gitea API client (list repos, fetch trees, fetch files)
  cache/             In-memory content cache (atomic swap per repo)
  render/            goldmark rendering pipeline
  server/            HTTP server, chi routes, htmx handlers
web/
  templates/         Go html/template files (index, repo, doc, error)
  static/            CSS, favicon

---

9. Routes

Method	Path	Description
GET	/	Top-level index (repo list)
GET	/{repo}/	Repo index (doc list)
GET	/{repo}/{path...}	Rendered document
POST	/webhook	Gitea push webhook receiver
GET	/health	Health check (200 if cache is populated, 503 if not)

htmx partial responses: when HX-Request header is present, return only the content fragment (no surrounding layout). This enables client-side navigation without full page reloads.

---

10. Future Work

Item	Description
Search	Full-text search across all docs (bleve or similar)
Cross-linking	Resolve relative markdown links across repos
Mermaid/diagrams	Render mermaid fenced blocks as SVG
Dark mode	Theme toggle (light/dark)
Pinned versions	Serve docs at a specific git tag