Compare commits
4 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 62df7ed6cd | |||
| 051abae390 | |||
| dd5142a48a | |||
| 2c3db6ea25 |
@@ -77,6 +77,9 @@ func runServer(configPath string) error {
|
||||
wsCfg := webserver.Config{
|
||||
ServiceName: cfg.MCIAS.ServiceName,
|
||||
Tags: cfg.MCIAS.Tags,
|
||||
MciasURL: cfg.MCIAS.ServerURL,
|
||||
CACert: cfg.MCIAS.CACert,
|
||||
RedirectURI: cfg.SSO.RedirectURI,
|
||||
}
|
||||
webSrv, err := webserver.New(wsCfg, database, authClient, logger)
|
||||
if err != nil {
|
||||
|
||||
755
docs/packaging-and-deployment.md
Normal file
755
docs/packaging-and-deployment.md
Normal file
@@ -0,0 +1,755 @@
|
||||
# Packaging and Deploying to the Metacircular Platform
|
||||
|
||||
This guide provides everything needed to build, package, and deploy a
|
||||
service to the Metacircular platform. It assumes no prior knowledge of
|
||||
the platform's internals.
|
||||
|
||||
---
|
||||
|
||||
## Platform Overview
|
||||
|
||||
Metacircular is a multi-service infrastructure platform. Services are
|
||||
Go binaries running as containers on Linux nodes, managed by these core
|
||||
components:
|
||||
|
||||
| Component | Role |
|
||||
|-----------|------|
|
||||
| **MCP** (Control Plane) | Deploys, monitors, and manages container lifecycle via rootless Podman |
|
||||
| **MCR** (Container Registry) | OCI container registry at `mcr.svc.mcp.metacircular.net:8443` |
|
||||
| **mc-proxy** (TLS Proxy) | Routes traffic to services via L4 (SNI passthrough) or L7 (TLS termination) |
|
||||
| **MCIAS** (Identity Service) | Central SSO/IAM — all services authenticate through it |
|
||||
| **MCNS** (DNS) | Authoritative DNS for `*.svc.mcp.metacircular.net` |
|
||||
|
||||
The operator workflow is: **build image → push to MCR → write service
|
||||
definition → deploy via MCP**. MCP handles port assignment, route
|
||||
registration, and container lifecycle.
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites
|
||||
|
||||
| Requirement | Details |
|
||||
|-------------|---------|
|
||||
| Go | 1.25+ |
|
||||
| Container engine | Docker or Podman (for building images) |
|
||||
| `mcp` CLI | Installed on the operator workstation |
|
||||
| MCR access | Credentials to push images to `mcr.svc.mcp.metacircular.net:8443` |
|
||||
| MCP agent | Running on the target node (currently `rift`) |
|
||||
| MCIAS account | For `mcp` CLI authentication to the agent |
|
||||
|
||||
---
|
||||
|
||||
## 1. Build the Container Image
|
||||
|
||||
### Dockerfile Pattern
|
||||
|
||||
All services use a two-stage Alpine build. This is the standard
|
||||
template:
|
||||
|
||||
```dockerfile
|
||||
FROM golang:1.25-alpine AS builder
|
||||
|
||||
RUN apk add --no-cache git
|
||||
WORKDIR /build
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
COPY . .
|
||||
|
||||
ARG VERSION=dev
|
||||
RUN CGO_ENABLED=0 go build -trimpath \
|
||||
-ldflags="-s -w -X main.version=${VERSION}" \
|
||||
-o /<binary> ./cmd/<binary>
|
||||
|
||||
FROM alpine:3.21
|
||||
|
||||
RUN apk add --no-cache ca-certificates tzdata
|
||||
COPY --from=builder /<binary> /usr/local/bin/<binary>
|
||||
|
||||
WORKDIR /srv/<service>
|
||||
EXPOSE <ports>
|
||||
|
||||
ENTRYPOINT ["<binary>"]
|
||||
CMD ["server", "--config", "/srv/<service>/<service>.toml"]
|
||||
```
|
||||
|
||||
### Dockerfile Rules
|
||||
|
||||
- **`CGO_ENABLED=0`** — all builds are statically linked. No CGo in
|
||||
production.
|
||||
- **`ca-certificates` and `tzdata`** — required in the runtime image
|
||||
for TLS verification and timezone-aware logging.
|
||||
- **No `USER` directive** — containers run as `--user 0:0` under MCP's
|
||||
rootless Podman. UID 0 inside the container maps to the unprivileged
|
||||
`mcp` host user. A non-root `USER` directive creates a subordinate
|
||||
UID that cannot access host-mounted volumes.
|
||||
- **No `VOLUME` directive** — causes layer unpacking failures under
|
||||
rootless Podman. The host volume mount is declared in the service
|
||||
definition, not the image.
|
||||
- **No `adduser`/`addgroup`** — unnecessary given the rootless Podman
|
||||
model.
|
||||
- **`WORKDIR /srv/<service>`** — so relative paths resolve correctly
|
||||
against the mounted data directory.
|
||||
- **Version injection** — pass the git tag via `--build-arg VERSION=...`
|
||||
so the binary can report its version.
|
||||
- **Stripped binaries** — `-trimpath -ldflags="-s -w"` removes debug
|
||||
symbols and build paths.
|
||||
|
||||
### Split Binaries
|
||||
|
||||
If the service has separate API and web UI binaries, create separate
|
||||
Dockerfiles:
|
||||
|
||||
- `Dockerfile.api` — builds the API/gRPC server
|
||||
- `Dockerfile.web` — builds the web UI server
|
||||
|
||||
Both follow the same template. The web binary communicates with the API
|
||||
server via gRPC (no direct database access).
|
||||
|
||||
### Makefile Target
|
||||
|
||||
Every service includes a `make docker` target:
|
||||
|
||||
```makefile
|
||||
docker:
|
||||
docker build --build-arg VERSION=$(shell git describe --tags --always --dirty) \
|
||||
-t <service> -f Dockerfile.api .
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Write a Service Definition
|
||||
|
||||
Service definitions are TOML files that tell MCP what to deploy. They
|
||||
live at `~/.config/mcp/services/<service>.toml` on the operator
|
||||
workstation.
|
||||
|
||||
### Minimal Example (Single Component, L7)
|
||||
|
||||
```toml
|
||||
name = "myservice"
|
||||
node = "rift"
|
||||
|
||||
[build.images]
|
||||
myservice = "Dockerfile"
|
||||
|
||||
[[components]]
|
||||
name = "web"
|
||||
image = "mcr.svc.mcp.metacircular.net:8443/myservice:v1.0.0"
|
||||
|
||||
[[components.routes]]
|
||||
port = 8443
|
||||
mode = "l7"
|
||||
```
|
||||
|
||||
### API Service Example (L4, Multiple Routes)
|
||||
|
||||
```toml
|
||||
name = "myservice"
|
||||
node = "rift"
|
||||
|
||||
[build.images]
|
||||
myservice = "Dockerfile"
|
||||
|
||||
[[components]]
|
||||
name = "api"
|
||||
image = "mcr.svc.mcp.metacircular.net:8443/myservice:v1.0.0"
|
||||
volumes = ["/srv/myservice:/srv/myservice"]
|
||||
cmd = ["server", "--config", "/srv/myservice/myservice.toml"]
|
||||
|
||||
[[components.routes]]
|
||||
name = "rest"
|
||||
port = 8443
|
||||
mode = "l4"
|
||||
|
||||
[[components.routes]]
|
||||
name = "grpc"
|
||||
port = 9443
|
||||
mode = "l4"
|
||||
```
|
||||
|
||||
### Full Example (API + Web)
|
||||
|
||||
```toml
|
||||
name = "myservice"
|
||||
node = "rift"
|
||||
|
||||
[build.images]
|
||||
myservice = "Dockerfile.api"
|
||||
myservice-web = "Dockerfile.web"
|
||||
|
||||
[[components]]
|
||||
name = "api"
|
||||
image = "mcr.svc.mcp.metacircular.net:8443/myservice:v1.0.0"
|
||||
volumes = ["/srv/myservice:/srv/myservice"]
|
||||
cmd = ["server", "--config", "/srv/myservice/myservice.toml"]
|
||||
|
||||
[[components.routes]]
|
||||
name = "rest"
|
||||
port = 8443
|
||||
mode = "l4"
|
||||
|
||||
[[components.routes]]
|
||||
name = "grpc"
|
||||
port = 9443
|
||||
mode = "l4"
|
||||
|
||||
[[components]]
|
||||
name = "web"
|
||||
image = "mcr.svc.mcp.metacircular.net:8443/myservice-web:v1.0.0"
|
||||
volumes = ["/srv/myservice:/srv/myservice"]
|
||||
cmd = ["server", "--config", "/srv/myservice/myservice.toml"]
|
||||
|
||||
[[components.routes]]
|
||||
port = 443
|
||||
mode = "l7"
|
||||
```
|
||||
|
||||
### Conventions
|
||||
|
||||
A few fields are derived by the agent at deploy time:
|
||||
|
||||
| Field | Default | Override when... |
|
||||
|-------|---------|------------------|
|
||||
| Source path | `<service>` relative to workspace root | Directory name differs from service name (use `path`) |
|
||||
| Hostname | `<service>.svc.mcp.metacircular.net` | Service needs a public hostname (use route `hostname`) |
|
||||
|
||||
All other fields must be explicit in the service definition.
|
||||
|
||||
### Service Definition Reference
|
||||
|
||||
**Top-level fields:**
|
||||
|
||||
| Field | Required | Purpose |
|
||||
|-------|----------|---------|
|
||||
| `name` | Yes | Service name (matches project name) |
|
||||
| `node` | Yes | Target node to deploy to |
|
||||
| `active` | No | Whether MCP keeps this running (default: `true`) |
|
||||
| `path` | No | Source directory relative to workspace (default: `name`) |
|
||||
|
||||
**Build fields:**
|
||||
|
||||
| Field | Purpose |
|
||||
|-------|---------|
|
||||
| `build.images.<name>` | Maps build image name to Dockerfile path. The `<name>` must match the repository name in a component's `image` field (the part after the last `/`, before the `:` tag). |
|
||||
|
||||
**Component fields:**
|
||||
|
||||
| Field | Required | Purpose |
|
||||
|-------|----------|---------|
|
||||
| `name` | Yes | Component name (e.g. `api`, `web`) |
|
||||
| `image` | Yes | Full image reference (e.g. `mcr.svc.mcp.metacircular.net:8443/myservice:v1.0.0`) |
|
||||
| `volumes` | No | Volume mounts (list of `host:container` strings) |
|
||||
| `cmd` | No | Command override (list of strings) |
|
||||
| `env` | No | Extra environment variables (list of `KEY=VALUE` strings) |
|
||||
| `network` | No | Container network (default: none) |
|
||||
| `user` | No | Container user (e.g. `0:0`) |
|
||||
| `restart` | No | Restart policy (e.g. `unless-stopped`) |
|
||||
|
||||
**Route fields (under `[[components.routes]]`):**
|
||||
|
||||
| Field | Purpose |
|
||||
|-------|---------|
|
||||
| `name` | Route name — determines `$PORT_<NAME>` env var |
|
||||
| `port` | External port on mc-proxy (e.g. `8443`, `9443`, `443`) |
|
||||
| `mode` | `l4` (TLS passthrough) or `l7` (TLS termination by mc-proxy) |
|
||||
| `hostname` | Public hostname override |
|
||||
|
||||
### Routing Modes
|
||||
|
||||
| Mode | TLS handled by | Use when... |
|
||||
|------|----------------|-------------|
|
||||
| `l4` | The service itself | Service manages its own TLS (API servers, gRPC) |
|
||||
| `l7` | mc-proxy | mc-proxy terminates TLS and proxies HTTP to the service (web UIs) |
|
||||
|
||||
### Version Pinning
|
||||
|
||||
Component `image` fields **must** pin an explicit semver tag (e.g.
|
||||
`mcr.svc.mcp.metacircular.net:8443/myservice:v1.1.0`). Never use
|
||||
`:latest`. This ensures deployments are reproducible and `mcp status`
|
||||
shows the actual running version. The version is extracted from the
|
||||
image tag.
|
||||
|
||||
---
|
||||
|
||||
## 3. Build, Push, and Deploy
|
||||
|
||||
### Tag the Release
|
||||
|
||||
```bash
|
||||
git tag -a v1.0.0 -m "v1.0.0"
|
||||
git push origin v1.0.0
|
||||
```
|
||||
|
||||
### Build and Push Images
|
||||
|
||||
```bash
|
||||
mcp build <service>
|
||||
```
|
||||
|
||||
This reads the `[build.images]` section of the service definition,
|
||||
builds each Dockerfile, tags the images with the version from the
|
||||
definition, and pushes them to MCR.
|
||||
|
||||
The workspace root is configured in `~/.config/mcp/mcp.toml`:
|
||||
|
||||
```toml
|
||||
[build]
|
||||
workspace = "~/src/metacircular"
|
||||
```
|
||||
|
||||
Each service's source is at `<workspace>/<path>` (where `path` defaults
|
||||
to the service name).
|
||||
|
||||
### Sync and Deploy
|
||||
|
||||
```bash
|
||||
# Push all service definitions to agents, auto-build missing images
|
||||
mcp sync
|
||||
|
||||
# Deploy (or redeploy) a specific service
|
||||
mcp deploy <service>
|
||||
```
|
||||
|
||||
`mcp sync` checks whether each component's image tag exists in MCR. If
|
||||
missing and the source tree is available, it builds and pushes
|
||||
automatically.
|
||||
|
||||
`mcp deploy` pulls the image on the target node and creates or
|
||||
recreates the containers.
|
||||
|
||||
### What Happens During Deploy
|
||||
|
||||
1. Agent assigns a free host port (10000–60000) for each declared route.
|
||||
2. Agent starts containers with `$PORT` / `$PORT_<NAME>` environment
|
||||
variables set to the assigned ports.
|
||||
3. Agent registers routes with mc-proxy (hostname → `127.0.0.1:<port>`,
|
||||
mode, TLS cert paths).
|
||||
4. Agent records the full state in its SQLite registry.
|
||||
|
||||
On stop (`mcp stop <service>`), the agent reverses the process: removes
|
||||
mc-proxy routes, then stops containers.
|
||||
|
||||
---
|
||||
|
||||
## 4. Data Directory Convention
|
||||
|
||||
All runtime data lives in `/srv/<service>/` on the host. This directory
|
||||
is bind-mounted into the container.
|
||||
|
||||
```
|
||||
/srv/<service>/
|
||||
├── <service>.toml # Configuration file
|
||||
├── <service>.db # SQLite database (created on first run)
|
||||
├── certs/ # TLS certificates
|
||||
│ ├── cert.pem
|
||||
│ └── key.pem
|
||||
└── backups/ # Database snapshots
|
||||
```
|
||||
|
||||
This directory must exist on the target node before the first deploy,
|
||||
owned by the `mcp` user (which runs rootless Podman). Create it with:
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /srv/<service>/certs
|
||||
sudo chown -R mcp:mcp /srv/<service>
|
||||
```
|
||||
|
||||
Place the service's TOML configuration and TLS certificates here before
|
||||
deploying.
|
||||
|
||||
---
|
||||
|
||||
## 5. Configuration
|
||||
|
||||
Services use TOML configuration with environment variable overrides.
|
||||
|
||||
### Standard Config Sections
|
||||
|
||||
```toml
|
||||
[server]
|
||||
listen_addr = ":8443"
|
||||
grpc_addr = ":9443"
|
||||
tls_cert = "/srv/<service>/certs/cert.pem"
|
||||
tls_key = "/srv/<service>/certs/key.pem"
|
||||
|
||||
[database]
|
||||
path = "/srv/<service>/<service>.db"
|
||||
|
||||
[mcias]
|
||||
server_url = "https://mcias.metacircular.net:8443"
|
||||
ca_cert = ""
|
||||
service_name = "<service>"
|
||||
tags = []
|
||||
|
||||
[log]
|
||||
level = "info"
|
||||
```
|
||||
|
||||
For services with SSO-enabled web UIs, add:
|
||||
|
||||
```toml
|
||||
[sso]
|
||||
redirect_uri = "https://<service>.svc.mcp.metacircular.net/sso/callback"
|
||||
```
|
||||
|
||||
For services with a separate web UI binary, add:
|
||||
|
||||
```toml
|
||||
[web]
|
||||
listen_addr = "127.0.0.1:8080"
|
||||
vault_grpc = "127.0.0.1:9443"
|
||||
vault_ca_cert = ""
|
||||
```
|
||||
|
||||
### $PORT Convention
|
||||
|
||||
When deployed via MCP, the agent assigns host ports and passes them as
|
||||
environment variables. **Applications should not hardcode listen
|
||||
addresses** — they will be overridden at deploy time.
|
||||
|
||||
| Env var | When set |
|
||||
|---------|----------|
|
||||
| `$PORT` | Component has a single unnamed route |
|
||||
| `$PORT_<NAME>` | Component has named routes |
|
||||
|
||||
Route names are uppercased: `name = "rest"` → `$PORT_REST`,
|
||||
`name = "grpc"` → `$PORT_GRPC`.
|
||||
|
||||
**Container listen address:** Services must bind to `0.0.0.0:$PORT`
|
||||
(or `:$PORT`), not `localhost:$PORT`. Podman port-forwards go through
|
||||
the container's network namespace — binding to `localhost` inside the
|
||||
container makes the port unreachable from outside.
|
||||
|
||||
Services built with **mcdsl v1.1.0+** handle this automatically —
|
||||
`config.Load` checks `$PORT` → overrides `Server.ListenAddr`, and
|
||||
`$PORT_GRPC` → overrides `Server.GRPCAddr`. These take precedence over
|
||||
TOML values.
|
||||
|
||||
Services not using mcdsl must check these environment variables in their
|
||||
own config loading.
|
||||
|
||||
### Environment Variable Overrides
|
||||
|
||||
Beyond `$PORT`, services support `$SERVICENAME_SECTION_KEY` overrides.
|
||||
For example, `$MCR_SERVER_LISTEN_ADDR=:9999` overrides
|
||||
`[server] listen_addr` in MCR's config. `$PORT` takes precedence over
|
||||
these.
|
||||
|
||||
---
|
||||
|
||||
## 6. Authentication (MCIAS Integration)
|
||||
|
||||
Every service delegates authentication to MCIAS. No service maintains
|
||||
its own user database. Services support two login modes: **SSO
|
||||
redirect** (recommended for web UIs) and **direct credentials**
|
||||
(fallback / API clients).
|
||||
|
||||
### SSO Login (Web UIs)
|
||||
|
||||
SSO is the preferred login method for web UIs. The flow is an OAuth
|
||||
2.0-style authorization code exchange:
|
||||
|
||||
1. User visits the service and is redirected to `/login`.
|
||||
2. Login page shows a "Sign in with MCIAS" button.
|
||||
3. Click redirects to MCIAS (`/sso/authorize`), which authenticates the
|
||||
user.
|
||||
4. MCIAS redirects back to the service's `/sso/callback` with an
|
||||
authorization code.
|
||||
5. The service exchanges the code for a JWT via a server-to-server call
|
||||
to MCIAS `POST /v1/sso/token`.
|
||||
6. The JWT is stored in a session cookie.
|
||||
|
||||
SSO is enabled by adding an `[sso]` section to the service config and
|
||||
registering the service as an SSO client in MCIAS.
|
||||
|
||||
**Service config:**
|
||||
|
||||
```toml
|
||||
[sso]
|
||||
redirect_uri = "https://<service>.svc.mcp.metacircular.net/sso/callback"
|
||||
```
|
||||
|
||||
**MCIAS config** (add to the `[[sso_clients]]` list):
|
||||
|
||||
```toml
|
||||
[[sso_clients]]
|
||||
client_id = "<service>"
|
||||
redirect_uri = "https://<service>.svc.mcp.metacircular.net/sso/callback"
|
||||
service_name = "<service>"
|
||||
```
|
||||
|
||||
The `redirect_uri` must match exactly between the service config and
|
||||
the MCIAS client registration.
|
||||
|
||||
When `[sso].redirect_uri` is empty or absent, the service falls back to
|
||||
the direct credentials form.
|
||||
|
||||
**Implementation:** Services use `mcdsl/sso` (v1.7.0+) which handles
|
||||
state management, CSRF-safe cookies, and the code exchange. The web
|
||||
server registers three routes:
|
||||
|
||||
| Route | Purpose |
|
||||
|-------|---------|
|
||||
| `GET /login` | Renders landing page with "Sign in with MCIAS" button |
|
||||
| `GET /sso/redirect` | Sets state cookies, redirects to MCIAS |
|
||||
| `GET /sso/callback` | Validates state, exchanges code for JWT, sets session |
|
||||
|
||||
### Direct Credentials (API / Fallback)
|
||||
|
||||
1. Client sends credentials to the service's `POST /v1/auth/login`.
|
||||
2. Service forwards them to MCIAS via `mcdsl/auth.Authenticator.Login()`.
|
||||
3. MCIAS validates and returns a bearer token.
|
||||
4. Subsequent requests include `Authorization: Bearer <token>`.
|
||||
5. Service validates tokens via `ValidateToken()`, cached for 30s
|
||||
(keyed by SHA-256 of the token).
|
||||
|
||||
Web UIs use this mode when SSO is not configured, presenting a
|
||||
username/password/TOTP form instead of the SSO button.
|
||||
|
||||
### Roles
|
||||
|
||||
| Role | Access |
|
||||
|------|--------|
|
||||
| `admin` | Full access, policy bypass |
|
||||
| `user` | Access governed by policy rules, default deny |
|
||||
| `guest` | Service-dependent restrictions, default deny |
|
||||
|
||||
Admin detection comes solely from the MCIAS `admin` role. Services
|
||||
never promote users locally.
|
||||
|
||||
---
|
||||
|
||||
## 7. Networking
|
||||
|
||||
### Hostnames
|
||||
|
||||
Every service gets `<service>.svc.mcp.metacircular.net` automatically.
|
||||
Public-facing services can declare additional hostnames:
|
||||
|
||||
```toml
|
||||
[[components.routes]]
|
||||
port = 443
|
||||
mode = "l7"
|
||||
hostname = "docs.metacircular.net"
|
||||
```
|
||||
|
||||
### TLS
|
||||
|
||||
- **Minimum TLS 1.3.** No exceptions.
|
||||
- L4 services manage their own TLS — certificates go in
|
||||
`/srv/<service>/certs/`.
|
||||
- L7 services have TLS terminated by mc-proxy — certs are stored at
|
||||
`/srv/mc-proxy/certs/<service>.pem`.
|
||||
- Certificate and key paths are required config — the service refuses
|
||||
to start without them.
|
||||
|
||||
### Container Networking
|
||||
|
||||
Containers join the `mcpnet` Podman network by default. Services
|
||||
communicate with each other over this network or via loopback (when
|
||||
co-located on the same node).
|
||||
|
||||
---
|
||||
|
||||
## 8. Command Reference
|
||||
|
||||
| Command | Purpose |
|
||||
|---------|---------|
|
||||
| `mcp build <service>` | Build and push images to MCR |
|
||||
| `mcp sync` | Push all service definitions to agents; auto-build missing images |
|
||||
| `mcp deploy <service>` | Pull image, (re)create containers, register routes |
|
||||
| `mcp undeploy <service>` | Full teardown: remove routes, DNS, certs, and containers |
|
||||
| `mcp stop <service>` | Remove routes, stop containers |
|
||||
| `mcp start <service>` | Start previously stopped containers |
|
||||
| `mcp restart <service>` | Restart containers in place |
|
||||
| `mcp ps` | List all managed containers and status |
|
||||
| `mcp status [service]` | Detailed status for a specific service |
|
||||
| `mcp logs <service>` | Stream container logs |
|
||||
| `mcp edit <service>` | Edit service definition |
|
||||
|
||||
---
|
||||
|
||||
## 9. Complete Walkthrough
|
||||
|
||||
Deploying a new service called `myservice` from scratch:
|
||||
|
||||
```bash
|
||||
# 1. Prepare the target node
|
||||
ssh rift
|
||||
sudo mkdir -p /srv/myservice/certs
|
||||
sudo chown -R mcp:mcp /srv/myservice
|
||||
# Place myservice.toml and TLS certs in /srv/myservice/
|
||||
exit
|
||||
|
||||
# 2. Tag the release
|
||||
cd ~/src/metacircular/myservice
|
||||
git tag -a v1.0.0 -m "v1.0.0"
|
||||
git push origin v1.0.0
|
||||
|
||||
# 3. Write the service definition
|
||||
cat > ~/.config/mcp/services/myservice.toml << 'EOF'
|
||||
name = "myservice"
|
||||
node = "rift"
|
||||
|
||||
[build.images]
|
||||
myservice = "Dockerfile.api"
|
||||
|
||||
[[components]]
|
||||
name = "api"
|
||||
image = "mcr.svc.mcp.metacircular.net:8443/myservice:v1.0.0"
|
||||
volumes = ["/srv/myservice:/srv/myservice"]
|
||||
|
||||
[[components.routes]]
|
||||
name = "rest"
|
||||
port = 8443
|
||||
mode = "l4"
|
||||
|
||||
[[components.routes]]
|
||||
name = "grpc"
|
||||
port = 9443
|
||||
mode = "l4"
|
||||
EOF
|
||||
|
||||
# 4. Build and push the image
|
||||
mcp build myservice
|
||||
|
||||
# 5. Deploy
|
||||
mcp deploy myservice
|
||||
|
||||
# 6. Verify
|
||||
mcp status myservice
|
||||
mcp ps
|
||||
```
|
||||
|
||||
The service is now running, with mc-proxy routing
|
||||
`myservice.svc.mcp.metacircular.net` traffic to the agent-assigned
|
||||
ports.
|
||||
|
||||
---
|
||||
|
||||
## Appendix: Repository Layout
|
||||
|
||||
Services follow a standard directory structure:
|
||||
|
||||
```
|
||||
.
|
||||
├── cmd/<service>/ CLI entry point (server, subcommands)
|
||||
├── cmd/<service>-web/ Web UI entry point (if separate)
|
||||
├── internal/ All service logic (not importable externally)
|
||||
│ ├── auth/ MCIAS integration
|
||||
│ ├── config/ TOML config loading
|
||||
│ ├── db/ Database setup, migrations
|
||||
│ ├── server/ REST API server
|
||||
│ ├── grpcserver/ gRPC server
|
||||
│ └── webserver/ Web UI server (if applicable)
|
||||
├── proto/<service>/v1/ Protobuf definitions
|
||||
├── gen/<service>/v1/ Generated gRPC code
|
||||
├── web/ Templates and static assets (embedded)
|
||||
├── deploy/
|
||||
│ ├── <service>-rift.toml Reference MCP service definition
|
||||
│ ├── docker/ Docker Compose files
|
||||
│ ├── examples/ Example config files
|
||||
│ └── systemd/ systemd units
|
||||
├── Dockerfile.api API server container
|
||||
├── Dockerfile.web Web UI container (if applicable)
|
||||
├── Makefile Standard build targets
|
||||
└── <service>.toml.example Example configuration
|
||||
```
|
||||
|
||||
### Standard Makefile Targets
|
||||
|
||||
| Target | Purpose |
|
||||
|--------|---------|
|
||||
| `make all` | vet → lint → test → build (the CI pipeline) |
|
||||
| `make build` | `go build ./...` |
|
||||
| `make test` | `go test ./...` |
|
||||
| `make vet` | `go vet ./...` |
|
||||
| `make lint` | `golangci-lint run ./...` |
|
||||
| `make docker` | Build the container image |
|
||||
| `make proto` | Regenerate gRPC code from .proto files |
|
||||
| `make devserver` | Build and run locally against `srv/` config |
|
||||
|
||||
---
|
||||
|
||||
## 10. Agent Management
|
||||
|
||||
MCP manages a fleet of nodes with heterogeneous operating systems and
|
||||
architectures. The agent binary lives at `/srv/mcp/mcp-agent` on every
|
||||
node — this is a mutable path that MCP controls, regardless of whether
|
||||
the node runs NixOS or Debian.
|
||||
|
||||
### Node Configuration
|
||||
|
||||
Each node in `~/.config/mcp/mcp.toml` includes SSH and architecture
|
||||
info for agent management:
|
||||
|
||||
```toml
|
||||
[[nodes]]
|
||||
name = "rift"
|
||||
address = "100.95.252.120:9444"
|
||||
ssh = "rift"
|
||||
arch = "amd64"
|
||||
|
||||
[[nodes]]
|
||||
name = "hyperborea"
|
||||
address = "100.x.x.x:9444"
|
||||
ssh = "hyperborea"
|
||||
arch = "arm64"
|
||||
```
|
||||
|
||||
### Upgrading Agents
|
||||
|
||||
After tagging a new MCP release:
|
||||
|
||||
```bash
|
||||
# Upgrade all nodes (recommended — prevents version skew)
|
||||
mcp agent upgrade
|
||||
|
||||
# Upgrade a single node
|
||||
mcp agent upgrade rift
|
||||
|
||||
# Check versions across the fleet
|
||||
mcp agent status
|
||||
```
|
||||
|
||||
`mcp agent upgrade` cross-compiles the agent binary for each target
|
||||
architecture, SSHs to each node, atomically replaces the binary, and
|
||||
restarts the systemd service. All nodes should be upgraded together
|
||||
because new CLI versions often depend on new agent RPCs.
|
||||
|
||||
### Provisioning New Nodes
|
||||
|
||||
One-time setup for a new Debian node:
|
||||
|
||||
```bash
|
||||
# 1. Provision the node (creates user, dirs, systemd unit, installs binary)
|
||||
mcp node provision <name>
|
||||
|
||||
# 2. Register the node
|
||||
mcp node add <name> <address>
|
||||
|
||||
# 3. Deploy services
|
||||
mcp deploy <service>
|
||||
```
|
||||
|
||||
For NixOS nodes, provisioning is handled by the NixOS configuration.
|
||||
The NixOS config creates the `mcp` user, systemd unit, and directories.
|
||||
The `ExecStart` path points to `/srv/mcp/mcp-agent` so that `mcp agent
|
||||
upgrade` works the same as on Debian nodes.
|
||||
|
||||
---
|
||||
|
||||
## Appendix: Currently Deployed Services
|
||||
|
||||
For reference, these services are operational on the platform:
|
||||
|
||||
| Service | Version | Node | Purpose |
|
||||
|---------|---------|------|---------|
|
||||
| MCIAS | v1.9.0 | (separate) | Identity and access |
|
||||
| Metacrypt | v1.4.1 | rift | Cryptographic service, PKI/CA |
|
||||
| MC-Proxy | v1.2.1 | rift | TLS proxy and router |
|
||||
| MCR | v1.2.1 | rift | Container registry |
|
||||
| MCNS | v1.1.1 | rift | Authoritative DNS |
|
||||
| MCDoc | v0.1.0 | rift | Documentation server |
|
||||
| MCQ | v0.4.0 | rift | Document review queue |
|
||||
| MCP | v0.7.6 | rift | Control plane agent |
|
||||
2
go.mod
2
go.mod
@@ -3,7 +3,7 @@ module git.wntrmute.dev/mc/mcq
|
||||
go 1.25.7
|
||||
|
||||
require (
|
||||
git.wntrmute.dev/mc/mcdsl v1.2.0
|
||||
git.wntrmute.dev/mc/mcdsl v1.7.0
|
||||
github.com/alecthomas/chroma/v2 v2.18.0
|
||||
github.com/go-chi/chi/v5 v5.2.5
|
||||
github.com/mark3labs/mcp-go v0.46.0
|
||||
|
||||
4
go.sum
4
go.sum
@@ -1,5 +1,5 @@
|
||||
git.wntrmute.dev/mc/mcdsl v1.2.0 h1:41hep7/PNZJfN0SN/nM+rQpyF1GSZcvNNjyVG81DI7U=
|
||||
git.wntrmute.dev/mc/mcdsl v1.2.0/go.mod h1:lXYrAt74ZUix6rx9oVN8d2zH1YJoyp4uxPVKQ+SSxuM=
|
||||
git.wntrmute.dev/mc/mcdsl v1.7.0 h1:dAh2SGdzjhz0H66i3KAMDm1eRYYgMaxqQ0Pj5NzF7fc=
|
||||
git.wntrmute.dev/mc/mcdsl v1.7.0/go.mod h1:MhYahIu7Sg53lE2zpQ20nlrsoNRjQzOJBAlCmom2wJc=
|
||||
github.com/alecthomas/assert/v2 v2.11.0 h1:2Q9r3ki8+JYXvGsDyBXwH3LcJ+WK5D0gc5E8vS6K3D0=
|
||||
github.com/alecthomas/assert/v2 v2.11.0/go.mod h1:Bze95FyfUr7x34QZrjL+XP+0qgp/zg8yS+TtBj1WA3k=
|
||||
github.com/alecthomas/chroma/v2 v2.2.0/go.mod h1:vf4zrexSH54oEjJ7EdB65tGNHmH3pGZmVkgTP5RHvAs=
|
||||
|
||||
@@ -14,10 +14,18 @@ import (
|
||||
|
||||
// Config is the MCQ configuration.
|
||||
type Config struct {
|
||||
Server ServerConfig `toml:"server"`
|
||||
Database DatabaseConfig `toml:"database"`
|
||||
Server ServerConfig `toml:"server"`
|
||||
Database DatabaseConfig `toml:"database"`
|
||||
MCIAS mcdslauth.Config `toml:"mcias"`
|
||||
Log LogConfig `toml:"log"`
|
||||
SSO SSOConfig `toml:"sso"`
|
||||
Log LogConfig `toml:"log"`
|
||||
}
|
||||
|
||||
// SSOConfig holds SSO redirect settings for the web UI.
|
||||
type SSOConfig struct {
|
||||
// RedirectURI is the callback URL that MCIAS redirects to after login.
|
||||
// Must exactly match the redirect_uri registered in MCIAS config.
|
||||
RedirectURI string `toml:"redirect_uri"`
|
||||
}
|
||||
|
||||
// ServerConfig holds HTTP/gRPC server settings. TLS fields are optional;
|
||||
|
||||
@@ -11,6 +11,7 @@ import (
|
||||
|
||||
"git.wntrmute.dev/mc/mcdsl/auth"
|
||||
"git.wntrmute.dev/mc/mcdsl/csrf"
|
||||
mcdsso "git.wntrmute.dev/mc/mcdsl/sso"
|
||||
"git.wntrmute.dev/mc/mcdsl/web"
|
||||
|
||||
mcqweb "git.wntrmute.dev/mc/mcq/web"
|
||||
@@ -25,16 +26,22 @@ const cookieName = "mcq_session"
|
||||
type Config struct {
|
||||
ServiceName string
|
||||
Tags []string
|
||||
// SSO fields — when RedirectURI is non-empty, the web UI uses SSO instead
|
||||
// of the direct username/password login form.
|
||||
MciasURL string
|
||||
CACert string
|
||||
RedirectURI string
|
||||
}
|
||||
|
||||
// Server is the MCQ web UI server.
|
||||
type Server struct {
|
||||
db *db.DB
|
||||
auth *auth.Authenticator
|
||||
csrf *csrf.Protect
|
||||
render *render.Renderer
|
||||
logger *slog.Logger
|
||||
config Config
|
||||
db *db.DB
|
||||
auth *auth.Authenticator
|
||||
csrf *csrf.Protect
|
||||
ssoClient *mcdsso.Client
|
||||
render *render.Renderer
|
||||
logger *slog.Logger
|
||||
config Config
|
||||
}
|
||||
|
||||
// New creates a web UI server.
|
||||
@@ -45,20 +52,43 @@ func New(cfg Config, database *db.DB, authenticator *auth.Authenticator, logger
|
||||
}
|
||||
csrfProtect := csrf.New(csrfSecret, "_csrf", "csrf_token")
|
||||
|
||||
return &Server{
|
||||
s := &Server{
|
||||
db: database,
|
||||
auth: authenticator,
|
||||
csrf: csrfProtect,
|
||||
render: render.New(),
|
||||
logger: logger,
|
||||
config: cfg,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// Create SSO client if the service has an SSO redirect_uri configured.
|
||||
if cfg.RedirectURI != "" {
|
||||
ssoClient, err := mcdsso.New(mcdsso.Config{
|
||||
MciasURL: cfg.MciasURL,
|
||||
ClientID: "mcq",
|
||||
RedirectURI: cfg.RedirectURI,
|
||||
CACert: cfg.CACert,
|
||||
})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("create SSO client: %w", err)
|
||||
}
|
||||
s.ssoClient = ssoClient
|
||||
logger.Info("SSO enabled: redirecting to MCIAS for login", "mcias_url", cfg.MciasURL)
|
||||
}
|
||||
|
||||
return s, nil
|
||||
}
|
||||
|
||||
// RegisterRoutes adds web UI routes to the given router.
|
||||
func (s *Server) RegisterRoutes(r chi.Router) {
|
||||
r.Get("/login", s.handleLoginPage)
|
||||
r.Post("/login", s.csrf.Middleware(http.HandlerFunc(s.handleLogin)).ServeHTTP)
|
||||
if s.ssoClient != nil {
|
||||
r.Get("/login", s.handleSSOLogin)
|
||||
r.Get("/sso/redirect", s.handleSSORedirect)
|
||||
r.Get("/sso/callback", s.handleSSOCallback)
|
||||
} else {
|
||||
r.Get("/login", s.handleLoginPage)
|
||||
r.Post("/login", s.csrf.Middleware(http.HandlerFunc(s.handleLogin)).ServeHTTP)
|
||||
}
|
||||
r.Get("/static/*", http.FileServer(http.FS(mcqweb.FS)).ServeHTTP)
|
||||
|
||||
// Authenticated routes.
|
||||
@@ -80,6 +110,7 @@ type pageData struct {
|
||||
Error string
|
||||
Title string
|
||||
Content any
|
||||
SSO bool
|
||||
}
|
||||
|
||||
func (s *Server) handleLoginPage(w http.ResponseWriter, r *http.Request) {
|
||||
@@ -101,6 +132,32 @@ func (s *Server) handleLogin(w http.ResponseWriter, r *http.Request) {
|
||||
http.Redirect(w, r, "/", http.StatusFound)
|
||||
}
|
||||
|
||||
// handleSSOLogin renders a landing page with a "Sign in with MCIAS" button.
|
||||
func (s *Server) handleSSOLogin(w http.ResponseWriter, r *http.Request) {
|
||||
web.RenderTemplate(w, mcqweb.FS, "login.html", pageData{SSO: true}, s.csrf.TemplateFunc(w))
|
||||
}
|
||||
|
||||
// handleSSORedirect initiates the SSO redirect to MCIAS.
|
||||
func (s *Server) handleSSORedirect(w http.ResponseWriter, r *http.Request) {
|
||||
if err := mcdsso.RedirectToLogin(w, r, s.ssoClient, "mcq"); err != nil {
|
||||
s.logger.Error("sso: redirect to login", "error", err)
|
||||
http.Error(w, "internal error", http.StatusInternalServerError)
|
||||
}
|
||||
}
|
||||
|
||||
// handleSSOCallback exchanges the authorization code for a JWT and sets the session.
|
||||
func (s *Server) handleSSOCallback(w http.ResponseWriter, r *http.Request) {
|
||||
token, returnTo, err := mcdsso.HandleCallback(w, r, s.ssoClient, "mcq")
|
||||
if err != nil {
|
||||
s.logger.Error("sso: callback", "error", err)
|
||||
http.Error(w, "Login failed. Please try again.", http.StatusUnauthorized)
|
||||
return
|
||||
}
|
||||
|
||||
web.SetSessionCookie(w, cookieName, token)
|
||||
http.Redirect(w, r, returnTo, http.StatusSeeOther)
|
||||
}
|
||||
|
||||
func (s *Server) handleLogout(w http.ResponseWriter, r *http.Request) {
|
||||
token := web.GetSessionToken(r, cookieName)
|
||||
if token != "" {
|
||||
|
||||
@@ -353,8 +353,75 @@ button:hover, .btn:hover {
|
||||
}
|
||||
|
||||
/* ===========================
|
||||
Read view
|
||||
Read view — two-column layout
|
||||
=========================== */
|
||||
.read-container {
|
||||
max-width: 1200px;
|
||||
margin: 0 auto;
|
||||
padding: 2rem 2rem 2rem 0;
|
||||
}
|
||||
.read-layout {
|
||||
display: grid;
|
||||
grid-template-columns: 220px 1fr;
|
||||
gap: 2rem;
|
||||
align-items: start;
|
||||
}
|
||||
.read-main {
|
||||
max-width: 900px;
|
||||
min-width: 0;
|
||||
}
|
||||
.read-toc {
|
||||
position: sticky;
|
||||
top: 1rem;
|
||||
max-height: calc(100vh - 2rem);
|
||||
overflow-y: auto;
|
||||
padding: 0.5rem 0;
|
||||
font-size: 0.75rem;
|
||||
line-height: 1.5;
|
||||
border-right: 1px solid var(--border-lt);
|
||||
padding-right: 1rem;
|
||||
}
|
||||
.read-toc ul {
|
||||
list-style: none;
|
||||
padding: 0;
|
||||
margin: 0;
|
||||
}
|
||||
.read-toc li {
|
||||
margin-bottom: 0.25rem;
|
||||
}
|
||||
.read-toc a {
|
||||
color: var(--text-lt);
|
||||
text-decoration: none;
|
||||
display: block;
|
||||
padding: 0.125rem 0;
|
||||
transition: color 0.1s;
|
||||
}
|
||||
.read-toc a:hover {
|
||||
color: var(--text);
|
||||
text-decoration: none;
|
||||
}
|
||||
.read-toc a.toc-active {
|
||||
color: var(--accent);
|
||||
}
|
||||
|
||||
@media (max-width: 900px) {
|
||||
.read-container {
|
||||
padding: 1.5rem 1rem;
|
||||
}
|
||||
.read-layout {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
.read-toc {
|
||||
position: static;
|
||||
max-height: none;
|
||||
border-right: none;
|
||||
border-bottom: 1px solid var(--border-lt);
|
||||
padding-right: 0;
|
||||
padding-bottom: 0.75rem;
|
||||
margin-bottom: 0.5rem;
|
||||
}
|
||||
}
|
||||
|
||||
.read-header {
|
||||
margin-bottom: 1.5rem;
|
||||
}
|
||||
|
||||
@@ -7,6 +7,11 @@
|
||||
</div>
|
||||
<div class="card">
|
||||
{{if .Error}}<div class="error">{{.Error}}</div>{{end}}
|
||||
{{if .SSO}}
|
||||
<div class="form-actions">
|
||||
<a href="/sso/redirect" style="display:block;text-align:center;text-decoration:none;"><button type="button" style="width:100%" class="btn">Sign in with MCIAS</button></a>
|
||||
</div>
|
||||
{{else}}
|
||||
<form method="POST" action="/login">
|
||||
{{csrfField}}
|
||||
<div class="form-group">
|
||||
@@ -25,5 +30,6 @@
|
||||
<button type="submit" class="btn">Login</button>
|
||||
</div>
|
||||
</form>
|
||||
{{end}}
|
||||
</div>
|
||||
{{end}}
|
||||
|
||||
@@ -1,32 +1,75 @@
|
||||
{{define "title"}} — {{.Doc.Title}}{{end}}
|
||||
{{define "container-class"}}read-container{{end}}
|
||||
{{define "content"}}
|
||||
<div class="read-header">
|
||||
<h2>{{.Doc.Title}}</h2>
|
||||
<div class="read-meta">
|
||||
<span>Pushed by {{.Doc.PushedBy}}</span>
|
||||
<span>{{.Doc.PushedAt}}</span>
|
||||
</div>
|
||||
<div class="read-actions">
|
||||
{{if .Doc.Read}}
|
||||
<form method="POST" action="/d/{{.Doc.Slug}}/unread" style="display:inline">
|
||||
{{csrfField}}
|
||||
<button type="submit" class="btn-ghost btn btn-sm">Mark unread</button>
|
||||
</form>
|
||||
{{else}}
|
||||
<form method="POST" action="/d/{{.Doc.Slug}}/read" style="display:inline">
|
||||
{{csrfField}}
|
||||
<button type="submit" class="btn-ghost btn btn-sm">Mark read</button>
|
||||
</form>
|
||||
{{end}}
|
||||
<form method="POST" action="/d/{{.Doc.Slug}}/delete" style="display:inline"
|
||||
onsubmit="return confirm('Delete this document?')">
|
||||
{{csrfField}}
|
||||
<button type="submit" class="btn-ghost btn btn-sm btn-danger">Unqueue</button>
|
||||
</form>
|
||||
<a href="/" class="btn-ghost btn btn-sm">Back to queue</a>
|
||||
<div class="read-layout">
|
||||
<nav class="read-toc" id="toc" aria-label="Table of contents"></nav>
|
||||
<div class="read-main">
|
||||
<div class="read-header">
|
||||
<h2>{{.Doc.Title}}</h2>
|
||||
<div class="read-meta">
|
||||
<span>Pushed by {{.Doc.PushedBy}}</span>
|
||||
<span>{{.Doc.PushedAt}}</span>
|
||||
</div>
|
||||
<div class="read-actions">
|
||||
{{if .Doc.Read}}
|
||||
<form method="POST" action="/d/{{.Doc.Slug}}/unread" style="display:inline">
|
||||
{{csrfField}}
|
||||
<button type="submit" class="btn-ghost btn btn-sm">Mark unread</button>
|
||||
</form>
|
||||
{{else}}
|
||||
<form method="POST" action="/d/{{.Doc.Slug}}/read" style="display:inline">
|
||||
{{csrfField}}
|
||||
<button type="submit" class="btn-ghost btn btn-sm">Mark read</button>
|
||||
</form>
|
||||
{{end}}
|
||||
<form method="POST" action="/d/{{.Doc.Slug}}/delete" style="display:inline"
|
||||
onsubmit="return confirm('Delete this document?')">
|
||||
{{csrfField}}
|
||||
<button type="submit" class="btn-ghost btn btn-sm btn-danger">Unqueue</button>
|
||||
</form>
|
||||
<a href="/" class="btn-ghost btn btn-sm">Back to queue</a>
|
||||
</div>
|
||||
</div>
|
||||
<div class="card markdown-body" id="article">
|
||||
{{.HTML}}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="card markdown-body">
|
||||
{{.HTML}}
|
||||
</div>
|
||||
<script>
|
||||
(function(){
|
||||
var toc=document.getElementById("toc");
|
||||
var headings=document.querySelectorAll("#article h1, #article h2, #article h3");
|
||||
if(headings.length<2){toc.style.display="none";return;}
|
||||
var ul=document.createElement("ul");
|
||||
var minLevel=6;
|
||||
headings.forEach(function(h){var l=parseInt(h.tagName[1]);if(l<minLevel)minLevel=l;});
|
||||
headings.forEach(function(h){
|
||||
if(!h.id)return;
|
||||
var li=document.createElement("li");
|
||||
var a=document.createElement("a");
|
||||
a.href="#"+h.id;
|
||||
a.textContent=h.textContent;
|
||||
var depth=parseInt(h.tagName[1])-minLevel;
|
||||
li.style.paddingLeft=(depth*0.75)+"rem";
|
||||
li.appendChild(a);
|
||||
ul.appendChild(li);
|
||||
});
|
||||
toc.appendChild(ul);
|
||||
/* highlight current section on scroll */
|
||||
var links=toc.querySelectorAll("a");
|
||||
var ids=[];links.forEach(function(a){ids.push(a.getAttribute("href").slice(1));});
|
||||
function onScroll(){
|
||||
var current="";
|
||||
for(var i=0;i<ids.length;i++){
|
||||
var el=document.getElementById(ids[i]);
|
||||
if(el&&el.getBoundingClientRect().top<=80)current=ids[i];
|
||||
}
|
||||
links.forEach(function(a){
|
||||
a.classList.toggle("toc-active",a.getAttribute("href")==="#"+current);
|
||||
});
|
||||
}
|
||||
window.addEventListener("scroll",onScroll,{passive:true});
|
||||
onScroll();
|
||||
})();
|
||||
</script>
|
||||
{{end}}
|
||||
|
||||
Reference in New Issue
Block a user