mc/metacircular
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

base: mc:docs/status-sync
Branches Tags
mc:master mc:docs/status-sync
..
compare: mc:4386fb08968eb0d7b03d511c9f6620f9993b0cec
Branches Tags
mc:master mc:docs/status-sync

11 Commits
docs/statu ... 4386fb0896

Author SHA1 Message Date
Kyle Isom
4386fb0896 Sync docs/metacircular.md versions and add undeploy capability 
Update version references to match current git tags: MCIAS v1.9.0,
Metacrypt v1.3.1, MCP v0.7.6. Add Phase D (DNS registration) to MCP
status, update RPC/CLI counts, and document undeploy as a first-class
capability. Also sync STATUS.md and packaging-and-deployment.md with
the same version updates.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-28 23:20:30 -07:00
Kyle Isom
5c4ed5c681 Update packaging docs: fix stale versions, add missing CLI commands 
Sync the deployed services version table with current git tags (MCIAS
v1.9.0, Metacrypt v1.3.1, MCR v1.2.1, MCNS v1.1.1, MCP v0.7.6) and
add mcp logs and mcp edit to the command reference.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-28 23:19:53 -07:00
Kyle Isom
4722fdb0da Sync platform docs: Phase D complete, Phase E planned, version updates 
- PLATFORM_EVOLUTION: Mark Phase D (DNS) complete, add Phase E
  (multi-node agent management) planning with items #10-12
- PLATFORM_EVOLUTION: Fix stale mcdsl reference (v1.2.0 adds → added,
  consuming services now on v1.4.0)
- STATUS: Update all service versions to current, note Phase A-D
  completion and Phase E planning
- docs/packaging-and-deployment: Add agent management section

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-28 23:19:53 -07:00
Kyle Isom
1715a53193 Update metacircular.md with system account auth model 
Document the three-tier identity model across MCIAS, MCR, Metacrypt,
MCNS, and MCP. Update version numbers for MCR (v1.2.1), MCNS (v1.1.1),
MCP (v0.7.2). Clarify that admin is reserved for MCIAS administration,
not routine deploy operations.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-28 23:18:53 -07:00
Kyle Isom
5e7e2522d7 Add CLI security standard: never echo passwords 
New standard requiring mcdsl/terminal.ReadPassword for interactive
password prompts. Codifies the fix applied to MCP's login command.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-28 23:18:53 -07:00
Kyle Isom
d041df01e4 Sync packaging docs with MCP implementation reality 
Fix service definition examples to require explicit image fields,
remove phantom version top-level field, document container listen
address requirement (0.0.0.0:$PORT not localhost), add undeploy
command, and update deployed service versions.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-28 23:18:53 -07:00
Kyle Isom
a04543028b Mark Phase C complete in PLATFORM_EVOLUTION.md 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-28 23:18:53 -07:00
Kyle Isom
90781014db Add mcdeploy to project maps, update MCDoc status 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-29 06:16:06 +00:00
Kyle Isom
24a8ae8c21 Sync STATUS.md: MC-Proxy v1.2.1, MCP v0.4.0, add MCDoc 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-29 06:16:06 +00:00
kyle
84610339aa Update deployed services versions in packaging guide 2026-03-29 06:15:57 +00:00
Kyle Isom
8550b8c2e5 Sync docs/metacircular.md versions and capabilities to current state 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-27 14:59:20 -07:00
7 changed files with 402 additions and 199 deletions
Expand all files Collapse all files

2
CLAUDE.md
View File

@@ -18,8 +18,10 @@ Metacircular is a multi-service personal infrastructure platform. This root repo
| `mcdsl/` | Standard library — shared packages for auth, db, config, HTTP/gRPC servers, CSRF, snapshots | Go | | `mcdsl/` | Standard library — shared packages for auth, db, config, HTTP/gRPC servers, CSRF, snapshots | Go |
| `mcdoc/` | Documentation server — renders markdown from Gitea, serves public docs via mc-proxy | Go | | `mcdoc/` | Documentation server — renders markdown from Gitea, serves public docs via mc-proxy | Go |
| `mcp/` | Control plane — operator-driven deployment, service registry, container lifecycle (master/agent) | Go | | `mcp/` | Control plane — operator-driven deployment, service registry, container lifecycle (master/agent) | Go |
| `mcdeploy/` | Deployment CLI — tactical bridge tool for build, push, deploy operations | Go |
| `mcns/` | Networking service — custom Go DNS server, authoritative for internal zones | Go | | `mcns/` | Networking service — custom Go DNS server, authoritative for internal zones | Go |
| `ca/` | PKI infrastructure and secrets for dev/test (not source code, gitignored) | — | | `ca/` | PKI infrastructure and secrets for dev/test (not source code, gitignored) | — |
| `docs/` | Platform-wide documentation (architecture overview, deployment guide) | Markdown |
Each subproject has its own `CLAUDE.md`, `ARCHITECTURE.md`, `Makefile`, and `go.mod`. When working in a subproject, read its own CLAUDE.md first. Each subproject has its own `CLAUDE.md`, `ARCHITECTURE.md`, `Makefile`, and `go.mod`. When working in a subproject, read its own CLAUDE.md first.

181
PLATFORM_EVOLUTION.md
View File

@@ -5,7 +5,7 @@ from its current manually-wired state to fully declarative deployment.
It is a living design document — not a spec, not a commitment, but a It is a living design document — not a spec, not a commitment, but a
record of where we are, where we want to be, and what's between. record of where we are, where we want to be, and what's between.
Last updated: 2026-03-27 (Phases A + B complete) Last updated: 2026-03-28 (Phases A + B + C + D complete)
--- ---
@@ -181,9 +181,9 @@ about one node, one mc-proxy, or loopback-only backends.
#### 1. mcdsl: Proper Module Versioning — DONE #### 1. mcdsl: Proper Module Versioning — DONE
mcdsl is already properly versioned and released: mcdsl is already properly versioned and released:
- Tagged releases: `v0.1.0`, `v1.0.0`, `v1.0.1` - Tagged releases: `v0.1.0`, `v1.0.0`, `v1.0.1`, `v1.1.0`, `v1.2.0`
- All consuming services import by URL with pinned versions - All consuming services import by URL with pinned versions
(mcr, mcat, mcns, mc-proxy → `v1.0.0`; metacrypt → `v1.0.1`) (all consuming services on `v1.2.0`)
- No `replace` directives anywhere - No `replace` directives anywhere
- Docker builds use standard `go mod download` - Docker builds use standard `go mod download`
- `uses_mcdsl` eliminated from service definitions and docs - `uses_mcdsl` eliminated from service definitions and docs
@@ -215,18 +215,14 @@ routes during deploy and stop:
- L4 routes: TLS passthrough, backend handles its own TLS - L4 routes: TLS passthrough, backend handles its own TLS
- Hostnames default to `<service>.svc.mcp.metacircular.net` - Hostnames default to `<service>.svc.mcp.metacircular.net`
#### 4. MCP Agent: TLS Cert Provisioning #### 4. MCP Agent: TLS Cert Provisioning — DONE
**Gap**: certs are manually provisioned and placed on disk. There is no Agent provisions TLS certificates from Metacrypt CA automatically during
automated issuance flow. deploy for L7 routes:
- ACME client library requests certs from Metacrypt CA via its API
**Work**: - Certs stored in `/srv/mc-proxy/certs/<service>.pem`
- Agent requests certs from Metacrypt CA via its API. - Provisioning happens during deploy before mc-proxy route registration
- Certs are stored in a standard location - L7 routes get agent-provisioned certs; L4 routes use service-managed TLS
(`/srv/mc-proxy/certs/<service>.pem`).
- Cert renewal is handled automatically before expiry.
**Depends on**: Metacrypt cert issuance policy (#7).
#### 5. mc-proxy: Route Persistence — DONE #### 5. mc-proxy: Route Persistence — DONE
@@ -243,57 +239,49 @@ mc-proxy routes are fully persisted in SQLite and survive restarts:
bootstrap before MCP is operational. The gRPC API and mcproxyctl bootstrap before MCP is operational. The gRPC API and mcproxyctl
are the primary route management interfaces going forward. are the primary route management interfaces going forward.
#### 6. MCP Agent: DNS Registration #### 6. MCP Agent: DNS Registration — DONE
**Gap**: DNS records are manually configured in MCNS zone files. Agent automatically manages DNS records during deploy and stop:
- Deploy: calls MCNS API to create/update A records for
`<service>.svc.mcp.metacircular.net` pointing to the node's address.
- Stop/undeploy: removes DNS records before stopping containers.
- Config: `[mcns]` section in agent config with server URL, CA cert,
token path, zone, and node address.
- Nil-safe: if MCNS not configured, silently skipped (backward compatible).
- Authorization: mcp-agent system account can manage any record name.
**Work**: #### 7. Metacrypt: Automated Cert Issuance Policy — DONE
- Agent creates/updates A records in MCNS for
`<service>.svc.mcp.metacircular.net`.
- Agent removes records on service teardown.
**Depends on**: MCNS record management API (#8). MCP agent has MCIAS credentials and Metacrypt policy for automated cert
issuance:
- MCP agent authenticates to Metacrypt with MCIAS service credentials
- Metacrypt policy allows cert issuance for
`*.svc.mcp.metacircular.net`
- One cert per hostname per service — no wildcard certs
#### 7. Metacrypt: Automated Cert Issuance Policy #### 8. MCNS: Record Management API — DONE
**Gap**: no policy exists for automated cert issuance. The MCP agent MCNS provides full CRUD for DNS records via REST and gRPC:
doesn't have a Metacrypt identity or permissions. - REST: POST/GET/PUT/DELETE on `/v1/zones/{zone}/records`
- gRPC: RecordService with ListRecords, CreateRecord, GetRecord,
**Work**: UpdateRecord, DeleteRecord RPCs
- MCP agent gets an MCIAS service account. - SQLite-backed with transactional writes, CNAME exclusivity enforcement,
- Metacrypt policy allows this account to issue certs scoped to and automatic SOA serial bumping on mutations
`*.svc.mcp.metacircular.net` (and explicitly listed public - Authorization: admin can manage any record, mcp-agent system account
hostnames). can manage any record name, other system accounts scoped to own name
- No wildcard certs — one cert per hostname per service. - MCP agent uses the REST API to register/deregister records on
deploy/stop
**Depends on**: MCIAS service account provisioning (exists today, just
needs the account created).
#### 8. MCNS: Record Management API
**Gap**: MCNS v1.0.0 has REST + gRPC APIs and SQLite storage, but
records are currently seeded from migrations (static). The API supports
CRUD operations but MCP does not yet call it for dynamic registration.
**Work**:
- MCP agent calls MCNS API to create/update/delete records on
deploy/stop.
- MCIAS auth scoping to allow MCP agent to manage
`*.svc.mcp.metacircular.net` records.
**Depends on**: MCNS API exists. Remaining work is MCP integration
and auth scoping.
#### 9. Application $PORT Convention — DONE #### 9. Application $PORT Convention — DONE
mcdsl v1.1.0 adds `$PORT` and `$PORT_GRPC` env var support: mcdsl v1.2.0 added `$PORT` and `$PORT_GRPC` env var support:
- `config.Load` checks `$PORT` → overrides `Server.ListenAddr` - `config.Load` checks `$PORT` → overrides `Server.ListenAddr`
- `config.Load` checks `$PORT_GRPC` → overrides `Server.GRPCAddr` - `config.Load` checks `$PORT_GRPC` → overrides `Server.GRPCAddr`
- Takes precedence over TOML and generic env overrides - Takes precedence over TOML and generic env overrides
(`$MCR_SERVER_LISTEN_ADDR`) — agent-assigned ports are authoritative (`$MCR_SERVER_LISTEN_ADDR`) — agent-assigned ports are authoritative
- Handles both `config.Base` embedding (MCR, MCNS, MCAT) and direct - Handles both `config.Base` embedding (MCR, MCNS, MCAT) and direct
`ServerConfig` embedding (Metacrypt) via struct tree walking `ServerConfig` embedding (Metacrypt) via struct tree walking
- MCR, Metacrypt, MCNS upgraded to mcdsl v1.1.0 - All consuming services on mcdsl v1.4.0
--- ---
@@ -311,33 +299,90 @@ Phase A — Independent groundwork: ✓ COMPLETE
Phase B — MCP route registration: ✓ COMPLETE Phase B — MCP route registration: ✓ COMPLETE
#3 Agent registers routes with mc-proxy ✓ DONE #3 Agent registers routes with mc-proxy ✓ DONE
Phase C — Automated TLS: Phase C — Automated TLS: ✓ COMPLETE
#7 Metacrypt cert issuance policy #7 Metacrypt cert issuance policy ✓ DONE
#4 Agent provisions certs #4 Agent provisions certs ✓ DONE
(depends on #7) (depends on #7)
Phase D — DNS: Phase D — DNS: ✓ COMPLETE
#8 MCNS record management API #8 MCNS record management API ✓ DONE
#6 Agent registers DNS #6 Agent registers DNS ✓ DONE
(depends on #8) (depends on #8)
Phase E — Multi-node agent management:
#10 Agent binary at /srv/mcp/mcp-agent on all nodes
#11 mcp agent upgrade (SSH-based cross-compiled push)
#12 Node provisioning tooling (Debian + NixOS)
(depends on #10)
``` ```
**Phases A and B are complete.** Services can be deployed with **Phases A, B, C, and D are complete.** Services can be deployed with
agent-assigned ports, `$PORT` env vars, and automatic mc-proxy route agent-assigned ports, `$PORT` env vars, automatic mc-proxy route
registration. No more manual port picking, mcproxyctl, or TOML editing. registration, automated TLS cert provisioning from Metacrypt CA, and
automatic DNS registration in MCNS. No more manual port picking,
The remaining manual steps are TLS cert provisioning (Phase C) and mcproxyctl, TOML editing, cert generation, or DNS zone editing.
DNS registration (Phase D).
### Immediate Next Steps ### Immediate Next Steps
1. **Phase C: Automated TLS** — Metacrypt cert issuance policy for MCP 1. **Phase E: Multi-node agent management** — see below.
agent, then agent provisions certs automatically during deploy. 2. **mcdoc implementation** — fully designed, no platform evolution
2. **Phase D: DNS** — MCNS record management API integration, then
agent registers DNS records during deploy.
3. **mcdoc implementation** — fully designed, no platform evolution
dependency. Deployable now with the new route system. dependency. Deployable now with the new route system.
#### 10. Agent Binary Location Convention
**Gap**: The agent binary is currently NixOS-managed on rift (lives in
`/nix/store/`, systemd `ExecStart` points there). This doesn't work for
Debian nodes and requires a full `nixos-rebuild` for every MCP release.
**Work**:
- Standardize agent binary at `/srv/mcp/mcp-agent` on all nodes.
- NixOS config: change `ExecStart` from nix store path to
`/srv/mcp/mcp-agent`. NixOS still owns user, systemd unit, podman,
directories — just not the binary version.
- Debian nodes: same layout, provisioned by setup script.
#### 11. Agent Upgrade via SSH Push
**Gap**: Updating the agent requires manual, OS-specific steps. On
NixOS: update flake lock, commit, push, rebuild. On Debian: build, scp,
restart. With multiple nodes and architectures (amd64 + arm64), this
doesn't scale.
**Work**:
- `mcp agent upgrade [node]` CLI command.
- Cross-compiles agent for each target arch (`GOARCH` from node config).
- Uses `golang.org/x/crypto/ssh` to push the binary and restart the
service. No external tool dependencies.
- Node config gains `ssh` (hostname) and `arch` (GOARCH) fields.
- Upgrades all nodes by default to prevent version skew. New RPCs cause
`Unimplemented` errors if agent and CLI are out of sync.
**Depends on**: #10 (binary location convention).
#### 12. Node Provisioning Tooling
**Gap**: Setting up a new node requires manual steps: create user,
create directories, install podman, write config, create systemd unit.
Different for NixOS vs Debian.
**Work**:
- Go-based provisioning tool (part of MCP CLI) or standalone script.
- `mcp node provision <name>` SSHs to the node and runs setup:
create `mcp` user with podman access, create `/srv/mcp/`, write
systemd unit, install initial binary, start service.
- For NixOS, provisioning remains in the NixOS config (declarative).
The provisioning tool targets Debian/generic Linux.
**Depends on**: #10 (binary location convention), #11 (SSH infra).
**Current fleet**:
| Node | OS | Arch | Status |
|------|----|------|--------|
| rift | NixOS | amd64 | Operational, single MCP agent |
| hyperborea | Debian (RPi) | arm64 | Online, needs agent provisioning |
| svc | Debian | amd64 | Runs MCIAS, needs agent for public edge services |
--- ---
## Open Questions ## Open Questions

4
README.md
View File

@@ -25,7 +25,8 @@ lives in [docs/metacircular.md](docs/metacircular.md).
| **MC-Proxy** | Node ingress — TLS proxy and router. L4 passthrough or L7 terminating (per-route), PROXY protocol, firewall with rate limiting and GeoIP. | Implemented | | **MC-Proxy** | Node ingress — TLS proxy and router. L4 passthrough or L7 terminating (per-route), PROXY protocol, firewall with rate limiting and GeoIP. | Implemented |
| **MCNS** | Networking — authoritative DNS for internal platform zones, upstream forwarding. | Implemented | | **MCNS** | Networking — authoritative DNS for internal platform zones, upstream forwarding. | Implemented |
| **MCP** | Control plane — operator-driven deployment, service registry, data transfer, master/agent container lifecycle. | Implemented | | **MCP** | Control plane — operator-driven deployment, service registry, data transfer, master/agent container lifecycle. | Implemented |
| **MCDoc** | Documentation server — renders markdown from Gitea, serves public docs. | In progress | | **MCDoc** | Documentation server — renders markdown from Gitea, serves public docs. | Implemented |
| **MCDeploy** | Deployment CLI — single-binary tool for build, push, deploy, cert renewal, and status. Tactical bridge tool while MCP capabilities mature. | Active dev |
Shared library: **MCDSL** — standard library for all services (auth, db, Shared library: **MCDSL** — standard library for all services (auth, db,
config, TLS server, CSRF, snapshots). config, TLS server, CSRF, snapshots).
@@ -102,6 +103,7 @@ metacircular/
├── mcns/ DNS server ├── mcns/ DNS server
├── mcat/ Login policy tester ├── mcat/ Login policy tester
├── mcdsl/ Standard library (shared packages) ├── mcdsl/ Standard library (shared packages)
├── mcdeploy/ Deployment CLI tool
├── mcdoc/ Documentation server ├── mcdoc/ Documentation server
├── ca/ PKI infrastructure (dev/test, not source code) ├── ca/ PKI infrastructure (dev/test, not source code)
└── docs/ Platform-wide documentation └── docs/ Platform-wide documentation

95
STATUS.md
View File

@@ -1,6 +1,6 @@
# Metacircular Platform Status # Metacircular Platform Status
Last updated: 2026-03-26 Last updated: 2026-03-28
## Platform Overview ## Platform Overview
@@ -8,27 +8,30 @@ One node operational (**rift**), running core infrastructure services as
containers fronted by MC-Proxy. MCIAS runs separately (not on rift). containers fronted by MC-Proxy. MCIAS runs separately (not on rift).
Bootstrap phases 0–4 complete (MCIAS, Metacrypt, MC-Proxy, MCR all Bootstrap phases 0–4 complete (MCIAS, Metacrypt, MC-Proxy, MCR all
operational). MCP is deployed and managing all platform containers. MCNS is operational). MCP is deployed and managing all platform containers. MCNS is
deployed on rift, serving authoritative DNS. deployed on rift, serving authoritative DNS. Platform evolution Phases A–D
complete (automated port assignment, route registration, TLS cert
provisioning, and DNS registration). Multi-node deployment is being planned
(Phase E).
## Service Status ## Service Status
| Service | Version | SDLC Phase | Deployed | Node | | Service | Version | SDLC Phase | Deployed | Node |
|---------|---------|------------|----------|------| |---------|---------|------------|----------|------|
| MCIAS | v1.8.0 | Maintenance | Yes | (separate) | | MCIAS | v1.9.0 | Maintenance | Yes | (separate) |
| Metacrypt | v1.1.0 | Production | Yes | rift | | Metacrypt | v1.3.1 | Production | Yes | rift |
| MC-Proxy | v1.1.0 | Maintenance | Yes | rift | | MC-Proxy | v1.2.1 | Maintenance | Yes | rift |
| MCR | v1.2.0 | Production | Yes | rift | | MCR | v1.2.1 | Production | Yes | rift |
| MCAT | v1.1.0 | Complete | Unknown | — | | MCAT | v1.1.1 | Complete | Unknown | — |
| MCDSL | v1.2.0 | Stable | N/A (library) | — | | MCDSL | v1.4.0 | Stable | N/A (library) | — |
| MCNS | v1.1.0 | Production | Yes | rift | | MCNS | v1.1.1 | Production | Yes | rift |
| MCP | v0.3.0 | Production | Yes | rift | | MCDoc | v0.1.0 | Production | Yes | rift |
| MCDeploy | v0.2.0 | Active dev | N/A (CLI tool) | — | | MCP | v0.7.6 | Production | Yes | rift |
## Service Details ## Service Details
### MCIAS — Identity and Access Service ### MCIAS — Identity and Access Service
- **Version:** v1.8.0 (client library: clients/go/v0.2.0) - **Version:** v1.9.0 (client library: clients/go/v0.2.0)
- **Phase:** Maintenance. Phases 0-14 complete. Feature-complete with active - **Phase:** Maintenance. Phases 0-14 complete. Feature-complete with active
refinement. refinement.
- **Deployment:** Running in production. All other services authenticate - **Deployment:** Running in production. All other services authenticate
@@ -40,7 +43,7 @@ deployed on rift, serving authoritative DNS.
### Metacrypt — Cryptographic Service Engine ### Metacrypt — Cryptographic Service Engine
- **Version:** v1.1.0. - **Version:** v1.3.1.
- **Phase:** Production. All four engine types implemented (CA, SSH CA, transit, - **Phase:** Production. All four engine types implemented (CA, SSH CA, transit,
user-to-user). Active work on integration test coverage. user-to-user). Active work on integration test coverage.
- **Deployment:** Running on rift as a container, fronted by MC-Proxy on - **Deployment:** Running on rift as a container, fronted by MC-Proxy on
@@ -52,18 +55,19 @@ deployed on rift, serving authoritative DNS.
### MC-Proxy — TLS Proxy and Router ### MC-Proxy — TLS Proxy and Router
- **Version:** v1.1.0. Phases 1-8 complete. - **Version:** v1.2.1.
- **Phase:** Maintenance. Stable and actively routing traffic on rift. - **Phase:** Maintenance. Stable and actively routing traffic on rift.
- **Deployment:** Running on rift. Fronts Metacrypt, MCR, and sgard on ports - **Deployment:** Running on rift. Fronts Metacrypt, MCR, and sgard on ports
443, 8443, and 9443. Prometheus metrics on 127.0.0.1:9091. 443, 8443, and 9443. Prometheus metrics on 127.0.0.1:9091. Routes persisted
- **Recent work:** MCR route additions, Nix flake, L7 backend cert handling, in SQLite and managed via gRPC API.
Prometheus metrics, L7 policies. - **Recent work:** Route persistence (SQLite), idempotent AddRoute (upsert),
golangci-lint v2 compliance, module path migration to mc/ org.
- **Artifacts:** systemd units (service + backup timer), Docker Compose - **Artifacts:** systemd units (service + backup timer), Docker Compose
(standard + rift), install and backup scripts, rift config. (standard + rift), install and backup scripts, rift config.
### MCR — Container Registry ### MCR — Container Registry
- **Version:** v1.2.0. All implementation phases complete. - **Version:** v1.2.1. All implementation phases complete.
- **Phase:** Production. Deployed on rift, serving container images. - **Phase:** Production. Deployed on rift, serving container images.
- **Deployment:** Running on rift as two containers (mcr API + mcr-web), - **Deployment:** Running on rift as two containers (mcr API + mcr-web),
fronted by MC-Proxy on ports 443 (web, L7), 8443 (API, L4), and fronted by MC-Proxy on ports 443 (web, L7), 8443 (API, L4), and
@@ -76,7 +80,7 @@ deployed on rift, serving authoritative DNS.
### MCAT — Login Policy Tester ### MCAT — Login Policy Tester
- **Version:** v1.1.0. - **Version:** v1.1.1.
- **Phase:** Complete. Diagnostic tool, not core infrastructure. - **Phase:** Complete. Diagnostic tool, not core infrastructure.
- **Deployment:** Available for ad-hoc use. Lightweight tool for testing - **Deployment:** Available for ad-hoc use. Lightweight tool for testing
MCIAS login policy rules. MCIAS login policy rules.
@@ -85,51 +89,56 @@ deployed on rift, serving authoritative DNS.
### MCDSL — Standard Library ### MCDSL — Standard Library
- **Version:** v1.2.0. - **Version:** v1.4.0.
- **Phase:** Stable. All 9 packages implemented and tested. Being adopted - **Phase:** Stable. All 9 packages implemented and tested. Being adopted
across the platform. across the platform.
- **Deployment:** N/A (Go library, imported by other services). - **Deployment:** N/A (Go library, imported by other services).
- **Packages:** auth, db, config, httpserver, grpcserver, csrf, web, health, - **Packages:** auth, db, config, httpserver, grpcserver, csrf, web, health,
archive. archive.
- **Adoption:** All services except mcias on v1.2.0. mcias pending. - **Adoption:** All services except mcias on v1.4.0. mcias pending.
### MCNS — Networking Service ### MCNS — Networking Service
- **Version:** v1.1.0. - **Version:** v1.1.1.
- **Phase:** Production. Custom Go DNS server replacing CoreDNS precursor. - **Phase:** Production. Custom Go DNS server replacing CoreDNS precursor.
- **Deployment:** Running on rift as a container managed by MCP. Serves two - **Deployment:** Running on rift as a container managed by MCP. Serves two
authoritative zones plus upstream forwarding. authoritative zones plus upstream forwarding. REST + gRPC APIs with MCIAS
auth and name-scoped system account authorization.
- **Recent work:** v1.0.0 implementation (custom Go DNS server), engineering - **Recent work:** v1.0.0 implementation (custom Go DNS server), engineering
review, deployed to rift replacing CoreDNS. review, deployed to rift replacing CoreDNS.
- **Artifacts:** Dockerfile, Docker Compose (rift), MCP service definition, - **Artifacts:** Dockerfile, Docker Compose (rift), MCP service definition,
systemd units, install script, example config. systemd units, install script, example config.
### MCDoc — Documentation Server
- **Version:** v0.1.0.
- **Phase:** Production. Fetches and renders markdown documentation from Gitea.
- **Deployment:** Running on rift as a container, fronted by MC-Proxy on
port 443 (L7).
- **Recent work:** Initial implementation, Gitea content fetching, goldmark
rendering with syntax highlighting, webhook-driven refresh.
- **Artifacts:** Dockerfile, MCP service definition.
### MCP — Control Plane ### MCP — Control Plane
- **Version:** v0.3.0. - **Version:** v0.7.6.
- **Phase:** Production. Phases 0-4 complete. Deployed to rift, managing all - **Phase:** Production. Phases A–D complete. Deployed to rift, managing all
platform containers. platform containers.
- **Deployment:** Running on rift. Agent as systemd service under `mcp` user - **Deployment:** Running on rift. Agent as systemd service under `mcp` user
with rootless podman. Manages metacrypt, mc-proxy, mcr, and mcns containers. with rootless podman. Manages metacrypt, mc-proxy, mcr, mcns, and mcdoc
containers.
- **Architecture:** Two components — `mcp` CLI (thin client on vade) and - **Architecture:** Two components — `mcp` CLI (thin client on vade) and
`mcp-agent` (per-node daemon with SQLite registry, podman management, `mcp-agent` (per-node daemon with SQLite registry, podman management,
monitoring with drift/flap detection). gRPC-only (no REST). monitoring with drift/flap detection, route registration with mc-proxy,
- **Recent work:** Full v1 implementation (12 RPCs, 15 CLI commands), automated TLS cert provisioning for L7 routes via Metacrypt CA, automated
deployment to rift, container migration from kyle→mcp user, service DNS registration in MCNS). gRPC-only (no REST). 15 RPCs, 17+ CLI commands.
definition authoring. - **Recent work:** Phase C (automated TLS cert provisioning), Phase D
(automated DNS registration via MCNS), undeploy command, logs command,
edit command, auto-login to MCR, system account auth model, module path
migration.
- **Artifacts:** systemd service (NixOS), TLS cert from Metacrypt, service - **Artifacts:** systemd service (NixOS), TLS cert from Metacrypt, service
definition files, design docs. definition files, design docs.
### MCDeploy — Deployment CLI
- **Version:** v0.2.0.
- **Phase:** Active development. Tactical bridge tool for deploying services
while MCP is being built.
- **Deployment:** N/A (local CLI tool, not a server).
- **Recent work:** Initial implementation, Nix flake.
- **Description:** Single-binary CLI that shells out to podman/ssh/scp/git
for build, push, deploy, cert renewal, and status. TOML-configured.
## Node Inventory ## Node Inventory
| Node | Address (LAN) | Address (Tailscale) | Role | | Node | Address (LAN) | Address (Tailscale) | Role |
@@ -138,10 +147,14 @@ deployed on rift, serving authoritative DNS.
## Rift Port Map ## Rift Port Map
Note: Services deployed via MCP receive dynamically assigned host ports
(10000–60000). The ports below are for infrastructure services with static
assignments or well-known ports.
| Port | Protocol | Services | | Port | Protocol | Services |
|------|----------|----------| |------|----------|----------|
| 53 | DNS (LAN + Tailscale) | mcns | | 53 | DNS (LAN + Tailscale) | mcns |
| 443 | L7 (TLS termination) | metacrypt-web, mcr-web | | 443 | L7 (TLS termination) | metacrypt-web, mcr-web, mcdoc |
| 8080 | HTTP (all interfaces) | exod | | 8080 | HTTP (all interfaces) | exod |
| 8443 | L4 (SNI passthrough) | metacrypt API, mcr API | | 8443 | L4 (SNI passthrough) | metacrypt API, mcr API |
| 9090 | HTTP (all interfaces) | exod | | 9090 | HTTP (all interfaces) | exod |

141
docs/metacircular.md
View File

@@ -48,11 +48,11 @@ the spec disagree, one of them has a bug.
## High-Level Overview ## High-Level Overview
Metacircular infrastructure is built from six core components, plus a shared Metacircular infrastructure is built from six core components and a
standard library (**MCDSL**) that provides the common patterns all services documentation server, plus a shared standard library (**MCDSL**) that provides
depend on (auth integration, database setup, config loading, HTTP/gRPC server the common patterns all services depend on (auth integration, database setup,
bootstrapping, CSRF, web session management, health checks, snapshots, and config loading, HTTP/gRPC server bootstrapping, CSRF, web session management,
service directory archiving): health checks, snapshots, and service directory archiving):
- **MCIAS** — Identity and access. The root of trust for all other services. - **MCIAS** — Identity and access. The root of trust for all other services.
Handles authentication, token issuance, role management, and login policy Handles authentication, token issuance, role management, and login policy
@@ -75,6 +75,10 @@ service directory archiving):
accepts outside connections, and routes them to the correct service — either accepts outside connections, and routes them to the correct service — either
as raw TCP passthrough or via TLS-terminating HTTP/2 reverse proxy. as raw TCP passthrough or via TLS-terminating HTTP/2 reverse proxy.
- **MCDoc** — Documentation server. Fetches markdown from Gitea repositories,
renders HTML with syntax highlighting, serves a navigable documentation site.
Public-facing, no MCIAS authentication required.
These components form a dependency graph rooted at MCIAS: These components form a dependency graph rooted at MCIAS:
``` ```
@@ -179,14 +183,19 @@ delegates authentication to it; no service maintains its own user database.
Services validate tokens by calling back to MCIAS (cached 30s by SHA-256 of Services validate tokens by calling back to MCIAS (cached 30s by SHA-256 of
the token). the token).
- **Role-based access.** Three roles — `admin` (full access, policy bypass), - **Role-based access.** Three roles — `admin` (MCIAS account management,
`user` (policy-governed), `guest` (service-dependent restrictions). Admin policy changes, zone mutations — reserved for human operators), `user`
detection comes solely from the MCIAS `admin` role; services never promote (policy-governed), `guest` (service-dependent restrictions, rejected by MCP
users locally. agent). Admin detection comes solely from the MCIAS `admin` role; services
never promote users locally. Routine operations (deploy, push, DNS updates)
do not require admin.
- **Account types.** Human accounts (interactive users) and system accounts - **Account types.** Human accounts (interactive users) and system accounts
(service-to-service). Both authenticate the same way; system accounts enable (service-to-service). Both produce standard JWTs validated the same way.
automated workflows. System accounts carry no roles — their authorization is handled by each
service's policy engine (Metacrypt policies, MCNS name-scoped access, MCR
default policies). System account tokens are long-lived (365-day default)
and do not require passwords for issuance.
- **Login policy.** Priority-based ACL rules that control who can log into - **Login policy.** Priority-based ACL rules that control who can log into
which services. Rules can target roles, account types, service names, and which services. Rules can target roles, account types, service names, and
@@ -204,7 +213,7 @@ MCIAS evaluates login policy against the service context, verifies credentials,
and returns a bearer token. The MCIAS Go client library and returns a bearer token. The MCIAS Go client library
(`git.wntrmute.dev/mc/mcias/clients/go`) handles this flow. (`git.wntrmute.dev/mc/mcias/clients/go`) handles this flow.
**Status:** Implemented. v1.7.0. Feature-complete with active refinement **Status:** Implemented. v1.9.0. Feature-complete with active refinement
(WebAuthn/FIDO2 passkeys, TOTP 2FA, service-context login policies). (WebAuthn/FIDO2 passkeys, TOTP 2FA, service-context login policies).
--- ---
@@ -255,7 +264,7 @@ core.
operations on which engine mounts. Priority-based evaluation, default deny, operations on which engine mounts. Priority-based evaluation, default deny,
admin bypass. See Metacrypt's `POLICY.md` for the full model. admin bypass. See Metacrypt's `POLICY.md` for the full model.
**Status:** Implemented. All four engine types complete — CA (with ACME **Status:** Implemented. v1.3.1. All four engine types complete — CA (with ACME
support), SSH CA, transit encryption, and user-to-user encryption. support), SSH CA, transit encryption, and user-to-user encryption.
--- ---
@@ -274,7 +283,9 @@ serves the container images that MCP deploys across the platform.
- **Authenticated access.** No anonymous access. MCR uses the OCI token - **Authenticated access.** No anonymous access. MCR uses the OCI token
authentication flow: clients hit `/v2/`, receive a 401 with a token authentication flow: clients hit `/v2/`, receive a 401 with a token
endpoint, authenticate via MCIAS, and use the returned JWT for subsequent endpoint, authenticate via MCIAS, and use the returned JWT for subsequent
requests. requests. The token endpoint accepts both username/password (standard
login) and pre-existing MCIAS JWTs as passwords (personal-access-token
pattern), enabling non-interactive push/pull for system accounts and CI.
- **Policy-controlled push/pull.** Fine-grained ACL rules govern who can push - **Policy-controlled push/pull.** Fine-grained ACL rules govern who can push
to or pull from which repositories. Integrated with MCIAS roles. to or pull from which repositories. Integrated with MCIAS roles.
@@ -286,7 +297,7 @@ serves the container images that MCP deploys across the platform.
is scheduled, MCP tells the node's agent which image to pull and where to get is scheduled, MCP tells the node's agent which image to pull and where to get
it. MCR sits behind an MC-Proxy instance for TLS routing. it. MCR sits behind an MC-Proxy instance for TLS routing.
**Status:** Implemented. Phase 13 (deployment artifacts) complete. **Status:** Implemented. v1.2.1. All implementation phases complete.
--- ---
@@ -333,7 +344,9 @@ two instances — an edge proxy on a public VPS and an origin proxy on the
private network, connected over the overlay with PROXY protocol preserving private network, connected over the overlay with PROXY protocol preserving
client IPs across the hop. client IPs across the hop.
**Status:** Implemented. **Status:** Implemented. v1.2.1. Route state persisted in SQLite with
write-through semantics. gRPC admin API with idempotent AddRoute for runtime
route management.
--- ---
@@ -365,9 +378,13 @@ into DNS records.
using internal DNS names automatically resolve to the right place without using internal DNS names automatically resolve to the right place without
config changes. config changes.
- **Record management API.** Authenticated via MCIAS. MCP is the primary - **Record management API.** Authenticated via MCIAS with name-scoped
consumer for dynamic updates. Operators can also manage records directly authorization. Admin can manage all records and zones. The `mcp-agent`
for static entries (node addresses, aliases). system account can create and delete any record. Other system accounts
can only manage records matching their own name (e.g., system account
`mcq` can manage `mcq.svc.mcp.metacircular.net` but not other records).
Human users have read-only access to records. Zone mutations (create,
update, delete zones) remain admin-only.
**How it fits in:** MCNS answers "what is the address of X?" MCP answers "where **How it fits in:** MCNS answers "what is the address of X?" MCP answers "where
is service α running?" and pushes the answer to MCNS. This separation means is service α running?" and pushes the answer to MCNS. This separation means
@@ -375,10 +392,11 @@ services can use stable DNS names in their configs (e.g.,
`mcias.svc.mcp.metacircular.net` in `[mcias] server_url`) that survive `mcias.svc.mcp.metacircular.net` in `[mcias] server_url`) that survive
migration without config changes. migration without config changes.
**Status:** Implemented. v1.0.0. Custom Go DNS server deployed on rift, **Status:** Implemented. v1.1.1. Custom Go DNS server deployed on rift,
serving two authoritative zones (`svc.mcp.metacircular.net` and serving two authoritative zones (`svc.mcp.metacircular.net` and
`mcp.metacircular.net`) plus upstream forwarding. REST + gRPC APIs with `mcp.metacircular.net`) plus upstream forwarding. REST + gRPC APIs with
MCIAS auth. Records stored in SQLite. MCIAS auth and name-scoped system account authorization. Records stored
in SQLite.
--- ---
@@ -403,6 +421,10 @@ each managed node.
the initial config, pulls the image from MCR, starts the container, and the initial config, pulls the image from MCR, starts the container, and
pushes a DNS update to MCNS (`α.svc.mcp.metacircular.net` → node address). pushes a DNS update to MCNS (`α.svc.mcp.metacircular.net` → node address).
- **Undeploy.** Full teardown of a service. Stops the container, removes
MC-Proxy routes, deletes DNS records from MCNS, and cleans up the service
registry entry. The inverse of deploy.
- **Migrate.** Move a service from one node to another. MCP snapshots the - **Migrate.** Move a service from one node to another. MCP snapshots the
service's `/srv/<service>/` directory on the source node (as a tar.zst service's `/srv/<service>/` directory on the source node (as a tar.zst
image), transfers it to the destination, extracts it, starts the service, image), transfers it to the destination, extracts it, starts the service,
@@ -429,9 +451,17 @@ each managed node.
- **Master/agent architecture.** MCP Master runs on the operator's machine. - **Master/agent architecture.** MCP Master runs on the operator's machine.
Agents run on every managed node, receiving C2 (command and control) from Agents run on every managed node, receiving C2 (command and control) from
Master, reporting node status, and managing local workloads. The C2 channel Master, reporting node status, and managing local workloads. The C2 channel
is authenticated via MCIAS. The master does not need to be always-on — is authenticated via MCIAS — any authenticated non-guest user or system
agents keep running their workloads independently; the master is needed only account is accepted (admin role is not required for deploy operations).
to issue new commands. The master does not need to be always-on — agents keep running their
workloads independently; the master is needed only to issue new commands.
- **System account automation.** The agent uses an `mcp-agent` system account
for all service-to-service communication: TLS cert provisioning (Metacrypt),
DNS record management (MCNS), and container image pulls (MCR). Each service
authorizes the agent through its own policy engine. Per-service system
accounts (e.g., `mcq`) can be created for scoped self-management — a service
account can only manage its own DNS records, not other services'.
- **Node management.** Track which nodes are in the platform, their health, - **Node management.** Track which nodes are in the platform, their health,
available resources, and running workloads. available resources, and running workloads.
@@ -452,11 +482,15 @@ services it depends on.
can deploy them. The systemd unit files exist as a fallback and for bootstrap — can deploy them. The systemd unit files exist as a fallback and for bootstrap —
the long-term deployment model is MCP-managed containers. the long-term deployment model is MCP-managed containers.
**Status:** Implemented. v0.1.0. Deployed on rift managing all platform **Status:** Implemented. v0.7.6. Deployed on rift managing all platform
containers. Two components — `mcp` CLI (operator workstation) and containers. Route declarations with automatic port allocation (`$PORT` /
`$PORT_<NAME>` env vars passed to containers). MC-Proxy route registration
during deploy and stop. Automated TLS cert provisioning for L7 routes via
Metacrypt CA (Phase C). Automated DNS registration in MCNS during deploy
and stop (Phase D). Two components — `mcp` CLI (operator workstation) and
`mcp-agent` (per-node daemon with SQLite registry, rootless Podman, `mcp-agent` (per-node daemon with SQLite registry, rootless Podman,
monitoring with drift/flap detection). gRPC-only (no REST). 12 RPCs, monitoring with drift/flap detection). gRPC-only (no REST). 15 RPCs,
15 CLI commands. 17+ CLI commands.
--- ---
@@ -663,20 +697,22 @@ renew certificates programmatically.
### How Services Get Certificates Today ### How Services Get Certificates Today
Currently, certificates are provisioned through Metacrypt's **REST API or web For services deployed via MCP with L7 routes, certificates are provisioned
UI** and placed into each service's `/srv/<service>/certs/` directory. This is automatically during deploy — MCP uses the Metacrypt ACME client library to
a manual process — the operator issues a certificate, downloads it, and obtain certs and transfers them to the node. For other services and during
deploys the files. The ACME client library exists but is not yet integrated bootstrap, certificates are provisioned through Metacrypt's **REST API or web
into any service. UI** and placed into each service's `/srv/<service>/certs/` directory manually.
### How It Will Work With MCP ### How MCP Automates Certificates
MCP is the natural place to automate certificate provisioning: MCP automates certificate provisioning for deploy workflows, with renewal and
migration automation planned:
- **Initial deploy.** When MCP deploys a new service, it can provision a - **Initial deploy.** When MCP deploys a new service, it provisions a
certificate from Metacrypt (via the ACME client library or the REST API), certificate from Metacrypt (via the ACME client library), transfers the cert
transfer the cert and key to the node as part of the config push to and key to the node as part of the config push to `/srv/<service>/certs/`,
`/srv/<service>/certs/`, and start the service with valid TLS material. and starts the service with valid TLS material. For L7 routes, MCP also
provisions a TLS certificate for MC-Proxy's termination endpoint.
- **Renewal.** MCP knows what services are running and when their certificates - **Renewal.** MCP knows what services are running and when their certificates
expire. It can renew certificates before expiry by re-running the ACME flow expire. It can renew certificates before expiry by re-running the ACME flow
@@ -689,10 +725,8 @@ MCP is the natural place to automate certificate provisioning:
for the new name. for the new name.
- **MC-Proxy L7 routes.** MC-Proxy's L7 mode requires certificate/key pairs - **MC-Proxy L7 routes.** MC-Proxy's L7 mode requires certificate/key pairs
for TLS termination. MCP (or the operator) can provision these from for TLS termination. MCP provisions these from Metacrypt during deploy and
Metacrypt and push them to MC-Proxy's cert directory. MC-Proxy's pushes them to the node alongside the route registration.
architecture doc lists ACME integration and Metacrypt key storage as future
work.
### Trust Distribution ### Trust Distribution
@@ -793,8 +827,13 @@ Operator workstation (vade)
│ │
├── Scheduling: select Node C (best fit) ├── Scheduling: select Node C (best fit)
│ │
├── Provision TLS certificate from Metacrypt ├── Port assignment: allocate a free host port for each
│ (ACME flow or REST API) │ declared route (passed as $PORT / $PORT_<NAME> env vars)
│
├── Provision TLS certificate from Metacrypt CA
│ (ACME client library) for the service
│ — for L7 routes, also provision a cert for MC-Proxy
│ TLS termination
│ │
├── C2 to Node C agent: ├── C2 to Node C agent:
│ 1. Create /srv/α/ directory structure │ 1. Create /srv/α/ directory structure
@@ -802,15 +841,15 @@ Operator workstation (vade)
│ 3. Transfer TLS cert+key → /srv/α/certs/ │ 3. Transfer TLS cert+key → /srv/α/certs/
│ 4. Transfer root CA cert → /srv/α/certs/ca.pem │ 4. Transfer root CA cert → /srv/α/certs/ca.pem
│ 5. Pull image from MCR │ 5. Pull image from MCR
│ 6. Start container │ 6. Start container with $PORT / $PORT_<NAME> env vars
│
├── Register routes with MC-Proxy
│ (gRPC AddRoute for each declared route)
│ │
├── Update service registry: α → Node C ├── Update service registry: α → Node C
│ │
├── Push DNS update to MCNS: └── Push DNS update to MCNS:
│ α.svc.mcp.metacircular.net → Node C address α.svc.mcp.metacircular.net → Node C address
│
└── (Optionally) update MC-Proxy route table
if α needs external ingress
``` ```
### 4. Migration ### 4. Migration

171
docs/packaging-and-deployment.md
View File

@@ -123,18 +123,38 @@ Service definitions are TOML files that tell MCP what to deploy. They
live at `~/.config/mcp/services/<service>.toml` on the operator live at `~/.config/mcp/services/<service>.toml` on the operator
workstation. workstation.
### Minimal Example (Single Component) ### Minimal Example (Single Component, L7)
```toml
name = "myservice"
node = "rift"
[build.images]
myservice = "Dockerfile"
[[components]]
name = "web"
image = "mcr.svc.mcp.metacircular.net:8443/myservice:v1.0.0"
[[components.routes]]
port = 8443
mode = "l7"
```
### API Service Example (L4, Multiple Routes)
```toml ```toml
name = "myservice" name = "myservice"
node = "rift" node = "rift"
version = "v1.0.0"
[build.images] [build.images]
myservice = "Dockerfile" myservice = "Dockerfile"
[[components]] [[components]]
name = "api" name = "api"
image = "mcr.svc.mcp.metacircular.net:8443/myservice:v1.0.0"
volumes = ["/srv/myservice:/srv/myservice"]
cmd = ["server", "--config", "/srv/myservice/myservice.toml"]
[[components.routes]] [[components.routes]]
name = "rest" name = "rest"
@@ -152,7 +172,6 @@ mode = "l4"
```toml ```toml
name = "myservice" name = "myservice"
node = "rift" node = "rift"
version = "v1.0.0"
[build.images] [build.images]
myservice = "Dockerfile.api" myservice = "Dockerfile.api"
@@ -160,6 +179,7 @@ myservice-web = "Dockerfile.web"
[[components]] [[components]]
name = "api" name = "api"
image = "mcr.svc.mcp.metacircular.net:8443/myservice:v1.0.0"
volumes = ["/srv/myservice:/srv/myservice"] volumes = ["/srv/myservice:/srv/myservice"]
cmd = ["server", "--config", "/srv/myservice/myservice.toml"] cmd = ["server", "--config", "/srv/myservice/myservice.toml"]
@@ -175,6 +195,7 @@ mode = "l4"
[[components]] [[components]]
name = "web" name = "web"
image = "mcr.svc.mcp.metacircular.net:8443/myservice-web:v1.0.0"
volumes = ["/srv/myservice:/srv/myservice"] volumes = ["/srv/myservice:/srv/myservice"]
cmd = ["server", "--config", "/srv/myservice/myservice.toml"] cmd = ["server", "--config", "/srv/myservice/myservice.toml"]
@@ -183,21 +204,16 @@ port = 443
mode = "l7" mode = "l7"
``` ```
### Convention-Derived Defaults ### Conventions
Most fields are optional — MCP derives them from conventions: A few fields are derived by the agent at deploy time:
| Field | Default | Override when... | | Field | Default | Override when... |
|-------|---------|------------------| |-------|---------|------------------|
| Image name | `<service>` (api), `<service>-<component>` (others) | Image name differs from convention | | Source path | `<service>` relative to workspace root | Directory name differs from service name (use `path`) |
| Image registry | `mcr.svc.mcp.metacircular.net:8443` (from global MCP config) | Never — always use MCR | | Hostname | `<service>.svc.mcp.metacircular.net` | Service needs a public hostname (use route `hostname`) |
| Version | Service-level `version` field | A component needs a different version |
| Volumes | `/srv/<service>:/srv/<service>` | Additional mounts are needed | All other fields must be explicit in the service definition.
| Network | `mcpnet` | Service needs host networking or a different network |
| User | `0:0` | Never change this for standard services |
| Restart | `unless-stopped` | Service should not auto-restart |
| Source path | `<service>` relative to workspace root | Directory name differs from service name |
| Hostname | `<service>.svc.mcp.metacircular.net` | Service needs a public hostname |
### Service Definition Reference ### Service Definition Reference
@@ -207,7 +223,6 @@ Most fields are optional — MCP derives them from conventions:
|-------|----------|---------| |-------|----------|---------|
| `name` | Yes | Service name (matches project name) | | `name` | Yes | Service name (matches project name) |
| `node` | Yes | Target node to deploy to | | `node` | Yes | Target node to deploy to |
| `version` | Yes | Image version tag (semver, e.g. `v1.0.0`) |
| `active` | No | Whether MCP keeps this running (default: `true`) | | `active` | No | Whether MCP keeps this running (default: `true`) |
| `path` | No | Source directory relative to workspace (default: `name`) | | `path` | No | Source directory relative to workspace (default: `name`) |
@@ -215,20 +230,20 @@ Most fields are optional — MCP derives them from conventions:
| Field | Purpose | | Field | Purpose |
|-------|---------| |-------|---------|
| `build.images.<name>` | Maps image name to Dockerfile path | | `build.images.<name>` | Maps build image name to Dockerfile path. The `<name>` must match the repository name in a component's `image` field (the part after the last `/`, before the `:` tag). |
**Component fields:** **Component fields:**
| Field | Purpose | | Field | Required | Purpose |
|-------|---------| |-------|----------|---------|
| `name` | Component name (e.g. `api`, `web`) | | `name` | Yes | Component name (e.g. `api`, `web`) |
| `image` | Full image reference override | | `image` | Yes | Full image reference (e.g. `mcr.svc.mcp.metacircular.net:8443/myservice:v1.0.0`) |
| `version` | Version override for this component | | `volumes` | No | Volume mounts (list of `host:container` strings) |
| `volumes` | Volume mounts (list of `host:container` strings) | | `cmd` | No | Command override (list of strings) |
| `cmd` | Command override (list of strings) | | `env` | No | Extra environment variables (list of `KEY=VALUE` strings) |
| `network` | Container network override | | `network` | No | Container network (default: none) |
| `user` | Container user override | | `user` | No | Container user (e.g. `0:0`) |
| `restart` | Restart policy override | | `restart` | No | Restart policy (e.g. `unless-stopped`) |
**Route fields (under `[[components.routes]]`):** **Route fields (under `[[components.routes]]`):**
@@ -248,9 +263,11 @@ Most fields are optional — MCP derives them from conventions:
### Version Pinning ### Version Pinning
Service definitions **must** pin an explicit semver tag (e.g. `v1.1.0`). Component `image` fields **must** pin an explicit semver tag (e.g.
Never use `:latest`. This ensures deployments are reproducible and `mcr.svc.mcp.metacircular.net:8443/myservice:v1.1.0`). Never use
`mcp status` shows the actual running version. `:latest`. This ensures deployments are reproducible and `mcp status`
shows the actual running version. The version is extracted from the
image tag.
--- ---
@@ -385,12 +402,17 @@ addresses** — they will be overridden at deploy time.
| Env var | When set | | Env var | When set |
|---------|----------| |---------|----------|
| `$PORT` | Component has a single route | | `$PORT` | Component has a single unnamed route |
| `$PORT_<NAME>` | Component has multiple named routes | | `$PORT_<NAME>` | Component has named routes |
Route names are uppercased: `name = "rest"` → `$PORT_REST`, Route names are uppercased: `name = "rest"` → `$PORT_REST`,
`name = "grpc"` → `$PORT_GRPC`. `name = "grpc"` → `$PORT_GRPC`.
**Container listen address:** Services must bind to `0.0.0.0:$PORT`
(or `:$PORT`), not `localhost:$PORT`. Podman port-forwards go through
the container's network namespace — binding to `localhost` inside the
container makes the port unreachable from outside.
Services built with **mcdsl v1.1.0+** handle this automatically — Services built with **mcdsl v1.1.0+** handle this automatically —
`config.Load` checks `$PORT` → overrides `Server.ListenAddr`, and `config.Load` checks `$PORT` → overrides `Server.ListenAddr`, and
`$PORT_GRPC` → overrides `Server.GRPCAddr`. These take precedence over `$PORT_GRPC` → overrides `Server.GRPCAddr`. These take precedence over
@@ -475,11 +497,14 @@ co-located on the same node).
| `mcp build <service>` | Build and push images to MCR | | `mcp build <service>` | Build and push images to MCR |
| `mcp sync` | Push all service definitions to agents; auto-build missing images | | `mcp sync` | Push all service definitions to agents; auto-build missing images |
| `mcp deploy <service>` | Pull image, (re)create containers, register routes | | `mcp deploy <service>` | Pull image, (re)create containers, register routes |
| `mcp undeploy <service>` | Full teardown: remove routes, DNS, certs, and containers |
| `mcp stop <service>` | Remove routes, stop containers | | `mcp stop <service>` | Remove routes, stop containers |
| `mcp start <service>` | Start previously stopped containers | | `mcp start <service>` | Start previously stopped containers |
| `mcp restart <service>` | Restart containers in place | | `mcp restart <service>` | Restart containers in place |
| `mcp ps` | List all managed containers and status | | `mcp ps` | List all managed containers and status |
| `mcp status [service]` | Detailed status for a specific service | | `mcp status [service]` | Detailed status for a specific service |
| `mcp logs <service>` | Stream container logs |
| `mcp edit <service>` | Edit service definition |
--- ---
@@ -504,13 +529,14 @@ git push origin v1.0.0
cat > ~/.config/mcp/services/myservice.toml << 'EOF' cat > ~/.config/mcp/services/myservice.toml << 'EOF'
name = "myservice" name = "myservice"
node = "rift" node = "rift"
version = "v1.0.0"
[build.images] [build.images]
myservice = "Dockerfile.api" myservice = "Dockerfile.api"
[[components]] [[components]]
name = "api" name = "api"
image = "mcr.svc.mcp.metacircular.net:8443/myservice:v1.0.0"
volumes = ["/srv/myservice:/srv/myservice"]
[[components.routes]] [[components.routes]]
name = "rest" name = "rest"
@@ -584,15 +610,84 @@ Services follow a standard directory structure:
--- ---
## 10. Agent Management
MCP manages a fleet of nodes with heterogeneous operating systems and
architectures. The agent binary lives at `/srv/mcp/mcp-agent` on every
node — this is a mutable path that MCP controls, regardless of whether
the node runs NixOS or Debian.
### Node Configuration
Each node in `~/.config/mcp/mcp.toml` includes SSH and architecture
info for agent management:
```toml
[[nodes]]
name = "rift"
address = "100.95.252.120:9444"
ssh = "rift"
arch = "amd64"
[[nodes]]
name = "hyperborea"
address = "100.x.x.x:9444"
ssh = "hyperborea"
arch = "arm64"
```
### Upgrading Agents
After tagging a new MCP release:
```bash
# Upgrade all nodes (recommended — prevents version skew)
mcp agent upgrade
# Upgrade a single node
mcp agent upgrade rift
# Check versions across the fleet
mcp agent status
```
`mcp agent upgrade` cross-compiles the agent binary for each target
architecture, SSHs to each node, atomically replaces the binary, and
restarts the systemd service. All nodes should be upgraded together
because new CLI versions often depend on new agent RPCs.
### Provisioning New Nodes
One-time setup for a new Debian node:
```bash
# 1. Provision the node (creates user, dirs, systemd unit, installs binary)
mcp node provision <name>
# 2. Register the node
mcp node add <name> <address>
# 3. Deploy services
mcp deploy <service>
```
For NixOS nodes, provisioning is handled by the NixOS configuration.
The NixOS config creates the `mcp` user, systemd unit, and directories.
The `ExecStart` path points to `/srv/mcp/mcp-agent` so that `mcp agent
upgrade` works the same as on Debian nodes.
---
## Appendix: Currently Deployed Services ## Appendix: Currently Deployed Services
For reference, these services are operational on the platform: For reference, these services are operational on the platform:
| Service | Version | Node | Purpose | | Service | Version | Node | Purpose |
|---------|---------|------|---------| |---------|---------|------|---------|
| MCIAS | v1.8.0 | (separate) | Identity and access | | MCIAS | v1.9.0 | (separate) | Identity and access |
| Metacrypt | v1.1.0 | rift | Cryptographic service, PKI/CA | | Metacrypt | v1.3.1 | rift | Cryptographic service, PKI/CA |
| MC-Proxy | v1.1.0 | rift | TLS proxy and router | | MC-Proxy | v1.2.1 | rift | TLS proxy and router |
| MCR | v1.2.0 | rift | Container registry | | MCR | v1.2.1 | rift | Container registry |
| MCNS | v1.1.0 | rift | Authoritative DNS | | MCNS | v1.1.1 | rift | Authoritative DNS |
| MCP | v0.3.0 | rift | Control plane agent | | MCDoc | v0.1.0 | rift | Documentation server |
| MCP | v0.7.6 | rift | Control plane agent |

7
engineering-standards.md
View File

@@ -1018,6 +1018,13 @@ Write these before writing code. They are the blueprint, not the afterthought.
- **Never log secrets.** Keys, passwords, tokens, and plaintext must never - **Never log secrets.** Keys, passwords, tokens, and plaintext must never
appear in log output. appear in log output.
### CLI Security
- **Never echo passwords.** Interactive password prompts must suppress
terminal echo. Use `mcdsl/terminal.ReadPassword` — it wraps
`golang.org/x/term.ReadPassword` with proper prompt and newline handling.
Never read passwords with `bufio.Scanner` or `fmt.Scanln`.
### Web Security ### Web Security
- CSRF tokens on all mutating requests. - CSRF tokens on all mutating requests.