mcp/PROJECT_PLAN_V1.md

# MCP v1 Project Plan

## Overview

This plan breaks MCP v1 into discrete implementation tasks organized into
phases. Tasks within a phase can often be parallelized. Dependencies
between tasks are noted explicitly.

The critical path is: proto → registry + runtime → agent deploy handler →
integration testing. Parallelizable work (CLI commands, monitoring, file
transfer) can proceed alongside the critical path once the proto and core
libraries are ready.

## Notation

- **[Pn]** = Phase n
- **[Pn.m]** = Task m in phase n
- **depends: [Px.y]** = must wait for that task
- **parallel: [Px.y, Px.z]** = can run alongside these tasks
- Each task includes scope, deliverables, and test criteria

---

## Phase 0: Project Scaffolding

One engineer. Serial. Establishes the project skeleton that everything
else builds on.

### P0.1: Repository and module setup

**Scope:** Initialize the Go module, create the standard directory
structure, and configure tooling.

**Deliverables:**
- `go.mod` with module path `git.wntrmute.dev/kyle/mcp`
- `Makefile` with standard targets (build, test, vet, lint, proto,
  proto-lint, clean, all)
- `.golangci.yaml` with platform-standard linter config
- `.gitignore`
- `CLAUDE.md` (project-specific AI context)
- Empty `cmd/mcp/main.go` and `cmd/mcp-agent/main.go` (compile to
  verify the skeleton works)

**Test criteria:** `make build` succeeds. `make vet` and `make lint` pass
on the empty project.

### P0.2: Proto definitions and code generation

**Scope:** Write the `mcp.proto` file from the ARCHITECTURE.md spec and
generate Go code.

**Depends:** P0.1

**Deliverables:**
- `proto/mcp/v1/mcp.proto` — full service definition from ARCHITECTURE.md
- `buf.yaml` configuration
- `gen/mcp/v1/` — generated Go code
- `make proto` and `make proto-lint` both pass

**Test criteria:** Generated code compiles. `buf lint` passes. All message
types and RPC methods from the architecture doc are present.

---

## Phase 1: Core Libraries

Four independent packages. **All can be built in parallel** once P0.2 is
complete. Each package has a well-defined interface, no dependencies on
other Phase 1 packages, and is fully testable in isolation.

### P1.1: Registry package (`internal/registry/`)

**Scope:** SQLite schema, migrations, and CRUD operations for the
node-local registry.

**Depends:** P0.1

**Deliverables:**
- `db.go` — open database, run migrations, close. Schema from
  ARCHITECTURE.md (services, components, component_ports,
  component_volumes, component_cmd, events tables).
- `services.go` — create, get, list, update, delete services.
- `components.go` — create, get, list (by service), update desired/observed
  state, update spec, delete. Support filtering by desired_state.
- `events.go` — insert event, query events by component+service+time
  range, count events in window (for flap detection), prune old events.

**Test criteria:** Full test coverage using `t.TempDir()` + real SQLite.
Tests cover:
- Schema migration is idempotent
- Service and component CRUD
- Desired/observed state updates
- Event insertion, time-range queries, pruning
- Foreign key cascading (delete service → components deleted)
- Component composite primary key (service, name) enforced

**parallel:** P1.2, P1.3, P1.4

### P1.2: Runtime package (`internal/runtime/`)

**Scope:** Container runtime abstraction with a podman implementation.

**Depends:** P0.1

**Deliverables:**
- `runtime.go` — `Runtime` interface:
  ```go
  type Runtime interface {
      Pull(ctx context.Context, image string) error
      Run(ctx context.Context, spec ContainerSpec) error
      Stop(ctx context.Context, name string) error
      Remove(ctx context.Context, name string) error
      Inspect(ctx context.Context, name string) (ContainerInfo, error)
      List(ctx context.Context) ([]ContainerInfo, error)
  }
  ```
  Plus `ContainerSpec` and `ContainerInfo` structs.
- `podman.go` — podman implementation. Builds command-line arguments from
  `ContainerSpec`, execs `podman` CLI, parses `podman inspect` JSON output.

**Test criteria:**
- Unit tests for command-line argument building (given a ContainerSpec,
  verify the constructed podman args are correct). These don't require
  podman to be installed.
- `ContainerSpec` → podman flag mapping matches the table in ARCHITECTURE.md.
- Container naming follows `<service>-<component>` convention.
- Version extraction from image tag works (e.g., `registry/img:v1.2.0`
  → `v1.2.0`, `registry/img:latest` → `latest`, `registry/img` → `""`).

**parallel:** P1.1, P1.3, P1.4

### P1.3: Service definition package (`internal/servicedef/`)

**Scope:** Parse, validate, and write TOML service definition files.

**Depends:** P0.1

**Deliverables:**
- `servicedef.go` — `Load(path) → ServiceDef`, `Write(path, ServiceDef)`,
  `LoadAll(dir) → []ServiceDef`. Validation: required fields (name, node,
  at least one component), component names unique within service. Converts
  between TOML representation and proto `ServiceSpec`.

**Test criteria:**
- Round-trip: write a ServiceDef, read it back, verify equality
- Validation rejects missing name, missing node, empty components,
  duplicate component names
- `LoadAll` loads all `.toml` files from a directory
- `active` field defaults to `true` if omitted
- Conversion to/from proto `ServiceSpec` is correct

**parallel:** P1.1, P1.2, P1.4

### P1.4: Config package (`internal/config/`)

**Scope:** Load and validate CLI and agent configuration from TOML files.

**Depends:** P0.1

**Deliverables:**
- `cli.go` — CLI config struct: services dir, MCIAS settings, auth
  (token path, optional username/password_file), nodes list. Load from
  TOML with env var overrides (`MCP_*`). Validate required fields.
- `agent.go` — Agent config struct: server (grpc_addr, tls_cert, tls_key),
  database path, MCIAS settings, agent (node_name, container_runtime),
  monitor settings, log level. Load from TOML with env var overrides
  (`MCP_AGENT_*`). Validate required fields.

**Test criteria:**
- Load from TOML file, verify all fields populated
- Required field validation (reject missing grpc_addr, missing tls_cert, etc.)
- Env var overrides work
- Nodes list parses correctly from `[[nodes]]`

**parallel:** P1.1, P1.2, P1.3

### P1.5: Auth package (`internal/auth/`)

**Scope:** MCIAS token validation for the agent, and token acquisition for
the CLI.

**Depends:** P0.1, P0.2 (uses proto-generated types for gRPC interceptor)

**Deliverables:**
- `auth.go`:
  - `Interceptor` — gRPC unary server interceptor that extracts bearer
    tokens, validates against MCIAS (with 30s SHA-256-keyed cache),
    checks admin role, audit-logs every RPC (method, caller, timestamp).
    Returns UNAUTHENTICATED or PERMISSION_DENIED on failure.
  - `Login(url, username, password) → token` — authenticate to MCIAS,
    return bearer token.
  - `LoadToken(path) → token` — read cached token from file.
  - `SaveToken(path, token)` — write token to file with 0600 permissions.

**Test criteria:**
- Interceptor rejects missing token (UNAUTHENTICATED)
- Interceptor rejects invalid token (UNAUTHENTICATED)
- Interceptor rejects non-admin token (PERMISSION_DENIED)
- Token caching works (same token within 30s returns cached result)
- Token file read/write with correct permissions
- Audit log entry emitted on every RPC (check slog output)

**Note:** Full interceptor testing requires an MCIAS mock or test instance.
Unit tests can mock the MCIAS validation call. Integration tests against a
real MCIAS instance are a Phase 4 concern.

**parallel:** P1.1, P1.2, P1.3, P1.4 (partially; needs P0.2 for proto types)

---

## Phase 2: Agent

The agent is the core of MCP. Tasks in this phase build on Phase 1
libraries. Some tasks can be parallelized; dependencies are noted.

### P2.1: Agent skeleton and gRPC server

**Scope:** Wire up the agent binary: config loading, database setup, gRPC
server with TLS and auth interceptor, graceful shutdown.

**Depends:** P0.2, P1.1, P1.4, P1.5

**Deliverables:**
- `cmd/mcp-agent/main.go` — cobra root command, `server` subcommand
- `internal/agent/agent.go` — Agent struct holding registry, runtime,
  config. Initializes database, starts gRPC server with TLS and auth
  interceptor, handles SIGINT/SIGTERM for graceful shutdown.
- Agent starts, listens on configured address, rejects unauthenticated
  RPCs, shuts down cleanly.

**Test criteria:** Agent starts with a test config, accepts TLS
connections, rejects RPCs without a valid token. Graceful shutdown
closes the database and stops the listener.

### P2.2: Deploy handler

**Scope:** Implement the `Deploy` RPC on the agent.

**Depends:** P2.1, P1.2

**Deliverables:**
- `internal/agent/deploy.go` — handles DeployRequest: records spec in
  registry, iterates components, calls runtime (pull, stop, remove, run,
  inspect), updates observed state and version, returns results.
- Supports single-component deploy (when `component` field is set).

**Test criteria:**
- Deploy with all components records spec in registry
- Deploy with single component only touches that component
- Failed pull returns error for that component, others continue
- Registry is updated with desired_state=running and observed_state
- Version is extracted from image tag

### P2.3: Lifecycle handlers (stop, start, restart)

**Scope:** Implement `StopService`, `StartService`, `RestartService` RPCs.

**Depends:** P2.1, P1.2

**parallel:** P2.2

**Deliverables:**
- `internal/agent/lifecycle.go`
- Stop: for each component, call runtime stop, update desired_state to
  `stopped`, update observed_state.
- Start: for each component, call runtime start (or run if removed),
  update desired_state to `running`, update observed_state.
- Restart: stop then start each component.

**Test criteria:**
- Stop sets desired_state=stopped, calls runtime stop
- Start sets desired_state=running, calls runtime start
- Restart cycles each component
- Returns per-component results

### P2.4: Status handlers (list, live check, get status)

**Scope:** Implement `ListServices`, `LiveCheck`, `GetServiceStatus` RPCs.

**Depends:** P2.1, P1.2

**parallel:** P2.2, P2.3

**Deliverables:**
- `internal/agent/status.go`
- `ListServices`: read from registry, no runtime query.
- `LiveCheck`: query runtime, reconcile registry, return updated state.
- `GetServiceStatus`: live check + drift detection + recent events.

**Test criteria:**
- ListServices returns registry contents without touching runtime
- LiveCheck updates observed_state from runtime
- GetServiceStatus includes drift info for mismatched desired/observed
- GetServiceStatus includes recent events

### P2.5: Sync handler

**Scope:** Implement `SyncDesiredState` RPC.

**Depends:** P2.1, P1.2

**parallel:** P2.2, P2.3, P2.4

**Deliverables:**
- `internal/agent/sync.go`
- Receives list of ServiceSpecs from CLI.
- For each service: create or update in registry, set desired_state based
  on `active` flag (running if active, stopped if not).
- Runs reconciliation (discover unmanaged containers, set to ignore).
- Returns per-service summary of what changed.

**Test criteria:**
- New services are created in registry
- Existing services have specs updated
- Active=false sets desired_state=stopped for all components
- Unmanaged containers discovered and set to ignore
- Returns accurate change summaries

### P2.6: File transfer handlers

**Scope:** Implement `PushFile` and `PullFile` RPCs.

**Depends:** P2.1

**parallel:** P2.2, P2.3, P2.4, P2.5

**Deliverables:**
- `internal/agent/files.go`
- Path validation: resolve `/srv/<service>/<path>`, reject `..` traversal,
  reject symlinks escaping the service directory.
- Push: atomic write (temp file + rename), create intermediate dirs.
- Pull: read file, return content and permissions.

**Test criteria:**
- Push creates file at correct path with correct permissions
- Push creates intermediate directories
- Push is atomic (partial write doesn't leave corrupt file)
- Pull returns file content and mode
- Path traversal rejected (`../etc/passwd`)
- Symlink escape rejected
- Service directory scoping enforced

### P2.7: Adopt handler

**Scope:** Implement `AdoptContainer` RPC.

**Depends:** P2.1, P1.2

**parallel:** P2.2, P2.3, P2.4, P2.5, P2.6

**Deliverables:**
- `internal/agent/adopt.go`
- Matches containers by `<service>-*` prefix in runtime.
- Creates service if needed.
- Strips prefix to derive component name.
- Sets desired_state based on current observed_state.
- Returns per-container results.

**Test criteria:**
- Matches containers by prefix
- Creates service when it doesn't exist
- Derives component names correctly (metacrypt-api → api, metacrypt-web → web)
- Single-component service (mc-proxy → mc-proxy) works
- Sets desired_state to running for running containers, stopped for stopped
- Returns results for each adopted container

### P2.8: Monitor subsystem

**Scope:** Implement the continuous monitoring loop and alerting.

**Depends:** P2.1, P1.1, P1.2

**parallel:** P2.2-P2.7 (can be built alongside other agent handlers)

**Deliverables:**
- `internal/monitor/monitor.go` — Monitor struct, Start/Stop methods.
  Runs a goroutine with a ticker at the configured interval. Each tick:
  queries runtime, reconciles registry, records events, evaluates alerts.
- `internal/monitor/alerting.go` — Alert evaluation: drift detection
  (desired != observed for managed components), flap detection (event
  count in window > threshold), cooldown tracking per component, alert
  command execution via `exec` (argv array, MCP_* env vars).
- Event pruning (delete events older than retention period).

**Test criteria:**
- Monitor detects state transitions and records events
- Drift alert fires on desired/observed mismatch
- Drift alert respects cooldown (doesn't fire again within window)
- Flap alert fires when transitions exceed threshold in window
- Alert command is exec'd with correct env vars
- Event pruning removes old events, retains recent ones
- Monitor can be stopped cleanly (goroutine exits)

### P2.9: Snapshot command

**Scope:** Implement `mcp-agent snapshot` for database backup.

**Depends:** P2.1, P1.1

**parallel:** P2.2-P2.8

**Deliverables:**
- `cmd/mcp-agent/snapshot.go` — cobra subcommand. Runs `VACUUM INTO`
  to create a consistent backup in `/srv/mcp/backups/`.

**Test criteria:**
- Creates a backup file with timestamp in name
- Backup is a valid SQLite database
- Original database is unchanged

---

## Phase 3: CLI

All CLI commands are thin gRPC clients. **Most can be built in parallel**
once the proto (P0.2) and servicedef/config packages (P1.3, P1.4) are
ready. CLI commands can be tested against a running agent (integration)
or with a mock gRPC server (unit).

### P3.1: CLI skeleton

**Scope:** Wire up the CLI binary: config loading, gRPC connection setup,
cobra command tree.

**Depends:** P0.2, P1.3, P1.4

**Deliverables:**
- `cmd/mcp/main.go` — cobra root command with `--config` flag.
  Subcommand stubs for all commands.
- gRPC dial helper: reads node address from config, establishes TLS
  connection with CA verification, attaches bearer token to metadata.

**Test criteria:** CLI starts, loads config, `--help` shows all
subcommands.

### P3.2: Login command

**Scope:** Implement `mcp login`.

**Depends:** P3.1, P1.5

**Deliverables:**
- `cmd/mcp/login.go` — prompts for username/password (or reads from
  config for unattended), calls MCIAS, saves token to configured path
  with 0600 permissions.

**Test criteria:** Token is saved to the correct path with correct
permissions.

### P3.3: Deploy command

**Scope:** Implement `mcp deploy`.

**Depends:** P3.1, P1.3

**parallel:** P3.4, P3.5, P3.6, P3.7, P3.8, P3.9, P3.10

**Deliverables:**
- `cmd/mcp/deploy.go`
- Resolves service spec: file (from `-f` or default path) > agent registry.
- Parses `<service>/<component>` syntax for single-component deploy.
- Pushes spec to agent via Deploy RPC.
- Prints per-component results.

**Test criteria:**
- Reads service definition from file
- Falls back to agent registry when no file exists
- Fails with clear error when neither exists
- Single-component syntax works
- Prints results

### P3.4: Lifecycle commands (stop, start, restart)

**Scope:** Implement `mcp stop`, `mcp start`, `mcp restart`.

**Depends:** P3.1, P1.3

**parallel:** P3.3, P3.5, P3.6, P3.7, P3.8, P3.9, P3.10

**Deliverables:**
- `cmd/mcp/lifecycle.go`
- Stop: sets `active = false` in service definition file, calls
  StopService RPC.
- Start: sets `active = true` in service definition file, calls
  StartService RPC.
- Restart: calls RestartService RPC (does not change active flag).

**Test criteria:**
- Stop updates the service definition file
- Start updates the service definition file
- Both call the correct RPC
- Restart does not modify the file

### P3.5: Status commands (list, ps, status)

**Scope:** Implement `mcp list`, `mcp ps`, `mcp status`.

**Depends:** P3.1

**parallel:** P3.3, P3.4, P3.6, P3.7, P3.8, P3.9, P3.10

**Deliverables:**
- `cmd/mcp/status.go`
- List: calls ListServices on all nodes, formats table output.
- Ps: calls LiveCheck on all nodes, formats with uptime and version.
- Status: calls GetServiceStatus, shows drift and recent events.

**Test criteria:**
- Queries all registered nodes
- Formats output as readable tables
- Status highlights drift clearly

### P3.6: Sync command

**Scope:** Implement `mcp sync`.

**Depends:** P3.1, P1.3

**parallel:** P3.3, P3.4, P3.5, P3.7, P3.8, P3.9, P3.10

**Deliverables:**
- `cmd/mcp/sync.go`
- Loads all service definitions from the services directory.
- Groups by node.
- Calls SyncDesiredState on each agent with that node's services.
- Prints summary of changes.

**Test criteria:**
- Loads all service definitions
- Filters by node correctly
- Pushes to correct agents
- Prints change summary

### P3.7: Adopt command

**Scope:** Implement `mcp adopt`.

**Depends:** P3.1

**parallel:** P3.3, P3.4, P3.5, P3.6, P3.8, P3.9, P3.10

**Deliverables:**
- `cmd/mcp/adopt.go`
- Calls AdoptContainer RPC on the agent.
- Prints adopted containers and their derived component names.

**Test criteria:**
- Calls RPC with service name
- Prints results

### P3.8: Service commands (show, edit, export)

**Scope:** Implement `mcp service show`, `mcp service edit`,
`mcp service export`.

**Depends:** P3.1, P1.3

**parallel:** P3.3, P3.4, P3.5, P3.6, P3.7, P3.9, P3.10

**Deliverables:**
- `cmd/mcp/service.go`
- Show: calls ListServices, filters to named service, prints spec.
- Edit: if file exists, open in $EDITOR. If not, export from agent
  first, then open. Save to standard path.
- Export: calls ListServices, converts to TOML, writes to file (default
  path or `-f`).

**Test criteria:**
- Show prints the correct spec
- Export writes a valid TOML file that can be loaded back
- Edit opens the correct file (or creates from agent spec)

### P3.9: Transfer commands (push, pull)

**Scope:** Implement `mcp push` and `mcp pull`.

**Depends:** P3.1

**parallel:** P3.3, P3.4, P3.5, P3.6, P3.7, P3.8, P3.10

**Deliverables:**
- `cmd/mcp/transfer.go`
- Push: reads local file, determines service and path, calls PushFile RPC.
  Default relative path = basename of local file.
- Pull: calls PullFile RPC, writes content to local file.

**Test criteria:**
- Push reads file and sends correct content
- Push derives path from basename when omitted
- Pull writes file locally with correct content

### P3.10: Node commands

**Scope:** Implement `mcp node list`, `mcp node add`, `mcp node remove`.

**Depends:** P3.1, P1.4

**parallel:** P3.3, P3.4, P3.5, P3.6, P3.7, P3.8, P3.9

**Deliverables:**
- `cmd/mcp/node.go`
- List: reads nodes from config, prints table.
- Add: appends a `[[nodes]]` entry to the config file.
- Remove: removes the named `[[nodes]]` entry from the config file.

**Test criteria:**
- List shows all configured nodes
- Add creates a new entry
- Remove deletes the named entry
- Config file remains valid TOML after add/remove

---

## Phase 4: Deployment Artifacts

Can be worked on in parallel with Phase 2 and 3.

### P4.1: Systemd units

**Scope:** Write systemd service and timer files.

**Depends:** None (these are static files)

**parallel:** All of Phase 2 and 3

**Deliverables:**
- `deploy/systemd/mcp-agent.service` — from ARCHITECTURE.md
- `deploy/systemd/mcp-agent-backup.service` — snapshot oneshot
- `deploy/systemd/mcp-agent-backup.timer` — daily 02:00 UTC, 5min jitter

**Test criteria:** Files match platform conventions (security hardening,
correct paths, correct user).

### P4.2: Example configs

**Scope:** Write example configuration files.

**Depends:** None

**parallel:** All of Phase 2 and 3

**Deliverables:**
- `deploy/examples/mcp.toml` — CLI config with all fields documented
- `deploy/examples/mcp-agent.toml` — agent config with all fields documented

**Test criteria:** Examples are valid TOML, loadable by the config package.

### P4.3: Install script

**Scope:** Write the agent install script.

**Depends:** None

**parallel:** All of Phase 2 and 3

**Deliverables:**
- `deploy/scripts/install-agent.sh` — idempotent: create user/group,
  install binary, create /srv/mcp/, install example config, install
  systemd units, reload daemon.

**Test criteria:** Script is idempotent (running twice produces the same
result).

---

## Phase 5: Integration Testing and Polish

Serial. Requires all previous phases to be complete.

### P5.1: Integration test suite

**Scope:** End-to-end tests: CLI → agent → podman → container lifecycle.

**Depends:** All of Phase 2 and 3

**Deliverables:**
- Test harness that starts an agent with a test config and temp database.
- Tests cover: deploy, stop, start, restart, sync, adopt, push/pull,
  list/ps/status.
- Tests verify registry state, runtime state, and CLI output.

**Test criteria:** All integration tests pass. Coverage of every CLI
command and agent RPC.

### P5.2: Bootstrap procedure test

**Scope:** Test the full MCP bootstrap on a clean node with existing
containers.

**Depends:** P5.1

**Deliverables:**
- Documented test procedure: start agent, sync (discover containers),
  adopt, export, verify service definitions match running state.
- Verify the container rename flow (bare names → <service>-<component>).

### P5.3: Documentation

**Scope:** Final docs pass.

**Depends:** P5.1

**Deliverables:**
- `CLAUDE.md` updated with final project structure and commands
- `README.md` with quick-start
- `RUNBOOK.md` with operational procedures
- Verify ARCHITECTURE.md matches implementation

---

## Parallelism Summary

```
Phase 0 (serial):  P0.1 → P0.2
                          │
                          ▼
Phase 1 (parallel): ┌─── P1.1 (registry)
                    ├─── P1.2 (runtime)
                    ├─── P1.3 (servicedef)
                    ├─── P1.4 (config)
                    └─── P1.5 (auth)
                          │
                    ┌─────┴──────┐
                    ▼            ▼
Phase 2 (agent):  P2.1 ──┐   Phase 3 (CLI):  P3.1 ──┐
                    │     │                     │     │
                    ▼     │                     ▼     │
                  P2.2  P2.3    Phase 4:      P3.2  P3.3
                  P2.4  P2.5    P4.1-P4.3     P3.4  P3.5
                  P2.6  P2.7    (parallel     P3.6  P3.7
                  P2.8  P2.9     with 2&3)    P3.8  P3.9
                    │                           │   P3.10
                    └──────────┬────────────────┘
                               ▼
Phase 5 (serial):  P5.1 → P5.2 → P5.3
```

Maximum parallelism: 5 engineers/agents during Phase 1, up to 8+
during Phase 2+3+4 combined.

Minimum serial path: P0.1 → P0.2 → P1.1 → P2.1 → P2.2 → P5.1 → P5.3