mcr/PROGRESS.md

# MCR Development Progress

Reverse-chronological log of development work. Most recent entries first.
See `PROJECT_PLAN.md` for the implementation roadmap and
`ARCHITECTURE.md` for the full design specification.

## Current State

**Phase:** 7 complete, ready for Phase 9
**Last updated:** 2026-03-19

### Completed

- Phase 0: Project scaffolding (all 4 steps)
- Phase 1: Configuration & database (all 3 steps)
- Phase 2: Blob storage layer (all 2 steps)
- Phase 3: MCIAS authentication (all 4 steps)
- Phase 4: Policy engine (all 4 steps)
- Phase 5: OCI pull path (all 5 steps)
- Phase 6: OCI push path (all 3 steps)
- Phase 7: OCI delete path (all 2 steps)
- Phase 8: Admin REST API (all 5 steps)
- `ARCHITECTURE.md` — Full design specification (18 sections)
- `CLAUDE.md` — AI development guidance
- `PROJECT_PLAN.md` — Implementation plan (14 phases, 40+ steps)
- `PROGRESS.md` — This file

### Next Steps

1. Phase 9 (garbage collection)
2. Phase 10 (gRPC admin API)

---

## Log

### 2026-03-19 — Phase 7: OCI delete path

**Task:** Implement manifest and blob deletion per OCI Distribution Spec.

**Changes:**

Step 7.1 — Manifest delete:
- `db/delete.go`: `DeleteManifest(repoID, digest)` — deletes manifest
  row; ON DELETE CASCADE handles manifest_blobs and tags
- `oci/delete.go`: `handleManifestDelete()` — policy check
  (registry:delete), rejects deletion by tag (405 UNSUPPORTED per OCI
  spec), returns 202 Accepted, writes `manifest_deleted` audit event
- Updated `oci/routes.go` dispatch to handle DELETE on manifests

Step 7.2 — Blob delete:
- `db/delete.go`: `DeleteBlobFromRepo(repoID, digest)` — removes
  manifest_blobs associations only; does NOT delete the blob row or
  file (GC's responsibility, since other repos may reference it)
- `oci/delete.go`: `handleBlobDelete()` — policy check, returns 202,
  writes `blob_deleted` audit event
- Updated `oci/routes.go` dispatch to handle DELETE on blobs
- Extended `DBQuerier` interface with delete methods

**Verification:**
- `make all` passes: vet clean, lint 0 issues, all tests passing,
  all 3 binaries built
- DB delete tests (5 new): delete manifest (verify cascade to tags and
  manifest_blobs, blob row preserved), manifest not found, delete blob
  from repo (manifest_blobs removed, blob row preserved, manifest
  preserved), blob not found, blob exists globally but not in repo
- OCI delete tests (8 new): manifest delete by digest (202), delete by
  tag (405 UNSUPPORTED), manifest not found (404 MANIFEST_UNKNOWN),
  repo not found (404 NAME_UNKNOWN), cascading tag deletion verified,
  blob delete (202), blob not in repo (404 BLOB_UNKNOWN), blob delete
  repo not found

---

### 2026-03-19 — Phase 6: OCI push path

**Task:** Implement blob uploads (monolithic and chunked) and manifest
push per ARCHITECTURE.md §5 and OCI Distribution Spec.

**Changes:**

Step 6.1 — Blob upload initiation:
- `db/upload.go`: `UploadRow` type, `ErrUploadNotFound` sentinel;
  `CreateUpload()`, `GetUpload()`, `UpdateUploadOffset()`, `DeleteUpload()`
- `db/push.go`: `GetOrCreateRepository()` (implicit repo creation on
  first push), `BlobExists()`, `InsertBlob()` (INSERT OR IGNORE for
  content-addressed dedup)
- `oci/upload.go`: `uploadManager` (sync.Mutex-protected map of in-progress
  BlobWriters by UUID); `generateUUID()` via crypto/rand; `handleUploadInitiate()`
  — policy check (registry:push), implicit repo creation, DB row + storage
  temp file, returns 202 with Location/Docker-Upload-UUID/Range headers
- Extended `DBQuerier` interface with push/upload methods
- Changed `BlobOpener` to `BlobStore` adding `StartUpload(uuid)` method

Step 6.2 — Blob upload chunked and monolithic:
- `oci/upload.go`: `handleUploadChunk()` (PATCH — append body, update offset),
  `handleUploadComplete()` (PUT — optional final body, commit with digest
  verification, insert blob row, cleanup upload, audit event),
  `handleUploadStatus()` (GET — 204 with Range header),
  `handleUploadCancel()` (DELETE — cancel BlobWriter, remove upload row)
- `oci/routes.go`: `parseOCIPath()` extended to handle `/blobs/uploads/`
  and `/blobs/uploads/<uuid>` patterns; `dispatchUpload()` routes by method

Step 6.3 — Manifest push:
- `db/push.go`: `PushManifestParams` struct, `PushManifest()` — single
  SQLite transaction: create repo if not exists, upsert manifest (ON
  CONFLICT DO UPDATE), clear and repopulate manifest_blobs join table,
  upsert tag if provided (atomic tag move)
- `oci/manifest.go`: `handleManifestPut()` — full push flow per §5:
  parse JSON, compute SHA-256, verify digest if push-by-digest, collect
  config+layer descriptors, verify all referenced blobs exist (400
  MANIFEST_BLOB_UNKNOWN if missing), call PushManifest(), audit event
  with tag details, returns 201 with Docker-Content-Digest/Location/
  Content-Type headers

**Verification:**
- `make all` passes: vet clean, lint 0 issues, 198 tests passing,
  all 3 binaries built
- DB tests (15 new): upload CRUD (create/get/update/delete/not-found),
  GetOrCreateRepository (new + existing), BlobExists (found/not-found),
  InsertBlob (new + idempotent), PushManifest by tag (verify repo creation,
  manifest, tag, manifest_blobs), by digest (no tag), tag move (atomic
  update), idempotent re-push
- OCI upload tests (8 new): initiate (202, Location/UUID/Range headers,
  implicit repo creation), unique UUIDs (5 initiates → 5 distinct UUIDs),
  monolithic upload (POST → PUT with body → 201), chunked upload
  (POST → PATCH → PATCH → PUT → 201), digest mismatch (400 DIGEST_INVALID),
  upload status (GET → 204), cancel (DELETE → 204), nonexistent UUID
  (PATCH → 404 BLOB_UPLOAD_UNKNOWN)
- OCI manifest push tests (7 new): push by tag (201, correct headers),
  push by digest, digest mismatch (400), missing blob (400
  MANIFEST_BLOB_UNKNOWN), malformed JSON (400 MANIFEST_INVALID), empty
  manifest, tag update (atomic move), re-push idempotent
- Route parsing tests (5 new subtests): upload initiate (trailing slash),
  upload initiate (no trailing slash), upload with UUID, multi-segment
  repo upload, multi-segment repo upload initiate

---

### 2026-03-19 — Batch B: Phase 5 (OCI pull) + Phase 8 (admin REST)

**Task:** Implement the OCI Distribution Spec pull path and the admin
REST API management endpoints. Both phases depend on Phase 4 (policy)
but not on each other — implemented in parallel.

**Changes:**

Phase 5 — `internal/db/` + `internal/oci/` (Steps 5.1–5.5):

Step 5.1 — OCI handler scaffolding:
- `db/errors.go`: `ErrRepoNotFound`, `ErrManifestNotFound`, `ErrBlobNotFound`
  sentinel errors
- `oci/handler.go`: `Handler` struct with `DBQuerier`, `BlobOpener`,
  `PolicyEval`, `AuditFunc` interfaces; `NewHandler()` constructor;
  `checkPolicy()` inline policy check; `audit()` helper
- `oci/ocierror.go`: `writeOCIError()` duplicated from server package
  (15 lines, avoids coupling)
- `oci/routes.go`: `parseOCIPath()` splits from the right to handle
  multi-segment repo names (e.g., `org/team/app/manifests/latest`);
  `Router()` returns chi router with `/_catalog` and `/*` catch-all;
  `dispatch()` routes to manifest/blob/tag handlers

Step 5.2 — Manifest pull:
- `db/repository.go`: `ManifestRow` type, `GetRepositoryByName()`,
  `GetManifestByTag()`, `GetManifestByDigest()`
- `oci/manifest.go`: GET/HEAD `/v2/{name}/manifests/{reference}`;
  resolves by tag or digest; sets Content-Type, Docker-Content-Digest,
  Content-Length headers

Step 5.3 — Blob download:
- `db/repository.go`: `BlobExistsInRepo()` — joins blobs+manifest_blobs+
  manifests to verify blob belongs to repo
- `oci/blob.go`: GET/HEAD `/v2/{name}/blobs/{digest}`; validates blob
  exists in repo before streaming from storage

Step 5.4 — Tag listing:
- `db/repository.go`: `ListTags()` with cursor-based pagination
  (after/limit)
- `oci/tags.go`: GET `/v2/{name}/tags/list` with OCI `?n=`/`?last=`
  pagination and `Link` header for next page

Step 5.5 — Catalog listing:
- `db/repository.go`: `ListRepositoryNames()` with cursor-based pagination
- `oci/catalog.go`: GET `/v2/_catalog` with same pagination pattern

Phase 8 — `internal/db/` + `internal/server/` (Steps 8.1–8.5):

Step 8.1 — Auth endpoints:
- `server/admin_auth.go`: POST `/v1/auth/login` (JSON body → MCIAS),
  POST `/v1/auth/logout` (204 stub), GET `/v1/health` (no auth)

Step 8.2 — Repository management:
- `db/admin.go`: `RepoMetadata`, `TagInfo`, `ManifestInfo`, `RepoDetail`
  types; `ListRepositoriesWithMetadata()`, `GetRepositoryDetail()`,
  `DeleteRepository()`
- `server/admin_repo.go`: GET/DELETE `/v1/repositories` and
  `/v1/repositories/*` (wildcard for multi-segment names)

Step 8.3 — Policy CRUD:
- `db/admin.go`: `PolicyRuleRow` type, `ErrPolicyRuleNotFound`;
  `CreatePolicyRule()`, `GetPolicyRule()`, `ListPolicyRules()`,
  `UpdatePolicyRule()`, `SetPolicyRuleEnabled()`, `DeletePolicyRule()`
- `server/admin_policy.go`: full CRUD on `/v1/policy/rules` and
  `/v1/policy/rules/{id}`; `PolicyReloader` interface; input validation
  (priority >= 1, valid effect/actions); mutations trigger engine reload

Step 8.4 — Audit endpoint:
- `server/admin_audit.go`: GET `/v1/audit` with query parameter filters
  (event_type, actor_id, repository, since, until, n, offset);
  delegates to `db.ListAuditEvents`

Step 8.5 — GC endpoints:
- `server/admin_gc.go`: `GCState` struct with `sync.Mutex`; POST `/v1/gc`
  returns 202 (stub for Phase 9); GET `/v1/gc/status`; concurrent
  trigger returns 409

Shared admin infrastructure:
- `server/admin.go`: `writeAdminError()` (platform `{"error":"..."}`
  format), `writeJSON()`, `RequireAdmin()` middleware, `hasRole()` helper
- `server/admin_routes.go`: `AdminDeps` struct, `MountAdminRoutes()`
  mounts all `/v1/*` endpoints with proper auth/admin middleware layers

**Verification:**
- `make all` passes: vet clean, lint 0 issues, 168 tests passing,
  all 3 binaries built
- Phase 5 (34 new tests): parseOCIPath (14 subtests covering simple/
  multi-segment/edge cases), manifest GET by tag/digest + HEAD + not
  found, blob GET/HEAD + not in repo + repo not found, tags list +
  pagination + empty + repo not found, catalog list + pagination + empty,
  DB repository methods (15 tests covering all 6 query methods)
- Phase 8 (51 new tests): DB admin methods (19 tests covering CRUD,
  pagination, cascade, not-found), admin auth (login ok/fail, health,
  logout), admin repos (list/detail/delete/non-admin 403), admin policy
  (full CRUD cycle, validation errors, non-admin 403, engine reload),
  admin audit (list with filters, pagination), admin GC (trigger 202,
  status, concurrent 409), RequireAdmin middleware (allowed/denied/
  no claims)

---

### 2026-03-19 — Phase 4: Policy engine

**Task:** Implement the registry-specific authorization engine with
priority-based, deny-wins, default-deny evaluation per ARCHITECTURE.md §4.

**Changes:**

Step 4.1 — `internal/policy/` core types and evaluation:
- `policy.go`: `Action` (6 constants), `Effect` (Allow/Deny), `PolicyInput`,
  `Rule` types per ARCHITECTURE.md §4
- `Evaluate(input, rules)` — stateless evaluation: sort by priority (stable),
  collect all matching rules, deny-wins, default-deny
- Rule matching: all populated fields ANDed; empty fields are wildcards;
  `Repositories` glob matching via `path.Match`; empty repo (global ops)
  only matches rules with empty Repositories list

Step 4.2 — `internal/policy/` built-in defaults:
- `defaults.go`: `DefaultRules()` returns 3 built-in rules (negative IDs,
  priority 0): admin wildcard (all actions), human user content access
  (pull/push/delete/catalog), version check (always accessible)

Step 4.3 — `internal/policy/` engine wrapper with DB integration:
- `engine.go`: `Engine` struct with `sync.RWMutex`-protected rule cache;
  `NewEngine()` pre-loaded with defaults; `SetRules()` merges with defaults;
  `Evaluate()` thread-safe evaluation; `Reload(RuleStore)` loads from DB
- `RuleStore` interface: `LoadEnabledPolicyRules() ([]Rule, error)`
- `internal/db/policy.go`: `LoadEnabledPolicyRules()` on `*DB` — loads
  enabled rules from `policy_rules` table, parses `rule_json` JSON column,
  returns `[]policy.Rule` ordered by priority

Step 4.4 — `internal/server/` policy middleware:
- `policy.go`: `PolicyEvaluator` interface, `AuditFunc` callback type,
  `RequirePolicy(evaluator, action, auditFn)` middleware — extracts claims
  from context, repo name from chi URL param, assembles `PolicyInput`,
  returns OCI DENIED (403) on deny with optional audit callback

**Verification:**
- `make all` passes: vet clean, lint 0 issues, 69 tests passing
  (17 policy + 14 server + 15 db + 9 auth + 7 config + 14 storage - some
  overlap from updated packages), all 3 binaries built
- Policy evaluation tests: admin wildcard, user allow, system account deny,
  exact repo match (allow + deny on different repo), glob match
  (production/* matches production/myapp, not production/team/myapp),
  deny-wins over allow, priority ordering, empty repo global operation
  (admin catalog allowed, repo-scoped rule doesn't match), multiple
  matching rules (highest-priority allow returned)
- Default rules tests: admin allowed for all 6 actions, user allowed for
  pull/push/delete/catalog but denied policy:manage, system account denied
  for all except version_check, version_check allowed for both human and
  system accounts
- Engine tests: defaults-only (admin allow, system deny), custom rules
  (matching subject allowed, different subject denied), reload picks up new
  rules (old rules gone), reload with empty store (disabled rules excluded,
  falls back to defaults)
- DB tests: LoadEnabledPolicyRules returns only enabled rules ordered by
  priority, parses rule_json correctly (effect, subject_uuid, actions,
  repositories), empty table returns nil
- Middleware tests: admin allowed, user allowed, system denied (403 with
  OCI DENIED error), system with matching rule allowed, explicit deny
  rule blocks access (403)

---

### 2026-03-19 — Batch A: Phase 2 (blob storage) + Phase 3 (MCIAS auth)

**Task:** Implement content-addressed blob storage and MCIAS authentication
with OCI token endpoint and auth middleware.

**Changes:**

Phase 2 — `internal/storage/` (Steps 2.1 + 2.2):
- `storage.go`: `Store` struct with `layersPath`/`uploadsPath`, `New()`
  constructor, digest validation (`^sha256:[a-f0-9]{64}$`), content-addressed
  path layout: `<layers>/sha256/<first-2-hex>/<full-64-hex>`
- `writer.go`: `BlobWriter` wrapping `*os.File` + `crypto/sha256` running hash
  via `io.MultiWriter`. `StartUpload(uuid)` creates temp file in uploads dir.
  `Write()` updates both file and hash. `Commit(expectedDigest)` finalizes hash,
  verifies digest, `MkdirAll` prefix dir, `Rename` atomically. `Cancel()` cleans
  up temp file. `BytesWritten()` returns offset.
- `reader.go`: `Open(digest)` returns `io.ReadCloser`, `Stat(digest)` returns
  size, `Delete(digest)` removes blob + best-effort prefix dir cleanup,
  `Exists(digest)` returns bool. All validate digest format first.
- `errors.go`: `ErrBlobNotFound`, `ErrDigestMismatch`, `ErrInvalidDigest`
- No new dependencies (stdlib only)

Phase 3 — `internal/auth/` (Steps 3.1) + `internal/server/` (Steps 3.2–3.4):
- `auth/client.go`: `Client` with `NewClient(serverURL, caCert, serviceName,
  tags)`, TLS 1.3 minimum, optional custom CA cert, 10s HTTP timeout.
  `Login()` POSTs to MCIAS `/v1/auth/login`. `ValidateToken()` with SHA-256
  cache keying and 30s TTL.
- `auth/claims.go`: `Claims` struct (Subject, AccountType, Roles) with context
  helpers `ContextWithClaims`/`ClaimsFromContext`
- `auth/cache.go`: `validationCache` with `sync.RWMutex`, lazy eviction,
  injectable `now` function for testing
- `auth/errors.go`: `ErrUnauthorized`, `ErrMCIASUnavailable`
- `server/middleware.go`: `TokenValidator` interface, `RequireAuth` middleware
  (Bearer token extraction, `WWW-Authenticate` header, OCI error format)
- `server/token.go`: `LoginClient` interface, `TokenHandler` (Basic auth →
  bearer token exchange via MCIAS, RFC 3339 `issued_at`)
- `server/v2.go`: `V2Handler` returning 200 `{}`
- `server/routes.go`: `NewRouter` with chi: `/v2/token` (no auth),
  `/v2/` (RequireAuth middleware)
- `server/ocierror.go`: `writeOCIError()` helper for OCI error JSON format
- New dependency: `github.com/go-chi/chi/v5`

**Verification:**
- `make all` passes: vet clean, lint 0 issues, 52 tests passing
  (7 config + 13 db/audit + 14 storage + 9 auth + 9 server), all 3 binaries built
- Storage tests: new store, digest validation (3 valid + 9 invalid), path layout,
  write+commit, digest mismatch rejection (temp cleanup verified), cancel cleanup,
  bytes written tracking, concurrent writes to different UUIDs, open after write,
  stat, exists, delete (verify gone), open not found, invalid digest format
  (covers Open/Stat/Delete/Exists)
- Auth tests: cache put/get, TTL expiry with clock injection, concurrent cache
  access, login success/failure (httptest mock), validate success/revoked,
  cache hit (request counter), cache expiry (clock advance)
- Server tests: RequireAuth valid/missing/invalid token, token handler
  success/invalid creds/missing auth, routes integration (authenticated /v2/,
  unauthenticated /v2/ → 401, token endpoint bypasses auth)

---

### 2026-03-19 — Phase 1: Configuration & database

**Task:** Implement TOML config loading with env overrides and validation,
SQLite database with migrations, and audit log helpers.

**Changes:**

Step 1.1 — `internal/config/`:
- `config.go`: `Config` struct matching ARCHITECTURE.md §10 (all 6 TOML
  sections: server, database, storage, mcias, web, log)
- Parsed with `go-toml/v2`; env overrides via `MCR_` prefix using
  reflection-based struct walker
- Startup validation: 6 required fields checked (listen_addr, tls_cert,
  tls_key, database.path, storage.layers_path, mcias.server_url)
- Same-filesystem check for layers_path/uploads_path via device ID
  comparison (walks to nearest existing parent if path doesn't exist yet)
- Default values: read_timeout=30s, write_timeout=0, idle_timeout=120s,
  shutdown_timeout=60s, uploads_path derived from layers_path, log.level=info
- `device_linux.go`: Linux-specific `extractDeviceID` using `syscall.Stat_t`
- `deploy/examples/mcr.toml`: annotated example config

Step 1.2 — `internal/db/`:
- `db.go`: `Open(path)` creates/opens SQLite via `modernc.org/sqlite`,
  sets pragmas (WAL, foreign_keys, busy_timeout=5000), chmod 0600
- `migrate.go`: migration framework with `schema_migrations` tracking table;
  `Migrate()` applies pending migrations in transactions; `SchemaVersion()`
  reports current version
- Migration 000001: `repositories`, `manifests`, `tags`, `blobs`,
  `manifest_blobs`, `uploads` — all tables, constraints, and indexes per
  ARCHITECTURE.md §8
- Migration 000002: `policy_rules`, `audit_log` — tables and indexes per §8

Step 1.3 — `internal/db/`:
- `audit.go`: `WriteAuditEvent(eventType, actorID, repository, digest, ip,
  details)` with JSON-serialized details map; `ListAuditEvents(AuditFilter)`
  with filtering by event_type, actor_id, repository, time range, and
  offset/limit pagination (default 50, descending by event_time)
- `AuditFilter` struct with all filter fields
- `AuditEvent` struct with JSON tags for API serialization

Lint fix:
- `.golangci.yaml`: disabled `fieldalignment` analyzer in govet (micro-
  optimization that hurts struct readability; not a security/correctness
  concern)

**Verification:**
- `make all` passes: vet clean, lint 0 issues, 20 tests passing
  (7 config + 13 db/audit), all 3 binaries built
- Config tests: valid load, defaults applied, uploads_path default,
  5 missing-required-field cases, env override (string + duration),
  same-filesystem check
- DB tests: open+migrate, idempotent migrate, 9 tables verified,
  foreign key enforcement, tag cascade on manifest delete,
  manifest_blobs cascade (blob row preserved), WAL mode verified
- Audit tests: write+list, filter by type, filter by actor, filter by
  repository, pagination (3 pages), null fields handled

---

### 2026-03-19 — Phase 0: Project scaffolding

**Task:** Set up Go module, build system, linter config, and binary
entry points with cobra subcommands.

**Changes:**
- `go.mod`: module `git.wntrmute.dev/kyle/mcr`, Go 1.25, cobra dependency
- Directory skeleton: `cmd/mcrsrv/`, `cmd/mcr-web/`, `cmd/mcrctl/`,
  `internal/`, `proto/mcr/v1/`, `gen/mcr/v1/`, `web/templates/`,
  `web/static/`, `deploy/docker/`, `deploy/examples/`, `deploy/scripts/`,
  `deploy/systemd/`, `docs/`
- `.gitignore`: binaries, `srv/`, `*.db*`, IDE/OS files
- `Makefile`: standard targets (`all`, `build`, `test`, `vet`, `lint`,
  `proto`, `proto-lint`, `clean`, `docker`, `devserver`); `all` runs
  `vet → lint → test → mcrsrv mcr-web mcrctl`; `CGO_ENABLED=0` on binary
  builds; version injection via `-X main.version`
- `.golangci.yaml`: golangci-lint v2 config matching mc-proxy conventions;
  linters: errcheck, govet, ineffassign, unused, errorlint, gosec,
  staticcheck, revive; formatters: gofmt, goimports; gosec G101 excluded
  in test files
- `buf.yaml`: protobuf linting (STANDARD) and breaking change detection (FILE)
- `cmd/mcrsrv/main.go`: root command with `server`, `init`, `snapshot`
  subcommands (stubs returning "not implemented")
- `cmd/mcr-web/main.go`: root command with `server` subcommand (stub)
- `cmd/mcrctl/main.go`: root command with `status`, `repo` (list/delete),
  `gc` (trigger/status), `policy` (list/create/update/delete),
  `audit` (tail), `snapshot` subcommands (stubs)
- All binaries accept `--version` flag

**Verification:**
- `make all` passes: vet clean, lint 0 issues, test (no test files),
  all three binaries built successfully
- `./mcrsrv --version` → `mcrsrv version 3695581`
- `./mcr-web --version` → `mcr-web version 3695581`
- All stubs return "not implemented" error as expected
- `make clean` removes binaries

---

### 2026-03-19 — Project planning

**Task:** Create design documents and implementation plan.

**Changes:**
- `README.md`: Existing one-line description
- `ARCHITECTURE.md`: Full design specification covering OCI Distribution
  Spec compliance, MCIAS authentication, policy engine, storage design,
  API surface (OCI + admin REST + gRPC), database schema, garbage collection,
  configuration, web UI, CLI tools, deployment, security model
- `CLAUDE.md`: Development guidance for AI-assisted implementation
- `PROJECT_PLAN.md`: 14-phase implementation plan with discrete steps,
  acceptance criteria, dependency graph, and batchable work identification
- `PROGRESS.md`: This progress tracker

**Notes:**
- No code written yet. All files are documentation/planning.
- ARCHITECTURE.md reviewed and corrected for: GC algorithm crash safety,
  policy glob semantics, tag FK cascade, OCI error format, API sync
  violations, timeout configuration, backup considerations, and other
  consistency issues.