Phases 5, 6, 8: OCI pull/push paths and admin REST API

Phase 5 (OCI pull): internal/oci/ package with manifest GET/HEAD by
tag/digest, blob GET/HEAD with repo membership check, tag listing with
OCI pagination, catalog listing. Multi-segment repo names via
parseOCIPath() right-split routing. DB query layer in
internal/db/repository.go.

Phase 6 (OCI push): blob uploads (monolithic and chunked) with
uploadManager tracking in-progress BlobWriters, manifest push
implementing full ARCHITECTURE.md §5 flow in a single SQLite
transaction (create repo, upsert manifest, populate manifest_blobs,
atomic tag move). Digest verification on both blob commit and manifest
push-by-digest.

Phase 8 (admin REST): /v1 endpoints for auth (login/logout/health),
repository management (list/detail/delete), policy CRUD with engine
reload, audit log listing with filters, GC trigger/status stubs.
RequireAdmin middleware, platform-standard error format.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-03-19 18:25:18 -07:00
parent f5e67bd4aa
commit dddc66f31b
40 changed files with 6832 additions and 7 deletions

View File

@@ -6,7 +6,7 @@ See `PROJECT_PLAN.md` for the implementation roadmap and
## Current State
**Phase:** 4 complete, ready for Batch B (Phase 5 + Phase 8)
**Phase:** 6 complete, ready for Phase 7
**Last updated:** 2026-03-19
### Completed
@@ -16,6 +16,9 @@ See `PROJECT_PLAN.md` for the implementation roadmap and
- Phase 2: Blob storage layer (all 2 steps)
- Phase 3: MCIAS authentication (all 4 steps)
- Phase 4: Policy engine (all 4 steps)
- Phase 5: OCI pull path (all 5 steps)
- Phase 6: OCI push path (all 3 steps)
- Phase 8: Admin REST API (all 5 steps)
- `ARCHITECTURE.md` — Full design specification (18 sections)
- `CLAUDE.md` — AI development guidance
- `PROJECT_PLAN.md` — Implementation plan (14 phases, 40+ steps)
@@ -23,14 +26,180 @@ See `PROJECT_PLAN.md` for the implementation roadmap and
### Next Steps
1. Batch B: Phase 5 (OCI pull) and Phase 8 (admin REST) — independent,
can be done in parallel
2. After Phase 5, Phase 6 (OCI push) then Phase 7 (OCI delete)
1. Phase 7 (OCI delete)
2. After Phase 7, Phase 9 (garbage collection)
3. Phase 10 (gRPC admin API)
---
## Log
### 2026-03-19 — Phase 6: OCI push path
**Task:** Implement blob uploads (monolithic and chunked) and manifest
push per ARCHITECTURE.md §5 and OCI Distribution Spec.
**Changes:**
Step 6.1 — Blob upload initiation:
- `db/upload.go`: `UploadRow` type, `ErrUploadNotFound` sentinel;
`CreateUpload()`, `GetUpload()`, `UpdateUploadOffset()`, `DeleteUpload()`
- `db/push.go`: `GetOrCreateRepository()` (implicit repo creation on
first push), `BlobExists()`, `InsertBlob()` (INSERT OR IGNORE for
content-addressed dedup)
- `oci/upload.go`: `uploadManager` (sync.Mutex-protected map of in-progress
BlobWriters by UUID); `generateUUID()` via crypto/rand; `handleUploadInitiate()`
— policy check (registry:push), implicit repo creation, DB row + storage
temp file, returns 202 with Location/Docker-Upload-UUID/Range headers
- Extended `DBQuerier` interface with push/upload methods
- Changed `BlobOpener` to `BlobStore` adding `StartUpload(uuid)` method
Step 6.2 — Blob upload chunked and monolithic:
- `oci/upload.go`: `handleUploadChunk()` (PATCH — append body, update offset),
`handleUploadComplete()` (PUT — optional final body, commit with digest
verification, insert blob row, cleanup upload, audit event),
`handleUploadStatus()` (GET — 204 with Range header),
`handleUploadCancel()` (DELETE — cancel BlobWriter, remove upload row)
- `oci/routes.go`: `parseOCIPath()` extended to handle `/blobs/uploads/`
and `/blobs/uploads/<uuid>` patterns; `dispatchUpload()` routes by method
Step 6.3 — Manifest push:
- `db/push.go`: `PushManifestParams` struct, `PushManifest()` — single
SQLite transaction: create repo if not exists, upsert manifest (ON
CONFLICT DO UPDATE), clear and repopulate manifest_blobs join table,
upsert tag if provided (atomic tag move)
- `oci/manifest.go`: `handleManifestPut()` — full push flow per §5:
parse JSON, compute SHA-256, verify digest if push-by-digest, collect
config+layer descriptors, verify all referenced blobs exist (400
MANIFEST_BLOB_UNKNOWN if missing), call PushManifest(), audit event
with tag details, returns 201 with Docker-Content-Digest/Location/
Content-Type headers
**Verification:**
- `make all` passes: vet clean, lint 0 issues, 198 tests passing,
all 3 binaries built
- DB tests (15 new): upload CRUD (create/get/update/delete/not-found),
GetOrCreateRepository (new + existing), BlobExists (found/not-found),
InsertBlob (new + idempotent), PushManifest by tag (verify repo creation,
manifest, tag, manifest_blobs), by digest (no tag), tag move (atomic
update), idempotent re-push
- OCI upload tests (8 new): initiate (202, Location/UUID/Range headers,
implicit repo creation), unique UUIDs (5 initiates → 5 distinct UUIDs),
monolithic upload (POST → PUT with body → 201), chunked upload
(POST → PATCH → PATCH → PUT → 201), digest mismatch (400 DIGEST_INVALID),
upload status (GET → 204), cancel (DELETE → 204), nonexistent UUID
(PATCH → 404 BLOB_UPLOAD_UNKNOWN)
- OCI manifest push tests (7 new): push by tag (201, correct headers),
push by digest, digest mismatch (400), missing blob (400
MANIFEST_BLOB_UNKNOWN), malformed JSON (400 MANIFEST_INVALID), empty
manifest, tag update (atomic move), re-push idempotent
- Route parsing tests (5 new subtests): upload initiate (trailing slash),
upload initiate (no trailing slash), upload with UUID, multi-segment
repo upload, multi-segment repo upload initiate
---
### 2026-03-19 — Batch B: Phase 5 (OCI pull) + Phase 8 (admin REST)
**Task:** Implement the OCI Distribution Spec pull path and the admin
REST API management endpoints. Both phases depend on Phase 4 (policy)
but not on each other — implemented in parallel.
**Changes:**
Phase 5 — `internal/db/` + `internal/oci/` (Steps 5.15.5):
Step 5.1 — OCI handler scaffolding:
- `db/errors.go`: `ErrRepoNotFound`, `ErrManifestNotFound`, `ErrBlobNotFound`
sentinel errors
- `oci/handler.go`: `Handler` struct with `DBQuerier`, `BlobOpener`,
`PolicyEval`, `AuditFunc` interfaces; `NewHandler()` constructor;
`checkPolicy()` inline policy check; `audit()` helper
- `oci/ocierror.go`: `writeOCIError()` duplicated from server package
(15 lines, avoids coupling)
- `oci/routes.go`: `parseOCIPath()` splits from the right to handle
multi-segment repo names (e.g., `org/team/app/manifests/latest`);
`Router()` returns chi router with `/_catalog` and `/*` catch-all;
`dispatch()` routes to manifest/blob/tag handlers
Step 5.2 — Manifest pull:
- `db/repository.go`: `ManifestRow` type, `GetRepositoryByName()`,
`GetManifestByTag()`, `GetManifestByDigest()`
- `oci/manifest.go`: GET/HEAD `/v2/{name}/manifests/{reference}`;
resolves by tag or digest; sets Content-Type, Docker-Content-Digest,
Content-Length headers
Step 5.3 — Blob download:
- `db/repository.go`: `BlobExistsInRepo()` — joins blobs+manifest_blobs+
manifests to verify blob belongs to repo
- `oci/blob.go`: GET/HEAD `/v2/{name}/blobs/{digest}`; validates blob
exists in repo before streaming from storage
Step 5.4 — Tag listing:
- `db/repository.go`: `ListTags()` with cursor-based pagination
(after/limit)
- `oci/tags.go`: GET `/v2/{name}/tags/list` with OCI `?n=`/`?last=`
pagination and `Link` header for next page
Step 5.5 — Catalog listing:
- `db/repository.go`: `ListRepositoryNames()` with cursor-based pagination
- `oci/catalog.go`: GET `/v2/_catalog` with same pagination pattern
Phase 8 — `internal/db/` + `internal/server/` (Steps 8.18.5):
Step 8.1 — Auth endpoints:
- `server/admin_auth.go`: POST `/v1/auth/login` (JSON body → MCIAS),
POST `/v1/auth/logout` (204 stub), GET `/v1/health` (no auth)
Step 8.2 — Repository management:
- `db/admin.go`: `RepoMetadata`, `TagInfo`, `ManifestInfo`, `RepoDetail`
types; `ListRepositoriesWithMetadata()`, `GetRepositoryDetail()`,
`DeleteRepository()`
- `server/admin_repo.go`: GET/DELETE `/v1/repositories` and
`/v1/repositories/*` (wildcard for multi-segment names)
Step 8.3 — Policy CRUD:
- `db/admin.go`: `PolicyRuleRow` type, `ErrPolicyRuleNotFound`;
`CreatePolicyRule()`, `GetPolicyRule()`, `ListPolicyRules()`,
`UpdatePolicyRule()`, `SetPolicyRuleEnabled()`, `DeletePolicyRule()`
- `server/admin_policy.go`: full CRUD on `/v1/policy/rules` and
`/v1/policy/rules/{id}`; `PolicyReloader` interface; input validation
(priority >= 1, valid effect/actions); mutations trigger engine reload
Step 8.4 — Audit endpoint:
- `server/admin_audit.go`: GET `/v1/audit` with query parameter filters
(event_type, actor_id, repository, since, until, n, offset);
delegates to `db.ListAuditEvents`
Step 8.5 — GC endpoints:
- `server/admin_gc.go`: `GCState` struct with `sync.Mutex`; POST `/v1/gc`
returns 202 (stub for Phase 9); GET `/v1/gc/status`; concurrent
trigger returns 409
Shared admin infrastructure:
- `server/admin.go`: `writeAdminError()` (platform `{"error":"..."}`
format), `writeJSON()`, `RequireAdmin()` middleware, `hasRole()` helper
- `server/admin_routes.go`: `AdminDeps` struct, `MountAdminRoutes()`
mounts all `/v1/*` endpoints with proper auth/admin middleware layers
**Verification:**
- `make all` passes: vet clean, lint 0 issues, 168 tests passing,
all 3 binaries built
- Phase 5 (34 new tests): parseOCIPath (14 subtests covering simple/
multi-segment/edge cases), manifest GET by tag/digest + HEAD + not
found, blob GET/HEAD + not in repo + repo not found, tags list +
pagination + empty + repo not found, catalog list + pagination + empty,
DB repository methods (15 tests covering all 6 query methods)
- Phase 8 (51 new tests): DB admin methods (19 tests covering CRUD,
pagination, cascade, not-found), admin auth (login ok/fail, health,
logout), admin repos (list/detail/delete/non-admin 403), admin policy
(full CRUD cycle, validation errors, non-admin 403, engine reload),
admin audit (list with filters, pagination), admin GC (trigger 202,
status, concurrent 409), RequireAdmin middleware (allowed/denied/
no claims)
---
### 2026-03-19 — Phase 4: Policy engine
**Task:** Implement the registry-specific authorization engine with