77 lines
3.6 KiB
Markdown
77 lines
3.6 KiB
Markdown
# P5c task 01: `POST /v1/extract`
|
|
|
|
**Branch:** `p5c` (`git switch -c p5c master` if it does not exist, else `git switch p5c`; `git status --short` must be empty, otherwise stop)
|
|
**Commit subject:** `api: POST /v1/extract answers an uploaded PDF like a fetched one (content-addressed cache)`
|
|
|
|
## Goal
|
|
|
|
`POST /v1/extract` with a PDF as the request body returns the `/v1/fetch` PDF envelope:
|
|
text with page markers or the map, `pages`, `grep`, or a cited digest. Nothing is fetched.
|
|
The bytes are hashed, the cache key is the hash, and the same file is extracted once.
|
|
|
|
## Files
|
|
|
|
- Copy: `internal/api/extract_test.go`
|
|
- Create: `internal/api/extract.go` (the handler and its envelope)
|
|
- Modify: `internal/api/api.go`: only the route in `New`, the `Server` field and constant
|
|
below, and the two lines in `ServeHTTP` (rule 1); `docs/implementer-log.md`
|
|
|
|
## Interfaces
|
|
|
|
```go
|
|
package api
|
|
|
|
const DefaultPDFMaxUpload = 100 << 20
|
|
|
|
// in Server:
|
|
PDFMaxUpload int64 // 0 = DefaultPDFMaxUpload
|
|
|
|
// route: s.Mux.HandleFunc("POST /v1/extract", s.v1extract)
|
|
// query parameters (all optional): pages, grep, question, mode ("clean" | "digest";
|
|
// default digest when question is set, else clean)
|
|
```
|
|
|
|
What exists and is reused, unchanged:
|
|
- `s.caller`, `s.callerClass`;
|
|
- `s.extractPDF(ctx, c, *fetch.Resource)`, `pdfError`, `s.renderPDF`, `s.storePDF(key, res, doc)`, `s.pdfFromEntry`;
|
|
- `s.cache()`, `dcache.PDFKey`, `clean.Kind`, `pdfdoc.Full`;
|
|
- `s.Digest.Run`, `s.Prom.PDFCacheHit`, `writeErr`, `writeJSON`.
|
|
|
|
## Rules
|
|
|
|
1. **`ServeHTTP`:**
|
|
- today it wraps every request body in `http.MaxBytesReader(w, r.Body, maxReqBody)` (64 KiB). For `/v1/extract` use `PDFMaxUpload` instead;
|
|
- add `/v1/extract` to the paths that get `requestDeadline`;
|
|
- every other path keeps 64 KiB (the given test checks that).
|
|
2. **Handler:**
|
|
- Authenticate as the other endpoints do.
|
|
- Read the whole body. Empty → `400 empty body`. Over the limit (`*http.MaxBytesError`) → `413 too large (>N MB)`.
|
|
- `clean.Kind(Content-Type, body)` must be `"pdf"`; otherwise `415 extract accepts PDFs only`.
|
|
3. **Cache:**
|
|
- `hash := "sha256:" + hex(sha256(body))`, `key := dcache.PDFKey("upload:"+hash, s.callerClass(c))`.
|
|
- An entry → the doc from it, `cached: true`, `PDFCacheHit`, no child. The content is addressed by its hash, so there is **no TTL and no revalidation**.
|
|
- Otherwise build `&fetch.Resource{URL: "upload", FinalURL: "upload", Status: 200, ContentType: ct, Body: body, Hash: hash, FetchedAt: now}`, then `extractPDF`, then `storePDF(key, res, doc)`.
|
|
- Errors go through `pdfError`, via the existing `writeFetchErr` (it adds `Retry-After`).
|
|
4. **Answer:** the same fields as `/v1/fetch` for a PDF:
|
|
- `content`, `mode`, `untrusted: true`, `warnings`, `sizes`, `timings`, `pdf`, and `digest` when a digest ran, including P5b's `retrieval` and `embedded`;
|
|
- `source` is `{"kind": "upload", "hash", "bytes", "cached", "fetched_at"}`, with **no `url`** key;
|
|
- bad `pages`/`grep` → 400 (as in `v1fetch`); the question limit → the same 422 as `v1fetch`.
|
|
- Do not call Honcho context for uploads.
|
|
5. **Logging:** one line, `"extract"`, with caller, bytes, `pdf_pages`, mode and cached. Never a title or any text. The server never sees a filename.
|
|
|
|
## Steps
|
|
|
|
- [ ] **1.** Branch; copy the given test. **2. See it fail.**
|
|
- [ ] **3.** `api.go` hooks, **4.** `extract.go`.
|
|
- [ ] **5.** `gofmt -w`; `go test -race -count=3 ./internal/api/` → `ok`. **6.** `make gate`.
|
|
- [ ] **7.** Row `p5c/01-extract-api`; commit.
|
|
|
|
```sh
|
|
git add internal/api docs/implementer-log.md
|
|
git commit -F .state/commit-msg.txt
|
|
```
|
|
|
|
## Done when
|
|
|
|
- All tests pass under `-race -count=3`; `make gate` ok; given file byte-identical.
|