# P5a task 07: the PDF document cache, `pages`, `grep`, and the question limit

**Branch:** `p5a` (`git switch p5a`; `git status --short` must be empty, otherwise stop)
**Commit subject:** `api: PDF cache with revalidation; pages and grep; refuse questions on very long documents`

## Goal

Task 06 serves PDFs but fetches and extracts every time. After this task, a PDF already
extracted is answered from the cache: fresh for 24 h, then revalidated with a conditional GET.
An agent can ask for `pages` or `grep`. A question on a document too long for the digest is
refused with a pointer to those two.

## Files

- Copy: nothing (the given `internal/api/pdf_test.go` came in task 06)
- Modify: `internal/api/pdf.go` (all new logic), `internal/api/api.go` (only the call in rule 1
  and the two request fields' use in rule 3), `docs/implementer-log.md`

## What already exists

- Task 06's `extractPDF`, `pdfError`, `renderPDF`, the `Server` fields (`PDFTTL`,
  `PDFMaxQuestionChunks`), the `v1Req` fields (`Pages`, `Grep`), and `DefaultPDFTTL` /
  `DefaultPDFMaxQuestionChunks`.
- `internal/dcache`:
  ```go
  type PDFEntry struct{ ContentHash, ETag, LastModified string; FetchedAt time.Time; Meta []byte; Pages []string }
  func PDFKey(url, callerClass string) string
  func (d *DB) PutPDF(key string, e PDFEntry)
  func (d *DB) GetPDF(key string) (PDFEntry, bool)
  func (d *DB) TouchPDF(key string, fetchedAt time.Time)
  ```
  A nil `*DB` is a no-op.
- `internal/fetch`: `func (f *Fetcher) FetchIf(ctx, rawURL, etag, lastModified string) (*Resource, error)`.
  A 304 returns `Status 304`, an empty body and a nil error.
- `internal/digest`: `Split(text string, maxTok int) []string` and `ChunkTokens`.

## Rules

1. **Cache first, in `doFetch`**, before `acquire` and `s.Fetch.Fetch`: call
   `s.pdfFromCache(ctx, c, rawURL) (res *fetch.Resource, doc *pdftext.Doc, ok bool, err error)`.
   The key is `dcache.PDFKey(rawURL, s.callerClass(c))`; the cache is `s.cache()` (may be nil).
   - **Miss:** `ok == false`; go on as today.
   - **Fresh** (`time.Since(e.FetchedAt) < ttl`, where `ttl` is `s.PDFTTL` or `DefaultPDFTTL`):
     build the doc from the entry (unmarshal `Meta` into `doc.Meta`; `doc.Text = e.Pages`), and a
     synthetic `res`: `URL` and `FinalURL` = rawURL, `Status` 200,
     `ContentType "application/pdf"`, `FetchedAt` = `e.FetchedAt`, `Hash` = `e.ContentHash`, no
     body. No network call and no child.
   - **Stale:** `s.Fetch.FetchIf(ctx, rawURL, e.ETag, e.LastModified)`.
     - 304 → `TouchPDF(key, now)`, then answer as fresh.
     - 200 → return that response with `ok == false`, so `doFetch` goes on without a second
       GET: the new document is extracted and stored.
     - An error → return it.
   - The source in the `/v1/fetch` envelope says `"cached": true` for an answer from the cache
     (fresh or after a 304).
2. **Store after extraction:** after `extractPDF` succeeds, `PutPDF` under the URL key with:
   - `ContentHash: res.SHA256()`;
   - `ETag` and `Last-Modified` from the response headers;
   - `FetchedAt: now`;
   - `Meta`: the JSON of `doc.Meta`;
   - `Pages: doc.Text`.
3. **`pages` and `grep` (`/v1/fetch`):**
   - Pass `req.Pages` and `req.Grep` into `pdfdoc.Options` in `renderPDF`.
   - `pdfdoc.ErrBadPages` or `pdfdoc.ErrBadGrep` (use `errors.Is`) → 400, with the error's own
     text as the message.
   - `pages` or `grep` on a non-PDF → `400 pages and grep apply to PDFs only`.
4. **Question limit:** in mode `digest` with a doc, compute
   `n := len(digest.Split(pdfdoc.Full(doc), digest.ChunkTokens))`. If `n` exceeds
   `s.PDFMaxQuestionChunks` (or its default), answer
   `422 document too long for question (<n> chunks); use grep or pages` **before** any digest
   call.

## Steps

- [ ] **1.** `git switch p5a`. **2. See the three tests fail:**
  `go test -count=1 -run 'TestPDFPagesAndGrep|TestPDFCache|TestPDFQuestionTooLong' ./internal/api/`.
- [ ] **3.** `pdfFromCache` and the store (rules 1–2), **4.** pages and grep, **5.** the question limit.
- [ ] **6.** `gofmt -w`; `go test -race -count=3 ./internal/api/` → `ok` (all ten PDF tests and
  every existing test).
- [ ] **7.** `make gate` → `gate: ok`. **8.** Row `p5a/07-api-cache-pages`; commit.

```sh
git add internal/api docs/implementer-log.md
git commit -F .state/commit-msg.txt
```

## Done when

- `go test -race -count=3 ./internal/api/` is ok; `make gate` ok; the given file is still
  byte-identical.