kyleandClaude Opus 5.5
cfde35397a
docs/plans/p5a: 01 kind+fetch limits, 02 pdftext child + runner, 03 dcache PDF tables, 04 pdfpool, 05 pdfdoc render (map/pages/grep), 06 api, 07 digest citations, metrics, CLI, skill, unit, README. Given tests compile against a panic-only skeleton and fail there for the intended reasons; PDFs for tests are built by internal/pdftest (given), plus two qpdf-encrypted fixtures. Owner changes for the run: vendor go-pdfkit/extract v0.3.0 + reader v0.6.0 (pinned via tools.go until task 02); AGENTS.md, tools/run-plan.sh (enforces a Co-Authored-By trailer for the model), docs/implementer-log.md, `make gate`; gofmt (whitespace) on clean.go and guard; TestUnsupportedType uses a zip now that PDFs are supported; requestDeadline 90→210 s and writeTimeout 120→240 s for the PDF path's worst case. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
83 lines
3.4 KiB
Markdown
83 lines
3.4 KiB
Markdown
# P5a task 03: a document cache for PDFs
|
|
|
|
**Branch:** `p5a` (`git switch p5a`; `git status --short` must be empty, otherwise stop)
|
|
**Commit subject:** `dcache: PDF document cache (per URL and caller class, whole-document eviction)`
|
|
|
|
## Goal
|
|
|
|
Extracting a thesis takes a second; downloading it can take a minute. Once a document is
|
|
extracted, later requests for other pages, a grep, or a question must not fetch or extract it
|
|
again. This task adds two tables to the existing SQLite cache and four functions. Freshness (the
|
|
24 h TTL and revalidation) is decided by the caller in task 06, not here.
|
|
|
|
## Files
|
|
|
|
- Copy: `internal/dcache/pdf_test.go`
|
|
- Create: `internal/dcache/pdf.go`
|
|
- Modify: `internal/dcache/dcache.go` (the two `CREATE TABLE`s, `Size`, `evict`), `docs/implementer-log.md`
|
|
|
|
## Interfaces
|
|
|
|
```go
|
|
package dcache
|
|
|
|
// PDFEntry is one cached PDF: how to revalidate it, its metadata (the pdftext.Meta JSON, opaque
|
|
// here) and the text of each extracted page (Pages[i] is page i+1).
|
|
type PDFEntry struct {
|
|
ContentHash string
|
|
ETag string
|
|
LastModified string
|
|
FetchedAt time.Time
|
|
Meta []byte
|
|
Pages []string
|
|
}
|
|
|
|
// PDFKey is the cache key for a document: a hash of the URL and the caller class, like the
|
|
// other keys here (the URL never appears in the clear).
|
|
func PDFKey(url, callerClass string) string
|
|
|
|
func (d *DB) PutPDF(key string, e PDFEntry) // replaces any entry under key, pages included
|
|
func (d *DB) GetPDF(key string) (PDFEntry, bool)
|
|
func (d *DB) TouchPDF(key string, fetchedAt time.Time) // after a 304: only FetchedAt moves; a miss is a no-op
|
|
```
|
|
|
|
## Rules
|
|
|
|
1. Tables:
|
|
- `pdf_doc(k TEXT PRIMARY KEY, content_hash, etag, last_modified, fetched_at INTEGER, meta BLOB, last_hit INTEGER, nbytes INTEGER)`
|
|
- `pdf_page(k TEXT, page INTEGER, text TEXT, PRIMARY KEY (k, page))`
|
|
`nbytes` is the document's size (meta plus every page's text) and counts toward `Size()`.
|
|
2. `PutPDF` writes the document and its pages in **one transaction**, deleting the old pages
|
|
first, then calls `evict`. `GetPDF` updates `last_hit`, and returns the pages in page order;
|
|
an empty page keeps its place.
|
|
3. `evict` treats a PDF document like any other entry, by `last_hit`, and removes it **whole**:
|
|
its `pdf_doc` row and all its `pdf_page` rows together. A cache never holds half a document.
|
|
4. Hit and miss counters: add `PDFH`, `PDFM` beside the others (the metrics task reads them later).
|
|
5. A nil `*DB` is a no-op everywhere, as for the existing methods; a SQL error is logged at
|
|
`Warn` (never with the key's text or page text) and behaves like a miss.
|
|
|
|
## Facts
|
|
|
|
- `DigestKey`, `EmbedKey` and `SearchKey` show how keys are hashed (`sha256` of the parts joined
|
|
with a separator). Follow the same pattern.
|
|
- The existing `evict` picks the oldest `last_hit` across the three existing tables; extend that
|
|
union with `pdf_doc`.
|
|
|
|
## Steps
|
|
|
|
- [ ] **1.** `git switch p5a`; copy the given test. **2. See it fail** (undefined names).
|
|
- [ ] **3.** `dcache.go` schema; `pdf.go`; `Size` and `evict`.
|
|
- [ ] **4.** `gofmt -w`; `go test -race -count=3 ./internal/dcache/` → `ok` (the existing dcache
|
|
tests too).
|
|
- [ ] **5.** `make gate`. **6.** Row `p5a/03-dcache`; commit.
|
|
|
|
```sh
|
|
git add internal/dcache docs/implementer-log.md
|
|
git commit -F .state/commit-msg.txt
|
|
```
|
|
|
|
## Done when
|
|
|
|
- The given and existing dcache tests pass under `-race -count=3`; `make gate` ok; given file
|
|
byte-identical.
|