kyleandClaude Opus 5.5
cfde35397a
docs/plans/p5a: 01 kind+fetch limits, 02 pdftext child + runner, 03 dcache PDF tables, 04 pdfpool, 05 pdfdoc render (map/pages/grep), 06 api, 07 digest citations, metrics, CLI, skill, unit, README. Given tests compile against a panic-only skeleton and fail there for the intended reasons; PDFs for tests are built by internal/pdftest (given), plus two qpdf-encrypted fixtures. Owner changes for the run: vendor go-pdfkit/extract v0.3.0 + reader v0.6.0 (pinned via tools.go until task 02); AGENTS.md, tools/run-plan.sh (enforces a Co-Authored-By trailer for the model), docs/implementer-log.md, `make gate`; gofmt (whitespace) on clean.go and guard; TestUnsupportedType uses a zip now that PDFs are supported; requestDeadline 90→210 s and writeTimeout 120→240 s for the PDF path's worst case. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
92 lines
3.8 KiB
Markdown
92 lines
3.8 KiB
Markdown
# extract
|
|
|
|
[](https://github.com/go-pdfkit/extract/actions/workflows/ci.yml)
|
|
[](https://pkg.go.dev/github.com/go-pdfkit/extract)
|
|
[](LICENSE)
|
|
[](#how-it-is-checked)
|
|
|
|
Reads a PDF page back: the text on it, with where each piece of it sits, and
|
|
the images it places.
|
|
|
|
```go
|
|
text, err := extract.Text(doc, 1) // the page as words, lines put back together
|
|
runs, err := extract.Runs(doc, 1) // every piece, with its place and its size
|
|
imgs, err := extract.Images(doc, 1) // every picture, and where it lands
|
|
```
|
|
|
|
Pure Go, no C. It builds for `GOOS=js/wasm` and every architecture the fleet
|
|
targets.
|
|
|
|
## What a page actually holds
|
|
|
|
Not text. A page holds instructions for drawing glyphs, and the text is what
|
|
those glyphs were meant to say — which the document says only if it was asked
|
|
to. Where it does not, this says so rather than guessing: a run whose codes
|
|
could not be worked out comes back marked `Unreadable`, so a caller can tell
|
|
*this page says nothing* from *this page could not be read*.
|
|
|
|
Three things are tried, in order:
|
|
|
|
1. the font's `/ToUnicode` map, which is the document's own word on what its
|
|
text says;
|
|
2. the name its encoding gives the code — but only a name the document chose
|
|
itself, since a **symbolic** font read through an assumed encoding gives
|
|
the wrong letter with nothing to say so;
|
|
3. what the **embedded font program** calls the code, read with
|
|
[`go-opentype`](https://github.com/go-opentype/opentype) — which is the
|
|
honest answer for a symbolic font, and the reason a page of mathematics
|
|
comes back as mathematics.
|
|
|
|
Text drawn in the mode that puts no ink on the page — how a scanner writes
|
|
what it read underneath the picture it read it from — is returned and marked
|
|
`Invisible`.
|
|
|
|
## Images
|
|
|
|
A picture comes back as the file holds it, with the box it covers on the page.
|
|
A JPEG comes out as a JPEG: re-encoding it would lose something for nothing.
|
|
Turning the other kinds into pixels means reading their colour space, which is
|
|
[`render`](https://github.com/go-pdfkit/render)'s work rather than this
|
|
package's.
|
|
|
|
## How it is checked
|
|
|
|
100% of statements, including every branch that handles a page saying
|
|
something it should not.
|
|
|
|
Then on the corpus — 118 833 real PDFs, 121 946 pages:
|
|
|
|
```
|
|
pages read 121946
|
|
with text 94865 (77.8%)
|
|
saying nothing 27081, of which 4812 did draw glyphs
|
|
characters 33976414 (17272226 of them letters)
|
|
runs 10797045
|
|
unreadable 203607 (1.89%)
|
|
invisible 29917
|
|
images 300685
|
|
panics 0
|
|
```
|
|
|
|
The 22% of pages that say nothing are figures — this corpus is arXiv's, and
|
|
most of it is plots. Of the pages that *did* draw glyphs and still said
|
|
nothing, nearly all are Type 3 fonts with no `/ToUnicode`: dvips bitmap fonts
|
|
whose glyphs are called `a1` and `s32`, which no reader can turn back into
|
|
letters.
|
|
|
|
## The rest of the family
|
|
|
|
[**reader**](https://github.com/go-pdfkit/reader) reads and writes the format ·
|
|
[**ops**](https://github.com/go-pdfkit/ops) is the verbs and the `pdfops` command ·
|
|
[**render**](https://github.com/go-pdfkit/render) turns a page into pixels ·
|
|
[**pdffont**](https://github.com/go-pdfkit/pdffont) is what a document says about a font ·
|
|
[**extract**](https://github.com/go-pdfkit/extract) reads a page back ·
|
|
[**coedit**](https://github.com/go-pdfkit/coedit) is a PDF several people edit at once ·
|
|
[**app**](https://github.com/go-pdfkit/app) is the browser workbench.
|
|
|
|
Documentation for all of it: <https://go-pdfkit.github.io/docs/>
|
|
|
|
## Licence
|
|
|
|
BSD-3-Clause.
|