kyle/sift
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

P5a plan: PDF documents — seven Ornith tasks with acceptance tests first

Browse Source 
docs/plans/p5a: 01 kind+fetch limits, 02 pdftext child + runner, 03 dcache PDF tables,
04 pdfpool, 05 pdfdoc render (map/pages/grep), 06 api, 07 digest citations, metrics, CLI,
skill, unit, README. Given tests compile against a panic-only skeleton and fail there for the
intended reasons; PDFs for tests are built by internal/pdftest (given), plus two qpdf-encrypted
fixtures.

Owner changes for the run: vendor go-pdfkit/extract v0.3.0 + reader v0.6.0 (pinned via
tools.go until task 02); AGENTS.md, tools/run-plan.sh (enforces a Co-Authored-By trailer for
the model), docs/implementer-log.md, `make gate`; gofmt (whitespace) on clean.go and guard;
TestUnsupportedType uses a zip now that PDFs are supported; requestDeadline 90→210 s and
writeTimeout 120→240 s for the PDF path's worst case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
kyle
2026-09-25 20:47:38 -07:00
co-authored by Claude Opus 5.5
parent 3f2b78d4aa
commit cfde35397a
115 changed files with 26422 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+1
View File
@@ -5,3 +5,4 @@
/sift
/tests/testdata/bakeoff-out/
/tests/testdata/bakeoff-raw/
.state/
AGENTS.md
+98
View File
@@ -0,0 +1,98 @@
# AGENTS.md
sift is the fleet's fetch and search service for LLM agents: it fetches a URL through an SSRF
guard, cleans the result (HTML → Markdown, JSON, text, and from P5a PDF), optionally digests it
with a small model, and serves Hermes agents through a Firecrawl-compatible API. It is written in
Go and runs on orion under a hardened systemd unit. You are implementing it one task at a time.
## How you work
1. The owner gives you one task file, `docs/plans/<plan>/NN-name.md`. Read it fully. Do that task
and nothing else. Do not start the next task.
2. Do the steps in order. Where a step shows a command and its expected output, run it and compare.
3. Do not read other task files, `docs/PLAN.md` or other phase docs unless the task tells you to.
The task file quotes what you need.
4. If something in the task is impossible, contradictory, or fails twice in the same way, **stop**.
Do not improvise, do not change a test, do not weaken a check. Add your row to
`docs/implementer-log.md` with status `stopped`, say what you tried and what happened, commit
only that file, and tell the owner.
5. Work only from files inside this repository. Never read another checkout, and never search the
file system for code. If you are stuck, stop as in point 4.
## Files you must never edit
- `docs/plans/`, `docs/PLAN.md`, `docs/P*.md`, `AGENTS.md`, `tools/run-plan.sh`
- `internal/guard/` and `internal/auth/` — the SSRF guard and the key store are the security
boundary; no task changes them.
- `go.mod`, `go.sum`, `vendor/` — every dependency is already vendored. Never add a module and
never run `go get` or `go mod tidy`/`vendor`; the build is offline.
- `nix/` and `flake.nix`, unless the task names the exact change.
- Anything a task told you to copy from `docs/plans/**/_files/` — tests, fixtures,
`internal/pdftest` — in this plan **or any earlier one**. If a copied test fails, your code is
wrong. If a copied test can no longer be right because the task changes what it tested, that
is the owner's error: stop and report it.
## Code rules
- Go 1.26 (`go.mod`). Standard library plus what is already in `vendor/`.
- No **new** source file over 400 lines. Do not grow `internal/api/api.go`: new API code goes in
the new file the task names.
- Library code (`internal/...`) never panics on input: no `panic`, no indexing that can go out of
range on data from a file, a request, a PDF or a peer. Check lengths. `main` may exit with a
message.
- Errors are values: return them, wrap with `fmt.Errorf("...: %w", err)`. A check that cannot do
its job fails; it does not return "ok".
- **Never log, print or store a request or response body, a document's text or its title.** Log
lines carry caller, host, sizes, counts, durations and outcomes. For a restricted caller
(`PermitRestricted`) use the existing helpers (`logHost`, `restrictedLog`, `logErr`): they
already blank what must not be written.
- Anything read from the network, a child process or a file is bounded (`io.LimitReader`,
deadlines).
- Exported names, field names and JSON tags are exactly as the task gives them; the tests compile
against them.
- Comments say why, not what. `gofmt` decides layout.
## Lessons from earlier work (crossbar, same owner, same process)
- A type assertion on a value from outside your package uses the two-value form and handles
`false`.
- When a rule says "every" or "everywhere", finish by listing each place it applies and checking
them one by one.
- Never end a turn by describing what you are about to do. Do it, then report.
- Work only inside this repository. Scratch programs under `/tmp` or anywhere else are refused by
the sandbox, and **a refused tool call is not a reason to end the turn**: write the experiment
as a `_test.go` file inside the repository (delete it before committing), or reason it out.
- Never change when production code releases, flushes or records something just to make a given
test's timing pass. If a given test seems to check a value before the code could settle it,
that is the owner's bug: stop and report it.
## The gate
`make gate` must pass before a task is done:
```
make gate # gofmt check, go vet, go test -race ./..., staticcheck if installed
```
It runs offline against `vendor/`. Run `gofmt -w` on the files you touched first. `go test
./internal/<pkg>/` runs one package.
## Git
- Work on the branch the task names. One task is one commit.
- Stage only the paths the task lists: `git add <path> ...`. Never `git add -A` or `git add .`.
- Never push, amend, rebase, reset, or switch to another branch.
- Commit message: the subject line the task gives, a blank line, a short body if useful, then the
`Co-Authored-By:` trailer line the owner's prompt gives you, exactly, as the last line.
- A message with more than one line goes in `.state/commit-msg.txt` (inside the repository and
ignored by git; `/tmp` is refused) and is committed with `git commit -F .state/commit-msg.txt`.
## The implementer log
Before you commit, add one row to the table in `docs/implementer-log.md` and include the file in
the commit:
`| <plan>/<task> | <date> | done or stopped | <sessions> | gate pass/fail | deviations | what you did |`
Be honest: the log is how the owner judges the process. A deviation you log is a finding; one
you hide is a defect.
Makefile
+8 -1
View File
@@ -10,7 +10,7 @@ CGO_ENABLED ?= 0
GO ?= go
STATICCHECK ?= $(shell command -v staticcheck 2>/dev/null)
.PHONY: all build test lint contract install clean
.PHONY: all build test lint contract install clean gate
all: build
@@ -20,6 +20,13 @@ build:
test:
$(GO) test -race ./...
# The implementer's gate (AGENTS.md): formatting, vet, race tests, staticcheck when present. Offline.
gate:
@test -z "$$(gofmt -l cmd internal tests 2>&1)" || { echo "gofmt: these files need formatting:"; gofmt -l cmd internal tests; exit 1; }
$(MAKE) lint
$(GO) test -race -count=1 ./...
@echo "gate: ok"
lint:
$(GO) vet ./...
@if [ -n "$(STATICCHECK)" ]; then $(STATICCHECK) ./...; else echo "staticcheck not found; skipped"; fi
docs/P5a.md
+17 -9
View File
@@ -61,9 +61,7 @@ Notes:
**Known extract defect:** the Russian UDHR (Acrobat Distiller 5, 2002) comes out with words glued
together and digits dropped (recall 0.03; gopdf and pdfium read it cleanly).
- Keep it as a regression fixture: `tests/testdata/pdf/udhr-ru.pdf`, expected to fail today and
marked as such.
- File the upstream issue.
- File the upstream issue. No fixture is committed: the document is not ours to redistribute.
- Pin `extract v0.3.0` / `reader v0.6.0` in `go.mod` and vendor them, as for every other dependency.
## 4. Detection and fetch
@@ -115,8 +113,9 @@ Kyle, 2026-09-25: "one or two workers seems tenable".
- **Pool:** 2 extraction slots and 4 waiting places. A request that waits more than 20 s, or
finds the queue full, gets `503 {"error":"pdf workers busy"}` with `Retry-After: 10`.
- **Single-flight:** keyed by (caller class, final URL). Concurrent requests for the same
document share one download and one extraction.
- **Single-flight:** keyed by (caller class, content hash). Concurrent requests for the same
bytes share one extraction. Downloads may still happen twice: a URL is not known to be a PDF
until it has been fetched.
- **Per caller:** one PDF job in flight. A second concurrent one gets the existing
`429 busy` path.
- **Cache first:** a request answered from the cache (§7) never touches the pool.
@@ -136,9 +135,11 @@ Both count toward `dcache` `maxBytes` and its eviction, whole documents at a tim
## 8. What callers get back
Text is assembled with page markers, so both agents and digests can cite pages:
Text is assembled with a marker before **every** page, so both agents and digests can cite pages:
```
--- page 1 ---
<page 1 text>
--- page 2 ---
@@ -158,7 +159,7 @@ Text is assembled with page markers, so both agents and digests can cite pages:
- `pages` and `grep` together means grep within those pages.
- The JSON envelope gains:
```
"pdf": {"pages": N, "title": "…", "outline": [...], "returned": [a, b], "map": bool, "grep_hits": n}
"pdf": {"pages", "title", "outline", "extracted", "first", "last", "map", "grep_hits", "note"}
```
**`/v1/fetch`, mode `digest` (`question`):**
@@ -231,8 +232,7 @@ All run offline against `tests/testdata/pdf/` fixtures served by an httptest ser
13. **Digest:** `question` on the 3-page fixture returns a digest containing `(p. ` for a fact
known to be on one page (the model is stubbed, as in the existing digest tests; the check is
that the page markers reach the prompt and the prompt line is present).
14. **The `udhr-ru` regression fixture** is present and marked as a known failure (skip with a
reason naming the upstream issue).
14. (Dropped: no third-party fixture is committed; the extract defect is tracked upstream.)
## 12. Rollout
@@ -242,3 +242,11 @@ All run offline against `tests/testdata/pdf/` fixtures served by an httptest ser
3. The skill update ships with the pin bump to talos, straylight and titan.
4. Watch `sift_pdf_*` and tirith for a week. The success measure is that the
`cp … pdftotext …` shell pattern disappears from agents' commands.
## 13. Changes after the plan was written (2026-09-25)
- The implementation plan is `docs/plans/p5a/` (seven tasks, Ornith implementing, acceptance tests
first). Where this doc and a task file differ, the task file and its given tests win.
- `requestDeadline` 90 s → 210 s and `writeTimeout` 120 s → 240 s, so the PDF path's worst case
(120 s download + 20 s queue + 60 s extraction) fits; other paths keep their own timeouts.
- The map line reads `pages 1-E extracted` (ASCII hyphen) when the child stopped at its page cap.
docs/implementer-log.md
+6
View File
@@ -0,0 +1,6 @@
# Implementer log
One row per task session that ends in a commit. Newest last.
| Task | Date | Status | Sessions | Gate | Deviations | Summary |
|---|---|---|---|---|---|---|
docs/plans/p5a/01-kind-fetch.md
+100
View File
@@ -0,0 +1,100 @@
# P5a task 01: recognise PDFs; give them a larger body cap, a longer deadline, and revalidation
**Branch:** `p5a` (`git switch -c p5a master` if it does not exist, else `git switch p5a`; `git status --short` must be empty, otherwise stop)
**Commit subject:** `clean, fetch: recognise PDFs; 100 MB / 120 s for PDFs; FetchIf revalidates`
## Goal
Today every response that is not HTML, JSON or text is `415 unsupported content type`. After this
task `clean.Kind` names PDFs, and the fetcher lets a PDF be as large as 100 MB and take 120 s
while a web page keeps 5 MB and 20 s. `FetchIf` sends `If-None-Match` / `If-Modified-Since` so a
cached document can be revalidated with a 304. Nothing uses these yet (task 06 does).
## Files
- Copy: `internal/clean/kind_test.go`, `internal/fetch/pdf_test.go`
- Create: `internal/fetch/pdf.go` (the PDF rule and `FetchIf` live here; keep `fetch.go`'s
changes to what the new file needs)
- Modify: `internal/clean/clean.go`, `internal/fetch/fetch.go`, `docs/implementer-log.md`
## Interfaces
```go
package clean
// Kind classifies a fetched body: "html", "json", "text", "pdf" or "other".
func Kind(contentType string, body []byte) string
package fetch
const (
DefaultPDFMaxBody = 100 << 20
DefaultPDFTimeout = 120 * time.Second
)
// in Fetcher:
PDFMaxBody int64 // 0 = DefaultPDFMaxBody
PDFTimeout time.Duration // 0 = DefaultPDFTimeout
// FetchIf is Fetch with validators: a non-empty etag is sent as If-None-Match and a non-empty
// lastModified as If-Modified-Since. A 304 answer returns a Resource with Status 304, an empty
// Body and a nil error. With both empty it is exactly Fetch.
func (f *Fetcher) FetchIf(ctx context.Context, rawURL, etag, lastModified string) (*Resource, error)
```
## Rules
1. **`Kind`** (the media type is lower-cased and its parameters dropped, as `MediaKind` does):
- `application/pdf` or `application/x-pdf` → `"pdf"`, whatever the body.
- `text/html`, `application/xhtml+xml` → `"html"`; `application/json` or `*+json` → `"json"`;
`text/plain`, `text/markdown`, `text/csv` → `"text"` (as today).
- `application/octet-stream` or `binary/octet-stream` → `"pdf"` if `%PDF-` occurs within the
first 1024 bytes of the body, else `"other"`.
- empty Content-Type → `"pdf"` if `%PDF-` occurs within the first 1024 bytes, else today's
`Sniff` (html / json / text).
- anything else → `"other"`.
`MediaKind` also returns `"pdf"` for the two PDF types; its other answers do not change.
2. **Fetch limits.** A response is a PDF when its Content-Type is a PDF type, or when it is empty
or octet-stream and the body starts (within 1024 bytes) with `%PDF-`. For a PDF the body cap
is `PDFMaxBody` and the deadline is `PDFTimeout`, both counted from the start of the request;
for anything else they stay `MaxBody` and `Timeout`. Over the cap is the existing
`413 too large (>N MB decompressed)` with N from the cap that applied.
3. **How to get the deadline right.** `Fetch` today makes one context with `Timeout`, and the
shared `http.Client` has `Timeout` set too — both would cut a PDF off at 20 s. Make the
client's `Timeout` at least the PDF deadline (or unset it and rely on contexts), give the
request a context with the **PDF** deadline, and cancel it at the ordinary deadline unless the
response has been recognised as a PDF by then (`time.AfterFunc` + a flag you set once the
headers, or the first bytes of an octet-stream body, say PDF). Read the body through
`io.LimitReader` with the cap that applies (+1 to detect overflow), as today.
4. `FetchIf` goes through the same guard, client, limits and error mapping as `Fetch`; only the
two request headers differ. Do not copy `Fetch`: factor the shared part.
## Facts
- The tests reach an `httptest` server on 127.0.0.1 through a `guard.Guard` with
`AllowPrefixes: 127.0.0.0/8`; the guard itself is not yours to change.
- `internal/fetch` has no other tests; `internal/api` tests exercise `Fetch` through the API and
must stay green.
## Steps
- [ ] **1.** Branch as above; copy the two given tests.
- [ ] **2. See them fail:** `go test -count=1 ./internal/clean/ ./internal/fetch/` → `Kind`,
`FetchIf`, `DefaultPDFMaxBody` undefined.
- [ ] **3.** `clean.go`. **4.** `fetch/pdf.go` and the smallest changes to `fetch.go`.
- [ ] **5.** `gofmt -w`; `go test -race -count=3 ./internal/clean/ ./internal/fetch/ ./internal/api/` → `ok`.
- [ ] **6.** `make gate`. **7.** Row `p5a/01-kind-fetch`; commit.
```sh
git add internal/clean internal/fetch docs/implementer-log.md
git commit -F .state/commit-msg.txt
```
## Done when
- The given tests and every earlier test pass under `-race -count=3`; `make gate` prints
`gate: ok`; given files byte-identical.
## Stop and report if
- A given test needs a change to `internal/guard`.
docs/plans/p5a/02-pdftext.md
+152
View File
@@ -0,0 +1,152 @@
# P5a task 02: `pdftext` — extract a PDF in a child process
**Branch:** `p5a` (`git switch p5a`; `git status --short` must be empty, otherwise stop)
**Commit subject:** `pdftext: extract PDF text in a re-exec'd child (sift pdftext) with a deadline`
## Goal
Parsing a PDF from the internet is the riskiest thing sift will do. The parser is pure Go
(`github.com/go-pdfkit/extract` + `github.com/go-pdfkit/reader`, already in `vendor/`), but a
goroutine cannot be killed, so it runs in a **child process**: the sift binary re-executed as
`sift pdftext`, reading the PDF on stdin and writing JSON Lines on stdout. The parent kills it at
a deadline. This task builds both halves; nothing calls them yet (task 06 does).
## Files
- Copy: `internal/pdftest/pdftest.go` (test support that builds PDFs; never edit it),
`tests/testdata/pdf/encrypted-user.pdf`, `tests/testdata/pdf/encrypted-owner.pdf`,
`tests/testdata/pdf/README.md`, `internal/pdftext/pdftext_test.go`
- Create: `internal/pdftext/pdftext.go` (types, errors, `Extract`, `ChildMain`),
`internal/pdftext/outline.go` (the outline walk), `internal/pdftext/runner.go` (`Runner`)
- Modify: `cmd/sift/main.go` (one `case "pdftext":` in the command switch), `docs/implementer-log.md`
- Delete: `tools.go` (it only pinned the two modules until this package imports them)
## Interfaces
```go
package pdftext
type OutlineItem struct {
Title string `json:"title"`
Page int `json:"page"` // 1-based
Level int `json:"level"` // 1 = top level
}
type Meta struct {
Pages int `json:"pages"` // pages in the document (not only those extracted)
Title string `json:"title"` // /Info /Title, "" when absent
Outline []OutlineItem `json:"outline"` // never nil: [] when the document has none
}
// Doc is an extracted document: Text[i] is page i+1's text, for the pages extracted
// (len(Text) ≤ Pages). Failed lists pages whose extraction failed (their Text is "").
type Doc struct {
Meta
Text []string
Failed []int
}
var (
ErrEncrypted = errors.New("encrypted PDF (password required)")
ErrUnreadable = errors.New("unreadable PDF")
ErrNoText = errors.New("no text layer (scanned PDF?)")
ErrTimeout = errors.New("pdf extraction timed out")
ErrTooMuchText = errors.New("pdf text over the output limit")
)
// Extract writes one {"meta":{…}} line, then one {"page":n,"text":"…"} line per page from 1 up
// to min(Pages, maxPages); a page that fails yields {"page":n,"error":"…"} and extraction goes
// on. It returns ErrEncrypted when the document needs a user password, ErrUnreadable when it
// cannot be opened or has no pages, and never panics.
func Extract(pdf []byte, maxPages int, w io.Writer) error
// ChildMain is `sift pdftext`: flags (-max-pages N, default 2000), the PDF on stdin (read with a
// 200 MiB cap), Extract to stdout. Exit codes: 0 ok, 3 ErrEncrypted, 4 ErrUnreadable, 1 anything
// else.
func ChildMain(args []string, stdin io.Reader, stdout io.Writer) int
type Runner struct {
Exe string // "" = /proc/self/exe
Args []string // nil = []string{"pdftext"}; -max-pages is appended
Env []string // the child's whole environment; nil = none
Deadline time.Duration // 0 = 60 s
MaxText int64 // bytes of child output; 0 = 64 MiB
MaxPages int // 0 = 2000
}
// Run executes the child, writes pdf to its stdin, and parses its output into a Doc.
func (r *Runner) Run(ctx context.Context, pdf []byte) (*Doc, error)
```
## Rules
1. **Extract.**
- Open with `reader.Open(pdf)`. `reader.ErrWrongPassword` (from `errors.Is`) → `ErrEncrypted`.
Any other open error → `ErrUnreadable`. `PageCount() == 0` → `ErrUnreadable`.
- Each page: `extract.Text(doc, n)` with `n` counting from 1, inside a `recover()`; an error
or a panic becomes that page's `"error"` line.
- Title: the trailer's `/Info` dictionary, key `Title`, as a text string (rule 3).
- The meta line comes first and carries the full page count even when `maxPages` is smaller.
2. **Outline** (`outline.go`). From the trailer's `/Root` (the catalog) follow `/Outlines` →
`/First`, then each item's `/Next`; an item's children start at its `/First` (level + 1).
- An item's page comes from `/Dest` (an array whose first element is a page reference), or
from `/A` when it is a dictionary with `/S /GoTo` and a `/D` array. Map the reference to a
1-based page number by comparing it with `doc.PageRef(i)` for `i` = 1…`PageCount()`.
- Skip an item whose destination is a name or cannot be resolved; keep walking.
- Stop after 10,000 items, and never visit an object twice (a malformed outline can loop).
3. **Text strings** (`/Title` in Info and in outline items): a `reader.String` is raw bytes.
Bytes starting `FE FF` are UTF-16BE (decode the rest with `unicode/utf16`); otherwise treat
each byte as a Latin-1 code point (close enough to PDFDocEncoding for titles). Trim spaces.
4. **ChildMain** runs with a soft memory limit: call `debug.SetMemoryLimit(768 << 20)` first.
It must not read any environment variable and must not open the network.
5. **Runner.Run.**
- `exec.CommandContext` with a context that ends at `Deadline` (or `ctx`, whichever is sooner);
`cmd.Env = r.Env` exactly (nil means an empty environment — the parent's never leaks in).
- Write the PDF to stdin from a goroutine, then close stdin.
- Read stdout through `io.LimitReader(…, MaxText+1)`; more than `MaxText` → kill the child and
return `ErrTooMuchText`. Keep at most 4 KiB of stderr, for the error message only.
- Always `Wait` for the child, so it is reaped before `Run` returns.
- Deadline passed → `ErrTimeout` (even if the kill produced some other error).
- Exit 3 → `ErrEncrypted`, exit 4 → `ErrUnreadable`, any other non-zero exit or unparseable
output → `ErrUnreadable` wrapped with the first line of stderr.
- Parse the lines into a `Doc`. If every extracted page's text is blank after
`strings.TrimSpace` (or no page was extracted), return `ErrNoText`.
6. **`cmd/sift/main.go`:** `case "pdftext": os.Exit(pdftext.ChildMain(os.Args[1:], os.Stdin, os.Stdout))`.
It is not listed in the help text: users never run it.
## Facts
- The tests run the **test binary itself** as the child: `TestMain` becomes the child when
`SIFT_PDFTEXT_CHILD=1` is in its environment. The `SIFT_PDFTEXT_SLEEP` and
`SIFT_PDFTEXT_PIDFILE` variables exist only in that test file; production code never reads
them.
- `extract.Text(doc, 1)` returns `""` with a nil error for an image-only page.
- Reader API you need: `reader.Open`, `doc.PageCount()`, `doc.PageRef(i) (Ref, bool)`,
`doc.Trailer() Dict`, `doc.Resolve(o) (Object, error)`, `Dict.Get(Name) Object`, and the types
`reader.Dict`, `reader.Array`, `reader.Ref`, `reader.String`, `reader.Name`. Read their
definitions under `vendor/github.com/go-pdfkit/reader/` (`object.go`, `document.go`) — that is
inside this repository.
## Steps
- [ ] **1.** `git switch p5a`; copy the given files; `git rm tools.go`.
- [ ] **2. See them fail:** `go test -count=1 ./internal/pdftext/` (undefined names).
- [ ] **3.** `pdftext.go`, **4.** `outline.go`, **5.** `runner.go`, **6.** `cmd/sift/main.go`.
- [ ] **7.** `gofmt -w`; `go test -race -count=3 ./internal/pdftext/` → `ok`.
- [ ] **8.** `make build && printf 'nope' | bin/sift pdftext; echo "exit $?"` → `exit 4`.
- [ ] **9.** `make gate`. **10.** Row `p5a/02-pdftext`; commit.
```sh
git add internal/pdftext internal/pdftest tests/testdata/pdf cmd/sift docs/implementer-log.md # tools.go is already staged by git rm
git commit -F .state/commit-msg.txt
```
## Done when
- The given tests pass under `-race -count=3`, including the deadline test (the child is gone
after `Run` returns); `make gate` ok; given files byte-identical; `go.mod` untouched.
## Stop and report if
- Removing `tools.go` changes `go.mod`, `go.sum` or `vendor/modules.txt` (it must not: the
package now imports both modules), or a vendored package is missing.
docs/plans/p5a/03-dcache.md
+82
View File
@@ -0,0 +1,82 @@
# P5a task 03: a document cache for PDFs
**Branch:** `p5a` (`git switch p5a`; `git status --short` must be empty, otherwise stop)
**Commit subject:** `dcache: PDF document cache (per URL and caller class, whole-document eviction)`
## Goal
Extracting a thesis takes a second; downloading it can take a minute. Once a document is
extracted, later requests for other pages, a grep, or a question must not fetch or extract it
again. This task adds two tables to the existing SQLite cache and four functions. Freshness (the
24 h TTL and revalidation) is decided by the caller in task 06, not here.
## Files
- Copy: `internal/dcache/pdf_test.go`
- Create: `internal/dcache/pdf.go`
- Modify: `internal/dcache/dcache.go` (the two `CREATE TABLE`s, `Size`, `evict`), `docs/implementer-log.md`
## Interfaces
```go
package dcache
// PDFEntry is one cached PDF: how to revalidate it, its metadata (the pdftext.Meta JSON, opaque
// here) and the text of each extracted page (Pages[i] is page i+1).
type PDFEntry struct {
ContentHash string
ETag string
LastModified string
FetchedAt time.Time
Meta []byte
Pages []string
}
// PDFKey is the cache key for a document: a hash of the URL and the caller class, like the
// other keys here (the URL never appears in the clear).
func PDFKey(url, callerClass string) string
func (d *DB) PutPDF(key string, e PDFEntry) // replaces any entry under key, pages included
func (d *DB) GetPDF(key string) (PDFEntry, bool)
func (d *DB) TouchPDF(key string, fetchedAt time.Time) // after a 304: only FetchedAt moves; a miss is a no-op
```
## Rules
1. Tables:
- `pdf_doc(k TEXT PRIMARY KEY, content_hash, etag, last_modified, fetched_at INTEGER, meta BLOB, last_hit INTEGER, nbytes INTEGER)`
- `pdf_page(k TEXT, page INTEGER, text TEXT, PRIMARY KEY (k, page))`
`nbytes` is the document's size (meta plus every page's text) and counts toward `Size()`.
2. `PutPDF` writes the document and its pages in **one transaction**, deleting the old pages
first, then calls `evict`. `GetPDF` updates `last_hit`, and returns the pages in page order;
an empty page keeps its place.
3. `evict` treats a PDF document like any other entry, by `last_hit`, and removes it **whole**:
its `pdf_doc` row and all its `pdf_page` rows together. A cache never holds half a document.
4. Hit and miss counters: add `PDFH`, `PDFM` beside the others (the metrics task reads them later).
5. A nil `*DB` is a no-op everywhere, as for the existing methods; a SQL error is logged at
`Warn` (never with the key's text or page text) and behaves like a miss.
## Facts
- `DigestKey`, `EmbedKey` and `SearchKey` show how keys are hashed (`sha256` of the parts joined
with a separator). Follow the same pattern.
- The existing `evict` picks the oldest `last_hit` across the three existing tables; extend that
union with `pdf_doc`.
## Steps
- [ ] **1.** `git switch p5a`; copy the given test. **2. See it fail** (undefined names).
- [ ] **3.** `dcache.go` schema; `pdf.go`; `Size` and `evict`.
- [ ] **4.** `gofmt -w`; `go test -race -count=3 ./internal/dcache/` → `ok` (the existing dcache
tests too).
- [ ] **5.** `make gate`. **6.** Row `p5a/03-dcache`; commit.
```sh
git add internal/dcache docs/implementer-log.md
git commit -F .state/commit-msg.txt
```
## Done when
- The given and existing dcache tests pass under `-race -count=3`; `make gate` ok; given file
byte-identical.
docs/plans/p5a/04-pdfpool.md
+81
View File
@@ -0,0 +1,81 @@
# P5a task 04: the extraction pool
**Branch:** `p5a` (`git switch p5a`; `git status --short` must be empty, otherwise stop)
**Commit subject:** `pdfpool: a few extraction slots, a short queue, single-flight, one job per caller`
## Goal
A PDF extraction uses a child process and up to a few hundred MB, and sift shares 1 GB with it.
Kyle, 2026-09-25: "I don't think we're seeing a ton of PDF requests simultaneously, so having one
or two workers seems tenable." This task is the small in-process scheduler that enforces that:
two slots, four waiting places, identical requests sharing one job, and no single caller
holding more than one job at a time. Generic: it knows nothing about PDFs.
## Files
- Copy: `internal/pdfpool/pdfpool_test.go`
- Create: `internal/pdfpool/pdfpool.go`
- Modify: `docs/implementer-log.md`
## Interfaces
```go
package pdfpool
var (
ErrBusy = errors.New("pdf workers busy") // queue full, or waited longer than Wait
ErrCallerBusy = errors.New("busy") // this caller already has a job in flight
)
// New returns a pool with `slots` jobs running at once, at most `queue` waiting, and a waiter
// refused after `wait`.
func New(slots, queue int, wait time.Duration) *Pool
// Do runs fn under the pool and returns its result.
func (p *Pool) Do(ctx context.Context, key, caller string, fn func(context.Context) (any, error)) (any, error)
func (p *Pool) Running() int // jobs holding a slot
func (p *Pool) Queued() int // jobs waiting for a slot
func (p *Pool) Waiting(key string) int // callers sharing key's job, not counting the one that started it
```
## Rules, in the order Do applies them
1. **Join.** If a job for `key` is already running or queued, wait for its result (count it in
`Waiting(key)`) and return it. Joining takes no slot, no queue place, and is allowed even if
the caller has another job in flight. `fn` may be nil when joining.
2. **Caller.** If `caller` already has a job in flight (running or queued) for another key,
return `ErrCallerBusy` at once.
3. **Slot.** If fewer than `slots` jobs are running, start at once. Otherwise, if `queue` jobs
are already waiting, return `ErrBusy` at once. Otherwise wait, first come first served, until
a slot frees (start), `wait` passes (`ErrBusy`), or `ctx` ends (`ctx.Err()`). A waiter that
leaves gives up its place.
4. **The job's context is not the caller's.** `fn` runs with a context that is **not** cancelled
when the caller that started it goes away (`context.WithoutCancel(ctx)`): other callers may
be sharing it. The work has its own deadline (the child's). A caller whose `ctx` ends while
waiting for a result gets `ctx.Err()`; the job goes on for the others.
5. **Results are not cached.** When the job ends, the key is forgotten: the next `Do` for it runs
`fn` again. An error from `fn` reaches every sharer. A **panic** in `fn` is recovered,
returned as an error to every sharer, and frees the slot and the caller.
6. Everything is guarded by one mutex; no goroutine leaks when a waiter gives up.
## Steps
- [ ] **1.** `git switch p5a`; copy the given test. **2. See it fail** (undefined names).
- [ ] **3.** `pdfpool.go`.
- [ ] **4.** `gofmt -w`; `go test -race -count=5 ./internal/pdfpool/` → `ok`.
- [ ] **5.** `make gate`. **6.** Row `p5a/04-pdfpool`; commit.
```sh
git add internal/pdfpool docs/implementer-log.md
git commit -F .state/commit-msg.txt
```
## Done when
- The given test passes under `-race -count=5`; `make gate` ok; given file byte-identical.
## Stop and report if
- A given test's timing looks impossible to meet without sleeping in production code (that is
the owner's bug, per `AGENTS.md`).
docs/plans/p5a/05-pdfdoc.md
+115
View File
@@ -0,0 +1,115 @@
# P5a task 05: render an extracted document for an agent
**Branch:** `p5a` (`git switch p5a`; `git status --short` must be empty, otherwise stop)
**Commit subject:** `pdfdoc: page markers, the map for long documents, page ranges, grep`
## Goal
An agent's context is small and a thesis is 60k–150k tokens. This task is the pure function that
decides what an agent sees: the whole text with page markers when it fits; otherwise a **map**
(title, outline with page numbers, the first pages, and how to ask for more); a **page range**;
or **grep** results with page numbers. No I/O, no PDFs: it works on a `*pdftext.Doc`.
## Files
- Copy: `internal/pdfdoc/pdfdoc_test.go`
- Create: `internal/pdfdoc/pdfdoc.go` (`Render`, `Full`, `Marker`, the map, page ranges),
`internal/pdfdoc/grep.go`
- Modify: `docs/implementer-log.md`
## Interfaces
```go
package pdfdoc
type Options struct {
Pages string // "" = no range; "a-b", "a" or "a-" (1-based, inclusive)
Grep string // "" = no grep; an RE2 pattern
OutCap int // characters (runes) the content may use
}
type Rendered struct {
Content string
Map bool // Content is the map of a document longer than OutCap
First, Last int // pages whose text is in Content; 0, 0 for a grep answer
GrepHits int // matches found (more may exist than are shown)
Note string // "" or how the answer was cut and how to continue
}
var (
ErrBadPages = errors.New("invalid pages")
ErrBadGrep = errors.New("invalid grep")
)
func Marker(page int) string // "--- page N ---"
func Full(doc *pdftext.Doc) string // every extracted page with its marker
func Render(doc *pdftext.Doc, opt Options) (Rendered, error)
```
## Rules
1. **Full.** For each extracted page p: `Marker(p) + "\n\n" + strings.TrimSpace(text)`, joined
with `"\n\n"`. The given test shows the exact string.
2. **No Pages, no Grep.**
- If the rune count of `Full(doc)` is at most `OutCap`: `Content = Full(doc)`, `First = 1`,
`Last` = pages extracted.
- Otherwise `Map = true` and the content is, in this order:
```
# <Title, or "Untitled PDF">
<Pages> pages · ~<T> tokens[ · pages 1-<E> extracted]
## Outline (only when the outline is not empty)
- <title> (p. <page>) (one line per item, indented two spaces per level below 1)
This document is longer than one answer: ask for pages="a-b" or grep="pattern".
<leading pages>
```
- `T` is the rune count of `Full(doc)` divided by 4.
- The `pages 1-E extracted` part appears only when fewer pages were extracted than
`Pages`. The separator is " · " (U+00B7).
- Leading pages are whole pages rendered as in `Full`, from page 1, as many as fit in
`OutCap/4` runes. At least page 1 is shown, cut to fit if it alone is larger.
- The whole content is at most `OutCap` runes; if the outline is too long, cut the outline,
not the header or the instruction line.
- `First = 1`, `Last` = the last whole page shown.
3. **Pages.**
- Parse after trimming spaces: `a-b`, `a`, `a-` (a to the last extracted page). Anything else
→ `ErrBadPages`.
- `a < 1`, `b < a`, or `a` past the last extracted page → an error wrapping `ErrBadPages`
whose text is `pages out of range (1-E)`. `b` past the end is clamped to `E`.
- At most **100 pages** and at most `OutCap` runes: whole pages from `a`. A single page larger
than `OutCap` is cut to fit.
- When the range was cut, `Note` says so and names the rest as `pages="k-b"`: for example
`cut at page 13; ask for pages="14-40"`.
4. **Grep.**
- The pattern must be at most 200 bytes and compile with `regexp` (RE2), else an error
wrapping `ErrBadGrep`. Prefix `(?i)` unless the pattern contains an upper-case letter
("smart case").
- Search every line of every page in the range (all extracted pages when `Pages` is empty).
- The first line of the content is `grep "<pattern>": N matches` (`1 match` for one; `no matches`
for none). Add `, showing the first 100` when N is over 100.
- Then, per match, up to the first 100: the previous line of the same page as
`p. P- <line>`, the matching line as `p. P: <line>`, the next line as `p. P- <line>`,
then a `--` line. Lines are trimmed of surrounding spaces.
- `GrepHits = N`, `First = Last = 0`. The content is at most `OutCap` runes (drop whole
match blocks at the end and say so in `Note`). No match is not an error.
5. Every length here is in **runes**, never bytes: the cap protects an agent's context, and CJK
text is three bytes a rune.
## Steps
- [ ] **1.** `git switch p5a`; copy the given test. **2. See it fail** (undefined names).
- [ ] **3.** `pdfdoc.go`, **4.** `grep.go`.
- [ ] **5.** `gofmt -w`; `go test -race -count=3 ./internal/pdfdoc/` → `ok`.
- [ ] **6.** `make gate`. **7.** Row `p5a/05-pdfdoc`; commit.
```sh
git add internal/pdfdoc docs/implementer-log.md
git commit -F .state/commit-msg.txt
```
## Done when
- The given test passes; `make gate` ok; given file byte-identical.
docs/plans/p5a/06-api.md
+150
View File
@@ -0,0 +1,150 @@
# P5a task 06: PDFs through `/v1/fetch` and `/v2/scrape`
**Branch:** `p5a` (`git switch p5a`; `git status --short` must be empty, otherwise stop)
**Commit subject:** `api: serve PDFs — child extraction via the pool, document cache, map, pages, grep`
## Goal
Wire tasks 01–05 into the API. A PDF URL fetched through `/v1/fetch` or `/v2/scrape` is:
1. extracted once in the child, under the pool;
2. cached per page;
3. answered with the whole text, the map, a page range, or grep results.
Later requests for the same document are served from the cache, revalidated after 24 h with a
conditional GET. `question` digests the document's text, page markers included, unless the
document is too long.
## Files
- Copy: `internal/api/pdf_test.go`
- Create: `internal/api/pdf.go` — everything PDF lives here
- Modify: `internal/api/api.go` — **only** the hooks listed in rule 7. `api.go` is already 840
lines; do not move code into it.
- Modify: `docs/implementer-log.md`
## Interfaces
```go
package api
const (
DefaultPDFTTL = 24 * time.Hour
DefaultPDFMaxQuestionChunks = 200
)
// in Server:
PDFRunner *pdftext.Runner // nil = &pdftext.Runner{} (the child is /proc/self/exe pdftext)
PDFPool *pdfpool.Pool // nil = pdfpool.New(2, 4, 20*time.Second)
PDFTTL time.Duration // 0 = DefaultPDFTTL
PDFMaxQuestionChunks int // 0 = DefaultPDFMaxQuestionChunks
// in v1Req:
Pages string `json:"pages"`
Grep string `json:"grep"`
```
Defaults are applied where the field is read: a zero `Server` from `New` must work. Build the
default pool once, with a `sync.Once` — never one per request.
## Rules
1. **Cache first.**
- Before fetching, look up `dcache.PDFKey(rawURL, s.callerClass(c))` in `s.cache()` (which
may be nil: then there is no cache).
- **Fresh** (now − `FetchedAt` < TTL): build the `*pdftext.Doc` from the entry (unmarshal
`Meta`; `Text` = `Pages`). Answer with no network call and no child. The source is marked
`"cached": true`.
- **Stale:** `s.Fetch.FetchIf(ctx, rawURL, e.ETag, e.LastModified)`.
- 304 → `TouchPDF(key, now)` and answer from the entry, with no child.
- 200 → go on as for a new document.
- **Not cached:** the usual `s.Fetch.Fetch`, under the existing `acquire`/`release`.
2. **A PDF** is `clean.Kind(res.ContentType, res.Body) == "pdf"`. Every other kind goes through
today's path unchanged. Use `clean.Kind` in place of the `MediaKind`/`Sniff` pair in both
`doFetch` and `cleanFetched`. `"other"` stays `415`.
3. **Extraction.**
- Call `s.PDFPool.Do(ctx, class+"|"+res.SHA256(), c.Name, fn)`, where `fn` runs
`s.PDFRunner.Run(jobCtx, res.Body)`.
- The key is the **content hash**, so identical bytes are extracted once however many callers
or URLs ask. The key includes the caller class, so a restricted caller never shares a job
with the fleet.
- Then `PutPDF` under the URL key with: the content hash; the response's `ETag` and
`Last-Modified`; `FetchedAt` = now; `Meta` = the JSON of `doc.Meta`; `Pages` = `doc.Text`.
4. **Errors**, all through the existing `writeErr` shape (`{"success":false,"error":…,"hint":…}`):
| cause | status | error | hint |
|---|---|---|---|
| `pdftext.ErrNoText` | 422 | `no text layer (scanned PDF?)` | `ocr` |
| `pdftext.ErrEncrypted` | 422 | `encrypted PDF (password required)` | |
| `pdftext.ErrUnreadable` | 422 | `unreadable PDF` | |
| `pdftext.ErrTimeout` | 422 | `pdf extraction timed out` | |
| `pdftext.ErrTooMuchText` | 422 | `pdf text over the output limit` | |
| `pdfpool.ErrBusy` | 503 | `pdf workers busy`, **header `Retry-After: 10`** | |
| `pdfpool.ErrCallerBusy` | 429 | `busy` | |
| `pdfdoc.ErrBadPages` / `ErrBadGrep` | 400 | the error's text | |
| `pages` or `grep` on a non-PDF | 400 | `pages and grep apply to PDFs only` | |
| `question` on a document over `PDFMaxQuestionChunks` chunks | 422 | `document too long for question (N chunks); use grep or pages` | |
A PDF path never answers 500 for anything the document did.
5. **`/v1/fetch`.**
- **Mode `clean`:** `content` is `pdfdoc.Render(doc, {Pages, Grep, OutCap: s.OutCap}).Content`.
- The envelope gains:
```
"pdf": {"pages", "title", "outline", "extracted", "first", "last", "map", "grep_hits", "note"}
```
`extracted` is `len(doc.Text)`; `outline` is `doc.Outline`; the rest come from `Rendered`.
- **Mode `digest`:** the text is `pdfdoc.Full(doc)`. First count
`len(digest.Split(full, digest.ChunkTokens))`; if it is over `PDFMaxQuestionChunks`, answer
the 422 above **before** any model call. Otherwise run the digest as for any page, with
`full` as the content.
- **Mode `raw`** on a PDF: 400 `raw mode does not apply to PDFs`.
- `sizes.raw_tokens` is the rune count of `Full(doc)` / 4 for a PDF (there is no body on a
cache hit).
6. **`/v2/scrape`.**
- `markdown` is `Render(doc, {OutCap: s.OutCap}).Content`; `html` is `""`.
- `metadata.title` is `doc.Title`; `metadata.numPages` is `doc.Pages`; `metadata.contentType`
is the response's (`application/pdf` on a cache hit).
7. **`api.go` hooks (the only edits allowed there):**
- the four `Server` fields and two `v1Req` fields;
- the `Kind` switch in `doFetch`/`cleanFetched`, now returning the doc;
- one call into `pdf.go` in `v1fetch` and in `scrape`;
- `Retry-After` in `writeFetchErr` for a 503.
`doFetch` and `cleanFetched` may gain a result, `doc *pdftext.Doc` (nil unless a PDF).
8. **Logging:**
- Keep today's `fetch`/`scrape` log lines and add `pdf_pages` (a count) to them.
- Never log the title, the outline or any text.
- A restricted caller's lines go through the existing helpers (host blanked); the given test
reads every log record.
## Facts
- The tests set `s.Digest = &digest.Engine{Cache: db}` only to give `s.cache()` a database: no
model is configured.
- The child in the tests is the test binary (`TestMain` in the given file).
- `setupAPI`, `postJSON` and `recHandler` come from `api_test.go` in the same package.
- `requestDeadline` (210 s) and `writeTimeout` (240 s) were raised by the owner for this path:
download ≤ 120 s + queue ≤ 20 s + extraction ≤ 60 s.
## Steps
- [ ] **1.** `git switch p5a`; copy the given test. **2. See it fail** (undefined fields).
- [ ] **3.** `pdf.go` (cache, extraction, errors, render), then the `api.go` hooks.
- [ ] **4.** `gofmt -w`; `go test -race -count=3 ./internal/api/` → `ok`: the new tests and
every existing api test (search, scrape, restricted logging, write timeout).
- [ ] **5.** `make gate`. **6.** Row `p5a/06-api`; commit.
```sh
git add internal/api docs/implementer-log.md
git commit -F .state/commit-msg.txt
```
## Done when
- `go test -race -count=3 ./internal/api/` is ok; `make gate` ok; given file byte-identical;
`api.go` grew by the hooks only (the owner will diff it).
## Stop and report if
- A rule here contradicts an existing api test.
- The pool's default wait (20 s) makes a test take longer than the test allows. The given tests
set their own pool where timing matters.
docs/plans/p5a/07-digest-cli-docs.md
+108
View File
@@ -0,0 +1,108 @@
# P5a task 07: page citations, metrics, CLI flags, skill, unit, README
**Branch:** `p5a` (`git switch p5a`; `git status --short` must be empty, otherwise stop)
**Commit subject:** `P5a: digests cite pages; sift_pdf_* metrics; sift fetch --pages/--grep; skill, unit and README`
## Goal
Finish P5a:
- digests of PDFs cite `(p. N)`;
- the PDF path is visible in `/metrics`;
- `sift fetch` can ask for pages and grep;
- the agents' skill tells them PDFs work;
- the unit gets the memory the child needs;
- the README describes all of it.
## Files
- Copy: `internal/digest/pages_test.go`, `internal/metrics/pdf_test.go`, `cmd/sift/fetchpdf_test.go`
- Modify:
- `internal/digest/digest.go` (prompt line, `PromptVersion`)
- `internal/metrics/metrics.go`
- `internal/api/pdf.go` (the metric calls)
- `cmd/sift/fetchcmd.go`
- `skill/SKILL.md`
- `nix/module.nix` (**only** `MemoryMax`)
- `README.md`
- `docs/implementer-log.md`
## Rules
1. **Digest.**
- Add one sentence to `systemPrompt`: `When the document has "--- page N ---" markers, cite
the page of each fact as (p. N).`
- Set `PromptVersion = "3"`. The digest cache keys on it, so digests made with the old
prompt are not served.
- `Split` already keeps markers whole; the given test pins that.
2. **Metrics** (`metrics.go`):
- `func (s *Set) ObservePDF(outcome string, d time.Duration, pages int)`. It feeds:
- the counter `sift_pdf_extract_total{outcome}`;
- the histogram `sift_pdf_extract_seconds`;
- the counter `sift_pdf_pages_total`.
`outcome` must be one of `ok`, `no_text`, `encrypted`, `unreadable`, `timeout`,
`too_large`, `busy`; anything else is counted as `other`. The label never carries raw
text: the given test sends `blocked: evil.example`.
- `func (s *Set) ObservePDFWait(d time.Duration)` feeds the histogram
`sift_pdf_pool_wait_seconds`.
- `func (s *Set) PDFCacheHit()` feeds the counter `sift_pdf_cache_hits_total`.
- Register them on the Set's own registry, like `sift_requests_total`.
- In `internal/api/pdf.go`:
- call `ObservePDF` once per extraction attempt, with the outcome and the pages extracted;
- call `ObservePDFWait` with the time spent waiting in `PDFPool.Do` before the job
started (measure around `Do`: total minus the job's own time);
- call `PDFCacheHit` for every answer served from the cache, fresh or after a 304.
3. **CLI** (`fetchcmd.go`).
- Parsing becomes a struct, and today's tuple function stays as a wrapper so the existing
tests keep compiling:
```go
type fetchOpts struct {
URL, Mode, JQ, Question, Pages, Grep string
NoContext, Recover, JSON bool
}
func parseFetchOpts(args []string) (fetchOpts, error) // the parser; knows --pages and --grep (both take a value)
func parseFetchArgs(args []string) (…the same 8 results as today…) // calls parseFetchOpts
func fetchBody(o fetchOpts) map[string]any // the /v1/fetch request body
```
- `fetchBody` builds today's body: url, mode (defaulting to digest when there is a question,
else clean), jq, question, context (question set and not `--no-context`), recover. It adds
`"pages"` and `"grep"` **only when set**.
- The command uses `parseFetchOpts` and `fetchBody`.
- Add `[--pages a-b] [--grep RE]` to `fetchUsage`.
- The given `cmd/sift/fetchpdf_test.go` and the existing flag tests must pass.
4. **Skill** (`skill/SKILL.md`):
- Remove the `415 unsupported type` row's mention of PDF.
- Add a short "PDFs" section:
- PDFs work;
- a long one answers with a map (outline with page numbers, first pages);
- then use `--pages 41-60` or `--grep 'pattern'`, or `--question` for a digest that cites
pages;
- scanned PDFs return `no text layer`;
- never download a PDF and run `pdftotext` yourself.
5. **Unit** (`nix/module.nix`): `MemoryMax = "1G";` (was `"512M"`). Nothing else in the file
changes.
6. **README:**
- a "PDF documents" section covering: detection, limits (100 MB, 120 s download, 60 s
extraction, 2,000 pages), the child process, the pool (2 slots, 4 waiting, 20 s), the
cache (24 h, conditional GET), the answers (full / map / pages / grep / digest with
citations), the errors table from task 06, and the metrics;
- note that the engine is `go-pdfkit/extract` v0.3.0, chosen by the bake-off in `docs/P5a.md` §3.
## Steps
- [ ] **1.** `git switch p5a`; copy the three given tests. **2. See them fail.**
- [ ] **3.** digest, **4.** metrics and the calls in `pdf.go`, **5.** CLI, **6.** skill, unit,
README.
- [ ] **7.** `gofmt -w`; `make gate`.
- [ ] **8.** `make build && bin/sift fetch 2>&1 | grep -c -- '--pages'` prints `1`.
- [ ] **9.** Row `p5a/07-digest-cli-docs`; commit.
```sh
git add internal/digest internal/metrics internal/api/pdf.go cmd/sift skill nix/module.nix README.md docs/implementer-log.md
git commit -F .state/commit-msg.txt
```
## Done when
- All tests pass; `make gate` ok; given files byte-identical; the `nix/module.nix` diff is the
one line.
docs/plans/p5a/README.md
+46
View File
@@ -0,0 +1,46 @@
# P5a implementation plan: PDF documents
> **For the implementing model:** do not work from this file. The owner gives you one task file
> at a time. This file is the index for the owner and the reviewer.
**Goal:** `docs/P5a.md`. sift fetches, extracts and serves PDFs, so agents stop running
`pdftotext` in the shell. Engine: `github.com/go-pdfkit/extract` v0.3.0 (bake-off, P5a §3),
vendored by the owner before the run.
| Task | Package(s) | Given |
|---|---|---|
| 01-kind-fetch | `internal/clean`, `internal/fetch` | `clean/kind_test.go`, `fetch/pdf_test.go` |
| 02-pdftext | `internal/pdftext`, `cmd/sift` (one dispatch line) | `pdftext/pdftext_test.go`, `internal/pdftest/pdftest.go`, `tests/testdata/pdf/*` |
| 03-dcache | `internal/dcache` | `dcache/pdf_test.go` |
| 04-pdfpool | `internal/pdfpool` | `pdfpool/pdfpool_test.go` |
| 05-pdfdoc | `internal/pdfdoc` | `pdfdoc/pdfdoc_test.go` |
| 06-api | `internal/api` | `api/pdf_test.go` |
| 07-digest-cli-docs | `internal/digest`, `internal/metrics`, `cmd/sift`, `internal/api` (metric calls), `skill/`, `nix/module.nix`, `README.md` | `digest/pages_test.go`, `metrics/pdf_test.go` |
**Order matters:** 05 imports 02's types; 06 uses 01–05; 07 wires metrics into 06's file.
**How this plan was made:** acceptance tests first, no reference implementation. The given tests
compiled against a panic-only skeleton of every new name, on this repository at the plan
commit, and failed there for the intended reasons; every earlier test stayed green. The PDFs the
tests use are built by `internal/pdftest` (a given file) at test time. The only binary fixtures
are two small qpdf-encrypted copies of the three-page test document (`tests/testdata/pdf/README.md`).
**Owner changes made before the run** (plan commit):
- `go.mod`, `go.sum`, `vendor/`: `go-pdfkit/extract` v0.3.0, `go-pdfkit/reader` v0.6.0 (and its
`pdffont`), pinned through `tools.go` until task 02 imports them. `flake.nix` uses
`vendorHash = null`, so the nix build reads `vendor/`.
- `internal/api/api_test.go`: `TestUnsupportedType` now uses `application/zip`; a PDF is no
longer unsupported.
- `gofmt -w` on `internal/clean/clean.go`, `internal/guard/guard_test.go`,
`internal/guard/iana.go` (whitespace only), so the new `make gate` target passes on master.
- `AGENTS.md`, `tools/run-plan.sh`, `docs/implementer-log.md`, `make gate`.
**Attribution:** `tools/run-plan.sh` gives the model the exact trailer
`Co-Authored-By: <model> <<model>@llama.cpp.invalid>` and refuses a commit without it.
**Where it runs:** a separate checkout (`~/src/sift-impl` on straylight), branch `p5a`. Nothing
reaches orion until the owner's review, the merge, and straylight's pin bump.
## Changes during the run
(none yet)
docs/plans/p5a/_files/cmd/sift/fetchpdf_test.go
+44
View File
@@ -0,0 +1,44 @@
package main
// P5a task 07: `sift fetch --pages` and `--grep`.
import (
"encoding/json"
"strings"
"testing"
)
func TestParseFetchOptsPDF(t *testing.T) {
o, err := parseFetchOpts([]string{"https://example.org/t.pdf", "--pages", "41-60", "--grep", "attention head"})
if err != nil {
t.Fatal(err)
}
if o.URL != "https://example.org/t.pdf" || o.Pages != "41-60" || o.Grep != "attention head" {
t.Errorf("got %+v", o)
}
o, err = parseFetchOpts([]string{"--grep", "x", "--question", "why", "https://example.org/t.pdf"})
if err != nil || o.URL != "https://example.org/t.pdf" || o.Grep != "x" || o.Question != "why" {
t.Errorf("flags before the URL: %+v %v", o, err)
}
if !strings.Contains(fetchUsage, "--pages") || !strings.Contains(fetchUsage, "--grep") {
t.Errorf("usage lacks the new flags: %s", fetchUsage)
}
}
func TestFetchBodyPDFFields(t *testing.T) {
b, _ := json.Marshal(fetchBody(fetchOpts{URL: "https://example.org/a.pdf"}))
if strings.Contains(string(b), `"pages"`) || strings.Contains(string(b), `"grep"`) {
t.Errorf("unset pages/grep must not be sent: %s", b)
}
b, _ = json.Marshal(fetchBody(fetchOpts{URL: "https://example.org/a.pdf", Pages: "2-3", Grep: "fox"}))
var m map[string]any
_ = json.Unmarshal(b, &m)
if m["pages"] != "2-3" || m["grep"] != "fox" || m["url"] != "https://example.org/a.pdf" || m["mode"] != "clean" {
t.Errorf("body = %s", b)
}
b, _ = json.Marshal(fetchBody(fetchOpts{URL: "u", Question: "q"}))
_ = json.Unmarshal(b, &m)
if m["mode"] != "digest" || m["question"] != "q" || m["context"] != true {
t.Errorf("question body = %s (mode must default to digest, context on)", b)
}
}
docs/plans/p5a/_files/internal/api/pdf_test.go
+402
View File
@@ -0,0 +1,402 @@
package api
// P5a task 06: PDFs through /v1/fetch and /v2/scrape — extraction in the child, the pool, the
// document cache, the map, pages, grep, and the errors.
//
// The child is the test binary itself (TestMain below), as in internal/pdftext's tests.
// SIFT_PDFTEXT_RUNLOG makes every child start append a line to a file, so a test can count
// extractions; SIFT_PDFTEXT_SLEEP slows a child down. Both hooks live only in this file.
import (
"encoding/json"
"fmt"
"io"
"log/slog"
"net/http"
"net/http/httptest"
"net/netip"
"os"
"path/filepath"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
"git.wntrmute.dev/kyle/sift/internal/auth"
"git.wntrmute.dev/kyle/sift/internal/dcache"
"git.wntrmute.dev/kyle/sift/internal/digest"
"git.wntrmute.dev/kyle/sift/internal/fetch"
"git.wntrmute.dev/kyle/sift/internal/guard"
"git.wntrmute.dev/kyle/sift/internal/pdfpool"
"git.wntrmute.dev/kyle/sift/internal/pdftest"
"git.wntrmute.dev/kyle/sift/internal/pdftext"
)
func TestMain(m *testing.M) {
if os.Getenv("SIFT_PDFTEXT_CHILD") == "1" {
if p := os.Getenv("SIFT_PDFTEXT_RUNLOG"); p != "" {
f, err := os.OpenFile(p, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0o600)
if err == nil {
_, _ = f.WriteString("run\n")
_ = f.Close()
}
}
if d, err := time.ParseDuration(os.Getenv("SIFT_PDFTEXT_SLEEP")); err == nil {
time.Sleep(d)
}
os.Exit(pdftext.ChildMain(os.Args[1:], os.Stdin, os.Stdout))
}
os.Exit(m.Run())
}
func threePDF() []byte {
return pdftest.Build(pdftest.Doc{
Title: "Three Pages",
Pages: []string{"Chapter one begins here.", "The quick brown fox is on page two.", "Last page."},
Outline: []pdftest.Outline{{Title: "1 Intro", Page: 1, Level: 1}, {Title: "2 Fox", Page: 2, Level: 1}},
})
}
func longPDF(n int) []byte {
var pages []string
for p := 1; p <= n; p++ {
pages = append(pages, pdftest.Lorem(p, 40))
}
return pdftest.Build(pdftest.Doc{Title: "Long Thesis", Pages: pages, Outline: []pdftest.Outline{{Title: "1 Introduction", Page: 1, Level: 1}, {Title: "2 Method", Page: n / 2, Level: 1}}})
}
// origin serves one PDF and counts GETs. It honours If-None-Match against its current ETag.
type origin struct {
mu sync.Mutex
ct string
body []byte
etag string
gets atomic.Int32
srv *httptest.Server
}
func newOrigin(t *testing.T, ct string, body []byte) *origin {
o := &origin{ct: ct, body: body, etag: `"v1"`}
o.srv = httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
o.gets.Add(1)
o.mu.Lock()
defer o.mu.Unlock()
if r.Header.Get("If-None-Match") == o.etag {
w.WriteHeader(http.StatusNotModified)
return
}
w.Header().Set("ETag", o.etag)
if o.ct != "" {
w.Header().Set("Content-Type", o.ct)
}
_, _ = w.Write(o.body)
}))
t.Cleanup(o.srv.Close)
return o
}
func (o *origin) replace(body []byte, etag string) {
o.mu.Lock()
o.body, o.etag = body, etag
o.mu.Unlock()
}
type pdfRig struct {
s *Server
fleet string // bearer key of caller "test" (hermes-fleet)
other string // bearer key of caller "other" (hermes-fleet)
gs string // bearer key of restricted caller "straylight-gs"
runlog string
logs *recHandler
}
func newPDFRig(t *testing.T, childEnv ...string) *pdfRig {
t.Helper()
dir := t.TempDir()
keys := filepath.Join(dir, "keys.toml")
key := func(name, ws string) string {
out := filepath.Join(dir, name)
if err := auth.Add(keys, name, ws, out); err != nil {
t.Fatal(err)
}
b, err := os.ReadFile(out)
if err != nil {
t.Fatal(err)
}
return strings.TrimSpace(string(b))
}
r := &pdfRig{fleet: key("test", "hermes-fleet"), other: key("other", "hermes-fleet"), gs: key("straylight-gs", "guardian-star"), runlog: filepath.Join(dir, "runs"), logs: &recHandler{}}
store, err := auth.Load(keys)
if err != nil {
t.Fatal(err)
}
g := &guard.Guard{AllowPrefixes: []netip.Prefix{netip.MustParsePrefix("127.0.0.0/8")}}
s := New("test", g, &fetch.Fetcher{Guard: g, UA: "sift-test"}, store, slog.New(r.logs))
s.PermitRestricted = []string{"straylight-gs"}
db, err := dcache.Open(filepath.Join(dir, "cache.db"), 0, nil)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = db.Close() })
s.Digest = &digest.Engine{Cache: db} // the cache lives on the digest engine; no model is configured
s.PDFRunner = &pdftext.Runner{Exe: os.Args[0], Env: append([]string{"SIFT_PDFTEXT_CHILD=1", "SIFT_PDFTEXT_RUNLOG=" + r.runlog}, childEnv...), Deadline: 20 * time.Second}
r.s = s
return r
}
func (r *pdfRig) runs() int {
b, _ := os.ReadFile(r.runlog)
return strings.Count(string(b), "run\n")
}
func (r *pdfRig) fetch(t *testing.T, key string, body map[string]any) (int, map[string]any, http.Header) {
t.Helper()
rec := postJSON(t, r.s, "/v1/fetch", key, body)
var out map[string]any
_ = json.Unmarshal(rec.Body.Bytes(), &out)
return rec.Code, out, rec.Header()
}
func pdfInfo(t *testing.T, out map[string]any) map[string]any {
t.Helper()
p, ok := out["pdf"].(map[string]any)
if !ok {
t.Fatalf("envelope has no pdf object: %v", out)
}
return p
}
func TestPDFCleanFetch(t *testing.T) {
for _, ct := range []string{"application/pdf", "application/octet-stream", ""} {
r := newPDFRig(t)
o := newOrigin(t, ct, threePDF())
code, out, _ := r.fetch(t, r.fleet, map[string]any{"url": o.srv.URL})
if code != 200 {
t.Fatalf("%q: %d %v", ct, code, out)
}
content, _ := out["content"].(string)
if !strings.Contains(content, "--- page 2 ---") || !strings.Contains(content, "quick brown fox") || out["mode"] != "clean" {
t.Errorf("%q: content %q mode %v", ct, content, out["mode"])
}
p := pdfInfo(t, out)
if p["pages"] != 3.0 || p["title"] != "Three Pages" || p["map"] != false || p["first"] != 1.0 || p["last"] != 3.0 {
t.Errorf("%q: pdf = %v", ct, p)
}
if ol, _ := p["outline"].([]any); len(ol) != 2 {
t.Errorf("%q: outline = %v", ct, p["outline"])
}
}
}
func TestPDFMap(t *testing.T) {
r := newPDFRig(t)
r.s.OutCap = 20000
o := newOrigin(t, "application/pdf", longPDF(40)) // ~40 × 2,700 characters
code, out, _ := r.fetch(t, r.fleet, map[string]any{"url": o.srv.URL})
if code != 200 {
t.Fatalf("%d %v", code, out)
}
content, _ := out["content"].(string)
p := pdfInfo(t, out)
if p["map"] != true || !strings.Contains(content, "## Outline") || !strings.Contains(content, "- 2 Method (p. 20)") || len([]rune(content)) > 20000 {
t.Errorf("map: pdf %v, %d chars:\n%.400s", p, len([]rune(content)), content)
}
}
func TestPDFPagesAndGrep(t *testing.T) {
r := newPDFRig(t)
o := newOrigin(t, "application/pdf", threePDF())
code, out, _ := r.fetch(t, r.fleet, map[string]any{"url": o.srv.URL, "pages": "2-3"})
content, _ := out["content"].(string)
if code != 200 || strings.Contains(content, "--- page 1 ---") || !strings.Contains(content, "--- page 3 ---") {
t.Errorf("pages 2-3: %d %q", code, content)
}
if p := pdfInfo(t, out); p["first"] != 2.0 || p["last"] != 3.0 {
t.Errorf("pages 2-3: pdf %v", p)
}
for _, bad := range []map[string]any{{"pages": "0-1"}, {"pages": "9"}, {"grep": strings.Repeat("a", 201)}, {"grep": "(unclosed"}} {
bad["url"] = o.srv.URL
if code, out, _ := r.fetch(t, r.fleet, bad); code != 400 || out["success"] != false {
t.Errorf("%v: %d %v, want 400", bad, code, out)
}
}
code, out, _ = r.fetch(t, r.fleet, map[string]any{"url": o.srv.URL, "grep": "quick brown"})
content, _ = out["content"].(string)
if code != 200 || !strings.Contains(content, "p. 2: The quick brown fox is on page two.") || pdfInfo(t, out)["grep_hits"] != 1.0 {
t.Errorf("grep: %d %q %v", code, content, out["pdf"])
}
// pages and grep on a non-PDF are refused rather than ignored.
html := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set("Content-Type", "text/html")
_, _ = io.WriteString(w, "<html><body><p>hi</p></body></html>")
}))
defer html.Close()
if code, out, _ := r.fetch(t, r.fleet, map[string]any{"url": html.URL, "pages": "1"}); code != 400 {
t.Errorf("pages on html: %d %v, want 400", code, out)
}
}
func TestPDFCache(t *testing.T) {
r := newPDFRig(t)
o := newOrigin(t, "application/pdf", threePDF())
get := func(extra map[string]any) string {
body := map[string]any{"url": o.srv.URL}
for k, v := range extra {
body[k] = v
}
code, out, _ := r.fetch(t, r.fleet, body)
if code != 200 {
t.Fatalf("%v: %d %v", extra, code, out)
}
c, _ := out["content"].(string)
return c
}
get(nil)
if o.gets.Load() != 1 || r.runs() != 1 {
t.Fatalf("first fetch: %d GETs, %d extractions; want 1 and 1", o.gets.Load(), r.runs())
}
if c := get(map[string]any{"pages": "2"}); !strings.Contains(c, "quick brown fox") {
t.Errorf("cached pages=2: %q", c)
}
if o.gets.Load() != 1 || r.runs() != 1 {
t.Errorf("fresh cache hit: %d GETs, %d extractions; want still 1 and 1", o.gets.Load(), r.runs())
}
// Stale: a conditional GET, answered 304, re-uses the text.
r.s.PDFTTL = time.Nanosecond
time.Sleep(2 * time.Millisecond)
get(map[string]any{"grep": "Chapter"})
if o.gets.Load() != 2 || r.runs() != 1 {
t.Errorf("after 304: %d GETs, %d extractions; want 2 and 1", o.gets.Load(), r.runs())
}
// The document changed: a full GET and a new extraction.
o.replace(pdftest.Build(pdftest.Doc{Pages: []string{"Brand new text."}}), `"v2"`)
if c := get(nil); !strings.Contains(c, "Brand new text.") {
t.Errorf("changed document: %q", c)
}
if o.gets.Load() != 3 || r.runs() != 2 {
t.Errorf("after change: %d GETs, %d extractions; want 3 and 2", o.gets.Load(), r.runs())
}
if DefaultPDFTTL != 24*time.Hour {
t.Errorf("DefaultPDFTTL = %v, want 24h", DefaultPDFTTL)
}
}
func TestPDFErrors(t *testing.T) {
r := newPDFRig(t)
enc, err := os.ReadFile(filepath.Join("..", "..", "tests", "testdata", "pdf", "encrypted-user.pdf"))
if err != nil {
t.Fatal(err)
}
for _, tc := range []struct {
name string
body []byte
want string
hint string
}{
{"scanned", pdftest.Build(pdftest.Doc{Pages: []string{"", ""}, Scanned: []int{1, 2}}), "no text layer", "ocr"},
{"encrypted", enc, "encrypted", ""},
{"garbage", []byte("%PDF-1.4\nthis is not really a pdf"), "unreadable", ""},
} {
o := newOrigin(t, "application/pdf", tc.body)
code, out, _ := r.fetch(t, r.fleet, map[string]any{"url": o.srv.URL})
msg, _ := out["error"].(string)
if code != 422 || !strings.Contains(msg, tc.want) || (tc.hint != "" && out["hint"] != tc.hint) {
t.Errorf("%s: %d %v; want 422 %q hint %q", tc.name, code, out, tc.want, tc.hint)
}
}
}
func TestPDFTimeout(t *testing.T) {
r := newPDFRig(t, "SIFT_PDFTEXT_SLEEP=10s")
r.s.PDFRunner.Deadline = 300 * time.Millisecond
o := newOrigin(t, "application/pdf", threePDF())
code, out, _ := r.fetch(t, r.fleet, map[string]any{"url": o.srv.URL})
if msg, _ := out["error"].(string); code != 422 || !strings.Contains(msg, "timed out") {
t.Errorf("slow child: %d %v; want 422 … timed out", code, out)
}
}
func TestPDFBusy(t *testing.T) {
r := newPDFRig(t, "SIFT_PDFTEXT_SLEEP=1500ms")
r.s.PDFPool = pdfpool.New(1, 0, 100*time.Millisecond)
a := newOrigin(t, "application/pdf", threePDF())
b := newOrigin(t, "application/pdf", longPDF(2))
done := make(chan int, 1)
go func() {
code, _, _ := r.fetch(t, r.fleet, map[string]any{"url": a.srv.URL})
done <- code
}()
time.Sleep(400 * time.Millisecond) // the first extraction holds the only slot
code, out, hdr := r.fetch(t, r.other, map[string]any{"url": b.srv.URL})
if code != 503 || hdr.Get("Retry-After") != "10" || !strings.Contains(fmt.Sprint(out["error"]), "busy") {
t.Errorf("second caller with the pool full: %d %v Retry-After %q; want 503, busy, 10", code, out, hdr.Get("Retry-After"))
}
code, out, _ = r.fetch(t, r.fleet, map[string]any{"url": b.srv.URL})
if code != 429 {
t.Errorf("same caller, second document while its first is running: %d %v; want 429", code, out)
}
if c := <-done; c != 200 {
t.Errorf("first fetch: %d", c)
}
}
func TestPDFRestrictedCaller(t *testing.T) {
r := newPDFRig(t)
o := newOrigin(t, "application/pdf", threePDF())
if code, out, _ := r.fetch(t, r.gs, map[string]any{"url": o.srv.URL}); code != 200 {
t.Fatalf("restricted fetch: %d %v", code, out)
}
r.logs.mu.Lock()
for _, rec := range r.logs.recs {
var b strings.Builder
b.WriteString(rec.Message)
rec.Attrs(func(a slog.Attr) bool { fmt.Fprintf(&b, " %s=%v", a.Key, a.Value); return true })
line := b.String()
if strings.Contains(line, "127.0.0.1") || strings.Contains(line, "Three Pages") || strings.Contains(line, "quick brown") {
t.Errorf("a restricted caller's PDF fetch logged %q", line)
}
}
r.logs.mu.Unlock()
// The fleet does not see the restricted caller's cached text: it extracts its own copy.
if code, _, _ := r.fetch(t, r.fleet, map[string]any{"url": o.srv.URL}); code != 200 || r.runs() != 2 {
t.Errorf("fleet after restricted: %d, %d extractions; want 200 and 2 (separate cache partitions)", code, r.runs())
}
}
func TestPDFScrape(t *testing.T) {
r := newPDFRig(t)
o := newOrigin(t, "application/pdf", threePDF())
rec := postJSON(t, r.s, "/v2/scrape", r.fleet, map[string]any{"url": o.srv.URL, "formats": []string{"markdown"}})
var out struct {
Success bool `json:"success"`
Data struct {
Markdown string `json:"markdown"`
Metadata map[string]any `json:"metadata"`
} `json:"data"`
}
if err := json.Unmarshal(rec.Body.Bytes(), &out); err != nil || rec.Code != 200 || !out.Success {
t.Fatalf("%d %s", rec.Code, rec.Body.String())
}
if !strings.Contains(out.Data.Markdown, "--- page 2 ---") || out.Data.Metadata["numPages"] != 3.0 || out.Data.Metadata["title"] != "Three Pages" {
t.Errorf("scrape: markdown %q metadata %v", out.Data.Markdown, out.Data.Metadata)
}
if ct, _ := out.Data.Metadata["contentType"].(string); !strings.HasPrefix(ct, "application/pdf") {
t.Errorf("contentType = %q", ct)
}
}
// A question on a document longer than PDFMaxQuestionChunks is refused with a pointer to grep
// and pages, before any model call.
func TestPDFQuestionTooLong(t *testing.T) {
r := newPDFRig(t)
r.s.PDFMaxQuestionChunks = 2
o := newOrigin(t, "application/pdf", longPDF(12))
code, out, _ := r.fetch(t, r.fleet, map[string]any{"url": o.srv.URL, "question": "what is the method?"})
if msg, _ := out["error"].(string); code != 422 || !strings.Contains(msg, "too long") || !strings.Contains(msg, "grep") {
t.Errorf("long question: %d %v; want 422 … too long … grep", code, out)
}
if DefaultPDFMaxQuestionChunks != 200 {
t.Errorf("DefaultPDFMaxQuestionChunks = %d, want 200", DefaultPDFMaxQuestionChunks)
}
}
docs/plans/p5a/_files/internal/clean/kind_test.go
+55
View File
@@ -0,0 +1,55 @@
package clean_test
// P5a task 01: Kind decides which cleaner a fetched body gets, now including "pdf".
import (
"strings"
"testing"
"git.wntrmute.dev/kyle/sift/internal/clean"
)
func TestKind(t *testing.T) {
pdf := []byte("%PDF-1.7\n%\xe2\xe3\xcf\xd3\n1 0 obj << >> endobj\n")
junkThenPDF := append([]byte(strings.Repeat(" ", 900)), pdf...) // the spec allows leading junk
lateMagic := append([]byte(strings.Repeat("x", 2000)), pdf...) // past the first 1024 bytes
binary := []byte{0x50, 0x4b, 0x03, 0x04, 0x00, 0x00} // a zip
for _, tc := range []struct {
name, ct string
body []byte
want string
}{
{"pdf by type", "application/pdf", pdf, "pdf"},
{"pdf by type with params", "Application/PDF; qs=0.9", pdf, "pdf"},
{"x-pdf", "application/x-pdf", pdf, "pdf"},
{"pdf type, body not checked", "application/pdf", []byte("oops"), "pdf"},
{"octet-stream sniffed as pdf", "application/octet-stream", pdf, "pdf"},
{"binary/octet-stream sniffed as pdf", "binary/octet-stream", pdf, "pdf"},
{"no type sniffed as pdf", "", pdf, "pdf"},
{"leading junk within 1024 bytes", "application/octet-stream", junkThenPDF, "pdf"},
{"magic too late", "application/octet-stream", lateMagic, "other"},
{"octet-stream that is not a pdf", "application/octet-stream", binary, "other"},
{"html stays html even if it mentions %PDF-", "text/html; charset=utf-8", []byte("<html>%PDF-1.4</html>"), "html"},
{"json", "application/json", []byte(`{"a":1}`), "json"},
{"problem+json", "application/problem+json", []byte(`{}`), "json"},
{"text", "text/plain", []byte("hello"), "text"},
{"no type, html", "", []byte("<!doctype html><html></html>"), "html"},
{"no type, json", "", []byte(`[1,2]`), "json"},
{"no type, text", "", []byte("just words"), "text"},
{"zip", "application/zip", binary, "other"},
{"image", "image/png", []byte{0x89, 'P', 'N', 'G'}, "other"},
} {
if got := clean.Kind(tc.ct, tc.body); got != tc.want {
t.Errorf("%s: Kind(%q, …) = %q, want %q", tc.name, tc.ct, got, tc.want)
}
}
}
// MediaKind keeps answering for callers that have no body: the pdf types are "pdf" there too.
func TestMediaKindKnowsPDF(t *testing.T) {
for ct, want := range map[string]string{"application/pdf": "pdf", "application/x-pdf": "pdf", "text/html": "html", "": "sniff"} {
if got := clean.MediaKind(ct); got != want {
t.Errorf("MediaKind(%q) = %q, want %q", ct, got, want)
}
}
}
docs/plans/p5a/_files/internal/dcache/pdf_test.go
+128
View File
@@ -0,0 +1,128 @@
package dcache_test
// P5a task 03: the PDF document cache. One entry per (URL, caller class): how to revalidate it,
// its metadata (opaque JSON here) and the text of every extracted page.
import (
"path/filepath"
"strings"
"testing"
"time"
"git.wntrmute.dev/kyle/sift/internal/dcache"
)
func openDB(t *testing.T, maxBytes int64) *dcache.DB {
t.Helper()
db, err := dcache.Open(filepath.Join(t.TempDir(), "c.db"), maxBytes, nil)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = db.Close() })
return db
}
func entry(pages ...string) dcache.PDFEntry {
return dcache.PDFEntry{
ContentHash: "sha256:abc",
ETag: `"v1"`,
LastModified: "Fri, 25 Sep 2026 12:00:00 GMT",
FetchedAt: time.Date(2026, 9, 25, 20, 0, 0, 0, time.UTC),
Meta: []byte(`{"pages":3,"title":"T","outline":[]}`),
Pages: pages,
}
}
func TestPDFRoundTrip(t *testing.T) {
db := openDB(t, 0)
k := dcache.PDFKey("https://example.org/a.pdf", dcache.ClassFleet)
if _, ok := db.GetPDF(k); ok {
t.Fatal("hit on an empty cache")
}
want := entry("page one", "page two", "", "page four") // an empty page keeps its place
db.PutPDF(k, want)
got, ok := db.GetPDF(k)
if !ok {
t.Fatal("miss after put")
}
if got.ContentHash != want.ContentHash || got.ETag != want.ETag || got.LastModified != want.LastModified ||
!got.FetchedAt.Equal(want.FetchedAt) || string(got.Meta) != string(want.Meta) {
t.Errorf("got %+v, want %+v", got, want)
}
if len(got.Pages) != 4 || got.Pages[0] != "page one" || got.Pages[2] != "" || got.Pages[3] != "page four" {
t.Errorf("pages = %q", got.Pages)
}
// A second put replaces the whole entry, pages included.
db.PutPDF(k, entry("only"))
got, _ = db.GetPDF(k)
if len(got.Pages) != 1 || got.Pages[0] != "only" {
t.Errorf("after replace, pages = %q; want [only]", got.Pages)
}
}
func TestPDFKeyPartitions(t *testing.T) {
db := openDB(t, 0)
u := "https://example.org/a.pdf"
fleet, restricted := dcache.PDFKey(u, dcache.ClassFleet), dcache.PDFKey(u, dcache.ClassRestricted)
if fleet == restricted {
t.Fatal("the same key for both caller classes")
}
db.PutPDF(restricted, entry("secret page"))
if _, ok := db.GetPDF(fleet); ok {
t.Error("a restricted caller's document is visible to the fleet")
}
if dcache.PDFKey(u, dcache.ClassFleet) != fleet || dcache.PDFKey("https://example.org/b.pdf", dcache.ClassFleet) == fleet {
t.Error("PDFKey must be deterministic and differ per URL")
}
if strings.Contains(fleet, "example.org") {
t.Error("the key must not carry the URL in the clear (it is hashed like the other keys)")
}
}
func TestPDFTouch(t *testing.T) {
db := openDB(t, 0)
k := dcache.PDFKey("https://example.org/a.pdf", dcache.ClassFleet)
db.PutPDF(k, entry("x"))
later := time.Date(2026, 9, 26, 20, 0, 0, 0, time.UTC)
db.TouchPDF(k, later) // after a 304: the text is unchanged, only its freshness moves
got, _ := db.GetPDF(k)
if !got.FetchedAt.Equal(later) || len(got.Pages) != 1 {
t.Errorf("after touch: %+v", got)
}
db.TouchPDF(dcache.PDFKey("https://nope", dcache.ClassFleet), later) // a miss is a no-op
}
// Documents count toward maxBytes and are evicted whole, oldest use first.
func TestPDFEviction(t *testing.T) {
db := openDB(t, 300<<10) // 300 KiB
big := strings.Repeat("z", 50<<10)
var keys []string
for i := 0; i < 10; i++ { // 10 × 2 pages × 50 KiB = 1 MiB of text
k := dcache.PDFKey("https://example.org/"+string(rune('a'+i))+".pdf", dcache.ClassFleet)
keys = append(keys, k)
db.PutPDF(k, entry(big, big))
}
if db.Size() > 300<<10 {
t.Errorf("size %d after puts, want ≤ maxBytes %d", db.Size(), 300<<10)
}
if _, ok := db.GetPDF(keys[9]); !ok {
t.Error("the newest document was evicted")
}
if _, ok := db.GetPDF(keys[0]); ok {
t.Error("the oldest document survived; eviction did not reach PDF entries")
}
for _, k := range keys {
if e, ok := db.GetPDF(k); ok && len(e.Pages) != 2 {
t.Errorf("a partly evicted document: %d pages", len(e.Pages))
}
}
}
func TestPDFNilDB(t *testing.T) {
var db *dcache.DB
db.PutPDF("k", entry("x"))
db.TouchPDF("k", time.Now())
if _, ok := db.GetPDF("k"); ok {
t.Error("nil DB returned a hit")
}
}
docs/plans/p5a/_files/internal/digest/pages_test.go
+40
View File
@@ -0,0 +1,40 @@
package digest
// P5a task 07: a digest of a PDF cites pages. The page markers come from internal/pdfdoc; the
// prompt tells the model to use them, and the prompt version moves so cached digests made with
// the old prompt are not served.
import (
"regexp"
"strconv"
"strings"
"testing"
)
func TestPromptCitesPages(t *testing.T) {
if !strings.Contains(systemPrompt, "--- page N ---") || !strings.Contains(systemPrompt, "(p. N)") {
t.Errorf("system prompt must tell the model to cite pages from --- page N --- markers:\n%s", systemPrompt)
}
if PromptVersion == "2" {
t.Error("PromptVersion must change with the prompt (was \"2\")")
}
}
// Page markers survive chunking whole: none is cut across two chunks. (This holds for today's
// Split, whose paragraph boundaries fall around each marker; the test keeps it that way.)
func TestSplitKeepsPageMarkers(t *testing.T) {
var b strings.Builder
for p := 1; p <= 40; p++ {
b.WriteString("--- page " + strconv.Itoa(p) + " ---\n\n")
b.WriteString(strings.Repeat("word ", 600))
b.WriteString("\n\n")
}
marker := regexp.MustCompile(`--- page \d+ ---`)
n := 0
for _, c := range Split(b.String(), ChunkTokens) {
n += len(marker.FindAllString(c, -1))
}
if n != 40 {
t.Errorf("%d whole page markers across the chunks, want 40", n)
}
}
docs/plans/p5a/_files/internal/fetch/pdf_test.go
+178
View File
@@ -0,0 +1,178 @@
package fetch_test
// P5a task 01: a PDF response gets a larger body cap and a longer deadline than a web page, and
// FetchIf revalidates a cached document with a conditional GET.
import (
"bytes"
"context"
"errors"
"net/http"
"net/http/httptest"
"net/netip"
"strings"
"sync"
"testing"
"time"
"git.wntrmute.dev/kyle/sift/internal/fetch"
"git.wntrmute.dev/kyle/sift/internal/guard"
)
func loopbackFetcher() *fetch.Fetcher {
g := &guard.Guard{AllowPrefixes: []netip.Prefix{netip.MustParsePrefix("127.0.0.0/8")}}
return &fetch.Fetcher{Guard: g, UA: "sift-test"}
}
func body(prefix string, n int) []byte {
b := bytes.Repeat([]byte("x"), n)
copy(b, prefix)
return b
}
func serve(t *testing.T, ct string, b []byte) string {
t.Helper()
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if ct != "" {
w.Header().Set("Content-Type", ct)
}
_, _ = w.Write(b)
}))
t.Cleanup(ts.Close)
return ts.URL
}
func status(err error) int {
var fe *fetch.Error
if errors.As(err, &fe) {
return fe.Status
}
return 0
}
func TestPDFBodyCap(t *testing.T) {
six := 6 << 20 // over DefaultMaxBody (5 MiB), under DefaultPDFMaxBody (100 MiB)
for _, tc := range []struct {
name, ct string
b []byte
want int // 0 = ok
}{
{"pdf by type", "application/pdf", body("%PDF-1.7\n", six), 0},
{"pdf by sniffing octet-stream", "application/octet-stream", body("%PDF-1.7\n", six), 0},
{"pdf with no type", "", body("%PDF-1.7\n", six), 0},
{"html keeps the 5 MiB cap", "text/html", body("<html>", six), 413},
{"octet-stream that is not a pdf keeps the cap", "application/octet-stream", body("PK\x03\x04", six), 413},
} {
f := loopbackFetcher()
res, err := f.Fetch(context.Background(), serve(t, tc.ct, tc.b))
if tc.want == 0 {
if err != nil || res == nil || len(res.Body) != six {
t.Errorf("%s: err %v, body %d bytes; want the whole %d", tc.name, err, lenBody(res), six)
}
continue
}
if status(err) != tc.want {
t.Errorf("%s: err %v (status %d), want %d", tc.name, err, status(err), tc.want)
}
}
}
func lenBody(r *fetch.Resource) int {
if r == nil {
return -1
}
return len(r.Body)
}
func TestPDFMaxBodyIsConfigurable(t *testing.T) {
f := loopbackFetcher()
f.PDFMaxBody = 1 << 20
_, err := f.Fetch(context.Background(), serve(t, "application/pdf", body("%PDF-1.7\n", 2<<20)))
if status(err) != 413 || !strings.Contains(err.Error(), "too large") {
t.Errorf("2 MiB pdf over a 1 MiB PDFMaxBody: %v, want 413 too large", err)
}
if fetch.DefaultPDFMaxBody != 100<<20 || fetch.DefaultPDFTimeout != 120*time.Second {
t.Errorf("defaults = %d, %v; want 100 MiB, 120s", fetch.DefaultPDFMaxBody, fetch.DefaultPDFTimeout)
}
}
// slow sends the headers, then the body in pieces over about 600 ms.
func slow(t *testing.T, ct string, first string) string {
t.Helper()
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", ct)
w.WriteHeader(200)
fl, _ := w.(http.Flusher)
_, _ = w.Write([]byte(first))
if fl != nil {
fl.Flush()
}
for i := 0; i < 6; i++ {
time.Sleep(100 * time.Millisecond)
_, _ = w.Write(bytes.Repeat([]byte("y"), 1024))
if fl != nil {
fl.Flush()
}
}
}))
t.Cleanup(ts.Close)
return ts.URL
}
// The web-page deadline (Timeout) still applies to HTML; a PDF gets PDFTimeout.
func TestPDFDeadline(t *testing.T) {
f := loopbackFetcher()
f.Timeout = 250 * time.Millisecond
f.PDFTimeout = 3 * time.Second
if _, err := f.Fetch(context.Background(), slow(t, "application/pdf", "%PDF-1.7\n")); err != nil {
t.Errorf("slow pdf under PDFTimeout: %v", err)
}
if _, err := f.Fetch(context.Background(), slow(t, "application/octet-stream", "%PDF-1.7\n")); err != nil {
t.Errorf("slow sniffed pdf under PDFTimeout: %v", err)
}
start := time.Now()
if _, err := f.Fetch(context.Background(), slow(t, "text/html", "<html>")); err == nil {
t.Errorf("slow html past Timeout succeeded; want an error")
}
if d := time.Since(start); d > 2*time.Second {
t.Errorf("slow html took %v to fail; the 250 ms deadline was not applied", d)
}
}
func TestFetchIf(t *testing.T) {
var mu sync.Mutex
var got []string
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
mu.Lock()
got = append(got, r.Header.Get("If-None-Match")+"|"+r.Header.Get("If-Modified-Since"))
mu.Unlock()
if r.Header.Get("If-None-Match") == `"v1"` {
w.WriteHeader(http.StatusNotModified)
return
}
w.Header().Set("Content-Type", "application/pdf")
w.Header().Set("ETag", `"v2"`)
w.Header().Set("Last-Modified", "Fri, 25 Sep 2026 12:00:00 GMT")
_, _ = w.Write([]byte("%PDF-1.7\nnew"))
}))
defer ts.Close()
f := loopbackFetcher()
res, err := f.FetchIf(context.Background(), ts.URL, `"v1"`, "")
if err != nil || res == nil || res.Status != http.StatusNotModified || len(res.Body) != 0 {
t.Fatalf("matching ETag: res %+v err %v; want status 304, empty body, nil error", res, err)
}
res, err = f.FetchIf(context.Background(), ts.URL, `"old"`, "Thu, 24 Sep 2026 12:00:00 GMT")
if err != nil || res.Status != 200 || string(res.Body) != "%PDF-1.7\nnew" || res.Header.Get("ETag") != `"v2"` {
t.Fatalf("changed document: res %+v err %v; want 200 with the new body and ETag", res, err)
}
mu.Lock()
defer mu.Unlock()
if len(got) != 2 || got[0] != `"v1"|` || got[1] != `"old"|Thu, 24 Sep 2026 12:00:00 GMT` {
t.Errorf("validators sent: %q", got)
}
// With no validators FetchIf is a plain Fetch.
if _, err := f.FetchIf(context.Background(), ts.URL, "", ""); err != nil {
t.Errorf("FetchIf with no validators: %v", err)
}
}
docs/plans/p5a/_files/internal/metrics/pdf_test.go
+49
View File
@@ -0,0 +1,49 @@
package metrics
// P5a task 07: the sift_pdf_* series.
import (
"io"
"net/http/httptest"
"strings"
"testing"
"time"
)
type zeroSrc struct{}
func (zeroSrc) Snapshot() Snapshot { return Snapshot{} }
func TestPDFSeries(t *testing.T) {
s := New(zeroSrc{})
s.ObservePDF("ok", 1500*time.Millisecond, 212)
s.ObservePDF("ok", 200*time.Millisecond, 3)
s.ObservePDF("no_text", 50*time.Millisecond, 0)
s.ObservePDF("blocked: evil.example", time.Millisecond, 0) // raw text never becomes a label
s.ObservePDFWait(2 * time.Second)
s.PDFCacheHit()
s.PDFCacheHit()
req := httptest.NewRequest("GET", "/metrics", nil)
req.RemoteAddr, req.Host = "127.0.0.1:9", "127.0.0.1:8890"
rec := httptest.NewRecorder()
s.Handler().ServeHTTP(rec, req)
body, _ := io.ReadAll(rec.Body)
out := string(body)
for _, want := range []string{
`sift_pdf_extract_total{outcome="ok"} 2`,
`sift_pdf_extract_total{outcome="no_text"} 1`,
`sift_pdf_extract_total{outcome="other"} 1`,
`sift_pdf_pages_total 215`,
`sift_pdf_cache_hits_total 2`,
`sift_pdf_extract_seconds_count 4`,
`sift_pdf_pool_wait_seconds_count 1`,
} {
if !strings.Contains(out, want) {
t.Errorf("metrics lack %q", want)
}
}
if strings.Contains(out, "evil.example") {
t.Error("a raw outcome string leaked into a label")
}
}
docs/plans/p5a/_files/internal/pdfdoc/pdfdoc_test.go
+249
View File
@@ -0,0 +1,249 @@
package pdfdoc_test
// P5a task 05: rendering an extracted document for an agent — the whole text with page markers,
// the map for a long document, a page range, and grep.
import (
"errors"
"fmt"
"strings"
"testing"
"time"
"unicode/utf8"
"git.wntrmute.dev/kyle/sift/internal/pdfdoc"
"git.wntrmute.dev/kyle/sift/internal/pdftext"
)
func small() *pdftext.Doc {
return &pdftext.Doc{
Meta: pdftext.Meta{Pages: 3, Title: "Three Pages", Outline: []pdftext.OutlineItem{{Title: "1 Intro", Page: 1, Level: 1}, {Title: "2 Fox", Page: 2, Level: 1}, {Title: "2.1 Den", Page: 2, Level: 2}}},
Text: []string{"Chapter one begins here.\nSecond line.\n", " The quick brown fox is on page two.\nIt jumps.", "Last page."},
}
}
// big returns n pages of about 1,000 characters each, every line naming its page.
func big(n int) *pdftext.Doc {
d := &pdftext.Doc{Meta: pdftext.Meta{Pages: n, Title: "Long Thesis", Outline: []pdftext.OutlineItem{{Title: "1 Introduction", Page: 1, Level: 1}, {Title: "2 Method", Page: 12, Level: 1}, {Title: "2.3 Setup", Page: 15, Level: 2}}}}
for p := 1; p <= n; p++ {
var lines []string
for l := 1; l <= 12; l++ {
lines = append(lines, fmt.Sprintf("page %d line %d lorem ipsum dolor sit amet consectetur adipiscing elit sed do eiusmod", p, l))
}
if p == 37 {
lines[4] = "the Needle is here on page 37"
}
d.Text = append(d.Text, strings.Join(lines, "\n"))
}
return d
}
func TestFull(t *testing.T) {
want := "--- page 1 ---\n\nChapter one begins here.\nSecond line.\n\n--- page 2 ---\n\nThe quick brown fox is on page two.\nIt jumps.\n\n--- page 3 ---\n\nLast page."
if got := pdfdoc.Full(small()); got != want {
t.Errorf("Full =\n%q\nwant\n%q", got, want)
}
if pdfdoc.Marker(12) != "--- page 12 ---" {
t.Errorf("Marker(12) = %q", pdfdoc.Marker(12))
}
}
func TestWholeDocumentWhenItFits(t *testing.T) {
r, err := pdfdoc.Render(small(), pdfdoc.Options{OutCap: 10000})
if err != nil {
t.Fatal(err)
}
if r.Map || r.First != 1 || r.Last != 3 || r.Content != pdfdoc.Full(small()) {
t.Errorf("got %+v", r)
}
}
func TestMapForALongDocument(t *testing.T) {
d := big(60) // ~60k characters
r, err := pdfdoc.Render(d, pdfdoc.Options{OutCap: 20000})
if err != nil {
t.Fatal(err)
}
if !r.Map || r.First != 1 || r.Last < 1 {
t.Fatalf("got Map=%v First=%d Last=%d, want a map from page 1", r.Map, r.First, r.Last)
}
c := r.Content
for _, want := range []string{
"# Long Thesis\n",
"60 pages · ~",
"## Outline\n",
"- 1 Introduction (p. 1)\n",
"- 2 Method (p. 12)\n",
" - 2.3 Setup (p. 15)\n", // level 2 is indented two spaces
`pages="a-b"`,
`grep="pattern"`,
"--- page 1 ---",
} {
if !strings.Contains(c, want) {
t.Errorf("map lacks %q", want)
}
}
if utf8.RuneCountInString(c) > 20000 {
t.Errorf("map is %d characters, over OutCap 20000", utf8.RuneCountInString(c))
}
// Leading pages take at most a quarter of the cap, and only whole pages.
if strings.Contains(c, fmt.Sprintf("--- page %d ---", r.Last+1)) {
t.Errorf("content has page %d past Last %d", r.Last+1, r.Last)
}
if !strings.Contains(c, fmt.Sprintf("page %d line 12", r.Last)) {
t.Errorf("the last page shown (%d) is cut; pages in a map are whole", r.Last)
}
idx := strings.Index(c, "--- page 1 ---")
if pagesPart := c[idx:]; utf8.RuneCountInString(pagesPart) > 20000/4 {
t.Errorf("leading pages take %d characters, over OutCap/4", utf8.RuneCountInString(pagesPart))
}
}
func TestMapNotesPartialExtraction(t *testing.T) {
d := big(30)
d.Pages = 2500 // the child stopped at its page cap
r, _ := pdfdoc.Render(d, pdfdoc.Options{OutCap: 5000})
if !strings.Contains(r.Content, "2500 pages") || !strings.Contains(r.Content, "pages 1-30 extracted") {
t.Errorf("map must say how much was extracted:\n%s", r.Content[:min(600, len(r.Content))])
}
}
func TestUntitledAndNoOutline(t *testing.T) {
d := big(40)
d.Title, d.Outline = "", nil
r, _ := pdfdoc.Render(d, pdfdoc.Options{OutCap: 5000})
if !strings.HasPrefix(r.Content, "# Untitled PDF\n") || strings.Contains(r.Content, "## Outline") {
t.Errorf("untitled, outline-less map:\n%s", r.Content[:min(400, len(r.Content))])
}
}
func TestPages(t *testing.T) {
d := big(60)
for _, tc := range []struct {
spec string
first, last int
}{
{"2-3", 2, 3},
{"7", 7, 7},
{" 58- ", 58, 60},
{"60-60", 60, 60},
} {
r, err := pdfdoc.Render(d, pdfdoc.Options{Pages: tc.spec, OutCap: 200000})
if err != nil {
t.Errorf("%q: %v", tc.spec, err)
continue
}
if r.Map || r.First != tc.first || r.Last != tc.last {
t.Errorf("%q: First %d Last %d Map %v, want %d-%d", tc.spec, r.First, r.Last, r.Map, tc.first, tc.last)
}
if !strings.HasPrefix(r.Content, pdfdoc.Marker(tc.first)+"\n\n") || strings.Contains(r.Content, pdfdoc.Marker(tc.last+1)) || (tc.first > 1 && strings.Contains(r.Content, pdfdoc.Marker(tc.first-1)+"\n")) {
t.Errorf("%q: wrong pages in content: %.120q…", tc.spec, r.Content)
}
}
for _, bad := range []string{"0-1", "5-4", "61", "999", "a-b", "1-2-3", "", "-5"} {
if bad == "" {
continue // empty means "no range", tested above
}
_, err := pdfdoc.Render(d, pdfdoc.Options{Pages: bad, OutCap: 200000})
if !errors.Is(err, pdfdoc.ErrBadPages) {
t.Errorf("%q: %v, want ErrBadPages", bad, err)
}
}
_, err := pdfdoc.Render(d, pdfdoc.Options{Pages: "61", OutCap: 200000})
if err == nil || !strings.Contains(err.Error(), "pages out of range (1-60)") {
t.Errorf("out-of-range message: %v", err)
}
}
// A range is at most 100 pages and at most OutCap characters, cut at a whole page with a note
// that says how to continue.
func TestPagesAreCapped(t *testing.T) {
d := big(300)
r, err := pdfdoc.Render(d, pdfdoc.Options{Pages: "1-300", OutCap: 1 << 30})
if err != nil {
t.Fatal(err)
}
if r.Last != 100 || !strings.Contains(r.Note, `pages="101-300"`) {
t.Errorf("1-300: Last %d note %q; want 100 and a pointer to 101-300", r.Last, r.Note)
}
r, _ = pdfdoc.Render(d, pdfdoc.Options{Pages: "10-40", OutCap: 5000})
if utf8.RuneCountInString(r.Content) > 5000 || r.First != 10 || r.Last < 10 || r.Last >= 40 {
t.Errorf("10-40 under a 5000 cap: First %d Last %d, %d chars", r.First, r.Last, utf8.RuneCountInString(r.Content))
}
if !strings.Contains(r.Note, fmt.Sprintf(`pages="%d-40"`, r.Last+1)) {
t.Errorf("note %q must point at the rest", r.Note)
}
// One page bigger than the cap is truncated rather than dropped.
huge := &pdftext.Doc{Meta: pdftext.Meta{Pages: 1}, Text: []string{strings.Repeat("w ", 10000)}}
r, _ = pdfdoc.Render(huge, pdfdoc.Options{Pages: "1", OutCap: 1000})
if r.First != 1 || r.Last != 1 || utf8.RuneCountInString(r.Content) > 1000 || r.Note == "" {
t.Errorf("oversized single page: %+v (%d chars)", r, utf8.RuneCountInString(r.Content))
}
}
func TestGrep(t *testing.T) {
d := big(60)
r, err := pdfdoc.Render(d, pdfdoc.Options{Grep: "needle", OutCap: 200000})
if err != nil {
t.Fatal(err)
}
if r.GrepHits != 1 || r.First != 0 || r.Map {
t.Errorf("got %+v", r)
}
for _, want := range []string{
`grep "needle": 1 match`,
"p. 37- page 37 line 4",
"p. 37: the Needle is here on page 37",
"p. 37- page 37 line 6",
} {
if !strings.Contains(r.Content, want) {
t.Errorf("grep output lacks %q:\n%s", want, r.Content)
}
}
// Smart case: an upper-case letter makes the pattern case-sensitive.
if r, _ := pdfdoc.Render(d, pdfdoc.Options{Grep: "NEEDLE", OutCap: 200000}); r.GrepHits != 0 {
t.Errorf("NEEDLE matched %d times, want 0 (case-sensitive)", r.GrepHits)
}
if r, _ := pdfdoc.Render(d, pdfdoc.Options{Grep: "Needle", OutCap: 200000}); r.GrepHits != 1 {
t.Errorf("Needle matched %d times, want 1", r.GrepHits)
}
// No match is an answer, not an error.
r, err = pdfdoc.Render(d, pdfdoc.Options{Grep: "zebra", OutCap: 200000})
if err != nil || r.GrepHits != 0 || !strings.Contains(r.Content, `grep "zebra": no matches`) {
t.Errorf("no match: %+v %v", r, err)
}
// Within a page range.
r, _ = pdfdoc.Render(d, pdfdoc.Options{Grep: "line 3 ", Pages: "5-6", OutCap: 200000})
if r.GrepHits != 2 || strings.Contains(r.Content, "p. 7") {
t.Errorf("grep in 5-6: %d hits\n%s", r.GrepHits, r.Content)
}
}
func TestGrepLimits(t *testing.T) {
d := big(200)
r, err := pdfdoc.Render(d, pdfdoc.Options{Grep: "lorem", OutCap: 1 << 30}) // 2,399 matches: every line but the needle
if err != nil {
t.Fatal(err)
}
if r.GrepHits != 2399 || strings.Count(r.Content, ": page ") != 100 || !strings.Contains(r.Content, "showing the first 100") {
t.Errorf("hits %d, shown %d", r.GrepHits, strings.Count(r.Content, ": page "))
}
if _, err := pdfdoc.Render(d, pdfdoc.Options{Grep: strings.Repeat("a", 201), OutCap: 1000}); !errors.Is(err, pdfdoc.ErrBadGrep) {
t.Errorf("201-byte pattern: %v, want ErrBadGrep", err)
}
if _, err := pdfdoc.Render(d, pdfdoc.Options{Grep: "(unclosed", OutCap: 1000}); !errors.Is(err, pdfdoc.ErrBadGrep) {
t.Errorf("bad regexp: %v, want ErrBadGrep", err)
}
// RE2 is linear: the classic catastrophic pattern over a 1 MB page finishes quickly.
evil := &pdftext.Doc{Meta: pdftext.Meta{Pages: 1}, Text: []string{strings.Repeat("a", 1<<20) + "!"}}
start := time.Now()
_, _ = pdfdoc.Render(evil, pdfdoc.Options{Grep: "(a+)+$", OutCap: 1000})
if d := time.Since(start); d > time.Second {
t.Errorf("(a+)+$ took %v", d)
}
// Output stays under the cap.
r, _ = pdfdoc.Render(d, pdfdoc.Options{Grep: "lorem", OutCap: 3000})
if utf8.RuneCountInString(r.Content) > 3000 {
t.Errorf("grep output %d characters over a 3000 cap", utf8.RuneCountInString(r.Content))
}
}
docs/plans/p5a/_files/internal/pdfpool/pdfpool_test.go
+226
View File
@@ -0,0 +1,226 @@
package pdfpool_test
// P5a task 04: the extraction pool. A few slots, a short queue, single-flight per key, and at
// most one job in flight per caller.
import (
"context"
"errors"
"sync"
"sync/atomic"
"testing"
"time"
"git.wntrmute.dev/kyle/sift/internal/pdfpool"
)
func waitUntil(t *testing.T, cond func() bool) {
t.Helper()
deadline := time.Now().Add(3 * time.Second)
for !cond() {
if time.Now().After(deadline) {
t.Fatal("condition not reached within 3 s")
}
time.Sleep(5 * time.Millisecond)
}
}
func TestSingleFlight(t *testing.T) {
p := pdfpool.New(2, 4, time.Second)
var calls atomic.Int32
release := make(chan struct{})
fn := func(ctx context.Context) (any, error) {
calls.Add(1)
<-release
return "text", nil
}
var wg sync.WaitGroup
results := make(chan any, 5)
for i := 0; i < 5; i++ {
wg.Add(1)
go func(i int) {
defer wg.Done()
v, err := p.Do(context.Background(), "doc-1", "caller-"+string(rune('a'+i)), fn)
if err != nil {
t.Error(err)
}
results <- v
}(i)
}
waitUntil(t, func() bool { return calls.Load() == 1 && p.Waiting("doc-1") == 4 })
close(release)
wg.Wait()
close(results)
for v := range results {
if v != "text" {
t.Errorf("a sharer got %v", v)
}
}
if n := calls.Load(); n != 1 {
t.Errorf("fn ran %d times for one key, want 1", n)
}
// Nothing is cached: the next call runs fn again.
_, _ = p.Do(context.Background(), "doc-1", "x", func(ctx context.Context) (any, error) { calls.Add(1); return nil, nil })
if n := calls.Load(); n != 2 {
t.Errorf("after the flight ended, fn ran %d times in total, want 2", n)
}
}
// Two slots and four waiting places: the seventh distinct job is refused at once.
func TestSlotsAndQueue(t *testing.T) {
p := pdfpool.New(2, 4, 5*time.Second)
release := make(chan struct{})
var running atomic.Int32
block := func(ctx context.Context) (any, error) {
running.Add(1)
defer running.Add(-1)
<-release
return nil, nil
}
var wg sync.WaitGroup
for i := 0; i < 6; i++ {
wg.Add(1)
go func(i int) {
defer wg.Done()
if _, err := p.Do(context.Background(), "doc-"+string(rune('a'+i)), "caller-"+string(rune('a'+i)), block); err != nil {
t.Errorf("job %d: %v", i, err)
}
}(i)
}
waitUntil(t, func() bool { return running.Load() == 2 && p.Queued() == 4 })
start := time.Now()
_, err := p.Do(context.Background(), "doc-z", "caller-z", block)
if !errors.Is(err, pdfpool.ErrBusy) {
t.Errorf("seventh job: %v, want ErrBusy", err)
}
if d := time.Since(start); d > 200*time.Millisecond {
t.Errorf("a full queue refused after %v; it must refuse at once", d)
}
if running.Load() > 2 {
t.Errorf("%d jobs ran at once, want at most 2", running.Load())
}
close(release)
wg.Wait()
}
// A queued job that waits longer than Wait is refused with ErrBusy and gives up its place.
func TestQueueWait(t *testing.T) {
p := pdfpool.New(1, 1, 150*time.Millisecond)
release := make(chan struct{})
defer close(release)
go func() {
_, _ = p.Do(context.Background(), "a", "ca", func(ctx context.Context) (any, error) { <-release; return nil, nil })
}()
waitUntil(t, func() bool { return p.Running() == 1 })
start := time.Now()
_, err := p.Do(context.Background(), "b", "cb", func(ctx context.Context) (any, error) { return nil, nil })
if !errors.Is(err, pdfpool.ErrBusy) {
t.Fatalf("waiter: %v, want ErrBusy", err)
}
if d := time.Since(start); d < 100*time.Millisecond || d > time.Second {
t.Errorf("waiter refused after %v, want about 150 ms", d)
}
if p.Queued() != 0 {
t.Errorf("queued = %d after the waiter gave up, want 0", p.Queued())
}
}
func TestCallerBusy(t *testing.T) {
p := pdfpool.New(2, 4, time.Second)
release := make(chan struct{})
go func() {
_, _ = p.Do(context.Background(), "doc-1", "straylight", func(ctx context.Context) (any, error) { <-release; return nil, nil })
}()
waitUntil(t, func() bool { return p.Running() == 1 })
if _, err := p.Do(context.Background(), "doc-2", "straylight", func(ctx context.Context) (any, error) { return nil, nil }); !errors.Is(err, pdfpool.ErrCallerBusy) {
t.Errorf("second job from the same caller: %v, want ErrCallerBusy", err)
}
// The same caller asking for the same document again joins the flight instead.
done := make(chan error, 1)
go func() {
_, err := p.Do(context.Background(), "doc-1", "straylight", func(ctx context.Context) (any, error) { return nil, nil })
done <- err
}()
waitUntil(t, func() bool { return p.Waiting("doc-1") == 1 })
close(release)
if err := <-done; err != nil {
t.Errorf("joining its own flight: %v", err)
}
// Once the job is done the caller may start another.
if _, err := p.Do(context.Background(), "doc-2", "straylight", func(ctx context.Context) (any, error) { return nil, nil }); err != nil {
t.Errorf("after the first job ended: %v", err)
}
}
func TestErrorsAndPanicsReachEverySharer(t *testing.T) {
p := pdfpool.New(2, 4, time.Second)
boom := errors.New("boom")
if _, err := p.Do(context.Background(), "k", "c", func(ctx context.Context) (any, error) { return nil, boom }); !errors.Is(err, boom) {
t.Errorf("fn error: %v", err)
}
_, err := p.Do(context.Background(), "k", "c", func(ctx context.Context) (any, error) { panic("kaboom") })
if err == nil {
t.Fatal("a panicking fn returned no error")
}
if p.Running() != 0 {
t.Errorf("running = %d after a panic, want 0 (the slot leaked)", p.Running())
}
if _, err := p.Do(context.Background(), "k2", "c", func(ctx context.Context) (any, error) { return 1, nil }); err != nil {
t.Errorf("the caller is still marked busy after its job panicked: %v", err)
}
}
func TestContextCancelWhileQueued(t *testing.T) {
p := pdfpool.New(1, 2, 5*time.Second)
release := make(chan struct{})
defer close(release)
go func() {
_, _ = p.Do(context.Background(), "a", "ca", func(ctx context.Context) (any, error) { <-release; return nil, nil })
}()
waitUntil(t, func() bool { return p.Running() == 1 })
ctx, cancel := context.WithCancel(context.Background())
errc := make(chan error, 1)
go func() {
_, err := p.Do(ctx, "b", "cb", func(ctx context.Context) (any, error) { return nil, nil })
errc <- err
}()
waitUntil(t, func() bool { return p.Queued() == 1 })
cancel()
if err := <-errc; !errors.Is(err, context.Canceled) {
t.Errorf("cancelled waiter: %v, want context.Canceled", err)
}
if p.Queued() != 0 {
t.Errorf("queued = %d after cancel, want 0", p.Queued())
}
}
// The job's own context is not the first caller's: if the caller that started a shared flight
// goes away, the sharers still get the result.
func TestFlightOutlivesItsStarter(t *testing.T) {
p := pdfpool.New(1, 2, 5*time.Second)
release := make(chan struct{})
ctx, cancel := context.WithCancel(context.Background())
jobCtxErr := make(chan error, 1)
go func() {
_, _ = p.Do(ctx, "doc", "first", func(jctx context.Context) (any, error) {
<-release
jobCtxErr <- jctx.Err()
return "done", nil
})
}()
waitUntil(t, func() bool { return p.Running() == 1 })
res := make(chan any, 1)
go func() {
v, _ := p.Do(context.Background(), "doc", "second", nil)
res <- v
}()
waitUntil(t, func() bool { return p.Waiting("doc") == 1 })
cancel()
close(release)
if v := <-res; v != "done" {
t.Errorf("sharer got %v after the starter left, want done", v)
}
if e := <-jobCtxErr; e != nil {
t.Errorf("the job's context was cancelled with its starter: %v", e)
}
}
docs/plans/p5a/_files/internal/pdftest/pdftest.go
+133
View File
@@ -0,0 +1,133 @@
// Package pdftest builds small, deterministic PDFs for tests: text pages in Helvetica, an optional
// outline (bookmarks), and image-only ("scanned") pages. It exists so the P5a tests need no binary
// fixtures from elsewhere. It is test support: nothing outside _test.go files imports it.
package pdftest
import (
"bytes"
"fmt"
"strings"
)
// Outline is one bookmark: a title that points at a 1-based page, nested at Level (1 = top).
// Build supports one level; Level is kept so tests can state what they expect back.
type Outline struct {
Title string
Page int
Level int
}
// Doc describes a PDF to build. Pages holds each page's text; a page's lines are split on "\n"
// and set one per line (long lines are not wrapped: callers keep them under ~90 characters).
// Scanned pages (1-based) carry a small image and no text at all.
type Doc struct {
Title string
Pages []string
Outline []Outline
Scanned []int
}
// Build returns the PDF bytes for d. It panics on a Doc it cannot express (an outline entry
// pointing past the last page), since that is a bug in the test that built it.
func Build(d Doc) []byte {
var b bytes.Buffer
var offsets []int
obj := func(body string) int {
offsets = append(offsets, b.Len())
n := len(offsets)
fmt.Fprintf(&b, "%d 0 obj\n%s\nendobj\n", n, body)
return n
}
b.WriteString("%PDF-1.4\n%\xe2\xe3\xcf\xd3\n")
// Object numbers are fixed by position: 1 catalog, 2 pages, 3 font, 4 image, 5 info, then
// two per page (page, content), then the outline.
nPages := len(d.Pages)
firstPage := 6
pageObj := func(i int) int { return firstPage + 2*i }
outlineRoot := firstPage + 2*nPages
scanned := map[int]bool{}
for _, p := range d.Scanned {
scanned[p] = true
}
catalog := "<< /Type /Catalog /Pages 2 0 R"
if len(d.Outline) > 0 {
catalog += fmt.Sprintf(" /Outlines %d 0 R /PageMode /UseOutlines", outlineRoot)
}
obj(catalog + " >>")
var kids []string
for i := range d.Pages {
kids = append(kids, fmt.Sprintf("%d 0 R", pageObj(i)))
}
obj(fmt.Sprintf("<< /Type /Pages /Kids [%s] /Count %d >>", strings.Join(kids, " "), nPages))
obj("<< /Type /Font /Subtype /Type1 /BaseFont /Helvetica /Encoding /WinAnsiEncoding >>")
img := []byte{0x00, 0x80, 0x80, 0x00} // 2x2 grey pixels
obj(fmt.Sprintf("<< /Type /XObject /Subtype /Image /Width 2 /Height 2 /ColorSpace /DeviceGray /BitsPerComponent 8 /Length %d >>\nstream\n%s\nendstream", len(img), img))
obj(fmt.Sprintf("<< /Title (%s) /Producer (sift pdftest) >>", esc(d.Title)))
for i, text := range d.Pages {
var content string
if scanned[i+1] {
content = "q 400 0 0 500 100 150 cm /Im1 Do Q"
} else {
var c strings.Builder
c.WriteString("BT /F1 11 Tf 13 TL 60 760 Td\n")
for _, line := range strings.Split(text, "\n") {
fmt.Fprintf(&c, "(%s) Tj T*\n", esc(line))
}
c.WriteString("ET")
content = c.String()
}
obj(fmt.Sprintf("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 612 792] /Resources << /Font << /F1 3 0 R >> /XObject << /Im1 4 0 R >> >> /Contents %d 0 R >>", pageObj(i)+1))
obj(fmt.Sprintf("<< /Length %d >>\nstream\n%s\nendstream", len(content), content))
}
if len(d.Outline) > 0 {
first := outlineRoot + 1
last := outlineRoot + len(d.Outline)
obj(fmt.Sprintf("<< /Type /Outlines /First %d 0 R /Last %d 0 R /Count %d >>", first, last, len(d.Outline)))
for i, o := range d.Outline {
if o.Page < 1 || o.Page > nPages {
panic(fmt.Sprintf("pdftest: outline %q points at page %d of %d", o.Title, o.Page, nPages))
}
n := first + i
item := fmt.Sprintf("<< /Title (%s) /Parent %d 0 R /Dest [%d 0 R /XYZ null null null]", esc(o.Title), outlineRoot, pageObj(o.Page-1))
if i > 0 {
item += fmt.Sprintf(" /Prev %d 0 R", n-1)
}
if n < last {
item += fmt.Sprintf(" /Next %d 0 R", n+1)
}
obj(item + " >>")
}
}
xref := b.Len()
fmt.Fprintf(&b, "xref\n0 %d\n0000000000 65535 f \n", len(offsets)+1)
for _, off := range offsets {
fmt.Fprintf(&b, "%010d 00000 n \n", off)
}
fmt.Fprintf(&b, "trailer\n<< /Size %d /Root 1 0 R /Info 5 0 R >>\nstartxref\n%d\n%%%%EOF\n", len(offsets)+1, xref)
return b.Bytes()
}
// Lorem returns n lines of distinct, searchable filler: "p<page> l<line> …", so a test can tell
// which page a line came from.
func Lorem(page, n int) string {
words := []string{"alpha", "bravo", "charlie", "delta", "echo", "foxtrot", "golf", "hotel", "india", "juliet", "kilo", "lima"}
var lines []string
for i := 1; i <= n; i++ {
var w []string
for j := 0; j < 10; j++ {
w = append(w, words[(page*7+i*3+j)%len(words)])
}
lines = append(lines, fmt.Sprintf("p%d l%d %s", page, i, strings.Join(w, " ")))
}
return strings.Join(lines, "\n")
}
func esc(s string) string {
r := strings.NewReplacer(`\`, `\\`, `(`, `\(`, `)`, `\)`)
return r.Replace(s)
}
docs/plans/p5a/_files/internal/pdftext/pdftext_test.go
+252
View File
@@ -0,0 +1,252 @@
package pdftext_test
// P5a task 02: Extract (the child's work), ChildMain (the `sift pdftext` entry point) and Runner
// (the parent: exec, deadline, kill, parse).
//
// The Runner tests run the test binary itself as the child: TestMain turns into the child when
// SIFT_PDFTEXT_CHILD=1 is set, exactly as `sift pdftext` would. The two other variables are test
// hooks that live only here: SIFT_PDFTEXT_SLEEP delays the child (for the deadline test) and
// SIFT_PDFTEXT_PIDFILE records its pid (to prove it is gone afterwards).
import (
"bufio"
"bytes"
"context"
"encoding/json"
"errors"
"os"
"path/filepath"
"strconv"
"strings"
"syscall"
"testing"
"time"
"git.wntrmute.dev/kyle/sift/internal/pdftest"
"git.wntrmute.dev/kyle/sift/internal/pdftext"
)
func TestMain(m *testing.M) {
if os.Getenv("SIFT_PDFTEXT_CHILD") == "1" {
if p := os.Getenv("SIFT_PDFTEXT_PIDFILE"); p != "" {
_ = os.WriteFile(p, []byte(strconv.Itoa(os.Getpid())), 0o600)
}
if d, err := time.ParseDuration(os.Getenv("SIFT_PDFTEXT_SLEEP")); err == nil {
time.Sleep(d)
}
os.Exit(pdftext.ChildMain(os.Args[1:], os.Stdin, os.Stdout))
}
os.Exit(m.Run())
}
func three() []byte {
return pdftest.Build(pdftest.Doc{
Title: "Three Pages",
Pages: []string{
"Chapter one begins here.\nSecond line (with parens).",
"The quick brown fox is on page two.",
pdftest.Lorem(3, 5),
},
Outline: []pdftest.Outline{{Title: "1 Intro", Page: 1, Level: 1}, {Title: "2 Fox", Page: 2, Level: 1}, {Title: "3 Filler", Page: 3, Level: 1}},
})
}
type line struct {
Meta *pdftext.Meta `json:"meta"`
Page int `json:"page"`
Text *string `json:"text"`
Error string `json:"error"`
}
func lines(t *testing.T, out []byte) []line {
t.Helper()
var ls []line
sc := bufio.NewScanner(bytes.NewReader(out))
sc.Buffer(make([]byte, 1<<20), 64<<20)
for sc.Scan() {
var l line
if err := json.Unmarshal(sc.Bytes(), &l); err != nil {
t.Fatalf("output line is not JSON: %q (%v)", sc.Text(), err)
}
ls = append(ls, l)
}
return ls
}
func TestExtractWritesMetaThenPages(t *testing.T) {
var out bytes.Buffer
if err := pdftext.Extract(three(), 2000, &out); err != nil {
t.Fatal(err)
}
ls := lines(t, out.Bytes())
if len(ls) != 4 || ls[0].Meta == nil {
t.Fatalf("got %d lines, first meta=%v; want a meta line then 3 page lines:\n%s", len(ls), ls[0].Meta, out.String())
}
m := ls[0].Meta
if m.Pages != 3 || m.Title != "Three Pages" {
t.Errorf("meta = %+v, want 3 pages titled Three Pages", *m)
}
want := []pdftext.OutlineItem{{Title: "1 Intro", Page: 1, Level: 1}, {Title: "2 Fox", Page: 2, Level: 1}, {Title: "3 Filler", Page: 3, Level: 1}}
if len(m.Outline) != 3 {
t.Fatalf("outline = %+v, want %+v", m.Outline, want)
}
for i := range want {
if m.Outline[i] != want[i] {
t.Errorf("outline[%d] = %+v, want %+v", i, m.Outline[i], want[i])
}
}
for i, l := range ls[1:] {
if l.Page != i+1 || l.Text == nil || l.Error != "" {
t.Errorf("line %d = %+v, want page %d with text", i+1, l, i+1)
}
}
if !strings.Contains(*ls[1].Text, "Chapter one begins here.") || !strings.Contains(*ls[2].Text, "quick brown fox") || !strings.Contains(*ls[3].Text, "p3 l5") {
t.Errorf("page texts wrong: %q / %q / %q", *ls[1].Text, *ls[2].Text, *ls[3].Text)
}
}
func TestExtractPageCap(t *testing.T) {
var out bytes.Buffer
if err := pdftext.Extract(three(), 2, &out); err != nil {
t.Fatal(err)
}
ls := lines(t, out.Bytes())
if len(ls) != 3 || ls[0].Meta.Pages != 3 || ls[2].Page != 2 {
t.Errorf("maxPages 2: %d lines, meta %+v; want meta (pages 3) and pages 1-2", len(ls), ls[0].Meta)
}
}
func TestExtractNoOutline(t *testing.T) {
var out bytes.Buffer
if err := pdftext.Extract(pdftest.Build(pdftest.Doc{Pages: []string{"a", "b"}}), 10, &out); err != nil {
t.Fatal(err)
}
ls := lines(t, out.Bytes())
if ls[0].Meta.Outline == nil || len(ls[0].Meta.Outline) != 0 {
t.Errorf("outline = %#v, want an empty non-nil slice (JSON [])", ls[0].Meta.Outline)
}
if !strings.Contains(out.String(), `"outline":[]`) {
t.Errorf("meta line must carry \"outline\":[]: %s", out.String())
}
}
func TestExtractErrors(t *testing.T) {
enc, err := os.ReadFile(filepath.Join("..", "..", "tests", "testdata", "pdf", "encrypted-user.pdf"))
if err != nil {
t.Fatal(err)
}
if err := pdftext.Extract(enc, 10, &bytes.Buffer{}); !errors.Is(err, pdftext.ErrEncrypted) {
t.Errorf("user-password pdf: %v, want ErrEncrypted", err)
}
owner, err := os.ReadFile(filepath.Join("..", "..", "tests", "testdata", "pdf", "encrypted-owner.pdf"))
if err != nil {
t.Fatal(err)
}
var out bytes.Buffer
if err := pdftext.Extract(owner, 10, &out); err != nil || !strings.Contains(out.String(), "quick brown fox") {
t.Errorf("owner-password-only pdf: err %v; want its text", err)
}
if err := pdftext.Extract([]byte("not a pdf at all"), 10, &bytes.Buffer{}); !errors.Is(err, pdftext.ErrUnreadable) {
t.Errorf("garbage: %v, want ErrUnreadable", err)
}
// A truncated file either yields what it can or ErrUnreadable; it never panics.
b := three()
_ = pdftext.Extract(b[:len(b)*6/10], 10, &bytes.Buffer{})
}
func TestChildMainExitCodes(t *testing.T) {
enc, _ := os.ReadFile(filepath.Join("..", "..", "tests", "testdata", "pdf", "encrypted-user.pdf"))
for _, tc := range []struct {
name string
in []byte
code int
}{
{"ok", three(), 0},
{"encrypted", enc, 3},
{"unreadable", []byte("nope"), 4},
} {
var out bytes.Buffer
if code := pdftext.ChildMain([]string{"-max-pages", "5"}, bytes.NewReader(tc.in), &out); code != tc.code {
t.Errorf("%s: exit %d, want %d", tc.name, code, tc.code)
}
}
}
func runner(env ...string) *pdftext.Runner {
return &pdftext.Runner{Exe: os.Args[0], Env: append([]string{"SIFT_PDFTEXT_CHILD=1"}, env...), Deadline: 20 * time.Second}
}
func TestRunnerRun(t *testing.T) {
doc, err := runner().Run(context.Background(), three())
if err != nil {
t.Fatal(err)
}
if doc.Pages != 3 || doc.Title != "Three Pages" || len(doc.Outline) != 3 || len(doc.Text) != 3 {
t.Fatalf("doc = %+v", doc)
}
if !strings.Contains(doc.Text[1], "quick brown fox") {
t.Errorf("page 2 = %q", doc.Text[1])
}
}
func TestRunnerErrors(t *testing.T) {
enc, _ := os.ReadFile(filepath.Join("..", "..", "tests", "testdata", "pdf", "encrypted-user.pdf"))
if _, err := runner().Run(context.Background(), enc); !errors.Is(err, pdftext.ErrEncrypted) {
t.Errorf("encrypted: %v, want ErrEncrypted", err)
}
if _, err := runner().Run(context.Background(), []byte("nope")); !errors.Is(err, pdftext.ErrUnreadable) {
t.Errorf("garbage: %v, want ErrUnreadable", err)
}
scanned := pdftest.Build(pdftest.Doc{Pages: []string{"", "", ""}, Scanned: []int{1, 2, 3}})
if _, err := runner().Run(context.Background(), scanned); !errors.Is(err, pdftext.ErrNoText) {
t.Errorf("scanned: %v, want ErrNoText", err)
}
r := runner()
r.MaxText = 100 // bytes of child output
if _, err := r.Run(context.Background(), three()); !errors.Is(err, pdftext.ErrTooMuchText) {
t.Errorf("output over MaxText: %v, want ErrTooMuchText", err)
}
}
// The child is killed at the deadline and reaped: no process is left behind.
func TestRunnerDeadlineKillsTheChild(t *testing.T) {
pidfile := filepath.Join(t.TempDir(), "pid")
r := runner("SIFT_PDFTEXT_SLEEP=30s", "SIFT_PDFTEXT_PIDFILE="+pidfile)
r.Deadline = 300 * time.Millisecond
start := time.Now()
_, err := r.Run(context.Background(), three())
if !errors.Is(err, pdftext.ErrTimeout) {
t.Fatalf("err %v, want ErrTimeout", err)
}
if d := time.Since(start); d > 3*time.Second {
t.Errorf("Run returned after %v; the 300 ms deadline was not enforced", d)
}
b, err := os.ReadFile(pidfile)
if err != nil {
t.Fatalf("child never wrote its pid: %v", err)
}
pid, _ := strconv.Atoi(string(b))
if err := syscall.Kill(pid, 0); !errors.Is(err, syscall.ESRCH) {
t.Errorf("child %d still exists after Run returned (kill -0: %v)", pid, err)
}
}
// The caller's context cancels the child too.
func TestRunnerContextCancel(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 300*time.Millisecond)
defer cancel()
_, err := runner("SIFT_PDFTEXT_SLEEP=30s").Run(ctx, three())
if err == nil {
t.Fatal("cancelled run succeeded")
}
}
// Runner passes only Env to the child: nothing of the parent's environment leaks in.
func TestRunnerEnvIsExplicit(t *testing.T) {
t.Setenv("SIFT_PDFTEXT_SLEEP", "30s") // in the parent only
r := runner()
r.Deadline = 5 * time.Second
if _, err := r.Run(context.Background(), three()); err != nil {
t.Errorf("parent env leaked into the child (it slept): %v", err)
}
}
docs/plans/p5a/_files/tests/testdata/pdf/README.md
Vendored
+14
View File
@@ -0,0 +1,14 @@
# PDF fixtures
Only the two encrypted files are binary; every other PDF the tests use is built at test time by
`internal/pdftest`.
- `encrypted-user.pdf`: the `pdftest` three-page document, AES-256, user password `userpw`,
owner password `ownerpw`. Made with
`qpdf --encrypt userpw ownerpw 256 -- three.pdf encrypted-user.pdf`.
- `encrypted-owner.pdf`: the same document, AES-256 with an empty user password
(opens without one), owner password `ownerpw`:
`qpdf --encrypt "" ownerpw 256 -- three.pdf encrypted-owner.pdf`.
Both contain the phrases "Chapter one begins here." (page 1) and
"The quick brown fox is on page two." (page 2).
docs/plans/p5a/_files/tests/testdata/pdf/encrypted-owner.pdf
Vendored
BIN
View File
Binary file not shown.
docs/plans/p5a/_files/tests/testdata/pdf/encrypted-user.pdf
Vendored
+81
View File
@@ -0,0 +1,81 @@
%PDF-1.7
%¿÷¢þ
1 0 obj
<< /Extensions << /ADBE << /BaseVersion /1.7 /ExtensionLevel 8 >> >> /Outlines 3 0 R /PageMode /UseOutlines /Pages 4 0 R /Type /Catalog >>
endobj
2 0 obj
<< /Producer <43b5e97c0a7de15bc0f4dbcc052b5f80f21e0e3bede0eda33137092879ea433d> /Title <29f3ced7effab532ee871ab3ebc38a6869b6b7379f2fa12e139f3e2a1d1ee664> >>
endobj
3 0 obj
<< /Count 3 /First 5 0 R /Last 6 0 R /Type /Outlines >>
endobj
4 0 obj
<< /Count 3 /Kids [ 7 0 R 8 0 R 9 0 R ] /Type /Pages >>
endobj
5 0 obj
<< /Dest [ 7 0 R /XYZ null null null ] /Next 10 0 R /Parent 3 0 R /Title <d2951c68344226cca87b8f66af448a05c4ba3c4baf7c7863293f2328ce407d09> >>
endobj
6 0 obj
<< /Dest [ 9 0 R /XYZ null null null ] /Parent 3 0 R /Prev 10 0 R /Title <13b9a70c4f34e06e182eb823c2cb79a1f82c91f6c69ee89bf2093cd61d451407> >>
endobj
7 0 obj
<< /Contents 11 0 R /MediaBox [ 0 0 612 792 ] /Parent 4 0 R /Resources << /Font << /F1 12 0 R >> /XObject << /Im1 13 0 R >> >> /Type /Page >>
endobj
8 0 obj
<< /Contents 14 0 R /MediaBox [ 0 0 612 792 ] /Parent 4 0 R /Resources << /Font << /F1 12 0 R >> /XObject << /Im1 13 0 R >> >> /Type /Page >>
endobj
9 0 obj
<< /Contents 15 0 R /MediaBox [ 0 0 612 792 ] /Parent 4 0 R /Resources << /Font << /F1 12 0 R >> /XObject << /Im1 13 0 R >> >> /Type /Page >>
endobj
10 0 obj
<< /Dest [ 8 0 R /XYZ null null null ] /Next 6 0 R /Parent 3 0 R /Prev 5 0 R /Title <b734ddb0e84e981224c172818d645644e95ae16c94cf45d02ef870d181f97422> >>
endobj
11 0 obj
<< /Length 128 /Filter /FlateDecode >>
stream
jl<ħ|Ýkj>Kþ"æ"-‚'agcYÓ“¡Áù ‹ä¾Îç„Â^›™h9Àu¢ò4DY­>€ä‡ÝhSkvJxŠý␍÷À'‡ÿ­ù/טò£ Ð#¼Æo=ÖÛ:³1áÇ¢hÇôªÀè<Â|Hü“J‹xYz4�ÿÄendstream
endobj
12 0 obj
<< /BaseFont /Helvetica /Encoding /WinAnsiEncoding /Subtype /Type1 /Type /Font >>
endobj
13 0 obj
<< /BitsPerComponent 8 /ColorSpace /DeviceGray /Height 2 /Subtype /Image /Type /XObject /Width 2 /Length 32 /Filter /FlateDecode >>
stream
Ë™"€_œ¼Ë57Ó”Ì60Ë%œ1´V\Ó$mG&0FåMendstream
endobj
14 0 obj
<< /Length 112 /Filter /FlateDecode >>
stream
,#c¾}jè¹Ájw�)æ�„›d7Éwp¨íi»gvÍñÎâH»6oA•ï´vD}–Ð9·G`1wÆŽ×XŒÝ‰ž ÄrÜ›Vf:I¡)x9ÅéÌß“>��WãxŽÄ‡¡ãŠ¤Ô«âÊ‘dendstream
endobj
15 0 obj
<< /Length 160 /Filter /FlateDecode >>
stream
yx÷â°í%"(hß]�T~BùÞ]+$ík±Mû†e.ŒbÿPת®0ž;äòãæë%½Kû¾¤z‹C”¤²!̘£)(žœÛôŒáéÄ;/¬@yèºÚh¥Á qR|Ôcݹ²Ã((¼Õ{úc¿AÆeýÏΧDíÚ¼)‡(íÄ! · #5u·ké#¹âW’<8pŽ÷I1XQ"¥G endstream
endobj
16 0 obj
<< /CF << /StdCF << /AuthEvent /DocOpen /CFM /AESV3 /Length 32 >> >> /Filter /Standard /Length 256 /O <2a0820bbf93fff95a410525ae20d4abca153af1e3464706a4f83e49c8caa92e62e160b6871c75b9e47a6cd6dd15fe6ab> /OE <210b63a9a2305637af02dc6e89409285f61f82adaafa125e950042528164e2a3> /P -4 /Perms <23688a77e80aeb8b536511ce497c729e> /R 6 /StmF /StdCF /StrF /StdCF /U <98531c24f9d0bbabd16cc589be7618a59de91065126679a37b26b5d684cbcbc76d34bf68dc61eafe1681b4d5a21826d1> /UE <75ac70233a6a28bcd012febedaf3cccf6c25af218f3318a9fa9bc0f94a0e1b53> /V 5 >>
endobj
xref
0 17
0000000000 65535 f
0000000015 00000 n
0000000169 00000 n
0000000341 00000 n
0000000412 00000 n
0000000483 00000 n
0000000641 00000 n
0000000799 00000 n
0000000956 00000 n
0000001113 00000 n
0000001270 00000 n
0000001440 00000 n
0000001640 00000 n
0000001738 00000 n
0000001935 00000 n
0000002119 00000 n
0000002351 00000 n
trailer << /Info 2 0 R /Root 1 0 R /Size 17 /ID [<39c444018179222865caf844f4fb7254><39c444018179222865caf844f4fb7254>] /Encrypt 16 0 R >>
startxref
2899
%%EOF
go.mod
+5 -1
View File
@@ -1,9 +1,11 @@
module git.wntrmute.dev/kyle/sift
go 1.26.0
go 1.26.4
require (
github.com/JohannesKaufmann/html-to-markdown/v2 v2.5.2
github.com/go-pdfkit/extract v0.3.0
github.com/go-pdfkit/reader v0.6.0
github.com/go-shiori/go-readability v0.0.0-20251205110129-5db1dc9836f0
github.com/itchyny/gojq v0.12.19
github.com/pelletier/go-toml/v2 v2.4.3
@@ -21,6 +23,8 @@ require (
github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/go-opentype/opentype v0.12.0 // indirect
github.com/go-pdfkit/pdffont v0.3.0 // indirect
github.com/go-shiori/dom v0.0.0-20230515143342-73569d674e1c // indirect
github.com/gogs/chardet v0.0.0-20211120154057-b7413eaefb8f // indirect
github.com/google/uuid v1.6.0 // indirect
go.sum
+10
View File
@@ -13,6 +13,16 @@ github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XL
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/go-opentype/fonts v0.8.0 h1:77i3VPIH90GbstzNb21mk+an4WvEOe2idC6W+J0n0fw=
github.com/go-opentype/fonts v0.8.0/go.mod h1:C6yQL2apHItfEZ5hztpsHF0S5mlX/hklLlq/Z5fRG/g=
github.com/go-opentype/opentype v0.12.0 h1:wBlcDi+3ZaNZXEt5z+Ixr11/cYYwi5W+jX6yTl/qr1I=
github.com/go-opentype/opentype v0.12.0/go.mod h1:AOixevJf7XQaH7+WG+OMIOZEbYPXfMqklVk26Y6YTUU=
github.com/go-pdfkit/extract v0.3.0 h1:w1ansrvMNqd+Hy1p5fo1orh6hM8meb+xCNKNap/7omM=
github.com/go-pdfkit/extract v0.3.0/go.mod h1:EOWOcx3o0HDC3ryCMoE9pwNIb3mwVmnXBSpTRfO1VdY=
github.com/go-pdfkit/pdffont v0.3.0 h1:G5DKcAmsZJ0e17QhSrcUaL7PKEXKjUx4P/iGMl7bAbI=
github.com/go-pdfkit/pdffont v0.3.0/go.mod h1:bfmNLna1l1CljNX/Utg55YzFylovfmI7sJnvgA3bzKI=
github.com/go-pdfkit/reader v0.6.0 h1:KAabNOYUcTlZlNBTbG9bEhWP1NiZhjuhzUdavdTdfes=
github.com/go-pdfkit/reader v0.6.0/go.mod h1:fQFOVfCMUui1AdvD4qhimdyvvNr9KvvJ1S7IuKZjyV8=
github.com/go-shiori/dom v0.0.0-20230515143342-73569d674e1c h1:wpkoddUomPfHiOziHZixGO5ZBS73cKqVzZipfrLmO1w=
github.com/go-shiori/dom v0.0.0-20230515143342-73569d674e1c/go.mod h1:oVDCh3qjJMLVUSILBRwrm+Bc6RNXGZYtoh9xdvf1ffM=
github.com/go-shiori/go-readability v0.0.0-20251205110129-5db1dc9836f0 h1:A3B75Yp163FAIf9nLlFMl4pwIj+T3uKxfI7mbvvY2Ls=
internal/api/api.go
+2 -2
View File
@@ -30,8 +30,8 @@ import (
const (
maxReqBody = 64 << 10
writeTimeout = 120 * time.Second
requestDeadline = 90 * time.Second
writeTimeout = 240 * time.Second // P5a: a PDF may take 120 s to download + 20 s queued + 60 s to extract
requestDeadline = 210 * time.Second // P5a: the PDF path's worst case (see writeTimeout); other paths keep their own shorter timeouts
)
type Server struct {
internal/api/api_test.go
+4 -3
View File
@@ -222,8 +222,9 @@ func TestJSONJQ(t *testing.T) {
func TestUnsupportedType(t *testing.T) {
g := &guard.Guard{AllowPrefixes: []netip.Prefix{netip.MustParsePrefix("127.0.0.0/8")}}
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/pdf")
_, _ = w.Write([]byte("%PDF"))
// P5a: a PDF is supported now; a zip is still not.
w.Header().Set("Content-Type", "application/zip")
_, _ = w.Write([]byte("PK\x03\x04"))
}))
defer ts.Close()
s, key := setupAPI(t, g)
@@ -260,7 +261,7 @@ func TestJQTooLarge(t *testing.T) {
func TestHTTPServerWriteTimeout(t *testing.T) {
s := New("test", &guard.Guard{}, nil, nil, slog.New(slog.NewTextHandler(io.Discard, nil)))
hs := s.HTTPServer("127.0.0.1:0")
if hs.WriteTimeout != 120*time.Second {
if hs.WriteTimeout != 240*time.Second { // P5a: 120 s download + 20 s queue + 60 s extraction fit inside
t.Fatalf("WriteTimeout %s", hs.WriteTimeout)
}
}
internal/clean/clean.go
+1 -1
View File
@@ -11,8 +11,8 @@ import (
"time"
"unicode/utf8"
readability "github.com/go-shiori/go-readability"
htmltomarkdown "github.com/JohannesKaufmann/html-to-markdown/v2"
readability "github.com/go-shiori/go-readability"
"github.com/itchyny/gojq"
"golang.org/x/net/html"
"golang.org/x/net/html/charset"
internal/guard/iana.go
+1
View File
@@ -7,6 +7,7 @@ import "net/netip"
// rather than decoded. IsGlobalUnicast remains a final gate in CheckAddr.
//
// Sources:
//
// https://www.iana.org/assignments/iana-ipv4-special-registry
// https://www.iana.org/assignments/iana-ipv6-special-registry
var ianaDeny = []struct {
tools.go
+10
View File
@@ -0,0 +1,10 @@
//go:build tools
// Package sift pins the PDF engine chosen for P5a (docs/P5a.md §3) so go.mod and vendor/ carry it
// before the code that imports it exists. Remove this file once internal/pdftext imports both.
package main
import (
_ "github.com/go-pdfkit/extract"
_ "github.com/go-pdfkit/reader"
)
tools/run-plan.sh
Executable
+71
View File
@@ -0,0 +1,71 @@
#!/bin/sh
# Runs the tasks of a plan one after another, each in its own fresh OpenCode session, and stops at
# the first task that does not end cleanly. This is the owner's tool; the implementing model never
# runs it. Adapted from crossbar's tools/run-plan.sh.
#
# Usage: tools/run-plan.sh docs/plans/<plan> [FIRST] FIRST is a task number such as 03
# Options (environment):
# SIFT_MODEL model to use (default llama.cpp/ornith-1.5-35b-a3b)
# OPENCODE_FLAGS extra flags for `opencode run` (default --pure)
# DRY_RUN=1 print what would be run and stop
set -eu
plan="${1:?usage: tools/run-plan.sh docs/plans/<plan> [FIRST]}"
first="${2:-00}"
model="${SIFT_MODEL:-llama.cpp/ornith-1.5-35b-a3b}"
flags="${OPENCODE_FLAGS:---pure}"
log=docs/implementer-log.md
milestone=$(basename "$plan")
runs=".state/runs/$milestone"
# The model's own name for the attribution trailer: the part after the provider prefix.
mname="${model#*/}"
trailer="Co-Authored-By: $mname <$mname@llama.cpp.invalid>"
[ -d "$plan" ] || { echo "run-plan: $plan is not a directory" >&2; exit 2; }
[ -f "$log" ] || { echo "run-plan: run this from the repository root" >&2; exit 2; }
mkdir -p "$runs"
for task in "$plan"/[0-9][0-9]-*.md; do
name=$(basename "$task" .md)
number=$(echo "$name" | cut -c1-2)
[ "$number" -ge "$first" ] || continue
prompt="Read \`AGENTS.md\`, then read \`$task\` and do exactly that task. Your commit message must end with this exact trailer line: $trailer"
if [ "${DRY_RUN:-0}" = 1 ]; then
echo "opencode run -m $model $flags --title $milestone/$name \"$prompt\""
continue
fi
if [ -n "$(git status --porcelain)" ]; then
echo "run-plan: the working tree is not clean before $name; stopping" >&2
exit 1
fi
before=$(git rev-parse HEAD)
echo "=== $milestone/$name: started $(date '+%H:%M:%S')"
# shellcheck disable=SC2086
if ! opencode run -m "$model" $flags --title "$milestone/$name" "$prompt" \
> "$runs/$name.log" 2>&1; then
echo "run-plan: opencode exited with an error in $name; see $runs/$name.log" >&2
exit 1
fi
echo "=== $milestone/$name: session ended $(date '+%H:%M:%S')"
if [ "$(git rev-parse HEAD)" = "$before" ]; then
echo "run-plan: $name made no commit; stopping" >&2
exit 1
fi
if [ -n "$(git status --porcelain)" ]; then
echo "run-plan: $name left uncommitted changes; stopping" >&2
exit 1
fi
if ! git log -1 --format=%B | grep -qxF "$trailer"; then
echo "run-plan: $name's commit lacks the trailer '$trailer'; stopping" >&2
exit 1
fi
if ! grep -Eq "^\| *$milestone/$name *\|[^|]*\| *done *\|" "$log"; then
echo "run-plan: $name has no 'done' row in $log; stopping" >&2
exit 1
fi
echo "=== $milestone/$name: ok ($(git rev-parse --short HEAD))"
done
echo "run-plan: all tasks done"
vendor/github.com/go-opentype/opentype/.gitignore
Generated
+2
View File
@@ -0,0 +1,2 @@
coverage.out
*.out
vendor/github.com/go-opentype/opentype/LICENSE
Generated Vendored
+29
View File
@@ -0,0 +1,29 @@
BSD 3-Clause License
Copyright (c) 2026 the go-opentype/opentype authors
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
vendor/github.com/go-opentype/opentype/README.md
Generated Vendored
+204
View File
@@ -0,0 +1,204 @@
# go-opentype/opentype
[![CI](https://github.com/go-opentype/opentype/actions/workflows/ci.yml/badge.svg)](https://github.com/go-opentype/opentype/actions/workflows/ci.yml)
[![pkg.go.dev](https://img.shields.io/badge/pkg.go.dev-opentype-007d9c?logo=go&logoColor=white)](https://pkg.go.dev/github.com/go-opentype/opentype)
![coverage](https://img.shields.io/badge/coverage-100%25-1a7f37)
![go](https://img.shields.io/badge/Go-1.26.4%2B-00ADD8?logo=go&logoColor=white)
[![license](https://img.shields.io/badge/license-BSD--3--Clause-blue)](./LICENSE)
Pure-Go, **CGO=0, standard-library-only** parser, shaper and anti-aliased
rasteriser for TrueType/OpenType fonts. No `golang.org/x/*`, no third-party
modules — it imports only `image`, `encoding/binary`, `math`, `errors` and
friends, so it builds for every Go target including `GOOS=js GOARCH=wasm`.
It exists to functionally replace the narrow slice of
`golang.org/x/image/font/opentype` that a glyph-blitting UI needs — parse a
font blob, build a face at a pixel size, pull per-rune advances and 8-bit
alpha coverage masks — and then goes further: CFF/CFF2 outlines, variable
fonts, GPOS/GSUB shaping, kerning and TrueType/CFF hinting are all in scope.
## Install
```sh
go get github.com/go-opentype/opentype
```
## Quick start
```go
package main
import (
"fmt"
"os"
"github.com/go-opentype/opentype"
)
func main() {
ttf, err := os.ReadFile("MyFont.ttf") // or .otf (CFF/CFF2)
if err != nil {
panic(err)
}
f, err := opentype.Parse(ttf)
if err != nil {
panic(err)
}
face := f.NewFace(16) // 16px per em
m := face.Metrics() // Metrics{Ascent, Descent, Height} in pixels
w := face.Measure("Hello") // total advance width in pixels
adv := face.Advance('H') // one rune's advance in pixels
fmt.Println(m, w, adv)
// Rasterise a glyph with (penX, baselineY) as the pen origin on the baseline.
bounds, mask, maskp, advance, ok := face.GlyphMask('H', 0, 16)
if ok && mask != nil {
// Composite the coverage mask onto your destination:
// pixel (maskp.X+i, maskp.Y+j) covers destination (bounds.Min.X+i, bounds.Min.Y+j).
_ = bounds
_ = maskp
_ = advance
}
}
```
The `GlyphMask` shape deliberately mirrors `x/image`'s `font.Face.Glyph`, so
swapping this in for the x/image face is mechanical.
## Features
- **sfnt container** parsing (`0x00010000` TrueType and `OTTO` CFF magics)
- **TrueType `glyf` outlines** — simple and composite glyphs, implied
on-curve midpoint synthesis, cycle/depth-guarded composites
- **CFF and CFF2 outlines** — Type 2 charstrings, subroutines, `seac`
accent composition, CFF2 variable-font blend operators
- **Variable fonts** — `fvar` axes and named instances, `avar` axis-value
mapping, `gvar`/CFF2 glyph-outline interpolation, and `HVAR`/`VVAR`/`MVAR`
metric variation, via `(*Face).SetVariation`
- **GSUB/GPOS shaping** — ligatures, contextual and positional-form
substitution (`isol`/`init`/`medi`/`fina`), pair/single/cursive
positioning, mark-to-base and mark-to-mark attachment
- **Kerning** — GPOS pair positioning with legacy `kern`-table fallback
through a single `Kerner`
- **Hinting** — TrueType instruction interpreter and a CFF/Type 2 stem
grid-fitter, toggled per `Face` with `SetHinting`; optional
`SetStemDarkening`
- **Vertical metrics** — `vhea`/`vmtx`/`VORG`-aware vertical advances and
origins for vertical writing modes
- **OpenType `MATH` table** — math-typesetting metrics (constants, per-glyph
italic correction / top-accent attachment / four-corner kerning, and
stretchy-glyph size variants and assemblies), exposed pixel-scaled through
a `Face`
- **Font descriptors** — the OS/2- and `head`/`post`-derived scalars a PDF
`/FontDescriptor` or CSS `@font-face` needs (`Ascent`, `Descent`,
`FontBBox`, `CapHeight`, `XHeight`, `ItalicAngle`, `WeightClass`,
`WidthClass`, `StemV`, `Flags`, `IsFixedPitch`/`IsItalic`/`IsSerif`)
- **Subsetting & instancing** — `SubsetTrueType`/`SubsetCFF` cut a font down
to a glyph set (closing composite/`seac` dependencies and remapping ids),
and `Instance`/`InstanceBytes` bake a variable font to a static instance —
the writer side needed to embed fonts in PDFs
- **Anti-aliased rasterisation** via 4×4 supersampling under the non-zero
winding rule
A `Font` is immutable after `Parse` and safe for concurrent use. A `Face`
caches rasterised glyphs and is **not** safe for concurrent use; build one
`Face` per goroutine if needed.
## API tour
| Symbol | Purpose |
| --- | --- |
| `Parse(b []byte) (*Font, error)` | Decode an sfnt (TrueType or CFF/OTTO) font blob. |
| `(*Font).NumGlyphs() int` | Glyph count (from `maxp`). |
| `(*Font).GlyphIndex(r rune) (GlyphIndex, bool)` | Map a rune via the cmap. |
| `(*Font).GlyphIndexVariation(r, vs rune) (GlyphIndex, bool)` | Map a rune + Unicode variation selector. |
| `(*Font).Axes() []Axis` | Variable-font design axes (`fvar`). |
| `(*Font).NamedInstances() []NamedInstance` | Named positions in the variation space. |
| `(*Font).GPOS() *GPOS` / `(*Font).GSUB() *GSUB` | Parsed layout tables, or `nil`. |
| `(*Font).HasVerticalMetrics() bool` | Whether `vhea`/`vmtx` are present. |
| `(*Font).NewFace(sizePx int) *Face` | Build a sized face (scale = `sizePx / unitsPerEm`). |
| `(*Face).Metrics() Metrics` | `Ascent`, `Descent`, `Height` in pixels. |
| `(*Face).Advance(r rune) int` / `(*Face).Measure(s string) int` | Rune / string advance in pixels. |
| `(*Face).Kern(prev, r rune) int` / `(*Face).MeasureKerned(s string) int` | GPOS-or-`kern` pair adjustment. |
| `(*Face).GlyphMask(r, x, y) (image.Rectangle, *image.Alpha, image.Point, int, bool)` | Rasterised glyph + advance. |
| `(*Face).Shape(text string, features ...string) []GlyphIndex` | GSUB-substituted glyph run. |
| `(*Face).ShapePositioned(text string, features ...string) []PositionedGlyph` | GSUB + GPOS glyph run with pen offsets. |
| `(*Face).SetVariation(coords map[string]float64)` | Instance a variable font at the given axis coordinates. |
| `(*Face).SetHinting(on bool)` / `(*Face).SetStemDarkening(on bool)` | Toggle the TrueType/CFF hinter. |
| `(*Face).VerticalAdvance(r rune) int` / `(*Face).VerticalOrigin(r rune) (int, bool)` | Vertical writing-mode metrics. |
| `(*Font).HasMath() bool` | Whether the font carries an OpenType `MATH` table. |
| `(*Face).MathConstant(w MathConstant) int` / `(*Face).ItalicCorrection(g GlyphIndex) int` / `(*Face).MathVariants(g, vertical) ([]MathVariant, *MathAssembly)` | Pixel-scaled `MATH`-table metrics for a math-layout engine. |
| `(*Font).Ascent/Descent/LineGap/FontBBox/CapHeight/XHeight/ItalicAngle/WeightClass/WidthClass/StemV/Flags/IsFixedPitch/IsItalic/IsSerif` | Font-descriptor scalars for PDF `/FontDescriptor` and CSS `@font-face`. |
| `(*Font).Instance(coords map[string]float64) (*Font, error)` / `(*Font).InstanceBytes(coords) ([]byte, error)` | Bake a variable font to a static instance (as a `*Font` or a new sfnt blob). |
| `(*Font).SubsetTrueType(gids []GlyphIndex) ([]byte, map[GlyphIndex]GlyphIndex, error)` | Cut a TrueType font to a glyph set, closing composites and returning the old→new id map. |
| `(*Font).SubsetCFF(gids []GlyphIndex) ([]byte, error)` | Cut a CFF/OTTO font to a glyph set (closing `seac` accents). |
See [`example_test.go`](./example_test.go) for runnable examples of each of
these, and `go doc github.com/go-opentype/opentype` for the full reference.
## Support matrix
- sfnt container (`0x00010000` and `true` TrueType magics, `OTTO` CFF magic)
- `head`, `maxp`, `hhea`, `hmtx` (including the trailing-run shared advance)
- `cmap` formats **0, 2, 4, 6, 8, 10, 12, 13** and **14** (Unicode variation
sequences), preferring the richest available (12/13 over 4/6)
- `loca` (short and long), `glyf` simple and composite TrueType glyphs
- CFF and CFF2 (`OTTO`) Type 2 charstring outlines, including `seac`
- `fvar`/`avar`/`gvar`/`HVAR`/`VVAR`/`MVAR` and CFF2 blends for variable
fonts, plus `Instance`/`InstanceBytes` to bake a static instance
- `GSUB`/`GPOS`/`GDEF`, the legacy `kern` table
- OS/2 + `head`/`post` **font-descriptor** scalars (`Ascent`, `Descent`,
`FontBBox`, `CapHeight`, `XHeight`, `ItalicAngle`, `WeightClass`,
`WidthClass`, `StemV`, `Flags`, …) for PDF/CSS embedding
- **subsetting** — `SubsetTrueType` (glyf, composite-closure aware, with an
old→new id map) and `SubsetCFF` (`seac`-closure aware)
- the OpenType **`MATH`** table — math-typesetting metrics (constants, per-glyph
italic correction / top-accent attachment / corner kerning, and stretchy-glyph
size variants and assemblies), exposed pixel-scaled through a `Face`
(`HasMath`, `MathConstant`, `ItalicCorrection`, `TopAccentAttachment`,
`MathKern`, `MathVariants`); math *layout* is left to a higher-level engine
- TrueType instruction hinting and a CFF stem grid-fitter/darkener
- `vhea`/`vmtx`/`VORG` vertical metrics
- anti-aliased rasterisation via 4×4 supersampling under the non-zero
winding rule
Rasterisation uses uniform supersampling (not the delta-hinted rasterisation
of a native TrueType/CFF hinter's drop-out control), so it favours
correctness and portability over the very last drop of small-size sharpness.
## Testing
Tests never depend on an external font to reach 100% coverage: they
synthesise minimal-but-valid TrueType and CFF fonts in memory to
deterministically exercise every parse, shape and raster branch, including
the error paths. A real-world font, Adobe's Source Serif 4 (SIL Open Font
License, bundled under [`testdata/`](./testdata)), is used in
[`example_test.go`](./example_test.go) and in a handful of sanity-check
tests, so the documented examples double as an end-to-end smoke test against
production CFF charstrings and Private DICTs. CI enforces **100.0% statement
coverage**, `go vet`, and a cross-compile smoke over
`linux/{amd64,arm64,riscv64,loong64,ppc64le,s390x}`, `js/wasm`,
`darwin/arm64` and `windows/amd64`.
## Part of the go-opentype pure-Go text stack
`go-opentype/opentype` is the parsing/shaping/rasterising engine at the base
of a dependency-free text stack:
- **[opentype](https://github.com/go-opentype/opentype)** (this repo) — font
parsing, GSUB/GPOS shaping, hinting and rasterisation.
- **[bidi](https://github.com/go-opentype/bidi)** — a full Unicode
Bidirectional Algorithm (UBA) implementation, for ordering mixed
left-to-right/right-to-left text before it is shaped.
- **[shape](https://github.com/go-opentype/shape)** — a HarfBuzz-lite
complex-script shaper (Arabic, Indic, Hangul, USE, Egyptian
hieroglyphs, ...) built on this package's GSUB/GPOS engine.
- **[fonts](https://github.com/go-opentype/fonts)** — 46 bundled OFL/BSD
font families (Latin, non-Latin scripts and CJK), per-family lazily
`go:embed`-ed, ready to feed to `Parse`.
## License
BSD-3-Clause. See [LICENSE](./LICENSE).
vendor/github.com/go-opentype/opentype/advance.go
Generated Vendored
+43
View File
@@ -0,0 +1,43 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
// This file adds by-glyph-index advance accessors that honour the face's current
// variation instance. Face.Advance and Face.VerticalAdvance take a rune and go
// through the cmap; a text layer that has already resolved glyph indices (a
// shaper, or a PDF writer emitting a /W width array) needs the advance of a glyph
// it names directly, at the same instance the outlines are rendered for. Without
// these it would have to rasterise the glyph or reach past the cmap by hand.
// AdvanceIndex returns the horizontal advance of glyph gid in whole pixels at the
// face's size, honouring the face's current variation (the HVAR delta, or the
// gvar phantom-point fallback, is folded in exactly as Face.Advance does for a
// rune). With no variation set it is the base advance scaled to pixels. It is the
// by-glyph-index counterpart of Face.Advance, for a caller that already holds a
// glyph id rather than a rune. An out-of-range gid advances by zero.
func (fc *Face) AdvanceIndex(gid GlyphIndex) int {
return roundInt(fc.hAdvanceUnits(gid) * fc.scale)
}
// AdvanceIndexUnits returns the horizontal advance of glyph gid in font units at
// the face's current variation, as a float64 (the value AdvanceIndex rounds and
// scales to pixels). It is the exact width a PDF /W array wants — font-unit widths
// that track the instanced font — without the caller re-deriving the HVAR delta.
// An out-of-range gid advances by zero.
func (fc *Face) AdvanceIndexUnits(gid GlyphIndex) float64 {
return fc.hAdvanceUnits(gid)
}
// VerticalAdvanceIndex returns the vertical advance (advance height) of glyph gid
// in whole pixels at the face's size, honouring the face's current variation (the
// VVAR delta is folded in). When the font has no vmtx table the advance is one em.
// It is the by-glyph-index counterpart of Face.VerticalAdvance.
func (fc *Face) VerticalAdvanceIndex(gid GlyphIndex) int {
adv := float64(fc.font.verticalAdvance(gid))
if fc.varCoords != nil {
adv += fc.font.verticalAdvanceDelta(int(gid), fc.varNorm)
}
return roundInt(adv * fc.scale)
}
vendor/github.com/go-opentype/opentype/avar.go
Generated Vendored
+77
View File
@@ -0,0 +1,77 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import "fmt"
// This file implements the optional 'avar' (Axis Variations) table. It refines
// the linear normalization performed by NormalizeCoords: for each axis it maps
// the normalized coordinate through a piecewise-linear segment map, letting a
// font make, say, the visual midpoint of a weight axis land somewhere other
// than the arithmetic midpoint. A font without 'avar' normalizes linearly.
// avarMap is one (from, to) control point of an axis segment map, both in the
// normalized F2Dot14 domain decoded to float64.
type avarMap struct {
from, to float64
}
// avarTable holds one segment map per axis, in axis order.
type avarTable struct {
segments [][]avarMap
}
// parseAvar decodes the 'avar' table: a header followed by one SegmentMaps
// record per axis, each a count and that many (fromCoordinate, toCoordinate)
// F2Dot14 pairs.
func (f *Font) parseAvar(b []byte) error {
r := reader{b: b}
r.u16() // majorVersion
r.u16() // minorVersion
r.u16() // reserved
axisCount := int(r.u16())
segments := make([][]avarMap, axisCount)
for i := range segments {
count := int(r.u16())
maps := make([]avarMap, count)
for j := range maps {
maps[j].from = r.f2dot14()
maps[j].to = r.f2dot14()
}
segments[i] = maps
}
if r.err != nil {
return fmt.Errorf("opentype: avar table: %w", r.err)
}
f.avar = &avarTable{segments: segments}
return nil
}
// apply maps a normalized coordinate v for the given axis through its segment
// map. Control points are assumed sorted by ascending fromCoordinate (as the
// specification requires). An axis with fewer than two control points, or an
// index past the table, is treated as identity.
func (a *avarTable) apply(axis int, v float64) float64 {
if axis >= len(a.segments) {
return v
}
seg := a.segments[axis]
if len(seg) < 2 {
return v
}
if v <= seg[0].from {
return seg[0].to
}
for i := 1; i < len(seg); i++ {
if v < seg[i].from {
lo, hi := seg[i-1], seg[i]
return lo.to + (hi.to-lo.to)*(v-lo.from)/(hi.from-lo.from)
}
if v == seg[i].from {
return seg[i].to
}
}
return seg[len(seg)-1].to
}
vendor/github.com/go-opentype/opentype/barecff.go
Generated Vendored
+202
View File
@@ -0,0 +1,202 @@
// Copyright (c) 2026, the go-opentype authors
// All rights reserved.
//
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import (
"fmt"
"math"
)
// ParseCFF decodes a bare CFF font program — a Compact Font Format font that
// is not wrapped in an sfnt container, which is how a PDF carries one as a
// FontFile3 of subtype Type1C. Such a program has no head, no hmtx and no
// character map: the size of its units comes from its font matrix, its
// advances from the charstrings themselves, and the way from a name or a byte
// to a glyph from its own charset and encoding. [Font.HasCharacterMap] is
// false for the result, and [Font.GlyphIndexByName] and
// [Font.GlyphIndexByCode] are how it is addressed.
//
// The byte slice is retained, not copied, and must not be mutated afterwards.
func ParseCFF(b []byte) (*Font, error) {
cff, err := parseCFF(b)
if err != nil {
return nil, err
}
n := len(cff.charStrings)
if n == 0 {
return nil, fmt.Errorf("opentype: cff: the font has no glyphs")
}
f := &Font{
cff: cff,
numGlyphs: n,
unitsPerEm: unitsPerEmOf(cff.fontMatrix),
numberOfHMetrics: n,
}
f.advances, f.lsbs = cff.metrics(n)
// A bare program says nothing about how the line is spaced; a page of one
// is laid out by whatever carries it, which for a PDF is the text matrix.
f.ascender, f.descender = f.unitsPerEm, 0
f.buildNames()
return f, nil
}
// unitsPerEmOf turns a font matrix into the number of units to the em, which
// is what the rest of this package counts in. A matrix of a thousandth is the
// usual one and gives a thousand.
func unitsPerEmOf(m [6]float64) int {
upem := int(math.Round(1 / m[0]))
if upem < 16 || upem > 16384 {
return 1000
}
return upem
}
// metrics is the advance and left side bearing of every glyph, read from the
// charstrings. A charstring that does not say gets the font's default.
func (c *cffTable) metrics(n int) (advances, lsbs []int) {
advances = make([]int, n)
lsbs = make([]int, n)
def, nominal := 0.0, 0.0
if c.priv != nil {
def, nominal = c.priv.defaultWidthX, c.priv.nominalWidthX
}
for gid := 0; gid < n; gid++ {
w := def
if delta, ok := charstringWidth(c.charStrings[gid]); ok {
w = nominal + delta
}
advances[gid] = int(math.Round(w))
}
return advances, lsbs
}
// charstringWidth reads the optional width a Type 2 charstring may carry in
// front of its first stem, move or endchar operator. It reads operands only,
// and gives up at the first operator that could hide one behind a subroutine
// rather than guess.
func charstringWidth(cs []byte) (float64, bool) {
var nOperands int
var first float64
for i := 0; i < len(cs); {
b := cs[i]
switch {
case b >= 32 || b == 28:
v, next, ok := t2Operand(cs, i)
if !ok {
return 0, false
}
if nOperands == 0 {
first = v
}
nOperands++
i = next
case b == 1 || b == 3 || b == 18 || b == 23: // hstem, vstem, hstemhm, vstemhm
return first, nOperands%2 == 1
case b == 19 || b == 20: // hintmask, cntrmask
return first, nOperands%2 == 1
case b == 21: // rmoveto
return first, nOperands > 2
case b == 22 || b == 4: // hmoveto, vmoveto
return first, nOperands > 1
case b == 14: // endchar
return first, nOperands == 1 || nOperands == 5
default:
// A subroutine call, or an arithmetic operator: the width, if
// there is one, is no longer in front of us.
return 0, false
}
}
return 0, false
}
// t2Operand reads one Type 2 charstring operand, giving its value and where
// the next byte is.
func t2Operand(cs []byte, i int) (float64, int, bool) {
b := cs[i]
switch {
case b == 28:
if i+2 >= len(cs) {
return 0, 0, false
}
return float64(int16(uint16(cs[i+1])<<8 | uint16(cs[i+2]))), i + 3, true
case b < 247:
return float64(int(b) - 139), i + 1, true
case b < 251:
if i+1 >= len(cs) {
return 0, 0, false
}
return float64((int(b)-247)*256 + int(cs[i+1]) + 108), i + 2, true
case b < 255:
if i+1 >= len(cs) {
return 0, 0, false
}
return float64(-(int(b)-251)*256 - int(cs[i+1]) - 108), i + 2, true
default: // 255: a sixteen-sixteen fixed-point number
if i+4 >= len(cs) {
return 0, 0, false
}
v := int32(uint32(cs[i+1])<<24 | uint32(cs[i+2])<<16 | uint32(cs[i+3])<<8 | uint32(cs[i+4]))
return float64(v) / 65536, i + 5, true
}
}
// buildNames indexes the font by the names it gives its glyphs, so that a
// document addressing it by name has somewhere to look. A font addressed by
// identifier names nothing, and this leaves it alone.
func (f *Font) buildNames() {
if f.cff == nil || f.cff.isCID {
return
}
f.glyphNames = make(map[string]GlyphIndex, f.numGlyphs)
for gid := 0; gid < f.numGlyphs; gid++ {
name, ok := f.cff.glyphName(gid)
if !ok || name == "" {
continue
}
if _, seen := f.glyphNames[name]; !seen {
f.glyphNames[name] = GlyphIndex(gid)
}
}
}
// GlyphName is what the font calls a glyph. ok is false for a font that does
// not name its glyphs — a TrueType font usually does not, and a CFF font
// addressed by character identifier never does.
func (f *Font) GlyphName(gid GlyphIndex) (string, bool) {
if f.t1 != nil {
return f.t1.glyphName(int(gid))
}
if f.cff == nil || f.cff.isCID {
return "", false
}
return f.cff.glyphName(int(gid))
}
// GlyphIndexByName maps a glyph name to its index. ok is false when the font
// names no glyphs or has none of that name.
func (f *Font) GlyphIndexByName(name string) (GlyphIndex, bool) {
gid, ok := f.glyphNames[name]
return gid, ok
}
// GlyphIndexByCode maps a byte to a glyph through the font program's own
// built-in encoding — the Standard Encoding unless the program said otherwise.
// ok is false for a font that carries no such encoding, which is every
// TrueType font and every CFF font addressed by character identifier.
func (f *Font) GlyphIndexByCode(code byte) (GlyphIndex, bool) {
if f.t1 != nil {
return f.t1.glyphByCode(code)
}
if f.cff == nil || f.cff.isCID {
return 0, false
}
gid, ok := f.cff.glyphByCode(code)
if !ok || gid >= f.numGlyphs {
return 0, false
}
return GlyphIndex(gid), true
}
vendor/github.com/go-opentype/opentype/cff.go
Generated Vendored
+1202
View File
File diff suppressed because it is too large Load Diff
vendor/github.com/go-opentype/opentype/cff2.go
Generated Vendored
+724
View File
@@ -0,0 +1,724 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import "fmt"
// This file adds CFF2 (variable Compact Font Format) outline support, so
// variable OpenType/CFF fonts (variable .otf, e.g. Source Serif Variable and
// many Adobe families) can be parsed and instanced. CFF2 is a close variant of
// CFF1/Type2 (cff.go): it shares the INDEX/DICT encodings and nearly all of the
// Type2 charstring path operators, so this file reuses cff.go's INDEX/DICT
// parsers, the Type2 operand decoder, and the whole Bézier-flattening pen
// (t2machine's moveTo/lineTo/curveTo and the curve helpers) to emit the same
// []contour model glyf.go produces.
//
// CFF2 differs from CFF1 in these ways, all handled here:
// - Header carries a 16-bit topDictLength; the Top DICT is a single DICT (no
// Top DICT INDEX), and every INDEX uses a 32-bit count (parseIndex2).
// - There is no per-glyph advance width, no endchar, no seac and no charset;
// charstrings simply end at their last byte.
// - Outlines vary with the design space: the 'blend' and 'vsindex' charstring
// operators pull deltas from the shared OpenType ItemVariationStore
// ('vstore') and interpolate operands in place at a normalized coordinate.
// - The Private DICT and its Local Subrs live per Font DICT in the FDArray,
// selected per glyph by FDSelect.
//
// CFF2 hinting is supported: hstem/vstem/hintmask stems and each Font DICT's
// Private-DICT hint parameters are recorded here and grid-fitted by cffhint.go
// exactly as for CFF1, so Face.SetHinting(true) grid-fits CFF2 glyphs too.
// Deferred (not implemented here, matching cff.go's scope): FDSelect format 4
// (for >65535 glyphs) and non-format-1 ItemVariationStores are rejected; metric
// variation (HVAR and friends) is out of scope. The interpreter's operand stack
// limit follows the CFF2 maximum.
// maxCFF2Stack is the CFF2 charstring operand-stack limit (the spec permits 513
// entries; blend expands the stack well past the Type2 limit of 48).
const maxCFF2Stack = 513
// cff2Table is a parsed CFF2 table: the CharStrings INDEX (one charstring per
// glyph), the Global Subr INDEX with its bias, the per-Font-DICT Local Subr
// INDEXes (with biases and default variation-store indices), the per-glyph
// Font-DICT selector, and the shared ItemVariationStore driving blends.
type cff2Table struct {
charStrings [][]byte
globalSubrs [][]byte
gsubrBias int
fdSelect []int // gid -> Font DICT index; nil means every glyph uses 0
localSubrs [][][]byte // per Font DICT: its Local Subr INDEX
localBias []int // per Font DICT: its Local Subr bias
privVsindex []int // per Font DICT: default vsindex from the Private DICT
privHints []*cffPrivate // per Font DICT: parsed Private-DICT hint parameters
vstore *itemVarStore
}
// regionAxis is one axis of one variation region: the tent function's lower,
// peak and upper normalized coordinates.
type regionAxis struct {
start, peak, end float64
}
// ivData is one ItemVariationData subtable reduced to what a CFF2 blend needs:
// the ordered list of region indices whose deltas the blend supplies. The delta
// values themselves come from the charstring, not from the store.
type ivData struct {
regionIndices []int
}
// itemVarStore is the shared OpenType ItemVariationStore (format 1) referenced
// by the Top DICT 'vstore' operator: the region list plus each variation-data
// subtable's region ordering.
type itemVarStore struct {
axisCount int
regions [][]regionAxis
data []ivData
}
// parseIndex2 decodes a CFF2 INDEX (identical to a CFF1 INDEX but with a 32-bit
// count) at r's current position, returning each stored object as a sub-slice of
// r.b and leaving r positioned just past the INDEX.
func parseIndex2(r *reader) ([][]byte, error) {
count := int(r.u32())
if r.err != nil {
return nil, fmt.Errorf("opentype: cff2 index count: %w", r.err)
}
if count == 0 {
return nil, nil
}
offSize := int(r.u8())
if offSize < 1 || offSize > 4 {
return nil, fmt.Errorf("opentype: cff2 index: bad offSize %d", offSize)
}
offsets := make([]int, count+1)
for k := range offsets {
offsets[k] = readOffset(r, offSize)
}
if r.err != nil {
return nil, fmt.Errorf("opentype: cff2 index offsets: %w", r.err)
}
base := r.pos - offsets[0]
items := make([][]byte, count)
for k := 0; k < count; k++ {
start, end := base+offsets[k], base+offsets[k+1]
if start < 0 || end > len(r.b) || start > end {
return nil, fmt.Errorf("opentype: cff2 index: item %d out of range", k)
}
items[k] = r.b[start:end]
}
r.pos = base + offsets[count]
return items, nil
}
// parseDict2 decodes a CFF2 DICT into a map from operator to its operand list.
// It differs from cff.go's parseDict only in the operator range: CFF2 adds the
// single-byte operators vsindex (22) and blend (23) to the Private DICT and
// vstore (24) to the Top DICT, all of which are reserved in CFF1. Operand bytes
// (28, 29, 30 and 32..255) share cff.go's parseDictOperand decoder.
func parseDict2(b []byte) (map[int][]float64, error) {
d := map[int][]float64{}
var operands []float64
i := 0
for i < len(b) {
b0 := b[i]
if b0 < 28 { // operator (12 introduces a two-byte operator)
op := int(b0)
i++
if b0 == 12 {
if i >= len(b) {
return nil, fmt.Errorf("opentype: cff2 dict: truncated operator")
}
op = 1200 + int(b[i])
i++
}
d[op] = operands
operands = nil
continue
}
v, n, err := parseDictOperand(b, i)
if err != nil {
return nil, err
}
operands = append(operands, v)
i += n
}
return d, nil
}
// parseCFF2 decodes a CFF2 table: the 5-byte header, the Top DICT, the Global
// Subr INDEX, the CharStrings INDEX, the optional ItemVariationStore, and the
// FDArray/FDSelect that assign each glyph its Private DICT and Local Subrs.
func parseCFF2(data []byte) (*cff2Table, error) {
if len(data) < 5 {
return nil, fmt.Errorf("opentype: cff2 header: %w", errTruncated)
}
if data[0] != 2 {
return nil, fmt.Errorf("opentype: cff2: unsupported major version %d", data[0])
}
hdrSize := int(data[2])
hr := reader{b: data, pos: 3}
topDictLen := int(hr.u16())
if hdrSize < 5 || hdrSize > len(data) {
return nil, fmt.Errorf("opentype: cff2: bad headerSize %d", hdrSize)
}
if hdrSize+topDictLen > len(data) {
return nil, fmt.Errorf("opentype: cff2: Top DICT out of range")
}
top, err := parseDict2(data[hdrSize : hdrSize+topDictLen])
if err != nil {
return nil, err
}
r := &reader{b: data, pos: hdrSize + topDictLen}
gsubrs, err := parseIndex2(r) // Global Subr INDEX
if err != nil {
return nil, err
}
csOff, ok := dictInt(top, 17, 0)
if !ok {
return nil, fmt.Errorf("opentype: cff2: missing CharStrings offset")
}
if csOff < 0 || csOff > len(data) {
return nil, fmt.Errorf("opentype: cff2: CharStrings offset out of range")
}
charStrings, err := parseIndex2(&reader{b: data, pos: csOff})
if err != nil {
return nil, err
}
var vstore *itemVarStore
if vsOff, ok := dictInt(top, 24, 0); ok {
if vstore, err = parseItemVarStore(data, vsOff); err != nil {
return nil, err
}
}
localSubrs, localBias, privVsindex, privHints, err := parseCFF2FDArray(data, top)
if err != nil {
return nil, err
}
fdSelect, err := parseCFF2FDSelect(data, top, len(charStrings), len(localSubrs))
if err != nil {
return nil, err
}
return &cff2Table{
charStrings: charStrings,
globalSubrs: gsubrs,
gsubrBias: subrBias(len(gsubrs)),
fdSelect: fdSelect,
localSubrs: localSubrs,
localBias: localBias,
privVsindex: privVsindex,
privHints: privHints,
vstore: vstore,
}, nil
}
// parseItemVarStore decodes the ItemVariationStore at offset off in data, whose
// data is prefixed with a 16-bit length. Only format 1 is supported. For CFF2
// only the region list and each subtable's region ordering are needed; the
// stored delta sets are not read.
func parseItemVarStore(data []byte, off int) (*itemVarStore, error) {
lr := reader{b: data, pos: off}
length := int(lr.u16())
if lr.err != nil {
return nil, fmt.Errorf("opentype: cff2 vstore length: %w", lr.err)
}
start := lr.pos
if start+length > len(data) {
return nil, fmt.Errorf("opentype: cff2 vstore out of range")
}
ivs := data[start : start+length]
r := reader{b: ivs}
if format := r.u16(); format != 1 {
return nil, fmt.Errorf("opentype: cff2 vstore: unsupported format %d", format)
}
regionListOff := int(r.u32())
ivdCount := int(r.u16())
ivdOffsets := make([]int, ivdCount)
for i := range ivdOffsets {
ivdOffsets[i] = int(r.u32())
}
if r.err != nil {
return nil, fmt.Errorf("opentype: cff2 vstore header: %w", r.err)
}
rr := reader{b: ivs, pos: regionListOff}
axisCount := int(rr.u16())
regionCount := int(rr.u16())
regions := make([][]regionAxis, regionCount)
for i := range regions {
axes := make([]regionAxis, axisCount)
for a := range axes {
axes[a].start = rr.f2dot14()
axes[a].peak = rr.f2dot14()
axes[a].end = rr.f2dot14()
}
regions[i] = axes
}
if rr.err != nil {
return nil, fmt.Errorf("opentype: cff2 vstore regions: %w", rr.err)
}
datas := make([]ivData, ivdCount)
for i, o := range ivdOffsets {
dr := reader{b: ivs, pos: o}
dr.u16() // itemCount (delta sets are not consumed by blend)
dr.u16() // wordDeltaCount
regionIndexCount := int(dr.u16())
idx := make([]int, regionIndexCount)
for k := range idx {
ri := int(dr.u16())
if ri >= regionCount {
return nil, fmt.Errorf("opentype: cff2 vstore: region index %d out of range", ri)
}
idx[k] = ri
}
if dr.err != nil {
return nil, fmt.Errorf("opentype: cff2 vstore data %d: %w", i, dr.err)
}
datas[i] = ivData{regionIndices: idx}
}
return &itemVarStore{axisCount: axisCount, regions: regions, data: datas}, nil
}
// regionScalar returns how strongly region ri applies at the normalized
// coordinate coords, reusing the shared tent-function evaluator (gvar.go).
func (vs *itemVarStore) regionScalar(ri int, coords []float64) float64 {
reg := vs.regions[ri]
lower := make([]float64, len(reg))
peak := make([]float64, len(reg))
upper := make([]float64, len(reg))
for a, ax := range reg {
lower[a], peak[a], upper[a] = ax.start, ax.peak, ax.end
}
return supportScalar(coords, lower, peak, upper)
}
// parseCFF2FDArray decodes the FDArray (an INDEX of Font DICTs) and each Font
// DICT's Private DICT, returning the per-Font-DICT Local Subr INDEXes, their
// biases and their default vsindex values. A CFF2 table without an FDArray is
// treated as having a single Font DICT with no Local Subrs.
func parseCFF2FDArray(data []byte, top map[int][]float64) (subrs [][][]byte, bias, vsindex []int, hints []*cffPrivate, err error) {
off, ok := dictInt(top, 1236, 0)
if !ok {
return [][][]byte{nil}, []int{subrBias(0)}, []int{0}, []*cffPrivate{nil}, nil
}
if off < 0 || off > len(data) {
return nil, nil, nil, nil, fmt.Errorf("opentype: cff2: FDArray offset out of range")
}
fontDicts, err := parseIndex2(&reader{b: data, pos: off})
if err != nil {
return nil, nil, nil, nil, err
}
if len(fontDicts) == 0 {
return nil, nil, nil, nil, fmt.Errorf("opentype: cff2: empty FDArray")
}
subrs = make([][][]byte, len(fontDicts))
bias = make([]int, len(fontDicts))
vsindex = make([]int, len(fontDicts))
hints = make([]*cffPrivate, len(fontDicts))
for i, fd := range fontDicts {
d, err := parseDict2(fd)
if err != nil {
return nil, nil, nil, nil, err
}
ls, vs, h, err := parseCFF2Private(data, d)
if err != nil {
return nil, nil, nil, nil, err
}
subrs[i] = ls
bias[i] = subrBias(len(ls))
vsindex[i] = vs
hints[i] = h
}
return subrs, bias, vsindex, hints, nil
}
// parseCFF2Private resolves a Font DICT's Private DICT and, if present, its
// Local Subr INDEX and default vsindex, and parses its hint parameters (blue
// zones, standard/snap stem widths) for the grid-fitter (cffhint.go).
func parseCFF2Private(data []byte, fd map[int][]float64) ([][]byte, int, *cffPrivate, error) {
pv, ok := fd[18]
if !ok {
return nil, 0, nil, nil
}
if len(pv) < 2 {
return nil, 0, nil, fmt.Errorf("opentype: cff2: bad Private DICT entry")
}
psize, poff := int(pv[0]), int(pv[1])
if psize < 0 || poff < 0 || poff+psize > len(data) {
return nil, 0, nil, fmt.Errorf("opentype: cff2: Private DICT out of range")
}
priv, err := parseDict2(data[poff : poff+psize])
if err != nil {
return nil, 0, nil, err
}
hints := parsePrivateHints(priv)
vsindex := 0
if v, ok := dictInt(priv, 22, 0); ok {
vsindex = v
}
soff, ok := dictInt(priv, 19, 0)
if !ok {
return nil, vsindex, hints, nil
}
soff += poff
if soff < 0 || soff > len(data) {
return nil, 0, nil, fmt.Errorf("opentype: cff2: Local Subrs offset out of range")
}
ls, err := parseIndex2(&reader{b: data, pos: soff})
if err != nil {
return nil, 0, nil, err
}
return ls, vsindex, hints, nil
}
// parseCFF2FDSelect decodes the FDSelect (glyph -> Font DICT index) in format 0
// or 3. A table without an FDSelect returns nil (every glyph uses Font DICT 0).
func parseCFF2FDSelect(data []byte, top map[int][]float64, numGlyphs, numFDs int) ([]int, error) {
off, ok := dictInt(top, 1237, 0)
if !ok {
return nil, nil
}
if off < 0 || off >= len(data) {
return nil, fmt.Errorf("opentype: cff2: FDSelect offset out of range")
}
r := reader{b: data, pos: off}
format := r.u8()
sel := make([]int, numGlyphs)
switch format {
case 0:
for g := 0; g < numGlyphs; g++ {
fd := int(r.u8())
if fd >= numFDs {
return nil, fmt.Errorf("opentype: cff2: FDSelect fd %d out of range", fd)
}
sel[g] = fd
}
if r.err != nil {
return nil, fmt.Errorf("opentype: cff2 FDSelect: %w", r.err)
}
case 3:
nRanges := int(r.u16())
firsts := make([]int, nRanges)
fds := make([]int, nRanges)
for i := 0; i < nRanges; i++ {
firsts[i] = int(r.u16())
fds[i] = int(r.u8())
}
sentinel := int(r.u16())
if r.err != nil {
return nil, fmt.Errorf("opentype: cff2 FDSelect: %w", r.err)
}
for i := 0; i < nRanges; i++ {
if fds[i] >= numFDs {
return nil, fmt.Errorf("opentype: cff2: FDSelect fd %d out of range", fds[i])
}
next := sentinel
if i+1 < nRanges {
next = firsts[i+1]
}
for g := firsts[i]; g < next && g < numGlyphs; g++ {
sel[g] = fds[i]
}
}
default:
return nil, fmt.Errorf("opentype: cff2: unsupported FDSelect format %d", format)
}
return sel, nil
}
// cff2machine interprets one glyph's CFF2 charstring. It borrows a t2machine as
// its pen (pen position, contour under construction, finished contours and the
// Bézier-flattening path operators) and adds the CFF2-only variation state: the
// normalized coordinate, the active variation-store index and the blend vector
// (the per-region scalars, with a leading 1 for the default master).
type cff2machine struct {
c *cff2Table
tm *t2machine
localSubrs [][]byte
localBias int
coords []float64
vsindex int
bv []float64
}
// outline interprets glyph gid's CFF2 charstring at the normalized variation
// coordinate coords (F2Dot14 int16 per axis, as produced by NormalizeCoords);
// a nil or short coords vector leaves the remaining axes at their default. The
// result is the instanced outline as contours in font units, all points
// on-curve with cubics pre-flattened.
func (c *cff2Table) outline(gid int, coords []int16) ([]contour, error) {
cs, _, err := c.outlineHints(gid, coords)
return cs, err
}
// outlineHints is outline plus the hinting data (recorded stems, per-hintmask
// activation and the glyph's Font-DICT Private-DICT hint parameters) that
// cffhint.go's grid-fitter consumes. The outline is still instanced at coords.
func (c *cff2Table) outlineHints(gid int, coords []int16) ([]contour, *cffGlyphHints, error) {
if gid < 0 || gid >= len(c.charStrings) {
return nil, nil, fmt.Errorf("opentype: cff2 glyph %d out of range", gid)
}
fd := 0
if c.fdSelect != nil {
fd = c.fdSelect[gid]
}
axisCount := 0
if c.vstore != nil {
axisCount = c.vstore.axisCount
}
m := &cff2machine{
c: c,
tm: &t2machine{},
localSubrs: c.localSubrs[fd],
localBias: c.localBias[fd],
coords: make([]float64, axisCount),
vsindex: c.privVsindex[fd],
}
for a := 0; a < axisCount && a < len(coords); a++ {
m.coords[a] = float64(coords[a]) / 16384.0
}
if err := m.computeBV(); err != nil {
return nil, nil, err
}
if err := m.run(c.charStrings[gid], 0); err != nil {
return nil, nil, err
}
m.tm.finishContour()
gh := &cffGlyphHints{
stems: m.tm.stemHints,
hintMasks: m.tm.hintMasks,
cntrGroups: m.tm.cntrGroups,
flexRanges: m.tm.flexRanges,
priv: c.privHints[fd],
}
return m.tm.contours, gh, nil
}
// computeBV rebuilds the blend vector for the active vsindex: a leading 1 (the
// default master's scalar) followed by one scalar per region of the selected
// ItemVariationData. With no variation store the vector is just [1], so blend
// becomes a no-op that leaves its default operands unchanged.
func (m *cff2machine) computeBV() error {
m.bv = append(m.bv[:0], 1)
vs := m.c.vstore
if vs == nil {
return nil
}
if m.vsindex < 0 || m.vsindex >= len(vs.data) {
return fmt.Errorf("opentype: cff2: vsindex %d out of range", m.vsindex)
}
for _, ri := range vs.data[m.vsindex].regionIndices {
m.bv = append(m.bv, vs.regionScalar(ri, m.coords))
}
return nil
}
// push appends v to the operand stack, guarding the CFF2 stack-depth limit.
func (m *cff2machine) push(v float64) error {
if len(m.tm.stack) >= maxCFF2Stack {
return fmt.Errorf("opentype: cff2: charstring stack overflow")
}
m.tm.stack = append(m.tm.stack, v)
return nil
}
// run executes a charstring (or subroutine) body; depth guards recursion. It
// returns when the code is exhausted or a 'return' operator ends a subroutine.
func (m *cff2machine) run(code []byte, depth int) error {
if depth > maxT2Depth {
return fmt.Errorf("opentype: cff2: subroutine recursion too deep")
}
i := 0
for i < len(code) {
b0 := code[i]
i++
if b0 >= 32 || b0 == 28 {
v, n, err := m.tm.operand(code, i-1)
if err != nil {
return err
}
if err := m.push(v); err != nil {
return err
}
i += n - 1
continue
}
ni, ret, err := m.operator(b0, code, i, depth)
if err != nil {
return err
}
i = ni
if ret {
return nil
}
}
return nil
}
// operator executes one CFF2 operator byte b0. code/i give the stream position
// just past b0 (for hintmask bytes and the escape second byte); it returns the
// updated stream index, whether the current subroutine should stop ('return'),
// and any error. Unlike Type2 there is no leading-width handling.
func (m *cff2machine) operator(b0 byte, code []byte, i, depth int) (int, bool, error) {
tm := m.tm
switch b0 {
case 1, 3, 18, 23: // hstem, vstem, hstemhm, vstemhm
tm.recordStems(b0 == 1 || b0 == 18)
case 19, 20: // hintmask, cntrmask
tm.recordStems(false) // any operands here are implicit vstem hints
ni, herr := tm.readHintMask(code, i, b0 == 19)
if herr != nil {
return i, false, herr
}
i = ni
case 21: // rmoveto
if len(tm.stack) < 2 {
return i, false, fmt.Errorf("opentype: cff2: rmoveto needs 2 args")
}
n := len(tm.stack)
tm.moveTo(tm.stack[n-2], tm.stack[n-1])
tm.stack = tm.stack[:0]
case 22: // hmoveto
if len(tm.stack) < 1 {
return i, false, fmt.Errorf("opentype: cff2: hmoveto needs 1 arg")
}
tm.moveTo(tm.stack[len(tm.stack)-1], 0)
tm.stack = tm.stack[:0]
case 4: // vmoveto
if len(tm.stack) < 1 {
return i, false, fmt.Errorf("opentype: cff2: vmoveto needs 1 arg")
}
tm.moveTo(0, tm.stack[len(tm.stack)-1])
tm.stack = tm.stack[:0]
case 5: // rlineto
s := tm.stack
for j := 0; j+2 <= len(s); j += 2 {
tm.lineTo(s[j], s[j+1])
}
tm.stack = tm.stack[:0]
case 6: // hlineto
tm.altLine(true)
case 7: // vlineto
tm.altLine(false)
case 8: // rrcurveto
tm.rrcurveto(tm.stack)
tm.stack = tm.stack[:0]
case 24: // rcurveline
tm.rcurveline(tm.stack)
tm.stack = tm.stack[:0]
case 25: // rlinecurve
tm.rlinecurve(tm.stack)
tm.stack = tm.stack[:0]
case 26: // vvcurveto
tm.vvcurveto(tm.stack)
tm.stack = tm.stack[:0]
case 27: // hhcurveto
tm.hhcurveto(tm.stack)
tm.stack = tm.stack[:0]
case 30: // vhcurveto
tm.altCurve(tm.stack, false)
tm.stack = tm.stack[:0]
case 31: // hvcurveto
tm.altCurve(tm.stack, true)
tm.stack = tm.stack[:0]
case 10: // callsubr
if err := m.callSubr(m.localSubrs, m.localBias, depth); err != nil {
return i, false, err
}
case 29: // callgsubr
if err := m.callSubr(m.c.globalSubrs, m.c.gsubrBias, depth); err != nil {
return i, false, err
}
case 11: // return
return i, true, nil
case 15: // vsindex
if err := m.vsindexOp(); err != nil {
return i, false, err
}
case 16: // blend
if err := m.blendOp(); err != nil {
return i, false, err
}
case 12: // escape: two-byte operator (flex family)
if i >= len(code) {
return i, false, fmt.Errorf("opentype: cff2 charstring: truncated escape")
}
b1 := code[i]
i++
if err := tm.escape(b1); err != nil {
return i, false, err
}
default:
return i, false, fmt.Errorf("opentype: cff2: unknown operator %d", b0)
}
return i, false, nil
}
// callSubr executes the subroutine selected by the top-of-stack index (plus the
// bias) from subrs. A 'return' inside the subroutine ends only that subroutine.
func (m *cff2machine) callSubr(subrs [][]byte, bias, depth int) error {
st := m.tm.stack
if len(st) < 1 {
return fmt.Errorf("opentype: cff2: callsubr needs an index")
}
idx := int(st[len(st)-1]) + bias
m.tm.stack = st[:len(st)-1]
if idx < 0 || idx >= len(subrs) {
return fmt.Errorf("opentype: cff2: subr index %d out of range", idx)
}
return m.run(subrs[idx], depth+1)
}
// vsindexOp handles the vsindex operator: it selects the ItemVariationData that
// subsequent blends draw from and recomputes the blend vector.
func (m *cff2machine) vsindexOp() error {
st := m.tm.stack
if len(st) < 1 {
return fmt.Errorf("opentype: cff2: vsindex needs 1 arg")
}
m.vsindex = int(st[len(st)-1])
m.tm.stack = st[:0]
return m.computeBV()
}
// blendOp handles the blend operator: the top operand n gives the number of
// resulting values; below it are n default values followed by n groups of k
// region deltas (k = regions of the active vsindex). Each default value is
// replaced in place by default + Σ bv[region]·delta, and the deltas and n are
// popped, leaving the n blended values as operands for the following operator.
func (m *cff2machine) blendOp() error {
st := m.tm.stack
if len(st) < 1 {
return fmt.Errorf("opentype: cff2: blend needs a count")
}
numBlends := int(st[len(st)-1])
st = st[:len(st)-1]
if numBlends < 0 {
return fmt.Errorf("opentype: cff2: blend count %d invalid", numBlends)
}
k := len(m.bv) - 1
need := numBlends * (k + 1)
if need > len(st) {
return fmt.Errorf("opentype: cff2: blend needs %d operands, have %d", need, len(st))
}
base := len(st) - need
di := base + numBlends
for vi := 0; vi < numBlends; vi++ {
sum := st[base+vi]
for j := 1; j <= k; j++ {
sum += m.bv[j] * st[di]
di++
}
st[base+vi] = sum
}
m.tm.stack = st[:base+numBlends]
return nil
}
vendor/github.com/go-opentype/opentype/cffhint.go
Generated Vendored
+649
View File
@@ -0,0 +1,649 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import (
"math"
"sort"
)
// This file grid-fits CFF/CFF2 (Type2) outlines the way FreeType's CFF hinter
// does: it aligns stem edges and blue-zone reference points to the pixel grid
// at a target ppem, then interpolates the rest of the outline between them. It
// is the CFF counterpart of the TrueType instruction interpreter (hint.go);
// where TrueType grid-fitting is imperative (a bytecode VM moves points), CFF
// grid-fitting is declarative — the font only states its stems (recorded by the
// Type2 interpreter in cff.go/cff2.go) and its Private-DICT hint parameters
// (blue zones, standard and snap stem widths, parsed here), and the renderer
// decides how to snap them.
//
// Pipeline (per glyph, per axis, independent x and y):
// 1. Scale every stem edge and blue-zone edge to device space (pixels/em).
// 2. Snap each active stem's width toward StdHW/StdVW / StemSnap and round it,
// then position the stem so one edge lands on an integer pixel boundary,
// anchoring horizontal (hstem) edges that fall in a blue zone to the rounded
// zone position instead. This yields, per stem edge, an original device
// position and its hinted (grid-fitted) position.
// 3. Build a piecewise-linear control map from those (original -> hinted)
// edge pairs plus a blue-zone flat-edge anchor per zone, and remap every
// outline point: a point on a control edge snaps with it, a point between
// two edges interpolates proportionally, and a point outside the outermost
// edges shifts rigidly with the nearest one. This is the same interpolation
// TrueType uses for untouched points (IUP), applied here to the whole
// outline.
//
// What is implemented: Private-DICT hint-parameter parsing (BlueValues,
// OtherBlues, FamilyBlues, FamilyOtherBlues, BlueScale, BlueShift, BlueFuzz,
// StdHW, StdVW, StemSnapH, StemSnapV); top/bottom blue-zone construction with
// overshoot, family-blue substitution and the BlueScale overshoot-suppression
// rule; stem-width snapping and edge grid-fitting; blue-zone alignment of stem
// edges and reference points; and per-point hintmask hint selection.
//
// The finer FreeType sub-nuances are also applied here:
// - counter control: a cntrmask hint group's counters (the gaps between the
// group's same-orientation stems, e.g. the three verticals of 'm') are
// equalised after grid-fitting so the stems keep even spacing;
// - mid-subpath hintmask changes: each point is remapped through the control
// map built from the stems active at that point (not the subpath's first
// point), so a mask that switches mid-contour hints each portion correctly;
// - flex regions: the near-flat span a flex (flex/flex1/hflex/hflex1) draws is
// interpolated smoothly between its grid-fitted anchors instead of letting an
// interior point snap to a stem edge as though it were a straight stem;
// - stem darkening: an optional ppem-dependent emboldening of stem edges that
// improves contrast on the anti-aliased rasteriser at small sizes (off by
// default, toggled by Face.SetStemDarkening; the output is byte-identical to
// the undarkened result when it is off).
// cffStemHint is one resolved stem hint in font units: the two edge positions
// (min < max) and whether it is a horizontal stem (hstem, edges are Y
// positions, controlling horizontal features and blue alignment) or a vertical
// stem (vstem, edges are X positions).
type cffStemHint struct {
horizontal bool
min, max float64
}
// cffHintMask is one recorded hintmask activation: the outline-point index at
// which it takes effect (so per-subpath hint selection can find the mask in
// force for a given point) and, per declared stem, whether that stem is active.
type cffHintMask struct {
pointIndex int
active []bool
}
// flexRange is one flex region's outline-point span (inclusive global point
// indices of the points the two flex curves emitted). The point at start-1 is
// the flex's entry anchor and the point at end is its exit anchor; the interior
// points between them are interpolated smoothly by the grid-fitter.
type flexRange struct {
start, end int
}
// cffGlyphHints bundles a glyph's recorded stems, hintmask activations,
// counter-control (cntrmask) stem groups and flex-region point spans with the
// font's parsed Private-DICT hint parameters, as returned by the CFF/CFF2
// interpreters alongside the outline.
type cffGlyphHints struct {
stems []cffStemHint
hintMasks []cffHintMask
cntrGroups [][]int
flexRanges []flexRange
priv *cffPrivate
}
// maskSelAt returns the index into hintMasks of the hintmask in force at the
// given outline-point index — the last hintmask whose pointIndex does not exceed
// it — or -1 when none applies yet (every stem is then treated as active). It is
// the primitive the grid-fitter keys its per-point control maps on so a hintmask
// that changes mid-subpath selects the right stems at each point.
func (gh *cffGlyphHints) maskSelAt(pointIndex int) int {
sel := -1
for i, hm := range gh.hintMasks {
if hm.pointIndex <= pointIndex {
sel = i
}
}
return sel
}
// activeMaskAt returns the per-stem activation of the hintmask in force at the
// given outline-point index. A nil result means "all stems active" (no hintmask
// applies yet, or the glyph declared none), which the grid-fitter treats as
// every stem on.
func (gh *cffGlyphHints) activeMaskAt(pointIndex int) []bool {
sel := gh.maskSelAt(pointIndex)
if sel < 0 {
return nil
}
return gh.hintMasks[sel].active
}
// cffPrivate holds the Private-DICT hint parameters that drive grid-fitting.
// Blue arrays are stored already accumulated (the DICT delta encoding undone)
// and in font units; BlueScale/BlueShift/BlueFuzz carry their CFF defaults when
// the DICT omits them.
type cffPrivate struct {
blueValues []float64 // paired: [0:2] baseline (bottom) zone, rest top zones
otherBlues []float64 // paired: additional bottom zones
familyBlues []float64 // paired: family baseline+top zones (consistency)
familyOtherBlues []float64 // paired: family bottom zones
blueScale float64 // overshoot-suppression threshold (default 0.039625)
blueShift float64 // minimum overshoot before it is shown (default 7)
blueFuzz float64 // blue-zone capture extension (default 1)
stdHW float64 // standard horizontal-stem width (hstem)
stdVW float64 // standard vertical-stem width (vstem)
stemSnapH []float64 // horizontal-stem snap widths
stemSnapV []float64 // vertical-stem snap widths
defaultWidthX float64 // the advance of a glyph whose charstring gives none
nominalWidthX float64 // what a charstring's width is measured from
}
// CFF Private-DICT defaults for the overshoot-control parameters (CFF spec).
const (
defaultBlueScale = 0.039625
defaultBlueShift = 7
defaultBlueFuzz = 1
)
// snapTolerancePx is how close (in device pixels) a stem's scaled width must be
// to a standard/snap width for the width to be rendered as that standard width
// (unifying near-equal stems) before rounding to whole pixels.
const snapTolerancePx = 0.5
// parsePrivateHints extracts the hint parameters from an already-decoded
// Private DICT (CFF or CFF2 share the operator codes). Absent overshoot-control
// operators take their CFF defaults; the delta-encoded blue and stem-snap
// arrays are accumulated into absolute font-unit values.
func parsePrivateHints(d map[int][]float64) *cffPrivate {
p := &cffPrivate{
blueScale: defaultBlueScale,
blueShift: defaultBlueShift,
blueFuzz: defaultBlueFuzz,
}
p.blueValues = accumDelta(d[6])
p.otherBlues = accumDelta(d[7])
p.familyBlues = accumDelta(d[8])
p.familyOtherBlues = accumDelta(d[9])
if v, ok := dictFloat(d, 10); ok {
p.stdHW = v
}
if v, ok := dictFloat(d, 11); ok {
p.stdVW = v
}
if v, ok := dictFloat(d, 1209); ok {
p.blueScale = v
}
if v, ok := dictFloat(d, 1210); ok {
p.blueShift = v
}
if v, ok := dictFloat(d, 1211); ok {
p.blueFuzz = v
}
p.stemSnapH = accumDelta(d[1212])
p.stemSnapV = accumDelta(d[1213])
if v, ok := d[20]; ok && len(v) > 0 {
p.defaultWidthX = v[0]
}
if v, ok := d[21]; ok && len(v) > 0 {
p.nominalWidthX = v[0]
}
return p
}
// dictFloat returns the first operand of operator op in d as a float, or
// ok=false when the operator is absent or carries no operand.
func dictFloat(d map[int][]float64, op int) (float64, bool) {
v, ok := d[op]
if !ok || len(v) == 0 {
return 0, false
}
return v[0], true
}
// accumDelta undoes the CFF DICT "delta" encoding: the stored operands are
// successive differences, so the absolute values are their running sum. A nil
// or empty input yields nil.
func accumDelta(v []float64) []float64 {
if len(v) == 0 {
return nil
}
out := make([]float64, len(v))
sum := 0.0
for i, d := range v {
sum += d
out[i] = sum
}
return out
}
// blueZone is one alignment zone in font units: flat is the reference edge that
// snaps to the grid (the baseline, x-height, cap height, …); over is the
// overshoot extent (below flat for a bottom zone, above for a top zone) where
// round glyph features spill past the flat edge.
type blueZone struct {
flat float64
over float64
}
// makeZone builds a zone from a (a, b) blue pair. A top zone's flat edge is the
// lower value with overshoot above; a bottom zone's flat edge is the upper
// value with overshoot below.
func makeZone(a, b float64, isTop bool) blueZone {
lo, hi := math.Min(a, b), math.Max(a, b)
if isTop {
return blueZone{flat: lo, over: hi}
}
return blueZone{flat: hi, over: lo}
}
// applyFamily substitutes the corresponding FamilyBlues flat edge for a zone's
// flat edge when the two are within one device pixel, so members of a type
// family align identically at small sizes (the CFF FamilyBlues rule). fam is
// the family array paired the same way as the primary array; k is the pair
// index.
func applyFamily(z *blueZone, fam []float64, k int, isTop bool, scale float64) {
if k+1 >= len(fam) {
return
}
fz := makeZone(fam[k], fam[k+1], isTop)
if math.Abs((fz.flat-z.flat)*scale) < 1 {
z.flat = fz.flat
}
}
// buildBlueZones assembles the blue zones from the Private-DICT arrays: the
// first BlueValues pair is the baseline (a bottom zone), the remaining
// BlueValues pairs are top zones, and every OtherBlues pair is a bottom zone.
// Family blues substitute for consistency (applyFamily).
func buildBlueZones(p *cffPrivate, scale float64) []blueZone {
var zones []blueZone
bv := p.blueValues
for k := 0; k+1 < len(bv); k += 2 {
isTop := k >= 2
z := makeZone(bv[k], bv[k+1], isTop)
applyFamily(&z, p.familyBlues, k, isTop, scale)
zones = append(zones, z)
}
ob := p.otherBlues
for k := 0; k+1 < len(ob); k += 2 {
z := makeZone(ob[k], ob[k+1], false)
applyFamily(&z, p.familyOtherBlues, k, false, scale)
zones = append(zones, z)
}
return zones
}
// capture reports whether edgeDev (a device-space stem edge) falls in this zone
// and, if so, the device position it should be aligned to. Within BlueFuzz of
// the zone the edge is captured; it snaps to the rounded flat edge unless
// overshoot is enabled (not suppressed) and the edge lies at least BlueShift
// beyond the flat edge, in which case a rounded overshoot of at least one pixel
// is preserved.
func (z blueZone) capture(edgeDev, scale, blueShift, blueFuzz float64, suppress bool) (float64, bool) {
flatDev := z.flat * scale
overDev := z.over * scale
lo, hi := math.Min(flatDev, overDev), math.Max(flatDev, overDev)
fuzz := blueFuzz * scale
if edgeDev < lo-fuzz || edgeDev > hi+fuzz {
return 0, false
}
base := math.Round(flatDev)
if suppress {
return base, true
}
over := edgeDev - flatDev
if math.Abs(over) < blueShift*scale {
return base, true
}
o := math.Round(math.Abs(over))
if o < 1 {
o = 1
}
if over < 0 {
o = -o
}
return base + o, true
}
// hintedStem is one stem after grid-fitting: the original device-space edge
// positions (oLo < oHi) and their hinted (snapped) positions.
type hintedStem struct {
oLo, oHi float64
hLo, hHi float64
}
// hintStem grid-fits one stem: its width is snapped toward the standard/snap
// widths and rounded; for a horizontal stem an edge that lands in a blue zone
// is anchored to the rounded zone position (and the opposite edge placed one
// snapped width away), otherwise the lower edge is rounded to the pixel grid.
func hintStem(s cffStemHint, scale float64, p *cffPrivate, blues []blueZone, suppress bool) hintedStem {
lo := math.Min(s.min, s.max) * scale
hi := math.Max(s.min, s.max) * scale
std, snaps := p.stdVW, p.stemSnapV
if s.horizontal {
std, snaps = p.stdHW, p.stemSnapH
}
w := snapWidth(hi-lo, std*scale, snaps, scale)
if s.horizontal {
if t, ok := captureEdge(hi, blues, p, scale, suppress); ok {
return hintedStem{oLo: lo, oHi: hi, hLo: t - w, hHi: t}
}
if t, ok := captureEdge(lo, blues, p, scale, suppress); ok {
return hintedStem{oLo: lo, oHi: hi, hLo: t, hHi: t + w}
}
}
hLo := math.Round(lo)
return hintedStem{oLo: lo, oHi: hi, hLo: hLo, hHi: hLo + w}
}
// captureEdge returns the aligned device position for a horizontal-stem edge
// that falls in one of the blue zones, testing each zone in turn.
func captureEdge(edgeDev float64, blues []blueZone, p *cffPrivate, scale float64, suppress bool) (float64, bool) {
for _, z := range blues {
if t, ok := z.capture(edgeDev, scale, p.blueShift, p.blueFuzz, suppress); ok {
return t, true
}
}
return 0, false
}
// snapWidth rounds a scaled stem width to whole pixels, first snapping it to the
// nearest standard or snap width (std and snaps are in font units, scaled here)
// when within snapTolerancePx so near-equal stems render at one consistent
// width. The result is at least one pixel.
func snapWidth(w, stdDev float64, snaps []float64, scale float64) float64 {
best, bestD := w, math.Inf(1)
consider := func(v float64) {
if v <= 0 {
return
}
if d := math.Abs(w - v); d < bestD {
best, bestD = v, d
}
}
consider(stdDev)
for _, s := range snaps {
consider(s * scale)
}
if bestD > snapTolerancePx {
best = w
}
r := math.Round(best)
if r < 1 {
r = 1
}
return r
}
// controlMap is a piecewise-linear 1-D remap built from (original -> hinted)
// device positions, sorted and deduplicated by original position.
type controlMap struct {
orig []float64
hint []float64
}
// add records one control point (an original device position and where it is
// hinted to).
func (cm *controlMap) add(o, h float64) {
cm.orig = append(cm.orig, o)
cm.hint = append(cm.hint, h)
}
// finalize sorts the control points by original position and merges points that
// share an original position (averaging their hinted targets), leaving a
// strictly increasing orig sequence for interpolation.
func (cm *controlMap) finalize() {
n := len(cm.orig)
if n == 0 {
return
}
idx := make([]int, n)
for i := range idx {
idx[i] = i
}
sort.SliceStable(idx, func(a, b int) bool { return cm.orig[idx[a]] < cm.orig[idx[b]] })
var os, hs []float64
for _, i := range idx {
o, h := cm.orig[i], cm.hint[i]
if len(os) > 0 && o == os[len(os)-1] {
// Same original edge from two hints: average the targets and keep one.
hs[len(hs)-1] = (hs[len(hs)-1] + h) / 2
continue
}
os = append(os, o)
hs = append(hs, h)
}
cm.orig, cm.hint = os, hs
}
// remap maps a device coordinate through the control map: below the first
// control point or above the last it shifts by that endpoint's delta; between
// two it interpolates linearly. An empty map is the identity.
func (cm *controlMap) remap(v float64) float64 {
n := len(cm.orig)
switch {
case n == 0:
return v
case v <= cm.orig[0]:
return v + (cm.hint[0] - cm.orig[0])
case v >= cm.orig[n-1]:
return v + (cm.hint[n-1] - cm.orig[n-1])
}
i := sort.Search(n, func(k int) bool { return cm.orig[k] >= v })
a, b := cm.orig[i-1], cm.orig[i]
ha, hb := cm.hint[i-1], cm.hint[i]
t := (v - a) / (b - a)
return ha + t*(hb-ha)
}
// hintPoint holds one outline point through grid-fitting: its original scaled
// (device) position (ox, oy), its hinted position (hx, hy) and its on-curve flag.
type hintPoint struct {
ox, oy, hx, hy float64
on bool
}
// cffGridFit grid-fits contours (in font units) using the recorded stems and
// Private-DICT hint parameters, returning the hinted contours in font units.
// scale is font-units-to-pixels and ppem is pixels/em (the BlueScale rule);
// darken enables stem darkening (weight compensation). It is a no-op returning
// equivalent geometry when there are no stems and no blue zones.
func cffGridFit(contours []contour, gh *cffGlyphHints, scale, ppem float64, darken bool) []contour {
priv := gh.priv
if priv == nil {
priv = &cffPrivate{blueScale: defaultBlueScale, blueShift: defaultBlueShift, blueFuzz: defaultBlueFuzz}
}
suppress := ppem*priv.blueScale < 1
blues := buildBlueZones(priv, scale)
// Grid-fit every stem once (its hinted position is independent of subpath),
// then equalise counter-group counters and optionally darken the stems.
hs := make([]hintedStem, len(gh.stems))
for i, s := range gh.stems {
hs[i] = hintStem(s, scale, priv, blues, suppress)
}
applyCounterControl(hs, gh.stems, gh.cntrGroups)
if darken {
darkenStems(hs, stemDarkenPixels(ppem))
}
// Remap every point through the control map built from the stems active at
// that point (per-point so a mid-subpath hintmask change hints correctly). The
// maps for a given active-hintmask selection are built once and cached.
cache := map[int]axisMaps{}
getMaps := func(sel int) axisMaps {
if m, ok := cache[sel]; ok {
return m
}
var active []bool
if sel >= 0 {
active = gh.hintMasks[sel].active
}
x, y := buildMaps(hs, gh.stems, active, blues, scale)
m := axisMaps{x: x, y: y}
cache[sel] = m
return m
}
var total int
for _, c := range contours {
total += len(c)
}
pts := make([]hintPoint, 0, total)
for _, c := range contours {
for _, p := range c {
ox, oy := p.x*scale, p.y*scale
m := getMaps(gh.maskSelAt(len(pts)))
pts = append(pts, hintPoint{ox: ox, oy: oy, hx: m.x.remap(ox), hy: m.y.remap(oy), on: p.on})
}
}
// Keep flex spans smooth: re-derive each flex region's interior points by
// interpolating between its grid-fitted entry (start-1) and exit (end) anchors.
for _, fr := range gh.flexRanges {
if fr.start-1 < 0 {
continue
}
interpolateFlex(pts, fr.start-1, fr.end)
}
out := make([]contour, len(contours))
idx := 0
for ci, c := range contours {
nc := make(contour, len(c))
for j := range c {
p := pts[idx]
nc[j] = outlinePoint{x: p.hx / scale, y: p.hy / scale, on: p.on}
idx++
}
out[ci] = nc
}
return out
}
// axisMaps pairs the x- and y-axis control maps for one active-hintmask
// selection.
type axisMaps struct {
x, y controlMap
}
// applyCounterControl equalises the counters of every recorded cntrmask group:
// within each group the same-orientation stems (independently for horizontal and
// vertical) are chained at an even, whole-pixel gap so features like the three
// verticals of 'm' keep uniform spacing after grid-fitting.
func applyCounterControl(hs []hintedStem, stems []cffStemHint, groups [][]int) {
for _, g := range groups {
equalizeCounters(hs, stems, g, true)
equalizeCounters(hs, stems, g, false)
}
}
// equalizeCounters repositions the horizontal (or vertical) stems of one counter
// group so the gaps between them are all the group's rounded average gap, keeping
// each stem's grid-fitted width and anchoring on the first (lowest) stem.
func equalizeCounters(hs []hintedStem, stems []cffStemHint, group []int, horizontal bool) {
var idx []int
for _, i := range group {
if stems[i].horizontal == horizontal {
idx = append(idx, i)
}
}
if len(idx) < 2 {
return
}
sort.Slice(idx, func(a, b int) bool { return hs[idx[a]].oLo < hs[idx[b]].oLo })
totalGap := 0.0
for k := 1; k < len(idx); k++ {
totalGap += hs[idx[k]].oLo - hs[idx[k-1]].oHi
}
gap := math.Round(totalGap / float64(len(idx)-1))
if gap < 1 {
gap = 1
}
for k := 1; k < len(idx); k++ {
w := hs[idx[k]].hHi - hs[idx[k]].hLo
lo := hs[idx[k-1]].hHi + gap
hs[idx[k]].hLo = lo
hs[idx[k]].hHi = lo + w
}
}
// darkenStems embiggens every stem by d device pixels per edge (its lower edge
// moves down and its upper edge up), the weight compensation that improves stem
// contrast on the anti-aliased rasteriser at small sizes.
func darkenStems(hs []hintedStem, d float64) {
for i := range hs {
hs[i].hLo -= d
hs[i].hHi += d
}
}
// stemDarkenPixels returns the per-edge stem emboldening in device pixels at the
// given ppem: strongest at small sizes and fading linearly to none at and above
// darkenMaxPPEM, mirroring FreeType's default stem darkening (a contrast boost
// for small text that is invisible once glyphs are large enough to hint cleanly).
func stemDarkenPixels(ppem float64) float64 {
if ppem <= 0 || ppem >= darkenMaxPPEM {
return 0
}
return darkenMaxPixels * (1 - ppem/darkenMaxPPEM)
}
// darkenMaxPPEM is the ppem at (and above) which stem darkening stops, and
// darkenMaxPixels is its per-edge strength (in device pixels) at ppem→0.
const (
darkenMaxPPEM = 48
darkenMaxPixels = 0.15
)
// interpolateFlex re-derives the hinted positions of a flex region's interior
// points (before+1 .. end-1) by IUP-style interpolation between the grid-fitted
// anchors at indices before and end, so the shallow flex curve keeps its smooth
// shape rather than each interior point snapping to a stem edge.
func interpolateFlex(pts []hintPoint, before, end int) {
for k := before + 1; k < end; k++ {
pts[k].hx = interp1D(pts[before].ox, pts[end].ox, pts[before].hx, pts[end].hx, pts[k].ox)
pts[k].hy = interp1D(pts[before].oy, pts[end].oy, pts[before].hy, pts[end].hy, pts[k].oy)
}
}
// interp1D maps o from the original interval [oA, oB] to the hinted interval,
// carrying the endpoint deltas: when the endpoints share an original coordinate
// (a degenerate interval) the point takes the anchor A's delta.
func interp1D(oA, oB, hA, hB, o float64) float64 {
dA := hA - oA
if oB == oA {
return o + dA
}
t := (o - oA) / (oB - oA)
return o + dA + t*((hB-oB)-dA)
}
// buildMaps assembles the x- and y-axis control maps for one subpath: each
// active vertical stem contributes its two edges to the x map and each active
// horizontal stem to the y map, and every blue zone contributes a flat-edge
// anchor to the y map (blue zones are always active). A nil active mask means
// every stem is active.
func buildMaps(hs []hintedStem, stems []cffStemHint, active []bool, blues []blueZone, scale float64) (x, y controlMap) {
for i, st := range hs {
if active != nil && !active[i] {
continue
}
if stems[i].horizontal {
y.add(st.oLo, st.hLo)
y.add(st.oHi, st.hHi)
} else {
x.add(st.oLo, st.hLo)
x.add(st.oHi, st.hHi)
}
}
for _, z := range blues {
f := z.flat * scale
y.add(f, math.Round(f))
}
x.finalize()
y.finalize()
return x, y
}
vendor/github.com/go-opentype/opentype/cffstrings.go
Generated Vendored
+88
View File
@@ -0,0 +1,88 @@
// Copyright (c) 2026, the go-opentype authors
// All rights reserved.
//
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
// cffStandardStrings are the 391 names every CFF font shares. A string id
// below this many is one of these; anything higher is an index into the
// font's own String INDEX. The list is fixed by the Compact Font Format
// specification and is the only way to learn what a glyph in such a font is
// called — which is what a PDF needs, because it addresses a simple font by
// name rather than by number.
//
// The first ninety-five of them are the printable ASCII characters in order,
// which is what makes the table checkable against the Standard Encoding
// rather than merely transcribed: see TestTheStandardStringsAgreeWithTheStandardEncoding.
var cffStandardStrings = [...]string{
".notdef", "space", "exclam", "quotedbl", "numbersign", "dollar", "percent",
"ampersand", "quoteright", "parenleft", "parenright", "asterisk", "plus",
"comma", "hyphen", "period", "slash", "zero", "one", "two", "three", "four",
"five", "six", "seven", "eight", "nine", "colon", "semicolon", "less",
"equal", "greater", "question", "at", "A", "B", "C", "D", "E", "F", "G",
"H", "I", "J", "K", "L", "M", "N", "O", "P", "Q", "R", "S", "T", "U", "V",
"W", "X", "Y", "Z", "bracketleft", "backslash", "bracketright",
"asciicircum", "underscore", "quoteleft", "a", "b", "c", "d", "e", "f", "g",
"h", "i", "j", "k", "l", "m", "n", "o", "p", "q", "r", "s", "t", "u", "v",
"w", "x", "y", "z", "braceleft", "bar", "braceright", "asciitilde",
"exclamdown", "cent", "sterling", "fraction", "yen", "florin", "section",
"currency", "quotesingle", "quotedblleft", "guillemotleft", "guilsinglleft",
"guilsinglright", "fi", "fl", "endash", "dagger", "daggerdbl",
"periodcentered", "paragraph", "bullet", "quotesinglbase", "quotedblbase",
"quotedblright", "guillemotright", "ellipsis", "perthousand", "questiondown",
"grave", "acute", "circumflex", "tilde", "macron", "breve", "dotaccent",
"dieresis", "ring", "cedilla", "hungarumlaut", "ogonek", "caron", "emdash",
"AE", "ordfeminine", "Lslash", "Oslash", "OE", "ordmasculine", "ae",
"dotlessi", "lslash", "oslash", "oe", "germandbls", "onesuperior",
"logicalnot", "mu", "trademark", "Eth", "onehalf", "plusminus", "Thorn",
"onequarter", "divide", "brokenbar", "degree", "thorn", "threequarters",
"twosuperior", "registered", "minus", "eth", "multiply", "threesuperior",
"copyright", "Aacute", "Acircumflex", "Adieresis", "Agrave", "Aring",
"Atilde", "Ccedilla", "Eacute", "Ecircumflex", "Edieresis", "Egrave",
"Iacute", "Icircumflex", "Idieresis", "Igrave", "Ntilde", "Oacute",
"Ocircumflex", "Odieresis", "Ograve", "Otilde", "Scaron", "Uacute",
"Ucircumflex", "Udieresis", "Ugrave", "Yacute", "Ydieresis", "Zcaron",
"aacute", "acircumflex", "adieresis", "agrave", "aring", "atilde",
"ccedilla", "eacute", "ecircumflex", "edieresis", "egrave", "iacute",
"icircumflex", "idieresis", "igrave", "ntilde", "oacute", "ocircumflex",
"odieresis", "ograve", "otilde", "scaron", "uacute", "ucircumflex",
"udieresis", "ugrave", "yacute", "ydieresis", "zcaron", "exclamsmall",
"Hungarumlautsmall", "dollaroldstyle", "dollarsuperior", "ampersandsmall",
"Acutesmall", "parenleftsuperior", "parenrightsuperior", "twodotenleader",
"onedotenleader", "zerooldstyle", "oneoldstyle", "twooldstyle",
"threeoldstyle", "fouroldstyle", "fiveoldstyle", "sixoldstyle",
"sevenoldstyle", "eightoldstyle", "nineoldstyle", "commasuperior",
"threequartersemdash", "periodsuperior", "questionsmall", "asuperior",
"bsuperior", "centsuperior", "dsuperior", "esuperior", "isuperior",
"lsuperior", "msuperior", "nsuperior", "osuperior", "rsuperior", "ssuperior",
"tsuperior", "ff", "ffi", "ffl", "parenleftinferior", "parenrightinferior",
"Circumflexsmall", "hyphensuperior", "Gravesmall", "Asmall", "Bsmall",
"Csmall", "Dsmall", "Esmall", "Fsmall", "Gsmall", "Hsmall", "Ismall",
"Jsmall", "Ksmall", "Lsmall", "Msmall", "Nsmall", "Osmall", "Psmall",
"Qsmall", "Rsmall", "Ssmall", "Tsmall", "Usmall", "Vsmall", "Wsmall",
"Xsmall", "Ysmall", "Zsmall", "colonmonetary", "onefitted", "rupiah",
"Tildesmall", "exclamdownsmall", "centoldstyle", "Lslashsmall",
"Scaronsmall", "Zcaronsmall", "Dieresissmall", "Brevesmall", "Caronsmall",
"Dotaccentsmall", "Macronsmall", "figuredash", "hypheninferior",
"Ogoneksmall", "Ringsmall", "Cedillasmall", "questiondownsmall",
"oneeighth", "threeeighths", "fiveeighths", "seveneighths", "onethird",
"twothirds", "zerosuperior", "foursuperior", "fivesuperior", "sixsuperior",
"sevensuperior", "eightsuperior", "ninesuperior", "zeroinferior",
"oneinferior", "twoinferior", "threeinferior", "fourinferior",
"fiveinferior", "sixinferior", "seveninferior", "eightinferior",
"nineinferior", "centinferior", "dollarinferior", "periodinferior",
"commainferior", "Agravesmall", "Aacutesmall", "Acircumflexsmall",
"Atildesmall", "Adieresissmall", "Aringsmall", "AEsmall", "Ccedillasmall",
"Egravesmall", "Eacutesmall", "Ecircumflexsmall", "Edieresissmall",
"Igravesmall", "Iacutesmall", "Icircumflexsmall", "Idieresissmall",
"Ethsmall", "Ntildesmall", "Ogravesmall", "Oacutesmall", "Ocircumflexsmall",
"Otildesmall", "Odieresissmall", "OEsmall", "Oslashsmall", "Ugravesmall",
"Uacutesmall", "Ucircumflexsmall", "Udieresissmall", "Yacutesmall",
"Thornsmall", "Ydieresissmall", "001.000", "001.001", "001.002", "001.003",
"Black", "Bold", "Book", "Light", "Medium", "Regular", "Roman", "Semibold",
}
// nStdStrings is how many names every CFF font shares.
const nStdStrings = len(cffStandardStrings)
vendor/github.com/go-opentype/opentype/charmap.go
Generated Vendored
+75
View File
@@ -0,0 +1,75 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
// A font's cmap table carries several subtables, each written for one
// platform and encoding, and Font.GlyphIndex answers through whichever of
// them can represent the most codepoints. That is the right answer for a font
// addressed by character, and the wrong one for a font addressed some other
// way: a PDF says how its fonts are addressed, and a symbolic TrueType font
// embedded in one is addressed through its own Microsoft Symbol or Macintosh
// Roman subtable by raw byte, not through its Unicode subtable by codepoint.
//
// Only the document can make that choice, so the subtables are listed here
// rather than chosen here.
// NumCharacterMaps returns how many cmap subtables the font carries in a form
// this package decodes. It is zero for a font with no character map at all.
//
// Format 14 (Unicode Variation Sequences) is not one of them: it maps a pair
// rather than a code, and is reached through Font.GlyphIndexVariation.
func (f *Font) NumCharacterMaps() int { return len(f.cmaps) }
// CharacterMap reports which platform and encoding the i'th subtable was
// written for, and which format it is written in. ok is false when i is not
// the index of a subtable.
//
// The pairs worth recognising are (3,1) and (3,10), Microsoft Unicode, and
// platform 0, Unicode; (3,0), Microsoft Symbol, whose codes are the font's
// own and conventionally live at 0xF000 + code; and (1,0), Macintosh Roman.
func (f *Font) CharacterMap(i int) (platform, encoding, format uint16, ok bool) {
if i < 0 || i >= len(f.cmaps) {
return 0, 0, 0, false
}
c := f.cmaps[i]
return c.platform, c.encoding, c.format, true
}
// GlyphIndexInMap maps a code to a glyph through one named subtable rather
// than through the one the font would choose. ok is false when i is not the
// index of a subtable, or when that subtable does not map the code (including
// a mapping to the .notdef glyph).
//
// The code is whatever the subtable is indexed by: a Unicode codepoint for a
// Unicode subtable, and a raw byte of the font's own encoding for a Microsoft
// Symbol or Macintosh Roman one.
func (f *Font) GlyphIndexInMap(i int, code rune) (GlyphIndex, bool) {
if i < 0 || i >= len(f.cmaps) {
return 0, false
}
return f.cmaps[i].lookup.lookup(code)
}
// RuneOfGlyphInMap inverts one subtable: which code reaches this glyph. ok is
// false when i is not the index of a subtable, or when no code in it reaches
// that glyph.
//
// A subtable maps many codes onto one glyph — a space and a no-break space
// commonly share one — and the lowest of them is the one reported. Inverting a
// Unicode subtable is how the character a glyph stands for is recovered when
// the document says nothing about it and the program names nothing: the font
// itself said which codepoint that glyph is for.
//
// The inverse is built once per subtable, on the first call, and kept.
func (f *Font) RuneOfGlyphInMap(i int, gid GlyphIndex) (rune, bool) {
if i < 0 || i >= len(f.cmaps) {
return 0, false
}
if f.cmaps[i].inverse == nil {
f.cmaps[i].inverse = f.cmaps[i].lookup.reverse(maxReverseCodes)
}
r, ok := f.cmaps[i].inverse[gid]
return r, ok
}
vendor/github.com/go-opentype/opentype/cmap.go
Generated Vendored
+933
View File
@@ -0,0 +1,933 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import "fmt"
// cmapLookup maps a rune to a glyph index. ok is false when the rune is not
// mapped by the selected subtable (including a mapping to the .notdef glyph).
//
// reverse answers the other question: which code reaches a given glyph. A
// subtable is a many-to-one map — a space and a no-break space commonly share
// one glyph — so the inverse keeps the lowest code that reaches each glyph,
// and is built once for the whole subtable rather than searched per glyph.
//
// budget bounds how many codes the walk may visit. Most formats describe no
// more codes than their own bytes can hold and ignore it; formats 4, 8 and 12
// do not — one group of a format-12 subtable can claim the whole Unicode
// space, and a damaged one can claim more — so those three spend it.
type cmapLookup interface {
lookup(r rune) (GlyphIndex, bool)
reverse(budget int) map[GlyphIndex]rune
}
// maxReverseCodes is the budget Font.RuneOfGlyphInMap gives a walk. A subtable
// that describes more than a million codes is describing far more than any
// inverse of it will be asked about, and the bound keeps a malformed font from
// costing more than a well-formed one.
const maxReverseCodes = 1 << 20
// reverseRange records the codes [first, first+count) as reaching the glyphs
// starting at gid, one glyph per code, into inv. It keeps the lowest code for
// each glyph and reports how many codes it consumed of the budget left.
func reverseRange(inv map[GlyphIndex]rune, first, count, gid uint32, budget int) int {
if uint32(budget) < count {
count = uint32(budget)
}
for i := uint32(0); i < count; i++ {
g := GlyphIndex(gid + i)
if g == 0 {
continue
}
if _, seen := inv[g]; !seen {
inv[g] = rune(first + i)
}
}
return int(count)
}
// cmapSubtable is one decoded cmap subtable together with the platform and
// encoding it was written for. Which of them to address a font through is the
// caller's decision and cannot be made here: a font embedded in a PDF is
// addressed the way the document says, and only the document knows.
type cmapSubtable struct {
platform uint16
encoding uint16
format uint16
lookup cmapLookup
inverse map[GlyphIndex]rune // built on first use by Font.RuneOfGlyphInMap
}
// parseCmap selects and decodes a Unicode cmap subtable. It understands
// formats 0, 2, 4, 6, 8, 10, 12 and 13 for the base rune->glyph mapping,
// preferring whichever can represent the most codepoints:
// 13 = 12 > 10 > 8 > 4 > 6 > 2 > 0.
// Format 14 (Unicode Variation Sequences) is a separate, always-parsed side
// table exposed through Font.GlyphIndexVariation; it never competes for f.cmap
// and its presence does not change ordinary GlyphIndex behaviour.
func (f *Font) parseCmap(b []byte) error {
r := reader{b: b}
r.skip(2) // version
numTables := int(r.u16())
if r.err != nil {
return fmt.Errorf("opentype: cmap header: %w", r.err)
}
var best cmapLookup
bestScore := 0
for i := 0; i < numTables; i++ {
platform := r.u16()
encoding := r.u16()
offset := int(r.u32())
if r.err != nil {
return fmt.Errorf("opentype: cmap record: %w", r.err)
}
if offset+2 > len(b) {
return fmt.Errorf("opentype: cmap subtable offset: %w", errTruncated)
}
sub := b[offset:]
format := be16(sub)
var (
lk cmapLookup
score int
err error
)
switch format {
case 0:
lk, err = parseCmap0(sub)
score = 1
case 2:
lk, err = parseCmap2(sub)
score = 2
case 6:
lk, err = parseCmap6(sub)
score = 3
case 4:
lk, err = parseCmap4(sub)
score = 4
case 8:
lk, err = parseCmap8(sub)
score = 5
case 10:
lk, err = parseCmap10(sub)
score = 6
case 12:
lk, err = parseCmap12(sub)
score = 7
case 13:
lk, err = parseCmap13(sub)
score = 7
case 14:
vs, verr := parseCmap14(sub)
if verr != nil {
return verr
}
f.cmapVS = vs
continue
default:
continue
}
if err != nil {
return err
}
f.cmaps = append(f.cmaps, cmapSubtable{
platform: platform,
encoding: encoding,
format: format,
lookup: lk,
})
if score > bestScore {
best = lk
bestScore = score
}
}
if best == nil {
return fmt.Errorf("opentype: no supported cmap subtable")
}
f.cmap = best
return nil
}
// cmap4 is a decoded format-4 (segmented, BMP) cmap subtable.
type cmap4 struct {
segCount int
endCode []uint16
startCode []uint16
idDelta []int16
idRangeOffset []uint16
glyphIDArray []uint16
}
// parseCmap4 decodes a format-4 subtable from sub (which starts at the format
// field).
func parseCmap4(sub []byte) (cmapLookup, error) {
r := reader{b: sub}
r.skip(6) // format, length, language
segX2 := int(r.u16())
segCount := segX2 / 2
r.skip(6) // searchRange, entrySelector, rangeShift
c := &cmap4{
segCount: segCount,
endCode: make([]uint16, segCount),
startCode: make([]uint16, segCount),
idDelta: make([]int16, segCount),
idRangeOffset: make([]uint16, segCount),
}
for i := 0; i < segCount; i++ {
c.endCode[i] = r.u16()
}
r.skip(2) // reservedPad
for i := 0; i < segCount; i++ {
c.startCode[i] = r.u16()
}
for i := 0; i < segCount; i++ {
c.idDelta[i] = r.i16()
}
for i := 0; i < segCount; i++ {
c.idRangeOffset[i] = r.u16()
}
if r.err != nil {
return nil, fmt.Errorf("opentype: cmap format 4: %w", r.err)
}
// The remaining bytes are the glyphIdArray (uint16 each).
rest := (len(sub) - r.pos) / 2
c.glyphIDArray = make([]uint16, rest)
for i := 0; i < rest; i++ {
c.glyphIDArray[i] = r.u16()
}
return c, nil
}
// lookup implements the format-4 segment search with idDelta/idRangeOffset.
func (c *cmap4) lookup(r rune) (GlyphIndex, bool) {
if uint32(r) > 0xFFFF {
return 0, false
}
cc := uint16(r)
for i := 0; i < c.segCount; i++ {
if c.endCode[i] < cc {
continue
}
if c.startCode[i] > cc {
return 0, false
}
var g uint16
if c.idRangeOffset[i] == 0 {
g = cc + uint16(c.idDelta[i])
} else {
idx := int(c.idRangeOffset[i]/2) + int(cc-c.startCode[i]) - (c.segCount - i)
if idx < 0 || idx >= len(c.glyphIDArray) {
return 0, false
}
g = c.glyphIDArray[idx]
if g != 0 {
g += uint16(c.idDelta[i])
}
}
if g == 0 {
return 0, false
}
return GlyphIndex(g), true
}
return 0, false
}
// reverse walks the segments in code order, resolving each code the same way
// lookup does, and keeps the lowest code that reaches each glyph.
func (c *cmap4) reverse(budget int) map[GlyphIndex]rune {
inv := make(map[GlyphIndex]rune)
for i := 0; i < c.segCount; i++ {
if c.startCode[i] > c.endCode[i] {
continue
}
for cc := uint32(c.startCode[i]); cc <= uint32(c.endCode[i]); cc++ {
if budget == 0 {
return inv
}
budget--
g, ok := c.lookup(rune(cc))
if !ok {
continue
}
if _, seen := inv[g]; !seen {
inv[g] = rune(cc)
}
}
}
return inv
}
// cmap12group is one sequential-map group of a format-12 subtable.
type cmap12group struct {
start uint32
end uint32
startGID uint32
}
// cmap12 is a decoded format-12 (segmented coverage, full Unicode) subtable.
type cmap12 struct {
groups []cmap12group
}
// parseCmap12 decodes a format-12 subtable from sub (starting at the format
// field).
func parseCmap12(sub []byte) (cmapLookup, error) {
r := reader{b: sub}
r.skip(12) // format, reserved, length, language
numGroups := int(r.u32())
if r.err != nil {
return nil, fmt.Errorf("opentype: cmap format 12 header: %w", r.err)
}
c := &cmap12{groups: make([]cmap12group, numGroups)}
for i := 0; i < numGroups; i++ {
c.groups[i].start = r.u32()
c.groups[i].end = r.u32()
c.groups[i].startGID = r.u32()
}
if r.err != nil {
return nil, fmt.Errorf("opentype: cmap format 12 groups: %w", r.err)
}
return c, nil
}
// lookup implements the format-12 binary search over sorted groups.
func (c *cmap12) lookup(r rune) (GlyphIndex, bool) {
cc := uint32(r)
lo, hi := 0, len(c.groups)
for lo < hi {
mid := (lo + hi) / 2
g := c.groups[mid]
if cc < g.start {
hi = mid
} else if cc > g.end {
lo = mid + 1
} else {
return GlyphIndex(g.startGID + (cc - g.start)), true
}
}
return 0, false
}
// reverse expands each group: a group advances one glyph per code point, so
// the codes it covers reach a contiguous run of glyphs.
func (c *cmap12) reverse(budget int) map[GlyphIndex]rune {
inv := make(map[GlyphIndex]rune)
for _, g := range c.groups {
if g.end < g.start {
continue
}
budget -= reverseRange(inv, g.start, g.end-g.start+1, g.startGID, budget)
if budget == 0 {
break
}
}
return inv
}
// cmap13group is one constant-map group of a format-13 subtable: every code
// point in [start, end] maps to the single glyph glyphID.
type cmap13group struct {
start uint32
end uint32
glyphID uint32
}
// cmap13 is a decoded format-13 (many-to-one range mappings) subtable. Its
// on-disk layout is identical to format 12, but the third group field is read
// with different semantics: format 12's startGlyphID advances one glyph per
// code point across a group, whereas format 13's glyphID is a constant so the
// whole range collapses onto a single glyph. This is what LastResort and other
// fallback fonts use to point every code point in a block at one "missing
// glyph" placeholder.
type cmap13 struct {
groups []cmap13group
}
// parseCmap13 decodes a format-13 subtable from sub (starting at the format
// field).
func parseCmap13(sub []byte) (cmapLookup, error) {
r := reader{b: sub}
r.skip(12) // format, reserved, length, language
numGroups := int(r.u32())
if r.err != nil {
return nil, fmt.Errorf("opentype: cmap format 13 header: %w", r.err)
}
c := &cmap13{groups: make([]cmap13group, numGroups)}
for i := 0; i < numGroups; i++ {
c.groups[i].start = r.u32()
c.groups[i].end = r.u32()
c.groups[i].glyphID = r.u32()
}
if r.err != nil {
return nil, fmt.Errorf("opentype: cmap format 13 groups: %w", r.err)
}
return c, nil
}
// lookup implements the format-13 binary search over sorted groups, returning
// the group's constant glyph for any code point it covers.
func (c *cmap13) lookup(r rune) (GlyphIndex, bool) {
cc := uint32(r)
lo, hi := 0, len(c.groups)
for lo < hi {
mid := (lo + hi) / 2
g := c.groups[mid]
if cc < g.start {
hi = mid
} else if cc > g.end {
lo = mid + 1
} else {
return GlyphIndex(g.glyphID), true
}
}
return 0, false
}
// reverse keeps one code per group: every code point a format-13 group covers
// reaches the same glyph, so only the lowest of them is worth recording, and a
// group covering a whole Unicode block costs one entry rather than a million.
// The budget goes unspent: a group costs one entry however many codes it
// covers, so the walk is only as long as the subtable itself.
func (c *cmap13) reverse(int) map[GlyphIndex]rune {
inv := make(map[GlyphIndex]rune)
for _, g := range c.groups {
if g.end < g.start || g.glyphID == 0 {
continue
}
gid := GlyphIndex(g.glyphID)
if _, seen := inv[gid]; !seen {
inv[gid] = rune(g.start)
}
}
return inv
}
// cmap8Group is one sequential-map group of a format-8 subtable; identical in
// shape and semantics to a format-12 group (a range advancing one glyph per
// code point from startGID).
type cmap8Group struct {
start uint32
end uint32
startGID uint32
}
// cmap8 is a decoded format-8 (mixed 16/32-bit coverage) subtable. is32 is
// the on-disk 65536-bit bitmap (8192 bytes) flagging which 16-bit lead code
// units begin a 32-bit (surrogate-pair) character rather than standing alone
// as a BMP character; it is retained for spec fidelity but, like other
// implementations, is not consulted by lookup, since lookup already receives
// a fully decoded rune rather than raw UTF-16 code units — the groups alone
// are sufficient to resolve it, exactly as in format 12.
type cmap8 struct {
is32 [8192]byte
groups []cmap8Group
}
// parseCmap8 decodes a format-8 subtable from sub (starting at the format
// field).
func parseCmap8(sub []byte) (cmapLookup, error) {
r := reader{b: sub}
r.skip(12) // format, reserved, length, language
var c cmap8
for i := range c.is32 {
c.is32[i] = r.u8()
}
if r.err != nil {
return nil, fmt.Errorf("opentype: cmap format 8 is32: %w", r.err)
}
numGroups := int(r.u32())
if r.err != nil {
return nil, fmt.Errorf("opentype: cmap format 8 header: %w", r.err)
}
c.groups = make([]cmap8Group, numGroups)
for i := range c.groups {
c.groups[i].start = r.u32()
c.groups[i].end = r.u32()
c.groups[i].startGID = r.u32()
}
if r.err != nil {
return nil, fmt.Errorf("opentype: cmap format 8 groups: %w", r.err)
}
return &c, nil
}
// lookup implements the format-8 binary search over sorted groups, the same
// algorithm as format 12's lookup (the is32 bitmap plays no role once the
// input is already a decoded rune).
func (c *cmap8) lookup(r rune) (GlyphIndex, bool) {
cc := uint32(r)
lo, hi := 0, len(c.groups)
for lo < hi {
mid := (lo + hi) / 2
g := c.groups[mid]
if cc < g.start {
hi = mid
} else if cc > g.end {
lo = mid + 1
} else {
return GlyphIndex(g.startGID + (cc - g.start)), true
}
}
return 0, false
}
// reverse expands each group, exactly as format 12's does: the two formats
// describe their groups the same way.
func (c *cmap8) reverse(budget int) map[GlyphIndex]rune {
inv := make(map[GlyphIndex]rune)
for _, g := range c.groups {
if g.end < g.start {
continue
}
budget -= reverseRange(inv, g.start, g.end-g.start+1, g.startGID, budget)
if budget == 0 {
break
}
}
return inv
}
// cmap0 is a decoded format-0 (byte encoding table) subtable: a flat 256-byte
// glyph-id array covering U+0000-U+00FF.
type cmap0 struct {
glyphIDArray [256]byte
}
// parseCmap0 decodes a format-0 subtable from sub (starting at the format
// field).
func parseCmap0(sub []byte) (cmapLookup, error) {
r := reader{b: sub}
r.skip(6) // format, length, language
var c cmap0
for i := range c.glyphIDArray {
c.glyphIDArray[i] = r.u8()
}
if r.err != nil {
return nil, fmt.Errorf("opentype: cmap format 0: %w", r.err)
}
return &c, nil
}
// lookup returns the direct array entry for r, or false when r is outside
// U+0000-U+00FF or maps to the .notdef glyph.
func (c *cmap0) lookup(r rune) (GlyphIndex, bool) {
if uint32(r) > 0xFF {
return 0, false
}
g := c.glyphIDArray[r]
if g == 0 {
return 0, false
}
return GlyphIndex(g), true
}
// reverse walks the 256-entry array, keeping the lowest code per glyph.
// The budget goes unspent: the array is 256 entries.
func (c *cmap0) reverse(int) map[GlyphIndex]rune {
inv := make(map[GlyphIndex]rune)
for code, g := range c.glyphIDArray {
if g == 0 {
continue
}
if _, seen := inv[GlyphIndex(g)]; !seen {
inv[GlyphIndex(g)] = rune(code)
}
}
return inv
}
// cmap6 is a decoded format-6 (trimmed table mapping) subtable: a
// contiguous, zero-based glyph-id array starting at firstCode.
type cmap6 struct {
firstCode uint16
glyphIDArray []uint16
}
// parseCmap6 decodes a format-6 subtable from sub (starting at the format
// field).
func parseCmap6(sub []byte) (cmapLookup, error) {
r := reader{b: sub}
r.skip(6) // format, length, language
firstCode := r.u16()
entryCount := int(r.u16())
if r.err != nil {
return nil, fmt.Errorf("opentype: cmap format 6 header: %w", r.err)
}
c := &cmap6{firstCode: firstCode, glyphIDArray: make([]uint16, entryCount)}
for i := range c.glyphIDArray {
c.glyphIDArray[i] = r.u16()
}
if r.err != nil {
return nil, fmt.Errorf("opentype: cmap format 6: %w", r.err)
}
return c, nil
}
// lookup returns the trimmed-table entry for r, or false when r falls
// outside [firstCode, firstCode+len(glyphIDArray)) or maps to .notdef.
func (c *cmap6) lookup(r rune) (GlyphIndex, bool) {
if uint32(r) < uint32(c.firstCode) {
return 0, false
}
idx := uint32(r) - uint32(c.firstCode)
if idx >= uint32(len(c.glyphIDArray)) {
return 0, false
}
g := c.glyphIDArray[idx]
if g == 0 {
return 0, false
}
return GlyphIndex(g), true
}
// reverse walks the trimmed array, keeping the lowest code per glyph.
// The budget goes unspent: the array is no longer than the subtable said.
func (c *cmap6) reverse(int) map[GlyphIndex]rune {
inv := make(map[GlyphIndex]rune)
for i, g := range c.glyphIDArray {
if g == 0 {
continue
}
if _, seen := inv[GlyphIndex(g)]; !seen {
inv[GlyphIndex(g)] = rune(uint32(c.firstCode) + uint32(i))
}
}
return inv
}
// cmap10 is a decoded format-10 (trimmed array) subtable: the 32-bit
// analogue of format 6 — a contiguous, zero-based glyph-id array starting at
// startCharCode, wide enough to cover astral code points.
type cmap10 struct {
startCharCode uint32
glyphIDArray []uint16
}
// parseCmap10 decodes a format-10 subtable from sub (starting at the format
// field).
func parseCmap10(sub []byte) (cmapLookup, error) {
r := reader{b: sub}
r.skip(12) // format, reserved, length, language
startCharCode := r.u32()
numChars := int(r.u32())
if r.err != nil {
return nil, fmt.Errorf("opentype: cmap format 10 header: %w", r.err)
}
c := &cmap10{startCharCode: startCharCode, glyphIDArray: make([]uint16, numChars)}
for i := range c.glyphIDArray {
c.glyphIDArray[i] = r.u16()
}
if r.err != nil {
return nil, fmt.Errorf("opentype: cmap format 10: %w", r.err)
}
return c, nil
}
// lookup returns the trimmed-table entry for r, or false when r falls
// outside [startCharCode, startCharCode+len(glyphIDArray)) or maps to
// .notdef.
func (c *cmap10) lookup(r rune) (GlyphIndex, bool) {
if uint32(r) < c.startCharCode {
return 0, false
}
idx := uint32(r) - c.startCharCode
if idx >= uint32(len(c.glyphIDArray)) {
return 0, false
}
g := c.glyphIDArray[idx]
if g == 0 {
return 0, false
}
return GlyphIndex(g), true
}
// reverse walks the trimmed array, keeping the lowest code per glyph.
// The budget goes unspent: the array is no longer than the subtable said.
func (c *cmap10) reverse(int) map[GlyphIndex]rune {
inv := make(map[GlyphIndex]rune)
for i, g := range c.glyphIDArray {
if g == 0 {
continue
}
if _, seen := inv[GlyphIndex(g)]; !seen {
inv[GlyphIndex(g)] = rune(c.startCharCode + uint32(i))
}
}
return inv
}
// cmap2SubHeader is one decoded subHeader record of a format-2 subtable.
// fieldOffset is the byte offset of this subHeader's idRangeOffset field
// relative to the start of glyphIndexArray, precomputed so lookup can turn
// (fieldOffset + idRangeOffset + 2*index) into a glyphIndexArray index the
// same way format 4 turns its own idRangeOffset into an array index.
type cmap2SubHeader struct {
firstCode uint16
entryCount uint16
idDelta int16
idRangeOffset uint16
fieldOffset int
}
// cmap2 is a decoded format-2 (high-byte mapping through table) subtable,
// the legacy encoding for mixed single/double-byte CJK charsets. The code
// passed to lookup is treated as a raw 1- or 2-byte legacy code (its low 16
// bits), exactly like the other cmap formats treat their input as a raw
// code rather than interpreting charset semantics.
type cmap2 struct {
subHeaderKeys [256]uint16 // per high-byte value, subHeaders' byte offset (index*8)
subHeaders []cmap2SubHeader
glyphIndexArray []uint16
}
// parseCmap2 decodes a format-2 subtable from sub (starting at the format
// field). The subtable has no explicit subHeader count; it is derived from
// the highest offset referenced by subHeaderKeys, following the same
// convention used by other OpenType/TrueType cmap format-2 parsers.
func parseCmap2(sub []byte) (cmapLookup, error) {
r := reader{b: sub}
r.skip(6) // format, length, language
var c cmap2
for i := range c.subHeaderKeys {
c.subHeaderKeys[i] = r.u16()
}
if r.err != nil {
return nil, fmt.Errorf("opentype: cmap format 2 keys: %w", r.err)
}
var maxKey uint16
for _, k := range c.subHeaderKeys {
if k > maxKey {
maxKey = k
}
}
numSubHeaders := int(maxKey)/8 + 1
glyphIndexArrayStart := r.pos + numSubHeaders*8
c.subHeaders = make([]cmap2SubHeader, numSubHeaders)
for i := range c.subHeaders {
fieldPos := r.pos + 6 // firstCode(2)+entryCount(2)+idDelta(2) precede idRangeOffset
c.subHeaders[i] = cmap2SubHeader{
firstCode: r.u16(),
entryCount: r.u16(),
idDelta: r.i16(),
idRangeOffset: r.u16(),
fieldOffset: fieldPos - glyphIndexArrayStart,
}
}
if r.err != nil {
return nil, fmt.Errorf("opentype: cmap format 2 subheaders: %w", r.err)
}
// The remaining bytes are the glyphIndexArray (uint16 each); rest is sized
// to whatever fits, so this run of reads never overruns.
rest := (len(sub) - r.pos) / 2
c.glyphIndexArray = make([]uint16, rest)
for i := range c.glyphIndexArray {
c.glyphIndexArray[i] = r.u16()
}
return &c, nil
}
// lookup implements the format-2 high-byte dispatch: the high byte of r
// selects a subHeader via subHeaderKeys (a value of 0 selects subHeader 0,
// which also serves single-byte codes, since a single-byte code has an
// implicit high byte of 0), then the low byte is checked against that
// subHeader's [firstCode, firstCode+entryCount) range and translated through
// glyphIndexArray and idDelta, mirroring format 4's idRangeOffset scheme.
func (c *cmap2) lookup(r rune) (GlyphIndex, bool) {
if uint32(r) > 0xFFFF {
return 0, false
}
code := uint32(r)
hi := (code >> 8) & 0xFF
lo := code & 0xFF
sh := c.subHeaders[int(c.subHeaderKeys[hi])/8]
rel := lo - uint32(sh.firstCode)
if rel >= uint32(sh.entryCount) || sh.idRangeOffset == 0 {
return 0, false
}
gidx := (sh.fieldOffset + int(sh.idRangeOffset) + 2*int(rel)) / 2
if gidx < 0 || gidx >= len(c.glyphIndexArray) {
return 0, false
}
g := c.glyphIndexArray[gidx]
if g == 0 {
return 0, false
}
return GlyphIndex(uint16(int32(g) + int32(sh.idDelta))), true
}
// reverse walks every code the high-byte dispatch can reach, resolving each
// the same way lookup does. A single-byte code has an implicit high byte of
// zero, so the walk covers the whole 16-bit space rather than only the codes
// some subHeader claims.
// The budget goes unspent: the walk is the 16-bit code space, once.
func (c *cmap2) reverse(int) map[GlyphIndex]rune {
inv := make(map[GlyphIndex]rune)
for cc := uint32(0); cc <= 0xFFFF; cc++ {
g, ok := c.lookup(rune(cc))
if !ok {
continue
}
if _, seen := inv[g]; !seen {
inv[g] = rune(cc)
}
}
return inv
}
// cmap14DefaultRange is one entry of a format-14 default-UVS table: base
// runes in [start, start+count) pair with this variation selector to
// resolve through the font's ordinary cmap rather than an explicit glyph.
type cmap14DefaultRange struct {
start uint32
count uint32
}
// cmap14NonDefault is one entry of a format-14 non-default-UVS table: base
// rune unicode paired with this variation selector resolves to glyph
// directly, bypassing the ordinary cmap.
type cmap14NonDefault struct {
unicode uint32
glyph GlyphIndex
}
// cmap14Record is one variation-selector record of a format-14 subtable.
type cmap14Record struct {
varSelector uint32
defaultRanges []cmap14DefaultRange
nonDefault []cmap14NonDefault
}
// cmap14 is a decoded format-14 (Unicode Variation Sequences) subtable. It is
// never installed as Font.cmap: it is a side table consulted only through
// GlyphIndexVariation.
type cmap14 struct {
records []cmap14Record
}
// parseCmap14 decodes a format-14 subtable from sub (starting at the format
// field).
func parseCmap14(sub []byte) (*cmap14, error) {
r := reader{b: sub}
r.skip(6) // format, length
numRecords := int(r.u32())
if r.err != nil {
return nil, fmt.Errorf("opentype: cmap format 14 header: %w", r.err)
}
type rawRecord struct {
varSelector uint32
defaultOff, nonDefaultOff uint32
}
raws := make([]rawRecord, numRecords)
for i := range raws {
raws[i] = rawRecord{
varSelector: r.u24(),
defaultOff: r.u32(),
nonDefaultOff: r.u32(),
}
}
if r.err != nil {
return nil, fmt.Errorf("opentype: cmap format 14 records: %w", r.err)
}
c := &cmap14{records: make([]cmap14Record, numRecords)}
for i, raw := range raws {
c.records[i].varSelector = raw.varSelector
if raw.defaultOff != 0 {
ranges, err := parseUVSDefaultTable(sub, int(raw.defaultOff))
if err != nil {
return nil, err
}
c.records[i].defaultRanges = ranges
}
if raw.nonDefaultOff != 0 {
maps, err := parseUVSNonDefaultTable(sub, int(raw.nonDefaultOff))
if err != nil {
return nil, err
}
c.records[i].nonDefault = maps
}
}
return c, nil
}
// parseUVSDefaultTable decodes a Default UVS table at byte offset off within
// sub (the format-14 subtable, offsets are subtable-relative). The offset
// itself is only checked against sub's length here; a short read of the
// table's own fields is reported separately by the reader below.
func parseUVSDefaultTable(sub []byte, off int) ([]cmap14DefaultRange, error) {
if off > len(sub) {
return nil, fmt.Errorf("opentype: cmap format 14 default UVS offset: %w", errTruncated)
}
r := reader{b: sub[off:]}
numRanges := int(r.u32())
if r.err != nil {
return nil, fmt.Errorf("opentype: cmap format 14 default UVS header: %w", r.err)
}
ranges := make([]cmap14DefaultRange, numRanges)
for i := range ranges {
start := r.u24()
additionalCount := r.u8()
ranges[i] = cmap14DefaultRange{start: start, count: uint32(additionalCount) + 1}
}
if r.err != nil {
return nil, fmt.Errorf("opentype: cmap format 14 default UVS ranges: %w", r.err)
}
return ranges, nil
}
// parseUVSNonDefaultTable decodes a Non-Default UVS table at byte offset off
// within sub (the format-14 subtable, offsets are subtable-relative). The
// offset itself is only checked against sub's length here; a short read of
// the table's own fields is reported separately by the reader below.
func parseUVSNonDefaultTable(sub []byte, off int) ([]cmap14NonDefault, error) {
if off > len(sub) {
return nil, fmt.Errorf("opentype: cmap format 14 non-default UVS offset: %w", errTruncated)
}
r := reader{b: sub[off:]}
numMappings := int(r.u32())
if r.err != nil {
return nil, fmt.Errorf("opentype: cmap format 14 non-default UVS header: %w", r.err)
}
maps := make([]cmap14NonDefault, numMappings)
for i := range maps {
maps[i] = cmap14NonDefault{unicode: r.u24(), glyph: GlyphIndex(r.u16())}
}
if r.err != nil {
return nil, fmt.Errorf("opentype: cmap format 14 non-default UVS mappings: %w", r.err)
}
return maps, nil
}
// lookupVariation resolves (r, vs) against c's records. A non-default
// mapping (an explicit glyph) takes precedence over a default-range match
// (which falls back to f's ordinary cmap); a vs with no record, or an r
// matched by neither its non-default mappings nor its default ranges,
// reports not-ok.
func (c *cmap14) lookupVariation(f *Font, r, vs rune) (GlyphIndex, bool) {
target := uint32(vs)
for _, rec := range c.records {
if rec.varSelector != target {
continue
}
cc := uint32(r)
for _, nd := range rec.nonDefault {
if nd.unicode == cc {
return nd.glyph, true
}
}
for _, dr := range rec.defaultRanges {
if cc >= dr.start && cc-dr.start < dr.count {
return f.GlyphIndex(r)
}
}
return 0, false
}
return 0, false
}
vendor/github.com/go-opentype/opentype/descriptor.go
Generated Vendored
+163
View File
@@ -0,0 +1,163 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
// This file surfaces the font-level descriptor scalars a PDF FontDescriptor (and
// general typographic use) needs, without a consumer having to re-parse the sfnt
// container itself. The values are decoded from head (bounding box, macStyle),
// OS/2 (weight/width class, cap/x height, family class, selection flags) and post
// (italic angle, fixed-pitch) at Parse time; the accessors below expose them.
//
// Every scalar has a defined value even when its source table is absent: the
// head-derived ones are always available, while the OS/2 and post ones fall back
// to a documented default (0, or the ascender for a missing cap height) so a
// caller never has to special-case a partial font.
// parseDescriptor decodes the optional OS/2 and post tables into the descriptor
// scalars. Both are optional: a font without them keeps the zero defaults, which
// the accessors translate into sensible fallbacks. A too-short table is treated
// as absent rather than an error, matching the tolerant handling a descriptor
// consumer expects (the required metrics all come from head/hhea, already parsed).
func (f *Font) parseDescriptor(tables map[string][]byte) {
if b, ok := tables["OS/2"]; ok && len(b) >= 78 {
f.os2Present = true
f.os2Version = int(be16(b))
f.weightClass = int(be16(b[4:]))
f.widthClass = int(be16(b[6:]))
f.sFamilyClass = int(sbe16(b[30:]))
f.fsSelection = be16(b[62:])
// sxHeight and sCapHeight were added in OS/2 version 2; older tables (and
// truncated version-2 ones) simply do not carry them.
if f.os2Version >= 2 && len(b) >= 90 {
f.sxHeight = int(sbe16(b[86:]))
f.sCapHeight = int(sbe16(b[88:]))
}
}
if b, ok := tables["post"]; ok && len(b) >= 16 {
f.postPresent = true
f.italicAngle = float64(int32(be32(b[4:]))) / 65536.0
f.isFixedPitch = be32(b[12:]) != 0
}
}
// UnitsPerEm returns the font's design units per em (from head), the scale in
// which every other font-unit metric this package reports is expressed. It is
// never zero (Parse rejects a font whose head declares zero).
func (f *Font) UnitsPerEm() int { return f.unitsPerEm }
// Ascent returns the typographic ascender in font units (hhea.ascender): the
// height above the baseline. It is the same value Face.Metrics scales to pixels.
func (f *Font) Ascent() int { return f.ascender }
// Descent returns the typographic descender in font units (hhea.descender). It
// is conventionally negative (below the baseline), as stored in hhea.
func (f *Font) Descent() int { return f.descender }
// LineGap returns the typographic line gap in font units (hhea.lineGap): the
// recommended extra spacing between lines beyond ascent minus descent.
func (f *Font) LineGap() int { return f.lineGap }
// FontBBox returns the font bounding box in font units (head.xMin/yMin/xMax/yMax):
// the union of every glyph's bounds. A PDF FontDescriptor's /FontBBox is this box
// scaled to glyph space.
func (f *Font) FontBBox() (xMin, yMin, xMax, yMax int) {
return f.xMin, f.yMin, f.xMax, f.yMax
}
// CapHeight returns the cap height in font units (OS/2.sCapHeight). It is zero
// when the font has no OS/2 table or an OS/2 version below 2, which do not carry
// it; a PDF consumer that needs a non-zero value substitutes the ascender.
func (f *Font) CapHeight() int { return f.sCapHeight }
// XHeight returns the x height in font units (OS/2.sxHeight). It is zero when the
// font has no OS/2 table or an OS/2 version below 2 (which do not carry it).
func (f *Font) XHeight() int { return f.sxHeight }
// ItalicAngle returns the italic (slant) angle in counter-clockwise degrees from
// vertical (post.italicAngle): zero for an upright font, negative for the usual
// forward slant. It is zero when the font has no post table.
func (f *Font) ItalicAngle() float64 { return f.italicAngle }
// WeightClass returns the OS/2 usWeightClass (100 thin .. 400 normal .. 900 black),
// or 0 when the font carries no OS/2 table.
func (f *Font) WeightClass() int { return f.weightClass }
// WidthClass returns the OS/2 usWidthClass (1 ultra-condensed .. 5 normal ..
// 9 ultra-expanded), or 0 when the font carries no OS/2 table.
func (f *Font) WidthClass() int { return f.widthClass }
// IsFixedPitch reports whether the font is monospaced (post.isFixedPitch).
func (f *Font) IsFixedPitch() bool { return f.isFixedPitch }
// IsItalic reports whether the font is italic or oblique, from any of the three
// places a font may record it: a non-zero post italic angle, the head macStyle
// italic bit, or the OS/2 fsSelection italic bit.
func (f *Font) IsItalic() bool {
return f.italicAngle != 0 || f.macStyle&0x02 != 0 || f.fsSelection&0x01 != 0
}
// IsSerif reports whether the font is a serif design, read from the high byte of
// OS/2.sFamilyClass (the IBM font-class id): classes 1-7 are serif families,
// class 8 is sans-serif, and the rest (script, symbolic, ...) are neither. It is
// false when the font has no OS/2 table.
func (f *Font) IsSerif() bool {
class := f.sFamilyClass >> 8
return class >= 1 && class <= 7
}
// isSymbolic reports whether the font is a symbol font (OS/2 sFamilyClass 12).
// It drives the PDF Symbolic/Nonsymbolic flag: symbol fonts must be flagged
// Symbolic, every other font Nonsymbolic.
func (f *Font) isSymbolic() bool { return f.sFamilyClass>>8 == 12 }
// PDF FontDescriptor flag bits (PDF 32000-1:2008, Table 123).
const (
pdfFlagFixedPitch = 1 << 0
pdfFlagSerif = 1 << 1
pdfFlagSymbolic = 1 << 2
pdfFlagNonsymbolic = 1 << 5
pdfFlagItalic = 1 << 6
)
// Flags returns the PDF FontDescriptor /Flags value for the font: a bit set
// combining FixedPitch, Serif, Symbolic-or-Nonsymbolic and Italic, derived from
// the post, OS/2 and head tables. A caller embeds it verbatim as /Flags. Exactly
// one of the Symbolic and Nonsymbolic bits is always set.
func (f *Font) Flags() int {
flags := 0
if f.isFixedPitch {
flags |= pdfFlagFixedPitch
}
if f.IsSerif() {
flags |= pdfFlagSerif
}
if f.isSymbolic() {
flags |= pdfFlagSymbolic
} else {
flags |= pdfFlagNonsymbolic
}
if f.IsItalic() {
flags |= pdfFlagItalic
}
return flags
}
// StemV returns an estimate, in font units, of the dominant vertical stem width
// a PDF FontDescriptor requires as /StemV. OpenType fonts do not store StemV, so
// it is approximated from the OS/2 weight class with the linear relation
// 50 + (weightClass-100)*11/100 (about 83 at the normal weight 400, ~116 at bold
// 700), clamped to a floor of 50. A font without an OS/2 table is treated as the
// normal weight. Callers that can do better should read WeightClass directly.
func (f *Font) StemV() int {
w := f.weightClass
if w == 0 {
w = 400
}
v := 50 + (w-100)*11/100
if v < 50 {
v = 50
}
return v
}
vendor/github.com/go-opentype/opentype/doc.go
Generated Vendored
+51
View File
@@ -0,0 +1,51 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
// Package opentype is a pure-Go, CGO=0, standard-library-only parser and
// anti-aliased rasteriser for TrueType/OpenType fonts.
//
// It is a functional replacement for the narrow slice of
// golang.org/x/image/font/opentype needed by a glyph-blitting UI: parse a
// font blob, build a Face at a pixel size, then obtain per-rune advances and
// 8-bit alpha coverage masks.
//
// # Scope
//
// Phase 1 covers TrueType 'glyf' outlines: the sfnt container, the head,
// maxp, hhea, hmtx, cmap (formats 4 and 12), loca and glyf tables, simple and
// composite glyphs, implied on-curve points, and a non-zero-winding
// supersampling rasteriser. OpenType/CFF ('OTTO') outlines, GPOS/GSUB
// shaping, kerning and hinting are not implemented; see the README for the
// full support matrix.
//
// The OpenType MATH table (math-typesetting metrics: constants, per-glyph
// italic correction and kerning, and stretchy-glyph size variants and
// assemblies) is decoded and exposed pixel-scaled through a Face; see
// [Font.HasMath] and math.go. Math layout itself (box building, the TeX math
// rules) is a higher-level engine that consumes these values.
//
// # Usage
//
// f, err := opentype.Parse(ttf)
// if err != nil { /* handle */ }
// face := f.NewFace(16)
// adv := face.Measure("Hello")
// bounds, mask, maskp, advance, ok := face.GlyphMask('H', penX, baselineY)
//
// A Font is immutable after Parse and safe for concurrent use; a Face caches
// rasterised glyphs and is not safe for concurrent use.
//
// # Embedding and subsetting
//
// For a consumer that embeds a font (a PDF or EPUB writer), the package exposes
// the primitives such an embedder otherwise re-implements by re-parsing the sfnt:
// the PDF FontDescriptor scalars ([Font.UnitsPerEm], [Font.FontBBox],
// [Font.CapHeight], [Font.XHeight], [Font.ItalicAngle], [Font.StemV],
// [Font.Flags], ...); raw table access ([Font.TableTags], [Font.Table]);
// by-glyph-index advances that honour the current variation
// ([Face.AdvanceIndex], [Face.AdvanceIndexUnits], [Face.VerticalAdvanceIndex]);
// TrueType glyph subsetting ([Font.SubsetTrueType]) and CFF charstring subsetting
// ([Font.SubsetCFF]); and baking a variable font at a chosen axis position into a
// static instance ([Font.Instance], [Font.InstanceBytes]).
package opentype
vendor/github.com/go-opentype/opentype/face.go
Generated Vendored
+358
View File
@@ -0,0 +1,358 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import (
"image"
"math"
)
// Metrics holds a Face's vertical metrics in whole pixels.
type Metrics struct {
Ascent int // baseline-to-top distance (a positive height above the baseline)
Descent int // baseline-to-bottom distance (a positive depth below the baseline)
Height int // line height: ascent + descent + line gap
}
// Face renders a Font at a fixed pixel size. It caches rasterised glyphs and
// is not safe for concurrent use; build one Face per goroutine if needed.
type Face struct {
font *Font
sizePx int
scale float64
cache map[GlyphIndex]cachedGlyph
// varCoords, when non-nil, instances a variable font's glyf outlines at the
// given user-space axis coordinates before rasterising (see SetVariation).
// varNorm caches those coordinates normalized (NormalizeCoords) so the
// metric-variation tables (HVAR/VVAR/MVAR) can be evaluated per call.
varCoords map[string]float64
varNorm []int16
// hinting enables the TrueType instruction interpreter for glyf glyphs that
// carry instructions (see SetHinting). interp is built lazily on first use
// and reused across glyphs; interpReady guards that one-time build.
hinting bool
interp *interp
interpReady bool
// stemDarken enables CFF/CFF2 stem darkening (weight compensation) when
// hinting is on: stems are embiggened by a small ppem-dependent amount to
// improve contrast on the anti-aliased rasteriser at small sizes (see
// SetStemDarkening). Off by default, so the default hinted output is
// byte-identical to the undarkened result.
stemDarken bool
}
// cachedGlyph is a rasterised glyph held in the Face cache. ok is false when
// the glyph could not be decoded (a corrupt outline), in which case it renders
// as nothing.
type cachedGlyph struct {
bounds image.Rectangle
mask *image.Alpha
ok bool
}
// roundInt rounds a float to the nearest integer (halves away from zero).
func roundInt(v float64) int { return int(math.Round(v)) }
// NewFace returns a Face that renders f at sizePx pixels per em. The scale
// factor is sizePx / unitsPerEm; sizePx should be positive.
func (f *Font) NewFace(sizePx int) *Face {
return &Face{
font: f,
sizePx: sizePx,
scale: float64(sizePx) / float64(f.unitsPerEm),
cache: map[GlyphIndex]cachedGlyph{},
}
}
// Font returns the Font this Face renders. It lets a shaper reach the font's
// layout tables ([Font.GSUB], [Font.GPOS]) and per-glyph metrics
// ([Font.GlyphAdvance]) while keeping the Face for pixel sizing.
func (fc *Face) Font() *Font { return fc.font }
// Scale returns the face's font-unit-to-pixel scale factor (the render size in
// pixels divided by the font's unitsPerEm). A shaper multiplies a font-unit
// advance or offset by it to obtain whole pixels at the face's size.
func (fc *Face) Scale() float64 { return fc.scale }
// Metrics returns the face's vertical metrics in pixels. When a variation is
// set and the font carries an MVAR table, the global metric deltas for the
// ascender ("hasc"), descender ("hdsc") and line gap ("hlgp") are folded in.
func (fc *Face) Metrics() Metrics {
s := fc.scale
asc := float64(fc.font.ascender)
desc := float64(fc.font.descender)
gap := float64(fc.font.lineGap)
if fc.varCoords != nil && fc.font.mvar != nil {
asc += fc.font.mvar.delta("hasc", fc.varNorm)
desc += fc.font.mvar.delta("hdsc", fc.varNorm)
gap += fc.font.mvar.delta("hlgp", fc.varNorm)
}
return Metrics{
Ascent: roundInt(asc * s),
Descent: roundInt(-desc * s),
Height: roundInt((asc - desc + gap) * s),
}
}
// hAdvanceUnits returns glyph gid's horizontal advance in font units at the
// face's current variation: the base advance plus, when a variation is set, the
// HVAR delta (or the gvar phantom-point fallback). With no variation set the
// result is exactly the base advance.
func (fc *Face) hAdvanceUnits(gid GlyphIndex) float64 {
adv := float64(fc.font.GlyphAdvance(gid))
if fc.varCoords != nil {
adv += fc.font.horizontalAdvanceDelta(int(gid), fc.varNorm)
}
return adv
}
// Advance returns the horizontal advance of r in pixels, or 0 if the rune is
// not mapped by the font's cmap. When a variation is set the HVAR delta (or the
// gvar phantom-point fallback) is included.
func (fc *Face) Advance(r rune) int {
gid, ok := fc.font.GlyphIndex(r)
if !ok {
return 0
}
return roundInt(fc.hAdvanceUnits(gid) * fc.scale)
}
// Measure returns the total advance width of s in pixels (the sum of each
// rune's Advance).
func (fc *Face) Measure(s string) int {
total := 0
for _, r := range s {
total += fc.Advance(r)
}
return total
}
// glyph returns the cached rasterisation of glyph gid, rendering it on first
// use.
func (fc *Face) glyph(gid GlyphIndex) cachedGlyph {
if cg, ok := fc.cache[gid]; ok {
return cg
}
cg := fc.render(gid)
fc.cache[gid] = cg
return cg
}
// render decodes and rasterises glyph gid at the face's size.
func (fc *Face) render(gid GlyphIndex) cachedGlyph {
contours, err := fc.outline(gid)
if err != nil {
return cachedGlyph{ok: false}
}
polys := make([][]fpoint, 0, len(contours))
for _, c := range contours {
p := flattenContour(c, fc.scale)
if len(p) >= 3 {
polys = append(polys, p)
}
}
bounds, mask := rasterize(polys)
return cachedGlyph{bounds: bounds, mask: mask, ok: true}
}
// GlyphMask rasterises r and positions it with (x, y) as the pen origin on the
// baseline. It returns the destination bounds, an *image.Alpha coverage mask,
// the offset into that mask corresponding to bounds.Min (always the origin),
// the advance width in pixels, and ok.
//
// ok is false when the rune is not mapped by the cmap or its glyph outline is
// corrupt; callers should render nothing in that case. A mapped-but-empty
// glyph (for example a space) returns ok true with a nil mask, an empty
// bounds, and its advance.
func (fc *Face) GlyphMask(r rune, x, y int) (bounds image.Rectangle, mask *image.Alpha, maskp image.Point, advance int, ok bool) {
gid, mapped := fc.font.GlyphIndex(r)
if !mapped {
return image.Rectangle{}, nil, image.Point{}, 0, false
}
cg := fc.glyph(gid)
if !cg.ok {
return image.Rectangle{}, nil, image.Point{}, 0, false
}
advance = roundInt(fc.hAdvanceUnits(gid) * fc.scale)
bounds = cg.bounds.Add(image.Point{X: x, Y: y})
return bounds, cg.mask, image.Point{}, advance, true
}
// GlyphMaskIndex rasterises the glyph with index gid directly — bypassing the
// cmap — and positions it with (x, y) as the pen origin on the baseline. It is
// the by-glyph-index counterpart of [Face.GlyphMask]: a complex-text shaper
// (see github.com/go-opentype/shape) emits glyph indices after applying GSUB
// substitutions and GPOS positioning, so the runes have already been resolved
// to glyphs and a caller must render each glyph by its index rather than by a
// rune. Callers add the shaper's per-glyph pixel offset to (x, y) before the
// call and advance the pen by the shaper's advance afterwards.
//
// It returns the destination bounds, an *image.Alpha coverage mask, the offset
// into that mask corresponding to bounds.Min (always the origin), the glyph's
// own horizontal advance width in pixels, and ok.
//
// ok is false when gid is out of range or its outline is corrupt; callers
// should render nothing in that case. A valid-but-empty glyph (for example a
// space) returns ok true with a nil mask, an empty bounds, and its advance.
func (fc *Face) GlyphMaskIndex(gid GlyphIndex, x, y int) (bounds image.Rectangle, mask *image.Alpha, maskp image.Point, advance int, ok bool) {
cg := fc.glyph(gid)
if !cg.ok {
return image.Rectangle{}, nil, image.Point{}, 0, false
}
advance = roundInt(fc.hAdvanceUnits(gid) * fc.scale)
bounds = cg.bounds.Add(image.Point{X: x, Y: y})
return bounds, cg.mask, image.Point{}, advance, true
}
// outline returns glyph gid's outline as contours in font units, selecting the
// outline source (CFF2 vs CFF vs glyf), applying variable-font instancing when a
// variation is set, and applying hinting when it is enabled.
//
// A CFF2 font is inherently variable: its outlines are instanced at the face's
// variation coordinates (the default master when none is set), converted to the
// normalized F2Dot14 form cff2.outline expects via NormalizeCoords. When hinting
// is enabled both CFF2 and CFF1 outlines are grid-fitted (see cffhint.go), so
// SetHinting affects all three outline flavours; with hinting off they are
// returned unhinted.
func (fc *Face) outline(gid GlyphIndex) ([]contour, error) {
if fc.font.t1 != nil {
// A Type 1 program carries hints of its own, in a form nothing here
// grid-fits; its outlines are drawn as they were written.
return fc.font.t1.outline(int(gid))
}
if fc.font.cff2 != nil {
contours, gh, err := fc.font.cff2.outlineHints(int(gid), fc.font.NormalizeCoords(fc.varCoords))
if err != nil {
return nil, err
}
if fc.hinting {
contours = cffGridFit(contours, gh, fc.scale, float64(fc.sizePx), fc.stemDarken)
}
return contours, nil
}
if fc.font.cff != nil {
contours, gh, err := fc.font.cff.outlineHints(int(gid))
if err != nil {
return nil, err
}
if fc.hinting {
contours = cffGridFit(contours, gh, fc.scale, float64(fc.sizePx), fc.stemDarken)
}
return contours, nil
}
var (
contours []contour
err error
)
if fc.varCoords != nil {
contours, err = fc.font.InstancePoints(int(gid), fc.varCoords)
} else {
contours, err = fc.font.glyphContours(gid)
}
if err != nil {
return nil, err
}
if fc.hinting {
contours = fc.hintedOutline(gid, contours)
}
return contours, nil
}
// SetVariation instances the font at the user-space axis coordinates coords
// (keyed by axis tag, e.g. {"wght": 700}); axes absent from coords take their
// default. Subsequent GlyphMask calls reflect the instanced outlines, for both
// glyf (via gvar) and CFF2 (via its blend/vsindex operators) variable fonts.
// Pass nil to return to the default master. A non-variable font (or a CFF1 font,
// which has no variation) is unaffected. The glyph cache is invalidated so the
// change takes effect at once.
func (fc *Face) SetVariation(coords map[string]float64) {
fc.varCoords = coords
fc.varNorm = fc.font.NormalizeCoords(coords)
fc.cache = map[GlyphIndex]cachedGlyph{}
}
// SetHinting enables or disables the TrueType instruction interpreter. When
// enabled, a glyf glyph that carries instructions is grid-fitted at the face's
// pixel size before rasterising; glyphs without instructions, composite glyphs
// and CFF fonts are unaffected. Hinting is off by default (the default output is
// unhinted anti-aliased coverage). The glyph cache is invalidated.
func (fc *Face) SetHinting(on bool) {
fc.hinting = on
fc.cache = map[GlyphIndex]cachedGlyph{}
}
// SetStemDarkening enables or disables stem darkening (weight compensation) for
// CFF/CFF2 glyphs. When both hinting and darkening are on, every grid-fitted stem
// is embiggened by a small ppem-dependent amount (strongest at small sizes,
// fading to none at larger sizes) so stems keep enough contrast on the
// anti-aliased rasteriser at small text sizes, as FreeType's CFF engine does by
// default. It has no effect while hinting is off or on glyf (TrueType) outlines.
// Darkening is off by default, so the default hinted output is byte-identical to
// the undarkened result. The glyph cache is invalidated.
func (fc *Face) SetStemDarkening(on bool) {
fc.stemDarken = on
fc.cache = map[GlyphIndex]cachedGlyph{}
}
// hintInterp lazily builds the interpreter for this face (running the font
// program and control-value program once) and reuses it across glyphs. It
// returns nil when the font has no interpreter or its programs fail to run, in
// which case the caller renders the glyph unhinted.
func (fc *Face) hintInterp() *interp {
if !fc.interpReady {
fc.interpReady = true
if it, err := newInterp(fc.font, fc.sizePx); err == nil {
fc.interp = it
}
}
return fc.interp
}
// hintedOutline grid-fits glyph gid's contours through the TrueType interpreter
// and returns the hinted contours in font units. It falls back to the input
// contours when the glyph carries no instructions, the interpreter is
// unavailable, or the program fails. The whole outline is treated as one closed
// contour for IUP, matching the interpreter's model (see hint.go).
func (fc *Face) hintedOutline(gid GlyphIndex, contours []contour) []contour {
instr := fc.font.glyphInstructions(gid)
if len(instr) == 0 {
return contours
}
it := fc.hintInterp()
if it == nil {
return contours
}
// Scale to pixel space (what the interpreter expects), recording contour
// boundaries so the hinted points can be re-split into contours.
var pts []outlinePoint
ends := make([]int, 0, len(contours))
for _, c := range contours {
for _, p := range c {
pts = append(pts, outlinePoint{x: p.x * fc.scale, y: p.y * fc.scale, on: p.on})
}
ends = append(ends, len(pts)-1)
}
out, err := it.run(pts, instr)
if err != nil {
return contours
}
// Back to font units so the shared rasteriser (which re-applies scale)
// reproduces the hinted pixel positions.
result := make([]contour, len(contours))
start := 0
for i, e := range ends {
cc := make(contour, e-start+1)
for j := range cc {
p := out[start+j]
cc[j] = outlinePoint{x: p.x / fc.scale, y: p.y / fc.scale, on: p.on}
}
result[i] = cc
start = e + 1
}
return result
}
vendor/github.com/go-opentype/opentype/font.go
Generated Vendored
+345
View File
@@ -0,0 +1,345 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import (
"encoding/binary"
"errors"
"fmt"
)
// GlyphIndex is a glyph identifier within a font: an index into the font's
// glyph store, as produced by the cmap. Zero is the ".notdef" glyph.
type GlyphIndex uint16
// errTruncated is the sentinel a bounded reader records when a read runs past
// the end of its slice. Callers wrap it with table context.
var errTruncated = errors.New("unexpected end of data")
// Font is a parsed TrueType/OpenType font. It is immutable after Parse and
// safe for concurrent use by multiple goroutines. Build a Face from it with
// NewFace to obtain sized metrics and rasterised glyphs.
type Font struct {
unitsPerEm int
indexToLocFormat int
numGlyphs int
ascender int
descender int
lineGap int
numberOfHMetrics int
advances []int // advanceWidth per glyph, in font units
lsbs []int // left side bearing per glyph, in font units
loca []uint32
glyf []byte
cmap cmapLookup
cmaps []cmapSubtable // every decoded cmap subtable, with the platform and encoding it was written for
cmapVS *cmap14 // format-14 Unicode Variation Sequences subtable, if present
cff *cffTable // CFF/Type2 outlines for an OpenType ("OTTO") font, if present
t1 *type1Font // PostScript Type 1 outlines, for a program read by ParseType1
glyphNames map[string]GlyphIndex // what the font calls its glyphs, when it names them
cff2 *cff2Table // CFF2 (variable Compact Font Format) outlines, if present
fvar *fvarTable // optional: variation axes and named instances
avar *avarTable // optional: axis-value segment maps
gvar *gvarTable // optional: glyph variation (delta) data
hvar *hvarTable // optional: horizontal metrics variation (advances)
vvar *vvarTable // optional: vertical metrics variation (advances)
mvar *mvarTable // optional: global font-metric variation
// OpenType Layout tables, all optional. Absence is not an error.
gdef *gdefTable // glyph definitions (classes, mark sets) for lookup flags
gsub *gsub // glyph substitution (ligatures, single substitution)
gpos *gpos // glyph positioning (pair kerning)
kern *kern // legacy kern table (kerning fallback)
// OpenType MATH table (math-typesetting metrics), optional. Absence is not an
// error; it powers Font.HasMath and the Face math accessors (see math.go).
math *mathTable
// Vertical writing-mode metrics (vhea/vmtx/VORG), all optional. They enable
// top-to-bottom (CJK tategaki) layout; absence means vertical metrics are
// unavailable. See vertical.go.
vertAscender int // vhea vertTypoAscender, font units
vertDescender int // vhea vertTypoDescender, font units
vertLineGap int // vhea vertTypoLineGap, font units
numOfLongVerMetrics int // vhea count of full vmtx entries
vertAdvances []int // advanceHeight per glyph, font units (vmtx)
tsbs []int // top side bearing per glyph, font units (vmtx)
vorgDefault int // VORG defaultVertOriginY, font units
vorg map[GlyphIndex]int // VORG per-glyph vertical origin overrides
hasVhea bool
hasVmtx bool
hasVORG bool
// TrueType instruction (hinting) tables and limits. These are optional:
// a font without them simply cannot be hinted. See hint.go.
fpgm []byte // font program (function definitions)
prep []byte // control-value program
cvt []int16
maxStorage int
maxFunctionDefs int
maxStackElements int
maxTwilightPts int
// Descriptor scalars decoded from head, OS/2 and post at Parse time. They
// back the PDF FontDescriptor accessors and general layout use (descriptor.go).
xMin, yMin, xMax, yMax int // head glyph bounding box, font units
macStyle uint16 // head macStyle bits (bit 0 bold, bit 1 italic)
os2Present bool // an OS/2 table was present and long enough to decode
os2Version int // OS/2 table version (governs the availability of x/CapHeight)
weightClass int // OS/2 usWeightClass (100..900), 0 when absent
widthClass int // OS/2 usWidthClass (1..9), 0 when absent
sFamilyClass int // OS/2 sFamilyClass (high byte is the IBM font class)
fsSelection uint16 // OS/2 fsSelection bits (bit 0 italic)
sCapHeight int // OS/2 sCapHeight, font units (version >= 2), 0 when absent
sxHeight int // OS/2 sxHeight, font units (version >= 2), 0 when absent
postPresent bool // a post table was present and long enough to decode
italicAngle float64 // post italicAngle (degrees, 0 for upright)
isFixedPitch bool // post isFixedPitch (monospaced)
// tables retains the raw, undecoded sfnt table slices keyed by four-byte tag,
// so callers can read table bytes (Font.Table/Font.TableTags) and the
// subsetting/instancing code can rebuild a container (subset.go, instance.go).
tables map[string][]byte
}
// be16 reads a big-endian uint16 from b[0:2]; the caller guarantees len(b)>=2.
func be16(b []byte) uint16 { return binary.BigEndian.Uint16(b) }
// be32 reads a big-endian uint32 from b[0:4]; the caller guarantees len(b)>=4.
func be32(b []byte) uint32 { return binary.BigEndian.Uint32(b) }
// sbe16 reads a big-endian int16 from b[0:2]; the caller guarantees len(b)>=2.
func sbe16(b []byte) int16 { return int16(binary.BigEndian.Uint16(b)) }
// Recognised sfnt version tags.
const (
versionTrueType = 0x00010000 // 'glyf' outlines
versionTrue = 0x74727565 // "true", Apple TrueType
versionOTTO = 0x4F54544F // "OTTO", CFF/OpenType outlines (phase 2)
)
// Parse decodes a TrueType/OpenType font from b and returns a Font. The byte
// slice is retained (not copied) and must not be mutated by the caller.
//
// It fails on a corrupt or unsupported container: a bad sfnt magic, truncated
// data, or a missing required table — which does not include the character
// map: see [Font.HasCharacterMap]. All three outline flavours are supported:
// TrueType ('glyf'/'loca'), CFF/OpenType (a "OTTO" sfnt, or any sfnt carrying a
// 'CFF ' table, decoded via cff.go) and variable CFF2 (a sfnt carrying a 'CFF2'
// table, decoded via cff2.go).
func Parse(b []byte) (*Font, error) {
if len(b) < 12 {
return nil, fmt.Errorf("opentype: short header: %w", errTruncated)
}
version := be32(b)
switch version {
case versionTrueType, versionTrue, versionOTTO:
// supported: TrueType ('glyf') and CFF/OpenType ("OTTO") outlines.
default:
return nil, fmt.Errorf("opentype: bad sfnt version 0x%08x", version)
}
numTables := int(be16(b[4:]))
tables := make(map[string][]byte, numTables)
// The table directory is numTables records of 16 bytes starting at 12.
if 12+numTables*16 > len(b) {
return nil, fmt.Errorf("opentype: table directory: %w", errTruncated)
}
for i := 0; i < numTables; i++ {
rec := b[12+i*16:]
tag := string(rec[0:4])
off := int(be32(rec[8:]))
length := int(be32(rec[12:]))
if off+length > len(b) {
return nil, fmt.Errorf("opentype: table %q out of range", tag)
}
tables[tag] = b[off : off+length]
}
// The outline flavour is CFF/CFF2 when the sfnt is "OTTO" or carries a 'CFF '
// or 'CFF2' table; such a font needs its CFF table but not 'glyf'/'loca'. A
// 'CFF2' table (variable CFF) takes precedence and requires 'CFF2'; otherwise
// a CFF font requires 'CFF '. A TrueType font needs 'glyf'/'loca'.
_, hasCFF := tables["CFF "]
_, hasCFF2 := tables["CFF2"]
isCFF := version == versionOTTO || hasCFF || hasCFF2
// A 'cmap' is not among them. A font embedded in a PDF as a CIDFontType2
// is addressed by glyph number through the document's own map, and the
// subset that is embedded routinely leaves its character map out; refusing
// such a font would leave every glyph in it undrawable for want of a table
// nothing was going to consult.
required := []string{"head", "maxp", "hhea", "hmtx"}
switch {
case hasCFF2:
required = append(required, "CFF2")
case isCFF:
required = append(required, "CFF ")
default:
required = append(required, "loca", "glyf")
}
for _, name := range required {
if _, ok := tables[name]; !ok {
return nil, fmt.Errorf("opentype: missing required table %q", name)
}
}
f := &Font{tables: tables}
if err := f.parseHead(tables["head"]); err != nil {
return nil, err
}
if err := f.parseMaxp(tables["maxp"]); err != nil {
return nil, err
}
if err := f.parseHhea(tables["hhea"]); err != nil {
return nil, err
}
if err := f.parseHmtx(tables["hmtx"]); err != nil {
return nil, err
}
switch {
case hasCFF2:
c2, err := parseCFF2(tables["CFF2"])
if err != nil {
return nil, err
}
f.cff2 = c2
case isCFF:
cff, err := parseCFF(tables["CFF "])
if err != nil {
return nil, err
}
f.cff = cff
default:
if err := f.parseLoca(tables["loca"]); err != nil {
return nil, err
}
f.glyf = tables["glyf"]
}
if cm, ok := tables["cmap"]; ok {
if err := f.parseCmap(cm); err != nil {
return nil, err
}
}
// Optional TrueType instruction tables. Absence is not an error; it just
// means the font carries no hinting program (see hint.go).
f.fpgm = tables["fpgm"]
f.prep = tables["prep"]
f.parseCvt(tables["cvt "])
// Optional OpenType Font Variations tables. A font without them parses and
// renders exactly as before; when present they enable instancing at a
// chosen axis coordinate (see fvar.go, avar.go, gvar.go).
if err := f.parseVariations(tables); err != nil {
return nil, err
}
// Optional metric-variation tables (HVAR, VVAR, MVAR): advance widths,
// advance heights and global font metrics that track the axis coordinate.
// Absence is not an error (see metricvar.go).
if err := f.parseMetricVariations(tables); err != nil {
return nil, err
}
// Optional OpenType Layout tables: substitution (GSUB), positioning (GPOS)
// and the legacy kern table. Absence is not an error (see gsub.go, gpos.go,
// kern.go); they power Face.Shape and Face.Kern.
if err := f.parseLayout(tables); err != nil {
return nil, err
}
// Optional vertical writing-mode tables (vhea, vmtx, VORG). Absence is not an
// error; they supply the metrics for top-to-bottom layout (see vertical.go).
if err := f.parseVertical(tables); err != nil {
return nil, err
}
// Optional OpenType MATH table: the font-level metrics a math-typesetting
// engine consumes. Absence is not an error (see math.go).
if err := f.parseMath(tables); err != nil {
return nil, err
}
// Optional descriptor tables (OS/2, post): the metadata a PDF FontDescriptor
// and general styling consume. Absence is not an error (see descriptor.go).
f.parseDescriptor(tables)
return f, nil
}
// parseLayout decodes the optional GSUB, GPOS and kern tables. Each is
// independent; any subset (or none) may be present, and a malformed table is
// reported as an error so a corrupt font fails cleanly.
func (f *Font) parseLayout(tables map[string][]byte) error {
// GDEF is parsed first so its glyph classes and mark sets can be shared with
// the GSUB and GPOS appliers, which honour it through their lookup flags.
if b, ok := tables["GDEF"]; ok {
gd, err := parseGDEF(b)
if err != nil {
return err
}
f.gdef = gd
}
if b, ok := tables["GSUB"]; ok {
g, err := parseGSUB(b)
if err != nil {
return err
}
g.gdef = f.gdef
f.gsub = g
}
if b, ok := tables["GPOS"]; ok {
g, err := parseGPOS(b)
if err != nil {
return err
}
g.gdef = f.gdef
f.gpos = g
}
if b, ok := tables["kern"]; ok {
k, err := parseKern(b)
if err != nil {
return err
}
f.kern = k
}
return nil
}
// parseCvt decodes the Control Value Table: a run of int16 FUnit values. A
// missing or odd-length table yields as many whole entries as fit.
func (f *Font) parseCvt(b []byte) {
f.cvt = make([]int16, len(b)/2)
for i := range f.cvt {
f.cvt[i] = sbe16(b[i*2:])
}
}
// NumGlyphs returns the number of glyphs in the font (from the maxp table).
func (f *Font) NumGlyphs() int { return f.numGlyphs }
// GlyphIndex maps a rune to its glyph index via the selected cmap subtable.
// ok is false when the rune has no glyph in that subtable, and for a font that
// carries no character map at all — one addressed by glyph number, which
// [Font.HasCharacterMap] reports.
func (f *Font) GlyphIndex(r rune) (GlyphIndex, bool) {
if f.cmap == nil {
return 0, false
}
return f.cmap.lookup(r)
}
// HasCharacterMap reports whether the font can say which glyph a rune is. A
// font subset embedded in a PDF as a CIDFontType2 usually cannot: the document
// carries that map instead, and the font is addressed by glyph number.
func (f *Font) HasCharacterMap() bool { return f.cmap != nil }
// GlyphIndexVariation resolves a Unicode variation sequence (a base rune
// followed by a variation selector) to a glyph index, using the font's
// format-14 cmap subtable.
//
// ok is false when the font has no format-14 subtable, the variation
// selector is not registered in it, or the sequence is registered as a
// "default" mapping whose base rune has no glyph in the ordinary cmap.
func (f *Font) GlyphIndexVariation(r, vs rune) (GlyphIndex, bool) {
if f.cmapVS == nil {
return 0, false
}
return f.cmapVS.lookupVariation(f, r, vs)
}
vendor/github.com/go-opentype/opentype/fvar.go
Generated Vendored
+213
View File
@@ -0,0 +1,213 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import (
"fmt"
"math"
)
// This file implements the 'fvar' (Font Variations) table: the declaration of
// a variable font's design-variation axes and its named instances. Together
// with 'avar' (avar.go) and 'gvar' (gvar.go) it lets a variable font be
// instantiated at a chosen point in its design space instead of only at its
// default master.
//
// The three tables are parsed as optional add-ons: a font without them is a
// plain static font and is unaffected. Metric variations (HVAR/VVAR/MVAR) are
// implemented separately in metricvar.go; CFF2 outline variation lives in
// cff2.go. Deferred (not implemented here): style-attribute variations (STAT).
// Axis is one design-variation axis of a variable font, decoded from the
// 'fvar' table. Min, Default and Max are user-space coordinates (for example
// 100, 400 and 900 for a typical weight axis).
type Axis struct {
Tag string // four-character axis tag, e.g. "wght", "wdth", "slnt"
Min float64 // minimum user coordinate
Default float64 // default user coordinate (the font's default master)
Max float64 // maximum user coordinate
Flags uint16 // axis flags (bit 0: hidden axis)
NameID uint16 // 'name' table entry describing the axis
}
// NamedInstance is a named position in the variation space (for example
// "Bold" or "Condensed"), decoded from the 'fvar' table.
type NamedInstance struct {
SubfamilyNameID uint16 // 'name' entry for the instance subfamily
Flags uint16 // instance flags (reserved)
Coordinates map[string]float64 // axis tag -> user coordinate
PostScriptNameID uint16 // 'name' entry for the PostScript name, 0 if absent
}
// fvarTable is the parsed 'fvar' table.
type fvarTable struct {
axes []Axis
instances []NamedInstance
}
// parseVariations parses the optional variation tables. Each is independent:
// any subset (or none) may be present. A malformed table is reported as an
// error so a corrupt font fails cleanly rather than silently mis-rendering.
func (f *Font) parseVariations(tables map[string][]byte) error {
if b, ok := tables["fvar"]; ok {
if err := f.parseFvar(b); err != nil {
return err
}
}
if b, ok := tables["avar"]; ok {
if err := f.parseAvar(b); err != nil {
return err
}
}
if b, ok := tables["gvar"]; ok {
if err := f.parseGvar(b); err != nil {
return err
}
}
return nil
}
// readTag reads a four-byte OpenType tag as a string.
func readTag(r *reader) string {
if r.pos+4 > len(r.b) {
r.err = errTruncated
return ""
}
s := string(r.b[r.pos : r.pos+4])
r.pos += 4
return s
}
// readFixed reads a 16.16 signed fixed-point value as a float64.
func readFixed(r *reader) float64 {
return float64(int32(r.u32())) / 65536.0
}
// parseFvar decodes the 'fvar' table: a header, an array of axis records and
// an array of named-instance records. The header carries the offset to the
// axis array and the (forward-compatible) record sizes, which are honoured so
// that unknown trailing fields are skipped.
func (f *Font) parseFvar(b []byte) error {
r := reader{b: b}
r.u16() // majorVersion
r.u16() // minorVersion
axesOffset := int(r.u16())
r.u16() // reserved
axisCount := int(r.u16())
axisSize := int(r.u16())
instanceCount := int(r.u16())
instanceSize := int(r.u16())
if r.err != nil {
return fmt.Errorf("opentype: fvar table: %w", r.err)
}
axes := make([]Axis, axisCount)
for i := range axes {
ar := reader{b: b, pos: axesOffset + i*axisSize}
tag := readTag(&ar)
min := readFixed(&ar)
def := readFixed(&ar)
max := readFixed(&ar)
flags := ar.u16()
nameID := ar.u16()
if ar.err != nil {
return fmt.Errorf("opentype: fvar axis %d: %w", i, ar.err)
}
axes[i] = Axis{Tag: tag, Min: min, Default: def, Max: max, Flags: flags, NameID: nameID}
}
instances := make([]NamedInstance, instanceCount)
for i := range instances {
ir := reader{b: b, pos: axesOffset + axisCount*axisSize + i*instanceSize}
sub := ir.u16()
flags := ir.u16()
coords := make(map[string]float64, axisCount)
for a := 0; a < axisCount; a++ {
coords[axes[a].Tag] = readFixed(&ir)
}
var psID uint16
if instanceSize >= axisCount*4+6 {
psID = ir.u16()
}
if ir.err != nil {
return fmt.Errorf("opentype: fvar instance %d: %w", i, ir.err)
}
instances[i] = NamedInstance{
SubfamilyNameID: sub,
Flags: flags,
Coordinates: coords,
PostScriptNameID: psID,
}
}
f.fvar = &fvarTable{axes: axes, instances: instances}
return nil
}
// Axes returns the font's variation axes, or nil for a non-variable font.
func (f *Font) Axes() []Axis {
if f.fvar == nil {
return nil
}
return f.fvar.axes
}
// NamedInstances returns the font's named instances, or nil if the font is not
// variable or declares none.
func (f *Font) NamedInstances() []NamedInstance {
if f.fvar == nil {
return nil
}
return f.fvar.instances
}
// NormalizeCoords converts a set of user-space axis coordinates (keyed by axis
// tag) into per-axis normalized F2Dot14 values in the range [-1, 1], the form
// gvar and avar operate in. Axes not present in user take their default (which
// normalizes to 0). Values outside an axis's [min, max] are clamped. If the
// font declares an 'avar' table its segment maps are applied. The result has
// one entry per fvar axis, in axis order; it is nil for a non-variable font.
func (f *Font) NormalizeCoords(user map[string]float64) []int16 {
if f.fvar == nil {
return nil
}
axes := f.fvar.axes
out := make([]int16, len(axes))
for i, ax := range axes {
v := ax.Default
if u, ok := user[ax.Tag]; ok {
v = u
}
if v < ax.Min {
v = ax.Min
}
if v > ax.Max {
v = ax.Max
}
// v is clamped to [min, max]; whenever v < default the axis has
// default > min, and whenever v > default it has max > default, so the
// divisors below are always positive. v == default yields n == 0.
var n float64
switch {
case v < ax.Default:
n = (v - ax.Default) / (ax.Default - ax.Min)
case v > ax.Default:
n = (v - ax.Default) / (ax.Max - ax.Default)
}
if f.avar != nil {
n = f.avar.apply(i, n)
}
if n < -1 {
n = -1
}
if n > 1 {
n = 1
}
out[i] = int16(math.Round(n * 16384))
}
return out
}
vendor/github.com/go-opentype/opentype/gdef.go
Generated Vendored
+263
View File
@@ -0,0 +1,263 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import "fmt"
// This file decodes the GDEF (Glyph Definition) table and implements the
// OpenType Layout lookup flags that GSUB and GPOS honour during matching. GDEF
// classifies glyphs (base / ligature / mark / component), assigns marks to
// attachment classes and defines mark glyph sets; the lookupFlag of each lookup
// then selects, via those classifications, which glyphs a lookup skips as it
// walks the run. Without a GDEF table (or with a zero lookupFlag) nothing is
// skipped and matching is identical to a plain contiguous walk.
//
// The flag bits (OpenType "LookupFlag") are:
//
// - RightToLeft (0x0001): cursive attachment joins right-to-left.
// - IgnoreBaseGlyphs (0x0002): skip GDEF class 1 (base) glyphs.
// - IgnoreLigatures (0x0004): skip GDEF class 2 (ligature) glyphs.
// - IgnoreMarks (0x0008): skip GDEF class 3 (mark) glyphs.
// - UseMarkFilteringSet (0x0010): skip marks absent from the referenced
// mark glyph set (its index is the lookup's markFilteringSet field).
// - MarkAttachmentType (0xFF00): skip marks whose GDEF mark-attachment class
// differs from the type in the flag's high byte.
// lookupFlag bit masks.
const (
flagRightToLeft = 0x0001
flagIgnoreBaseGlyphs = 0x0002
flagIgnoreLigatures = 0x0004
flagIgnoreMarks = 0x0008
flagUseMarkFilteringSet = 0x0010
flagMarkAttachTypeMask = 0xFF00
)
// GDEF GlyphClassDef class values.
const (
gdefClassBase = 1 // simple glyph (a letter)
gdefClassLigature = 2 // ligature glyph (multiple combined glyphs)
gdefClassMark = 3 // combining mark (diacritic)
gdefClassComponent = 4 // ligature component (part of a ligature)
)
// gdefTable is a decoded GDEF table. All three maps/slices are optional (an
// absent subtable yields an empty map or nil slice); a font may carry a GDEF
// table with only some of them present.
type gdefTable struct {
glyphClass map[GlyphIndex]int // GlyphClassDef: base/ligature/mark/component
markAttach map[GlyphIndex]int // MarkAttachClassDef: mark-attachment class per mark
markGlyphSets []map[GlyphIndex]int // MarkGlyphSets: one Coverage per set
}
// parseGDEF decodes a GDEF table. AttachList and LigCaretList are tolerated but
// not decoded (they carry hinting/caret data GSUB/GPOS matching does not need).
func parseGDEF(b []byte) (*gdefTable, error) {
r := reader{b: b}
r.skip(2) // majorVersion
minor := r.u16()
glyphClassOff := int(r.u16())
r.skip(2) // attachListOffset (tolerated, not decoded)
r.skip(2) // ligCaretListOffset (tolerated, not decoded)
markAttachOff := int(r.u16())
markSetsOff := 0
if minor >= 2 {
markSetsOff = int(r.u16())
}
// minor >= 3 adds an itemVarStore Offset32 here; it is not needed for
// matching and is left unread.
if r.err != nil {
return nil, fmt.Errorf("opentype: gdef header: %w", r.err)
}
t := &gdefTable{glyphClass: map[GlyphIndex]int{}, markAttach: map[GlyphIndex]int{}}
if glyphClassOff != 0 {
cd, err := parseClassDef(subslice(b, glyphClassOff))
if err != nil {
return nil, err
}
t.glyphClass = cd
}
if markAttachOff != 0 {
cd, err := parseClassDef(subslice(b, markAttachOff))
if err != nil {
return nil, err
}
t.markAttach = cd
}
if markSetsOff != 0 {
sets, err := parseMarkGlyphSets(subslice(b, markSetsOff))
if err != nil {
return nil, err
}
t.markGlyphSets = sets
}
return t, nil
}
// parseMarkGlyphSets decodes a MarkGlyphSetsDef table: a format, a set count and
// that many Offset32 to Coverage tables (from the start of b).
func parseMarkGlyphSets(b []byte) ([]map[GlyphIndex]int, error) {
r := reader{b: b}
r.skip(2) // format (1)
n := int(r.u16())
offs := make([]int, n)
for i := 0; i < n; i++ {
offs[i] = int(r.u32())
}
if r.err != nil {
return nil, fmt.Errorf("opentype: markGlyphSets: %w", r.err)
}
sets := make([]map[GlyphIndex]int, n)
for i, off := range offs {
cov, err := parseCoverage(subslice(b, off))
if err != nil {
return nil, err
}
sets[i] = cov
}
return sets, nil
}
// markSetCovers reports whether mark glyph g is a member of mark glyph set i.
func (t *gdefTable) markSetCovers(i int, g GlyphIndex) bool {
if i < 0 || i >= len(t.markGlyphSets) {
return false
}
_, ok := t.markGlyphSets[i][g]
return ok
}
// skipper decides which glyphs a lookup ignores while matching, from the
// lookup's flag (and mark-filtering-set index) against the font's GDEF table. A
// zero-value skipper (no GDEF) skips nothing, so matching degrades to a plain
// contiguous walk and behaviour is identical to a font without GDEF.
type skipper struct {
gdef *gdefTable
flag uint16
markFilterSet uint16
}
// rtl reports whether the lookup's RightToLeft flag is set (consulted by
// cursive attachment).
func (sk skipper) rtl() bool { return sk.flag&flagRightToLeft != 0 }
// isMark reports whether g is a GDEF mark glyph (class 3). It is false without a
// GDEF table, so mark-aware base finding degrades to the plain search.
func (sk skipper) isMark(g GlyphIndex) bool {
return sk.gdef != nil && sk.gdef.glyphClass[g] == gdefClassMark
}
// skip reports whether glyph g is ignored by this lookup: a base/ligature/mark
// the corresponding Ignore flag suppresses, or a mark filtered out by the
// mark-attachment type or the mark-filtering set.
func (sk skipper) skip(g GlyphIndex) bool {
if sk.gdef == nil {
return false
}
class := sk.gdef.glyphClass[g]
if sk.flag&flagIgnoreBaseGlyphs != 0 && class == gdefClassBase {
return true
}
if sk.flag&flagIgnoreLigatures != 0 && class == gdefClassLigature {
return true
}
if sk.flag&flagIgnoreMarks != 0 && class == gdefClassMark {
return true
}
if class == gdefClassMark {
if sk.flag&flagUseMarkFilteringSet != 0 {
return !sk.gdef.markSetCovers(int(sk.markFilterSet), g)
}
if attach := sk.flag & flagMarkAttachTypeMask; attach != 0 {
return sk.gdef.markAttach[g] != int(attach>>8)
}
}
return false
}
// next returns the index of the first non-skipped glyph at position >= from, or
// len(g) when none remains.
func (sk skipper) next(g []GlyphIndex, from int) int {
for from < len(g) && sk.skip(g[from]) {
from++
}
return from
}
// prev returns the index of the first non-skipped glyph at position <= from, or
// -1 when none remains.
func (sk skipper) prev(g []GlyphIndex, from int) int {
for from >= 0 && sk.skip(g[from]) {
from--
}
return from
}
// glyphPred tests one glyph during sequence matching (by glyph id, class or
// coverage membership, depending on the sub-format).
type glyphPred func(GlyphIndex) bool
// matchForward matches preds against successive non-skipped positions starting
// at the first non-skipped glyph at or after `from`. It returns the matched
// absolute positions, the index just past the last match (where a following
// lookahead search begins), and whether every predicate matched. An empty preds
// list matches trivially at `from`.
func (sk skipper) matchForward(g []GlyphIndex, from int, preds []glyphPred) (pos []int, end int, ok bool) {
pos = make([]int, len(preds))
p := from
for k, pred := range preds {
p = sk.next(g, p)
if p >= len(g) || !pred(g[p]) {
return nil, 0, false
}
pos[k] = p
p++
}
return pos, p, true
}
// matchBackward matches preds against successive non-skipped positions walking
// backward from `from` (used for backtrack sequences, which are stored in the
// reverse of text order).
func (sk skipper) matchBackward(g []GlyphIndex, from int, preds []glyphPred) bool {
p := from
for _, pred := range preds {
p = sk.prev(g, p)
if p < 0 || !pred(g[p]) {
return false
}
p--
}
return true
}
// matchContext matches a format-3 coverage input sequence: covs[0] must cover
// the anchor glyph g[i], and covs[1:] must cover successive non-skipped
// positions after i. It returns the matched absolute positions (starting with
// i), the index just past the input and whether it matched. An empty covs list
// does not match.
func (sk skipper) matchContext(g []GlyphIndex, i int, covs []map[GlyphIndex]int) (pos []int, end int, ok bool) {
if len(covs) == 0 {
return nil, 0, false
}
if _, ok := covs[0][g[i]]; !ok {
return nil, 0, false
}
rest, end, ok := sk.matchForward(g, i+1, covPreds(covs[1:]))
if !ok {
return nil, 0, false
}
return append([]int{i}, rest...), end, true
}
// covPreds builds coverage-membership predicates from a list of Coverage maps.
func covPreds(covs []map[GlyphIndex]int) []glyphPred {
preds := make([]glyphPred, len(covs))
for k := range covs {
c := covs[k]
preds[k] = func(g GlyphIndex) bool { _, ok := c[g]; return ok }
}
return preds
}
vendor/github.com/go-opentype/opentype/glyf.go
Generated Vendored
+291
View File
@@ -0,0 +1,291 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import (
"errors"
"fmt"
)
// outlinePoint is one point of a glyph contour in font units. on reports
// whether it lies on the curve (true) or is an off-curve quadratic control
// point (false).
type outlinePoint struct {
x, y float64
on bool
}
// contour is one closed sequence of outline points.
type contour []outlinePoint
// Simple-glyph flag bits (TrueType spec).
const (
flagOnCurve = 0x01
flagXShort = 0x02
flagYShort = 0x04
flagRepeat = 0x08
flagXSamePosX = 0x10 // X is same (long form) or positive (short form)
flagYSamePosY = 0x20 // Y is same (long form) or positive (short form)
)
// Composite-glyph flag bits (TrueType spec).
const (
compArgsAreWords = 0x0001
compArgsAreXY = 0x0002
compWeHaveScale = 0x0008
compMoreComponents = 0x0020
compXAndYScale = 0x0040
compTwoByTwo = 0x0080
)
// maxCompositeDepth bounds composite-glyph recursion; anything deeper is
// rejected as malformed (in addition to explicit cycle detection).
const maxCompositeDepth = 8
// glyphContours returns the outline of glyph gid, resolving composites, with
// a fresh recursion-tracking state. It returns the default-master outline; a
// nil norm disables variation (see loadGlyph).
func (f *Font) glyphContours(gid GlyphIndex) ([]contour, error) {
return f.loadGlyph(gid, 0, map[GlyphIndex]bool{}, nil)
}
// glyphInstructions returns the TrueType instruction bytecode attached to a
// simple glyph gid, or nil for an empty glyph, a composite glyph, or a glyph
// with no instructions. It is only called after glyphContours has successfully
// decoded gid, so the loca range and the simple-glyph header are known valid.
func (f *Font) glyphInstructions(gid GlyphIndex) []byte {
start, end := f.loca[gid], f.loca[gid+1]
if start == end {
return nil // empty glyph (e.g. space)
}
r := reader{b: f.glyf[start:end]}
n := int(r.i16())
if n < 0 {
return nil // composite glyph: not hinted here
}
r.skip(8) // xMin, yMin, xMax, yMax
for i := 0; i < n; i++ {
r.u16() // endPtsOfContours[i]
}
instrLen := int(r.u16())
if instrLen == 0 {
return nil
}
return r.b[r.pos : r.pos+instrLen]
}
// loadGlyph decodes glyph gid's contours in font units. depth and visited
// guard composite recursion against over-deep nesting and cycles. norm, when
// non-nil, is the normalized variation coordinate at which the glyph is
// instanced: simple-glyph points get their 'gvar' deltas (with IUP) and
// composite-component offsets get theirs (see gvar.go). A nil norm yields the
// unvaried default-master outline.
func (f *Font) loadGlyph(gid GlyphIndex, depth int, visited map[GlyphIndex]bool, norm []int16) ([]contour, error) {
if depth > maxCompositeDepth {
return nil, errors.New("opentype: composite glyph nesting too deep")
}
if int(gid) >= f.numGlyphs {
return nil, fmt.Errorf("opentype: glyph index %d out of range", int(gid))
}
if visited[gid] {
return nil, fmt.Errorf("opentype: cyclic composite glyph %d", int(gid))
}
start, end := f.loca[gid], f.loca[gid+1]
if start > end {
return nil, fmt.Errorf("opentype: glyph %d: loca not monotonic", int(gid))
}
if int(end) > len(f.glyf) {
return nil, fmt.Errorf("opentype: glyph %d: %w", int(gid), errTruncated)
}
if start == end {
return nil, nil // empty glyph (e.g. space)
}
visited[gid] = true
defer delete(visited, gid)
r := reader{b: f.glyf[start:end]}
numberOfContours := int(r.i16())
r.skip(8) // xMin, yMin, xMax, yMax
if numberOfContours >= 0 {
contours, err := f.simpleGlyph(&r, numberOfContours)
if err != nil {
return nil, err
}
return f.varySimple(int(gid), contours, norm), nil
}
return f.compositeGlyph(int(gid), &r, depth, visited, norm)
}
// simpleGlyph decodes a simple (non-composite) glyph's n contours.
func (f *Font) simpleGlyph(r *reader, n int) ([]contour, error) {
endPts := make([]uint16, n)
for i := range endPts {
endPts[i] = r.u16()
}
numPoints := 0
if n > 0 {
numPoints = int(endPts[n-1]) + 1
}
instrLen := int(r.u16())
r.skip(instrLen)
flags := make([]uint8, numPoints)
for i := 0; i < numPoints; {
fl := r.u8()
flags[i] = fl
i++
if fl&flagRepeat != 0 {
rep := int(r.u8())
for j := 0; j < rep && i < numPoints; j++ {
flags[i] = fl
i++
}
}
}
xs := readCoords(r, flags, numPoints, flagXShort, flagXSamePosX)
ys := readCoords(r, flags, numPoints, flagYShort, flagYSamePosY)
if r.err != nil {
return nil, fmt.Errorf("opentype: simple glyph: %w", r.err)
}
contours := make([]contour, n)
p := 0
for ci := 0; ci < n; ci++ {
endp := int(endPts[ci])
var cnt contour
for ; p <= endp; p++ {
cnt = append(cnt, outlinePoint{
x: float64(xs[p]),
y: float64(ys[p]),
on: flags[p]&flagOnCurve != 0,
})
}
contours[ci] = cnt
}
return contours, nil
}
// readCoords decodes one axis (x or y) of point coordinates, accumulating the
// signed deltas per the short/same flag encoding.
func readCoords(r *reader, flags []uint8, numPoints int, shortBit, samePosBit uint8) []int {
coords := make([]int, numPoints)
v := 0
for i := 0; i < numPoints; i++ {
fl := flags[i]
if fl&shortBit != 0 {
d := int(r.u8())
if fl&samePosBit == 0 {
d = -d
}
v += d
} else if fl&samePosBit == 0 {
v += int(r.i16())
}
coords[i] = v
}
return coords
}
// glyphComponent is one decoded component record of a composite glyph: the
// referenced glyph, its placement offset (arg1/arg2, when argsAreXY) and its
// 2x2 transform (a,b,c,d).
type glyphComponent struct {
flags uint16
gid GlyphIndex
arg1, arg2 int
a, b, c, d float64
argsAreXY bool
}
// parseComponents decodes a composite glyph's component records; r must be
// positioned just past the glyph header. Components are returned in order.
func parseComponents(r *reader) ([]glyphComponent, error) {
var comps []glyphComponent
for {
var comp glyphComponent
comp.flags = r.u16()
comp.gid = GlyphIndex(r.u16())
if comp.flags&compArgsAreWords != 0 {
comp.arg1 = int(r.i16())
comp.arg2 = int(r.i16())
} else {
comp.arg1 = int(int8(r.u8()))
comp.arg2 = int(int8(r.u8()))
}
comp.argsAreXY = comp.flags&compArgsAreXY != 0
comp.a, comp.b, comp.c, comp.d = 1.0, 0.0, 0.0, 1.0
switch {
case comp.flags&compWeHaveScale != 0:
comp.a = r.f2dot14()
comp.d = comp.a
case comp.flags&compXAndYScale != 0:
comp.a = r.f2dot14()
comp.d = r.f2dot14()
case comp.flags&compTwoByTwo != 0:
comp.a = r.f2dot14()
comp.b = r.f2dot14()
comp.c = r.f2dot14()
comp.d = r.f2dot14()
}
if r.err != nil {
return nil, fmt.Errorf("opentype: composite glyph: %w", r.err)
}
comps = append(comps, comp)
if comp.flags&compMoreComponents == 0 {
return comps, nil
}
}
}
// compositeGlyph assembles a composite glyph from its components, recursively
// decoding and transforming each referenced glyph. When norm is non-nil the
// component offsets are first varied by 'gvar' (varyComponentOffsets) and each
// component is instanced recursively at the same coordinate.
func (f *Font) compositeGlyph(gid int, r *reader, depth int, visited map[GlyphIndex]bool, norm []int16) ([]contour, error) {
comps, err := parseComponents(r)
if err != nil {
return nil, err
}
dxs := make([]float64, len(comps))
dys := make([]float64, len(comps))
for i := range comps {
dxs[i] = float64(comps[i].arg1)
dys[i] = float64(comps[i].arg2)
}
f.varyComponentOffsets(gid, comps, dxs, dys, norm)
var out []contour
for i := range comps {
comp := comps[i]
if !comp.argsAreXY {
return nil, errors.New("opentype: unsupported: composite point matching")
}
sub, err := f.loadGlyph(comp.gid, depth+1, visited, norm)
if err != nil {
return nil, err
}
out = append(out, transformContours(sub, comp.a, comp.b, comp.c, comp.d, dxs[i], dys[i])...)
}
return out, nil
}
// transformContours applies the affine transform [a b c d] with translation
// (dx, dy) to every point of cs, returning fresh contours.
func transformContours(cs []contour, a, b, c, d, dx, dy float64) []contour {
out := make([]contour, len(cs))
for ci, cont := range cs {
nc := make(contour, len(cont))
for i, pt := range cont {
nc[i] = outlinePoint{
x: a*pt.x + c*pt.y + dx,
y: b*pt.x + d*pt.y + dy,
on: pt.on,
}
}
out[ci] = nc
}
return out
}
vendor/github.com/go-opentype/opentype/glyphpath.go
Generated Vendored
+164
View File
@@ -0,0 +1,164 @@
// Copyright (c) the go-opentype/opentype authors.
// SPDX-License-Identifier: BSD-3-Clause
package opentype
import (
"strconv"
"strings"
)
// Point is a 2-D coordinate in a glyph outline.
type Point struct{ X, Y float64 }
// SegmentOp identifies a glyph-outline path command.
type SegmentOp uint8
const (
// SegMoveTo starts a new contour at P[0].
SegMoveTo SegmentOp = iota
// SegLineTo draws a straight line to P[0].
SegLineTo
// SegQuadTo draws a quadratic Bézier with control point P[0] to endpoint P[1].
SegQuadTo
// SegClose closes the current contour; it carries no points.
SegClose
)
// Segment is one command of a glyph outline. SegMoveTo and SegLineTo use P[0];
// SegQuadTo uses P[0] as the quadratic control point and P[1] as the on-curve
// endpoint; SegClose uses neither.
type Segment struct {
Op SegmentOp
P [2]Point
}
// GlyphOutline returns glyph gid's outline as path segments in font design
// units, in the font's native Y-up orientation with the pen origin at (0, 0).
//
// TrueType off-curve points are emitted as [SegQuadTo] control points, with the
// implied on-curve midpoint synthesised between two consecutive off-curve
// points. CFF/CFF2 outlines are stored by this package pre-flattened to
// polylines, so they yield [SegLineTo] segments only.
//
// ok is false when gid is out of range or its outline is corrupt; a valid but
// empty glyph (for example a space) returns nil, true.
func (fc *Face) GlyphOutline(gid GlyphIndex) (segs []Segment, ok bool) {
cs, err := fc.outline(gid)
if err != nil {
return nil, false
}
for _, c := range cs {
segs = appendContour(segs, c)
}
return segs, true
}
// mid returns the midpoint of a and b.
func mid(a, b outlinePoint) Point {
return Point{X: (a.x + b.x) / 2, Y: (a.y + b.y) / 2}
}
// appendContour converts one closed contour of on/off-curve points into path
// segments (a MoveTo, the body, and a Close), following the TrueType rule that
// consecutive off-curve points imply an on-curve midpoint between them.
func appendContour(dst []Segment, c contour) []Segment {
n := len(c)
if n == 0 {
return dst
}
// Choose the starting on-curve point; an all-off-curve contour starts at the
// synthesised midpoint of its last and first points.
var start Point
var body contour
switch {
case c[0].on:
start, body = Point{c[0].x, c[0].y}, c[1:]
case c[n-1].on:
start, body = Point{c[n-1].x, c[n-1].y}, c[:n-1]
default:
start, body = mid(c[n-1], c[0]), c
}
dst = append(dst, Segment{Op: SegMoveTo, P: [2]Point{start}})
var ctrl Point
haveCtrl := false
for _, p := range body {
cur := Point{p.x, p.y}
if p.on {
if haveCtrl {
dst = append(dst, Segment{Op: SegQuadTo, P: [2]Point{ctrl, cur}})
haveCtrl = false
} else {
dst = append(dst, Segment{Op: SegLineTo, P: [2]Point{cur}})
}
continue
}
if haveCtrl {
// Two consecutive off-curve points: emit the implied midpoint.
m := Point{(ctrl.X + cur.X) / 2, (ctrl.Y + cur.Y) / 2}
dst = append(dst, Segment{Op: SegQuadTo, P: [2]Point{ctrl, m}})
}
ctrl, haveCtrl = cur, true
}
if haveCtrl {
dst = append(dst, Segment{Op: SegQuadTo, P: [2]Point{ctrl, start}})
}
return append(dst, Segment{Op: SegClose})
}
// GlyphSVGPath returns glyph gid's outline as an SVG path "d" string at the
// face's pixel size, in SVG's Y-down coordinate space with the pen origin at
// (0, 0) — so the baseline is y = 0 and ascenders have negative y. Callers place
// the glyph by wrapping the path in a translate transform.
//
// ok is false when gid is out of range or its outline is corrupt; a valid but
// empty glyph returns "", true.
func (fc *Face) GlyphSVGPath(gid GlyphIndex) (d string, ok bool) {
segs, ok := fc.GlyphOutline(gid)
if !ok {
return "", false
}
return segmentsToSVG(segs, fc.scale), true
}
// segmentsToSVG renders outline segments (font units, Y-up) into an SVG path
// "d" string scaled by scale and flipped to SVG's Y-down space.
func segmentsToSVG(segs []Segment, scale float64) string {
var b strings.Builder
writePoint := func(p Point) {
b.WriteString(ftoa(p.X * scale))
b.WriteByte(' ')
b.WriteString(ftoa(-p.Y * scale))
}
for _, s := range segs {
switch s.Op {
case SegMoveTo:
b.WriteByte('M')
writePoint(s.P[0])
case SegLineTo:
b.WriteByte('L')
writePoint(s.P[0])
case SegQuadTo:
b.WriteByte('Q')
writePoint(s.P[0])
b.WriteByte(' ')
writePoint(s.P[1])
case SegClose:
b.WriteByte('Z')
}
}
return b.String()
}
// ftoa formats v with up to three decimals and no trailing zeros (so "12.000"
// becomes "12" and "1.250" becomes "1.25"), keeping SVG paths compact. The
// fixed-precision form always contains a decimal point, so trimming is safe.
func ftoa(v float64) string {
if v == 0 { // collapse both +0 and the -0 produced by the Y flip
return "0"
}
s := strings.TrimRight(strconv.FormatFloat(v, 'f', 3, 64), "0")
return strings.TrimSuffix(s, ".")
}
vendor/github.com/go-opentype/opentype/gpos.go
Generated Vendored
+1653
View File
File diff suppressed because it is too large Load Diff
vendor/github.com/go-opentype/opentype/gsub.go
Generated Vendored
+1197
View File
File diff suppressed because it is too large Load Diff
vendor/github.com/go-opentype/opentype/gvar.go
Generated Vendored
+504
View File
@@ -0,0 +1,504 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import "fmt"
// This file implements the optional 'gvar' (Glyph Variations) table for 'glyf'
// outlines: the per-glyph store of point deltas that move a glyph's outline
// from the default master to a requested position in the variation space.
//
// A glyph's variation is a set of "tuple variations". Each tuple has a peak
// (and optionally an intermediate region) in normalized axis space, a set of
// point numbers it touches and a delta (dx, dy) per touched point. At a given
// normalized coordinate every tuple gets a scalar (how strongly it applies);
// the scaled deltas of all tuples are summed. Deltas for points a tuple does
// not touch are recovered by IUP (Interpolation of Untouched Points), done per
// contour against the default-master coordinates.
//
// Both simple and composite glyphs are variable: simple-glyph points get their
// deltas with per-contour IUP, while a composite glyph's per-component
// placement offsets (and the four phantom points) get theirs directly, with no
// IUP (each component point is independent). The four TrueType phantom points
// are carried through delta application (so "all points" and explicit point
// sets index correctly) but are dropped from the result.
//
// Metric variation via HVAR/VVAR/MVAR (advance widths, advance heights and
// global font metrics) is implemented separately in metricvar.go.
// gvarTable is the parsed 'gvar' table.
type gvarTable struct {
axisCount int
sharedTuples [][]float64 // sharedTupleCount peak tuples, each axisCount F2Dot14
dataOffsets []uint32 // glyphCount+1 offsets into the data array
dataArrayOff int // offset of the glyph-variation-data array within data
data []byte // the whole 'gvar' table
}
// Tuple index / header flag bits (gvar TupleVariationHeader.tupleIndex).
const (
gvEmbeddedPeak = 0x8000 // an embedded peak tuple follows the header
gvIntermediate = 0x4000 // embedded intermediate-region tuples follow
gvPrivatePoints = 0x2000 // this tuple has private point numbers
gvTupleIndexMsk = 0x0FFF // shared-tuple index (when no embedded peak)
)
// gvSharedPointNumbers is the bit in GlyphVariationData.tupleVariationCount
// flagging a shared point-number set at the head of the serialized data.
const gvSharedPointNumbers = 0x8000
// parseGvar decodes the 'gvar' header, the per-glyph offset table and the
// shared-tuple array. Per-glyph variation data is decoded lazily in
// applyVariation, so a glyph is only touched when it is actually instanced.
func (f *Font) parseGvar(b []byte) error {
r := reader{b: b}
r.u16() // majorVersion
r.u16() // minorVersion
axisCount := int(r.u16())
sharedTupleCount := int(r.u16())
sharedTuplesOffset := int(r.u32())
glyphCount := int(r.u16())
flags := r.u16()
dataArrayOffset := int(r.u32())
if r.err != nil {
return fmt.Errorf("opentype: gvar table: %w", r.err)
}
longOffsets := flags&1 != 0
offsets := make([]uint32, glyphCount+1)
for i := range offsets {
if longOffsets {
offsets[i] = r.u32()
} else {
offsets[i] = uint32(r.u16()) * 2
}
}
if r.err != nil {
return fmt.Errorf("opentype: gvar offsets: %w", r.err)
}
shared := make([][]float64, sharedTupleCount)
sr := reader{b: b, pos: sharedTuplesOffset}
for i := range shared {
t := make([]float64, axisCount)
for a := range t {
t[a] = sr.f2dot14()
}
shared[i] = t
}
if sr.err != nil {
return fmt.Errorf("opentype: gvar shared tuples: %w", sr.err)
}
f.gvar = &gvarTable{
axisCount: axisCount,
sharedTuples: shared,
dataOffsets: offsets,
dataArrayOff: dataArrayOffset,
data: b,
}
return nil
}
// applyVariation returns pts with glyph gid's 'gvar' deltas applied at the
// normalized coordinate norm (as produced by NormalizeCoords). pts are the
// glyph's default-master outline points in point order; ends holds the last
// point index of each contour (as in glyf's endPtsOfContours). Points untouched
// by a tuple are filled by per-contour IUP against pts. A font without 'gvar',
// a glyph with no variation data, or malformed data yields an unchanged copy.
func (f *Font) applyVariation(gid int, pts []outlinePoint, ends []int, norm []int16) []outlinePoint {
out := make([]outlinePoint, len(pts))
copy(out, pts)
gv := f.gvar
if gv == nil || gid < 0 || gid+1 >= len(gv.dataOffsets) {
return out
}
start := gv.dataArrayOff + int(gv.dataOffsets[gid])
end := gv.dataArrayOff + int(gv.dataOffsets[gid+1])
if start >= end || end > len(gv.data) {
return out
}
coords := make([]float64, gv.axisCount)
for i := range coords {
if i < len(norm) {
coords[i] = float64(norm[i]) / 16384.0
}
}
numReal := len(pts)
accX := make([]float64, numReal+4) // + four phantom points
accY := make([]float64, numReal+4)
if err := gv.applyGlyph(gv.data[start:end], coords, ends, pts, accX, accY, false); err != nil {
return out
}
for i := 0; i < numReal; i++ {
out[i].x += accX[i]
out[i].y += accY[i]
}
return out
}
// tupleHeader is a decoded TupleVariationHeader: the region it applies to and
// the size and point-set kind of its serialized data.
type tupleHeader struct {
size int // serialized data size in bytes (private points + deltas)
peak []float64 // per-axis peak coordinate
lower []float64 // per-axis region lower bound
upper []float64 // per-axis region upper bound
privatePoint bool // has private point numbers
}
// applyGlyph decodes one glyph's GlyphVariationData and accumulates every
// applicable tuple's scaled deltas into accX/accY (length numPts). For a simple
// glyph (composite false) untouched points of an explicit set are filled by
// per-contour IUP against pts; for a composite glyph (composite true, ends and
// pts nil) each point is a component offset or phantom point and untouched
// points simply stay zero — no IUP.
func (gv *gvarTable) applyGlyph(gd []byte, coords []float64, ends []int, pts []outlinePoint, accX, accY []float64, composite bool) error {
numPts := len(accX)
r := reader{b: gd}
countAndFlags := r.u16()
dataOffset := int(r.u16())
tupleCount := int(countAndFlags & gvTupleIndexMsk)
hasShared := countAndFlags&gvSharedPointNumbers != 0
headers := make([]tupleHeader, tupleCount)
for i := range headers {
h := tupleHeader{
size: int(r.u16()),
lower: make([]float64, gv.axisCount),
upper: make([]float64, gv.axisCount),
}
idx := r.u16()
if idx&gvEmbeddedPeak != 0 {
h.peak = make([]float64, gv.axisCount)
for a := range h.peak {
h.peak[a] = r.f2dot14()
}
} else {
si := int(idx & gvTupleIndexMsk)
if si >= len(gv.sharedTuples) {
return fmt.Errorf("opentype: gvar: shared tuple index %d out of range", si)
}
h.peak = gv.sharedTuples[si]
}
if idx&gvIntermediate != 0 {
for a := 0; a < gv.axisCount; a++ {
h.lower[a] = r.f2dot14()
}
for a := 0; a < gv.axisCount; a++ {
h.upper[a] = r.f2dot14()
}
} else {
for a := 0; a < gv.axisCount; a++ {
if h.peak[a] < 0 {
h.lower[a] = h.peak[a]
}
if h.peak[a] > 0 {
h.upper[a] = h.peak[a]
}
}
}
h.privatePoint = idx&gvPrivatePoints != 0
headers[i] = h
}
if r.err != nil {
return fmt.Errorf("opentype: gvar tuple headers: %w", r.err)
}
sd := reader{b: gd, pos: dataOffset}
var sharedPoints []int
sharedAll := false
if hasShared {
sharedPoints, sharedAll = readPackedPoints(&sd)
}
if sd.err != nil {
return fmt.Errorf("opentype: gvar shared points: %w", sd.err)
}
pos := sd.pos
for i := range headers {
h := headers[i]
tr := reader{b: gd, pos: pos}
points, all := sharedPoints, sharedAll
if h.privatePoint {
points, all = readPackedPoints(&tr)
}
n := len(points)
if all {
n = numPts
}
dx := readPackedDeltas(&tr, n)
dy := readPackedDeltas(&tr, n)
if tr.err != nil {
return fmt.Errorf("opentype: gvar tuple %d data: %w", i, tr.err)
}
scalar := supportScalar(coords, h.lower, h.peak, h.upper)
if scalar != 0 {
accumulate(scalar, all, points, dx, dy, ends, pts, accX, accY, !composite)
}
pos += h.size
}
return nil
}
// readPackedPoints decodes a packed point-number set. all is true for the
// "all points" encoding (a leading count of zero), in which case pts is nil.
// Numbers are stored as run-length-encoded deltas from the previous number.
func readPackedPoints(r *reader) (pts []int, all bool) {
first := r.u8()
count := int(first)
if first&0x80 != 0 {
count = int(first&0x7f)<<8 | int(r.u8())
}
if count == 0 {
return nil, true
}
pts = make([]int, 0, count)
val := 0
for len(pts) < count {
control := r.u8()
run := int(control&0x7f) + 1
words := control&0x80 != 0
for j := 0; j < run && len(pts) < count; j++ {
if words {
val += int(r.u16())
} else {
val += int(r.u8())
}
pts = append(pts, val)
}
if r.err != nil {
return pts, false
}
}
return pts, false
}
// readPackedDeltas decodes count run-length-encoded deltas (one axis's worth).
func readPackedDeltas(r *reader, count int) []float64 {
out := make([]float64, 0, count)
for len(out) < count {
control := r.u8()
run := int(control&0x3f) + 1
switch {
case control&0x80 != 0: // DELTAS_ARE_ZERO
for j := 0; j < run && len(out) < count; j++ {
out = append(out, 0)
}
case control&0x40 != 0: // DELTAS_ARE_WORDS
for j := 0; j < run && len(out) < count; j++ {
out = append(out, float64(int16(r.u16())))
}
default:
for j := 0; j < run && len(out) < count; j++ {
out = append(out, float64(int8(r.u8())))
}
}
if r.err != nil {
return out
}
}
return out
}
// supportScalar returns how strongly a tuple with the given per-axis region
// (lower <= peak <= upper) applies at coords: 1 at the peak, tapering linearly
// to 0 at the region edges, and 0 outside. An axis whose peak is 0, or whose
// region is degenerate or spans zero, does not constrain the tuple.
func supportScalar(coords, lower, peak, upper []float64) float64 {
scalar := 1.0
for a := range peak {
p := peak[a]
if p == 0 {
continue
}
lo, up := lower[a], upper[a]
if lo > p || p > up {
continue
}
if lo < 0 && up > 0 {
continue
}
v := coords[a]
if v == p {
continue
}
if v <= lo || up <= v {
return 0
}
if v < p {
scalar *= (v - lo) / (p - lo)
} else {
scalar *= (up - v) / (up - p)
}
}
return scalar
}
// accumulate adds one tuple's contribution to accX/accY. For an explicit point
// set it places each touched point's delta; iup then fills the untouched points
// per contour by interpolation (simple glyphs) or leaves them at zero
// (composite glyphs). For the "all points" set every point is touched and
// deltas apply directly.
func accumulate(scalar float64, all bool, points []int, dx, dy []float64, ends []int, pts []outlinePoint, accX, accY []float64, iup bool) {
numPts := len(accX)
fullX := make([]float64, numPts)
fullY := make([]float64, numPts)
if all {
for i := 0; i < numPts && i < len(dx); i++ {
fullX[i] = dx[i]
fullY[i] = dy[i]
}
} else {
touched := make([]bool, numPts)
for k, pi := range points {
if pi >= numPts {
continue
}
fullX[pi] = dx[k]
fullY[pi] = dy[k]
touched[pi] = true
}
if iup {
startIdx := 0
for _, e := range ends {
iupContour(pts[startIdx:e+1], fullX[startIdx:e+1], fullY[startIdx:e+1], touched[startIdx:e+1])
startIdx = e + 1
}
}
}
for i := 0; i < numPts; i++ {
accX[i] += scalar * fullX[i]
accY[i] += scalar * fullY[i]
}
}
// iupContour fills deltas for the untouched points of a single contour in
// place, interpolating each coordinate against the default-master positions in
// pts. With no touched point the contour is left unmoved; with exactly one the
// whole contour is shifted by that delta; otherwise untouched points between
// each pair of consecutive touched points (wrapping around the contour) are
// interpolated.
func iupContour(pts []outlinePoint, dx, dy []float64, touched []bool) {
n := len(pts)
var tp []int
for i := 0; i < n; i++ {
if touched[i] {
tp = append(tp, i)
}
}
if len(tp) == 0 {
return
}
if len(tp) == 1 {
for i := 0; i < n; i++ {
dx[i] = dx[tp[0]]
dy[i] = dy[tp[0]]
}
return
}
for k := 0; k < len(tp); k++ {
a := tp[k]
b := tp[(k+1)%len(tp)]
for i := (a + 1) % n; i != b; i = (i + 1) % n {
dx[i] = iupInterp(pts[i].x, pts[a].x, pts[b].x, dx[a], dx[b])
dy[i] = iupInterp(pts[i].y, pts[a].y, pts[b].y, dy[a], dy[b])
}
}
}
// iupInterp interpolates the delta of one untouched point at coordinate target,
// bracketed by two touched points at coordinates lo/hi with deltas dlo/dhi.
// Outside the bracket the nearer touched delta is held constant; inside, the
// delta is linearly interpolated by coordinate.
func iupInterp(target, lo, hi, dlo, dhi float64) float64 {
if lo > hi {
lo, hi = hi, lo
dlo, dhi = dhi, dlo
}
if target <= lo {
return dlo
}
if target >= hi {
return dhi
}
return dlo + (dhi-dlo)*(target-lo)/(hi-lo)
}
// varySimple applies glyph gid's 'gvar' point deltas to a simple glyph's
// contours at normalized coordinate norm, returning fresh contours. A nil norm
// or a font without 'gvar' returns contours unchanged (the default outline).
func (f *Font) varySimple(gid int, contours []contour, norm []int16) []contour {
if norm == nil || f.gvar == nil {
return contours
}
var pts []outlinePoint
var ends []int
for _, c := range contours {
pts = append(pts, c...)
ends = append(ends, len(pts)-1)
}
varied := f.applyVariation(gid, pts, ends, norm)
out := make([]contour, len(contours))
startIdx := 0
for i, e := range ends {
out[i] = varied[startIdx : e+1]
startIdx = e + 1
}
return out
}
// varyComponentOffsets applies glyph gid's 'gvar' deltas to a composite glyph's
// component placement offsets in place (dxs/dys, one entry per component). The
// gvar point set for a composite indexes the components in order followed by
// four phantom points; deltas apply with no IUP. A nil norm, a font without
// 'gvar', or malformed data leaves dxs/dys unchanged.
func (f *Font) varyComponentOffsets(gid int, comps []glyphComponent, dxs, dys []float64, norm []int16) {
if norm == nil {
return
}
gv := f.gvar
if gv == nil || gid < 0 || gid+1 >= len(gv.dataOffsets) {
return
}
start := gv.dataArrayOff + int(gv.dataOffsets[gid])
end := gv.dataArrayOff + int(gv.dataOffsets[gid+1])
if start >= end || end > len(gv.data) {
return
}
coords := make([]float64, gv.axisCount)
for i := range coords {
if i < len(norm) {
coords[i] = float64(norm[i]) / 16384.0
}
}
accX := make([]float64, len(comps)+4) // component offsets + four phantom points
accY := make([]float64, len(comps)+4)
if err := gv.applyGlyph(gv.data[start:end], coords, nil, nil, accX, accY, true); err != nil {
return
}
for i := range comps {
dxs[i] += accX[i]
dys[i] += accY[i]
}
}
// InstancePoints returns glyph gid's outline, as contours in font units,
// instanced at the user-space axis coordinates coords (keyed by axis tag).
// Axes absent from coords take their default. For a non-variable font, or a
// glyph with no variation data, it returns the default outline. Both simple
// and composite glyphs are varied: a composite's component offsets get their
// deltas and each component is instanced recursively at the same coordinate.
func (f *Font) InstancePoints(gid int, coords map[string]float64) ([]contour, error) {
norm := f.NormalizeCoords(coords)
return f.loadGlyph(GlyphIndex(gid), 0, map[GlyphIndex]bool{}, norm)
}
vendor/github.com/go-opentype/opentype/hint.go
Generated Vendored
+1755
View File
File diff suppressed because it is too large Load Diff
vendor/github.com/go-opentype/opentype/instance.go
Generated Vendored
+295
View File
@@ -0,0 +1,295 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import (
"encoding/binary"
"fmt"
)
// This file bakes a variable ('glyf'/'gvar') font at a chosen axis position into
// a plain static font, so an embedder that cannot ship a variable font (a PDF
// writer, say) can ship the exact instance a document uses. Every glyph's outline
// is instanced (gvar deltas applied, composites flattened) and re-encoded as a
// simple glyph, every advance width is instanced (HVAR, or the gvar phantom-point
// fallback), and the variation tables (fvar, avar, gvar, HVAR/VVAR/MVAR, STAT) are
// dropped so the result advertises — and behaves as — a single static master.
//
// Scope (documented): the CFF2 (variable CFF) outline path is not baked; a CFF2
// font returns an error. TrueType instructions and the shared hinting programs
// (cvt/fpgm/prep) are dropped, since flattening composites into simple glyphs
// renumbers the points the instructions address; the instance renders unhinted.
// Global metric variations (MVAR) are not folded into hhea/OS-2; the instance
// keeps the default master's global metrics.
// glyfWriter is a minimal big-endian byte writer for re-encoding glyf/loca and
// patching the cloned metric tables.
type glyfWriter struct{ b []byte }
func (w *glyfWriter) u16(v uint16) { w.b = binary.BigEndian.AppendUint16(w.b, v) }
func (w *glyfWriter) i16(v int16) { w.u16(uint16(v)) }
// Instance bakes the font at the user-space axis coordinates coords into a new,
// static Font: a font with the requested variation permanently applied and no
// variation axes. Axes absent from coords take their default. It is the parsed
// counterpart of InstanceBytes.
//
// It returns an error for a non-variable font, a CFF2 (variable CFF) font, or when
// coords names an axis the font does not have.
func (f *Font) Instance(coords map[string]float64) (*Font, error) {
b, err := f.InstanceBytes(coords)
if err != nil {
return nil, err
}
return Parse(b)
}
// InstanceBytes bakes the font at coords into the bytes of a new static TrueType
// sfnt (see Instance for the semantics). The returned font carries instanced glyf
// outlines and advance widths and no variation tables; Instance parses these bytes.
func (f *Font) InstanceBytes(coords map[string]float64) ([]byte, error) {
if f.cff2 != nil {
return nil, fmt.Errorf("opentype: Instance: CFF2 (variable CFF) fonts are not supported")
}
if f.fvar == nil {
return nil, fmt.Errorf("opentype: Instance: font is not variable (no fvar table)")
}
if f.glyf == nil {
return nil, fmt.Errorf("opentype: Instance: font has no glyf outlines to instance")
}
known := make(map[string]bool, len(f.fvar.axes))
for _, a := range f.fvar.axes {
known[a.Tag] = true
}
for tag := range coords {
if !known[tag] {
return nil, fmt.Errorf("opentype: Instance: unknown axis %q", tag)
}
}
norm := f.NormalizeCoords(coords)
glyphs := make([][]byte, f.numGlyphs)
advances := make([]int, f.numGlyphs)
lsbs := make([]int, f.numGlyphs)
haveBox := false
var xMin, yMin, xMax, yMax int
for gid := 0; gid < f.numGlyphs; gid++ {
contours, err := f.loadGlyph(GlyphIndex(gid), 0, map[GlyphIndex]bool{}, norm)
if err != nil {
return nil, err
}
glyphs[gid] = encodeSimpleGlyph(contours)
advances[gid] = roundInt(float64(f.advances[gid]) + f.horizontalAdvanceDelta(gid, norm))
gx0, gy0, gx1, gy1, ok := contourBounds(contours)
if ok {
lsbs[gid] = gx0
if !haveBox {
xMin, yMin, xMax, yMax, haveBox = gx0, gy0, gx1, gy1, true
} else {
xMin, yMin = min(xMin, gx0), min(yMin, gy0)
xMax, yMax = max(xMax, gx1), max(yMax, gy1)
}
}
}
loca, glyf := packGlyf(glyphs)
tables := map[string][]byte{
"glyf": glyf,
"loca": loca,
"head": f.instanceHead(xMin, yMin, xMax, yMax),
"hhea": f.subsetHhea(f.numGlyphs),
"maxp": instanceMaxp(f.tables["maxp"], glyphs),
"hmtx": buildHmtx(advances, lsbs),
}
// Carry over every non-variation table verbatim. The variation tables and the
// hinting tables are dropped (see the file comment); the six rebuilt tables
// above are already set and must not be overwritten by the originals.
drop := map[string]bool{
"fvar": true, "avar": true, "gvar": true,
"HVAR": true, "VVAR": true, "MVAR": true, "STAT": true,
"cvt ": true, "fpgm": true, "prep": true,
"glyf": true, "loca": true, "head": true,
"hhea": true, "maxp": true, "hmtx": true,
}
for tag, b := range f.tables {
if drop[tag] {
continue
}
tables[tag] = append([]byte(nil), b...)
}
return buildSFNT(versionTrueType, tables), nil
}
// packGlyf concatenates the per-glyph blobs into a glyf table and its matching
// long-format loca (numGlyphs+1 offsets), 2-byte-aligning each glyph.
func packGlyf(glyphs [][]byte) (loca, glyf []byte) {
offsets := make([]uint32, len(glyphs)+1)
for i, g := range glyphs {
offsets[i] = uint32(len(glyf))
glyf = append(glyf, g...)
for len(glyf)%2 != 0 {
glyf = append(glyf, 0)
}
}
offsets[len(glyphs)] = uint32(len(glyf))
loca = make([]byte, len(offsets)*4)
for i, o := range offsets {
binary.BigEndian.PutUint32(loca[i*4:], o)
}
return loca, glyf
}
// buildHmtx builds a full hmtx table: one (advanceWidth, leftSideBearing) pair for
// every glyph (numberOfHMetrics equals the glyph count).
func buildHmtx(advances, lsbs []int) []byte {
w := &glyfWriter{}
for i := range advances {
w.u16(uint16(advances[i]))
w.i16(int16(lsbs[i]))
}
return w.b
}
// instanceHead clones the head table with checkSumAdjustment zeroed (buildSFNT
// recomputes it), indexToLocFormat set to long, and the recomputed instanced glyph
// bounding box.
func (f *Font) instanceHead(xMin, yMin, xMax, yMax int) []byte {
head := append([]byte(nil), f.tables["head"]...)
binary.BigEndian.PutUint32(head[8:], 0)
binary.BigEndian.PutUint16(head[36:], uint16(int16(xMin)))
binary.BigEndian.PutUint16(head[38:], uint16(int16(yMin)))
binary.BigEndian.PutUint16(head[40:], uint16(int16(xMax)))
binary.BigEndian.PutUint16(head[42:], uint16(int16(yMax)))
binary.BigEndian.PutUint16(head[50:], 1)
return head
}
// instanceMaxp clones maxp with the outline statistics of the instanced glyphs:
// the max simple-glyph point and contour counts are recomputed, and the composite
// statistics are zeroed since the instance carries only simple glyphs. A
// version-0.5 maxp (too short to hold these fields) is returned unchanged.
func instanceMaxp(orig []byte, glyphs [][]byte) []byte {
maxp := append([]byte(nil), orig...)
if len(maxp) < 32 {
return maxp
}
maxPoints, maxContours := 0, 0
for _, g := range glyphs {
p, c := simpleGlyphStats(g)
maxPoints = max(maxPoints, p)
maxContours = max(maxContours, c)
}
binary.BigEndian.PutUint16(maxp[6:], uint16(maxPoints))
binary.BigEndian.PutUint16(maxp[8:], uint16(maxContours))
binary.BigEndian.PutUint16(maxp[10:], 0) // maxCompositePoints
binary.BigEndian.PutUint16(maxp[12:], 0) // maxCompositeContours
return maxp
}
// simpleGlyphStats returns the point and contour counts of a re-encoded simple
// glyph blob (as encodeSimpleGlyph produced it), or (0,0) for an empty glyph.
func simpleGlyphStats(g []byte) (points, contours int) {
if len(g) < 10 {
return 0, 0
}
n := int(int16(binary.BigEndian.Uint16(g)))
if n <= 0 {
return 0, 0
}
// endPtsOfContours[n-1] + 1 is the point count; the last end point sits at
// offset 10 + 2*(n-1).
last := 10 + 2*(n-1)
if last+2 > len(g) {
return 0, 0
}
return int(binary.BigEndian.Uint16(g[last:])) + 1, n
}
// contourBounds returns the integer bounding box of the (rounded) contour points
// and whether any point was present.
func contourBounds(contours []contour) (xMin, yMin, xMax, yMax int, ok bool) {
for _, c := range contours {
for _, p := range c {
x, y := roundInt(p.x), roundInt(p.y)
if !ok {
xMin, yMin, xMax, yMax, ok = x, y, x, y, true
continue
}
xMin, yMin = min(xMin, x), min(yMin, y)
xMax, yMax = max(xMax, x), max(yMax, y)
}
}
return xMin, yMin, xMax, yMax, ok
}
// encodeSimpleGlyph re-encodes instanced contours as a TrueType simple glyph, with
// coordinates rounded to integers and no instructions. It returns nil (an empty
// glyph) when the contours hold no points. Off-curve (quadratic control) points
// are preserved, so the re-encoded glyph re-parses to the same outline.
func encodeSimpleGlyph(contours []contour) []byte {
var pts []outlinePoint
var ends []int
for _, c := range contours {
if len(c) == 0 {
continue
}
pts = append(pts, c...)
ends = append(ends, len(pts)-1)
}
if len(pts) == 0 {
return nil
}
xMin, yMin, xMax, yMax, _ := contourBounds(contours)
w := &glyfWriter{}
w.i16(int16(len(ends)))
w.i16(int16(xMin))
w.i16(int16(yMin))
w.i16(int16(xMax))
w.i16(int16(yMax))
for _, e := range ends {
w.u16(uint16(e))
}
w.u16(0) // instructionLength
var flags, xs, ys []byte
px, py := 0, 0
for _, p := range pts {
x, y := roundInt(p.x), roundInt(p.y)
var fl byte
if p.on {
fl |= flagOnCurve
}
fx, xb := encodeGlyfDelta(x-px, flagXShort, flagXSamePosX)
fy, yb := encodeGlyfDelta(y-py, flagYShort, flagYSamePosY)
fl |= fx | fy
flags = append(flags, fl)
xs = append(xs, xb...)
ys = append(ys, yb...)
px, py = x, y
}
w.b = append(w.b, flags...)
w.b = append(w.b, xs...)
w.b = append(w.b, ys...)
return w.b
}
// encodeGlyfDelta encodes one coordinate delta with the short/same/long flag
// scheme readCoords decodes: zero deltas set the "same" bit and emit no bytes,
// deltas in [-255,255] use one byte with a sign carried by the short/positive
// bits, and larger deltas fall back to a signed 16-bit value.
func encodeGlyfDelta(delta int, shortBit, samePosBit byte) (flag byte, data []byte) {
switch {
case delta == 0:
return samePosBit, nil
case delta > 0 && delta <= 255:
return shortBit | samePosBit, []byte{byte(delta)}
case delta < 0 && delta >= -255:
return shortBit, []byte{byte(-delta)}
default:
d := int16(delta)
return 0, []byte{byte(uint16(d) >> 8), byte(d)}
}
}
vendor/github.com/go-opentype/opentype/kern.go
Generated Vendored
+81
View File
@@ -0,0 +1,81 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import "fmt"
// This file parses the legacy 'kern' table in its Microsoft/OpenType form
// (table version 0), decoding only format-0 subtables: a flat list of
// (left, right) glyph pairs each with a signed kerning value in font units.
// Non-format-0 subtables are skipped by their declared length. The Apple
// version-1 ('kern' with a 32-bit version) header is not handled.
// kern is a decoded legacy kern table: kerning values keyed by the packed
// (left, right) glyph pair.
type kern struct {
pairs map[uint32]int
}
// kernKey packs a glyph pair into the map key.
func kernKey(left, right GlyphIndex) uint32 {
return uint32(left)<<16 | uint32(right)
}
// parseKern decodes a legacy kern table (version 0).
func parseKern(b []byte) (*kern, error) {
r := reader{b: b}
r.skip(2) // version (0)
nTables := int(r.u16())
if r.err != nil {
return nil, fmt.Errorf("opentype: kern header: %w", r.err)
}
k := &kern{pairs: map[uint32]int{}}
for t := 0; t < nTables; t++ {
stStart := r.pos
r.skip(2) // subtable version
length := int(r.u16())
coverage := r.u16()
if r.err != nil {
return nil, fmt.Errorf("opentype: kern subtable header: %w", r.err)
}
if coverage>>8 == 0 { // format is the high byte of the coverage field
if err := k.parseFormat0(&r); err != nil {
return nil, err
}
}
// Advance to the next subtable using its declared length. For format 0
// this matches where the sequential read left off; for skipped formats
// it steps over the unread body.
next := stStart + length
if next > len(r.b) {
return nil, fmt.Errorf("opentype: kern subtable length: %w", errTruncated)
}
r.pos = next
}
return k, nil
}
// parseFormat0 reads a format-0 subtable body (positioned just after the 6-byte
// subtable header) into the pair map.
func (k *kern) parseFormat0(r *reader) error {
nPairs := int(r.u16())
r.skip(6) // searchRange, entrySelector, rangeShift
for i := 0; i < nPairs; i++ {
left := GlyphIndex(r.u16())
right := GlyphIndex(r.u16())
value := int(r.i16())
k.pairs[kernKey(left, right)] = value
}
if r.err != nil {
return fmt.Errorf("opentype: kern format 0: %w", r.err)
}
return nil
}
// Pair returns the legacy kerning adjustment between left and right in font
// units, or 0 when the pair is absent.
func (k *kern) Pair(left, right GlyphIndex) int {
return k.pairs[kernKey(left, right)]
}
vendor/github.com/go-opentype/opentype/layout.go
Generated Vendored
+376
View File
@@ -0,0 +1,376 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import (
"fmt"
"sort"
)
// This file parses the structures shared by the OpenType Layout tables GSUB
// and GPOS: the ScriptList, FeatureList and LookupList, plus the Coverage and
// ClassDef tables their subtables point at. The lookup subtables themselves are
// table-specific and decoded in gsub.go and gpos.go.
//
// All offsets inside a Layout table are unsigned 16-bit values measured from
// the start of the structure that contains them, so parsing proceeds by
// re-slicing at each offset with subslice and recursing.
// subslice returns b[off:], or nil when off is past the end of b. A nil result
// makes any subsequent bounded read record errTruncated, so callers can defer
// the error check to a single site.
func subslice(b []byte, off int) []byte {
if off > len(b) {
return nil
}
return b[off:]
}
// tag reads a 4-byte OpenType tag as a string, recording errTruncated when the
// four bytes are not available.
func (r *reader) tag() string {
if r.pos+4 > len(r.b) {
r.err = errTruncated
return ""
}
s := string(r.b[r.pos : r.pos+4])
r.pos += 4
return s
}
// langSys is a decoded LangSys table: the (optional) required feature index and
// the list of feature indices enabled for a script/language.
type langSys struct {
required uint16 // 0xFFFF means "no required feature"
features []uint16
}
// langSysRecord binds a 4-byte language tag to its LangSys table.
type langSysRecord struct {
tag string
ls *langSys
}
// scriptRecord is a decoded Script table: its default LangSys (may be nil) and
// any language-specific LangSys tables, keyed by tag.
type scriptRecord struct {
tag string
defaultLang *langSys
langs []langSysRecord
}
// featureRecord is a decoded Feature table: the lookup indices it activates.
type featureRecord struct {
tag string
lookups []uint16
}
// layoutHeader holds the script and feature tables shared by a GSUB or GPOS
// table. It is embedded into gsub and gpos to provide selectLookups.
type layoutHeader struct {
scripts []scriptRecord
features []featureRecord
}
// parseLayoutCommon decodes the version header of a GSUB/GPOS table and its
// ScriptList, FeatureList and LookupList, returning the shared header plus the
// raw bytes of each Lookup table (decoded by the caller per lookup type).
func parseLayoutCommon(data []byte) (layoutHeader, [][]byte, error) {
r := reader{b: data}
r.skip(4) // majorVersion, minorVersion (1.1's featureVariations is ignored)
scriptOff := int(r.u16())
featureOff := int(r.u16())
lookupOff := int(r.u16())
if r.err != nil {
return layoutHeader{}, nil, fmt.Errorf("opentype: layout header: %w", r.err)
}
scripts, err := parseScriptList(subslice(data, scriptOff))
if err != nil {
return layoutHeader{}, nil, err
}
features, err := parseFeatureList(subslice(data, featureOff))
if err != nil {
return layoutHeader{}, nil, err
}
lookups, err := parseLookupList(subslice(data, lookupOff))
if err != nil {
return layoutHeader{}, nil, err
}
return layoutHeader{scripts: scripts, features: features}, lookups, nil
}
// parseScriptList decodes a ScriptList table.
func parseScriptList(b []byte) ([]scriptRecord, error) {
r := reader{b: b}
count := int(r.u16())
type rec struct {
tag string
off int
}
recs := make([]rec, count)
for i := 0; i < count; i++ {
recs[i].tag = r.tag()
recs[i].off = int(r.u16())
}
if r.err != nil {
return nil, fmt.Errorf("opentype: scriptList: %w", r.err)
}
out := make([]scriptRecord, count)
for i, rc := range recs {
sc, err := parseScript(subslice(b, rc.off))
if err != nil {
return nil, err
}
sc.tag = rc.tag
out[i] = sc
}
return out, nil
}
// parseScript decodes a Script table (its default and language LangSys tables).
func parseScript(b []byte) (scriptRecord, error) {
r := reader{b: b}
defOff := int(r.u16())
langCount := int(r.u16())
type lr struct {
tag string
off int
}
lrs := make([]lr, langCount)
for i := 0; i < langCount; i++ {
lrs[i].tag = r.tag()
lrs[i].off = int(r.u16())
}
if r.err != nil {
return scriptRecord{}, fmt.Errorf("opentype: script: %w", r.err)
}
var sc scriptRecord
if defOff != 0 {
ls, err := parseLangSys(subslice(b, defOff))
if err != nil {
return scriptRecord{}, err
}
sc.defaultLang = ls
}
sc.langs = make([]langSysRecord, langCount)
for i, l := range lrs {
ls, err := parseLangSys(subslice(b, l.off))
if err != nil {
return scriptRecord{}, err
}
sc.langs[i] = langSysRecord{tag: l.tag, ls: ls}
}
return sc, nil
}
// parseLangSys decodes a LangSys table.
func parseLangSys(b []byte) (*langSys, error) {
r := reader{b: b}
r.skip(2) // lookupOrderOffset (reserved, always 0)
required := r.u16()
n := int(r.u16())
feats := make([]uint16, n)
for i := 0; i < n; i++ {
feats[i] = r.u16()
}
if r.err != nil {
return nil, fmt.Errorf("opentype: langSys: %w", r.err)
}
return &langSys{required: required, features: feats}, nil
}
// parseFeatureList decodes a FeatureList table.
func parseFeatureList(b []byte) ([]featureRecord, error) {
r := reader{b: b}
count := int(r.u16())
type rec struct {
tag string
off int
}
recs := make([]rec, count)
for i := 0; i < count; i++ {
recs[i].tag = r.tag()
recs[i].off = int(r.u16())
}
if r.err != nil {
return nil, fmt.Errorf("opentype: featureList: %w", r.err)
}
out := make([]featureRecord, count)
for i, rc := range recs {
fr, err := parseFeature(subslice(b, rc.off))
if err != nil {
return nil, err
}
fr.tag = rc.tag
out[i] = fr
}
return out, nil
}
// parseFeature decodes a Feature table (its lookup index list).
func parseFeature(b []byte) (featureRecord, error) {
r := reader{b: b}
r.skip(2) // featureParamsOffset (unused here)
n := int(r.u16())
idx := make([]uint16, n)
for i := 0; i < n; i++ {
idx[i] = r.u16()
}
if r.err != nil {
return featureRecord{}, fmt.Errorf("opentype: feature: %w", r.err)
}
return featureRecord{lookups: idx}, nil
}
// parseLookupList decodes a LookupList table into the raw bytes of each Lookup
// table. Type-specific decoding happens in gsub.go and gpos.go.
func parseLookupList(b []byte) ([][]byte, error) {
r := reader{b: b}
count := int(r.u16())
offs := make([]int, count)
for i := 0; i < count; i++ {
offs[i] = int(r.u16())
}
if r.err != nil {
return nil, fmt.Errorf("opentype: lookupList: %w", r.err)
}
out := make([][]byte, count)
for i, off := range offs {
out[i] = subslice(b, off)
}
return out, nil
}
// parseCoverage decodes a Coverage table (format 1 or 2) into a map from glyph
// to its coverage index.
func parseCoverage(b []byte) (map[GlyphIndex]int, error) {
r := reader{b: b}
format := r.u16()
m := map[GlyphIndex]int{}
switch format {
case 1:
n := int(r.u16())
for i := 0; i < n; i++ {
m[GlyphIndex(r.u16())] = i
}
case 2:
n := int(r.u16())
for i := 0; i < n; i++ {
start := int(r.u16())
end := int(r.u16())
startCov := int(r.u16())
for g := start; g <= end; g++ {
m[GlyphIndex(g)] = startCov + (g - start)
}
}
default:
return nil, fmt.Errorf("opentype: coverage format %d", format)
}
if r.err != nil {
return nil, fmt.Errorf("opentype: coverage: %w", r.err)
}
return m, nil
}
// parseClassDef decodes a ClassDef table (format 1 or 2) into a map from glyph
// to class. Glyphs absent from the map belong to class 0.
func parseClassDef(b []byte) (map[GlyphIndex]int, error) {
r := reader{b: b}
format := r.u16()
m := map[GlyphIndex]int{}
switch format {
case 1:
start := int(r.u16())
n := int(r.u16())
for i := 0; i < n; i++ {
m[GlyphIndex(start+i)] = int(r.u16())
}
case 2:
n := int(r.u16())
for i := 0; i < n; i++ {
s := int(r.u16())
e := int(r.u16())
c := int(r.u16())
for g := s; g <= e; g++ {
m[GlyphIndex(g)] = c
}
}
default:
return nil, fmt.Errorf("opentype: classdef format %d", format)
}
if r.err != nil {
return nil, fmt.Errorf("opentype: classdef: %w", r.err)
}
return m, nil
}
// findScript resolves a script tag, falling back to "DFLT" and then to the
// first script present, matching the behaviour of common shapers.
func (h *layoutHeader) findScript(tag string) *scriptRecord {
for i := range h.scripts {
if h.scripts[i].tag == tag {
return &h.scripts[i]
}
}
for i := range h.scripts {
if h.scripts[i].tag == "DFLT" {
return &h.scripts[i]
}
}
if len(h.scripts) > 0 {
return &h.scripts[0]
}
return nil
}
// findLang resolves a language tag within a script, falling back to the script's
// default LangSys.
func (s *scriptRecord) findLang(tag string) *langSys {
for i := range s.langs {
if s.langs[i].tag == tag {
return s.langs[i].ls
}
}
return s.defaultLang
}
// selectLookups returns the lookup indices activated by the wanted feature tags
// for a (script, language) pair, in ascending lookup order and de-duplicated.
// The required feature (if any) is included when its tag is wanted.
func (h *layoutHeader) selectLookups(script, lang string, want map[string]bool) []int {
sr := h.findScript(script)
if sr == nil {
return nil
}
ls := sr.findLang(lang)
if ls == nil {
return nil
}
seen := map[int]bool{}
var out []int
add := func(featIdx uint16) {
if int(featIdx) >= len(h.features) {
return
}
fr := h.features[featIdx]
if !want[fr.tag] {
return
}
for _, li := range fr.lookups {
if !seen[int(li)] {
seen[int(li)] = true
out = append(out, int(li))
}
}
}
if ls.required != 0xFFFF {
add(ls.required)
}
for _, fi := range ls.features {
add(fi)
}
sort.Ints(out)
return out
}
vendor/github.com/go-opentype/opentype/math.go
Generated Vendored
+674
View File
@@ -0,0 +1,674 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import (
"fmt"
"sort"
)
// This file decodes the OpenType MATH table: the font-level metrics a math
// typesetter needs. It is the layer LuaTeX, unicode-math, MathJax and Word's
// equation editor build their math layout on. The MATH table has three parts:
//
// - MathConstants: ~56 global scalars (subscript/superscript shifts, fraction
// and radical gaps, rule thicknesses, the math axis height, …).
// - MathGlyphInfo: per-glyph italic correction, top-accent attachment points,
// an "extended shape" set and cut-in math kerning for the four glyph corners.
// - MathVariants: larger size variants of a glyph plus the recipe (a
// GlyphAssembly of repeatable parts) for building an arbitrarily tall or wide
// stretchy glyph (integrals, braces, radicals, arrows).
//
// This package exposes those metrics, pixel-scaled through a Face. The actual
// math LAYOUT — box building, TeX's math list rules, choosing a variant or
// assembling a stretchy glyph at a target size — is a higher-level engine that
// consumes these values; it is out of scope here.
//
// All offsets inside the table are unsigned 16-bit values measured from the
// start of the structure that contains them, so parsing re-slices with subslice
// and recurses, exactly as gsub.go/gpos.go/layout.go do.
// MathConstant identifies one scalar in the MATH table's MathConstants record.
// The constants are listed in their on-disk order. Face.MathConstant returns
// each pixel-scaled, except the three percentage constants
// (ScriptPercentScaleDown, ScriptScriptPercentScaleDown and
// RadicalDegreeBottomRaisePercent), which are pure ratios and returned verbatim.
type MathConstant int
// The MathConstants record fields, in table order. mathConstCount is the count
// sentinel (not a real constant).
const (
ScriptPercentScaleDown MathConstant = iota
ScriptScriptPercentScaleDown
DelimitedSubFormulaMinHeight
DisplayOperatorMinHeight
MathLeading
AxisHeight
AccentBaseHeight
FlattenedAccentBaseHeight
SubscriptShiftDown
SubscriptTopMax
SubscriptBaselineDropMin
SuperscriptShiftUp
SuperscriptShiftUpCramped
SuperscriptBottomMin
SuperscriptBaselineDropMax
SubSuperscriptGapMin
SuperscriptBottomMaxWithSubscript
SpaceAfterScript
UpperLimitGapMin
UpperLimitBaselineRiseMin
LowerLimitGapMin
LowerLimitBaselineDropMin
StackTopShiftUp
StackTopDisplayStyleShiftUp
StackBottomShiftDown
StackBottomDisplayStyleShiftDown
StackGapMin
StackDisplayStyleGapMin
StretchStackTopShiftUp
StretchStackBottomShiftDown
StretchStackGapAboveMin
StretchStackGapBelowMin
FractionNumeratorShiftUp
FractionNumeratorDisplayStyleShiftUp
FractionDenominatorShiftDown
FractionDenominatorDisplayStyleShiftDown
FractionNumeratorGapMin
FractionNumDisplayStyleGapMin
FractionRuleThickness
FractionDenominatorGapMin
FractionDenomDisplayStyleGapMin
SkewedFractionHorizontalGap
SkewedFractionVerticalGap
OverbarVerticalGap
OverbarRuleThickness
OverbarExtraAscender
UnderbarVerticalGap
UnderbarRuleThickness
UnderbarExtraDescender
RadicalVerticalGap
RadicalDisplayStyleVerticalGap
RadicalRuleThickness
RadicalExtraAscender
RadicalKernBeforeDegree
RadicalKernAfterDegree
RadicalDegreeBottomRaisePercent
mathConstCount
)
// isPercent reports whether c is a percentage ratio rather than a design-unit
// length; such a constant is returned unscaled.
func (c MathConstant) isPercent() bool {
return c == ScriptPercentScaleDown ||
c == ScriptScriptPercentScaleDown ||
c == RadicalDegreeBottomRaisePercent
}
// MathKernCorner selects one of a glyph's four math-kern corners, used for the
// staircase cut-in between a base glyph and an adjacent script.
type MathKernCorner int
// The four math-kern corners, in the on-disk order of a MathKernInfoRecord.
const (
MathKernTopRight MathKernCorner = iota
MathKernTopLeft
MathKernBottomRight
MathKernBottomLeft
)
// partFlagExtender marks a GlyphPart that may be repeated to lengthen a stretchy
// glyph assembly (PART_FLAG_EXTENDER).
const partFlagExtender = 0x0001
// MathVariant is one size variant of a stretchy glyph: the variant's glyph and
// its advance (height for a vertical variant, width for a horizontal one) in
// pixels at the Face's size.
type MathVariant struct {
Glyph GlyphIndex
Advance int
}
// MathAssembly is the recipe for building a stretchy glyph out of repeatable
// parts, with all measurements in pixels at the Face's size. A layout engine
// stacks the parts (vertically or horizontally), overlapping neighbours by at
// least MinConnectorOverlap and repeating the Extender parts until the assembly
// reaches the target size.
type MathAssembly struct {
ItalicsCorrection int
MinConnectorOverlap int
Parts []MathAssemblyPart
}
// MathAssemblyPart is one component of a MathAssembly. StartConnector and
// EndConnector are the maximum overlap lengths at the part's leading and
// trailing ends; FullAdvance is the part's full extent along the stretch axis.
type MathAssemblyPart struct {
Glyph GlyphIndex
StartConnector int
EndConnector int
FullAdvance int
Extender bool
}
// mathTable is a decoded MATH table. Every sub-part is optional; a font may
// carry constants only, glyph info only, variants only, or any combination.
type mathTable struct {
constants [mathConstCount]int32 // design-unit or percent scalars, indexed by MathConstant
hasConstants bool
italicCorr map[GlyphIndex]int32 // MathItalicsCorrectionInfo: per-glyph italic correction
topAccent map[GlyphIndex]int32 // MathTopAccentAttachment: per-glyph top-accent x
extendedShape map[GlyphIndex]int // ExtendedShapeCoverage, used as a set
kern map[GlyphIndex]*mathKernRecord
minConnectorOverlap int32
vertConstruct map[GlyphIndex]*mathGlyphConstruction // vertical (tall) stretch
horizConstruct map[GlyphIndex]*mathGlyphConstruction // horizontal (wide) stretch
}
// mathKernRecord holds a glyph's four corner MathKerns (any may be nil).
type mathKernRecord struct {
corners [4]*mathKern
}
// mathKern is a step function of correction height: heights has n entries and
// kerns has n+1, so a height selects the kern in the interval it falls into.
type mathKern struct {
heights []int32 // ascending correctionHeight values (design units)
kerns []int32 // kern values (design units), one more than heights
}
// valueAt returns the kern for correction height h (design units) by locating
// the interval h falls into: the first heights[i] strictly greater than h, or
// the final kern when h is at or above every height.
func (k *mathKern) valueAt(h int32) int32 {
i := sort.Search(len(k.heights), func(i int) bool { return h < k.heights[i] })
return k.kerns[i]
}
// mathGlyphConstruction lists a glyph's size variants and, optionally, the
// assembly recipe for an arbitrarily stretched glyph.
type mathGlyphConstruction struct {
variants []mathVariantRecord
assembly *mathAssemblyData
}
// mathVariantRecord is one MathGlyphVariantRecord (design units).
type mathVariantRecord struct {
glyph GlyphIndex
advance int32
}
// mathAssemblyData is a decoded GlyphAssembly (design units).
type mathAssemblyData struct {
italicsCorrection int32
parts []mathPart
}
// mathPart is one GlyphPart of an assembly (design units).
type mathPart struct {
glyph GlyphIndex
startConnector int32
endConnector int32
fullAdvance int32
extender bool
}
// mathValue reads a MathValueRecord: an int16 design-unit value followed by an
// Offset16 to an optional Device table, which this package tolerates and
// ignores (device deltas fine-tune a value at a specific ppem; a scalable
// renderer does not need them).
func (r *reader) mathValue() int32 {
v := r.i16()
r.skip(2) // deviceOffset, ignored
return int32(v)
}
// parseMath decodes the optional MATH table and stores it on the font. Absence
// is not an error; a malformed table fails the whole parse so a corrupt font is
// rejected cleanly, matching the other layout tables.
func (f *Font) parseMath(tables map[string][]byte) error {
b, ok := tables["MATH"]
if !ok {
return nil
}
m, err := parseMATH(b)
if err != nil {
return err
}
f.math = m
return nil
}
// parseMATH decodes a MATH table: a version header followed by up to three
// independent sub-offsets.
func parseMATH(b []byte) (*mathTable, error) {
r := reader{b: b}
r.skip(4) // majorVersion, minorVersion
constOff := int(r.u16())
glyphInfoOff := int(r.u16())
variantsOff := int(r.u16())
if r.err != nil {
return nil, fmt.Errorf("opentype: MATH header: %w", r.err)
}
m := &mathTable{}
if constOff != 0 {
if err := m.parseConstants(subslice(b, constOff)); err != nil {
return nil, err
}
}
if glyphInfoOff != 0 {
if err := m.parseGlyphInfo(subslice(b, glyphInfoOff)); err != nil {
return nil, err
}
}
if variantsOff != 0 {
if err := m.parseVariants(subslice(b, variantsOff)); err != nil {
return nil, err
}
}
return m, nil
}
// parseConstants decodes the MathConstants record into the constants array. The
// layout is two int16 percentages, two UFWORD minimum heights, 51 MathValue
// records and a trailing int16 percentage.
func (m *mathTable) parseConstants(b []byte) error {
r := reader{b: b}
var c [mathConstCount]int32
c[ScriptPercentScaleDown] = int32(r.i16())
c[ScriptScriptPercentScaleDown] = int32(r.i16())
c[DelimitedSubFormulaMinHeight] = int32(r.u16())
c[DisplayOperatorMinHeight] = int32(r.u16())
for i := MathLeading; i <= RadicalKernAfterDegree; i++ {
c[i] = r.mathValue()
}
c[RadicalDegreeBottomRaisePercent] = int32(r.i16())
if r.err != nil {
return fmt.Errorf("opentype: MATH constants: %w", r.err)
}
m.constants = c
m.hasConstants = true
return nil
}
// parseGlyphInfo decodes the MathGlyphInfo table: four optional sub-offsets.
func (m *mathTable) parseGlyphInfo(b []byte) error {
r := reader{b: b}
italicOff := int(r.u16())
topAccentOff := int(r.u16())
extShapeOff := int(r.u16())
kernOff := int(r.u16())
if r.err != nil {
return fmt.Errorf("opentype: MATH glyphInfo: %w", r.err)
}
if italicOff != 0 {
vals, err := parsePerGlyphValues(subslice(b, italicOff))
if err != nil {
return err
}
m.italicCorr = vals
}
if topAccentOff != 0 {
vals, err := parsePerGlyphValues(subslice(b, topAccentOff))
if err != nil {
return err
}
m.topAccent = vals
}
if extShapeOff != 0 {
cov, err := parseCoverage(subslice(b, extShapeOff))
if err != nil {
return err
}
m.extendedShape = cov
}
if kernOff != 0 {
k, err := parseMathKernInfo(subslice(b, kernOff))
if err != nil {
return err
}
m.kern = k
}
return nil
}
// parsePerGlyphValues decodes the shape shared by MathItalicsCorrectionInfo and
// MathTopAccentAttachment: a Coverage offset, a count and that many MathValue
// records, joined into a per-glyph map.
func parsePerGlyphValues(b []byte) (map[GlyphIndex]int32, error) {
r := reader{b: b}
covOff := int(r.u16())
count := int(r.u16())
vals := make([]int32, count)
for i := range vals {
vals[i] = r.mathValue()
}
if r.err != nil {
return nil, fmt.Errorf("opentype: MATH per-glyph values: %w", r.err)
}
cov, err := parseCoverage(subslice(b, covOff))
if err != nil {
return nil, err
}
return coverageValues(cov, vals), nil
}
// coverageValues joins a Coverage (glyph→index) with a parallel value slice into
// a glyph→value map.
func coverageValues(cov map[GlyphIndex]int, vals []int32) map[GlyphIndex]int32 {
out := make(map[GlyphIndex]int32, len(cov))
for g, i := range cov {
if i < len(vals) {
out[g] = vals[i]
}
}
return out
}
// parseMathKernInfo decodes the MathKernInfo table: a Coverage plus one
// MathKernInfoRecord (four corner offsets) per covered glyph.
func parseMathKernInfo(b []byte) (map[GlyphIndex]*mathKernRecord, error) {
r := reader{b: b}
covOff := int(r.u16())
count := int(r.u16())
offs := make([][4]int, count)
for i := range offs {
for c := 0; c < 4; c++ {
offs[i][c] = int(r.u16())
}
}
if r.err != nil {
return nil, fmt.Errorf("opentype: MATH kernInfo: %w", r.err)
}
recs := make([]*mathKernRecord, count)
for i := range offs {
rec := &mathKernRecord{}
for c := 0; c < 4; c++ {
if offs[i][c] == 0 {
continue // this corner has no math kern
}
k, err := parseMathKern(subslice(b, offs[i][c]))
if err != nil {
return nil, err
}
rec.corners[c] = k
}
recs[i] = rec
}
cov, err := parseCoverage(subslice(b, covOff))
if err != nil {
return nil, err
}
out := make(map[GlyphIndex]*mathKernRecord, len(cov))
for g, i := range cov {
if i < len(recs) {
out[g] = recs[i]
}
}
return out, nil
}
// parseMathKern decodes a MathKern subtable: heightCount correction heights and
// heightCount+1 kern values.
func parseMathKern(b []byte) (*mathKern, error) {
r := reader{b: b}
n := int(r.u16())
heights := make([]int32, n)
for i := range heights {
heights[i] = r.mathValue()
}
kerns := make([]int32, n+1)
for i := range kerns {
kerns[i] = r.mathValue()
}
if r.err != nil {
return nil, fmt.Errorf("opentype: MATH kern: %w", r.err)
}
return &mathKern{heights: heights, kerns: kerns}, nil
}
// parseVariants decodes the MathVariants table: the minimum connector overlap,
// two Coverage offsets and the vertical and horizontal glyph constructions.
func (m *mathTable) parseVariants(b []byte) error {
r := reader{b: b}
m.minConnectorOverlap = int32(r.u16())
vertCovOff := int(r.u16())
horizCovOff := int(r.u16())
vertCount := int(r.u16())
horizCount := int(r.u16())
vertOffs := make([]int, vertCount)
for i := range vertOffs {
vertOffs[i] = int(r.u16())
}
horizOffs := make([]int, horizCount)
for i := range horizOffs {
horizOffs[i] = int(r.u16())
}
if r.err != nil {
return fmt.Errorf("opentype: MATH variants: %w", r.err)
}
vert, err := buildConstructions(b, vertCovOff, vertOffs)
if err != nil {
return err
}
horiz, err := buildConstructions(b, horizCovOff, horizOffs)
if err != nil {
return err
}
m.vertConstruct = vert
m.horizConstruct = horiz
return nil
}
// buildConstructions decodes each MathGlyphConstruction at offs and keys them by
// the glyphs in the Coverage at covOff. A zero covOff (an empty axis) yields an
// empty map without touching the coverage bytes.
func buildConstructions(b []byte, covOff int, offs []int) (map[GlyphIndex]*mathGlyphConstruction, error) {
out := map[GlyphIndex]*mathGlyphConstruction{}
if covOff == 0 {
return out, nil
}
constructs := make([]*mathGlyphConstruction, len(offs))
for i, off := range offs {
gc, err := parseGlyphConstruction(subslice(b, off))
if err != nil {
return nil, err
}
constructs[i] = gc
}
cov, err := parseCoverage(subslice(b, covOff))
if err != nil {
return nil, err
}
for g, i := range cov {
if i < len(constructs) {
out[g] = constructs[i]
}
}
return out, nil
}
// parseGlyphConstruction decodes a MathGlyphConstruction: an optional
// GlyphAssembly offset and a list of MathGlyphVariantRecords.
func parseGlyphConstruction(b []byte) (*mathGlyphConstruction, error) {
r := reader{b: b}
asmOff := int(r.u16())
variantCount := int(r.u16())
variants := make([]mathVariantRecord, variantCount)
for i := range variants {
variants[i].glyph = GlyphIndex(r.u16())
variants[i].advance = int32(r.u16())
}
if r.err != nil {
return nil, fmt.Errorf("opentype: MATH glyph construction: %w", r.err)
}
gc := &mathGlyphConstruction{variants: variants}
if asmOff != 0 {
asm, err := parseGlyphAssembly(subslice(b, asmOff))
if err != nil {
return nil, err
}
gc.assembly = asm
}
return gc, nil
}
// parseGlyphAssembly decodes a GlyphAssembly: an italics-correction MathValue
// and a list of GlyphParts.
func parseGlyphAssembly(b []byte) (*mathAssemblyData, error) {
r := reader{b: b}
italic := r.mathValue()
partCount := int(r.u16())
parts := make([]mathPart, partCount)
for i := range parts {
parts[i].glyph = GlyphIndex(r.u16())
parts[i].startConnector = int32(r.u16())
parts[i].endConnector = int32(r.u16())
parts[i].fullAdvance = int32(r.u16())
parts[i].extender = r.u16()&partFlagExtender != 0
}
if r.err != nil {
return nil, fmt.Errorf("opentype: MATH glyph assembly: %w", r.err)
}
return &mathAssemblyData{italicsCorrection: italic, parts: parts}, nil
}
// --- public API -------------------------------------------------------------
// HasMath reports whether the font carries an OpenType MATH table (the metrics
// a math-typesetting engine needs). When false, all Face math accessors return
// their zero result.
func (f *Font) HasMath() bool { return f.math != nil }
// IsExtendedShapeGlyph reports whether gid is in the MATH table's extended-shape
// coverage: a tall glyph (such as a large integral) whose superscript is
// positioned as if the glyph were a stretched, rather than a fixed-size, shape.
func (f *Font) IsExtendedShapeGlyph(gid GlyphIndex) bool {
if f.math == nil {
return false
}
_, ok := f.math.extendedShape[gid]
return ok
}
// scaleUnit converts a design-unit value to whole pixels at the face's size.
func (fc *Face) scaleUnit(v int32) int { return roundInt(float64(v) * fc.scale) }
// MathConstant returns the MathConstants value which, pixel-scaled at the face's
// size — except the three percentage constants, which are pure ratios and
// returned verbatim. It returns 0 when the font has no MATH constants or which
// is out of range.
func (fc *Face) MathConstant(which MathConstant) int {
m := fc.font.math
if m == nil || !m.hasConstants || which < 0 || which >= mathConstCount {
return 0
}
v := m.constants[which]
if which.isPercent() {
return int(v)
}
return fc.scaleUnit(v)
}
// ItalicCorrection returns glyph gid's math italic correction in pixels, or 0
// when the glyph has none (or the font carries no MATH table). It is the space
// to insert after a slanted glyph before a following upright one, and the
// horizontal offset for a subscript that follows a superscript.
func (fc *Face) ItalicCorrection(gid GlyphIndex) int {
m := fc.font.math
if m == nil {
return 0
}
return fc.scaleUnit(m.italicCorr[gid])
}
// TopAccentAttachment returns the x position (in pixels from the glyph origin)
// at which a top accent should be centred over glyph gid. ok is false when the
// glyph has no entry, in which case a layout engine centres on the glyph's
// advance instead.
func (fc *Face) TopAccentAttachment(gid GlyphIndex) (int, bool) {
m := fc.font.math
if m == nil {
return 0, false
}
v, ok := m.topAccent[gid]
if !ok {
return 0, false
}
return fc.scaleUnit(v), true
}
// MathKern returns the math cut-in kern (in pixels) for glyph gid at the given
// corner and correction height, the height (in pixels above the baseline) at
// which the adjacent script attaches. It returns 0 when the glyph has no kern
// for that corner or the font carries no MATH table.
func (fc *Face) MathKern(gid GlyphIndex, corner MathKernCorner, correctionHeight int) int {
m := fc.font.math
if m == nil {
return 0
}
rec := m.kern[gid]
if rec == nil {
return 0
}
if corner < 0 || corner > MathKernBottomLeft {
return 0
}
k := rec.corners[corner]
if k == nil {
return 0
}
hUnits := int32(roundInt(float64(correctionHeight) / fc.scale))
return fc.scaleUnit(k.valueAt(hUnits))
}
// MathVariants returns the size variants and the stretchy-glyph assembly for
// glyph gid along the requested axis (vertical when vertical is true, otherwise
// horizontal), with every measurement pixel-scaled at the face's size. Variants
// are ordered smallest-first as stored in the font; the assembly is nil when the
// glyph has no assembly recipe. Both results are nil when the glyph has no
// construction on that axis (or the font carries no MATH table). A math layout
// engine picks the first variant whose advance meets the target size, or builds
// the assembly when even the largest variant is too small.
func (fc *Face) MathVariants(gid GlyphIndex, vertical bool) ([]MathVariant, *MathAssembly) {
m := fc.font.math
if m == nil {
return nil, nil
}
table := m.horizConstruct
if vertical {
table = m.vertConstruct
}
gc := table[gid]
if gc == nil {
return nil, nil
}
var variants []MathVariant
for _, v := range gc.variants {
variants = append(variants, MathVariant{Glyph: v.glyph, Advance: fc.scaleUnit(v.advance)})
}
var asm *MathAssembly
if gc.assembly != nil {
a := gc.assembly
asm = &MathAssembly{
ItalicsCorrection: fc.scaleUnit(a.italicsCorrection),
MinConnectorOverlap: fc.scaleUnit(m.minConnectorOverlap),
Parts: make([]MathAssemblyPart, len(a.parts)),
}
for i, p := range a.parts {
asm.Parts[i] = MathAssemblyPart{
Glyph: p.glyph,
StartConnector: fc.scaleUnit(p.startConnector),
EndConnector: fc.scaleUnit(p.endConnector),
FullAdvance: fc.scaleUnit(p.fullAdvance),
Extender: p.extender,
}
}
}
return variants, asm
}
vendor/github.com/go-opentype/opentype/metricvar.go
Generated Vendored
+465
View File
@@ -0,0 +1,465 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import "fmt"
// This file implements the optional metric-variation tables HVAR, VVAR and
// MVAR, so a variable font's advance widths, advance heights and global font
// metrics track the axis coordinate rather than staying at the default master.
//
// - HVAR (Horizontal Metrics Variations): per-glyph horizontal-advance deltas
// (plus optional left/right side-bearing maps), applied by Face.Advance,
// Face.Measure and the GlyphMask family.
// - VVAR (Vertical Metrics Variations): the vertical analogue, per-glyph
// advance-height deltas, applied by Face.VerticalAdvance.
// - MVAR (Metrics Variations): global font-metric deltas keyed by four-byte
// value tags (hasc/hdsc/hlgp/...), folded into Face.Metrics.
//
// All three share one machinery: an ItemVariationStore (the same format 1 store
// CFF2 references, but here the delta sets are fully decoded because the deltas
// come from the store rather than from a charstring) plus, per metric, an
// optional DeltaSetIndexMap translating a glyph index into an (outer, inner)
// pair addressing the store. The tent-function region evaluator supportScalar
// (gvar.go) is reused unchanged.
//
// When a variable glyf font ships no HVAR table, horizontal advances fall back
// to the per-glyph gvar phantom-point deltas the specification prescribes (see
// gvarAdvanceWidthDelta); this fallback is implemented for simple glyphs.
// Composite-glyph advance fallback is out of scope (it returns no delta).
// metricVarData is one ItemVariationData subtable: the ordered region indices
// its columns correspond to, and one fully-decoded delta row per item.
type metricVarData struct {
regionIndices []int
deltaSets [][]int32 // itemCount rows, each len(regionIndices) deltas
}
// metricVarStore is a format-1 ItemVariationStore with its delta sets decoded.
// Each region is kept as parallel per-axis lower/peak/upper coordinate slices so
// the shared supportScalar evaluator can be called without repacking.
type metricVarStore struct {
axisCount int
regLower [][]float64
regPeak [][]float64
regUpper [][]float64
data []metricVarData
}
// delta returns the interpolated delta for item (outer, inner) at the normalized
// coordinate norm (F2Dot14 int16 per axis, as produced by NormalizeCoords): the
// sum over the subtable's regions of the region's tent scalar times the stored
// delta. An out-of-range outer or inner index yields zero.
func (s *metricVarStore) delta(outer, inner int, norm []int16) float64 {
if outer < 0 || outer >= len(s.data) {
return 0
}
d := s.data[outer]
if inner < 0 || inner >= len(d.deltaSets) {
return 0
}
coords := make([]float64, s.axisCount)
for i := range coords {
if i < len(norm) {
coords[i] = float64(norm[i]) / 16384.0
}
}
row := d.deltaSets[inner]
sum := 0.0
for j, ri := range d.regionIndices {
sum += supportScalar(coords, s.regLower[ri], s.regPeak[ri], s.regUpper[ri]) * float64(row[j])
}
return sum
}
// parseMetricVarStore decodes the ItemVariationStore at offset off in b, whose
// internal region-list and subtable offsets are relative to off. Only format 1
// is supported (the sole format defined for OpenType variations).
func parseMetricVarStore(b []byte, off int) (*metricVarStore, error) {
r := reader{b: b, pos: off}
if format := r.u16(); format != 1 {
return nil, fmt.Errorf("opentype: ItemVariationStore: unsupported format %d", format)
}
regionListOff := off + int(r.u32())
ivdCount := int(r.u16())
ivdOffsets := make([]int, ivdCount)
for i := range ivdOffsets {
ivdOffsets[i] = off + int(r.u32())
}
if r.err != nil {
return nil, fmt.Errorf("opentype: ItemVariationStore header: %w", r.err)
}
rr := reader{b: b, pos: regionListOff}
axisCount := int(rr.u16())
regionCount := int(rr.u16())
lower := make([][]float64, regionCount)
peak := make([][]float64, regionCount)
upper := make([][]float64, regionCount)
for i := 0; i < regionCount; i++ {
lo := make([]float64, axisCount)
pk := make([]float64, axisCount)
up := make([]float64, axisCount)
for a := 0; a < axisCount; a++ {
lo[a] = rr.f2dot14()
pk[a] = rr.f2dot14()
up[a] = rr.f2dot14()
}
lower[i], peak[i], upper[i] = lo, pk, up
}
if rr.err != nil {
return nil, fmt.Errorf("opentype: ItemVariationStore regions: %w", rr.err)
}
data := make([]metricVarData, ivdCount)
for i, o := range ivdOffsets {
d, err := parseItemVarData(b, o, regionCount)
if err != nil {
return nil, err
}
data[i] = d
}
return &metricVarStore{axisCount: axisCount, regLower: lower, regPeak: peak, regUpper: upper, data: data}, nil
}
// parseItemVarData decodes one ItemVariationData subtable at offset off, reading
// its region ordering and the full grid of item deltas. regionCount bounds the
// region indices. The wordDeltaCount field's high bit (LONG_WORDS) and its low
// 15 bits (the count of wide leading columns) select each delta's byte width.
func parseItemVarData(b []byte, off, regionCount int) (metricVarData, error) {
r := reader{b: b, pos: off}
itemCount := int(r.u16())
wordDeltaCount := int(r.u16())
regionIndexCount := int(r.u16())
idx := make([]int, regionIndexCount)
for k := range idx {
ri := int(r.u16())
if ri >= regionCount {
return metricVarData{}, fmt.Errorf("opentype: ItemVariationData: region index %d out of range", ri)
}
idx[k] = ri
}
if r.err != nil {
return metricVarData{}, fmt.Errorf("opentype: ItemVariationData header: %w", r.err)
}
longWords := wordDeltaCount&0x8000 != 0
wordCount := wordDeltaCount & 0x7FFF
if wordCount > regionIndexCount {
return metricVarData{}, fmt.Errorf("opentype: ItemVariationData: word count %d exceeds region count %d", wordCount, regionIndexCount)
}
sets := make([][]int32, itemCount)
for i := 0; i < itemCount; i++ {
row := make([]int32, regionIndexCount)
for j := 0; j < regionIndexCount; j++ {
row[j] = readVarDelta(&r, longWords, j < wordCount)
}
sets[i] = row
}
if r.err != nil {
return metricVarData{}, fmt.Errorf("opentype: ItemVariationData deltas: %w", r.err)
}
return metricVarData{regionIndices: idx, deltaSets: sets}, nil
}
// readVarDelta reads one ItemVariationData delta. With LONG_WORDS the wide
// columns are 32-bit and the narrow ones 16-bit; without it the wide columns are
// 16-bit and the narrow ones 8-bit. wide selects the wide width for this column.
func readVarDelta(r *reader, longWords, wide bool) int32 {
if longWords && wide {
return int32(r.u32())
}
if longWords || wide {
return int32(r.i16())
}
return int32(int8(r.u8()))
}
// deltaSetMap is a decoded DeltaSetIndexMap: per glyph index, the (outer, inner)
// pair addressing the ItemVariationStore. A nil or empty map maps a glyph index
// to itself (outer 0, inner = glyph index).
type deltaSetMap struct {
outer []uint32
inner []uint32
}
// lookup returns the (outer, inner) store indices for glyph gid. A nil or empty
// map is the identity mapping (outer 0, inner gid); a gid past the map's last
// entry clamps to that entry (as the specification prescribes).
func (m *deltaSetMap) lookup(gid int) (int, int) {
if m == nil || len(m.outer) == 0 {
return 0, gid
}
if gid >= len(m.outer) {
gid = len(m.outer) - 1
}
return int(m.outer[gid]), int(m.inner[gid])
}
// parseDeltaSetMap decodes the DeltaSetIndexMap at offset off in b (format 0 or
// 1). A zero offset means the map is absent and yields a nil map (the identity
// mapping). entryFormat's low nibble gives the inner-index bit count minus one
// and bits 4-5 give the per-entry byte size minus one.
func parseDeltaSetMap(b []byte, off int) (*deltaSetMap, error) {
if off == 0 {
return nil, nil
}
r := reader{b: b, pos: off}
format := r.u8()
entryFormat := r.u8()
var count int
switch format {
case 0:
count = int(r.u16())
case 1:
count = int(r.u32())
default:
return nil, fmt.Errorf("opentype: DeltaSetIndexMap: unsupported format %d", format)
}
innerBits := uint(entryFormat&0x0F) + 1
entrySize := int((entryFormat&0x30)>>4) + 1
innerMask := uint32(1)<<innerBits - 1
m := &deltaSetMap{outer: make([]uint32, count), inner: make([]uint32, count)}
for i := 0; i < count; i++ {
var entry uint32
for k := 0; k < entrySize; k++ {
entry = entry<<8 | uint32(r.u8())
}
m.inner[i] = entry & innerMask
m.outer[i] = entry >> innerBits
}
if r.err != nil {
return nil, fmt.Errorf("opentype: DeltaSetIndexMap: %w", r.err)
}
return m, nil
}
// hvarTable is the parsed HVAR table: the shared store plus the advance-width
// and optional left/right side-bearing DeltaSetIndexMaps. Only the advance-width
// map drives a metric here; the side-bearing maps are decoded for completeness.
type hvarTable struct {
store *metricVarStore
advMap *deltaSetMap
lsbMap *deltaSetMap
rsbMap *deltaSetMap
}
// vvarTable is the parsed VVAR table: the shared store plus the advance-height
// and optional top/bottom side-bearing and vertical-origin maps.
type vvarTable struct {
store *metricVarStore
advMap *deltaSetMap
tsbMap *deltaSetMap
bsbMap *deltaSetMap
vorgMap *deltaSetMap
}
// mvarTable is the parsed MVAR table: the shared store plus a map from each
// four-byte value tag to its (outer, inner) store indices.
type mvarTable struct {
store *metricVarStore
values map[string][2]int
}
// delta returns the MVAR delta for value tag at norm, or zero when the tag is
// absent or the table carries no store.
func (m *mvarTable) delta(tag string, norm []int16) float64 {
v, ok := m.values[tag]
if !ok || m.store == nil {
return 0
}
return m.store.delta(v[0], v[1], norm)
}
// parseMetricVarTable decodes the header common to HVAR and VVAR: a version, a
// required ItemVariationStore offset, and mapCount DeltaSetIndexMap offsets (3
// for HVAR, 4 for VVAR), all uint32 and relative to the table start.
func parseMetricVarTable(b []byte, mapCount int) (*metricVarStore, []*deltaSetMap, error) {
r := reader{b: b}
r.u16() // majorVersion
r.u16() // minorVersion
storeOff := int(r.u32())
offs := make([]int, mapCount)
for i := range offs {
offs[i] = int(r.u32())
}
if r.err != nil {
return nil, nil, fmt.Errorf("opentype: metric variations header: %w", r.err)
}
store, err := parseMetricVarStore(b, storeOff)
if err != nil {
return nil, nil, err
}
maps := make([]*deltaSetMap, mapCount)
for i, o := range offs {
m, err := parseDeltaSetMap(b, o)
if err != nil {
return nil, nil, err
}
maps[i] = m
}
return store, maps, nil
}
// parseHVAR decodes the HVAR table into f.hvar.
func (f *Font) parseHVAR(b []byte) error {
store, maps, err := parseMetricVarTable(b, 3)
if err != nil {
return err
}
f.hvar = &hvarTable{store: store, advMap: maps[0], lsbMap: maps[1], rsbMap: maps[2]}
return nil
}
// parseVVAR decodes the VVAR table into f.vvar.
func (f *Font) parseVVAR(b []byte) error {
store, maps, err := parseMetricVarTable(b, 4)
if err != nil {
return err
}
f.vvar = &vvarTable{store: store, advMap: maps[0], tsbMap: maps[1], bsbMap: maps[2], vorgMap: maps[3]}
return nil
}
// parseMVAR decodes the MVAR table into f.mvar: a header, a run of value records
// (each a four-byte tag and an (outer, inner) index pair) and, unless there are
// none, a shared ItemVariationStore. The store offset here is a uint16.
func (f *Font) parseMVAR(b []byte) error {
r := reader{b: b}
r.u16() // majorVersion
r.u16() // minorVersion
r.u16() // reserved
valueRecordSize := int(r.u16())
valueRecordCount := int(r.u16())
storeOff := int(r.u16())
if r.err != nil {
return fmt.Errorf("opentype: MVAR header: %w", r.err)
}
base := r.pos
values := make(map[string][2]int, valueRecordCount)
for i := 0; i < valueRecordCount; i++ {
vr := reader{b: b, pos: base + i*valueRecordSize}
tag := readTag(&vr)
outer := int(vr.u16())
inner := int(vr.u16())
if vr.err != nil {
return fmt.Errorf("opentype: MVAR value record %d: %w", i, vr.err)
}
values[tag] = [2]int{outer, inner}
}
var store *metricVarStore
if storeOff != 0 {
s, err := parseMetricVarStore(b, storeOff)
if err != nil {
return err
}
store = s
}
f.mvar = &mvarTable{store: store, values: values}
return nil
}
// parseMetricVariations parses the optional HVAR, VVAR and MVAR tables. Each is
// independent; absence is not an error. A present-but-malformed table is
// reported so a corrupt font fails cleanly.
func (f *Font) parseMetricVariations(tables map[string][]byte) error {
if b, ok := tables["HVAR"]; ok {
if err := f.parseHVAR(b); err != nil {
return err
}
}
if b, ok := tables["VVAR"]; ok {
if err := f.parseVVAR(b); err != nil {
return err
}
}
if b, ok := tables["MVAR"]; ok {
if err := f.parseMVAR(b); err != nil {
return err
}
}
return nil
}
// horizontalAdvanceDelta returns glyph gid's horizontal-advance delta in font
// units at normalized coordinate norm: the HVAR advance-width delta when the
// font carries HVAR, otherwise the gvar phantom-point fallback for a variable
// glyf font. It is zero for a static font (nil norm) or when nothing applies.
func (f *Font) horizontalAdvanceDelta(gid int, norm []int16) float64 {
if norm == nil {
return 0
}
if f.hvar != nil {
outer, inner := f.hvar.advMap.lookup(gid)
return f.hvar.store.delta(outer, inner, norm)
}
return f.gvarAdvanceWidthDelta(gid, norm)
}
// verticalAdvanceDelta returns glyph gid's advance-height delta in font units at
// normalized coordinate norm from the VVAR table, or zero when the font has no
// VVAR table or norm is nil.
func (f *Font) verticalAdvanceDelta(gid int, norm []int16) float64 {
if norm == nil || f.vvar == nil {
return 0
}
outer, inner := f.vvar.advMap.lookup(gid)
return f.vvar.store.delta(outer, inner, norm)
}
// gvarAdvanceWidthDelta computes glyph gid's horizontal-advance delta from its
// 'gvar' phantom points — the fallback the specification prescribes when a
// variable font varies advances through gvar but ships no HVAR table. It decodes
// the simple glyph's default points, runs the same gvar delta accumulation used
// for outlines (gvar.go) over a buffer that includes the four TrueType phantom
// points, and returns the advance-width phantom's x delta (pp2) minus the left
// phantom's (pp1). It is zero for an empty glyph, a composite glyph, a
// missing/short gvar, or malformed data.
func (f *Font) gvarAdvanceWidthDelta(gid int, norm []int16) float64 {
gv := f.gvar
if gv == nil || gid < 0 || gid+1 >= len(gv.dataOffsets) {
return 0
}
start, end := f.loca[gid], f.loca[gid+1]
if start >= end || int(end) > len(f.glyf) {
return 0 // empty or out-of-range glyph
}
r := reader{b: f.glyf[start:end]}
n := int(r.i16())
r.skip(8) // xMin, yMin, xMax, yMax
if n < 0 {
return 0 // composite-glyph advance fallback is out of scope
}
contours, err := f.simpleGlyph(&r, n)
if err != nil {
return 0
}
var pts []outlinePoint
var ends []int
for _, c := range contours {
pts = append(pts, c...)
ends = append(ends, len(pts)-1)
}
numReal := len(pts)
gstart := gv.dataArrayOff + int(gv.dataOffsets[gid])
gend := gv.dataArrayOff + int(gv.dataOffsets[gid+1])
if gstart >= gend || gend > len(gv.data) {
return 0
}
coords := make([]float64, gv.axisCount)
for i := range coords {
if i < len(norm) {
coords[i] = float64(norm[i]) / 16384.0
}
}
accX := make([]float64, numReal+4)
accY := make([]float64, numReal+4)
if err := gv.applyGlyph(gv.data[gstart:gend], coords, ends, pts, accX, accY, false); err != nil {
return 0
}
return accX[numReal+1] - accX[numReal]
}
vendor/github.com/go-opentype/opentype/positional.go
Generated Vendored
+185
View File
@@ -0,0 +1,185 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
// This file exposes the parsed GSUB and GPOS tables to external callers and
// adds positional GSUB application: applying a feature's lookups only at
// selected glyph positions. That per-glyph masking is the mechanism a
// complex-text shaper needs for Arabic cursive joining, where the isol, init,
// medi and fina single-substitution features must each be applied only at the
// glyphs that take that joining form. Whole-run application (as done by Apply)
// remains available by passing a nil mask.
// GSUB is the exported handle to a font's parsed GSUB (glyph substitution)
// table, obtained via [Font.GSUB]. It exposes [GSUB.Apply] (whole-run) and
// [GSUB.ApplyMasked] (positional) so a shaper outside this package can drive
// substitution lookups directly.
type GSUB = gsub
// GPOS is the exported handle to a font's parsed GPOS (glyph positioning)
// table, obtained via [Font.GPOS]. Its [GPOS.Position] method runs positioning
// lookups (pair kerning, mark attachment, cursive attachment, ...) over a glyph
// run a caller has already substituted.
type GPOS = gpos
// GSUB returns the font's parsed GSUB table, or nil when the font carries no
// GSUB table (in which case no substitution is available).
func (f *Font) GSUB() *GSUB { return f.gsub }
// GPOS returns the font's parsed GPOS table, or nil when the font carries no
// GPOS table (in which case no positioning is available).
func (f *Font) GPOS() *GPOS { return f.gpos }
// GlyphAdvance returns the horizontal advance of glyph g in font units, or 0
// when g is out of range. A shaper multiplies it by [Face.Scale] to obtain
// pixels, and feeds the per-glyph advances to [GPOS.Position] so mark
// attachment can pull a diacritic back onto its base.
func (f *Font) GlyphAdvance(g GlyphIndex) int {
if int(g) < len(f.advances) {
return f.advances[g]
}
return 0
}
// FeatureApp requests one GSUB feature applied over a subset of a glyph run.
// Tag is the 4-byte feature tag (for example "init" or "liga"). Positions
// selects, by run index, the glyphs the feature's lookups may start at: a
// lookup is attempted at glyph i only when Positions[i] is true. A nil
// Positions applies the feature over the whole run (identical to Apply); a
// non-nil Positions restricts it, with indices at or past its end treated as
// false.
//
// Masking is intended for the length-preserving single substitutions OpenType
// files the Arabic positional-form features (isol/init/medi/fina) as, so the
// mask indices stay aligned to the run as it is rewritten.
type FeatureApp struct {
Tag string
Positions []bool
}
// ApplyMasked applies each feature in feats, in order, to the glyph run and
// returns the substituted run; the input slice is not modified. For a feature
// with a nil Positions mask the feature's lookups run over the whole run (as
// [GSUB.Apply] does); with a non-nil mask they are attempted only at the glyphs
// the mask selects — the mechanism Arabic cursive shaping needs to apply
// init/medi/fina/isol each at just the glyphs in that joining position.
//
// Features are resolved with the script-agnostic default fallback (the DFLT
// script, then the union across every script's default LangSys), so features
// filed only under a real script such as "arab" or "latn" still resolve. An
// empty feature list or an empty run leaves the input unchanged.
//
// Use [GSUB.ApplyMaskedTracked] instead when the caller must follow which
// source glyph each output glyph derives from across a decomposition or
// ligature (as a shaper does to re-derive Arabic joining masks after ccmp
// decomposes letters, and to map output glyphs back to text clusters).
func (g *gsub) ApplyMasked(glyphs []GlyphIndex, feats []FeatureApp) []GlyphIndex {
out, _ := g.applyMasked(glyphs, nil, feats)
return out
}
// ApplyMaskedTracked is [GSUB.ApplyMasked] that also propagates a per-glyph
// cluster (source-position) slice through the substitution. clusters gives the
// starting cluster of each input glyph; nil starts them at 0, 1, 2, ... The
// returned cluster slice is the same length as the returned run: a glyph
// produced by decomposing input glyph k carries cluster[k], and a ligature
// carries the cluster of its first component. Neither input slice is modified.
//
// This is the primitive a complex-text shaper drives: apply ccmp with tracking
// to learn where each decomposed glyph came from, rebuild the init/medi/fina
// masks against the decomposed run, apply those, and read the final clusters to
// map glyphs back to the source text.
func (g *gsub) ApplyMaskedTracked(glyphs []GlyphIndex, clusters []int, feats []FeatureApp) ([]GlyphIndex, []int) {
if clusters == nil {
clusters = make([]int, len(glyphs))
for i := range clusters {
clusters[i] = i
}
}
return g.applyMasked(glyphs, clusters, feats)
}
// applyMasked is the shared implementation of ApplyMasked and
// ApplyMaskedTracked. A nil clusters disables cluster tracking (the plain
// ApplyMasked fast path); a non-nil clusters is threaded through the applier
// and returned alongside the substituted run.
func (g *gsub) applyMasked(glyphs []GlyphIndex, clusters []int, feats []FeatureApp) ([]GlyphIndex, []int) {
if len(feats) == 0 || len(glyphs) == 0 {
return glyphs, clusters
}
out := append([]GlyphIndex(nil), glyphs...)
a := &gsubApplier{lookups: g.lookups, gdef: g.gdef}
if clusters != nil {
a.clusters = append([]int(nil), clusters...)
}
for _, fa := range feats {
idxs := g.selectLookupsDefault(map[string]bool{fa.Tag: true})
for _, li := range idxs {
if li >= len(g.lookups) {
continue
}
out = a.applyLookupMasked(g.lookups[li], out, fa.Positions)
}
}
return out, a.clusters
}
// applyLookupMasked applies one lookup like applyLookup, but restricted to the
// positions the mask selects. A nil mask means the whole run (delegating to
// applyLookup). With a mask, a substitution is attempted at glyph i only when
// mask[i] is true (indices past the mask are treated as false); this is meant
// for length-preserving single substitutions so the mask stays aligned.
func (a *gsubApplier) applyLookupMasked(lk gsubLookup, glyphs []GlyphIndex, mask []bool) []GlyphIndex {
if mask == nil {
return a.applyLookup(lk, glyphs)
}
a.skip = a.skipperFor(lk)
masked := func(i int) bool { return i < len(mask) && mask[i] && !a.skip.skip(glyphs[i]) }
if lk.reverse {
for i := len(glyphs) - 1; i >= 0; i-- {
if !masked(i) {
continue
}
for _, st := range lk.subtables {
if out, _, ok := st.sub(a, glyphs, i); ok {
glyphs = out
break
}
}
}
return glyphs
}
i := 0
for i < len(glyphs) {
if !masked(i) {
i++
continue
}
applied := false
for _, st := range lk.subtables {
if out, next, ok := st.sub(a, glyphs, i); ok {
glyphs = out
i = next
applied = true
break
}
}
if !applied {
i++
}
}
return glyphs
}
// Position runs the GPOS positioning lookups activated by the given feature
// tags over the glyph run and returns one adjustment per glyph, in font units.
// advances gives each glyph's base horizontal advance (used to pull marks back
// onto their base) and may be nil (treated as all-zero). Features are resolved
// with the same script-agnostic default fallback as kerning, so kern/mark/mkmk
// filed under a real script resolve. It is the exported entry point a shaper
// uses to position a run it substituted itself.
func (p *gpos) Position(glyphs []GlyphIndex, advances []int, features ...string) []GlyphPosition {
return p.position(glyphs, advances, features...)
}
vendor/github.com/go-opentype/opentype/raster.go
Generated Vendored
+171
View File
@@ -0,0 +1,171 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import (
"image"
"image/color"
"math"
)
// fpoint is a point in device (pixel) space.
type fpoint struct{ x, y float64 }
// supersample is the per-axis oversampling factor; the rasteriser takes
// ss*ss coverage samples per output pixel for anti-aliasing.
const supersample = 4
// flattenContour scales a font-unit contour to device space (flipping the
// y axis so font "up" becomes image "down") and flattens its quadratic
// segments into a closed polygon. It returns nil for a degenerate contour of
// fewer than two points.
func flattenContour(c contour, scale float64) []fpoint {
n := len(c)
if n < 2 {
return nil
}
type dp struct {
x, y float64
on bool
}
pts := make([]dp, n)
for i, p := range c {
pts[i] = dp{x: p.x * scale, y: -p.y * scale, on: p.on}
}
// Insert an implied on-curve midpoint between any two consecutive
// off-curve points (walking the contour cyclically).
q := make([]dp, 0, n*2)
for i := 0; i < n; i++ {
a := pts[i]
q = append(q, a)
b := pts[(i+1)%n]
if !a.on && !b.on {
q = append(q, dp{x: (a.x + b.x) / 2, y: (a.y + b.y) / 2, on: true})
}
}
// Rotate so the walk starts on an on-curve point (one is guaranteed to
// exist for n >= 2 after midpoint insertion).
s := 0
for !q[s].on {
s++
}
ordered := append(append([]dp{}, q[s:]...), q[:s]...)
m := len(ordered)
poly := []fpoint{{x: ordered[0].x, y: ordered[0].y}}
i := 0
for i < m {
next := ordered[(i+1)%m]
if next.on {
poly = append(poly, fpoint{x: next.x, y: next.y})
i++
continue
}
cur := ordered[i]
end := ordered[(i+2)%m]
flattenQuad(&poly, cur.x, cur.y, next.x, next.y, end.x, end.y)
i += 2
}
return poly
}
// flattenQuad appends a flattened quadratic Bézier (from the start point,
// already the last element of *poly, through control point (cx,cy) to end
// point (x1,y1)) using a step count adapted to the curve's device-space size.
func flattenQuad(poly *[]fpoint, x0, y0, cx, cy, x1, y1 float64) {
d := math.Hypot(cx-x0, cy-y0) + math.Hypot(x1-cx, y1-cy)
steps := int(d / 3.0)
if steps < 1 {
steps = 1
}
for i := 1; i <= steps; i++ {
t := float64(i) / float64(steps)
mt := 1 - t
x := mt*mt*x0 + 2*mt*t*cx + t*t*x1
y := mt*mt*y0 + 2*mt*t*cy + t*t*y1
*poly = append(*poly, fpoint{x: x, y: y})
}
}
// isLeft returns a positive value if point (x,y) is left of the directed edge
// a→b, negative if right, and zero if collinear.
func isLeft(a, b fpoint, x, y float64) float64 {
return (b.x-a.x)*(y-a.y) - (x-a.x)*(b.y-a.y)
}
// insideNonZero reports whether the point (x,y) is inside the polygon set
// under the non-zero winding rule.
func insideNonZero(polys [][]fpoint, x, y float64) bool {
wn := 0
for _, poly := range polys {
n := len(poly)
for i := 0; i < n; i++ {
a := poly[i]
b := poly[(i+1)%n]
if a.y <= y {
if b.y > y && isLeft(a, b, x, y) > 0 {
wn++
}
} else if b.y <= y && isLeft(a, b, x, y) < 0 {
wn--
}
}
}
return wn != 0
}
// rasterize scan-converts the device-space polygons into an 8-bit alpha
// coverage mask using supersampled non-zero winding. It returns the mask's
// device-space bounds (relative to the glyph origin) and an *image.Alpha whose
// own rectangle starts at (0,0). An empty polygon set yields an empty rect and
// a nil mask.
func rasterize(polys [][]fpoint) (image.Rectangle, *image.Alpha) {
if len(polys) == 0 {
return image.Rectangle{}, nil
}
minX, minY := math.Inf(1), math.Inf(1)
maxX, maxY := math.Inf(-1), math.Inf(-1)
for _, poly := range polys {
for _, p := range poly {
if p.x < minX {
minX = p.x
}
if p.x > maxX {
maxX = p.x
}
if p.y < minY {
minY = p.y
}
if p.y > maxY {
maxY = p.y
}
}
}
x0 := int(math.Floor(minX))
y0 := int(math.Floor(minY))
x1 := int(math.Ceil(maxX))
y1 := int(math.Ceil(maxY))
w, h := x1-x0, y1-y0
alpha := image.NewAlpha(image.Rect(0, 0, w, h))
for py := 0; py < h; py++ {
for px := 0; px < w; px++ {
count := 0
for sy := 0; sy < supersample; sy++ {
for sx := 0; sx < supersample; sx++ {
fx := float64(x0+px) + (float64(sx)+0.5)/supersample
fy := float64(y0+py) + (float64(sy)+0.5)/supersample
if insideNonZero(polys, fx, fy) {
count++
}
}
}
alpha.SetAlpha(px, py, color.Alpha{A: uint8(count * 255 / (supersample * supersample))})
}
}
return image.Rect(x0, y0, x1, y1), alpha
}
vendor/github.com/go-opentype/opentype/rawtable.go
Generated Vendored
+38
View File
@@ -0,0 +1,38 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import "sort"
// This file exposes read-only access to the raw, undecoded sfnt table bytes the
// font was parsed from. A consumer that embeds a font (a PDF or EPUB writer) or
// inspects a table this package does not decode can read the exact table data
// without re-parsing the container itself.
// TableTags returns the four-byte tags of every table in the font, sorted
// lexicographically. The returned slice is freshly allocated and owned by the
// caller. Tags retain their sfnt spelling, including any trailing space (for
// example "cvt " and "OS/2").
func (f *Font) TableTags() []string {
tags := make([]string, 0, len(f.tables))
for t := range f.tables {
tags = append(tags, t)
}
sort.Strings(tags)
return tags
}
// Table returns a copy of the raw bytes of the table named by its four-byte tag,
// and ok reporting whether the font carries that table. The tag must be the exact
// sfnt spelling including any trailing space (for example "cvt " or "OS/2"). The
// returned slice is a fresh copy the caller may retain or mutate; it never aliases
// the font's backing data.
func (f *Font) Table(tag string) (data []byte, ok bool) {
b, ok := f.tables[tag]
if !ok {
return nil, false
}
return append([]byte(nil), b...), true
}
vendor/github.com/go-opentype/opentype/reader.go
Generated Vendored
+77
View File
@@ -0,0 +1,77 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import "encoding/binary"
// reader is a big-endian sequential reader over a byte slice with sticky
// error tracking: any read past the end records errTruncated and returns zero,
// so a parser can read a run of fields and check err once at the end.
type reader struct {
b []byte
pos int
err error
}
// u8 reads one unsigned byte.
func (r *reader) u8() uint8 {
if r.pos+1 > len(r.b) {
r.err = errTruncated
return 0
}
v := r.b[r.pos]
r.pos++
return v
}
// u16 reads a big-endian uint16.
func (r *reader) u16() uint16 {
if r.pos+2 > len(r.b) {
r.err = errTruncated
return 0
}
v := binary.BigEndian.Uint16(r.b[r.pos:])
r.pos += 2
return v
}
// u32 reads a big-endian uint32.
func (r *reader) u32() uint32 {
if r.pos+4 > len(r.b) {
r.err = errTruncated
return 0
}
v := binary.BigEndian.Uint32(r.b[r.pos:])
r.pos += 4
return v
}
// i16 reads a big-endian signed int16.
func (r *reader) i16() int16 { return int16(r.u16()) }
// u24 reads a big-endian unsigned 24-bit integer (used by cmap format 14's
// variation-selector and Unicode-value fields).
func (r *reader) u24() uint32 {
if r.pos+3 > len(r.b) {
r.err = errTruncated
return 0
}
v := uint32(r.b[r.pos])<<16 | uint32(r.b[r.pos+1])<<8 | uint32(r.b[r.pos+2])
r.pos += 3
return v
}
// f2dot14 reads an F2Dot14 fixed-point value as a float64.
func (r *reader) f2dot14() float64 { return float64(r.i16()) / 16384.0 }
// skip advances the position by n bytes, recording an error if that runs past
// the end of the slice.
func (r *reader) skip(n int) {
if r.pos+n > len(r.b) {
r.err = errTruncated
return
}
r.pos += n
}
vendor/github.com/go-opentype/opentype/sfntbuild.go
Generated Vendored
+125
View File
@@ -0,0 +1,125 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import (
"encoding/binary"
"sort"
)
// This file holds the shared machinery for writing a fresh sfnt container: the
// table directory, per-table checksums and the head checkSumAdjustment. Both the
// TrueType subsetter (subset.go) and the variable-instance serialiser
// (instance.go) rebuild a font from a set of tables and emit it through
// buildSFNT, so a single, tested writer covers every produced font.
// buildSFNT packs tables into an sfnt container with the given sfnt version tag,
// laying out the table directory (records sorted by tag, 4-byte-aligned bodies),
// filling in each record's checksum, offset and length, and patching head's
// checkSumAdjustment so the emitted font is a valid, self-consistent sfnt. A
// "head" table, when present, must be at least 12 bytes (its checkSumAdjustment
// field lives at offset 8); the callers always supply the full 54-byte table.
func buildSFNT(version uint32, tables map[string][]byte) []byte {
tags := make([]string, 0, len(tables))
for t := range tables {
tags = append(tags, t)
}
sort.Strings(tags)
n := len(tags)
// searchRange/entrySelector/rangeShift describe the largest power of two not
// exceeding n, per the sfnt spec's binary-search hint fields.
entrySelector := 0
for (1 << (entrySelector + 1)) <= n {
entrySelector++
}
searchRange := (1 << entrySelector) * 16
rangeShift := n*16 - searchRange
headerLen := 12 + 16*n
offsets := make(map[string]int, n)
var body []byte
for _, t := range tags {
offsets[t] = headerLen + len(body)
body = append(body, tables[t]...)
for len(body)%4 != 0 {
body = append(body, 0)
}
}
out := make([]byte, headerLen)
binary.BigEndian.PutUint32(out[0:], version)
binary.BigEndian.PutUint16(out[4:], uint16(n))
binary.BigEndian.PutUint16(out[6:], uint16(searchRange))
binary.BigEndian.PutUint16(out[8:], uint16(entrySelector))
binary.BigEndian.PutUint16(out[10:], uint16(rangeShift))
for i, t := range tags {
rec := 12 + i*16
copy(out[rec:], t)
binary.BigEndian.PutUint32(out[rec+4:], sfntChecksum(tables[t]))
binary.BigEndian.PutUint32(out[rec+8:], uint32(offsets[t]))
binary.BigEndian.PutUint32(out[rec+12:], uint32(len(tables[t])))
}
out = append(out, body...)
// head.checkSumAdjustment = 0xB1B0AFBA - checksum(whole file), computed with
// the field itself read as zero (it is still zero in out at this point).
if headOff, ok := offsets["head"]; ok {
adj := 0xB1B0AFBA - sfntChecksum(out)
binary.BigEndian.PutUint32(out[headOff+8:], adj)
}
return out
}
// sfntChecksum is the sfnt table checksum: the sum modulo 2^32 of the data read
// as big-endian uint32 words, the final word zero-padded when the length is not a
// multiple of four.
func sfntChecksum(b []byte) uint32 {
var sum uint32
var i int
for ; i+4 <= len(b); i += 4 {
sum += binary.BigEndian.Uint32(b[i:])
}
if rem := len(b) - i; rem > 0 {
var tail [4]byte
copy(tail[:], b[i:])
sum += binary.BigEndian.Uint32(tail[:])
}
return sum
}
// minimalCmap returns a structurally valid, essentially empty cmap table: a
// single format-4 (3,1 Unicode BMP) subtable whose only segment is the mandatory
// 0xFFFF sentinel, so it maps no character to a real glyph. A subset embedded in
// a PDF as a CIDFontType2 is addressed by glyph id through the CIDToGIDMap and
// never consults a cmap, but opentype.Parse (and other consumers) require a cmap
// table to be present and decodable; this satisfies that without carrying the
// original, now-misindexed mappings.
func minimalCmap() []byte {
var sub []byte
put16 := func(v uint16) { sub = binary.BigEndian.AppendUint16(sub, v) }
put16(4) // format
put16(24) // length (4-header + 8 fixed + one segment of 8)
put16(0) // language
put16(2) // segCountX2 (segCount 1)
put16(2) // searchRange
put16(0) // entrySelector
put16(0) // rangeShift
put16(0xFFFF) // endCode[0] (sentinel)
put16(0) // reservedPad
put16(0xFFFF) // startCode[0]
put16(1) // idDelta[0]
put16(0) // idRangeOffset[0]
var t []byte
putT16 := func(v uint16) { t = binary.BigEndian.AppendUint16(t, v) }
putT32 := func(v uint32) { t = binary.BigEndian.AppendUint32(t, v) }
putT16(0) // version
putT16(1) // numTables
putT16(3) // platformID (Windows)
putT16(1) // encodingID (Unicode BMP)
putT32(12)
return append(t, sub...)
}
vendor/github.com/go-opentype/opentype/shape.go
Generated Vendored
+161
View File
@@ -0,0 +1,161 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
// This file wires the OpenType Layout tables (GSUB, GPOS and the legacy kern
// table, parsed in gsub.go/gpos.go/kern.go) into the public Face API: Shape
// runs glyph substitution over a text run, while Kern and MeasureKerned apply
// pair-adjustment kerning. A font without the relevant tables behaves as a
// plain unshaped, unkerned font.
// Shape maps text to its glyph run and applies the GSUB lookups activated by
// the given feature tags (for example "liga" for standard ligatures, "smcp"
// for small caps). Each rune maps through the font's cmap; an unmapped rune
// becomes glyph 0 (.notdef). With no GSUB table, no features, or features that
// match no lookups, the run is the plain cmap mapping.
func (fc *Face) Shape(text string, features ...string) []GlyphIndex {
var glyphs []GlyphIndex
for _, r := range text {
gid, _ := fc.font.GlyphIndex(r) // unmapped -> 0 (.notdef)
glyphs = append(glyphs, gid)
}
if fc.font.gsub != nil {
glyphs = fc.font.gsub.Apply(glyphs, features...)
}
return glyphs
}
// PositionedGlyph is one glyph of a positioned run produced by ShapePositioned:
// the glyph to draw, the pen-relative placement offsets (XOffset, YOffset) GPOS
// assigned it (a non-zero YOffset lifts an attached diacritic onto its base),
// and the advance to move the pen by afterwards, all in whole pixels at the
// face's size.
type PositionedGlyph struct {
Glyph GlyphIndex
XOffset int
YOffset int
XAdvance int
}
// defaultPosFeatures are the GPOS features ShapePositioned always activates in
// addition to any the caller requests: pair/positional kerning ("kern"),
// mark-to-base/ligature attachment ("mark") and mark-to-mark stacking ("mkmk").
var defaultPosFeatures = []string{"kern", "mark", "mkmk"}
// ShapePositioned maps text to a fully positioned glyph run: each rune maps
// through the cmap (an unmapped rune becomes glyph 0), the requested GSUB
// features are applied (ligatures, contextual substitution, ...), then GPOS
// positions the resulting glyphs — pair kerning, single/cursive adjustment and
// mark attachment (types 4/5/6 pull diacritics onto their base). The positioning
// features are the caller's features plus the always-on kern/mark/mkmk set.
//
// The result has one PositionedGlyph per output glyph, with placement and
// advance in whole pixels at the face's size. A font lacking GSUB or GPOS simply
// skips that stage; with neither the run is the plain cmap mapping at its
// unadjusted advances.
func (fc *Face) ShapePositioned(text string, features ...string) []PositionedGlyph {
var glyphs []GlyphIndex
for _, r := range text {
gid, _ := fc.font.GlyphIndex(r) // unmapped -> 0 (.notdef)
glyphs = append(glyphs, gid)
}
if fc.font.gsub != nil {
glyphs = fc.font.gsub.Apply(glyphs, features...)
}
// Base horizontal advances in font units, indexed by glyph, feed both the
// mark-attachment pull-back and the final per-glyph advance.
advances := make([]int, len(glyphs))
for i, g := range glyphs {
advances[i] = fc.font.advances[g]
}
var positions []GlyphPosition
if fc.font.gpos != nil {
posFeatures := append(append([]string(nil), features...), defaultPosFeatures...)
positions = fc.font.gpos.position(glyphs, advances, posFeatures...)
}
out := make([]PositionedGlyph, len(glyphs))
for i, g := range glyphs {
adv := advances[i]
var xOff, yOff int
if positions != nil {
xOff = positions[i].XOffset
yOff = positions[i].YOffset
adv += positions[i].XAdvance
}
out[i] = PositionedGlyph{
Glyph: g,
XOffset: roundInt(float64(xOff) * fc.scale),
YOffset: roundInt(float64(yOff) * fc.scale),
XAdvance: roundInt(float64(adv) * fc.scale),
}
}
return out
}
// Kern returns the horizontal kerning adjustment between the consecutive runes
// prev and r, in whole pixels at the face's size. GPOS pair positioning is
// preferred; the legacy kern table is used as a fallback. It is zero when either
// rune is unmapped or the font carries no kerning for the pair.
//
// The result is rounded to a whole pixel. A text layer that sets type at (or
// near) the font's own em — where one pixel spans a large design quantum and a
// single-pixel kern would quantise away most of the sub-pixel pair adjustment —
// should measure with KernUnits (or KernIndexUnits) instead, which return the
// unrounded font-unit value, exactly as AdvanceIndexUnits complements
// AdvanceIndex.
func (fc *Face) Kern(prev, r rune) int {
return roundInt(fc.KernUnits(prev, r) * fc.scale)
}
// KernIndexUnits returns the horizontal kerning adjustment between the
// consecutive glyphs left and right in font units (unrounded) — the value Kern
// scales and rounds to pixels. GPOS pair positioning is preferred; the legacy
// kern table is the fallback. It is the by-glyph-index, font-unit counterpart of
// Kern, the exact primitive a shaper or a text engine wants: kerning applies
// between shaped glyphs (which may be ligatures), and a caller typesetting at
// the design em keeps the sub-pixel adjustment that Kern's whole-pixel rounding
// would discard. A caller multiplies the result by Face.Scale to obtain pixels
// (mirroring AdvanceIndexUnits). The adjustment is zero when the font carries no
// kerning for the pair.
func (fc *Face) KernIndexUnits(left, right GlyphIndex) float64 {
kn := Kerner{gpos: fc.font.gpos, kern: fc.font.kern}
return float64(kn.Kerning(left, right))
}
// KernUnits returns the horizontal kerning adjustment between the consecutive
// runes prev and r in font units (unrounded) — the rune-keyed, font-unit
// counterpart of Kern. Each rune maps through the cmap; the result is zero when
// either rune is unmapped or the font carries no kerning for the pair. A caller
// multiplies by Face.Scale to obtain pixels; see KernIndexUnits for the
// by-glyph-index form a shaper drives on an already-substituted run.
func (fc *Face) KernUnits(prev, r rune) float64 {
lg, ok := fc.font.GlyphIndex(prev)
if !ok {
return 0
}
rg, ok := fc.font.GlyphIndex(r)
if !ok {
return 0
}
return fc.KernIndexUnits(lg, rg)
}
// MeasureKerned returns the total advance width of s in pixels like Measure,
// but additionally applies the font's kerning between each pair of consecutive
// runes. For a font with no kerning it equals Measure.
func (fc *Face) MeasureKerned(s string) int {
total := 0
var prev rune
havePrev := false
for _, r := range s {
if havePrev {
total += fc.Kern(prev, r)
}
total += fc.Advance(r)
prev = r
havePrev = true
}
return total
}
vendor/github.com/go-opentype/opentype/subset.go
Generated Vendored
+224
View File
@@ -0,0 +1,224 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import (
"encoding/binary"
"errors"
"fmt"
"sort"
)
// This file implements TrueType ('glyf') glyph subsetting: rebuilding a minimal,
// self-contained sfnt that carries only a requested set of glyphs (and every
// glyph a composite among them references). It is the primitive a PDF or EPUB
// embedder needs so it can ship a subset font program instead of the whole face,
// and so it does not have to re-parse and re-serialise the sfnt container itself.
// CFF ('CFF ') charstring subsetting lives in subsetcff.go.
// componentRef locates one component-glyph reference inside a composite glyph's
// raw bytes: the byte offset of its two-byte glyph-id field and the glyph id
// stored there. Subsetting reads the ids to compute the closure and rewrites the
// field at the offset to the component's new id.
type componentRef struct {
offset int
gid GlyphIndex
}
// componentRefs returns the component-glyph references of a raw glyph. It returns
// nil for an empty or simple glyph (numberOfContours >= 0) and, for a composite,
// one entry per component in stream order. The walk is bounds-checked: a record
// that would read past the end of data ends the scan, so a truncated composite
// yields the components decodable so far rather than a panic.
func componentRefs(data []byte) []componentRef {
if len(data) < 10 || sbe16(data) >= 0 {
return nil
}
var refs []componentRef
p := 10
for p+4 <= len(data) {
flags := be16(data[p:])
refs = append(refs, componentRef{offset: p + 2, gid: GlyphIndex(be16(data[p+2:]))})
p += 4
if flags&compArgsAreWords != 0 {
p += 4
} else {
p += 2
}
switch {
case flags&compWeHaveScale != 0:
p += 2
case flags&compXAndYScale != 0:
p += 4
case flags&compTwoByTwo != 0:
p += 8
}
if flags&compMoreComponents == 0 {
break
}
}
return refs
}
// rawGlyph returns glyph gid's raw bytes from the glyf table (the loca-delimited
// slice), or nil for an empty glyph. The caller must not mutate the result; it
// aliases the font's backing data.
func (f *Font) rawGlyph(gid GlyphIndex) []byte {
start, end := f.loca[gid], f.loca[gid+1]
if start >= end || int(end) > len(f.glyf) {
return nil
}
return f.glyf[start:end]
}
// subsetClosure returns the sorted set of glyph ids to keep: the requested gids,
// glyph 0 (.notdef, always retained so the subset has a valid missing glyph), and
// every glyph transitively referenced by a composite among them. A requested id
// outside the font's glyph range is an error. Cycles are terminated by the
// membership set, so a self- or mutually-referential composite is handled.
func (f *Font) subsetClosure(gids []GlyphIndex) ([]GlyphIndex, error) {
needed := map[GlyphIndex]bool{}
var walk func(gid GlyphIndex) error
walk = func(gid GlyphIndex) error {
if int(gid) >= f.numGlyphs {
return fmt.Errorf("opentype: subset: glyph index %d out of range", int(gid))
}
if needed[gid] {
return nil
}
needed[gid] = true
for _, ref := range componentRefs(f.rawGlyph(gid)) {
if err := walk(ref.gid); err != nil {
return err
}
}
return nil
}
if err := walk(0); err != nil {
return nil, err
}
for _, g := range gids {
if err := walk(g); err != nil {
return nil, err
}
}
order := make([]GlyphIndex, 0, len(needed))
for g := range needed {
order = append(order, g)
}
sort.Slice(order, func(i, j int) bool { return order[i] < order[j] })
return order, nil
}
// SubsetTrueType builds a minimal, valid TrueType sfnt carrying only the glyphs
// in gids plus glyph 0 and every component a composite among them references. The
// kept glyphs are renumbered compactly (glyph 0 stays 0); the returned oldToNew
// map gives each retained original glyph id its id in the subset, which a PDF
// CIDFontType2 embedder turns into a /CIDToGIDMap (CID = original id -> subset id)
// and a content-stream writer uses to re-address glyphs.
//
// The rebuilt font contains glyf, loca (long form), head, hhea, maxp, hmtx and a
// minimal cmap (enough to re-parse; a PDF addresses the subset by glyph id, not
// through the cmap), plus the TrueType instruction tables (cvt/fpgm/prep) when the
// original font carried them so hinted glyphs still render as designed. Composite
// component references are rewritten to the new glyph numbering.
//
// It returns an error for a font with no glyf table (a CFF/OpenType font — use
// SubsetCFF) or when a requested glyph id is out of range.
func (f *Font) SubsetTrueType(gids []GlyphIndex) (data []byte, oldToNew map[GlyphIndex]GlyphIndex, err error) {
if f.glyf == nil {
return nil, nil, errors.New("opentype: SubsetTrueType: font has no glyf table (not a TrueType font)")
}
kept, err := f.subsetClosure(gids)
if err != nil {
return nil, nil, err
}
oldToNew = make(map[GlyphIndex]GlyphIndex, len(kept))
for newGID, oldGID := range kept {
oldToNew[oldGID] = GlyphIndex(newGID)
}
// Rebuild glyf and a long-format loca in the new, compact glyph order,
// rewriting each composite's component ids to their subset ids.
blobs := make([][]byte, len(kept))
for newGID, oldGID := range kept {
blobs[newGID] = remapGlyph(f.rawGlyph(oldGID), oldToNew)
}
locaBytes, glyf := packGlyf(blobs)
tables := map[string][]byte{
"glyf": glyf,
"loca": locaBytes,
"head": f.subsetHead(),
"hhea": f.subsetHhea(len(kept)),
"maxp": f.subsetMaxp(len(kept)),
"hmtx": f.subsetHmtx(kept),
"cmap": minimalCmap(),
}
for _, opt := range []string{"cvt ", "fpgm", "prep"} {
if b, ok := f.tables[opt]; ok {
tables[opt] = append([]byte(nil), b...)
}
}
return buildSFNT(versionTrueType, tables), oldToNew, nil
}
// remapGlyph returns a copy of a raw glyph with every composite component id
// rewritten to its subset id via oldToNew. A simple or empty glyph is copied
// unchanged. A component id absent from oldToNew (which cannot happen for a glyph
// produced by subsetClosure, whose closure includes every component) is left as
// is, so the function never panics on unexpected input.
func remapGlyph(data []byte, oldToNew map[GlyphIndex]GlyphIndex) []byte {
refs := componentRefs(data)
out := append([]byte(nil), data...)
for _, ref := range refs {
if nb, ok := oldToNew[ref.gid]; ok {
binary.BigEndian.PutUint16(out[ref.offset:], uint16(nb))
}
}
return out
}
// subsetHead clones the head table with checkSumAdjustment zeroed (buildSFNT
// recomputes it) and indexToLocFormat set to long, matching the long-format loca
// the subsetter always emits.
func (f *Font) subsetHead() []byte {
head := append([]byte(nil), f.tables["head"]...)
binary.BigEndian.PutUint32(head[8:], 0) // checkSumAdjustment: recomputed by buildSFNT
binary.BigEndian.PutUint16(head[50:], 1) // indexToLocFormat: long
return head
}
// subsetHhea clones the hhea table with numberOfHMetrics set to the subset glyph
// count, matching the full-width hmtx subsetHmtx emits.
func (f *Font) subsetHhea(n int) []byte {
hhea := append([]byte(nil), f.tables["hhea"]...)
binary.BigEndian.PutUint16(hhea[34:], uint16(n))
return hhea
}
// subsetMaxp clones the maxp table with numGlyphs set to the subset glyph count.
// The other maxp limits (max points, max component depth, ...) remain valid upper
// bounds for a subset of the original glyphs, so they are carried over unchanged.
func (f *Font) subsetMaxp(n int) []byte {
maxp := append([]byte(nil), f.tables["maxp"]...)
binary.BigEndian.PutUint16(maxp[4:], uint16(n))
return maxp
}
// subsetHmtx builds an hmtx table for the subset glyph order: one (advanceWidth,
// leftSideBearing) pair per kept glyph, taken from the original per-glyph metrics.
// Emitting a full entry for every glyph (numberOfHMetrics == glyph count, set in
// subsetHhea) keeps the table trivially correct without the trailing shared-advance
// optimisation.
func (f *Font) subsetHmtx(kept []GlyphIndex) []byte {
out := make([]byte, 4*len(kept))
for i, oldGID := range kept {
binary.BigEndian.PutUint16(out[i*4:], uint16(f.advances[oldGID]))
binary.BigEndian.PutUint16(out[i*4+2:], uint16(int16(f.lsbs[oldGID])))
}
return out
}
vendor/github.com/go-opentype/opentype/subsetcff.go
Generated Vendored
+406
View File
@@ -0,0 +1,406 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import "fmt"
// This file implements CFF (Type2 charstring) subsetting: rebuilding a 'CFF '
// table that keeps the outlines of a requested set of glyphs and replaces every
// other glyph's charstring with an empty one (a lone endchar). It is a real
// charstring subset — the CharStrings INDEX, the bulk of a CFF program, no longer
// carries the dropped glyphs' outlines — while glyph numbering is preserved, so a
// PDF embeds the result with an Identity CIDToGIDMap and the font's charset,
// encoding, Private DICT and subroutines all stay valid unchanged.
//
// Scope (documented, deliberate): the global and local subroutine INDEXes are
// retained whole rather than tree-shaken; CID-keyed CFF (a ROS/FDArray/FDSelect
// font) and CFF2 are rejected with an error rather than subset; and endchar-seac
// accent composition (a deprecated CFF feature) is not followed, so a caller
// embedding a seac-built accented glyph should also request its base and accent
// components. These edges keep the subsetter tractable and exact for the common
// name-keyed OTF, which is what PDF and EPUB embedders overwhelmingly handle.
// dictRawEntry is one operator of a CFF DICT together with the raw bytes of its
// operands and their decoded values. The raw bytes let a DICT be re-emitted
// verbatim except for the operators whose operands the subsetter rewrites (the
// offset operators of the Top DICT); copying them avoids decoding and re-encoding
// real-number operands (such as FontMatrix), which would not round-trip exactly.
// The decoded values let the subsetter read the offset and flag operators it acts
// on without a second parse pass.
type dictRawEntry struct {
operands []byte
values []float64
op int
}
// parseDictRaw splits a CFF DICT into its operator entries, each carrying the raw
// operand bytes that preceded the operator and their decoded values. It reuses
// parseDictOperand for the per-operand byte lengths and values, so it accepts
// exactly the operands parseDict does.
func parseDictRaw(b []byte) ([]dictRawEntry, error) {
var entries []dictRawEntry
opStart := 0
var values []float64
i := 0
for i < len(b) {
b0 := b[i]
if b0 <= 21 {
operands := b[opStart:i]
op := int(b0)
i++
if b0 == 12 {
if i >= len(b) {
return nil, fmt.Errorf("opentype: cff dict: truncated operator")
}
op = 1200 + int(b[i])
i++
}
entries = append(entries, dictRawEntry{operands: operands, values: values, op: op})
opStart = i
values = nil
continue
}
v, n, err := parseDictOperand(b, i)
if err != nil {
return nil, err
}
values = append(values, v)
i += n
}
return entries, nil
}
// findEntry returns the DICT entry for operator op, or nil when the operator is
// absent.
func findEntry(entries []dictRawEntry, op int) *dictRawEntry {
for i := range entries {
if entries[i].op == op {
return &entries[i]
}
}
return nil
}
// encodeCFFInt5 encodes v as a five-byte CFF DICT integer operand (marker 29 and
// a 32-bit big-endian value). A fixed five-byte width keeps the Top DICT the same
// size no matter what offset value it holds, which lets the layout be computed in
// one pass: the offsets an operand carries do not change the size of the DICT that
// carries them.
func encodeCFFInt5(v int) []byte {
return []byte{29, byte(v >> 24), byte(v >> 16), byte(v >> 8), byte(v)}
}
// encodeCFFOp encodes a DICT operator (a one-byte operator, or the escape byte 12
// followed by the second byte for a two-byte operator keyed as 1200+second).
func encodeCFFOp(op int) []byte {
if op >= 1200 {
return []byte{12, byte(op - 1200)}
}
return []byte{byte(op)}
}
// topDictOffsets carries the rebuilt Top DICT's offset operands. charset is -1
// when the original charset is predefined (its operand is copied unchanged);
// otherwise it is the new absolute offset of the relocated charset. hasPrivate is
// false when the font has no Private DICT (no op-18 entry is emitted then).
type topDictOffsets struct {
charset int
charStrings int
privateSize int
privateOff int
hasPrivate bool
charsetIsPre bool
}
// encodeTopDict re-emits the Top DICT, copying every operator verbatim except the
// ones that carry absolute offsets: CharStrings (17) and Private (18) get the new
// offsets, charset (15) gets its new offset unless it is predefined, and Encoding
// (16) is forced to the predefined Standard encoding (0) since a custom encoding
// is neither relocated nor needed for embedding. All rewritten offset operands use
// the fixed five-byte form, so the encoded size does not depend on the offsets.
func encodeTopDict(entries []dictRawEntry, off topDictOffsets) []byte {
var out []byte
for _, e := range entries {
switch e.op {
case 15: // charset
if off.charsetIsPre {
out = append(out, e.operands...)
} else {
out = append(out, encodeCFFInt5(off.charset)...)
}
out = append(out, encodeCFFOp(15)...)
case 16: // Encoding -> Standard
out = append(out, encodeCFFInt5(0)...)
out = append(out, encodeCFFOp(16)...)
case 17: // CharStrings
out = append(out, encodeCFFInt5(off.charStrings)...)
out = append(out, encodeCFFOp(17)...)
case 18: // Private [size offset]
out = append(out, encodeCFFInt5(off.privateSize)...)
out = append(out, encodeCFFInt5(off.privateOff)...)
out = append(out, encodeCFFOp(18)...)
default:
out = append(out, e.operands...)
out = append(out, encodeCFFOp(e.op)...)
}
}
return out
}
// writeCFFIndex encodes items as a CFF INDEX (a count, an offset size, count+1
// relative offsets, then the concatenated items). An empty item list encodes as a
// count of zero, the shortest valid INDEX.
func writeCFFIndex(items [][]byte) []byte {
count := len(items)
out := []byte{byte(count >> 8), byte(count)}
if count == 0 {
return out
}
offs := make([]int, count+1)
offs[0] = 1
for i, it := range items {
offs[i+1] = offs[i] + len(it)
}
offSize := 1
for v := offs[count]; v > 0xFF; v >>= 8 {
offSize++
}
out = append(out, byte(offSize))
for _, o := range offs {
for b := offSize - 1; b >= 0; b-- {
out = append(out, byte(o>>(8*b)))
}
}
for _, it := range items {
out = append(out, it...)
}
return out
}
// cffIndexEnd returns the byte offset just past the CFF INDEX beginning at pos.
func cffIndexEnd(data []byte, pos int) (int, error) {
r := &reader{b: data, pos: pos}
if _, err := parseIndex(r); err != nil {
return 0, err
}
return r.pos, nil
}
// cffCharsetLength returns the byte length of the (non-predefined) charset at off
// for a font of nGlyphs glyphs, decoding format 0, 1 or 2 exactly as parseCharset
// walks them.
func cffCharsetLength(data []byte, off, nGlyphs int) (int, error) {
if off > len(data) {
return 0, fmt.Errorf("opentype: cff charset offset out of range")
}
r := &reader{b: data, pos: off}
format := r.u8()
gid := 1
switch format {
case 0:
for gid < nGlyphs {
r.u16()
gid++
}
case 1:
for gid < nGlyphs {
r.u16()
gid += int(r.u8()) + 1
}
case 2:
for gid < nGlyphs {
r.u16()
gid += int(r.u16()) + 1
}
default:
return 0, fmt.Errorf("opentype: cff charset: unsupported format %d", format)
}
if r.err != nil {
return 0, fmt.Errorf("opentype: cff charset: %w", r.err)
}
return r.pos - off, nil
}
// cffPrivateBlob returns a copy of the Private DICT at [privOff, privOff+privSize)
// together with its Local Subr INDEX when it declares one, as a single contiguous
// blob. The blob preserves the byte distance between the Private DICT and its
// local subrs, so the DICT's (relative) Subrs offset stays valid after the blob is
// relocated as a whole.
func cffPrivateBlob(data []byte, privOff, privSize int) ([]byte, error) {
if privOff < 0 || privSize < 0 || privOff+privSize > len(data) {
return nil, fmt.Errorf("opentype: cff: Private DICT out of range")
}
priv, err := parseDict(data[privOff : privOff+privSize])
if err != nil {
return nil, err
}
soff, ok := dictInt(priv, 19, 0)
if !ok {
return append([]byte(nil), data[privOff:privOff+privSize]...), nil
}
subrAbs := privOff + soff
if subrAbs < 0 || subrAbs > len(data) {
return nil, fmt.Errorf("opentype: cff: Local Subrs offset out of range")
}
end, err := cffIndexEnd(data, subrAbs)
if err != nil {
return nil, err
}
return append([]byte(nil), data[privOff:end]...), nil
}
// SubsetCFF builds a subset 'CFF ' table (the raw font program, as a PDF /FontFile3
// CIDFontType0C stream embeds it) that keeps the charstrings of the glyphs in gids
// plus glyph 0, replacing every other glyph's charstring with an empty one. Glyph
// numbering is preserved, so the font's charset, encoding, Private DICT and
// subroutines remain valid and a PDF embeds the result with an Identity CIDToGIDMap
// (no glyph-id remap is needed, unlike SubsetTrueType).
//
// It returns an error for a font with no CFF table (a TrueType font — use
// SubsetTrueType), for a CFF2 (variable) font, for a CID-keyed CFF (one with a ROS,
// FDArray or FDSelect operator), or when a requested glyph id is out of range. See
// the file-level comment for the retained-subroutine and endchar-seac scope notes.
func (f *Font) SubsetCFF(gids []GlyphIndex) ([]byte, error) {
if f.cff2 != nil {
return nil, fmt.Errorf("opentype: SubsetCFF: CFF2 (variable) fonts are not supported; bake an instance first")
}
if f.cff == nil {
return nil, fmt.Errorf("opentype: SubsetCFF: font has no CFF table (not a CFF/OpenType font)")
}
// f.cff being non-nil means Parse decoded a 'CFF ' table, which it also
// retained in f.tables, so the lookup always succeeds here.
return subsetCFFTable(f.tables["CFF "], f.cff, gids)
}
// subsetCFFTable rebuilds the CFF table bytes: header and the Name, String and
// Global Subr INDEXes are copied verbatim; the Top DICT is re-emitted with new
// offsets; the CharStrings INDEX is rebuilt keeping the requested glyphs; and the
// (relocated) charset and Private+Local-Subr blobs are appended. The layout is
// header, Name INDEX, Top DICT INDEX, String INDEX, Global Subr INDEX, charset,
// CharStrings INDEX, Private+Local Subrs.
func subsetCFFTable(data []byte, cff *cffTable, gids []GlyphIndex) ([]byte, error) {
if len(data) < 4 {
return nil, fmt.Errorf("opentype: cff header: %w", errTruncated)
}
hdrSize := int(data[2])
if hdrSize < 4 || hdrSize > len(data) {
return nil, fmt.Errorf("opentype: cff: bad hdrSize %d", hdrSize)
}
// Re-walk the top-level INDEXes to recover their raw byte spans (Name, Top
// DICT, String, Global Subr), which are copied unchanged into the subset.
r := &reader{b: data, pos: hdrSize}
nameStart := r.pos
if _, err := parseIndex(r); err != nil {
return nil, err
}
topStart := r.pos
topDicts, err := parseIndex(r)
if err != nil {
return nil, err
}
if len(topDicts) < 1 {
return nil, fmt.Errorf("opentype: cff: empty Top DICT INDEX")
}
strStart := r.pos
if _, err := parseIndex(r); err != nil {
return nil, err
}
gsubrStart := r.pos
if _, err := parseIndex(r); err != nil {
return nil, err
}
gsubrEnd := r.pos
nameRaw := data[nameStart:topStart]
strRaw := data[strStart:gsubrStart]
gsubrRaw := data[gsubrStart:gsubrEnd]
entries, err := parseDictRaw(topDicts[0])
if err != nil {
return nil, err
}
// CID-keyed CFF (ROS/FDArray/FDSelect) is out of scope: its charset and
// FDSelect index the same glyph ids, and its Font DICTs carry their own
// Private offsets, none of which this preserve-numbering subsetter rewrites.
if findEntry(entries, 1230) != nil {
return nil, fmt.Errorf("opentype: SubsetCFF: CID-keyed CFF is not supported")
}
if findEntry(entries, 1236) != nil {
return nil, fmt.Errorf("opentype: SubsetCFF: CID-keyed CFF (FDArray) is not supported")
}
n := len(cff.charStrings)
kept := map[GlyphIndex]bool{0: true}
for _, g := range gids {
if int(g) >= n {
return nil, fmt.Errorf("opentype: SubsetCFF: glyph index %d out of range", int(g))
}
kept[g] = true
}
items := make([][]byte, n)
for gid := 0; gid < n; gid++ {
if kept[GlyphIndex(gid)] {
items[gid] = cff.charStrings[gid]
} else {
items[gid] = []byte{14} // endchar: an empty glyph
}
}
newCharStrings := writeCFFIndex(items)
// Resolve the charset: predefined (offset <= 2) charsets are copied by operand
// and need no blob; a custom charset is relocated and its length measured.
var charsetBlob []byte
offsets := topDictOffsets{charset: -1, charsetIsPre: true}
if e := findEntry(entries, 15); e != nil && len(e.values) > 0 && int(e.values[0]) > 2 {
csOff := int(e.values[0])
csLen, err := cffCharsetLength(data, csOff, n)
if err != nil {
return nil, err
}
charsetBlob = append([]byte(nil), data[csOff:csOff+csLen]...)
offsets.charsetIsPre = false
}
// Resolve the Private DICT and its Local Subrs into one relocatable blob.
var privBlob []byte
if pv := findEntry(entries, 18); pv != nil {
if len(pv.values) < 2 {
return nil, fmt.Errorf("opentype: cff: bad Private DICT entry")
}
privSize, privOff := int(pv.values[0]), int(pv.values[1])
privBlob, err = cffPrivateBlob(data, privOff, privSize)
if err != nil {
return nil, err
}
offsets.hasPrivate = true
offsets.privateSize = privSize
}
// Measure the Top DICT INDEX (fixed size thanks to five-byte offset operands),
// then compute every downstream offset and assemble.
topDictSize := len(writeCFFIndex([][]byte{encodeTopDict(entries, offsets)}))
pos := hdrSize + len(nameRaw) + topDictSize + len(strRaw) + len(gsubrRaw)
if !offsets.charsetIsPre {
offsets.charset = pos
pos += len(charsetBlob)
}
offsets.charStrings = pos
pos += len(newCharStrings)
if offsets.hasPrivate {
offsets.privateOff = pos
}
topDict := encodeTopDict(entries, offsets)
out := append([]byte(nil), data[:hdrSize]...)
out = append(out, nameRaw...)
out = append(out, writeCFFIndex([][]byte{topDict})...)
out = append(out, strRaw...)
out = append(out, gsubrRaw...)
out = append(out, charsetBlob...)
out = append(out, newCharStrings...)
out = append(out, privBlob...)
return out, nil
}
vendor/github.com/go-opentype/opentype/tables.go
Generated Vendored
+122
View File
@@ -0,0 +1,122 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import "fmt"
// parseHead reads unitsPerEm and indexToLocFormat from the head table.
// The full head table is 54 bytes; indexToLocFormat sits at offset 50.
func (f *Font) parseHead(b []byte) error {
if len(b) < 54 {
return fmt.Errorf("opentype: head table: %w", errTruncated)
}
f.unitsPerEm = int(be16(b[18:]))
if f.unitsPerEm == 0 {
return fmt.Errorf("opentype: head table: unitsPerEm is zero")
}
f.macStyle = be16(b[44:])
f.xMin = int(sbe16(b[36:]))
f.yMin = int(sbe16(b[38:]))
f.xMax = int(sbe16(b[40:]))
f.yMax = int(sbe16(b[42:]))
f.indexToLocFormat = int(sbe16(b[50:]))
if f.indexToLocFormat != 0 && f.indexToLocFormat != 1 {
return fmt.Errorf("opentype: head table: bad indexToLocFormat %d", f.indexToLocFormat)
}
return nil
}
// parseMaxp reads numGlyphs from the maxp table (offset 4).
func (f *Font) parseMaxp(b []byte) error {
if len(b) < 6 {
return fmt.Errorf("opentype: maxp table: %w", errTruncated)
}
f.numGlyphs = int(be16(b[4:]))
if f.numGlyphs == 0 {
return fmt.Errorf("opentype: maxp table: zero glyphs")
}
// Version-1.0 maxp additionally carries the interpreter limits used by the
// hinting VM. A shorter (version-0.5) table leaves conservative defaults.
f.maxTwilightPts = 16
f.maxStorage = 64
f.maxFunctionDefs = 64
f.maxStackElements = 256
if len(b) >= 26 {
f.maxTwilightPts = int(be16(b[16:]))
f.maxStorage = int(be16(b[18:]))
f.maxFunctionDefs = int(be16(b[20:]))
f.maxStackElements = int(be16(b[24:]))
}
return nil
}
// parseHhea reads the vertical metrics and numberOfHMetrics from hhea.
// ascender/descender/lineGap are int16 at offsets 4/6/8; numberOfHMetrics is
// the uint16 at offset 34.
func (f *Font) parseHhea(b []byte) error {
if len(b) < 36 {
return fmt.Errorf("opentype: hhea table: %w", errTruncated)
}
f.ascender = int(sbe16(b[4:]))
f.descender = int(sbe16(b[6:]))
f.lineGap = int(sbe16(b[8:]))
f.numberOfHMetrics = int(be16(b[34:]))
if f.numberOfHMetrics == 0 || f.numberOfHMetrics > f.numGlyphs {
return fmt.Errorf("opentype: hhea table: bad numberOfHMetrics %d", f.numberOfHMetrics)
}
return nil
}
// numberOfHMetrics is kept only for the span of parseHhea→parseHmtx.
// It is declared on Font to pass the value between the two parsers.
// parseHmtx expands the hmtx table into per-glyph advanceWidth and left side
// bearing slices. The first numberOfHMetrics entries are (advance, lsb) pairs;
// the trailing glyphs share the last entry's advance and carry their own lsb.
func (f *Font) parseHmtx(b []byte) error {
n := f.numberOfHMetrics
need := 4*n + 2*(f.numGlyphs-n)
if len(b) < need {
return fmt.Errorf("opentype: hmtx table: %w", errTruncated)
}
f.advances = make([]int, f.numGlyphs)
f.lsbs = make([]int, f.numGlyphs)
lastAdvance := 0
for i := 0; i < f.numGlyphs; i++ {
if i < n {
lastAdvance = int(be16(b[i*4:]))
f.advances[i] = lastAdvance
f.lsbs[i] = int(sbe16(b[i*4+2:]))
} else {
f.advances[i] = lastAdvance
f.lsbs[i] = int(sbe16(b[4*n+2*(i-n):]))
}
}
return nil
}
// parseLoca decodes the loca table into numGlyphs+1 glyph offsets into glyf.
// Short format stores uint16 half-offsets (doubled); long format stores
// uint32 offsets directly, selected by head.indexToLocFormat.
func (f *Font) parseLoca(b []byte) error {
count := f.numGlyphs + 1
f.loca = make([]uint32, count)
if f.indexToLocFormat == 0 {
if len(b) < count*2 {
return fmt.Errorf("opentype: loca table (short): %w", errTruncated)
}
for i := 0; i < count; i++ {
f.loca[i] = uint32(be16(b[i*2:])) * 2
}
return nil
}
if len(b) < count*4 {
return fmt.Errorf("opentype: loca table (long): %w", errTruncated)
}
for i := 0; i < count; i++ {
f.loca[i] = be32(b[i*4:])
}
return nil
}
vendor/github.com/go-opentype/opentype/type1.go
Generated Vendored
+467
View File
@@ -0,0 +1,467 @@
// Copyright (c) 2026, the go-opentype authors
// All rights reserved.
//
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import (
"bytes"
"fmt"
"math"
"strconv"
)
// type1Font is a PostScript Type 1 font program: a dictionary of charstrings
// addressed by name, the subroutines they call, and the encoding the program
// was written with. It is the format PDF calls FontFile, and the one every
// PostScript printer of the nineteen-nineties spoke.
type type1Font struct {
// names is every glyph, in the order they were read, which is the order
// their indices are handed out in. .notdef is put first when the font has
// one, so that glyph zero means what it means everywhere else.
names []string
charstrings map[string][]byte
byName map[string]GlyphIndex
subrs [][]byte
// encoding is what the program itself says a byte stands for; nil means
// it said to use the standard one.
encoding map[byte]string
fontMatrix [6]float64
}
// The three constants the eexec cipher is built on, and the two keys: one for
// the program's private half, one for each charstring inside it.
const (
eexecC1 = 52845
eexecC2 = 22719
eexecKey = 55665
charstringR = 4330
defaultLenIV = 4
)
// ParseType1 decodes a PostScript Type 1 font program — what a PDF carries as
// a FontFile, and what a .pfb or .pfa file holds. Such a program addresses its
// glyphs by name and carries an encoding of its own, so the result answers
// [Font.GlyphIndexByName] and [Font.GlyphIndexByCode] and reports no character
// map.
//
// The bytes are not retained: a Type 1 program is decrypted on the way in, so
// what is kept is the plaintext.
func ParseType1(b []byte) (*Font, error) {
clear, enc, err := type1Halves(b)
if err != nil {
return nil, err
}
private := eexecDecrypt(enc, eexecKey, 4)
if len(private) == 0 {
return nil, fmt.Errorf("opentype: type1: the private half is empty")
}
t1 := &type1Font{
charstrings: map[string][]byte{},
byName: map[string]GlyphIndex{},
fontMatrix: defaultFontMatrix,
}
t1.readClear(clear)
if err := t1.readPrivate(private); err != nil {
return nil, err
}
if len(t1.names) == 0 {
return nil, fmt.Errorf("opentype: type1: the font has no glyphs")
}
f := &Font{
t1: t1,
numGlyphs: len(t1.names),
unitsPerEm: unitsPerEmOf(t1.fontMatrix),
numberOfHMetrics: len(t1.names),
glyphNames: t1.byName,
}
f.ascender = f.unitsPerEm
f.advances, f.lsbs = t1.metrics()
return f, nil
}
// type1Halves separates the program's readable half from its encrypted one,
// whichever container they arrived in: the segments of a .pfb file, the hex of
// a .pfa one, or the plain concatenation a PDF stream holds.
func type1Halves(b []byte) (clear, enc []byte, err error) {
if len(b) >= 2 && b[0] == 0x80 {
if b, err = joinPFB(b); err != nil {
return nil, nil, err
}
}
i := bytes.Index(b, []byte("eexec"))
if i < 0 {
return nil, nil, fmt.Errorf("opentype: type1: the program has no eexec section")
}
clear = b[:i]
rest := b[i+len("eexec"):]
for len(rest) > 0 && isPSSpace(rest[0]) {
rest = rest[1:]
}
if isHexSection(rest) {
return clear, unhex(rest), nil
}
return clear, rest, nil
}
// joinPFB concatenates the segments of a .pfb file, which wraps each half in a
// six-byte header rather than running them together.
func joinPFB(b []byte) ([]byte, error) {
var out []byte
for len(b) >= 2 && b[0] == 0x80 {
switch b[1] {
case 3:
return out, nil
case 1, 2:
if len(b) < 6 {
return nil, fmt.Errorf("opentype: type1: a pfb segment header is cut short")
}
n := int(b[2]) | int(b[3])<<8 | int(b[4])<<16 | int(b[5])<<24
if n < 0 || 6+n > len(b) {
return nil, fmt.Errorf("opentype: type1: a pfb segment runs past the end")
}
out = append(out, b[6:6+n]...)
b = b[6+n:]
default:
return nil, fmt.Errorf("opentype: type1: unknown pfb segment %d", b[1])
}
}
return out, nil
}
// hexLookahead is how many bytes must all be hexadecimal digits before the
// section is read as hexadecimal. The specification says four, which is not
// enough: a binary section begins with four random bytes, and four random
// bytes are all hexadecimal digits about once in twenty thousand times — often
// enough to happen across a corpus, and when it does the whole font is lost.
// Sixteen makes that impossible while still holding for every real one, whose
// hexadecimal runs are lines long.
const hexLookahead = 16
// isHexSection reports whether the encrypted half was written as hexadecimal,
// which is what a .pfa file does.
func isHexSection(b []byte) bool {
n := 0
for _, c := range b {
if isPSSpace(c) {
continue
}
if !isHexDigit(c) {
return false
}
if n++; n == hexLookahead {
return true
}
}
return false
}
func isHexDigit(c byte) bool {
return c >= '0' && c <= '9' || c >= 'a' && c <= 'f' || c >= 'A' && c <= 'F'
}
func isPSSpace(c byte) bool {
return c == ' ' || c == '\t' || c == '\r' || c == '\n' || c == '\f' || c == 0
}
// unhex reads a hexadecimal section, ignoring whatever whitespace it was laid
// out with.
func unhex(b []byte) []byte {
out := make([]byte, 0, len(b)/2)
var hi byte
have := false
for _, c := range b {
if !isHexDigit(c) {
continue
}
v := hexVal(c)
if !have {
hi, have = v, true
continue
}
out = append(out, hi<<4|v)
have = false
}
return out
}
func hexVal(c byte) byte {
switch {
case c <= '9':
return c - '0'
case c <= 'F':
return c - 'A' + 10
default:
return c - 'a' + 10
}
}
// eexecDecrypt undoes the cipher a Type 1 program is written in, throwing away
// the leading bytes that exist only to randomise it.
func eexecDecrypt(data []byte, key uint16, skip int) []byte {
r := key
out := make([]byte, len(data))
for i, c := range data {
out[i] = c ^ byte(r>>8)
r = (uint16(c)+r)*eexecC1 + eexecC2
}
if len(out) < skip {
return nil
}
return out[skip:]
}
// readClear takes what the readable half says: the font's matrix, and the
// encoding it was written with.
func (t *type1Font) readClear(b []byte) {
if m, ok := readMatrix(b); ok {
t.fontMatrix = m
}
t.encoding = readType1Encoding(b)
}
// readMatrix reads /FontMatrix, which says how big the program's own units are.
func readMatrix(b []byte) ([6]float64, bool) {
var m [6]float64
i := bytes.Index(b, []byte("/FontMatrix"))
if i < 0 {
return m, false
}
open := bytes.IndexByte(b[i:], '[')
if open < 0 {
return m, false
}
close := bytes.IndexByte(b[i+open:], ']')
if close < 0 {
return m, false
}
fields := bytes.Fields(b[i+open+1 : i+open+close])
if len(fields) < 6 {
return m, false
}
for k := 0; k < 6; k++ {
v, err := strconv.ParseFloat(string(fields[k]), 64)
if err != nil {
return m, false
}
m[k] = v
}
if m[0] == 0 {
return m, false
}
return m, true
}
// readType1Encoding reads the /Encoding the program was written with: either a
// name saying to use the standard one, or an array built up by putting a name
// at a code at a time.
func readType1Encoding(b []byte) map[byte]string {
i := bytes.Index(b, []byte("/Encoding"))
if i < 0 {
return nil
}
rest := b[i+len("/Encoding"):]
// Whatever comes first decides: a name, or an array being built.
if j := bytes.Index(rest, []byte("StandardEncoding")); j >= 0 && j < 40 {
return nil
}
out := map[byte]string{}
for {
j := bytes.Index(rest, []byte("dup "))
if j < 0 {
break
}
rest = rest[j+4:]
fields := bytes.Fields(rest)
if len(fields) < 3 || len(fields[1]) < 2 || fields[1][0] != '/' || string(fields[2]) != "put" {
continue
}
code, err := strconv.Atoi(string(fields[0]))
if err != nil || code < 0 || code > 255 {
continue
}
out[byte(code)] = string(fields[1][1:])
}
if len(out) == 0 {
return nil
}
return out
}
// readPrivate takes what the encrypted half holds: how much of each charstring
// is padding, the subroutines, and the charstrings themselves.
func (t *type1Font) readPrivate(b []byte) error {
lenIV := defaultLenIV
if i := bytes.Index(b, []byte("/lenIV")); i >= 0 {
if fields := bytes.Fields(b[i+len("/lenIV"):]); len(fields) > 0 {
if v, err := strconv.Atoi(string(fields[0])); err == nil && v >= 0 && v < 16 {
lenIV = v
}
}
}
t.readSubrs(b, lenIV)
return t.readCharstrings(b, lenIV)
}
// readSubrs reads the /Subrs array: a numbered list of shared charstrings.
func (t *type1Font) readSubrs(b []byte, lenIV int) {
i := bytes.Index(b, []byte("/Subrs"))
if i < 0 {
return
}
rest := b[i+len("/Subrs"):]
fields := bytes.Fields(rest)
if len(fields) == 0 {
return
}
n, err := strconv.Atoi(string(fields[0]))
if err != nil || n < 0 || n > 65535 {
return
}
t.subrs = make([][]byte, n)
for k := 0; k < n; k++ {
j := bytes.Index(rest, []byte("dup "))
if j < 0 {
return
}
rest = rest[j+4:]
idx, data, next, ok := readBinaryEntry(rest, lenIV)
if !ok {
return
}
if num, err := strconv.Atoi(string(idx)); err == nil && num >= 0 && num < n {
t.subrs[num] = data
}
rest = next
}
}
// readCharstrings reads the /CharStrings dictionary: a glyph name and its
// charstring, over and over.
func (t *type1Font) readCharstrings(b []byte, lenIV int) error {
i := bytes.Index(b, []byte("/CharStrings"))
if i < 0 {
return fmt.Errorf("opentype: type1: the program has no CharStrings")
}
rest := b[i+len("/CharStrings"):]
// Past the dictionary's own preamble, every entry begins with a name.
if j := bytes.Index(rest, []byte("begin")); j >= 0 {
rest = rest[j+len("begin"):]
}
for {
j := bytes.IndexByte(rest, '/')
if j < 0 {
break
}
rest = rest[j+1:]
raw, data, next, ok := readBinaryEntry(rest, lenIV)
if !ok {
break
}
name := string(raw)
if _, seen := t.charstrings[name]; !seen && name != "" {
t.charstrings[name] = data
t.names = append(t.names, name)
}
rest = next
}
t.order()
return nil
}
// readBinaryEntry reads "<key> <length> RD <length bytes> ..." — the shape
// both a subroutine and a charstring arrive in. The operator that introduces
// the bytes is spelt RD by some programs and -| by others, and neither is a
// keyword worth insisting on: what matters is that a number is followed by
// something that is not a number, and then that many bytes.
func readBinaryEntry(b []byte, lenIV int) (key, data []byte, rest []byte, ok bool) {
name, after := psToken(b)
if name == nil {
return nil, nil, nil, false
}
lengthTok, after := psToken(after)
n, err := strconv.Atoi(string(lengthTok))
if err != nil || n < 0 || n > len(after) {
return nil, nil, nil, false
}
_, after = psToken(after) // RD, -|, or whatever this program calls it
if len(after) < 1+n {
return nil, nil, nil, false
}
// Exactly one space separates the operator from the bytes.
raw := after[1 : 1+n]
return name, eexecDecrypt(raw, charstringR, lenIV), after[1+n:], true
}
// psToken reads one whitespace-delimited token.
func psToken(b []byte) (tok, rest []byte) {
i := 0
for i < len(b) && isPSSpace(b[i]) {
i++
}
start := i
for i < len(b) && !isPSSpace(b[i]) {
i++
}
if start == i {
return nil, b[i:]
}
return b[start:i], b[i:]
}
// order puts .notdef first, so that glyph zero means what it means in every
// other format, and indexes the rest where they fell.
func (t *type1Font) order() {
for i, n := range t.names {
if n == ".notdef" && i != 0 {
t.names = append([]string{n}, append(t.names[:i:i], t.names[i+1:]...)...)
break
}
}
for i, n := range t.names {
t.byName[n] = GlyphIndex(i)
}
}
// metrics is what each glyph's charstring says it is wide, which for a Type 1
// program is the operand of its very first operator.
func (t *type1Font) metrics() (advances, lsbs []int) {
advances = make([]int, len(t.names))
lsbs = make([]int, len(t.names))
for i, name := range t.names {
sb, w, ok := type1Width(t.charstrings[name])
if !ok {
continue
}
advances[i] = int(math.Round(w))
lsbs[i] = int(math.Round(sb))
}
return advances, lsbs
}
// glyphByCode maps a byte through the encoding the program was written with.
func (t *type1Font) glyphByCode(code byte) (GlyphIndex, bool) {
name, ok := t.encoding[code]
if !ok {
if t.encoding != nil {
return 0, false
}
sid, in := standardEncodingSID(int(code))
if !in || int(sid) >= nStdStrings {
return 0, false
}
name = cffStandardStrings[sid]
}
gid, ok := t.byName[name]
return gid, ok
}
// glyphName is what the program calls a glyph.
func (t *type1Font) glyphName(gid int) (string, bool) {
if gid < 0 || gid >= len(t.names) {
return "", false
}
return t.names[gid], true
}
vendor/github.com/go-opentype/opentype/type1exec.go
Generated Vendored
+493
View File
@@ -0,0 +1,493 @@
// Copyright (c) 2026, the go-opentype authors
// All rights reserved.
//
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import "fmt"
// The Type 1 charstring operators. They are not the Type 2 ones: the two
// formats share a lineage and very little else.
const (
t1HStem = 1
t1VStem = 3
t1VMoveTo = 4
t1RLineTo = 5
t1HLineTo = 6
t1VLineTo = 7
t1RRCurveTo = 8
t1ClosePath = 9
t1CallSubr = 10
t1Return = 11
t1Escape = 12
t1HSbw = 13
t1EndChar = 14
t1RMoveTo = 21
t1HMoveTo = 22
t1VHCurveTo = 30
t1HVCurveTo = 31
)
// The escaped operators, reached through operator 12.
const (
t1DotSection = 0
t1VStem3 = 1
t1HStem3 = 2
t1Seac = 6
t1Sbw = 7
t1Div = 12
t1CallOtherSubr = 16
t1Pop = 17
t1SetCurrentPoint = 33
maxT1Depth = 30
maxT1Stack = 48
maxT1PostScriptSize = 32
)
// t1machine runs one Type 1 charstring.
type t1machine struct {
f *type1Font
stack []float64
ps []float64 // the interpreter's other stack, which othersubrs use
contours []contour
cur contour
x, y float64
sbx, sby float64
width float64
haveWidth bool
// flex collects the seven points an othersubr-driven curve is built from;
// while it is running, a move records a point instead of starting a
// contour.
inFlex bool
flexPts []outlinePoint
seacDepth int
done bool
}
// outline draws a glyph of a Type 1 program.
func (t *type1Font) outline(gid int) ([]contour, error) {
name, ok := t.glyphName(gid)
if !ok {
return nil, fmt.Errorf("opentype: type1: glyph index %d out of range", gid)
}
return t.outlineByName(name, 0)
}
// outlineByName draws the glyph a name stands for.
func (t *type1Font) outlineByName(name string, seacDepth int) ([]contour, error) {
cs, ok := t.charstrings[name]
if !ok {
return nil, fmt.Errorf("opentype: type1: no glyph called %q", name)
}
m := &t1machine{f: t, seacDepth: seacDepth}
if err := m.run(cs, 0); err != nil {
return nil, err
}
m.finishContour()
return m.contours, nil
}
// type1Width reads what a charstring says its glyph is wide, which is the
// operand of hsbw or sbw — always the first operator of a Type 1 glyph.
func type1Width(cs []byte) (sb, w float64, ok bool) {
var stack []float64
for i := 0; i < len(cs); {
v, next, isNum, valid := t1Operand(cs, i)
if !valid {
return 0, 0, false
}
if isNum {
stack = append(stack, v)
i = next
continue
}
switch cs[i] {
case t1HSbw:
if len(stack) < 2 {
return 0, 0, false
}
return stack[0], stack[1], true
case t1Escape:
if i+1 < len(cs) && cs[i+1] == t1Sbw {
if len(stack) < 4 {
return 0, 0, false
}
return stack[0], stack[2], true
}
return 0, 0, false
default:
return 0, 0, false
}
}
return 0, 0, false
}
// t1Operand reads one charstring byte, saying whether it was a number and
// where the next one is.
func t1Operand(cs []byte, i int) (v float64, next int, isNum, ok bool) {
b := cs[i]
switch {
case b >= 32 && b <= 246:
return float64(int(b) - 139), i + 1, true, true
case b >= 247 && b <= 250:
if i+1 >= len(cs) {
return 0, 0, false, false
}
return float64((int(b)-247)*256 + int(cs[i+1]) + 108), i + 2, true, true
case b >= 251 && b <= 254:
if i+1 >= len(cs) {
return 0, 0, false, false
}
return float64(-(int(b)-251)*256 - int(cs[i+1]) - 108), i + 2, true, true
case b == 255:
if i+4 >= len(cs) {
return 0, 0, false, false
}
n := int32(uint32(cs[i+1])<<24 | uint32(cs[i+2])<<16 | uint32(cs[i+3])<<8 | uint32(cs[i+4]))
return float64(n), i + 5, true, true
}
return 0, i, false, true
}
// run executes a charstring or a subroutine.
func (m *t1machine) run(cs []byte, depth int) error {
if depth > maxT1Depth {
return fmt.Errorf("opentype: type1: subroutine nesting too deep")
}
for i := 0; i < len(cs); {
v, next, isNum, ok := t1Operand(cs, i)
if !ok {
return fmt.Errorf("opentype: type1: a charstring operand runs past the end")
}
if isNum {
if len(m.stack) >= maxT1Stack {
return fmt.Errorf("opentype: type1: too many operands")
}
m.stack = append(m.stack, v)
i = next
continue
}
op := cs[i]
i++
if op == t1Escape {
if i >= len(cs) {
return fmt.Errorf("opentype: type1: a charstring ends in an escape")
}
esc := cs[i]
i++
stop, err := m.escaped(esc, depth)
if err != nil {
return err
}
if stop {
return nil
}
continue
}
stop, err := m.operator(op, depth)
if err != nil {
return err
}
if stop {
return nil
}
}
return nil
}
// operator runs one plain operator, reporting whether the charstring is over.
func (m *t1machine) operator(op byte, depth int) (bool, error) {
s := m.stack
switch op {
case t1HStem, t1VStem:
m.clear()
case t1HSbw:
if len(s) >= 2 {
m.sbx, m.width, m.haveWidth = s[0], s[1], true
m.x, m.y = m.sbx, 0
}
m.clear()
case t1RMoveTo:
if len(s) >= 2 {
m.moveTo(s[len(s)-2], s[len(s)-1])
}
m.clear()
case t1HMoveTo:
if len(s) >= 1 {
m.moveTo(s[len(s)-1], 0)
}
m.clear()
case t1VMoveTo:
if len(s) >= 1 {
m.moveTo(0, s[len(s)-1])
}
m.clear()
case t1RLineTo:
if len(s) >= 2 {
m.lineTo(s[0], s[1])
}
m.clear()
case t1HLineTo:
if len(s) >= 1 {
m.lineTo(s[0], 0)
}
m.clear()
case t1VLineTo:
if len(s) >= 1 {
m.lineTo(0, s[0])
}
m.clear()
case t1RRCurveTo:
if len(s) >= 6 {
m.rrcurveTo(s[0], s[1], s[2], s[3], s[4], s[5])
}
m.clear()
case t1HVCurveTo:
if len(s) >= 4 {
m.rrcurveTo(s[0], 0, s[1], s[2], 0, s[3])
}
m.clear()
case t1VHCurveTo:
if len(s) >= 4 {
m.rrcurveTo(0, s[0], s[1], s[2], s[3], 0)
}
m.clear()
case t1ClosePath:
m.closePath()
m.clear()
case t1CallSubr:
return false, m.callSubr(depth)
case t1Return:
return true, nil
case t1EndChar:
m.done = true
return true, nil
default:
// An operator this format has not got: the rest of the charstring
// cannot be trusted, but what has been drawn so far can.
return true, nil
}
return false, nil
}
// escaped runs one of the operators reached through the escape byte.
func (m *t1machine) escaped(esc byte, depth int) (bool, error) {
s := m.stack
switch esc {
case t1DotSection, t1VStem3, t1HStem3:
m.clear()
case t1Sbw:
if len(s) >= 4 {
m.sbx, m.sby, m.width, m.haveWidth = s[0], s[1], s[2], true
m.x, m.y = m.sbx, m.sby
}
m.clear()
case t1Div:
if len(s) >= 2 && s[len(s)-1] != 0 {
m.stack = append(s[:len(s)-2], s[len(s)-2]/s[len(s)-1])
}
case t1Seac:
return true, m.seac()
case t1CallOtherSubr:
return false, m.callOtherSubr()
case t1Pop:
if len(m.ps) > 0 {
m.stack = append(m.stack, m.ps[len(m.ps)-1])
m.ps = m.ps[:len(m.ps)-1]
} else {
m.stack = append(m.stack, 0)
}
case t1SetCurrentPoint:
if len(s) >= 2 {
m.x, m.y = s[0], s[1]
}
m.clear()
default:
return true, nil
}
return false, nil
}
// callSubr runs a numbered subroutine.
func (m *t1machine) callSubr(depth int) error {
if len(m.stack) == 0 {
return nil
}
n := int(m.stack[len(m.stack)-1])
m.stack = m.stack[:len(m.stack)-1]
if n < 0 || n >= len(m.f.subrs) || m.f.subrs[n] == nil {
return nil
}
return m.run(m.f.subrs[n], depth+1)
}
// callOtherSubr handles the four subroutines a Type 1 program calls out to the
// interpreter for. Numbers 0 to 2 build a flex — a curve so shallow it is
// drawn as one — and number 3 asks which hints to use, which nothing here
// needs. Anything else is answered by handing its arguments straight back, so
// that a program using an othersubr of its own carries on rather than stops.
func (m *t1machine) callOtherSubr() error {
if len(m.stack) < 2 {
m.clear()
return nil
}
which := int(m.stack[len(m.stack)-1])
n := int(m.stack[len(m.stack)-2])
m.stack = m.stack[:len(m.stack)-2]
if n < 0 || n > len(m.stack) {
m.clear()
return nil
}
args := m.stack[len(m.stack)-n:]
m.stack = m.stack[:len(m.stack)-n]
switch which {
case 0: // the flex is over: draw the two curves its seven points describe
m.endFlex()
// The convention is that the two coordinates it ends at are read back
// with two pops.
m.ps = append(m.ps[:0], m.y, m.x)
case 1: // a flex begins
m.inFlex = true
m.flexPts = m.flexPts[:0]
case 2: // a flex point has been collected by the move that followed
case 3:
// Hint replacement. The program pushes the number of the subroutine
// holding the hints it wants, and expects to read it straight back
// with a pop and then call it. Handing back anything else — the
// othersubr's own number, say — calls the wrong subroutine, and the
// glyph comes out empty with nothing to say why.
if len(args) > 0 {
m.ps = append(m.ps[:0], args[0])
} else {
m.ps = append(m.ps[:0], 3)
}
default:
// Hand the arguments back in the order the pops will take them.
m.ps = m.ps[:0]
for i := len(args) - 1; i >= 0; i-- {
m.ps = append(m.ps, args[i])
}
if len(m.ps) > maxT1PostScriptSize {
m.ps = m.ps[:maxT1PostScriptSize]
}
}
return nil
}
// endFlex turns the seven collected points into the two curves they stand for.
func (m *t1machine) endFlex() {
m.inFlex = false
if len(m.flexPts) < 7 {
return
}
p := m.flexPts[len(m.flexPts)-7:]
// The first of the seven is the reference point the shape is measured
// against and is not drawn.
m.curveTo(p[1].x, p[1].y, p[2].x, p[2].y, p[3].x, p[3].y)
m.curveTo(p[4].x, p[4].y, p[5].x, p[5].y, p[6].x, p[6].y)
}
// seac composes an accented glyph out of two others, named by their Standard
// Encoding codes.
func (m *t1machine) seac() error {
if m.seacDepth >= maxSeacDepth {
return fmt.Errorf("opentype: type1: accent composition nesting too deep")
}
s := m.stack
if len(s) < 5 {
m.clear()
return nil
}
asb, adx, ady, bchar, achar := s[0], s[1], s[2], int(s[3]), int(s[4])
m.clear()
base, err := m.component(bchar)
if err != nil {
return err
}
accent, err := m.component(achar)
if err != nil {
return err
}
m.finishContour()
m.contours = append(m.contours, base...)
// The accent is placed relative to the base's own left side bearing.
dx, dy := m.sbx-asb+adx, ady
for _, c := range accent {
nc := make(contour, len(c))
for i, p := range c {
nc[i] = outlinePoint{x: p.x + dx, y: p.y + dy, on: p.on}
}
m.contours = append(m.contours, nc)
}
m.done = true
return nil
}
// component draws one half of an accented glyph, named by a Standard Encoding
// code.
func (m *t1machine) component(code int) ([]contour, error) {
sid, ok := standardEncodingSID(code)
if !ok || int(sid) >= nStdStrings {
return nil, fmt.Errorf("opentype: type1: code %d is not in the standard encoding", code)
}
return m.f.outlineByName(cffStandardStrings[sid], m.seacDepth+1)
}
// clear empties the operand stack, which every operator that consumes one does.
func (m *t1machine) clear() { m.stack = m.stack[:0] }
// moveTo begins a contour, or collects a flex point while one is being built.
func (m *t1machine) moveTo(dx, dy float64) {
m.x += dx
m.y += dy
if m.inFlex {
m.flexPts = append(m.flexPts, outlinePoint{x: m.x, y: m.y, on: true})
return
}
m.finishContour()
m.cur = contour{{x: m.x, y: m.y, on: true}}
}
// lineTo adds a straight segment.
func (m *t1machine) lineTo(dx, dy float64) {
m.x += dx
m.y += dy
m.cur = append(m.cur, outlinePoint{x: m.x, y: m.y, on: true})
}
// rrcurveTo adds a curve given by three relative steps.
func (m *t1machine) rrcurveTo(dx1, dy1, dx2, dy2, dx3, dy3 float64) {
x1, y1 := m.x+dx1, m.y+dy1
x2, y2 := x1+dx2, y1+dy2
m.curveTo(x1, y1, x2, y2, x2+dx3, y2+dy3)
}
// curveTo flattens a cubic curve into the on-curve points a contour is made of.
func (m *t1machine) curveTo(x1, y1, x2, y2, x3, y3 float64) {
x0, y0 := m.x, m.y
for k := 1; k <= cffCurveSteps; k++ {
t := float64(k) / float64(cffCurveSteps)
u := 1 - t
bx := u*u*u*x0 + 3*u*u*t*x1 + 3*u*t*t*x2 + t*t*t*x3
by := u*u*u*y0 + 3*u*u*t*y1 + 3*u*t*t*y2 + t*t*t*y3
m.cur = append(m.cur, outlinePoint{x: bx, y: by, on: true})
}
m.x, m.y = x3, y3
}
// closePath ends the contour in progress. A Type 1 program says so out loud,
// where later formats leave it implied.
func (m *t1machine) closePath() {
m.finishContour()
}
// finishContour puts the contour under construction with the others.
func (m *t1machine) finishContour() {
if len(m.cur) > 0 {
m.contours = append(m.contours, m.cur)
m.cur = nil
}
}
vendor/github.com/go-opentype/opentype/vertical.go
Generated Vendored
+193
View File
@@ -0,0 +1,193 @@
// Copyright (c) 2026 the go-opentype/opentype authors. All rights reserved.
// Use of this source code is governed by a BSD-3-Clause license that can be
// found in the LICENSE file at the root of this repository.
package opentype
import "fmt"
// This file adds vertical writing-mode metrics: the data needed to lay glyphs
// out top-to-bottom for CJK tategaki and other vertical text. Three optional
// tables carry it:
//
// - vhea: the vertical header (ascender/descender/lineGap for vertical layout
// plus numOfLongVerMetrics, the horizontal analogue of hhea);
// - vmtx: per-glyph advanceHeight and topSideBearing (analogous to hmtx);
// - VORG: the Vertical Origin table, a default and per-glyph y coordinate of
// each glyph's vertical origin.
//
// Absence of these tables is not an error; it simply means vertical metrics are
// unavailable (HasVerticalMetrics reports false) and the caller falls back to
// the em square.
//
// Selecting vertical GLYPH FORMS (the OpenType 'vert'/'vrt2' features that swap
// horizontal glyphs for their upright vertical variants) is a shaping concern
// handled through GSUB (see gsub.go and Face.Shape). This file supplies the
// vertical METRICS and origin those laid-out glyphs are positioned with; it does
// not itself perform substitution.
// parseVertical decodes the optional vhea, vmtx and VORG tables. Each is
// independent, but vmtx is meaningful only alongside vhea (which supplies
// numOfLongVerMetrics), so a vmtx without a vhea is ignored. A malformed table
// that is present is reported as an error so a corrupt font fails cleanly.
func (f *Font) parseVertical(tables map[string][]byte) error {
if b, ok := tables["vhea"]; ok {
if err := f.parseVhea(b); err != nil {
return err
}
}
if b, ok := tables["vmtx"]; ok && f.hasVhea {
if err := f.parseVmtx(b); err != nil {
return err
}
}
if b, ok := tables["VORG"]; ok {
if err := f.parseVORG(b); err != nil {
return err
}
}
return nil
}
// parseVhea reads the vertical metrics and numOfLongVerMetrics from vhea. Its
// layout mirrors hhea: vertTypoAscender/Descender/LineGap are int16 at offsets
// 4/6/8 and numOfLongVerMetrics is the uint16 at offset 34.
func (f *Font) parseVhea(b []byte) error {
if len(b) < 36 {
return fmt.Errorf("opentype: vhea table: %w", errTruncated)
}
f.vertAscender = int(sbe16(b[4:]))
f.vertDescender = int(sbe16(b[6:]))
f.vertLineGap = int(sbe16(b[8:]))
f.numOfLongVerMetrics = int(be16(b[34:]))
if f.numOfLongVerMetrics == 0 || f.numOfLongVerMetrics > f.numGlyphs {
return fmt.Errorf("opentype: vhea table: bad numOfLongVerMetrics %d", f.numOfLongVerMetrics)
}
f.hasVhea = true
return nil
}
// parseVmtx expands the vmtx table into per-glyph advanceHeight and top side
// bearing slices, mirroring parseHmtx. The first numOfLongVerMetrics entries are
// (advanceHeight, topSideBearing) pairs; the trailing glyphs share the last
// entry's advance and carry their own top side bearing.
func (f *Font) parseVmtx(b []byte) error {
n := f.numOfLongVerMetrics
need := 4*n + 2*(f.numGlyphs-n)
if len(b) < need {
return fmt.Errorf("opentype: vmtx table: %w", errTruncated)
}
f.vertAdvances = make([]int, f.numGlyphs)
f.tsbs = make([]int, f.numGlyphs)
lastAdvance := 0
for i := 0; i < f.numGlyphs; i++ {
if i < n {
lastAdvance = int(be16(b[i*4:]))
f.vertAdvances[i] = lastAdvance
f.tsbs[i] = int(sbe16(b[i*4+2:]))
} else {
f.vertAdvances[i] = lastAdvance
f.tsbs[i] = int(sbe16(b[4*n+2*(i-n):]))
}
}
f.hasVmtx = true
return nil
}
// parseVORG decodes the Vertical Origin table: an 8-byte header carrying the
// defaultVertOriginY (int16 at offset 4) and numVertOriginYMetrics (uint16 at
// offset 6), followed by that many 4-byte {glyphIndex uint16, vertOriginY int16}
// records overriding the default for specific glyphs.
func (f *Font) parseVORG(b []byte) error {
if len(b) < 8 {
return fmt.Errorf("opentype: VORG table: %w", errTruncated)
}
f.vorgDefault = int(sbe16(b[4:]))
count := int(be16(b[6:]))
if len(b) < 8+4*count {
return fmt.Errorf("opentype: VORG table: %w", errTruncated)
}
f.vorg = make(map[GlyphIndex]int, count)
for i := 0; i < count; i++ {
rec := b[8+4*i:]
f.vorg[GlyphIndex(be16(rec))] = int(sbe16(rec[2:]))
}
f.hasVORG = true
return nil
}
// HasVerticalMetrics reports whether the font carries the vertical header and
// per-glyph vertical advances (vhea and vmtx) required to lay text out
// top-to-bottom. When it returns false, VerticalAdvance falls back to the em
// square and VerticalMetrics returns zeroes.
func (f *Font) HasVerticalMetrics() bool {
return f.hasVhea && f.hasVmtx
}
// verticalAdvance returns glyph gid's advance height in font units. When the
// font has no vmtx table it falls back to one em (unitsPerEm), the conventional
// full-width advance for vertical CJK layout.
func (f *Font) verticalAdvance(gid GlyphIndex) int {
if !f.hasVmtx {
return f.unitsPerEm
}
return f.vertAdvances[gid]
}
// verticalOriginY returns glyph gid's vertical origin y in font units and
// whether the font supplies one (a VORG table). A glyph without its own record
// takes the table's default vertical origin.
func (f *Font) verticalOriginY(gid GlyphIndex) (int, bool) {
if !f.hasVORG {
return 0, false
}
if y, ok := f.vorg[gid]; ok {
return y, true
}
return f.vorgDefault, true
}
// VerticalAdvance returns the vertical advance (advance height) of r in pixels,
// or 0 if the rune is not mapped by the font's cmap. When the font has no vmtx
// table the advance is one em, scaled to the face's size.
//
// This is the top-to-bottom analogue of Advance. Choosing the upright vertical
// glyph form for r (the 'vert'/'vrt2' features) is a separate shaping step done
// through GSUB; VerticalAdvance measures whichever glyph the cmap maps.
func (fc *Face) VerticalAdvance(r rune) int {
gid, ok := fc.font.GlyphIndex(r)
if !ok {
return 0
}
adv := float64(fc.font.verticalAdvance(gid))
if fc.varCoords != nil {
adv += fc.font.verticalAdvanceDelta(int(gid), fc.varNorm)
}
return roundInt(adv * fc.scale)
}
// VerticalMetrics returns the face's vertical-layout line metrics in pixels: the
// vhea vertTypoAscender, vertTypoDescender and vertTypoLineGap scaled to the
// face's size. All three are zero when the font has no vhea table.
func (fc *Face) VerticalMetrics() (ascent, descent, lineGap int) {
s := fc.scale
return roundInt(float64(fc.font.vertAscender) * s),
roundInt(float64(fc.font.vertDescender) * s),
roundInt(float64(fc.font.vertLineGap) * s)
}
// VerticalOrigin returns the y coordinate of r's vertical origin in pixels and
// whether the font supplies one (a VORG table). The origin locates the glyph
// relative to the pen when advancing top-to-bottom; ok is false when r is
// unmapped or the font has no VORG table.
func (fc *Face) VerticalOrigin(r rune) (int, bool) {
gid, mapped := fc.font.GlyphIndex(r)
if !mapped {
return 0, false
}
y, ok := fc.font.verticalOriginY(gid)
if !ok {
return 0, false
}
return roundInt(float64(y) * fc.scale), true
}
vendor/github.com/go-pdfkit/extract/.gitignore
Generated
+1
View File
@@ -0,0 +1 @@
coverage.out
vendor/github.com/go-pdfkit/extract/LICENSE
Generated Vendored
+29
View File
@@ -0,0 +1,29 @@
BSD 3-Clause License
Copyright (c) 2026 the go-pdfkit/render authors
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
vendor/github.com/go-pdfkit/extract/README.md
Generated Vendored
+91
View File
@@ -0,0 +1,91 @@
# extract
[![CI](https://github.com/go-pdfkit/extract/actions/workflows/ci.yml/badge.svg)](https://github.com/go-pdfkit/extract/actions/workflows/ci.yml)
[![Go Reference](https://pkg.go.dev/badge/github.com/go-pdfkit/extract.svg)](https://pkg.go.dev/github.com/go-pdfkit/extract)
[![License: BSD-3-Clause](https://img.shields.io/badge/License-BSD--3--Clause-blue.svg)](LICENSE)
[![Coverage](https://img.shields.io/badge/coverage-100%25-brightgreen.svg)](#how-it-is-checked)
Reads a PDF page back: the text on it, with where each piece of it sits, and
the images it places.
```go
text, err := extract.Text(doc, 1) // the page as words, lines put back together
runs, err := extract.Runs(doc, 1) // every piece, with its place and its size
imgs, err := extract.Images(doc, 1) // every picture, and where it lands
```
Pure Go, no C. It builds for `GOOS=js/wasm` and every architecture the fleet
targets.
## What a page actually holds
Not text. A page holds instructions for drawing glyphs, and the text is what
those glyphs were meant to say — which the document says only if it was asked
to. Where it does not, this says so rather than guessing: a run whose codes
could not be worked out comes back marked `Unreadable`, so a caller can tell
*this page says nothing* from *this page could not be read*.
Three things are tried, in order:
1. the font's `/ToUnicode` map, which is the document's own word on what its
text says;
2. the name its encoding gives the code — but only a name the document chose
itself, since a **symbolic** font read through an assumed encoding gives
the wrong letter with nothing to say so;
3. what the **embedded font program** calls the code, read with
[`go-opentype`](https://github.com/go-opentype/opentype) — which is the
honest answer for a symbolic font, and the reason a page of mathematics
comes back as mathematics.
Text drawn in the mode that puts no ink on the page — how a scanner writes
what it read underneath the picture it read it from — is returned and marked
`Invisible`.
## Images
A picture comes back as the file holds it, with the box it covers on the page.
A JPEG comes out as a JPEG: re-encoding it would lose something for nothing.
Turning the other kinds into pixels means reading their colour space, which is
[`render`](https://github.com/go-pdfkit/render)'s work rather than this
package's.
## How it is checked
100% of statements, including every branch that handles a page saying
something it should not.
Then on the corpus — 118 833 real PDFs, 121 946 pages:
```
pages read 121946
with text 94865 (77.8%)
saying nothing 27081, of which 4812 did draw glyphs
characters 33976414 (17272226 of them letters)
runs 10797045
unreadable 203607 (1.89%)
invisible 29917
images 300685
panics 0
```
The 22% of pages that say nothing are figures — this corpus is arXiv's, and
most of it is plots. Of the pages that *did* draw glyphs and still said
nothing, nearly all are Type 3 fonts with no `/ToUnicode`: dvips bitmap fonts
whose glyphs are called `a1` and `s32`, which no reader can turn back into
letters.
## The rest of the family
[**reader**](https://github.com/go-pdfkit/reader) reads and writes the format ·
[**ops**](https://github.com/go-pdfkit/ops) is the verbs and the `pdfops` command ·
[**render**](https://github.com/go-pdfkit/render) turns a page into pixels ·
[**pdffont**](https://github.com/go-pdfkit/pdffont) is what a document says about a font ·
[**extract**](https://github.com/go-pdfkit/extract) reads a page back ·
[**coedit**](https://github.com/go-pdfkit/coedit) is a PDF several people edit at once ·
[**app**](https://github.com/go-pdfkit/app) is the browser workbench.
Documentation for all of it: <https://go-pdfkit.github.io/docs/>
## Licence
BSD-3-Clause.
vendor/github.com/go-pdfkit/extract/extract.go
Generated Vendored
+176
View File
@@ -0,0 +1,176 @@
// Package extract reads a PDF page back: the text on it, with where each piece
// of it sits, and the images it places.
//
// A page does not hold text. It holds instructions for drawing glyphs, and the
// text is what those glyphs were meant to say — which the document says only
// if it was asked to. Where it does not, this says so rather than guessing:
// a run whose characters could not be worked out comes back marked, so that a
// caller can tell "this page says nothing" from "this page could not be read".
package extract
import (
"sort"
"strings"
"github.com/go-pdfkit/pdffont"
"github.com/go-pdfkit/reader"
)
// A Run is a piece of text drawn in one go: one font, one size, one place.
type Run struct {
// Text is what the run says.
Text string
// X and Y are where it starts, in the page's own coordinates — up from
// the bottom left corner, in points.
X, Y float64
// Width is how far the pen moved drawing it, and Size how tall the text
// is, both in points.
Width, Size float64
// Font is the name the page's resources give the font.
Font string
// Space is how wide a space would be here, which is what says whether a
// gap between two runs is a word break or only kerning.
Space float64
// Invisible reports text drawn in the mode that puts no ink on the page.
// It is how a scanner puts what it read underneath the picture it read it
// from, so it is text worth having — and worth knowing about.
Invisible bool
// Unreadable reports a run some of whose codes said nothing about which
// characters they were: the font carried no map and named its glyphs
// after numbers. What Text holds is then only the part that could be read.
Unreadable bool
}
// Text is everything the page says, in reading order, with the lines put back
// together.
func Text(d *reader.Document, page int) (string, error) {
runs, err := Runs(d, page)
if err != nil {
return "", err
}
return Assemble(runs), nil
}
// Runs is every piece of text on the page, in the order it was drawn.
func Runs(d *reader.Document, page int) ([]Run, error) {
e, err := walk(d, page, false)
if err != nil {
return nil, err
}
return e.runs, nil
}
// walk reads a page's content stream once, keeping what it finds.
//
// A page whose content decoded only part of the way is read as far as it went:
// 263 streams in 212 of the 1 633 real forms in the corpus cannot be decoded
// cleanly, and half a page of text is worth more to somebody searching than
// none of it. A page that decoded no bytes at all is a different answer and
// stays an error — a conversion that silently produces an empty document from
// an unreadable page is worse than one that says it could not read it, and
// go-pdfkit/latex relies on being told.
//
// wantPictures says whether the caller is going to look at the pictures. A
// page's images have nothing to do with its text, and undoing their filters is
// the most expensive thing on the page: one arXiv figure holds 378 MB of image
// once decompressed, so asking that page what it *said* used to cost 1 094 MB
// and 1.3 seconds spent inflating pictures the answer throws away. Where they
// land is still noted either way, since that costs a matrix multiply; only the
// bytes are left alone.
func walk(d *reader.Document, page int, wantPictures bool) (*extractor, error) {
dec, err := d.PageContentDecoded(page)
if err != nil {
return nil, err
}
if len(dec.Data) == 0 && dec.Cause != nil {
return nil, dec.Cause
}
content := dec.Data
p, _ := d.Page(page)
resources, _ := d.GetDict(p, "Resources")
e := &extractor{doc: d, fonts: map[int]*pdffont.Font{}, wantPictures: wantPictures}
e.run(content, resources, initialState(d, p), 0)
return e, nil
}
// Assemble puts runs back together as lines of text: everything on one
// baseline in the order it sits across the page, with a space wherever the
// gap between two pieces is wide enough to be one.
func Assemble(runs []Run) string {
if len(runs) == 0 {
return ""
}
lines := groupIntoLines(runs)
var out strings.Builder
for i, line := range lines {
if i > 0 {
out.WriteByte('\n')
}
out.WriteString(joinLine(line))
}
return out.String()
}
// groupIntoLines gathers runs that sit on the same baseline, in the order they
// appear down the page.
func groupIntoLines(runs []Run) [][]Run {
sorted := append([]Run{}, runs...)
sort.SliceStable(sorted, func(i, j int) bool {
if !sameLine(sorted[i], sorted[j]) {
return sorted[i].Y > sorted[j].Y
}
return sorted[i].X < sorted[j].X
})
var lines [][]Run
for _, r := range sorted {
if n := len(lines); n > 0 && sameLine(lines[n-1][0], r) {
lines[n-1] = append(lines[n-1], r)
continue
}
lines = append(lines, []Run{r})
}
return lines
}
// sameLine reports whether two runs sit on the same baseline. A line is
// allowed to wobble by a fraction of its own height, which is what a
// superscript or a slightly different font does.
func sameLine(a, b Run) bool {
tol := a.Size * 0.3
if tol <= 0 {
tol = 1
}
d := a.Y - b.Y
return d < tol && d > -tol
}
// joinLine writes one line out, putting a space wherever the gap between two
// pieces is wide enough to be one.
func joinLine(line []Run) string {
var out strings.Builder
for i, r := range line {
if i > 0 {
prev := line[i-1]
gap := r.X - (prev.X + prev.Width)
if wantsSpace(prev, r, gap) {
out.WriteByte(' ')
}
}
out.WriteString(r.Text)
}
return strings.TrimRight(out.String(), " ")
}
// wantsSpace decides whether a gap between two pieces of a line is a word
// break. A space of its own is never doubled, and a gap of about a third of a
// space is where kerning ends and words begin.
func wantsSpace(prev, next Run, gap float64) bool {
if strings.HasSuffix(prev.Text, " ") || strings.HasPrefix(next.Text, " ") {
return false
}
space := prev.Space
if space <= 0 {
space = prev.Size * 0.25
}
return gap > space*0.3
}
vendor/github.com/go-pdfkit/extract/image.go
Generated Vendored
+188
View File
@@ -0,0 +1,188 @@
package extract
import (
"github.com/go-pdfkit/reader"
)
// An Image is a picture the page places, and where it puts it.
//
// What comes back is the picture as the file holds it — the bytes and the
// dictionary describing them — rather than pixels. A JPEG comes out as a JPEG,
// which is what anyone pulling pictures out of a document wants: re-encoding
// it would lose something for nothing. Turning the other kinds into pixels
// means reading their colour space, which is the renderer's work rather than
// this package's.
type Image struct {
// Name is what the page's resources call it, or empty for one written
// into the content stream.
Name string
// Width and Height are the picture's own, in samples.
Width, Height int
// X and Y are where its bottom left corner lands on the page, and
// DrawnWidth and DrawnHeight how large it is drawn, in points. A picture
// turned or skewed by the page still reports the box it covers.
X, Y float64
DrawnWidth, DrawnHeight float64
// Filter is the last filter still on the data: DCTDecode for a JPEG that
// comes out as one, JPXDecode for a JPEG 2000, and empty for data this
// has unfiltered into plain samples.
Filter reader.Name
// Data is the picture's bytes, with every filter this understands undone.
Data []byte
// Dict is the image's own dictionary, which says how to read the data:
// its colour space, how many bits a sample takes, and whether it is a
// mask.
Dict reader.Dict
// Inline reports a picture written into the content stream rather than
// held as a resource of its own.
Inline bool
}
// Images is every picture the page places, in the order it places them. The
// same picture placed twice comes back twice, since where it lands is part of
// what is being asked for.
func Images(d *reader.Document, page int) ([]Image, error) {
e, err := walk(d, page, true)
if err != nil {
return nil, err
}
return e.images, nil
}
// doXObject handles the Do operator: a form is walked into, and a picture is
// noted where it lands.
func (e *extractor) doXObject(g *state, operands []reader.Object, resources reader.Dict, depth int) {
if len(operands) == 0 {
return
}
name, ok := reader.ToName(operands[0])
if !ok {
return
}
xobjects, ok := e.doc.GetDict(resources, "XObject")
if !ok {
return
}
stream, ok := reader.ToStream(resolve(e.doc, xobjects.Get(name)))
if !ok {
return
}
switch sub, _ := reader.ToName(resolve(e.doc, stream.Dict.Get("Subtype"))); sub {
case "Form":
e.doForm(g, stream, resources, depth)
case "Image":
e.noteImage(g, string(name), stream.Dict, stream.Raw, false)
}
}
// doForm walks into a form, which is a piece of a page held apart and drawn
// where the page says.
func (e *extractor) doForm(g *state, stream *reader.Stream, resources reader.Dict, depth int) {
data, img, err := e.doc.DecodeStream(stream)
if err != nil || img != "" {
return
}
inner := *g
if m := floatArray(e.doc, stream.Dict.Get("Matrix")); len(m) >= 6 {
inner.ctm = matrix{m[0], m[1], m[2], m[3], m[4], m[5]}.mul(g.ctm)
}
own, ok := e.doc.GetDict(stream.Dict, "Resources")
if !ok {
own = resources
}
e.run(data, own, inner, depth+1)
}
// inlineImage notes a picture written into the content stream.
func (e *extractor) inlineImage(g *state, im *reader.InlineImage) {
if im == nil {
return
}
e.noteImage(g, "", im.Expanded(), im.Raw, true)
}
// noteImage works out where a picture lands and records it. A picture occupies
// the unit square of the transform in force, so its corners are where that
// square's are.
func (e *extractor) noteImage(g *state, name string, dict reader.Dict, raw []byte, inline bool) {
minX, minY, maxX, maxY := unitSquareBox(g.ctm)
img := Image{
Name: name,
Width: int(intOf(resolve(e.doc, dict.Get("Width")), 0)),
Height: int(intOf(resolve(e.doc, dict.Get("Height")), 0)),
X: minX,
Y: minY,
DrawnWidth: maxX - minX,
DrawnHeight: maxY - minY,
Dict: dict,
Inline: inline,
}
// A caller after the text is not after the pixels, and undoing a
// picture's filters to hand it nothing is the most expensive thing a page
// can be made to do.
if e.wantPictures {
data, filter, err := reader.Decode(dict, raw, e.doc.Resolver())
if err == nil {
img.Data, img.Filter = data, filter
} else {
img.Data = raw
}
}
e.images = append(e.images, img)
}
// unitSquareBox is the box the unit square covers once the transform has had
// its way with it.
func unitSquareBox(m matrix) (minX, minY, maxX, maxY float64) {
first := true
for _, c := range [][2]float64{{0, 0}, {1, 0}, {1, 1}, {0, 1}} {
x, y := m.apply(c[0], c[1])
if first {
minX, minY, maxX, maxY = x, y, x, y
first = false
continue
}
minX, maxX = smaller(minX, x), larger(maxX, x)
minY, maxY = smaller(minY, y), larger(maxY, y)
}
return minX, minY, maxX, maxY
}
func smaller(a, b float64) float64 {
if a < b {
return a
}
return b
}
func larger(a, b float64) float64 {
if a > b {
return a
}
return b
}
// intOf reads a whole number, or gives a default.
func intOf(o reader.Object, def int64) int64 {
if v, ok := reader.ToInt(o); ok {
return v
}
return def
}
// floatArray reads an array of numbers, or nothing.
func floatArray(d *reader.Document, o reader.Object) []float64 {
arr, ok := reader.ToArray(resolve(d, o))
if !ok {
return nil
}
out := make([]float64, 0, len(arr))
for _, e := range arr {
v, ok := reader.ToFloat(resolve(d, e))
if !ok {
return nil
}
out = append(out, v)
}
return out
}
vendor/github.com/go-pdfkit/extract/program.go
Generated Vendored
+175
View File
@@ -0,0 +1,175 @@
package extract
import (
"github.com/go-opentype/opentype"
"github.com/go-pdfkit/pdffont"
"github.com/go-pdfkit/reader"
)
// attachProgram gives a font a last resort for the codes the document says
// nothing about: what its own embedded program calls them.
//
// A font that says it is symbolic is addressed through its own character map,
// and an assumed encoding is a bad guess at it — a mathematical font puts a
// capital gamma where the standard encoding puts an inverted exclamation mark.
// The program knows, and this is how it is asked.
//
// There are two ways to ask, and a program answers at most one of them. A
// PostScript program names its glyphs and carries an encoding from code to
// name, so the name is the answer. A TrueType program usually does neither —
// a subset of one names nothing worth reading and has no code-to-name
// encoding at all — but it does carry character maps, and walking through one
// to the glyph and back out of another says which character the glyph is for.
func attachProgram(f *pdffont.Font) {
if f.Kind() == pdffont.Composite {
return
}
key, data, ok := f.Program()
if !ok {
return
}
program, err := readProgram(key, data)
if err != nil {
return
}
maps := chooseCharacterMaps(program)
// Only a simple font reaches here, and its codes are bytes.
f.SetFallback(func(code int) (string, bool) {
if r, ok := runeByName(program, code); ok {
return string(r), true
}
if r, ok := runeByCharacterMap(program, maps, code); ok {
return string(r), true
}
return "", false
})
}
// runeByName asks the program's own encoding what it calls a code, and reads
// the character out of that name. Only a PostScript program — a Type 1 one, or
// the CFF outlines of an OpenType font — carries such an encoding.
func runeByName(program *opentype.Font, code int) (rune, bool) {
gid, ok := program.GlyphIndexByCode(byte(code))
if !ok {
return 0, false
}
// A glyph the program does not name comes back as no name at all,
// which names no character either.
name, _ := program.GlyphName(gid)
return pdffont.RuneOfGlyphName(name)
}
// characterMaps says which of a program's cmap subtables are worth addressing,
// by index, or -1 for one the program does not carry.
type characterMaps struct {
// symbol is the Microsoft Symbol subtable, platform 3 encoding 0: a font's
// own codes, conventionally written at 0xF000 + code.
symbol int
// mac is the Macintosh Roman subtable, platform 1 encoding 0, indexed by
// single bytes.
mac int
// unicode is a Unicode subtable — Microsoft Unicode, platform 3 encoding 1
// or 10, or anything on platform 0 — indexed by codepoint. This is the one
// that is inverted, because it is the only one whose codes are characters.
unicode int
}
// chooseCharacterMaps picks out the subtables a font is addressed through and
// the one that says which character a glyph is for.
//
// A font can carry several subtables of a kind; the first of each is taken,
// which is what a font that repeats one means by repeating it.
func chooseCharacterMaps(program *opentype.Font) characterMaps {
m := characterMaps{symbol: -1, mac: -1, unicode: -1}
for i := range program.NumCharacterMaps() {
platform, encoding, _, _ := program.CharacterMap(i)
switch {
case platform == 3 && encoding == 0:
if m.symbol < 0 {
m.symbol = i
}
case platform == 1 && encoding == 0:
if m.mac < 0 {
m.mac = i
}
case platform == 0, platform == 3 && (encoding == 1 || encoding == 10):
if m.unicode < 0 {
m.unicode = i
}
}
}
return m
}
// runeByCharacterMap walks a code through the font's own character map to a
// glyph, and back out of its Unicode character map to the character that glyph
// stands for.
//
// The way in is the one poppler uses for a font the document gave no encoding:
// the Microsoft Symbol subtable if there is one, else the Macintosh Roman one,
// addressed by the raw code and then, failing that, by 0xF000 + code, which is
// where such subtables are conventionally written.
//
// The way out is the Unicode subtable, inverted. Without one there is no way
// out: a glyph on its own says nothing about which character it is, and a
// guess would be exactly the wrong letter the caller's guard exists to refuse.
func runeByCharacterMap(program *opentype.Font, m characterMaps, code int) (rune, bool) {
if m.unicode < 0 {
return 0, false
}
in := m.symbol
if in < 0 {
in = m.mac
}
if in < 0 {
return 0, false
}
gid, ok := program.GlyphIndexInMap(in, rune(code))
if !ok {
gid, ok = program.GlyphIndexInMap(in, rune(0xF000|code))
}
if !ok {
return 0, false
}
r, ok := program.RuneOfGlyphInMap(m.unicode, gid)
if !ok || !readableRune(r) {
return 0, false
}
return r, true
}
// readableRune reports whether a character recovered this way is worth
// reporting as text.
//
// A private-use codepoint is not: it means whatever the font decided it means
// and nothing outside the font can read it, so a page full of them searches no
// better than a page of nothing and looks, wrongly, like it was read. Control
// characters are refused for the same reason — a page does not say them.
func readableRune(r rune) bool {
switch {
case r < 0x20, r >= 0x7F && r <= 0x9F:
return false
case r >= 0xE000 && r <= 0xF8FF:
return false
case r >= 0xF0000:
return false
}
return true
}
// readProgram decodes an embedded font program. Which key it arrived under
// says what it is: FontFile2 is TrueType, FontFile a PostScript Type 1
// program, and FontFile3 either a bare CFF one or a whole OpenType font — the
// key alone does not settle that one, so both are tried.
func readProgram(key reader.Name, data []byte) (*opentype.Font, error) {
switch key {
case "FontFile":
return opentype.ParseType1(data)
case "FontFile3":
if f, err := opentype.Parse(data); err == nil {
return f, nil
}
return opentype.ParseCFF(data)
}
return opentype.Parse(data)
}
vendor/github.com/go-pdfkit/extract/renovate.json
Generated Vendored
+4
View File
@@ -0,0 +1,4 @@
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": ["github>go-pdfkit/.github"]
}
vendor/github.com/go-pdfkit/extract/run.go
Generated Vendored
+273
View File
@@ -0,0 +1,273 @@
package extract
import (
"strings"
"github.com/go-pdfkit/pdffont"
"github.com/go-pdfkit/reader"
)
// maxFormDepth is how deeply one form may draw another before a page is taken
// to be drawing itself.
const maxFormDepth = 12
// An extractor walks a content stream keeping just enough state to say where
// each piece of text was drawn.
type extractor struct {
doc *reader.Document
runs []Run
fonts map[int]*pdffont.Font
// tm is where the next glyph goes and tlm where the line began; both
// belong to the page rather than to the graphics state.
tm, tlm matrix
images []Image
// wantPictures says whether anybody is going to read Image.Data. When
// nobody is, the filters are left undone.
wantPictures bool
}
// run walks a content stream.
func (e *extractor) run(content []byte, resources reader.Dict, g state, depth int) {
if depth > maxFormDepth {
return
}
// The operations are taken one at a time rather than all at once. The
// walk only ever goes forwards, so nothing needs the list; keeping it
// costs a struct and a slice header for every operator on the page, and
// pages are not always small. One arXiv figure holds a single page of
// 84.8 MB of content stream and 4 579 973 operations: holding them cost
// 2 166 MB where reading them one by one costs 935 MB.
scan := reader.NewContentScanner(content)
var stack []state
for {
op, more := scan.Next()
if !more {
break
}
n := numbers(op.Operands)
switch op.Operator {
case "q":
stack = append(stack, g)
case "Q":
if len(stack) > 0 {
g = stack[len(stack)-1]
stack = stack[:len(stack)-1]
}
case "cm":
if len(n) >= 6 {
g.ctm = matrix{n[0], n[1], n[2], n[3], n[4], n[5]}.mul(g.ctm)
}
case "BT":
e.tm, e.tlm = identity, identity
case "ET":
e.tm, e.tlm = identity, identity
case "Tf":
e.setFont(&g, op.Operands, n, resources)
case "Tc":
if len(n) >= 1 {
g.text.charSpace = n[0]
}
case "Tw":
if len(n) >= 1 {
g.text.wordSpace = n[0]
}
case "Tz":
if len(n) >= 1 {
g.text.scale = n[0] / 100
}
case "TL":
if len(n) >= 1 {
g.text.leading = n[0]
}
case "Ts":
if len(n) >= 1 {
g.text.rise = n[0]
}
case "Tr":
if len(n) >= 1 {
g.text.mode = int(n[0])
}
case "Td":
if len(n) >= 2 {
e.tlm = matrix{1, 0, 0, 1, n[0], n[1]}.mul(e.tlm)
e.tm = e.tlm
}
case "TD":
if len(n) >= 2 {
g.text.leading = -n[1]
e.tlm = matrix{1, 0, 0, 1, n[0], n[1]}.mul(e.tlm)
e.tm = e.tlm
}
case "Tm":
if len(n) >= 6 {
e.tlm = matrix{n[0], n[1], n[2], n[3], n[4], n[5]}
e.tm = e.tlm
}
case "T*":
e.nextLine(&g)
case "Tj":
e.showOperands(&g, op.Operands)
case "'":
e.nextLine(&g)
e.showOperands(&g, op.Operands)
case "\"":
if len(n) >= 2 {
g.text.wordSpace, g.text.charSpace = n[0], n[1]
}
e.nextLine(&g)
e.showOperands(&g, op.Operands)
case "TJ":
e.showArray(&g, op.Operands)
case "Do":
e.doXObject(&g, op.Operands, resources, depth)
case "BI":
e.inlineImage(&g, op.Image)
}
}
}
// setFont reads the font a Tf operator names.
func (e *extractor) setFont(g *state, operands []reader.Object, n []float64, resources reader.Dict) {
if len(operands) < 2 {
return
}
name, ok := reader.ToName(operands[0])
if !ok {
return
}
if len(n) >= 1 {
g.text.size = n[len(n)-1]
}
g.text.fontName = string(name)
g.text.font = e.font(name, resources)
}
// font reads a font out of the page's resources, remembering the ones already
// read so that a page setting the same font a thousand times reads it once.
func (e *extractor) font(name reader.Name, resources reader.Dict) *pdffont.Font {
fonts, ok := e.doc.GetDict(resources, "Font")
if !ok {
return nil
}
if ref, ok := fonts.Get(name).(reader.Ref); ok {
if f, seen := e.fonts[ref.Num]; seen {
return f
}
dict, ok := e.doc.GetDict(fonts, name)
if !ok {
return nil
}
f := pdffont.Read(e.doc, dict)
attachProgram(f)
e.fonts[ref.Num] = f
return f
}
dict, ok := e.doc.GetDict(fonts, name)
if !ok {
return nil
}
f := pdffont.Read(e.doc, dict)
attachProgram(f)
return f
}
// nextLine moves down by the leading, which is what T* and the quote
// operators do.
func (e *extractor) nextLine(g *state) {
e.tlm = matrix{1, 0, 0, 1, 0, -g.text.leading}.mul(e.tlm)
e.tm = e.tlm
}
// showOperands shows the string a Tj-like operator carries.
func (e *extractor) showOperands(g *state, operands []reader.Object) {
for _, o := range operands {
if s, ok := reader.ToString(o); ok {
e.show(g, s)
return
}
}
}
// showArray shows the pieces a TJ operator carries, moving the pen by the
// numbers between them.
func (e *extractor) showArray(g *state, operands []reader.Object) {
for _, o := range operands {
arr, ok := reader.ToArray(o)
if !ok {
continue
}
for _, item := range arr {
if s, ok := reader.ToString(item); ok {
e.show(g, s)
continue
}
if v, ok := reader.ToFloat(item); ok {
e.advance(g, -v/1000*g.text.size*g.text.scale)
}
}
}
}
// show reads one string as text and records where it was drawn.
func (e *extractor) show(g *state, s []byte) {
f := g.text.font
if f == nil || len(s) == 0 {
return
}
start := e.tm.mul(g.ctm)
x, y := start.apply(0, g.text.rise)
var text strings.Builder
unreadable := false
width := 0.0
for _, code := range f.Codes(s) {
if t, ok := f.Text(code); ok {
text.WriteString(pdffont.TrimText(t))
} else {
unreadable = true
}
adv := (f.Width(code)*g.text.size + g.text.charSpace) * g.text.scale
if f.Kind() != pdffont.Composite && code == ' ' {
adv += g.text.wordSpace * g.text.scale
}
width += adv
}
e.advance(g, width)
if text.Len() == 0 && !unreadable {
return
}
// How far a length in text space reaches on the page: the text matrix and
// the page's transform together, which is the same matrix the run's own
// position came through.
//
// Only the page's transform used to be counted here, and the text matrix
// was left out. A document is free to put the whole of its scale there —
// "/F1 1 Tf" and then a text matrix of ten is how TeX writes every PDF it
// has ever produced — and such a document reported its text as one point
// tall with letters half a point wide, while saying quite correctly where
// on the page each run began.
//
// Those are the two numbers a word break is decided by. A gap measured in
// points was being weighed against a space width ten times too small, so
// every ordinary kern between two letters looked like a space: "Original
// Domain" came back as "Or ig inal D omain". Across four thousand arXiv
// figures, 57% of the words a reader got back were one or two letters
// long. Nothing failed, and nothing was slow; the text was simply no
// longer text.
scale := start.scale()
e.runs = append(e.runs, Run{
Text: text.String(),
X: x,
Y: y,
Width: width * scale,
Size: g.text.size * scale,
Font: g.text.fontName,
Space: f.Width(' ') * g.text.size * g.text.scale * scale,
Invisible: g.text.mode == 3,
Unreadable: unreadable,
})
}
// advance moves the pen along the line by a distance in text space.
func (e *extractor) advance(g *state, by float64) {
e.tm = matrix{1, 0, 0, 1, by, 0}.mul(e.tm)
}
vendor/github.com/go-pdfkit/extract/state.go
Generated Vendored
+134
View File
@@ -0,0 +1,134 @@
package extract
import (
"github.com/go-pdfkit/pdffont"
"github.com/go-pdfkit/reader"
)
// A matrix is the six numbers a PDF transform is written as. Extraction needs
// only to multiply them and to move a point, so it carries its own rather than
// depending on a drawing library it has no other use for.
type matrix struct{ a, b, c, d, e, f float64 }
// identity is the transform that changes nothing.
var identity = matrix{1, 0, 0, 1, 0, 0}
// mul is the transform that applies m first and then n, which is the order a
// PDF composes them in.
func (m matrix) mul(n matrix) matrix {
return matrix{
a: m.a*n.a + m.b*n.c,
b: m.a*n.b + m.b*n.d,
c: m.c*n.a + m.d*n.c,
d: m.c*n.b + m.d*n.d,
e: m.e*n.a + m.f*n.c + n.e,
f: m.e*n.b + m.f*n.d + n.f,
}
}
// apply moves a point.
func (m matrix) apply(x, y float64) (float64, float64) {
return m.a*x + m.c*y + m.e, m.b*x + m.d*y + m.f
}
// scale is how much the transform stretches a length, taken as the geometric
// mean of its two axes so that a rotation counts as no stretch at all.
func (m matrix) scale() float64 {
x := m.a*m.a + m.b*m.b
y := m.c*m.c + m.d*m.d
return sqrt(sqrt(x * y))
}
// sqrt without pulling in the whole of math for one use.
func sqrt(v float64) float64 {
if v <= 0 {
return 0
}
x := v
for i := 0; i < 24; i++ {
x = (x + v/x) / 2
}
return x
}
// A textState is everything the showing operators read besides the string.
type textState struct {
font *pdffont.Font
fontName string
size float64
charSpace float64
wordSpace float64
scale float64 // horizontal scaling, as a fraction
leading float64
rise float64
mode int
}
// A state is the graphics state, of which extraction needs only the transform
// and the text parameters.
type state struct {
ctm matrix
text textState
}
// initialState is the transform a page starts in, which for extraction is the
// one that undoes the page's own box: text comes back in the page's
// coordinates, counting up from the bottom left of what is visible.
func initialState(d *reader.Document, page reader.Dict) state {
box := pageBox(d, page)
return state{
ctm: matrix{1, 0, 0, 1, -box[0], -box[1]},
text: textState{scale: 1},
}
}
// pageBox is the area of the page that is visible.
func pageBox(d *reader.Document, page reader.Dict) [4]float64 {
for _, key := range []reader.Name{"CropBox", "MediaBox"} {
if b, ok := rectangle(d, page.Get(key)); ok {
return b
}
}
return [4]float64{0, 0, 612, 792}
}
// rectangle reads a PDF rectangle, put the right way round.
func rectangle(d *reader.Document, o reader.Object) ([4]float64, bool) {
var out [4]float64
arr, ok := reader.ToArray(resolve(d, o))
if !ok || len(arr) < 4 {
return out, false
}
for i := 0; i < 4; i++ {
v, ok := reader.ToFloat(resolve(d, arr[i]))
if !ok {
return out, false
}
out[i] = v
}
if out[0] > out[2] {
out[0], out[2] = out[2], out[0]
}
if out[1] > out[3] {
out[1], out[3] = out[3], out[1]
}
return out, true
}
// resolve follows an indirect reference. A document that opened cannot fail to
// resolve one.
func resolve(d *reader.Document, o reader.Object) reader.Object {
out, _ := d.Resolve(o)
return out
}
// numbers reads the operands that are numbers.
func numbers(operands []reader.Object) []float64 {
out := make([]float64, 0, len(operands))
for _, o := range operands {
if v, ok := reader.ToFloat(o); ok {
out = append(out, v)
}
}
return out
}
vendor/github.com/go-pdfkit/pdffont/.gitignore
Generated
+1
View File
@@ -0,0 +1 @@
coverage.out
vendor/github.com/go-pdfkit/pdffont/LICENSE
Generated Vendored
+29
View File
@@ -0,0 +1,29 @@
BSD 3-Clause License
Copyright (c) 2026 the go-pdfkit/render authors
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
vendor/github.com/go-pdfkit/pdffont/README.md
Generated Vendored
+75
View File
@@ -0,0 +1,75 @@
# pdffont
[![CI](https://github.com/go-pdfkit/pdffont/actions/workflows/ci.yml/badge.svg)](https://github.com/go-pdfkit/pdffont/actions/workflows/ci.yml)
[![Go Reference](https://pkg.go.dev/badge/github.com/go-pdfkit/pdffont.svg)](https://pkg.go.dev/github.com/go-pdfkit/pdffont)
[![License: BSD-3-Clause](https://img.shields.io/badge/License-BSD--3--Clause-blue.svg)](LICENSE)
[![Coverage](https://img.shields.io/badge/coverage-100%25-brightgreen.svg)](#how-it-is-checked)
The PDF side of a font: not the outlines, which are the font program's
business, but everything the document itself says about how a string of
bytes is to be read.
Which codes a string holds, how wide each one is, what each is called, and
what text each stands for. It is what a renderer needs before it can ask a
font program for a glyph, and all an extractor needs to read a page back as
words — so neither has to say it twice.
Pure Go, no C, nothing outside [`reader`](https://github.com/go-pdfkit/reader)
and the standard library.
## What it reads
Simple fonts, addressed one byte at a time through a base encoding and
whatever differences the document names on top of it. Composite fonts,
addressed by character identifier, with their widths and their map from
identifier to glyph. Type 3 fonts, whose glyphs are little drawings and
whose widths are in a space of their own.
And `/ToUnicode`, which is the one thing in a PDF that says what its text
actually **says**. Without it a page can be drawn but not read.
## What it refuses to guess
A font that says it is **symbolic** is addressed through its own character
map, and a base encoding is a guess at it — a bad one. A mathematical font
puts a capital gamma where the standard encoding puts an inverted
exclamation mark, so reading the name gives the wrong letter with nothing
to say so.
Such a code is left unread unless the document named it itself, or unless a
caller installs [`Font.SetFallback`](https://pkg.go.dev/github.com/go-pdfkit/pdffont#Font.SetFallback)
to say what the embedded program calls it. Reading the program is another
package's work; `extract` does it with
[`go-opentype`](https://github.com/go-opentype/opentype).
## How it is checked
100% of statements, including every branch that handles a dictionary saying
something it should not.
And the glyph-name table — which is the one piece of data here, and where a
single wrong entry would silently give the wrong letter — is held up
against the documents themselves. A font that both names a code and carries
a `/ToUnicode` entry for it states the same fact twice, so the two can be
compared. Over 12 000 corpus files: **63 515 names checked, 63 074 agreed —
99.31%**. The disagreements are producers whose own maps are wrong
(`alpha` → `↵`, `ff` → `!`), plus `Delta` and `Omega`, which Adobe's list
gives as the increment and ohm signs; the Greek letters written here are
the same characters by Unicode's own equivalence, and the ones a person
searching would type.
## The rest of the family
[**reader**](https://github.com/go-pdfkit/reader) reads and writes the format ·
[**ops**](https://github.com/go-pdfkit/ops) is the verbs and the `pdfops` command ·
[**render**](https://github.com/go-pdfkit/render) turns a page into pixels ·
[**pdffont**](https://github.com/go-pdfkit/pdffont) is what a document says about a font ·
[**extract**](https://github.com/go-pdfkit/extract) reads a page back ·
[**coedit**](https://github.com/go-pdfkit/coedit) is a PDF several people edit at once ·
[**app**](https://github.com/go-pdfkit/app) is the browser workbench.
Documentation for all of it: <https://go-pdfkit.github.io/docs/>
## Licence
BSD-3-Clause.
vendor/github.com/go-pdfkit/pdffont/encoding.go
Generated Vendored
+302
View File
@@ -0,0 +1,302 @@
package pdffont
import "strings"
// The two encodings a simple font is addressed through when it does not carry
// one of its own, as a name per code. Only the codes that name a glyph appear;
// the rest are blank.
//
// These are the tables the specification prints. They matter because a font
// program is addressed by glyph name or by character, and a content stream is
// written in single bytes: without them, every byte would have to be guessed
// at.
// StandardEncoding is Adobe's own, which most Type 1 fonts assume, and what a
// simple font is read with when it says nothing else.
var StandardEncoding = [256]string{
32: "space", 33: "exclam", 34: "quotedbl", 35: "numbersign", 36: "dollar",
37: "percent", 38: "ampersand", 39: "quoteright", 40: "parenleft",
41: "parenright", 42: "asterisk", 43: "plus", 44: "comma", 45: "hyphen",
46: "period", 47: "slash", 48: "zero", 49: "one", 50: "two", 51: "three",
52: "four", 53: "five", 54: "six", 55: "seven", 56: "eight", 57: "nine",
58: "colon", 59: "semicolon", 60: "less", 61: "equal", 62: "greater",
63: "question", 64: "at", 65: "A", 66: "B", 67: "C", 68: "D", 69: "E",
70: "F", 71: "G", 72: "H", 73: "I", 74: "J", 75: "K", 76: "L", 77: "M",
78: "N", 79: "O", 80: "P", 81: "Q", 82: "R", 83: "S", 84: "T", 85: "U",
86: "V", 87: "W", 88: "X", 89: "Y", 90: "Z", 91: "bracketleft",
92: "backslash", 93: "bracketright", 94: "asciicircum", 95: "underscore",
96: "quoteleft", 97: "a", 98: "b", 99: "c", 100: "d", 101: "e", 102: "f",
103: "g", 104: "h", 105: "i", 106: "j", 107: "k", 108: "l", 109: "m",
110: "n", 111: "o", 112: "p", 113: "q", 114: "r", 115: "s", 116: "t",
117: "u", 118: "v", 119: "w", 120: "x", 121: "y", 122: "z",
123: "braceleft", 124: "bar", 125: "braceright", 126: "asciitilde",
161: "exclamdown", 162: "cent", 163: "sterling", 164: "fraction",
165: "yen", 166: "florin", 167: "section", 168: "currency",
169: "quotesingle", 170: "quotedblleft", 171: "guillemotleft",
172: "guilsinglleft", 173: "guilsinglright", 174: "fi", 175: "fl",
177: "endash", 178: "dagger", 179: "daggerdbl", 180: "periodcentered",
182: "paragraph", 183: "bullet", 184: "quotesinglbase",
185: "quotedblbase", 186: "quotedblright", 187: "guillemotright",
188: "ellipsis", 189: "perthousand", 191: "questiondown", 193: "grave",
194: "acute", 195: "circumflex", 196: "tilde", 197: "macron", 198: "breve",
199: "dotaccent", 200: "dieresis", 202: "ring", 203: "cedilla",
205: "hungarumlaut", 206: "ogonek", 207: "caron", 208: "emdash",
225: "AE", 227: "ordfeminine", 232: "Lslash", 233: "Oslash", 234: "OE",
235: "ordmasculine", 241: "ae", 245: "dotlessi", 248: "lslash",
249: "oslash", 250: "oe", 251: "germandbls",
}
// WinAnsiEncoding is the one a file written on Windows assumes, and the one
// most producers name.
var WinAnsiEncoding = [256]string{
32: "space", 33: "exclam", 34: "quotedbl", 35: "numbersign", 36: "dollar",
37: "percent", 38: "ampersand", 39: "quotesingle", 40: "parenleft",
41: "parenright", 42: "asterisk", 43: "plus", 44: "comma", 45: "hyphen",
46: "period", 47: "slash", 48: "zero", 49: "one", 50: "two", 51: "three",
52: "four", 53: "five", 54: "six", 55: "seven", 56: "eight", 57: "nine",
58: "colon", 59: "semicolon", 60: "less", 61: "equal", 62: "greater",
63: "question", 64: "at", 65: "A", 66: "B", 67: "C", 68: "D", 69: "E",
70: "F", 71: "G", 72: "H", 73: "I", 74: "J", 75: "K", 76: "L", 77: "M",
78: "N", 79: "O", 80: "P", 81: "Q", 82: "R", 83: "S", 84: "T", 85: "U",
86: "V", 87: "W", 88: "X", 89: "Y", 90: "Z", 91: "bracketleft",
92: "backslash", 93: "bracketright", 94: "asciicircum", 95: "underscore",
96: "grave", 97: "a", 98: "b", 99: "c", 100: "d", 101: "e", 102: "f",
103: "g", 104: "h", 105: "i", 106: "j", 107: "k", 108: "l", 109: "m",
110: "n", 111: "o", 112: "p", 113: "q", 114: "r", 115: "s", 116: "t",
117: "u", 118: "v", 119: "w", 120: "x", 121: "y", 122: "z",
123: "braceleft", 124: "bar", 125: "braceright", 126: "asciitilde",
128: "Euro", 130: "quotesinglbase", 131: "florin", 132: "quotedblbase",
133: "ellipsis", 134: "dagger", 135: "daggerdbl", 136: "circumflex",
137: "perthousand", 138: "Scaron", 139: "guilsinglleft", 140: "OE",
142: "Zcaron", 145: "quoteleft", 146: "quoteright", 147: "quotedblleft",
148: "quotedblright", 149: "bullet", 150: "endash", 151: "emdash",
152: "tilde", 153: "trademark", 154: "scaron", 155: "guilsinglright",
156: "oe", 158: "zcaron", 159: "Ydieresis", 160: "space",
161: "exclamdown", 162: "cent", 163: "sterling", 164: "currency",
165: "yen", 166: "brokenbar", 167: "section", 168: "dieresis",
169: "copyright", 170: "ordfeminine", 171: "guillemotleft",
172: "logicalnot", 173: "hyphen", 174: "registered", 175: "macron",
176: "degree", 177: "plusminus", 178: "twosuperior",
179: "threesuperior", 180: "acute", 181: "mu", 182: "paragraph",
183: "periodcentered", 184: "cedilla", 185: "onesuperior",
186: "ordmasculine", 187: "guillemotright", 188: "onequarter",
189: "onehalf", 190: "threequarters", 191: "questiondown", 192: "Agrave",
193: "Aacute", 194: "Acircumflex", 195: "Atilde", 196: "Adieresis",
197: "Aring", 198: "AE", 199: "Ccedilla", 200: "Egrave", 201: "Eacute",
202: "Ecircumflex", 203: "Edieresis", 204: "Igrave", 205: "Iacute",
206: "Icircumflex", 207: "Idieresis", 208: "Eth", 209: "Ntilde",
210: "Ograve", 211: "Oacute", 212: "Ocircumflex", 213: "Otilde",
214: "Odieresis", 215: "multiply", 216: "Oslash", 217: "Ugrave",
218: "Uacute", 219: "Ucircumflex", 220: "Udieresis", 221: "Yacute",
222: "Thorn", 223: "germandbls", 224: "agrave", 225: "aacute",
226: "acircumflex", 227: "atilde", 228: "adieresis", 229: "aring",
230: "ae", 231: "ccedilla", 232: "egrave", 233: "eacute",
234: "ecircumflex", 235: "edieresis", 236: "igrave", 237: "iacute",
238: "icircumflex", 239: "idieresis", 240: "eth", 241: "ntilde",
242: "ograve", 243: "oacute", 244: "ocircumflex", 245: "otilde",
246: "odieresis", 247: "divide", 248: "oslash", 249: "ugrave",
250: "uacute", 251: "ucircumflex", 252: "udieresis", 253: "yacute",
254: "thorn", 255: "ydieresis",
}
// glyphRunes is what the names above stand for. A font program is looked up by
// character, so a name has to become one; anything not here, and not written
// in one of the two conventional forms, cannot be looked up that way.
var glyphRunes = map[string]rune{
"space": ' ', "exclam": '!', "quotedbl": '"', "numbersign": '#',
"dollar": '$', "percent": '%', "ampersand": '&', "quotesingle": '\'',
"quoteright": '’', "quoteleft": '‘', "parenleft": '(',
"parenright": ')', "asterisk": '*', "plus": '+', "comma": ',',
"hyphen": '-', "period": '.', "slash": '/', "zero": '0', "one": '1',
"two": '2', "three": '3', "four": '4', "five": '5', "six": '6',
"seven": '7', "eight": '8', "nine": '9', "colon": ':', "semicolon": ';',
"less": '<', "equal": '=', "greater": '>', "question": '?', "at": '@',
"bracketleft": '[', "backslash": '\\', "bracketright": ']',
"asciicircum": '^', "underscore": '_', "braceleft": '{', "bar": '|',
"braceright": '}', "asciitilde": '~', "exclamdown": '¡', "cent": '¢',
"sterling": '£', "fraction": '⁄', "yen": '¥', "florin": 'ƒ',
"section": '§', "currency": '¤', "quotedblleft": '“',
"quotedblright": '”', "guillemotleft": '«', "guillemotright": '»',
"guilsinglleft": '‹', "guilsinglright": '›', "fi": 'fi', "fl": 'fl',
"endash": '–', "emdash": '—', "dagger": '†', "daggerdbl": '‡',
"periodcentered": '·', "paragraph": '¶', "bullet": '•',
"quotesinglbase": '‚', "quotedblbase": '„', "ellipsis": '…',
"perthousand": '‰', "questiondown": '¿', "grave": '`', "acute": '´',
"circumflex": 'ˆ', "tilde": '˜', "macron": '¯', "breve": '˘',
"dotaccent": '˙', "dieresis": '¨', "ring": '˚', "cedilla": '¸',
"hungarumlaut": '˝', "ogonek": '˛', "caron": 'ˇ', "AE": 'Æ', "ae": 'æ',
"ordfeminine": 'ª', "ordmasculine": 'º', "Lslash": 'Ł', "lslash": 'ł',
"Oslash": 'Ø', "oslash": 'ø', "OE": 'Œ', "oe": 'œ', "dotlessi": 'ı',
"germandbls": 'ß', "Euro": '€', "Scaron": 'Š', "scaron": 'š',
"Zcaron": 'Ž', "zcaron": 'ž', "Ydieresis": 'Ÿ', "trademark": '™',
"brokenbar": '¦', "copyright": '©', "logicalnot": '¬', "registered": '®',
"degree": '°', "plusminus": '±', "twosuperior": '²',
"threesuperior": '³', "mu": 'µ', "onesuperior": '¹', "onequarter": '¼',
"onehalf": '½', "threequarters": '¾', "multiply": '×', "divide": '÷',
"Agrave": 'À', "Aacute": 'Á', "Acircumflex": 'Â', "Atilde": 'Ã',
"Adieresis": 'Ä', "Aring": 'Å', "Ccedilla": 'Ç', "Egrave": 'È',
"Eacute": 'É', "Ecircumflex": 'Ê', "Edieresis": 'Ë', "Igrave": 'Ì',
"Iacute": 'Í', "Icircumflex": 'Î', "Idieresis": 'Ï', "Eth": 'Ð',
"Ntilde": 'Ñ', "Ograve": 'Ò', "Oacute": 'Ó', "Ocircumflex": 'Ô',
"Otilde": 'Õ', "Odieresis": 'Ö', "Ugrave": 'Ù', "Uacute": 'Ú',
"Ucircumflex": 'Û', "Udieresis": 'Ü', "Yacute": 'Ý', "Thorn": 'Þ',
"agrave": 'à', "aacute": 'á', "acircumflex": 'â', "atilde": 'ã',
"adieresis": 'ä', "aring": 'å', "ccedilla": 'ç', "egrave": 'è',
"eacute": 'é', "ecircumflex": 'ê', "edieresis": 'ë', "igrave": 'ì',
"iacute": 'í', "icircumflex": 'î', "idieresis": 'ï', "eth": 'ð',
"ntilde": 'ñ', "ograve": 'ò', "oacute": 'ó', "ocircumflex": 'ô',
"otilde": 'õ', "odieresis": 'ö', "ugrave": 'ù', "uacute": 'ú',
"ucircumflex": 'û', "udieresis": 'ü', "yacute": 'ý', "thorn": 'þ',
"ydieresis": 'ÿ',
}
// RuneOfGlyphName turns a glyph name into the character it stands for, through
// the table above or through the two conventions the specification defines for
// naming a character directly. ok is false for a name that says nothing about
// which character it is, which a subsetted font's own names often do not.
func RuneOfGlyphName(name string) (rune, bool) {
s, ok := TextOfGlyphName(name)
if !ok {
return 0, false
}
r := []rune(s)
if len(r) != 1 {
return 0, false
}
return r[0], true
}
// TextOfGlyphName turns a glyph name into the text it stands for, which is not
// always one character: a name may say it is a ligature of several.
//
// Three of the rules here are the ones the Adobe Glyph List lays down for
// reading a name nobody has listed, and each was costing real text. A name may
// carry a variant after a full stop — "a.sc" is a small-capital A and is still
// an A, 7 272 of them in the corpus. A name may be several component names
// with underscores between them — "f_i" is the fi ligature, and without this
// "Definition" comes back "Denition", which is not a missing character so much
// as a misspelt word. And a name may simply be one nobody thought to list.
func TextOfGlyphName(name string) (string, bool) {
if name == "" {
return "", false
}
if r, ok := oneGlyphRune(name); ok {
return string(r), true
}
if seq, ok := unicodeSequence(name); ok {
return seq, true
}
// A variant of a character is still that character: the part before the
// first full stop is the name, and what follows says which cut of it.
if base, _, found := strings.Cut(name, "."); found {
if base == "" {
return "", false
}
return TextOfGlyphName(base)
}
// A name made of parts with underscores between them is the parts, in
// order — which is how a ligature is named when it has no name of its own.
if strings.Contains(name, "_") {
// Every part that can be read contributes at least one character, and
// a part that cannot gives up on the whole name, so what comes out of
// this is never empty.
var out strings.Builder
for _, part := range strings.Split(name, "_") {
piece, ok := TextOfGlyphName(part)
if !ok {
return "", false
}
out.WriteString(piece)
}
return out.String(), true
}
return "", false
}
// oneGlyphRune is a name that stands for exactly one character, by any of the
// ways a name can.
func oneGlyphRune(name string) (rune, bool) {
if r, ok := glyphRunes[name]; ok {
return r, true
}
if r, ok := latinExtendedRunes[name]; ok {
return r, true
}
if r, ok := greekAndMathRunes[name]; ok {
return r, true
}
if len(name) == 1 {
return rune(name[0]), true
}
if r, ok := parseHexName(name, "uni", 4); ok {
return r, true
}
if r, ok := parseHexName(name, "u", 4); ok {
return r, true
}
return 0, false
}
// unicodeSequence reads the uniXXXXYYYY form, which names several characters
// at once: a letter and the accent that goes over it, or the three pieces a
// Hebrew cluster is written in.
//
// It has to be told apart from something that looks exactly like it. Producers
// write a glyph's own number in the same shape — uni00000048 is glyph 72, not
// U+0000 followed by U+0048 — and 49 740 names in the corpus are that. The two
// are separable by one observation: a real sequence never begins with U+0000,
// because nothing is written after a character that does not exist. Of the
// 3 875 genuine sequences found across 14 823 embedded fonts, not one starts
// with a zero group; of the disguised glyph numbers, all of them do.
func unicodeSequence(name string) (string, bool) {
digits, ok := strings.CutPrefix(name, "uni")
if !ok || len(digits) < 8 || len(digits)%4 != 0 {
return "", false
}
var out strings.Builder
for i := 0; i < len(digits); i += 4 {
r, ok := parseHexName("uni"+digits[i:i+4], "uni", 4)
if !ok {
return "", false
}
if r == 0 {
// A glyph number wearing a character's clothes.
return "", false
}
out.WriteRune(r)
}
return out.String(), true
}
// parseHexName reads the uniXXXX and uXXXX conventions.
func parseHexName(name, prefix string, minDigits int) (rune, bool) {
if len(name) <= len(prefix) || name[:len(prefix)] != prefix {
return 0, false
}
digits := name[len(prefix):]
if len(digits) < minDigits || len(digits) > 6 {
return 0, false
}
var v rune
for i := 0; i < len(digits); i++ {
d := hexDigit(digits[i])
if d < 0 {
return 0, false
}
v = v<<4 | rune(d)
}
return v, true
}
// hexDigit reads one hexadecimal digit, or reports -1.
func hexDigit(c byte) int {
switch {
case c >= '0' && c <= '9':
return int(c - '0')
case c >= 'A' && c <= 'F':
return int(c-'A') + 10
case c >= 'a' && c <= 'f':
return int(c-'a') + 10
}
return -1
}
vendor/github.com/go-pdfkit/pdffont/extended.go
Generated Vendored
+71
View File
@@ -0,0 +1,71 @@
package pdffont
// The glyph names above cover the Latin every English document is set in and
// stop there. A document in Czech, Polish, Slovak, Hungarian, Turkish,
// Romanian, Latvian, Lithuanian or Maltese is set in the block that follows,
// and every one of its accented letters was being read as nothing at all —
// which is worse than it sounds, because a dropped character leaves no mark:
// "Příliš" comes back "Píliš" and looks like a word.
//
// Counted across the 118 833-file corpus: of 52.4 million glyph names, 483 382
// could not be turned into text, and the largest nameable share of those was
// this block — rcaron 650, zdotaccent 617, Ccaron 611, nacute 602, aogonek
// 602, scedilla 597, gbreve 597, and so on for a hundred more.
//
// These are the names the Adobe Glyph List gives, and the characters are not
// in doubt: unlike the subsetted fonts that call their glyphs g0 or dress a
// glyph number up as uni00000048, a name like "rcaron" says exactly one thing.
var latinExtendedRunes = map[string]rune{
"Amacron": 'Ā', "amacron": 'ā', "Abreve": 'Ă', "abreve": 'ă',
"Aogonek": 'Ą', "aogonek": 'ą', "Cacute": 'Ć', "cacute": 'ć',
"Ccircumflex": 'Ĉ', "ccircumflex": 'ĉ', "Cdotaccent": 'Ċ', "cdotaccent": 'ċ',
"Ccaron": 'Č', "ccaron": 'č', "Dcaron": 'Ď', "dcaron": 'ď',
"Dcroat": 'Đ', "dcroat": 'đ', "Dslash": 'Đ', "dmacron": 'đ',
"Emacron": 'Ē', "emacron": 'ē', "Ebreve": 'Ĕ', "ebreve": 'ĕ',
"Edotaccent": 'Ė', "edotaccent": 'ė', "Eogonek": 'Ę', "eogonek": 'ę',
"Ecaron": 'Ě', "ecaron": 'ě', "Gcircumflex": 'Ĝ', "gcircumflex": 'ĝ',
"Gbreve": 'Ğ', "gbreve": 'ğ', "Gdotaccent": 'Ġ', "gdotaccent": 'ġ',
"Gcommaaccent": 'Ģ', "gcommaaccent": 'ģ',
"Hcircumflex": 'Ĥ', "hcircumflex": 'ĥ', "Hbar": 'Ħ', "hbar": 'ħ',
"Itilde": 'Ĩ', "itilde": 'ĩ', "Imacron": 'Ī', "imacron": 'ī',
"Ibreve": 'Ĭ', "ibreve": 'ĭ', "Iogonek": 'Į', "iogonek": 'į',
"Idotaccent": 'İ', "dotlessi": 'ı', "IJ": 'IJ', "ij": 'ij',
"Jcircumflex": 'Ĵ', "jcircumflex": 'ĵ',
"Kcommaaccent": 'Ķ', "kcommaaccent": 'ķ', "kgreenlandic": 'ĸ',
"Lacute": 'Ĺ', "lacute": 'ĺ', "Lcommaaccent": 'Ļ', "lcommaaccent": 'ļ',
"Lcaron": 'Ľ', "lcaron": 'ľ', "Ldot": 'Ŀ', "ldot": 'ŀ',
"Nacute": 'Ń', "nacute": 'ń', "Ncommaaccent": 'Ņ', "ncommaaccent": 'ņ',
"Ncaron": 'Ň', "ncaron": 'ň', "napostrophe": 'ʼn', "Eng": 'Ŋ', "eng": 'ŋ',
"Omacron": 'Ō', "omacron": 'ō', "Obreve": 'Ŏ', "obreve": 'ŏ',
"Ohungarumlaut": 'Ő', "ohungarumlaut": 'ő',
"Racute": 'Ŕ', "racute": 'ŕ', "Rcommaaccent": 'Ŗ', "rcommaaccent": 'ŗ',
"Rcaron": 'Ř', "rcaron": 'ř', "Sacute": 'Ś', "sacute": 'ś',
"Scircumflex": 'Ŝ', "scircumflex": 'ŝ', "Scedilla": 'Ş', "scedilla": 'ş',
"Tcommaaccent": 'Ţ', "tcommaaccent": 'ţ', "Tcaron": 'Ť', "tcaron": 'ť',
"Tbar": 'Ŧ', "tbar": 'ŧ', "Utilde": 'Ũ', "utilde": 'ũ',
"Umacron": 'Ū', "umacron": 'ū', "Ubreve": 'Ŭ', "ubreve": 'ŭ',
"Uring": 'Ů', "uring": 'ů', "Uhungarumlaut": 'Ű', "uhungarumlaut": 'ű',
"Uogonek": 'Ų', "uogonek": 'ų', "Wcircumflex": 'Ŵ', "wcircumflex": 'ŵ',
"Ycircumflex": 'Ŷ', "ycircumflex": 'ŷ',
"Zacute": 'Ź', "zacute": 'ź', "Zdotaccent": 'Ż', "zdotaccent": 'ż',
"longs": 'ſ',
// A handful outside that block that the same documents use: the Romanian
// letters written with a comma below rather than a cedilla, the Turkish
// and Azeri schwa, and the two the Adobe list names for Welsh and Maltese.
"Scommaaccent": 'Ș', "scommaaccent": 'ș',
"Afii10017": 'А', "afii10017": 'А',
"Wgrave": 'Ẁ', "wgrave": 'ẁ', "Wacute": 'Ẃ', "wacute": 'ẃ',
"Wdieresis": 'Ẅ', "wdieresis": 'ẅ',
"Ygrave": 'Ỳ', "ygrave": 'ỳ',
"Hcaron": 'Ȟ', "hcaron": 'ȟ',
// The rest of what the corpus asks for and nothing could answer, each of
// them a name the Adobe list gives and none of them in doubt: the cedilla
// spellings of the Romanian letters, the capital sharp s, the script ell
// mathematicians write lengths with, and the angle brackets.
"Tcedilla": 'Ţ', "tcedilla": 'ţ', "Germandbls": 'ẞ',
"lscript": 'ℓ', "openbullet": '◦',
"angbracketleft": '〈', "angbracketright": '〉',
"Scommabelow": 'Ș', "scommabelow": 'ș',
"Tcommabelow": 'Ț', "tcommabelow": 'ț',
}
vendor/github.com/go-pdfkit/pdffont/font.go
Generated Vendored
+245
View File
@@ -0,0 +1,245 @@
// Package pdffont is the PDF side of a font: not the outlines, which are the
// font program's business, but everything the document itself says about how a
// string of bytes is to be read. Which codes a string holds, how wide each one
// is, what each is called, and what text each stands for.
//
// It is what a renderer needs before it can ask a font program for a glyph,
// and all an extractor needs to read a page back as words. Both would
// otherwise have to say the same things twice.
package pdffont
import (
"github.com/go-pdfkit/reader"
)
// A Kind says how a font's codes are read.
type Kind int
const (
// Simple fonts take one byte a code and are addressed by glyph name.
Simple Kind = iota
// Composite fonts take a character identifier, usually two bytes wide.
Composite
// Type3 fonts draw each glyph by running a little content stream.
Type3
)
// A Font is a font as the document describes it.
type Font struct {
kind Kind
dict reader.Dict
descriptor reader.Dict
doc *reader.Document
names map[int]string // what each code is called, for a simple font
// chosen says which of those names the document chose itself, in a
// Differences array, rather than inheriting from a base encoding.
chosen map[int]bool
widths map[int]float64 // in text space: a thousandth of the point size
defaultW float64
toUni map[int]string
symbolic bool
// fallback is a last resort for codes the document itself says nothing
// about: what the embedded font program calls them. A caller that can
// read the program installs one.
fallback func(code int) (string, bool)
// cidToGID maps a character identifier to a glyph number, for a composite
// font whose descendant carries a map rather than using them as they are.
cidToGID []byte
// The pieces only a Type 3 font has.
fontMatrix [6]float64
charProcs reader.Dict
t3Res reader.Dict
}
// Read takes what a font dictionary says. It never fails: a dictionary that
// says nothing useful gives a font that reads its codes one byte at a time and
// advances by half an em, which is what a reader has to do with one anyway.
func Read(d *reader.Document, dict reader.Dict) *Font {
f := &Font{
doc: d, dict: dict, widths: map[int]float64{}, defaultW: 0.5,
fontMatrix: [6]float64{0.001, 0, 0, 0.001, 0, 0},
}
switch sub, _ := reader.ToName(resolve(d, dict.Get("Subtype"))); sub {
case "Type0":
f.kind = Composite
f.readComposite()
case "Type3":
f.kind = Type3
f.readType3()
default:
f.kind = Simple
f.readSimple()
}
f.toUni = f.readToUnicode()
return f
}
// Kind says how this font's codes are read.
func (f *Font) Kind() Kind { return f.kind }
// Dict is the font dictionary itself, for whatever a caller needs that this
// does not say.
func (f *Font) Dict() reader.Dict { return f.dict }
// Descriptor is the font descriptor, which is where an embedded program lives.
func (f *Font) Descriptor() reader.Dict { return f.descriptor }
// Symbolic reports a font that says it is addressed through its own character
// map rather than through an encoding of names.
func (f *Font) Symbolic() bool { return f.symbolic }
// FontMatrix is how a Type 3 font's glyph space relates to text space. For
// every other kind it is a thousandth, which is what the format assumes.
func (f *Font) FontMatrix() [6]float64 { return f.fontMatrix }
// CharProcs are a Type 3 font's glyph drawings, by name.
func (f *Font) CharProcs() reader.Dict { return f.charProcs }
// Type3Resources are what a Type 3 font's glyph drawings draw with.
func (f *Font) Type3Resources() reader.Dict { return f.t3Res }
// Codes cuts a string into the codes it holds: one byte at a time for a simple
// font, two for a composite one, which is what every composite encoding seen in
// the wild uses.
func (f *Font) Codes(s []byte) []int {
if f.kind != Composite {
out := make([]int, len(s))
for i, b := range s {
out[i] = int(b)
}
return out
}
out := make([]int, 0, len(s)/2)
for i := 0; i+1 < len(s); i += 2 {
out = append(out, int(s[i])<<8|int(s[i+1]))
}
return out
}
// Width is how far the pen moves for one code, in text space — a multiple of
// the point size, so a width of half means half an em.
func (f *Font) Width(code int) float64 {
if w, ok := f.widths[code]; ok {
return w
}
return f.defaultW
}
// HasWidth reports whether the document said how wide this code is, rather
// than the width being the fallback [Font.Width] gives for one it did not.
//
// It matters for a font the document does not carry. One of the fourteen
// standard faces is written with no widths at all — every reader is expected
// to know Times and Helvetica and Courier by heart — so a reader drawing a
// stand-in has to take the advances from the stand-in too. Half an em for
// every letter reads as a typewriter, which is not what the page says.
func (f *Font) HasWidth(code int) bool {
_, ok := f.widths[code]
return ok
}
// GlyphName is what the document's encoding calls a code. ok is false for a
// composite font, which names nothing, and for a code the encoding passes over.
func (f *Font) GlyphName(code int) (string, bool) {
name, ok := f.names[code]
return name, ok
}
// Text is what a code stands for as characters: what the font's ToUnicode map
// says, or failing that what its glyph name says. ok is false when neither
// does — a subsetted font naming its glyphs g17 and carrying no map cannot be
// read back, and saying so is better than guessing.
func (f *Font) Text(code int) (string, bool) {
if s, ok := f.toUni[code]; ok && s != "" {
return s, true
}
if name, ok := f.names[code]; ok && f.namedByTheDocument(code) {
// Not one character: a name may say it is a ligature of several, and
// a code that stands for two letters has to give back two.
if text, ok := TextOfGlyphName(name); ok {
return text, true
}
}
if f.fallback != nil {
if s, ok := f.fallback(code); ok && s != "" {
return s, true
}
}
return "", false
}
// SetFallback installs a last resort for codes the document says nothing
// about: what the embedded font program calls them.
//
// A font that says it is symbolic is addressed through its own character map,
// and what that map says is the only honest answer for a code the document did
// not name. Reading the program is another package's work, so a caller that
// can do it hands the answer in here.
func (f *Font) SetFallback(fn func(code int) (string, bool)) { f.fallback = fn }
// namedByTheDocument reports whether a code's name can be believed as a
// statement about which character it is.
//
// A font that says it is symbolic is addressed through its own character map,
// and the base encoding is then a guess. It is a bad one: a mathematical font
// puts a capital gamma where the standard encoding puts an inverted
// exclamation mark, and reading the name would give the wrong letter with
// nothing to say so. A name the document chose itself, in a Differences
// array, is a different matter — that one it meant.
func (f *Font) namedByTheDocument(code int) bool {
return !f.symbolic || f.chosen[code]
}
// Chosen reports whether the document named this code itself rather than
// inheriting the name from a base encoding.
func (f *Font) Chosen(code int) bool { return f.chosen[code] }
// CIDToGID maps a character identifier to a glyph number for a composite font.
// ok is false when the font's map does not reach that far.
func (f *Font) CIDToGID(cid int) (int, bool) {
if f.cidToGID == nil {
return cid, true
}
i := cid * 2
if i+1 >= len(f.cidToGID) {
return 0, false
}
return int(f.cidToGID[i])<<8 | int(f.cidToGID[i+1]), true
}
// Program is the embedded font program: which key it arrived under, and its
// bytes. ok is false for a font whose program is not embedded, which is every
// standard face and many others besides.
func (f *Font) Program() (reader.Name, []byte, bool) {
for _, key := range []reader.Name{"FontFile2", "FontFile3", "FontFile"} {
stream, ok := reader.ToStream(resolve(f.doc, f.descriptor.Get(key)))
if !ok {
continue
}
data, img, err := f.doc.DecodeStream(stream)
if err != nil || img != "" {
continue
}
return key, data, true
}
return "", nil, false
}
// resolve follows an indirect reference. A document that opened cannot fail to
// resolve one, so there is nothing to handle at every use.
func resolve(d *reader.Document, o reader.Object) reader.Object {
out, _ := d.Resolve(o)
return out
}
// floatOf reads a number, or gives a default.
func floatOf(o reader.Object, def float64) float64 {
if v, ok := reader.ToFloat(o); ok {
return v
}
return def
}
vendor/github.com/go-pdfkit/pdffont/greek.go
Generated Vendored
+69
View File
@@ -0,0 +1,69 @@
package pdffont
// greekAndMathRunes are the names a mathematical document gives its glyphs.
// They are not in the Latin encodings, so a page of equations addressed by
// name — which is what every TeX document is — could not be read back without
// them: the words came out and the symbols did not.
//
// The names are Adobe's, from the Symbol font and the glyph list every
// producer follows. They are checked against the documents themselves rather
// than taken on trust: a font that carries both a name for a code and a
// ToUnicode entry for it states the same fact twice, and the corpus has tens
// of thousands of such pairs to compare against.
var greekAndMathRunes = map[string]rune{
// The Greek alphabet, lower case and upper.
"alpha": 'α', "beta": 'β', "gamma": 'γ', "delta": 'δ', "epsilon": 'ε',
"zeta": 'ζ', "eta": 'η', "theta": 'θ', "iota": 'ι', "kappa": 'κ',
"lambda": 'λ', "mu1": 'μ', "nu": 'ν', "xi": 'ξ', "omicron": 'ο',
"pi": 'π', "rho": 'ρ', "sigma": 'σ', "sigma1": 'ς', "tau": 'τ',
"upsilon": 'υ', "phi": 'φ', "chi": 'χ', "psi": 'ψ', "omega": 'ω',
"theta1": 'ϑ', "phi1": 'ϕ', "epsilon1": 'ϵ', "rho1": 'ϱ', "omega1": 'ϖ',
"kappa1": 'ϰ', "pi1": 'ϖ',
// Adobe's list gives Delta and Omega the increment and ohm signs, which
// Unicode says are the same characters as the Greek letters written
// here — canonically equivalent, and these are the forms a person
// searching the text would type.
"Alpha": 'Α', "Beta": 'Β', "Gamma": 'Γ', "Delta": 'Δ', "Epsilon": 'Ε',
"Zeta": 'Ζ', "Eta": 'Η', "Theta": 'Θ', "Iota": 'Ι', "Kappa": 'Κ',
"Lambda": 'Λ', "Mu": 'Μ', "Nu": 'Ν', "Xi": 'Ξ', "Omicron": 'Ο',
"Pi": 'Π', "Rho": 'Ρ', "Sigma": 'Σ', "Tau": 'Τ', "Upsilon": 'Υ',
"Phi": 'Φ', "Chi": 'Χ', "Psi": 'Ψ', "Omega": 'Ω', "Upsilon1": 'ϒ',
// The signs an equation is built from.
"minus": '−', "plus": '+', "equal": '=', "asteriskmath": '∗',
"lessequal": '≤', "greaterequal": '≥', "notequal": '≠', "approxequal": '≈',
"equivalence": '≡', "similar": '∼', "congruent": '≅', "proportional": '∝',
"infinity": '∞', "partialdiff": '∂', "gradient": '∇', "integral": '∫',
"summation": '∑', "product": '∏', "radical": '√', "angle": '∠',
"perpendicular": '⊥', "therefore": '∴', "dotmath": '⋅', "bullet3": '∙',
"element": '∈', "notelement": '∉', "universal": '∀', "existential": '∃',
"emptyset": '∅', "intersection": '∩', "union": '∪',
"propersubset": '⊂', "propersuperset": '⊃',
"reflexsubset": '⊆', "reflexsuperset": '⊇', "notsubset": '⊄',
"logicaland": '∧', "logicalor": '∨', "logicalnot1": '¬',
"circleplus": '⊕', "circlemultiply": '⊗',
"prime": '′', "second": '″', "minute": '′',
// Arrows, and the brackets a display equation is built out of.
"arrowleft": '←', "arrowup": '↑', "arrowright": '→', "arrowdown": '↓',
"arrowboth": '↔', "arrowupdn": '↕',
"arrowdblleft": '⇐', "arrowdblup": '⇑', "arrowdblright": '⇒',
"arrowdbldown": '⇓', "arrowdblboth": '⇔',
"angleleft": '⟨', "angleright": '⟩',
"bracketlefttp": '⎡', "bracketleftbt": '⎣', "bracketleftex": '⎢',
"bracketrighttp": '⎤', "bracketrightbt": '⎦', "bracketrightex": '⎥',
"parenlefttp": '⎛', "parenleftbt": '⎝', "parenleftex": '⎜',
"parenrighttp": '⎞', "parenrightbt": '⎠', "parenrightex": '⎟',
"bracelefttp": '⎧', "braceleftmid": '⎨', "braceleftbt": '⎩',
"bracerighttp": '⎫', "bracerightmid": '⎬', "bracerightbt": '⎭',
"braceex": '⎪',
// The ligatures and marks a book is set with.
"ff": 'ff', "ffi": 'ffi', "ffl": 'ffl', "dotlessj": 'ȷ',
"circumflexbig": 'ˆ', "tildebig": '˜',
"suchthat": '∍', "carriagereturn": '↵', "aleph": 'ℵ',
"Ifraktur": 'ℑ', "Rfraktur": 'ℜ', "weierstrass": '℘',
"circlecopyrt": '©', "trademarksans": '™', "trademarkserif": '™',
"degree1": '°', "florin1": 'ƒ',
"lozenge": '◊', "spade": '♠', "club": '♣', "heart": '♥', "diamond": '♦',
}
vendor/github.com/go-pdfkit/pdffont/read.go
Generated Vendored
+223
View File
@@ -0,0 +1,223 @@
package pdffont
import "github.com/go-pdfkit/reader"
// readSimple reads a font addressed one byte at a time: what each code is
// called, and how wide it is.
func (f *Font) readSimple() {
f.descriptor, _ = f.doc.GetDict(f.dict, "FontDescriptor")
f.symbolic = symbolicFlag(f.doc, f.descriptor)
f.names = f.readEncoding()
f.readSimpleWidths()
}
// readType3 reads a font whose glyphs are little drawings.
func (f *Font) readType3() {
f.descriptor, _ = f.doc.GetDict(f.dict, "FontDescriptor")
f.names = f.readEncoding()
f.charProcs, _ = f.doc.GetDict(f.dict, "CharProcs")
f.t3Res, _ = f.doc.GetDict(f.dict, "Resources")
if m := floatArray(f.doc, f.dict.Get("FontMatrix")); len(m) >= 6 {
copy(f.fontMatrix[:], m[:6])
}
f.readSimpleWidths()
// A Type 3 font's widths are in its own glyph space, which the font
// matrix maps onto text space.
for code, w := range f.widths {
f.widths[code] = w * 1000 * f.fontMatrix[0]
}
f.defaultW = 0
}
// symbolicFlag reads whether a font says it is addressed through its own
// character map rather than through an encoding of names.
func symbolicFlag(d *reader.Document, descriptor reader.Dict) bool {
flags, ok := reader.ToInt(resolve(d, descriptor.Get("Flags")))
if !ok {
return false
}
const symbolic, nonSymbolic = 1 << 2, 1 << 5
return flags&symbolic != 0 && flags&nonSymbolic == 0
}
// readEncoding works out what each code of a simple font is called: a base
// table, and then whatever differences the font names on top of it.
func (f *Font) readEncoding() map[int]string {
base := StandardEncoding
out := map[int]string{}
f.chosen = map[int]bool{}
enc := resolve(f.doc, f.dict.Get("Encoding"))
if name, ok := reader.ToName(enc); ok {
table := NamedEncoding(name, base)
fillNames(out, table)
// A named encoding is the document's own word on what its codes are,
// symbolic font or not.
for code, n := range table {
if n != "" {
f.chosen[code] = true
}
}
return out
}
encDict, ok := reader.ToDict(enc)
if !ok {
fillNames(out, base)
return out
}
if name, ok := reader.ToName(resolve(f.doc, encDict.Get("BaseEncoding"))); ok {
base = NamedEncoding(name, base)
}
fillNames(out, base)
f.applyDifferences(out, encDict)
return out
}
// NamedEncoding is the table a name stands for, or the fallback for a name
// that is not one of them.
func NamedEncoding(name reader.Name, fallback [256]string) [256]string {
switch name {
case "WinAnsiEncoding":
return WinAnsiEncoding
case "StandardEncoding", "MacRomanEncoding", "MacExpertEncoding":
// Mac Roman differs from the standard encoding above code 127, where
// almost nothing in a Latin document lives; using the standard table
// is closer than using none.
return StandardEncoding
}
return fallback
}
// fillNames copies a table into the map the rest of this works from.
func fillNames(out map[int]string, table [256]string) {
for code, name := range table {
if name != "" {
out[code] = name
}
}
}
// applyDifferences reads the array that names glyphs a code at a time: a
// number, then the names of the codes from there on.
func (f *Font) applyDifferences(out map[int]string, encDict reader.Dict) {
arr, ok := reader.ToArray(resolve(f.doc, encDict.Get("Differences")))
if !ok {
return
}
code := 0
for _, e := range arr {
v := resolve(f.doc, e)
if n, ok := reader.ToInt(v); ok {
code = int(n)
continue
}
if name, ok := reader.ToName(v); ok {
if code >= 0 && code < 256 {
out[code] = string(name)
f.chosen[code] = true
}
code++
}
}
}
// readSimpleWidths reads /Widths, which gives one width per code from
// /FirstChar on.
func (f *Font) readSimpleWidths() {
first, ok := reader.ToInt(resolve(f.doc, f.dict.Get("FirstChar")))
if !ok {
return
}
arr, ok := reader.ToArray(resolve(f.doc, f.dict.Get("Widths")))
if !ok {
return
}
for i, e := range arr {
if v, ok := reader.ToFloat(resolve(f.doc, e)); ok {
f.widths[int(first)+i] = v / 1000
}
}
if mw := floatOf(resolve(f.doc, f.descriptor.Get("MissingWidth")), -1); mw >= 0 {
f.defaultW = mw / 1000
}
}
// readComposite reads a font addressed by character identifier: its descendant
// holds the widths, the descriptor and the map from identifier to glyph.
func (f *Font) readComposite() {
f.defaultW = 1
arr, ok := reader.ToArray(resolve(f.doc, f.dict.Get("DescendantFonts")))
if !ok || len(arr) == 0 {
return
}
kid, ok := reader.ToDict(resolve(f.doc, arr[0]))
if !ok {
return
}
if v, ok := reader.ToFloat(resolve(f.doc, kid.Get("DW"))); ok {
f.defaultW = v / 1000
}
f.readCIDWidths(kid)
f.descriptor, _ = f.doc.GetDict(kid, "FontDescriptor")
f.symbolic = symbolicFlag(f.doc, f.descriptor)
if stream, ok := reader.ToStream(resolve(f.doc, kid.Get("CIDToGIDMap"))); ok {
if data, img, err := f.doc.DecodeStream(stream); err == nil && img == "" {
f.cidToGID = data
}
}
}
// readCIDWidths reads the /W array, which names widths one identifier at a
// time or a run at a time.
func (f *Font) readCIDWidths(kid reader.Dict) {
arr, ok := reader.ToArray(resolve(f.doc, kid.Get("W")))
if !ok {
return
}
for i := 0; i < len(arr); {
first, ok := reader.ToInt(resolve(f.doc, arr[i]))
if !ok || i+1 >= len(arr) {
return
}
next := resolve(f.doc, arr[i+1])
if list, ok := reader.ToArray(next); ok {
for k, e := range list {
if v, ok := reader.ToFloat(resolve(f.doc, e)); ok {
f.widths[int(first)+k] = v / 1000
}
}
i += 2
continue
}
last, ok := reader.ToInt(next)
if !ok || i+2 >= len(arr) {
return
}
w, ok := reader.ToFloat(resolve(f.doc, arr[i+2]))
if !ok {
return
}
if last >= first && last-first < 1<<20 {
for c := first; c <= last; c++ {
f.widths[int(c)] = w / 1000
}
}
i += 3
}
}
// floatArray reads an array of numbers, or nothing.
func floatArray(d *reader.Document, o reader.Object) []float64 {
arr, ok := reader.ToArray(resolve(d, o))
if !ok {
return nil
}
out := make([]float64, 0, len(arr))
for _, e := range arr {
v, ok := reader.ToFloat(resolve(d, e))
if !ok {
return nil
}
out = append(out, v)
}
return out
}
vendor/github.com/go-pdfkit/pdffont/tounicode.go
Generated Vendored
+252
View File
@@ -0,0 +1,252 @@
package pdffont
import (
"strings"
"unicode/utf16"
"github.com/go-pdfkit/reader"
)
// readToUnicode reads the font's /ToUnicode map, which is the one thing in a
// PDF that says what its text actually says. Without it a page can be drawn
// but not read: a subsetted font naming its glyphs by number carries no other
// clue.
func (f *Font) readToUnicode() map[int]string {
stream, ok := reader.ToStream(resolve(f.doc, f.dict.Get("ToUnicode")))
if !ok {
return nil
}
data, img, err := f.doc.DecodeStream(stream)
if err != nil || img != "" {
return nil
}
return ReadToUnicode(data)
}
// ReadToUnicode decodes a ToUnicode CMap: a little PostScript program whose
// only meaningful parts are the bfchar and bfrange blocks saying which
// characters each code stands for.
func ReadToUnicode(data []byte) map[int]string {
out := map[int]string{}
toks := cmapTokens(data)
for i := 0; i < len(toks); i++ {
switch toks[i].word {
case "beginbfchar":
i = readBFChar(toks, i+1, out)
case "beginbfrange":
i = readBFRange(toks, i+1, out)
}
}
if len(out) == 0 {
return nil
}
return out
}
// maxToUnicodeEntries is how many codes one map may name.
//
// A bfrange names a run of codes in about twenty bytes, and only the width of
// a single run was bounded — not how many runs a map could hold. So the size
// of the answer had nothing to do with the size of the question: 585 bytes
// produced 655 360 entries in 2.3 seconds, and 10 655 bytes produced
// 13 107 200 entries and a gigabyte. Every font on every page is read this
// way, so that was a gigabyte per font.
//
// The bound is not a guess. Across 5 338 /ToUnicode maps taken out of real
// documents — arXiv figures, government forms, and mozilla's pdf.js corpus —
// the median names 13 codes, the 99th percentile names 538, and the largest
// names exactly 65 536: one whole two-byte code space, which is as many codes
// as a font of that shape can have. This allows four times that, so a
// document has to be malformed or hostile to reach it, and a map that does is
// cut off there rather than being allowed to ask for everything.
const maxToUnicodeEntries = 1 << 18
// A cmapToken is one piece of such a program: a hexadecimal string, an array
// bracket, or a bare word.
type cmapToken struct {
word string
hex []byte
isHex bool
}
// cmapTokens cuts the program up. Only three shapes matter — a hexadecimal
// string, a bracket, and a word — and everything else can be a word.
func cmapTokens(b []byte) []cmapToken {
var out []cmapToken
for i := 0; i < len(b); {
c := b[i]
switch {
case c == '%':
for i < len(b) && b[i] != '\n' && b[i] != '\r' {
i++
}
case c == '<':
j := i + 1
for j < len(b) && b[j] != '>' {
j++
}
out = append(out, cmapToken{hex: unhexBytes(b[i+1 : min(j, len(b))]), isHex: true})
i = min(j+1, len(b))
case c == '[' || c == ']':
out = append(out, cmapToken{word: string(c)})
i++
case c <= ' ':
i++
default:
j := i
for j < len(b) && b[j] > ' ' && b[j] != '<' && b[j] != '[' && b[j] != ']' && b[j] != '%' {
j++
}
out = append(out, cmapToken{word: string(b[i:j])})
i = j
}
}
return out
}
// readBFChar reads pairs of "code, what it stands for" up to the block's end.
func readBFChar(toks []cmapToken, i int, out map[int]string) int {
for i+1 < len(toks) {
if toks[i].word == "endbfchar" {
return i
}
if len(out) >= maxToUnicodeEntries {
return i
}
if !toks[i].isHex || !toks[i+1].isHex {
return i
}
out[codeOf(toks[i].hex)] = textOf(toks[i+1].hex)
i += 2
}
return i
}
// readBFRange reads runs of codes: either a first character the run counts on
// from, or a list naming each one.
func readBFRange(toks []cmapToken, i int, out map[int]string) int {
for i+2 < len(toks) {
if toks[i].word == "endbfrange" {
return i
}
if !toks[i].isHex || !toks[i+1].isHex {
return i
}
lo, hi := codeOf(toks[i].hex), codeOf(toks[i+1].hex)
if hi < lo || hi-lo > 1<<16 {
return i
}
if len(out) >= maxToUnicodeEntries {
return i
}
switch {
case toks[i+2].isHex:
base := utf16Runes(toks[i+2].hex)
for c := lo; c <= hi && len(out) < maxToUnicodeEntries; c++ {
out[c] = countOn(base, c-lo)
}
i += 3
case toks[i+2].word == "[":
j := i + 3
for c := lo; c <= hi && j < len(toks) && toks[j].word != "]"; c++ {
if toks[j].isHex && len(out) < maxToUnicodeEntries {
out[c] = textOf(toks[j].hex)
}
j++
}
for j < len(toks) && toks[j].word != "]" {
j++
}
i = j + 1
default:
return i
}
}
return i
}
// codeOf reads a code, which is one or two bytes written as hexadecimal.
func codeOf(b []byte) int {
v := 0
for _, c := range b {
v = v<<8 | int(c)
}
return v
}
// textOf reads what a code stands for: characters written as sixteen-bit
// units, the way the format carries text.
func textOf(b []byte) string {
return string(utf16Runes(b))
}
// utf16Runes decodes the sixteen-bit units a ToUnicode value is written in.
func utf16Runes(b []byte) []rune {
units := make([]uint16, 0, len(b)/2)
for i := 0; i+1 < len(b); i += 2 {
units = append(units, uint16(b[i])<<8|uint16(b[i+1]))
}
return utf16.Decode(units)
}
// countOn is what the n'th code of a range stands for, counting on from the
// first: the last character moves and the rest stay, which is what the
// specification says a range means.
func countOn(base []rune, n int) string {
if len(base) == 0 {
return ""
}
out := append([]rune{}, base...)
out[len(out)-1] += rune(n)
return string(out)
}
// unhexBytes reads a hexadecimal string, ignoring whatever it was laid out
// with. An odd digit at the end is taken as the high half of a byte, which is
// what the format says.
func unhexBytes(b []byte) []byte {
var digits []byte
for _, c := range b {
if isHexDigit(c) {
digits = append(digits, hexVal(c))
}
}
if len(digits)%2 == 1 {
digits = append(digits, 0)
}
out := make([]byte, 0, len(digits)/2)
for i := 0; i+1 < len(digits); i += 2 {
out = append(out, digits[i]<<4|digits[i+1])
}
return out
}
func isHexDigit(c byte) bool {
return c >= '0' && c <= '9' || c >= 'a' && c <= 'f' || c >= 'A' && c <= 'F'
}
func hexVal(c byte) byte {
switch {
case c <= '9':
return c - '0'
case c <= 'F':
return c - 'A' + 10
default:
return c - 'a' + 10
}
}
// min is the smaller of two numbers.
func min(a, b int) int {
if a < b {
return a
}
return b
}
// TrimText tidies what a map gives back: the format allows a value to carry
// the replacement character or nothing at all, and neither is text.
func TrimText(s string) string {
s = strings.TrimFunc(s, func(r rune) bool { return r == 0 || r == 0xFFFD })
return s
}
vendor/github.com/go-pdfkit/reader/.gitignore
Generated
+3
View File
@@ -0,0 +1,3 @@
cov.out
coverage.out
*.pdf
vendor/github.com/go-pdfkit/reader/LICENSE
Generated Vendored
+29
View File
@@ -0,0 +1,29 @@
BSD 3-Clause License
Copyright (c) 2026 the go-pdfkit/reader authors
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
vendor/github.com/go-pdfkit/reader/README.md
Generated Vendored
+144
View File
@@ -0,0 +1,144 @@
# reader
[![CI](https://github.com/go-pdfkit/reader/actions/workflows/ci.yml/badge.svg)](https://github.com/go-pdfkit/reader/actions/workflows/ci.yml)
[![Go Reference](https://pkg.go.dev/badge/github.com/go-pdfkit/reader.svg)](https://pkg.go.dev/github.com/go-pdfkit/reader)
[![Go Report Card](https://goreportcard.com/badge/github.com/go-pdfkit/reader)](https://goreportcard.com/report/github.com/go-pdfkit/reader)
[![License: BSD-3-Clause](https://img.shields.io/badge/License-BSD--3--Clause-blue.svg)](LICENSE)
[![Coverage](https://img.shields.io/badge/coverage-100%25-brightgreen.svg)](#testing)
A pure-Go, **zero-C** PDF **reader** — the parsing half of
[go-pdfkit](https://github.com/go-pdfkit). Where
[`pdfkit`](https://github.com/go-pdfkit/pdfkit) authors new PDF 1.7, this
module takes existing PDF bytes apart: lexer, object model, cross-reference
tables and streams, stream filters, the standard security handler, the page
tree, and content streams. It also writes the same object graph back out,
which is what every operation on an existing file needs.
Nothing outside the Go standard library is required, so it builds for
`GOOS=js/wasm` and every 64-bit architecture the fleet targets.
## Status
Landing in waves. This module currently provides the **object layer** and
the **document structure**:
- **Lexer** — the full PDF object syntax: integers and reals in the shapes
producers actually write, literal strings with their escapes, hex strings,
names with `#xx` escapes, arrays, dictionaries, streams, and keywords.
- **Objects** — `Null`, `Bool`, `Integer`, `Real`, `String`, `Name`, `Array`,
`Dict`, `Stream`, and indirect `Ref`, with the `N G R` / `N G obj` lookahead
resolved the way the specification requires.
- **Filters** — `FlateDecode`, `LZWDecode` (with PDF's `EarlyChange`, which the
standard library's LZW does not implement), `ASCIIHexDecode`,
`ASCII85Decode`, `RunLengthDecode`, `Crypt`, plus the PNG and TIFF
predictors. Image filters (`DCTDecode`, `JPXDecode`, `CCITTFaxDecode`,
`JBIG2Decode`) are reported rather than applied, so an image consumer gets
the encoded bytes.
- **Filters that fail** — a chain that cannot be run to the end is not the end
of the stream. `DecodeRecovering` returns what the filters before the failure
produced — the prefix a damaged Flate stream did inflate, not its compressed
bytes — and says so through `Decoded.Recovered` and `Decoded.Cause`. Bytes no
filter decoded arrive in `Decoded.Undecoded`, never in `Decoded.Data`, so a
compressed stream cannot be painted as samples or tokenised as content by a
caller that forgot to check a flag.
`Decode` is the strict reading and refuses such a stream outright, so a
caller that must not act on damaged data says which it wants by which one it
calls. Page content takes the lenient reading, because a page whose content
will not decode is still a page; `PageContentDecoded` reports whether any
salvaging happened. Cross-reference streams take the strict one: a table that
has to be guessed at is worse than no table, and the repair below is the
answer to one.
- **Cross-references** — classic tables, cross-reference **streams**, **object
streams**, `/Prev` chains, and the `/XRefStm` of a hybrid file, newest
definition winning.
- **Repair** — a file whose tables are missing, truncated or simply wrong is
rebuilt by scanning it for object headers, trailers and, failing those, for
a catalogue; a file that kept its pages but lost its catalogue gets one.
This is not an exceptional path: it is what makes a reader usable. The
rebuild establishes the file key before it reads object streams, so a file
whose catalogue lives in an encrypted one can be rebuilt at all; it takes
`/Encrypt` from a cross-reference stream's dictionary when the file has no
trailer keyword to name it in. Where two object streams claim the same
object, the later one in the file wins, the same rule the header scan
applies to objects written directly — so the same damaged file reads the
same way twice. When a rebuild fails, the error says what the rebuild found
and not merely that the tables could not be read.
- **Documents** — `Open`, object resolution with cycle and recursion guards,
the trailer, the catalogue, and the page tree with the four attributes a
page inherits from its ancestors.
- **Encryption** — the standard security handler in every revision, both ways:
reading RC4 at 40 and 128 bits, AESV2, and the AES-256 of `/R 5` and `/R 6`,
with crypt filters, `/Identity` and `/EncryptMetadata`; and writing AES-256
or AES-128, with permissions, a user password and an owner password.
A password is tried as the user password and as the owner password, and
`Open` uses the empty one, so a file protected only against editing opens
with no password at all.
and steps over rubbish rather than losing the operations around it, with
inline images read whole. Where an inline image ends is the one genuinely
ambiguous thing in a content stream, since its data may spell EI itself;
the length is computed from the image's own geometry where it can be, and
otherwise every candidate EI is tried until the data before one actually
decodes.
- **Writing** — the exact inverse of the parser: objects rendered in PDF
syntax with stable dictionary ordering and strings in whichever of the two
forms is shorter, and a `Writer` that copies whole object graphs out of one
or more documents, renumbering as it goes, then lays down a cross-reference
table and a trailer. `NewPackedWriter` instead puts what it can into
compressed **object streams** and ends in a **cross-reference stream**,
which is what makes a file small.
Measured against a corpus of **118 863 real PDFs** — Matplotlib, cairo,
pdfTeX, Ghostscript, Adobe, R, Apache FOP, PDF 1.3 through 1.7 — `Open`
succeeds on **118 833** of them and finds 138 337 pages, in 8 seconds, with no
panics. Of the thirty it refuses, twenty-seven are PNG files with a `.pdf`
extension, one is PostScript, and two are PDFs truncated past recovery.
The one encrypted PDF the corpus holds — `/V 4 /R 4`, written by macOS
Quartz — decrypts to readable metadata and a valid content stream, which is
the only genuinely independent check of the key derivation there is here;
the other revisions are round-tripped against a test encryptor written from
the same algorithms.
The content-stream tokeniser reads **1 536 769 753 operations** across those
138 337 pages in a minute and a half, with no panics, no page failing to
decode, and no operator outside the seventy the format defines — beyond four
`arc` and six `nan` written by a producer that was simply wrong.
Rewriting is checked the same way: every one of the **118 833** files that
open is copied object by object into a new file, which is then re-read and
compared on page count, media boxes and the bytes of every content stream.
**All 118 833 match**, 35.5 GB in and 35.2 GB out, in twenty-eight seconds —
and **all 118 833 match again** when the packed writer is used instead, which
brings the output down to **94.6%** of the input.
Next wave: the operations built on all of this.
## Install
```sh
go get github.com/go-pdfkit/reader
```
## Testing
```sh
go test -covermode=set ./...
```
CI gates on **exact 100% statement coverage**, `go vet`, and a cross-compile
across `linux/{amd64,arm64,riscv64,loong64,ppc64le,s390x}`, `js/wasm`,
`darwin/arm64` and `windows/amd64`.
## The rest of the family
[**reader**](https://github.com/go-pdfkit/reader) reads and writes the format ·
[**ops**](https://github.com/go-pdfkit/ops) is the verbs and the `pdfops` command ·
[**render**](https://github.com/go-pdfkit/render) turns a page into pixels ·
[**pdffont**](https://github.com/go-pdfkit/pdffont) is what a document says about a font ·
[**extract**](https://github.com/go-pdfkit/extract) reads a page back ·
[**coedit**](https://github.com/go-pdfkit/coedit) is a PDF several people edit at once ·
[**app**](https://github.com/go-pdfkit/app) is the browser workbench.
Documentation for all of it: <https://go-pdfkit.github.io/docs/>
## License
BSD-3-Clause — see [LICENSE](LICENSE). Copyright the go-pdfkit/reader authors.
vendor/github.com/go-pdfkit/reader/ccitt.go
Generated Vendored
+579
View File
@@ -0,0 +1,579 @@
package reader
import "fmt"
// CCITT Group 3 and Group 4 fax decoding, as used by /CCITTFaxDecode.
//
// WHY THIS IS HERE
//
// A scanned form is a fax. 67 of the 1 633 real forms in the corpus — the
// eleven issuing bodies, not the vendor test suites — carry a CCITT-encoded
// image, 263 images between them, and 6 of their pages have nothing else on
// them at all. Those six pages drew entirely blank, and a blank page is the
// failure a reader notices before any other.
//
// WHY IT BELONGS IN THE READER RATHER THAN IN A RENDERER
//
// The other filters this package stops at — DCT, JPX, JBIG2 — carry an image
// with its own idea of how many components it has and how deep they are. This
// one does not: it produces bilevel samples, one bit a pixel, rows padded to a
// byte, and the stream dictionary says what they mean. That is a byte stream,
// so it is a filter, and every caller gets it rather than each writing its own.
//
// THE REFERENCE READ BEFORE WRITING THIS
//
// ITU-T Recommendations T.4 (Group 3) and T.6 (Group 4), by way of the tables
// and the changing-element algorithm in golang.org/x/image/ccitt — read, not
// imported, because this package has no dependencies outside the standard
// library and gains none. The code tables below were extracted mechanically
// from that package's gen.go rather than retyped, because a table of 218
// variable-length codes transcribed by hand is a table with a mistake in it.
//
// The algorithm is the one T.6 Figure 1 describes. A row is decoded into one
// byte a pixel — 0xFF white, 0x00 black — and the row before it is the
// reference against which the two-dimensional modes are resolved:
//
// b1 b2
// v v
// prev: BBBBBwwwwwBBBwwwww
// curr: BBBwwwwwBBBBBBwwww
// ^ ^ ^
// a0 a1 a2
//
// a0 is where the pen is; a1 the next colour change to its right on this row;
// b1 the first change on the row above that is to the right of a0 and of the
// opposite colour to it; b2 the next change after b1. Pass mode says a1 is at
// or beyond b2; horizontal mode reads two runs and ignores the row above;
// vertical mode puts a1 at b1 plus an offset between -3 and +3.
// The two-dimensional modes of T.4 Table 1.
const (
modePass = iota
modeH
modeV0
modeVR1
modeVR2
modeVR3
modeVL1
modeVL2
modeVL3
modeExt
)
// A ccittCode is one row of a code table: the value a code names, and the bits
// that name it, most significant first.
type ccittCode struct {
value int
bits string
}
// ccittModeCodes is Table 1 of ITU-T T.4: the two-dimensional modes.
var ccittModeCodes = []ccittCode{
{modePass, "0001"}, {modeH, "001"}, {modeV0, "1"}, {modeVR1, "011"},
{modeVR2, "000011"}, {modeVR3, "0000011"}, {modeVL1, "010"}, {modeVL2, "000010"},
{modeVL3, "0000010"}, {modeExt, "0000001"},
}
// ccittWhiteCodes is Tables 2 and 3 of ITU-T T.4 for a white run: the
// terminating codes 0 to 63, then the make-up codes in steps of 64.
var ccittWhiteCodes = []ccittCode{
{0, "00110101"}, {1, "000111"}, {2, "0111"}, {3, "1000"},
{4, "1011"}, {5, "1100"}, {6, "1110"}, {7, "1111"},
{8, "10011"}, {9, "10100"}, {10, "00111"}, {11, "01000"},
{12, "001000"}, {13, "000011"}, {14, "110100"}, {15, "110101"},
{16, "101010"}, {17, "101011"}, {18, "0100111"}, {19, "0001100"},
{20, "0001000"}, {21, "0010111"}, {22, "0000011"}, {23, "0000100"},
{24, "0101000"}, {25, "0101011"}, {26, "0010011"}, {27, "0100100"},
{28, "0011000"}, {29, "00000010"}, {30, "00000011"}, {31, "00011010"},
{32, "00011011"}, {33, "00010010"}, {34, "00010011"}, {35, "00010100"},
{36, "00010101"}, {37, "00010110"}, {38, "00010111"}, {39, "00101000"},
{40, "00101001"}, {41, "00101010"}, {42, "00101011"}, {43, "00101100"},
{44, "00101101"}, {45, "00000100"}, {46, "00000101"}, {47, "00001010"},
{48, "00001011"}, {49, "01010010"}, {50, "01010011"}, {51, "01010100"},
{52, "01010101"}, {53, "00100100"}, {54, "00100101"}, {55, "01011000"},
{56, "01011001"}, {57, "01011010"}, {58, "01011011"}, {59, "01001010"},
{60, "01001011"}, {61, "00110010"}, {62, "00110011"}, {63, "00110100"},
{64, "11011"}, {128, "10010"}, {192, "010111"}, {256, "0110111"},
{320, "00110110"}, {384, "00110111"}, {448, "01100100"}, {512, "01100101"},
{576, "01101000"}, {640, "01100111"}, {704, "011001100"}, {768, "011001101"},
{832, "011010010"}, {896, "011010011"}, {960, "011010100"}, {1024, "011010101"},
{1088, "011010110"}, {1152, "011010111"}, {1216, "011011000"}, {1280, "011011001"},
{1344, "011011010"}, {1408, "011011011"}, {1472, "010011000"}, {1536, "010011001"},
{1600, "010011010"}, {1664, "011000"}, {1728, "010011011"}, {1792, "00000001000"},
{1856, "00000001100"}, {1920, "00000001101"}, {1984, "000000010010"}, {2048, "000000010011"},
{2112, "000000010100"}, {2176, "000000010101"}, {2240, "000000010110"}, {2304, "000000010111"},
{2368, "000000011100"}, {2432, "000000011101"}, {2496, "000000011110"}, {2560, "000000011111"},
}
// ccittBlackCodes is Tables 2 and 3 of ITU-T T.4 for a black run.
var ccittBlackCodes = []ccittCode{
{0, "0000110111"}, {1, "010"}, {2, "11"}, {3, "10"},
{4, "011"}, {5, "0011"}, {6, "0010"}, {7, "00011"},
{8, "000101"}, {9, "000100"}, {10, "0000100"}, {11, "0000101"},
{12, "0000111"}, {13, "00000100"}, {14, "00000111"}, {15, "000011000"},
{16, "0000010111"}, {17, "0000011000"}, {18, "0000001000"}, {19, "00001100111"},
{20, "00001101000"}, {21, "00001101100"}, {22, "00000110111"}, {23, "00000101000"},
{24, "00000010111"}, {25, "00000011000"}, {26, "000011001010"}, {27, "000011001011"},
{28, "000011001100"}, {29, "000011001101"}, {30, "000001101000"}, {31, "000001101001"},
{32, "000001101010"}, {33, "000001101011"}, {34, "000011010010"}, {35, "000011010011"},
{36, "000011010100"}, {37, "000011010101"}, {38, "000011010110"}, {39, "000011010111"},
{40, "000001101100"}, {41, "000001101101"}, {42, "000011011010"}, {43, "000011011011"},
{44, "000001010100"}, {45, "000001010101"}, {46, "000001010110"}, {47, "000001010111"},
{48, "000001100100"}, {49, "000001100101"}, {50, "000001010010"}, {51, "000001010011"},
{52, "000000100100"}, {53, "000000110111"}, {54, "000000111000"}, {55, "000000100111"},
{56, "000000101000"}, {57, "000001011000"}, {58, "000001011001"}, {59, "000000101011"},
{60, "000000101100"}, {61, "000001011010"}, {62, "000001100110"}, {63, "000001100111"},
{64, "0000001111"}, {128, "000011001000"}, {192, "000011001001"}, {256, "000001011011"},
{320, "000000110011"}, {384, "000000110100"}, {448, "000000110101"}, {512, "0000001101100"},
{576, "0000001101101"}, {640, "0000001001010"}, {704, "0000001001011"}, {768, "0000001001100"},
{832, "0000001001101"}, {896, "0000001110010"}, {960, "0000001110011"}, {1024, "0000001110100"},
{1088, "0000001110101"}, {1152, "0000001110110"}, {1216, "0000001110111"}, {1280, "0000001010010"},
{1344, "0000001010011"}, {1408, "0000001010100"}, {1472, "0000001010101"}, {1536, "0000001011010"},
{1600, "0000001011011"}, {1664, "0000001100100"}, {1728, "0000001100101"}, {1792, "00000001000"},
{1856, "00000001100"}, {1920, "00000001101"}, {1984, "000000010010"}, {2048, "000000010011"},
{2112, "000000010100"}, {2176, "000000010101"}, {2240, "000000010110"}, {2304, "000000010111"},
{2368, "000000011100"}, {2432, "000000011101"}, {2496, "000000011110"}, {2560, "000000011111"},
}
// A ccittTable decodes one of the code tables. Codes are grouped by their
// length, which is the whole of the decoding: read a bit, lengthen the code by
// one, and ask whether a code of that length says anything. A prefix-free code
// cannot answer twice, so the first answer is the only one.
type ccittTable struct {
byLength []map[uint32]int
}
// newCCITTTable groups a table's codes by length. The bits are read from the
// strings the tables are written in, so what the code says and what the
// specification says are the same text.
func newCCITTTable(codes []ccittCode) *ccittTable {
longest := 0
for _, c := range codes {
if len(c.bits) > longest {
longest = len(c.bits)
}
}
t := &ccittTable{byLength: make([]map[uint32]int, longest+1)}
for _, c := range codes {
var v uint32
for _, b := range c.bits {
v <<= 1
if b == '1' {
v |= 1
}
}
n := len(c.bits)
if t.byLength[n] == nil {
t.byLength[n] = map[uint32]int{}
}
t.byLength[n][v] = c.value
}
return t
}
var (
ccittModes = newCCITTTable(ccittModeCodes)
ccittWhite = newCCITTTable(ccittWhiteCodes)
ccittBlack = newCCITTTable(ccittBlackCodes)
)
// A ccittBits reads one bit at a time, most significant first, which is the
// order a fax is written in.
type ccittBits struct {
data []byte
pos int // in bits
}
func (b *ccittBits) next() (uint32, bool) {
if b.pos >= 8*len(b.data) {
return 0, false
}
bit := (b.data[b.pos/8] >> (7 - uint(b.pos%8))) & 1
b.pos++
return uint32(bit), true
}
// align moves to the next byte boundary, which /EncodedByteAlign asks for at
// the start of every row.
func (b *ccittBits) align() { b.pos = (b.pos + 7) &^ 7 }
func (b *ccittBits) exhausted() bool { return b.pos >= 8*len(b.data) }
// restIsFill says whether nothing but zero bits remain. A fax is padded to a
// byte, and often to rather more than a byte, and zeros decode as perfectly
// good two-dimensional modes: pass mode is 0001 and vertical-left-3 is 0000010,
// so a decoder that does not stop here goes on inventing rows that look like
// the last real one. One French form gave 1 636 rows for a 208-row image that
// way, and every one of the surplus rows had plausible ink on it.
func (b *ccittBits) restIsFill() bool {
for i := b.pos; i < 8*len(b.data); i++ {
if b.data[i/8]&(1<<(7-uint(i%8))) != 0 {
return false
}
}
return true
}
// read decodes one code. It returns false at the end of the data or on a code
// the table does not name — the caller decides which of those is fatal, since a
// truncated fax is common and worth showing as far as it goes.
func (t *ccittTable) read(b *ccittBits) (int, bool) {
var code uint32
for n := 1; n < len(t.byLength); n++ {
bit, ok := b.next()
if !ok {
return 0, false
}
code = code<<1 | bit
if v, ok := t.byLength[n][code]; ok {
return v, true
}
}
return 0, false
}
// skipEOL consumes an end-of-line code, 000000000001, if one is next, and any
// fill bits before it. A Group 3 fax may have one at the end of every row, may
// have none at all, and may have several together at the end; all three are
// found in the corpus, so none of them may be an error.
func (b *ccittBits) skipEOL() bool {
start := b.pos
zeros := 0
for {
bit, ok := b.next()
if !ok {
b.pos = start
return false
}
if bit == 0 {
zeros++
continue
}
if zeros >= 11 {
return true
}
b.pos = start
return false
}
}
// ccittParams are the decode parameters of /CCITTFaxDecode, with the defaults
// the specification gives.
type ccittParams struct {
k int // <0 Group 4, 0 Group 3 one-dimensional, >0 Group 3 mixed
columns int // default 1728
rows int // 0 means as many as the data holds
blackIs1 bool // false: a 0 bit is black
encodedByteAlign bool
endOfBlock bool // default true: the data ends with a block marker
}
func ccittParamsOf(parm Dict, r Resolver) ccittParams {
return ccittParams{
k: intParm(parm, "K", 0, r),
columns: intParm(parm, "Columns", 1728, r),
rows: intParm(parm, "Rows", 0, r),
blackIs1: boolParm(parm, "BlackIs1", false, r),
encodedByteAlign: boolParm(parm, "EncodedByteAlign", false, r),
endOfBlock: boolParm(parm, "EndOfBlock", true, r),
}
}
// endOfBlockEOLs is how many end-of-line codes in a row mean the data has
// ended: two for Group 4, which has none between its rows, and six for Group 3,
// where every row may legitimately be followed by one.
func endOfBlockEOLs(k int) int {
if k < 0 {
return 2
}
return 6
}
// maxCCITTPixels bounds what one image may decode to, the way maxDecodedSize
// bounds a Flate stream. A fax names its own width and height in the decode
// parameters, so a file can ask for an image of any size it likes; 2^28 pixels
// is more than four times A4 at 600 dots to the inch.
var maxCCITTPixels = 1 << 28
// ccittDecode turns a fax into bilevel samples: one bit a pixel, each row
// padded to a byte boundary, which is what every other image filter in this
// package produces and what a stream dictionary's /Width, /Height and
// /BitsPerComponent describe.
func ccittDecode(data []byte, p ccittParams) ([]byte, error) {
if p.columns <= 0 {
return nil, fmt.Errorf("reader: /CCITTFaxDecode names %d columns", p.columns)
}
if p.rows < 0 {
return nil, fmt.Errorf("reader: /CCITTFaxDecode names %d rows", p.rows)
}
if p.rows > 0 && p.columns > maxCCITTPixels/p.rows {
return nil, fmt.Errorf("reader: /CCITTFaxDecode names %d by %d pixels, "+
"past the limit of %d", p.columns, p.rows, maxCCITTPixels)
}
b := &ccittBits{data: data}
stride := (p.columns + 7) / 8
curr := make([]byte, p.columns)
prev := []byte(nil)
out := []byte(nil)
for row := 0; p.rows == 0 || row < p.rows; row++ {
if p.rows == 0 && len(out)/stride*p.columns >= maxCCITTPixels {
return nil, fmt.Errorf("reader: /CCITTFaxDecode ran past %d pixels "+
"without naming how many rows it has", maxCCITTPixels)
}
if p.encodedByteAlign {
b.align()
}
// A Group 3 row may be introduced by an end-of-line code. Consuming it
// here rather than after the row means a file that has them and a file
// that does not are read the same way — and counting them is how the
// end of the data announces itself, since the block marker is nothing
// but end-of-line codes in a row.
eols := 0
for b.skipEOL() {
eols++
}
if p.endOfBlock && eols >= endOfBlockEOLs(p.k) {
break
}
if b.exhausted() || b.restIsFill() {
break
}
twoDimensional := p.k < 0
if p.k > 0 {
// In mixed mode a bit after the end-of-line says which the row is:
// 1 for one-dimensional, 0 for two.
// The data was checked for bits a moment ago, so this cannot
// fail; the value is taken rather than the error handled, because
// a branch that cannot be reached cannot be tested.
bit, _ := b.next()
twoDimensional = bit == 0
}
if err := ccittRow(b, curr, prev, twoDimensional); err != nil {
if len(out) == 0 {
return nil, err
}
// A truncated fax is worth showing as far as it got: the rows
// already decoded are real, and refusing them turns a damaged
// scan into a blank page. Leaving the loop rather than returning
// here matters — the padding below is what makes the answer as
// long as /Rows promised, and a fuzz case of two bytes found that
// returning early handed the caller one row where it had asked
// for four, with no error to say so.
break
}
out = append(out, ccittPack(curr, stride, p.blackIs1)...)
if prev == nil {
prev = make([]byte, p.columns)
}
copy(prev, curr)
}
// A file that says how many rows it has gets that many, padded with white
// if the data ran out, so the samples match the /Height the dictionary
// promises rather than being short by a row.
if p.rows > 0 {
white := make([]byte, stride)
if !p.blackIs1 {
for i := range white {
white[i] = 0xFF
}
}
for len(out) < p.rows*stride {
out = append(out, white...)
}
}
return out, nil
}
// ccittPack turns a row of one byte a pixel into one bit a pixel. With
// /BlackIs1 false — the default — a 0 bit is black, so white becomes a 1.
func ccittPack(row []byte, stride int, blackIs1 bool) []byte {
out := make([]byte, stride)
for x, v := range row {
white := v != 0
if white != blackIs1 {
out[x/8] |= 1 << (7 - uint(x%8))
}
}
return out
}
// ccittRow decodes one row into curr, one byte a pixel: 0xFF white, 0x00 black.
func ccittRow(b *ccittBits, curr, prev []byte, twoDimensional bool) error {
for i := range curr {
curr[i] = 0
}
at, white, first := 0, true, true
for at < len(curr) {
if !twoDimensional {
n, err := ccittRun(b, white)
if err != nil {
return err
}
if at+n > len(curr) {
return fmt.Errorf("reader: a fax row of %d pixels named %d",
len(curr), at+n)
}
fill(curr[at:at+n], white)
at += n
white = !white
first = false
continue
}
mode, ok := ccittModes.read(b)
if !ok {
return fmt.Errorf("reader: a fax row named no mode this reader knows")
}
switch mode {
case modePass:
// ccittFindB starts where the pen is and only walks forward,
// stopping at the end of the row, so b2 is always a pixel of this
// row at or after the pen. There is nothing to check.
b2 := ccittFindB(prev, curr, at, white, first, true)
fill(curr[at:b2], white)
at = b2
case modeH:
// Two runs, of the current colour and then the other, neither of
// them looking at the row above.
for i := 0; i < 2; i++ {
n, err := ccittRun(b, white)
if err != nil {
return err
}
if at+n > len(curr) {
return fmt.Errorf("reader: a fax row of %d pixels named %d",
len(curr), at+n)
}
fill(curr[at:at+n], white)
at += n
white = !white
}
// Horizontal mode reads a pair, so the two flips above have put
// the colour back where it started; there is nothing to undo.
case modeV0, modeVR1, modeVR2, modeVR3, modeVL1, modeVL2, modeVL3:
a1 := ccittFindB(prev, curr, at, white, first, false) + ccittOffset(mode)
if a1 < at || a1 > len(curr) {
return fmt.Errorf("reader: a fax vertical mode named pixel %d of %d",
a1, len(curr))
}
fill(curr[at:a1], white)
at = a1
white = !white
default:
return fmt.Errorf("reader: a fax named an extension mode, which " +
"this reader does not decode")
}
first = false
}
return nil
}
// ccittOffset is how far a vertical mode moves a1 from b1.
func ccittOffset(mode int) int {
switch mode {
case modeVR1:
return 1
case modeVR2:
return 2
case modeVR3:
return 3
case modeVL1:
return -1
case modeVL2:
return -2
case modeVL3:
return -3
}
return 0
}
// ccittRun reads one run length, which is a sequence of make-up codes ending in
// a terminating code below 64.
func ccittRun(b *ccittBits, white bool) (int, error) {
table := ccittBlack
if white {
table = ccittWhite
}
total := 0
for {
n, ok := table.read(b)
if !ok {
return 0, fmt.Errorf("reader: a fax run named no length this reader knows")
}
total += n
if total > maxCCITTPixels {
return 0, fmt.Errorf("reader: a fax named a run of %d pixels", total)
}
if n <= 63 {
return total, nil
}
}
}
// ccittFindB finds b1, or b2 when second is set: the changing elements on the
// row above, as T.6 Figure 1 defines them.
func ccittFindB(prev, curr []byte, at int, white, first, second bool) int {
// The row above the first row is implicitly all white, so it has no
// changing elements and both b values are at the end of the row.
if prev == nil {
return len(curr)
}
i := at
if first {
// a0 is implicitly one pixel before the row, on white. b1 is the first
// black pixel above; b2 the first white pixel after that.
for i < len(prev) && prev[i] != 0 {
i++
}
if second {
for i < len(prev) && prev[i] == 0 {
i++
}
}
return i
}
// Walk past the run above that is of the opposite colour to the pen, then
// past the run of the pen's own colour: what follows is b1.
opposite := byte(0xFF)
if white {
opposite = 0
}
for i < len(prev) && prev[i] == opposite {
i++
}
same := ^opposite
for i < len(prev) && prev[i] == same {
i++
}
if second {
for i < len(prev) && prev[i] == opposite {
i++
}
}
return i
}
// boolParm reads a boolean decode parameter, falling back to its default.
func boolParm(parm Dict, key Name, def bool, r Resolver) bool {
if parm == nil {
return def
}
v, err := Resolve(parm.Get(key), r)
if err != nil {
return def
}
if b, ok := v.(Bool); ok {
return bool(b)
}
return def
}
// fill paints a run of pixels, one byte each.
func fill(row []byte, white bool) {
v := byte(0)
if white {
v = 0xFF
}
for i := range row {
row[i] = v
}
}
vendor/github.com/go-pdfkit/reader/content.go
Generated Vendored
+339
View File
@@ -0,0 +1,339 @@
package reader
import (
"bytes"
"fmt"
)
// An Operation is one step of a content stream: an operator and the operands
// that precede it. An inline image is reported as the operator "BI" with its
// dictionary and data attached, since the three keywords it spans are one
// thing.
type Operation struct {
Operator string
Operands []Object
Image *InlineImage
}
// An InlineImage is the dictionary and the still-encoded bytes of a BI … ID …
// EI sequence. Its dictionary uses the abbreviated keys inline images are
// written with; [InlineImage.Expanded] gives the ordinary spelling.
type InlineImage struct {
Dict Dict
Raw []byte
}
// inlineKeys maps the abbreviations an inline image dictionary uses to the
// names their long form would have.
var inlineKeys = map[Name]Name{
"BPC": "BitsPerComponent",
"CS": "ColorSpace",
"D": "Decode",
"DP": "DecodeParms",
"F": "Filter",
"H": "Height",
"IM": "ImageMask",
"I": "Interpolate",
"W": "Width",
"L": "Length",
}
// inlineColourSpaces maps the abbreviated colour space names.
var inlineColourSpaces = map[Name]Name{
"G": "DeviceGray",
"RGB": "DeviceRGB",
"CMYK": "DeviceCMYK",
"I": "Indexed",
}
// Expanded returns the image's dictionary with the abbreviated keys and colour
// space names written out, so it can be read like any image XObject.
//
// A dictionary may say the same thing twice and disagree with itself: /W 20
// beside /Width 10, /CS /RGB beside /ColorSpace /3chanRGB. The specification
// permits both spellings in an inline image and says nothing about which wins,
// and the public implementations disagree — pdf.js asks for the abbreviation
// and falls back to the written-out name, MuPDF asks the other way round.
//
// The abbreviation wins here. Two reasons: it is the spelling Table 93 gives
// for an inline image, the written-out name being the tolerated alias; and
// safedocs' own fixture for this case marks the abbreviation as the line to
// remove "to see the effect", which is a statement about what the reference
// viewer does.
//
// What is not defensible is the answer changing between runs. Expanding in one
// pass over the map made the winner depend on Go's map iteration order, which
// is deliberately random: the same file drew differently each time it was
// opened, with no way for a caller to notice. Two passes settle it — the
// written-out names first, then the abbreviations over them — and no order
// within either pass can matter, because the names in each are distinct.
func (im *InlineImage) Expanded() Dict {
out := Dict{}
for k, v := range im.Dict {
if _, abbreviated := inlineKeys[k]; abbreviated {
continue
}
out[k] = expandColourSpace(k, v)
}
for k, v := range im.Dict {
long, abbreviated := inlineKeys[k]
if !abbreviated {
continue
}
out[long] = expandColourSpace(long, v)
}
return out
}
// expandColourSpace writes out an abbreviated colour space name, which is the
// one value that is abbreviated as well as its key.
func expandColourSpace(k Name, v Object) Object {
if k != "ColorSpace" {
return v
}
n, ok := ToName(v)
if !ok {
return v
}
if long, ok := inlineColourSpaces[n]; ok {
return long
}
return v
}
// A ContentScanner walks a content stream one operation at a time. A stream
// with rubbish in it yields the operations around the rubbish rather than
// nothing at all, which is what a renderer needs; [ContentScanner.Err] reports
// the first thing that did not parse.
type ContentScanner struct {
lex lexer
pending []Object
err error
}
// NewContentScanner reads operations from a decoded content stream.
func NewContentScanner(data []byte) *ContentScanner {
return &ContentScanner{lex: lexer{buf: data}}
}
// Err reports the first malformed token the scan stepped over, if any.
func (s *ContentScanner) Err() error { return s.err }
// Next returns the next operation, and false once the stream is exhausted.
func (s *ContentScanner) Next() (Operation, bool) {
for {
t, err := s.lex.next()
if err != nil {
s.note(err)
// Step over the byte that would not parse and carry on.
if s.lex.pos < len(s.lex.buf) {
s.lex.pos++
continue
}
return Operation{}, false
}
if t.kind == tokEOF {
return Operation{}, false
}
if t.kind != tokKeyword {
s.operand(t)
continue
}
switch string(t.text) {
case "true":
s.pending = append(s.pending, Bool(true))
continue
case "false":
s.pending = append(s.pending, Bool(false))
continue
case "null":
s.pending = append(s.pending, Null{})
continue
case "BI":
img, err := s.inlineImage()
if err != nil {
s.note(err)
return Operation{}, false
}
return s.emit("BI", img), true
}
return s.emit(string(t.text), nil), true
}
}
// operand parses one operand, an already-read token starting it.
func (s *ContentScanner) operand(t token) {
p := &parser{lex: s.lex}
o, err := p.parseFrom(t)
s.lex = p.lex
if err != nil {
s.note(err)
if s.lex.pos < len(s.lex.buf) {
s.lex.pos++
}
return
}
s.pending = append(s.pending, o)
}
// emit finishes an operation and clears the operands waiting for it.
func (s *ContentScanner) emit(op string, img *InlineImage) Operation {
out := Operation{Operator: op, Operands: s.pending, Image: img}
s.pending = nil
return out
}
// note keeps the first error the scan met.
func (s *ContentScanner) note(err error) {
if s.err == nil {
s.err = err
}
}
// inlineImage reads the dictionary after BI and the data after ID.
func (s *ContentScanner) inlineImage() (*InlineImage, error) {
d := Dict{}
for {
t, err := s.lex.next()
if err != nil {
return nil, err
}
if t.kind == tokEOF {
return nil, &SyntaxError{t.pos, "an inline image has no ID"}
}
if t.kind == tokKeyword && string(t.text) == "ID" {
break
}
if t.kind != tokName {
return nil, &SyntaxError{t.pos, "an inline image key is not a name"}
}
p := &parser{lex: s.lex}
v, err := p.parseObject()
s.lex = p.lex
if err != nil {
return nil, err
}
d[Name(t.text)] = v
}
// Exactly one white-space byte separates ID from the data.
if s.lex.pos < len(s.lex.buf) && isSpace(s.lex.buf[s.lex.pos]) {
s.lex.pos++
}
start := s.lex.pos
end, err := inlineImageEnd(s.lex.buf, start, d)
if err != nil {
return nil, err
}
raw := s.lex.buf[start:end]
s.lex.pos = end
// Step over the EI that follows.
for s.lex.pos < len(s.lex.buf) && isSpace(s.lex.buf[s.lex.pos]) {
s.lex.pos++
}
if bytes.HasPrefix(s.lex.buf[s.lex.pos:], []byte("EI")) {
s.lex.pos += 2
}
return &InlineImage{Dict: d, Raw: raw}, nil
}
// Operations tokenises a whole content stream. The error it returns describes
// what did not parse; the operations it returns are still worth having.
func Operations(data []byte) ([]Operation, error) {
s := NewContentScanner(data)
var out []Operation
for {
op, ok := s.Next()
if !ok {
return out, s.Err()
}
out = append(out, op)
}
}
// PageContent returns the decoded content stream of the i'th page, counting
// from one. A page whose /Contents is an array has its streams joined, as the
// specification requires, with a newline between them.
//
// A content stream whose filters cannot be applied contributes what could be
// salvaged from it rather than failing the page, which is what a renderer
// needs; the error is for a page that is not there at all.
// [Document.PageContentDecoded] says whether any salvaging happened.
func (d *Document) PageContent(i int) ([]byte, error) {
dec, err := d.PageContentDecoded(i)
return dec.Data, err
}
// PageContentDecoded is [Document.PageContent] with the outcome of the decode
// attached: [Decoded.Recovered] reports that at least one of the page's content
// streams could not be decoded cleanly, and [Decoded.Cause] says why.
func (d *Document) PageContentDecoded(i int) (Decoded, error) {
page, err := d.Page(i)
if err != nil {
return Decoded{}, err
}
return d.contentOf(page)
}
// contentOf decodes a page's /Contents, whether one stream or several.
func (d *Document) contentOf(page Dict) (Decoded, error) {
o, err := d.Resolve(page.Get("Contents"))
if err != nil {
return Decoded{}, err
}
switch v := o.(type) {
case *Stream:
return d.decodedContent(v), nil
case Array:
var out Decoded
for _, e := range v {
eo, err := d.Resolve(e)
if err != nil {
return Decoded{}, err
}
s, ok := ToStream(eo)
if !ok {
continue
}
part := d.decodedContent(s)
if part.Recovered && !out.Recovered {
out.Recovered, out.Cause, out.Filter = true, part.Cause, part.Filter
out.Undecoded = part.Undecoded
}
if len(part.Data) == 0 {
continue
}
if len(out.Data) > 0 {
out.Data = append(out.Data, '\n')
}
out.Data = append(out.Data, part.Data...)
}
return out, nil
}
return Decoded{}, nil
}
// decodedContent decodes one content stream. An image filter has no business
// being there, so the bytes it holds are not content: they are reported as
// salvage, not handed over as if they could be tokenised.
func (d *Document) decodedContent(s *Stream) Decoded {
dec := d.DecodeStreamRecovering(s)
if dec.Image != "" {
return Decoded{
Undecoded: dec.Data,
Recovered: true,
Filter: dec.Image,
Cause: fmt.Errorf("reader: a content stream is filtered as an image (/%s)", dec.Image),
}
}
return dec
}
// PageOperations tokenises the i'th page's content stream, counting from one.
func (d *Document) PageOperations(i int) ([]Operation, error) {
data, err := d.PageContent(i)
if err != nil {
return nil, err
}
ops, _ := Operations(data)
return ops, nil
}
vendor/github.com/go-pdfkit/reader/crypt.go
Generated Vendored
+572
View File
@@ -0,0 +1,572 @@
package reader
import (
"bytes"
"crypto/aes"
"crypto/cipher"
"crypto/md5"
"crypto/rc4"
"crypto/sha256"
"crypto/sha512"
"errors"
"fmt"
"hash"
)
// ErrWrongPassword is returned when a file is encrypted and neither the
// password given nor the empty one opens it.
var ErrWrongPassword = errors.New("reader: the password does not open this file")
// ErrUnsupportedEncryption is returned for a security handler this package
// does not implement — anything other than the standard one.
var ErrUnsupportedEncryption = errors.New("reader: unsupported security handler")
// pad is the 32-byte string every pre-2.0 password is padded with.
var pad = []byte{
0x28, 0xBF, 0x4E, 0x5E, 0x4E, 0x75, 0x8A, 0x41, 0x64, 0x00, 0x4E, 0x56,
0xFF, 0xFA, 0x01, 0x08, 0x2E, 0x2E, 0x00, 0xB6, 0xD0, 0x68, 0x3E, 0x80,
0x2F, 0x0C, 0xA9, 0xFE, 0x64, 0x53, 0x69, 0x7A,
}
// cryptMethod is how one class of data — strings or streams — is protected.
type cryptMethod uint8
const (
cryptNone cryptMethod = iota // /Identity: not encrypted at all
cryptRC4
cryptAESV2 // 128-bit AES, per-object key
cryptAESV3 // 256-bit AES, the file key used directly
)
// A decryptor holds the file encryption key and how to apply it.
type decryptor struct {
key []byte
strings cryptMethod
streams cryptMethod
revision int
perm Permissions
owner bool // the password given was the owner's, not the user's
skipObj int // the /Encrypt dictionary's own object number, never decrypted
skipKnown bool // whether skipObj is meaningful
}
// A Protection is what a file's security handler says about it: how it is
// encrypted, and what a reader that opened it with the user password may do.
type Protection struct {
// Method names the algorithm the way a person would say it: "RC4-40",
// "RC4-128", "AES-128", "AES-256", or "none" for a file that declares an
// /Encrypt dictionary and then encrypts nothing with it.
Method string
// Revision is the standard security handler's revision: 2, 3, 4, 5 or 6.
Revision int
// Permissions is what the file grants whoever opened it with the user
// password.
Permissions Permissions
// Owner is true when the password the file was opened with was the
// owner's, in which case the permissions do not apply to this reader.
Owner bool
}
// Protection reports how the file is protected, and false when it is not
// protected at all. A document that opened has already been decrypted; this
// says what it said about itself on the way.
func (d *Document) Protection() (Protection, bool) {
if d.decrypt == nil {
return Protection{}, false
}
return Protection{
Method: d.decrypt.methodName(),
Revision: d.decrypt.revision,
Permissions: d.decrypt.perm,
Owner: d.decrypt.owner,
}, true
}
// methodName says which algorithm protects the streams, which is the one that
// matters: it is where the content is.
func (dec *decryptor) methodName() string {
switch dec.streams {
case cryptAESV3:
return "AES-256"
case cryptAESV2:
return "AES-128"
case cryptRC4:
return fmt.Sprintf("RC4-%d", len(dec.key)*8)
}
return "none"
}
// Encrypted reports whether the file declares an /Encrypt dictionary.
func (d *Document) Encrypted() bool {
return d.trailer != nil && d.trailer.Get("Encrypt").Kind() != KindNull
}
// setUpDecryption reads /Encrypt and derives the file key. It is called once,
// before any string or stream is handed out.
func (d *Document) setUpDecryption(password string) error {
if !d.Encrypted() {
return nil
}
if ref, ok := d.trailer.Get("Encrypt").(Ref); ok {
d.encryptNum, d.encryptKnown = ref.Num, true
}
o, err := d.Resolve(d.trailer.Get("Encrypt"))
if err != nil {
return err
}
enc, ok := ToDict(o)
if !ok {
return fmt.Errorf("reader: /Encrypt is a %s, not a dictionary", o.Kind())
}
id := d.firstID()
dec, err := newDecryptor(enc, id, password, d.Get)
if err != nil {
return err
}
dec.skipObj, dec.skipKnown = d.encryptNum, d.encryptKnown
d.decrypt = dec
return nil
}
// firstID returns the first element of the trailer's /ID, which takes part in
// the pre-2.0 key derivation. A file without one contributes nothing.
func (d *Document) firstID() []byte {
arr, ok := ToArray(d.trailer.Get("ID"))
if !ok || len(arr) == 0 {
return nil
}
s, ok := ToString(arr[0])
if !ok {
return nil
}
return s
}
// newDecryptor derives the file encryption key from a password.
func newDecryptor(enc Dict, id []byte, password string, r Resolver) (*decryptor, error) {
if f, ok := ToName(resolved(enc, "Filter", r)); ok && f != "Standard" {
return nil, ErrUnsupportedEncryption
}
v := int(intOf(resolved(enc, "V", r), 0))
rev := int(intOf(resolved(enc, "R", r), 0))
if v == 0 || rev == 0 {
return nil, fmt.Errorf("reader: /Encrypt has no usable /V and /R")
}
length := int(intOf(resolved(enc, "Length", r), 40))
if length < 40 || length > 256 || length%8 != 0 {
length = 40
}
dec := &decryptor{revision: rev}
if err := dec.readMethods(enc, v, r); err != nil {
return nil, err
}
owner, _ := ToString(resolved(enc, "O", r))
user, _ := ToString(resolved(enc, "U", r))
perm := int32(intOf(resolved(enc, "P", r), 0))
metadata := true
if b, ok := ToBool(resolved(enc, "EncryptMetadata", r)); ok {
metadata = b
}
dec.perm = Permissions(uint32(perm)) & AllPermissions
if dec.streams == cryptNone && dec.strings == cryptNone {
// /StmF and /StrF both name the identity crypt filter, so nothing in
// the document body is encrypted and no key is needed to read it. A
// file like that is opened without validating the password: refusing
// it would refuse a document whose every byte is already plain, which
// is what every other reader shows. Only /EFF — the embedded files —
// would need the key, and this reader does not hand those out.
//
// Nothing was authenticated, so dec.owner stays false and Protection
// reports the method as "none": a caller can see exactly what it got.
return dec, nil
}
if rev >= 5 {
key, asOwner, err := deriveKeyR5(enc, password, r)
if err != nil {
return nil, err
}
dec.key, dec.owner = key, asOwner
return dec, nil
}
n := length / 8
if rev == 2 {
n = 5
}
key, asOwner, err := deriveKeyLegacy(password, owner, user, id, perm, n, rev, metadata)
if err != nil {
return nil, err
}
dec.key, dec.owner = key, asOwner
return dec, nil
}
// readMethods works out how strings and streams are protected. Before /V 4
// there is one method for everything; from /V 4 the crypt filters in /CF are
// named by /StmF and /StrF.
func (dec *decryptor) readMethods(enc Dict, v int, r Resolver) error {
if v < 4 {
dec.strings, dec.streams = cryptRC4, cryptRC4
return nil
}
cf, _ := ToDict(resolved(enc, "CF", r))
pick := func(key Name) (cryptMethod, error) {
name, ok := ToName(resolved(enc, key, r))
if !ok || name == "Identity" {
return cryptNone, nil
}
f, ok := ToDict(resolved(cf, name, r))
if !ok {
return cryptNone, nil
}
switch m, _ := ToName(resolved(f, "CFM", r)); m {
case "V2":
return cryptRC4, nil
case "AESV2":
return cryptAESV2, nil
case "AESV3":
return cryptAESV3, nil
case "None":
return cryptNone, nil
default:
return cryptNone, fmt.Errorf("reader: unsupported crypt filter method /%s", m)
}
}
var err error
if dec.streams, err = pick("StmF"); err != nil {
return err
}
dec.strings, err = pick("StrF")
return err
}
// deriveKeyLegacy is the pre-2.0 key derivation, trying the password as the
// user password and then as the owner password.
func deriveKeyLegacy(password string, owner, user, id []byte, perm int32, n, rev int, metadata bool) (key []byte, asOwner bool, err error) {
for _, candidate := range legacyCandidates(password, owner, n, rev) {
k := legacyFileKey(candidate.padded, owner, id, perm, n, rev, metadata)
if legacyUserKeyMatches(k, user, id, rev) {
return k, candidate.owner, nil
}
}
return nil, false, ErrWrongPassword
}
// A legacyCandidate is one padded password to try, and whether reaching it
// meant knowing the owner's rather than the user's.
type legacyCandidate struct {
padded []byte
owner bool
}
// legacyCandidates lists the padded passwords worth trying: the one given, the
// empty one, and the user password the owner password unlocks.
func legacyCandidates(password string, owner []byte, n, rev int) []legacyCandidate {
out := []legacyCandidate{{padded: padPassword([]byte(password))}}
if password != "" {
out = append(out, legacyCandidate{padded: padPassword(nil)})
}
if u := userFromOwner([]byte(password), owner, n, rev); u != nil {
out = append(out, legacyCandidate{padded: u, owner: true})
}
return out
}
// padPassword truncates or pads a password to the 32 bytes the algorithm wants.
func padPassword(p []byte) []byte {
out := make([]byte, 32)
k := copy(out, p)
copy(out[k:], pad)
return out
}
// legacyFileKey is algorithm 2: the file encryption key from a padded password.
func legacyFileKey(padded, owner, id []byte, perm int32, n, rev int, metadata bool) []byte {
h := md5.New()
h.Write(padded)
h.Write(owner)
h.Write([]byte{byte(perm), byte(perm >> 8), byte(perm >> 16), byte(perm >> 24)})
h.Write(id)
if rev >= 4 && !metadata {
h.Write([]byte{0xFF, 0xFF, 0xFF, 0xFF})
}
key := h.Sum(nil)
if rev >= 3 {
for i := 0; i < 50; i++ {
sum := md5.Sum(key[:n])
key = sum[:]
}
}
return key[:n]
}
// legacyUserKeyMatches is algorithm 6: does this key reproduce /U?
func legacyUserKeyMatches(key, user, id []byte, rev int) bool {
if rev == 2 {
return bytes.Equal(rc4Bytes(key, pad), user)
}
h := md5.New()
h.Write(pad)
h.Write(id)
x := rc4Bytes(key, h.Sum(nil))
for i := 1; i <= 19; i++ {
x = rc4Bytes(xorKey(key, i), x)
}
return len(user) >= 16 && bytes.Equal(x, user[:16])
}
// userFromOwner is algorithm 7: recover the padded user password from /O using
// the owner password.
func userFromOwner(password, owner []byte, n, rev int) []byte {
if len(owner) < 32 {
return nil
}
sum := md5.Sum(padPassword(password))
key := sum[:]
if rev >= 3 {
for i := 0; i < 50; i++ {
s := md5.Sum(key)
key = s[:]
}
}
key = key[:n]
if rev == 2 {
return rc4Bytes(key, owner[:32])
}
x := owner[:32]
for i := 19; i >= 0; i-- {
x = rc4Bytes(xorKey(key, i), x)
}
return x
}
// xorKey returns key with every byte exclusive-ored with v, which is how both
// password algorithms walk their nineteen rounds.
func xorKey(key []byte, v int) []byte {
out := make([]byte, len(key))
for i := range key {
out[i] = key[i] ^ byte(v)
}
return out
}
// deriveKeyR5 is the PDF 2.0 derivation, /R 5 and /R 6: the password is
// validated against a salted hash and then unwraps the file key.
func deriveKeyR5(enc Dict, password string, r Resolver) (key []byte, asOwner bool, err error) {
user, _ := ToString(resolved(enc, "U", r))
userE, _ := ToString(resolved(enc, "UE", r))
owner, _ := ToString(resolved(enc, "O", r))
ownerE, _ := ToString(resolved(enc, "OE", r))
rev := int(intOf(resolved(enc, "R", r), 6))
if len(user) < 48 {
return nil, false, fmt.Errorf("reader: /U is %d bytes, not the 48 this revision needs", len(user))
}
pw := []byte(password)
for _, candidate := range [][]byte{pw, nil} {
if candidate == nil && password == "" {
break
}
if key := unlockR5(candidate, user, userE, nil, rev); key != nil {
return key, false, nil
}
if len(owner) >= 48 {
if key := unlockR5(candidate, owner, ownerE, user[:48], rev); key != nil {
return key, true, nil
}
}
}
return nil, false, ErrWrongPassword
}
// unlockR5 checks one password against a 48-byte /U or /O entry and, when it
// matches, unwraps the file key from the matching /UE or /OE.
func unlockR5(password, entry, wrapped, udata []byte, rev int) []byte {
valSalt, keySalt := entry[32:40], entry[40:48]
if !bytes.Equal(hash2B(password, valSalt, udata, rev), entry[:32]) {
return nil
}
if len(wrapped) < 32 {
return nil
}
inter := hash2B(password, keySalt, udata, rev)
// inter is always the 32 bytes hash2B returns, so the cipher cannot refuse it.
block, _ := aes.NewCipher(inter)
out := make([]byte, 32)
cipher.NewCBCDecrypter(block, make([]byte, aes.BlockSize)).CryptBlocks(out, wrapped[:32])
return out
}
// hash2B is algorithm 2.B: SHA-256 for revision 5, and for revision 6 the
// hardening loop that repeatedly encrypts and re-hashes.
func hash2B(password, salt, udata []byte, rev int) []byte {
h := sha256.New()
h.Write(password)
h.Write(salt)
h.Write(udata)
k := h.Sum(nil)
if rev < 6 {
return k
}
for round := 0; ; round++ {
var k1 []byte
one := append(append(append([]byte{}, password...), k...), udata...)
for i := 0; i < 64; i++ {
k1 = append(k1, one...)
}
// k is always 32 bytes, so a 16-byte key is always well formed.
block, _ := aes.NewCipher(k[:16])
e := make([]byte, len(k1)-len(k1)%aes.BlockSize)
cipher.NewCBCEncrypter(block, k[16:32]).CryptBlocks(e, k1[:len(e)])
sum := 0
for _, b := range e[:16] {
sum += int(b)
}
var next hash.Hash
switch sum % 3 {
case 0:
next = sha256.New()
case 1:
next = sha512.New384()
default:
next = sha512.New()
}
next.Write(e)
k = next.Sum(nil)
if round >= 63 && int(e[len(e)-1]) <= round-31 {
break
}
}
return k[:32]
}
// objectKey is the per-object key the pre-2.0 methods use; AESV3 uses the file
// key unchanged.
func (dec *decryptor) objectKey(num, gen int, method cryptMethod) []byte {
if method == cryptAESV3 {
return dec.key
}
h := md5.New()
h.Write(dec.key)
h.Write([]byte{byte(num), byte(num >> 8), byte(num >> 16), byte(gen), byte(gen >> 8)})
if method == cryptAESV2 {
h.Write([]byte{0x73, 0x41, 0x6C, 0x54}) // "sAlT"
}
k := h.Sum(nil)
if n := len(dec.key) + 5; n < 16 {
return k[:n]
}
return k
}
// decryptBytes undoes one method over one object's data.
func (dec *decryptor) decryptBytes(num, gen int, method cryptMethod, data []byte) []byte {
switch method {
case cryptNone:
return data
case cryptRC4:
return rc4Bytes(dec.objectKey(num, gen, method), data)
default:
return aesDecrypt(dec.objectKey(num, gen, method), data)
}
}
// rc4Bytes applies RC4, which is its own inverse. A key length the cipher
// refuses yields nil, which every caller reads as a password that does not
// fit — the derivations here never produce one.
func rc4Bytes(key, data []byte) []byte {
c, err := rc4.NewCipher(key)
if err != nil {
return nil
}
out := make([]byte, len(data))
c.XORKeyStream(out, data)
return out
}
// aesDecrypt undoes AES in CBC mode, the initialisation vector being the first
// block of the data and the padding the one CBC calls for.
func aesDecrypt(key, data []byte) []byte {
if len(data) < 2*aes.BlockSize || len(data)%aes.BlockSize != 0 {
return nil
}
block, err := aes.NewCipher(key)
if err != nil {
return nil
}
out := make([]byte, len(data)-aes.BlockSize)
cipher.NewCBCDecrypter(block, data[:aes.BlockSize]).CryptBlocks(out, data[aes.BlockSize:])
n := int(out[len(out)-1])
if n < 1 || n > aes.BlockSize || n > len(out) {
return out
}
return out[:len(out)-n]
}
// decryptObject walks an object, decrypting every string in it and, for a
// stream, its data. The /Encrypt dictionary itself, and any cross-reference
// stream, are left alone: neither is encrypted.
func (dec *decryptor) decryptObject(num, gen int, o Object) Object {
if dec.skipKnown && num == dec.skipObj {
return o
}
return dec.walk(num, gen, o)
}
// walk is decryptObject's recursion.
func (dec *decryptor) walk(num, gen int, o Object) Object {
switch v := o.(type) {
case String:
return String(dec.decryptBytes(num, gen, dec.strings, v))
case Array:
for i, e := range v {
v[i] = dec.walk(num, gen, e)
}
return v
case Dict:
for k, e := range v {
v[k] = dec.walk(num, gen, e)
}
return v
case *Stream:
if t, ok := ToName(v.Dict.Get("Type")); ok && t == "XRef" {
return v
}
v.Dict, _ = ToDict(dec.walk(num, gen, v.Dict))
if streamIsPlain(v.Dict) {
return v
}
v.Raw = dec.decryptBytes(num, gen, dec.streams, v.Raw)
return v
}
return o
}
// streamIsPlain reports whether a stream's own filter chain says its bytes were
// left unencrypted: a leading /Crypt filter naming /Identity, which is what
// /Crypt with no /Name means too. Producers use it for the metadata stream of a
// file whose /EncryptMetadata is false, and decrypting such a stream turns
// readable XML into noise.
func streamIsPlain(d Dict) bool {
if first, ok := firstFilter(d); !ok || first != "Crypt" {
return false
}
name, ok := ToName(firstDecodeParms(d).Get("Name"))
return !ok || name == "Identity"
}
// resolved is a helper for reading an /Encrypt entry that may be indirect.
func resolved(d Dict, key Name, r Resolver) Object {
o, err := Resolve(d.Get(key), r)
if err != nil {
return Null{}
}
return o
}
// intOf reads an integer with a default.
func intOf(o Object, def int64) int64 {
if n, ok := ToInt(o); ok {
return n
}
return def
}

Some files were not shown because too many files have changed in this diff Show More