kyle/sift
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

P5a plan: PDF documents — seven Ornith tasks with acceptance tests first

Browse Source 
docs/plans/p5a: 01 kind+fetch limits, 02 pdftext child + runner, 03 dcache PDF tables,
04 pdfpool, 05 pdfdoc render (map/pages/grep), 06 api, 07 digest citations, metrics, CLI,
skill, unit, README. Given tests compile against a panic-only skeleton and fail there for the
intended reasons; PDFs for tests are built by internal/pdftest (given), plus two qpdf-encrypted
fixtures.

Owner changes for the run: vendor go-pdfkit/extract v0.3.0 + reader v0.6.0 (pinned via
tools.go until task 02); AGENTS.md, tools/run-plan.sh (enforces a Co-Authored-By trailer for
the model), docs/implementer-log.md, `make gate`; gofmt (whitespace) on clean.go and guard;
TestUnsupportedType uses a zip now that PDFs are supported; requestDeadline 90→210 s and
writeTimeout 120→240 s for the PDF path's worst case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
kyle
2026-09-25 20:47:38 -07:00
co-authored by Claude Opus 5.5
parent 3f2b78d4aa
commit cfde35397a
115 changed files with 26422 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files
vendor/github.com/go-pdfkit/extract/.gitignore
Generated
+1
View File
@@ -0,0 +1 @@
coverage.out
vendor/github.com/go-pdfkit/extract/LICENSE
Generated Vendored
+29
View File
@@ -0,0 +1,29 @@
BSD 3-Clause License
Copyright (c) 2026 the go-pdfkit/render authors
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
vendor/github.com/go-pdfkit/extract/README.md
Generated Vendored
+91
View File
@@ -0,0 +1,91 @@
# extract
[![CI](https://github.com/go-pdfkit/extract/actions/workflows/ci.yml/badge.svg)](https://github.com/go-pdfkit/extract/actions/workflows/ci.yml)
[![Go Reference](https://pkg.go.dev/badge/github.com/go-pdfkit/extract.svg)](https://pkg.go.dev/github.com/go-pdfkit/extract)
[![License: BSD-3-Clause](https://img.shields.io/badge/License-BSD--3--Clause-blue.svg)](LICENSE)
[![Coverage](https://img.shields.io/badge/coverage-100%25-brightgreen.svg)](#how-it-is-checked)
Reads a PDF page back: the text on it, with where each piece of it sits, and
the images it places.
```go
text, err := extract.Text(doc, 1) // the page as words, lines put back together
runs, err := extract.Runs(doc, 1) // every piece, with its place and its size
imgs, err := extract.Images(doc, 1) // every picture, and where it lands
```
Pure Go, no C. It builds for `GOOS=js/wasm` and every architecture the fleet
targets.
## What a page actually holds
Not text. A page holds instructions for drawing glyphs, and the text is what
those glyphs were meant to say — which the document says only if it was asked
to. Where it does not, this says so rather than guessing: a run whose codes
could not be worked out comes back marked `Unreadable`, so a caller can tell
*this page says nothing* from *this page could not be read*.
Three things are tried, in order:
1. the font's `/ToUnicode` map, which is the document's own word on what its
text says;
2. the name its encoding gives the code — but only a name the document chose
itself, since a **symbolic** font read through an assumed encoding gives
the wrong letter with nothing to say so;
3. what the **embedded font program** calls the code, read with
[`go-opentype`](https://github.com/go-opentype/opentype) — which is the
honest answer for a symbolic font, and the reason a page of mathematics
comes back as mathematics.
Text drawn in the mode that puts no ink on the page — how a scanner writes
what it read underneath the picture it read it from — is returned and marked
`Invisible`.
## Images
A picture comes back as the file holds it, with the box it covers on the page.
A JPEG comes out as a JPEG: re-encoding it would lose something for nothing.
Turning the other kinds into pixels means reading their colour space, which is
[`render`](https://github.com/go-pdfkit/render)'s work rather than this
package's.
## How it is checked
100% of statements, including every branch that handles a page saying
something it should not.
Then on the corpus — 118 833 real PDFs, 121 946 pages:
```
pages read 121946
with text 94865 (77.8%)
saying nothing 27081, of which 4812 did draw glyphs
characters 33976414 (17272226 of them letters)
runs 10797045
unreadable 203607 (1.89%)
invisible 29917
images 300685
panics 0
```
The 22% of pages that say nothing are figures — this corpus is arXiv's, and
most of it is plots. Of the pages that *did* draw glyphs and still said
nothing, nearly all are Type 3 fonts with no `/ToUnicode`: dvips bitmap fonts
whose glyphs are called `a1` and `s32`, which no reader can turn back into
letters.
## The rest of the family
[**reader**](https://github.com/go-pdfkit/reader) reads and writes the format ·
[**ops**](https://github.com/go-pdfkit/ops) is the verbs and the `pdfops` command ·
[**render**](https://github.com/go-pdfkit/render) turns a page into pixels ·
[**pdffont**](https://github.com/go-pdfkit/pdffont) is what a document says about a font ·
[**extract**](https://github.com/go-pdfkit/extract) reads a page back ·
[**coedit**](https://github.com/go-pdfkit/coedit) is a PDF several people edit at once ·
[**app**](https://github.com/go-pdfkit/app) is the browser workbench.
Documentation for all of it: <https://go-pdfkit.github.io/docs/>
## Licence
BSD-3-Clause.
vendor/github.com/go-pdfkit/extract/extract.go
Generated Vendored
+176
View File
@@ -0,0 +1,176 @@
// Package extract reads a PDF page back: the text on it, with where each piece
// of it sits, and the images it places.
//
// A page does not hold text. It holds instructions for drawing glyphs, and the
// text is what those glyphs were meant to say — which the document says only
// if it was asked to. Where it does not, this says so rather than guessing:
// a run whose characters could not be worked out comes back marked, so that a
// caller can tell "this page says nothing" from "this page could not be read".
package extract
import (
"sort"
"strings"
"github.com/go-pdfkit/pdffont"
"github.com/go-pdfkit/reader"
)
// A Run is a piece of text drawn in one go: one font, one size, one place.
type Run struct {
// Text is what the run says.
Text string
// X and Y are where it starts, in the page's own coordinates — up from
// the bottom left corner, in points.
X, Y float64
// Width is how far the pen moved drawing it, and Size how tall the text
// is, both in points.
Width, Size float64
// Font is the name the page's resources give the font.
Font string
// Space is how wide a space would be here, which is what says whether a
// gap between two runs is a word break or only kerning.
Space float64
// Invisible reports text drawn in the mode that puts no ink on the page.
// It is how a scanner puts what it read underneath the picture it read it
// from, so it is text worth having — and worth knowing about.
Invisible bool
// Unreadable reports a run some of whose codes said nothing about which
// characters they were: the font carried no map and named its glyphs
// after numbers. What Text holds is then only the part that could be read.
Unreadable bool
}
// Text is everything the page says, in reading order, with the lines put back
// together.
func Text(d *reader.Document, page int) (string, error) {
runs, err := Runs(d, page)
if err != nil {
return "", err
}
return Assemble(runs), nil
}
// Runs is every piece of text on the page, in the order it was drawn.
func Runs(d *reader.Document, page int) ([]Run, error) {
e, err := walk(d, page, false)
if err != nil {
return nil, err
}
return e.runs, nil
}
// walk reads a page's content stream once, keeping what it finds.
//
// A page whose content decoded only part of the way is read as far as it went:
// 263 streams in 212 of the 1 633 real forms in the corpus cannot be decoded
// cleanly, and half a page of text is worth more to somebody searching than
// none of it. A page that decoded no bytes at all is a different answer and
// stays an error — a conversion that silently produces an empty document from
// an unreadable page is worse than one that says it could not read it, and
// go-pdfkit/latex relies on being told.
//
// wantPictures says whether the caller is going to look at the pictures. A
// page's images have nothing to do with its text, and undoing their filters is
// the most expensive thing on the page: one arXiv figure holds 378 MB of image
// once decompressed, so asking that page what it *said* used to cost 1 094 MB
// and 1.3 seconds spent inflating pictures the answer throws away. Where they
// land is still noted either way, since that costs a matrix multiply; only the
// bytes are left alone.
func walk(d *reader.Document, page int, wantPictures bool) (*extractor, error) {
dec, err := d.PageContentDecoded(page)
if err != nil {
return nil, err
}
if len(dec.Data) == 0 && dec.Cause != nil {
return nil, dec.Cause
}
content := dec.Data
p, _ := d.Page(page)
resources, _ := d.GetDict(p, "Resources")
e := &extractor{doc: d, fonts: map[int]*pdffont.Font{}, wantPictures: wantPictures}
e.run(content, resources, initialState(d, p), 0)
return e, nil
}
// Assemble puts runs back together as lines of text: everything on one
// baseline in the order it sits across the page, with a space wherever the
// gap between two pieces is wide enough to be one.
func Assemble(runs []Run) string {
if len(runs) == 0 {
return ""
}
lines := groupIntoLines(runs)
var out strings.Builder
for i, line := range lines {
if i > 0 {
out.WriteByte('\n')
}
out.WriteString(joinLine(line))
}
return out.String()
}
// groupIntoLines gathers runs that sit on the same baseline, in the order they
// appear down the page.
func groupIntoLines(runs []Run) [][]Run {
sorted := append([]Run{}, runs...)
sort.SliceStable(sorted, func(i, j int) bool {
if !sameLine(sorted[i], sorted[j]) {
return sorted[i].Y > sorted[j].Y
}
return sorted[i].X < sorted[j].X
})
var lines [][]Run
for _, r := range sorted {
if n := len(lines); n > 0 && sameLine(lines[n-1][0], r) {
lines[n-1] = append(lines[n-1], r)
continue
}
lines = append(lines, []Run{r})
}
return lines
}
// sameLine reports whether two runs sit on the same baseline. A line is
// allowed to wobble by a fraction of its own height, which is what a
// superscript or a slightly different font does.
func sameLine(a, b Run) bool {
tol := a.Size * 0.3
if tol <= 0 {
tol = 1
}
d := a.Y - b.Y
return d < tol && d > -tol
}
// joinLine writes one line out, putting a space wherever the gap between two
// pieces is wide enough to be one.
func joinLine(line []Run) string {
var out strings.Builder
for i, r := range line {
if i > 0 {
prev := line[i-1]
gap := r.X - (prev.X + prev.Width)
if wantsSpace(prev, r, gap) {
out.WriteByte(' ')
}
}
out.WriteString(r.Text)
}
return strings.TrimRight(out.String(), " ")
}
// wantsSpace decides whether a gap between two pieces of a line is a word
// break. A space of its own is never doubled, and a gap of about a third of a
// space is where kerning ends and words begin.
func wantsSpace(prev, next Run, gap float64) bool {
if strings.HasSuffix(prev.Text, " ") || strings.HasPrefix(next.Text, " ") {
return false
}
space := prev.Space
if space <= 0 {
space = prev.Size * 0.25
}
return gap > space*0.3
}
vendor/github.com/go-pdfkit/extract/image.go
Generated Vendored
+188
View File
@@ -0,0 +1,188 @@
package extract
import (
"github.com/go-pdfkit/reader"
)
// An Image is a picture the page places, and where it puts it.
//
// What comes back is the picture as the file holds it — the bytes and the
// dictionary describing them — rather than pixels. A JPEG comes out as a JPEG,
// which is what anyone pulling pictures out of a document wants: re-encoding
// it would lose something for nothing. Turning the other kinds into pixels
// means reading their colour space, which is the renderer's work rather than
// this package's.
type Image struct {
// Name is what the page's resources call it, or empty for one written
// into the content stream.
Name string
// Width and Height are the picture's own, in samples.
Width, Height int
// X and Y are where its bottom left corner lands on the page, and
// DrawnWidth and DrawnHeight how large it is drawn, in points. A picture
// turned or skewed by the page still reports the box it covers.
X, Y float64
DrawnWidth, DrawnHeight float64
// Filter is the last filter still on the data: DCTDecode for a JPEG that
// comes out as one, JPXDecode for a JPEG 2000, and empty for data this
// has unfiltered into plain samples.
Filter reader.Name
// Data is the picture's bytes, with every filter this understands undone.
Data []byte
// Dict is the image's own dictionary, which says how to read the data:
// its colour space, how many bits a sample takes, and whether it is a
// mask.
Dict reader.Dict
// Inline reports a picture written into the content stream rather than
// held as a resource of its own.
Inline bool
}
// Images is every picture the page places, in the order it places them. The
// same picture placed twice comes back twice, since where it lands is part of
// what is being asked for.
func Images(d *reader.Document, page int) ([]Image, error) {
e, err := walk(d, page, true)
if err != nil {
return nil, err
}
return e.images, nil
}
// doXObject handles the Do operator: a form is walked into, and a picture is
// noted where it lands.
func (e *extractor) doXObject(g *state, operands []reader.Object, resources reader.Dict, depth int) {
if len(operands) == 0 {
return
}
name, ok := reader.ToName(operands[0])
if !ok {
return
}
xobjects, ok := e.doc.GetDict(resources, "XObject")
if !ok {
return
}
stream, ok := reader.ToStream(resolve(e.doc, xobjects.Get(name)))
if !ok {
return
}
switch sub, _ := reader.ToName(resolve(e.doc, stream.Dict.Get("Subtype"))); sub {
case "Form":
e.doForm(g, stream, resources, depth)
case "Image":
e.noteImage(g, string(name), stream.Dict, stream.Raw, false)
}
}
// doForm walks into a form, which is a piece of a page held apart and drawn
// where the page says.
func (e *extractor) doForm(g *state, stream *reader.Stream, resources reader.Dict, depth int) {
data, img, err := e.doc.DecodeStream(stream)
if err != nil || img != "" {
return
}
inner := *g
if m := floatArray(e.doc, stream.Dict.Get("Matrix")); len(m) >= 6 {
inner.ctm = matrix{m[0], m[1], m[2], m[3], m[4], m[5]}.mul(g.ctm)
}
own, ok := e.doc.GetDict(stream.Dict, "Resources")
if !ok {
own = resources
}
e.run(data, own, inner, depth+1)
}
// inlineImage notes a picture written into the content stream.
func (e *extractor) inlineImage(g *state, im *reader.InlineImage) {
if im == nil {
return
}
e.noteImage(g, "", im.Expanded(), im.Raw, true)
}
// noteImage works out where a picture lands and records it. A picture occupies
// the unit square of the transform in force, so its corners are where that
// square's are.
func (e *extractor) noteImage(g *state, name string, dict reader.Dict, raw []byte, inline bool) {
minX, minY, maxX, maxY := unitSquareBox(g.ctm)
img := Image{
Name: name,
Width: int(intOf(resolve(e.doc, dict.Get("Width")), 0)),
Height: int(intOf(resolve(e.doc, dict.Get("Height")), 0)),
X: minX,
Y: minY,
DrawnWidth: maxX - minX,
DrawnHeight: maxY - minY,
Dict: dict,
Inline: inline,
}
// A caller after the text is not after the pixels, and undoing a
// picture's filters to hand it nothing is the most expensive thing a page
// can be made to do.
if e.wantPictures {
data, filter, err := reader.Decode(dict, raw, e.doc.Resolver())
if err == nil {
img.Data, img.Filter = data, filter
} else {
img.Data = raw
}
}
e.images = append(e.images, img)
}
// unitSquareBox is the box the unit square covers once the transform has had
// its way with it.
func unitSquareBox(m matrix) (minX, minY, maxX, maxY float64) {
first := true
for _, c := range [][2]float64{{0, 0}, {1, 0}, {1, 1}, {0, 1}} {
x, y := m.apply(c[0], c[1])
if first {
minX, minY, maxX, maxY = x, y, x, y
first = false
continue
}
minX, maxX = smaller(minX, x), larger(maxX, x)
minY, maxY = smaller(minY, y), larger(maxY, y)
}
return minX, minY, maxX, maxY
}
func smaller(a, b float64) float64 {
if a < b {
return a
}
return b
}
func larger(a, b float64) float64 {
if a > b {
return a
}
return b
}
// intOf reads a whole number, or gives a default.
func intOf(o reader.Object, def int64) int64 {
if v, ok := reader.ToInt(o); ok {
return v
}
return def
}
// floatArray reads an array of numbers, or nothing.
func floatArray(d *reader.Document, o reader.Object) []float64 {
arr, ok := reader.ToArray(resolve(d, o))
if !ok {
return nil
}
out := make([]float64, 0, len(arr))
for _, e := range arr {
v, ok := reader.ToFloat(resolve(d, e))
if !ok {
return nil
}
out = append(out, v)
}
return out
}
vendor/github.com/go-pdfkit/extract/program.go
Generated Vendored
+175
View File
@@ -0,0 +1,175 @@
package extract
import (
"github.com/go-opentype/opentype"
"github.com/go-pdfkit/pdffont"
"github.com/go-pdfkit/reader"
)
// attachProgram gives a font a last resort for the codes the document says
// nothing about: what its own embedded program calls them.
//
// A font that says it is symbolic is addressed through its own character map,
// and an assumed encoding is a bad guess at it — a mathematical font puts a
// capital gamma where the standard encoding puts an inverted exclamation mark.
// The program knows, and this is how it is asked.
//
// There are two ways to ask, and a program answers at most one of them. A
// PostScript program names its glyphs and carries an encoding from code to
// name, so the name is the answer. A TrueType program usually does neither —
// a subset of one names nothing worth reading and has no code-to-name
// encoding at all — but it does carry character maps, and walking through one
// to the glyph and back out of another says which character the glyph is for.
func attachProgram(f *pdffont.Font) {
if f.Kind() == pdffont.Composite {
return
}
key, data, ok := f.Program()
if !ok {
return
}
program, err := readProgram(key, data)
if err != nil {
return
}
maps := chooseCharacterMaps(program)
// Only a simple font reaches here, and its codes are bytes.
f.SetFallback(func(code int) (string, bool) {
if r, ok := runeByName(program, code); ok {
return string(r), true
}
if r, ok := runeByCharacterMap(program, maps, code); ok {
return string(r), true
}
return "", false
})
}
// runeByName asks the program's own encoding what it calls a code, and reads
// the character out of that name. Only a PostScript program — a Type 1 one, or
// the CFF outlines of an OpenType font — carries such an encoding.
func runeByName(program *opentype.Font, code int) (rune, bool) {
gid, ok := program.GlyphIndexByCode(byte(code))
if !ok {
return 0, false
}
// A glyph the program does not name comes back as no name at all,
// which names no character either.
name, _ := program.GlyphName(gid)
return pdffont.RuneOfGlyphName(name)
}
// characterMaps says which of a program's cmap subtables are worth addressing,
// by index, or -1 for one the program does not carry.
type characterMaps struct {
// symbol is the Microsoft Symbol subtable, platform 3 encoding 0: a font's
// own codes, conventionally written at 0xF000 + code.
symbol int
// mac is the Macintosh Roman subtable, platform 1 encoding 0, indexed by
// single bytes.
mac int
// unicode is a Unicode subtable — Microsoft Unicode, platform 3 encoding 1
// or 10, or anything on platform 0 — indexed by codepoint. This is the one
// that is inverted, because it is the only one whose codes are characters.
unicode int
}
// chooseCharacterMaps picks out the subtables a font is addressed through and
// the one that says which character a glyph is for.
//
// A font can carry several subtables of a kind; the first of each is taken,
// which is what a font that repeats one means by repeating it.
func chooseCharacterMaps(program *opentype.Font) characterMaps {
m := characterMaps{symbol: -1, mac: -1, unicode: -1}
for i := range program.NumCharacterMaps() {
platform, encoding, _, _ := program.CharacterMap(i)
switch {
case platform == 3 && encoding == 0:
if m.symbol < 0 {
m.symbol = i
}
case platform == 1 && encoding == 0:
if m.mac < 0 {
m.mac = i
}
case platform == 0, platform == 3 && (encoding == 1 || encoding == 10):
if m.unicode < 0 {
m.unicode = i
}
}
}
return m
}
// runeByCharacterMap walks a code through the font's own character map to a
// glyph, and back out of its Unicode character map to the character that glyph
// stands for.
//
// The way in is the one poppler uses for a font the document gave no encoding:
// the Microsoft Symbol subtable if there is one, else the Macintosh Roman one,
// addressed by the raw code and then, failing that, by 0xF000 + code, which is
// where such subtables are conventionally written.
//
// The way out is the Unicode subtable, inverted. Without one there is no way
// out: a glyph on its own says nothing about which character it is, and a
// guess would be exactly the wrong letter the caller's guard exists to refuse.
func runeByCharacterMap(program *opentype.Font, m characterMaps, code int) (rune, bool) {
if m.unicode < 0 {
return 0, false
}
in := m.symbol
if in < 0 {
in = m.mac
}
if in < 0 {
return 0, false
}
gid, ok := program.GlyphIndexInMap(in, rune(code))
if !ok {
gid, ok = program.GlyphIndexInMap(in, rune(0xF000|code))
}
if !ok {
return 0, false
}
r, ok := program.RuneOfGlyphInMap(m.unicode, gid)
if !ok || !readableRune(r) {
return 0, false
}
return r, true
}
// readableRune reports whether a character recovered this way is worth
// reporting as text.
//
// A private-use codepoint is not: it means whatever the font decided it means
// and nothing outside the font can read it, so a page full of them searches no
// better than a page of nothing and looks, wrongly, like it was read. Control
// characters are refused for the same reason — a page does not say them.
func readableRune(r rune) bool {
switch {
case r < 0x20, r >= 0x7F && r <= 0x9F:
return false
case r >= 0xE000 && r <= 0xF8FF:
return false
case r >= 0xF0000:
return false
}
return true
}
// readProgram decodes an embedded font program. Which key it arrived under
// says what it is: FontFile2 is TrueType, FontFile a PostScript Type 1
// program, and FontFile3 either a bare CFF one or a whole OpenType font — the
// key alone does not settle that one, so both are tried.
func readProgram(key reader.Name, data []byte) (*opentype.Font, error) {
switch key {
case "FontFile":
return opentype.ParseType1(data)
case "FontFile3":
if f, err := opentype.Parse(data); err == nil {
return f, nil
}
return opentype.ParseCFF(data)
}
return opentype.Parse(data)
}
vendor/github.com/go-pdfkit/extract/renovate.json
Generated Vendored
+4
View File
@@ -0,0 +1,4 @@
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": ["github>go-pdfkit/.github"]
}
vendor/github.com/go-pdfkit/extract/run.go
Generated Vendored
+273
View File
@@ -0,0 +1,273 @@
package extract
import (
"strings"
"github.com/go-pdfkit/pdffont"
"github.com/go-pdfkit/reader"
)
// maxFormDepth is how deeply one form may draw another before a page is taken
// to be drawing itself.
const maxFormDepth = 12
// An extractor walks a content stream keeping just enough state to say where
// each piece of text was drawn.
type extractor struct {
doc *reader.Document
runs []Run
fonts map[int]*pdffont.Font
// tm is where the next glyph goes and tlm where the line began; both
// belong to the page rather than to the graphics state.
tm, tlm matrix
images []Image
// wantPictures says whether anybody is going to read Image.Data. When
// nobody is, the filters are left undone.
wantPictures bool
}
// run walks a content stream.
func (e *extractor) run(content []byte, resources reader.Dict, g state, depth int) {
if depth > maxFormDepth {
return
}
// The operations are taken one at a time rather than all at once. The
// walk only ever goes forwards, so nothing needs the list; keeping it
// costs a struct and a slice header for every operator on the page, and
// pages are not always small. One arXiv figure holds a single page of
// 84.8 MB of content stream and 4 579 973 operations: holding them cost
// 2 166 MB where reading them one by one costs 935 MB.
scan := reader.NewContentScanner(content)
var stack []state
for {
op, more := scan.Next()
if !more {
break
}
n := numbers(op.Operands)
switch op.Operator {
case "q":
stack = append(stack, g)
case "Q":
if len(stack) > 0 {
g = stack[len(stack)-1]
stack = stack[:len(stack)-1]
}
case "cm":
if len(n) >= 6 {
g.ctm = matrix{n[0], n[1], n[2], n[3], n[4], n[5]}.mul(g.ctm)
}
case "BT":
e.tm, e.tlm = identity, identity
case "ET":
e.tm, e.tlm = identity, identity
case "Tf":
e.setFont(&g, op.Operands, n, resources)
case "Tc":
if len(n) >= 1 {
g.text.charSpace = n[0]
}
case "Tw":
if len(n) >= 1 {
g.text.wordSpace = n[0]
}
case "Tz":
if len(n) >= 1 {
g.text.scale = n[0] / 100
}
case "TL":
if len(n) >= 1 {
g.text.leading = n[0]
}
case "Ts":
if len(n) >= 1 {
g.text.rise = n[0]
}
case "Tr":
if len(n) >= 1 {
g.text.mode = int(n[0])
}
case "Td":
if len(n) >= 2 {
e.tlm = matrix{1, 0, 0, 1, n[0], n[1]}.mul(e.tlm)
e.tm = e.tlm
}
case "TD":
if len(n) >= 2 {
g.text.leading = -n[1]
e.tlm = matrix{1, 0, 0, 1, n[0], n[1]}.mul(e.tlm)
e.tm = e.tlm
}
case "Tm":
if len(n) >= 6 {
e.tlm = matrix{n[0], n[1], n[2], n[3], n[4], n[5]}
e.tm = e.tlm
}
case "T*":
e.nextLine(&g)
case "Tj":
e.showOperands(&g, op.Operands)
case "'":
e.nextLine(&g)
e.showOperands(&g, op.Operands)
case "\"":
if len(n) >= 2 {
g.text.wordSpace, g.text.charSpace = n[0], n[1]
}
e.nextLine(&g)
e.showOperands(&g, op.Operands)
case "TJ":
e.showArray(&g, op.Operands)
case "Do":
e.doXObject(&g, op.Operands, resources, depth)
case "BI":
e.inlineImage(&g, op.Image)
}
}
}
// setFont reads the font a Tf operator names.
func (e *extractor) setFont(g *state, operands []reader.Object, n []float64, resources reader.Dict) {
if len(operands) < 2 {
return
}
name, ok := reader.ToName(operands[0])
if !ok {
return
}
if len(n) >= 1 {
g.text.size = n[len(n)-1]
}
g.text.fontName = string(name)
g.text.font = e.font(name, resources)
}
// font reads a font out of the page's resources, remembering the ones already
// read so that a page setting the same font a thousand times reads it once.
func (e *extractor) font(name reader.Name, resources reader.Dict) *pdffont.Font {
fonts, ok := e.doc.GetDict(resources, "Font")
if !ok {
return nil
}
if ref, ok := fonts.Get(name).(reader.Ref); ok {
if f, seen := e.fonts[ref.Num]; seen {
return f
}
dict, ok := e.doc.GetDict(fonts, name)
if !ok {
return nil
}
f := pdffont.Read(e.doc, dict)
attachProgram(f)
e.fonts[ref.Num] = f
return f
}
dict, ok := e.doc.GetDict(fonts, name)
if !ok {
return nil
}
f := pdffont.Read(e.doc, dict)
attachProgram(f)
return f
}
// nextLine moves down by the leading, which is what T* and the quote
// operators do.
func (e *extractor) nextLine(g *state) {
e.tlm = matrix{1, 0, 0, 1, 0, -g.text.leading}.mul(e.tlm)
e.tm = e.tlm
}
// showOperands shows the string a Tj-like operator carries.
func (e *extractor) showOperands(g *state, operands []reader.Object) {
for _, o := range operands {
if s, ok := reader.ToString(o); ok {
e.show(g, s)
return
}
}
}
// showArray shows the pieces a TJ operator carries, moving the pen by the
// numbers between them.
func (e *extractor) showArray(g *state, operands []reader.Object) {
for _, o := range operands {
arr, ok := reader.ToArray(o)
if !ok {
continue
}
for _, item := range arr {
if s, ok := reader.ToString(item); ok {
e.show(g, s)
continue
}
if v, ok := reader.ToFloat(item); ok {
e.advance(g, -v/1000*g.text.size*g.text.scale)
}
}
}
}
// show reads one string as text and records where it was drawn.
func (e *extractor) show(g *state, s []byte) {
f := g.text.font
if f == nil || len(s) == 0 {
return
}
start := e.tm.mul(g.ctm)
x, y := start.apply(0, g.text.rise)
var text strings.Builder
unreadable := false
width := 0.0
for _, code := range f.Codes(s) {
if t, ok := f.Text(code); ok {
text.WriteString(pdffont.TrimText(t))
} else {
unreadable = true
}
adv := (f.Width(code)*g.text.size + g.text.charSpace) * g.text.scale
if f.Kind() != pdffont.Composite && code == ' ' {
adv += g.text.wordSpace * g.text.scale
}
width += adv
}
e.advance(g, width)
if text.Len() == 0 && !unreadable {
return
}
// How far a length in text space reaches on the page: the text matrix and
// the page's transform together, which is the same matrix the run's own
// position came through.
//
// Only the page's transform used to be counted here, and the text matrix
// was left out. A document is free to put the whole of its scale there —
// "/F1 1 Tf" and then a text matrix of ten is how TeX writes every PDF it
// has ever produced — and such a document reported its text as one point
// tall with letters half a point wide, while saying quite correctly where
// on the page each run began.
//
// Those are the two numbers a word break is decided by. A gap measured in
// points was being weighed against a space width ten times too small, so
// every ordinary kern between two letters looked like a space: "Original
// Domain" came back as "Or ig inal D omain". Across four thousand arXiv
// figures, 57% of the words a reader got back were one or two letters
// long. Nothing failed, and nothing was slow; the text was simply no
// longer text.
scale := start.scale()
e.runs = append(e.runs, Run{
Text: text.String(),
X: x,
Y: y,
Width: width * scale,
Size: g.text.size * scale,
Font: g.text.fontName,
Space: f.Width(' ') * g.text.size * g.text.scale * scale,
Invisible: g.text.mode == 3,
Unreadable: unreadable,
})
}
// advance moves the pen along the line by a distance in text space.
func (e *extractor) advance(g *state, by float64) {
e.tm = matrix{1, 0, 0, 1, by, 0}.mul(e.tm)
}
vendor/github.com/go-pdfkit/extract/state.go
Generated Vendored
+134
View File
@@ -0,0 +1,134 @@
package extract
import (
"github.com/go-pdfkit/pdffont"
"github.com/go-pdfkit/reader"
)
// A matrix is the six numbers a PDF transform is written as. Extraction needs
// only to multiply them and to move a point, so it carries its own rather than
// depending on a drawing library it has no other use for.
type matrix struct{ a, b, c, d, e, f float64 }
// identity is the transform that changes nothing.
var identity = matrix{1, 0, 0, 1, 0, 0}
// mul is the transform that applies m first and then n, which is the order a
// PDF composes them in.
func (m matrix) mul(n matrix) matrix {
return matrix{
a: m.a*n.a + m.b*n.c,
b: m.a*n.b + m.b*n.d,
c: m.c*n.a + m.d*n.c,
d: m.c*n.b + m.d*n.d,
e: m.e*n.a + m.f*n.c + n.e,
f: m.e*n.b + m.f*n.d + n.f,
}
}
// apply moves a point.
func (m matrix) apply(x, y float64) (float64, float64) {
return m.a*x + m.c*y + m.e, m.b*x + m.d*y + m.f
}
// scale is how much the transform stretches a length, taken as the geometric
// mean of its two axes so that a rotation counts as no stretch at all.
func (m matrix) scale() float64 {
x := m.a*m.a + m.b*m.b
y := m.c*m.c + m.d*m.d
return sqrt(sqrt(x * y))
}
// sqrt without pulling in the whole of math for one use.
func sqrt(v float64) float64 {
if v <= 0 {
return 0
}
x := v
for i := 0; i < 24; i++ {
x = (x + v/x) / 2
}
return x
}
// A textState is everything the showing operators read besides the string.
type textState struct {
font *pdffont.Font
fontName string
size float64
charSpace float64
wordSpace float64
scale float64 // horizontal scaling, as a fraction
leading float64
rise float64
mode int
}
// A state is the graphics state, of which extraction needs only the transform
// and the text parameters.
type state struct {
ctm matrix
text textState
}
// initialState is the transform a page starts in, which for extraction is the
// one that undoes the page's own box: text comes back in the page's
// coordinates, counting up from the bottom left of what is visible.
func initialState(d *reader.Document, page reader.Dict) state {
box := pageBox(d, page)
return state{
ctm: matrix{1, 0, 0, 1, -box[0], -box[1]},
text: textState{scale: 1},
}
}
// pageBox is the area of the page that is visible.
func pageBox(d *reader.Document, page reader.Dict) [4]float64 {
for _, key := range []reader.Name{"CropBox", "MediaBox"} {
if b, ok := rectangle(d, page.Get(key)); ok {
return b
}
}
return [4]float64{0, 0, 612, 792}
}
// rectangle reads a PDF rectangle, put the right way round.
func rectangle(d *reader.Document, o reader.Object) ([4]float64, bool) {
var out [4]float64
arr, ok := reader.ToArray(resolve(d, o))
if !ok || len(arr) < 4 {
return out, false
}
for i := 0; i < 4; i++ {
v, ok := reader.ToFloat(resolve(d, arr[i]))
if !ok {
return out, false
}
out[i] = v
}
if out[0] > out[2] {
out[0], out[2] = out[2], out[0]
}
if out[1] > out[3] {
out[1], out[3] = out[3], out[1]
}
return out, true
}
// resolve follows an indirect reference. A document that opened cannot fail to
// resolve one.
func resolve(d *reader.Document, o reader.Object) reader.Object {
out, _ := d.Resolve(o)
return out
}
// numbers reads the operands that are numbers.
func numbers(operands []reader.Object) []float64 {
out := make([]float64, 0, len(operands))
for _, o := range operands {
if v, ok := reader.ToFloat(o); ok {
out = append(out, v)
}
}
return out
}
vendor/github.com/go-pdfkit/pdffont/.gitignore
Generated
+1
View File
@@ -0,0 +1 @@
coverage.out
vendor/github.com/go-pdfkit/pdffont/LICENSE
Generated Vendored
+29
View File
@@ -0,0 +1,29 @@
BSD 3-Clause License
Copyright (c) 2026 the go-pdfkit/render authors
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
vendor/github.com/go-pdfkit/pdffont/README.md
Generated Vendored
+75
View File
@@ -0,0 +1,75 @@
# pdffont
[![CI](https://github.com/go-pdfkit/pdffont/actions/workflows/ci.yml/badge.svg)](https://github.com/go-pdfkit/pdffont/actions/workflows/ci.yml)
[![Go Reference](https://pkg.go.dev/badge/github.com/go-pdfkit/pdffont.svg)](https://pkg.go.dev/github.com/go-pdfkit/pdffont)
[![License: BSD-3-Clause](https://img.shields.io/badge/License-BSD--3--Clause-blue.svg)](LICENSE)
[![Coverage](https://img.shields.io/badge/coverage-100%25-brightgreen.svg)](#how-it-is-checked)
The PDF side of a font: not the outlines, which are the font program's
business, but everything the document itself says about how a string of
bytes is to be read.
Which codes a string holds, how wide each one is, what each is called, and
what text each stands for. It is what a renderer needs before it can ask a
font program for a glyph, and all an extractor needs to read a page back as
words — so neither has to say it twice.
Pure Go, no C, nothing outside [`reader`](https://github.com/go-pdfkit/reader)
and the standard library.
## What it reads
Simple fonts, addressed one byte at a time through a base encoding and
whatever differences the document names on top of it. Composite fonts,
addressed by character identifier, with their widths and their map from
identifier to glyph. Type 3 fonts, whose glyphs are little drawings and
whose widths are in a space of their own.
And `/ToUnicode`, which is the one thing in a PDF that says what its text
actually **says**. Without it a page can be drawn but not read.
## What it refuses to guess
A font that says it is **symbolic** is addressed through its own character
map, and a base encoding is a guess at it — a bad one. A mathematical font
puts a capital gamma where the standard encoding puts an inverted
exclamation mark, so reading the name gives the wrong letter with nothing
to say so.
Such a code is left unread unless the document named it itself, or unless a
caller installs [`Font.SetFallback`](https://pkg.go.dev/github.com/go-pdfkit/pdffont#Font.SetFallback)
to say what the embedded program calls it. Reading the program is another
package's work; `extract` does it with
[`go-opentype`](https://github.com/go-opentype/opentype).
## How it is checked
100% of statements, including every branch that handles a dictionary saying
something it should not.
And the glyph-name table — which is the one piece of data here, and where a
single wrong entry would silently give the wrong letter — is held up
against the documents themselves. A font that both names a code and carries
a `/ToUnicode` entry for it states the same fact twice, so the two can be
compared. Over 12 000 corpus files: **63 515 names checked, 63 074 agreed —
99.31%**. The disagreements are producers whose own maps are wrong
(`alpha` → `↵`, `ff` → `!`), plus `Delta` and `Omega`, which Adobe's list
gives as the increment and ohm signs; the Greek letters written here are
the same characters by Unicode's own equivalence, and the ones a person
searching would type.
## The rest of the family
[**reader**](https://github.com/go-pdfkit/reader) reads and writes the format ·
[**ops**](https://github.com/go-pdfkit/ops) is the verbs and the `pdfops` command ·
[**render**](https://github.com/go-pdfkit/render) turns a page into pixels ·
[**pdffont**](https://github.com/go-pdfkit/pdffont) is what a document says about a font ·
[**extract**](https://github.com/go-pdfkit/extract) reads a page back ·
[**coedit**](https://github.com/go-pdfkit/coedit) is a PDF several people edit at once ·
[**app**](https://github.com/go-pdfkit/app) is the browser workbench.
Documentation for all of it: <https://go-pdfkit.github.io/docs/>
## Licence
BSD-3-Clause.
vendor/github.com/go-pdfkit/pdffont/encoding.go
Generated Vendored
+302
View File
@@ -0,0 +1,302 @@
package pdffont
import "strings"
// The two encodings a simple font is addressed through when it does not carry
// one of its own, as a name per code. Only the codes that name a glyph appear;
// the rest are blank.
//
// These are the tables the specification prints. They matter because a font
// program is addressed by glyph name or by character, and a content stream is
// written in single bytes: without them, every byte would have to be guessed
// at.
// StandardEncoding is Adobe's own, which most Type 1 fonts assume, and what a
// simple font is read with when it says nothing else.
var StandardEncoding = [256]string{
32: "space", 33: "exclam", 34: "quotedbl", 35: "numbersign", 36: "dollar",
37: "percent", 38: "ampersand", 39: "quoteright", 40: "parenleft",
41: "parenright", 42: "asterisk", 43: "plus", 44: "comma", 45: "hyphen",
46: "period", 47: "slash", 48: "zero", 49: "one", 50: "two", 51: "three",
52: "four", 53: "five", 54: "six", 55: "seven", 56: "eight", 57: "nine",
58: "colon", 59: "semicolon", 60: "less", 61: "equal", 62: "greater",
63: "question", 64: "at", 65: "A", 66: "B", 67: "C", 68: "D", 69: "E",
70: "F", 71: "G", 72: "H", 73: "I", 74: "J", 75: "K", 76: "L", 77: "M",
78: "N", 79: "O", 80: "P", 81: "Q", 82: "R", 83: "S", 84: "T", 85: "U",
86: "V", 87: "W", 88: "X", 89: "Y", 90: "Z", 91: "bracketleft",
92: "backslash", 93: "bracketright", 94: "asciicircum", 95: "underscore",
96: "quoteleft", 97: "a", 98: "b", 99: "c", 100: "d", 101: "e", 102: "f",
103: "g", 104: "h", 105: "i", 106: "j", 107: "k", 108: "l", 109: "m",
110: "n", 111: "o", 112: "p", 113: "q", 114: "r", 115: "s", 116: "t",
117: "u", 118: "v", 119: "w", 120: "x", 121: "y", 122: "z",
123: "braceleft", 124: "bar", 125: "braceright", 126: "asciitilde",
161: "exclamdown", 162: "cent", 163: "sterling", 164: "fraction",
165: "yen", 166: "florin", 167: "section", 168: "currency",
169: "quotesingle", 170: "quotedblleft", 171: "guillemotleft",
172: "guilsinglleft", 173: "guilsinglright", 174: "fi", 175: "fl",
177: "endash", 178: "dagger", 179: "daggerdbl", 180: "periodcentered",
182: "paragraph", 183: "bullet", 184: "quotesinglbase",
185: "quotedblbase", 186: "quotedblright", 187: "guillemotright",
188: "ellipsis", 189: "perthousand", 191: "questiondown", 193: "grave",
194: "acute", 195: "circumflex", 196: "tilde", 197: "macron", 198: "breve",
199: "dotaccent", 200: "dieresis", 202: "ring", 203: "cedilla",
205: "hungarumlaut", 206: "ogonek", 207: "caron", 208: "emdash",
225: "AE", 227: "ordfeminine", 232: "Lslash", 233: "Oslash", 234: "OE",
235: "ordmasculine", 241: "ae", 245: "dotlessi", 248: "lslash",
249: "oslash", 250: "oe", 251: "germandbls",
}
// WinAnsiEncoding is the one a file written on Windows assumes, and the one
// most producers name.
var WinAnsiEncoding = [256]string{
32: "space", 33: "exclam", 34: "quotedbl", 35: "numbersign", 36: "dollar",
37: "percent", 38: "ampersand", 39: "quotesingle", 40: "parenleft",
41: "parenright", 42: "asterisk", 43: "plus", 44: "comma", 45: "hyphen",
46: "period", 47: "slash", 48: "zero", 49: "one", 50: "two", 51: "three",
52: "four", 53: "five", 54: "six", 55: "seven", 56: "eight", 57: "nine",
58: "colon", 59: "semicolon", 60: "less", 61: "equal", 62: "greater",
63: "question", 64: "at", 65: "A", 66: "B", 67: "C", 68: "D", 69: "E",
70: "F", 71: "G", 72: "H", 73: "I", 74: "J", 75: "K", 76: "L", 77: "M",
78: "N", 79: "O", 80: "P", 81: "Q", 82: "R", 83: "S", 84: "T", 85: "U",
86: "V", 87: "W", 88: "X", 89: "Y", 90: "Z", 91: "bracketleft",
92: "backslash", 93: "bracketright", 94: "asciicircum", 95: "underscore",
96: "grave", 97: "a", 98: "b", 99: "c", 100: "d", 101: "e", 102: "f",
103: "g", 104: "h", 105: "i", 106: "j", 107: "k", 108: "l", 109: "m",
110: "n", 111: "o", 112: "p", 113: "q", 114: "r", 115: "s", 116: "t",
117: "u", 118: "v", 119: "w", 120: "x", 121: "y", 122: "z",
123: "braceleft", 124: "bar", 125: "braceright", 126: "asciitilde",
128: "Euro", 130: "quotesinglbase", 131: "florin", 132: "quotedblbase",
133: "ellipsis", 134: "dagger", 135: "daggerdbl", 136: "circumflex",
137: "perthousand", 138: "Scaron", 139: "guilsinglleft", 140: "OE",
142: "Zcaron", 145: "quoteleft", 146: "quoteright", 147: "quotedblleft",
148: "quotedblright", 149: "bullet", 150: "endash", 151: "emdash",
152: "tilde", 153: "trademark", 154: "scaron", 155: "guilsinglright",
156: "oe", 158: "zcaron", 159: "Ydieresis", 160: "space",
161: "exclamdown", 162: "cent", 163: "sterling", 164: "currency",
165: "yen", 166: "brokenbar", 167: "section", 168: "dieresis",
169: "copyright", 170: "ordfeminine", 171: "guillemotleft",
172: "logicalnot", 173: "hyphen", 174: "registered", 175: "macron",
176: "degree", 177: "plusminus", 178: "twosuperior",
179: "threesuperior", 180: "acute", 181: "mu", 182: "paragraph",
183: "periodcentered", 184: "cedilla", 185: "onesuperior",
186: "ordmasculine", 187: "guillemotright", 188: "onequarter",
189: "onehalf", 190: "threequarters", 191: "questiondown", 192: "Agrave",
193: "Aacute", 194: "Acircumflex", 195: "Atilde", 196: "Adieresis",
197: "Aring", 198: "AE", 199: "Ccedilla", 200: "Egrave", 201: "Eacute",
202: "Ecircumflex", 203: "Edieresis", 204: "Igrave", 205: "Iacute",
206: "Icircumflex", 207: "Idieresis", 208: "Eth", 209: "Ntilde",
210: "Ograve", 211: "Oacute", 212: "Ocircumflex", 213: "Otilde",
214: "Odieresis", 215: "multiply", 216: "Oslash", 217: "Ugrave",
218: "Uacute", 219: "Ucircumflex", 220: "Udieresis", 221: "Yacute",
222: "Thorn", 223: "germandbls", 224: "agrave", 225: "aacute",
226: "acircumflex", 227: "atilde", 228: "adieresis", 229: "aring",
230: "ae", 231: "ccedilla", 232: "egrave", 233: "eacute",
234: "ecircumflex", 235: "edieresis", 236: "igrave", 237: "iacute",
238: "icircumflex", 239: "idieresis", 240: "eth", 241: "ntilde",
242: "ograve", 243: "oacute", 244: "ocircumflex", 245: "otilde",
246: "odieresis", 247: "divide", 248: "oslash", 249: "ugrave",
250: "uacute", 251: "ucircumflex", 252: "udieresis", 253: "yacute",
254: "thorn", 255: "ydieresis",
}
// glyphRunes is what the names above stand for. A font program is looked up by
// character, so a name has to become one; anything not here, and not written
// in one of the two conventional forms, cannot be looked up that way.
var glyphRunes = map[string]rune{
"space": ' ', "exclam": '!', "quotedbl": '"', "numbersign": '#',
"dollar": '$', "percent": '%', "ampersand": '&', "quotesingle": '\'',
"quoteright": '’', "quoteleft": '‘', "parenleft": '(',
"parenright": ')', "asterisk": '*', "plus": '+', "comma": ',',
"hyphen": '-', "period": '.', "slash": '/', "zero": '0', "one": '1',
"two": '2', "three": '3', "four": '4', "five": '5', "six": '6',
"seven": '7', "eight": '8', "nine": '9', "colon": ':', "semicolon": ';',
"less": '<', "equal": '=', "greater": '>', "question": '?', "at": '@',
"bracketleft": '[', "backslash": '\\', "bracketright": ']',
"asciicircum": '^', "underscore": '_', "braceleft": '{', "bar": '|',
"braceright": '}', "asciitilde": '~', "exclamdown": '¡', "cent": '¢',
"sterling": '£', "fraction": '⁄', "yen": '¥', "florin": 'ƒ',
"section": '§', "currency": '¤', "quotedblleft": '“',
"quotedblright": '”', "guillemotleft": '«', "guillemotright": '»',
"guilsinglleft": '‹', "guilsinglright": '›', "fi": 'fi', "fl": 'fl',
"endash": '–', "emdash": '—', "dagger": '†', "daggerdbl": '‡',
"periodcentered": '·', "paragraph": '¶', "bullet": '•',
"quotesinglbase": '‚', "quotedblbase": '„', "ellipsis": '…',
"perthousand": '‰', "questiondown": '¿', "grave": '`', "acute": '´',
"circumflex": 'ˆ', "tilde": '˜', "macron": '¯', "breve": '˘',
"dotaccent": '˙', "dieresis": '¨', "ring": '˚', "cedilla": '¸',
"hungarumlaut": '˝', "ogonek": '˛', "caron": 'ˇ', "AE": 'Æ', "ae": 'æ',
"ordfeminine": 'ª', "ordmasculine": 'º', "Lslash": 'Ł', "lslash": 'ł',
"Oslash": 'Ø', "oslash": 'ø', "OE": 'Œ', "oe": 'œ', "dotlessi": 'ı',
"germandbls": 'ß', "Euro": '€', "Scaron": 'Š', "scaron": 'š',
"Zcaron": 'Ž', "zcaron": 'ž', "Ydieresis": 'Ÿ', "trademark": '™',
"brokenbar": '¦', "copyright": '©', "logicalnot": '¬', "registered": '®',
"degree": '°', "plusminus": '±', "twosuperior": '²',
"threesuperior": '³', "mu": 'µ', "onesuperior": '¹', "onequarter": '¼',
"onehalf": '½', "threequarters": '¾', "multiply": '×', "divide": '÷',
"Agrave": 'À', "Aacute": 'Á', "Acircumflex": 'Â', "Atilde": 'Ã',
"Adieresis": 'Ä', "Aring": 'Å', "Ccedilla": 'Ç', "Egrave": 'È',
"Eacute": 'É', "Ecircumflex": 'Ê', "Edieresis": 'Ë', "Igrave": 'Ì',
"Iacute": 'Í', "Icircumflex": 'Î', "Idieresis": 'Ï', "Eth": 'Ð',
"Ntilde": 'Ñ', "Ograve": 'Ò', "Oacute": 'Ó', "Ocircumflex": 'Ô',
"Otilde": 'Õ', "Odieresis": 'Ö', "Ugrave": 'Ù', "Uacute": 'Ú',
"Ucircumflex": 'Û', "Udieresis": 'Ü', "Yacute": 'Ý', "Thorn": 'Þ',
"agrave": 'à', "aacute": 'á', "acircumflex": 'â', "atilde": 'ã',
"adieresis": 'ä', "aring": 'å', "ccedilla": 'ç', "egrave": 'è',
"eacute": 'é', "ecircumflex": 'ê', "edieresis": 'ë', "igrave": 'ì',
"iacute": 'í', "icircumflex": 'î', "idieresis": 'ï', "eth": 'ð',
"ntilde": 'ñ', "ograve": 'ò', "oacute": 'ó', "ocircumflex": 'ô',
"otilde": 'õ', "odieresis": 'ö', "ugrave": 'ù', "uacute": 'ú',
"ucircumflex": 'û', "udieresis": 'ü', "yacute": 'ý', "thorn": 'þ',
"ydieresis": 'ÿ',
}
// RuneOfGlyphName turns a glyph name into the character it stands for, through
// the table above or through the two conventions the specification defines for
// naming a character directly. ok is false for a name that says nothing about
// which character it is, which a subsetted font's own names often do not.
func RuneOfGlyphName(name string) (rune, bool) {
s, ok := TextOfGlyphName(name)
if !ok {
return 0, false
}
r := []rune(s)
if len(r) != 1 {
return 0, false
}
return r[0], true
}
// TextOfGlyphName turns a glyph name into the text it stands for, which is not
// always one character: a name may say it is a ligature of several.
//
// Three of the rules here are the ones the Adobe Glyph List lays down for
// reading a name nobody has listed, and each was costing real text. A name may
// carry a variant after a full stop — "a.sc" is a small-capital A and is still
// an A, 7 272 of them in the corpus. A name may be several component names
// with underscores between them — "f_i" is the fi ligature, and without this
// "Definition" comes back "Denition", which is not a missing character so much
// as a misspelt word. And a name may simply be one nobody thought to list.
func TextOfGlyphName(name string) (string, bool) {
if name == "" {
return "", false
}
if r, ok := oneGlyphRune(name); ok {
return string(r), true
}
if seq, ok := unicodeSequence(name); ok {
return seq, true
}
// A variant of a character is still that character: the part before the
// first full stop is the name, and what follows says which cut of it.
if base, _, found := strings.Cut(name, "."); found {
if base == "" {
return "", false
}
return TextOfGlyphName(base)
}
// A name made of parts with underscores between them is the parts, in
// order — which is how a ligature is named when it has no name of its own.
if strings.Contains(name, "_") {
// Every part that can be read contributes at least one character, and
// a part that cannot gives up on the whole name, so what comes out of
// this is never empty.
var out strings.Builder
for _, part := range strings.Split(name, "_") {
piece, ok := TextOfGlyphName(part)
if !ok {
return "", false
}
out.WriteString(piece)
}
return out.String(), true
}
return "", false
}
// oneGlyphRune is a name that stands for exactly one character, by any of the
// ways a name can.
func oneGlyphRune(name string) (rune, bool) {
if r, ok := glyphRunes[name]; ok {
return r, true
}
if r, ok := latinExtendedRunes[name]; ok {
return r, true
}
if r, ok := greekAndMathRunes[name]; ok {
return r, true
}
if len(name) == 1 {
return rune(name[0]), true
}
if r, ok := parseHexName(name, "uni", 4); ok {
return r, true
}
if r, ok := parseHexName(name, "u", 4); ok {
return r, true
}
return 0, false
}
// unicodeSequence reads the uniXXXXYYYY form, which names several characters
// at once: a letter and the accent that goes over it, or the three pieces a
// Hebrew cluster is written in.
//
// It has to be told apart from something that looks exactly like it. Producers
// write a glyph's own number in the same shape — uni00000048 is glyph 72, not
// U+0000 followed by U+0048 — and 49 740 names in the corpus are that. The two
// are separable by one observation: a real sequence never begins with U+0000,
// because nothing is written after a character that does not exist. Of the
// 3 875 genuine sequences found across 14 823 embedded fonts, not one starts
// with a zero group; of the disguised glyph numbers, all of them do.
func unicodeSequence(name string) (string, bool) {
digits, ok := strings.CutPrefix(name, "uni")
if !ok || len(digits) < 8 || len(digits)%4 != 0 {
return "", false
}
var out strings.Builder
for i := 0; i < len(digits); i += 4 {
r, ok := parseHexName("uni"+digits[i:i+4], "uni", 4)
if !ok {
return "", false
}
if r == 0 {
// A glyph number wearing a character's clothes.
return "", false
}
out.WriteRune(r)
}
return out.String(), true
}
// parseHexName reads the uniXXXX and uXXXX conventions.
func parseHexName(name, prefix string, minDigits int) (rune, bool) {
if len(name) <= len(prefix) || name[:len(prefix)] != prefix {
return 0, false
}
digits := name[len(prefix):]
if len(digits) < minDigits || len(digits) > 6 {
return 0, false
}
var v rune
for i := 0; i < len(digits); i++ {
d := hexDigit(digits[i])
if d < 0 {
return 0, false
}
v = v<<4 | rune(d)
}
return v, true
}
// hexDigit reads one hexadecimal digit, or reports -1.
func hexDigit(c byte) int {
switch {
case c >= '0' && c <= '9':
return int(c - '0')
case c >= 'A' && c <= 'F':
return int(c-'A') + 10
case c >= 'a' && c <= 'f':
return int(c-'a') + 10
}
return -1
}
vendor/github.com/go-pdfkit/pdffont/extended.go
Generated Vendored
+71
View File
@@ -0,0 +1,71 @@
package pdffont
// The glyph names above cover the Latin every English document is set in and
// stop there. A document in Czech, Polish, Slovak, Hungarian, Turkish,
// Romanian, Latvian, Lithuanian or Maltese is set in the block that follows,
// and every one of its accented letters was being read as nothing at all —
// which is worse than it sounds, because a dropped character leaves no mark:
// "Příliš" comes back "Píliš" and looks like a word.
//
// Counted across the 118 833-file corpus: of 52.4 million glyph names, 483 382
// could not be turned into text, and the largest nameable share of those was
// this block — rcaron 650, zdotaccent 617, Ccaron 611, nacute 602, aogonek
// 602, scedilla 597, gbreve 597, and so on for a hundred more.
//
// These are the names the Adobe Glyph List gives, and the characters are not
// in doubt: unlike the subsetted fonts that call their glyphs g0 or dress a
// glyph number up as uni00000048, a name like "rcaron" says exactly one thing.
var latinExtendedRunes = map[string]rune{
"Amacron": 'Ā', "amacron": 'ā', "Abreve": 'Ă', "abreve": 'ă',
"Aogonek": 'Ą', "aogonek": 'ą', "Cacute": 'Ć', "cacute": 'ć',
"Ccircumflex": 'Ĉ', "ccircumflex": 'ĉ', "Cdotaccent": 'Ċ', "cdotaccent": 'ċ',
"Ccaron": 'Č', "ccaron": 'č', "Dcaron": 'Ď', "dcaron": 'ď',
"Dcroat": 'Đ', "dcroat": 'đ', "Dslash": 'Đ', "dmacron": 'đ',
"Emacron": 'Ē', "emacron": 'ē', "Ebreve": 'Ĕ', "ebreve": 'ĕ',
"Edotaccent": 'Ė', "edotaccent": 'ė', "Eogonek": 'Ę', "eogonek": 'ę',
"Ecaron": 'Ě', "ecaron": 'ě', "Gcircumflex": 'Ĝ', "gcircumflex": 'ĝ',
"Gbreve": 'Ğ', "gbreve": 'ğ', "Gdotaccent": 'Ġ', "gdotaccent": 'ġ',
"Gcommaaccent": 'Ģ', "gcommaaccent": 'ģ',
"Hcircumflex": 'Ĥ', "hcircumflex": 'ĥ', "Hbar": 'Ħ', "hbar": 'ħ',
"Itilde": 'Ĩ', "itilde": 'ĩ', "Imacron": 'Ī', "imacron": 'ī',
"Ibreve": 'Ĭ', "ibreve": 'ĭ', "Iogonek": 'Į', "iogonek": 'į',
"Idotaccent": 'İ', "dotlessi": 'ı', "IJ": 'IJ', "ij": 'ij',
"Jcircumflex": 'Ĵ', "jcircumflex": 'ĵ',
"Kcommaaccent": 'Ķ', "kcommaaccent": 'ķ', "kgreenlandic": 'ĸ',
"Lacute": 'Ĺ', "lacute": 'ĺ', "Lcommaaccent": 'Ļ', "lcommaaccent": 'ļ',
"Lcaron": 'Ľ', "lcaron": 'ľ', "Ldot": 'Ŀ', "ldot": 'ŀ',
"Nacute": 'Ń', "nacute": 'ń', "Ncommaaccent": 'Ņ', "ncommaaccent": 'ņ',
"Ncaron": 'Ň', "ncaron": 'ň', "napostrophe": 'ʼn', "Eng": 'Ŋ', "eng": 'ŋ',
"Omacron": 'Ō', "omacron": 'ō', "Obreve": 'Ŏ', "obreve": 'ŏ',
"Ohungarumlaut": 'Ő', "ohungarumlaut": 'ő',
"Racute": 'Ŕ', "racute": 'ŕ', "Rcommaaccent": 'Ŗ', "rcommaaccent": 'ŗ',
"Rcaron": 'Ř', "rcaron": 'ř', "Sacute": 'Ś', "sacute": 'ś',
"Scircumflex": 'Ŝ', "scircumflex": 'ŝ', "Scedilla": 'Ş', "scedilla": 'ş',
"Tcommaaccent": 'Ţ', "tcommaaccent": 'ţ', "Tcaron": 'Ť', "tcaron": 'ť',
"Tbar": 'Ŧ', "tbar": 'ŧ', "Utilde": 'Ũ', "utilde": 'ũ',
"Umacron": 'Ū', "umacron": 'ū', "Ubreve": 'Ŭ', "ubreve": 'ŭ',
"Uring": 'Ů', "uring": 'ů', "Uhungarumlaut": 'Ű', "uhungarumlaut": 'ű',
"Uogonek": 'Ų', "uogonek": 'ų', "Wcircumflex": 'Ŵ', "wcircumflex": 'ŵ',
"Ycircumflex": 'Ŷ', "ycircumflex": 'ŷ',
"Zacute": 'Ź', "zacute": 'ź', "Zdotaccent": 'Ż', "zdotaccent": 'ż',
"longs": 'ſ',
// A handful outside that block that the same documents use: the Romanian
// letters written with a comma below rather than a cedilla, the Turkish
// and Azeri schwa, and the two the Adobe list names for Welsh and Maltese.
"Scommaaccent": 'Ș', "scommaaccent": 'ș',
"Afii10017": 'А', "afii10017": 'А',
"Wgrave": 'Ẁ', "wgrave": 'ẁ', "Wacute": 'Ẃ', "wacute": 'ẃ',
"Wdieresis": 'Ẅ', "wdieresis": 'ẅ',
"Ygrave": 'Ỳ', "ygrave": 'ỳ',
"Hcaron": 'Ȟ', "hcaron": 'ȟ',
// The rest of what the corpus asks for and nothing could answer, each of
// them a name the Adobe list gives and none of them in doubt: the cedilla
// spellings of the Romanian letters, the capital sharp s, the script ell
// mathematicians write lengths with, and the angle brackets.
"Tcedilla": 'Ţ', "tcedilla": 'ţ', "Germandbls": 'ẞ',
"lscript": 'ℓ', "openbullet": '◦',
"angbracketleft": '〈', "angbracketright": '〉',
"Scommabelow": 'Ș', "scommabelow": 'ș',
"Tcommabelow": 'Ț', "tcommabelow": 'ț',
}
vendor/github.com/go-pdfkit/pdffont/font.go
Generated Vendored
+245
View File
@@ -0,0 +1,245 @@
// Package pdffont is the PDF side of a font: not the outlines, which are the
// font program's business, but everything the document itself says about how a
// string of bytes is to be read. Which codes a string holds, how wide each one
// is, what each is called, and what text each stands for.
//
// It is what a renderer needs before it can ask a font program for a glyph,
// and all an extractor needs to read a page back as words. Both would
// otherwise have to say the same things twice.
package pdffont
import (
"github.com/go-pdfkit/reader"
)
// A Kind says how a font's codes are read.
type Kind int
const (
// Simple fonts take one byte a code and are addressed by glyph name.
Simple Kind = iota
// Composite fonts take a character identifier, usually two bytes wide.
Composite
// Type3 fonts draw each glyph by running a little content stream.
Type3
)
// A Font is a font as the document describes it.
type Font struct {
kind Kind
dict reader.Dict
descriptor reader.Dict
doc *reader.Document
names map[int]string // what each code is called, for a simple font
// chosen says which of those names the document chose itself, in a
// Differences array, rather than inheriting from a base encoding.
chosen map[int]bool
widths map[int]float64 // in text space: a thousandth of the point size
defaultW float64
toUni map[int]string
symbolic bool
// fallback is a last resort for codes the document itself says nothing
// about: what the embedded font program calls them. A caller that can
// read the program installs one.
fallback func(code int) (string, bool)
// cidToGID maps a character identifier to a glyph number, for a composite
// font whose descendant carries a map rather than using them as they are.
cidToGID []byte
// The pieces only a Type 3 font has.
fontMatrix [6]float64
charProcs reader.Dict
t3Res reader.Dict
}
// Read takes what a font dictionary says. It never fails: a dictionary that
// says nothing useful gives a font that reads its codes one byte at a time and
// advances by half an em, which is what a reader has to do with one anyway.
func Read(d *reader.Document, dict reader.Dict) *Font {
f := &Font{
doc: d, dict: dict, widths: map[int]float64{}, defaultW: 0.5,
fontMatrix: [6]float64{0.001, 0, 0, 0.001, 0, 0},
}
switch sub, _ := reader.ToName(resolve(d, dict.Get("Subtype"))); sub {
case "Type0":
f.kind = Composite
f.readComposite()
case "Type3":
f.kind = Type3
f.readType3()
default:
f.kind = Simple
f.readSimple()
}
f.toUni = f.readToUnicode()
return f
}
// Kind says how this font's codes are read.
func (f *Font) Kind() Kind { return f.kind }
// Dict is the font dictionary itself, for whatever a caller needs that this
// does not say.
func (f *Font) Dict() reader.Dict { return f.dict }
// Descriptor is the font descriptor, which is where an embedded program lives.
func (f *Font) Descriptor() reader.Dict { return f.descriptor }
// Symbolic reports a font that says it is addressed through its own character
// map rather than through an encoding of names.
func (f *Font) Symbolic() bool { return f.symbolic }
// FontMatrix is how a Type 3 font's glyph space relates to text space. For
// every other kind it is a thousandth, which is what the format assumes.
func (f *Font) FontMatrix() [6]float64 { return f.fontMatrix }
// CharProcs are a Type 3 font's glyph drawings, by name.
func (f *Font) CharProcs() reader.Dict { return f.charProcs }
// Type3Resources are what a Type 3 font's glyph drawings draw with.
func (f *Font) Type3Resources() reader.Dict { return f.t3Res }
// Codes cuts a string into the codes it holds: one byte at a time for a simple
// font, two for a composite one, which is what every composite encoding seen in
// the wild uses.
func (f *Font) Codes(s []byte) []int {
if f.kind != Composite {
out := make([]int, len(s))
for i, b := range s {
out[i] = int(b)
}
return out
}
out := make([]int, 0, len(s)/2)
for i := 0; i+1 < len(s); i += 2 {
out = append(out, int(s[i])<<8|int(s[i+1]))
}
return out
}
// Width is how far the pen moves for one code, in text space — a multiple of
// the point size, so a width of half means half an em.
func (f *Font) Width(code int) float64 {
if w, ok := f.widths[code]; ok {
return w
}
return f.defaultW
}
// HasWidth reports whether the document said how wide this code is, rather
// than the width being the fallback [Font.Width] gives for one it did not.
//
// It matters for a font the document does not carry. One of the fourteen
// standard faces is written with no widths at all — every reader is expected
// to know Times and Helvetica and Courier by heart — so a reader drawing a
// stand-in has to take the advances from the stand-in too. Half an em for
// every letter reads as a typewriter, which is not what the page says.
func (f *Font) HasWidth(code int) bool {
_, ok := f.widths[code]
return ok
}
// GlyphName is what the document's encoding calls a code. ok is false for a
// composite font, which names nothing, and for a code the encoding passes over.
func (f *Font) GlyphName(code int) (string, bool) {
name, ok := f.names[code]
return name, ok
}
// Text is what a code stands for as characters: what the font's ToUnicode map
// says, or failing that what its glyph name says. ok is false when neither
// does — a subsetted font naming its glyphs g17 and carrying no map cannot be
// read back, and saying so is better than guessing.
func (f *Font) Text(code int) (string, bool) {
if s, ok := f.toUni[code]; ok && s != "" {
return s, true
}
if name, ok := f.names[code]; ok && f.namedByTheDocument(code) {
// Not one character: a name may say it is a ligature of several, and
// a code that stands for two letters has to give back two.
if text, ok := TextOfGlyphName(name); ok {
return text, true
}
}
if f.fallback != nil {
if s, ok := f.fallback(code); ok && s != "" {
return s, true
}
}
return "", false
}
// SetFallback installs a last resort for codes the document says nothing
// about: what the embedded font program calls them.
//
// A font that says it is symbolic is addressed through its own character map,
// and what that map says is the only honest answer for a code the document did
// not name. Reading the program is another package's work, so a caller that
// can do it hands the answer in here.
func (f *Font) SetFallback(fn func(code int) (string, bool)) { f.fallback = fn }
// namedByTheDocument reports whether a code's name can be believed as a
// statement about which character it is.
//
// A font that says it is symbolic is addressed through its own character map,
// and the base encoding is then a guess. It is a bad one: a mathematical font
// puts a capital gamma where the standard encoding puts an inverted
// exclamation mark, and reading the name would give the wrong letter with
// nothing to say so. A name the document chose itself, in a Differences
// array, is a different matter — that one it meant.
func (f *Font) namedByTheDocument(code int) bool {
return !f.symbolic || f.chosen[code]
}
// Chosen reports whether the document named this code itself rather than
// inheriting the name from a base encoding.
func (f *Font) Chosen(code int) bool { return f.chosen[code] }
// CIDToGID maps a character identifier to a glyph number for a composite font.
// ok is false when the font's map does not reach that far.
func (f *Font) CIDToGID(cid int) (int, bool) {
if f.cidToGID == nil {
return cid, true
}
i := cid * 2
if i+1 >= len(f.cidToGID) {
return 0, false
}
return int(f.cidToGID[i])<<8 | int(f.cidToGID[i+1]), true
}
// Program is the embedded font program: which key it arrived under, and its
// bytes. ok is false for a font whose program is not embedded, which is every
// standard face and many others besides.
func (f *Font) Program() (reader.Name, []byte, bool) {
for _, key := range []reader.Name{"FontFile2", "FontFile3", "FontFile"} {
stream, ok := reader.ToStream(resolve(f.doc, f.descriptor.Get(key)))
if !ok {
continue
}
data, img, err := f.doc.DecodeStream(stream)
if err != nil || img != "" {
continue
}
return key, data, true
}
return "", nil, false
}
// resolve follows an indirect reference. A document that opened cannot fail to
// resolve one, so there is nothing to handle at every use.
func resolve(d *reader.Document, o reader.Object) reader.Object {
out, _ := d.Resolve(o)
return out
}
// floatOf reads a number, or gives a default.
func floatOf(o reader.Object, def float64) float64 {
if v, ok := reader.ToFloat(o); ok {
return v
}
return def
}
vendor/github.com/go-pdfkit/pdffont/greek.go
Generated Vendored
+69
View File
@@ -0,0 +1,69 @@
package pdffont
// greekAndMathRunes are the names a mathematical document gives its glyphs.
// They are not in the Latin encodings, so a page of equations addressed by
// name — which is what every TeX document is — could not be read back without
// them: the words came out and the symbols did not.
//
// The names are Adobe's, from the Symbol font and the glyph list every
// producer follows. They are checked against the documents themselves rather
// than taken on trust: a font that carries both a name for a code and a
// ToUnicode entry for it states the same fact twice, and the corpus has tens
// of thousands of such pairs to compare against.
var greekAndMathRunes = map[string]rune{
// The Greek alphabet, lower case and upper.
"alpha": 'α', "beta": 'β', "gamma": 'γ', "delta": 'δ', "epsilon": 'ε',
"zeta": 'ζ', "eta": 'η', "theta": 'θ', "iota": 'ι', "kappa": 'κ',
"lambda": 'λ', "mu1": 'μ', "nu": 'ν', "xi": 'ξ', "omicron": 'ο',
"pi": 'π', "rho": 'ρ', "sigma": 'σ', "sigma1": 'ς', "tau": 'τ',
"upsilon": 'υ', "phi": 'φ', "chi": 'χ', "psi": 'ψ', "omega": 'ω',
"theta1": 'ϑ', "phi1": 'ϕ', "epsilon1": 'ϵ', "rho1": 'ϱ', "omega1": 'ϖ',
"kappa1": 'ϰ', "pi1": 'ϖ',
// Adobe's list gives Delta and Omega the increment and ohm signs, which
// Unicode says are the same characters as the Greek letters written
// here — canonically equivalent, and these are the forms a person
// searching the text would type.
"Alpha": 'Α', "Beta": 'Β', "Gamma": 'Γ', "Delta": 'Δ', "Epsilon": 'Ε',
"Zeta": 'Ζ', "Eta": 'Η', "Theta": 'Θ', "Iota": 'Ι', "Kappa": 'Κ',
"Lambda": 'Λ', "Mu": 'Μ', "Nu": 'Ν', "Xi": 'Ξ', "Omicron": 'Ο',
"Pi": 'Π', "Rho": 'Ρ', "Sigma": 'Σ', "Tau": 'Τ', "Upsilon": 'Υ',
"Phi": 'Φ', "Chi": 'Χ', "Psi": 'Ψ', "Omega": 'Ω', "Upsilon1": 'ϒ',
// The signs an equation is built from.
"minus": '−', "plus": '+', "equal": '=', "asteriskmath": '∗',
"lessequal": '≤', "greaterequal": '≥', "notequal": '≠', "approxequal": '≈',
"equivalence": '≡', "similar": '∼', "congruent": '≅', "proportional": '∝',
"infinity": '∞', "partialdiff": '∂', "gradient": '∇', "integral": '∫',
"summation": '∑', "product": '∏', "radical": '√', "angle": '∠',
"perpendicular": '⊥', "therefore": '∴', "dotmath": '⋅', "bullet3": '∙',
"element": '∈', "notelement": '∉', "universal": '∀', "existential": '∃',
"emptyset": '∅', "intersection": '∩', "union": '∪',
"propersubset": '⊂', "propersuperset": '⊃',
"reflexsubset": '⊆', "reflexsuperset": '⊇', "notsubset": '⊄',
"logicaland": '∧', "logicalor": '∨', "logicalnot1": '¬',
"circleplus": '⊕', "circlemultiply": '⊗',
"prime": '′', "second": '″', "minute": '′',
// Arrows, and the brackets a display equation is built out of.
"arrowleft": '←', "arrowup": '↑', "arrowright": '→', "arrowdown": '↓',
"arrowboth": '↔', "arrowupdn": '↕',
"arrowdblleft": '⇐', "arrowdblup": '⇑', "arrowdblright": '⇒',
"arrowdbldown": '⇓', "arrowdblboth": '⇔',
"angleleft": '⟨', "angleright": '⟩',
"bracketlefttp": '⎡', "bracketleftbt": '⎣', "bracketleftex": '⎢',
"bracketrighttp": '⎤', "bracketrightbt": '⎦', "bracketrightex": '⎥',
"parenlefttp": '⎛', "parenleftbt": '⎝', "parenleftex": '⎜',
"parenrighttp": '⎞', "parenrightbt": '⎠', "parenrightex": '⎟',
"bracelefttp": '⎧', "braceleftmid": '⎨', "braceleftbt": '⎩',
"bracerighttp": '⎫', "bracerightmid": '⎬', "bracerightbt": '⎭',
"braceex": '⎪',
// The ligatures and marks a book is set with.
"ff": 'ff', "ffi": 'ffi', "ffl": 'ffl', "dotlessj": 'ȷ',
"circumflexbig": 'ˆ', "tildebig": '˜',
"suchthat": '∍', "carriagereturn": '↵', "aleph": 'ℵ',
"Ifraktur": 'ℑ', "Rfraktur": 'ℜ', "weierstrass": '℘',
"circlecopyrt": '©', "trademarksans": '™', "trademarkserif": '™',
"degree1": '°', "florin1": 'ƒ',
"lozenge": '◊', "spade": '♠', "club": '♣', "heart": '♥', "diamond": '♦',
}
vendor/github.com/go-pdfkit/pdffont/read.go
Generated Vendored
+223
View File
@@ -0,0 +1,223 @@
package pdffont
import "github.com/go-pdfkit/reader"
// readSimple reads a font addressed one byte at a time: what each code is
// called, and how wide it is.
func (f *Font) readSimple() {
f.descriptor, _ = f.doc.GetDict(f.dict, "FontDescriptor")
f.symbolic = symbolicFlag(f.doc, f.descriptor)
f.names = f.readEncoding()
f.readSimpleWidths()
}
// readType3 reads a font whose glyphs are little drawings.
func (f *Font) readType3() {
f.descriptor, _ = f.doc.GetDict(f.dict, "FontDescriptor")
f.names = f.readEncoding()
f.charProcs, _ = f.doc.GetDict(f.dict, "CharProcs")
f.t3Res, _ = f.doc.GetDict(f.dict, "Resources")
if m := floatArray(f.doc, f.dict.Get("FontMatrix")); len(m) >= 6 {
copy(f.fontMatrix[:], m[:6])
}
f.readSimpleWidths()
// A Type 3 font's widths are in its own glyph space, which the font
// matrix maps onto text space.
for code, w := range f.widths {
f.widths[code] = w * 1000 * f.fontMatrix[0]
}
f.defaultW = 0
}
// symbolicFlag reads whether a font says it is addressed through its own
// character map rather than through an encoding of names.
func symbolicFlag(d *reader.Document, descriptor reader.Dict) bool {
flags, ok := reader.ToInt(resolve(d, descriptor.Get("Flags")))
if !ok {
return false
}
const symbolic, nonSymbolic = 1 << 2, 1 << 5
return flags&symbolic != 0 && flags&nonSymbolic == 0
}
// readEncoding works out what each code of a simple font is called: a base
// table, and then whatever differences the font names on top of it.
func (f *Font) readEncoding() map[int]string {
base := StandardEncoding
out := map[int]string{}
f.chosen = map[int]bool{}
enc := resolve(f.doc, f.dict.Get("Encoding"))
if name, ok := reader.ToName(enc); ok {
table := NamedEncoding(name, base)
fillNames(out, table)
// A named encoding is the document's own word on what its codes are,
// symbolic font or not.
for code, n := range table {
if n != "" {
f.chosen[code] = true
}
}
return out
}
encDict, ok := reader.ToDict(enc)
if !ok {
fillNames(out, base)
return out
}
if name, ok := reader.ToName(resolve(f.doc, encDict.Get("BaseEncoding"))); ok {
base = NamedEncoding(name, base)
}
fillNames(out, base)
f.applyDifferences(out, encDict)
return out
}
// NamedEncoding is the table a name stands for, or the fallback for a name
// that is not one of them.
func NamedEncoding(name reader.Name, fallback [256]string) [256]string {
switch name {
case "WinAnsiEncoding":
return WinAnsiEncoding
case "StandardEncoding", "MacRomanEncoding", "MacExpertEncoding":
// Mac Roman differs from the standard encoding above code 127, where
// almost nothing in a Latin document lives; using the standard table
// is closer than using none.
return StandardEncoding
}
return fallback
}
// fillNames copies a table into the map the rest of this works from.
func fillNames(out map[int]string, table [256]string) {
for code, name := range table {
if name != "" {
out[code] = name
}
}
}
// applyDifferences reads the array that names glyphs a code at a time: a
// number, then the names of the codes from there on.
func (f *Font) applyDifferences(out map[int]string, encDict reader.Dict) {
arr, ok := reader.ToArray(resolve(f.doc, encDict.Get("Differences")))
if !ok {
return
}
code := 0
for _, e := range arr {
v := resolve(f.doc, e)
if n, ok := reader.ToInt(v); ok {
code = int(n)
continue
}
if name, ok := reader.ToName(v); ok {
if code >= 0 && code < 256 {
out[code] = string(name)
f.chosen[code] = true
}
code++
}
}
}
// readSimpleWidths reads /Widths, which gives one width per code from
// /FirstChar on.
func (f *Font) readSimpleWidths() {
first, ok := reader.ToInt(resolve(f.doc, f.dict.Get("FirstChar")))
if !ok {
return
}
arr, ok := reader.ToArray(resolve(f.doc, f.dict.Get("Widths")))
if !ok {
return
}
for i, e := range arr {
if v, ok := reader.ToFloat(resolve(f.doc, e)); ok {
f.widths[int(first)+i] = v / 1000
}
}
if mw := floatOf(resolve(f.doc, f.descriptor.Get("MissingWidth")), -1); mw >= 0 {
f.defaultW = mw / 1000
}
}
// readComposite reads a font addressed by character identifier: its descendant
// holds the widths, the descriptor and the map from identifier to glyph.
func (f *Font) readComposite() {
f.defaultW = 1
arr, ok := reader.ToArray(resolve(f.doc, f.dict.Get("DescendantFonts")))
if !ok || len(arr) == 0 {
return
}
kid, ok := reader.ToDict(resolve(f.doc, arr[0]))
if !ok {
return
}
if v, ok := reader.ToFloat(resolve(f.doc, kid.Get("DW"))); ok {
f.defaultW = v / 1000
}
f.readCIDWidths(kid)
f.descriptor, _ = f.doc.GetDict(kid, "FontDescriptor")
f.symbolic = symbolicFlag(f.doc, f.descriptor)
if stream, ok := reader.ToStream(resolve(f.doc, kid.Get("CIDToGIDMap"))); ok {
if data, img, err := f.doc.DecodeStream(stream); err == nil && img == "" {
f.cidToGID = data
}
}
}
// readCIDWidths reads the /W array, which names widths one identifier at a
// time or a run at a time.
func (f *Font) readCIDWidths(kid reader.Dict) {
arr, ok := reader.ToArray(resolve(f.doc, kid.Get("W")))
if !ok {
return
}
for i := 0; i < len(arr); {
first, ok := reader.ToInt(resolve(f.doc, arr[i]))
if !ok || i+1 >= len(arr) {
return
}
next := resolve(f.doc, arr[i+1])
if list, ok := reader.ToArray(next); ok {
for k, e := range list {
if v, ok := reader.ToFloat(resolve(f.doc, e)); ok {
f.widths[int(first)+k] = v / 1000
}
}
i += 2
continue
}
last, ok := reader.ToInt(next)
if !ok || i+2 >= len(arr) {
return
}
w, ok := reader.ToFloat(resolve(f.doc, arr[i+2]))
if !ok {
return
}
if last >= first && last-first < 1<<20 {
for c := first; c <= last; c++ {
f.widths[int(c)] = w / 1000
}
}
i += 3
}
}
// floatArray reads an array of numbers, or nothing.
func floatArray(d *reader.Document, o reader.Object) []float64 {
arr, ok := reader.ToArray(resolve(d, o))
if !ok {
return nil
}
out := make([]float64, 0, len(arr))
for _, e := range arr {
v, ok := reader.ToFloat(resolve(d, e))
if !ok {
return nil
}
out = append(out, v)
}
return out
}
vendor/github.com/go-pdfkit/pdffont/tounicode.go
Generated Vendored
+252
View File
@@ -0,0 +1,252 @@
package pdffont
import (
"strings"
"unicode/utf16"
"github.com/go-pdfkit/reader"
)
// readToUnicode reads the font's /ToUnicode map, which is the one thing in a
// PDF that says what its text actually says. Without it a page can be drawn
// but not read: a subsetted font naming its glyphs by number carries no other
// clue.
func (f *Font) readToUnicode() map[int]string {
stream, ok := reader.ToStream(resolve(f.doc, f.dict.Get("ToUnicode")))
if !ok {
return nil
}
data, img, err := f.doc.DecodeStream(stream)
if err != nil || img != "" {
return nil
}
return ReadToUnicode(data)
}
// ReadToUnicode decodes a ToUnicode CMap: a little PostScript program whose
// only meaningful parts are the bfchar and bfrange blocks saying which
// characters each code stands for.
func ReadToUnicode(data []byte) map[int]string {
out := map[int]string{}
toks := cmapTokens(data)
for i := 0; i < len(toks); i++ {
switch toks[i].word {
case "beginbfchar":
i = readBFChar(toks, i+1, out)
case "beginbfrange":
i = readBFRange(toks, i+1, out)
}
}
if len(out) == 0 {
return nil
}
return out
}
// maxToUnicodeEntries is how many codes one map may name.
//
// A bfrange names a run of codes in about twenty bytes, and only the width of
// a single run was bounded — not how many runs a map could hold. So the size
// of the answer had nothing to do with the size of the question: 585 bytes
// produced 655 360 entries in 2.3 seconds, and 10 655 bytes produced
// 13 107 200 entries and a gigabyte. Every font on every page is read this
// way, so that was a gigabyte per font.
//
// The bound is not a guess. Across 5 338 /ToUnicode maps taken out of real
// documents — arXiv figures, government forms, and mozilla's pdf.js corpus —
// the median names 13 codes, the 99th percentile names 538, and the largest
// names exactly 65 536: one whole two-byte code space, which is as many codes
// as a font of that shape can have. This allows four times that, so a
// document has to be malformed or hostile to reach it, and a map that does is
// cut off there rather than being allowed to ask for everything.
const maxToUnicodeEntries = 1 << 18
// A cmapToken is one piece of such a program: a hexadecimal string, an array
// bracket, or a bare word.
type cmapToken struct {
word string
hex []byte
isHex bool
}
// cmapTokens cuts the program up. Only three shapes matter — a hexadecimal
// string, a bracket, and a word — and everything else can be a word.
func cmapTokens(b []byte) []cmapToken {
var out []cmapToken
for i := 0; i < len(b); {
c := b[i]
switch {
case c == '%':
for i < len(b) && b[i] != '\n' && b[i] != '\r' {
i++
}
case c == '<':
j := i + 1
for j < len(b) && b[j] != '>' {
j++
}
out = append(out, cmapToken{hex: unhexBytes(b[i+1 : min(j, len(b))]), isHex: true})
i = min(j+1, len(b))
case c == '[' || c == ']':
out = append(out, cmapToken{word: string(c)})
i++
case c <= ' ':
i++
default:
j := i
for j < len(b) && b[j] > ' ' && b[j] != '<' && b[j] != '[' && b[j] != ']' && b[j] != '%' {
j++
}
out = append(out, cmapToken{word: string(b[i:j])})
i = j
}
}
return out
}
// readBFChar reads pairs of "code, what it stands for" up to the block's end.
func readBFChar(toks []cmapToken, i int, out map[int]string) int {
for i+1 < len(toks) {
if toks[i].word == "endbfchar" {
return i
}
if len(out) >= maxToUnicodeEntries {
return i
}
if !toks[i].isHex || !toks[i+1].isHex {
return i
}
out[codeOf(toks[i].hex)] = textOf(toks[i+1].hex)
i += 2
}
return i
}
// readBFRange reads runs of codes: either a first character the run counts on
// from, or a list naming each one.
func readBFRange(toks []cmapToken, i int, out map[int]string) int {
for i+2 < len(toks) {
if toks[i].word == "endbfrange" {
return i
}
if !toks[i].isHex || !toks[i+1].isHex {
return i
}
lo, hi := codeOf(toks[i].hex), codeOf(toks[i+1].hex)
if hi < lo || hi-lo > 1<<16 {
return i
}
if len(out) >= maxToUnicodeEntries {
return i
}
switch {
case toks[i+2].isHex:
base := utf16Runes(toks[i+2].hex)
for c := lo; c <= hi && len(out) < maxToUnicodeEntries; c++ {
out[c] = countOn(base, c-lo)
}
i += 3
case toks[i+2].word == "[":
j := i + 3
for c := lo; c <= hi && j < len(toks) && toks[j].word != "]"; c++ {
if toks[j].isHex && len(out) < maxToUnicodeEntries {
out[c] = textOf(toks[j].hex)
}
j++
}
for j < len(toks) && toks[j].word != "]" {
j++
}
i = j + 1
default:
return i
}
}
return i
}
// codeOf reads a code, which is one or two bytes written as hexadecimal.
func codeOf(b []byte) int {
v := 0
for _, c := range b {
v = v<<8 | int(c)
}
return v
}
// textOf reads what a code stands for: characters written as sixteen-bit
// units, the way the format carries text.
func textOf(b []byte) string {
return string(utf16Runes(b))
}
// utf16Runes decodes the sixteen-bit units a ToUnicode value is written in.
func utf16Runes(b []byte) []rune {
units := make([]uint16, 0, len(b)/2)
for i := 0; i+1 < len(b); i += 2 {
units = append(units, uint16(b[i])<<8|uint16(b[i+1]))
}
return utf16.Decode(units)
}
// countOn is what the n'th code of a range stands for, counting on from the
// first: the last character moves and the rest stay, which is what the
// specification says a range means.
func countOn(base []rune, n int) string {
if len(base) == 0 {
return ""
}
out := append([]rune{}, base...)
out[len(out)-1] += rune(n)
return string(out)
}
// unhexBytes reads a hexadecimal string, ignoring whatever it was laid out
// with. An odd digit at the end is taken as the high half of a byte, which is
// what the format says.
func unhexBytes(b []byte) []byte {
var digits []byte
for _, c := range b {
if isHexDigit(c) {
digits = append(digits, hexVal(c))
}
}
if len(digits)%2 == 1 {
digits = append(digits, 0)
}
out := make([]byte, 0, len(digits)/2)
for i := 0; i+1 < len(digits); i += 2 {
out = append(out, digits[i]<<4|digits[i+1])
}
return out
}
func isHexDigit(c byte) bool {
return c >= '0' && c <= '9' || c >= 'a' && c <= 'f' || c >= 'A' && c <= 'F'
}
func hexVal(c byte) byte {
switch {
case c <= '9':
return c - '0'
case c <= 'F':
return c - 'A' + 10
default:
return c - 'a' + 10
}
}
// min is the smaller of two numbers.
func min(a, b int) int {
if a < b {
return a
}
return b
}
// TrimText tidies what a map gives back: the format allows a value to carry
// the replacement character or nothing at all, and neither is text.
func TrimText(s string) string {
s = strings.TrimFunc(s, func(r rune) bool { return r == 0 || r == 0xFFFD })
return s
}
vendor/github.com/go-pdfkit/reader/.gitignore
Generated
+3
View File
@@ -0,0 +1,3 @@
cov.out
coverage.out
*.pdf
vendor/github.com/go-pdfkit/reader/LICENSE
Generated Vendored
+29
View File
@@ -0,0 +1,29 @@
BSD 3-Clause License
Copyright (c) 2026 the go-pdfkit/reader authors
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
vendor/github.com/go-pdfkit/reader/README.md
Generated Vendored
+144
View File
@@ -0,0 +1,144 @@
# reader
[![CI](https://github.com/go-pdfkit/reader/actions/workflows/ci.yml/badge.svg)](https://github.com/go-pdfkit/reader/actions/workflows/ci.yml)
[![Go Reference](https://pkg.go.dev/badge/github.com/go-pdfkit/reader.svg)](https://pkg.go.dev/github.com/go-pdfkit/reader)
[![Go Report Card](https://goreportcard.com/badge/github.com/go-pdfkit/reader)](https://goreportcard.com/report/github.com/go-pdfkit/reader)
[![License: BSD-3-Clause](https://img.shields.io/badge/License-BSD--3--Clause-blue.svg)](LICENSE)
[![Coverage](https://img.shields.io/badge/coverage-100%25-brightgreen.svg)](#testing)
A pure-Go, **zero-C** PDF **reader** — the parsing half of
[go-pdfkit](https://github.com/go-pdfkit). Where
[`pdfkit`](https://github.com/go-pdfkit/pdfkit) authors new PDF 1.7, this
module takes existing PDF bytes apart: lexer, object model, cross-reference
tables and streams, stream filters, the standard security handler, the page
tree, and content streams. It also writes the same object graph back out,
which is what every operation on an existing file needs.
Nothing outside the Go standard library is required, so it builds for
`GOOS=js/wasm` and every 64-bit architecture the fleet targets.
## Status
Landing in waves. This module currently provides the **object layer** and
the **document structure**:
- **Lexer** — the full PDF object syntax: integers and reals in the shapes
producers actually write, literal strings with their escapes, hex strings,
names with `#xx` escapes, arrays, dictionaries, streams, and keywords.
- **Objects** — `Null`, `Bool`, `Integer`, `Real`, `String`, `Name`, `Array`,
`Dict`, `Stream`, and indirect `Ref`, with the `N G R` / `N G obj` lookahead
resolved the way the specification requires.
- **Filters** — `FlateDecode`, `LZWDecode` (with PDF's `EarlyChange`, which the
standard library's LZW does not implement), `ASCIIHexDecode`,
`ASCII85Decode`, `RunLengthDecode`, `Crypt`, plus the PNG and TIFF
predictors. Image filters (`DCTDecode`, `JPXDecode`, `CCITTFaxDecode`,
`JBIG2Decode`) are reported rather than applied, so an image consumer gets
the encoded bytes.
- **Filters that fail** — a chain that cannot be run to the end is not the end
of the stream. `DecodeRecovering` returns what the filters before the failure
produced — the prefix a damaged Flate stream did inflate, not its compressed
bytes — and says so through `Decoded.Recovered` and `Decoded.Cause`. Bytes no
filter decoded arrive in `Decoded.Undecoded`, never in `Decoded.Data`, so a
compressed stream cannot be painted as samples or tokenised as content by a
caller that forgot to check a flag.
`Decode` is the strict reading and refuses such a stream outright, so a
caller that must not act on damaged data says which it wants by which one it
calls. Page content takes the lenient reading, because a page whose content
will not decode is still a page; `PageContentDecoded` reports whether any
salvaging happened. Cross-reference streams take the strict one: a table that
has to be guessed at is worse than no table, and the repair below is the
answer to one.
- **Cross-references** — classic tables, cross-reference **streams**, **object
streams**, `/Prev` chains, and the `/XRefStm` of a hybrid file, newest
definition winning.
- **Repair** — a file whose tables are missing, truncated or simply wrong is
rebuilt by scanning it for object headers, trailers and, failing those, for
a catalogue; a file that kept its pages but lost its catalogue gets one.
This is not an exceptional path: it is what makes a reader usable. The
rebuild establishes the file key before it reads object streams, so a file
whose catalogue lives in an encrypted one can be rebuilt at all; it takes
`/Encrypt` from a cross-reference stream's dictionary when the file has no
trailer keyword to name it in. Where two object streams claim the same
object, the later one in the file wins, the same rule the header scan
applies to objects written directly — so the same damaged file reads the
same way twice. When a rebuild fails, the error says what the rebuild found
and not merely that the tables could not be read.
- **Documents** — `Open`, object resolution with cycle and recursion guards,
the trailer, the catalogue, and the page tree with the four attributes a
page inherits from its ancestors.
- **Encryption** — the standard security handler in every revision, both ways:
reading RC4 at 40 and 128 bits, AESV2, and the AES-256 of `/R 5` and `/R 6`,
with crypt filters, `/Identity` and `/EncryptMetadata`; and writing AES-256
or AES-128, with permissions, a user password and an owner password.
A password is tried as the user password and as the owner password, and
`Open` uses the empty one, so a file protected only against editing opens
with no password at all.
and steps over rubbish rather than losing the operations around it, with
inline images read whole. Where an inline image ends is the one genuinely
ambiguous thing in a content stream, since its data may spell EI itself;
the length is computed from the image's own geometry where it can be, and
otherwise every candidate EI is tried until the data before one actually
decodes.
- **Writing** — the exact inverse of the parser: objects rendered in PDF
syntax with stable dictionary ordering and strings in whichever of the two
forms is shorter, and a `Writer` that copies whole object graphs out of one
or more documents, renumbering as it goes, then lays down a cross-reference
table and a trailer. `NewPackedWriter` instead puts what it can into
compressed **object streams** and ends in a **cross-reference stream**,
which is what makes a file small.
Measured against a corpus of **118 863 real PDFs** — Matplotlib, cairo,
pdfTeX, Ghostscript, Adobe, R, Apache FOP, PDF 1.3 through 1.7 — `Open`
succeeds on **118 833** of them and finds 138 337 pages, in 8 seconds, with no
panics. Of the thirty it refuses, twenty-seven are PNG files with a `.pdf`
extension, one is PostScript, and two are PDFs truncated past recovery.
The one encrypted PDF the corpus holds — `/V 4 /R 4`, written by macOS
Quartz — decrypts to readable metadata and a valid content stream, which is
the only genuinely independent check of the key derivation there is here;
the other revisions are round-tripped against a test encryptor written from
the same algorithms.
The content-stream tokeniser reads **1 536 769 753 operations** across those
138 337 pages in a minute and a half, with no panics, no page failing to
decode, and no operator outside the seventy the format defines — beyond four
`arc` and six `nan` written by a producer that was simply wrong.
Rewriting is checked the same way: every one of the **118 833** files that
open is copied object by object into a new file, which is then re-read and
compared on page count, media boxes and the bytes of every content stream.
**All 118 833 match**, 35.5 GB in and 35.2 GB out, in twenty-eight seconds —
and **all 118 833 match again** when the packed writer is used instead, which
brings the output down to **94.6%** of the input.
Next wave: the operations built on all of this.
## Install
```sh
go get github.com/go-pdfkit/reader
```
## Testing
```sh
go test -covermode=set ./...
```
CI gates on **exact 100% statement coverage**, `go vet`, and a cross-compile
across `linux/{amd64,arm64,riscv64,loong64,ppc64le,s390x}`, `js/wasm`,
`darwin/arm64` and `windows/amd64`.
## The rest of the family
[**reader**](https://github.com/go-pdfkit/reader) reads and writes the format ·
[**ops**](https://github.com/go-pdfkit/ops) is the verbs and the `pdfops` command ·
[**render**](https://github.com/go-pdfkit/render) turns a page into pixels ·
[**pdffont**](https://github.com/go-pdfkit/pdffont) is what a document says about a font ·
[**extract**](https://github.com/go-pdfkit/extract) reads a page back ·
[**coedit**](https://github.com/go-pdfkit/coedit) is a PDF several people edit at once ·
[**app**](https://github.com/go-pdfkit/app) is the browser workbench.
Documentation for all of it: <https://go-pdfkit.github.io/docs/>
## License
BSD-3-Clause — see [LICENSE](LICENSE). Copyright the go-pdfkit/reader authors.
vendor/github.com/go-pdfkit/reader/ccitt.go
Generated Vendored
+579
View File
@@ -0,0 +1,579 @@
package reader
import "fmt"
// CCITT Group 3 and Group 4 fax decoding, as used by /CCITTFaxDecode.
//
// WHY THIS IS HERE
//
// A scanned form is a fax. 67 of the 1 633 real forms in the corpus — the
// eleven issuing bodies, not the vendor test suites — carry a CCITT-encoded
// image, 263 images between them, and 6 of their pages have nothing else on
// them at all. Those six pages drew entirely blank, and a blank page is the
// failure a reader notices before any other.
//
// WHY IT BELONGS IN THE READER RATHER THAN IN A RENDERER
//
// The other filters this package stops at — DCT, JPX, JBIG2 — carry an image
// with its own idea of how many components it has and how deep they are. This
// one does not: it produces bilevel samples, one bit a pixel, rows padded to a
// byte, and the stream dictionary says what they mean. That is a byte stream,
// so it is a filter, and every caller gets it rather than each writing its own.
//
// THE REFERENCE READ BEFORE WRITING THIS
//
// ITU-T Recommendations T.4 (Group 3) and T.6 (Group 4), by way of the tables
// and the changing-element algorithm in golang.org/x/image/ccitt — read, not
// imported, because this package has no dependencies outside the standard
// library and gains none. The code tables below were extracted mechanically
// from that package's gen.go rather than retyped, because a table of 218
// variable-length codes transcribed by hand is a table with a mistake in it.
//
// The algorithm is the one T.6 Figure 1 describes. A row is decoded into one
// byte a pixel — 0xFF white, 0x00 black — and the row before it is the
// reference against which the two-dimensional modes are resolved:
//
// b1 b2
// v v
// prev: BBBBBwwwwwBBBwwwww
// curr: BBBwwwwwBBBBBBwwww
// ^ ^ ^
// a0 a1 a2
//
// a0 is where the pen is; a1 the next colour change to its right on this row;
// b1 the first change on the row above that is to the right of a0 and of the
// opposite colour to it; b2 the next change after b1. Pass mode says a1 is at
// or beyond b2; horizontal mode reads two runs and ignores the row above;
// vertical mode puts a1 at b1 plus an offset between -3 and +3.
// The two-dimensional modes of T.4 Table 1.
const (
modePass = iota
modeH
modeV0
modeVR1
modeVR2
modeVR3
modeVL1
modeVL2
modeVL3
modeExt
)
// A ccittCode is one row of a code table: the value a code names, and the bits
// that name it, most significant first.
type ccittCode struct {
value int
bits string
}
// ccittModeCodes is Table 1 of ITU-T T.4: the two-dimensional modes.
var ccittModeCodes = []ccittCode{
{modePass, "0001"}, {modeH, "001"}, {modeV0, "1"}, {modeVR1, "011"},
{modeVR2, "000011"}, {modeVR3, "0000011"}, {modeVL1, "010"}, {modeVL2, "000010"},
{modeVL3, "0000010"}, {modeExt, "0000001"},
}
// ccittWhiteCodes is Tables 2 and 3 of ITU-T T.4 for a white run: the
// terminating codes 0 to 63, then the make-up codes in steps of 64.
var ccittWhiteCodes = []ccittCode{
{0, "00110101"}, {1, "000111"}, {2, "0111"}, {3, "1000"},
{4, "1011"}, {5, "1100"}, {6, "1110"}, {7, "1111"},
{8, "10011"}, {9, "10100"}, {10, "00111"}, {11, "01000"},
{12, "001000"}, {13, "000011"}, {14, "110100"}, {15, "110101"},
{16, "101010"}, {17, "101011"}, {18, "0100111"}, {19, "0001100"},
{20, "0001000"}, {21, "0010111"}, {22, "0000011"}, {23, "0000100"},
{24, "0101000"}, {25, "0101011"}, {26, "0010011"}, {27, "0100100"},
{28, "0011000"}, {29, "00000010"}, {30, "00000011"}, {31, "00011010"},
{32, "00011011"}, {33, "00010010"}, {34, "00010011"}, {35, "00010100"},
{36, "00010101"}, {37, "00010110"}, {38, "00010111"}, {39, "00101000"},
{40, "00101001"}, {41, "00101010"}, {42, "00101011"}, {43, "00101100"},
{44, "00101101"}, {45, "00000100"}, {46, "00000101"}, {47, "00001010"},
{48, "00001011"}, {49, "01010010"}, {50, "01010011"}, {51, "01010100"},
{52, "01010101"}, {53, "00100100"}, {54, "00100101"}, {55, "01011000"},
{56, "01011001"}, {57, "01011010"}, {58, "01011011"}, {59, "01001010"},
{60, "01001011"}, {61, "00110010"}, {62, "00110011"}, {63, "00110100"},
{64, "11011"}, {128, "10010"}, {192, "010111"}, {256, "0110111"},
{320, "00110110"}, {384, "00110111"}, {448, "01100100"}, {512, "01100101"},
{576, "01101000"}, {640, "01100111"}, {704, "011001100"}, {768, "011001101"},
{832, "011010010"}, {896, "011010011"}, {960, "011010100"}, {1024, "011010101"},
{1088, "011010110"}, {1152, "011010111"}, {1216, "011011000"}, {1280, "011011001"},
{1344, "011011010"}, {1408, "011011011"}, {1472, "010011000"}, {1536, "010011001"},
{1600, "010011010"}, {1664, "011000"}, {1728, "010011011"}, {1792, "00000001000"},
{1856, "00000001100"}, {1920, "00000001101"}, {1984, "000000010010"}, {2048, "000000010011"},
{2112, "000000010100"}, {2176, "000000010101"}, {2240, "000000010110"}, {2304, "000000010111"},
{2368, "000000011100"}, {2432, "000000011101"}, {2496, "000000011110"}, {2560, "000000011111"},
}
// ccittBlackCodes is Tables 2 and 3 of ITU-T T.4 for a black run.
var ccittBlackCodes = []ccittCode{
{0, "0000110111"}, {1, "010"}, {2, "11"}, {3, "10"},
{4, "011"}, {5, "0011"}, {6, "0010"}, {7, "00011"},
{8, "000101"}, {9, "000100"}, {10, "0000100"}, {11, "0000101"},
{12, "0000111"}, {13, "00000100"}, {14, "00000111"}, {15, "000011000"},
{16, "0000010111"}, {17, "0000011000"}, {18, "0000001000"}, {19, "00001100111"},
{20, "00001101000"}, {21, "00001101100"}, {22, "00000110111"}, {23, "00000101000"},
{24, "00000010111"}, {25, "00000011000"}, {26, "000011001010"}, {27, "000011001011"},
{28, "000011001100"}, {29, "000011001101"}, {30, "000001101000"}, {31, "000001101001"},
{32, "000001101010"}, {33, "000001101011"}, {34, "000011010010"}, {35, "000011010011"},
{36, "000011010100"}, {37, "000011010101"}, {38, "000011010110"}, {39, "000011010111"},
{40, "000001101100"}, {41, "000001101101"}, {42, "000011011010"}, {43, "000011011011"},
{44, "000001010100"}, {45, "000001010101"}, {46, "000001010110"}, {47, "000001010111"},
{48, "000001100100"}, {49, "000001100101"}, {50, "000001010010"}, {51, "000001010011"},
{52, "000000100100"}, {53, "000000110111"}, {54, "000000111000"}, {55, "000000100111"},
{56, "000000101000"}, {57, "000001011000"}, {58, "000001011001"}, {59, "000000101011"},
{60, "000000101100"}, {61, "000001011010"}, {62, "000001100110"}, {63, "000001100111"},
{64, "0000001111"}, {128, "000011001000"}, {192, "000011001001"}, {256, "000001011011"},
{320, "000000110011"}, {384, "000000110100"}, {448, "000000110101"}, {512, "0000001101100"},
{576, "0000001101101"}, {640, "0000001001010"}, {704, "0000001001011"}, {768, "0000001001100"},
{832, "0000001001101"}, {896, "0000001110010"}, {960, "0000001110011"}, {1024, "0000001110100"},
{1088, "0000001110101"}, {1152, "0000001110110"}, {1216, "0000001110111"}, {1280, "0000001010010"},
{1344, "0000001010011"}, {1408, "0000001010100"}, {1472, "0000001010101"}, {1536, "0000001011010"},
{1600, "0000001011011"}, {1664, "0000001100100"}, {1728, "0000001100101"}, {1792, "00000001000"},
{1856, "00000001100"}, {1920, "00000001101"}, {1984, "000000010010"}, {2048, "000000010011"},
{2112, "000000010100"}, {2176, "000000010101"}, {2240, "000000010110"}, {2304, "000000010111"},
{2368, "000000011100"}, {2432, "000000011101"}, {2496, "000000011110"}, {2560, "000000011111"},
}
// A ccittTable decodes one of the code tables. Codes are grouped by their
// length, which is the whole of the decoding: read a bit, lengthen the code by
// one, and ask whether a code of that length says anything. A prefix-free code
// cannot answer twice, so the first answer is the only one.
type ccittTable struct {
byLength []map[uint32]int
}
// newCCITTTable groups a table's codes by length. The bits are read from the
// strings the tables are written in, so what the code says and what the
// specification says are the same text.
func newCCITTTable(codes []ccittCode) *ccittTable {
longest := 0
for _, c := range codes {
if len(c.bits) > longest {
longest = len(c.bits)
}
}
t := &ccittTable{byLength: make([]map[uint32]int, longest+1)}
for _, c := range codes {
var v uint32
for _, b := range c.bits {
v <<= 1
if b == '1' {
v |= 1
}
}
n := len(c.bits)
if t.byLength[n] == nil {
t.byLength[n] = map[uint32]int{}
}
t.byLength[n][v] = c.value
}
return t
}
var (
ccittModes = newCCITTTable(ccittModeCodes)
ccittWhite = newCCITTTable(ccittWhiteCodes)
ccittBlack = newCCITTTable(ccittBlackCodes)
)
// A ccittBits reads one bit at a time, most significant first, which is the
// order a fax is written in.
type ccittBits struct {
data []byte
pos int // in bits
}
func (b *ccittBits) next() (uint32, bool) {
if b.pos >= 8*len(b.data) {
return 0, false
}
bit := (b.data[b.pos/8] >> (7 - uint(b.pos%8))) & 1
b.pos++
return uint32(bit), true
}
// align moves to the next byte boundary, which /EncodedByteAlign asks for at
// the start of every row.
func (b *ccittBits) align() { b.pos = (b.pos + 7) &^ 7 }
func (b *ccittBits) exhausted() bool { return b.pos >= 8*len(b.data) }
// restIsFill says whether nothing but zero bits remain. A fax is padded to a
// byte, and often to rather more than a byte, and zeros decode as perfectly
// good two-dimensional modes: pass mode is 0001 and vertical-left-3 is 0000010,
// so a decoder that does not stop here goes on inventing rows that look like
// the last real one. One French form gave 1 636 rows for a 208-row image that
// way, and every one of the surplus rows had plausible ink on it.
func (b *ccittBits) restIsFill() bool {
for i := b.pos; i < 8*len(b.data); i++ {
if b.data[i/8]&(1<<(7-uint(i%8))) != 0 {
return false
}
}
return true
}
// read decodes one code. It returns false at the end of the data or on a code
// the table does not name — the caller decides which of those is fatal, since a
// truncated fax is common and worth showing as far as it goes.
func (t *ccittTable) read(b *ccittBits) (int, bool) {
var code uint32
for n := 1; n < len(t.byLength); n++ {
bit, ok := b.next()
if !ok {
return 0, false
}
code = code<<1 | bit
if v, ok := t.byLength[n][code]; ok {
return v, true
}
}
return 0, false
}
// skipEOL consumes an end-of-line code, 000000000001, if one is next, and any
// fill bits before it. A Group 3 fax may have one at the end of every row, may
// have none at all, and may have several together at the end; all three are
// found in the corpus, so none of them may be an error.
func (b *ccittBits) skipEOL() bool {
start := b.pos
zeros := 0
for {
bit, ok := b.next()
if !ok {
b.pos = start
return false
}
if bit == 0 {
zeros++
continue
}
if zeros >= 11 {
return true
}
b.pos = start
return false
}
}
// ccittParams are the decode parameters of /CCITTFaxDecode, with the defaults
// the specification gives.
type ccittParams struct {
k int // <0 Group 4, 0 Group 3 one-dimensional, >0 Group 3 mixed
columns int // default 1728
rows int // 0 means as many as the data holds
blackIs1 bool // false: a 0 bit is black
encodedByteAlign bool
endOfBlock bool // default true: the data ends with a block marker
}
func ccittParamsOf(parm Dict, r Resolver) ccittParams {
return ccittParams{
k: intParm(parm, "K", 0, r),
columns: intParm(parm, "Columns", 1728, r),
rows: intParm(parm, "Rows", 0, r),
blackIs1: boolParm(parm, "BlackIs1", false, r),
encodedByteAlign: boolParm(parm, "EncodedByteAlign", false, r),
endOfBlock: boolParm(parm, "EndOfBlock", true, r),
}
}
// endOfBlockEOLs is how many end-of-line codes in a row mean the data has
// ended: two for Group 4, which has none between its rows, and six for Group 3,
// where every row may legitimately be followed by one.
func endOfBlockEOLs(k int) int {
if k < 0 {
return 2
}
return 6
}
// maxCCITTPixels bounds what one image may decode to, the way maxDecodedSize
// bounds a Flate stream. A fax names its own width and height in the decode
// parameters, so a file can ask for an image of any size it likes; 2^28 pixels
// is more than four times A4 at 600 dots to the inch.
var maxCCITTPixels = 1 << 28
// ccittDecode turns a fax into bilevel samples: one bit a pixel, each row
// padded to a byte boundary, which is what every other image filter in this
// package produces and what a stream dictionary's /Width, /Height and
// /BitsPerComponent describe.
func ccittDecode(data []byte, p ccittParams) ([]byte, error) {
if p.columns <= 0 {
return nil, fmt.Errorf("reader: /CCITTFaxDecode names %d columns", p.columns)
}
if p.rows < 0 {
return nil, fmt.Errorf("reader: /CCITTFaxDecode names %d rows", p.rows)
}
if p.rows > 0 && p.columns > maxCCITTPixels/p.rows {
return nil, fmt.Errorf("reader: /CCITTFaxDecode names %d by %d pixels, "+
"past the limit of %d", p.columns, p.rows, maxCCITTPixels)
}
b := &ccittBits{data: data}
stride := (p.columns + 7) / 8
curr := make([]byte, p.columns)
prev := []byte(nil)
out := []byte(nil)
for row := 0; p.rows == 0 || row < p.rows; row++ {
if p.rows == 0 && len(out)/stride*p.columns >= maxCCITTPixels {
return nil, fmt.Errorf("reader: /CCITTFaxDecode ran past %d pixels "+
"without naming how many rows it has", maxCCITTPixels)
}
if p.encodedByteAlign {
b.align()
}
// A Group 3 row may be introduced by an end-of-line code. Consuming it
// here rather than after the row means a file that has them and a file
// that does not are read the same way — and counting them is how the
// end of the data announces itself, since the block marker is nothing
// but end-of-line codes in a row.
eols := 0
for b.skipEOL() {
eols++
}
if p.endOfBlock && eols >= endOfBlockEOLs(p.k) {
break
}
if b.exhausted() || b.restIsFill() {
break
}
twoDimensional := p.k < 0
if p.k > 0 {
// In mixed mode a bit after the end-of-line says which the row is:
// 1 for one-dimensional, 0 for two.
// The data was checked for bits a moment ago, so this cannot
// fail; the value is taken rather than the error handled, because
// a branch that cannot be reached cannot be tested.
bit, _ := b.next()
twoDimensional = bit == 0
}
if err := ccittRow(b, curr, prev, twoDimensional); err != nil {
if len(out) == 0 {
return nil, err
}
// A truncated fax is worth showing as far as it got: the rows
// already decoded are real, and refusing them turns a damaged
// scan into a blank page. Leaving the loop rather than returning
// here matters — the padding below is what makes the answer as
// long as /Rows promised, and a fuzz case of two bytes found that
// returning early handed the caller one row where it had asked
// for four, with no error to say so.
break
}
out = append(out, ccittPack(curr, stride, p.blackIs1)...)
if prev == nil {
prev = make([]byte, p.columns)
}
copy(prev, curr)
}
// A file that says how many rows it has gets that many, padded with white
// if the data ran out, so the samples match the /Height the dictionary
// promises rather than being short by a row.
if p.rows > 0 {
white := make([]byte, stride)
if !p.blackIs1 {
for i := range white {
white[i] = 0xFF
}
}
for len(out) < p.rows*stride {
out = append(out, white...)
}
}
return out, nil
}
// ccittPack turns a row of one byte a pixel into one bit a pixel. With
// /BlackIs1 false — the default — a 0 bit is black, so white becomes a 1.
func ccittPack(row []byte, stride int, blackIs1 bool) []byte {
out := make([]byte, stride)
for x, v := range row {
white := v != 0
if white != blackIs1 {
out[x/8] |= 1 << (7 - uint(x%8))
}
}
return out
}
// ccittRow decodes one row into curr, one byte a pixel: 0xFF white, 0x00 black.
func ccittRow(b *ccittBits, curr, prev []byte, twoDimensional bool) error {
for i := range curr {
curr[i] = 0
}
at, white, first := 0, true, true
for at < len(curr) {
if !twoDimensional {
n, err := ccittRun(b, white)
if err != nil {
return err
}
if at+n > len(curr) {
return fmt.Errorf("reader: a fax row of %d pixels named %d",
len(curr), at+n)
}
fill(curr[at:at+n], white)
at += n
white = !white
first = false
continue
}
mode, ok := ccittModes.read(b)
if !ok {
return fmt.Errorf("reader: a fax row named no mode this reader knows")
}
switch mode {
case modePass:
// ccittFindB starts where the pen is and only walks forward,
// stopping at the end of the row, so b2 is always a pixel of this
// row at or after the pen. There is nothing to check.
b2 := ccittFindB(prev, curr, at, white, first, true)
fill(curr[at:b2], white)
at = b2
case modeH:
// Two runs, of the current colour and then the other, neither of
// them looking at the row above.
for i := 0; i < 2; i++ {
n, err := ccittRun(b, white)
if err != nil {
return err
}
if at+n > len(curr) {
return fmt.Errorf("reader: a fax row of %d pixels named %d",
len(curr), at+n)
}
fill(curr[at:at+n], white)
at += n
white = !white
}
// Horizontal mode reads a pair, so the two flips above have put
// the colour back where it started; there is nothing to undo.
case modeV0, modeVR1, modeVR2, modeVR3, modeVL1, modeVL2, modeVL3:
a1 := ccittFindB(prev, curr, at, white, first, false) + ccittOffset(mode)
if a1 < at || a1 > len(curr) {
return fmt.Errorf("reader: a fax vertical mode named pixel %d of %d",
a1, len(curr))
}
fill(curr[at:a1], white)
at = a1
white = !white
default:
return fmt.Errorf("reader: a fax named an extension mode, which " +
"this reader does not decode")
}
first = false
}
return nil
}
// ccittOffset is how far a vertical mode moves a1 from b1.
func ccittOffset(mode int) int {
switch mode {
case modeVR1:
return 1
case modeVR2:
return 2
case modeVR3:
return 3
case modeVL1:
return -1
case modeVL2:
return -2
case modeVL3:
return -3
}
return 0
}
// ccittRun reads one run length, which is a sequence of make-up codes ending in
// a terminating code below 64.
func ccittRun(b *ccittBits, white bool) (int, error) {
table := ccittBlack
if white {
table = ccittWhite
}
total := 0
for {
n, ok := table.read(b)
if !ok {
return 0, fmt.Errorf("reader: a fax run named no length this reader knows")
}
total += n
if total > maxCCITTPixels {
return 0, fmt.Errorf("reader: a fax named a run of %d pixels", total)
}
if n <= 63 {
return total, nil
}
}
}
// ccittFindB finds b1, or b2 when second is set: the changing elements on the
// row above, as T.6 Figure 1 defines them.
func ccittFindB(prev, curr []byte, at int, white, first, second bool) int {
// The row above the first row is implicitly all white, so it has no
// changing elements and both b values are at the end of the row.
if prev == nil {
return len(curr)
}
i := at
if first {
// a0 is implicitly one pixel before the row, on white. b1 is the first
// black pixel above; b2 the first white pixel after that.
for i < len(prev) && prev[i] != 0 {
i++
}
if second {
for i < len(prev) && prev[i] == 0 {
i++
}
}
return i
}
// Walk past the run above that is of the opposite colour to the pen, then
// past the run of the pen's own colour: what follows is b1.
opposite := byte(0xFF)
if white {
opposite = 0
}
for i < len(prev) && prev[i] == opposite {
i++
}
same := ^opposite
for i < len(prev) && prev[i] == same {
i++
}
if second {
for i < len(prev) && prev[i] == opposite {
i++
}
}
return i
}
// boolParm reads a boolean decode parameter, falling back to its default.
func boolParm(parm Dict, key Name, def bool, r Resolver) bool {
if parm == nil {
return def
}
v, err := Resolve(parm.Get(key), r)
if err != nil {
return def
}
if b, ok := v.(Bool); ok {
return bool(b)
}
return def
}
// fill paints a run of pixels, one byte each.
func fill(row []byte, white bool) {
v := byte(0)
if white {
v = 0xFF
}
for i := range row {
row[i] = v
}
}
vendor/github.com/go-pdfkit/reader/content.go
Generated Vendored
+339
View File
@@ -0,0 +1,339 @@
package reader
import (
"bytes"
"fmt"
)
// An Operation is one step of a content stream: an operator and the operands
// that precede it. An inline image is reported as the operator "BI" with its
// dictionary and data attached, since the three keywords it spans are one
// thing.
type Operation struct {
Operator string
Operands []Object
Image *InlineImage
}
// An InlineImage is the dictionary and the still-encoded bytes of a BI … ID …
// EI sequence. Its dictionary uses the abbreviated keys inline images are
// written with; [InlineImage.Expanded] gives the ordinary spelling.
type InlineImage struct {
Dict Dict
Raw []byte
}
// inlineKeys maps the abbreviations an inline image dictionary uses to the
// names their long form would have.
var inlineKeys = map[Name]Name{
"BPC": "BitsPerComponent",
"CS": "ColorSpace",
"D": "Decode",
"DP": "DecodeParms",
"F": "Filter",
"H": "Height",
"IM": "ImageMask",
"I": "Interpolate",
"W": "Width",
"L": "Length",
}
// inlineColourSpaces maps the abbreviated colour space names.
var inlineColourSpaces = map[Name]Name{
"G": "DeviceGray",
"RGB": "DeviceRGB",
"CMYK": "DeviceCMYK",
"I": "Indexed",
}
// Expanded returns the image's dictionary with the abbreviated keys and colour
// space names written out, so it can be read like any image XObject.
//
// A dictionary may say the same thing twice and disagree with itself: /W 20
// beside /Width 10, /CS /RGB beside /ColorSpace /3chanRGB. The specification
// permits both spellings in an inline image and says nothing about which wins,
// and the public implementations disagree — pdf.js asks for the abbreviation
// and falls back to the written-out name, MuPDF asks the other way round.
//
// The abbreviation wins here. Two reasons: it is the spelling Table 93 gives
// for an inline image, the written-out name being the tolerated alias; and
// safedocs' own fixture for this case marks the abbreviation as the line to
// remove "to see the effect", which is a statement about what the reference
// viewer does.
//
// What is not defensible is the answer changing between runs. Expanding in one
// pass over the map made the winner depend on Go's map iteration order, which
// is deliberately random: the same file drew differently each time it was
// opened, with no way for a caller to notice. Two passes settle it — the
// written-out names first, then the abbreviations over them — and no order
// within either pass can matter, because the names in each are distinct.
func (im *InlineImage) Expanded() Dict {
out := Dict{}
for k, v := range im.Dict {
if _, abbreviated := inlineKeys[k]; abbreviated {
continue
}
out[k] = expandColourSpace(k, v)
}
for k, v := range im.Dict {
long, abbreviated := inlineKeys[k]
if !abbreviated {
continue
}
out[long] = expandColourSpace(long, v)
}
return out
}
// expandColourSpace writes out an abbreviated colour space name, which is the
// one value that is abbreviated as well as its key.
func expandColourSpace(k Name, v Object) Object {
if k != "ColorSpace" {
return v
}
n, ok := ToName(v)
if !ok {
return v
}
if long, ok := inlineColourSpaces[n]; ok {
return long
}
return v
}
// A ContentScanner walks a content stream one operation at a time. A stream
// with rubbish in it yields the operations around the rubbish rather than
// nothing at all, which is what a renderer needs; [ContentScanner.Err] reports
// the first thing that did not parse.
type ContentScanner struct {
lex lexer
pending []Object
err error
}
// NewContentScanner reads operations from a decoded content stream.
func NewContentScanner(data []byte) *ContentScanner {
return &ContentScanner{lex: lexer{buf: data}}
}
// Err reports the first malformed token the scan stepped over, if any.
func (s *ContentScanner) Err() error { return s.err }
// Next returns the next operation, and false once the stream is exhausted.
func (s *ContentScanner) Next() (Operation, bool) {
for {
t, err := s.lex.next()
if err != nil {
s.note(err)
// Step over the byte that would not parse and carry on.
if s.lex.pos < len(s.lex.buf) {
s.lex.pos++
continue
}
return Operation{}, false
}
if t.kind == tokEOF {
return Operation{}, false
}
if t.kind != tokKeyword {
s.operand(t)
continue
}
switch string(t.text) {
case "true":
s.pending = append(s.pending, Bool(true))
continue
case "false":
s.pending = append(s.pending, Bool(false))
continue
case "null":
s.pending = append(s.pending, Null{})
continue
case "BI":
img, err := s.inlineImage()
if err != nil {
s.note(err)
return Operation{}, false
}
return s.emit("BI", img), true
}
return s.emit(string(t.text), nil), true
}
}
// operand parses one operand, an already-read token starting it.
func (s *ContentScanner) operand(t token) {
p := &parser{lex: s.lex}
o, err := p.parseFrom(t)
s.lex = p.lex
if err != nil {
s.note(err)
if s.lex.pos < len(s.lex.buf) {
s.lex.pos++
}
return
}
s.pending = append(s.pending, o)
}
// emit finishes an operation and clears the operands waiting for it.
func (s *ContentScanner) emit(op string, img *InlineImage) Operation {
out := Operation{Operator: op, Operands: s.pending, Image: img}
s.pending = nil
return out
}
// note keeps the first error the scan met.
func (s *ContentScanner) note(err error) {
if s.err == nil {
s.err = err
}
}
// inlineImage reads the dictionary after BI and the data after ID.
func (s *ContentScanner) inlineImage() (*InlineImage, error) {
d := Dict{}
for {
t, err := s.lex.next()
if err != nil {
return nil, err
}
if t.kind == tokEOF {
return nil, &SyntaxError{t.pos, "an inline image has no ID"}
}
if t.kind == tokKeyword && string(t.text) == "ID" {
break
}
if t.kind != tokName {
return nil, &SyntaxError{t.pos, "an inline image key is not a name"}
}
p := &parser{lex: s.lex}
v, err := p.parseObject()
s.lex = p.lex
if err != nil {
return nil, err
}
d[Name(t.text)] = v
}
// Exactly one white-space byte separates ID from the data.
if s.lex.pos < len(s.lex.buf) && isSpace(s.lex.buf[s.lex.pos]) {
s.lex.pos++
}
start := s.lex.pos
end, err := inlineImageEnd(s.lex.buf, start, d)
if err != nil {
return nil, err
}
raw := s.lex.buf[start:end]
s.lex.pos = end
// Step over the EI that follows.
for s.lex.pos < len(s.lex.buf) && isSpace(s.lex.buf[s.lex.pos]) {
s.lex.pos++
}
if bytes.HasPrefix(s.lex.buf[s.lex.pos:], []byte("EI")) {
s.lex.pos += 2
}
return &InlineImage{Dict: d, Raw: raw}, nil
}
// Operations tokenises a whole content stream. The error it returns describes
// what did not parse; the operations it returns are still worth having.
func Operations(data []byte) ([]Operation, error) {
s := NewContentScanner(data)
var out []Operation
for {
op, ok := s.Next()
if !ok {
return out, s.Err()
}
out = append(out, op)
}
}
// PageContent returns the decoded content stream of the i'th page, counting
// from one. A page whose /Contents is an array has its streams joined, as the
// specification requires, with a newline between them.
//
// A content stream whose filters cannot be applied contributes what could be
// salvaged from it rather than failing the page, which is what a renderer
// needs; the error is for a page that is not there at all.
// [Document.PageContentDecoded] says whether any salvaging happened.
func (d *Document) PageContent(i int) ([]byte, error) {
dec, err := d.PageContentDecoded(i)
return dec.Data, err
}
// PageContentDecoded is [Document.PageContent] with the outcome of the decode
// attached: [Decoded.Recovered] reports that at least one of the page's content
// streams could not be decoded cleanly, and [Decoded.Cause] says why.
func (d *Document) PageContentDecoded(i int) (Decoded, error) {
page, err := d.Page(i)
if err != nil {
return Decoded{}, err
}
return d.contentOf(page)
}
// contentOf decodes a page's /Contents, whether one stream or several.
func (d *Document) contentOf(page Dict) (Decoded, error) {
o, err := d.Resolve(page.Get("Contents"))
if err != nil {
return Decoded{}, err
}
switch v := o.(type) {
case *Stream:
return d.decodedContent(v), nil
case Array:
var out Decoded
for _, e := range v {
eo, err := d.Resolve(e)
if err != nil {
return Decoded{}, err
}
s, ok := ToStream(eo)
if !ok {
continue
}
part := d.decodedContent(s)
if part.Recovered && !out.Recovered {
out.Recovered, out.Cause, out.Filter = true, part.Cause, part.Filter
out.Undecoded = part.Undecoded
}
if len(part.Data) == 0 {
continue
}
if len(out.Data) > 0 {
out.Data = append(out.Data, '\n')
}
out.Data = append(out.Data, part.Data...)
}
return out, nil
}
return Decoded{}, nil
}
// decodedContent decodes one content stream. An image filter has no business
// being there, so the bytes it holds are not content: they are reported as
// salvage, not handed over as if they could be tokenised.
func (d *Document) decodedContent(s *Stream) Decoded {
dec := d.DecodeStreamRecovering(s)
if dec.Image != "" {
return Decoded{
Undecoded: dec.Data,
Recovered: true,
Filter: dec.Image,
Cause: fmt.Errorf("reader: a content stream is filtered as an image (/%s)", dec.Image),
}
}
return dec
}
// PageOperations tokenises the i'th page's content stream, counting from one.
func (d *Document) PageOperations(i int) ([]Operation, error) {
data, err := d.PageContent(i)
if err != nil {
return nil, err
}
ops, _ := Operations(data)
return ops, nil
}
vendor/github.com/go-pdfkit/reader/crypt.go
Generated Vendored
+572
View File
@@ -0,0 +1,572 @@
package reader
import (
"bytes"
"crypto/aes"
"crypto/cipher"
"crypto/md5"
"crypto/rc4"
"crypto/sha256"
"crypto/sha512"
"errors"
"fmt"
"hash"
)
// ErrWrongPassword is returned when a file is encrypted and neither the
// password given nor the empty one opens it.
var ErrWrongPassword = errors.New("reader: the password does not open this file")
// ErrUnsupportedEncryption is returned for a security handler this package
// does not implement — anything other than the standard one.
var ErrUnsupportedEncryption = errors.New("reader: unsupported security handler")
// pad is the 32-byte string every pre-2.0 password is padded with.
var pad = []byte{
0x28, 0xBF, 0x4E, 0x5E, 0x4E, 0x75, 0x8A, 0x41, 0x64, 0x00, 0x4E, 0x56,
0xFF, 0xFA, 0x01, 0x08, 0x2E, 0x2E, 0x00, 0xB6, 0xD0, 0x68, 0x3E, 0x80,
0x2F, 0x0C, 0xA9, 0xFE, 0x64, 0x53, 0x69, 0x7A,
}
// cryptMethod is how one class of data — strings or streams — is protected.
type cryptMethod uint8
const (
cryptNone cryptMethod = iota // /Identity: not encrypted at all
cryptRC4
cryptAESV2 // 128-bit AES, per-object key
cryptAESV3 // 256-bit AES, the file key used directly
)
// A decryptor holds the file encryption key and how to apply it.
type decryptor struct {
key []byte
strings cryptMethod
streams cryptMethod
revision int
perm Permissions
owner bool // the password given was the owner's, not the user's
skipObj int // the /Encrypt dictionary's own object number, never decrypted
skipKnown bool // whether skipObj is meaningful
}
// A Protection is what a file's security handler says about it: how it is
// encrypted, and what a reader that opened it with the user password may do.
type Protection struct {
// Method names the algorithm the way a person would say it: "RC4-40",
// "RC4-128", "AES-128", "AES-256", or "none" for a file that declares an
// /Encrypt dictionary and then encrypts nothing with it.
Method string
// Revision is the standard security handler's revision: 2, 3, 4, 5 or 6.
Revision int
// Permissions is what the file grants whoever opened it with the user
// password.
Permissions Permissions
// Owner is true when the password the file was opened with was the
// owner's, in which case the permissions do not apply to this reader.
Owner bool
}
// Protection reports how the file is protected, and false when it is not
// protected at all. A document that opened has already been decrypted; this
// says what it said about itself on the way.
func (d *Document) Protection() (Protection, bool) {
if d.decrypt == nil {
return Protection{}, false
}
return Protection{
Method: d.decrypt.methodName(),
Revision: d.decrypt.revision,
Permissions: d.decrypt.perm,
Owner: d.decrypt.owner,
}, true
}
// methodName says which algorithm protects the streams, which is the one that
// matters: it is where the content is.
func (dec *decryptor) methodName() string {
switch dec.streams {
case cryptAESV3:
return "AES-256"
case cryptAESV2:
return "AES-128"
case cryptRC4:
return fmt.Sprintf("RC4-%d", len(dec.key)*8)
}
return "none"
}
// Encrypted reports whether the file declares an /Encrypt dictionary.
func (d *Document) Encrypted() bool {
return d.trailer != nil && d.trailer.Get("Encrypt").Kind() != KindNull
}
// setUpDecryption reads /Encrypt and derives the file key. It is called once,
// before any string or stream is handed out.
func (d *Document) setUpDecryption(password string) error {
if !d.Encrypted() {
return nil
}
if ref, ok := d.trailer.Get("Encrypt").(Ref); ok {
d.encryptNum, d.encryptKnown = ref.Num, true
}
o, err := d.Resolve(d.trailer.Get("Encrypt"))
if err != nil {
return err
}
enc, ok := ToDict(o)
if !ok {
return fmt.Errorf("reader: /Encrypt is a %s, not a dictionary", o.Kind())
}
id := d.firstID()
dec, err := newDecryptor(enc, id, password, d.Get)
if err != nil {
return err
}
dec.skipObj, dec.skipKnown = d.encryptNum, d.encryptKnown
d.decrypt = dec
return nil
}
// firstID returns the first element of the trailer's /ID, which takes part in
// the pre-2.0 key derivation. A file without one contributes nothing.
func (d *Document) firstID() []byte {
arr, ok := ToArray(d.trailer.Get("ID"))
if !ok || len(arr) == 0 {
return nil
}
s, ok := ToString(arr[0])
if !ok {
return nil
}
return s
}
// newDecryptor derives the file encryption key from a password.
func newDecryptor(enc Dict, id []byte, password string, r Resolver) (*decryptor, error) {
if f, ok := ToName(resolved(enc, "Filter", r)); ok && f != "Standard" {
return nil, ErrUnsupportedEncryption
}
v := int(intOf(resolved(enc, "V", r), 0))
rev := int(intOf(resolved(enc, "R", r), 0))
if v == 0 || rev == 0 {
return nil, fmt.Errorf("reader: /Encrypt has no usable /V and /R")
}
length := int(intOf(resolved(enc, "Length", r), 40))
if length < 40 || length > 256 || length%8 != 0 {
length = 40
}
dec := &decryptor{revision: rev}
if err := dec.readMethods(enc, v, r); err != nil {
return nil, err
}
owner, _ := ToString(resolved(enc, "O", r))
user, _ := ToString(resolved(enc, "U", r))
perm := int32(intOf(resolved(enc, "P", r), 0))
metadata := true
if b, ok := ToBool(resolved(enc, "EncryptMetadata", r)); ok {
metadata = b
}
dec.perm = Permissions(uint32(perm)) & AllPermissions
if dec.streams == cryptNone && dec.strings == cryptNone {
// /StmF and /StrF both name the identity crypt filter, so nothing in
// the document body is encrypted and no key is needed to read it. A
// file like that is opened without validating the password: refusing
// it would refuse a document whose every byte is already plain, which
// is what every other reader shows. Only /EFF — the embedded files —
// would need the key, and this reader does not hand those out.
//
// Nothing was authenticated, so dec.owner stays false and Protection
// reports the method as "none": a caller can see exactly what it got.
return dec, nil
}
if rev >= 5 {
key, asOwner, err := deriveKeyR5(enc, password, r)
if err != nil {
return nil, err
}
dec.key, dec.owner = key, asOwner
return dec, nil
}
n := length / 8
if rev == 2 {
n = 5
}
key, asOwner, err := deriveKeyLegacy(password, owner, user, id, perm, n, rev, metadata)
if err != nil {
return nil, err
}
dec.key, dec.owner = key, asOwner
return dec, nil
}
// readMethods works out how strings and streams are protected. Before /V 4
// there is one method for everything; from /V 4 the crypt filters in /CF are
// named by /StmF and /StrF.
func (dec *decryptor) readMethods(enc Dict, v int, r Resolver) error {
if v < 4 {
dec.strings, dec.streams = cryptRC4, cryptRC4
return nil
}
cf, _ := ToDict(resolved(enc, "CF", r))
pick := func(key Name) (cryptMethod, error) {
name, ok := ToName(resolved(enc, key, r))
if !ok || name == "Identity" {
return cryptNone, nil
}
f, ok := ToDict(resolved(cf, name, r))
if !ok {
return cryptNone, nil
}
switch m, _ := ToName(resolved(f, "CFM", r)); m {
case "V2":
return cryptRC4, nil
case "AESV2":
return cryptAESV2, nil
case "AESV3":
return cryptAESV3, nil
case "None":
return cryptNone, nil
default:
return cryptNone, fmt.Errorf("reader: unsupported crypt filter method /%s", m)
}
}
var err error
if dec.streams, err = pick("StmF"); err != nil {
return err
}
dec.strings, err = pick("StrF")
return err
}
// deriveKeyLegacy is the pre-2.0 key derivation, trying the password as the
// user password and then as the owner password.
func deriveKeyLegacy(password string, owner, user, id []byte, perm int32, n, rev int, metadata bool) (key []byte, asOwner bool, err error) {
for _, candidate := range legacyCandidates(password, owner, n, rev) {
k := legacyFileKey(candidate.padded, owner, id, perm, n, rev, metadata)
if legacyUserKeyMatches(k, user, id, rev) {
return k, candidate.owner, nil
}
}
return nil, false, ErrWrongPassword
}
// A legacyCandidate is one padded password to try, and whether reaching it
// meant knowing the owner's rather than the user's.
type legacyCandidate struct {
padded []byte
owner bool
}
// legacyCandidates lists the padded passwords worth trying: the one given, the
// empty one, and the user password the owner password unlocks.
func legacyCandidates(password string, owner []byte, n, rev int) []legacyCandidate {
out := []legacyCandidate{{padded: padPassword([]byte(password))}}
if password != "" {
out = append(out, legacyCandidate{padded: padPassword(nil)})
}
if u := userFromOwner([]byte(password), owner, n, rev); u != nil {
out = append(out, legacyCandidate{padded: u, owner: true})
}
return out
}
// padPassword truncates or pads a password to the 32 bytes the algorithm wants.
func padPassword(p []byte) []byte {
out := make([]byte, 32)
k := copy(out, p)
copy(out[k:], pad)
return out
}
// legacyFileKey is algorithm 2: the file encryption key from a padded password.
func legacyFileKey(padded, owner, id []byte, perm int32, n, rev int, metadata bool) []byte {
h := md5.New()
h.Write(padded)
h.Write(owner)
h.Write([]byte{byte(perm), byte(perm >> 8), byte(perm >> 16), byte(perm >> 24)})
h.Write(id)
if rev >= 4 && !metadata {
h.Write([]byte{0xFF, 0xFF, 0xFF, 0xFF})
}
key := h.Sum(nil)
if rev >= 3 {
for i := 0; i < 50; i++ {
sum := md5.Sum(key[:n])
key = sum[:]
}
}
return key[:n]
}
// legacyUserKeyMatches is algorithm 6: does this key reproduce /U?
func legacyUserKeyMatches(key, user, id []byte, rev int) bool {
if rev == 2 {
return bytes.Equal(rc4Bytes(key, pad), user)
}
h := md5.New()
h.Write(pad)
h.Write(id)
x := rc4Bytes(key, h.Sum(nil))
for i := 1; i <= 19; i++ {
x = rc4Bytes(xorKey(key, i), x)
}
return len(user) >= 16 && bytes.Equal(x, user[:16])
}
// userFromOwner is algorithm 7: recover the padded user password from /O using
// the owner password.
func userFromOwner(password, owner []byte, n, rev int) []byte {
if len(owner) < 32 {
return nil
}
sum := md5.Sum(padPassword(password))
key := sum[:]
if rev >= 3 {
for i := 0; i < 50; i++ {
s := md5.Sum(key)
key = s[:]
}
}
key = key[:n]
if rev == 2 {
return rc4Bytes(key, owner[:32])
}
x := owner[:32]
for i := 19; i >= 0; i-- {
x = rc4Bytes(xorKey(key, i), x)
}
return x
}
// xorKey returns key with every byte exclusive-ored with v, which is how both
// password algorithms walk their nineteen rounds.
func xorKey(key []byte, v int) []byte {
out := make([]byte, len(key))
for i := range key {
out[i] = key[i] ^ byte(v)
}
return out
}
// deriveKeyR5 is the PDF 2.0 derivation, /R 5 and /R 6: the password is
// validated against a salted hash and then unwraps the file key.
func deriveKeyR5(enc Dict, password string, r Resolver) (key []byte, asOwner bool, err error) {
user, _ := ToString(resolved(enc, "U", r))
userE, _ := ToString(resolved(enc, "UE", r))
owner, _ := ToString(resolved(enc, "O", r))
ownerE, _ := ToString(resolved(enc, "OE", r))
rev := int(intOf(resolved(enc, "R", r), 6))
if len(user) < 48 {
return nil, false, fmt.Errorf("reader: /U is %d bytes, not the 48 this revision needs", len(user))
}
pw := []byte(password)
for _, candidate := range [][]byte{pw, nil} {
if candidate == nil && password == "" {
break
}
if key := unlockR5(candidate, user, userE, nil, rev); key != nil {
return key, false, nil
}
if len(owner) >= 48 {
if key := unlockR5(candidate, owner, ownerE, user[:48], rev); key != nil {
return key, true, nil
}
}
}
return nil, false, ErrWrongPassword
}
// unlockR5 checks one password against a 48-byte /U or /O entry and, when it
// matches, unwraps the file key from the matching /UE or /OE.
func unlockR5(password, entry, wrapped, udata []byte, rev int) []byte {
valSalt, keySalt := entry[32:40], entry[40:48]
if !bytes.Equal(hash2B(password, valSalt, udata, rev), entry[:32]) {
return nil
}
if len(wrapped) < 32 {
return nil
}
inter := hash2B(password, keySalt, udata, rev)
// inter is always the 32 bytes hash2B returns, so the cipher cannot refuse it.
block, _ := aes.NewCipher(inter)
out := make([]byte, 32)
cipher.NewCBCDecrypter(block, make([]byte, aes.BlockSize)).CryptBlocks(out, wrapped[:32])
return out
}
// hash2B is algorithm 2.B: SHA-256 for revision 5, and for revision 6 the
// hardening loop that repeatedly encrypts and re-hashes.
func hash2B(password, salt, udata []byte, rev int) []byte {
h := sha256.New()
h.Write(password)
h.Write(salt)
h.Write(udata)
k := h.Sum(nil)
if rev < 6 {
return k
}
for round := 0; ; round++ {
var k1 []byte
one := append(append(append([]byte{}, password...), k...), udata...)
for i := 0; i < 64; i++ {
k1 = append(k1, one...)
}
// k is always 32 bytes, so a 16-byte key is always well formed.
block, _ := aes.NewCipher(k[:16])
e := make([]byte, len(k1)-len(k1)%aes.BlockSize)
cipher.NewCBCEncrypter(block, k[16:32]).CryptBlocks(e, k1[:len(e)])
sum := 0
for _, b := range e[:16] {
sum += int(b)
}
var next hash.Hash
switch sum % 3 {
case 0:
next = sha256.New()
case 1:
next = sha512.New384()
default:
next = sha512.New()
}
next.Write(e)
k = next.Sum(nil)
if round >= 63 && int(e[len(e)-1]) <= round-31 {
break
}
}
return k[:32]
}
// objectKey is the per-object key the pre-2.0 methods use; AESV3 uses the file
// key unchanged.
func (dec *decryptor) objectKey(num, gen int, method cryptMethod) []byte {
if method == cryptAESV3 {
return dec.key
}
h := md5.New()
h.Write(dec.key)
h.Write([]byte{byte(num), byte(num >> 8), byte(num >> 16), byte(gen), byte(gen >> 8)})
if method == cryptAESV2 {
h.Write([]byte{0x73, 0x41, 0x6C, 0x54}) // "sAlT"
}
k := h.Sum(nil)
if n := len(dec.key) + 5; n < 16 {
return k[:n]
}
return k
}
// decryptBytes undoes one method over one object's data.
func (dec *decryptor) decryptBytes(num, gen int, method cryptMethod, data []byte) []byte {
switch method {
case cryptNone:
return data
case cryptRC4:
return rc4Bytes(dec.objectKey(num, gen, method), data)
default:
return aesDecrypt(dec.objectKey(num, gen, method), data)
}
}
// rc4Bytes applies RC4, which is its own inverse. A key length the cipher
// refuses yields nil, which every caller reads as a password that does not
// fit — the derivations here never produce one.
func rc4Bytes(key, data []byte) []byte {
c, err := rc4.NewCipher(key)
if err != nil {
return nil
}
out := make([]byte, len(data))
c.XORKeyStream(out, data)
return out
}
// aesDecrypt undoes AES in CBC mode, the initialisation vector being the first
// block of the data and the padding the one CBC calls for.
func aesDecrypt(key, data []byte) []byte {
if len(data) < 2*aes.BlockSize || len(data)%aes.BlockSize != 0 {
return nil
}
block, err := aes.NewCipher(key)
if err != nil {
return nil
}
out := make([]byte, len(data)-aes.BlockSize)
cipher.NewCBCDecrypter(block, data[:aes.BlockSize]).CryptBlocks(out, data[aes.BlockSize:])
n := int(out[len(out)-1])
if n < 1 || n > aes.BlockSize || n > len(out) {
return out
}
return out[:len(out)-n]
}
// decryptObject walks an object, decrypting every string in it and, for a
// stream, its data. The /Encrypt dictionary itself, and any cross-reference
// stream, are left alone: neither is encrypted.
func (dec *decryptor) decryptObject(num, gen int, o Object) Object {
if dec.skipKnown && num == dec.skipObj {
return o
}
return dec.walk(num, gen, o)
}
// walk is decryptObject's recursion.
func (dec *decryptor) walk(num, gen int, o Object) Object {
switch v := o.(type) {
case String:
return String(dec.decryptBytes(num, gen, dec.strings, v))
case Array:
for i, e := range v {
v[i] = dec.walk(num, gen, e)
}
return v
case Dict:
for k, e := range v {
v[k] = dec.walk(num, gen, e)
}
return v
case *Stream:
if t, ok := ToName(v.Dict.Get("Type")); ok && t == "XRef" {
return v
}
v.Dict, _ = ToDict(dec.walk(num, gen, v.Dict))
if streamIsPlain(v.Dict) {
return v
}
v.Raw = dec.decryptBytes(num, gen, dec.streams, v.Raw)
return v
}
return o
}
// streamIsPlain reports whether a stream's own filter chain says its bytes were
// left unencrypted: a leading /Crypt filter naming /Identity, which is what
// /Crypt with no /Name means too. Producers use it for the metadata stream of a
// file whose /EncryptMetadata is false, and decrypting such a stream turns
// readable XML into noise.
func streamIsPlain(d Dict) bool {
if first, ok := firstFilter(d); !ok || first != "Crypt" {
return false
}
name, ok := ToName(firstDecodeParms(d).Get("Name"))
return !ok || name == "Identity"
}
// resolved is a helper for reading an /Encrypt entry that may be indirect.
func resolved(d Dict, key Name, r Resolver) Object {
o, err := Resolve(d.Get(key), r)
if err != nil {
return Null{}
}
return o
}
// intOf reads an integer with a default.
func intOf(o Object, def int64) int64 {
if n, ok := ToInt(o); ok {
return n
}
return def
}
vendor/github.com/go-pdfkit/reader/document.go
Generated Vendored
+431
View File
@@ -0,0 +1,431 @@
package reader
import (
"bytes"
"fmt"
)
// maxPageTreeNodes bounds a page tree walk. /Count is a claim, not a fact, and
// /Kids can be made to point back at an ancestor.
const maxPageTreeNodes = 1 << 20
// inheritable lists the page attributes a node may take from an ancestor.
var inheritable = []Name{"Resources", "MediaBox", "CropBox", "Rotate"}
// A Document is a parsed PDF file: its cross-reference information, its
// trailer, and lazy access to every object in it.
type Document struct {
buf []byte
xref map[int]xrefEntry
trailer Dict
cache map[int]Object
loading map[int]bool
objStms map[int]map[int]Object
pages []Ref
password string
decrypt *decryptor
encryptNum int
encryptKnown bool
repaired bool
}
// Open parses the cross-reference information of a PDF file held in memory,
// using the empty password. A file whose tables are damaged is rebuilt by
// scanning it, so Open fails only when there is no usable document structure
// at all, or when the file is encrypted and the empty password does not open
// it — [OpenWithPassword] then does.
func Open(b []byte) (*Document, error) { return OpenWithPassword(b, "") }
// OpenWithPassword is Open with a password to try. The password is tried both
// as the user password and as the owner password, the empty one as well.
func OpenWithPassword(b []byte, password string) (*Document, error) {
d := &Document{
buf: b,
xref: map[int]xrefEntry{},
cache: map[int]Object{},
loading: map[int]bool{},
objStms: map[int]map[int]Object{},
password: password,
}
err := d.loadXref()
if err == nil {
if derr := d.setUpDecryption(password); derr != nil {
return nil, derr
}
d.cache = map[int]Object{}
cerr := error(nil)
if _, cerr = d.Catalog(); cerr == nil {
return d, nil
}
// Tables that parse but lead nowhere are worse than none: rebuild.
// What the catalogue lookup actually said is kept: "no page tree" and
// "/Root is a null" send a reader to different places.
err = fmt.Errorf("reader: the cross-reference tables do not lead to a catalogue: %w", cerr)
}
if rerr := d.repair(); rerr != nil {
// Both diagnoses matter, and the rebuild's is the one about the file
// as it really is: it has read every object header in it, where the
// tables only failed to be read. Returning the tables' error alone
// reported "no startxref" for a file that turned out to hold no
// objects at all, or an encrypted one whose catalogue could not be
// reached — a different problem with a different answer.
return nil, fmt.Errorf("%w (the cross-reference information was unusable too: %v)", rerr, err)
}
// Setting the key up again here used to be necessary, because the rebuild
// ran before anything knew the file was encrypted. repair() establishes it
// itself now, so by this point it is either known or not needed.
return d, nil
}
// Trailer returns the file's trailer dictionary, the newest value of every key
// across the chain of cross-reference sections.
func (d *Document) Trailer() Dict { return d.trailer }
// Repaired reports whether the document's structure was rebuilt by scanning
// the file rather than read from its cross-reference tables.
func (d *Document) Repaired() bool { return d.repaired }
// Resolver returns the function that turns this document's indirect references
// into objects, for [Resolve] and [Decode].
func (d *Document) Resolver() Resolver { return d.Get }
// Resolve follows indirect references until it reaches a direct object.
func (d *Document) Resolve(o Object) (Object, error) { return Resolve(o, d.Get) }
// GetDict resolves an entry of a dictionary to a dictionary.
func (d *Document) GetDict(from Dict, key Name) (Dict, bool) {
o, err := d.Resolve(from.Get(key))
if err != nil {
return nil, false
}
return ToDict(o)
}
// Get returns the object an indirect reference names, or [Null] when the file
// does not define it.
func (d *Document) Get(ref Ref) (Object, error) {
if o, ok := d.cache[ref.Num]; ok {
return o, nil
}
e, ok := d.xref[ref.Num]
if !ok || e.kind == 'f' {
return Null{}, nil
}
if d.loading[ref.Num] {
// An object stream whose own /Length or /Filter points back at an
// object inside itself would otherwise recurse forever.
return Null{}, nil
}
d.loading[ref.Num] = true
defer delete(d.loading, ref.Num)
var obj Object
var err error
if e.kind == 'o' {
obj, err = d.getFromObjectStream(ref.Num, e)
} else {
obj, err = d.getAtOffset(ref.Num, e)
}
if err != nil {
return nil, err
}
d.cache[ref.Num] = obj
return obj, nil
}
// getAtOffset reads an object written directly in the file. An offset that
// does not hold the expected object number means the tables are wrong, which
// is common enough that it triggers a rebuild rather than an error.
func (d *Document) getAtOffset(num int, e xrefEntry) (Object, error) {
if e.offset < 0 || e.offset >= int64(len(d.buf)) {
return d.retryAfterRepair(num, fmt.Errorf("reader: object %d is at offset %d, outside the file", num, e.offset))
}
got, obj, _, err := ParseIndirectObject(d.buf[e.offset:], d.Get)
if err != nil {
return d.retryAfterRepair(num, err)
}
if got.Num != num {
return d.retryAfterRepair(num, fmt.Errorf("reader: offset %d holds object %d, not %d", e.offset, got.Num, num))
}
if d.decrypt != nil {
obj = d.decrypt.decryptObject(num, got.Gen, obj)
}
return obj, nil
}
// retryAfterRepair rebuilds the tables once and looks the object up again.
func (d *Document) retryAfterRepair(num int, cause error) (Object, error) {
if d.repaired {
return Null{}, nil
}
if err := d.repair(); err != nil {
return nil, cause
}
e, ok := d.xref[num]
if !ok || e.kind != 'n' || e.offset >= int64(len(d.buf)) {
return Null{}, nil
}
got, obj, _, err := ParseIndirectObject(d.buf[e.offset:], d.Get)
if err != nil || got.Num != num {
return Null{}, nil
}
if d.decrypt != nil {
obj = d.decrypt.decryptObject(num, got.Gen, obj)
}
return obj, nil
}
// getFromObjectStream reads an object held inside a /Type /ObjStm stream.
func (d *Document) getFromObjectStream(num int, e xrefEntry) (Object, error) {
objs, err := d.objectStream(e.strmNum)
if err != nil {
return nil, err
}
if o, ok := objs[num]; ok {
return o, nil
}
// The index is a hint; a stream that does not hold the object simply
// does not define it.
return Null{}, nil
}
// objectStream parses an object stream once and keeps its contents.
func (d *Document) objectStream(num int) (map[int]Object, error) {
if objs, ok := d.objStms[num]; ok {
return objs, nil
}
objs := map[int]Object{}
d.objStms[num] = objs
o, err := d.Get(Ref{Num: num})
if err != nil {
return objs, err
}
s, ok := ToStream(o)
if !ok {
return objs, nil
}
// An object stream that stops in the middle still holds whole objects in
// the part that did decode, and losing them loses pages.
dec := d.DecodeStreamRecovering(s)
if dec.Image != "" {
return objs, nil
}
data := dec.Data
n := int(intOr(s.Dict.Get("N"), 0))
first := int(intOr(s.Dict.Get("First"), 0))
if n <= 0 || first <= 0 || first > len(data) {
return objs, nil
}
l := &lexer{buf: data[:first]}
type pair struct{ num, off int }
pairs := make([]pair, 0, n)
for i := 0; i < n; i++ {
t1, err1 := l.next()
t2, err2 := l.next()
if err1 != nil || err2 != nil || t1.kind != tokInteger || t2.kind != tokInteger {
break
}
pairs = append(pairs, pair{int(t1.i), first + int(t2.i)})
}
for _, p := range pairs {
if p.off < 0 || p.off > len(data) {
continue
}
obj, _, err := ParseObject(data[p.off:])
if err != nil {
continue
}
objs[p.num] = obj
}
return objs, nil
}
// intOr resolves nothing; it reads a direct integer or returns a default.
func intOr(o Object, def int64) int64 {
if n, ok := ToInt(o); ok {
return n
}
return def
}
// DecodeStream applies a stream's filter chain, resolving any indirect decode
// parameters against this document. A chain that cannot be run to the end is
// an error; [Document.DecodeStreamRecovering] salvages instead.
func (d *Document) DecodeStream(s *Stream) ([]byte, Name, error) {
return Decode(s.Dict, s.Raw, d.Get)
}
// DecodeStreamRecovering applies a stream's filter chain, salvaging what it can
// from a chain that cannot be run to the end and saying that it did so.
func (d *Document) DecodeStreamRecovering(s *Stream) Decoded {
return DecodeRecovering(s.Dict, s.Raw, d.Get)
}
// Catalog returns the document catalogue the trailer's /Root names.
func (d *Document) Catalog() (Dict, error) {
if d.trailer == nil {
return nil, fmt.Errorf("reader: the file has no trailer")
}
o, err := d.Resolve(d.trailer.Get("Root"))
if err != nil {
return nil, err
}
cat, ok := ToDict(o)
if !ok {
return nil, fmt.Errorf("reader: /Root is a %s, not a dictionary", o.Kind())
}
if _, ok := d.GetDict(cat, "Pages"); !ok {
return nil, fmt.Errorf("reader: the catalogue has no page tree")
}
return cat, nil
}
// PageCount reports how many pages the document has.
func (d *Document) PageCount() int { return len(d.pageRefs()) }
// PageRef returns the reference of the i'th page, counting from one.
func (d *Document) PageRef(i int) (Ref, bool) {
refs := d.pageRefs()
if i < 1 || i > len(refs) {
return Ref{}, false
}
return refs[i-1], true
}
// Page returns the i'th page's dictionary, counting from one, with the
// attributes it inherits from its ancestors filled in.
func (d *Document) Page(i int) (Dict, error) {
ref, ok := d.PageRef(i)
if !ok {
return nil, fmt.Errorf("reader: page %d is out of range (the document has %d)", i, d.PageCount())
}
o, err := d.Get(ref)
if err != nil {
return nil, err
}
page, ok := ToDict(o)
if !ok {
return nil, fmt.Errorf("reader: page %d is a %s, not a dictionary", i, o.Kind())
}
return d.withInherited(page), nil
}
// withInherited copies the page and fills in what its ancestors provide.
func (d *Document) withInherited(page Dict) Dict {
out := Dict{}
for k, v := range page {
out[k] = v
}
node := page
for depth := 0; depth < maxRefDepth; depth++ {
missing := false
for _, k := range inheritable {
if _, ok := out[k]; !ok {
missing = true
}
}
if !missing {
break
}
parent, ok := d.GetDict(node, "Parent")
if !ok {
break
}
for _, k := range inheritable {
if _, ok := out[k]; ok {
continue
}
if v, ok := parent[k]; ok {
out[k] = v
}
}
node = parent
}
return out
}
// pageRefs walks the page tree once, in document order.
func (d *Document) pageRefs() []Ref {
if d.pages != nil {
return d.pages
}
d.pages = []Ref{}
cat, err := d.Catalog()
if err != nil {
return d.pages
}
root := cat["Pages"]
seen := map[Ref]bool{}
d.walkPages(root, seen, 0)
if len(d.pages) == 0 {
// A tree that yields nothing but a catalogue that names one page node
// is still a one-page document.
d.pages = d.scanForPages()
}
return d.pages
}
// walkPages appends the leaves of a page tree node in order.
func (d *Document) walkPages(node Object, seen map[Ref]bool, depth int) {
if depth > 64 || len(d.pages) >= maxPageTreeNodes {
return
}
ref, isRef := node.(Ref)
if isRef {
if seen[ref] {
return
}
seen[ref] = true
}
o, err := d.Resolve(node)
if err != nil {
return
}
dict, ok := ToDict(o)
if !ok {
return
}
kids, hasKids := d.kidsOf(dict)
if !hasKids {
if isRef {
d.pages = append(d.pages, ref)
}
return
}
for _, kid := range kids {
d.walkPages(kid, seen, depth+1)
}
}
// kidsOf reports a node's /Kids when it is an interior node of the page tree.
func (d *Document) kidsOf(node Dict) (Array, bool) {
if t, ok := ToName(node.Get("Type")); ok && t == "Page" {
return nil, false
}
o, err := d.Resolve(node.Get("Kids"))
if err != nil {
return nil, false
}
kids, ok := ToArray(o)
return kids, ok
}
// Version reports the PDF version the header declares, "1.7" and the like.
func (d *Document) Version() string {
i := bytes.Index(d.buf, []byte("%PDF-"))
if i < 0 || i+8 > len(d.buf) {
return ""
}
return string(bytes.TrimRight(d.buf[i+5:i+8], "\r\n \t"))
}
// scanForPages is the last resort for a page tree that resolves to nothing:
// every object in the file that calls itself a page, in object-number order.
func (d *Document) scanForPages() []Ref {
var out []Ref
for _, num := range d.objectsOfType("Page") {
out = append(out, Ref{Num: num})
}
return out
}
vendor/github.com/go-pdfkit/reader/encrypt.go
Generated Vendored
+365
View File
@@ -0,0 +1,365 @@
package reader
import (
"bytes"
"crypto/aes"
"crypto/cipher"
"crypto/md5"
"crypto/rand"
"fmt"
"strings"
)
// Permissions say what a reader may do with a file it can open with the user
// password. They are advisory — nothing enforces them but the reader's own
// good manners — and a file opened with the owner password ignores them
// entirely.
type Permissions uint32
// The permissions a standard security handler can express. Anything not
// granted is refused; [AllPermissions] grants everything.
const (
PermPrint Permissions = 1 << 2
PermModify Permissions = 1 << 3
PermCopy Permissions = 1 << 4
PermAnnotate Permissions = 1 << 5
PermFillForms Permissions = 1 << 8
PermExtract Permissions = 1 << 9 // for accessibility
PermAssemble Permissions = 1 << 10
PermPrintFaithful Permissions = 1 << 11 // at full resolution
// AllPermissions grants every one of them.
AllPermissions = PermPrint | PermModify | PermCopy | PermAnnotate |
PermFillForms | PermExtract | PermAssemble | PermPrintFaithful
)
// permissionNames pairs each permission with what it lets a reader do, in the
// order a person would want to read them.
var permissionNames = []struct {
bit Permissions
name string
}{
{PermPrint, "print"},
{PermPrintFaithful, "print at full resolution"},
{PermModify, "modify"},
{PermAssemble, "assemble"},
{PermCopy, "copy"},
{PermExtract, "extract for accessibility"},
{PermAnnotate, "annotate"},
{PermFillForms, "fill in forms"},
}
// Allows reports whether every one of the given permissions is granted.
func (p Permissions) Allows(want Permissions) bool { return p&want == want }
// String lists what is granted, in words.
func (p Permissions) String() string {
var out []string
for _, e := range permissionNames {
if p.Allows(e.bit) {
out = append(out, e.name)
}
}
if len(out) == 0 {
return "nothing"
}
return strings.Join(out, ", ")
}
// permissionBase is the pattern of reserved bits the specification requires
// around the permissions themselves.
const permissionBase uint32 = 0xFFFFF0C0
// An Encryption says how a file is to be protected.
//
// Two people can open it: whoever knows the user password, subject to the
// permissions, and whoever knows the owner password, subject to nothing. An
// empty user password means the file opens without one and the permissions are
// all it says.
type Encryption struct {
UserPassword string
OwnerPassword string
Permissions Permissions
// AES128 asks for the older method, which readers before 2008 understand.
// The default is AES-256, which is what a file should use today.
AES128 bool
}
// Encrypt protects everything written from here on. It must be called before
// anything is written, and the file it produces is not byte-for-byte
// reproducible: encryption needs randomness, by design.
func (w *Writer) Encrypt(e Encryption) {
if len(w.offsets) > 0 || len(w.pending) > 0 {
w.note(fmt.Errorf("reader: a file cannot be encrypted after it has been written to"))
return
}
enc, err := newEncrypter(e)
if err != nil {
w.note(err)
return
}
w.encrypt = enc
}
// An encrypter holds the file key and the dictionary that describes it.
type encrypter struct {
key []byte
method cryptMethod
dict Dict
id []byte
err error
// number is the object the /Encrypt dictionary is written under, which is
// the one thing in the file that is never encrypted.
number int
}
// newEncrypter derives a file key and builds the dictionary that lets a reader
// derive it again from a password.
func newEncrypter(e Encryption) (*encrypter, error) {
perm := permissionBase | uint32(e.Permissions)
id, err := randomBytes(16)
if err != nil {
return nil, err
}
if e.AES128 {
return newLegacyEncrypter(e, perm, id)
}
return newAES256Encrypter(e, perm, id)
}
// newAES256Encrypter builds the revision 6 form: a random file key, wrapped
// twice, once for each password.
func newAES256Encrypter(e Encryption, perm uint32, id []byte) (*encrypter, error) {
key, err := randomBytes(32)
if err != nil {
return nil, err
}
// An owner password nobody set is the user's: leaving it empty would
// mean anything at all opens the file with every permission, which is
// the opposite of what asking for encryption means.
ownerPassword := e.OwnerPassword
if ownerPassword == "" {
ownerPassword = e.UserPassword
}
user, userE, err := wrapKey([]byte(e.UserPassword), key, nil)
if err != nil {
return nil, err
}
owner, ownerE, err := wrapKey([]byte(ownerPassword), key, user)
if err != nil {
return nil, err
}
perms, err := permsEntry(key, perm)
if err != nil {
return nil, err
}
return &encrypter{
key: key,
method: cryptAESV3,
id: id,
dict: Dict{
"Filter": Name("Standard"),
"V": Integer(5),
"R": Integer(6),
"Length": Integer(256),
"P": Integer(int32(perm)),
"U": String(user),
"UE": String(userE),
"O": String(owner),
"OE": String(ownerE),
"Perms": String(perms),
"CF": Dict{"StdCF": Dict{"CFM": Name("AESV3"), "Length": Integer(32)}},
"StmF": Name("StdCF"),
"StrF": Name("StdCF"),
"EncryptMetadata": Bool(true),
},
}, nil
}
// wrapKey builds one 48-byte password entry and the 32 bytes that hold the
// file key wrapped for it.
func wrapKey(password, key, udata []byte) (entry, wrapped []byte, err error) {
salts, err := randomBytes(16)
if err != nil {
return nil, nil, err
}
validation, keySalt := salts[:8], salts[8:]
entry = append(append(hash2B(password, validation, udata, 6), validation...), keySalt...)
// hash2B always returns 32 bytes, so the cipher cannot refuse the key.
block, _ := aes.NewCipher(hash2B(password, keySalt, udata, 6))
wrapped = make([]byte, 32)
cipher.NewCBCEncrypter(block, make([]byte, aes.BlockSize)).CryptBlocks(wrapped, key)
return entry, wrapped, nil
}
// permsEntry is the block that lets a reader check the permissions have not
// been tampered with: they are encrypted with the file key itself.
func permsEntry(key []byte, perm uint32) ([]byte, error) {
tail, err := randomBytes(4)
if err != nil {
return nil, err
}
plain := []byte{
byte(perm), byte(perm >> 8), byte(perm >> 16), byte(perm >> 24),
0xFF, 0xFF, 0xFF, 0xFF,
'T', // metadata is encrypted too
'a', 'd', 'b', // the specification's own marker
tail[0], tail[1], tail[2], tail[3],
}
// The file key is 32 bytes by construction.
block, _ := aes.NewCipher(key)
out := make([]byte, 16)
// One block, with no chaining: this is the one place the format uses it.
block.Encrypt(out, plain)
return out, nil
}
// newLegacyEncrypter builds the revision 4 form, whose key comes from the
// passwords rather than the other way round.
func newLegacyEncrypter(e Encryption, perm uint32, id []byte) (*encrypter, error) {
const n = 16 // 128 bits
owner := legacyOwnerValue([]byte(e.OwnerPassword), []byte(e.UserPassword), n)
key := legacyFileKey(padPassword([]byte(e.UserPassword)), owner, id, int32(perm), n, 4, true)
user := legacyUserValue(key, id)
return &encrypter{
key: key,
method: cryptAESV2,
id: id,
dict: Dict{
"Filter": Name("Standard"),
"V": Integer(4),
"R": Integer(4),
"Length": Integer(128),
"P": Integer(int32(perm)),
"O": String(owner),
"U": String(user),
"CF": Dict{"StdCF": Dict{"CFM": Name("AESV2"), "Length": Integer(16)}},
"StmF": Name("StdCF"),
"StrF": Name("StdCF"),
"EncryptMetadata": Bool(true),
},
}, nil
}
// legacyOwnerValue is algorithm 3: the /O entry, which holds the user password
// encrypted under the owner's.
func legacyOwnerValue(ownerPassword, userPassword []byte, n int) []byte {
if len(ownerPassword) == 0 {
ownerPassword = userPassword
}
sum := md5Sum(padPassword(ownerPassword))
key := sum
for i := 0; i < 50; i++ {
key = md5Sum(key)
}
key = key[:n]
x := padPassword(userPassword)
for i := 0; i <= 19; i++ {
x = rc4Bytes(xorKey(key, i), x)
}
return x
}
// legacyUserValue is algorithm 5: the /U entry.
func legacyUserValue(key, id []byte) []byte {
x := rc4Bytes(key, md5Sum(append(append([]byte{}, pad...), id...)))
for i := 1; i <= 19; i++ {
x = rc4Bytes(xorKey(key, i), x)
}
return append(x, make([]byte, 16)...)
}
// randomSource is where randomness comes from. It is a variable so a test
// can take it away and see that a file is refused rather than written
// without the protection it was asked for.
var randomSource = rand.Read
// randomBytes is the only source of randomness here, and the reason an
// encrypted file is not reproducible.
func randomBytes(n int) ([]byte, error) {
out := make([]byte, n)
if _, err := randomSource(out); err != nil {
return nil, fmt.Errorf("reader: no randomness available: %w", err)
}
return out, nil
}
// apply encrypts one object's strings and, for a stream, its data.
func (e *encrypter) apply(num, gen int, o Object) Object {
switch v := o.(type) {
case String:
return String(e.encryptBytes(num, gen, v))
case Array:
out := make(Array, len(v))
for i, x := range v {
out[i] = e.apply(num, gen, x)
}
return out
case Dict:
out := Dict{}
for k, x := range v {
out[k] = e.apply(num, gen, x)
}
return out
case *Stream:
out := &Stream{Dict: Dict{}, Raw: e.encryptBytes(num, gen, v.Raw)}
for k, x := range v.Dict {
out.Dict[k] = e.apply(num, gen, x)
}
return out
}
return o
}
// encryptBytes protects one object's bytes.
func (e *encrypter) encryptBytes(num, gen int, data []byte) []byte {
dec := &decryptor{key: e.key}
key := dec.objectKey(num, gen, e.method)
iv, err := randomBytes(aes.BlockSize)
if err != nil {
// Writing the bytes as they are would hand out in the clear what was
// asked to be protected, so the failure is remembered and the whole
// file is refused.
e.note(err)
return data
}
// The file key is 32 bytes by construction.
block, _ := aes.NewCipher(key)
// The padding CBC calls for, always at least one whole block of it.
n := aes.BlockSize - len(data)%aes.BlockSize
padded := append(append([]byte{}, data...), bytes.Repeat([]byte{byte(n)}, n)...)
out := make([]byte, len(padded))
cipher.NewCBCEncrypter(block, iv).CryptBlocks(out, padded)
return append(iv, out...)
}
// md5Sum is the hash the pre-2.0 handlers are built on.
func md5Sum(b []byte) []byte {
sum := md5.Sum(b)
return sum[:]
}
// writeEncryptDict writes the /Encrypt dictionary and returns the trailer
// entries a reader needs to find and use it. The dictionary is the one object
// in the file that is never encrypted, since it says how to decrypt the rest.
func (w *Writer) writeEncryptDict() Dict {
if w.encrypt == nil {
return nil
}
ref := w.Reserve()
w.encrypt.number = ref.Num
w.writeInlineRaw(ref, w.encrypt.dict)
// The identifier takes part in the older key derivation and has to be in
// the trailer for a reader to reach it.
id := String(w.encrypt.id)
return Dict{"Encrypt": ref, "ID": Array{id, id}}
}
// note keeps the first thing that went wrong while encrypting, so a file that
// could not be protected is refused rather than written in the clear.
func (e *encrypter) note(err error) {
if e.err == nil {
e.err = err
}
}
vendor/github.com/go-pdfkit/reader/filter.go
Generated Vendored
+475
View File
@@ -0,0 +1,475 @@
package reader
import (
"bytes"
"compress/flate"
"compress/zlib"
"fmt"
"io"
)
// maxDecodedSize caps what one stream may expand to. A few hundred bytes of
// Flate can name gigabytes, and a reader that runs in a browser tab must
// refuse rather than fill the heap. Tests lower it.
var maxDecodedSize int64 = 1 << 30
// ImageFilter reports whether a filter yields an encoded image rather than a
// byte stream. [Decode] stops at one of these and hands the caller the still
// encoded bytes, because decoding them is an image decoder's job.
//
// /CCITTFaxDecode was on this list and is not any more. It does not carry an
// image with its own idea of how many components it has and how deep they are,
// the way DCT and JPX do: it produces bilevel samples, one bit a pixel, and the
// stream dictionary says what they mean. That is a byte stream, so it is a
// filter — and decoding it here means every caller gets it rather than each
// writing its own. See ccitt.go.
func ImageFilter(n Name) bool {
switch n {
case "DCTDecode", "DCT", "JPXDecode", "JBIG2Decode":
return true
}
return false
}
// A Decoded is the outcome of applying a stream's filter chain, including the
// outcome of a chain that could not be finished.
//
// The point of the type is that a caller can tell the three outcomes apart
// without having to be careful. Bytes a filter decoded and bytes no filter
// decoded arrive in different fields, so the second kind cannot be painted as
// samples or tokenised as content by a caller that forgot to check a flag.
type Decoded struct {
// Data is what the chain decoded. A recovered decode leaves fewer bytes
// here than the stream meant to carry, but they are bytes of the kind it
// meant to carry: Data never holds bytes that no filter has decoded.
//
// The one case where Data is still encoded is the deliberate one: a chain
// that stopped at an image filter, which Image names and Recovered does
// not flag, because stopping there is the contract.
Data []byte
// Undecoded holds the bytes the chain could not get past, still in the
// encoding Filter names. It is set instead of Data — never beside it —
// when the filter that failed produced nothing at all.
//
// This is where a compressed content stream, a truncated fax, or a filter
// nobody implements ends up. A caller that wants the stream as it lies has
// to ask for it by a name that says what it is; a caller that reads Data
// cannot be handed it by accident. Painting undecoded bytes as a one-bit
// image looks like a page with something on it, which is why nothing about
// this is left to a flag.
Undecoded []byte
// Image names the image filter the chain stopped at, when it stopped at
// one; Data is then the bytes still encoded in it.
Image Name
// Recovered says the chain could not be run to the end. A caller that must
// not act on damaged data stops here — or calls [Decode], which refuses
// outright.
Recovered bool
// Cause says why the chain stopped, and is set exactly when Recovered is.
Cause error
// Filter names the filter that could not be applied, when one is to blame:
// a chain whose /Filter entry itself is unreadable blames nothing.
Filter Name
}
// DecodeRecovering applies a stream dictionary's filter chain to raw and never
// fails. A filter that cannot be applied — corrupt Flate data, a stream that
// stops in the middle, a filter name nobody implements — ends the chain, and
// what the filters before it produced is returned with [Decoded.Recovered]
// set. That is deliberately what every other reader does: refusing the stream
// loses a page the file can still show.
//
// The salvage is always as far down the chain as the filters got, never the
// bytes as they arrived: a damaged Flate stream yields the prefix it did
// inflate, in [Decoded.Data]. A filter that produced nothing at all leaves the
// bytes in [Decoded.Undecoded] instead, still in its encoding, because that is
// what they are.
func DecodeRecovering(d Dict, raw []byte, resolve Resolver) Decoded {
filters, parms, err := filterChain(d, resolve)
if err != nil {
return Decoded{Undecoded: raw, Recovered: true, Cause: err}
}
data := raw
for i, f := range filters {
if ImageFilter(f) {
return Decoded{Data: data, Image: f}
}
out, err := applyFilter(f, data, parms[i], resolve)
if err != nil {
if len(out) == 0 {
return Decoded{Undecoded: data, Recovered: true, Cause: err, Filter: f}
}
return Decoded{Data: out, Recovered: true, Cause: err, Filter: f}
}
data = out
}
return Decoded{Data: data}
}
// Decode applies a stream dictionary's filter chain to raw. It returns the
// decoded bytes and, when the chain ends in an image filter, that filter's
// name together with the bytes still encoded in it.
//
// Decode is the strict reading: a chain that cannot be run to the end is an
// error and yields no bytes. [DecodeRecovering] is the lenient one, and says
// which of the two it gave you.
func Decode(d Dict, raw []byte, resolve Resolver) ([]byte, Name, error) {
r := DecodeRecovering(d, raw, resolve)
if r.Recovered {
return nil, "", r.Cause
}
return r.Data, r.Image, nil
}
// DecodeStream is Decode for a parsed stream.
func DecodeStream(s *Stream, resolve Resolver) ([]byte, Name, error) {
return Decode(s.Dict, s.Raw, resolve)
}
// DecodeStreamRecovering is DecodeRecovering for a parsed stream.
func DecodeStreamRecovering(s *Stream, resolve Resolver) Decoded {
return DecodeRecovering(s.Dict, s.Raw, resolve)
}
// filterChain reads /Filter and /DecodeParms, each of which may be a single
// value or an array, and returns them aligned.
func filterChain(d Dict, r Resolver) ([]Name, []Dict, error) {
fo, err := Resolve(d.Get("Filter"), r)
if err != nil {
return nil, nil, err
}
var names []Name
switch v := fo.(type) {
case Null:
case Name:
names = []Name{v}
case Array:
for _, e := range v {
eo, err := Resolve(e, r)
if err != nil {
return nil, nil, err
}
n, ok := ToName(eo)
if !ok {
return nil, nil, fmt.Errorf("reader: /Filter array holds a %s, not a name", eo.Kind())
}
names = append(names, n)
}
default:
return nil, nil, fmt.Errorf("reader: /Filter is a %s, not a name or an array", fo.Kind())
}
parms := make([]Dict, len(names))
po, err := Resolve(d.Get("DecodeParms"), r)
if err != nil {
return nil, nil, err
}
switch v := po.(type) {
case Null:
case Dict:
if len(parms) > 0 {
parms[0] = v
}
case Array:
for i, e := range v {
if i >= len(parms) {
break
}
eo, err := Resolve(e, r)
if err != nil {
return nil, nil, err
}
if pd, ok := ToDict(eo); ok {
parms[i] = pd
}
}
default:
return nil, nil, fmt.Errorf("reader: /DecodeParms is a %s, not a dictionary or an array", po.Kind())
}
return names, parms, nil
}
// applyFilter runs one filter. The abbreviated names are the ones inline
// images use; regular streams may legally carry them too.
func applyFilter(f Name, data []byte, parm Dict, r Resolver) ([]byte, error) {
switch f {
case "FlateDecode", "Fl":
out, err := flateDecode(data)
if err != nil {
return salvage(out, err, parm, r)
}
return applyPredictor(out, parm, r)
case "LZWDecode", "LZW":
early := intParm(parm, "EarlyChange", 1, r)
out, err := lzwDecode(data, early != 0)
if err != nil {
return salvage(out, err, parm, r)
}
return applyPredictor(out, parm, r)
case "ASCIIHexDecode", "AHx":
return asciiHexDecode(data)
case "ASCII85Decode", "A85":
return ascii85Decode(data)
case "RunLengthDecode", "RL":
return runLengthDecode(data)
case "CCITTFaxDecode", "CCF":
return ccittDecode(data, ccittParamsOf(parm, r))
case "Crypt":
return cryptFilter(data, parm, r)
}
return nil, fmt.Errorf("reader: unsupported filter /%s", f)
}
// cryptFilter applies the /Crypt filter, which does not transform anything: it
// names the crypt filter a stream's bytes were encrypted with, and /Identity —
// which is also what /Crypt with no /Name means — says they were not encrypted
// at all. Either way the document that owns the stream has already dealt with
// its encryption by the time the filter chain runs, so there is nothing here
// left to undo.
//
// A /Name this reader cannot account for is reported rather than waved through,
// because the bytes would then be ciphertext that only looks like data.
func cryptFilter(data []byte, parm Dict, r Resolver) ([]byte, error) {
o, err := Resolve(parm.Get("Name"), r)
if err != nil {
return nil, fmt.Errorf("reader: /Crypt filter: %w", err)
}
if n, ok := ToName(o); ok && n != "Identity" {
// No bytes come back: they are ciphertext, and handing them over as
// though the filter had run is how ciphertext gets painted.
return nil, fmt.Errorf("reader: /Crypt filter names /%s, which this reader cannot apply", n)
}
return data, nil
}
// firstFilter names the first filter of a stream's chain. Only direct values
// are read: it is consulted while the file key is being established, before
// following an indirect reference is safe.
func firstFilter(d Dict) (Name, bool) {
switch v := d.Get("Filter").(type) {
case Name:
return v, true
case Array:
if len(v) > 0 {
n, ok := ToName(v[0])
return n, ok
}
}
return "", false
}
// firstDecodeParms is the parameter dictionary belonging to the first filter,
// read directly for the same reason.
func firstDecodeParms(d Dict) Dict {
switch v := d.Get("DecodeParms").(type) {
case Dict:
return v
case Array:
if len(v) > 0 {
if pd, ok := ToDict(v[0]); ok {
return pd
}
}
}
return nil
}
// salvage finishes a filter that stopped part-way. The prefix it did produce is
// still worth having, and the predictor still applies to it: the predictors a
// PDF may name all undo one row at a time, so a buffer that stops in the middle
// undoes up to where it stops.
func salvage(out []byte, err error, parm Dict, r Resolver) ([]byte, error) {
if len(out) == 0 {
return nil, err
}
if p, perr := applyPredictor(out, parm, r); perr == nil {
return p, err
}
return out, err
}
// intParm reads an integer decode parameter, falling back to its default.
func intParm(parm Dict, key Name, def int, r Resolver) int {
if parm == nil {
return def
}
o, err := Resolve(parm.Get(key), r)
if err != nil {
return def
}
n, ok := ToInt(o)
if !ok {
return def
}
return int(n)
}
// flateDecode inflates a stream. Producers emit both zlib-wrapped and bare
// deflate data, sometimes with leading white-space, and truncate the last
// stream in a damaged file. All four cases yield whatever bytes are there — but
// a prefix comes back with the error that ended it, never dressed up as a whole
// stream, so the caller can tell the difference.
func flateDecode(data []byte) ([]byte, error) {
i := 0
for i < len(data) && isSpace(data[i]) {
i++
}
data = data[i:]
if zr, err := zlib.NewReader(bytes.NewReader(data)); err == nil {
out, err := readAllCapped(zr)
if err == nil {
return out, nil
}
if len(out) > 0 {
return out, fmt.Errorf("reader: FlateDecode: %w", err)
}
}
out, err := readAllCapped(flate.NewReader(bytes.NewReader(data)))
if err != nil {
return out, fmt.Errorf("reader: FlateDecode: %w", err)
}
return out, nil
}
// readAllCapped reads r, refusing to grow past maxDecodedSize.
func readAllCapped(r io.Reader) ([]byte, error) {
var buf bytes.Buffer
n, err := io.Copy(&buf, io.LimitReader(r, maxDecodedSize+1))
if err != nil {
return buf.Bytes(), err
}
if n > maxDecodedSize {
return nil, fmt.Errorf("reader: decoded stream exceeds %d bytes", maxDecodedSize)
}
return buf.Bytes(), nil
}
// asciiHexDecode reads hexadecimal digits up to the '>' terminator, padding a
// final odd digit with zero.
func asciiHexDecode(data []byte) ([]byte, error) {
out := make([]byte, 0, len(data)/2)
hi := -1
for i := 0; i < len(data); i++ {
c := data[i]
if c == '>' {
break
}
if isSpace(c) {
continue
}
v := hexVal(c)
if v < 0 {
return out, fmt.Errorf("reader: ASCIIHexDecode: invalid digit %q", rune(c))
}
if hi < 0 {
hi = v
continue
}
out = append(out, byte(hi<<4|v))
hi = -1
}
if hi >= 0 {
out = append(out, byte(hi<<4))
}
return out, nil
}
// ascii85Decode reads base-85 groups up to the "~>" terminator.
func ascii85Decode(data []byte) ([]byte, error) {
if bytes.HasPrefix(data, []byte("<~")) {
data = data[2:]
}
out := make([]byte, 0, len(data)*4/5)
var group [5]byte
n := 0
flush := func(n int) {
for i := n; i < 5; i++ {
group[i] = 'u'
}
v := uint32(0)
for _, c := range group {
v = v*85 + uint32(c-'!')
}
var b [4]byte
b[0], b[1], b[2], b[3] = byte(v>>24), byte(v>>16), byte(v>>8), byte(v)
out = append(out, b[:n-1]...)
}
for i := 0; i < len(data); i++ {
c := data[i]
if isSpace(c) {
continue
}
if c == '~' {
break
}
if c == 'z' && n == 0 {
out = append(out, 0, 0, 0, 0)
continue
}
if c < '!' || c > 'u' {
return out, fmt.Errorf("reader: ASCII85Decode: invalid character %q", rune(c))
}
group[n] = c
n++
if n == 5 {
if err := checkA85(group); err != nil {
return out, err
}
flush(5)
n = 0
}
}
if n == 1 {
return out, fmt.Errorf("reader: ASCII85Decode: truncated final group")
}
if n > 1 {
flush(n)
}
return out, nil
}
// checkA85 rejects a full group that names more than 2^32-1.
func checkA85(g [5]byte) error {
v := uint64(0)
for _, c := range g {
v = v*85 + uint64(c-'!')
}
if v > 0xFFFFFFFF {
return fmt.Errorf("reader: ASCII85Decode: group overflows 32 bits")
}
return nil
}
// runLengthDecode expands the byte-oriented run-length encoding: a length byte
// below 128 introduces that many literal bytes plus one, above 128 repeats the
// next byte, and 128 ends the data.
func runLengthDecode(data []byte) ([]byte, error) {
out := []byte{}
for i := 0; i < len(data); {
n := int(data[i])
i++
switch {
case n == 128:
return out, nil
case n < 128:
end := i + n + 1
if end > len(data) {
return out, fmt.Errorf("reader: RunLengthDecode: truncated literal run")
}
out = append(out, data[i:end]...)
i = end
default:
if i >= len(data) {
return out, fmt.Errorf("reader: RunLengthDecode: truncated repeat run")
}
out = append(out, bytes.Repeat(data[i:i+1], 257-n)...)
i++
}
}
return out, nil
}
vendor/github.com/go-pdfkit/reader/inline.go
Generated Vendored
+155
View File
@@ -0,0 +1,155 @@
package reader
import "bytes"
// inlineImageEnd finds where an inline image's data stops — the one genuinely
// ambiguous thing in a content stream, since the data is raw bytes that may
// spell EI themselves.
//
// The answers are tried in order of how much they can be trusted: an image
// with no filter says exactly how long it is through its width, height, depth
// and colour space; a declared /L is believed when an EI really does follow
// it; and failing both, every EI in the data is tried in turn and the first
// one whose data actually decodes is taken. That last step is what a scan
// alone gets wrong, because compressed bytes do spell EI by accident.
func inlineImageEnd(b []byte, start int, d Dict) (int, error) {
if n, ok := inlineImageDataLength(d); ok && start+n <= len(b) && eiFollows(b, start+n) {
return start + n, nil
}
if n, ok := ToInt(d.Get("L")); ok && n >= 0 && start+int(n) <= len(b) && eiFollows(b, start+int(n)) {
return start + int(n), nil
}
expanded := (&InlineImage{Dict: d}).Expanded()
want, haveWant := inlineSampleBytes(d)
// White-space before EI is how the keyword is meant to be written; a
// second pass allows for producers that run the data straight into it.
for _, needSpace := range []bool{true, false} {
for from := start; ; {
end, resume := nextEI(b, start, from, needSpace)
if end < 0 {
break
}
if inlineDataDecodes(expanded, b[start:end], want, haveWant) {
return end, nil
}
from = resume
}
}
return 0, &SyntaxError{start, "an inline image has no EI"}
}
// nextEI returns where the image data would end for the next EI candidate at
// or after from, and where to resume searching. It reports -1 when there is no
// candidate left.
func nextEI(b []byte, start, from int, needSpace bool) (end, resume int) {
for i := from; i+1 < len(b); i++ {
if b[i] != 'E' || b[i+1] != 'I' {
continue
}
if i+2 < len(b) && isRegular(b[i+2]) {
continue
}
if i > start && isSpace(b[i-1]) {
return i - 1, i + 2
}
if !needSpace {
return i, i + 2
}
}
return -1, len(b)
}
// eiFollows reports whether an EI keyword stands at i, after white-space.
func eiFollows(b []byte, i int) bool {
for i < len(b) && isSpace(b[i]) {
i++
}
if !bytes.HasPrefix(b[i:], []byte("EI")) {
return false
}
return i+2 >= len(b) || !isRegular(b[i+2])
}
// inlineDataDecodes reports whether a candidate stretch of bytes is really the
// whole of the image: it has to run through the declared filters without
// complaint, produce the number of samples the image says it has, and — for a
// JPEG, which no filter here can check — begin and end where a JPEG does.
func inlineDataDecodes(expanded Dict, data []byte, want int, haveWant bool) bool {
if expanded.Get("Filter").Kind() == KindNull {
return true
}
out, img, err := Decode(expanded, data, nil)
if err != nil {
return false
}
if img == "DCTDecode" {
return len(out) > 4 && out[0] == 0xFF && out[1] == 0xD8 &&
out[len(out)-2] == 0xFF && out[len(out)-1] == 0xD9
}
if img != "" {
return true
}
return !haveWant || len(out) == want
}
// inlineComponents reports how many samples one pixel of an inline image has,
// for the colour spaces that can be named inline.
func inlineComponents(d Dict) (int, bool) {
if b, ok := ToBool(d.Get("IM")); ok && b {
return 1, true
}
cs, ok := ToName(d.Get("CS"))
if !ok {
if _, isArray := ToArray(d.Get("CS")); isArray {
// An inline [/Indexed …] space has one component per sample.
return 1, true
}
return 0, false
}
switch cs {
case "G", "DeviceGray", "CalGray", "I", "Indexed":
return 1, true
case "RGB", "DeviceRGB", "CalRGB":
return 3, true
case "CMYK", "DeviceCMYK":
return 4, true
}
// A name that refers to a colour space in the page's resources: not
// something the arithmetic can settle here.
return 0, false
}
// inlineSampleBytes computes how many bytes an inline image's samples occupy
// once decoded.
func inlineSampleBytes(d Dict) (int, bool) {
w, okW := ToInt(d.Get("W"))
h, okH := ToInt(d.Get("H"))
if !okW || !okH || w <= 0 || h <= 0 {
return 0, false
}
bpc := int64(8)
if b, ok := ToInt(d.Get("BPC")); ok {
bpc = b
}
if im, ok := ToBool(d.Get("IM")); ok && im {
bpc = 1
}
if bpc <= 0 || bpc > 16 {
return 0, false
}
comps, ok := inlineComponents(d)
if !ok {
return 0, false
}
row := (w*int64(comps)*bpc + 7) / 8
return int(row * h), true
}
// inlineImageDataLength is inlineSampleBytes for an image with no filter, for
// which the samples are the data.
func inlineImageDataLength(d Dict) (int, bool) {
if d.Get("F").Kind() != KindNull || d.Get("Filter").Kind() != KindNull {
return 0, false
}
return inlineSampleBytes(d)
}
vendor/github.com/go-pdfkit/reader/lex.go
Generated Vendored
+340
View File
@@ -0,0 +1,340 @@
package reader
import (
"fmt"
"strconv"
)
// A SyntaxError reports malformed PDF syntax and where it was found, so a
// repair pass can restart from a known offset instead of giving up.
type SyntaxError struct {
Offset int
Msg string
}
// Error implements the error interface.
func (e *SyntaxError) Error() string {
return fmt.Sprintf("reader: %s at offset %d", e.Msg, e.Offset)
}
// isSpace reports the six bytes the specification calls white-space,
// NUL included.
func isSpace(c byte) bool {
return c == 0 || c == 9 || c == 10 || c == 12 || c == 13 || c == 32
}
// isDelim reports the eight delimiter bytes.
func isDelim(c byte) bool {
switch c {
case '(', ')', '<', '>', '[', ']', '{', '}', '/', '%':
return true
}
return false
}
// isRegular reports a byte that may appear inside a name, a number or a
// keyword: anything that is neither white-space nor a delimiter.
func isRegular(c byte) bool { return !isSpace(c) && !isDelim(c) }
// hexVal decodes one hexadecimal digit, or reports -1.
func hexVal(c byte) int {
switch {
case c >= '0' && c <= '9':
return int(c - '0')
case c >= 'a' && c <= 'f':
return int(c-'a') + 10
case c >= 'A' && c <= 'F':
return int(c-'A') + 10
}
return -1
}
type tokKind uint8
const (
tokEOF tokKind = iota
tokInteger
tokReal
tokString
tokName
tokArrayOpen
tokArrayClose
tokDictOpen
tokDictClose
tokBraceOpen
tokBraceClose
tokKeyword
)
// A token is one lexical unit. text carries the bytes of a string, name or
// keyword; i and f carry the value of a number.
type token struct {
kind tokKind
text []byte
i int64
f float64
pos int
}
// A lexer walks the PDF object syntax over a byte slice. It never copies the
// input except where an escape sequence forces it.
type lexer struct {
buf []byte
pos int
}
// skipSpace consumes white-space and comments.
func (l *lexer) skipSpace() {
for l.pos < len(l.buf) {
c := l.buf[l.pos]
if isSpace(c) {
l.pos++
continue
}
if c == '%' {
for l.pos < len(l.buf) && l.buf[l.pos] != '\n' && l.buf[l.pos] != '\r' {
l.pos++
}
continue
}
return
}
}
// next returns the token at the current position and advances past it.
func (l *lexer) next() (token, error) {
l.skipSpace()
start := l.pos
if l.pos >= len(l.buf) {
return token{kind: tokEOF, pos: start}, nil
}
switch c := l.buf[l.pos]; {
case c == '[':
l.pos++
return token{kind: tokArrayOpen, pos: start}, nil
case c == ']':
l.pos++
return token{kind: tokArrayClose, pos: start}, nil
case c == '{':
l.pos++
return token{kind: tokBraceOpen, pos: start}, nil
case c == '}':
l.pos++
return token{kind: tokBraceClose, pos: start}, nil
case c == '<':
if l.pos+1 < len(l.buf) && l.buf[l.pos+1] == '<' {
l.pos += 2
return token{kind: tokDictOpen, pos: start}, nil
}
return l.hexString(start)
case c == '>':
if l.pos+1 < len(l.buf) && l.buf[l.pos+1] == '>' {
l.pos += 2
return token{kind: tokDictClose, pos: start}, nil
}
l.pos++
return token{}, &SyntaxError{start, "stray '>'"}
case c == '(':
return l.literalString(start)
case c == ')':
l.pos++
return token{}, &SyntaxError{start, "stray ')'"}
case c == '/':
return l.name(start)
case c == '+' || c == '-' || c == '.' || (c >= '0' && c <= '9'):
return l.number(start)
default:
return l.keyword(start), nil
}
}
// keyword reads a run of regular characters: an operator, obj, endobj, R,
// true, false, null.
func (l *lexer) keyword(start int) token {
p := l.pos
for p < len(l.buf) && isRegular(l.buf[p]) {
p++
}
l.pos = p
return token{kind: tokKeyword, text: l.buf[start:p], pos: start}
}
// number reads an integer or a real. It stops at the first byte that cannot
// continue the number rather than swallowing the whole run of regular
// characters: producers do write "3.4-5" for two numbers, and a reader that
// rejects that loses the rest of the content stream.
func (l *lexer) number(start int) (token, error) {
p := l.pos
for p < len(l.buf) && (l.buf[p] == '+' || l.buf[p] == '-') {
p++
}
digits, dot := 0, false
for p < len(l.buf) {
c := l.buf[p]
switch {
case c >= '0' && c <= '9':
digits++
case c == '.' && !dot:
dot = true
default:
goto done
}
p++
}
done:
s := l.buf[l.pos:p]
l.pos = p
if digits == 0 {
return token{}, &SyntaxError{start, "malformed number " + strconv.Quote(string(s))}
}
if i, err := strconv.ParseInt(string(s), 10, 64); err == nil {
return token{kind: tokInteger, i: i, f: float64(i), pos: start}, nil
}
f, err := parseReal(s)
if err != nil {
return token{}, &SyntaxError{start, "malformed number " + strconv.Quote(string(s))}
}
return token{kind: tokReal, i: int64(f), f: f, pos: start}, nil
}
// parseReal accepts the doubled sign of "--5", which producers do write; only
// the first sign counts. Everything else has already been filtered out by the
// scan above.
func parseReal(s []byte) (float64, error) {
clean := make([]byte, 0, len(s))
i := 0
if i < len(s) && (s[i] == '+' || s[i] == '-') {
clean = append(clean, s[i])
i++
}
for i < len(s) && (s[i] == '+' || s[i] == '-') {
i++
}
clean = append(clean, s[i:]...)
return strconv.ParseFloat(string(clean), 64)
}
// literalString reads a (parenthesised) string, resolving escapes. Nested
// parentheses nest; an end-of-line inside the string, however written, becomes
// a single line feed.
func (l *lexer) literalString(start int) (token, error) {
l.pos++
depth := 1
out := []byte{}
for l.pos < len(l.buf) {
c := l.buf[l.pos]
l.pos++
switch c {
case '\\':
if l.pos >= len(l.buf) {
return token{}, &SyntaxError{start, "unterminated string"}
}
e := l.buf[l.pos]
l.pos++
switch {
case e == 'n':
out = append(out, '\n')
case e == 'r':
out = append(out, '\r')
case e == 't':
out = append(out, '\t')
case e == 'b':
out = append(out, '\b')
case e == 'f':
out = append(out, '\f')
case e == '\r':
// A backslash before an end-of-line continues the line.
if l.pos < len(l.buf) && l.buf[l.pos] == '\n' {
l.pos++
}
case e == '\n':
case e >= '0' && e <= '7':
v := int(e - '0')
for k := 0; k < 2 && l.pos < len(l.buf) && l.buf[l.pos] >= '0' && l.buf[l.pos] <= '7'; k++ {
v = v*8 + int(l.buf[l.pos]-'0')
l.pos++
}
out = append(out, byte(v))
default:
// \( \) \\ and, per the specification, any other escaped
// character stands for itself.
out = append(out, e)
}
case '(':
depth++
out = append(out, c)
case ')':
depth--
if depth == 0 {
return token{kind: tokString, text: out, pos: start}, nil
}
out = append(out, c)
case '\r':
if l.pos < len(l.buf) && l.buf[l.pos] == '\n' {
l.pos++
}
out = append(out, '\n')
default:
out = append(out, c)
}
}
return token{}, &SyntaxError{start, "unterminated string"}
}
// hexString reads a <hex> string. White-space is ignored and a final odd digit
// is padded with zero, as the specification requires.
func (l *lexer) hexString(start int) (token, error) {
l.pos++
out := []byte{}
hi := -1
for l.pos < len(l.buf) {
c := l.buf[l.pos]
l.pos++
if c == '>' {
if hi >= 0 {
out = append(out, byte(hi<<4))
}
return token{kind: tokString, text: out, pos: start}, nil
}
if isSpace(c) {
continue
}
v := hexVal(c)
if v < 0 {
// Producers do emit a stray byte inside <…>; skipping it is what
// every other reader does, and losing the whole object is worse.
continue
}
if hi < 0 {
hi = v
continue
}
out = append(out, byte(hi<<4|v))
hi = -1
}
return token{}, &SyntaxError{start, "unterminated hex string"}
}
// name reads a /Name, resolving #xx escapes.
func (l *lexer) name(start int) (token, error) {
l.pos++
out := []byte{}
for l.pos < len(l.buf) && isRegular(l.buf[l.pos]) {
c := l.buf[l.pos]
l.pos++
if c != '#' {
out = append(out, c)
continue
}
if l.pos+1 >= len(l.buf) {
return token{}, &SyntaxError{start, "truncated #xx escape in name"}
}
h1, h2 := hexVal(l.buf[l.pos]), hexVal(l.buf[l.pos+1])
if h1 < 0 || h2 < 0 {
return token{}, &SyntaxError{start, "invalid #xx escape in name"}
}
out = append(out, byte(h1<<4|h2))
l.pos += 2
}
return token{kind: tokName, text: out, pos: start}, nil
}
vendor/github.com/go-pdfkit/reader/lzw.go
Generated Vendored
+86
View File
@@ -0,0 +1,86 @@
package reader
import "fmt"
// LZW code points that are not table entries.
const (
lzwClear = 256
lzwEOD = 257
lzwFirst = 258
lzwMax = 4096
)
// lzwDecode expands PDF's variable-code-width LZW.
//
// The standard library's compress/lzw cannot be used here: PDF's /EarlyChange
// parameter defaults to 1, which widens the code one entry before the table
// actually fills, and compress/lzw implements neither that convention nor a
// way to ask for it. Getting this wrong does not fail — it silently produces
// plausible rubbish a few hundred bytes in.
func lzwDecode(data []byte, early bool) ([]byte, error) {
var table [lzwMax][]byte
for i := 0; i < 256; i++ {
table[i] = []byte{byte(i)}
}
next := lzwFirst
width := 9
bit := 0
var out, prev []byte
for {
if bit+width > len(data)*8 {
// A truncated stream keeps what it managed to say.
return out, nil
}
code := 0
for k := 0; k < width; k++ {
code = code<<1 | int(data[(bit+k)/8]>>(7-(bit+k)%8)&1)
}
bit += width
switch code {
case lzwEOD:
return out, nil
case lzwClear:
next, width, prev = lzwFirst, 9, nil
continue
}
var entry []byte
switch {
case code < next && table[code] != nil:
entry = table[code]
case code == next && prev != nil:
// The encoder may name the entry it is about to define.
entry = append(append([]byte{}, prev...), prev[0])
default:
// What the stream did say is kept, with the reason it stopped.
return out, fmt.Errorf("reader: LZWDecode: code %d is not in the table", code)
}
out = append(out, entry...)
if prev != nil && next < lzwMax {
table[next] = append(append([]byte{}, prev...), entry[0])
next++
}
prev = entry
// With EarlyChange the width grows one entry sooner. Both conventions
// are the same comparison against a shifted count.
count := next
if early {
count++
}
switch {
case count >= 2048:
width = 12
case count >= 1024:
width = 11
case count >= 512:
width = 10
default:
width = 9
}
}
}
vendor/github.com/go-pdfkit/reader/object.go
Generated Vendored
+244
View File
@@ -0,0 +1,244 @@
// Package reader takes existing PDF bytes apart: the object syntax, the
// cross-reference machinery, stream filters, and the document structure built
// on top of them. It is the parsing counterpart of github.com/go-pdfkit/pdfkit,
// which writes PDF.
//
// Nothing outside the Go standard library is used, so the package builds for
// js/wasm and for every architecture the fleet targets.
package reader
import "fmt"
// Kind names the eight basic PDF object types plus the two the file format
// adds on top of them: streams and indirect references.
type Kind uint8
// The object kinds. [Object.Kind] reports one of these.
const (
KindNull Kind = iota
KindBool
KindInteger
KindReal
KindString
KindName
KindArray
KindDict
KindStream
KindRef
)
var kindNames = [...]string{
KindNull: "null",
KindBool: "boolean",
KindInteger: "integer",
KindReal: "real",
KindString: "string",
KindName: "name",
KindArray: "array",
KindDict: "dictionary",
KindStream: "stream",
KindRef: "reference",
}
// String names the kind, for error messages.
func (k Kind) String() string {
if int(k) >= len(kindNames) {
return fmt.Sprintf("Kind(%d)", uint8(k))
}
return kindNames[k]
}
// An Object is any value the PDF object syntax can express. The concrete types
// are [Null], [Bool], [Integer], [Real], [String], [Name], [Array], [Dict],
// [Ref] and *[Stream]; Kind tells them apart without a type switch.
type Object interface {
Kind() Kind
}
// Null is the PDF null object. A missing dictionary entry resolves to Null
// rather than to a nil Object, so callers never have to nil-check.
type Null struct{}
// Kind implements [Object].
func (Null) Kind() Kind { return KindNull }
// Bool is a PDF boolean.
type Bool bool
// Kind implements [Object].
func (Bool) Kind() Kind { return KindBool }
// Integer is a PDF integer. Numbers too large for an int64 are parsed as
// [Real], which is what every other reader does.
type Integer int64
// Kind implements [Object].
func (Integer) Kind() Kind { return KindInteger }
// Real is a PDF real number.
type Real float64
// Kind implements [Object].
func (Real) Kind() Kind { return KindReal }
// String is a PDF string, already unescaped: the bytes a literal or hex string
// denotes, with no interpretation of their text encoding.
type String []byte
// Kind implements [Object].
func (String) Kind() Kind { return KindString }
// Name is a PDF name with its #xx escapes resolved and the leading slash
// dropped, so /Type is Name("Type").
type Name string
// Kind implements [Object].
func (Name) Kind() Kind { return KindName }
// Array is a PDF array.
type Array []Object
// Kind implements [Object].
func (Array) Kind() Kind { return KindArray }
// Dict is a PDF dictionary.
type Dict map[Name]Object
// Kind implements [Object].
func (Dict) Kind() Kind { return KindDict }
// Get returns the entry, or [Null] when it is absent. The value may still be a
// [Ref]; use [Resolve] to follow it.
func (d Dict) Get(k Name) Object {
if v, ok := d[k]; ok && v != nil {
return v
}
return Null{}
}
// Ref is an indirect reference: the "12 0 R" that stands in for an object
// stored elsewhere in the file.
type Ref struct {
Num int
Gen int
}
// Kind implements [Object].
func (Ref) Kind() Kind { return KindRef }
// String renders the reference the way it appears in a file.
func (r Ref) String() string { return fmt.Sprintf("%d %d R", r.Num, r.Gen) }
// Stream is a PDF stream: a dictionary plus the raw, still-encoded bytes
// between the stream and endstream keywords. [Decode] applies the filter chain.
type Stream struct {
Dict Dict
Raw []byte
}
// Kind implements [Object].
func (*Stream) Kind() Kind { return KindStream }
// A Resolver fetches the object an indirect reference names. A document
// supplies one; a nil Resolver makes every reference resolve to [Null], which
// is what a caller parsing an isolated fragment wants.
type Resolver func(Ref) (Object, error)
// maxRefDepth bounds a chain of references pointing at references. A file can
// make that chain a cycle, and a cycle must not hang the reader.
const maxRefDepth = 64
// Resolve follows indirect references until it reaches a direct object. A nil
// object, an absent entry and an unresolvable reference all yield [Null].
func Resolve(o Object, r Resolver) (Object, error) {
if o == nil {
return Null{}, nil
}
for depth := 0; ; depth++ {
ref, ok := o.(Ref)
if !ok {
return o, nil
}
if r == nil {
return Null{}, nil
}
if depth >= maxRefDepth {
return nil, fmt.Errorf("reader: reference chain longer than %d at %s", maxRefDepth, ref)
}
v, err := r(ref)
if err != nil {
return nil, err
}
if v == nil {
return Null{}, nil
}
o = v
}
}
// ToBool reports the value of a boolean object.
func ToBool(o Object) (bool, bool) {
v, ok := o.(Bool)
return bool(v), ok
}
// ToInt reports the value of an integer object, and of a real whose value is a
// whole number — files do write "3.0" where an integer belongs.
func ToInt(o Object) (int64, bool) {
switch v := o.(type) {
case Integer:
return int64(v), true
case Real:
if float64(int64(v)) == float64(v) {
return int64(v), true
}
}
return 0, false
}
// ToFloat reports the value of any numeric object.
func ToFloat(o Object) (float64, bool) {
switch v := o.(type) {
case Integer:
return float64(v), true
case Real:
return float64(v), true
}
return 0, false
}
// ToString reports the bytes of a string object.
func ToString(o Object) ([]byte, bool) {
v, ok := o.(String)
return []byte(v), ok
}
// ToName reports the value of a name object.
func ToName(o Object) (Name, bool) {
v, ok := o.(Name)
return v, ok
}
// ToArray reports the elements of an array object.
func ToArray(o Object) (Array, bool) {
v, ok := o.(Array)
return v, ok
}
// ToDict reports the dictionary of a dictionary object, or a stream's own
// dictionary — a stream is a dictionary everywhere a dictionary is expected.
func ToDict(o Object) (Dict, bool) {
switch v := o.(type) {
case Dict:
return v, true
case *Stream:
return v.Dict, true
}
return nil, false
}
// ToStream reports the stream object.
func ToStream(o Object) (*Stream, bool) {
v, ok := o.(*Stream)
return v, ok
}
vendor/github.com/go-pdfkit/reader/objstm.go
Generated Vendored
+128
View File
@@ -0,0 +1,128 @@
package reader
import (
"bytes"
"compress/zlib"
"fmt"
)
// objectsPerStream bounds how many objects go into one object stream. A
// stream has to be inflated whole to read any object in it, so packing
// everything into one would make opening a file read all of it.
const objectsPerStream = 200
// packedAt says which object stream an object was packed into, and where.
type packedAt struct {
stream int
index int
}
// pendingObject is an object waiting to be packed.
type pendingObject struct {
ref Ref
obj Object
}
// packObjects writes every waiting object into object streams. It is called
// once, from Finish.
func (w *Writer) packObjects() {
for start := 0; start < len(w.pending); start += objectsPerStream {
end := start + objectsPerStream
if end > len(w.pending) {
end = len(w.pending)
}
w.packGroup(w.pending[start:end])
}
w.pending = nil
}
// packGroup writes one object stream holding the given objects.
func (w *Writer) packGroup(group []pendingObject) {
var head, payload bytes.Buffer
for _, p := range group {
fmt.Fprintf(&head, "%d %d ", p.ref.Num, payload.Len())
payload.Write(AppendObject(nil, p.obj))
payload.WriteByte('\n')
}
body := append(head.Bytes(), payload.Bytes()...)
stream := w.Reserve()
for i, p := range group {
w.packed[p.ref.Num] = packedAt{stream: stream.Num, index: i}
}
w.writeInline(stream, &Stream{
Dict: Dict{
"Type": Name("ObjStm"),
"N": Integer(len(group)),
"First": Integer(head.Len()),
},
Raw: body,
})
}
// flateCompress compresses a stream's data, which is what makes packing worth doing.
func flateCompress(data []byte) []byte {
var buf bytes.Buffer
zw := zlib.NewWriter(&buf)
// A bytes.Buffer never refuses bytes, and Close only flushes.
zw.Write(data)
zw.Close()
return buf.Bytes()
}
// finishWithXrefStream writes the cross-reference information as a stream
// rather than a table, which is the only form that can name an object inside
// an object stream.
func (w *Writer) finishWithXrefStream(trailer Dict) ([]byte, error) {
// Finish has already written the /Encrypt dictionary and put what a
// reader needs into the trailer.
w.packObjects()
xref := w.Reserve()
// The cross-reference stream is reserved after everything else — the
// object streams included — so it always carries the highest number.
high := xref.Num
start := w.buf.Len()
w.offsets[xref.Num] = start
rows := make([]byte, 0, (high+1)*7)
for num := 0; num <= high; num++ {
switch {
case num == 0:
rows = append(rows, 0, 0, 0, 0, 0, 0xFF, 0xFF)
case w.offsets[num] != 0 || num == xref.Num:
rows = append(rows, xrefRow(1, int64(w.offsets[num]), 0)...)
default:
if at, ok := w.packed[num]; ok {
rows = append(rows, xrefRow(2, int64(at.stream), int64(at.index))...)
continue
}
rows = append(rows, 0, 0, 0, 0, 0, 0xFF, 0xFF)
}
}
dict := Dict{
"Type": Name("XRef"),
"Size": Integer(high + 1),
"W": Array{Integer(1), Integer(4), Integer(2)},
}
for k, v := range trailer {
dict[k] = v
}
w.writeInlineRaw(xref, &Stream{Dict: dict, Raw: rows})
fmt.Fprintf(&w.buf, "startxref\n%d\n%%%%EOF\n", start)
if w.encrypt != nil && w.encrypt.err != nil {
w.note(w.encrypt.err)
}
if w.err != nil {
return nil, w.err
}
return w.buf.Bytes(), nil
}
// xrefRow renders one three-field row of a cross-reference stream.
func xrefRow(kind byte, f2, f3 int64) []byte {
return []byte{
kind,
byte(f2 >> 24), byte(f2 >> 16), byte(f2 >> 8), byte(f2),
byte(f3 >> 8), byte(f3),
}
}
vendor/github.com/go-pdfkit/reader/parse.go
Generated Vendored
+295
View File
@@ -0,0 +1,295 @@
package reader
import (
"bytes"
"strconv"
)
// A parser turns the token stream into objects. resolve is consulted only for
// a stream's /Length, which files are allowed to store indirectly.
type parser struct {
lex lexer
resolve Resolver
}
// ParseObject parses a single direct object from the start of b and reports how
// many bytes it consumed.
func ParseObject(b []byte) (Object, int, error) {
p := &parser{lex: lexer{buf: b}}
o, err := p.parseObject()
if err != nil {
return nil, 0, err
}
return o, p.lex.pos, nil
}
// ParseIndirectObject parses "N G obj … endobj" from the start of b, returning
// the reference it defines, the object itself, and how many bytes it consumed.
// A stream's /Length may be an indirect reference; resolve supplies it, and a
// nil or unhelpful resolve falls back to scanning for the endstream keyword.
func ParseIndirectObject(b []byte, resolve Resolver) (Ref, Object, int, error) {
p := &parser{lex: lexer{buf: b}, resolve: resolve}
num, err := p.expectUint("object number")
if err != nil {
return Ref{}, nil, 0, err
}
gen, err := p.expectUint("generation number")
if err != nil {
return Ref{}, nil, 0, err
}
if err := p.expectKeyword("obj"); err != nil {
return Ref{}, nil, 0, err
}
ref := Ref{Num: num, Gen: gen}
// An indirect object may have no value at all — "7 0 obj endobj" —
// which the corpus does contain. Its value is null.
if p.accept("endobj") {
return ref, Null{}, p.lex.pos, nil
}
obj, err := p.parseObject()
if err != nil {
return ref, nil, 0, err
}
obj, err = p.maybeStream(obj)
if err != nil {
return ref, nil, 0, err
}
p.accept("endobj")
return ref, obj, p.lex.pos, nil
}
// expectUint reads a non-negative integer token.
func (p *parser) expectUint(what string) (int, error) {
t, err := p.lex.next()
if err != nil {
return 0, err
}
if t.kind != tokInteger || t.i < 0 {
return 0, &SyntaxError{t.pos, what + " expected"}
}
return int(t.i), nil
}
// expectKeyword requires the given keyword next.
func (p *parser) expectKeyword(kw string) error {
t, err := p.lex.next()
if err != nil {
return err
}
if t.kind != tokKeyword || string(t.text) != kw {
return &SyntaxError{t.pos, strconv.Quote(kw) + " expected"}
}
return nil
}
// accept consumes the keyword if it is next, and reports whether it was.
func (p *parser) accept(kw string) bool {
save := p.lex.pos
t, err := p.lex.next()
if err == nil && t.kind == tokKeyword && string(t.text) == kw {
return true
}
p.lex.pos = save
return false
}
// maybeStream turns a dictionary followed by the stream keyword into a Stream.
func (p *parser) maybeStream(obj Object) (Object, error) {
save := p.lex.pos
t, err := p.lex.next()
if err != nil || t.kind != tokKeyword || string(t.text) != "stream" {
p.lex.pos = save
return obj, nil
}
d, ok := obj.(Dict)
if !ok {
return nil, &SyntaxError{t.pos, "stream keyword after a " + obj.Kind().String()}
}
return p.readStream(d, t.pos)
}
// readStream extracts the bytes between stream and endstream. The declared
// /Length is trusted only when endstream really does follow it — producers get
// it wrong often enough that the scan below is the common path, not the
// exception.
func (p *parser) readStream(d Dict, kwPos int) (Object, error) {
b := p.lex.buf
i := p.lex.pos
for i < len(b) && (b[i] == ' ' || b[i] == '\t') {
i++
}
if i < len(b) && b[i] == '\r' {
i++
}
if i < len(b) && b[i] == '\n' {
i++
}
start := i
if n, ok := p.streamLength(d); ok && start+n <= len(b) {
if end := endstreamAt(b, start+n); end >= 0 {
p.lex.pos = end
return &Stream{Dict: d, Raw: b[start : start+n]}, nil
}
}
j := bytes.Index(b[start:], []byte("endstream"))
if j < 0 {
return nil, &SyntaxError{kwPos, "unterminated stream"}
}
end := start + j
p.lex.pos = end + len("endstream")
// The end-of-line that precedes endstream belongs to the file, not to the
// stream's data.
if end > start && b[end-1] == '\n' {
end--
}
if end > start && b[end-1] == '\r' {
end--
}
return &Stream{Dict: d, Raw: b[start:end]}, nil
}
// streamLength reads /Length, following an indirect reference when it can.
func (p *parser) streamLength(d Dict) (int, bool) {
o, err := Resolve(d.Get("Length"), p.resolve)
if err != nil {
return 0, false
}
n, ok := ToInt(o)
if !ok || n < 0 {
return 0, false
}
return int(n), true
}
// endstreamAt reports the offset just past an endstream keyword that follows
// white-space at i, or -1 when something else is there.
func endstreamAt(b []byte, i int) int {
for i < len(b) && isSpace(b[i]) {
i++
}
if bytes.HasPrefix(b[i:], []byte("endstream")) {
return i + len("endstream")
}
return -1
}
// parseObject reads one object.
func (p *parser) parseObject() (Object, error) {
t, err := p.lex.next()
if err != nil {
return nil, err
}
return p.parseFrom(t)
}
// parseFrom reads the object that starts with an already-read token.
func (p *parser) parseFrom(t token) (Object, error) {
switch t.kind {
case tokEOF:
return nil, &SyntaxError{t.pos, "unexpected end of input"}
case tokInteger:
return p.maybeRef(t)
case tokReal:
return Real(t.f), nil
case tokString:
return String(t.text), nil
case tokName:
return Name(t.text), nil
case tokArrayOpen:
return p.parseArray()
case tokDictOpen:
return p.parseDict()
case tokKeyword:
switch string(t.text) {
case "true":
return Bool(true), nil
case "false":
return Bool(false), nil
case "null":
return Null{}, nil
}
return nil, &SyntaxError{t.pos, "unexpected keyword " + strconv.Quote(string(t.text))}
}
return nil, &SyntaxError{t.pos, "unexpected token"}
}
// maybeRef decides between the integer just read and the "N G R" that starts
// the same way. Only two tokens of lookahead separate them.
func (p *parser) maybeRef(t token) (Object, error) {
save := p.lex.pos
if t.i >= 0 {
if t2, err := p.lex.next(); err == nil && t2.kind == tokInteger && t2.i >= 0 {
if t3, err := p.lex.next(); err == nil && t3.kind == tokKeyword && string(t3.text) == "R" {
return Ref{Num: int(t.i), Gen: int(t2.i)}, nil
}
}
}
p.lex.pos = save
return Integer(t.i), nil
}
// parseArray reads the body of an array, the opening bracket already consumed.
func (p *parser) parseArray() (Object, error) {
arr := Array{}
for {
t, err := p.lex.next()
if err != nil {
return nil, err
}
switch t.kind {
case tokArrayClose:
return arr, nil
case tokEOF:
return nil, &SyntaxError{t.pos, "unterminated array"}
}
o, err := p.parseFrom(t)
if err != nil {
return nil, err
}
arr = append(arr, o)
}
}
// parseDict reads the body of a dictionary, "<<" already consumed.
func (p *parser) parseDict() (Object, error) {
d := Dict{}
for {
t, err := p.lex.next()
if err != nil {
return nil, err
}
switch t.kind {
case tokDictClose:
return d, nil
case tokEOF:
return nil, &SyntaxError{t.pos, "unterminated dictionary"}
case tokName:
default:
return nil, &SyntaxError{t.pos, "dictionary key is a " + t.kind.describe() + ", not a name"}
}
v, err := p.parseObject()
if err != nil {
return nil, err
}
d[Name(t.text)] = v
}
}
// describe names a token kind for an error message.
func (k tokKind) describe() string {
switch k {
case tokInteger, tokReal:
return "number"
case tokString:
return "string"
case tokArrayOpen, tokArrayClose:
return "bracket"
case tokDictOpen, tokDictClose:
return "dictionary delimiter"
case tokBraceOpen, tokBraceClose:
return "brace"
case tokKeyword:
return "keyword"
}
return "token"
}
vendor/github.com/go-pdfkit/reader/predictor.go
Generated Vendored
+130
View File
@@ -0,0 +1,130 @@
package reader
import "fmt"
// applyPredictor undoes the PNG or TIFF predictor a Flate or LZW stream was
// filtered through. /Predictor 1, and an absent parameter dictionary, mean
// there is nothing to undo.
func applyPredictor(data []byte, parm Dict, r Resolver) ([]byte, error) {
pred := intParm(parm, "Predictor", 1, r)
if pred <= 1 {
return data, nil
}
colors := intParm(parm, "Colors", 1, r)
bpc := intParm(parm, "BitsPerComponent", 8, r)
columns := intParm(parm, "Columns", 1, r)
if colors < 1 || bpc < 1 || columns < 1 {
return nil, fmt.Errorf("reader: predictor: /Colors %d /BitsPerComponent %d /Columns %d", colors, bpc, columns)
}
switch {
case pred == 2:
return tiffPredictor(data, colors, bpc, columns)
case pred >= 10:
return pngPredictor(data, colors, bpc, columns)
}
return nil, fmt.Errorf("reader: predictor: /Predictor %d is not defined", pred)
}
// rowGeometry gives the bytes per row and the byte distance between a sample
// and the one to its left, both rounded up as the specification requires.
func rowGeometry(colors, bpc, columns int) (rowLen, bpp int) {
return (colors*bpc*columns + 7) / 8, (colors*bpc + 7) / 8
}
// pngPredictor undoes the per-row filters PNG defines, each row of the stream
// being preceded by its filter type byte.
func pngPredictor(data []byte, colors, bpc, columns int) ([]byte, error) {
rowLen, bpp := rowGeometry(colors, bpc, columns)
prev := make([]byte, rowLen)
cur := make([]byte, rowLen)
out := make([]byte, 0, len(data))
for i := 0; i < len(data); {
ft := data[i]
i++
n := copy(cur, data[i:])
// A damaged file can end mid-row; treat the missing bytes as zero
// rather than dropping the row.
clear(cur[n:])
i += n
switch ft {
case 0:
case 1:
for k := bpp; k < rowLen; k++ {
cur[k] += cur[k-bpp]
}
case 2:
for k := 0; k < rowLen; k++ {
cur[k] += prev[k]
}
case 3:
for k := 0; k < rowLen; k++ {
left := 0
if k >= bpp {
left = int(cur[k-bpp])
}
cur[k] += byte((left + int(prev[k])) / 2)
}
case 4:
for k := 0; k < rowLen; k++ {
var left, upLeft byte
if k >= bpp {
left, upLeft = cur[k-bpp], prev[k-bpp]
}
cur[k] += paeth(left, prev[k], upLeft)
}
default:
return nil, fmt.Errorf("reader: predictor: unknown PNG row filter %d", ft)
}
out = append(out, cur...)
copy(prev, cur)
}
return out, nil
}
// paeth is the PNG predictor of that name: the neighbour closest to a+b-c.
func paeth(a, b, c byte) byte {
p := int(a) + int(b) - int(c)
pa, pb, pc := abs(p-int(a)), abs(p-int(b)), abs(p-int(c))
if pa <= pb && pa <= pc {
return a
}
if pb <= pc {
return b
}
return c
}
// abs is the integer absolute value.
func abs(v int) int {
if v < 0 {
return -v
}
return v
}
// tiffPredictor undoes horizontal differencing. Only the 8- and 16-bit
// component depths are implemented, which is every one seen in practice; a
// sub-byte depth is reported rather than silently mis-decoded.
func tiffPredictor(data []byte, colors, bpc, columns int) ([]byte, error) {
if bpc != 8 && bpc != 16 {
return nil, fmt.Errorf("reader: predictor: TIFF differencing with /BitsPerComponent %d is not supported", bpc)
}
rowLen, bpp := rowGeometry(colors, bpc, columns)
out := append([]byte{}, data...)
for r := 0; r+rowLen <= len(out); r += rowLen {
row := out[r : r+rowLen]
if bpc == 8 {
for k := bpp; k < rowLen; k++ {
row[k] += row[k-bpp]
}
continue
}
for k := bpp; k+1 < rowLen; k += 2 {
v := uint16(row[k])<<8 | uint16(row[k+1])
p := uint16(row[k-bpp])<<8 | uint16(row[k-bpp+1])
v += p
row[k], row[k+1] = byte(v>>8), byte(v)
}
}
return out, nil
}
vendor/github.com/go-pdfkit/reader/repair.go
Generated Vendored
+339
View File
@@ -0,0 +1,339 @@
package reader
import (
"bytes"
"cmp"
"fmt"
"slices"
)
// repair rebuilds the cross-reference information by reading the file itself
// rather than what it says about itself: every "N G obj" header in order, the
// later definition of an object number winning, plus whatever trailer or
// catalogue can be found. Damaged files are the normal case for a reader, not
// an exceptional one.
func (d *Document) repair() error {
d.repaired = true
d.cache = map[int]Object{}
d.objStms = map[int]map[int]Object{}
d.pages = nil
d.xref = map[int]xrefEntry{}
for _, h := range scanObjectHeaders(d.buf) {
// A later definition wins, so the entry is overwritten unconditionally
// rather than through setEntry.
d.xref[h.num] = xrefEntry{kind: 'n', offset: int64(h.offset), gen: h.gen}
}
if len(d.xref) == 0 {
return fmt.Errorf("reader: the file holds no indirect objects")
}
// A rebuild reads object streams, and in an encrypted file it cannot read
// one until the file key is known. The trailers a scan can see are where
// /Encrypt is named, so the key is established before the streams are
// indexed rather than after: otherwise a file whose catalogue lives in an
// encrypted object stream is rebuilt from bytes nobody can read, and the
// rebuild fails for a reason that has nothing to do with the file.
if err := d.establishDecryption(); err != nil {
return err
}
// Objects held in object streams are invisible to a header scan; take them
// from every object stream the scan did find.
d.indexObjectStreams()
d.loadRepairedTrailer()
if d.trailer == nil {
return fmt.Errorf("reader: no document catalogue found")
}
return nil
}
// establishDecryption derives the file key from a trailer the scan can see, for
// a rebuild that runs before the cross-reference tables could name /Encrypt.
// A file that is not encrypted, or whose key is already known, needs nothing.
func (d *Document) establishDecryption() error {
if d.decrypt != nil || !bytes.Contains(d.buf, []byte("/Encrypt")) {
// /Encrypt can only be named by a trailer, and no trailer is
// compressed, so a file that does not hold those bytes anywhere is not
// encrypted. Saying so cheaply keeps the scan below off the common path.
return nil
}
previous := d.trailer
for _, tr := range d.trailerCandidates() {
if tr == nil || tr.Get("Encrypt").Kind() == KindNull {
continue
}
d.trailer = tr
err := d.setUpDecryption(d.password)
d.trailer = previous
if err != nil {
return err
}
// Whatever was read while the key was unknown was read wrong.
d.cache = map[int]Object{}
return nil
}
return nil
}
// trailerCandidates lists every dictionary that could carry trailer entries:
// what the tables managed to read, then the file's trailer keywords, then the
// dictionary of each cross-reference stream.
//
// The last of those is not optional. A file written with cross-reference
// streams has no trailer keyword anywhere in it, and /Encrypt then exists only
// in a /Type /XRef stream's dictionary — which is the one stream a PDF never
// encrypts, precisely so that it can be read before the key is known.
//
// The cross-reference streams come highest object number first, an incremental
// update having appended both its new objects and its new table to the file.
func (d *Document) trailerCandidates() []Dict {
out := append([]Dict{d.trailer}, scanTrailers(d.buf)...)
nums := make([]int, 0, len(d.xref))
for num := range d.xref {
nums = append(nums, num)
}
slices.Sort(nums)
var streams []Dict
for _, num := range nums {
// An error here leaves a nil object, which is not a stream; the
// rebuild has already been told about anything unreadable.
o, _ := d.Get(Ref{Num: num})
s, ok := ToStream(o)
if !ok {
continue
}
if t, ok := ToName(s.Dict.Get("Type")); ok && t == "XRef" {
streams = append(streams, s.Dict)
}
}
slices.Reverse(streams)
return append(out, streams...)
}
// loadRepairedTrailer finds a trailer that leads to a catalogue: the file's own
// trailer dictionaries first, newest first; then any object that calls itself a
// catalogue; and failing both, a catalogue built over whatever pages survive.
func (d *Document) loadRepairedTrailer() {
previous := d.trailer
for _, tr := range scanTrailers(d.buf) {
d.trailer = tr
if _, err := d.Catalog(); err == nil {
return
}
}
if previous != nil {
d.trailer = previous
if _, err := d.Catalog(); err == nil {
return
}
}
d.trailer = nil
for _, num := range d.objectsOfType("Catalog") {
d.trailer = Dict{"Root": Ref{Num: num}}
if _, err := d.Catalog(); err == nil {
return
}
d.trailer = nil
}
d.synthesiseCatalogue()
}
// objectsOfType lists, in object-number order, the objects whose /Type is the
// given name.
func (d *Document) objectsOfType(want Name) []int {
// The objects that exist are walked, in order, rather than every number
// up to the largest of them. A file may name any object number it likes:
// a 219-byte one in the wild declares object 2147483647, and counting up
// to that is two thousand million map lookups for three objects — which
// is a quarter of a minute of somebody's afternoon for a file that fits
// in a tweet.
nums := make([]int, 0, len(d.xref))
for num := range d.xref {
nums = append(nums, num)
}
slices.Sort(nums)
var out []int
for _, num := range nums {
o, err := d.Get(Ref{Num: num})
if err != nil {
continue
}
dict, ok := ToDict(o)
if !ok {
continue
}
if t, ok := ToName(dict.Get("Type")); ok && t == want {
out = append(out, num)
}
}
return out
}
// synthesiseCatalogue is the last resort for a file whose catalogue and page
// tree did not survive: build one over whatever objects still call themselves
// pages. A truncated file that kept its pages is worth opening; one that kept
// none is not a document.
func (d *Document) synthesiseCatalogue() {
var kids Array
for _, num := range d.objectsOfType("Page") {
kids = append(kids, Ref{Num: num})
}
if len(kids) == 0 {
return
}
d.trailer = Dict{"Root": Dict{
"Type": Name("Catalog"),
"Pages": Dict{
"Type": Name("Pages"),
"Kids": kids,
"Count": Integer(len(kids)),
},
}}
}
// indexObjectStreams adds the objects held inside every object stream the scan
// found, without overwriting an object written directly in the file.
func (d *Document) indexObjectStreams() {
// Two object streams in a damaged file may both claim the same object
// number — a partial rewrite leaves the old one in place beside the new —
// and the loop below lets the first stream walked define it. Walking
// d.xref in map order therefore made the same file read differently from
// one run to the next.
//
// They are walked latest in the file first, which is the rule the header
// scan above already applies to objects written directly: a later
// definition wins, an incremental update having appended it.
nums := make([]int, 0, len(d.xref))
for num := range d.xref {
nums = append(nums, num)
}
slices.Sort(nums)
type located struct {
offset int64
num int
}
var found []located
for _, num := range nums {
o, err := d.Get(Ref{Num: num})
if err != nil {
continue
}
s, ok := ToStream(o)
if !ok {
continue
}
if t, ok := ToName(s.Dict.Get("Type")); ok && t == "ObjStm" {
found = append(found, located{d.xref[num].offset, num})
}
}
slices.SortFunc(found, func(a, b located) int { return cmp.Compare(b.offset, a.offset) })
streams := make([]int, 0, len(found))
for _, f := range found {
streams = append(streams, f.num)
}
for _, num := range streams {
// The stream object is already cached by the pass above, so this
// cannot fail; an empty result simply contributes nothing.
objs, _ := d.objectStream(num)
for objNum, obj := range objs {
if _, ok := d.xref[objNum]; ok {
continue
}
d.xref[objNum] = xrefEntry{kind: 'o', strmNum: num}
d.cache[objNum] = obj
}
}
}
// An objectHeader is one "N G obj" found by scanning.
type objectHeader struct {
num, gen, offset int
}
// scanObjectHeaders finds every indirect object header in the file, in the
// order they appear.
func scanObjectHeaders(b []byte) []objectHeader {
var out []objectHeader
for i := 0; i+3 <= len(b); {
j := bytes.Index(b[i:], []byte("obj"))
if j < 0 {
return out
}
at := i + j
i = at + 3
if at+3 < len(b) && isRegular(b[at+3]) {
continue
}
h, ok := headerBefore(b, at)
if !ok {
continue
}
out = append(out, h)
}
return out
}
// headerBefore reads the "N G" that must precede an obj keyword at at.
func headerBefore(b []byte, at int) (objectHeader, bool) {
p := skipSpaceBack(b, at)
if p == at {
return objectHeader{}, false
}
gen, p, ok := digitsBack(b, p)
if !ok {
return objectHeader{}, false
}
q := skipSpaceBack(b, p)
if q == p {
return objectHeader{}, false
}
num, p, ok := digitsBack(b, q)
if !ok {
return objectHeader{}, false
}
return objectHeader{num: num, gen: gen, offset: p}, true
}
// skipSpaceBack walks back over white-space and returns the new position.
func skipSpaceBack(b []byte, p int) int {
for p > 0 && isSpace(b[p-1]) {
p--
}
return p
}
// digitsBack reads a decimal number that ends at p, going backwards.
func digitsBack(b []byte, p int) (int, int, bool) {
end := p
for p > 0 && b[p-1] >= '0' && b[p-1] <= '9' {
p--
}
if p == end || end-p > 10 {
return 0, p, false
}
v := 0
for _, c := range b[p:end] {
v = v*10 + int(c-'0')
}
return v, p, true
}
// scanTrailers finds the file's trailer dictionaries, newest first.
func scanTrailers(b []byte) []Dict {
var out []Dict
for i := len(b); i > 0; {
j := bytes.LastIndex(b[:i], []byte("trailer"))
if j < 0 {
break
}
i = j
o, _, err := ParseObject(b[j+len("trailer"):])
if err != nil {
continue
}
if tr, ok := ToDict(o); ok {
out = append(out, tr)
}
}
return out
}
vendor/github.com/go-pdfkit/reader/writer.go
Generated Vendored
+432
View File
@@ -0,0 +1,432 @@
package reader
import (
"bytes"
"fmt"
"math"
"sort"
"strconv"
)
// AppendObject writes an object in PDF syntax, appending to dst. Dictionary
// keys are written in order, so the same object always produces the same
// bytes — which is what makes a rewritten file comparable with the one it came
// from.
func AppendObject(dst []byte, o Object) []byte {
switch v := o.(type) {
case nil:
return append(dst, "null"...)
case Null:
return append(dst, "null"...)
case Bool:
if v {
return append(dst, "true"...)
}
return append(dst, "false"...)
case Integer:
return strconv.AppendInt(dst, int64(v), 10)
case Real:
return appendReal(dst, float64(v))
case String:
return appendString(dst, v)
case Name:
return appendName(dst, v)
case Ref:
dst = strconv.AppendInt(dst, int64(v.Num), 10)
dst = append(dst, ' ')
dst = strconv.AppendInt(dst, int64(v.Gen), 10)
return append(dst, " R"...)
case Array:
dst = append(dst, '[')
for i, e := range v {
if i > 0 {
dst = append(dst, ' ')
}
dst = AppendObject(dst, e)
}
return append(dst, ']')
case Dict:
return appendDict(dst, v)
case *Stream:
return appendStream(dst, v)
}
return append(dst, "null"...)
}
// FormatObject renders an object in PDF syntax.
func FormatObject(o Object) []byte { return AppendObject(nil, o) }
// appendReal writes a number the way PDF spells one: no exponent, since the
// format has no notation for it, and no infinity or not-a-number, which it
// cannot represent at all.
func appendReal(dst []byte, f float64) []byte {
if math.IsNaN(f) || math.IsInf(f, 0) {
return append(dst, '0')
}
if f == 0 {
// Negative zero is legal and meaningless, and writing it back out
// would be read as the integer zero, so a rewrite would not be a
// fixpoint. Producers do emit it.
return append(dst, '0')
}
return strconv.AppendFloat(dst, f, 'f', -1, 64)
}
// appendString writes a string in whichever of the two forms a PDF allows is
// shorter. A literal string spells an unprintable byte as a four-character
// octal escape, so text in UTF-16 — which is how a PDF says anything that is
// not Latin-1 — comes out four times its length; the hex form costs two
// characters a byte whatever the byte is.
func appendString(dst []byte, s []byte) []byte {
if literalStringCost(s) > 2*len(s)+2 {
return appendHexString(dst, s)
}
return appendLiteralString(dst, s)
}
// literalStringCost is how many characters the literal form would take.
func literalStringCost(s []byte) int {
n := 2
for _, c := range s {
switch {
case c == '(' || c == ')' || c == '\\' || c == '\n' || c == '\r' || c == '\t':
n += 2
case c < 32 || c > 126:
n += 4
default:
n++
}
}
return n
}
// appendHexString writes the <hex> form.
func appendHexString(dst []byte, s []byte) []byte {
const hex = "0123456789ABCDEF"
dst = append(dst, '<')
for _, c := range s {
dst = append(dst, hex[c>>4], hex[c&15])
}
return append(dst, '>')
}
// appendLiteralString writes the (parenthesised) form, escaping only what has
// to be escaped and rendering anything unprintable as an octal escape.
func appendLiteralString(dst []byte, s []byte) []byte {
dst = append(dst, '(')
for _, c := range s {
switch {
case c == '(' || c == ')' || c == '\\':
dst = append(dst, '\\', c)
case c == '\n':
dst = append(dst, '\\', 'n')
case c == '\r':
dst = append(dst, '\\', 'r')
case c == '\t':
dst = append(dst, '\\', 't')
case c < 32 || c > 126:
dst = append(dst, '\\')
dst = append(dst, '0'+c>>6&7, '0'+c>>3&7, '0'+c&7)
default:
dst = append(dst, c)
}
}
return append(dst, ')')
}
// appendName writes a name, escaping every byte that may not appear in one.
func appendName(dst []byte, n Name) []byte {
dst = append(dst, '/')
const hex = "0123456789ABCDEF"
for i := 0; i < len(n); i++ {
c := n[i]
if !isRegular(c) || c == '#' || c < '!' || c > '~' {
dst = append(dst, '#', hex[c>>4], hex[c&15])
continue
}
dst = append(dst, c)
}
return dst
}
// appendDict writes a dictionary with its keys in order.
func appendDict(dst []byte, d Dict) []byte {
keys := make([]string, 0, len(d))
for k := range d {
keys = append(keys, string(k))
}
sort.Strings(keys)
dst = append(dst, "<<"...)
for _, k := range keys {
dst = appendName(dst, Name(k))
dst = append(dst, ' ')
dst = AppendObject(dst, d[Name(k)])
dst = append(dst, ' ')
}
if len(keys) > 0 {
dst = dst[:len(dst)-1]
}
return append(dst, ">>"...)
}
// appendStream writes a stream, its /Length rewritten to the length of the
// data actually being written.
func appendStream(dst []byte, s *Stream) []byte {
d := Dict{}
for k, v := range s.Dict {
d[k] = v
}
d["Length"] = Integer(len(s.Raw))
dst = appendDict(dst, d)
dst = append(dst, "\nstream\n"...)
dst = append(dst, s.Raw...)
return append(dst, "\nendstream"...)
}
// A Writer builds a PDF file out of objects. Numbers are handed out by
// [Writer.Reserve], objects are written with [Writer.Put], and [Writer.Finish]
// adds the cross-reference table and the trailer.
type Writer struct {
buf bytes.Buffer
offsets map[int]int
packed map[int]packedAt
pending []pendingObject
seen map[int]bool
next int
copied map[*Document]map[int]Ref
err error
pack bool
encrypt *encrypter
}
// NewWriter starts a file with the given version in its header, "1.7" when the
// version is empty. Objects are written one after another and the file ends in
// a cross-reference table, which every reader that has ever existed can read.
func NewWriter(version string) *Writer { return newWriter(version, false) }
// NewPackedWriter starts a file that puts what it can into compressed object
// streams and ends in a cross-reference stream. That is smaller — usually by
// a lot, since a file is mostly small dictionaries — and has been readable
// since PDF 1.5, which is 2003; the version in the header is raised to 1.5 if
// it is lower.
func NewPackedWriter(version string) *Writer { return newWriter(version, true) }
// newWriter starts a file of either kind.
func newWriter(version string, pack bool) *Writer {
if version == "" {
version = "1.7"
}
if pack && version < "1.5" {
version = "1.5"
}
w := &Writer{
offsets: map[int]int{},
packed: map[int]packedAt{},
seen: map[int]bool{},
next: 1,
copied: map[*Document]map[int]Ref{},
pack: pack,
}
fmt.Fprintf(&w.buf, "%%PDF-%s\n", version)
// The four bytes above 127 tell every tool downstream that this file is
// not text, which is what the specification asks for.
w.buf.Write([]byte{'%', 0xE2, 0xE3, 0xCF, 0xD3, '\n'})
return w
}
// Reserve hands out an object number that nothing has been written to yet.
func (w *Writer) Reserve() Ref {
r := Ref{Num: w.next}
w.next++
return r
}
// Put writes an object under a reserved number.
func (w *Writer) Put(ref Ref, o Object) {
if w.seen[ref.Num] {
w.note(fmt.Errorf("reader: object %d written twice", ref.Num))
return
}
w.seen[ref.Num] = true
if ref.Num >= w.next {
w.next = ref.Num + 1
}
// A stream cannot go inside another stream, and only generation zero may
// be packed; everything else can wait to be gathered up at the end.
if _, isStream := o.(*Stream); w.pack && !isStream && ref.Gen == 0 {
w.pending = append(w.pending, pendingObject{ref: ref, obj: o})
return
}
w.writeInline(ref, o)
}
// writeInline writes an object where it stands in the file, compressed and
// then encrypted — in that order, since encrypted bytes do not compress.
func (w *Writer) writeInline(ref Ref, o Object) {
o = w.compressStream(o)
if w.encrypt != nil {
o = w.encrypt.apply(ref.Num, ref.Gen, o)
}
w.emit(ref, o)
}
// writeInlineRaw writes an object that must not be encrypted: the /Encrypt
// dictionary itself, and the cross-reference stream, neither of which a
// reader could decrypt, since it needs them to work out how.
func (w *Writer) writeInlineRaw(ref Ref, o Object) {
w.emit(ref, w.compressStream(o))
}
// compressStream deflates a stream that arrives with no filter of its own,
// when the file is being written to be small.
func (w *Writer) compressStream(o Object) Object {
s, ok := o.(*Stream)
if !ok || !w.pack || s.Dict.Get("Filter").Kind() != KindNull {
return o
}
// Nothing this package generated arrives compressed.
out := &Stream{Dict: cloneDict(s.Dict), Raw: flateCompress(s.Raw)}
out.Dict["Filter"] = Name("FlateDecode")
return out
}
// emit writes an object and remembers where it went.
func (w *Writer) emit(ref Ref, o Object) {
w.offsets[ref.Num] = w.buf.Len()
fmt.Fprintf(&w.buf, "%d %d obj\n", ref.Num, ref.Gen)
w.buf.Write(AppendObject(nil, o))
w.buf.WriteString("\nendobj\n")
}
// cloneDict copies a dictionary one level deep.
func cloneDict(d Dict) Dict {
out := Dict{}
for k, v := range d {
out[k] = v
}
return out
}
// Add reserves a number, writes the object under it and returns the reference.
func (w *Writer) Add(o Object) Ref {
ref := w.Reserve()
w.Put(ref, o)
return ref
}
// Err reports the first thing that went wrong while building the file.
func (w *Writer) Err() error { return w.err }
// note keeps the first error.
func (w *Writer) note(err error) {
if w.err == nil {
w.err = err
}
}
// Finish writes the cross-reference table and the trailer and returns the
// file. /Size is filled in; the caller supplies /Root and whatever else the
// trailer needs.
func (w *Writer) Finish(trailer Dict) ([]byte, error) {
// The caller's dictionary is left as it was; what a file needs to say
// about its own encryption is added to a copy.
trailer = cloneDict(trailer)
for k, v := range w.writeEncryptDict() {
trailer[k] = v
}
if w.pack {
return w.finishWithXrefStream(trailer)
}
high := 0
for num := range w.offsets {
if num > high {
high = num
}
}
start := w.buf.Len()
fmt.Fprintf(&w.buf, "xref\n0 %d\n0000000000 65535 f \n", high+1)
for num := 1; num <= high; num++ {
off, ok := w.offsets[num]
if !ok {
w.buf.WriteString("0000000000 65535 f \n")
continue
}
fmt.Fprintf(&w.buf, "%010d 00000 n \n", off)
}
out := Dict{}
for k, v := range trailer {
out[k] = v
}
out["Size"] = Integer(high + 1)
w.buf.WriteString("trailer\n")
w.buf.Write(AppendObject(nil, out))
fmt.Fprintf(&w.buf, "\nstartxref\n%d\n%%%%EOF\n", start)
if w.encrypt != nil && w.encrypt.err != nil {
w.note(w.encrypt.err)
}
if w.err != nil {
return nil, w.err
}
return w.buf.Bytes(), nil
}
// Copy writes an object from a document into this file, following every
// indirect reference it reaches and renumbering as it goes, so that objects
// from several documents can live side by side. An object already copied from
// the same document keeps the number it was given.
func (w *Writer) Copy(d *Document, o Object) Object {
seen, ok := w.copied[d]
if !ok {
seen = map[int]Ref{}
w.copied[d] = seen
}
return w.copyObject(d, seen, o, 0)
}
// copyObject is Copy's recursion.
func (w *Writer) copyObject(d *Document, seen map[int]Ref, o Object, depth int) Object {
if depth > maxCopyDepth {
w.note(fmt.Errorf("reader: an object graph nested deeper than %d was truncated", maxCopyDepth))
return Null{}
}
switch v := o.(type) {
case Ref:
if to, ok := seen[v.Num]; ok {
return to
}
// The number is reserved before the object is read, so a reference
// back to an ancestor finds it rather than recursing.
to := w.Reserve()
seen[v.Num] = to
src, err := d.Get(v)
if err != nil {
w.note(err)
src = Null{}
}
w.Put(to, w.copyObject(d, seen, src, depth+1))
return to
case Array:
out := make(Array, len(v))
for i, e := range v {
out[i] = w.copyObject(d, seen, e, depth+1)
}
return out
case Dict:
out := Dict{}
for k, e := range v {
out[k] = w.copyObject(d, seen, e, depth+1)
}
return out
case *Stream:
out := &Stream{Dict: Dict{}, Raw: v.Raw}
for k, e := range v.Dict {
out.Dict[k] = w.copyObject(d, seen, e, depth+1)
}
return out
}
return o
}
// maxCopyDepth bounds a copy, since a direct object may nest arbitrarily even
// when no reference repeats.
const maxCopyDepth = 256
vendor/github.com/go-pdfkit/reader/xref.go
Generated Vendored
+303
View File
@@ -0,0 +1,303 @@
package reader
import (
"bytes"
"fmt"
)
// An xrefEntry says where object N lives.
type xrefEntry struct {
kind byte // 'n' at a byte offset, 'o' inside an object stream, 'f' free
offset int64 // kind 'n'
gen int // kind 'n'
strmNum int // kind 'o': the object stream's own number
strmIdx int // kind 'o': the index within it
}
// maxXrefSections bounds a /Prev chain, which a file is free to make a cycle.
const maxXrefSections = 1024
// startxrefWindow is how far back from the end the startxref keyword is looked
// for. The specification allows only a short tail; real files sometimes append
// junk, so the window is generous.
const startxrefWindow = 4096
// findStartxref reads the offset the file's last startxref names.
func (d *Document) findStartxref() (int64, error) {
from := len(d.buf) - startxrefWindow
if from < 0 {
from = 0
}
i := bytes.LastIndex(d.buf[from:], []byte("startxref"))
if i < 0 {
return 0, fmt.Errorf("reader: no startxref in the last %d bytes", startxrefWindow)
}
l := &lexer{buf: d.buf, pos: from + i + len("startxref")}
t, err := l.next()
if err != nil {
return 0, err
}
if t.kind != tokInteger || t.i < 0 || t.i >= int64(len(d.buf)) {
return 0, &SyntaxError{t.pos, "startxref does not name an offset inside the file"}
}
return t.i, nil
}
// loadXref follows the chain of cross-reference sections from the last one
// back through /Prev, taking the newest definition of every object and of
// every trailer entry.
func (d *Document) loadXref() error {
off, err := d.findStartxref()
if err != nil {
return err
}
seen := map[int64]bool{}
for n := 0; off > 0 && n < maxXrefSections; n++ {
if seen[off] {
return fmt.Errorf("reader: cross-reference sections form a cycle at offset %d", off)
}
seen[off] = true
tr, err := d.readXrefSection(off)
if err != nil {
return err
}
d.mergeTrailer(tr)
// A hybrid file keeps a second, stream-shaped section for the objects
// its table deliberately hides from old readers.
if x, ok := ToInt(tr.Get("XRefStm")); ok && x > 0 && !seen[x] {
seen[x] = true
if hy, err := d.readXrefSection(x); err == nil {
d.mergeTrailer(hy)
}
}
p, ok := ToInt(tr.Get("Prev"))
if !ok || p <= 0 {
break
}
off = p
}
if len(d.xref) == 0 {
return fmt.Errorf("reader: the cross-reference table is empty")
}
return nil
}
// mergeTrailer keeps the first value seen for each key, sections being read
// newest first.
func (d *Document) mergeTrailer(tr Dict) {
if d.trailer == nil {
d.trailer = Dict{}
}
for k, v := range tr {
if _, ok := d.trailer[k]; !ok {
d.trailer[k] = v
}
}
}
// setEntry records an entry unless a newer section already defined the object.
func (d *Document) setEntry(num int, e xrefEntry) {
if num < 0 {
return
}
if _, ok := d.xref[num]; !ok {
d.xref[num] = e
}
}
// readXrefSection reads whichever of the two forms is at off and returns its
// trailer dictionary.
func (d *Document) readXrefSection(off int64) (Dict, error) {
if off < 0 || off >= int64(len(d.buf)) {
return nil, fmt.Errorf("reader: cross-reference offset %d is outside the file", off)
}
l := &lexer{buf: d.buf, pos: int(off)}
t, err := l.next()
if err != nil {
return nil, err
}
if t.kind == tokKeyword && string(t.text) == "xref" {
return d.readXrefTable(l)
}
_, obj, _, err := ParseIndirectObject(d.buf[off:], nil)
if err != nil {
return nil, err
}
s, ok := ToStream(obj)
if !ok {
return nil, &SyntaxError{int(off), "neither an xref table nor an xref stream"}
}
return d.readXrefStream(s)
}
// readXrefTable reads the classic subsection form, the xref keyword already
// consumed, up to and including its trailer dictionary.
func (d *Document) readXrefTable(l *lexer) (Dict, error) {
p := &parser{lex: *l}
for {
save := p.lex.pos
t, err := p.lex.next()
if err != nil {
return nil, err
}
if t.kind == tokKeyword && string(t.text) == "trailer" {
o, err := p.parseObject()
if err != nil {
return nil, err
}
tr, ok := ToDict(o)
if !ok {
return nil, &SyntaxError{save, "the trailer is not a dictionary"}
}
return tr, nil
}
if t.kind != tokInteger {
return nil, &SyntaxError{t.pos, "a cross-reference subsection header was expected"}
}
start := int(t.i)
count, err := p.expectUint("a subsection entry count")
if err != nil {
return nil, err
}
for i := 0; i < count; i++ {
if err := d.readXrefTableEntry(p, start+i); err != nil {
return nil, err
}
}
}
}
// readXrefTableEntry reads one "offset generation n|f" line.
func (d *Document) readXrefTableEntry(p *parser, num int) error {
t1, err := p.lex.next()
if err != nil {
return err
}
if t1.kind != tokInteger {
return &SyntaxError{t1.pos, "a cross-reference entry offset was expected"}
}
t2, err := p.lex.next()
if err != nil {
return err
}
if t2.kind != tokInteger {
return &SyntaxError{t2.pos, "a cross-reference entry generation was expected"}
}
t3, err := p.lex.next()
if err != nil {
return err
}
if t3.kind != tokKeyword || len(t3.text) != 1 || (t3.text[0] != 'n' && t3.text[0] != 'f') {
return &SyntaxError{t3.pos, "a cross-reference entry must end in n or f"}
}
if t3.text[0] == 'n' {
d.setEntry(num, xrefEntry{kind: 'n', offset: t1.i, gen: int(t2.i)})
return nil
}
d.setEntry(num, xrefEntry{kind: 'f'})
return nil
}
// readXrefStream reads the /Type /XRef stream form. Its own data is never
// encrypted, so it is decoded without the document's decryptor.
func (d *Document) readXrefStream(s *Stream) (Dict, error) {
data, img, err := Decode(s.Dict, s.Raw, nil)
if err != nil {
return nil, err
}
if img != "" {
return nil, fmt.Errorf("reader: the cross-reference stream is filtered as an image")
}
widths, err := xrefWidths(s.Dict)
if err != nil {
return nil, err
}
index, err := xrefIndex(s.Dict)
if err != nil {
return nil, err
}
row := widths[0] + widths[1] + widths[2]
if row == 0 {
return nil, fmt.Errorf("reader: the cross-reference stream's /W entries are all zero")
}
pos := 0
for i := 0; i+1 < len(index); i += 2 {
for k := 0; k < index[i+1]; k++ {
if pos+row > len(data) {
// A section that claims more rows than it carries still
// contributes the rows it does carry.
return s.Dict, nil
}
f1 := xrefField(data[pos:], widths[0], 1)
f2 := xrefField(data[pos+widths[0]:], widths[1], 0)
f3 := xrefField(data[pos+widths[0]+widths[1]:], widths[2], 0)
pos += row
d.storeXrefStreamEntry(index[i]+k, f1, f2, f3)
}
}
return s.Dict, nil
}
// storeXrefStreamEntry turns one decoded row into an entry.
func (d *Document) storeXrefStreamEntry(num int, f1, f2, f3 int64) {
switch f1 {
case 1:
d.setEntry(num, xrefEntry{kind: 'n', offset: f2, gen: int(f3)})
case 2:
d.setEntry(num, xrefEntry{kind: 'o', strmNum: int(f2), strmIdx: int(f3)})
default:
d.setEntry(num, xrefEntry{kind: 'f'})
}
}
// xrefWidths reads /W, which says how many bytes each of the three fields
// occupies.
func xrefWidths(d Dict) ([3]int, error) {
var w [3]int
arr, ok := ToArray(d.Get("W"))
if !ok || len(arr) < 3 {
return w, fmt.Errorf("reader: the cross-reference stream has no usable /W")
}
for i := 0; i < 3; i++ {
n, ok := ToInt(arr[i])
if !ok || n < 0 || n > 8 {
return w, fmt.Errorf("reader: /W[%d] is not a byte width", i)
}
w[i] = int(n)
}
return w, nil
}
// xrefIndex reads /Index, defaulting to the whole range /Size describes.
func xrefIndex(d Dict) ([]int, error) {
arr, ok := ToArray(d.Get("Index"))
if !ok {
size, ok := ToInt(d.Get("Size"))
if !ok || size < 0 {
return nil, fmt.Errorf("reader: the cross-reference stream has neither /Index nor /Size")
}
return []int{0, int(size)}, nil
}
out := make([]int, 0, len(arr))
for _, e := range arr {
n, ok := ToInt(e)
if !ok || n < 0 {
return nil, fmt.Errorf("reader: /Index holds a value that is not a count")
}
out = append(out, int(n))
}
return out, nil
}
// xrefField reads one big-endian field of the given width, or its default when
// the width is zero.
func xrefField(b []byte, width int, def int64) int64 {
if width == 0 {
return def
}
v := int64(0)
for i := 0; i < width; i++ {
v = v<<8 | int64(b[i])
}
return v
}