kyle/sift
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

P5a plan: PDF documents — seven Ornith tasks with acceptance tests first

Browse Source 
docs/plans/p5a: 01 kind+fetch limits, 02 pdftext child + runner, 03 dcache PDF tables,
04 pdfpool, 05 pdfdoc render (map/pages/grep), 06 api, 07 digest citations, metrics, CLI,
skill, unit, README. Given tests compile against a panic-only skeleton and fail there for the
intended reasons; PDFs for tests are built by internal/pdftest (given), plus two qpdf-encrypted
fixtures.

Owner changes for the run: vendor go-pdfkit/extract v0.3.0 + reader v0.6.0 (pinned via
tools.go until task 02); AGENTS.md, tools/run-plan.sh (enforces a Co-Authored-By trailer for
the model), docs/implementer-log.md, `make gate`; gofmt (whitespace) on clean.go and guard;
TestUnsupportedType uses a zip now that PDFs are supported; requestDeadline 90→210 s and
writeTimeout 120→240 s for the PDF path's worst case.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
kyle
2026-09-25 20:47:38 -07:00
co-authored by Claude Opus 5.5
parent 3f2b78d4aa
commit cfde35397a
115 changed files with 26422 additions and 43 deletions
Download Patch File Download Diff File Expand all files Collapse all files
vendor/github.com/go-pdfkit/reader/.gitignore
Generated
+3
View File
@@ -0,0 +1,3 @@
cov.out
coverage.out
*.pdf
vendor/github.com/go-pdfkit/reader/LICENSE
Generated Vendored
+29
View File
@@ -0,0 +1,29 @@
BSD 3-Clause License
Copyright (c) 2026 the go-pdfkit/reader authors
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:
1. Redistributions of source code must retain the above copyright notice,
this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
3. Neither the name of the copyright holder nor the names of its contributors
may be used to endorse or promote products derived from this software
without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.
vendor/github.com/go-pdfkit/reader/README.md
Generated Vendored
+144
View File
@@ -0,0 +1,144 @@
# reader
[![CI](https://github.com/go-pdfkit/reader/actions/workflows/ci.yml/badge.svg)](https://github.com/go-pdfkit/reader/actions/workflows/ci.yml)
[![Go Reference](https://pkg.go.dev/badge/github.com/go-pdfkit/reader.svg)](https://pkg.go.dev/github.com/go-pdfkit/reader)
[![Go Report Card](https://goreportcard.com/badge/github.com/go-pdfkit/reader)](https://goreportcard.com/report/github.com/go-pdfkit/reader)
[![License: BSD-3-Clause](https://img.shields.io/badge/License-BSD--3--Clause-blue.svg)](LICENSE)
[![Coverage](https://img.shields.io/badge/coverage-100%25-brightgreen.svg)](#testing)
A pure-Go, **zero-C** PDF **reader** — the parsing half of
[go-pdfkit](https://github.com/go-pdfkit). Where
[`pdfkit`](https://github.com/go-pdfkit/pdfkit) authors new PDF 1.7, this
module takes existing PDF bytes apart: lexer, object model, cross-reference
tables and streams, stream filters, the standard security handler, the page
tree, and content streams. It also writes the same object graph back out,
which is what every operation on an existing file needs.
Nothing outside the Go standard library is required, so it builds for
`GOOS=js/wasm` and every 64-bit architecture the fleet targets.
## Status
Landing in waves. This module currently provides the **object layer** and
the **document structure**:
- **Lexer** — the full PDF object syntax: integers and reals in the shapes
producers actually write, literal strings with their escapes, hex strings,
names with `#xx` escapes, arrays, dictionaries, streams, and keywords.
- **Objects** — `Null`, `Bool`, `Integer`, `Real`, `String`, `Name`, `Array`,
`Dict`, `Stream`, and indirect `Ref`, with the `N G R` / `N G obj` lookahead
resolved the way the specification requires.
- **Filters** — `FlateDecode`, `LZWDecode` (with PDF's `EarlyChange`, which the
standard library's LZW does not implement), `ASCIIHexDecode`,
`ASCII85Decode`, `RunLengthDecode`, `Crypt`, plus the PNG and TIFF
predictors. Image filters (`DCTDecode`, `JPXDecode`, `CCITTFaxDecode`,
`JBIG2Decode`) are reported rather than applied, so an image consumer gets
the encoded bytes.
- **Filters that fail** — a chain that cannot be run to the end is not the end
of the stream. `DecodeRecovering` returns what the filters before the failure
produced — the prefix a damaged Flate stream did inflate, not its compressed
bytes — and says so through `Decoded.Recovered` and `Decoded.Cause`. Bytes no
filter decoded arrive in `Decoded.Undecoded`, never in `Decoded.Data`, so a
compressed stream cannot be painted as samples or tokenised as content by a
caller that forgot to check a flag.
`Decode` is the strict reading and refuses such a stream outright, so a
caller that must not act on damaged data says which it wants by which one it
calls. Page content takes the lenient reading, because a page whose content
will not decode is still a page; `PageContentDecoded` reports whether any
salvaging happened. Cross-reference streams take the strict one: a table that
has to be guessed at is worse than no table, and the repair below is the
answer to one.
- **Cross-references** — classic tables, cross-reference **streams**, **object
streams**, `/Prev` chains, and the `/XRefStm` of a hybrid file, newest
definition winning.
- **Repair** — a file whose tables are missing, truncated or simply wrong is
rebuilt by scanning it for object headers, trailers and, failing those, for
a catalogue; a file that kept its pages but lost its catalogue gets one.
This is not an exceptional path: it is what makes a reader usable. The
rebuild establishes the file key before it reads object streams, so a file
whose catalogue lives in an encrypted one can be rebuilt at all; it takes
`/Encrypt` from a cross-reference stream's dictionary when the file has no
trailer keyword to name it in. Where two object streams claim the same
object, the later one in the file wins, the same rule the header scan
applies to objects written directly — so the same damaged file reads the
same way twice. When a rebuild fails, the error says what the rebuild found
and not merely that the tables could not be read.
- **Documents** — `Open`, object resolution with cycle and recursion guards,
the trailer, the catalogue, and the page tree with the four attributes a
page inherits from its ancestors.
- **Encryption** — the standard security handler in every revision, both ways:
reading RC4 at 40 and 128 bits, AESV2, and the AES-256 of `/R 5` and `/R 6`,
with crypt filters, `/Identity` and `/EncryptMetadata`; and writing AES-256
or AES-128, with permissions, a user password and an owner password.
A password is tried as the user password and as the owner password, and
`Open` uses the empty one, so a file protected only against editing opens
with no password at all.
and steps over rubbish rather than losing the operations around it, with
inline images read whole. Where an inline image ends is the one genuinely
ambiguous thing in a content stream, since its data may spell EI itself;
the length is computed from the image's own geometry where it can be, and
otherwise every candidate EI is tried until the data before one actually
decodes.
- **Writing** — the exact inverse of the parser: objects rendered in PDF
syntax with stable dictionary ordering and strings in whichever of the two
forms is shorter, and a `Writer` that copies whole object graphs out of one
or more documents, renumbering as it goes, then lays down a cross-reference
table and a trailer. `NewPackedWriter` instead puts what it can into
compressed **object streams** and ends in a **cross-reference stream**,
which is what makes a file small.
Measured against a corpus of **118 863 real PDFs** — Matplotlib, cairo,
pdfTeX, Ghostscript, Adobe, R, Apache FOP, PDF 1.3 through 1.7 — `Open`
succeeds on **118 833** of them and finds 138 337 pages, in 8 seconds, with no
panics. Of the thirty it refuses, twenty-seven are PNG files with a `.pdf`
extension, one is PostScript, and two are PDFs truncated past recovery.
The one encrypted PDF the corpus holds — `/V 4 /R 4`, written by macOS
Quartz — decrypts to readable metadata and a valid content stream, which is
the only genuinely independent check of the key derivation there is here;
the other revisions are round-tripped against a test encryptor written from
the same algorithms.
The content-stream tokeniser reads **1 536 769 753 operations** across those
138 337 pages in a minute and a half, with no panics, no page failing to
decode, and no operator outside the seventy the format defines — beyond four
`arc` and six `nan` written by a producer that was simply wrong.
Rewriting is checked the same way: every one of the **118 833** files that
open is copied object by object into a new file, which is then re-read and
compared on page count, media boxes and the bytes of every content stream.
**All 118 833 match**, 35.5 GB in and 35.2 GB out, in twenty-eight seconds —
and **all 118 833 match again** when the packed writer is used instead, which
brings the output down to **94.6%** of the input.
Next wave: the operations built on all of this.
## Install
```sh
go get github.com/go-pdfkit/reader
```
## Testing
```sh
go test -covermode=set ./...
```
CI gates on **exact 100% statement coverage**, `go vet`, and a cross-compile
across `linux/{amd64,arm64,riscv64,loong64,ppc64le,s390x}`, `js/wasm`,
`darwin/arm64` and `windows/amd64`.
## The rest of the family
[**reader**](https://github.com/go-pdfkit/reader) reads and writes the format ·
[**ops**](https://github.com/go-pdfkit/ops) is the verbs and the `pdfops` command ·
[**render**](https://github.com/go-pdfkit/render) turns a page into pixels ·
[**pdffont**](https://github.com/go-pdfkit/pdffont) is what a document says about a font ·
[**extract**](https://github.com/go-pdfkit/extract) reads a page back ·
[**coedit**](https://github.com/go-pdfkit/coedit) is a PDF several people edit at once ·
[**app**](https://github.com/go-pdfkit/app) is the browser workbench.
Documentation for all of it: <https://go-pdfkit.github.io/docs/>
## License
BSD-3-Clause — see [LICENSE](LICENSE). Copyright the go-pdfkit/reader authors.
vendor/github.com/go-pdfkit/reader/ccitt.go
Generated Vendored
+579
View File
@@ -0,0 +1,579 @@
package reader
import "fmt"
// CCITT Group 3 and Group 4 fax decoding, as used by /CCITTFaxDecode.
//
// WHY THIS IS HERE
//
// A scanned form is a fax. 67 of the 1 633 real forms in the corpus — the
// eleven issuing bodies, not the vendor test suites — carry a CCITT-encoded
// image, 263 images between them, and 6 of their pages have nothing else on
// them at all. Those six pages drew entirely blank, and a blank page is the
// failure a reader notices before any other.
//
// WHY IT BELONGS IN THE READER RATHER THAN IN A RENDERER
//
// The other filters this package stops at — DCT, JPX, JBIG2 — carry an image
// with its own idea of how many components it has and how deep they are. This
// one does not: it produces bilevel samples, one bit a pixel, rows padded to a
// byte, and the stream dictionary says what they mean. That is a byte stream,
// so it is a filter, and every caller gets it rather than each writing its own.
//
// THE REFERENCE READ BEFORE WRITING THIS
//
// ITU-T Recommendations T.4 (Group 3) and T.6 (Group 4), by way of the tables
// and the changing-element algorithm in golang.org/x/image/ccitt — read, not
// imported, because this package has no dependencies outside the standard
// library and gains none. The code tables below were extracted mechanically
// from that package's gen.go rather than retyped, because a table of 218
// variable-length codes transcribed by hand is a table with a mistake in it.
//
// The algorithm is the one T.6 Figure 1 describes. A row is decoded into one
// byte a pixel — 0xFF white, 0x00 black — and the row before it is the
// reference against which the two-dimensional modes are resolved:
//
// b1 b2
// v v
// prev: BBBBBwwwwwBBBwwwww
// curr: BBBwwwwwBBBBBBwwww
// ^ ^ ^
// a0 a1 a2
//
// a0 is where the pen is; a1 the next colour change to its right on this row;
// b1 the first change on the row above that is to the right of a0 and of the
// opposite colour to it; b2 the next change after b1. Pass mode says a1 is at
// or beyond b2; horizontal mode reads two runs and ignores the row above;
// vertical mode puts a1 at b1 plus an offset between -3 and +3.
// The two-dimensional modes of T.4 Table 1.
const (
modePass = iota
modeH
modeV0
modeVR1
modeVR2
modeVR3
modeVL1
modeVL2
modeVL3
modeExt
)
// A ccittCode is one row of a code table: the value a code names, and the bits
// that name it, most significant first.
type ccittCode struct {
value int
bits string
}
// ccittModeCodes is Table 1 of ITU-T T.4: the two-dimensional modes.
var ccittModeCodes = []ccittCode{
{modePass, "0001"}, {modeH, "001"}, {modeV0, "1"}, {modeVR1, "011"},
{modeVR2, "000011"}, {modeVR3, "0000011"}, {modeVL1, "010"}, {modeVL2, "000010"},
{modeVL3, "0000010"}, {modeExt, "0000001"},
}
// ccittWhiteCodes is Tables 2 and 3 of ITU-T T.4 for a white run: the
// terminating codes 0 to 63, then the make-up codes in steps of 64.
var ccittWhiteCodes = []ccittCode{
{0, "00110101"}, {1, "000111"}, {2, "0111"}, {3, "1000"},
{4, "1011"}, {5, "1100"}, {6, "1110"}, {7, "1111"},
{8, "10011"}, {9, "10100"}, {10, "00111"}, {11, "01000"},
{12, "001000"}, {13, "000011"}, {14, "110100"}, {15, "110101"},
{16, "101010"}, {17, "101011"}, {18, "0100111"}, {19, "0001100"},
{20, "0001000"}, {21, "0010111"}, {22, "0000011"}, {23, "0000100"},
{24, "0101000"}, {25, "0101011"}, {26, "0010011"}, {27, "0100100"},
{28, "0011000"}, {29, "00000010"}, {30, "00000011"}, {31, "00011010"},
{32, "00011011"}, {33, "00010010"}, {34, "00010011"}, {35, "00010100"},
{36, "00010101"}, {37, "00010110"}, {38, "00010111"}, {39, "00101000"},
{40, "00101001"}, {41, "00101010"}, {42, "00101011"}, {43, "00101100"},
{44, "00101101"}, {45, "00000100"}, {46, "00000101"}, {47, "00001010"},
{48, "00001011"}, {49, "01010010"}, {50, "01010011"}, {51, "01010100"},
{52, "01010101"}, {53, "00100100"}, {54, "00100101"}, {55, "01011000"},
{56, "01011001"}, {57, "01011010"}, {58, "01011011"}, {59, "01001010"},
{60, "01001011"}, {61, "00110010"}, {62, "00110011"}, {63, "00110100"},
{64, "11011"}, {128, "10010"}, {192, "010111"}, {256, "0110111"},
{320, "00110110"}, {384, "00110111"}, {448, "01100100"}, {512, "01100101"},
{576, "01101000"}, {640, "01100111"}, {704, "011001100"}, {768, "011001101"},
{832, "011010010"}, {896, "011010011"}, {960, "011010100"}, {1024, "011010101"},
{1088, "011010110"}, {1152, "011010111"}, {1216, "011011000"}, {1280, "011011001"},
{1344, "011011010"}, {1408, "011011011"}, {1472, "010011000"}, {1536, "010011001"},
{1600, "010011010"}, {1664, "011000"}, {1728, "010011011"}, {1792, "00000001000"},
{1856, "00000001100"}, {1920, "00000001101"}, {1984, "000000010010"}, {2048, "000000010011"},
{2112, "000000010100"}, {2176, "000000010101"}, {2240, "000000010110"}, {2304, "000000010111"},
{2368, "000000011100"}, {2432, "000000011101"}, {2496, "000000011110"}, {2560, "000000011111"},
}
// ccittBlackCodes is Tables 2 and 3 of ITU-T T.4 for a black run.
var ccittBlackCodes = []ccittCode{
{0, "0000110111"}, {1, "010"}, {2, "11"}, {3, "10"},
{4, "011"}, {5, "0011"}, {6, "0010"}, {7, "00011"},
{8, "000101"}, {9, "000100"}, {10, "0000100"}, {11, "0000101"},
{12, "0000111"}, {13, "00000100"}, {14, "00000111"}, {15, "000011000"},
{16, "0000010111"}, {17, "0000011000"}, {18, "0000001000"}, {19, "00001100111"},
{20, "00001101000"}, {21, "00001101100"}, {22, "00000110111"}, {23, "00000101000"},
{24, "00000010111"}, {25, "00000011000"}, {26, "000011001010"}, {27, "000011001011"},
{28, "000011001100"}, {29, "000011001101"}, {30, "000001101000"}, {31, "000001101001"},
{32, "000001101010"}, {33, "000001101011"}, {34, "000011010010"}, {35, "000011010011"},
{36, "000011010100"}, {37, "000011010101"}, {38, "000011010110"}, {39, "000011010111"},
{40, "000001101100"}, {41, "000001101101"}, {42, "000011011010"}, {43, "000011011011"},
{44, "000001010100"}, {45, "000001010101"}, {46, "000001010110"}, {47, "000001010111"},
{48, "000001100100"}, {49, "000001100101"}, {50, "000001010010"}, {51, "000001010011"},
{52, "000000100100"}, {53, "000000110111"}, {54, "000000111000"}, {55, "000000100111"},
{56, "000000101000"}, {57, "000001011000"}, {58, "000001011001"}, {59, "000000101011"},
{60, "000000101100"}, {61, "000001011010"}, {62, "000001100110"}, {63, "000001100111"},
{64, "0000001111"}, {128, "000011001000"}, {192, "000011001001"}, {256, "000001011011"},
{320, "000000110011"}, {384, "000000110100"}, {448, "000000110101"}, {512, "0000001101100"},
{576, "0000001101101"}, {640, "0000001001010"}, {704, "0000001001011"}, {768, "0000001001100"},
{832, "0000001001101"}, {896, "0000001110010"}, {960, "0000001110011"}, {1024, "0000001110100"},
{1088, "0000001110101"}, {1152, "0000001110110"}, {1216, "0000001110111"}, {1280, "0000001010010"},
{1344, "0000001010011"}, {1408, "0000001010100"}, {1472, "0000001010101"}, {1536, "0000001011010"},
{1600, "0000001011011"}, {1664, "0000001100100"}, {1728, "0000001100101"}, {1792, "00000001000"},
{1856, "00000001100"}, {1920, "00000001101"}, {1984, "000000010010"}, {2048, "000000010011"},
{2112, "000000010100"}, {2176, "000000010101"}, {2240, "000000010110"}, {2304, "000000010111"},
{2368, "000000011100"}, {2432, "000000011101"}, {2496, "000000011110"}, {2560, "000000011111"},
}
// A ccittTable decodes one of the code tables. Codes are grouped by their
// length, which is the whole of the decoding: read a bit, lengthen the code by
// one, and ask whether a code of that length says anything. A prefix-free code
// cannot answer twice, so the first answer is the only one.
type ccittTable struct {
byLength []map[uint32]int
}
// newCCITTTable groups a table's codes by length. The bits are read from the
// strings the tables are written in, so what the code says and what the
// specification says are the same text.
func newCCITTTable(codes []ccittCode) *ccittTable {
longest := 0
for _, c := range codes {
if len(c.bits) > longest {
longest = len(c.bits)
}
}
t := &ccittTable{byLength: make([]map[uint32]int, longest+1)}
for _, c := range codes {
var v uint32
for _, b := range c.bits {
v <<= 1
if b == '1' {
v |= 1
}
}
n := len(c.bits)
if t.byLength[n] == nil {
t.byLength[n] = map[uint32]int{}
}
t.byLength[n][v] = c.value
}
return t
}
var (
ccittModes = newCCITTTable(ccittModeCodes)
ccittWhite = newCCITTTable(ccittWhiteCodes)
ccittBlack = newCCITTTable(ccittBlackCodes)
)
// A ccittBits reads one bit at a time, most significant first, which is the
// order a fax is written in.
type ccittBits struct {
data []byte
pos int // in bits
}
func (b *ccittBits) next() (uint32, bool) {
if b.pos >= 8*len(b.data) {
return 0, false
}
bit := (b.data[b.pos/8] >> (7 - uint(b.pos%8))) & 1
b.pos++
return uint32(bit), true
}
// align moves to the next byte boundary, which /EncodedByteAlign asks for at
// the start of every row.
func (b *ccittBits) align() { b.pos = (b.pos + 7) &^ 7 }
func (b *ccittBits) exhausted() bool { return b.pos >= 8*len(b.data) }
// restIsFill says whether nothing but zero bits remain. A fax is padded to a
// byte, and often to rather more than a byte, and zeros decode as perfectly
// good two-dimensional modes: pass mode is 0001 and vertical-left-3 is 0000010,
// so a decoder that does not stop here goes on inventing rows that look like
// the last real one. One French form gave 1 636 rows for a 208-row image that
// way, and every one of the surplus rows had plausible ink on it.
func (b *ccittBits) restIsFill() bool {
for i := b.pos; i < 8*len(b.data); i++ {
if b.data[i/8]&(1<<(7-uint(i%8))) != 0 {
return false
}
}
return true
}
// read decodes one code. It returns false at the end of the data or on a code
// the table does not name — the caller decides which of those is fatal, since a
// truncated fax is common and worth showing as far as it goes.
func (t *ccittTable) read(b *ccittBits) (int, bool) {
var code uint32
for n := 1; n < len(t.byLength); n++ {
bit, ok := b.next()
if !ok {
return 0, false
}
code = code<<1 | bit
if v, ok := t.byLength[n][code]; ok {
return v, true
}
}
return 0, false
}
// skipEOL consumes an end-of-line code, 000000000001, if one is next, and any
// fill bits before it. A Group 3 fax may have one at the end of every row, may
// have none at all, and may have several together at the end; all three are
// found in the corpus, so none of them may be an error.
func (b *ccittBits) skipEOL() bool {
start := b.pos
zeros := 0
for {
bit, ok := b.next()
if !ok {
b.pos = start
return false
}
if bit == 0 {
zeros++
continue
}
if zeros >= 11 {
return true
}
b.pos = start
return false
}
}
// ccittParams are the decode parameters of /CCITTFaxDecode, with the defaults
// the specification gives.
type ccittParams struct {
k int // <0 Group 4, 0 Group 3 one-dimensional, >0 Group 3 mixed
columns int // default 1728
rows int // 0 means as many as the data holds
blackIs1 bool // false: a 0 bit is black
encodedByteAlign bool
endOfBlock bool // default true: the data ends with a block marker
}
func ccittParamsOf(parm Dict, r Resolver) ccittParams {
return ccittParams{
k: intParm(parm, "K", 0, r),
columns: intParm(parm, "Columns", 1728, r),
rows: intParm(parm, "Rows", 0, r),
blackIs1: boolParm(parm, "BlackIs1", false, r),
encodedByteAlign: boolParm(parm, "EncodedByteAlign", false, r),
endOfBlock: boolParm(parm, "EndOfBlock", true, r),
}
}
// endOfBlockEOLs is how many end-of-line codes in a row mean the data has
// ended: two for Group 4, which has none between its rows, and six for Group 3,
// where every row may legitimately be followed by one.
func endOfBlockEOLs(k int) int {
if k < 0 {
return 2
}
return 6
}
// maxCCITTPixels bounds what one image may decode to, the way maxDecodedSize
// bounds a Flate stream. A fax names its own width and height in the decode
// parameters, so a file can ask for an image of any size it likes; 2^28 pixels
// is more than four times A4 at 600 dots to the inch.
var maxCCITTPixels = 1 << 28
// ccittDecode turns a fax into bilevel samples: one bit a pixel, each row
// padded to a byte boundary, which is what every other image filter in this
// package produces and what a stream dictionary's /Width, /Height and
// /BitsPerComponent describe.
func ccittDecode(data []byte, p ccittParams) ([]byte, error) {
if p.columns <= 0 {
return nil, fmt.Errorf("reader: /CCITTFaxDecode names %d columns", p.columns)
}
if p.rows < 0 {
return nil, fmt.Errorf("reader: /CCITTFaxDecode names %d rows", p.rows)
}
if p.rows > 0 && p.columns > maxCCITTPixels/p.rows {
return nil, fmt.Errorf("reader: /CCITTFaxDecode names %d by %d pixels, "+
"past the limit of %d", p.columns, p.rows, maxCCITTPixels)
}
b := &ccittBits{data: data}
stride := (p.columns + 7) / 8
curr := make([]byte, p.columns)
prev := []byte(nil)
out := []byte(nil)
for row := 0; p.rows == 0 || row < p.rows; row++ {
if p.rows == 0 && len(out)/stride*p.columns >= maxCCITTPixels {
return nil, fmt.Errorf("reader: /CCITTFaxDecode ran past %d pixels "+
"without naming how many rows it has", maxCCITTPixels)
}
if p.encodedByteAlign {
b.align()
}
// A Group 3 row may be introduced by an end-of-line code. Consuming it
// here rather than after the row means a file that has them and a file
// that does not are read the same way — and counting them is how the
// end of the data announces itself, since the block marker is nothing
// but end-of-line codes in a row.
eols := 0
for b.skipEOL() {
eols++
}
if p.endOfBlock && eols >= endOfBlockEOLs(p.k) {
break
}
if b.exhausted() || b.restIsFill() {
break
}
twoDimensional := p.k < 0
if p.k > 0 {
// In mixed mode a bit after the end-of-line says which the row is:
// 1 for one-dimensional, 0 for two.
// The data was checked for bits a moment ago, so this cannot
// fail; the value is taken rather than the error handled, because
// a branch that cannot be reached cannot be tested.
bit, _ := b.next()
twoDimensional = bit == 0
}
if err := ccittRow(b, curr, prev, twoDimensional); err != nil {
if len(out) == 0 {
return nil, err
}
// A truncated fax is worth showing as far as it got: the rows
// already decoded are real, and refusing them turns a damaged
// scan into a blank page. Leaving the loop rather than returning
// here matters — the padding below is what makes the answer as
// long as /Rows promised, and a fuzz case of two bytes found that
// returning early handed the caller one row where it had asked
// for four, with no error to say so.
break
}
out = append(out, ccittPack(curr, stride, p.blackIs1)...)
if prev == nil {
prev = make([]byte, p.columns)
}
copy(prev, curr)
}
// A file that says how many rows it has gets that many, padded with white
// if the data ran out, so the samples match the /Height the dictionary
// promises rather than being short by a row.
if p.rows > 0 {
white := make([]byte, stride)
if !p.blackIs1 {
for i := range white {
white[i] = 0xFF
}
}
for len(out) < p.rows*stride {
out = append(out, white...)
}
}
return out, nil
}
// ccittPack turns a row of one byte a pixel into one bit a pixel. With
// /BlackIs1 false — the default — a 0 bit is black, so white becomes a 1.
func ccittPack(row []byte, stride int, blackIs1 bool) []byte {
out := make([]byte, stride)
for x, v := range row {
white := v != 0
if white != blackIs1 {
out[x/8] |= 1 << (7 - uint(x%8))
}
}
return out
}
// ccittRow decodes one row into curr, one byte a pixel: 0xFF white, 0x00 black.
func ccittRow(b *ccittBits, curr, prev []byte, twoDimensional bool) error {
for i := range curr {
curr[i] = 0
}
at, white, first := 0, true, true
for at < len(curr) {
if !twoDimensional {
n, err := ccittRun(b, white)
if err != nil {
return err
}
if at+n > len(curr) {
return fmt.Errorf("reader: a fax row of %d pixels named %d",
len(curr), at+n)
}
fill(curr[at:at+n], white)
at += n
white = !white
first = false
continue
}
mode, ok := ccittModes.read(b)
if !ok {
return fmt.Errorf("reader: a fax row named no mode this reader knows")
}
switch mode {
case modePass:
// ccittFindB starts where the pen is and only walks forward,
// stopping at the end of the row, so b2 is always a pixel of this
// row at or after the pen. There is nothing to check.
b2 := ccittFindB(prev, curr, at, white, first, true)
fill(curr[at:b2], white)
at = b2
case modeH:
// Two runs, of the current colour and then the other, neither of
// them looking at the row above.
for i := 0; i < 2; i++ {
n, err := ccittRun(b, white)
if err != nil {
return err
}
if at+n > len(curr) {
return fmt.Errorf("reader: a fax row of %d pixels named %d",
len(curr), at+n)
}
fill(curr[at:at+n], white)
at += n
white = !white
}
// Horizontal mode reads a pair, so the two flips above have put
// the colour back where it started; there is nothing to undo.
case modeV0, modeVR1, modeVR2, modeVR3, modeVL1, modeVL2, modeVL3:
a1 := ccittFindB(prev, curr, at, white, first, false) + ccittOffset(mode)
if a1 < at || a1 > len(curr) {
return fmt.Errorf("reader: a fax vertical mode named pixel %d of %d",
a1, len(curr))
}
fill(curr[at:a1], white)
at = a1
white = !white
default:
return fmt.Errorf("reader: a fax named an extension mode, which " +
"this reader does not decode")
}
first = false
}
return nil
}
// ccittOffset is how far a vertical mode moves a1 from b1.
func ccittOffset(mode int) int {
switch mode {
case modeVR1:
return 1
case modeVR2:
return 2
case modeVR3:
return 3
case modeVL1:
return -1
case modeVL2:
return -2
case modeVL3:
return -3
}
return 0
}
// ccittRun reads one run length, which is a sequence of make-up codes ending in
// a terminating code below 64.
func ccittRun(b *ccittBits, white bool) (int, error) {
table := ccittBlack
if white {
table = ccittWhite
}
total := 0
for {
n, ok := table.read(b)
if !ok {
return 0, fmt.Errorf("reader: a fax run named no length this reader knows")
}
total += n
if total > maxCCITTPixels {
return 0, fmt.Errorf("reader: a fax named a run of %d pixels", total)
}
if n <= 63 {
return total, nil
}
}
}
// ccittFindB finds b1, or b2 when second is set: the changing elements on the
// row above, as T.6 Figure 1 defines them.
func ccittFindB(prev, curr []byte, at int, white, first, second bool) int {
// The row above the first row is implicitly all white, so it has no
// changing elements and both b values are at the end of the row.
if prev == nil {
return len(curr)
}
i := at
if first {
// a0 is implicitly one pixel before the row, on white. b1 is the first
// black pixel above; b2 the first white pixel after that.
for i < len(prev) && prev[i] != 0 {
i++
}
if second {
for i < len(prev) && prev[i] == 0 {
i++
}
}
return i
}
// Walk past the run above that is of the opposite colour to the pen, then
// past the run of the pen's own colour: what follows is b1.
opposite := byte(0xFF)
if white {
opposite = 0
}
for i < len(prev) && prev[i] == opposite {
i++
}
same := ^opposite
for i < len(prev) && prev[i] == same {
i++
}
if second {
for i < len(prev) && prev[i] == opposite {
i++
}
}
return i
}
// boolParm reads a boolean decode parameter, falling back to its default.
func boolParm(parm Dict, key Name, def bool, r Resolver) bool {
if parm == nil {
return def
}
v, err := Resolve(parm.Get(key), r)
if err != nil {
return def
}
if b, ok := v.(Bool); ok {
return bool(b)
}
return def
}
// fill paints a run of pixels, one byte each.
func fill(row []byte, white bool) {
v := byte(0)
if white {
v = 0xFF
}
for i := range row {
row[i] = v
}
}
vendor/github.com/go-pdfkit/reader/content.go
Generated Vendored
+339
View File
@@ -0,0 +1,339 @@
package reader
import (
"bytes"
"fmt"
)
// An Operation is one step of a content stream: an operator and the operands
// that precede it. An inline image is reported as the operator "BI" with its
// dictionary and data attached, since the three keywords it spans are one
// thing.
type Operation struct {
Operator string
Operands []Object
Image *InlineImage
}
// An InlineImage is the dictionary and the still-encoded bytes of a BI … ID …
// EI sequence. Its dictionary uses the abbreviated keys inline images are
// written with; [InlineImage.Expanded] gives the ordinary spelling.
type InlineImage struct {
Dict Dict
Raw []byte
}
// inlineKeys maps the abbreviations an inline image dictionary uses to the
// names their long form would have.
var inlineKeys = map[Name]Name{
"BPC": "BitsPerComponent",
"CS": "ColorSpace",
"D": "Decode",
"DP": "DecodeParms",
"F": "Filter",
"H": "Height",
"IM": "ImageMask",
"I": "Interpolate",
"W": "Width",
"L": "Length",
}
// inlineColourSpaces maps the abbreviated colour space names.
var inlineColourSpaces = map[Name]Name{
"G": "DeviceGray",
"RGB": "DeviceRGB",
"CMYK": "DeviceCMYK",
"I": "Indexed",
}
// Expanded returns the image's dictionary with the abbreviated keys and colour
// space names written out, so it can be read like any image XObject.
//
// A dictionary may say the same thing twice and disagree with itself: /W 20
// beside /Width 10, /CS /RGB beside /ColorSpace /3chanRGB. The specification
// permits both spellings in an inline image and says nothing about which wins,
// and the public implementations disagree — pdf.js asks for the abbreviation
// and falls back to the written-out name, MuPDF asks the other way round.
//
// The abbreviation wins here. Two reasons: it is the spelling Table 93 gives
// for an inline image, the written-out name being the tolerated alias; and
// safedocs' own fixture for this case marks the abbreviation as the line to
// remove "to see the effect", which is a statement about what the reference
// viewer does.
//
// What is not defensible is the answer changing between runs. Expanding in one
// pass over the map made the winner depend on Go's map iteration order, which
// is deliberately random: the same file drew differently each time it was
// opened, with no way for a caller to notice. Two passes settle it — the
// written-out names first, then the abbreviations over them — and no order
// within either pass can matter, because the names in each are distinct.
func (im *InlineImage) Expanded() Dict {
out := Dict{}
for k, v := range im.Dict {
if _, abbreviated := inlineKeys[k]; abbreviated {
continue
}
out[k] = expandColourSpace(k, v)
}
for k, v := range im.Dict {
long, abbreviated := inlineKeys[k]
if !abbreviated {
continue
}
out[long] = expandColourSpace(long, v)
}
return out
}
// expandColourSpace writes out an abbreviated colour space name, which is the
// one value that is abbreviated as well as its key.
func expandColourSpace(k Name, v Object) Object {
if k != "ColorSpace" {
return v
}
n, ok := ToName(v)
if !ok {
return v
}
if long, ok := inlineColourSpaces[n]; ok {
return long
}
return v
}
// A ContentScanner walks a content stream one operation at a time. A stream
// with rubbish in it yields the operations around the rubbish rather than
// nothing at all, which is what a renderer needs; [ContentScanner.Err] reports
// the first thing that did not parse.
type ContentScanner struct {
lex lexer
pending []Object
err error
}
// NewContentScanner reads operations from a decoded content stream.
func NewContentScanner(data []byte) *ContentScanner {
return &ContentScanner{lex: lexer{buf: data}}
}
// Err reports the first malformed token the scan stepped over, if any.
func (s *ContentScanner) Err() error { return s.err }
// Next returns the next operation, and false once the stream is exhausted.
func (s *ContentScanner) Next() (Operation, bool) {
for {
t, err := s.lex.next()
if err != nil {
s.note(err)
// Step over the byte that would not parse and carry on.
if s.lex.pos < len(s.lex.buf) {
s.lex.pos++
continue
}
return Operation{}, false
}
if t.kind == tokEOF {
return Operation{}, false
}
if t.kind != tokKeyword {
s.operand(t)
continue
}
switch string(t.text) {
case "true":
s.pending = append(s.pending, Bool(true))
continue
case "false":
s.pending = append(s.pending, Bool(false))
continue
case "null":
s.pending = append(s.pending, Null{})
continue
case "BI":
img, err := s.inlineImage()
if err != nil {
s.note(err)
return Operation{}, false
}
return s.emit("BI", img), true
}
return s.emit(string(t.text), nil), true
}
}
// operand parses one operand, an already-read token starting it.
func (s *ContentScanner) operand(t token) {
p := &parser{lex: s.lex}
o, err := p.parseFrom(t)
s.lex = p.lex
if err != nil {
s.note(err)
if s.lex.pos < len(s.lex.buf) {
s.lex.pos++
}
return
}
s.pending = append(s.pending, o)
}
// emit finishes an operation and clears the operands waiting for it.
func (s *ContentScanner) emit(op string, img *InlineImage) Operation {
out := Operation{Operator: op, Operands: s.pending, Image: img}
s.pending = nil
return out
}
// note keeps the first error the scan met.
func (s *ContentScanner) note(err error) {
if s.err == nil {
s.err = err
}
}
// inlineImage reads the dictionary after BI and the data after ID.
func (s *ContentScanner) inlineImage() (*InlineImage, error) {
d := Dict{}
for {
t, err := s.lex.next()
if err != nil {
return nil, err
}
if t.kind == tokEOF {
return nil, &SyntaxError{t.pos, "an inline image has no ID"}
}
if t.kind == tokKeyword && string(t.text) == "ID" {
break
}
if t.kind != tokName {
return nil, &SyntaxError{t.pos, "an inline image key is not a name"}
}
p := &parser{lex: s.lex}
v, err := p.parseObject()
s.lex = p.lex
if err != nil {
return nil, err
}
d[Name(t.text)] = v
}
// Exactly one white-space byte separates ID from the data.
if s.lex.pos < len(s.lex.buf) && isSpace(s.lex.buf[s.lex.pos]) {
s.lex.pos++
}
start := s.lex.pos
end, err := inlineImageEnd(s.lex.buf, start, d)
if err != nil {
return nil, err
}
raw := s.lex.buf[start:end]
s.lex.pos = end
// Step over the EI that follows.
for s.lex.pos < len(s.lex.buf) && isSpace(s.lex.buf[s.lex.pos]) {
s.lex.pos++
}
if bytes.HasPrefix(s.lex.buf[s.lex.pos:], []byte("EI")) {
s.lex.pos += 2
}
return &InlineImage{Dict: d, Raw: raw}, nil
}
// Operations tokenises a whole content stream. The error it returns describes
// what did not parse; the operations it returns are still worth having.
func Operations(data []byte) ([]Operation, error) {
s := NewContentScanner(data)
var out []Operation
for {
op, ok := s.Next()
if !ok {
return out, s.Err()
}
out = append(out, op)
}
}
// PageContent returns the decoded content stream of the i'th page, counting
// from one. A page whose /Contents is an array has its streams joined, as the
// specification requires, with a newline between them.
//
// A content stream whose filters cannot be applied contributes what could be
// salvaged from it rather than failing the page, which is what a renderer
// needs; the error is for a page that is not there at all.
// [Document.PageContentDecoded] says whether any salvaging happened.
func (d *Document) PageContent(i int) ([]byte, error) {
dec, err := d.PageContentDecoded(i)
return dec.Data, err
}
// PageContentDecoded is [Document.PageContent] with the outcome of the decode
// attached: [Decoded.Recovered] reports that at least one of the page's content
// streams could not be decoded cleanly, and [Decoded.Cause] says why.
func (d *Document) PageContentDecoded(i int) (Decoded, error) {
page, err := d.Page(i)
if err != nil {
return Decoded{}, err
}
return d.contentOf(page)
}
// contentOf decodes a page's /Contents, whether one stream or several.
func (d *Document) contentOf(page Dict) (Decoded, error) {
o, err := d.Resolve(page.Get("Contents"))
if err != nil {
return Decoded{}, err
}
switch v := o.(type) {
case *Stream:
return d.decodedContent(v), nil
case Array:
var out Decoded
for _, e := range v {
eo, err := d.Resolve(e)
if err != nil {
return Decoded{}, err
}
s, ok := ToStream(eo)
if !ok {
continue
}
part := d.decodedContent(s)
if part.Recovered && !out.Recovered {
out.Recovered, out.Cause, out.Filter = true, part.Cause, part.Filter
out.Undecoded = part.Undecoded
}
if len(part.Data) == 0 {
continue
}
if len(out.Data) > 0 {
out.Data = append(out.Data, '\n')
}
out.Data = append(out.Data, part.Data...)
}
return out, nil
}
return Decoded{}, nil
}
// decodedContent decodes one content stream. An image filter has no business
// being there, so the bytes it holds are not content: they are reported as
// salvage, not handed over as if they could be tokenised.
func (d *Document) decodedContent(s *Stream) Decoded {
dec := d.DecodeStreamRecovering(s)
if dec.Image != "" {
return Decoded{
Undecoded: dec.Data,
Recovered: true,
Filter: dec.Image,
Cause: fmt.Errorf("reader: a content stream is filtered as an image (/%s)", dec.Image),
}
}
return dec
}
// PageOperations tokenises the i'th page's content stream, counting from one.
func (d *Document) PageOperations(i int) ([]Operation, error) {
data, err := d.PageContent(i)
if err != nil {
return nil, err
}
ops, _ := Operations(data)
return ops, nil
}
vendor/github.com/go-pdfkit/reader/crypt.go
Generated Vendored
+572
View File
@@ -0,0 +1,572 @@
package reader
import (
"bytes"
"crypto/aes"
"crypto/cipher"
"crypto/md5"
"crypto/rc4"
"crypto/sha256"
"crypto/sha512"
"errors"
"fmt"
"hash"
)
// ErrWrongPassword is returned when a file is encrypted and neither the
// password given nor the empty one opens it.
var ErrWrongPassword = errors.New("reader: the password does not open this file")
// ErrUnsupportedEncryption is returned for a security handler this package
// does not implement — anything other than the standard one.
var ErrUnsupportedEncryption = errors.New("reader: unsupported security handler")
// pad is the 32-byte string every pre-2.0 password is padded with.
var pad = []byte{
0x28, 0xBF, 0x4E, 0x5E, 0x4E, 0x75, 0x8A, 0x41, 0x64, 0x00, 0x4E, 0x56,
0xFF, 0xFA, 0x01, 0x08, 0x2E, 0x2E, 0x00, 0xB6, 0xD0, 0x68, 0x3E, 0x80,
0x2F, 0x0C, 0xA9, 0xFE, 0x64, 0x53, 0x69, 0x7A,
}
// cryptMethod is how one class of data — strings or streams — is protected.
type cryptMethod uint8
const (
cryptNone cryptMethod = iota // /Identity: not encrypted at all
cryptRC4
cryptAESV2 // 128-bit AES, per-object key
cryptAESV3 // 256-bit AES, the file key used directly
)
// A decryptor holds the file encryption key and how to apply it.
type decryptor struct {
key []byte
strings cryptMethod
streams cryptMethod
revision int
perm Permissions
owner bool // the password given was the owner's, not the user's
skipObj int // the /Encrypt dictionary's own object number, never decrypted
skipKnown bool // whether skipObj is meaningful
}
// A Protection is what a file's security handler says about it: how it is
// encrypted, and what a reader that opened it with the user password may do.
type Protection struct {
// Method names the algorithm the way a person would say it: "RC4-40",
// "RC4-128", "AES-128", "AES-256", or "none" for a file that declares an
// /Encrypt dictionary and then encrypts nothing with it.
Method string
// Revision is the standard security handler's revision: 2, 3, 4, 5 or 6.
Revision int
// Permissions is what the file grants whoever opened it with the user
// password.
Permissions Permissions
// Owner is true when the password the file was opened with was the
// owner's, in which case the permissions do not apply to this reader.
Owner bool
}
// Protection reports how the file is protected, and false when it is not
// protected at all. A document that opened has already been decrypted; this
// says what it said about itself on the way.
func (d *Document) Protection() (Protection, bool) {
if d.decrypt == nil {
return Protection{}, false
}
return Protection{
Method: d.decrypt.methodName(),
Revision: d.decrypt.revision,
Permissions: d.decrypt.perm,
Owner: d.decrypt.owner,
}, true
}
// methodName says which algorithm protects the streams, which is the one that
// matters: it is where the content is.
func (dec *decryptor) methodName() string {
switch dec.streams {
case cryptAESV3:
return "AES-256"
case cryptAESV2:
return "AES-128"
case cryptRC4:
return fmt.Sprintf("RC4-%d", len(dec.key)*8)
}
return "none"
}
// Encrypted reports whether the file declares an /Encrypt dictionary.
func (d *Document) Encrypted() bool {
return d.trailer != nil && d.trailer.Get("Encrypt").Kind() != KindNull
}
// setUpDecryption reads /Encrypt and derives the file key. It is called once,
// before any string or stream is handed out.
func (d *Document) setUpDecryption(password string) error {
if !d.Encrypted() {
return nil
}
if ref, ok := d.trailer.Get("Encrypt").(Ref); ok {
d.encryptNum, d.encryptKnown = ref.Num, true
}
o, err := d.Resolve(d.trailer.Get("Encrypt"))
if err != nil {
return err
}
enc, ok := ToDict(o)
if !ok {
return fmt.Errorf("reader: /Encrypt is a %s, not a dictionary", o.Kind())
}
id := d.firstID()
dec, err := newDecryptor(enc, id, password, d.Get)
if err != nil {
return err
}
dec.skipObj, dec.skipKnown = d.encryptNum, d.encryptKnown
d.decrypt = dec
return nil
}
// firstID returns the first element of the trailer's /ID, which takes part in
// the pre-2.0 key derivation. A file without one contributes nothing.
func (d *Document) firstID() []byte {
arr, ok := ToArray(d.trailer.Get("ID"))
if !ok || len(arr) == 0 {
return nil
}
s, ok := ToString(arr[0])
if !ok {
return nil
}
return s
}
// newDecryptor derives the file encryption key from a password.
func newDecryptor(enc Dict, id []byte, password string, r Resolver) (*decryptor, error) {
if f, ok := ToName(resolved(enc, "Filter", r)); ok && f != "Standard" {
return nil, ErrUnsupportedEncryption
}
v := int(intOf(resolved(enc, "V", r), 0))
rev := int(intOf(resolved(enc, "R", r), 0))
if v == 0 || rev == 0 {
return nil, fmt.Errorf("reader: /Encrypt has no usable /V and /R")
}
length := int(intOf(resolved(enc, "Length", r), 40))
if length < 40 || length > 256 || length%8 != 0 {
length = 40
}
dec := &decryptor{revision: rev}
if err := dec.readMethods(enc, v, r); err != nil {
return nil, err
}
owner, _ := ToString(resolved(enc, "O", r))
user, _ := ToString(resolved(enc, "U", r))
perm := int32(intOf(resolved(enc, "P", r), 0))
metadata := true
if b, ok := ToBool(resolved(enc, "EncryptMetadata", r)); ok {
metadata = b
}
dec.perm = Permissions(uint32(perm)) & AllPermissions
if dec.streams == cryptNone && dec.strings == cryptNone {
// /StmF and /StrF both name the identity crypt filter, so nothing in
// the document body is encrypted and no key is needed to read it. A
// file like that is opened without validating the password: refusing
// it would refuse a document whose every byte is already plain, which
// is what every other reader shows. Only /EFF — the embedded files —
// would need the key, and this reader does not hand those out.
//
// Nothing was authenticated, so dec.owner stays false and Protection
// reports the method as "none": a caller can see exactly what it got.
return dec, nil
}
if rev >= 5 {
key, asOwner, err := deriveKeyR5(enc, password, r)
if err != nil {
return nil, err
}
dec.key, dec.owner = key, asOwner
return dec, nil
}
n := length / 8
if rev == 2 {
n = 5
}
key, asOwner, err := deriveKeyLegacy(password, owner, user, id, perm, n, rev, metadata)
if err != nil {
return nil, err
}
dec.key, dec.owner = key, asOwner
return dec, nil
}
// readMethods works out how strings and streams are protected. Before /V 4
// there is one method for everything; from /V 4 the crypt filters in /CF are
// named by /StmF and /StrF.
func (dec *decryptor) readMethods(enc Dict, v int, r Resolver) error {
if v < 4 {
dec.strings, dec.streams = cryptRC4, cryptRC4
return nil
}
cf, _ := ToDict(resolved(enc, "CF", r))
pick := func(key Name) (cryptMethod, error) {
name, ok := ToName(resolved(enc, key, r))
if !ok || name == "Identity" {
return cryptNone, nil
}
f, ok := ToDict(resolved(cf, name, r))
if !ok {
return cryptNone, nil
}
switch m, _ := ToName(resolved(f, "CFM", r)); m {
case "V2":
return cryptRC4, nil
case "AESV2":
return cryptAESV2, nil
case "AESV3":
return cryptAESV3, nil
case "None":
return cryptNone, nil
default:
return cryptNone, fmt.Errorf("reader: unsupported crypt filter method /%s", m)
}
}
var err error
if dec.streams, err = pick("StmF"); err != nil {
return err
}
dec.strings, err = pick("StrF")
return err
}
// deriveKeyLegacy is the pre-2.0 key derivation, trying the password as the
// user password and then as the owner password.
func deriveKeyLegacy(password string, owner, user, id []byte, perm int32, n, rev int, metadata bool) (key []byte, asOwner bool, err error) {
for _, candidate := range legacyCandidates(password, owner, n, rev) {
k := legacyFileKey(candidate.padded, owner, id, perm, n, rev, metadata)
if legacyUserKeyMatches(k, user, id, rev) {
return k, candidate.owner, nil
}
}
return nil, false, ErrWrongPassword
}
// A legacyCandidate is one padded password to try, and whether reaching it
// meant knowing the owner's rather than the user's.
type legacyCandidate struct {
padded []byte
owner bool
}
// legacyCandidates lists the padded passwords worth trying: the one given, the
// empty one, and the user password the owner password unlocks.
func legacyCandidates(password string, owner []byte, n, rev int) []legacyCandidate {
out := []legacyCandidate{{padded: padPassword([]byte(password))}}
if password != "" {
out = append(out, legacyCandidate{padded: padPassword(nil)})
}
if u := userFromOwner([]byte(password), owner, n, rev); u != nil {
out = append(out, legacyCandidate{padded: u, owner: true})
}
return out
}
// padPassword truncates or pads a password to the 32 bytes the algorithm wants.
func padPassword(p []byte) []byte {
out := make([]byte, 32)
k := copy(out, p)
copy(out[k:], pad)
return out
}
// legacyFileKey is algorithm 2: the file encryption key from a padded password.
func legacyFileKey(padded, owner, id []byte, perm int32, n, rev int, metadata bool) []byte {
h := md5.New()
h.Write(padded)
h.Write(owner)
h.Write([]byte{byte(perm), byte(perm >> 8), byte(perm >> 16), byte(perm >> 24)})
h.Write(id)
if rev >= 4 && !metadata {
h.Write([]byte{0xFF, 0xFF, 0xFF, 0xFF})
}
key := h.Sum(nil)
if rev >= 3 {
for i := 0; i < 50; i++ {
sum := md5.Sum(key[:n])
key = sum[:]
}
}
return key[:n]
}
// legacyUserKeyMatches is algorithm 6: does this key reproduce /U?
func legacyUserKeyMatches(key, user, id []byte, rev int) bool {
if rev == 2 {
return bytes.Equal(rc4Bytes(key, pad), user)
}
h := md5.New()
h.Write(pad)
h.Write(id)
x := rc4Bytes(key, h.Sum(nil))
for i := 1; i <= 19; i++ {
x = rc4Bytes(xorKey(key, i), x)
}
return len(user) >= 16 && bytes.Equal(x, user[:16])
}
// userFromOwner is algorithm 7: recover the padded user password from /O using
// the owner password.
func userFromOwner(password, owner []byte, n, rev int) []byte {
if len(owner) < 32 {
return nil
}
sum := md5.Sum(padPassword(password))
key := sum[:]
if rev >= 3 {
for i := 0; i < 50; i++ {
s := md5.Sum(key)
key = s[:]
}
}
key = key[:n]
if rev == 2 {
return rc4Bytes(key, owner[:32])
}
x := owner[:32]
for i := 19; i >= 0; i-- {
x = rc4Bytes(xorKey(key, i), x)
}
return x
}
// xorKey returns key with every byte exclusive-ored with v, which is how both
// password algorithms walk their nineteen rounds.
func xorKey(key []byte, v int) []byte {
out := make([]byte, len(key))
for i := range key {
out[i] = key[i] ^ byte(v)
}
return out
}
// deriveKeyR5 is the PDF 2.0 derivation, /R 5 and /R 6: the password is
// validated against a salted hash and then unwraps the file key.
func deriveKeyR5(enc Dict, password string, r Resolver) (key []byte, asOwner bool, err error) {
user, _ := ToString(resolved(enc, "U", r))
userE, _ := ToString(resolved(enc, "UE", r))
owner, _ := ToString(resolved(enc, "O", r))
ownerE, _ := ToString(resolved(enc, "OE", r))
rev := int(intOf(resolved(enc, "R", r), 6))
if len(user) < 48 {
return nil, false, fmt.Errorf("reader: /U is %d bytes, not the 48 this revision needs", len(user))
}
pw := []byte(password)
for _, candidate := range [][]byte{pw, nil} {
if candidate == nil && password == "" {
break
}
if key := unlockR5(candidate, user, userE, nil, rev); key != nil {
return key, false, nil
}
if len(owner) >= 48 {
if key := unlockR5(candidate, owner, ownerE, user[:48], rev); key != nil {
return key, true, nil
}
}
}
return nil, false, ErrWrongPassword
}
// unlockR5 checks one password against a 48-byte /U or /O entry and, when it
// matches, unwraps the file key from the matching /UE or /OE.
func unlockR5(password, entry, wrapped, udata []byte, rev int) []byte {
valSalt, keySalt := entry[32:40], entry[40:48]
if !bytes.Equal(hash2B(password, valSalt, udata, rev), entry[:32]) {
return nil
}
if len(wrapped) < 32 {
return nil
}
inter := hash2B(password, keySalt, udata, rev)
// inter is always the 32 bytes hash2B returns, so the cipher cannot refuse it.
block, _ := aes.NewCipher(inter)
out := make([]byte, 32)
cipher.NewCBCDecrypter(block, make([]byte, aes.BlockSize)).CryptBlocks(out, wrapped[:32])
return out
}
// hash2B is algorithm 2.B: SHA-256 for revision 5, and for revision 6 the
// hardening loop that repeatedly encrypts and re-hashes.
func hash2B(password, salt, udata []byte, rev int) []byte {
h := sha256.New()
h.Write(password)
h.Write(salt)
h.Write(udata)
k := h.Sum(nil)
if rev < 6 {
return k
}
for round := 0; ; round++ {
var k1 []byte
one := append(append(append([]byte{}, password...), k...), udata...)
for i := 0; i < 64; i++ {
k1 = append(k1, one...)
}
// k is always 32 bytes, so a 16-byte key is always well formed.
block, _ := aes.NewCipher(k[:16])
e := make([]byte, len(k1)-len(k1)%aes.BlockSize)
cipher.NewCBCEncrypter(block, k[16:32]).CryptBlocks(e, k1[:len(e)])
sum := 0
for _, b := range e[:16] {
sum += int(b)
}
var next hash.Hash
switch sum % 3 {
case 0:
next = sha256.New()
case 1:
next = sha512.New384()
default:
next = sha512.New()
}
next.Write(e)
k = next.Sum(nil)
if round >= 63 && int(e[len(e)-1]) <= round-31 {
break
}
}
return k[:32]
}
// objectKey is the per-object key the pre-2.0 methods use; AESV3 uses the file
// key unchanged.
func (dec *decryptor) objectKey(num, gen int, method cryptMethod) []byte {
if method == cryptAESV3 {
return dec.key
}
h := md5.New()
h.Write(dec.key)
h.Write([]byte{byte(num), byte(num >> 8), byte(num >> 16), byte(gen), byte(gen >> 8)})
if method == cryptAESV2 {
h.Write([]byte{0x73, 0x41, 0x6C, 0x54}) // "sAlT"
}
k := h.Sum(nil)
if n := len(dec.key) + 5; n < 16 {
return k[:n]
}
return k
}
// decryptBytes undoes one method over one object's data.
func (dec *decryptor) decryptBytes(num, gen int, method cryptMethod, data []byte) []byte {
switch method {
case cryptNone:
return data
case cryptRC4:
return rc4Bytes(dec.objectKey(num, gen, method), data)
default:
return aesDecrypt(dec.objectKey(num, gen, method), data)
}
}
// rc4Bytes applies RC4, which is its own inverse. A key length the cipher
// refuses yields nil, which every caller reads as a password that does not
// fit — the derivations here never produce one.
func rc4Bytes(key, data []byte) []byte {
c, err := rc4.NewCipher(key)
if err != nil {
return nil
}
out := make([]byte, len(data))
c.XORKeyStream(out, data)
return out
}
// aesDecrypt undoes AES in CBC mode, the initialisation vector being the first
// block of the data and the padding the one CBC calls for.
func aesDecrypt(key, data []byte) []byte {
if len(data) < 2*aes.BlockSize || len(data)%aes.BlockSize != 0 {
return nil
}
block, err := aes.NewCipher(key)
if err != nil {
return nil
}
out := make([]byte, len(data)-aes.BlockSize)
cipher.NewCBCDecrypter(block, data[:aes.BlockSize]).CryptBlocks(out, data[aes.BlockSize:])
n := int(out[len(out)-1])
if n < 1 || n > aes.BlockSize || n > len(out) {
return out
}
return out[:len(out)-n]
}
// decryptObject walks an object, decrypting every string in it and, for a
// stream, its data. The /Encrypt dictionary itself, and any cross-reference
// stream, are left alone: neither is encrypted.
func (dec *decryptor) decryptObject(num, gen int, o Object) Object {
if dec.skipKnown && num == dec.skipObj {
return o
}
return dec.walk(num, gen, o)
}
// walk is decryptObject's recursion.
func (dec *decryptor) walk(num, gen int, o Object) Object {
switch v := o.(type) {
case String:
return String(dec.decryptBytes(num, gen, dec.strings, v))
case Array:
for i, e := range v {
v[i] = dec.walk(num, gen, e)
}
return v
case Dict:
for k, e := range v {
v[k] = dec.walk(num, gen, e)
}
return v
case *Stream:
if t, ok := ToName(v.Dict.Get("Type")); ok && t == "XRef" {
return v
}
v.Dict, _ = ToDict(dec.walk(num, gen, v.Dict))
if streamIsPlain(v.Dict) {
return v
}
v.Raw = dec.decryptBytes(num, gen, dec.streams, v.Raw)
return v
}
return o
}
// streamIsPlain reports whether a stream's own filter chain says its bytes were
// left unencrypted: a leading /Crypt filter naming /Identity, which is what
// /Crypt with no /Name means too. Producers use it for the metadata stream of a
// file whose /EncryptMetadata is false, and decrypting such a stream turns
// readable XML into noise.
func streamIsPlain(d Dict) bool {
if first, ok := firstFilter(d); !ok || first != "Crypt" {
return false
}
name, ok := ToName(firstDecodeParms(d).Get("Name"))
return !ok || name == "Identity"
}
// resolved is a helper for reading an /Encrypt entry that may be indirect.
func resolved(d Dict, key Name, r Resolver) Object {
o, err := Resolve(d.Get(key), r)
if err != nil {
return Null{}
}
return o
}
// intOf reads an integer with a default.
func intOf(o Object, def int64) int64 {
if n, ok := ToInt(o); ok {
return n
}
return def
}
vendor/github.com/go-pdfkit/reader/document.go
Generated Vendored
+431
View File
@@ -0,0 +1,431 @@
package reader
import (
"bytes"
"fmt"
)
// maxPageTreeNodes bounds a page tree walk. /Count is a claim, not a fact, and
// /Kids can be made to point back at an ancestor.
const maxPageTreeNodes = 1 << 20
// inheritable lists the page attributes a node may take from an ancestor.
var inheritable = []Name{"Resources", "MediaBox", "CropBox", "Rotate"}
// A Document is a parsed PDF file: its cross-reference information, its
// trailer, and lazy access to every object in it.
type Document struct {
buf []byte
xref map[int]xrefEntry
trailer Dict
cache map[int]Object
loading map[int]bool
objStms map[int]map[int]Object
pages []Ref
password string
decrypt *decryptor
encryptNum int
encryptKnown bool
repaired bool
}
// Open parses the cross-reference information of a PDF file held in memory,
// using the empty password. A file whose tables are damaged is rebuilt by
// scanning it, so Open fails only when there is no usable document structure
// at all, or when the file is encrypted and the empty password does not open
// it — [OpenWithPassword] then does.
func Open(b []byte) (*Document, error) { return OpenWithPassword(b, "") }
// OpenWithPassword is Open with a password to try. The password is tried both
// as the user password and as the owner password, the empty one as well.
func OpenWithPassword(b []byte, password string) (*Document, error) {
d := &Document{
buf: b,
xref: map[int]xrefEntry{},
cache: map[int]Object{},
loading: map[int]bool{},
objStms: map[int]map[int]Object{},
password: password,
}
err := d.loadXref()
if err == nil {
if derr := d.setUpDecryption(password); derr != nil {
return nil, derr
}
d.cache = map[int]Object{}
cerr := error(nil)
if _, cerr = d.Catalog(); cerr == nil {
return d, nil
}
// Tables that parse but lead nowhere are worse than none: rebuild.
// What the catalogue lookup actually said is kept: "no page tree" and
// "/Root is a null" send a reader to different places.
err = fmt.Errorf("reader: the cross-reference tables do not lead to a catalogue: %w", cerr)
}
if rerr := d.repair(); rerr != nil {
// Both diagnoses matter, and the rebuild's is the one about the file
// as it really is: it has read every object header in it, where the
// tables only failed to be read. Returning the tables' error alone
// reported "no startxref" for a file that turned out to hold no
// objects at all, or an encrypted one whose catalogue could not be
// reached — a different problem with a different answer.
return nil, fmt.Errorf("%w (the cross-reference information was unusable too: %v)", rerr, err)
}
// Setting the key up again here used to be necessary, because the rebuild
// ran before anything knew the file was encrypted. repair() establishes it
// itself now, so by this point it is either known or not needed.
return d, nil
}
// Trailer returns the file's trailer dictionary, the newest value of every key
// across the chain of cross-reference sections.
func (d *Document) Trailer() Dict { return d.trailer }
// Repaired reports whether the document's structure was rebuilt by scanning
// the file rather than read from its cross-reference tables.
func (d *Document) Repaired() bool { return d.repaired }
// Resolver returns the function that turns this document's indirect references
// into objects, for [Resolve] and [Decode].
func (d *Document) Resolver() Resolver { return d.Get }
// Resolve follows indirect references until it reaches a direct object.
func (d *Document) Resolve(o Object) (Object, error) { return Resolve(o, d.Get) }
// GetDict resolves an entry of a dictionary to a dictionary.
func (d *Document) GetDict(from Dict, key Name) (Dict, bool) {
o, err := d.Resolve(from.Get(key))
if err != nil {
return nil, false
}
return ToDict(o)
}
// Get returns the object an indirect reference names, or [Null] when the file
// does not define it.
func (d *Document) Get(ref Ref) (Object, error) {
if o, ok := d.cache[ref.Num]; ok {
return o, nil
}
e, ok := d.xref[ref.Num]
if !ok || e.kind == 'f' {
return Null{}, nil
}
if d.loading[ref.Num] {
// An object stream whose own /Length or /Filter points back at an
// object inside itself would otherwise recurse forever.
return Null{}, nil
}
d.loading[ref.Num] = true
defer delete(d.loading, ref.Num)
var obj Object
var err error
if e.kind == 'o' {
obj, err = d.getFromObjectStream(ref.Num, e)
} else {
obj, err = d.getAtOffset(ref.Num, e)
}
if err != nil {
return nil, err
}
d.cache[ref.Num] = obj
return obj, nil
}
// getAtOffset reads an object written directly in the file. An offset that
// does not hold the expected object number means the tables are wrong, which
// is common enough that it triggers a rebuild rather than an error.
func (d *Document) getAtOffset(num int, e xrefEntry) (Object, error) {
if e.offset < 0 || e.offset >= int64(len(d.buf)) {
return d.retryAfterRepair(num, fmt.Errorf("reader: object %d is at offset %d, outside the file", num, e.offset))
}
got, obj, _, err := ParseIndirectObject(d.buf[e.offset:], d.Get)
if err != nil {
return d.retryAfterRepair(num, err)
}
if got.Num != num {
return d.retryAfterRepair(num, fmt.Errorf("reader: offset %d holds object %d, not %d", e.offset, got.Num, num))
}
if d.decrypt != nil {
obj = d.decrypt.decryptObject(num, got.Gen, obj)
}
return obj, nil
}
// retryAfterRepair rebuilds the tables once and looks the object up again.
func (d *Document) retryAfterRepair(num int, cause error) (Object, error) {
if d.repaired {
return Null{}, nil
}
if err := d.repair(); err != nil {
return nil, cause
}
e, ok := d.xref[num]
if !ok || e.kind != 'n' || e.offset >= int64(len(d.buf)) {
return Null{}, nil
}
got, obj, _, err := ParseIndirectObject(d.buf[e.offset:], d.Get)
if err != nil || got.Num != num {
return Null{}, nil
}
if d.decrypt != nil {
obj = d.decrypt.decryptObject(num, got.Gen, obj)
}
return obj, nil
}
// getFromObjectStream reads an object held inside a /Type /ObjStm stream.
func (d *Document) getFromObjectStream(num int, e xrefEntry) (Object, error) {
objs, err := d.objectStream(e.strmNum)
if err != nil {
return nil, err
}
if o, ok := objs[num]; ok {
return o, nil
}
// The index is a hint; a stream that does not hold the object simply
// does not define it.
return Null{}, nil
}
// objectStream parses an object stream once and keeps its contents.
func (d *Document) objectStream(num int) (map[int]Object, error) {
if objs, ok := d.objStms[num]; ok {
return objs, nil
}
objs := map[int]Object{}
d.objStms[num] = objs
o, err := d.Get(Ref{Num: num})
if err != nil {
return objs, err
}
s, ok := ToStream(o)
if !ok {
return objs, nil
}
// An object stream that stops in the middle still holds whole objects in
// the part that did decode, and losing them loses pages.
dec := d.DecodeStreamRecovering(s)
if dec.Image != "" {
return objs, nil
}
data := dec.Data
n := int(intOr(s.Dict.Get("N"), 0))
first := int(intOr(s.Dict.Get("First"), 0))
if n <= 0 || first <= 0 || first > len(data) {
return objs, nil
}
l := &lexer{buf: data[:first]}
type pair struct{ num, off int }
pairs := make([]pair, 0, n)
for i := 0; i < n; i++ {
t1, err1 := l.next()
t2, err2 := l.next()
if err1 != nil || err2 != nil || t1.kind != tokInteger || t2.kind != tokInteger {
break
}
pairs = append(pairs, pair{int(t1.i), first + int(t2.i)})
}
for _, p := range pairs {
if p.off < 0 || p.off > len(data) {
continue
}
obj, _, err := ParseObject(data[p.off:])
if err != nil {
continue
}
objs[p.num] = obj
}
return objs, nil
}
// intOr resolves nothing; it reads a direct integer or returns a default.
func intOr(o Object, def int64) int64 {
if n, ok := ToInt(o); ok {
return n
}
return def
}
// DecodeStream applies a stream's filter chain, resolving any indirect decode
// parameters against this document. A chain that cannot be run to the end is
// an error; [Document.DecodeStreamRecovering] salvages instead.
func (d *Document) DecodeStream(s *Stream) ([]byte, Name, error) {
return Decode(s.Dict, s.Raw, d.Get)
}
// DecodeStreamRecovering applies a stream's filter chain, salvaging what it can
// from a chain that cannot be run to the end and saying that it did so.
func (d *Document) DecodeStreamRecovering(s *Stream) Decoded {
return DecodeRecovering(s.Dict, s.Raw, d.Get)
}
// Catalog returns the document catalogue the trailer's /Root names.
func (d *Document) Catalog() (Dict, error) {
if d.trailer == nil {
return nil, fmt.Errorf("reader: the file has no trailer")
}
o, err := d.Resolve(d.trailer.Get("Root"))
if err != nil {
return nil, err
}
cat, ok := ToDict(o)
if !ok {
return nil, fmt.Errorf("reader: /Root is a %s, not a dictionary", o.Kind())
}
if _, ok := d.GetDict(cat, "Pages"); !ok {
return nil, fmt.Errorf("reader: the catalogue has no page tree")
}
return cat, nil
}
// PageCount reports how many pages the document has.
func (d *Document) PageCount() int { return len(d.pageRefs()) }
// PageRef returns the reference of the i'th page, counting from one.
func (d *Document) PageRef(i int) (Ref, bool) {
refs := d.pageRefs()
if i < 1 || i > len(refs) {
return Ref{}, false
}
return refs[i-1], true
}
// Page returns the i'th page's dictionary, counting from one, with the
// attributes it inherits from its ancestors filled in.
func (d *Document) Page(i int) (Dict, error) {
ref, ok := d.PageRef(i)
if !ok {
return nil, fmt.Errorf("reader: page %d is out of range (the document has %d)", i, d.PageCount())
}
o, err := d.Get(ref)
if err != nil {
return nil, err
}
page, ok := ToDict(o)
if !ok {
return nil, fmt.Errorf("reader: page %d is a %s, not a dictionary", i, o.Kind())
}
return d.withInherited(page), nil
}
// withInherited copies the page and fills in what its ancestors provide.
func (d *Document) withInherited(page Dict) Dict {
out := Dict{}
for k, v := range page {
out[k] = v
}
node := page
for depth := 0; depth < maxRefDepth; depth++ {
missing := false
for _, k := range inheritable {
if _, ok := out[k]; !ok {
missing = true
}
}
if !missing {
break
}
parent, ok := d.GetDict(node, "Parent")
if !ok {
break
}
for _, k := range inheritable {
if _, ok := out[k]; ok {
continue
}
if v, ok := parent[k]; ok {
out[k] = v
}
}
node = parent
}
return out
}
// pageRefs walks the page tree once, in document order.
func (d *Document) pageRefs() []Ref {
if d.pages != nil {
return d.pages
}
d.pages = []Ref{}
cat, err := d.Catalog()
if err != nil {
return d.pages
}
root := cat["Pages"]
seen := map[Ref]bool{}
d.walkPages(root, seen, 0)
if len(d.pages) == 0 {
// A tree that yields nothing but a catalogue that names one page node
// is still a one-page document.
d.pages = d.scanForPages()
}
return d.pages
}
// walkPages appends the leaves of a page tree node in order.
func (d *Document) walkPages(node Object, seen map[Ref]bool, depth int) {
if depth > 64 || len(d.pages) >= maxPageTreeNodes {
return
}
ref, isRef := node.(Ref)
if isRef {
if seen[ref] {
return
}
seen[ref] = true
}
o, err := d.Resolve(node)
if err != nil {
return
}
dict, ok := ToDict(o)
if !ok {
return
}
kids, hasKids := d.kidsOf(dict)
if !hasKids {
if isRef {
d.pages = append(d.pages, ref)
}
return
}
for _, kid := range kids {
d.walkPages(kid, seen, depth+1)
}
}
// kidsOf reports a node's /Kids when it is an interior node of the page tree.
func (d *Document) kidsOf(node Dict) (Array, bool) {
if t, ok := ToName(node.Get("Type")); ok && t == "Page" {
return nil, false
}
o, err := d.Resolve(node.Get("Kids"))
if err != nil {
return nil, false
}
kids, ok := ToArray(o)
return kids, ok
}
// Version reports the PDF version the header declares, "1.7" and the like.
func (d *Document) Version() string {
i := bytes.Index(d.buf, []byte("%PDF-"))
if i < 0 || i+8 > len(d.buf) {
return ""
}
return string(bytes.TrimRight(d.buf[i+5:i+8], "\r\n \t"))
}
// scanForPages is the last resort for a page tree that resolves to nothing:
// every object in the file that calls itself a page, in object-number order.
func (d *Document) scanForPages() []Ref {
var out []Ref
for _, num := range d.objectsOfType("Page") {
out = append(out, Ref{Num: num})
}
return out
}
vendor/github.com/go-pdfkit/reader/encrypt.go
Generated Vendored
+365
View File
@@ -0,0 +1,365 @@
package reader
import (
"bytes"
"crypto/aes"
"crypto/cipher"
"crypto/md5"
"crypto/rand"
"fmt"
"strings"
)
// Permissions say what a reader may do with a file it can open with the user
// password. They are advisory — nothing enforces them but the reader's own
// good manners — and a file opened with the owner password ignores them
// entirely.
type Permissions uint32
// The permissions a standard security handler can express. Anything not
// granted is refused; [AllPermissions] grants everything.
const (
PermPrint Permissions = 1 << 2
PermModify Permissions = 1 << 3
PermCopy Permissions = 1 << 4
PermAnnotate Permissions = 1 << 5
PermFillForms Permissions = 1 << 8
PermExtract Permissions = 1 << 9 // for accessibility
PermAssemble Permissions = 1 << 10
PermPrintFaithful Permissions = 1 << 11 // at full resolution
// AllPermissions grants every one of them.
AllPermissions = PermPrint | PermModify | PermCopy | PermAnnotate |
PermFillForms | PermExtract | PermAssemble | PermPrintFaithful
)
// permissionNames pairs each permission with what it lets a reader do, in the
// order a person would want to read them.
var permissionNames = []struct {
bit Permissions
name string
}{
{PermPrint, "print"},
{PermPrintFaithful, "print at full resolution"},
{PermModify, "modify"},
{PermAssemble, "assemble"},
{PermCopy, "copy"},
{PermExtract, "extract for accessibility"},
{PermAnnotate, "annotate"},
{PermFillForms, "fill in forms"},
}
// Allows reports whether every one of the given permissions is granted.
func (p Permissions) Allows(want Permissions) bool { return p&want == want }
// String lists what is granted, in words.
func (p Permissions) String() string {
var out []string
for _, e := range permissionNames {
if p.Allows(e.bit) {
out = append(out, e.name)
}
}
if len(out) == 0 {
return "nothing"
}
return strings.Join(out, ", ")
}
// permissionBase is the pattern of reserved bits the specification requires
// around the permissions themselves.
const permissionBase uint32 = 0xFFFFF0C0
// An Encryption says how a file is to be protected.
//
// Two people can open it: whoever knows the user password, subject to the
// permissions, and whoever knows the owner password, subject to nothing. An
// empty user password means the file opens without one and the permissions are
// all it says.
type Encryption struct {
UserPassword string
OwnerPassword string
Permissions Permissions
// AES128 asks for the older method, which readers before 2008 understand.
// The default is AES-256, which is what a file should use today.
AES128 bool
}
// Encrypt protects everything written from here on. It must be called before
// anything is written, and the file it produces is not byte-for-byte
// reproducible: encryption needs randomness, by design.
func (w *Writer) Encrypt(e Encryption) {
if len(w.offsets) > 0 || len(w.pending) > 0 {
w.note(fmt.Errorf("reader: a file cannot be encrypted after it has been written to"))
return
}
enc, err := newEncrypter(e)
if err != nil {
w.note(err)
return
}
w.encrypt = enc
}
// An encrypter holds the file key and the dictionary that describes it.
type encrypter struct {
key []byte
method cryptMethod
dict Dict
id []byte
err error
// number is the object the /Encrypt dictionary is written under, which is
// the one thing in the file that is never encrypted.
number int
}
// newEncrypter derives a file key and builds the dictionary that lets a reader
// derive it again from a password.
func newEncrypter(e Encryption) (*encrypter, error) {
perm := permissionBase | uint32(e.Permissions)
id, err := randomBytes(16)
if err != nil {
return nil, err
}
if e.AES128 {
return newLegacyEncrypter(e, perm, id)
}
return newAES256Encrypter(e, perm, id)
}
// newAES256Encrypter builds the revision 6 form: a random file key, wrapped
// twice, once for each password.
func newAES256Encrypter(e Encryption, perm uint32, id []byte) (*encrypter, error) {
key, err := randomBytes(32)
if err != nil {
return nil, err
}
// An owner password nobody set is the user's: leaving it empty would
// mean anything at all opens the file with every permission, which is
// the opposite of what asking for encryption means.
ownerPassword := e.OwnerPassword
if ownerPassword == "" {
ownerPassword = e.UserPassword
}
user, userE, err := wrapKey([]byte(e.UserPassword), key, nil)
if err != nil {
return nil, err
}
owner, ownerE, err := wrapKey([]byte(ownerPassword), key, user)
if err != nil {
return nil, err
}
perms, err := permsEntry(key, perm)
if err != nil {
return nil, err
}
return &encrypter{
key: key,
method: cryptAESV3,
id: id,
dict: Dict{
"Filter": Name("Standard"),
"V": Integer(5),
"R": Integer(6),
"Length": Integer(256),
"P": Integer(int32(perm)),
"U": String(user),
"UE": String(userE),
"O": String(owner),
"OE": String(ownerE),
"Perms": String(perms),
"CF": Dict{"StdCF": Dict{"CFM": Name("AESV3"), "Length": Integer(32)}},
"StmF": Name("StdCF"),
"StrF": Name("StdCF"),
"EncryptMetadata": Bool(true),
},
}, nil
}
// wrapKey builds one 48-byte password entry and the 32 bytes that hold the
// file key wrapped for it.
func wrapKey(password, key, udata []byte) (entry, wrapped []byte, err error) {
salts, err := randomBytes(16)
if err != nil {
return nil, nil, err
}
validation, keySalt := salts[:8], salts[8:]
entry = append(append(hash2B(password, validation, udata, 6), validation...), keySalt...)
// hash2B always returns 32 bytes, so the cipher cannot refuse the key.
block, _ := aes.NewCipher(hash2B(password, keySalt, udata, 6))
wrapped = make([]byte, 32)
cipher.NewCBCEncrypter(block, make([]byte, aes.BlockSize)).CryptBlocks(wrapped, key)
return entry, wrapped, nil
}
// permsEntry is the block that lets a reader check the permissions have not
// been tampered with: they are encrypted with the file key itself.
func permsEntry(key []byte, perm uint32) ([]byte, error) {
tail, err := randomBytes(4)
if err != nil {
return nil, err
}
plain := []byte{
byte(perm), byte(perm >> 8), byte(perm >> 16), byte(perm >> 24),
0xFF, 0xFF, 0xFF, 0xFF,
'T', // metadata is encrypted too
'a', 'd', 'b', // the specification's own marker
tail[0], tail[1], tail[2], tail[3],
}
// The file key is 32 bytes by construction.
block, _ := aes.NewCipher(key)
out := make([]byte, 16)
// One block, with no chaining: this is the one place the format uses it.
block.Encrypt(out, plain)
return out, nil
}
// newLegacyEncrypter builds the revision 4 form, whose key comes from the
// passwords rather than the other way round.
func newLegacyEncrypter(e Encryption, perm uint32, id []byte) (*encrypter, error) {
const n = 16 // 128 bits
owner := legacyOwnerValue([]byte(e.OwnerPassword), []byte(e.UserPassword), n)
key := legacyFileKey(padPassword([]byte(e.UserPassword)), owner, id, int32(perm), n, 4, true)
user := legacyUserValue(key, id)
return &encrypter{
key: key,
method: cryptAESV2,
id: id,
dict: Dict{
"Filter": Name("Standard"),
"V": Integer(4),
"R": Integer(4),
"Length": Integer(128),
"P": Integer(int32(perm)),
"O": String(owner),
"U": String(user),
"CF": Dict{"StdCF": Dict{"CFM": Name("AESV2"), "Length": Integer(16)}},
"StmF": Name("StdCF"),
"StrF": Name("StdCF"),
"EncryptMetadata": Bool(true),
},
}, nil
}
// legacyOwnerValue is algorithm 3: the /O entry, which holds the user password
// encrypted under the owner's.
func legacyOwnerValue(ownerPassword, userPassword []byte, n int) []byte {
if len(ownerPassword) == 0 {
ownerPassword = userPassword
}
sum := md5Sum(padPassword(ownerPassword))
key := sum
for i := 0; i < 50; i++ {
key = md5Sum(key)
}
key = key[:n]
x := padPassword(userPassword)
for i := 0; i <= 19; i++ {
x = rc4Bytes(xorKey(key, i), x)
}
return x
}
// legacyUserValue is algorithm 5: the /U entry.
func legacyUserValue(key, id []byte) []byte {
x := rc4Bytes(key, md5Sum(append(append([]byte{}, pad...), id...)))
for i := 1; i <= 19; i++ {
x = rc4Bytes(xorKey(key, i), x)
}
return append(x, make([]byte, 16)...)
}
// randomSource is where randomness comes from. It is a variable so a test
// can take it away and see that a file is refused rather than written
// without the protection it was asked for.
var randomSource = rand.Read
// randomBytes is the only source of randomness here, and the reason an
// encrypted file is not reproducible.
func randomBytes(n int) ([]byte, error) {
out := make([]byte, n)
if _, err := randomSource(out); err != nil {
return nil, fmt.Errorf("reader: no randomness available: %w", err)
}
return out, nil
}
// apply encrypts one object's strings and, for a stream, its data.
func (e *encrypter) apply(num, gen int, o Object) Object {
switch v := o.(type) {
case String:
return String(e.encryptBytes(num, gen, v))
case Array:
out := make(Array, len(v))
for i, x := range v {
out[i] = e.apply(num, gen, x)
}
return out
case Dict:
out := Dict{}
for k, x := range v {
out[k] = e.apply(num, gen, x)
}
return out
case *Stream:
out := &Stream{Dict: Dict{}, Raw: e.encryptBytes(num, gen, v.Raw)}
for k, x := range v.Dict {
out.Dict[k] = e.apply(num, gen, x)
}
return out
}
return o
}
// encryptBytes protects one object's bytes.
func (e *encrypter) encryptBytes(num, gen int, data []byte) []byte {
dec := &decryptor{key: e.key}
key := dec.objectKey(num, gen, e.method)
iv, err := randomBytes(aes.BlockSize)
if err != nil {
// Writing the bytes as they are would hand out in the clear what was
// asked to be protected, so the failure is remembered and the whole
// file is refused.
e.note(err)
return data
}
// The file key is 32 bytes by construction.
block, _ := aes.NewCipher(key)
// The padding CBC calls for, always at least one whole block of it.
n := aes.BlockSize - len(data)%aes.BlockSize
padded := append(append([]byte{}, data...), bytes.Repeat([]byte{byte(n)}, n)...)
out := make([]byte, len(padded))
cipher.NewCBCEncrypter(block, iv).CryptBlocks(out, padded)
return append(iv, out...)
}
// md5Sum is the hash the pre-2.0 handlers are built on.
func md5Sum(b []byte) []byte {
sum := md5.Sum(b)
return sum[:]
}
// writeEncryptDict writes the /Encrypt dictionary and returns the trailer
// entries a reader needs to find and use it. The dictionary is the one object
// in the file that is never encrypted, since it says how to decrypt the rest.
func (w *Writer) writeEncryptDict() Dict {
if w.encrypt == nil {
return nil
}
ref := w.Reserve()
w.encrypt.number = ref.Num
w.writeInlineRaw(ref, w.encrypt.dict)
// The identifier takes part in the older key derivation and has to be in
// the trailer for a reader to reach it.
id := String(w.encrypt.id)
return Dict{"Encrypt": ref, "ID": Array{id, id}}
}
// note keeps the first thing that went wrong while encrypting, so a file that
// could not be protected is refused rather than written in the clear.
func (e *encrypter) note(err error) {
if e.err == nil {
e.err = err
}
}
vendor/github.com/go-pdfkit/reader/filter.go
Generated Vendored
+475
View File
@@ -0,0 +1,475 @@
package reader
import (
"bytes"
"compress/flate"
"compress/zlib"
"fmt"
"io"
)
// maxDecodedSize caps what one stream may expand to. A few hundred bytes of
// Flate can name gigabytes, and a reader that runs in a browser tab must
// refuse rather than fill the heap. Tests lower it.
var maxDecodedSize int64 = 1 << 30
// ImageFilter reports whether a filter yields an encoded image rather than a
// byte stream. [Decode] stops at one of these and hands the caller the still
// encoded bytes, because decoding them is an image decoder's job.
//
// /CCITTFaxDecode was on this list and is not any more. It does not carry an
// image with its own idea of how many components it has and how deep they are,
// the way DCT and JPX do: it produces bilevel samples, one bit a pixel, and the
// stream dictionary says what they mean. That is a byte stream, so it is a
// filter — and decoding it here means every caller gets it rather than each
// writing its own. See ccitt.go.
func ImageFilter(n Name) bool {
switch n {
case "DCTDecode", "DCT", "JPXDecode", "JBIG2Decode":
return true
}
return false
}
// A Decoded is the outcome of applying a stream's filter chain, including the
// outcome of a chain that could not be finished.
//
// The point of the type is that a caller can tell the three outcomes apart
// without having to be careful. Bytes a filter decoded and bytes no filter
// decoded arrive in different fields, so the second kind cannot be painted as
// samples or tokenised as content by a caller that forgot to check a flag.
type Decoded struct {
// Data is what the chain decoded. A recovered decode leaves fewer bytes
// here than the stream meant to carry, but they are bytes of the kind it
// meant to carry: Data never holds bytes that no filter has decoded.
//
// The one case where Data is still encoded is the deliberate one: a chain
// that stopped at an image filter, which Image names and Recovered does
// not flag, because stopping there is the contract.
Data []byte
// Undecoded holds the bytes the chain could not get past, still in the
// encoding Filter names. It is set instead of Data — never beside it —
// when the filter that failed produced nothing at all.
//
// This is where a compressed content stream, a truncated fax, or a filter
// nobody implements ends up. A caller that wants the stream as it lies has
// to ask for it by a name that says what it is; a caller that reads Data
// cannot be handed it by accident. Painting undecoded bytes as a one-bit
// image looks like a page with something on it, which is why nothing about
// this is left to a flag.
Undecoded []byte
// Image names the image filter the chain stopped at, when it stopped at
// one; Data is then the bytes still encoded in it.
Image Name
// Recovered says the chain could not be run to the end. A caller that must
// not act on damaged data stops here — or calls [Decode], which refuses
// outright.
Recovered bool
// Cause says why the chain stopped, and is set exactly when Recovered is.
Cause error
// Filter names the filter that could not be applied, when one is to blame:
// a chain whose /Filter entry itself is unreadable blames nothing.
Filter Name
}
// DecodeRecovering applies a stream dictionary's filter chain to raw and never
// fails. A filter that cannot be applied — corrupt Flate data, a stream that
// stops in the middle, a filter name nobody implements — ends the chain, and
// what the filters before it produced is returned with [Decoded.Recovered]
// set. That is deliberately what every other reader does: refusing the stream
// loses a page the file can still show.
//
// The salvage is always as far down the chain as the filters got, never the
// bytes as they arrived: a damaged Flate stream yields the prefix it did
// inflate, in [Decoded.Data]. A filter that produced nothing at all leaves the
// bytes in [Decoded.Undecoded] instead, still in its encoding, because that is
// what they are.
func DecodeRecovering(d Dict, raw []byte, resolve Resolver) Decoded {
filters, parms, err := filterChain(d, resolve)
if err != nil {
return Decoded{Undecoded: raw, Recovered: true, Cause: err}
}
data := raw
for i, f := range filters {
if ImageFilter(f) {
return Decoded{Data: data, Image: f}
}
out, err := applyFilter(f, data, parms[i], resolve)
if err != nil {
if len(out) == 0 {
return Decoded{Undecoded: data, Recovered: true, Cause: err, Filter: f}
}
return Decoded{Data: out, Recovered: true, Cause: err, Filter: f}
}
data = out
}
return Decoded{Data: data}
}
// Decode applies a stream dictionary's filter chain to raw. It returns the
// decoded bytes and, when the chain ends in an image filter, that filter's
// name together with the bytes still encoded in it.
//
// Decode is the strict reading: a chain that cannot be run to the end is an
// error and yields no bytes. [DecodeRecovering] is the lenient one, and says
// which of the two it gave you.
func Decode(d Dict, raw []byte, resolve Resolver) ([]byte, Name, error) {
r := DecodeRecovering(d, raw, resolve)
if r.Recovered {
return nil, "", r.Cause
}
return r.Data, r.Image, nil
}
// DecodeStream is Decode for a parsed stream.
func DecodeStream(s *Stream, resolve Resolver) ([]byte, Name, error) {
return Decode(s.Dict, s.Raw, resolve)
}
// DecodeStreamRecovering is DecodeRecovering for a parsed stream.
func DecodeStreamRecovering(s *Stream, resolve Resolver) Decoded {
return DecodeRecovering(s.Dict, s.Raw, resolve)
}
// filterChain reads /Filter and /DecodeParms, each of which may be a single
// value or an array, and returns them aligned.
func filterChain(d Dict, r Resolver) ([]Name, []Dict, error) {
fo, err := Resolve(d.Get("Filter"), r)
if err != nil {
return nil, nil, err
}
var names []Name
switch v := fo.(type) {
case Null:
case Name:
names = []Name{v}
case Array:
for _, e := range v {
eo, err := Resolve(e, r)
if err != nil {
return nil, nil, err
}
n, ok := ToName(eo)
if !ok {
return nil, nil, fmt.Errorf("reader: /Filter array holds a %s, not a name", eo.Kind())
}
names = append(names, n)
}
default:
return nil, nil, fmt.Errorf("reader: /Filter is a %s, not a name or an array", fo.Kind())
}
parms := make([]Dict, len(names))
po, err := Resolve(d.Get("DecodeParms"), r)
if err != nil {
return nil, nil, err
}
switch v := po.(type) {
case Null:
case Dict:
if len(parms) > 0 {
parms[0] = v
}
case Array:
for i, e := range v {
if i >= len(parms) {
break
}
eo, err := Resolve(e, r)
if err != nil {
return nil, nil, err
}
if pd, ok := ToDict(eo); ok {
parms[i] = pd
}
}
default:
return nil, nil, fmt.Errorf("reader: /DecodeParms is a %s, not a dictionary or an array", po.Kind())
}
return names, parms, nil
}
// applyFilter runs one filter. The abbreviated names are the ones inline
// images use; regular streams may legally carry them too.
func applyFilter(f Name, data []byte, parm Dict, r Resolver) ([]byte, error) {
switch f {
case "FlateDecode", "Fl":
out, err := flateDecode(data)
if err != nil {
return salvage(out, err, parm, r)
}
return applyPredictor(out, parm, r)
case "LZWDecode", "LZW":
early := intParm(parm, "EarlyChange", 1, r)
out, err := lzwDecode(data, early != 0)
if err != nil {
return salvage(out, err, parm, r)
}
return applyPredictor(out, parm, r)
case "ASCIIHexDecode", "AHx":
return asciiHexDecode(data)
case "ASCII85Decode", "A85":
return ascii85Decode(data)
case "RunLengthDecode", "RL":
return runLengthDecode(data)
case "CCITTFaxDecode", "CCF":
return ccittDecode(data, ccittParamsOf(parm, r))
case "Crypt":
return cryptFilter(data, parm, r)
}
return nil, fmt.Errorf("reader: unsupported filter /%s", f)
}
// cryptFilter applies the /Crypt filter, which does not transform anything: it
// names the crypt filter a stream's bytes were encrypted with, and /Identity —
// which is also what /Crypt with no /Name means — says they were not encrypted
// at all. Either way the document that owns the stream has already dealt with
// its encryption by the time the filter chain runs, so there is nothing here
// left to undo.
//
// A /Name this reader cannot account for is reported rather than waved through,
// because the bytes would then be ciphertext that only looks like data.
func cryptFilter(data []byte, parm Dict, r Resolver) ([]byte, error) {
o, err := Resolve(parm.Get("Name"), r)
if err != nil {
return nil, fmt.Errorf("reader: /Crypt filter: %w", err)
}
if n, ok := ToName(o); ok && n != "Identity" {
// No bytes come back: they are ciphertext, and handing them over as
// though the filter had run is how ciphertext gets painted.
return nil, fmt.Errorf("reader: /Crypt filter names /%s, which this reader cannot apply", n)
}
return data, nil
}
// firstFilter names the first filter of a stream's chain. Only direct values
// are read: it is consulted while the file key is being established, before
// following an indirect reference is safe.
func firstFilter(d Dict) (Name, bool) {
switch v := d.Get("Filter").(type) {
case Name:
return v, true
case Array:
if len(v) > 0 {
n, ok := ToName(v[0])
return n, ok
}
}
return "", false
}
// firstDecodeParms is the parameter dictionary belonging to the first filter,
// read directly for the same reason.
func firstDecodeParms(d Dict) Dict {
switch v := d.Get("DecodeParms").(type) {
case Dict:
return v
case Array:
if len(v) > 0 {
if pd, ok := ToDict(v[0]); ok {
return pd
}
}
}
return nil
}
// salvage finishes a filter that stopped part-way. The prefix it did produce is
// still worth having, and the predictor still applies to it: the predictors a
// PDF may name all undo one row at a time, so a buffer that stops in the middle
// undoes up to where it stops.
func salvage(out []byte, err error, parm Dict, r Resolver) ([]byte, error) {
if len(out) == 0 {
return nil, err
}
if p, perr := applyPredictor(out, parm, r); perr == nil {
return p, err
}
return out, err
}
// intParm reads an integer decode parameter, falling back to its default.
func intParm(parm Dict, key Name, def int, r Resolver) int {
if parm == nil {
return def
}
o, err := Resolve(parm.Get(key), r)
if err != nil {
return def
}
n, ok := ToInt(o)
if !ok {
return def
}
return int(n)
}
// flateDecode inflates a stream. Producers emit both zlib-wrapped and bare
// deflate data, sometimes with leading white-space, and truncate the last
// stream in a damaged file. All four cases yield whatever bytes are there — but
// a prefix comes back with the error that ended it, never dressed up as a whole
// stream, so the caller can tell the difference.
func flateDecode(data []byte) ([]byte, error) {
i := 0
for i < len(data) && isSpace(data[i]) {
i++
}
data = data[i:]
if zr, err := zlib.NewReader(bytes.NewReader(data)); err == nil {
out, err := readAllCapped(zr)
if err == nil {
return out, nil
}
if len(out) > 0 {
return out, fmt.Errorf("reader: FlateDecode: %w", err)
}
}
out, err := readAllCapped(flate.NewReader(bytes.NewReader(data)))
if err != nil {
return out, fmt.Errorf("reader: FlateDecode: %w", err)
}
return out, nil
}
// readAllCapped reads r, refusing to grow past maxDecodedSize.
func readAllCapped(r io.Reader) ([]byte, error) {
var buf bytes.Buffer
n, err := io.Copy(&buf, io.LimitReader(r, maxDecodedSize+1))
if err != nil {
return buf.Bytes(), err
}
if n > maxDecodedSize {
return nil, fmt.Errorf("reader: decoded stream exceeds %d bytes", maxDecodedSize)
}
return buf.Bytes(), nil
}
// asciiHexDecode reads hexadecimal digits up to the '>' terminator, padding a
// final odd digit with zero.
func asciiHexDecode(data []byte) ([]byte, error) {
out := make([]byte, 0, len(data)/2)
hi := -1
for i := 0; i < len(data); i++ {
c := data[i]
if c == '>' {
break
}
if isSpace(c) {
continue
}
v := hexVal(c)
if v < 0 {
return out, fmt.Errorf("reader: ASCIIHexDecode: invalid digit %q", rune(c))
}
if hi < 0 {
hi = v
continue
}
out = append(out, byte(hi<<4|v))
hi = -1
}
if hi >= 0 {
out = append(out, byte(hi<<4))
}
return out, nil
}
// ascii85Decode reads base-85 groups up to the "~>" terminator.
func ascii85Decode(data []byte) ([]byte, error) {
if bytes.HasPrefix(data, []byte("<~")) {
data = data[2:]
}
out := make([]byte, 0, len(data)*4/5)
var group [5]byte
n := 0
flush := func(n int) {
for i := n; i < 5; i++ {
group[i] = 'u'
}
v := uint32(0)
for _, c := range group {
v = v*85 + uint32(c-'!')
}
var b [4]byte
b[0], b[1], b[2], b[3] = byte(v>>24), byte(v>>16), byte(v>>8), byte(v)
out = append(out, b[:n-1]...)
}
for i := 0; i < len(data); i++ {
c := data[i]
if isSpace(c) {
continue
}
if c == '~' {
break
}
if c == 'z' && n == 0 {
out = append(out, 0, 0, 0, 0)
continue
}
if c < '!' || c > 'u' {
return out, fmt.Errorf("reader: ASCII85Decode: invalid character %q", rune(c))
}
group[n] = c
n++
if n == 5 {
if err := checkA85(group); err != nil {
return out, err
}
flush(5)
n = 0
}
}
if n == 1 {
return out, fmt.Errorf("reader: ASCII85Decode: truncated final group")
}
if n > 1 {
flush(n)
}
return out, nil
}
// checkA85 rejects a full group that names more than 2^32-1.
func checkA85(g [5]byte) error {
v := uint64(0)
for _, c := range g {
v = v*85 + uint64(c-'!')
}
if v > 0xFFFFFFFF {
return fmt.Errorf("reader: ASCII85Decode: group overflows 32 bits")
}
return nil
}
// runLengthDecode expands the byte-oriented run-length encoding: a length byte
// below 128 introduces that many literal bytes plus one, above 128 repeats the
// next byte, and 128 ends the data.
func runLengthDecode(data []byte) ([]byte, error) {
out := []byte{}
for i := 0; i < len(data); {
n := int(data[i])
i++
switch {
case n == 128:
return out, nil
case n < 128:
end := i + n + 1
if end > len(data) {
return out, fmt.Errorf("reader: RunLengthDecode: truncated literal run")
}
out = append(out, data[i:end]...)
i = end
default:
if i >= len(data) {
return out, fmt.Errorf("reader: RunLengthDecode: truncated repeat run")
}
out = append(out, bytes.Repeat(data[i:i+1], 257-n)...)
i++
}
}
return out, nil
}
vendor/github.com/go-pdfkit/reader/inline.go
Generated Vendored
+155
View File
@@ -0,0 +1,155 @@
package reader
import "bytes"
// inlineImageEnd finds where an inline image's data stops — the one genuinely
// ambiguous thing in a content stream, since the data is raw bytes that may
// spell EI themselves.
//
// The answers are tried in order of how much they can be trusted: an image
// with no filter says exactly how long it is through its width, height, depth
// and colour space; a declared /L is believed when an EI really does follow
// it; and failing both, every EI in the data is tried in turn and the first
// one whose data actually decodes is taken. That last step is what a scan
// alone gets wrong, because compressed bytes do spell EI by accident.
func inlineImageEnd(b []byte, start int, d Dict) (int, error) {
if n, ok := inlineImageDataLength(d); ok && start+n <= len(b) && eiFollows(b, start+n) {
return start + n, nil
}
if n, ok := ToInt(d.Get("L")); ok && n >= 0 && start+int(n) <= len(b) && eiFollows(b, start+int(n)) {
return start + int(n), nil
}
expanded := (&InlineImage{Dict: d}).Expanded()
want, haveWant := inlineSampleBytes(d)
// White-space before EI is how the keyword is meant to be written; a
// second pass allows for producers that run the data straight into it.
for _, needSpace := range []bool{true, false} {
for from := start; ; {
end, resume := nextEI(b, start, from, needSpace)
if end < 0 {
break
}
if inlineDataDecodes(expanded, b[start:end], want, haveWant) {
return end, nil
}
from = resume
}
}
return 0, &SyntaxError{start, "an inline image has no EI"}
}
// nextEI returns where the image data would end for the next EI candidate at
// or after from, and where to resume searching. It reports -1 when there is no
// candidate left.
func nextEI(b []byte, start, from int, needSpace bool) (end, resume int) {
for i := from; i+1 < len(b); i++ {
if b[i] != 'E' || b[i+1] != 'I' {
continue
}
if i+2 < len(b) && isRegular(b[i+2]) {
continue
}
if i > start && isSpace(b[i-1]) {
return i - 1, i + 2
}
if !needSpace {
return i, i + 2
}
}
return -1, len(b)
}
// eiFollows reports whether an EI keyword stands at i, after white-space.
func eiFollows(b []byte, i int) bool {
for i < len(b) && isSpace(b[i]) {
i++
}
if !bytes.HasPrefix(b[i:], []byte("EI")) {
return false
}
return i+2 >= len(b) || !isRegular(b[i+2])
}
// inlineDataDecodes reports whether a candidate stretch of bytes is really the
// whole of the image: it has to run through the declared filters without
// complaint, produce the number of samples the image says it has, and — for a
// JPEG, which no filter here can check — begin and end where a JPEG does.
func inlineDataDecodes(expanded Dict, data []byte, want int, haveWant bool) bool {
if expanded.Get("Filter").Kind() == KindNull {
return true
}
out, img, err := Decode(expanded, data, nil)
if err != nil {
return false
}
if img == "DCTDecode" {
return len(out) > 4 && out[0] == 0xFF && out[1] == 0xD8 &&
out[len(out)-2] == 0xFF && out[len(out)-1] == 0xD9
}
if img != "" {
return true
}
return !haveWant || len(out) == want
}
// inlineComponents reports how many samples one pixel of an inline image has,
// for the colour spaces that can be named inline.
func inlineComponents(d Dict) (int, bool) {
if b, ok := ToBool(d.Get("IM")); ok && b {
return 1, true
}
cs, ok := ToName(d.Get("CS"))
if !ok {
if _, isArray := ToArray(d.Get("CS")); isArray {
// An inline [/Indexed …] space has one component per sample.
return 1, true
}
return 0, false
}
switch cs {
case "G", "DeviceGray", "CalGray", "I", "Indexed":
return 1, true
case "RGB", "DeviceRGB", "CalRGB":
return 3, true
case "CMYK", "DeviceCMYK":
return 4, true
}
// A name that refers to a colour space in the page's resources: not
// something the arithmetic can settle here.
return 0, false
}
// inlineSampleBytes computes how many bytes an inline image's samples occupy
// once decoded.
func inlineSampleBytes(d Dict) (int, bool) {
w, okW := ToInt(d.Get("W"))
h, okH := ToInt(d.Get("H"))
if !okW || !okH || w <= 0 || h <= 0 {
return 0, false
}
bpc := int64(8)
if b, ok := ToInt(d.Get("BPC")); ok {
bpc = b
}
if im, ok := ToBool(d.Get("IM")); ok && im {
bpc = 1
}
if bpc <= 0 || bpc > 16 {
return 0, false
}
comps, ok := inlineComponents(d)
if !ok {
return 0, false
}
row := (w*int64(comps)*bpc + 7) / 8
return int(row * h), true
}
// inlineImageDataLength is inlineSampleBytes for an image with no filter, for
// which the samples are the data.
func inlineImageDataLength(d Dict) (int, bool) {
if d.Get("F").Kind() != KindNull || d.Get("Filter").Kind() != KindNull {
return 0, false
}
return inlineSampleBytes(d)
}
vendor/github.com/go-pdfkit/reader/lex.go
Generated Vendored
+340
View File
@@ -0,0 +1,340 @@
package reader
import (
"fmt"
"strconv"
)
// A SyntaxError reports malformed PDF syntax and where it was found, so a
// repair pass can restart from a known offset instead of giving up.
type SyntaxError struct {
Offset int
Msg string
}
// Error implements the error interface.
func (e *SyntaxError) Error() string {
return fmt.Sprintf("reader: %s at offset %d", e.Msg, e.Offset)
}
// isSpace reports the six bytes the specification calls white-space,
// NUL included.
func isSpace(c byte) bool {
return c == 0 || c == 9 || c == 10 || c == 12 || c == 13 || c == 32
}
// isDelim reports the eight delimiter bytes.
func isDelim(c byte) bool {
switch c {
case '(', ')', '<', '>', '[', ']', '{', '}', '/', '%':
return true
}
return false
}
// isRegular reports a byte that may appear inside a name, a number or a
// keyword: anything that is neither white-space nor a delimiter.
func isRegular(c byte) bool { return !isSpace(c) && !isDelim(c) }
// hexVal decodes one hexadecimal digit, or reports -1.
func hexVal(c byte) int {
switch {
case c >= '0' && c <= '9':
return int(c - '0')
case c >= 'a' && c <= 'f':
return int(c-'a') + 10
case c >= 'A' && c <= 'F':
return int(c-'A') + 10
}
return -1
}
type tokKind uint8
const (
tokEOF tokKind = iota
tokInteger
tokReal
tokString
tokName
tokArrayOpen
tokArrayClose
tokDictOpen
tokDictClose
tokBraceOpen
tokBraceClose
tokKeyword
)
// A token is one lexical unit. text carries the bytes of a string, name or
// keyword; i and f carry the value of a number.
type token struct {
kind tokKind
text []byte
i int64
f float64
pos int
}
// A lexer walks the PDF object syntax over a byte slice. It never copies the
// input except where an escape sequence forces it.
type lexer struct {
buf []byte
pos int
}
// skipSpace consumes white-space and comments.
func (l *lexer) skipSpace() {
for l.pos < len(l.buf) {
c := l.buf[l.pos]
if isSpace(c) {
l.pos++
continue
}
if c == '%' {
for l.pos < len(l.buf) && l.buf[l.pos] != '\n' && l.buf[l.pos] != '\r' {
l.pos++
}
continue
}
return
}
}
// next returns the token at the current position and advances past it.
func (l *lexer) next() (token, error) {
l.skipSpace()
start := l.pos
if l.pos >= len(l.buf) {
return token{kind: tokEOF, pos: start}, nil
}
switch c := l.buf[l.pos]; {
case c == '[':
l.pos++
return token{kind: tokArrayOpen, pos: start}, nil
case c == ']':
l.pos++
return token{kind: tokArrayClose, pos: start}, nil
case c == '{':
l.pos++
return token{kind: tokBraceOpen, pos: start}, nil
case c == '}':
l.pos++
return token{kind: tokBraceClose, pos: start}, nil
case c == '<':
if l.pos+1 < len(l.buf) && l.buf[l.pos+1] == '<' {
l.pos += 2
return token{kind: tokDictOpen, pos: start}, nil
}
return l.hexString(start)
case c == '>':
if l.pos+1 < len(l.buf) && l.buf[l.pos+1] == '>' {
l.pos += 2
return token{kind: tokDictClose, pos: start}, nil
}
l.pos++
return token{}, &SyntaxError{start, "stray '>'"}
case c == '(':
return l.literalString(start)
case c == ')':
l.pos++
return token{}, &SyntaxError{start, "stray ')'"}
case c == '/':
return l.name(start)
case c == '+' || c == '-' || c == '.' || (c >= '0' && c <= '9'):
return l.number(start)
default:
return l.keyword(start), nil
}
}
// keyword reads a run of regular characters: an operator, obj, endobj, R,
// true, false, null.
func (l *lexer) keyword(start int) token {
p := l.pos
for p < len(l.buf) && isRegular(l.buf[p]) {
p++
}
l.pos = p
return token{kind: tokKeyword, text: l.buf[start:p], pos: start}
}
// number reads an integer or a real. It stops at the first byte that cannot
// continue the number rather than swallowing the whole run of regular
// characters: producers do write "3.4-5" for two numbers, and a reader that
// rejects that loses the rest of the content stream.
func (l *lexer) number(start int) (token, error) {
p := l.pos
for p < len(l.buf) && (l.buf[p] == '+' || l.buf[p] == '-') {
p++
}
digits, dot := 0, false
for p < len(l.buf) {
c := l.buf[p]
switch {
case c >= '0' && c <= '9':
digits++
case c == '.' && !dot:
dot = true
default:
goto done
}
p++
}
done:
s := l.buf[l.pos:p]
l.pos = p
if digits == 0 {
return token{}, &SyntaxError{start, "malformed number " + strconv.Quote(string(s))}
}
if i, err := strconv.ParseInt(string(s), 10, 64); err == nil {
return token{kind: tokInteger, i: i, f: float64(i), pos: start}, nil
}
f, err := parseReal(s)
if err != nil {
return token{}, &SyntaxError{start, "malformed number " + strconv.Quote(string(s))}
}
return token{kind: tokReal, i: int64(f), f: f, pos: start}, nil
}
// parseReal accepts the doubled sign of "--5", which producers do write; only
// the first sign counts. Everything else has already been filtered out by the
// scan above.
func parseReal(s []byte) (float64, error) {
clean := make([]byte, 0, len(s))
i := 0
if i < len(s) && (s[i] == '+' || s[i] == '-') {
clean = append(clean, s[i])
i++
}
for i < len(s) && (s[i] == '+' || s[i] == '-') {
i++
}
clean = append(clean, s[i:]...)
return strconv.ParseFloat(string(clean), 64)
}
// literalString reads a (parenthesised) string, resolving escapes. Nested
// parentheses nest; an end-of-line inside the string, however written, becomes
// a single line feed.
func (l *lexer) literalString(start int) (token, error) {
l.pos++
depth := 1
out := []byte{}
for l.pos < len(l.buf) {
c := l.buf[l.pos]
l.pos++
switch c {
case '\\':
if l.pos >= len(l.buf) {
return token{}, &SyntaxError{start, "unterminated string"}
}
e := l.buf[l.pos]
l.pos++
switch {
case e == 'n':
out = append(out, '\n')
case e == 'r':
out = append(out, '\r')
case e == 't':
out = append(out, '\t')
case e == 'b':
out = append(out, '\b')
case e == 'f':
out = append(out, '\f')
case e == '\r':
// A backslash before an end-of-line continues the line.
if l.pos < len(l.buf) && l.buf[l.pos] == '\n' {
l.pos++
}
case e == '\n':
case e >= '0' && e <= '7':
v := int(e - '0')
for k := 0; k < 2 && l.pos < len(l.buf) && l.buf[l.pos] >= '0' && l.buf[l.pos] <= '7'; k++ {
v = v*8 + int(l.buf[l.pos]-'0')
l.pos++
}
out = append(out, byte(v))
default:
// \( \) \\ and, per the specification, any other escaped
// character stands for itself.
out = append(out, e)
}
case '(':
depth++
out = append(out, c)
case ')':
depth--
if depth == 0 {
return token{kind: tokString, text: out, pos: start}, nil
}
out = append(out, c)
case '\r':
if l.pos < len(l.buf) && l.buf[l.pos] == '\n' {
l.pos++
}
out = append(out, '\n')
default:
out = append(out, c)
}
}
return token{}, &SyntaxError{start, "unterminated string"}
}
// hexString reads a <hex> string. White-space is ignored and a final odd digit
// is padded with zero, as the specification requires.
func (l *lexer) hexString(start int) (token, error) {
l.pos++
out := []byte{}
hi := -1
for l.pos < len(l.buf) {
c := l.buf[l.pos]
l.pos++
if c == '>' {
if hi >= 0 {
out = append(out, byte(hi<<4))
}
return token{kind: tokString, text: out, pos: start}, nil
}
if isSpace(c) {
continue
}
v := hexVal(c)
if v < 0 {
// Producers do emit a stray byte inside <…>; skipping it is what
// every other reader does, and losing the whole object is worse.
continue
}
if hi < 0 {
hi = v
continue
}
out = append(out, byte(hi<<4|v))
hi = -1
}
return token{}, &SyntaxError{start, "unterminated hex string"}
}
// name reads a /Name, resolving #xx escapes.
func (l *lexer) name(start int) (token, error) {
l.pos++
out := []byte{}
for l.pos < len(l.buf) && isRegular(l.buf[l.pos]) {
c := l.buf[l.pos]
l.pos++
if c != '#' {
out = append(out, c)
continue
}
if l.pos+1 >= len(l.buf) {
return token{}, &SyntaxError{start, "truncated #xx escape in name"}
}
h1, h2 := hexVal(l.buf[l.pos]), hexVal(l.buf[l.pos+1])
if h1 < 0 || h2 < 0 {
return token{}, &SyntaxError{start, "invalid #xx escape in name"}
}
out = append(out, byte(h1<<4|h2))
l.pos += 2
}
return token{kind: tokName, text: out, pos: start}, nil
}
vendor/github.com/go-pdfkit/reader/lzw.go
Generated Vendored
+86
View File
@@ -0,0 +1,86 @@
package reader
import "fmt"
// LZW code points that are not table entries.
const (
lzwClear = 256
lzwEOD = 257
lzwFirst = 258
lzwMax = 4096
)
// lzwDecode expands PDF's variable-code-width LZW.
//
// The standard library's compress/lzw cannot be used here: PDF's /EarlyChange
// parameter defaults to 1, which widens the code one entry before the table
// actually fills, and compress/lzw implements neither that convention nor a
// way to ask for it. Getting this wrong does not fail — it silently produces
// plausible rubbish a few hundred bytes in.
func lzwDecode(data []byte, early bool) ([]byte, error) {
var table [lzwMax][]byte
for i := 0; i < 256; i++ {
table[i] = []byte{byte(i)}
}
next := lzwFirst
width := 9
bit := 0
var out, prev []byte
for {
if bit+width > len(data)*8 {
// A truncated stream keeps what it managed to say.
return out, nil
}
code := 0
for k := 0; k < width; k++ {
code = code<<1 | int(data[(bit+k)/8]>>(7-(bit+k)%8)&1)
}
bit += width
switch code {
case lzwEOD:
return out, nil
case lzwClear:
next, width, prev = lzwFirst, 9, nil
continue
}
var entry []byte
switch {
case code < next && table[code] != nil:
entry = table[code]
case code == next && prev != nil:
// The encoder may name the entry it is about to define.
entry = append(append([]byte{}, prev...), prev[0])
default:
// What the stream did say is kept, with the reason it stopped.
return out, fmt.Errorf("reader: LZWDecode: code %d is not in the table", code)
}
out = append(out, entry...)
if prev != nil && next < lzwMax {
table[next] = append(append([]byte{}, prev...), entry[0])
next++
}
prev = entry
// With EarlyChange the width grows one entry sooner. Both conventions
// are the same comparison against a shifted count.
count := next
if early {
count++
}
switch {
case count >= 2048:
width = 12
case count >= 1024:
width = 11
case count >= 512:
width = 10
default:
width = 9
}
}
}
vendor/github.com/go-pdfkit/reader/object.go
Generated Vendored
+244
View File
@@ -0,0 +1,244 @@
// Package reader takes existing PDF bytes apart: the object syntax, the
// cross-reference machinery, stream filters, and the document structure built
// on top of them. It is the parsing counterpart of github.com/go-pdfkit/pdfkit,
// which writes PDF.
//
// Nothing outside the Go standard library is used, so the package builds for
// js/wasm and for every architecture the fleet targets.
package reader
import "fmt"
// Kind names the eight basic PDF object types plus the two the file format
// adds on top of them: streams and indirect references.
type Kind uint8
// The object kinds. [Object.Kind] reports one of these.
const (
KindNull Kind = iota
KindBool
KindInteger
KindReal
KindString
KindName
KindArray
KindDict
KindStream
KindRef
)
var kindNames = [...]string{
KindNull: "null",
KindBool: "boolean",
KindInteger: "integer",
KindReal: "real",
KindString: "string",
KindName: "name",
KindArray: "array",
KindDict: "dictionary",
KindStream: "stream",
KindRef: "reference",
}
// String names the kind, for error messages.
func (k Kind) String() string {
if int(k) >= len(kindNames) {
return fmt.Sprintf("Kind(%d)", uint8(k))
}
return kindNames[k]
}
// An Object is any value the PDF object syntax can express. The concrete types
// are [Null], [Bool], [Integer], [Real], [String], [Name], [Array], [Dict],
// [Ref] and *[Stream]; Kind tells them apart without a type switch.
type Object interface {
Kind() Kind
}
// Null is the PDF null object. A missing dictionary entry resolves to Null
// rather than to a nil Object, so callers never have to nil-check.
type Null struct{}
// Kind implements [Object].
func (Null) Kind() Kind { return KindNull }
// Bool is a PDF boolean.
type Bool bool
// Kind implements [Object].
func (Bool) Kind() Kind { return KindBool }
// Integer is a PDF integer. Numbers too large for an int64 are parsed as
// [Real], which is what every other reader does.
type Integer int64
// Kind implements [Object].
func (Integer) Kind() Kind { return KindInteger }
// Real is a PDF real number.
type Real float64
// Kind implements [Object].
func (Real) Kind() Kind { return KindReal }
// String is a PDF string, already unescaped: the bytes a literal or hex string
// denotes, with no interpretation of their text encoding.
type String []byte
// Kind implements [Object].
func (String) Kind() Kind { return KindString }
// Name is a PDF name with its #xx escapes resolved and the leading slash
// dropped, so /Type is Name("Type").
type Name string
// Kind implements [Object].
func (Name) Kind() Kind { return KindName }
// Array is a PDF array.
type Array []Object
// Kind implements [Object].
func (Array) Kind() Kind { return KindArray }
// Dict is a PDF dictionary.
type Dict map[Name]Object
// Kind implements [Object].
func (Dict) Kind() Kind { return KindDict }
// Get returns the entry, or [Null] when it is absent. The value may still be a
// [Ref]; use [Resolve] to follow it.
func (d Dict) Get(k Name) Object {
if v, ok := d[k]; ok && v != nil {
return v
}
return Null{}
}
// Ref is an indirect reference: the "12 0 R" that stands in for an object
// stored elsewhere in the file.
type Ref struct {
Num int
Gen int
}
// Kind implements [Object].
func (Ref) Kind() Kind { return KindRef }
// String renders the reference the way it appears in a file.
func (r Ref) String() string { return fmt.Sprintf("%d %d R", r.Num, r.Gen) }
// Stream is a PDF stream: a dictionary plus the raw, still-encoded bytes
// between the stream and endstream keywords. [Decode] applies the filter chain.
type Stream struct {
Dict Dict
Raw []byte
}
// Kind implements [Object].
func (*Stream) Kind() Kind { return KindStream }
// A Resolver fetches the object an indirect reference names. A document
// supplies one; a nil Resolver makes every reference resolve to [Null], which
// is what a caller parsing an isolated fragment wants.
type Resolver func(Ref) (Object, error)
// maxRefDepth bounds a chain of references pointing at references. A file can
// make that chain a cycle, and a cycle must not hang the reader.
const maxRefDepth = 64
// Resolve follows indirect references until it reaches a direct object. A nil
// object, an absent entry and an unresolvable reference all yield [Null].
func Resolve(o Object, r Resolver) (Object, error) {
if o == nil {
return Null{}, nil
}
for depth := 0; ; depth++ {
ref, ok := o.(Ref)
if !ok {
return o, nil
}
if r == nil {
return Null{}, nil
}
if depth >= maxRefDepth {
return nil, fmt.Errorf("reader: reference chain longer than %d at %s", maxRefDepth, ref)
}
v, err := r(ref)
if err != nil {
return nil, err
}
if v == nil {
return Null{}, nil
}
o = v
}
}
// ToBool reports the value of a boolean object.
func ToBool(o Object) (bool, bool) {
v, ok := o.(Bool)
return bool(v), ok
}
// ToInt reports the value of an integer object, and of a real whose value is a
// whole number — files do write "3.0" where an integer belongs.
func ToInt(o Object) (int64, bool) {
switch v := o.(type) {
case Integer:
return int64(v), true
case Real:
if float64(int64(v)) == float64(v) {
return int64(v), true
}
}
return 0, false
}
// ToFloat reports the value of any numeric object.
func ToFloat(o Object) (float64, bool) {
switch v := o.(type) {
case Integer:
return float64(v), true
case Real:
return float64(v), true
}
return 0, false
}
// ToString reports the bytes of a string object.
func ToString(o Object) ([]byte, bool) {
v, ok := o.(String)
return []byte(v), ok
}
// ToName reports the value of a name object.
func ToName(o Object) (Name, bool) {
v, ok := o.(Name)
return v, ok
}
// ToArray reports the elements of an array object.
func ToArray(o Object) (Array, bool) {
v, ok := o.(Array)
return v, ok
}
// ToDict reports the dictionary of a dictionary object, or a stream's own
// dictionary — a stream is a dictionary everywhere a dictionary is expected.
func ToDict(o Object) (Dict, bool) {
switch v := o.(type) {
case Dict:
return v, true
case *Stream:
return v.Dict, true
}
return nil, false
}
// ToStream reports the stream object.
func ToStream(o Object) (*Stream, bool) {
v, ok := o.(*Stream)
return v, ok
}
vendor/github.com/go-pdfkit/reader/objstm.go
Generated Vendored
+128
View File
@@ -0,0 +1,128 @@
package reader
import (
"bytes"
"compress/zlib"
"fmt"
)
// objectsPerStream bounds how many objects go into one object stream. A
// stream has to be inflated whole to read any object in it, so packing
// everything into one would make opening a file read all of it.
const objectsPerStream = 200
// packedAt says which object stream an object was packed into, and where.
type packedAt struct {
stream int
index int
}
// pendingObject is an object waiting to be packed.
type pendingObject struct {
ref Ref
obj Object
}
// packObjects writes every waiting object into object streams. It is called
// once, from Finish.
func (w *Writer) packObjects() {
for start := 0; start < len(w.pending); start += objectsPerStream {
end := start + objectsPerStream
if end > len(w.pending) {
end = len(w.pending)
}
w.packGroup(w.pending[start:end])
}
w.pending = nil
}
// packGroup writes one object stream holding the given objects.
func (w *Writer) packGroup(group []pendingObject) {
var head, payload bytes.Buffer
for _, p := range group {
fmt.Fprintf(&head, "%d %d ", p.ref.Num, payload.Len())
payload.Write(AppendObject(nil, p.obj))
payload.WriteByte('\n')
}
body := append(head.Bytes(), payload.Bytes()...)
stream := w.Reserve()
for i, p := range group {
w.packed[p.ref.Num] = packedAt{stream: stream.Num, index: i}
}
w.writeInline(stream, &Stream{
Dict: Dict{
"Type": Name("ObjStm"),
"N": Integer(len(group)),
"First": Integer(head.Len()),
},
Raw: body,
})
}
// flateCompress compresses a stream's data, which is what makes packing worth doing.
func flateCompress(data []byte) []byte {
var buf bytes.Buffer
zw := zlib.NewWriter(&buf)
// A bytes.Buffer never refuses bytes, and Close only flushes.
zw.Write(data)
zw.Close()
return buf.Bytes()
}
// finishWithXrefStream writes the cross-reference information as a stream
// rather than a table, which is the only form that can name an object inside
// an object stream.
func (w *Writer) finishWithXrefStream(trailer Dict) ([]byte, error) {
// Finish has already written the /Encrypt dictionary and put what a
// reader needs into the trailer.
w.packObjects()
xref := w.Reserve()
// The cross-reference stream is reserved after everything else — the
// object streams included — so it always carries the highest number.
high := xref.Num
start := w.buf.Len()
w.offsets[xref.Num] = start
rows := make([]byte, 0, (high+1)*7)
for num := 0; num <= high; num++ {
switch {
case num == 0:
rows = append(rows, 0, 0, 0, 0, 0, 0xFF, 0xFF)
case w.offsets[num] != 0 || num == xref.Num:
rows = append(rows, xrefRow(1, int64(w.offsets[num]), 0)...)
default:
if at, ok := w.packed[num]; ok {
rows = append(rows, xrefRow(2, int64(at.stream), int64(at.index))...)
continue
}
rows = append(rows, 0, 0, 0, 0, 0, 0xFF, 0xFF)
}
}
dict := Dict{
"Type": Name("XRef"),
"Size": Integer(high + 1),
"W": Array{Integer(1), Integer(4), Integer(2)},
}
for k, v := range trailer {
dict[k] = v
}
w.writeInlineRaw(xref, &Stream{Dict: dict, Raw: rows})
fmt.Fprintf(&w.buf, "startxref\n%d\n%%%%EOF\n", start)
if w.encrypt != nil && w.encrypt.err != nil {
w.note(w.encrypt.err)
}
if w.err != nil {
return nil, w.err
}
return w.buf.Bytes(), nil
}
// xrefRow renders one three-field row of a cross-reference stream.
func xrefRow(kind byte, f2, f3 int64) []byte {
return []byte{
kind,
byte(f2 >> 24), byte(f2 >> 16), byte(f2 >> 8), byte(f2),
byte(f3 >> 8), byte(f3),
}
}
vendor/github.com/go-pdfkit/reader/parse.go
Generated Vendored
+295
View File
@@ -0,0 +1,295 @@
package reader
import (
"bytes"
"strconv"
)
// A parser turns the token stream into objects. resolve is consulted only for
// a stream's /Length, which files are allowed to store indirectly.
type parser struct {
lex lexer
resolve Resolver
}
// ParseObject parses a single direct object from the start of b and reports how
// many bytes it consumed.
func ParseObject(b []byte) (Object, int, error) {
p := &parser{lex: lexer{buf: b}}
o, err := p.parseObject()
if err != nil {
return nil, 0, err
}
return o, p.lex.pos, nil
}
// ParseIndirectObject parses "N G obj … endobj" from the start of b, returning
// the reference it defines, the object itself, and how many bytes it consumed.
// A stream's /Length may be an indirect reference; resolve supplies it, and a
// nil or unhelpful resolve falls back to scanning for the endstream keyword.
func ParseIndirectObject(b []byte, resolve Resolver) (Ref, Object, int, error) {
p := &parser{lex: lexer{buf: b}, resolve: resolve}
num, err := p.expectUint("object number")
if err != nil {
return Ref{}, nil, 0, err
}
gen, err := p.expectUint("generation number")
if err != nil {
return Ref{}, nil, 0, err
}
if err := p.expectKeyword("obj"); err != nil {
return Ref{}, nil, 0, err
}
ref := Ref{Num: num, Gen: gen}
// An indirect object may have no value at all — "7 0 obj endobj" —
// which the corpus does contain. Its value is null.
if p.accept("endobj") {
return ref, Null{}, p.lex.pos, nil
}
obj, err := p.parseObject()
if err != nil {
return ref, nil, 0, err
}
obj, err = p.maybeStream(obj)
if err != nil {
return ref, nil, 0, err
}
p.accept("endobj")
return ref, obj, p.lex.pos, nil
}
// expectUint reads a non-negative integer token.
func (p *parser) expectUint(what string) (int, error) {
t, err := p.lex.next()
if err != nil {
return 0, err
}
if t.kind != tokInteger || t.i < 0 {
return 0, &SyntaxError{t.pos, what + " expected"}
}
return int(t.i), nil
}
// expectKeyword requires the given keyword next.
func (p *parser) expectKeyword(kw string) error {
t, err := p.lex.next()
if err != nil {
return err
}
if t.kind != tokKeyword || string(t.text) != kw {
return &SyntaxError{t.pos, strconv.Quote(kw) + " expected"}
}
return nil
}
// accept consumes the keyword if it is next, and reports whether it was.
func (p *parser) accept(kw string) bool {
save := p.lex.pos
t, err := p.lex.next()
if err == nil && t.kind == tokKeyword && string(t.text) == kw {
return true
}
p.lex.pos = save
return false
}
// maybeStream turns a dictionary followed by the stream keyword into a Stream.
func (p *parser) maybeStream(obj Object) (Object, error) {
save := p.lex.pos
t, err := p.lex.next()
if err != nil || t.kind != tokKeyword || string(t.text) != "stream" {
p.lex.pos = save
return obj, nil
}
d, ok := obj.(Dict)
if !ok {
return nil, &SyntaxError{t.pos, "stream keyword after a " + obj.Kind().String()}
}
return p.readStream(d, t.pos)
}
// readStream extracts the bytes between stream and endstream. The declared
// /Length is trusted only when endstream really does follow it — producers get
// it wrong often enough that the scan below is the common path, not the
// exception.
func (p *parser) readStream(d Dict, kwPos int) (Object, error) {
b := p.lex.buf
i := p.lex.pos
for i < len(b) && (b[i] == ' ' || b[i] == '\t') {
i++
}
if i < len(b) && b[i] == '\r' {
i++
}
if i < len(b) && b[i] == '\n' {
i++
}
start := i
if n, ok := p.streamLength(d); ok && start+n <= len(b) {
if end := endstreamAt(b, start+n); end >= 0 {
p.lex.pos = end
return &Stream{Dict: d, Raw: b[start : start+n]}, nil
}
}
j := bytes.Index(b[start:], []byte("endstream"))
if j < 0 {
return nil, &SyntaxError{kwPos, "unterminated stream"}
}
end := start + j
p.lex.pos = end + len("endstream")
// The end-of-line that precedes endstream belongs to the file, not to the
// stream's data.
if end > start && b[end-1] == '\n' {
end--
}
if end > start && b[end-1] == '\r' {
end--
}
return &Stream{Dict: d, Raw: b[start:end]}, nil
}
// streamLength reads /Length, following an indirect reference when it can.
func (p *parser) streamLength(d Dict) (int, bool) {
o, err := Resolve(d.Get("Length"), p.resolve)
if err != nil {
return 0, false
}
n, ok := ToInt(o)
if !ok || n < 0 {
return 0, false
}
return int(n), true
}
// endstreamAt reports the offset just past an endstream keyword that follows
// white-space at i, or -1 when something else is there.
func endstreamAt(b []byte, i int) int {
for i < len(b) && isSpace(b[i]) {
i++
}
if bytes.HasPrefix(b[i:], []byte("endstream")) {
return i + len("endstream")
}
return -1
}
// parseObject reads one object.
func (p *parser) parseObject() (Object, error) {
t, err := p.lex.next()
if err != nil {
return nil, err
}
return p.parseFrom(t)
}
// parseFrom reads the object that starts with an already-read token.
func (p *parser) parseFrom(t token) (Object, error) {
switch t.kind {
case tokEOF:
return nil, &SyntaxError{t.pos, "unexpected end of input"}
case tokInteger:
return p.maybeRef(t)
case tokReal:
return Real(t.f), nil
case tokString:
return String(t.text), nil
case tokName:
return Name(t.text), nil
case tokArrayOpen:
return p.parseArray()
case tokDictOpen:
return p.parseDict()
case tokKeyword:
switch string(t.text) {
case "true":
return Bool(true), nil
case "false":
return Bool(false), nil
case "null":
return Null{}, nil
}
return nil, &SyntaxError{t.pos, "unexpected keyword " + strconv.Quote(string(t.text))}
}
return nil, &SyntaxError{t.pos, "unexpected token"}
}
// maybeRef decides between the integer just read and the "N G R" that starts
// the same way. Only two tokens of lookahead separate them.
func (p *parser) maybeRef(t token) (Object, error) {
save := p.lex.pos
if t.i >= 0 {
if t2, err := p.lex.next(); err == nil && t2.kind == tokInteger && t2.i >= 0 {
if t3, err := p.lex.next(); err == nil && t3.kind == tokKeyword && string(t3.text) == "R" {
return Ref{Num: int(t.i), Gen: int(t2.i)}, nil
}
}
}
p.lex.pos = save
return Integer(t.i), nil
}
// parseArray reads the body of an array, the opening bracket already consumed.
func (p *parser) parseArray() (Object, error) {
arr := Array{}
for {
t, err := p.lex.next()
if err != nil {
return nil, err
}
switch t.kind {
case tokArrayClose:
return arr, nil
case tokEOF:
return nil, &SyntaxError{t.pos, "unterminated array"}
}
o, err := p.parseFrom(t)
if err != nil {
return nil, err
}
arr = append(arr, o)
}
}
// parseDict reads the body of a dictionary, "<<" already consumed.
func (p *parser) parseDict() (Object, error) {
d := Dict{}
for {
t, err := p.lex.next()
if err != nil {
return nil, err
}
switch t.kind {
case tokDictClose:
return d, nil
case tokEOF:
return nil, &SyntaxError{t.pos, "unterminated dictionary"}
case tokName:
default:
return nil, &SyntaxError{t.pos, "dictionary key is a " + t.kind.describe() + ", not a name"}
}
v, err := p.parseObject()
if err != nil {
return nil, err
}
d[Name(t.text)] = v
}
}
// describe names a token kind for an error message.
func (k tokKind) describe() string {
switch k {
case tokInteger, tokReal:
return "number"
case tokString:
return "string"
case tokArrayOpen, tokArrayClose:
return "bracket"
case tokDictOpen, tokDictClose:
return "dictionary delimiter"
case tokBraceOpen, tokBraceClose:
return "brace"
case tokKeyword:
return "keyword"
}
return "token"
}
vendor/github.com/go-pdfkit/reader/predictor.go
Generated Vendored
+130
View File
@@ -0,0 +1,130 @@
package reader
import "fmt"
// applyPredictor undoes the PNG or TIFF predictor a Flate or LZW stream was
// filtered through. /Predictor 1, and an absent parameter dictionary, mean
// there is nothing to undo.
func applyPredictor(data []byte, parm Dict, r Resolver) ([]byte, error) {
pred := intParm(parm, "Predictor", 1, r)
if pred <= 1 {
return data, nil
}
colors := intParm(parm, "Colors", 1, r)
bpc := intParm(parm, "BitsPerComponent", 8, r)
columns := intParm(parm, "Columns", 1, r)
if colors < 1 || bpc < 1 || columns < 1 {
return nil, fmt.Errorf("reader: predictor: /Colors %d /BitsPerComponent %d /Columns %d", colors, bpc, columns)
}
switch {
case pred == 2:
return tiffPredictor(data, colors, bpc, columns)
case pred >= 10:
return pngPredictor(data, colors, bpc, columns)
}
return nil, fmt.Errorf("reader: predictor: /Predictor %d is not defined", pred)
}
// rowGeometry gives the bytes per row and the byte distance between a sample
// and the one to its left, both rounded up as the specification requires.
func rowGeometry(colors, bpc, columns int) (rowLen, bpp int) {
return (colors*bpc*columns + 7) / 8, (colors*bpc + 7) / 8
}
// pngPredictor undoes the per-row filters PNG defines, each row of the stream
// being preceded by its filter type byte.
func pngPredictor(data []byte, colors, bpc, columns int) ([]byte, error) {
rowLen, bpp := rowGeometry(colors, bpc, columns)
prev := make([]byte, rowLen)
cur := make([]byte, rowLen)
out := make([]byte, 0, len(data))
for i := 0; i < len(data); {
ft := data[i]
i++
n := copy(cur, data[i:])
// A damaged file can end mid-row; treat the missing bytes as zero
// rather than dropping the row.
clear(cur[n:])
i += n
switch ft {
case 0:
case 1:
for k := bpp; k < rowLen; k++ {
cur[k] += cur[k-bpp]
}
case 2:
for k := 0; k < rowLen; k++ {
cur[k] += prev[k]
}
case 3:
for k := 0; k < rowLen; k++ {
left := 0
if k >= bpp {
left = int(cur[k-bpp])
}
cur[k] += byte((left + int(prev[k])) / 2)
}
case 4:
for k := 0; k < rowLen; k++ {
var left, upLeft byte
if k >= bpp {
left, upLeft = cur[k-bpp], prev[k-bpp]
}
cur[k] += paeth(left, prev[k], upLeft)
}
default:
return nil, fmt.Errorf("reader: predictor: unknown PNG row filter %d", ft)
}
out = append(out, cur...)
copy(prev, cur)
}
return out, nil
}
// paeth is the PNG predictor of that name: the neighbour closest to a+b-c.
func paeth(a, b, c byte) byte {
p := int(a) + int(b) - int(c)
pa, pb, pc := abs(p-int(a)), abs(p-int(b)), abs(p-int(c))
if pa <= pb && pa <= pc {
return a
}
if pb <= pc {
return b
}
return c
}
// abs is the integer absolute value.
func abs(v int) int {
if v < 0 {
return -v
}
return v
}
// tiffPredictor undoes horizontal differencing. Only the 8- and 16-bit
// component depths are implemented, which is every one seen in practice; a
// sub-byte depth is reported rather than silently mis-decoded.
func tiffPredictor(data []byte, colors, bpc, columns int) ([]byte, error) {
if bpc != 8 && bpc != 16 {
return nil, fmt.Errorf("reader: predictor: TIFF differencing with /BitsPerComponent %d is not supported", bpc)
}
rowLen, bpp := rowGeometry(colors, bpc, columns)
out := append([]byte{}, data...)
for r := 0; r+rowLen <= len(out); r += rowLen {
row := out[r : r+rowLen]
if bpc == 8 {
for k := bpp; k < rowLen; k++ {
row[k] += row[k-bpp]
}
continue
}
for k := bpp; k+1 < rowLen; k += 2 {
v := uint16(row[k])<<8 | uint16(row[k+1])
p := uint16(row[k-bpp])<<8 | uint16(row[k-bpp+1])
v += p
row[k], row[k+1] = byte(v>>8), byte(v)
}
}
return out, nil
}
vendor/github.com/go-pdfkit/reader/repair.go
Generated Vendored
+339
View File
@@ -0,0 +1,339 @@
package reader
import (
"bytes"
"cmp"
"fmt"
"slices"
)
// repair rebuilds the cross-reference information by reading the file itself
// rather than what it says about itself: every "N G obj" header in order, the
// later definition of an object number winning, plus whatever trailer or
// catalogue can be found. Damaged files are the normal case for a reader, not
// an exceptional one.
func (d *Document) repair() error {
d.repaired = true
d.cache = map[int]Object{}
d.objStms = map[int]map[int]Object{}
d.pages = nil
d.xref = map[int]xrefEntry{}
for _, h := range scanObjectHeaders(d.buf) {
// A later definition wins, so the entry is overwritten unconditionally
// rather than through setEntry.
d.xref[h.num] = xrefEntry{kind: 'n', offset: int64(h.offset), gen: h.gen}
}
if len(d.xref) == 0 {
return fmt.Errorf("reader: the file holds no indirect objects")
}
// A rebuild reads object streams, and in an encrypted file it cannot read
// one until the file key is known. The trailers a scan can see are where
// /Encrypt is named, so the key is established before the streams are
// indexed rather than after: otherwise a file whose catalogue lives in an
// encrypted object stream is rebuilt from bytes nobody can read, and the
// rebuild fails for a reason that has nothing to do with the file.
if err := d.establishDecryption(); err != nil {
return err
}
// Objects held in object streams are invisible to a header scan; take them
// from every object stream the scan did find.
d.indexObjectStreams()
d.loadRepairedTrailer()
if d.trailer == nil {
return fmt.Errorf("reader: no document catalogue found")
}
return nil
}
// establishDecryption derives the file key from a trailer the scan can see, for
// a rebuild that runs before the cross-reference tables could name /Encrypt.
// A file that is not encrypted, or whose key is already known, needs nothing.
func (d *Document) establishDecryption() error {
if d.decrypt != nil || !bytes.Contains(d.buf, []byte("/Encrypt")) {
// /Encrypt can only be named by a trailer, and no trailer is
// compressed, so a file that does not hold those bytes anywhere is not
// encrypted. Saying so cheaply keeps the scan below off the common path.
return nil
}
previous := d.trailer
for _, tr := range d.trailerCandidates() {
if tr == nil || tr.Get("Encrypt").Kind() == KindNull {
continue
}
d.trailer = tr
err := d.setUpDecryption(d.password)
d.trailer = previous
if err != nil {
return err
}
// Whatever was read while the key was unknown was read wrong.
d.cache = map[int]Object{}
return nil
}
return nil
}
// trailerCandidates lists every dictionary that could carry trailer entries:
// what the tables managed to read, then the file's trailer keywords, then the
// dictionary of each cross-reference stream.
//
// The last of those is not optional. A file written with cross-reference
// streams has no trailer keyword anywhere in it, and /Encrypt then exists only
// in a /Type /XRef stream's dictionary — which is the one stream a PDF never
// encrypts, precisely so that it can be read before the key is known.
//
// The cross-reference streams come highest object number first, an incremental
// update having appended both its new objects and its new table to the file.
func (d *Document) trailerCandidates() []Dict {
out := append([]Dict{d.trailer}, scanTrailers(d.buf)...)
nums := make([]int, 0, len(d.xref))
for num := range d.xref {
nums = append(nums, num)
}
slices.Sort(nums)
var streams []Dict
for _, num := range nums {
// An error here leaves a nil object, which is not a stream; the
// rebuild has already been told about anything unreadable.
o, _ := d.Get(Ref{Num: num})
s, ok := ToStream(o)
if !ok {
continue
}
if t, ok := ToName(s.Dict.Get("Type")); ok && t == "XRef" {
streams = append(streams, s.Dict)
}
}
slices.Reverse(streams)
return append(out, streams...)
}
// loadRepairedTrailer finds a trailer that leads to a catalogue: the file's own
// trailer dictionaries first, newest first; then any object that calls itself a
// catalogue; and failing both, a catalogue built over whatever pages survive.
func (d *Document) loadRepairedTrailer() {
previous := d.trailer
for _, tr := range scanTrailers(d.buf) {
d.trailer = tr
if _, err := d.Catalog(); err == nil {
return
}
}
if previous != nil {
d.trailer = previous
if _, err := d.Catalog(); err == nil {
return
}
}
d.trailer = nil
for _, num := range d.objectsOfType("Catalog") {
d.trailer = Dict{"Root": Ref{Num: num}}
if _, err := d.Catalog(); err == nil {
return
}
d.trailer = nil
}
d.synthesiseCatalogue()
}
// objectsOfType lists, in object-number order, the objects whose /Type is the
// given name.
func (d *Document) objectsOfType(want Name) []int {
// The objects that exist are walked, in order, rather than every number
// up to the largest of them. A file may name any object number it likes:
// a 219-byte one in the wild declares object 2147483647, and counting up
// to that is two thousand million map lookups for three objects — which
// is a quarter of a minute of somebody's afternoon for a file that fits
// in a tweet.
nums := make([]int, 0, len(d.xref))
for num := range d.xref {
nums = append(nums, num)
}
slices.Sort(nums)
var out []int
for _, num := range nums {
o, err := d.Get(Ref{Num: num})
if err != nil {
continue
}
dict, ok := ToDict(o)
if !ok {
continue
}
if t, ok := ToName(dict.Get("Type")); ok && t == want {
out = append(out, num)
}
}
return out
}
// synthesiseCatalogue is the last resort for a file whose catalogue and page
// tree did not survive: build one over whatever objects still call themselves
// pages. A truncated file that kept its pages is worth opening; one that kept
// none is not a document.
func (d *Document) synthesiseCatalogue() {
var kids Array
for _, num := range d.objectsOfType("Page") {
kids = append(kids, Ref{Num: num})
}
if len(kids) == 0 {
return
}
d.trailer = Dict{"Root": Dict{
"Type": Name("Catalog"),
"Pages": Dict{
"Type": Name("Pages"),
"Kids": kids,
"Count": Integer(len(kids)),
},
}}
}
// indexObjectStreams adds the objects held inside every object stream the scan
// found, without overwriting an object written directly in the file.
func (d *Document) indexObjectStreams() {
// Two object streams in a damaged file may both claim the same object
// number — a partial rewrite leaves the old one in place beside the new —
// and the loop below lets the first stream walked define it. Walking
// d.xref in map order therefore made the same file read differently from
// one run to the next.
//
// They are walked latest in the file first, which is the rule the header
// scan above already applies to objects written directly: a later
// definition wins, an incremental update having appended it.
nums := make([]int, 0, len(d.xref))
for num := range d.xref {
nums = append(nums, num)
}
slices.Sort(nums)
type located struct {
offset int64
num int
}
var found []located
for _, num := range nums {
o, err := d.Get(Ref{Num: num})
if err != nil {
continue
}
s, ok := ToStream(o)
if !ok {
continue
}
if t, ok := ToName(s.Dict.Get("Type")); ok && t == "ObjStm" {
found = append(found, located{d.xref[num].offset, num})
}
}
slices.SortFunc(found, func(a, b located) int { return cmp.Compare(b.offset, a.offset) })
streams := make([]int, 0, len(found))
for _, f := range found {
streams = append(streams, f.num)
}
for _, num := range streams {
// The stream object is already cached by the pass above, so this
// cannot fail; an empty result simply contributes nothing.
objs, _ := d.objectStream(num)
for objNum, obj := range objs {
if _, ok := d.xref[objNum]; ok {
continue
}
d.xref[objNum] = xrefEntry{kind: 'o', strmNum: num}
d.cache[objNum] = obj
}
}
}
// An objectHeader is one "N G obj" found by scanning.
type objectHeader struct {
num, gen, offset int
}
// scanObjectHeaders finds every indirect object header in the file, in the
// order they appear.
func scanObjectHeaders(b []byte) []objectHeader {
var out []objectHeader
for i := 0; i+3 <= len(b); {
j := bytes.Index(b[i:], []byte("obj"))
if j < 0 {
return out
}
at := i + j
i = at + 3
if at+3 < len(b) && isRegular(b[at+3]) {
continue
}
h, ok := headerBefore(b, at)
if !ok {
continue
}
out = append(out, h)
}
return out
}
// headerBefore reads the "N G" that must precede an obj keyword at at.
func headerBefore(b []byte, at int) (objectHeader, bool) {
p := skipSpaceBack(b, at)
if p == at {
return objectHeader{}, false
}
gen, p, ok := digitsBack(b, p)
if !ok {
return objectHeader{}, false
}
q := skipSpaceBack(b, p)
if q == p {
return objectHeader{}, false
}
num, p, ok := digitsBack(b, q)
if !ok {
return objectHeader{}, false
}
return objectHeader{num: num, gen: gen, offset: p}, true
}
// skipSpaceBack walks back over white-space and returns the new position.
func skipSpaceBack(b []byte, p int) int {
for p > 0 && isSpace(b[p-1]) {
p--
}
return p
}
// digitsBack reads a decimal number that ends at p, going backwards.
func digitsBack(b []byte, p int) (int, int, bool) {
end := p
for p > 0 && b[p-1] >= '0' && b[p-1] <= '9' {
p--
}
if p == end || end-p > 10 {
return 0, p, false
}
v := 0
for _, c := range b[p:end] {
v = v*10 + int(c-'0')
}
return v, p, true
}
// scanTrailers finds the file's trailer dictionaries, newest first.
func scanTrailers(b []byte) []Dict {
var out []Dict
for i := len(b); i > 0; {
j := bytes.LastIndex(b[:i], []byte("trailer"))
if j < 0 {
break
}
i = j
o, _, err := ParseObject(b[j+len("trailer"):])
if err != nil {
continue
}
if tr, ok := ToDict(o); ok {
out = append(out, tr)
}
}
return out
}
vendor/github.com/go-pdfkit/reader/writer.go
Generated Vendored
+432
View File
@@ -0,0 +1,432 @@
package reader
import (
"bytes"
"fmt"
"math"
"sort"
"strconv"
)
// AppendObject writes an object in PDF syntax, appending to dst. Dictionary
// keys are written in order, so the same object always produces the same
// bytes — which is what makes a rewritten file comparable with the one it came
// from.
func AppendObject(dst []byte, o Object) []byte {
switch v := o.(type) {
case nil:
return append(dst, "null"...)
case Null:
return append(dst, "null"...)
case Bool:
if v {
return append(dst, "true"...)
}
return append(dst, "false"...)
case Integer:
return strconv.AppendInt(dst, int64(v), 10)
case Real:
return appendReal(dst, float64(v))
case String:
return appendString(dst, v)
case Name:
return appendName(dst, v)
case Ref:
dst = strconv.AppendInt(dst, int64(v.Num), 10)
dst = append(dst, ' ')
dst = strconv.AppendInt(dst, int64(v.Gen), 10)
return append(dst, " R"...)
case Array:
dst = append(dst, '[')
for i, e := range v {
if i > 0 {
dst = append(dst, ' ')
}
dst = AppendObject(dst, e)
}
return append(dst, ']')
case Dict:
return appendDict(dst, v)
case *Stream:
return appendStream(dst, v)
}
return append(dst, "null"...)
}
// FormatObject renders an object in PDF syntax.
func FormatObject(o Object) []byte { return AppendObject(nil, o) }
// appendReal writes a number the way PDF spells one: no exponent, since the
// format has no notation for it, and no infinity or not-a-number, which it
// cannot represent at all.
func appendReal(dst []byte, f float64) []byte {
if math.IsNaN(f) || math.IsInf(f, 0) {
return append(dst, '0')
}
if f == 0 {
// Negative zero is legal and meaningless, and writing it back out
// would be read as the integer zero, so a rewrite would not be a
// fixpoint. Producers do emit it.
return append(dst, '0')
}
return strconv.AppendFloat(dst, f, 'f', -1, 64)
}
// appendString writes a string in whichever of the two forms a PDF allows is
// shorter. A literal string spells an unprintable byte as a four-character
// octal escape, so text in UTF-16 — which is how a PDF says anything that is
// not Latin-1 — comes out four times its length; the hex form costs two
// characters a byte whatever the byte is.
func appendString(dst []byte, s []byte) []byte {
if literalStringCost(s) > 2*len(s)+2 {
return appendHexString(dst, s)
}
return appendLiteralString(dst, s)
}
// literalStringCost is how many characters the literal form would take.
func literalStringCost(s []byte) int {
n := 2
for _, c := range s {
switch {
case c == '(' || c == ')' || c == '\\' || c == '\n' || c == '\r' || c == '\t':
n += 2
case c < 32 || c > 126:
n += 4
default:
n++
}
}
return n
}
// appendHexString writes the <hex> form.
func appendHexString(dst []byte, s []byte) []byte {
const hex = "0123456789ABCDEF"
dst = append(dst, '<')
for _, c := range s {
dst = append(dst, hex[c>>4], hex[c&15])
}
return append(dst, '>')
}
// appendLiteralString writes the (parenthesised) form, escaping only what has
// to be escaped and rendering anything unprintable as an octal escape.
func appendLiteralString(dst []byte, s []byte) []byte {
dst = append(dst, '(')
for _, c := range s {
switch {
case c == '(' || c == ')' || c == '\\':
dst = append(dst, '\\', c)
case c == '\n':
dst = append(dst, '\\', 'n')
case c == '\r':
dst = append(dst, '\\', 'r')
case c == '\t':
dst = append(dst, '\\', 't')
case c < 32 || c > 126:
dst = append(dst, '\\')
dst = append(dst, '0'+c>>6&7, '0'+c>>3&7, '0'+c&7)
default:
dst = append(dst, c)
}
}
return append(dst, ')')
}
// appendName writes a name, escaping every byte that may not appear in one.
func appendName(dst []byte, n Name) []byte {
dst = append(dst, '/')
const hex = "0123456789ABCDEF"
for i := 0; i < len(n); i++ {
c := n[i]
if !isRegular(c) || c == '#' || c < '!' || c > '~' {
dst = append(dst, '#', hex[c>>4], hex[c&15])
continue
}
dst = append(dst, c)
}
return dst
}
// appendDict writes a dictionary with its keys in order.
func appendDict(dst []byte, d Dict) []byte {
keys := make([]string, 0, len(d))
for k := range d {
keys = append(keys, string(k))
}
sort.Strings(keys)
dst = append(dst, "<<"...)
for _, k := range keys {
dst = appendName(dst, Name(k))
dst = append(dst, ' ')
dst = AppendObject(dst, d[Name(k)])
dst = append(dst, ' ')
}
if len(keys) > 0 {
dst = dst[:len(dst)-1]
}
return append(dst, ">>"...)
}
// appendStream writes a stream, its /Length rewritten to the length of the
// data actually being written.
func appendStream(dst []byte, s *Stream) []byte {
d := Dict{}
for k, v := range s.Dict {
d[k] = v
}
d["Length"] = Integer(len(s.Raw))
dst = appendDict(dst, d)
dst = append(dst, "\nstream\n"...)
dst = append(dst, s.Raw...)
return append(dst, "\nendstream"...)
}
// A Writer builds a PDF file out of objects. Numbers are handed out by
// [Writer.Reserve], objects are written with [Writer.Put], and [Writer.Finish]
// adds the cross-reference table and the trailer.
type Writer struct {
buf bytes.Buffer
offsets map[int]int
packed map[int]packedAt
pending []pendingObject
seen map[int]bool
next int
copied map[*Document]map[int]Ref
err error
pack bool
encrypt *encrypter
}
// NewWriter starts a file with the given version in its header, "1.7" when the
// version is empty. Objects are written one after another and the file ends in
// a cross-reference table, which every reader that has ever existed can read.
func NewWriter(version string) *Writer { return newWriter(version, false) }
// NewPackedWriter starts a file that puts what it can into compressed object
// streams and ends in a cross-reference stream. That is smaller — usually by
// a lot, since a file is mostly small dictionaries — and has been readable
// since PDF 1.5, which is 2003; the version in the header is raised to 1.5 if
// it is lower.
func NewPackedWriter(version string) *Writer { return newWriter(version, true) }
// newWriter starts a file of either kind.
func newWriter(version string, pack bool) *Writer {
if version == "" {
version = "1.7"
}
if pack && version < "1.5" {
version = "1.5"
}
w := &Writer{
offsets: map[int]int{},
packed: map[int]packedAt{},
seen: map[int]bool{},
next: 1,
copied: map[*Document]map[int]Ref{},
pack: pack,
}
fmt.Fprintf(&w.buf, "%%PDF-%s\n", version)
// The four bytes above 127 tell every tool downstream that this file is
// not text, which is what the specification asks for.
w.buf.Write([]byte{'%', 0xE2, 0xE3, 0xCF, 0xD3, '\n'})
return w
}
// Reserve hands out an object number that nothing has been written to yet.
func (w *Writer) Reserve() Ref {
r := Ref{Num: w.next}
w.next++
return r
}
// Put writes an object under a reserved number.
func (w *Writer) Put(ref Ref, o Object) {
if w.seen[ref.Num] {
w.note(fmt.Errorf("reader: object %d written twice", ref.Num))
return
}
w.seen[ref.Num] = true
if ref.Num >= w.next {
w.next = ref.Num + 1
}
// A stream cannot go inside another stream, and only generation zero may
// be packed; everything else can wait to be gathered up at the end.
if _, isStream := o.(*Stream); w.pack && !isStream && ref.Gen == 0 {
w.pending = append(w.pending, pendingObject{ref: ref, obj: o})
return
}
w.writeInline(ref, o)
}
// writeInline writes an object where it stands in the file, compressed and
// then encrypted — in that order, since encrypted bytes do not compress.
func (w *Writer) writeInline(ref Ref, o Object) {
o = w.compressStream(o)
if w.encrypt != nil {
o = w.encrypt.apply(ref.Num, ref.Gen, o)
}
w.emit(ref, o)
}
// writeInlineRaw writes an object that must not be encrypted: the /Encrypt
// dictionary itself, and the cross-reference stream, neither of which a
// reader could decrypt, since it needs them to work out how.
func (w *Writer) writeInlineRaw(ref Ref, o Object) {
w.emit(ref, w.compressStream(o))
}
// compressStream deflates a stream that arrives with no filter of its own,
// when the file is being written to be small.
func (w *Writer) compressStream(o Object) Object {
s, ok := o.(*Stream)
if !ok || !w.pack || s.Dict.Get("Filter").Kind() != KindNull {
return o
}
// Nothing this package generated arrives compressed.
out := &Stream{Dict: cloneDict(s.Dict), Raw: flateCompress(s.Raw)}
out.Dict["Filter"] = Name("FlateDecode")
return out
}
// emit writes an object and remembers where it went.
func (w *Writer) emit(ref Ref, o Object) {
w.offsets[ref.Num] = w.buf.Len()
fmt.Fprintf(&w.buf, "%d %d obj\n", ref.Num, ref.Gen)
w.buf.Write(AppendObject(nil, o))
w.buf.WriteString("\nendobj\n")
}
// cloneDict copies a dictionary one level deep.
func cloneDict(d Dict) Dict {
out := Dict{}
for k, v := range d {
out[k] = v
}
return out
}
// Add reserves a number, writes the object under it and returns the reference.
func (w *Writer) Add(o Object) Ref {
ref := w.Reserve()
w.Put(ref, o)
return ref
}
// Err reports the first thing that went wrong while building the file.
func (w *Writer) Err() error { return w.err }
// note keeps the first error.
func (w *Writer) note(err error) {
if w.err == nil {
w.err = err
}
}
// Finish writes the cross-reference table and the trailer and returns the
// file. /Size is filled in; the caller supplies /Root and whatever else the
// trailer needs.
func (w *Writer) Finish(trailer Dict) ([]byte, error) {
// The caller's dictionary is left as it was; what a file needs to say
// about its own encryption is added to a copy.
trailer = cloneDict(trailer)
for k, v := range w.writeEncryptDict() {
trailer[k] = v
}
if w.pack {
return w.finishWithXrefStream(trailer)
}
high := 0
for num := range w.offsets {
if num > high {
high = num
}
}
start := w.buf.Len()
fmt.Fprintf(&w.buf, "xref\n0 %d\n0000000000 65535 f \n", high+1)
for num := 1; num <= high; num++ {
off, ok := w.offsets[num]
if !ok {
w.buf.WriteString("0000000000 65535 f \n")
continue
}
fmt.Fprintf(&w.buf, "%010d 00000 n \n", off)
}
out := Dict{}
for k, v := range trailer {
out[k] = v
}
out["Size"] = Integer(high + 1)
w.buf.WriteString("trailer\n")
w.buf.Write(AppendObject(nil, out))
fmt.Fprintf(&w.buf, "\nstartxref\n%d\n%%%%EOF\n", start)
if w.encrypt != nil && w.encrypt.err != nil {
w.note(w.encrypt.err)
}
if w.err != nil {
return nil, w.err
}
return w.buf.Bytes(), nil
}
// Copy writes an object from a document into this file, following every
// indirect reference it reaches and renumbering as it goes, so that objects
// from several documents can live side by side. An object already copied from
// the same document keeps the number it was given.
func (w *Writer) Copy(d *Document, o Object) Object {
seen, ok := w.copied[d]
if !ok {
seen = map[int]Ref{}
w.copied[d] = seen
}
return w.copyObject(d, seen, o, 0)
}
// copyObject is Copy's recursion.
func (w *Writer) copyObject(d *Document, seen map[int]Ref, o Object, depth int) Object {
if depth > maxCopyDepth {
w.note(fmt.Errorf("reader: an object graph nested deeper than %d was truncated", maxCopyDepth))
return Null{}
}
switch v := o.(type) {
case Ref:
if to, ok := seen[v.Num]; ok {
return to
}
// The number is reserved before the object is read, so a reference
// back to an ancestor finds it rather than recursing.
to := w.Reserve()
seen[v.Num] = to
src, err := d.Get(v)
if err != nil {
w.note(err)
src = Null{}
}
w.Put(to, w.copyObject(d, seen, src, depth+1))
return to
case Array:
out := make(Array, len(v))
for i, e := range v {
out[i] = w.copyObject(d, seen, e, depth+1)
}
return out
case Dict:
out := Dict{}
for k, e := range v {
out[k] = w.copyObject(d, seen, e, depth+1)
}
return out
case *Stream:
out := &Stream{Dict: Dict{}, Raw: v.Raw}
for k, e := range v.Dict {
out.Dict[k] = w.copyObject(d, seen, e, depth+1)
}
return out
}
return o
}
// maxCopyDepth bounds a copy, since a direct object may nest arbitrarily even
// when no reference repeats.
const maxCopyDepth = 256
vendor/github.com/go-pdfkit/reader/xref.go
Generated Vendored
+303
View File
@@ -0,0 +1,303 @@
package reader
import (
"bytes"
"fmt"
)
// An xrefEntry says where object N lives.
type xrefEntry struct {
kind byte // 'n' at a byte offset, 'o' inside an object stream, 'f' free
offset int64 // kind 'n'
gen int // kind 'n'
strmNum int // kind 'o': the object stream's own number
strmIdx int // kind 'o': the index within it
}
// maxXrefSections bounds a /Prev chain, which a file is free to make a cycle.
const maxXrefSections = 1024
// startxrefWindow is how far back from the end the startxref keyword is looked
// for. The specification allows only a short tail; real files sometimes append
// junk, so the window is generous.
const startxrefWindow = 4096
// findStartxref reads the offset the file's last startxref names.
func (d *Document) findStartxref() (int64, error) {
from := len(d.buf) - startxrefWindow
if from < 0 {
from = 0
}
i := bytes.LastIndex(d.buf[from:], []byte("startxref"))
if i < 0 {
return 0, fmt.Errorf("reader: no startxref in the last %d bytes", startxrefWindow)
}
l := &lexer{buf: d.buf, pos: from + i + len("startxref")}
t, err := l.next()
if err != nil {
return 0, err
}
if t.kind != tokInteger || t.i < 0 || t.i >= int64(len(d.buf)) {
return 0, &SyntaxError{t.pos, "startxref does not name an offset inside the file"}
}
return t.i, nil
}
// loadXref follows the chain of cross-reference sections from the last one
// back through /Prev, taking the newest definition of every object and of
// every trailer entry.
func (d *Document) loadXref() error {
off, err := d.findStartxref()
if err != nil {
return err
}
seen := map[int64]bool{}
for n := 0; off > 0 && n < maxXrefSections; n++ {
if seen[off] {
return fmt.Errorf("reader: cross-reference sections form a cycle at offset %d", off)
}
seen[off] = true
tr, err := d.readXrefSection(off)
if err != nil {
return err
}
d.mergeTrailer(tr)
// A hybrid file keeps a second, stream-shaped section for the objects
// its table deliberately hides from old readers.
if x, ok := ToInt(tr.Get("XRefStm")); ok && x > 0 && !seen[x] {
seen[x] = true
if hy, err := d.readXrefSection(x); err == nil {
d.mergeTrailer(hy)
}
}
p, ok := ToInt(tr.Get("Prev"))
if !ok || p <= 0 {
break
}
off = p
}
if len(d.xref) == 0 {
return fmt.Errorf("reader: the cross-reference table is empty")
}
return nil
}
// mergeTrailer keeps the first value seen for each key, sections being read
// newest first.
func (d *Document) mergeTrailer(tr Dict) {
if d.trailer == nil {
d.trailer = Dict{}
}
for k, v := range tr {
if _, ok := d.trailer[k]; !ok {
d.trailer[k] = v
}
}
}
// setEntry records an entry unless a newer section already defined the object.
func (d *Document) setEntry(num int, e xrefEntry) {
if num < 0 {
return
}
if _, ok := d.xref[num]; !ok {
d.xref[num] = e
}
}
// readXrefSection reads whichever of the two forms is at off and returns its
// trailer dictionary.
func (d *Document) readXrefSection(off int64) (Dict, error) {
if off < 0 || off >= int64(len(d.buf)) {
return nil, fmt.Errorf("reader: cross-reference offset %d is outside the file", off)
}
l := &lexer{buf: d.buf, pos: int(off)}
t, err := l.next()
if err != nil {
return nil, err
}
if t.kind == tokKeyword && string(t.text) == "xref" {
return d.readXrefTable(l)
}
_, obj, _, err := ParseIndirectObject(d.buf[off:], nil)
if err != nil {
return nil, err
}
s, ok := ToStream(obj)
if !ok {
return nil, &SyntaxError{int(off), "neither an xref table nor an xref stream"}
}
return d.readXrefStream(s)
}
// readXrefTable reads the classic subsection form, the xref keyword already
// consumed, up to and including its trailer dictionary.
func (d *Document) readXrefTable(l *lexer) (Dict, error) {
p := &parser{lex: *l}
for {
save := p.lex.pos
t, err := p.lex.next()
if err != nil {
return nil, err
}
if t.kind == tokKeyword && string(t.text) == "trailer" {
o, err := p.parseObject()
if err != nil {
return nil, err
}
tr, ok := ToDict(o)
if !ok {
return nil, &SyntaxError{save, "the trailer is not a dictionary"}
}
return tr, nil
}
if t.kind != tokInteger {
return nil, &SyntaxError{t.pos, "a cross-reference subsection header was expected"}
}
start := int(t.i)
count, err := p.expectUint("a subsection entry count")
if err != nil {
return nil, err
}
for i := 0; i < count; i++ {
if err := d.readXrefTableEntry(p, start+i); err != nil {
return nil, err
}
}
}
}
// readXrefTableEntry reads one "offset generation n|f" line.
func (d *Document) readXrefTableEntry(p *parser, num int) error {
t1, err := p.lex.next()
if err != nil {
return err
}
if t1.kind != tokInteger {
return &SyntaxError{t1.pos, "a cross-reference entry offset was expected"}
}
t2, err := p.lex.next()
if err != nil {
return err
}
if t2.kind != tokInteger {
return &SyntaxError{t2.pos, "a cross-reference entry generation was expected"}
}
t3, err := p.lex.next()
if err != nil {
return err
}
if t3.kind != tokKeyword || len(t3.text) != 1 || (t3.text[0] != 'n' && t3.text[0] != 'f') {
return &SyntaxError{t3.pos, "a cross-reference entry must end in n or f"}
}
if t3.text[0] == 'n' {
d.setEntry(num, xrefEntry{kind: 'n', offset: t1.i, gen: int(t2.i)})
return nil
}
d.setEntry(num, xrefEntry{kind: 'f'})
return nil
}
// readXrefStream reads the /Type /XRef stream form. Its own data is never
// encrypted, so it is decoded without the document's decryptor.
func (d *Document) readXrefStream(s *Stream) (Dict, error) {
data, img, err := Decode(s.Dict, s.Raw, nil)
if err != nil {
return nil, err
}
if img != "" {
return nil, fmt.Errorf("reader: the cross-reference stream is filtered as an image")
}
widths, err := xrefWidths(s.Dict)
if err != nil {
return nil, err
}
index, err := xrefIndex(s.Dict)
if err != nil {
return nil, err
}
row := widths[0] + widths[1] + widths[2]
if row == 0 {
return nil, fmt.Errorf("reader: the cross-reference stream's /W entries are all zero")
}
pos := 0
for i := 0; i+1 < len(index); i += 2 {
for k := 0; k < index[i+1]; k++ {
if pos+row > len(data) {
// A section that claims more rows than it carries still
// contributes the rows it does carry.
return s.Dict, nil
}
f1 := xrefField(data[pos:], widths[0], 1)
f2 := xrefField(data[pos+widths[0]:], widths[1], 0)
f3 := xrefField(data[pos+widths[0]+widths[1]:], widths[2], 0)
pos += row
d.storeXrefStreamEntry(index[i]+k, f1, f2, f3)
}
}
return s.Dict, nil
}
// storeXrefStreamEntry turns one decoded row into an entry.
func (d *Document) storeXrefStreamEntry(num int, f1, f2, f3 int64) {
switch f1 {
case 1:
d.setEntry(num, xrefEntry{kind: 'n', offset: f2, gen: int(f3)})
case 2:
d.setEntry(num, xrefEntry{kind: 'o', strmNum: int(f2), strmIdx: int(f3)})
default:
d.setEntry(num, xrefEntry{kind: 'f'})
}
}
// xrefWidths reads /W, which says how many bytes each of the three fields
// occupies.
func xrefWidths(d Dict) ([3]int, error) {
var w [3]int
arr, ok := ToArray(d.Get("W"))
if !ok || len(arr) < 3 {
return w, fmt.Errorf("reader: the cross-reference stream has no usable /W")
}
for i := 0; i < 3; i++ {
n, ok := ToInt(arr[i])
if !ok || n < 0 || n > 8 {
return w, fmt.Errorf("reader: /W[%d] is not a byte width", i)
}
w[i] = int(n)
}
return w, nil
}
// xrefIndex reads /Index, defaulting to the whole range /Size describes.
func xrefIndex(d Dict) ([]int, error) {
arr, ok := ToArray(d.Get("Index"))
if !ok {
size, ok := ToInt(d.Get("Size"))
if !ok || size < 0 {
return nil, fmt.Errorf("reader: the cross-reference stream has neither /Index nor /Size")
}
return []int{0, int(size)}, nil
}
out := make([]int, 0, len(arr))
for _, e := range arr {
n, ok := ToInt(e)
if !ok || n < 0 {
return nil, fmt.Errorf("reader: /Index holds a value that is not a count")
}
out = append(out, int(n))
}
return out, nil
}
// xrefField reads one big-endian field of the given width, or its default when
// the width is zero.
func xrefField(b []byte, width int, def int64) int64 {
if width == 0 {
return def
}
v := int64(0)
for i := 0; i < width; i++ {
v = v<<8 | int64(b[i])
}
return v
}