P5a plan: PDF documents — seven Ornith tasks with acceptance tests first
docs/plans/p5a: 01 kind+fetch limits, 02 pdftext child + runner, 03 dcache PDF tables, 04 pdfpool, 05 pdfdoc render (map/pages/grep), 06 api, 07 digest citations, metrics, CLI, skill, unit, README. Given tests compile against a panic-only skeleton and fail there for the intended reasons; PDFs for tests are built by internal/pdftest (given), plus two qpdf-encrypted fixtures. Owner changes for the run: vendor go-pdfkit/extract v0.3.0 + reader v0.6.0 (pinned via tools.go until task 02); AGENTS.md, tools/run-plan.sh (enforces a Co-Authored-By trailer for the model), docs/implementer-log.md, `make gate`; gofmt (whitespace) on clean.go and guard; TestUnsupportedType uses a zip now that PDFs are supported; requestDeadline 90→210 s and writeTimeout 120→240 s for the PDF path's worst case. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5.5
+1
@@ -0,0 +1 @@
|
||||
coverage.out
|
||||
+29
@@ -0,0 +1,29 @@
|
||||
BSD 3-Clause License
|
||||
|
||||
Copyright (c) 2026 the go-pdfkit/render authors
|
||||
|
||||
Redistribution and use in source and binary forms, with or without
|
||||
modification, are permitted provided that the following conditions are met:
|
||||
|
||||
1. Redistributions of source code must retain the above copyright notice,
|
||||
this list of conditions and the following disclaimer.
|
||||
|
||||
2. Redistributions in binary form must reproduce the above copyright notice,
|
||||
this list of conditions and the following disclaimer in the documentation
|
||||
and/or other materials provided with the distribution.
|
||||
|
||||
3. Neither the name of the copyright holder nor the names of its contributors
|
||||
may be used to endorse or promote products derived from this software
|
||||
without specific prior written permission.
|
||||
|
||||
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
|
||||
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
||||
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
|
||||
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
|
||||
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
|
||||
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
|
||||
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
|
||||
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
|
||||
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
|
||||
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
|
||||
POSSIBILITY OF SUCH DAMAGE.
|
||||
+75
@@ -0,0 +1,75 @@
|
||||
# pdffont
|
||||
|
||||
[](https://github.com/go-pdfkit/pdffont/actions/workflows/ci.yml)
|
||||
[](https://pkg.go.dev/github.com/go-pdfkit/pdffont)
|
||||
[](LICENSE)
|
||||
[](#how-it-is-checked)
|
||||
|
||||
The PDF side of a font: not the outlines, which are the font program's
|
||||
business, but everything the document itself says about how a string of
|
||||
bytes is to be read.
|
||||
|
||||
Which codes a string holds, how wide each one is, what each is called, and
|
||||
what text each stands for. It is what a renderer needs before it can ask a
|
||||
font program for a glyph, and all an extractor needs to read a page back as
|
||||
words — so neither has to say it twice.
|
||||
|
||||
Pure Go, no C, nothing outside [`reader`](https://github.com/go-pdfkit/reader)
|
||||
and the standard library.
|
||||
|
||||
## What it reads
|
||||
|
||||
Simple fonts, addressed one byte at a time through a base encoding and
|
||||
whatever differences the document names on top of it. Composite fonts,
|
||||
addressed by character identifier, with their widths and their map from
|
||||
identifier to glyph. Type 3 fonts, whose glyphs are little drawings and
|
||||
whose widths are in a space of their own.
|
||||
|
||||
And `/ToUnicode`, which is the one thing in a PDF that says what its text
|
||||
actually **says**. Without it a page can be drawn but not read.
|
||||
|
||||
## What it refuses to guess
|
||||
|
||||
A font that says it is **symbolic** is addressed through its own character
|
||||
map, and a base encoding is a guess at it — a bad one. A mathematical font
|
||||
puts a capital gamma where the standard encoding puts an inverted
|
||||
exclamation mark, so reading the name gives the wrong letter with nothing
|
||||
to say so.
|
||||
|
||||
Such a code is left unread unless the document named it itself, or unless a
|
||||
caller installs [`Font.SetFallback`](https://pkg.go.dev/github.com/go-pdfkit/pdffont#Font.SetFallback)
|
||||
to say what the embedded program calls it. Reading the program is another
|
||||
package's work; `extract` does it with
|
||||
[`go-opentype`](https://github.com/go-opentype/opentype).
|
||||
|
||||
## How it is checked
|
||||
|
||||
100% of statements, including every branch that handles a dictionary saying
|
||||
something it should not.
|
||||
|
||||
And the glyph-name table — which is the one piece of data here, and where a
|
||||
single wrong entry would silently give the wrong letter — is held up
|
||||
against the documents themselves. A font that both names a code and carries
|
||||
a `/ToUnicode` entry for it states the same fact twice, so the two can be
|
||||
compared. Over 12 000 corpus files: **63 515 names checked, 63 074 agreed —
|
||||
99.31%**. The disagreements are producers whose own maps are wrong
|
||||
(`alpha` → `↵`, `ff` → `!`), plus `Delta` and `Omega`, which Adobe's list
|
||||
gives as the increment and ohm signs; the Greek letters written here are
|
||||
the same characters by Unicode's own equivalence, and the ones a person
|
||||
searching would type.
|
||||
|
||||
## The rest of the family
|
||||
|
||||
[**reader**](https://github.com/go-pdfkit/reader) reads and writes the format ·
|
||||
[**ops**](https://github.com/go-pdfkit/ops) is the verbs and the `pdfops` command ·
|
||||
[**render**](https://github.com/go-pdfkit/render) turns a page into pixels ·
|
||||
[**pdffont**](https://github.com/go-pdfkit/pdffont) is what a document says about a font ·
|
||||
[**extract**](https://github.com/go-pdfkit/extract) reads a page back ·
|
||||
[**coedit**](https://github.com/go-pdfkit/coedit) is a PDF several people edit at once ·
|
||||
[**app**](https://github.com/go-pdfkit/app) is the browser workbench.
|
||||
|
||||
Documentation for all of it: <https://go-pdfkit.github.io/docs/>
|
||||
|
||||
## Licence
|
||||
|
||||
BSD-3-Clause.
|
||||
+302
@@ -0,0 +1,302 @@
|
||||
package pdffont
|
||||
|
||||
import "strings"
|
||||
|
||||
// The two encodings a simple font is addressed through when it does not carry
|
||||
// one of its own, as a name per code. Only the codes that name a glyph appear;
|
||||
// the rest are blank.
|
||||
//
|
||||
// These are the tables the specification prints. They matter because a font
|
||||
// program is addressed by glyph name or by character, and a content stream is
|
||||
// written in single bytes: without them, every byte would have to be guessed
|
||||
// at.
|
||||
|
||||
// StandardEncoding is Adobe's own, which most Type 1 fonts assume, and what a
|
||||
// simple font is read with when it says nothing else.
|
||||
var StandardEncoding = [256]string{
|
||||
32: "space", 33: "exclam", 34: "quotedbl", 35: "numbersign", 36: "dollar",
|
||||
37: "percent", 38: "ampersand", 39: "quoteright", 40: "parenleft",
|
||||
41: "parenright", 42: "asterisk", 43: "plus", 44: "comma", 45: "hyphen",
|
||||
46: "period", 47: "slash", 48: "zero", 49: "one", 50: "two", 51: "three",
|
||||
52: "four", 53: "five", 54: "six", 55: "seven", 56: "eight", 57: "nine",
|
||||
58: "colon", 59: "semicolon", 60: "less", 61: "equal", 62: "greater",
|
||||
63: "question", 64: "at", 65: "A", 66: "B", 67: "C", 68: "D", 69: "E",
|
||||
70: "F", 71: "G", 72: "H", 73: "I", 74: "J", 75: "K", 76: "L", 77: "M",
|
||||
78: "N", 79: "O", 80: "P", 81: "Q", 82: "R", 83: "S", 84: "T", 85: "U",
|
||||
86: "V", 87: "W", 88: "X", 89: "Y", 90: "Z", 91: "bracketleft",
|
||||
92: "backslash", 93: "bracketright", 94: "asciicircum", 95: "underscore",
|
||||
96: "quoteleft", 97: "a", 98: "b", 99: "c", 100: "d", 101: "e", 102: "f",
|
||||
103: "g", 104: "h", 105: "i", 106: "j", 107: "k", 108: "l", 109: "m",
|
||||
110: "n", 111: "o", 112: "p", 113: "q", 114: "r", 115: "s", 116: "t",
|
||||
117: "u", 118: "v", 119: "w", 120: "x", 121: "y", 122: "z",
|
||||
123: "braceleft", 124: "bar", 125: "braceright", 126: "asciitilde",
|
||||
161: "exclamdown", 162: "cent", 163: "sterling", 164: "fraction",
|
||||
165: "yen", 166: "florin", 167: "section", 168: "currency",
|
||||
169: "quotesingle", 170: "quotedblleft", 171: "guillemotleft",
|
||||
172: "guilsinglleft", 173: "guilsinglright", 174: "fi", 175: "fl",
|
||||
177: "endash", 178: "dagger", 179: "daggerdbl", 180: "periodcentered",
|
||||
182: "paragraph", 183: "bullet", 184: "quotesinglbase",
|
||||
185: "quotedblbase", 186: "quotedblright", 187: "guillemotright",
|
||||
188: "ellipsis", 189: "perthousand", 191: "questiondown", 193: "grave",
|
||||
194: "acute", 195: "circumflex", 196: "tilde", 197: "macron", 198: "breve",
|
||||
199: "dotaccent", 200: "dieresis", 202: "ring", 203: "cedilla",
|
||||
205: "hungarumlaut", 206: "ogonek", 207: "caron", 208: "emdash",
|
||||
225: "AE", 227: "ordfeminine", 232: "Lslash", 233: "Oslash", 234: "OE",
|
||||
235: "ordmasculine", 241: "ae", 245: "dotlessi", 248: "lslash",
|
||||
249: "oslash", 250: "oe", 251: "germandbls",
|
||||
}
|
||||
|
||||
// WinAnsiEncoding is the one a file written on Windows assumes, and the one
|
||||
// most producers name.
|
||||
var WinAnsiEncoding = [256]string{
|
||||
32: "space", 33: "exclam", 34: "quotedbl", 35: "numbersign", 36: "dollar",
|
||||
37: "percent", 38: "ampersand", 39: "quotesingle", 40: "parenleft",
|
||||
41: "parenright", 42: "asterisk", 43: "plus", 44: "comma", 45: "hyphen",
|
||||
46: "period", 47: "slash", 48: "zero", 49: "one", 50: "two", 51: "three",
|
||||
52: "four", 53: "five", 54: "six", 55: "seven", 56: "eight", 57: "nine",
|
||||
58: "colon", 59: "semicolon", 60: "less", 61: "equal", 62: "greater",
|
||||
63: "question", 64: "at", 65: "A", 66: "B", 67: "C", 68: "D", 69: "E",
|
||||
70: "F", 71: "G", 72: "H", 73: "I", 74: "J", 75: "K", 76: "L", 77: "M",
|
||||
78: "N", 79: "O", 80: "P", 81: "Q", 82: "R", 83: "S", 84: "T", 85: "U",
|
||||
86: "V", 87: "W", 88: "X", 89: "Y", 90: "Z", 91: "bracketleft",
|
||||
92: "backslash", 93: "bracketright", 94: "asciicircum", 95: "underscore",
|
||||
96: "grave", 97: "a", 98: "b", 99: "c", 100: "d", 101: "e", 102: "f",
|
||||
103: "g", 104: "h", 105: "i", 106: "j", 107: "k", 108: "l", 109: "m",
|
||||
110: "n", 111: "o", 112: "p", 113: "q", 114: "r", 115: "s", 116: "t",
|
||||
117: "u", 118: "v", 119: "w", 120: "x", 121: "y", 122: "z",
|
||||
123: "braceleft", 124: "bar", 125: "braceright", 126: "asciitilde",
|
||||
128: "Euro", 130: "quotesinglbase", 131: "florin", 132: "quotedblbase",
|
||||
133: "ellipsis", 134: "dagger", 135: "daggerdbl", 136: "circumflex",
|
||||
137: "perthousand", 138: "Scaron", 139: "guilsinglleft", 140: "OE",
|
||||
142: "Zcaron", 145: "quoteleft", 146: "quoteright", 147: "quotedblleft",
|
||||
148: "quotedblright", 149: "bullet", 150: "endash", 151: "emdash",
|
||||
152: "tilde", 153: "trademark", 154: "scaron", 155: "guilsinglright",
|
||||
156: "oe", 158: "zcaron", 159: "Ydieresis", 160: "space",
|
||||
161: "exclamdown", 162: "cent", 163: "sterling", 164: "currency",
|
||||
165: "yen", 166: "brokenbar", 167: "section", 168: "dieresis",
|
||||
169: "copyright", 170: "ordfeminine", 171: "guillemotleft",
|
||||
172: "logicalnot", 173: "hyphen", 174: "registered", 175: "macron",
|
||||
176: "degree", 177: "plusminus", 178: "twosuperior",
|
||||
179: "threesuperior", 180: "acute", 181: "mu", 182: "paragraph",
|
||||
183: "periodcentered", 184: "cedilla", 185: "onesuperior",
|
||||
186: "ordmasculine", 187: "guillemotright", 188: "onequarter",
|
||||
189: "onehalf", 190: "threequarters", 191: "questiondown", 192: "Agrave",
|
||||
193: "Aacute", 194: "Acircumflex", 195: "Atilde", 196: "Adieresis",
|
||||
197: "Aring", 198: "AE", 199: "Ccedilla", 200: "Egrave", 201: "Eacute",
|
||||
202: "Ecircumflex", 203: "Edieresis", 204: "Igrave", 205: "Iacute",
|
||||
206: "Icircumflex", 207: "Idieresis", 208: "Eth", 209: "Ntilde",
|
||||
210: "Ograve", 211: "Oacute", 212: "Ocircumflex", 213: "Otilde",
|
||||
214: "Odieresis", 215: "multiply", 216: "Oslash", 217: "Ugrave",
|
||||
218: "Uacute", 219: "Ucircumflex", 220: "Udieresis", 221: "Yacute",
|
||||
222: "Thorn", 223: "germandbls", 224: "agrave", 225: "aacute",
|
||||
226: "acircumflex", 227: "atilde", 228: "adieresis", 229: "aring",
|
||||
230: "ae", 231: "ccedilla", 232: "egrave", 233: "eacute",
|
||||
234: "ecircumflex", 235: "edieresis", 236: "igrave", 237: "iacute",
|
||||
238: "icircumflex", 239: "idieresis", 240: "eth", 241: "ntilde",
|
||||
242: "ograve", 243: "oacute", 244: "ocircumflex", 245: "otilde",
|
||||
246: "odieresis", 247: "divide", 248: "oslash", 249: "ugrave",
|
||||
250: "uacute", 251: "ucircumflex", 252: "udieresis", 253: "yacute",
|
||||
254: "thorn", 255: "ydieresis",
|
||||
}
|
||||
|
||||
// glyphRunes is what the names above stand for. A font program is looked up by
|
||||
// character, so a name has to become one; anything not here, and not written
|
||||
// in one of the two conventional forms, cannot be looked up that way.
|
||||
var glyphRunes = map[string]rune{
|
||||
"space": ' ', "exclam": '!', "quotedbl": '"', "numbersign": '#',
|
||||
"dollar": '$', "percent": '%', "ampersand": '&', "quotesingle": '\'',
|
||||
"quoteright": '’', "quoteleft": '‘', "parenleft": '(',
|
||||
"parenright": ')', "asterisk": '*', "plus": '+', "comma": ',',
|
||||
"hyphen": '-', "period": '.', "slash": '/', "zero": '0', "one": '1',
|
||||
"two": '2', "three": '3', "four": '4', "five": '5', "six": '6',
|
||||
"seven": '7', "eight": '8', "nine": '9', "colon": ':', "semicolon": ';',
|
||||
"less": '<', "equal": '=', "greater": '>', "question": '?', "at": '@',
|
||||
"bracketleft": '[', "backslash": '\\', "bracketright": ']',
|
||||
"asciicircum": '^', "underscore": '_', "braceleft": '{', "bar": '|',
|
||||
"braceright": '}', "asciitilde": '~', "exclamdown": '¡', "cent": '¢',
|
||||
"sterling": '£', "fraction": '⁄', "yen": '¥', "florin": 'ƒ',
|
||||
"section": '§', "currency": '¤', "quotedblleft": '“',
|
||||
"quotedblright": '”', "guillemotleft": '«', "guillemotright": '»',
|
||||
"guilsinglleft": '‹', "guilsinglright": '›', "fi": 'fi', "fl": 'fl',
|
||||
"endash": '–', "emdash": '—', "dagger": '†', "daggerdbl": '‡',
|
||||
"periodcentered": '·', "paragraph": '¶', "bullet": '•',
|
||||
"quotesinglbase": '‚', "quotedblbase": '„', "ellipsis": '…',
|
||||
"perthousand": '‰', "questiondown": '¿', "grave": '`', "acute": '´',
|
||||
"circumflex": 'ˆ', "tilde": '˜', "macron": '¯', "breve": '˘',
|
||||
"dotaccent": '˙', "dieresis": '¨', "ring": '˚', "cedilla": '¸',
|
||||
"hungarumlaut": '˝', "ogonek": '˛', "caron": 'ˇ', "AE": 'Æ', "ae": 'æ',
|
||||
"ordfeminine": 'ª', "ordmasculine": 'º', "Lslash": 'Ł', "lslash": 'ł',
|
||||
"Oslash": 'Ø', "oslash": 'ø', "OE": 'Œ', "oe": 'œ', "dotlessi": 'ı',
|
||||
"germandbls": 'ß', "Euro": '€', "Scaron": 'Š', "scaron": 'š',
|
||||
"Zcaron": 'Ž', "zcaron": 'ž', "Ydieresis": 'Ÿ', "trademark": '™',
|
||||
"brokenbar": '¦', "copyright": '©', "logicalnot": '¬', "registered": '®',
|
||||
"degree": '°', "plusminus": '±', "twosuperior": '²',
|
||||
"threesuperior": '³', "mu": 'µ', "onesuperior": '¹', "onequarter": '¼',
|
||||
"onehalf": '½', "threequarters": '¾', "multiply": '×', "divide": '÷',
|
||||
"Agrave": 'À', "Aacute": 'Á', "Acircumflex": 'Â', "Atilde": 'Ã',
|
||||
"Adieresis": 'Ä', "Aring": 'Å', "Ccedilla": 'Ç', "Egrave": 'È',
|
||||
"Eacute": 'É', "Ecircumflex": 'Ê', "Edieresis": 'Ë', "Igrave": 'Ì',
|
||||
"Iacute": 'Í', "Icircumflex": 'Î', "Idieresis": 'Ï', "Eth": 'Ð',
|
||||
"Ntilde": 'Ñ', "Ograve": 'Ò', "Oacute": 'Ó', "Ocircumflex": 'Ô',
|
||||
"Otilde": 'Õ', "Odieresis": 'Ö', "Ugrave": 'Ù', "Uacute": 'Ú',
|
||||
"Ucircumflex": 'Û', "Udieresis": 'Ü', "Yacute": 'Ý', "Thorn": 'Þ',
|
||||
"agrave": 'à', "aacute": 'á', "acircumflex": 'â', "atilde": 'ã',
|
||||
"adieresis": 'ä', "aring": 'å', "ccedilla": 'ç', "egrave": 'è',
|
||||
"eacute": 'é', "ecircumflex": 'ê', "edieresis": 'ë', "igrave": 'ì',
|
||||
"iacute": 'í', "icircumflex": 'î', "idieresis": 'ï', "eth": 'ð',
|
||||
"ntilde": 'ñ', "ograve": 'ò', "oacute": 'ó', "ocircumflex": 'ô',
|
||||
"otilde": 'õ', "odieresis": 'ö', "ugrave": 'ù', "uacute": 'ú',
|
||||
"ucircumflex": 'û', "udieresis": 'ü', "yacute": 'ý', "thorn": 'þ',
|
||||
"ydieresis": 'ÿ',
|
||||
}
|
||||
|
||||
// RuneOfGlyphName turns a glyph name into the character it stands for, through
|
||||
// the table above or through the two conventions the specification defines for
|
||||
// naming a character directly. ok is false for a name that says nothing about
|
||||
// which character it is, which a subsetted font's own names often do not.
|
||||
func RuneOfGlyphName(name string) (rune, bool) {
|
||||
s, ok := TextOfGlyphName(name)
|
||||
if !ok {
|
||||
return 0, false
|
||||
}
|
||||
r := []rune(s)
|
||||
if len(r) != 1 {
|
||||
return 0, false
|
||||
}
|
||||
return r[0], true
|
||||
}
|
||||
|
||||
// TextOfGlyphName turns a glyph name into the text it stands for, which is not
|
||||
// always one character: a name may say it is a ligature of several.
|
||||
//
|
||||
// Three of the rules here are the ones the Adobe Glyph List lays down for
|
||||
// reading a name nobody has listed, and each was costing real text. A name may
|
||||
// carry a variant after a full stop — "a.sc" is a small-capital A and is still
|
||||
// an A, 7 272 of them in the corpus. A name may be several component names
|
||||
// with underscores between them — "f_i" is the fi ligature, and without this
|
||||
// "Definition" comes back "Denition", which is not a missing character so much
|
||||
// as a misspelt word. And a name may simply be one nobody thought to list.
|
||||
func TextOfGlyphName(name string) (string, bool) {
|
||||
if name == "" {
|
||||
return "", false
|
||||
}
|
||||
if r, ok := oneGlyphRune(name); ok {
|
||||
return string(r), true
|
||||
}
|
||||
if seq, ok := unicodeSequence(name); ok {
|
||||
return seq, true
|
||||
}
|
||||
// A variant of a character is still that character: the part before the
|
||||
// first full stop is the name, and what follows says which cut of it.
|
||||
if base, _, found := strings.Cut(name, "."); found {
|
||||
if base == "" {
|
||||
return "", false
|
||||
}
|
||||
return TextOfGlyphName(base)
|
||||
}
|
||||
// A name made of parts with underscores between them is the parts, in
|
||||
// order — which is how a ligature is named when it has no name of its own.
|
||||
if strings.Contains(name, "_") {
|
||||
// Every part that can be read contributes at least one character, and
|
||||
// a part that cannot gives up on the whole name, so what comes out of
|
||||
// this is never empty.
|
||||
var out strings.Builder
|
||||
for _, part := range strings.Split(name, "_") {
|
||||
piece, ok := TextOfGlyphName(part)
|
||||
if !ok {
|
||||
return "", false
|
||||
}
|
||||
out.WriteString(piece)
|
||||
}
|
||||
return out.String(), true
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
|
||||
// oneGlyphRune is a name that stands for exactly one character, by any of the
|
||||
// ways a name can.
|
||||
func oneGlyphRune(name string) (rune, bool) {
|
||||
if r, ok := glyphRunes[name]; ok {
|
||||
return r, true
|
||||
}
|
||||
if r, ok := latinExtendedRunes[name]; ok {
|
||||
return r, true
|
||||
}
|
||||
if r, ok := greekAndMathRunes[name]; ok {
|
||||
return r, true
|
||||
}
|
||||
if len(name) == 1 {
|
||||
return rune(name[0]), true
|
||||
}
|
||||
if r, ok := parseHexName(name, "uni", 4); ok {
|
||||
return r, true
|
||||
}
|
||||
if r, ok := parseHexName(name, "u", 4); ok {
|
||||
return r, true
|
||||
}
|
||||
return 0, false
|
||||
}
|
||||
|
||||
// unicodeSequence reads the uniXXXXYYYY form, which names several characters
|
||||
// at once: a letter and the accent that goes over it, or the three pieces a
|
||||
// Hebrew cluster is written in.
|
||||
//
|
||||
// It has to be told apart from something that looks exactly like it. Producers
|
||||
// write a glyph's own number in the same shape — uni00000048 is glyph 72, not
|
||||
// U+0000 followed by U+0048 — and 49 740 names in the corpus are that. The two
|
||||
// are separable by one observation: a real sequence never begins with U+0000,
|
||||
// because nothing is written after a character that does not exist. Of the
|
||||
// 3 875 genuine sequences found across 14 823 embedded fonts, not one starts
|
||||
// with a zero group; of the disguised glyph numbers, all of them do.
|
||||
func unicodeSequence(name string) (string, bool) {
|
||||
digits, ok := strings.CutPrefix(name, "uni")
|
||||
if !ok || len(digits) < 8 || len(digits)%4 != 0 {
|
||||
return "", false
|
||||
}
|
||||
var out strings.Builder
|
||||
for i := 0; i < len(digits); i += 4 {
|
||||
r, ok := parseHexName("uni"+digits[i:i+4], "uni", 4)
|
||||
if !ok {
|
||||
return "", false
|
||||
}
|
||||
if r == 0 {
|
||||
// A glyph number wearing a character's clothes.
|
||||
return "", false
|
||||
}
|
||||
out.WriteRune(r)
|
||||
}
|
||||
return out.String(), true
|
||||
}
|
||||
|
||||
// parseHexName reads the uniXXXX and uXXXX conventions.
|
||||
func parseHexName(name, prefix string, minDigits int) (rune, bool) {
|
||||
if len(name) <= len(prefix) || name[:len(prefix)] != prefix {
|
||||
return 0, false
|
||||
}
|
||||
digits := name[len(prefix):]
|
||||
if len(digits) < minDigits || len(digits) > 6 {
|
||||
return 0, false
|
||||
}
|
||||
var v rune
|
||||
for i := 0; i < len(digits); i++ {
|
||||
d := hexDigit(digits[i])
|
||||
if d < 0 {
|
||||
return 0, false
|
||||
}
|
||||
v = v<<4 | rune(d)
|
||||
}
|
||||
return v, true
|
||||
}
|
||||
|
||||
// hexDigit reads one hexadecimal digit, or reports -1.
|
||||
func hexDigit(c byte) int {
|
||||
switch {
|
||||
case c >= '0' && c <= '9':
|
||||
return int(c - '0')
|
||||
case c >= 'A' && c <= 'F':
|
||||
return int(c-'A') + 10
|
||||
case c >= 'a' && c <= 'f':
|
||||
return int(c-'a') + 10
|
||||
}
|
||||
return -1
|
||||
}
|
||||
+71
@@ -0,0 +1,71 @@
|
||||
package pdffont
|
||||
|
||||
// The glyph names above cover the Latin every English document is set in and
|
||||
// stop there. A document in Czech, Polish, Slovak, Hungarian, Turkish,
|
||||
// Romanian, Latvian, Lithuanian or Maltese is set in the block that follows,
|
||||
// and every one of its accented letters was being read as nothing at all —
|
||||
// which is worse than it sounds, because a dropped character leaves no mark:
|
||||
// "Příliš" comes back "Píliš" and looks like a word.
|
||||
//
|
||||
// Counted across the 118 833-file corpus: of 52.4 million glyph names, 483 382
|
||||
// could not be turned into text, and the largest nameable share of those was
|
||||
// this block — rcaron 650, zdotaccent 617, Ccaron 611, nacute 602, aogonek
|
||||
// 602, scedilla 597, gbreve 597, and so on for a hundred more.
|
||||
//
|
||||
// These are the names the Adobe Glyph List gives, and the characters are not
|
||||
// in doubt: unlike the subsetted fonts that call their glyphs g0 or dress a
|
||||
// glyph number up as uni00000048, a name like "rcaron" says exactly one thing.
|
||||
var latinExtendedRunes = map[string]rune{
|
||||
"Amacron": 'Ā', "amacron": 'ā', "Abreve": 'Ă', "abreve": 'ă',
|
||||
"Aogonek": 'Ą', "aogonek": 'ą', "Cacute": 'Ć', "cacute": 'ć',
|
||||
"Ccircumflex": 'Ĉ', "ccircumflex": 'ĉ', "Cdotaccent": 'Ċ', "cdotaccent": 'ċ',
|
||||
"Ccaron": 'Č', "ccaron": 'č', "Dcaron": 'Ď', "dcaron": 'ď',
|
||||
"Dcroat": 'Đ', "dcroat": 'đ', "Dslash": 'Đ', "dmacron": 'đ',
|
||||
"Emacron": 'Ē', "emacron": 'ē', "Ebreve": 'Ĕ', "ebreve": 'ĕ',
|
||||
"Edotaccent": 'Ė', "edotaccent": 'ė', "Eogonek": 'Ę', "eogonek": 'ę',
|
||||
"Ecaron": 'Ě', "ecaron": 'ě', "Gcircumflex": 'Ĝ', "gcircumflex": 'ĝ',
|
||||
"Gbreve": 'Ğ', "gbreve": 'ğ', "Gdotaccent": 'Ġ', "gdotaccent": 'ġ',
|
||||
"Gcommaaccent": 'Ģ', "gcommaaccent": 'ģ',
|
||||
"Hcircumflex": 'Ĥ', "hcircumflex": 'ĥ', "Hbar": 'Ħ', "hbar": 'ħ',
|
||||
"Itilde": 'Ĩ', "itilde": 'ĩ', "Imacron": 'Ī', "imacron": 'ī',
|
||||
"Ibreve": 'Ĭ', "ibreve": 'ĭ', "Iogonek": 'Į', "iogonek": 'į',
|
||||
"Idotaccent": 'İ', "dotlessi": 'ı', "IJ": 'IJ', "ij": 'ij',
|
||||
"Jcircumflex": 'Ĵ', "jcircumflex": 'ĵ',
|
||||
"Kcommaaccent": 'Ķ', "kcommaaccent": 'ķ', "kgreenlandic": 'ĸ',
|
||||
"Lacute": 'Ĺ', "lacute": 'ĺ', "Lcommaaccent": 'Ļ', "lcommaaccent": 'ļ',
|
||||
"Lcaron": 'Ľ', "lcaron": 'ľ', "Ldot": 'Ŀ', "ldot": 'ŀ',
|
||||
"Nacute": 'Ń', "nacute": 'ń', "Ncommaaccent": 'Ņ', "ncommaaccent": 'ņ',
|
||||
"Ncaron": 'Ň', "ncaron": 'ň', "napostrophe": 'ʼn', "Eng": 'Ŋ', "eng": 'ŋ',
|
||||
"Omacron": 'Ō', "omacron": 'ō', "Obreve": 'Ŏ', "obreve": 'ŏ',
|
||||
"Ohungarumlaut": 'Ő', "ohungarumlaut": 'ő',
|
||||
"Racute": 'Ŕ', "racute": 'ŕ', "Rcommaaccent": 'Ŗ', "rcommaaccent": 'ŗ',
|
||||
"Rcaron": 'Ř', "rcaron": 'ř', "Sacute": 'Ś', "sacute": 'ś',
|
||||
"Scircumflex": 'Ŝ', "scircumflex": 'ŝ', "Scedilla": 'Ş', "scedilla": 'ş',
|
||||
"Tcommaaccent": 'Ţ', "tcommaaccent": 'ţ', "Tcaron": 'Ť', "tcaron": 'ť',
|
||||
"Tbar": 'Ŧ', "tbar": 'ŧ', "Utilde": 'Ũ', "utilde": 'ũ',
|
||||
"Umacron": 'Ū', "umacron": 'ū', "Ubreve": 'Ŭ', "ubreve": 'ŭ',
|
||||
"Uring": 'Ů', "uring": 'ů', "Uhungarumlaut": 'Ű', "uhungarumlaut": 'ű',
|
||||
"Uogonek": 'Ų', "uogonek": 'ų', "Wcircumflex": 'Ŵ', "wcircumflex": 'ŵ',
|
||||
"Ycircumflex": 'Ŷ', "ycircumflex": 'ŷ',
|
||||
"Zacute": 'Ź', "zacute": 'ź', "Zdotaccent": 'Ż', "zdotaccent": 'ż',
|
||||
"longs": 'ſ',
|
||||
|
||||
// A handful outside that block that the same documents use: the Romanian
|
||||
// letters written with a comma below rather than a cedilla, the Turkish
|
||||
// and Azeri schwa, and the two the Adobe list names for Welsh and Maltese.
|
||||
"Scommaaccent": 'Ș', "scommaaccent": 'ș',
|
||||
"Afii10017": 'А', "afii10017": 'А',
|
||||
"Wgrave": 'Ẁ', "wgrave": 'ẁ', "Wacute": 'Ẃ', "wacute": 'ẃ',
|
||||
"Wdieresis": 'Ẅ', "wdieresis": 'ẅ',
|
||||
"Ygrave": 'Ỳ', "ygrave": 'ỳ',
|
||||
"Hcaron": 'Ȟ', "hcaron": 'ȟ',
|
||||
// The rest of what the corpus asks for and nothing could answer, each of
|
||||
// them a name the Adobe list gives and none of them in doubt: the cedilla
|
||||
// spellings of the Romanian letters, the capital sharp s, the script ell
|
||||
// mathematicians write lengths with, and the angle brackets.
|
||||
"Tcedilla": 'Ţ', "tcedilla": 'ţ', "Germandbls": 'ẞ',
|
||||
"lscript": 'ℓ', "openbullet": '◦',
|
||||
"angbracketleft": '〈', "angbracketright": '〉',
|
||||
"Scommabelow": 'Ș', "scommabelow": 'ș',
|
||||
"Tcommabelow": 'Ț', "tcommabelow": 'ț',
|
||||
}
|
||||
+245
@@ -0,0 +1,245 @@
|
||||
// Package pdffont is the PDF side of a font: not the outlines, which are the
|
||||
// font program's business, but everything the document itself says about how a
|
||||
// string of bytes is to be read. Which codes a string holds, how wide each one
|
||||
// is, what each is called, and what text each stands for.
|
||||
//
|
||||
// It is what a renderer needs before it can ask a font program for a glyph,
|
||||
// and all an extractor needs to read a page back as words. Both would
|
||||
// otherwise have to say the same things twice.
|
||||
package pdffont
|
||||
|
||||
import (
|
||||
"github.com/go-pdfkit/reader"
|
||||
)
|
||||
|
||||
// A Kind says how a font's codes are read.
|
||||
type Kind int
|
||||
|
||||
const (
|
||||
// Simple fonts take one byte a code and are addressed by glyph name.
|
||||
Simple Kind = iota
|
||||
// Composite fonts take a character identifier, usually two bytes wide.
|
||||
Composite
|
||||
// Type3 fonts draw each glyph by running a little content stream.
|
||||
Type3
|
||||
)
|
||||
|
||||
// A Font is a font as the document describes it.
|
||||
type Font struct {
|
||||
kind Kind
|
||||
dict reader.Dict
|
||||
descriptor reader.Dict
|
||||
doc *reader.Document
|
||||
|
||||
names map[int]string // what each code is called, for a simple font
|
||||
// chosen says which of those names the document chose itself, in a
|
||||
// Differences array, rather than inheriting from a base encoding.
|
||||
chosen map[int]bool
|
||||
widths map[int]float64 // in text space: a thousandth of the point size
|
||||
defaultW float64
|
||||
toUni map[int]string
|
||||
symbolic bool
|
||||
|
||||
// fallback is a last resort for codes the document itself says nothing
|
||||
// about: what the embedded font program calls them. A caller that can
|
||||
// read the program installs one.
|
||||
fallback func(code int) (string, bool)
|
||||
|
||||
// cidToGID maps a character identifier to a glyph number, for a composite
|
||||
// font whose descendant carries a map rather than using them as they are.
|
||||
cidToGID []byte
|
||||
|
||||
// The pieces only a Type 3 font has.
|
||||
fontMatrix [6]float64
|
||||
charProcs reader.Dict
|
||||
t3Res reader.Dict
|
||||
}
|
||||
|
||||
// Read takes what a font dictionary says. It never fails: a dictionary that
|
||||
// says nothing useful gives a font that reads its codes one byte at a time and
|
||||
// advances by half an em, which is what a reader has to do with one anyway.
|
||||
func Read(d *reader.Document, dict reader.Dict) *Font {
|
||||
f := &Font{
|
||||
doc: d, dict: dict, widths: map[int]float64{}, defaultW: 0.5,
|
||||
fontMatrix: [6]float64{0.001, 0, 0, 0.001, 0, 0},
|
||||
}
|
||||
switch sub, _ := reader.ToName(resolve(d, dict.Get("Subtype"))); sub {
|
||||
case "Type0":
|
||||
f.kind = Composite
|
||||
f.readComposite()
|
||||
case "Type3":
|
||||
f.kind = Type3
|
||||
f.readType3()
|
||||
default:
|
||||
f.kind = Simple
|
||||
f.readSimple()
|
||||
}
|
||||
f.toUni = f.readToUnicode()
|
||||
return f
|
||||
}
|
||||
|
||||
// Kind says how this font's codes are read.
|
||||
func (f *Font) Kind() Kind { return f.kind }
|
||||
|
||||
// Dict is the font dictionary itself, for whatever a caller needs that this
|
||||
// does not say.
|
||||
func (f *Font) Dict() reader.Dict { return f.dict }
|
||||
|
||||
// Descriptor is the font descriptor, which is where an embedded program lives.
|
||||
func (f *Font) Descriptor() reader.Dict { return f.descriptor }
|
||||
|
||||
// Symbolic reports a font that says it is addressed through its own character
|
||||
// map rather than through an encoding of names.
|
||||
func (f *Font) Symbolic() bool { return f.symbolic }
|
||||
|
||||
// FontMatrix is how a Type 3 font's glyph space relates to text space. For
|
||||
// every other kind it is a thousandth, which is what the format assumes.
|
||||
func (f *Font) FontMatrix() [6]float64 { return f.fontMatrix }
|
||||
|
||||
// CharProcs are a Type 3 font's glyph drawings, by name.
|
||||
func (f *Font) CharProcs() reader.Dict { return f.charProcs }
|
||||
|
||||
// Type3Resources are what a Type 3 font's glyph drawings draw with.
|
||||
func (f *Font) Type3Resources() reader.Dict { return f.t3Res }
|
||||
|
||||
// Codes cuts a string into the codes it holds: one byte at a time for a simple
|
||||
// font, two for a composite one, which is what every composite encoding seen in
|
||||
// the wild uses.
|
||||
func (f *Font) Codes(s []byte) []int {
|
||||
if f.kind != Composite {
|
||||
out := make([]int, len(s))
|
||||
for i, b := range s {
|
||||
out[i] = int(b)
|
||||
}
|
||||
return out
|
||||
}
|
||||
out := make([]int, 0, len(s)/2)
|
||||
for i := 0; i+1 < len(s); i += 2 {
|
||||
out = append(out, int(s[i])<<8|int(s[i+1]))
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// Width is how far the pen moves for one code, in text space — a multiple of
|
||||
// the point size, so a width of half means half an em.
|
||||
func (f *Font) Width(code int) float64 {
|
||||
if w, ok := f.widths[code]; ok {
|
||||
return w
|
||||
}
|
||||
return f.defaultW
|
||||
}
|
||||
|
||||
// HasWidth reports whether the document said how wide this code is, rather
|
||||
// than the width being the fallback [Font.Width] gives for one it did not.
|
||||
//
|
||||
// It matters for a font the document does not carry. One of the fourteen
|
||||
// standard faces is written with no widths at all — every reader is expected
|
||||
// to know Times and Helvetica and Courier by heart — so a reader drawing a
|
||||
// stand-in has to take the advances from the stand-in too. Half an em for
|
||||
// every letter reads as a typewriter, which is not what the page says.
|
||||
func (f *Font) HasWidth(code int) bool {
|
||||
_, ok := f.widths[code]
|
||||
return ok
|
||||
}
|
||||
|
||||
// GlyphName is what the document's encoding calls a code. ok is false for a
|
||||
// composite font, which names nothing, and for a code the encoding passes over.
|
||||
func (f *Font) GlyphName(code int) (string, bool) {
|
||||
name, ok := f.names[code]
|
||||
return name, ok
|
||||
}
|
||||
|
||||
// Text is what a code stands for as characters: what the font's ToUnicode map
|
||||
// says, or failing that what its glyph name says. ok is false when neither
|
||||
// does — a subsetted font naming its glyphs g17 and carrying no map cannot be
|
||||
// read back, and saying so is better than guessing.
|
||||
func (f *Font) Text(code int) (string, bool) {
|
||||
if s, ok := f.toUni[code]; ok && s != "" {
|
||||
return s, true
|
||||
}
|
||||
if name, ok := f.names[code]; ok && f.namedByTheDocument(code) {
|
||||
// Not one character: a name may say it is a ligature of several, and
|
||||
// a code that stands for two letters has to give back two.
|
||||
if text, ok := TextOfGlyphName(name); ok {
|
||||
return text, true
|
||||
}
|
||||
}
|
||||
if f.fallback != nil {
|
||||
if s, ok := f.fallback(code); ok && s != "" {
|
||||
return s, true
|
||||
}
|
||||
}
|
||||
return "", false
|
||||
}
|
||||
|
||||
// SetFallback installs a last resort for codes the document says nothing
|
||||
// about: what the embedded font program calls them.
|
||||
//
|
||||
// A font that says it is symbolic is addressed through its own character map,
|
||||
// and what that map says is the only honest answer for a code the document did
|
||||
// not name. Reading the program is another package's work, so a caller that
|
||||
// can do it hands the answer in here.
|
||||
func (f *Font) SetFallback(fn func(code int) (string, bool)) { f.fallback = fn }
|
||||
|
||||
// namedByTheDocument reports whether a code's name can be believed as a
|
||||
// statement about which character it is.
|
||||
//
|
||||
// A font that says it is symbolic is addressed through its own character map,
|
||||
// and the base encoding is then a guess. It is a bad one: a mathematical font
|
||||
// puts a capital gamma where the standard encoding puts an inverted
|
||||
// exclamation mark, and reading the name would give the wrong letter with
|
||||
// nothing to say so. A name the document chose itself, in a Differences
|
||||
// array, is a different matter — that one it meant.
|
||||
func (f *Font) namedByTheDocument(code int) bool {
|
||||
return !f.symbolic || f.chosen[code]
|
||||
}
|
||||
|
||||
// Chosen reports whether the document named this code itself rather than
|
||||
// inheriting the name from a base encoding.
|
||||
func (f *Font) Chosen(code int) bool { return f.chosen[code] }
|
||||
|
||||
// CIDToGID maps a character identifier to a glyph number for a composite font.
|
||||
// ok is false when the font's map does not reach that far.
|
||||
func (f *Font) CIDToGID(cid int) (int, bool) {
|
||||
if f.cidToGID == nil {
|
||||
return cid, true
|
||||
}
|
||||
i := cid * 2
|
||||
if i+1 >= len(f.cidToGID) {
|
||||
return 0, false
|
||||
}
|
||||
return int(f.cidToGID[i])<<8 | int(f.cidToGID[i+1]), true
|
||||
}
|
||||
|
||||
// Program is the embedded font program: which key it arrived under, and its
|
||||
// bytes. ok is false for a font whose program is not embedded, which is every
|
||||
// standard face and many others besides.
|
||||
func (f *Font) Program() (reader.Name, []byte, bool) {
|
||||
for _, key := range []reader.Name{"FontFile2", "FontFile3", "FontFile"} {
|
||||
stream, ok := reader.ToStream(resolve(f.doc, f.descriptor.Get(key)))
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
data, img, err := f.doc.DecodeStream(stream)
|
||||
if err != nil || img != "" {
|
||||
continue
|
||||
}
|
||||
return key, data, true
|
||||
}
|
||||
return "", nil, false
|
||||
}
|
||||
|
||||
// resolve follows an indirect reference. A document that opened cannot fail to
|
||||
// resolve one, so there is nothing to handle at every use.
|
||||
func resolve(d *reader.Document, o reader.Object) reader.Object {
|
||||
out, _ := d.Resolve(o)
|
||||
return out
|
||||
}
|
||||
|
||||
// floatOf reads a number, or gives a default.
|
||||
func floatOf(o reader.Object, def float64) float64 {
|
||||
if v, ok := reader.ToFloat(o); ok {
|
||||
return v
|
||||
}
|
||||
return def
|
||||
}
|
||||
+69
@@ -0,0 +1,69 @@
|
||||
package pdffont
|
||||
|
||||
// greekAndMathRunes are the names a mathematical document gives its glyphs.
|
||||
// They are not in the Latin encodings, so a page of equations addressed by
|
||||
// name — which is what every TeX document is — could not be read back without
|
||||
// them: the words came out and the symbols did not.
|
||||
//
|
||||
// The names are Adobe's, from the Symbol font and the glyph list every
|
||||
// producer follows. They are checked against the documents themselves rather
|
||||
// than taken on trust: a font that carries both a name for a code and a
|
||||
// ToUnicode entry for it states the same fact twice, and the corpus has tens
|
||||
// of thousands of such pairs to compare against.
|
||||
var greekAndMathRunes = map[string]rune{
|
||||
// The Greek alphabet, lower case and upper.
|
||||
"alpha": 'α', "beta": 'β', "gamma": 'γ', "delta": 'δ', "epsilon": 'ε',
|
||||
"zeta": 'ζ', "eta": 'η', "theta": 'θ', "iota": 'ι', "kappa": 'κ',
|
||||
"lambda": 'λ', "mu1": 'μ', "nu": 'ν', "xi": 'ξ', "omicron": 'ο',
|
||||
"pi": 'π', "rho": 'ρ', "sigma": 'σ', "sigma1": 'ς', "tau": 'τ',
|
||||
"upsilon": 'υ', "phi": 'φ', "chi": 'χ', "psi": 'ψ', "omega": 'ω',
|
||||
"theta1": 'ϑ', "phi1": 'ϕ', "epsilon1": 'ϵ', "rho1": 'ϱ', "omega1": 'ϖ',
|
||||
"kappa1": 'ϰ', "pi1": 'ϖ',
|
||||
// Adobe's list gives Delta and Omega the increment and ohm signs, which
|
||||
// Unicode says are the same characters as the Greek letters written
|
||||
// here — canonically equivalent, and these are the forms a person
|
||||
// searching the text would type.
|
||||
"Alpha": 'Α', "Beta": 'Β', "Gamma": 'Γ', "Delta": 'Δ', "Epsilon": 'Ε',
|
||||
"Zeta": 'Ζ', "Eta": 'Η', "Theta": 'Θ', "Iota": 'Ι', "Kappa": 'Κ',
|
||||
"Lambda": 'Λ', "Mu": 'Μ', "Nu": 'Ν', "Xi": 'Ξ', "Omicron": 'Ο',
|
||||
"Pi": 'Π', "Rho": 'Ρ', "Sigma": 'Σ', "Tau": 'Τ', "Upsilon": 'Υ',
|
||||
"Phi": 'Φ', "Chi": 'Χ', "Psi": 'Ψ', "Omega": 'Ω', "Upsilon1": 'ϒ',
|
||||
|
||||
// The signs an equation is built from.
|
||||
"minus": '−', "plus": '+', "equal": '=', "asteriskmath": '∗',
|
||||
"lessequal": '≤', "greaterequal": '≥', "notequal": '≠', "approxequal": '≈',
|
||||
"equivalence": '≡', "similar": '∼', "congruent": '≅', "proportional": '∝',
|
||||
"infinity": '∞', "partialdiff": '∂', "gradient": '∇', "integral": '∫',
|
||||
"summation": '∑', "product": '∏', "radical": '√', "angle": '∠',
|
||||
"perpendicular": '⊥', "therefore": '∴', "dotmath": '⋅', "bullet3": '∙',
|
||||
"element": '∈', "notelement": '∉', "universal": '∀', "existential": '∃',
|
||||
"emptyset": '∅', "intersection": '∩', "union": '∪',
|
||||
"propersubset": '⊂', "propersuperset": '⊃',
|
||||
"reflexsubset": '⊆', "reflexsuperset": '⊇', "notsubset": '⊄',
|
||||
"logicaland": '∧', "logicalor": '∨', "logicalnot1": '¬',
|
||||
"circleplus": '⊕', "circlemultiply": '⊗',
|
||||
"prime": '′', "second": '″', "minute": '′',
|
||||
|
||||
// Arrows, and the brackets a display equation is built out of.
|
||||
"arrowleft": '←', "arrowup": '↑', "arrowright": '→', "arrowdown": '↓',
|
||||
"arrowboth": '↔', "arrowupdn": '↕',
|
||||
"arrowdblleft": '⇐', "arrowdblup": '⇑', "arrowdblright": '⇒',
|
||||
"arrowdbldown": '⇓', "arrowdblboth": '⇔',
|
||||
"angleleft": '⟨', "angleright": '⟩',
|
||||
"bracketlefttp": '⎡', "bracketleftbt": '⎣', "bracketleftex": '⎢',
|
||||
"bracketrighttp": '⎤', "bracketrightbt": '⎦', "bracketrightex": '⎥',
|
||||
"parenlefttp": '⎛', "parenleftbt": '⎝', "parenleftex": '⎜',
|
||||
"parenrighttp": '⎞', "parenrightbt": '⎠', "parenrightex": '⎟',
|
||||
"bracelefttp": '⎧', "braceleftmid": '⎨', "braceleftbt": '⎩',
|
||||
"bracerighttp": '⎫', "bracerightmid": '⎬', "bracerightbt": '⎭',
|
||||
"braceex": '⎪',
|
||||
|
||||
// The ligatures and marks a book is set with.
|
||||
"ff": 'ff', "ffi": 'ffi', "ffl": 'ffl', "dotlessj": 'ȷ',
|
||||
"circumflexbig": 'ˆ', "tildebig": '˜',
|
||||
"suchthat": '∍', "carriagereturn": '↵', "aleph": 'ℵ',
|
||||
"Ifraktur": 'ℑ', "Rfraktur": 'ℜ', "weierstrass": '℘',
|
||||
"circlecopyrt": '©', "trademarksans": '™', "trademarkserif": '™',
|
||||
"degree1": '°', "florin1": 'ƒ',
|
||||
"lozenge": '◊', "spade": '♠', "club": '♣', "heart": '♥', "diamond": '♦',
|
||||
}
|
||||
+223
@@ -0,0 +1,223 @@
|
||||
package pdffont
|
||||
|
||||
import "github.com/go-pdfkit/reader"
|
||||
|
||||
// readSimple reads a font addressed one byte at a time: what each code is
|
||||
// called, and how wide it is.
|
||||
func (f *Font) readSimple() {
|
||||
f.descriptor, _ = f.doc.GetDict(f.dict, "FontDescriptor")
|
||||
f.symbolic = symbolicFlag(f.doc, f.descriptor)
|
||||
f.names = f.readEncoding()
|
||||
f.readSimpleWidths()
|
||||
}
|
||||
|
||||
// readType3 reads a font whose glyphs are little drawings.
|
||||
func (f *Font) readType3() {
|
||||
f.descriptor, _ = f.doc.GetDict(f.dict, "FontDescriptor")
|
||||
f.names = f.readEncoding()
|
||||
f.charProcs, _ = f.doc.GetDict(f.dict, "CharProcs")
|
||||
f.t3Res, _ = f.doc.GetDict(f.dict, "Resources")
|
||||
if m := floatArray(f.doc, f.dict.Get("FontMatrix")); len(m) >= 6 {
|
||||
copy(f.fontMatrix[:], m[:6])
|
||||
}
|
||||
f.readSimpleWidths()
|
||||
// A Type 3 font's widths are in its own glyph space, which the font
|
||||
// matrix maps onto text space.
|
||||
for code, w := range f.widths {
|
||||
f.widths[code] = w * 1000 * f.fontMatrix[0]
|
||||
}
|
||||
f.defaultW = 0
|
||||
}
|
||||
|
||||
// symbolicFlag reads whether a font says it is addressed through its own
|
||||
// character map rather than through an encoding of names.
|
||||
func symbolicFlag(d *reader.Document, descriptor reader.Dict) bool {
|
||||
flags, ok := reader.ToInt(resolve(d, descriptor.Get("Flags")))
|
||||
if !ok {
|
||||
return false
|
||||
}
|
||||
const symbolic, nonSymbolic = 1 << 2, 1 << 5
|
||||
return flags&symbolic != 0 && flags&nonSymbolic == 0
|
||||
}
|
||||
|
||||
// readEncoding works out what each code of a simple font is called: a base
|
||||
// table, and then whatever differences the font names on top of it.
|
||||
func (f *Font) readEncoding() map[int]string {
|
||||
base := StandardEncoding
|
||||
out := map[int]string{}
|
||||
f.chosen = map[int]bool{}
|
||||
enc := resolve(f.doc, f.dict.Get("Encoding"))
|
||||
if name, ok := reader.ToName(enc); ok {
|
||||
table := NamedEncoding(name, base)
|
||||
fillNames(out, table)
|
||||
// A named encoding is the document's own word on what its codes are,
|
||||
// symbolic font or not.
|
||||
for code, n := range table {
|
||||
if n != "" {
|
||||
f.chosen[code] = true
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
encDict, ok := reader.ToDict(enc)
|
||||
if !ok {
|
||||
fillNames(out, base)
|
||||
return out
|
||||
}
|
||||
if name, ok := reader.ToName(resolve(f.doc, encDict.Get("BaseEncoding"))); ok {
|
||||
base = NamedEncoding(name, base)
|
||||
}
|
||||
fillNames(out, base)
|
||||
f.applyDifferences(out, encDict)
|
||||
return out
|
||||
}
|
||||
|
||||
// NamedEncoding is the table a name stands for, or the fallback for a name
|
||||
// that is not one of them.
|
||||
func NamedEncoding(name reader.Name, fallback [256]string) [256]string {
|
||||
switch name {
|
||||
case "WinAnsiEncoding":
|
||||
return WinAnsiEncoding
|
||||
case "StandardEncoding", "MacRomanEncoding", "MacExpertEncoding":
|
||||
// Mac Roman differs from the standard encoding above code 127, where
|
||||
// almost nothing in a Latin document lives; using the standard table
|
||||
// is closer than using none.
|
||||
return StandardEncoding
|
||||
}
|
||||
return fallback
|
||||
}
|
||||
|
||||
// fillNames copies a table into the map the rest of this works from.
|
||||
func fillNames(out map[int]string, table [256]string) {
|
||||
for code, name := range table {
|
||||
if name != "" {
|
||||
out[code] = name
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// applyDifferences reads the array that names glyphs a code at a time: a
|
||||
// number, then the names of the codes from there on.
|
||||
func (f *Font) applyDifferences(out map[int]string, encDict reader.Dict) {
|
||||
arr, ok := reader.ToArray(resolve(f.doc, encDict.Get("Differences")))
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
code := 0
|
||||
for _, e := range arr {
|
||||
v := resolve(f.doc, e)
|
||||
if n, ok := reader.ToInt(v); ok {
|
||||
code = int(n)
|
||||
continue
|
||||
}
|
||||
if name, ok := reader.ToName(v); ok {
|
||||
if code >= 0 && code < 256 {
|
||||
out[code] = string(name)
|
||||
f.chosen[code] = true
|
||||
}
|
||||
code++
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// readSimpleWidths reads /Widths, which gives one width per code from
|
||||
// /FirstChar on.
|
||||
func (f *Font) readSimpleWidths() {
|
||||
first, ok := reader.ToInt(resolve(f.doc, f.dict.Get("FirstChar")))
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
arr, ok := reader.ToArray(resolve(f.doc, f.dict.Get("Widths")))
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
for i, e := range arr {
|
||||
if v, ok := reader.ToFloat(resolve(f.doc, e)); ok {
|
||||
f.widths[int(first)+i] = v / 1000
|
||||
}
|
||||
}
|
||||
if mw := floatOf(resolve(f.doc, f.descriptor.Get("MissingWidth")), -1); mw >= 0 {
|
||||
f.defaultW = mw / 1000
|
||||
}
|
||||
}
|
||||
|
||||
// readComposite reads a font addressed by character identifier: its descendant
|
||||
// holds the widths, the descriptor and the map from identifier to glyph.
|
||||
func (f *Font) readComposite() {
|
||||
f.defaultW = 1
|
||||
arr, ok := reader.ToArray(resolve(f.doc, f.dict.Get("DescendantFonts")))
|
||||
if !ok || len(arr) == 0 {
|
||||
return
|
||||
}
|
||||
kid, ok := reader.ToDict(resolve(f.doc, arr[0]))
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
if v, ok := reader.ToFloat(resolve(f.doc, kid.Get("DW"))); ok {
|
||||
f.defaultW = v / 1000
|
||||
}
|
||||
f.readCIDWidths(kid)
|
||||
f.descriptor, _ = f.doc.GetDict(kid, "FontDescriptor")
|
||||
f.symbolic = symbolicFlag(f.doc, f.descriptor)
|
||||
if stream, ok := reader.ToStream(resolve(f.doc, kid.Get("CIDToGIDMap"))); ok {
|
||||
if data, img, err := f.doc.DecodeStream(stream); err == nil && img == "" {
|
||||
f.cidToGID = data
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// readCIDWidths reads the /W array, which names widths one identifier at a
|
||||
// time or a run at a time.
|
||||
func (f *Font) readCIDWidths(kid reader.Dict) {
|
||||
arr, ok := reader.ToArray(resolve(f.doc, kid.Get("W")))
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
for i := 0; i < len(arr); {
|
||||
first, ok := reader.ToInt(resolve(f.doc, arr[i]))
|
||||
if !ok || i+1 >= len(arr) {
|
||||
return
|
||||
}
|
||||
next := resolve(f.doc, arr[i+1])
|
||||
if list, ok := reader.ToArray(next); ok {
|
||||
for k, e := range list {
|
||||
if v, ok := reader.ToFloat(resolve(f.doc, e)); ok {
|
||||
f.widths[int(first)+k] = v / 1000
|
||||
}
|
||||
}
|
||||
i += 2
|
||||
continue
|
||||
}
|
||||
last, ok := reader.ToInt(next)
|
||||
if !ok || i+2 >= len(arr) {
|
||||
return
|
||||
}
|
||||
w, ok := reader.ToFloat(resolve(f.doc, arr[i+2]))
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
if last >= first && last-first < 1<<20 {
|
||||
for c := first; c <= last; c++ {
|
||||
f.widths[int(c)] = w / 1000
|
||||
}
|
||||
}
|
||||
i += 3
|
||||
}
|
||||
}
|
||||
|
||||
// floatArray reads an array of numbers, or nothing.
|
||||
func floatArray(d *reader.Document, o reader.Object) []float64 {
|
||||
arr, ok := reader.ToArray(resolve(d, o))
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
out := make([]float64, 0, len(arr))
|
||||
for _, e := range arr {
|
||||
v, ok := reader.ToFloat(resolve(d, e))
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
out = append(out, v)
|
||||
}
|
||||
return out
|
||||
}
|
||||
+252
@@ -0,0 +1,252 @@
|
||||
package pdffont
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"unicode/utf16"
|
||||
|
||||
"github.com/go-pdfkit/reader"
|
||||
)
|
||||
|
||||
// readToUnicode reads the font's /ToUnicode map, which is the one thing in a
|
||||
// PDF that says what its text actually says. Without it a page can be drawn
|
||||
// but not read: a subsetted font naming its glyphs by number carries no other
|
||||
// clue.
|
||||
func (f *Font) readToUnicode() map[int]string {
|
||||
stream, ok := reader.ToStream(resolve(f.doc, f.dict.Get("ToUnicode")))
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
data, img, err := f.doc.DecodeStream(stream)
|
||||
if err != nil || img != "" {
|
||||
return nil
|
||||
}
|
||||
return ReadToUnicode(data)
|
||||
}
|
||||
|
||||
// ReadToUnicode decodes a ToUnicode CMap: a little PostScript program whose
|
||||
// only meaningful parts are the bfchar and bfrange blocks saying which
|
||||
// characters each code stands for.
|
||||
func ReadToUnicode(data []byte) map[int]string {
|
||||
out := map[int]string{}
|
||||
toks := cmapTokens(data)
|
||||
for i := 0; i < len(toks); i++ {
|
||||
switch toks[i].word {
|
||||
case "beginbfchar":
|
||||
i = readBFChar(toks, i+1, out)
|
||||
case "beginbfrange":
|
||||
i = readBFRange(toks, i+1, out)
|
||||
}
|
||||
}
|
||||
if len(out) == 0 {
|
||||
return nil
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// maxToUnicodeEntries is how many codes one map may name.
|
||||
//
|
||||
// A bfrange names a run of codes in about twenty bytes, and only the width of
|
||||
// a single run was bounded — not how many runs a map could hold. So the size
|
||||
// of the answer had nothing to do with the size of the question: 585 bytes
|
||||
// produced 655 360 entries in 2.3 seconds, and 10 655 bytes produced
|
||||
// 13 107 200 entries and a gigabyte. Every font on every page is read this
|
||||
// way, so that was a gigabyte per font.
|
||||
//
|
||||
// The bound is not a guess. Across 5 338 /ToUnicode maps taken out of real
|
||||
// documents — arXiv figures, government forms, and mozilla's pdf.js corpus —
|
||||
// the median names 13 codes, the 99th percentile names 538, and the largest
|
||||
// names exactly 65 536: one whole two-byte code space, which is as many codes
|
||||
// as a font of that shape can have. This allows four times that, so a
|
||||
// document has to be malformed or hostile to reach it, and a map that does is
|
||||
// cut off there rather than being allowed to ask for everything.
|
||||
const maxToUnicodeEntries = 1 << 18
|
||||
|
||||
// A cmapToken is one piece of such a program: a hexadecimal string, an array
|
||||
// bracket, or a bare word.
|
||||
type cmapToken struct {
|
||||
word string
|
||||
hex []byte
|
||||
isHex bool
|
||||
}
|
||||
|
||||
// cmapTokens cuts the program up. Only three shapes matter — a hexadecimal
|
||||
// string, a bracket, and a word — and everything else can be a word.
|
||||
func cmapTokens(b []byte) []cmapToken {
|
||||
var out []cmapToken
|
||||
for i := 0; i < len(b); {
|
||||
c := b[i]
|
||||
switch {
|
||||
case c == '%':
|
||||
for i < len(b) && b[i] != '\n' && b[i] != '\r' {
|
||||
i++
|
||||
}
|
||||
case c == '<':
|
||||
j := i + 1
|
||||
for j < len(b) && b[j] != '>' {
|
||||
j++
|
||||
}
|
||||
out = append(out, cmapToken{hex: unhexBytes(b[i+1 : min(j, len(b))]), isHex: true})
|
||||
i = min(j+1, len(b))
|
||||
case c == '[' || c == ']':
|
||||
out = append(out, cmapToken{word: string(c)})
|
||||
i++
|
||||
case c <= ' ':
|
||||
i++
|
||||
default:
|
||||
j := i
|
||||
for j < len(b) && b[j] > ' ' && b[j] != '<' && b[j] != '[' && b[j] != ']' && b[j] != '%' {
|
||||
j++
|
||||
}
|
||||
out = append(out, cmapToken{word: string(b[i:j])})
|
||||
i = j
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// readBFChar reads pairs of "code, what it stands for" up to the block's end.
|
||||
func readBFChar(toks []cmapToken, i int, out map[int]string) int {
|
||||
for i+1 < len(toks) {
|
||||
if toks[i].word == "endbfchar" {
|
||||
return i
|
||||
}
|
||||
if len(out) >= maxToUnicodeEntries {
|
||||
return i
|
||||
}
|
||||
if !toks[i].isHex || !toks[i+1].isHex {
|
||||
return i
|
||||
}
|
||||
out[codeOf(toks[i].hex)] = textOf(toks[i+1].hex)
|
||||
i += 2
|
||||
}
|
||||
return i
|
||||
}
|
||||
|
||||
// readBFRange reads runs of codes: either a first character the run counts on
|
||||
// from, or a list naming each one.
|
||||
func readBFRange(toks []cmapToken, i int, out map[int]string) int {
|
||||
for i+2 < len(toks) {
|
||||
if toks[i].word == "endbfrange" {
|
||||
return i
|
||||
}
|
||||
if !toks[i].isHex || !toks[i+1].isHex {
|
||||
return i
|
||||
}
|
||||
lo, hi := codeOf(toks[i].hex), codeOf(toks[i+1].hex)
|
||||
if hi < lo || hi-lo > 1<<16 {
|
||||
return i
|
||||
}
|
||||
if len(out) >= maxToUnicodeEntries {
|
||||
return i
|
||||
}
|
||||
switch {
|
||||
case toks[i+2].isHex:
|
||||
base := utf16Runes(toks[i+2].hex)
|
||||
for c := lo; c <= hi && len(out) < maxToUnicodeEntries; c++ {
|
||||
out[c] = countOn(base, c-lo)
|
||||
}
|
||||
i += 3
|
||||
case toks[i+2].word == "[":
|
||||
j := i + 3
|
||||
for c := lo; c <= hi && j < len(toks) && toks[j].word != "]"; c++ {
|
||||
if toks[j].isHex && len(out) < maxToUnicodeEntries {
|
||||
out[c] = textOf(toks[j].hex)
|
||||
}
|
||||
j++
|
||||
}
|
||||
for j < len(toks) && toks[j].word != "]" {
|
||||
j++
|
||||
}
|
||||
i = j + 1
|
||||
default:
|
||||
return i
|
||||
}
|
||||
}
|
||||
return i
|
||||
}
|
||||
|
||||
// codeOf reads a code, which is one or two bytes written as hexadecimal.
|
||||
func codeOf(b []byte) int {
|
||||
v := 0
|
||||
for _, c := range b {
|
||||
v = v<<8 | int(c)
|
||||
}
|
||||
return v
|
||||
}
|
||||
|
||||
// textOf reads what a code stands for: characters written as sixteen-bit
|
||||
// units, the way the format carries text.
|
||||
func textOf(b []byte) string {
|
||||
return string(utf16Runes(b))
|
||||
}
|
||||
|
||||
// utf16Runes decodes the sixteen-bit units a ToUnicode value is written in.
|
||||
func utf16Runes(b []byte) []rune {
|
||||
units := make([]uint16, 0, len(b)/2)
|
||||
for i := 0; i+1 < len(b); i += 2 {
|
||||
units = append(units, uint16(b[i])<<8|uint16(b[i+1]))
|
||||
}
|
||||
return utf16.Decode(units)
|
||||
}
|
||||
|
||||
// countOn is what the n'th code of a range stands for, counting on from the
|
||||
// first: the last character moves and the rest stay, which is what the
|
||||
// specification says a range means.
|
||||
func countOn(base []rune, n int) string {
|
||||
if len(base) == 0 {
|
||||
return ""
|
||||
}
|
||||
out := append([]rune{}, base...)
|
||||
out[len(out)-1] += rune(n)
|
||||
return string(out)
|
||||
}
|
||||
|
||||
// unhexBytes reads a hexadecimal string, ignoring whatever it was laid out
|
||||
// with. An odd digit at the end is taken as the high half of a byte, which is
|
||||
// what the format says.
|
||||
func unhexBytes(b []byte) []byte {
|
||||
var digits []byte
|
||||
for _, c := range b {
|
||||
if isHexDigit(c) {
|
||||
digits = append(digits, hexVal(c))
|
||||
}
|
||||
}
|
||||
if len(digits)%2 == 1 {
|
||||
digits = append(digits, 0)
|
||||
}
|
||||
out := make([]byte, 0, len(digits)/2)
|
||||
for i := 0; i+1 < len(digits); i += 2 {
|
||||
out = append(out, digits[i]<<4|digits[i+1])
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func isHexDigit(c byte) bool {
|
||||
return c >= '0' && c <= '9' || c >= 'a' && c <= 'f' || c >= 'A' && c <= 'F'
|
||||
}
|
||||
|
||||
func hexVal(c byte) byte {
|
||||
switch {
|
||||
case c <= '9':
|
||||
return c - '0'
|
||||
case c <= 'F':
|
||||
return c - 'A' + 10
|
||||
default:
|
||||
return c - 'a' + 10
|
||||
}
|
||||
}
|
||||
|
||||
// min is the smaller of two numbers.
|
||||
func min(a, b int) int {
|
||||
if a < b {
|
||||
return a
|
||||
}
|
||||
return b
|
||||
}
|
||||
|
||||
// TrimText tidies what a map gives back: the format allows a value to carry
|
||||
// the replacement character or nothing at all, and neither is text.
|
||||
func TrimText(s string) string {
|
||||
s = strings.TrimFunc(s, func(r rune) bool { return r == 0 || r == 0xFFFD })
|
||||
return s
|
||||
}
|
||||
Reference in New Issue
Block a user