pdftext: extract PDF text in a re-exec'd child (sift pdftext) with a deadline
The parser is pure Go but a goroutine cannot be killed, so extraction runs in a child process: the binary re-executed as `sift pdftext`, reading the PDF on stdin and writing JSON Lines on stdout, killed at a deadline by the parent. pdftext.go holds the types, the five errors, Extract (reader.Open with ErrWrongPassword -> ErrEncrypted, per-page recover, trailer /Info /Title, and rule-3 UTF-16BE/Latin-1 text decode) and ChildMain (soft 768 MiB memory limit, -no env read, no network, exit codes 0/3/4/1). outline.go walks /Outlines -> /First -> /Next with /First children at level+1, resolving /Dest and /GoTo /A dests to 1-based pages, bounded at 10,000 items and never visiting an object twice. runner.go is the Runner: env-isolated child (nil env is an empty environment), 60 s deadline, 64 MiB output cap, 200 MiB stdin cap, stderr capped at 4 KiB, exit-code to error mapping, and Doc parsing with ErrNoText. cmd/sift/main.go adds the unlisted `pdftext` case. tools.go is removed; the go-pdfkit modules stay vendored. Co-Authored-By: ornith-1.5-35b-a3b <ornith-1.5-35b-a3b@llama.cpp.invalid>
This commit is contained in:
Vendored
+14
@@ -0,0 +1,14 @@
|
||||
# PDF fixtures
|
||||
|
||||
Only the two encrypted files are binary; every other PDF the tests use is built at test time by
|
||||
`internal/pdftest`.
|
||||
|
||||
- `encrypted-user.pdf`: the `pdftest` three-page document, AES-256, user password `userpw`,
|
||||
owner password `ownerpw`. Made with
|
||||
`qpdf --encrypt userpw ownerpw 256 -- three.pdf encrypted-user.pdf`.
|
||||
- `encrypted-owner.pdf`: the same document, AES-256 with an empty user password
|
||||
(opens without one), owner password `ownerpw`:
|
||||
`qpdf --encrypt "" ownerpw 256 -- three.pdf encrypted-owner.pdf`.
|
||||
|
||||
Both contain the phrases "Chapter one begins here." (page 1) and
|
||||
"The quick brown fox is on page two." (page 2).
|
||||
BIN
Binary file not shown.
+81
@@ -0,0 +1,81 @@
|
||||
%PDF-1.7
|
||||
%¿÷¢þ
|
||||
1 0 obj
|
||||
<< /Extensions << /ADBE << /BaseVersion /1.7 /ExtensionLevel 8 >> >> /Outlines 3 0 R /PageMode /UseOutlines /Pages 4 0 R /Type /Catalog >>
|
||||
endobj
|
||||
2 0 obj
|
||||
<< /Producer <43b5e97c0a7de15bc0f4dbcc052b5f80f21e0e3bede0eda33137092879ea433d> /Title <29f3ced7effab532ee871ab3ebc38a6869b6b7379f2fa12e139f3e2a1d1ee664> >>
|
||||
endobj
|
||||
3 0 obj
|
||||
<< /Count 3 /First 5 0 R /Last 6 0 R /Type /Outlines >>
|
||||
endobj
|
||||
4 0 obj
|
||||
<< /Count 3 /Kids [ 7 0 R 8 0 R 9 0 R ] /Type /Pages >>
|
||||
endobj
|
||||
5 0 obj
|
||||
<< /Dest [ 7 0 R /XYZ null null null ] /Next 10 0 R /Parent 3 0 R /Title <d2951c68344226cca87b8f66af448a05c4ba3c4baf7c7863293f2328ce407d09> >>
|
||||
endobj
|
||||
6 0 obj
|
||||
<< /Dest [ 9 0 R /XYZ null null null ] /Parent 3 0 R /Prev 10 0 R /Title <13b9a70c4f34e06e182eb823c2cb79a1f82c91f6c69ee89bf2093cd61d451407> >>
|
||||
endobj
|
||||
7 0 obj
|
||||
<< /Contents 11 0 R /MediaBox [ 0 0 612 792 ] /Parent 4 0 R /Resources << /Font << /F1 12 0 R >> /XObject << /Im1 13 0 R >> >> /Type /Page >>
|
||||
endobj
|
||||
8 0 obj
|
||||
<< /Contents 14 0 R /MediaBox [ 0 0 612 792 ] /Parent 4 0 R /Resources << /Font << /F1 12 0 R >> /XObject << /Im1 13 0 R >> >> /Type /Page >>
|
||||
endobj
|
||||
9 0 obj
|
||||
<< /Contents 15 0 R /MediaBox [ 0 0 612 792 ] /Parent 4 0 R /Resources << /Font << /F1 12 0 R >> /XObject << /Im1 13 0 R >> >> /Type /Page >>
|
||||
endobj
|
||||
10 0 obj
|
||||
<< /Dest [ 8 0 R /XYZ null null null ] /Next 6 0 R /Parent 3 0 R /Prev 5 0 R /Title <b734ddb0e84e981224c172818d645644e95ae16c94cf45d02ef870d181f97422> >>
|
||||
endobj
|
||||
11 0 obj
|
||||
<< /Length 128 /Filter /FlateDecode >>
|
||||
stream
|
||||
jl<ħ|Ýkj>Kþ"æ"-‚'agcYÓ“¡Áù ‹ä¾Îç„Â^›™h9Àu¢ò4DY>€ä‡ÝhSkvJxŠý␍÷À'‡ÿù/טò£Ð#¼Æo=ÖÛ:³1áÇ¢hÇôªÀè<Â|Hü“J‹xYz4�ÿÄendstream
|
||||
endobj
|
||||
12 0 obj
|
||||
<< /BaseFont /Helvetica /Encoding /WinAnsiEncoding /Subtype /Type1 /Type /Font >>
|
||||
endobj
|
||||
13 0 obj
|
||||
<< /BitsPerComponent 8 /ColorSpace /DeviceGray /Height 2 /Subtype /Image /Type /XObject /Width 2 /Length 32 /Filter /FlateDecode >>
|
||||
stream
|
||||
Ë™"€_œ¼Ë57Ó”Ì60Ë%œ1´V\Ó$mG&0FåMendstream
|
||||
endobj
|
||||
14 0 obj
|
||||
<< /Length 112 /Filter /FlateDecode >>
|
||||
stream
|
||||
,#c¾}jè¹Ájw�)æ�„›d7Éwp¨íi»gvÍñÎâH»6oA•ï´vD}–Ð9·G`1wÆŽ×XŒÝ‰žÄrÜ›Vf:I¡)x9ÅéÌß“>��WãxŽÄ‡¡ãŠ¤Ô«âÊ‘dendstream
|
||||
endobj
|
||||
15 0 obj
|
||||
<< /Length 160 /Filter /FlateDecode >>
|
||||
stream
|
||||
yx÷â°í%"(hß]�T~BùÞ]+$ík±Mû†e.ŒbÿPת®0ž;äòãæë%½Kû¾¤z‹C”¤²!̘£)(žœÛôŒáéÄ;/¬@yèºÚh¥Á qR|Ôcݹ²Ã((¼Õ{úc¿AÆeýÏΧDíÚ¼)‡(íÄ! ·#5u·ké#¹âW’<8pŽ÷I1XQ"¥Gendstream
|
||||
endobj
|
||||
16 0 obj
|
||||
<< /CF << /StdCF << /AuthEvent /DocOpen /CFM /AESV3 /Length 32 >> >> /Filter /Standard /Length 256 /O <2a0820bbf93fff95a410525ae20d4abca153af1e3464706a4f83e49c8caa92e62e160b6871c75b9e47a6cd6dd15fe6ab> /OE <210b63a9a2305637af02dc6e89409285f61f82adaafa125e950042528164e2a3> /P -4 /Perms <23688a77e80aeb8b536511ce497c729e> /R 6 /StmF /StdCF /StrF /StdCF /U <98531c24f9d0bbabd16cc589be7618a59de91065126679a37b26b5d684cbcbc76d34bf68dc61eafe1681b4d5a21826d1> /UE <75ac70233a6a28bcd012febedaf3cccf6c25af218f3318a9fa9bc0f94a0e1b53> /V 5 >>
|
||||
endobj
|
||||
xref
|
||||
0 17
|
||||
0000000000 65535 f
|
||||
0000000015 00000 n
|
||||
0000000169 00000 n
|
||||
0000000341 00000 n
|
||||
0000000412 00000 n
|
||||
0000000483 00000 n
|
||||
0000000641 00000 n
|
||||
0000000799 00000 n
|
||||
0000000956 00000 n
|
||||
0000001113 00000 n
|
||||
0000001270 00000 n
|
||||
0000001440 00000 n
|
||||
0000001640 00000 n
|
||||
0000001738 00000 n
|
||||
0000001935 00000 n
|
||||
0000002119 00000 n
|
||||
0000002351 00000 n
|
||||
trailer << /Info 2 0 R /Root 1 0 R /Size 17 /ID [<39c444018179222865caf844f4fb7254><39c444018179222865caf844f4fb7254>] /Encrypt 16 0 R >>
|
||||
startxref
|
||||
2899
|
||||
%%EOF
|
||||
Reference in New Issue
Block a user