pdftext: extract PDF text in a re-exec'd child (sift pdftext) with a deadline

The parser is pure Go but a goroutine cannot be killed, so extraction runs in
a child process: the binary re-executed as `sift pdftext`, reading the PDF on
stdin and writing JSON Lines on stdout, killed at a deadline by the parent.

pdftext.go holds the types, the five errors, Extract (reader.Open with
ErrWrongPassword -> ErrEncrypted, per-page recover, trailer /Info /Title, and
rule-3 UTF-16BE/Latin-1 text decode) and ChildMain (soft 768 MiB memory limit,
-no env read, no network, exit codes 0/3/4/1). outline.go walks /Outlines ->
/First -> /Next with /First children at level+1, resolving /Dest and /GoTo /A
dests to 1-based pages, bounded at 10,000 items and never visiting an object
twice. runner.go is the Runner: env-isolated child (nil env is an empty
environment), 60 s deadline, 64 MiB output cap, 200 MiB stdin cap, stderr
capped at 4 KiB, exit-code to error mapping, and Doc parsing with ErrNoText.
cmd/sift/main.go adds the unlisted `pdftext` case. tools.go is removed; the
go-pdfkit modules stay vendored.

Co-Authored-By: ornith-1.5-35b-a3b <ornith-1.5-35b-a3b@llama.cpp.invalid>
This commit is contained in:
2026-09-25 22:58:05 -07:00
co-authored by ornith-1.5-35b-a3b
parent 13365a2daa
commit f57aaef59e
11 changed files with 1112 additions and 10 deletions
+81
View File
@@ -0,0 +1,81 @@
%PDF-1.7
%¿÷¢þ
1 0 obj
<< /Extensions << /ADBE << /BaseVersion /1.7 /ExtensionLevel 8 >> >> /Outlines 3 0 R /PageMode /UseOutlines /Pages 4 0 R /Type /Catalog >>
endobj
2 0 obj
<< /Producer <43b5e97c0a7de15bc0f4dbcc052b5f80f21e0e3bede0eda33137092879ea433d> /Title <29f3ced7effab532ee871ab3ebc38a6869b6b7379f2fa12e139f3e2a1d1ee664> >>
endobj
3 0 obj
<< /Count 3 /First 5 0 R /Last 6 0 R /Type /Outlines >>
endobj
4 0 obj
<< /Count 3 /Kids [ 7 0 R 8 0 R 9 0 R ] /Type /Pages >>
endobj
5 0 obj
<< /Dest [ 7 0 R /XYZ null null null ] /Next 10 0 R /Parent 3 0 R /Title <d2951c68344226cca87b8f66af448a05c4ba3c4baf7c7863293f2328ce407d09> >>
endobj
6 0 obj
<< /Dest [ 9 0 R /XYZ null null null ] /Parent 3 0 R /Prev 10 0 R /Title <13b9a70c4f34e06e182eb823c2cb79a1f82c91f6c69ee89bf2093cd61d451407> >>
endobj
7 0 obj
<< /Contents 11 0 R /MediaBox [ 0 0 612 792 ] /Parent 4 0 R /Resources << /Font << /F1 12 0 R >> /XObject << /Im1 13 0 R >> >> /Type /Page >>
endobj
8 0 obj
<< /Contents 14 0 R /MediaBox [ 0 0 612 792 ] /Parent 4 0 R /Resources << /Font << /F1 12 0 R >> /XObject << /Im1 13 0 R >> >> /Type /Page >>
endobj
9 0 obj
<< /Contents 15 0 R /MediaBox [ 0 0 612 792 ] /Parent 4 0 R /Resources << /Font << /F1 12 0 R >> /XObject << /Im1 13 0 R >> >> /Type /Page >>
endobj
10 0 obj
<< /Dest [ 8 0 R /XYZ null null null ] /Next 6 0 R /Parent 3 0 R /Prev 5 0 R /Title <b734ddb0e84e981224c172818d645644e95ae16c94cf45d02ef870d181f97422> >>
endobj
11 0 obj
<< /Length 128 /Filter /FlateDecode >>
stream
jl<ħ|Ýkj>Kþ"æ"-‚'agcYÓ“¡Áù ‹ä¾Îç„Â^›™h9Àu¢ò4DY­>€ä‡ÝhSkvJxŠý␍÷À'‡ÿ­ù/טò£ Ð#¼Æo=ÖÛ:³1áÇ¢hÇôªÀè<Â|Hü“J‹xYz4�ÿÄendstream
endobj
12 0 obj
<< /BaseFont /Helvetica /Encoding /WinAnsiEncoding /Subtype /Type1 /Type /Font >>
endobj
13 0 obj
<< /BitsPerComponent 8 /ColorSpace /DeviceGray /Height 2 /Subtype /Image /Type /XObject /Width 2 /Length 32 /Filter /FlateDecode >>
stream
Ë™"€_œ¼Ë57Ó”Ì60Ë%œ1´V\Ó$mG&0FåMendstream
endobj
14 0 obj
<< /Length 112 /Filter /FlateDecode >>
stream
,#c¾}jè¹Ájw�)æ�„›d7Éwp¨íi»gvÍñÎâH»6oA•ï´vD}–Ð9·G`1wÆŽ×XŒÝ‰ž ÄrÜ›Vf:I¡)x9ÅéÌß“>��WãxŽÄ‡¡ãФÔ«âÊ‘dendstream
endobj
15 0 obj
<< /Length 160 /Filter /FlateDecode >>
stream
yx÷â°í%"(hß]�T~BùÞ]+$ík±Mû†e.ŒbÿPת®0ž;äòãæë%½Kû¾¤z‹C”¤²!̘£)(žœÛôŒáéÄ;/¬@yèºÚh¥Á qR|Ôcݹ²Ã((¼Õ{úc¿AÆeýÏΧDíÚ¼)‡(íÄ! · #5u·ké#¹âW’<8pŽ÷I1XQ"¥G endstream
endobj
16 0 obj
<< /CF << /StdCF << /AuthEvent /DocOpen /CFM /AESV3 /Length 32 >> >> /Filter /Standard /Length 256 /O <2a0820bbf93fff95a410525ae20d4abca153af1e3464706a4f83e49c8caa92e62e160b6871c75b9e47a6cd6dd15fe6ab> /OE <210b63a9a2305637af02dc6e89409285f61f82adaafa125e950042528164e2a3> /P -4 /Perms <23688a77e80aeb8b536511ce497c729e> /R 6 /StmF /StdCF /StrF /StdCF /U <98531c24f9d0bbabd16cc589be7618a59de91065126679a37b26b5d684cbcbc76d34bf68dc61eafe1681b4d5a21826d1> /UE <75ac70233a6a28bcd012febedaf3cccf6c25af218f3318a9fa9bc0f94a0e1b53> /V 5 >>
endobj
xref
0 17
0000000000 65535 f
0000000015 00000 n
0000000169 00000 n
0000000341 00000 n
0000000412 00000 n
0000000483 00000 n
0000000641 00000 n
0000000799 00000 n
0000000956 00000 n
0000001113 00000 n
0000001270 00000 n
0000001440 00000 n
0000001640 00000 n
0000001738 00000 n
0000001935 00000 n
0000002119 00000 n
0000002351 00000 n
trailer << /Info 2 0 R /Root 1 0 R /Size 17 /ID [<39c444018179222865caf844f4fb7254><39c444018179222865caf844f4fb7254>] /Encrypt 16 0 R >>
startxref
2899
%%EOF