Swap: chunked checkpoints for large buffers; scaled thresholds; single replay

- A checkpoint larger than one record (16 MiB) is written as CHKPT_BEGIN
  [size u64, CRC-32], CHKPT_DATA chunks and CHKPT_END. Replay assembles it
  and applies it only when complete with the size and CRC matching; an
  unfinished one is a torn tail (valid_bytes stops before it). Buffers
  over 16 MiB could not be checkpointed before: a lost record left their
  journal behind for good, and they were never compacted. Records are
  written as segments straight from the snapshot. Header version 2;
  version 1 journals are still read.
- Checkpoint thresholds scale with the buffer size (edits of half its
  size; the timer only after a sixteenth; compaction at twice its size;
  gap retries spaced by size), and a checkpoint that would compact is
  written straight into the compacted journal instead of being appended
  and then copied.
- The recovery prompt keeps the content it replayed for its preview and
  installs it on 'y', instead of replaying the journal a second time.
- A base file over 64 MiB gets its CRC from the writer thread before it
  creates the journal (it had none, so touching such a file made its
  journal unusable for recovery).
- kte_tests takes optional name substrings to run a subset of tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
This commit is contained in:
Claude
2026-09-24 09:16:04 -07:00
committed by Kyle Isom
parent fc15a58a2b
commit a33c1759e7
9 changed files with 491 additions and 96 deletions
+18 -3
View File
@@ -526,6 +526,24 @@ before the fix (unless noted).
through matches in the regex replace prompt; Up/Down in a regex prompt
use the regex.
**Follow-up: journals for large buffers**
- Buffers over 16 MiB are checkpointed as CHKPT_BEGIN (size, CRC-32),
data chunks and CHKPT_END; replay applies the checkpoint only when it
is complete and its CRC matches, and treats an unfinished one as a
torn tail. Such buffers could not be checkpointed at all: a lost
record left the journal permanently behind and it was never
compacted. The journal header is now version 2 (version 1 journals
are still read; older kte builds will not read version 2).
- Checkpoint thresholds scale with the buffer: after edits amounting to
half its size, on the timer only after a sixteenth, compaction at
twice its size, gap retries spaced by size. A checkpoint that would
compact is written straight into the compacted journal.
- The recovery prompt keeps the content it replayed for the preview;
answering y installs it instead of replaying the journal again.
- The CRC of a base file over 64 MiB is computed by the journal writer
thread (it was skipped), so a touched but unchanged large file no
longer makes its journal unusable for recovery.
**Known limitations (not fixed)**
- Catastrophic regex backtracking (e.g. `(a*)*b`) can still take very
long; std::regex has no time limit.
@@ -539,6 +557,3 @@ before the fix (unless noted).
session that is locked out is told at its first edit instead.
- A regex that matches rarely or not at all still scans the whole
buffer with std::regex (1.6 s on 105 MB).
- A journal whose buffer exceeds 16 MiB cannot be checkpointed; after a
lost record such a journal stays incomplete until the file is saved
(reported to the user).