kte/docs/plans/swap-files.md

Swap files for kte — design plan

Goals

• Preserve user work across crashes, power failures, and OS kills.
• Keep the editor responsive; avoid blocking the UI on disk I/O.
• Bound recovery time and swap size.
• Favor simple, robust primitives that work well on POSIX and macOS; keep Windows feasibility in mind.

Model overview

Per open buffer, maintain a sidecar swap journal next to the file:

• Path: .<basename>.kte.swp in the same directory as the file (for unnamed/unsaved buffers, use a per‑session temp dir like $TMPDIR/kte/ with a random UUID).
• Format: append‑only journal of editing operations with periodic checkpoints.
• Crash safety: only append, fsync as per policy; checkpoint via write‑to‑temp + fsync + atomic rename.

File format (v1)

Header (fixed 64 bytes):

• Magic: KTE_SWP\0 (8 bytes)
• Version: 1 (u32)
• Flags: bitset (u32) — e.g., compression, checksums, endian.
• Created time (u64)
• Host info hash (u64) — optional, for telemetry/debug.
• File identity: hash of canonical path (u64) and original file size+mtime (u64+u64) at start.
• Reserved/padding.

Records (stream after header):

• Each record: [type u8][len u24][payload][crc32 u32]
• Types:
  • CHKPT — full snapshot checkpoint of entire buffer content and minimal metadata (cursor pos, filetype). Payload optionally compressed. Written occasionally to cap replay time.
  • INS — insert at (row, col) text bytes (text may contain newlines). Encoded with varints.
  • DEL — delete length at (row, col). If spanning lines, semantics defined as in Buffer::delete_text.
  • SPLIT, JOIN — explicit structural ops (optional; can be expressed via INS/DEL).
  • META — update metadata (e.g., filetype, encoding hints).

Durability policy

Configurable knobs (sane defaults in parentheses):

• Time‑based flush: group edits and flush every 150–300 ms (200 ms).
• Operation count flush: after N ops (200).
• Idle flush: on 500 ms idle lull, flush immediately.
• Checkpoint cadence: after M KB of journal (512–2048 KB) or T seconds ( 30–120 s), whichever first.
• fsync policy:
  • always: fsync every flush (safest, slowest).
  • grouped (default): fsync at most every 1–2 s or on idle/blur/quit.
  • never: rely on OS flush (fastest, riskier).
  • On POSIX, prefer fdatasync when available; fall back to fsync.

Performance & threading

• Background writer thread per editor instance (shared) with a bounded MPSC queue of per‑buffer records.
• Each Buffer has a small in‑memory journal buffer; UI thread enqueues ops (non‑blocking) and may coalesce adjacent inserts/deletes.
• Writer batch‑writes records to the swap file, computes CRCs, and decides checkpoint boundaries.
• Backpressure: if the queue grows beyond a high watermark, signal the UI to start coalescing more aggressively and slow enqueue (never block hard editing path; at worst drop optional META).

Recovery flow

On opening a file:

1. Detect swap sidecar .<basename>.kte.swp.
2. Validate header, iterate records verifying CRCs.
3. Compare recorded original file identity against actual file; if mismatch, warn user but allow recovery (content wins).
4. Reconstruct buffer: start from the last good CHKPT (if any), then replay subsequent ops. If trailing partial record encountered (EOF mid‑record), truncate at last good offset.
5. Present a choice: Recover (load recovered buffer; keep the swap file until user saves) or Discard (delete swap file and open clean file).

Stability & corruption mitigation

• Append‑only with per‑record CRC32 guards against torn writes.
• Atomic checkpoint rotation: write .<basename>.kte.swp.tmp, fsync, then rename over old .swp.
• Size caps: rotate or compact when .swp exceeds a threshold (e.g., 64–128 MB). Compaction creates a fresh file with a single checkpoint.
• Low‑disk‑space behavior: on write failures, surface a non‑modal warning and temporarily fall back to in‑memory only; retry opportunistically.

Security considerations

• Swap files mirror buffer content, which may be sensitive. Options:
  • Configurable location (same dir vs. $XDG_STATE_HOME/kte/swap).
  • Optional per‑file encryption (future work) using OS keychain.
  • Ensure permissions are 0600.

Interoperability & UX

• Use a distinctive extension .kte.swp to avoid conflicts with other editors.
• Status bar indicator when swap is active; commands to purge/compact.
• On save: do not delete swap immediately; keep until the buffer is clean and idle for a short grace period (allows undo of accidental external changes).

Implementation plan (staged)

1. Minimal journal writer (append‑only INS/DEL) with grouped fsync; single per‑editor writer thread.
2. Reader/recovery path with CRC validation and replay.
3. Checkpoints + atomic rotation; compaction path.
4. Config surface and UI prompts; telemetry counters.
5. Optional compression and advanced coalescing.

Defaults balancing performance and stability

• Grouped flush with fsync every ~1 s or on idle/quit.
• Checkpoint every 1 MB or 60 s.
• Bounded queue and batch writes to minimize syscalls.
• Immediate flush on critical events (buffer close, app quit, power source change on laptops if detectable).