Each edit shifted every later line start in the line index, O(lines) per
keystroke (0.5 ms near the top of a 1M-line file). The shift is now kept
pending ("from entry k, add d") and applied on read; further edits at the
same line only adjust it, and it is folded in when an edit lands
elsewhere.
locate() walked the piece list for every lookup (two per rendered line
per frame on an edited buffer); it now binary-searches prefix sums
rebuilt once per modification (keyed on version_, which every mutation
bumps; AdoptOriginal now bumps it after installing its piece).
Typing on line 10 of a 1M-line file with a frame's worth of line reads
per keystroke: 0.52 ms -> 0.017 ms. A new randomized test checks every
read path against a string model with consolidation forced; it fails
under each of three deliberate breakages of the new bookkeeping.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
The row cache rebuilt a string for every line of the file after each edit
and had one production caller left (filetype detection in the ImGui
frontend, which needs only the first line). Removed with it: rows_,
nrows_, the dirty flag set on every edit, ensure_rows_cache and a mutex
guarding only that cache. Tests read lines with GetLineString, Nrows or,
where they compare every line, a test-only LinesForTests().
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
GetLine and LineColToByteOffset fetched each line's last byte (a piece
lookup, an allocation and a cache write) to see whether it was a newline;
the line index already says so: every line but the last ends at one.
LineColToByteOffset runs on every insert. RebuildLineIndex scans with
memchr instead of byte by byte.
Removed, having no callers: OptimizedSearch (built only into its own
test), UndoNodePool.h, SwapManager::RecordSplit/RecordJoin (replay still
reads SPLIT/JOIN), PieceTable::Reserve/Capacity and its capacity
constructor, Buffer's write-only render column, Editor's mode, no-kill,
dirtyex and repeatable flags and the UCount getter, and
RetryPolicy::Conservative.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
SearchHighlight compiles the search pattern once per frame and yields each
line's match ranges; the terminal, ImGui and Qt renderers had three copies
of this, and Qt compiled a std::regex for every visible line on every
frame. Qt now uses kte::Regex like the others.
kte::Regex::RunGuarded replaces Command.cc's run_regex_work, which chose
the large stack by comparing engine names. ForEachRegexMatch had no
callers left and is removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- PCRE2: compile with the ANYCRLF newline convention and DOLLAR_ENDONLY,
as ECMAScript reads '.' and '$'. With PCRE2's default (LF only) '.'
matched the CR of a CRLF line, so regex replace-all deleted CRs.
- Replacement $0 is the whole match, two digits are read greedily and a
group the pattern lacks expands to nothing, as std::regex_replace
(libstdc++) does; these cases are now compared in the tests.
- Plain search visits overlapping matches going forward too, and counts
them, as the backward search found them ('aa' in 'aaa' read 2/1).
- The recovery prompt's replayed content is used on 'y' only if the
journal is unchanged since; otherwise it is replayed again.
- A version 1 journal continued after recovery is rewritten as a
compacted version 2 journal when it needs a chunked checkpoint,
instead of receiving records an older kte would skip. (The version is
read through a separate descriptor: the journal fd is write-only.)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
RegexEngine.h/.cc wrap the regex engine for search, regex replace-all and
the terminal and ImGui renderers' match highlighting:
- With KTE_USE_PCRE2 (default ON; used when pkg-config finds libpcre2-8,
linked as -lpcre2-8) patterns are compiled by PCRE2 with its JIT and
matched under a match limit: catastrophic backtracking stops with a
status note instead of hanging the editor, and matching does not
recurse on the C stack (no large-stack thread; the per-keystroke and
per-frame line limit is 1 MiB instead of 2,000 bytes). Otherwise
std::regex is used as before.
- Replacement strings follow std::regex_replace (ECMAScript as libstdc++
reads it) with either engine: the PCRE2 path expands $&, $n, $`, $',
$$ itself and steps over empty matches the same way.
- Regex search walks lines over the materialized text instead of calling
GetLineView per row. The terminal renderer compiles the pattern once
per frame (it compiled it for every row).
- Only one checkpoint per journal is queued at a time: whole-file edits
in a row on a large buffer each queued a full copy of the buffer.
- default.nix and the Dockerfile add pcre2 and pkg-config.
On 26 MB: regex keystroke 1.6 -> 0.14 ms, next 62 -> 6 ms, replace-all
929 -> 438 ms; a rarely matching regex over 105 MB: 1.55 s -> 0.10 s.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- A checkpoint larger than one record (16 MiB) is written as CHKPT_BEGIN
[size u64, CRC-32], CHKPT_DATA chunks and CHKPT_END. Replay assembles it
and applies it only when complete with the size and CRC matching; an
unfinished one is a torn tail (valid_bytes stops before it). Buffers
over 16 MiB could not be checkpointed before: a lost record left their
journal behind for good, and they were never compacted. Records are
written as segments straight from the snapshot. Header version 2;
version 1 journals are still read.
- Checkpoint thresholds scale with the buffer size (edits of half its
size; the timer only after a sixteenth; compaction at twice its size;
gap retries spaced by size), and a checkpoint that would compact is
written straight into the compacted journal instead of being appended
and then copied.
- The recovery prompt keeps the content it replayed for its preview and
installs it on 'y', instead of replaying the journal a second time.
- A base file over 64 MiB gets its CRC from the writer thread before it
creates the journal (it had none, so touching such a file made its
journal unusable for recovery).
- kte_tests takes optional name substrings to run a subset of tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
run_search() replaces collecting every match in the buffer (and building
the full list again for every next/previous):
- Plain search finds forward from the search origin (then the current
match) in the materialized text, or backwards through 1 MiB blocks
searched forward (string_view::rfind scans a byte at a time: 511 ms
per previous-match on 105 MB, now 6 ms). Regex search walks lines
from the position with early exit.
- The i/N count stops at 1,000 matches, and for regex after 4 MiB; the
status says 'N+' when capped.
- The first match is the first at or after the cursor (it was the first
in the buffer). Extending the query keeps the current match while it
still matches. Left/Right also step through matches in the regex
replace prompt; Up/Down in a regex prompt use the regex.
On a 26 MB file: plain keystroke 30 ms -> 0.1 ms, next 24 -> 3 ms, regex
keystroke 434 -> 1.6 ms, regex next 420 -> 66 ms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- Undo texts of 4 KiB or more (Insert/Paste from the buffer, Delete from
the spans the piece table remembers for its last large deletion) are
held as TextSpan references into the piece table's append-only
storage, after checking they hold exactly the recorded bytes. Undo and
redo insert them with Buffer::insert_spans, which adds pieces over the
stored bytes instead of copying them into storage again (the journal
still gets the bytes, as one record). A replace-all session on a 26 MB
file peaks at 181 MB instead of 332 MB; undo/redo cycles of a 6.6 MB
kill no longer grow memory by 6.6 MB each.
- UndoSystem keeps a byte budget (default max(64 MiB, twice the buffer
size); SetByteBudget). Over it, history is pruned to 75% of the budget:
branches off the current path, then the oldest edits (an undo group
goes whole, and never the current state's group), then the redo line.
When the saved state is dropped the buffer stays modified until saved.
- replace_all_bytes clears undo history, which was recorded against (and
may reference the storage of) the previous content.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- HighlighterEngine caches spans only for rows near the requested row
and trims the cache around it past 16k entries; per-row end states are
still kept to resume lexing. Jumping to the end of a 78 MB C++ file
cached every row's spans (854 MB RSS, now 197 MB), and the first edit
near the top spent 270 ms freeing them.
- Plain and regex search read lines as views into the materialized
buffer instead of Buffer::Rows(), which built (and kept) a string per
line, O(lines x pieces) after edits. Plain search scans the whole text
once and tracks rows by newline.
- transform_rows() applies a per-row rewrite of a row range as one edit
(replace_rows_text). Indent/unindent region and visual-line insert,
backspace, delete, newline and yank used one piece-table edit per row,
each linear in the buffer: 1.1 s per keystroke with 5,000 rows selected
in a 26 MB file (now 4 ms), and minutes to indent a large region.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- The CRLF handling applied only when every line of the paragraph ended
in CR, so the last paragraph of a CRLF file without a final newline
(and mixed CRLF/LF paragraphs) still had CRs joined into the middle of
lines. Each line's CR is now stripped; if any line had one, every new
line but the last ends in CR, and the last keeps what the paragraph's
last line had.
- Wrapping never starts a line with a word that reads as a list marker
('-', '*', '+', '1.', '2)'); the next reflow turned such a paragraph
into a list.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- Indent-region skips empty lines. With the whole buffer selected, the
empty row after the final newline became a tab-only last line.
- Reflow no longer wraps before a line's first word: a first word longer
than the width (a URL) got a prefix-only line before it, a blank line
or a bullet marker split from its text, and another on every reflow.
- A whitespace-only line (spaces, tabs, a CR) separates paragraphs for
reflow; such paragraphs were merged.
- Reflow of CRLF lines strips the CR before wrapping and restores it on
every output line; the CRs ended up in the middle of joined lines.
- A counted backspace or delete-char is one undo step even when it joins
lines.
- The C++ highlighter stops looking for a raw-string '(' after 16
delimiter characters; scanning the whole line at every R" was
quadratic.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- The shortened fallback swap name was capped at exactly NAME_MAX, so
compaction's '<journal>.tmp' failed with ENAMETOOLONG on every
checkpoint and the journal grew by a full copy of the buffer each
time. Cap it at NAME_MAX - 4.
- A lone UTF-8 lead byte (8-bit Meta terminals) waited a second for
continuation bytes on every key; wait 250 ms. An invalid byte now ends
a pending ESC/C-k prefix instead of leaving it for the next key.
- Journal notices are shown after a deferred open's own status (which
used to overwrite them), all pending ones at once, and the lock-out
notice is not repeated after every save while the lock is still held.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- SwapManager queues user notices when a journal is held by another kte
(this session's edits are then not journaled) and, once per gap, when
journal writes fail. Editor shows them in the status line from its
per-frame hook. Both went only to the error log, so crash recovery was
silently off or incomplete.
- The fallback swap name '<basename>.<hash>.swp' exceeded NAME_MAX for
basenames over ~233 bytes, so such files were never journaled; the
basename is shortened (on a UTF-8 boundary) only in that case.
- Audit doc: sixth-round findings.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- TerminalInputHandler reads bytes with getch() and assembles UTF-8
itself, waiting up to a second for a character's continuation bytes.
The frontend reads the rest of an input batch with a zero timeout, and
get_wch() then gave up on a character whose bytes had not all arrived;
ncurses discarded the rest as an invalid sequence, so text pasted over
a slow link silently lost characters and what followed them (typing
'ab\xc3', then '\xa9cd' 50 ms later saved 'ab').
- Visual-line insert, delete, backspace and newline carried the cursor's
byte column to the other selected lines, splitting multibyte
characters there (the saved file became invalid UTF-8). The column is
now carried as a character count.
- Visual-line delete-char left the cursor on the last selected line.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- Buffer::insert_row recorded its text and newline as two swap records
after inserting both. A checkpoint triggered by the first (1 MiB of
edits, or the checkpoint timer) snapshotted the newline already, and
the second added it again on replay: recovery after undoing C-k C-k
silently gained a blank line. It is now one record.
- Visual-line yank placed the cursor relative to its row assuming that
row was inside the selection; after C-k a or Enter it was not, and the
cursor ended past the end of a line or the buffer, where typed text was
recorded for undo at a position it was not inserted at and could not
be undone. The formula uses the nearest selected row, and run_handler
now clamps the cursor to the buffer before and after every command.
- ENABLE_ASAN added -fsanitize=address to CMAKE_C_FLAGS only; the
project is C++, so nothing was instrumented. Add it to
CMAKE_CXX_FLAGS.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
After a lost record the journal drops position-based records until a
full checkpoint. That checkpoint was only requested from later edits,
rate-limited to once a second; when the writer handled the failing
record before its edit asked for a checkpoint, the checkpoint failed as
well and consumed the slot, so edits made within the next second were
not recovered and, if the user stopped typing, the journal stayed behind
indefinitely. SwapManager::RetryGapCheckpoints(), called once per frame
from Editor::ProcessPendingOpens, requests it once the interval has
passed. This also made SwapReplay_LostRecord_ResyncsWithCheckpoint flaky
under load; the test now exercises the retry.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- Save's in-place fallbacks (no temp file; no hard-link copy) now apply
only when the directory refuses a new name (EACCES, EPERM,
ENAMETOOLONG). On ENOSPC, EDQUOT or EIO the in-place write truncated
the file and then failed, leaving it corrupt; now the save fails and
the file is untouched (checked on a full tmpfs).
- replace_rows_text/replace_buffer_text delete and insert only the span
between the old and new text's common prefix and suffix (kept on
UTF-8 boundaries). Replace-all recorded the whole buffer twice in the
undo tree per run.
- Regex search skips long lines only while typing (limit lowered to
2,000 bytes, as for renderer highlighting; '.*q' on a 20 KB line took
12 s), reports how many were skipped, and next/previous match search
every line. Previously matches on long lines could not be found.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- main: when the environment's locale is not UTF-8 (LANG unset, C/POSIX),
switch LC_CTYPE to a UTF-8 locale so ncurses delivers and draws
multibyte characters instead of dropping them.
- Buffer::Id(): a process-unique id that survives moves and is fresh for
copies. The terminal renderer's skip and cursor caches and the reload
confirmation now key on it rather than the buffer's address, which is
reused after a buffer is closed.
- The reload confirmation is cancelled by any other command, like the
quit confirmation.
- TerminalFrontend redraws when a deferred open starts a prompt or sets a
status (e.g. the swap recovery question), not only when a buffer opens.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
Each piece-table edit costs time linear in the buffer (piece lookup,
line-index update), so operations issuing one edit per match, line or
repetition were quadratic:
- Plain and regex replace-all build the new text in one pass and apply
it as one delete + one insert (one undo group, one chunked journal
record). 12 MB with 229k matches: minutes -> 0.14 s.
- Yank inserts the whole kill in one edit (it split and re-trimmed line
by line): 200k lines in 0.02 s.
- InsertText with a repeat count inserts the repeated text once; the
total is capped at 256 MiB (also across visual-line broadcast).
- Incremental regex search skips lines over 20,000 bytes and renderer
match highlighting over 10,000 (the previous change raised the render
limit to 1 MB): std::regex can be quadratic in line length even for
ordinary patterns such as '.*q', uninterruptibly, and these run on
every keystroke or frame. Explicit regex replace still processes all.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- The writer's periodic fsync now holds a separate io_mtx_, taken also
(before mtx_) by the main-thread paths that close journal fds, instead
of mtx_. Every keystroke takes mtx_ to enqueue its record, so a slow
fsync (writeback pressure, NFS) froze typing.
- A journal's base identity (stat + content CRC) is computed before
taking mtx_ and applied under it; the CRC is skipped above 64 MiB
(size and mtime still identify the file). It re-read the whole file
under the lock on every open and save.
- A buffer that was never journaled (a virtual buffer saved under a
real name) is attached by NotifyFilenameChanged.
- ThreadSanitizer: open_ctx (writer thread) no longer writes ctx.path,
and locked_out is recorded by the caller under mtx_; both raced with
main-thread reads.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- OpenFromFile builds the new content in a separate PieceTable and
swaps it in only when complete. A reload that ran out of memory used
to leave the buffer empty (filename, journal and dirty state intact),
and the next save truncated the file.
- Editor::ProcessPendingOpens catches exceptions from opening (it runs
in every frontend's loop, outside command dispatch, so bad_alloc on a
huge file ended the editor); the terminal draw is guarded too.
- The hard-link save copy uses a unique mkstemp name, so a leftover
copy no longer blocks every later save; if no copy can be created the
file is still written in place.
- When no temp file can be created in the directory (file writable,
directory not), an existing regular file is written in place instead
of the save failing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- Redoing an insert and undoing a delete put the cursor at col + size
on the start row even when the text spans lines, i.e. past the end of
the line. Later edits and their undo records then went to the wrong
place (a join recorded at column 2 of 'a' re-split the text wrongly)
and delete-word-prev read past the line (heap-buffer-overflow under
ASan). The cursor now goes after the text's last newline, and undo and
redo clamp it to the buffer.
- A recovered buffer stays modified in every undo state
(mark_base_unsaved): one edit and its undo made it look clean, and
closing it then discarded the recovered text and its journal.
- Save-and-quit records the saved undo state.
- Visual-line selections are clamped to the buffer when read; the
visual newline counts the cursor's own line only if it was selected.
A stale selection put the cursor rows past the end, where typed text
could not be undone.
- The yank size cap accounts for visual-line yanks (once per line).
- After an exception escapes a command, open undo groups are closed so
later edits do not join them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- kte refuses to start when stdin is not a terminal: piped bytes were
executed as editor commands (C-k s plus a path writes a file), and
after EOF the loop spun at full CPU with no way to quit.
- The terminal frontend handles all pending input before drawing (a
paste used to cost a frame and a full redraw per character) and
redraws only after input, a resize or deferred opens, else once a
second. The renderer caches, per row and buffer version, where the
horizontally scrolled part of a line starts and the cursor's render
column, instead of rescanning the line every frame. Idle CPU with the
cursor at the end of a 20 MB line: one full core -> ~1%.
- Buffers with a virtual name (+HELP+) are not journaled and saving one
asks for a file name; it used to write ./+HELP+ and journal under
that path.
- Yank with a repeat count is capped at 256 MiB in total instead of
reserving count x size bytes (C-u 1000000 C-y of a large kill).
- Remove an overload clang reports as unused (-Werror on macOS).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- Ownership is applied before permissions: chown clears setuid/setgid,
so saving a 06755 file left it 0755.
- The umask is read once during static initialisation. Reading it at
save time needed umask(0)/umask(old), briefly changing it for the swap
writer thread creating files.
- Before rewriting a hard-linked file in place, the new content is
written and synced to a copy beside it; if the in-place write fails
(ENOSPC, EIO) the copy is kept and named in the error.
- Journal headers also record a CRC-32 of the base file (flags bit 1,
bytes 40..43). A changed mtime with the same size and content
(touch, checkout back and forth) no longer marks a journal stale.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- Quit checks every buffer for unsaved changes and names them; it used
to look only at the current buffer and exit, dropping the rest (an
unnamed buffer's text was unrecoverable). Save-and-quit does not quit
while other buffers are dirty.
- Reload of a dirty buffer needs a second C-k l with no edit in between
(it discarded edits, undo history and the journal at once), and a file
deleted on disk is not 'reloaded' into an empty buffer.
- At the destructive y/n prompts ("Recover swap edits?" and "Save
changes to X?"), only an explicit 'n' discards; Enter or any other key
cancels. Keys typed while the startup recovery prompt appeared used to
delete the crash journal. SwapRecoveryPrompt_Discard now answers 'n'.
- Saving from the close prompt refuses to overwrite a file changed on
disk, like save.
- Save-as onto a file open in another buffer is refused.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- A session never unlinks a journal it does not own: Detach, ResetJournal
and NotifyFilenameChanged skip removal when this session was locked
out or another session holds the lock, and PruneSwapDir skips held
journals. A second session used to delete the first session's live
journal on save or close.
- The writer thread's periodic fsync now runs under mtx_. It fsynced fds
that the main thread could close concurrently (or that had been
reused); an in-flight fsync also kept a just-closed journal's flock
alive, so the owner's own cleanup saw its journal as held.
- Compaction locks the compacted file before renaming it into place and
keeps using it, so the journal is never unlocked in between; a failed
lock on the truncating reopen marks the session locked out.
- Unnamed-buffer journals include the pid, so sessions no longer share
(and delete) each other's unnamed-N.swp.
- RowsView keeps only the previous generation of cached rows: keeping
all of them made C-u 2000 M-d on a 1 MiB line use ~2 GB.
- Renderers run regex match highlighting through ForEachRegexMatch,
which uses the large-stack worker for lines over 512 bytes; some
patterns overflowed the main stack well under the previous 10,000
byte limit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- OpenFromFile opens non-blocking, fstat()s and accepts regular files
only. A directory made tellg() report a huge size and bad_alloc exit
the editor (reachable by Tab-completing a directory in the open
prompt); a FIFO blocked open() forever with SIGINT ignored; a device
node opened as an empty, file-backed buffer that saving then replaced
with a regular file. Files are read to EOF in chunks; a file too large
to load is reported, not fatal.
- Saving refuses to replace an existing non-regular file.
- Exceptions escaping a command handler are reported in the status line
and error log instead of propagating to main() and exiting, which lost
every unsaved buffer (unnamed buffers' journals are never recovered).
- Commands use non-throwing filesystem queries for existence checks,
the current directory and Tab completion (ENAMETOOLONG, EACCES or a
deleted working directory used to throw).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- open_ctx takes a non-blocking flock on the journal. A journal locked
by another kte process is left alone: this session reports it once
and does not write (records from two sessions used to interleave in
one file). Opening a file whose journal is live in another session
skips recovery instead of replaying or removing that journal.
- The header records the size and mtime of the file the journal's
records apply to (flags bit 0, bytes 24..39; previously reserved, so
no version change). Before offering recovery, a journal whose base no
longer matches the file on disk (changed after a crash, or the crash
fell between a save's rename and the journal reset) is treated as
stale and offered for deletion, instead of replaying its edits onto
different text. Journals from older kte (flag clear) replay as before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- C0 controls and DEL were passed to ncurses, which acted on them: a
carriage return moved to column 0 and the line padding then erased
the text, so every line of a CRLF file drew blank, and a backspace
ate the previous character. They are now drawn in caret notation
(^M, ^H, ^?) and counted as two cells. C1 controls (and stray bytes
0x80-0x9F, which decode to them; U+009B is CSI) are drawn as '?'.
- The width rule lives in TermWidth.h and is shared by the renderer's
drawing loop, its cursor placement and the command layer's
horizontal scrolling and mouse mapping, which also disagreed on NUL
and invalid bytes.
- GUI config: columns/rows must be 1..1000 and the font size a finite
4..200 (TOML 'size = inf' and huge values were accepted, leading to
UB in float-to-int conversion and int overflow downstream).
Verified in tmux: a CRLF file now shows 'first line^M' where the
baseline showed a blank line.
Audit findings B9 (and the CR/BS case from the follow-up review).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- fsync is never retried. After a writeback error Linux may mark the
pages clean, so a retried fsync could succeed although the data was
lost, and the rename then replaced a good file with a bad one.
- New files get 0666 & ~umask instead of mkstemp's 0600. Replaced files
keep their mode (file-type bits masked) and, best effort, their owner
and group.
- Saving through a symlink writes the link's target instead of
replacing the link with a regular file (SaveAs did not resolve it).
- A file with several hard links is rewritten in place, so the other
names see the new content (as vim does by default).
- Save-as (prompt and ':save-as') asks before replacing any existing
file other than the buffer's own; it used to ask only for unnamed
buffers. Save-and-quit refuses to overwrite a file changed on disk,
like save. The first save of a new file records the saved undo state.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- Recovery works when another named buffer is current. On that path
OpenFile's NotifyFilenameChanged deleted the journal (old path ==
new path) that was about to be replayed; the swap was lost and the
file opened without the edits (pre-existing).
- After recovering from a journal with a torn final record, the torn
bytes are truncated before the session appends to it (ReplayFile now
reports the valid prefix length). A journal that recovers nothing is
removed instead of appended to.
- Reload restarts the journal; recovery used to replay the edits the
reload discarded.
- Opening a file that is already open switches to its buffer (in
OpenFile and before the recovery check). Two buffers for one file
interleaved records in one journal, and closing either unlinked it.
- Swap file names escape '!' and '%', so distinct paths no longer
share a journal ('/x/a!b' vs '/x/a/b'). Swap files for paths that
contain those characters get new names.
- The gap flag is cleared when the journal restarts, and a gap that
cannot be closed (buffer too large to checkpoint) is reported once.
- crc32's table is initialised thread-safely; writer-thread error
reports identify the buffer by journal path instead of racing on
Buffer::Filename(); Close() no longer retries on EINTR (the fd is
already released and may have been reused).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
libstdc++'s std::regex matcher recurses about once per matched
character, so '.*', 'a+' or '(a|b)*' on a line of a few tens of
thousands of characters (minified JS/JSON) overflowed the 8 MiB stack
and crashed the editor from the regex search prompt or regex replace.
Regex search and regex replace now run their matching on a worker
thread with a 1 GiB reserved stack (RunWithLargeStack; the caller
waits, so nothing is shared concurrently). Renderers skip regex match
highlighting on lines over 10,000 bytes. Catastrophic backtracking
(e.g. (a*)*b) is still possible: std::regex has no time limit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- Motion and delete commands read rows through RowsView (on-demand,
version-cached) instead of Buffer::Rows(), which rebuilt every line of
the file after each edit. On a 500k-line buffer after an edit:
backspace/delete/motions 80-90 ms -> ~0.1 ms; C-u 3000 C-d 273 s ->
0.25 s. Whole-buffer search scans keep Rows() (cached between edits).
- While a prompt is open only prompt-aware and view-only commands run.
Others edited the buffer behind the prompt (bypassing the read-only
check) or switched/closed buffers under a pending confirmation, which
then acted on the wrong buffer. The ':' command path now applies the
read-only check too.
- Pasting into a prompt (ImGui) flattens line breaks to spaces instead
of emitting Newline, which accepted the prompt and sent the rest of
the paste to the buffer (or ran replace-all).
- The mark region clamps both ends; a stale mark past the end of the
buffer is treated as end of buffer. This fixes kill-region deleting
the wrong text and leaving the cursor past end-of-line.
- Kill-line with a count stops at end of buffer instead of killing
lines above the cursor (regression from the D3 change).
- Undo groups nest, so SmartNewline in visual-line mode is one step.
- Word motions and word deletion treat non-ASCII bytes as word
characters and so never split a UTF-8 character.
- The universal-argument count saturates at 1,000,000 instead of
overflowing int.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
Consolidation rewrites pieces without changing content or offsets but
invalidated the line index, so once a buffer passed the piece limit
every edit paid a full rescan again. A moved-from table kept the index
of the text it gave away, and Insert now updates that index in place,
producing ranges past the end of its buffer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
Every edit bumped the buffer version (through SetDirty) and called
InvalidateFrom(0), so all cached spans and line states were discarded
and stateful highlighters re-lexed from row 0 to the viewport on each
keystroke.
Buffer mutators now report the row they changed (edited_at_), which
bumps the version and tells the engine to drop only rows from there
down; a row's highlighting depends only on rows above it. A version
change that arrives without that notice still clears everything, so a
missed path cannot show stale spans. The engine keeps states as a
contiguous vector and caches spans for every row it lexes. SetDirty no
longer touches the version: the edits themselves do, including undo
and redo back to the saved state (D9 still holds; its test now checks
the highlighting itself).
Typing near the end of a 100k-line C++ file: 46.3 ms -> 0.14 ms of
highlighting work per keystroke.
Audit finding P3.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- PieceTable::Insert/Delete adjust a valid line index directly (shift
later line starts, add or drop starts for inserted or deleted
newlines) instead of invalidating it, which forced a byte-by-byte
rescan of the whole buffer on the next line query. On a 38 MB,
1M-line buffer, insert + line query drops from 14.7 ms to 0.28 ms.
- The terminal and ImGui renderers fetch only the rows they draw
(Nrows + GetLineString) instead of Buffer::Rows(), which rebuilt a
string for every line of the file on the first frame after each edit.
Qt's line-count uses switch to Nrows().
ImGuiRenderer.cc was syntax-checked with g++ and clang++ against the
vendored ImGui headers; the GUI targets could not be built here (no
SDL2 or Qt6).
Audit findings P1, P2.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- Backspace and delete-char remove a whole UTF-8 character (they
removed one byte, leaving an invalid lead byte in the file), and
record the full character for undo.
- Left/right step over whole characters; up/down snap the column to a
character boundary and clamp the row, which also removes an unsigned
underflow in move-down when the cursor was past the last row.
- Horizontal scroll offsets are computed in display cells, the way the
terminal renderer draws (tabs, wide characters), instead of bytes, so
long non-ASCII lines no longer scroll to blank with the cursor off
screen. The mouse column mapping uses the same measure.
Audit findings B5, B6.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- Regex replace-all, reflow and kill-region replaced or deleted whole
rows, which added or left a stray newline when the text ended on a
last line without one. They now edit the byte range in place.
- Regex replace-all no longer matches the empty position after a final
newline. Before, a zero-width pattern such as ^ matched it, added a
line, and matched the new empty last row again: the editor hung.
- Kill-line on the last row kills an unterminated line with the newline
before it, and does nothing on the empty row after a final newline
(it used to record an undo step that added a line).
- The visual-line newline records its splits as one undo step, and
places the cursor correctly for a repeat count.
- Buffer::delete_text computes its byte range directly instead of
materializing every line it spans.
Audit findings D3, D4, D5.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- Replay recovers every complete record before a torn final record (the
normal result of a crash mid-append) and reports it through err,
instead of rejecting the whole journal. SwapReplay_TruncatedLog is
updated to the new behaviour.
- A record that cannot be written (open/write failure, breaker open)
marks the journal as having a gap. Later position-based records are
dropped until a checkpoint re-establishes the content; recorders
request one on the next edit (at most once per second).
- A failed append is truncated back off the file, and a failed header
write closes the file so the next attempt starts fresh.
- Inserts over the 24-bit record limit are split into consecutive
records; buffers too large to checkpoint are no longer snapshotted
and rejected on every checkpoint.
- Declining to delete an unreadable swap moves it to .corrupt so new
edits are not appended behind the bad data.
- NotifyFilenameChanged no longer writes a full checkpoint into the
journal it is about to delete.
Audit findings D7, P5.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
A secondary GUI window shared the primary's buffer vector but attached
buffers through its own SwapManager. Closing the window destroyed that
manager while buffers still held its recorders, so the next edit called
into freed memory; Rehome through the wrong manager also dropped
journaling for the primary's buffers.
Audit finding M3.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
The ImGui Cmd+V/Ctrl+V paste handler split clipboard text only on '\n',
leaving '\r' inside each segment. Pasting text with CRLF (Windows) or bare
CR (classic-Mac / some macOS apps) line endings enqueued an InsertText arg
containing '\r', which the InsertText command rejects with "InsertText arg
must not contain newlines".
Extract the paste-to-commands logic into a pure, SDL-free helper
(PasteSplit.h) that treats '\n', '\r\n', and bare '\r' each as one line
break, and add unit tests covering all line-ending forms plus blank-line
and trailing-newline preservation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Undo/redo:
- cmd_newline recorded the undo node's position after the cursor moved past
the split point, so undoing Enter joined the wrong pair of lines.
- Backspace/Delete line-joins recorded UndoType::Newline, whose apply()
semantics are inverted for a join (forward=split); add UndoType::JoinLines
with correct forward/backward behavior.
- Regex replace-all, indent/unindent region, kill-to-eol, kill-line,
kill-region, and delete-word-prev/next mutated the buffer via the
no-undo-recording raw APIs; they now record undo, grouped atomically via a
shared UndoGroupGuard.
- Plain-text replace-all with an empty replacement advanced the scan
position one character too far, skipping adjacent/overlapping matches.
Crash recovery / memory safety:
- Buffer's move ctor/assignment never carried over swap_rec_,
on_disk_identity_, or the visual-line-mode fields, so the crash-recovery
journal silently stopped tracking a buffer whenever Editor's
std::vector<Buffer> reallocated or shifted (e.g. opening/closing files).
Added SwapManager::Rehome() plus Flush()-before-mutate in
Editor::AddBuffer/CloseBuffer so the journal's Buffer* key and the
background writer thread never reference a stale address.
- UndoTree leaked its entire node graph (including all edit text) on every
buffer close/reload; added ~UndoTree() to free it.
- main.cc didn't restore the terminal if an exception escaped the run loop;
added an RAII guard around Frontend::Shutdown().
Input handling:
- Terminal frontend used getch() instead of get_wch(), silently dropping
non-ASCII keyboard input despite linking wide-char ncurses.
- KKeymap's C-k lookup switches on an already-lowercased key, making
`case 'E'` unreachable dead code; C-k Shift-E silently aliased to C-k e.
- ImGui file picker wasn't modal: keystrokes typed while it was open still
reached the buffer underneath as edit commands, and Escape didn't close it.
- Gated ImGuiInputHandler's unconditional fprintf/fflush diagnostics behind
an IMGUI_IH_DEBUG macro, matching QtInputHandler's existing convention.
Syntax highlighting:
- Go/Rust/SQL highlighters weren't stateful, so multi-line /* */ comments
mis-highlighted as code starting on the second line.
- Python's triple-quoted-string handler didn't re-scan the remainder of a
line after a string closed mid-line, missing a same-line reopen.
- HighlighterEngine's state_last_contig_ field was declared but unused,
forcing an O(n) scan of state_cache_ on every stateful lookup; wired it up
as a real fast path and fixed InvalidateFrom() to keep it in sync.
- kge's :syntax off was silently undone the next frame because
apply_syntax_to_buffer() re-applied the config default unconditionally
every frame; added a user-override flag mirroring the existing
edit-mode-detection guard.
Added regression tests for all of the above (test_undo.cc, test_kkeymap.cc,
test_command_semantics.cc, test_search_replace_flow.cc, and a new
test_syntax_highlighting.cc). Full suite (163 tests) passes, including under
AddressSanitizer.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Multi-window:
- Per-window ImGui contexts (fixes input, scroll, and rendering isolation)
- Per-instance scroll and mouse state in ImGuiRenderer (no more statics)
- Proper GL context activation during window destruction
- ValidateBufferIndex guards against stale curbuf_ across shared buffers
- Editor methods (CurrentBuffer, SwitchTo, CloseBuffer, etc.) use Buffers()
accessor to respect shared buffer lists
- New windows open with an untitled buffer
- Scratch buffer reuse works in secondary windows
- CMD-w on macOS closes only the focused window
- Deferred new-window creation to avoid mid-frame ImGui context corruption
Swap file cleanup:
- SaveAs prompt handler now calls ResetJournal
- cmd_save_and_quit now calls ResetJournal
- Editor::Reset detaches all buffers before clearing
- Tests for save-and-quit and editor-reset swap cleanup
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1. Static linking - Added KTE_STATIC_LINK CMake option and
disabled it in default.nix to avoid the "attempted
static link of dynamic object" error
2. Missing include - Added <cstring> to
test_swap_edge_cases.cc for std::memset/std::memcpy (GCC
14 is stricter about transitive includes)
- Added `test_reflow_undo.cc` to validate undo/redo workflows for reflow operations.
- Introduced `UndoType::InsertRow` in `UndoSystem` for tracking row insertion changes in undo history.
- Updated `UndoNode.h` and `UndoSystem.cc` to support row insertion as a standalone undo step.
- Enhanced reflow paragraph functionality to properly record undo/redo actions for both row deletion and insertion.
- Enabled legacy/extended undo tests in `test_undo.cc` for comprehensive validation.
- Updated `CMakeLists.txt` to include new test file in the build target.
- Introduced `CommandId::SmartNewline` for auto-indented newlines, enhancing text editing workflows.
- Added `cmd_smart_newline` to implement indentation-aware newline logic.
- Integrated SmartNewline with keymaps, mouse/keyboard input handlers, and terminal/editor commands.
- Wrote comprehensive tests in `test_smart_newline.cc` to validate behavior for spaces, tabs, and no-indentation cases.
- Updated `Command.h` and `CMakeLists.txt` to register and build the new command.
- Introduced `test_swap_edge_cases.cc` with extensive tests for minimum payload sizes, truncated payloads, data overflows, unsupported encoding versions, CRC mismatches, and mixed valid/invalid records to ensure reliability under complex scenarios.
- Enhanced `main.cc` with a top-level exception handler to prevent data loss and ensure cleanup during unexpected failures.
Claude
kyleandClaude Opus 4.8