Highlighters emit one span per byte of a non-ASCII character, and the
ImGui and Qt renderers decoded each span on its own, so every such
character was drawn as replacement glyphs (reproduced under Xvfb:
"héllo" showed as "h??llo"). kte::SanitizeSpans (Highlight.h) widens
spans to whole UTF-8 characters and removes overlaps (the earlier span
wins), and replaces the three renderers' copies of the clamp/sort code.
The terminal renderer's token lookup rescanned the spans from the first
for every byte drawn; it now walks them with a cursor.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
HighlighterEngine::GetLine copied the span vector on every call (twice
on a miss) and did a dynamic_cast per call; it now returns the cached
entry (valid until the next call) and dispatches on Stateful().
PieceTable and HighlighterEngine took a mutex in their read paths while
their writes took none, and PieceTable.h claimed concurrent reads were
safe; they are not (reads update caches). Every Buffer is used from the
main thread only (the swap writer thread receives copies; the regex
large-stack thread runs while its caller waits), so the mutexes are
removed and the headers say so. ASan and the threaded tests under TSan
are clean.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
ImGui measured every span, highlight and the cursor from column 0 of the
line (and rescanned the line for each column conversion), which is
quadratic in line length on minified files. Columns now come from one
table per line and pixel offsets from the nearest position already
measured. Output is pixel-identical to before (compared under Xvfb, with
syntax colouring, tabs, UTF-8 and search highlights).
kte::kTabWidth replaces the literal 8s, and Qt's 4: Qt expanded tabs to
4 columns while the command layer scrolled by 8, misplacing the cursor
and horizontal scroll on lines with tabs. MeasureChar and
DisplayColumns (TermWidth.h) are the command layer's column model, now
shared, and Qt draws by it instead of one column per byte, which
shifted everything after a multi-byte character.
Qt also:
- drew operators in the dark-mode background colour, so they vanished;
- computed the editor's column count three ways (different glyph,
padding and status-row accounting) from painting, and positioned text
with a rounded cell width, so the cursor drifted off the glyphs on long
lines. One TextLayout now serves painting, resizing, font changes and
wheel scrolling, with a fractional cell width; runs are drawn only
where visible;
- drew each non-ASCII character at its cell, since a fallback-font glyph
wider than a cell pushed the rest of the run out of place.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
SearchHighlight compiles the search pattern once per frame and yields each
line's match ranges; the terminal, ImGui and Qt renderers had three copies
of this, and Qt compiled a std::regex for every visible line on every
frame. Qt now uses kte::Regex like the others.
kte::Regex::RunGuarded replaces Command.cc's run_regex_work, which chose
the large stack by comparing engine names. ForEachRegexMatch had no
callers left and is removed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
RegexEngine.h/.cc wrap the regex engine for search, regex replace-all and
the terminal and ImGui renderers' match highlighting:
- With KTE_USE_PCRE2 (default ON; used when pkg-config finds libpcre2-8,
linked as -lpcre2-8) patterns are compiled by PCRE2 with its JIT and
matched under a match limit: catastrophic backtracking stops with a
status note instead of hanging the editor, and matching does not
recurse on the C stack (no large-stack thread; the per-keystroke and
per-frame line limit is 1 MiB instead of 2,000 bytes). Otherwise
std::regex is used as before.
- Replacement strings follow std::regex_replace (ECMAScript as libstdc++
reads it) with either engine: the PCRE2 path expands $&, $n, $`, $',
$$ itself and steps over empty matches the same way.
- Regex search walks lines over the materialized text instead of calling
GetLineView per row. The terminal renderer compiles the pattern once
per frame (it compiled it for every row).
- Only one checkpoint per journal is queued at a time: whole-file edits
in a row on a large buffer each queued a full copy of the buffer.
- default.nix and the Dockerfile add pcre2 and pkg-config.
On 26 MB: regex keystroke 1.6 -> 0.14 ms, next 62 -> 6 ms, replace-all
929 -> 438 ms; a rarely matching regex over 105 MB: 1.55 s -> 0.10 s.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- main: when the environment's locale is not UTF-8 (LANG unset, C/POSIX),
switch LC_CTYPE to a UTF-8 locale so ncurses delivers and draws
multibyte characters instead of dropping them.
- Buffer::Id(): a process-unique id that survives moves and is fresh for
copies. The terminal renderer's skip and cursor caches and the reload
confirmation now key on it rather than the buffer's address, which is
reused after a buffer is closed.
- The reload confirmation is cancelled by any other command, like the
quit confirmation.
- TerminalFrontend redraws when a deferred open starts a prompt or sets a
status (e.g. the swap recovery question), not only when a buffer opens.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- kte refuses to start when stdin is not a terminal: piped bytes were
executed as editor commands (C-k s plus a path writes a file), and
after EOF the loop spun at full CPU with no way to quit.
- The terminal frontend handles all pending input before drawing (a
paste used to cost a frame and a full redraw per character) and
redraws only after input, a resize or deferred opens, else once a
second. The renderer caches, per row and buffer version, where the
horizontally scrolled part of a line starts and the cursor's render
column, instead of rescanning the line every frame. Idle CPU with the
cursor at the end of a 20 MB line: one full core -> ~1%.
- Buffers with a virtual name (+HELP+) are not journaled and saving one
asks for a file name; it used to write ./+HELP+ and journal under
that path.
- Yank with a repeat count is capped at 256 MiB in total instead of
reserving count x size bytes (C-u 1000000 C-y of a large kill).
- Remove an overload clang reports as unused (-Werror on macOS).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- A session never unlinks a journal it does not own: Detach, ResetJournal
and NotifyFilenameChanged skip removal when this session was locked
out or another session holds the lock, and PruneSwapDir skips held
journals. A second session used to delete the first session's live
journal on save or close.
- The writer thread's periodic fsync now runs under mtx_. It fsynced fds
that the main thread could close concurrently (or that had been
reused); an in-flight fsync also kept a just-closed journal's flock
alive, so the owner's own cleanup saw its journal as held.
- Compaction locks the compacted file before renaming it into place and
keeps using it, so the journal is never unlocked in between; a failed
lock on the truncating reopen marks the session locked out.
- Unnamed-buffer journals include the pid, so sessions no longer share
(and delete) each other's unnamed-N.swp.
- RowsView keeps only the previous generation of cached rows: keeping
all of them made C-u 2000 M-d on a 1 MiB line use ~2 GB.
- Renderers run regex match highlighting through ForEachRegexMatch,
which uses the large-stack worker for lines over 512 bytes; some
patterns overflowed the main stack well under the previous 10,000
byte limit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- C0 controls and DEL were passed to ncurses, which acted on them: a
carriage return moved to column 0 and the line padding then erased
the text, so every line of a CRLF file drew blank, and a backspace
ate the previous character. They are now drawn in caret notation
(^M, ^H, ^?) and counted as two cells. C1 controls (and stray bytes
0x80-0x9F, which decode to them; U+009B is CSI) are drawn as '?'.
- The width rule lives in TermWidth.h and is shared by the renderer's
drawing loop, its cursor placement and the command layer's
horizontal scrolling and mouse mapping, which also disagreed on NUL
and invalid bytes.
- GUI config: columns/rows must be 1..1000 and the font size a finite
4..200 (TOML 'size = inf' and huge values were accepted, leading to
UB in float-to-int conversion and int overflow downstream).
Verified in tmux: a CRLF file now shows 'first line^M' where the
baseline showed a blank line.
Audit findings B9 (and the CR/BS case from the follow-up review).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
libstdc++'s std::regex matcher recurses about once per matched
character, so '.*', 'a+' or '(a|b)*' on a line of a few tens of
thousands of characters (minified JS/JSON) overflowed the 8 MiB stack
and crashed the editor from the regex search prompt or regex replace.
Regex search and regex replace now run their matching on a worker
thread with a 1 GiB reserved stack (RunWithLargeStack; the caller
waits, so nothing is shared concurrently). Renderers skip regex match
highlighting on lines over 10,000 bytes. Catastrophic backtracking
(e.g. (a*)*b) is still possible: std::regex has no time limit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- PieceTable::Insert/Delete adjust a valid line index directly (shift
later line starts, add or drop starts for inserted or deleted
newlines) instead of invalidating it, which forced a byte-by-byte
rescan of the whole buffer on the next line query. On a 38 MB,
1M-line buffer, insert + line query drops from 14.7 ms to 0.28 ms.
- The terminal and ImGui renderers fetch only the rows they draw
(Nrows + GetLineString) instead of Buffer::Rows(), which rebuilt a
string for every line of the file on the first frame after each edit.
Qt's line-count uses switch to Nrows().
ImGuiRenderer.cc was syntax-checked with g++ and clang++ against the
vendored ImGui headers; the GUI targets could not be built here (no
SDL2 or Qt6).
Audit findings P1, P2.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
- Introduced SwapManager for sidecar journaling of buffer mutations, with a safe recovery mechanism.
- Added group undo/redo functionality, allowing atomic grouping of related edits.
- Implemented `SwapRecorder` and integrated it as a callback interface for mutations.
- Added unit tests for swap journaling (save/load/replay) and undo grouping.
- Refactored undo to support group tracking and ID management.
- Updated CMake to include the new tests and swap journaling logic.
- Introduced visual-line mode for multi-line selection and edits.
- Implemented commands, rendering, and keyboard shortcuts.
- Added tests for broadcast operations in visual-line mode.
- Added full UTF-8 support for terminal rendering, including multi-width character handling.
- Improved font handling in ImGui with expanded glyph support (Greek, Mathematical Operators).
- Updated locale initialization to enable proper character rendering.
- Bumped version to 1.5.8.
- Added prefetching in both terminal and GUI renderers to optimize visible row highlights.
- Introduced background worker for offscreen highlight warming to improve scrolling performance.
- Refactored `HighlighterEngine` to manage thread-safety, caching, and stateful re-computation.
- Integrated changes into `HighlighterEngine`, `TerminalRenderer`, and `GUIRenderer`.
- Bumped version to 1.2.0 in preparation for the release.
- Introduced `HighlighterRegistry` with support for multiple language highlighters (e.g., JSON, Markdown, Python).
- Added `JsonHighlighter` implementation for basic JSON syntax highlighting.
Release / Bump Homebrew formula (push) Has been cancelled
Release / Build Linux amd64 (push) Has been cancelled
Release / Build Linux arm64 (push) Has been cancelled
Release / Build macOS arm64 (.app) (push) Has been cancelled
Release / Create GitHub Release (push) Has been cancelled
- Expanded help text and command documentation with detailed keybinding descriptions.
- Added theme customization support to GUIConfig (Nord default, light/dark variants).
- Adjusted for consistent indentation and debug instrumentation in undo system.
- Enhanced test cases for multi-line, UTF-8, and branching scenarios.
Release / Bump Homebrew formula (push) Has been cancelled
- Display buffer position prefix "[x/N]" in GUI and terminal renderers.
- Improve `kte` and `kge` man pages with frontend usage details and project homepage.
- Update README with GUI invocation instructions.
- Bump version to 1.0.0.
- Add visual file picker for GUI with toggle support.
- Introduce `GUIConfig` class for loading GUI settings from configuration file.
- Refactor window initialization to support dynamic sizing based on configuration.
- Add macOS-specific handling for fullscreen behavior.
- Improve header inclusion order and minor code cleanup.
- Normalize path handling for buffer operations, supporting tilde expansion and absolute paths.
- Introduce `DisplayNameFor` to uniquely resolve buffer display names, minimizing filename clashes.
- Add new commands: `ShowWorkingDirectory` and `ChangeWorkingDirectory`.
- Refine keybindings and enhance existing commands for improved command flow.
- Adjust GUI and terminal renderers to display total line counts alongside filenames.
- Update coding style to align with project guidelines.
Claude
kyle