kyle/kte
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Fix undo correctness, crash-recovery, and syntax highlighting bugs from full codebase review

Browse Source 
Undo/redo:
- cmd_newline recorded the undo node's position after the cursor moved past
  the split point, so undoing Enter joined the wrong pair of lines.
- Backspace/Delete line-joins recorded UndoType::Newline, whose apply()
  semantics are inverted for a join (forward=split); add UndoType::JoinLines
  with correct forward/backward behavior.
- Regex replace-all, indent/unindent region, kill-to-eol, kill-line,
  kill-region, and delete-word-prev/next mutated the buffer via the
  no-undo-recording raw APIs; they now record undo, grouped atomically via a
  shared UndoGroupGuard.
- Plain-text replace-all with an empty replacement advanced the scan
  position one character too far, skipping adjacent/overlapping matches.

Crash recovery / memory safety:
- Buffer's move ctor/assignment never carried over swap_rec_,
  on_disk_identity_, or the visual-line-mode fields, so the crash-recovery
  journal silently stopped tracking a buffer whenever Editor's
  std::vector<Buffer> reallocated or shifted (e.g. opening/closing files).
  Added SwapManager::Rehome() plus Flush()-before-mutate in
  Editor::AddBuffer/CloseBuffer so the journal's Buffer* key and the
  background writer thread never reference a stale address.
- UndoTree leaked its entire node graph (including all edit text) on every
  buffer close/reload; added ~UndoTree() to free it.
- main.cc didn't restore the terminal if an exception escaped the run loop;
  added an RAII guard around Frontend::Shutdown().

Input handling:
- Terminal frontend used getch() instead of get_wch(), silently dropping
  non-ASCII keyboard input despite linking wide-char ncurses.
- KKeymap's C-k lookup switches on an already-lowercased key, making
  `case 'E'` unreachable dead code; C-k Shift-E silently aliased to C-k e.
- ImGui file picker wasn't modal: keystrokes typed while it was open still
  reached the buffer underneath as edit commands, and Escape didn't close it.
- Gated ImGuiInputHandler's unconditional fprintf/fflush diagnostics behind
  an IMGUI_IH_DEBUG macro, matching QtInputHandler's existing convention.

Syntax highlighting:
- Go/Rust/SQL highlighters weren't stateful, so multi-line /* */ comments
  mis-highlighted as code starting on the second line.
- Python's triple-quoted-string handler didn't re-scan the remainder of a
  line after a string closed mid-line, missing a same-line reopen.
- HighlighterEngine's state_last_contig_ field was declared but unused,
  forcing an O(n) scan of state_cache_ on every stateful lookup; wired it up
  as a real fast path and fixed InvalidateFrom() to keep it in sync.
- kge's :syntax off was silently undone the next frame because
  apply_syntax_to_buffer() re-applied the config default unconditionally
  every frame; added a user-override flag mirroring the existing
  edit-mode-detection guard.

Added regression tests for all of the above (test_undo.cc, test_kkeymap.cc,
test_command_semantics.cc, test_search_replace_flow.cc, and a new
test_syntax_highlighting.cc). Full suite (163 tests) passes, including under
AddressSanitizer.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Kyle Isom
2026-07-01 14:19:15 -07:00
parent b60a8dc491
commit 3126a5e523
30 changed files with 1191 additions and 126 deletions
Download Patch File Download Diff File Expand all files Collapse all files

37
syntax/GoHighlighter.cc
View File

@@ -46,14 +46,45 @@ GoHighlighter::GoHighlighter()
void
GoHighlighter::HighlightLine(const Buffer &buf, int row, std::vector<HighlightSpan> &out) const
{
StatefulHighlighter::LineState prev;
(void) HighlightLineStateful(buf, row, prev, out);
}
StatefulHighlighter::LineState
GoHighlighter::HighlightLineStateful(const Buffer &buf,
int row,
const LineState &prev,
std::vector<HighlightSpan> &out) const
{
StatefulHighlighter::LineState state = prev;
if (row < 0 || static_cast<std::size_t>(row) >= buf.Nrows())
return;
return state;
std::string s = buf.GetLineString(static_cast<std::size_t>(row));
int n = static_cast<int>(s.size());
int i = 0;
int bol = 0;
while (bol < n && (s[bol] == ' ' || s[bol] == '\t'))
++bol;
// Continue a multi-line block comment from the previous line.
if (state.in_block_comment) {
int j = i;
while (i + 1 < n) {
if (s[i] == '*' && s[i + 1] == '/') {
i += 2;
push(out, j, i, TokenKind::Comment);
state.in_block_comment = false;
break;
}
++i;
}
if (state.in_block_comment) {
push(out, j, n, TokenKind::Comment);
return state;
}
}
// line comment
while (i < n) {
char c = s[i];
@@ -82,7 +113,8 @@ GoHighlighter::HighlightLine(const Buffer &buf, int row, std::vector<HighlightSp
}
if (!closed) {
push(out, i, n, TokenKind::Comment);
break;
state.in_block_comment = true;
return state;
} else {
push(out, i, j, TokenKind::Comment);
i = j;
@@ -152,5 +184,6 @@ GoHighlighter::HighlightLine(const Buffer &buf, int row, std::vector<HighlightSp
push(out, i, i + 1, TokenKind::Default);
++i;
}
return state;
}
} // namespace kte

7
syntax/GoHighlighter.h
View File

@@ -5,12 +5,17 @@
#include <unordered_set>
namespace kte {
class GoHighlighter final : public LanguageHighlighter {
class GoHighlighter final : public StatefulHighlighter {
public:
GoHighlighter();
void HighlightLine(const Buffer &buf, int row, std::vector<HighlightSpan> &out) const override;
LineState HighlightLineStateful(const Buffer &buf,
int row,
const LineState &prev,
std::vector<HighlightSpan> &out) const override;
private:
std::unordered_set<std::string> kws_;
std::unordered_set<std::string> types_;

28
syntax/HighlighterEngine.cc
View File

@@ -74,7 +74,22 @@ HighlighterEngine::GetLine(const Buffer &buf, int row, std::uint64_t buf_version
StatefulHighlighter::LineState prev_state;
int start_row = -1;
if (!state_cache_.empty()) {
// Fast path: state_last_contig_ tracks the highest row we've cached state
// for, per version. If that row is already below our target, it's
// necessarily the best anchor the O(n) scan below would have found, so we
// can skip the scan entirely. Re-validated against state_cache_ before
// use since InvalidateFrom()/SetHighlighter() may have raced/cleared it.
auto contig_it = state_last_contig_.find(buf_version);
if (contig_it != state_last_contig_.end() && contig_it->second < row) {
auto sc_it = state_cache_.find(contig_it->second);
if (sc_it != state_cache_.end() && sc_it->second.version == buf_version) {
start_row = contig_it->second;
prev_state = sc_it->second.state;
}
}
if (start_row < 0 && !state_cache_.empty()) {
// linear search over map (unordered), track best candidate
int best = -1;
for (const auto &kv: state_cache_) {
@@ -109,6 +124,9 @@ HighlighterEngine::GetLine(const Buffer &buf, int row, std::uint64_t buf_version
se.state = next_state;
state_cache_[r] = se;
cur_state = next_state;
int &contig = state_last_contig_[buf_version];
if (r > contig)
contig = r;
}
// Store in cache and return by value
@@ -139,6 +157,14 @@ HighlighterEngine::InvalidateFrom(int row)
++it;
}
}
// A version's tracked contiguous-state row is no longer valid once any
// row at or above it has been evicted from state_cache_ above.
for (auto it = state_last_contig_.begin(); it != state_last_contig_.end();) {
if (it->second >= row)
it = state_last_contig_.erase(it);
else
++it;
}
}

27
syntax/PythonHighlighter.cc
View File

@@ -55,31 +55,26 @@ PythonHighlighter::HighlightLineStateful(const Buffer &buf, int row, const LineS
std::string s = buf.GetLineString(static_cast<std::size_t>(row));
int n = static_cast<int>(s.size());
int i = 0;
// Triple-quoted string continuation uses in_raw_string with raw_delim either "'''" or "\"\"\""
if (state.in_raw_string && (state.raw_delim == "'''" || state.raw_delim == "\"\"\"")) {
auto pos = s.find(state.raw_delim);
if (pos == std::string::npos) {
push(out, 0, n, TokenKind::String);
return state; // still inside
} else {
int end = static_cast<int>(pos + static_cast<int>(state.raw_delim.size()));
push(out, 0, end, TokenKind::String);
// remainder processed normally
s = s.substr(end);
n = static_cast<int>(s.size());
state.in_raw_string = false;
state.raw_delim.clear();
// Continue parsing remainder as a separate small loop
int base = end;
// original offset, but we already emitted to 'out' with base=0; following spans should be from 'end'
// For simplicity, mark rest as Default
if (n > 0)
push(out, base, base + n, TokenKind::Default);
return state;
}
int end = static_cast<int>(pos + static_cast<int>(state.raw_delim.size()));
push(out, 0, end, TokenKind::String);
state.in_raw_string = false;
state.raw_delim.clear();
// Resume the normal tokenizer at the closing delimiter's end, on the
// same (unmodified) `s`/`n`, so anything after it - including a new
// triple-quoted string opening on this same line - is re-scanned
// instead of being dumped into a single opaque Default span.
i = end;
}
int i = 0;
// Detect comment start '#', ignoring inside strings
while (i < n) {
char c = s[i];

37
syntax/RustHighlighter.cc
View File

@@ -47,11 +47,42 @@ RustHighlighter::RustHighlighter()
void
RustHighlighter::HighlightLine(const Buffer &buf, int row, std::vector<HighlightSpan> &out) const
{
StatefulHighlighter::LineState prev;
(void) HighlightLineStateful(buf, row, prev, out);
}
StatefulHighlighter::LineState
RustHighlighter::HighlightLineStateful(const Buffer &buf,
int row,
const LineState &prev,
std::vector<HighlightSpan> &out) const
{
StatefulHighlighter::LineState state = prev;
if (row < 0 || static_cast<std::size_t>(row) >= buf.Nrows())
return;
return state;
std::string s = buf.GetLineString(static_cast<std::size_t>(row));
int n = static_cast<int>(s.size());
int i = 0;
// Continue a multi-line block comment from the previous line.
if (state.in_block_comment) {
int j = i;
while (i + 1 < n) {
if (s[i] == '*' && s[i + 1] == '/') {
i += 2;
push(out, j, i, TokenKind::Comment);
state.in_block_comment = false;
break;
}
++i;
}
if (state.in_block_comment) {
push(out, j, n, TokenKind::Comment);
return state;
}
}
while (i < n) {
char c = s[i];
if (c == ' ' || c == '\t') {
@@ -79,7 +110,8 @@ RustHighlighter::HighlightLine(const Buffer &buf, int row, std::vector<Highlight
}
if (!closed) {
push(out, i, n, TokenKind::Comment);
break;
state.in_block_comment = true;
return state;
} else {
push(out, i, j, TokenKind::Comment);
i = j;
@@ -140,5 +172,6 @@ RustHighlighter::HighlightLine(const Buffer &buf, int row, std::vector<Highlight
push(out, i, i + 1, TokenKind::Default);
++i;
}
return state;
}
} // namespace kte

7
syntax/RustHighlighter.h
View File

@@ -5,12 +5,17 @@
#include <unordered_set>
namespace kte {
class RustHighlighter final : public LanguageHighlighter {
class RustHighlighter final : public StatefulHighlighter {
public:
RustHighlighter();
void HighlightLine(const Buffer &buf, int row, std::vector<HighlightSpan> &out) const override;
LineState HighlightLineStateful(const Buffer &buf,
int row,
const LineState &prev,
std::vector<HighlightSpan> &out) const override;
private:
std::unordered_set<std::string> kws_;
std::unordered_set<std::string> types_;

38
syntax/SqlHighlighter.cc
View File

@@ -47,12 +47,42 @@ SqlHighlighter::SqlHighlighter()
void
SqlHighlighter::HighlightLine(const Buffer &buf, int row, std::vector<HighlightSpan> &out) const
{
StatefulHighlighter::LineState prev;
(void) HighlightLineStateful(buf, row, prev, out);
}
StatefulHighlighter::LineState
SqlHighlighter::HighlightLineStateful(const Buffer &buf,
int row,
const LineState &prev,
std::vector<HighlightSpan> &out) const
{
StatefulHighlighter::LineState state = prev;
if (row < 0 || static_cast<std::size_t>(row) >= buf.Nrows())
return;
return state;
std::string s = buf.GetLineString(static_cast<std::size_t>(row));
int n = static_cast<int>(s.size());
int i = 0;
// Continue a multi-line block comment from the previous line.
if (state.in_block_comment) {
int j = i;
while (i + 1 < n) {
if (s[i] == '*' && s[i + 1] == '/') {
i += 2;
push(out, j, i, TokenKind::Comment);
state.in_block_comment = false;
break;
}
++i;
}
if (state.in_block_comment) {
push(out, j, n, TokenKind::Comment);
return state;
}
}
while (i < n) {
char c = s[i];
if (c == ' ' || c == '\t') {
@@ -68,7 +98,7 @@ SqlHighlighter::HighlightLine(const Buffer &buf, int row, std::vector<HighlightS
push(out, i, n, TokenKind::Comment);
break;
}
// simple block comment on same line: /* ... */
// block comment: /* ... */ (may span multiple lines)
if (c == '/' && i + 1 < n && s[i + 1] == '*') {
int j = i + 2;
bool closed = false;
@@ -82,7 +112,8 @@ SqlHighlighter::HighlightLine(const Buffer &buf, int row, std::vector<HighlightS
}
if (!closed) {
push(out, i, n, TokenKind::Comment);
break;
state.in_block_comment = true;
return state;
} else {
push(out, i, j, TokenKind::Comment);
i = j;
@@ -151,5 +182,6 @@ SqlHighlighter::HighlightLine(const Buffer &buf, int row, std::vector<HighlightS
push(out, i, i + 1, TokenKind::Default);
++i;
}
return state;
}
} // namespace kte

7
syntax/SqlHighlighter.h
View File

@@ -5,12 +5,17 @@
#include <unordered_set>
namespace kte {
class SqlHighlighter final : public LanguageHighlighter {
class SqlHighlighter final : public StatefulHighlighter {
public:
SqlHighlighter();
void HighlightLine(const Buffer &buf, int row, std::vector<HighlightSpan> &out) const override;
LineState HighlightLineStateful(const Buffer &buf,
int row,
const LineState &prev,
std::vector<HighlightSpan> &out) const override;
private:
std::unordered_set<std::string> kws_;
std::unordered_set<std::string> types_;