kyle/kte
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Draw non-ASCII text whole in the GUIs (B7); one span sanitizer

Browse Source 
Highlighters emit one span per byte of a non-ASCII character, and the
ImGui and Qt renderers decoded each span on its own, so every such
character was drawn as replacement glyphs (reproduced under Xvfb:
"héllo" showed as "h??llo"). kte::SanitizeSpans (Highlight.h) widens
spans to whole UTF-8 characters and removes overlaps (the earlier span
wins), and replaces the three renderers' copies of the clamp/sort code.

The terminal renderer's token lookup rescanned the spans from the first
for every byte drawn; it now walks them with a cursor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015rcVQiiWyTzoNrsvLSaAfB
This commit is contained in:
Claude
2026-09-24 12:19:59 -07:00
committed by Kyle Isom
parent d490245f30
commit c5df5035fa
5 changed files with 95 additions and 87 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Highlight.h
+42
View File
@@ -1,7 +1,10 @@
// Highlight.h - core syntax highlighting types for kte
#pragma once
#include <algorithm>
#include <cstddef>
#include <cstdint>
#include <string_view>
#include <vector>
namespace kte {
@@ -34,4 +37,43 @@ struct LineHighlight {
std::vector<HighlightSpan> spans;
std::uint64_t version{0}; // buffer version used for this line
};
// A line's spans ready to draw: clamped to the line, ordered by start, not
// overlapping (the earlier span wins), and widened to whole UTF-8
// characters. Highlighters may emit one span per byte of a non-ASCII
// character; drawn separately, each byte showed as a replacement glyph.
inline void
SanitizeSpans(std::string_view line, const std::vector<HighlightSpan> &in, std::vector<HighlightSpan> &out)
{
out.clear();
const long len = static_cast<long>(line.size());
for (const HighlightSpan &sp: in) {
long s = std::clamp<long>(std::min(sp.col_start, sp.col_end), 0, len);
long e = std::clamp<long>(std::max(sp.col_start, sp.col_end), 0, len);
if (e > s)
out.push_back(HighlightSpan{static_cast<int>(s), static_cast<int>(e), sp.kind});
}
std::stable_sort(out.begin(), out.end(), [](const HighlightSpan &a, const HighlightSpan &b) {
return a.col_start < b.col_start;
});
auto continuation = [&](long i) {
return i < len && (static_cast<unsigned char>(line[static_cast<std::size_t>(i)]) & 0xC0) == 0x80;
};
std::size_t w = 0;
long prev_end = 0;
for (const HighlightSpan &sp: out) {
long s = sp.col_start, e = sp.col_end;
while (s > 0 && continuation(s))
--s;
while (e < len && continuation(e))
++e;
s = std::max(s, prev_end);
if (e <= s)
continue;
out[w++] = HighlightSpan{static_cast<int>(s), static_cast<int>(e), sp.kind};
prev_end = e;
}
out.resize(w);
}
} // namespace kte