mirror of
https://github.com/pewdiepie-archdaemon/odysseus.git
synced 2026-09-10 10:12:20 +02:00
Léo
d0bf771f9d
* fix(static): vendor KaTeX and Mermaid instead of loading them from a CDN
index.html pulled katex.min.{js,css} and mermaid.min.js from cdn.jsdelivr.net on
every page load. For self-hosted software that is three problems at once: an
air-gapped or offline install renders no math and no diagrams at all, every
session announces its IP, User-Agent and Referer to a third party, and the "runs
on your own hardware" promise quietly isn't true.
static/lib/ already vendors highlight.js, docx, xlsx, mammoth, html2pdf and
qrcode, so the CDN usage was an inconsistency rather than a policy. Vendoring
also pins Mermaid, which was floating on the `11` tag, to 11.16.1.
Behaviour is unchanged: both libraries still load eagerly from <head>, just from
this machine.
- KaTeX goes in its own directory because its stylesheet resolves fonts with a
relative url(fonts/...), so the vendored CSS needs no rewrite. Only the .woff2
variants ship, matching static/fonts/, since a browser that supports woff2
never requests the .woff/.ttf alternatives the stylesheet also lists.
- The service worker precaches KaTeX and its fonts so offline math is typeset
rather than falling back to system glyphs, and CACHE_NAME is bumped. Mermaid
is left to the existing cache-first rule: at 3.5 MB, precaching it would mean
re-downloading it on every cache bump for a library most sessions never touch.
- Licence texts travel with the bundles in licenses/, following the convention
the repo already uses for OpenDyslexic and DeepResearch.
- .gitattributes turns the whitespace check off for static/lib/ so `git diff
--check` passes without stripping bytes from the published npm artifacts,
which would desync them from upstream.
* perf(markdown): load KaTeX and Mermaid on first use, not on every page load
Both libraries loaded eagerly from <head>, costing every session ~985 KB on the
wire (929 KB of that Mermaid) even though most chats contain neither a formula
nor a diagram. Measured on a cold profile via the Resource Timing API: JS bytes
per page load drop from 3,102,141 to 2,098,634, a saving of 1,003,507 bytes, and
third-party requests per load go from 3 to 0.
markdown.js now fetches each library the first time one is actually needed:
- renderMermaid() checks for an unprocessed mermaid fence before touching the
network, and re-queries the DOM after the load so a diagram replaced mid-stream
still renders.
- mdToHtml() is synchronous, so when KaTeX is not in yet it banks the math source
in an inert placeholder and schedules a flush that loads the library and swaps
the placeholders in. Once KaTeX is loaded it typesets inline exactly as before,
so callers that never call a render helper still get their math.
Both loaders memoise the promise rather than the module, so concurrent callers
share one fetch and a double trigger cannot start two loads; a failed load clears
the memo so the next formula retries instead of being poisoned for the session.
The flush is scheduled with setTimeout rather than requestAnimationFrame, which
is throttled to a stop in a background tab and never fires at all in a headless
browser, so math would have sat as plain source text until the tab was focused.
If neither library ever loads, math degrades to readable source text and diagrams
to their fence contents, rather than to nothing.
* fix(markdown): unescape & last so math entities survive intact
The math pass unescaped & before < and >. mdToHtml escapes the source
first, so a literal "<" typed inside a formula arrives here as "&lt;",
turns back into "<" on the ampersand pass, and is then eaten by the very next
one. Typing $a < b$ rendered as "a < b" instead of the literal text.
The code-block pass in the same function already unescapes & last; only the
math paths were the outlier, in all four of the copies this branch consolidated
into pushMath(). Reordering to match makes them consistent and clears the
js/double-escaping alert CodeQL raised on this PR.
Math containing a genuinely typed "<" is unaffected, which is why this went
unnoticed for so long. Covered by a regression test asserting both cases.
* fix(markdown): decode entity-spelled math in one pass
mdToHtml escapes the source before the math pass, so a typed "<" reaches
the delimiters as "<" and a typed "<" reaches them as "&lt;".
KaTeX has no entity syntax and reads the leftover "&" as an alignment
marker, so "$a < b$" rendered as a red .katex-error instead of a
formula, on both the inline and the deferred path.
Chained replaces cannot fix it in either order: unescaping "&" first
lets the next pass eat the "<" it just wrote, and unescaping it last
leaves the entity spelling for KaTeX to choke on. One alternation,
longest form first, decodes every spelling and never rescans its own
output.
The tests now drive the vendored KaTeX build rather than a renderer that
echoes its input, which is why the old assertion looked correct.
* fix(document): typeset deferred math before the PDF export
exportAsPdf() renders the document into a detached container and hands
it straight to html2pdf. On a page where KaTeX has not loaded yet,
mdToHtml() returns pending placeholders and schedules a flush scoped to
document, which never reaches a node that was never attached, so the
PDF printed raw formula source.
Render the container's own math first. renderMath() returns immediately
without fetching anything when there is nothing pending, so a document
with no formulas still exports without pulling KaTeX.
1230 lines
51 KiB
JavaScript
1230 lines
51 KiB
JavaScript
// static/js/markdown.js
|
||
|
||
/**
|
||
* Markdown rendering and content processing utilities
|
||
*/
|
||
|
||
import uiModule from './ui.js';
|
||
import { splitTableRow } from './markdown/tableRow.js';
|
||
import { replaceEmojiShortcodes, hasEmojiShortcode } from './emojiShortcodes.js';
|
||
|
||
var escapeHtml = uiModule.esc;
|
||
|
||
// Mermaid and KaTeX are vendored under /static/lib and fetched on first use.
|
||
// Loading them from <head> cost every session ~985 KB on the wire even though
|
||
// most chats never contain a diagram or a formula. Both loaders memoise the
|
||
// *promise* rather than the resolved library, so concurrent callers share one
|
||
// fetch and a double trigger cannot start two loads. A failed load clears the
|
||
// memo so the next diagram/formula retries instead of being poisoned forever.
|
||
const MERMAID_SRC = '/static/lib/mermaid.min.js';
|
||
const KATEX_SRC = '/static/lib/katex/katex.min.js';
|
||
const KATEX_CSS = '/static/lib/katex/katex.min.css';
|
||
// Marks math emitted before KaTeX finished loading; renderMath() swaps these
|
||
// for typeset output. The source stays as readable text inside the span, so a
|
||
// load that never completes degrades to plain text rather than to nothing.
|
||
const MATH_PENDING_CLASS = 'ody-math-pending';
|
||
|
||
// KaTeX has no entity syntax: it reads a bare "&" as an alignment marker and
|
||
// errors out on anything that is not a valid column break, so "a < b" comes
|
||
// back as a red .katex-error instead of a formula. mdToHtml escapes the whole
|
||
// string before the math pass, which leaves two spellings of the same
|
||
// character at the delimiters — a typed "<" arrives as "<", while a typed
|
||
// "<" arrives as "&lt;" — and both have to reach KaTeX as "<".
|
||
//
|
||
// One alternation, longest form first, so nothing this writes is scanned
|
||
// again. Chained .replace() calls cannot do it: unescaping "&" first lets
|
||
// the next pass eat the "<" it just produced (the double-unescape CodeQL
|
||
// flags), and unescaping it last leaves the entity spelling intact and breaks
|
||
// the render. The code-block pass upstream keeps its chained order on purpose
|
||
// — Markdown does not decode entities inside code, so "<" there is meant to
|
||
// stay visible.
|
||
const MATH_SOURCE_ENTITY_RE = /&(?:lt|gt|amp|quot|#39);|<|>|&/g;
|
||
const MATH_SOURCE_ENTITIES = {
|
||
'&lt;': '<',
|
||
'&gt;': '>',
|
||
'&amp;': '&',
|
||
'&quot;': '"',
|
||
'&#39;': "'",
|
||
'<': '<',
|
||
'>': '>',
|
||
'&': '&',
|
||
};
|
||
|
||
function decodeMathSource(text) {
|
||
return String(text).replace(MATH_SOURCE_ENTITY_RE, (entity) => MATH_SOURCE_ENTITIES[entity]);
|
||
}
|
||
|
||
let _mermaidPromise = null;
|
||
let _katexPromise = null;
|
||
let _mathFlushScheduled = false;
|
||
|
||
function _loadScript(src) {
|
||
return new Promise((resolve, reject) => {
|
||
const script = document.createElement('script');
|
||
script.src = src;
|
||
script.addEventListener('load', () => resolve(), { once: true });
|
||
script.addEventListener('error', () => reject(new Error('Failed to load ' + src)), { once: true });
|
||
document.head.appendChild(script);
|
||
});
|
||
}
|
||
|
||
function _loadStylesheet(href) {
|
||
// Resolves either way: without the stylesheet KaTeX still produces correct
|
||
// markup, just unstyled, which beats failing the whole math render.
|
||
return new Promise((resolve) => {
|
||
const link = document.createElement('link');
|
||
link.rel = 'stylesheet';
|
||
link.href = href;
|
||
link.addEventListener('load', () => resolve(), { once: true });
|
||
link.addEventListener('error', () => resolve(), { once: true });
|
||
document.head.appendChild(link);
|
||
});
|
||
}
|
||
|
||
/**
|
||
* Load Mermaid on first use and initialize it once.
|
||
*/
|
||
export function ensureMermaid() {
|
||
return (_mermaidPromise ??= _loadScript(MERMAID_SRC)
|
||
.then(() => {
|
||
if (!window.mermaid) throw new Error('mermaid global missing after load');
|
||
window.mermaid.initialize({ startOnLoad: false, theme: 'dark', securityLevel: 'loose' });
|
||
return window.mermaid;
|
||
})
|
||
.catch((err) => {
|
||
_mermaidPromise = null;
|
||
throw err;
|
||
}));
|
||
}
|
||
|
||
/**
|
||
* Load KaTeX (script + stylesheet) on first use.
|
||
*/
|
||
export function ensureKatex() {
|
||
return (_katexPromise ??= Promise.all([_loadScript(KATEX_SRC), _loadStylesheet(KATEX_CSS)])
|
||
.then(() => {
|
||
if (!window.katex) throw new Error('katex global missing after load');
|
||
return window.katex;
|
||
})
|
||
.catch((err) => {
|
||
_katexPromise = null;
|
||
throw err;
|
||
}));
|
||
}
|
||
|
||
// mdToHtml() is synchronous and its callers insert the returned string into the
|
||
// DOM themselves, so the placeholders are usually not attached yet when this
|
||
// fires. Loading first and scanning afterwards covers that gap: by the time
|
||
// KaTeX is in, the caller's innerHTML assignment has long since happened.
|
||
//
|
||
// setTimeout, not requestAnimationFrame: this has nothing to do with paint, and
|
||
// rAF is throttled to a stop in a background tab (and never fires at all in a
|
||
// headless browser), which would leave math untypeset until the tab is focused.
|
||
function _scheduleMathFlush() {
|
||
if (_mathFlushScheduled) return;
|
||
_mathFlushScheduled = true;
|
||
setTimeout(() => {
|
||
_mathFlushScheduled = false;
|
||
ensureKatex()
|
||
.then(() => renderMath(document))
|
||
.catch((e) => console.warn('KaTeX load error:', e));
|
||
}, 0);
|
||
}
|
||
|
||
function safeLinkUrl(rawUrl) {
|
||
const url = String(rawUrl || '').trim();
|
||
if (url.startsWith('#')) {
|
||
return /^#[A-Za-z0-9_-]*$/.test(url) ? url : '';
|
||
}
|
||
try {
|
||
const parsed = new URL(url, window.location.origin);
|
||
if (parsed.protocol === 'http:' || parsed.protocol === 'https:') {
|
||
return parsed.href;
|
||
}
|
||
} catch (_) {
|
||
return '';
|
||
}
|
||
return '';
|
||
}
|
||
|
||
function linkHtml(text, url) {
|
||
const safeUrl = safeLinkUrl(url);
|
||
const safeText = escapeHtml(text);
|
||
if (!safeUrl) return safeText;
|
||
if (safeUrl.startsWith('#')) {
|
||
return `<a href="${safeUrl}" class="chat-link">${safeText}</a>`;
|
||
}
|
||
return `<a href="${escapeHtml(safeUrl)}" target="_blank" rel="noopener noreferrer">${safeText}</a>`;
|
||
}
|
||
|
||
function imageHtml(alt, url, title) {
|
||
const safeUrl = safeLinkUrl(url);
|
||
if (!safeUrl || safeUrl.startsWith('#')) return escapeHtml(alt || '');
|
||
const safeAlt = escapeHtml(alt || '');
|
||
const safeTitle = title ? ` title="${escapeHtml(title)}"` : '';
|
||
return `<img src="${escapeHtml(safeUrl)}" alt="${safeAlt}"${safeTitle} loading="lazy" decoding="async">`;
|
||
}
|
||
|
||
function _isModelEndpointUrl(rawUrl) {
|
||
try {
|
||
const parsed = new URL(String(rawUrl || ''), window.location.origin);
|
||
if (parsed.protocol !== 'http:' && parsed.protocol !== 'https:') return false;
|
||
const path = parsed.pathname.replace(/\/+$/, '');
|
||
return path === '/v1';
|
||
} catch (_) {
|
||
return false;
|
||
}
|
||
}
|
||
|
||
/**
|
||
* Sanitize the raw-HTML fragments that mdToHtml deliberately preserves from
|
||
* the source text — <details> blocks (collapsible agent output) and <a> tags
|
||
* (emitted by the markdown link pass). Those fragments are later restored
|
||
* verbatim into innerHTML, so without scrubbing them a model — or any content
|
||
* routed through here — could smuggle in an `<img onerror=...>`, an
|
||
* `<a href="javascript:...">`, an `onmouseover=` handler, etc. and execute
|
||
* script in the authenticated page (DOM XSS).
|
||
*
|
||
* Parsing into a <template> is inert: assigning to template.innerHTML neither
|
||
* fetches resources nor runs scripts, so we can walk the resulting tree,
|
||
* drop script-capable elements, and strip event-handler attributes and
|
||
* dangerous URL schemes before the (now safe) fragment is handed back.
|
||
*/
|
||
const _ALLOWED_HTML_BAD_TAGS = new Set([
|
||
'SCRIPT', 'IFRAME', 'OBJECT', 'EMBED', 'LINK', 'META',
|
||
'STYLE', 'BASE', 'FORM', 'NOSCRIPT', 'TEMPLATE',
|
||
// Foreign-content roots. SVG/MathML have their own parser rules and are a
|
||
// classic mutation-XSS vehicle — e.g. an SVG-namespaced <script>, whose
|
||
// `tagName` is the lower-case 'script' and would slip a name check that
|
||
// assumed HTML's upper-casing. They aren't needed in the <details>/<a>
|
||
// fragments we preserve, so drop the whole subtree.
|
||
'SVG', 'MATH',
|
||
]);
|
||
const _ALLOWED_HTML_URL_ATTRS = new Set([
|
||
'href', 'src', 'srcset', 'xlink:href', 'action', 'formaction', 'background', 'poster',
|
||
]);
|
||
|
||
function _compactUrlSchemeValue(value) {
|
||
return String(value || '').replace(/[\u0000-\u0020\u007f-\u009f]+/g, '').toLowerCase();
|
||
}
|
||
|
||
function _isDangerousUrl(value) {
|
||
return /^(javascript|vbscript|data):/.test(_compactUrlSchemeValue(value));
|
||
}
|
||
|
||
function _isDangerousSrcset(value) {
|
||
return String(value || '').split(',').some(candidate => _isDangerousUrl(candidate));
|
||
}
|
||
|
||
function _cleanAllowedHtmlOnce(htmlString) {
|
||
const tpl = document.createElement('template');
|
||
tpl.innerHTML = htmlString;
|
||
for (const el of Array.from(tpl.content.querySelectorAll('*'))) {
|
||
// Upper-case the tag for comparison: HTML tagNames are upper-case, but
|
||
// SVG/MathML elements preserve their original (lower/camel) case, so a
|
||
// raw `Set.has(el.tagName)` would miss e.g. a namespaced <script>.
|
||
if (_ALLOWED_HTML_BAD_TAGS.has(el.tagName.toUpperCase())) {
|
||
el.remove();
|
||
continue;
|
||
}
|
||
for (const attr of Array.from(el.attributes)) {
|
||
const name = attr.name.toLowerCase();
|
||
// Drop every inline event handler (onerror, onclick, onmouseover, ...)
|
||
// and srcdoc (a frame-less script vector).
|
||
if (name.startsWith('on') || name === 'srcdoc') {
|
||
el.removeAttribute(attr.name);
|
||
continue;
|
||
}
|
||
if (name === 'style') {
|
||
const value = _compactUrlSchemeValue(attr.value);
|
||
if (/javascript:|vbscript:|data:|expression\(/.test(value)) {
|
||
el.removeAttribute(attr.name);
|
||
}
|
||
continue;
|
||
}
|
||
// Neutralize javascript:/vbscript:/data: in URL-bearing attributes.
|
||
// Strip control/space chars first so e.g. "java\tscript:" can't slip by.
|
||
if (_ALLOWED_HTML_URL_ATTRS.has(name)) {
|
||
if (name === 'srcset' ? _isDangerousSrcset(attr.value) : _isDangerousUrl(attr.value)) {
|
||
el.removeAttribute(attr.name);
|
||
}
|
||
}
|
||
}
|
||
}
|
||
return tpl.innerHTML;
|
||
}
|
||
|
||
export function sanitizeAllowedHtml(html) {
|
||
const raw = String(html == null ? '' : html);
|
||
// Non-browser context (e.g. a future SSR/Node import): fail closed by
|
||
// escaping rather than trusting the markup.
|
||
if (typeof document === 'undefined') return escapeHtml(raw);
|
||
|
||
// Sanitize to a fixpoint. Re-parsing the serialized output can mutate the
|
||
// tree (the basis of mutation-XSS), so re-clean until it stops changing.
|
||
let out = raw;
|
||
for (let i = 0; i < 4; i++) {
|
||
const next = _cleanAllowedHtmlOnce(out);
|
||
if (next === out) break;
|
||
out = next;
|
||
}
|
||
return out;
|
||
}
|
||
|
||
/**
|
||
* Check if text has unclosed think tag
|
||
*/
|
||
export function hasUnclosedThinkTag(text) {
|
||
text = normalizeThinkingMarkup(text || '');
|
||
const openCount =
|
||
(text.match(/<(?:think(?:ing)?|thought)(?:\s+[^>]*)?>/gi) || []).length
|
||
+ (text.match(/<\|channel>thought/gi) || []).length;
|
||
const closeCount =
|
||
(text.match(/<\/(?:think(?:ing)?|thought)>/gi) || []).length
|
||
+ (text.match(/<channel\|>/gi) || []).length;
|
||
return openCount > closeCount;
|
||
}
|
||
|
||
export function startsWithReasoningPrefix(text) {
|
||
return /^\s*(?:thinking(?:\s+process)?\s*:|the user |user wants|we need |i need |i should |i will |i'll |i am going |let me (?:think|look|see|check|read|review|analyze|parse|figure|draft|write)|they are |the question |i can )/i.test(text || '');
|
||
}
|
||
|
||
export function normalizeThinkingMarkup(text) {
|
||
if (!text) return text;
|
||
let normalized = text;
|
||
// MiniMax M-series can emit namespaced reasoning tags like
|
||
// <mm:think>...</mm:think>. Normalize them into the shared thinking parser.
|
||
normalized = normalized.replace(/<mm:think(\s+[^>]*)?>/gi, (_m, attrs = '') => `<think${attrs || ''}>`);
|
||
normalized = normalized.replace(/<\/mm:think>/gi, '</think>');
|
||
normalized = normalized.replace(/<thought(\s+[^>]*)?>/gi, (_m, attrs = '') => `<think${attrs || ''}>`);
|
||
normalized = normalized.replace(/<\/thought>/gi, '</think>');
|
||
normalized = normalized.replace(/<\|channel>thought\s*\n?([\s\S]*?)<channel\|>\s*/gi, (_m, content = '') => {
|
||
const thought = String(content || '').trim();
|
||
return thought ? `<think>${thought}</think>\n` : '';
|
||
});
|
||
normalized = normalized.replace(/<\|channel>response\s*\n?([\s\S]*?)<channel\|>/gi, (_m, content = '') => content || '');
|
||
normalized = normalized.replace(/<\|channel>response\s*\n?/gi, '');
|
||
normalized = normalized.replace(/<channel\|>/gi, '');
|
||
return normalized;
|
||
}
|
||
|
||
function normalizePlainThinking(text) {
|
||
if (!text) return text;
|
||
text = normalizeThinkingMarkup(text);
|
||
if (/<think/i.test(text)) return text;
|
||
|
||
const trimmed = text.trimStart();
|
||
if (!startsWithReasoningPrefix(trimmed)) return text;
|
||
|
||
const replyStarts = [
|
||
'Hey', 'Hi ', 'Hi!', 'Hello', 'Sure', 'Yes', 'No ', 'No,', 'Yo', 'OK',
|
||
'Here', 'Absolutely', 'Of course', 'Great', 'Alright', 'Thanks', 'Welcome',
|
||
'Good ', "I'm happy", "I'd be"
|
||
];
|
||
const prefixRegex = /^(thinking(?:\s+process)?\s*:)\s*/i;
|
||
const escapedReplyStarts = replyStarts.map((value) => value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'));
|
||
const boundaryRegex = new RegExp(
|
||
`^([\\s\\S]*?)(\\n\\n(?=${escapedReplyStarts.join('|')}|I |What|Let|This |As ))[\\s\\S]*$`,
|
||
'i'
|
||
);
|
||
const boundaryMatch = boundaryRegex.exec(trimmed);
|
||
|
||
if (boundaryMatch) {
|
||
const thinkBlock = boundaryMatch[1].replace(prefixRegex, '').trim();
|
||
const reply = trimmed.slice(boundaryMatch[1].length).trimStart();
|
||
if (thinkBlock && reply) return `<think>${thinkBlock}</think>\n\n${reply}`;
|
||
}
|
||
|
||
const lines = trimmed.split('\n');
|
||
for (let index = 1; index < lines.length; index += 1) {
|
||
const line = lines[index].trim();
|
||
if (!line) continue;
|
||
if (replyStarts.some((prefix) => line.startsWith(prefix))) {
|
||
const thinkBlock = lines.slice(0, index).join('\n').replace(prefixRegex, '').trim();
|
||
const reply = lines.slice(index).join('\n').trim();
|
||
if (thinkBlock && reply) return `<think>${thinkBlock}</think>\n${reply}`;
|
||
}
|
||
}
|
||
|
||
const withoutPrefix = trimmed.replace(prefixRegex, '');
|
||
for (const prefix of replyStarts) {
|
||
const rx = new RegExp(`[.!?]\\s*(${prefix.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')})`);
|
||
const match = rx.exec(withoutPrefix);
|
||
if (match && match.index > 20) {
|
||
const thinkBlock = withoutPrefix.slice(0, match.index + 1).trim();
|
||
const reply = withoutPrefix.slice(match.index + 1).trim();
|
||
if (thinkBlock && reply) return `<think>${thinkBlock}</think>\n${reply}`;
|
||
}
|
||
}
|
||
|
||
if (/^\s*(?:thinking(?:\s+process)?\s*:|the user |user wants|we need |let me (?:think|look|see|check|read|review|analyze|parse|figure|draft|write)|i need to |i should |i will |i'll |i am going )/i.test(trimmed)) {
|
||
const thinkBlock = withoutPrefix.trim();
|
||
if (thinkBlock) return `<think>${thinkBlock}</think>`;
|
||
}
|
||
|
||
return text;
|
||
}
|
||
|
||
/**
|
||
* Extract all complete thinking blocks and remaining content
|
||
*/
|
||
export function extractThinkingBlocks(text) {
|
||
// Handle malformed patterns: <think></think>\n...actual thinking...\n</think>
|
||
// Some models emit an empty <think></think> then put thinking text outside,
|
||
// closed by a second orphaned </think>.
|
||
let normalized = normalizePlainThinking(text);
|
||
// Collapse <think>short</think>...real thinking...</think> into one block
|
||
// Models sometimes emit a trivial first block then continue thinking outside tags
|
||
normalized = normalized.replace(/<think(?:ing)?(?:\s+[^>]*)?>.{0,30}<\/think(?:ing)?>\s*([\s\S]*?)<\/think(?:ing)?>/gi, (m, content) => {
|
||
return '<think>' + content.trim() + '</think>';
|
||
});
|
||
|
||
// Merge consecutive <think> blocks (some models split thinking across multiple tags)
|
||
normalized = normalized.replace(/<\/think(?:ing)?>\s*<think(?:ing)?(?:\s+[^>]*)?>/gi, '\n\n');
|
||
|
||
// Extract thinking time attribute if present
|
||
const timeMatch = normalized.match(/<think(?:ing)?\s+time="([\d.]+)"/i);
|
||
const thinkingTime = timeMatch ? timeMatch[1] : null;
|
||
// Strip time attribute for content extraction
|
||
normalized = normalized.replace(/<think(?:ing)?\s+time="[\d.]+"/gi, '<think');
|
||
|
||
const thinkRegex = /<think(?:ing)?(?:\s+[^>]*)?>([\s\S]*?)<\/think(?:ing)?>/gi;
|
||
const thinkingBlocks = [];
|
||
let match;
|
||
|
||
// Extract all complete thinking blocks
|
||
while ((match = thinkRegex.exec(normalized)) !== null) {
|
||
const content = match[1].trim();
|
||
if (content) thinkingBlocks.push(content);
|
||
}
|
||
|
||
// Remove all complete <think>/<thinking> blocks
|
||
let cleanContent = normalized.replace(thinkRegex, '');
|
||
|
||
// If there's an unclosed tag, decide between two cases:
|
||
// (a) Stray opener at the very start with no real reply before it — typical
|
||
// of quantized models (MiniMax-AWQ) that emit a literal `<think>` token
|
||
// at the start of every reply without ever closing it. Strip just the
|
||
// opener and keep the body as the reply, otherwise the bubble looks
|
||
// blank on reload (the body was being treated as collapsed thinking).
|
||
// (b) Cut-off mid-generation — there's already real reply text before the
|
||
// opener. Drop from the tag onward as before (it's truncated thinking).
|
||
if (hasUnclosedThinkTag(normalized)) {
|
||
const gemmaThoughtStart = cleanContent.search(/<\|channel>thought/i);
|
||
if (gemmaThoughtStart >= 0) {
|
||
const leakedThought = cleanContent
|
||
.slice(gemmaThoughtStart)
|
||
.replace(/^<\|channel>thought\s*\n?/i, '')
|
||
.trim();
|
||
if (gemmaThoughtStart === 0 && leakedThought) thinkingBlocks.push(leakedThought);
|
||
cleanContent = cleanContent.slice(0, gemmaThoughtStart);
|
||
} else {
|
||
const strayOpener = cleanContent.match(/^\s*<think(?:ing)?(?:\s+[^>]*)?>([\s\S]*)$/i);
|
||
if (strayOpener) {
|
||
cleanContent = strayOpener[1];
|
||
} else {
|
||
cleanContent = cleanContent.replace(/<think(?:ing)?(?:\s+[^>]*)?>[\s\S]*$/gi, '');
|
||
}
|
||
}
|
||
}
|
||
|
||
// Handle orphaned </think> with no opening tag — text before it is leaked thinking
|
||
const orphanMatch = cleanContent.match(/^([\s\S]+?)<\/think(?:ing)?>/i);
|
||
if (orphanMatch && orphanMatch[1].trim()) {
|
||
thinkingBlocks.push(orphanMatch[1].trim());
|
||
cleanContent = cleanContent.slice(orphanMatch[0].length);
|
||
}
|
||
|
||
// Strip any remaining orphaned closing tags
|
||
cleanContent = cleanContent.replace(/<\/think(?:ing)?>/gi, '');
|
||
|
||
// Merge all thinking blocks into one — no reason to show multiple dropdowns
|
||
const mergedBlocks = thinkingBlocks.length > 1
|
||
? [thinkingBlocks.join('\n\n')]
|
||
: thinkingBlocks;
|
||
|
||
return {
|
||
thinkingBlocks: mergedBlocks,
|
||
content: cleanContent.trim(),
|
||
thinkingTime,
|
||
};
|
||
}
|
||
|
||
/**
|
||
* Create a collapsible thinking section
|
||
*/
|
||
function createThinkingSection(thinkingContent, index = 0, thinkingTime = null) {
|
||
const id = `thinking-${Date.now()}-${index}`;
|
||
const timeHtml = thinkingTime ? `<span style="font-size:11px;opacity:0.4;font-variant-numeric:tabular-nums;">${thinkingTime}s</span>` : '';
|
||
return `
|
||
<div class="thinking-section">
|
||
<div class="thinking-header" data-thinking-id="${id}">
|
||
<div class="thinking-header-left">
|
||
<span>View thinking process</span>
|
||
</div>
|
||
<div style="display:flex;align-items:center;gap:6px;">
|
||
${timeHtml}
|
||
<span class="thinking-toggle" id="${id}-toggle"></span>
|
||
</div>
|
||
</div>
|
||
<div class="thinking-content" id="${id}">
|
||
<div class="thinking-content-inner">
|
||
${mdToHtml(thinkingContent)}
|
||
</div>
|
||
</div>
|
||
</div>
|
||
`;
|
||
}
|
||
|
||
function createTaskCompletedMarker() {
|
||
return `
|
||
<div class="task-completed-marker" role="status" aria-label="Task completed">
|
||
<span class="task-completed-icon" aria-hidden="true">
|
||
<svg viewBox="0 0 24 24" width="14" height="14" fill="none" stroke="currentColor" stroke-width="2.6" stroke-linecap="round" stroke-linejoin="round"><polyline points="20 6 9 17 4 12"/></svg>
|
||
</span>
|
||
<span>Task completed</span>
|
||
</div>
|
||
`;
|
||
}
|
||
|
||
/**
|
||
* Process text and render with thinking sections
|
||
*/
|
||
// ── Emoji → monochrome SVG (OpenMoji-black via same-origin /api/emoji proxy) ──
|
||
// Replace colorful system/Twemoji emoji with single-color line icons tinted to
|
||
// the surrounding text color (project rule: never colorful emoji). Operates on
|
||
// rendered HTML: only touches text outside tags and skips <code>/<pre>.
|
||
const _EMOJI_RE = /\p{Extended_Pictographic}/u;
|
||
const _emojiSeg = (typeof Intl !== 'undefined' && Intl.Segmenter)
|
||
? new Intl.Segmenter(undefined, { granularity: 'grapheme' }) : null;
|
||
|
||
function _emojiCodepoints(emoji) {
|
||
// Twemoji filename rule: strip U+FE0F unless the sequence has a ZWJ (U+200D).
|
||
const s = emoji.indexOf('') >= 0 ? emoji : emoji.replace(/️/g, '');
|
||
const cps = [];
|
||
for (const ch of s) { const c = ch.codePointAt(0); if (c) cps.push(c.toString(16)); }
|
||
return cps.join('-');
|
||
}
|
||
function _emojiImg(emoji) {
|
||
const code = _emojiCodepoints(emoji);
|
||
if (!code) return emoji;
|
||
// Monochrome line icon: the OpenMoji black SVG is used as a CSS mask filled
|
||
// with the surrounding text color (currentColor), so emoji render as a single
|
||
// theme-tinted line glyph — never colorful (project rule). If the proxy can't
|
||
// supply the glyph it returns a transparent SVG, so the mask shows nothing.
|
||
return `<span class="emoji" role="img" aria-label="${emoji}" style="--em:url('/api/emoji/${code}.svg')"></span>`;
|
||
}
|
||
function _svgifyText(text) {
|
||
if (!_emojiSeg) return text;
|
||
let out = '';
|
||
for (const { segment } of _emojiSeg.segment(text)) {
|
||
out += _EMOJI_RE.test(segment) ? _emojiImg(segment) : segment;
|
||
}
|
||
return out;
|
||
}
|
||
/** When "Text-only Emojis" is on, keep Unicode in HTML so deEmojify() can strip them. */
|
||
function _useSvgEmoji() {
|
||
return typeof document === 'undefined' || !document.body?.classList.contains('text-emojis');
|
||
}
|
||
|
||
// `opts.shortcodes` (default true) controls the issue-#345 `:name:` → emoji
|
||
// expansion. Chat passes it through as true; document/email body renderers pass
|
||
// false so author-typed `:shortcode:` text stays literal (see mdToHtml callers).
|
||
// The Unicode-emoji → monochrome-SVG pass always runs regardless, so a real 😀
|
||
// in a document still renders as the themed line icon as it always has.
|
||
export function svgifyEmoji(html, opts) {
|
||
if (!_useSvgEmoji() || !html) return html;
|
||
const allowShortcodes = !opts || opts.shortcodes !== false;
|
||
// Two reasons to walk the HTML: real Unicode emoji to turn into SVG icons,
|
||
// or `:shortcode:` text the model emitted instead of an emoji (issue #345).
|
||
const hasUnicode = _EMOJI_RE.test(html);
|
||
const hasShortcode = allowShortcodes && hasEmojiShortcode(html);
|
||
if (!hasUnicode && !hasShortcode) return html;
|
||
const parts = html.split(/(<[^>]*>)/); // odd indices = tags
|
||
let codeDepth = 0;
|
||
for (let i = 0; i < parts.length; i++) {
|
||
if (i % 2 === 1) {
|
||
const t = parts[i].toLowerCase();
|
||
if (/^<(pre|code)[\s>]/.test(t)) codeDepth++;
|
||
else if (/^<\/(pre|code)\s*>/.test(t)) codeDepth = Math.max(0, codeDepth - 1);
|
||
continue;
|
||
}
|
||
if (codeDepth !== 0) continue;
|
||
let seg = parts[i];
|
||
// Expand shortcodes to Unicode first, then both they and any pre-existing
|
||
// Unicode emoji get rendered as the same monochrome line icons below.
|
||
if (hasShortcode) seg = replaceEmojiShortcodes(seg);
|
||
if (_EMOJI_RE.test(seg)) seg = _svgifyText(seg);
|
||
parts[i] = seg;
|
||
}
|
||
return parts.join('');
|
||
}
|
||
/**
|
||
* Generic collapsible section that reuses the thinking-dropdown styling and its
|
||
* delegated toggle (any `.thinking-header[data-thinking-id]`). The label drives
|
||
* the "View <label>" / "Hide <label>" text via data-label. Used e.g. for the
|
||
* vision-model image description on a user's photo message.
|
||
*/
|
||
export function createCollapsible(contentMarkdown, label = 'details') {
|
||
const id = `collapse-${Date.now()}-${Math.floor(Math.random() * 1e6)}`;
|
||
const safeLabel = escapeHtml(label);
|
||
return `
|
||
<div class="thinking-section">
|
||
<div class="thinking-header" data-thinking-id="${id}">
|
||
<div class="thinking-header-left"><span data-label="${safeLabel}">View ${safeLabel}</span></div>
|
||
<div style="display:flex;align-items:center;gap:6px;"><span class="thinking-toggle" id="${id}-toggle"></span></div>
|
||
</div>
|
||
<div class="thinking-content" id="${id}"><div class="thinking-content-inner">${mdToHtml(contentMarkdown)}</div></div>
|
||
</div>`;
|
||
}
|
||
|
||
export function processWithThinking(text) {
|
||
const { thinkingBlocks, content, thinkingTime } = extractThinkingBlocks(text);
|
||
|
||
let html = '';
|
||
let visibleContent = content || '';
|
||
const doneOnly = /^\s*\[DONE\]\s*$/i.test(visibleContent);
|
||
const hadTrailingDone = !doneOnly && /(?:^|\n)\s*\[DONE\]\s*$/i.test(visibleContent);
|
||
|
||
// Add thinking sections (collapsed by default)
|
||
thinkingBlocks.forEach((block, index) => {
|
||
html += createThinkingSection(block, index, thinkingTime);
|
||
});
|
||
|
||
// Add the actual content
|
||
if (doneOnly) {
|
||
html += createTaskCompletedMarker();
|
||
} else {
|
||
if (hadTrailingDone) visibleContent = visibleContent.replace(/\n?\s*\[DONE\]\s*$/i, '').trimEnd();
|
||
if (visibleContent) html += mdToHtml(visibleContent);
|
||
if (hadTrailingDone) html += createTaskCompletedMarker();
|
||
}
|
||
|
||
return _useSvgEmoji() ? svgifyEmoji(html) : html;
|
||
}
|
||
|
||
/**
|
||
* Convert markdown to HTML
|
||
*/
|
||
export function mdToHtml(src, opts) {
|
||
const allowedHtmlBlocks = [];
|
||
const codeBlocks = [];
|
||
const inlineCodeBlocks = [];
|
||
const mermaidBlocks = [];
|
||
let s = (src ?? '');
|
||
|
||
// Extract fenced code blocks before any markdown/HTML preservation passes.
|
||
// Otherwise placeholders from the allowed-HTML sanitizer (e.g.
|
||
// ___ALLOWED_HTML_0___) can leak into quoted HTML/JS samples, because the
|
||
// placeholder gets captured as literal code content and never restored inside
|
||
// the final <pre><code> block.
|
||
s = s.replace(/```(\w+)?\n([\s\S]*?)```/g, (_, lang, code) => {
|
||
const cleaned = code
|
||
.replace(/\r\n/g, '\n')
|
||
.replace(/[ \t]+$/gm, '')
|
||
.replace(/^\s*\n+/, '')
|
||
.replace(/\n+\s*$/g, '');
|
||
|
||
// Mermaid diagrams: render as diagram instead of code block
|
||
if (lang && lang.toLowerCase() === 'mermaid') {
|
||
const mermaidId = 'mermaid-' + Date.now() + '-' + mermaidBlocks.length;
|
||
const raw = cleaned.replace(/</g, '<').replace(/>/g, '>').replace(/&/g, '&');
|
||
const placeholder = `___MERMAID_BLOCK_${mermaidBlocks.length}___`;
|
||
mermaidBlocks.push(`<div class="mermaid-container"><pre class="mermaid" id="${mermaidId}">${escapeHtml(raw)}</pre></div>`);
|
||
return placeholder;
|
||
}
|
||
|
||
const escaped = cleaned.replace(/</g, '<').replace(/>/g, '>').replace(/&/g, '&');
|
||
const placeholder = `___CODE_BLOCK_${codeBlocks.length}___`;
|
||
|
||
const langClass = lang ? ` class="language-${lang}"` : '';
|
||
const runnableLangs = ['python','py','javascript','js','html','bash','sh','shell','zsh'];
|
||
const runBtn = (lang && runnableLangs.includes(lang.toLowerCase()))
|
||
? `<button type="button" class="run-code" data-code="${escapeHtml(escaped)}" data-lang="${lang}" title="Run code"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><polygon points="5 3 19 12 5 21 5 3"/></svg></button>`
|
||
: '';
|
||
const editBtn = `<button type="button" class="edit-code" title="Edit"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M11 4H4a2 2 0 0 0-2 2v14a2 2 0 0 0 2 2h14a2 2 0 0 0 2-2v-7"/><path d="M18.5 2.5a2.121 2.121 0 0 1 3 3L12 15l-4 1 1-4 9.5-9.5z"/></svg></button>`;
|
||
codeBlocks.push(`<pre><code${langClass} data-lang="${lang || ''}">${escapeHtml(escaped)}</code>${runBtn}${editBtn}<button type="button" class="copy-code" data-code="${escapeHtml(escaped)}"><svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><rect x="9" y="9" width="13" height="13" rx="2"/><path d="M5 15H4a2 2 0 0 1-2-2V4a2 2 0 0 1 2-2h9a2 2 0 0 1 2 2v1"/></svg></button></pre>`);
|
||
|
||
return placeholder;
|
||
});
|
||
|
||
// Extract inline code spans before the link/autolink/HTML passes, mirroring
|
||
// the fenced-block handling above. A URL inside `inline code` (e.g.
|
||
// `irm http://127.0.0.1:3000/x`) is preceded by a space, so the bare-URL
|
||
// autolink matches it, wraps it in an <a> tag, and swaps that for an
|
||
// ___ALLOWED_HTML_ placeholder — corrupting the command. The old inline-code
|
||
// pass ran after those passes, too late to protect it.
|
||
s = s.replace(/`([^`]+?)`/g, (match, code) => {
|
||
if (code.startsWith('___CODE_BLOCK_') || code.startsWith('___MERMAID_BLOCK_')) return match;
|
||
const placeholder = `___INLINE_CODE_${inlineCodeBlocks.length}___`;
|
||
inlineCodeBlocks.push(`<code>${escapeHtml(code)}</code>`);
|
||
return placeholder;
|
||
});
|
||
|
||
// Repair common ways the agent mangles the entity-anchor convention
|
||
// (`[Name](#kind-<id>)`). Models reliably get the single-link case
|
||
// right but slip into other formats when listing many in a table.
|
||
// These regexes upgrade the broken forms to proper markdown links so
|
||
// the standard `[text](url)` handler below picks them up.
|
||
const ANCHOR_KIND = '(?:session|document|note|image|email|event|task|skill|research)';
|
||
// Case A: `[Name] [#kind-id]` — agent put the URL in brackets, often
|
||
// in a table cell next to the label. Pair them.
|
||
s = s.replace(
|
||
new RegExp(`\\[([^\\]\\n]+?)\\]\\s*\\[#(${ANCHOR_KIND}-[A-Za-z0-9_-]+)\\]`, 'g'),
|
||
'[$1](#$2)',
|
||
);
|
||
// Case B: bare `[#kind-id]` with no preceding label — give it a
|
||
// generic "→ open" link text so it still renders as a button.
|
||
s = s.replace(
|
||
new RegExp(`\\[#(${ANCHOR_KIND}-[A-Za-z0-9_-]+)\\]`, 'g'),
|
||
'[→ open](#$1)',
|
||
);
|
||
// Case C: bare `#kind-id` in plain text — only when it's word-
|
||
// boundary delimited and NOT already inside a markdown link or
|
||
// anchor syntax. Use a lookbehind for `](` or `[` to skip those.
|
||
s = s.replace(
|
||
new RegExp(`(^|[^\\[(])#(${ANCHOR_KIND}-[A-Za-z0-9_-]+)\\b`, 'g'),
|
||
'$1[#$2](#$2)',
|
||
);
|
||
// Legacy search_chats output used bare session hashes (`#<uuid>`). Upgrade
|
||
// those too so old answers and model summaries remain clickable.
|
||
s = s.replace(
|
||
/(^|[^\[(])#([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})\b/gi,
|
||
'$1[#session-$2](#session-$2)',
|
||
);
|
||
|
||
// Convert markdown images before links so  does not become
|
||
// literal "!" plus a normal link.
|
||
s = s.replace(/!\[([^\]\n]*)\]\(([^)\s]+)(?:\s+"([^"]*)")?\)/g, (match, alt, url, title) => {
|
||
return imageHtml(alt, url, title);
|
||
});
|
||
|
||
// Convert markdown links [text](url) to clickable links
|
||
// Internal #hash links navigate in-page; external links open in new tab
|
||
s = s.replace(/\[([^\]]+)\]\(([^)]+)\)/g, (match, text, url) => {
|
||
return linkHtml(text, url);
|
||
});
|
||
|
||
// Autolink bare URLs (http/https). Skips URLs already inside <a> tags
|
||
// (placed by markdown link replacement above) and URLs in backticks.
|
||
s = s.replace(
|
||
/(^|[\s(<])(https?:\/\/[^\s<>"'`\]]+[^\s<>"'`\].,;:!?])/g,
|
||
(match, prefix, url) => `${prefix}${linkHtml(url, url)}`
|
||
);
|
||
|
||
// Autolink scheme-less domains the model often emits as plain text
|
||
// (e.g. "techcrunch.com/ai", "perplexity.ai", "www.wired.com"). The TLD
|
||
// allowlist keeps it from matching file names / versions ("package.json",
|
||
// "node.js", "v1.2.3"); the required start/[\s(<] prefix means domains
|
||
// already inside an http link (preceded by "//") or an email ("@") are
|
||
// skipped. Require the TLD to end at a real domain boundary so dotted code
|
||
// identifiers like `sklearn.metrics` do not link `sklearn.me` and leave
|
||
// placeholder fragments in the remaining text.
|
||
s = s.replace(
|
||
/(^|[\s(<])((?:www\.)?[a-z0-9](?:[a-z0-9-]*[a-z0-9])?(?:\.[a-z0-9-]+)*\.(?:com|org|net|io|ai|co|dev|app|gov|edu|news|info|tech|xyz|me)(?=$|[\/\s<>"'`\]).,;:!?])(?:\/[^\s<>"'`\])]*)?)/gi,
|
||
(match, prefix, domain) => {
|
||
const trail = (domain.match(/[.,;:!?)]+$/) || [''])[0];
|
||
const core = trail ? domain.slice(0, -trail.length) : domain;
|
||
return `${prefix}${linkHtml(core, 'https://' + core)}${trail}`;
|
||
}
|
||
);
|
||
|
||
// Extract <details>...</details> blocks and replace with placeholders
|
||
// Default to open so agent output is visible
|
||
s = s.replace(/<details>([\s\S]*?)<\/details>/gi, (match) => {
|
||
const placeholder = `___ALLOWED_HTML_${allowedHtmlBlocks.length}___`;
|
||
allowedHtmlBlocks.push(sanitizeAllowedHtml(match.replace(/<details>/i, '<details open>')));
|
||
return placeholder;
|
||
});
|
||
|
||
// ALSO preserve <a>/<img> tags the same way (they're now in the HTML from
|
||
// markdown conversion)
|
||
s = s.replace(/<(?:a\s+[^>]*>.*?<\/a|img\s+[^>]*?)>/gi, (match) => {
|
||
const placeholder = `___ALLOWED_HTML_${allowedHtmlBlocks.length}___`;
|
||
allowedHtmlBlocks.push(sanitizeAllowedHtml(match));
|
||
return placeholder;
|
||
});
|
||
|
||
// Now escape everything else
|
||
s = s.replace(/&/g, '&').replace(/</g, '<').replace(/>/g, '>');
|
||
|
||
s = s.replace(/\n{3,}/g, '\n\n');
|
||
|
||
// KaTeX math rendering (after code blocks are extracted, so math in code is safe)
|
||
const mathBlocks = [];
|
||
let sawPendingMath = false;
|
||
|
||
// Typeset straight away when KaTeX is already in, otherwise bank the source in
|
||
// an inert placeholder for renderMath() to swap once the library lands.
|
||
const pushMath = (math, displayMode) => {
|
||
const raw = decodeMathSource(math).trim();
|
||
const placeholder = `___MATH_BLOCK_${mathBlocks.length}___`;
|
||
if (window.katex) {
|
||
mathBlocks.push(katex.renderToString(raw, { displayMode, throwOnError: false }));
|
||
} else {
|
||
sawPendingMath = true;
|
||
mathBlocks.push(`<span class="${MATH_PENDING_CLASS}" data-display="${displayMode}">${escapeHtml(raw)}</span>`);
|
||
}
|
||
return placeholder;
|
||
};
|
||
|
||
// Display math: \[ ... \] — GPT-style delimiter (gpt-5.x, Claude, etc.).
|
||
// Handle before $$/$ so all common delimiters render.
|
||
s = s.replace(/\\\[([\s\S]*?)\\\]/g, (match, math) => {
|
||
try { return pushMath(math, true); } catch (e) { return match; }
|
||
});
|
||
// Inline math: \( ... \) — GPT-style inline delimiter. Single-line only
|
||
// ([^\n]) so a stray escaped paren in prose can't swallow across lines.
|
||
s = s.replace(/\\\(([^\n]*?)\\\)/g, (match, math) => {
|
||
try { return pushMath(math, false); } catch (e) { return match; }
|
||
});
|
||
// Display math: $$...$$
|
||
s = s.replace(/\$\$([\s\S]*?)\$\$/g, (match, math) => {
|
||
try { return pushMath(math, true); } catch (e) { return match; }
|
||
});
|
||
// Inline math: $...$ — single line only, and Pandoc-style delimiter rules so
|
||
// currency doesn't render as math ("$5 to $10"): the opening $ must be
|
||
// immediately followed by a non-space, the closing $ must be immediately
|
||
// preceded by a non-space and not followed by a digit.
|
||
s = s.replace(/(?<![\$\d])\$(?!\$)(?=\S)([^\$\n]+?)(?<=\S)\$(?!\$|\d)/g, (match, math) => {
|
||
try { return pushMath(math, false); } catch (e) { return match; }
|
||
});
|
||
|
||
if (sawPendingMath) _scheduleMathFlush();
|
||
|
||
// Handle pipe tables
|
||
s = s.replace(/(?:^|\n)([^\n]*\|[^\n]*\|[^\n]*)(?:\n([^\n]*\|[^\n]*\|[^\n]*))*/g, (table) => {
|
||
if (table.includes('___CODE_BLOCK_') || table.includes('___ALLOWED_HTML_')) return table;
|
||
|
||
const rows = table.trim().split('\n');
|
||
if (rows.length < 2) return table;
|
||
|
||
let html = '<table style="border-collapse: collapse; width: 100%; margin: 10px 0;">';
|
||
|
||
rows.forEach((row, idx) => {
|
||
if (idx === 1 && /^[\s|:\-]+$/.test(row)) {
|
||
html += '<tbody>';
|
||
return;
|
||
}
|
||
const cells = splitTableRow(row);
|
||
if (cells.length === 0) return;
|
||
|
||
html += '<tr>';
|
||
|
||
cells.forEach(cell => {
|
||
const tag = idx === 0 ? 'th' : 'td';
|
||
html += `<${tag} style="padding: 8px; text-align: left; border-bottom: 1px solid var(--border);">${cell.trim()}</${tag}>`;
|
||
});
|
||
|
||
html += '</tr>';
|
||
});
|
||
|
||
html += '</tbody></table>';
|
||
return html;
|
||
});
|
||
|
||
// Horizontal rules (must come before bold/italic to avoid * conflicts)
|
||
s = s.replace(/^(?:---|\*\*\*|___)\s*$/gm, '<hr>');
|
||
|
||
// Bold, italic, strikethrough
|
||
s = s.replace(/\*\*([^*]+)\*\*/g, '<strong>$1</strong>').replace(/\*([^*]+)\*/g, '<em>$1</em>');
|
||
s = s.replace(/~~([^~]+)~~/g, '<del>$1</del>');
|
||
|
||
// Headers
|
||
s = s.replace(/^###### (.*)$/gm, '<h6>$1</h6>')
|
||
.replace(/^##### (.*)$/gm, '<h5>$1</h5>')
|
||
.replace(/^#### (.*)$/gm, '<h4>$1</h4>')
|
||
.replace(/^### (.*)$/gm, '<h3>$1</h3>')
|
||
.replace(/^## (.*)$/gm, '<h2>$1</h2>')
|
||
.replace(/^# (.*)$/gm, '<h1>$1</h1>');
|
||
|
||
// Ordered lists (1. 2. 3. etc.)
|
||
s = s.replace(/^(\d+)\. (.*)$/gm, '<oli>$2</oli>');
|
||
s = s.replace(/(?:^|\n)(<oli>[\s\S]*?)(?=\n(?!<oli>)|$)/g, m => `<ol>${m.trim().replace(/<\/?oli>/g, (t) => t === '<oli>' ? '<li>' : '</li>')}</ol>`);
|
||
|
||
// GitHub-style task lists (- [ ] / - [x]) → checkbox items. Must run before
|
||
// the generic unordered-list rule so the "- " prefix isn't consumed first.
|
||
// Emits <uli> (with a class) so the unordered-list wrapper below treats it
|
||
// as a list item. Used by plan mode: plan + progress render as a checklist.
|
||
s = s.replace(/^(?:- |\* )\[([ xX])\] (.*)$/gm, (_m, mark, text) => {
|
||
const done = mark.toLowerCase() === 'x';
|
||
return `<uli class="task-item${done ? ' task-done' : ''}"><span class="task-check" aria-hidden="true"></span><span class="task-text">${text}</span></uli>`;
|
||
});
|
||
|
||
// Unordered lists. <uli> may carry attributes (task-item class), so the
|
||
// wrapper preserves them when converting <uli ...> → <li ...>.
|
||
s = s.replace(/^(?:- |\* )(.*)$/gm, '<uli>$1</uli>');
|
||
s = s.replace(/(^|\n)((?:<uli\b[^>]*>[^\n]*<\/uli>(?:\n|$))+)/g, (_, prefix, block) =>
|
||
`${prefix}<ul>${block.trim().replace(/<uli\b([^>]*)>/g, '<li$1>').replace(/<\/uli>/g, '</li>')}</ul>`);
|
||
|
||
// Blockquotes
|
||
s = s.replace(/^> (.*)$/gm, '<bq>$1</bq>');
|
||
s = s.replace(/(?:^|\n)(<bq>[\s\S]*?)(?=\n(?!<bq>)|$)/g, m =>
|
||
`<blockquote>${m.trim().replace(/<\/?bq>/g, (t) => t === '<bq>' ? '<p>' : '</p>')}</blockquote>`);
|
||
|
||
// Paragraphs - but NOT for code block placeholders or allowed HTML
|
||
s = s.replace(/^(?!<h\d|<ul>|<ol>|<li|<oli>|<\/li>|<pre>|<blockquote>|<bq>|<hr>|___CODE_BLOCK_|___ALLOWED_HTML_|___MATH_BLOCK_|___MERMAID_BLOCK_)([^\n]+)$/gm, '<p>$1</p>');
|
||
|
||
// Line breaks within paragraphs
|
||
s = s.replace(/<p>([\s\S]*?)<\/p>/g, (match, content) => {
|
||
if (content.includes('___CODE_BLOCK_') || content.includes('___ALLOWED_HTML_') || content.includes('___MATH_BLOCK_') || content.includes('___MERMAID_BLOCK_')) return match;
|
||
const withLineBreaks = content.replace(/\n{2,}/g, '</p><p>').replace(/\n/g, '<br>');
|
||
return `<p>${withLineBreaks}</p>`;
|
||
});
|
||
|
||
// Remove empty paragraphs
|
||
s = s.replace(/<p><\/p>/g, '');
|
||
|
||
// Every restore below passes a function replacer rather than the block string
|
||
// itself. With a string replacement, `String.replace` reads `$&`, `` $` ``,
|
||
// `$'` and `$$` in the *replacement* as substitution patterns, so a restored
|
||
// block containing them is corrupted: `$&` re-inserts the placeholder, `` $` ``
|
||
// and `$'` splice in the surrounding document, and `$$` collapses to `$`. Those
|
||
// sequences are ordinary content in fenced code (`perl -pe 's/x/$& y/'`,
|
||
// `echo "$$USD"`). A function replacer inserts its return value verbatim.
|
||
|
||
// CRITICAL: Restore allowed HTML blocks first
|
||
allowedHtmlBlocks.forEach((block, index) => {
|
||
s = s.replace(`___ALLOWED_HTML_${index}___`, () => block);
|
||
});
|
||
|
||
// Restore math blocks
|
||
mathBlocks.forEach((block, index) => {
|
||
s = s.replace(`___MATH_BLOCK_${index}___`, () => block);
|
||
});
|
||
|
||
// Restore mermaid diagram blocks
|
||
mermaidBlocks.forEach((block, index) => {
|
||
s = s.replace(`___MERMAID_BLOCK_${index}___`, () => block);
|
||
});
|
||
|
||
// CRITICAL: Restore code blocks at the end
|
||
codeBlocks.forEach((block, index) => {
|
||
s = s.replace(`___CODE_BLOCK_${index}___`, () => block);
|
||
});
|
||
|
||
// Restore inline code spans last, so placeholders carried inside restored
|
||
// <a>/allowed-HTML blocks are resolved too.
|
||
inlineCodeBlocks.forEach((block, index) => {
|
||
s = s.replace(`___INLINE_CODE_${index}___`, () => block);
|
||
});
|
||
|
||
return _useSvgEmoji() ? svgifyEmoji(s, opts) : s;
|
||
}
|
||
|
||
/**
|
||
* Reduce excessive whitespace outside of code blocks
|
||
*/
|
||
export function squashOutsideCode(s) {
|
||
if (!s) return "";
|
||
const parts = String(s).split(/```/);
|
||
for (let i = 0; i < parts.length; i += 2) {
|
||
parts[i] = parts[i]
|
||
.replace(/\r\n/g, '\n')
|
||
.replace(/[ \t]+\n/g, '\n')
|
||
.replace(/\n{3,}/g, '\n\n');
|
||
}
|
||
return parts.join('```');
|
||
}
|
||
|
||
/**
|
||
* Render content that may be text or array of content blocks
|
||
*/
|
||
export function renderContent(content) {
|
||
if (Array.isArray(content)) {
|
||
const texts = [];
|
||
for (const blk of content) {
|
||
if (blk.type === 'text') texts.push(blk.text);
|
||
else if (blk.type === 'image_url') texts.push('[image]');
|
||
}
|
||
return texts.join('\n');
|
||
}
|
||
return content;
|
||
}
|
||
|
||
/**
|
||
* Initialize any unprocessed Mermaid diagrams in a container (or whole document).
|
||
* Returns a promise so callers can await the (lazy) library load if they need to.
|
||
*/
|
||
export function renderMermaid(container) {
|
||
const target = container || document;
|
||
if (!target || typeof target.querySelectorAll !== 'function') return Promise.resolve();
|
||
// Cheap pre-check: no fence on the page means Mermaid is never fetched.
|
||
if (target.querySelectorAll('pre.mermaid:not([data-processed])').length === 0) return Promise.resolve();
|
||
return ensureMermaid()
|
||
.then((mermaid) => {
|
||
// Re-query after the load: during streaming the renderer replaces the
|
||
// message body repeatedly, so the nodes seen before the fetch are stale.
|
||
const nodes = [...target.querySelectorAll('pre.mermaid:not([data-processed])')]
|
||
.filter((node) => node.isConnected);
|
||
if (nodes.length === 0) return;
|
||
return mermaid.run({ nodes });
|
||
})
|
||
.catch((e) => { console.warn('Mermaid render error:', e); });
|
||
}
|
||
|
||
/**
|
||
* Typeset any math that mdToHtml() had to defer because KaTeX was not loaded
|
||
* yet. Once KaTeX is in, mdToHtml() renders inline and this finds nothing.
|
||
*/
|
||
export function renderMath(container) {
|
||
const target = container || document;
|
||
if (!target || typeof target.querySelectorAll !== 'function') return Promise.resolve();
|
||
if (target.querySelectorAll('.' + MATH_PENDING_CLASS).length === 0) return Promise.resolve();
|
||
return ensureKatex()
|
||
.then((katex) => {
|
||
target.querySelectorAll('.' + MATH_PENDING_CLASS).forEach((el) => {
|
||
const displayMode = el.getAttribute('data-display') === 'true';
|
||
try {
|
||
el.outerHTML = katex.renderToString(el.textContent || '', { displayMode, throwOnError: false });
|
||
} catch (e) {
|
||
// Leave the source visible — readable, just not typeset.
|
||
el.classList.remove(MATH_PENDING_CLASS);
|
||
}
|
||
});
|
||
})
|
||
.catch((e) => { console.warn('KaTeX render error:', e); });
|
||
}
|
||
|
||
const markdownModule = {
|
||
escapeHtml,
|
||
mdToHtml,
|
||
sanitizeAllowedHtml,
|
||
squashOutsideCode,
|
||
renderContent,
|
||
processWithThinking,
|
||
createCollapsible,
|
||
hasUnclosedThinkTag,
|
||
extractThinkingBlocks,
|
||
normalizeThinkingMarkup,
|
||
startsWithReasoningPrefix,
|
||
renderMermaid,
|
||
renderMath,
|
||
ensureMermaid,
|
||
ensureKatex
|
||
};
|
||
|
||
export default markdownModule;
|
||
|
||
// Persist which thinking sections were expanded across page refreshes.
|
||
// IDs are render-generated (Date.now-based) so we key by a stable hash of
|
||
// the inner text content instead — same content reproduces the same hash on
|
||
// reload. LocalStorage holds a Set of expanded hashes; we observe the chat
|
||
// history and re-expand matching sections as they're inserted.
|
||
const THINK_EXPANDED_KEY = 'odysseus-thinking-expanded';
|
||
function _loadExpandedSet() {
|
||
try { return new Set(JSON.parse(localStorage.getItem(THINK_EXPANDED_KEY) || '[]')); }
|
||
catch { return new Set(); }
|
||
}
|
||
function _saveExpandedSet(set) {
|
||
try {
|
||
const arr = [...set];
|
||
// Bound storage growth — keep the most recent 200 entries.
|
||
if (arr.length > 200) arr.splice(0, arr.length - 200);
|
||
localStorage.setItem(THINK_EXPANDED_KEY, JSON.stringify(arr));
|
||
} catch {}
|
||
}
|
||
function _hashThinkingContent(el) {
|
||
if (!el) return '';
|
||
const text = (el.textContent || '').trim();
|
||
if (!text) return '';
|
||
let h = 0;
|
||
for (let i = 0; i < text.length; i++) {
|
||
h = (h * 31 + text.charCodeAt(i)) | 0;
|
||
}
|
||
return String(h);
|
||
}
|
||
function _setThinkingExpanded(content, toggle, header, expanded) {
|
||
if (!content || !toggle) return;
|
||
content.classList.toggle('expanded', expanded);
|
||
toggle.classList.toggle('expanded', expanded);
|
||
const label_el = header?.querySelector('.thinking-header-left span');
|
||
if (label_el) {
|
||
const label = label_el.dataset.label || 'thinking process';
|
||
label_el.textContent = expanded ? `Hide ${label}` : `View ${label}`;
|
||
}
|
||
}
|
||
|
||
// Delegated click handler for thinking toggle (CSP-safe, no inline onclick)
|
||
document.addEventListener('click', function(e) {
|
||
const header = e.target.closest('.thinking-header[data-thinking-id]');
|
||
if (!header) return;
|
||
const id = header.dataset.thinkingId;
|
||
const content = document.getElementById(id);
|
||
const toggle = document.getElementById(id + '-toggle');
|
||
if (!content || !toggle) return;
|
||
|
||
const willExpand = !content.classList.contains('expanded');
|
||
_setThinkingExpanded(content, toggle, header, willExpand);
|
||
|
||
// Persist by content hash so the choice survives a refresh.
|
||
const hash = _hashThinkingContent(content);
|
||
if (!hash) return;
|
||
const set = _loadExpandedSet();
|
||
if (willExpand) set.add(hash);
|
||
else set.delete(hash);
|
||
_saveExpandedSet(set);
|
||
});
|
||
|
||
// Watch the chat history; whenever a thinking section appears, expand it if
|
||
// its hash matches one the user previously expanded.
|
||
(function _watchThinking() {
|
||
if (window._thinkingWatcherWired) return;
|
||
window._thinkingWatcherWired = true;
|
||
const _apply = (root) => {
|
||
if (!root || !root.querySelectorAll) return;
|
||
const sections = root.matches?.('.thinking-section')
|
||
? [root]
|
||
: [...root.querySelectorAll('.thinking-section')];
|
||
if (!sections.length) return;
|
||
const set = _loadExpandedSet();
|
||
if (!set.size) return;
|
||
for (const sec of sections) {
|
||
const content = sec.querySelector('.thinking-content');
|
||
if (!content) continue;
|
||
if (content.classList.contains('expanded')) continue;
|
||
const hash = _hashThinkingContent(content);
|
||
if (!hash || !set.has(hash)) continue;
|
||
const header = sec.querySelector('.thinking-header[data-thinking-id]');
|
||
const id = header?.dataset.thinkingId;
|
||
const toggle = id ? document.getElementById(id + '-toggle') : null;
|
||
_setThinkingExpanded(content, toggle, header, true);
|
||
}
|
||
};
|
||
const start = () => {
|
||
const root = document.body;
|
||
if (!root) return;
|
||
_apply(root);
|
||
new MutationObserver((mutations) => {
|
||
for (const m of mutations) {
|
||
for (const node of m.addedNodes) {
|
||
if (node.nodeType === 1) _apply(node);
|
||
}
|
||
}
|
||
}).observe(root, { childList: true, subtree: true });
|
||
};
|
||
if (document.readyState === 'loading') {
|
||
document.addEventListener('DOMContentLoaded', start, { once: true });
|
||
} else {
|
||
start();
|
||
}
|
||
})();
|
||
|
||
function _endpointNameFromUrl(url) {
|
||
try {
|
||
const parsed = new URL(url, window.location.origin);
|
||
return parsed.host || parsed.hostname || 'Model endpoint';
|
||
} catch (_) {
|
||
return 'Model endpoint';
|
||
}
|
||
}
|
||
|
||
function _appendEndpointAddButtons(root) {
|
||
if (!root || !root.querySelectorAll) return;
|
||
const anchors = root.matches?.('a[href]')
|
||
? [root]
|
||
: [...root.querySelectorAll('a[href]')];
|
||
for (const anchor of anchors) {
|
||
if (anchor.dataset.endpointAddChecked === '1') continue;
|
||
anchor.dataset.endpointAddChecked = '1';
|
||
const href = anchor.getAttribute('href') || '';
|
||
if (!_isModelEndpointUrl(href)) continue;
|
||
if (anchor.nextElementSibling?.classList?.contains('model-endpoint-add-btn')) continue;
|
||
|
||
const btn = document.createElement('button');
|
||
btn.type = 'button';
|
||
btn.className = 'model-endpoint-add-btn';
|
||
btn.dataset.endpointUrl = new URL(href, window.location.origin).href.replace(/\/+$/, '');
|
||
btn.title = 'Add this OpenAI-compatible endpoint to the model picker';
|
||
btn.innerHTML = '<span aria-hidden="true">+</span><span>Add to model picker</span>';
|
||
anchor.insertAdjacentElement('afterend', btn);
|
||
}
|
||
}
|
||
|
||
async function _registerEndpointFromButton(btn) {
|
||
const baseUrl = String(btn?.dataset?.endpointUrl || '').trim();
|
||
if (!baseUrl || !_isModelEndpointUrl(baseUrl)) return;
|
||
const original = btn.innerHTML;
|
||
btn.disabled = true;
|
||
btn.innerHTML = '<span aria-hidden="true">...</span><span>Adding</span>';
|
||
try {
|
||
const existingRes = await fetch('/api/model-endpoints', { credentials: 'same-origin' });
|
||
if (existingRes.ok) {
|
||
const endpoints = await existingRes.json();
|
||
const existing = Array.isArray(endpoints)
|
||
? endpoints.find((ep) => String(ep.base_url || '').replace(/\/+$/, '') === baseUrl)
|
||
: null;
|
||
if (existing) {
|
||
btn.classList.add('added');
|
||
btn.innerHTML = '<span aria-hidden="true">✓</span><span>Already added</span>';
|
||
window.dispatchEvent(new CustomEvent('ge:model-endpoints-updated', { detail: { baseUrl } }));
|
||
if (window.modelsModule?.refreshModels) window.modelsModule.refreshModels(true);
|
||
if (window.sessionModule?.updateModelPicker) window.sessionModule.updateModelPicker();
|
||
uiModule.showToast?.(`Already in model picker: ${existing.name || _endpointNameFromUrl(baseUrl)}`);
|
||
return;
|
||
}
|
||
}
|
||
|
||
const parsed = new URL(baseUrl, window.location.origin);
|
||
const fd = new FormData();
|
||
fd.append('base_url', baseUrl);
|
||
fd.append('name', _endpointNameFromUrl(baseUrl));
|
||
fd.append('model_type', 'llm');
|
||
fd.append('endpoint_kind', 'auto');
|
||
fd.append('skip_probe', 'true');
|
||
if (/^(localhost|127\.0\.0\.1|0\.0\.0\.0)$/i.test(parsed.hostname)) {
|
||
fd.append('container_local', 'true');
|
||
}
|
||
const res = await fetch('/api/model-endpoints', {
|
||
method: 'POST',
|
||
credentials: 'same-origin',
|
||
body: fd,
|
||
});
|
||
if (!res.ok) {
|
||
const body = await res.text().catch(() => '');
|
||
throw new Error(`HTTP ${res.status}${body ? ': ' + body.slice(0, 160) : ''}`);
|
||
}
|
||
btn.classList.add('added');
|
||
btn.innerHTML = '<span aria-hidden="true">✓</span><span>Added</span>';
|
||
window.dispatchEvent(new CustomEvent('ge:model-endpoints-updated', { detail: { baseUrl } }));
|
||
if (window.modelsModule?.refreshModels) await window.modelsModule.refreshModels(true);
|
||
if (window.sessionModule?.updateModelPicker) window.sessionModule.updateModelPicker();
|
||
uiModule.showToast?.(`Model endpoint added: ${_endpointNameFromUrl(baseUrl)}`);
|
||
} catch (err) {
|
||
btn.disabled = false;
|
||
btn.innerHTML = original;
|
||
uiModule.showError?.(`Add endpoint failed: ${err.message || err}`);
|
||
}
|
||
}
|
||
|
||
(function _watchModelEndpointLinks() {
|
||
if (window._modelEndpointLinkWatcherWired) return;
|
||
window._modelEndpointLinkWatcherWired = true;
|
||
|
||
document.addEventListener('click', (e) => {
|
||
const btn = e.target.closest?.('.model-endpoint-add-btn');
|
||
if (!btn) return;
|
||
e.preventDefault();
|
||
e.stopPropagation();
|
||
_registerEndpointFromButton(btn);
|
||
});
|
||
|
||
const start = () => {
|
||
const root = document.body;
|
||
if (!root) return;
|
||
_appendEndpointAddButtons(root);
|
||
new MutationObserver((mutations) => {
|
||
for (const m of mutations) {
|
||
for (const node of m.addedNodes) {
|
||
if (node.nodeType === 1) _appendEndpointAddButtons(node);
|
||
}
|
||
}
|
||
}).observe(root, { childList: true, subtree: true });
|
||
};
|
||
if (document.readyState === 'loading') {
|
||
document.addEventListener('DOMContentLoaded', start, { once: true });
|
||
} else {
|
||
start();
|
||
}
|
||
})();
|