fix(skills): stop SKILL.md frontmatter escapes compounding on every save (#5883)

_emit_scalar quotes a frontmatter scalar with json.dumps when it holds
punctuation that would change how the line reads back. _parse_scalar undid that
with a bare raw[1:-1]: it stripped the quotes but never decoded the escapes. So
a description containing ü was written as the escape sequence \u00fc, read
back with that escape still sitting literally in the value, and re-escaped on
the next save. The backslash run doubles every save, so a non-English skill
description degrades into backslash noise after a few edits, and the escapes are
shown verbatim in the skills list and the /skills catalog.

This is not limited to non-ASCII. Any description containing a quote takes the
same path, since the quote is itself what forces the quoted form.

Make the two halves symmetric: emit with ensure_ascii=False, since SKILL.md is
UTF-8 at both ends (skills.py reads it, atomic_write_text writes it) and the
ASCII-escaped form bought nothing; and parse double-quoted scalars with
json.loads, falling back to the previous literal reading when the value is not
valid JSON. Files already corrupted heal one level per load.

ensure_ascii=False on its own would open a smaller hole. json.dumps escapes
every C0 control character but passes NEL, LINE SEPARATOR and PARAGRAPH
SEPARATOR through literally, and parse_frontmatter reads one scalar per line via
str.splitlines(), which breaks on all three. Re-escape those three, and add them
plus the remaining splitlines characters to the set that forces a quoted scalar,
so none of them can reach the file bare.

Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
This commit is contained in:
Ashvin
2026-08-12 01:09:02 +01:00
committed by GitHub
co-authored by Alexandre Teixeira
parent c2b9666def
commit 93653120d6
2 changed files with 255 additions and 2 deletions
+41 -2
View File
@@ -100,6 +100,18 @@ def _parse_scalar(raw: str) -> Any:
if raw.lower() in ("null", "none", "~"):
return None
if (raw[0] == raw[-1]) and raw[0] in ("'", '"'):
if raw[0] == '"':
# _emit_scalar writes double-quoted scalars with json.dumps, so
# decode the escapes instead of only stripping the quotes. Without
# this, `\"` / `\\` / `\uXXXX` stayed verbatim in the value and the
# next save escaped their backslashes again, doubling them on every
# load/save cycle (issue #5210).
try:
return json.loads(raw)
except ValueError:
# Hand-written file using escapes JSON rejects (e.g. a bare
# Windows path). Keep the previous literal reading.
pass
return raw[1:-1]
# Try number
try:
@@ -171,6 +183,26 @@ def parse_frontmatter(text: str) -> tuple[Dict[str, Any], str]:
return fm, body
# Characters that force a quoted scalar. The punctuation would otherwise change
# how the value reads back; the second row is every character str.splitlines()
# treats as a line break, and parse_frontmatter() reads one scalar per line, so
# emitting one of those bare would split the value across lines.
_FM_MUST_QUOTE = (
":", "#", "[", "]", "{", "}", ",", "&", "*", "!", "|", ">", "'", '"', "%", "@",
"\n", "\r", "\v", "\f", "\x1c", "\x1d", "\x1e", "\x85", "\u2028", "\u2029",
)
# json.dumps escapes every C0 control character, but with ensure_ascii=False it
# passes NEL / LINE SEPARATOR / PARAGRAPH SEPARATOR through literally, and
# str.splitlines() still breaks on all three. Re-escape exactly those, which
# json.loads decodes again on the way in, so the pair stays symmetric.
_FM_POST_DUMPS_ESCAPES = (
("\x85", "\\u0085"),
("\u2028", "\\u2028"),
("\u2029", "\\u2029"),
)
def _emit_scalar(v: Any) -> str:
if v is None:
return "null"
@@ -181,8 +213,15 @@ def _emit_scalar(v: Any) -> str:
if isinstance(v, list):
return "[" + ", ".join(_emit_scalar(x) for x in v) + "]"
s = str(v)
if any(c in s for c in (":", "#", "\n", "[", "]", "{", "}", ",", "&", "*", "!", "|", ">", "'", '"', "%", "@")):
return json.dumps(s)
if any(c in s for c in _FM_MUST_QUOTE):
# ensure_ascii=False keeps non-ASCII text as itself. SKILL.md is UTF-8 at
# both ends (skills.py reads it, atomic_write_text writes it), so the
# \uXXXX form bought nothing and leaked into the parsed value (#5210).
out = json.dumps(s, ensure_ascii=False)
for ch, esc in _FM_POST_DUMPS_ESCAPES:
if ch in out:
out = out.replace(ch, esc)
return out
return s