clide/docs/spikes/cc-stream-json-control-protocol-2.1.150.md

Spike: Claude Code stream-json control protocol (T-165 / T-166)

Pinned to: claude 2.1.150. Undocumented, version-drifting internal contracts (per D-75/D-77) — re-validate on a CC bump, keyed off the transcript/event version field and claude_code_version in the init event.

Method: drove the real claude binary as a subprocess over stdin/stdout (docs/spikes/ driver scripts were throwaway; the captured logs are the source of truth) and read the shipped implementation directly — the CLI is a ~238 MB node SEA at ~/.local/share/claude/versions/2.1.150; strings on it exposes the zod schemas and the request/response builders. Every shape below was confirmed live (a real prompt round-tripped) unless marked otherwise. Cost a handful of paid turns.

---

0. TL;DR for the implementation

Spawn:

claude --input-format stream-json --output-format stream-json --verbose \
       --permission-prompt-tool stdio  [ --resume <id> | --session-id <id> ]

• --permission-prompt-tool stdio is mandatory to receive permission prompts. Without it, any tool that resolves to "ask" is auto-denied (no prompt reaches the client) — confirmed: a Write came back as permission_denials + an is_error tool_result "you haven't granted it yet", never a control request. stdio is a hidden value (not in --help); the SDK uses it (f.sdkUrl?"stdio":…).
• Read stdout as line-delimited JSON. Most lines are normal stream events (system/assistant/user/result/rate_limit_event); some are control_requests you must answer or Claude hangs.
• Send a prompt as {"type":"user","message":{"role":"user","content":"…"}}. stream-json does not echo your user messages back unless --replay-user-messages — local-echo them yourself.

---

1. Output event stream (stdout)

One JSON object per line. Types seen:

• system/subtype:"hook_started"|"hook_progress"|"hook_response" — session hooks (e.g. SessionStart). Informational; ignore for rendering.
• system/subtype:"init" — the config goldmine. Carries session_id, cwd, model (claude-opus-4-7[1m]), permissionMode, tools[], mcp_servers[], slash_commands[], skills[], agents[], output_style, claude_code_version, apiKeySource, plugins[]. (This is a strong source for ClaudeConfig / status.)
• assistant — {message:{model,id,content:[…],usage:{…}}, uuid, session_id, request_id}. Content blocks are emitted as separate assistant events sharing one message.id (e.g. a text block, then a tool_use block). Block shapes are identical to the transcript: text / thinking (+signature) / tool_use(id,name,input). message.usage carries input_tokens + cache_read_input_tokens + cache_creation_input_tokens (→ context-token count).
• user — tool results Claude received: message.content:[{type:"tool_result", tool_use_id,content,is_error}] plus a top-level tool_use_result with richer structured data (e.g. Bash {stdout,stderr,interrupted,…}).
• result — terminal turn summary: result (final text), usage, total_cost_usd, permission_denials[], num_turns, modelUsage.<model>.contextWindow (e.g. 1000000) and maxOutputTokens — i.e. the context-window size IS exposed here (relevant to the T-158 budget gap; the remaining-budget % still is not).
• rate_limit_event — {rate_limit_info:{status,resetsAt,rateLimitType,…}}.

The existing parseTranscriptChunk (transcript_reader.dart) parses the assistant/user events as-is (same message.content shapes; missing uuid/timestamp default harmlessly). Permission mode comes from init, not a permission-mode record.

2. Control protocol (bidirectional, same stdin/stdout)

Envelope

Claude → client:

{"type":"control_request","request_id":"<uuid>","request":{"subtype":"…", …}}

Client → Claude (the reply):

{"type":"control_response","response":{"subtype":"success","request_id":"<uuid>","response":{…}}}

On failure use {"subtype":"error","request_id":"…","error":"…"}. request_id must echo exactly. An unanswered control_request hangs the turn.

can_use_tool (the permission ask) — confirmed live

Request:

{"type":"control_request","request_id":"70c8…","request":{
  "subtype":"can_use_tool","tool_name":"Write","display_name":"Write",
  "input":{"file_path":"…","content":"…"},
  "description":"banana.txt",
  "permission_suggestions":[{"type":"setMode","mode":"acceptEdits","destination":"session"}],
  "tool_use_id":"toolu_…"}}

ALLOW — updatedInput is required (a bare {"behavior":"allow"} is rejected with a ZodError: "updatedInput: expected record, received undefined"). Echo the input back unchanged to allow as-is, or modify it to alter the call:

{"type":"control_response","response":{"subtype":"success","request_id":"70c8…",
  "response":{"behavior":"allow","updatedInput":{ …the tool input… }}}}

DENY — message is required:

{"type":"control_response","response":{"subtype":"success","request_id":"70c8…",
  "response":{"behavior":"deny","message":"User declined."}}}

(Decision zod union also allows optional updatedPermissions on allow.)

AskUserQuestion — confirmed live (the non-obvious one)

AskUserQuestion is not answered with a tool_result (that path is rejected as a dismissal — it even shows up in permission_denials). It is permission-gated and answered through the same can_use_tool channel. Its input is exactly clide's own AskUserQuestion shape: {questions:[{question,header,multiSelect,options:[{label,description}]}]}. Return the user's choice by injecting an answers map into updatedInput: answers : record(question-text → chosen-label) (multi-select = comma-separated labels).

{"behavior":"allow","updatedInput":{
   "questions":[ …echoed… ],
   "answers":{"Do you prefer cats or dogs?":"Dogs"}}}

Confirmed: Claude then reported "You chose Dogs"; the resulting tool_use_result was {questions:[…],answers:{"Do you prefer cats or dogs?":"Dogs"}}. Leaving answers empty makes Claude say "your selection didn't come through".

Other control_request subtypes (from the binary's zod schemas)

Client→Claude requests it accepts: initialize, interrupt, set_permission_mode {mode}, mcp_message {server_name,message}. Claude→client requests you may receive: can_use_tool, hook_callback {callback_id,input,tool_use_id?}. Answer every inbound control_request (even unknown subtypes — reply error "Unsupported…") or the turn stalls. There is also a control_cancel_request for in-flight cancellation.

initialize handshake — optional, but a config goldmine

Sending it first is not what enables permissions (the stdio flag does that — verified: initialize-then-Write was still auto-denied). But the response is rich:

// → {"type":"control_request","request_id":"init-1","request":{"subtype":"initialize","hooks":{},"sdkMcpServers":[]}}
// ← response.response = {commands:[{name,description,argumentHint,aliases?}…],
//      agents:[…], models:[…], output_style, available_output_styles,
//      account:{email,organization,subscriptionType,apiProvider}, pid}

commands[] carries descriptions + argumentHints the init event's bare slash_commands[] lacks — better source for the typeahead (T-152) and ClaudeConfig.

3. Implications for the tickets

• T-165: spawn with the flags in §0; feed parseTranscriptChunk from the assistant/user events; pull permissionMode/model from init; local-echo user sends. Isolate all protocol framing in one module (drift-containment, D-77).
• T-166: wire can_use_tool → native prompt card (allow/deny, using display_name/description/permission_suggestions); AskUserQuestion → option picker, returning updatedInput.answers. Answer every control_request.
• ClaudeConfig (D-76): prefer the initialize response's commands[] (has descriptions) over the init slash_commands[].

4. MCP vs the control channel — which layer does what

These are complementary, not competing — a recurring point of confusion:

• The conversation + permissions + AskUserQuestion ride the stream-json control channel (can_use_tool over stdin/stdout). This is the SDK-blessed transport and what canUseTool maps to. T-165/T-166 use it. MCP is not involved here.
• MCP is how you give Claude extra tools/capabilities. That's exactly what the VSCode/JetBrains extensions do: they host an MCP server exposing IDE features to Claude (mcp__ide__getDiagnostics, mcp__ide__executeCode, openDiff) — they do not route the conversation or permissions through MCP; permissions still go through the CLI/control channel. D-77's clide-hosted MCP broker (T-170) is the same idea: provide team messaging / task-sync tools to the agents.
• One overlap exists: --permission-prompt-tool accepts either the special stdio value (control channel — what we use) or an MCP tool name (Claude calls your MCP tool to get the allow/deny). So permissions could be routed through MCP — but it's strictly more machinery than stdio, needs a server, and loses the structured permission_suggestions/display_name the control request carries. Recommendation: keep permissions on stdio; reserve MCP for capability/tool provision (IDE context later, the team broker in T-170).

5. Resilience — the stdio channel is brittle; the fallback menu (no empty slate)

Decision (this spike): carry the conversation + permissions + AskUserQuestion on the stdio control channel (--permission-prompt-tool stdio + can_use_tool). Rationale: most direct — no extra process, no loopback "networking", and Claude hands us structured prompt metadata for free. But it is an undocumented internal contract; Anthropic can change or remove it on any version bump. This section exists so that if it shifts we start from a researched menu, not a blank page.

How we'd detect a break (canaries):

• Pin claude_code_version (here 2.1.150); diff it on every CC upgrade.
• Symptoms of regression: tools auto-denied despite --permission-prompt-tool stdio; can_use_tool requests missing/renamed; ZodError tool_results rejecting our control_response (e.g. the updatedInput-required quirk changing).
• The captured spike logs (this doc's source) double as regression fixtures — wire them into the parser/control-handler tests so a CC bump that breaks shapes goes red.

Fallback menu, in rough preference order if stdio is withdrawn:

1. MCP permission tool — --permission-prompt-tool mcp__clide__approve against the clide-hosted MCP server (T-170 builds that server anyway). Claude calls our MCP tool to get the allow/deny. More layers, but MCP is a stable, public protocol — the strongest fallback precisely because the broker will already exist.
2. Permission modes — degrade to --permission-mode acceptEdits / dontAsk / bypassPermissions for a reduced-fidelity UX (no per-call prompt) as a stopgap. Public, stable flags; buys time to build a better path.
3. Agent SDK — if Anthropic ships/keeps a stable public canUseTool SDK surface (TS/Python), shell out to or port it. Supported, but a heavier dep + language seam.
4. ACP (Agent Client Protocol) — if the ecosystem converges on ACP (session/request_permission, Zed's path) as the blessed editor-integration standard, adopt its adapter. Open standard; would reframe the whole transport.

Containment: all protocol framing lives behind one module (per D-77), so swapping transports is a one-seam change. Re-capture fixtures per pinned version.

Not done / open

• hook_callback and mcp_message round-trips not exercised live (shapes from the binary only) — needed for the clide-hosted MCP broker (T-170).
• Image/file paste intake over stream-json content blocks not tested here.
• Remaining-usage budget % still not exposed (only contextWindow size, in result).