Squash Odysseus development history

specs/model-providers/llama-cpp.md

@@ -0,0 +1,47 @@
+# llama.cpp Provider Shape
+
+Last updated: dev@e57f60b | 2026-07-20
+
+## Scope
+
+Canonical provider ID `llamacpp`; OpenAI Chat/Responses and Anthropic Messages
+compatibility plus native server metadata; reader
+`src/model_capability_readers/llamacpp.py`.
+
+## Metadata Shapes
+
+`/v1/models` provides served identity and can include server model entries;
+native `/props` is authoritative for the running model/server combination:
+
+- `model_alias`/`model_path`;
+- `default_generation_settings.n_ctx` and sampling `params`;
+- `total_slots` and optional `/slots[].n_ctx` fallback;
+- `chat_template_caps` for tools/system role;
+- `modalities.vision|audio`;
+- current server/build state.
+
+Capability depends on weights, projection/model assets, chat template, parser,
+and launch flags. It is endpoint evidence, not a checkpoint-name claim.
+`/props` and `/v1/models` can be merged only for the same served identity.
+
+## Request And Response Shape
+
+llama-server supports several OpenAI-compatible tasks and native extensions.
+Do not infer embeddings/rerank/chat solely from the OpenAI model card; use an
+explicit server model capability field or endpoint configuration. Tool and
+reasoning correctness can depend on selected chat template and parser.
+
+## Fallback And Safety
+
+The registry selects llama.cpp through an explicit vendor or endpoint kind; it
+does not auto-detect `/props` from payload shape. Port 8000 currently maps to
+the vLLM placeholder, while 8080 falls through to generic OpenAI-compatible.
+llama.cpp-only `session_id` and `cache_prompt` affinity fields must remain local
+endpoint behavior and never leak to strict cloud providers (#4640 and current
+affinity tests).
+
+## Current Gaps
+
+- Multi-model routing requires per-served-model `/props` association.
+- Parser/template configuration is not yet fully represented in canonical
+  endpoint metadata.