jpmschweitzer/odysseus
1
0
Fork 0
mirror of https://github.com/pewdiepie-archdaemon/odysseus.git synced 2026-10-16 11:52:25 +02:00
Code Issues Packages Projects Releases Wiki Activity

docs(specs): routine specifications update

Browse Source
This commit is contained in:
RaresKeY
2026-10-15 16:52:16 +00:00
parent c74f475679
commit bead6a6797
29 changed files with 767 additions and 511 deletions
Download Patch File Download Diff File Expand all files Collapse all files
specs/model-providers/llama-cpp.md
+8 -19
View File
@@ -1,17 +1,14 @@
# llama.cpp Provider Shape
Last updated: dev@e57f60b | 2026-07-20
Last updated: dev@86b886c | 2026-10-14
## Scope
Canonical provider ID `llamacpp`; OpenAI Chat/Responses and Anthropic Messages
compatibility plus native server metadata; reader
`src/model_capability_readers/llamacpp.py`.
Canonical provider ID `llamacpp`; OpenAI Chat/Responses and Anthropic Messages compatibility plus native server metadata; reader `src/model_capability_readers/llamacpp.py`.
## Metadata Shapes
`/v1/models` provides served identity and can include server model entries;
native `/props` is authoritative for the running model/server combination:
`/v1/models` provides served identity and can include server model entries; native `/props` is authoritative for the running model/server combination:
- `model_alias`/`model_path`;
- `default_generation_settings.n_ctx` and sampling `params`;
@@ -20,25 +17,17 @@ native `/props` is authoritative for the running model/server combination:
- `modalities.vision|audio`;
- current server/build state.
Capability depends on weights, projection/model assets, chat template, parser,
and launch flags. It is endpoint evidence, not a checkpoint-name claim.
`/props` and `/v1/models` can be merged only for the same served identity.
Capability depends on weights, projection/model assets, chat template, parser, and launch flags. It is endpoint evidence, not a checkpoint-name claim. `/props` and `/v1/models` can be merged only for the same served identity.
## Request And Response Shape
llama-server supports several OpenAI-compatible tasks and native extensions.
Do not infer embeddings/rerank/chat solely from the OpenAI model card; use an
explicit server model capability field or endpoint configuration. Tool and
reasoning correctness can depend on selected chat template and parser.
llama-server supports several OpenAI-compatible tasks and native extensions. Do not infer embeddings/rerank/chat solely from the OpenAI model card; use an explicit server model capability field or endpoint configuration. Tool and reasoning correctness can depend on selected chat template and parser.
## Fallback And Safety
The registry selects llama.cpp through an explicit vendor or endpoint kind; it
does not auto-detect `/props` from payload shape. Port 8000 currently maps to
the vLLM placeholder, while 8080 falls through to generic OpenAI-compatible.
llama.cpp-only `session_id` and `cache_prompt` affinity fields must remain local
endpoint behavior and never leak to strict cloud providers (#4640 and current
affinity tests).
The registry selects llama.cpp through an explicit vendor or endpoint kind; it does not auto-detect `/props` from payload shape. Port 8000 currently maps to the vLLM placeholder, while 8080 falls through to generic OpenAI-compatible. llama.cpp-only `session_id` and `cache_prompt` affinity fields must remain local endpoint behavior and never leak to strict cloud providers (#4640 and current affinity tests).
Runtime context lookup in `src/model_context.py` tries `/slots` first and falls back to `/props.default_generation_settings.n_ctx`, covering servers launched without `--slots`. This runtime path remains separate from canonical reader merging.
## Current Gaps