mirror of
https://github.com/pewdiepie-archdaemon/odysseus.git
synced 2026-10-12 09:52:19 +02:00
45 lines
1.9 KiB
Markdown
45 lines
1.9 KiB
Markdown
# vLLM Provider Shape
|
|
|
|
Last updated: dev@e57f60b | 2026-07-20
|
|
|
|
## Scope
|
|
|
|
Canonical placeholder provider ID `vllm`; OpenAI Chat and Responses serving;
|
|
generic identity-only inventory normalization. There is no dedicated vLLM
|
|
reader or model-card detector on current `dev`.
|
|
|
|
## Catalog Shape
|
|
|
|
Current `GET /v1/models` returns `object: list`, `data[]` model cards with
|
|
`id`, `object`, `owned_by: vllm`, `root`, `parent`, `max_model_len`, and
|
|
`permission[]`. The generic reader retains only identity/raw data and does not
|
|
inspect `owned_by`, `root`, `parent`, `max_model_len`, or `permission`. The card
|
|
does not prove chat template, tools,
|
|
reasoning parser, vision assets, embeddings, transcription, or rerank.
|
|
|
|
LoRA cards can use a different `id`, root path, and parent. Keep each served ID
|
|
endpoint scoped and do not merge it globally with the base checkpoint.
|
|
|
|
## Runtime Capability
|
|
|
|
vLLM's supported API surface is broad, but actual behavior depends on the
|
|
loaded model task, chat template, multimodal assets, tool-call parser,
|
|
reasoning parser, structured-output configuration, and launch flags. Current
|
|
Odysseus reasoning regressions cover structured `reasoning`, legacy
|
|
`reasoning_content`, and compatible fields (#602). These response channels are
|
|
transport evidence, not a claim that every vLLM model reasons.
|
|
|
|
## Fallback And Safety
|
|
|
|
Current reader detection identifies port 8000 as vLLM, or accepts an explicit
|
|
endpoint kind, then dispatches to the generic identity-only reader. It does not
|
|
infer vLLM from the model-card payload. Do not consume `/server_info`
|
|
environment/config dumps for normal discovery because they can be large and
|
|
operationally sensitive.
|
|
|
|
## Current Gaps
|
|
|
|
- A small safe native capability endpoint is not part of the canonical probe.
|
|
- Deployment parser/template flags are not persisted with endpoint capability.
|
|
- No dedicated reader maps vLLM model-card fields today.
|