docs(specs): routine specifications update

This commit is contained in:
RaresKeY
2026-10-15 16:52:16 +00:00
parent c74f475679
commit bead6a6797
29 changed files with 767 additions and 511 deletions
+2 -1
View File
@@ -1,6 +1,6 @@
# Provider Capability Specs
Last updated: dev@e71f8ce | 2026-08-25
Last updated: dev@86b886c | 2026-10-14
## Scope
@@ -54,6 +54,7 @@ Current reader dispatch does not infer a provider from payload shape. It uses an
- [github-copilot.md](github-copilot.md): account model-list observations and required runtime headers.
- [chatgpt-subscription.md](chatgpt-subscription.md): Codex model identity and Responses event shape.
- [cohere.md](cohere.md): native endpoint/catalog observations; not currently normalized.
- [featherless.md](featherless.md): plan-filtered catalog search and explicit picker pins without bulk discovery.
### Other provider identity and general/identity-only observations
+21 -24
View File
@@ -1,43 +1,40 @@
# ChatGPT Subscription Provider Shape
Last updated: dev@e57f60b | 2026-07-20
Last updated: dev@86b886c | 2026-10-15
## Scope
Canonical provider ID `chatgpt_subscription`; Codex Responses transport;
auth and runtime code in `src/chatgpt_subscription.py`,
`routes/chatgpt_subscription_routes.py`, and `src/llm_core.py`.
There is no dedicated ChatGPT Subscription canonical reader on current `dev`.
Canonical provider ID `chatgpt_subscription`; Codex Responses transport; auth and runtime code in `src/chatgpt_subscription.py`, `routes/chatgpt_subscription_routes.py`, and `src/llm_core.py`. There is no dedicated ChatGPT Subscription canonical reader on current `dev`.
## Catalog Shape
The account-scoped Codex models endpoint returns root `models[]`; `slug` is the
request identity and `visibility`/`priority` control availability/order. These
fields do not prove tools, reasoning, vision, or context. Null/malformed model
lists fail soft rather than crashing discovery (#5280/#5281).
The account-scoped Codex models endpoint returns root `models[]`; `slug` is the request identity and `visibility`/`priority` control availability/order. These fields do not prove tools, reasoning, vision, or context. Null/malformed model lists fail soft rather than crashing discovery (#5280/#5281).
The canonical generic reader does not accept `slug`-only items, so this runtime
catalog is not currently normalized into `ModelCapabilityRecord` values.
The canonical generic reader does not accept `slug`-only items, so this runtime catalog is not currently normalized into `ModelCapabilityRecord` values.
## Request And Response Shape
Transport uses a ChatGPT backend Responses endpoint, `input` items, flattened
function tools, streamed function-call argument events, exact `call_id`, and
`function_call_output` continuation. Parallel calls and encrypted reasoning
continuity require preserving typed output/history rather than coercing all
roles to text. This shape is supported by the existing adapter and the focused
tool-calling follow-up evidence in #5490; unmerged observations remain claimed
until integrated/reproduced.
Transport uses an inference-only ChatGPT backend Responses payload allowlist: model, instructions, input, stream, store, optional temperature and reasoning effort. It drops native tools, tool choice, previous-response/background fields and unrelated options. `build_responses_input()` flattens conversation history into text input/output items; multipart image blocks are not preserved. Native tool calls/results and encrypted typed reasoning continuity are not preserved as a general protocol. Agent tools therefore use the text protocol on this provider rather than a claimed native function-call continuation.
OAuth/device credentials and refresh are provider-session behavior. Expired
credentials should return an actionable reconnect error, not generic model
failure.
OAuth/device credentials and refresh are provider-session behavior. Expired credentials should return an actionable reconnect error, not generic model failure.
## Fallback And Safety
Only the explicit internal base/ChatGPT host selects this provider. Never send
subscription credentials to a custom OpenAI-compatible URL. Catalog slugs stay
identity-only unless account-scoped fields or probes supply capability.
Only the explicit internal base/ChatGPT host selects this provider. Never send subscription credentials to a custom OpenAI-compatible URL. Catalog slugs alone remain identity-only. Runtime catalog metadata can retain advertised reasoning levels, but the runtime also supplies maintained defaults and model-name fallback metadata; those are not canonical verified capability cards.
## Account And Usage Ownership
Device-flow setup supports multiple accounts using stable provider-auth IDs; display labels are not identity. Reconnect, delete, inventory and account usage are owner-scoped. Subscription endpoint inventory applies exact effective-owner equality even for admins, and refresh deduplication includes the provider-auth ID to keep accounts separate.
`GET /api/chatgpt-subscription/accounts/{auth_id}/usage` returns normalized rate-limit telemetry for exactly one owned auth session. `src.chatgpt_subscription.UsageCache` keeps deep-copied normalized snapshots for 45 seconds, bounded to 256 auth-id entries with FIFO eviction rather than single-flight refresh, and ownership is checked before cache hits. Forced refresh replaces the snapshot; reconnect/deletion invalidates cached usage. Errors return credential-free unavailable payloads. `static/js/chatgptSubscriptionUsage.js` owns the frontend read/display flow.
Runtime catalog metadata retains supported/default reasoning levels and is exposed through `/api/models` model metadata. `validate_reasoning_effort()` omits unsupported overrides; its maintained defaults and name-based fallback are runtime compatibility rules rather than canonical provider evidence.
## Binding And Telemetry Limits
New account setup creates a distinct auth/endpoint pair. Sessions persist endpoint id so the bound endpoint wins over labels; removed or disabled bound endpoints fail closed rather than borrowing another login. Legacy unbound sessions can bind the oldest enabled owner-visible URL-matching endpoint before resolving its exact owner-scoped auth session. Runtime bearer credentials are attached request-locally only to the registered subscription route. Forked session account identity and usage cache ownership remain independent of chat approval grants.
Usage parsing bounds finite numeric telemetry, preserves missing values as unknown and shapes credential-free errors; a JWT claim can supply unverified routing metadata without becoming verified identity. Usage cache and optimistic UI expansion do not guarantee fresh quota or live provider acceptance. Session `effort:<level>` is validated on model changes/partial generation settings, and reasoning effort partitions inference cache identity; metadata defaults/name fallbacks remain compatibility evidence.
## Current Gaps
+26
View File
@@ -0,0 +1,26 @@
# Featherless Provider Shape
Last updated: dev@86b886c | 2026-10-15
## Scope
Runtime provider ID `featherless`; OpenAI-compatible calls in `src/llm_core.py`, catalog and picker ownership in `routes/model_routes.py`, and setup/selection UI in `static/js/admin.js` and `static/js/providers.js`. No dedicated canonical capability reader exists.
## Catalog And Picker
Normal endpoint probing returns no bulk Featherless inventory. New rows use API endpoint kind and an explicit empty pin list; only selected model IDs become picker inventory. Admin `GET /api/model-endpoints/{ep_id}/catalog-search` queries `/v1/models` with `q`, `available_on_current_plan=true`, `status=active`, `conversational=true`, and bounded `page`/`per_page` (maximum 100). Its in-memory cache lasts 45 seconds and keys endpoint ID, normalized query and pagination. Upstream authorization, rate-limit and timeout failures become route errors rather than fabricated model cards.
Reachability tries `/v1/plan` and then a one-item plan-filtered model query. Endpoint status and selected model inventory are distinct facts; a reachable account need not have any pinned models.
## Request And Capability Boundaries
Calls use the OpenAI-compatible runtime adapter. Model availability on the current plan does not prove native tools, reasoning, modality, or context. Per-model endpoint tool modes are operator configuration; canonical generic normalization remains identity-only.
## Selection UI Limits
The admin picker debounces queries after two characters, pages bounded catalog results and saves pins/tool modes optimistically. Initial search cancellation does not establish equivalent load-more generation guards, and ignored PATCH response status does not prove accepted persistence. Lowercased query cache identity, explicit empty pins and endpoint reachability remain separate from provider plan/capability acceptance.
## Current Gaps
- No native canonical reader maps plan/catalog evidence into capability records.
- Catalog search is a live provider dependency; cache and pins do not guarantee continued account availability.
+8 -19
View File
@@ -1,17 +1,14 @@
# llama.cpp Provider Shape
Last updated: dev@e57f60b | 2026-07-20
Last updated: dev@86b886c | 2026-10-14
## Scope
Canonical provider ID `llamacpp`; OpenAI Chat/Responses and Anthropic Messages
compatibility plus native server metadata; reader
`src/model_capability_readers/llamacpp.py`.
Canonical provider ID `llamacpp`; OpenAI Chat/Responses and Anthropic Messages compatibility plus native server metadata; reader `src/model_capability_readers/llamacpp.py`.
## Metadata Shapes
`/v1/models` provides served identity and can include server model entries;
native `/props` is authoritative for the running model/server combination:
`/v1/models` provides served identity and can include server model entries; native `/props` is authoritative for the running model/server combination:
- `model_alias`/`model_path`;
- `default_generation_settings.n_ctx` and sampling `params`;
@@ -20,25 +17,17 @@ native `/props` is authoritative for the running model/server combination:
- `modalities.vision|audio`;
- current server/build state.
Capability depends on weights, projection/model assets, chat template, parser,
and launch flags. It is endpoint evidence, not a checkpoint-name claim.
`/props` and `/v1/models` can be merged only for the same served identity.
Capability depends on weights, projection/model assets, chat template, parser, and launch flags. It is endpoint evidence, not a checkpoint-name claim. `/props` and `/v1/models` can be merged only for the same served identity.
## Request And Response Shape
llama-server supports several OpenAI-compatible tasks and native extensions.
Do not infer embeddings/rerank/chat solely from the OpenAI model card; use an
explicit server model capability field or endpoint configuration. Tool and
reasoning correctness can depend on selected chat template and parser.
llama-server supports several OpenAI-compatible tasks and native extensions. Do not infer embeddings/rerank/chat solely from the OpenAI model card; use an explicit server model capability field or endpoint configuration. Tool and reasoning correctness can depend on selected chat template and parser.
## Fallback And Safety
The registry selects llama.cpp through an explicit vendor or endpoint kind; it
does not auto-detect `/props` from payload shape. Port 8000 currently maps to
the vLLM placeholder, while 8080 falls through to generic OpenAI-compatible.
llama.cpp-only `session_id` and `cache_prompt` affinity fields must remain local
endpoint behavior and never leak to strict cloud providers (#4640 and current
affinity tests).
The registry selects llama.cpp through an explicit vendor or endpoint kind; it does not auto-detect `/props` from payload shape. Port 8000 currently maps to the vLLM placeholder, while 8080 falls through to generic OpenAI-compatible. llama.cpp-only `session_id` and `cache_prompt` affinity fields must remain local endpoint behavior and never leak to strict cloud providers (#4640 and current affinity tests).
Runtime context lookup in `src/model_context.py` tries `/slots` first and falls back to `/props.default_generation_settings.n_ctx`, covering servers launched without `--slots`. This runtime path remains separate from canonical reader merging.
## Current Gaps