Sync ollama docs from eab97e9f on 2026-10-12
This commit is contained in:
The Librarian
+2
-2
@@ -4,8 +4,8 @@ This is a mirror of the Ollama repository.
|
||||
|
||||
**Synced from:** https://github.com/ollama/ollama.git
|
||||
**Branch:** main
|
||||
**Commit:** 53fed26112817f7c55f664efb9e3f65f06cab7db
|
||||
**Sync Date:** 2026-09-12
|
||||
**Commit:** eab97e9f92b9a25c2d52d2cc6c1b1c99bd9fae21
|
||||
**Sync Date:** 2026-10-12
|
||||
**Content:** Paths: docs
|
||||
|
||||
---
|
||||
|
||||
+48
-49
@@ -378,6 +378,8 @@ curl http://localhost:11434/api/generate -d '{
|
||||
|
||||
If you want to set custom options for the model at runtime rather than in the Modelfile, you can do so with the `options` parameter. This example sets every available option, but you can set any of them individually and omit the ones you do not want to override.
|
||||
|
||||
`typical_p` is deprecated and may be removed in a future release.
|
||||
|
||||
##### Request
|
||||
|
||||
```shell
|
||||
@@ -393,7 +395,6 @@ curl http://localhost:11434/api/generate -d '{
|
||||
"top_k": 20,
|
||||
"top_p": 0.9,
|
||||
"min_p": 0.0,
|
||||
"typical_p": 0.7,
|
||||
"repeat_last_n": 33,
|
||||
"temperature": 0.8,
|
||||
"repeat_penalty": 1.2,
|
||||
@@ -1178,16 +1179,17 @@ Create a model from:
|
||||
|
||||
- another model;
|
||||
- a safetensors directory; or
|
||||
- a GGUF file.
|
||||
- one or more GGUF files.
|
||||
|
||||
If you are creating a model from a safetensors directory or from a GGUF file, you must [push a blob](#push-a-blob) for each of the files and then use the file name and SHA256 digest associated with each blob in the `files` field.
|
||||
Use GGUF community tools to prepare and quantize GGUF files before importing.
|
||||
|
||||
If you are creating a model from a safetensors directory or from GGUF files, you must [push a blob](#push-a-blob) for each file and then use its file name and SHA256 digest in the `files` field.
|
||||
|
||||
### Parameters
|
||||
|
||||
- `model`: name of the model to create
|
||||
- `from`: (optional) name of an existing model to create the new model from
|
||||
- `files`: (optional) a dictionary of file names to SHA256 digests of blobs to create the model from
|
||||
- `adapters`: (optional) a dictionary of file names to SHA256 digests of blobs for LORA adapters
|
||||
- `template`: (optional) the prompt template for the model
|
||||
- `renderer`: (optional) the name of the renderer for the model
|
||||
- `parser`: (optional) the name of the parser for the model
|
||||
@@ -1195,16 +1197,19 @@ If you are creating a model from a safetensors directory or from a GGUF file, yo
|
||||
- `system`: (optional) a string containing the system prompt for the model
|
||||
- `parameters`: (optional) a dictionary of parameters for the model (see [Modelfile](./modelfile.mdx#valid-parameters-and-values) for a list of parameters)
|
||||
- `messages`: (optional) a list of message objects used to create a conversation
|
||||
- `capabilities`: (optional) capabilities to add without removing inherited or inferred capabilities, such as `["decision"]` for compatible decision models
|
||||
- `stream`: (optional) if `false` the response will be returned as a single response object, rather than a stream of objects
|
||||
- `quantize` (optional): quantize a non-quantized (e.g. float16) model
|
||||
- `quantize`: (optional) quantize safetensors model weights for MLX during import
|
||||
|
||||
#### Quantization types
|
||||
|
||||
| Type | Recommended |
|
||||
| ------ | :---------: |
|
||||
| q4_K_M | \* |
|
||||
| q4_K_S | |
|
||||
| q8_0 | \* |
|
||||
| Type | Recommended |
|
||||
| ----- | :---------: |
|
||||
| nvfp4 | \* |
|
||||
| mxfp8 | \* |
|
||||
| mxfp4 | |
|
||||
| int4 | |
|
||||
| int8 | |
|
||||
|
||||
### Examples
|
||||
|
||||
@@ -1240,40 +1245,9 @@ A stream of JSON objects is returned:
|
||||
{"status":"success"}
|
||||
```
|
||||
|
||||
#### Quantize a model
|
||||
|
||||
Quantize a non-quantized model.
|
||||
|
||||
##### Request
|
||||
|
||||
```shell
|
||||
curl http://localhost:11434/api/create -d '{
|
||||
"model": "llama3.2:quantized",
|
||||
"from": "llama3.2:3b-instruct-fp16",
|
||||
"quantize": "q4_K_M"
|
||||
}'
|
||||
```
|
||||
|
||||
##### Response
|
||||
|
||||
A stream of JSON objects is returned:
|
||||
|
||||
```json
|
||||
{"status":"quantizing F16 model to Q4_K_M","digest":"0","total":6433687776,"completed":12302}
|
||||
{"status":"quantizing F16 model to Q4_K_M","digest":"0","total":6433687776,"completed":6433687552}
|
||||
{"status":"verifying conversion"}
|
||||
{"status":"creating new layer sha256:fb7f4f211b89c6c4928ff4ddb73db9f9c0cfca3e000c3e40d6cf27ddc6ca72eb"}
|
||||
{"status":"using existing layer sha256:966de95ca8a62200913e3f8bfbf84c8494536f1b94b49166851e76644e966396"}
|
||||
{"status":"using existing layer sha256:fcc5a6bec9daf9b561a68827b67ab6088e1dba9d1fa2a50d7bbcc8384e0a265d"}
|
||||
{"status":"using existing layer sha256:a70ff7e570d97baaf4e62ac6e6ad9975e04caa6d900d3742d37698494479e0cd"}
|
||||
{"status":"using existing layer sha256:56bb8bd477a519ffa694fc449c2413c6f0e1d3b1c88fa7e3c9d88d3ae49d4dcb"}
|
||||
{"status":"writing manifest"}
|
||||
{"status":"success"}
|
||||
```
|
||||
|
||||
#### Create a model from GGUF
|
||||
|
||||
Create a model from a GGUF file. The `files` parameter should be filled out with the file name and SHA256 digest of the GGUF file you wish to use. Use [/api/blobs/:digest](#push-a-blob) to push the GGUF file to the server before calling this API.
|
||||
Create a model from GGUF files. The `files` parameter should contain the file name and SHA256 digest of each GGUF file. For a split GGUF model, include each shard under its original split filename. Use [/api/blobs/:digest](#push-a-blob) to push each file before calling this API.
|
||||
|
||||
##### Request
|
||||
|
||||
@@ -1299,7 +1273,7 @@ A stream of JSON objects is returned:
|
||||
|
||||
#### Create a model from a Safetensors directory
|
||||
|
||||
The `files` parameter should include a dictionary of files for the safetensors model which includes the file names and SHA256 digest of each file. Use [/api/blobs/:digest](#push-a-blob) to first push each of the files to the server before calling this API. Files will remain in the cache until the Ollama server is restarted.
|
||||
The `files` parameter should include the file name and SHA256 digest of each file in the Safetensors model. Use [/api/blobs/:digest](#push-a-blob) to push each file before calling this API. Files remain in the cache until the Ollama server is restarted.
|
||||
|
||||
##### Request
|
||||
|
||||
@@ -1322,11 +1296,8 @@ curl http://localhost:11434/api/create -d '{
|
||||
A stream of JSON objects is returned:
|
||||
|
||||
```shell
|
||||
{"status":"converting model"}
|
||||
{"status":"creating new layer sha256:05ca5b813af4a53d2c2922933936e398958855c44ee534858fcfd830940618b6"}
|
||||
{"status":"using autodetected template llama3-instruct"}
|
||||
{"status":"using existing layer sha256:56bb8bd477a519ffa694fc449c2413c6f0e1d3b1c88fa7e3c9d88d3ae49d4dcb"}
|
||||
{"status":"writing manifest"}
|
||||
{"status":"importing fred (2 tensors)"}
|
||||
{"status":"writing manifest for fred"}
|
||||
{"status":"success"}
|
||||
```
|
||||
|
||||
@@ -1523,6 +1494,12 @@ curl http://localhost:11434/api/copy -d '{
|
||||
|
||||
Returns a 200 OK if successful, or a 404 Not Found if the source model doesn't exist.
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "success"
|
||||
}
|
||||
```
|
||||
|
||||
## Delete a Model
|
||||
|
||||
```
|
||||
@@ -1697,7 +1674,7 @@ Generate embeddings from a model
|
||||
### Parameters
|
||||
|
||||
- `model`: name of model to generate embeddings from
|
||||
- `input`: text or list of text to generate embeddings for
|
||||
- `input`: text or list of text to generate embeddings for. To embed media (images or audio) alongside text on models that accept multimodal embeddings, each entry may instead be an object with any of `text` (string), `image` (base64-encoded bytes), `audio` (base64-encoded bytes). In a list, entries can mix plain strings and such objects; each object owns its own media. `video` is not currently accepted.
|
||||
|
||||
Advanced parameters:
|
||||
|
||||
@@ -1761,6 +1738,28 @@ curl http://localhost:11434/api/embed -d '{
|
||||
}
|
||||
```
|
||||
|
||||
#### Request (With image)
|
||||
|
||||
```shell
|
||||
curl http://localhost:11434/api/embed -d '{
|
||||
"model": "embeddinggemma2",
|
||||
"input": {"text": "a cat sitting on a windowsill", "image": "iVBORw0KGgo..."}
|
||||
}'
|
||||
```
|
||||
|
||||
#### Request (Batched, mixed modalities)
|
||||
|
||||
```shell
|
||||
curl http://localhost:11434/api/embed -d '{
|
||||
"model": "embeddinggemma2",
|
||||
"input": [
|
||||
"a plain text query",
|
||||
{"image": "iVBORw0KGgo..."},
|
||||
{"audio": "UklGRiQAAABXQVZF..."}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
## List Running Models
|
||||
|
||||
```
|
||||
|
||||
@@ -2,9 +2,42 @@
|
||||
title: Anthropic compatibility
|
||||
---
|
||||
|
||||
Ollama provides compatibility with the [Anthropic Messages API](https://docs.anthropic.com/en/api/messages) to help connect existing applications to Ollama, including tools like Claude Code.
|
||||
Connect Anthropic clients and tools such as Claude Code to Ollama. Ollama supports a subset of the [Anthropic Messages API](https://platform.claude.com/docs/en/api/http/messages/create).
|
||||
|
||||
## Usage
|
||||
## Direct cloud access
|
||||
|
||||
Set your [API key](https://ollama.com/settings/keys) in `OLLAMA_API_KEY`. No Ollama installation required.
|
||||
|
||||
```shell
|
||||
export OLLAMA_API_KEY="your_api_key"
|
||||
```
|
||||
|
||||
Send a request with bearer authentication:
|
||||
|
||||
```shell
|
||||
curl https://ollama.com/v1/messages \
|
||||
-H "Authorization: Bearer $OLLAMA_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "anthropic-version: 2023-06-01" \
|
||||
-d '{
|
||||
"model": "gemma4:31b",
|
||||
"max_tokens": 1024,
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Say hello in one sentence."
|
||||
}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
Read text blocks from `content`. The cloud endpoint requires `Authorization: Bearer`; it does not accept `x-api-key` alone.
|
||||
|
||||
Compatible models support messages, streaming, and function calling. Tool-choice controls, deferred tools, and hosted web search are not fully supported.
|
||||
|
||||
## Local server usage
|
||||
|
||||
These examples connect to your local Ollama server. Use `ollama` as the placeholder API key. To use cloud models through this server, [sign in to Ollama](/api/authentication#signing-in).
|
||||
|
||||
### Environment variables
|
||||
|
||||
@@ -234,55 +267,7 @@ curl -X POST http://localhost:11434/v1/messages \
|
||||
|
||||
## Using with Claude Code
|
||||
|
||||
[Claude Code](https://code.claude.com/docs/en/overview) can be configured to use Ollama as its backend.
|
||||
|
||||
### Recommended models
|
||||
|
||||
For coding use cases, models like `glm-4.7`, `minimax-m2.1`, and `qwen3-coder` are recommended.
|
||||
|
||||
Download a model before use:
|
||||
|
||||
```shell
|
||||
ollama pull qwen3-coder
|
||||
```
|
||||
> Note: Qwen 3 coder is a 30B parameter model requiring at least 24GB of VRAM to run smoothly. More is required for longer context lengths.
|
||||
|
||||
```shell
|
||||
ollama pull glm-4.7:cloud
|
||||
```
|
||||
|
||||
### Quick setup
|
||||
|
||||
```shell
|
||||
ollama launch claude
|
||||
```
|
||||
|
||||
This will prompt you to select a model, configure Claude Code automatically, and launch it. To configure without launching:
|
||||
|
||||
```shell
|
||||
ollama launch claude --config
|
||||
```
|
||||
|
||||
### Manual setup
|
||||
|
||||
Set the environment variables and run Claude Code:
|
||||
|
||||
```shell
|
||||
ANTHROPIC_AUTH_TOKEN=ollama ANTHROPIC_BASE_URL=http://localhost:11434 claude --model qwen3-coder
|
||||
```
|
||||
|
||||
Or set the environment variables in your shell profile:
|
||||
|
||||
```shell
|
||||
export ANTHROPIC_AUTH_TOKEN=ollama
|
||||
export ANTHROPIC_BASE_URL=http://localhost:11434
|
||||
```
|
||||
|
||||
Then run Claude Code with any Ollama model:
|
||||
|
||||
```shell
|
||||
claude --model qwen3-coder
|
||||
```
|
||||
See the [Claude Code guide](/integrations/claude-code) for Ollama Launch and [direct cloud setup](/integrations/claude-code#connect-directly-to-ollama-cloud).
|
||||
|
||||
## Endpoints
|
||||
|
||||
@@ -318,6 +303,8 @@ claude --model qwen3-coder
|
||||
- [x] `stop_sequences`
|
||||
- [x] `tools`
|
||||
- [x] `thinking`
|
||||
- [x] `output_config`
|
||||
- [x] `effort` (model-defined names)
|
||||
- [ ] `tool_choice`
|
||||
- [ ] `metadata`
|
||||
|
||||
@@ -342,6 +329,8 @@ claude --model qwen3-coder
|
||||
- [x] `ping`
|
||||
- [x] `error`
|
||||
|
||||
Use `/api/show` to discover a model's supported thinking values and default. `thinking.type: "enabled"` and `"disabled"` act as explicit on/off controls. When a model uses its metadata to resolve named levels, supported `output_config.effort` names are applied exactly and unsupported names resolve to the model default. Models without metadata keep the existing compatibility behavior, which trims and lowercases effort names and maps `"xhigh"` to `"high"`.
|
||||
|
||||
## Models
|
||||
|
||||
Ollama supports both local and cloud models.
|
||||
@@ -360,12 +349,9 @@ Recommended local models:
|
||||
|
||||
### Cloud models
|
||||
|
||||
Cloud models are available immediately without pulling:
|
||||
Browse the [cloud model catalog](https://ollama.com/search?c=cloud). Through a signed-in Ollama server, use a cloud name such as `gemma4:cloud` without a separate pull. For direct cloud requests, use the hosted identifier, such as `gemma4:31b`.
|
||||
|
||||
- `glm-4.7:cloud` - High-performance cloud model
|
||||
- `minimax-m2.1:cloud` - Fast cloud model
|
||||
|
||||
### Default model names
|
||||
### Default model names on a local server
|
||||
|
||||
For tooling that relies on default Anthropic model names such as `claude-3-5-sonnet`, use `ollama cp` to copy an existing model name:
|
||||
|
||||
@@ -394,8 +380,8 @@ curl http://localhost:11434/v1/messages \
|
||||
|
||||
### Behavior differences
|
||||
|
||||
- API key is accepted but not validated
|
||||
- `anthropic-version` header is accepted but not used
|
||||
- The local server does not validate API keys. Direct cloud inference requires a valid bearer token.
|
||||
- The hosted endpoint rejects `anthropic-version` values older than `2023-06-01`. Use `2023-06-01` in requests.
|
||||
- Token counts are approximations based on the underlying model's tokenizer
|
||||
|
||||
### Not supported
|
||||
|
||||
+29
-35
@@ -2,34 +2,52 @@
|
||||
title: Authentication
|
||||
---
|
||||
|
||||
No authentication is required when accessing Ollama's API locally via `http://localhost:11434`.
|
||||
The local API at `http://localhost:11434` does not require authentication.
|
||||
|
||||
Authentication is required for the following:
|
||||
To use cloud models, publish models, or download private models, sign in to Ollama. For direct API access to ollama.com, use an API key.
|
||||
|
||||
* Running cloud models via ollama.com
|
||||
* Publishing models
|
||||
* Downloading private models
|
||||
## API keys
|
||||
|
||||
Ollama supports two authentication methods:
|
||||
Direct cloud inference at `https://ollama.com/api` and `https://ollama.com/v1` requires an API key. No Ollama installation or local server is required.
|
||||
|
||||
* **Signing in**: sign in from your local installation, and Ollama will automatically take care of authenticating requests to ollama.com when running commands
|
||||
* **API keys**: API keys for programmatic access to ollama.com's API
|
||||
Create an [API key](https://ollama.com/settings/keys), then set it in your terminal:
|
||||
|
||||
```shell
|
||||
export OLLAMA_API_KEY="your_api_key"
|
||||
```
|
||||
|
||||
Use the API key in the `Authorization: Bearer` header. This also applies to the hosted Anthropic-compatible `/v1/messages` endpoint; `x-api-key` alone is not supported.
|
||||
|
||||
```shell
|
||||
curl https://ollama.com/api/chat \
|
||||
-H "Authorization: Bearer $OLLAMA_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "gemma4:31b",
|
||||
"messages": [{"role": "user", "content": "Why is the sky blue?"}],
|
||||
"stream": false
|
||||
}'
|
||||
```
|
||||
|
||||
API keys do not expire. Revoke a key in [API keys settings](https://ollama.com/settings/keys).
|
||||
|
||||
Keep keys outside browser code and source control. See the [API quickstart](/quickstart#build-an-application) for shell setup and compatibility examples.
|
||||
|
||||
## Signing in
|
||||
|
||||
To sign in to ollama.com from your local installation of Ollama, run:
|
||||
Sign in from your terminal:
|
||||
|
||||
```
|
||||
ollama signin
|
||||
```
|
||||
|
||||
Once signed in, Ollama will automatically authenticate commands as required:
|
||||
Ollama then authenticates cloud requests for you:
|
||||
|
||||
```
|
||||
ollama run gpt-oss:120b-cloud
|
||||
```
|
||||
|
||||
Similarly, when accessing a local API endpoint that requires cloud access, Ollama will automatically authenticate the request:
|
||||
This also works through the local API:
|
||||
|
||||
```shell
|
||||
curl http://localhost:11434/api/generate -d '{
|
||||
@@ -37,27 +55,3 @@ curl http://localhost:11434/api/generate -d '{
|
||||
"prompt": "Why is the sky blue?"
|
||||
}'
|
||||
```
|
||||
|
||||
## API keys
|
||||
|
||||
For direct access to ollama.com's API served at `https://ollama.com/api`, authentication via API keys is required.
|
||||
|
||||
First, create an [API key](https://ollama.com/settings/keys), then set the `OLLAMA_API_KEY` environment variable:
|
||||
|
||||
```shell
|
||||
export OLLAMA_API_KEY=your_api_key
|
||||
```
|
||||
|
||||
Then use the API key in the Authorization header:
|
||||
|
||||
```shell
|
||||
curl https://ollama.com/api/generate \
|
||||
-H "Authorization: Bearer $OLLAMA_API_KEY" \
|
||||
-d '{
|
||||
"model": "gpt-oss:120b",
|
||||
"prompt": "Why is the sky blue?",
|
||||
"stream": false
|
||||
}'
|
||||
```
|
||||
|
||||
API keys don't currently expire, however you can revoke them at any time in your [API keys settings](https://ollama.com/settings/keys).
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
---
|
||||
title: Balance
|
||||
sidebarTitle: /api/balance
|
||||
openapi: /openapi.yaml GET /api/balance
|
||||
---
|
||||
|
||||
<RequestExample>
|
||||
|
||||
```shell Request
|
||||
curl https://ollama.com/api/balance \
|
||||
-H "Authorization: Bearer $OLLAMA_API_KEY"
|
||||
```
|
||||
|
||||
</RequestExample>
|
||||
|
||||
<ResponseExample>
|
||||
|
||||
```json Response
|
||||
{
|
||||
"included": {
|
||||
"balance_usd": 72.5,
|
||||
"allowance_usd": 100,
|
||||
"period": {
|
||||
"from": "2026-09-15T09:30:00Z",
|
||||
"until": "2026-10-15T09:30:00Z"
|
||||
}
|
||||
},
|
||||
"purchased": {
|
||||
"balance_usd": 25
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
</ResponseExample>
|
||||
|
||||
This endpoint takes no query parameters. For usage history, see [Cloud usage](/api/cloud-usage).
|
||||
|
||||
## Balances
|
||||
|
||||
The included period follows your plan's monthly reset schedule, including for annual subscriptions. Period timestamps are in UTC, with `from` inclusive and `until` exclusive.
|
||||
|
||||
The purchased balance includes only unexpired credits.
|
||||
|
||||
<Accordion title="Legacy plans">
|
||||
|
||||
Plans with session and weekly limits return `session` and `weekly` in `included`:
|
||||
|
||||
```json
|
||||
{
|
||||
"included": {
|
||||
"session": {
|
||||
"remaining_percent": 75,
|
||||
"resets_at": "2026-10-01T07:00:00Z"
|
||||
},
|
||||
"weekly": {
|
||||
"remaining_percent": 40,
|
||||
"resets_at": "2026-10-05T00:00:00Z"
|
||||
}
|
||||
},
|
||||
"purchased": {
|
||||
"balance_usd": 25
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`remaining_percent` ranges from `0` to `100`; `75` means 75% remains. `resets_at` is the next reset time in UTC.
|
||||
|
||||
</Accordion>
|
||||
|
||||
## Rate limits
|
||||
|
||||
This endpoint allows 10 requests per minute per user, shared across API keys and devices. We recommend polling once per minute. A `429` response includes a `Retry-After` header with the number of seconds to wait before retrying.
|
||||
|
||||
## Teams
|
||||
|
||||
All team members share the same included and purchased balances.
|
||||
|
||||
## Reference
|
||||
@@ -0,0 +1,120 @@
|
||||
---
|
||||
title: Cloud usage
|
||||
sidebarTitle: /api/usage
|
||||
openapi: /openapi.yaml GET /api/usage
|
||||
---
|
||||
|
||||
<RequestExample>
|
||||
|
||||
```shell Request
|
||||
curl 'https://ollama.com/api/usage?range=24h' \
|
||||
-H "Authorization: Bearer $OLLAMA_API_KEY"
|
||||
```
|
||||
|
||||
</RequestExample>
|
||||
|
||||
<ResponseExample>
|
||||
|
||||
```json Response (22 empty buckets omitted)
|
||||
{
|
||||
"range": "24h",
|
||||
"scope": "self",
|
||||
"granularity": "hour",
|
||||
"from": "2026-09-30T02:00:00Z",
|
||||
"until": "2026-10-01T02:30:00Z",
|
||||
"totals": {
|
||||
"request_count": 15,
|
||||
"usage_usd": 0.01718,
|
||||
"input_tokens": 106000,
|
||||
"cached_input_tokens": 46000,
|
||||
"output_tokens": 13600
|
||||
},
|
||||
"buckets": [
|
||||
{
|
||||
"from": "2026-09-30T12:00:00Z",
|
||||
"until": "2026-09-30T13:00:00Z",
|
||||
"request_count": 4,
|
||||
"usage_usd": 0.0052,
|
||||
"input_tokens": 24000,
|
||||
"cached_input_tokens": 0,
|
||||
"output_tokens": 3200
|
||||
},
|
||||
{
|
||||
"from": "2026-10-01T00:00:00Z",
|
||||
"until": "2026-10-01T01:00:00Z",
|
||||
"request_count": 8,
|
||||
"usage_usd": 0.0088,
|
||||
"input_tokens": 64000,
|
||||
"cached_input_tokens": 40000,
|
||||
"output_tokens": 8000
|
||||
},
|
||||
{
|
||||
"from": "2026-10-01T02:00:00Z",
|
||||
"until": "2026-10-01T02:30:00Z",
|
||||
"partial": true,
|
||||
"request_count": 3,
|
||||
"usage_usd": 0.00318,
|
||||
"input_tokens": 18000,
|
||||
"cached_input_tokens": 6000,
|
||||
"output_tokens": 2400
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
</ResponseExample>
|
||||
|
||||
New usage may take time to appear.
|
||||
|
||||
For your remaining included and purchased credits, use [Balance](/api/balance).
|
||||
|
||||
Unsupported or repeated query parameters return `400`. Errors follow Ollama's [JSON error format](/api/errors#error-messages).
|
||||
|
||||
## Time ranges
|
||||
|
||||
| Range | Starts at | Bucket size | Buckets |
|
||||
| ----- | ------------------------------------------------- | ----------- | ------- |
|
||||
| `24h` | 24 hours before the start of the current UTC hour | Hour | 25 |
|
||||
| `7d` | Seven days ago at 00:00 UTC | Day | 8 |
|
||||
| `30d` | Thirty days ago at 00:00 UTC | Day | 31 |
|
||||
|
||||
Every range ends at the time of the request. For example, at 02:30 UTC on October 1, `24h` starts at 02:00 UTC on September 30. It returns 24 complete hourly buckets followed by the current 02:00–02:30 bucket. The `7d` and `30d` ranges return seven and thirty complete days, followed by today so far.
|
||||
|
||||
Buckets appear in chronological order, including zeroes for hours or days with no recorded requests. The final bucket covers the current hour or day and has `partial: true`.
|
||||
|
||||
All timestamps are in UTC. Each `from` is inclusive and each `until` is exclusive.
|
||||
|
||||
## Usage totals and buckets
|
||||
|
||||
`totals` summarizes usage across the entire range. `buckets` breaks that usage into hours or days.
|
||||
|
||||
`granularity` is set automatically: `hour` for `24h` and `day` for `7d` and `30d`. The top-level `from` and `until` describe the entire range; each bucket has its own boundaries.
|
||||
|
||||
`partial` indicates whether the hour or day is still in progress, rather than whether all usage has been recorded.
|
||||
|
||||
<Accordion title="Legacy plans">
|
||||
|
||||
For legacy plans, only request counts are available. If a bucket or total includes legacy requests, cost and token counts are omitted.
|
||||
|
||||
</Accordion>
|
||||
|
||||
## Rate limits
|
||||
|
||||
This endpoint allows 10 requests per minute per user, shared across API keys and devices. We recommend polling once per minute. A `429` response includes a `Retry-After` header with the number of seconds to wait before retrying.
|
||||
|
||||
## Team usage
|
||||
|
||||
By default, everyone sees their own usage (`scope=self`).
|
||||
|
||||
Team admins can use `scope=team` to see usage for the whole team:
|
||||
|
||||
```shell
|
||||
curl 'https://ollama.com/api/usage?range=7d&scope=team' \
|
||||
-H "Authorization: Bearer $OLLAMA_API_KEY"
|
||||
```
|
||||
|
||||
## Coming soon
|
||||
|
||||
We plan to add custom time ranges, usage breakdowns by model and API key, and breakdowns by team member for team admins.
|
||||
|
||||
## Reference
|
||||
+29
-19
@@ -2,45 +2,55 @@
|
||||
title: Introduction
|
||||
---
|
||||
|
||||
Use Ollama's API to run and interact with models.
|
||||
Use Ollama's API to build applications with cloud or local models.
|
||||
|
||||
## Get started
|
||||
|
||||
Follow the [quickstart](/quickstart) to install Ollama and make your first request.
|
||||
Call cloud models with an [API key](https://ollama.com/settings/keys). No Ollama installation required. Start with the [API quickstart](/quickstart#build-an-application).
|
||||
|
||||
## Base URL
|
||||
To run models on your computer, [install Ollama and run a local model](/quickstart#run-a-model-locally).
|
||||
|
||||
After installation, Ollama's API is served by default at:
|
||||
## Base URLs
|
||||
|
||||
```
|
||||
http://localhost:11434/api
|
||||
```
|
||||
| API | Direct cloud access | Local Ollama server |
|
||||
| --- | --- | --- |
|
||||
| Ollama | `https://ollama.com/api` | `http://localhost:11434/api` |
|
||||
| OpenAI compatibility | `https://ollama.com/v1` | `http://localhost:11434/v1` |
|
||||
| Anthropic client base URL | `https://ollama.com` | `http://localhost:11434` |
|
||||
|
||||
For running cloud models on [ollama.com](https://ollama.com), the same API is available with the following base URL:
|
||||
The Anthropic client appends `/v1/messages` to its base URL.
|
||||
|
||||
```
|
||||
https://ollama.com/api
|
||||
```
|
||||
Cloud requests need an API key. Local requests do not. To use cloud models through your local server, [sign in to Ollama](/api/authentication#signing-in).
|
||||
|
||||
## Example request
|
||||
Creating or deleting models requires a local Ollama server. [System One](/capabilities/decision) also requires a local server and is not yet available on Ollama Cloud. For supported API features, see [OpenAI compatibility](/api/openai-compatibility) and [Anthropic compatibility](/api/anthropic-compatibility).
|
||||
|
||||
Once Ollama is running, its API is automatically available and can be accessed via `curl`:
|
||||
## Ollama API example
|
||||
|
||||
Set `OLLAMA_API_KEY` to your [API key](https://ollama.com/settings/keys), then call a cloud model:
|
||||
|
||||
```shell
|
||||
curl http://localhost:11434/api/generate -d '{
|
||||
"model": "gemma4",
|
||||
"prompt": "Why is the sky blue?"
|
||||
}'
|
||||
curl https://ollama.com/api/chat \
|
||||
-H "Authorization: Bearer $OLLAMA_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "gemma4:31b",
|
||||
"messages": [{"role": "user", "content": "Why is the sky blue?"}],
|
||||
"stream": false
|
||||
}'
|
||||
```
|
||||
|
||||
Read the answer from `message.content`. For local inference, use `http://localhost:11434/api/chat` with a downloaded model and omit the authorization header.
|
||||
|
||||
## Libraries
|
||||
|
||||
Ollama has official libraries for Python and JavaScript:
|
||||
Use Ollama's official libraries with either a local server or the hosted API:
|
||||
|
||||
- [Python](https://github.com/ollama/ollama-python)
|
||||
- [JavaScript](https://github.com/ollama/ollama-js)
|
||||
|
||||
Several community-maintained libraries are available for Ollama. For a full list, see the [Ollama GitHub repository](https://github.com/ollama/ollama?tab=readme-ov-file#libraries--sdks).
|
||||
For direct cloud access, set the host to `https://ollama.com` and supply your API key in the `Authorization: Bearer` header.
|
||||
|
||||
Browse [community libraries](https://github.com/ollama/ollama?tab=readme-ov-file#libraries--sdks).
|
||||
|
||||
## Versioning
|
||||
|
||||
|
||||
@@ -2,9 +2,43 @@
|
||||
title: OpenAI compatibility
|
||||
---
|
||||
|
||||
Ollama provides compatibility with parts of the [OpenAI API](https://platform.openai.com/docs/api-reference) to help connect existing applications to Ollama.
|
||||
Connect OpenAI clients to Ollama. Ollama supports a subset of the [OpenAI API](https://platform.openai.com/docs/api-reference).
|
||||
|
||||
## Usage
|
||||
## Direct cloud access
|
||||
|
||||
Set your [API key](https://ollama.com/settings/keys) in `OLLAMA_API_KEY`. Install the client with `pip install openai`. No Ollama installation required.
|
||||
|
||||
```python
|
||||
import os
|
||||
from openai import OpenAI
|
||||
|
||||
client = OpenAI(
|
||||
base_url="https://ollama.com/v1",
|
||||
api_key=os.environ["OLLAMA_API_KEY"],
|
||||
)
|
||||
|
||||
response = client.chat.completions.create(
|
||||
model="gemma4:31b",
|
||||
messages=[{"role": "user", "content": "Say hello in one sentence."}],
|
||||
)
|
||||
print(response.choices[0].message.content)
|
||||
```
|
||||
|
||||
The same client supports stateless Responses requests:
|
||||
|
||||
```python
|
||||
response = client.responses.create(
|
||||
model="gemma4:31b",
|
||||
input="Say hello in one sentence.",
|
||||
)
|
||||
print(response.output_text)
|
||||
```
|
||||
|
||||
Choose a [cloud model](https://ollama.com/search?c=cloud). The cloud API does not support stateful Responses, built-in web search through `/v1/responses`, or custom/freeform tool-call replay.
|
||||
|
||||
## Local server usage
|
||||
|
||||
These examples connect to your local Ollama server. The client requires an API key value, but Ollama ignores it. To use cloud models through this server, [sign in to Ollama](/api/authentication#signing-in).
|
||||
|
||||
### Simple `/v1/chat/completions` example
|
||||
|
||||
@@ -208,14 +242,16 @@ curl -X POST http://localhost:11434/v1/chat/completions \
|
||||
- [x] `top_p`
|
||||
- [x] `max_tokens`
|
||||
- [x] `tools`
|
||||
- [x] `reasoning_effort` (`"high"`, `"medium"`, `"low"`, `"max"`, `"none"`)
|
||||
- [x] `reasoning_effort` (model-defined names and compatibility aliases)
|
||||
- [x] `reasoning`
|
||||
- [x] `effort` (`"high"`, `"medium"`, `"low"`, `"max"`, `"none"`)
|
||||
- [x] `effort` (model-defined names and compatibility aliases)
|
||||
- [ ] `tool_choice`
|
||||
- [ ] `logit_bias`
|
||||
- [ ] `user`
|
||||
- [ ] `n`
|
||||
|
||||
`reasoning_effort` and `reasoning.effort` control model thinking. Use `/api/show` to discover each model's supported values and default. When a model uses its metadata to resolve named levels, supported names are applied exactly and unsupported names resolve to the model default. Boolean-only models that support enabling thinking map recognized compatibility efforts to `true`, while `"none"` requests `false`. Models without metadata retain the existing compatibility aliases: `"minimal"` maps to `"low"`, and `"xhigh"` or `"ultra"` map to `"max"`. GPT-OSS keeps its existing OpenAI compatibility conversion: `"minimal"` uses `"low"`, `"xhigh"` or `"ultra"` use `"high"`, and `"none"` requests no thinking output.
|
||||
|
||||
### `/v1/completions`
|
||||
|
||||
#### Supported features
|
||||
@@ -279,7 +315,9 @@ curl -X POST http://localhost:11434/v1/chat/completions \
|
||||
- [x] `dimensions`
|
||||
- [ ] `user`
|
||||
|
||||
### `/v1/responses`
|
||||
### Responses API
|
||||
|
||||
`POST /v1/responses`
|
||||
|
||||
> Note: Added in Ollama v0.13.3
|
||||
|
||||
@@ -302,19 +340,24 @@ Ollama supports the [OpenAI Responses API](https://platform.openai.com/docs/api-
|
||||
- [x] `temperature`
|
||||
- [x] `top_p`
|
||||
- [x] `max_output_tokens`
|
||||
- [x] `reasoning`
|
||||
- [x] `effort` (model-defined names and compatibility aliases)
|
||||
- [x] `think` (Ollama extension; boolean, model-defined string, or `null`)
|
||||
- [ ] `previous_response_id` (stateful v1/responses not supported)
|
||||
- [ ] `conversation` (stateful v1/responses not supported)
|
||||
- [ ] `truncation`
|
||||
|
||||
## Models
|
||||
|
||||
Before using a model, pull it locally `ollama pull`:
|
||||
For local inference, download a model before using it:
|
||||
|
||||
```shell
|
||||
ollama pull llama3.2
|
||||
```
|
||||
|
||||
### Default model names
|
||||
Direct cloud requests use the identifiers from `https://ollama.com/api/tags` and do not require a pull. Through a signed-in local server, select a cloud model such as `gemma4:cloud`.
|
||||
|
||||
### Default model names on a local server
|
||||
|
||||
For tooling that relies on default OpenAI model names such as `gpt-3.5-turbo`, use `ollama cp` to copy an existing model name to a temporary name:
|
||||
|
||||
@@ -338,7 +381,7 @@ curl http://localhost:11434/v1/chat/completions \
|
||||
}'
|
||||
```
|
||||
|
||||
### Setting the context size
|
||||
### Setting the local context size
|
||||
|
||||
The OpenAI API does not have a way of setting the context size for a model. If you need to change the context size, create a `Modelfile` which looks like:
|
||||
|
||||
|
||||
@@ -0,0 +1,165 @@
|
||||
---
|
||||
title: Decision
|
||||
description: Classify text, estimate yes/no probabilities, and score ordered criteria.
|
||||
---
|
||||
|
||||
<Note>
|
||||
Currently available locally in Ollama.
|
||||
</Note>
|
||||
|
||||
Use System One to classify text, answer yes/no questions, or score text against a rubric.
|
||||
|
||||
See the [full list of decision models](https://ollama.com/search?c=decision).
|
||||
|
||||
## Setup
|
||||
|
||||
Start [Ollama](/quickstart) v0.35.0 or later, then download [Nimble](https://ollama.com/library/nimble):
|
||||
|
||||
```shell setup.sh
|
||||
ollama pull nimble
|
||||
```
|
||||
|
||||
Nimble, Tev1, [Clef](https://ollama.com/library/clef), and [Clef Flash](https://ollama.com/library/clef-flash) are decision models. The Clef models can also judge images and require v0.35.1 or later.
|
||||
|
||||
Local requests do not need an API key.
|
||||
|
||||
## Choose a label
|
||||
|
||||
Put your input in `state` and describe the possible answers in `criteria`:
|
||||
|
||||
```shell choice.sh
|
||||
curl http://localhost:11434/v1/systemone \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "nimble",
|
||||
"state": "Our checkout has returned 500 errors since 9am.",
|
||||
"questions": {
|
||||
"label": {
|
||||
"type": "choice",
|
||||
"instructions": "Which label fits this ticket?",
|
||||
"criteria": {
|
||||
"billing": "Payments and refunds",
|
||||
"bug": "Software errors",
|
||||
"account": "Login and account access"
|
||||
}
|
||||
}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
Example response:
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "nimble",
|
||||
"answers": {
|
||||
"label": {
|
||||
"type": "choice",
|
||||
"choice": "bug",
|
||||
"probabilities": {"billing": 0.0125, "bug": 0.9781, "account": 0.0093},
|
||||
"confidence": 0.8906
|
||||
}
|
||||
},
|
||||
"usage": {"input_tokens": 174, "output_tokens": 1}
|
||||
}
|
||||
```
|
||||
|
||||
## Judge an image
|
||||
|
||||
[Clef](https://ollama.com/library/clef) and [Clef Flash](https://ollama.com/library/clef-flash) read images alongside the state. Send base64-encoded PNG, JPEG, or WebP files in `images`; they are shared by all questions, in request order. URLs and data URLs are not accepted. `state` is still required — it holds the context for the decision.
|
||||
|
||||
```shell image.sh
|
||||
curl http://localhost:11434/v1/systemone \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "clef-flash",
|
||||
"state": "A user took this screenshot and wants to know what it shows.",
|
||||
"images": ["<base64-encoded image>"],
|
||||
"questions": {
|
||||
"has_ollama": {
|
||||
"type": "noul",
|
||||
"instructions": "Does this image contain Ollama?"
|
||||
},
|
||||
"app": {
|
||||
"type": "choice",
|
||||
"instructions": "Which application is shown in this screenshot?",
|
||||
"criteria": {
|
||||
"vscode": "Visual Studio Code",
|
||||
"terminal": "A terminal emulator",
|
||||
"browser": "A web browser",
|
||||
"other": "A different application"
|
||||
}
|
||||
}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
Example response:
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "clef-flash",
|
||||
"answers": {
|
||||
"has_ollama": {"type": "noul", "noul": 0.959},
|
||||
"app": {
|
||||
"type": "choice",
|
||||
"choice": "vscode",
|
||||
"probabilities": {"vscode": 0.966, "other": 0.017, "browser": 0.010, "terminal": 0.007},
|
||||
"confidence": 0.868
|
||||
}
|
||||
},
|
||||
"usage": {"input_tokens": 677, "output_tokens": 0}
|
||||
}
|
||||
```
|
||||
|
||||
## Question types
|
||||
|
||||
| Type | Criteria | Result |
|
||||
| --- | --- | --- |
|
||||
| `choice` | 2–26 named options with descriptions | The most likely option and each option's probability |
|
||||
| `noul` | Descriptions for `"false"` and `"true"` | The probability of `true`, from 0 to 1 |
|
||||
| `score` | 2–26 descriptions, ordered from lowest to highest | The probability-weighted average of the zero-based levels |
|
||||
|
||||
For the urgency criteria below:
|
||||
|
||||
```text
|
||||
score = 0 × P(Routine) + 1 × P(Soon) + 2 × P(Immediate)
|
||||
```
|
||||
|
||||
For `choice` and `score`, `confidence` measures how strongly the model favors one answer over the others, from 0 to 1. A higher value does not guarantee the answer is correct.
|
||||
|
||||
## Ask multiple questions
|
||||
|
||||
Combine question types in one request. Here, both questions use the same ticket:
|
||||
|
||||
```shell multiple.sh
|
||||
curl http://localhost:11434/v1/systemone \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "nimble",
|
||||
"state": {"ticket": "I was charged twice. Please refund the extra payment."},
|
||||
"questions": {
|
||||
"refund": {
|
||||
"type": "noul",
|
||||
"instructions": "Is the customer requesting a refund?",
|
||||
"criteria": {
|
||||
"false": "No refund is requested",
|
||||
"true": "The customer requests a refund"
|
||||
}
|
||||
},
|
||||
"urgency": {
|
||||
"type": "score",
|
||||
"instructions": "How urgently does this ticket need a response?",
|
||||
"criteria": [
|
||||
"Routine: no time pressure",
|
||||
"Soon: a customer is inconvenienced",
|
||||
"Immediate: a critical service is unavailable"
|
||||
]
|
||||
}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
This returns a refund probability of `0.9989` and an urgency score of `0.8308` on the 0–2 scale (rounded). Results can vary with the model and configuration.
|
||||
|
||||
See the [API reference](/api/systemone) for request fields, limits, errors, and token usage.
|
||||
@@ -6,21 +6,46 @@ Thinking-capable models emit a `thinking` field that separates their reasoning t
|
||||
|
||||
Use this capability to audit model steps, animate the model *thinking* in a UI, or hide the trace entirely when you only need the final response.
|
||||
|
||||
## Supported models
|
||||
See the [full list of thinking models](https://ollama.com/search?c=thinking).
|
||||
|
||||
- [Qwen 3](https://ollama.com/library/qwen3)
|
||||
- [GPT-OSS](https://ollama.com/library/gpt-oss) *(use `think` levels: `low`, `medium`, `high` — the trace cannot be fully disabled)*
|
||||
- [DeepSeek-v3.1](https://ollama.com/library/deepseek-v3.1)
|
||||
- [DeepSeek R1](https://ollama.com/library/deepseek-r1)
|
||||
- Browse the latest additions under [thinking models](https://ollama.com/search?c=thinking)
|
||||
## Discover a model's thinking controls
|
||||
|
||||
Thinking controls vary by model. Use `/api/show` to discover the values a model supports and the value Ollama uses by default:
|
||||
|
||||
```shell
|
||||
curl http://localhost:11434/api/show -d '{
|
||||
"model": "gpt-oss"
|
||||
}'
|
||||
```
|
||||
|
||||
The response includes a top-level `thinking` object:
|
||||
|
||||
```json
|
||||
{
|
||||
"thinking": {
|
||||
"values": ["low", "medium", "high"],
|
||||
"default": "medium"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- `values` can contain booleans (`true` or `false`) for on/off controls. It can also contain model-defined strings for named levels.
|
||||
- `default` is used when `think` is not set.
|
||||
- `values: [false]` means the model does not support thinking.
|
||||
- If `thinking` is omitted, the model has no thinking metadata. The model might conduct thinking based on its existing behavior.
|
||||
|
||||
## Enable thinking in API calls
|
||||
|
||||
Set the `think` field on chat or generate requests. Most models accept booleans (`true`/`false`) or levels (`low`, `medium`, `high`, `max`), where `max` requests the highest thinking level.
|
||||
Set the `think` field on a chat or generate request:
|
||||
|
||||
GPT-OSS instead expects one of `low`, `medium`, or `high` to tune the trace length.
|
||||
- `true`: request thinking output.
|
||||
- `false`: request no thinking output, if the model permits it.
|
||||
- `null`: use the model default.
|
||||
- A string: select a supported level from `thinking.values`. Use the exact value from `/api/show`. Numbers are not supported.
|
||||
|
||||
The `message.thinking` (chat endpoint) or `thinking` (generate endpoint) field contains the reasoning trace while `message.content` / `response` holds the final answer.
|
||||
If the model resolves named levels from `/api/show` metadata, Ollama applies supported names exactly. Unsupported names use the model default.
|
||||
|
||||
The reasoning output and answer use separate fields. Chat returns `message.thinking` and `message.content`. Generate returns `thinking` and `response`.
|
||||
|
||||
<Tabs>
|
||||
<Tab title="cURL">
|
||||
@@ -68,10 +93,6 @@ The `message.thinking` (chat endpoint) or `thinking` (generate endpoint) field c
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
<Note>
|
||||
GPT-OSS requires `think` to be set to `"low"`, `"medium"`, or `"high"`. Passing `true`/`false` is ignored for that model.
|
||||
</Note>
|
||||
|
||||
## Stream the reasoning trace
|
||||
|
||||
Thinking streams interleave reasoning tokens before answer tokens. Detect the first `thinking` chunk to render a "thinking" section, then switch to the final reply once `message.content` arrives.
|
||||
@@ -141,13 +162,3 @@ Thinking streams interleave reasoning tokens before answer tokens. Detect the fi
|
||||
```
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
## CLI quick reference
|
||||
|
||||
- Enable thinking for a single run: `ollama run deepseek-r1 --think "Where should I visit in Lisbon?"`
|
||||
- Disable thinking: `ollama run deepseek-r1 --think=false "Summarize this article"`
|
||||
- Hide the trace while still using a thinking model: `ollama run deepseek-r1 --hidethinking "Is 9.9 bigger or 9.11?"`
|
||||
- Inside interactive sessions, toggle with `/set think` or `/set nothink`.
|
||||
- GPT-OSS only accepts levels: `ollama run gpt-oss --think=low "Draft a headline"` (replace `low` with `medium` or `high` as needed).
|
||||
|
||||
<Note>Thinking is enabled by default in the CLI and API for supported models.</Note>
|
||||
|
||||
+66
-266
@@ -3,290 +3,90 @@ title: Cloud
|
||||
sidebarTitle: Cloud
|
||||
---
|
||||
|
||||
## Cloud Models
|
||||
Run models in Ollama's cloud from your apps or terminal. No model or app download required.
|
||||
|
||||
Ollama's cloud models are a new kind of model in Ollama that can run without a powerful GPU. Instead, cloud models are automatically offloaded to Ollama's cloud service while offering the same capabilities as local models, making it possible to keep using your local tools while running larger models that wouldn't fit on a personal computer.
|
||||
## API Key
|
||||
|
||||
### Supported models
|
||||
Create an [API key](https://ollama.com/settings/keys), then set it in Bash or Zsh:
|
||||
|
||||
For a list of supported models, see Ollama's [model library](https://ollama.com/search?c=cloud).
|
||||
|
||||
### Running Cloud models
|
||||
|
||||
Ollama's cloud models require an account on [ollama.com](https://ollama.com). To sign in or create an account, run:
|
||||
|
||||
```
|
||||
ollama signin
|
||||
```shell
|
||||
export OLLAMA_API_KEY="your_api_key"
|
||||
```
|
||||
|
||||
<Tabs>
|
||||
<Tab title="CLI">
|
||||
Send a request from the same terminal:
|
||||
|
||||
To run a cloud model, open the terminal and run:
|
||||
|
||||
```
|
||||
ollama run gpt-oss:120b-cloud
|
||||
```
|
||||
|
||||
</Tab>
|
||||
<Tab title="Python">
|
||||
|
||||
First, pull a cloud model so it can be accessed:
|
||||
|
||||
```
|
||||
ollama pull gpt-oss:120b-cloud
|
||||
```
|
||||
|
||||
Next, install [Ollama's Python library](https://github.com/ollama/ollama-python):
|
||||
|
||||
```
|
||||
pip install ollama
|
||||
```
|
||||
|
||||
Next, create and run a simple Python script:
|
||||
|
||||
```python
|
||||
from ollama import Client
|
||||
|
||||
client = Client()
|
||||
|
||||
messages = [
|
||||
{
|
||||
'role': 'user',
|
||||
'content': 'Why is the sky blue?',
|
||||
},
|
||||
]
|
||||
|
||||
for part in client.chat('gpt-oss:120b-cloud', messages=messages, stream=True):
|
||||
print(part['message']['content'], end='', flush=True)
|
||||
```
|
||||
|
||||
</Tab>
|
||||
<Tab title="JavaScript">
|
||||
|
||||
First, pull a cloud model so it can be accessed:
|
||||
|
||||
```
|
||||
ollama pull gpt-oss:120b-cloud
|
||||
```
|
||||
|
||||
Next, install [Ollama's JavaScript library](https://github.com/ollama/ollama-js):
|
||||
|
||||
```
|
||||
npm i ollama
|
||||
```
|
||||
|
||||
Then use the library to run a cloud model:
|
||||
|
||||
```typescript
|
||||
import { Ollama } from "ollama";
|
||||
|
||||
const ollama = new Ollama();
|
||||
|
||||
const response = await ollama.chat({
|
||||
model: "gpt-oss:120b-cloud",
|
||||
messages: [{ role: "user", content: "Explain quantum computing" }],
|
||||
stream: true,
|
||||
});
|
||||
|
||||
for await (const part of response) {
|
||||
process.stdout.write(part.message.content);
|
||||
}
|
||||
```
|
||||
|
||||
</Tab>
|
||||
<Tab title="cURL">
|
||||
|
||||
First, pull a cloud model so it can be accessed:
|
||||
|
||||
```
|
||||
ollama pull gpt-oss:120b-cloud
|
||||
```
|
||||
|
||||
Run the following cURL command to run the command via Ollama's API:
|
||||
|
||||
```
|
||||
curl http://localhost:11434/api/chat -d '{
|
||||
"model": "gpt-oss:120b-cloud",
|
||||
"messages": [{
|
||||
"role": "user",
|
||||
"content": "Why is the sky blue?"
|
||||
}],
|
||||
"stream": false
|
||||
}'
|
||||
```
|
||||
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
## Cloud API access
|
||||
|
||||
Cloud models can also be accessed directly on ollama.com's API. In this mode, ollama.com acts as a remote Ollama host.
|
||||
|
||||
### Authentication
|
||||
|
||||
For direct access to ollama.com's API, first create an [API key](https://ollama.com/settings/keys).
|
||||
|
||||
Then, set the `OLLAMA_API_KEY` environment variable to your API key.
|
||||
|
||||
```
|
||||
export OLLAMA_API_KEY=your_api_key
|
||||
```
|
||||
|
||||
### Listing models
|
||||
|
||||
For models available directly via Ollama's API, models can be listed via:
|
||||
|
||||
```
|
||||
curl https://ollama.com/api/tags
|
||||
```
|
||||
|
||||
### Generating a response
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Python">
|
||||
|
||||
First, install [Ollama's Python library](https://github.com/ollama/ollama-python)
|
||||
|
||||
```
|
||||
pip install ollama
|
||||
```
|
||||
|
||||
Then make a request
|
||||
|
||||
```python
|
||||
import os
|
||||
from ollama import Client
|
||||
|
||||
client = Client(
|
||||
host="https://ollama.com",
|
||||
headers={'Authorization': 'Bearer ' + os.environ.get('OLLAMA_API_KEY')}
|
||||
)
|
||||
|
||||
messages = [
|
||||
{
|
||||
'role': 'user',
|
||||
'content': 'Why is the sky blue?',
|
||||
},
|
||||
]
|
||||
|
||||
for part in client.chat('gpt-oss:120b', messages=messages, stream=True):
|
||||
print(part['message']['content'], end='', flush=True)
|
||||
```
|
||||
|
||||
</Tab>
|
||||
<Tab title="JavaScript">
|
||||
|
||||
First, install [Ollama's JavaScript library](https://github.com/ollama/ollama-js):
|
||||
|
||||
```
|
||||
npm i ollama
|
||||
```
|
||||
|
||||
Next, make a request to the model:
|
||||
|
||||
```typescript
|
||||
import { Ollama } from "ollama";
|
||||
|
||||
const ollama = new Ollama({
|
||||
host: "https://ollama.com",
|
||||
headers: {
|
||||
Authorization: "Bearer " + process.env.OLLAMA_API_KEY,
|
||||
},
|
||||
});
|
||||
|
||||
const response = await ollama.chat({
|
||||
model: "gpt-oss:120b",
|
||||
messages: [{ role: "user", content: "Explain quantum computing" }],
|
||||
stream: true,
|
||||
});
|
||||
|
||||
for await (const part of response) {
|
||||
process.stdout.write(part.message.content);
|
||||
}
|
||||
```
|
||||
|
||||
</Tab>
|
||||
<Tab title="cURL">
|
||||
|
||||
Generate a response via Ollama's chat API:
|
||||
|
||||
```
|
||||
```shell
|
||||
curl https://ollama.com/api/chat \
|
||||
-H "Authorization: Bearer $OLLAMA_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "gpt-oss:120b",
|
||||
"messages": [{
|
||||
"role": "user",
|
||||
"content": "Why is the sky blue?"
|
||||
}],
|
||||
"model": "gemma4:31b",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Say hello in one sentence."
|
||||
}
|
||||
],
|
||||
"stream": false
|
||||
}'
|
||||
```
|
||||
|
||||
</Tab>
|
||||
</Tabs>
|
||||
Read the answer from `message.content`. Keep your API key out of browser code and source control.
|
||||
|
||||
## Local only
|
||||
You can also use [OpenAI](/api/openai-compatibility) or [Anthropic](/api/anthropic-compatibility) clients. Each supports a subset of the original API. No Ollama installation is required.
|
||||
|
||||
Ollama can run in local-only mode by [disabling Ollama's cloud](./faq#how-do-i-disable-ollama-cloud-features) features.
|
||||
## Ollama App or CLI
|
||||
|
||||
[Download Ollama](https://ollama.com/download). Open the app, or run:
|
||||
|
||||
```shell
|
||||
ollama
|
||||
```
|
||||
|
||||
Follow the setup prompts and sign in to use cloud models.
|
||||
|
||||
On macOS, open **Apps** and connect Claude or ChatGPT. For coding agents, run a launch command:
|
||||
|
||||
<CodeGroup>
|
||||
|
||||
```shell Claude Code
|
||||
ollama launch claude
|
||||
```
|
||||
|
||||
```shell Codex CLI
|
||||
ollama launch codex
|
||||
```
|
||||
|
||||
```shell OpenCode
|
||||
ollama launch opencode
|
||||
```
|
||||
|
||||
</CodeGroup>
|
||||
|
||||
For more integrations, open the Ollama app or run `ollama` in your terminal.
|
||||
|
||||
Ollama handles sign-in for connected apps. See the [quickstart](/quickstart) for setup.
|
||||
|
||||
## Models
|
||||
|
||||
Browse [cloud models](https://ollama.com/search?c=cloud), or list them with the API:
|
||||
|
||||
```shell
|
||||
curl https://ollama.com/api/tags
|
||||
```
|
||||
|
||||
For API requests to ollama.com, use the name returned by this list, such as `gemma4:31b`. In the Ollama app or CLI, use `gemma4:cloud`. Cloud models do not need to be downloaded.
|
||||
|
||||
## Usage
|
||||
|
||||
Check your [usage](https://ollama.com/settings/usage) or compare [plans](https://ollama.com/pricing).
|
||||
|
||||
## Retirements
|
||||
|
||||
Ollama will occasionally deprecate and retire older cloud models as newer and better open-source models are released.
|
||||
Tools and applications relying on Ollama Cloud models may need to be updated to keep working. Impacted users will be
|
||||
notified in advance of model deprecation and retirement. Deprecations will be communicated through email and on the
|
||||
Ollama website.
|
||||
[Usage settings](https://ollama.com/settings) show upcoming retirements for models you've recently used. Switch models before the retirement date. Downloaded local models are not affected.
|
||||
|
||||
Ollama Cloud model retirement does not affect local models.
|
||||
## Data handling
|
||||
|
||||
### Upcoming retirements
|
||||
Ollama processes cloud prompts and responses to answer your requests. We do not use them to train models. Read the [Privacy Policy](https://ollama.com/privacy) for details.
|
||||
|
||||
| Retirement date | Model | Recommended alternative |
|
||||
| --- | --- | --- |
|
||||
| July 31, 2026 | `minimax-m2.5` | `minimax-m2.7` |
|
||||
| July 31, 2026 | `kimi-k2.5` | `kimi-k2.6` |
|
||||
|
||||
### Past retirements
|
||||
|
||||
<AccordionGroup>
|
||||
<Accordion title="July 15, 2026">
|
||||
| Model | Recommended alternative |
|
||||
| --- | --- |
|
||||
| `deepseek-v3.1:671b` | `deepseek-v4-flash` |
|
||||
| `deepseek-v3.2` | `deepseek-v4-flash` |
|
||||
| `devstral-2:123b` | `mistral-large-3:675b` |
|
||||
| `devstral-small-2:24b` | |
|
||||
| `ministral-3:14b` | |
|
||||
| `ministral-3:3b` | |
|
||||
| `ministral-3:8b` | |
|
||||
| `gemini-3-flash-preview` | `minimax-m3` |
|
||||
| `gemma3:12b` | `gemma4:31b` |
|
||||
| `gemma3:27b` | `gemma4:31b` |
|
||||
| `gemma3:4b` | `gemma4:31b` |
|
||||
| `glm-4.7` | `glm-5.2` |
|
||||
| `glm-5` | `glm-5.2` |
|
||||
| `minimax-m2.1` | `minimax-m3` |
|
||||
| `qwen3-coder-next` | `qwen3.5:397b` |
|
||||
| `qwen3-coder:480b` | `qwen3.5:397b` |
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="June 30, 2026">
|
||||
| Model | Recommended alternative |
|
||||
| --- | --- |
|
||||
| `rnj-1:8b` | |
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="June 16, 2026">
|
||||
| Model | Recommended alternative |
|
||||
| --- | --- |
|
||||
| `kimi-k2-thinking` | `kimi-k2.6` |
|
||||
| `kimi-k2:1t` | `kimi-k2.6` |
|
||||
| `minimax-m2` | `minimax-m3` |
|
||||
| `glm-4.6` | `glm-5.1` |
|
||||
| `qwen3-next:80b` | `qwen3.5` |
|
||||
| `qwen3-vl:235b` | `qwen3.5` |
|
||||
| `qwen3-vl:235b-instruct` | `qwen3.5` |
|
||||
| `cogito-2.1:671b` | `deepseek-v4-flash` |
|
||||
</Accordion>
|
||||
</AccordionGroup>
|
||||
To use only local models, [disable cloud features](/faq#how-do-i-disable-ollama-cloud-features).
|
||||
|
||||
+14
-2
@@ -96,6 +96,14 @@
|
||||
"source": "/integrations/clawdbot",
|
||||
"destination": "/integrations/openclaw"
|
||||
},
|
||||
{
|
||||
"source": "/integrations/codex-app",
|
||||
"destination": "/integrations/chatgpt"
|
||||
},
|
||||
{
|
||||
"source": "/integrations/codex-app.mdx",
|
||||
"destination": "/integrations/chatgpt"
|
||||
},
|
||||
{
|
||||
"source": "/integrations/poolside",
|
||||
"destination": "/integrations/pool"
|
||||
@@ -120,6 +128,7 @@
|
||||
"/capabilities/streaming",
|
||||
"/capabilities/thinking",
|
||||
"/capabilities/structured-outputs",
|
||||
"/capabilities/decision",
|
||||
"/capabilities/vision",
|
||||
"/capabilities/embeddings",
|
||||
"/capabilities/tool-calling",
|
||||
@@ -171,8 +180,8 @@
|
||||
"/integrations/opencode",
|
||||
"/integrations/deepseek-harness",
|
||||
"/integrations/cline-cli",
|
||||
"/integrations/codex-app",
|
||||
"/integrations/codex",
|
||||
"/integrations/chatgpt",
|
||||
"/integrations/copilot-cli",
|
||||
"/integrations/droid",
|
||||
"/integrations/goose",
|
||||
@@ -241,6 +250,7 @@
|
||||
"POST /api/generate",
|
||||
"POST /api/chat",
|
||||
"POST /api/embed",
|
||||
"POST /v1/systemone",
|
||||
"GET /api/tags",
|
||||
"GET /api/ps",
|
||||
"POST /api/show",
|
||||
@@ -249,7 +259,9 @@
|
||||
"POST /api/pull",
|
||||
"POST /api/push",
|
||||
"DELETE /api/delete",
|
||||
"GET /api/version"
|
||||
"GET /api/version",
|
||||
"/api/cloud-usage",
|
||||
"/api/balance"
|
||||
]
|
||||
}
|
||||
]
|
||||
|
||||
@@ -0,0 +1,4 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 320 320">
|
||||
<style>@media (prefers-color-scheme: dark) { path { fill: #fff; } }</style>
|
||||
<path d="m297.06 130.97c7.26-21.79 4.76-45.66-6.85-65.48-17.46-30.4-52.56-46.04-86.84-38.68-15.25-17.18-37.16-26.95-60.13-26.81-35.04-.08-66.13 22.48-76.91 55.82-22.51 4.61-41.94 18.7-53.31 38.67-17.59 30.32-13.58 68.54 9.92 94.54-7.26 21.79-4.76 45.66 6.85 65.48 17.46 30.4 52.56 46.04 86.84 38.68 15.24 17.18 37.16 26.95 60.13 26.8 35.06.09 66.16-22.49 76.94-55.86 22.51-4.61 41.94-18.7 53.31-38.67 17.57-30.32 13.55-68.51-9.94-94.51zm-120.28 168.11c-14.03.02-27.62-4.89-38.39-13.88.49-.26 1.34-.73 1.89-1.07l63.72-36.8c3.26-1.85 5.26-5.32 5.24-9.07v-89.83l26.93 15.55c.29.14.48.42.52.74v74.39c-.04 33.08-26.83 59.9-59.91 59.97zm-128.84-55.03c-7.03-12.14-9.56-26.37-7.15-40.18.47.28 1.3.79 1.89 1.13l63.72 36.8c3.23 1.89 7.23 1.89 10.47 0l77.79-44.92v31.1c.02.32-.13.63-.38.83l-64.41 37.19c-28.69 16.52-65.33 6.7-81.92-21.95zm-16.77-139.09c7-12.16 18.05-21.46 31.21-26.29 0 .55-.03 1.52-.03 2.2v73.61c-.02 3.74 1.98 7.21 5.23 9.06l77.79 44.91-26.93 15.55c-.27.18-.61.21-.91.08l-64.42-37.22c-28.63-16.58-38.45-53.21-21.95-81.89zm221.26 51.49-77.79-44.92 26.93-15.54c.27-.18.61-.21.91-.08l64.42 37.19c28.68 16.57 38.51 53.26 21.94 81.94-7.01 12.14-18.05 21.44-31.2 26.28v-75.81c.03-3.74-1.96-7.2-5.2-9.06zm26.8-40.34c-.47-.29-1.3-.79-1.89-1.13l-63.72-36.8c-3.23-1.89-7.23-1.89-10.47 0l-77.79 44.92v-31.1c-.02-.32.13-.63.38-.83l64.41-37.16c28.69-16.55 65.37-6.7 81.91 22 6.99 12.12 9.52 26.31 7.15 40.1zm-168.51 55.43-26.94-15.55c-.29-.14-.48-.42-.52-.74v-74.39c.02-33.12 26.89-59.96 60.01-59.94 14.01 0 27.57 4.92 38.34 13.88-.49.26-1.33.73-1.89 1.07l-63.72 36.8c-3.26 1.85-5.26 5.31-5.24 9.06l-.04 89.79zm14.63-31.54 34.65-20.01 34.65 20v40.01l-34.65 20-34.65-20z"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 1.8 KiB |
@@ -0,0 +1,7 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!-- Generated by Pixelmator Pro 3.6.17 -->
|
||||
<svg width="1200" height="1200" viewBox="0 0 1200 1200" xmlns="http://www.w3.org/2000/svg">
|
||||
<g id="g314">
|
||||
<path id="path147" fill="#d97757" stroke="none" d="M 233.959793 800.214905 L 468.644287 668.536987 L 472.590637 657.100647 L 468.644287 650.738403 L 457.208069 650.738403 L 417.986633 648.322144 L 283.892639 644.69812 L 167.597321 639.865845 L 54.926208 633.825623 L 26.577238 627.785339 L 3.3e-05 592.751709 L 2.73832 575.27533 L 26.577238 559.248352 L 60.724873 562.228149 L 136.187973 567.382629 L 249.422867 575.194763 L 331.570496 580.026978 L 453.261841 592.671082 L 472.590637 592.671082 L 475.328857 584.859009 L 468.724915 580.026978 L 463.570557 575.194763 L 346.389313 495.785217 L 219.543671 411.865906 L 153.100723 363.543762 L 117.181267 339.060425 L 99.060455 316.107361 L 91.248367 266.01355 L 123.865784 230.093994 L 167.677887 233.073853 L 178.872513 236.053772 L 223.248367 270.201477 L 318.040283 343.570496 L 441.825592 434.738342 L 459.946411 449.798706 L 467.194672 444.64447 L 468.080597 441.020203 L 459.946411 427.409485 L 392.617493 305.718323 L 320.778564 181.932983 L 288.80542 130.630859 L 280.348999 99.865845 C 277.369171 87.221436 275.194641 76.590698 275.194641 63.624268 L 312.322174 13.20813 L 332.8591 6.604126 L 382.389313 13.20813 L 403.248352 31.328979 L 434.013519 101.71814 L 483.865753 212.537048 L 561.181274 363.221497 L 583.812134 407.919434 L 595.892639 449.315491 L 600.40271 461.959839 L 608.214783 461.959839 L 608.214783 454.711609 L 614.577271 369.825623 L 626.335632 265.61084 L 637.771851 131.516846 L 641.718201 93.745117 L 660.402832 48.483276 L 697.530334 24.000122 L 726.52356 37.852417 L 750.362549 72 L 747.060486 94.067139 L 732.886047 186.201416 L 705.100708 330.52356 L 686.979919 427.167847 L 697.530334 427.167847 L 709.61084 415.087341 L 758.496704 350.174561 L 840.644348 247.490051 L 876.885925 206.738342 L 919.167847 161.71814 L 946.308838 140.29541 L 997.61084 140.29541 L 1035.38269 196.429626 L 1018.469849 254.416199 L 965.637634 321.422852 L 921.825562 378.201538 L 859.006714 462.765259 L 819.785278 530.41626 L 823.409424 535.812073 L 832.75177 534.92627 L 974.657776 504.724915 L 1051.328979 490.872559 L 1142.818848 475.167786 L 1184.214844 494.496582 L 1188.724854 514.147644 L 1172.456421 554.335693 L 1074.604126 578.496765 L 959.838989 601.449829 L 788.939636 641.879272 L 786.845764 643.409485 L 789.261841 646.389343 L 866.255127 653.637634 L 899.194702 655.409424 L 979.812134 655.409424 L 1129.932861 666.604187 L 1169.154419 692.537109 L 1192.671265 724.268677 L 1188.724854 748.429688 L 1128.322144 779.194641 L 1046.818848 759.865845 L 856.590759 714.604126 L 791.355774 698.335754 L 782.335693 698.335754 L 782.335693 703.731567 L 836.69812 756.885986 L 936.322205 846.845581 L 1061.073975 962.81897 L 1067.436279 991.490112 L 1051.409424 1014.120911 L 1034.496704 1011.704712 L 924.885986 929.234924 L 882.604126 892.107544 L 786.845764 811.48999 L 780.483276 811.48999 L 780.483276 819.946289 L 802.550415 852.241699 L 919.087341 1027.409424 L 925.127625 1081.127686 L 916.671204 1098.604126 L 886.469849 1109.154419 L 853.288696 1103.114136 L 785.073914 1007.355835 L 714.684631 899.516785 L 657.906067 802.872498 L 650.979858 806.81897 L 617.476624 1167.704834 L 601.771851 1186.147705 L 565.530212 1200 L 535.328857 1177.046997 L 519.302124 1139.919556 L 535.328857 1066.550537 L 554.657776 970.792053 L 570.362488 894.68457 L 584.536926 800.134277 L 592.993347 768.724976 L 592.429626 766.630859 L 585.503479 767.516968 L 514.22821 865.369263 L 405.825531 1011.865906 L 320.053711 1103.677979 L 299.516815 1111.812256 L 263.919525 1093.369263 L 267.221497 1060.429688 L 287.114136 1031.114136 L 405.825531 880.107361 L 477.422913 786.52356 L 523.651062 732.483276 L 523.328918 724.671265 L 520.590698 724.671265 L 205.288605 929.395935 L 149.154434 936.644409 L 124.993355 914.01355 L 127.973183 876.885986 L 139.409409 864.80542 L 234.201385 799.570435 L 233.879227 799.8927 Z"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 4.0 KiB |
+6
-95
@@ -4,46 +4,10 @@ title: Importing a Model
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Importing a Safetensors adapter](#importing-a-fine-tuned-adapter-from-safetensors-weights)
|
||||
- [Importing a Safetensors model](#importing-a-model-from-safetensors-weights)
|
||||
- [Importing a GGUF file](#importing-a-gguf-based-model-or-adapter)
|
||||
- [Importing a GGUF model](#importing-a-gguf-model)
|
||||
- [Sharing models on ollama.com](#sharing-your-model-on-ollama-com)
|
||||
|
||||
## Importing a fine tuned adapter from Safetensors weights
|
||||
|
||||
First, create a `Modelfile` with a `FROM` command pointing at the base model you used for fine tuning, and an `ADAPTER` command which points to the directory with your Safetensors adapter:
|
||||
|
||||
```dockerfile
|
||||
FROM <base model name>
|
||||
ADAPTER /path/to/safetensors/adapter/directory
|
||||
```
|
||||
|
||||
Make sure that you use the same base model in the `FROM` command as you used to create the adapter otherwise you will get erratic results. Most frameworks use different quantization methods, so it's best to use non-quantized (i.e. non-QLoRA) adapters. If your adapter is in the same directory as your `Modelfile`, use `ADAPTER .` to specify the adapter path.
|
||||
|
||||
Now run `ollama create` from the directory where the `Modelfile` was created:
|
||||
|
||||
```shell
|
||||
ollama create my-model
|
||||
```
|
||||
|
||||
Lastly, test the model:
|
||||
|
||||
```shell
|
||||
ollama run my-model
|
||||
```
|
||||
|
||||
Ollama supports importing adapters based on several different model architectures including:
|
||||
|
||||
- Llama (including Llama 2, Llama 3, Llama 3.1, and Llama 3.2);
|
||||
- Mistral (including Mistral 1, Mistral 2, and Mixtral); and
|
||||
- Gemma (including Gemma 1 and Gemma 2)
|
||||
|
||||
You can create the adapter using a fine tuning framework or tool which can output adapters in the Safetensors format, such as:
|
||||
|
||||
- Hugging Face [fine tuning framework](https://huggingface.co/docs/transformers/en/training)
|
||||
- [Unsloth](https://github.com/unslothai/unsloth)
|
||||
- [MLX](https://github.com/ml-explore/mlx)
|
||||
|
||||
## Importing a model from Safetensors weights
|
||||
|
||||
First, create a `Modelfile` with a `FROM` command which points to the directory containing your Safetensors weights:
|
||||
@@ -66,81 +30,28 @@ Lastly, test the model:
|
||||
ollama run my-model
|
||||
```
|
||||
|
||||
Ollama supports importing models for several different architectures including:
|
||||
## Importing a GGUF model
|
||||
|
||||
- Llama (including Llama 2, Llama 3, Llama 3.1, and Llama 3.2);
|
||||
- Mistral (including Mistral 1, Mistral 2, and Mixtral);
|
||||
- Gemma (including Gemma 1 and Gemma 2); and
|
||||
- Phi3
|
||||
Ollama does not quantize GGUF models during import. Prepare and quantize them first with a GGUF tool such as llama.cpp's [`llama-quantize`](https://github.com/ggml-org/llama.cpp/tree/master/tools/quantize).
|
||||
|
||||
This includes importing foundation models as well as any fine tuned models which have been _fused_ with a foundation model.
|
||||
|
||||
## Importing a GGUF based model or adapter
|
||||
|
||||
If you have a GGUF based model or adapter it is possible to import it into Ollama. You can obtain a GGUF model or adapter by:
|
||||
|
||||
- converting a Safetensors model with the `convert_hf_to_gguf.py` from Llama.cpp;
|
||||
- converting a Safetensors adapter with the `convert_lora_to_gguf.py` from Llama.cpp; or
|
||||
- downloading a model or adapter from a place such as HuggingFace
|
||||
|
||||
To import a GGUF model, create a `Modelfile` containing:
|
||||
To import a single-file GGUF model, create a `Modelfile` containing:
|
||||
|
||||
```dockerfile
|
||||
FROM /path/to/file.gguf
|
||||
```
|
||||
|
||||
For a GGUF adapter, create the `Modelfile` with:
|
||||
For a split GGUF model, keep the original split filenames and use a wildcard that matches every shard.
|
||||
|
||||
```dockerfile
|
||||
FROM <model name>
|
||||
ADAPTER /path/to/file.gguf
|
||||
FROM /path/to/model-*.gguf
|
||||
```
|
||||
|
||||
When importing a GGUF adapter, it's important to use the same base model as the base model that the adapter was created with. You can use:
|
||||
|
||||
- a model from Ollama
|
||||
- a GGUF file
|
||||
- a Safetensors based model
|
||||
|
||||
Once you have created your `Modelfile`, use the `ollama create` command to build the model.
|
||||
|
||||
```shell
|
||||
ollama create my-model
|
||||
```
|
||||
|
||||
## Quantizing a Model
|
||||
|
||||
Quantizing a model allows you to run models faster and with less memory consumption but at reduced accuracy. This allows you to run a model on more modest hardware.
|
||||
|
||||
Ollama can quantize FP16 and FP32 based models into different quantization levels using the `-q/--quantize` flag with the `ollama create` command.
|
||||
|
||||
First, create a Modelfile with the FP16 or FP32 based model you wish to quantize.
|
||||
|
||||
```dockerfile
|
||||
FROM /path/to/my/gemma/f16/model
|
||||
```
|
||||
|
||||
Use `ollama create` to then create the quantized model.
|
||||
|
||||
```shell
|
||||
$ ollama create --quantize q4_K_M mymodel
|
||||
transferring model data
|
||||
quantizing F16 model to Q4_K_M
|
||||
creating new layer sha256:735e246cc1abfd06e9cdcf95504d6789a6cd1ad7577108a70d9902fef503c1bd
|
||||
creating new layer sha256:0853f0ad24e5865173bbf9ffcc7b0f5d56b66fd690ab1009867e45e7d2c4db0f
|
||||
writing manifest
|
||||
success
|
||||
```
|
||||
|
||||
### Supported Quantizations
|
||||
|
||||
- `q8_0`
|
||||
|
||||
#### K-means Quantizations
|
||||
|
||||
- `q4_K_S`
|
||||
- `q4_K_M`
|
||||
|
||||
## Sharing your model on ollama.com
|
||||
|
||||
You can share any model you have created by pushing it to [ollama.com](https://ollama.com) so that other users can try it out.
|
||||
|
||||
+22
-38
@@ -3,54 +3,38 @@ title: Ollama documentation
|
||||
sidebarTitle: Welcome
|
||||
---
|
||||
|
||||
Start building with open models.
|
||||
|
||||
Follow the [quickstart](/quickstart), then choose a model, integration, or API.
|
||||
|
||||
## Models
|
||||
|
||||
Run models locally or use larger models in Ollama's cloud.
|
||||
Use open models in your desktop apps and coding agents, or build them into your application.
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Browse models" icon="star" href="https://ollama.com/search">
|
||||
Find models for chat, coding, vision, embeddings, and reasoning.
|
||||
<Card title="Use your apps" icon="desktop" href="/quickstart#get-started">
|
||||
Connect Claude or ChatGPT on your desktop, or launch a coding agent from your terminal.
|
||||
</Card>
|
||||
<Card title="Build an application" icon="code" href="/quickstart#build-an-application">
|
||||
Call cloud models with Ollama, OpenAI, or Anthropic APIs.
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Explore models
|
||||
|
||||
Choose cloud models or run models on your computer.
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Cloud models" icon="cloud" href="/cloud">
|
||||
Run larger models on Ollama's Cloud without the download.
|
||||
Run models without a local GPU.
|
||||
</Card>
|
||||
<Card title="Local models" icon="hard-drive" href="/quickstart#run-a-model-locally">
|
||||
Download Ollama and run a model on your computer.
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Next steps
|
||||
Browse the [model library](https://ollama.com/search) for coding, vision, embeddings, and reasoning models.
|
||||
|
||||
Connect Ollama to an app, or build with the API.
|
||||
## Documentation
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Integrations" icon="plug" href="/integrations">
|
||||
Connect Ollama to an app, editor, or agent.
|
||||
</Card>
|
||||
|
||||
<Card title="First API request" icon="code" href="/api/introduction">
|
||||
Learn the local and cloud base URLs, then send a request with `curl`.
|
||||
</Card>
|
||||
|
||||
<Card title="Python library" icon="python" href="https://github.com/ollama/ollama-python">
|
||||
Use Ollama from Python.
|
||||
</Card>
|
||||
|
||||
<Card title="JavaScript library" icon="js" href="https://github.com/ollama/ollama-js">
|
||||
Use Ollama from JavaScript or TypeScript.
|
||||
</Card>
|
||||
</CardGroup>
|
||||
- [Integrations](/integrations) — connect an app, editor, or agent.
|
||||
- [API reference](/api/introduction) — use Ollama's API or compatible OpenAI and Anthropic clients.
|
||||
- [Python library](https://github.com/ollama/ollama-python) and [JavaScript library](https://github.com/ollama/ollama-js) — use Ollama from your application.
|
||||
|
||||
## Community
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Discord" icon="discord" href="https://discord.gg/ollama">
|
||||
Join the Ollama Discord.
|
||||
</Card>
|
||||
|
||||
<Card title="Reddit" icon="reddit" href="https://reddit.com/r/ollama">
|
||||
Join the Ollama subreddit.
|
||||
</Card>
|
||||
</CardGroup>
|
||||
Join the conversation on [Discord](https://discord.gg/ollama) or [Reddit](https://reddit.com/r/ollama).
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
---
|
||||
title: ChatGPT Desktop
|
||||
description: Use Ollama models in ChatGPT Desktop.
|
||||
---
|
||||
|
||||
ChatGPT Desktop includes Codex, OpenAI's desktop coding agent. Ollama adds compatible local and cloud models to Codex's model picker alongside ChatGPT's native models.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Ollama v0.34.0 or newer
|
||||
|
||||
If ChatGPT is not installed, Ollama offers to download it during setup.
|
||||
|
||||
## Connect ChatGPT
|
||||
|
||||
1. Open the Ollama desktop app and select **Apps**.
|
||||
2. Toggle **ChatGPT (Desktop)** to On.
|
||||
3. Follow the setup prompts. If ChatGPT is already open, Ollama asks to restart it before the new models appear.
|
||||
4. Open the ChatGPT desktop app and use Codex mode.
|
||||
|
||||
## Choose models
|
||||
|
||||
Open Ollama **Settings**, select **Apps**, then select ChatGPT. Choose up to five compatible local or cloud models and save your changes. Ollama asks to restart ChatGPT when a model change needs one.
|
||||
|
||||
## Disconnect ChatGPT
|
||||
|
||||
Open Ollama **Apps** and turn ChatGPT off. Ollama restores ChatGPT's connection settings and usual profile. If ChatGPT is open, Ollama asks to restart it so the change takes effect.
|
||||
|
||||
## Codex behavior and limits
|
||||
|
||||
- Only requests made with an Ollama model go to Ollama. ChatGPT's native models keep using their usual provider.
|
||||
- Ollama models are for Codex mode. Regular Chat and voice keep using their usual ChatGPT connection.
|
||||
- Codex manages sessions. Ollama routes model requests and compaction when Codex asks for it, but does not store ChatGPT sessions.
|
||||
- Web search uses Ollama's web-search service when you use an Ollama model. It requires Ollama sign-in and is limited to three calls per response.
|
||||
|
||||
## Related integrations
|
||||
|
||||
- Use Codex from a terminal with [Codex CLI](/integrations/codex).
|
||||
- Connect other desktop apps from [Claude Desktop](/integrations/claude-desktop).
|
||||
@@ -2,7 +2,7 @@
|
||||
title: Claude Code
|
||||
---
|
||||
|
||||
[Claude Code](https://code.claude.com/docs/en/overview) is an agentic coding tool that reads your codebase, edits files, and runs commands.
|
||||
[Claude Code](https://code.claude.com/docs/en/overview) reads your codebase, edits files, and runs commands.
|
||||
|
||||
Ollama connects Claude Code to local and cloud models through its Anthropic-compatible API.
|
||||
|
||||
@@ -155,6 +155,23 @@ Claude Code prompts for permission on most actions. To allow the bot to work aut
|
||||
|
||||
See the [plugin README](https://github.com/anthropics/claude-plugins-official/tree/main/external_plugins/telegram) for setup instructions.
|
||||
|
||||
## Connect directly to Ollama Cloud
|
||||
|
||||
Install Claude Code, then set your [API key](/quickstart#build-an-application) in `OLLAMA_API_KEY`. No Ollama installation required.
|
||||
|
||||
In Bash or Zsh, run:
|
||||
|
||||
```shell
|
||||
ANTHROPIC_BASE_URL=https://ollama.com \
|
||||
ANTHROPIC_AUTH_TOKEN="$OLLAMA_API_KEY" \
|
||||
ANTHROPIC_API_KEY="" \
|
||||
claude --model glm-5.3-flash
|
||||
```
|
||||
|
||||
Use `ANTHROPIC_AUTH_TOKEN` for bearer authentication. Claude settings can override shell variables. Update any base URL or credentials in those settings, then run `/status` to check the connection.
|
||||
|
||||
Choose a [cloud model](https://ollama.com/search?c=cloud) that supports tools. Basic chat and file edits work with compatible models. Hosted WebSearch and advanced tool controls are not fully supported. See [Anthropic compatibility](/api/anthropic-compatibility).
|
||||
|
||||
## Manual setup
|
||||
|
||||
<p className="manual-step-title">1. Install Claude Code</p>
|
||||
|
||||
@@ -7,7 +7,7 @@ description: Use Ollama models in Claude on macOS.
|
||||
## Prerequisites
|
||||
|
||||
- [Ollama](https://ollama.com/download/). Windows support coming soon.
|
||||
- To use a local model, [download it](/quickstart#3-start-a-chat) before setup.
|
||||
- To use a local model, [download it](/quickstart#run-a-model-locally) before setup.
|
||||
- To use a cloud model, sign in to Ollama and enable cloud models. Some models require a [paid plan](https://ollama.com/pricing).
|
||||
|
||||
If Claude is not installed, Ollama offers to download it during setup.
|
||||
@@ -15,12 +15,12 @@ If Claude is not installed, Ollama offers to download it during setup.
|
||||
## Set up Claude
|
||||
|
||||
1. Open Ollama and select **Apps**.
|
||||
2. Toggle Claude to On.
|
||||
3. If Claude is not installed, Ollama will download and finish the installation.
|
||||
2. Connect **Claude**.
|
||||
3. Follow the setup prompts to install Claude if needed and restart it when prompted.
|
||||
|
||||
## Switch models
|
||||
|
||||
Open Ollama **Settings**. Under **Apps**, choose a model you want to use, then select **Restart Claude**. You can assign the same Ollama model to more than one option.
|
||||
Open Ollama **Settings → Apps**. Choose your models, then select **Restart Claude**. You can use the same model for more than one option.
|
||||
|
||||
## Supported features
|
||||
|
||||
@@ -31,7 +31,7 @@ Open Ollama **Settings**. Under **Apps**, choose a model you want to use, then s
|
||||
|
||||
## Disconnect Claude
|
||||
|
||||
Open Ollama **Apps** and turn Claude off. Ollama restores Claude's previous configuration. If Claude is open, Ollama asks to restart it so the change takes effect.
|
||||
Open Ollama **Apps** and disconnect Claude. Ollama restores Claude's previous settings. Restart Claude when prompted.
|
||||
|
||||
You can also restore Claude from the terminal:
|
||||
|
||||
|
||||
@@ -13,7 +13,7 @@ Install [Cline](https://docs.cline.bot/getting-started/installing-cline) in your
|
||||
2. Select a model under `Model` or type one (e.g. `qwen3`)
|
||||
3. Update the context window to at least 32K tokens under `Context Window`
|
||||
|
||||
<Note>Coding tools require a larger context window. It is recommended to use a context window of at least 32K tokens. See [Context length](/context-length) for more information.</Note>
|
||||
<Note>Use a [context window](/context-length) of at least 32K tokens for coding.</Note>
|
||||
|
||||
<div style={{ display: 'flex', justifyContent: 'center' }}>
|
||||
<img
|
||||
@@ -32,7 +32,6 @@ Install [Cline](https://docs.cline.bot/getting-started/installing-cline) in your
|
||||
4. Select a model from the list
|
||||
|
||||
|
||||
### Recommended Models
|
||||
### Models
|
||||
|
||||
- `qwen3-coder:480b`
|
||||
- `deepseek-v3.1:671b`
|
||||
Choose a [cloud model](https://ollama.com/search?c=cloud) that supports tool calling.
|
||||
|
||||
@@ -1,82 +0,0 @@
|
||||
---
|
||||
title: Codex App
|
||||
---
|
||||
|
||||
Codex App is OpenAI's desktop coding agent for macOS and Windows. Ollama configures the app to use Ollama's OpenAI-compatible endpoint, so Codex can work with local models and Ollama Cloud models in the desktop app.
|
||||
|
||||
<img
|
||||
src="/images/codex-app-home.png"
|
||||
alt="Codex App with Ollama selected"
|
||||
style={{ borderRadius: "12px" }}
|
||||
/>
|
||||
|
||||
## Install
|
||||
|
||||
Install the [Codex App](https://developers.openai.com/codex/quickstart/) for macOS or Windows.
|
||||
|
||||
<Note>Codex App support is available in Ollama v0.24.0 and newer.</Note>
|
||||
|
||||
|
||||
## Quick setup
|
||||
|
||||
```shell
|
||||
ollama launch codex-app
|
||||
```
|
||||
|
||||
Once Codex App opens, start a task or open a repository as usual.
|
||||
|
||||
## Built-in browser
|
||||
|
||||
Codex App can open local servers and sites in its built-in browser. Annotate directly on the page to request changes.
|
||||
|
||||
<img
|
||||
src="/images/codex-app-annotate.png"
|
||||
alt="Codex App browser annotations"
|
||||
style={{ borderRadius: "12px" }}
|
||||
/>
|
||||
|
||||
## Review mode
|
||||
|
||||
Use review mode to inspect code changes, leave comments, and iterate on fixes without leaving the app.
|
||||
|
||||
<img
|
||||
src="/images/codex-app-review.png"
|
||||
alt="Codex App review comments"
|
||||
style={{ borderRadius: "12px" }}
|
||||
/>
|
||||
|
||||
### Run directly with a model
|
||||
|
||||
```shell
|
||||
ollama launch codex-app --model kimi-k2.6:cloud
|
||||
```
|
||||
|
||||
Use a local model by passing its model name:
|
||||
|
||||
```shell
|
||||
ollama launch codex-app --model gemma4:31b
|
||||
```
|
||||
|
||||
Running `ollama launch codex-app` is persistent and will have your model selected next time you open Codex.
|
||||
|
||||
|
||||
### Restore Codex App
|
||||
|
||||
To switch Codex App back to the profile you were using before `ollama launch codex-app`, run:
|
||||
|
||||
```shell
|
||||
ollama launch codex-app --restore
|
||||
```
|
||||
|
||||
Ollama restores Codex App's settings and configs. If Codex App is open, Ollama asks before restarting it.
|
||||
|
||||
|
||||
The Codex CLI profile managed by `ollama launch codex` is left separate from the Codex App profile.
|
||||
|
||||
Before overwriting Codex App config files, Ollama Launch saves backups under `~/.ollama/backup/codex-app/`. On Windows, `~` resolves to your user profile directory.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
If Codex App does not open after setup, open Codex manually once and run `ollama launch codex-app` again.
|
||||
|
||||
If Codex App is already running and does not switch models, allow Ollama to restart it when prompted, or quit Codex App and run `ollama launch codex-app` again.
|
||||
@@ -5,7 +5,7 @@ title: Codex CLI
|
||||
|
||||
## Install
|
||||
|
||||
Install the [Codex CLI](https://developers.openai.com/codex/cli/). For the desktop app, see [Codex App](/integrations/codex-app).
|
||||
Install the [Codex CLI](https://developers.openai.com/codex/cli/). For the ChatGPT desktop app, see [ChatGPT Desktop](/integrations/chatgpt).
|
||||
|
||||
```
|
||||
npm install -g @openai/codex
|
||||
@@ -13,7 +13,7 @@ npm install -g @openai/codex
|
||||
|
||||
## Usage with Ollama
|
||||
|
||||
<Note>Codex requires a larger context window. It is recommended to use a context window of at least 64k tokens.</Note>
|
||||
<Note>Use a context window of at least 64k tokens for Codex.</Note>
|
||||
|
||||
### Quick setup
|
||||
|
||||
@@ -77,7 +77,7 @@ Then run:
|
||||
codex --profile ollama-launch
|
||||
```
|
||||
|
||||
## Web search
|
||||
## Web search through local Ollama
|
||||
|
||||
Codex web-search requests sent through the Ollama profile are executed by
|
||||
Ollama for both local and cloud models. Sign in with `ollama signin` to use the
|
||||
|
||||
+23
-16
@@ -2,51 +2,58 @@
|
||||
title: Overview
|
||||
---
|
||||
|
||||
Use Ollama from coding agents, personal assistants, and editors.
|
||||
Use Ollama models in your apps and tools.
|
||||
|
||||
Run `ollama launch` to see the latest integrations you can run from the terminal.
|
||||
|
||||
## Code in the terminal
|
||||
## Launch a coding agent
|
||||
|
||||
Run `ollama launch` in a terminal to choose a coding agent.
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Claude Code" icon="/images/launch-icons/claude-code.svg" href="/integrations/claude-code">
|
||||
Terminal coding agent with tools, vision, web search, and long context.
|
||||
Read code, edit files, and run commands.
|
||||
</Card>
|
||||
|
||||
<Card title="OpenCode" icon="/images/launch-icons/opencode.svg" href="/integrations/opencode">
|
||||
Open-source coding agent that edits, runs, and iterates on code.
|
||||
Edit and run code from your terminal.
|
||||
</Card>
|
||||
|
||||
<Card title="DeepSeek Harness" icon="/images/launch-icons/deepseek-harness.svg" href="/integrations/deepseek-harness">
|
||||
DeepSeek's open-source agent harness with subagents and web search.
|
||||
Use DeepSeek's open-source coding agent in your browser.
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Connect an assistant
|
||||
## Connect a desktop app
|
||||
|
||||
Use open models in assistant apps.
|
||||
On macOS, open the [Ollama app](https://ollama.com/download) and select **Apps** in the sidebar. Connect Claude or ChatGPT.
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="Claude" href="/integrations/claude-desktop">
|
||||
Desktop assistant with local and cloud Ollama models.
|
||||
<Card title="Claude Desktop" icon="/images/launch-icons/claude.svg" href="/integrations/claude-desktop">
|
||||
Use local and cloud Ollama models.
|
||||
</Card>
|
||||
|
||||
<Card title="ChatGPT Desktop" icon="/images/launch-icons/chatgpt.svg" href="/integrations/chatgpt">
|
||||
Use Ollama models in Codex mode.
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Run a personal assistant
|
||||
|
||||
Run `ollama launch` in a terminal and choose OpenClaw or Hermes Agent.
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="OpenClaw" icon="/images/launch-icons/openclaw.svg" href="/integrations/openclaw">
|
||||
Personal assistant for messaging apps and everyday tasks.
|
||||
Chat with an assistant through your messaging apps.
|
||||
</Card>
|
||||
|
||||
<Card title="Hermes Agent" icon="/images/launch-icons/hermes-agent.svg" href="/integrations/hermes">
|
||||
Open-source agent with self-improving skills, memory, and messaging.
|
||||
Personal assistant with memory, reusable skills, and messaging.
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## Work in your editor
|
||||
|
||||
Use Ollama models inside your editor.
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card title="VS Code" icon="/images/launch-icons/vscode.svg" href="/integrations/vscode">
|
||||
Use Ollama models in VS Code Chat.
|
||||
Install the Ollama extension to use models in VS Code Chat.
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
+18
-33
@@ -12,15 +12,15 @@ A Modelfile is the blueprint to create and share customized models using Ollama.
|
||||
- [FROM (Required)](#from-required)
|
||||
- [Build from existing model](#build-from-existing-model)
|
||||
- [Build from a Safetensors model](#build-from-a-safetensors-model)
|
||||
- [Build from a GGUF file](#build-from-a-gguf-file)
|
||||
- [Build from a GGUF model](#build-from-a-gguf-model)
|
||||
- [PARAMETER](#parameter)
|
||||
- [Valid Parameters and Values](#valid-parameters-and-values)
|
||||
- [TEMPLATE](#template)
|
||||
- [Template Variables](#template-variables)
|
||||
- [SYSTEM](#system)
|
||||
- [ADAPTER](#adapter)
|
||||
- [LICENSE](#license)
|
||||
- [MESSAGE](#message)
|
||||
- [CAPABILITY](#capability)
|
||||
- [Notes](#notes)
|
||||
|
||||
## Format
|
||||
@@ -38,10 +38,10 @@ INSTRUCTION arguments
|
||||
| [`PARAMETER`](#parameter) | Sets the parameters for how Ollama will run the model. |
|
||||
| [`TEMPLATE`](#template) | The full prompt template to be sent to the model. |
|
||||
| [`SYSTEM`](#system) | Specifies the system message that will be set in the template. |
|
||||
| [`ADAPTER`](#adapter) | Defines the (Q)LoRA adapters to apply to the model. |
|
||||
| [`LICENSE`](#license) | Specifies the legal license. |
|
||||
| [`MESSAGE`](#message) | Specify message history. |
|
||||
| [`REQUIRES`](#requires) | Specify the minimum version of Ollama required by the model. |
|
||||
| [`CAPABILITY`](#capability) | Declare an additional model capability. |
|
||||
|
||||
## Examples
|
||||
|
||||
@@ -119,14 +119,7 @@ FROM <model directory>
|
||||
|
||||
The model directory should contain the Safetensors weights for a supported architecture.
|
||||
|
||||
Currently supported model architectures:
|
||||
|
||||
- Llama (including Llama 2, Llama 3, Llama 3.1, and Llama 3.2)
|
||||
- Mistral (including Mistral 1, Mistral 2, and Mixtral)
|
||||
- Gemma (including Gemma 1 and Gemma 2)
|
||||
- Phi3
|
||||
|
||||
#### Build from a GGUF file
|
||||
#### Build from a GGUF model
|
||||
|
||||
```
|
||||
FROM ./ollama-model.gguf
|
||||
@@ -134,6 +127,12 @@ FROM ./ollama-model.gguf
|
||||
|
||||
The GGUF file location should be specified as an absolute path or relative to the `Modelfile` location.
|
||||
|
||||
For a split GGUF model, keep the original split filenames and use a wildcard that matches every shard. Shards can also be listed with separate `FROM` instructions.
|
||||
|
||||
```
|
||||
FROM ./model-*.gguf
|
||||
```
|
||||
|
||||
### PARAMETER
|
||||
|
||||
The `PARAMETER` instruction defines a parameter that can be set when the model is run.
|
||||
@@ -187,28 +186,6 @@ The `SYSTEM` instruction specifies the system message to be used in the template
|
||||
SYSTEM """<system message>"""
|
||||
```
|
||||
|
||||
### ADAPTER
|
||||
|
||||
The `ADAPTER` instruction specifies a fine tuned LoRA adapter that should apply to the base model. The value of the adapter should be an absolute path or a path relative to the Modelfile. The base model should be specified with a `FROM` instruction. If the base model is not the same as the base model that the adapter was tuned from the behaviour will be erratic.
|
||||
|
||||
#### Safetensor adapter
|
||||
|
||||
```
|
||||
ADAPTER <path to safetensor adapter>
|
||||
```
|
||||
|
||||
Currently supported Safetensor adapters:
|
||||
|
||||
- Llama (including Llama 2, Llama 3, and Llama 3.1)
|
||||
- Mistral (including Mistral 1, Mistral 2, and Mixtral)
|
||||
- Gemma (including Gemma 1 and Gemma 2)
|
||||
|
||||
#### GGUF adapter
|
||||
|
||||
```
|
||||
ADAPTER ./ollama-lora.gguf
|
||||
```
|
||||
|
||||
### LICENSE
|
||||
|
||||
The `LICENSE` instruction allows you to specify the legal license under which the model used with this Modelfile is shared or distributed.
|
||||
@@ -246,6 +223,14 @@ MESSAGE user Is Ontario in Canada?
|
||||
MESSAGE assistant yes
|
||||
```
|
||||
|
||||
### CAPABILITY
|
||||
|
||||
Adds a capability to a GGUF or MLX model without removing inherited or inferred capabilities. Repeat the instruction to add more than one. Declare `decision` only for weights trained for System One's decision prompts:
|
||||
|
||||
```
|
||||
CAPABILITY decision
|
||||
```
|
||||
|
||||
### REQUIRES
|
||||
|
||||
The `REQUIRES` instruction allows you to specify the minimum version of Ollama required by the model.
|
||||
|
||||
+723
-15
@@ -96,11 +96,7 @@ components:
|
||||
type: boolean
|
||||
default: true
|
||||
think:
|
||||
oneOf:
|
||||
- type: boolean
|
||||
- type: string
|
||||
enum: [high, medium, low, max]
|
||||
description: When true, returns separate thinking output in addition to content. Can be a boolean (true/false) or a string ("high", "medium", "low", "max") for supported models, with "max" requesting the highest thinking level.
|
||||
$ref: "#/components/schemas/ThinkValue"
|
||||
raw:
|
||||
type: boolean
|
||||
description: When true, returns the raw response from the model without any prompt templating
|
||||
@@ -294,11 +290,7 @@ components:
|
||||
type: boolean
|
||||
default: true
|
||||
think:
|
||||
oneOf:
|
||||
- type: boolean
|
||||
- type: string
|
||||
enum: [high, medium, low, max]
|
||||
description: When true, returns separate thinking output in addition to content. Can be a boolean (true/false) or a string ("high", "medium", "low", "max") for supported models, with "max" requesting the highest thinking level.
|
||||
$ref: "#/components/schemas/ThinkValue"
|
||||
keep_alive:
|
||||
oneOf:
|
||||
- type: string
|
||||
@@ -410,6 +402,29 @@ components:
|
||||
done:
|
||||
type: boolean
|
||||
description: True for the final event in the stream
|
||||
ThinkValue:
|
||||
description: Controls a model's thinking output. Use `/api/show` to discover the supported values and default for the selected model. `true` requests thinking, `false` requests no thinking output, and `null` uses the model default. String values are model-defined; supported names must match `/api/show` exactly. Numbers are not supported.
|
||||
oneOf:
|
||||
- type: boolean
|
||||
- type: string
|
||||
- type: "null"
|
||||
Thinking:
|
||||
type: object
|
||||
required: [values, default]
|
||||
description: Thinking controls advertised by a model. Models without thinking metadata omit this field.
|
||||
properties:
|
||||
values:
|
||||
type: array
|
||||
description: Values explicitly supported by the model's `think` request field. Booleans represent on/off controls and strings represent model-defined levels. An array containing only `false` identifies a model without thinking support.
|
||||
items:
|
||||
oneOf:
|
||||
- type: boolean
|
||||
- type: string
|
||||
default:
|
||||
description: Value used when `think` is not set. For models that use this metadata for named thinking levels, unsupported names resolve to this default.
|
||||
oneOf:
|
||||
- type: boolean
|
||||
- type: string
|
||||
StatusEvent:
|
||||
type: object
|
||||
properties:
|
||||
@@ -498,6 +513,16 @@ components:
|
||||
parser:
|
||||
type: string
|
||||
description: Name of the parser for the model
|
||||
files:
|
||||
type: object
|
||||
additionalProperties:
|
||||
type: string
|
||||
description: Source file names mapped to their SHA-256 digests. Split GGUF models must include each shard under its original split filename.
|
||||
draft_files:
|
||||
type: object
|
||||
additionalProperties:
|
||||
type: string
|
||||
description: Draft source file names mapped to their SHA-256 digests
|
||||
license:
|
||||
oneOf:
|
||||
- type: string
|
||||
@@ -518,7 +543,18 @@ components:
|
||||
$ref: "#/components/schemas/ChatMessage"
|
||||
quantize:
|
||||
type: string
|
||||
description: Quantization level to apply (e.g. `q4_K_M`, `q8_0`)
|
||||
description: Quantization level to apply during import (e.g. `nvfp4`)
|
||||
draft_quantize:
|
||||
type: string
|
||||
description: Quantization level to apply to draft weights during import
|
||||
requires:
|
||||
type: string
|
||||
description: Minimum Ollama version required by the model
|
||||
capabilities:
|
||||
type: array
|
||||
items:
|
||||
type: string
|
||||
description: Capabilities to add without removing inherited or inferred capabilities (e.g. `decision` for compatible decision models)
|
||||
stream:
|
||||
type: boolean
|
||||
default: true
|
||||
@@ -581,6 +617,8 @@ components:
|
||||
ShowResponse:
|
||||
type: object
|
||||
properties:
|
||||
thinking:
|
||||
$ref: "#/components/schemas/Thinking"
|
||||
parameters:
|
||||
type: string
|
||||
description: Model parameter settings serialized as text
|
||||
@@ -790,6 +828,324 @@ components:
|
||||
items:
|
||||
$ref: "#/components/schemas/TokenLogprob"
|
||||
description: Most likely tokens and their log probabilities at this position
|
||||
SystemOneContent:
|
||||
description: A nonempty string, or an object or array serialized as JSON text. Not interpreted as chat messages or multimodal input.
|
||||
oneOf:
|
||||
- type: string
|
||||
pattern: '\S'
|
||||
- type: object
|
||||
additionalProperties: true
|
||||
- type: array
|
||||
items: {}
|
||||
SystemOneChoiceQuestion:
|
||||
title: Choice
|
||||
type: object
|
||||
required: [type, instructions, criteria]
|
||||
properties:
|
||||
type:
|
||||
type: string
|
||||
enum: [choice]
|
||||
instructions:
|
||||
$ref: "#/components/schemas/SystemOneContent"
|
||||
criteria:
|
||||
type: object
|
||||
description: Option keys mapped to descriptions, or null for a bare label. Keys must not be blank; ties follow the model's option order. The option limit depends on the model.
|
||||
minProperties: 2
|
||||
maxProperties: 255
|
||||
propertyNames:
|
||||
pattern: '\S'
|
||||
additionalProperties:
|
||||
type: [string, "null"]
|
||||
SystemOneNoulQuestion:
|
||||
title: Noul
|
||||
type: object
|
||||
required: [type, instructions]
|
||||
properties:
|
||||
type:
|
||||
type: string
|
||||
enum: [noul]
|
||||
instructions:
|
||||
$ref: "#/components/schemas/SystemOneContent"
|
||||
criteria:
|
||||
type: object
|
||||
description: Optional descriptions for the two outcomes. Omitted entries use model-specific defaults.
|
||||
properties:
|
||||
"false":
|
||||
type: string
|
||||
default: "No"
|
||||
"true":
|
||||
type: string
|
||||
default: "Yes"
|
||||
additionalProperties: false
|
||||
SystemOneScoreQuestion:
|
||||
title: Score
|
||||
type: object
|
||||
required: [type, instructions, criteria]
|
||||
properties:
|
||||
type:
|
||||
type: string
|
||||
enum: [score]
|
||||
instructions:
|
||||
$ref: "#/components/schemas/SystemOneContent"
|
||||
criteria:
|
||||
type: array
|
||||
description: Descriptions ordered from the lowest score (index 0) to the highest. Defines a scale from 0 to the number of criteria minus 1. The maximum number of levels depends on the model.
|
||||
minItems: 2
|
||||
maxItems: 26
|
||||
items:
|
||||
type: string
|
||||
SystemOneRequest:
|
||||
type: object
|
||||
required: [model, state, questions]
|
||||
properties:
|
||||
model:
|
||||
type: string
|
||||
pattern: '\S'
|
||||
description: Local model trained for System One, such as nimble. Requires compatible GGUF weights and a scoring-capable runner; cloud and MLX/Safetensors models are not supported.
|
||||
state:
|
||||
$ref: "#/components/schemas/SystemOneContent"
|
||||
images:
|
||||
type: array
|
||||
description: Base64-encoded images shared by all questions, in request order. Requires Clef or Clef Flash with vision weights. URLs and data URLs are not supported.
|
||||
items:
|
||||
type: string
|
||||
format: byte
|
||||
questions:
|
||||
type: object
|
||||
description: Named questions about the shared state. Answers are not passed to later questions.
|
||||
minProperties: 1
|
||||
maxProperties: 64
|
||||
propertyNames:
|
||||
pattern: '\S'
|
||||
additionalProperties:
|
||||
oneOf:
|
||||
- $ref: "#/components/schemas/SystemOneChoiceQuestion"
|
||||
- $ref: "#/components/schemas/SystemOneNoulQuestion"
|
||||
- $ref: "#/components/schemas/SystemOneScoreQuestion"
|
||||
keep_alive:
|
||||
oneOf:
|
||||
- type: string
|
||||
- type: number
|
||||
description: How long to keep the model loaded after the request, as a duration string (such as 5m) or seconds. Zero unloads after the request; a negative value keeps it loaded. Defaults to the server's keep-alive setting (5m unless configured otherwise).
|
||||
SystemOneProbabilities:
|
||||
type: object
|
||||
description: Probabilities normalized over the supplied candidates, summing to 1 subject to floating-point precision.
|
||||
additionalProperties:
|
||||
type: number
|
||||
minimum: 0
|
||||
maximum: 1
|
||||
SystemOneConfidence:
|
||||
type: number
|
||||
minimum: 0
|
||||
maximum: 1
|
||||
description: Distribution concentration, calculated as 1 - H(p) / ln(N), where H(p) is entropy and N is the candidate count. Zero means uniform probabilities; values near 1 mean one candidate dominates. Not calibrated correctness.
|
||||
SystemOneChoiceAnswer:
|
||||
title: Choice
|
||||
type: object
|
||||
required: [type, choice, probabilities, confidence]
|
||||
properties:
|
||||
type:
|
||||
type: string
|
||||
enum: [choice]
|
||||
choice:
|
||||
type: string
|
||||
description: Option key with the highest probability. Ties follow the model's option order.
|
||||
probabilities:
|
||||
$ref: "#/components/schemas/SystemOneProbabilities"
|
||||
confidence:
|
||||
$ref: "#/components/schemas/SystemOneConfidence"
|
||||
SystemOneNoulAnswer:
|
||||
title: Noul
|
||||
type: object
|
||||
required: [type, noul]
|
||||
properties:
|
||||
type:
|
||||
type: string
|
||||
enum: [noul]
|
||||
noul:
|
||||
type: number
|
||||
minimum: 0
|
||||
maximum: 1
|
||||
description: Probability of true among the false and true candidates. This is a number, not a Boolean.
|
||||
SystemOneScoreAnswer:
|
||||
title: Score
|
||||
type: object
|
||||
required: [type, score, legend, probabilities, confidence]
|
||||
properties:
|
||||
type:
|
||||
type: string
|
||||
enum: [score]
|
||||
score:
|
||||
type: number
|
||||
minimum: 0
|
||||
maximum: 25
|
||||
description: Probability-weighted average of the zero-based criterion indices, from 0 to the number of criteria minus 1. Not rounded to a level or normalized to 0–1.
|
||||
legend:
|
||||
type: object
|
||||
description: Zero-based indices as string keys mapped to the criterion descriptions.
|
||||
additionalProperties:
|
||||
type: string
|
||||
probabilities:
|
||||
$ref: "#/components/schemas/SystemOneProbabilities"
|
||||
description: Probabilities keyed by zero-based criterion indices as strings.
|
||||
confidence:
|
||||
$ref: "#/components/schemas/SystemOneConfidence"
|
||||
SystemOneResponse:
|
||||
type: object
|
||||
required: [model, answers, usage]
|
||||
properties:
|
||||
model:
|
||||
type: string
|
||||
description: Model name from the request.
|
||||
answers:
|
||||
type: object
|
||||
description: Answers keyed by the question names in the request.
|
||||
additionalProperties:
|
||||
oneOf:
|
||||
- $ref: "#/components/schemas/SystemOneChoiceAnswer"
|
||||
- $ref: "#/components/schemas/SystemOneNoulAnswer"
|
||||
- $ref: "#/components/schemas/SystemOneScoreAnswer"
|
||||
usage:
|
||||
type: object
|
||||
required: [input_tokens, output_tokens]
|
||||
properties:
|
||||
input_tokens:
|
||||
type: integer
|
||||
minimum: 0
|
||||
description: Total evaluated input tokens, including image positions. Shared context is counted again when the model scores questions separately.
|
||||
output_tokens:
|
||||
type: integer
|
||||
minimum: 0
|
||||
description: Tokens generated internally for scoring, including prefix preparation and retries. May exceed the question count; not the length of the JSON response.
|
||||
UsageMetrics:
|
||||
type: object
|
||||
required: [request_count]
|
||||
properties:
|
||||
request_count:
|
||||
type: integer
|
||||
minimum: 0
|
||||
description: Number of recorded requests.
|
||||
usage_usd:
|
||||
type: number
|
||||
description: Total value of the requests in USD, including usage covered by your plan and usage paid from purchased credits.
|
||||
input_tokens:
|
||||
type: integer
|
||||
description: Recorded input tokens, including cached input tokens.
|
||||
cached_input_tokens:
|
||||
type: integer
|
||||
description: Recorded input tokens read from cache, already included in input_tokens.
|
||||
output_tokens:
|
||||
type: integer
|
||||
description: Recorded output tokens.
|
||||
UsageBucket:
|
||||
allOf:
|
||||
- $ref: "#/components/schemas/UsageMetrics"
|
||||
- type: object
|
||||
required: [from, until]
|
||||
properties:
|
||||
from:
|
||||
type: string
|
||||
format: date-time
|
||||
description: Start of the bucket in UTC, inclusive.
|
||||
until:
|
||||
type: string
|
||||
format: date-time
|
||||
description: End of the bucket in UTC, exclusive.
|
||||
partial:
|
||||
type: boolean
|
||||
description: Present and true for the current hour or day, which is still in progress.
|
||||
UsageResponse:
|
||||
type: object
|
||||
required: [range, scope, granularity, from, until, totals, buckets]
|
||||
properties:
|
||||
range:
|
||||
type: string
|
||||
enum: [24h, 7d, 30d]
|
||||
description: Selected time range.
|
||||
scope:
|
||||
type: string
|
||||
enum: [self, team]
|
||||
description: Selected usage scope.
|
||||
granularity:
|
||||
type: string
|
||||
enum: [hour, day]
|
||||
description: "Set automatically: hour for 24h, day for 7d and 30d."
|
||||
from:
|
||||
type: string
|
||||
format: date-time
|
||||
description: Start of the range in UTC, inclusive.
|
||||
until:
|
||||
type: string
|
||||
format: date-time
|
||||
description: End of the range in UTC, exclusive.
|
||||
totals:
|
||||
$ref: "#/components/schemas/UsageMetrics"
|
||||
description: Usage across the entire range.
|
||||
buckets:
|
||||
type: array
|
||||
description: Usage in chronological order, including buckets with no recorded requests.
|
||||
items:
|
||||
$ref: "#/components/schemas/UsageBucket"
|
||||
IncludedBalance:
|
||||
title: Included credits
|
||||
type: object
|
||||
required: [balance_usd, allowance_usd, period]
|
||||
properties:
|
||||
balance_usd:
|
||||
type: number
|
||||
description: Remaining included credits in USD.
|
||||
allowance_usd:
|
||||
type: number
|
||||
description: Included credits available for the full period in USD.
|
||||
period:
|
||||
type: object
|
||||
required: [from, until]
|
||||
properties:
|
||||
from:
|
||||
type: string
|
||||
format: date-time
|
||||
description: Start of the included period in UTC, inclusive.
|
||||
until:
|
||||
type: string
|
||||
format: date-time
|
||||
description: End of the included period in UTC, exclusive.
|
||||
LegacyBalanceLimit:
|
||||
type: object
|
||||
required: [remaining_percent, resets_at]
|
||||
properties:
|
||||
remaining_percent:
|
||||
type: number
|
||||
minimum: 0
|
||||
maximum: 100
|
||||
description: Percentage of the limit remaining.
|
||||
resets_at:
|
||||
type: string
|
||||
format: date-time
|
||||
description: Next reset time in UTC.
|
||||
LegacyIncludedBalance:
|
||||
title: Legacy plan limits
|
||||
type: object
|
||||
required: [session, weekly]
|
||||
properties:
|
||||
session:
|
||||
$ref: "#/components/schemas/LegacyBalanceLimit"
|
||||
weekly:
|
||||
$ref: "#/components/schemas/LegacyBalanceLimit"
|
||||
BalanceResponse:
|
||||
type: object
|
||||
required: [included, purchased]
|
||||
properties:
|
||||
included:
|
||||
oneOf:
|
||||
- $ref: "#/components/schemas/IncludedBalance"
|
||||
- $ref: "#/components/schemas/LegacyIncludedBalance"
|
||||
purchased:
|
||||
type: object
|
||||
required: [balance_usd]
|
||||
properties:
|
||||
balance_usd:
|
||||
type: number
|
||||
description: Remaining unexpired purchased credits in USD.
|
||||
ErrorResponse:
|
||||
type: object
|
||||
properties:
|
||||
@@ -797,12 +1153,117 @@ components:
|
||||
type: string
|
||||
description: Error message describing what went wrong
|
||||
paths:
|
||||
/v1/systemone:
|
||||
post:
|
||||
summary: Answer typed questions
|
||||
description: |
|
||||
Answer choice, yes/no, and scoring questions with a local System One model. Requires Ollama v0.35.0 or later. See the [decision guide](/capabilities/decision) for examples.
|
||||
operationId: systemone
|
||||
x-mint:
|
||||
href: /api/systemone
|
||||
metadata:
|
||||
title: System One
|
||||
sidebarTitle: /v1/systemone
|
||||
content: |
|
||||
<Note>
|
||||
Currently available locally in Ollama.
|
||||
</Note>
|
||||
|
||||
- Returns one JSON response; streaming, video, tools, and generation controls are not supported.
|
||||
- Requests without images must fit within 64 KiB; requests with images must fit within 32 MiB, including base64 and JSON. The complete input must fit the loaded context window. Nimble and Tev also require two token positions for scoring. Input is never truncated.
|
||||
x-codeSamples:
|
||||
- lang: bash
|
||||
label: Choose a label
|
||||
source: |
|
||||
curl http://localhost:11434/v1/systemone \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"model": "nimble",
|
||||
"state": "Our checkout has returned 500 errors since 9am.",
|
||||
"questions": {
|
||||
"label": {
|
||||
"type": "choice",
|
||||
"instructions": "Which label fits this ticket?",
|
||||
"criteria": {
|
||||
"billing": "Payments and refunds",
|
||||
"bug": "Software errors",
|
||||
"account": "Login and account access"
|
||||
}
|
||||
}
|
||||
}
|
||||
}'
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/SystemOneRequest"
|
||||
example:
|
||||
model: nimble
|
||||
state: Our checkout has returned 500 errors since 9am.
|
||||
questions:
|
||||
label:
|
||||
type: choice
|
||||
instructions: Which label fits this ticket?
|
||||
criteria:
|
||||
billing: Payments and refunds
|
||||
bug: Software errors
|
||||
account: Login and account access
|
||||
responses:
|
||||
"200":
|
||||
description: Answers and token usage for all questions. Example probabilities and confidence are rounded to four decimal places; results and usage can vary with the model and server configuration.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/SystemOneResponse"
|
||||
example:
|
||||
model: nimble
|
||||
answers:
|
||||
label:
|
||||
type: choice
|
||||
choice: bug
|
||||
probabilities:
|
||||
billing: 0.0125
|
||||
bug: 0.9781
|
||||
account: 0.0093
|
||||
confidence: 0.8906
|
||||
usage:
|
||||
input_tokens: 174
|
||||
output_tokens: 1
|
||||
"400":
|
||||
description: Invalid request, unsupported model or runner, or a rendered prompt that exceeds the loaded context. Cloud models are rejected.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorResponse"
|
||||
"404":
|
||||
description: Local model not found. Download the model before making a request.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorResponse"
|
||||
"413":
|
||||
description: Request body exceeds 64 KiB without images or 32 MiB with images.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorResponse"
|
||||
example:
|
||||
error: request body must not exceed 64 KiB without images
|
||||
"500":
|
||||
description: Model loading, rendering, or scoring failed.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorResponse"
|
||||
/api/generate:
|
||||
post:
|
||||
summary: Generate a response
|
||||
description: Generates a response for the provided prompt
|
||||
operationId: generate
|
||||
x-mint:
|
||||
metadata:
|
||||
sidebarTitle: /api/generate
|
||||
href: /api/generate
|
||||
x-codeSamples:
|
||||
- lang: bash
|
||||
@@ -820,6 +1281,15 @@ paths:
|
||||
"prompt": "Why is the sky blue?",
|
||||
"stream": false
|
||||
}'
|
||||
- lang: bash
|
||||
label: Thinking
|
||||
source: |
|
||||
curl http://localhost:11434/api/generate -d '{
|
||||
"model": "gpt-oss",
|
||||
"prompt": "What is 1+1?",
|
||||
"think": "low",
|
||||
"stream": false
|
||||
}'
|
||||
- lang: bash
|
||||
label: With options
|
||||
source: |
|
||||
@@ -915,6 +1385,8 @@ paths:
|
||||
description: Generate the next chat message in a conversation between a user and an assistant.
|
||||
operationId: chat
|
||||
x-mint:
|
||||
metadata:
|
||||
sidebarTitle: /api/chat
|
||||
href: /api/chat
|
||||
x-codeSamples:
|
||||
- lang: bash
|
||||
@@ -1073,6 +1545,8 @@ paths:
|
||||
description: Creates vector embeddings representing the input text
|
||||
operationId: embed
|
||||
x-mint:
|
||||
metadata:
|
||||
sidebarTitle: /api/embed
|
||||
href: /api/embed
|
||||
x-codeSamples:
|
||||
- lang: bash
|
||||
@@ -1148,6 +1622,8 @@ paths:
|
||||
description: Fetch a list of models and their details
|
||||
operationId: list
|
||||
x-mint:
|
||||
metadata:
|
||||
sidebarTitle: /api/tags
|
||||
href: /api/tags
|
||||
x-codeSamples:
|
||||
- lang: bash
|
||||
@@ -1181,6 +1657,8 @@ paths:
|
||||
description: Retrieve a list of models that are currently running
|
||||
operationId: ps
|
||||
x-mint:
|
||||
metadata:
|
||||
sidebarTitle: /api/ps
|
||||
href: /api/ps
|
||||
x-codeSamples:
|
||||
- lang: bash
|
||||
@@ -1215,6 +1693,9 @@ paths:
|
||||
post:
|
||||
summary: Show model details
|
||||
operationId: show
|
||||
x-mint:
|
||||
metadata:
|
||||
sidebarTitle: /api/show
|
||||
x-codeSamples:
|
||||
- lang: bash
|
||||
label: Default
|
||||
@@ -1229,6 +1710,18 @@ paths:
|
||||
"model": "gemma4",
|
||||
"verbose": true
|
||||
}'
|
||||
- lang: bash
|
||||
label: Thinking controls
|
||||
source: |
|
||||
curl http://localhost:11434/api/show -d '{
|
||||
"model": "gpt-oss"
|
||||
}'
|
||||
- lang: bash
|
||||
label: Thinking controls
|
||||
source: |
|
||||
curl http://localhost:11434/api/show -d '{
|
||||
"model": "gpt-oss"
|
||||
}'
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
@@ -1249,7 +1742,13 @@ paths:
|
||||
license: "Gemma Terms of Use \n\nLast modified: February 21, 2024..."
|
||||
capabilities:
|
||||
- "completion"
|
||||
- "thinking"
|
||||
- "vision"
|
||||
thinking:
|
||||
values:
|
||||
- false
|
||||
- true
|
||||
default: true
|
||||
modified_at: "2025-08-14T15:49:43.634137516-07:00"
|
||||
details:
|
||||
parent_model: ""
|
||||
@@ -1321,6 +1820,8 @@ paths:
|
||||
summary: Create a model
|
||||
operationId: create
|
||||
x-mint:
|
||||
metadata:
|
||||
sidebarTitle: /api/create
|
||||
href: /api/create
|
||||
x-codeSamples:
|
||||
- lang: bash
|
||||
@@ -1340,12 +1841,13 @@ paths:
|
||||
"system": "You are Ollama the llama."
|
||||
}'
|
||||
- lang: bash
|
||||
label: Quantize
|
||||
label: Create from GGUF
|
||||
source: |
|
||||
curl http://localhost:11434/api/create -d '{
|
||||
"model": "llama3.1:8b-instruct-Q4_K_M",
|
||||
"from": "llama3.1:8b-instruct-fp16",
|
||||
"quantize": "q4_K_M"
|
||||
"model": "my-gguf-model",
|
||||
"files": {
|
||||
"model.gguf": "sha256:432f310a77f4650a88d0fd59ecdd7cebed8d684bafea53cbff0473542964f0c3"
|
||||
}
|
||||
}'
|
||||
requestBody:
|
||||
required: true
|
||||
@@ -1371,11 +1873,69 @@ paths:
|
||||
$ref: "#/components/schemas/StatusEvent"
|
||||
example:
|
||||
status: "success"
|
||||
/api/blobs/{digest}:
|
||||
head:
|
||||
summary: Check if a blob exists
|
||||
operationId: headBlob
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/DigestParam"
|
||||
responses:
|
||||
"200":
|
||||
description: Blob exists
|
||||
"400":
|
||||
description: Invalid digest
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorResponse"
|
||||
"404":
|
||||
description: Blob not found
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorResponse"
|
||||
"500":
|
||||
description: Failed to inspect blob
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorResponse"
|
||||
post:
|
||||
summary: Push a blob
|
||||
operationId: createBlob
|
||||
parameters:
|
||||
- $ref: "#/components/parameters/DigestParam"
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/octet-stream:
|
||||
schema:
|
||||
type: string
|
||||
format: binary
|
||||
responses:
|
||||
"200":
|
||||
description: Blob already exists
|
||||
"201":
|
||||
description: Blob created
|
||||
"400":
|
||||
description: Invalid digest or blob content
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorResponse"
|
||||
"500":
|
||||
description: Failed to write blob
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorResponse"
|
||||
/api/copy:
|
||||
post:
|
||||
summary: Copy a model
|
||||
operationId: copy
|
||||
x-mint:
|
||||
metadata:
|
||||
sidebarTitle: /api/copy
|
||||
href: /api/copy
|
||||
x-codeSamples:
|
||||
- lang: bash
|
||||
@@ -1397,11 +1957,31 @@ paths:
|
||||
responses:
|
||||
"200":
|
||||
description: Model successfully copied
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/StatusResponse"
|
||||
example:
|
||||
status: "success"
|
||||
"400":
|
||||
description: Model could not be copied
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorResponse"
|
||||
"404":
|
||||
description: Source model not found
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorResponse"
|
||||
/api/pull:
|
||||
post:
|
||||
summary: Pull a model
|
||||
operationId: pull
|
||||
x-mint:
|
||||
metadata:
|
||||
sidebarTitle: /api/pull
|
||||
href: /api/pull
|
||||
x-codeSamples:
|
||||
- lang: bash
|
||||
@@ -1444,6 +2024,8 @@ paths:
|
||||
summary: Push a model
|
||||
operationId: push
|
||||
x-mint:
|
||||
metadata:
|
||||
sidebarTitle: /api/push
|
||||
href: /api/push
|
||||
x-codeSamples:
|
||||
- lang: bash
|
||||
@@ -1486,6 +2068,8 @@ paths:
|
||||
summary: Delete a model
|
||||
operationId: delete
|
||||
x-mint:
|
||||
metadata:
|
||||
sidebarTitle: /api/delete
|
||||
href: /api/delete
|
||||
x-codeSamples:
|
||||
- lang: bash
|
||||
@@ -1510,6 +2094,9 @@ paths:
|
||||
summary: Get version
|
||||
description: Retrieve the version of the Ollama
|
||||
operationId: version
|
||||
x-mint:
|
||||
metadata:
|
||||
sidebarTitle: /api/version
|
||||
x-codeSamples:
|
||||
- lang: bash
|
||||
label: Default
|
||||
@@ -1524,3 +2111,124 @@ paths:
|
||||
$ref: "#/components/schemas/VersionResponse"
|
||||
example:
|
||||
version: "0.12.6"
|
||||
/api/usage:
|
||||
get:
|
||||
summary: Cloud usage
|
||||
description: Usage statistics for cloud inference, web search, and web fetch.
|
||||
operationId: usage
|
||||
servers:
|
||||
- url: https://ollama.com
|
||||
description: Ollama Cloud
|
||||
security:
|
||||
- bearerAuth: []
|
||||
parameters:
|
||||
- name: range
|
||||
in: query
|
||||
description: Completed UTC hours or days to include, plus the current partial hour or day.
|
||||
schema:
|
||||
type: string
|
||||
enum: [24h, 7d, 30d]
|
||||
default: 7d
|
||||
- name: scope
|
||||
in: query
|
||||
description: Your requests, or your team's requests. Team scope requires being a team admin.
|
||||
schema:
|
||||
type: string
|
||||
enum: [self, team]
|
||||
default: self
|
||||
responses:
|
||||
"200":
|
||||
description: Usage for the selected range and scope.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/UsageResponse"
|
||||
"400":
|
||||
description: Invalid query parameters.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorResponse"
|
||||
"401":
|
||||
description: Invalid credentials.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorResponse"
|
||||
"403":
|
||||
description: Account suspended or team scope requested without team admin access.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorResponse"
|
||||
"429":
|
||||
description: Too many requests.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorResponse"
|
||||
headers:
|
||||
Retry-After:
|
||||
description: Seconds to wait before retrying.
|
||||
schema:
|
||||
type: integer
|
||||
minimum: 1
|
||||
"503":
|
||||
description: Usage reporting is temporarily unavailable.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorResponse"
|
||||
/api/balance:
|
||||
get:
|
||||
summary: Balance
|
||||
description: Check your remaining usage credits.
|
||||
operationId: balance
|
||||
servers:
|
||||
- url: https://ollama.com
|
||||
description: Ollama Cloud
|
||||
security:
|
||||
- bearerAuth: []
|
||||
responses:
|
||||
"200":
|
||||
description: Current included and purchased balances.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/BalanceResponse"
|
||||
"400":
|
||||
description: Invalid query parameters.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorResponse"
|
||||
"401":
|
||||
description: Invalid credentials.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorResponse"
|
||||
"403":
|
||||
description: Account suspended.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorResponse"
|
||||
"429":
|
||||
description: Too many requests.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorResponse"
|
||||
headers:
|
||||
Retry-After:
|
||||
description: Seconds to wait before retrying.
|
||||
schema:
|
||||
type: integer
|
||||
minimum: 1
|
||||
"503":
|
||||
description: Balance reporting is temporarily unavailable.
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: "#/components/schemas/ErrorResponse"
|
||||
|
||||
+269
-28
@@ -2,59 +2,300 @@
|
||||
title: Quickstart
|
||||
---
|
||||
|
||||
Install Ollama and get your first response.
|
||||
Use Ollama in [desktop apps](#connect-a-desktop-app) and [coding agents](#use-a-coding-agent), or [build an application](#build-an-application) with an API key.
|
||||
|
||||
## 1. Download Ollama
|
||||
## Get started
|
||||
|
||||
Ollama runs on macOS, Windows, and Linux.
|
||||
|
||||
<a
|
||||
href="https://ollama.com/download"
|
||||
target="_blank"
|
||||
className="inline-block px-6 py-2 bg-black rounded-full dark:bg-neutral-700 text-white font-normal border-none"
|
||||
>
|
||||
Download Ollama
|
||||
</a>
|
||||
|
||||
## 2. Open the menu
|
||||
|
||||
Run `ollama` in your terminal to open the interactive menu:
|
||||
[Download Ollama](https://ollama.com/download) for macOS, Windows, or Linux. Open the app, or get started from your terminal:
|
||||
|
||||
```shell
|
||||
ollama
|
||||
```
|
||||
|
||||
From the menu you can:
|
||||
Follow the setup prompts. Sign in to use cloud models, or choose a local model.
|
||||
|
||||
- **Run a model** - Start an interactive chat
|
||||
- **Launch tools** - [Claude Code](/integrations/claude-code), [OpenClaw](/integrations/openclaw), [VS Code](/integrations/vscode), and more
|
||||
## Connect a desktop app
|
||||
|
||||
## 3. Start a chat
|
||||
On macOS, open Ollama and select **Apps**. Connect **Claude** or **ChatGPT (Desktop)**. Follow the prompts to install or restart the app.
|
||||
|
||||
Run a model to start your first chat.
|
||||
Choose your Ollama models in **Settings → Apps**.
|
||||
|
||||
- [Claude Desktop](/integrations/claude-desktop) — use Ollama models in Claude.
|
||||
- [ChatGPT Desktop](/integrations/chatgpt) — use Ollama models in Codex mode. Regular Chat and voice use your usual ChatGPT connection.
|
||||
|
||||
## Use a coding agent
|
||||
|
||||
From your project directory, launch your agent:
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Claude Code">
|
||||
|
||||
```shell
|
||||
ollama run gemma4
|
||||
ollama launch claude
|
||||
```
|
||||
|
||||
Cloud models work the same way:
|
||||
Ollama offers to install Claude Code if needed. See [Claude Code](/integrations/claude-code) for details.
|
||||
|
||||
</Tab>
|
||||
<Tab title="Codex CLI">
|
||||
|
||||
[Install Codex CLI](/integrations/codex#install) first, then launch it:
|
||||
|
||||
```shell
|
||||
ollama run gemma4:cloud
|
||||
ollama launch codex
|
||||
```
|
||||
|
||||
Send your first message:
|
||||
See [Codex CLI](/integrations/codex) for setup.
|
||||
|
||||
</Tab>
|
||||
<Tab title="OpenCode">
|
||||
|
||||
```shell
|
||||
ollama launch opencode
|
||||
```
|
||||
|
||||
Ollama offers to install OpenCode if needed. See [OpenCode](/integrations/opencode) for details.
|
||||
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
Choose a model when prompted, then try:
|
||||
|
||||
```text
|
||||
Explain why the sky is blue in one paragraph.
|
||||
Explain how this repository is organized.
|
||||
```
|
||||
|
||||
To leave the chat, type:
|
||||
For more tools, see [Integrations](/integrations). To connect an agent directly with an API key, see [Claude Code](/integrations/claude-code#connect-directly-to-ollama-cloud).
|
||||
|
||||
## Build an application
|
||||
|
||||
Use cloud models with an API key, or run models locally without one. Cloud requests do not require an Ollama installation.
|
||||
|
||||
### 1. Create an API key
|
||||
|
||||
For local models, skip this step and select **Local** under [Send a request](#2-send-a-request).
|
||||
|
||||
For cloud models, sign in or create an account, then create an [API key](https://ollama.com/settings/keys).
|
||||
|
||||
Set your key in the terminal:
|
||||
|
||||
<CodeGroup>
|
||||
|
||||
```shell macOS / Linux
|
||||
export OLLAMA_API_KEY="your_api_key"
|
||||
```
|
||||
|
||||
```powershell Windows
|
||||
$env:OLLAMA_API_KEY = "your_api_key"
|
||||
```
|
||||
|
||||
</CodeGroup>
|
||||
|
||||
Keep your key on your application's server, outside browser code and source control.
|
||||
|
||||
### 2. Send a request
|
||||
|
||||
Make your first request to Ollama
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Cloud">
|
||||
|
||||
These examples use the `gemma4:31b` [cloud model](https://ollama.com/search?c=cloud).
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Ollama API">
|
||||
|
||||
```shell
|
||||
/bye
|
||||
curl https://ollama.com/api/chat \
|
||||
-H "Authorization: Bearer $OLLAMA_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "gemma4:31b",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Say hello in one sentence."
|
||||
}
|
||||
],
|
||||
"stream": false
|
||||
}'
|
||||
```
|
||||
|
||||
Read the answer from `message.content`. See [Ollama's API and libraries](/api/introduction).
|
||||
|
||||
</Tab>
|
||||
<Tab title="OpenAI Chat Completions">
|
||||
|
||||
```shell
|
||||
curl https://ollama.com/v1/chat/completions \
|
||||
-H "Authorization: Bearer $OLLAMA_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "gemma4:31b",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Say hello in one sentence."
|
||||
}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
Read the answer from `choices[0].message.content`. See [OpenAI compatibility](/api/openai-compatibility) for client setup and supported features.
|
||||
|
||||
</Tab>
|
||||
<Tab title="OpenAI Responses">
|
||||
|
||||
```shell
|
||||
curl https://ollama.com/v1/responses \
|
||||
-H "Authorization: Bearer $OLLAMA_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "gemma4:31b",
|
||||
"input": "Say hello in one sentence."
|
||||
}'
|
||||
```
|
||||
|
||||
Read the `output_text` content blocks inside `output`. The OpenAI Python client exposes the text as `response.output_text`.
|
||||
|
||||
Responses requests are stateless: include conversation history in each request. `previous_response_id` and `conversation` aren't supported. See [Responses compatibility](/api/openai-compatibility#responses-api).
|
||||
|
||||
</Tab>
|
||||
<Tab title="Anthropic Messages">
|
||||
|
||||
```shell
|
||||
curl https://ollama.com/v1/messages \
|
||||
-H "Authorization: Bearer $OLLAMA_API_KEY" \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "anthropic-version: 2023-06-01" \
|
||||
-d '{
|
||||
"model": "gemma4:31b",
|
||||
"max_tokens": 1024,
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Say hello in one sentence."
|
||||
}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
Read text blocks from `content`. Direct cloud requests use bearer authentication. See [Anthropic compatibility](/api/anthropic-compatibility) for client setup and supported features.
|
||||
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
OpenAI and Anthropic compatibility each cover a subset of the original API. See [Cloud](/cloud) for models and usage limits.
|
||||
|
||||
</Tab>
|
||||
<Tab title="Local">
|
||||
|
||||
Run [Gemma 4 E2B](https://ollama.com/library/gemma4:e2b) on your computer. No API key required.
|
||||
|
||||
<Note>
|
||||
The model download is about 7.2 GB. We recommend 8 GB of available VRAM, or unified memory on a Mac. Larger context windows need more memory. With less VRAM, Ollama can use system RAM, but responses may be slower.
|
||||
</Note>
|
||||
|
||||
[Download Ollama](https://ollama.com/download) and open the app. On Linux, start the server with `ollama serve` if it is not already running.
|
||||
|
||||
Download the model:
|
||||
|
||||
```shell
|
||||
ollama pull gemma4:e2b
|
||||
```
|
||||
|
||||
Send a request to your local server:
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Ollama API">
|
||||
|
||||
```shell
|
||||
curl http://localhost:11434/api/chat \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "gemma4:e2b",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Say hello in one sentence."
|
||||
}
|
||||
],
|
||||
"stream": false
|
||||
}'
|
||||
```
|
||||
|
||||
</Tab>
|
||||
<Tab title="OpenAI Chat Completions">
|
||||
|
||||
```shell
|
||||
curl http://localhost:11434/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "gemma4:e2b",
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Say hello in one sentence."
|
||||
}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
Read the answer from `choices[0].message.content`. See [OpenAI compatibility](/api/openai-compatibility) for client setup and supported features.
|
||||
|
||||
</Tab>
|
||||
<Tab title="OpenAI Responses">
|
||||
|
||||
```shell
|
||||
curl http://localhost:11434/v1/responses \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "gemma4:e2b",
|
||||
"input": "Say hello in one sentence."
|
||||
}'
|
||||
```
|
||||
|
||||
Read the `output_text` content blocks inside `output`. The OpenAI Python client exposes the text as `response.output_text`.
|
||||
|
||||
Responses requests are stateless: include conversation history in each request. `previous_response_id` and `conversation` aren't supported. See [Responses compatibility](/api/openai-compatibility#responses-api).
|
||||
|
||||
</Tab>
|
||||
<Tab title="Anthropic Messages">
|
||||
|
||||
```shell
|
||||
curl http://localhost:11434/v1/messages \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "anthropic-version: 2023-06-01" \
|
||||
-d '{
|
||||
"model": "gemma4:e2b",
|
||||
"max_tokens": 1024,
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": "Say hello in one sentence."
|
||||
}
|
||||
]
|
||||
}'
|
||||
```
|
||||
|
||||
Read text blocks from `content`. See [Anthropic compatibility](/api/anthropic-compatibility) for client setup and supported features.
|
||||
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
## Run a model locally
|
||||
|
||||
[Download Ollama](https://ollama.com/download), then run:
|
||||
|
||||
```shell
|
||||
ollama run gemma4:e2b
|
||||
```
|
||||
|
||||
Ollama downloads the model and starts a chat on your computer. Type `/bye` to leave.
|
||||
|
||||
## Next steps
|
||||
|
||||
Use a model with an [integration](/integrations), make an [API request](/api/introduction), or browse more [models](https://ollama.com/search).
|
||||
Add [tool calling](/capabilities/tool-calling), [stream responses](/capabilities/streaming), or browse [more models](https://ollama.com/search).
|
||||
|
||||
Reference in New Issue
Block a user