portainer-core/services/core-ai

Core-AI Service

AI agent service built on PydanticAI for infrastructure management and automation.

Architecture

Core-AI provides two agents with distinct capabilities:

┌─────────────────────────────────────────────────────────────┐
│                      Core-AI Service                        │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  ┌──────────────────┐         ┌──────────────────────┐    │
│  │  PydanticAgent   │         │ SimpleLiteLLMAgent   │    │
│  │  (Primary)       │         │  (Fallback)          │    │
│  │                  │         │                      │    │
│  │  • Tool calling  │         │  • No tools          │    │
│  │  • Memory (3-tier)│         │  • Direct LiteLLM   │    │
│  │  • OpenAPI tools │         │  • Minimal overhead  │    │
│  └────────┬─────────┘         └──────────┬───────────┘    │
│           │                              │                 │
│           └──────────┬───────────────────┘                 │
│                      │                                     │
│                      ▼                                     │
│           ┌──────────────────────┐                        │
│           │  PydanticAI Runtime  │                        │
│           │  (Ollama backend)    │                        │
│           └──────────────────────┘                        │
└─────────────────────────────────────────────────────────────┘

PydanticAgent (Primary)

Endpoint: /v1/chat/completions (default)

Advanced agent using the PydanticAI framework with:

• Tool Calling: Automatic function calling with proper validation
• Memory System: 3-tier conversation memory (buffer + Qdrant)
• Local Tools: Time, calculations, web search (SearXNG)
• OpenAPI Tools: Auto-discovered from core-api infrastructure endpoints
• Streaming Support: Server-sent events for real-time responses

SimpleLiteLLMAgent (Fallback)

Endpoint: /v1/chat/simple

Lightweight agent for direct LLM interaction:

• No Tools: Pure conversational mode
• Direct LiteLLM: Minimal abstraction layer
• No Memory: Stateless request/response
• Low Latency: Fastest response times

Tool System

Local Tools

Built-in utilities available immediately (defined in src/tools/local.py):

• get_current_time(timezone) - Timezone-aware time with IANA timezone support
• get_current_date() - Current date in ISO format
• calculate(expression) - Safe mathematical calculations
• calculate_date_difference(date1, date2) - Date arithmetic
• add_days_to_date(date, days) - Date manipulation
• web_search(query, category, max_results) - SearXNG metasearch integration

OpenAPI Discovery

Dynamically discovers infrastructure tools from core-api's OpenAPI spec:

• Auto-Discovery: Fetches /openapi.json on startup
• REST Mapping: Converts endpoints to callable functions
• Prefixed Names: Tools prefixed with service name (e.g., core-api__list_containers)
• Type Safety: Preserves parameter types and validation

Configuration:

OPENAPI_ENABLED=true
OPENAPI_ENDPOINTS=http://core-api:8083/openapi.json

List Available Tools:

curl http://localhost:8086/v1/tools

Memory System

3-tier multi-tenant memory with per-user data isolation:

Tier 1: Conversation Buffer (RAM)

• Storage: In-memory per-user buffers
• Scope: Recent N turns (configurable, default: 10)
• Speed: Instant access
• Purpose: Fast context for ongoing conversations

Tier 2: Persistent Storage (Qdrant)

• Storage: Per-user Qdrant collections
• Scope: Complete conversation history
• Speed: Fast retrieval by conversation ID
• Purpose: Conversation continuity across sessions

Tier 3: Semantic Search (Qdrant)

• Storage: Same as Tier 2 with vector embeddings
• Scope: Cross-conversation semantic search
• Speed: Sub-second similarity search
• Purpose: Contextual recall across all user conversations

Multi-Tenancy

• Per-User Collections: Each user gets isolated Qdrant collection
• User ID Format: Sanitized email (username_at_domain_com)
• GDPR Compliance: Complete user data deletion support
• Automatic Isolation: No cross-user data leakage

Memory Configuration:

MEMORY_ENABLED=true
MEMORY_TIER1_SIZE=10
QDRANT_URL=http://qdrant:6333
EMBEDDING_MODEL=nomic-embed-text
DEFAULT_USER_ID=llmdefault_at_schweitz_net

API Endpoints

Chat Completions

POST /v1/chat/completions (Default: PydanticAI)

OpenAI-compatible chat endpoint using PydanticAgent.

Request:

{
  "messages": [
    {"role": "user", "content": "What containers are running?"}
  ],
  "conversation_id": "optional-conversation-id",
  "enable_tools": true,
  "stream": false
}

Response:

{
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "I found 5 running containers..."
    },
    "finish_reason": "stop"
  }],
  "model": "pydantic",
  "tools_enabled": true,
  "tools_count": 12
}

Streaming: Set "stream": true for SSE response

Simple Chat

POST /v1/chat/simple

No-tools fallback endpoint using SimpleLiteLLMAgent.

Same request/response format as above, but tools_enabled will be false.

List Models

GET /v1/models

Returns available agent types:

• pydantic - PydanticAgent (primary)
• simple - SimpleLiteLLMAgent (fallback)

List Tools

GET /v1/tools

Returns all available tools (local + discovered OpenAPI tools).

Health Check

GET /health

Service health status with agent availability.

Configuration

All configuration via environment variables (see src/config.py):

Core Settings

Variable	Default	Description
HOST	0.0.0.0	Server host
PORT	8086	Server port
LOG_LEVEL	INFO	Logging level

Ollama Integration

Variable	Default	Description
OLLAMA_BASE_URL	http://ollama:11434	Ollama API URL
AGENT_MODEL	mistral-nemo:latest	Primary model (tool-calling optimized)
OLLAMA_TIMEOUT	300	Request timeout (seconds)

System Prompts

Variable	Default	Description
SYSTEM_PROMPT_VARIANT	minimal_agent	Prompt for SimpleLiteLLMAgent
PYDANTIC_SYSTEM_PROMPT_VARIANT	pydantic_agent	Prompt for PydanticAgent

Tool Discovery

Variable	Default	Description
OPENAPI_ENABLED	true	Enable OpenAPI tool discovery
OPENAPI_ENDPOINTS	http://core-api:8083/openapi.json	OpenAPI spec URLs (comma-separated)

Memory System

Variable	Default	Description
MEMORY_ENABLED	true	Enable conversation memory
MEMORY_TIER1_SIZE	10	Max turns in RAM buffer
QDRANT_URL	http://qdrant:6333	Qdrant vector DB URL
QDRANT_COLLECTION_PREFIX	core_ai_user	Prefix for user collections
EMBEDDING_MODEL	nomic-embed-text	Ollama embedding model
EMBEDDING_DIMENSION	768	Embedding vector size
DEFAULT_USER_ID	llmdefault_at_schweitz_net	Default user (until auth integration)

Quick Start

1. Install Dependencies

pip install -r requirements.txt

2. Configure Environment

Create .env file:

OLLAMA_BASE_URL=http://ollama:11434
AGENT_MODEL=mistral-nemo:latest
QDRANT_URL=http://qdrant:6333
MEMORY_ENABLED=true
OPENAPI_ENABLED=true

3. Start Service

python main.py

Service available at http://localhost:8086

4. Test Chat

curl -X POST http://localhost:8086/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "messages": [
      {"role": "user", "content": "What time is it in Amsterdam?"}
    ],
    "enable_tools": true
  }'

The agent will automatically use the get_current_time tool.

Testing

Unit Tests

# Run all tests
pytest tests/ -v

# Run specific test suite
pytest tests/test_ai_flow_quality.py -v

# Run with coverage
pytest tests/ --cov=src --cov-report=html

Integration Tests

Quality tests for end-to-end AI flows:

pytest tests/test_ai_flow_quality.py -v

See tests/QUALITY_TESTS.md for test documentation.

Manual Testing

# Test PydanticAgent (with tools)
curl -X POST http://localhost:8086/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"messages": [{"role": "user", "content": "Calculate 123 * 456"}]}'

# Test SimpleLiteLLMAgent (no tools)
curl -X POST http://localhost:8086/v1/chat/simple \
  -H 'Content-Type: application/json' \
  -d '{"messages": [{"role": "user", "content": "Hello!"}]}'

# List available tools
curl http://localhost:8086/v1/tools

# Health check
curl http://localhost:8086/health

Docker Deployment

Build

docker build -t core-ai:latest .

Run

docker run -d \
  --name core-ai \
  -p 8086:8086 \
  -e OLLAMA_BASE_URL=http://ollama:11434 \
  -e QDRANT_URL=http://qdrant:6333 \
  -e AGENT_MODEL=mistral-nemo:latest \
  --network docker-dataplane \
  core-ai:latest

Using Docker Compose

docker-compose -f ../../stacks/core-ai.yml up

Project Structure

services/core-ai/
├── main.py                          # HTTP server (aiohttp)
├── src/
│   ├── agents/
│   │   ├── __init__.py             # Agent exports
│   │   ├── pydantic_agent.py       # PydanticAgent (primary)
│   │   └── simple.py               # SimpleLiteLLMAgent (fallback)
│   ├── memory/
│   │   ├── manager.py              # Multi-tenant memory manager
│   │   ├── tier1_buffer.py         # RAM conversation buffer
│   │   ├── qdrant_memory.py        # Qdrant persistent + semantic
│   │   ├── base.py                 # Base memory interfaces
│   │   └── schemas.py              # Memory data schemas
│   ├── tools/
│   │   ├── local.py                # Local utility tools
│   │   ├── openapi_discovery.py    # OpenAPI tool discovery
│   │   └── registry.py             # Tool registration system
│   ├── config.py                   # Configuration (Pydantic Settings)
│   ├── prompts.py                  # System prompts
│   └── utils.py                    # Utilities
├── tests/
│   ├── test_ai_flow_quality.py     # End-to-end AI quality tests
│   └── QUALITY_TESTS.md            # Test documentation
├── requirements.txt
├── Dockerfile
└── README.md (this file)

Development

Adding Local Tools

Edit src/tools/local.py:

from src.tools.registry import register_tool

@register_tool
async def my_new_tool(param: str) -> str:
    """
    Tool description for LLM.

    Args:
        param: Parameter description

    Returns:
        Result description
    """
    # Implementation
    return f"Result: {param}"

Tool automatically available to PydanticAgent.

Adding OpenAPI Sources

Add endpoints to configuration:

OPENAPI_ENDPOINTS=http://core-api:8083/openapi.json,http://automation:8080/openapi.json

Tools auto-discovered on startup with service prefix:

• core-api__list_containers
• automation__deploy_stack

Modifying System Prompts

Edit src/prompts.py:

PROMPTS = {
    "pydantic_agent": "Your custom PydanticAgent prompt...",
    "minimal_agent": "Your custom SimpleLiteLLMAgent prompt..."
}

Update environment:

PYDANTIC_SYSTEM_PROMPT_VARIANT=pydantic_agent

Memory System Usage

Memory automatically managed per user:

from src.memory import get_memory_manager_for_user

# Get user's memory manager
memory = get_memory_manager_for_user(user_id="user_at_example_com")

# Memory automatically used by PydanticAgent when conversation_id provided
# See: src/agents/pydantic_agent.py

Troubleshooting

PydanticAI Not Available

Error: PydanticAI not available. Install with: pip install pydantic-ai

Solution:

pip install pydantic-ai

Tools Not Discovered

Issue: /v1/tools returns empty list or only local tools

Check:

1. Verify OPENAPI_ENABLED=true
2. Check core-api is running: curl http://core-api:8083/openapi.json
3. Review logs for discovery errors: docker logs core-ai

Memory Errors

Issue: Memory operations failing

Check:

1. Verify Qdrant running: curl http://qdrant:6333/collections
2. Check embedding model available: docker exec ollama ollama list | grep nomic-embed-text
3. Review logs for initialization errors

Model Timeouts

Issue: Requests timing out

Solutions:

1. Increase timeout: OLLAMA_TIMEOUT=600
2. Use smaller model: AGENT_MODEL=mistral-tools:7b
3. Check GPU access: docker exec ollama nvidia-smi

Tool Calling Failures

Issue: Agent not using tools correctly

Check:

1. Verify model supports tool calling: mistral-nemo, mistral-tools:7b
2. Test with enable_tools=false to isolate issue
3. Review tool logs: Look for 🔧 TOOL CALL: in logs

Model Recommendations

For Tool Calling (PydanticAgent)

• mistral-nemo:latest (default) - Best balance
• mistral-tools:7b - Faster, less accurate
• llama3.1:8b - Good alternative

For Simple Chat (SimpleLiteLLMAgent)

• gemma2:9b - Fast conversational
• llama3.2:3b - Minimal resources
• Any model works (no tool calling required)

Migration Notes

This service has migrated from:

• ADK (Agent Development Kit) → PydanticAI
• LangChain/LangGraph → PydanticAI native
• OllamaNativeAgent → Removed (superseded by PydanticAgent)

All references to these frameworks have been removed. The codebase now exclusively uses PydanticAI for agent orchestration.

Contributing

When making changes:

1. Add tests in tests/
2. Update docstrings
3. Test with both agents (/v1/chat/completions and /v1/chat/simple)
4. Verify tool discovery works
5. Test memory persistence

License

Part of the portainer-core project.