# Core-AI Service

AI agent service built on PydanticAI for infrastructure management and automation.

## Architecture

Core-AI provides two agents with distinct capabilities:

```
┌─────────────────────────────────────────────────────────────┐
│                      Core-AI Service                        │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  ┌──────────────────┐         ┌──────────────────────┐    │
│  │  PydanticAgent   │         │ SimpleLiteLLMAgent   │    │
│  │  (Primary)       │         │  (Fallback)          │    │
│  │                  │         │                      │    │
│  │  • Tool calling  │         │  • No tools          │    │
│  │  • Memory (3-tier)│         │  • Direct LiteLLM   │    │
│  │  • OpenAPI tools │         │  • Minimal overhead  │    │
│  └────────┬─────────┘         └──────────┬───────────┘    │
│           │                              │                 │
│           └──────────┬───────────────────┘                 │
│                      │                                     │
│                      ▼                                     │
│           ┌──────────────────────┐                        │
│           │  PydanticAI Runtime  │                        │
│           │  (Ollama backend)    │                        │
│           └──────────────────────┘                        │
└─────────────────────────────────────────────────────────────┘
```

### PydanticAgent (Primary)

**Endpoint:** `/v1/chat/completions` (default)

Advanced agent using the PydanticAI framework with:

- **Tool Calling:** Automatic function calling with proper validation
- **Memory System:** 3-tier conversation memory (buffer + Qdrant)
- **Local Tools:** Time, calculations, web search (SearXNG)
- **OpenAPI Tools:** Auto-discovered from core-api infrastructure endpoints
- **Streaming Support:** Server-sent events for real-time responses

### SimpleLiteLLMAgent (Fallback)

**Endpoint:** `/v1/chat/simple`

Lightweight agent for direct LLM interaction:

- **No Tools:** Pure conversational mode
- **Direct LiteLLM:** Minimal abstraction layer
- **No Memory:** Stateless request/response
- **Low Latency:** Fastest response times

## Tool System

### Local Tools

Built-in utilities available immediately (defined in `src/tools/local.py`):

- `get_current_time(timezone)` - Timezone-aware time with IANA timezone support
- `get_current_date()` - Current date in ISO format
- `calculate(expression)` - Safe mathematical calculations
- `calculate_date_difference(date1, date2)` - Date arithmetic
- `add_days_to_date(date, days)` - Date manipulation
- `web_search(query, category, max_results)` - SearXNG metasearch integration

### OpenAPI Discovery

Dynamically discovers infrastructure tools from core-api's OpenAPI spec:

- **Auto-Discovery:** Fetches `/openapi.json` on startup
- **REST Mapping:** Converts endpoints to callable functions
- **Prefixed Names:** Tools prefixed with service name (e.g., `core-api__list_containers`)
- **Type Safety:** Preserves parameter types and validation

**Configuration:**
```bash
OPENAPI_ENABLED=true
OPENAPI_ENDPOINTS=http://core-api:8083/openapi.json
```

**List Available Tools:**
```bash
curl http://localhost:8086/v1/tools
```

## Memory System

3-tier multi-tenant memory with per-user data isolation:

### Tier 1: Conversation Buffer (RAM)
- **Storage:** In-memory per-user buffers
- **Scope:** Recent N turns (configurable, default: 10)
- **Speed:** Instant access
- **Purpose:** Fast context for ongoing conversations

### Tier 2: Persistent Storage (Qdrant)
- **Storage:** Per-user Qdrant collections
- **Scope:** Complete conversation history
- **Speed:** Fast retrieval by conversation ID
- **Purpose:** Conversation continuity across sessions

### Tier 3: Semantic Search (Qdrant)
- **Storage:** Same as Tier 2 with vector embeddings
- **Scope:** Cross-conversation semantic search
- **Speed:** Sub-second similarity search
- **Purpose:** Contextual recall across all user conversations

### Multi-Tenancy

- **Per-User Collections:** Each user gets isolated Qdrant collection
- **User ID Format:** Sanitized email (`username_at_domain_com`)
- **GDPR Compliance:** Complete user data deletion support
- **Automatic Isolation:** No cross-user data leakage

**Memory Configuration:**
```bash
MEMORY_ENABLED=true
MEMORY_TIER1_SIZE=10
QDRANT_URL=http://qdrant:6333
EMBEDDING_MODEL=nomic-embed-text
DEFAULT_USER_ID=llmdefault_at_schweitz_net
```

## API Endpoints

### Chat Completions

**POST /v1/chat/completions** (Default: PydanticAI)

OpenAI-compatible chat endpoint using PydanticAgent.

**Request:**
```json
{
  "messages": [
    {"role": "user", "content": "What containers are running?"}
  ],
  "conversation_id": "optional-conversation-id",
  "enable_tools": true,
  "stream": false
}
```

**Response:**
```json
{
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "I found 5 running containers..."
    },
    "finish_reason": "stop"
  }],
  "model": "pydantic",
  "tools_enabled": true,
  "tools_count": 12
}
```

**Streaming:** Set `"stream": true` for SSE response

### Simple Chat

**POST /v1/chat/simple**

No-tools fallback endpoint using SimpleLiteLLMAgent.

Same request/response format as above, but `tools_enabled` will be `false`.

### List Models

**GET /v1/models**

Returns available agent types:
- `pydantic` - PydanticAgent (primary)
- `simple` - SimpleLiteLLMAgent (fallback)

### List Tools

**GET /v1/tools**

Returns all available tools (local + discovered OpenAPI tools).

### Health Check

**GET /health**

Service health status with agent availability.

## Configuration

All configuration via environment variables (see `src/config.py`):

### Core Settings

| Variable | Default | Description |
|----------|---------|-------------|
| `HOST` | `0.0.0.0` | Server host |
| `PORT` | `8086` | Server port |
| `LOG_LEVEL` | `INFO` | Logging level |

### Ollama Integration

| Variable | Default | Description |
|----------|---------|-------------|
| `OLLAMA_BASE_URL` | `http://ollama:11434` | Ollama API URL |
| `AGENT_MODEL` | `mistral-nemo:latest` | Primary model (tool-calling optimized) |
| `OLLAMA_TIMEOUT` | `300` | Request timeout (seconds) |

### System Prompts

| Variable | Default | Description |
|----------|---------|-------------|
| `SYSTEM_PROMPT_VARIANT` | `minimal_agent` | Prompt for SimpleLiteLLMAgent |
| `PYDANTIC_SYSTEM_PROMPT_VARIANT` | `pydantic_agent` | Prompt for PydanticAgent |

### Tool Discovery

| Variable | Default | Description |
|----------|---------|-------------|
| `OPENAPI_ENABLED` | `true` | Enable OpenAPI tool discovery |
| `OPENAPI_ENDPOINTS` | `http://core-api:8083/openapi.json` | OpenAPI spec URLs (comma-separated) |

### Memory System

| Variable | Default | Description |
|----------|---------|-------------|
| `MEMORY_ENABLED` | `true` | Enable conversation memory |
| `MEMORY_TIER1_SIZE` | `10` | Max turns in RAM buffer |
| `QDRANT_URL` | `http://qdrant:6333` | Qdrant vector DB URL |
| `QDRANT_COLLECTION_PREFIX` | `core_ai_user` | Prefix for user collections |
| `EMBEDDING_MODEL` | `nomic-embed-text` | Ollama embedding model |
| `EMBEDDING_DIMENSION` | `768` | Embedding vector size |
| `DEFAULT_USER_ID` | `llmdefault_at_schweitz_net` | Default user (until auth integration) |

## Quick Start

### 1. Install Dependencies

```bash
pip install -r requirements.txt
```

### 2. Configure Environment

Create `.env` file:

```bash
OLLAMA_BASE_URL=http://ollama:11434
AGENT_MODEL=mistral-nemo:latest
QDRANT_URL=http://qdrant:6333
MEMORY_ENABLED=true
OPENAPI_ENABLED=true
```

### 3. Start Service

```bash
python main.py
```

Service available at `http://localhost:8086`

### 4. Test Chat

```bash
curl -X POST http://localhost:8086/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "messages": [
      {"role": "user", "content": "What time is it in Amsterdam?"}
    ],
    "enable_tools": true
  }'
```

The agent will automatically use the `get_current_time` tool.

## Testing

### Unit Tests

```bash
# Run all tests
pytest tests/ -v

# Run specific test suite
pytest tests/test_ai_flow_quality.py -v

# Run with coverage
pytest tests/ --cov=src --cov-report=html
```

### Integration Tests

Quality tests for end-to-end AI flows:

```bash
pytest tests/test_ai_flow_quality.py -v
```

See `tests/QUALITY_TESTS.md` for test documentation.

### Manual Testing

```bash
# Test PydanticAgent (with tools)
curl -X POST http://localhost:8086/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"messages": [{"role": "user", "content": "Calculate 123 * 456"}]}'

# Test SimpleLiteLLMAgent (no tools)
curl -X POST http://localhost:8086/v1/chat/simple \
  -H 'Content-Type: application/json' \
  -d '{"messages": [{"role": "user", "content": "Hello!"}]}'

# List available tools
curl http://localhost:8086/v1/tools

# Health check
curl http://localhost:8086/health
```

## Docker Deployment

### Build

```bash
docker build -t core-ai:latest .
```

### Run

```bash
docker run -d \
  --name core-ai \
  -p 8086:8086 \
  -e OLLAMA_BASE_URL=http://ollama:11434 \
  -e QDRANT_URL=http://qdrant:6333 \
  -e AGENT_MODEL=mistral-nemo:latest \
  --network docker-dataplane \
  core-ai:latest
```

### Using Docker Compose

```bash
docker-compose -f ../../stacks/core-ai.yml up
```

## Project Structure

```
services/core-ai/
├── main.py                          # HTTP server (aiohttp)
├── src/
│   ├── agents/
│   │   ├── __init__.py             # Agent exports
│   │   ├── pydantic_agent.py       # PydanticAgent (primary)
│   │   └── simple.py               # SimpleLiteLLMAgent (fallback)
│   ├── memory/
│   │   ├── manager.py              # Multi-tenant memory manager
│   │   ├── tier1_buffer.py         # RAM conversation buffer
│   │   ├── qdrant_memory.py        # Qdrant persistent + semantic
│   │   ├── base.py                 # Base memory interfaces
│   │   └── schemas.py              # Memory data schemas
│   ├── tools/
│   │   ├── local.py                # Local utility tools
│   │   ├── openapi_discovery.py    # OpenAPI tool discovery
│   │   └── registry.py             # Tool registration system
│   ├── config.py                   # Configuration (Pydantic Settings)
│   ├── prompts.py                  # System prompts
│   └── utils.py                    # Utilities
├── tests/
│   ├── test_ai_flow_quality.py     # End-to-end AI quality tests
│   └── QUALITY_TESTS.md            # Test documentation
├── requirements.txt
├── Dockerfile
└── README.md (this file)
```

## Development

### Adding Local Tools

Edit `src/tools/local.py`:

```python
from src.tools.registry import register_tool

@register_tool
async def my_new_tool(param: str) -> str:
    """
    Tool description for LLM.

    Args:
        param: Parameter description

    Returns:
        Result description
    """
    # Implementation
    return f"Result: {param}"
```

Tool automatically available to PydanticAgent.

### Adding OpenAPI Sources

Add endpoints to configuration:

```bash
OPENAPI_ENDPOINTS=http://core-api:8083/openapi.json,http://automation:8080/openapi.json
```

Tools auto-discovered on startup with service prefix:
- `core-api__list_containers`
- `automation__deploy_stack`

### Modifying System Prompts

Edit `src/prompts.py`:

```python
PROMPTS = {
    "pydantic_agent": "Your custom PydanticAgent prompt...",
    "minimal_agent": "Your custom SimpleLiteLLMAgent prompt..."
}
```

Update environment:
```bash
PYDANTIC_SYSTEM_PROMPT_VARIANT=pydantic_agent
```

### Memory System Usage

Memory automatically managed per user:

```python
from src.memory import get_memory_manager_for_user

# Get user's memory manager
memory = get_memory_manager_for_user(user_id="user_at_example_com")

# Memory automatically used by PydanticAgent when conversation_id provided
# See: src/agents/pydantic_agent.py
```

## Troubleshooting

### PydanticAI Not Available

**Error:** `PydanticAI not available. Install with: pip install pydantic-ai`

**Solution:**
```bash
pip install pydantic-ai
```

### Tools Not Discovered

**Issue:** `/v1/tools` returns empty list or only local tools

**Check:**
1. Verify `OPENAPI_ENABLED=true`
2. Check core-api is running: `curl http://core-api:8083/openapi.json`
3. Review logs for discovery errors: `docker logs core-ai`

### Memory Errors

**Issue:** Memory operations failing

**Check:**
1. Verify Qdrant running: `curl http://qdrant:6333/collections`
2. Check embedding model available: `docker exec ollama ollama list | grep nomic-embed-text`
3. Review logs for initialization errors

### Model Timeouts

**Issue:** Requests timing out

**Solutions:**
1. Increase timeout: `OLLAMA_TIMEOUT=600`
2. Use smaller model: `AGENT_MODEL=mistral-tools:7b`
3. Check GPU access: `docker exec ollama nvidia-smi`

### Tool Calling Failures

**Issue:** Agent not using tools correctly

**Check:**
1. Verify model supports tool calling: `mistral-nemo`, `mistral-tools:7b`
2. Test with `enable_tools=false` to isolate issue
3. Review tool logs: Look for `🔧 TOOL CALL:` in logs

## Model Recommendations

### For Tool Calling (PydanticAgent)

- **mistral-nemo:latest** (default) - Best balance
- **mistral-tools:7b** - Faster, less accurate
- **llama3.1:8b** - Good alternative

### For Simple Chat (SimpleLiteLLMAgent)

- **gemma2:9b** - Fast conversational
- **llama3.2:3b** - Minimal resources
- Any model works (no tool calling required)

## Migration Notes

This service has migrated from:
- **ADK (Agent Development Kit)** → PydanticAI
- **LangChain/LangGraph** → PydanticAI native
- **OllamaNativeAgent** → Removed (superseded by PydanticAgent)

All references to these frameworks have been removed. The codebase now exclusively uses PydanticAI for agent orchestration.

## Contributing

When making changes:
1. Add tests in `tests/`
2. Update docstrings
3. Test with both agents (`/v1/chat/completions` and `/v1/chat/simple`)
4. Verify tool discovery works
5. Test memory persistence

## License

Part of the portainer-core project.