portainer-core/services/core-ai/ARCHITECTURE.md

Core-AI Dual-Mode Architecture

Overview

Core-AI provides two AI endpoints with different complexity levels:

1. Simple Mode - Direct LiteLLM (existing)
2. ADK Mode - Full Google ADK with tool calling (new)

Core-API becomes a pure tools platform providing REST endpoints.

---

Architecture Diagram

┌─────────────────────────────────────────────────────────────┐
│                         Core-AI                             │
│                                                             │
│  ┌─────────────────────┐      ┌─────────────────────┐     │
│  │   Simple Endpoint   │      │    ADK Endpoint     │     │
│  │ /v1/chat/simple     │      │  /v1/chat/adk       │     │
│  │                     │      │                     │     │
│  │  SimpleLiteLLMAgent │      │   ADKAgent          │     │
│  │       ↓             │      │      ↓              │     │
│  │    LiteLLM          │      │   ADK Runtime       │     │
│  │       ↓             │      │      ↓              │     │
│  │    [No Tools]       │      │   Tool Registry     │     │
│  └─────────────────────┘      │      ↓              │     │
│                               │   REST Calls ───────┼─────┼─┐
│                               └─────────────────────┘     │ │
└───────────────────────────────────────────────────────────┘ │
                                                              │
         ┌────────────────────────────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────────────────────────────┐
│                        Core-API                             │
│                     (Tools Platform)                        │
│                                                             │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐        │
│  │   System    │  │  Services   │  │   Docker    │        │
│  │   Tools     │  │   Tools     │  │   Tools     │        │
│  │   /system/* │  │ /services/* │  │  /docker/*  │        │
│  └─────────────┘  └─────────────┘  └─────────────┘        │
│                                                             │
│  [No AI Agent - Pure REST API]                             │
└─────────────────────────────────────────────────────────────┘
         │
         ▼
┌─────────────────────────────────────────────────────────────┐
│                    External Systems                         │
│         (Portainer, Uptime Kuma, Ollama, etc.)             │
└─────────────────────────────────────────────────────────────┘

---

API Routes

Core-AI Routes

Endpoint	Mode	Agent	Tools	Use Case
POST /v1/chat/simple	Simple	SimpleLiteLLMAgent	None	Fast Q&A, text generation
POST /v1/chat/adk	ADK	ADKAgent	Full toolset	Complex tasks, orchestration
GET /health	N/A	N/A	N/A	Health check

Core-API Routes (No Changes - Tools Only)

Endpoint	Purpose
GET /v1/system/status	System status
GET /v1/services/*	Service management
GET /v1/docker/*	Docker operations
POST /v1/tools/*	Tool execution

---

Request/Response Formats

Simple Mode

POST /v1/chat/simple
{
  "messages": [
    {"role": "user", "content": "What is 2+2?"}
  ],
  "stream": false
}

Response:
{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "model": "simple",
  "choices": [{
    "message": {"role": "assistant", "content": "4"},
    "finish_reason": "stop"
  }]
}

ADK Mode

POST /v1/chat/adk
{
  "messages": [
    {"role": "user", "content": "Check the system status and tell me if everything is ok"}
  ],
  "stream": true
}

Response (streaming):
data: {"type": "tool_call", "tool": "get_system_status", ...}
data: {"type": "tool_result", "result": {...}}
data: {"type": "content", "content": "Everything looks good..."}
data: {"type": "content", "finish_reason": "stop"}

---

File Structure

services/core-ai/
├── main.py                          # HTTP server with both routes
├── src/
│   ├── __init__.py
│   ├── config.py                    # Configuration
│   ├── prompts.py                   # System prompts (simple + ADK)
│   ├── agents/
│   │   ├── __init__.py
│   │   ├── simple.py                # SimpleLiteLLMAgent (existing)
│   │   └── adk_agent.py             # ADKAgent (new)
│   └── tools/
│       ├── __init__.py
│       ├── registry.py              # ADK tool registry
│       ├── system_tools.py          # System tools (REST calls to core-api)
│       ├── service_tools.py         # Service tools (REST calls to core-api)
│       └── knowledge_tools.py       # Knowledge tools (web search, etc.)
├── diagnostics/
│   ├── check_ollama.py
│   ├── test_litellm_direct.py
│   └── test_adk_direct.py           # New: Test ADK without HTTP
├── tests/
│   ├── test_01_environment.py       # ✓ Existing
│   ├── test_02_litellm_raw.py       # ✓ Existing
│   ├── test_03_message_format.py    # ✓ Existing
│   ├── test_04_agent.py             # ✓ Existing (simple agent)
│   ├── test_05_api.py               # ✓ Existing (simple API)
│   ├── test_06_adk_setup.py         # New: ADK initialization
│   ├── test_07_adk_tools.py         # New: ADK tool registration
│   ├── test_08_adk_agent.py         # New: ADK agent logic
│   ├── test_09_adk_tool_calling.py  # New: ADK tool execution
│   ├── test_10_adk_api.py           # New: ADK API endpoint
│   ├── run_all_tests.sh
│   └── README.md
└── README.md

---

Implementation Phases

Phase 1: ADK Agent Setup ✓

• Create src/agents/adk_agent.py
• Initialize ADK runtime
• Test basic ADK completion
• Create diagnostic: diagnostics/test_adk_direct.py
• Create test: tests/test_06_adk_setup.py

Phase 2: Tool Integration

• Create tool registry with ADK FunctionTool format
• Implement REST-based tools (call core-api endpoints)
• Test tool registration
• Create test: tests/test_07_adk_tools.py

Phase 3: ADK Agent with Tools

• Integrate tools into ADK agent
• Test tool calling flow
• Verify REST calls to core-api
• Create tests: test_08_adk_agent.py, test_09_adk_tool_calling.py

Phase 4: API Routes

• Add /v1/chat/adk endpoint
• Rename existing to /v1/chat/simple (keep /v1/chat/completions as alias)
• Test both endpoints
• Create test: tests/test_10_adk_api.py

Phase 5: Documentation & Cleanup

• Update README.md
• Update test documentation
• Add architecture diagrams
• Document migration from core-api

---

Tool Design: REST-First

All tools in core-ai make REST calls to core-api:

# Example: System Status Tool
@log_tool_call
async def get_system_status() -> str:
    """Get current system status from core-api."""
    async with httpx.AsyncClient() as client:
        response = await client.get(f"{CORE_API_BASE_URL}/system/status")
        data = response.json()
        return json.dumps(data, indent=2)

Benefits:

• Clean separation: core-ai = AI, core-api = tools
• Tools can be used by both AI and direct API calls
• Easy to test tools independently
• No code duplication

---

Testing Strategy

Layer 1-5: Simple Mode (Existing)

Already tested and passing ✓

Layer 6: ADK Setup

# Test ADK runtime initialization
# Test ADK basic completion (no tools)
# Test ADK message handling

Layer 7: ADK Tools

# Test tool registration
# Test tool discovery
# Test REST connectivity to core-api

Layer 8: ADK Agent

# Test agent with tools
# Test prompt handling
# Test error handling

Layer 9: ADK Tool Calling

# Test tool invocation
# Test tool results
# Test multi-tool workflows

Layer 10: ADK API

# Test /v1/chat/adk endpoint
# Test streaming with tools
# Test non-streaming with tools

---

Configuration

Environment Variables

# Existing
OLLAMA_BASE_URL=http://ollama:11434
AGENT_MODEL=gemma2:9b-instruct-q5_K_M
SYSTEM_PROMPT_VARIANT=minimal_agent
HOST=0.0.0.0
PORT=8086

# New
CORE_API_BASE_URL=http://core-api:8083/v1  # For tool REST calls
ADK_ENABLED=true                            # Enable ADK endpoint
SIMPLE_ENABLED=true                         # Enable simple endpoint
ADK_SYSTEM_PROMPT_VARIANT=adk_agent        # Different prompt for ADK

Prompts

PROMPTS = {
    "minimal_agent": "You are a helpful assistant.",  # Simple mode
    "adk_agent": """You are a system management assistant with access to tools.
    Use tools when needed to answer questions about system status, services, and docker."""
}

---

Migration Path (Core-API)

Later cleanup - not in this phase:

1. Remove AI agent code from core-api
2. Remove ADK dependencies from core-api
3. Keep only REST endpoints
4. Update core-api to be pure API
5. Redirect any AI requests to core-ai

---

Backward Compatibility

• /v1/chat/completions → alias for /v1/chat/simple
• Existing clients keep working
• New clients can choose mode

---

Performance Considerations

Aspect	Simple Mode	ADK Mode
Latency	~0.5-1s	~1-3s (with tools)
Overhead	Minimal	ADK runtime
Memory	Low	Medium (tool registry)
Use Case	Fast Q&A	Complex tasks

---

Next Steps

1. ✅ Design architecture (this document)
2. ⏳ Implement Phase 1: ADK Agent Setup
3. ⏳ Implement Phase 2: Tool Integration
4. ⏳ Implement Phase 3: ADK Agent with Tools
5. ⏳ Implement Phase 4: API Routes
6. ⏳ Implement Phase 5: Documentation

Let's start with Phase 1!