# Core-AI Dual-Mode Architecture ## Overview Core-AI provides **two AI endpoints** with different complexity levels: 1. **Simple Mode** - Direct LiteLLM (existing) 2. **ADK Mode** - Full Google ADK with tool calling (new) Core-API becomes a **pure tools platform** providing REST endpoints. --- ## Architecture Diagram ``` ┌─────────────────────────────────────────────────────────────┐ │ Core-AI │ │ │ │ ┌─────────────────────┐ ┌─────────────────────┐ │ │ │ Simple Endpoint │ │ ADK Endpoint │ │ │ │ /v1/chat/simple │ │ /v1/chat/adk │ │ │ │ │ │ │ │ │ │ SimpleLiteLLMAgent │ │ ADKAgent │ │ │ │ ↓ │ │ ↓ │ │ │ │ LiteLLM │ │ ADK Runtime │ │ │ │ ↓ │ │ ↓ │ │ │ │ [No Tools] │ │ Tool Registry │ │ │ └─────────────────────┘ │ ↓ │ │ │ │ REST Calls ───────┼─────┼─┐ │ └─────────────────────┘ │ │ └───────────────────────────────────────────────────────────┘ │ │ ┌────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ Core-API │ │ (Tools Platform) │ │ │ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │ │ System │ │ Services │ │ Docker │ │ │ │ Tools │ │ Tools │ │ Tools │ │ │ │ /system/* │ │ /services/* │ │ /docker/* │ │ │ └─────────────┘ └─────────────┘ └─────────────┘ │ │ │ │ [No AI Agent - Pure REST API] │ └─────────────────────────────────────────────────────────────┘ │ ▼ ┌─────────────────────────────────────────────────────────────┐ │ External Systems │ │ (Portainer, Uptime Kuma, Ollama, etc.) │ └─────────────────────────────────────────────────────────────┘ ``` --- ## API Routes ### Core-AI Routes | Endpoint | Mode | Agent | Tools | Use Case | |----------|------|-------|-------|----------| | `POST /v1/chat/simple` | Simple | SimpleLiteLLMAgent | None | Fast Q&A, text generation | | `POST /v1/chat/adk` | ADK | ADKAgent | Full toolset | Complex tasks, orchestration | | `GET /health` | N/A | N/A | N/A | Health check | ### Core-API Routes (No Changes - Tools Only) | Endpoint | Purpose | |----------|---------| | `GET /v1/system/status` | System status | | `GET /v1/services/*` | Service management | | `GET /v1/docker/*` | Docker operations | | `POST /v1/tools/*` | Tool execution | --- ## Request/Response Formats ### Simple Mode ```json POST /v1/chat/simple { "messages": [ {"role": "user", "content": "What is 2+2?"} ], "stream": false } Response: { "id": "chatcmpl-...", "object": "chat.completion", "model": "simple", "choices": [{ "message": {"role": "assistant", "content": "4"}, "finish_reason": "stop" }] } ``` ### ADK Mode ```json POST /v1/chat/adk { "messages": [ {"role": "user", "content": "Check the system status and tell me if everything is ok"} ], "stream": true } Response (streaming): data: {"type": "tool_call", "tool": "get_system_status", ...} data: {"type": "tool_result", "result": {...}} data: {"type": "content", "content": "Everything looks good..."} data: {"type": "content", "finish_reason": "stop"} ``` --- ## File Structure ``` services/core-ai/ ├── main.py # HTTP server with both routes ├── src/ │ ├── __init__.py │ ├── config.py # Configuration │ ├── prompts.py # System prompts (simple + ADK) │ ├── agents/ │ │ ├── __init__.py │ │ ├── simple.py # SimpleLiteLLMAgent (existing) │ │ └── adk_agent.py # ADKAgent (new) │ └── tools/ │ ├── __init__.py │ ├── registry.py # ADK tool registry │ ├── system_tools.py # System tools (REST calls to core-api) │ ├── service_tools.py # Service tools (REST calls to core-api) │ └── knowledge_tools.py # Knowledge tools (web search, etc.) ├── diagnostics/ │ ├── check_ollama.py │ ├── test_litellm_direct.py │ └── test_adk_direct.py # New: Test ADK without HTTP ├── tests/ │ ├── test_01_environment.py # ✓ Existing │ ├── test_02_litellm_raw.py # ✓ Existing │ ├── test_03_message_format.py # ✓ Existing │ ├── test_04_agent.py # ✓ Existing (simple agent) │ ├── test_05_api.py # ✓ Existing (simple API) │ ├── test_06_adk_setup.py # New: ADK initialization │ ├── test_07_adk_tools.py # New: ADK tool registration │ ├── test_08_adk_agent.py # New: ADK agent logic │ ├── test_09_adk_tool_calling.py # New: ADK tool execution │ ├── test_10_adk_api.py # New: ADK API endpoint │ ├── run_all_tests.sh │ └── README.md └── README.md ``` --- ## Implementation Phases ### Phase 1: ADK Agent Setup ✓ - [x] Create `src/agents/adk_agent.py` - [x] Initialize ADK runtime - [x] Test basic ADK completion - [x] Create diagnostic: `diagnostics/test_adk_direct.py` - [x] Create test: `tests/test_06_adk_setup.py` ### Phase 2: Tool Integration - [ ] Create tool registry with ADK FunctionTool format - [ ] Implement REST-based tools (call core-api endpoints) - [ ] Test tool registration - [ ] Create test: `tests/test_07_adk_tools.py` ### Phase 3: ADK Agent with Tools - [ ] Integrate tools into ADK agent - [ ] Test tool calling flow - [ ] Verify REST calls to core-api - [ ] Create tests: `test_08_adk_agent.py`, `test_09_adk_tool_calling.py` ### Phase 4: API Routes - [ ] Add `/v1/chat/adk` endpoint - [ ] Rename existing to `/v1/chat/simple` (keep `/v1/chat/completions` as alias) - [ ] Test both endpoints - [ ] Create test: `tests/test_10_adk_api.py` ### Phase 5: Documentation & Cleanup - [ ] Update README.md - [ ] Update test documentation - [ ] Add architecture diagrams - [ ] Document migration from core-api --- ## Tool Design: REST-First All tools in core-ai make REST calls to core-api: ```python # Example: System Status Tool @log_tool_call async def get_system_status() -> str: """Get current system status from core-api.""" async with httpx.AsyncClient() as client: response = await client.get(f"{CORE_API_BASE_URL}/system/status") data = response.json() return json.dumps(data, indent=2) ``` **Benefits:** - Clean separation: core-ai = AI, core-api = tools - Tools can be used by both AI and direct API calls - Easy to test tools independently - No code duplication --- ## Testing Strategy ### Layer 1-5: Simple Mode (Existing) Already tested and passing ✓ ### Layer 6: ADK Setup ```python # Test ADK runtime initialization # Test ADK basic completion (no tools) # Test ADK message handling ``` ### Layer 7: ADK Tools ```python # Test tool registration # Test tool discovery # Test REST connectivity to core-api ``` ### Layer 8: ADK Agent ```python # Test agent with tools # Test prompt handling # Test error handling ``` ### Layer 9: ADK Tool Calling ```python # Test tool invocation # Test tool results # Test multi-tool workflows ``` ### Layer 10: ADK API ```python # Test /v1/chat/adk endpoint # Test streaming with tools # Test non-streaming with tools ``` --- ## Configuration ### Environment Variables ```bash # Existing OLLAMA_BASE_URL=http://ollama:11434 AGENT_MODEL=gemma2:9b-instruct-q5_K_M SYSTEM_PROMPT_VARIANT=minimal_agent HOST=0.0.0.0 PORT=8086 # New CORE_API_BASE_URL=http://core-api:8083/v1 # For tool REST calls ADK_ENABLED=true # Enable ADK endpoint SIMPLE_ENABLED=true # Enable simple endpoint ADK_SYSTEM_PROMPT_VARIANT=adk_agent # Different prompt for ADK ``` ### Prompts ```python PROMPTS = { "minimal_agent": "You are a helpful assistant.", # Simple mode "adk_agent": """You are a system management assistant with access to tools. Use tools when needed to answer questions about system status, services, and docker.""" } ``` --- ## Migration Path (Core-API) **Later cleanup - not in this phase:** 1. Remove AI agent code from core-api 2. Remove ADK dependencies from core-api 3. Keep only REST endpoints 4. Update core-api to be pure API 5. Redirect any AI requests to core-ai --- ## Backward Compatibility - `/v1/chat/completions` → alias for `/v1/chat/simple` - Existing clients keep working - New clients can choose mode --- ## Performance Considerations | Aspect | Simple Mode | ADK Mode | |--------|-------------|----------| | **Latency** | ~0.5-1s | ~1-3s (with tools) | | **Overhead** | Minimal | ADK runtime | | **Memory** | Low | Medium (tool registry) | | **Use Case** | Fast Q&A | Complex tasks | --- ## Next Steps 1. ✅ Design architecture (this document) 2. ⏳ Implement Phase 1: ADK Agent Setup 3. ⏳ Implement Phase 2: Tool Integration 4. ⏳ Implement Phase 3: ADK Agent with Tools 5. ⏳ Implement Phase 4: API Routes 6. ⏳ Implement Phase 5: Documentation **Let's start with Phase 1!**