# Core-AI Service

Simplified AI service for testing LiteLLM → Ollama → Model integration without ADK complexity.

## Purpose

This service strips away the ADK layer to isolate and debug the fundamental LiteLLM/Ollama integration. It provides:

- **Direct LiteLLM integration** - No ADK overhead
- **OpenAI-compatible API** - Drop-in replacement for testing
- **Comprehensive diagnostics** - Layered testing to identify issues
- **Minimal complexity** - Easy to understand and debug

## Architecture

```
HTTP Request → SimpleLiteLLMAgent → LiteLLM → Ollama → Model → Response
```

**Bypassed:** Google ADK, tool calling, complex orchestration

## Quick Start

### 1. Install Dependencies

```bash
pip install -r requirements.txt
```

### 2. Configure Environment

Create `.env` file or set environment variables:

```bash
OLLAMA_BASE_URL=http://ollama:11434
AGENT_MODEL=gemma2:9b-instruct-q5_K_M
SYSTEM_PROMPT_VARIANT=minimal_agent
HOST=0.0.0.0
PORT=8086
```

### 3. Run Diagnostics

```bash
# Check Ollama connectivity
python diagnostics/check_ollama.py

# Test direct LiteLLM
python diagnostics/test_litellm_direct.py

# Run full test suite
bash tests/run_all_tests.sh
```

### 4. Start Service

```bash
python main.py
```

Service will be available at `http://localhost:8086`

### 5. Test It

```bash
curl -X POST http://localhost:8086/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "messages": [
      {"role": "user", "content": "What is the capital of France?"}
    ]
  }'
```

## API Endpoints

### `GET /health`
Health check endpoint

**Response:**
```json
{
  "status": "ok",
  "service": "core-ai"
}
```

### `POST /v1/chat/completions`
OpenAI-compatible chat completions endpoint

**Request:**
```json
{
  "model": "test",
  "messages": [
    {"role": "user", "content": "Your question here"}
  ],
  "stream": false
}
```

**Response (non-streaming):**
```json
{
  "id": "chatcmpl-...",
  "object": "chat.completion",
  "created": 1234567890,
  "model": "test",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "Response here"
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "prompt_tokens": 0,
    "completion_tokens": 0,
    "total_tokens": 0
  }
}
```

**Streaming:** Set `"stream": true` for Server-Sent Events response

## Project Structure

```
services/core-ai/
├── main.py                 # HTTP server (aiohttp)
├── src/
│   ├── agent.py           # SimpleLiteLLMAgent
│   ├── config.py          # Configuration (Pydantic)
│   ├── prompts.py         # System prompts
│   └── tools.py           # (Unused in this version)
├── diagnostics/
│   ├── check_ollama.py    # Ollama connectivity check
│   └── test_litellm_direct.py  # Direct LiteLLM test
├── tests/
│   ├── test_01_environment.py   # Config tests
│   ├── test_02_litellm_raw.py   # Raw LiteLLM tests
│   ├── test_03_message_format.py # Message formatting
│   ├── test_04_agent.py         # Agent logic tests
│   ├── test_05_api.py           # API endpoint tests
│   ├── run_all_tests.sh         # Run all tests
│   └── README.md                # Test documentation
├── requirements.txt
├── Dockerfile
└── README.md (this file)
```

## Configuration

Configuration is managed via `src/config.py` using Pydantic Settings.

### Environment Variables

| Variable | Default | Description |
|----------|---------|-------------|
| `HOST` | `0.0.0.0` | Server host |
| `PORT` | `8086` | Server port |
| `OLLAMA_BASE_URL` | `http://ollama:11434` | Ollama API URL |
| `AGENT_MODEL` | `gemma2:9b-instruct-q5_K_M` | Model name |
| `SYSTEM_PROMPT_VARIANT` | `minimal_agent` | Prompt variant to use |
| `DEBUG` | `false` | Enable debug mode |
| `LOG_LEVEL` | `INFO` | Logging level |

## Testing

See [tests/README.md](tests/README.md) for comprehensive testing documentation.

**Quick test:**
```bash
bash tests/run_all_tests.sh
```

This runs 5 layers of tests to isolate issues:
1. Environment & Configuration
2. Raw LiteLLM Connection
3. Message Formatting
4. Agent Logic
5. API Integration

## Docker Deployment

### Build

```bash
docker build -t core-ai:latest .
```

### Run

```bash
docker run -d \
  --name core-ai \
  -p 8086:8086 \
  -e OLLAMA_BASE_URL=http://ollama:11434 \
  -e AGENT_MODEL=gemma2:9b-instruct-q5_K_M \
  --network docker-dataplane \
  core-ai:latest
```

### Using Docker Compose

```bash
docker-compose -f ../../stacks/core-ai.yml up
```

## Troubleshooting

### Service won't start

1. Check logs: `docker logs core-ai`
2. Verify Ollama is running: `docker ps | grep ollama`
3. Run diagnostics: `python diagnostics/check_ollama.py`

### No response or timeout

1. Check Ollama logs: `docker logs ollama`
2. Model may be loading (first run takes 30-60s)
3. Verify model exists: `docker exec ollama ollama list`
4. Test directly: `docker exec ollama ollama run gemma2:9b-instruct-q5_K_M "test"`

### Wrong or empty responses

1. Check system prompt is loaded (see agent logs)
2. Verify prompt variant exists in `src/prompts.py`
3. Run Layer 3 tests: `pytest tests/test_03_message_format.py -v`

### Connection refused

1. Check network: `docker network inspect docker-dataplane`
2. Verify both services are on the same network
3. Try using container IP instead of hostname

## Development

### Adding New Prompts

Edit `src/prompts.py`:

```python
PROMPTS = {
    "minimal_agent": "You are a helpful assistant.",
    "my_new_prompt": "Your custom system prompt here."
}
```

Update environment variable:
```bash
SYSTEM_PROMPT_VARIANT=my_new_prompt
```

### Modifying Agent Behavior

Edit `src/agent.py` - specifically the `SimpleLiteLLMAgent` class.

**Key methods:**
- `__init__()` - Initialization and configuration
- `chat()` - Streaming chat handler
- `chat_completion()` - Non-streaming completion handler

### Adding Tests

Add to appropriate test layer in `tests/`:
- Configuration changes → `test_01_environment.py`
- LiteLLM behavior → `test_02_litellm_raw.py`
- Message formatting → `test_03_message_format.py`
- Agent logic → `test_04_agent.py`
- API changes → `test_05_api.py`

## Comparison with Core-API

| Feature | Core-AI | Core-API |
|---------|---------|----------|
| **ADK Integration** | ❌ No | ✅ Yes |
| **Tool Calling** | ❌ No | ✅ Yes |
| **System Orchestration** | ❌ No | ✅ Yes |
| **Complexity** | Low | High |
| **Purpose** | Debugging | Production |
| **Direct LiteLLM** | ✅ Yes | ❌ No |
| **Diagnostics** | ✅ Comprehensive | Limited |

## Next Steps

### If Tests Pass

1. ✅ Foundation is solid
2. Consider migrating fixes to core-api
3. Add ADK layer back in phases
4. Test tool calling integration

### If Tests Fail

1. Run diagnostics to identify layer
2. Fix that specific layer
3. Re-run tests
4. Proceed once all pass

## Contributing

When making changes:
1. Run diagnostics first
2. Make changes
3. Run full test suite
4. Update relevant documentation
5. Test in Docker environment

## License

Part of the tower-of-joy project.