# Scheduler Task Registration Guide

## Overview

The Scheduler is **DB-driven** (not YAML-based). Tasks are registered via REST API and stored in PostgreSQL.

## Quick Reference

### Register a New Task

```bash
curl -X POST http://192.168.86.149:8090/tasks \
  -H "Authorization: Bearer $SCHEDULER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "task_name": "my_task",
    "service": "my-service",
    "executor": "rest_api",
    "priority": 25,
    "description": "Task description",
    "minute": 0,
    "hour": -1,
    "config": {
      "method": "POST",
      "url": "http://service:port/endpoint",
      "headers": {
        "Authorization": "Bearer ${ENV_VAR}"
      },
      "body": {}
    }
  }'
```

### List All Tasks

```bash
curl http://192.168.86.149:8090/tasks \
  -H "Authorization: Bearer $SCHEDULER_API_KEY"
```

### Trigger Task Manually

```bash
curl -X POST http://192.168.86.149:8090/tasks/{task_name}/trigger \
  -H "Authorization: Bearer $SCHEDULER_API_KEY"
```

### View Task Execution History

```bash
curl http://192.168.86.149:8090/executions?task_name={task_name} \
  -H "Authorization: Bearer $SCHEDULER_API_KEY"
```

## Schedule Patterns

Use cron-style fields where **`-1` means "every"**:

| Pattern | minute | hour | day_of_month | month | day_of_week | Description |
|---------|--------|------|--------------|-------|-------------|-------------|
| Every minute | -1 | -1 | -1 | -1 | -1 | Runs every minute |
| Hourly | 0 | -1 | -1 | -1 | -1 | Runs at minute 0 of every hour |
| Daily at 2 AM | 0 | 2 | -1 | -1 | -1 | Runs at 2:00 AM every day |
| Weekly (Monday 2 AM) | 0 | 2 | -1 | -1 | 0 | Runs at 2:00 AM every Monday |
| Monthly (1st at 2 AM) | 0 | 2 | 1 | -1 | -1 | Runs at 2:00 AM on 1st of month |

**Note:** `day_of_week` is 0-6 (Monday-Sunday)

## Priority Levels

| Priority Range | Level | Use Cases |
|----------------|-------|-----------|
| 1-5 | Emergency/System | Critical system tasks |
| 10-30 | User | User-initiated tasks |
| 40-70 | Maintenance | Background maintenance |
| 70+ | Low | Low priority cleanup |

## Executor Types

### `rest_api` Executor

Calls HTTP endpoints. Supports environment variable substitution in headers/body.

**Config Format:**
```json
{
  "method": "POST|GET|PUT|DELETE|PATCH",
  "url": "http://service:port/endpoint",
  "headers": {
    "Authorization": "Bearer ${ENV_VAR}",
    "Content-Type": "application/json"
  },
  "body": {
    "key": "value"
  },
  "timeout": 30,
  "auth": {
    "type": "bearer|basic|api_key",
    "value": "${TOKEN_ENV_VAR}"
  }
}
```

**Environment Variables:** Use `${VAR_NAME}` syntax to reference env vars from the scheduler container.

### Other Executors

- `shell`: Execute shell commands
- `python`: Execute Python scripts
- `docker`: Docker operations
- `backup`: Backup operations
- `doc_sync`: Documentation sync

## Complete Task Schema

### Required Fields

- `task_name` (string): Unique task identifier
- `service` (string): Service that owns this task (e.g., "library-desk", "core-api")
- `executor` (string): Executor type ("rest_api", "shell", etc.)
- `priority` (int): Priority level (1-100)

### Optional Fields

- `description` (string): Human-readable description
- `minute` (int): -1 to 59, default: -1 (every minute)
- `hour` (int): -1 to 23, default: -1 (every hour)
- `day_of_month` (int): -1 to 31, default: -1 (every day)
- `month` (int): -1 to 12, default: -1 (every month)
- `day_of_week` (int): -1 to 6, default: -1 (every day)
- `enabled` (bool): Whether task is enabled, default: true
- `max_retries` (int): Max retry attempts, default: 3
- `timeout_seconds` (int): Execution timeout, default: 3600
- `config` (object): Executor-specific configuration
- `created_by` (string): Creator identifier, default: "api"

## Example: Librarian Knowledge Consolidation

This task processes unprocessed search queries and consolidates knowledge into wiki pages.

**Registration:**
```bash
curl -X POST http://192.168.86.149:8090/tasks \
  -H "Authorization: Bearer $SCHEDULER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "task_name": "librarian_consolidation",
    "service": "library-desk",
    "executor": "rest_api",
    "priority": 25,
    "description": "Processes unprocessed search queries and consolidates knowledge",
    "minute": 0,
    "hour": -1,
    "config": {
      "method": "POST",
      "url": "http://library-desk:8089/consolidation/run",
      "headers": {
        "Authorization": "Bearer ${LIBRARY_API_KEY}",
        "Content-Type": "application/json"
      },
      "body": {
        "dry_run": false,
        "process_limit": 10,
        "lookback_days": 7,
        "min_web_results": 2
      }
    }
  }'
```

**Schedule:** Runs hourly (at minute 0 of every hour)

**What it does:**
1. Queries Neo4j for unprocessed `SearchQuery` nodes
2. Analyzes web results with Ollama
3. Creates/updates wiki pages via WikiPageWriter
4. Extracts and adds new entities to knowledge graph
5. Marks searches as processed

## API Endpoints

| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/tasks` | List all tasks |
| GET | `/tasks/{task_name}` | Get task details |
| POST | `/tasks` | Create new task |
| PUT | `/tasks/{task_name}` | Update task |
| DELETE | `/tasks/{task_name}` | Delete task |
| POST | `/tasks/{task_name}/trigger` | Manually trigger task |
| GET | `/executions` | View execution history |
| GET | `/stats` | Get scheduler statistics |
| GET | `/health` | Health check |

## Interactive Documentation

Visit **http://192.168.86.149:8090/docs** for interactive Swagger UI with:
- Full API documentation
- Request/response schemas
- Try-it-out functionality
- Example payloads

## Environment Variables

Get API keys from containers:
```bash
# Scheduler API key
docker exec scheduler printenv SCHEDULER_API_KEY

# Library Desk API key
docker exec library-desk printenv LIBRARY_API_KEY
```

## Task Management Tips

1. **Test with dry_run**: Set `enabled: false` initially or use dry_run in task config
2. **Monitor executions**: Check `/executions` endpoint regularly
3. **Start conservative**: Use longer intervals (hourly, daily) before going to frequent runs
4. **Use priorities wisely**: Reserve 1-10 for critical tasks, use 20-30 for user tasks
5. **Set timeouts appropriately**: Long-running tasks need higher timeout_seconds

## Troubleshooting

### Task not running

1. Check if task is enabled: `GET /tasks/{task_name}`
2. Check execution history: `GET /executions?task_name={task_name}`
3. Verify schedule matches current time
4. Check scheduler logs: `docker logs scheduler`

### Task failing

1. Check execution history for error messages
2. Verify endpoint is accessible from scheduler container
3. Test endpoint manually with curl
4. Check API keys and authentication
5. Verify executor configuration (especially for rest_api)

### Debug a task

```bash
# Get task details
curl http://192.168.86.149:8090/tasks/librarian_consolidation \
  -H "Authorization: Bearer $SCHEDULER_API_KEY"

# View recent executions
curl http://192.168.86.149:8090/executions?task_name=librarian_consolidation&limit=5 \
  -H "Authorization: Bearer $SCHEDULER_API_KEY"

# Trigger manually
curl -X POST http://192.168.86.149:8090/tasks/librarian_consolidation/trigger \
  -H "Authorization: Bearer $SCHEDULER_API_KEY"
```

## Architecture

```
┌─────────────────────────────────────────────────────────┐
│  The Scheduler (AsyncIOScheduler)                       │
│                                                          │
│  ┌────────────────────────────────────────────┐        │
│  │  Cron Job (every minute)                   │        │
│  │  ↓                                          │        │
│  │  Query PostgreSQL for tasks scheduled      │        │
│  │  for current minute/hour/day               │        │
│  │  ↓                                          │        │
│  │  Execute up to 5 tasks concurrently        │        │
│  │  (based on priority)                       │        │
│  │  ↓                                          │        │
│  │  Call appropriate executor:                │        │
│  │  - rest_api → HTTP request                │        │
│  │  - shell → subprocess                      │        │
│  │  - python → exec                           │        │
│  └────────────────────────────────────────────┘        │
│                                                          │
│  PostgreSQL:                                            │
│  - scheduled_tasks (task definitions)                  │
│  - task_executions (execution history)                 │
└─────────────────────────────────────────────────────────┘
```

## Current Registered Tasks

As of 2025-12-09:

1. **backup_docker_configs_daily** - Daily at 3:05 AM
2. **librarian_consolidation** - Hourly (minute 0)
3. **test_example_task** - Every minute (testing)
4. **sync_fastapi_docs_monthly** - 11th of month at 4:00 AM
5. **sync_ollama_docs_monthly** - 12th of month at 4:00 AM