# Changelog

All notable changes to The Scheduler will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

## [Unreleased]

## [1.4.0] - 2026-08-08

### Added
- **Portainer Backup Executor** (`portainer_backup_executor.py`) — archives Portainer's
  own state through its `/api/backup` endpoint. Portainer's BoltDB lives in a Docker
  volume that the daily config backup does not cover, so losing that volume would take
  every stack definition with it. Uses the API rather than tarring the live volume, and
  rejects a 200 whose body is not a readable archive.


## [1.3.0] - 2026-08-08

### Added
- **Postgres Retention Executor** (`postgres_retention_executor.py`) — deletes rows
  past a retention window from a table on the shared Postgres server. Uses the
  Scheduler's own credentials with only the database name overridden, so the target
  database grants `scheduler_user` SELECT and DELETE on the table.
- **Docker Prune Executor** (`docker_prune_executor.py`) — scheduled reclaim of Docker
  disk usage. Build cache and dangling images are pruned by default; unused images and
  volumes are opt-in, since volume pruning also removes volumes belonging to stopped
  containers.

### Fixed
- `POST /tasks` returned HTTP 500 after successfully creating the task. The response
  model declared `created_at`/`updated_at` as strings while the database returns
  timestamps, so every create looked like a failure and retrying hit a duplicate-key
  error.

### Changed
- `TASK_REGISTRATION.md` now lists the executors that exist. It previously advertised
  `shell`, `python` and `docker` executors that were never implemented.

## [1.2.0] - 2026-03-30

### Added
- **GCS Backup Executor** (`gcs_backup_executor.py`) — offsite backup to Google Cloud
  Storage.

## [1.1.3] - 2026-01-08

### Changed
- Test release to validate CI/CD auto-deploy workflow

## [1.1.2] - 2026-01-03

### Fixed
- CI: Use curl for release creation (release-action requires Go)

## [1.1.1] - 2026-01-03

### Changed
- CI: Auto-create Gitea release on version tag push (v*) instead of manual release trigger

## [1.1.0] - 2025-12-14

### Added
- **Gitea Release Cleanup Executor** (`gitea_release_cleanup_executor.py`)
  - Automatically cleans up old releases across all Gitea repositories
  - Configurable retention count (default: 5 releases per repo)
  - Repository exclusion list support
  - Dry-run mode for safe testing
  - Designed to run before Watchtower to prevent image tag accumulation
- **GITEA_TOKEN setting** in config for API token authentication (separate from password)

## [1.0.4] - 2025-12-14

### Fixed
- CI/CD: Correct Watchtower port (8080)

## [1.0.3] - 2025-12-14

### Added
- CI/CD: Trigger Watchtower update after successful Docker build

## [1.0.2] - 2025-12-14

### Fixed
- Removed unused `setup_database.sql` from Dockerfile (database schema managed externally)

## [1.0.1] - 2025-12-14

### Changed
- **Version tracking now uses pyproject.toml** as single source of truth
  - Added `pyproject.toml` with project metadata and dependencies
  - `config.py` reads version from pyproject.toml using `tomllib`
  - FastAPI app title and version dynamically loaded from config
  - Health endpoint now includes version in response
  - Dockerfile updated to include pyproject.toml

## [1.0.0] - 2025-12-07

### Added

#### Core Scheduling System
- **Hybrid APScheduler + PostgreSQL architecture** for minute-based task scheduling
  - Single scheduler job runs every minute
  - Queries database for tasks scheduled for current minute
  - Executes up to 5 tasks concurrently by priority
  - Priority queue system (1-100, lower = higher priority)
- **Cron-like scheduling** with wildcard support (`-1` = any)
  - Supports minute, hour, day_of_month, month, day_of_week patterns
  - Flexible scheduling from every-minute to specific dates
- **Task execution tracking** with full audit trail
  - Database tables: `scheduled_tasks` (definitions) and `task_executions` (history)
  - Tracks status, duration, output, errors, and retry attempts
  - Execution metadata stored as JSONB

#### REST API
- **Full CRUD API** for task management with FastAPI
  - `POST /tasks` - Create new scheduled task
  - `GET /tasks` - List all tasks with filtering (service, enabled)
  - `GET /tasks/{name}` - Get task details
  - `PUT /tasks/{name}` - Update task configuration
  - `DELETE /tasks/{name}` - Remove task
  - `POST /tasks/{name}/trigger` - Manually trigger task execution
- **Execution history endpoints**
  - `GET /executions` - Query execution history
  - Filter by task_name, status, service
  - Pagination support (limit parameter)
- **System monitoring endpoints**
  - `GET /health` - Health check
  - `GET /stats` - System statistics (enabled tasks, running tasks, 24h execution counts)
- **API Key authentication** (Bearer token) for all protected endpoints
- **OpenAPI documentation** at `/docs`

#### Task Executors
- **Example Executor** (`example_executor.py`)
  - Simple test implementation with configurable message and delay
  - Demonstrates executor pattern
- **Documentation Sync Executor** (`doc_sync_executor.py`)
  - Mirrors documentation from upstream Git repositories to Gitea
  - Supports full repository mirroring or selective path syncing
  - Creates date-tagged snapshots (YYYY-MM-DD format)
  - Generates `.SYNC_INFO.md` with sync metadata
  - Configurable upstream repo, paths, branch, and Gitea destination
- **Config Backup Executor** (`config_backup_executor.py`)
  - Backs up Docker configurations and data directories
  - Supports multiple source paths with exclusion patterns
  - Optional compression (tar.gz)
  - Retention policy (days-based cleanup)
  - Creates timestamped backups

#### Pre-configured Tasks
- **FastAPI Documentation Sync** (monthly on 11th at 04:00)
  - Syncs entire FastAPI repository to `library/docs-fastapi`
  - Priority: 60 (maintenance)
- **Ollama Documentation Sync** (monthly on 12th at 04:00)
  - Syncs only `/docs` folder from Ollama repository to `library/docs-ollama`
  - Priority: 60 (maintenance)
- **Docker Config Backup** (daily at 03:05)
  - Backs up Docker data and configurations
  - Priority: 20 (user task)
  - 30-day retention
- **Example Test Task** (every minute, can be disabled)
  - Test task for validation
  - Priority: 50 (maintenance)

#### Testing Infrastructure
- **Comprehensive test suite** with 80% code coverage
  - 85 total tests across multiple test files
  - Pytest configuration with markers (unit, integration, api, executor)
  - Coverage reporting with pytest-cov
- **Test categories**:
  - Unit tests: Fast tests with mocked dependencies
  - API tests: Comprehensive endpoint testing (24 tests)
  - Executor tests: Task executor validation
  - Integration tests: Real database operations
- **Test database setup**
  - Dedicated `test_scheduler` database on postgres-shared
  - Automatic schema creation and cleanup
  - Database fixtures for clean test state
  - Test user: `test_scheduler_user`
- **Test fixtures** (conftest.py)
  - Mock database connections
  - Mock scheduler and executor
  - Sample task data
  - Authentication headers
  - Clean database state management

#### Documentation
- **Comprehensive README.md** (500+ lines)
  - Architecture overview with ASCII diagram
  - Quick start guide
  - Complete API reference with curl examples
  - Task scheduling patterns and examples
  - Executor development guide
  - Testing guide with coverage metrics
  - Development and debugging information
  - Security and performance notes
- **Test database setup guide** (test_database_setup.sql)
  - SQL script for creating test environment
  - Schema matching production
  - Test data fixtures

#### Configuration
- **Pydantic Settings** for environment-based configuration
  - PostgreSQL connection settings
  - Redis connection (for future use)
  - Gitea authentication
  - API key configuration
  - Computed properties (database_url, redis_url)
- **Docker stack configuration** (stacks/scheduler.yml)
  - Virtual environment setup on startup
  - Git configuration for librarian user
  - Health checks
  - Network isolation (docker-dataplane)
  - Resource limits

### Technical Details

#### Database Schema
```sql
scheduled_tasks:
  - Task definitions/templates
  - Scheduling configuration (minute/hour/day patterns)
  - Priority, enabled status, retry settings
  - Task configuration as JSONB
  - Execution tracking fields

task_executions:
  - Individual execution records
  - Status tracking (pending, running, success, failed, timeout)
  - Duration and timestamp tracking
  - Output and error details
  - Metadata as JSONB
```

#### Performance
- Minute-based processing with lightweight scheduler ticks
- Connection pooling for database efficiency
- Database indexes for optimized task queries
- Concurrent execution with configurable limit (default: 5)
- Priority-based execution order

#### Security
- API key authentication required for protected endpoints
- Network isolation on docker-dataplane
- Environment variable-based secrets
- Gitea token authentication for git operations
- Database credentials in environment

### Dependencies
- FastAPI (web framework)
- APScheduler (task scheduling)
- psycopg2-binary (PostgreSQL driver)
- Pydantic (configuration management)
- pytest + pytest-asyncio + pytest-cov (testing)
- GitPython (git operations)

### Coverage Metrics
```
Module                          Coverage
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
config.py                       100% ✅
example_executor.py             100% ✅
main.py (API endpoints)          95% ✅
doc_sync_executor.py             78% ✅
executor.py (core logic)         71% ✅
config_backup_executor.py        50% 📈
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
TOTAL                            80% 🎯
```

## [Unreleased]

### Planned
- Redis integration for distributed locking
- Webhook notifications for task completion
- Task dependencies (run task B after task A succeeds)
- Task groups and tags
- More executors (database backup, log rotation, etc.)
- Web UI for task management
- Metrics export (Prometheus)
- Advanced scheduling (last business day of month, etc.)