Commit Graph
6 Commits
Author SHA1 Message Date
jpmschweitzer 8c0ced68eb feat(scheduler): add Pydantic models and improve task API
Scheduler API:
- Add Pydantic models for request/response validation
- Improve API documentation with examples
- Add detailed schedule pattern documentation
- Document priority levels and executor types

Models:
- TaskCreate, TaskUpdate, TaskResponse models
- Field validation and constraints
- Type safety for task operations

Documentation:
- Add TASK_REGISTRATION.md guide
- Document schedule patterns and executor configs
2025-12-10 01:26:21 +01:00
jpmschweitzer 65a91ab6f5 feat(scheduler): add generic REST API executor for universal HTTP task execution
Add rest_api_executor as a universal executor that can call any REST API
endpoint across the system. This provides a standard way to trigger HTTP
operations from scheduled tasks.

Features:
- All HTTP methods: GET, POST, PUT, DELETE, PATCH
- Authentication: Bearer token, Basic auth, API key
- Environment variable substitution: ${VAR_NAME}
- JSONPath response extraction
- Configurable timeouts and SSL verification
- Sensitive data redaction in logs
- Custom headers support

This executor enables scheduler to call any service endpoint (Library Desk,
Core API, external webhooks) without needing service-specific executors.

Example usage:
{
  "executor": "rest_api_executor",
  "config": {
    "url": "http://library-desk:8089/consolidate/knowledge",
    "method": "POST",
    "payload": {"process_limit": 10},
    "auth": {"type": "bearer", "token": "${API_KEY}"}
  }
}
2025-12-09 14:26:46 +01:00
jpmschweitzerandClaude Sonnet 4.5 be0ff6780c docs(scheduler): add comprehensive README and CHANGELOG
Add complete documentation for The Scheduler service.

README.md (500+ lines):
- Architecture overview with ASCII diagram
- Quick start guide
- Complete API reference with curl examples
- Task scheduling patterns and examples
- Priority system documentation
- Built-in executors documentation (example, doc_sync, config_backup)
- Custom executor development guide
- Current tasks table
- Database schema documentation
- Testing guide with coverage metrics
- Development and debugging information
- Monitoring and troubleshooting
- Security and performance notes
- API reference with response codes and filtering

CHANGELOG.md:
- Initial v1.0.0 release documentation
- Core features and architecture
- REST API endpoints
- Task executors and pre-configured tasks
- Testing infrastructure and metrics
- Technical details and dependencies
- Coverage metrics breakdown
- Planned features for future releases

Documentation covers:
- All API endpoints and authentication
- Scheduling examples (every minute, daily, monthly, etc.)
- Priority ranges and usage
- Executor configuration
- Test database setup
- Docker stack configuration
- Common issues and solutions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-07 23:19:38 +01:00
jpmschweitzerandClaude Sonnet 4.5 4e4ce38db5 test(scheduler): add comprehensive test suite with 80% coverage
Add complete testing infrastructure with unit, integration, and API tests.

Test coverage: 80% overall
- config.py: 100%
- example_executor.py: 100%
- main.py (API endpoints): 95%
- doc_sync_executor.py: 78%
- executor.py (core logic): 71%
- config_backup_executor.py: 50%

Test categories:
- Unit tests: Fast tests with mocked dependencies
- API tests: Comprehensive endpoint testing (24 tests)
- Executor tests: Task executor validation
- Integration tests: Real database operations

Test infrastructure:
- pytest configuration with markers (unit, integration, api, executor)
- Coverage reporting with pytest-cov
- Dedicated test database (test_scheduler on postgres-shared)
- Database fixtures for clean test state
- Mock fixtures for unit testing

Test database:
- Database: test_scheduler
- User: test_scheduler_user
- Automatic schema creation and cleanup
- Integration tests use real PostgreSQL

Files:
- pytest.ini - pytest configuration
- tests/conftest.py - shared fixtures
- tests/test_api.py - API endpoint tests
- tests/test_api_comprehensive.py - comprehensive API tests
- tests/test_config.py - configuration tests
- tests/test_database_integration.py - database integration tests
- tests/test_integration.py - general integration tests
- tests/test_*_executor.py - executor-specific tests
- tests/test_database_setup.sql - test database schema

85 total tests with 54 passing core tests

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-07 23:15:31 +01:00
jpmschweitzerandClaude Sonnet 4.5 ff8a3a1009 feat(scheduler): add task executors for common operations
Add three built-in task executors for various automation tasks.

Executors:
1. example_executor - Simple test implementation with configurable message and delay
2. doc_sync_executor - Mirror documentation from upstream Git repos to Gitea
3. config_backup_executor - Backup Docker configs and data directories

doc_sync_executor features:
- Clones upstream repository (GitHub, GitLab, etc.)
- Supports full repository mirroring or selective path syncing
- Pushes to Gitea with authentication
- Creates date-tagged snapshots (YYYY-MM-DD)
- Generates .SYNC_INFO.md with sync metadata

config_backup_executor features:
- Backs up multiple source paths with exclusion patterns
- Optional compression (tar.gz)
- Retention policy (days-based cleanup)
- Timestamped backups

Pre-configured tasks:
- backup_docker_configs_daily (priority 20, daily 03:05)
- sync_fastapi_docs_monthly (priority 60, 11th @ 04:00)
- sync_ollama_docs_monthly (priority 60, 12th @ 04:00)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-07 23:13:07 +01:00
jpmschweitzerandClaude Sonnet 4.5 455d16ce8f feat(scheduler): implement core scheduler service
Add hybrid APScheduler + PostgreSQL-based task scheduling system with minute-based execution and priority queue.

Core features:
- Minute-based scheduling with cron-like patterns (-1 = wildcard)
- Priority queue system (1-100, lower = higher priority)
- Concurrent execution (max 5 tasks simultaneously)
- Full REST API for task management (CRUD operations)
- Task execution tracking with audit trail
- API key authentication (Bearer token)
- Health checks and system statistics

Architecture:
- APScheduler runs every minute
- Queries PostgreSQL for tasks scheduled for current minute
- Executes tasks concurrently by priority
- Records execution history in database

Database schema:
- scheduled_tasks: Task definitions/templates
- task_executions: Individual execution records

Technical stack:
- FastAPI for REST API
- APScheduler for scheduling
- PostgreSQL for persistence
- Pydantic for configuration

Endpoints:
- POST/GET/PUT/DELETE /tasks - Task management
- POST /tasks/{name}/trigger - Manual execution
- GET /executions - Execution history
- GET /health - Health check
- GET /stats - System statistics

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2025-12-07 23:12:28 +01:00