Files
library-desk/docs/scheduler-tasks.md
T
jpmschweitzerandClaude Fable 5 0b346d3a57 feat: add job cleanup loop, Scheduler task definitions, and registrar
- job_cleanup_loop (src/jobs/job_manager.py): hourly in-process pass over
  JobManager.cleanup_expired_jobs, started at app startup and cancelled
  at shutdown; Redis job payloads auto-expire but set memberships do not.
- docs/scheduler-tasks.md: the four production Scheduler task payloads
  for the deploy checklist - nightly integrity check 04:30, weekly
  quality report Sunday 03:00 (day_of_week=6, 0=Monday), daily Paperless
  orphan-cleanup 05:00 hitting the existing
  /maintenance/cleanup/paperless?user=jpmschweitzer&dry_run=false
  endpoint, and disabling test_example_task - with exact HTTP bodies
  (explicit user=jpmschweitzer, Authorization: Bearer ${LIBRARY_API_KEY}
  placeholder).
- scripts/register_scheduler_tasks.py: reads SCHEDULER_URL from env,
  DRY-RUN BY DEFAULT (prints the exact payloads, provably contacts
  nothing), --execute gated and requiring LIBRARY_API_KEY to fill the
  placeholder. NOT executed - definitions delivered for the deploy
  checklist only.

9 new offline tests (loop passes/error-resilience/cancellation, payload
schedules, explicit production user, placeholder, dry-run default).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QbFZyDvYksazX6nYQYZ67L
2026-07-14 12:30:27 +02:00

160 lines
4.5 KiB
Markdown

# Scheduler Task Definitions (Phase C deploy checklist)
Production task payloads for the homelab's database-driven **Scheduler**
service. These are **definitions only** — nothing in this repo registers
them automatically. Register them as part of the deploy checklist, either
via the Scheduler UI/API or with the helper script:
```bash
# Preview exactly what would be sent (default):
SCHEDULER_URL=http://<scheduler-host>:8090 \
.venv/bin/python scripts/register_scheduler_tasks.py
# Actually register/update the tasks (deploy checklist step):
SCHEDULER_URL=http://<scheduler-host>:8090 \
LIBRARY_API_KEY=<the library-desk API key> \
.venv/bin/python scripts/register_scheduler_tasks.py --execute
```
Conventions:
- All tasks call the **production** library-desk container
(`http://library-desk:8089`) with the explicit production tenant
`user=jpmschweitzer` (there is no default tenant — Phase B).
- `${LIBRARY_API_KEY}` is a placeholder for the library-desk API key
(`LIBRARY_API_KEY` in the container env). Never commit the real value.
- Schedule fields use the Scheduler's convention: `-1` = every,
`day_of_week`: `0` = Monday … `6` = Sunday.
---
## 1. Nightly integrity check — 04:30 daily
Read-only report: pages without vectors, orphaned vectors, unexpected
Qdrant collections, Document nodes without wiki pages. Caches its result
in Redis for the weekly quality report.
```json
{
"task_name": "library_integrity_check",
"service": "library-desk",
"executor": "rest_api_executor",
"priority": 60,
"description": "Nightly read-only integrity check for the library (vectors/graph/wiki/collections)",
"enabled": true,
"max_retries": 2,
"timeout_seconds": 900,
"minute": 30,
"hour": 4,
"day_of_month": -1,
"month": -1,
"day_of_week": -1,
"config": {
"method": "POST",
"url": "http://library-desk:8089/maintenance/integrity-check",
"headers": {
"Content-Type": "application/json",
"Authorization": "Bearer ${LIBRARY_API_KEY}"
},
"body": {
"user": "jpmschweitzer"
}
}
}
```
## 2. Weekly quality report — Sunday 03:00
Runs the duplicate scan, flags stale/metadata-poor pages, folds in the
latest integrity results, and writes the dated report page to
`users/jpmschweitzer/system/quality-reports/YYYY-MM-DD`.
```json
{
"task_name": "library_quality_report",
"service": "library-desk",
"executor": "rest_api_executor",
"priority": 60,
"description": "Weekly library quality report (dedup, stale pages, missing metadata, integrity) written to the wiki",
"enabled": true,
"max_retries": 2,
"timeout_seconds": 1800,
"minute": 0,
"hour": 3,
"day_of_month": -1,
"month": -1,
"day_of_week": 6,
"config": {
"method": "POST",
"url": "http://library-desk:8089/maintenance/quality-report",
"headers": {
"Content-Type": "application/json",
"Authorization": "Bearer ${LIBRARY_API_KEY}"
},
"body": {
"user": "jpmschweitzer",
"stale_days": 30,
"dedup_threshold": 0.9,
"write_page": true
}
}
}
```
## 3. Daily Paperless orphan cleanup — 05:00
Hits the **existing** cleanup endpoint (query parameters, empty body).
`dry_run=false` deletes vectors/graph nodes for documents that were
removed from Paperless-ngx.
```json
{
"task_name": "library_paperless_orphan_cleanup",
"service": "library-desk",
"executor": "rest_api_executor",
"priority": 60,
"description": "Daily cleanup of vectors/graph nodes for documents deleted from Paperless-ngx",
"enabled": true,
"max_retries": 2,
"timeout_seconds": 900,
"minute": 0,
"hour": 5,
"day_of_month": -1,
"month": -1,
"day_of_week": -1,
"config": {
"method": "POST",
"url": "http://library-desk:8089/maintenance/cleanup/paperless?user=jpmschweitzer&dry_run=false",
"headers": {
"Authorization": "Bearer ${LIBRARY_API_KEY}"
},
"body": {}
}
}
```
## 4. Disable `test_example_task`
Not a new task: the leftover example task must be **disabled** (not
deleted, so its history is preserved).
```
PUT ${SCHEDULER_URL}/tasks/test_example_task
Content-Type: application/json
{"enabled": false}
```
---
## Related (already registered / in-process)
- `knowledge_consolidation` — every 30 minutes, POST
`/consolidate/knowledge` (already registered; after the Phase C
consolidation repair its runs log `searches_processed` and
`duration_ms`, and searches are no longer consumed while the LLM is
unavailable).
- Redis job-set cleanup — runs **in-process** inside library-desk
(hourly `job_cleanup_loop` started at app startup); no Scheduler task
needed.