- job_cleanup_loop (src/jobs/job_manager.py): hourly in-process pass over
JobManager.cleanup_expired_jobs, started at app startup and cancelled
at shutdown; Redis job payloads auto-expire but set memberships do not.
- docs/scheduler-tasks.md: the four production Scheduler task payloads
for the deploy checklist - nightly integrity check 04:30, weekly
quality report Sunday 03:00 (day_of_week=6, 0=Monday), daily Paperless
orphan-cleanup 05:00 hitting the existing
/maintenance/cleanup/paperless?user=jpmschweitzer&dry_run=false
endpoint, and disabling test_example_task - with exact HTTP bodies
(explicit user=jpmschweitzer, Authorization: Bearer ${LIBRARY_API_KEY}
placeholder).
- scripts/register_scheduler_tasks.py: reads SCHEDULER_URL from env,
DRY-RUN BY DEFAULT (prints the exact payloads, provably contacts
nothing), --execute gated and requiring LIBRARY_API_KEY to fill the
placeholder. NOT executed - definitions delivered for the deploy
checklist only.
9 new offline tests (loop passes/error-resilience/cancellation, payload
schedules, explicit production user, placeholder, dry-run default).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QbFZyDvYksazX6nYQYZ67L
160 lines
4.5 KiB
Markdown
160 lines
4.5 KiB
Markdown
# Scheduler Task Definitions (Phase C deploy checklist)
|
|
|
|
Production task payloads for the homelab's database-driven **Scheduler**
|
|
service. These are **definitions only** — nothing in this repo registers
|
|
them automatically. Register them as part of the deploy checklist, either
|
|
via the Scheduler UI/API or with the helper script:
|
|
|
|
```bash
|
|
# Preview exactly what would be sent (default):
|
|
SCHEDULER_URL=http://<scheduler-host>:8090 \
|
|
.venv/bin/python scripts/register_scheduler_tasks.py
|
|
|
|
# Actually register/update the tasks (deploy checklist step):
|
|
SCHEDULER_URL=http://<scheduler-host>:8090 \
|
|
LIBRARY_API_KEY=<the library-desk API key> \
|
|
.venv/bin/python scripts/register_scheduler_tasks.py --execute
|
|
```
|
|
|
|
Conventions:
|
|
|
|
- All tasks call the **production** library-desk container
|
|
(`http://library-desk:8089`) with the explicit production tenant
|
|
`user=jpmschweitzer` (there is no default tenant — Phase B).
|
|
- `${LIBRARY_API_KEY}` is a placeholder for the library-desk API key
|
|
(`LIBRARY_API_KEY` in the container env). Never commit the real value.
|
|
- Schedule fields use the Scheduler's convention: `-1` = every,
|
|
`day_of_week`: `0` = Monday … `6` = Sunday.
|
|
|
|
---
|
|
|
|
## 1. Nightly integrity check — 04:30 daily
|
|
|
|
Read-only report: pages without vectors, orphaned vectors, unexpected
|
|
Qdrant collections, Document nodes without wiki pages. Caches its result
|
|
in Redis for the weekly quality report.
|
|
|
|
```json
|
|
{
|
|
"task_name": "library_integrity_check",
|
|
"service": "library-desk",
|
|
"executor": "rest_api_executor",
|
|
"priority": 60,
|
|
"description": "Nightly read-only integrity check for the library (vectors/graph/wiki/collections)",
|
|
"enabled": true,
|
|
"max_retries": 2,
|
|
"timeout_seconds": 900,
|
|
"minute": 30,
|
|
"hour": 4,
|
|
"day_of_month": -1,
|
|
"month": -1,
|
|
"day_of_week": -1,
|
|
"config": {
|
|
"method": "POST",
|
|
"url": "http://library-desk:8089/maintenance/integrity-check",
|
|
"headers": {
|
|
"Content-Type": "application/json",
|
|
"Authorization": "Bearer ${LIBRARY_API_KEY}"
|
|
},
|
|
"body": {
|
|
"user": "jpmschweitzer"
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
## 2. Weekly quality report — Sunday 03:00
|
|
|
|
Runs the duplicate scan, flags stale/metadata-poor pages, folds in the
|
|
latest integrity results, and writes the dated report page to
|
|
`users/jpmschweitzer/system/quality-reports/YYYY-MM-DD`.
|
|
|
|
```json
|
|
{
|
|
"task_name": "library_quality_report",
|
|
"service": "library-desk",
|
|
"executor": "rest_api_executor",
|
|
"priority": 60,
|
|
"description": "Weekly library quality report (dedup, stale pages, missing metadata, integrity) written to the wiki",
|
|
"enabled": true,
|
|
"max_retries": 2,
|
|
"timeout_seconds": 1800,
|
|
"minute": 0,
|
|
"hour": 3,
|
|
"day_of_month": -1,
|
|
"month": -1,
|
|
"day_of_week": 6,
|
|
"config": {
|
|
"method": "POST",
|
|
"url": "http://library-desk:8089/maintenance/quality-report",
|
|
"headers": {
|
|
"Content-Type": "application/json",
|
|
"Authorization": "Bearer ${LIBRARY_API_KEY}"
|
|
},
|
|
"body": {
|
|
"user": "jpmschweitzer",
|
|
"stale_days": 30,
|
|
"dedup_threshold": 0.9,
|
|
"write_page": true
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
## 3. Daily Paperless orphan cleanup — 05:00
|
|
|
|
Hits the **existing** cleanup endpoint (query parameters, empty body).
|
|
`dry_run=false` deletes vectors/graph nodes for documents that were
|
|
removed from Paperless-ngx.
|
|
|
|
```json
|
|
{
|
|
"task_name": "library_paperless_orphan_cleanup",
|
|
"service": "library-desk",
|
|
"executor": "rest_api_executor",
|
|
"priority": 60,
|
|
"description": "Daily cleanup of vectors/graph nodes for documents deleted from Paperless-ngx",
|
|
"enabled": true,
|
|
"max_retries": 2,
|
|
"timeout_seconds": 900,
|
|
"minute": 0,
|
|
"hour": 5,
|
|
"day_of_month": -1,
|
|
"month": -1,
|
|
"day_of_week": -1,
|
|
"config": {
|
|
"method": "POST",
|
|
"url": "http://library-desk:8089/maintenance/cleanup/paperless?user=jpmschweitzer&dry_run=false",
|
|
"headers": {
|
|
"Authorization": "Bearer ${LIBRARY_API_KEY}"
|
|
},
|
|
"body": {}
|
|
}
|
|
}
|
|
```
|
|
|
|
## 4. Disable `test_example_task`
|
|
|
|
Not a new task: the leftover example task must be **disabled** (not
|
|
deleted, so its history is preserved).
|
|
|
|
```
|
|
PUT ${SCHEDULER_URL}/tasks/test_example_task
|
|
Content-Type: application/json
|
|
|
|
{"enabled": false}
|
|
```
|
|
|
|
---
|
|
|
|
## Related (already registered / in-process)
|
|
|
|
- `knowledge_consolidation` — every 30 minutes, POST
|
|
`/consolidate/knowledge` (already registered; after the Phase C
|
|
consolidation repair its runs log `searches_processed` and
|
|
`duration_ms`, and searches are no longer consumed while the LLM is
|
|
unavailable).
|
|
- Redis job-set cleanup — runs **in-process** inside library-desk
|
|
(hourly `job_cleanup_loop` started at app startup); no Scheduler task
|
|
needed.
|