161 lines
7.4 KiB
Markdown
161 lines
7.4 KiB
Markdown
# CLAUDE.md — scheduler
|
|
|
|
"The Scheduler" — system-wide maintenance orchestration for tower-of-joy: config backups, doc
|
|
mirroring to Gitea, cleanup and retention, and arbitrary REST calls on a cron. Python 3.12 /
|
|
FastAPI, APScheduler, PostgreSQL and Redis. Container `scheduler` on `docker-dataplane`,
|
|
port **8090**, Redis DB **3**, Postgres DB **scheduler**.
|
|
|
|
It is the homelab's cron. Recurring work belongs here rather than in a systemd timer (workspace D-5).
|
|
|
|
## Live contract
|
|
|
|
`http://localhost:8090/openapi.json` — 10 paths, `version: 1.9.0` (verified 2026-08-11). Human
|
|
docs at `/docs`. Generated from running code, so read it instead of inferring routes.
|
|
|
|
**Every route except `/health` requires `Authorization: Bearer $SCHEDULER_API_KEY`.** An
|
|
unauthenticated call returns `{"detail": "Missing API key"}` at 200-shape JSON, not a 401 body
|
|
you might pattern-match on.
|
|
|
|
Note the old AGENTS.md told you to verify deploys against `192.168.86.149:8000/health`. That is
|
|
**tatlock's** port, not this service's. It is 8090.
|
|
|
|
## The thing that will mislead you: executors are chosen by data, not code
|
|
|
|
`src/executors/*.py` are **never statically imported**. `src/tasks/executor.py:202` does:
|
|
|
|
```python
|
|
module_path = f"src.executors.{executor_name}"
|
|
module = __import__(module_path, fromlist=['execute'])
|
|
```
|
|
|
|
where `executor_name` comes from a **row in the `scheduled_tasks` table**. Consequences, and
|
|
they defeat both of the usual checks:
|
|
|
|
- **grep finds nothing.** No file imports `config_backup_executor`; the name only ever exists as
|
|
a database string.
|
|
- **`sys.modules` finds nothing either.** A cold `import src.main` loads only `src`, `src.config`,
|
|
`src.main`, `src.models`, `src.tasks`, `src.tasks.executor`. Every executor is absent until a
|
|
task actually fires. Absence there is a timing artifact, not evidence of death.
|
|
|
|
**The authoritative source is the database.** As of 2026-08-09:
|
|
|
|
| Executor | Rows | Enabled |
|
|
|---|---|---|
|
|
| `rest_api_executor` | 16 | yes |
|
|
| `doc_sync_executor` | 2 | yes |
|
|
| `config_backup_executor`, `docker_prune_executor`, `gitea_release_cleanup_executor`, `portainer_backup_executor`, `postgres_retention_executor` | 1 each | yes |
|
|
| `example_executor` | 1 | **no** |
|
|
| `gcs_backup_executor` | **0** | — |
|
|
|
|
`gcs_backup_executor` has no rows at all. That does **not** make it dead code: it becomes live
|
|
the instant someone inserts a row naming it, with no code change and no deploy. Treat unreferenced
|
|
executors as *dormant*, not removable. An executor's contract is a module-level
|
|
`execute(config, settings)` — a missing one is caught at run time and reported as
|
|
`Executor <name> missing execute() function`, not at import or startup.
|
|
|
|
Re-check with the query rather than trusting the table above:
|
|
|
|
```bash
|
|
docker exec scheduler python3 -c "
|
|
import psycopg2
|
|
from src.config import get_settings
|
|
s = get_settings()
|
|
c = psycopg2.connect(host=s.postgres_host, port=s.postgres_port, dbname=s.postgres_db,
|
|
user=s.postgres_user, password=s.postgres_password)
|
|
cur = c.cursor()
|
|
cur.execute('SELECT executor, count(*), bool_or(enabled) FROM scheduled_tasks GROUP BY executor ORDER BY 1')
|
|
[print(r) for r in cur.fetchall()]"
|
|
```
|
|
|
|
Build the connection from `get_settings()` fields as above. Do not print the assembled URL — it
|
|
carries the Postgres password.
|
|
|
|
## Database
|
|
|
|
Three tables, and the names do not match the API paths: **`scheduled_tasks`** (not `tasks` —
|
|
`SELECT … FROM tasks` fails with `UndefinedTable`), `task_executions`, `doc_sources`. Schema is
|
|
SQLAlchemy (`src/models.py`); there is no Alembic here, unlike core-api.
|
|
|
|
## Layout, and one trap in it
|
|
|
|
`src/main.py` (app + routes), `src/config.py` (pydantic-settings), `src/models.py`,
|
|
`src/tasks/executor.py` (the scheduling engine), `src/executors/` (the dynamically-loaded units).
|
|
|
|
**`src/config/` also exists and is an empty directory.** `import src.config` resolves to
|
|
`src/config.py` — verified in the container, `__file__` is `/app/src/config.py`, because a
|
|
regular module wins over a namespace package. Do not "fix" this by moving config into the
|
|
directory, and do not assume the directory is a package with contents.
|
|
|
|
## Registering tasks
|
|
|
|
Tasks are DB-driven, registered over the API — not YAML, not a file in this repo. See
|
|
`TASK_REGISTRATION.md` for the payload shape and the cron-field conventions (`hour: -1` means
|
|
every hour). There is also a workspace-level `scheduler` skill for driving it conversationally.
|
|
|
|
## Working here
|
|
|
|
Group new work by domain rather than by file type; a single large `routers/` folder is the thing
|
|
to avoid. Reference: [FastAPI best practices](https://github.com/zhanymkanov/fastapi-best-practices).
|
|
|
|
```bash
|
|
.venv/bin/python -m pytest tests/ # or: pytest tests/
|
|
```
|
|
|
|
Test dependencies are the `test` extra in `pyproject.toml` (pytest, pytest-asyncio, pytest-cov,
|
|
freezegun). `pytest.ini` is at the repo root. No linter is configured — no ruff/flake8 config and
|
|
neither in the dependencies — so do not assume `ruff check` exists here.
|
|
|
|
## CI
|
|
|
|
`.gitea/workflows/build.yml` is the only workflow and triggers **only on `v*` tag push**: build,
|
|
push image, ping Watchtower. There is **no CI test or lint gate**. Run the tests yourself before
|
|
tagging.
|
|
|
|
## Work tracking
|
|
|
|
Work lives in **pql**, not a markdown TODO. **This repo's vault is standalone** — its tickets
|
|
and its internal decisions live here in `.pql/` and `governance/`, and travel with a clone,
|
|
because `.pql/changelog/` is committed and replayed by the git hooks (workspace D-15). The databases are
|
|
gitignored and rebuildable with `pql plan rebuild`.
|
|
|
|
`pql` is **not** on the non-interactive `PATH` — invoke it as `/home/jpmschweitzer/.local/bin/pql`.
|
|
From inside this repo no `--vault` is needed: pql anchors at the nearest `.git/` ancestor, which
|
|
is this repo.
|
|
|
|
```bash
|
|
/home/jpmschweitzer/.local/bin/pql ticket list # this repo's open work
|
|
/home/jpmschweitzer/.local/bin/pql plan whatsnext # next unblocked item, with context
|
|
/home/jpmschweitzer/.local/bin/pql decisions list # this repo's own decisions
|
|
```
|
|
|
|
Stack-level decisions that constrain this service — the host, the network, deploy mechanics,
|
|
and the fact that recurring work belongs here at all (workspace D-5) — live in the **workspace** vault
|
|
and need the flag:
|
|
|
|
```bash
|
|
/home/jpmschweitzer/.local/bin/pql --vault /mnt/media/Projects decisions list --domain scheduler
|
|
```
|
|
|
|
Note `ticket new --decision D-N` resolves ids within **one** vault, so a ticket here cannot link
|
|
to a workspace decision. Cite the id in the ticket body instead.
|
|
|
|
## Git
|
|
|
|
- **History is linear — no merge commits.** Work on `main`, or a short-lived branch that is
|
|
fast-forwarded and deleted. The previous AGENTS.md mandated a feature branch per change; that
|
|
rule was retired workspace-wide on 2026-08-08.
|
|
- **Conventional Commits**: `feat:`, `fix:`, `refactor:`, `docs:`, `chore:`.
|
|
- **Stage explicitly. Never `git add -A`** — it is denied by policy, and it sweeps in whatever
|
|
else is dirty, including secrets.
|
|
- Update `CHANGELOG.md` for every user-facing change, under `[Unreleased]`.
|
|
|
|
## Releasing
|
|
|
|
Ask whether a deploy is wanted first — it is not automatic.
|
|
|
|
1. Bump `version` in `pyproject.toml` (patch for fixes, minor for features).
|
|
2. Move `[Unreleased]` entries into a dated section in `CHANGELOG.md`.
|
|
3. Stage the changed files by name, commit, tag `vX.Y.Z`, `git push origin main --tags`.
|
|
4. Gitea CI builds and pushes on the tag; Watchtower deploys it.
|
|
5. Verify: `curl http://192.168.86.149:8090/health`.
|