feat: backup executors report their own outcome to check_history

The health record used to learn about backups by polling the mtime of the
newest archive, hourly, against a 48-hour threshold — for a job that runs once
a day. Forty-seven of every forty-eight runs could not produce a new answer.

Worse, file age cannot distinguish a failed backup from one that has not run
yet: when tonight's job dies, yesterday's archive is 24 hours old and still
reads healthy, and keeps reading healthy until hour 48. A failure stayed
invisible for two days to the check whose only job was noticing it. These
executors know at 03:05.

Reporting wraps the work rather than living inside it, so the failure path
cannot be forgotten — an exception is reported as critical and re-raised,
leaving the task's own status untouched. Reporting only success would reproduce
exactly the blind spot this replaces.

A reporting failure never fails the backup. Everything in health_report is
caught and logged, which means the absence of rows is the only symptom a broken
reporter produces — so monitor for rows, not for errors.

Not yet functional in production: scheduler_user holds DELETE and SELECT on
check_history (it prunes the table nightly) but not INSERT. Until that grant is
made the report logs a refusal and skips. The ticket predicted this as the step
that would fail silently, which is why it is named in the changelog and handled
as its own exception rather than folded into a generic catch.

Verified against baseline: tests/test_config_backup_executor.py and
tests/test_portainer_backup_executor.py report 7 failed / 17 passed both with
and without this change, so the pre-existing failures are untouched.

Workspace D-33, T-69.
This commit is contained in:
2026-08-11 00:11:52 +02:00
parent 0c2199667f
commit d9cbaee1fc
4 changed files with 209 additions and 2 deletions
+18
View File
@@ -6,6 +6,15 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
## [Unreleased]
### Added
- Backup executors report their own outcome to the homelab health record — one row in
`check_history` per run, success or failure. Replaces a monitor that inferred backup health
from file age and could not tell a failed backup from one that had not run yet.
### Notes
- Requires `GRANT INSERT ON check_history TO scheduler_user` in the `sysmon` database. Without
it the report is refused, logged, and skipped; the backup itself is unaffected.
## [1.4.0] - 2026-08-08
### Added
@@ -271,6 +280,15 @@ TOTAL 80% 🎯
## [Unreleased]
### Added
- Backup executors report their own outcome to the homelab health record — one row in
`check_history` per run, success or failure. Replaces a monitor that inferred backup health
from file age and could not tell a failed backup from one that had not run yet.
### Notes
- Requires `GRANT INSERT ON check_history TO scheduler_user` in the `sysmon` database. Without
it the report is refused, logged, and skipped; the backup itself is unaffected.
### Planned
- Redis integration for distributed locking
- Webhook notifications for task completion