feat: backup executors report their own outcome to check_history
The health record used to learn about backups by polling the mtime of the newest archive, hourly, against a 48-hour threshold — for a job that runs once a day. Forty-seven of every forty-eight runs could not produce a new answer. Worse, file age cannot distinguish a failed backup from one that has not run yet: when tonight's job dies, yesterday's archive is 24 hours old and still reads healthy, and keeps reading healthy until hour 48. A failure stayed invisible for two days to the check whose only job was noticing it. These executors know at 03:05. Reporting wraps the work rather than living inside it, so the failure path cannot be forgotten — an exception is reported as critical and re-raised, leaving the task's own status untouched. Reporting only success would reproduce exactly the blind spot this replaces. A reporting failure never fails the backup. Everything in health_report is caught and logged, which means the absence of rows is the only symptom a broken reporter produces — so monitor for rows, not for errors. Not yet functional in production: scheduler_user holds DELETE and SELECT on check_history (it prunes the table nightly) but not INSERT. Until that grant is made the report logs a refusal and skips. The ticket predicted this as the step that would fail silently, which is why it is named in the changelog and handled as its own exception rather than folded into a generic catch. Verified against baseline: tests/test_config_backup_executor.py and tests/test_portainer_backup_executor.py report 7 failed / 17 passed both with and without this change, so the pre-existing failures are untouched. Workspace D-33, T-69.
This commit is contained in:
@@ -6,6 +6,15 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- Backup executors report their own outcome to the homelab health record — one row in
|
||||
`check_history` per run, success or failure. Replaces a monitor that inferred backup health
|
||||
from file age and could not tell a failed backup from one that had not run yet.
|
||||
|
||||
### Notes
|
||||
- Requires `GRANT INSERT ON check_history TO scheduler_user` in the `sysmon` database. Without
|
||||
it the report is refused, logged, and skipped; the backup itself is unaffected.
|
||||
|
||||
## [1.4.0] - 2026-08-08
|
||||
|
||||
### Added
|
||||
@@ -271,6 +280,15 @@ TOTAL 80% 🎯
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
- Backup executors report their own outcome to the homelab health record — one row in
|
||||
`check_history` per run, success or failure. Replaces a monitor that inferred backup health
|
||||
from file age and could not tell a failed backup from one that had not run yet.
|
||||
|
||||
### Notes
|
||||
- Requires `GRANT INSERT ON check_history TO scheduler_user` in the `sysmon` database. Without
|
||||
it the report is refused, logged, and skipped; the backup itself is unaffected.
|
||||
|
||||
### Planned
|
||||
- Redis integration for distributed locking
|
||||
- Webhook notifications for task completion
|
||||
|
||||
Reference in New Issue
Block a user