9 Commits
Author SHA1 Message Date
jpmschweitzerandClaude Fable 5 b4a5a92fee fix: send Scheduler API Bearer auth from the task registrar
The Scheduler's task-management endpoints (GET/POST /tasks, PUT
/tasks/{name}) are guarded by verify_api_key, but execute() built a bare
httpx.Client with no Authorization header: the existence probe 401'd
(misread as 'task absent') and every POST/PUT registration failed, so
--execute was never runnable end-to-end against the real Scheduler.

--execute now requires SCHEDULER_API_KEY from the environment (never
stored) and sends Authorization: Bearer on all registrar HTTP calls.
Deploy notes updated alongside the LIBRARY_API_KEY requirement.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QbFZyDvYksazX6nYQYZ67L
2026-07-14 15:13:16 +02:00
jpmschweitzerandClaude Fable 5 9ceec1464a fix: WS5 hazards batch - CORS, scheduler auth, reranker dedupe, wiring, write txns
- CORS: drop allow_credentials (wildcard origin + credentials told
  browsers to attach credentials for any site); origins configurable via
  CORS_ALLOW_ORIGINS (default * is safe without credentials). Verified
  live: preflight no longer advertises access-control-allow-credentials.
- Scheduler tasks: auth moved from a plain Authorization header (which
  the Scheduler's rest_api_executor does NOT env-substitute) to its
  auth {type: bearer, token: ${LIBRARY_API_KEY}} block, substituted from
  the Scheduler's own environment at execution time. The registrar no
  longer resolves the real key client-side, so it can never be persisted
  into the scheduled_tasks.config JSONB column. Also fixed: JSON bodies
  moved from the ignored "body" key to "payload" (the executor only
  reads config["payload"], so the tasks would have POSTed empty bodies
  and failed required-user validation).
- Reranker: parsed ranking indices are deduplicated preserving first
  occurrence (an LLM answer like "3,3,1" duplicated a result).
- HybridRAG wiring consolidated into dependencies.get_hybrid_rag_service
  (now including volatile_service); the inline copies in /query/hybrid
  and /wiki/pages/smart-create are gone - smart-create previously ran
  without the volatile leg, and the singleton was unused.
- Remaining Neo4j writes (GraphService ingestion/deletes/purges/entity
  mentions, webhook rename+delete cleanup, document-sync _index_graph,
  consolidation mark-processed/add-entity) moved from auto-commit
  execute_query to execute_write managed transactions with retry.

Verified end-to-end on the local dev server as llm_tester: /query/hybrid
200 with all five legs ok (volatile now active), background persistence
landed as one transaction.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QbFZyDvYksazX6nYQYZ67L
2026-07-14 14:50:03 +02:00
jpmschweitzerandClaude Fable 5 0b346d3a57 feat: add job cleanup loop, Scheduler task definitions, and registrar
- job_cleanup_loop (src/jobs/job_manager.py): hourly in-process pass over
  JobManager.cleanup_expired_jobs, started at app startup and cancelled
  at shutdown; Redis job payloads auto-expire but set memberships do not.
- docs/scheduler-tasks.md: the four production Scheduler task payloads
  for the deploy checklist - nightly integrity check 04:30, weekly
  quality report Sunday 03:00 (day_of_week=6, 0=Monday), daily Paperless
  orphan-cleanup 05:00 hitting the existing
  /maintenance/cleanup/paperless?user=jpmschweitzer&dry_run=false
  endpoint, and disabling test_example_task - with exact HTTP bodies
  (explicit user=jpmschweitzer, Authorization: Bearer ${LIBRARY_API_KEY}
  placeholder).
- scripts/register_scheduler_tasks.py: reads SCHEDULER_URL from env,
  DRY-RUN BY DEFAULT (prints the exact payloads, provably contacts
  nothing), --execute gated and requiring LIBRARY_API_KEY to fill the
  placeholder. NOT executed - definitions delivered for the deploy
  checklist only.

9 new offline tests (loop passes/error-resilience/cancellation, payload
schedules, explicit production user, placeholder, dry-run default).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QbFZyDvYksazX6nYQYZ67L
2026-07-14 12:30:27 +02:00
jpmschweitzerandClaude Opus 4.5 0a8c2639a0 docs: update MEMORY_REMEMBER_PLAN with implementation status
Mark all phases as complete (v1.5.0-v1.6.0):
- Settings DB, Phase A, B, C all implemented
- Updated files summary with actual implementations
- Added remaining work section for file upload placeholder

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-29 21:33:10 +01:00
jpmschweitzerandClaude Opus 4.5 5d4a8dba95 feat: add external API providers and central settings database
- Add central settings database client (system_settings on postgres-shared)
  - User-scoped settings with global fallback
  - API config storage with enabled/disabled toggle
  - Per-source category filtering for news

- Add modular external API providers in src/apis/:
  - OpenMeteoProvider: weather with geocoding (free, no key)
  - NOSProvider: Dutch news RSS feeds
  - BBCProvider: English news RSS feeds
  - AggregatedNewsProvider: merges sources with category filtering
  - AlphaVantageProvider: financial quotes (API key from settings DB)

- Add provider dependencies and lifecycle management
- Add requirements-dev.txt with pip-audit for security auditing
- Add MEMORY_REMEMBER_PLAN.md documenting volatile/document memory architecture

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-26 12:07:58 +01:00
jpmschweitzerandClaude Opus 4.5 4ff3fc4c7a feat: add Paperless-ngx document storage integration
- Add /documents router with webhook, upload, search, health endpoints
- Create DocumentSyncService for indexing documents to vectors/graph
- Add PaperlessClient for REST API integration
- Configure dependency injection for Paperless client
- Add document models for webhook payloads and responses
- Event-driven architecture via Paperless workflow webhooks

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-25 14:16:40 +01:00
jpmschweitzerandClaude Opus 4.5 e6e65d6d78 feat: add test data cleanup endpoint
Build and Push / build (release) Successful in 28s
Add POST /maintenance/cleanup/test-data endpoint to purge LLM test data
from wiki, graph, and vectors. Security-restricted to test user namespace
only (users/llm-tester/*, users/llm_tester/*).

- Supports dry_run=true (default) to preview before deleting
- Cleans vectors, graph nodes, and wiki pages
- Scheduler task configured for weekly cleanup (Sunday 3:00 AM)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-24 21:12:16 +01:00
jpmschweitzerandClaude Opus 4.5 f139f518ff docs: add scheduler integration and memory system plan
Documentation updates:
- AGENTS.md: Added wakeup.sh usage and local testing instructions
- LIBRARIAN_INTEGRATION.md: Complete maintenance scheduler docs
  - Scheduled task configuration for reconcile-index
  - Endpoint specifications and response formats
- docs/MEMORY_SYSTEM_PLAN.md: Three-tier memory architecture
  - Volatile (Redis TTL) for ephemeral context
  - Documents (TBD) for git mirrors, PDFs, images
  - Knowledge (Wiki + Neo4j) for permanent research

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-24 16:33:06 +01:00
jpmschweitzerandClaude Opus 4.5 f095de1162 docs: add HybridRAG architecture documentation
Documents two-stage RRF, configuration options, and notes
potential vector search noise improvements for future reference.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-15 17:54:38 +01:00