jpmschweitzerandClaude
aac293a053
Migrates TODO.md into pql and removes it. Six tickets: the double- ingestion bug with its full investigation preserved, and an epic covering the four stub endpoints in src/main.py. Markdown TODO lists cannot express blocking, parentage or status, and nothing notices when they go stale. Tickets travel with the repo — .pql/changelog/ is committed and replayed by the git hooks, while the databases are ignored and rebuildable with `pql plan rebuild`. Replaces the feature-branch mandate with the workspace convention: linear history, no merge commits, work on main or a short-lived branch that is fast-forwarded away. tatlock remains the one repo that requires branches. Adds a committed .claude/settings.json. `pql init` writes one containing only allow rules, which is the wrong shape — an allowlist with no floor under it. Every git deny appears in both `git <verb>` and `git * <verb>` form; the second catches `git -C <path>`, and without it the git denies would be decorative. .gitignore gains two entries. `.claude/settings.local.json` was only protected by a global gitignore on this machine, so the protection did not travel with the repo. The .pql rules ignore everything except the changelog, deliberately, since that file is what makes tickets portable. Co-Authored-By: Claude <noreply@anthropic.com>
50 lines
16 KiB
SQL
50 lines
16 KiB
SQL
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06FY6QDAA8EWH7D2KKQF3TBHXR', 'bug', NULL, 'Every wiki page change is ingested twice', NULL, 'backlog', 'high', NULL, NULL, NULL, '2026-08-08 21:53:14.322', '2026-08-08 21:53:14.322', NULL, 'e93392a341072352b147a3466ed3ab5b', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
|
|
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06FY6QDAA8EWH7D2KKQF3TBHXR', 'bug', NULL, 'Every wiki page change is ingested twice', 'Dockerfile runs `uvicorn ... --workers 2`, and `startup_event` in src/main.py creates a
|
|
WikiChangeListener per worker. Each opens its own PostgreSQL LISTEN connection on
|
|
`wiki_page_changes`, and Postgres delivers NOTIFY to *every* listening session — so both
|
|
workers process the same event.
|
|
|
|
Proven against production v1.9.1 on 2026-08-08:
|
|
- pg_stat_activity shows two library_desk_listener sessions, both LISTEN "wiki_page_changes"
|
|
- "Wiki.js change listener started successfully" is logged twice at startup
|
|
- a single pg_notify(''wiki_page_changes'', ''claude-probe-no-colons'') produced exactly two
|
|
"Invalid notification payload" lines, one per worker (that payload is rejected before any
|
|
ingestion runs, so it is a side-effect-free probe)
|
|
|
|
The debounce in wiki_change_listener.py (self._recent_notifications) is an in-process dict
|
|
and cannot dedupe across workers. No Redis lock or other cross-process guard exists in the
|
|
process_wiki_page_change path in src/routers/webhooks.py.
|
|
|
|
IMPACT: each edit runs ingestion twice — duplicate embedding generation against Ollama plus
|
|
duplicate Qdrant and Neo4j writes. On tower-of-joy, Ollama shares the GPU with
|
|
Speaches/Whisper, so the wasted work has a cost beyond CPU.
|
|
|
|
NOT YET ESTABLISHED: whether this is merely wasteful or actually corrupting. Check whether
|
|
repeated ingestion of the same page creates duplicate Neo4j entities/relationships or
|
|
duplicate Qdrant points — that changes the severity.
|
|
|
|
OPTIONS TO WEIGH:
|
|
- run the listener in one place only (single-worker sidecar, or elect an owner) so the
|
|
subscription is singular by construction
|
|
- keep per-worker listeners and add a short-TTL Redis claim lock keyed on page_id, so only
|
|
the first worker to claim a notification processes it. Redis is already a dependency —
|
|
the job manager uses it.
|
|
|
|
Whatever approach is taken must preserve the reconnect supervision added in v1.9.1 and its
|
|
tests in tests/test_wiki_change_listener.py.
|
|
|
|
Migrated from TODO.md 2026-08-08.', 'backlog', 'high', NULL, NULL, NULL, '2026-08-08 21:53:14.322', '2026-08-08 21:53:25.051', NULL, '34ff99ff78472b24fc35096faa05c8df', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
|
|
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06FY6QF8M3ZYNMK67A0VQYYQ7W', 'epic', NULL, 'Implement the stub endpoints in src/main.py', NULL, 'backlog', 'medium', NULL, NULL, NULL, '2026-08-08 21:53:30.273', '2026-08-08 21:53:30.273', NULL, '84c45755f930b52d22238199335e6641', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
|
|
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06FY6QF94YZ1RRP8MYEZCZKJRM', 'task', NULL, 'Implement POST /ingest/check-updates', NULL, 'backlog', 'medium', NULL, NULL, NULL, '2026-08-08 21:53:30.407', '2026-08-08 21:53:30.407', NULL, '2811d4e9372b5c838f8d524a57fc0b7c', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
|
|
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06FY6QF9KJ96C95T493C8WHQM0', 'task', NULL, 'Implement GET /ingest/status/{document_id}', NULL, 'backlog', 'medium', NULL, NULL, NULL, '2026-08-08 21:53:30.525', '2026-08-08 21:53:30.525', NULL, 'b9f5120a8a63b817dc815bda5615a75a', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
|
|
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06FY6QFA3CB42112FNPTADNCCG', 'task', NULL, 'Implement GET /ingest/repo-status/{repository}', NULL, 'backlog', 'medium', NULL, NULL, NULL, '2026-08-08 21:53:30.651', '2026-08-08 21:53:30.651', NULL, 'cb00e75f1f91e819f15f345eb18db460', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
|
|
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06FY6QFAHYRXZBK58NY73K3YSM', 'task', NULL, 'Implement POST /deduplicate/check', NULL, 'backlog', 'medium', NULL, NULL, NULL, '2026-08-08 21:53:30.767', '2026-08-08 21:53:30.767', NULL, '0300426ec940c0d43e36241d0b5968d1', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
|
|
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06FY6QF94YZ1RRP8MYEZCZKJRM', 'task', '06FY6QF8M3ZYNMK67A0VQYYQ7W', 'Implement POST /ingest/check-updates', NULL, 'backlog', 'medium', NULL, NULL, NULL, '2026-08-08 21:53:30.407', '2026-08-08 21:53:37.143', NULL, '2401d236cc1b289afa88a791a69caeec', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
|
|
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06FY6QF9KJ96C95T493C8WHQM0', 'task', '06FY6QF8M3ZYNMK67A0VQYYQ7W', 'Implement GET /ingest/status/{document_id}', NULL, 'backlog', 'medium', NULL, NULL, NULL, '2026-08-08 21:53:30.525', '2026-08-08 21:53:37.178', NULL, '36d4868e94a8f66a98fc370ac83b1e7c', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
|
|
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06FY6QFA3CB42112FNPTADNCCG', 'task', '06FY6QF8M3ZYNMK67A0VQYYQ7W', 'Implement GET /ingest/repo-status/{repository}', NULL, 'backlog', 'medium', NULL, NULL, NULL, '2026-08-08 21:53:30.651', '2026-08-08 21:53:37.179', NULL, '8b44d54948e21faf5a7fa6bf20f25281', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
|
|
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06FY6QFAHYRXZBK58NY73K3YSM', 'task', '06FY6QF8M3ZYNMK67A0VQYYQ7W', 'Implement POST /deduplicate/check', NULL, 'backlog', 'medium', NULL, NULL, NULL, '2026-08-08 21:53:30.767', '2026-08-08 21:53:37.179', NULL, 'd78fc0b019f24a626f6c9c800fc54023', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
|
|
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06FY6QF94YZ1RRP8MYEZCZKJRM', 'task', '06FY6QF8M3ZYNMK67A0VQYYQ7W', 'Implement POST /ingest/check-updates', 'Check which documents need updating based on content hashes. Used by Scheduler to determine what changed since last sync. Needs: query existing documents by path, compare content hashes, return list of updates needed. Migrated from TODO.md 2026-08-08.', 'backlog', 'medium', NULL, NULL, NULL, '2026-08-08 21:53:30.407', '2026-08-08 21:53:37.284', NULL, 'a4005f13ed2272d82bcd4b3172e9aa57', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
|
|
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06FY6QF9KJ96C95T493C8WHQM0', 'task', '06FY6QF8M3ZYNMK67A0VQYYQ7W', 'Implement GET /ingest/status/{document_id}', 'Get processing status for a document. Needs a status tracking system (Redis or database) and per-document ingestion progress. Migrated from TODO.md 2026-08-08.', 'backlog', 'medium', NULL, NULL, NULL, '2026-08-08 21:53:30.525', '2026-08-08 21:53:37.421', NULL, '29670c509669343cababf536be083e6f', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
|
|
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06FY6QFA3CB42112FNPTADNCCG', 'task', '06FY6QF8M3ZYNMK67A0VQYYQ7W', 'Implement GET /ingest/repo-status/{repository}', 'Get indexing status for an entire repository. Needs repository-level statistics and tracking of which documents from a repo are indexed. Migrated from TODO.md 2026-08-08.', 'backlog', 'medium', NULL, NULL, NULL, '2026-08-08 21:53:30.651', '2026-08-08 21:53:37.572', NULL, '76be99d15816829ef4cd210107630d76', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
|
|
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06FY6QFAHYRXZBK58NY73K3YSM', 'task', '06FY6QF8M3ZYNMK67A0VQYYQ7W', 'Implement POST /deduplicate/check', 'Check for duplicate or highly similar documents using vector similarity and graph analysis. Needs: get document embedding from Qdrant, find similar vectors above threshold, check graph relationships, return candidates with similarity scores. Migrated from TODO.md 2026-08-08.', 'backlog', 'medium', NULL, NULL, NULL, '2026-08-08 21:53:30.767', '2026-08-08 21:53:37.698', NULL, '2643ba102e78a305f8b70e329cbba84d', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
|