Compare commits

...
17 Commits
Author SHA1 Message Date
jpmschweitzer 3536c40608 fix(permissions): narrow rm -rf deny globs to their exact forms
The trailing wildcard on the three rm -rf deny entries spanned path
separators, so Bash(rm -rf /*) matched every absolute path on the
machine rather than the filesystem root, and the ~ and $HOME entries
had the same shape. Narrowed to the exact literal forms.

These rules match literal command text, so they still stop a typo on
rm -rf /, rm -rf ~ or rm -rf $HOME exactly, but they no longer stop a
recursive delete aimed at any other path. That reduced cover is
deliberate, not an oversight.
2026-08-25 20:31:15 +02:00
jpmschweitzerandClaude 0c7e1cb04e fix(tests): mock the live DNS service instance, not a dead legacy class (T-55)
test_dns_lookup_returns_result patched src.controllers.tools_controller.
DNSService, which is never imported by the request path under test.
client wraps src.main.app, which routes /tools/dns/lookup through
src.domains.tools.controller.tools_controller — a singleton constructed
at import time from src.domains.tools.dns.service.DNSService. The patched
class was dead; the mock was never consulted, so the test issued a real
DNS query for example.com and asserted on its outcome. With no network
the query times out and the assertion fails (D-26).

The three sibling tests in the same class patch the same dead class and
also run unmocked, but happen not to notice: DNS failures are caught
inside DNSService.lookup() and returned as a normal 200 response with
success=False, and their assertions only check status_code / DNSQueryError
branches that don't depend on resolution actually succeeding. Only this
test's `data["success"] is True` assertion is sensitive to the real
network outcome, which is why it's the only one D-26's namespace run
catches. Not touched here — out of T-55's scope, flagging for the record.

Fix patches tools_controller.dns_service, the actual instance attribute
the live route calls, via patch.object on the singleton rather than
patch() on the constructor class (the instance already exists by the
time a class-level patch would apply).

Verified:
- unshare -rn (lo up): 381 passed, exit 0 (was 1 failed, 380 passed, exit 2)
- with network: 381 passed, exit 0 (unchanged from before the fix)
- mutation check: retargeted the patch.object to a nonexistent attribute
  name, confirmed count==1 before editing; namespace run then reproduced
  the original failure (1 failed, 380 passed); reverted and reconfirmed
  381 passed, exit 0.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-18 15:33:58 +02:00
jpmschweitzerandClaude 6695215aeb build(make): prove setup worked instead of assuming pip's exit code (T-47)
pip install exits 0 whether the result is usable or not — that is the D-24
shape exactly, a step whose job is to not fail. On 2026-08-09 the venv here
existed and pip had already succeeded, but sqlalchemy was declared in
requirements.txt and not installed. That surfaced as 11 pytest collection
errors that read as broken imports rather than as an environment problem.

setup now ends with `pytest --collect-only tests/`, which exercises every
import the suite touches without running anything. pip check was considered
as a cheaper alternative and rejected: it only checks the installed set's
internal consistency against itself, so it would not have caught this case —
sqlalchemy was still present as another package's transitive dependency even
after being dropped from requirements.txt. collect-only checks declared vs.
actually usable directly, which is the axis that broke.

Verified: two clean runs (33.5s cold, 5.7s idempotent re-run, second changes
nothing). Reproduced the original failure by uninstalling sqlalchemy from an
otherwise-correct venv — collect-only alone then exits 2 with 11 collection
errors; make setup against that same state reinstalls it and exits 0 with
381/381 collected.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-17 12:05:28 +02:00
jpmschweitzerandClaude d2e0a49e7c build(ci): move the pre-push gate into the Makefile
The hook carried ~50 lines of gitleaks logic and a comment explaining it was
self-contained because "this repo has no Makefile". It has one now, so the
reason is gone and the arrangement is backwards: a hook is a trigger, and
logic belongs where it can be read, run by hand, and changed under review.

.githooks/pre-push is now a byte-identical shim onto `make pre-push` in every
repo in the workspace. The scan itself moves to ci/secrets.sh unchanged, and
`make secrets` runs it on its own.

The call surface is identical everywhere; what it runs is not, and should not
be — each repo gates what it actually has. That is the point of standardising
the name rather than the contents: nobody has to read a repo to find out how
to check it.

secrets runs first, deliberately. It is the only failure here that cannot be
undone by fixing it afterwards — a failed lint costs another commit, a pushed
credential is cached and indexed whether or not it is later deleted.

Some of these gates fail today, on lint debt that predates them, and they are
left wired anyway. The board was measured once and written down in T-56
instead of being worked around here. Narrowing each gate to whatever already
passes would produce a gate that reports success for doing nothing, which is
the failure this workspace keeps rediscovering.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 18:57:22 +02:00
jpmschweitzerandClaude 755aa61107 ci(make): reserve exit 69 for "could not run" (D-26)
Environment guards now exit 69 rather than 1, so a caller can tell a suite
that could not start from one that ran and failed. The first toj test sweep
reported "3 repositories failed" and none of the three had executed a test —
two could not find go, one had no venv. That points the reader at the tests
when the fault is in the environment.

Only the environment guards change. A gitleaks finding, a failed test run and
a vulncheck hit still exit 1, because those did run and did fail.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 15:56:19 +02:00
jpmschweitzerandClaude 01349a83f2 test: repair the suite against the current API
The suite could not even collect: the venv was missing declared dependencies,
and five tests asserted an API that had moved on. 11 collection errors to 381
passing.

test_oidc.py was written for the single-issuer API and 6243f29 replaced it.
issuer and audience became lists, jwks_uri stopped being an attribute in
favour of get_jwks_uri(issuer), get_jwks became get_jwks_for_issuer, and the
lru_cache became a per-issuer dict so cache_clear no longer exists. Rewritten
against the current surface, with coverage added for the two behaviours the
multi-issuer change introduced and never tested: is_valid_issuer rejecting an
unconfigured issuer, and the cache keying per issuer. Both are
security-relevant — a shared cache would serve one issuer keys for another.

Three /auth/me tests asserted a path that does not exist. The route is
declared as /me inside AuthController.create_router() and mounts at
/auth/users/me; the generated spec is authoritative and the local app and the
deployed service agree on it. Those tests had never passed.

test_handles_empty_groups expected groups == [""] for an empty header. oidc.py
has returned [] since the initial commit, and [] is correct — [""] would also
be unsafe, since any check doing "" in groups would match.

test_model_aliases_property covered Settings.model_aliases, deleted with the
Ollama integration in c1f16d4. Removed rather than repaired.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 15:21:43 +02:00
jpmschweitzerandClaude 7815e1c231 build: add the Makefile command surface (D-27)
Every repo gets one at the root: help, plus test and lint where those exist.
The point is that a target name means the same thing in every repo, so an
agent or a person can act without reading the repo first.

Paths resolve here rather than in callers (D-10). python3 on this host is 3.8
and cannot parse these sources, and a bare pytest or ruff resolves only in a
login shell — so both are named explicitly through the venv, and a missing
venv fails with the command to fix it rather than a bare no-such-file.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 15:10:31 +02:00
jpmschweitzerandClaude e9fc09af26 chore(claude): pin PQL_VAULT per project so cwd stops choosing the vault
pql is now a bare word on PATH, which removed the long incantation that had
been forcing --vault into every call by habit. Convenience lowered the cost
of the wrong thing without lowering the cost of the right one: a three-word
pql ticket new targets whichever vault the cwd happens to sit in, and there
are nine of them with colliding id sequences.

PQL_VAULT in each project settings file makes the vault a property of the
session rather than of the working directory — the same lesson Rule 3 records
for git -C, applied to pql. Verified the env var overrides cwd discovery,
that an explicit --vault still beats the env var, and that the harness
hot-reloads it without a restart.

This does not make provenance visible: no output says which vault answered,
so a forgotten --vault still returns a well-formed answer about the wrong
dataset. That remains T-37.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 13:48:58 +02:00
jpmschweitzerandClaude 56b991c965 chore(claude): deny toj in the sub-repos
toj is now on the global PATH as /usr/local/bin/toj, so its scope boundary
had to stop being "the absolute path is inconvenient to type" and start
being a rule. Its repo and settings verbs operate on the workspace root; run
from inside this repo they answer about the wrong tree.

Both spellings are denied, bare and absolute, because a deny with one
spelling left open is decorative.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 13:42:03 +02:00
jpmschweitzerandClaude 387a3dbb02 ci: gate pushes on a gitleaks scan of the outgoing commits
No repo here scanned for committed credentials. The hook is self-contained
rather than delegating to a Makefile, because this repo has none and a hook
reaching into a sibling repo breaks the moment this one is cloned elsewhere.

Scans the outgoing range rather than full history: history carries settled
findings — test fixtures, vendored third-party code — and a gate that fails
on something unfixable gets bypassed within a week.

Setting core.hooksPath means pql init must replant its replication shims into
.githooks, which is why they are gitignored here alongside the tracked
pre-push. Same layout pql itself uses.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 12:48:53 +02:00
jpmschweitzerandClaude 7f90369621 docs: qualify workspace decision ids cited from this repo
Decision ids are per-vault sequences, so they collide by construction
once there is more than one vault -- and every repo now has one. A bare
D-15 here will mean this repo's D-15 the moment this repo records one.
Cross-vault references are therefore qualified: workspace D-15.

Not hypothetical: pql holds D-1 through D-31 while the workspace holds
D-1 through D-21, so every workspace id currently collides with an
unrelated pql one. A bare id is not wrong the day it is written -- it
decays into wrong as the other vault grows, and nothing flags it.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 04:17:07 +02:00
jpmschweitzerandClaude 659f08e576 chore: adopt the workspace agent-config baseline
Commits a .claude/settings.json rather than leaving permissions to
per-developer local state, and initialises a pql vault for this repo's
tickets and internal decisions.

Every git deny rule appears in both the `git <verb>` and `git * <verb>`
forms. Only the second catches `git -C <path>`, and without it the whole
deny list is decorative -- it looks like a policy and stops nothing.

The allow list carries pql's absolute path alongside the bare name.
pql is installed to ~/.local/bin, which is on the login PATH but not the
one a non-interactive shell gets, so the bare-name rules match nothing on
their own and every call would prompt anyway.

.gitignore now covers .claude/settings.local.json, which is machine-local
and must never be shared. `pql init` contributed the .pql/* rules with an
exception for the changelog, which is the replication log of record and
has to be committed for tickets to travel with a clone.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 03:13:08 +02:00
jpmschweitzerandClaude 29bfa3259d docs: replace AGENTS.md with a repo-specific CLAUDE.md
One agent doc per repo, and it is CLAUDE.md. Two docs describing one repo
drift, and the one nobody read is always the one holding the rule that
mattered. Written fresh rather than reformatted, so the structure follows
what someone working here actually needs.

Two rules from the old file are gone deliberately. The mandate to branch
for every change was retired in favour of one linear-history policy, and
the release snippet used `git add -A`, which sweeps in whatever else is
dirty.

The architecture section is the part worth reading. An earlier draft
called src/auth, src/controllers, src/clients, src/dns and src/models
dead code, derived from grepping main.py's imports. That was wrong:
main.py:55 calls initialize_oidc(), which imports and configures
src.auth.oidc from inside the function body, so src/auth is configured
with live Authentik issuers on every boot. It also missed four genuinely
unreferenced packages. The section now states the method used -- import
the app in the container and read sys.modules -- and its blind spot, that
a cold snapshot cannot see a module imported on a request path.

Also records that the README's "runs as non-root user (uid 1000)" is
false: the Dockerfile has no USER directive. Flagged rather than fixed,
since changing the runtime user is not a docs change.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 03:12:02 +02:00
jpmschweitzerandClaude Fable 5 b3383f19b3 chore(ci): push images via git.schweitz.net registry
The .internal registry domain is being retired; git.schweitz.net now
serves the registry without SSO on /v2/.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 11:09:58 +02:00
Jeroen SchweitzerandClaude Opus 4.5 fcf5c8ccee feat: add /tools/news endpoint for news ticker integration
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m15s
- Add NewsHeadline and NewsResponse schemas
- Add NewsService for parsing news from Qdrant volatile collection
- Add GET /tools/news endpoint to tools controller
- News fetched from 'news' namespace in volatile_{user} collection

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 21:49:05 +01:00
Jeroen SchweitzerandClaude Opus 4.5 768cea2c89 fix: pass forecast raw_data to service for parsing
Build and Push / build (push) Successful in 1m14s
Build and Push / release (push) Successful in 2s
Qdrant client was extracting 'days' (int) instead of 'daily' (list)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 18:45:24 +01:00
Jeroen SchweitzerandClaude Opus 4.5 26ecc3e5fd fix: convert wind direction degrees to cardinal string
Build and Push / release (push) Successful in 2s
Build and Push / build (push) Successful in 1m14s
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 18:36:38 +01:00
28 changed files with 1598 additions and 172 deletions
+69
View File
@@ -0,0 +1,69 @@
{
"env": {
"PQL_VAULT": "/mnt/media/Projects/core-api"
},
"permissions": {
"allow": [
"Bash(pql)",
"Bash(pql *)",
"Bash(/home/jpmschweitzer/.local/bin/pql:*)",
"Bash(git status:*)",
"Bash(git log:*)",
"Bash(git diff:*)",
"Bash(git branch:*)",
"Bash(.venv/bin/python -m pytest:*)",
"Bash(.venv/bin/pytest:*)",
"Bash(pytest:*)",
"Bash(alembic *)",
"Bash(docker logs core-api:*)",
"Bash(curl -s http://localhost:8083/*)"
],
"deny": [
"Bash(/mnt/media/Projects/cladmin/ops/bin/toj)",
"Bash(/mnt/media/Projects/cladmin/ops/bin/toj:*)",
"Bash(alembic downgrade base*)",
"Bash(chmod -R 777 *)",
"Bash(chmod 777 *)",
"Bash(dd if=*)",
"Bash(find * -delete*)",
"Bash(find * -exec*)",
"Bash(git * add --all*)",
"Bash(git * add -A*)",
"Bash(git * add .)",
"Bash(git * branch -D *)",
"Bash(git * checkout -- *)",
"Bash(git * clean -fd*)",
"Bash(git * clean -fdx*)",
"Bash(git * commit --no-verify*)",
"Bash(git * merge --no-ff*)",
"Bash(git * push --force*)",
"Bash(git * push -f*)",
"Bash(git * reset --hard*)",
"Bash(git * restore .*)",
"Bash(git add --all*)",
"Bash(git add -A*)",
"Bash(git add .)",
"Bash(git branch -D *)",
"Bash(git checkout -- *)",
"Bash(git clean -fd*)",
"Bash(git clean -fdx*)",
"Bash(git commit --no-verify*)",
"Bash(git merge --no-ff*)",
"Bash(git push --force*)",
"Bash(git push -f*)",
"Bash(git reset --hard*)",
"Bash(git restore .*)",
"Bash(mkfs*)",
"Bash(psql * -c DROP*)",
"Bash(psql * DROP DATABASE*)",
"Bash(psql * TRUNCATE*)",
"Bash(rm -rf $HOME)",
"Bash(rm -rf /)",
"Bash(rm -rf ~)",
"Bash(su *)",
"Bash(sudo *)",
"Bash(toj)",
"Bash(toj:*)"
]
}
}
+1
View File
@@ -0,0 +1 @@
.pql/changelog/*.sql merge=union
+3 -3
View File
@@ -26,7 +26,7 @@ jobs:
- name: Login to Gitea Registry - name: Login to Gitea Registry
uses: docker/login-action@v3 uses: docker/login-action@v3
with: with:
registry: git.schweitz.internal registry: git.schweitz.net
username: ${{ secrets.REGISTRY_USER }} username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }} password: ${{ secrets.REGISTRY_PASSWORD }}
@@ -36,8 +36,8 @@ jobs:
context: . context: .
push: true push: true
tags: | tags: |
git.schweitz.internal/jpmschweitzer/core-api:latest git.schweitz.net/jpmschweitzer/core-api:latest
git.schweitz.internal/jpmschweitzer/core-api:${{ github.ref_name }} git.schweitz.net/jpmschweitzer/core-api:${{ github.ref_name }}
- name: Trigger Watchtower update - name: Trigger Watchtower update
if: success() if: success()
+13
View File
@@ -0,0 +1,13 @@
#!/usr/bin/env bash
# Trigger only. The checks live in the Makefile, where they can be read, run by
# hand (`make pre-push`), and changed under review.
#
# This file is identical in every repo in this workspace, deliberately: the call
# surface is the same everywhere even though what each gate runs is not, so
# nobody has to read a repo to find out how to check it (D-27).
#
# Enable per clone with: git config core.hooksPath .githooks
# Never bypass with --no-verify. Suppress a specific finding deliberately
# instead, with a reason — see `make pre-push`.
set -euo pipefail
exec make -C "$(git rev-parse --show-toplevel)" pre-push
+17
View File
@@ -136,3 +136,20 @@ Thumbs.db
# profiling data # profiling data
.prof .prof
# Claude Code local settings (machine-specific, may contain credentials)
.claude/settings.local.json
# pql — ignore everything except the changelog, which is the replication log of
# record and must be committed for tickets to travel with the repo.
.pql/*
!.pql/changelog/
# pql shims planted by `pql init` into the dir core.hooksPath points at.
# Per-clone: each embeds the absolute path of the pql binary that planted it.
# Only .githooks/pre-push is shared.
.githooks/pre-commit
.githooks/post-merge
.githooks/post-checkout
.githooks/post-rewrite
+11
View File
@@ -0,0 +1,11 @@
-- Changelog format marker, written by pql. Comments only: this file
-- is never executed — Import descends into the per-table directories
-- and does not read the changelog root.
--
-- A changelog carrying no marker is format 1, the shape that existed
-- before formats were versioned. An older format is migrated forward
-- by `pql plan upgrade` (and automatically from the post-merge hook);
-- a newer one is refused rather than replayed under rules this binary
-- does not know. See D-28 and docs/versions.md.
-- pql:changelog_format: 2.0.0
-- pql:written_by: 2.2.0
+139
View File
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
+139
View File
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
+139
View File
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
-72
View File
@@ -1,72 +0,0 @@
# AGENTS.md
> **Start every session by reading this file.**
> This file outlines the operational protocols, coding standards, and architectural decisions for this FastAPI project.
## 1. Agent Operational Protocols
### 🧠 Work Patterns (Plan-Act-Reflect)
* **Plan:** Before writing code, briefly outline your plan. Identify which files you will touch and what the side effects might be.
* **Act:** Execute the changes in small, atomic steps.
* **Reflect:** After coding, verify your work. Did you break existing tests? Did you add new tests?
### 🛡️ Git Discipline
* **NEVER commit to `main` or `master` directly.** Always create a feature branch: `feature/your-feature-name` or `fix/issue-description`.
* **Commit Messages:** Use the [Conventional Commits](https://www.conventionalcommits.org/) format.
* `feat: add user login endpoint`
* `fix: resolve database connection timeout`
* `refactor: split monolith dependency file`
* **Atomic Commits:** Keep commits small. One logical change = one commit.
### 📝 Changelog Maintenance
* **Update `CHANGELOG.md`** with every user-facing change.
* Format: `## [Unreleased] - YYYY-MM-DD` followed by `### Added`, `### Changed`, or `### Fixed`.
### 🚀 Release Flow
When changes are ready for deployment:
1. **Ask user if deploy cycle is desired **
2. **Update version** in `pyproject.toml`:
- Bug fixes: bump patch version (1.8.3 → 1.8.4)
- New features: bump minor version (1.8.4 → 1.9.0)
3. **Update CHANGELOG.md**:
- Move items from `[Unreleased]` to new version section
- Add release date: `## [1.8.4] - 2025-12-16`
4. **Commit and tag**:
```bash
git add -A
git commit -m "fix: description of changes"
git tag v1.8.4
git push origin main --tags
```
5. **CI/CD triggers automatically**:
- Gitea CI builds Docker image on new tag
- Watchtower pulls and deploys to production
- Verify deployment: `curl http://192.168.86.149:8083/health`
---
## 2. FastAPI Architecture & Best Practices
*Reference: [FastAPI Best Practices](https://github.com/zhanymkanov/fastapi-best-practices)*
### 📂 Project Structure (Directory-based, NOT File-type based)
Do **not** group files by type (e.g., one huge `routers` folder). Group by **domain/module** inside a `src/` directory.
**Correct Structure:**
```text
src/
├── auth/
│ ├── router.py # Endpoints
│ ├── schemas.py # Pydantic models
│ ├── service.py # Business logic (CRUD, etc.)
│ ├── dependencies.py# Module-specific dependencies
│ └── config.py # Module-specific settings
├── posts/
│ ├── router.py
│ └── ...
└── main.py # App entry point
+30
View File
@@ -5,6 +5,36 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [1.11.0] - 2026-01-08
### Added
- **News Headlines API** - New endpoint for news ticker integration
- `GET /tools/news` - Fetch news headlines from user's volatile collection
- Returns headlines with title, description, source, and URL
- Data sourced from `volatile_{user}` Qdrant collection (news namespace)
- Uses `preferred_username` from OIDC, falls back to `default`
- News subdomain under tools (`src/domains/tools/news/`)
- `NewsHeadline` and `NewsResponse` Pydantic schemas
- `NewsService` for parsing news data from Qdrant
## [1.10.12] - 2026-01-08
### Fixed
- **Forecast data retrieval** - Pass raw_data to service instead of extracting wrong field
- Qdrant client was extracting `days` (integer 7) instead of `daily` (list)
- Now passes full raw_data for service to parse correctly
## [1.10.11] - 2026-01-08
### Fixed
- **Weather wind direction** - Convert integer degrees to cardinal direction string
- Scheduler stores wind_direction as degrees (e.g., 135)
- Schema expects string (e.g., "SE")
- Added `_degrees_to_cardinal()` conversion
## [1.10.10] - 2026-01-08 ## [1.10.10] - 2026-01-08
### Fixed ### Fixed
+177
View File
@@ -0,0 +1,177 @@
# CLAUDE.md — core-api
FastAPI service providing infrastructure management, home automation, and utility
endpoints for the homelab. Talks to Portainer, Nginx Proxy Manager, Home Assistant,
Postgres (via SQLAlchemy async + Alembic), Qdrant, and Authentik (OIDC). Deployed on
tower-of-joy at **:8083**.
## Ports — these differ, deliberately
| | Port | How |
|---|---|---|
| Local dev | **8788** | `./wakeup.sh`, uvicorn `--reload`, logs to `logs/server.log` |
| Production | **8083** | container; health at `http://192.168.86.149:8083/health` |
Testing `localhost:8083` on the dev box hits the *container*, not your reload server.
## Live contract
The live contract is always `http://localhost:8083/openapi.json` (62 paths, verified
2026-08-09) and human docs at `http://localhost:8083/docs` / `/redoc` — generated from
running code, so query it rather than inferring routes from source or from the README's
endpoint list, which can drift.
## Architecture
Domain-first layout under `src/domains/<name>/{controller,models,schemas,service}.py`
(auth, dashboard, health, housekeeping, infrastructure, static, tools). `src/main.py`
wires only `src.domains.*` — verify by reading its imports.
**Legacy top-level packages — "not in `main.py`" does not mean dead.** Routes are wired
only from `src.domains.*`, so grepping `main.py`'s imports looks like it settles which
packages are live. It does not. `main.py:55` calls `initialize_oidc()`, and that function
(`src/shared/security.py:21`) deliberately imports and configures **both** `src.auth.oidc`
and `src.domains.auth.oidc` — a function-body import, invisible to a grep of `main.py`.
That one call drags in `src/auth/`, `src/controllers/`, `src/db/`, `src/logging_config.py`
and `src/base_schema.py` at startup.
Three tiers, established by importing the app inside the container and reading
`sys.modules` (verified 2026-08-09):
| Tier | Packages |
|---|---|
| Serving routes | `src/domains/`, `src/shared/`, `src/service_groups/` |
| **Loaded and configured**, but serving no routes | `src/auth/`, `src/db/`, `src/controllers/`, `src/logging_config.py`, `src/base_schema.py` |
| Genuinely unreferenced | `src/agent/`, `src/api/`, `src/clients/`, `src/dns/`, `src/memory/`, `src/models/` |
`src/auth/` is the trap. Its `oidc_config` singleton is configured at every startup with
the real Authentik issuers — the log line `src.auth.oidc:configure` proves it — so a test
importing `src.auth.oidc` is exercising live, configured code, not a fossil. No `src/domains/*`
module depends on it, so it is configured defensively rather than used; that makes it a
deletion candidate, but a considered one, not obvious cleanup.
**Before deleting anything from `src/`, import the app and read `sys.modules`** rather than
grepping `main.py`. Function-body imports exist here specifically to dodge circular imports,
and they are exactly what a grep misses.
**That check has its own blind spot, so do not read the third tier as a delete list.** The
table above is a snapshot taken after a cold `import src.main` — it shows what *startup*
loads. A module imported inside a request handler would be absent from it while being
entirely live, and absence would then be a timing artifact rather than evidence of death.
This bit on webber, where a tool package imported from inside an agent method looked
unloaded and was serving every request. Nothing in core-api is currently known to work that
way, but that is the weaker claim — it means nobody has exercised the routes and re-checked,
not that nobody does it. Before deleting a third-tier package, drive the endpoints that
would plausibly load it and take the snapshot again.
Some tests (`test_auth_controller.py`, `test_oidc.py`, `test_npm_client.py`,
`test_portainer_client.py`, `test_static_controller.py`, `test_tools_controller.py`) import
from the top-level paths rather than `src.domains.*`. Which of those cover live code follows
the table above — `test_oidc.py` does; the client tests target the unreferenced tier. Not
cleaned up in this pass; flagged, not fixed.
Shared infra (config, database, logging, security/OIDC, external API clients) lives in
`src/shared/`.
Group new work by **domain, not by file type** — a single large `routers/` folder is
the thing to avoid. Reference: [FastAPI best practices](https://github.com/zhanymkanov/fastapi-best-practices).
## Database
SQLAlchemy 2.0 async + asyncpg, migrations via Alembic (`alembic/versions/`). Models
live under `src/domains/<name>/models.py` and must be imported in `alembic/env.py` to
register with `Base.metadata` — check that file when adding a new model or `alembic
revision --autogenerate` will silently miss it.
## Working here
**Plan, act, reflect.** Outline which files you will touch and the side effects before
writing. Change in small atomic steps. Afterwards, verify: did existing tests break, and
does the new behaviour have a test?
**Test locally first — the build-deploy loop is slow.** `./wakeup.sh` auto-reloads on
code changes (not on `requirements.txt` changes; restart the container/script after
adding a dependency). Deploy only when a feature is complete and tested.
Run tests through the venv explicitly, to avoid environment mismatch:
```bash
.venv/bin/python -m pytest tests/
# or, with coverage:
.venv/bin/python -m pytest --cov=src --cov-report=term-missing
```
Copy `.env.example` to `.env` and configure Portainer, NPM, Home Assistant, SearXNG,
Postgres, and Qdrant hosts/credentials.
No linter is configured in this repo (no ruff/flake8 config, none in `requirements.txt`
or `dev-requirements.txt`) — unlike some sibling repos, don't assume `ruff check` exists
here.
## CI
`.gitea/workflows/build.yml` is the only workflow: triggered on `v*` tag push, it
creates a Gitea release, builds and pushes the image, then pings Watchtower. There is
**no CI test/lint gate** — pytest only runs locally or on request. Verify tests pass
before tagging a release.
## Work tracking
Work lives in **pql**, not a markdown TODO. **This repo's vault is standalone** — its tickets
and its internal decisions live here in `.pql/` and `governance/`, and travel with a clone,
because `.pql/changelog/` is committed and replayed by the git hooks (workspace D-15). The databases are
gitignored and rebuildable with `pql plan rebuild`.
`pql` is **not** on the non-interactive `PATH` — invoke it as
`/home/jpmschweitzer/.local/bin/pql`. From inside this repo no `--vault` is needed: pql anchors
at the nearest `.git/` ancestor, which is this repo.
```bash
/home/jpmschweitzer/.local/bin/pql ticket list # this repo's open work
/home/jpmschweitzer/.local/bin/pql plan whatsnext # next unblocked item, with context
/home/jpmschweitzer/.local/bin/pql decisions list # this repo's own decisions
```
Stack-level decisions that constrain this service live in the **workspace** vault and need the
flag:
```bash
/home/jpmschweitzer/.local/bin/pql --vault /mnt/media/Projects decisions list --domain core-api
```
Note `ticket new --decision D-N` resolves ids within **one** vault, so a ticket here cannot link
to a workspace decision. Cite the id in the ticket body instead.
Do not add a TODO section to a markdown file.
## Git
- **History is linear — no merge commits.** Work on `main`, or a short-lived branch
that is fast-forwarded and deleted. (This repo's AGENTS.md previously mandated a
feature branch for every change; that rule was retired workspace-wide on 2026-08-08
and does not apply here anymore.)
- **Conventional Commits**: `feat:`, `fix:`, `refactor:`, `docs:`, `chore:`.
- **Atomic commits** — one logical change each.
- **Stage explicitly. Never `git add -A`** — it is denied by policy, and it sweeps in
whatever else is dirty, including secrets.
- Update `CHANGELOG.md` with every user-facing change, under `[Unreleased]` in `Added` /
`Changed` / `Fixed`.
## Releasing
Ask whether a deploy is wanted first — it is not automatic.
1. Bump the version in `pyproject.toml` (patch for fixes, minor for features).
2. Move `[Unreleased]` entries into a dated version section in `CHANGELOG.md`.
3. Stage the changed files by name, commit, tag `vX.Y.Z`, `git push origin main --tags`.
4. Gitea CI (`build.yml`) builds and pushes the image on the tag; Watchtower deploys it.
5. Verify: `curl http://192.168.86.149:8083/health`.
## Security
- OIDC authentication via Authentik, multi-issuer/multi-audience support.
- Admin endpoints require authentication when `OIDC_ENABLED=true`.
- README claims the container "runs as non-root user (uid 1000)" — **checked and
false**: the Dockerfile has no `USER` directive, so the container runs as root.
Not fixed here (out of scope for a docs normalization pass); flagging so it isn't
restated as fact.
+67
View File
@@ -0,0 +1,67 @@
# core-api — the repo's command surface (D-27).
#
# Paths resolve here rather than in callers (D-10). `python3` on this host is
# 3.8 and cannot parse these sources, and a bare `pytest` resolves only in a
# login shell — so both are named explicitly through the venv.
VENV := $(CURDIR)/.venv
PYTHON ?= python3.12
.DEFAULT_GOAL := help
.PHONY: help
help: ## Show this help
@grep -hE '^[a-z][a-z0-9_-]*:.*?## ' $(MAKEFILE_LIST) \
| awk 'BEGIN{FS=":.*?## "}{printf " \033[36m%-14s\033[0m %s\n", $$1, $$2}'
.PHONY: setup
setup: ## Create the venv, install dependencies, and prove the result is usable
$(PYTHON) -m venv .venv
$(VENV)/bin/pip install -r requirements.txt -r dev-requirements.txt
# Exit 0 from pip install is not evidence (D-24) — it is the step's job not
# to fail, so a broken result and a working one look identical from here.
# On 2026-08-09 the venv existed and pip had exited 0, but sqlalchemy was
# declared in requirements.txt and not installed; that surfaced as 11
# pytest collection errors that read as broken imports, not as a setup
# problem. `--collect-only` exercises every import the suite touches
# without running a single test, so it catches exactly that class of
# drift and stays cheap. `pip check` was considered too, but it only
# verifies the *installed* set's internal consistency against itself —
# it would not have caught this case, because sqlalchemy was still
# present as another package's transitive dependency even when dropped
# from requirements.txt. collect-only checks declared-vs-actually-usable
# directly, which is the axis that broke.
$(VENV)/bin/python -m pytest --collect-only tests/
.PHONY: test
test: ## Run the test suite
@test -x $(VENV)/bin/python || { echo "FAIL — no venv; run: make setup"; exit 69; }
$(VENV)/bin/python -m pytest tests/
# No `lint` target, deliberately. This repo configures no linter — no ruff or
# flake8 config, and neither in requirements. Per D-27 the name is reserved for
# repos that lint; an empty target here would report clean for something never
# run. Add the target when a linter is added, not before.
# git hands a hook a non-login shell, which never sees ~/.local/bin — where
# gitleaks lands. Without this the scan reports "not installed" on every push,
# which is a check that fails open (D-24).
export PATH := $(HOME)/.local/bin:/usr/local/bin:$(PATH)
.PHONY: secrets
secrets: ## Scan the commits about to be pushed for credentials
@ci/secrets.sh
# The call surface is identical in every repo; what it runs is not.
#
# `secrets` runs first, deliberately: it is the only failure here that cannot be
# undone by fixing it afterwards. A failed lint costs another commit; a pushed
# credential is cached and indexed whether or not it is later deleted.
#
# Some of these fail today, and are left wired anyway. The state was measured
# once and written down in T-56 rather than being worked around here — a gate
# quietly narrowed to what already passes is a gate that reports success for
# doing nothing, which is the failure this workspace keeps rediscovering.
.PHONY: pre-push
pre-push: secrets ## Everything the pre-push hook runs
@echo " -- not gated here yet: lint (no linter configured) and test (T-56)"
Executable
+50
View File
@@ -0,0 +1,50 @@
#!/usr/bin/env bash
# Secret scan over the commits about to be pushed.
#
# Lives here rather than inside .githooks/pre-push so it can be read, run by
# hand (`make secrets`), and changed under review. A hook is a trigger; it is
# not a home for logic. Identical in every repo in this workspace (D-27).
set -euo pipefail
cd "$(git rev-parse --show-toplevel)"
# A non-login shell — which is what git gives a hook — skips /etc/profile.d
# and never sees ~/.local/bin, where the gitleaks release tarball lands.
# Without this the scan reports "not installed" on every push.
[ -d "$HOME/.local/bin" ] && PATH="$HOME/.local/bin:$PATH"
if ! command -v gitleaks >/dev/null 2>&1; then
echo "FAIL secrets — gitleaks not installed, so this check would be a no-op pretending to pass." >&2
echo " https://github.com/gitleaks/gitleaks/releases → ~/.local/bin/gitleaks" >&2
exit 1
fi
# Scan the outgoing range, not full history. History here carries findings
# that are settled — test fixtures and vendored third-party code — and a gate
# that fails on something unfixable gets bypassed within a week. What matters
# is what is about to leave this machine.
if upstream=$(git rev-parse --abbrev-ref --symbolic-full-name '@{u}' 2>/dev/null); then
range="$upstream..HEAD"
elif git rev-parse --verify --quiet origin/main >/dev/null; then
range="origin/main..HEAD"
else
range=""
fi
if [ -z "$range" ]; then
gitleaks dir . --redact --no-banner --exit-code 1 || {
echo "FAIL secrets — gitleaks found a credential in the working tree." >&2; exit 1; }
exit 0
fi
[ -n "$(git log --oneline "$range" 2>/dev/null)" ] || exit 0
gitleaks git . --log-opts="$range" --redact --no-banner --exit-code 1 >/dev/null 2>&1 || {
echo "FAIL secrets — gitleaks found a credential in the commits being pushed." >&2
echo " inspect (values redacted): gitleaks git . --log-opts=\"$range\" --redact" >&2
echo " then remove and rotate it, or suppress deliberately:" >&2
echo " inline '# gitleaks:allow <reason>'" >&2
echo " or add the fingerprint to .gitleaksignore WITH a reason" >&2
exit 1
}
echo " ok secrets"
+54
View File
@@ -0,0 +1,54 @@
# Decisions, Questions, Rejected
This directory holds structured planning records that pql parses
into pql.db. Each record is a `### [DQR]-N: Title` heading inside
a markdown file. Files live in three per-type subdirectories:
- `decisions/<domain>.md` — confirmed design decisions
- `questions/<domain>.md` — open questions that may resolve into
decisions or rejected proposals
- `rejected/<domain>.md` — rejected proposals (kept for the audit
trail)
The parser infers domain from the filename stem and record type
from the parent subdirectory.
D-records that propose implementation work link to `initiative`-type
tickets via `decision_ref`. Run `pql decisions show <id>
--with-tickets` to inspect implementation status.
## Recommended domains
Start with this canonical set; create files as records land in
each domain:
- **architecture** — structural commitments (storage, layering,
languages, libraries)
- **process** — team workflow (commits, branches, releases, reviews)
- **design** — user-facing surface (UX, UI, public APIs)
- **coding-conventions** — team-internal code shape (style, lint,
file layout)
- **testing** — quality strategy (coverage, layers, gates)
You might also want, project-permitting:
- `accessibility` — if you ship user-facing software
- `security` — if you handle user data or network surfaces
- `licensing` — if you release open-source or commercial
- `documentation` — if user-docs are non-trivial
- `deployment` — if shipping is non-trivial
- `performance` — if you have perf budgets / SLOs
<!-- pql:records (auto-generated; do not edit manually) -->
## Decisions
- _(none)_
## Open questions
- _(none)_
## Rejected
- _(none)_
+1 -1
View File
@@ -1,6 +1,6 @@
[project] [project]
name = "core-api" name = "core-api"
version = "1.10.10" version = "1.11.0"
description = "Core Code API - Infrastructure management and tools API" description = "Core Code API - Infrastructure management and tools API"
readme = "README.md" readme = "README.md"
requires-python = ">=3.12" requires-python = ">=3.12"
+62
View File
@@ -18,6 +18,8 @@ from src.domains.tools.system.schemas import SystemStatsResponse
from src.domains.tools.system.service import SystemStatsService from src.domains.tools.system.service import SystemStatsService
from src.domains.tools.environment.schemas import EnvironmentResponse from src.domains.tools.environment.schemas import EnvironmentResponse
from src.domains.tools.environment.service import EnvironmentService from src.domains.tools.environment.service import EnvironmentService
from src.domains.tools.news.schemas import NewsResponse
from src.domains.tools.news.service import NewsService
from src.domains.auth.oidc import get_optional_user from src.domains.auth.oidc import get_optional_user
logger = get_logger(__name__) logger = get_logger(__name__)
@@ -38,6 +40,7 @@ class ToolsController(BaseController):
self.dns_service = DNSService() self.dns_service = DNSService()
self.system_stats_service = SystemStatsService() self.system_stats_service = SystemStatsService()
self.environment_service = EnvironmentService() self.environment_service = EnvironmentService()
self.news_service = NewsService()
def create_router(self) -> APIRouter: def create_router(self) -> APIRouter:
"""Create and configure the router""" """Create and configure the router"""
@@ -213,6 +216,65 @@ class ToolsController(BaseController):
detail=f"Failed to fetch environment data: {str(e)}" detail=f"Failed to fetch environment data: {str(e)}"
) )
@router.get(
"/news",
response_model=NewsResponse,
status_code=status.HTTP_200_OK,
summary="Get news headlines",
description="""
Get news headlines for the authenticated user.
Fetches news data from the Qdrant volatile collection for the authenticated user.
Falls back to 'default' user if not authenticated.
**Data Returned:**
- **Headlines:** List of news headlines with title, description, source, url
- **Category:** News category (general, technology, etc.)
- **Sources:** List of news sources
**Data Source:** Qdrant volatile_{user} collection (news namespace)
**Use Cases:**
- Dashboard news ticker
- News feed widgets
- Information display
"""
)
async def get_news(
user: Optional[Dict] = Depends(get_optional_user),
) -> NewsResponse:
"""
Get news headlines
Args:
user: Optional authenticated user from OIDC
Returns:
News headlines response
Raises:
HTTPException: 500 for processing errors
"""
try:
# Get user identifier from OIDC claims, fallback to 'default'
user_id = "default"
if user:
user_id = user.get("preferred_username") or user.get("sub", "default")
# Strip email domain if present (e.g., "user@example.com" -> "user")
if "@" in user_id:
user_id = user_id.split("@")[0]
logger.info(f"Fetching news data for user: {user_id}")
result = await self.news_service.get_news(user_id)
return result
except Exception as e:
logger.error(f"Failed to get news data: {str(e)}", exc_info=True)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Failed to fetch news data: {str(e)}"
)
return router return router
+13 -1
View File
@@ -43,13 +43,18 @@ class EnvironmentService:
return None return None
try: try:
# Handle wind direction - convert degrees to cardinal if integer
wind_dir = raw_data.get("wind_direction") or raw_data.get("wind_dir")
if isinstance(wind_dir, (int, float)):
wind_dir = self._degrees_to_cardinal(wind_dir)
return WeatherData( return WeatherData(
temperature=raw_data.get("temperature") or raw_data.get("temp"), temperature=raw_data.get("temperature") or raw_data.get("temp"),
feels_like=raw_data.get("feels_like") or raw_data.get("feelslike"), feels_like=raw_data.get("feels_like") or raw_data.get("feelslike"),
conditions=raw_data.get("conditions") or raw_data.get("weather") or raw_data.get("description"), conditions=raw_data.get("conditions") or raw_data.get("weather") or raw_data.get("description"),
humidity=raw_data.get("humidity"), humidity=raw_data.get("humidity"),
wind_speed=raw_data.get("wind_speed") or raw_data.get("windspeed") or raw_data.get("wind"), wind_speed=raw_data.get("wind_speed") or raw_data.get("windspeed") or raw_data.get("wind"),
wind_direction=raw_data.get("wind_direction") or raw_data.get("wind_dir"), wind_direction=wind_dir,
pressure=raw_data.get("pressure"), pressure=raw_data.get("pressure"),
visibility=raw_data.get("visibility"), visibility=raw_data.get("visibility"),
uv_index=raw_data.get("uv_index") or raw_data.get("uv"), uv_index=raw_data.get("uv_index") or raw_data.get("uv"),
@@ -60,6 +65,13 @@ class EnvironmentService:
logger.warning(f"Failed to parse weather data: {e}") logger.warning(f"Failed to parse weather data: {e}")
return None return None
def _degrees_to_cardinal(self, degrees: float) -> str:
"""Convert wind direction degrees to cardinal direction."""
directions = ["N", "NNE", "NE", "ENE", "E", "ESE", "SE", "SSE",
"S", "SSW", "SW", "WSW", "W", "WNW", "NW", "NNW"]
index = round(degrees / 22.5) % 16
return directions[index]
def _parse_forecast(self, raw_data: Any) -> Optional[List[ForecastDay]]: def _parse_forecast(self, raw_data: Any) -> Optional[List[ForecastDay]]:
""" """
Parse raw forecast data into list of ForecastDay schemas. Parse raw forecast data into list of ForecastDay schemas.
+5
View File
@@ -0,0 +1,5 @@
"""
News subdomain for Tools.
Provides news headlines from Qdrant volatile collection.
"""
+56
View File
@@ -0,0 +1,56 @@
"""
News data schemas for Tools domain.
Provides Pydantic models for news headlines retrieved from the Qdrant volatile collection.
"""
from datetime import datetime
from typing import Optional, List
from pydantic import Field
from src.shared.base import BaseSchema
class NewsHeadline(BaseSchema):
"""Single news headline."""
title: str = Field(
...,
description="Headline title"
)
description: Optional[str] = Field(
None,
description="Brief description or summary"
)
source: Optional[str] = Field(
None,
description="News source name"
)
url: Optional[str] = Field(
None,
description="Link to full article"
)
class NewsResponse(BaseSchema):
"""News headlines response."""
headlines: List[NewsHeadline] = Field(
default_factory=list,
description="List of news headlines"
)
category: Optional[str] = Field(
None,
description="News category (e.g., 'general', 'technology')"
)
sources: Optional[List[str]] = Field(
None,
description="List of source names"
)
updated_at: datetime = Field(
default_factory=datetime.utcnow,
description="Timestamp when data was fetched"
)
user: Optional[str] = Field(
None,
description="User identifier used for data lookup"
)
+113
View File
@@ -0,0 +1,113 @@
"""
News data service for Tools domain.
Fetches news headlines from the Qdrant volatile collection.
"""
from datetime import datetime
from typing import Optional, Dict, Any, List
from src.shared.logging import get_logger
from src.shared.clients.qdrant_client import get_qdrant_client
from src.domains.tools.news.schemas import (
NewsHeadline,
NewsResponse,
)
logger = get_logger(__name__)
class NewsService:
"""
Service for fetching news data from Qdrant volatile collection.
Retrieves news headlines for a specific user.
"""
def __init__(self):
"""Initialize news service with Qdrant client."""
self.qdrant = get_qdrant_client()
def _parse_headlines(self, raw_data: Any) -> List[NewsHeadline]:
"""
Parse raw news data into list of NewsHeadline schemas.
Handles various formats from different news sources.
"""
if not raw_data:
return []
try:
# Handle dict with nested headlines list
headlines_list = raw_data
if isinstance(raw_data, dict):
headlines_list = raw_data.get("headlines") or raw_data.get("articles") or []
if not isinstance(headlines_list, list):
return []
headlines = []
for item in headlines_list:
if isinstance(item, dict):
headlines.append(NewsHeadline(
title=item.get("title", ""),
description=item.get("description") or item.get("summary"),
source=item.get("source") or item.get("provider"),
url=item.get("url") or item.get("link"),
))
elif isinstance(item, str):
# Simple string headlines
headlines.append(NewsHeadline(title=item))
return headlines
except Exception as e:
logger.warning(f"Failed to parse news headlines: {e}")
return []
async def get_news(self, user: str = "default") -> NewsResponse:
"""
Get news headlines for a user.
Fetches news from the user's volatile collection.
Args:
user: User identifier (default: 'default')
Returns:
NewsResponse with headlines
"""
logger.info(f"Fetching news data for user: {user}")
# Get raw data from Qdrant
news_records = await self.qdrant.get_by_namespace(user, "news")
headlines = []
category = None
sources = None
if news_records:
raw_data = news_records[0].get("raw_data", {})
headlines = self._parse_headlines(raw_data)
if isinstance(raw_data, dict):
category = raw_data.get("category")
sources = raw_data.get("sources")
return NewsResponse(
headlines=headlines,
category=category,
sources=sources,
updated_at=datetime.utcnow(),
user=user,
)
# Singleton instance
_news_service: Optional[NewsService] = None
def get_news_service() -> NewsService:
"""Get or create singleton news service instance."""
global _news_service
if _news_service is None:
_news_service = NewsService()
return _news_service
+2 -11
View File
@@ -190,19 +190,10 @@ class QdrantReadClient:
if aqi: if aqi:
result["air_quality"] = aqi if isinstance(aqi, dict) else {"aqi": aqi} result["air_quality"] = aqi if isinstance(aqi, dict) else {"aqi": aqi}
# Fetch forecast data # Fetch forecast data - pass raw_data to service for parsing
forecast_records = await self.get_by_namespace(user, "forecast") forecast_records = await self.get_by_namespace(user, "forecast")
if forecast_records: if forecast_records:
# Forecast might be a single record with list or multiple records result["forecast"] = forecast_records[0].get("raw_data")
first_record = forecast_records[0].get("raw_data")
if isinstance(first_record, list):
result["forecast"] = first_record
elif isinstance(first_record, dict):
# Could be a dict with 'days' or 'forecast' key
result["forecast"] = first_record.get(
"days",
first_record.get("forecast", [first_record])
)
# Fetch sun times data # Fetch sun times data
sun_records = await self.get_by_namespace(user, "sun") sun_records = await self.get_by_namespace(user, "sun")
+21 -7
View File
@@ -815,22 +815,32 @@ class TestPhase4Schemas:
# ============================================================================= # =============================================================================
# GET /auth/me Endpoint Tests (NPM Forward Auth) # GET /auth/users/me Endpoint Tests (NPM Forward Auth)
# ============================================================================= # =============================================================================
class TestAuthMeEndpoint: class TestAuthMeEndpoint:
"""Test GET /auth/me endpoint with NPM forward auth.""" """
Test GET /auth/users/me with NPM forward auth.
The path is /auth/users/me, not /auth/me — the route is declared as "/me"
inside AuthController.create_router(), which mounts under a users prefix.
These tests asserted /auth/me and had never passed; the generated spec is
authoritative and both the local app and the deployed service agree on 62
paths including this one.
"""
PATH = "/auth/users/me"
def test_auth_me_in_openapi(self, client): def test_auth_me_in_openapi(self, client):
"""Auth me endpoint should be in OpenAPI spec.""" """Auth me endpoint should be in OpenAPI spec."""
response = client.get("/openapi.json") response = client.get("/openapi.json")
spec = response.json() spec = response.json()
assert "/auth/me" in spec["paths"] assert self.PATH in spec["paths"]
assert "get" in spec["paths"]["/auth/me"] assert "get" in spec["paths"][self.PATH]
def test_auth_me_returns_401_without_forward_auth(self, client): def test_auth_me_returns_401_without_forward_auth(self, client):
"""Should return 401 when accessed without forward auth headers.""" """Should return 401 when accessed without forward auth headers."""
response = client.get("/auth/me") response = client.get(self.PATH)
# Without NPM forward auth headers, should return 401 # Without NPM forward auth headers, should return 401
assert response.status_code == 401 assert response.status_code == 401
@@ -840,7 +850,7 @@ class TestAuthMeEndpoint:
spec = response.json() spec = response.json()
# Check response schema references AuthSyncResponse # Check response schema references AuthSyncResponse
me_endpoint = spec["paths"]["/auth/me"]["get"] me_endpoint = spec["paths"][self.PATH]["get"]
assert "responses" in me_endpoint assert "responses" in me_endpoint
assert "200" in me_endpoint["responses"] assert "200" in me_endpoint["responses"]
@@ -920,7 +930,11 @@ class TestForwardAuthParsing:
result = await get_forward_auth_user(mock_request) result = await get_forward_auth_user(mock_request)
assert result["groups"] == [""] # An empty groups header means no groups, not one group named "".
# oidc.py has guarded this since the initial commit — this assertion
# expected [""] and had never passed. [""] would also be unsafe: any
# authorization check doing `"" in groups` would match.
assert result["groups"] == []
assert result["name"] == "jdoe" # Falls back to username assert result["name"] == "jdoe" # Falls back to username
@pytest.mark.asyncio @pytest.mark.asyncio
+3 -7
View File
@@ -89,10 +89,6 @@ class TestGetSettings:
settings2 = get_settings() settings2 = get_settings()
assert settings1 is settings2 assert settings1 is settings2
def test_model_aliases_property(self): # Removed: test_model_aliases_property. Settings.model_aliases mapped
"""Model aliases property should return dict.""" # gpt-3.5-turbo and gpt-4 onto local models, and was deleted along with the
settings = get_settings() # Ollama integration in c1f16d4. The test outlived the feature it covered.
aliases = settings.model_aliases
assert isinstance(aliases, dict)
assert "gpt-3.5-turbo" in aliases
assert "gpt-4" in aliases
+114 -62
View File
@@ -1,118 +1,173 @@
"""Tests for OIDC authentication module.""" """Tests for OIDC authentication module."""
import pytest import pytest
from unittest.mock import patch, MagicMock, AsyncMock from unittest.mock import patch, MagicMock
from fastapi import HTTPException from fastapi import HTTPException
from src.auth.oidc import OIDCConfig, oidc_config, get_jwks, get_current_user from src.auth.oidc import (
OIDCConfig,
oidc_config,
get_jwks_for_issuer,
get_current_user,
_jwks_cache,
)
@pytest.fixture(autouse=True)
def clear_jwks_cache():
"""
The JWKS cache is module-level state, so a fetch in one test would satisfy
the next one and hide a regression. Clearing on both sides keeps the tests
order-independent.
"""
_jwks_cache.clear()
yield
_jwks_cache.clear()
class TestOIDCConfig: class TestOIDCConfig:
"""Test OIDCConfig class.""" """Test OIDCConfig class."""
def test_init_defaults(self): def test_init_defaults(self):
"""Config should initialize with disabled state.""" """Config should initialize disabled with no issuers or audiences."""
config = OIDCConfig() config = OIDCConfig()
assert config.enabled is False assert config.enabled is False
assert config.issuer == "" assert config.issuers == []
assert config.audience == "" assert config.audiences == []
assert config.jwks_uri == ""
def test_configure_sets_values(self): def test_configure_sets_values(self):
"""configure should set all values.""" """configure should set all values."""
config = OIDCConfig() config = OIDCConfig()
config.configure( config.configure(
enabled=True, enabled=True,
issuer="https://auth.example.com", issuers=["https://auth.example.com"],
audience="core-api" audiences=["core-api"],
) )
assert config.enabled is True assert config.enabled is True
assert config.issuer == "https://auth.example.com" assert config.issuers == ["https://auth.example.com"]
assert config.audience == "core-api" assert config.audiences == ["core-api"]
assert config.jwks_uri == "https://auth.example.com/jwks/"
def test_configure_strips_trailing_slash(self): def test_configure_strips_trailing_slash(self):
"""configure should handle trailing slash in issuer.""" """configure should normalise issuers by dropping the trailing slash."""
config = OIDCConfig() config = OIDCConfig()
config.configure( config.configure(
enabled=True, enabled=True,
issuer="https://auth.example.com/", issuers=["https://auth.example.com/"],
audience="core-api" audiences=["core-api"],
) )
assert config.jwks_uri == "https://auth.example.com/jwks/" assert config.issuers == ["https://auth.example.com"]
def test_configure_accepts_multiple_issuers(self):
"""The point of the multi-issuer change: more than one is allowed."""
config = OIDCConfig()
config.configure(
enabled=True,
issuers=["https://a.example.com/", "https://b.example.com"],
audiences=["core-api", "other"],
)
assert config.issuers == ["https://a.example.com", "https://b.example.com"]
assert config.audiences == ["core-api", "other"]
def test_get_jwks_uri_derives_from_issuer(self):
"""The JWKS URI is derived per issuer rather than configured."""
config = OIDCConfig()
assert config.get_jwks_uri("https://auth.example.com") == "https://auth.example.com/jwks/"
assert config.get_jwks_uri("https://auth.example.com/") == "https://auth.example.com/jwks/"
def test_is_valid_issuer_only_accepts_configured(self):
"""
An unconfigured issuer must be rejected. This is the security-relevant
half of multi-issuer support: accepting any issuer would let a token
from an unrelated identity provider through.
"""
config = OIDCConfig()
config.configure(
enabled=True,
issuers=["https://auth.example.com"],
audiences=["core-api"],
)
assert config.is_valid_issuer("https://auth.example.com") is True
assert config.is_valid_issuer("https://auth.example.com/") is True
assert config.is_valid_issuer("https://evil.example.com") is False
class TestGetJWKS: class TestGetJWKSForIssuer:
"""Test get_jwks function.""" """Test get_jwks_for_issuer function."""
ISSUER = "https://auth.example.com"
def test_returns_empty_when_disabled(self): def test_returns_empty_when_disabled(self):
"""get_jwks should return empty dict when OIDC disabled.""" """Should return an empty dict when OIDC is disabled."""
# Save original state
original_enabled = oidc_config.enabled original_enabled = oidc_config.enabled
try: try:
oidc_config.enabled = False oidc_config.enabled = False
# Clear the cache
get_jwks.cache_clear()
result = get_jwks() assert get_jwks_for_issuer(self.ISSUER) == {}
assert result == {}
finally: finally:
# Restore original state
oidc_config.enabled = original_enabled oidc_config.enabled = original_enabled
get_jwks.cache_clear()
@patch("src.auth.oidc.httpx.get") @patch("src.auth.oidc.httpx.get")
def test_fetches_jwks_when_enabled(self, mock_get): def test_fetches_jwks_when_enabled(self, mock_get):
"""get_jwks should fetch JWKS when enabled.""" """Should fetch from the issuer's derived JWKS URI."""
# Save original state
original_enabled = oidc_config.enabled original_enabled = oidc_config.enabled
original_jwks_uri = oidc_config.jwks_uri
try: try:
oidc_config.enabled = True oidc_config.enabled = True
oidc_config.jwks_uri = "https://auth.example.com/jwks/"
get_jwks.cache_clear()
mock_response = MagicMock() mock_response = MagicMock()
mock_response.json.return_value = {"keys": [{"kid": "test"}]} mock_response.json.return_value = {"keys": [{"kid": "abc"}]}
mock_response.raise_for_status = MagicMock()
mock_get.return_value = mock_response mock_get.return_value = mock_response
result = get_jwks() result = get_jwks_for_issuer(self.ISSUER)
assert "keys" in result assert result == {"keys": [{"kid": "abc"}]}
mock_get.assert_called_once() mock_get.assert_called_once()
assert mock_get.call_args[0][0] == f"{self.ISSUER}/jwks/"
finally: finally:
oidc_config.enabled = original_enabled oidc_config.enabled = original_enabled
oidc_config.jwks_uri = original_jwks_uri
get_jwks.cache_clear()
@patch("src.auth.oidc.httpx.get") @patch("src.auth.oidc.httpx.get")
def test_raises_exception_on_error(self, mock_get): def test_caches_per_issuer(self, mock_get):
"""get_jwks should raise HTTPException on fetch error.""" """
# Save original state A second call for the same issuer must not refetch, and a different
issuer must. Caching by issuer is the behaviour the multi-issuer change
introduced, and a shared cache would have served one issuer's keys for
another — which would be a verification bypass, not just a slow path.
"""
original_enabled = oidc_config.enabled original_enabled = oidc_config.enabled
original_jwks_uri = oidc_config.jwks_uri
try: try:
oidc_config.enabled = True oidc_config.enabled = True
oidc_config.jwks_uri = "https://auth.example.com/jwks/" mock_response = MagicMock()
get_jwks.cache_clear() mock_response.json.return_value = {"keys": []}
mock_get.return_value = mock_response
mock_get.side_effect = Exception("Connection error") get_jwks_for_issuer(self.ISSUER)
get_jwks_for_issuer(self.ISSUER + "/") # same issuer, normalised
assert mock_get.call_count == 1
get_jwks_for_issuer("https://other.example.com")
assert mock_get.call_count == 2
finally:
oidc_config.enabled = original_enabled
@patch("src.auth.oidc.httpx.get")
def test_raises_503_on_fetch_error(self, mock_get):
"""A JWKS fetch failure should surface as 503, not leak the cause."""
original_enabled = oidc_config.enabled
try:
oidc_config.enabled = True
mock_get.side_effect = Exception("Connection failed")
with pytest.raises(HTTPException) as exc_info: with pytest.raises(HTTPException) as exc_info:
get_jwks() get_jwks_for_issuer(self.ISSUER)
assert exc_info.value.status_code == 503 assert exc_info.value.status_code == 503
finally: finally:
oidc_config.enabled = original_enabled oidc_config.enabled = original_enabled
oidc_config.jwks_uri = original_jwks_uri
get_jwks.cache_clear()
class TestGetCurrentUser: class TestGetCurrentUser:
@@ -121,9 +176,7 @@ class TestGetCurrentUser:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_returns_none_when_disabled(self): async def test_returns_none_when_disabled(self):
"""get_current_user should return None when OIDC disabled.""" """get_current_user should return None when OIDC disabled."""
# Save original state
original_enabled = oidc_config.enabled original_enabled = oidc_config.enabled
try: try:
oidc_config.enabled = False oidc_config.enabled = False
@@ -136,9 +189,7 @@ class TestGetCurrentUser:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_raises_401_when_enabled_without_token(self): async def test_raises_401_when_enabled_without_token(self):
"""get_current_user should raise 401 when enabled but no token.""" """get_current_user should raise 401 when enabled but no token."""
# Save original state
original_enabled = oidc_config.enabled original_enabled = oidc_config.enabled
try: try:
oidc_config.enabled = True oidc_config.enabled = True
@@ -157,11 +208,12 @@ class TestOIDCGlobalConfig:
"""oidc_config should be an OIDCConfig instance.""" """oidc_config should be an OIDCConfig instance."""
assert isinstance(oidc_config, OIDCConfig) assert isinstance(oidc_config, OIDCConfig)
def test_global_config_starts_disabled(self): def test_global_config_exposes_the_multi_issuer_surface(self):
"""oidc_config should start disabled by default.""" """
# This tests the initial state before any configure() is called Asserts the shape rather than the values, since the live state depends
# The actual state depends on app configuration on app configuration. These four are what callers depend on.
assert hasattr(oidc_config, 'enabled') """
assert hasattr(oidc_config, 'issuer') assert hasattr(oidc_config, "enabled")
assert hasattr(oidc_config, 'audience') assert hasattr(oidc_config, "issuers")
assert hasattr(oidc_config, 'jwks_uri') assert hasattr(oidc_config, "audiences")
assert callable(oidc_config.get_jwks_uri)
+17 -4
View File
@@ -4,6 +4,7 @@ from fastapi.testclient import TestClient
from unittest.mock import patch, AsyncMock, MagicMock from unittest.mock import patch, AsyncMock, MagicMock
from src.main import app from src.main import app
from src.domains.tools.controller import tools_controller
@pytest.fixture @pytest.fixture
@@ -36,9 +37,21 @@ class TestDNSLookup:
) )
assert response.status_code == 200 assert response.status_code == 200
@patch("src.controllers.tools_controller.DNSService") def test_dns_lookup_returns_result(self, client):
def test_dns_lookup_returns_result(self, mock_dns_class, client): """DNS lookup should return lookup results.
"""DNS lookup should return lookup results."""
Patches the live singleton's `dns_service` attribute, not the
`src.controllers.tools_controller.DNSService` class: that module is
the legacy top-level package (not wired into `src.main`, see
CLAUDE.md "Legacy top-level packages"). `client` exercises
`src.main.app`, which routes through
`src.domains.tools.controller.tools_controller`, a singleton built
at import time — so patching the class there would also miss,
since `tools_controller.dns_service` is already a constructed
instance by the time a test patches the class. Patching the
instance attribute directly is the only patch that actually
intercepts this request path.
"""
mock_response = MagicMock() mock_response = MagicMock()
mock_response.success = True mock_response.success = True
mock_response.domain = "example.com" mock_response.domain = "example.com"
@@ -59,8 +72,8 @@ class TestDNSLookup:
mock_service = MagicMock() mock_service = MagicMock()
mock_service.lookup = AsyncMock(return_value=mock_response) mock_service.lookup = AsyncMock(return_value=mock_response)
mock_dns_class.return_value = mock_service
with patch.object(tools_controller, "dns_service", mock_service):
response = client.post( response = client.post(
"/tools/dns/lookup", "/tools/dns/lookup",
json={"domain": "example.com", "record_type": "A"} json={"domain": "example.com", "record_type": "A"}