Compare commits

...
79 Commits
Author SHA1 Message Date
jpmschweitzerandClaude Opus 5.5 44c5facc65 chore(release): 1.14.0 — archive and trash per mailbox, lookup by message number
Build and Push / release (push) Successful in 2s
Build and Push / build (push) Successful in 13s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 22:58:17 +02:00
jpmschweitzerandClaude Sonnet 5 e25a3ab4f4 docs(changelog): record T-13/T-9 — archive and trash a mailbox entry, lookup by message number
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-19 22:55:00 +02:00
jpmschweitzerandClaude Sonnet 5 7c67a4a46b feat(scribe): archive, trash and look up a mailbox entry by message number (T-13/T-9)
The user wants a way to get rid of a message that is no longer useful. A
message is stored once and shared by every recipient; `history` is how
agents rebuild context (D-66), so deleting the message itself would take
it from every other reader. Deletion is therefore per mailbox entry
(D-70): `archive` (kept, out of the default view) and `trash` (out of the
mailbox, restorable). True erasure of content stays `redact` — not this
round.

Migration 009 (down_revision="008") adds
scribe_mailbox_entries.archived_at and trashed_at — independent nullable
timestamps, mirroring book_records.archived_at/trashed_at (migration
005). No data step: every existing entry is neither.

PATCH /scribe/mailbox/{entry_id} takes `archived` and `trashed` booleans
beside `read` (true sets the timestamp if not already set, false clears
it); the response carries both. GET /scribe/mailbox takes `view`: `inbox`
(default, neither archived nor trashed), `archive` (archived and not
trashed), `trash` (trashed, regardless of archived_at — trashing hides an
entry from the archive view too), `all` (no filter on either column).
`find` now excludes trashed entries (archived ones still match).

T-9: GET /scribe/mailbox?message_id=N is a direct lookup, not a filtered
list — the unique constraint on (address, message_id) guarantees zero or
one, in any view, ignoring unread/tag/before_id entirely. That is the
point: a client that already knows the message number never pages the
unread-first/id-descending listing to find it.

Judgement call: the listing's own cursor (`before_id`, a single `id`
comparison) still does not match its two-key sort (unread-first, then id
descending) — the pre-existing gap the dossier flagged as optional to
fix. Left unchanged: T-9's message_id lookup is the actual answer to "a
client pages to find one," and reworking the general cursor is a larger,
separate concern (which field(s) a keyset cursor needs) that risks the
already-passing test_mailbox_lists_unread_first for no requirement this
round asked for.

Judgement call: archived_at and trashed_at are independent timestamps,
never cross-clearing each other. Trashing an already-archived entry does
not clear archived_at, so restoring it (trashed=false) returns it to the
archive view, not the inbox — the view filter's own trash-takes-priority
rule is what keeps it out of view=archive while trashed, not a write-time
side effect.

Tests: 12 new (6 archive/trash flow + 4 message_id lookup + 2 migration
round-trip). Full suite 554 -> 566 passed.

Mutation checks (Edit, reverted after each):
- view=inbox dropped its trashed_at filter ->
  test_trash_hides_from_inbox_archive_and_find_shows_in_trash failed
  (assert 1 == 0).
- message_id lookup dropped the caller-address filter ->
  test_message_id_answers_nothing_for_a_message_not_in_their_mailbox
  failed (assert 1 == 0).
- trash updated every mailbox entry for the message instead of the
  caller's own row ->
  test_trashing_one_recipients_entry_leaves_the_others_untouched failed
  (assert 0 == 1).
All three reverted; full suite re-verified green (566 passed).

Migration 008 -> 009 -> 008 -> 009 round-tripped against core_api_test,
both manually (alembic CLI) and in
TestMigration009RoundTrip. Production stays at 008 — every alembic
invocation derived POSTGRES_* from TEST_DATABASE_URL.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-19 22:54:35 +02:00
jpmschweitzerandClaude Opus 5.5 0bba3baf32 chore(plan): T-13 — archive and trash a message in your own mailbox
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 22:35:21 +02:00
jpmschweitzerandClaude Opus 5.5 c21c03c289 chore(plan): T-11 done — live in 1.13.0
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 22:16:05 +02:00
jpmschweitzerandClaude Opus 5.5 fca545aa49 chore(plan): T-12 — the test database URL leaks through pytest -v
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 22:08:36 +02:00
jpmschweitzerandClaude Opus 5.5 dbb924da59 chore(release): 1.13.0 — a default address on each consumer grant
Build and Push / release (push) Successful in 1s
Build and Push / build (push) Successful in 13s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 22:08:36 +02:00
jpmschweitzerandClaude Sonnet 5 ff54fcc94d docs(changelog): record T-11 — a default address on each consumer grant
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-19 22:06:18 +02:00
jpmschweitzerandClaude Sonnet 5 80034bbacf feat(tenants): add a default address on each consumer grant (T-11)
claude.ai's connector keeps no MCP session between tool calls, so the
Office cannot remember which address a conversation chose. Each consumer
grant now carries a fallback: the Office uses it when a call names none
and has no session binding (tatlock-office T-5, in a parallel round,
reads it).

Migration 008 (down_revision="007") adds
tenant_consumers.default_address — nullable varchar(63) — and a
composite foreign key (tenant_id, default_address) ->
scribe_addresses(owner_tenant, name), so a default always names a real
address in the same tenant. scribe_addresses already carries both a
primary key and a named unique constraint on (owner_tenant, name)
(migration 006), so the composite FK has something to bind to; Postgres's
default MATCH SIMPLE means a NULL default_address never triggers the
check, which is the state every grant but one is left in. The data step
sets jpmschweitzer/office-claude -> claude-ai only if that address exists
at upgrade time (true in production, false in core_api_test, where no
scribe_addresses are seeded) — an UPDATE ... WHERE EXISTS, not a bare
UPDATE, so it is a no-op rather than a failure when absent.
llm_tester/office-claude is untouched and stays null. Downgrade drops the
FK then the column.

TenantConsumer (models.py) gains the column and a matching
ForeignKeyConstraint. TenantsService.get_me now returns a third element,
the caller's own consumer's default_address (looked up only when
caller.consumer is not None — a user-kind caller carries no consumer, so
there is no grant row to hold a default for it) or None. GET /tenants/me
answers it; GET /tenants and its TenantSummary schema are unchanged.

Tests: TestMigration008RoundTrip/ForeignKey/DataStep added to
tests/test_tenants_migration.py (round-trip 007<->008, the FK existing
and refusing an unknown address while allowing null, the data step's
no-op and positive cases). tests/conftest.py gains a
default_address_grant fixture (registers a throwaway address, points a
grant at it, reverses both on teardown). tests/test_tenants.py gains
TestMeDefaultAddress (an office caller with/without a default, a user
caller always null).

Friction found and fixed in the same commit: the new composite FK made
Postgres refuse tests/test_scribe.py's existing clean_scribe_tables
TRUNCATE ("cannot truncate a table referenced in a foreign key
constraint") — the restriction is on the constraint's existence, not on
whether any row's default_address is actually non-null. tenant_consumers
joins that TRUNCATE; CASCADE was rejected because it would silently drop
every grant rather than only addresses, so the four grants
(jpmschweitzer/llm_tester x office-claude/office-gemini) are reinserted
in the same fixture, right after.

Mutation-checked: forcing default_address to null in the controller
fails exactly test_office_caller_with_a_default_sees_it, leaving the
without-a-default and user-caller tests green. Disabling the FK's
creation (and, symmetrically, its drop in downgrade — to keep the
round-trip self-consistent against the live core_api_test schema) fails
test_fk_exists and test_fk_refuses_a_default_naming_an_address_that_does_not_exist,
leaving test_fk_allows_a_null_default_regardless_of_tenant green. Both
mutations reverted; the full suite re-verified green (543 + 11 = 554
passed).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-19 22:05:46 +02:00
jpmschweitzerandClaude Opus 5.5 008996e030 chore(plan): T-11 — a default address on each consumer grant
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 21:38:18 +02:00
jpmschweitzerandClaude Opus 5.5 d1641f1480 chore(plan): T-10 — Scribe messages number per tenant
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 19:12:23 +02:00
jpmschweitzerandClaude Opus 5.5 1c4cfc4f1b chore(plan): T-9 — the mailbox answers by message number
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 19:08:02 +02:00
jpmschweitzerandClaude Opus 5.5 87481a016a chore(plan): T-1 and T-8 done — live in 1.12.0
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 17:48:04 +02:00
jpmschweitzerandClaude Opus 5.5 8f72aa0e6e chore(release): 1.12.0 — the Book, the Scribe, caller auth, the tenant registry
Build and Push / release (push) Successful in 1s
Build and Push / build (push) Successful in 1m2s
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 17:37:48 +02:00
jpmschweitzerandClaude Sonnet 5 d32caabe92 docs(changelog): record T-8 — the tenant registry
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-19 17:31:43 +02:00
jpmschweitzerandClaude Sonnet 5 99ee994023 feat(tenants): add the tenant registry, the one enforcement point Book and Scribe now share (T-8)
A tenant has been an unchecked string since T-4/T-7: the Office's
X-Office-Tenant header or a user JWT's tatlock_user claim, with Book and
Scribe each running their own _require_tenant that checked only presence.
A typo'd claim opened a fresh, empty namespace rather than being refused.

src/domains/tenants/ (controller/models/schemas/service) adds tenants and
tenant_consumers (migration 007, down_revision="006"): tenants.id is the
slug a tatlock tenant carries, kind (household|person|test) and status
(active|suspended) CHECK-constrained; tenant_consumers is the grant pairing
a consumer to a tenant. Migration 007 seeds the two tenants in use today —
jpmschweitzer (household, "Jeroen") and llm_tester (test, "Playground") —
each granted to office-claude, the only Office consumer configured in
Authentik. Every existing owner_tenant column (book_records,
book_tenant_counters, scribe_addresses, scribe_messages,
scribe_mailbox_entries — five, grepped and confirmed) now carries a
foreign key to tenants.id, ON DELETE RESTRICT, at both the migration and
the ORM model.

TenantsService.require_tenant is the one enforcement point: no tenant is
400 tenant_required (unchanged); an unknown tenant is 403 tenant_unknown;
a suspended one is 403 tenant_suspended; an Office caller whose consumer
holds no grant in that tenant is 403 consumer_not_granted. One query — a
LEFT JOIN against the grant — so presence, status and the grant are all
read in a single round trip. Book's and Scribe's own _require_tenant now
delegate to it, translating TenantError to their own exception type so
every call site and each controller's single except clause keep working
unchanged.

Read-only this round: GET /tenants/me answers the caller's own tenant and
its granted consumers (the cheap "am I allowed here" probe); GET /tenants
answers every tenant for a tatlock/scheduler caller, or a list of one for
a tenant-bearing caller — another tenant's existence is never revealed.
No write routes; provisioning a tenant is workspace T-195.

Tests: tests/test_tenants.py and tests/test_tenants_migration.py (new);
a TestTenantRegistryEnforcement class added to both tests/test_book.py and
tests/test_scribe.py proving all four cases (unknown, suspended,
ungranted-consumer, and the control) on a live Book and Scribe route, not
only on /tenants itself. tests/conftest.py gains a tenant_registry schema
fixture and a suspended_tenant fixture (inserted and deleted per test; no
FK references it); book_schema and scribe_schema each independently grant
a second, test-environment-only consumer (office-gemini) in both seeded
tenants, since test_book.py and test_scribe.py have used it as a stand-in
second consumer since before this grant check existed — the migration
itself still seeds only office-claude, matching the dossier.

Mutation-checked: dropping the grant check fails the three
consumer_not_granted tests (200 instead of 403); dropping the status
check fails the three tenant_suspended tests; making Book bypass the
helper fails its unknown-tenant test (200 instead of 403). All three
restored and the full suite re-verified green (508 + 35 = 543 passed).

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-19 17:30:55 +02:00
jpmschweitzerandClaude Sonnet 5 e02173aefb feat(book): tenant boundary and per-tenant numbering on the Book (T-7)
005 has never run in production (still at 004), so it is amended in place
rather than patched later by a 007: book_records gains owner_tenant (the
tatlock tenant, Caller.tenant from T-4) and number, a short outward id
allocated per tenant via a new book_tenant_counters row locked FOR UPDATE
in the same transaction as the insert — INSERT ... ON CONFLICT DO NOTHING
bootstraps the counter row so a tenant's very first allocation doesn't
race its own creation. Unique on (owner_tenant, number); composite index
on (owner_tenant, kind, status). 006 applies on top unchanged.

Every Book route now requires a tenant (BookService._require_tenant,
mirroring the Scribe's own pattern) and filters on it: a record in
another tenant answers exactly like one that does not exist, in both
directions. link and add's parent convenience both route through
_get_or_404, so a link can never reach another tenant — which is also
what keeps show's links/backlinks free of their own tenant filter, since
every link touching a tenant's record is created tenant-safe already.
Responses include number; list accepts number= to find one.

Verified: core_api_test downgraded to 004 (with the original migration,
to match its already-applied pre-amendment schema), amended, upgraded
back to head — clean round trip. 508 tests pass (489 baseline + 19 new:
3 migration-shape, 16 tenant/numbering). Mutation checks confirmed and
reverted: dropping list's tenant filter, allocating numbers from a global
counter, and dropping link's tenant check on its target each fail the
test built to catch it. Raw REST against a dev server on :8777 against
core_api_test confirms the same behaviour over the wire: two tenants'
first records both land on number 1, and a cross-tenant read answers 404
not_found.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-19 16:28:08 +02:00
jpmschweitzerandClaude Opus 5.5 e52c727d99 chore(plan): T-7 — the Book schema and tenant boundary round
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 16:07:03 +02:00
jpmschweitzerandClaude Opus 5.5 c5baaeea99 chore(plan): T-5 done; T-6 filed
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 15:24:11 +02:00
jpmschweitzerandClaude Sonnet 5 93b00b6369 docs(changelog): record T-5 — the Scribe's messaging
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-19 15:19:17 +02:00
jpmschweitzerandClaude Sonnet 5 ccdbc44d67 feat(scribe): add the Scribe domain — addresses, send, mailbox, history, find, mark, tag (T-5)
The Office (workspace D-66, D-68, D-73) needs a message bus: an address is
a topic named for an identity, owned by a tenant and a consumer. D-68 puts
it here, in core-api's own database, behind its own auth, following
src/domains/book/.

Migration 006 lands the whole schema by hand (down_revision="005"):
scribe_addresses (owner_tenant + name primary key — no surrogate id, since
no route ever references an address except by name), scribe_messages
(retained; a generated tsvector search column + GIN index on subject/body,
and a GIN index on the topics array for history's lookup), and
scribe_mailbox_entries (one row per recipient address per message, unique
on address+message_id). No data: seeding `jeroen` and `tower-control` is a
deploy step through `register`, not this migration.

Routes, every one behind require_caller("office", "user"), tenant from the
caller (T-4):

- register/list addresses — register is idempotent for the same consumer,
  409 address_taken for another consumer in the tenant; list marks which
  addresses the caller's consumer owns.
- send — `from` is always the caller's own identity, stamped server-side,
  never read from the body; 403 identity_not_owned if that identity is not
  a registered address owned by the caller's consumer; 404 unknown_address
  (naming the missing ones) if any recipient does not exist in the tenant;
  writes one mailbox entry per recipient.
- mailbox / find — the caller's own identity's entries (mailbox: unread
  first; find: full text via websearch_to_tsquery), both requiring the
  identity be owned by the caller's consumer.
- history — the retained messages sent to an address, oldest first;
  reading never consumes or marks, and reads scribe_messages directly
  rather than joining through mailbox entries.
- mark / tag / untag — share one PATCH route; an entry outside the
  caller's tenant, or addressed to an identity the caller isn't reading
  as, answers 404 not_found — indistinguishable from a nonexistent entry.

Every query filters on the caller's tenant (D-73): an address or message
in another tenant answers exactly like one that does not exist, so no id
is ever confirmed across that boundary (test_scribe.py::TestTenantIsolation,
including two tests proving the tenant filter is load-bearing even when
the same address *name* legitimately exists, differently owned, in two
tenants — not just when the caller's own identity is unregistered there).

Messaging only this round: groups, subscriptions, the notification matrix
and delivery are explicitly not built (workspace T-176) — the schema
already carries what `topics`/`tags` need for them.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-19 15:18:52 +02:00
jpmschweitzerandClaude Opus 5.5 739b33b198 chore(plan): T-5 — the Scribe messaging round
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 14:57:49 +02:00
jpmschweitzerandClaude Opus 5.5 e395220c1f chore(plan): T-4 done; T-3 gains the dev-server target
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 13:46:48 +02:00
jpmschweitzerandClaude Sonnet 5 ce2ec61684 docs(changelog): record T-4 — consumer and tenant on the caller
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-19 13:43:57 +02:00
jpmschweitzerandClaude Sonnet 5 0453d08206 feat(auth): T-4 — the caller carries consumer and tenant from the Office
The Office (workspace D-73) now signs every visitor in through Authentik
with one OAuth client per consumer, and every token carries a signed
tatlock_user claim. Caller gains consumer (X-Office-Consumer, the
Authentik application slug) and tenant (X-Office-Tenant, the
tatlock_user claim). For kind "office" both are required on every call,
reads included — missing is 400 consumer_required/tenant_required,
malformed is 400 consumer_invalid/tenant_invalid. Every other kind
ignores both headers outright: tatlock and scheduler carry no tenant,
and a "user" caller's tenant comes only from their own JWT's
tatlock_user claim, never from a header — a header must not be able to
put a non-Office caller into a tenant.

X-Office-Client (informational) is retired in favour of
X-Office-Consumer. book_events.client keeps its name (no migration this
round) and now records the consumer instead.

Book routes do not start filtering by tenant this round — that is
core-api T-2.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-19 13:43:36 +02:00
jpmschweitzerandClaude Opus 5.5 1da057bea6 chore(plan): round office-2 ticket
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 13:29:43 +02:00
jpmschweitzerandClaude Opus 5.5 3171389583 chore(plan): T-2 — the Book carries an owner, the tatlock tenant
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 12:37:01 +02:00
jpmschweitzerandClaude Opus 5.5 e31990c217 chore(plan): T-1 reviewed; T-2 and T-3 filed
T-2 finishes the Book before its first deploy: the remaining verbs,
short outward ids (amending migration 005), and work/personal as tags.
T-3 names the alembic target instead of inheriting production.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 11:20:53 +02:00
jpmschweitzerandClaude Sonnet 5 63feb88912 test(book): prove a task is found by full-text, not only a note
The dossier's verification wants a note *and* a task found by find();
the previous version only ever searched for the note's words. Search
for the task's own body text and assert it is the record returned.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-19 11:17:10 +02:00
jpmschweitzerandClaude Sonnet 5 a334abd523 docs(changelog): record T-1 — caller auth, locked housekeeping reads, the Book
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-19 11:14:56 +02:00
jpmschweitzerandClaude Sonnet 5 1166f18b53 feat(book): add the Book domain — notes, tasks, decisions, questions, rejected (T-1)
The Office (workspace D-64..D-72) needs a store for Notes, Tasks,
Decisions, Questions and Rejected options. D-68 puts it here, in
core-api's own database, behind its own auth.

Migration 005 lands the whole schema now, by hand (down_revision="004";
the chain has a hash-named revision between 003 and 004, confirmed
before trusting the numbering): book_records (kind, status validated
per kind in the service, scope work/personal, a generated tsvector
search column + GIN index), book_tags, book_record_links, book_attachments
(table only, no file handling yet), and book_events — append-only via a
database trigger that raises on UPDATE and DELETE for every role,
including the owning application.

Routes this round (D-70's core verbs), every one behind
require_caller("office", "user"), writes with identity_required=True,
every change writing a book_events row: add, show, list (the nine
views), find (full-text), change, close (task done/dropped; question
resolved via a `resolves` link into `settled`; decision superseded via
a `supersedes` link into `superseded`), link/unlink, tag/untag, history.

Append, remind, reschedule, undo, purge, redact, pin, check, snooze,
archive/trash/restore, pause/resume, due, next, batch calls and
idempotency keys are explicitly not this round — the schema already
carries what they need.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-19 11:14:46 +02:00
jpmschweitzerandClaude Sonnet 5 3f77cc2926 fix(housekeeping): require a caller on reads; writes unchanged (T-1)
api.schweitz.net is internet-facing with no forward-auth, and every
GET under /housekeeping/* (health, devices, areas, scenes, scripts,
automations, history) was unauthenticated. It answers 500 today only
because Home Assistant's token is dead; once that token is restored it
would serve the house's device list to anyone.

Reads now go through require_caller("office", "tatlock", "user").
tatlock is in the list because tatlock's Housekeeper
(tatlock/src/agents/housekeeper/client.py) is the only caller of these
routes today and already sends Bearer CORE_API_KEY when it is set.

Writes are untouched — control_device, activate_scene, run_script and
toggle_automation still depend on get_admin_user. Whether tatlock may
write is a separate decision, not this one.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-19 11:14:35 +02:00
jpmschweitzerandClaude Sonnet 5 04ec54b635 feat(auth): add require_caller — static service keys before any JWT (T-1)
core-api had no service-key auth: AuthService.validate_api_key was never
called, api_keys stayed empty, and CORE_API_KEY was read by nothing. This
is the first thing that reads a service key.

require_caller(*allowed, identity_required=False) resolves, in order:
office_service_key -> tatlock_service_key -> scheduler_service_key ->
the existing get_current_user() JWT path. Each static key is compared
with hmac.compare_digest, gated on the configured key being non-empty so
an unset key can never match. A disallowed kind is 403 forbidden.

The office caller's identity comes from X-Office-Identity: required on
writes (400 identity_required if missing), validated against
^[a-z0-9][a-z0-9-]{0,62}$ and against the reserved/group- set (400
identity_invalid). Other kinds carry a fixed identity and ignore the
header.

New settings (src/shared/config.py, default ""): office_service_key,
tatlock_service_key, scheduler_service_key; and book_timezone
("Europe/Amsterdam") for the Book's due/remind handling.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-10-19 11:14:15 +02:00
jpmschweitzerandClaude Opus 5.5 0df3fdddcb chore(plan): T-1 — caller auth, locked housekeeping routes, the Book
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 10:43:41 +02:00
jpmschweitzer 3536c40608 fix(permissions): narrow rm -rf deny globs to their exact forms
The trailing wildcard on the three rm -rf deny entries spanned path
separators, so Bash(rm -rf /*) matched every absolute path on the
machine rather than the filesystem root, and the ~ and $HOME entries
had the same shape. Narrowed to the exact literal forms.

These rules match literal command text, so they still stop a typo on
rm -rf /, rm -rf ~ or rm -rf $HOME exactly, but they no longer stop a
recursive delete aimed at any other path. That reduced cover is
deliberate, not an oversight.
2026-08-25 20:31:15 +02:00
jpmschweitzerandClaude 0c7e1cb04e fix(tests): mock the live DNS service instance, not a dead legacy class (T-55)
test_dns_lookup_returns_result patched src.controllers.tools_controller.
DNSService, which is never imported by the request path under test.
client wraps src.main.app, which routes /tools/dns/lookup through
src.domains.tools.controller.tools_controller — a singleton constructed
at import time from src.domains.tools.dns.service.DNSService. The patched
class was dead; the mock was never consulted, so the test issued a real
DNS query for example.com and asserted on its outcome. With no network
the query times out and the assertion fails (D-26).

The three sibling tests in the same class patch the same dead class and
also run unmocked, but happen not to notice: DNS failures are caught
inside DNSService.lookup() and returned as a normal 200 response with
success=False, and their assertions only check status_code / DNSQueryError
branches that don't depend on resolution actually succeeding. Only this
test's `data["success"] is True` assertion is sensitive to the real
network outcome, which is why it's the only one D-26's namespace run
catches. Not touched here — out of T-55's scope, flagging for the record.

Fix patches tools_controller.dns_service, the actual instance attribute
the live route calls, via patch.object on the singleton rather than
patch() on the constructor class (the instance already exists by the
time a class-level patch would apply).

Verified:
- unshare -rn (lo up): 381 passed, exit 0 (was 1 failed, 380 passed, exit 2)
- with network: 381 passed, exit 0 (unchanged from before the fix)
- mutation check: retargeted the patch.object to a nonexistent attribute
  name, confirmed count==1 before editing; namespace run then reproduced
  the original failure (1 failed, 380 passed); reverted and reconfirmed
  381 passed, exit 0.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-18 15:33:58 +02:00
jpmschweitzerandClaude 6695215aeb build(make): prove setup worked instead of assuming pip's exit code (T-47)
pip install exits 0 whether the result is usable or not — that is the D-24
shape exactly, a step whose job is to not fail. On 2026-08-09 the venv here
existed and pip had already succeeded, but sqlalchemy was declared in
requirements.txt and not installed. That surfaced as 11 pytest collection
errors that read as broken imports rather than as an environment problem.

setup now ends with `pytest --collect-only tests/`, which exercises every
import the suite touches without running anything. pip check was considered
as a cheaper alternative and rejected: it only checks the installed set's
internal consistency against itself, so it would not have caught this case —
sqlalchemy was still present as another package's transitive dependency even
after being dropped from requirements.txt. collect-only checks declared vs.
actually usable directly, which is the axis that broke.

Verified: two clean runs (33.5s cold, 5.7s idempotent re-run, second changes
nothing). Reproduced the original failure by uninstalling sqlalchemy from an
otherwise-correct venv — collect-only alone then exits 2 with 11 collection
errors; make setup against that same state reinstalls it and exits 0 with
381/381 collected.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-17 12:05:28 +02:00
jpmschweitzerandClaude d2e0a49e7c build(ci): move the pre-push gate into the Makefile
The hook carried ~50 lines of gitleaks logic and a comment explaining it was
self-contained because "this repo has no Makefile". It has one now, so the
reason is gone and the arrangement is backwards: a hook is a trigger, and
logic belongs where it can be read, run by hand, and changed under review.

.githooks/pre-push is now a byte-identical shim onto `make pre-push` in every
repo in the workspace. The scan itself moves to ci/secrets.sh unchanged, and
`make secrets` runs it on its own.

The call surface is identical everywhere; what it runs is not, and should not
be — each repo gates what it actually has. That is the point of standardising
the name rather than the contents: nobody has to read a repo to find out how
to check it.

secrets runs first, deliberately. It is the only failure here that cannot be
undone by fixing it afterwards — a failed lint costs another commit, a pushed
credential is cached and indexed whether or not it is later deleted.

Some of these gates fail today, on lint debt that predates them, and they are
left wired anyway. The board was measured once and written down in T-56
instead of being worked around here. Narrowing each gate to whatever already
passes would produce a gate that reports success for doing nothing, which is
the failure this workspace keeps rediscovering.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 18:57:22 +02:00
jpmschweitzerandClaude 755aa61107 ci(make): reserve exit 69 for "could not run" (D-26)
Environment guards now exit 69 rather than 1, so a caller can tell a suite
that could not start from one that ran and failed. The first toj test sweep
reported "3 repositories failed" and none of the three had executed a test —
two could not find go, one had no venv. That points the reader at the tests
when the fault is in the environment.

Only the environment guards change. A gitleaks finding, a failed test run and
a vulncheck hit still exit 1, because those did run and did fail.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 15:56:19 +02:00
jpmschweitzerandClaude 01349a83f2 test: repair the suite against the current API
The suite could not even collect: the venv was missing declared dependencies,
and five tests asserted an API that had moved on. 11 collection errors to 381
passing.

test_oidc.py was written for the single-issuer API and 6243f29 replaced it.
issuer and audience became lists, jwks_uri stopped being an attribute in
favour of get_jwks_uri(issuer), get_jwks became get_jwks_for_issuer, and the
lru_cache became a per-issuer dict so cache_clear no longer exists. Rewritten
against the current surface, with coverage added for the two behaviours the
multi-issuer change introduced and never tested: is_valid_issuer rejecting an
unconfigured issuer, and the cache keying per issuer. Both are
security-relevant — a shared cache would serve one issuer keys for another.

Three /auth/me tests asserted a path that does not exist. The route is
declared as /me inside AuthController.create_router() and mounts at
/auth/users/me; the generated spec is authoritative and the local app and the
deployed service agree on it. Those tests had never passed.

test_handles_empty_groups expected groups == [""] for an empty header. oidc.py
has returned [] since the initial commit, and [] is correct — [""] would also
be unsafe, since any check doing "" in groups would match.

test_model_aliases_property covered Settings.model_aliases, deleted with the
Ollama integration in c1f16d4. Removed rather than repaired.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 15:21:43 +02:00
jpmschweitzerandClaude 7815e1c231 build: add the Makefile command surface (D-27)
Every repo gets one at the root: help, plus test and lint where those exist.
The point is that a target name means the same thing in every repo, so an
agent or a person can act without reading the repo first.

Paths resolve here rather than in callers (D-10). python3 on this host is 3.8
and cannot parse these sources, and a bare pytest or ruff resolves only in a
login shell — so both are named explicitly through the venv, and a missing
venv fails with the command to fix it rather than a bare no-such-file.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 15:10:31 +02:00
jpmschweitzerandClaude e9fc09af26 chore(claude): pin PQL_VAULT per project so cwd stops choosing the vault
pql is now a bare word on PATH, which removed the long incantation that had
been forcing --vault into every call by habit. Convenience lowered the cost
of the wrong thing without lowering the cost of the right one: a three-word
pql ticket new targets whichever vault the cwd happens to sit in, and there
are nine of them with colliding id sequences.

PQL_VAULT in each project settings file makes the vault a property of the
session rather than of the working directory — the same lesson Rule 3 records
for git -C, applied to pql. Verified the env var overrides cwd discovery,
that an explicit --vault still beats the env var, and that the harness
hot-reloads it without a restart.

This does not make provenance visible: no output says which vault answered,
so a forgotten --vault still returns a well-formed answer about the wrong
dataset. That remains T-37.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 13:48:58 +02:00
jpmschweitzerandClaude 56b991c965 chore(claude): deny toj in the sub-repos
toj is now on the global PATH as /usr/local/bin/toj, so its scope boundary
had to stop being "the absolute path is inconvenient to type" and start
being a rule. Its repo and settings verbs operate on the workspace root; run
from inside this repo they answer about the wrong tree.

Both spellings are denied, bare and absolute, because a deny with one
spelling left open is decorative.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 13:42:03 +02:00
jpmschweitzerandClaude 387a3dbb02 ci: gate pushes on a gitleaks scan of the outgoing commits
No repo here scanned for committed credentials. The hook is self-contained
rather than delegating to a Makefile, because this repo has none and a hook
reaching into a sibling repo breaks the moment this one is cloned elsewhere.

Scans the outgoing range rather than full history: history carries settled
findings — test fixtures, vendored third-party code — and a gate that fails
on something unfixable gets bypassed within a week.

Setting core.hooksPath means pql init must replant its replication shims into
.githooks, which is why they are gitignored here alongside the tracked
pre-push. Same layout pql itself uses.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 12:48:53 +02:00
jpmschweitzerandClaude 7f90369621 docs: qualify workspace decision ids cited from this repo
Decision ids are per-vault sequences, so they collide by construction
once there is more than one vault -- and every repo now has one. A bare
D-15 here will mean this repo's D-15 the moment this repo records one.
Cross-vault references are therefore qualified: workspace D-15.

Not hypothetical: pql holds D-1 through D-31 while the workspace holds
D-1 through D-21, so every workspace id currently collides with an
unrelated pql one. A bare id is not wrong the day it is written -- it
decays into wrong as the other vault grows, and nothing flags it.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 04:17:07 +02:00
jpmschweitzerandClaude 659f08e576 chore: adopt the workspace agent-config baseline
Commits a .claude/settings.json rather than leaving permissions to
per-developer local state, and initialises a pql vault for this repo's
tickets and internal decisions.

Every git deny rule appears in both the `git <verb>` and `git * <verb>`
forms. Only the second catches `git -C <path>`, and without it the whole
deny list is decorative -- it looks like a policy and stops nothing.

The allow list carries pql's absolute path alongside the bare name.
pql is installed to ~/.local/bin, which is on the login PATH but not the
one a non-interactive shell gets, so the bare-name rules match nothing on
their own and every call would prompt anyway.

.gitignore now covers .claude/settings.local.json, which is machine-local
and must never be shared. `pql init` contributed the .pql/* rules with an
exception for the changelog, which is the replication log of record and
has to be committed for tickets to travel with a clone.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 03:13:08 +02:00
jpmschweitzerandClaude 29bfa3259d docs: replace AGENTS.md with a repo-specific CLAUDE.md
One agent doc per repo, and it is CLAUDE.md. Two docs describing one repo
drift, and the one nobody read is always the one holding the rule that
mattered. Written fresh rather than reformatted, so the structure follows
what someone working here actually needs.

Two rules from the old file are gone deliberately. The mandate to branch
for every change was retired in favour of one linear-history policy, and
the release snippet used `git add -A`, which sweeps in whatever else is
dirty.

The architecture section is the part worth reading. An earlier draft
called src/auth, src/controllers, src/clients, src/dns and src/models
dead code, derived from grepping main.py's imports. That was wrong:
main.py:55 calls initialize_oidc(), which imports and configures
src.auth.oidc from inside the function body, so src/auth is configured
with live Authentik issuers on every boot. It also missed four genuinely
unreferenced packages. The section now states the method used -- import
the app in the container and read sys.modules -- and its blind spot, that
a cold snapshot cannot see a module imported on a request path.

Also records that the README's "runs as non-root user (uid 1000)" is
false: the Dockerfile has no USER directive. Flagged rather than fixed,
since changing the runtime user is not a docs change.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 03:12:02 +02:00
jpmschweitzerandClaude Fable 5 b3383f19b3 chore(ci): push images via git.schweitz.net registry
The .internal registry domain is being retired; git.schweitz.net now
serves the registry without SSO on /v2/.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 11:09:58 +02:00
Jeroen SchweitzerandClaude Opus 4.5 fcf5c8ccee feat: add /tools/news endpoint for news ticker integration
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m15s
- Add NewsHeadline and NewsResponse schemas
- Add NewsService for parsing news from Qdrant volatile collection
- Add GET /tools/news endpoint to tools controller
- News fetched from 'news' namespace in volatile_{user} collection

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 21:49:05 +01:00
Jeroen SchweitzerandClaude Opus 4.5 768cea2c89 fix: pass forecast raw_data to service for parsing
Build and Push / build (push) Successful in 1m14s
Build and Push / release (push) Successful in 2s
Qdrant client was extracting 'days' (int) instead of 'daily' (list)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 18:45:24 +01:00
Jeroen SchweitzerandClaude Opus 4.5 26ecc3e5fd fix: convert wind direction degrees to cardinal string
Build and Push / release (push) Successful in 2s
Build and Push / build (push) Successful in 1m14s
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 18:36:38 +01:00
Jeroen SchweitzerandClaude Opus 4.5 5ff4ba0a43 fix: environment data parsing for scheduler format
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m15s
- Forecast: Check 'daily' key first (scheduler stores count in 'days')
- Sun times: Use ISO fields, calculate daylight from various sources
- Air quality: Support aqi_us/aqi_european and nitrogen_dioxide fields

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 18:30:30 +01:00
Jeroen SchweitzerandClaude Opus 4.5 bb438e22d6 fix: strip email domain from user identifier for environment endpoint
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m15s
If preferred_username is an email (user@domain.com), extract just the
username part to match Qdrant collection naming (volatile_user).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 15:46:16 +01:00
Jeroen SchweitzerandClaude Opus 4.5 3bb3b01dbd fix: OIDC audience validation - use string not list
Build and Push / build (push) Successful in 1m15s
Build and Push / release (push) Successful in 3s
python-jose jwt.decode() requires audience as string or None, not list.
Now extract and validate audience from unverified claims first,
then use token's actual audience for JWT decode.

Fixes "audience must be a string or None" error.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 15:35:05 +01:00
Jeroen SchweitzerandClaude Opus 4.5 ce761a9d2c debug: add logging for OIDC token validation
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 13:58:39 +01:00
Jeroen SchweitzerandClaude Opus 4.5 6243f29aae feat: multi-issuer OIDC support with config consolidation
Build and Push / release (push) Successful in 2s
Build and Push / build (push) Successful in 1m15s
- Support multiple OAuth providers (core-api, tatlock-ui, tatlock)
- Changed oidc_issuer (string) to oidc_issuers (list)
- Per-issuer JWKS caching
- Validates token issuer against allowed list
- Consolidated config files (removed deprecated src/config.py, src/security.py)
- Updated imports to use src/shared/config and src/shared/security

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 13:09:29 +01:00
Jeroen SchweitzerandClaude Opus 4.5 c4d32952db fix: OIDC multi-audience support and Swagger UI fix
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m13s
- Accept tokens from multiple clients (core-api, tatlock-ui, tatlock)
- Fixed main.py to use oidc_audiences[0] for Swagger UI OAuth

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 12:17:16 +01:00
Jeroen Schweitzer 4c45f139d9 correct documentation 2026-01-08 12:11:01 +01:00
Jeroen Schweitzer 67b33314fe fix: update main.py to use oidc_audiences list
Build and Push / release (push) Failing after 3s
Build and Push / build (push) Has been skipped
2026-01-07 19:34:05 +01:00
Jeroen SchweitzerandClaude Opus 4.5 6ce34cc016 fix: accept multiple OIDC audiences for cross-client auth
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m15s
- Changed oidc_audience (string) to oidc_audiences (list)
- Now accepts tokens with audience: core-api, tatlock-ui, tatlock
- Fixes environment endpoint returning "default" user when using
  tatlock-ui token (audience mismatch was causing JWT claims error)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 19:01:39 +01:00
Jeroen SchweitzerandClaude Opus 4.5 c1f16d44e5 refactor: remove Ollama integration and unused AI configuration
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m19s
- Remove src/models/ollama_client.py, embeddings.py, embeddings_ollama.py
- Remove model aliases and AI config from settings (both config.py files)
- Update health endpoints to only check database connectivity
- Update tests to reflect database-only health checks
- Update README, .env.example, and OIDC docstrings

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 17:56:48 +01:00
Jeroen SchweitzerandClaude Opus 4.5 4df5cfc106 fix: initialize OIDC config for both auth modules
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m30s
The domains.auth.oidc module had its own oidc_config instance that
wasn't being configured, causing environment endpoint to always use
hardcoded "local" user instead of authenticated user.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 16:00:14 +01:00
Jeroen SchweitzerandClaude Opus 4.5 0a16688cc8 chore: release v1.10.2
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 2m7s
Enhanced environment endpoint logging for debugging user resolution.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 15:32:53 +01:00
Jeroen SchweitzerandClaude Opus 4.5 a7535fe8ea fix: correct OIDC import path in tools controller
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m14s
The environment endpoint was importing from non-existent path
`src.oidc.dependencies` instead of `src.auth.oidc`, causing
authentication to fail and queries to go to wrong Qdrant collection.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 23:36:24 +01:00
Jeroen SchweitzerandClaude Opus 4.5 6045c6ac6a feat: add environment endpoint for weather, forecast, and sun data
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 2m14s
Add GET /tools/environment endpoint that fetches weather, forecast, sun times,
and air quality data from user's volatile Qdrant collection.

- Add qdrant-client dependency
- Create QdrantReadClient wrapper for read-only queries
- Add environment schemas and service in tools domain
- Parse weather, forecast, sun times, and air quality from Qdrant payloads
- Support user-specific collections via preferred_username from OIDC
- Add comprehensive service tests (13 tests)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 23:04:59 +01:00
Jeroen Schweitzer b8fca7060f npm config 2026-01-04 21:51:21 +01:00
Jeroen SchweitzerandClaude Opus 4.5 e3a49c800a fix: SQLAlchemy async lazy loading for new users
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m11s
Initialize user.roles=[] and user.preferences on new user creation
to avoid MissingGreenlet error in async context.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 18:05:34 +01:00
Jeroen SchweitzerandClaude Opus 4.5 7ab9f73a1d chore: release v1.9.3
Build and Push / release (push) Successful in 2s
Build and Push / build (push) Successful in 1m10s
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 17:49:26 +01:00
Jeroen SchweitzerandClaude Opus 4.5 dd5b794de4 fix: handle non-UUID sub claim in auth sync
Authentik JWT sub claim may not be a valid UUID.
Now derives a deterministic UUID from the sub string if parsing fails.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 17:45:05 +01:00
Jeroen SchweitzerandClaude Opus 4.5 dd0997679f fix: /auth/users/me now supports NPM forward auth headers
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m11s
Added get_current_user_or_forward_auth() combined dependency that:
- First checks for X-authentik-* headers from NPM forward auth (web)
- Falls back to JWT Bearer token validation (mobile/native)

This fixes web authentication where browsers don't send Bearer tokens
but rely on NPM's forward auth proxy to pass user info via headers.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 00:24:41 +01:00
Jeroen SchweitzerandClaude Opus 4.5 e2226cd923 fix(build): add production API URLs to Docker build
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m8s
Dockerfile now passes --dart-define flags for CORE_API_URL and
TATLOCK_API_URL pointing to schweitz.net domains. This enables
requiresAuth=true, fixing auth being completely skipped in production.

Also added service port reference table to AGENTS.md.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 23:18:34 +01:00
Jeroen SchweitzerandClaude Opus 4.5 7bf3c76a1b fix(cors): use explicit origins instead of wildcard
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m11s
When allow_credentials=True, browsers reject wildcard (*) origins.
Added specific allowed origins for Tatlock domains.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 23:09:47 +01:00
Jeroen SchweitzerandClaude Opus 4.5 4ae8cfbc0b chore: release v1.9.0
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m10s
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 22:24:36 +01:00
Jeroen SchweitzerandClaude Opus 4.5 69045a78a7 test(auth): add tests for GET /auth/me endpoint
Add comprehensive tests for NPM forward auth endpoint:
- Forward auth header parsing
- User lookup methods (get_user_by_email, get_user_by_authentik_id)
- Admin gate authorization logic
- OpenAPI spec validation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 22:14:13 +01:00
Jeroen SchweitzerandClaude Opus 4.5 884996fd83 feat(auth): implement GET /auth/me for NPM forward auth
Add endpoint to get current user profile from NPM forward auth headers.
Enables web authentication flow where NPM handles Authentik login.

- Read X-authentik-* headers set by NPM forward auth
- Auto-create user if not in database (first login via web)
- Sync roles from current Authentik groups
- Add get_user_by_email and get_user_by_authentik_id helpers

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 21:54:44 +01:00
Jeroen SchweitzerandClaude Opus 4.5 8bd13eab09 ci: trigger build on version tag push with auto-release
Changed workflow to:
- Trigger on push of v* tags instead of release publish
- Auto-create Gitea release via API
- Then build and push Docker image

This simplifies deployment: just push a version tag.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 20:40:44 +01:00
Jeroen SchweitzerandClaude Opus 4.5 397a47c8fc feat(auth): implement Phase 4 user profile and API key endpoints
Build and Push / build (release) Successful in 1m10s
Add user profile, preferences, and API key management endpoints:
- GET /auth/users/me - full user profile with roles and preferences
- GET/PATCH /auth/users/me/preferences - user preferences management
- GET/POST/DELETE /auth/users/me/api-keys - API key lifecycle

API keys use tak_ prefix, SHA-256 hashing, and are shown only once on creation.
Preferences support partial updates with JSON merge behavior.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 20:24:46 +01:00
Jeroen SchweitzerandClaude Opus 4.5 7752cd9d23 feat(auth): implement group-role mapping and permission system
Architecture changes:
- Permission format: domain.category:action (e.g., control-room.general:admin)
- Decoupled groups from roles via group_roles mapping table
- Groups are organizational (synced from Authentik)
- Roles are permissions (admin-managed via API)

New features:
- require_permission() and require_any_permission() dependency factories
- Action hierarchy: admin > editor > user > viewer
- Global admin override (admin.general:admin grants all)
- Group-role management endpoints (assign/remove roles)
- GET /auth/roles endpoint to list all roles

Database changes:
- Added category column to roles table (default: general)
- Removed authentik_group column (decoupled)
- Added group_roles association table
- Added user_groups association table
- Migration updates role names to domain.general:action format

Tests:
- 67 new tests for auth service and controller
- Covers token validation, user sync, role sync
- Covers group-role assignment/removal
- Covers schema conversions and permission system

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 19:52:10 +01:00
Jeroen SchweitzerandClaude Opus 4.5 075b0ec297 feat: add system stats API for dashboard monitoring
Build and Push / build (release) Successful in 1m44s
- GET /tools/system/stats - Real-time host system statistics
- CPU usage, memory, all mounted disks, network I/O
- GPU/VRAM stats via nvidia-smi (if available)
- Uses psutil for cross-platform host metrics
- Auto-discovers and filters real filesystems

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 17:30:45 +01:00
103 changed files with 14002 additions and 1620 deletions
+69
View File
@@ -0,0 +1,69 @@
{
"env": {
"PQL_VAULT": "/mnt/media/Projects/core-api"
},
"permissions": {
"allow": [
"Bash(pql)",
"Bash(pql *)",
"Bash(/home/jpmschweitzer/.local/bin/pql:*)",
"Bash(git status:*)",
"Bash(git log:*)",
"Bash(git diff:*)",
"Bash(git branch:*)",
"Bash(.venv/bin/python -m pytest:*)",
"Bash(.venv/bin/pytest:*)",
"Bash(pytest:*)",
"Bash(alembic *)",
"Bash(docker logs core-api:*)",
"Bash(curl -s http://localhost:8083/*)"
],
"deny": [
"Bash(/mnt/media/Projects/cladmin/ops/bin/toj)",
"Bash(/mnt/media/Projects/cladmin/ops/bin/toj:*)",
"Bash(alembic downgrade base*)",
"Bash(chmod -R 777 *)",
"Bash(chmod 777 *)",
"Bash(dd if=*)",
"Bash(find * -delete*)",
"Bash(find * -exec*)",
"Bash(git * add --all*)",
"Bash(git * add -A*)",
"Bash(git * add .)",
"Bash(git * branch -D *)",
"Bash(git * checkout -- *)",
"Bash(git * clean -fd*)",
"Bash(git * clean -fdx*)",
"Bash(git * commit --no-verify*)",
"Bash(git * merge --no-ff*)",
"Bash(git * push --force*)",
"Bash(git * push -f*)",
"Bash(git * reset --hard*)",
"Bash(git * restore .*)",
"Bash(git add --all*)",
"Bash(git add -A*)",
"Bash(git add .)",
"Bash(git branch -D *)",
"Bash(git checkout -- *)",
"Bash(git clean -fd*)",
"Bash(git clean -fdx*)",
"Bash(git commit --no-verify*)",
"Bash(git merge --no-ff*)",
"Bash(git push --force*)",
"Bash(git push -f*)",
"Bash(git reset --hard*)",
"Bash(git restore .*)",
"Bash(mkfs*)",
"Bash(psql * -c DROP*)",
"Bash(psql * DROP DATABASE*)",
"Bash(psql * TRUNCATE*)",
"Bash(rm -rf $HOME)",
"Bash(rm -rf /)",
"Bash(rm -rf ~)",
"Bash(su *)",
"Bash(sudo *)",
"Bash(toj)",
"Bash(toj:*)"
]
}
}
+1 -4
View File
@@ -39,12 +39,9 @@ HOMEASSISTANT_URL=http://localhost:8123
HOMEASSISTANT_TOKEN=your-long-lived-access-token
# =============================================================================
# AI Services
# Search
# =============================================================================
# Ollama API
OLLAMA_BASE_URL=http://localhost:11434
# SearXNG (self-hosted search)
SEARXNG_URL=http://localhost:8080
+1
View File
@@ -0,0 +1 @@
.pql/changelog/*.sql merge=union
+18 -5
View File
@@ -1,19 +1,32 @@
name: Build and Push
on:
release:
types: [published]
push:
tags:
- 'v*'
jobs:
release:
runs-on: ubuntu-latest
steps:
- name: Create Gitea Release
run: |
curl -sf -X POST \
-H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \
-H "Content-Type: application/json" \
-d '{"tag_name": "${{ github.ref_name }}", "name": "Release ${{ github.ref_name }}", "body": "Automated release for ${{ github.ref_name }}"}' \
"${{ github.server_url }}/api/v1/repos/${{ github.repository }}/releases"
build:
runs-on: ubuntu-latest
needs: release
steps:
- uses: actions/checkout@v4
- name: Login to Gitea Registry
uses: docker/login-action@v3
with:
registry: git.schweitz.internal
registry: git.schweitz.net
username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }}
@@ -23,8 +36,8 @@ jobs:
context: .
push: true
tags: |
git.schweitz.internal/jpmschweitzer/core-api:latest
git.schweitz.internal/jpmschweitzer/core-api:${{ github.ref_name }}
git.schweitz.net/jpmschweitzer/core-api:latest
git.schweitz.net/jpmschweitzer/core-api:${{ github.ref_name }}
- name: Trigger Watchtower update
if: success()
+13
View File
@@ -0,0 +1,13 @@
#!/usr/bin/env bash
# Trigger only. The checks live in the Makefile, where they can be read, run by
# hand (`make pre-push`), and changed under review.
#
# This file is identical in every repo in this workspace, deliberately: the call
# surface is the same everywhere even though what each gate runs is not, so
# nobody has to read a repo to find out how to check it (D-27).
#
# Enable per clone with: git config core.hooksPath .githooks
# Never bypass with --no-verify. Suppress a specific finding deliberately
# instead, with a reason — see `make pre-push`.
set -euo pipefail
exec make -C "$(git rev-parse --show-toplevel)" pre-push
+17
View File
@@ -136,3 +136,20 @@ Thumbs.db
# profiling data
.prof
# Claude Code local settings (machine-specific, may contain credentials)
.claude/settings.local.json
# pql — ignore everything except the changelog, which is the replication log of
# record and must be committed for tickets to travel with the repo.
.pql/*
!.pql/changelog/
# pql shims planted by `pql init` into the dir core.hooksPath points at.
# Per-clone: each embeds the absolute path of the pql binary that planted it.
# Only .githooks/pre-push is shared.
.githooks/pre-commit
.githooks/post-merge
.githooks/post-checkout
.githooks/post-rewrite
+11
View File
@@ -0,0 +1,11 @@
-- Changelog format marker, written by pql. Comments only: this file
-- is never executed — Import descends into the per-table directories
-- and does not read the changelog root.
--
-- A changelog carrying no marker is format 1, the shape that existed
-- before formats were versioned. An older format is migrated forward
-- by `pql plan upgrade` (and automatically from the post-merge hook);
-- a newer one is refused rather than replayed under rules this binary
-- does not know. See D-28 and docs/versions.md.
-- pql:changelog_format: 2.0.0
-- pql:written_by: 2.2.0
+139
View File
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
+109
View File
@@ -0,0 +1,109 @@
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN6KV13SSS2KZP10SKTT903C', 'description', 'Caller authentication for service callers, the housekeeping routes locked behind it, and the `book` domain with migration 005 — the first half of the household''s Office storage. Driven by workspace T-166; the design is workspace D-64…D-72 (`cladmin/governance/decisions/office.md`), above all D-68 (the Book lives here), D-70 (the verbs) and D-71.
- `require_caller()`: the Office''s service key with a forwarded identity, the Scheduler''s key (dispatch only), else an Authentik JWT. On every new route and every housekeeping route — core-api is internet-facing at api.schweitz.net with no forward-auth, and `/housekeeping/devices` answers unauthenticated today.
- `src/domains/book/` and migration 005: records (note, task, decision, question, rejected), tags with system tags `work`/`personal`, links, attachments, append-only events.', 'Caller authentication for service callers, the housekeeping routes locked behind it, and the `book` domain with migration 005 — the first half of the household''s Office storage. Driven by workspace T-166; the design is workspace D-64…D-72 (`cladmin/governance/decisions/office.md`), above all D-68 (the Book lives here), D-70 (the verbs) and D-71.
- `require_caller()`: the Office''s service key with a forwarded identity, the Scheduler''s key (dispatch only), else an Authentik JWT. On every new route and every housekeeping route — core-api is internet-facing at api.schweitz.net with no forward-auth, and `/housekeeping/devices` answers unauthenticated today.
- `src/domains/book/` and migration 005: records (note, task, decision, question, rejected), tags with system tags `work`/`personal`, links, attachments, append-only events.
**Code done 2026-10-19**, commits `04ec54b`…`63feb88` (round core-api-T-1-001): `make test` 381 → 430 passed; migration 005 round-trips on `core_api_test`; production `core_api` still at 004; mutation-checked (key compare, the housekeeping lock, the append-only trigger). Follow-ups: T-2 (remaining verbs, short ids, scope tags — amends 005 before its first deploy), T-3 (alembic target). Held for the milestone.', NULL, '2026-10-19 09:20:45', '2026-10-19 09:20:45.376', '2026-10-19 09:20:45.376', NULL, 'f74c7130a534e2a97136986fb545eedd', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN6XMY61H4P86X0TKGD3D37W', 'description', 'The rest of the Book''s verbs (workspace D-70), and two corrections to what T-1 built — all before the Book''s first deploy. Driven by workspace T-166/T-167.
**Corrections, found reviewing T-1 (2026-10-19):**
- **Outward ids.** Records use UUIDs, but the Office names items in words — "Task 12", "Note 40" (workspace D-67). Add a short sequential `number` per record (unique, never reused) and accept it wherever a UUID is accepted. Migration 005 has never run against production `core_api` (still at 004), so **amend 005** rather than adding 006 — and re-prove its round trip on `core_api_test`.
- **`work` and `personal` as tags.** `tag`/`untag` treat them as free text; they must map onto the `scope` column (tagging `work` sets scope work; untagging the only scope is refused with a code that names the fix), so the outward "baked-in tags" of D-69 hold.
**The remaining verbs:** `append`, `remind`, `reschedule` (batch), `undo` (this identity''s last change, from `book_events`), `purge` (from trash only), `redact` (scrubs a value from the record and its events — the one audited exception to the append-only trigger), `pin`/`unpin`, `check`/`uncheck`, `snooze`, `archive`/`unarchive`, `trash`/`restore`, `pause`/`resume` (a project and its descendants through `parent`; statuses saved in `paused_statuses` and restored exactly), `due`, `next` (filters energy, estimate, context tags, scope); batch forms for every note- and write-class verb; an idempotency key on `add`.', 'The rest of the Book''s verbs (workspace D-70), and two corrections to what T-1 built — all before the Book''s first deploy. Driven by workspace T-166/T-167.
**Corrections, found reviewing T-1 (2026-10-19):**
- **Outward ids.** Records use UUIDs, but the Office names items in words — "Task 12", "Note 40" (workspace D-67). Add a short sequential `number` per record (unique, never reused) and accept it wherever a UUID is accepted. Migration 005 has never run against production `core_api` (still at 004), so **amend 005** rather than adding 006 — and re-prove its round trip on `core_api_test`.
- **`work` and `personal` as tags.** `tag`/`untag` treat them as free text; they must map onto the `scope` column (tagging `work` sets scope work; untagging the only scope is refused with a code that names the fix), so the outward "baked-in tags" of D-69 hold.
**The remaining verbs:** `append`, `remind`, `reschedule` (batch), `undo` (this identity''s last change, from `book_events`), `purge` (from trash only), `redact` (scrubs a value from the record and its events — the one audited exception to the append-only trigger), `pin`/`unpin`, `check`/`uncheck`, `snooze`, `archive`/`unarchive`, `trash`/`restore`, `pause`/`resume` (a project and its descendants through `parent`; statuses saved in `paused_statuses` and restored exactly), `due`, `next` (filters energy, estimate, context tags, scope); batch forms for every note- and write-class verb; an idempotency key on `add`.
**Add an owner (D-73).** Book records need an `owner` — the tatlock tenant id from the token''s signed `tatlock_user` claim (production `jpmschweitzer`), forwarded by the Office — and every query scoped by it. Not in migration 005 as written; add it while amending 005. The Scribe''s tables (workspace T-176) carry the same owner on addresses.', NULL, '2026-10-19 10:36:41', '2026-10-19 10:36:41.675', '2026-10-19 10:36:41.675', NULL, '2f858098d03ce3b28a64f4dcbdbea26a', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN6XMY61H4P86X0TKGD3D37W', 'description', 'The rest of the Book''s verbs (workspace D-70), and two corrections to what T-1 built — all before the Book''s first deploy. Driven by workspace T-166/T-167.
**Corrections, found reviewing T-1 (2026-10-19):**
- **Outward ids.** Records use UUIDs, but the Office names items in words — "Task 12", "Note 40" (workspace D-67). Add a short sequential `number` per record (unique, never reused) and accept it wherever a UUID is accepted. Migration 005 has never run against production `core_api` (still at 004), so **amend 005** rather than adding 006 — and re-prove its round trip on `core_api_test`.
- **`work` and `personal` as tags.** `tag`/`untag` treat them as free text; they must map onto the `scope` column (tagging `work` sets scope work; untagging the only scope is refused with a code that names the fix), so the outward "baked-in tags" of D-69 hold.
**The remaining verbs:** `append`, `remind`, `reschedule` (batch), `undo` (this identity''s last change, from `book_events`), `purge` (from trash only), `redact` (scrubs a value from the record and its events — the one audited exception to the append-only trigger), `pin`/`unpin`, `check`/`uncheck`, `snooze`, `archive`/`unarchive`, `trash`/`restore`, `pause`/`resume` (a project and its descendants through `parent`; statuses saved in `paused_statuses` and restored exactly), `due`, `next` (filters energy, estimate, context tags, scope); batch forms for every note- and write-class verb; an idempotency key on `add`.
**Add an owner (D-73).** Book records need an `owner` — the tatlock tenant id from the token''s signed `tatlock_user` claim (production `jpmschweitzer`), forwarded by the Office — and every query scoped by it. Not in migration 005 as written; add it while amending 005. The Scribe''s tables (workspace T-176) carry the same owner on addresses.', 'The rest of the Book''s verbs (workspace D-70), and two corrections to what T-1 built — all before the Book''s first deploy. Driven by workspace T-166/T-167.
**Corrections, found reviewing T-1 (2026-10-19):**
- **Outward ids.** Records use UUIDs, but the Office names items in words — "Task 12", "Note 40" (workspace D-67). Add a short sequential `number` per record (unique, never reused) and accept it wherever a UUID is accepted. Migration 005 has never run against production `core_api` (still at 004), so **amend 005** rather than adding 006 — and re-prove its round trip on `core_api_test`.
- **`work` and `personal` as tags.** `tag`/`untag` treat them as free text; they must map onto the `scope` column (tagging `work` sets scope work; untagging the only scope is refused with a code that names the fix), so the outward "baked-in tags" of D-69 hold.
**The remaining verbs:** `append`, `remind`, `reschedule` (batch), `undo` (this identity''s last change, from `book_events`), `purge` (from trash only), `redact` (scrubs a value from the record and its events — the one audited exception to the append-only trigger), `pin`/`unpin`, `check`/`uncheck`, `snooze`, `archive`/`unarchive`, `trash`/`restore`, `pause`/`resume` (a project and its descendants through `parent`; statuses saved in `paused_statuses` and restored exactly), `due`, `next` (filters energy, estimate, context tags, scope); batch forms for every note- and write-class verb; an idempotency key on `add`.
**Add an owner (D-73).** Book records need an `owner` — the tatlock tenant id from the token''s signed `tatlock_user` claim (production `jpmschweitzer`), forwarded by the Office — and every query scoped by it. Not in migration 005 as written; add it while amending 005. The Scribe''s tables (workspace T-176) carry the same owner on addresses.
**Isolation (D-73).** Every Book query and write filters on the owner tenant; a test proves an `llm_tester` caller cannot read, change, link to or close a `jpmschweitzer` record (and the reverse) — the error is `not_found`, so another tenant''s ids are not even confirmed to exist.', NULL, '2026-10-19 10:44:56', '2026-10-19 10:44:56.770', '2026-10-19 10:44:56.770', NULL, '5a9e8eb56b5634bdb34d36f9b5a6cdbf', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN7T48Z610867H891MZCVGA4', 'description', '`require_caller()` (T-1) learns who the Office''s caller is beyond a declared identity: the **consumer** (the Authentik client the request was signed in through, e.g. `office-claude`) and the **tenant** (the signed `tatlock_user` claim, e.g. `jpmschweitzer`, `llm_tester`). The Office has already verified the token; core-api trusts these only alongside the Office''s service key, as it does the identity. Driven by workspace T-185; design in workspace D-73.', '`require_caller()` (T-1) learns who the Office''s caller is beyond a declared identity: the **consumer** (the Authentik client the request was signed in through, e.g. `office-claude`) and the **tenant** (the signed `tatlock_user` claim, e.g. `jpmschweitzer`, `llm_tester`). The Office has already verified the token; core-api trusts these only alongside the Office''s service key, as it does the identity. Driven by workspace T-185; design in workspace D-73.
**Code done 2026-10-19** (round core-api-T-4-001): `0453d08`, `ce2ec61` — `Caller` carries consumer and tenant; the Office must send `X-Office-Consumer` and `X-Office-Tenant` on every call (400 `consumer_required`/`_invalid`, `tenant_required`/`_invalid`); other kinds ignore both headers; a JWT user''s tenant comes only from its own `tatlock_user` claim; the consumer lands in `book_events.client`. `make test` 430 → 451; mutation-checked (tenant requirement, header spoofing). Held — deploys with core-api T-1 at the milestone.', NULL, '2026-10-19 11:46:47', '2026-10-19 11:46:47.566', '2026-10-19 11:46:47.566', NULL, '1c452fa8a2ac5f5c326e9bf08e7981b6', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN7T48Z610867H891MZCVGA4', 'status', 'backlog', 'done', NULL, '2026-10-19 11:46:47', '2026-10-19 11:46:47.688', '2026-10-19 11:46:47.688', NULL, '877161504860711bfc61a86a6ed44e46', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN6XMYNQF0VP25QMY7YTB41W', 'description', '`alembic/env.py` always takes its target from `get_settings().database_url`, composed from `.env`''s `POSTGRES_*` — which point at **production** `core_api`. `get_settings()` is `lru_cache`d and is warmed by anything that imports `src.main`, so a test that needs to migrate `core_api_test` has to run alembic in a subprocess with every `POSTGRES_*` overridden. T-1''s tests do exactly that, correctly — but one missed override migrates production.
Fix: honour `-x db_url=…` (or a `TEST_DATABASE_URL`-style variable) in `env.py` before falling back to settings, and have the migration tests use it, so the target is named rather than inherited. Found by the T-1 round''s friction report.', '`alembic/env.py` always takes its target from `get_settings().database_url`, composed from `.env`''s `POSTGRES_*` — which point at **production** `core_api`. `get_settings()` is `lru_cache`d and is warmed by anything that imports `src.main`, so a test that needs to migrate `core_api_test` has to run alembic in a subprocess with every `POSTGRES_*` overridden. T-1''s tests do exactly that, correctly — but one missed override migrates production.
Fix: honour `-x db_url=…` (or a `TEST_DATABASE_URL`-style variable) in `env.py` before falling back to settings, and have the migration tests use it, so the target is named rather than inherited. Found by the T-1 round''s friction report.
Related, from the T-4 round: there is no make target to start a dev server against `core_api_test` — the round built the `POSTGRES_*` overrides from `TEST_DATABASE_URL` by hand, carefully unprinted. A `make dev-test-server` that does it would remove a recurring hand-built step that one slip turns into a run against production.', NULL, '2026-10-19 11:46:47', '2026-10-19 11:46:47.827', '2026-10-19 11:46:47.827', NULL, '1cf3bed3f8970e8f70a0b78845c75c35', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN8F5Z00KKNGTADK9BHG99N8', 'description', 'The messaging half of the Scribe, the household''s message bus: addresses, sending, mailboxes, history, search, read state and tags — enough for the first end-to-end flow through the Office. Driven by workspace T-186; design in workspace D-66 and D-73. Builds on T-1''s caller and T-4''s consumer and tenant.', 'The messaging half of the Scribe, the household''s message bus: addresses, sending, mailboxes, history, search, read state and tags — enough for the first end-to-end flow through the Office. Driven by workspace T-186; design in workspace D-66 and D-73. Builds on T-1''s caller and T-4''s consumer and tenant.
**Code done 2026-10-19** (round core-api-T-5-001): `ccdbc44`, `93b00b6` — migration 006 (`scribe_addresses` keyed on tenant + name, retained `scribe_messages` with full text, `scribe_mailbox_entries`), routes register, list, send, mailbox, history, find and one PATCH for mark/tag/untag; `from` stamped from the caller; every query filtered by tenant, another tenant''s ids answered as missing; an entry not addressed to the caller''s identity answers 404 even inside the tenant. `make test` 451 → 489; mutation-checked (tenant filter, from-stamping, ownership). Held for release. **The round printed the `core_api_test` password into its output; the lead rotated it** (role and `.env`, never printed; tests pass on the new one).', NULL, '2026-10-19 13:24:10', '2026-10-19 13:24:10.811', '2026-10-19 13:24:10.811', NULL, 'a43a09f6533c7433c348d6a15f669920', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN8F5Z00KKNGTADK9BHG99N8', 'status', 'backlog', 'done', NULL, '2026-10-19 13:24:10', '2026-10-19 13:24:10.937', '2026-10-19 13:24:10.937', NULL, '07a38b83b95b5a7d62f1ef4704408375', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN6XMYNQF0VP25QMY7YTB41W', 'description', '`alembic/env.py` always takes its target from `get_settings().database_url`, composed from `.env`''s `POSTGRES_*` — which point at **production** `core_api`. `get_settings()` is `lru_cache`d and is warmed by anything that imports `src.main`, so a test that needs to migrate `core_api_test` has to run alembic in a subprocess with every `POSTGRES_*` overridden. T-1''s tests do exactly that, correctly — but one missed override migrates production.
Fix: honour `-x db_url=…` (or a `TEST_DATABASE_URL`-style variable) in `env.py` before falling back to settings, and have the migration tests use it, so the target is named rather than inherited. Found by the T-1 round''s friction report.
Related, from the T-4 round: there is no make target to start a dev server against `core_api_test` — the round built the `POSTGRES_*` overrides from `TEST_DATABASE_URL` by hand, carefully unprinted. A `make dev-test-server` that does it would remove a recurring hand-built step that one slip turns into a run against production.', '`alembic/env.py` always takes its target from `get_settings().database_url`, composed from `.env`''s `POSTGRES_*` — which point at **production** `core_api`. `get_settings()` is `lru_cache`d and is warmed by anything that imports `src.main`, so a test that needs to migrate `core_api_test` has to run alembic in a subprocess with every `POSTGRES_*` overridden. T-1''s tests do exactly that, correctly — but one missed override migrates production.
Fix: honour `-x db_url=…` (or a `TEST_DATABASE_URL`-style variable) in `env.py` before falling back to settings, and have the migration tests use it, so the target is named rather than inherited. Found by the T-1 round''s friction report.
Related, from the T-4 round: there is no make target to start a dev server against `core_api_test` — the round built the `POSTGRES_*` overrides from `TEST_DATABASE_URL` by hand, carefully unprinted. A `make dev-test-server` that does it would remove a recurring hand-built step that one slip turns into a run against production.
Raised again by the T-5 round: every round rebuilds the `:8777` dev server against `core_api_test` by hand, and the T-5 round leaked the test password doing it. Make this target the next thing core-api does.', NULL, '2026-10-19 13:24:11', '2026-10-19 13:24:11.249', '2026-10-19 13:24:11.249', NULL, 'fe81a166cb0353f3a1e0747758c04e44', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN6XMY61H4P86X0TKGD3D37W', 'description', 'The rest of the Book''s verbs (workspace D-70), and two corrections to what T-1 built — all before the Book''s first deploy. Driven by workspace T-166/T-167.
**Corrections, found reviewing T-1 (2026-10-19):**
- **Outward ids.** Records use UUIDs, but the Office names items in words — "Task 12", "Note 40" (workspace D-67). Add a short sequential `number` per record (unique, never reused) and accept it wherever a UUID is accepted. Migration 005 has never run against production `core_api` (still at 004), so **amend 005** rather than adding 006 — and re-prove its round trip on `core_api_test`.
- **`work` and `personal` as tags.** `tag`/`untag` treat them as free text; they must map onto the `scope` column (tagging `work` sets scope work; untagging the only scope is refused with a code that names the fix), so the outward "baked-in tags" of D-69 hold.
**The remaining verbs:** `append`, `remind`, `reschedule` (batch), `undo` (this identity''s last change, from `book_events`), `purge` (from trash only), `redact` (scrubs a value from the record and its events — the one audited exception to the append-only trigger), `pin`/`unpin`, `check`/`uncheck`, `snooze`, `archive`/`unarchive`, `trash`/`restore`, `pause`/`resume` (a project and its descendants through `parent`; statuses saved in `paused_statuses` and restored exactly), `due`, `next` (filters energy, estimate, context tags, scope); batch forms for every note- and write-class verb; an idempotency key on `add`.
**Add an owner (D-73).** Book records need an `owner` — the tatlock tenant id from the token''s signed `tatlock_user` claim (production `jpmschweitzer`), forwarded by the Office — and every query scoped by it. Not in migration 005 as written; add it while amending 005. The Scribe''s tables (workspace T-176) carry the same owner on addresses.
**Isolation (D-73).** Every Book query and write filters on the owner tenant; a test proves an `llm_tester` caller cannot read, change, link to or close a `jpmschweitzer` record (and the reverse) — the error is `not_found`, so another tenant''s ids are not even confirmed to exist.', 'The rest of the Book''s verbs (workspace D-70), and two corrections to what T-1 built — all before the Book''s first deploy. Driven by workspace T-166/T-167.
**Corrections, found reviewing T-1 (2026-10-19):**
- **Outward ids.** Records use UUIDs, but the Office names items in words — "Task 12", "Note 40" (workspace D-67). Add a short sequential `number` per record (unique, never reused) and accept it wherever a UUID is accepted. Migration 005 has never run against production `core_api` (still at 004), so **amend 005** rather than adding 006 — and re-prove its round trip on `core_api_test`.
- **`work` and `personal` as tags.** `tag`/`untag` treat them as free text; they must map onto the `scope` column (tagging `work` sets scope work; untagging the only scope is refused with a code that names the fix), so the outward "baked-in tags" of D-69 hold.
**The remaining verbs:** `append`, `remind`, `reschedule` (batch), `undo` (this identity''s last change, from `book_events`), `purge` (from trash only), `redact` (scrubs a value from the record and its events — the one audited exception to the append-only trigger), `pin`/`unpin`, `check`/`uncheck`, `snooze`, `archive`/`unarchive`, `trash`/`restore`, `pause`/`resume` (a project and its descendants through `parent`; statuses saved in `paused_statuses` and restored exactly), `due`, `next` (filters energy, estimate, context tags, scope); batch forms for every note- and write-class verb; an idempotency key on `add`.
**Add an owner (D-73).** Book records need an `owner` — the tatlock tenant id from the token''s signed `tatlock_user` claim (production `jpmschweitzer`), forwarded by the Office — and every query scoped by it. Not in migration 005 as written; add it while amending 005. The Scribe''s tables (workspace T-176) carry the same owner on addresses.
**Isolation (D-73).** Every Book query and write filters on the owner tenant; a test proves an `llm_tester` caller cannot read, change, link to or close a `jpmschweitzer` record (and the reverse) — the error is `not_found`, so another tenant''s ids are not even confirmed to exist.
**Split 2026-10-19:** the short outward ids and the owner tenant (amending 005), and the Book''s tenant filter, moved to **T-7**, so core-api can release safely. T-2 keeps the remaining verbs and work/personal as tags.', NULL, '2026-10-19 14:06:27', '2026-10-19 14:06:27.325', '2026-10-19 14:06:27.325', NULL, '0e233f93ee1e85162d05109749d33032', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN8Z1FF1QDGR4QE4405AP8GC', 'status', 'backlog', 'done', NULL, '2026-10-19 14:51:47', '2026-10-19 14:51:47.496', '2026-10-19 14:51:47.496', NULL, '625ca10cfc9bcaf576a2fd52a3fe2792', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN6KV13SSS2KZP10SKTT903C', 'status', 'backlog', 'done', NULL, '2026-10-19 15:48:03', '2026-10-19 15:48:03.951', '2026-10-19 15:48:03.951', NULL, '51baa7767d5176263b817493d40ff5e9', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN99CB552PSF82EE2P8PJZP0', 'status', 'backlog', 'done', NULL, '2026-10-19 15:48:04', '2026-10-19 15:48:04.079', '2026-10-19 15:48:04.079', NULL, '252442ffd3a53fbb45bfdd6b24d2752c', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GNBAZ6KW704AF07ZBR3YC868', 'description', '**A default address per consumer grant**, so a client that keeps no MCP session (claude.ai''s connector) need not name itself on every call. The Office uses it when a call names no address and has no session binding (tatlock-office T-5).
- `tenant_consumers.default_address` — nullable; a composite foreign key `(tenant_id, default_address)` → `scribe_addresses(owner_tenant, name)`, so a default always names a real address in the same tenant. The Office still checks the consumer owns it.
- Migration 008 sets `jpmschweitzer` / `office-claude` → `claude-ai`; `llm_tester` / `office-claude` stays null.
- `GET /tenants/me` answers `default_address` — the caller''s own consumer''s default, or null (user-kind callers carry no consumer: null).
- No write route; set by migration until provisioning lands (workspace T-195).', '**A default address per consumer grant**, so a client that keeps no MCP session (claude.ai''s connector) need not name itself on every call. The Office uses it when a call names no address and has no session binding (tatlock-office T-5).
- `tenant_consumers.default_address` — nullable; a composite foreign key `(tenant_id, default_address)` → `scribe_addresses(owner_tenant, name)`, so a default always names a real address in the same tenant. The Office still checks the consumer owns it.
- Migration 008 sets `jpmschweitzer` / `office-claude` → `claude-ai`; `llm_tester` / `office-claude` stays null.
- `GET /tenants/me` answers `default_address` — the caller''s own consumer''s default, or null (user-kind callers carry no consumer: null).
- No write route; set by migration until provisioning lands (workspace T-195).
Live in core-api 1.13.0; production migrated to 008 — jpmschweitzer/office-claude → claude-ai, llm_tester none; /tenants/me answers it.', NULL, '2026-10-19 20:16:05', '2026-10-19 20:16:05.428', '2026-10-19 20:16:05.428', NULL, 'e1691631641e5a8a37743c8a53fe6c75', 2) ON CONFLICT(hash) DO NOTHING;
INSERT INTO ticket_history (ticket_record_id, field, old_value, new_value, changed_by, changed_at, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GNBAZ6KW704AF07ZBR3YC868', 'status', 'backlog', 'done', NULL, '2026-10-19 20:16:05', '2026-10-19 20:16:05.567', '2026-10-19 20:16:05.567', NULL, 'd4fb4422de5f1d47f6ab52904623cd0d', 2) ON CONFLICT(hash) DO NOTHING;
+139
View File
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
+13
View File
@@ -0,0 +1,13 @@
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN6KV13SSS2KZP10SKTT903C', 'T-1', '2026-10-19 08:37:53.567', '2026-10-19 08:37:53.567', NULL, '624cc203712b63205f39eecd40c86d95', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN6XMY61H4P86X0TKGD3D37W', 'T-2', '2026-10-19 09:20:45.104', '2026-10-19 09:20:45.104', NULL, '6d58bd9debd724db63e6067a82013b5f', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN6XMYNQF0VP25QMY7YTB41W', 'T-3', '2026-10-19 09:20:45.230', '2026-10-19 09:20:45.230', NULL, 'fdd572c5cfad3a68c1477ecede1db7f4', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN7T48Z610867H891MZCVGA4', 'T-4', '2026-10-19 11:25:10.777', '2026-10-19 11:25:10.777', NULL, 'e30571204475f25d624a8d593829cda9', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN8F5Z00KKNGTADK9BHG99N8', 'T-5', '2026-10-19 12:57:09.632', '2026-10-19 12:57:09.632', NULL, 'a8774787d37deb533e701914864fd560', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN8NBWVQ53ZX7WMZNE5GJJP4', 'T-6', '2026-10-19 13:24:11.102', '2026-10-19 13:24:11.102', NULL, 'f6351f2d6190ae5fbd9e55e6eadc04e5', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN8Z1FF1QDGR4QE4405AP8GC', 'T-7', '2026-10-19 14:06:27.193', '2026-10-19 14:06:27.193', NULL, '09267e8234ec21c6d75767d7cd21bb51', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN99CB552PSF82EE2P8PJZP0', 'T-8', '2026-10-19 14:51:37.641', '2026-10-19 14:51:37.641', NULL, 'd8c7f6122652cb1c3e448f1adedd54ab', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GNA8JCWVP31DKSE7F4EMF6G4', 'T-9', '2026-10-19 17:07:53.702', '2026-10-19 17:07:53.702', NULL, 'f7caf295773331bd290e8f9fd196ce6d', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GNA9K6GSTB09D1SWV649DZ3M', 'T-10', '2026-10-19 17:12:22.406', '2026-10-19 17:12:22.406', NULL, '3bd35329f2ace334c5ef64d8e52ae9cf', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GNBAZ6KW704AF07ZBR3YC868', 'T-11', '2026-10-19 19:38:11.488', '2026-10-19 19:38:11.488', NULL, '8150ae921a6b2682265917fb669c67e0', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GNBHXYDJEEDQEADCP3JTR7BR', 'T-12', '2026-10-19 20:08:36.204', '2026-10-19 20:08:36.204', NULL, '087d8b4674d3e12ca3b274fc4e255ade', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
INSERT INTO ticket_idmap (record_id, ticket_id, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GNBR1VBM8SZP8DDK93SFS608', 'T-13', '2026-10-19 20:35:21.053', '2026-10-19 20:35:21.053', NULL, 'aab7f828f5f01bd8310c85f53ff0dc19', 2) ON CONFLICT(record_id) DO UPDATE SET ticket_id=excluded.ticket_id, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= ticket_idmap.updated_at;
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
+139
View File
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
+152
View File
@@ -0,0 +1,152 @@
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN6KV13SSS2KZP10SKTT903C', 'task', NULL, 'Caller auth, locked housekeeping routes, and the Book domain', 'Caller authentication for service callers, the housekeeping routes locked behind it, and the `book` domain with migration 005 — the first half of the household''s Office storage. Driven by workspace T-166; the design is workspace D-64…D-72 (`cladmin/governance/decisions/office.md`), above all D-68 (the Book lives here), D-70 (the verbs) and D-71.
- `require_caller()`: the Office''s service key with a forwarded identity, the Scheduler''s key (dispatch only), else an Authentik JWT. On every new route and every housekeeping route — core-api is internet-facing at api.schweitz.net with no forward-auth, and `/housekeeping/devices` answers unauthenticated today.
- `src/domains/book/` and migration 005: records (note, task, decision, question, rejected), tags with system tags `work`/`personal`, links, attachments, append-only events.', 'backlog', 'high', NULL, NULL, NULL, '2026-10-19 08:37:53.566', '2026-10-19 08:37:53.566', NULL, '2caf93d2159440ddba20ebe336115a3c', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN6XMY61H4P86X0TKGD3D37W', 'task', NULL, 'The Book''s remaining verbs, short outward ids, and work/personal as tags', 'The rest of the Book''s verbs (workspace D-70), and two corrections to what T-1 built — all before the Book''s first deploy. Driven by workspace T-166/T-167.
**Corrections, found reviewing T-1 (2026-10-19):**
- **Outward ids.** Records use UUIDs, but the Office names items in words — "Task 12", "Note 40" (workspace D-67). Add a short sequential `number` per record (unique, never reused) and accept it wherever a UUID is accepted. Migration 005 has never run against production `core_api` (still at 004), so **amend 005** rather than adding 006 — and re-prove its round trip on `core_api_test`.
- **`work` and `personal` as tags.** `tag`/`untag` treat them as free text; they must map onto the `scope` column (tagging `work` sets scope work; untagging the only scope is refused with a code that names the fix), so the outward "baked-in tags" of D-69 hold.
**The remaining verbs:** `append`, `remind`, `reschedule` (batch), `undo` (this identity''s last change, from `book_events`), `purge` (from trash only), `redact` (scrubs a value from the record and its events — the one audited exception to the append-only trigger), `pin`/`unpin`, `check`/`uncheck`, `snooze`, `archive`/`unarchive`, `trash`/`restore`, `pause`/`resume` (a project and its descendants through `parent`; statuses saved in `paused_statuses` and restored exactly), `due`, `next` (filters energy, estimate, context tags, scope); batch forms for every note- and write-class verb; an idempotency key on `add`.', 'backlog', 'high', NULL, NULL, NULL, '2026-10-19 09:20:45.104', '2026-10-19 09:20:45.104', NULL, '9034c13962b3a44d6572e1e2d92e3ab3', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN6XMYNQF0VP25QMY7YTB41W', 'task', NULL, 'Alembic takes its target from production settings; name it instead', '`alembic/env.py` always takes its target from `get_settings().database_url`, composed from `.env`''s `POSTGRES_*` — which point at **production** `core_api`. `get_settings()` is `lru_cache`d and is warmed by anything that imports `src.main`, so a test that needs to migrate `core_api_test` has to run alembic in a subprocess with every `POSTGRES_*` overridden. T-1''s tests do exactly that, correctly — but one missed override migrates production.
Fix: honour `-x db_url=…` (or a `TEST_DATABASE_URL`-style variable) in `env.py` before falling back to settings, and have the migration tests use it, so the target is named rather than inherited. Found by the T-1 round''s friction report.', 'backlog', 'medium', NULL, NULL, NULL, '2026-10-19 09:20:45.229', '2026-10-19 09:20:45.229', NULL, 'b096eb96b790b241405fe4703d4911af', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN6KV13SSS2KZP10SKTT903C', 'task', NULL, 'Caller auth, locked housekeeping routes, and the Book domain', 'Caller authentication for service callers, the housekeeping routes locked behind it, and the `book` domain with migration 005 — the first half of the household''s Office storage. Driven by workspace T-166; the design is workspace D-64…D-72 (`cladmin/governance/decisions/office.md`), above all D-68 (the Book lives here), D-70 (the verbs) and D-71.
- `require_caller()`: the Office''s service key with a forwarded identity, the Scheduler''s key (dispatch only), else an Authentik JWT. On every new route and every housekeeping route — core-api is internet-facing at api.schweitz.net with no forward-auth, and `/housekeeping/devices` answers unauthenticated today.
- `src/domains/book/` and migration 005: records (note, task, decision, question, rejected), tags with system tags `work`/`personal`, links, attachments, append-only events.
**Code done 2026-10-19**, commits `04ec54b`…`63feb88` (round core-api-T-1-001): `make test` 381 → 430 passed; migration 005 round-trips on `core_api_test`; production `core_api` still at 004; mutation-checked (key compare, the housekeeping lock, the append-only trigger). Follow-ups: T-2 (remaining verbs, short ids, scope tags — amends 005 before its first deploy), T-3 (alembic target). Held for the milestone.', 'backlog', 'high', NULL, NULL, NULL, '2026-10-19 08:37:53.566', '2026-10-19 09:20:45.375', NULL, '27ad92892863649997bb7a0ca147cbed', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN6XMY61H4P86X0TKGD3D37W', 'task', NULL, 'The Book''s remaining verbs, short outward ids, and work/personal as tags', 'The rest of the Book''s verbs (workspace D-70), and two corrections to what T-1 built — all before the Book''s first deploy. Driven by workspace T-166/T-167.
**Corrections, found reviewing T-1 (2026-10-19):**
- **Outward ids.** Records use UUIDs, but the Office names items in words — "Task 12", "Note 40" (workspace D-67). Add a short sequential `number` per record (unique, never reused) and accept it wherever a UUID is accepted. Migration 005 has never run against production `core_api` (still at 004), so **amend 005** rather than adding 006 — and re-prove its round trip on `core_api_test`.
- **`work` and `personal` as tags.** `tag`/`untag` treat them as free text; they must map onto the `scope` column (tagging `work` sets scope work; untagging the only scope is refused with a code that names the fix), so the outward "baked-in tags" of D-69 hold.
**The remaining verbs:** `append`, `remind`, `reschedule` (batch), `undo` (this identity''s last change, from `book_events`), `purge` (from trash only), `redact` (scrubs a value from the record and its events — the one audited exception to the append-only trigger), `pin`/`unpin`, `check`/`uncheck`, `snooze`, `archive`/`unarchive`, `trash`/`restore`, `pause`/`resume` (a project and its descendants through `parent`; statuses saved in `paused_statuses` and restored exactly), `due`, `next` (filters energy, estimate, context tags, scope); batch forms for every note- and write-class verb; an idempotency key on `add`.
**Add an owner (D-73).** Book records need an `owner` — the tatlock tenant id from the token''s signed `tatlock_user` claim (production `jpmschweitzer`), forwarded by the Office — and every query scoped by it. Not in migration 005 as written; add it while amending 005. The Scribe''s tables (workspace T-176) carry the same owner on addresses.', 'backlog', 'high', NULL, NULL, NULL, '2026-10-19 09:20:45.104', '2026-10-19 10:36:41.674', NULL, '39efc332e98d1f72eff1763b33125b19', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN6XMY61H4P86X0TKGD3D37W', 'task', NULL, 'The Book''s remaining verbs, short outward ids, and work/personal as tags', 'The rest of the Book''s verbs (workspace D-70), and two corrections to what T-1 built — all before the Book''s first deploy. Driven by workspace T-166/T-167.
**Corrections, found reviewing T-1 (2026-10-19):**
- **Outward ids.** Records use UUIDs, but the Office names items in words — "Task 12", "Note 40" (workspace D-67). Add a short sequential `number` per record (unique, never reused) and accept it wherever a UUID is accepted. Migration 005 has never run against production `core_api` (still at 004), so **amend 005** rather than adding 006 — and re-prove its round trip on `core_api_test`.
- **`work` and `personal` as tags.** `tag`/`untag` treat them as free text; they must map onto the `scope` column (tagging `work` sets scope work; untagging the only scope is refused with a code that names the fix), so the outward "baked-in tags" of D-69 hold.
**The remaining verbs:** `append`, `remind`, `reschedule` (batch), `undo` (this identity''s last change, from `book_events`), `purge` (from trash only), `redact` (scrubs a value from the record and its events — the one audited exception to the append-only trigger), `pin`/`unpin`, `check`/`uncheck`, `snooze`, `archive`/`unarchive`, `trash`/`restore`, `pause`/`resume` (a project and its descendants through `parent`; statuses saved in `paused_statuses` and restored exactly), `due`, `next` (filters energy, estimate, context tags, scope); batch forms for every note- and write-class verb; an idempotency key on `add`.
**Add an owner (D-73).** Book records need an `owner` — the tatlock tenant id from the token''s signed `tatlock_user` claim (production `jpmschweitzer`), forwarded by the Office — and every query scoped by it. Not in migration 005 as written; add it while amending 005. The Scribe''s tables (workspace T-176) carry the same owner on addresses.
**Isolation (D-73).** Every Book query and write filters on the owner tenant; a test proves an `llm_tester` caller cannot read, change, link to or close a `jpmschweitzer` record (and the reverse) — the error is `not_found`, so another tenant''s ids are not even confirmed to exist.', 'backlog', 'high', NULL, NULL, NULL, '2026-10-19 09:20:45.104', '2026-10-19 10:44:56.770', NULL, 'd930f0dbc1197dff3600619b31a6b0c6', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN7T48Z610867H891MZCVGA4', 'task', NULL, 'The caller carries consumer and tenant from the Office', '`require_caller()` (T-1) learns who the Office''s caller is beyond a declared identity: the **consumer** (the Authentik client the request was signed in through, e.g. `office-claude`) and the **tenant** (the signed `tatlock_user` claim, e.g. `jpmschweitzer`, `llm_tester`). The Office has already verified the token; core-api trusts these only alongside the Office''s service key, as it does the identity. Driven by workspace T-185; design in workspace D-73.', 'backlog', 'high', NULL, NULL, NULL, '2026-10-19 11:25:10.777', '2026-10-19 11:25:10.777', NULL, 'e73c46c5e77917c049e92ff1f627f447', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN7T48Z610867H891MZCVGA4', 'task', NULL, 'The caller carries consumer and tenant from the Office', '`require_caller()` (T-1) learns who the Office''s caller is beyond a declared identity: the **consumer** (the Authentik client the request was signed in through, e.g. `office-claude`) and the **tenant** (the signed `tatlock_user` claim, e.g. `jpmschweitzer`, `llm_tester`). The Office has already verified the token; core-api trusts these only alongside the Office''s service key, as it does the identity. Driven by workspace T-185; design in workspace D-73.
**Code done 2026-10-19** (round core-api-T-4-001): `0453d08`, `ce2ec61` — `Caller` carries consumer and tenant; the Office must send `X-Office-Consumer` and `X-Office-Tenant` on every call (400 `consumer_required`/`_invalid`, `tenant_required`/`_invalid`); other kinds ignore both headers; a JWT user''s tenant comes only from its own `tatlock_user` claim; the consumer lands in `book_events.client`. `make test` 430 → 451; mutation-checked (tenant requirement, header spoofing). Held — deploys with core-api T-1 at the milestone.', 'backlog', 'high', NULL, NULL, NULL, '2026-10-19 11:25:10.777', '2026-10-19 11:46:47.566', NULL, 'ed07dd9dcd17801b51cbbf8cb3145451', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN7T48Z610867H891MZCVGA4', 'task', NULL, 'The caller carries consumer and tenant from the Office', '`require_caller()` (T-1) learns who the Office''s caller is beyond a declared identity: the **consumer** (the Authentik client the request was signed in through, e.g. `office-claude`) and the **tenant** (the signed `tatlock_user` claim, e.g. `jpmschweitzer`, `llm_tester`). The Office has already verified the token; core-api trusts these only alongside the Office''s service key, as it does the identity. Driven by workspace T-185; design in workspace D-73.
**Code done 2026-10-19** (round core-api-T-4-001): `0453d08`, `ce2ec61` — `Caller` carries consumer and tenant; the Office must send `X-Office-Consumer` and `X-Office-Tenant` on every call (400 `consumer_required`/`_invalid`, `tenant_required`/`_invalid`); other kinds ignore both headers; a JWT user''s tenant comes only from its own `tatlock_user` claim; the consumer lands in `book_events.client`. `make test` 430 → 451; mutation-checked (tenant requirement, header spoofing). Held — deploys with core-api T-1 at the milestone.', 'done', 'high', NULL, NULL, NULL, '2026-10-19 11:25:10.777', '2026-10-19 11:46:47.688', NULL, '4fbb627f0d1f48350ebd4d97fc64fd94', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN6XMYNQF0VP25QMY7YTB41W', 'task', NULL, 'Alembic takes its target from production settings; name it instead', '`alembic/env.py` always takes its target from `get_settings().database_url`, composed from `.env`''s `POSTGRES_*` — which point at **production** `core_api`. `get_settings()` is `lru_cache`d and is warmed by anything that imports `src.main`, so a test that needs to migrate `core_api_test` has to run alembic in a subprocess with every `POSTGRES_*` overridden. T-1''s tests do exactly that, correctly — but one missed override migrates production.
Fix: honour `-x db_url=…` (or a `TEST_DATABASE_URL`-style variable) in `env.py` before falling back to settings, and have the migration tests use it, so the target is named rather than inherited. Found by the T-1 round''s friction report.
Related, from the T-4 round: there is no make target to start a dev server against `core_api_test` — the round built the `POSTGRES_*` overrides from `TEST_DATABASE_URL` by hand, carefully unprinted. A `make dev-test-server` that does it would remove a recurring hand-built step that one slip turns into a run against production.', 'backlog', 'medium', NULL, NULL, NULL, '2026-10-19 09:20:45.229', '2026-10-19 11:46:47.827', NULL, '3d53cf5d73b33f32d80232456e23f5ad', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN8F5Z00KKNGTADK9BHG99N8', 'task', NULL, 'The Scribe''s messaging: addresses, send, mailbox, history, find, mark, tag', 'The messaging half of the Scribe, the household''s message bus: addresses, sending, mailboxes, history, search, read state and tags — enough for the first end-to-end flow through the Office. Driven by workspace T-186; design in workspace D-66 and D-73. Builds on T-1''s caller and T-4''s consumer and tenant.', 'backlog', 'high', NULL, NULL, NULL, '2026-10-19 12:57:09.632', '2026-10-19 12:57:09.632', NULL, 'ae66ca9641212daeb46527917c324175', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN8F5Z00KKNGTADK9BHG99N8', 'task', NULL, 'The Scribe''s messaging: addresses, send, mailbox, history, find, mark, tag', 'The messaging half of the Scribe, the household''s message bus: addresses, sending, mailboxes, history, search, read state and tags — enough for the first end-to-end flow through the Office. Driven by workspace T-186; design in workspace D-66 and D-73. Builds on T-1''s caller and T-4''s consumer and tenant.
**Code done 2026-10-19** (round core-api-T-5-001): `ccdbc44`, `93b00b6` — migration 006 (`scribe_addresses` keyed on tenant + name, retained `scribe_messages` with full text, `scribe_mailbox_entries`), routes register, list, send, mailbox, history, find and one PATCH for mark/tag/untag; `from` stamped from the caller; every query filtered by tenant, another tenant''s ids answered as missing; an entry not addressed to the caller''s identity answers 404 even inside the tenant. `make test` 451 → 489; mutation-checked (tenant filter, from-stamping, ownership). Held for release. **The round printed the `core_api_test` password into its output; the lead rotated it** (role and `.env`, never printed; tests pass on the new one).', 'backlog', 'high', NULL, NULL, NULL, '2026-10-19 12:57:09.632', '2026-10-19 13:24:10.810', NULL, '5351a56b3c5e6fb9594f0b2d87717774', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN8F5Z00KKNGTADK9BHG99N8', 'task', NULL, 'The Scribe''s messaging: addresses, send, mailbox, history, find, mark, tag', 'The messaging half of the Scribe, the household''s message bus: addresses, sending, mailboxes, history, search, read state and tags — enough for the first end-to-end flow through the Office. Driven by workspace T-186; design in workspace D-66 and D-73. Builds on T-1''s caller and T-4''s consumer and tenant.
**Code done 2026-10-19** (round core-api-T-5-001): `ccdbc44`, `93b00b6` — migration 006 (`scribe_addresses` keyed on tenant + name, retained `scribe_messages` with full text, `scribe_mailbox_entries`), routes register, list, send, mailbox, history, find and one PATCH for mark/tag/untag; `from` stamped from the caller; every query filtered by tenant, another tenant''s ids answered as missing; an entry not addressed to the caller''s identity answers 404 even inside the tenant. `make test` 451 → 489; mutation-checked (tenant filter, from-stamping, ownership). Held for release. **The round printed the `core_api_test` password into its output; the lead rotated it** (role and `.env`, never printed; tests pass on the new one).', 'done', 'high', NULL, NULL, NULL, '2026-10-19 12:57:09.632', '2026-10-19 13:24:10.937', NULL, 'a75c4791f0cf9c91a56afc65e474bd43', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN8NBWVQ53ZX7WMZNE5GJJP4', 'bug', NULL, 'User-kind callers have no consumer, so their address ownership matches any other''s', 'Callers of kind `user` (an Authentik user straight to core-api, e.g. tatlock-ui later) carry no consumer, so `scribe_addresses.owner_consumer` is `NULL` for anything they register — and ownership checks compare consumers, so two different user-kind callers in one tenant would each pass as owner of the other''s addresses (`None == None`). `book_events.client` has the same nullable shape. Not reachable through the Office (every Office call carries a real consumer since T-4), so not on the messaging slice''s path — but it must be closed before any user-kind caller registers or sends. Fix: give a user-kind caller a consumer of its own (e.g. `user:<preferred_username>`), or refuse address ownership to callers without one. Found in the T-5 round''s report.', 'backlog', 'medium', NULL, NULL, NULL, '2026-10-19 13:24:11.102', '2026-10-19 13:24:11.102', NULL, 'd7e0fab068383f77f9a0dd0b7b421a92', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN6XMYNQF0VP25QMY7YTB41W', 'task', NULL, 'Alembic takes its target from production settings; name it instead', '`alembic/env.py` always takes its target from `get_settings().database_url`, composed from `.env`''s `POSTGRES_*` — which point at **production** `core_api`. `get_settings()` is `lru_cache`d and is warmed by anything that imports `src.main`, so a test that needs to migrate `core_api_test` has to run alembic in a subprocess with every `POSTGRES_*` overridden. T-1''s tests do exactly that, correctly — but one missed override migrates production.
Fix: honour `-x db_url=…` (or a `TEST_DATABASE_URL`-style variable) in `env.py` before falling back to settings, and have the migration tests use it, so the target is named rather than inherited. Found by the T-1 round''s friction report.
Related, from the T-4 round: there is no make target to start a dev server against `core_api_test` — the round built the `POSTGRES_*` overrides from `TEST_DATABASE_URL` by hand, carefully unprinted. A `make dev-test-server` that does it would remove a recurring hand-built step that one slip turns into a run against production.
Raised again by the T-5 round: every round rebuilds the `:8777` dev server against `core_api_test` by hand, and the T-5 round leaked the test password doing it. Make this target the next thing core-api does.', 'backlog', 'medium', NULL, NULL, NULL, '2026-10-19 09:20:45.229', '2026-10-19 13:24:11.248', NULL, '54942cb45b6635f1d21a053e8a8e25c6', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN8Z1FF1QDGR4QE4405AP8GC', 'task', NULL, 'The Book''s final schema and tenant boundary, before 005''s first release', 'The Book''s final schema and its tenant boundary, before core-api''s first release applies migration 005 to production. Split from T-2 (whose remaining verbs stay there). Amend 005 — never run in production — rather than add a migration: per-tenant short outward numbers ("Task 12", workspace D-67), an owner tenant on every record (D-73), and every Book route filtered by it, so the routes are safe to expose the moment they deploy.', 'backlog', 'high', NULL, NULL, NULL, '2026-10-19 14:06:27.192', '2026-10-19 14:06:27.192', NULL, 'cd22ba2ca9a28b7690305eda1f8153b4', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN6XMY61H4P86X0TKGD3D37W', 'task', NULL, 'The Book''s remaining verbs, short outward ids, and work/personal as tags', 'The rest of the Book''s verbs (workspace D-70), and two corrections to what T-1 built — all before the Book''s first deploy. Driven by workspace T-166/T-167.
**Corrections, found reviewing T-1 (2026-10-19):**
- **Outward ids.** Records use UUIDs, but the Office names items in words — "Task 12", "Note 40" (workspace D-67). Add a short sequential `number` per record (unique, never reused) and accept it wherever a UUID is accepted. Migration 005 has never run against production `core_api` (still at 004), so **amend 005** rather than adding 006 — and re-prove its round trip on `core_api_test`.
- **`work` and `personal` as tags.** `tag`/`untag` treat them as free text; they must map onto the `scope` column (tagging `work` sets scope work; untagging the only scope is refused with a code that names the fix), so the outward "baked-in tags" of D-69 hold.
**The remaining verbs:** `append`, `remind`, `reschedule` (batch), `undo` (this identity''s last change, from `book_events`), `purge` (from trash only), `redact` (scrubs a value from the record and its events — the one audited exception to the append-only trigger), `pin`/`unpin`, `check`/`uncheck`, `snooze`, `archive`/`unarchive`, `trash`/`restore`, `pause`/`resume` (a project and its descendants through `parent`; statuses saved in `paused_statuses` and restored exactly), `due`, `next` (filters energy, estimate, context tags, scope); batch forms for every note- and write-class verb; an idempotency key on `add`.
**Add an owner (D-73).** Book records need an `owner` — the tatlock tenant id from the token''s signed `tatlock_user` claim (production `jpmschweitzer`), forwarded by the Office — and every query scoped by it. Not in migration 005 as written; add it while amending 005. The Scribe''s tables (workspace T-176) carry the same owner on addresses.
**Isolation (D-73).** Every Book query and write filters on the owner tenant; a test proves an `llm_tester` caller cannot read, change, link to or close a `jpmschweitzer` record (and the reverse) — the error is `not_found`, so another tenant''s ids are not even confirmed to exist.
**Split 2026-10-19:** the short outward ids and the owner tenant (amending 005), and the Book''s tenant filter, moved to **T-7**, so core-api can release safely. T-2 keeps the remaining verbs and work/personal as tags.', 'backlog', 'high', NULL, NULL, NULL, '2026-10-19 09:20:45.104', '2026-10-19 14:06:27.324', NULL, '58063dda134a948a461cd97325286e48', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN99CB552PSF82EE2P8PJZP0', 'story', NULL, 'The tenant registry: tenants, consumer grants, and one enforcement point', 'The tenant registry: one table that says which tenants exist, which consumers may act in each, and whether a tenant is active — the anchor every service''s tenancy conforms to (workspace D-73).
Today a tenant is an unchecked string: the Office header or a JWT''s `tatlock_user` claim is trusted as-is, and the Book and the Scribe file rows under whatever value arrives. A typo''d claim opens a fresh, empty namespace instead of being refused.
**Tables (migration 007):**
- `tenants` — `id` (the slug, the value of `tatlock_user`; same pattern `_require_office_tenant` accepts), `display_name`, `kind` (`household` | `person` | `test`), `status` (`active` | `suspended`), timestamps. Seeded: `jpmschweitzer` (household), `llm_tester` (test).
- `tenant_consumers` — a grant: consumer X may act in tenant Y. PK `(tenant_id, consumer)`. Seeded: `office-claude` in both seeded tenants.
- Foreign keys from every `owner_tenant` column in the Book and the Scribe to `tenants.id`.
**Enforcement:** every tenant-scoped operation goes through one helper. No tenant → 400 `tenant_required`. Tenant unknown → 403 `tenant_unknown`. Tenant suspended → 403 `tenant_suspended`. Office caller whose consumer holds no grant in that tenant → 403 `consumer_not_granted`.
**Read API, for other services to conform to:** `GET /tenants/me` (the caller''s own tenant and its consumer''s grant); `GET /tenants` (service callers without a tenant see all; a tenant-bearing caller sees only its own — the hard boundary).
**Not in v1:** write routes for provisioning (T-195 on the workspace board adds them, or an operator verb); per-consumer verb classes (workspace T-184 — a column on `tenant_consumers` when it lands).', 'backlog', 'high', NULL, NULL, NULL, '2026-10-19 14:51:37.641', '2026-10-19 14:51:37.641', NULL, '756a467f5dc4a84c7ee9080ef21ca5fb', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN8Z1FF1QDGR4QE4405AP8GC', 'task', NULL, 'The Book''s final schema and tenant boundary, before 005''s first release', 'The Book''s final schema and its tenant boundary, before core-api''s first release applies migration 005 to production. Split from T-2 (whose remaining verbs stay there). Amend 005 — never run in production — rather than add a migration: per-tenant short outward numbers ("Task 12", workspace D-67), an owner tenant on every record (D-73), and every Book route filtered by it, so the routes are safe to expose the moment they deploy.', 'done', 'high', NULL, NULL, NULL, '2026-10-19 14:06:27.192', '2026-10-19 14:51:47.496', NULL, '8797eb16319aa33c8317b0a1f0defc3d', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN6KV13SSS2KZP10SKTT903C', 'task', NULL, 'Caller auth, locked housekeeping routes, and the Book domain', 'Caller authentication for service callers, the housekeeping routes locked behind it, and the `book` domain with migration 005 — the first half of the household''s Office storage. Driven by workspace T-166; the design is workspace D-64…D-72 (`cladmin/governance/decisions/office.md`), above all D-68 (the Book lives here), D-70 (the verbs) and D-71.
- `require_caller()`: the Office''s service key with a forwarded identity, the Scheduler''s key (dispatch only), else an Authentik JWT. On every new route and every housekeeping route — core-api is internet-facing at api.schweitz.net with no forward-auth, and `/housekeeping/devices` answers unauthenticated today.
- `src/domains/book/` and migration 005: records (note, task, decision, question, rejected), tags with system tags `work`/`personal`, links, attachments, append-only events.
**Code done 2026-10-19**, commits `04ec54b`…`63feb88` (round core-api-T-1-001): `make test` 381 → 430 passed; migration 005 round-trips on `core_api_test`; production `core_api` still at 004; mutation-checked (key compare, the housekeeping lock, the append-only trigger). Follow-ups: T-2 (remaining verbs, short ids, scope tags — amends 005 before its first deploy), T-3 (alembic target). Held for the milestone.', 'done', 'high', NULL, NULL, NULL, '2026-10-19 08:37:53.566', '2026-10-19 15:48:03.950', NULL, '4f256fd107a235ce29e881208eca9fbc', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GN99CB552PSF82EE2P8PJZP0', 'story', NULL, 'The tenant registry: tenants, consumer grants, and one enforcement point', 'The tenant registry: one table that says which tenants exist, which consumers may act in each, and whether a tenant is active — the anchor every service''s tenancy conforms to (workspace D-73).
Today a tenant is an unchecked string: the Office header or a JWT''s `tatlock_user` claim is trusted as-is, and the Book and the Scribe file rows under whatever value arrives. A typo''d claim opens a fresh, empty namespace instead of being refused.
**Tables (migration 007):**
- `tenants` — `id` (the slug, the value of `tatlock_user`; same pattern `_require_office_tenant` accepts), `display_name`, `kind` (`household` | `person` | `test`), `status` (`active` | `suspended`), timestamps. Seeded: `jpmschweitzer` (household), `llm_tester` (test).
- `tenant_consumers` — a grant: consumer X may act in tenant Y. PK `(tenant_id, consumer)`. Seeded: `office-claude` in both seeded tenants.
- Foreign keys from every `owner_tenant` column in the Book and the Scribe to `tenants.id`.
**Enforcement:** every tenant-scoped operation goes through one helper. No tenant → 400 `tenant_required`. Tenant unknown → 403 `tenant_unknown`. Tenant suspended → 403 `tenant_suspended`. Office caller whose consumer holds no grant in that tenant → 403 `consumer_not_granted`.
**Read API, for other services to conform to:** `GET /tenants/me` (the caller''s own tenant and its consumer''s grant); `GET /tenants` (service callers without a tenant see all; a tenant-bearing caller sees only its own — the hard boundary).
**Not in v1:** write routes for provisioning (T-195 on the workspace board adds them, or an operator verb); per-consumer verb classes (workspace T-184 — a column on `tenant_consumers` when it lands).', 'done', 'high', NULL, NULL, NULL, '2026-10-19 14:51:37.641', '2026-10-19 15:48:04.079', NULL, '4adf867d5e7e4424e6a2e97b53f958d1', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GNA8JCWVP31DKSE7F4EMF6G4', 'task', NULL, 'The mailbox answers by message number, so no client pages to find an entry', '`GET /scribe/mailbox` sorts unread first, then by id descending, behind one `before_id` cursor — two sort keys, one cursor. No cursor choice is safe in every case: paging below the last row is correct whenever a page holds a read entry, but a page of only unread entries (more than 200 unread) skips read entries with higher ids. Both clients that turn a message number into a mailbox entry — the Office (`src/systems/scribe.py` `_resolve_entry_id`) and `toj scribe mark` — page the mailbox to do it.
**Fix:** a `message_id` query parameter on `GET /scribe/mailbox` that answers the caller''s own entry for that message (zero or one), so neither client pages. Optionally, a cursor that matches the sort (`before` over `(read, id)`) for the listing itself.
Done when: both clients drop their paging for a single call, and a test seeds an old unread entry ahead of more than a page of read ones and finds a read entry beyond the first page.', 'backlog', 'medium', NULL, NULL, NULL, '2026-10-19 17:07:53.702', '2026-10-19 17:07:53.702', NULL, 'c6599dbbb04302146398615a0caf9e2d', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GNA9K6GSTB09D1SWV649DZ3M', 'task', NULL, 'Scribe messages number per tenant, like the Book', 'Scribe message numbers are global: the playground tenant''s first message was "message 2" because message 1 lives in the household''s tenant. That lets one tenant count another''s mail — a small leak through the hard boundary (workspace D-73), and exactly why the Book numbers per tenant (T-7). Give messages a per-tenant `number` the same way (`owner_tenant`, a per-tenant counter taken FOR UPDATE), answer every route with it, and accept it wherever a message id is taken today. A migration on live data: existing messages get numbers in id order per tenant.', 'backlog', 'medium', NULL, NULL, NULL, '2026-10-19 17:12:22.406', '2026-10-19 17:12:22.406', NULL, '3d3b298bf83b044360d3141137a259a6', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GNBAZ6KW704AF07ZBR3YC868', 'story', NULL, 'A default address on each consumer grant, answered by /tenants/me', '**A default address per consumer grant**, so a client that keeps no MCP session (claude.ai''s connector) need not name itself on every call. The Office uses it when a call names no address and has no session binding (tatlock-office T-5).
- `tenant_consumers.default_address` — nullable; a composite foreign key `(tenant_id, default_address)` → `scribe_addresses(owner_tenant, name)`, so a default always names a real address in the same tenant. The Office still checks the consumer owns it.
- Migration 008 sets `jpmschweitzer` / `office-claude` → `claude-ai`; `llm_tester` / `office-claude` stays null.
- `GET /tenants/me` answers `default_address` — the caller''s own consumer''s default, or null (user-kind callers carry no consumer: null).
- No write route; set by migration until provisioning lands (workspace T-195).', 'backlog', 'high', NULL, NULL, NULL, '2026-10-19 19:38:11.488', '2026-10-19 19:38:11.488', NULL, '0bcc2ad2e1df4cc4410f73b9f55092ca', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GNBHXYDJEEDQEADCP3JTR7BR', 'task', NULL, 'A failing database test under pytest -v prints the test database password', 'Found in round core-api-T-11-001: when a test that takes the `test_database_url` fixture fails under `pytest -v`, pytest shows the fixture''s value in the traceback — the `core_api_test` URL with its password. Local test credential, not production''s, but Rule 3 draws no line on severity, and every debugging session that lets one fail under -v repeats it. Fix the fixture so its value never renders (a repr-safe wrapper, or pass the parts rather than the URL), and say in CLAUDE.md how to debug a DB test without it.', 'backlog', 'medium', NULL, NULL, NULL, '2026-10-19 20:08:36.204', '2026-10-19 20:08:36.204', NULL, '0ed2ee8eb049c3f0b32d57c0a37b3f6b', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GNBAZ6KW704AF07ZBR3YC868', 'story', NULL, 'A default address on each consumer grant, answered by /tenants/me', '**A default address per consumer grant**, so a client that keeps no MCP session (claude.ai''s connector) need not name itself on every call. The Office uses it when a call names no address and has no session binding (tatlock-office T-5).
- `tenant_consumers.default_address` — nullable; a composite foreign key `(tenant_id, default_address)` → `scribe_addresses(owner_tenant, name)`, so a default always names a real address in the same tenant. The Office still checks the consumer owns it.
- Migration 008 sets `jpmschweitzer` / `office-claude` → `claude-ai`; `llm_tester` / `office-claude` stays null.
- `GET /tenants/me` answers `default_address` — the caller''s own consumer''s default, or null (user-kind callers carry no consumer: null).
- No write route; set by migration until provisioning lands (workspace T-195).
Live in core-api 1.13.0; production migrated to 008 — jpmschweitzer/office-claude → claude-ai, llm_tester none; /tenants/me answers it.', 'backlog', 'high', NULL, NULL, NULL, '2026-10-19 19:38:11.488', '2026-10-19 20:16:05.428', NULL, '5ce494e5d0f29bf5fc2371d4f1c0c4a1', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GNBAZ6KW704AF07ZBR3YC868', 'story', NULL, 'A default address on each consumer grant, answered by /tenants/me', '**A default address per consumer grant**, so a client that keeps no MCP session (claude.ai''s connector) need not name itself on every call. The Office uses it when a call names no address and has no session binding (tatlock-office T-5).
- `tenant_consumers.default_address` — nullable; a composite foreign key `(tenant_id, default_address)` → `scribe_addresses(owner_tenant, name)`, so a default always names a real address in the same tenant. The Office still checks the consumer owns it.
- Migration 008 sets `jpmschweitzer` / `office-claude` → `claude-ai`; `llm_tester` / `office-claude` stays null.
- `GET /tenants/me` answers `default_address` — the caller''s own consumer''s default, or null (user-kind callers carry no consumer: null).
- No write route; set by migration until provisioning lands (workspace T-195).
Live in core-api 1.13.0; production migrated to 008 — jpmschweitzer/office-claude → claude-ai, llm_tester none; /tenants/me answers it.', 'done', 'high', NULL, NULL, NULL, '2026-10-19 19:38:11.488', '2026-10-19 20:16:05.567', NULL, 'fe9a3bd6efdd20903e4e7563c59f4bba', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
INSERT INTO tickets (record_id, type, parent_record_id, title, description, status, priority, assigned_to, team, decision_ref, created_at, updated_at, deleted_at, hash, canonical_version) VALUES ('06GNBR1VBM8SZP8DDK93SFS608', 'story', NULL, 'Archive and trash a message in your own mailbox', 'The user wants a way to get rid of a message that is no longer useful. A message is retained and shared by every recipient (workspace D-66 — history is how agents rebuild context), so deletion is **per mailbox**, in D-70''s vocabulary:
- **archive / unarchive** — the entry leaves the default mailbox listing; still in `history`, still findable.
- **trash / restore** — the entry leaves the mailbox, restorable; trashed entries are listed only when asked for.
- True erasure of a message''s content stays `redact` (write, audited) — not this ticket.
**core-api:** `scribe_mailbox_entries` gains `archived_at` and `trashed_at`; `PATCH /scribe/mailbox/{entry_id}` takes `archived` and `trashed` booleans beside `read`; `GET /scribe/mailbox` takes `view` (`inbox` default: neither archived nor trashed · `archive` · `trash` · `all`). Paired with T-9 (look an entry up by message number) in one round, since both touch the same route.', 'backlog', 'high', NULL, NULL, NULL, '2026-10-19 20:35:21.053', '2026-10-19 20:35:21.053', NULL, 'd4fe2d5d06329b627a04dec089ca8a8b', 2) ON CONFLICT(record_id) DO UPDATE SET type=excluded.type, parent_record_id=excluded.parent_record_id, title=excluded.title, description=excluded.description, status=excluded.status, priority=excluded.priority, assigned_to=excluded.assigned_to, team=excluded.team, decision_ref=excluded.decision_ref, updated_at=excluded.updated_at, deleted_at=excluded.deleted_at, hash=excluded.hash, canonical_version=excluded.canonical_version WHERE excluded.updated_at >= tickets.updated_at;
-72
View File
@@ -1,72 +0,0 @@
# AGENTS.md
> **Start every session by reading this file.**
> This file outlines the operational protocols, coding standards, and architectural decisions for this FastAPI project.
## 1. Agent Operational Protocols
### 🧠 Work Patterns (Plan-Act-Reflect)
* **Plan:** Before writing code, briefly outline your plan. Identify which files you will touch and what the side effects might be.
* **Act:** Execute the changes in small, atomic steps.
* **Reflect:** After coding, verify your work. Did you break existing tests? Did you add new tests?
### 🛡️ Git Discipline
* **NEVER commit to `main` or `master` directly.** Always create a feature branch: `feature/your-feature-name` or `fix/issue-description`.
* **Commit Messages:** Use the [Conventional Commits](https://www.conventionalcommits.org/) format.
* `feat: add user login endpoint`
* `fix: resolve database connection timeout`
* `refactor: split monolith dependency file`
* **Atomic Commits:** Keep commits small. One logical change = one commit.
### 📝 Changelog Maintenance
* **Update `CHANGELOG.md`** with every user-facing change.
* Format: `## [Unreleased] - YYYY-MM-DD` followed by `### Added`, `### Changed`, or `### Fixed`.
### 🚀 Release Flow
When changes are ready for deployment:
1. **Ask user if deploy cycle is desired **
2. **Update version** in `pyproject.toml`:
- Bug fixes: bump patch version (1.8.3 → 1.8.4)
- New features: bump minor version (1.8.4 → 1.9.0)
3. **Update CHANGELOG.md**:
- Move items from `[Unreleased]` to new version section
- Add release date: `## [1.8.4] - 2025-12-16`
4. **Commit and tag**:
```bash
git add -A
git commit -m "fix: description of changes"
git tag v1.8.4
git push origin main --tags
```
5. **CI/CD triggers automatically**:
- Gitea CI builds Docker image on new tag
- Watchtower pulls and deploys to production
- Verify deployment: `curl http://192.168.86.149:8000/health`
---
## 2. FastAPI Architecture & Best Practices
*Reference: [FastAPI Best Practices](https://github.com/zhanymkanov/fastapi-best-practices)*
### 📂 Project Structure (Directory-based, NOT File-type based)
Do **not** group files by type (e.g., one huge `routers` folder). Group by **domain/module** inside a `src/` directory.
**Correct Structure:**
```text
src/
├── auth/
│ ├── router.py # Endpoints
│ ├── schemas.py # Pydantic models
│ ├── service.py # Business logic (CRUD, etc.)
│ ├── dependencies.py# Module-specific dependencies
│ └── config.py # Module-specific settings
├── posts/
│ ├── router.py
│ └── ...
└── main.py # App entry point
+335
View File
@@ -5,6 +5,341 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
## [1.14.0] - 2026-10-19
### Added
- **Archive and trash a message in your own mailbox** (`scribe_mailbox_entries.archived_at`
and `trashed_at`, migration 009, core-api T-13) — deletion is per mailbox entry, never per
message, since a message is retained and shared by every recipient (D-66/D-70). `PATCH
/scribe/mailbox/{entry_id}` takes `archived` and `trashed` booleans beside `read` (true sets
the timestamp, false clears it); the response carries both states. `GET /scribe/mailbox`
takes `view`: `inbox` (default, neither archived nor trashed), `archive` (archived, not
trashed), `trash` (trashed, regardless of `archived_at`), `all`. `find` now excludes trashed
entries.
- **Look up a mailbox entry by message number** (core-api T-9) — `GET
/scribe/mailbox?message_id=N` answers the caller's own entry for that message, zero or one,
in any view, bypassing `unread`/`tag`/`before_id` so no client pages the unread-first/
id-descending listing to find one.
## [1.13.0] - 2026-10-19
### Added
- **A default address on each consumer grant** (`tenant_consumers.default_address`,
migration 008, core-api T-11) — the Office's fallback when a call names no
address and holds no session binding, since claude.ai's connector keeps no
MCP session between tool calls. A composite foreign key
`(tenant_id, default_address)` -> `scribe_addresses(owner_tenant, name)`
ensures a default always names a real address in the same tenant.
`jpmschweitzer`/`office-claude` defaults to `claude-ai` where that address
already exists; every other grant stays null. `GET /tenants/me` answers it
for the caller's own consumer grant, or null for a user-kind caller or a
grant without one; `GET /tenants` is unchanged. No write route this round —
provisioning is later.
## [1.12.0] - 2026-10-19
### Added
- **The tenant registry** (`src/domains/tenants/`, core-api T-8 / workspace
D-73) - `tenants` and `tenant_consumers` (migration 007), and
`TenantsService.require_tenant`, the one enforcement point Book's and
Scribe's own `_require_tenant` now delegate to instead of checking only
presence: a tenant that is not in the table is 403 `tenant_unknown`, one
marked `suspended` is 403 `tenant_suspended`, and an Office caller whose
consumer holds no grant in that tenant is 403 `consumer_not_granted` -
closing the gap where a typo'd `tatlock_user` claim opened a fresh, empty
namespace rather than being refused. Every existing `owner_tenant` column
(`book_records`, `book_tenant_counters`, `scribe_addresses`,
`scribe_messages`, `scribe_mailbox_entries`) now carries a foreign key to
`tenants.id` (`ON DELETE RESTRICT`). Migration 007 seeds the two tenants
in use today - `jpmschweitzer` (household) and `llm_tester` (test) - both
granted to `office-claude`, the only Office consumer configured in
Authentik. Read-only this round: `GET /tenants/me` answers the caller's
own tenant and its granted consumers (the cheap "am I allowed here"
probe); `GET /tenants` answers every tenant for a `tatlock`/`scheduler`
caller or a list of one for a tenant-bearing caller, never revealing
another tenant's existence. No write routes - provisioning a tenant is
later (workspace T-195).
- **The Scribe** (`src/domains/scribe/`) - the household's message bus
(dossier core-api T-5 / workspace D-66, D-68, D-73): `register`/`list`
addresses, `send`, `mailbox`, `history`, `find`, `mark`, `tag`/`untag`, all
under `/scribe/*`, behind `require_caller("office", "user")`. An address is
a topic owned by a tenant and a consumer; `send`'s `from` is always the
caller's own identity, never the request body. Every query filters on the
caller's tenant (D-73) - an address or message in another tenant answers
exactly like one that does not exist, so no id is ever confirmed across
that boundary. Messaging only this round: groups, subscriptions, the
notification matrix and delivery are later work (workspace T-176).
Migration 006 creates `scribe_addresses`, `scribe_messages` (retained,
full-text indexed), `scribe_mailbox_entries`.
- **Caller carries consumer and tenant from the Office** (`src/domains/auth/
callers.py`, core-api T-4 / workspace D-73) - `Caller` gains `consumer` and
`tenant`. For an `office` caller both `X-Office-Consumer` and
`X-Office-Tenant` are required on every call, reads included: missing is
400 `consumer_required`/`tenant_required`, malformed is 400
`consumer_invalid`/`tenant_invalid`. Every other kind ignores both headers
outright - `tatlock` and `scheduler` carry no tenant, and a `user` caller's
tenant comes only from their own JWT's `tatlock_user` claim, never from a
header.
- **Caller auth** (`src/domains/auth/callers.py`) - the first thing in this repo
that reads a service key. `require_caller()` resolves a static office/tatlock/
scheduler key (constant-time compare, checked before any JWT parsing) or falls
through to the existing OIDC path. New settings: `office_service_key`,
`tatlock_service_key`, `scheduler_service_key`, `book_timezone`.
- **The Book** (`src/domains/book/`) - Notes, Tasks, Decisions, Questions and
Rejected options (workspace D-68/D-69/D-70). Routes: `add`, `show`, `list`,
`find`, `change`, `close`, `link`/`unlink`, `tag`/`untag`, `history`, all under
`/book/*`, behind `require_caller("office", "user")`. Migration 005 creates the
whole schema: `book_records`, `book_tags`, `book_record_links`,
`book_attachments`, `book_events` (append-only, DB trigger-enforced),
`book_tenant_counters`.
- **The Book is behind the tenant boundary** (core-api T-7 / workspace D-73) -
every route now requires a tenant on the caller (400 `tenant_required` for
a `user`-kind caller with no `tatlock_user` claim; an `office` caller
already has one from every call per T-4) and filters on it: a record in
another tenant answers exactly like one that does not exist (404
`not_found`), in both directions, and `link`/`add`'s `parent` convenience
refuse a target in another tenant the same way. `add` sets `owner_tenant`
from the caller and allocates `number` - a short outward id numbered per
tenant (each tenant's Book starts at 1, unique on `(owner_tenant,
number)`, never reused), allocated via a per-tenant counter row
(`book_tenant_counters`) locked `FOR UPDATE` in the same transaction as
the insert. Responses include `number`; `list` accepts `number=` to find
one. Migration 005 (unreleased - production was still at 004) is amended
in place to add `book_records.owner_tenant`/`number` and
`book_tenant_counters`, rather than patching them in with a later
migration.
### Changed
- **`X-Office-Client` is retired in favour of `X-Office-Consumer`** (core-api
T-4) - the informational header is no longer read; `book_events.client`
(no migration this round) now records the consumer instead.
- **Housekeeping reads are locked** - `GET /housekeeping/*` (health, devices,
areas, scenes, scripts, automations, history) now require a caller
(`office`, `tatlock`, or `user`) instead of being unauthenticated. Writes are
unchanged (`get_admin_user`). `api.schweitz.net` carries no forward-auth, so
this was an open read of the house's device list once Home Assistant's token
is restored (T-1 / D-68).
## [1.11.0] - 2026-01-08
### Added
- **News Headlines API** - New endpoint for news ticker integration
- `GET /tools/news` - Fetch news headlines from user's volatile collection
- Returns headlines with title, description, source, and URL
- Data sourced from `volatile_{user}` Qdrant collection (news namespace)
- Uses `preferred_username` from OIDC, falls back to `default`
- News subdomain under tools (`src/domains/tools/news/`)
- `NewsHeadline` and `NewsResponse` Pydantic schemas
- `NewsService` for parsing news data from Qdrant
## [1.10.12] - 2026-01-08
### Fixed
- **Forecast data retrieval** - Pass raw_data to service instead of extracting wrong field
- Qdrant client was extracting `days` (integer 7) instead of `daily` (list)
- Now passes full raw_data for service to parse correctly
## [1.10.11] - 2026-01-08
### Fixed
- **Weather wind direction** - Convert integer degrees to cardinal direction string
- Scheduler stores wind_direction as degrees (e.g., 135)
- Schema expects string (e.g., "SE")
- Added `_degrees_to_cardinal()` conversion
## [1.10.10] - 2026-01-08
### Fixed
- **Environment data parsing** - Fix parsing of scheduler-generated Qdrant data
- Forecast: Check `daily` key first (scheduler stores day count in `days`, list in `daily`)
- Sun times: Use `sunrise_iso`/`sunset_iso` fields, handle time-only format fallback
- Sun times: Calculate daylight from `daylight_duration_seconds` or `daylight_hours`
- Air quality: Support `aqi_us`/`aqi_european` and `nitrogen_dioxide` field names
## [1.10.9] - 2026-01-08
### Fixed
- **Environment user ID cleanup** - Strip email domain from user identifier
- If `preferred_username` is an email, extract just the username part
- Ensures Qdrant collection name matches (e.g., `volatile_jpmschweitzer` not `volatile_jpmschweitzer@gmail.com`)
## [1.10.8] - 2026-01-08
### Fixed
- **OIDC audience validation** - python-jose requires string audience, not list
- Extract and validate audience from unverified claims first
- Use token's actual audience for JWT decode (after validating it's allowed)
- Fixes "audience must be a string or None" error
## [1.10.7] - 2026-01-08
### Added
- **Multi-issuer OIDC support** - Accept tokens from multiple OAuth providers
- Changed `oidc_issuer` (string) to `oidc_issuers` (list)
- Each issuer has its own JWKS endpoint, now cached per-issuer
- Validates token issuer against allowed list before fetching JWKS
- Supports tokens from: `core-api`, `tatlock-ui`, `tatlock` OAuth applications
- Completes fix for environment endpoint user resolution
### Removed
- Deprecated `src/config.py` - consolidated to `src/shared/config.py`
- Deprecated `src/security.py` - consolidated to `src/shared/security.py`
## [1.10.6] - 2026-01-08
### Fixed
- **OIDC audience mismatch** - Accept tokens from multiple clients
- Changed `oidc_audience` (string) to `oidc_audiences` (list)
- Now accepts tokens with audience: `core-api`, `tatlock-ui`, or `tatlock`
- Fixes environment endpoint returning "default" user instead of authenticated username
- Fixed main.py to use `oidc_audiences[0]` for Swagger UI OAuth client
### Changed
- Documentation cleanup in README
## [1.10.4] - 2026-01-07
### Removed
- **Ollama integration removed** - AI inference is no longer handled by this API
- Removed `src/models/ollama_client.py` and all Ollama-related configuration
- Removed `src/models/embeddings.py` and `src/models/embeddings_ollama.py`
- Removed model aliases and AI configuration from settings
- Health endpoints no longer check Ollama status
- Tests updated to reflect database-only health checks
### Changed
- Health check `/health/full` now only checks database connectivity
- Diagnostics endpoint simplified (removed Ollama component info)
## [1.10.3] - 2026-01-07
### Fixed
- **OIDC config not applied to domains module** - Both `src.auth.oidc` and `src.domains.auth.oidc` configs are now initialized
- Previously only `src.auth.oidc` was configured, leaving domains tools using hardcoded "local" user
- Environment endpoint now correctly uses authenticated user from OIDC token
## [1.10.2] - 2026-01-07
### Changed
- **Enhanced environment endpoint logging** - Added detailed user claim logging for debugging
- Logs both `preferred_username` and `sub` claims when resolving user
- Distinguishes between authenticated and unauthenticated requests
## [1.10.1] - 2026-01-06
### Fixed
- Fix OIDC import path in tools controller (`src.oidc.dependencies` → `src.auth.oidc`)
- Environment endpoint was returning `user: "local"` instead of authenticated username
- Caused queries to wrong Qdrant collection (`volatile_local` vs `volatile_{username}`)
## [1.10.0] - 2026-01-06
### Added
- **Environment Data API** - Qdrant-backed endpoint for weather, forecast, and sun position data
- `GET /tools/environment` - Fetch environment data from user's volatile collection
- Weather: current temperature, conditions, humidity, wind speed
- Forecast: multi-day outlook with high/low temperatures
- Sun times: sunrise, sunset, daylight duration
- Air quality: AQI and quality level (when available)
- Data sourced from `volatile_{user}` Qdrant collection
- Uses `preferred_username` from OIDC, falls back to `default`
- `qdrant-client` dependency for vector database access
- `QdrantReadClient` wrapper for read-only collection queries
- Comprehensive test suite for environment service parsing
## [1.9.4] - 2026-01-04
### Fixed
- Fix SQLAlchemy async lazy loading error for new users in `/auth/sync`
- Initialize `user.roles = []` and `user.preferences` to avoid greenlet error
- Was causing "MissingGreenlet: greenlet_spawn has not been called" on new user creation
## [1.9.3] - 2026-01-04
### Fixed
- Handle non-UUID `sub` claim in `/auth/sync` - Authentik JWT may return non-UUID subject identifiers
- Now derives deterministic UUID from sub string if direct parsing fails
## [1.9.2] - 2026-01-04
### Fixed
- `/auth/users/me` endpoint now supports both NPM forward auth headers AND JWT Bearer tokens
- Added `get_current_user_or_forward_auth()` combined auth dependency
- Fixes web authentication where NPM passes `X-authentik-*` headers instead of JWT
- Mobile/native clients continue to use JWT Bearer tokens as before
## [1.9.1] - 2026-01-03
### Fixed
- CORS configuration now uses explicit origins instead of `"*"`
- When `allow_credentials=True`, wildcard origins are rejected by browsers
- Added `home.schweitz.net`, `tatlock.schweitz.net`, and localhost origins
## [1.9.0] - 2026-01-03
### Added
- **NPM Forward Auth Support** - Web authentication via Nginx Proxy Manager forward auth
- `GET /auth/me` - Get current user from NPM forward auth headers (X-authentik-uid, X-authentik-email, etc.)
- Auto-creates user on first web login if not in database
- Syncs roles from NPM forward auth groups header
- `get_user_by_email` and `get_user_by_authentik_id` methods in AuthService
- Comprehensive tests for `/auth/me` endpoint
## [1.8.0] - 2026-01-03
### Added
- **Group-Role Mapping & Permissions** (Phase 3)
- Decoupled group-role architecture (groups from Authentik, roles admin-managed)
- Permission format: `domain.category:action` with action hierarchy
- `require_permission` and `require_any_permission` dependency factories
- Global admin override (`admin.general:admin`)
- **User Profile & API Keys** (Phase 4)
- `GET /auth/users/me` - Full user profile with roles and preferences
- `GET/PATCH /auth/users/me/preferences` - User preferences management
- `GET/POST/DELETE /auth/users/me/api-keys` - API key lifecycle
- API keys with `tak_` prefix, SHA-256 hashing, shown only once on creation
## [1.7.0] - 2026-01-03
### Added
- **System Stats API** - Host system resource monitoring for dashboard widgets
- `GET /tools/system/stats` - Real-time host system statistics
- CPU: usage percentage, core count, load averages
- Memory: usage percentage, total/used/available bytes
- Disks: all mounted filesystems with usage stats (auto-discovers mounts)
- Network: total bytes sent/received
- GPU/VRAM: NVIDIA GPU memory usage (via nvidia-smi if available)
- `psutil` dependency for cross-platform system metrics
## [1.6.1] - 2026-01-03
### Added
+177
View File
@@ -0,0 +1,177 @@
# CLAUDE.md — core-api
FastAPI service providing infrastructure management, home automation, and utility
endpoints for the homelab. Talks to Portainer, Nginx Proxy Manager, Home Assistant,
Postgres (via SQLAlchemy async + Alembic), Qdrant, and Authentik (OIDC). Deployed on
tower-of-joy at **:8083**.
## Ports — these differ, deliberately
| | Port | How |
|---|---|---|
| Local dev | **8788** | `./wakeup.sh`, uvicorn `--reload`, logs to `logs/server.log` |
| Production | **8083** | container; health at `http://192.168.86.149:8083/health` |
Testing `localhost:8083` on the dev box hits the *container*, not your reload server.
## Live contract
The live contract is always `http://localhost:8083/openapi.json` (62 paths, verified
2026-08-09) and human docs at `http://localhost:8083/docs` / `/redoc` — generated from
running code, so query it rather than inferring routes from source or from the README's
endpoint list, which can drift.
## Architecture
Domain-first layout under `src/domains/<name>/{controller,models,schemas,service}.py`
(auth, dashboard, health, housekeeping, infrastructure, static, tools). `src/main.py`
wires only `src.domains.*` — verify by reading its imports.
**Legacy top-level packages — "not in `main.py`" does not mean dead.** Routes are wired
only from `src.domains.*`, so grepping `main.py`'s imports looks like it settles which
packages are live. It does not. `main.py:55` calls `initialize_oidc()`, and that function
(`src/shared/security.py:21`) deliberately imports and configures **both** `src.auth.oidc`
and `src.domains.auth.oidc` — a function-body import, invisible to a grep of `main.py`.
That one call drags in `src/auth/`, `src/controllers/`, `src/db/`, `src/logging_config.py`
and `src/base_schema.py` at startup.
Three tiers, established by importing the app inside the container and reading
`sys.modules` (verified 2026-08-09):
| Tier | Packages |
|---|---|
| Serving routes | `src/domains/`, `src/shared/`, `src/service_groups/` |
| **Loaded and configured**, but serving no routes | `src/auth/`, `src/db/`, `src/controllers/`, `src/logging_config.py`, `src/base_schema.py` |
| Genuinely unreferenced | `src/agent/`, `src/api/`, `src/clients/`, `src/dns/`, `src/memory/`, `src/models/` |
`src/auth/` is the trap. Its `oidc_config` singleton is configured at every startup with
the real Authentik issuers — the log line `src.auth.oidc:configure` proves it — so a test
importing `src.auth.oidc` is exercising live, configured code, not a fossil. No `src/domains/*`
module depends on it, so it is configured defensively rather than used; that makes it a
deletion candidate, but a considered one, not obvious cleanup.
**Before deleting anything from `src/`, import the app and read `sys.modules`** rather than
grepping `main.py`. Function-body imports exist here specifically to dodge circular imports,
and they are exactly what a grep misses.
**That check has its own blind spot, so do not read the third tier as a delete list.** The
table above is a snapshot taken after a cold `import src.main` — it shows what *startup*
loads. A module imported inside a request handler would be absent from it while being
entirely live, and absence would then be a timing artifact rather than evidence of death.
This bit on webber, where a tool package imported from inside an agent method looked
unloaded and was serving every request. Nothing in core-api is currently known to work that
way, but that is the weaker claim — it means nobody has exercised the routes and re-checked,
not that nobody does it. Before deleting a third-tier package, drive the endpoints that
would plausibly load it and take the snapshot again.
Some tests (`test_auth_controller.py`, `test_oidc.py`, `test_npm_client.py`,
`test_portainer_client.py`, `test_static_controller.py`, `test_tools_controller.py`) import
from the top-level paths rather than `src.domains.*`. Which of those cover live code follows
the table above — `test_oidc.py` does; the client tests target the unreferenced tier. Not
cleaned up in this pass; flagged, not fixed.
Shared infra (config, database, logging, security/OIDC, external API clients) lives in
`src/shared/`.
Group new work by **domain, not by file type** — a single large `routers/` folder is
the thing to avoid. Reference: [FastAPI best practices](https://github.com/zhanymkanov/fastapi-best-practices).
## Database
SQLAlchemy 2.0 async + asyncpg, migrations via Alembic (`alembic/versions/`). Models
live under `src/domains/<name>/models.py` and must be imported in `alembic/env.py` to
register with `Base.metadata` — check that file when adding a new model or `alembic
revision --autogenerate` will silently miss it.
## Working here
**Plan, act, reflect.** Outline which files you will touch and the side effects before
writing. Change in small atomic steps. Afterwards, verify: did existing tests break, and
does the new behaviour have a test?
**Test locally first — the build-deploy loop is slow.** `./wakeup.sh` auto-reloads on
code changes (not on `requirements.txt` changes; restart the container/script after
adding a dependency). Deploy only when a feature is complete and tested.
Run tests through the venv explicitly, to avoid environment mismatch:
```bash
.venv/bin/python -m pytest tests/
# or, with coverage:
.venv/bin/python -m pytest --cov=src --cov-report=term-missing
```
Copy `.env.example` to `.env` and configure Portainer, NPM, Home Assistant, SearXNG,
Postgres, and Qdrant hosts/credentials.
No linter is configured in this repo (no ruff/flake8 config, none in `requirements.txt`
or `dev-requirements.txt`) — unlike some sibling repos, don't assume `ruff check` exists
here.
## CI
`.gitea/workflows/build.yml` is the only workflow: triggered on `v*` tag push, it
creates a Gitea release, builds and pushes the image, then pings Watchtower. There is
**no CI test/lint gate** — pytest only runs locally or on request. Verify tests pass
before tagging a release.
## Work tracking
Work lives in **pql**, not a markdown TODO. **This repo's vault is standalone** — its tickets
and its internal decisions live here in `.pql/` and `governance/`, and travel with a clone,
because `.pql/changelog/` is committed and replayed by the git hooks (workspace D-15). The databases are
gitignored and rebuildable with `pql plan rebuild`.
`pql` is **not** on the non-interactive `PATH` — invoke it as
`/home/jpmschweitzer/.local/bin/pql`. From inside this repo no `--vault` is needed: pql anchors
at the nearest `.git/` ancestor, which is this repo.
```bash
/home/jpmschweitzer/.local/bin/pql ticket list # this repo's open work
/home/jpmschweitzer/.local/bin/pql plan whatsnext # next unblocked item, with context
/home/jpmschweitzer/.local/bin/pql decisions list # this repo's own decisions
```
Stack-level decisions that constrain this service live in the **workspace** vault and need the
flag:
```bash
/home/jpmschweitzer/.local/bin/pql --vault /mnt/media/Projects decisions list --domain core-api
```
Note `ticket new --decision D-N` resolves ids within **one** vault, so a ticket here cannot link
to a workspace decision. Cite the id in the ticket body instead.
Do not add a TODO section to a markdown file.
## Git
- **History is linear — no merge commits.** Work on `main`, or a short-lived branch
that is fast-forwarded and deleted. (This repo's AGENTS.md previously mandated a
feature branch for every change; that rule was retired workspace-wide on 2026-08-08
and does not apply here anymore.)
- **Conventional Commits**: `feat:`, `fix:`, `refactor:`, `docs:`, `chore:`.
- **Atomic commits** — one logical change each.
- **Stage explicitly. Never `git add -A`** — it is denied by policy, and it sweeps in
whatever else is dirty, including secrets.
- Update `CHANGELOG.md` with every user-facing change, under `[Unreleased]` in `Added` /
`Changed` / `Fixed`.
## Releasing
Ask whether a deploy is wanted first — it is not automatic.
1. Bump the version in `pyproject.toml` (patch for fixes, minor for features).
2. Move `[Unreleased]` entries into a dated version section in `CHANGELOG.md`.
3. Stage the changed files by name, commit, tag `vX.Y.Z`, `git push origin main --tags`.
4. Gitea CI (`build.yml`) builds and pushes the image on the tag; Watchtower deploys it.
5. Verify: `curl http://192.168.86.149:8083/health`.
## Security
- OIDC authentication via Authentik, multi-issuer/multi-audience support.
- Admin endpoints require authentication when `OIDC_ENABLED=true`.
- README claims the container "runs as non-root user (uid 1000)" — **checked and
false**: the Dockerfile has no `USER` directive, so the container runs as root.
Not fixed here (out of scope for a docs normalization pass); flagging so it isn't
restated as fact.
+67
View File
@@ -0,0 +1,67 @@
# core-api — the repo's command surface (D-27).
#
# Paths resolve here rather than in callers (D-10). `python3` on this host is
# 3.8 and cannot parse these sources, and a bare `pytest` resolves only in a
# login shell — so both are named explicitly through the venv.
VENV := $(CURDIR)/.venv
PYTHON ?= python3.12
.DEFAULT_GOAL := help
.PHONY: help
help: ## Show this help
@grep -hE '^[a-z][a-z0-9_-]*:.*?## ' $(MAKEFILE_LIST) \
| awk 'BEGIN{FS=":.*?## "}{printf " \033[36m%-14s\033[0m %s\n", $$1, $$2}'
.PHONY: setup
setup: ## Create the venv, install dependencies, and prove the result is usable
$(PYTHON) -m venv .venv
$(VENV)/bin/pip install -r requirements.txt -r dev-requirements.txt
# Exit 0 from pip install is not evidence (D-24) — it is the step's job not
# to fail, so a broken result and a working one look identical from here.
# On 2026-08-09 the venv existed and pip had exited 0, but sqlalchemy was
# declared in requirements.txt and not installed; that surfaced as 11
# pytest collection errors that read as broken imports, not as a setup
# problem. `--collect-only` exercises every import the suite touches
# without running a single test, so it catches exactly that class of
# drift and stays cheap. `pip check` was considered too, but it only
# verifies the *installed* set's internal consistency against itself —
# it would not have caught this case, because sqlalchemy was still
# present as another package's transitive dependency even when dropped
# from requirements.txt. collect-only checks declared-vs-actually-usable
# directly, which is the axis that broke.
$(VENV)/bin/python -m pytest --collect-only tests/
.PHONY: test
test: ## Run the test suite
@test -x $(VENV)/bin/python || { echo "FAIL — no venv; run: make setup"; exit 69; }
$(VENV)/bin/python -m pytest tests/
# No `lint` target, deliberately. This repo configures no linter — no ruff or
# flake8 config, and neither in requirements. Per D-27 the name is reserved for
# repos that lint; an empty target here would report clean for something never
# run. Add the target when a linter is added, not before.
# git hands a hook a non-login shell, which never sees ~/.local/bin — where
# gitleaks lands. Without this the scan reports "not installed" on every push,
# which is a check that fails open (D-24).
export PATH := $(HOME)/.local/bin:/usr/local/bin:$(PATH)
.PHONY: secrets
secrets: ## Scan the commits about to be pushed for credentials
@ci/secrets.sh
# The call surface is identical in every repo; what it runs is not.
#
# `secrets` runs first, deliberately: it is the only failure here that cannot be
# undone by fixing it afterwards. A failed lint costs another commit; a pushed
# credential is cached and indexed whether or not it is later deleted.
#
# Some of these fail today, and are left wired anyway. The state was measured
# once and written down in T-56 rather than being worked around here — a gate
# quietly narrowed to what already passes is a gate that reports success for
# doing nothing, which is the failure this workspace keeps rediscovering.
.PHONY: pre-push
pre-push: secrets ## Everything the pre-push hook runs
@echo " -- not gated here yet: lint (no linter configured) and test (T-56)"
+6 -17
View File
@@ -19,8 +19,7 @@ Central API service providing infrastructure management, home automation, and ut
### Utilities
- **DNS Lookup**: Query DNS records (A, AAAA, MX, TXT, CNAME, NS, SOA, PTR)
- **Health Checks**: Comprehensive service health monitoring
- **AI Metrics Proxy**: Forward metrics requests to Core-AI service
- **Health Checks**: Service health monitoring with database connectivity status
## Architecture
@@ -35,10 +34,8 @@ src/
├── clients/
│ ├── homeassistant_client.py # Home Assistant REST client
│ ├── npm_client.py # Nginx Proxy Manager client
│ ├── ollama_client.py # Ollama LLM client
│ └── portainer_client.py # Portainer API client
├── controllers/
│ ├── ai_controller.py # AI metrics proxy
│ ├── health_controller.py # Health endpoints
│ ├── housekeeping_controller.py # Home automation endpoints
│ ├── infrastructure_controller.py # Infrastructure management
@@ -88,9 +85,6 @@ src/
### Tools (`/tools`)
- `POST /tools/dns/lookup` - DNS record lookup
### AI (`/ai`)
- `GET /ai/metrics` - Proxy to Core-AI metrics
## Development
### Requirements
@@ -103,10 +97,6 @@ src/
# Install dependencies
pip install -r requirements.txt
# Copy credentials template
cp src/credentials.example.py src/credentials.py
# Edit src/credentials.py with your values
# Run locally
uvicorn src.main:app --reload --host 0.0.0.0 --port 8083
```
@@ -169,10 +159,9 @@ docker run -p 8083:8083 core-code:latest
| `NPM_PASSWORD` | NPM admin password | - |
| `HOMEASSISTANT_URL` | Home Assistant URL | `http://localhost:8123` |
| `HOMEASSISTANT_TOKEN` | HA long-lived access token | - |
| `OLLAMA_URL` | Ollama API URL | `http://localhost:11434` |
| `OIDC_ENABLED` | Enable OIDC auth | `false` |
| `OIDC_ISSUER` | OIDC issuer URL | - |
| `OIDC_AUDIENCE` | OIDC audience | - |
| `OIDC_ISSUERS` | OIDC issuer URLs (comma-separated) | See config.py |
| `OIDC_AUDIENCES` | OIDC audiences (comma-separated) | See config.py |
## API Documentation
@@ -183,9 +172,9 @@ Once deployed, access documentation at:
## Health Checks
- **Basic**: `GET /health` - Returns status and Ollama connection
- **Full**: `GET /health/full` - Returns all component statuses (503 if unhealthy)
- **Diagnostics**: `GET /health/diagnostics` - Detailed service information
- **Basic**: `GET /health` - Fast liveness check for container orchestration
- **Full**: `GET /health/full` - Returns database status (503 if unhealthy)
- **Diagnostics**: `GET /health/diagnostics` - Service info and configuration
## Security
+7
View File
@@ -19,6 +19,13 @@ from src.shared.database import Base
# Import all models to ensure they're registered with Base.metadata
from src.domains.auth.models import User, Role, UserRole, UserPreferences, ApiKey # noqa: F401
from src.domains.dashboard.models import QuickLink, DashboardWidget # noqa: F401
from src.domains.book.models import ( # noqa: F401
BookRecord, BookTag, BookRecordLink, BookAttachment, BookEvent, BookTenantCounter,
)
from src.domains.scribe.models import ( # noqa: F401
ScribeAddress, ScribeMessage, ScribeMailboxEntry,
)
from src.domains.tenants.models import Tenant, TenantConsumer # noqa: F401
# Alembic Config object
config = context.config
@@ -0,0 +1,141 @@
"""Add group_roles mapping and update role schema
Revision ID: 004
Revises: f0349c95aa5d
Create Date: 2026-01-03
Changes:
- Add category column to roles (default 'general')
- Drop authentik_group column from roles (decoupled architecture)
- Create user_groups association table
- Create group_roles association table
- Update role names from domain:action to domain.general:action
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql
# revision identifiers, used by Alembic.
revision: str = "004"
down_revision: Union[str, None] = "f0349c95aa5d"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Add category column to roles
op.add_column(
"roles",
sa.Column(
"category",
sa.String(50),
nullable=False,
server_default="general",
comment="Permission category within domain (general for full access, or specific tool)",
),
)
# Update role names from domain:action to domain.general:action
op.execute(
"""
UPDATE roles
SET name = REPLACE(name, ':', '.general:')
WHERE name NOT LIKE '%.%:%'
"""
)
# Update the comment on the name column
op.alter_column(
"roles",
"name",
comment="Role name in format domain.category:action (e.g., control-room.general:admin)",
)
# Drop the authentik_group unique index first
op.drop_index("ix_roles_authentik_group", table_name="roles")
# Drop authentik_group column (no longer needed with group_roles mapping)
op.drop_column("roles", "authentik_group")
# Create user_groups association table
op.create_table(
"user_groups",
sa.Column(
"user_id",
postgresql.UUID(as_uuid=True),
sa.ForeignKey("users.id", ondelete="CASCADE"),
primary_key=True,
),
sa.Column(
"group_id",
postgresql.UUID(as_uuid=True),
sa.ForeignKey("groups.id", ondelete="CASCADE"),
primary_key=True,
),
)
# Create group_roles association table
op.create_table(
"group_roles",
sa.Column(
"group_id",
postgresql.UUID(as_uuid=True),
sa.ForeignKey("groups.id", ondelete="CASCADE"),
primary_key=True,
),
sa.Column(
"role_id",
postgresql.UUID(as_uuid=True),
sa.ForeignKey("roles.id", ondelete="CASCADE"),
primary_key=True,
),
)
def downgrade() -> None:
# Drop association tables
op.drop_table("group_roles")
op.drop_table("user_groups")
# Add back authentik_group column
op.add_column(
"roles",
sa.Column(
"authentik_group",
sa.String(255),
nullable=True,
comment="Corresponding Authentik group name",
),
)
# Restore authentik_group values from role names
op.execute(
"""
UPDATE roles
SET authentik_group = 'tatlock-' || REPLACE(REPLACE(name, '.general:', '-'), ':', '-')
"""
)
# Recreate the unique index
op.create_index("ix_roles_authentik_group", "roles", ["authentik_group"], unique=True)
# Revert role names from domain.general:action to domain:action
op.execute(
"""
UPDATE roles
SET name = REPLACE(name, '.general:', ':')
WHERE name LIKE '%.general:%'
"""
)
# Update the comment on the name column
op.alter_column(
"roles",
"name",
comment="Role name in format domain:action",
)
# Drop category column
op.drop_column("roles", "category")
@@ -0,0 +1,198 @@
"""Create Book tables (T-1, D-68/D-69/D-70; tenant/number amended by T-7)
Revision ID: 005
Revises: 004
Create Date: 2026-10-19
The whole Book schema lands in this one migration — book_records,
book_tags, book_record_links, book_attachments, book_events,
book_tenant_counters — so later rounds (append, remind, reschedule,
pause/resume, redact, ...) need no migration of their own for these tables.
book_events is append-only: a trigger raises on UPDATE and DELETE, for
every role including the owning application. `redact` (D-70 rule 4) is the
one sanctioned exception and is a later round; this migration does not
soften the trigger.
core-api T-7: amended before 005's first release (production was still at
004) to add book_records.owner_tenant/number and book_tenant_counters,
rather than patching them in with a later migration — see T-7's dossier.
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects.postgresql import JSONB, TSVECTOR, UUID
# revision identifiers, used by Alembic.
revision: str = "005"
down_revision: Union[str, None] = "004"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""Create the Book tables, its search column/index, and the append-only trigger."""
op.create_table(
"book_records",
sa.Column("id", UUID(as_uuid=True), nullable=False),
sa.Column("owner_tenant", sa.String(length=63), nullable=False),
sa.Column("number", sa.Integer(), nullable=False),
sa.Column("kind", sa.String(length=20), nullable=False),
sa.Column("title", sa.String(length=500), nullable=True),
sa.Column("body", sa.Text(), nullable=False, server_default=""),
sa.Column("checklist", JSONB(), nullable=False, server_default=sa.text("'[]'::jsonb")),
sa.Column("status", sa.String(length=20), nullable=False),
sa.Column("scope", sa.String(length=10), nullable=False, server_default="personal"),
sa.Column("priority", sa.Integer(), nullable=True),
sa.Column("pinned", sa.Boolean(), nullable=False, server_default=sa.false()),
sa.Column("energy", sa.String(length=10), nullable=True),
sa.Column("estimate_minutes", sa.Integer(), nullable=True),
sa.Column("next_step", sa.Text(), nullable=True),
sa.Column("start_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("due_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("all_day", sa.Boolean(), nullable=False, server_default=sa.false()),
sa.Column("remind_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("recurrence", sa.String(length=255), nullable=True),
sa.Column("recurrence_mode", sa.String(length=20), nullable=True),
sa.Column("remind_to", sa.String(length=63), nullable=True),
sa.Column("last_touched_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column("archived_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("trashed_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("paused_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("paused_statuses", JSONB(), nullable=True),
sa.Column("source", sa.String(length=50), nullable=True),
sa.Column("external_id", sa.String(length=255), nullable=True),
sa.Column("colour", sa.String(length=20), nullable=True),
sa.Column("created_by", sa.String(length=63), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.UniqueConstraint("source", "external_id", name="uq_book_records_source_external_id"),
sa.UniqueConstraint("owner_tenant", "number", name="uq_book_records_tenant_number"),
)
op.create_index("ix_book_records_kind", "book_records", ["kind"])
op.create_index("ix_book_records_status", "book_records", ["status"])
op.create_index("ix_book_records_due_at", "book_records", ["due_at"])
op.create_index(
"ix_book_records_tenant_kind_status",
"book_records",
["owner_tenant", "kind", "status"],
)
# Generated column — 'simple' because the household writes mixed Dutch
# and English, and a language-specific config would stem one of the two
# badly. Added via raw SQL: Alembic's op.add_column has no first-class
# GENERATED ALWAYS AS support.
op.execute(
"""
ALTER TABLE book_records
ADD COLUMN search tsvector
GENERATED ALWAYS AS (to_tsvector('simple', coalesce(title, '') || ' ' || body)) STORED
"""
)
op.create_index(
"ix_book_records_search", "book_records", ["search"], postgresql_using="gin"
)
# One row per tenant: the next `number` to allocate on book_records.
# Locked FOR UPDATE in the same transaction as the record insert it
# numbers (BookService._allocate_number) — see that method's docstring
# for why this shape is safe under concurrency.
op.create_table(
"book_tenant_counters",
sa.Column("owner_tenant", sa.String(length=63), nullable=False),
sa.Column("next_number", sa.Integer(), nullable=False, server_default="1"),
sa.PrimaryKeyConstraint("owner_tenant"),
)
op.create_table(
"book_tags",
sa.Column("record_id", UUID(as_uuid=True), sa.ForeignKey("book_records.id", ondelete="CASCADE"), nullable=False),
sa.Column("tag", sa.String(length=63), nullable=False),
sa.PrimaryKeyConstraint("record_id", "tag"),
)
op.create_index("ix_book_tags_tag", "book_tags", ["tag"])
op.create_table(
"book_record_links",
sa.Column("from_id", UUID(as_uuid=True), sa.ForeignKey("book_records.id", ondelete="CASCADE"), nullable=False),
sa.Column("relation", sa.String(length=20), nullable=False),
sa.Column("to_id", UUID(as_uuid=True), sa.ForeignKey("book_records.id", ondelete="CASCADE"), nullable=False),
sa.Column("created_by", sa.String(length=63), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.PrimaryKeyConstraint("from_id", "relation", "to_id"),
)
op.create_index("ix_book_record_links_to_id", "book_record_links", ["to_id"])
op.create_table(
"book_attachments",
sa.Column("id", UUID(as_uuid=True), nullable=False),
sa.Column("record_id", UUID(as_uuid=True), sa.ForeignKey("book_records.id", ondelete="CASCADE"), nullable=False),
sa.Column("path", sa.String(length=1000), nullable=False),
sa.Column("mime", sa.String(length=255), nullable=True),
sa.Column("size", sa.Integer(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.PrimaryKeyConstraint("id"),
)
op.create_index("ix_book_attachments_record_id", "book_attachments", ["record_id"])
op.create_table(
"book_events",
sa.Column("id", UUID(as_uuid=True), nullable=False),
sa.Column("record_id", UUID(as_uuid=True), sa.ForeignKey("book_records.id", ondelete="CASCADE"), nullable=False),
sa.Column("at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column("identity", sa.String(length=63), nullable=False),
sa.Column("client", sa.String(length=63), nullable=True),
sa.Column("action", sa.String(length=20), nullable=False),
sa.Column("diff", JSONB(), nullable=False, server_default=sa.text("'{}'::jsonb")),
sa.PrimaryKeyConstraint("id"),
)
op.create_index("ix_book_events_record_id", "book_events", ["record_id"])
# Append-only, for every role including the owning application.
op.execute(
"""
CREATE FUNCTION book_events_append_only() RETURNS trigger AS $$
BEGIN
RAISE EXCEPTION 'book_events is append-only: % not allowed', TG_OP;
END;
$$ LANGUAGE plpgsql
"""
)
op.execute(
"""
CREATE TRIGGER book_events_no_update_delete
BEFORE UPDATE OR DELETE ON book_events
FOR EACH ROW EXECUTE FUNCTION book_events_append_only()
"""
)
def downgrade() -> None:
"""Drop the Book tables, in dependency order."""
op.execute("DROP TRIGGER IF EXISTS book_events_no_update_delete ON book_events")
op.execute("DROP FUNCTION IF EXISTS book_events_append_only()")
op.drop_index("ix_book_events_record_id", table_name="book_events")
op.drop_table("book_events")
op.drop_index("ix_book_attachments_record_id", table_name="book_attachments")
op.drop_table("book_attachments")
op.drop_index("ix_book_record_links_to_id", table_name="book_record_links")
op.drop_table("book_record_links")
op.drop_index("ix_book_tags_tag", table_name="book_tags")
op.drop_table("book_tags")
op.drop_table("book_tenant_counters")
op.drop_index("ix_book_records_search", table_name="book_records")
op.drop_index("ix_book_records_tenant_kind_status", table_name="book_records")
op.drop_index("ix_book_records_due_at", table_name="book_records")
op.drop_index("ix_book_records_status", table_name="book_records")
op.drop_index("ix_book_records_kind", table_name="book_records")
op.drop_table("book_records")
@@ -0,0 +1,113 @@
"""Create Scribe tables (dossier core-api T-5, D-66/D-68/D-73)
Revision ID: 006
Revises: 005
Create Date: 2026-10-19
scribe_addresses, scribe_messages, scribe_mailbox_entries — messaging only
(no groups, subscriptions, notification matrix or delivery this round;
workspace T-176). No data: seeding `jeroen` and `tower-control` is a
deploy step through `register`, not this migration.
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects.postgresql import ARRAY, TSVECTOR
# revision identifiers, used by Alembic.
revision: str = "006"
down_revision: Union[str, None] = "005"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""Create the Scribe tables, their search/topics GIN indexes."""
op.create_table(
"scribe_addresses",
sa.Column("owner_tenant", sa.String(length=63), nullable=False),
sa.Column("name", sa.String(length=63), nullable=False),
sa.Column("owner_consumer", sa.String(length=63), nullable=True),
sa.Column("created_by", sa.String(length=63), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.PrimaryKeyConstraint("owner_tenant", "name"),
sa.UniqueConstraint("owner_tenant", "name", name="uq_scribe_addresses_tenant_name"),
)
op.create_table(
"scribe_messages",
sa.Column("id", sa.Integer(), sa.Identity(always=False), nullable=False),
sa.Column("owner_tenant", sa.String(length=63), nullable=False),
sa.Column("topics", ARRAY(sa.Text()), nullable=False, server_default=sa.text("'{}'::text[]")),
sa.Column("from_identity", sa.String(length=63), nullable=False),
sa.Column("from_consumer", sa.String(length=63), nullable=True),
sa.Column("subject", sa.String(length=500), nullable=True),
sa.Column("body", sa.Text(), nullable=False),
sa.Column("tags", ARRAY(sa.Text()), nullable=False, server_default=sa.text("'{}'::text[]")),
sa.Column("reply_to", sa.Integer(), nullable=True),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.ForeignKeyConstraint(["reply_to"], ["scribe_messages.id"], name="fk_scribe_messages_reply_to"),
)
op.create_index("ix_scribe_messages_owner_tenant", "scribe_messages", ["owner_tenant"])
op.create_index(
"ix_scribe_messages_topics", "scribe_messages", ["topics"], postgresql_using="gin"
)
# Generated column — 'simple' because the household writes mixed Dutch
# and English, matching book_records.search (migration 005). Added via
# raw SQL: Alembic's op.add_column has no first-class GENERATED ALWAYS
# AS support.
op.execute(
"""
ALTER TABLE scribe_messages
ADD COLUMN search tsvector
GENERATED ALWAYS AS (to_tsvector('simple', coalesce(subject, '') || ' ' || body)) STORED
"""
)
op.create_index(
"ix_scribe_messages_search", "scribe_messages", ["search"], postgresql_using="gin"
)
op.create_table(
"scribe_mailbox_entries",
sa.Column("id", sa.Integer(), sa.Identity(always=False), nullable=False),
sa.Column("address", sa.String(length=63), nullable=False),
sa.Column("owner_tenant", sa.String(length=63), nullable=False),
sa.Column("message_id", sa.Integer(), nullable=False),
sa.Column("read_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("tags", ARRAY(sa.Text()), nullable=False, server_default=sa.text("'{}'::text[]")),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.ForeignKeyConstraint(
["message_id"], ["scribe_messages.id"], name="fk_scribe_mailbox_entries_message_id"
),
sa.UniqueConstraint("address", "message_id", name="uq_scribe_mailbox_entries_address_message"),
)
op.create_index(
"ix_scribe_mailbox_entries_owner_tenant_address",
"scribe_mailbox_entries",
["owner_tenant", "address"],
)
op.create_index(
"ix_scribe_mailbox_entries_message_id", "scribe_mailbox_entries", ["message_id"]
)
def downgrade() -> None:
"""Drop the Scribe tables, in dependency order."""
op.drop_index("ix_scribe_mailbox_entries_message_id", table_name="scribe_mailbox_entries")
op.drop_index(
"ix_scribe_mailbox_entries_owner_tenant_address", table_name="scribe_mailbox_entries"
)
op.drop_table("scribe_mailbox_entries")
op.drop_index("ix_scribe_messages_search", table_name="scribe_messages")
op.drop_index("ix_scribe_messages_topics", table_name="scribe_messages")
op.drop_index("ix_scribe_messages_owner_tenant", table_name="scribe_messages")
op.drop_table("scribe_messages")
op.drop_table("scribe_addresses")
@@ -0,0 +1,97 @@
"""Create the tenant registry (core-api T-8, workspace D-73)
Revision ID: 007
Revises: 006
Create Date: 2026-10-19
`tenants` and `tenant_consumers` — the registry every Book and Scribe
`owner_tenant` column now conforms to via a foreign key (`ON DELETE
RESTRICT`: a tenant with any Book or Scribe row can never be dropped out
from under it). Seeds the two tenants in use today — `jpmschweitzer`
(household, "Jeroen") and `llm_tester` (test, "Playground") — each granted
to `office-claude`, the only Office consumer configured in Authentik
(dossier core-api T-8). No write routes ship this round; provisioning a
third tenant is later (workspace T-195) and happens by hand until then.
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = "007"
down_revision: Union[str, None] = "006"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
# Every existing owner_tenant column (core-api T-8's dossier: "five
# owner_tenant columns exist ... find them by grep rather than trusting
# this count" — grepped and confirmed at five, here).
OWNER_TENANT_FKS = (
("book_records", "fk_book_records_owner_tenant"),
("book_tenant_counters", "fk_book_tenant_counters_owner_tenant"),
("scribe_addresses", "fk_scribe_addresses_owner_tenant"),
("scribe_messages", "fk_scribe_messages_owner_tenant"),
("scribe_mailbox_entries", "fk_scribe_mailbox_entries_owner_tenant"),
)
def upgrade() -> None:
"""Create the tenant registry, seed it, then bind every existing
owner_tenant column to it."""
op.create_table(
"tenants",
sa.Column("id", sa.String(length=63), nullable=False),
sa.Column("display_name", sa.Text(), nullable=False),
sa.Column("kind", sa.String(length=20), nullable=False),
sa.Column("status", sa.String(length=20), nullable=False, server_default="active"),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column("updated_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.PrimaryKeyConstraint("id"),
sa.CheckConstraint("kind IN ('household', 'person', 'test')", name="ck_tenants_kind"),
sa.CheckConstraint("status IN ('active', 'suspended')", name="ck_tenants_status"),
)
op.create_table(
"tenant_consumers",
sa.Column("tenant_id", sa.String(length=63), nullable=False),
sa.Column("consumer", sa.String(length=63), nullable=False),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.PrimaryKeyConstraint("tenant_id", "consumer"),
sa.ForeignKeyConstraint(["tenant_id"], ["tenants.id"], name="fk_tenant_consumers_tenant_id"),
)
# Seed: the two tenants in use today, both granted to office-claude —
# the only Office consumer configured in Authentik (dossier T-8).
op.execute(
"""
INSERT INTO tenants (id, display_name, kind, status) VALUES
('jpmschweitzer', 'Jeroen', 'household', 'active'),
('llm_tester', 'Playground', 'test', 'active')
"""
)
op.execute(
"""
INSERT INTO tenant_consumers (tenant_id, consumer) VALUES
('jpmschweitzer', 'office-claude'),
('llm_tester', 'office-claude')
"""
)
# Every existing owner_tenant column now conforms to the registry.
for table, fk_name in OWNER_TENANT_FKS:
op.create_foreign_key(
fk_name, table, "tenants", ["owner_tenant"], ["id"], ondelete="RESTRICT"
)
def downgrade() -> None:
"""Unbind every owner_tenant column, then drop the registry."""
for table, fk_name in OWNER_TENANT_FKS:
op.drop_constraint(fk_name, table, type_="foreignkey")
op.drop_table("tenant_consumers")
op.drop_table("tenants")
@@ -0,0 +1,77 @@
"""Add a default address to each consumer grant (core-api T-11)
Revision ID: 008
Revises: 007
Create Date: 2026-10-19
claude.ai's connector keeps no MCP session between tool calls, so the
Office cannot remember which address a conversation chose. Each consumer
grant now carries `default_address`: the Office uses it when a call names
none and has no session binding (tatlock-office T-5 reads it).
`tenant_consumers.default_address` is nullable, and a composite foreign
key `(tenant_id, default_address)` -> `scribe_addresses(owner_tenant,
name)` ensures a default always names a real address in the same tenant.
Postgres's default `MATCH SIMPLE` means a NULL `default_address` never
triggers the check, so an ungranted default is simply absent rather than
a constraint violation waiting to happen.
Data step: `jpmschweitzer`/`office-claude` defaults to `claude-ai` only if
that address already exists at upgrade time - true in production, false
in `core_api_test` (no scribe_addresses are seeded there), so the `UPDATE
... WHERE EXISTS` must be - and is - a no-op rather than a failure when
the address is absent. `llm_tester`/`office-claude` is not touched and
stays null.
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = "008"
down_revision: Union[str, None] = "007"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""Add the column, bind it to scribe_addresses, then seed the one
production default the dossier specifies."""
op.add_column(
"tenant_consumers",
sa.Column("default_address", sa.String(length=63), nullable=True),
)
op.create_foreign_key(
"fk_tenant_consumers_default_address",
"tenant_consumers",
"scribe_addresses",
["tenant_id", "default_address"],
["owner_tenant", "name"],
)
# Only if claude-ai already exists in jpmschweitzer (production: yes;
# core_api_test: no) - an UPDATE ... WHERE EXISTS is a no-op rather
# than a failure when it does not.
op.execute(
"""
UPDATE tenant_consumers
SET default_address = 'claude-ai'
WHERE tenant_id = 'jpmschweitzer'
AND consumer = 'office-claude'
AND EXISTS (
SELECT 1 FROM scribe_addresses
WHERE owner_tenant = 'jpmschweitzer' AND name = 'claude-ai'
)
"""
)
def downgrade() -> None:
"""Unbind and drop the column."""
op.drop_constraint(
"fk_tenant_consumers_default_address", "tenant_consumers", type_="foreignkey"
)
op.drop_column("tenant_consumers", "default_address")
@@ -0,0 +1,45 @@
"""Archive and trash a message in your own mailbox (core-api T-13)
Revision ID: 009
Revises: 008
Create Date: 2026-10-19
Deletion is per mailbox entry, never per message (workspace D-66/D-70): a
message is retained and shared by every recipient, so only the entry that
represents one recipient's copy can be archived or trashed. `archived_at`
and `trashed_at` are independent nullable timestamps on
`scribe_mailbox_entries`, mirroring `book_records.archived_at`/
`trashed_at` (migration 005) — the service layer's `view=trash` takes
priority over `view=archive` when both are set, so the column pair alone
does not encode that precedence.
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = "009"
down_revision: Union[str, None] = "008"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""Add the two nullable timestamp columns. No data step: every existing
entry is neither archived nor trashed."""
op.add_column(
"scribe_mailbox_entries",
sa.Column("archived_at", sa.DateTime(timezone=True), nullable=True),
)
op.add_column(
"scribe_mailbox_entries",
sa.Column("trashed_at", sa.DateTime(timezone=True), nullable=True),
)
def downgrade() -> None:
"""Drop both columns."""
op.drop_column("scribe_mailbox_entries", "trashed_at")
op.drop_column("scribe_mailbox_entries", "archived_at")
Executable
+50
View File
@@ -0,0 +1,50 @@
#!/usr/bin/env bash
# Secret scan over the commits about to be pushed.
#
# Lives here rather than inside .githooks/pre-push so it can be read, run by
# hand (`make secrets`), and changed under review. A hook is a trigger; it is
# not a home for logic. Identical in every repo in this workspace (D-27).
set -euo pipefail
cd "$(git rev-parse --show-toplevel)"
# A non-login shell — which is what git gives a hook — skips /etc/profile.d
# and never sees ~/.local/bin, where the gitleaks release tarball lands.
# Without this the scan reports "not installed" on every push.
[ -d "$HOME/.local/bin" ] && PATH="$HOME/.local/bin:$PATH"
if ! command -v gitleaks >/dev/null 2>&1; then
echo "FAIL secrets — gitleaks not installed, so this check would be a no-op pretending to pass." >&2
echo " https://github.com/gitleaks/gitleaks/releases → ~/.local/bin/gitleaks" >&2
exit 1
fi
# Scan the outgoing range, not full history. History here carries findings
# that are settled — test fixtures and vendored third-party code — and a gate
# that fails on something unfixable gets bypassed within a week. What matters
# is what is about to leave this machine.
if upstream=$(git rev-parse --abbrev-ref --symbolic-full-name '@{u}' 2>/dev/null); then
range="$upstream..HEAD"
elif git rev-parse --verify --quiet origin/main >/dev/null; then
range="origin/main..HEAD"
else
range=""
fi
if [ -z "$range" ]; then
gitleaks dir . --redact --no-banner --exit-code 1 || {
echo "FAIL secrets — gitleaks found a credential in the working tree." >&2; exit 1; }
exit 0
fi
[ -n "$(git log --oneline "$range" 2>/dev/null)" ] || exit 0
gitleaks git . --log-opts="$range" --redact --no-banner --exit-code 1 >/dev/null 2>&1 || {
echo "FAIL secrets — gitleaks found a credential in the commits being pushed." >&2
echo " inspect (values redacted): gitleaks git . --log-opts=\"$range\" --redact" >&2
echo " then remove and rotate it, or suppress deliberately:" >&2
echo " inline '# gitleaks:allow <reason>'" >&2
echo " or add the fingerprint to .gitleaksignore WITH a reason" >&2
exit 1
}
echo " ok secrets"
+54
View File
@@ -0,0 +1,54 @@
# Decisions, Questions, Rejected
This directory holds structured planning records that pql parses
into pql.db. Each record is a `### [DQR]-N: Title` heading inside
a markdown file. Files live in three per-type subdirectories:
- `decisions/<domain>.md` — confirmed design decisions
- `questions/<domain>.md` — open questions that may resolve into
decisions or rejected proposals
- `rejected/<domain>.md` — rejected proposals (kept for the audit
trail)
The parser infers domain from the filename stem and record type
from the parent subdirectory.
D-records that propose implementation work link to `initiative`-type
tickets via `decision_ref`. Run `pql decisions show <id>
--with-tickets` to inspect implementation status.
## Recommended domains
Start with this canonical set; create files as records land in
each domain:
- **architecture** — structural commitments (storage, layering,
languages, libraries)
- **process** — team workflow (commits, branches, releases, reviews)
- **design** — user-facing surface (UX, UI, public APIs)
- **coding-conventions** — team-internal code shape (style, lint,
file layout)
- **testing** — quality strategy (coverage, layers, gates)
You might also want, project-permitting:
- `accessibility` — if you ship user-facing software
- `security` — if you handle user data or network surfaces
- `licensing` — if you release open-source or commercial
- `documentation` — if user-docs are non-trivial
- `deployment` — if shipping is non-trivial
- `performance` — if you have perf budgets / SLOs
<!-- pql:records (auto-generated; do not edit manually) -->
## Decisions
- _(none)_
## Open questions
- _(none)_
## Rejected
- _(none)_
+12
View File
@@ -0,0 +1,12 @@
proxy_buffers 8 16k;
proxy_buffer_size 32k;
# CORS headers for Flutter web
add_header Access-Control-Allow-Origin "https://home.schweitz.net" always;
add_header Access-Control-Allow-Credentials true always;
add_header Access-Control-Allow-Methods "GET, POST, PUT, DELETE, PATCH, OPTIONS" always;
add_header Access-Control-Allow-Headers "Content-Type, Authorization" always;
if ($request_method = OPTIONS) {
return 204;
}
+1 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "core-api"
version = "1.6.1"
version = "1.14.0"
description = "Core Code API - Infrastructure management and tools API"
readme = "README.md"
requires-python = ">=3.12"
+4
View File
@@ -21,6 +21,7 @@ python-dotenv~=1.0.0
python-json-logger~=2.0.0
pytz~=2024.1
dnspython~=2.7.0
psutil~=6.1.0
# Authentication & Security
PyJWT[crypto]>=2.9.0
@@ -31,3 +32,6 @@ cryptography>=44.0.1 # CVE-2024-12797
sqlalchemy[asyncio]~=2.0.0
asyncpg>=0.30.0
alembic~=1.13.0
# Vector Database
qdrant-client>=1.9.0
+70 -7
View File
@@ -13,6 +13,7 @@ from src.logging_config import get_logger
from src.db import get_async_session
from src.auth.schemas import AuthSyncRequest, AuthSyncResponse, UsersListResponse, BulkSyncResultSchema, GroupsListResponse
from src.auth.service import AuthService
from src.auth.oidc import get_forward_auth_user
logger = get_logger(__name__)
@@ -222,21 +223,83 @@ class AuthController(BaseController):
responses={
200: {"description": "User profile"},
401: {"description": "Not authenticated"},
404: {"description": "User not found in database"},
},
)
async def get_me(
forward_auth_user: Optional[dict] = Depends(get_forward_auth_user),
session: AsyncSession = Depends(get_async_session),
) -> JSONResponse:
) -> AuthSyncResponse:
"""
Get the current authenticated user's profile
Note: This endpoint requires a valid session or API key.
For now, returns 501 Not Implemented until session management is added.
Authentication is handled by NPM forward auth with Authentik.
The proxy sets X-authentik-* headers which this endpoint reads.
For internal/LAN access (no forward auth headers), returns 401.
Use POST /auth/sync with an OIDC token for mobile app authentication.
"""
# TODO: Implement with get_current_user dependency
raise HTTPException(
status_code=501,
detail="Not implemented - use /auth/sync with access token",
# Require forward auth for this endpoint
if forward_auth_user is None:
raise HTTPException(
status_code=401,
detail="Authentication required - access via authenticated proxy or use /auth/sync",
)
service = AuthService(session)
# Try to find user by Authentik UID first, then by email
user = None
uid = forward_auth_user.get("uid")
if uid:
try:
import uuid
authentik_id = uuid.UUID(uid)
user = await service.get_user_by_authentik_id(authentik_id)
except (ValueError, TypeError):
pass # Invalid UUID, try email
if user is None:
email = forward_auth_user.get("email")
if email:
user = await service.get_user_by_email(email)
if user is None:
# User authenticated with Authentik but not synced to database yet
# This can happen on first login via web
logger.info(f"User {forward_auth_user.get('email')} not found, creating from forward auth")
# Create user from forward auth headers
from src.auth.schemas import TokenInfoSchema
token_info = TokenInfoSchema(
sub=forward_auth_user.get("uid", ""),
email=forward_auth_user.get("email", ""),
name=forward_auth_user.get("name"),
groups=forward_auth_user.get("groups", []),
)
try:
user, _ = await service.sync_user(token_info)
await service.sync_roles(user, token_info.groups)
await session.commit()
await session.refresh(user, ["preferences", "roles"])
except Exception as e:
logger.error(f"Failed to create user from forward auth: {e}")
raise HTTPException(
status_code=500,
detail="Failed to create user profile",
)
# Sync roles from current groups (in case they changed)
groups = forward_auth_user.get("groups", [])
roles = await service.sync_roles(user, groups)
await session.commit()
return AuthSyncResponse(
user=service.user_to_schema(user),
roles=service.roles_to_schema(roles),
preferences=service.preferences_to_schema(user.preferences),
is_new_user=False,
)
return router
+60 -23
View File
@@ -22,29 +22,42 @@ class OIDCConfig:
def __init__(self):
# These will be set from environment variables in config.py
self.enabled = False
self.issuer = ""
self.audience = ""
self.jwks_uri = ""
self.issuers: list[str] = []
self.audiences: list[str] = []
def configure(self, enabled: bool, issuer: str, audience: str):
def configure(self, enabled: bool, issuers: list[str], audiences: list[str]):
"""Configure OIDC settings"""
self.enabled = enabled
self.issuer = issuer
self.audience = audience
self.jwks_uri = f"{issuer.rstrip('/')}/jwks/"
logger.info(f"OIDC configured: enabled={enabled}, issuer={issuer}")
self.issuers = [iss.rstrip('/') for iss in issuers] # Normalize without trailing slash
self.audiences = audiences
logger.info(f"OIDC configured: enabled={enabled}, issuers={self.issuers}, audiences={audiences}")
def get_jwks_uri(self, issuer: str) -> str:
"""Get JWKS URI for a specific issuer"""
return f"{issuer.rstrip('/')}/jwks/"
def is_valid_issuer(self, issuer: str) -> bool:
"""Check if issuer is in the allowed list"""
normalized = issuer.rstrip('/')
return normalized in self.issuers
# Global OIDC config instance
oidc_config = OIDCConfig()
@lru_cache(maxsize=1)
def get_jwks() -> Dict:
"""
Fetch JSON Web Key Set (JWKS) from Authentik
# Per-issuer JWKS cache
_jwks_cache: Dict[str, Dict] = {}
Cached to avoid repeated requests. Cache is cleared on server restart.
def get_jwks_for_issuer(issuer: str) -> Dict:
"""
Fetch JSON Web Key Set (JWKS) for a specific issuer.
Cached per-issuer to avoid repeated requests. Cache is cleared on server restart.
Args:
issuer: The token issuer URL
Returns:
JWKS dictionary containing public keys for token verification
@@ -55,15 +68,24 @@ def get_jwks() -> Dict:
if not oidc_config.enabled:
return {}
normalized_issuer = issuer.rstrip('/')
# Return cached JWKS if available
if normalized_issuer in _jwks_cache:
return _jwks_cache[normalized_issuer]
jwks_uri = oidc_config.get_jwks_uri(normalized_issuer)
try:
logger.debug(f"Fetching JWKS from {oidc_config.jwks_uri}")
response = httpx.get(oidc_config.jwks_uri, timeout=10.0)
logger.debug(f"Fetching JWKS from {jwks_uri}")
response = httpx.get(jwks_uri, timeout=10.0)
response.raise_for_status()
jwks = response.json()
logger.info(f"JWKS fetched successfully ({len(jwks.get('keys', []))} keys)")
logger.info(f"JWKS fetched successfully for {normalized_issuer} ({len(jwks.get('keys', []))} keys)")
_jwks_cache[normalized_issuer] = jwks
return jwks
except Exception as e:
logger.error(f"Failed to fetch JWKS: {e}")
logger.error(f"Failed to fetch JWKS from {jwks_uri}: {e}")
raise HTTPException(
status_code=503,
detail="Authentication service unavailable"
@@ -109,6 +131,21 @@ async def get_current_user(
token = credentials.credentials
try:
# First, extract issuer and audience from unverified claims
unverified_claims = jwt.get_unverified_claims(token)
token_issuer = unverified_claims.get("iss", "")
token_audience = unverified_claims.get("aud", "")
# Validate issuer is in our allowed list
if not oidc_config.is_valid_issuer(token_issuer):
logger.warning(f"Invalid token issuer: {token_issuer}")
raise HTTPException(status_code=401, detail="Invalid token issuer")
# Validate audience is in our allowed list
if token_audience not in oidc_config.audiences:
logger.warning(f"Invalid token audience: {token_audience}")
raise HTTPException(status_code=401, detail="Invalid token audience")
# Decode token header to get key ID
unverified_header = jwt.get_unverified_header(token)
kid = unverified_header.get("kid")
@@ -116,8 +153,8 @@ async def get_current_user(
if not kid:
raise HTTPException(status_code=401, detail="Invalid token format")
# Find matching key in JWKS
jwks = get_jwks()
# Find matching key in JWKS for this specific issuer
jwks = get_jwks_for_issuer(token_issuer)
rsa_key = None
for key in jwks.get("keys", []):
@@ -129,17 +166,17 @@ async def get_current_user(
logger.warning(f"No matching key found for kid: {kid}")
raise HTTPException(status_code=401, detail="Invalid token key")
# Verify and decode token
# Verify and decode token using the token's actual issuer and audience
payload = jwt.decode(
token,
rsa_key,
algorithms=["RS256"],
audience=oidc_config.audience,
issuer=oidc_config.issuer,
audience=token_audience, # Use the token's audience (already validated)
issuer=token_issuer, # Use the token's issuer (already validated)
)
user_email = payload.get("email", "unknown")
logger.info(f"Authenticated user: {user_email}")
logger.info(f"Authenticated user: {user_email} (issuer: {token_issuer})")
return payload
+37 -1
View File
@@ -13,7 +13,7 @@ from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from src.config import get_settings
from src.shared.config import get_settings
from src.logging_config import get_logger
from src.db.models import User, Role, UserPreferences, Group
from src.auth.schemas import TokenInfoSchema, UserSchema, RoleSchema, UserPreferencesSchema, UserListItemSchema, BulkSyncResultSchema, GroupListItemSchema
@@ -86,6 +86,42 @@ class AuthService:
logger.error(f"Authentik userinfo request error: {e}")
raise ValueError("Authentication service unavailable")
async def get_user_by_email(self, email: str) -> Optional[User]:
"""
Get user by email address
Args:
email: User email address
Returns:
User if found, None otherwise
"""
stmt = (
select(User)
.options(selectinload(User.roles), selectinload(User.preferences))
.where(User.email == email)
)
result = await self.session.execute(stmt)
return result.scalar_one_or_none()
async def get_user_by_authentik_id(self, authentik_id: uuid.UUID) -> Optional[User]:
"""
Get user by Authentik UUID
Args:
authentik_id: Authentik user UUID
Returns:
User if found, None otherwise
"""
stmt = (
select(User)
.options(selectinload(User.roles), selectinload(User.preferences))
.where(User.authentik_id == authentik_id)
)
result = await self.session.execute(stmt)
return result.scalar_one_or_none()
async def sync_user(self, token_info: TokenInfoSchema) -> tuple[User, bool]:
"""
Create or update user from OIDC token info
+1 -1
View File
@@ -10,7 +10,7 @@ import json
from typing import Optional, Dict, List, Any
from datetime import datetime, timedelta, timezone
from src.logging_config import get_logger
from src.config import get_settings
from src.shared.config import get_settings
logger = get_logger(__name__)
settings = get_settings()
+1 -1
View File
@@ -7,7 +7,7 @@ import httpx
from typing import Optional, Dict, List, Any
from datetime import datetime, timedelta
from src.logging_config import get_logger
from src.config import get_settings
from src.shared.config import get_settings
logger = get_logger(__name__)
settings = get_settings()
+1 -1
View File
@@ -8,7 +8,7 @@ import httpx
import json
from typing import Optional, Dict, List, Any
from src.logging_config import get_logger
from src.config import get_settings
from src.shared.config import get_settings
logger = get_logger(__name__)
settings = get_settings()
-160
View File
@@ -1,160 +0,0 @@
"""
Global configuration for Core Code API
All configuration is loaded from environment variables or .env file.
See .env.example for available settings.
"""
import tomllib
from pathlib import Path
from pydantic_settings import BaseSettings
from functools import lru_cache
def _get_version_from_pyproject() -> str:
"""Load version from pyproject.toml"""
pyproject_path = Path(__file__).parent.parent / "pyproject.toml"
try:
with open(pyproject_path, "rb") as f:
data = tomllib.load(f)
return data.get("project", {}).get("version", "0.0.0")
except FileNotFoundError:
return "0.0.0"
__version__ = _get_version_from_pyproject()
class Settings(BaseSettings):
"""Global application settings"""
# Application
app_name: str = "Core Code API"
app_version: str = __version__
debug: bool = False
# Server
host: str = "0.0.0.0"
port: int = 8083
# CORS
cors_origins: list[str] = ["*"]
cors_credentials: bool = True
cors_methods: list[str] = ["*"]
cors_headers: list[str] = ["*"]
# Logging
log_level: str = "DEBUG"
# Ollama Configuration (for AI orchestration)
ollama_base_url: str # Required - set OLLAMA_BASE_URL in .env
ollama_timeout: int = 300 # 5 minutes
# Model Configuration
default_model: str = "mistral-nemo-large:latest"
agent_model: str = "mistral-nemo-large:latest" # Must support tool calling with ADK (~4GB VRAM)
code_models: str = "mistral-nemo-large:latest"
# Previous config (gemma3:12b used ~10GB VRAM)
# default_model: str = "gemma3:12b"
# agent_model: str = "gemma3:12b"
# System Prompt Variant (for A/B testing)
# Options: v1_verbose, v2_concise, v3_imperative, v4_minimal, v4_gemini_suggestion, v5_adk_optimized, v7_adk_best_practice, v8_holistic
system_prompt_variant: str = "v8_holistic"
# Agent Configuration
agent_fallback_enabled: bool = True
# Model Aliases (OpenAI → Local)
alias_gpt35: str = "gemma:7b"
alias_gpt4: str = "mistral:7b"
alias_gpt4_turbo: str = "mixtral:8x7b"
alias_gpt4_code: str = "codestral:latest"
# Memory Configuration
memory_tier1_max_turns: int = 10
memory_consolidation_threshold: int = 10
# Qdrant Configuration
qdrant_host: str = "qdrant"
qdrant_port: int = 6333
qdrant_collection_conversations: str = "core_api_conversations"
qdrant_collection_documents: str = "core_api_documents"
qdrant_collection_user_facts: str = "core_api_user_facts"
# Embeddings (using Ollama - no local models needed)
embedding_model: str = "nomic-embed-text" # Ollama embedding model
embedding_dimension: int = 768 # nomic-embed-text dimension
embedding_batch_size: int = 32
# Search Configuration
search_provider: str = "searxng"
searxng_url: str # Required - set SEARXNG_URL in .env
# Infrastructure Management (Portainer)
portainer_url: str # Required - set PORTAINER_URL in .env
portainer_api_key: str # Required - set PORTAINER_API_KEY in .env
# Infrastructure Management (Nginx Proxy Manager)
npm_url: str # Required - set NPM_URL in .env
npm_email: str # Required - set NPM_EMAIL in .env
npm_password: str # Required - set NPM_PASSWORD in .env
# Home Assistant Configuration
homeassistant_url: str # Required - set HOMEASSISTANT_URL in .env
homeassistant_token: str # Required - set HOMEASSISTANT_TOKEN in .env
homeassistant_timeout: int = 30
# PostgreSQL Database
postgres_host: str # Required - set POSTGRES_HOST in .env (e.g., localhost:5432)
postgres_user: str = "core_api"
postgres_password: str # Required - set POSTGRES_PASSWORD in .env
postgres_database: str = "core_api"
@property
def database_url(self) -> str:
"""Construct database URL from components"""
return f"postgresql://{self.postgres_user}:{self.postgres_password}@{self.postgres_host}/{self.postgres_database}"
# OIDC Authentication (Authentik)
oidc_enabled: bool = False # Set to True to require authentication
oidc_issuer: str = "https://auth.schweitz.net/application/o/core-api/"
oidc_audience: str = "core-api"
# Authentik API (for token validation and user management)
# Must use domain name (not IP) when AUTHENTIK_COOKIE_DOMAIN is set
authentik_url: str = "https://auth.schweitz.net" # Authentik base URL
authentik_username: str = "" # Admin username for API access (AUTHENTIK_USERNAME env var)
authentik_password: str = "" # Admin password for API access (AUTHENTIK_PASSWORD env var)
@property
def model_aliases(self) -> dict:
"""Computed property for model aliases"""
return {
"gpt-3.5-turbo": self.alias_gpt35,
"gpt-4": self.alias_gpt4,
"gpt-4-turbo": self.alias_gpt4_turbo,
"gpt-4-code": self.alias_gpt4_code,
}
def get_lightweight_models(self) -> list[str]:
"""Parse comma-separated lightweight models"""
return [m.strip().strip('"').strip("'") for m in self.lightweight_models.split(",") if m.strip()]
def get_heavy_models(self) -> list[str]:
"""Parse comma-separated heavy models"""
return [m.strip().strip('"').strip("'") for m in self.heavy_models.split(",") if m.strip()]
def get_code_models(self) -> list[str]:
"""Parse comma-separated code models"""
return [m.strip().strip('"').strip("'") for m in self.code_models.split(",") if m.strip()]
class Config:
env_file = ".env"
case_sensitive = False
extra = "ignore" # Ignore extra env vars not defined in Settings
@lru_cache()
def get_settings() -> Settings:
"""Cached settings instance"""
return Settings()
+13 -96
View File
@@ -7,13 +7,11 @@ from fastapi import APIRouter, Response
from fastapi.responses import JSONResponse
from src.controllers.base import BaseController
from src.config import get_settings
from src.shared.config import get_settings
from src.logging_config import get_logger
from src.models.ollama_client import get_ollama_client
from src.db import get_database
logger = get_logger(__name__)
@@ -73,63 +71,18 @@ class HealthController(BaseController):
@router.get(
"/health/full",
summary="Fast health check for Docker",
summary="Full health check with database",
)
async def full_health_check(response: Response):
"""
Fast health check for container orchestration (Docker/K8s).
Health check including database connectivity.
Checks component availability WITHOUT running expensive operations.
Returns 200 OK if all components are available, otherwise 503.
For detailed diagnostics, use /health/diagnostics instead.
Returns 200 OK if database is available, otherwise 503.
"""
import time
start_time = time.time()
# Check 1: Ollama connection + verify agent model is available
ollama_client = get_ollama_client()
ollama_healthy = False
ollama_error = None
model_available = False
try:
# Ping Ollama
ollama_healthy = await ollama_client.health_check()
# Verify the agent model is pulled and check what's currently loaded
models_info = {}
if ollama_healthy:
try:
models_response = await ollama_client.list_models()
available_models = [m.get('name', '') for m in models_response.get('models', [])]
model_available = settings.agent_model in available_models
# Get info about currently loaded models (those with size in memory)
loaded_models = [
m.get('name', '') for m in models_response.get('models', [])
if m.get('size', 0) > 0
]
models_info = {
"configured": settings.agent_model,
"available": model_available,
"total_in_ollama": len(available_models),
"currently_loaded": loaded_models if loaded_models else ["none"]
}
if not model_available:
ollama_error = f"Model '{settings.agent_model}' not found in Ollama. Available: {', '.join(available_models[:3])}"
ollama_healthy = False
except Exception as e:
ollama_error = f"Could not list Ollama models: {str(e)}"
ollama_healthy = False
except Exception as e:
ollama_error = str(e)
logger.warning(f"Ollama health check failed: {ollama_error}")
# Check 2: Database connection
# Check database connection
database = get_database()
db_healthy = False
db_error = None
@@ -140,27 +93,17 @@ class HealthController(BaseController):
db_error = str(e)
logger.warning(f"Database health check failed: {db_error}")
is_healthy = ollama_healthy and db_healthy
elapsed_ms = int((time.time() - start_time) * 1000)
status_code = 200 if is_healthy else 503
status_code = 200 if db_healthy else 503
response.status_code = status_code
return {
"status": "healthy" if is_healthy else "unhealthy",
"status": "healthy" if db_healthy else "unhealthy",
"status_code": status_code,
"response_time_ms": elapsed_ms,
"components": {
"ollama": {
"status": "✅ healthy" if ollama_healthy else "❌ unhealthy",
"models": models_info if models_info else {
"configured": settings.agent_model,
"available": False
},
"error": ollama_error
},
"database": {
"status": "✅ healthy" if db_healthy else "❌ unhealthy",
"status": "healthy" if db_healthy else "unhealthy",
"error": db_error
}
}
@@ -170,18 +113,13 @@ class HealthController(BaseController):
"/health/diagnostics",
summary="Detailed system diagnostics",
)
async def diagnostics(deep_test: bool = False):
async def diagnostics():
"""
Comprehensive system diagnostics with detailed component information.
Query Parameters:
- deep_test: Set to true to actually test agent generation (slow, ~5-10s)
Returns detailed information about all system components.
System diagnostics with service information.
"""
import time
start_time = time.time()
diagnostics = {
"timestamp": time.time(),
"service": {
@@ -189,30 +127,9 @@ class HealthController(BaseController):
"version": settings.app_version,
"purpose": "Infrastructure management and tools API"
},
"components": {}
}
# 1. Ollama Connection
ollama_client = get_ollama_client()
try:
ollama_healthy = await ollama_client.health_check()
diagnostics["components"]["ollama"] = {
"status": "✅ connected",
"url": settings.ollama_base_url,
"timeout": settings.ollama_timeout,
"default_model": settings.default_model
"configuration": {
"cors_origins": settings.cors_origins[:2] if len(settings.cors_origins) > 2 else settings.cors_origins
}
except Exception as e:
diagnostics["components"]["ollama"] = {
"status": "❌ error",
"error": str(e)
}
# 2. Configuration
diagnostics["configuration"] = {
"agent_fallback_enabled": settings.agent_fallback_enabled,
"memory_tier1_max_turns": settings.memory_tier1_max_turns,
"cors_origins": settings.cors_origins[:2] if len(settings.cors_origins) > 2 else settings.cors_origins
}
elapsed_ms = int((time.time() - start_time) * 1000)
+62 -1
View File
@@ -3,14 +3,20 @@ Tools Controller
Provides utility tool endpoints including:
- DNS lookups
- Environment data (weather, forecast, sun times, air quality)
"""
from fastapi import APIRouter, HTTPException, status
from typing import Dict, Optional
from fastapi import APIRouter, Depends, HTTPException, status
from src.controllers.base import BaseController
from src.logging_config import get_logger
from src.dns.schemas import DNSLookupRequest, DNSLookupResponse
from src.dns.service import DNSService
from src.dns.exceptions import DNSQueryError
from src.domains.tools.environment.schemas import EnvironmentResponse
from src.domains.tools.environment.service import get_environment_service
from src.auth.oidc import get_optional_user
logger = get_logger(__name__)
@@ -26,6 +32,7 @@ class ToolsController(BaseController):
def __init__(self):
super().__init__(prefix="/tools", tags=["Tools"])
self.dns_service = DNSService()
self.environment_service = get_environment_service()
def create_router(self) -> APIRouter:
"""Create and configure the router"""
@@ -95,6 +102,60 @@ class ToolsController(BaseController):
detail="An unexpected error occurred during DNS lookup"
)
@router.get(
"/environment",
response_model=EnvironmentResponse,
status_code=status.HTTP_200_OK,
summary="Get environment data",
description="""
Fetch current environment data including weather, forecast, sun times,
and optionally air quality.
Data is retrieved from the user's volatile Qdrant collection which is
populated by background data collectors.
**Data Sources:**
- Weather: Current temperature, conditions, humidity, wind
- Forecast: Multi-day weather outlook
- Sun Times: Sunrise, sunset, daylight duration
- Air Quality: AQI and pollutant levels (when available)
**Authentication:**
- Uses authenticated user's `preferred_username` if available
- Falls back to 'default' for unauthenticated requests
"""
)
async def get_environment(
user: Optional[Dict] = Depends(get_optional_user),
) -> EnvironmentResponse:
"""
Get current environment data.
Args:
user: Optional authenticated user info
Returns:
Environment data with weather, forecast, sun times, and air quality
"""
try:
# Determine user identifier
user_id = "default"
if user:
logger.debug(f"User claims: {user}")
user_id = user.get("preferred_username") or user.get("sub", "default")
logger.info(f"Fetching environment data for user: {user_id} (preferred_username={user.get('preferred_username')}, sub={user.get('sub')})")
else:
logger.info(f"Fetching environment data for user: {user_id} (no auth)")
result = await self.environment_service.get_current(user_id)
return result
except Exception as e:
logger.error(f"Error fetching environment data: {str(e)}", exc_info=True)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Failed to fetch environment data"
)
return router
+1 -1
View File
@@ -15,7 +15,7 @@ from sqlalchemy.ext.asyncio import (
from sqlalchemy.orm import DeclarativeBase
from sqlalchemy.pool import NullPool
from src.config import get_settings
from src.shared.config import get_settings
from src.logging_config import get_logger
logger = get_logger(__name__)
+18
View File
@@ -11,9 +11,16 @@ from src.domains.auth.oidc import (
get_forward_auth_user,
get_forward_auth_admin,
oidc_config,
# Permission system
require_permission,
require_any_permission,
ACTION_HIERARCHY,
VALID_DOMAINS,
DEFAULT_CATEGORY,
)
from src.domains.auth.service import AuthService, get_auth_service
from src.domains.auth.controller import auth_controller
from src.domains.auth.callers import Caller, require_caller
from src.domains.auth.models import (
User,
Role,
@@ -22,6 +29,7 @@ from src.domains.auth.models import (
UserPreferences,
ApiKey,
user_groups,
group_roles,
)
__all__ = [
@@ -32,11 +40,20 @@ __all__ = [
"get_forward_auth_user",
"get_forward_auth_admin",
"oidc_config",
# Permission system
"require_permission",
"require_any_permission",
"ACTION_HIERARCHY",
"VALID_DOMAINS",
"DEFAULT_CATEGORY",
# Service
"AuthService",
"get_auth_service",
# Controller
"auth_controller",
# Callers (T-1)
"Caller",
"require_caller",
# Models
"User",
"Role",
@@ -45,4 +62,5 @@ __all__ = [
"UserPreferences",
"ApiKey",
"user_groups",
"group_roles",
]
+181
View File
@@ -0,0 +1,181 @@
"""
Caller Authentication
core-api has no service-key auth today: AuthService.validate_api_key is
never called in any route, api_keys stays empty, and `CORE_API_KEY` is read
by nothing. This module is the first thing that reads it.
Resolution order, static keys before any JWT parsing:
1. Bearer == office_service_key -> kind "office"
2. Bearer == tatlock_service_key -> kind "tatlock"
3. Bearer == scheduler_service_key -> kind "scheduler"
4. Otherwise -> get_current_user() -> kind "user"
Each static key is matched with hmac.compare_digest, and an unconfigured
(empty-string) key never matches — checked explicitly rather than trusting
compare_digest("", "") to be the right thing to compare, which would make an
empty configured key match an empty bearer.
core-api T-4 (workspace D-73): the Office now signs every visitor in through
Authentik with one OAuth client per consumer, and every token carries a
signed `tatlock_user` claim. The Office verifies the token and forwards the
result in two headers, trusted only alongside the office service key:
- `X-Office-Consumer` — the Authentik application slug the token came from
(`office-claude`, `office-gemini`, ...).
- `X-Office-Tenant` — the token's `tatlock_user` claim (the tatlock tenant).
For kind "office" both are required on **every** call, reads included — the
tenant scopes reads too. For every other kind they are ignored outright:
"tatlock" and "scheduler" carry no tenant (`None`); a "user" caller gets its
tenant from its own JWT's `tatlock_user` claim if it has one, else `None` —
never from a header, so a header can never put a non-Office caller into a
tenant.
`X-Office-Client` (informational, pre-T-4) is retired in favour of
`X-Office-Consumer`; nothing reads it any more.
"""
import hmac
import re
from dataclasses import dataclass
from typing import Callable, Literal, Optional
from fastapi import Depends, HTTPException, Request, Security
from fastapi.security import HTTPAuthorizationCredentials
from src.domains.auth.oidc import get_current_user, security
from src.shared.config import get_settings
from src.shared.logging import get_logger
logger = get_logger(__name__)
CallerKind = Literal["office", "tatlock", "scheduler", "user"]
# Reserved identities: the system nouns, never a caller's own identity.
RESERVED_IDENTITIES = {"book", "office", "scheduler", "tatlock"}
# ^[a-z0-9][a-z0-9-]{0,62}$, and additionally not a "group-*" shape (checked
# separately below, since the character class alone can't exclude a prefix).
_IDENTITY_RE = re.compile(r"^[a-z0-9][a-z0-9-]{0,62}$")
# X-Office-Consumer: the Authentik application slug, same shape as an identity.
_CONSUMER_RE = re.compile(r"^[a-z0-9][a-z0-9-]{0,62}$")
# X-Office-Tenant: tatlock's own tenant ids contain underscores (e.g.
# `llm_tester`), so the identity/consumer shape doesn't fit.
_TENANT_RE = re.compile(r"^[a-z0-9_][a-z0-9_.-]{0,62}$")
@dataclass(frozen=True)
class Caller:
"""Who is calling, resolved once per request."""
kind: CallerKind
identity: str # stamped on every write and event
consumer: Optional[str] # X-Office-Consumer; office only, else None
tenant: Optional[str] # the tatlock tenant; office header or a user's own JWT claim
def _bad_request(code: str) -> HTTPException:
return HTTPException(status_code=400, detail={"code": code})
def _validate_identity(identity: str) -> None:
"""Raise 400 identity_invalid for a malformed or reserved identity."""
if not _IDENTITY_RE.match(identity):
raise _bad_request("identity_invalid")
if identity.startswith("group-") or identity in RESERVED_IDENTITIES:
raise _bad_request("identity_invalid")
def _require_office_consumer(request: Request) -> str:
"""X-Office-Consumer is required on every office call; raise if absent/malformed."""
value = request.headers.get("X-Office-Consumer")
if not value:
raise _bad_request("consumer_required")
if not _CONSUMER_RE.match(value):
raise _bad_request("consumer_invalid")
return value
def _require_office_tenant(request: Request) -> str:
"""X-Office-Tenant is required on every office call, reads included."""
value = request.headers.get("X-Office-Tenant")
if not value:
raise _bad_request("tenant_required")
if not _TENANT_RE.match(value):
raise _bad_request("tenant_invalid")
return value
def require_caller(*allowed: CallerKind, identity_required: bool = False) -> Callable:
"""
Build a FastAPI dependency that resolves the caller and enforces it is
one of `allowed`.
Args:
allowed: caller kinds permitted on this route.
identity_required: for an "office" caller, whether X-Office-Identity
must be present (true on writes). Other kinds carry a fixed
identity and ignore this. Unrelated to consumer/tenant, which an
"office" caller must carry on every call regardless of this flag.
"""
async def dependency(
request: Request,
credentials: Optional[HTTPAuthorizationCredentials] = Security(security),
) -> Caller:
settings = get_settings()
token = credentials.credentials if credentials else None
caller: Optional[Caller] = None
if (
token
and settings.office_service_key
and hmac.compare_digest(token, settings.office_service_key)
):
consumer = _require_office_consumer(request)
tenant = _require_office_tenant(request)
identity_header = request.headers.get("X-Office-Identity")
if not identity_header:
if identity_required:
raise _bad_request("identity_required")
identity = ""
else:
_validate_identity(identity_header)
identity = identity_header
caller = Caller(kind="office", identity=identity, consumer=consumer, tenant=tenant)
elif (
token
and settings.tatlock_service_key
and hmac.compare_digest(token, settings.tatlock_service_key)
):
caller = Caller(kind="tatlock", identity="tatlock", consumer=None, tenant=None)
elif (
token
and settings.scheduler_service_key
and hmac.compare_digest(token, settings.scheduler_service_key)
):
caller = Caller(kind="scheduler", identity="scheduler", consumer=None, tenant=None)
else:
# No static key matched (or none configured) — fall through to
# the existing OIDC/JWT path. A static key never reaches here.
user = await get_current_user(credentials)
identity = (user or {}).get("preferred_username") or "user"
# The tenant comes only from the JWT's own claim — never from a
# header, which would let any caller spoof another tenant.
tenant = (user or {}).get("tatlock_user") or None
caller = Caller(kind="user", identity=identity, consumer=None, tenant=tenant)
if caller.kind not in allowed:
logger.warning(f"Caller kind '{caller.kind}' forbidden on this route (allowed: {allowed})")
raise HTTPException(status_code=403, detail={"code": "forbidden"})
return caller
return dependency
+377 -14
View File
@@ -3,8 +3,9 @@ Authentication Controller
Provides authentication endpoints for OIDC token sync and user management.
"""
import uuid
from typing import Optional
from fastapi import APIRouter, Depends, HTTPException, Query
from fastapi import APIRouter, Depends, HTTPException, Path, Query
from fastapi.responses import JSONResponse
from sqlalchemy.ext.asyncio import AsyncSession
@@ -13,9 +14,13 @@ from src.shared.logging import get_logger
from src.shared.database import get_async_session
from src.domains.auth.schemas import (
AuthSyncRequest, AuthSyncResponse, UsersListResponse,
BulkSyncResultSchema, GroupsListResponse
BulkSyncResultSchema, GroupsListResponse, RolesListResponse,
GroupRoleAssignmentResponse, UserProfileResponse, PreferencesUpdateRequest,
UserPreferencesSchema, ApiKeyCreateRequest, ApiKeyCreateResponse,
ApiKeysListResponse,
)
from src.domains.auth.service import AuthService
from src.domains.auth.oidc import get_current_user, get_current_user_or_forward_auth
logger = get_logger(__name__)
@@ -219,29 +224,387 @@ class AuthController(BaseController):
raise HTTPException(status_code=401, detail=str(e))
@router.get(
"/me",
summary="Get current user profile",
response_model=AuthSyncResponse,
"/roles",
summary="List all roles",
response_model=RolesListResponse,
responses={
200: {"description": "User profile"},
200: {"description": "List of all available roles"},
},
)
async def list_roles(
session: AsyncSession = Depends(get_async_session),
) -> RolesListResponse:
"""
List all available roles in the system
Returns all domain.category:action role combinations.
Use these when assigning roles to groups.
"""
service = AuthService(session)
roles = await service.list_roles()
return RolesListResponse(
items=service.roles_to_schema(roles),
total=len(roles),
)
@router.post(
"/groups/{group_id}/roles/{role_id}",
summary="Assign role to group",
response_model=GroupRoleAssignmentResponse,
responses={
200: {"description": "Role assigned successfully"},
404: {"description": "Group or role not found"},
},
)
async def assign_role_to_group(
group_id: uuid.UUID = Path(..., description="Group ID"),
role_id: uuid.UUID = Path(..., description="Role ID to assign"),
session: AsyncSession = Depends(get_async_session),
) -> GroupRoleAssignmentResponse:
"""
Assign a role to a group
All users in this group will inherit this role's permissions.
"""
service = AuthService(session)
try:
group = await service.assign_role_to_group(group_id, role_id)
await session.commit()
return GroupRoleAssignmentResponse(
group_id=group.id,
group_name=group.name,
roles=[role.name for role in group.roles],
)
except ValueError as e:
raise HTTPException(status_code=404, detail=str(e))
@router.delete(
"/groups/{group_id}/roles/{role_id}",
summary="Remove role from group",
response_model=GroupRoleAssignmentResponse,
responses={
200: {"description": "Role removed successfully"},
404: {"description": "Group or role not found"},
},
)
async def remove_role_from_group(
group_id: uuid.UUID = Path(..., description="Group ID"),
role_id: uuid.UUID = Path(..., description="Role ID to remove"),
session: AsyncSession = Depends(get_async_session),
) -> GroupRoleAssignmentResponse:
"""
Remove a role from a group
Users in this group will no longer inherit this role's permissions.
"""
service = AuthService(session)
try:
group = await service.remove_role_from_group(group_id, role_id)
await session.commit()
return GroupRoleAssignmentResponse(
group_id=group.id,
group_name=group.name,
roles=[role.name for role in group.roles],
)
except ValueError as e:
raise HTTPException(status_code=404, detail=str(e))
# =====================================================================
# Phase 4: User Profile & Settings
# =====================================================================
@router.get(
"/users/me",
summary="Get current user profile",
response_model=UserProfileResponse,
responses={
200: {"description": "User profile with roles and preferences"},
401: {"description": "Not authenticated"},
},
)
async def get_me(
async def get_current_user_profile(
user_claims: dict = Depends(get_current_user_or_forward_auth),
session: AsyncSession = Depends(get_async_session),
) -> JSONResponse:
) -> UserProfileResponse:
"""
Get the current authenticated user's profile
Note: This endpoint requires a valid session or API key.
For now, returns 501 Not Implemented until session management is added.
Returns the user's profile, roles, and preferences.
Supports both:
- Bearer token (mobile/native clients)
- NPM forward auth headers (web clients via proxy)
For forward auth users, auto-creates the user in the database
if they don't exist yet (first login via web).
"""
# TODO: Implement with get_current_user dependency
raise HTTPException(
status_code=501,
detail="Not implemented - use /auth/sync with access token",
service = AuthService(session)
# Get authentik_id from claims (JWT 'sub' field or forward auth 'uid')
authentik_id_str = user_claims.get("sub")
if not authentik_id_str or authentik_id_str == "local-user":
raise HTTPException(status_code=401, detail="Authentication required")
try:
authentik_id = uuid.UUID(authentik_id_str)
except ValueError:
raise HTTPException(status_code=401, detail="Invalid user identifier")
user = await service.get_user_by_authentik_id(authentik_id)
# If user not found and using forward auth, auto-create them
if user is None:
auth_method = user_claims.get("auth_method")
if auth_method == "forward_auth":
# Auto-sync user from forward auth headers
logger.info(f"Auto-creating user from forward auth: {user_claims.get('email')}")
user, is_new = await service.sync_user_from_claims(
authentik_id=authentik_id,
email=user_claims.get("email", ""),
name=user_claims.get("name", user_claims.get("preferred_username", "")),
groups=user_claims.get("groups", []),
)
await session.commit()
await session.refresh(user, ["preferences", "roles"])
else:
# JWT auth but user not in DB - they need to sync first
raise HTTPException(
status_code=404,
detail="User not found - please sync via /auth/sync first",
)
return UserProfileResponse(
user=service.user_to_schema(user),
roles=service.roles_to_schema(user.roles),
preferences=service.preferences_to_schema(user.preferences),
)
@router.get(
"/users/me/preferences",
summary="Get user preferences",
response_model=UserPreferencesSchema,
responses={
200: {"description": "User preferences"},
401: {"description": "Not authenticated"},
},
)
async def get_preferences(
user_claims: dict = Depends(get_current_user),
session: AsyncSession = Depends(get_async_session),
) -> UserPreferencesSchema:
"""
Get the current user's preferences
"""
service = AuthService(session)
authentik_id_str = user_claims.get("sub")
if not authentik_id_str or authentik_id_str == "local-user":
raise HTTPException(status_code=401, detail="Authentication required")
try:
authentik_id = uuid.UUID(authentik_id_str)
except ValueError:
raise HTTPException(status_code=401, detail="Invalid user identifier")
user = await service.get_user_by_authentik_id(authentik_id)
if user is None:
raise HTTPException(status_code=404, detail="User not found")
return service.preferences_to_schema(user.preferences)
@router.patch(
"/users/me/preferences",
summary="Update user preferences",
response_model=UserPreferencesSchema,
responses={
200: {"description": "Updated preferences"},
401: {"description": "Not authenticated"},
422: {"description": "Invalid preference value"},
},
)
async def update_preferences(
request: PreferencesUpdateRequest,
user_claims: dict = Depends(get_current_user),
session: AsyncSession = Depends(get_async_session),
) -> UserPreferencesSchema:
"""
Update the current user's preferences
Only provided fields are updated. preferences_json is merged
with existing values (not replaced).
"""
service = AuthService(session)
authentik_id_str = user_claims.get("sub")
if not authentik_id_str or authentik_id_str == "local-user":
raise HTTPException(status_code=401, detail="Authentication required")
try:
authentik_id = uuid.UUID(authentik_id_str)
except ValueError:
raise HTTPException(status_code=401, detail="Invalid user identifier")
user = await service.get_user_by_authentik_id(authentik_id)
if user is None:
raise HTTPException(status_code=404, detail="User not found")
try:
prefs = await service.update_preferences(
user_id=user.id,
theme=request.theme,
default_room=request.default_room,
preferences_json=request.preferences_json,
)
await session.commit()
return service.preferences_to_schema(prefs)
except ValueError as e:
raise HTTPException(status_code=422, detail=str(e))
# =====================================================================
# Phase 4: API Keys
# =====================================================================
@router.get(
"/users/me/api-keys",
summary="List user's API keys",
response_model=ApiKeysListResponse,
responses={
200: {"description": "List of API keys"},
401: {"description": "Not authenticated"},
},
)
async def list_api_keys(
user_claims: dict = Depends(get_current_user),
session: AsyncSession = Depends(get_async_session),
) -> ApiKeysListResponse:
"""
List all API keys for the current user
Returns key metadata only - the actual key values are never
retrievable after creation.
"""
service = AuthService(session)
authentik_id_str = user_claims.get("sub")
if not authentik_id_str or authentik_id_str == "local-user":
raise HTTPException(status_code=401, detail="Authentication required")
try:
authentik_id = uuid.UUID(authentik_id_str)
except ValueError:
raise HTTPException(status_code=401, detail="Invalid user identifier")
user = await service.get_user_by_authentik_id(authentik_id)
if user is None:
raise HTTPException(status_code=404, detail="User not found")
keys = await service.list_user_api_keys(user.id)
return ApiKeysListResponse(
items=[service.api_key_to_schema(k) for k in keys],
total=len(keys),
)
@router.post(
"/users/me/api-keys",
summary="Create a new API key",
response_model=ApiKeyCreateResponse,
responses={
201: {"description": "API key created"},
401: {"description": "Not authenticated"},
403: {"description": "API keys disabled for user"},
},
)
async def create_api_key(
request: ApiKeyCreateRequest,
user_claims: dict = Depends(get_current_user),
session: AsyncSession = Depends(get_async_session),
) -> ApiKeyCreateResponse:
"""
Create a new API key for the current user
**IMPORTANT**: The full API key is only returned once in this response!
Store it securely - it cannot be retrieved again.
"""
service = AuthService(session)
authentik_id_str = user_claims.get("sub")
if not authentik_id_str or authentik_id_str == "local-user":
raise HTTPException(status_code=401, detail="Authentication required")
try:
authentik_id = uuid.UUID(authentik_id_str)
except ValueError:
raise HTTPException(status_code=401, detail="Invalid user identifier")
user = await service.get_user_by_authentik_id(authentik_id)
if user is None:
raise HTTPException(status_code=404, detail="User not found")
try:
api_key, full_key = await service.create_api_key(
user_id=user.id,
name=request.name,
scopes=request.scopes,
expires_in_days=request.expires_in_days,
)
await session.commit()
return ApiKeyCreateResponse(
id=api_key.id,
name=api_key.name,
key=full_key, # Only time this is returned!
key_prefix=api_key.key_prefix,
scopes=api_key.scopes,
expires_at=api_key.expires_at,
created_at=api_key.created_at,
)
except ValueError as e:
raise HTTPException(status_code=403, detail=str(e))
@router.delete(
"/users/me/api-keys/{key_id}",
summary="Delete an API key",
responses={
204: {"description": "API key deleted"},
401: {"description": "Not authenticated"},
404: {"description": "API key not found"},
},
)
async def delete_api_key(
key_id: uuid.UUID = Path(..., description="API key ID to delete"),
user_claims: dict = Depends(get_current_user),
session: AsyncSession = Depends(get_async_session),
) -> JSONResponse:
"""
Delete an API key
The key will be immediately invalidated.
"""
service = AuthService(session)
authentik_id_str = user_claims.get("sub")
if not authentik_id_str or authentik_id_str == "local-user":
raise HTTPException(status_code=401, detail="Authentication required")
try:
authentik_id = uuid.UUID(authentik_id_str)
except ValueError:
raise HTTPException(status_code=401, detail="Invalid user identifier")
user = await service.get_user_by_authentik_id(authentik_id)
if user is None:
raise HTTPException(status_code=404, detail="User not found")
try:
deleted = await service.delete_api_key(user.id, key_id)
if not deleted:
raise HTTPException(status_code=404, detail="API key not found")
await session.commit()
return JSONResponse(status_code=204, content=None)
except ValueError as e:
raise HTTPException(status_code=403, detail=str(e))
return router
+34 -8
View File
@@ -26,6 +26,13 @@ user_groups = Table(
Column("group_id", UUID(as_uuid=True), ForeignKey("groups.id", ondelete="CASCADE"), primary_key=True),
)
group_roles = Table(
"group_roles",
Base.metadata,
Column("group_id", UUID(as_uuid=True), ForeignKey("groups.id", ondelete="CASCADE"), primary_key=True),
Column("role_id", UUID(as_uuid=True), ForeignKey("roles.id", ondelete="CASCADE"), primary_key=True),
)
# =============================================================================
# User Model
@@ -114,8 +121,13 @@ class Role(Base):
"""
Role model for domain-scoped permissions
Permission format: domain.category:action
- domain: Main area (control-room, library, media, ai, etc.)
- category: Sub-area within domain (general for full access, or specific tools)
- action: Permission level (viewer, user, editor, admin)
Roles are seeded from configuration, not user-editable.
Each role maps to an Authentik group (e.g., tatlock-control-room-admin).
Groups are assigned roles via the group_roles mapping table.
Domains: control-room, library, media, ai, housekeeper, developer, documents, gaming, admin
Actions: viewer, user, editor, admin (hierarchical)
@@ -133,7 +145,7 @@ class Role(Base):
unique=True,
nullable=False,
index=True,
comment="Role name in format domain:action (e.g., control-room:admin)",
comment="Role name in format domain.category:action (e.g., control-room.general:admin)",
)
domain: Mapped[str] = mapped_column(
String(50),
@@ -141,17 +153,17 @@ class Role(Base):
index=True,
comment="Permission domain (e.g., control-room, media, ai)",
)
category: Mapped[str] = mapped_column(
String(50),
nullable=False,
default="general",
comment="Permission category within domain (general for full access, or specific tool)",
)
action: Mapped[str] = mapped_column(
String(20),
nullable=False,
comment="Permission action (viewer, user, editor, admin)",
)
authentik_group: Mapped[str | None] = mapped_column(
String(255),
nullable=True,
unique=True,
comment="Corresponding Authentik group name (e.g., tatlock-control-room-admin)",
)
# Relationships
users: Mapped[List["User"]] = relationship(
@@ -160,6 +172,12 @@ class Role(Base):
back_populates="roles",
lazy="selectin",
)
groups: Mapped[List["Group"]] = relationship(
"Group",
secondary="group_roles",
back_populates="roles",
lazy="selectin",
)
def __repr__(self) -> str:
return f"<Role {self.name}>"
@@ -247,6 +265,14 @@ class Group(Base):
comment="Last sync from Authentik",
)
# Relationships
roles: Mapped[List["Role"]] = relationship(
"Role",
secondary="group_roles",
back_populates="groups",
lazy="selectin",
)
def __repr__(self) -> str:
return f"<Group {self.name}>"
+449 -27
View File
@@ -3,15 +3,56 @@ OIDC Authentication Module
Provides OAuth2/OIDC token validation for FastAPI using Authentik as IdP.
Implements bearer token authentication with JWT verification.
Permission Format: domain.category:action
- domain: Main area (control-room, library, media, ai, etc.)
- category: Sub-area within domain (general for full domain, or specific tools)
- action: Permission level (viewer, user, editor, admin)
Examples:
- control-room.general:admin - Full access to Control Room
- media.general:viewer - View-only access to Media area
Action Hierarchy (higher implies lower):
- admin > editor > user > viewer
"""
from fastapi import Depends, HTTPException, Security, Request
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
from jose import jwt, JWTError
import httpx
from functools import lru_cache
from typing import Dict, Optional
from typing import Callable, Dict, List, Optional
from src.shared.logging import get_logger
# =============================================================================
# Permission System
# =============================================================================
# Action hierarchy: higher actions imply lower ones
ACTION_HIERARCHY: Dict[str, int] = {
"viewer": 1,
"user": 2,
"editor": 3,
"admin": 4,
}
# Valid domains (main areas)
VALID_DOMAINS = {
"control-room",
"library",
"media",
"ai",
"housekeeper",
"developer",
"documents",
"gaming",
"admin", # Global admin domain
}
# Default category for general domain access
DEFAULT_CATEGORY = "general"
logger = get_logger(__name__)
security = HTTPBearer(auto_error=False)
@@ -22,29 +63,42 @@ class OIDCConfig:
def __init__(self):
# These will be set from environment variables in config.py
self.enabled = False
self.issuer = ""
self.audience = ""
self.jwks_uri = ""
self.issuers: list[str] = []
self.audiences: list[str] = []
def configure(self, enabled: bool, issuer: str, audience: str):
def configure(self, enabled: bool, issuers: list[str], audiences: list[str]):
"""Configure OIDC settings"""
self.enabled = enabled
self.issuer = issuer
self.audience = audience
self.jwks_uri = f"{issuer.rstrip('/')}/jwks/"
logger.info(f"OIDC configured: enabled={enabled}, issuer={issuer}")
self.issuers = [iss.rstrip('/') for iss in issuers] # Normalize without trailing slash
self.audiences = audiences
logger.info(f"OIDC configured: enabled={enabled}, issuers={self.issuers}, audiences={audiences}")
def get_jwks_uri(self, issuer: str) -> str:
"""Get JWKS URI for a specific issuer"""
return f"{issuer.rstrip('/')}/jwks/"
def is_valid_issuer(self, issuer: str) -> bool:
"""Check if issuer is in the allowed list"""
normalized = issuer.rstrip('/')
return normalized in self.issuers
# Global OIDC config instance
oidc_config = OIDCConfig()
@lru_cache(maxsize=1)
def get_jwks() -> Dict:
"""
Fetch JSON Web Key Set (JWKS) from Authentik
# Per-issuer JWKS cache
_jwks_cache: Dict[str, Dict] = {}
Cached to avoid repeated requests. Cache is cleared on server restart.
def get_jwks_for_issuer(issuer: str) -> Dict:
"""
Fetch JSON Web Key Set (JWKS) for a specific issuer.
Cached per-issuer to avoid repeated requests. Cache is cleared on server restart.
Args:
issuer: The token issuer URL
Returns:
JWKS dictionary containing public keys for token verification
@@ -55,15 +109,24 @@ def get_jwks() -> Dict:
if not oidc_config.enabled:
return {}
normalized_issuer = issuer.rstrip('/')
# Return cached JWKS if available
if normalized_issuer in _jwks_cache:
return _jwks_cache[normalized_issuer]
jwks_uri = oidc_config.get_jwks_uri(normalized_issuer)
try:
logger.debug(f"Fetching JWKS from {oidc_config.jwks_uri}")
response = httpx.get(oidc_config.jwks_uri, timeout=10.0)
logger.debug(f"Fetching JWKS from {jwks_uri}")
response = httpx.get(jwks_uri, timeout=10.0)
response.raise_for_status()
jwks = response.json()
logger.info(f"JWKS fetched successfully ({len(jwks.get('keys', []))} keys)")
logger.info(f"JWKS fetched successfully for {normalized_issuer} ({len(jwks.get('keys', []))} keys)")
_jwks_cache[normalized_issuer] = jwks
return jwks
except Exception as e:
logger.error(f"Failed to fetch JWKS: {e}")
logger.error(f"Failed to fetch JWKS from {jwks_uri}: {e}")
raise HTTPException(
status_code=503,
detail="Authentication service unavailable"
@@ -116,15 +179,30 @@ async def get_current_user(
token = credentials.credentials
try:
# Decode token header to get key ID
# First, decode token without verification to get issuer and key ID
unverified_header = jwt.get_unverified_header(token)
unverified_claims = jwt.get_unverified_claims(token)
kid = unverified_header.get("kid")
token_issuer = unverified_claims.get("iss", "")
token_audience = unverified_claims.get("aud", "")
if not kid:
raise HTTPException(status_code=401, detail="Invalid token format")
# Find matching key in JWKS
jwks = get_jwks()
# Validate issuer is in allowed list
logger.debug(f"Token issuer: {token_issuer}, allowed issuers: {oidc_config.issuers}")
if not oidc_config.is_valid_issuer(token_issuer):
logger.warning(f"Invalid token issuer: {token_issuer} (allowed: {oidc_config.issuers})")
raise HTTPException(status_code=401, detail="Invalid token issuer")
# Validate audience is in allowed list
if token_audience not in oidc_config.audiences:
logger.warning(f"Invalid token audience: {token_audience} (allowed: {oidc_config.audiences})")
raise HTTPException(status_code=401, detail="Invalid token audience")
# Get JWKS for this specific issuer
jwks = get_jwks_for_issuer(token_issuer)
rsa_key = None
for key in jwks.get("keys", []):
@@ -136,17 +214,17 @@ async def get_current_user(
logger.warning(f"No matching key found for kid: {kid}")
raise HTTPException(status_code=401, detail="Invalid token key")
# Verify and decode token
# Verify and decode token using the token's actual issuer and audience
payload = jwt.decode(
token,
rsa_key,
algorithms=["RS256"],
audience=oidc_config.audience,
issuer=oidc_config.issuer,
audience=token_audience, # Use the token's audience (already validated)
issuer=token_issuer, # Use the token's issuer (already validated)
)
user_email = payload.get("email", "unknown")
logger.info(f"Authenticated user: {user_email}")
logger.info(f"Authenticated user: {user_email} (issuer: {token_issuer})")
return payload
@@ -244,12 +322,14 @@ async def get_optional_user(
}
if not credentials:
logger.debug("No credentials provided for optional auth")
return None
try:
return await get_current_user(credentials)
except HTTPException:
# Invalid token - return None instead of raising
except HTTPException as e:
# Invalid token - log and return None instead of raising
logger.warning(f"Optional auth failed: {e.detail}")
return None
@@ -352,3 +432,345 @@ async def get_forward_auth_admin(
)
return user
async def get_current_user_or_forward_auth(
request: Request,
credentials: Optional[HTTPAuthorizationCredentials] = Security(security)
) -> Dict:
"""
Combined auth: Try forward auth headers first, then JWT Bearer token.
Supports both:
- Web clients via NPM forward auth (X-authentik-* headers from proxy)
- Mobile/native clients via OIDC JWT Bearer tokens
This is the preferred dependency for /auth/users/me and similar endpoints
that need to work with both web (cookie-based via NPM) and mobile (token-based).
Args:
request: FastAPI request object containing headers
credentials: HTTP Bearer token from Authorization header
Returns:
User claims dictionary with at minimum: sub, email, name, groups, auth_method
Raises:
HTTPException 401: If neither forward auth headers nor valid JWT provided
"""
# 1. Try forward auth headers first (web via NPM)
username = request.headers.get("x-authentik-username")
email = request.headers.get("x-authentik-email")
if username and email:
# Forward auth headers present - use them
groups = request.headers.get("x-authentik-groups", "")
name = request.headers.get("x-authentik-name", username)
uid = request.headers.get("x-authentik-uid")
user_info = {
"sub": uid, # Use authentik UID as subject (for user lookup)
"email": email,
"preferred_username": username,
"name": name,
"groups": [g.strip() for g in groups.split(",")] if groups else [],
"auth_method": "forward_auth"
}
logger.info(f"Authenticated via forward auth: {email}")
return user_info
# 2. Fall back to JWT Bearer token (mobile/native)
return await get_current_user(credentials)
# =============================================================================
# Permission-Based Access Control
# =============================================================================
def _parse_permission(permission: str) -> tuple[str, str, str]:
"""
Parse a permission string into (domain, category, action)
Supports formats:
- domain.category:action (full): "control-room.general:admin"
- domain:action (shorthand): "control-room:admin" -> ("control-room", "general", "admin")
Returns:
Tuple of (domain, category, action)
Raises:
ValueError: If permission format is invalid
"""
# Split on colon first to get action
if ":" not in permission:
raise ValueError(f"Invalid permission format (missing ':'): {permission}")
location, action = permission.rsplit(":", 1)
# Split location on dot to get domain and category
if "." in location:
domain, category = location.split(".", 1)
else:
# Shorthand: domain:action -> domain.general:action
domain = location
category = DEFAULT_CATEGORY
return domain, category, action
def _action_satisfies(user_action: str, required_action: str) -> bool:
"""
Check if user's action level satisfies the required action
Due to hierarchy, admin satisfies editor, editor satisfies user, etc.
Args:
user_action: The action the user has
required_action: The action required for access
Returns:
True if user's action is >= required action
"""
user_level = ACTION_HIERARCHY.get(user_action, 0)
required_level = ACTION_HIERARCHY.get(required_action, 0)
return user_level >= required_level
def _user_has_permission(
user_permissions: List[str],
required_domain: str,
required_category: str,
required_action: str,
) -> bool:
"""
Check if user has a permission that satisfies the requirement
Checks:
1. Exact match: domain.category:action
2. Domain-wide: domain.general:action (if category != general)
3. Global admin: admin.general:admin (superuser)
Args:
user_permissions: List of user's permission strings
required_domain: Required domain
required_category: Required category
required_action: Required action
Returns:
True if user has sufficient permission
"""
for perm in user_permissions:
try:
dom, cat, act = _parse_permission(perm)
except ValueError:
continue
# Global admin (admin.general:admin) grants all permissions
if dom == "admin" and cat == "general" and act == "admin":
return True
# Check if this permission covers the requirement
if dom == required_domain:
# Exact category match
if cat == required_category and _action_satisfies(act, required_action):
return True
# Domain-wide permission (general category) covers all categories in domain
if cat == "general" and _action_satisfies(act, required_action):
return True
return False
def _extract_permissions_from_groups(groups: List[str]) -> List[str]:
"""
Extract permission strings from Authentik group names
Authentik groups follow naming: tatlock-{domain}-{category}-{action}
or shorthand: tatlock-{domain}-{action} (implies category=general)
Examples:
- tatlock-control-room-general-admin -> control-room.general:admin
- tatlock-media-viewer -> media.general:viewer (shorthand)
- tatlock-tools-dns-user -> tools.dns:user
Args:
groups: List of Authentik group names
Returns:
List of permission strings
"""
permissions = []
for group in groups:
if not group.startswith("tatlock-"):
continue
# Remove prefix
parts = group[8:].split("-") # Remove "tatlock-"
if len(parts) >= 3:
# Could be domain-category-action or domain-with-hyphen-action
# Try to find a valid action at the end
action = parts[-1]
if action in ACTION_HIERARCHY:
# Check if domain-category or single domain with hyphen
remaining = parts[:-1]
# Try to find known domain (greedy match from start)
for i in range(len(remaining), 0, -1):
potential_domain = "-".join(remaining[:i])
if potential_domain in VALID_DOMAINS:
category_parts = remaining[i:]
category = "-".join(category_parts) if category_parts else DEFAULT_CATEGORY
permissions.append(f"{potential_domain}.{category}:{action}")
break
elif len(parts) == 2:
# Shorthand: domain-action (domain might have hyphen)
action = parts[-1]
if action in ACTION_HIERARCHY:
domain = parts[0]
if domain in VALID_DOMAINS:
permissions.append(f"{domain}.{DEFAULT_CATEGORY}:{action}")
return permissions
def require_permission(
domain: str,
action: str,
category: str = DEFAULT_CATEGORY,
) -> Callable:
"""
Dependency factory for permission-based access control
Creates a FastAPI dependency that checks if the current user has
the required permission. Considers action hierarchy and global admin.
Usage:
@router.get("/containers")
async def list_containers(
user: Dict = Depends(require_permission("control-room", "viewer"))
):
...
@router.delete("/container/{id}")
async def delete_container(
user: Dict = Depends(require_permission("control-room", "admin"))
):
...
Args:
domain: Permission domain (e.g., "control-room", "media")
action: Required action level (viewer, user, editor, admin)
category: Permission category within domain, defaults to "general"
Returns:
FastAPI dependency function
"""
perm_str = f"{domain}.{category}:{action}"
async def permission_checker(
user: Optional[Dict] = Depends(get_current_user)
) -> Dict:
"""Check if user has required permission"""
# If OIDC disabled, allow all (local dev mode)
if not oidc_config.enabled:
logger.debug(f"OIDC disabled - allowing {perm_str}")
return user or {"email": "local", "groups": ["admin"]}
if user is None:
raise HTTPException(
status_code=401,
detail="Authentication required",
headers={"WWW-Authenticate": "Bearer"},
)
# Extract permissions from user's groups
groups = user.get("groups", [])
permissions = _extract_permissions_from_groups(groups)
# Check if user has required permission
if _user_has_permission(permissions, domain, category, action):
logger.debug(f"User {user.get('email')} granted {perm_str}")
return user
# Permission denied
user_email = user.get("email", "unknown")
logger.warning(
f"User {user_email} denied {perm_str} "
f"(groups: {groups}, permissions: {permissions})"
)
raise HTTPException(
status_code=403,
detail=f"Permission required: {perm_str}",
)
return permission_checker
def require_any_permission(*required_permissions: str) -> Callable:
"""
Dependency factory requiring any one of multiple permissions
Useful for endpoints accessible to multiple roles.
Usage:
@router.get("/shared-resource")
async def get_shared(
user: Dict = Depends(require_any_permission(
"control-room:viewer",
"media:viewer",
))
):
...
Args:
*required_permissions: Permission strings (domain.category:action or domain:action)
Returns:
FastAPI dependency function
"""
async def permission_checker(
user: Optional[Dict] = Depends(get_current_user)
) -> Dict:
"""Check if user has any of the required permissions"""
# If OIDC disabled, allow all
if not oidc_config.enabled:
return user or {"email": "local", "groups": ["admin"]}
if user is None:
raise HTTPException(
status_code=401,
detail="Authentication required",
headers={"WWW-Authenticate": "Bearer"},
)
groups = user.get("groups", [])
permissions = _extract_permissions_from_groups(groups)
# Check each required permission
for perm in required_permissions:
try:
dom, cat, act = _parse_permission(perm)
if _user_has_permission(permissions, dom, cat, act):
logger.debug(f"User {user.get('email')} granted via {perm}")
return user
except ValueError:
logger.warning(f"Invalid permission format: {perm}")
continue
# None matched
user_email = user.get("email", "unknown")
logger.warning(
f"User {user_email} denied (required any of: {required_permissions})"
)
raise HTTPException(
status_code=403,
detail=f"One of these permissions required: {', '.join(required_permissions)}",
)
return permission_checker
+84 -2
View File
@@ -26,10 +26,12 @@ class AuthSyncRequest(BaseSchema):
class RoleSchema(BaseSchema):
"""Role information in domain:action format"""
"""Role information in domain.category:action format"""
name: str = Field(..., description="Role name (e.g., 'control-room:admin')")
id: uuid.UUID = Field(..., description="Role ID")
name: str = Field(..., description="Role name (e.g., 'control-room.general:admin')")
domain: str = Field(..., description="Permission domain (e.g., 'control-room')")
category: str = Field(default="general", description="Permission category (e.g., 'general')")
action: str = Field(..., description="Permission action (e.g., 'admin')")
@@ -120,6 +122,7 @@ class GroupListItemSchema(BaseSchema):
parent_name: Optional[str] = Field(None, description="Parent group name")
member_count: int = Field(default=0, description="Number of users in this group")
synced_at: datetime = Field(..., description="Last sync timestamp")
roles: list[str] = Field(default_factory=list, description="Assigned role names")
class GroupsListResponse(BaseSchema):
@@ -127,3 +130,82 @@ class GroupsListResponse(BaseSchema):
items: list[GroupListItemSchema] = Field(..., description="List of groups")
total: int = Field(..., description="Total count of groups")
class RolesListResponse(BaseSchema):
"""Response from GET /auth/roles"""
items: list[RoleSchema] = Field(..., description="List of all roles")
total: int = Field(..., description="Total count of roles")
class GroupRoleAssignmentResponse(BaseSchema):
"""Response from group role assignment operations"""
group_id: uuid.UUID = Field(..., description="Group ID")
group_name: str = Field(..., description="Group name")
roles: list[str] = Field(..., description="Currently assigned role names")
# =============================================================================
# User Profile (Phase 4)
# =============================================================================
class UserProfileResponse(BaseSchema):
"""Response from GET /users/me - full user profile"""
user: UserSchema = Field(..., description="User profile")
roles: list[RoleSchema] = Field(..., description="User's permission roles")
preferences: UserPreferencesSchema = Field(..., description="User preferences")
class PreferencesUpdateRequest(BaseSchema):
"""Request for PATCH /users/me/preferences"""
theme: Optional[str] = Field(None, description="Theme preference: system, light, dark")
default_room: Optional[str] = Field(None, description="Default room for housekeeping")
preferences_json: Optional[dict] = Field(None, description="Extended preferences (merged)")
# =============================================================================
# API Keys (Phase 4)
# =============================================================================
class ApiKeyCreateRequest(BaseSchema):
"""Request for POST /users/me/api-keys"""
name: str = Field(..., min_length=1, max_length=100, description="Human-readable key name")
scopes: Optional[list[str]] = Field(None, description="Optional scope restriction (role names)")
expires_in_days: Optional[int] = Field(None, ge=1, le=365, description="Days until expiration (optional)")
class ApiKeyCreateResponse(BaseSchema):
"""Response from POST /users/me/api-keys - includes the key (shown only once)"""
id: uuid.UUID = Field(..., description="API key ID")
name: str = Field(..., description="Key name")
key: str = Field(..., description="The API key (shown only once!)")
key_prefix: str = Field(..., description="Key prefix for identification")
scopes: Optional[list[str]] = Field(None, description="Scope restriction")
expires_at: Optional[datetime] = Field(None, description="Expiration timestamp")
created_at: datetime = Field(..., description="Creation timestamp")
class ApiKeySchema(BaseSchema):
"""API key information (without the actual key)"""
id: uuid.UUID = Field(..., description="API key ID")
name: str = Field(..., description="Key name")
key_prefix: str = Field(..., description="Key prefix for identification (e.g., 'tak_abc1')")
scopes: Optional[list[str]] = Field(None, description="Scope restriction")
expires_at: Optional[datetime] = Field(None, description="Expiration timestamp")
last_used_at: Optional[datetime] = Field(None, description="Last usage timestamp")
created_at: datetime = Field(..., description="Creation timestamp")
is_expired: bool = Field(..., description="Whether the key has expired")
class ApiKeysListResponse(BaseSchema):
"""Response from GET /users/me/api-keys"""
items: list[ApiKeySchema] = Field(..., description="List of API keys")
total: int = Field(..., description="Total count of keys")
+466 -17
View File
@@ -3,9 +3,11 @@ Authentication Service
Business logic for user synchronization from Authentik.
"""
import hashlib
import re
import secrets
import uuid
from datetime import datetime, timezone
from datetime import datetime, timezone, timedelta
from typing import Optional
import httpx
@@ -15,10 +17,11 @@ from sqlalchemy.orm import selectinload
from src.shared.config import get_settings
from src.shared.logging import get_logger
from src.domains.auth.models import User, Role, UserPreferences, Group
from src.domains.auth.models import User, Role, UserPreferences, Group, ApiKey
from src.domains.auth.schemas import (
TokenInfoSchema, UserSchema, RoleSchema, UserPreferencesSchema,
UserListItemSchema, BulkSyncResultSchema, GroupListItemSchema
UserListItemSchema, BulkSyncResultSchema, GroupListItemSchema,
ApiKeySchema, ApiKeyCreateResponse,
)
logger = get_logger(__name__)
@@ -99,7 +102,13 @@ class AuthService:
Returns:
Tuple of (User, is_new_user)
"""
authentik_id = uuid.UUID(token_info.sub)
# Parse authentik_id - may be UUID or other format
try:
authentik_id = uuid.UUID(token_info.sub)
except ValueError:
# If sub is not a valid UUID, derive one deterministically
logger.warning(f"sub claim is not a UUID: {token_info.sub}, deriving UUID")
authentik_id = uuid.uuid5(uuid.NAMESPACE_OID, token_info.sub)
# Try to find existing user
stmt = (
@@ -121,11 +130,14 @@ class AuthService:
avatar_url=token_info.picture,
last_login=datetime.now(timezone.utc),
)
# Initialize relationships to avoid lazy loading issues in async
user.roles = []
self.session.add(user)
await self.session.flush() # Get the user ID
# Create default preferences
# Create default preferences and attach to user
preferences = UserPreferences(user_id=user.id)
user.preferences = preferences
self.session.add(preferences)
logger.info(f"Created new user: {token_info.email}")
@@ -141,30 +153,111 @@ class AuthService:
await self.session.flush()
return user, is_new
async def sync_roles(self, user: User, groups: list[str]) -> list[Role]:
async def sync_user_from_claims(
self,
authentik_id: uuid.UUID,
email: str,
name: str,
groups: list[str],
avatar_url: Optional[str] = None,
) -> tuple[User, bool]:
"""
Synchronize user roles from Authentik groups
Create or update user from forward auth claims (NPM X-authentik-* headers)
Maps Authentik groups (e.g., 'tatlock-control-room-admin')
to application roles (e.g., 'control-room:admin').
This is similar to sync_user() but works with raw claims instead of
TokenInfoSchema. Used for auto-syncing users on first web login via NPM.
Args:
authentik_id: The Authentik user UUID (from X-authentik-uid)
email: User email (from X-authentik-email)
name: User display name (from X-authentik-name)
groups: List of group names (from X-authentik-groups)
avatar_url: Optional avatar URL
Returns:
Tuple of (User, is_new_user)
"""
# Try to find existing user
stmt = (
select(User)
.options(selectinload(User.roles), selectinload(User.preferences))
.where(User.authentik_id == authentik_id)
)
result = await self.session.execute(stmt)
user = result.scalar_one_or_none()
is_new = user is None
if is_new:
# Create new user
user = User(
authentik_id=authentik_id,
email=email,
name=name or email,
avatar_url=avatar_url,
last_login=datetime.now(timezone.utc),
)
self.session.add(user)
await self.session.flush() # Get the user ID
# Create default preferences
preferences = UserPreferences(user_id=user.id)
self.session.add(preferences)
logger.info(f"Created new user from forward auth: {email}")
else:
# Update existing user
user.email = email
user.name = name or email
if avatar_url:
user.avatar_url = avatar_url
user.last_login = datetime.now(timezone.utc)
logger.info(f"Updated existing user from forward auth: {email}")
# Sync roles from groups
await self.sync_roles(user, groups)
await self.session.flush()
return user, is_new
async def sync_roles(self, user: User, group_names: list[str]) -> list[Role]:
"""
Synchronize user roles from Authentik groups via group_roles mapping
Looks up the user's groups in the database, then retrieves all roles
assigned to those groups via the group_roles mapping table.
Args:
user: User to sync roles for
groups: List of Authentik group names
group_names: List of Authentik group names
Returns:
List of synced Role objects
"""
# Get all roles that match the user's Authentik groups
stmt = select(Role).where(Role.authentik_group.in_(groups))
# Find local Group records matching the Authentik group names
stmt = (
select(Group)
.options(selectinload(Group.roles))
.where(Group.name.in_(group_names))
)
result = await self.session.execute(stmt)
matching_roles = list(result.scalars().all())
matching_groups = list(result.scalars().all())
# Collect all unique roles from all matching groups
roles_set: dict[uuid.UUID, Role] = {}
for group in matching_groups:
for role in group.roles:
roles_set[role.id] = role
matching_roles = list(roles_set.values())
# Clear existing roles and set new ones
user.roles = matching_roles
role_names = [r.name for r in matching_roles]
logger.info(f"Synced roles for {user.email}: {role_names}")
group_names_found = [g.name for g in matching_groups]
logger.info(f"Synced roles for {user.email} via groups {group_names_found}: {role_names}")
return matching_roles
@@ -183,7 +276,7 @@ class AuthService:
def roles_to_schema(self, roles: list[Role]) -> list[RoleSchema]:
"""Convert Role models to schemas"""
return [
RoleSchema(name=r.name, domain=r.domain, action=r.action)
RoleSchema(id=r.id, name=r.name, domain=r.domain, category=r.category, action=r.action)
for r in roles
]
@@ -492,8 +585,8 @@ class AuthService:
"""
from sqlalchemy import func
# Base query
base_query = select(Group)
# Base query with roles loaded
base_query = select(Group).options(selectinload(Group.roles))
# Apply search filter if provided
if search:
@@ -520,12 +613,106 @@ class AuthService:
parent_name=group.parent_name,
member_count=group.member_count,
synced_at=group.synced_at,
roles=[role.name for role in group.roles],
)
for group in groups
]
return items, total
async def list_roles(self) -> list[Role]:
"""
List all available roles
Returns:
List of all Role objects
"""
stmt = select(Role).order_by(Role.domain, Role.category, Role.action)
result = await self.session.execute(stmt)
return list(result.scalars().all())
async def get_group_by_id(self, group_id: uuid.UUID) -> Optional[Group]:
"""
Get a group by its ID with roles loaded
Args:
group_id: The group's UUID
Returns:
Group object or None if not found
"""
stmt = (
select(Group)
.options(selectinload(Group.roles))
.where(Group.id == group_id)
)
result = await self.session.execute(stmt)
return result.scalar_one_or_none()
async def assign_role_to_group(self, group_id: uuid.UUID, role_id: uuid.UUID) -> Group:
"""
Assign a role to a group
Args:
group_id: The group's UUID
role_id: The role's UUID to assign
Returns:
Updated Group object
Raises:
ValueError: If group or role not found
"""
group = await self.get_group_by_id(group_id)
if not group:
raise ValueError(f"Group not found: {group_id}")
stmt = select(Role).where(Role.id == role_id)
result = await self.session.execute(stmt)
role = result.scalar_one_or_none()
if not role:
raise ValueError(f"Role not found: {role_id}")
# Add role if not already assigned
if role not in group.roles:
group.roles.append(role)
await self.session.flush()
logger.info(f"Assigned role {role.name} to group {group.name}")
return group
async def remove_role_from_group(self, group_id: uuid.UUID, role_id: uuid.UUID) -> Group:
"""
Remove a role from a group
Args:
group_id: The group's UUID
role_id: The role's UUID to remove
Returns:
Updated Group object
Raises:
ValueError: If group or role not found
"""
group = await self.get_group_by_id(group_id)
if not group:
raise ValueError(f"Group not found: {group_id}")
stmt = select(Role).where(Role.id == role_id)
result = await self.session.execute(stmt)
role = result.scalar_one_or_none()
if not role:
raise ValueError(f"Role not found: {role_id}")
# Remove role if assigned
if role in group.roles:
group.roles.remove(role)
await self.session.flush()
logger.info(f"Removed role {role.name} from group {group.name}")
return group
async def bulk_sync_groups_from_authentik(self) -> BulkSyncResultSchema:
"""
Fetch all groups from Authentik admin API and sync to local database
@@ -627,6 +814,268 @@ class AuthService:
errors=errors,
)
# =========================================================================
# Phase 4: User Profile & Settings
# =========================================================================
async def get_user_by_authentik_id(self, authentik_id: uuid.UUID) -> Optional[User]:
"""
Get user by Authentik UUID with roles and preferences loaded
Args:
authentik_id: The Authentik user UUID (from JWT 'sub' claim)
Returns:
User object or None if not found
"""
stmt = (
select(User)
.options(selectinload(User.roles), selectinload(User.preferences))
.where(User.authentik_id == authentik_id)
)
result = await self.session.execute(stmt)
return result.scalar_one_or_none()
async def get_user_by_id(self, user_id: uuid.UUID) -> Optional[User]:
"""
Get user by internal UUID with roles and preferences loaded
Args:
user_id: The internal user UUID
Returns:
User object or None if not found
"""
stmt = (
select(User)
.options(selectinload(User.roles), selectinload(User.preferences))
.where(User.id == user_id)
)
result = await self.session.execute(stmt)
return result.scalar_one_or_none()
async def update_preferences(
self,
user_id: uuid.UUID,
theme: Optional[str] = None,
default_room: Optional[str] = None,
preferences_json: Optional[dict] = None,
) -> UserPreferences:
"""
Update user preferences
Args:
user_id: User's UUID
theme: New theme value (or None to keep existing)
default_room: New default room (or None to keep existing)
preferences_json: JSON to merge with existing (or None to keep existing)
Returns:
Updated UserPreferences object
Raises:
ValueError: If user not found
"""
stmt = select(UserPreferences).where(UserPreferences.user_id == user_id)
result = await self.session.execute(stmt)
prefs = result.scalar_one_or_none()
if prefs is None:
# Create preferences if they don't exist
prefs = UserPreferences(user_id=user_id)
self.session.add(prefs)
if theme is not None:
if theme not in ("system", "light", "dark"):
raise ValueError(f"Invalid theme: {theme}")
prefs.theme = theme
if default_room is not None:
prefs.default_room = default_room
if preferences_json is not None:
# Merge with existing preferences
existing = prefs.preferences_json or {}
existing.update(preferences_json)
prefs.preferences_json = existing
await self.session.flush()
logger.info(f"Updated preferences for user {user_id}")
return prefs
# =========================================================================
# Phase 4: API Keys
# =========================================================================
def _generate_api_key(self) -> tuple[str, str, str]:
"""
Generate a new API key
Returns:
Tuple of (full_key, key_hash, key_prefix)
"""
# Generate 32 random bytes = 256 bits of entropy
random_bytes = secrets.token_bytes(32)
# Encode as base64-like string (URL-safe)
key_body = secrets.token_urlsafe(32)
# Prefix with 'tak_' (tatlock api key)
full_key = f"tak_{key_body}"
# Hash for storage
key_hash = hashlib.sha256(full_key.encode()).hexdigest()
# Prefix for identification (first 8 chars after 'tak_')
key_prefix = f"tak_{key_body[:4]}"
return full_key, key_hash, key_prefix
async def create_api_key(
self,
user_id: uuid.UUID,
name: str,
scopes: Optional[list[str]] = None,
expires_in_days: Optional[int] = None,
) -> tuple[ApiKey, str]:
"""
Create a new API key for a user
Args:
user_id: User's UUID
name: Human-readable key name
scopes: Optional list of scope restrictions
expires_in_days: Optional expiration in days
Returns:
Tuple of (ApiKey object, full key string)
The full key is only returned once at creation!
Raises:
ValueError: If user not found or API keys disabled
"""
# Check user exists and has API keys enabled
user = await self.get_user_by_id(user_id)
if user is None:
raise ValueError(f"User not found: {user_id}")
if not user.api_keys_enabled:
raise ValueError("API keys are disabled for this user")
# Generate the key
full_key, key_hash, key_prefix = self._generate_api_key()
# Calculate expiration
expires_at = None
if expires_in_days:
expires_at = datetime.now(timezone.utc) + timedelta(days=expires_in_days)
# Create the key record
api_key = ApiKey(
user_id=user_id,
name=name,
key_hash=key_hash,
key_prefix=key_prefix,
scopes=scopes,
expires_at=expires_at,
)
self.session.add(api_key)
await self.session.flush()
logger.info(f"Created API key '{name}' for user {user_id}")
return api_key, full_key
async def list_user_api_keys(self, user_id: uuid.UUID) -> list[ApiKey]:
"""
List all API keys for a user
Args:
user_id: User's UUID
Returns:
List of ApiKey objects (without the actual keys)
"""
stmt = (
select(ApiKey)
.where(ApiKey.user_id == user_id)
.order_by(ApiKey.created_at.desc())
)
result = await self.session.execute(stmt)
return list(result.scalars().all())
async def delete_api_key(self, user_id: uuid.UUID, key_id: uuid.UUID) -> bool:
"""
Delete an API key
Args:
user_id: User's UUID (for authorization)
key_id: API key UUID
Returns:
True if deleted, False if not found
Raises:
ValueError: If key belongs to different user
"""
stmt = select(ApiKey).where(ApiKey.id == key_id)
result = await self.session.execute(stmt)
api_key = result.scalar_one_or_none()
if api_key is None:
return False
if api_key.user_id != user_id:
raise ValueError("API key belongs to different user")
await self.session.delete(api_key)
await self.session.flush()
logger.info(f"Deleted API key {key_id} for user {user_id}")
return True
async def validate_api_key(self, key: str) -> Optional[User]:
"""
Validate an API key and return the associated user
Args:
key: The full API key string
Returns:
User object if valid, None if invalid/expired
"""
# Hash the provided key
key_hash = hashlib.sha256(key.encode()).hexdigest()
# Look up by hash
stmt = (
select(ApiKey)
.options(selectinload(ApiKey.user).selectinload(User.roles))
.where(ApiKey.key_hash == key_hash)
)
result = await self.session.execute(stmt)
api_key = result.scalar_one_or_none()
if api_key is None:
return None
# Check expiration
if api_key.is_expired:
logger.warning(f"Expired API key used: {api_key.key_prefix}...")
return None
# Update last used timestamp
api_key.last_used_at = datetime.now(timezone.utc)
logger.debug(f"API key authenticated: {api_key.key_prefix}... for user {api_key.user.email}")
return api_key.user
def api_key_to_schema(self, api_key: ApiKey) -> ApiKeySchema:
"""Convert ApiKey model to schema"""
return ApiKeySchema(
id=api_key.id,
name=api_key.name,
key_prefix=api_key.key_prefix,
scopes=api_key.scopes,
expires_at=api_key.expires_at,
last_used_at=api_key.last_used_at,
created_at=api_key.created_at,
is_expired=api_key.is_expired,
)
# Factory function for dependency injection
def get_auth_service(session: AsyncSession) -> AuthService:
+9
View File
@@ -0,0 +1,9 @@
"""
Book Domain
The household's Notes, Tasks, Decisions, Questions and Rejected options
(workspace D-68, D-69, D-70). Served under /book/*.
"""
from src.domains.book.controller import book_controller
__all__ = ["book_controller"]
+327
View File
@@ -0,0 +1,327 @@
"""
Book Controller
The core Book verbs (D-70): add, show, list, find, change, close,
link/unlink, tag/untag, history. Every route requires a caller (office or
user); writes additionally require an identity (T-1). Every route also
requires a tenant on the caller — enforced in the service layer
(`BookService._require_tenant`) rather than here, so reads and writes share
one check (core-api T-7).
"""
import uuid
from typing import Optional
from fastapi import APIRouter, Depends, HTTPException, Query
from src.domains.auth.callers import Caller, require_caller
from src.domains.book.schemas import (
BookCloseRequest,
BookEventResponse,
BookHistoryResponse,
BookLinkRequest,
BookLinkResponse,
BookRecordCreate,
BookRecordDetailResponse,
BookRecordListResponse,
BookRecordResponse,
BookRecordUpdate,
BookTagRequest,
ListView,
)
from src.domains.book.service import BookError, get_book_service
from src.shared.base import BaseController
from src.shared.database import get_async_session
from src.shared.logging import get_logger
from sqlalchemy.ext.asyncio import AsyncSession
logger = get_logger(__name__)
require_book_reader = require_caller("office", "user")
require_book_writer = require_caller("office", "user", identity_required=True)
def _record_response(record, tags) -> BookRecordResponse:
schema = BookRecordResponse.model_validate(record, from_attributes=True)
return schema.model_copy(update={"tags": tags})
def _link_response(link) -> BookLinkResponse:
return BookLinkResponse.model_validate(link, from_attributes=True)
def _detail_response(record, tags, links, backlinks) -> BookRecordDetailResponse:
base = BookRecordResponse.model_validate(record, from_attributes=True).model_dump()
base["tags"] = tags
return BookRecordDetailResponse(
**base,
links=[_link_response(link) for link in links],
backlinks=[_link_response(link) for link in backlinks],
)
def _event_response(event) -> BookEventResponse:
return BookEventResponse.model_validate(event, from_attributes=True)
class BookController(BaseController):
"""Controller for the Book domain."""
def __init__(self):
super().__init__(prefix="/book", tags=["Book"])
def create_router(self) -> APIRouter:
router = APIRouter(prefix=self.prefix, tags=self.tags)
service = get_book_service()
# =====================================================================
# add
# =====================================================================
@router.post(
"/records",
response_model=BookRecordResponse,
status_code=201,
summary="Add a record to the Book",
)
async def add_record(
data: BookRecordCreate,
caller: Caller = Depends(require_book_writer),
session: AsyncSession = Depends(get_async_session),
):
try:
record, tags = await service.add_record(session, data, caller)
except BookError as error:
raise HTTPException(status_code=error.status_code, detail={"code": error.code}) from error
return _record_response(record, tags)
# =====================================================================
# list
# =====================================================================
@router.get(
"/records",
response_model=BookRecordListResponse,
summary="List records",
)
async def list_records(
view: Optional[ListView] = Query(None, description="inbox, today, upcoming, waiting, someday, pinned, archive, trash, done"),
tag: Optional[str] = Query(None),
scope: Optional[str] = Query(None),
parent: Optional[uuid.UUID] = Query(None),
number: Optional[int] = Query(None, description="find the one record with this outward id"),
limit: int = Query(50, ge=1, le=200),
offset: int = Query(0, ge=0),
caller: Caller = Depends(require_book_reader),
session: AsyncSession = Depends(get_async_session),
):
try:
results, total = await service.list_records(
session, caller, view=view, tag=tag, scope=scope, parent=parent,
number=number, limit=limit, offset=offset,
)
except BookError as error:
raise HTTPException(status_code=error.status_code, detail={"code": error.code}) from error
return BookRecordListResponse(
records=[_record_response(record, tags) for record, tags in results],
total=total,
limit=limit,
offset=offset,
)
# =====================================================================
# find — declared before /{record_id} so "find" is never parsed as an id
# =====================================================================
@router.get(
"/records/find",
response_model=BookRecordListResponse,
summary="Full-text search the Book",
)
async def find_records(
q: str = Query(..., min_length=1),
limit: int = Query(50, ge=1, le=200),
offset: int = Query(0, ge=0),
caller: Caller = Depends(require_book_reader),
session: AsyncSession = Depends(get_async_session),
):
try:
results, total = await service.find_records(session, caller, q=q, limit=limit, offset=offset)
except BookError as error:
raise HTTPException(status_code=error.status_code, detail={"code": error.code}) from error
return BookRecordListResponse(
records=[_record_response(record, tags) for record, tags in results],
total=total,
limit=limit,
offset=offset,
)
# =====================================================================
# show
# =====================================================================
@router.get(
"/records/{record_id}",
response_model=BookRecordDetailResponse,
summary="Show a record, its links and its tags",
)
async def show_record(
record_id: uuid.UUID,
caller: Caller = Depends(require_book_reader),
session: AsyncSession = Depends(get_async_session),
):
try:
record, tags, links, backlinks = await service.show_record(session, record_id, caller)
except BookError as error:
raise HTTPException(status_code=error.status_code, detail={"code": error.code}) from error
return _detail_response(record, tags, links, backlinks)
# =====================================================================
# change
# =====================================================================
@router.patch(
"/records/{record_id}",
response_model=BookRecordResponse,
summary="Change a record",
)
async def change_record(
record_id: uuid.UUID,
data: BookRecordUpdate,
caller: Caller = Depends(require_book_writer),
session: AsyncSession = Depends(get_async_session),
):
try:
record, tags = await service.change_record(session, record_id, data, caller)
except BookError as error:
raise HTTPException(status_code=error.status_code, detail={"code": error.code}) from error
return _record_response(record, tags)
# =====================================================================
# close
# =====================================================================
@router.post(
"/records/{record_id}/close",
response_model=BookRecordResponse,
summary="Close a record (task done/dropped, question resolved, decision superseded)",
)
async def close_record(
record_id: uuid.UUID,
data: BookCloseRequest,
caller: Caller = Depends(require_book_writer),
session: AsyncSession = Depends(get_async_session),
):
try:
record, tags = await service.close_record(session, record_id, data, caller)
except BookError as error:
raise HTTPException(status_code=error.status_code, detail={"code": error.code}) from error
return _record_response(record, tags)
# =====================================================================
# link / unlink
# =====================================================================
@router.post(
"/records/{record_id}/link",
response_model=BookRecordResponse,
summary="Link a record to another",
)
async def link_record(
record_id: uuid.UUID,
data: BookLinkRequest,
caller: Caller = Depends(require_book_writer),
session: AsyncSession = Depends(get_async_session),
):
try:
record, tags = await service.link_records(
session, record_id, data.to_id, data.relation, caller
)
except BookError as error:
raise HTTPException(status_code=error.status_code, detail={"code": error.code}) from error
return _record_response(record, tags)
@router.post(
"/records/{record_id}/unlink",
response_model=BookRecordResponse,
summary="Remove a link between two records",
)
async def unlink_record(
record_id: uuid.UUID,
data: BookLinkRequest,
caller: Caller = Depends(require_book_writer),
session: AsyncSession = Depends(get_async_session),
):
try:
record, tags = await service.unlink_records(
session, record_id, data.to_id, data.relation, caller
)
except BookError as error:
raise HTTPException(status_code=error.status_code, detail={"code": error.code}) from error
return _record_response(record, tags)
# =====================================================================
# tag / untag
# =====================================================================
@router.post(
"/records/{record_id}/tag",
response_model=BookRecordResponse,
summary="Tag a record",
)
async def tag_record(
record_id: uuid.UUID,
data: BookTagRequest,
caller: Caller = Depends(require_book_writer),
session: AsyncSession = Depends(get_async_session),
):
try:
record, tags = await service.tag_record(session, record_id, data.tag, caller)
except BookError as error:
raise HTTPException(status_code=error.status_code, detail={"code": error.code}) from error
return _record_response(record, tags)
@router.post(
"/records/{record_id}/untag",
response_model=BookRecordResponse,
summary="Remove a tag from a record",
)
async def untag_record(
record_id: uuid.UUID,
data: BookTagRequest,
caller: Caller = Depends(require_book_writer),
session: AsyncSession = Depends(get_async_session),
):
try:
record, tags = await service.untag_record(session, record_id, data.tag, caller)
except BookError as error:
raise HTTPException(status_code=error.status_code, detail={"code": error.code}) from error
return _record_response(record, tags)
# =====================================================================
# history
# =====================================================================
@router.get(
"/records/{record_id}/history",
response_model=BookHistoryResponse,
summary="The append-only event history for a record",
)
async def record_history(
record_id: uuid.UUID,
caller: Caller = Depends(require_book_reader),
session: AsyncSession = Depends(get_async_session),
):
try:
events = await service.get_history(session, record_id, caller)
except BookError as error:
raise HTTPException(status_code=error.status_code, detail={"code": error.code}) from error
return BookHistoryResponse(
record_id=record_id,
events=[_event_response(event) for event in events],
)
return router
# Create controller instance
book_controller = BookController()
+290
View File
@@ -0,0 +1,290 @@
"""
Book Domain Models
The Book holds Notes, Tasks, Decisions, Questions and Rejected options for
the Office (workspace D-68, D-69, D-70). SQLAlchemy 2.0 Mapped/mapped_column
style, timestamptz, following src/domains/auth/models.py.
Table prefix is `book_` (D-68). `book_events` is append-only — migration 005
adds a trigger that raises on UPDATE and DELETE, enforced at the database,
not here.
"""
import uuid
from datetime import datetime
from typing import Optional
from sqlalchemy import (
Boolean,
Computed,
DateTime,
ForeignKey,
Index,
Integer,
String,
Text,
UniqueConstraint,
func,
)
from sqlalchemy.dialects.postgresql import JSONB, TSVECTOR, UUID
from sqlalchemy.orm import Mapped, mapped_column
from src.shared.database import Base
# Kinds and their valid statuses. Enforced in the service layer (per kind),
# not as a database CHECK — the valid status set depends on the kind, which
# a single-column CHECK cannot express without a trigger.
RECORD_KINDS = ("note", "task", "decision", "question", "rejected")
STATUSES_BY_KIND = {
"note": {"active"},
"task": {"inbox", "open", "waiting", "someday", "done", "dropped"},
"decision": {"proposed", "accepted", "superseded"},
"question": {"open", "settled", "dropped"},
"rejected": {"rejected"},
}
DEFAULT_STATUS_BY_KIND = {
"note": "active",
"task": "inbox",
"decision": "proposed",
"question": "open",
"rejected": "rejected",
}
SCOPES = ("work", "personal")
DEFAULT_SCOPE = "personal"
LINK_RELATIONS = ("parent", "blocks", "resolves", "supersedes", "related")
class BookRecord(Base):
"""A Note, Task, Decision, Question or Rejected option."""
__tablename__ = "book_records"
__table_args__ = (
UniqueConstraint("source", "external_id", name="uq_book_records_source_external_id"),
UniqueConstraint("owner_tenant", "number", name="uq_book_records_tenant_number"),
Index("ix_book_records_tenant_kind_status", "owner_tenant", "kind", "status"),
)
id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
primary_key=True,
default=uuid.uuid4,
)
# The tatlock tenant the record belongs to (Caller.tenant, T-4/D-73).
# Every query in the service layer filters on it — there is no
# database-level RLS here, so that filter is load-bearing in Python,
# following src/domains/scribe/models.py's same note. FK to
# tenants.id added by migration 007 (core-api T-8).
owner_tenant: Mapped[str] = mapped_column(
String(63), ForeignKey("tenants.id", ondelete="RESTRICT"), nullable=False
)
# A short outward id, numbered per tenant (never globally) so a tenant
# can never infer another's activity from its numbers. Allocated by
# BookService._allocate_number, which locks a per-tenant counter row
# FOR UPDATE in the same transaction as this row's insert.
number: Mapped[int] = mapped_column(Integer, nullable=False)
kind: Mapped[str] = mapped_column(String(20), nullable=False, index=True)
title: Mapped[Optional[str]] = mapped_column(String(500), nullable=True)
body: Mapped[str] = mapped_column(Text, nullable=False, default="")
checklist: Mapped[list] = mapped_column(JSONB, nullable=False, default=list)
status: Mapped[str] = mapped_column(String(20), nullable=False, index=True)
scope: Mapped[str] = mapped_column(String(10), nullable=False, default=DEFAULT_SCOPE)
priority: Mapped[Optional[int]] = mapped_column(Integer, nullable=True)
pinned: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
energy: Mapped[Optional[str]] = mapped_column(String(10), nullable=True)
estimate_minutes: Mapped[Optional[int]] = mapped_column(Integer, nullable=True)
next_step: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
start_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
due_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True, index=True)
all_day: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False)
remind_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
recurrence: Mapped[Optional[str]] = mapped_column(String(255), nullable=True)
recurrence_mode: Mapped[Optional[str]] = mapped_column(String(20), nullable=True)
remind_to: Mapped[Optional[str]] = mapped_column(String(63), nullable=True)
last_touched_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
server_default=func.now(),
nullable=False,
)
archived_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
trashed_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
paused_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
paused_statuses: Mapped[Optional[dict]] = mapped_column(JSONB, nullable=True)
source: Mapped[Optional[str]] = mapped_column(String(50), nullable=True)
external_id: Mapped[Optional[str]] = mapped_column(String(255), nullable=True)
colour: Mapped[Optional[str]] = mapped_column(String(20), nullable=True)
created_by: Mapped[str] = mapped_column(String(63), nullable=False)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
server_default=func.now(),
nullable=False,
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
server_default=func.now(),
onupdate=func.now(),
nullable=False,
)
# Generated by migration 005 (`to_tsvector('simple', ...)`, GIN-indexed).
# 'simple' because the household writes mixed Dutch and English — a
# language-specific config would stem one of the two badly. `Computed`
# mirrors the migration's SQL so SQLAlchemy knows this column is
# database-generated and excludes it from INSERT/UPDATE — without it,
# the ORM tries to write NULL into a GENERATED ALWAYS column and
# Postgres rejects the insert outright.
search: Mapped[Optional[str]] = mapped_column(
TSVECTOR,
Computed("to_tsvector('simple', coalesce(title, '') || ' ' || body)", persisted=True),
nullable=True,
)
# Deliberately no `tags` relationship here. The service always queries
# book_tags explicitly (`BookService._tags_for`) rather than traversing
# an ORM relationship — partly because async SQLAlchemy can't lazy-load
# on an unloaded attribute without a greenlet error, but mainly because
# a relationship named `tags` collides with the response schema's
# `tags: List[str]` field: `BookRecordResponse.model_validate(record,
# from_attributes=True)` would read the relationship (a list of BookTag
# ORM objects) instead of the list of strings the controller builds
# separately, and fail validation. Cascade delete is handled by the
# column's own `ondelete="CASCADE"`, so the relationship bought nothing.
def __repr__(self) -> str:
return f"<BookRecord {self.kind} {self.id}>"
class BookTag(Base):
"""A free tag on a record. `work`/`personal` live on `scope`, not here."""
__tablename__ = "book_tags"
record_id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
ForeignKey("book_records.id", ondelete="CASCADE"),
primary_key=True,
)
tag: Mapped[str] = mapped_column(String(63), primary_key=True)
def __repr__(self) -> str:
return f"<BookTag {self.tag} on {self.record_id}>"
class BookRecordLink(Base):
"""A directed relation between two records."""
__tablename__ = "book_record_links"
from_id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
ForeignKey("book_records.id", ondelete="CASCADE"),
primary_key=True,
)
relation: Mapped[str] = mapped_column(String(20), primary_key=True)
to_id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
ForeignKey("book_records.id", ondelete="CASCADE"),
primary_key=True,
)
created_by: Mapped[str] = mapped_column(String(63), nullable=False)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
server_default=func.now(),
nullable=False,
)
def __repr__(self) -> str:
return f"<BookRecordLink {self.from_id} {self.relation} {self.to_id}>"
class BookAttachment(Base):
"""A file attached to a record. Table only — no file handling yet."""
__tablename__ = "book_attachments"
id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
primary_key=True,
default=uuid.uuid4,
)
record_id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
ForeignKey("book_records.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
path: Mapped[str] = mapped_column(String(1000), nullable=False)
mime: Mapped[Optional[str]] = mapped_column(String(255), nullable=True)
size: Mapped[Optional[int]] = mapped_column(Integer, nullable=True)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
server_default=func.now(),
nullable=False,
)
def __repr__(self) -> str:
return f"<BookAttachment {self.path} on {self.record_id}>"
class BookEvent(Base):
"""
An append-only audit row for every change to a record.
Migration 005 adds a trigger that raises on UPDATE and DELETE against
this table — enforced for every role, the owning application included.
`redact` (D-70 rule 4) is the one sanctioned exception, and it is a
later round; nothing here weakens the trigger.
"""
__tablename__ = "book_events"
id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
primary_key=True,
default=uuid.uuid4,
)
record_id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
ForeignKey("book_records.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
server_default=func.now(),
nullable=False,
)
identity: Mapped[str] = mapped_column(String(63), nullable=False)
client: Mapped[Optional[str]] = mapped_column(String(63), nullable=True)
action: Mapped[str] = mapped_column(String(20), nullable=False)
diff: Mapped[dict] = mapped_column(JSONB, nullable=False, default=dict)
def __repr__(self) -> str:
return f"<BookEvent {self.action} on {self.record_id} by {self.identity}>"
class BookTenantCounter(Base):
"""
One row per tenant: the next `number` to allocate on `book_records`.
`BookService._allocate_number` upserts this row (`INSERT ... ON
CONFLICT DO NOTHING`, so a tenant's first `add` never races its own
counter row into existence) and then locks it `FOR UPDATE` in the same
transaction as the record insert it numbers — a second concurrent `add`
in the same tenant blocks on that lock until the first commits, so two
concurrent creates never read the same `next_number`. A different
tenant's counter row is untouched, so tenants never contend with each
other.
"""
__tablename__ = "book_tenant_counters"
# FK to tenants.id added by migration 007 (core-api T-8).
owner_tenant: Mapped[str] = mapped_column(
String(63), ForeignKey("tenants.id", ondelete="RESTRICT"), primary_key=True
)
next_number: Mapped[int] = mapped_column(Integer, nullable=False, default=1)
def __repr__(self) -> str:
return f"<BookTenantCounter {self.owner_tenant} next={self.next_number}>"
+194
View File
@@ -0,0 +1,194 @@
"""
Book Domain Schemas
Pydantic request/response payloads for the Book (D-68, D-70). This round
ships `add`, `show`, `list`, `find`, `change`, `close`, `link`/`unlink`,
`tag`/`untag`, `history` — the verbs the Office needs next round are not
schematized here (dossier: append, remind, reschedule, undo, purge, redact,
pin, check, snooze, archive/trash/restore, pause/resume, due, next, batch,
idempotency keys).
"""
import uuid
from datetime import datetime
from typing import Any, List, Literal, Optional
from pydantic import Field
from src.shared.base import BaseSchema
RecordKind = Literal["note", "task", "decision", "question", "rejected"]
RecordScope = Literal["work", "personal"]
RecordEnergy = Literal["low", "medium", "high"]
RecurrenceMode = Literal["schedule", "after_completion"]
LinkRelation = Literal["parent", "blocks", "resolves", "supersedes", "related"]
ListView = Literal[
"inbox", "today", "upcoming", "waiting", "someday",
"pinned", "archive", "trash", "done",
]
class BookChecklistItem(BaseSchema):
"""One line of a record's checklist."""
text: str
checked: bool = False
position: int = 0
class BookRecordCreate(BaseSchema):
"""`add` — kind defaults to a Note so text-only capture costs nothing (D-72)."""
kind: RecordKind = "note"
title: Optional[str] = Field(None, max_length=500)
body: str = ""
checklist: List[BookChecklistItem] = Field(default_factory=list)
status: Optional[str] = None
scope: RecordScope = "personal"
priority: Optional[int] = None
pinned: bool = False
energy: Optional[RecordEnergy] = None
estimate_minutes: Optional[int] = None
next_step: Optional[str] = None
start_at: Optional[datetime] = None
due_at: Optional[datetime] = None
all_day: bool = False
remind_at: Optional[datetime] = None
recurrence: Optional[str] = None
recurrence_mode: Optional[RecurrenceMode] = None
remind_to: Optional[str] = None
source: Optional[str] = None
external_id: Optional[str] = None
colour: Optional[str] = None
tags: List[str] = Field(default_factory=list)
parent: Optional[uuid.UUID] = Field(
None, description="Convenience: also create a `parent` link to this record"
)
class BookRecordUpdate(BaseSchema):
"""`change` — every field optional; only what is set is applied."""
kind: Optional[RecordKind] = None
title: Optional[str] = Field(None, max_length=500)
body: Optional[str] = None
checklist: Optional[List[BookChecklistItem]] = None
status: Optional[str] = None
scope: Optional[RecordScope] = None
priority: Optional[int] = None
pinned: Optional[bool] = None
energy: Optional[RecordEnergy] = None
estimate_minutes: Optional[int] = None
next_step: Optional[str] = None
start_at: Optional[datetime] = None
due_at: Optional[datetime] = None
all_day: Optional[bool] = None
remind_at: Optional[datetime] = None
recurrence: Optional[str] = None
recurrence_mode: Optional[RecurrenceMode] = None
remind_to: Optional[str] = None
source: Optional[str] = None
external_id: Optional[str] = None
colour: Optional[str] = None
class BookCloseRequest(BaseSchema):
"""
`close` — the field that applies depends on the record's kind:
- task: `outcome` ("done" or "dropped")
- question: `resolved_by` (the Decision id that resolves it); sets `settled`
- decision: `superseded_by` (the new Decision id); the old one becomes `superseded`
"""
outcome: Optional[Literal["done", "dropped"]] = None
resolved_by: Optional[uuid.UUID] = None
superseded_by: Optional[uuid.UUID] = None
class BookLinkRequest(BaseSchema):
"""`link` / `unlink` — direction is from the record in the path to `to_id`."""
to_id: uuid.UUID
relation: LinkRelation
class BookTagRequest(BaseSchema):
"""`tag` / `untag`."""
tag: str = Field(..., min_length=1, max_length=63)
class BookLinkResponse(BaseSchema):
from_id: uuid.UUID
relation: str
to_id: uuid.UUID
created_by: str
created_at: datetime
class BookRecordResponse(BaseSchema):
"""Every write returns the stored record (D-70 rule 7). `number` is the
outward id, numbered per tenant (core-api T-7); `owner_tenant` is not
echoed back — it is implicit in the caller, following
src/domains/scribe/schemas.py's AddressResponse."""
id: uuid.UUID
number: int
kind: str
title: Optional[str]
body: str
checklist: List[dict] = Field(default_factory=list)
status: str
scope: str
priority: Optional[int]
pinned: bool
energy: Optional[str]
estimate_minutes: Optional[int]
next_step: Optional[str]
start_at: Optional[datetime]
due_at: Optional[datetime]
all_day: bool
remind_at: Optional[datetime]
recurrence: Optional[str]
recurrence_mode: Optional[str]
remind_to: Optional[str]
last_touched_at: datetime
archived_at: Optional[datetime]
trashed_at: Optional[datetime]
paused_at: Optional[datetime]
paused_statuses: Optional[dict]
source: Optional[str]
external_id: Optional[str]
colour: Optional[str]
created_by: str
created_at: datetime
updated_at: datetime
tags: List[str] = Field(default_factory=list)
class BookRecordDetailResponse(BookRecordResponse):
"""`show` — the record plus its outgoing links and backlinks."""
links: List[BookLinkResponse] = Field(default_factory=list)
backlinks: List[BookLinkResponse] = Field(default_factory=list)
class BookRecordListResponse(BaseSchema):
records: List[BookRecordResponse]
total: int
limit: int
offset: int
class BookEventResponse(BaseSchema):
id: uuid.UUID
record_id: uuid.UUID
at: datetime
identity: str
client: Optional[str]
action: str
diff: dict
class BookHistoryResponse(BaseSchema):
record_id: uuid.UUID
events: List[BookEventResponse]
+609
View File
@@ -0,0 +1,609 @@
"""
Book Domain Service
Business logic for the Book — Notes, Tasks, Decisions, Questions and
Rejected options (D-68, D-70). Every mutating method stamps a `book_events`
row via `_write_event`; the table itself is append-only (migration 005's
trigger), so this module never updates or deletes an event.
Every method starts by requiring a tenant on the caller (`_require_tenant`,
same pattern as `src/domains/scribe/service.py`) and ends by filtering its
query on it: **a record in another tenant answers exactly like one that
does not exist** (core-api T-7 / workspace D-73), so no method here ever
confirms an id across that boundary. `link` and `add`'s `parent` convenience
both route through `_get_or_404`, so neither can create a link into another
tenant — which is also what keeps `show`'s links/backlinks free of a
separate tenant filter: every link touching a tenant's record is created
tenant-safe in the first place.
Methods take `session: AsyncSession` per call rather than storing it on
`self`, following `src/domains/dashboard/service.py`.
"""
from datetime import datetime, timezone
from typing import List, Optional, Tuple
from zoneinfo import ZoneInfo
from sqlalchemy import delete, func, select
from sqlalchemy.dialects.postgresql import insert as pg_insert
from sqlalchemy.ext.asyncio import AsyncSession
from src.domains.auth.callers import Caller
from src.domains.book.models import (
DEFAULT_STATUS_BY_KIND,
STATUSES_BY_KIND,
BookEvent,
BookRecord,
BookRecordLink,
BookTag,
BookTenantCounter,
)
from src.domains.book.schemas import BookCloseRequest, BookRecordCreate, BookRecordUpdate
from src.domains.tenants.service import TenantError, get_tenants_service
from src.shared.config import get_settings
from src.shared.logging import get_logger
logger = get_logger(__name__)
class BookError(Exception):
"""Raised for a Book-domain failure; the controller maps it to a response."""
def __init__(self, code: str, status_code: int = 400):
self.code = code
self.status_code = status_code
super().__init__(code)
class BookService:
"""Service for Book operations."""
# =========================================================================
# Internal helpers
# =========================================================================
async def _require_tenant(self, session: AsyncSession, caller: Caller) -> str:
"""Every Book operation needs a tenant; a caller without one gets a
clear error rather than a query that silently matches nothing. An
Office caller always has one (callers.py enforces it on every call);
a user-kind caller without a `tatlock_user` claim does not.
Delegates to the tenant registry's one enforcement point (core-api
T-8 / workspace D-73), which additionally refuses an unknown
tenant, a suspended one, or an Office consumer with no grant in it.
`TenantError` is translated to `BookError` so every call site and
the controller's single `except BookError` keep working unchanged.
"""
try:
return await get_tenants_service().require_tenant(session, caller)
except TenantError as error:
raise BookError(error.code, error.status_code) from error
async def _get_or_404(self, session: AsyncSession, record_id, tenant: str) -> BookRecord:
"""A record outside `tenant` answers exactly like one that does not
exist — `session.get()` alone can't express that filter, since the
primary key is `id` only, so this is a `select` rather than a get."""
stmt = select(BookRecord).where(BookRecord.id == record_id, BookRecord.owner_tenant == tenant)
record = (await session.execute(stmt)).scalar_one_or_none()
if record is None:
raise BookError("not_found", 404)
return record
async def _allocate_number(self, session: AsyncSession, tenant: str) -> int:
"""
Allocate the next outward `number` for `tenant`, safe under
concurrency.
`INSERT ... ON CONFLICT DO NOTHING` guarantees the tenant's counter
row exists without racing its own creation (two simultaneous first
`add`s in a brand-new tenant can't both try to INSERT the same row
and fail), then `SELECT ... FOR UPDATE` locks that row for the rest
of this transaction — a second concurrent `add` in the same tenant
blocks on the lock until this one commits or rolls back, so two
concurrent creates in one tenant can never read the same
`next_number`. A different tenant's row is untouched, so tenants
never contend with each other. This is the dossier's suggested
shape (a per-tenant counter row taken FOR UPDATE in the same
transaction as the insert); the ON CONFLICT DO NOTHING bootstrap is
the one addition, needed because the plain shape alone still races
on a tenant's very first allocation.
"""
upsert = (
pg_insert(BookTenantCounter)
.values(owner_tenant=tenant, next_number=1)
.on_conflict_do_nothing(index_elements=["owner_tenant"])
)
await session.execute(upsert)
locked = (
select(BookTenantCounter)
.where(BookTenantCounter.owner_tenant == tenant)
.with_for_update()
)
counter = (await session.execute(locked)).scalar_one()
number = counter.next_number
counter.next_number = number + 1
return number
async def _tags_for(self, session: AsyncSession, record_id) -> List[str]:
stmt = select(BookTag.tag).where(BookTag.record_id == record_id).order_by(BookTag.tag)
result = await session.execute(stmt)
return list(result.scalars().all())
async def _write_event(
self,
session: AsyncSession,
record_id,
caller: Caller,
action: str,
diff: dict,
) -> None:
event = BookEvent(
record_id=record_id,
identity=caller.identity,
client=caller.consumer, # column name unchanged (no migration this round); T-4 retires X-Office-Client and records the consumer here instead
action=action,
diff=diff,
)
session.add(event)
await session.flush()
def _validate_status(self, kind: str, status: Optional[str]) -> str:
"""Resolve and validate a status for `kind`; defaults when unset."""
resolved = status or DEFAULT_STATUS_BY_KIND[kind]
if resolved not in STATUSES_BY_KIND[kind]:
raise BookError("invalid_status", 400)
return resolved
def _touch(self, record: BookRecord) -> None:
record.last_touched_at = datetime.now(timezone.utc)
# =========================================================================
# add / show / list / find
# =========================================================================
async def add_record(
self,
session: AsyncSession,
data: BookRecordCreate,
caller: Caller,
) -> Tuple[BookRecord, List[str]]:
"""`add` — text-only input lands an unfiled Note (D-72). Sets
`owner_tenant` from the caller and allocates `number` (core-api T-7)."""
tenant = await self._require_tenant(session, caller)
status = self._validate_status(data.kind, data.status)
number = await self._allocate_number(session, tenant)
record = BookRecord(
owner_tenant=tenant,
number=number,
kind=data.kind,
title=data.title,
body=data.body,
checklist=[item.model_dump() for item in data.checklist],
status=status,
scope=data.scope,
priority=data.priority,
pinned=data.pinned,
energy=data.energy,
estimate_minutes=data.estimate_minutes,
next_step=data.next_step,
start_at=data.start_at,
due_at=data.due_at,
all_day=data.all_day,
remind_at=data.remind_at,
recurrence=data.recurrence,
recurrence_mode=data.recurrence_mode,
remind_to=data.remind_to,
source=data.source,
external_id=data.external_id,
colour=data.colour,
created_by=caller.identity,
)
session.add(record)
await session.flush() # assign record.id
tag_names = list(dict.fromkeys(data.tags)) # de-dup, keep order
for tag in tag_names:
session.add(BookTag(record_id=record.id, tag=tag))
if data.parent is not None:
# The parent is a link target (`relation="parent"`), so it goes
# through the same tenant-scoped lookup `link` uses — a parent
# in another tenant (or no such record at all) answers 404
# not_found, the same as an explicit `link` call would.
await self._get_or_404(session, data.parent, tenant)
session.add(BookRecordLink(
from_id=record.id,
relation="parent",
to_id=data.parent,
created_by=caller.identity,
))
await self._write_event(session, record.id, caller, "add", {"kind": data.kind, "status": status})
await session.commit()
await session.refresh(record)
logger.info(f"Book record added: kind={data.kind} id={record.id} number={number} tenant={tenant} by={caller.identity}")
return record, tag_names
async def show_record(
self,
session: AsyncSession,
record_id,
caller: Caller,
) -> Tuple[BookRecord, List[str], List[BookRecordLink], List[BookRecordLink]]:
"""`show` — the record, its tags, outgoing links and backlinks."""
tenant = await self._require_tenant(session, caller)
record = await self._get_or_404(session, record_id, tenant)
tags = await self._tags_for(session, record_id)
links_stmt = select(BookRecordLink).where(BookRecordLink.from_id == record_id)
links = list((await session.execute(links_stmt)).scalars().all())
backlinks_stmt = select(BookRecordLink).where(BookRecordLink.to_id == record_id)
backlinks = list((await session.execute(backlinks_stmt)).scalars().all())
return record, tags, links, backlinks
async def list_records(
self,
session: AsyncSession,
caller: Caller,
view: Optional[str] = None,
tag: Optional[str] = None,
scope: Optional[str] = None,
parent: Optional[object] = None,
number: Optional[int] = None,
limit: int = 50,
offset: int = 0,
) -> Tuple[List[Tuple[BookRecord, List[str]]], int]:
"""
`list` — the nine views of D-70, scoped to the caller's tenant.
Status-named views (`inbox`, `waiting`, `someday`, `done`) filter on
the matching task status and simply return nothing for kinds that
never carry it. `today` and `upcoming` bucket on `due_at` in
`book_timezone`; richer due/overdue handling is `due`/`next`, both
next round. `number=` finds the one record with that outward id in
the caller's tenant (core-api T-7).
"""
tenant = await self._require_tenant(session, caller)
conditions = [BookRecord.owner_tenant == tenant]
if view in ("inbox", "waiting", "someday", "done"):
conditions.append(BookRecord.status == view)
elif view == "pinned":
conditions.append(BookRecord.pinned.is_(True))
elif view == "archive":
conditions.append(BookRecord.archived_at.isnot(None))
elif view == "trash":
conditions.append(BookRecord.trashed_at.isnot(None))
elif view in ("today", "upcoming"):
tz = ZoneInfo(get_settings().book_timezone)
local_now = datetime.now(tz)
day_start = local_now.replace(hour=0, minute=0, second=0, microsecond=0)
day_end = day_start.replace(hour=23, minute=59, second=59, microsecond=999999)
conditions.append(BookRecord.due_at.isnot(None))
if view == "today":
conditions.append(BookRecord.due_at >= day_start)
conditions.append(BookRecord.due_at <= day_end)
else:
conditions.append(BookRecord.due_at > day_end)
if view not in ("archive", "trash"):
conditions.append(BookRecord.archived_at.is_(None))
conditions.append(BookRecord.trashed_at.is_(None))
if scope is not None:
conditions.append(BookRecord.scope == scope)
if parent is not None:
parent_subquery = (
select(BookRecordLink.from_id)
.where(BookRecordLink.relation == "parent", BookRecordLink.to_id == parent)
)
conditions.append(BookRecord.id.in_(parent_subquery))
if tag is not None:
tag_subquery = select(BookTag.record_id).where(BookTag.tag == tag)
conditions.append(BookRecord.id.in_(tag_subquery))
if number is not None:
conditions.append(BookRecord.number == number)
base_query = select(BookRecord).where(*conditions)
count_query = select(func.count()).select_from(base_query.subquery())
total = (await session.execute(count_query)).scalar() or 0
query = (
base_query
.order_by(BookRecord.last_touched_at.desc())
.offset(offset)
.limit(limit)
)
records = list((await session.execute(query)).scalars().all())
results = []
for record in records:
tags = await self._tags_for(session, record.id)
results.append((record, tags))
return results, total
async def find_records(
self,
session: AsyncSession,
caller: Caller,
q: str,
limit: int = 50,
offset: int = 0,
) -> Tuple[List[Tuple[BookRecord, List[str]]], int]:
"""`find` — full-text search over title and body, scoped to the
caller's tenant."""
tenant = await self._require_tenant(session, caller)
tsquery = func.websearch_to_tsquery("simple", q)
base_query = select(BookRecord).where(
BookRecord.owner_tenant == tenant, BookRecord.search.op("@@")(tsquery)
)
count_query = select(func.count()).select_from(base_query.subquery())
total = (await session.execute(count_query)).scalar() or 0
query = base_query.order_by(BookRecord.updated_at.desc()).offset(offset).limit(limit)
records = list((await session.execute(query)).scalars().all())
results = []
for record in records:
tags = await self._tags_for(session, record.id)
results.append((record, tags))
return results, total
# =========================================================================
# change / close
# =========================================================================
async def change_record(
self,
session: AsyncSession,
record_id,
data: BookRecordUpdate,
caller: Caller,
) -> Tuple[BookRecord, List[str]]:
"""`change` — a partial update; only fields set on the request move."""
tenant = await self._require_tenant(session, caller)
record = await self._get_or_404(session, record_id, tenant)
update_data = data.model_dump(exclude_unset=True)
diff: dict = {}
new_kind = update_data.get("kind", record.kind)
if "status" in update_data or "kind" in update_data:
resolved_status = self._validate_status(new_kind, update_data.get("status"))
update_data["status"] = resolved_status
if "checklist" in update_data:
update_data["checklist"] = [
item if isinstance(item, dict) else item.model_dump()
for item in update_data["checklist"]
]
for field, value in update_data.items():
before = getattr(record, field)
if before != value:
diff[field] = {"from": str(before), "to": str(value)}
setattr(record, field, value)
self._touch(record)
if diff:
await self._write_event(session, record_id, caller, "change", diff)
await session.commit()
await session.refresh(record)
tags = await self._tags_for(session, record_id)
logger.info(f"Book record changed: id={record_id} by={caller.identity} fields={list(diff)}")
return record, tags
async def close_record(
self,
session: AsyncSession,
record_id,
data: BookCloseRequest,
caller: Caller,
) -> Tuple[BookRecord, List[str]]:
"""
`close` — a Task is done or dropped; a Question is resolved into a
Decision (`resolves` link, status `settled`); a Decision is
superseded by a newer one (`supersedes` link, status `superseded`).
`resolved_by`/`superseded_by` are link targets, so each is looked up
in the caller's tenant the same as `link` — a target in another
tenant answers 404 not_found.
"""
tenant = await self._require_tenant(session, caller)
record = await self._get_or_404(session, record_id, tenant)
diff: dict = {}
if record.kind == "task":
if data.outcome is None:
raise BookError("invalid_status", 400)
record.status = data.outcome
diff = {"status": data.outcome}
elif record.kind == "question":
if data.resolved_by is None:
raise BookError("invalid_status", 400)
decision = await self._get_or_404(session, data.resolved_by, tenant)
if decision.kind != "decision":
raise BookError("invalid_status", 400)
session.add(BookRecordLink(
from_id=decision.id,
relation="resolves",
to_id=record.id,
created_by=caller.identity,
))
record.status = "settled"
diff = {"status": "settled", "resolved_by": str(decision.id)}
elif record.kind == "decision":
if data.superseded_by is None:
raise BookError("invalid_status", 400)
new_decision = await self._get_or_404(session, data.superseded_by, tenant)
if new_decision.kind != "decision":
raise BookError("invalid_status", 400)
session.add(BookRecordLink(
from_id=new_decision.id,
relation="supersedes",
to_id=record.id,
created_by=caller.identity,
))
record.status = "superseded"
diff = {"status": "superseded", "superseded_by": str(new_decision.id)}
else:
# Notes and Rejected options have no `close` transition.
raise BookError("invalid_status", 400)
self._touch(record)
await self._write_event(session, record_id, caller, "close", diff)
await session.commit()
await session.refresh(record)
tags = await self._tags_for(session, record_id)
logger.info(f"Book record closed: id={record_id} by={caller.identity} diff={diff}")
return record, tags
# =========================================================================
# link / unlink
# =========================================================================
async def link_records(
self,
session: AsyncSession,
record_id,
to_id,
relation: str,
caller: Caller,
) -> Tuple[BookRecord, List[str]]:
"""`link` refuses a target in another tenant the same way a missing
one is refused — both are 404 not_found (core-api T-7)."""
tenant = await self._require_tenant(session, caller)
record = await self._get_or_404(session, record_id, tenant)
await self._get_or_404(session, to_id, tenant) # the target must exist in the same tenant
session.add(BookRecordLink(
from_id=record_id,
relation=relation,
to_id=to_id,
created_by=caller.identity,
))
await self._write_event(
session, record_id, caller, "link", {"relation": relation, "to_id": str(to_id)}
)
await session.commit()
await session.refresh(record)
tags = await self._tags_for(session, record_id)
return record, tags
async def unlink_records(
self,
session: AsyncSession,
record_id,
to_id,
relation: str,
caller: Caller,
) -> Tuple[BookRecord, List[str]]:
tenant = await self._require_tenant(session, caller)
record = await self._get_or_404(session, record_id, tenant)
stmt = delete(BookRecordLink).where(
BookRecordLink.from_id == record_id,
BookRecordLink.relation == relation,
BookRecordLink.to_id == to_id,
)
result = await session.execute(stmt)
if result.rowcount == 0:
raise BookError("not_found", 404)
await self._write_event(
session, record_id, caller, "unlink", {"relation": relation, "to_id": str(to_id)}
)
await session.commit()
await session.refresh(record)
tags = await self._tags_for(session, record_id)
return record, tags
# =========================================================================
# tag / untag
# =========================================================================
async def tag_record(
self,
session: AsyncSession,
record_id,
tag: str,
caller: Caller,
) -> Tuple[BookRecord, List[str]]:
tenant = await self._require_tenant(session, caller)
record = await self._get_or_404(session, record_id, tenant)
existing = await session.get(BookTag, (record_id, tag))
if existing is None:
session.add(BookTag(record_id=record_id, tag=tag))
await self._write_event(session, record_id, caller, "tag", {"tag": tag})
await session.commit()
tags = await self._tags_for(session, record_id)
return record, tags
async def untag_record(
self,
session: AsyncSession,
record_id,
tag: str,
caller: Caller,
) -> Tuple[BookRecord, List[str]]:
tenant = await self._require_tenant(session, caller)
record = await self._get_or_404(session, record_id, tenant)
stmt = delete(BookTag).where(BookTag.record_id == record_id, BookTag.tag == tag)
result = await session.execute(stmt)
if result.rowcount > 0:
await self._write_event(session, record_id, caller, "untag", {"tag": tag})
await session.commit()
tags = await self._tags_for(session, record_id)
return record, tags
# =========================================================================
# history
# =========================================================================
async def get_history(
self, session: AsyncSession, record_id, caller: Caller, limit: int = 100
) -> List[BookEvent]:
tenant = await self._require_tenant(session, caller)
await self._get_or_404(session, record_id, tenant)
stmt = (
select(BookEvent)
.where(BookEvent.record_id == record_id)
.order_by(BookEvent.at.desc())
.limit(limit)
)
return list((await session.execute(stmt)).scalars().all())
# Singleton instance
_book_service: Optional[BookService] = None
def get_book_service() -> BookService:
"""Get singleton Book service instance"""
global _book_service
if _book_service is None:
_book_service = BookService()
return _book_service
+11 -96
View File
@@ -70,66 +70,18 @@ class HealthController(BaseController):
@router.get(
"/health/full",
summary="Fast health check for Docker",
summary="Full health check with database",
)
async def full_health_check(response: Response):
"""
Fast health check for container orchestration (Docker/K8s).
Health check including database connectivity.
Checks component availability WITHOUT running expensive operations.
Returns 200 OK if all components are available, otherwise 503.
For detailed diagnostics, use /health/diagnostics instead.
Returns 200 OK if database is available, otherwise 503.
"""
import time
start_time = time.time()
# Import here to avoid circular imports
from src.models.ollama_client import get_ollama_client
# Check 1: Ollama connection + verify agent model is available
ollama_client = get_ollama_client()
ollama_healthy = False
ollama_error = None
model_available = False
try:
# Ping Ollama
ollama_healthy = await ollama_client.health_check()
# Verify the agent model is pulled and check what's currently loaded
models_info = {}
if ollama_healthy:
try:
models_response = await ollama_client.list_models()
available_models = [m.get('name', '') for m in models_response.get('models', [])]
model_available = settings.agent_model in available_models
# Get info about currently loaded models (those with size in memory)
loaded_models = [
m.get('name', '') for m in models_response.get('models', [])
if m.get('size', 0) > 0
]
models_info = {
"configured": settings.agent_model,
"available": model_available,
"total_in_ollama": len(available_models),
"currently_loaded": loaded_models if loaded_models else ["none"]
}
if not model_available:
ollama_error = f"Model '{settings.agent_model}' not found in Ollama. Available: {', '.join(available_models[:3])}"
ollama_healthy = False
except Exception as e:
ollama_error = f"Could not list Ollama models: {str(e)}"
ollama_healthy = False
except Exception as e:
ollama_error = str(e)
logger.warning(f"Ollama health check failed: {ollama_error}")
# Check 2: Database connection
# Check database connection
database = get_database()
db_healthy = False
db_error = None
@@ -140,25 +92,15 @@ class HealthController(BaseController):
db_error = str(e)
logger.warning(f"Database health check failed: {db_error}")
is_healthy = ollama_healthy and db_healthy
elapsed_ms = int((time.time() - start_time) * 1000)
status_code = 200 if is_healthy else 503
status_code = 200 if db_healthy else 503
response.status_code = status_code
return {
"status": "healthy" if is_healthy else "unhealthy",
"status": "healthy" if db_healthy else "unhealthy",
"status_code": status_code,
"response_time_ms": elapsed_ms,
"components": {
"ollama": {
"status": "healthy" if ollama_healthy else "unhealthy",
"models": models_info if models_info else {
"configured": settings.agent_model,
"available": False
},
"error": ollama_error
},
"database": {
"status": "healthy" if db_healthy else "unhealthy",
"error": db_error
@@ -170,19 +112,13 @@ class HealthController(BaseController):
"/health/diagnostics",
summary="Detailed system diagnostics",
)
async def diagnostics(deep_test: bool = False):
async def diagnostics():
"""
Comprehensive system diagnostics with detailed component information.
Query Parameters:
- deep_test: Set to true to actually test agent generation (slow, ~5-10s)
Returns detailed information about all system components.
System diagnostics with service information.
"""
import time
from src.models.ollama_client import get_ollama_client
start_time = time.time()
diagnostics = {
"timestamp": time.time(),
"service": {
@@ -190,30 +126,9 @@ class HealthController(BaseController):
"version": settings.app_version,
"purpose": "Infrastructure management and tools API"
},
"components": {}
}
# 1. Ollama Connection
ollama_client = get_ollama_client()
try:
ollama_healthy = await ollama_client.health_check()
diagnostics["components"]["ollama"] = {
"status": "connected",
"url": settings.ollama_base_url,
"timeout": settings.ollama_timeout,
"default_model": settings.default_model
"configuration": {
"cors_origins": settings.cors_origins[:2] if len(settings.cors_origins) > 2 else settings.cors_origins
}
except Exception as e:
diagnostics["components"]["ollama"] = {
"status": "error",
"error": str(e)
}
# 2. Configuration
diagnostics["configuration"] = {
"agent_fallback_enabled": settings.agent_fallback_enabled,
"memory_tier1_max_turns": settings.memory_tier1_max_turns,
"cors_origins": settings.cors_origins[:2] if len(settings.cors_origins) > 2 else settings.cors_origins
}
elapsed_ms = int((time.time() - start_time) * 1000)
+19 -8
View File
@@ -13,9 +13,15 @@ from src.shared.base import BaseController
from src.shared.clients import get_homeassistant_client
from src.shared.logging import get_logger
from src.domains.auth.oidc import get_admin_user
from src.domains.auth.callers import Caller, require_caller
logger = get_logger(__name__)
# api.schweitz.net carries no forward-auth, so these reads were otherwise
# open to the internet (T-1 / D-68). Writes are untouched — see controller
# comments at each `get_admin_user` dependency below.
require_housekeeping_reader = require_caller("office", "tatlock", "user")
# Pydantic Schemas
class Device(BaseModel):
@@ -188,7 +194,7 @@ class HousekeepingController(BaseController):
response_model=HealthResponse,
summary="Home automation health check"
)
async def get_health():
async def get_health(caller: Caller = Depends(require_housekeeping_reader)):
"""Check Home Assistant connection health"""
ha = get_homeassistant_client()
return await ha.health_check()
@@ -200,7 +206,8 @@ class HousekeepingController(BaseController):
)
async def list_devices(
domain: Optional[str] = Query(None, description="Filter by domain (light, switch, climate, etc.)"),
area: Optional[str] = Query(None, description="Filter by area/room name")
area: Optional[str] = Query(None, description="Filter by area/room name"),
caller: Caller = Depends(require_housekeeping_reader),
):
"""List all available devices with optional filtering"""
ha = get_homeassistant_client()
@@ -256,7 +263,10 @@ class HousekeepingController(BaseController):
response_model=DeviceDetailResponse,
responses={404: {"model": ErrorResponse}}
)
async def get_device(entity_id: str):
async def get_device(
entity_id: str,
caller: Caller = Depends(require_housekeeping_reader),
):
"""Get detailed state of a specific device"""
ha = get_homeassistant_client()
@@ -296,7 +306,7 @@ class HousekeepingController(BaseController):
response_model=AreaListResponse,
summary="List areas/rooms"
)
async def list_areas():
async def list_areas(caller: Caller = Depends(require_housekeeping_reader)):
"""List all configured areas/rooms in Home Assistant"""
ha = get_homeassistant_client()
@@ -389,7 +399,7 @@ class HousekeepingController(BaseController):
response_model=SceneListResponse,
summary="List available scenes"
)
async def list_scenes():
async def list_scenes(caller: Caller = Depends(require_housekeeping_reader)):
"""List all available scenes in Home Assistant"""
ha = get_homeassistant_client()
@@ -459,7 +469,7 @@ class HousekeepingController(BaseController):
response_model=ScriptListResponse,
summary="List available scripts"
)
async def list_scripts():
async def list_scripts(caller: Caller = Depends(require_housekeeping_reader)):
"""List all available scripts/sequences in Home Assistant"""
ha = get_homeassistant_client()
@@ -531,7 +541,7 @@ class HousekeepingController(BaseController):
response_model=AutomationListResponse,
summary="List automations"
)
async def list_automations():
async def list_automations(caller: Caller = Depends(require_housekeeping_reader)):
"""List all automations with their enabled/disabled status"""
ha = get_homeassistant_client()
@@ -609,7 +619,8 @@ class HousekeepingController(BaseController):
)
async def get_history(
entity_id: str = Query(..., description="Entity ID to get history for"),
hours: int = Query(24, ge=1, le=168, description="Hours of history (1-168)")
hours: int = Query(24, ge=1, le=168, description="Hours of history (1-168)"),
caller: Caller = Depends(require_housekeeping_reader),
):
"""Get state history for a device"""
ha = get_homeassistant_client()
+9
View File
@@ -0,0 +1,9 @@
"""
Scribe Domain
The household's message bus (workspace D-66, D-68, D-73): addresses, send,
mailbox, history, find, mark, tag. Served under /scribe/*.
"""
from src.domains.scribe.controller import scribe_controller
__all__ = ["scribe_controller"]
+264
View File
@@ -0,0 +1,264 @@
"""
Scribe Controller
The Scribe's messaging verbs (dossier core-api T-5): register, list
(addresses), send, mailbox, history, find, mark, tag/untag, archive/trash
a mailbox entry and look one up by message number (core-api T-13/T-9).
Every route is behind an Office or user caller (T-1/T-4); every query
filters on the caller's tenant (D-73) — an address or message in another
tenant answers exactly like one that does not exist.
"""
from typing import Optional
from fastapi import APIRouter, Depends, HTTPException, Query
from src.domains.auth.callers import Caller, require_caller
from src.domains.scribe.schemas import (
AddressListResponse,
AddressRegisterRequest,
AddressResponse,
HistoryResponse,
MailboxEntryResponse,
MailboxListResponse,
MailboxPatchRequest,
MailboxView,
MessageResponse,
MessageSendRequest,
)
from src.domains.scribe.service import ScribeError, get_scribe_service
from src.shared.base import BaseController
from src.shared.database import get_async_session
from src.shared.logging import get_logger
from sqlalchemy.ext.asyncio import AsyncSession
logger = get_logger(__name__)
require_scribe_reader = require_caller("office", "user")
require_scribe_writer = require_caller("office", "user", identity_required=True)
def _address_response(address, owned: bool) -> AddressResponse:
return AddressResponse(
name=address.name,
owner_consumer=address.owner_consumer,
created_by=address.created_by,
created_at=address.created_at,
owned=owned,
)
def _message_response(message) -> MessageResponse:
return MessageResponse.model_validate(message, from_attributes=True)
def _entry_response(entry, message) -> MailboxEntryResponse:
return MailboxEntryResponse(
id=entry.id,
address=entry.address,
read_at=entry.read_at,
archived_at=entry.archived_at,
trashed_at=entry.trashed_at,
tags=entry.tags,
created_at=entry.created_at,
message=_message_response(message),
)
class ScribeController(BaseController):
"""Controller for the Scribe domain."""
def __init__(self):
super().__init__(prefix="/scribe", tags=["Scribe"])
def create_router(self) -> APIRouter:
router = APIRouter(prefix=self.prefix, tags=self.tags)
service = get_scribe_service()
# =====================================================================
# register / list addresses
# =====================================================================
@router.post(
"/addresses",
response_model=AddressResponse,
summary="Register an address (idempotent for the same consumer)",
)
async def register_address(
data: AddressRegisterRequest,
caller: Caller = Depends(require_scribe_writer),
session: AsyncSession = Depends(get_async_session),
):
try:
address, owned = await service.register_address(session, data.name, caller)
except ScribeError as error:
raise HTTPException(
status_code=error.status_code, detail={"code": error.code, **error.extra}
) from error
return _address_response(address, owned)
@router.get(
"/addresses",
response_model=AddressListResponse,
summary="List addresses in the caller's tenant",
)
async def list_addresses(
caller: Caller = Depends(require_scribe_reader),
session: AsyncSession = Depends(get_async_session),
):
try:
results = await service.list_addresses(session, caller)
except ScribeError as error:
raise HTTPException(
status_code=error.status_code, detail={"code": error.code, **error.extra}
) from error
return AddressListResponse(
addresses=[_address_response(address, owned) for address, owned in results]
)
# =====================================================================
# send
# =====================================================================
@router.post(
"/messages",
response_model=MessageResponse,
status_code=201,
summary="Send a message to one or more addresses",
)
async def send_message(
data: MessageSendRequest,
caller: Caller = Depends(require_scribe_writer),
session: AsyncSession = Depends(get_async_session),
):
try:
message = await service.send_message(session, data, caller)
except ScribeError as error:
raise HTTPException(
status_code=error.status_code, detail={"code": error.code, **error.extra}
) from error
return _message_response(message)
# =====================================================================
# find
# =====================================================================
@router.get(
"/messages",
response_model=MailboxListResponse,
summary="Full text search over messages in the caller identity's mailbox",
)
async def find_messages(
q: str = Query(..., min_length=1),
limit: int = Query(50, ge=1, le=200),
caller: Caller = Depends(require_scribe_reader),
session: AsyncSession = Depends(get_async_session),
):
try:
results, total = await service.find_messages(session, caller, q, limit)
except ScribeError as error:
raise HTTPException(
status_code=error.status_code, detail={"code": error.code, **error.extra}
) from error
return MailboxListResponse(
entries=[_entry_response(entry, message) for entry, message in results],
total=total,
limit=limit,
)
# =====================================================================
# mailbox
# =====================================================================
@router.get(
"/mailbox",
response_model=MailboxListResponse,
summary="The caller identity's mailbox, unread first",
)
async def get_mailbox(
view: MailboxView = Query(
"inbox", description="inbox (default, neither archived nor trashed), archive, trash, all"
),
unread: bool = Query(False),
tag: Optional[str] = Query(None),
limit: int = Query(50, ge=1, le=200),
before_id: Optional[int] = Query(None),
message_id: Optional[int] = Query(
None,
description="look up the caller's own entry for this message number directly, "
"in any view, ignoring unread/tag/before_id (core-api T-9)",
),
caller: Caller = Depends(require_scribe_reader),
session: AsyncSession = Depends(get_async_session),
):
try:
results, total = await service.get_mailbox(
session,
caller,
view=view,
unread=unread,
tag=tag,
limit=limit,
before_id=before_id,
message_id=message_id,
)
except ScribeError as error:
raise HTTPException(
status_code=error.status_code, detail={"code": error.code, **error.extra}
) from error
return MailboxListResponse(
entries=[_entry_response(entry, message) for entry, message in results],
total=total,
limit=limit,
)
@router.patch(
"/mailbox/{entry_id}",
response_model=MailboxEntryResponse,
summary="Mark read/unread, or tag/untag a mailbox entry",
)
async def patch_mailbox_entry(
entry_id: int,
data: MailboxPatchRequest,
caller: Caller = Depends(require_scribe_writer),
session: AsyncSession = Depends(get_async_session),
):
try:
entry, message = await service.patch_mailbox_entry(session, entry_id, data, caller)
except ScribeError as error:
raise HTTPException(
status_code=error.status_code, detail={"code": error.code, **error.extra}
) from error
return _entry_response(entry, message)
# =====================================================================
# history
# =====================================================================
@router.get(
"/addresses/{name}/history",
response_model=HistoryResponse,
summary="The retained messages sent to an address, oldest first",
)
async def address_history(
name: str,
since_id: Optional[int] = Query(None),
limit: int = Query(100, ge=1, le=500),
caller: Caller = Depends(require_scribe_reader),
session: AsyncSession = Depends(get_async_session),
):
try:
messages = await service.get_history(session, name, caller, since_id, limit)
except ScribeError as error:
raise HTTPException(
status_code=error.status_code, detail={"code": error.code, **error.extra}
) from error
return HistoryResponse(
address=name,
messages=[_message_response(message) for message in messages],
)
return router
# Create controller instance
scribe_controller = ScribeController()
+151
View File
@@ -0,0 +1,151 @@
"""
Scribe Domain Models
The Scribe is the household's message bus (workspace D-66, D-68, D-73): a
retained pub/sub log where an address is a topic. This round ships the
primitives migration 006 creates — addresses, messages, mailbox entries —
for direct messaging only; groups, subscriptions, the notification matrix
and delivery are later work (workspace T-176).
SQLAlchemy 2.0 Mapped/mapped_column style, timestamptz, following
src/domains/book/models.py. Every table carries `owner_tenant` and every
query in the service layer filters on it (D-73's hard tenant boundary) —
there is no database-level RLS here, so that filter is load-bearing in
Python, not just in SQL.
"""
from datetime import datetime
from typing import List, Optional
from sqlalchemy import (
Computed,
DateTime,
ForeignKey,
Identity,
Integer,
String,
Text,
UniqueConstraint,
func,
)
from sqlalchemy.dialects.postgresql import ARRAY, TSVECTOR
from sqlalchemy.orm import Mapped, mapped_column
from src.shared.database import Base
# Address name shape, identical to src/domains/auth/callers.py's identity
# rule (an address is named for an identity, D-66) — not imported from there
# because that module's constants are private to it; kept here so a drift
# between the two never happens silently, the regex is asserted equal in
# tests/test_scribe.py instead.
ADDRESS_NAME_PATTERN = r"^[a-z0-9][a-z0-9-]{0,62}$"
RESERVED_ADDRESS_NAMES = {"book", "office", "scheduler", "tatlock"}
class ScribeAddress(Base):
"""
An address: a topic named for an identity. `register` creates it once
(D-66) — the pair (owner_tenant, name) is the natural key, so no
surrogate id: no route ever references an address except by name.
"""
__tablename__ = "scribe_addresses"
__table_args__ = (
UniqueConstraint("owner_tenant", "name", name="uq_scribe_addresses_tenant_name"),
)
# FK to tenants.id added by migration 007 (core-api T-8).
owner_tenant: Mapped[str] = mapped_column(
String(63), ForeignKey("tenants.id", ondelete="RESTRICT"), primary_key=True
)
name: Mapped[str] = mapped_column(String(63), primary_key=True)
# Nullable: a "user"-kind caller (direct OIDC, not through the Office)
# carries no consumer (src/domains/auth/callers.py), so an address it
# registers has none either.
owner_consumer: Mapped[Optional[str]] = mapped_column(String(63), nullable=True)
created_by: Mapped[str] = mapped_column(String(63), nullable=False)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
server_default=func.now(),
nullable=False,
)
def __repr__(self) -> str:
return f"<ScribeAddress {self.owner_tenant}/{self.name}>"
class ScribeMessage(Base):
"""
A retained message. `topics` is the set of addresses (and, later,
groups) it was published to — an integer array is denormalized from
the mailbox entries so `history` can read the retained log directly
without joining through them (D-66: "topics are retained and
re-readable"). Nothing deletes a message.
"""
__tablename__ = "scribe_messages"
id: Mapped[int] = mapped_column(Integer, Identity(always=False), primary_key=True)
# FK to tenants.id added by migration 007 (core-api T-8).
owner_tenant: Mapped[str] = mapped_column(
String(63), ForeignKey("tenants.id", ondelete="RESTRICT"), nullable=False, index=True
)
topics: Mapped[List[str]] = mapped_column(ARRAY(Text), nullable=False, default=list)
from_identity: Mapped[str] = mapped_column(String(63), nullable=False)
from_consumer: Mapped[Optional[str]] = mapped_column(String(63), nullable=True)
subject: Mapped[Optional[str]] = mapped_column(String(500), nullable=True)
body: Mapped[str] = mapped_column(Text, nullable=False)
tags: Mapped[List[str]] = mapped_column(ARRAY(Text), nullable=False, default=list)
reply_to: Mapped[Optional[int]] = mapped_column(
Integer, ForeignKey("scribe_messages.id"), nullable=True
)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
server_default=func.now(),
nullable=False,
)
# Generated by migration 006 (`to_tsvector('simple', ...)`, GIN-indexed),
# 'simple' for the same reason as book_records.search: mixed Dutch/
# English content. `Computed` mirrors the migration's SQL.
search: Mapped[Optional[str]] = mapped_column(
TSVECTOR,
Computed("to_tsvector('simple', coalesce(subject, '') || ' ' || body)", persisted=True),
nullable=True,
)
def __repr__(self) -> str:
return f"<ScribeMessage {self.id} from={self.from_identity}>"
class ScribeMailboxEntry(Base):
"""One row per recipient address per message — what lands in a mailbox."""
__tablename__ = "scribe_mailbox_entries"
__table_args__ = (
UniqueConstraint("address", "message_id", name="uq_scribe_mailbox_entries_address_message"),
)
id: Mapped[int] = mapped_column(Integer, Identity(always=False), primary_key=True)
address: Mapped[str] = mapped_column(String(63), nullable=False)
# FK to tenants.id added by migration 007 (core-api T-8).
owner_tenant: Mapped[str] = mapped_column(
String(63), ForeignKey("tenants.id", ondelete="RESTRICT"), nullable=False
)
message_id: Mapped[int] = mapped_column(
Integer, ForeignKey("scribe_messages.id"), nullable=False
)
read_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
# Migration 009 (core-api T-13): deletion is per mailbox entry, never
# per message (D-66/D-70) — independent nullable timestamps, mirroring
# book_records.archived_at/trashed_at. The service layer's view=trash
# takes priority over view=archive when both are set.
archived_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
trashed_at: Mapped[Optional[datetime]] = mapped_column(DateTime(timezone=True), nullable=True)
tags: Mapped[List[str]] = mapped_column(ARRAY(Text), nullable=False, default=list)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
server_default=func.now(),
nullable=False,
)
def __repr__(self) -> str:
return f"<ScribeMailboxEntry {self.id} address={self.address} message={self.message_id}>"
+95
View File
@@ -0,0 +1,95 @@
"""
Scribe Domain Schemas
Pydantic request/response payloads for the Scribe (D-66, D-68, D-73). This
round ships `register`, `list` (addresses), `send`, `mailbox`, `history`,
`find`, `mark`, `tag`/`untag`, archive/trash a mailbox entry and look one
up by message number (core-api T-13/T-9) — groups, subscriptions, the
notification matrix and delivery are later work (dossier core-api T-5).
"""
from datetime import datetime
from typing import List, Literal, Optional
from pydantic import Field
from src.domains.scribe.models import ADDRESS_NAME_PATTERN
from src.shared.base import BaseSchema
MailboxView = Literal["inbox", "archive", "trash", "all"]
class AddressRegisterRequest(BaseSchema):
"""`register` — the name only; owner (tenant, consumer) comes from the caller."""
name: str = Field(..., pattern=ADDRESS_NAME_PATTERN)
class AddressResponse(BaseSchema):
name: str
owner_consumer: Optional[str]
created_by: str
created_at: datetime
owned: bool = Field(..., description="True when the caller's consumer registered this address")
class AddressListResponse(BaseSchema):
addresses: List[AddressResponse]
class MessageSendRequest(BaseSchema):
"""`send` — `from` is never in the body; it is the caller's own identity."""
to: List[str] = Field(..., min_length=1)
subject: Optional[str] = Field(None, max_length=500)
body: str = Field(..., min_length=1)
tags: List[str] = Field(default_factory=list)
reply_to: Optional[int] = None
class MessageResponse(BaseSchema):
id: int
topics: List[str]
from_identity: str
from_consumer: Optional[str]
subject: Optional[str]
body: str
tags: List[str]
reply_to: Optional[int]
created_at: datetime
class MailboxEntryResponse(BaseSchema):
"""`mailbox` / `find` rows, and the result of `mark`/`tag`/`untag`."""
id: int
address: str
read_at: Optional[datetime]
archived_at: Optional[datetime]
trashed_at: Optional[datetime]
tags: List[str]
created_at: datetime
message: MessageResponse
class MailboxListResponse(BaseSchema):
entries: List[MailboxEntryResponse]
total: int
limit: int
class HistoryResponse(BaseSchema):
address: str
messages: List[MessageResponse]
class MailboxPatchRequest(BaseSchema):
"""`mark` (`read`), `archive`/`unarchive`, `trash`/`restore` and
`tag`/`untag` (`add_tags`/`remove_tags`) share one route. `archived`/
`trashed` are independent: `true` sets the timestamp (if not already
set), `false` clears it (workspace core-api T-13)."""
read: Optional[bool] = None
archived: Optional[bool] = None
trashed: Optional[bool] = None
add_tags: Optional[List[str]] = None
remove_tags: Optional[List[str]] = None
+401
View File
@@ -0,0 +1,401 @@
"""
Scribe Domain Service
Business logic for the Scribe — the household's message bus (D-66, D-68,
D-73). Every method starts by requiring a tenant on the caller and ends by
filtering its query on it: **an address or message in another tenant
answers exactly like one that does not exist** (dossier core-api T-5), so
no method here ever confirms an id across that boundary.
Methods take `session: AsyncSession` per call rather than storing it on
`self`, following src/domains/book/service.py.
"""
from datetime import datetime, timezone
from typing import List, Optional, Tuple
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
from src.domains.auth.callers import Caller
from src.domains.scribe.models import (
RESERVED_ADDRESS_NAMES,
ScribeAddress,
ScribeMailboxEntry,
ScribeMessage,
)
from src.domains.scribe.schemas import MailboxPatchRequest, MessageSendRequest
from src.domains.tenants.service import TenantError, get_tenants_service
from src.shared.logging import get_logger
logger = get_logger(__name__)
class ScribeError(Exception):
"""Raised for a Scribe-domain failure; the controller maps it to a response."""
def __init__(self, code: str, status_code: int = 400, **extra):
self.code = code
self.status_code = status_code
self.extra = extra
super().__init__(code)
class ScribeService:
"""Service for Scribe operations."""
# =========================================================================
# Internal helpers
# =========================================================================
async def _require_tenant(self, session: AsyncSession, caller: Caller) -> str:
"""Every Scribe operation needs a tenant; a caller without one gets
a clear error rather than a query that silently matches nothing.
Delegates to the tenant registry's one enforcement point (core-api
T-8 / workspace D-73), which additionally refuses an unknown
tenant, a suspended one, or an Office consumer with no grant in it.
`TenantError` is translated to `ScribeError` so every call site and
the controller's single `except ScribeError` keep working
unchanged."""
try:
return await get_tenants_service().require_tenant(session, caller)
except TenantError as error:
raise ScribeError(error.code, error.status_code) from error
def _validate_address_name(self, name: str) -> None:
"""Beyond the schema's regex (shape only): not `group-*`, not reserved.
Mirrors src/domains/auth/callers.py's `_validate_identity` rule —
an address is named for an identity (D-66)."""
if name.startswith("group-") or name in RESERVED_ADDRESS_NAMES:
raise ScribeError("address_invalid", 400)
async def _get_address(self, session: AsyncSession, tenant: str, name: str) -> Optional[ScribeAddress]:
return await session.get(ScribeAddress, (tenant, name))
async def _require_owned_identity(self, session: AsyncSession, caller: Caller, tenant: str) -> None:
"""mailbox/find read the caller's own identity's mailbox: the identity
must be a registered address owned by the caller's consumer, else
403 identity_not_owned — the same refusal `send` uses for `from`."""
address = await self._get_address(session, tenant, caller.identity)
if address is None or address.owner_consumer != caller.consumer:
raise ScribeError("identity_not_owned", 403)
async def _messages_for_entries(
self, session: AsyncSession, entries: List[ScribeMailboxEntry]
) -> dict:
if not entries:
return {}
ids = [entry.message_id for entry in entries]
stmt = select(ScribeMessage).where(ScribeMessage.id.in_(ids))
messages = (await session.execute(stmt)).scalars().all()
return {message.id: message for message in messages}
# =========================================================================
# register / list addresses
# =========================================================================
async def register_address(
self, session: AsyncSession, name: str, caller: Caller
) -> Tuple[ScribeAddress, bool]:
"""`register` — idempotent for the same consumer; 409 address_taken
for another consumer in the tenant (D-66)."""
tenant = await self._require_tenant(session, caller)
self._validate_address_name(name)
existing = await self._get_address(session, tenant, name)
if existing is not None:
if existing.owner_consumer == caller.consumer:
return existing, True
raise ScribeError("address_taken", 409)
address = ScribeAddress(
owner_tenant=tenant,
name=name,
owner_consumer=caller.consumer,
created_by=caller.identity,
)
session.add(address)
await session.commit()
await session.refresh(address)
logger.info(f"Scribe address registered: {tenant}/{name} by={caller.identity}")
return address, True
async def list_addresses(
self, session: AsyncSession, caller: Caller
) -> List[Tuple[ScribeAddress, bool]]:
"""`list` — the caller's tenant; mark which the caller's consumer owns."""
tenant = await self._require_tenant(session, caller)
stmt = (
select(ScribeAddress)
.where(ScribeAddress.owner_tenant == tenant)
.order_by(ScribeAddress.name)
)
addresses = (await session.execute(stmt)).scalars().all()
return [(a, a.owner_consumer == caller.consumer) for a in addresses]
# =========================================================================
# send
# =========================================================================
async def send_message(
self, session: AsyncSession, data: MessageSendRequest, caller: Caller
) -> ScribeMessage:
"""`send` — `from` is the caller's own identity, never the body."""
tenant = await self._require_tenant(session, caller)
from_address = await self._get_address(session, tenant, caller.identity)
if from_address is None or from_address.owner_consumer != caller.consumer:
raise ScribeError("identity_not_owned", 403)
recipients = list(dict.fromkeys(data.to)) # de-dup, keep order
found_stmt = select(ScribeAddress.name).where(
ScribeAddress.owner_tenant == tenant, ScribeAddress.name.in_(recipients)
)
found = set((await session.execute(found_stmt)).scalars().all())
missing = [name for name in recipients if name not in found]
if missing:
raise ScribeError("unknown_address", 404, names=missing)
if data.reply_to is not None:
reply_target = await session.get(ScribeMessage, data.reply_to)
if reply_target is None or reply_target.owner_tenant != tenant:
raise ScribeError("not_found", 404)
tags = list(dict.fromkeys(data.tags))
message = ScribeMessage(
owner_tenant=tenant,
topics=recipients,
from_identity=caller.identity,
from_consumer=caller.consumer,
subject=data.subject,
body=data.body,
tags=tags,
reply_to=data.reply_to,
)
session.add(message)
await session.flush() # assign message.id
for recipient in recipients:
session.add(
ScribeMailboxEntry(address=recipient, owner_tenant=tenant, message_id=message.id)
)
await session.commit()
await session.refresh(message)
logger.info(
f"Scribe message sent: id={message.id} from={caller.identity} to={recipients}"
)
return message
# =========================================================================
# mailbox / find
# =========================================================================
async def get_mailbox(
self,
session: AsyncSession,
caller: Caller,
view: str = "inbox",
unread: Optional[bool] = None,
tag: Optional[str] = None,
limit: int = 50,
before_id: Optional[int] = None,
message_id: Optional[int] = None,
) -> Tuple[List[Tuple[ScribeMailboxEntry, ScribeMessage]], int]:
"""`mailbox` — the caller's identity's entries, unread first.
`message_id`, when given, is a direct lookup rather than a filtered
list: zero or one entry (the unique constraint on
(address, message_id) guarantees at most one), in any view — it
ignores `view`/`unread`/`tag`/`before_id` entirely, which is the
point (core-api T-9): a client that already knows the message
number never pages the unread-first/id-descending listing to find
it.
Otherwise `view` selects which of the independent `archived_at`/
`trashed_at` states to show: `inbox` (default) is neither,
`archive` is archived and not trashed, `trash` is trashed
(regardless of archived_at — trashing hides an entry from the
archive view too), `all` applies no filter on either column.
"""
tenant = await self._require_tenant(session, caller)
await self._require_owned_identity(session, caller, tenant)
if message_id is not None:
lookup_query = select(ScribeMailboxEntry).where(
ScribeMailboxEntry.owner_tenant == tenant,
ScribeMailboxEntry.address == caller.identity,
ScribeMailboxEntry.message_id == message_id,
)
entries = list((await session.execute(lookup_query)).scalars().all())
messages_by_id = await self._messages_for_entries(session, entries)
results = [(entry, messages_by_id[entry.message_id]) for entry in entries]
return results, len(results)
conditions = [
ScribeMailboxEntry.owner_tenant == tenant,
ScribeMailboxEntry.address == caller.identity,
]
if view == "archive":
conditions.append(ScribeMailboxEntry.archived_at.isnot(None))
conditions.append(ScribeMailboxEntry.trashed_at.is_(None))
elif view == "trash":
conditions.append(ScribeMailboxEntry.trashed_at.isnot(None))
elif view == "all":
pass
else: # "inbox" (default)
conditions.append(ScribeMailboxEntry.archived_at.is_(None))
conditions.append(ScribeMailboxEntry.trashed_at.is_(None))
if unread:
conditions.append(ScribeMailboxEntry.read_at.is_(None))
if tag is not None:
conditions.append(ScribeMailboxEntry.tags.contains([tag]))
if before_id is not None:
conditions.append(ScribeMailboxEntry.id < before_id)
base_query = select(ScribeMailboxEntry).where(*conditions)
count_query = select(func.count()).select_from(base_query.subquery())
total = (await session.execute(count_query)).scalar() or 0
query = (
base_query
.order_by(ScribeMailboxEntry.read_at.is_(None).desc(), ScribeMailboxEntry.id.desc())
.limit(limit)
)
entries = list((await session.execute(query)).scalars().all())
messages_by_id = await self._messages_for_entries(session, entries)
return [(entry, messages_by_id[entry.message_id]) for entry in entries], total
async def find_messages(
self, session: AsyncSession, caller: Caller, q: str, limit: int = 50
) -> Tuple[List[Tuple[ScribeMailboxEntry, ScribeMessage]], int]:
"""`find` — full text over messages in the caller identity's
mailbox, every entry that is not trashed (core-api T-13: trash
takes an entry out of `find` the same way it takes it out of
`inbox` and `archive`); archived entries still match."""
tenant = await self._require_tenant(session, caller)
await self._require_owned_identity(session, caller, tenant)
tsquery = func.websearch_to_tsquery("simple", q)
base_query = (
select(ScribeMailboxEntry)
.join(ScribeMessage, ScribeMailboxEntry.message_id == ScribeMessage.id)
.where(
ScribeMailboxEntry.owner_tenant == tenant,
ScribeMailboxEntry.address == caller.identity,
ScribeMailboxEntry.trashed_at.is_(None),
ScribeMessage.search.op("@@")(tsquery),
)
)
count_query = select(func.count()).select_from(base_query.subquery())
total = (await session.execute(count_query)).scalar() or 0
query = base_query.order_by(ScribeMailboxEntry.id.desc()).limit(limit)
entries = list((await session.execute(query)).scalars().all())
messages_by_id = await self._messages_for_entries(session, entries)
return [(entry, messages_by_id[entry.message_id]) for entry in entries], total
# =========================================================================
# history
# =========================================================================
async def get_history(
self,
session: AsyncSession,
name: str,
caller: Caller,
since_id: Optional[int] = None,
limit: int = 100,
) -> List[ScribeMessage]:
"""`history` — the retained messages sent to that address, oldest
first. Reading never consumes or marks: this reads scribe_messages
directly, never scribe_mailbox_entries."""
tenant = await self._require_tenant(session, caller)
address = await self._get_address(session, tenant, name)
if address is None:
raise ScribeError("unknown_address", 404)
conditions = [
ScribeMessage.owner_tenant == tenant,
ScribeMessage.topics.contains([name]),
]
if since_id is not None:
conditions.append(ScribeMessage.id > since_id)
stmt = (
select(ScribeMessage)
.where(*conditions)
.order_by(ScribeMessage.id.asc())
.limit(limit)
)
return list((await session.execute(stmt)).scalars().all())
# =========================================================================
# mark / tag / untag
# =========================================================================
async def patch_mailbox_entry(
self, session: AsyncSession, entry_id: int, data: MailboxPatchRequest, caller: Caller
) -> Tuple[ScribeMailboxEntry, ScribeMessage]:
"""`mark` (`read`), `archive`/`unarchive`, `trash`/`restore` and
`tag`/`untag` (`add_tags`/`remove_tags`) share this route. An entry
outside the caller's tenant, or addressed to an identity the caller
is not reading as, answers 404 not_found — indistinguishable from a
nonexistent entry, so an id never confirms someone else's mail
exists. `archived`/`trashed` are independent timestamps (core-api
T-13): `true` sets the one not already set, `false` clears it —
trashing does not clear `archived_at`, so restoring a message that
was archived before it was trashed returns it to the archive view,
not the inbox."""
tenant = await self._require_tenant(session, caller)
entry = await session.get(ScribeMailboxEntry, entry_id)
if entry is None or entry.owner_tenant != tenant or entry.address != caller.identity:
raise ScribeError("not_found", 404)
if data.read is True and entry.read_at is None:
entry.read_at = datetime.now(timezone.utc)
elif data.read is False:
entry.read_at = None
if data.archived is True and entry.archived_at is None:
entry.archived_at = datetime.now(timezone.utc)
elif data.archived is False:
entry.archived_at = None
if data.trashed is True and entry.trashed_at is None:
entry.trashed_at = datetime.now(timezone.utc)
elif data.trashed is False:
entry.trashed_at = None
if data.add_tags:
current = set(entry.tags)
entry.tags = list(entry.tags) + [t for t in data.add_tags if t not in current]
if data.remove_tags:
remove = set(data.remove_tags)
entry.tags = [t for t in entry.tags if t not in remove]
await session.commit()
await session.refresh(entry)
message = await session.get(ScribeMessage, entry.message_id)
return entry, message
# Singleton instance
_scribe_service: Optional[ScribeService] = None
def get_scribe_service() -> ScribeService:
"""Get singleton Scribe service instance"""
global _scribe_service
if _scribe_service is None:
_scribe_service = ScribeService()
return _scribe_service
+10
View File
@@ -0,0 +1,10 @@
"""
Tenants Domain
The tenant registry (core-api T-8 / workspace D-73): tenants, consumer
grants, and the one enforcement point every tenant-scoped domain's own
`_require_tenant` calls through to (Book, Scribe). Served under /tenants/*.
"""
from src.domains.tenants.controller import tenants_controller
__all__ = ["tenants_controller"]
+93
View File
@@ -0,0 +1,93 @@
"""
Tenants Controller
The tenant registry's read API (core-api T-8 / workspace D-73): `GET
/tenants/me` and `GET /tenants`. No write routes this round — provisioning
is later (workspace T-195).
"""
from fastapi import APIRouter, Depends, HTTPException
from sqlalchemy.ext.asyncio import AsyncSession
from src.domains.auth.callers import Caller, require_caller
from src.domains.tenants.schemas import TenantListResponse, TenantMeResponse, TenantSummary
from src.domains.tenants.service import TenantError, get_tenants_service
from src.shared.base import BaseController
from src.shared.database import get_async_session
from src.shared.logging import get_logger
logger = get_logger(__name__)
require_tenants_reader = require_caller("office", "user")
require_tenants_lister = require_caller("office", "user", "tatlock", "scheduler")
def _summary(tenant) -> TenantSummary:
return TenantSummary(
id=tenant.id, display_name=tenant.display_name, kind=tenant.kind, status=tenant.status
)
class TenantsController(BaseController):
"""Controller for the tenant registry's read API."""
def __init__(self):
super().__init__(prefix="/tenants", tags=["Tenants"])
def create_router(self) -> APIRouter:
router = APIRouter(prefix=self.prefix, tags=self.tags)
service = get_tenants_service()
# =====================================================================
# me
# =====================================================================
@router.get(
"/me",
response_model=TenantMeResponse,
summary="The caller's own tenant and the consumers granted in it",
)
async def get_me(
caller: Caller = Depends(require_tenants_reader),
session: AsyncSession = Depends(get_async_session),
):
try:
tenant, consumers, default_address = await service.get_me(session, caller)
except TenantError as error:
raise HTTPException(
status_code=error.status_code, detail={"code": error.code}
) from error
return TenantMeResponse(
id=tenant.id,
display_name=tenant.display_name,
kind=tenant.kind,
status=tenant.status,
consumers=consumers,
default_address=default_address,
)
# =====================================================================
# list
# =====================================================================
@router.get(
"",
response_model=TenantListResponse,
summary="Every tenant (tatlock/scheduler) or the caller's own, as a list of one",
)
async def list_tenants(
caller: Caller = Depends(require_tenants_lister),
session: AsyncSession = Depends(get_async_session),
):
try:
tenants = await service.list_tenants(session, caller)
except TenantError as error:
raise HTTPException(
status_code=error.status_code, detail={"code": error.code}
) from error
return TenantListResponse(tenants=[_summary(tenant) for tenant in tenants])
return router
# Create controller instance
tenants_controller = TenantsController()
+98
View File
@@ -0,0 +1,98 @@
"""
Tenants Domain Models
The tenant registry (core-api T-8 / workspace D-73): every Book and Scribe
`owner_tenant` column carries a foreign key to `tenants.id` (migration 007),
and `TenantsService.require_tenant` is the one place a caller's tenant is
checked known, active, and — for an Office caller — granted to the calling
consumer. SQLAlchemy 2.0 Mapped/mapped_column style, timestamptz, following
src/domains/book/models.py.
`TenantConsumer.default_address` (migration 008, core-api T-11) is each
grant's fallback address for a caller that names none and holds no session
binding — claude.ai's connector keeps no MCP session between tool calls, so
the Office needs this to answer without one. The composite FK keeps it
honest: a default always names a real address in the same tenant.
"""
from datetime import datetime
from typing import Optional
from sqlalchemy import (
CheckConstraint,
DateTime,
ForeignKey,
ForeignKeyConstraint,
String,
Text,
func,
)
from sqlalchemy.orm import Mapped, mapped_column
from src.shared.database import Base
TENANT_KINDS = ("household", "person", "test")
TENANT_STATUSES = ("active", "suspended")
# The slug shape a tenant id carries, identical to
# src/domains/auth/callers.py's `_TENANT_RE` (tatlock's own tenant ids carry
# underscores, e.g. `llm_tester`) — not imported from there because that
# module's constants are private to it, following
# src/domains/scribe/models.py's same note about ADDRESS_NAME_PATTERN.
# Asserted equal in tests/test_tenants.py.
TENANT_ID_PATTERN = r"^[a-z0-9_][a-z0-9_.-]{0,62}$"
class Tenant(Base):
"""A tenant: tatlock's own per-household/person/test namespace."""
__tablename__ = "tenants"
__table_args__ = (
CheckConstraint("kind IN ('household', 'person', 'test')", name="ck_tenants_kind"),
CheckConstraint("status IN ('active', 'suspended')", name="ck_tenants_status"),
)
id: Mapped[str] = mapped_column(String(63), primary_key=True)
display_name: Mapped[str] = mapped_column(Text, nullable=False)
kind: Mapped[str] = mapped_column(String(20), nullable=False)
status: Mapped[str] = mapped_column(String(20), nullable=False, default="active")
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False
)
updated_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
server_default=func.now(),
onupdate=func.now(),
nullable=False,
)
def __repr__(self) -> str:
return f"<Tenant {self.id} kind={self.kind} status={self.status}>"
class TenantConsumer(Base):
"""A grant: consumer `consumer` may act in tenant `tenant_id`, with an
optional default address (migration 008, core-api T-11) — the composite
FK below means a non-null default always names a real
`scribe_addresses` row in this same tenant."""
__tablename__ = "tenant_consumers"
__table_args__ = (
ForeignKeyConstraint(
["tenant_id", "default_address"],
["scribe_addresses.owner_tenant", "scribe_addresses.name"],
name="fk_tenant_consumers_default_address",
),
)
tenant_id: Mapped[str] = mapped_column(
String(63), ForeignKey("tenants.id"), primary_key=True
)
consumer: Mapped[str] = mapped_column(String(63), primary_key=True)
# FK to scribe_addresses (owner_tenant, name) added by migration 008.
default_address: Mapped[Optional[str]] = mapped_column(String(63), nullable=True)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True), server_default=func.now(), nullable=False
)
def __repr__(self) -> str:
return f"<TenantConsumer {self.tenant_id}/{self.consumer}>"
+35
View File
@@ -0,0 +1,35 @@
"""
Tenants Domain Schemas
Pydantic request/response payloads for the tenant registry (core-api T-8 /
workspace D-73). No write routes this round — provisioning is later
(workspace T-195).
"""
from typing import List, Optional
from src.shared.base import BaseSchema
class TenantSummary(BaseSchema):
"""The shape `GET /tenants` answers with, per tenant."""
id: str
display_name: str
kind: str
status: str
class TenantMeResponse(TenantSummary):
"""`GET /tenants/me` — the caller's own tenant plus the consumers
granted in it, the cheap "am I allowed here" probe (D-73).
`default_address` (core-api T-11) is the caller's own consumer's
default address, or null — null for a user-kind caller (no consumer to
look a default up for) and for an office caller whose grant has none."""
consumers: List[str]
default_address: Optional[str] = None
class TenantListResponse(BaseSchema):
tenants: List[TenantSummary]
+142
View File
@@ -0,0 +1,142 @@
"""
Tenants Domain Service
The tenant registry (core-api T-8 / workspace D-73): `require_tenant` is
the one enforcement point Book's and Scribe's own `_require_tenant` now
delegate to, rather than each checking only presence (core-api T-1/T-5's
original shape, which let a typo'd claim open a fresh, empty namespace
instead of being refused).
It takes the session, so it is one query: a LEFT JOIN against the grant
table, so presence, status and (for an Office caller) the grant are all
read in a single round trip. When `caller.consumer` is `None` (tatlock,
scheduler, or a user-kind caller — core-api T-6 carries no consumer for
those), the join condition `TenantConsumer.consumer == None` compiles to
`IS NULL`, which never matches a grant row (consumer is NOT NULL there),
so the grant side of the row is always empty for them — correct, since the
grant check below is skipped outright when `caller.consumer` is `None`.
"""
from typing import List, Optional, Tuple
from sqlalchemy import and_, select
from sqlalchemy.ext.asyncio import AsyncSession
from src.domains.auth.callers import Caller
from src.domains.tenants.models import Tenant, TenantConsumer
from src.shared.logging import get_logger
logger = get_logger(__name__)
class TenantError(Exception):
"""Raised for a tenant-registry failure; the controller maps it to a response."""
def __init__(self, code: str, status_code: int = 400):
self.code = code
self.status_code = status_code
super().__init__(code)
class TenantsService:
"""The tenant registry's one enforcement point, and its read API."""
async def require_tenant(self, session: AsyncSession, caller: Caller) -> str:
"""
The one enforcement point (core-api T-8 / workspace D-73):
- no tenant on the caller -> 400 `tenant_required` (unchanged)
- tenant not in the table -> 403 `tenant_unknown`
- tenant `suspended` -> 403 `tenant_suspended`
- an Office caller (carries a consumer) whose consumer holds no
grant in that tenant -> 403 `consumer_not_granted`
- a user-kind caller carries no consumer (core-api T-6, not this
round): tenant checks only.
Returns the tenant id on success.
"""
if not caller.tenant:
raise TenantError("tenant_required", 400)
tenant = caller.tenant
stmt = (
select(Tenant.status, TenantConsumer.consumer)
.outerjoin(
TenantConsumer,
and_(
TenantConsumer.tenant_id == Tenant.id,
TenantConsumer.consumer == caller.consumer,
),
)
.where(Tenant.id == tenant)
)
row = (await session.execute(stmt)).first()
if row is None:
raise TenantError("tenant_unknown", 403)
status, granted_consumer = row
if status == "suspended":
raise TenantError("tenant_suspended", 403)
if caller.consumer is not None and granted_consumer is None:
raise TenantError("consumer_not_granted", 403)
return tenant
async def get_me(
self, session: AsyncSession, caller: Caller
) -> Tuple[Tenant, List[str], Optional[str]]:
"""`GET /tenants/me` — through the same helper, so "am I allowed
here" shares its answer with every other tenant-scoped call.
The third element is the caller's own consumer's default address
(core-api T-11), or None — looked up only for an office caller
(`caller.consumer is not None`); a user-kind caller carries no
consumer, so there is no grant row to hold a default for it."""
tenant_id = await self.require_tenant(session, caller)
tenant = await session.get(Tenant, tenant_id)
consumers_stmt = (
select(TenantConsumer.consumer)
.where(TenantConsumer.tenant_id == tenant_id)
.order_by(TenantConsumer.consumer)
)
consumers = list((await session.execute(consumers_stmt)).scalars().all())
default_address: Optional[str] = None
if caller.consumer is not None:
default_address = await session.scalar(
select(TenantConsumer.default_address).where(
TenantConsumer.tenant_id == tenant_id,
TenantConsumer.consumer == caller.consumer,
)
)
return tenant, consumers, default_address
async def list_tenants(self, session: AsyncSession, caller: Caller) -> List[Tenant]:
"""
`GET /tenants` — `tatlock` and `scheduler` callers (no tenant) see
every tenant; a tenant-bearing caller sees only its own, as a list
of one (the hard boundary: another tenant's existence is never
revealed, so this still runs through `require_tenant`); a user
caller with no tenant is 400 `tenant_required`.
"""
if caller.tenant is None:
if caller.kind in ("tatlock", "scheduler"):
stmt = select(Tenant).order_by(Tenant.id)
return list((await session.execute(stmt)).scalars().all())
raise TenantError("tenant_required", 400)
tenant_id = await self.require_tenant(session, caller)
tenant = await session.get(Tenant, tenant_id)
return [tenant]
# Singleton instance
_tenants_service: Optional[TenantsService] = None
def get_tenants_service() -> TenantsService:
"""Get singleton Tenants service instance"""
global _tenants_service
if _tenants_service is None:
_tenants_service = TenantsService()
return _tenants_service
+9 -2
View File
@@ -1,9 +1,16 @@
"""
Tools Domain
Provides utility tool endpoints including DNS lookups.
Provides utility tool endpoints including DNS lookups and system stats.
"""
from src.domains.tools.controller import tools_controller
from src.domains.tools.dns import DNSService, DNSQueryError
from src.domains.tools.system import SystemStatsService, SystemStatsResponse
__all__ = ["tools_controller", "DNSService", "DNSQueryError"]
__all__ = [
"tools_controller",
"DNSService",
"DNSQueryError",
"SystemStatsService",
"SystemStatsResponse",
]
+181 -1
View File
@@ -3,14 +3,24 @@ Tools Controller
Provides utility tool endpoints including:
- DNS lookups
- System stats
- Environment data (weather, forecast, sun times)
"""
from fastapi import APIRouter, HTTPException, status
from typing import Dict, Optional
from fastapi import APIRouter, HTTPException, status, Depends
from src.shared.base import BaseController
from src.shared.logging import get_logger
from src.domains.tools.dns.schemas import DNSLookupRequest, DNSLookupResponse
from src.domains.tools.dns.service import DNSService
from src.domains.tools.dns.exceptions import DNSQueryError
from src.domains.tools.system.schemas import SystemStatsResponse
from src.domains.tools.system.service import SystemStatsService
from src.domains.tools.environment.schemas import EnvironmentResponse
from src.domains.tools.environment.service import EnvironmentService
from src.domains.tools.news.schemas import NewsResponse
from src.domains.tools.news.service import NewsService
from src.domains.auth.oidc import get_optional_user
logger = get_logger(__name__)
@@ -21,11 +31,16 @@ class ToolsController(BaseController):
Provides endpoints for:
- DNS lookups
- System stats
- Environment data (weather, forecast, sun times)
"""
def __init__(self):
super().__init__(prefix="/tools", tags=["Tools"])
self.dns_service = DNSService()
self.system_stats_service = SystemStatsService()
self.environment_service = EnvironmentService()
self.news_service = NewsService()
def create_router(self) -> APIRouter:
"""Create and configure the router"""
@@ -95,6 +110,171 @@ class ToolsController(BaseController):
detail="An unexpected error occurred during DNS lookup"
)
@router.get(
"/system/stats",
response_model=SystemStatsResponse,
status_code=status.HTTP_200_OK,
summary="Get host system statistics",
description="""
Get real-time host system resource statistics.
Returns CPU, memory, disk, network, and GPU/VRAM usage for the host machine
(not Docker container metrics).
**Metrics Returned:**
- **CPU:** Usage percentage, core count, load averages
- **Memory:** Usage percentage, total/used/available bytes
- **Disk:** Usage percentage, total/used/free bytes (root partition)
- **Network:** Total bytes sent/received
- **GPU:** VRAM usage (if NVIDIA GPU available via nvidia-smi)
**Use Cases:**
- Dashboard system monitoring widgets
- Health checks and alerting
- Capacity planning
"""
)
async def get_system_stats() -> SystemStatsResponse:
"""
Get current host system statistics
Returns:
System statistics including CPU, memory, disk, network, and GPU
Raises:
HTTPException: 500 for processing errors
"""
try:
logger.info("Fetching system stats")
result = await self.system_stats_service.get_stats()
return result
except Exception as e:
logger.error(f"Failed to get system stats: {str(e)}", exc_info=True)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Failed to collect system stats: {str(e)}"
)
@router.get(
"/environment",
response_model=EnvironmentResponse,
status_code=status.HTTP_200_OK,
summary="Get environment data",
description="""
Get current environment data including weather, forecast, and sun times.
Fetches data from the Qdrant volatile collection for the authenticated user.
Falls back to 'default' user if not authenticated.
**Data Returned:**
- **Weather:** Current temperature, conditions, humidity, wind
- **Forecast:** Multi-day weather outlook
- **Sun Times:** Sunrise, sunset, daylight duration
- **Air Quality:** AQI and pollutant levels (if available)
**Data Source:** Qdrant volatile_{user} collection
**Use Cases:**
- Dashboard environment widgets
- Home automation context
- Weather-based automations
"""
)
async def get_environment(
user: Optional[Dict] = Depends(get_optional_user),
) -> EnvironmentResponse:
"""
Get current environment data
Args:
user: Optional authenticated user from OIDC
Returns:
Environment data including weather, forecast, sun times
Raises:
HTTPException: 500 for processing errors
"""
try:
# Get user identifier from OIDC claims, fallback to 'default'
user_id = "default"
if user:
user_id = user.get("preferred_username") or user.get("sub", "default")
# Strip email domain if present (e.g., "user@example.com" -> "user")
if "@" in user_id:
user_id = user_id.split("@")[0]
logger.info(f"Fetching environment data for user: {user_id}")
result = await self.environment_service.get_current(user_id)
return result
except Exception as e:
logger.error(f"Failed to get environment data: {str(e)}", exc_info=True)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Failed to fetch environment data: {str(e)}"
)
@router.get(
"/news",
response_model=NewsResponse,
status_code=status.HTTP_200_OK,
summary="Get news headlines",
description="""
Get news headlines for the authenticated user.
Fetches news data from the Qdrant volatile collection for the authenticated user.
Falls back to 'default' user if not authenticated.
**Data Returned:**
- **Headlines:** List of news headlines with title, description, source, url
- **Category:** News category (general, technology, etc.)
- **Sources:** List of news sources
**Data Source:** Qdrant volatile_{user} collection (news namespace)
**Use Cases:**
- Dashboard news ticker
- News feed widgets
- Information display
"""
)
async def get_news(
user: Optional[Dict] = Depends(get_optional_user),
) -> NewsResponse:
"""
Get news headlines
Args:
user: Optional authenticated user from OIDC
Returns:
News headlines response
Raises:
HTTPException: 500 for processing errors
"""
try:
# Get user identifier from OIDC claims, fallback to 'default'
user_id = "default"
if user:
user_id = user.get("preferred_username") or user.get("sub", "default")
# Strip email domain if present (e.g., "user@example.com" -> "user")
if "@" in user_id:
user_id = user_id.split("@")[0]
logger.info(f"Fetching news data for user: {user_id}")
result = await self.news_service.get_news(user_id)
return result
except Exception as e:
logger.error(f"Failed to get news data: {str(e)}", exc_info=True)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Failed to fetch news data: {str(e)}"
)
return router
+23
View File
@@ -0,0 +1,23 @@
"""
Environment data module for Tools domain.
Provides access to weather, forecast, sun times, and air quality data
from the Qdrant volatile collection.
"""
from src.domains.tools.environment.schemas import (
WeatherData,
ForecastDay,
SunTimesData,
AirQualityData,
EnvironmentResponse,
)
from src.domains.tools.environment.service import EnvironmentService
__all__ = [
"WeatherData",
"ForecastDay",
"SunTimesData",
"AirQualityData",
"EnvironmentResponse",
"EnvironmentService",
]
+185
View File
@@ -0,0 +1,185 @@
"""
Environment data schemas for Tools domain.
Provides Pydantic models for weather, forecast, sun times, and air quality data
retrieved from the Qdrant volatile collection.
"""
from datetime import datetime
from typing import Optional, List, Any
from pydantic import Field
from src.shared.base import BaseSchema
class WeatherData(BaseSchema):
"""Current weather conditions."""
temperature: Optional[float] = Field(
None,
description="Current temperature in Celsius"
)
feels_like: Optional[float] = Field(
None,
description="Feels-like temperature in Celsius"
)
conditions: Optional[str] = Field(
None,
description="Weather conditions description (e.g., 'Partly Cloudy')"
)
humidity: Optional[int] = Field(
None,
ge=0,
le=100,
description="Humidity percentage"
)
wind_speed: Optional[float] = Field(
None,
description="Wind speed in km/h"
)
wind_direction: Optional[str] = Field(
None,
description="Wind direction (e.g., 'NW')"
)
pressure: Optional[float] = Field(
None,
description="Atmospheric pressure in hPa"
)
visibility: Optional[float] = Field(
None,
description="Visibility in km"
)
uv_index: Optional[float] = Field(
None,
description="UV index"
)
location: Optional[str] = Field(
None,
description="Location name"
)
icon: Optional[str] = Field(
None,
description="Weather icon code or URL"
)
class ForecastDay(BaseSchema):
"""Single day forecast data."""
date: str = Field(
...,
description="Date string (e.g., '2025-01-07')"
)
high: Optional[float] = Field(
None,
description="High temperature in Celsius"
)
low: Optional[float] = Field(
None,
description="Low temperature in Celsius"
)
conditions: Optional[str] = Field(
None,
description="Weather conditions description"
)
precipitation_chance: Optional[int] = Field(
None,
ge=0,
le=100,
description="Chance of precipitation percentage"
)
icon: Optional[str] = Field(
None,
description="Weather icon code or URL"
)
class SunTimesData(BaseSchema):
"""Sunrise and sunset times."""
sunrise: Optional[datetime] = Field(
None,
description="Sunrise time"
)
sunset: Optional[datetime] = Field(
None,
description="Sunset time"
)
daylight_minutes: Optional[int] = Field(
None,
description="Total daylight duration in minutes"
)
solar_noon: Optional[datetime] = Field(
None,
description="Solar noon time"
)
dawn: Optional[datetime] = Field(
None,
description="Civil dawn time"
)
dusk: Optional[datetime] = Field(
None,
description="Civil dusk time"
)
class AirQualityData(BaseSchema):
"""Air quality information."""
aqi: Optional[int] = Field(
None,
ge=0,
description="Air Quality Index"
)
quality: Optional[str] = Field(
None,
description="Quality category (Good, Moderate, Unhealthy, etc.)"
)
pm25: Optional[float] = Field(
None,
description="PM2.5 concentration in microg/m3"
)
pm10: Optional[float] = Field(
None,
description="PM10 concentration in microg/m3"
)
o3: Optional[float] = Field(
None,
description="Ozone concentration in ppb"
)
no2: Optional[float] = Field(
None,
description="Nitrogen dioxide concentration in ppb"
)
location: Optional[str] = Field(
None,
description="Location name"
)
class EnvironmentResponse(BaseSchema):
"""Combined environment data response."""
weather: Optional[WeatherData] = Field(
None,
description="Current weather conditions"
)
forecast: Optional[List[ForecastDay]] = Field(
None,
description="Multi-day weather forecast"
)
sun_times: Optional[SunTimesData] = Field(
None,
description="Sunrise/sunset times"
)
air_quality: Optional[AirQualityData] = Field(
None,
description="Air quality data (None if not available)"
)
updated_at: datetime = Field(
default_factory=datetime.utcnow,
description="Timestamp when data was fetched"
)
user: Optional[str] = Field(
None,
description="User identifier used for data lookup"
)
+287
View File
@@ -0,0 +1,287 @@
"""
Environment data service for Tools domain.
Fetches weather, forecast, sun times, and air quality data from
the Qdrant volatile collection.
"""
from datetime import datetime
from typing import Optional, Dict, Any, List
from src.shared.logging import get_logger
from src.shared.clients.qdrant_client import get_qdrant_client
from src.domains.tools.environment.schemas import (
WeatherData,
ForecastDay,
SunTimesData,
AirQualityData,
EnvironmentResponse,
)
logger = get_logger(__name__)
class EnvironmentService:
"""
Service for fetching environment data from Qdrant volatile collection.
Retrieves weather, forecast, sun times, and optionally air quality
data for a specific user.
"""
def __init__(self):
"""Initialize environment service with Qdrant client."""
self.qdrant = get_qdrant_client()
def _parse_weather(self, raw_data: Optional[Dict[str, Any]]) -> Optional[WeatherData]:
"""
Parse raw weather data into WeatherData schema.
Handles various field naming conventions that might come from
different weather APIs.
"""
if not raw_data:
return None
try:
# Handle wind direction - convert degrees to cardinal if integer
wind_dir = raw_data.get("wind_direction") or raw_data.get("wind_dir")
if isinstance(wind_dir, (int, float)):
wind_dir = self._degrees_to_cardinal(wind_dir)
return WeatherData(
temperature=raw_data.get("temperature") or raw_data.get("temp"),
feels_like=raw_data.get("feels_like") or raw_data.get("feelslike"),
conditions=raw_data.get("conditions") or raw_data.get("weather") or raw_data.get("description"),
humidity=raw_data.get("humidity"),
wind_speed=raw_data.get("wind_speed") or raw_data.get("windspeed") or raw_data.get("wind"),
wind_direction=wind_dir,
pressure=raw_data.get("pressure"),
visibility=raw_data.get("visibility"),
uv_index=raw_data.get("uv_index") or raw_data.get("uv"),
location=raw_data.get("location") or raw_data.get("city"),
icon=raw_data.get("icon") or raw_data.get("icon_url"),
)
except Exception as e:
logger.warning(f"Failed to parse weather data: {e}")
return None
def _degrees_to_cardinal(self, degrees: float) -> str:
"""Convert wind direction degrees to cardinal direction."""
directions = ["N", "NNE", "NE", "ENE", "E", "ESE", "SE", "SSE",
"S", "SSW", "SW", "WSW", "W", "WNW", "NW", "NNW"]
index = round(degrees / 22.5) % 16
return directions[index]
def _parse_forecast(self, raw_data: Any) -> Optional[List[ForecastDay]]:
"""
Parse raw forecast data into list of ForecastDay schemas.
Handles both list format and dict with nested list.
"""
if not raw_data:
return None
try:
# Normalize to list
forecast_list = raw_data
if isinstance(raw_data, dict):
# Check 'daily' first (scheduler format), then 'forecast', then 'days'
# Note: 'days' might be an integer count, so check 'daily' first
forecast_list = raw_data.get("daily") or raw_data.get("forecast")
if forecast_list is None:
days_value = raw_data.get("days")
if isinstance(days_value, list):
forecast_list = days_value
else:
forecast_list = []
if not isinstance(forecast_list, list):
return None
days = []
for day in forecast_list:
if isinstance(day, dict):
days.append(ForecastDay(
date=day.get("date", ""),
high=day.get("high") or day.get("temp_high") or day.get("maxtemp") or day.get("temp_max"),
low=day.get("low") or day.get("temp_low") or day.get("mintemp") or day.get("temp_min"),
conditions=day.get("conditions") or day.get("weather") or day.get("description"),
precipitation_chance=day.get("precipitation_chance") or day.get("pop") or day.get("precip"),
icon=day.get("icon"),
))
return days if days else None
except Exception as e:
logger.warning(f"Failed to parse forecast data: {e}")
return None
def _parse_sun_times(self, raw_data: Optional[Dict[str, Any]]) -> Optional[SunTimesData]:
"""
Parse raw sun times data into SunTimesData schema.
Handles datetime strings and calculates daylight minutes if not provided.
"""
if not raw_data:
return None
try:
# Prefer ISO format fields (sunrise_iso, sunset_iso) over time-only fields
sunrise = raw_data.get("sunrise_iso") or raw_data.get("sunrise")
sunset = raw_data.get("sunset_iso") or raw_data.get("sunset")
# Parse datetime strings if needed
if isinstance(sunrise, str):
# Handle time-only format (HH:MM) by combining with today's date
if len(sunrise) <= 5 and ":" in sunrise:
today = datetime.now().date()
sunrise = datetime.strptime(f"{today} {sunrise}", "%Y-%m-%d %H:%M")
else:
sunrise = datetime.fromisoformat(sunrise.replace("Z", "+00:00"))
if isinstance(sunset, str):
# Handle time-only format (HH:MM) by combining with today's date
if len(sunset) <= 5 and ":" in sunset:
today = datetime.now().date()
sunset = datetime.strptime(f"{today} {sunset}", "%Y-%m-%d %H:%M")
else:
sunset = datetime.fromisoformat(sunset.replace("Z", "+00:00"))
# Get daylight from various field names
daylight_minutes = raw_data.get("daylight_minutes") or raw_data.get("daylight")
if daylight_minutes is None:
# Try to calculate from daylight_duration_seconds or daylight_hours
daylight_seconds = raw_data.get("daylight_duration_seconds")
if daylight_seconds:
daylight_minutes = int(daylight_seconds / 60)
else:
daylight_hours = raw_data.get("daylight_hours")
if daylight_hours:
daylight_minutes = int(daylight_hours * 60)
elif sunrise and sunset:
daylight_minutes = int((sunset - sunrise).total_seconds() / 60)
# Parse optional fields
solar_noon = raw_data.get("solar_noon")
if isinstance(solar_noon, str):
solar_noon = datetime.fromisoformat(solar_noon.replace("Z", "+00:00"))
dawn = raw_data.get("dawn") or raw_data.get("civil_dawn")
if isinstance(dawn, str):
dawn = datetime.fromisoformat(dawn.replace("Z", "+00:00"))
dusk = raw_data.get("dusk") or raw_data.get("civil_dusk")
if isinstance(dusk, str):
dusk = datetime.fromisoformat(dusk.replace("Z", "+00:00"))
return SunTimesData(
sunrise=sunrise,
sunset=sunset,
daylight_minutes=daylight_minutes,
solar_noon=solar_noon,
dawn=dawn,
dusk=dusk,
)
except Exception as e:
logger.warning(f"Failed to parse sun times data: {e}")
return None
def _parse_air_quality(self, raw_data: Any) -> Optional[AirQualityData]:
"""
Parse raw air quality data into AirQualityData schema.
Handles both dict format and simple integer AQI value.
"""
if raw_data is None:
return None
try:
# Handle simple integer AQI
if isinstance(raw_data, (int, float)):
aqi = int(raw_data)
return AirQualityData(
aqi=aqi,
quality=self._aqi_to_quality(aqi),
)
if not isinstance(raw_data, dict):
return None
# Try various AQI field names - prefer US AQI, then European, then generic
aqi = raw_data.get("aqi") or raw_data.get("aqi_us") or raw_data.get("aqi_european") or raw_data.get("index")
if isinstance(aqi, (int, float)):
aqi = int(aqi)
return AirQualityData(
aqi=aqi,
quality=raw_data.get("quality") or (self._aqi_to_quality(aqi) if aqi else None),
pm25=raw_data.get("pm25") or raw_data.get("pm2_5"),
pm10=raw_data.get("pm10"),
o3=raw_data.get("o3") or raw_data.get("ozone"),
no2=raw_data.get("no2") or raw_data.get("nitrogen_dioxide"),
location=raw_data.get("location"),
)
except Exception as e:
logger.warning(f"Failed to parse air quality data: {e}")
return None
def _aqi_to_quality(self, aqi: int) -> str:
"""Convert AQI value to quality category string."""
if aqi <= 50:
return "Good"
elif aqi <= 100:
return "Moderate"
elif aqi <= 150:
return "Unhealthy for Sensitive Groups"
elif aqi <= 200:
return "Unhealthy"
elif aqi <= 300:
return "Very Unhealthy"
else:
return "Hazardous"
async def get_current(self, user: str = "default") -> EnvironmentResponse:
"""
Get current environment data for a user.
Fetches weather, forecast, sun times, and air quality from
the user's volatile collection.
Args:
user: User identifier (default: 'default')
Returns:
EnvironmentResponse with all available data
"""
logger.info(f"Fetching environment data for user: {user}")
# Get raw data from Qdrant
raw_data = await self.qdrant.get_environment_data(user)
# Parse each data type
weather = self._parse_weather(raw_data.get("weather"))
forecast = self._parse_forecast(raw_data.get("forecast"))
sun_times = self._parse_sun_times(raw_data.get("sun_times"))
air_quality = self._parse_air_quality(raw_data.get("air_quality"))
return EnvironmentResponse(
weather=weather,
forecast=forecast,
sun_times=sun_times,
air_quality=air_quality,
updated_at=datetime.utcnow(),
user=user,
)
# Singleton instance
_environment_service: Optional[EnvironmentService] = None
def get_environment_service() -> EnvironmentService:
"""Get or create singleton environment service instance."""
global _environment_service
if _environment_service is None:
_environment_service = EnvironmentService()
return _environment_service
+5
View File
@@ -0,0 +1,5 @@
"""
News subdomain for Tools.
Provides news headlines from Qdrant volatile collection.
"""
+56
View File
@@ -0,0 +1,56 @@
"""
News data schemas for Tools domain.
Provides Pydantic models for news headlines retrieved from the Qdrant volatile collection.
"""
from datetime import datetime
from typing import Optional, List
from pydantic import Field
from src.shared.base import BaseSchema
class NewsHeadline(BaseSchema):
"""Single news headline."""
title: str = Field(
...,
description="Headline title"
)
description: Optional[str] = Field(
None,
description="Brief description or summary"
)
source: Optional[str] = Field(
None,
description="News source name"
)
url: Optional[str] = Field(
None,
description="Link to full article"
)
class NewsResponse(BaseSchema):
"""News headlines response."""
headlines: List[NewsHeadline] = Field(
default_factory=list,
description="List of news headlines"
)
category: Optional[str] = Field(
None,
description="News category (e.g., 'general', 'technology')"
)
sources: Optional[List[str]] = Field(
None,
description="List of source names"
)
updated_at: datetime = Field(
default_factory=datetime.utcnow,
description="Timestamp when data was fetched"
)
user: Optional[str] = Field(
None,
description="User identifier used for data lookup"
)
+113
View File
@@ -0,0 +1,113 @@
"""
News data service for Tools domain.
Fetches news headlines from the Qdrant volatile collection.
"""
from datetime import datetime
from typing import Optional, Dict, Any, List
from src.shared.logging import get_logger
from src.shared.clients.qdrant_client import get_qdrant_client
from src.domains.tools.news.schemas import (
NewsHeadline,
NewsResponse,
)
logger = get_logger(__name__)
class NewsService:
"""
Service for fetching news data from Qdrant volatile collection.
Retrieves news headlines for a specific user.
"""
def __init__(self):
"""Initialize news service with Qdrant client."""
self.qdrant = get_qdrant_client()
def _parse_headlines(self, raw_data: Any) -> List[NewsHeadline]:
"""
Parse raw news data into list of NewsHeadline schemas.
Handles various formats from different news sources.
"""
if not raw_data:
return []
try:
# Handle dict with nested headlines list
headlines_list = raw_data
if isinstance(raw_data, dict):
headlines_list = raw_data.get("headlines") or raw_data.get("articles") or []
if not isinstance(headlines_list, list):
return []
headlines = []
for item in headlines_list:
if isinstance(item, dict):
headlines.append(NewsHeadline(
title=item.get("title", ""),
description=item.get("description") or item.get("summary"),
source=item.get("source") or item.get("provider"),
url=item.get("url") or item.get("link"),
))
elif isinstance(item, str):
# Simple string headlines
headlines.append(NewsHeadline(title=item))
return headlines
except Exception as e:
logger.warning(f"Failed to parse news headlines: {e}")
return []
async def get_news(self, user: str = "default") -> NewsResponse:
"""
Get news headlines for a user.
Fetches news from the user's volatile collection.
Args:
user: User identifier (default: 'default')
Returns:
NewsResponse with headlines
"""
logger.info(f"Fetching news data for user: {user}")
# Get raw data from Qdrant
news_records = await self.qdrant.get_by_namespace(user, "news")
headlines = []
category = None
sources = None
if news_records:
raw_data = news_records[0].get("raw_data", {})
headlines = self._parse_headlines(raw_data)
if isinstance(raw_data, dict):
category = raw_data.get("category")
sources = raw_data.get("sources")
return NewsResponse(
headlines=headlines,
category=category,
sources=sources,
updated_at=datetime.utcnow(),
user=user,
)
# Singleton instance
_news_service: Optional[NewsService] = None
def get_news_service() -> NewsService:
"""Get or create singleton news service instance."""
global _news_service
if _news_service is None:
_news_service = NewsService()
return _news_service
+6
View File
@@ -0,0 +1,6 @@
"""System stats module for host system resource monitoring."""
from src.domains.tools.system.service import SystemStatsService
from src.domains.tools.system.schemas import SystemStatsResponse
__all__ = ["SystemStatsService", "SystemStatsResponse"]
+197
View File
@@ -0,0 +1,197 @@
"""
Pydantic schemas for system stats module
"""
from pydantic import Field
from typing import Optional, List
from datetime import datetime
from src.shared.base import BaseSchema
class CpuStats(BaseSchema):
"""CPU usage statistics"""
usage_percent: float = Field(
...,
description="CPU usage percentage (0-100)",
ge=0,
le=100
)
cores: int = Field(
...,
description="Number of CPU cores"
)
load_1m: Optional[float] = Field(
default=None,
description="1-minute load average"
)
load_5m: Optional[float] = Field(
default=None,
description="5-minute load average"
)
load_15m: Optional[float] = Field(
default=None,
description="15-minute load average"
)
class MemoryStats(BaseSchema):
"""Memory usage statistics"""
usage_percent: float = Field(
...,
description="Memory usage percentage (0-100)",
ge=0,
le=100
)
total_bytes: int = Field(
...,
description="Total memory in bytes"
)
used_bytes: int = Field(
...,
description="Used memory in bytes"
)
available_bytes: int = Field(
...,
description="Available memory in bytes"
)
class DiskStats(BaseSchema):
"""Disk usage statistics for a single mount point"""
mount_point: str = Field(
...,
description="Mount point path"
)
device: str = Field(
...,
description="Device name (e.g., /dev/sda1)"
)
fstype: str = Field(
...,
description="Filesystem type (e.g., ext4, xfs)"
)
usage_percent: float = Field(
...,
description="Disk usage percentage (0-100)",
ge=0,
le=100
)
total_bytes: int = Field(
...,
description="Total disk space in bytes"
)
used_bytes: int = Field(
...,
description="Used disk space in bytes"
)
free_bytes: int = Field(
...,
description="Free disk space in bytes"
)
class NetworkStats(BaseSchema):
"""Network I/O statistics"""
bytes_sent: int = Field(
...,
description="Total bytes sent"
)
bytes_recv: int = Field(
...,
description="Total bytes received"
)
bytes_total: int = Field(
...,
description="Total bytes (sent + received)"
)
class GpuStats(BaseSchema):
"""GPU/VRAM statistics (if available)"""
available: bool = Field(
...,
description="Whether GPU stats are available"
)
name: Optional[str] = Field(
default=None,
description="GPU name"
)
usage_percent: Optional[float] = Field(
default=None,
description="VRAM usage percentage (0-100)"
)
total_bytes: Optional[int] = Field(
default=None,
description="Total VRAM in bytes"
)
used_bytes: Optional[int] = Field(
default=None,
description="Used VRAM in bytes"
)
free_bytes: Optional[int] = Field(
default=None,
description="Free VRAM in bytes"
)
class SystemStatsResponse(BaseSchema):
"""Response model for system stats"""
cpu: CpuStats = Field(
...,
description="CPU statistics"
)
memory: MemoryStats = Field(
...,
description="Memory statistics"
)
disks: List[DiskStats] = Field(
...,
description="Disk statistics for all mounted filesystems"
)
network: NetworkStats = Field(
...,
description="Network I/O statistics"
)
gpu: GpuStats = Field(
...,
description="GPU/VRAM statistics"
)
hostname: str = Field(
...,
description="System hostname"
)
queried_at: datetime = Field(
...,
description="UTC timestamp when stats were collected"
)
+190
View File
@@ -0,0 +1,190 @@
"""
System stats service for collecting host system metrics
"""
import subprocess
import socket
from datetime import datetime, timezone
import psutil
from src.shared.logging import get_logger
from typing import List
from src.domains.tools.system.schemas import (
SystemStatsResponse,
CpuStats,
MemoryStats,
DiskStats,
NetworkStats,
GpuStats,
)
# Filesystem types to exclude (virtual/system filesystems)
EXCLUDED_FSTYPES = {
"tmpfs", "devtmpfs", "devfs", "squashfs", "overlay",
"aufs", "proc", "sysfs", "cgroup", "cgroup2",
"debugfs", "tracefs", "securityfs", "pstore",
"hugetlbfs", "mqueue", "binfmt_misc", "autofs",
"fuse.lxcfs", "nsfs", "efivarfs",
}
logger = get_logger(__name__)
class SystemStatsService:
"""Service for collecting host system statistics"""
async def get_stats(self) -> SystemStatsResponse:
"""
Collect current system statistics.
Returns:
SystemStatsResponse with CPU, memory, disks, network, and GPU stats
"""
cpu = self._get_cpu_stats()
memory = self._get_memory_stats()
disks = self._get_all_disk_stats()
network = self._get_network_stats()
gpu = self._get_gpu_stats()
return SystemStatsResponse(
cpu=cpu,
memory=memory,
disks=disks,
network=network,
gpu=gpu,
hostname=socket.gethostname(),
queried_at=datetime.now(timezone.utc),
)
def _get_cpu_stats(self) -> CpuStats:
"""Get CPU usage statistics"""
# Get CPU percentage (blocking call with interval for accuracy)
cpu_percent = psutil.cpu_percent(interval=0.1)
cpu_count = psutil.cpu_count()
# Get load averages (Unix only)
try:
load_1, load_5, load_15 = psutil.getloadavg()
except (AttributeError, OSError):
load_1 = load_5 = load_15 = None
return CpuStats(
usage_percent=cpu_percent,
cores=cpu_count or 1,
load_1m=load_1,
load_5m=load_5,
load_15m=load_15,
)
def _get_memory_stats(self) -> MemoryStats:
"""Get memory usage statistics"""
mem = psutil.virtual_memory()
return MemoryStats(
usage_percent=mem.percent,
total_bytes=mem.total,
used_bytes=mem.used,
available_bytes=mem.available,
)
def _get_all_disk_stats(self) -> List[DiskStats]:
"""Get disk usage statistics for all mounted real filesystems"""
disks = []
seen_devices = set()
for partition in psutil.disk_partitions(all=False):
# Skip excluded filesystem types
if partition.fstype.lower() in EXCLUDED_FSTYPES:
continue
# Skip duplicate devices (same device mounted multiple times)
if partition.device in seen_devices:
continue
seen_devices.add(partition.device)
# Skip Docker/container overlays
if partition.mountpoint.startswith("/var/lib/docker"):
continue
try:
usage = psutil.disk_usage(partition.mountpoint)
disks.append(DiskStats(
mount_point=partition.mountpoint,
device=partition.device,
fstype=partition.fstype,
usage_percent=usage.percent,
total_bytes=usage.total,
used_bytes=usage.used,
free_bytes=usage.free,
))
except (PermissionError, OSError) as e:
logger.debug(f"Skipping {partition.mountpoint}: {e}")
continue
# Sort by mount point for consistent ordering
disks.sort(key=lambda d: d.mount_point)
return disks
def _get_network_stats(self) -> NetworkStats:
"""Get network I/O statistics"""
net_io = psutil.net_io_counters()
return NetworkStats(
bytes_sent=net_io.bytes_sent,
bytes_recv=net_io.bytes_recv,
bytes_total=net_io.bytes_sent + net_io.bytes_recv,
)
def _get_gpu_stats(self) -> GpuStats:
"""Get GPU/VRAM statistics using nvidia-smi"""
try:
# Query nvidia-smi for GPU memory info
result = subprocess.run(
[
"nvidia-smi",
"--query-gpu=name,memory.total,memory.used,memory.free",
"--format=csv,noheader,nounits",
],
capture_output=True,
text=True,
timeout=5,
)
if result.returncode != 0:
logger.debug("nvidia-smi not available or failed")
return GpuStats(available=False)
# Parse output: "NVIDIA GeForce RTX 3080, 10240, 2048, 8192"
line = result.stdout.strip().split("\n")[0] # First GPU
parts = [p.strip() for p in line.split(",")]
if len(parts) >= 4:
name = parts[0]
total_mb = int(parts[1])
used_mb = int(parts[2])
free_mb = int(parts[3])
total_bytes = total_mb * 1024 * 1024
used_bytes = used_mb * 1024 * 1024
free_bytes = free_mb * 1024 * 1024
usage_percent = (used_mb / total_mb * 100) if total_mb > 0 else 0
return GpuStats(
available=True,
name=name,
usage_percent=round(usage_percent, 1),
total_bytes=total_bytes,
used_bytes=used_bytes,
free_bytes=free_bytes,
)
except FileNotFoundError:
logger.debug("nvidia-smi not found - no NVIDIA GPU available")
except subprocess.TimeoutExpired:
logger.warning("nvidia-smi timed out")
except Exception as e:
logger.warning(f"Failed to get GPU stats: {e}")
return GpuStats(available=False)
+7 -12
View File
@@ -10,7 +10,6 @@ from src.shared.config import get_settings
from src.shared.logging import setup_logging, get_logger
from src.shared.database import get_database
from src.shared.security import initialize_oidc
from src.models.ollama_client import get_ollama_client, close_ollama_client
# Import domain controllers
from src.domains.health import health_controller
@@ -20,6 +19,9 @@ from src.domains.infrastructure import infrastructure_controller
from src.domains.housekeeping import housekeeping_controller
from src.domains.static import static_controller
from src.domains.dashboard import dashboard_controller
from src.domains.book import book_controller
from src.domains.scribe import scribe_controller
from src.domains.tenants import tenants_controller
# Initialize settings
settings = get_settings()
@@ -42,17 +44,8 @@ async def lifespan(app: FastAPI):
logger.info(f"Starting {settings.app_name} v{settings.app_version}")
logger.info(f"Debug mode: {settings.debug}")
logger.info(f"Log level: {settings.log_level}")
logger.info(f"Ollama URL: {settings.ollama_base_url}")
logger.info("=" * 60)
# Check Ollama connectivity
ollama_client = get_ollama_client()
ollama_healthy = await ollama_client.health_check()
if ollama_healthy:
logger.info("Ollama connection successful")
else:
logger.warning("Ollama connection failed - AI features may not work")
# Check database connectivity
database = get_database()
db_healthy = await database.health_check()
@@ -68,7 +61,6 @@ async def lifespan(app: FastAPI):
# Shutdown
logger.info("Shutting down application")
await close_ollama_client()
await database.close()
@@ -94,7 +86,7 @@ See `/docs` for the full API reference.
lifespan=lifespan,
debug=settings.debug,
swagger_ui_init_oauth={
"clientId": settings.oidc_audience,
"clientId": settings.oidc_audiences[0] if settings.oidc_audiences else "core-api",
"usePkceWithAuthorizationCodeGrant": True,
} if settings.oidc_enabled else None
)
@@ -117,6 +109,9 @@ app.include_router(infrastructure_controller.router) # /infrastructure
app.include_router(housekeeping_controller.router) # /housekeeping/*
app.include_router(static_controller.router) # /static/*
app.include_router(dashboard_controller.router) # /dashboard/*
app.include_router(book_controller.router) # /book/*
app.include_router(scribe_controller.router) # /scribe/*
app.include_router(tenants_controller.router) # /tenants/*
# Global exception handler
-128
View File
@@ -1,128 +0,0 @@
"""
Embedding model client for text vectorization
Uses sentence-transformers for generating embeddings.
"""
import logging
from typing import List, Optional
from sentence_transformers import SentenceTransformer
from src.config import get_settings
logger = logging.getLogger(__name__)
settings = get_settings()
class EmbeddingClient:
"""Client for generating text embeddings"""
def __init__(self, model_name: Optional[str] = None):
"""
Initialize embedding client
Args:
model_name: Optional model name, defaults to config
"""
self.model_name = model_name or settings.embedding_model
self.dimension = settings.embedding_dimension
self._model: Optional[SentenceTransformer] = None
logger.info(f"Initializing EmbeddingClient with model: {self.model_name}")
def _load_model(self) -> SentenceTransformer:
"""
Lazy load the embedding model
Returns:
Loaded SentenceTransformer model
"""
if self._model is None:
logger.info(f"Loading embedding model: {self.model_name}")
self._model = SentenceTransformer(self.model_name)
logger.info(f"Model loaded successfully. Embedding dimension: {self.dimension}")
return self._model
def embed_text(self, text: str) -> List[float]:
"""
Generate embedding for a single text
Args:
text: Input text to embed
Returns:
List of floats representing the embedding vector
"""
model = self._load_model()
embedding = model.encode(text, convert_to_numpy=True)
return embedding.tolist()
def embed_batch(self, texts: List[str]) -> List[List[float]]:
"""
Generate embeddings for multiple texts
Args:
texts: List of input texts
Returns:
List of embedding vectors
"""
model = self._load_model()
embeddings = model.encode(
texts,
batch_size=settings.embedding_batch_size,
convert_to_numpy=True,
show_progress_bar=False
)
return embeddings.tolist()
def get_dimension(self) -> int:
"""
Get embedding dimension
Returns:
Embedding vector dimension
"""
return self.dimension
# Global instance
_embedding_client: Optional[EmbeddingClient] = None
def get_embedding_client() -> EmbeddingClient:
"""
Get or create global embedding client instance
Returns:
EmbeddingClient instance
"""
global _embedding_client
if _embedding_client is None:
_embedding_client = EmbeddingClient()
return _embedding_client
async def embed_text_async(text: str) -> List[float]:
"""
Async wrapper for embedding text
Args:
text: Input text
Returns:
Embedding vector
"""
client = get_embedding_client()
return client.embed_text(text)
async def embed_batch_async(texts: List[str]) -> List[List[float]]:
"""
Async wrapper for batch embedding
Args:
texts: List of input texts
Returns:
List of embedding vectors
"""
client = get_embedding_client()
return client.embed_batch(texts)
-136
View File
@@ -1,136 +0,0 @@
"""
Ollama-based embedding client for text vectorization
Uses Ollama's embedding API instead of local sentence-transformers.
This eliminates the need for PyTorch and heavy ML dependencies.
"""
import logging
import httpx
from typing import List, Optional
from src.config import get_settings
logger = logging.getLogger(__name__)
settings = get_settings()
class OllamaEmbeddingClient:
"""Client for generating text embeddings using Ollama"""
def __init__(
self,
model_name: Optional[str] = None,
base_url: Optional[str] = None,
timeout: int = 30
):
"""
Initialize Ollama embedding client
Args:
model_name: Embedding model name (default: nomic-embed-text)
base_url: Ollama base URL (default from settings)
timeout: Request timeout in seconds
"""
self.model_name = model_name or settings.embedding_model
self.base_url = (base_url or settings.ollama_base_url).rstrip("/")
self.timeout = timeout
self.dimension = settings.embedding_dimension
logger.info(f"Initializing OllamaEmbeddingClient with model: {self.model_name}")
logger.info(f"Ollama URL: {self.base_url}")
async def embed_text(self, text: str) -> List[float]:
"""
Generate embedding for a single text using Ollama
Args:
text: Input text to embed
Returns:
List of floats representing the embedding vector
"""
try:
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.post(
f"{self.base_url}/api/embeddings",
json={
"model": self.model_name,
"prompt": text
}
)
response.raise_for_status()
result = response.json()
return result["embedding"]
except Exception as e:
logger.error(f"Error generating embedding via Ollama: {e}")
raise
async def embed_batch(self, texts: List[str]) -> List[List[float]]:
"""
Generate embeddings for multiple texts
Args:
texts: List of input texts
Returns:
List of embedding vectors
"""
embeddings = []
for text in texts:
embedding = await self.embed_text(text)
embeddings.append(embedding)
return embeddings
def get_dimension(self) -> int:
"""
Get embedding dimension
Returns:
Embedding vector dimension
"""
return self.dimension
# Global instance
_embedding_client: Optional[OllamaEmbeddingClient] = None
def get_embedding_client() -> OllamaEmbeddingClient:
"""
Get or create global Ollama embedding client instance
Returns:
OllamaEmbeddingClient instance
"""
global _embedding_client
if _embedding_client is None:
_embedding_client = OllamaEmbeddingClient()
return _embedding_client
async def embed_text_async(text: str) -> List[float]:
"""
Async wrapper for embedding text
Args:
text: Input text
Returns:
Embedding vector
"""
client = get_embedding_client()
return await client.embed_text(text)
async def embed_batch_async(texts: List[str]) -> List[List[float]]:
"""
Async wrapper for batch embedding
Args:
texts: List of input texts
Returns:
List of embedding vectors
"""
client = get_embedding_client()
return await client.embed_batch(texts)
-223
View File
@@ -1,223 +0,0 @@
"""
Ollama client for model inference.
Handles both streaming and non-streaming requests.
"""
import httpx
import json
import logging
from typing import AsyncIterator, Dict, Any, Optional
from src.config import get_settings
logger = logging.getLogger(__name__)
settings = get_settings()
class OllamaClient:
"""Client for interacting with Ollama API."""
def __init__(self):
self.base_url = settings.ollama_base_url
self.timeout = settings.ollama_timeout
self.client = httpx.AsyncClient(timeout=self.timeout)
logger.info(f"Initialized Ollama client: {self.base_url}")
async def close(self):
"""Close the HTTP client."""
await self.client.aclose()
def resolve_model(self, model_name: str) -> str:
"""
Resolve model alias to actual Ollama model.
Args:
model_name: Requested model name (e.g., "gpt-3.5-turbo")
Returns:
Actual Ollama model name (e.g., "gemma:7b")
"""
resolved = settings.model_aliases.get(model_name, model_name)
if resolved != model_name:
logger.info(f"Model resolution: {model_name} → {resolved}")
return resolved
async def generate_non_streaming(
self,
model: str,
prompt: str,
temperature: float = 0.7,
max_tokens: Optional[int] = None
) -> Dict[str, Any]:
"""
Generate non-streaming response from Ollama using chat endpoint.
Args:
model: Model name
prompt: User prompt
temperature: Sampling temperature
max_tokens: Maximum tokens to generate
Returns:
Dict with 'response' and 'tokens' keys
"""
actual_model = self.resolve_model(model)
payload = {
"model": actual_model,
"messages": [
{"role": "user", "content": prompt}
],
"stream": False,
"options": {
"temperature": temperature,
}
}
if max_tokens:
payload["options"]["num_predict"] = max_tokens
logger.debug(f"Ollama request to {actual_model}")
try:
response = await self.client.post(
f"{self.base_url}/api/chat",
json=payload
)
response.raise_for_status()
result = response.json()
return {
"response": result.get("message", {}).get("content", ""),
"tokens": {
"prompt": result.get("prompt_eval_count", 0),
"completion": result.get("eval_count", 0),
"total": result.get("prompt_eval_count", 0) + result.get("eval_count", 0)
}
}
except httpx.HTTPError as e:
logger.error(f"Ollama request failed: {e}")
raise
async def generate_streaming(
self,
model: str,
prompt: str,
temperature: float = 0.7,
max_tokens: Optional[int] = None
) -> AsyncIterator[str]:
"""
Generate streaming response from Ollama using chat endpoint.
Args:
model: Model name
prompt: User prompt
temperature: Sampling temperature
max_tokens: Maximum tokens to generate
Yields:
Token strings
"""
actual_model = self.resolve_model(model)
payload = {
"model": actual_model,
"messages": [
{"role": "user", "content": prompt}
],
"stream": True,
"options": {
"temperature": temperature,
}
}
if max_tokens:
payload["options"]["num_predict"] = max_tokens
logger.debug(f"Ollama streaming request to {actual_model}")
try:
async with self.client.stream(
"POST",
f"{self.base_url}/api/chat",
json=payload
) as response:
response.raise_for_status()
async for line in response.aiter_lines():
if not line:
continue
try:
chunk = json.loads(line)
if "message" in chunk:
content = chunk["message"].get("content", "")
if content:
yield content
# Check if done
if chunk.get("done", False):
break
except json.JSONDecodeError:
logger.warning(f"Failed to parse JSON: {line}")
continue
except httpx.HTTPError as e:
logger.error(f"Ollama streaming request failed: {e}")
raise
async def health_check(self) -> bool:
"""
Check if Ollama is healthy.
Returns:
True if healthy, False otherwise
"""
try:
response = await self.client.get(
f"{self.base_url}/api/tags",
timeout=5.0
)
return response.status_code == 200
except Exception as e:
logger.error(f"Ollama health check failed: {e}")
return False
async def list_models(self) -> Dict[str, Any]:
"""
List all available models in Ollama.
Returns:
Dict with 'models' key containing list of model info
"""
try:
response = await self.client.get(
f"{self.base_url}/api/tags",
timeout=5.0
)
response.raise_for_status()
return response.json()
except Exception as e:
logger.error(f"Failed to list Ollama models: {e}")
raise
# Global client instance
_ollama_client: Optional[OllamaClient] = None
def get_ollama_client() -> OllamaClient:
"""Get or create the global Ollama client instance."""
global _ollama_client
if _ollama_client is None:
_ollama_client = OllamaClient()
return _ollama_client
async def close_ollama_client():
"""Close the global Ollama client."""
global _ollama_client
if _ollama_client is not None:
await _ollama_client.close()
_ollama_client = None
-32
View File
@@ -1,32 +0,0 @@
"""
Security initialization module
Handles OIDC configuration and authentication setup
"""
from src.config import Settings
from src.auth.oidc import oidc_config
from src.logging_config import get_logger
logger = get_logger(__name__)
def initialize_oidc(settings: Settings) -> None:
"""
Initialize OIDC authentication configuration
Configures the global oidc_config instance with settings from environment.
If OIDC is enabled, logs the issuer URL for verification.
Args:
settings: Application settings containing OIDC configuration
"""
oidc_config.configure(
enabled=settings.oidc_enabled,
issuer=settings.oidc_issuer,
audience=settings.oidc_audience
)
if settings.oidc_enabled:
logger.info(f"✓ OIDC authentication enabled (issuer: {settings.oidc_issuer})")
else:
logger.info("○ OIDC authentication disabled - API is publicly accessible")
+3
View File
@@ -7,6 +7,7 @@ from src.shared.clients.portainer_client import PortainerClient, get_portainer_c
from src.shared.clients.npm_client import NPMClient, get_npm_client
from src.shared.clients.homeassistant_client import HomeAssistantClient, get_homeassistant_client
from src.shared.clients.authentik_client import AuthentikClient, get_authentik_client
from src.shared.clients.qdrant_client import QdrantReadClient, get_qdrant_client
__all__ = [
"PortainerClient",
@@ -17,4 +18,6 @@ __all__ = [
"get_homeassistant_client",
"AuthentikClient",
"get_authentik_client",
"QdrantReadClient",
"get_qdrant_client",
]
+249
View File
@@ -0,0 +1,249 @@
"""
Qdrant Vector Database Client (Read-Only)
Provides read-only access to Qdrant collections for querying volatile data.
Used to fetch weather, forecast, and sun times from the volatile_{user} collection.
"""
import time
from typing import List, Dict, Any, Optional
from qdrant_client import QdrantClient
from qdrant_client.models import Filter, FieldCondition, MatchValue, Range
from src.shared.logging import get_logger
from src.shared.config import get_settings
logger = get_logger(__name__)
settings = get_settings()
class QdrantReadClient:
"""
Read-only Qdrant client for accessing volatile data.
Connects to Qdrant and provides methods to query collections
with filtering by namespace and TTL expiry.
"""
VOLATILE_COLLECTION_PREFIX = "volatile_"
def __init__(
self,
host: Optional[str] = None,
port: Optional[int] = None,
):
"""
Initialize Qdrant read client.
Args:
host: Qdrant server host (default from settings)
port: Qdrant server port (default from settings)
"""
self.host = host or settings.qdrant_host
self.port = port or settings.qdrant_port
self._client: Optional[QdrantClient] = None
logger.info(f"Initialized QdrantReadClient: {self.host}:{self.port}")
@property
def client(self) -> QdrantClient:
"""Lazy-load Qdrant client connection."""
if self._client is None:
self._client = QdrantClient(
host=self.host,
port=self.port,
)
return self._client
def _get_volatile_collection(self, user: str) -> str:
"""Get volatile collection name for user."""
return f"{self.VOLATILE_COLLECTION_PREFIX}{user}"
def _current_timestamp_ms(self) -> int:
"""Get current timestamp in milliseconds."""
return int(time.time() * 1000)
async def collection_exists(self, collection_name: str) -> bool:
"""
Check if a collection exists.
Args:
collection_name: Name of collection to check
Returns:
True if collection exists
"""
try:
collections = self.client.get_collections()
existing = [c.name for c in collections.collections]
return collection_name in existing
except Exception as e:
logger.error(f"Error checking collection existence: {e}")
return False
async def get_by_namespace(
self,
user: str,
namespace: str,
include_expired: bool = False
) -> List[Dict[str, Any]]:
"""
Get all records for a specific namespace from user's volatile collection.
Args:
user: User identifier (e.g., 'jpmschweitzer' or 'default')
namespace: Namespace to filter (e.g., 'weather', 'forecast', 'sun')
include_expired: Whether to include expired records (default False)
Returns:
List of records with payload data
"""
collection_name = self._get_volatile_collection(user)
if not await self.collection_exists(collection_name):
logger.debug(f"Collection {collection_name} does not exist")
return []
# Build filter conditions
conditions = [
FieldCondition(
key="namespace",
match=MatchValue(value=namespace)
)
]
# Add TTL expiry filter unless including expired
if not include_expired:
now_ms = self._current_timestamp_ms()
conditions.append(
FieldCondition(
key="ttl_expiry",
range=Range(gt=now_ms)
)
)
query_filter = Filter(must=conditions)
try:
# Scroll through matching records
points, _ = self.client.scroll(
collection_name=collection_name,
scroll_filter=query_filter,
limit=100,
with_payload=True,
with_vectors=False
)
results = []
for point in points:
payload = dict(point.payload) if point.payload else {}
results.append({
"id": str(point.id),
"namespace": payload.get("namespace"),
"key": payload.get("key"),
"raw_data": payload.get("raw_data", {}),
"source": payload.get("source"),
"ttl_expiry": payload.get("ttl_expiry"),
"updated_at": payload.get("updated_at"),
})
logger.debug(
f"Found {len(results)} records in {collection_name}/{namespace}"
)
return results
except Exception as e:
logger.error(f"Error fetching from {collection_name}/{namespace}: {e}")
return []
async def get_environment_data(
self,
user: str
) -> Dict[str, Any]:
"""
Get all environment data (weather, forecast, sun times) for a user.
Convenience method that fetches all environment-related namespaces
in a single call.
Args:
user: User identifier
Returns:
Dict with 'weather', 'forecast', 'sun_times', 'air_quality' keys
(each may be None if no data found)
"""
result = {
"weather": None,
"forecast": None,
"sun_times": None,
"air_quality": None,
}
# Fetch weather data
weather_records = await self.get_by_namespace(user, "weather")
if weather_records:
# Get the first/most recent weather record
result["weather"] = weather_records[0].get("raw_data")
# Check if air quality is embedded in weather data
if result["weather"]:
aqi = result["weather"].get("aqi") or result["weather"].get("air_quality")
if aqi:
result["air_quality"] = aqi if isinstance(aqi, dict) else {"aqi": aqi}
# Fetch forecast data - pass raw_data to service for parsing
forecast_records = await self.get_by_namespace(user, "forecast")
if forecast_records:
result["forecast"] = forecast_records[0].get("raw_data")
# Fetch sun times data
sun_records = await self.get_by_namespace(user, "sun")
if sun_records:
result["sun_times"] = sun_records[0].get("raw_data")
# Check for separate air quality namespace if not embedded
if result["air_quality"] is None:
aq_records = await self.get_by_namespace(user, "air_quality")
if aq_records:
result["air_quality"] = aq_records[0].get("raw_data")
return result
async def health_check(self) -> Dict[str, Any]:
"""
Check Qdrant connectivity.
Returns:
Dict with connection status and info
"""
try:
collections = self.client.get_collections()
volatile_collections = [
c.name for c in collections.collections
if c.name.startswith(self.VOLATILE_COLLECTION_PREFIX)
]
return {
"status": "healthy",
"connected": True,
"host": f"{self.host}:{self.port}",
"volatile_collections": volatile_collections,
}
except Exception as e:
logger.error(f"Qdrant health check failed: {e}")
return {
"status": "unhealthy",
"connected": False,
"host": f"{self.host}:{self.port}",
"error": str(e),
}
# Singleton instance for reuse
_qdrant_client: Optional[QdrantReadClient] = None
def get_qdrant_client() -> QdrantReadClient:
"""Get or create singleton Qdrant client instance."""
global _qdrant_client
if _qdrant_client is None:
_qdrant_client = QdrantReadClient()
return _qdrant_client
+24 -46
View File
@@ -36,8 +36,15 @@ class Settings(BaseSettings):
host: str = "0.0.0.0"
port: int = 8083
# CORS
cors_origins: list[str] = ["*"]
# CORS - Note: When cors_credentials is True, cannot use "*" for origins
# Set CORS_ORIGINS env var to override (comma-separated list)
cors_origins: list[str] = [
"https://home.schweitz.net",
"https://tatlock.schweitz.net",
"http://localhost:8080",
"http://localhost:3000",
"http://127.0.0.1:8080",
]
cors_credentials: bool = True
cors_methods: list[str] = ["*"]
cors_headers: list[str] = ["*"]
@@ -45,31 +52,6 @@ class Settings(BaseSettings):
# Logging
log_level: str = "DEBUG"
# Ollama Configuration (for AI orchestration)
ollama_base_url: str # Required - set OLLAMA_BASE_URL in .env
ollama_timeout: int = 300 # 5 minutes
# Model Configuration
default_model: str = "mistral-nemo-large:latest"
agent_model: str = "mistral-nemo-large:latest"
code_models: str = "mistral-nemo-large:latest"
# System Prompt Variant (for A/B testing)
system_prompt_variant: str = "v8_holistic"
# Agent Configuration
agent_fallback_enabled: bool = True
# Model Aliases (OpenAI → Local)
alias_gpt35: str = "gemma:7b"
alias_gpt4: str = "mistral:7b"
alias_gpt4_turbo: str = "mixtral:8x7b"
alias_gpt4_code: str = "codestral:latest"
# Memory Configuration
memory_tier1_max_turns: int = 10
memory_consolidation_threshold: int = 10
# Qdrant Configuration
qdrant_host: str = "qdrant"
qdrant_port: int = 6333
@@ -77,11 +59,6 @@ class Settings(BaseSettings):
qdrant_collection_documents: str = "core_api_documents"
qdrant_collection_user_facts: str = "core_api_user_facts"
# Embeddings (using Ollama)
embedding_model: str = "nomic-embed-text"
embedding_dimension: int = 768
embedding_batch_size: int = 32
# Search Configuration
search_provider: str = "searxng"
searxng_url: str # Required - set SEARXNG_URL in .env
@@ -113,27 +90,28 @@ class Settings(BaseSettings):
# OIDC Authentication (Authentik)
oidc_enabled: bool = False
oidc_issuer: str = "https://auth.schweitz.net/application/o/core-api/"
oidc_audience: str = "core-api"
# Accept tokens from multiple OAuth providers (each has its own issuer/JWKS)
oidc_issuers: list[str] = [
"https://auth.schweitz.net/application/o/core-api/",
"https://auth.schweitz.net/application/o/tatlock-ui/",
"https://auth.schweitz.net/application/o/tatlock/",
]
# Accept tokens from multiple clients
oidc_audiences: list[str] = ["core-api", "tatlock-ui", "tatlock"]
# Authentik API (for token validation and user management)
authentik_url: str = "https://auth.schweitz.net"
authentik_username: str = ""
authentik_password: str = ""
@property
def model_aliases(self) -> dict:
"""Computed property for model aliases"""
return {
"gpt-3.5-turbo": self.alias_gpt35,
"gpt-4": self.alias_gpt4,
"gpt-4-turbo": self.alias_gpt4_turbo,
"gpt-4-code": self.alias_gpt4_code,
}
# Caller auth — static service keys (T-1). Empty by default so an unset
# key can never match an empty Bearer token (see src.domains.auth.callers).
office_service_key: str = ""
tatlock_service_key: str = ""
scheduler_service_key: str = ""
def get_code_models(self) -> list[str]:
"""Parse comma-separated code models"""
return [m.strip().strip('"').strip("'") for m in self.code_models.split(",") if m.strip()]
# The Book (T-1) — local time for due/remind/recurrence handling.
book_timezone: str = "Europe/Amsterdam"
class Config:
env_file = ".env"
+13 -5
View File
@@ -17,15 +17,23 @@ def initialize_oidc(settings: Settings) -> None:
settings: Application settings containing OIDC configuration
"""
# Import here to avoid circular imports
from src.auth.oidc import oidc_config
# Configure BOTH oidc modules (src.auth and src.domains.auth)
from src.auth.oidc import oidc_config as auth_oidc_config
from src.domains.auth.oidc import oidc_config as domains_oidc_config
oidc_config.configure(
auth_oidc_config.configure(
enabled=settings.oidc_enabled,
issuer=settings.oidc_issuer,
audience=settings.oidc_audience
issuers=settings.oidc_issuers,
audiences=settings.oidc_audiences
)
domains_oidc_config.configure(
enabled=settings.oidc_enabled,
issuers=settings.oidc_issuers,
audiences=settings.oidc_audiences
)
if settings.oidc_enabled:
logger.info(f"OIDC authentication enabled (issuer: {settings.oidc_issuer})")
logger.info(f"OIDC authentication enabled (issuers: {settings.oidc_issuers})")
else:
logger.info("OIDC authentication disabled - API is publicly accessible")
+371
View File
@@ -0,0 +1,371 @@
"""
Shared test fixtures.
Database-backed tests need `TEST_DATABASE_URL` (core-api's gitignored .env).
They **skip** when it is unset and **refuse** (hard failure, not a skip)
when it points at `core_api` — that is production, and the dev `.env`'s
other `POSTGRES_*` values point there too, so this is the one guard that
stands between a test run and the production database.
Running the actual migration against the test database is done through the
`alembic` CLI in a subprocess rather than in-process, because
`alembic/env.py` always builds its connection URL from `get_settings()`
(composed from `POSTGRES_*`), with no override hook of its own — and
`get_settings()` is `lru_cache`d, so by the time a test runs, other test
modules have almost certainly already imported `src.main` and cached a
Settings instance pointed at production. A subprocess gets a clean
environment and a clean cache; overriding `POSTGRES_HOST`/`USER`/
`PASSWORD`/`DATABASE` in that child's environment, derived from
`TEST_DATABASE_URL`, is what actually redirects the migration.
"""
import asyncio
import os
import subprocess
from pathlib import Path
from urllib.parse import urlsplit
import pytest
from dotenv import load_dotenv
REPO_ROOT = Path(__file__).resolve().parent.parent
ALEMBIC_BIN = REPO_ROOT / ".venv" / "bin" / "alembic"
load_dotenv(REPO_ROOT / ".env")
def _components(url: str) -> dict:
"""POSTGRES_* overrides for a subprocess, derived from a database URL."""
parts = urlsplit(url)
host = parts.hostname or ""
if parts.port:
host = f"{host}:{parts.port}"
return {
"POSTGRES_HOST": host,
"POSTGRES_USER": parts.username or "",
"POSTGRES_PASSWORD": parts.password or "",
"POSTGRES_DATABASE": parts.path.lstrip("/"),
}
def run_alembic(url: str, *args: str) -> subprocess.CompletedProcess:
"""Run `alembic <args>` against `url`, never against the repo's own .env database."""
env = os.environ.copy()
env.update(_components(url))
result = subprocess.run(
[str(ALEMBIC_BIN), *args],
cwd=str(REPO_ROOT),
env=env,
capture_output=True,
text=True,
)
return result
@pytest.fixture(scope="session")
def test_database_url() -> str:
"""The core_api_test database URL, or skip/refuse per the rules above."""
url = os.environ.get("TEST_DATABASE_URL")
if not url:
pytest.skip("TEST_DATABASE_URL is not set — database tests skipped")
db_name = urlsplit(url).path.lstrip("/")
if db_name == "core_api":
pytest.fail(
"TEST_DATABASE_URL resolves to 'core_api' (production) — refusing to use it for tests"
)
return url
def _grant_test_office_gemini(url: str) -> None:
"""
Grant a second, test-environment-only consumer, `office-gemini`, in
both seeded tenants: test_book.py and test_scribe.py have used
`office-gemini` as a stand-in second Office consumer since before
core-api T-8's grant check existed, to prove behaviour that depends on
two *distinct* consumers (address ownership across consumers,
`book_events.client` recording the actual caller) rather than on any
one consumer being granted. Migration 007 itself seeds only
`office-claude` — the only consumer actually configured in Authentik
today (T-8's dossier) — so this grant is inserted directly here,
test-only, rather than through the migration; production's seed stays
exactly what the dossier specifies. A consumer this never grants
(e.g. `office-ungranted` in a test) is what proves the
`consumer_not_granted` refusal — this must never widen to cover one of
those.
A plain function, not a fixture: called from each of `book_schema`,
`scribe_schema` and `tenant_registry` right after their own `run_alembic
upgrade head`, so the grant is restored whenever any one of them is the
first to re-establish the schema in a session — the same
self-healing-on-first-use property those fixtures already rely on to
survive a migration test downgrading the schema mid-suite. A session-
scoped fixture shared between them would cache the *string* across that
downgrade while the real schema moved on underneath it — which is what
a first version of this did, and `tests/test_scribe.py` failing with
"relation tenants does not exist" is what caught it.
"""
from sqlalchemy import text
from sqlalchemy.ext.asyncio import create_async_engine
from sqlalchemy.pool import NullPool
asyncpg_url = url.replace("postgresql://", "postgresql+asyncpg://", 1)
async def _grant() -> None:
engine = create_async_engine(asyncpg_url, poolclass=NullPool)
try:
async with engine.begin() as conn:
await conn.execute(
text(
"INSERT INTO tenant_consumers (tenant_id, consumer) VALUES "
"('jpmschweitzer', 'office-gemini'), ('llm_tester', 'office-gemini') "
"ON CONFLICT DO NOTHING"
)
)
finally:
await engine.dispose()
asyncio.run(_grant())
@pytest.fixture(scope="session")
def tenant_registry(test_database_url: str) -> str:
"""Ensure migration 007 (and everything before it) is applied to
core_api_test, with its seed tenants/grants plus the test-only
office-gemini grant (see `_grant_test_office_gemini`). Used directly by
tests/test_tenants.py; book_schema and scribe_schema do the same thing
independently rather than depending on this, so each keeps its own
self-healing-on-first-use property (see that function's docstring)."""
result = run_alembic(test_database_url, "upgrade", "head")
if result.returncode != 0:
pytest.fail(f"alembic upgrade head failed:\n{result.stdout}\n{result.stderr}")
_grant_test_office_gemini(test_database_url)
return test_database_url
@pytest.fixture(scope="session")
def book_schema(test_database_url: str) -> str:
"""Ensure migration 005 (and everything before it, now including the
tenant registry) is applied to core_api_test."""
result = run_alembic(test_database_url, "upgrade", "head")
if result.returncode != 0:
pytest.fail(f"alembic upgrade head failed:\n{result.stdout}\n{result.stderr}")
_grant_test_office_gemini(test_database_url)
return test_database_url
@pytest.fixture
async def book_session(book_schema: str):
"""An AsyncSession bound to core_api_test, rolled back after each test."""
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from sqlalchemy.pool import NullPool
asyncpg_url = book_schema.replace("postgresql://", "postgresql+asyncpg://", 1)
engine = create_async_engine(asyncpg_url, poolclass=NullPool)
session_factory = async_sessionmaker(bind=engine, expire_on_commit=False)
async with session_factory() as session:
yield session
await session.rollback()
await engine.dispose()
@pytest.fixture
async def clean_book_tables(book_session):
"""
Book service methods commit internally (same pattern as the Dashboard
service), so `book_session`'s post-test rollback never runs against
anything — the data is already durable. TRUNCATE before each test
instead, which the append-only trigger does not block: it is a
row-level BEFORE UPDATE OR DELETE trigger, and TRUNCATE fires neither.
"""
from sqlalchemy import text
await book_session.execute(
text(
"TRUNCATE book_events, book_attachments, book_record_links, book_tags, "
"book_records, book_tenant_counters"
)
)
await book_session.commit()
yield book_session
@pytest.fixture(scope="session")
def scribe_schema(test_database_url: str) -> str:
"""Ensure migration 006 (and everything before it, now including the
tenant registry) is applied to core_api_test."""
result = run_alembic(test_database_url, "upgrade", "head")
if result.returncode != 0:
pytest.fail(f"alembic upgrade head failed:\n{result.stdout}\n{result.stderr}")
_grant_test_office_gemini(test_database_url)
return test_database_url
@pytest.fixture
async def scribe_session(scribe_schema: str):
"""An AsyncSession bound to core_api_test — same per-test engine pattern
as book_session (see its docstring for why: TestClient drives the app
on a different event loop than pytest-asyncio's fixture setup)."""
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from sqlalchemy.pool import NullPool
asyncpg_url = scribe_schema.replace("postgresql://", "postgresql+asyncpg://", 1)
engine = create_async_engine(asyncpg_url, poolclass=NullPool)
session_factory = async_sessionmaker(bind=engine, expire_on_commit=False)
async with session_factory() as session:
yield session
await session.rollback()
await engine.dispose()
@pytest.fixture
async def clean_scribe_tables(scribe_session):
"""Scribe service methods commit internally (same reason as Book's
clean_book_tables). TRUNCATE all tables together so Postgres resolves
the FK order (mailbox_entries -> messages, messages' self-referencing
reply_to) across the one statement.
`tenant_consumers` joined the list for migration 008 (core-api T-11):
its new composite FK `(tenant_id, default_address)` ->
`scribe_addresses(owner_tenant, name)` means Postgres now refuses to
truncate `scribe_addresses` alone — "cannot truncate a table
referenced in a foreign key constraint" — a restriction on the
constraint's existence, not on whether any row's `default_address` is
actually non-null (it never is here; migration 008's data step is a
no-op in `core_api_test`). `TRUNCATE ... CASCADE` was rejected: it
would silently drop every grant rather than only addresses, so the
four grants this file's tests depend on — both seeded tenants x
office-claude/office-gemini, the same pairing `_grant_test_office_gemini`
establishes once per session — are reinserted immediately after."""
from sqlalchemy import text
await scribe_session.execute(
text(
"TRUNCATE scribe_mailbox_entries, scribe_messages, scribe_addresses, "
"tenant_consumers"
)
)
await scribe_session.execute(
text(
"INSERT INTO tenant_consumers (tenant_id, consumer) VALUES "
"('jpmschweitzer', 'office-claude'), ('llm_tester', 'office-claude'), "
"('jpmschweitzer', 'office-gemini'), ('llm_tester', 'office-gemini')"
)
)
await scribe_session.commit()
yield scribe_session
@pytest.fixture
async def suspended_tenant(test_database_url: str):
"""
A third tenant, inserted with `status='suspended'`, for proving the
`tenant_suspended` refusal (core-api T-8) without touching either
seeded tenant that the rest of the suite depends on being active.
Nothing else ever references this id, so inserting and deleting it
per test is safe — no FK points at it.
Depends on `test_database_url` and runs its own `alembic upgrade head`
rather than on `tenant_registry`/`book_schema`/`scribe_schema`: this
fixture is reused from test_book.py, test_scribe.py and test_tenants.py,
so whichever of those files runs first would permanently cache
*any* session-scoped schema fixture it depended on — and a migration
test downgrading the schema afterwards (test_book_migration.py,
test_scribe_migration.py) would leave that cache stale for the other
two files. Self-contained and function-scoped avoids it outright, at
the cost of a redundant `alembic upgrade head` call (cheap relative to
the many such calls the migration test files already make).
"""
from sqlalchemy import text
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from sqlalchemy.pool import NullPool
result = run_alembic(test_database_url, "upgrade", "head")
if result.returncode != 0:
pytest.fail(f"alembic upgrade head failed:\n{result.stdout}\n{result.stderr}")
tenant_id = "suspended-tenant"
asyncpg_url = test_database_url.replace("postgresql://", "postgresql+asyncpg://", 1)
engine = create_async_engine(asyncpg_url, poolclass=NullPool)
session_factory = async_sessionmaker(bind=engine, expire_on_commit=False)
async with session_factory() as session:
await session.execute(
text(
"INSERT INTO tenants (id, display_name, kind, status) "
"VALUES (:id, 'Suspended Test Tenant', 'test', 'suspended') "
"ON CONFLICT (id) DO UPDATE SET status = 'suspended'"
),
{"id": tenant_id},
)
await session.commit()
yield tenant_id
async with session_factory() as session:
await session.execute(text("DELETE FROM tenants WHERE id = :id"), {"id": tenant_id})
await session.commit()
await engine.dispose()
@pytest.fixture
async def default_address_grant(tenant_registry: str, test_database_url: str):
"""
Give `jpmschweitzer`/`office-claude` a default address (core-api T-11):
registers a throwaway `scribe_addresses` row first — migration 008's
composite FK means the grant's default must name a real address in the
same tenant — then points the grant at it. Reversed on teardown (default
back to null, the address row deleted) so the rest of the suite sees the
null default that `core_api_test`'s own copy of migration 008 actually
leaves this grant with (see test_tenants_migration.py's
TestMigration008DataStep: claude-ai is never seeded here).
"""
from sqlalchemy import text
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from sqlalchemy.pool import NullPool
tenant_id = "jpmschweitzer"
consumer = "office-claude"
address = "test-default-address"
asyncpg_url = test_database_url.replace("postgresql://", "postgresql+asyncpg://", 1)
engine = create_async_engine(asyncpg_url, poolclass=NullPool)
session_factory = async_sessionmaker(bind=engine, expire_on_commit=False)
async with session_factory() as session:
await session.execute(
text(
"INSERT INTO scribe_addresses (owner_tenant, name, created_by) "
"VALUES (:tenant, :address, 'test-harness')"
),
{"tenant": tenant_id, "address": address},
)
await session.execute(
text(
"UPDATE tenant_consumers SET default_address = :address "
"WHERE tenant_id = :tenant AND consumer = :consumer"
),
{"tenant": tenant_id, "address": address, "consumer": consumer},
)
await session.commit()
yield address
async with session_factory() as session:
await session.execute(
text(
"UPDATE tenant_consumers SET default_address = NULL "
"WHERE tenant_id = :tenant AND consumer = :consumer"
),
{"tenant": tenant_id, "consumer": consumer},
)
await session.execute(
text("DELETE FROM scribe_addresses WHERE owner_tenant = :tenant AND name = :address"),
{"tenant": tenant_id, "address": address},
)
await session.commit()
await engine.dispose()
File diff suppressed because it is too large Load Diff
+627
View File
@@ -0,0 +1,627 @@
"""Tests for authentication service."""
import uuid
from datetime import datetime, timezone
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from sqlalchemy.ext.asyncio import AsyncSession
from src.domains.auth.models import User, Role, Group, UserPreferences
from src.domains.auth.schemas import TokenInfoSchema, RoleSchema
from src.domains.auth.service import AuthService, get_auth_service
# =============================================================================
# Fixtures
# =============================================================================
@pytest.fixture
def mock_session():
"""Create a mock async database session."""
session = AsyncMock(spec=AsyncSession)
session.execute = AsyncMock()
session.commit = AsyncMock()
session.flush = AsyncMock()
session.refresh = AsyncMock()
session.add = MagicMock()
return session
@pytest.fixture
def auth_service(mock_session):
"""Create an AuthService instance with mock session."""
return AuthService(mock_session)
@pytest.fixture
def sample_user():
"""Create a sample user for testing."""
user = User(
id=uuid.uuid4(),
authentik_id=uuid.uuid4(),
email="test@example.com",
name="Test User",
avatar_url="https://example.com/avatar.jpg",
created_at=datetime.now(timezone.utc),
last_login=datetime.now(timezone.utc),
)
user.roles = []
user.preferences = None
return user
@pytest.fixture
def sample_role():
"""Create a sample role for testing."""
return Role(
id=uuid.uuid4(),
name="control-room.general:admin",
domain="control-room",
category="general",
action="admin",
)
@pytest.fixture
def sample_group(sample_role):
"""Create a sample group for testing."""
group = Group(
id=uuid.uuid4(),
authentik_id=uuid.uuid4(),
name="Administrators",
is_superuser=True,
parent_name=None,
member_count=5,
created_at=datetime.now(timezone.utc),
synced_at=datetime.now(timezone.utc),
)
group.roles = [sample_role]
return group
@pytest.fixture
def sample_token_info():
"""Create sample token info from Authentik."""
return TokenInfoSchema(
sub=str(uuid.uuid4()),
email="test@example.com",
name="Test User",
preferred_username="testuser",
groups=["Administrators", "Developers"],
picture="https://example.com/avatar.jpg",
)
@pytest.fixture
def sample_preferences():
"""Create sample user preferences."""
return UserPreferences(
user_id=uuid.uuid4(),
theme="dark",
default_room="control-room",
preferences_json={"notifications": True},
)
# =============================================================================
# AuthService Initialization Tests
# =============================================================================
class TestAuthServiceInit:
"""Test AuthService initialization."""
def test_init_with_session(self, mock_session):
"""AuthService should initialize with session."""
service = AuthService(mock_session)
assert service.session is mock_session
def test_init_sets_userinfo_url(self, mock_session):
"""AuthService should set userinfo URL from settings."""
service = AuthService(mock_session)
assert "userinfo" in service.userinfo_url
def test_get_auth_service_factory(self, mock_session):
"""get_auth_service should return AuthService instance."""
service = get_auth_service(mock_session)
assert isinstance(service, AuthService)
assert service.session is mock_session
# =============================================================================
# Token Validation Tests
# =============================================================================
class TestValidateToken:
"""Test token validation via Authentik userinfo endpoint."""
@pytest.mark.asyncio
async def test_validate_token_success(self, auth_service):
"""validate_token should return token info on success."""
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {
"sub": str(uuid.uuid4()),
"email": "test@example.com",
"name": "Test User",
"preferred_username": "testuser",
"groups": ["Administrators"],
"picture": "https://example.com/avatar.jpg",
}
mock_response.raise_for_status = MagicMock()
with patch("src.domains.auth.service.httpx.AsyncClient") as mock_client:
mock_client.return_value.__aenter__.return_value.get = AsyncMock(
return_value=mock_response
)
result = await auth_service.validate_token("valid_token")
assert result.email == "test@example.com"
assert result.name == "Test User"
assert "Administrators" in result.groups
@pytest.mark.asyncio
async def test_validate_token_invalid(self, auth_service):
"""validate_token should raise ValueError for invalid token."""
mock_response = MagicMock()
mock_response.status_code = 401
with patch("src.domains.auth.service.httpx.AsyncClient") as mock_client:
mock_client.return_value.__aenter__.return_value.get = AsyncMock(
return_value=mock_response
)
with pytest.raises(ValueError, match="Invalid or expired token"):
await auth_service.validate_token("invalid_token")
@pytest.mark.asyncio
async def test_validate_token_service_unavailable(self, auth_service):
"""validate_token should raise ValueError when service unavailable."""
import httpx
with patch("src.domains.auth.service.httpx.AsyncClient") as mock_client:
mock_client.return_value.__aenter__.return_value.get = AsyncMock(
side_effect=httpx.RequestError("Connection failed")
)
with pytest.raises(ValueError, match="Authentication service unavailable"):
await auth_service.validate_token("token")
# =============================================================================
# User Sync Tests
# =============================================================================
class TestSyncUser:
"""Test user synchronization from OIDC token."""
@pytest.mark.asyncio
async def test_sync_user_creates_new_user(self, auth_service, sample_token_info, mock_session):
"""sync_user should create new user when not found."""
# Mock no existing user found
mock_result = MagicMock()
mock_result.scalar_one_or_none.return_value = None
mock_session.execute.return_value = mock_result
user, is_new = await auth_service.sync_user(sample_token_info)
assert is_new is True
assert mock_session.add.call_count == 2 # User and Preferences
@pytest.mark.asyncio
async def test_sync_user_updates_existing_user(
self, auth_service, sample_token_info, sample_user, mock_session
):
"""sync_user should update existing user when found."""
# Mock existing user found
mock_result = MagicMock()
mock_result.scalar_one_or_none.return_value = sample_user
mock_session.execute.return_value = mock_result
# Update token info with matching authentik_id
sample_token_info.sub = str(sample_user.authentik_id)
user, is_new = await auth_service.sync_user(sample_token_info)
assert is_new is False
assert user.email == sample_token_info.email
assert user.name == sample_token_info.name
@pytest.mark.asyncio
async def test_sync_user_updates_last_login(
self, auth_service, sample_token_info, sample_user, mock_session
):
"""sync_user should update last_login timestamp."""
old_login = sample_user.last_login
mock_result = MagicMock()
mock_result.scalar_one_or_none.return_value = sample_user
mock_session.execute.return_value = mock_result
sample_token_info.sub = str(sample_user.authentik_id)
user, _ = await auth_service.sync_user(sample_token_info)
assert user.last_login is not None
# last_login should be updated (or same if happened in same second)
assert user.last_login >= old_login or user.last_login is not None
# =============================================================================
# Role Sync Tests
# =============================================================================
class TestSyncRoles:
"""Test role synchronization from Authentik groups via group_roles."""
@pytest.mark.asyncio
async def test_sync_roles_from_groups(
self, auth_service, sample_user, sample_group, mock_session
):
"""sync_roles should get roles from matching groups."""
# Mock finding groups with roles
mock_result = MagicMock()
mock_result.scalars.return_value.all.return_value = [sample_group]
mock_session.execute.return_value = mock_result
roles = await auth_service.sync_roles(sample_user, ["Administrators"])
assert len(roles) == 1
assert roles[0].name == "control-room.general:admin"
assert sample_user.roles == roles
@pytest.mark.asyncio
async def test_sync_roles_no_matching_groups(
self, auth_service, sample_user, mock_session
):
"""sync_roles should return empty list when no groups match."""
mock_result = MagicMock()
mock_result.scalars.return_value.all.return_value = []
mock_session.execute.return_value = mock_result
roles = await auth_service.sync_roles(sample_user, ["NonExistentGroup"])
assert len(roles) == 0
assert sample_user.roles == []
@pytest.mark.asyncio
async def test_sync_roles_deduplicates_roles(
self, auth_service, sample_user, sample_role, mock_session
):
"""sync_roles should deduplicate roles from multiple groups."""
# Create two groups with the same role
group1 = Group(
id=uuid.uuid4(),
authentik_id=uuid.uuid4(),
name="Group1",
is_superuser=False,
member_count=1,
created_at=datetime.now(timezone.utc),
synced_at=datetime.now(timezone.utc),
)
group1.roles = [sample_role]
group2 = Group(
id=uuid.uuid4(),
authentik_id=uuid.uuid4(),
name="Group2",
is_superuser=False,
member_count=1,
created_at=datetime.now(timezone.utc),
synced_at=datetime.now(timezone.utc),
)
group2.roles = [sample_role] # Same role
mock_result = MagicMock()
mock_result.scalars.return_value.all.return_value = [group1, group2]
mock_session.execute.return_value = mock_result
roles = await auth_service.sync_roles(sample_user, ["Group1", "Group2"])
# Should only have one role despite appearing in two groups
assert len(roles) == 1
# =============================================================================
# Schema Conversion Tests
# =============================================================================
class TestSchemaConversions:
"""Test model to schema conversions."""
def test_user_to_schema(self, auth_service, sample_user):
"""user_to_schema should convert User model to UserSchema."""
schema = auth_service.user_to_schema(sample_user)
assert schema.id == sample_user.id
assert schema.authentik_id == sample_user.authentik_id
assert schema.email == sample_user.email
assert schema.name == sample_user.name
assert schema.avatar_url == sample_user.avatar_url
def test_roles_to_schema(self, auth_service, sample_role):
"""roles_to_schema should convert Role models to RoleSchemas."""
schemas = auth_service.roles_to_schema([sample_role])
assert len(schemas) == 1
assert schemas[0].id == sample_role.id
assert schemas[0].name == sample_role.name
assert schemas[0].domain == sample_role.domain
assert schemas[0].category == sample_role.category
assert schemas[0].action == sample_role.action
def test_roles_to_schema_empty_list(self, auth_service):
"""roles_to_schema should handle empty list."""
schemas = auth_service.roles_to_schema([])
assert schemas == []
def test_preferences_to_schema(self, auth_service, sample_preferences):
"""preferences_to_schema should convert UserPreferences to schema."""
schema = auth_service.preferences_to_schema(sample_preferences)
assert schema.theme == sample_preferences.theme
assert schema.default_room == sample_preferences.default_room
assert schema.preferences_json == sample_preferences.preferences_json
def test_preferences_to_schema_none(self, auth_service):
"""preferences_to_schema should return defaults for None."""
schema = auth_service.preferences_to_schema(None)
assert schema.theme == "system"
assert schema.default_room == "front-hall"
assert schema.preferences_json == {}
# =============================================================================
# List Operations Tests
# =============================================================================
class TestListOperations:
"""Test list operations for users, groups, and roles."""
@pytest.mark.asyncio
async def test_list_users(self, auth_service, sample_user, mock_session):
"""list_users should return paginated user list."""
sample_user.roles = []
# Mock count query
count_result = MagicMock()
count_result.scalar.return_value = 1
# Mock users query
users_result = MagicMock()
users_result.scalars.return_value.all.return_value = [sample_user]
mock_session.execute.side_effect = [count_result, users_result]
items, total = await auth_service.list_users()
assert total == 1
assert len(items) == 1
assert items[0].email == sample_user.email
@pytest.mark.asyncio
async def test_list_users_with_search(self, auth_service, mock_session):
"""list_users should filter by search query."""
count_result = MagicMock()
count_result.scalar.return_value = 0
users_result = MagicMock()
users_result.scalars.return_value.all.return_value = []
mock_session.execute.side_effect = [count_result, users_result]
items, total = await auth_service.list_users(search="nonexistent")
assert total == 0
assert len(items) == 0
@pytest.mark.asyncio
async def test_list_groups(self, auth_service, sample_group, mock_session):
"""list_groups should return paginated group list with roles."""
count_result = MagicMock()
count_result.scalar.return_value = 1
groups_result = MagicMock()
groups_result.scalars.return_value.all.return_value = [sample_group]
mock_session.execute.side_effect = [count_result, groups_result]
items, total = await auth_service.list_groups()
assert total == 1
assert len(items) == 1
assert items[0].name == sample_group.name
assert len(items[0].roles) == 1 # Should include role names
@pytest.mark.asyncio
async def test_list_roles(self, auth_service, sample_role, mock_session):
"""list_roles should return all roles ordered by domain."""
mock_result = MagicMock()
mock_result.scalars.return_value.all.return_value = [sample_role]
mock_session.execute.return_value = mock_result
roles = await auth_service.list_roles()
assert len(roles) == 1
assert roles[0].name == sample_role.name
# =============================================================================
# Group-Role Management Tests
# =============================================================================
class TestGroupRoleManagement:
"""Test group-role assignment and removal."""
@pytest.mark.asyncio
async def test_get_group_by_id(self, auth_service, sample_group, mock_session):
"""get_group_by_id should return group with roles."""
mock_result = MagicMock()
mock_result.scalar_one_or_none.return_value = sample_group
mock_session.execute.return_value = mock_result
group = await auth_service.get_group_by_id(sample_group.id)
assert group is not None
assert group.id == sample_group.id
assert len(group.roles) == 1
@pytest.mark.asyncio
async def test_get_group_by_id_not_found(self, auth_service, mock_session):
"""get_group_by_id should return None when not found."""
mock_result = MagicMock()
mock_result.scalar_one_or_none.return_value = None
mock_session.execute.return_value = mock_result
group = await auth_service.get_group_by_id(uuid.uuid4())
assert group is None
@pytest.mark.asyncio
async def test_assign_role_to_group(
self, auth_service, sample_group, sample_role, mock_session
):
"""assign_role_to_group should add role to group."""
# Clear existing roles for this test
sample_group.roles = []
# Mock group lookup
group_result = MagicMock()
group_result.scalar_one_or_none.return_value = sample_group
# Mock role lookup
role_result = MagicMock()
role_result.scalar_one_or_none.return_value = sample_role
mock_session.execute.side_effect = [group_result, role_result]
group = await auth_service.assign_role_to_group(sample_group.id, sample_role.id)
assert sample_role in group.roles
mock_session.flush.assert_called()
@pytest.mark.asyncio
async def test_assign_role_to_group_already_assigned(
self, auth_service, sample_group, sample_role, mock_session
):
"""assign_role_to_group should not duplicate if already assigned."""
# Group already has this role
sample_group.roles = [sample_role]
original_count = len(sample_group.roles)
group_result = MagicMock()
group_result.scalar_one_or_none.return_value = sample_group
role_result = MagicMock()
role_result.scalar_one_or_none.return_value = sample_role
mock_session.execute.side_effect = [group_result, role_result]
group = await auth_service.assign_role_to_group(sample_group.id, sample_role.id)
assert len(group.roles) == original_count # No duplicate
@pytest.mark.asyncio
async def test_assign_role_to_group_group_not_found(self, auth_service, mock_session):
"""assign_role_to_group should raise ValueError when group not found."""
mock_result = MagicMock()
mock_result.scalar_one_or_none.return_value = None
mock_session.execute.return_value = mock_result
with pytest.raises(ValueError, match="Group not found"):
await auth_service.assign_role_to_group(uuid.uuid4(), uuid.uuid4())
@pytest.mark.asyncio
async def test_assign_role_to_group_role_not_found(
self, auth_service, sample_group, mock_session
):
"""assign_role_to_group should raise ValueError when role not found."""
sample_group.roles = []
group_result = MagicMock()
group_result.scalar_one_or_none.return_value = sample_group
role_result = MagicMock()
role_result.scalar_one_or_none.return_value = None
mock_session.execute.side_effect = [group_result, role_result]
with pytest.raises(ValueError, match="Role not found"):
await auth_service.assign_role_to_group(sample_group.id, uuid.uuid4())
@pytest.mark.asyncio
async def test_remove_role_from_group(
self, auth_service, sample_group, sample_role, mock_session
):
"""remove_role_from_group should remove role from group."""
# Group has this role
sample_group.roles = [sample_role]
group_result = MagicMock()
group_result.scalar_one_or_none.return_value = sample_group
role_result = MagicMock()
role_result.scalar_one_or_none.return_value = sample_role
mock_session.execute.side_effect = [group_result, role_result]
group = await auth_service.remove_role_from_group(sample_group.id, sample_role.id)
assert sample_role not in group.roles
mock_session.flush.assert_called()
@pytest.mark.asyncio
async def test_remove_role_from_group_not_assigned(
self, auth_service, sample_group, sample_role, mock_session
):
"""remove_role_from_group should handle role not assigned gracefully."""
# Group does not have this role
sample_group.roles = []
group_result = MagicMock()
group_result.scalar_one_or_none.return_value = sample_group
role_result = MagicMock()
role_result.scalar_one_or_none.return_value = sample_role
mock_session.execute.side_effect = [group_result, role_result]
group = await auth_service.remove_role_from_group(sample_group.id, sample_role.id)
# Should complete without error
assert len(group.roles) == 0
# =============================================================================
# Role Schema Tests
# =============================================================================
class TestRoleSchema:
"""Test RoleSchema validation."""
def test_role_schema_creation(self):
"""RoleSchema should be creatable with valid data."""
schema = RoleSchema(
id=uuid.uuid4(),
name="control-room.general:admin",
domain="control-room",
category="general",
action="admin",
)
assert schema.name == "control-room.general:admin"
assert schema.domain == "control-room"
assert schema.category == "general"
assert schema.action == "admin"
def test_role_schema_category_default(self):
"""RoleSchema should default category to 'general'."""
schema = RoleSchema(
id=uuid.uuid4(),
name="media.general:viewer",
domain="media",
action="viewer",
)
assert schema.category == "general"
+757
View File
@@ -0,0 +1,757 @@
"""
Tests for the Book domain (T-1 / D-68, D-70, T-7) against `core_api_test`.
`get_async_session` is overridden to the test database for every test here
— the Book routes list `caller` before `session` in their dependencies, but
overriding is what actually guarantees the suite never opens a connection
to production regardless of FastAPI's dependency resolution order.
"""
import asyncio
import uuid
from unittest.mock import patch
import pytest
from fastapi.testclient import TestClient
from src.domains.auth.oidc import oidc_config
from src.main import app
from src.shared.config import get_settings
from src.shared.database import get_async_session
settings = get_settings()
OTHER_TENANT = "llm_tester"
OFFICE_KEY = "test-office-key"
@pytest.fixture(autouse=True)
def office_key():
"""Every Book test authenticates as the office caller unless it says otherwise."""
original = settings.office_service_key
settings.office_service_key = OFFICE_KEY
yield
settings.office_service_key = original
@pytest.fixture
def client(clean_book_tables, test_database_url):
"""
A TestClient whose DB dependency is overridden to core_api_test.
The override must build its own engine/session per call rather than
reuse `clean_book_tables`'s already-open one: TestClient drives the
ASGI app from a different event loop than the one pytest-asyncio ran
the fixture's setup in, and an asyncpg connection is not portable
across loops — reusing it here raised "Future ... attached to a
different loop" and then hung. Building fresh per call keeps
everything inside whichever loop is actually running the request.
"""
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from sqlalchemy.pool import NullPool
asyncpg_url = test_database_url.replace("postgresql://", "postgresql+asyncpg://", 1)
async def _override():
engine = create_async_engine(asyncpg_url, poolclass=NullPool)
session_factory = async_sessionmaker(bind=engine, expire_on_commit=False)
async with session_factory() as session:
yield session
await engine.dispose()
app.dependency_overrides[get_async_session] = _override
yield TestClient(app)
app.dependency_overrides.pop(get_async_session, None)
VALID_CONSUMER = "office-claude"
VALID_TENANT = "jpmschweitzer"
def office_headers(
identity: str = "jeroen",
consumer: str = VALID_CONSUMER,
tenant: str = VALID_TENANT,
) -> dict:
return {
"Authorization": f"Bearer {OFFICE_KEY}",
"X-Office-Identity": identity,
"X-Office-Consumer": consumer,
"X-Office-Tenant": tenant,
}
class TestCallerMatrixOnBookRoutes:
"""Dossier verification #3, the Book-specific half."""
def test_no_auth_is_401_on_book_routes(self, client):
oidc_config.configure(enabled=True, issuers=[], audiences=[])
try:
response = client.post("/book/records", json={"body": "hi"})
finally:
oidc_config.configure(enabled=False, issuers=[], audiences=[])
assert response.status_code == 401
def test_office_write_without_identity_is_400(self, client):
response = client.post(
"/book/records",
json={"body": "hi"},
headers={
"Authorization": f"Bearer {OFFICE_KEY}",
"X-Office-Consumer": VALID_CONSUMER,
"X-Office-Tenant": VALID_TENANT,
},
)
assert response.status_code == 400
assert response.json()["detail"]["code"] == "identity_required"
def test_office_write_without_consumer_is_400(self, client):
response = client.post(
"/book/records",
json={"body": "hi"},
headers={
"Authorization": f"Bearer {OFFICE_KEY}",
"X-Office-Identity": "jeroen",
"X-Office-Tenant": VALID_TENANT,
},
)
assert response.status_code == 400
assert response.json()["detail"]["code"] == "consumer_required"
def test_office_write_without_tenant_is_400(self, client):
response = client.post(
"/book/records",
json={"body": "hi"},
headers={
"Authorization": f"Bearer {OFFICE_KEY}",
"X-Office-Identity": "jeroen",
"X-Office-Consumer": VALID_CONSUMER,
},
)
assert response.status_code == 400
assert response.json()["detail"]["code"] == "tenant_required"
def test_office_read_without_tenant_is_400(self, client):
"""The tenant scopes reads too — a list call with no tenant is 400,
even though list carries no identity_required."""
response = client.get(
"/book/records",
headers={
"Authorization": f"Bearer {OFFICE_KEY}",
"X-Office-Consumer": VALID_CONSUMER,
},
)
assert response.status_code == 400
assert response.json()["detail"]["code"] == "tenant_required"
def test_office_malformed_consumer_is_400(self, client):
response = client.get(
"/book/records",
headers={
"Authorization": f"Bearer {OFFICE_KEY}",
"X-Office-Consumer": "Not Valid",
"X-Office-Tenant": VALID_TENANT,
},
)
assert response.status_code == 400
assert response.json()["detail"]["code"] == "consumer_invalid"
def test_office_malformed_tenant_is_400(self, client):
response = client.get(
"/book/records",
headers={
"Authorization": f"Bearer {OFFICE_KEY}",
"X-Office-Consumer": VALID_CONSUMER,
"X-Office-Tenant": "Not Valid",
},
)
assert response.status_code == 400
assert response.json()["detail"]["code"] == "tenant_invalid"
def test_scheduler_key_is_403_on_book_records(self, client):
settings.scheduler_service_key = "test-scheduler-key"
try:
response = client.post(
"/book/records",
json={"body": "hi"},
headers={"Authorization": "Bearer test-scheduler-key"},
)
finally:
settings.scheduler_service_key = ""
assert response.status_code == 403
assert response.json()["detail"]["code"] == "forbidden"
def test_malformed_identity_is_400(self, client):
response = client.post(
"/book/records",
json={"body": "hi"},
headers={
"Authorization": f"Bearer {OFFICE_KEY}",
"X-Office-Identity": "Not Valid",
"X-Office-Consumer": VALID_CONSUMER,
"X-Office-Tenant": VALID_TENANT,
},
)
assert response.status_code == 400
assert response.json()["detail"]["code"] == "identity_invalid"
class TestAddShowListFind:
def test_add_note_defaults_to_personal_scope_and_inbox_free(self, client):
response = client.post(
"/book/records", json={"body": "buy milk"}, headers=office_headers()
)
assert response.status_code == 201
data = response.json()
assert data["kind"] == "note"
assert data["status"] == "active"
assert data["scope"] == "personal"
assert data["created_by"] == "jeroen"
def test_add_rejects_bad_status_for_kind(self, client):
response = client.post(
"/book/records",
json={"kind": "task", "body": "x", "status": "settled"},
headers=office_headers(),
)
assert response.status_code == 400
assert response.json()["detail"]["code"] == "invalid_status"
def test_every_record_carries_exactly_one_of_work_or_personal(self, client):
default_scope = client.post(
"/book/records", json={"body": "default scope"}, headers=office_headers()
).json()["scope"]
assert default_scope in ("work", "personal")
work_scope = client.post(
"/book/records",
json={"body": "work item", "scope": "work"},
headers=office_headers(),
).json()["scope"]
assert work_scope == "work"
rejected = client.post(
"/book/records",
json={"body": "bad scope", "scope": "both"},
headers=office_headers(),
)
assert rejected.status_code == 422 # pydantic Literal rejects a third value
def test_show_returns_record_tags_links_and_backlinks(self, client):
parent = client.post(
"/book/records", json={"kind": "task", "body": "project"}, headers=office_headers()
).json()
child = client.post(
"/book/records",
json={"kind": "task", "body": "subtask", "parent": parent["id"], "tags": ["chore"]},
headers=office_headers(),
).json()
child_shown = client.get(f"/book/records/{child['id']}", headers=office_headers()).json()
assert child_shown["tags"] == ["chore"]
assert len(child_shown["links"]) == 1
assert child_shown["links"][0]["relation"] == "parent"
assert child_shown["links"][0]["to_id"] == parent["id"]
parent_shown = client.get(f"/book/records/{parent['id']}", headers=office_headers()).json()
assert len(parent_shown["backlinks"]) == 1
assert parent_shown["backlinks"][0]["from_id"] == child["id"]
def test_show_missing_record_is_404(self, client):
response = client.get(f"/book/records/{uuid.uuid4()}", headers=office_headers())
assert response.status_code == 404
assert response.json()["detail"]["code"] == "not_found"
def test_list_filters_by_scope(self, client):
client.post("/book/records", json={"body": "a", "scope": "work"}, headers=office_headers())
client.post("/book/records", json={"body": "b", "scope": "personal"}, headers=office_headers())
work_only = client.get("/book/records", params={"scope": "work"}, headers=office_headers()).json()
assert work_only["total"] == 1
assert work_only["records"][0]["scope"] == "work"
def test_find_full_text_matches_a_note_and_a_task(self, client):
"""Dossier verification #4: a note and a task, both found by full-text."""
note = client.post(
"/book/records",
json={"kind": "note", "title": "Weekend plans", "body": "visit the garden centre"},
headers=office_headers(),
).json()
task = client.post(
"/book/records",
json={"kind": "task", "body": "renew the car insurance"},
headers=office_headers(),
).json()
found_note_by_body = client.get(
"/book/records/find", params={"q": "garden"}, headers=office_headers()
).json()
assert found_note_by_body["total"] == 1
assert found_note_by_body["records"][0]["id"] == note["id"]
found_note_by_title = client.get(
"/book/records/find", params={"q": "weekend"}, headers=office_headers()
).json()
assert found_note_by_title["total"] == 1
assert found_note_by_title["records"][0]["id"] == note["id"]
found_task = client.get(
"/book/records/find", params={"q": "insurance"}, headers=office_headers()
).json()
assert found_task["total"] == 1
assert found_task["records"][0]["id"] == task["id"]
assert found_task["records"][0]["kind"] == "task"
class TestChange:
def test_change_updates_only_given_fields(self, client):
record = client.post(
"/book/records", json={"kind": "task", "body": "water the plants"}, headers=office_headers()
).json()
changed = client.patch(
f"/book/records/{record['id']}",
json={"priority": 1},
headers=office_headers(),
).json()
assert changed["priority"] == 1
assert changed["body"] == "water the plants"
def test_change_without_identity_is_400(self, client):
record = client.post(
"/book/records", json={"body": "x"}, headers=office_headers()
).json()
response = client.patch(
f"/book/records/{record['id']}",
json={"priority": 1},
headers={
"Authorization": f"Bearer {OFFICE_KEY}",
"X-Office-Consumer": VALID_CONSUMER,
"X-Office-Tenant": VALID_TENANT,
},
)
assert response.status_code == 400
assert response.json()["detail"]["code"] == "identity_required"
class TestClose:
def test_close_task_done(self, client):
task = client.post(
"/book/records", json={"kind": "task", "body": "mow the lawn"}, headers=office_headers()
).json()
closed = client.post(
f"/book/records/{task['id']}/close", json={"outcome": "done"}, headers=office_headers()
).json()
assert closed["status"] == "done"
def test_close_question_via_resolves_sets_settled(self, client):
question = client.post(
"/book/records",
json={"kind": "question", "body": "what colour for the fence?"},
headers=office_headers(),
).json()
decision = client.post(
"/book/records",
json={"kind": "decision", "body": "fence will be green"},
headers=office_headers(),
).json()
closed = client.post(
f"/book/records/{question['id']}/close",
json={"resolved_by": decision["id"]},
headers=office_headers(),
).json()
assert closed["status"] == "settled"
shown = client.get(f"/book/records/{question['id']}", headers=office_headers()).json()
assert any(b["relation"] == "resolves" and b["from_id"] == decision["id"] for b in shown["backlinks"])
def test_close_decision_via_supersedes_sets_superseded(self, client):
old_decision = client.post(
"/book/records", json={"kind": "decision", "body": "fence will be green"}, headers=office_headers()
).json()
new_decision = client.post(
"/book/records", json={"kind": "decision", "body": "fence will be red"}, headers=office_headers()
).json()
closed = client.post(
f"/book/records/{old_decision['id']}/close",
json={"superseded_by": new_decision["id"]},
headers=office_headers(),
).json()
assert closed["status"] == "superseded"
def test_close_note_has_no_transition(self, client):
note = client.post("/book/records", json={"body": "just a note"}, headers=office_headers()).json()
response = client.post(
f"/book/records/{note['id']}/close", json={}, headers=office_headers()
)
assert response.status_code == 400
assert response.json()["detail"]["code"] == "invalid_status"
class TestLinkUnlink:
def test_link_then_unlink(self, client):
a = client.post("/book/records", json={"body": "a"}, headers=office_headers()).json()
b = client.post("/book/records", json={"body": "b"}, headers=office_headers()).json()
client.post(
f"/book/records/{a['id']}/link",
json={"to_id": b["id"], "relation": "related"},
headers=office_headers(),
)
shown = client.get(f"/book/records/{a['id']}", headers=office_headers()).json()
assert len(shown["links"]) == 1
unlinked = client.post(
f"/book/records/{a['id']}/unlink",
json={"to_id": b["id"], "relation": "related"},
headers=office_headers(),
)
assert unlinked.status_code == 200
shown_after = client.get(f"/book/records/{a['id']}", headers=office_headers()).json()
assert len(shown_after["links"]) == 0
def test_unlink_nonexistent_link_is_404(self, client):
a = client.post("/book/records", json={"body": "a"}, headers=office_headers()).json()
b = client.post("/book/records", json={"body": "b"}, headers=office_headers()).json()
response = client.post(
f"/book/records/{a['id']}/unlink",
json={"to_id": b["id"], "relation": "blocks"},
headers=office_headers(),
)
assert response.status_code == 404
class TestTagUntag:
def test_tag_then_untag(self, client):
record = client.post("/book/records", json={"body": "x"}, headers=office_headers()).json()
tagged = client.post(
f"/book/records/{record['id']}/tag", json={"tag": "chore"}, headers=office_headers()
).json()
assert tagged["tags"] == ["chore"]
untagged = client.post(
f"/book/records/{record['id']}/untag", json={"tag": "chore"}, headers=office_headers()
).json()
assert untagged["tags"] == []
def test_tagging_twice_is_idempotent(self, client):
record = client.post("/book/records", json={"body": "x"}, headers=office_headers()).json()
client.post(f"/book/records/{record['id']}/tag", json={"tag": "chore"}, headers=office_headers())
second = client.post(
f"/book/records/{record['id']}/tag", json={"tag": "chore"}, headers=office_headers()
)
assert second.status_code == 200
assert second.json()["tags"] == ["chore"]
class TestHistory:
def test_every_write_appends_an_event_carrying_identity(self, client):
record = client.post(
"/book/records", json={"body": "x"}, headers=office_headers(identity="jeroen")
).json()
client.patch(
f"/book/records/{record['id']}",
json={"priority": 2},
headers=office_headers(identity="tower-control"),
)
history = client.get(f"/book/records/{record['id']}/history", headers=office_headers()).json()
actions = [e["action"] for e in history["events"]]
identities = {e["identity"] for e in history["events"]}
assert "add" in actions
assert "change" in actions
assert identities == {"jeroen", "tower-control"}
def test_history_events_are_immutable_at_the_api(self, client):
"""There is no update/delete verb for events — only `add` and `change`
write them, and the append-only trigger (tested in
test_book_migration.py) is what actually enforces it."""
record = client.post("/book/records", json={"body": "x"}, headers=office_headers()).json()
history = client.get(f"/book/records/{record['id']}/history", headers=office_headers()).json()
assert len(history["events"]) == 1
assert history["events"][0]["action"] == "add"
def test_a_write_s_event_records_the_consumer(self, client):
"""T-4: X-Office-Client is retired; book_events.client (no migration
this round) now carries the consumer instead."""
record = client.post(
"/book/records",
json={"body": "x"},
headers=office_headers(consumer="office-gemini"),
).json()
history = client.get(
f"/book/records/{record['id']}/history", headers=office_headers()
).json()
assert history["events"][0]["action"] == "add"
assert history["events"][0]["client"] == "office-gemini"
class TestNumbering:
"""core-api T-7: `number` is allocated per tenant, never globally."""
def test_numbering_starts_at_one_per_tenant_and_increments(self, client):
first = client.post(
"/book/records", json={"body": "a"}, headers=office_headers(tenant=VALID_TENANT)
).json()
second = client.post(
"/book/records", json={"body": "b"}, headers=office_headers(tenant=VALID_TENANT)
).json()
other_tenant_first = client.post(
"/book/records", json={"body": "c"}, headers=office_headers(tenant=OTHER_TENANT)
).json()
assert first["number"] == 1
assert second["number"] == 2
# A different tenant starts its own sequence at 1 — it can never
# infer VALID_TENANT's activity from its own numbers.
assert other_tenant_first["number"] == 1
def test_list_number_finds_the_record(self, client):
record = client.post(
"/book/records", json={"body": "find me"}, headers=office_headers()
).json()
client.post("/book/records", json={"body": "noise"}, headers=office_headers())
found = client.get(
"/book/records", params={"number": record["number"]}, headers=office_headers()
).json()
assert found["total"] == 1
assert found["records"][0]["id"] == record["id"]
@pytest.mark.asyncio
async def test_two_concurrent_creates_in_one_tenant_get_distinct_numbers(
self, clean_book_tables, test_database_url
):
"""Direct service calls on two independent sessions, run
concurrently via asyncio.gather, rather than through TestClient
(whose calls are issued one at a time and so would never actually
overlap at the database). Exercises BookService._allocate_number's
FOR UPDATE lock for real."""
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from sqlalchemy.pool import NullPool
from src.domains.auth.callers import Caller
from src.domains.book.schemas import BookRecordCreate
from src.domains.book.service import get_book_service
asyncpg_url = test_database_url.replace("postgresql://", "postgresql+asyncpg://", 1)
service = get_book_service()
caller = Caller(kind="office", identity="jeroen", consumer="office-claude", tenant=VALID_TENANT)
async def _create(body: str) -> int:
engine = create_async_engine(asyncpg_url, poolclass=NullPool)
session_factory = async_sessionmaker(bind=engine, expire_on_commit=False)
try:
async with session_factory() as session:
record, _ = await service.add_record(session, BookRecordCreate(body=body), caller)
return record.number
finally:
await engine.dispose()
numbers = await asyncio.gather(_create("one"), _create("two"))
assert sorted(numbers) == [1, 2]
class TestTenantIsolation:
"""core-api T-7 / workspace D-73: a record in another tenant answers
exactly like one that does not exist, checked in both directions."""
def test_llm_tester_cannot_list_jpmschweitzer_records(self, client):
client.post(
"/book/records", json={"body": "secret"}, headers=office_headers(tenant=VALID_TENANT)
)
as_other = client.get("/book/records", headers=office_headers(tenant=OTHER_TENANT)).json()
assert as_other["total"] == 0
def test_llm_tester_cannot_show_a_jpmschweitzer_record(self, client):
record = client.post(
"/book/records", json={"body": "secret"}, headers=office_headers(tenant=VALID_TENANT)
).json()
response = client.get(
f"/book/records/{record['id']}", headers=office_headers(tenant=OTHER_TENANT)
)
assert response.status_code == 404
assert response.json()["detail"]["code"] == "not_found"
def test_jpmschweitzer_cannot_show_an_llm_tester_record(self, client):
"""The reverse direction of the previous test — the boundary is
symmetric, not a one-way allowlist for the production tenant."""
record = client.post(
"/book/records", json={"body": "secret"}, headers=office_headers(tenant=OTHER_TENANT)
).json()
response = client.get(
f"/book/records/{record['id']}", headers=office_headers(tenant=VALID_TENANT)
)
assert response.status_code == 404
assert response.json()["detail"]["code"] == "not_found"
def test_llm_tester_cannot_change_a_jpmschweitzer_record(self, client):
record = client.post(
"/book/records", json={"body": "secret"}, headers=office_headers(tenant=VALID_TENANT)
).json()
response = client.patch(
f"/book/records/{record['id']}",
json={"priority": 1},
headers=office_headers(tenant=OTHER_TENANT),
)
assert response.status_code == 404
assert response.json()["detail"]["code"] == "not_found"
def test_llm_tester_cannot_close_a_jpmschweitzer_record(self, client):
task = client.post(
"/book/records",
json={"kind": "task", "body": "secret"},
headers=office_headers(tenant=VALID_TENANT),
).json()
response = client.post(
f"/book/records/{task['id']}/close",
json={"outcome": "done"},
headers=office_headers(tenant=OTHER_TENANT),
)
assert response.status_code == 404
assert response.json()["detail"]["code"] == "not_found"
def test_llm_tester_cannot_tag_a_jpmschweitzer_record(self, client):
record = client.post(
"/book/records", json={"body": "secret"}, headers=office_headers(tenant=VALID_TENANT)
).json()
response = client.post(
f"/book/records/{record['id']}/tag",
json={"tag": "chore"},
headers=office_headers(tenant=OTHER_TENANT),
)
assert response.status_code == 404
assert response.json()["detail"]["code"] == "not_found"
def test_llm_tester_cannot_link_to_a_jpmschweitzer_record(self, client):
"""`link` refuses a target in another tenant the same way (dossier)."""
a = client.post(
"/book/records", json={"body": "a"}, headers=office_headers(tenant=OTHER_TENANT)
).json()
b = client.post(
"/book/records", json={"body": "b"}, headers=office_headers(tenant=VALID_TENANT)
).json()
response = client.post(
f"/book/records/{a['id']}/link",
json={"to_id": b["id"], "relation": "related"},
headers=office_headers(tenant=OTHER_TENANT),
)
assert response.status_code == 404
assert response.json()["detail"]["code"] == "not_found"
def test_llm_tester_cannot_read_history_of_a_jpmschweitzer_record(self, client):
record = client.post(
"/book/records", json={"body": "secret"}, headers=office_headers(tenant=VALID_TENANT)
).json()
response = client.get(
f"/book/records/{record['id']}/history", headers=office_headers(tenant=OTHER_TENANT)
)
assert response.status_code == 404
assert response.json()["detail"]["code"] == "not_found"
def test_add_with_a_parent_in_another_tenant_is_404_and_nothing_persists(self, client):
"""The `parent` convenience link in `add` is a link target too, so
it is refused the same way `link` is. The attempted child must not
persist either — the whole add is one transaction."""
parent = client.post(
"/book/records", json={"body": "parent"}, headers=office_headers(tenant=OTHER_TENANT)
).json()
response = client.post(
"/book/records",
json={"body": "child", "parent": parent["id"]},
headers=office_headers(tenant=VALID_TENANT),
)
assert response.status_code == 404
assert response.json()["detail"]["code"] == "not_found"
listing = client.get("/book/records", headers=office_headers(tenant=VALID_TENANT)).json()
assert listing["total"] == 0
def test_find_does_not_leak_a_jpmschweitzer_record_to_llm_tester(self, client):
client.post(
"/book/records",
json={"body": "unique-secret-garden-term"},
headers=office_headers(tenant=VALID_TENANT),
)
found = client.get(
"/book/records/find",
params={"q": "unique-secret-garden-term"},
headers=office_headers(tenant=OTHER_TENANT),
).json()
assert found["total"] == 0
class TestUserCallerTenantRequired:
"""A user-kind caller without a `tatlock_user` claim gets 400
tenant_required — the Office caller gets this for free from
callers.py, but a "user"-kind caller's tenant is optional there, so
Book's own `_require_tenant` is what actually enforces it (dossier
verification #2)."""
def test_user_without_tenant_claim_is_400_on_read(self, client):
with patch("src.domains.auth.callers.get_current_user") as mock_get_current_user:
mock_get_current_user.return_value = {"preferred_username": "jeroen"}
response = client.get("/book/records", headers={"Authorization": "Bearer some-jwt"})
assert response.status_code == 400
assert response.json()["detail"]["code"] == "tenant_required"
def test_user_without_tenant_claim_is_400_on_write(self, client):
with patch("src.domains.auth.callers.get_current_user") as mock_get_current_user:
mock_get_current_user.return_value = {"preferred_username": "jeroen"}
response = client.post(
"/book/records", json={"body": "hi"}, headers={"Authorization": "Bearer some-jwt"}
)
assert response.status_code == 400
assert response.json()["detail"]["code"] == "tenant_required"
def test_user_with_tenant_claim_succeeds(self, client):
with patch("src.domains.auth.callers.get_current_user") as mock_get_current_user:
mock_get_current_user.return_value = {
"preferred_username": "jeroen",
"tatlock_user": VALID_TENANT,
}
response = client.post(
"/book/records", json={"body": "hi"}, headers={"Authorization": "Bearer some-jwt"}
)
assert response.status_code == 201
assert response.json()["created_by"] == "jeroen"
assert response.json()["number"] == 1
class TestTenantRegistryEnforcement:
"""core-api T-8 / workspace D-73: Book's own `_require_tenant` now
delegates to the tenant registry's one enforcement point, so a Book
route gains three refusal paths it did not have before, alongside the
control that a properly granted, active tenant still succeeds."""
def test_unknown_tenant_is_403(self, client):
response = client.get(
"/book/records", headers=office_headers(tenant="no-such-tenant")
)
assert response.status_code == 403
assert response.json()["detail"]["code"] == "tenant_unknown"
def test_suspended_tenant_is_403(self, client, suspended_tenant):
response = client.get(
"/book/records", headers=office_headers(tenant=suspended_tenant)
)
assert response.status_code == 403
assert response.json()["detail"]["code"] == "tenant_suspended"
def test_consumer_without_a_grant_is_403(self, client):
response = client.get(
"/book/records", headers=office_headers(consumer="office-ungranted")
)
assert response.status_code == 403
assert response.json()["detail"]["code"] == "consumer_not_granted"
def test_granted_consumer_in_an_active_tenant_succeeds(self, client):
"""The control: office-claude, granted in jpmschweitzer (active),
still reaches the route."""
response = client.get("/book/records", headers=office_headers())
assert response.status_code == 200
+157
View File
@@ -0,0 +1,157 @@
"""
Tests for Book migration 005.
Runs against `core_api_test` only (see conftest.py — skip when
TEST_DATABASE_URL is unset, hard failure if it resolves to `core_api`).
Self-contained: starts and ends at head regardless of what ran before it,
so test order never matters for the schema the rest of the suite needs.
"""
import pytest
from sqlalchemy import text
from sqlalchemy.ext.asyncio import create_async_engine
from sqlalchemy.pool import NullPool
from tests.conftest import run_alembic
async def _table_exists(url: str, table: str) -> bool:
asyncpg_url = url.replace("postgresql://", "postgresql+asyncpg://", 1)
engine = create_async_engine(asyncpg_url, poolclass=NullPool)
try:
async with engine.connect() as conn:
result = await conn.execute(
text("SELECT to_regclass(:name) IS NOT NULL AS present"),
{"name": table},
)
return bool(result.scalar())
finally:
await engine.dispose()
async def _scalar(url: str, query: str, **params) -> object:
asyncpg_url = url.replace("postgresql://", "postgresql+asyncpg://", 1)
engine = create_async_engine(asyncpg_url, poolclass=NullPool)
try:
async with engine.connect() as conn:
result = await conn.execute(text(query), params)
return result.scalar()
finally:
await engine.dispose()
class TestMigration005RoundTrip:
"""Migration 005 must upgrade and downgrade cleanly (dossier verification #2)."""
@pytest.mark.asyncio
async def test_upgrade_creates_book_tables(self, test_database_url):
result = run_alembic(test_database_url, "upgrade", "head")
assert result.returncode == 0, result.stderr
for table in (
"book_records", "book_tags", "book_record_links", "book_attachments",
"book_events", "book_tenant_counters",
):
assert await _table_exists(test_database_url, table), f"{table} missing after upgrade"
@pytest.mark.asyncio
async def test_downgrade_then_upgrade_round_trips_cleanly(self, test_database_url):
# Baseline: make sure we start at head.
baseline = run_alembic(test_database_url, "upgrade", "head")
assert baseline.returncode == 0, baseline.stderr
down = run_alembic(test_database_url, "downgrade", "004")
assert down.returncode == 0, down.stderr
assert not await _table_exists(test_database_url, "book_records")
assert not await _table_exists(test_database_url, "book_events")
assert not await _table_exists(test_database_url, "book_tenant_counters")
# Leave the schema at head for the rest of the suite.
up = run_alembic(test_database_url, "upgrade", "005")
assert up.returncode == 0, up.stderr
assert await _table_exists(test_database_url, "book_records")
assert await _table_exists(test_database_url, "book_events")
assert await _table_exists(test_database_url, "book_tenant_counters")
class TestMigration005TenantAndNumber:
"""core-api T-7: owner_tenant/number amended onto 005 before its first release."""
@pytest.mark.asyncio
async def test_book_records_owner_tenant_and_number_are_not_null(self, test_database_url):
run_alembic(test_database_url, "upgrade", "head")
for column in ("owner_tenant", "number"):
nullable = await _scalar(
test_database_url,
"SELECT is_nullable FROM information_schema.columns "
"WHERE table_name = 'book_records' AND column_name = :column",
column=column,
)
assert nullable == "NO", f"book_records.{column} must be NOT NULL, got {nullable!r}"
@pytest.mark.asyncio
async def test_book_records_has_tenant_number_unique_constraint(self, test_database_url):
run_alembic(test_database_url, "upgrade", "head")
found = await _scalar(
test_database_url,
"SELECT conname FROM pg_constraint WHERE conname = 'uq_book_records_tenant_number'",
)
assert found == "uq_book_records_tenant_number"
@pytest.mark.asyncio
async def test_book_records_has_tenant_kind_status_index(self, test_database_url):
run_alembic(test_database_url, "upgrade", "head")
found = await _scalar(
test_database_url,
"SELECT indexname FROM pg_indexes WHERE indexname = 'ix_book_records_tenant_kind_status'",
)
assert found == "ix_book_records_tenant_kind_status"
class TestBookEventsAppendOnly:
"""The trigger must block UPDATE and DELETE on book_events, no exceptions."""
@pytest.mark.asyncio
async def test_update_on_book_events_raises(self, clean_book_tables):
from src.domains.book.models import BookEvent, BookRecord
session = clean_book_tables
record = BookRecord(
owner_tenant="jpmschweitzer", number=1,
kind="note", body="trigger test", status="active", created_by="test-harness",
)
session.add(record)
await session.flush()
event = BookEvent(record_id=record.id, identity="test-harness", client=None, action="add", diff={})
session.add(event)
await session.commit()
with pytest.raises(Exception) as exc_info:
await session.execute(
text("UPDATE book_events SET action = 'tampered' WHERE id = :id").bindparams(id=event.id)
)
await session.commit()
assert "append-only" in str(exc_info.value).lower() or "book_events" in str(exc_info.value)
await session.rollback()
@pytest.mark.asyncio
async def test_delete_on_book_events_raises(self, clean_book_tables):
from src.domains.book.models import BookEvent, BookRecord
session = clean_book_tables
record = BookRecord(
owner_tenant="jpmschweitzer", number=1,
kind="note", body="trigger test", status="active", created_by="test-harness",
)
session.add(record)
await session.flush()
event = BookEvent(record_id=record.id, identity="test-harness", client=None, action="add", diff={})
session.add(event)
await session.commit()
with pytest.raises(Exception) as exc_info:
await session.execute(
text("DELETE FROM book_events WHERE id = :id").bindparams(id=event.id)
)
await session.commit()
assert "append-only" in str(exc_info.value).lower() or "book_events" in str(exc_info.value)
await session.rollback()
+411
View File
@@ -0,0 +1,411 @@
"""
Tests for require_caller (T-1, extended by T-4).
Builds a tiny standalone FastAPI app rather than reusing src.main's real
routes, so these are pure caller-resolution tests with no database and no
coupling to a particular domain's routing.
"""
import pytest
from fastapi import Depends, FastAPI
from fastapi.testclient import TestClient
from unittest.mock import patch
from src.domains.auth.callers import Caller, require_caller
from src.domains.auth.oidc import oidc_config
from src.shared.config import get_settings
settings = get_settings()
VALID_CONSUMER = "office-claude"
VALID_TENANT = "jpmschweitzer"
def _reset_service_keys():
settings.office_service_key = ""
settings.tatlock_service_key = ""
settings.scheduler_service_key = ""
@pytest.fixture(autouse=True)
def clean_caller_state():
"""Service keys and OIDC state are process-global; isolate every test."""
original_keys = (
settings.office_service_key,
settings.tatlock_service_key,
settings.scheduler_service_key,
)
original_oidc = (oidc_config.enabled, oidc_config.issuers, oidc_config.audiences)
_reset_service_keys()
yield
(
settings.office_service_key,
settings.tatlock_service_key,
settings.scheduler_service_key,
) = original_keys
oidc_config.enabled, oidc_config.issuers, oidc_config.audiences = original_oidc
# Named with a leading underscore so pytest does not try to collect it as
# a test module attribute (it previously warned: "cannot collect 'test_app'
# because it is not a function").
_caller_app = FastAPI()
@_caller_app.get("/only-office")
async def only_office(caller: Caller = Depends(require_caller("office"))):
return {
"kind": caller.kind,
"identity": caller.identity,
"consumer": caller.consumer,
"tenant": caller.tenant,
}
@_caller_app.get("/office-tatlock-user")
async def office_tatlock_user(
caller: Caller = Depends(require_caller("office", "tatlock", "user")),
):
return {"kind": caller.kind, "identity": caller.identity, "tenant": caller.tenant}
@_caller_app.post("/office-user-write")
async def office_user_write(
caller: Caller = Depends(require_caller("office", "user", identity_required=True)),
):
return {"kind": caller.kind, "identity": caller.identity}
@_caller_app.get("/all-kinds")
async def all_kinds(
caller: Caller = Depends(require_caller("office", "tatlock", "scheduler", "user")),
):
return {"kind": caller.kind, "identity": caller.identity, "tenant": caller.tenant}
@pytest.fixture
def client():
return TestClient(_caller_app)
class TestNoAuth:
def test_no_token_defaults_to_local_user_when_oidc_disabled(self, client):
"""Matches existing housekeeping/dashboard tests: OIDC off -> local user."""
response = client.get("/office-tatlock-user")
assert response.status_code == 200
assert response.json() == {"kind": "user", "identity": "local", "tenant": None}
def test_no_token_401_when_oidc_enabled(self, client):
oidc_config.configure(enabled=True, issuers=[], audiences=[])
response = client.get("/office-tatlock-user")
assert response.status_code == 401
class TestOfficeKey:
def test_office_key_with_identity_resolves(self, client):
settings.office_service_key = "office-secret"
response = client.get(
"/office-tatlock-user",
headers={
"Authorization": "Bearer office-secret",
"X-Office-Identity": "jeroen",
"X-Office-Consumer": VALID_CONSUMER,
"X-Office-Tenant": VALID_TENANT,
},
)
assert response.status_code == 200
assert response.json() == {"kind": "office", "identity": "jeroen", "tenant": VALID_TENANT}
def test_office_key_consumer_and_tenant_are_set_on_caller(self, client):
"""Dossier verification: both present -> Caller.consumer and .tenant set."""
settings.office_service_key = "office-secret"
response = client.get(
"/only-office",
headers={
"Authorization": "Bearer office-secret",
"X-Office-Identity": "jeroen",
"X-Office-Consumer": VALID_CONSUMER,
"X-Office-Tenant": VALID_TENANT,
},
)
assert response.status_code == 200
assert response.json() == {
"kind": "office",
"identity": "jeroen",
"consumer": VALID_CONSUMER,
"tenant": VALID_TENANT,
}
def test_office_key_x_office_client_header_is_ignored(self, client):
"""X-Office-Client is retired (T-4) in favour of X-Office-Consumer."""
settings.office_service_key = "office-secret"
response = client.get(
"/only-office",
headers={
"Authorization": "Bearer office-secret",
"X-Office-Identity": "jeroen",
"X-Office-Consumer": VALID_CONSUMER,
"X-Office-Tenant": VALID_TENANT,
"X-Office-Client": "claude-desktop",
},
)
assert response.status_code == 200
body = response.json()
assert body["consumer"] == VALID_CONSUMER # not "claude-desktop"
assert "client" not in body
def test_office_key_missing_identity_on_read_defaults_empty(self, client):
settings.office_service_key = "office-secret"
response = client.get(
"/office-tatlock-user",
headers={
"Authorization": "Bearer office-secret",
"X-Office-Consumer": VALID_CONSUMER,
"X-Office-Tenant": VALID_TENANT,
},
)
assert response.status_code == 200
assert response.json() == {"kind": "office", "identity": "", "tenant": VALID_TENANT}
def test_office_key_missing_identity_on_write_is_400(self, client):
settings.office_service_key = "office-secret"
response = client.post(
"/office-user-write",
headers={
"Authorization": "Bearer office-secret",
"X-Office-Consumer": VALID_CONSUMER,
"X-Office-Tenant": VALID_TENANT,
},
)
assert response.status_code == 400
assert response.json()["detail"]["code"] == "identity_required"
@pytest.mark.parametrize("identity", ["Jeroen", "group-home", "tatlock", "office", "book", "scheduler", "-abc", "a_b"])
def test_office_key_malformed_or_reserved_identity_is_400(self, client, identity):
settings.office_service_key = "office-secret"
response = client.post(
"/office-user-write",
headers={
"Authorization": "Bearer office-secret",
"X-Office-Identity": identity,
"X-Office-Consumer": VALID_CONSUMER,
"X-Office-Tenant": VALID_TENANT,
},
)
assert response.status_code == 400
assert response.json()["detail"]["code"] == "identity_invalid"
def test_office_key_valid_identity_on_write_succeeds(self, client):
settings.office_service_key = "office-secret"
response = client.post(
"/office-user-write",
headers={
"Authorization": "Bearer office-secret",
"X-Office-Identity": "jeroen",
"X-Office-Consumer": VALID_CONSUMER,
"X-Office-Tenant": VALID_TENANT,
},
)
assert response.status_code == 200
assert response.json() == {"kind": "office", "identity": "jeroen"}
class TestOfficeConsumerAndTenant:
"""T-4: both are required on every office call, reads included — the
tenant scopes reads too. Missing -> 400 */_required; malformed -> 400
*/_invalid."""
def test_missing_consumer_is_400(self, client):
settings.office_service_key = "office-secret"
response = client.get(
"/office-tatlock-user",
headers={
"Authorization": "Bearer office-secret",
"X-Office-Tenant": VALID_TENANT,
},
)
assert response.status_code == 400
assert response.json()["detail"]["code"] == "consumer_required"
@pytest.mark.parametrize("consumer", ["Office-Claude", "office_claude", "-office"])
def test_malformed_consumer_is_400(self, client, consumer):
settings.office_service_key = "office-secret"
response = client.get(
"/office-tatlock-user",
headers={
"Authorization": "Bearer office-secret",
"X-Office-Consumer": consumer,
"X-Office-Tenant": VALID_TENANT,
},
)
assert response.status_code == 400
assert response.json()["detail"]["code"] == "consumer_invalid"
def test_missing_tenant_is_400_even_on_a_read(self, client):
"""Dossier verification: an Office read without a tenant is 400 —
/office-tatlock-user carries no identity_required, and this still
fails before reaching the handler."""
settings.office_service_key = "office-secret"
response = client.get(
"/office-tatlock-user",
headers={
"Authorization": "Bearer office-secret",
"X-Office-Consumer": VALID_CONSUMER,
},
)
assert response.status_code == 400
assert response.json()["detail"]["code"] == "tenant_required"
@pytest.mark.parametrize("tenant", ["Jeroen", "-abc", "a b"])
def test_malformed_tenant_is_400(self, client, tenant):
settings.office_service_key = "office-secret"
response = client.get(
"/office-tatlock-user",
headers={
"Authorization": "Bearer office-secret",
"X-Office-Consumer": VALID_CONSUMER,
"X-Office-Tenant": tenant,
},
)
assert response.status_code == 400
assert response.json()["detail"]["code"] == "tenant_invalid"
def test_tenant_allows_underscores(self, client):
"""tatlock's own tenant ids contain underscores (e.g. llm_tester)."""
settings.office_service_key = "office-secret"
response = client.get(
"/office-tatlock-user",
headers={
"Authorization": "Bearer office-secret",
"X-Office-Consumer": VALID_CONSUMER,
"X-Office-Tenant": "llm_tester",
},
)
assert response.status_code == 200
assert response.json()["tenant"] == "llm_tester"
def test_missing_both_reports_consumer_first(self, client):
"""Lock down the check order: consumer before tenant when both are absent."""
settings.office_service_key = "office-secret"
response = client.get(
"/office-tatlock-user", headers={"Authorization": "Bearer office-secret"}
)
assert response.status_code == 400
assert response.json()["detail"]["code"] == "consumer_required"
class TestTatlockAndSchedulerKeys:
def test_tatlock_key_resolves_and_ignores_identity_header(self, client):
settings.tatlock_service_key = "tatlock-secret"
response = client.get(
"/office-tatlock-user",
headers={
"Authorization": "Bearer tatlock-secret",
"X-Office-Identity": "jeroen",
"X-Office-Consumer": VALID_CONSUMER,
"X-Office-Tenant": VALID_TENANT,
},
)
assert response.status_code == 200
assert response.json() == {"kind": "tatlock", "identity": "tatlock", "tenant": None}
def test_scheduler_key_forbidden_on_a_route_that_excludes_it(self, client):
settings.scheduler_service_key = "scheduler-secret"
response = client.get(
"/office-tatlock-user", headers={"Authorization": "Bearer scheduler-secret"}
)
assert response.status_code == 403
assert response.json()["detail"]["code"] == "forbidden"
def test_scheduler_key_resolves_and_ignores_consumer_and_tenant_headers(self, client):
"""Every kind but 'office' ignores X-Office-Consumer/X-Office-Tenant outright."""
settings.scheduler_service_key = "scheduler-secret"
response = client.get(
"/all-kinds",
headers={
"Authorization": "Bearer scheduler-secret",
"X-Office-Consumer": VALID_CONSUMER,
"X-Office-Tenant": VALID_TENANT,
},
)
assert response.status_code == 200
assert response.json() == {"kind": "scheduler", "identity": "scheduler", "tenant": None}
class TestServiceKeyBypassesJWT:
def test_matched_service_key_never_calls_get_current_user(self, client):
"""Static keys are checked before any JWT parsing (dossier T-1 §1)."""
settings.office_service_key = "office-secret"
with patch("src.domains.auth.callers.get_current_user") as mock_get_current_user:
mock_get_current_user.side_effect = AssertionError(
"get_current_user must not run when a service key matched"
)
response = client.get(
"/office-tatlock-user",
headers={
"Authorization": "Bearer office-secret",
"X-Office-Identity": "jeroen",
"X-Office-Consumer": VALID_CONSUMER,
"X-Office-Tenant": VALID_TENANT,
},
)
assert response.status_code == 200
mock_get_current_user.assert_not_called()
def test_unmatched_token_falls_through_to_get_current_user(self, client):
"""A wrong-looking token with no matching key must still reach the JWT path."""
settings.office_service_key = "office-secret"
response = client.get(
"/office-tatlock-user", headers={"Authorization": "Bearer not-the-office-key"}
)
# OIDC disabled -> get_current_user ignores the bad bearer and returns the local user.
assert response.status_code == 200
assert response.json() == {"kind": "user", "identity": "local", "tenant": None}
class TestUserTenant:
"""A JWT user's tenant comes only from their own token's `tatlock_user`
claim — never from a header (T-4). This is the tenant-spoofing guard."""
def test_user_tenant_comes_from_jwt_claim(self, client):
with patch("src.domains.auth.callers.get_current_user") as mock_get_current_user:
mock_get_current_user.return_value = {
"preferred_username": "jpmschweitzer",
"tatlock_user": "jpmschweitzer",
}
response = client.get(
"/office-tatlock-user",
headers={"Authorization": "Bearer some-jwt"},
)
assert response.status_code == 200
assert response.json() == {"kind": "user", "identity": "jpmschweitzer", "tenant": "jpmschweitzer"}
def test_user_tenant_ignores_x_office_tenant_header_spoofing_attempt(self, client):
"""The header must not be able to put a non-Office caller into a tenant:
a JWT carrying the real claim wins over a forged header every time."""
with patch("src.domains.auth.callers.get_current_user") as mock_get_current_user:
mock_get_current_user.return_value = {
"preferred_username": "jpmschweitzer",
"tatlock_user": "jpmschweitzer",
}
response = client.get(
"/office-tatlock-user",
headers={
"Authorization": "Bearer some-jwt",
"X-Office-Tenant": "someone-elses-tenant",
},
)
assert response.status_code == 200
assert response.json() == {"kind": "user", "identity": "jpmschweitzer", "tenant": "jpmschweitzer"}
def test_user_tenant_is_none_when_jwt_carries_no_claim_even_with_header_sent(self, client):
with patch("src.domains.auth.callers.get_current_user") as mock_get_current_user:
mock_get_current_user.return_value = {"preferred_username": "jpmschweitzer"}
response = client.get(
"/office-tatlock-user",
headers={
"Authorization": "Bearer some-jwt",
"X-Office-Tenant": "someone-elses-tenant",
},
)
assert response.status_code == 200
assert response.json() == {"kind": "user", "identity": "jpmschweitzer", "tenant": None}
+4 -8
View File
@@ -1,6 +1,6 @@
"""Tests for config module."""
import pytest
from src.config import (
from src.shared.config import (
__version__,
Settings,
get_settings,
@@ -89,10 +89,6 @@ class TestGetSettings:
settings2 = get_settings()
assert settings1 is settings2
def test_model_aliases_property(self):
"""Model aliases property should return dict."""
settings = get_settings()
aliases = settings.model_aliases
assert isinstance(aliases, dict)
assert "gpt-3.5-turbo" in aliases
assert "gpt-4" in aliases
# Removed: test_model_aliases_property. Settings.model_aliases mapped
# gpt-3.5-turbo and gpt-4 onto local models, and was deleted along with the
# Ollama integration in c1f16d4. The test outlived the feature it covered.
+159
View File
@@ -0,0 +1,159 @@
"""Tests for environment service and schemas."""
import pytest
from datetime import datetime
class TestEnvironmentService:
"""Test EnvironmentService methods."""
def test_aqi_to_quality_good(self):
"""AQI 0-50 should return Good."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
assert service._aqi_to_quality(0) == "Good"
assert service._aqi_to_quality(25) == "Good"
assert service._aqi_to_quality(50) == "Good"
def test_aqi_to_quality_moderate(self):
"""AQI 51-100 should return Moderate."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
assert service._aqi_to_quality(51) == "Moderate"
assert service._aqi_to_quality(75) == "Moderate"
assert service._aqi_to_quality(100) == "Moderate"
def test_aqi_to_quality_unhealthy_sensitive(self):
"""AQI 101-150 should return Unhealthy for Sensitive Groups."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
assert service._aqi_to_quality(101) == "Unhealthy for Sensitive Groups"
assert service._aqi_to_quality(150) == "Unhealthy for Sensitive Groups"
def test_aqi_to_quality_unhealthy(self):
"""AQI 151-200 should return Unhealthy."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
assert service._aqi_to_quality(151) == "Unhealthy"
assert service._aqi_to_quality(200) == "Unhealthy"
def test_aqi_to_quality_very_unhealthy(self):
"""AQI 201-300 should return Very Unhealthy."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
assert service._aqi_to_quality(201) == "Very Unhealthy"
assert service._aqi_to_quality(300) == "Very Unhealthy"
def test_aqi_to_quality_hazardous(self):
"""AQI >300 should return Hazardous."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
assert service._aqi_to_quality(301) == "Hazardous"
assert service._aqi_to_quality(500) == "Hazardous"
def test_parse_weather_with_valid_data(self):
"""Parse weather should return WeatherData for valid input."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
raw = {
"temperature": 15.5,
"conditions": "Cloudy",
"humidity": 72,
"location": "Rotterdam",
}
result = service._parse_weather(raw)
assert result is not None
assert result.temperature == 15.5
assert result.conditions == "Cloudy"
assert result.humidity == 72
def test_parse_weather_with_none(self):
"""Parse weather should return None for None input."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
result = service._parse_weather(None)
assert result is None
def test_parse_forecast_with_list(self):
"""Parse forecast should handle list format."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
raw = [
{"date": "2026-01-06", "high": 12, "low": 5, "conditions": "Cloudy"},
{"date": "2026-01-07", "high": 14, "low": 6, "conditions": "Sunny"},
]
result = service._parse_forecast(raw)
assert result is not None
assert len(result) == 2
assert result[0].date == "2026-01-06"
assert result[0].high == 12
def test_parse_forecast_with_dict(self):
"""Parse forecast should handle dict with days key."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
raw = {
"days": [
{"date": "2026-01-06", "high": 12, "low": 5, "conditions": "Cloudy"},
]
}
result = service._parse_forecast(raw)
assert result is not None
assert len(result) == 1
def test_parse_sun_times_with_strings(self):
"""Parse sun times should handle ISO datetime strings."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
raw = {
"sunrise": "2026-01-06T08:45:00",
"sunset": "2026-01-06T16:50:00",
}
result = service._parse_sun_times(raw)
assert result is not None
assert result.sunrise.hour == 8
assert result.sunrise.minute == 45
assert result.sunset.hour == 16
assert result.daylight_minutes == 485
def test_parse_air_quality_with_int(self):
"""Parse air quality should handle simple integer AQI."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
result = service._parse_air_quality(42)
assert result is not None
assert result.aqi == 42
assert result.quality == "Good"
def test_parse_air_quality_with_dict(self):
"""Parse air quality should handle dict format."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
raw = {
"aqi": 75,
"pm25": 8.5,
"pm10": 15,
}
result = service._parse_air_quality(raw)
assert result is not None
assert result.aqi == 75
assert result.quality == "Moderate"
assert result.pm25 == 8.5
+19 -87
View File
@@ -80,24 +80,24 @@ class TestFullHealthCheck:
"""Test /health/full endpoint."""
@patch("src.shared.database.Database.health_check")
@patch("src.models.ollama_client.get_ollama_client")
def test_full_health_returns_503_when_unhealthy(self, mock_get_ollama, mock_db_health, client):
"""Full health should return 503 when Ollama unhealthy."""
mock_client = AsyncMock()
mock_client.health_check.return_value = False
mock_get_ollama.return_value = mock_client
def test_full_health_returns_200_when_healthy(self, mock_db_health, client):
"""Full health should return 200 when database is healthy."""
mock_db_health.return_value = True
response = client.get("/health/full")
assert response.status_code == 200
@patch("src.shared.database.Database.health_check")
def test_full_health_returns_503_when_unhealthy(self, mock_db_health, client):
"""Full health should return 503 when database is unhealthy."""
mock_db_health.return_value = False
response = client.get("/health/full")
assert response.status_code == 503
@patch("src.shared.database.Database.health_check")
@patch("src.models.ollama_client.get_ollama_client")
def test_full_health_returns_components_status(self, mock_get_ollama, mock_db_health, client):
def test_full_health_returns_components_status(self, mock_db_health, client):
"""Full health should return component status."""
mock_client = AsyncMock()
mock_client.health_check.return_value = False
mock_get_ollama.return_value = mock_client
mock_db_health.return_value = True
response = client.get("/health/full")
@@ -105,63 +105,32 @@ class TestFullHealthCheck:
assert "status" in data
assert "components" in data
assert "ollama" in data["components"]
assert "database" in data["components"]
assert "response_time_ms" in data
@patch("src.shared.database.Database.health_check")
@patch("src.models.ollama_client.get_ollama_client")
def test_full_health_handles_list_models_error(self, mock_get_ollama, mock_db_health, client):
"""Full health should handle list_models errors."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_client.list_models.side_effect = Exception("Connection error")
mock_get_ollama.return_value = mock_client
mock_db_health.return_value = True
def test_full_health_handles_database_error(self, mock_db_health, client):
"""Full health should handle database errors gracefully."""
mock_db_health.side_effect = Exception("Connection error")
response = client.get("/health/full")
data = response.json()
# Should report error in component status
assert "ollama" in data["components"]
@patch("src.shared.database.Database.health_check")
@patch("src.models.ollama_client.get_ollama_client")
def test_full_health_handles_health_check_exception(self, mock_get_ollama, mock_db_health, client):
"""Full health should handle health check exceptions gracefully."""
mock_client = AsyncMock()
# Return False instead of raising exception to test unhealthy path
mock_client.health_check.return_value = False
mock_get_ollama.return_value = mock_client
mock_db_health.return_value = True
response = client.get("/health/full")
# Should return 503 for unhealthy
assert response.status_code == 503
data = response.json()
assert data["status"] == "unhealthy"
assert "error" in data["components"]["database"]
class TestDiagnosticsEndpoint:
"""Test /health/diagnostics endpoint."""
@patch("src.models.ollama_client.get_ollama_client")
def test_diagnostics_returns_200(self, mock_get_ollama, client):
def test_diagnostics_returns_200(self, client):
"""Diagnostics should return 200."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics")
assert response.status_code == 200
@patch("src.models.ollama_client.get_ollama_client")
def test_diagnostics_returns_service_info(self, mock_get_ollama, client):
def test_diagnostics_returns_service_info(self, client):
"""Diagnostics should return service information."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics")
data = response.json()
@@ -169,54 +138,17 @@ class TestDiagnosticsEndpoint:
assert "name" in data["service"]
assert "version" in data["service"]
@patch("src.models.ollama_client.get_ollama_client")
def test_diagnostics_returns_components(self, mock_get_ollama, client):
"""Diagnostics should return component details."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics")
data = response.json()
assert "components" in data
assert "ollama" in data["components"]
@patch("src.models.ollama_client.get_ollama_client")
def test_diagnostics_returns_configuration(self, mock_get_ollama, client):
def test_diagnostics_returns_configuration(self, client):
"""Diagnostics should return configuration info."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics")
data = response.json()
assert "configuration" in data
@patch("src.models.ollama_client.get_ollama_client")
def test_diagnostics_returns_response_time(self, mock_get_ollama, client):
def test_diagnostics_returns_response_time(self, client):
"""Diagnostics should return response time."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics")
data = response.json()
assert "response_time_ms" in data
assert isinstance(data["response_time_ms"], int)
@patch("src.models.ollama_client.get_ollama_client")
def test_diagnostics_handles_ollama_error(self, mock_get_ollama, client):
"""Diagnostics should handle Ollama connection errors."""
mock_client = AsyncMock()
mock_client.health_check.side_effect = Exception("Connection refused")
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics")
data = response.json()
# Should still return 200 with error info
assert response.status_code == 200
assert "error" in data["components"]["ollama"]
+46
View File
@@ -3,6 +3,7 @@ import pytest
from fastapi.testclient import TestClient
from unittest.mock import patch, AsyncMock
from src.domains.auth.oidc import oidc_config
from src.main import app
@@ -12,6 +13,51 @@ def client():
return TestClient(app)
class TestHousekeepingReadsAreLocked:
"""
T-1 / D-68: api.schweitz.net carries no forward-auth, so an
unauthenticated read used to reach Home Assistant directly. These
assert the caller gate in front of the GET routes, independent of
whatever Home Assistant itself returns.
"""
@pytest.fixture(autouse=True)
def oidc_enabled(self):
original = (oidc_config.enabled, oidc_config.issuers, oidc_config.audiences)
oidc_config.configure(enabled=True, issuers=[], audiences=[])
yield
oidc_config.enabled, oidc_config.issuers, oidc_config.audiences = original
def test_no_auth_is_401_on_devices(self, client):
response = client.get("/housekeeping/devices")
assert response.status_code == 401
def test_no_auth_is_401_on_health(self, client):
response = client.get("/housekeeping/health")
assert response.status_code == 401
@patch("src.domains.housekeeping.controller.get_homeassistant_client")
def test_tatlock_key_passes_auth_on_devices(self, mock_get_ha, client):
"""The one caller this lock exists to keep working (dossier §2)."""
from src.shared.config import get_settings
settings = get_settings()
original = settings.tatlock_service_key
settings.tatlock_service_key = "test-tatlock-key"
try:
mock_client = AsyncMock()
mock_client.get_states.return_value = []
mock_get_ha.return_value = mock_client
response = client.get(
"/housekeeping/devices",
headers={"Authorization": "Bearer test-tatlock-key"},
)
assert response.status_code == 200
finally:
settings.tatlock_service_key = original
@pytest.fixture
def mock_ha_client():
"""Create a mock Home Assistant client."""
+114 -62
View File
@@ -1,118 +1,173 @@
"""Tests for OIDC authentication module."""
import pytest
from unittest.mock import patch, MagicMock, AsyncMock
from unittest.mock import patch, MagicMock
from fastapi import HTTPException
from src.auth.oidc import OIDCConfig, oidc_config, get_jwks, get_current_user
from src.auth.oidc import (
OIDCConfig,
oidc_config,
get_jwks_for_issuer,
get_current_user,
_jwks_cache,
)
@pytest.fixture(autouse=True)
def clear_jwks_cache():
"""
The JWKS cache is module-level state, so a fetch in one test would satisfy
the next one and hide a regression. Clearing on both sides keeps the tests
order-independent.
"""
_jwks_cache.clear()
yield
_jwks_cache.clear()
class TestOIDCConfig:
"""Test OIDCConfig class."""
def test_init_defaults(self):
"""Config should initialize with disabled state."""
"""Config should initialize disabled with no issuers or audiences."""
config = OIDCConfig()
assert config.enabled is False
assert config.issuer == ""
assert config.audience == ""
assert config.jwks_uri == ""
assert config.issuers == []
assert config.audiences == []
def test_configure_sets_values(self):
"""configure should set all values."""
config = OIDCConfig()
config.configure(
enabled=True,
issuer="https://auth.example.com",
audience="core-api"
issuers=["https://auth.example.com"],
audiences=["core-api"],
)
assert config.enabled is True
assert config.issuer == "https://auth.example.com"
assert config.audience == "core-api"
assert config.jwks_uri == "https://auth.example.com/jwks/"
assert config.issuers == ["https://auth.example.com"]
assert config.audiences == ["core-api"]
def test_configure_strips_trailing_slash(self):
"""configure should handle trailing slash in issuer."""
"""configure should normalise issuers by dropping the trailing slash."""
config = OIDCConfig()
config.configure(
enabled=True,
issuer="https://auth.example.com/",
audience="core-api"
issuers=["https://auth.example.com/"],
audiences=["core-api"],
)
assert config.jwks_uri == "https://auth.example.com/jwks/"
assert config.issuers == ["https://auth.example.com"]
def test_configure_accepts_multiple_issuers(self):
"""The point of the multi-issuer change: more than one is allowed."""
config = OIDCConfig()
config.configure(
enabled=True,
issuers=["https://a.example.com/", "https://b.example.com"],
audiences=["core-api", "other"],
)
assert config.issuers == ["https://a.example.com", "https://b.example.com"]
assert config.audiences == ["core-api", "other"]
def test_get_jwks_uri_derives_from_issuer(self):
"""The JWKS URI is derived per issuer rather than configured."""
config = OIDCConfig()
assert config.get_jwks_uri("https://auth.example.com") == "https://auth.example.com/jwks/"
assert config.get_jwks_uri("https://auth.example.com/") == "https://auth.example.com/jwks/"
def test_is_valid_issuer_only_accepts_configured(self):
"""
An unconfigured issuer must be rejected. This is the security-relevant
half of multi-issuer support: accepting any issuer would let a token
from an unrelated identity provider through.
"""
config = OIDCConfig()
config.configure(
enabled=True,
issuers=["https://auth.example.com"],
audiences=["core-api"],
)
assert config.is_valid_issuer("https://auth.example.com") is True
assert config.is_valid_issuer("https://auth.example.com/") is True
assert config.is_valid_issuer("https://evil.example.com") is False
class TestGetJWKS:
"""Test get_jwks function."""
class TestGetJWKSForIssuer:
"""Test get_jwks_for_issuer function."""
ISSUER = "https://auth.example.com"
def test_returns_empty_when_disabled(self):
"""get_jwks should return empty dict when OIDC disabled."""
# Save original state
"""Should return an empty dict when OIDC is disabled."""
original_enabled = oidc_config.enabled
try:
oidc_config.enabled = False
# Clear the cache
get_jwks.cache_clear()
result = get_jwks()
assert result == {}
assert get_jwks_for_issuer(self.ISSUER) == {}
finally:
# Restore original state
oidc_config.enabled = original_enabled
get_jwks.cache_clear()
@patch("src.auth.oidc.httpx.get")
def test_fetches_jwks_when_enabled(self, mock_get):
"""get_jwks should fetch JWKS when enabled."""
# Save original state
"""Should fetch from the issuer's derived JWKS URI."""
original_enabled = oidc_config.enabled
original_jwks_uri = oidc_config.jwks_uri
try:
oidc_config.enabled = True
oidc_config.jwks_uri = "https://auth.example.com/jwks/"
get_jwks.cache_clear()
mock_response = MagicMock()
mock_response.json.return_value = {"keys": [{"kid": "test"}]}
mock_response.raise_for_status = MagicMock()
mock_response.json.return_value = {"keys": [{"kid": "abc"}]}
mock_get.return_value = mock_response
result = get_jwks()
result = get_jwks_for_issuer(self.ISSUER)
assert "keys" in result
assert result == {"keys": [{"kid": "abc"}]}
mock_get.assert_called_once()
assert mock_get.call_args[0][0] == f"{self.ISSUER}/jwks/"
finally:
oidc_config.enabled = original_enabled
oidc_config.jwks_uri = original_jwks_uri
get_jwks.cache_clear()
@patch("src.auth.oidc.httpx.get")
def test_raises_exception_on_error(self, mock_get):
"""get_jwks should raise HTTPException on fetch error."""
# Save original state
def test_caches_per_issuer(self, mock_get):
"""
A second call for the same issuer must not refetch, and a different
issuer must. Caching by issuer is the behaviour the multi-issuer change
introduced, and a shared cache would have served one issuer's keys for
another — which would be a verification bypass, not just a slow path.
"""
original_enabled = oidc_config.enabled
original_jwks_uri = oidc_config.jwks_uri
try:
oidc_config.enabled = True
oidc_config.jwks_uri = "https://auth.example.com/jwks/"
get_jwks.cache_clear()
mock_response = MagicMock()
mock_response.json.return_value = {"keys": []}
mock_get.return_value = mock_response
mock_get.side_effect = Exception("Connection error")
get_jwks_for_issuer(self.ISSUER)
get_jwks_for_issuer(self.ISSUER + "/") # same issuer, normalised
assert mock_get.call_count == 1
get_jwks_for_issuer("https://other.example.com")
assert mock_get.call_count == 2
finally:
oidc_config.enabled = original_enabled
@patch("src.auth.oidc.httpx.get")
def test_raises_503_on_fetch_error(self, mock_get):
"""A JWKS fetch failure should surface as 503, not leak the cause."""
original_enabled = oidc_config.enabled
try:
oidc_config.enabled = True
mock_get.side_effect = Exception("Connection failed")
with pytest.raises(HTTPException) as exc_info:
get_jwks()
get_jwks_for_issuer(self.ISSUER)
assert exc_info.value.status_code == 503
finally:
oidc_config.enabled = original_enabled
oidc_config.jwks_uri = original_jwks_uri
get_jwks.cache_clear()
class TestGetCurrentUser:
@@ -121,9 +176,7 @@ class TestGetCurrentUser:
@pytest.mark.asyncio
async def test_returns_none_when_disabled(self):
"""get_current_user should return None when OIDC disabled."""
# Save original state
original_enabled = oidc_config.enabled
try:
oidc_config.enabled = False
@@ -136,9 +189,7 @@ class TestGetCurrentUser:
@pytest.mark.asyncio
async def test_raises_401_when_enabled_without_token(self):
"""get_current_user should raise 401 when enabled but no token."""
# Save original state
original_enabled = oidc_config.enabled
try:
oidc_config.enabled = True
@@ -157,11 +208,12 @@ class TestOIDCGlobalConfig:
"""oidc_config should be an OIDCConfig instance."""
assert isinstance(oidc_config, OIDCConfig)
def test_global_config_starts_disabled(self):
"""oidc_config should start disabled by default."""
# This tests the initial state before any configure() is called
# The actual state depends on app configuration
assert hasattr(oidc_config, 'enabled')
assert hasattr(oidc_config, 'issuer')
assert hasattr(oidc_config, 'audience')
assert hasattr(oidc_config, 'jwks_uri')
def test_global_config_exposes_the_multi_issuer_surface(self):
"""
Asserts the shape rather than the values, since the live state depends
on app configuration. These four are what callers depend on.
"""
assert hasattr(oidc_config, "enabled")
assert hasattr(oidc_config, "issuers")
assert hasattr(oidc_config, "audiences")
assert callable(oidc_config.get_jwks_uri)
-307
View File
@@ -1,307 +0,0 @@
"""Tests for Ollama client."""
import pytest
from unittest.mock import patch, AsyncMock, MagicMock
import json
from src.models.ollama_client import OllamaClient, get_ollama_client, close_ollama_client
class TestOllamaClientInit:
"""Test OllamaClient initialization."""
@patch("src.models.ollama_client.settings")
def test_uses_settings_defaults(self, mock_settings):
"""Client should use settings for defaults."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 60
client = OllamaClient()
assert client.base_url == "http://ollama:11434"
assert client.timeout == 60
@patch("src.models.ollama_client.settings")
def test_creates_http_client(self, mock_settings):
"""Client should create httpx AsyncClient."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
assert client.client is not None
class TestOllamaClientClose:
"""Test client close functionality."""
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_close_closes_client(self, mock_settings):
"""close should close the HTTP client."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
with patch.object(client.client, "aclose", new_callable=AsyncMock) as mock_close:
await client.close()
mock_close.assert_called_once()
class TestOllamaClientResolveModel:
"""Test model resolution."""
@patch("src.models.ollama_client.settings")
def test_resolves_aliased_model(self, mock_settings):
"""resolve_model should map alias to actual model."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
mock_settings.model_aliases = {"gpt-3.5-turbo": "gemma:7b"}
client = OllamaClient()
result = client.resolve_model("gpt-3.5-turbo")
assert result == "gemma:7b"
@patch("src.models.ollama_client.settings")
def test_returns_original_if_no_alias(self, mock_settings):
"""resolve_model should return original if no alias found."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
mock_settings.model_aliases = {}
client = OllamaClient()
result = client.resolve_model("llama2")
assert result == "llama2"
class TestOllamaClientHealthCheck:
"""Test health check functionality."""
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_health_check_returns_true_on_200(self, mock_settings):
"""Health check should return True when Ollama responds 200."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
mock_response = MagicMock()
mock_response.status_code = 200
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.return_value = mock_response
result = await client.health_check()
assert result is True
mock_get.assert_called_once()
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_health_check_returns_false_on_error(self, mock_settings):
"""Health check should return False on connection error."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.side_effect = Exception("Connection refused")
result = await client.health_check()
assert result is False
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_health_check_returns_false_on_non_200(self, mock_settings):
"""Health check should return False on non-200 status."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
mock_response = MagicMock()
mock_response.status_code = 500
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.return_value = mock_response
result = await client.health_check()
assert result is False
class TestOllamaClientListModels:
"""Test list models functionality."""
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_list_models_returns_dict(self, mock_settings):
"""list_models should return dict with models."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
models_data = {
"models": [
{"name": "llama2", "size": 1000000},
{"name": "gemma:7b", "size": 2000000}
]
}
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = models_data
mock_response.raise_for_status = MagicMock()
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.return_value = mock_response
result = await client.list_models()
assert result == models_data
assert len(result["models"]) == 2
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_list_models_raises_on_error(self, mock_settings):
"""list_models should raise on error."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.side_effect = Exception("Connection error")
with pytest.raises(Exception):
await client.list_models()
class TestOllamaClientGenerateNonStreaming:
"""Test non-streaming generation."""
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_generate_non_streaming_returns_response(self, mock_settings):
"""generate_non_streaming should return response dict."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
mock_settings.model_aliases = {}
client = OllamaClient()
response_data = {
"message": {"content": "Hello! How can I help?"},
"prompt_eval_count": 10,
"eval_count": 20
}
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = response_data
mock_response.raise_for_status = MagicMock()
with patch.object(client.client, "post", new_callable=AsyncMock) as mock_post:
mock_post.return_value = mock_response
result = await client.generate_non_streaming("llama2", "Hello")
assert result["response"] == "Hello! How can I help?"
assert result["tokens"]["prompt"] == 10
assert result["tokens"]["completion"] == 20
assert result["tokens"]["total"] == 30
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_generate_non_streaming_includes_max_tokens(self, mock_settings):
"""generate_non_streaming should include max_tokens in payload."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
mock_settings.model_aliases = {}
client = OllamaClient()
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {"message": {"content": "Hi"}}
mock_response.raise_for_status = MagicMock()
with patch.object(client.client, "post", new_callable=AsyncMock) as mock_post:
mock_post.return_value = mock_response
await client.generate_non_streaming("llama2", "Hello", max_tokens=100)
call_args = mock_post.call_args
assert call_args[1]["json"]["options"]["num_predict"] == 100
class TestOllamaClientGenerateStreaming:
"""Test streaming generation."""
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_generate_streaming_yields_content(self, mock_settings):
"""generate_streaming should yield content chunks."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
mock_settings.model_aliases = {}
client = OllamaClient()
# Create mock streaming response
async def mock_aiter_lines():
yield json.dumps({"message": {"content": "Hello"}})
yield json.dumps({"message": {"content": " world"}})
yield json.dumps({"done": True})
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.raise_for_status = MagicMock()
mock_response.aiter_lines = mock_aiter_lines
mock_stream_context = AsyncMock()
mock_stream_context.__aenter__.return_value = mock_response
mock_stream_context.__aexit__.return_value = None
with patch.object(client.client, "stream", return_value=mock_stream_context):
chunks = []
async for chunk in client.generate_streaming("llama2", "Hi"):
chunks.append(chunk)
assert "Hello" in chunks
assert " world" in chunks
class TestOllamaClientSingleton:
"""Test singleton pattern."""
@patch("src.models.ollama_client.settings")
def test_get_ollama_client_returns_same_instance(self, mock_settings):
"""get_ollama_client should return singleton."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
import src.models.ollama_client as module
module._ollama_client = None
client1 = get_ollama_client()
client2 = get_ollama_client()
assert client1 is client2
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_close_ollama_client_clears_singleton(self, mock_settings):
"""close_ollama_client should clear the singleton."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
import src.models.ollama_client as module
module._ollama_client = None
client = get_ollama_client()
with patch.object(client.client, "aclose", new_callable=AsyncMock):
await close_ollama_client()
assert module._ollama_client is None
+917
View File
@@ -0,0 +1,917 @@
"""
Tests for the Scribe domain (dossier core-api T-5 / D-66, D-68, D-73)
against `core_api_test`.
Follows tests/test_book.py's TestClient pattern: `get_async_session` is
overridden to the test database for every test, building a fresh
engine/session per call because TestClient drives the ASGI app on a
different event loop than pytest-asyncio's fixture setup.
"""
import pytest
from fastapi.testclient import TestClient
from src.domains.auth.callers import _IDENTITY_RE, RESERVED_IDENTITIES
from src.domains.scribe.models import ADDRESS_NAME_PATTERN, RESERVED_ADDRESS_NAMES
from src.main import app
from src.shared.config import get_settings
from src.shared.database import get_async_session
settings = get_settings()
OFFICE_KEY = "test-office-key"
TENANT = "jpmschweitzer"
OTHER_TENANT = "llm_tester"
def test_address_name_rule_matches_callers_identity_rule():
"""models.py deliberately duplicates callers.py's identity shape rather
than importing its private constants (see models.py's comment) — assert
the two never drift apart instead."""
assert ADDRESS_NAME_PATTERN == _IDENTITY_RE.pattern
assert RESERVED_ADDRESS_NAMES == RESERVED_IDENTITIES
@pytest.fixture(autouse=True)
def office_key():
"""Every Scribe test authenticates as the office caller unless it says otherwise."""
original = settings.office_service_key
settings.office_service_key = OFFICE_KEY
yield
settings.office_service_key = original
@pytest.fixture
def client(clean_scribe_tables, test_database_url):
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from sqlalchemy.pool import NullPool
asyncpg_url = test_database_url.replace("postgresql://", "postgresql+asyncpg://", 1)
async def _override():
engine = create_async_engine(asyncpg_url, poolclass=NullPool)
session_factory = async_sessionmaker(bind=engine, expire_on_commit=False)
async with session_factory() as session:
yield session
await engine.dispose()
app.dependency_overrides[get_async_session] = _override
yield TestClient(app)
app.dependency_overrides.pop(get_async_session, None)
def office_headers(identity: str, consumer: str, tenant: str = TENANT) -> dict:
return {
"Authorization": f"Bearer {OFFICE_KEY}",
"X-Office-Identity": identity,
"X-Office-Consumer": consumer,
"X-Office-Tenant": tenant,
}
def register(client, identity: str, consumer: str, tenant: str = TENANT):
return client.post(
"/scribe/addresses",
json={"name": identity},
headers=office_headers(identity, consumer, tenant),
)
# =============================================================================
# Caller matrix — a representative slice; the full matrix is test_callers.py
# and test_book.py's TestCallerMatrixOnBookRoutes already exercises
# require_caller itself end to end on a sibling domain.
# =============================================================================
class TestCallerMatrixOnScribeRoutes:
def test_no_auth_is_401_on_scribe_routes(self, client):
from src.domains.auth.oidc import oidc_config
oidc_config.configure(enabled=True, issuers=[], audiences=[])
try:
response = client.post("/scribe/addresses", json={"name": "jeroen"})
finally:
oidc_config.configure(enabled=False, issuers=[], audiences=[])
assert response.status_code == 401
def test_office_write_without_identity_is_400(self, client):
response = client.post(
"/scribe/addresses",
json={"name": "jeroen"},
headers={
"Authorization": f"Bearer {OFFICE_KEY}",
"X-Office-Consumer": "office-claude",
"X-Office-Tenant": TENANT,
},
)
assert response.status_code == 400
assert response.json()["detail"]["code"] == "identity_required"
def test_office_write_without_tenant_is_400(self, client):
response = client.post(
"/scribe/addresses",
json={"name": "jeroen"},
headers={
"Authorization": f"Bearer {OFFICE_KEY}",
"X-Office-Identity": "jeroen",
"X-Office-Consumer": "office-claude",
},
)
assert response.status_code == 400
assert response.json()["detail"]["code"] == "tenant_required"
def test_scheduler_key_is_403_on_scribe_addresses(self, client):
settings.scheduler_service_key = "test-scheduler-key"
try:
response = client.post(
"/scribe/addresses",
json={"name": "jeroen"},
headers={"Authorization": "Bearer test-scheduler-key"},
)
finally:
settings.scheduler_service_key = ""
assert response.status_code == 403
assert response.json()["detail"]["code"] == "forbidden"
# =============================================================================
# register / list addresses
# =============================================================================
class TestRegisterAndListAddresses:
def test_register_creates_address_owned_by_caller(self, client):
response = register(client, "jeroen", "office-claude")
assert response.status_code == 200
data = response.json()
assert data["name"] == "jeroen"
assert data["owner_consumer"] == "office-claude"
assert data["created_by"] == "jeroen"
assert data["owned"] is True
def test_register_is_idempotent_for_the_same_consumer(self, client):
first = register(client, "jeroen", "office-claude")
second = register(client, "jeroen", "office-claude")
assert first.status_code == 200
assert second.status_code == 200
assert second.json()["owned"] is True
def test_register_by_another_consumer_is_409_address_taken(self, client):
register(client, "jeroen", "office-claude")
response = register(client, "jeroen", "office-gemini")
assert response.status_code == 409
assert response.json()["detail"]["code"] == "address_taken"
@pytest.mark.parametrize("name", ["book", "office", "scheduler", "tatlock", "group-home"])
def test_register_reserved_or_group_name_is_400(self, client, name):
"""The registering identity ("jeroen") is valid and distinct from
the name being registered, so this exercises the service's own
reserved/group- check, not the caller header's identity check."""
response = client.post(
"/scribe/addresses",
json={"name": name},
headers=office_headers("jeroen", "office-claude"),
)
assert response.status_code == 400
assert response.json()["detail"]["code"] == "address_invalid"
@pytest.mark.parametrize("name", ["Jeroen", "-abc", "a_b", ""])
def test_register_malformed_name_is_422(self, client, name):
"""Shape alone is a pydantic concern (422), not a ScribeError (400)."""
response = client.post(
"/scribe/addresses",
json={"name": name},
headers=office_headers("jeroen", "office-claude"),
)
assert response.status_code == 422
def test_list_addresses_marks_ownership(self, client):
register(client, "jeroen", "office-claude")
register(client, "tower-control", "office-gemini")
as_claude = client.get(
"/scribe/addresses", headers=office_headers("jeroen", "office-claude")
).json()
owned = {a["name"]: a["owned"] for a in as_claude["addresses"]}
assert owned == {"jeroen": True, "tower-control": False}
def test_list_addresses_is_scoped_to_the_callers_tenant(self, client):
register(client, "jeroen", "office-claude", tenant=TENANT)
register(client, "jeroen", "office-claude", tenant=OTHER_TENANT)
as_other_tenant = client.get(
"/scribe/addresses", headers=office_headers("jeroen", "office-claude", OTHER_TENANT)
).json()
assert len(as_other_tenant["addresses"]) == 1
# =============================================================================
# The flow (dossier verification #2): register, send, mailbox, mark,
# history, find, from shows identity and consumer.
# =============================================================================
class TestMessagingFlow:
def test_full_flow(self, client):
register(client, "jeroen", "office-claude")
register(client, "tower-control", "office-gemini")
sent = client.post(
"/scribe/messages",
json={"to": ["tower-control"], "subject": "status", "body": "all green"},
headers=office_headers("jeroen", "office-claude"),
)
assert sent.status_code == 201
message = sent.json()
assert message["from_identity"] == "jeroen"
assert message["from_consumer"] == "office-claude"
assert message["topics"] == ["tower-control"]
mailbox = client.get(
"/scribe/mailbox", headers=office_headers("tower-control", "office-gemini")
).json()
assert mailbox["total"] == 1
entry = mailbox["entries"][0]
assert entry["read_at"] is None
assert entry["message"]["id"] == message["id"]
assert entry["message"]["body"] == "all green"
marked = client.patch(
f"/scribe/mailbox/{entry['id']}",
json={"read": True},
headers=office_headers("tower-control", "office-gemini"),
)
assert marked.status_code == 200
assert marked.json()["read_at"] is not None
mailbox_after = client.get(
"/scribe/mailbox", headers=office_headers("tower-control", "office-gemini")
).json()
assert mailbox_after["entries"][0]["read_at"] is not None
history = client.get(
"/scribe/addresses/tower-control/history",
headers=office_headers("tower-control", "office-gemini"),
).json()
assert history["address"] == "tower-control"
assert len(history["messages"]) == 1
assert history["messages"][0]["id"] == message["id"]
found = client.get(
"/scribe/messages",
params={"q": "green"},
headers=office_headers("tower-control", "office-gemini"),
).json()
assert found["total"] == 1
assert found["entries"][0]["message"]["id"] == message["id"]
def test_history_reading_never_marks_or_consumes(self, client):
register(client, "jeroen", "office-claude")
register(client, "tower-control", "office-gemini")
client.post(
"/scribe/messages",
json={"to": ["tower-control"], "body": "hi"},
headers=office_headers("jeroen", "office-claude"),
)
client.get(
"/scribe/addresses/tower-control/history",
headers=office_headers("tower-control", "office-gemini"),
)
mailbox = client.get(
"/scribe/mailbox", headers=office_headers("tower-control", "office-gemini")
).json()
assert mailbox["entries"][0]["read_at"] is None
def test_mailbox_lists_unread_first(self, client):
register(client, "jeroen", "office-claude")
register(client, "tower-control", "office-gemini")
for body in ("first", "second", "third"):
client.post(
"/scribe/messages",
json={"to": ["tower-control"], "body": body},
headers=office_headers("jeroen", "office-claude"),
)
mailbox = client.get(
"/scribe/mailbox", headers=office_headers("tower-control", "office-gemini")
).json()
first_entry_id = mailbox["entries"][0]["id"]
client.patch(
f"/scribe/mailbox/{first_entry_id}",
json={"read": True},
headers=office_headers("tower-control", "office-gemini"),
)
mailbox_after = client.get(
"/scribe/mailbox", headers=office_headers("tower-control", "office-gemini")
).json()
read_states = [e["read_at"] is None for e in mailbox_after["entries"]]
# unread (True) entries sort before the one read (False) entry
assert read_states == sorted(read_states, reverse=True)
def test_tag_then_untag_mailbox_entry(self, client):
register(client, "jeroen", "office-claude")
register(client, "tower-control", "office-gemini")
client.post(
"/scribe/messages",
json={"to": ["tower-control"], "body": "hi"},
headers=office_headers("jeroen", "office-claude"),
)
entry_id = client.get(
"/scribe/mailbox", headers=office_headers("tower-control", "office-gemini")
).json()["entries"][0]["id"]
tagged = client.patch(
f"/scribe/mailbox/{entry_id}",
json={"add_tags": ["urgent"]},
headers=office_headers("tower-control", "office-gemini"),
)
assert tagged.json()["tags"] == ["urgent"]
untagged = client.patch(
f"/scribe/mailbox/{entry_id}",
json={"remove_tags": ["urgent"]},
headers=office_headers("tower-control", "office-gemini"),
)
assert untagged.json()["tags"] == []
# =============================================================================
# send validation
# =============================================================================
class TestSendValidation:
def test_send_to_unknown_address_is_404_naming_missing_ones(self, client):
register(client, "jeroen", "office-claude")
response = client.post(
"/scribe/messages",
json={"to": ["nobody", "tower-control"], "body": "hi"},
headers=office_headers("jeroen", "office-claude"),
)
assert response.status_code == 404
detail = response.json()["detail"]
assert detail["code"] == "unknown_address"
assert set(detail["names"]) == {"nobody", "tower-control"}
def test_send_from_identity_not_registered_is_403(self, client):
register(client, "tower-control", "office-gemini")
response = client.post(
"/scribe/messages",
json={"to": ["tower-control"], "body": "hi"},
headers=office_headers("jeroen", "office-claude"), # jeroen never registered
)
assert response.status_code == 403
assert response.json()["detail"]["code"] == "identity_not_owned"
def test_cannot_send_as_an_address_another_consumer_owns(self, client):
"""dossier verification #3: a consumer cannot send *as* an address
another consumer owns."""
register(client, "jeroen", "office-claude")
register(client, "tower-control", "office-gemini")
response = client.post(
"/scribe/messages",
json={"to": ["jeroen"], "body": "impersonating"},
# office-gemini claiming to BE jeroen, which office-claude owns
headers=office_headers("jeroen", "office-gemini"),
)
assert response.status_code == 403
assert response.json()["detail"]["code"] == "identity_not_owned"
def test_send_with_reply_to_in_another_tenant_is_404(self, client):
register(client, "jeroen", "office-claude", tenant=TENANT)
register(client, "tower-control", "office-gemini", tenant=TENANT)
sent = client.post(
"/scribe/messages",
json={"to": ["tower-control"], "body": "original"},
headers=office_headers("jeroen", "office-claude", tenant=TENANT),
).json()
register(client, "jeroen", "office-claude", tenant=OTHER_TENANT)
register(client, "tower-control", "office-gemini", tenant=OTHER_TENANT)
reply = client.post(
"/scribe/messages",
json={
"to": ["tower-control"],
"body": "cross-tenant reply",
"reply_to": sent["id"],
},
headers=office_headers("jeroen", "office-claude", tenant=OTHER_TENANT),
)
assert reply.status_code == 404
assert reply.json()["detail"]["code"] == "not_found"
# =============================================================================
# Isolation (dossier verification #3): the hard tenant boundary, both
# directions — every cross-tenant answer is indistinguishable from "does
# not exist".
# =============================================================================
class TestTenantIsolation:
def test_llm_tester_cannot_list_jpmschweitzer_addresses(self, client):
register(client, "jeroen", "office-claude", tenant=TENANT)
as_other = client.get(
"/scribe/addresses", headers=office_headers("jeroen", "office-claude", OTHER_TENANT)
).json()
assert as_other["addresses"] == []
def test_llm_tester_cannot_send_to_a_jpmschweitzer_address(self, client):
register(client, "tower-control", "office-gemini", tenant=TENANT) # TENANT only
register(client, "jeroen", "office-claude", tenant=OTHER_TENANT)
response = client.post(
"/scribe/messages",
json={"to": ["tower-control"], "body": "leaking?"},
headers=office_headers("jeroen", "office-claude", OTHER_TENANT),
)
# From OTHER_TENANT's point of view, "tower-control" does not exist.
assert response.status_code == 404
assert response.json()["detail"]["code"] == "unknown_address"
def test_llm_tester_cannot_read_mailbox_of_jpmschweitzer_address(self, client):
register(client, "jeroen", "office-claude", tenant=TENANT)
register(client, "tower-control", "office-gemini", tenant=TENANT)
client.post(
"/scribe/messages",
json={"to": ["tower-control"], "body": "secret"},
headers=office_headers("jeroen", "office-claude", tenant=TENANT),
)
# llm_tester has never registered tower-control, so it cannot even
# bind the identity -> 403, indistinguishable from "no such address".
response = client.get(
"/scribe/mailbox",
headers=office_headers("tower-control", "office-gemini", OTHER_TENANT),
)
assert response.status_code == 403
assert response.json()["detail"]["code"] == "identity_not_owned"
def test_llm_tester_cannot_see_history_of_a_jpmschweitzer_address(self, client):
register(client, "jeroen", "office-claude", tenant=TENANT)
register(client, "tower-control", "office-gemini", tenant=TENANT)
client.post(
"/scribe/messages",
json={"to": ["tower-control"], "body": "secret"},
headers=office_headers("jeroen", "office-claude", tenant=TENANT),
)
response = client.get(
"/scribe/addresses/tower-control/history",
headers=office_headers("jeroen", "office-claude", OTHER_TENANT),
)
assert response.status_code == 404
assert response.json()["detail"]["code"] == "unknown_address"
def test_llm_tester_cannot_find_a_message_of_a_jpmschweitzer_address(self, client):
register(client, "jeroen", "office-claude", tenant=TENANT)
register(client, "tower-control", "office-gemini", tenant=TENANT)
client.post(
"/scribe/messages",
json={"to": ["tower-control"], "body": "unique-secret-word"},
headers=office_headers("jeroen", "office-claude", tenant=TENANT),
)
response = client.get(
"/scribe/messages",
params={"q": "unique-secret-word"},
headers=office_headers("tower-control", "office-gemini", OTHER_TENANT),
)
assert response.status_code == 403
assert response.json()["detail"]["code"] == "identity_not_owned"
def test_mark_on_an_entry_in_another_tenant_is_404(self, client):
register(client, "jeroen", "office-claude", tenant=TENANT)
register(client, "tower-control", "office-gemini", tenant=TENANT)
client.post(
"/scribe/messages",
json={"to": ["tower-control"], "body": "secret"},
headers=office_headers("jeroen", "office-claude", tenant=TENANT),
)
entry_id = client.get(
"/scribe/mailbox", headers=office_headers("tower-control", "office-gemini", TENANT)
).json()["entries"][0]["id"]
register(client, "tower-control", "office-gemini", tenant=OTHER_TENANT)
response = client.patch(
f"/scribe/mailbox/{entry_id}",
json={"read": True},
headers=office_headers("tower-control", "office-gemini", OTHER_TENANT),
)
assert response.status_code == 404
assert response.json()["detail"]["code"] == "not_found"
def test_mark_on_someone_elses_entry_within_the_same_tenant_is_404(self, client):
"""Not a tenant crossing, but the same non-disclosure principle:
an entry addressed to another identity answers like it doesn't
exist, rather than confirming it belongs to someone else."""
register(client, "jeroen", "office-claude", tenant=TENANT)
register(client, "tower-control", "office-gemini", tenant=TENANT)
client.post(
"/scribe/messages",
json={"to": ["tower-control"], "body": "secret"},
headers=office_headers("jeroen", "office-claude", tenant=TENANT),
)
entry_id = client.get(
"/scribe/mailbox", headers=office_headers("tower-control", "office-gemini", TENANT)
).json()["entries"][0]["id"]
response = client.patch(
f"/scribe/mailbox/{entry_id}",
json={"read": True},
headers=office_headers("jeroen", "office-claude", TENANT), # not tower-control
)
assert response.status_code == 404
assert response.json()["detail"]["code"] == "not_found"
def test_mailbox_does_not_leak_entries_from_another_tenants_same_named_address(self, client):
"""The identity-ownership precheck alone is not the isolation
guarantee: a caller legitimately owns "jeroen" in their own tenant,
and a *different* "jeroen" (same name, different owner) exists in
another tenant with mail of its own. The mailbox query's own tenant
filter is what keeps that mail out — this is what mutation check #1
(dossier) targets."""
register(client, "jeroen", "office-claude", tenant=TENANT)
register(client, "tower-control", "office-gemini", tenant=TENANT)
client.post(
"/scribe/messages",
json={"to": ["jeroen"], "body": "for the TENANT jeroen"},
headers=office_headers("tower-control", "office-gemini", tenant=TENANT),
)
register(client, "jeroen", "office-claude", tenant=OTHER_TENANT)
register(client, "tower-control", "office-gemini", tenant=OTHER_TENANT)
client.post(
"/scribe/messages",
json={"to": ["jeroen"], "body": "for the OTHER_TENANT jeroen"},
headers=office_headers("tower-control", "office-gemini", tenant=OTHER_TENANT),
)
mailbox = client.get(
"/scribe/mailbox", headers=office_headers("jeroen", "office-claude", TENANT)
).json()
assert mailbox["total"] == 1
assert mailbox["entries"][0]["message"]["body"] == "for the TENANT jeroen"
def test_find_does_not_leak_messages_from_another_tenants_same_named_address(self, client):
"""Same concern as above, for `find` (mutation check #1's other half)."""
register(client, "jeroen", "office-claude", tenant=TENANT)
register(client, "tower-control", "office-gemini", tenant=TENANT)
client.post(
"/scribe/messages",
json={"to": ["jeroen"], "body": "shared-search-term in TENANT"},
headers=office_headers("tower-control", "office-gemini", tenant=TENANT),
)
register(client, "jeroen", "office-claude", tenant=OTHER_TENANT)
register(client, "tower-control", "office-gemini", tenant=OTHER_TENANT)
client.post(
"/scribe/messages",
json={"to": ["jeroen"], "body": "shared-search-term in OTHER_TENANT"},
headers=office_headers("tower-control", "office-gemini", tenant=OTHER_TENANT),
)
found = client.get(
"/scribe/messages",
params={"q": "shared-search-term"},
headers=office_headers("jeroen", "office-claude", TENANT),
).json()
assert found["total"] == 1
assert found["entries"][0]["message"]["body"] == "shared-search-term in TENANT"
class TestTenantRegistryEnforcement:
"""core-api T-8 / workspace D-73: Scribe's own `_require_tenant` now
delegates to the tenant registry's one enforcement point, so a Scribe
route gains three refusal paths it did not have before, alongside the
control that a properly granted, active tenant still succeeds."""
def test_unknown_tenant_is_403(self, client):
response = client.get(
"/scribe/addresses",
headers=office_headers("jeroen", "office-claude", tenant="no-such-tenant"),
)
assert response.status_code == 403
assert response.json()["detail"]["code"] == "tenant_unknown"
def test_suspended_tenant_is_403(self, client, suspended_tenant):
response = client.get(
"/scribe/addresses",
headers=office_headers("jeroen", "office-claude", tenant=suspended_tenant),
)
assert response.status_code == 403
assert response.json()["detail"]["code"] == "tenant_suspended"
def test_consumer_without_a_grant_is_403(self, client):
response = client.get(
"/scribe/addresses",
headers=office_headers("jeroen", "office-ungranted", tenant=TENANT),
)
assert response.status_code == 403
assert response.json()["detail"]["code"] == "consumer_not_granted"
def test_granted_consumer_in_an_active_tenant_succeeds(self, client):
"""The control: office-claude, granted in jpmschweitzer (active),
still reaches the route."""
response = client.get(
"/scribe/addresses", headers=office_headers("jeroen", "office-claude", tenant=TENANT)
)
assert response.status_code == 200
# =============================================================================
# Archive and trash a mailbox entry (core-api T-13 / D-66, D-70): deletion
# is per mailbox entry, never per message — the message stays in `history`
# and another recipient's entry is untouched throughout.
# =============================================================================
def _send_and_get_entry_id(client, body: str, to: str = "tower-control") -> tuple:
"""Register the usual pair if needed by the caller, send one message,
and return (message, entry_id) for the recipient's own mailbox entry."""
sent = client.post(
"/scribe/messages",
json={"to": [to], "body": body},
headers=office_headers("jeroen", "office-claude"),
).json()
entry_id = client.get(
"/scribe/mailbox", headers=office_headers("tower-control", "office-gemini")
).json()["entries"][0]["id"]
return sent, entry_id
class TestArchiveAndTrash:
def test_archive_hides_from_inbox_and_shows_in_archive_view(self, client):
register(client, "jeroen", "office-claude")
register(client, "tower-control", "office-gemini")
_, entry_id = _send_and_get_entry_id(client, "archive me")
archived = client.patch(
f"/scribe/mailbox/{entry_id}",
json={"archived": True},
headers=office_headers("tower-control", "office-gemini"),
)
assert archived.status_code == 200
assert archived.json()["archived_at"] is not None
assert archived.json()["trashed_at"] is None
inbox = client.get(
"/scribe/mailbox", headers=office_headers("tower-control", "office-gemini")
).json()
assert inbox["total"] == 0
archive_view = client.get(
"/scribe/mailbox",
params={"view": "archive"},
headers=office_headers("tower-control", "office-gemini"),
).json()
assert archive_view["total"] == 1
assert archive_view["entries"][0]["id"] == entry_id
def test_unarchive_brings_it_back_to_inbox(self, client):
register(client, "jeroen", "office-claude")
register(client, "tower-control", "office-gemini")
_, entry_id = _send_and_get_entry_id(client, "archive then unarchive")
client.patch(
f"/scribe/mailbox/{entry_id}",
json={"archived": True},
headers=office_headers("tower-control", "office-gemini"),
)
unarchived = client.patch(
f"/scribe/mailbox/{entry_id}",
json={"archived": False},
headers=office_headers("tower-control", "office-gemini"),
)
assert unarchived.json()["archived_at"] is None
inbox = client.get(
"/scribe/mailbox", headers=office_headers("tower-control", "office-gemini")
).json()
assert inbox["total"] == 1
assert inbox["entries"][0]["id"] == entry_id
def test_trash_hides_from_inbox_archive_and_find_shows_in_trash(self, client):
register(client, "jeroen", "office-claude")
register(client, "tower-control", "office-gemini")
_, entry_id = _send_and_get_entry_id(client, "trash-findable-term")
trashed = client.patch(
f"/scribe/mailbox/{entry_id}",
json={"trashed": True},
headers=office_headers("tower-control", "office-gemini"),
)
assert trashed.status_code == 200
assert trashed.json()["trashed_at"] is not None
inbox = client.get(
"/scribe/mailbox", headers=office_headers("tower-control", "office-gemini")
).json()
assert inbox["total"] == 0
archive_view = client.get(
"/scribe/mailbox",
params={"view": "archive"},
headers=office_headers("tower-control", "office-gemini"),
).json()
assert archive_view["total"] == 0
found = client.get(
"/scribe/messages",
params={"q": "trash-findable-term"},
headers=office_headers("tower-control", "office-gemini"),
).json()
assert found["total"] == 0
trash_view = client.get(
"/scribe/mailbox",
params={"view": "trash"},
headers=office_headers("tower-control", "office-gemini"),
).json()
assert trash_view["total"] == 1
assert trash_view["entries"][0]["id"] == entry_id
def test_restore_brings_it_back_to_inbox(self, client):
register(client, "jeroen", "office-claude")
register(client, "tower-control", "office-gemini")
_, entry_id = _send_and_get_entry_id(client, "trash then restore")
client.patch(
f"/scribe/mailbox/{entry_id}",
json={"trashed": True},
headers=office_headers("tower-control", "office-gemini"),
)
restored = client.patch(
f"/scribe/mailbox/{entry_id}",
json={"trashed": False},
headers=office_headers("tower-control", "office-gemini"),
)
assert restored.json()["trashed_at"] is None
inbox = client.get(
"/scribe/mailbox", headers=office_headers("tower-control", "office-gemini")
).json()
assert inbox["total"] == 1
assert inbox["entries"][0]["id"] == entry_id
def test_message_stays_in_history_throughout_archive_and_trash(self, client):
register(client, "jeroen", "office-claude")
register(client, "tower-control", "office-gemini")
message, entry_id = _send_and_get_entry_id(client, "history survives")
for payload in ({"archived": True}, {"trashed": True}):
client.patch(
f"/scribe/mailbox/{entry_id}",
json=payload,
headers=office_headers("tower-control", "office-gemini"),
)
history = client.get(
"/scribe/addresses/tower-control/history",
headers=office_headers("tower-control", "office-gemini"),
).json()
assert [m["id"] for m in history["messages"]] == [message["id"]]
def test_trashing_one_recipients_entry_leaves_the_others_untouched(self, client):
register(client, "jeroen", "office-claude")
register(client, "tower-control", "office-gemini")
register(client, "jeroen-2", "office-claude")
client.post(
"/scribe/messages",
json={"to": ["tower-control", "jeroen-2"], "body": "broadcast"},
headers=office_headers("jeroen", "office-claude"),
)
tower_entry_id = client.get(
"/scribe/mailbox", headers=office_headers("tower-control", "office-gemini")
).json()["entries"][0]["id"]
client.patch(
f"/scribe/mailbox/{tower_entry_id}",
json={"trashed": True},
headers=office_headers("tower-control", "office-gemini"),
)
other_inbox = client.get(
"/scribe/mailbox", headers=office_headers("jeroen-2", "office-claude")
).json()
assert other_inbox["total"] == 1
assert other_inbox["entries"][0]["trashed_at"] is None
# =============================================================================
# Look an entry up by message number (core-api T-9): zero or one, in any
# view, no paging required.
# =============================================================================
class TestMailboxMessageIdLookup:
def test_message_id_finds_the_entry_in_any_view(self, client):
register(client, "jeroen", "office-claude")
register(client, "tower-control", "office-gemini")
message, entry_id = _send_and_get_entry_id(client, "locate me")
client.patch(
f"/scribe/mailbox/{entry_id}",
json={"trashed": True},
headers=office_headers("tower-control", "office-gemini"),
)
found = client.get(
"/scribe/mailbox",
params={"message_id": message["id"]},
headers=office_headers("tower-control", "office-gemini"),
).json()
assert found["total"] == 1
assert found["entries"][0]["id"] == entry_id
assert found["entries"][0]["trashed_at"] is not None
def test_message_id_answers_nothing_for_a_message_not_in_their_mailbox(self, client):
register(client, "jeroen", "office-claude")
register(client, "tower-control", "office-gemini")
sent = client.post(
"/scribe/messages",
json={"to": ["tower-control"], "body": "not jeroen's"},
headers=office_headers("jeroen", "office-claude"),
).json()
found = client.get(
"/scribe/mailbox",
params={"message_id": sent["id"]},
headers=office_headers("jeroen", "office-claude"),
).json()
assert found["total"] == 0
assert found["entries"] == []
def test_message_id_answers_nothing_across_tenants(self, client):
register(client, "jeroen", "office-claude", tenant=TENANT)
register(client, "tower-control", "office-gemini", tenant=TENANT)
sent = client.post(
"/scribe/messages",
json={"to": ["tower-control"], "body": "tenant-scoped"},
headers=office_headers("jeroen", "office-claude", tenant=TENANT),
).json()
register(client, "tower-control", "office-gemini", tenant=OTHER_TENANT)
found = client.get(
"/scribe/mailbox",
params={"message_id": sent["id"]},
headers=office_headers("tower-control", "office-gemini", OTHER_TENANT),
).json()
assert found["total"] == 0
def test_message_id_finds_a_read_entry_past_the_first_page_in_one_call(self, client):
"""dossier verification #4: today's listing sorts unread-first,
then by id descending, behind a single before_id cursor. Seed one
old entry that stays unread (so it sorts first regardless of its
low id) ahead of 51 read ones (one more than the default page size
of 50) — the oldest read entry (lowest id among them) sorts last,
past the first page. `message_id` answers it in one call, with no
`before_id` paging."""
register(client, "jeroen", "office-claude")
register(client, "tower-control", "office-gemini")
old_unread = client.post(
"/scribe/messages",
json={"to": ["tower-control"], "body": "old and unread"},
headers=office_headers("jeroen", "office-claude"),
).json()
read_message_ids = []
for i in range(51):
sent = client.post(
"/scribe/messages",
json={"to": ["tower-control"], "body": f"read {i}"},
headers=office_headers("jeroen", "office-claude"),
).json()
read_message_ids.append(sent["id"])
all_entries = client.get(
"/scribe/mailbox",
params={"view": "all", "limit": 200},
headers=office_headers("tower-control", "office-gemini"),
).json()["entries"]
for entry in all_entries:
if entry["message"]["id"] != old_unread["id"]:
client.patch(
f"/scribe/mailbox/{entry['id']}",
json={"read": True},
headers=office_headers("tower-control", "office-gemini"),
)
# The oldest (lowest-id) read message: last among the reads when
# sorted id-descending, so page 1 (the default limit of 50) never
# reaches it — plain listing would need a second before_id call.
target_message_id = read_message_ids[0]
page_one = client.get(
"/scribe/mailbox", headers=office_headers("tower-control", "office-gemini")
).json()
assert target_message_id not in {e["message"]["id"] for e in page_one["entries"]}
found = client.get(
"/scribe/mailbox",
params={"message_id": target_message_id},
headers=office_headers("tower-control", "office-gemini"),
).json()
assert found["total"] == 1
assert found["entries"][0]["message"]["id"] == target_message_id
+110
View File
@@ -0,0 +1,110 @@
"""
Tests for Scribe migrations 006 and 009.
Runs against `core_api_test` only (see conftest.py — skip when
TEST_DATABASE_URL is unset, hard failure if it resolves to `core_api`).
Self-contained: starts and ends at head regardless of what ran before it.
"""
import pytest
from sqlalchemy import text
from sqlalchemy.ext.asyncio import create_async_engine
from sqlalchemy.pool import NullPool
from tests.conftest import run_alembic
async def _table_exists(url: str, table: str) -> bool:
asyncpg_url = url.replace("postgresql://", "postgresql+asyncpg://", 1)
engine = create_async_engine(asyncpg_url, poolclass=NullPool)
try:
async with engine.connect() as conn:
result = await conn.execute(
text("SELECT to_regclass(:name) IS NOT NULL AS present"),
{"name": table},
)
return bool(result.scalar())
finally:
await engine.dispose()
async def _columns(url: str, table: str) -> set:
asyncpg_url = url.replace("postgresql://", "postgresql+asyncpg://", 1)
engine = create_async_engine(asyncpg_url, poolclass=NullPool)
try:
async with engine.connect() as conn:
result = await conn.execute(
text(
"SELECT column_name FROM information_schema.columns "
"WHERE table_name = :table"
),
{"table": table},
)
return {row[0] for row in result.all()}
finally:
await engine.dispose()
class TestMigration006RoundTrip:
"""Migration 006 must upgrade and downgrade cleanly (dossier verification #1)."""
@pytest.mark.asyncio
async def test_upgrade_creates_scribe_tables(self, test_database_url):
result = run_alembic(test_database_url, "upgrade", "head")
assert result.returncode == 0, result.stderr
for table in ("scribe_addresses", "scribe_messages", "scribe_mailbox_entries"):
assert await _table_exists(test_database_url, table), f"{table} missing after upgrade"
@pytest.mark.asyncio
async def test_downgrade_then_upgrade_round_trips_cleanly(self, test_database_url):
baseline = run_alembic(test_database_url, "upgrade", "head")
assert baseline.returncode == 0, baseline.stderr
down = run_alembic(test_database_url, "downgrade", "005")
assert down.returncode == 0, down.stderr
assert not await _table_exists(test_database_url, "scribe_addresses")
assert not await _table_exists(test_database_url, "scribe_messages")
assert not await _table_exists(test_database_url, "scribe_mailbox_entries")
# Book tables, created by the migration below this one, must survive
# a downgrade of 006 — proves 006 only touches its own tables.
assert await _table_exists(test_database_url, "book_records")
# Leave the schema at head for the rest of the suite.
up = run_alembic(test_database_url, "upgrade", "006")
assert up.returncode == 0, up.stderr
assert await _table_exists(test_database_url, "scribe_addresses")
assert await _table_exists(test_database_url, "scribe_messages")
assert await _table_exists(test_database_url, "scribe_mailbox_entries")
class TestMigration009RoundTrip:
"""Migration 009 (core-api T-13) must upgrade and downgrade cleanly —
dossier verification #2: 008 -> 009 -> 008 -> 009."""
@pytest.mark.asyncio
async def test_upgrade_adds_archived_and_trashed_at(self, test_database_url):
result = run_alembic(test_database_url, "upgrade", "head")
assert result.returncode == 0, result.stderr
columns = await _columns(test_database_url, "scribe_mailbox_entries")
assert {"archived_at", "trashed_at"} <= columns
@pytest.mark.asyncio
async def test_downgrade_then_upgrade_round_trips_cleanly(self, test_database_url):
baseline = run_alembic(test_database_url, "upgrade", "head")
assert baseline.returncode == 0, baseline.stderr
down = run_alembic(test_database_url, "downgrade", "008")
assert down.returncode == 0, down.stderr
columns_at_008 = await _columns(test_database_url, "scribe_mailbox_entries")
assert "archived_at" not in columns_at_008
assert "trashed_at" not in columns_at_008
# The table itself, created by 006, must survive a downgrade of
# 009 — proves 009 only adds/drops its own two columns.
assert await _table_exists(test_database_url, "scribe_mailbox_entries")
# Leave the schema at head (009 is head) for the rest of the suite.
up = run_alembic(test_database_url, "upgrade", "head")
assert up.returncode == 0, up.stderr
columns_at_head = await _columns(test_database_url, "scribe_mailbox_entries")
assert {"archived_at", "trashed_at"} <= columns_at_head

Some files were not shown because too many files have changed in this diff Show More