Compare commits

...
46 Commits
Author SHA1 Message Date
jpmschweitzer 3536c40608 fix(permissions): narrow rm -rf deny globs to their exact forms
The trailing wildcard on the three rm -rf deny entries spanned path
separators, so Bash(rm -rf /*) matched every absolute path on the
machine rather than the filesystem root, and the ~ and $HOME entries
had the same shape. Narrowed to the exact literal forms.

These rules match literal command text, so they still stop a typo on
rm -rf /, rm -rf ~ or rm -rf $HOME exactly, but they no longer stop a
recursive delete aimed at any other path. That reduced cover is
deliberate, not an oversight.
2026-08-25 20:31:15 +02:00
jpmschweitzerandClaude 0c7e1cb04e fix(tests): mock the live DNS service instance, not a dead legacy class (T-55)
test_dns_lookup_returns_result patched src.controllers.tools_controller.
DNSService, which is never imported by the request path under test.
client wraps src.main.app, which routes /tools/dns/lookup through
src.domains.tools.controller.tools_controller — a singleton constructed
at import time from src.domains.tools.dns.service.DNSService. The patched
class was dead; the mock was never consulted, so the test issued a real
DNS query for example.com and asserted on its outcome. With no network
the query times out and the assertion fails (D-26).

The three sibling tests in the same class patch the same dead class and
also run unmocked, but happen not to notice: DNS failures are caught
inside DNSService.lookup() and returned as a normal 200 response with
success=False, and their assertions only check status_code / DNSQueryError
branches that don't depend on resolution actually succeeding. Only this
test's `data["success"] is True` assertion is sensitive to the real
network outcome, which is why it's the only one D-26's namespace run
catches. Not touched here — out of T-55's scope, flagging for the record.

Fix patches tools_controller.dns_service, the actual instance attribute
the live route calls, via patch.object on the singleton rather than
patch() on the constructor class (the instance already exists by the
time a class-level patch would apply).

Verified:
- unshare -rn (lo up): 381 passed, exit 0 (was 1 failed, 380 passed, exit 2)
- with network: 381 passed, exit 0 (unchanged from before the fix)
- mutation check: retargeted the patch.object to a nonexistent attribute
  name, confirmed count==1 before editing; namespace run then reproduced
  the original failure (1 failed, 380 passed); reverted and reconfirmed
  381 passed, exit 0.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-18 15:33:58 +02:00
jpmschweitzerandClaude 6695215aeb build(make): prove setup worked instead of assuming pip's exit code (T-47)
pip install exits 0 whether the result is usable or not — that is the D-24
shape exactly, a step whose job is to not fail. On 2026-08-09 the venv here
existed and pip had already succeeded, but sqlalchemy was declared in
requirements.txt and not installed. That surfaced as 11 pytest collection
errors that read as broken imports rather than as an environment problem.

setup now ends with `pytest --collect-only tests/`, which exercises every
import the suite touches without running anything. pip check was considered
as a cheaper alternative and rejected: it only checks the installed set's
internal consistency against itself, so it would not have caught this case —
sqlalchemy was still present as another package's transitive dependency even
after being dropped from requirements.txt. collect-only checks declared vs.
actually usable directly, which is the axis that broke.

Verified: two clean runs (33.5s cold, 5.7s idempotent re-run, second changes
nothing). Reproduced the original failure by uninstalling sqlalchemy from an
otherwise-correct venv — collect-only alone then exits 2 with 11 collection
errors; make setup against that same state reinstalls it and exits 0 with
381/381 collected.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-17 12:05:28 +02:00
jpmschweitzerandClaude d2e0a49e7c build(ci): move the pre-push gate into the Makefile
The hook carried ~50 lines of gitleaks logic and a comment explaining it was
self-contained because "this repo has no Makefile". It has one now, so the
reason is gone and the arrangement is backwards: a hook is a trigger, and
logic belongs where it can be read, run by hand, and changed under review.

.githooks/pre-push is now a byte-identical shim onto `make pre-push` in every
repo in the workspace. The scan itself moves to ci/secrets.sh unchanged, and
`make secrets` runs it on its own.

The call surface is identical everywhere; what it runs is not, and should not
be — each repo gates what it actually has. That is the point of standardising
the name rather than the contents: nobody has to read a repo to find out how
to check it.

secrets runs first, deliberately. It is the only failure here that cannot be
undone by fixing it afterwards — a failed lint costs another commit, a pushed
credential is cached and indexed whether or not it is later deleted.

Some of these gates fail today, on lint debt that predates them, and they are
left wired anyway. The board was measured once and written down in T-56
instead of being worked around here. Narrowing each gate to whatever already
passes would produce a gate that reports success for doing nothing, which is
the failure this workspace keeps rediscovering.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 18:57:22 +02:00
jpmschweitzerandClaude 755aa61107 ci(make): reserve exit 69 for "could not run" (D-26)
Environment guards now exit 69 rather than 1, so a caller can tell a suite
that could not start from one that ran and failed. The first toj test sweep
reported "3 repositories failed" and none of the three had executed a test —
two could not find go, one had no venv. That points the reader at the tests
when the fault is in the environment.

Only the environment guards change. A gitleaks finding, a failed test run and
a vulncheck hit still exit 1, because those did run and did fail.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 15:56:19 +02:00
jpmschweitzerandClaude 01349a83f2 test: repair the suite against the current API
The suite could not even collect: the venv was missing declared dependencies,
and five tests asserted an API that had moved on. 11 collection errors to 381
passing.

test_oidc.py was written for the single-issuer API and 6243f29 replaced it.
issuer and audience became lists, jwks_uri stopped being an attribute in
favour of get_jwks_uri(issuer), get_jwks became get_jwks_for_issuer, and the
lru_cache became a per-issuer dict so cache_clear no longer exists. Rewritten
against the current surface, with coverage added for the two behaviours the
multi-issuer change introduced and never tested: is_valid_issuer rejecting an
unconfigured issuer, and the cache keying per issuer. Both are
security-relevant — a shared cache would serve one issuer keys for another.

Three /auth/me tests asserted a path that does not exist. The route is
declared as /me inside AuthController.create_router() and mounts at
/auth/users/me; the generated spec is authoritative and the local app and the
deployed service agree on it. Those tests had never passed.

test_handles_empty_groups expected groups == [""] for an empty header. oidc.py
has returned [] since the initial commit, and [] is correct — [""] would also
be unsafe, since any check doing "" in groups would match.

test_model_aliases_property covered Settings.model_aliases, deleted with the
Ollama integration in c1f16d4. Removed rather than repaired.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 15:21:43 +02:00
jpmschweitzerandClaude 7815e1c231 build: add the Makefile command surface (D-27)
Every repo gets one at the root: help, plus test and lint where those exist.
The point is that a target name means the same thing in every repo, so an
agent or a person can act without reading the repo first.

Paths resolve here rather than in callers (D-10). python3 on this host is 3.8
and cannot parse these sources, and a bare pytest or ruff resolves only in a
login shell — so both are named explicitly through the venv, and a missing
venv fails with the command to fix it rather than a bare no-such-file.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 15:10:31 +02:00
jpmschweitzerandClaude e9fc09af26 chore(claude): pin PQL_VAULT per project so cwd stops choosing the vault
pql is now a bare word on PATH, which removed the long incantation that had
been forcing --vault into every call by habit. Convenience lowered the cost
of the wrong thing without lowering the cost of the right one: a three-word
pql ticket new targets whichever vault the cwd happens to sit in, and there
are nine of them with colliding id sequences.

PQL_VAULT in each project settings file makes the vault a property of the
session rather than of the working directory — the same lesson Rule 3 records
for git -C, applied to pql. Verified the env var overrides cwd discovery,
that an explicit --vault still beats the env var, and that the harness
hot-reloads it without a restart.

This does not make provenance visible: no output says which vault answered,
so a forgotten --vault still returns a well-formed answer about the wrong
dataset. That remains T-37.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 13:48:58 +02:00
jpmschweitzerandClaude 56b991c965 chore(claude): deny toj in the sub-repos
toj is now on the global PATH as /usr/local/bin/toj, so its scope boundary
had to stop being "the absolute path is inconvenient to type" and start
being a rule. Its repo and settings verbs operate on the workspace root; run
from inside this repo they answer about the wrong tree.

Both spellings are denied, bare and absolute, because a deny with one
spelling left open is decorative.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 13:42:03 +02:00
jpmschweitzerandClaude 387a3dbb02 ci: gate pushes on a gitleaks scan of the outgoing commits
No repo here scanned for committed credentials. The hook is self-contained
rather than delegating to a Makefile, because this repo has none and a hook
reaching into a sibling repo breaks the moment this one is cloned elsewhere.

Scans the outgoing range rather than full history: history carries settled
findings — test fixtures, vendored third-party code — and a gate that fails
on something unfixable gets bypassed within a week.

Setting core.hooksPath means pql init must replant its replication shims into
.githooks, which is why they are gitignored here alongside the tracked
pre-push. Same layout pql itself uses.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 12:48:53 +02:00
jpmschweitzerandClaude 7f90369621 docs: qualify workspace decision ids cited from this repo
Decision ids are per-vault sequences, so they collide by construction
once there is more than one vault -- and every repo now has one. A bare
D-15 here will mean this repo's D-15 the moment this repo records one.
Cross-vault references are therefore qualified: workspace D-15.

Not hypothetical: pql holds D-1 through D-31 while the workspace holds
D-1 through D-21, so every workspace id currently collides with an
unrelated pql one. A bare id is not wrong the day it is written -- it
decays into wrong as the other vault grows, and nothing flags it.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 04:17:07 +02:00
jpmschweitzerandClaude 659f08e576 chore: adopt the workspace agent-config baseline
Commits a .claude/settings.json rather than leaving permissions to
per-developer local state, and initialises a pql vault for this repo's
tickets and internal decisions.

Every git deny rule appears in both the `git <verb>` and `git * <verb>`
forms. Only the second catches `git -C <path>`, and without it the whole
deny list is decorative -- it looks like a policy and stops nothing.

The allow list carries pql's absolute path alongside the bare name.
pql is installed to ~/.local/bin, which is on the login PATH but not the
one a non-interactive shell gets, so the bare-name rules match nothing on
their own and every call would prompt anyway.

.gitignore now covers .claude/settings.local.json, which is machine-local
and must never be shared. `pql init` contributed the .pql/* rules with an
exception for the changelog, which is the replication log of record and
has to be committed for tickets to travel with a clone.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 03:13:08 +02:00
jpmschweitzerandClaude 29bfa3259d docs: replace AGENTS.md with a repo-specific CLAUDE.md
One agent doc per repo, and it is CLAUDE.md. Two docs describing one repo
drift, and the one nobody read is always the one holding the rule that
mattered. Written fresh rather than reformatted, so the structure follows
what someone working here actually needs.

Two rules from the old file are gone deliberately. The mandate to branch
for every change was retired in favour of one linear-history policy, and
the release snippet used `git add -A`, which sweeps in whatever else is
dirty.

The architecture section is the part worth reading. An earlier draft
called src/auth, src/controllers, src/clients, src/dns and src/models
dead code, derived from grepping main.py's imports. That was wrong:
main.py:55 calls initialize_oidc(), which imports and configures
src.auth.oidc from inside the function body, so src/auth is configured
with live Authentik issuers on every boot. It also missed four genuinely
unreferenced packages. The section now states the method used -- import
the app in the container and read sys.modules -- and its blind spot, that
a cold snapshot cannot see a module imported on a request path.

Also records that the README's "runs as non-root user (uid 1000)" is
false: the Dockerfile has no USER directive. Flagged rather than fixed,
since changing the runtime user is not a docs change.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 03:12:02 +02:00
jpmschweitzerandClaude Fable 5 b3383f19b3 chore(ci): push images via git.schweitz.net registry
The .internal registry domain is being retired; git.schweitz.net now
serves the registry without SSO on /v2/.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 11:09:58 +02:00
Jeroen SchweitzerandClaude Opus 4.5 fcf5c8ccee feat: add /tools/news endpoint for news ticker integration
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m15s
- Add NewsHeadline and NewsResponse schemas
- Add NewsService for parsing news from Qdrant volatile collection
- Add GET /tools/news endpoint to tools controller
- News fetched from 'news' namespace in volatile_{user} collection

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 21:49:05 +01:00
Jeroen SchweitzerandClaude Opus 4.5 768cea2c89 fix: pass forecast raw_data to service for parsing
Build and Push / build (push) Successful in 1m14s
Build and Push / release (push) Successful in 2s
Qdrant client was extracting 'days' (int) instead of 'daily' (list)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 18:45:24 +01:00
Jeroen SchweitzerandClaude Opus 4.5 26ecc3e5fd fix: convert wind direction degrees to cardinal string
Build and Push / release (push) Successful in 2s
Build and Push / build (push) Successful in 1m14s
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 18:36:38 +01:00
Jeroen SchweitzerandClaude Opus 4.5 5ff4ba0a43 fix: environment data parsing for scheduler format
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m15s
- Forecast: Check 'daily' key first (scheduler stores count in 'days')
- Sun times: Use ISO fields, calculate daylight from various sources
- Air quality: Support aqi_us/aqi_european and nitrogen_dioxide fields

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 18:30:30 +01:00
Jeroen SchweitzerandClaude Opus 4.5 bb438e22d6 fix: strip email domain from user identifier for environment endpoint
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m15s
If preferred_username is an email (user@domain.com), extract just the
username part to match Qdrant collection naming (volatile_user).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 15:46:16 +01:00
Jeroen SchweitzerandClaude Opus 4.5 3bb3b01dbd fix: OIDC audience validation - use string not list
Build and Push / build (push) Successful in 1m15s
Build and Push / release (push) Successful in 3s
python-jose jwt.decode() requires audience as string or None, not list.
Now extract and validate audience from unverified claims first,
then use token's actual audience for JWT decode.

Fixes "audience must be a string or None" error.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 15:35:05 +01:00
Jeroen SchweitzerandClaude Opus 4.5 ce761a9d2c debug: add logging for OIDC token validation
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 13:58:39 +01:00
Jeroen SchweitzerandClaude Opus 4.5 6243f29aae feat: multi-issuer OIDC support with config consolidation
Build and Push / release (push) Successful in 2s
Build and Push / build (push) Successful in 1m15s
- Support multiple OAuth providers (core-api, tatlock-ui, tatlock)
- Changed oidc_issuer (string) to oidc_issuers (list)
- Per-issuer JWKS caching
- Validates token issuer against allowed list
- Consolidated config files (removed deprecated src/config.py, src/security.py)
- Updated imports to use src/shared/config and src/shared/security

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 13:09:29 +01:00
Jeroen SchweitzerandClaude Opus 4.5 c4d32952db fix: OIDC multi-audience support and Swagger UI fix
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m13s
- Accept tokens from multiple clients (core-api, tatlock-ui, tatlock)
- Fixed main.py to use oidc_audiences[0] for Swagger UI OAuth

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 12:17:16 +01:00
Jeroen Schweitzer 4c45f139d9 correct documentation 2026-01-08 12:11:01 +01:00
Jeroen Schweitzer 67b33314fe fix: update main.py to use oidc_audiences list
Build and Push / release (push) Failing after 3s
Build and Push / build (push) Has been skipped
2026-01-07 19:34:05 +01:00
Jeroen SchweitzerandClaude Opus 4.5 6ce34cc016 fix: accept multiple OIDC audiences for cross-client auth
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m15s
- Changed oidc_audience (string) to oidc_audiences (list)
- Now accepts tokens with audience: core-api, tatlock-ui, tatlock
- Fixes environment endpoint returning "default" user when using
  tatlock-ui token (audience mismatch was causing JWT claims error)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 19:01:39 +01:00
Jeroen SchweitzerandClaude Opus 4.5 c1f16d44e5 refactor: remove Ollama integration and unused AI configuration
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m19s
- Remove src/models/ollama_client.py, embeddings.py, embeddings_ollama.py
- Remove model aliases and AI config from settings (both config.py files)
- Update health endpoints to only check database connectivity
- Update tests to reflect database-only health checks
- Update README, .env.example, and OIDC docstrings

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 17:56:48 +01:00
Jeroen SchweitzerandClaude Opus 4.5 4df5cfc106 fix: initialize OIDC config for both auth modules
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m30s
The domains.auth.oidc module had its own oidc_config instance that
wasn't being configured, causing environment endpoint to always use
hardcoded "local" user instead of authenticated user.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 16:00:14 +01:00
Jeroen SchweitzerandClaude Opus 4.5 0a16688cc8 chore: release v1.10.2
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 2m7s
Enhanced environment endpoint logging for debugging user resolution.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 15:32:53 +01:00
Jeroen SchweitzerandClaude Opus 4.5 a7535fe8ea fix: correct OIDC import path in tools controller
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m14s
The environment endpoint was importing from non-existent path
`src.oidc.dependencies` instead of `src.auth.oidc`, causing
authentication to fail and queries to go to wrong Qdrant collection.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 23:36:24 +01:00
Jeroen SchweitzerandClaude Opus 4.5 6045c6ac6a feat: add environment endpoint for weather, forecast, and sun data
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 2m14s
Add GET /tools/environment endpoint that fetches weather, forecast, sun times,
and air quality data from user's volatile Qdrant collection.

- Add qdrant-client dependency
- Create QdrantReadClient wrapper for read-only queries
- Add environment schemas and service in tools domain
- Parse weather, forecast, sun times, and air quality from Qdrant payloads
- Support user-specific collections via preferred_username from OIDC
- Add comprehensive service tests (13 tests)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 23:04:59 +01:00
Jeroen Schweitzer b8fca7060f npm config 2026-01-04 21:51:21 +01:00
Jeroen SchweitzerandClaude Opus 4.5 e3a49c800a fix: SQLAlchemy async lazy loading for new users
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m11s
Initialize user.roles=[] and user.preferences on new user creation
to avoid MissingGreenlet error in async context.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 18:05:34 +01:00
Jeroen SchweitzerandClaude Opus 4.5 7ab9f73a1d chore: release v1.9.3
Build and Push / release (push) Successful in 2s
Build and Push / build (push) Successful in 1m10s
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 17:49:26 +01:00
Jeroen SchweitzerandClaude Opus 4.5 dd5b794de4 fix: handle non-UUID sub claim in auth sync
Authentik JWT sub claim may not be a valid UUID.
Now derives a deterministic UUID from the sub string if parsing fails.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 17:45:05 +01:00
Jeroen SchweitzerandClaude Opus 4.5 dd0997679f fix: /auth/users/me now supports NPM forward auth headers
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m11s
Added get_current_user_or_forward_auth() combined dependency that:
- First checks for X-authentik-* headers from NPM forward auth (web)
- Falls back to JWT Bearer token validation (mobile/native)

This fixes web authentication where browsers don't send Bearer tokens
but rely on NPM's forward auth proxy to pass user info via headers.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 00:24:41 +01:00
Jeroen SchweitzerandClaude Opus 4.5 e2226cd923 fix(build): add production API URLs to Docker build
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m8s
Dockerfile now passes --dart-define flags for CORE_API_URL and
TATLOCK_API_URL pointing to schweitz.net domains. This enables
requiresAuth=true, fixing auth being completely skipped in production.

Also added service port reference table to AGENTS.md.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 23:18:34 +01:00
Jeroen SchweitzerandClaude Opus 4.5 7bf3c76a1b fix(cors): use explicit origins instead of wildcard
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m11s
When allow_credentials=True, browsers reject wildcard (*) origins.
Added specific allowed origins for Tatlock domains.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 23:09:47 +01:00
Jeroen SchweitzerandClaude Opus 4.5 4ae8cfbc0b chore: release v1.9.0
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m10s
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 22:24:36 +01:00
Jeroen SchweitzerandClaude Opus 4.5 69045a78a7 test(auth): add tests for GET /auth/me endpoint
Add comprehensive tests for NPM forward auth endpoint:
- Forward auth header parsing
- User lookup methods (get_user_by_email, get_user_by_authentik_id)
- Admin gate authorization logic
- OpenAPI spec validation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 22:14:13 +01:00
Jeroen SchweitzerandClaude Opus 4.5 884996fd83 feat(auth): implement GET /auth/me for NPM forward auth
Add endpoint to get current user profile from NPM forward auth headers.
Enables web authentication flow where NPM handles Authentik login.

- Read X-authentik-* headers set by NPM forward auth
- Auto-create user if not in database (first login via web)
- Sync roles from current Authentik groups
- Add get_user_by_email and get_user_by_authentik_id helpers

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 21:54:44 +01:00
Jeroen SchweitzerandClaude Opus 4.5 8bd13eab09 ci: trigger build on version tag push with auto-release
Changed workflow to:
- Trigger on push of v* tags instead of release publish
- Auto-create Gitea release via API
- Then build and push Docker image

This simplifies deployment: just push a version tag.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 20:40:44 +01:00
Jeroen SchweitzerandClaude Opus 4.5 397a47c8fc feat(auth): implement Phase 4 user profile and API key endpoints
Build and Push / build (release) Successful in 1m10s
Add user profile, preferences, and API key management endpoints:
- GET /auth/users/me - full user profile with roles and preferences
- GET/PATCH /auth/users/me/preferences - user preferences management
- GET/POST/DELETE /auth/users/me/api-keys - API key lifecycle

API keys use tak_ prefix, SHA-256 hashing, and are shown only once on creation.
Preferences support partial updates with JSON merge behavior.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 20:24:46 +01:00
Jeroen SchweitzerandClaude Opus 4.5 7752cd9d23 feat(auth): implement group-role mapping and permission system
Architecture changes:
- Permission format: domain.category:action (e.g., control-room.general:admin)
- Decoupled groups from roles via group_roles mapping table
- Groups are organizational (synced from Authentik)
- Roles are permissions (admin-managed via API)

New features:
- require_permission() and require_any_permission() dependency factories
- Action hierarchy: admin > editor > user > viewer
- Global admin override (admin.general:admin grants all)
- Group-role management endpoints (assign/remove roles)
- GET /auth/roles endpoint to list all roles

Database changes:
- Added category column to roles table (default: general)
- Removed authentik_group column (decoupled)
- Added group_roles association table
- Added user_groups association table
- Migration updates role names to domain.general:action format

Tests:
- 67 new tests for auth service and controller
- Covers token validation, user sync, role sync
- Covers group-role assignment/removal
- Covers schema conversions and permission system

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 19:52:10 +01:00
Jeroen SchweitzerandClaude Opus 4.5 075b0ec297 feat: add system stats API for dashboard monitoring
Build and Push / build (release) Successful in 1m44s
- GET /tools/system/stats - Real-time host system statistics
- CPU usage, memory, all mounted disks, network I/O
- GPU/VRAM stats via nvidia-smi (if available)
- Uses psutil for cross-platform host metrics
- Auto-discovers and filters real filesystems

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 17:30:45 +01:00
Jeroen SchweitzerandClaude Opus 4.5 49be935b5d feat: add link_type field to quick links
Build and Push / build (release) Successful in 1m10s
Adds link_type column to quick_links table for iframe vs new_tab behavior.
Includes Alembic migration and schema updates.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 14:38:10 +01:00
71 changed files with 6777 additions and 1614 deletions
+69
View File
@@ -0,0 +1,69 @@
{
"env": {
"PQL_VAULT": "/mnt/media/Projects/core-api"
},
"permissions": {
"allow": [
"Bash(pql)",
"Bash(pql *)",
"Bash(/home/jpmschweitzer/.local/bin/pql:*)",
"Bash(git status:*)",
"Bash(git log:*)",
"Bash(git diff:*)",
"Bash(git branch:*)",
"Bash(.venv/bin/python -m pytest:*)",
"Bash(.venv/bin/pytest:*)",
"Bash(pytest:*)",
"Bash(alembic *)",
"Bash(docker logs core-api:*)",
"Bash(curl -s http://localhost:8083/*)"
],
"deny": [
"Bash(/mnt/media/Projects/cladmin/ops/bin/toj)",
"Bash(/mnt/media/Projects/cladmin/ops/bin/toj:*)",
"Bash(alembic downgrade base*)",
"Bash(chmod -R 777 *)",
"Bash(chmod 777 *)",
"Bash(dd if=*)",
"Bash(find * -delete*)",
"Bash(find * -exec*)",
"Bash(git * add --all*)",
"Bash(git * add -A*)",
"Bash(git * add .)",
"Bash(git * branch -D *)",
"Bash(git * checkout -- *)",
"Bash(git * clean -fd*)",
"Bash(git * clean -fdx*)",
"Bash(git * commit --no-verify*)",
"Bash(git * merge --no-ff*)",
"Bash(git * push --force*)",
"Bash(git * push -f*)",
"Bash(git * reset --hard*)",
"Bash(git * restore .*)",
"Bash(git add --all*)",
"Bash(git add -A*)",
"Bash(git add .)",
"Bash(git branch -D *)",
"Bash(git checkout -- *)",
"Bash(git clean -fd*)",
"Bash(git clean -fdx*)",
"Bash(git commit --no-verify*)",
"Bash(git merge --no-ff*)",
"Bash(git push --force*)",
"Bash(git push -f*)",
"Bash(git reset --hard*)",
"Bash(git restore .*)",
"Bash(mkfs*)",
"Bash(psql * -c DROP*)",
"Bash(psql * DROP DATABASE*)",
"Bash(psql * TRUNCATE*)",
"Bash(rm -rf $HOME)",
"Bash(rm -rf /)",
"Bash(rm -rf ~)",
"Bash(su *)",
"Bash(sudo *)",
"Bash(toj)",
"Bash(toj:*)"
]
}
}
+1 -4
View File
@@ -39,12 +39,9 @@ HOMEASSISTANT_URL=http://localhost:8123
HOMEASSISTANT_TOKEN=your-long-lived-access-token HOMEASSISTANT_TOKEN=your-long-lived-access-token
# ============================================================================= # =============================================================================
# AI Services # Search
# ============================================================================= # =============================================================================
# Ollama API
OLLAMA_BASE_URL=http://localhost:11434
# SearXNG (self-hosted search) # SearXNG (self-hosted search)
SEARXNG_URL=http://localhost:8080 SEARXNG_URL=http://localhost:8080
+1
View File
@@ -0,0 +1 @@
.pql/changelog/*.sql merge=union
+18 -5
View File
@@ -1,19 +1,32 @@
name: Build and Push name: Build and Push
on: on:
release: push:
types: [published] tags:
- 'v*'
jobs: jobs:
release:
runs-on: ubuntu-latest
steps:
- name: Create Gitea Release
run: |
curl -sf -X POST \
-H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \
-H "Content-Type: application/json" \
-d '{"tag_name": "${{ github.ref_name }}", "name": "Release ${{ github.ref_name }}", "body": "Automated release for ${{ github.ref_name }}"}' \
"${{ github.server_url }}/api/v1/repos/${{ github.repository }}/releases"
build: build:
runs-on: ubuntu-latest runs-on: ubuntu-latest
needs: release
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- name: Login to Gitea Registry - name: Login to Gitea Registry
uses: docker/login-action@v3 uses: docker/login-action@v3
with: with:
registry: git.schweitz.internal registry: git.schweitz.net
username: ${{ secrets.REGISTRY_USER }} username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }} password: ${{ secrets.REGISTRY_PASSWORD }}
@@ -23,8 +36,8 @@ jobs:
context: . context: .
push: true push: true
tags: | tags: |
git.schweitz.internal/jpmschweitzer/core-api:latest git.schweitz.net/jpmschweitzer/core-api:latest
git.schweitz.internal/jpmschweitzer/core-api:${{ github.ref_name }} git.schweitz.net/jpmschweitzer/core-api:${{ github.ref_name }}
- name: Trigger Watchtower update - name: Trigger Watchtower update
if: success() if: success()
+13
View File
@@ -0,0 +1,13 @@
#!/usr/bin/env bash
# Trigger only. The checks live in the Makefile, where they can be read, run by
# hand (`make pre-push`), and changed under review.
#
# This file is identical in every repo in this workspace, deliberately: the call
# surface is the same everywhere even though what each gate runs is not, so
# nobody has to read a repo to find out how to check it (D-27).
#
# Enable per clone with: git config core.hooksPath .githooks
# Never bypass with --no-verify. Suppress a specific finding deliberately
# instead, with a reason — see `make pre-push`.
set -euo pipefail
exec make -C "$(git rev-parse --show-toplevel)" pre-push
+17
View File
@@ -136,3 +136,20 @@ Thumbs.db
# profiling data # profiling data
.prof .prof
# Claude Code local settings (machine-specific, may contain credentials)
.claude/settings.local.json
# pql — ignore everything except the changelog, which is the replication log of
# record and must be committed for tickets to travel with the repo.
.pql/*
!.pql/changelog/
# pql shims planted by `pql init` into the dir core.hooksPath points at.
# Per-clone: each embeds the absolute path of the pql binary that planted it.
# Only .githooks/pre-push is shared.
.githooks/pre-commit
.githooks/post-merge
.githooks/post-checkout
.githooks/post-rewrite
+11
View File
@@ -0,0 +1,11 @@
-- Changelog format marker, written by pql. Comments only: this file
-- is never executed — Import descends into the per-table directories
-- and does not read the changelog root.
--
-- A changelog carrying no marker is format 1, the shape that existed
-- before formats were versioned. An older format is migrated forward
-- by `pql plan upgrade` (and automatically from the post-merge hook);
-- a newer one is refused rather than replayed under rules this binary
-- does not know. See D-28 and docs/versions.md.
-- pql:changelog_format: 2.0.0
-- pql:written_by: 2.2.0
+139
View File
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
+139
View File
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
+139
View File
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
-72
View File
@@ -1,72 +0,0 @@
# AGENTS.md
> **Start every session by reading this file.**
> This file outlines the operational protocols, coding standards, and architectural decisions for this FastAPI project.
## 1. Agent Operational Protocols
### 🧠 Work Patterns (Plan-Act-Reflect)
* **Plan:** Before writing code, briefly outline your plan. Identify which files you will touch and what the side effects might be.
* **Act:** Execute the changes in small, atomic steps.
* **Reflect:** After coding, verify your work. Did you break existing tests? Did you add new tests?
### 🛡️ Git Discipline
* **NEVER commit to `main` or `master` directly.** Always create a feature branch: `feature/your-feature-name` or `fix/issue-description`.
* **Commit Messages:** Use the [Conventional Commits](https://www.conventionalcommits.org/) format.
* `feat: add user login endpoint`
* `fix: resolve database connection timeout`
* `refactor: split monolith dependency file`
* **Atomic Commits:** Keep commits small. One logical change = one commit.
### 📝 Changelog Maintenance
* **Update `CHANGELOG.md`** with every user-facing change.
* Format: `## [Unreleased] - YYYY-MM-DD` followed by `### Added`, `### Changed`, or `### Fixed`.
### 🚀 Release Flow
When changes are ready for deployment:
1. **Ask user if deploy cycle is desired **
2. **Update version** in `pyproject.toml`:
- Bug fixes: bump patch version (1.8.3 → 1.8.4)
- New features: bump minor version (1.8.4 → 1.9.0)
3. **Update CHANGELOG.md**:
- Move items from `[Unreleased]` to new version section
- Add release date: `## [1.8.4] - 2025-12-16`
4. **Commit and tag**:
```bash
git add -A
git commit -m "fix: description of changes"
git tag v1.8.4
git push origin main --tags
```
5. **CI/CD triggers automatically**:
- Gitea CI builds Docker image on new tag
- Watchtower pulls and deploys to production
- Verify deployment: `curl http://192.168.86.149:8000/health`
---
## 2. FastAPI Architecture & Best Practices
*Reference: [FastAPI Best Practices](https://github.com/zhanymkanov/fastapi-best-practices)*
### 📂 Project Structure (Directory-based, NOT File-type based)
Do **not** group files by type (e.g., one huge `routers` folder). Group by **domain/module** inside a `src/` directory.
**Correct Structure:**
```text
src/
├── auth/
│ ├── router.py # Endpoints
│ ├── schemas.py # Pydantic models
│ ├── service.py # Business logic (CRUD, etc.)
│ ├── dependencies.py# Module-specific dependencies
│ └── config.py # Module-specific settings
├── posts/
│ ├── router.py
│ └── ...
└── main.py # App entry point
+220
View File
@@ -5,6 +5,226 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [1.11.0] - 2026-01-08
### Added
- **News Headlines API** - New endpoint for news ticker integration
- `GET /tools/news` - Fetch news headlines from user's volatile collection
- Returns headlines with title, description, source, and URL
- Data sourced from `volatile_{user}` Qdrant collection (news namespace)
- Uses `preferred_username` from OIDC, falls back to `default`
- News subdomain under tools (`src/domains/tools/news/`)
- `NewsHeadline` and `NewsResponse` Pydantic schemas
- `NewsService` for parsing news data from Qdrant
## [1.10.12] - 2026-01-08
### Fixed
- **Forecast data retrieval** - Pass raw_data to service instead of extracting wrong field
- Qdrant client was extracting `days` (integer 7) instead of `daily` (list)
- Now passes full raw_data for service to parse correctly
## [1.10.11] - 2026-01-08
### Fixed
- **Weather wind direction** - Convert integer degrees to cardinal direction string
- Scheduler stores wind_direction as degrees (e.g., 135)
- Schema expects string (e.g., "SE")
- Added `_degrees_to_cardinal()` conversion
## [1.10.10] - 2026-01-08
### Fixed
- **Environment data parsing** - Fix parsing of scheduler-generated Qdrant data
- Forecast: Check `daily` key first (scheduler stores day count in `days`, list in `daily`)
- Sun times: Use `sunrise_iso`/`sunset_iso` fields, handle time-only format fallback
- Sun times: Calculate daylight from `daylight_duration_seconds` or `daylight_hours`
- Air quality: Support `aqi_us`/`aqi_european` and `nitrogen_dioxide` field names
## [1.10.9] - 2026-01-08
### Fixed
- **Environment user ID cleanup** - Strip email domain from user identifier
- If `preferred_username` is an email, extract just the username part
- Ensures Qdrant collection name matches (e.g., `volatile_jpmschweitzer` not `volatile_jpmschweitzer@gmail.com`)
## [1.10.8] - 2026-01-08
### Fixed
- **OIDC audience validation** - python-jose requires string audience, not list
- Extract and validate audience from unverified claims first
- Use token's actual audience for JWT decode (after validating it's allowed)
- Fixes "audience must be a string or None" error
## [1.10.7] - 2026-01-08
### Added
- **Multi-issuer OIDC support** - Accept tokens from multiple OAuth providers
- Changed `oidc_issuer` (string) to `oidc_issuers` (list)
- Each issuer has its own JWKS endpoint, now cached per-issuer
- Validates token issuer against allowed list before fetching JWKS
- Supports tokens from: `core-api`, `tatlock-ui`, `tatlock` OAuth applications
- Completes fix for environment endpoint user resolution
### Removed
- Deprecated `src/config.py` - consolidated to `src/shared/config.py`
- Deprecated `src/security.py` - consolidated to `src/shared/security.py`
## [1.10.6] - 2026-01-08
### Fixed
- **OIDC audience mismatch** - Accept tokens from multiple clients
- Changed `oidc_audience` (string) to `oidc_audiences` (list)
- Now accepts tokens with audience: `core-api`, `tatlock-ui`, or `tatlock`
- Fixes environment endpoint returning "default" user instead of authenticated username
- Fixed main.py to use `oidc_audiences[0]` for Swagger UI OAuth client
### Changed
- Documentation cleanup in README
## [1.10.4] - 2026-01-07
### Removed
- **Ollama integration removed** - AI inference is no longer handled by this API
- Removed `src/models/ollama_client.py` and all Ollama-related configuration
- Removed `src/models/embeddings.py` and `src/models/embeddings_ollama.py`
- Removed model aliases and AI configuration from settings
- Health endpoints no longer check Ollama status
- Tests updated to reflect database-only health checks
### Changed
- Health check `/health/full` now only checks database connectivity
- Diagnostics endpoint simplified (removed Ollama component info)
## [1.10.3] - 2026-01-07
### Fixed
- **OIDC config not applied to domains module** - Both `src.auth.oidc` and `src.domains.auth.oidc` configs are now initialized
- Previously only `src.auth.oidc` was configured, leaving domains tools using hardcoded "local" user
- Environment endpoint now correctly uses authenticated user from OIDC token
## [1.10.2] - 2026-01-07
### Changed
- **Enhanced environment endpoint logging** - Added detailed user claim logging for debugging
- Logs both `preferred_username` and `sub` claims when resolving user
- Distinguishes between authenticated and unauthenticated requests
## [1.10.1] - 2026-01-06
### Fixed
- Fix OIDC import path in tools controller (`src.oidc.dependencies``src.auth.oidc`)
- Environment endpoint was returning `user: "local"` instead of authenticated username
- Caused queries to wrong Qdrant collection (`volatile_local` vs `volatile_{username}`)
## [1.10.0] - 2026-01-06
### Added
- **Environment Data API** - Qdrant-backed endpoint for weather, forecast, and sun position data
- `GET /tools/environment` - Fetch environment data from user's volatile collection
- Weather: current temperature, conditions, humidity, wind speed
- Forecast: multi-day outlook with high/low temperatures
- Sun times: sunrise, sunset, daylight duration
- Air quality: AQI and quality level (when available)
- Data sourced from `volatile_{user}` Qdrant collection
- Uses `preferred_username` from OIDC, falls back to `default`
- `qdrant-client` dependency for vector database access
- `QdrantReadClient` wrapper for read-only collection queries
- Comprehensive test suite for environment service parsing
## [1.9.4] - 2026-01-04
### Fixed
- Fix SQLAlchemy async lazy loading error for new users in `/auth/sync`
- Initialize `user.roles = []` and `user.preferences` to avoid greenlet error
- Was causing "MissingGreenlet: greenlet_spawn has not been called" on new user creation
## [1.9.3] - 2026-01-04
### Fixed
- Handle non-UUID `sub` claim in `/auth/sync` - Authentik JWT may return non-UUID subject identifiers
- Now derives deterministic UUID from sub string if direct parsing fails
## [1.9.2] - 2026-01-04
### Fixed
- `/auth/users/me` endpoint now supports both NPM forward auth headers AND JWT Bearer tokens
- Added `get_current_user_or_forward_auth()` combined auth dependency
- Fixes web authentication where NPM passes `X-authentik-*` headers instead of JWT
- Mobile/native clients continue to use JWT Bearer tokens as before
## [1.9.1] - 2026-01-03
### Fixed
- CORS configuration now uses explicit origins instead of `"*"`
- When `allow_credentials=True`, wildcard origins are rejected by browsers
- Added `home.schweitz.net`, `tatlock.schweitz.net`, and localhost origins
## [1.9.0] - 2026-01-03
### Added
- **NPM Forward Auth Support** - Web authentication via Nginx Proxy Manager forward auth
- `GET /auth/me` - Get current user from NPM forward auth headers (X-authentik-uid, X-authentik-email, etc.)
- Auto-creates user on first web login if not in database
- Syncs roles from NPM forward auth groups header
- `get_user_by_email` and `get_user_by_authentik_id` methods in AuthService
- Comprehensive tests for `/auth/me` endpoint
## [1.8.0] - 2026-01-03
### Added
- **Group-Role Mapping & Permissions** (Phase 3)
- Decoupled group-role architecture (groups from Authentik, roles admin-managed)
- Permission format: `domain.category:action` with action hierarchy
- `require_permission` and `require_any_permission` dependency factories
- Global admin override (`admin.general:admin`)
- **User Profile & API Keys** (Phase 4)
- `GET /auth/users/me` - Full user profile with roles and preferences
- `GET/PATCH /auth/users/me/preferences` - User preferences management
- `GET/POST/DELETE /auth/users/me/api-keys` - API key lifecycle
- API keys with `tak_` prefix, SHA-256 hashing, shown only once on creation
## [1.7.0] - 2026-01-03
### Added
- **System Stats API** - Host system resource monitoring for dashboard widgets
- `GET /tools/system/stats` - Real-time host system statistics
- CPU: usage percentage, core count, load averages
- Memory: usage percentage, total/used/available bytes
- Disks: all mounted filesystems with usage stats (auto-discovers mounts)
- Network: total bytes sent/received
- GPU/VRAM: NVIDIA GPU memory usage (via nvidia-smi if available)
- `psutil` dependency for cross-platform system metrics
## [1.6.1] - 2026-01-03
### Added
- `link_type` field to quick links for iframe vs new tab behavior
## [1.6.0] - 2026-01-03 ## [1.6.0] - 2026-01-03
### Added ### Added
+177
View File
@@ -0,0 +1,177 @@
# CLAUDE.md — core-api
FastAPI service providing infrastructure management, home automation, and utility
endpoints for the homelab. Talks to Portainer, Nginx Proxy Manager, Home Assistant,
Postgres (via SQLAlchemy async + Alembic), Qdrant, and Authentik (OIDC). Deployed on
tower-of-joy at **:8083**.
## Ports — these differ, deliberately
| | Port | How |
|---|---|---|
| Local dev | **8788** | `./wakeup.sh`, uvicorn `--reload`, logs to `logs/server.log` |
| Production | **8083** | container; health at `http://192.168.86.149:8083/health` |
Testing `localhost:8083` on the dev box hits the *container*, not your reload server.
## Live contract
The live contract is always `http://localhost:8083/openapi.json` (62 paths, verified
2026-08-09) and human docs at `http://localhost:8083/docs` / `/redoc` — generated from
running code, so query it rather than inferring routes from source or from the README's
endpoint list, which can drift.
## Architecture
Domain-first layout under `src/domains/<name>/{controller,models,schemas,service}.py`
(auth, dashboard, health, housekeeping, infrastructure, static, tools). `src/main.py`
wires only `src.domains.*` — verify by reading its imports.
**Legacy top-level packages — "not in `main.py`" does not mean dead.** Routes are wired
only from `src.domains.*`, so grepping `main.py`'s imports looks like it settles which
packages are live. It does not. `main.py:55` calls `initialize_oidc()`, and that function
(`src/shared/security.py:21`) deliberately imports and configures **both** `src.auth.oidc`
and `src.domains.auth.oidc` — a function-body import, invisible to a grep of `main.py`.
That one call drags in `src/auth/`, `src/controllers/`, `src/db/`, `src/logging_config.py`
and `src/base_schema.py` at startup.
Three tiers, established by importing the app inside the container and reading
`sys.modules` (verified 2026-08-09):
| Tier | Packages |
|---|---|
| Serving routes | `src/domains/`, `src/shared/`, `src/service_groups/` |
| **Loaded and configured**, but serving no routes | `src/auth/`, `src/db/`, `src/controllers/`, `src/logging_config.py`, `src/base_schema.py` |
| Genuinely unreferenced | `src/agent/`, `src/api/`, `src/clients/`, `src/dns/`, `src/memory/`, `src/models/` |
`src/auth/` is the trap. Its `oidc_config` singleton is configured at every startup with
the real Authentik issuers — the log line `src.auth.oidc:configure` proves it — so a test
importing `src.auth.oidc` is exercising live, configured code, not a fossil. No `src/domains/*`
module depends on it, so it is configured defensively rather than used; that makes it a
deletion candidate, but a considered one, not obvious cleanup.
**Before deleting anything from `src/`, import the app and read `sys.modules`** rather than
grepping `main.py`. Function-body imports exist here specifically to dodge circular imports,
and they are exactly what a grep misses.
**That check has its own blind spot, so do not read the third tier as a delete list.** The
table above is a snapshot taken after a cold `import src.main` — it shows what *startup*
loads. A module imported inside a request handler would be absent from it while being
entirely live, and absence would then be a timing artifact rather than evidence of death.
This bit on webber, where a tool package imported from inside an agent method looked
unloaded and was serving every request. Nothing in core-api is currently known to work that
way, but that is the weaker claim — it means nobody has exercised the routes and re-checked,
not that nobody does it. Before deleting a third-tier package, drive the endpoints that
would plausibly load it and take the snapshot again.
Some tests (`test_auth_controller.py`, `test_oidc.py`, `test_npm_client.py`,
`test_portainer_client.py`, `test_static_controller.py`, `test_tools_controller.py`) import
from the top-level paths rather than `src.domains.*`. Which of those cover live code follows
the table above — `test_oidc.py` does; the client tests target the unreferenced tier. Not
cleaned up in this pass; flagged, not fixed.
Shared infra (config, database, logging, security/OIDC, external API clients) lives in
`src/shared/`.
Group new work by **domain, not by file type** — a single large `routers/` folder is
the thing to avoid. Reference: [FastAPI best practices](https://github.com/zhanymkanov/fastapi-best-practices).
## Database
SQLAlchemy 2.0 async + asyncpg, migrations via Alembic (`alembic/versions/`). Models
live under `src/domains/<name>/models.py` and must be imported in `alembic/env.py` to
register with `Base.metadata` — check that file when adding a new model or `alembic
revision --autogenerate` will silently miss it.
## Working here
**Plan, act, reflect.** Outline which files you will touch and the side effects before
writing. Change in small atomic steps. Afterwards, verify: did existing tests break, and
does the new behaviour have a test?
**Test locally first — the build-deploy loop is slow.** `./wakeup.sh` auto-reloads on
code changes (not on `requirements.txt` changes; restart the container/script after
adding a dependency). Deploy only when a feature is complete and tested.
Run tests through the venv explicitly, to avoid environment mismatch:
```bash
.venv/bin/python -m pytest tests/
# or, with coverage:
.venv/bin/python -m pytest --cov=src --cov-report=term-missing
```
Copy `.env.example` to `.env` and configure Portainer, NPM, Home Assistant, SearXNG,
Postgres, and Qdrant hosts/credentials.
No linter is configured in this repo (no ruff/flake8 config, none in `requirements.txt`
or `dev-requirements.txt`) — unlike some sibling repos, don't assume `ruff check` exists
here.
## CI
`.gitea/workflows/build.yml` is the only workflow: triggered on `v*` tag push, it
creates a Gitea release, builds and pushes the image, then pings Watchtower. There is
**no CI test/lint gate** — pytest only runs locally or on request. Verify tests pass
before tagging a release.
## Work tracking
Work lives in **pql**, not a markdown TODO. **This repo's vault is standalone** — its tickets
and its internal decisions live here in `.pql/` and `governance/`, and travel with a clone,
because `.pql/changelog/` is committed and replayed by the git hooks (workspace D-15). The databases are
gitignored and rebuildable with `pql plan rebuild`.
`pql` is **not** on the non-interactive `PATH` — invoke it as
`/home/jpmschweitzer/.local/bin/pql`. From inside this repo no `--vault` is needed: pql anchors
at the nearest `.git/` ancestor, which is this repo.
```bash
/home/jpmschweitzer/.local/bin/pql ticket list # this repo's open work
/home/jpmschweitzer/.local/bin/pql plan whatsnext # next unblocked item, with context
/home/jpmschweitzer/.local/bin/pql decisions list # this repo's own decisions
```
Stack-level decisions that constrain this service live in the **workspace** vault and need the
flag:
```bash
/home/jpmschweitzer/.local/bin/pql --vault /mnt/media/Projects decisions list --domain core-api
```
Note `ticket new --decision D-N` resolves ids within **one** vault, so a ticket here cannot link
to a workspace decision. Cite the id in the ticket body instead.
Do not add a TODO section to a markdown file.
## Git
- **History is linear — no merge commits.** Work on `main`, or a short-lived branch
that is fast-forwarded and deleted. (This repo's AGENTS.md previously mandated a
feature branch for every change; that rule was retired workspace-wide on 2026-08-08
and does not apply here anymore.)
- **Conventional Commits**: `feat:`, `fix:`, `refactor:`, `docs:`, `chore:`.
- **Atomic commits** — one logical change each.
- **Stage explicitly. Never `git add -A`** — it is denied by policy, and it sweeps in
whatever else is dirty, including secrets.
- Update `CHANGELOG.md` with every user-facing change, under `[Unreleased]` in `Added` /
`Changed` / `Fixed`.
## Releasing
Ask whether a deploy is wanted first — it is not automatic.
1. Bump the version in `pyproject.toml` (patch for fixes, minor for features).
2. Move `[Unreleased]` entries into a dated version section in `CHANGELOG.md`.
3. Stage the changed files by name, commit, tag `vX.Y.Z`, `git push origin main --tags`.
4. Gitea CI (`build.yml`) builds and pushes the image on the tag; Watchtower deploys it.
5. Verify: `curl http://192.168.86.149:8083/health`.
## Security
- OIDC authentication via Authentik, multi-issuer/multi-audience support.
- Admin endpoints require authentication when `OIDC_ENABLED=true`.
- README claims the container "runs as non-root user (uid 1000)" — **checked and
false**: the Dockerfile has no `USER` directive, so the container runs as root.
Not fixed here (out of scope for a docs normalization pass); flagging so it isn't
restated as fact.
+67
View File
@@ -0,0 +1,67 @@
# core-api — the repo's command surface (D-27).
#
# Paths resolve here rather than in callers (D-10). `python3` on this host is
# 3.8 and cannot parse these sources, and a bare `pytest` resolves only in a
# login shell — so both are named explicitly through the venv.
VENV := $(CURDIR)/.venv
PYTHON ?= python3.12
.DEFAULT_GOAL := help
.PHONY: help
help: ## Show this help
@grep -hE '^[a-z][a-z0-9_-]*:.*?## ' $(MAKEFILE_LIST) \
| awk 'BEGIN{FS=":.*?## "}{printf " \033[36m%-14s\033[0m %s\n", $$1, $$2}'
.PHONY: setup
setup: ## Create the venv, install dependencies, and prove the result is usable
$(PYTHON) -m venv .venv
$(VENV)/bin/pip install -r requirements.txt -r dev-requirements.txt
# Exit 0 from pip install is not evidence (D-24) — it is the step's job not
# to fail, so a broken result and a working one look identical from here.
# On 2026-08-09 the venv existed and pip had exited 0, but sqlalchemy was
# declared in requirements.txt and not installed; that surfaced as 11
# pytest collection errors that read as broken imports, not as a setup
# problem. `--collect-only` exercises every import the suite touches
# without running a single test, so it catches exactly that class of
# drift and stays cheap. `pip check` was considered too, but it only
# verifies the *installed* set's internal consistency against itself —
# it would not have caught this case, because sqlalchemy was still
# present as another package's transitive dependency even when dropped
# from requirements.txt. collect-only checks declared-vs-actually-usable
# directly, which is the axis that broke.
$(VENV)/bin/python -m pytest --collect-only tests/
.PHONY: test
test: ## Run the test suite
@test -x $(VENV)/bin/python || { echo "FAIL — no venv; run: make setup"; exit 69; }
$(VENV)/bin/python -m pytest tests/
# No `lint` target, deliberately. This repo configures no linter — no ruff or
# flake8 config, and neither in requirements. Per D-27 the name is reserved for
# repos that lint; an empty target here would report clean for something never
# run. Add the target when a linter is added, not before.
# git hands a hook a non-login shell, which never sees ~/.local/bin — where
# gitleaks lands. Without this the scan reports "not installed" on every push,
# which is a check that fails open (D-24).
export PATH := $(HOME)/.local/bin:/usr/local/bin:$(PATH)
.PHONY: secrets
secrets: ## Scan the commits about to be pushed for credentials
@ci/secrets.sh
# The call surface is identical in every repo; what it runs is not.
#
# `secrets` runs first, deliberately: it is the only failure here that cannot be
# undone by fixing it afterwards. A failed lint costs another commit; a pushed
# credential is cached and indexed whether or not it is later deleted.
#
# Some of these fail today, and are left wired anyway. The state was measured
# once and written down in T-56 rather than being worked around here — a gate
# quietly narrowed to what already passes is a gate that reports success for
# doing nothing, which is the failure this workspace keeps rediscovering.
.PHONY: pre-push
pre-push: secrets ## Everything the pre-push hook runs
@echo " -- not gated here yet: lint (no linter configured) and test (T-56)"
+6 -17
View File
@@ -19,8 +19,7 @@ Central API service providing infrastructure management, home automation, and ut
### Utilities ### Utilities
- **DNS Lookup**: Query DNS records (A, AAAA, MX, TXT, CNAME, NS, SOA, PTR) - **DNS Lookup**: Query DNS records (A, AAAA, MX, TXT, CNAME, NS, SOA, PTR)
- **Health Checks**: Comprehensive service health monitoring - **Health Checks**: Service health monitoring with database connectivity status
- **AI Metrics Proxy**: Forward metrics requests to Core-AI service
## Architecture ## Architecture
@@ -35,10 +34,8 @@ src/
├── clients/ ├── clients/
│ ├── homeassistant_client.py # Home Assistant REST client │ ├── homeassistant_client.py # Home Assistant REST client
│ ├── npm_client.py # Nginx Proxy Manager client │ ├── npm_client.py # Nginx Proxy Manager client
│ ├── ollama_client.py # Ollama LLM client
│ └── portainer_client.py # Portainer API client │ └── portainer_client.py # Portainer API client
├── controllers/ ├── controllers/
│ ├── ai_controller.py # AI metrics proxy
│ ├── health_controller.py # Health endpoints │ ├── health_controller.py # Health endpoints
│ ├── housekeeping_controller.py # Home automation endpoints │ ├── housekeeping_controller.py # Home automation endpoints
│ ├── infrastructure_controller.py # Infrastructure management │ ├── infrastructure_controller.py # Infrastructure management
@@ -88,9 +85,6 @@ src/
### Tools (`/tools`) ### Tools (`/tools`)
- `POST /tools/dns/lookup` - DNS record lookup - `POST /tools/dns/lookup` - DNS record lookup
### AI (`/ai`)
- `GET /ai/metrics` - Proxy to Core-AI metrics
## Development ## Development
### Requirements ### Requirements
@@ -103,10 +97,6 @@ src/
# Install dependencies # Install dependencies
pip install -r requirements.txt pip install -r requirements.txt
# Copy credentials template
cp src/credentials.example.py src/credentials.py
# Edit src/credentials.py with your values
# Run locally # Run locally
uvicorn src.main:app --reload --host 0.0.0.0 --port 8083 uvicorn src.main:app --reload --host 0.0.0.0 --port 8083
``` ```
@@ -169,10 +159,9 @@ docker run -p 8083:8083 core-code:latest
| `NPM_PASSWORD` | NPM admin password | - | | `NPM_PASSWORD` | NPM admin password | - |
| `HOMEASSISTANT_URL` | Home Assistant URL | `http://localhost:8123` | | `HOMEASSISTANT_URL` | Home Assistant URL | `http://localhost:8123` |
| `HOMEASSISTANT_TOKEN` | HA long-lived access token | - | | `HOMEASSISTANT_TOKEN` | HA long-lived access token | - |
| `OLLAMA_URL` | Ollama API URL | `http://localhost:11434` |
| `OIDC_ENABLED` | Enable OIDC auth | `false` | | `OIDC_ENABLED` | Enable OIDC auth | `false` |
| `OIDC_ISSUER` | OIDC issuer URL | - | | `OIDC_ISSUERS` | OIDC issuer URLs (comma-separated) | See config.py |
| `OIDC_AUDIENCE` | OIDC audience | - | | `OIDC_AUDIENCES` | OIDC audiences (comma-separated) | See config.py |
## API Documentation ## API Documentation
@@ -183,9 +172,9 @@ Once deployed, access documentation at:
## Health Checks ## Health Checks
- **Basic**: `GET /health` - Returns status and Ollama connection - **Basic**: `GET /health` - Fast liveness check for container orchestration
- **Full**: `GET /health/full` - Returns all component statuses (503 if unhealthy) - **Full**: `GET /health/full` - Returns database status (503 if unhealthy)
- **Diagnostics**: `GET /health/diagnostics` - Detailed service information - **Diagnostics**: `GET /health/diagnostics` - Service info and configuration
## Security ## Security
@@ -0,0 +1,26 @@
"""add_link_type_to_quick_links
Revision ID: f0349c95aa5d
Revises: 003
Create Date: 2026-01-03 13:14:46.911770+00:00
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = 'f0349c95aa5d'
down_revision: Union[str, None] = '003'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column('quick_links', sa.Column('link_type', sa.String(length=20), nullable=True, server_default='iframe'))
# Update existing rows to have the default value
op.execute("UPDATE quick_links SET link_type = 'iframe' WHERE link_type IS NULL")
def downgrade() -> None:
op.drop_column('quick_links', 'link_type')
@@ -0,0 +1,141 @@
"""Add group_roles mapping and update role schema
Revision ID: 004
Revises: f0349c95aa5d
Create Date: 2026-01-03
Changes:
- Add category column to roles (default 'general')
- Drop authentik_group column from roles (decoupled architecture)
- Create user_groups association table
- Create group_roles association table
- Update role names from domain:action to domain.general:action
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql
# revision identifiers, used by Alembic.
revision: str = "004"
down_revision: Union[str, None] = "f0349c95aa5d"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Add category column to roles
op.add_column(
"roles",
sa.Column(
"category",
sa.String(50),
nullable=False,
server_default="general",
comment="Permission category within domain (general for full access, or specific tool)",
),
)
# Update role names from domain:action to domain.general:action
op.execute(
"""
UPDATE roles
SET name = REPLACE(name, ':', '.general:')
WHERE name NOT LIKE '%.%:%'
"""
)
# Update the comment on the name column
op.alter_column(
"roles",
"name",
comment="Role name in format domain.category:action (e.g., control-room.general:admin)",
)
# Drop the authentik_group unique index first
op.drop_index("ix_roles_authentik_group", table_name="roles")
# Drop authentik_group column (no longer needed with group_roles mapping)
op.drop_column("roles", "authentik_group")
# Create user_groups association table
op.create_table(
"user_groups",
sa.Column(
"user_id",
postgresql.UUID(as_uuid=True),
sa.ForeignKey("users.id", ondelete="CASCADE"),
primary_key=True,
),
sa.Column(
"group_id",
postgresql.UUID(as_uuid=True),
sa.ForeignKey("groups.id", ondelete="CASCADE"),
primary_key=True,
),
)
# Create group_roles association table
op.create_table(
"group_roles",
sa.Column(
"group_id",
postgresql.UUID(as_uuid=True),
sa.ForeignKey("groups.id", ondelete="CASCADE"),
primary_key=True,
),
sa.Column(
"role_id",
postgresql.UUID(as_uuid=True),
sa.ForeignKey("roles.id", ondelete="CASCADE"),
primary_key=True,
),
)
def downgrade() -> None:
# Drop association tables
op.drop_table("group_roles")
op.drop_table("user_groups")
# Add back authentik_group column
op.add_column(
"roles",
sa.Column(
"authentik_group",
sa.String(255),
nullable=True,
comment="Corresponding Authentik group name",
),
)
# Restore authentik_group values from role names
op.execute(
"""
UPDATE roles
SET authentik_group = 'tatlock-' || REPLACE(REPLACE(name, '.general:', '-'), ':', '-')
"""
)
# Recreate the unique index
op.create_index("ix_roles_authentik_group", "roles", ["authentik_group"], unique=True)
# Revert role names from domain.general:action to domain:action
op.execute(
"""
UPDATE roles
SET name = REPLACE(name, '.general:', ':')
WHERE name LIKE '%.general:%'
"""
)
# Update the comment on the name column
op.alter_column(
"roles",
"name",
comment="Role name in format domain:action",
)
# Drop category column
op.drop_column("roles", "category")
Executable
+50
View File
@@ -0,0 +1,50 @@
#!/usr/bin/env bash
# Secret scan over the commits about to be pushed.
#
# Lives here rather than inside .githooks/pre-push so it can be read, run by
# hand (`make secrets`), and changed under review. A hook is a trigger; it is
# not a home for logic. Identical in every repo in this workspace (D-27).
set -euo pipefail
cd "$(git rev-parse --show-toplevel)"
# A non-login shell — which is what git gives a hook — skips /etc/profile.d
# and never sees ~/.local/bin, where the gitleaks release tarball lands.
# Without this the scan reports "not installed" on every push.
[ -d "$HOME/.local/bin" ] && PATH="$HOME/.local/bin:$PATH"
if ! command -v gitleaks >/dev/null 2>&1; then
echo "FAIL secrets — gitleaks not installed, so this check would be a no-op pretending to pass." >&2
echo " https://github.com/gitleaks/gitleaks/releases → ~/.local/bin/gitleaks" >&2
exit 1
fi
# Scan the outgoing range, not full history. History here carries findings
# that are settled — test fixtures and vendored third-party code — and a gate
# that fails on something unfixable gets bypassed within a week. What matters
# is what is about to leave this machine.
if upstream=$(git rev-parse --abbrev-ref --symbolic-full-name '@{u}' 2>/dev/null); then
range="$upstream..HEAD"
elif git rev-parse --verify --quiet origin/main >/dev/null; then
range="origin/main..HEAD"
else
range=""
fi
if [ -z "$range" ]; then
gitleaks dir . --redact --no-banner --exit-code 1 || {
echo "FAIL secrets — gitleaks found a credential in the working tree." >&2; exit 1; }
exit 0
fi
[ -n "$(git log --oneline "$range" 2>/dev/null)" ] || exit 0
gitleaks git . --log-opts="$range" --redact --no-banner --exit-code 1 >/dev/null 2>&1 || {
echo "FAIL secrets — gitleaks found a credential in the commits being pushed." >&2
echo " inspect (values redacted): gitleaks git . --log-opts=\"$range\" --redact" >&2
echo " then remove and rotate it, or suppress deliberately:" >&2
echo " inline '# gitleaks:allow <reason>'" >&2
echo " or add the fingerprint to .gitleaksignore WITH a reason" >&2
exit 1
}
echo " ok secrets"
+54
View File
@@ -0,0 +1,54 @@
# Decisions, Questions, Rejected
This directory holds structured planning records that pql parses
into pql.db. Each record is a `### [DQR]-N: Title` heading inside
a markdown file. Files live in three per-type subdirectories:
- `decisions/<domain>.md` — confirmed design decisions
- `questions/<domain>.md` — open questions that may resolve into
decisions or rejected proposals
- `rejected/<domain>.md` — rejected proposals (kept for the audit
trail)
The parser infers domain from the filename stem and record type
from the parent subdirectory.
D-records that propose implementation work link to `initiative`-type
tickets via `decision_ref`. Run `pql decisions show <id>
--with-tickets` to inspect implementation status.
## Recommended domains
Start with this canonical set; create files as records land in
each domain:
- **architecture** — structural commitments (storage, layering,
languages, libraries)
- **process** — team workflow (commits, branches, releases, reviews)
- **design** — user-facing surface (UX, UI, public APIs)
- **coding-conventions** — team-internal code shape (style, lint,
file layout)
- **testing** — quality strategy (coverage, layers, gates)
You might also want, project-permitting:
- `accessibility` — if you ship user-facing software
- `security` — if you handle user data or network surfaces
- `licensing` — if you release open-source or commercial
- `documentation` — if user-docs are non-trivial
- `deployment` — if shipping is non-trivial
- `performance` — if you have perf budgets / SLOs
<!-- pql:records (auto-generated; do not edit manually) -->
## Decisions
- _(none)_
## Open questions
- _(none)_
## Rejected
- _(none)_
+12
View File
@@ -0,0 +1,12 @@
proxy_buffers 8 16k;
proxy_buffer_size 32k;
# CORS headers for Flutter web
add_header Access-Control-Allow-Origin "https://home.schweitz.net" always;
add_header Access-Control-Allow-Credentials true always;
add_header Access-Control-Allow-Methods "GET, POST, PUT, DELETE, PATCH, OPTIONS" always;
add_header Access-Control-Allow-Headers "Content-Type, Authorization" always;
if ($request_method = OPTIONS) {
return 204;
}
+1 -1
View File
@@ -1,6 +1,6 @@
[project] [project]
name = "core-api" name = "core-api"
version = "1.6.0" version = "1.11.0"
description = "Core Code API - Infrastructure management and tools API" description = "Core Code API - Infrastructure management and tools API"
readme = "README.md" readme = "README.md"
requires-python = ">=3.12" requires-python = ">=3.12"
+4
View File
@@ -21,6 +21,7 @@ python-dotenv~=1.0.0
python-json-logger~=2.0.0 python-json-logger~=2.0.0
pytz~=2024.1 pytz~=2024.1
dnspython~=2.7.0 dnspython~=2.7.0
psutil~=6.1.0
# Authentication & Security # Authentication & Security
PyJWT[crypto]>=2.9.0 PyJWT[crypto]>=2.9.0
@@ -31,3 +32,6 @@ cryptography>=44.0.1 # CVE-2024-12797
sqlalchemy[asyncio]~=2.0.0 sqlalchemy[asyncio]~=2.0.0
asyncpg>=0.30.0 asyncpg>=0.30.0
alembic~=1.13.0 alembic~=1.13.0
# Vector Database
qdrant-client>=1.9.0
+69 -6
View File
@@ -13,6 +13,7 @@ from src.logging_config import get_logger
from src.db import get_async_session from src.db import get_async_session
from src.auth.schemas import AuthSyncRequest, AuthSyncResponse, UsersListResponse, BulkSyncResultSchema, GroupsListResponse from src.auth.schemas import AuthSyncRequest, AuthSyncResponse, UsersListResponse, BulkSyncResultSchema, GroupsListResponse
from src.auth.service import AuthService from src.auth.service import AuthService
from src.auth.oidc import get_forward_auth_user
logger = get_logger(__name__) logger = get_logger(__name__)
@@ -222,21 +223,83 @@ class AuthController(BaseController):
responses={ responses={
200: {"description": "User profile"}, 200: {"description": "User profile"},
401: {"description": "Not authenticated"}, 401: {"description": "Not authenticated"},
404: {"description": "User not found in database"},
}, },
) )
async def get_me( async def get_me(
forward_auth_user: Optional[dict] = Depends(get_forward_auth_user),
session: AsyncSession = Depends(get_async_session), session: AsyncSession = Depends(get_async_session),
) -> JSONResponse: ) -> AuthSyncResponse:
""" """
Get the current authenticated user's profile Get the current authenticated user's profile
Note: This endpoint requires a valid session or API key. Authentication is handled by NPM forward auth with Authentik.
For now, returns 501 Not Implemented until session management is added. The proxy sets X-authentik-* headers which this endpoint reads.
For internal/LAN access (no forward auth headers), returns 401.
Use POST /auth/sync with an OIDC token for mobile app authentication.
""" """
# TODO: Implement with get_current_user dependency # Require forward auth for this endpoint
if forward_auth_user is None:
raise HTTPException( raise HTTPException(
status_code=501, status_code=401,
detail="Not implemented - use /auth/sync with access token", detail="Authentication required - access via authenticated proxy or use /auth/sync",
)
service = AuthService(session)
# Try to find user by Authentik UID first, then by email
user = None
uid = forward_auth_user.get("uid")
if uid:
try:
import uuid
authentik_id = uuid.UUID(uid)
user = await service.get_user_by_authentik_id(authentik_id)
except (ValueError, TypeError):
pass # Invalid UUID, try email
if user is None:
email = forward_auth_user.get("email")
if email:
user = await service.get_user_by_email(email)
if user is None:
# User authenticated with Authentik but not synced to database yet
# This can happen on first login via web
logger.info(f"User {forward_auth_user.get('email')} not found, creating from forward auth")
# Create user from forward auth headers
from src.auth.schemas import TokenInfoSchema
token_info = TokenInfoSchema(
sub=forward_auth_user.get("uid", ""),
email=forward_auth_user.get("email", ""),
name=forward_auth_user.get("name"),
groups=forward_auth_user.get("groups", []),
)
try:
user, _ = await service.sync_user(token_info)
await service.sync_roles(user, token_info.groups)
await session.commit()
await session.refresh(user, ["preferences", "roles"])
except Exception as e:
logger.error(f"Failed to create user from forward auth: {e}")
raise HTTPException(
status_code=500,
detail="Failed to create user profile",
)
# Sync roles from current groups (in case they changed)
groups = forward_auth_user.get("groups", [])
roles = await service.sync_roles(user, groups)
await session.commit()
return AuthSyncResponse(
user=service.user_to_schema(user),
roles=service.roles_to_schema(roles),
preferences=service.preferences_to_schema(user.preferences),
is_new_user=False,
) )
return router return router
+60 -23
View File
@@ -22,29 +22,42 @@ class OIDCConfig:
def __init__(self): def __init__(self):
# These will be set from environment variables in config.py # These will be set from environment variables in config.py
self.enabled = False self.enabled = False
self.issuer = "" self.issuers: list[str] = []
self.audience = "" self.audiences: list[str] = []
self.jwks_uri = ""
def configure(self, enabled: bool, issuer: str, audience: str): def configure(self, enabled: bool, issuers: list[str], audiences: list[str]):
"""Configure OIDC settings""" """Configure OIDC settings"""
self.enabled = enabled self.enabled = enabled
self.issuer = issuer self.issuers = [iss.rstrip('/') for iss in issuers] # Normalize without trailing slash
self.audience = audience self.audiences = audiences
self.jwks_uri = f"{issuer.rstrip('/')}/jwks/" logger.info(f"OIDC configured: enabled={enabled}, issuers={self.issuers}, audiences={audiences}")
logger.info(f"OIDC configured: enabled={enabled}, issuer={issuer}")
def get_jwks_uri(self, issuer: str) -> str:
"""Get JWKS URI for a specific issuer"""
return f"{issuer.rstrip('/')}/jwks/"
def is_valid_issuer(self, issuer: str) -> bool:
"""Check if issuer is in the allowed list"""
normalized = issuer.rstrip('/')
return normalized in self.issuers
# Global OIDC config instance # Global OIDC config instance
oidc_config = OIDCConfig() oidc_config = OIDCConfig()
@lru_cache(maxsize=1) # Per-issuer JWKS cache
def get_jwks() -> Dict: _jwks_cache: Dict[str, Dict] = {}
"""
Fetch JSON Web Key Set (JWKS) from Authentik
Cached to avoid repeated requests. Cache is cleared on server restart.
def get_jwks_for_issuer(issuer: str) -> Dict:
"""
Fetch JSON Web Key Set (JWKS) for a specific issuer.
Cached per-issuer to avoid repeated requests. Cache is cleared on server restart.
Args:
issuer: The token issuer URL
Returns: Returns:
JWKS dictionary containing public keys for token verification JWKS dictionary containing public keys for token verification
@@ -55,15 +68,24 @@ def get_jwks() -> Dict:
if not oidc_config.enabled: if not oidc_config.enabled:
return {} return {}
normalized_issuer = issuer.rstrip('/')
# Return cached JWKS if available
if normalized_issuer in _jwks_cache:
return _jwks_cache[normalized_issuer]
jwks_uri = oidc_config.get_jwks_uri(normalized_issuer)
try: try:
logger.debug(f"Fetching JWKS from {oidc_config.jwks_uri}") logger.debug(f"Fetching JWKS from {jwks_uri}")
response = httpx.get(oidc_config.jwks_uri, timeout=10.0) response = httpx.get(jwks_uri, timeout=10.0)
response.raise_for_status() response.raise_for_status()
jwks = response.json() jwks = response.json()
logger.info(f"JWKS fetched successfully ({len(jwks.get('keys', []))} keys)") logger.info(f"JWKS fetched successfully for {normalized_issuer} ({len(jwks.get('keys', []))} keys)")
_jwks_cache[normalized_issuer] = jwks
return jwks return jwks
except Exception as e: except Exception as e:
logger.error(f"Failed to fetch JWKS: {e}") logger.error(f"Failed to fetch JWKS from {jwks_uri}: {e}")
raise HTTPException( raise HTTPException(
status_code=503, status_code=503,
detail="Authentication service unavailable" detail="Authentication service unavailable"
@@ -109,6 +131,21 @@ async def get_current_user(
token = credentials.credentials token = credentials.credentials
try: try:
# First, extract issuer and audience from unverified claims
unverified_claims = jwt.get_unverified_claims(token)
token_issuer = unverified_claims.get("iss", "")
token_audience = unverified_claims.get("aud", "")
# Validate issuer is in our allowed list
if not oidc_config.is_valid_issuer(token_issuer):
logger.warning(f"Invalid token issuer: {token_issuer}")
raise HTTPException(status_code=401, detail="Invalid token issuer")
# Validate audience is in our allowed list
if token_audience not in oidc_config.audiences:
logger.warning(f"Invalid token audience: {token_audience}")
raise HTTPException(status_code=401, detail="Invalid token audience")
# Decode token header to get key ID # Decode token header to get key ID
unverified_header = jwt.get_unverified_header(token) unverified_header = jwt.get_unverified_header(token)
kid = unverified_header.get("kid") kid = unverified_header.get("kid")
@@ -116,8 +153,8 @@ async def get_current_user(
if not kid: if not kid:
raise HTTPException(status_code=401, detail="Invalid token format") raise HTTPException(status_code=401, detail="Invalid token format")
# Find matching key in JWKS # Find matching key in JWKS for this specific issuer
jwks = get_jwks() jwks = get_jwks_for_issuer(token_issuer)
rsa_key = None rsa_key = None
for key in jwks.get("keys", []): for key in jwks.get("keys", []):
@@ -129,17 +166,17 @@ async def get_current_user(
logger.warning(f"No matching key found for kid: {kid}") logger.warning(f"No matching key found for kid: {kid}")
raise HTTPException(status_code=401, detail="Invalid token key") raise HTTPException(status_code=401, detail="Invalid token key")
# Verify and decode token # Verify and decode token using the token's actual issuer and audience
payload = jwt.decode( payload = jwt.decode(
token, token,
rsa_key, rsa_key,
algorithms=["RS256"], algorithms=["RS256"],
audience=oidc_config.audience, audience=token_audience, # Use the token's audience (already validated)
issuer=oidc_config.issuer, issuer=token_issuer, # Use the token's issuer (already validated)
) )
user_email = payload.get("email", "unknown") user_email = payload.get("email", "unknown")
logger.info(f"Authenticated user: {user_email}") logger.info(f"Authenticated user: {user_email} (issuer: {token_issuer})")
return payload return payload
+37 -1
View File
@@ -13,7 +13,7 @@ from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload from sqlalchemy.orm import selectinload
from src.config import get_settings from src.shared.config import get_settings
from src.logging_config import get_logger from src.logging_config import get_logger
from src.db.models import User, Role, UserPreferences, Group from src.db.models import User, Role, UserPreferences, Group
from src.auth.schemas import TokenInfoSchema, UserSchema, RoleSchema, UserPreferencesSchema, UserListItemSchema, BulkSyncResultSchema, GroupListItemSchema from src.auth.schemas import TokenInfoSchema, UserSchema, RoleSchema, UserPreferencesSchema, UserListItemSchema, BulkSyncResultSchema, GroupListItemSchema
@@ -86,6 +86,42 @@ class AuthService:
logger.error(f"Authentik userinfo request error: {e}") logger.error(f"Authentik userinfo request error: {e}")
raise ValueError("Authentication service unavailable") raise ValueError("Authentication service unavailable")
async def get_user_by_email(self, email: str) -> Optional[User]:
"""
Get user by email address
Args:
email: User email address
Returns:
User if found, None otherwise
"""
stmt = (
select(User)
.options(selectinload(User.roles), selectinload(User.preferences))
.where(User.email == email)
)
result = await self.session.execute(stmt)
return result.scalar_one_or_none()
async def get_user_by_authentik_id(self, authentik_id: uuid.UUID) -> Optional[User]:
"""
Get user by Authentik UUID
Args:
authentik_id: Authentik user UUID
Returns:
User if found, None otherwise
"""
stmt = (
select(User)
.options(selectinload(User.roles), selectinload(User.preferences))
.where(User.authentik_id == authentik_id)
)
result = await self.session.execute(stmt)
return result.scalar_one_or_none()
async def sync_user(self, token_info: TokenInfoSchema) -> tuple[User, bool]: async def sync_user(self, token_info: TokenInfoSchema) -> tuple[User, bool]:
""" """
Create or update user from OIDC token info Create or update user from OIDC token info
+1 -1
View File
@@ -10,7 +10,7 @@ import json
from typing import Optional, Dict, List, Any from typing import Optional, Dict, List, Any
from datetime import datetime, timedelta, timezone from datetime import datetime, timedelta, timezone
from src.logging_config import get_logger from src.logging_config import get_logger
from src.config import get_settings from src.shared.config import get_settings
logger = get_logger(__name__) logger = get_logger(__name__)
settings = get_settings() settings = get_settings()
+1 -1
View File
@@ -7,7 +7,7 @@ import httpx
from typing import Optional, Dict, List, Any from typing import Optional, Dict, List, Any
from datetime import datetime, timedelta from datetime import datetime, timedelta
from src.logging_config import get_logger from src.logging_config import get_logger
from src.config import get_settings from src.shared.config import get_settings
logger = get_logger(__name__) logger = get_logger(__name__)
settings = get_settings() settings = get_settings()
+1 -1
View File
@@ -8,7 +8,7 @@ import httpx
import json import json
from typing import Optional, Dict, List, Any from typing import Optional, Dict, List, Any
from src.logging_config import get_logger from src.logging_config import get_logger
from src.config import get_settings from src.shared.config import get_settings
logger = get_logger(__name__) logger = get_logger(__name__)
settings = get_settings() settings = get_settings()
-160
View File
@@ -1,160 +0,0 @@
"""
Global configuration for Core Code API
All configuration is loaded from environment variables or .env file.
See .env.example for available settings.
"""
import tomllib
from pathlib import Path
from pydantic_settings import BaseSettings
from functools import lru_cache
def _get_version_from_pyproject() -> str:
"""Load version from pyproject.toml"""
pyproject_path = Path(__file__).parent.parent / "pyproject.toml"
try:
with open(pyproject_path, "rb") as f:
data = tomllib.load(f)
return data.get("project", {}).get("version", "0.0.0")
except FileNotFoundError:
return "0.0.0"
__version__ = _get_version_from_pyproject()
class Settings(BaseSettings):
"""Global application settings"""
# Application
app_name: str = "Core Code API"
app_version: str = __version__
debug: bool = False
# Server
host: str = "0.0.0.0"
port: int = 8083
# CORS
cors_origins: list[str] = ["*"]
cors_credentials: bool = True
cors_methods: list[str] = ["*"]
cors_headers: list[str] = ["*"]
# Logging
log_level: str = "DEBUG"
# Ollama Configuration (for AI orchestration)
ollama_base_url: str # Required - set OLLAMA_BASE_URL in .env
ollama_timeout: int = 300 # 5 minutes
# Model Configuration
default_model: str = "mistral-nemo-large:latest"
agent_model: str = "mistral-nemo-large:latest" # Must support tool calling with ADK (~4GB VRAM)
code_models: str = "mistral-nemo-large:latest"
# Previous config (gemma3:12b used ~10GB VRAM)
# default_model: str = "gemma3:12b"
# agent_model: str = "gemma3:12b"
# System Prompt Variant (for A/B testing)
# Options: v1_verbose, v2_concise, v3_imperative, v4_minimal, v4_gemini_suggestion, v5_adk_optimized, v7_adk_best_practice, v8_holistic
system_prompt_variant: str = "v8_holistic"
# Agent Configuration
agent_fallback_enabled: bool = True
# Model Aliases (OpenAI → Local)
alias_gpt35: str = "gemma:7b"
alias_gpt4: str = "mistral:7b"
alias_gpt4_turbo: str = "mixtral:8x7b"
alias_gpt4_code: str = "codestral:latest"
# Memory Configuration
memory_tier1_max_turns: int = 10
memory_consolidation_threshold: int = 10
# Qdrant Configuration
qdrant_host: str = "qdrant"
qdrant_port: int = 6333
qdrant_collection_conversations: str = "core_api_conversations"
qdrant_collection_documents: str = "core_api_documents"
qdrant_collection_user_facts: str = "core_api_user_facts"
# Embeddings (using Ollama - no local models needed)
embedding_model: str = "nomic-embed-text" # Ollama embedding model
embedding_dimension: int = 768 # nomic-embed-text dimension
embedding_batch_size: int = 32
# Search Configuration
search_provider: str = "searxng"
searxng_url: str # Required - set SEARXNG_URL in .env
# Infrastructure Management (Portainer)
portainer_url: str # Required - set PORTAINER_URL in .env
portainer_api_key: str # Required - set PORTAINER_API_KEY in .env
# Infrastructure Management (Nginx Proxy Manager)
npm_url: str # Required - set NPM_URL in .env
npm_email: str # Required - set NPM_EMAIL in .env
npm_password: str # Required - set NPM_PASSWORD in .env
# Home Assistant Configuration
homeassistant_url: str # Required - set HOMEASSISTANT_URL in .env
homeassistant_token: str # Required - set HOMEASSISTANT_TOKEN in .env
homeassistant_timeout: int = 30
# PostgreSQL Database
postgres_host: str # Required - set POSTGRES_HOST in .env (e.g., localhost:5432)
postgres_user: str = "core_api"
postgres_password: str # Required - set POSTGRES_PASSWORD in .env
postgres_database: str = "core_api"
@property
def database_url(self) -> str:
"""Construct database URL from components"""
return f"postgresql://{self.postgres_user}:{self.postgres_password}@{self.postgres_host}/{self.postgres_database}"
# OIDC Authentication (Authentik)
oidc_enabled: bool = False # Set to True to require authentication
oidc_issuer: str = "https://auth.schweitz.net/application/o/core-api/"
oidc_audience: str = "core-api"
# Authentik API (for token validation and user management)
# Must use domain name (not IP) when AUTHENTIK_COOKIE_DOMAIN is set
authentik_url: str = "https://auth.schweitz.net" # Authentik base URL
authentik_username: str = "" # Admin username for API access (AUTHENTIK_USERNAME env var)
authentik_password: str = "" # Admin password for API access (AUTHENTIK_PASSWORD env var)
@property
def model_aliases(self) -> dict:
"""Computed property for model aliases"""
return {
"gpt-3.5-turbo": self.alias_gpt35,
"gpt-4": self.alias_gpt4,
"gpt-4-turbo": self.alias_gpt4_turbo,
"gpt-4-code": self.alias_gpt4_code,
}
def get_lightweight_models(self) -> list[str]:
"""Parse comma-separated lightweight models"""
return [m.strip().strip('"').strip("'") for m in self.lightweight_models.split(",") if m.strip()]
def get_heavy_models(self) -> list[str]:
"""Parse comma-separated heavy models"""
return [m.strip().strip('"').strip("'") for m in self.heavy_models.split(",") if m.strip()]
def get_code_models(self) -> list[str]:
"""Parse comma-separated code models"""
return [m.strip().strip('"').strip("'") for m in self.code_models.split(",") if m.strip()]
class Config:
env_file = ".env"
case_sensitive = False
extra = "ignore" # Ignore extra env vars not defined in Settings
@lru_cache()
def get_settings() -> Settings:
"""Cached settings instance"""
return Settings()
+13 -96
View File
@@ -7,13 +7,11 @@ from fastapi import APIRouter, Response
from fastapi.responses import JSONResponse from fastapi.responses import JSONResponse
from src.controllers.base import BaseController from src.controllers.base import BaseController
from src.config import get_settings from src.shared.config import get_settings
from src.logging_config import get_logger from src.logging_config import get_logger
from src.models.ollama_client import get_ollama_client
from src.db import get_database from src.db import get_database
logger = get_logger(__name__) logger = get_logger(__name__)
@@ -73,63 +71,18 @@ class HealthController(BaseController):
@router.get( @router.get(
"/health/full", "/health/full",
summary="Fast health check for Docker", summary="Full health check with database",
) )
async def full_health_check(response: Response): async def full_health_check(response: Response):
""" """
Fast health check for container orchestration (Docker/K8s). Health check including database connectivity.
Checks component availability WITHOUT running expensive operations. Returns 200 OK if database is available, otherwise 503.
Returns 200 OK if all components are available, otherwise 503.
For detailed diagnostics, use /health/diagnostics instead.
""" """
import time import time
start_time = time.time() start_time = time.time()
# Check 1: Ollama connection + verify agent model is available # Check database connection
ollama_client = get_ollama_client()
ollama_healthy = False
ollama_error = None
model_available = False
try:
# Ping Ollama
ollama_healthy = await ollama_client.health_check()
# Verify the agent model is pulled and check what's currently loaded
models_info = {}
if ollama_healthy:
try:
models_response = await ollama_client.list_models()
available_models = [m.get('name', '') for m in models_response.get('models', [])]
model_available = settings.agent_model in available_models
# Get info about currently loaded models (those with size in memory)
loaded_models = [
m.get('name', '') for m in models_response.get('models', [])
if m.get('size', 0) > 0
]
models_info = {
"configured": settings.agent_model,
"available": model_available,
"total_in_ollama": len(available_models),
"currently_loaded": loaded_models if loaded_models else ["none"]
}
if not model_available:
ollama_error = f"Model '{settings.agent_model}' not found in Ollama. Available: {', '.join(available_models[:3])}"
ollama_healthy = False
except Exception as e:
ollama_error = f"Could not list Ollama models: {str(e)}"
ollama_healthy = False
except Exception as e:
ollama_error = str(e)
logger.warning(f"Ollama health check failed: {ollama_error}")
# Check 2: Database connection
database = get_database() database = get_database()
db_healthy = False db_healthy = False
db_error = None db_error = None
@@ -140,27 +93,17 @@ class HealthController(BaseController):
db_error = str(e) db_error = str(e)
logger.warning(f"Database health check failed: {db_error}") logger.warning(f"Database health check failed: {db_error}")
is_healthy = ollama_healthy and db_healthy
elapsed_ms = int((time.time() - start_time) * 1000) elapsed_ms = int((time.time() - start_time) * 1000)
status_code = 200 if is_healthy else 503 status_code = 200 if db_healthy else 503
response.status_code = status_code response.status_code = status_code
return { return {
"status": "healthy" if is_healthy else "unhealthy", "status": "healthy" if db_healthy else "unhealthy",
"status_code": status_code, "status_code": status_code,
"response_time_ms": elapsed_ms, "response_time_ms": elapsed_ms,
"components": { "components": {
"ollama": {
"status": "✅ healthy" if ollama_healthy else "❌ unhealthy",
"models": models_info if models_info else {
"configured": settings.agent_model,
"available": False
},
"error": ollama_error
},
"database": { "database": {
"status": "✅ healthy" if db_healthy else "❌ unhealthy", "status": "healthy" if db_healthy else "unhealthy",
"error": db_error "error": db_error
} }
} }
@@ -170,18 +113,13 @@ class HealthController(BaseController):
"/health/diagnostics", "/health/diagnostics",
summary="Detailed system diagnostics", summary="Detailed system diagnostics",
) )
async def diagnostics(deep_test: bool = False): async def diagnostics():
""" """
Comprehensive system diagnostics with detailed component information. System diagnostics with service information.
Query Parameters:
- deep_test: Set to true to actually test agent generation (slow, ~5-10s)
Returns detailed information about all system components.
""" """
import time import time
start_time = time.time() start_time = time.time()
diagnostics = { diagnostics = {
"timestamp": time.time(), "timestamp": time.time(),
"service": { "service": {
@@ -189,31 +127,10 @@ class HealthController(BaseController):
"version": settings.app_version, "version": settings.app_version,
"purpose": "Infrastructure management and tools API" "purpose": "Infrastructure management and tools API"
}, },
"components": {} "configuration": {
}
# 1. Ollama Connection
ollama_client = get_ollama_client()
try:
ollama_healthy = await ollama_client.health_check()
diagnostics["components"]["ollama"] = {
"status": "✅ connected",
"url": settings.ollama_base_url,
"timeout": settings.ollama_timeout,
"default_model": settings.default_model
}
except Exception as e:
diagnostics["components"]["ollama"] = {
"status": "❌ error",
"error": str(e)
}
# 2. Configuration
diagnostics["configuration"] = {
"agent_fallback_enabled": settings.agent_fallback_enabled,
"memory_tier1_max_turns": settings.memory_tier1_max_turns,
"cors_origins": settings.cors_origins[:2] if len(settings.cors_origins) > 2 else settings.cors_origins "cors_origins": settings.cors_origins[:2] if len(settings.cors_origins) > 2 else settings.cors_origins
} }
}
elapsed_ms = int((time.time() - start_time) * 1000) elapsed_ms = int((time.time() - start_time) * 1000)
diagnostics["response_time_ms"] = elapsed_ms diagnostics["response_time_ms"] = elapsed_ms
+62 -1
View File
@@ -3,14 +3,20 @@ Tools Controller
Provides utility tool endpoints including: Provides utility tool endpoints including:
- DNS lookups - DNS lookups
- Environment data (weather, forecast, sun times, air quality)
""" """
from fastapi import APIRouter, HTTPException, status from typing import Dict, Optional
from fastapi import APIRouter, Depends, HTTPException, status
from src.controllers.base import BaseController from src.controllers.base import BaseController
from src.logging_config import get_logger from src.logging_config import get_logger
from src.dns.schemas import DNSLookupRequest, DNSLookupResponse from src.dns.schemas import DNSLookupRequest, DNSLookupResponse
from src.dns.service import DNSService from src.dns.service import DNSService
from src.dns.exceptions import DNSQueryError from src.dns.exceptions import DNSQueryError
from src.domains.tools.environment.schemas import EnvironmentResponse
from src.domains.tools.environment.service import get_environment_service
from src.auth.oidc import get_optional_user
logger = get_logger(__name__) logger = get_logger(__name__)
@@ -26,6 +32,7 @@ class ToolsController(BaseController):
def __init__(self): def __init__(self):
super().__init__(prefix="/tools", tags=["Tools"]) super().__init__(prefix="/tools", tags=["Tools"])
self.dns_service = DNSService() self.dns_service = DNSService()
self.environment_service = get_environment_service()
def create_router(self) -> APIRouter: def create_router(self) -> APIRouter:
"""Create and configure the router""" """Create and configure the router"""
@@ -95,6 +102,60 @@ class ToolsController(BaseController):
detail="An unexpected error occurred during DNS lookup" detail="An unexpected error occurred during DNS lookup"
) )
@router.get(
"/environment",
response_model=EnvironmentResponse,
status_code=status.HTTP_200_OK,
summary="Get environment data",
description="""
Fetch current environment data including weather, forecast, sun times,
and optionally air quality.
Data is retrieved from the user's volatile Qdrant collection which is
populated by background data collectors.
**Data Sources:**
- Weather: Current temperature, conditions, humidity, wind
- Forecast: Multi-day weather outlook
- Sun Times: Sunrise, sunset, daylight duration
- Air Quality: AQI and pollutant levels (when available)
**Authentication:**
- Uses authenticated user's `preferred_username` if available
- Falls back to 'default' for unauthenticated requests
"""
)
async def get_environment(
user: Optional[Dict] = Depends(get_optional_user),
) -> EnvironmentResponse:
"""
Get current environment data.
Args:
user: Optional authenticated user info
Returns:
Environment data with weather, forecast, sun times, and air quality
"""
try:
# Determine user identifier
user_id = "default"
if user:
logger.debug(f"User claims: {user}")
user_id = user.get("preferred_username") or user.get("sub", "default")
logger.info(f"Fetching environment data for user: {user_id} (preferred_username={user.get('preferred_username')}, sub={user.get('sub')})")
else:
logger.info(f"Fetching environment data for user: {user_id} (no auth)")
result = await self.environment_service.get_current(user_id)
return result
except Exception as e:
logger.error(f"Error fetching environment data: {str(e)}", exc_info=True)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Failed to fetch environment data"
)
return router return router
+1 -1
View File
@@ -15,7 +15,7 @@ from sqlalchemy.ext.asyncio import (
from sqlalchemy.orm import DeclarativeBase from sqlalchemy.orm import DeclarativeBase
from sqlalchemy.pool import NullPool from sqlalchemy.pool import NullPool
from src.config import get_settings from src.shared.config import get_settings
from src.logging_config import get_logger from src.logging_config import get_logger
logger = get_logger(__name__) logger = get_logger(__name__)
+14
View File
@@ -11,6 +11,12 @@ from src.domains.auth.oidc import (
get_forward_auth_user, get_forward_auth_user,
get_forward_auth_admin, get_forward_auth_admin,
oidc_config, oidc_config,
# Permission system
require_permission,
require_any_permission,
ACTION_HIERARCHY,
VALID_DOMAINS,
DEFAULT_CATEGORY,
) )
from src.domains.auth.service import AuthService, get_auth_service from src.domains.auth.service import AuthService, get_auth_service
from src.domains.auth.controller import auth_controller from src.domains.auth.controller import auth_controller
@@ -22,6 +28,7 @@ from src.domains.auth.models import (
UserPreferences, UserPreferences,
ApiKey, ApiKey,
user_groups, user_groups,
group_roles,
) )
__all__ = [ __all__ = [
@@ -32,6 +39,12 @@ __all__ = [
"get_forward_auth_user", "get_forward_auth_user",
"get_forward_auth_admin", "get_forward_auth_admin",
"oidc_config", "oidc_config",
# Permission system
"require_permission",
"require_any_permission",
"ACTION_HIERARCHY",
"VALID_DOMAINS",
"DEFAULT_CATEGORY",
# Service # Service
"AuthService", "AuthService",
"get_auth_service", "get_auth_service",
@@ -45,4 +58,5 @@ __all__ = [
"UserPreferences", "UserPreferences",
"ApiKey", "ApiKey",
"user_groups", "user_groups",
"group_roles",
] ]
+377 -14
View File
@@ -3,8 +3,9 @@ Authentication Controller
Provides authentication endpoints for OIDC token sync and user management. Provides authentication endpoints for OIDC token sync and user management.
""" """
import uuid
from typing import Optional from typing import Optional
from fastapi import APIRouter, Depends, HTTPException, Query from fastapi import APIRouter, Depends, HTTPException, Path, Query
from fastapi.responses import JSONResponse from fastapi.responses import JSONResponse
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
@@ -13,9 +14,13 @@ from src.shared.logging import get_logger
from src.shared.database import get_async_session from src.shared.database import get_async_session
from src.domains.auth.schemas import ( from src.domains.auth.schemas import (
AuthSyncRequest, AuthSyncResponse, UsersListResponse, AuthSyncRequest, AuthSyncResponse, UsersListResponse,
BulkSyncResultSchema, GroupsListResponse BulkSyncResultSchema, GroupsListResponse, RolesListResponse,
GroupRoleAssignmentResponse, UserProfileResponse, PreferencesUpdateRequest,
UserPreferencesSchema, ApiKeyCreateRequest, ApiKeyCreateResponse,
ApiKeysListResponse,
) )
from src.domains.auth.service import AuthService from src.domains.auth.service import AuthService
from src.domains.auth.oidc import get_current_user, get_current_user_or_forward_auth
logger = get_logger(__name__) logger = get_logger(__name__)
@@ -219,28 +224,386 @@ class AuthController(BaseController):
raise HTTPException(status_code=401, detail=str(e)) raise HTTPException(status_code=401, detail=str(e))
@router.get( @router.get(
"/me", "/roles",
summary="Get current user profile", summary="List all roles",
response_model=AuthSyncResponse, response_model=RolesListResponse,
responses={ responses={
200: {"description": "User profile"}, 200: {"description": "List of all available roles"},
},
)
async def list_roles(
session: AsyncSession = Depends(get_async_session),
) -> RolesListResponse:
"""
List all available roles in the system
Returns all domain.category:action role combinations.
Use these when assigning roles to groups.
"""
service = AuthService(session)
roles = await service.list_roles()
return RolesListResponse(
items=service.roles_to_schema(roles),
total=len(roles),
)
@router.post(
"/groups/{group_id}/roles/{role_id}",
summary="Assign role to group",
response_model=GroupRoleAssignmentResponse,
responses={
200: {"description": "Role assigned successfully"},
404: {"description": "Group or role not found"},
},
)
async def assign_role_to_group(
group_id: uuid.UUID = Path(..., description="Group ID"),
role_id: uuid.UUID = Path(..., description="Role ID to assign"),
session: AsyncSession = Depends(get_async_session),
) -> GroupRoleAssignmentResponse:
"""
Assign a role to a group
All users in this group will inherit this role's permissions.
"""
service = AuthService(session)
try:
group = await service.assign_role_to_group(group_id, role_id)
await session.commit()
return GroupRoleAssignmentResponse(
group_id=group.id,
group_name=group.name,
roles=[role.name for role in group.roles],
)
except ValueError as e:
raise HTTPException(status_code=404, detail=str(e))
@router.delete(
"/groups/{group_id}/roles/{role_id}",
summary="Remove role from group",
response_model=GroupRoleAssignmentResponse,
responses={
200: {"description": "Role removed successfully"},
404: {"description": "Group or role not found"},
},
)
async def remove_role_from_group(
group_id: uuid.UUID = Path(..., description="Group ID"),
role_id: uuid.UUID = Path(..., description="Role ID to remove"),
session: AsyncSession = Depends(get_async_session),
) -> GroupRoleAssignmentResponse:
"""
Remove a role from a group
Users in this group will no longer inherit this role's permissions.
"""
service = AuthService(session)
try:
group = await service.remove_role_from_group(group_id, role_id)
await session.commit()
return GroupRoleAssignmentResponse(
group_id=group.id,
group_name=group.name,
roles=[role.name for role in group.roles],
)
except ValueError as e:
raise HTTPException(status_code=404, detail=str(e))
# =====================================================================
# Phase 4: User Profile & Settings
# =====================================================================
@router.get(
"/users/me",
summary="Get current user profile",
response_model=UserProfileResponse,
responses={
200: {"description": "User profile with roles and preferences"},
401: {"description": "Not authenticated"}, 401: {"description": "Not authenticated"},
}, },
) )
async def get_me( async def get_current_user_profile(
user_claims: dict = Depends(get_current_user_or_forward_auth),
session: AsyncSession = Depends(get_async_session), session: AsyncSession = Depends(get_async_session),
) -> JSONResponse: ) -> UserProfileResponse:
""" """
Get the current authenticated user's profile Get the current authenticated user's profile
Note: This endpoint requires a valid session or API key. Returns the user's profile, roles, and preferences.
For now, returns 501 Not Implemented until session management is added. Supports both:
- Bearer token (mobile/native clients)
- NPM forward auth headers (web clients via proxy)
For forward auth users, auto-creates the user in the database
if they don't exist yet (first login via web).
""" """
# TODO: Implement with get_current_user dependency service = AuthService(session)
raise HTTPException(
status_code=501, # Get authentik_id from claims (JWT 'sub' field or forward auth 'uid')
detail="Not implemented - use /auth/sync with access token", authentik_id_str = user_claims.get("sub")
if not authentik_id_str or authentik_id_str == "local-user":
raise HTTPException(status_code=401, detail="Authentication required")
try:
authentik_id = uuid.UUID(authentik_id_str)
except ValueError:
raise HTTPException(status_code=401, detail="Invalid user identifier")
user = await service.get_user_by_authentik_id(authentik_id)
# If user not found and using forward auth, auto-create them
if user is None:
auth_method = user_claims.get("auth_method")
if auth_method == "forward_auth":
# Auto-sync user from forward auth headers
logger.info(f"Auto-creating user from forward auth: {user_claims.get('email')}")
user, is_new = await service.sync_user_from_claims(
authentik_id=authentik_id,
email=user_claims.get("email", ""),
name=user_claims.get("name", user_claims.get("preferred_username", "")),
groups=user_claims.get("groups", []),
) )
await session.commit()
await session.refresh(user, ["preferences", "roles"])
else:
# JWT auth but user not in DB - they need to sync first
raise HTTPException(
status_code=404,
detail="User not found - please sync via /auth/sync first",
)
return UserProfileResponse(
user=service.user_to_schema(user),
roles=service.roles_to_schema(user.roles),
preferences=service.preferences_to_schema(user.preferences),
)
@router.get(
"/users/me/preferences",
summary="Get user preferences",
response_model=UserPreferencesSchema,
responses={
200: {"description": "User preferences"},
401: {"description": "Not authenticated"},
},
)
async def get_preferences(
user_claims: dict = Depends(get_current_user),
session: AsyncSession = Depends(get_async_session),
) -> UserPreferencesSchema:
"""
Get the current user's preferences
"""
service = AuthService(session)
authentik_id_str = user_claims.get("sub")
if not authentik_id_str or authentik_id_str == "local-user":
raise HTTPException(status_code=401, detail="Authentication required")
try:
authentik_id = uuid.UUID(authentik_id_str)
except ValueError:
raise HTTPException(status_code=401, detail="Invalid user identifier")
user = await service.get_user_by_authentik_id(authentik_id)
if user is None:
raise HTTPException(status_code=404, detail="User not found")
return service.preferences_to_schema(user.preferences)
@router.patch(
"/users/me/preferences",
summary="Update user preferences",
response_model=UserPreferencesSchema,
responses={
200: {"description": "Updated preferences"},
401: {"description": "Not authenticated"},
422: {"description": "Invalid preference value"},
},
)
async def update_preferences(
request: PreferencesUpdateRequest,
user_claims: dict = Depends(get_current_user),
session: AsyncSession = Depends(get_async_session),
) -> UserPreferencesSchema:
"""
Update the current user's preferences
Only provided fields are updated. preferences_json is merged
with existing values (not replaced).
"""
service = AuthService(session)
authentik_id_str = user_claims.get("sub")
if not authentik_id_str or authentik_id_str == "local-user":
raise HTTPException(status_code=401, detail="Authentication required")
try:
authentik_id = uuid.UUID(authentik_id_str)
except ValueError:
raise HTTPException(status_code=401, detail="Invalid user identifier")
user = await service.get_user_by_authentik_id(authentik_id)
if user is None:
raise HTTPException(status_code=404, detail="User not found")
try:
prefs = await service.update_preferences(
user_id=user.id,
theme=request.theme,
default_room=request.default_room,
preferences_json=request.preferences_json,
)
await session.commit()
return service.preferences_to_schema(prefs)
except ValueError as e:
raise HTTPException(status_code=422, detail=str(e))
# =====================================================================
# Phase 4: API Keys
# =====================================================================
@router.get(
"/users/me/api-keys",
summary="List user's API keys",
response_model=ApiKeysListResponse,
responses={
200: {"description": "List of API keys"},
401: {"description": "Not authenticated"},
},
)
async def list_api_keys(
user_claims: dict = Depends(get_current_user),
session: AsyncSession = Depends(get_async_session),
) -> ApiKeysListResponse:
"""
List all API keys for the current user
Returns key metadata only - the actual key values are never
retrievable after creation.
"""
service = AuthService(session)
authentik_id_str = user_claims.get("sub")
if not authentik_id_str or authentik_id_str == "local-user":
raise HTTPException(status_code=401, detail="Authentication required")
try:
authentik_id = uuid.UUID(authentik_id_str)
except ValueError:
raise HTTPException(status_code=401, detail="Invalid user identifier")
user = await service.get_user_by_authentik_id(authentik_id)
if user is None:
raise HTTPException(status_code=404, detail="User not found")
keys = await service.list_user_api_keys(user.id)
return ApiKeysListResponse(
items=[service.api_key_to_schema(k) for k in keys],
total=len(keys),
)
@router.post(
"/users/me/api-keys",
summary="Create a new API key",
response_model=ApiKeyCreateResponse,
responses={
201: {"description": "API key created"},
401: {"description": "Not authenticated"},
403: {"description": "API keys disabled for user"},
},
)
async def create_api_key(
request: ApiKeyCreateRequest,
user_claims: dict = Depends(get_current_user),
session: AsyncSession = Depends(get_async_session),
) -> ApiKeyCreateResponse:
"""
Create a new API key for the current user
**IMPORTANT**: The full API key is only returned once in this response!
Store it securely - it cannot be retrieved again.
"""
service = AuthService(session)
authentik_id_str = user_claims.get("sub")
if not authentik_id_str or authentik_id_str == "local-user":
raise HTTPException(status_code=401, detail="Authentication required")
try:
authentik_id = uuid.UUID(authentik_id_str)
except ValueError:
raise HTTPException(status_code=401, detail="Invalid user identifier")
user = await service.get_user_by_authentik_id(authentik_id)
if user is None:
raise HTTPException(status_code=404, detail="User not found")
try:
api_key, full_key = await service.create_api_key(
user_id=user.id,
name=request.name,
scopes=request.scopes,
expires_in_days=request.expires_in_days,
)
await session.commit()
return ApiKeyCreateResponse(
id=api_key.id,
name=api_key.name,
key=full_key, # Only time this is returned!
key_prefix=api_key.key_prefix,
scopes=api_key.scopes,
expires_at=api_key.expires_at,
created_at=api_key.created_at,
)
except ValueError as e:
raise HTTPException(status_code=403, detail=str(e))
@router.delete(
"/users/me/api-keys/{key_id}",
summary="Delete an API key",
responses={
204: {"description": "API key deleted"},
401: {"description": "Not authenticated"},
404: {"description": "API key not found"},
},
)
async def delete_api_key(
key_id: uuid.UUID = Path(..., description="API key ID to delete"),
user_claims: dict = Depends(get_current_user),
session: AsyncSession = Depends(get_async_session),
) -> JSONResponse:
"""
Delete an API key
The key will be immediately invalidated.
"""
service = AuthService(session)
authentik_id_str = user_claims.get("sub")
if not authentik_id_str or authentik_id_str == "local-user":
raise HTTPException(status_code=401, detail="Authentication required")
try:
authentik_id = uuid.UUID(authentik_id_str)
except ValueError:
raise HTTPException(status_code=401, detail="Invalid user identifier")
user = await service.get_user_by_authentik_id(authentik_id)
if user is None:
raise HTTPException(status_code=404, detail="User not found")
try:
deleted = await service.delete_api_key(user.id, key_id)
if not deleted:
raise HTTPException(status_code=404, detail="API key not found")
await session.commit()
return JSONResponse(status_code=204, content=None)
except ValueError as e:
raise HTTPException(status_code=403, detail=str(e))
return router return router
+34 -8
View File
@@ -26,6 +26,13 @@ user_groups = Table(
Column("group_id", UUID(as_uuid=True), ForeignKey("groups.id", ondelete="CASCADE"), primary_key=True), Column("group_id", UUID(as_uuid=True), ForeignKey("groups.id", ondelete="CASCADE"), primary_key=True),
) )
group_roles = Table(
"group_roles",
Base.metadata,
Column("group_id", UUID(as_uuid=True), ForeignKey("groups.id", ondelete="CASCADE"), primary_key=True),
Column("role_id", UUID(as_uuid=True), ForeignKey("roles.id", ondelete="CASCADE"), primary_key=True),
)
# ============================================================================= # =============================================================================
# User Model # User Model
@@ -114,8 +121,13 @@ class Role(Base):
""" """
Role model for domain-scoped permissions Role model for domain-scoped permissions
Permission format: domain.category:action
- domain: Main area (control-room, library, media, ai, etc.)
- category: Sub-area within domain (general for full access, or specific tools)
- action: Permission level (viewer, user, editor, admin)
Roles are seeded from configuration, not user-editable. Roles are seeded from configuration, not user-editable.
Each role maps to an Authentik group (e.g., tatlock-control-room-admin). Groups are assigned roles via the group_roles mapping table.
Domains: control-room, library, media, ai, housekeeper, developer, documents, gaming, admin Domains: control-room, library, media, ai, housekeeper, developer, documents, gaming, admin
Actions: viewer, user, editor, admin (hierarchical) Actions: viewer, user, editor, admin (hierarchical)
@@ -133,7 +145,7 @@ class Role(Base):
unique=True, unique=True,
nullable=False, nullable=False,
index=True, index=True,
comment="Role name in format domain:action (e.g., control-room:admin)", comment="Role name in format domain.category:action (e.g., control-room.general:admin)",
) )
domain: Mapped[str] = mapped_column( domain: Mapped[str] = mapped_column(
String(50), String(50),
@@ -141,17 +153,17 @@ class Role(Base):
index=True, index=True,
comment="Permission domain (e.g., control-room, media, ai)", comment="Permission domain (e.g., control-room, media, ai)",
) )
category: Mapped[str] = mapped_column(
String(50),
nullable=False,
default="general",
comment="Permission category within domain (general for full access, or specific tool)",
)
action: Mapped[str] = mapped_column( action: Mapped[str] = mapped_column(
String(20), String(20),
nullable=False, nullable=False,
comment="Permission action (viewer, user, editor, admin)", comment="Permission action (viewer, user, editor, admin)",
) )
authentik_group: Mapped[str | None] = mapped_column(
String(255),
nullable=True,
unique=True,
comment="Corresponding Authentik group name (e.g., tatlock-control-room-admin)",
)
# Relationships # Relationships
users: Mapped[List["User"]] = relationship( users: Mapped[List["User"]] = relationship(
@@ -160,6 +172,12 @@ class Role(Base):
back_populates="roles", back_populates="roles",
lazy="selectin", lazy="selectin",
) )
groups: Mapped[List["Group"]] = relationship(
"Group",
secondary="group_roles",
back_populates="roles",
lazy="selectin",
)
def __repr__(self) -> str: def __repr__(self) -> str:
return f"<Role {self.name}>" return f"<Role {self.name}>"
@@ -247,6 +265,14 @@ class Group(Base):
comment="Last sync from Authentik", comment="Last sync from Authentik",
) )
# Relationships
roles: Mapped[List["Role"]] = relationship(
"Role",
secondary="group_roles",
back_populates="groups",
lazy="selectin",
)
def __repr__(self) -> str: def __repr__(self) -> str:
return f"<Group {self.name}>" return f"<Group {self.name}>"
+449 -27
View File
@@ -3,15 +3,56 @@ OIDC Authentication Module
Provides OAuth2/OIDC token validation for FastAPI using Authentik as IdP. Provides OAuth2/OIDC token validation for FastAPI using Authentik as IdP.
Implements bearer token authentication with JWT verification. Implements bearer token authentication with JWT verification.
Permission Format: domain.category:action
- domain: Main area (control-room, library, media, ai, etc.)
- category: Sub-area within domain (general for full domain, or specific tools)
- action: Permission level (viewer, user, editor, admin)
Examples:
- control-room.general:admin - Full access to Control Room
- media.general:viewer - View-only access to Media area
Action Hierarchy (higher implies lower):
- admin > editor > user > viewer
""" """
from fastapi import Depends, HTTPException, Security, Request from fastapi import Depends, HTTPException, Security, Request
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
from jose import jwt, JWTError from jose import jwt, JWTError
import httpx import httpx
from functools import lru_cache from functools import lru_cache
from typing import Dict, Optional from typing import Callable, Dict, List, Optional
from src.shared.logging import get_logger from src.shared.logging import get_logger
# =============================================================================
# Permission System
# =============================================================================
# Action hierarchy: higher actions imply lower ones
ACTION_HIERARCHY: Dict[str, int] = {
"viewer": 1,
"user": 2,
"editor": 3,
"admin": 4,
}
# Valid domains (main areas)
VALID_DOMAINS = {
"control-room",
"library",
"media",
"ai",
"housekeeper",
"developer",
"documents",
"gaming",
"admin", # Global admin domain
}
# Default category for general domain access
DEFAULT_CATEGORY = "general"
logger = get_logger(__name__) logger = get_logger(__name__)
security = HTTPBearer(auto_error=False) security = HTTPBearer(auto_error=False)
@@ -22,29 +63,42 @@ class OIDCConfig:
def __init__(self): def __init__(self):
# These will be set from environment variables in config.py # These will be set from environment variables in config.py
self.enabled = False self.enabled = False
self.issuer = "" self.issuers: list[str] = []
self.audience = "" self.audiences: list[str] = []
self.jwks_uri = ""
def configure(self, enabled: bool, issuer: str, audience: str): def configure(self, enabled: bool, issuers: list[str], audiences: list[str]):
"""Configure OIDC settings""" """Configure OIDC settings"""
self.enabled = enabled self.enabled = enabled
self.issuer = issuer self.issuers = [iss.rstrip('/') for iss in issuers] # Normalize without trailing slash
self.audience = audience self.audiences = audiences
self.jwks_uri = f"{issuer.rstrip('/')}/jwks/" logger.info(f"OIDC configured: enabled={enabled}, issuers={self.issuers}, audiences={audiences}")
logger.info(f"OIDC configured: enabled={enabled}, issuer={issuer}")
def get_jwks_uri(self, issuer: str) -> str:
"""Get JWKS URI for a specific issuer"""
return f"{issuer.rstrip('/')}/jwks/"
def is_valid_issuer(self, issuer: str) -> bool:
"""Check if issuer is in the allowed list"""
normalized = issuer.rstrip('/')
return normalized in self.issuers
# Global OIDC config instance # Global OIDC config instance
oidc_config = OIDCConfig() oidc_config = OIDCConfig()
@lru_cache(maxsize=1) # Per-issuer JWKS cache
def get_jwks() -> Dict: _jwks_cache: Dict[str, Dict] = {}
"""
Fetch JSON Web Key Set (JWKS) from Authentik
Cached to avoid repeated requests. Cache is cleared on server restart.
def get_jwks_for_issuer(issuer: str) -> Dict:
"""
Fetch JSON Web Key Set (JWKS) for a specific issuer.
Cached per-issuer to avoid repeated requests. Cache is cleared on server restart.
Args:
issuer: The token issuer URL
Returns: Returns:
JWKS dictionary containing public keys for token verification JWKS dictionary containing public keys for token verification
@@ -55,15 +109,24 @@ def get_jwks() -> Dict:
if not oidc_config.enabled: if not oidc_config.enabled:
return {} return {}
normalized_issuer = issuer.rstrip('/')
# Return cached JWKS if available
if normalized_issuer in _jwks_cache:
return _jwks_cache[normalized_issuer]
jwks_uri = oidc_config.get_jwks_uri(normalized_issuer)
try: try:
logger.debug(f"Fetching JWKS from {oidc_config.jwks_uri}") logger.debug(f"Fetching JWKS from {jwks_uri}")
response = httpx.get(oidc_config.jwks_uri, timeout=10.0) response = httpx.get(jwks_uri, timeout=10.0)
response.raise_for_status() response.raise_for_status()
jwks = response.json() jwks = response.json()
logger.info(f"JWKS fetched successfully ({len(jwks.get('keys', []))} keys)") logger.info(f"JWKS fetched successfully for {normalized_issuer} ({len(jwks.get('keys', []))} keys)")
_jwks_cache[normalized_issuer] = jwks
return jwks return jwks
except Exception as e: except Exception as e:
logger.error(f"Failed to fetch JWKS: {e}") logger.error(f"Failed to fetch JWKS from {jwks_uri}: {e}")
raise HTTPException( raise HTTPException(
status_code=503, status_code=503,
detail="Authentication service unavailable" detail="Authentication service unavailable"
@@ -116,15 +179,30 @@ async def get_current_user(
token = credentials.credentials token = credentials.credentials
try: try:
# Decode token header to get key ID # First, decode token without verification to get issuer and key ID
unverified_header = jwt.get_unverified_header(token) unverified_header = jwt.get_unverified_header(token)
unverified_claims = jwt.get_unverified_claims(token)
kid = unverified_header.get("kid") kid = unverified_header.get("kid")
token_issuer = unverified_claims.get("iss", "")
token_audience = unverified_claims.get("aud", "")
if not kid: if not kid:
raise HTTPException(status_code=401, detail="Invalid token format") raise HTTPException(status_code=401, detail="Invalid token format")
# Find matching key in JWKS # Validate issuer is in allowed list
jwks = get_jwks() logger.debug(f"Token issuer: {token_issuer}, allowed issuers: {oidc_config.issuers}")
if not oidc_config.is_valid_issuer(token_issuer):
logger.warning(f"Invalid token issuer: {token_issuer} (allowed: {oidc_config.issuers})")
raise HTTPException(status_code=401, detail="Invalid token issuer")
# Validate audience is in allowed list
if token_audience not in oidc_config.audiences:
logger.warning(f"Invalid token audience: {token_audience} (allowed: {oidc_config.audiences})")
raise HTTPException(status_code=401, detail="Invalid token audience")
# Get JWKS for this specific issuer
jwks = get_jwks_for_issuer(token_issuer)
rsa_key = None rsa_key = None
for key in jwks.get("keys", []): for key in jwks.get("keys", []):
@@ -136,17 +214,17 @@ async def get_current_user(
logger.warning(f"No matching key found for kid: {kid}") logger.warning(f"No matching key found for kid: {kid}")
raise HTTPException(status_code=401, detail="Invalid token key") raise HTTPException(status_code=401, detail="Invalid token key")
# Verify and decode token # Verify and decode token using the token's actual issuer and audience
payload = jwt.decode( payload = jwt.decode(
token, token,
rsa_key, rsa_key,
algorithms=["RS256"], algorithms=["RS256"],
audience=oidc_config.audience, audience=token_audience, # Use the token's audience (already validated)
issuer=oidc_config.issuer, issuer=token_issuer, # Use the token's issuer (already validated)
) )
user_email = payload.get("email", "unknown") user_email = payload.get("email", "unknown")
logger.info(f"Authenticated user: {user_email}") logger.info(f"Authenticated user: {user_email} (issuer: {token_issuer})")
return payload return payload
@@ -244,12 +322,14 @@ async def get_optional_user(
} }
if not credentials: if not credentials:
logger.debug("No credentials provided for optional auth")
return None return None
try: try:
return await get_current_user(credentials) return await get_current_user(credentials)
except HTTPException: except HTTPException as e:
# Invalid token - return None instead of raising # Invalid token - log and return None instead of raising
logger.warning(f"Optional auth failed: {e.detail}")
return None return None
@@ -352,3 +432,345 @@ async def get_forward_auth_admin(
) )
return user return user
async def get_current_user_or_forward_auth(
request: Request,
credentials: Optional[HTTPAuthorizationCredentials] = Security(security)
) -> Dict:
"""
Combined auth: Try forward auth headers first, then JWT Bearer token.
Supports both:
- Web clients via NPM forward auth (X-authentik-* headers from proxy)
- Mobile/native clients via OIDC JWT Bearer tokens
This is the preferred dependency for /auth/users/me and similar endpoints
that need to work with both web (cookie-based via NPM) and mobile (token-based).
Args:
request: FastAPI request object containing headers
credentials: HTTP Bearer token from Authorization header
Returns:
User claims dictionary with at minimum: sub, email, name, groups, auth_method
Raises:
HTTPException 401: If neither forward auth headers nor valid JWT provided
"""
# 1. Try forward auth headers first (web via NPM)
username = request.headers.get("x-authentik-username")
email = request.headers.get("x-authentik-email")
if username and email:
# Forward auth headers present - use them
groups = request.headers.get("x-authentik-groups", "")
name = request.headers.get("x-authentik-name", username)
uid = request.headers.get("x-authentik-uid")
user_info = {
"sub": uid, # Use authentik UID as subject (for user lookup)
"email": email,
"preferred_username": username,
"name": name,
"groups": [g.strip() for g in groups.split(",")] if groups else [],
"auth_method": "forward_auth"
}
logger.info(f"Authenticated via forward auth: {email}")
return user_info
# 2. Fall back to JWT Bearer token (mobile/native)
return await get_current_user(credentials)
# =============================================================================
# Permission-Based Access Control
# =============================================================================
def _parse_permission(permission: str) -> tuple[str, str, str]:
"""
Parse a permission string into (domain, category, action)
Supports formats:
- domain.category:action (full): "control-room.general:admin"
- domain:action (shorthand): "control-room:admin" -> ("control-room", "general", "admin")
Returns:
Tuple of (domain, category, action)
Raises:
ValueError: If permission format is invalid
"""
# Split on colon first to get action
if ":" not in permission:
raise ValueError(f"Invalid permission format (missing ':'): {permission}")
location, action = permission.rsplit(":", 1)
# Split location on dot to get domain and category
if "." in location:
domain, category = location.split(".", 1)
else:
# Shorthand: domain:action -> domain.general:action
domain = location
category = DEFAULT_CATEGORY
return domain, category, action
def _action_satisfies(user_action: str, required_action: str) -> bool:
"""
Check if user's action level satisfies the required action
Due to hierarchy, admin satisfies editor, editor satisfies user, etc.
Args:
user_action: The action the user has
required_action: The action required for access
Returns:
True if user's action is >= required action
"""
user_level = ACTION_HIERARCHY.get(user_action, 0)
required_level = ACTION_HIERARCHY.get(required_action, 0)
return user_level >= required_level
def _user_has_permission(
user_permissions: List[str],
required_domain: str,
required_category: str,
required_action: str,
) -> bool:
"""
Check if user has a permission that satisfies the requirement
Checks:
1. Exact match: domain.category:action
2. Domain-wide: domain.general:action (if category != general)
3. Global admin: admin.general:admin (superuser)
Args:
user_permissions: List of user's permission strings
required_domain: Required domain
required_category: Required category
required_action: Required action
Returns:
True if user has sufficient permission
"""
for perm in user_permissions:
try:
dom, cat, act = _parse_permission(perm)
except ValueError:
continue
# Global admin (admin.general:admin) grants all permissions
if dom == "admin" and cat == "general" and act == "admin":
return True
# Check if this permission covers the requirement
if dom == required_domain:
# Exact category match
if cat == required_category and _action_satisfies(act, required_action):
return True
# Domain-wide permission (general category) covers all categories in domain
if cat == "general" and _action_satisfies(act, required_action):
return True
return False
def _extract_permissions_from_groups(groups: List[str]) -> List[str]:
"""
Extract permission strings from Authentik group names
Authentik groups follow naming: tatlock-{domain}-{category}-{action}
or shorthand: tatlock-{domain}-{action} (implies category=general)
Examples:
- tatlock-control-room-general-admin -> control-room.general:admin
- tatlock-media-viewer -> media.general:viewer (shorthand)
- tatlock-tools-dns-user -> tools.dns:user
Args:
groups: List of Authentik group names
Returns:
List of permission strings
"""
permissions = []
for group in groups:
if not group.startswith("tatlock-"):
continue
# Remove prefix
parts = group[8:].split("-") # Remove "tatlock-"
if len(parts) >= 3:
# Could be domain-category-action or domain-with-hyphen-action
# Try to find a valid action at the end
action = parts[-1]
if action in ACTION_HIERARCHY:
# Check if domain-category or single domain with hyphen
remaining = parts[:-1]
# Try to find known domain (greedy match from start)
for i in range(len(remaining), 0, -1):
potential_domain = "-".join(remaining[:i])
if potential_domain in VALID_DOMAINS:
category_parts = remaining[i:]
category = "-".join(category_parts) if category_parts else DEFAULT_CATEGORY
permissions.append(f"{potential_domain}.{category}:{action}")
break
elif len(parts) == 2:
# Shorthand: domain-action (domain might have hyphen)
action = parts[-1]
if action in ACTION_HIERARCHY:
domain = parts[0]
if domain in VALID_DOMAINS:
permissions.append(f"{domain}.{DEFAULT_CATEGORY}:{action}")
return permissions
def require_permission(
domain: str,
action: str,
category: str = DEFAULT_CATEGORY,
) -> Callable:
"""
Dependency factory for permission-based access control
Creates a FastAPI dependency that checks if the current user has
the required permission. Considers action hierarchy and global admin.
Usage:
@router.get("/containers")
async def list_containers(
user: Dict = Depends(require_permission("control-room", "viewer"))
):
...
@router.delete("/container/{id}")
async def delete_container(
user: Dict = Depends(require_permission("control-room", "admin"))
):
...
Args:
domain: Permission domain (e.g., "control-room", "media")
action: Required action level (viewer, user, editor, admin)
category: Permission category within domain, defaults to "general"
Returns:
FastAPI dependency function
"""
perm_str = f"{domain}.{category}:{action}"
async def permission_checker(
user: Optional[Dict] = Depends(get_current_user)
) -> Dict:
"""Check if user has required permission"""
# If OIDC disabled, allow all (local dev mode)
if not oidc_config.enabled:
logger.debug(f"OIDC disabled - allowing {perm_str}")
return user or {"email": "local", "groups": ["admin"]}
if user is None:
raise HTTPException(
status_code=401,
detail="Authentication required",
headers={"WWW-Authenticate": "Bearer"},
)
# Extract permissions from user's groups
groups = user.get("groups", [])
permissions = _extract_permissions_from_groups(groups)
# Check if user has required permission
if _user_has_permission(permissions, domain, category, action):
logger.debug(f"User {user.get('email')} granted {perm_str}")
return user
# Permission denied
user_email = user.get("email", "unknown")
logger.warning(
f"User {user_email} denied {perm_str} "
f"(groups: {groups}, permissions: {permissions})"
)
raise HTTPException(
status_code=403,
detail=f"Permission required: {perm_str}",
)
return permission_checker
def require_any_permission(*required_permissions: str) -> Callable:
"""
Dependency factory requiring any one of multiple permissions
Useful for endpoints accessible to multiple roles.
Usage:
@router.get("/shared-resource")
async def get_shared(
user: Dict = Depends(require_any_permission(
"control-room:viewer",
"media:viewer",
))
):
...
Args:
*required_permissions: Permission strings (domain.category:action or domain:action)
Returns:
FastAPI dependency function
"""
async def permission_checker(
user: Optional[Dict] = Depends(get_current_user)
) -> Dict:
"""Check if user has any of the required permissions"""
# If OIDC disabled, allow all
if not oidc_config.enabled:
return user or {"email": "local", "groups": ["admin"]}
if user is None:
raise HTTPException(
status_code=401,
detail="Authentication required",
headers={"WWW-Authenticate": "Bearer"},
)
groups = user.get("groups", [])
permissions = _extract_permissions_from_groups(groups)
# Check each required permission
for perm in required_permissions:
try:
dom, cat, act = _parse_permission(perm)
if _user_has_permission(permissions, dom, cat, act):
logger.debug(f"User {user.get('email')} granted via {perm}")
return user
except ValueError:
logger.warning(f"Invalid permission format: {perm}")
continue
# None matched
user_email = user.get("email", "unknown")
logger.warning(
f"User {user_email} denied (required any of: {required_permissions})"
)
raise HTTPException(
status_code=403,
detail=f"One of these permissions required: {', '.join(required_permissions)}",
)
return permission_checker
+84 -2
View File
@@ -26,10 +26,12 @@ class AuthSyncRequest(BaseSchema):
class RoleSchema(BaseSchema): class RoleSchema(BaseSchema):
"""Role information in domain:action format""" """Role information in domain.category:action format"""
name: str = Field(..., description="Role name (e.g., 'control-room:admin')") id: uuid.UUID = Field(..., description="Role ID")
name: str = Field(..., description="Role name (e.g., 'control-room.general:admin')")
domain: str = Field(..., description="Permission domain (e.g., 'control-room')") domain: str = Field(..., description="Permission domain (e.g., 'control-room')")
category: str = Field(default="general", description="Permission category (e.g., 'general')")
action: str = Field(..., description="Permission action (e.g., 'admin')") action: str = Field(..., description="Permission action (e.g., 'admin')")
@@ -120,6 +122,7 @@ class GroupListItemSchema(BaseSchema):
parent_name: Optional[str] = Field(None, description="Parent group name") parent_name: Optional[str] = Field(None, description="Parent group name")
member_count: int = Field(default=0, description="Number of users in this group") member_count: int = Field(default=0, description="Number of users in this group")
synced_at: datetime = Field(..., description="Last sync timestamp") synced_at: datetime = Field(..., description="Last sync timestamp")
roles: list[str] = Field(default_factory=list, description="Assigned role names")
class GroupsListResponse(BaseSchema): class GroupsListResponse(BaseSchema):
@@ -127,3 +130,82 @@ class GroupsListResponse(BaseSchema):
items: list[GroupListItemSchema] = Field(..., description="List of groups") items: list[GroupListItemSchema] = Field(..., description="List of groups")
total: int = Field(..., description="Total count of groups") total: int = Field(..., description="Total count of groups")
class RolesListResponse(BaseSchema):
"""Response from GET /auth/roles"""
items: list[RoleSchema] = Field(..., description="List of all roles")
total: int = Field(..., description="Total count of roles")
class GroupRoleAssignmentResponse(BaseSchema):
"""Response from group role assignment operations"""
group_id: uuid.UUID = Field(..., description="Group ID")
group_name: str = Field(..., description="Group name")
roles: list[str] = Field(..., description="Currently assigned role names")
# =============================================================================
# User Profile (Phase 4)
# =============================================================================
class UserProfileResponse(BaseSchema):
"""Response from GET /users/me - full user profile"""
user: UserSchema = Field(..., description="User profile")
roles: list[RoleSchema] = Field(..., description="User's permission roles")
preferences: UserPreferencesSchema = Field(..., description="User preferences")
class PreferencesUpdateRequest(BaseSchema):
"""Request for PATCH /users/me/preferences"""
theme: Optional[str] = Field(None, description="Theme preference: system, light, dark")
default_room: Optional[str] = Field(None, description="Default room for housekeeping")
preferences_json: Optional[dict] = Field(None, description="Extended preferences (merged)")
# =============================================================================
# API Keys (Phase 4)
# =============================================================================
class ApiKeyCreateRequest(BaseSchema):
"""Request for POST /users/me/api-keys"""
name: str = Field(..., min_length=1, max_length=100, description="Human-readable key name")
scopes: Optional[list[str]] = Field(None, description="Optional scope restriction (role names)")
expires_in_days: Optional[int] = Field(None, ge=1, le=365, description="Days until expiration (optional)")
class ApiKeyCreateResponse(BaseSchema):
"""Response from POST /users/me/api-keys - includes the key (shown only once)"""
id: uuid.UUID = Field(..., description="API key ID")
name: str = Field(..., description="Key name")
key: str = Field(..., description="The API key (shown only once!)")
key_prefix: str = Field(..., description="Key prefix for identification")
scopes: Optional[list[str]] = Field(None, description="Scope restriction")
expires_at: Optional[datetime] = Field(None, description="Expiration timestamp")
created_at: datetime = Field(..., description="Creation timestamp")
class ApiKeySchema(BaseSchema):
"""API key information (without the actual key)"""
id: uuid.UUID = Field(..., description="API key ID")
name: str = Field(..., description="Key name")
key_prefix: str = Field(..., description="Key prefix for identification (e.g., 'tak_abc1')")
scopes: Optional[list[str]] = Field(None, description="Scope restriction")
expires_at: Optional[datetime] = Field(None, description="Expiration timestamp")
last_used_at: Optional[datetime] = Field(None, description="Last usage timestamp")
created_at: datetime = Field(..., description="Creation timestamp")
is_expired: bool = Field(..., description="Whether the key has expired")
class ApiKeysListResponse(BaseSchema):
"""Response from GET /users/me/api-keys"""
items: list[ApiKeySchema] = Field(..., description="List of API keys")
total: int = Field(..., description="Total count of keys")
+465 -16
View File
@@ -3,9 +3,11 @@ Authentication Service
Business logic for user synchronization from Authentik. Business logic for user synchronization from Authentik.
""" """
import hashlib
import re import re
import secrets
import uuid import uuid
from datetime import datetime, timezone from datetime import datetime, timezone, timedelta
from typing import Optional from typing import Optional
import httpx import httpx
@@ -15,10 +17,11 @@ from sqlalchemy.orm import selectinload
from src.shared.config import get_settings from src.shared.config import get_settings
from src.shared.logging import get_logger from src.shared.logging import get_logger
from src.domains.auth.models import User, Role, UserPreferences, Group from src.domains.auth.models import User, Role, UserPreferences, Group, ApiKey
from src.domains.auth.schemas import ( from src.domains.auth.schemas import (
TokenInfoSchema, UserSchema, RoleSchema, UserPreferencesSchema, TokenInfoSchema, UserSchema, RoleSchema, UserPreferencesSchema,
UserListItemSchema, BulkSyncResultSchema, GroupListItemSchema UserListItemSchema, BulkSyncResultSchema, GroupListItemSchema,
ApiKeySchema, ApiKeyCreateResponse,
) )
logger = get_logger(__name__) logger = get_logger(__name__)
@@ -99,7 +102,13 @@ class AuthService:
Returns: Returns:
Tuple of (User, is_new_user) Tuple of (User, is_new_user)
""" """
# Parse authentik_id - may be UUID or other format
try:
authentik_id = uuid.UUID(token_info.sub) authentik_id = uuid.UUID(token_info.sub)
except ValueError:
# If sub is not a valid UUID, derive one deterministically
logger.warning(f"sub claim is not a UUID: {token_info.sub}, deriving UUID")
authentik_id = uuid.uuid5(uuid.NAMESPACE_OID, token_info.sub)
# Try to find existing user # Try to find existing user
stmt = ( stmt = (
@@ -121,11 +130,14 @@ class AuthService:
avatar_url=token_info.picture, avatar_url=token_info.picture,
last_login=datetime.now(timezone.utc), last_login=datetime.now(timezone.utc),
) )
# Initialize relationships to avoid lazy loading issues in async
user.roles = []
self.session.add(user) self.session.add(user)
await self.session.flush() # Get the user ID await self.session.flush() # Get the user ID
# Create default preferences # Create default preferences and attach to user
preferences = UserPreferences(user_id=user.id) preferences = UserPreferences(user_id=user.id)
user.preferences = preferences
self.session.add(preferences) self.session.add(preferences)
logger.info(f"Created new user: {token_info.email}") logger.info(f"Created new user: {token_info.email}")
@@ -141,30 +153,111 @@ class AuthService:
await self.session.flush() await self.session.flush()
return user, is_new return user, is_new
async def sync_roles(self, user: User, groups: list[str]) -> list[Role]: async def sync_user_from_claims(
self,
authentik_id: uuid.UUID,
email: str,
name: str,
groups: list[str],
avatar_url: Optional[str] = None,
) -> tuple[User, bool]:
""" """
Synchronize user roles from Authentik groups Create or update user from forward auth claims (NPM X-authentik-* headers)
Maps Authentik groups (e.g., 'tatlock-control-room-admin') This is similar to sync_user() but works with raw claims instead of
to application roles (e.g., 'control-room:admin'). TokenInfoSchema. Used for auto-syncing users on first web login via NPM.
Args:
authentik_id: The Authentik user UUID (from X-authentik-uid)
email: User email (from X-authentik-email)
name: User display name (from X-authentik-name)
groups: List of group names (from X-authentik-groups)
avatar_url: Optional avatar URL
Returns:
Tuple of (User, is_new_user)
"""
# Try to find existing user
stmt = (
select(User)
.options(selectinload(User.roles), selectinload(User.preferences))
.where(User.authentik_id == authentik_id)
)
result = await self.session.execute(stmt)
user = result.scalar_one_or_none()
is_new = user is None
if is_new:
# Create new user
user = User(
authentik_id=authentik_id,
email=email,
name=name or email,
avatar_url=avatar_url,
last_login=datetime.now(timezone.utc),
)
self.session.add(user)
await self.session.flush() # Get the user ID
# Create default preferences
preferences = UserPreferences(user_id=user.id)
self.session.add(preferences)
logger.info(f"Created new user from forward auth: {email}")
else:
# Update existing user
user.email = email
user.name = name or email
if avatar_url:
user.avatar_url = avatar_url
user.last_login = datetime.now(timezone.utc)
logger.info(f"Updated existing user from forward auth: {email}")
# Sync roles from groups
await self.sync_roles(user, groups)
await self.session.flush()
return user, is_new
async def sync_roles(self, user: User, group_names: list[str]) -> list[Role]:
"""
Synchronize user roles from Authentik groups via group_roles mapping
Looks up the user's groups in the database, then retrieves all roles
assigned to those groups via the group_roles mapping table.
Args: Args:
user: User to sync roles for user: User to sync roles for
groups: List of Authentik group names group_names: List of Authentik group names
Returns: Returns:
List of synced Role objects List of synced Role objects
""" """
# Get all roles that match the user's Authentik groups # Find local Group records matching the Authentik group names
stmt = select(Role).where(Role.authentik_group.in_(groups)) stmt = (
select(Group)
.options(selectinload(Group.roles))
.where(Group.name.in_(group_names))
)
result = await self.session.execute(stmt) result = await self.session.execute(stmt)
matching_roles = list(result.scalars().all()) matching_groups = list(result.scalars().all())
# Collect all unique roles from all matching groups
roles_set: dict[uuid.UUID, Role] = {}
for group in matching_groups:
for role in group.roles:
roles_set[role.id] = role
matching_roles = list(roles_set.values())
# Clear existing roles and set new ones # Clear existing roles and set new ones
user.roles = matching_roles user.roles = matching_roles
role_names = [r.name for r in matching_roles] role_names = [r.name for r in matching_roles]
logger.info(f"Synced roles for {user.email}: {role_names}") group_names_found = [g.name for g in matching_groups]
logger.info(f"Synced roles for {user.email} via groups {group_names_found}: {role_names}")
return matching_roles return matching_roles
@@ -183,7 +276,7 @@ class AuthService:
def roles_to_schema(self, roles: list[Role]) -> list[RoleSchema]: def roles_to_schema(self, roles: list[Role]) -> list[RoleSchema]:
"""Convert Role models to schemas""" """Convert Role models to schemas"""
return [ return [
RoleSchema(name=r.name, domain=r.domain, action=r.action) RoleSchema(id=r.id, name=r.name, domain=r.domain, category=r.category, action=r.action)
for r in roles for r in roles
] ]
@@ -492,8 +585,8 @@ class AuthService:
""" """
from sqlalchemy import func from sqlalchemy import func
# Base query # Base query with roles loaded
base_query = select(Group) base_query = select(Group).options(selectinload(Group.roles))
# Apply search filter if provided # Apply search filter if provided
if search: if search:
@@ -520,12 +613,106 @@ class AuthService:
parent_name=group.parent_name, parent_name=group.parent_name,
member_count=group.member_count, member_count=group.member_count,
synced_at=group.synced_at, synced_at=group.synced_at,
roles=[role.name for role in group.roles],
) )
for group in groups for group in groups
] ]
return items, total return items, total
async def list_roles(self) -> list[Role]:
"""
List all available roles
Returns:
List of all Role objects
"""
stmt = select(Role).order_by(Role.domain, Role.category, Role.action)
result = await self.session.execute(stmt)
return list(result.scalars().all())
async def get_group_by_id(self, group_id: uuid.UUID) -> Optional[Group]:
"""
Get a group by its ID with roles loaded
Args:
group_id: The group's UUID
Returns:
Group object or None if not found
"""
stmt = (
select(Group)
.options(selectinload(Group.roles))
.where(Group.id == group_id)
)
result = await self.session.execute(stmt)
return result.scalar_one_or_none()
async def assign_role_to_group(self, group_id: uuid.UUID, role_id: uuid.UUID) -> Group:
"""
Assign a role to a group
Args:
group_id: The group's UUID
role_id: The role's UUID to assign
Returns:
Updated Group object
Raises:
ValueError: If group or role not found
"""
group = await self.get_group_by_id(group_id)
if not group:
raise ValueError(f"Group not found: {group_id}")
stmt = select(Role).where(Role.id == role_id)
result = await self.session.execute(stmt)
role = result.scalar_one_or_none()
if not role:
raise ValueError(f"Role not found: {role_id}")
# Add role if not already assigned
if role not in group.roles:
group.roles.append(role)
await self.session.flush()
logger.info(f"Assigned role {role.name} to group {group.name}")
return group
async def remove_role_from_group(self, group_id: uuid.UUID, role_id: uuid.UUID) -> Group:
"""
Remove a role from a group
Args:
group_id: The group's UUID
role_id: The role's UUID to remove
Returns:
Updated Group object
Raises:
ValueError: If group or role not found
"""
group = await self.get_group_by_id(group_id)
if not group:
raise ValueError(f"Group not found: {group_id}")
stmt = select(Role).where(Role.id == role_id)
result = await self.session.execute(stmt)
role = result.scalar_one_or_none()
if not role:
raise ValueError(f"Role not found: {role_id}")
# Remove role if assigned
if role in group.roles:
group.roles.remove(role)
await self.session.flush()
logger.info(f"Removed role {role.name} from group {group.name}")
return group
async def bulk_sync_groups_from_authentik(self) -> BulkSyncResultSchema: async def bulk_sync_groups_from_authentik(self) -> BulkSyncResultSchema:
""" """
Fetch all groups from Authentik admin API and sync to local database Fetch all groups from Authentik admin API and sync to local database
@@ -627,6 +814,268 @@ class AuthService:
errors=errors, errors=errors,
) )
# =========================================================================
# Phase 4: User Profile & Settings
# =========================================================================
async def get_user_by_authentik_id(self, authentik_id: uuid.UUID) -> Optional[User]:
"""
Get user by Authentik UUID with roles and preferences loaded
Args:
authentik_id: The Authentik user UUID (from JWT 'sub' claim)
Returns:
User object or None if not found
"""
stmt = (
select(User)
.options(selectinload(User.roles), selectinload(User.preferences))
.where(User.authentik_id == authentik_id)
)
result = await self.session.execute(stmt)
return result.scalar_one_or_none()
async def get_user_by_id(self, user_id: uuid.UUID) -> Optional[User]:
"""
Get user by internal UUID with roles and preferences loaded
Args:
user_id: The internal user UUID
Returns:
User object or None if not found
"""
stmt = (
select(User)
.options(selectinload(User.roles), selectinload(User.preferences))
.where(User.id == user_id)
)
result = await self.session.execute(stmt)
return result.scalar_one_or_none()
async def update_preferences(
self,
user_id: uuid.UUID,
theme: Optional[str] = None,
default_room: Optional[str] = None,
preferences_json: Optional[dict] = None,
) -> UserPreferences:
"""
Update user preferences
Args:
user_id: User's UUID
theme: New theme value (or None to keep existing)
default_room: New default room (or None to keep existing)
preferences_json: JSON to merge with existing (or None to keep existing)
Returns:
Updated UserPreferences object
Raises:
ValueError: If user not found
"""
stmt = select(UserPreferences).where(UserPreferences.user_id == user_id)
result = await self.session.execute(stmt)
prefs = result.scalar_one_or_none()
if prefs is None:
# Create preferences if they don't exist
prefs = UserPreferences(user_id=user_id)
self.session.add(prefs)
if theme is not None:
if theme not in ("system", "light", "dark"):
raise ValueError(f"Invalid theme: {theme}")
prefs.theme = theme
if default_room is not None:
prefs.default_room = default_room
if preferences_json is not None:
# Merge with existing preferences
existing = prefs.preferences_json or {}
existing.update(preferences_json)
prefs.preferences_json = existing
await self.session.flush()
logger.info(f"Updated preferences for user {user_id}")
return prefs
# =========================================================================
# Phase 4: API Keys
# =========================================================================
def _generate_api_key(self) -> tuple[str, str, str]:
"""
Generate a new API key
Returns:
Tuple of (full_key, key_hash, key_prefix)
"""
# Generate 32 random bytes = 256 bits of entropy
random_bytes = secrets.token_bytes(32)
# Encode as base64-like string (URL-safe)
key_body = secrets.token_urlsafe(32)
# Prefix with 'tak_' (tatlock api key)
full_key = f"tak_{key_body}"
# Hash for storage
key_hash = hashlib.sha256(full_key.encode()).hexdigest()
# Prefix for identification (first 8 chars after 'tak_')
key_prefix = f"tak_{key_body[:4]}"
return full_key, key_hash, key_prefix
async def create_api_key(
self,
user_id: uuid.UUID,
name: str,
scopes: Optional[list[str]] = None,
expires_in_days: Optional[int] = None,
) -> tuple[ApiKey, str]:
"""
Create a new API key for a user
Args:
user_id: User's UUID
name: Human-readable key name
scopes: Optional list of scope restrictions
expires_in_days: Optional expiration in days
Returns:
Tuple of (ApiKey object, full key string)
The full key is only returned once at creation!
Raises:
ValueError: If user not found or API keys disabled
"""
# Check user exists and has API keys enabled
user = await self.get_user_by_id(user_id)
if user is None:
raise ValueError(f"User not found: {user_id}")
if not user.api_keys_enabled:
raise ValueError("API keys are disabled for this user")
# Generate the key
full_key, key_hash, key_prefix = self._generate_api_key()
# Calculate expiration
expires_at = None
if expires_in_days:
expires_at = datetime.now(timezone.utc) + timedelta(days=expires_in_days)
# Create the key record
api_key = ApiKey(
user_id=user_id,
name=name,
key_hash=key_hash,
key_prefix=key_prefix,
scopes=scopes,
expires_at=expires_at,
)
self.session.add(api_key)
await self.session.flush()
logger.info(f"Created API key '{name}' for user {user_id}")
return api_key, full_key
async def list_user_api_keys(self, user_id: uuid.UUID) -> list[ApiKey]:
"""
List all API keys for a user
Args:
user_id: User's UUID
Returns:
List of ApiKey objects (without the actual keys)
"""
stmt = (
select(ApiKey)
.where(ApiKey.user_id == user_id)
.order_by(ApiKey.created_at.desc())
)
result = await self.session.execute(stmt)
return list(result.scalars().all())
async def delete_api_key(self, user_id: uuid.UUID, key_id: uuid.UUID) -> bool:
"""
Delete an API key
Args:
user_id: User's UUID (for authorization)
key_id: API key UUID
Returns:
True if deleted, False if not found
Raises:
ValueError: If key belongs to different user
"""
stmt = select(ApiKey).where(ApiKey.id == key_id)
result = await self.session.execute(stmt)
api_key = result.scalar_one_or_none()
if api_key is None:
return False
if api_key.user_id != user_id:
raise ValueError("API key belongs to different user")
await self.session.delete(api_key)
await self.session.flush()
logger.info(f"Deleted API key {key_id} for user {user_id}")
return True
async def validate_api_key(self, key: str) -> Optional[User]:
"""
Validate an API key and return the associated user
Args:
key: The full API key string
Returns:
User object if valid, None if invalid/expired
"""
# Hash the provided key
key_hash = hashlib.sha256(key.encode()).hexdigest()
# Look up by hash
stmt = (
select(ApiKey)
.options(selectinload(ApiKey.user).selectinload(User.roles))
.where(ApiKey.key_hash == key_hash)
)
result = await self.session.execute(stmt)
api_key = result.scalar_one_or_none()
if api_key is None:
return None
# Check expiration
if api_key.is_expired:
logger.warning(f"Expired API key used: {api_key.key_prefix}...")
return None
# Update last used timestamp
api_key.last_used_at = datetime.now(timezone.utc)
logger.debug(f"API key authenticated: {api_key.key_prefix}... for user {api_key.user.email}")
return api_key.user
def api_key_to_schema(self, api_key: ApiKey) -> ApiKeySchema:
"""Convert ApiKey model to schema"""
return ApiKeySchema(
id=api_key.id,
name=api_key.name,
key_prefix=api_key.key_prefix,
scopes=api_key.scopes,
expires_at=api_key.expires_at,
last_used_at=api_key.last_used_at,
created_at=api_key.created_at,
is_expired=api_key.is_expired,
)
# Factory function for dependency injection # Factory function for dependency injection
def get_auth_service(session: AsyncSession) -> AuthService: def get_auth_service(session: AsyncSession) -> AuthService:
+1
View File
@@ -31,6 +31,7 @@ class QuickLink(Base):
# Ordering and display # Ordering and display
position = Column(Integer, default=0) position = Column(Integer, default=0)
is_visible = Column(Boolean, default=True) is_visible = Column(Boolean, default=True)
link_type = Column(String(20), default="iframe") # "iframe", "new_tab", etc.
# Styling # Styling
color = Column(String(20), nullable=True) # Hex color for the link card color = Column(String(20), nullable=True) # Hex color for the link card
+3
View File
@@ -26,6 +26,7 @@ class QuickLinkCreate(QuickLinkBase):
"""Schema for creating a quick link""" """Schema for creating a quick link"""
position: Optional[int] = Field(0, ge=0, description="Display position") position: Optional[int] = Field(0, ge=0, description="Display position")
is_visible: Optional[bool] = Field(True, description="Whether link is visible") is_visible: Optional[bool] = Field(True, description="Whether link is visible")
link_type: Optional[str] = Field("iframe", max_length=20, description="Link type: iframe, new_tab")
class QuickLinkUpdate(BaseSchema): class QuickLinkUpdate(BaseSchema):
@@ -37,6 +38,7 @@ class QuickLinkUpdate(BaseSchema):
category: Optional[str] = Field(None, max_length=50) category: Optional[str] = Field(None, max_length=50)
position: Optional[int] = Field(None, ge=0) position: Optional[int] = Field(None, ge=0)
is_visible: Optional[bool] = None is_visible: Optional[bool] = None
link_type: Optional[str] = Field(None, max_length=20)
color: Optional[str] = Field(None, max_length=20) color: Optional[str] = Field(None, max_length=20)
background_color: Optional[str] = Field(None, max_length=20) background_color: Optional[str] = Field(None, max_length=20)
@@ -47,6 +49,7 @@ class QuickLinkResponse(QuickLinkBase):
user_id: Optional[str] = None user_id: Optional[str] = None
position: int position: int
is_visible: bool is_visible: bool
link_type: str = "iframe"
created_at: datetime created_at: datetime
updated_at: datetime updated_at: datetime
+11 -96
View File
@@ -70,66 +70,18 @@ class HealthController(BaseController):
@router.get( @router.get(
"/health/full", "/health/full",
summary="Fast health check for Docker", summary="Full health check with database",
) )
async def full_health_check(response: Response): async def full_health_check(response: Response):
""" """
Fast health check for container orchestration (Docker/K8s). Health check including database connectivity.
Checks component availability WITHOUT running expensive operations. Returns 200 OK if database is available, otherwise 503.
Returns 200 OK if all components are available, otherwise 503.
For detailed diagnostics, use /health/diagnostics instead.
""" """
import time import time
start_time = time.time() start_time = time.time()
# Import here to avoid circular imports # Check database connection
from src.models.ollama_client import get_ollama_client
# Check 1: Ollama connection + verify agent model is available
ollama_client = get_ollama_client()
ollama_healthy = False
ollama_error = None
model_available = False
try:
# Ping Ollama
ollama_healthy = await ollama_client.health_check()
# Verify the agent model is pulled and check what's currently loaded
models_info = {}
if ollama_healthy:
try:
models_response = await ollama_client.list_models()
available_models = [m.get('name', '') for m in models_response.get('models', [])]
model_available = settings.agent_model in available_models
# Get info about currently loaded models (those with size in memory)
loaded_models = [
m.get('name', '') for m in models_response.get('models', [])
if m.get('size', 0) > 0
]
models_info = {
"configured": settings.agent_model,
"available": model_available,
"total_in_ollama": len(available_models),
"currently_loaded": loaded_models if loaded_models else ["none"]
}
if not model_available:
ollama_error = f"Model '{settings.agent_model}' not found in Ollama. Available: {', '.join(available_models[:3])}"
ollama_healthy = False
except Exception as e:
ollama_error = f"Could not list Ollama models: {str(e)}"
ollama_healthy = False
except Exception as e:
ollama_error = str(e)
logger.warning(f"Ollama health check failed: {ollama_error}")
# Check 2: Database connection
database = get_database() database = get_database()
db_healthy = False db_healthy = False
db_error = None db_error = None
@@ -140,25 +92,15 @@ class HealthController(BaseController):
db_error = str(e) db_error = str(e)
logger.warning(f"Database health check failed: {db_error}") logger.warning(f"Database health check failed: {db_error}")
is_healthy = ollama_healthy and db_healthy
elapsed_ms = int((time.time() - start_time) * 1000) elapsed_ms = int((time.time() - start_time) * 1000)
status_code = 200 if is_healthy else 503 status_code = 200 if db_healthy else 503
response.status_code = status_code response.status_code = status_code
return { return {
"status": "healthy" if is_healthy else "unhealthy", "status": "healthy" if db_healthy else "unhealthy",
"status_code": status_code, "status_code": status_code,
"response_time_ms": elapsed_ms, "response_time_ms": elapsed_ms,
"components": { "components": {
"ollama": {
"status": "healthy" if ollama_healthy else "unhealthy",
"models": models_info if models_info else {
"configured": settings.agent_model,
"available": False
},
"error": ollama_error
},
"database": { "database": {
"status": "healthy" if db_healthy else "unhealthy", "status": "healthy" if db_healthy else "unhealthy",
"error": db_error "error": db_error
@@ -170,19 +112,13 @@ class HealthController(BaseController):
"/health/diagnostics", "/health/diagnostics",
summary="Detailed system diagnostics", summary="Detailed system diagnostics",
) )
async def diagnostics(deep_test: bool = False): async def diagnostics():
""" """
Comprehensive system diagnostics with detailed component information. System diagnostics with service information.
Query Parameters:
- deep_test: Set to true to actually test agent generation (slow, ~5-10s)
Returns detailed information about all system components.
""" """
import time import time
from src.models.ollama_client import get_ollama_client
start_time = time.time() start_time = time.time()
diagnostics = { diagnostics = {
"timestamp": time.time(), "timestamp": time.time(),
"service": { "service": {
@@ -190,31 +126,10 @@ class HealthController(BaseController):
"version": settings.app_version, "version": settings.app_version,
"purpose": "Infrastructure management and tools API" "purpose": "Infrastructure management and tools API"
}, },
"components": {} "configuration": {
}
# 1. Ollama Connection
ollama_client = get_ollama_client()
try:
ollama_healthy = await ollama_client.health_check()
diagnostics["components"]["ollama"] = {
"status": "connected",
"url": settings.ollama_base_url,
"timeout": settings.ollama_timeout,
"default_model": settings.default_model
}
except Exception as e:
diagnostics["components"]["ollama"] = {
"status": "error",
"error": str(e)
}
# 2. Configuration
diagnostics["configuration"] = {
"agent_fallback_enabled": settings.agent_fallback_enabled,
"memory_tier1_max_turns": settings.memory_tier1_max_turns,
"cors_origins": settings.cors_origins[:2] if len(settings.cors_origins) > 2 else settings.cors_origins "cors_origins": settings.cors_origins[:2] if len(settings.cors_origins) > 2 else settings.cors_origins
} }
}
elapsed_ms = int((time.time() - start_time) * 1000) elapsed_ms = int((time.time() - start_time) * 1000)
diagnostics["response_time_ms"] = elapsed_ms diagnostics["response_time_ms"] = elapsed_ms
+9 -2
View File
@@ -1,9 +1,16 @@
""" """
Tools Domain Tools Domain
Provides utility tool endpoints including DNS lookups. Provides utility tool endpoints including DNS lookups and system stats.
""" """
from src.domains.tools.controller import tools_controller from src.domains.tools.controller import tools_controller
from src.domains.tools.dns import DNSService, DNSQueryError from src.domains.tools.dns import DNSService, DNSQueryError
from src.domains.tools.system import SystemStatsService, SystemStatsResponse
__all__ = ["tools_controller", "DNSService", "DNSQueryError"] __all__ = [
"tools_controller",
"DNSService",
"DNSQueryError",
"SystemStatsService",
"SystemStatsResponse",
]
+181 -1
View File
@@ -3,14 +3,24 @@ Tools Controller
Provides utility tool endpoints including: Provides utility tool endpoints including:
- DNS lookups - DNS lookups
- System stats
- Environment data (weather, forecast, sun times)
""" """
from fastapi import APIRouter, HTTPException, status from typing import Dict, Optional
from fastapi import APIRouter, HTTPException, status, Depends
from src.shared.base import BaseController from src.shared.base import BaseController
from src.shared.logging import get_logger from src.shared.logging import get_logger
from src.domains.tools.dns.schemas import DNSLookupRequest, DNSLookupResponse from src.domains.tools.dns.schemas import DNSLookupRequest, DNSLookupResponse
from src.domains.tools.dns.service import DNSService from src.domains.tools.dns.service import DNSService
from src.domains.tools.dns.exceptions import DNSQueryError from src.domains.tools.dns.exceptions import DNSQueryError
from src.domains.tools.system.schemas import SystemStatsResponse
from src.domains.tools.system.service import SystemStatsService
from src.domains.tools.environment.schemas import EnvironmentResponse
from src.domains.tools.environment.service import EnvironmentService
from src.domains.tools.news.schemas import NewsResponse
from src.domains.tools.news.service import NewsService
from src.domains.auth.oidc import get_optional_user
logger = get_logger(__name__) logger = get_logger(__name__)
@@ -21,11 +31,16 @@ class ToolsController(BaseController):
Provides endpoints for: Provides endpoints for:
- DNS lookups - DNS lookups
- System stats
- Environment data (weather, forecast, sun times)
""" """
def __init__(self): def __init__(self):
super().__init__(prefix="/tools", tags=["Tools"]) super().__init__(prefix="/tools", tags=["Tools"])
self.dns_service = DNSService() self.dns_service = DNSService()
self.system_stats_service = SystemStatsService()
self.environment_service = EnvironmentService()
self.news_service = NewsService()
def create_router(self) -> APIRouter: def create_router(self) -> APIRouter:
"""Create and configure the router""" """Create and configure the router"""
@@ -95,6 +110,171 @@ class ToolsController(BaseController):
detail="An unexpected error occurred during DNS lookup" detail="An unexpected error occurred during DNS lookup"
) )
@router.get(
"/system/stats",
response_model=SystemStatsResponse,
status_code=status.HTTP_200_OK,
summary="Get host system statistics",
description="""
Get real-time host system resource statistics.
Returns CPU, memory, disk, network, and GPU/VRAM usage for the host machine
(not Docker container metrics).
**Metrics Returned:**
- **CPU:** Usage percentage, core count, load averages
- **Memory:** Usage percentage, total/used/available bytes
- **Disk:** Usage percentage, total/used/free bytes (root partition)
- **Network:** Total bytes sent/received
- **GPU:** VRAM usage (if NVIDIA GPU available via nvidia-smi)
**Use Cases:**
- Dashboard system monitoring widgets
- Health checks and alerting
- Capacity planning
"""
)
async def get_system_stats() -> SystemStatsResponse:
"""
Get current host system statistics
Returns:
System statistics including CPU, memory, disk, network, and GPU
Raises:
HTTPException: 500 for processing errors
"""
try:
logger.info("Fetching system stats")
result = await self.system_stats_service.get_stats()
return result
except Exception as e:
logger.error(f"Failed to get system stats: {str(e)}", exc_info=True)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Failed to collect system stats: {str(e)}"
)
@router.get(
"/environment",
response_model=EnvironmentResponse,
status_code=status.HTTP_200_OK,
summary="Get environment data",
description="""
Get current environment data including weather, forecast, and sun times.
Fetches data from the Qdrant volatile collection for the authenticated user.
Falls back to 'default' user if not authenticated.
**Data Returned:**
- **Weather:** Current temperature, conditions, humidity, wind
- **Forecast:** Multi-day weather outlook
- **Sun Times:** Sunrise, sunset, daylight duration
- **Air Quality:** AQI and pollutant levels (if available)
**Data Source:** Qdrant volatile_{user} collection
**Use Cases:**
- Dashboard environment widgets
- Home automation context
- Weather-based automations
"""
)
async def get_environment(
user: Optional[Dict] = Depends(get_optional_user),
) -> EnvironmentResponse:
"""
Get current environment data
Args:
user: Optional authenticated user from OIDC
Returns:
Environment data including weather, forecast, sun times
Raises:
HTTPException: 500 for processing errors
"""
try:
# Get user identifier from OIDC claims, fallback to 'default'
user_id = "default"
if user:
user_id = user.get("preferred_username") or user.get("sub", "default")
# Strip email domain if present (e.g., "user@example.com" -> "user")
if "@" in user_id:
user_id = user_id.split("@")[0]
logger.info(f"Fetching environment data for user: {user_id}")
result = await self.environment_service.get_current(user_id)
return result
except Exception as e:
logger.error(f"Failed to get environment data: {str(e)}", exc_info=True)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Failed to fetch environment data: {str(e)}"
)
@router.get(
"/news",
response_model=NewsResponse,
status_code=status.HTTP_200_OK,
summary="Get news headlines",
description="""
Get news headlines for the authenticated user.
Fetches news data from the Qdrant volatile collection for the authenticated user.
Falls back to 'default' user if not authenticated.
**Data Returned:**
- **Headlines:** List of news headlines with title, description, source, url
- **Category:** News category (general, technology, etc.)
- **Sources:** List of news sources
**Data Source:** Qdrant volatile_{user} collection (news namespace)
**Use Cases:**
- Dashboard news ticker
- News feed widgets
- Information display
"""
)
async def get_news(
user: Optional[Dict] = Depends(get_optional_user),
) -> NewsResponse:
"""
Get news headlines
Args:
user: Optional authenticated user from OIDC
Returns:
News headlines response
Raises:
HTTPException: 500 for processing errors
"""
try:
# Get user identifier from OIDC claims, fallback to 'default'
user_id = "default"
if user:
user_id = user.get("preferred_username") or user.get("sub", "default")
# Strip email domain if present (e.g., "user@example.com" -> "user")
if "@" in user_id:
user_id = user_id.split("@")[0]
logger.info(f"Fetching news data for user: {user_id}")
result = await self.news_service.get_news(user_id)
return result
except Exception as e:
logger.error(f"Failed to get news data: {str(e)}", exc_info=True)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Failed to fetch news data: {str(e)}"
)
return router return router
+23
View File
@@ -0,0 +1,23 @@
"""
Environment data module for Tools domain.
Provides access to weather, forecast, sun times, and air quality data
from the Qdrant volatile collection.
"""
from src.domains.tools.environment.schemas import (
WeatherData,
ForecastDay,
SunTimesData,
AirQualityData,
EnvironmentResponse,
)
from src.domains.tools.environment.service import EnvironmentService
__all__ = [
"WeatherData",
"ForecastDay",
"SunTimesData",
"AirQualityData",
"EnvironmentResponse",
"EnvironmentService",
]
+185
View File
@@ -0,0 +1,185 @@
"""
Environment data schemas for Tools domain.
Provides Pydantic models for weather, forecast, sun times, and air quality data
retrieved from the Qdrant volatile collection.
"""
from datetime import datetime
from typing import Optional, List, Any
from pydantic import Field
from src.shared.base import BaseSchema
class WeatherData(BaseSchema):
"""Current weather conditions."""
temperature: Optional[float] = Field(
None,
description="Current temperature in Celsius"
)
feels_like: Optional[float] = Field(
None,
description="Feels-like temperature in Celsius"
)
conditions: Optional[str] = Field(
None,
description="Weather conditions description (e.g., 'Partly Cloudy')"
)
humidity: Optional[int] = Field(
None,
ge=0,
le=100,
description="Humidity percentage"
)
wind_speed: Optional[float] = Field(
None,
description="Wind speed in km/h"
)
wind_direction: Optional[str] = Field(
None,
description="Wind direction (e.g., 'NW')"
)
pressure: Optional[float] = Field(
None,
description="Atmospheric pressure in hPa"
)
visibility: Optional[float] = Field(
None,
description="Visibility in km"
)
uv_index: Optional[float] = Field(
None,
description="UV index"
)
location: Optional[str] = Field(
None,
description="Location name"
)
icon: Optional[str] = Field(
None,
description="Weather icon code or URL"
)
class ForecastDay(BaseSchema):
"""Single day forecast data."""
date: str = Field(
...,
description="Date string (e.g., '2025-01-07')"
)
high: Optional[float] = Field(
None,
description="High temperature in Celsius"
)
low: Optional[float] = Field(
None,
description="Low temperature in Celsius"
)
conditions: Optional[str] = Field(
None,
description="Weather conditions description"
)
precipitation_chance: Optional[int] = Field(
None,
ge=0,
le=100,
description="Chance of precipitation percentage"
)
icon: Optional[str] = Field(
None,
description="Weather icon code or URL"
)
class SunTimesData(BaseSchema):
"""Sunrise and sunset times."""
sunrise: Optional[datetime] = Field(
None,
description="Sunrise time"
)
sunset: Optional[datetime] = Field(
None,
description="Sunset time"
)
daylight_minutes: Optional[int] = Field(
None,
description="Total daylight duration in minutes"
)
solar_noon: Optional[datetime] = Field(
None,
description="Solar noon time"
)
dawn: Optional[datetime] = Field(
None,
description="Civil dawn time"
)
dusk: Optional[datetime] = Field(
None,
description="Civil dusk time"
)
class AirQualityData(BaseSchema):
"""Air quality information."""
aqi: Optional[int] = Field(
None,
ge=0,
description="Air Quality Index"
)
quality: Optional[str] = Field(
None,
description="Quality category (Good, Moderate, Unhealthy, etc.)"
)
pm25: Optional[float] = Field(
None,
description="PM2.5 concentration in microg/m3"
)
pm10: Optional[float] = Field(
None,
description="PM10 concentration in microg/m3"
)
o3: Optional[float] = Field(
None,
description="Ozone concentration in ppb"
)
no2: Optional[float] = Field(
None,
description="Nitrogen dioxide concentration in ppb"
)
location: Optional[str] = Field(
None,
description="Location name"
)
class EnvironmentResponse(BaseSchema):
"""Combined environment data response."""
weather: Optional[WeatherData] = Field(
None,
description="Current weather conditions"
)
forecast: Optional[List[ForecastDay]] = Field(
None,
description="Multi-day weather forecast"
)
sun_times: Optional[SunTimesData] = Field(
None,
description="Sunrise/sunset times"
)
air_quality: Optional[AirQualityData] = Field(
None,
description="Air quality data (None if not available)"
)
updated_at: datetime = Field(
default_factory=datetime.utcnow,
description="Timestamp when data was fetched"
)
user: Optional[str] = Field(
None,
description="User identifier used for data lookup"
)
+287
View File
@@ -0,0 +1,287 @@
"""
Environment data service for Tools domain.
Fetches weather, forecast, sun times, and air quality data from
the Qdrant volatile collection.
"""
from datetime import datetime
from typing import Optional, Dict, Any, List
from src.shared.logging import get_logger
from src.shared.clients.qdrant_client import get_qdrant_client
from src.domains.tools.environment.schemas import (
WeatherData,
ForecastDay,
SunTimesData,
AirQualityData,
EnvironmentResponse,
)
logger = get_logger(__name__)
class EnvironmentService:
"""
Service for fetching environment data from Qdrant volatile collection.
Retrieves weather, forecast, sun times, and optionally air quality
data for a specific user.
"""
def __init__(self):
"""Initialize environment service with Qdrant client."""
self.qdrant = get_qdrant_client()
def _parse_weather(self, raw_data: Optional[Dict[str, Any]]) -> Optional[WeatherData]:
"""
Parse raw weather data into WeatherData schema.
Handles various field naming conventions that might come from
different weather APIs.
"""
if not raw_data:
return None
try:
# Handle wind direction - convert degrees to cardinal if integer
wind_dir = raw_data.get("wind_direction") or raw_data.get("wind_dir")
if isinstance(wind_dir, (int, float)):
wind_dir = self._degrees_to_cardinal(wind_dir)
return WeatherData(
temperature=raw_data.get("temperature") or raw_data.get("temp"),
feels_like=raw_data.get("feels_like") or raw_data.get("feelslike"),
conditions=raw_data.get("conditions") or raw_data.get("weather") or raw_data.get("description"),
humidity=raw_data.get("humidity"),
wind_speed=raw_data.get("wind_speed") or raw_data.get("windspeed") or raw_data.get("wind"),
wind_direction=wind_dir,
pressure=raw_data.get("pressure"),
visibility=raw_data.get("visibility"),
uv_index=raw_data.get("uv_index") or raw_data.get("uv"),
location=raw_data.get("location") or raw_data.get("city"),
icon=raw_data.get("icon") or raw_data.get("icon_url"),
)
except Exception as e:
logger.warning(f"Failed to parse weather data: {e}")
return None
def _degrees_to_cardinal(self, degrees: float) -> str:
"""Convert wind direction degrees to cardinal direction."""
directions = ["N", "NNE", "NE", "ENE", "E", "ESE", "SE", "SSE",
"S", "SSW", "SW", "WSW", "W", "WNW", "NW", "NNW"]
index = round(degrees / 22.5) % 16
return directions[index]
def _parse_forecast(self, raw_data: Any) -> Optional[List[ForecastDay]]:
"""
Parse raw forecast data into list of ForecastDay schemas.
Handles both list format and dict with nested list.
"""
if not raw_data:
return None
try:
# Normalize to list
forecast_list = raw_data
if isinstance(raw_data, dict):
# Check 'daily' first (scheduler format), then 'forecast', then 'days'
# Note: 'days' might be an integer count, so check 'daily' first
forecast_list = raw_data.get("daily") or raw_data.get("forecast")
if forecast_list is None:
days_value = raw_data.get("days")
if isinstance(days_value, list):
forecast_list = days_value
else:
forecast_list = []
if not isinstance(forecast_list, list):
return None
days = []
for day in forecast_list:
if isinstance(day, dict):
days.append(ForecastDay(
date=day.get("date", ""),
high=day.get("high") or day.get("temp_high") or day.get("maxtemp") or day.get("temp_max"),
low=day.get("low") or day.get("temp_low") or day.get("mintemp") or day.get("temp_min"),
conditions=day.get("conditions") or day.get("weather") or day.get("description"),
precipitation_chance=day.get("precipitation_chance") or day.get("pop") or day.get("precip"),
icon=day.get("icon"),
))
return days if days else None
except Exception as e:
logger.warning(f"Failed to parse forecast data: {e}")
return None
def _parse_sun_times(self, raw_data: Optional[Dict[str, Any]]) -> Optional[SunTimesData]:
"""
Parse raw sun times data into SunTimesData schema.
Handles datetime strings and calculates daylight minutes if not provided.
"""
if not raw_data:
return None
try:
# Prefer ISO format fields (sunrise_iso, sunset_iso) over time-only fields
sunrise = raw_data.get("sunrise_iso") or raw_data.get("sunrise")
sunset = raw_data.get("sunset_iso") or raw_data.get("sunset")
# Parse datetime strings if needed
if isinstance(sunrise, str):
# Handle time-only format (HH:MM) by combining with today's date
if len(sunrise) <= 5 and ":" in sunrise:
today = datetime.now().date()
sunrise = datetime.strptime(f"{today} {sunrise}", "%Y-%m-%d %H:%M")
else:
sunrise = datetime.fromisoformat(sunrise.replace("Z", "+00:00"))
if isinstance(sunset, str):
# Handle time-only format (HH:MM) by combining with today's date
if len(sunset) <= 5 and ":" in sunset:
today = datetime.now().date()
sunset = datetime.strptime(f"{today} {sunset}", "%Y-%m-%d %H:%M")
else:
sunset = datetime.fromisoformat(sunset.replace("Z", "+00:00"))
# Get daylight from various field names
daylight_minutes = raw_data.get("daylight_minutes") or raw_data.get("daylight")
if daylight_minutes is None:
# Try to calculate from daylight_duration_seconds or daylight_hours
daylight_seconds = raw_data.get("daylight_duration_seconds")
if daylight_seconds:
daylight_minutes = int(daylight_seconds / 60)
else:
daylight_hours = raw_data.get("daylight_hours")
if daylight_hours:
daylight_minutes = int(daylight_hours * 60)
elif sunrise and sunset:
daylight_minutes = int((sunset - sunrise).total_seconds() / 60)
# Parse optional fields
solar_noon = raw_data.get("solar_noon")
if isinstance(solar_noon, str):
solar_noon = datetime.fromisoformat(solar_noon.replace("Z", "+00:00"))
dawn = raw_data.get("dawn") or raw_data.get("civil_dawn")
if isinstance(dawn, str):
dawn = datetime.fromisoformat(dawn.replace("Z", "+00:00"))
dusk = raw_data.get("dusk") or raw_data.get("civil_dusk")
if isinstance(dusk, str):
dusk = datetime.fromisoformat(dusk.replace("Z", "+00:00"))
return SunTimesData(
sunrise=sunrise,
sunset=sunset,
daylight_minutes=daylight_minutes,
solar_noon=solar_noon,
dawn=dawn,
dusk=dusk,
)
except Exception as e:
logger.warning(f"Failed to parse sun times data: {e}")
return None
def _parse_air_quality(self, raw_data: Any) -> Optional[AirQualityData]:
"""
Parse raw air quality data into AirQualityData schema.
Handles both dict format and simple integer AQI value.
"""
if raw_data is None:
return None
try:
# Handle simple integer AQI
if isinstance(raw_data, (int, float)):
aqi = int(raw_data)
return AirQualityData(
aqi=aqi,
quality=self._aqi_to_quality(aqi),
)
if not isinstance(raw_data, dict):
return None
# Try various AQI field names - prefer US AQI, then European, then generic
aqi = raw_data.get("aqi") or raw_data.get("aqi_us") or raw_data.get("aqi_european") or raw_data.get("index")
if isinstance(aqi, (int, float)):
aqi = int(aqi)
return AirQualityData(
aqi=aqi,
quality=raw_data.get("quality") or (self._aqi_to_quality(aqi) if aqi else None),
pm25=raw_data.get("pm25") or raw_data.get("pm2_5"),
pm10=raw_data.get("pm10"),
o3=raw_data.get("o3") or raw_data.get("ozone"),
no2=raw_data.get("no2") or raw_data.get("nitrogen_dioxide"),
location=raw_data.get("location"),
)
except Exception as e:
logger.warning(f"Failed to parse air quality data: {e}")
return None
def _aqi_to_quality(self, aqi: int) -> str:
"""Convert AQI value to quality category string."""
if aqi <= 50:
return "Good"
elif aqi <= 100:
return "Moderate"
elif aqi <= 150:
return "Unhealthy for Sensitive Groups"
elif aqi <= 200:
return "Unhealthy"
elif aqi <= 300:
return "Very Unhealthy"
else:
return "Hazardous"
async def get_current(self, user: str = "default") -> EnvironmentResponse:
"""
Get current environment data for a user.
Fetches weather, forecast, sun times, and air quality from
the user's volatile collection.
Args:
user: User identifier (default: 'default')
Returns:
EnvironmentResponse with all available data
"""
logger.info(f"Fetching environment data for user: {user}")
# Get raw data from Qdrant
raw_data = await self.qdrant.get_environment_data(user)
# Parse each data type
weather = self._parse_weather(raw_data.get("weather"))
forecast = self._parse_forecast(raw_data.get("forecast"))
sun_times = self._parse_sun_times(raw_data.get("sun_times"))
air_quality = self._parse_air_quality(raw_data.get("air_quality"))
return EnvironmentResponse(
weather=weather,
forecast=forecast,
sun_times=sun_times,
air_quality=air_quality,
updated_at=datetime.utcnow(),
user=user,
)
# Singleton instance
_environment_service: Optional[EnvironmentService] = None
def get_environment_service() -> EnvironmentService:
"""Get or create singleton environment service instance."""
global _environment_service
if _environment_service is None:
_environment_service = EnvironmentService()
return _environment_service
+5
View File
@@ -0,0 +1,5 @@
"""
News subdomain for Tools.
Provides news headlines from Qdrant volatile collection.
"""
+56
View File
@@ -0,0 +1,56 @@
"""
News data schemas for Tools domain.
Provides Pydantic models for news headlines retrieved from the Qdrant volatile collection.
"""
from datetime import datetime
from typing import Optional, List
from pydantic import Field
from src.shared.base import BaseSchema
class NewsHeadline(BaseSchema):
"""Single news headline."""
title: str = Field(
...,
description="Headline title"
)
description: Optional[str] = Field(
None,
description="Brief description or summary"
)
source: Optional[str] = Field(
None,
description="News source name"
)
url: Optional[str] = Field(
None,
description="Link to full article"
)
class NewsResponse(BaseSchema):
"""News headlines response."""
headlines: List[NewsHeadline] = Field(
default_factory=list,
description="List of news headlines"
)
category: Optional[str] = Field(
None,
description="News category (e.g., 'general', 'technology')"
)
sources: Optional[List[str]] = Field(
None,
description="List of source names"
)
updated_at: datetime = Field(
default_factory=datetime.utcnow,
description="Timestamp when data was fetched"
)
user: Optional[str] = Field(
None,
description="User identifier used for data lookup"
)
+113
View File
@@ -0,0 +1,113 @@
"""
News data service for Tools domain.
Fetches news headlines from the Qdrant volatile collection.
"""
from datetime import datetime
from typing import Optional, Dict, Any, List
from src.shared.logging import get_logger
from src.shared.clients.qdrant_client import get_qdrant_client
from src.domains.tools.news.schemas import (
NewsHeadline,
NewsResponse,
)
logger = get_logger(__name__)
class NewsService:
"""
Service for fetching news data from Qdrant volatile collection.
Retrieves news headlines for a specific user.
"""
def __init__(self):
"""Initialize news service with Qdrant client."""
self.qdrant = get_qdrant_client()
def _parse_headlines(self, raw_data: Any) -> List[NewsHeadline]:
"""
Parse raw news data into list of NewsHeadline schemas.
Handles various formats from different news sources.
"""
if not raw_data:
return []
try:
# Handle dict with nested headlines list
headlines_list = raw_data
if isinstance(raw_data, dict):
headlines_list = raw_data.get("headlines") or raw_data.get("articles") or []
if not isinstance(headlines_list, list):
return []
headlines = []
for item in headlines_list:
if isinstance(item, dict):
headlines.append(NewsHeadline(
title=item.get("title", ""),
description=item.get("description") or item.get("summary"),
source=item.get("source") or item.get("provider"),
url=item.get("url") or item.get("link"),
))
elif isinstance(item, str):
# Simple string headlines
headlines.append(NewsHeadline(title=item))
return headlines
except Exception as e:
logger.warning(f"Failed to parse news headlines: {e}")
return []
async def get_news(self, user: str = "default") -> NewsResponse:
"""
Get news headlines for a user.
Fetches news from the user's volatile collection.
Args:
user: User identifier (default: 'default')
Returns:
NewsResponse with headlines
"""
logger.info(f"Fetching news data for user: {user}")
# Get raw data from Qdrant
news_records = await self.qdrant.get_by_namespace(user, "news")
headlines = []
category = None
sources = None
if news_records:
raw_data = news_records[0].get("raw_data", {})
headlines = self._parse_headlines(raw_data)
if isinstance(raw_data, dict):
category = raw_data.get("category")
sources = raw_data.get("sources")
return NewsResponse(
headlines=headlines,
category=category,
sources=sources,
updated_at=datetime.utcnow(),
user=user,
)
# Singleton instance
_news_service: Optional[NewsService] = None
def get_news_service() -> NewsService:
"""Get or create singleton news service instance."""
global _news_service
if _news_service is None:
_news_service = NewsService()
return _news_service
+6
View File
@@ -0,0 +1,6 @@
"""System stats module for host system resource monitoring."""
from src.domains.tools.system.service import SystemStatsService
from src.domains.tools.system.schemas import SystemStatsResponse
__all__ = ["SystemStatsService", "SystemStatsResponse"]
+197
View File
@@ -0,0 +1,197 @@
"""
Pydantic schemas for system stats module
"""
from pydantic import Field
from typing import Optional, List
from datetime import datetime
from src.shared.base import BaseSchema
class CpuStats(BaseSchema):
"""CPU usage statistics"""
usage_percent: float = Field(
...,
description="CPU usage percentage (0-100)",
ge=0,
le=100
)
cores: int = Field(
...,
description="Number of CPU cores"
)
load_1m: Optional[float] = Field(
default=None,
description="1-minute load average"
)
load_5m: Optional[float] = Field(
default=None,
description="5-minute load average"
)
load_15m: Optional[float] = Field(
default=None,
description="15-minute load average"
)
class MemoryStats(BaseSchema):
"""Memory usage statistics"""
usage_percent: float = Field(
...,
description="Memory usage percentage (0-100)",
ge=0,
le=100
)
total_bytes: int = Field(
...,
description="Total memory in bytes"
)
used_bytes: int = Field(
...,
description="Used memory in bytes"
)
available_bytes: int = Field(
...,
description="Available memory in bytes"
)
class DiskStats(BaseSchema):
"""Disk usage statistics for a single mount point"""
mount_point: str = Field(
...,
description="Mount point path"
)
device: str = Field(
...,
description="Device name (e.g., /dev/sda1)"
)
fstype: str = Field(
...,
description="Filesystem type (e.g., ext4, xfs)"
)
usage_percent: float = Field(
...,
description="Disk usage percentage (0-100)",
ge=0,
le=100
)
total_bytes: int = Field(
...,
description="Total disk space in bytes"
)
used_bytes: int = Field(
...,
description="Used disk space in bytes"
)
free_bytes: int = Field(
...,
description="Free disk space in bytes"
)
class NetworkStats(BaseSchema):
"""Network I/O statistics"""
bytes_sent: int = Field(
...,
description="Total bytes sent"
)
bytes_recv: int = Field(
...,
description="Total bytes received"
)
bytes_total: int = Field(
...,
description="Total bytes (sent + received)"
)
class GpuStats(BaseSchema):
"""GPU/VRAM statistics (if available)"""
available: bool = Field(
...,
description="Whether GPU stats are available"
)
name: Optional[str] = Field(
default=None,
description="GPU name"
)
usage_percent: Optional[float] = Field(
default=None,
description="VRAM usage percentage (0-100)"
)
total_bytes: Optional[int] = Field(
default=None,
description="Total VRAM in bytes"
)
used_bytes: Optional[int] = Field(
default=None,
description="Used VRAM in bytes"
)
free_bytes: Optional[int] = Field(
default=None,
description="Free VRAM in bytes"
)
class SystemStatsResponse(BaseSchema):
"""Response model for system stats"""
cpu: CpuStats = Field(
...,
description="CPU statistics"
)
memory: MemoryStats = Field(
...,
description="Memory statistics"
)
disks: List[DiskStats] = Field(
...,
description="Disk statistics for all mounted filesystems"
)
network: NetworkStats = Field(
...,
description="Network I/O statistics"
)
gpu: GpuStats = Field(
...,
description="GPU/VRAM statistics"
)
hostname: str = Field(
...,
description="System hostname"
)
queried_at: datetime = Field(
...,
description="UTC timestamp when stats were collected"
)
+190
View File
@@ -0,0 +1,190 @@
"""
System stats service for collecting host system metrics
"""
import subprocess
import socket
from datetime import datetime, timezone
import psutil
from src.shared.logging import get_logger
from typing import List
from src.domains.tools.system.schemas import (
SystemStatsResponse,
CpuStats,
MemoryStats,
DiskStats,
NetworkStats,
GpuStats,
)
# Filesystem types to exclude (virtual/system filesystems)
EXCLUDED_FSTYPES = {
"tmpfs", "devtmpfs", "devfs", "squashfs", "overlay",
"aufs", "proc", "sysfs", "cgroup", "cgroup2",
"debugfs", "tracefs", "securityfs", "pstore",
"hugetlbfs", "mqueue", "binfmt_misc", "autofs",
"fuse.lxcfs", "nsfs", "efivarfs",
}
logger = get_logger(__name__)
class SystemStatsService:
"""Service for collecting host system statistics"""
async def get_stats(self) -> SystemStatsResponse:
"""
Collect current system statistics.
Returns:
SystemStatsResponse with CPU, memory, disks, network, and GPU stats
"""
cpu = self._get_cpu_stats()
memory = self._get_memory_stats()
disks = self._get_all_disk_stats()
network = self._get_network_stats()
gpu = self._get_gpu_stats()
return SystemStatsResponse(
cpu=cpu,
memory=memory,
disks=disks,
network=network,
gpu=gpu,
hostname=socket.gethostname(),
queried_at=datetime.now(timezone.utc),
)
def _get_cpu_stats(self) -> CpuStats:
"""Get CPU usage statistics"""
# Get CPU percentage (blocking call with interval for accuracy)
cpu_percent = psutil.cpu_percent(interval=0.1)
cpu_count = psutil.cpu_count()
# Get load averages (Unix only)
try:
load_1, load_5, load_15 = psutil.getloadavg()
except (AttributeError, OSError):
load_1 = load_5 = load_15 = None
return CpuStats(
usage_percent=cpu_percent,
cores=cpu_count or 1,
load_1m=load_1,
load_5m=load_5,
load_15m=load_15,
)
def _get_memory_stats(self) -> MemoryStats:
"""Get memory usage statistics"""
mem = psutil.virtual_memory()
return MemoryStats(
usage_percent=mem.percent,
total_bytes=mem.total,
used_bytes=mem.used,
available_bytes=mem.available,
)
def _get_all_disk_stats(self) -> List[DiskStats]:
"""Get disk usage statistics for all mounted real filesystems"""
disks = []
seen_devices = set()
for partition in psutil.disk_partitions(all=False):
# Skip excluded filesystem types
if partition.fstype.lower() in EXCLUDED_FSTYPES:
continue
# Skip duplicate devices (same device mounted multiple times)
if partition.device in seen_devices:
continue
seen_devices.add(partition.device)
# Skip Docker/container overlays
if partition.mountpoint.startswith("/var/lib/docker"):
continue
try:
usage = psutil.disk_usage(partition.mountpoint)
disks.append(DiskStats(
mount_point=partition.mountpoint,
device=partition.device,
fstype=partition.fstype,
usage_percent=usage.percent,
total_bytes=usage.total,
used_bytes=usage.used,
free_bytes=usage.free,
))
except (PermissionError, OSError) as e:
logger.debug(f"Skipping {partition.mountpoint}: {e}")
continue
# Sort by mount point for consistent ordering
disks.sort(key=lambda d: d.mount_point)
return disks
def _get_network_stats(self) -> NetworkStats:
"""Get network I/O statistics"""
net_io = psutil.net_io_counters()
return NetworkStats(
bytes_sent=net_io.bytes_sent,
bytes_recv=net_io.bytes_recv,
bytes_total=net_io.bytes_sent + net_io.bytes_recv,
)
def _get_gpu_stats(self) -> GpuStats:
"""Get GPU/VRAM statistics using nvidia-smi"""
try:
# Query nvidia-smi for GPU memory info
result = subprocess.run(
[
"nvidia-smi",
"--query-gpu=name,memory.total,memory.used,memory.free",
"--format=csv,noheader,nounits",
],
capture_output=True,
text=True,
timeout=5,
)
if result.returncode != 0:
logger.debug("nvidia-smi not available or failed")
return GpuStats(available=False)
# Parse output: "NVIDIA GeForce RTX 3080, 10240, 2048, 8192"
line = result.stdout.strip().split("\n")[0] # First GPU
parts = [p.strip() for p in line.split(",")]
if len(parts) >= 4:
name = parts[0]
total_mb = int(parts[1])
used_mb = int(parts[2])
free_mb = int(parts[3])
total_bytes = total_mb * 1024 * 1024
used_bytes = used_mb * 1024 * 1024
free_bytes = free_mb * 1024 * 1024
usage_percent = (used_mb / total_mb * 100) if total_mb > 0 else 0
return GpuStats(
available=True,
name=name,
usage_percent=round(usage_percent, 1),
total_bytes=total_bytes,
used_bytes=used_bytes,
free_bytes=free_bytes,
)
except FileNotFoundError:
logger.debug("nvidia-smi not found - no NVIDIA GPU available")
except subprocess.TimeoutExpired:
logger.warning("nvidia-smi timed out")
except Exception as e:
logger.warning(f"Failed to get GPU stats: {e}")
return GpuStats(available=False)
+1 -12
View File
@@ -10,7 +10,6 @@ from src.shared.config import get_settings
from src.shared.logging import setup_logging, get_logger from src.shared.logging import setup_logging, get_logger
from src.shared.database import get_database from src.shared.database import get_database
from src.shared.security import initialize_oidc from src.shared.security import initialize_oidc
from src.models.ollama_client import get_ollama_client, close_ollama_client
# Import domain controllers # Import domain controllers
from src.domains.health import health_controller from src.domains.health import health_controller
@@ -42,17 +41,8 @@ async def lifespan(app: FastAPI):
logger.info(f"Starting {settings.app_name} v{settings.app_version}") logger.info(f"Starting {settings.app_name} v{settings.app_version}")
logger.info(f"Debug mode: {settings.debug}") logger.info(f"Debug mode: {settings.debug}")
logger.info(f"Log level: {settings.log_level}") logger.info(f"Log level: {settings.log_level}")
logger.info(f"Ollama URL: {settings.ollama_base_url}")
logger.info("=" * 60) logger.info("=" * 60)
# Check Ollama connectivity
ollama_client = get_ollama_client()
ollama_healthy = await ollama_client.health_check()
if ollama_healthy:
logger.info("Ollama connection successful")
else:
logger.warning("Ollama connection failed - AI features may not work")
# Check database connectivity # Check database connectivity
database = get_database() database = get_database()
db_healthy = await database.health_check() db_healthy = await database.health_check()
@@ -68,7 +58,6 @@ async def lifespan(app: FastAPI):
# Shutdown # Shutdown
logger.info("Shutting down application") logger.info("Shutting down application")
await close_ollama_client()
await database.close() await database.close()
@@ -94,7 +83,7 @@ See `/docs` for the full API reference.
lifespan=lifespan, lifespan=lifespan,
debug=settings.debug, debug=settings.debug,
swagger_ui_init_oauth={ swagger_ui_init_oauth={
"clientId": settings.oidc_audience, "clientId": settings.oidc_audiences[0] if settings.oidc_audiences else "core-api",
"usePkceWithAuthorizationCodeGrant": True, "usePkceWithAuthorizationCodeGrant": True,
} if settings.oidc_enabled else None } if settings.oidc_enabled else None
) )
-128
View File
@@ -1,128 +0,0 @@
"""
Embedding model client for text vectorization
Uses sentence-transformers for generating embeddings.
"""
import logging
from typing import List, Optional
from sentence_transformers import SentenceTransformer
from src.config import get_settings
logger = logging.getLogger(__name__)
settings = get_settings()
class EmbeddingClient:
"""Client for generating text embeddings"""
def __init__(self, model_name: Optional[str] = None):
"""
Initialize embedding client
Args:
model_name: Optional model name, defaults to config
"""
self.model_name = model_name or settings.embedding_model
self.dimension = settings.embedding_dimension
self._model: Optional[SentenceTransformer] = None
logger.info(f"Initializing EmbeddingClient with model: {self.model_name}")
def _load_model(self) -> SentenceTransformer:
"""
Lazy load the embedding model
Returns:
Loaded SentenceTransformer model
"""
if self._model is None:
logger.info(f"Loading embedding model: {self.model_name}")
self._model = SentenceTransformer(self.model_name)
logger.info(f"Model loaded successfully. Embedding dimension: {self.dimension}")
return self._model
def embed_text(self, text: str) -> List[float]:
"""
Generate embedding for a single text
Args:
text: Input text to embed
Returns:
List of floats representing the embedding vector
"""
model = self._load_model()
embedding = model.encode(text, convert_to_numpy=True)
return embedding.tolist()
def embed_batch(self, texts: List[str]) -> List[List[float]]:
"""
Generate embeddings for multiple texts
Args:
texts: List of input texts
Returns:
List of embedding vectors
"""
model = self._load_model()
embeddings = model.encode(
texts,
batch_size=settings.embedding_batch_size,
convert_to_numpy=True,
show_progress_bar=False
)
return embeddings.tolist()
def get_dimension(self) -> int:
"""
Get embedding dimension
Returns:
Embedding vector dimension
"""
return self.dimension
# Global instance
_embedding_client: Optional[EmbeddingClient] = None
def get_embedding_client() -> EmbeddingClient:
"""
Get or create global embedding client instance
Returns:
EmbeddingClient instance
"""
global _embedding_client
if _embedding_client is None:
_embedding_client = EmbeddingClient()
return _embedding_client
async def embed_text_async(text: str) -> List[float]:
"""
Async wrapper for embedding text
Args:
text: Input text
Returns:
Embedding vector
"""
client = get_embedding_client()
return client.embed_text(text)
async def embed_batch_async(texts: List[str]) -> List[List[float]]:
"""
Async wrapper for batch embedding
Args:
texts: List of input texts
Returns:
List of embedding vectors
"""
client = get_embedding_client()
return client.embed_batch(texts)
-136
View File
@@ -1,136 +0,0 @@
"""
Ollama-based embedding client for text vectorization
Uses Ollama's embedding API instead of local sentence-transformers.
This eliminates the need for PyTorch and heavy ML dependencies.
"""
import logging
import httpx
from typing import List, Optional
from src.config import get_settings
logger = logging.getLogger(__name__)
settings = get_settings()
class OllamaEmbeddingClient:
"""Client for generating text embeddings using Ollama"""
def __init__(
self,
model_name: Optional[str] = None,
base_url: Optional[str] = None,
timeout: int = 30
):
"""
Initialize Ollama embedding client
Args:
model_name: Embedding model name (default: nomic-embed-text)
base_url: Ollama base URL (default from settings)
timeout: Request timeout in seconds
"""
self.model_name = model_name or settings.embedding_model
self.base_url = (base_url or settings.ollama_base_url).rstrip("/")
self.timeout = timeout
self.dimension = settings.embedding_dimension
logger.info(f"Initializing OllamaEmbeddingClient with model: {self.model_name}")
logger.info(f"Ollama URL: {self.base_url}")
async def embed_text(self, text: str) -> List[float]:
"""
Generate embedding for a single text using Ollama
Args:
text: Input text to embed
Returns:
List of floats representing the embedding vector
"""
try:
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.post(
f"{self.base_url}/api/embeddings",
json={
"model": self.model_name,
"prompt": text
}
)
response.raise_for_status()
result = response.json()
return result["embedding"]
except Exception as e:
logger.error(f"Error generating embedding via Ollama: {e}")
raise
async def embed_batch(self, texts: List[str]) -> List[List[float]]:
"""
Generate embeddings for multiple texts
Args:
texts: List of input texts
Returns:
List of embedding vectors
"""
embeddings = []
for text in texts:
embedding = await self.embed_text(text)
embeddings.append(embedding)
return embeddings
def get_dimension(self) -> int:
"""
Get embedding dimension
Returns:
Embedding vector dimension
"""
return self.dimension
# Global instance
_embedding_client: Optional[OllamaEmbeddingClient] = None
def get_embedding_client() -> OllamaEmbeddingClient:
"""
Get or create global Ollama embedding client instance
Returns:
OllamaEmbeddingClient instance
"""
global _embedding_client
if _embedding_client is None:
_embedding_client = OllamaEmbeddingClient()
return _embedding_client
async def embed_text_async(text: str) -> List[float]:
"""
Async wrapper for embedding text
Args:
text: Input text
Returns:
Embedding vector
"""
client = get_embedding_client()
return await client.embed_text(text)
async def embed_batch_async(texts: List[str]) -> List[List[float]]:
"""
Async wrapper for batch embedding
Args:
texts: List of input texts
Returns:
List of embedding vectors
"""
client = get_embedding_client()
return await client.embed_batch(texts)
-223
View File
@@ -1,223 +0,0 @@
"""
Ollama client for model inference.
Handles both streaming and non-streaming requests.
"""
import httpx
import json
import logging
from typing import AsyncIterator, Dict, Any, Optional
from src.config import get_settings
logger = logging.getLogger(__name__)
settings = get_settings()
class OllamaClient:
"""Client for interacting with Ollama API."""
def __init__(self):
self.base_url = settings.ollama_base_url
self.timeout = settings.ollama_timeout
self.client = httpx.AsyncClient(timeout=self.timeout)
logger.info(f"Initialized Ollama client: {self.base_url}")
async def close(self):
"""Close the HTTP client."""
await self.client.aclose()
def resolve_model(self, model_name: str) -> str:
"""
Resolve model alias to actual Ollama model.
Args:
model_name: Requested model name (e.g., "gpt-3.5-turbo")
Returns:
Actual Ollama model name (e.g., "gemma:7b")
"""
resolved = settings.model_aliases.get(model_name, model_name)
if resolved != model_name:
logger.info(f"Model resolution: {model_name}{resolved}")
return resolved
async def generate_non_streaming(
self,
model: str,
prompt: str,
temperature: float = 0.7,
max_tokens: Optional[int] = None
) -> Dict[str, Any]:
"""
Generate non-streaming response from Ollama using chat endpoint.
Args:
model: Model name
prompt: User prompt
temperature: Sampling temperature
max_tokens: Maximum tokens to generate
Returns:
Dict with 'response' and 'tokens' keys
"""
actual_model = self.resolve_model(model)
payload = {
"model": actual_model,
"messages": [
{"role": "user", "content": prompt}
],
"stream": False,
"options": {
"temperature": temperature,
}
}
if max_tokens:
payload["options"]["num_predict"] = max_tokens
logger.debug(f"Ollama request to {actual_model}")
try:
response = await self.client.post(
f"{self.base_url}/api/chat",
json=payload
)
response.raise_for_status()
result = response.json()
return {
"response": result.get("message", {}).get("content", ""),
"tokens": {
"prompt": result.get("prompt_eval_count", 0),
"completion": result.get("eval_count", 0),
"total": result.get("prompt_eval_count", 0) + result.get("eval_count", 0)
}
}
except httpx.HTTPError as e:
logger.error(f"Ollama request failed: {e}")
raise
async def generate_streaming(
self,
model: str,
prompt: str,
temperature: float = 0.7,
max_tokens: Optional[int] = None
) -> AsyncIterator[str]:
"""
Generate streaming response from Ollama using chat endpoint.
Args:
model: Model name
prompt: User prompt
temperature: Sampling temperature
max_tokens: Maximum tokens to generate
Yields:
Token strings
"""
actual_model = self.resolve_model(model)
payload = {
"model": actual_model,
"messages": [
{"role": "user", "content": prompt}
],
"stream": True,
"options": {
"temperature": temperature,
}
}
if max_tokens:
payload["options"]["num_predict"] = max_tokens
logger.debug(f"Ollama streaming request to {actual_model}")
try:
async with self.client.stream(
"POST",
f"{self.base_url}/api/chat",
json=payload
) as response:
response.raise_for_status()
async for line in response.aiter_lines():
if not line:
continue
try:
chunk = json.loads(line)
if "message" in chunk:
content = chunk["message"].get("content", "")
if content:
yield content
# Check if done
if chunk.get("done", False):
break
except json.JSONDecodeError:
logger.warning(f"Failed to parse JSON: {line}")
continue
except httpx.HTTPError as e:
logger.error(f"Ollama streaming request failed: {e}")
raise
async def health_check(self) -> bool:
"""
Check if Ollama is healthy.
Returns:
True if healthy, False otherwise
"""
try:
response = await self.client.get(
f"{self.base_url}/api/tags",
timeout=5.0
)
return response.status_code == 200
except Exception as e:
logger.error(f"Ollama health check failed: {e}")
return False
async def list_models(self) -> Dict[str, Any]:
"""
List all available models in Ollama.
Returns:
Dict with 'models' key containing list of model info
"""
try:
response = await self.client.get(
f"{self.base_url}/api/tags",
timeout=5.0
)
response.raise_for_status()
return response.json()
except Exception as e:
logger.error(f"Failed to list Ollama models: {e}")
raise
# Global client instance
_ollama_client: Optional[OllamaClient] = None
def get_ollama_client() -> OllamaClient:
"""Get or create the global Ollama client instance."""
global _ollama_client
if _ollama_client is None:
_ollama_client = OllamaClient()
return _ollama_client
async def close_ollama_client():
"""Close the global Ollama client."""
global _ollama_client
if _ollama_client is not None:
await _ollama_client.close()
_ollama_client = None
-32
View File
@@ -1,32 +0,0 @@
"""
Security initialization module
Handles OIDC configuration and authentication setup
"""
from src.config import Settings
from src.auth.oidc import oidc_config
from src.logging_config import get_logger
logger = get_logger(__name__)
def initialize_oidc(settings: Settings) -> None:
"""
Initialize OIDC authentication configuration
Configures the global oidc_config instance with settings from environment.
If OIDC is enabled, logs the issuer URL for verification.
Args:
settings: Application settings containing OIDC configuration
"""
oidc_config.configure(
enabled=settings.oidc_enabled,
issuer=settings.oidc_issuer,
audience=settings.oidc_audience
)
if settings.oidc_enabled:
logger.info(f"✓ OIDC authentication enabled (issuer: {settings.oidc_issuer})")
else:
logger.info("○ OIDC authentication disabled - API is publicly accessible")
+3
View File
@@ -7,6 +7,7 @@ from src.shared.clients.portainer_client import PortainerClient, get_portainer_c
from src.shared.clients.npm_client import NPMClient, get_npm_client from src.shared.clients.npm_client import NPMClient, get_npm_client
from src.shared.clients.homeassistant_client import HomeAssistantClient, get_homeassistant_client from src.shared.clients.homeassistant_client import HomeAssistantClient, get_homeassistant_client
from src.shared.clients.authentik_client import AuthentikClient, get_authentik_client from src.shared.clients.authentik_client import AuthentikClient, get_authentik_client
from src.shared.clients.qdrant_client import QdrantReadClient, get_qdrant_client
__all__ = [ __all__ = [
"PortainerClient", "PortainerClient",
@@ -17,4 +18,6 @@ __all__ = [
"get_homeassistant_client", "get_homeassistant_client",
"AuthentikClient", "AuthentikClient",
"get_authentik_client", "get_authentik_client",
"QdrantReadClient",
"get_qdrant_client",
] ]
+249
View File
@@ -0,0 +1,249 @@
"""
Qdrant Vector Database Client (Read-Only)
Provides read-only access to Qdrant collections for querying volatile data.
Used to fetch weather, forecast, and sun times from the volatile_{user} collection.
"""
import time
from typing import List, Dict, Any, Optional
from qdrant_client import QdrantClient
from qdrant_client.models import Filter, FieldCondition, MatchValue, Range
from src.shared.logging import get_logger
from src.shared.config import get_settings
logger = get_logger(__name__)
settings = get_settings()
class QdrantReadClient:
"""
Read-only Qdrant client for accessing volatile data.
Connects to Qdrant and provides methods to query collections
with filtering by namespace and TTL expiry.
"""
VOLATILE_COLLECTION_PREFIX = "volatile_"
def __init__(
self,
host: Optional[str] = None,
port: Optional[int] = None,
):
"""
Initialize Qdrant read client.
Args:
host: Qdrant server host (default from settings)
port: Qdrant server port (default from settings)
"""
self.host = host or settings.qdrant_host
self.port = port or settings.qdrant_port
self._client: Optional[QdrantClient] = None
logger.info(f"Initialized QdrantReadClient: {self.host}:{self.port}")
@property
def client(self) -> QdrantClient:
"""Lazy-load Qdrant client connection."""
if self._client is None:
self._client = QdrantClient(
host=self.host,
port=self.port,
)
return self._client
def _get_volatile_collection(self, user: str) -> str:
"""Get volatile collection name for user."""
return f"{self.VOLATILE_COLLECTION_PREFIX}{user}"
def _current_timestamp_ms(self) -> int:
"""Get current timestamp in milliseconds."""
return int(time.time() * 1000)
async def collection_exists(self, collection_name: str) -> bool:
"""
Check if a collection exists.
Args:
collection_name: Name of collection to check
Returns:
True if collection exists
"""
try:
collections = self.client.get_collections()
existing = [c.name for c in collections.collections]
return collection_name in existing
except Exception as e:
logger.error(f"Error checking collection existence: {e}")
return False
async def get_by_namespace(
self,
user: str,
namespace: str,
include_expired: bool = False
) -> List[Dict[str, Any]]:
"""
Get all records for a specific namespace from user's volatile collection.
Args:
user: User identifier (e.g., 'jpmschweitzer' or 'default')
namespace: Namespace to filter (e.g., 'weather', 'forecast', 'sun')
include_expired: Whether to include expired records (default False)
Returns:
List of records with payload data
"""
collection_name = self._get_volatile_collection(user)
if not await self.collection_exists(collection_name):
logger.debug(f"Collection {collection_name} does not exist")
return []
# Build filter conditions
conditions = [
FieldCondition(
key="namespace",
match=MatchValue(value=namespace)
)
]
# Add TTL expiry filter unless including expired
if not include_expired:
now_ms = self._current_timestamp_ms()
conditions.append(
FieldCondition(
key="ttl_expiry",
range=Range(gt=now_ms)
)
)
query_filter = Filter(must=conditions)
try:
# Scroll through matching records
points, _ = self.client.scroll(
collection_name=collection_name,
scroll_filter=query_filter,
limit=100,
with_payload=True,
with_vectors=False
)
results = []
for point in points:
payload = dict(point.payload) if point.payload else {}
results.append({
"id": str(point.id),
"namespace": payload.get("namespace"),
"key": payload.get("key"),
"raw_data": payload.get("raw_data", {}),
"source": payload.get("source"),
"ttl_expiry": payload.get("ttl_expiry"),
"updated_at": payload.get("updated_at"),
})
logger.debug(
f"Found {len(results)} records in {collection_name}/{namespace}"
)
return results
except Exception as e:
logger.error(f"Error fetching from {collection_name}/{namespace}: {e}")
return []
async def get_environment_data(
self,
user: str
) -> Dict[str, Any]:
"""
Get all environment data (weather, forecast, sun times) for a user.
Convenience method that fetches all environment-related namespaces
in a single call.
Args:
user: User identifier
Returns:
Dict with 'weather', 'forecast', 'sun_times', 'air_quality' keys
(each may be None if no data found)
"""
result = {
"weather": None,
"forecast": None,
"sun_times": None,
"air_quality": None,
}
# Fetch weather data
weather_records = await self.get_by_namespace(user, "weather")
if weather_records:
# Get the first/most recent weather record
result["weather"] = weather_records[0].get("raw_data")
# Check if air quality is embedded in weather data
if result["weather"]:
aqi = result["weather"].get("aqi") or result["weather"].get("air_quality")
if aqi:
result["air_quality"] = aqi if isinstance(aqi, dict) else {"aqi": aqi}
# Fetch forecast data - pass raw_data to service for parsing
forecast_records = await self.get_by_namespace(user, "forecast")
if forecast_records:
result["forecast"] = forecast_records[0].get("raw_data")
# Fetch sun times data
sun_records = await self.get_by_namespace(user, "sun")
if sun_records:
result["sun_times"] = sun_records[0].get("raw_data")
# Check for separate air quality namespace if not embedded
if result["air_quality"] is None:
aq_records = await self.get_by_namespace(user, "air_quality")
if aq_records:
result["air_quality"] = aq_records[0].get("raw_data")
return result
async def health_check(self) -> Dict[str, Any]:
"""
Check Qdrant connectivity.
Returns:
Dict with connection status and info
"""
try:
collections = self.client.get_collections()
volatile_collections = [
c.name for c in collections.collections
if c.name.startswith(self.VOLATILE_COLLECTION_PREFIX)
]
return {
"status": "healthy",
"connected": True,
"host": f"{self.host}:{self.port}",
"volatile_collections": volatile_collections,
}
except Exception as e:
logger.error(f"Qdrant health check failed: {e}")
return {
"status": "unhealthy",
"connected": False,
"host": f"{self.host}:{self.port}",
"error": str(e),
}
# Singleton instance for reuse
_qdrant_client: Optional[QdrantReadClient] = None
def get_qdrant_client() -> QdrantReadClient:
"""Get or create singleton Qdrant client instance."""
global _qdrant_client
if _qdrant_client is None:
_qdrant_client = QdrantReadClient()
return _qdrant_client
+17 -48
View File
@@ -36,8 +36,15 @@ class Settings(BaseSettings):
host: str = "0.0.0.0" host: str = "0.0.0.0"
port: int = 8083 port: int = 8083
# CORS # CORS - Note: When cors_credentials is True, cannot use "*" for origins
cors_origins: list[str] = ["*"] # Set CORS_ORIGINS env var to override (comma-separated list)
cors_origins: list[str] = [
"https://home.schweitz.net",
"https://tatlock.schweitz.net",
"http://localhost:8080",
"http://localhost:3000",
"http://127.0.0.1:8080",
]
cors_credentials: bool = True cors_credentials: bool = True
cors_methods: list[str] = ["*"] cors_methods: list[str] = ["*"]
cors_headers: list[str] = ["*"] cors_headers: list[str] = ["*"]
@@ -45,31 +52,6 @@ class Settings(BaseSettings):
# Logging # Logging
log_level: str = "DEBUG" log_level: str = "DEBUG"
# Ollama Configuration (for AI orchestration)
ollama_base_url: str # Required - set OLLAMA_BASE_URL in .env
ollama_timeout: int = 300 # 5 minutes
# Model Configuration
default_model: str = "mistral-nemo-large:latest"
agent_model: str = "mistral-nemo-large:latest"
code_models: str = "mistral-nemo-large:latest"
# System Prompt Variant (for A/B testing)
system_prompt_variant: str = "v8_holistic"
# Agent Configuration
agent_fallback_enabled: bool = True
# Model Aliases (OpenAI → Local)
alias_gpt35: str = "gemma:7b"
alias_gpt4: str = "mistral:7b"
alias_gpt4_turbo: str = "mixtral:8x7b"
alias_gpt4_code: str = "codestral:latest"
# Memory Configuration
memory_tier1_max_turns: int = 10
memory_consolidation_threshold: int = 10
# Qdrant Configuration # Qdrant Configuration
qdrant_host: str = "qdrant" qdrant_host: str = "qdrant"
qdrant_port: int = 6333 qdrant_port: int = 6333
@@ -77,11 +59,6 @@ class Settings(BaseSettings):
qdrant_collection_documents: str = "core_api_documents" qdrant_collection_documents: str = "core_api_documents"
qdrant_collection_user_facts: str = "core_api_user_facts" qdrant_collection_user_facts: str = "core_api_user_facts"
# Embeddings (using Ollama)
embedding_model: str = "nomic-embed-text"
embedding_dimension: int = 768
embedding_batch_size: int = 32
# Search Configuration # Search Configuration
search_provider: str = "searxng" search_provider: str = "searxng"
searxng_url: str # Required - set SEARXNG_URL in .env searxng_url: str # Required - set SEARXNG_URL in .env
@@ -113,28 +90,20 @@ class Settings(BaseSettings):
# OIDC Authentication (Authentik) # OIDC Authentication (Authentik)
oidc_enabled: bool = False oidc_enabled: bool = False
oidc_issuer: str = "https://auth.schweitz.net/application/o/core-api/" # Accept tokens from multiple OAuth providers (each has its own issuer/JWKS)
oidc_audience: str = "core-api" oidc_issuers: list[str] = [
"https://auth.schweitz.net/application/o/core-api/",
"https://auth.schweitz.net/application/o/tatlock-ui/",
"https://auth.schweitz.net/application/o/tatlock/",
]
# Accept tokens from multiple clients
oidc_audiences: list[str] = ["core-api", "tatlock-ui", "tatlock"]
# Authentik API (for token validation and user management) # Authentik API (for token validation and user management)
authentik_url: str = "https://auth.schweitz.net" authentik_url: str = "https://auth.schweitz.net"
authentik_username: str = "" authentik_username: str = ""
authentik_password: str = "" authentik_password: str = ""
@property
def model_aliases(self) -> dict:
"""Computed property for model aliases"""
return {
"gpt-3.5-turbo": self.alias_gpt35,
"gpt-4": self.alias_gpt4,
"gpt-4-turbo": self.alias_gpt4_turbo,
"gpt-4-code": self.alias_gpt4_code,
}
def get_code_models(self) -> list[str]:
"""Parse comma-separated code models"""
return [m.strip().strip('"').strip("'") for m in self.code_models.split(",") if m.strip()]
class Config: class Config:
env_file = ".env" env_file = ".env"
case_sensitive = False case_sensitive = False
+13 -5
View File
@@ -17,15 +17,23 @@ def initialize_oidc(settings: Settings) -> None:
settings: Application settings containing OIDC configuration settings: Application settings containing OIDC configuration
""" """
# Import here to avoid circular imports # Import here to avoid circular imports
from src.auth.oidc import oidc_config # Configure BOTH oidc modules (src.auth and src.domains.auth)
from src.auth.oidc import oidc_config as auth_oidc_config
from src.domains.auth.oidc import oidc_config as domains_oidc_config
oidc_config.configure( auth_oidc_config.configure(
enabled=settings.oidc_enabled, enabled=settings.oidc_enabled,
issuer=settings.oidc_issuer, issuers=settings.oidc_issuers,
audience=settings.oidc_audience audiences=settings.oidc_audiences
)
domains_oidc_config.configure(
enabled=settings.oidc_enabled,
issuers=settings.oidc_issuers,
audiences=settings.oidc_audiences
) )
if settings.oidc_enabled: if settings.oidc_enabled:
logger.info(f"OIDC authentication enabled (issuer: {settings.oidc_issuer})") logger.info(f"OIDC authentication enabled (issuers: {settings.oidc_issuers})")
else: else:
logger.info("OIDC authentication disabled - API is publicly accessible") logger.info("OIDC authentication disabled - API is publicly accessible")
File diff suppressed because it is too large Load Diff
+627
View File
@@ -0,0 +1,627 @@
"""Tests for authentication service."""
import uuid
from datetime import datetime, timezone
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from sqlalchemy.ext.asyncio import AsyncSession
from src.domains.auth.models import User, Role, Group, UserPreferences
from src.domains.auth.schemas import TokenInfoSchema, RoleSchema
from src.domains.auth.service import AuthService, get_auth_service
# =============================================================================
# Fixtures
# =============================================================================
@pytest.fixture
def mock_session():
"""Create a mock async database session."""
session = AsyncMock(spec=AsyncSession)
session.execute = AsyncMock()
session.commit = AsyncMock()
session.flush = AsyncMock()
session.refresh = AsyncMock()
session.add = MagicMock()
return session
@pytest.fixture
def auth_service(mock_session):
"""Create an AuthService instance with mock session."""
return AuthService(mock_session)
@pytest.fixture
def sample_user():
"""Create a sample user for testing."""
user = User(
id=uuid.uuid4(),
authentik_id=uuid.uuid4(),
email="test@example.com",
name="Test User",
avatar_url="https://example.com/avatar.jpg",
created_at=datetime.now(timezone.utc),
last_login=datetime.now(timezone.utc),
)
user.roles = []
user.preferences = None
return user
@pytest.fixture
def sample_role():
"""Create a sample role for testing."""
return Role(
id=uuid.uuid4(),
name="control-room.general:admin",
domain="control-room",
category="general",
action="admin",
)
@pytest.fixture
def sample_group(sample_role):
"""Create a sample group for testing."""
group = Group(
id=uuid.uuid4(),
authentik_id=uuid.uuid4(),
name="Administrators",
is_superuser=True,
parent_name=None,
member_count=5,
created_at=datetime.now(timezone.utc),
synced_at=datetime.now(timezone.utc),
)
group.roles = [sample_role]
return group
@pytest.fixture
def sample_token_info():
"""Create sample token info from Authentik."""
return TokenInfoSchema(
sub=str(uuid.uuid4()),
email="test@example.com",
name="Test User",
preferred_username="testuser",
groups=["Administrators", "Developers"],
picture="https://example.com/avatar.jpg",
)
@pytest.fixture
def sample_preferences():
"""Create sample user preferences."""
return UserPreferences(
user_id=uuid.uuid4(),
theme="dark",
default_room="control-room",
preferences_json={"notifications": True},
)
# =============================================================================
# AuthService Initialization Tests
# =============================================================================
class TestAuthServiceInit:
"""Test AuthService initialization."""
def test_init_with_session(self, mock_session):
"""AuthService should initialize with session."""
service = AuthService(mock_session)
assert service.session is mock_session
def test_init_sets_userinfo_url(self, mock_session):
"""AuthService should set userinfo URL from settings."""
service = AuthService(mock_session)
assert "userinfo" in service.userinfo_url
def test_get_auth_service_factory(self, mock_session):
"""get_auth_service should return AuthService instance."""
service = get_auth_service(mock_session)
assert isinstance(service, AuthService)
assert service.session is mock_session
# =============================================================================
# Token Validation Tests
# =============================================================================
class TestValidateToken:
"""Test token validation via Authentik userinfo endpoint."""
@pytest.mark.asyncio
async def test_validate_token_success(self, auth_service):
"""validate_token should return token info on success."""
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {
"sub": str(uuid.uuid4()),
"email": "test@example.com",
"name": "Test User",
"preferred_username": "testuser",
"groups": ["Administrators"],
"picture": "https://example.com/avatar.jpg",
}
mock_response.raise_for_status = MagicMock()
with patch("src.domains.auth.service.httpx.AsyncClient") as mock_client:
mock_client.return_value.__aenter__.return_value.get = AsyncMock(
return_value=mock_response
)
result = await auth_service.validate_token("valid_token")
assert result.email == "test@example.com"
assert result.name == "Test User"
assert "Administrators" in result.groups
@pytest.mark.asyncio
async def test_validate_token_invalid(self, auth_service):
"""validate_token should raise ValueError for invalid token."""
mock_response = MagicMock()
mock_response.status_code = 401
with patch("src.domains.auth.service.httpx.AsyncClient") as mock_client:
mock_client.return_value.__aenter__.return_value.get = AsyncMock(
return_value=mock_response
)
with pytest.raises(ValueError, match="Invalid or expired token"):
await auth_service.validate_token("invalid_token")
@pytest.mark.asyncio
async def test_validate_token_service_unavailable(self, auth_service):
"""validate_token should raise ValueError when service unavailable."""
import httpx
with patch("src.domains.auth.service.httpx.AsyncClient") as mock_client:
mock_client.return_value.__aenter__.return_value.get = AsyncMock(
side_effect=httpx.RequestError("Connection failed")
)
with pytest.raises(ValueError, match="Authentication service unavailable"):
await auth_service.validate_token("token")
# =============================================================================
# User Sync Tests
# =============================================================================
class TestSyncUser:
"""Test user synchronization from OIDC token."""
@pytest.mark.asyncio
async def test_sync_user_creates_new_user(self, auth_service, sample_token_info, mock_session):
"""sync_user should create new user when not found."""
# Mock no existing user found
mock_result = MagicMock()
mock_result.scalar_one_or_none.return_value = None
mock_session.execute.return_value = mock_result
user, is_new = await auth_service.sync_user(sample_token_info)
assert is_new is True
assert mock_session.add.call_count == 2 # User and Preferences
@pytest.mark.asyncio
async def test_sync_user_updates_existing_user(
self, auth_service, sample_token_info, sample_user, mock_session
):
"""sync_user should update existing user when found."""
# Mock existing user found
mock_result = MagicMock()
mock_result.scalar_one_or_none.return_value = sample_user
mock_session.execute.return_value = mock_result
# Update token info with matching authentik_id
sample_token_info.sub = str(sample_user.authentik_id)
user, is_new = await auth_service.sync_user(sample_token_info)
assert is_new is False
assert user.email == sample_token_info.email
assert user.name == sample_token_info.name
@pytest.mark.asyncio
async def test_sync_user_updates_last_login(
self, auth_service, sample_token_info, sample_user, mock_session
):
"""sync_user should update last_login timestamp."""
old_login = sample_user.last_login
mock_result = MagicMock()
mock_result.scalar_one_or_none.return_value = sample_user
mock_session.execute.return_value = mock_result
sample_token_info.sub = str(sample_user.authentik_id)
user, _ = await auth_service.sync_user(sample_token_info)
assert user.last_login is not None
# last_login should be updated (or same if happened in same second)
assert user.last_login >= old_login or user.last_login is not None
# =============================================================================
# Role Sync Tests
# =============================================================================
class TestSyncRoles:
"""Test role synchronization from Authentik groups via group_roles."""
@pytest.mark.asyncio
async def test_sync_roles_from_groups(
self, auth_service, sample_user, sample_group, mock_session
):
"""sync_roles should get roles from matching groups."""
# Mock finding groups with roles
mock_result = MagicMock()
mock_result.scalars.return_value.all.return_value = [sample_group]
mock_session.execute.return_value = mock_result
roles = await auth_service.sync_roles(sample_user, ["Administrators"])
assert len(roles) == 1
assert roles[0].name == "control-room.general:admin"
assert sample_user.roles == roles
@pytest.mark.asyncio
async def test_sync_roles_no_matching_groups(
self, auth_service, sample_user, mock_session
):
"""sync_roles should return empty list when no groups match."""
mock_result = MagicMock()
mock_result.scalars.return_value.all.return_value = []
mock_session.execute.return_value = mock_result
roles = await auth_service.sync_roles(sample_user, ["NonExistentGroup"])
assert len(roles) == 0
assert sample_user.roles == []
@pytest.mark.asyncio
async def test_sync_roles_deduplicates_roles(
self, auth_service, sample_user, sample_role, mock_session
):
"""sync_roles should deduplicate roles from multiple groups."""
# Create two groups with the same role
group1 = Group(
id=uuid.uuid4(),
authentik_id=uuid.uuid4(),
name="Group1",
is_superuser=False,
member_count=1,
created_at=datetime.now(timezone.utc),
synced_at=datetime.now(timezone.utc),
)
group1.roles = [sample_role]
group2 = Group(
id=uuid.uuid4(),
authentik_id=uuid.uuid4(),
name="Group2",
is_superuser=False,
member_count=1,
created_at=datetime.now(timezone.utc),
synced_at=datetime.now(timezone.utc),
)
group2.roles = [sample_role] # Same role
mock_result = MagicMock()
mock_result.scalars.return_value.all.return_value = [group1, group2]
mock_session.execute.return_value = mock_result
roles = await auth_service.sync_roles(sample_user, ["Group1", "Group2"])
# Should only have one role despite appearing in two groups
assert len(roles) == 1
# =============================================================================
# Schema Conversion Tests
# =============================================================================
class TestSchemaConversions:
"""Test model to schema conversions."""
def test_user_to_schema(self, auth_service, sample_user):
"""user_to_schema should convert User model to UserSchema."""
schema = auth_service.user_to_schema(sample_user)
assert schema.id == sample_user.id
assert schema.authentik_id == sample_user.authentik_id
assert schema.email == sample_user.email
assert schema.name == sample_user.name
assert schema.avatar_url == sample_user.avatar_url
def test_roles_to_schema(self, auth_service, sample_role):
"""roles_to_schema should convert Role models to RoleSchemas."""
schemas = auth_service.roles_to_schema([sample_role])
assert len(schemas) == 1
assert schemas[0].id == sample_role.id
assert schemas[0].name == sample_role.name
assert schemas[0].domain == sample_role.domain
assert schemas[0].category == sample_role.category
assert schemas[0].action == sample_role.action
def test_roles_to_schema_empty_list(self, auth_service):
"""roles_to_schema should handle empty list."""
schemas = auth_service.roles_to_schema([])
assert schemas == []
def test_preferences_to_schema(self, auth_service, sample_preferences):
"""preferences_to_schema should convert UserPreferences to schema."""
schema = auth_service.preferences_to_schema(sample_preferences)
assert schema.theme == sample_preferences.theme
assert schema.default_room == sample_preferences.default_room
assert schema.preferences_json == sample_preferences.preferences_json
def test_preferences_to_schema_none(self, auth_service):
"""preferences_to_schema should return defaults for None."""
schema = auth_service.preferences_to_schema(None)
assert schema.theme == "system"
assert schema.default_room == "front-hall"
assert schema.preferences_json == {}
# =============================================================================
# List Operations Tests
# =============================================================================
class TestListOperations:
"""Test list operations for users, groups, and roles."""
@pytest.mark.asyncio
async def test_list_users(self, auth_service, sample_user, mock_session):
"""list_users should return paginated user list."""
sample_user.roles = []
# Mock count query
count_result = MagicMock()
count_result.scalar.return_value = 1
# Mock users query
users_result = MagicMock()
users_result.scalars.return_value.all.return_value = [sample_user]
mock_session.execute.side_effect = [count_result, users_result]
items, total = await auth_service.list_users()
assert total == 1
assert len(items) == 1
assert items[0].email == sample_user.email
@pytest.mark.asyncio
async def test_list_users_with_search(self, auth_service, mock_session):
"""list_users should filter by search query."""
count_result = MagicMock()
count_result.scalar.return_value = 0
users_result = MagicMock()
users_result.scalars.return_value.all.return_value = []
mock_session.execute.side_effect = [count_result, users_result]
items, total = await auth_service.list_users(search="nonexistent")
assert total == 0
assert len(items) == 0
@pytest.mark.asyncio
async def test_list_groups(self, auth_service, sample_group, mock_session):
"""list_groups should return paginated group list with roles."""
count_result = MagicMock()
count_result.scalar.return_value = 1
groups_result = MagicMock()
groups_result.scalars.return_value.all.return_value = [sample_group]
mock_session.execute.side_effect = [count_result, groups_result]
items, total = await auth_service.list_groups()
assert total == 1
assert len(items) == 1
assert items[0].name == sample_group.name
assert len(items[0].roles) == 1 # Should include role names
@pytest.mark.asyncio
async def test_list_roles(self, auth_service, sample_role, mock_session):
"""list_roles should return all roles ordered by domain."""
mock_result = MagicMock()
mock_result.scalars.return_value.all.return_value = [sample_role]
mock_session.execute.return_value = mock_result
roles = await auth_service.list_roles()
assert len(roles) == 1
assert roles[0].name == sample_role.name
# =============================================================================
# Group-Role Management Tests
# =============================================================================
class TestGroupRoleManagement:
"""Test group-role assignment and removal."""
@pytest.mark.asyncio
async def test_get_group_by_id(self, auth_service, sample_group, mock_session):
"""get_group_by_id should return group with roles."""
mock_result = MagicMock()
mock_result.scalar_one_or_none.return_value = sample_group
mock_session.execute.return_value = mock_result
group = await auth_service.get_group_by_id(sample_group.id)
assert group is not None
assert group.id == sample_group.id
assert len(group.roles) == 1
@pytest.mark.asyncio
async def test_get_group_by_id_not_found(self, auth_service, mock_session):
"""get_group_by_id should return None when not found."""
mock_result = MagicMock()
mock_result.scalar_one_or_none.return_value = None
mock_session.execute.return_value = mock_result
group = await auth_service.get_group_by_id(uuid.uuid4())
assert group is None
@pytest.mark.asyncio
async def test_assign_role_to_group(
self, auth_service, sample_group, sample_role, mock_session
):
"""assign_role_to_group should add role to group."""
# Clear existing roles for this test
sample_group.roles = []
# Mock group lookup
group_result = MagicMock()
group_result.scalar_one_or_none.return_value = sample_group
# Mock role lookup
role_result = MagicMock()
role_result.scalar_one_or_none.return_value = sample_role
mock_session.execute.side_effect = [group_result, role_result]
group = await auth_service.assign_role_to_group(sample_group.id, sample_role.id)
assert sample_role in group.roles
mock_session.flush.assert_called()
@pytest.mark.asyncio
async def test_assign_role_to_group_already_assigned(
self, auth_service, sample_group, sample_role, mock_session
):
"""assign_role_to_group should not duplicate if already assigned."""
# Group already has this role
sample_group.roles = [sample_role]
original_count = len(sample_group.roles)
group_result = MagicMock()
group_result.scalar_one_or_none.return_value = sample_group
role_result = MagicMock()
role_result.scalar_one_or_none.return_value = sample_role
mock_session.execute.side_effect = [group_result, role_result]
group = await auth_service.assign_role_to_group(sample_group.id, sample_role.id)
assert len(group.roles) == original_count # No duplicate
@pytest.mark.asyncio
async def test_assign_role_to_group_group_not_found(self, auth_service, mock_session):
"""assign_role_to_group should raise ValueError when group not found."""
mock_result = MagicMock()
mock_result.scalar_one_or_none.return_value = None
mock_session.execute.return_value = mock_result
with pytest.raises(ValueError, match="Group not found"):
await auth_service.assign_role_to_group(uuid.uuid4(), uuid.uuid4())
@pytest.mark.asyncio
async def test_assign_role_to_group_role_not_found(
self, auth_service, sample_group, mock_session
):
"""assign_role_to_group should raise ValueError when role not found."""
sample_group.roles = []
group_result = MagicMock()
group_result.scalar_one_or_none.return_value = sample_group
role_result = MagicMock()
role_result.scalar_one_or_none.return_value = None
mock_session.execute.side_effect = [group_result, role_result]
with pytest.raises(ValueError, match="Role not found"):
await auth_service.assign_role_to_group(sample_group.id, uuid.uuid4())
@pytest.mark.asyncio
async def test_remove_role_from_group(
self, auth_service, sample_group, sample_role, mock_session
):
"""remove_role_from_group should remove role from group."""
# Group has this role
sample_group.roles = [sample_role]
group_result = MagicMock()
group_result.scalar_one_or_none.return_value = sample_group
role_result = MagicMock()
role_result.scalar_one_or_none.return_value = sample_role
mock_session.execute.side_effect = [group_result, role_result]
group = await auth_service.remove_role_from_group(sample_group.id, sample_role.id)
assert sample_role not in group.roles
mock_session.flush.assert_called()
@pytest.mark.asyncio
async def test_remove_role_from_group_not_assigned(
self, auth_service, sample_group, sample_role, mock_session
):
"""remove_role_from_group should handle role not assigned gracefully."""
# Group does not have this role
sample_group.roles = []
group_result = MagicMock()
group_result.scalar_one_or_none.return_value = sample_group
role_result = MagicMock()
role_result.scalar_one_or_none.return_value = sample_role
mock_session.execute.side_effect = [group_result, role_result]
group = await auth_service.remove_role_from_group(sample_group.id, sample_role.id)
# Should complete without error
assert len(group.roles) == 0
# =============================================================================
# Role Schema Tests
# =============================================================================
class TestRoleSchema:
"""Test RoleSchema validation."""
def test_role_schema_creation(self):
"""RoleSchema should be creatable with valid data."""
schema = RoleSchema(
id=uuid.uuid4(),
name="control-room.general:admin",
domain="control-room",
category="general",
action="admin",
)
assert schema.name == "control-room.general:admin"
assert schema.domain == "control-room"
assert schema.category == "general"
assert schema.action == "admin"
def test_role_schema_category_default(self):
"""RoleSchema should default category to 'general'."""
schema = RoleSchema(
id=uuid.uuid4(),
name="media.general:viewer",
domain="media",
action="viewer",
)
assert schema.category == "general"
+4 -8
View File
@@ -1,6 +1,6 @@
"""Tests for config module.""" """Tests for config module."""
import pytest import pytest
from src.config import ( from src.shared.config import (
__version__, __version__,
Settings, Settings,
get_settings, get_settings,
@@ -89,10 +89,6 @@ class TestGetSettings:
settings2 = get_settings() settings2 = get_settings()
assert settings1 is settings2 assert settings1 is settings2
def test_model_aliases_property(self): # Removed: test_model_aliases_property. Settings.model_aliases mapped
"""Model aliases property should return dict.""" # gpt-3.5-turbo and gpt-4 onto local models, and was deleted along with the
settings = get_settings() # Ollama integration in c1f16d4. The test outlived the feature it covered.
aliases = settings.model_aliases
assert isinstance(aliases, dict)
assert "gpt-3.5-turbo" in aliases
assert "gpt-4" in aliases
+159
View File
@@ -0,0 +1,159 @@
"""Tests for environment service and schemas."""
import pytest
from datetime import datetime
class TestEnvironmentService:
"""Test EnvironmentService methods."""
def test_aqi_to_quality_good(self):
"""AQI 0-50 should return Good."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
assert service._aqi_to_quality(0) == "Good"
assert service._aqi_to_quality(25) == "Good"
assert service._aqi_to_quality(50) == "Good"
def test_aqi_to_quality_moderate(self):
"""AQI 51-100 should return Moderate."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
assert service._aqi_to_quality(51) == "Moderate"
assert service._aqi_to_quality(75) == "Moderate"
assert service._aqi_to_quality(100) == "Moderate"
def test_aqi_to_quality_unhealthy_sensitive(self):
"""AQI 101-150 should return Unhealthy for Sensitive Groups."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
assert service._aqi_to_quality(101) == "Unhealthy for Sensitive Groups"
assert service._aqi_to_quality(150) == "Unhealthy for Sensitive Groups"
def test_aqi_to_quality_unhealthy(self):
"""AQI 151-200 should return Unhealthy."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
assert service._aqi_to_quality(151) == "Unhealthy"
assert service._aqi_to_quality(200) == "Unhealthy"
def test_aqi_to_quality_very_unhealthy(self):
"""AQI 201-300 should return Very Unhealthy."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
assert service._aqi_to_quality(201) == "Very Unhealthy"
assert service._aqi_to_quality(300) == "Very Unhealthy"
def test_aqi_to_quality_hazardous(self):
"""AQI >300 should return Hazardous."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
assert service._aqi_to_quality(301) == "Hazardous"
assert service._aqi_to_quality(500) == "Hazardous"
def test_parse_weather_with_valid_data(self):
"""Parse weather should return WeatherData for valid input."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
raw = {
"temperature": 15.5,
"conditions": "Cloudy",
"humidity": 72,
"location": "Rotterdam",
}
result = service._parse_weather(raw)
assert result is not None
assert result.temperature == 15.5
assert result.conditions == "Cloudy"
assert result.humidity == 72
def test_parse_weather_with_none(self):
"""Parse weather should return None for None input."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
result = service._parse_weather(None)
assert result is None
def test_parse_forecast_with_list(self):
"""Parse forecast should handle list format."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
raw = [
{"date": "2026-01-06", "high": 12, "low": 5, "conditions": "Cloudy"},
{"date": "2026-01-07", "high": 14, "low": 6, "conditions": "Sunny"},
]
result = service._parse_forecast(raw)
assert result is not None
assert len(result) == 2
assert result[0].date == "2026-01-06"
assert result[0].high == 12
def test_parse_forecast_with_dict(self):
"""Parse forecast should handle dict with days key."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
raw = {
"days": [
{"date": "2026-01-06", "high": 12, "low": 5, "conditions": "Cloudy"},
]
}
result = service._parse_forecast(raw)
assert result is not None
assert len(result) == 1
def test_parse_sun_times_with_strings(self):
"""Parse sun times should handle ISO datetime strings."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
raw = {
"sunrise": "2026-01-06T08:45:00",
"sunset": "2026-01-06T16:50:00",
}
result = service._parse_sun_times(raw)
assert result is not None
assert result.sunrise.hour == 8
assert result.sunrise.minute == 45
assert result.sunset.hour == 16
assert result.daylight_minutes == 485
def test_parse_air_quality_with_int(self):
"""Parse air quality should handle simple integer AQI."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
result = service._parse_air_quality(42)
assert result is not None
assert result.aqi == 42
assert result.quality == "Good"
def test_parse_air_quality_with_dict(self):
"""Parse air quality should handle dict format."""
from src.domains.tools.environment.service import EnvironmentService
service = EnvironmentService.__new__(EnvironmentService)
raw = {
"aqi": 75,
"pm25": 8.5,
"pm10": 15,
}
result = service._parse_air_quality(raw)
assert result is not None
assert result.aqi == 75
assert result.quality == "Moderate"
assert result.pm25 == 8.5
+19 -87
View File
@@ -80,24 +80,24 @@ class TestFullHealthCheck:
"""Test /health/full endpoint.""" """Test /health/full endpoint."""
@patch("src.shared.database.Database.health_check") @patch("src.shared.database.Database.health_check")
@patch("src.models.ollama_client.get_ollama_client") def test_full_health_returns_200_when_healthy(self, mock_db_health, client):
def test_full_health_returns_503_when_unhealthy(self, mock_get_ollama, mock_db_health, client): """Full health should return 200 when database is healthy."""
"""Full health should return 503 when Ollama unhealthy."""
mock_client = AsyncMock()
mock_client.health_check.return_value = False
mock_get_ollama.return_value = mock_client
mock_db_health.return_value = True mock_db_health.return_value = True
response = client.get("/health/full")
assert response.status_code == 200
@patch("src.shared.database.Database.health_check")
def test_full_health_returns_503_when_unhealthy(self, mock_db_health, client):
"""Full health should return 503 when database is unhealthy."""
mock_db_health.return_value = False
response = client.get("/health/full") response = client.get("/health/full")
assert response.status_code == 503 assert response.status_code == 503
@patch("src.shared.database.Database.health_check") @patch("src.shared.database.Database.health_check")
@patch("src.models.ollama_client.get_ollama_client") def test_full_health_returns_components_status(self, mock_db_health, client):
def test_full_health_returns_components_status(self, mock_get_ollama, mock_db_health, client):
"""Full health should return component status.""" """Full health should return component status."""
mock_client = AsyncMock()
mock_client.health_check.return_value = False
mock_get_ollama.return_value = mock_client
mock_db_health.return_value = True mock_db_health.return_value = True
response = client.get("/health/full") response = client.get("/health/full")
@@ -105,63 +105,32 @@ class TestFullHealthCheck:
assert "status" in data assert "status" in data
assert "components" in data assert "components" in data
assert "ollama" in data["components"]
assert "database" in data["components"] assert "database" in data["components"]
assert "response_time_ms" in data assert "response_time_ms" in data
@patch("src.shared.database.Database.health_check") @patch("src.shared.database.Database.health_check")
@patch("src.models.ollama_client.get_ollama_client") def test_full_health_handles_database_error(self, mock_db_health, client):
def test_full_health_handles_list_models_error(self, mock_get_ollama, mock_db_health, client): """Full health should handle database errors gracefully."""
"""Full health should handle list_models errors.""" mock_db_health.side_effect = Exception("Connection error")
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_client.list_models.side_effect = Exception("Connection error")
mock_get_ollama.return_value = mock_client
mock_db_health.return_value = True
response = client.get("/health/full") response = client.get("/health/full")
data = response.json() data = response.json()
# Should report error in component status
assert "ollama" in data["components"]
@patch("src.shared.database.Database.health_check")
@patch("src.models.ollama_client.get_ollama_client")
def test_full_health_handles_health_check_exception(self, mock_get_ollama, mock_db_health, client):
"""Full health should handle health check exceptions gracefully."""
mock_client = AsyncMock()
# Return False instead of raising exception to test unhealthy path
mock_client.health_check.return_value = False
mock_get_ollama.return_value = mock_client
mock_db_health.return_value = True
response = client.get("/health/full")
# Should return 503 for unhealthy
assert response.status_code == 503 assert response.status_code == 503
data = response.json()
assert data["status"] == "unhealthy" assert data["status"] == "unhealthy"
assert "error" in data["components"]["database"]
class TestDiagnosticsEndpoint: class TestDiagnosticsEndpoint:
"""Test /health/diagnostics endpoint.""" """Test /health/diagnostics endpoint."""
@patch("src.models.ollama_client.get_ollama_client") def test_diagnostics_returns_200(self, client):
def test_diagnostics_returns_200(self, mock_get_ollama, client):
"""Diagnostics should return 200.""" """Diagnostics should return 200."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics") response = client.get("/health/diagnostics")
assert response.status_code == 200 assert response.status_code == 200
@patch("src.models.ollama_client.get_ollama_client") def test_diagnostics_returns_service_info(self, client):
def test_diagnostics_returns_service_info(self, mock_get_ollama, client):
"""Diagnostics should return service information.""" """Diagnostics should return service information."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics") response = client.get("/health/diagnostics")
data = response.json() data = response.json()
@@ -169,54 +138,17 @@ class TestDiagnosticsEndpoint:
assert "name" in data["service"] assert "name" in data["service"]
assert "version" in data["service"] assert "version" in data["service"]
@patch("src.models.ollama_client.get_ollama_client") def test_diagnostics_returns_configuration(self, client):
def test_diagnostics_returns_components(self, mock_get_ollama, client):
"""Diagnostics should return component details."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics")
data = response.json()
assert "components" in data
assert "ollama" in data["components"]
@patch("src.models.ollama_client.get_ollama_client")
def test_diagnostics_returns_configuration(self, mock_get_ollama, client):
"""Diagnostics should return configuration info.""" """Diagnostics should return configuration info."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics") response = client.get("/health/diagnostics")
data = response.json() data = response.json()
assert "configuration" in data assert "configuration" in data
@patch("src.models.ollama_client.get_ollama_client") def test_diagnostics_returns_response_time(self, client):
def test_diagnostics_returns_response_time(self, mock_get_ollama, client):
"""Diagnostics should return response time.""" """Diagnostics should return response time."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics") response = client.get("/health/diagnostics")
data = response.json() data = response.json()
assert "response_time_ms" in data assert "response_time_ms" in data
assert isinstance(data["response_time_ms"], int) assert isinstance(data["response_time_ms"], int)
@patch("src.models.ollama_client.get_ollama_client")
def test_diagnostics_handles_ollama_error(self, mock_get_ollama, client):
"""Diagnostics should handle Ollama connection errors."""
mock_client = AsyncMock()
mock_client.health_check.side_effect = Exception("Connection refused")
mock_get_ollama.return_value = mock_client
response = client.get("/health/diagnostics")
data = response.json()
# Should still return 200 with error info
assert response.status_code == 200
assert "error" in data["components"]["ollama"]
+114 -62
View File
@@ -1,118 +1,173 @@
"""Tests for OIDC authentication module.""" """Tests for OIDC authentication module."""
import pytest import pytest
from unittest.mock import patch, MagicMock, AsyncMock from unittest.mock import patch, MagicMock
from fastapi import HTTPException from fastapi import HTTPException
from src.auth.oidc import OIDCConfig, oidc_config, get_jwks, get_current_user from src.auth.oidc import (
OIDCConfig,
oidc_config,
get_jwks_for_issuer,
get_current_user,
_jwks_cache,
)
@pytest.fixture(autouse=True)
def clear_jwks_cache():
"""
The JWKS cache is module-level state, so a fetch in one test would satisfy
the next one and hide a regression. Clearing on both sides keeps the tests
order-independent.
"""
_jwks_cache.clear()
yield
_jwks_cache.clear()
class TestOIDCConfig: class TestOIDCConfig:
"""Test OIDCConfig class.""" """Test OIDCConfig class."""
def test_init_defaults(self): def test_init_defaults(self):
"""Config should initialize with disabled state.""" """Config should initialize disabled with no issuers or audiences."""
config = OIDCConfig() config = OIDCConfig()
assert config.enabled is False assert config.enabled is False
assert config.issuer == "" assert config.issuers == []
assert config.audience == "" assert config.audiences == []
assert config.jwks_uri == ""
def test_configure_sets_values(self): def test_configure_sets_values(self):
"""configure should set all values.""" """configure should set all values."""
config = OIDCConfig() config = OIDCConfig()
config.configure( config.configure(
enabled=True, enabled=True,
issuer="https://auth.example.com", issuers=["https://auth.example.com"],
audience="core-api" audiences=["core-api"],
) )
assert config.enabled is True assert config.enabled is True
assert config.issuer == "https://auth.example.com" assert config.issuers == ["https://auth.example.com"]
assert config.audience == "core-api" assert config.audiences == ["core-api"]
assert config.jwks_uri == "https://auth.example.com/jwks/"
def test_configure_strips_trailing_slash(self): def test_configure_strips_trailing_slash(self):
"""configure should handle trailing slash in issuer.""" """configure should normalise issuers by dropping the trailing slash."""
config = OIDCConfig() config = OIDCConfig()
config.configure( config.configure(
enabled=True, enabled=True,
issuer="https://auth.example.com/", issuers=["https://auth.example.com/"],
audience="core-api" audiences=["core-api"],
) )
assert config.jwks_uri == "https://auth.example.com/jwks/" assert config.issuers == ["https://auth.example.com"]
def test_configure_accepts_multiple_issuers(self):
"""The point of the multi-issuer change: more than one is allowed."""
config = OIDCConfig()
config.configure(
enabled=True,
issuers=["https://a.example.com/", "https://b.example.com"],
audiences=["core-api", "other"],
)
assert config.issuers == ["https://a.example.com", "https://b.example.com"]
assert config.audiences == ["core-api", "other"]
def test_get_jwks_uri_derives_from_issuer(self):
"""The JWKS URI is derived per issuer rather than configured."""
config = OIDCConfig()
assert config.get_jwks_uri("https://auth.example.com") == "https://auth.example.com/jwks/"
assert config.get_jwks_uri("https://auth.example.com/") == "https://auth.example.com/jwks/"
def test_is_valid_issuer_only_accepts_configured(self):
"""
An unconfigured issuer must be rejected. This is the security-relevant
half of multi-issuer support: accepting any issuer would let a token
from an unrelated identity provider through.
"""
config = OIDCConfig()
config.configure(
enabled=True,
issuers=["https://auth.example.com"],
audiences=["core-api"],
)
assert config.is_valid_issuer("https://auth.example.com") is True
assert config.is_valid_issuer("https://auth.example.com/") is True
assert config.is_valid_issuer("https://evil.example.com") is False
class TestGetJWKS: class TestGetJWKSForIssuer:
"""Test get_jwks function.""" """Test get_jwks_for_issuer function."""
ISSUER = "https://auth.example.com"
def test_returns_empty_when_disabled(self): def test_returns_empty_when_disabled(self):
"""get_jwks should return empty dict when OIDC disabled.""" """Should return an empty dict when OIDC is disabled."""
# Save original state
original_enabled = oidc_config.enabled original_enabled = oidc_config.enabled
try: try:
oidc_config.enabled = False oidc_config.enabled = False
# Clear the cache
get_jwks.cache_clear()
result = get_jwks() assert get_jwks_for_issuer(self.ISSUER) == {}
assert result == {}
finally: finally:
# Restore original state
oidc_config.enabled = original_enabled oidc_config.enabled = original_enabled
get_jwks.cache_clear()
@patch("src.auth.oidc.httpx.get") @patch("src.auth.oidc.httpx.get")
def test_fetches_jwks_when_enabled(self, mock_get): def test_fetches_jwks_when_enabled(self, mock_get):
"""get_jwks should fetch JWKS when enabled.""" """Should fetch from the issuer's derived JWKS URI."""
# Save original state
original_enabled = oidc_config.enabled original_enabled = oidc_config.enabled
original_jwks_uri = oidc_config.jwks_uri
try: try:
oidc_config.enabled = True oidc_config.enabled = True
oidc_config.jwks_uri = "https://auth.example.com/jwks/"
get_jwks.cache_clear()
mock_response = MagicMock() mock_response = MagicMock()
mock_response.json.return_value = {"keys": [{"kid": "test"}]} mock_response.json.return_value = {"keys": [{"kid": "abc"}]}
mock_response.raise_for_status = MagicMock()
mock_get.return_value = mock_response mock_get.return_value = mock_response
result = get_jwks() result = get_jwks_for_issuer(self.ISSUER)
assert "keys" in result assert result == {"keys": [{"kid": "abc"}]}
mock_get.assert_called_once() mock_get.assert_called_once()
assert mock_get.call_args[0][0] == f"{self.ISSUER}/jwks/"
finally: finally:
oidc_config.enabled = original_enabled oidc_config.enabled = original_enabled
oidc_config.jwks_uri = original_jwks_uri
get_jwks.cache_clear()
@patch("src.auth.oidc.httpx.get") @patch("src.auth.oidc.httpx.get")
def test_raises_exception_on_error(self, mock_get): def test_caches_per_issuer(self, mock_get):
"""get_jwks should raise HTTPException on fetch error.""" """
# Save original state A second call for the same issuer must not refetch, and a different
issuer must. Caching by issuer is the behaviour the multi-issuer change
introduced, and a shared cache would have served one issuer's keys for
another — which would be a verification bypass, not just a slow path.
"""
original_enabled = oidc_config.enabled original_enabled = oidc_config.enabled
original_jwks_uri = oidc_config.jwks_uri
try: try:
oidc_config.enabled = True oidc_config.enabled = True
oidc_config.jwks_uri = "https://auth.example.com/jwks/" mock_response = MagicMock()
get_jwks.cache_clear() mock_response.json.return_value = {"keys": []}
mock_get.return_value = mock_response
mock_get.side_effect = Exception("Connection error") get_jwks_for_issuer(self.ISSUER)
get_jwks_for_issuer(self.ISSUER + "/") # same issuer, normalised
assert mock_get.call_count == 1
get_jwks_for_issuer("https://other.example.com")
assert mock_get.call_count == 2
finally:
oidc_config.enabled = original_enabled
@patch("src.auth.oidc.httpx.get")
def test_raises_503_on_fetch_error(self, mock_get):
"""A JWKS fetch failure should surface as 503, not leak the cause."""
original_enabled = oidc_config.enabled
try:
oidc_config.enabled = True
mock_get.side_effect = Exception("Connection failed")
with pytest.raises(HTTPException) as exc_info: with pytest.raises(HTTPException) as exc_info:
get_jwks() get_jwks_for_issuer(self.ISSUER)
assert exc_info.value.status_code == 503 assert exc_info.value.status_code == 503
finally: finally:
oidc_config.enabled = original_enabled oidc_config.enabled = original_enabled
oidc_config.jwks_uri = original_jwks_uri
get_jwks.cache_clear()
class TestGetCurrentUser: class TestGetCurrentUser:
@@ -121,9 +176,7 @@ class TestGetCurrentUser:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_returns_none_when_disabled(self): async def test_returns_none_when_disabled(self):
"""get_current_user should return None when OIDC disabled.""" """get_current_user should return None when OIDC disabled."""
# Save original state
original_enabled = oidc_config.enabled original_enabled = oidc_config.enabled
try: try:
oidc_config.enabled = False oidc_config.enabled = False
@@ -136,9 +189,7 @@ class TestGetCurrentUser:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_raises_401_when_enabled_without_token(self): async def test_raises_401_when_enabled_without_token(self):
"""get_current_user should raise 401 when enabled but no token.""" """get_current_user should raise 401 when enabled but no token."""
# Save original state
original_enabled = oidc_config.enabled original_enabled = oidc_config.enabled
try: try:
oidc_config.enabled = True oidc_config.enabled = True
@@ -157,11 +208,12 @@ class TestOIDCGlobalConfig:
"""oidc_config should be an OIDCConfig instance.""" """oidc_config should be an OIDCConfig instance."""
assert isinstance(oidc_config, OIDCConfig) assert isinstance(oidc_config, OIDCConfig)
def test_global_config_starts_disabled(self): def test_global_config_exposes_the_multi_issuer_surface(self):
"""oidc_config should start disabled by default.""" """
# This tests the initial state before any configure() is called Asserts the shape rather than the values, since the live state depends
# The actual state depends on app configuration on app configuration. These four are what callers depend on.
assert hasattr(oidc_config, 'enabled') """
assert hasattr(oidc_config, 'issuer') assert hasattr(oidc_config, "enabled")
assert hasattr(oidc_config, 'audience') assert hasattr(oidc_config, "issuers")
assert hasattr(oidc_config, 'jwks_uri') assert hasattr(oidc_config, "audiences")
assert callable(oidc_config.get_jwks_uri)
-307
View File
@@ -1,307 +0,0 @@
"""Tests for Ollama client."""
import pytest
from unittest.mock import patch, AsyncMock, MagicMock
import json
from src.models.ollama_client import OllamaClient, get_ollama_client, close_ollama_client
class TestOllamaClientInit:
"""Test OllamaClient initialization."""
@patch("src.models.ollama_client.settings")
def test_uses_settings_defaults(self, mock_settings):
"""Client should use settings for defaults."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 60
client = OllamaClient()
assert client.base_url == "http://ollama:11434"
assert client.timeout == 60
@patch("src.models.ollama_client.settings")
def test_creates_http_client(self, mock_settings):
"""Client should create httpx AsyncClient."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
assert client.client is not None
class TestOllamaClientClose:
"""Test client close functionality."""
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_close_closes_client(self, mock_settings):
"""close should close the HTTP client."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
with patch.object(client.client, "aclose", new_callable=AsyncMock) as mock_close:
await client.close()
mock_close.assert_called_once()
class TestOllamaClientResolveModel:
"""Test model resolution."""
@patch("src.models.ollama_client.settings")
def test_resolves_aliased_model(self, mock_settings):
"""resolve_model should map alias to actual model."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
mock_settings.model_aliases = {"gpt-3.5-turbo": "gemma:7b"}
client = OllamaClient()
result = client.resolve_model("gpt-3.5-turbo")
assert result == "gemma:7b"
@patch("src.models.ollama_client.settings")
def test_returns_original_if_no_alias(self, mock_settings):
"""resolve_model should return original if no alias found."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
mock_settings.model_aliases = {}
client = OllamaClient()
result = client.resolve_model("llama2")
assert result == "llama2"
class TestOllamaClientHealthCheck:
"""Test health check functionality."""
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_health_check_returns_true_on_200(self, mock_settings):
"""Health check should return True when Ollama responds 200."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
mock_response = MagicMock()
mock_response.status_code = 200
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.return_value = mock_response
result = await client.health_check()
assert result is True
mock_get.assert_called_once()
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_health_check_returns_false_on_error(self, mock_settings):
"""Health check should return False on connection error."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.side_effect = Exception("Connection refused")
result = await client.health_check()
assert result is False
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_health_check_returns_false_on_non_200(self, mock_settings):
"""Health check should return False on non-200 status."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
mock_response = MagicMock()
mock_response.status_code = 500
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.return_value = mock_response
result = await client.health_check()
assert result is False
class TestOllamaClientListModels:
"""Test list models functionality."""
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_list_models_returns_dict(self, mock_settings):
"""list_models should return dict with models."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
models_data = {
"models": [
{"name": "llama2", "size": 1000000},
{"name": "gemma:7b", "size": 2000000}
]
}
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = models_data
mock_response.raise_for_status = MagicMock()
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.return_value = mock_response
result = await client.list_models()
assert result == models_data
assert len(result["models"]) == 2
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_list_models_raises_on_error(self, mock_settings):
"""list_models should raise on error."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
client = OllamaClient()
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.side_effect = Exception("Connection error")
with pytest.raises(Exception):
await client.list_models()
class TestOllamaClientGenerateNonStreaming:
"""Test non-streaming generation."""
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_generate_non_streaming_returns_response(self, mock_settings):
"""generate_non_streaming should return response dict."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
mock_settings.model_aliases = {}
client = OllamaClient()
response_data = {
"message": {"content": "Hello! How can I help?"},
"prompt_eval_count": 10,
"eval_count": 20
}
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = response_data
mock_response.raise_for_status = MagicMock()
with patch.object(client.client, "post", new_callable=AsyncMock) as mock_post:
mock_post.return_value = mock_response
result = await client.generate_non_streaming("llama2", "Hello")
assert result["response"] == "Hello! How can I help?"
assert result["tokens"]["prompt"] == 10
assert result["tokens"]["completion"] == 20
assert result["tokens"]["total"] == 30
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_generate_non_streaming_includes_max_tokens(self, mock_settings):
"""generate_non_streaming should include max_tokens in payload."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
mock_settings.model_aliases = {}
client = OllamaClient()
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {"message": {"content": "Hi"}}
mock_response.raise_for_status = MagicMock()
with patch.object(client.client, "post", new_callable=AsyncMock) as mock_post:
mock_post.return_value = mock_response
await client.generate_non_streaming("llama2", "Hello", max_tokens=100)
call_args = mock_post.call_args
assert call_args[1]["json"]["options"]["num_predict"] == 100
class TestOllamaClientGenerateStreaming:
"""Test streaming generation."""
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_generate_streaming_yields_content(self, mock_settings):
"""generate_streaming should yield content chunks."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
mock_settings.model_aliases = {}
client = OllamaClient()
# Create mock streaming response
async def mock_aiter_lines():
yield json.dumps({"message": {"content": "Hello"}})
yield json.dumps({"message": {"content": " world"}})
yield json.dumps({"done": True})
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.raise_for_status = MagicMock()
mock_response.aiter_lines = mock_aiter_lines
mock_stream_context = AsyncMock()
mock_stream_context.__aenter__.return_value = mock_response
mock_stream_context.__aexit__.return_value = None
with patch.object(client.client, "stream", return_value=mock_stream_context):
chunks = []
async for chunk in client.generate_streaming("llama2", "Hi"):
chunks.append(chunk)
assert "Hello" in chunks
assert " world" in chunks
class TestOllamaClientSingleton:
"""Test singleton pattern."""
@patch("src.models.ollama_client.settings")
def test_get_ollama_client_returns_same_instance(self, mock_settings):
"""get_ollama_client should return singleton."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
import src.models.ollama_client as module
module._ollama_client = None
client1 = get_ollama_client()
client2 = get_ollama_client()
assert client1 is client2
@pytest.mark.asyncio
@patch("src.models.ollama_client.settings")
async def test_close_ollama_client_clears_singleton(self, mock_settings):
"""close_ollama_client should clear the singleton."""
mock_settings.ollama_base_url = "http://ollama:11434"
mock_settings.ollama_timeout = 30
import src.models.ollama_client as module
module._ollama_client = None
client = get_ollama_client()
with patch.object(client.client, "aclose", new_callable=AsyncMock):
await close_ollama_client()
assert module._ollama_client is None
+17 -4
View File
@@ -4,6 +4,7 @@ from fastapi.testclient import TestClient
from unittest.mock import patch, AsyncMock, MagicMock from unittest.mock import patch, AsyncMock, MagicMock
from src.main import app from src.main import app
from src.domains.tools.controller import tools_controller
@pytest.fixture @pytest.fixture
@@ -36,9 +37,21 @@ class TestDNSLookup:
) )
assert response.status_code == 200 assert response.status_code == 200
@patch("src.controllers.tools_controller.DNSService") def test_dns_lookup_returns_result(self, client):
def test_dns_lookup_returns_result(self, mock_dns_class, client): """DNS lookup should return lookup results.
"""DNS lookup should return lookup results."""
Patches the live singleton's `dns_service` attribute, not the
`src.controllers.tools_controller.DNSService` class: that module is
the legacy top-level package (not wired into `src.main`, see
CLAUDE.md "Legacy top-level packages"). `client` exercises
`src.main.app`, which routes through
`src.domains.tools.controller.tools_controller`, a singleton built
at import time — so patching the class there would also miss,
since `tools_controller.dns_service` is already a constructed
instance by the time a test patches the class. Patching the
instance attribute directly is the only patch that actually
intercepts this request path.
"""
mock_response = MagicMock() mock_response = MagicMock()
mock_response.success = True mock_response.success = True
mock_response.domain = "example.com" mock_response.domain = "example.com"
@@ -59,8 +72,8 @@ class TestDNSLookup:
mock_service = MagicMock() mock_service = MagicMock()
mock_service.lookup = AsyncMock(return_value=mock_response) mock_service.lookup = AsyncMock(return_value=mock_response)
mock_dns_class.return_value = mock_service
with patch.object(tools_controller, "dns_service", mock_service):
response = client.post( response = client.post(
"/tools/dns/lookup", "/tools/dns/lookup",
json={"domain": "example.com", "record_type": "A"} json={"domain": "example.com", "record_type": "A"}