Compare commits

...
52 Commits
Author SHA1 Message Date
jpmschweitzer 3536c40608 fix(permissions): narrow rm -rf deny globs to their exact forms
The trailing wildcard on the three rm -rf deny entries spanned path
separators, so Bash(rm -rf /*) matched every absolute path on the
machine rather than the filesystem root, and the ~ and $HOME entries
had the same shape. Narrowed to the exact literal forms.

These rules match literal command text, so they still stop a typo on
rm -rf /, rm -rf ~ or rm -rf $HOME exactly, but they no longer stop a
recursive delete aimed at any other path. That reduced cover is
deliberate, not an oversight.
2026-08-25 20:31:15 +02:00
jpmschweitzerandClaude 0c7e1cb04e fix(tests): mock the live DNS service instance, not a dead legacy class (T-55)
test_dns_lookup_returns_result patched src.controllers.tools_controller.
DNSService, which is never imported by the request path under test.
client wraps src.main.app, which routes /tools/dns/lookup through
src.domains.tools.controller.tools_controller — a singleton constructed
at import time from src.domains.tools.dns.service.DNSService. The patched
class was dead; the mock was never consulted, so the test issued a real
DNS query for example.com and asserted on its outcome. With no network
the query times out and the assertion fails (D-26).

The three sibling tests in the same class patch the same dead class and
also run unmocked, but happen not to notice: DNS failures are caught
inside DNSService.lookup() and returned as a normal 200 response with
success=False, and their assertions only check status_code / DNSQueryError
branches that don't depend on resolution actually succeeding. Only this
test's `data["success"] is True` assertion is sensitive to the real
network outcome, which is why it's the only one D-26's namespace run
catches. Not touched here — out of T-55's scope, flagging for the record.

Fix patches tools_controller.dns_service, the actual instance attribute
the live route calls, via patch.object on the singleton rather than
patch() on the constructor class (the instance already exists by the
time a class-level patch would apply).

Verified:
- unshare -rn (lo up): 381 passed, exit 0 (was 1 failed, 380 passed, exit 2)
- with network: 381 passed, exit 0 (unchanged from before the fix)
- mutation check: retargeted the patch.object to a nonexistent attribute
  name, confirmed count==1 before editing; namespace run then reproduced
  the original failure (1 failed, 380 passed); reverted and reconfirmed
  381 passed, exit 0.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-18 15:33:58 +02:00
jpmschweitzerandClaude 6695215aeb build(make): prove setup worked instead of assuming pip's exit code (T-47)
pip install exits 0 whether the result is usable or not — that is the D-24
shape exactly, a step whose job is to not fail. On 2026-08-09 the venv here
existed and pip had already succeeded, but sqlalchemy was declared in
requirements.txt and not installed. That surfaced as 11 pytest collection
errors that read as broken imports rather than as an environment problem.

setup now ends with `pytest --collect-only tests/`, which exercises every
import the suite touches without running anything. pip check was considered
as a cheaper alternative and rejected: it only checks the installed set's
internal consistency against itself, so it would not have caught this case —
sqlalchemy was still present as another package's transitive dependency even
after being dropped from requirements.txt. collect-only checks declared vs.
actually usable directly, which is the axis that broke.

Verified: two clean runs (33.5s cold, 5.7s idempotent re-run, second changes
nothing). Reproduced the original failure by uninstalling sqlalchemy from an
otherwise-correct venv — collect-only alone then exits 2 with 11 collection
errors; make setup against that same state reinstalls it and exits 0 with
381/381 collected.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-17 12:05:28 +02:00
jpmschweitzerandClaude d2e0a49e7c build(ci): move the pre-push gate into the Makefile
The hook carried ~50 lines of gitleaks logic and a comment explaining it was
self-contained because "this repo has no Makefile". It has one now, so the
reason is gone and the arrangement is backwards: a hook is a trigger, and
logic belongs where it can be read, run by hand, and changed under review.

.githooks/pre-push is now a byte-identical shim onto `make pre-push` in every
repo in the workspace. The scan itself moves to ci/secrets.sh unchanged, and
`make secrets` runs it on its own.

The call surface is identical everywhere; what it runs is not, and should not
be — each repo gates what it actually has. That is the point of standardising
the name rather than the contents: nobody has to read a repo to find out how
to check it.

secrets runs first, deliberately. It is the only failure here that cannot be
undone by fixing it afterwards — a failed lint costs another commit, a pushed
credential is cached and indexed whether or not it is later deleted.

Some of these gates fail today, on lint debt that predates them, and they are
left wired anyway. The board was measured once and written down in T-56
instead of being worked around here. Narrowing each gate to whatever already
passes would produce a gate that reports success for doing nothing, which is
the failure this workspace keeps rediscovering.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 18:57:22 +02:00
jpmschweitzerandClaude 755aa61107 ci(make): reserve exit 69 for "could not run" (D-26)
Environment guards now exit 69 rather than 1, so a caller can tell a suite
that could not start from one that ran and failed. The first toj test sweep
reported "3 repositories failed" and none of the three had executed a test —
two could not find go, one had no venv. That points the reader at the tests
when the fault is in the environment.

Only the environment guards change. A gitleaks finding, a failed test run and
a vulncheck hit still exit 1, because those did run and did fail.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 15:56:19 +02:00
jpmschweitzerandClaude 01349a83f2 test: repair the suite against the current API
The suite could not even collect: the venv was missing declared dependencies,
and five tests asserted an API that had moved on. 11 collection errors to 381
passing.

test_oidc.py was written for the single-issuer API and 6243f29 replaced it.
issuer and audience became lists, jwks_uri stopped being an attribute in
favour of get_jwks_uri(issuer), get_jwks became get_jwks_for_issuer, and the
lru_cache became a per-issuer dict so cache_clear no longer exists. Rewritten
against the current surface, with coverage added for the two behaviours the
multi-issuer change introduced and never tested: is_valid_issuer rejecting an
unconfigured issuer, and the cache keying per issuer. Both are
security-relevant — a shared cache would serve one issuer keys for another.

Three /auth/me tests asserted a path that does not exist. The route is
declared as /me inside AuthController.create_router() and mounts at
/auth/users/me; the generated spec is authoritative and the local app and the
deployed service agree on it. Those tests had never passed.

test_handles_empty_groups expected groups == [""] for an empty header. oidc.py
has returned [] since the initial commit, and [] is correct — [""] would also
be unsafe, since any check doing "" in groups would match.

test_model_aliases_property covered Settings.model_aliases, deleted with the
Ollama integration in c1f16d4. Removed rather than repaired.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 15:21:43 +02:00
jpmschweitzerandClaude 7815e1c231 build: add the Makefile command surface (D-27)
Every repo gets one at the root: help, plus test and lint where those exist.
The point is that a target name means the same thing in every repo, so an
agent or a person can act without reading the repo first.

Paths resolve here rather than in callers (D-10). python3 on this host is 3.8
and cannot parse these sources, and a bare pytest or ruff resolves only in a
login shell — so both are named explicitly through the venv, and a missing
venv fails with the command to fix it rather than a bare no-such-file.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 15:10:31 +02:00
jpmschweitzerandClaude e9fc09af26 chore(claude): pin PQL_VAULT per project so cwd stops choosing the vault
pql is now a bare word on PATH, which removed the long incantation that had
been forcing --vault into every call by habit. Convenience lowered the cost
of the wrong thing without lowering the cost of the right one: a three-word
pql ticket new targets whichever vault the cwd happens to sit in, and there
are nine of them with colliding id sequences.

PQL_VAULT in each project settings file makes the vault a property of the
session rather than of the working directory — the same lesson Rule 3 records
for git -C, applied to pql. Verified the env var overrides cwd discovery,
that an explicit --vault still beats the env var, and that the harness
hot-reloads it without a restart.

This does not make provenance visible: no output says which vault answered,
so a forgotten --vault still returns a well-formed answer about the wrong
dataset. That remains T-37.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 13:48:58 +02:00
jpmschweitzerandClaude 56b991c965 chore(claude): deny toj in the sub-repos
toj is now on the global PATH as /usr/local/bin/toj, so its scope boundary
had to stop being "the absolute path is inconvenient to type" and start
being a rule. Its repo and settings verbs operate on the workspace root; run
from inside this repo they answer about the wrong tree.

Both spellings are denied, bare and absolute, because a deny with one
spelling left open is decorative.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 13:42:03 +02:00
jpmschweitzerandClaude 387a3dbb02 ci: gate pushes on a gitleaks scan of the outgoing commits
No repo here scanned for committed credentials. The hook is self-contained
rather than delegating to a Makefile, because this repo has none and a hook
reaching into a sibling repo breaks the moment this one is cloned elsewhere.

Scans the outgoing range rather than full history: history carries settled
findings — test fixtures, vendored third-party code — and a gate that fails
on something unfixable gets bypassed within a week.

Setting core.hooksPath means pql init must replant its replication shims into
.githooks, which is why they are gitignored here alongside the tracked
pre-push. Same layout pql itself uses.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 12:48:53 +02:00
jpmschweitzerandClaude 7f90369621 docs: qualify workspace decision ids cited from this repo
Decision ids are per-vault sequences, so they collide by construction
once there is more than one vault -- and every repo now has one. A bare
D-15 here will mean this repo's D-15 the moment this repo records one.
Cross-vault references are therefore qualified: workspace D-15.

Not hypothetical: pql holds D-1 through D-31 while the workspace holds
D-1 through D-21, so every workspace id currently collides with an
unrelated pql one. A bare id is not wrong the day it is written -- it
decays into wrong as the other vault grows, and nothing flags it.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 04:17:07 +02:00
jpmschweitzerandClaude 659f08e576 chore: adopt the workspace agent-config baseline
Commits a .claude/settings.json rather than leaving permissions to
per-developer local state, and initialises a pql vault for this repo's
tickets and internal decisions.

Every git deny rule appears in both the `git <verb>` and `git * <verb>`
forms. Only the second catches `git -C <path>`, and without it the whole
deny list is decorative -- it looks like a policy and stops nothing.

The allow list carries pql's absolute path alongside the bare name.
pql is installed to ~/.local/bin, which is on the login PATH but not the
one a non-interactive shell gets, so the bare-name rules match nothing on
their own and every call would prompt anyway.

.gitignore now covers .claude/settings.local.json, which is machine-local
and must never be shared. `pql init` contributed the .pql/* rules with an
exception for the changelog, which is the replication log of record and
has to be committed for tickets to travel with a clone.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 03:13:08 +02:00
jpmschweitzerandClaude 29bfa3259d docs: replace AGENTS.md with a repo-specific CLAUDE.md
One agent doc per repo, and it is CLAUDE.md. Two docs describing one repo
drift, and the one nobody read is always the one holding the rule that
mattered. Written fresh rather than reformatted, so the structure follows
what someone working here actually needs.

Two rules from the old file are gone deliberately. The mandate to branch
for every change was retired in favour of one linear-history policy, and
the release snippet used `git add -A`, which sweeps in whatever else is
dirty.

The architecture section is the part worth reading. An earlier draft
called src/auth, src/controllers, src/clients, src/dns and src/models
dead code, derived from grepping main.py's imports. That was wrong:
main.py:55 calls initialize_oidc(), which imports and configures
src.auth.oidc from inside the function body, so src/auth is configured
with live Authentik issuers on every boot. It also missed four genuinely
unreferenced packages. The section now states the method used -- import
the app in the container and read sys.modules -- and its blind spot, that
a cold snapshot cannot see a module imported on a request path.

Also records that the README's "runs as non-root user (uid 1000)" is
false: the Dockerfile has no USER directive. Flagged rather than fixed,
since changing the runtime user is not a docs change.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 03:12:02 +02:00
jpmschweitzerandClaude Fable 5 b3383f19b3 chore(ci): push images via git.schweitz.net registry
The .internal registry domain is being retired; git.schweitz.net now
serves the registry without SSO on /v2/.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 11:09:58 +02:00
Jeroen SchweitzerandClaude Opus 4.5 fcf5c8ccee feat: add /tools/news endpoint for news ticker integration
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m15s
- Add NewsHeadline and NewsResponse schemas
- Add NewsService for parsing news from Qdrant volatile collection
- Add GET /tools/news endpoint to tools controller
- News fetched from 'news' namespace in volatile_{user} collection

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 21:49:05 +01:00
Jeroen SchweitzerandClaude Opus 4.5 768cea2c89 fix: pass forecast raw_data to service for parsing
Build and Push / build (push) Successful in 1m14s
Build and Push / release (push) Successful in 2s
Qdrant client was extracting 'days' (int) instead of 'daily' (list)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 18:45:24 +01:00
Jeroen SchweitzerandClaude Opus 4.5 26ecc3e5fd fix: convert wind direction degrees to cardinal string
Build and Push / release (push) Successful in 2s
Build and Push / build (push) Successful in 1m14s
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 18:36:38 +01:00
Jeroen SchweitzerandClaude Opus 4.5 5ff4ba0a43 fix: environment data parsing for scheduler format
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m15s
- Forecast: Check 'daily' key first (scheduler stores count in 'days')
- Sun times: Use ISO fields, calculate daylight from various sources
- Air quality: Support aqi_us/aqi_european and nitrogen_dioxide fields

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 18:30:30 +01:00
Jeroen SchweitzerandClaude Opus 4.5 bb438e22d6 fix: strip email domain from user identifier for environment endpoint
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m15s
If preferred_username is an email (user@domain.com), extract just the
username part to match Qdrant collection naming (volatile_user).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 15:46:16 +01:00
Jeroen SchweitzerandClaude Opus 4.5 3bb3b01dbd fix: OIDC audience validation - use string not list
Build and Push / build (push) Successful in 1m15s
Build and Push / release (push) Successful in 3s
python-jose jwt.decode() requires audience as string or None, not list.
Now extract and validate audience from unverified claims first,
then use token's actual audience for JWT decode.

Fixes "audience must be a string or None" error.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 15:35:05 +01:00
Jeroen SchweitzerandClaude Opus 4.5 ce761a9d2c debug: add logging for OIDC token validation
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 13:58:39 +01:00
Jeroen SchweitzerandClaude Opus 4.5 6243f29aae feat: multi-issuer OIDC support with config consolidation
Build and Push / release (push) Successful in 2s
Build and Push / build (push) Successful in 1m15s
- Support multiple OAuth providers (core-api, tatlock-ui, tatlock)
- Changed oidc_issuer (string) to oidc_issuers (list)
- Per-issuer JWKS caching
- Validates token issuer against allowed list
- Consolidated config files (removed deprecated src/config.py, src/security.py)
- Updated imports to use src/shared/config and src/shared/security

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 13:09:29 +01:00
Jeroen SchweitzerandClaude Opus 4.5 c4d32952db fix: OIDC multi-audience support and Swagger UI fix
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m13s
- Accept tokens from multiple clients (core-api, tatlock-ui, tatlock)
- Fixed main.py to use oidc_audiences[0] for Swagger UI OAuth

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-08 12:17:16 +01:00
Jeroen Schweitzer 4c45f139d9 correct documentation 2026-01-08 12:11:01 +01:00
Jeroen Schweitzer 67b33314fe fix: update main.py to use oidc_audiences list
Build and Push / release (push) Failing after 3s
Build and Push / build (push) Has been skipped
2026-01-07 19:34:05 +01:00
Jeroen SchweitzerandClaude Opus 4.5 6ce34cc016 fix: accept multiple OIDC audiences for cross-client auth
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m15s
- Changed oidc_audience (string) to oidc_audiences (list)
- Now accepts tokens with audience: core-api, tatlock-ui, tatlock
- Fixes environment endpoint returning "default" user when using
  tatlock-ui token (audience mismatch was causing JWT claims error)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 19:01:39 +01:00
Jeroen SchweitzerandClaude Opus 4.5 c1f16d44e5 refactor: remove Ollama integration and unused AI configuration
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m19s
- Remove src/models/ollama_client.py, embeddings.py, embeddings_ollama.py
- Remove model aliases and AI config from settings (both config.py files)
- Update health endpoints to only check database connectivity
- Update tests to reflect database-only health checks
- Update README, .env.example, and OIDC docstrings

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 17:56:48 +01:00
Jeroen SchweitzerandClaude Opus 4.5 4df5cfc106 fix: initialize OIDC config for both auth modules
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m30s
The domains.auth.oidc module had its own oidc_config instance that
wasn't being configured, causing environment endpoint to always use
hardcoded "local" user instead of authenticated user.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 16:00:14 +01:00
Jeroen SchweitzerandClaude Opus 4.5 0a16688cc8 chore: release v1.10.2
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 2m7s
Enhanced environment endpoint logging for debugging user resolution.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 15:32:53 +01:00
Jeroen SchweitzerandClaude Opus 4.5 a7535fe8ea fix: correct OIDC import path in tools controller
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m14s
The environment endpoint was importing from non-existent path
`src.oidc.dependencies` instead of `src.auth.oidc`, causing
authentication to fail and queries to go to wrong Qdrant collection.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 23:36:24 +01:00
Jeroen SchweitzerandClaude Opus 4.5 6045c6ac6a feat: add environment endpoint for weather, forecast, and sun data
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 2m14s
Add GET /tools/environment endpoint that fetches weather, forecast, sun times,
and air quality data from user's volatile Qdrant collection.

- Add qdrant-client dependency
- Create QdrantReadClient wrapper for read-only queries
- Add environment schemas and service in tools domain
- Parse weather, forecast, sun times, and air quality from Qdrant payloads
- Support user-specific collections via preferred_username from OIDC
- Add comprehensive service tests (13 tests)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-06 23:04:59 +01:00
Jeroen Schweitzer b8fca7060f npm config 2026-01-04 21:51:21 +01:00
Jeroen SchweitzerandClaude Opus 4.5 e3a49c800a fix: SQLAlchemy async lazy loading for new users
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m11s
Initialize user.roles=[] and user.preferences on new user creation
to avoid MissingGreenlet error in async context.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 18:05:34 +01:00
Jeroen SchweitzerandClaude Opus 4.5 7ab9f73a1d chore: release v1.9.3
Build and Push / release (push) Successful in 2s
Build and Push / build (push) Successful in 1m10s
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 17:49:26 +01:00
Jeroen SchweitzerandClaude Opus 4.5 dd5b794de4 fix: handle non-UUID sub claim in auth sync
Authentik JWT sub claim may not be a valid UUID.
Now derives a deterministic UUID from the sub string if parsing fails.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 17:45:05 +01:00
Jeroen SchweitzerandClaude Opus 4.5 dd0997679f fix: /auth/users/me now supports NPM forward auth headers
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m11s
Added get_current_user_or_forward_auth() combined dependency that:
- First checks for X-authentik-* headers from NPM forward auth (web)
- Falls back to JWT Bearer token validation (mobile/native)

This fixes web authentication where browsers don't send Bearer tokens
but rely on NPM's forward auth proxy to pass user info via headers.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-04 00:24:41 +01:00
Jeroen SchweitzerandClaude Opus 4.5 e2226cd923 fix(build): add production API URLs to Docker build
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m8s
Dockerfile now passes --dart-define flags for CORE_API_URL and
TATLOCK_API_URL pointing to schweitz.net domains. This enables
requiresAuth=true, fixing auth being completely skipped in production.

Also added service port reference table to AGENTS.md.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 23:18:34 +01:00
Jeroen SchweitzerandClaude Opus 4.5 7bf3c76a1b fix(cors): use explicit origins instead of wildcard
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m11s
When allow_credentials=True, browsers reject wildcard (*) origins.
Added specific allowed origins for Tatlock domains.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 23:09:47 +01:00
Jeroen SchweitzerandClaude Opus 4.5 4ae8cfbc0b chore: release v1.9.0
Build and Push / release (push) Successful in 3s
Build and Push / build (push) Successful in 1m10s
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 22:24:36 +01:00
Jeroen SchweitzerandClaude Opus 4.5 69045a78a7 test(auth): add tests for GET /auth/me endpoint
Add comprehensive tests for NPM forward auth endpoint:
- Forward auth header parsing
- User lookup methods (get_user_by_email, get_user_by_authentik_id)
- Admin gate authorization logic
- OpenAPI spec validation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 22:14:13 +01:00
Jeroen SchweitzerandClaude Opus 4.5 884996fd83 feat(auth): implement GET /auth/me for NPM forward auth
Add endpoint to get current user profile from NPM forward auth headers.
Enables web authentication flow where NPM handles Authentik login.

- Read X-authentik-* headers set by NPM forward auth
- Auto-create user if not in database (first login via web)
- Sync roles from current Authentik groups
- Add get_user_by_email and get_user_by_authentik_id helpers

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 21:54:44 +01:00
Jeroen SchweitzerandClaude Opus 4.5 8bd13eab09 ci: trigger build on version tag push with auto-release
Changed workflow to:
- Trigger on push of v* tags instead of release publish
- Auto-create Gitea release via API
- Then build and push Docker image

This simplifies deployment: just push a version tag.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 20:40:44 +01:00
Jeroen SchweitzerandClaude Opus 4.5 397a47c8fc feat(auth): implement Phase 4 user profile and API key endpoints
Build and Push / build (release) Successful in 1m10s
Add user profile, preferences, and API key management endpoints:
- GET /auth/users/me - full user profile with roles and preferences
- GET/PATCH /auth/users/me/preferences - user preferences management
- GET/POST/DELETE /auth/users/me/api-keys - API key lifecycle

API keys use tak_ prefix, SHA-256 hashing, and are shown only once on creation.
Preferences support partial updates with JSON merge behavior.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 20:24:46 +01:00
Jeroen SchweitzerandClaude Opus 4.5 7752cd9d23 feat(auth): implement group-role mapping and permission system
Architecture changes:
- Permission format: domain.category:action (e.g., control-room.general:admin)
- Decoupled groups from roles via group_roles mapping table
- Groups are organizational (synced from Authentik)
- Roles are permissions (admin-managed via API)

New features:
- require_permission() and require_any_permission() dependency factories
- Action hierarchy: admin > editor > user > viewer
- Global admin override (admin.general:admin grants all)
- Group-role management endpoints (assign/remove roles)
- GET /auth/roles endpoint to list all roles

Database changes:
- Added category column to roles table (default: general)
- Removed authentik_group column (decoupled)
- Added group_roles association table
- Added user_groups association table
- Migration updates role names to domain.general:action format

Tests:
- 67 new tests for auth service and controller
- Covers token validation, user sync, role sync
- Covers group-role assignment/removal
- Covers schema conversions and permission system

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 19:52:10 +01:00
Jeroen SchweitzerandClaude Opus 4.5 075b0ec297 feat: add system stats API for dashboard monitoring
Build and Push / build (release) Successful in 1m44s
- GET /tools/system/stats - Real-time host system statistics
- CPU usage, memory, all mounted disks, network I/O
- GPU/VRAM stats via nvidia-smi (if available)
- Uses psutil for cross-platform host metrics
- Auto-discovers and filters real filesystems

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 17:30:45 +01:00
Jeroen SchweitzerandClaude Opus 4.5 49be935b5d feat: add link_type field to quick links
Build and Push / build (release) Successful in 1m10s
Adds link_type column to quick_links table for iframe vs new_tab behavior.
Includes Alembic migration and schema updates.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 14:38:10 +01:00
Jeroen SchweitzerandClaude Opus 4.5 381d43b60b feat: add dashboard API with quick links and widgets
Build and Push / build (release) Successful in 1m28s
- Dashboard domain with Quick Links CRUD + reorder endpoints
- Dashboard widgets management endpoints
- Database migrations for quick_links and dashboard_widgets tables
- Static file controller for Organizr widgets
- Default local user when OIDC is disabled
- Domain-based architecture refactor (src/domains/, src/shared/)
- Test suite updated for new structure (285 tests passing)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-03 12:27:57 +01:00
Jeroen SchweitzerandClaude Opus 4.5 e85c9a123d feat: add groups management API
Build and Push / build (release) Successful in 50s
- GET /auth/groups - list groups with search/pagination
- POST /auth/groups/sync-from-authentik - bulk sync from Authentik
- Group model and database migration

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 22:28:06 +01:00
Jeroen SchweitzerandClaude Opus 4.5 3516376d92 chore: cleanup auth code after debugging session
Build and Push / build (release) Successful in 50s
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 21:41:10 +01:00
Jeroen SchweitzerandClaude Opus 4.5 ffa984e271 fix: separate httpx and SQLAlchemy async contexts in bulk sync
Build and Push / build (release) Successful in 50s
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 21:34:34 +01:00
Jeroen SchweitzerandClaude Opus 4.5 7d13be6052 fix: use uuid field instead of pk for Authentik user sync
Build and Push / build (release) Successful in 50s
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 21:27:21 +01:00
Jeroen SchweitzerandClaude Opus 4.5 faff45db90 fix: manually handle session cookies for Authentik API authentication
Build and Push / build (release) Successful in 1m14s
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-01 21:01:29 +01:00
108 changed files with 15261 additions and 2246 deletions
+69
View File
@@ -0,0 +1,69 @@
{
"env": {
"PQL_VAULT": "/mnt/media/Projects/core-api"
},
"permissions": {
"allow": [
"Bash(pql)",
"Bash(pql *)",
"Bash(/home/jpmschweitzer/.local/bin/pql:*)",
"Bash(git status:*)",
"Bash(git log:*)",
"Bash(git diff:*)",
"Bash(git branch:*)",
"Bash(.venv/bin/python -m pytest:*)",
"Bash(.venv/bin/pytest:*)",
"Bash(pytest:*)",
"Bash(alembic *)",
"Bash(docker logs core-api:*)",
"Bash(curl -s http://localhost:8083/*)"
],
"deny": [
"Bash(/mnt/media/Projects/cladmin/ops/bin/toj)",
"Bash(/mnt/media/Projects/cladmin/ops/bin/toj:*)",
"Bash(alembic downgrade base*)",
"Bash(chmod -R 777 *)",
"Bash(chmod 777 *)",
"Bash(dd if=*)",
"Bash(find * -delete*)",
"Bash(find * -exec*)",
"Bash(git * add --all*)",
"Bash(git * add -A*)",
"Bash(git * add .)",
"Bash(git * branch -D *)",
"Bash(git * checkout -- *)",
"Bash(git * clean -fd*)",
"Bash(git * clean -fdx*)",
"Bash(git * commit --no-verify*)",
"Bash(git * merge --no-ff*)",
"Bash(git * push --force*)",
"Bash(git * push -f*)",
"Bash(git * reset --hard*)",
"Bash(git * restore .*)",
"Bash(git add --all*)",
"Bash(git add -A*)",
"Bash(git add .)",
"Bash(git branch -D *)",
"Bash(git checkout -- *)",
"Bash(git clean -fd*)",
"Bash(git clean -fdx*)",
"Bash(git commit --no-verify*)",
"Bash(git merge --no-ff*)",
"Bash(git push --force*)",
"Bash(git push -f*)",
"Bash(git reset --hard*)",
"Bash(git restore .*)",
"Bash(mkfs*)",
"Bash(psql * -c DROP*)",
"Bash(psql * DROP DATABASE*)",
"Bash(psql * TRUNCATE*)",
"Bash(rm -rf $HOME)",
"Bash(rm -rf /)",
"Bash(rm -rf ~)",
"Bash(su *)",
"Bash(sudo *)",
"Bash(toj)",
"Bash(toj:*)"
]
}
}
+1 -4
View File
@@ -39,12 +39,9 @@ HOMEASSISTANT_URL=http://localhost:8123
HOMEASSISTANT_TOKEN=your-long-lived-access-token HOMEASSISTANT_TOKEN=your-long-lived-access-token
# ============================================================================= # =============================================================================
# AI Services # Search
# ============================================================================= # =============================================================================
# Ollama API
OLLAMA_BASE_URL=http://localhost:11434
# SearXNG (self-hosted search) # SearXNG (self-hosted search)
SEARXNG_URL=http://localhost:8080 SEARXNG_URL=http://localhost:8080
+1
View File
@@ -0,0 +1 @@
.pql/changelog/*.sql merge=union
+18 -5
View File
@@ -1,19 +1,32 @@
name: Build and Push name: Build and Push
on: on:
release: push:
types: [published] tags:
- 'v*'
jobs: jobs:
release:
runs-on: ubuntu-latest
steps:
- name: Create Gitea Release
run: |
curl -sf -X POST \
-H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \
-H "Content-Type: application/json" \
-d '{"tag_name": "${{ github.ref_name }}", "name": "Release ${{ github.ref_name }}", "body": "Automated release for ${{ github.ref_name }}"}' \
"${{ github.server_url }}/api/v1/repos/${{ github.repository }}/releases"
build: build:
runs-on: ubuntu-latest runs-on: ubuntu-latest
needs: release
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- name: Login to Gitea Registry - name: Login to Gitea Registry
uses: docker/login-action@v3 uses: docker/login-action@v3
with: with:
registry: git.schweitz.internal registry: git.schweitz.net
username: ${{ secrets.REGISTRY_USER }} username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }} password: ${{ secrets.REGISTRY_PASSWORD }}
@@ -23,8 +36,8 @@ jobs:
context: . context: .
push: true push: true
tags: | tags: |
git.schweitz.internal/jpmschweitzer/core-api:latest git.schweitz.net/jpmschweitzer/core-api:latest
git.schweitz.internal/jpmschweitzer/core-api:${{ github.ref_name }} git.schweitz.net/jpmschweitzer/core-api:${{ github.ref_name }}
- name: Trigger Watchtower update - name: Trigger Watchtower update
if: success() if: success()
+13
View File
@@ -0,0 +1,13 @@
#!/usr/bin/env bash
# Trigger only. The checks live in the Makefile, where they can be read, run by
# hand (`make pre-push`), and changed under review.
#
# This file is identical in every repo in this workspace, deliberately: the call
# surface is the same everywhere even though what each gate runs is not, so
# nobody has to read a repo to find out how to check it (D-27).
#
# Enable per clone with: git config core.hooksPath .githooks
# Never bypass with --no-verify. Suppress a specific finding deliberately
# instead, with a reason — see `make pre-push`.
set -euo pipefail
exec make -C "$(git rev-parse --show-toplevel)" pre-push
+17
View File
@@ -136,3 +136,20 @@ Thumbs.db
# profiling data # profiling data
.prof .prof
# Claude Code local settings (machine-specific, may contain credentials)
.claude/settings.local.json
# pql — ignore everything except the changelog, which is the replication log of
# record and must be committed for tickets to travel with the repo.
.pql/*
!.pql/changelog/
# pql shims planted by `pql init` into the dir core.hooksPath points at.
# Per-clone: each embeds the absolute path of the pql binary that planted it.
# Only .githooks/pre-push is shared.
.githooks/pre-commit
.githooks/post-merge
.githooks/post-checkout
.githooks/post-rewrite
+11
View File
@@ -0,0 +1,11 @@
-- Changelog format marker, written by pql. Comments only: this file
-- is never executed — Import descends into the per-table directories
-- and does not read the changelog root.
--
-- A changelog carrying no marker is format 1, the shape that existed
-- before formats were versioned. An older format is migrated forward
-- by `pql plan upgrade` (and automatically from the post-merge hook);
-- a newer one is refused rather than replayed under rules this binary
-- does not know. See D-28 and docs/versions.md.
-- pql:changelog_format: 2.0.0
-- pql:written_by: 2.2.0
+139
View File
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
+139
View File
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
+139
View File
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
-72
View File
@@ -1,72 +0,0 @@
# AGENTS.md
> **Start every session by reading this file.**
> This file outlines the operational protocols, coding standards, and architectural decisions for this FastAPI project.
## 1. Agent Operational Protocols
### 🧠 Work Patterns (Plan-Act-Reflect)
* **Plan:** Before writing code, briefly outline your plan. Identify which files you will touch and what the side effects might be.
* **Act:** Execute the changes in small, atomic steps.
* **Reflect:** After coding, verify your work. Did you break existing tests? Did you add new tests?
### 🛡️ Git Discipline
* **NEVER commit to `main` or `master` directly.** Always create a feature branch: `feature/your-feature-name` or `fix/issue-description`.
* **Commit Messages:** Use the [Conventional Commits](https://www.conventionalcommits.org/) format.
* `feat: add user login endpoint`
* `fix: resolve database connection timeout`
* `refactor: split monolith dependency file`
* **Atomic Commits:** Keep commits small. One logical change = one commit.
### 📝 Changelog Maintenance
* **Update `CHANGELOG.md`** with every user-facing change.
* Format: `## [Unreleased] - YYYY-MM-DD` followed by `### Added`, `### Changed`, or `### Fixed`.
### 🚀 Release Flow
When changes are ready for deployment:
1. **Ask user if deploy cycle is desired **
2. **Update version** in `pyproject.toml`:
- Bug fixes: bump patch version (1.8.3 → 1.8.4)
- New features: bump minor version (1.8.4 → 1.9.0)
3. **Update CHANGELOG.md**:
- Move items from `[Unreleased]` to new version section
- Add release date: `## [1.8.4] - 2025-12-16`
4. **Commit and tag**:
```bash
git add -A
git commit -m "fix: description of changes"
git tag v1.8.4
git push origin main --tags
```
5. **CI/CD triggers automatically**:
- Gitea CI builds Docker image on new tag
- Watchtower pulls and deploys to production
- Verify deployment: `curl http://192.168.86.149:8000/health`
---
## 2. FastAPI Architecture & Best Practices
*Reference: [FastAPI Best Practices](https://github.com/zhanymkanov/fastapi-best-practices)*
### 📂 Project Structure (Directory-based, NOT File-type based)
Do **not** group files by type (e.g., one huge `routers` folder). Group by **domain/module** inside a `src/` directory.
**Correct Structure:**
```text
src/
├── auth/
│ ├── router.py # Endpoints
│ ├── schemas.py # Pydantic models
│ ├── service.py # Business logic (CRUD, etc.)
│ ├── dependencies.py# Module-specific dependencies
│ └── config.py # Module-specific settings
├── posts/
│ ├── router.py
│ └── ...
└── main.py # App entry point
+281
View File
@@ -5,6 +5,287 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [1.11.0] - 2026-01-08
### Added
- **News Headlines API** - New endpoint for news ticker integration
- `GET /tools/news` - Fetch news headlines from user's volatile collection
- Returns headlines with title, description, source, and URL
- Data sourced from `volatile_{user}` Qdrant collection (news namespace)
- Uses `preferred_username` from OIDC, falls back to `default`
- News subdomain under tools (`src/domains/tools/news/`)
- `NewsHeadline` and `NewsResponse` Pydantic schemas
- `NewsService` for parsing news data from Qdrant
## [1.10.12] - 2026-01-08
### Fixed
- **Forecast data retrieval** - Pass raw_data to service instead of extracting wrong field
- Qdrant client was extracting `days` (integer 7) instead of `daily` (list)
- Now passes full raw_data for service to parse correctly
## [1.10.11] - 2026-01-08
### Fixed
- **Weather wind direction** - Convert integer degrees to cardinal direction string
- Scheduler stores wind_direction as degrees (e.g., 135)
- Schema expects string (e.g., "SE")
- Added `_degrees_to_cardinal()` conversion
## [1.10.10] - 2026-01-08
### Fixed
- **Environment data parsing** - Fix parsing of scheduler-generated Qdrant data
- Forecast: Check `daily` key first (scheduler stores day count in `days`, list in `daily`)
- Sun times: Use `sunrise_iso`/`sunset_iso` fields, handle time-only format fallback
- Sun times: Calculate daylight from `daylight_duration_seconds` or `daylight_hours`
- Air quality: Support `aqi_us`/`aqi_european` and `nitrogen_dioxide` field names
## [1.10.9] - 2026-01-08
### Fixed
- **Environment user ID cleanup** - Strip email domain from user identifier
- If `preferred_username` is an email, extract just the username part
- Ensures Qdrant collection name matches (e.g., `volatile_jpmschweitzer` not `volatile_jpmschweitzer@gmail.com`)
## [1.10.8] - 2026-01-08
### Fixed
- **OIDC audience validation** - python-jose requires string audience, not list
- Extract and validate audience from unverified claims first
- Use token's actual audience for JWT decode (after validating it's allowed)
- Fixes "audience must be a string or None" error
## [1.10.7] - 2026-01-08
### Added
- **Multi-issuer OIDC support** - Accept tokens from multiple OAuth providers
- Changed `oidc_issuer` (string) to `oidc_issuers` (list)
- Each issuer has its own JWKS endpoint, now cached per-issuer
- Validates token issuer against allowed list before fetching JWKS
- Supports tokens from: `core-api`, `tatlock-ui`, `tatlock` OAuth applications
- Completes fix for environment endpoint user resolution
### Removed
- Deprecated `src/config.py` - consolidated to `src/shared/config.py`
- Deprecated `src/security.py` - consolidated to `src/shared/security.py`
## [1.10.6] - 2026-01-08
### Fixed
- **OIDC audience mismatch** - Accept tokens from multiple clients
- Changed `oidc_audience` (string) to `oidc_audiences` (list)
- Now accepts tokens with audience: `core-api`, `tatlock-ui`, or `tatlock`
- Fixes environment endpoint returning "default" user instead of authenticated username
- Fixed main.py to use `oidc_audiences[0]` for Swagger UI OAuth client
### Changed
- Documentation cleanup in README
## [1.10.4] - 2026-01-07
### Removed
- **Ollama integration removed** - AI inference is no longer handled by this API
- Removed `src/models/ollama_client.py` and all Ollama-related configuration
- Removed `src/models/embeddings.py` and `src/models/embeddings_ollama.py`
- Removed model aliases and AI configuration from settings
- Health endpoints no longer check Ollama status
- Tests updated to reflect database-only health checks
### Changed
- Health check `/health/full` now only checks database connectivity
- Diagnostics endpoint simplified (removed Ollama component info)
## [1.10.3] - 2026-01-07
### Fixed
- **OIDC config not applied to domains module** - Both `src.auth.oidc` and `src.domains.auth.oidc` configs are now initialized
- Previously only `src.auth.oidc` was configured, leaving domains tools using hardcoded "local" user
- Environment endpoint now correctly uses authenticated user from OIDC token
## [1.10.2] - 2026-01-07
### Changed
- **Enhanced environment endpoint logging** - Added detailed user claim logging for debugging
- Logs both `preferred_username` and `sub` claims when resolving user
- Distinguishes between authenticated and unauthenticated requests
## [1.10.1] - 2026-01-06
### Fixed
- Fix OIDC import path in tools controller (`src.oidc.dependencies``src.auth.oidc`)
- Environment endpoint was returning `user: "local"` instead of authenticated username
- Caused queries to wrong Qdrant collection (`volatile_local` vs `volatile_{username}`)
## [1.10.0] - 2026-01-06
### Added
- **Environment Data API** - Qdrant-backed endpoint for weather, forecast, and sun position data
- `GET /tools/environment` - Fetch environment data from user's volatile collection
- Weather: current temperature, conditions, humidity, wind speed
- Forecast: multi-day outlook with high/low temperatures
- Sun times: sunrise, sunset, daylight duration
- Air quality: AQI and quality level (when available)
- Data sourced from `volatile_{user}` Qdrant collection
- Uses `preferred_username` from OIDC, falls back to `default`
- `qdrant-client` dependency for vector database access
- `QdrantReadClient` wrapper for read-only collection queries
- Comprehensive test suite for environment service parsing
## [1.9.4] - 2026-01-04
### Fixed
- Fix SQLAlchemy async lazy loading error for new users in `/auth/sync`
- Initialize `user.roles = []` and `user.preferences` to avoid greenlet error
- Was causing "MissingGreenlet: greenlet_spawn has not been called" on new user creation
## [1.9.3] - 2026-01-04
### Fixed
- Handle non-UUID `sub` claim in `/auth/sync` - Authentik JWT may return non-UUID subject identifiers
- Now derives deterministic UUID from sub string if direct parsing fails
## [1.9.2] - 2026-01-04
### Fixed
- `/auth/users/me` endpoint now supports both NPM forward auth headers AND JWT Bearer tokens
- Added `get_current_user_or_forward_auth()` combined auth dependency
- Fixes web authentication where NPM passes `X-authentik-*` headers instead of JWT
- Mobile/native clients continue to use JWT Bearer tokens as before
## [1.9.1] - 2026-01-03
### Fixed
- CORS configuration now uses explicit origins instead of `"*"`
- When `allow_credentials=True`, wildcard origins are rejected by browsers
- Added `home.schweitz.net`, `tatlock.schweitz.net`, and localhost origins
## [1.9.0] - 2026-01-03
### Added
- **NPM Forward Auth Support** - Web authentication via Nginx Proxy Manager forward auth
- `GET /auth/me` - Get current user from NPM forward auth headers (X-authentik-uid, X-authentik-email, etc.)
- Auto-creates user on first web login if not in database
- Syncs roles from NPM forward auth groups header
- `get_user_by_email` and `get_user_by_authentik_id` methods in AuthService
- Comprehensive tests for `/auth/me` endpoint
## [1.8.0] - 2026-01-03
### Added
- **Group-Role Mapping & Permissions** (Phase 3)
- Decoupled group-role architecture (groups from Authentik, roles admin-managed)
- Permission format: `domain.category:action` with action hierarchy
- `require_permission` and `require_any_permission` dependency factories
- Global admin override (`admin.general:admin`)
- **User Profile & API Keys** (Phase 4)
- `GET /auth/users/me` - Full user profile with roles and preferences
- `GET/PATCH /auth/users/me/preferences` - User preferences management
- `GET/POST/DELETE /auth/users/me/api-keys` - API key lifecycle
- API keys with `tak_` prefix, SHA-256 hashing, shown only once on creation
## [1.7.0] - 2026-01-03
### Added
- **System Stats API** - Host system resource monitoring for dashboard widgets
- `GET /tools/system/stats` - Real-time host system statistics
- CPU: usage percentage, core count, load averages
- Memory: usage percentage, total/used/available bytes
- Disks: all mounted filesystems with usage stats (auto-discovers mounts)
- Network: total bytes sent/received
- GPU/VRAM: NVIDIA GPU memory usage (via nvidia-smi if available)
- `psutil` dependency for cross-platform system metrics
## [1.6.1] - 2026-01-03
### Added
- `link_type` field to quick links for iframe vs new tab behavior
## [1.6.0] - 2026-01-03
### Added
- **Dashboard API** - Quick links and widgets management for Organizr-style dashboard
- `GET /dashboard/quick-links` - List quick links with category/visibility filtering
- `GET /dashboard/quick-links/{id}` - Get single quick link
- `POST /dashboard/quick-links` - Create quick link
- `PUT /dashboard/quick-links/{id}` - Update quick link
- `DELETE /dashboard/quick-links/{id}` - Delete quick link
- `POST /dashboard/quick-links/reorder` - Reorder quick links by position
- `GET /dashboard/widgets` - List dashboard widgets
- `GET /dashboard/widgets/{id}` - Get single widget
- `POST /dashboard/widgets` - Create widget
- `PUT /dashboard/widgets/{id}` - Update widget
- `DELETE /dashboard/widgets/{id}` - Delete widget
- Database migrations for `quick_links` and `dashboard_widgets` tables
- Static file controller for serving Organizr widgets (`/static/widgets`)
- Default local user authentication when OIDC is disabled
### Changed
- **Domain-based architecture** - Refactored codebase to domain-driven structure
- `src/domains/` - Domain modules (auth, dashboard, health, housekeeping, infrastructure, tools)
- `src/shared/` - Shared utilities (base, config, database, logging, security, clients)
- Test suite updated for new domain structure (285 tests passing)
## [1.5.0] - 2026-01-01
### Added
- **Groups Management** - Authentik group synchronization
- `GET /auth/groups` - List all groups with search and pagination
- `POST /auth/groups/sync-from-authentik` - Bulk sync groups from Authentik admin API
- Database model and migration for groups table
## [1.4.6] - 2026-01-01
### Changed
- Code cleanup: move inline `re` import to top of auth/service.py
## [1.4.5] - 2026-01-01
### Fixed
- Separate httpx and SQLAlchemy async contexts in bulk sync (fixes greenlet error)
## [1.4.4] - 2026-01-01
### Fixed
- Use `uuid` field instead of `pk` for Authentik user sync (pk is integer, uuid is proper UUID)
- Skip internal_service_account type users during bulk sync
## [1.4.3] - 2026-01-01
### Fixed
- Manually extract and send session cookies for Authentik flow auth (fixes cross-domain cookie handling)
## [1.4.2] - 2026-01-01 ## [1.4.2] - 2026-01-01
### Fixed ### Fixed
+177
View File
@@ -0,0 +1,177 @@
# CLAUDE.md — core-api
FastAPI service providing infrastructure management, home automation, and utility
endpoints for the homelab. Talks to Portainer, Nginx Proxy Manager, Home Assistant,
Postgres (via SQLAlchemy async + Alembic), Qdrant, and Authentik (OIDC). Deployed on
tower-of-joy at **:8083**.
## Ports — these differ, deliberately
| | Port | How |
|---|---|---|
| Local dev | **8788** | `./wakeup.sh`, uvicorn `--reload`, logs to `logs/server.log` |
| Production | **8083** | container; health at `http://192.168.86.149:8083/health` |
Testing `localhost:8083` on the dev box hits the *container*, not your reload server.
## Live contract
The live contract is always `http://localhost:8083/openapi.json` (62 paths, verified
2026-08-09) and human docs at `http://localhost:8083/docs` / `/redoc` — generated from
running code, so query it rather than inferring routes from source or from the README's
endpoint list, which can drift.
## Architecture
Domain-first layout under `src/domains/<name>/{controller,models,schemas,service}.py`
(auth, dashboard, health, housekeeping, infrastructure, static, tools). `src/main.py`
wires only `src.domains.*` — verify by reading its imports.
**Legacy top-level packages — "not in `main.py`" does not mean dead.** Routes are wired
only from `src.domains.*`, so grepping `main.py`'s imports looks like it settles which
packages are live. It does not. `main.py:55` calls `initialize_oidc()`, and that function
(`src/shared/security.py:21`) deliberately imports and configures **both** `src.auth.oidc`
and `src.domains.auth.oidc` — a function-body import, invisible to a grep of `main.py`.
That one call drags in `src/auth/`, `src/controllers/`, `src/db/`, `src/logging_config.py`
and `src/base_schema.py` at startup.
Three tiers, established by importing the app inside the container and reading
`sys.modules` (verified 2026-08-09):
| Tier | Packages |
|---|---|
| Serving routes | `src/domains/`, `src/shared/`, `src/service_groups/` |
| **Loaded and configured**, but serving no routes | `src/auth/`, `src/db/`, `src/controllers/`, `src/logging_config.py`, `src/base_schema.py` |
| Genuinely unreferenced | `src/agent/`, `src/api/`, `src/clients/`, `src/dns/`, `src/memory/`, `src/models/` |
`src/auth/` is the trap. Its `oidc_config` singleton is configured at every startup with
the real Authentik issuers — the log line `src.auth.oidc:configure` proves it — so a test
importing `src.auth.oidc` is exercising live, configured code, not a fossil. No `src/domains/*`
module depends on it, so it is configured defensively rather than used; that makes it a
deletion candidate, but a considered one, not obvious cleanup.
**Before deleting anything from `src/`, import the app and read `sys.modules`** rather than
grepping `main.py`. Function-body imports exist here specifically to dodge circular imports,
and they are exactly what a grep misses.
**That check has its own blind spot, so do not read the third tier as a delete list.** The
table above is a snapshot taken after a cold `import src.main` — it shows what *startup*
loads. A module imported inside a request handler would be absent from it while being
entirely live, and absence would then be a timing artifact rather than evidence of death.
This bit on webber, where a tool package imported from inside an agent method looked
unloaded and was serving every request. Nothing in core-api is currently known to work that
way, but that is the weaker claim — it means nobody has exercised the routes and re-checked,
not that nobody does it. Before deleting a third-tier package, drive the endpoints that
would plausibly load it and take the snapshot again.
Some tests (`test_auth_controller.py`, `test_oidc.py`, `test_npm_client.py`,
`test_portainer_client.py`, `test_static_controller.py`, `test_tools_controller.py`) import
from the top-level paths rather than `src.domains.*`. Which of those cover live code follows
the table above — `test_oidc.py` does; the client tests target the unreferenced tier. Not
cleaned up in this pass; flagged, not fixed.
Shared infra (config, database, logging, security/OIDC, external API clients) lives in
`src/shared/`.
Group new work by **domain, not by file type** — a single large `routers/` folder is
the thing to avoid. Reference: [FastAPI best practices](https://github.com/zhanymkanov/fastapi-best-practices).
## Database
SQLAlchemy 2.0 async + asyncpg, migrations via Alembic (`alembic/versions/`). Models
live under `src/domains/<name>/models.py` and must be imported in `alembic/env.py` to
register with `Base.metadata` — check that file when adding a new model or `alembic
revision --autogenerate` will silently miss it.
## Working here
**Plan, act, reflect.** Outline which files you will touch and the side effects before
writing. Change in small atomic steps. Afterwards, verify: did existing tests break, and
does the new behaviour have a test?
**Test locally first — the build-deploy loop is slow.** `./wakeup.sh` auto-reloads on
code changes (not on `requirements.txt` changes; restart the container/script after
adding a dependency). Deploy only when a feature is complete and tested.
Run tests through the venv explicitly, to avoid environment mismatch:
```bash
.venv/bin/python -m pytest tests/
# or, with coverage:
.venv/bin/python -m pytest --cov=src --cov-report=term-missing
```
Copy `.env.example` to `.env` and configure Portainer, NPM, Home Assistant, SearXNG,
Postgres, and Qdrant hosts/credentials.
No linter is configured in this repo (no ruff/flake8 config, none in `requirements.txt`
or `dev-requirements.txt`) — unlike some sibling repos, don't assume `ruff check` exists
here.
## CI
`.gitea/workflows/build.yml` is the only workflow: triggered on `v*` tag push, it
creates a Gitea release, builds and pushes the image, then pings Watchtower. There is
**no CI test/lint gate** — pytest only runs locally or on request. Verify tests pass
before tagging a release.
## Work tracking
Work lives in **pql**, not a markdown TODO. **This repo's vault is standalone** — its tickets
and its internal decisions live here in `.pql/` and `governance/`, and travel with a clone,
because `.pql/changelog/` is committed and replayed by the git hooks (workspace D-15). The databases are
gitignored and rebuildable with `pql plan rebuild`.
`pql` is **not** on the non-interactive `PATH` — invoke it as
`/home/jpmschweitzer/.local/bin/pql`. From inside this repo no `--vault` is needed: pql anchors
at the nearest `.git/` ancestor, which is this repo.
```bash
/home/jpmschweitzer/.local/bin/pql ticket list # this repo's open work
/home/jpmschweitzer/.local/bin/pql plan whatsnext # next unblocked item, with context
/home/jpmschweitzer/.local/bin/pql decisions list # this repo's own decisions
```
Stack-level decisions that constrain this service live in the **workspace** vault and need the
flag:
```bash
/home/jpmschweitzer/.local/bin/pql --vault /mnt/media/Projects decisions list --domain core-api
```
Note `ticket new --decision D-N` resolves ids within **one** vault, so a ticket here cannot link
to a workspace decision. Cite the id in the ticket body instead.
Do not add a TODO section to a markdown file.
## Git
- **History is linear — no merge commits.** Work on `main`, or a short-lived branch
that is fast-forwarded and deleted. (This repo's AGENTS.md previously mandated a
feature branch for every change; that rule was retired workspace-wide on 2026-08-08
and does not apply here anymore.)
- **Conventional Commits**: `feat:`, `fix:`, `refactor:`, `docs:`, `chore:`.
- **Atomic commits** — one logical change each.
- **Stage explicitly. Never `git add -A`** — it is denied by policy, and it sweeps in
whatever else is dirty, including secrets.
- Update `CHANGELOG.md` with every user-facing change, under `[Unreleased]` in `Added` /
`Changed` / `Fixed`.
## Releasing
Ask whether a deploy is wanted first — it is not automatic.
1. Bump the version in `pyproject.toml` (patch for fixes, minor for features).
2. Move `[Unreleased]` entries into a dated version section in `CHANGELOG.md`.
3. Stage the changed files by name, commit, tag `vX.Y.Z`, `git push origin main --tags`.
4. Gitea CI (`build.yml`) builds and pushes the image on the tag; Watchtower deploys it.
5. Verify: `curl http://192.168.86.149:8083/health`.
## Security
- OIDC authentication via Authentik, multi-issuer/multi-audience support.
- Admin endpoints require authentication when `OIDC_ENABLED=true`.
- README claims the container "runs as non-root user (uid 1000)" — **checked and
false**: the Dockerfile has no `USER` directive, so the container runs as root.
Not fixed here (out of scope for a docs normalization pass); flagging so it isn't
restated as fact.
+67
View File
@@ -0,0 +1,67 @@
# core-api — the repo's command surface (D-27).
#
# Paths resolve here rather than in callers (D-10). `python3` on this host is
# 3.8 and cannot parse these sources, and a bare `pytest` resolves only in a
# login shell — so both are named explicitly through the venv.
VENV := $(CURDIR)/.venv
PYTHON ?= python3.12
.DEFAULT_GOAL := help
.PHONY: help
help: ## Show this help
@grep -hE '^[a-z][a-z0-9_-]*:.*?## ' $(MAKEFILE_LIST) \
| awk 'BEGIN{FS=":.*?## "}{printf " \033[36m%-14s\033[0m %s\n", $$1, $$2}'
.PHONY: setup
setup: ## Create the venv, install dependencies, and prove the result is usable
$(PYTHON) -m venv .venv
$(VENV)/bin/pip install -r requirements.txt -r dev-requirements.txt
# Exit 0 from pip install is not evidence (D-24) — it is the step's job not
# to fail, so a broken result and a working one look identical from here.
# On 2026-08-09 the venv existed and pip had exited 0, but sqlalchemy was
# declared in requirements.txt and not installed; that surfaced as 11
# pytest collection errors that read as broken imports, not as a setup
# problem. `--collect-only` exercises every import the suite touches
# without running a single test, so it catches exactly that class of
# drift and stays cheap. `pip check` was considered too, but it only
# verifies the *installed* set's internal consistency against itself —
# it would not have caught this case, because sqlalchemy was still
# present as another package's transitive dependency even when dropped
# from requirements.txt. collect-only checks declared-vs-actually-usable
# directly, which is the axis that broke.
$(VENV)/bin/python -m pytest --collect-only tests/
.PHONY: test
test: ## Run the test suite
@test -x $(VENV)/bin/python || { echo "FAIL — no venv; run: make setup"; exit 69; }
$(VENV)/bin/python -m pytest tests/
# No `lint` target, deliberately. This repo configures no linter — no ruff or
# flake8 config, and neither in requirements. Per D-27 the name is reserved for
# repos that lint; an empty target here would report clean for something never
# run. Add the target when a linter is added, not before.
# git hands a hook a non-login shell, which never sees ~/.local/bin — where
# gitleaks lands. Without this the scan reports "not installed" on every push,
# which is a check that fails open (D-24).
export PATH := $(HOME)/.local/bin:/usr/local/bin:$(PATH)
.PHONY: secrets
secrets: ## Scan the commits about to be pushed for credentials
@ci/secrets.sh
# The call surface is identical in every repo; what it runs is not.
#
# `secrets` runs first, deliberately: it is the only failure here that cannot be
# undone by fixing it afterwards. A failed lint costs another commit; a pushed
# credential is cached and indexed whether or not it is later deleted.
#
# Some of these fail today, and are left wired anyway. The state was measured
# once and written down in T-56 rather than being worked around here — a gate
# quietly narrowed to what already passes is a gate that reports success for
# doing nothing, which is the failure this workspace keeps rediscovering.
.PHONY: pre-push
pre-push: secrets ## Everything the pre-push hook runs
@echo " -- not gated here yet: lint (no linter configured) and test (T-56)"
+6 -17
View File
@@ -19,8 +19,7 @@ Central API service providing infrastructure management, home automation, and ut
### Utilities ### Utilities
- **DNS Lookup**: Query DNS records (A, AAAA, MX, TXT, CNAME, NS, SOA, PTR) - **DNS Lookup**: Query DNS records (A, AAAA, MX, TXT, CNAME, NS, SOA, PTR)
- **Health Checks**: Comprehensive service health monitoring - **Health Checks**: Service health monitoring with database connectivity status
- **AI Metrics Proxy**: Forward metrics requests to Core-AI service
## Architecture ## Architecture
@@ -35,10 +34,8 @@ src/
├── clients/ ├── clients/
│ ├── homeassistant_client.py # Home Assistant REST client │ ├── homeassistant_client.py # Home Assistant REST client
│ ├── npm_client.py # Nginx Proxy Manager client │ ├── npm_client.py # Nginx Proxy Manager client
│ ├── ollama_client.py # Ollama LLM client
│ └── portainer_client.py # Portainer API client │ └── portainer_client.py # Portainer API client
├── controllers/ ├── controllers/
│ ├── ai_controller.py # AI metrics proxy
│ ├── health_controller.py # Health endpoints │ ├── health_controller.py # Health endpoints
│ ├── housekeeping_controller.py # Home automation endpoints │ ├── housekeeping_controller.py # Home automation endpoints
│ ├── infrastructure_controller.py # Infrastructure management │ ├── infrastructure_controller.py # Infrastructure management
@@ -88,9 +85,6 @@ src/
### Tools (`/tools`) ### Tools (`/tools`)
- `POST /tools/dns/lookup` - DNS record lookup - `POST /tools/dns/lookup` - DNS record lookup
### AI (`/ai`)
- `GET /ai/metrics` - Proxy to Core-AI metrics
## Development ## Development
### Requirements ### Requirements
@@ -103,10 +97,6 @@ src/
# Install dependencies # Install dependencies
pip install -r requirements.txt pip install -r requirements.txt
# Copy credentials template
cp src/credentials.example.py src/credentials.py
# Edit src/credentials.py with your values
# Run locally # Run locally
uvicorn src.main:app --reload --host 0.0.0.0 --port 8083 uvicorn src.main:app --reload --host 0.0.0.0 --port 8083
``` ```
@@ -169,10 +159,9 @@ docker run -p 8083:8083 core-code:latest
| `NPM_PASSWORD` | NPM admin password | - | | `NPM_PASSWORD` | NPM admin password | - |
| `HOMEASSISTANT_URL` | Home Assistant URL | `http://localhost:8123` | | `HOMEASSISTANT_URL` | Home Assistant URL | `http://localhost:8123` |
| `HOMEASSISTANT_TOKEN` | HA long-lived access token | - | | `HOMEASSISTANT_TOKEN` | HA long-lived access token | - |
| `OLLAMA_URL` | Ollama API URL | `http://localhost:11434` |
| `OIDC_ENABLED` | Enable OIDC auth | `false` | | `OIDC_ENABLED` | Enable OIDC auth | `false` |
| `OIDC_ISSUER` | OIDC issuer URL | - | | `OIDC_ISSUERS` | OIDC issuer URLs (comma-separated) | See config.py |
| `OIDC_AUDIENCE` | OIDC audience | - | | `OIDC_AUDIENCES` | OIDC audiences (comma-separated) | See config.py |
## API Documentation ## API Documentation
@@ -183,9 +172,9 @@ Once deployed, access documentation at:
## Health Checks ## Health Checks
- **Basic**: `GET /health` - Returns status and Ollama connection - **Basic**: `GET /health` - Fast liveness check for container orchestration
- **Full**: `GET /health/full` - Returns all component statuses (503 if unhealthy) - **Full**: `GET /health/full` - Returns database status (503 if unhealthy)
- **Diagnostics**: `GET /health/diagnostics` - Detailed service information - **Diagnostics**: `GET /health/diagnostics` - Service info and configuration
## Security ## Security
+4 -3
View File
@@ -13,11 +13,12 @@ from sqlalchemy.ext.asyncio import async_engine_from_config
from alembic import context from alembic import context
# Import our models and config # Import our models and config
from src.config import get_settings from src.shared.config import get_settings
from src.db.database import Base from src.shared.database import Base
# Import all models to ensure they're registered with Base.metadata # Import all models to ensure they're registered with Base.metadata
from src.db.models import User, Role, UserRole, UserPreferences, ApiKey # noqa: F401 from src.domains.auth.models import User, Role, UserRole, UserPreferences, ApiKey # noqa: F401
from src.domains.dashboard.models import QuickLink, DashboardWidget # noqa: F401
# Alembic Config object # Alembic Config object
config = context.config config = context.config
@@ -0,0 +1,40 @@
"""Create groups table
Revision ID: 002
Revises: 001
Create Date: 2026-01-01
Creates the groups table for syncing Authentik groups.
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql
# revision identifiers, used by Alembic.
revision: str = "002"
down_revision: Union[str, None] = "001"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Groups table
op.create_table(
"groups",
sa.Column("id", postgresql.UUID(as_uuid=True), primary_key=True),
sa.Column("authentik_id", postgresql.UUID(as_uuid=True), nullable=False, comment="Authentik group UUID"),
sa.Column("name", sa.String(255), nullable=False),
sa.Column("is_superuser", sa.Boolean(), nullable=False, server_default="false", comment="Whether members have superuser privileges"),
sa.Column("parent_name", sa.String(255), nullable=True, comment="Parent group name for hierarchy"),
sa.Column("member_count", sa.Integer(), nullable=False, server_default="0", comment="Number of users in this group"),
sa.Column("created_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False),
sa.Column("synced_at", sa.DateTime(timezone=True), server_default=sa.func.now(), nullable=False, comment="Last sync from Authentik"),
)
op.create_index("ix_groups_authentik_id", "groups", ["authentik_id"], unique=True)
op.create_index("ix_groups_name", "groups", ["name"], unique=True)
def downgrade() -> None:
op.drop_table("groups")
@@ -0,0 +1,71 @@
"""Create dashboard tables
Revision ID: 003
Revises: 002
Create Date: 2026-01-03
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = '003'
down_revision: Union[str, None] = '002'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
"""Create quick_links and dashboard_widgets tables."""
# Create quick_links table
op.create_table(
'quick_links',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('title', sa.String(length=100), nullable=False),
sa.Column('url', sa.String(length=500), nullable=False),
sa.Column('icon', sa.String(length=100), nullable=True),
sa.Column('description', sa.String(length=255), nullable=True),
sa.Column('category', sa.String(length=50), nullable=True),
sa.Column('user_id', sa.String(length=255), nullable=True),
sa.Column('position', sa.Integer(), nullable=True, default=0),
sa.Column('is_visible', sa.Boolean(), nullable=True, default=True),
sa.Column('color', sa.String(length=20), nullable=True),
sa.Column('background_color', sa.String(length=20), nullable=True),
sa.Column('created_at', sa.DateTime(), nullable=True),
sa.Column('updated_at', sa.DateTime(), nullable=True),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_quick_links_id'), 'quick_links', ['id'], unique=False)
op.create_index(op.f('ix_quick_links_user_id'), 'quick_links', ['user_id'], unique=False)
# Create dashboard_widgets table
op.create_table(
'dashboard_widgets',
sa.Column('id', sa.Integer(), nullable=False),
sa.Column('widget_type', sa.String(length=50), nullable=False),
sa.Column('user_id', sa.String(length=255), nullable=True),
sa.Column('position_x', sa.Integer(), nullable=True, default=0),
sa.Column('position_y', sa.Integer(), nullable=True, default=0),
sa.Column('width', sa.Integer(), nullable=True, default=1),
sa.Column('height', sa.Integer(), nullable=True, default=1),
sa.Column('config', sa.Text(), nullable=True),
sa.Column('is_visible', sa.Boolean(), nullable=True, default=True),
sa.Column('created_at', sa.DateTime(), nullable=True),
sa.Column('updated_at', sa.DateTime(), nullable=True),
sa.PrimaryKeyConstraint('id')
)
op.create_index(op.f('ix_dashboard_widgets_id'), 'dashboard_widgets', ['id'], unique=False)
op.create_index(op.f('ix_dashboard_widgets_user_id'), 'dashboard_widgets', ['user_id'], unique=False)
def downgrade() -> None:
"""Drop dashboard tables."""
op.drop_index(op.f('ix_dashboard_widgets_user_id'), table_name='dashboard_widgets')
op.drop_index(op.f('ix_dashboard_widgets_id'), table_name='dashboard_widgets')
op.drop_table('dashboard_widgets')
op.drop_index(op.f('ix_quick_links_user_id'), table_name='quick_links')
op.drop_index(op.f('ix_quick_links_id'), table_name='quick_links')
op.drop_table('quick_links')
@@ -0,0 +1,26 @@
"""add_link_type_to_quick_links
Revision ID: f0349c95aa5d
Revises: 003
Create Date: 2026-01-03 13:14:46.911770+00:00
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
# revision identifiers, used by Alembic.
revision: str = 'f0349c95aa5d'
down_revision: Union[str, None] = '003'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.add_column('quick_links', sa.Column('link_type', sa.String(length=20), nullable=True, server_default='iframe'))
# Update existing rows to have the default value
op.execute("UPDATE quick_links SET link_type = 'iframe' WHERE link_type IS NULL")
def downgrade() -> None:
op.drop_column('quick_links', 'link_type')
@@ -0,0 +1,141 @@
"""Add group_roles mapping and update role schema
Revision ID: 004
Revises: f0349c95aa5d
Create Date: 2026-01-03
Changes:
- Add category column to roles (default 'general')
- Drop authentik_group column from roles (decoupled architecture)
- Create user_groups association table
- Create group_roles association table
- Update role names from domain:action to domain.general:action
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
from sqlalchemy.dialects import postgresql
# revision identifiers, used by Alembic.
revision: str = "004"
down_revision: Union[str, None] = "f0349c95aa5d"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Add category column to roles
op.add_column(
"roles",
sa.Column(
"category",
sa.String(50),
nullable=False,
server_default="general",
comment="Permission category within domain (general for full access, or specific tool)",
),
)
# Update role names from domain:action to domain.general:action
op.execute(
"""
UPDATE roles
SET name = REPLACE(name, ':', '.general:')
WHERE name NOT LIKE '%.%:%'
"""
)
# Update the comment on the name column
op.alter_column(
"roles",
"name",
comment="Role name in format domain.category:action (e.g., control-room.general:admin)",
)
# Drop the authentik_group unique index first
op.drop_index("ix_roles_authentik_group", table_name="roles")
# Drop authentik_group column (no longer needed with group_roles mapping)
op.drop_column("roles", "authentik_group")
# Create user_groups association table
op.create_table(
"user_groups",
sa.Column(
"user_id",
postgresql.UUID(as_uuid=True),
sa.ForeignKey("users.id", ondelete="CASCADE"),
primary_key=True,
),
sa.Column(
"group_id",
postgresql.UUID(as_uuid=True),
sa.ForeignKey("groups.id", ondelete="CASCADE"),
primary_key=True,
),
)
# Create group_roles association table
op.create_table(
"group_roles",
sa.Column(
"group_id",
postgresql.UUID(as_uuid=True),
sa.ForeignKey("groups.id", ondelete="CASCADE"),
primary_key=True,
),
sa.Column(
"role_id",
postgresql.UUID(as_uuid=True),
sa.ForeignKey("roles.id", ondelete="CASCADE"),
primary_key=True,
),
)
def downgrade() -> None:
# Drop association tables
op.drop_table("group_roles")
op.drop_table("user_groups")
# Add back authentik_group column
op.add_column(
"roles",
sa.Column(
"authentik_group",
sa.String(255),
nullable=True,
comment="Corresponding Authentik group name",
),
)
# Restore authentik_group values from role names
op.execute(
"""
UPDATE roles
SET authentik_group = 'tatlock-' || REPLACE(REPLACE(name, '.general:', '-'), ':', '-')
"""
)
# Recreate the unique index
op.create_index("ix_roles_authentik_group", "roles", ["authentik_group"], unique=True)
# Revert role names from domain.general:action to domain:action
op.execute(
"""
UPDATE roles
SET name = REPLACE(name, '.general:', ':')
WHERE name LIKE '%.general:%'
"""
)
# Update the comment on the name column
op.alter_column(
"roles",
"name",
comment="Role name in format domain:action",
)
# Drop category column
op.drop_column("roles", "category")
Executable
+50
View File
@@ -0,0 +1,50 @@
#!/usr/bin/env bash
# Secret scan over the commits about to be pushed.
#
# Lives here rather than inside .githooks/pre-push so it can be read, run by
# hand (`make secrets`), and changed under review. A hook is a trigger; it is
# not a home for logic. Identical in every repo in this workspace (D-27).
set -euo pipefail
cd "$(git rev-parse --show-toplevel)"
# A non-login shell — which is what git gives a hook — skips /etc/profile.d
# and never sees ~/.local/bin, where the gitleaks release tarball lands.
# Without this the scan reports "not installed" on every push.
[ -d "$HOME/.local/bin" ] && PATH="$HOME/.local/bin:$PATH"
if ! command -v gitleaks >/dev/null 2>&1; then
echo "FAIL secrets — gitleaks not installed, so this check would be a no-op pretending to pass." >&2
echo " https://github.com/gitleaks/gitleaks/releases → ~/.local/bin/gitleaks" >&2
exit 1
fi
# Scan the outgoing range, not full history. History here carries findings
# that are settled — test fixtures and vendored third-party code — and a gate
# that fails on something unfixable gets bypassed within a week. What matters
# is what is about to leave this machine.
if upstream=$(git rev-parse --abbrev-ref --symbolic-full-name '@{u}' 2>/dev/null); then
range="$upstream..HEAD"
elif git rev-parse --verify --quiet origin/main >/dev/null; then
range="origin/main..HEAD"
else
range=""
fi
if [ -z "$range" ]; then
gitleaks dir . --redact --no-banner --exit-code 1 || {
echo "FAIL secrets — gitleaks found a credential in the working tree." >&2; exit 1; }
exit 0
fi
[ -n "$(git log --oneline "$range" 2>/dev/null)" ] || exit 0
gitleaks git . --log-opts="$range" --redact --no-banner --exit-code 1 >/dev/null 2>&1 || {
echo "FAIL secrets — gitleaks found a credential in the commits being pushed." >&2
echo " inspect (values redacted): gitleaks git . --log-opts=\"$range\" --redact" >&2
echo " then remove and rotate it, or suppress deliberately:" >&2
echo " inline '# gitleaks:allow <reason>'" >&2
echo " or add the fingerprint to .gitleaksignore WITH a reason" >&2
exit 1
}
echo " ok secrets"
+54
View File
@@ -0,0 +1,54 @@
# Decisions, Questions, Rejected
This directory holds structured planning records that pql parses
into pql.db. Each record is a `### [DQR]-N: Title` heading inside
a markdown file. Files live in three per-type subdirectories:
- `decisions/<domain>.md` — confirmed design decisions
- `questions/<domain>.md` — open questions that may resolve into
decisions or rejected proposals
- `rejected/<domain>.md` — rejected proposals (kept for the audit
trail)
The parser infers domain from the filename stem and record type
from the parent subdirectory.
D-records that propose implementation work link to `initiative`-type
tickets via `decision_ref`. Run `pql decisions show <id>
--with-tickets` to inspect implementation status.
## Recommended domains
Start with this canonical set; create files as records land in
each domain:
- **architecture** — structural commitments (storage, layering,
languages, libraries)
- **process** — team workflow (commits, branches, releases, reviews)
- **design** — user-facing surface (UX, UI, public APIs)
- **coding-conventions** — team-internal code shape (style, lint,
file layout)
- **testing** — quality strategy (coverage, layers, gates)
You might also want, project-permitting:
- `accessibility` — if you ship user-facing software
- `security` — if you handle user data or network surfaces
- `licensing` — if you release open-source or commercial
- `documentation` — if user-docs are non-trivial
- `deployment` — if shipping is non-trivial
- `performance` — if you have perf budgets / SLOs
<!-- pql:records (auto-generated; do not edit manually) -->
## Decisions
- _(none)_
## Open questions
- _(none)_
## Rejected
- _(none)_
+12
View File
@@ -0,0 +1,12 @@
proxy_buffers 8 16k;
proxy_buffer_size 32k;
# CORS headers for Flutter web
add_header Access-Control-Allow-Origin "https://home.schweitz.net" always;
add_header Access-Control-Allow-Credentials true always;
add_header Access-Control-Allow-Methods "GET, POST, PUT, DELETE, PATCH, OPTIONS" always;
add_header Access-Control-Allow-Headers "Content-Type, Authorization" always;
if ($request_method = OPTIONS) {
return 204;
}
+1 -1
View File
@@ -1,6 +1,6 @@
[project] [project]
name = "core-api" name = "core-api"
version = "1.4.2" version = "1.11.0"
description = "Core Code API - Infrastructure management and tools API" description = "Core Code API - Infrastructure management and tools API"
readme = "README.md" readme = "README.md"
requires-python = ">=3.12" requires-python = ">=3.12"
+4
View File
@@ -21,6 +21,7 @@ python-dotenv~=1.0.0
python-json-logger~=2.0.0 python-json-logger~=2.0.0
pytz~=2024.1 pytz~=2024.1
dnspython~=2.7.0 dnspython~=2.7.0
psutil~=6.1.0
# Authentication & Security # Authentication & Security
PyJWT[crypto]>=2.9.0 PyJWT[crypto]>=2.9.0
@@ -31,3 +32,6 @@ cryptography>=44.0.1 # CVE-2024-12797
sqlalchemy[asyncio]~=2.0.0 sqlalchemy[asyncio]~=2.0.0
asyncpg>=0.30.0 asyncpg>=0.30.0
alembic~=1.13.0 alembic~=1.13.0
# Vector Database
qdrant-client>=1.9.0
+1 -1
View File
@@ -1,6 +1,6 @@
""" """
Core Code API - OpenAPI-compatible functions for Open WebUI Core Code API - OpenAPI-compatible functions for Open WebUI
""" """
from src.shared.config import __version__
__version__ = "1.0.0"
__author__ = "Core Code Team" __author__ = "Core Code Team"
+133 -7
View File
@@ -11,8 +11,9 @@ from sqlalchemy.ext.asyncio import AsyncSession
from src.controllers.base import BaseController from src.controllers.base import BaseController
from src.logging_config import get_logger from src.logging_config import get_logger
from src.db import get_async_session from src.db import get_async_session
from src.auth.schemas import AuthSyncRequest, AuthSyncResponse, UsersListResponse, BulkSyncResultSchema from src.auth.schemas import AuthSyncRequest, AuthSyncResponse, UsersListResponse, BulkSyncResultSchema, GroupsListResponse
from src.auth.service import AuthService from src.auth.service import AuthService
from src.auth.oidc import get_forward_auth_user
logger = get_logger(__name__) logger = get_logger(__name__)
@@ -152,6 +153,69 @@ class AuthController(BaseController):
logger.error(f"Bulk sync failed: {e}") logger.error(f"Bulk sync failed: {e}")
raise HTTPException(status_code=401, detail=str(e)) raise HTTPException(status_code=401, detail=str(e))
@router.get(
"/groups",
summary="List all groups",
response_model=GroupsListResponse,
responses={
200: {"description": "List of groups"},
},
)
async def list_groups(
search: Optional[str] = Query(None, description="Search by group name"),
offset: int = Query(0, ge=0, description="Number of records to skip"),
limit: int = Query(50, ge=1, le=100, description="Maximum records to return"),
session: AsyncSession = Depends(get_async_session),
) -> GroupsListResponse:
"""
List all groups synced from Authentik
Returns paginated list of groups with their details.
Supports search filtering by name.
"""
service = AuthService(session)
items, total = await service.list_groups(
search=search,
offset=offset,
limit=limit,
)
return GroupsListResponse(items=items, total=total)
@router.post(
"/groups/sync-from-authentik",
summary="Bulk sync groups from Authentik",
response_model=BulkSyncResultSchema,
responses={
200: {"description": "Sync completed"},
401: {"description": "Authentik API credentials invalid"},
503: {"description": "Authentik service unavailable"},
},
)
async def sync_groups_from_authentik(
session: AsyncSession = Depends(get_async_session),
) -> BulkSyncResultSchema:
"""
Fetch all groups from Authentik and sync to local database
This endpoint uses the Authentik admin API to fetch all groups
and create/update them in the local database. Requires
AUTHENTIK_USERNAME and AUTHENTIK_PASSWORD to be configured.
Use this to populate groups or to re-sync after changes in Authentik.
"""
service = AuthService(session)
try:
result = await service.bulk_sync_groups_from_authentik()
logger.info(
f"Groups bulk sync completed: {result.created} created, "
f"{result.updated} updated, {result.failed} failed"
)
return result
except ValueError as e:
logger.error(f"Groups bulk sync failed: {e}")
raise HTTPException(status_code=401, detail=str(e))
@router.get( @router.get(
"/me", "/me",
summary="Get current user profile", summary="Get current user profile",
@@ -159,21 +223,83 @@ class AuthController(BaseController):
responses={ responses={
200: {"description": "User profile"}, 200: {"description": "User profile"},
401: {"description": "Not authenticated"}, 401: {"description": "Not authenticated"},
404: {"description": "User not found in database"},
}, },
) )
async def get_me( async def get_me(
forward_auth_user: Optional[dict] = Depends(get_forward_auth_user),
session: AsyncSession = Depends(get_async_session), session: AsyncSession = Depends(get_async_session),
) -> JSONResponse: ) -> AuthSyncResponse:
""" """
Get the current authenticated user's profile Get the current authenticated user's profile
Note: This endpoint requires a valid session or API key. Authentication is handled by NPM forward auth with Authentik.
For now, returns 501 Not Implemented until session management is added. The proxy sets X-authentik-* headers which this endpoint reads.
For internal/LAN access (no forward auth headers), returns 401.
Use POST /auth/sync with an OIDC token for mobile app authentication.
""" """
# TODO: Implement with get_current_user dependency # Require forward auth for this endpoint
if forward_auth_user is None:
raise HTTPException( raise HTTPException(
status_code=501, status_code=401,
detail="Not implemented - use /auth/sync with access token", detail="Authentication required - access via authenticated proxy or use /auth/sync",
)
service = AuthService(session)
# Try to find user by Authentik UID first, then by email
user = None
uid = forward_auth_user.get("uid")
if uid:
try:
import uuid
authentik_id = uuid.UUID(uid)
user = await service.get_user_by_authentik_id(authentik_id)
except (ValueError, TypeError):
pass # Invalid UUID, try email
if user is None:
email = forward_auth_user.get("email")
if email:
user = await service.get_user_by_email(email)
if user is None:
# User authenticated with Authentik but not synced to database yet
# This can happen on first login via web
logger.info(f"User {forward_auth_user.get('email')} not found, creating from forward auth")
# Create user from forward auth headers
from src.auth.schemas import TokenInfoSchema
token_info = TokenInfoSchema(
sub=forward_auth_user.get("uid", ""),
email=forward_auth_user.get("email", ""),
name=forward_auth_user.get("name"),
groups=forward_auth_user.get("groups", []),
)
try:
user, _ = await service.sync_user(token_info)
await service.sync_roles(user, token_info.groups)
await session.commit()
await session.refresh(user, ["preferences", "roles"])
except Exception as e:
logger.error(f"Failed to create user from forward auth: {e}")
raise HTTPException(
status_code=500,
detail="Failed to create user profile",
)
# Sync roles from current groups (in case they changed)
groups = forward_auth_user.get("groups", [])
roles = await service.sync_roles(user, groups)
await session.commit()
return AuthSyncResponse(
user=service.user_to_schema(user),
roles=service.roles_to_schema(roles),
preferences=service.preferences_to_schema(user.preferences),
is_new_user=False,
) )
return router return router
+60 -23
View File
@@ -22,29 +22,42 @@ class OIDCConfig:
def __init__(self): def __init__(self):
# These will be set from environment variables in config.py # These will be set from environment variables in config.py
self.enabled = False self.enabled = False
self.issuer = "" self.issuers: list[str] = []
self.audience = "" self.audiences: list[str] = []
self.jwks_uri = ""
def configure(self, enabled: bool, issuer: str, audience: str): def configure(self, enabled: bool, issuers: list[str], audiences: list[str]):
"""Configure OIDC settings""" """Configure OIDC settings"""
self.enabled = enabled self.enabled = enabled
self.issuer = issuer self.issuers = [iss.rstrip('/') for iss in issuers] # Normalize without trailing slash
self.audience = audience self.audiences = audiences
self.jwks_uri = f"{issuer.rstrip('/')}/jwks/" logger.info(f"OIDC configured: enabled={enabled}, issuers={self.issuers}, audiences={audiences}")
logger.info(f"OIDC configured: enabled={enabled}, issuer={issuer}")
def get_jwks_uri(self, issuer: str) -> str:
"""Get JWKS URI for a specific issuer"""
return f"{issuer.rstrip('/')}/jwks/"
def is_valid_issuer(self, issuer: str) -> bool:
"""Check if issuer is in the allowed list"""
normalized = issuer.rstrip('/')
return normalized in self.issuers
# Global OIDC config instance # Global OIDC config instance
oidc_config = OIDCConfig() oidc_config = OIDCConfig()
@lru_cache(maxsize=1) # Per-issuer JWKS cache
def get_jwks() -> Dict: _jwks_cache: Dict[str, Dict] = {}
"""
Fetch JSON Web Key Set (JWKS) from Authentik
Cached to avoid repeated requests. Cache is cleared on server restart.
def get_jwks_for_issuer(issuer: str) -> Dict:
"""
Fetch JSON Web Key Set (JWKS) for a specific issuer.
Cached per-issuer to avoid repeated requests. Cache is cleared on server restart.
Args:
issuer: The token issuer URL
Returns: Returns:
JWKS dictionary containing public keys for token verification JWKS dictionary containing public keys for token verification
@@ -55,15 +68,24 @@ def get_jwks() -> Dict:
if not oidc_config.enabled: if not oidc_config.enabled:
return {} return {}
normalized_issuer = issuer.rstrip('/')
# Return cached JWKS if available
if normalized_issuer in _jwks_cache:
return _jwks_cache[normalized_issuer]
jwks_uri = oidc_config.get_jwks_uri(normalized_issuer)
try: try:
logger.debug(f"Fetching JWKS from {oidc_config.jwks_uri}") logger.debug(f"Fetching JWKS from {jwks_uri}")
response = httpx.get(oidc_config.jwks_uri, timeout=10.0) response = httpx.get(jwks_uri, timeout=10.0)
response.raise_for_status() response.raise_for_status()
jwks = response.json() jwks = response.json()
logger.info(f"JWKS fetched successfully ({len(jwks.get('keys', []))} keys)") logger.info(f"JWKS fetched successfully for {normalized_issuer} ({len(jwks.get('keys', []))} keys)")
_jwks_cache[normalized_issuer] = jwks
return jwks return jwks
except Exception as e: except Exception as e:
logger.error(f"Failed to fetch JWKS: {e}") logger.error(f"Failed to fetch JWKS from {jwks_uri}: {e}")
raise HTTPException( raise HTTPException(
status_code=503, status_code=503,
detail="Authentication service unavailable" detail="Authentication service unavailable"
@@ -109,6 +131,21 @@ async def get_current_user(
token = credentials.credentials token = credentials.credentials
try: try:
# First, extract issuer and audience from unverified claims
unverified_claims = jwt.get_unverified_claims(token)
token_issuer = unverified_claims.get("iss", "")
token_audience = unverified_claims.get("aud", "")
# Validate issuer is in our allowed list
if not oidc_config.is_valid_issuer(token_issuer):
logger.warning(f"Invalid token issuer: {token_issuer}")
raise HTTPException(status_code=401, detail="Invalid token issuer")
# Validate audience is in our allowed list
if token_audience not in oidc_config.audiences:
logger.warning(f"Invalid token audience: {token_audience}")
raise HTTPException(status_code=401, detail="Invalid token audience")
# Decode token header to get key ID # Decode token header to get key ID
unverified_header = jwt.get_unverified_header(token) unverified_header = jwt.get_unverified_header(token)
kid = unverified_header.get("kid") kid = unverified_header.get("kid")
@@ -116,8 +153,8 @@ async def get_current_user(
if not kid: if not kid:
raise HTTPException(status_code=401, detail="Invalid token format") raise HTTPException(status_code=401, detail="Invalid token format")
# Find matching key in JWKS # Find matching key in JWKS for this specific issuer
jwks = get_jwks() jwks = get_jwks_for_issuer(token_issuer)
rsa_key = None rsa_key = None
for key in jwks.get("keys", []): for key in jwks.get("keys", []):
@@ -129,17 +166,17 @@ async def get_current_user(
logger.warning(f"No matching key found for kid: {kid}") logger.warning(f"No matching key found for kid: {kid}")
raise HTTPException(status_code=401, detail="Invalid token key") raise HTTPException(status_code=401, detail="Invalid token key")
# Verify and decode token # Verify and decode token using the token's actual issuer and audience
payload = jwt.decode( payload = jwt.decode(
token, token,
rsa_key, rsa_key,
algorithms=["RS256"], algorithms=["RS256"],
audience=oidc_config.audience, audience=token_audience, # Use the token's audience (already validated)
issuer=oidc_config.issuer, issuer=token_issuer, # Use the token's issuer (already validated)
) )
user_email = payload.get("email", "unknown") user_email = payload.get("email", "unknown")
logger.info(f"Authenticated user: {user_email}") logger.info(f"Authenticated user: {user_email} (issuer: {token_issuer})")
return payload return payload
+19
View File
@@ -108,3 +108,22 @@ class BulkSyncResultSchema(BaseSchema):
failed: int = Field(..., description="Number of users that failed to sync") failed: int = Field(..., description="Number of users that failed to sync")
total_in_authentik: int = Field(..., description="Total users in Authentik") total_in_authentik: int = Field(..., description="Total users in Authentik")
errors: list[str] = Field(default_factory=list, description="Error messages for failed syncs") errors: list[str] = Field(default_factory=list, description="Error messages for failed syncs")
class GroupListItemSchema(BaseSchema):
"""Group item for list display"""
id: uuid.UUID = Field(..., description="Internal group ID")
authentik_id: uuid.UUID = Field(..., description="Authentik group ID")
name: str = Field(..., description="Group name")
is_superuser: bool = Field(default=False, description="Whether group has superuser privileges")
parent_name: Optional[str] = Field(None, description="Parent group name")
member_count: int = Field(default=0, description="Number of users in this group")
synced_at: datetime = Field(..., description="Last sync timestamp")
class GroupsListResponse(BaseSchema):
"""Response from GET /auth/groups"""
items: list[GroupListItemSchema] = Field(..., description="List of groups")
total: int = Field(..., description="Total count of groups")
+249 -35
View File
@@ -3,6 +3,7 @@ Authentication Service
Business logic for user synchronization from Authentik. Business logic for user synchronization from Authentik.
""" """
import re
import uuid import uuid
from datetime import datetime, timezone from datetime import datetime, timezone
from typing import Optional from typing import Optional
@@ -12,10 +13,10 @@ from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload from sqlalchemy.orm import selectinload
from src.config import get_settings from src.shared.config import get_settings
from src.logging_config import get_logger from src.logging_config import get_logger
from src.db.models import User, Role, UserPreferences from src.db.models import User, Role, UserPreferences, Group
from src.auth.schemas import TokenInfoSchema, UserSchema, RoleSchema, UserPreferencesSchema, UserListItemSchema, BulkSyncResultSchema from src.auth.schemas import TokenInfoSchema, UserSchema, RoleSchema, UserPreferencesSchema, UserListItemSchema, BulkSyncResultSchema, GroupListItemSchema
logger = get_logger(__name__) logger = get_logger(__name__)
settings = get_settings() settings = get_settings()
@@ -85,6 +86,42 @@ class AuthService:
logger.error(f"Authentik userinfo request error: {e}") logger.error(f"Authentik userinfo request error: {e}")
raise ValueError("Authentication service unavailable") raise ValueError("Authentication service unavailable")
async def get_user_by_email(self, email: str) -> Optional[User]:
"""
Get user by email address
Args:
email: User email address
Returns:
User if found, None otherwise
"""
stmt = (
select(User)
.options(selectinload(User.roles), selectinload(User.preferences))
.where(User.email == email)
)
result = await self.session.execute(stmt)
return result.scalar_one_or_none()
async def get_user_by_authentik_id(self, authentik_id: uuid.UUID) -> Optional[User]:
"""
Get user by Authentik UUID
Args:
authentik_id: Authentik user UUID
Returns:
User if found, None otherwise
"""
stmt = (
select(User)
.options(selectinload(User.roles), selectinload(User.preferences))
.where(User.authentik_id == authentik_id)
)
result = await self.session.execute(stmt)
return result.scalar_one_or_none()
async def sync_user(self, token_info: TokenInfoSchema) -> tuple[User, bool]: async def sync_user(self, token_info: TokenInfoSchema) -> tuple[User, bool]:
""" """
Create or update user from OIDC token info Create or update user from OIDC token info
@@ -249,24 +286,29 @@ class AuthService:
return items, total return items, total
def _get_csrf_token(self, client: httpx.AsyncClient) -> str: def _extract_cookie(self, headers: httpx.Headers, cookie_name: str) -> str:
"""Extract CSRF token from cookies""" """Extract a specific cookie value from Set-Cookie headers"""
for cookie in client.cookies.jar: for header in headers.get_list('set-cookie'):
if cookie.name == "authentik_csrf": if header.startswith(f'{cookie_name}='):
return cookie.value match = re.match(rf'{cookie_name}=([^;]+)', header)
if match:
return match.group(1)
return "" return ""
async def _authentik_session_login(self, client: httpx.AsyncClient) -> None: async def _authentik_session_login(self, client: httpx.AsyncClient) -> str:
""" """
Authenticate with Authentik using the flow API to establish a session Authenticate with Authentik using the flow API to establish a session
Authentik's flow API requires: Authentik's flow API requires:
1. Cookie persistence between requests 1. Cookie persistence between requests (manually handled due to domain restrictions)
2. X-authentik-CSRF header set to the authentik_csrf cookie value 2. X-authentik-CSRF header set to the authentik_csrf cookie value
3. Multi-stage flow handling (identification -> password -> done) 3. Multi-stage flow handling (identification -> password -> done)
Args: Args:
client: httpx client with cookie persistence client: httpx client
Returns:
Session cookie value for subsequent API calls
Raises: Raises:
ValueError: If authentication fails ValueError: If authentication fails
@@ -278,55 +320,66 @@ class AuthService:
resp.raise_for_status() resp.raise_for_status()
data = resp.json() data = resp.json()
logger.debug(f"Flow initial response: component={data.get('component')}, type={data.get('type')}") # Extract cookies manually from Set-Cookie headers (bypasses domain restrictions)
session_cookie = self._extract_cookie(resp.headers, "authentik_session")
csrf_cookie = self._extract_cookie(resp.headers, "authentik_csrf")
# Get CSRF token for subsequent requests logger.debug(f"Flow initial: component={data.get('component')}, session={bool(session_cookie)}, csrf={bool(csrf_cookie)}")
csrf_token = self._get_csrf_token(client)
logger.debug(f"CSRF token obtained: {bool(csrf_token)}")
# Build headers with CSRF token # Build headers with manual cookie and CSRF token
headers = { def build_headers():
hdrs = {
"Accept": "application/json", "Accept": "application/json",
"Content-Type": "application/json", "Content-Type": "application/json",
"Cookie": f"authentik_session={session_cookie}",
} }
if csrf_token: if csrf_cookie:
headers["X-authentik-CSRF"] = csrf_token hdrs["Cookie"] += f"; authentik_csrf={csrf_cookie}"
hdrs["X-authentik-CSRF"] = csrf_cookie
return hdrs
# Step 2: Handle identification stage - submit username # Step 2: Handle identification stage - submit username
if data.get("component") == "ak-stage-identification": if data.get("component") == "ak-stage-identification":
resp = await client.post( resp = await client.post(
flow_url, flow_url,
json={"uid_field": settings.authentik_username}, json={"uid_field": settings.authentik_username},
headers=headers, headers=build_headers(),
) )
resp.raise_for_status() resp.raise_for_status()
data = resp.json() data = resp.json()
logger.debug(f"After username: component={data.get('component')}, type={data.get('type')}")
# Update CSRF token (might change between stages) # Update session cookie if new one received
csrf_token = self._get_csrf_token(client) new_session = self._extract_cookie(resp.headers, "authentik_session")
if csrf_token: if new_session:
headers["X-authentik-CSRF"] = csrf_token session_cookie = new_session
logger.debug(f"After username: component={data.get('component')}")
# Step 3: Handle password stage if required # Step 3: Handle password stage if required
if data.get("component") == "ak-stage-password": if data.get("component") == "ak-stage-password":
resp = await client.post( resp = await client.post(
flow_url, flow_url,
json={"password": settings.authentik_password}, json={"password": settings.authentik_password},
headers=headers, headers=build_headers(),
) )
resp.raise_for_status() resp.raise_for_status()
data = resp.json() data = resp.json()
logger.debug(f"After password: component={data.get('component')}, type={data.get('type')}")
# Update session cookie if new one received
new_session = self._extract_cookie(resp.headers, "authentik_session")
if new_session:
session_cookie = new_session
logger.debug(f"After password: component={data.get('component')}")
# Check for access denied # Check for access denied
if data.get("component") == "ak-stage-access-denied": if data.get("component") == "ak-stage-access-denied":
raise ValueError("Authentik authentication failed: access denied") raise ValueError("Authentik authentication failed: access denied")
# Check for redirect (successful auth) # Check for redirect (successful auth)
if data.get("type") == "redirect" or data.get("to"): if data.get("component") == "xak-flow-redirect" or data.get("to"):
logger.info("Successfully authenticated with Authentik via flow") logger.info("Successfully authenticated with Authentik via flow")
return return session_cookie
# If we're still in identification stage, the username might be wrong # If we're still in identification stage, the username might be wrong
if data.get("component") == "ak-stage-identification": if data.get("component") == "ak-stage-identification":
@@ -334,6 +387,7 @@ class AuthService:
raise ValueError(f"Authentication stuck at identification stage: {response_errors}") raise ValueError(f"Authentication stuck at identification stage: {response_errors}")
logger.info(f"Authentik flow completed with component: {data.get('component')}") logger.info(f"Authentik flow completed with component: {data.get('component')}")
return session_cookie
async def bulk_sync_from_authentik(self) -> BulkSyncResultSchema: async def bulk_sync_from_authentik(self) -> BulkSyncResultSchema:
""" """
@@ -351,16 +405,21 @@ class AuthService:
errors = [] errors = []
total_in_authentik = 0 total_in_authentik = 0
# Step 1: Fetch all user data from Authentik API
authentik_users = []
try: try:
async with httpx.AsyncClient(timeout=30.0, follow_redirects=True) as client: async with httpx.AsyncClient(timeout=30.0, follow_redirects=True) as client:
# Authenticate with Authentik to get session # Authenticate with Authentik to get session cookie
await self._authentik_session_login(client) session_cookie = await self._authentik_session_login(client)
# Fetch users from Authentik admin API using session # Fetch users from Authentik admin API using session cookie
response = await client.get( response = await client.get(
f"{settings.authentik_url}/api/v3/core/users/", f"{settings.authentik_url}/api/v3/core/users/",
params={"page_size": 500}, params={"page_size": 500},
headers={"Accept": "application/json"}, headers={
"Accept": "application/json",
"Cookie": f"authentik_session={session_cookie}",
},
) )
if response.status_code == 401: if response.status_code == 401:
@@ -372,16 +431,22 @@ class AuthService:
authentik_users = data.get("results", []) authentik_users = data.get("results", [])
total_in_authentik = data.get("pagination", {}).get("count", len(authentik_users)) total_in_authentik = data.get("pagination", {}).get("count", len(authentik_users))
except httpx.HTTPStatusError as e:
raise ValueError(f"Authentik API error: {e.response.status_code}")
except httpx.RequestError as e:
raise ValueError(f"Failed to connect to Authentik: {str(e)}")
# Step 2: Sync users to database (outside of httpx context to avoid greenlet issues)
for auth_user in authentik_users: for auth_user in authentik_users:
try: try:
# Skip service accounts and inactive users # Skip service accounts and inactive users
if auth_user.get("type") == "service_account": if auth_user.get("type") in ("service_account", "internal_service_account"):
continue continue
if not auth_user.get("is_active", True): if not auth_user.get("is_active", True):
continue continue
# Extract user data from Authentik # Extract user data from Authentik
authentik_id = uuid.UUID(auth_user["pk"]) authentik_id = uuid.UUID(auth_user["uuid"])
email = auth_user.get("email") or f"{auth_user['username']}@local" email = auth_user.get("email") or f"{auth_user['username']}@local"
name = auth_user.get("name") or auth_user.get("username", "Unknown") name = auth_user.get("name") or auth_user.get("username", "Unknown")
avatar_url = auth_user.get("avatar") avatar_url = auth_user.get("avatar")
@@ -433,11 +498,160 @@ class AuthService:
# Commit all changes # Commit all changes
await self.session.commit() await self.session.commit()
return BulkSyncResultSchema(
created=created,
updated=updated,
failed=failed,
total_in_authentik=total_in_authentik,
errors=errors,
)
async def list_groups(
self,
search: Optional[str] = None,
offset: int = 0,
limit: int = 50,
) -> tuple[list[GroupListItemSchema], int]:
"""
List all groups with optional search and pagination
Args:
search: Optional search query (matches name)
offset: Number of records to skip
limit: Maximum number of records to return
Returns:
Tuple of (list of group schemas, total count)
"""
from sqlalchemy import func
# Base query
base_query = select(Group)
# Apply search filter if provided
if search:
search_filter = f"%{search}%"
base_query = base_query.where(Group.name.ilike(search_filter))
# Get total count
count_query = select(func.count()).select_from(base_query.subquery())
total_result = await self.session.execute(count_query)
total = total_result.scalar() or 0
# Apply pagination and ordering
query = base_query.order_by(Group.name).offset(offset).limit(limit)
result = await self.session.execute(query)
groups = list(result.scalars().all())
# Convert to schemas
items = [
GroupListItemSchema(
id=group.id,
authentik_id=group.authentik_id,
name=group.name,
is_superuser=group.is_superuser,
parent_name=group.parent_name,
member_count=group.member_count,
synced_at=group.synced_at,
)
for group in groups
]
return items, total
async def bulk_sync_groups_from_authentik(self) -> BulkSyncResultSchema:
"""
Fetch all groups from Authentik admin API and sync to local database
Returns:
BulkSyncResultSchema with counts of created/updated/failed groups
"""
if not settings.authentik_username or not settings.authentik_password:
raise ValueError("AUTHENTIK_USERNAME and AUTHENTIK_PASSWORD must be configured")
created = 0
updated = 0
failed = 0
errors = []
total_in_authentik = 0
# Step 1: Fetch all group data from Authentik API
authentik_groups = []
try:
async with httpx.AsyncClient(timeout=30.0, follow_redirects=True) as client:
# Authenticate with Authentik to get session cookie
session_cookie = await self._authentik_session_login(client)
# Fetch groups from Authentik admin API using session cookie
response = await client.get(
f"{settings.authentik_url}/api/v3/core/groups/",
params={"page_size": 500},
headers={
"Accept": "application/json",
"Cookie": f"authentik_session={session_cookie}",
},
)
if response.status_code == 401:
raise ValueError("Authentik API token is invalid or expired")
response.raise_for_status()
data = response.json()
authentik_groups = data.get("results", [])
total_in_authentik = data.get("pagination", {}).get("count", len(authentik_groups))
except httpx.HTTPStatusError as e: except httpx.HTTPStatusError as e:
raise ValueError(f"Authentik API error: {e.response.status_code}") raise ValueError(f"Authentik API error: {e.response.status_code}")
except httpx.RequestError as e: except httpx.RequestError as e:
raise ValueError(f"Failed to connect to Authentik: {str(e)}") raise ValueError(f"Failed to connect to Authentik: {str(e)}")
# Step 2: Sync groups to database (outside of httpx context to avoid greenlet issues)
for auth_group in authentik_groups:
try:
# Extract group data from Authentik
authentik_id = uuid.UUID(auth_group["pk"])
name = auth_group.get("name", "Unknown")
is_superuser = auth_group.get("is_superuser", False)
parent_name = auth_group.get("parent_name")
# users field contains list of user PKs
member_count = len(auth_group.get("users", []))
# Check if group exists
stmt = select(Group).where(Group.authentik_id == authentik_id)
result = await self.session.execute(stmt)
group = result.scalar_one_or_none()
if group is None:
# Create new group
group = Group(
authentik_id=authentik_id,
name=name,
is_superuser=is_superuser,
parent_name=parent_name,
member_count=member_count,
)
self.session.add(group)
created += 1
logger.info(f"Created group from Authentik: {name}")
else:
# Update existing group
group.name = name
group.is_superuser = is_superuser
group.parent_name = parent_name
group.member_count = member_count
updated += 1
logger.info(f"Updated group from Authentik: {name}")
except Exception as e:
failed += 1
error_msg = f"Failed to sync group {auth_group.get('name', 'unknown')}: {str(e)}"
errors.append(error_msg)
logger.warning(error_msg)
# Commit all changes
await self.session.commit()
return BulkSyncResultSchema( return BulkSyncResultSchema(
created=created, created=created,
updated=updated, updated=updated,
+1 -1
View File
@@ -10,7 +10,7 @@ import json
from typing import Optional, Dict, List, Any from typing import Optional, Dict, List, Any
from datetime import datetime, timedelta, timezone from datetime import datetime, timedelta, timezone
from src.logging_config import get_logger from src.logging_config import get_logger
from src.config import get_settings from src.shared.config import get_settings
logger = get_logger(__name__) logger = get_logger(__name__)
settings = get_settings() settings = get_settings()
+1 -1
View File
@@ -7,7 +7,7 @@ import httpx
from typing import Optional, Dict, List, Any from typing import Optional, Dict, List, Any
from datetime import datetime, timedelta from datetime import datetime, timedelta
from src.logging_config import get_logger from src.logging_config import get_logger
from src.config import get_settings from src.shared.config import get_settings
logger = get_logger(__name__) logger = get_logger(__name__)
settings = get_settings() settings = get_settings()
+1 -1
View File
@@ -8,7 +8,7 @@ import httpx
import json import json
from typing import Optional, Dict, List, Any from typing import Optional, Dict, List, Any
from src.logging_config import get_logger from src.logging_config import get_logger
from src.config import get_settings from src.shared.config import get_settings
logger = get_logger(__name__) logger = get_logger(__name__)
settings = get_settings() settings = get_settings()
-160
View File
@@ -1,160 +0,0 @@
"""
Global configuration for Core Code API
All configuration is loaded from environment variables or .env file.
See .env.example for available settings.
"""
import tomllib
from pathlib import Path
from pydantic_settings import BaseSettings
from functools import lru_cache
def _get_version_from_pyproject() -> str:
"""Load version from pyproject.toml"""
pyproject_path = Path(__file__).parent.parent / "pyproject.toml"
try:
with open(pyproject_path, "rb") as f:
data = tomllib.load(f)
return data.get("project", {}).get("version", "0.0.0")
except FileNotFoundError:
return "0.0.0"
__version__ = _get_version_from_pyproject()
class Settings(BaseSettings):
"""Global application settings"""
# Application
app_name: str = "Core Code API"
app_version: str = __version__
debug: bool = False
# Server
host: str = "0.0.0.0"
port: int = 8083
# CORS
cors_origins: list[str] = ["*"]
cors_credentials: bool = True
cors_methods: list[str] = ["*"]
cors_headers: list[str] = ["*"]
# Logging
log_level: str = "DEBUG"
# Ollama Configuration (for AI orchestration)
ollama_base_url: str # Required - set OLLAMA_BASE_URL in .env
ollama_timeout: int = 300 # 5 minutes
# Model Configuration
default_model: str = "mistral-nemo-large:latest"
agent_model: str = "mistral-nemo-large:latest" # Must support tool calling with ADK (~4GB VRAM)
code_models: str = "mistral-nemo-large:latest"
# Previous config (gemma3:12b used ~10GB VRAM)
# default_model: str = "gemma3:12b"
# agent_model: str = "gemma3:12b"
# System Prompt Variant (for A/B testing)
# Options: v1_verbose, v2_concise, v3_imperative, v4_minimal, v4_gemini_suggestion, v5_adk_optimized, v7_adk_best_practice, v8_holistic
system_prompt_variant: str = "v8_holistic"
# Agent Configuration
agent_fallback_enabled: bool = True
# Model Aliases (OpenAI → Local)
alias_gpt35: str = "gemma:7b"
alias_gpt4: str = "mistral:7b"
alias_gpt4_turbo: str = "mixtral:8x7b"
alias_gpt4_code: str = "codestral:latest"
# Memory Configuration
memory_tier1_max_turns: int = 10
memory_consolidation_threshold: int = 10
# Qdrant Configuration
qdrant_host: str = "qdrant"
qdrant_port: int = 6333
qdrant_collection_conversations: str = "core_api_conversations"
qdrant_collection_documents: str = "core_api_documents"
qdrant_collection_user_facts: str = "core_api_user_facts"
# Embeddings (using Ollama - no local models needed)
embedding_model: str = "nomic-embed-text" # Ollama embedding model
embedding_dimension: int = 768 # nomic-embed-text dimension
embedding_batch_size: int = 32
# Search Configuration
search_provider: str = "searxng"
searxng_url: str # Required - set SEARXNG_URL in .env
# Infrastructure Management (Portainer)
portainer_url: str # Required - set PORTAINER_URL in .env
portainer_api_key: str # Required - set PORTAINER_API_KEY in .env
# Infrastructure Management (Nginx Proxy Manager)
npm_url: str # Required - set NPM_URL in .env
npm_email: str # Required - set NPM_EMAIL in .env
npm_password: str # Required - set NPM_PASSWORD in .env
# Home Assistant Configuration
homeassistant_url: str # Required - set HOMEASSISTANT_URL in .env
homeassistant_token: str # Required - set HOMEASSISTANT_TOKEN in .env
homeassistant_timeout: int = 30
# PostgreSQL Database
postgres_host: str # Required - set POSTGRES_HOST in .env (e.g., localhost:5432)
postgres_user: str = "core_api"
postgres_password: str # Required - set POSTGRES_PASSWORD in .env
postgres_database: str = "core_api"
@property
def database_url(self) -> str:
"""Construct database URL from components"""
return f"postgresql://{self.postgres_user}:{self.postgres_password}@{self.postgres_host}/{self.postgres_database}"
# OIDC Authentication (Authentik)
oidc_enabled: bool = False # Set to True to require authentication
oidc_issuer: str = "https://auth.schweitz.net/application/o/core-api/"
oidc_audience: str = "core-api"
# Authentik API (for token validation and user management)
# Must use domain name (not IP) when AUTHENTIK_COOKIE_DOMAIN is set
authentik_url: str = "https://auth.schweitz.net" # Authentik base URL
authentik_username: str = "" # Admin username for API access (AUTHENTIK_USERNAME env var)
authentik_password: str = "" # Admin password for API access (AUTHENTIK_PASSWORD env var)
@property
def model_aliases(self) -> dict:
"""Computed property for model aliases"""
return {
"gpt-3.5-turbo": self.alias_gpt35,
"gpt-4": self.alias_gpt4,
"gpt-4-turbo": self.alias_gpt4_turbo,
"gpt-4-code": self.alias_gpt4_code,
}
def get_lightweight_models(self) -> list[str]:
"""Parse comma-separated lightweight models"""
return [m.strip().strip('"').strip("'") for m in self.lightweight_models.split(",") if m.strip()]
def get_heavy_models(self) -> list[str]:
"""Parse comma-separated heavy models"""
return [m.strip().strip('"').strip("'") for m in self.heavy_models.split(",") if m.strip()]
def get_code_models(self) -> list[str]:
"""Parse comma-separated code models"""
return [m.strip().strip('"').strip("'") for m in self.code_models.split(",") if m.strip()]
class Config:
env_file = ".env"
case_sensitive = False
extra = "ignore" # Ignore extra env vars not defined in Settings
@lru_cache()
def get_settings() -> Settings:
"""Cached settings instance"""
return Settings()
+13 -96
View File
@@ -7,13 +7,11 @@ from fastapi import APIRouter, Response
from fastapi.responses import JSONResponse from fastapi.responses import JSONResponse
from src.controllers.base import BaseController from src.controllers.base import BaseController
from src.config import get_settings from src.shared.config import get_settings
from src.logging_config import get_logger from src.logging_config import get_logger
from src.models.ollama_client import get_ollama_client
from src.db import get_database from src.db import get_database
logger = get_logger(__name__) logger = get_logger(__name__)
@@ -73,63 +71,18 @@ class HealthController(BaseController):
@router.get( @router.get(
"/health/full", "/health/full",
summary="Fast health check for Docker", summary="Full health check with database",
) )
async def full_health_check(response: Response): async def full_health_check(response: Response):
""" """
Fast health check for container orchestration (Docker/K8s). Health check including database connectivity.
Checks component availability WITHOUT running expensive operations. Returns 200 OK if database is available, otherwise 503.
Returns 200 OK if all components are available, otherwise 503.
For detailed diagnostics, use /health/diagnostics instead.
""" """
import time import time
start_time = time.time() start_time = time.time()
# Check 1: Ollama connection + verify agent model is available # Check database connection
ollama_client = get_ollama_client()
ollama_healthy = False
ollama_error = None
model_available = False
try:
# Ping Ollama
ollama_healthy = await ollama_client.health_check()
# Verify the agent model is pulled and check what's currently loaded
models_info = {}
if ollama_healthy:
try:
models_response = await ollama_client.list_models()
available_models = [m.get('name', '') for m in models_response.get('models', [])]
model_available = settings.agent_model in available_models
# Get info about currently loaded models (those with size in memory)
loaded_models = [
m.get('name', '') for m in models_response.get('models', [])
if m.get('size', 0) > 0
]
models_info = {
"configured": settings.agent_model,
"available": model_available,
"total_in_ollama": len(available_models),
"currently_loaded": loaded_models if loaded_models else ["none"]
}
if not model_available:
ollama_error = f"Model '{settings.agent_model}' not found in Ollama. Available: {', '.join(available_models[:3])}"
ollama_healthy = False
except Exception as e:
ollama_error = f"Could not list Ollama models: {str(e)}"
ollama_healthy = False
except Exception as e:
ollama_error = str(e)
logger.warning(f"Ollama health check failed: {ollama_error}")
# Check 2: Database connection
database = get_database() database = get_database()
db_healthy = False db_healthy = False
db_error = None db_error = None
@@ -140,27 +93,17 @@ class HealthController(BaseController):
db_error = str(e) db_error = str(e)
logger.warning(f"Database health check failed: {db_error}") logger.warning(f"Database health check failed: {db_error}")
is_healthy = ollama_healthy and db_healthy
elapsed_ms = int((time.time() - start_time) * 1000) elapsed_ms = int((time.time() - start_time) * 1000)
status_code = 200 if is_healthy else 503 status_code = 200 if db_healthy else 503
response.status_code = status_code response.status_code = status_code
return { return {
"status": "healthy" if is_healthy else "unhealthy", "status": "healthy" if db_healthy else "unhealthy",
"status_code": status_code, "status_code": status_code,
"response_time_ms": elapsed_ms, "response_time_ms": elapsed_ms,
"components": { "components": {
"ollama": {
"status": "✅ healthy" if ollama_healthy else "❌ unhealthy",
"models": models_info if models_info else {
"configured": settings.agent_model,
"available": False
},
"error": ollama_error
},
"database": { "database": {
"status": "✅ healthy" if db_healthy else "❌ unhealthy", "status": "healthy" if db_healthy else "unhealthy",
"error": db_error "error": db_error
} }
} }
@@ -170,18 +113,13 @@ class HealthController(BaseController):
"/health/diagnostics", "/health/diagnostics",
summary="Detailed system diagnostics", summary="Detailed system diagnostics",
) )
async def diagnostics(deep_test: bool = False): async def diagnostics():
""" """
Comprehensive system diagnostics with detailed component information. System diagnostics with service information.
Query Parameters:
- deep_test: Set to true to actually test agent generation (slow, ~5-10s)
Returns detailed information about all system components.
""" """
import time import time
start_time = time.time() start_time = time.time()
diagnostics = { diagnostics = {
"timestamp": time.time(), "timestamp": time.time(),
"service": { "service": {
@@ -189,31 +127,10 @@ class HealthController(BaseController):
"version": settings.app_version, "version": settings.app_version,
"purpose": "Infrastructure management and tools API" "purpose": "Infrastructure management and tools API"
}, },
"components": {} "configuration": {
}
# 1. Ollama Connection
ollama_client = get_ollama_client()
try:
ollama_healthy = await ollama_client.health_check()
diagnostics["components"]["ollama"] = {
"status": "✅ connected",
"url": settings.ollama_base_url,
"timeout": settings.ollama_timeout,
"default_model": settings.default_model
}
except Exception as e:
diagnostics["components"]["ollama"] = {
"status": "❌ error",
"error": str(e)
}
# 2. Configuration
diagnostics["configuration"] = {
"agent_fallback_enabled": settings.agent_fallback_enabled,
"memory_tier1_max_turns": settings.memory_tier1_max_turns,
"cors_origins": settings.cors_origins[:2] if len(settings.cors_origins) > 2 else settings.cors_origins "cors_origins": settings.cors_origins[:2] if len(settings.cors_origins) > 2 else settings.cors_origins
} }
}
elapsed_ms = int((time.time() - start_time) * 1000) elapsed_ms = int((time.time() - start_time) * 1000)
diagnostics["response_time_ms"] = elapsed_ms diagnostics["response_time_ms"] = elapsed_ms
+62 -1
View File
@@ -3,14 +3,20 @@ Tools Controller
Provides utility tool endpoints including: Provides utility tool endpoints including:
- DNS lookups - DNS lookups
- Environment data (weather, forecast, sun times, air quality)
""" """
from fastapi import APIRouter, HTTPException, status from typing import Dict, Optional
from fastapi import APIRouter, Depends, HTTPException, status
from src.controllers.base import BaseController from src.controllers.base import BaseController
from src.logging_config import get_logger from src.logging_config import get_logger
from src.dns.schemas import DNSLookupRequest, DNSLookupResponse from src.dns.schemas import DNSLookupRequest, DNSLookupResponse
from src.dns.service import DNSService from src.dns.service import DNSService
from src.dns.exceptions import DNSQueryError from src.dns.exceptions import DNSQueryError
from src.domains.tools.environment.schemas import EnvironmentResponse
from src.domains.tools.environment.service import get_environment_service
from src.auth.oidc import get_optional_user
logger = get_logger(__name__) logger = get_logger(__name__)
@@ -26,6 +32,7 @@ class ToolsController(BaseController):
def __init__(self): def __init__(self):
super().__init__(prefix="/tools", tags=["Tools"]) super().__init__(prefix="/tools", tags=["Tools"])
self.dns_service = DNSService() self.dns_service = DNSService()
self.environment_service = get_environment_service()
def create_router(self) -> APIRouter: def create_router(self) -> APIRouter:
"""Create and configure the router""" """Create and configure the router"""
@@ -95,6 +102,60 @@ class ToolsController(BaseController):
detail="An unexpected error occurred during DNS lookup" detail="An unexpected error occurred during DNS lookup"
) )
@router.get(
"/environment",
response_model=EnvironmentResponse,
status_code=status.HTTP_200_OK,
summary="Get environment data",
description="""
Fetch current environment data including weather, forecast, sun times,
and optionally air quality.
Data is retrieved from the user's volatile Qdrant collection which is
populated by background data collectors.
**Data Sources:**
- Weather: Current temperature, conditions, humidity, wind
- Forecast: Multi-day weather outlook
- Sun Times: Sunrise, sunset, daylight duration
- Air Quality: AQI and pollutant levels (when available)
**Authentication:**
- Uses authenticated user's `preferred_username` if available
- Falls back to 'default' for unauthenticated requests
"""
)
async def get_environment(
user: Optional[Dict] = Depends(get_optional_user),
) -> EnvironmentResponse:
"""
Get current environment data.
Args:
user: Optional authenticated user info
Returns:
Environment data with weather, forecast, sun times, and air quality
"""
try:
# Determine user identifier
user_id = "default"
if user:
logger.debug(f"User claims: {user}")
user_id = user.get("preferred_username") or user.get("sub", "default")
logger.info(f"Fetching environment data for user: {user_id} (preferred_username={user.get('preferred_username')}, sub={user.get('sub')})")
else:
logger.info(f"Fetching environment data for user: {user_id} (no auth)")
result = await self.environment_service.get_current(user_id)
return result
except Exception as e:
logger.error(f"Error fetching environment data: {str(e)}", exc_info=True)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="Failed to fetch environment data"
)
return router return router
+1 -1
View File
@@ -15,7 +15,7 @@ from sqlalchemy.ext.asyncio import (
from sqlalchemy.orm import DeclarativeBase from sqlalchemy.orm import DeclarativeBase
from sqlalchemy.pool import NullPool from sqlalchemy.pool import NullPool
from src.config import get_settings from src.shared.config import get_settings
from src.logging_config import get_logger from src.logging_config import get_logger
logger = get_logger(__name__) logger = get_logger(__name__)
+2
View File
@@ -7,6 +7,7 @@ from src.db.models.user import User
from src.db.models.role import Role, UserRole from src.db.models.role import Role, UserRole
from src.db.models.user_preferences import UserPreferences from src.db.models.user_preferences import UserPreferences
from src.db.models.api_key import ApiKey from src.db.models.api_key import ApiKey
from src.db.models.group import Group
__all__ = [ __all__ = [
"User", "User",
@@ -14,4 +15,5 @@ __all__ = [
"UserRole", "UserRole",
"UserPreferences", "UserPreferences",
"ApiKey", "ApiKey",
"Group",
] ]
+85
View File
@@ -0,0 +1,85 @@
"""
Group Model
Represents groups synced from Authentik.
Groups are used for access control and user organization.
"""
import uuid
from datetime import datetime
from typing import TYPE_CHECKING, List
from sqlalchemy import String, Boolean, DateTime, func, Table, Column, ForeignKey
from sqlalchemy.dialects.postgresql import UUID
from sqlalchemy.orm import Mapped, mapped_column, relationship
from src.db.database import Base
# Association table for User-Group many-to-many relationship
user_groups = Table(
"user_groups",
Base.metadata,
Column("user_id", UUID(as_uuid=True), ForeignKey("users.id", ondelete="CASCADE"), primary_key=True),
Column("group_id", UUID(as_uuid=True), ForeignKey("groups.id", ondelete="CASCADE"), primary_key=True),
)
class Group(Base):
"""
Group model synced from Authentik
Groups are fetched from Authentik admin API and cached locally.
They represent organizational units for access control.
"""
__tablename__ = "groups"
id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
primary_key=True,
default=uuid.uuid4,
)
authentik_id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
unique=True,
nullable=False,
index=True,
comment="Authentik group UUID",
)
name: Mapped[str] = mapped_column(
String(255),
unique=True,
nullable=False,
index=True,
)
is_superuser: Mapped[bool] = mapped_column(
Boolean,
default=False,
nullable=False,
comment="Whether members of this group have superuser privileges",
)
parent_name: Mapped[str | None] = mapped_column(
String(255),
nullable=True,
comment="Parent group name for hierarchy",
)
member_count: Mapped[int] = mapped_column(
default=0,
nullable=False,
comment="Number of users in this group",
)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
server_default=func.now(),
nullable=False,
)
synced_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
server_default=func.now(),
onupdate=func.now(),
nullable=False,
comment="Last sync from Authentik",
)
def __repr__(self) -> str:
return f"<Group {self.name}>"
+5
View File
@@ -0,0 +1,5 @@
"""
Domain modules for Core-API
Each domain contains its own models, schemas, services, and controllers.
"""
+62
View File
@@ -0,0 +1,62 @@
"""
Authentication Domain
Provides OIDC/OAuth2 authentication via Authentik, user management,
roles, groups, and API key authentication.
"""
from src.domains.auth.oidc import (
get_current_user,
get_admin_user,
get_optional_user,
get_forward_auth_user,
get_forward_auth_admin,
oidc_config,
# Permission system
require_permission,
require_any_permission,
ACTION_HIERARCHY,
VALID_DOMAINS,
DEFAULT_CATEGORY,
)
from src.domains.auth.service import AuthService, get_auth_service
from src.domains.auth.controller import auth_controller
from src.domains.auth.models import (
User,
Role,
UserRole,
Group,
UserPreferences,
ApiKey,
user_groups,
group_roles,
)
__all__ = [
# OIDC dependencies
"get_current_user",
"get_admin_user",
"get_optional_user",
"get_forward_auth_user",
"get_forward_auth_admin",
"oidc_config",
# Permission system
"require_permission",
"require_any_permission",
"ACTION_HIERARCHY",
"VALID_DOMAINS",
"DEFAULT_CATEGORY",
# Service
"AuthService",
"get_auth_service",
# Controller
"auth_controller",
# Models
"User",
"Role",
"UserRole",
"Group",
"UserPreferences",
"ApiKey",
"user_groups",
"group_roles",
]
+612
View File
@@ -0,0 +1,612 @@
"""
Authentication Controller
Provides authentication endpoints for OIDC token sync and user management.
"""
import uuid
from typing import Optional
from fastapi import APIRouter, Depends, HTTPException, Path, Query
from fastapi.responses import JSONResponse
from sqlalchemy.ext.asyncio import AsyncSession
from src.shared.base import BaseController
from src.shared.logging import get_logger
from src.shared.database import get_async_session
from src.domains.auth.schemas import (
AuthSyncRequest, AuthSyncResponse, UsersListResponse,
BulkSyncResultSchema, GroupsListResponse, RolesListResponse,
GroupRoleAssignmentResponse, UserProfileResponse, PreferencesUpdateRequest,
UserPreferencesSchema, ApiKeyCreateRequest, ApiKeyCreateResponse,
ApiKeysListResponse,
)
from src.domains.auth.service import AuthService
from src.domains.auth.oidc import get_current_user, get_current_user_or_forward_auth
logger = get_logger(__name__)
class AuthController(BaseController):
"""
Controller for authentication operations
Provides endpoints for:
- Token synchronization (login)
- User profile retrieval
"""
def __init__(self):
super().__init__(prefix="/auth", tags=["Authentication"])
def create_router(self) -> APIRouter:
"""Create and configure the router"""
router = APIRouter(prefix=self.prefix, tags=self.tags)
@router.post(
"/sync",
summary="Sync user from OIDC token",
response_model=AuthSyncResponse,
responses={
200: {"description": "User synced successfully"},
401: {"description": "Invalid or expired token"},
503: {"description": "Authentication service unavailable"},
},
)
async def sync_user(
request: AuthSyncRequest,
session: AsyncSession = Depends(get_async_session),
) -> AuthSyncResponse:
"""
Synchronize user from OIDC access token
This endpoint should be called after the client obtains an access token
from Authentik. It:
1. Validates the token via Authentik's userinfo endpoint
2. Creates or updates the user in the database
3. Syncs roles from Authentik groups
4. Returns the user profile with roles and preferences
The client should store the returned user info for local use.
"""
service = AuthService(session)
try:
# Validate token with Authentik
token_info = await service.validate_token(request.access_token)
except ValueError as e:
logger.warning(f"Token validation failed: {e}")
raise HTTPException(status_code=401, detail=str(e))
# Sync user to database
user, is_new = await service.sync_user(token_info)
# Sync roles from groups
roles = await service.sync_roles(user, token_info.groups)
# Commit the transaction
await session.commit()
# Refresh to get relationships
await session.refresh(user, ["preferences"])
# Build response
return AuthSyncResponse(
user=service.user_to_schema(user),
roles=service.roles_to_schema(roles),
preferences=service.preferences_to_schema(user.preferences),
is_new_user=is_new,
)
@router.get(
"/users",
summary="List all users",
response_model=UsersListResponse,
responses={
200: {"description": "List of users"},
},
)
async def list_users(
search: Optional[str] = Query(None, description="Search by name or email"),
offset: int = Query(0, ge=0, description="Number of records to skip"),
limit: int = Query(50, ge=1, le=100, description="Maximum records to return"),
session: AsyncSession = Depends(get_async_session),
) -> UsersListResponse:
"""
List all users who have logged in via Authentik
Returns paginated list of users with their roles.
Supports search filtering by name or email.
"""
service = AuthService(session)
items, total = await service.list_users(
search=search,
offset=offset,
limit=limit,
)
return UsersListResponse(items=items, total=total)
@router.post(
"/users/sync-from-authentik",
summary="Bulk sync users from Authentik",
response_model=BulkSyncResultSchema,
responses={
200: {"description": "Sync completed"},
401: {"description": "Authentik API token invalid"},
503: {"description": "Authentik service unavailable"},
},
)
async def sync_users_from_authentik(
session: AsyncSession = Depends(get_async_session),
) -> BulkSyncResultSchema:
"""
Fetch all users from Authentik and sync to local database
This endpoint uses the Authentik admin API to fetch all users
and create/update them in the local database. Requires
AUTHENTIK_CORE_API_TOKEN to be configured.
Use this to initially populate users or to re-sync after
changes in Authentik.
"""
service = AuthService(session)
try:
result = await service.bulk_sync_from_authentik()
logger.info(
f"Bulk sync completed: {result.created} created, "
f"{result.updated} updated, {result.failed} failed"
)
return result
except ValueError as e:
logger.error(f"Bulk sync failed: {e}")
raise HTTPException(status_code=401, detail=str(e))
@router.get(
"/groups",
summary="List all groups",
response_model=GroupsListResponse,
responses={
200: {"description": "List of groups"},
},
)
async def list_groups(
search: Optional[str] = Query(None, description="Search by group name"),
offset: int = Query(0, ge=0, description="Number of records to skip"),
limit: int = Query(50, ge=1, le=100, description="Maximum records to return"),
session: AsyncSession = Depends(get_async_session),
) -> GroupsListResponse:
"""
List all groups synced from Authentik
Returns paginated list of groups with their details.
Supports search filtering by name.
"""
service = AuthService(session)
items, total = await service.list_groups(
search=search,
offset=offset,
limit=limit,
)
return GroupsListResponse(items=items, total=total)
@router.post(
"/groups/sync-from-authentik",
summary="Bulk sync groups from Authentik",
response_model=BulkSyncResultSchema,
responses={
200: {"description": "Sync completed"},
401: {"description": "Authentik API credentials invalid"},
503: {"description": "Authentik service unavailable"},
},
)
async def sync_groups_from_authentik(
session: AsyncSession = Depends(get_async_session),
) -> BulkSyncResultSchema:
"""
Fetch all groups from Authentik and sync to local database
This endpoint uses the Authentik admin API to fetch all groups
and create/update them in the local database. Requires
AUTHENTIK_USERNAME and AUTHENTIK_PASSWORD to be configured.
Use this to populate groups or to re-sync after changes in Authentik.
"""
service = AuthService(session)
try:
result = await service.bulk_sync_groups_from_authentik()
logger.info(
f"Groups bulk sync completed: {result.created} created, "
f"{result.updated} updated, {result.failed} failed"
)
return result
except ValueError as e:
logger.error(f"Groups bulk sync failed: {e}")
raise HTTPException(status_code=401, detail=str(e))
@router.get(
"/roles",
summary="List all roles",
response_model=RolesListResponse,
responses={
200: {"description": "List of all available roles"},
},
)
async def list_roles(
session: AsyncSession = Depends(get_async_session),
) -> RolesListResponse:
"""
List all available roles in the system
Returns all domain.category:action role combinations.
Use these when assigning roles to groups.
"""
service = AuthService(session)
roles = await service.list_roles()
return RolesListResponse(
items=service.roles_to_schema(roles),
total=len(roles),
)
@router.post(
"/groups/{group_id}/roles/{role_id}",
summary="Assign role to group",
response_model=GroupRoleAssignmentResponse,
responses={
200: {"description": "Role assigned successfully"},
404: {"description": "Group or role not found"},
},
)
async def assign_role_to_group(
group_id: uuid.UUID = Path(..., description="Group ID"),
role_id: uuid.UUID = Path(..., description="Role ID to assign"),
session: AsyncSession = Depends(get_async_session),
) -> GroupRoleAssignmentResponse:
"""
Assign a role to a group
All users in this group will inherit this role's permissions.
"""
service = AuthService(session)
try:
group = await service.assign_role_to_group(group_id, role_id)
await session.commit()
return GroupRoleAssignmentResponse(
group_id=group.id,
group_name=group.name,
roles=[role.name for role in group.roles],
)
except ValueError as e:
raise HTTPException(status_code=404, detail=str(e))
@router.delete(
"/groups/{group_id}/roles/{role_id}",
summary="Remove role from group",
response_model=GroupRoleAssignmentResponse,
responses={
200: {"description": "Role removed successfully"},
404: {"description": "Group or role not found"},
},
)
async def remove_role_from_group(
group_id: uuid.UUID = Path(..., description="Group ID"),
role_id: uuid.UUID = Path(..., description="Role ID to remove"),
session: AsyncSession = Depends(get_async_session),
) -> GroupRoleAssignmentResponse:
"""
Remove a role from a group
Users in this group will no longer inherit this role's permissions.
"""
service = AuthService(session)
try:
group = await service.remove_role_from_group(group_id, role_id)
await session.commit()
return GroupRoleAssignmentResponse(
group_id=group.id,
group_name=group.name,
roles=[role.name for role in group.roles],
)
except ValueError as e:
raise HTTPException(status_code=404, detail=str(e))
# =====================================================================
# Phase 4: User Profile & Settings
# =====================================================================
@router.get(
"/users/me",
summary="Get current user profile",
response_model=UserProfileResponse,
responses={
200: {"description": "User profile with roles and preferences"},
401: {"description": "Not authenticated"},
},
)
async def get_current_user_profile(
user_claims: dict = Depends(get_current_user_or_forward_auth),
session: AsyncSession = Depends(get_async_session),
) -> UserProfileResponse:
"""
Get the current authenticated user's profile
Returns the user's profile, roles, and preferences.
Supports both:
- Bearer token (mobile/native clients)
- NPM forward auth headers (web clients via proxy)
For forward auth users, auto-creates the user in the database
if they don't exist yet (first login via web).
"""
service = AuthService(session)
# Get authentik_id from claims (JWT 'sub' field or forward auth 'uid')
authentik_id_str = user_claims.get("sub")
if not authentik_id_str or authentik_id_str == "local-user":
raise HTTPException(status_code=401, detail="Authentication required")
try:
authentik_id = uuid.UUID(authentik_id_str)
except ValueError:
raise HTTPException(status_code=401, detail="Invalid user identifier")
user = await service.get_user_by_authentik_id(authentik_id)
# If user not found and using forward auth, auto-create them
if user is None:
auth_method = user_claims.get("auth_method")
if auth_method == "forward_auth":
# Auto-sync user from forward auth headers
logger.info(f"Auto-creating user from forward auth: {user_claims.get('email')}")
user, is_new = await service.sync_user_from_claims(
authentik_id=authentik_id,
email=user_claims.get("email", ""),
name=user_claims.get("name", user_claims.get("preferred_username", "")),
groups=user_claims.get("groups", []),
)
await session.commit()
await session.refresh(user, ["preferences", "roles"])
else:
# JWT auth but user not in DB - they need to sync first
raise HTTPException(
status_code=404,
detail="User not found - please sync via /auth/sync first",
)
return UserProfileResponse(
user=service.user_to_schema(user),
roles=service.roles_to_schema(user.roles),
preferences=service.preferences_to_schema(user.preferences),
)
@router.get(
"/users/me/preferences",
summary="Get user preferences",
response_model=UserPreferencesSchema,
responses={
200: {"description": "User preferences"},
401: {"description": "Not authenticated"},
},
)
async def get_preferences(
user_claims: dict = Depends(get_current_user),
session: AsyncSession = Depends(get_async_session),
) -> UserPreferencesSchema:
"""
Get the current user's preferences
"""
service = AuthService(session)
authentik_id_str = user_claims.get("sub")
if not authentik_id_str or authentik_id_str == "local-user":
raise HTTPException(status_code=401, detail="Authentication required")
try:
authentik_id = uuid.UUID(authentik_id_str)
except ValueError:
raise HTTPException(status_code=401, detail="Invalid user identifier")
user = await service.get_user_by_authentik_id(authentik_id)
if user is None:
raise HTTPException(status_code=404, detail="User not found")
return service.preferences_to_schema(user.preferences)
@router.patch(
"/users/me/preferences",
summary="Update user preferences",
response_model=UserPreferencesSchema,
responses={
200: {"description": "Updated preferences"},
401: {"description": "Not authenticated"},
422: {"description": "Invalid preference value"},
},
)
async def update_preferences(
request: PreferencesUpdateRequest,
user_claims: dict = Depends(get_current_user),
session: AsyncSession = Depends(get_async_session),
) -> UserPreferencesSchema:
"""
Update the current user's preferences
Only provided fields are updated. preferences_json is merged
with existing values (not replaced).
"""
service = AuthService(session)
authentik_id_str = user_claims.get("sub")
if not authentik_id_str or authentik_id_str == "local-user":
raise HTTPException(status_code=401, detail="Authentication required")
try:
authentik_id = uuid.UUID(authentik_id_str)
except ValueError:
raise HTTPException(status_code=401, detail="Invalid user identifier")
user = await service.get_user_by_authentik_id(authentik_id)
if user is None:
raise HTTPException(status_code=404, detail="User not found")
try:
prefs = await service.update_preferences(
user_id=user.id,
theme=request.theme,
default_room=request.default_room,
preferences_json=request.preferences_json,
)
await session.commit()
return service.preferences_to_schema(prefs)
except ValueError as e:
raise HTTPException(status_code=422, detail=str(e))
# =====================================================================
# Phase 4: API Keys
# =====================================================================
@router.get(
"/users/me/api-keys",
summary="List user's API keys",
response_model=ApiKeysListResponse,
responses={
200: {"description": "List of API keys"},
401: {"description": "Not authenticated"},
},
)
async def list_api_keys(
user_claims: dict = Depends(get_current_user),
session: AsyncSession = Depends(get_async_session),
) -> ApiKeysListResponse:
"""
List all API keys for the current user
Returns key metadata only - the actual key values are never
retrievable after creation.
"""
service = AuthService(session)
authentik_id_str = user_claims.get("sub")
if not authentik_id_str or authentik_id_str == "local-user":
raise HTTPException(status_code=401, detail="Authentication required")
try:
authentik_id = uuid.UUID(authentik_id_str)
except ValueError:
raise HTTPException(status_code=401, detail="Invalid user identifier")
user = await service.get_user_by_authentik_id(authentik_id)
if user is None:
raise HTTPException(status_code=404, detail="User not found")
keys = await service.list_user_api_keys(user.id)
return ApiKeysListResponse(
items=[service.api_key_to_schema(k) for k in keys],
total=len(keys),
)
@router.post(
"/users/me/api-keys",
summary="Create a new API key",
response_model=ApiKeyCreateResponse,
responses={
201: {"description": "API key created"},
401: {"description": "Not authenticated"},
403: {"description": "API keys disabled for user"},
},
)
async def create_api_key(
request: ApiKeyCreateRequest,
user_claims: dict = Depends(get_current_user),
session: AsyncSession = Depends(get_async_session),
) -> ApiKeyCreateResponse:
"""
Create a new API key for the current user
**IMPORTANT**: The full API key is only returned once in this response!
Store it securely - it cannot be retrieved again.
"""
service = AuthService(session)
authentik_id_str = user_claims.get("sub")
if not authentik_id_str or authentik_id_str == "local-user":
raise HTTPException(status_code=401, detail="Authentication required")
try:
authentik_id = uuid.UUID(authentik_id_str)
except ValueError:
raise HTTPException(status_code=401, detail="Invalid user identifier")
user = await service.get_user_by_authentik_id(authentik_id)
if user is None:
raise HTTPException(status_code=404, detail="User not found")
try:
api_key, full_key = await service.create_api_key(
user_id=user.id,
name=request.name,
scopes=request.scopes,
expires_in_days=request.expires_in_days,
)
await session.commit()
return ApiKeyCreateResponse(
id=api_key.id,
name=api_key.name,
key=full_key, # Only time this is returned!
key_prefix=api_key.key_prefix,
scopes=api_key.scopes,
expires_at=api_key.expires_at,
created_at=api_key.created_at,
)
except ValueError as e:
raise HTTPException(status_code=403, detail=str(e))
@router.delete(
"/users/me/api-keys/{key_id}",
summary="Delete an API key",
responses={
204: {"description": "API key deleted"},
401: {"description": "Not authenticated"},
404: {"description": "API key not found"},
},
)
async def delete_api_key(
key_id: uuid.UUID = Path(..., description="API key ID to delete"),
user_claims: dict = Depends(get_current_user),
session: AsyncSession = Depends(get_async_session),
) -> JSONResponse:
"""
Delete an API key
The key will be immediately invalidated.
"""
service = AuthService(session)
authentik_id_str = user_claims.get("sub")
if not authentik_id_str or authentik_id_str == "local-user":
raise HTTPException(status_code=401, detail="Authentication required")
try:
authentik_id = uuid.UUID(authentik_id_str)
except ValueError:
raise HTTPException(status_code=401, detail="Invalid user identifier")
user = await service.get_user_by_authentik_id(authentik_id)
if user is None:
raise HTTPException(status_code=404, detail="User not found")
try:
deleted = await service.delete_api_key(user.id, key_id)
if not deleted:
raise HTTPException(status_code=404, detail="API key not found")
await session.commit()
return JSONResponse(status_code=204, content=None)
except ValueError as e:
raise HTTPException(status_code=403, detail=str(e))
return router
# Create controller instance
auth_controller = AuthController()
+408
View File
@@ -0,0 +1,408 @@
"""
Authentication Domain Models
SQLAlchemy models for users, roles, groups, API keys, and preferences.
All authentication-related database models consolidated in one file.
"""
import uuid
from datetime import datetime
from typing import TYPE_CHECKING, List
from sqlalchemy import String, Boolean, DateTime, func, ForeignKey, Table, Column
from sqlalchemy.dialects.postgresql import UUID, JSONB, ARRAY
from sqlalchemy.orm import Mapped, mapped_column, relationship
from src.shared.database import Base
# =============================================================================
# Association Tables
# =============================================================================
user_groups = Table(
"user_groups",
Base.metadata,
Column("user_id", UUID(as_uuid=True), ForeignKey("users.id", ondelete="CASCADE"), primary_key=True),
Column("group_id", UUID(as_uuid=True), ForeignKey("groups.id", ondelete="CASCADE"), primary_key=True),
)
group_roles = Table(
"group_roles",
Base.metadata,
Column("group_id", UUID(as_uuid=True), ForeignKey("groups.id", ondelete="CASCADE"), primary_key=True),
Column("role_id", UUID(as_uuid=True), ForeignKey("roles.id", ondelete="CASCADE"), primary_key=True),
)
# =============================================================================
# User Model
# =============================================================================
class User(Base):
"""
User model synced from Authentik
Users are created/updated when they authenticate via OIDC.
The authentik_id links to the Authentik user record.
"""
__tablename__ = "users"
id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
primary_key=True,
default=uuid.uuid4,
)
authentik_id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
unique=True,
nullable=False,
index=True,
)
email: Mapped[str] = mapped_column(
String(255),
unique=True,
nullable=False,
index=True,
)
name: Mapped[str] = mapped_column(
String(255),
nullable=False,
)
avatar_url: Mapped[str | None] = mapped_column(
String(500),
nullable=True,
)
api_keys_enabled: Mapped[bool] = mapped_column(
Boolean,
default=True,
nullable=False,
)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
server_default=func.now(),
nullable=False,
)
last_login: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
nullable=True,
)
# Relationships
roles: Mapped[List["Role"]] = relationship(
"Role",
secondary="user_roles",
back_populates="users",
lazy="selectin",
)
preferences: Mapped["UserPreferences"] = relationship(
"UserPreferences",
back_populates="user",
uselist=False,
lazy="selectin",
cascade="all, delete-orphan",
)
api_keys: Mapped[List["ApiKey"]] = relationship(
"ApiKey",
back_populates="user",
lazy="selectin",
cascade="all, delete-orphan",
)
def __repr__(self) -> str:
return f"<User {self.email}>"
# =============================================================================
# Role Models
# =============================================================================
class Role(Base):
"""
Role model for domain-scoped permissions
Permission format: domain.category:action
- domain: Main area (control-room, library, media, ai, etc.)
- category: Sub-area within domain (general for full access, or specific tools)
- action: Permission level (viewer, user, editor, admin)
Roles are seeded from configuration, not user-editable.
Groups are assigned roles via the group_roles mapping table.
Domains: control-room, library, media, ai, housekeeper, developer, documents, gaming, admin
Actions: viewer, user, editor, admin (hierarchical)
"""
__tablename__ = "roles"
id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
primary_key=True,
default=uuid.uuid4,
)
name: Mapped[str] = mapped_column(
String(100),
unique=True,
nullable=False,
index=True,
comment="Role name in format domain.category:action (e.g., control-room.general:admin)",
)
domain: Mapped[str] = mapped_column(
String(50),
nullable=False,
index=True,
comment="Permission domain (e.g., control-room, media, ai)",
)
category: Mapped[str] = mapped_column(
String(50),
nullable=False,
default="general",
comment="Permission category within domain (general for full access, or specific tool)",
)
action: Mapped[str] = mapped_column(
String(20),
nullable=False,
comment="Permission action (viewer, user, editor, admin)",
)
# Relationships
users: Mapped[List["User"]] = relationship(
"User",
secondary="user_roles",
back_populates="roles",
lazy="selectin",
)
groups: Mapped[List["Group"]] = relationship(
"Group",
secondary="group_roles",
back_populates="roles",
lazy="selectin",
)
def __repr__(self) -> str:
return f"<Role {self.name}>"
class UserRole(Base):
"""
Association table for User-Role many-to-many relationship
Synced from Authentik groups during user authentication.
"""
__tablename__ = "user_roles"
user_id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
ForeignKey("users.id", ondelete="CASCADE"),
primary_key=True,
)
role_id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
ForeignKey("roles.id", ondelete="CASCADE"),
primary_key=True,
)
# =============================================================================
# Group Model
# =============================================================================
class Group(Base):
"""
Group model synced from Authentik
Groups are fetched from Authentik admin API and cached locally.
They represent organizational units for access control.
"""
__tablename__ = "groups"
id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
primary_key=True,
default=uuid.uuid4,
)
authentik_id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
unique=True,
nullable=False,
index=True,
comment="Authentik group UUID",
)
name: Mapped[str] = mapped_column(
String(255),
unique=True,
nullable=False,
index=True,
)
is_superuser: Mapped[bool] = mapped_column(
Boolean,
default=False,
nullable=False,
comment="Whether members of this group have superuser privileges",
)
parent_name: Mapped[str | None] = mapped_column(
String(255),
nullable=True,
comment="Parent group name for hierarchy",
)
member_count: Mapped[int] = mapped_column(
default=0,
nullable=False,
comment="Number of users in this group",
)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
server_default=func.now(),
nullable=False,
)
synced_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
server_default=func.now(),
onupdate=func.now(),
nullable=False,
comment="Last sync from Authentik",
)
# Relationships
roles: Mapped[List["Role"]] = relationship(
"Role",
secondary="group_roles",
back_populates="groups",
lazy="selectin",
)
def __repr__(self) -> str:
return f"<Group {self.name}>"
# =============================================================================
# User Preferences Model
# =============================================================================
class UserPreferences(Base):
"""
User preferences model
Stores user-specific settings that persist across sessions.
Extended settings stored in preferences_json for flexibility.
"""
__tablename__ = "user_preferences"
user_id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
ForeignKey("users.id", ondelete="CASCADE"),
primary_key=True,
)
theme: Mapped[str] = mapped_column(
String(20),
default="system",
nullable=False,
comment="Theme preference: system, light, dark",
)
default_room: Mapped[str] = mapped_column(
String(50),
default="front-hall",
nullable=False,
comment="Default room for housekeeping features",
)
preferences_json: Mapped[dict] = mapped_column(
JSONB,
default=dict,
nullable=False,
comment="Extended preferences as JSON",
)
# Relationships
user: Mapped["User"] = relationship(
"User",
back_populates="preferences",
)
def __repr__(self) -> str:
return f"<UserPreferences user_id={self.user_id}>"
# =============================================================================
# API Key Model
# =============================================================================
class ApiKey(Base):
"""
API Key model for programmatic access
API keys provide an alternative to OIDC for:
- Local development without SSO
- Service-to-service communication
- Scripts and automation
Keys inherit the user's roles but can optionally
be restricted to a subset of scopes.
"""
__tablename__ = "api_keys"
id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
primary_key=True,
default=uuid.uuid4,
)
user_id: Mapped[uuid.UUID] = mapped_column(
UUID(as_uuid=True),
ForeignKey("users.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
name: Mapped[str] = mapped_column(
String(100),
nullable=False,
comment="Human-readable key name (e.g., 'Dev Laptop', 'CI/CD')",
)
key_hash: Mapped[str] = mapped_column(
String(255),
nullable=False,
comment="SHA-256 hash of the API key",
)
key_prefix: Mapped[str] = mapped_column(
String(8),
nullable=False,
comment="First 8 chars of key for identification (e.g., 'cak_abc1')",
)
scopes: Mapped[List[str] | None] = mapped_column(
ARRAY(String),
nullable=True,
comment="Optional scope restriction (subset of user roles)",
)
expires_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
nullable=True,
comment="Optional expiration timestamp",
)
last_used_at: Mapped[datetime | None] = mapped_column(
DateTime(timezone=True),
nullable=True,
comment="Last time this key was used",
)
created_at: Mapped[datetime] = mapped_column(
DateTime(timezone=True),
server_default=func.now(),
nullable=False,
)
# Relationships
user: Mapped["User"] = relationship(
"User",
back_populates="api_keys",
)
def __repr__(self) -> str:
return f"<ApiKey {self.key_prefix}... ({self.name})>"
@property
def is_expired(self) -> bool:
"""Check if the API key has expired"""
if self.expires_at is None:
return False
return datetime.now(self.expires_at.tzinfo) > self.expires_at
+776
View File
@@ -0,0 +1,776 @@
"""
OIDC Authentication Module
Provides OAuth2/OIDC token validation for FastAPI using Authentik as IdP.
Implements bearer token authentication with JWT verification.
Permission Format: domain.category:action
- domain: Main area (control-room, library, media, ai, etc.)
- category: Sub-area within domain (general for full domain, or specific tools)
- action: Permission level (viewer, user, editor, admin)
Examples:
- control-room.general:admin - Full access to Control Room
- media.general:viewer - View-only access to Media area
Action Hierarchy (higher implies lower):
- admin > editor > user > viewer
"""
from fastapi import Depends, HTTPException, Security, Request
from fastapi.security import HTTPBearer, HTTPAuthorizationCredentials
from jose import jwt, JWTError
import httpx
from functools import lru_cache
from typing import Callable, Dict, List, Optional
from src.shared.logging import get_logger
# =============================================================================
# Permission System
# =============================================================================
# Action hierarchy: higher actions imply lower ones
ACTION_HIERARCHY: Dict[str, int] = {
"viewer": 1,
"user": 2,
"editor": 3,
"admin": 4,
}
# Valid domains (main areas)
VALID_DOMAINS = {
"control-room",
"library",
"media",
"ai",
"housekeeper",
"developer",
"documents",
"gaming",
"admin", # Global admin domain
}
# Default category for general domain access
DEFAULT_CATEGORY = "general"
logger = get_logger(__name__)
security = HTTPBearer(auto_error=False)
class OIDCConfig:
"""OIDC configuration from environment"""
def __init__(self):
# These will be set from environment variables in config.py
self.enabled = False
self.issuers: list[str] = []
self.audiences: list[str] = []
def configure(self, enabled: bool, issuers: list[str], audiences: list[str]):
"""Configure OIDC settings"""
self.enabled = enabled
self.issuers = [iss.rstrip('/') for iss in issuers] # Normalize without trailing slash
self.audiences = audiences
logger.info(f"OIDC configured: enabled={enabled}, issuers={self.issuers}, audiences={audiences}")
def get_jwks_uri(self, issuer: str) -> str:
"""Get JWKS URI for a specific issuer"""
return f"{issuer.rstrip('/')}/jwks/"
def is_valid_issuer(self, issuer: str) -> bool:
"""Check if issuer is in the allowed list"""
normalized = issuer.rstrip('/')
return normalized in self.issuers
# Global OIDC config instance
oidc_config = OIDCConfig()
# Per-issuer JWKS cache
_jwks_cache: Dict[str, Dict] = {}
def get_jwks_for_issuer(issuer: str) -> Dict:
"""
Fetch JSON Web Key Set (JWKS) for a specific issuer.
Cached per-issuer to avoid repeated requests. Cache is cleared on server restart.
Args:
issuer: The token issuer URL
Returns:
JWKS dictionary containing public keys for token verification
Raises:
HTTPException: If JWKS fetch fails
"""
if not oidc_config.enabled:
return {}
normalized_issuer = issuer.rstrip('/')
# Return cached JWKS if available
if normalized_issuer in _jwks_cache:
return _jwks_cache[normalized_issuer]
jwks_uri = oidc_config.get_jwks_uri(normalized_issuer)
try:
logger.debug(f"Fetching JWKS from {jwks_uri}")
response = httpx.get(jwks_uri, timeout=10.0)
response.raise_for_status()
jwks = response.json()
logger.info(f"JWKS fetched successfully for {normalized_issuer} ({len(jwks.get('keys', []))} keys)")
_jwks_cache[normalized_issuer] = jwks
return jwks
except Exception as e:
logger.error(f"Failed to fetch JWKS from {jwks_uri}: {e}")
raise HTTPException(
status_code=503,
detail="Authentication service unavailable"
)
async def get_current_user(
credentials: Optional[HTTPAuthorizationCredentials] = Security(security)
) -> Optional[Dict]:
"""
Validate OIDC token from Authorization: Bearer header
Extracts and validates JWT token from request header. Verifies:
- Token signature using JWKS
- Token expiration
- Issuer matches Authentik
- Audience matches core-api
Args:
credentials: HTTP Bearer token from Authorization header
Returns:
User claims dictionary containing email, name, groups, etc.
Returns None if OIDC is disabled (allows unauthenticated access)
Raises:
HTTPException 401: If token is invalid, expired, or missing when OIDC enabled
"""
# If OIDC is disabled, return a default local user
if not oidc_config.enabled:
logger.debug("OIDC disabled - using local user")
return {
"sub": "local-user",
"email": "local@localhost",
"preferred_username": "local",
"name": "Local User",
"groups": ["admin"],
"auth_method": "local"
}
# OIDC enabled - token required
if not credentials:
logger.warning("Authentication required but no token provided")
raise HTTPException(
status_code=401,
detail="Authentication required",
headers={"WWW-Authenticate": "Bearer"},
)
token = credentials.credentials
try:
# First, decode token without verification to get issuer and key ID
unverified_header = jwt.get_unverified_header(token)
unverified_claims = jwt.get_unverified_claims(token)
kid = unverified_header.get("kid")
token_issuer = unverified_claims.get("iss", "")
token_audience = unverified_claims.get("aud", "")
if not kid:
raise HTTPException(status_code=401, detail="Invalid token format")
# Validate issuer is in allowed list
logger.debug(f"Token issuer: {token_issuer}, allowed issuers: {oidc_config.issuers}")
if not oidc_config.is_valid_issuer(token_issuer):
logger.warning(f"Invalid token issuer: {token_issuer} (allowed: {oidc_config.issuers})")
raise HTTPException(status_code=401, detail="Invalid token issuer")
# Validate audience is in allowed list
if token_audience not in oidc_config.audiences:
logger.warning(f"Invalid token audience: {token_audience} (allowed: {oidc_config.audiences})")
raise HTTPException(status_code=401, detail="Invalid token audience")
# Get JWKS for this specific issuer
jwks = get_jwks_for_issuer(token_issuer)
rsa_key = None
for key in jwks.get("keys", []):
if key.get("kid") == kid:
rsa_key = key
break
if not rsa_key:
logger.warning(f"No matching key found for kid: {kid}")
raise HTTPException(status_code=401, detail="Invalid token key")
# Verify and decode token using the token's actual issuer and audience
payload = jwt.decode(
token,
rsa_key,
algorithms=["RS256"],
audience=token_audience, # Use the token's audience (already validated)
issuer=token_issuer, # Use the token's issuer (already validated)
)
user_email = payload.get("email", "unknown")
logger.info(f"Authenticated user: {user_email} (issuer: {token_issuer})")
return payload
except jwt.ExpiredSignatureError:
logger.warning("Token expired")
raise HTTPException(
status_code=401,
detail="Token expired",
headers={"WWW-Authenticate": "Bearer"},
)
except jwt.JWTClaimsError as e:
logger.warning(f"Invalid token claims: {e}")
raise HTTPException(
status_code=401,
detail="Invalid token claims",
headers={"WWW-Authenticate": "Bearer"},
)
except JWTError as e:
logger.error(f"JWT validation error: {e}")
raise HTTPException(
status_code=401,
detail="Invalid authentication token",
headers={"WWW-Authenticate": "Bearer"},
)
except Exception as e:
logger.error(f"Unexpected authentication error: {e}")
raise HTTPException(
status_code=500,
detail="Authentication error",
)
async def get_admin_user(
user: Optional[Dict] = Depends(get_current_user)
) -> Dict:
"""
Require admin group membership
Use this dependency for endpoints that require admin access.
Checks if user is member of 'admin' group in Authentik.
Args:
user: User claims from get_current_user
Returns:
User claims dictionary if user is admin
Raises:
HTTPException 403: If user is not in admin group
HTTPException 401: If OIDC enabled but user not authenticated
"""
# If OIDC disabled, allow all (backward compatibility)
if not oidc_config.enabled or user is None:
logger.debug("OIDC disabled - allowing admin access")
return {"email": "unauthenticated", "groups": ["admin"]}
# Check admin group membership
groups = user.get("groups", [])
if "admin" not in groups and "authentik Admins" not in groups:
user_email = user.get("email", "unknown")
logger.warning(f"User {user_email} attempted admin access (groups: {groups})")
raise HTTPException(
status_code=403,
detail="Admin access required"
)
return user
async def get_optional_user(
credentials: Optional[HTTPAuthorizationCredentials] = Security(security)
) -> Optional[Dict]:
"""
Optional authentication - allows both authenticated and unauthenticated access
Use for endpoints that should be accessible to everyone but can provide
enhanced functionality for authenticated users.
Args:
credentials: HTTP Bearer token from Authorization header
Returns:
User claims if valid token provided, local user if OIDC disabled, None otherwise
"""
# If OIDC is disabled, return the local user
if not oidc_config.enabled:
return {
"sub": "local-user",
"email": "local@localhost",
"preferred_username": "local",
"name": "Local User",
"groups": ["admin"],
"auth_method": "local"
}
if not credentials:
logger.debug("No credentials provided for optional auth")
return None
try:
return await get_current_user(credentials)
except HTTPException as e:
# Invalid token - log and return None instead of raising
logger.warning(f"Optional auth failed: {e.detail}")
return None
async def get_forward_auth_user(
request: Request
) -> Optional[Dict]:
"""
Authentik Forward Auth authentication for external access via NPM
This dependency allows:
- External access through api.schweitz.net (with Authentik forward auth headers) - REQUIRES authentication
- Internal direct access (no forward auth headers) - ALLOWED without authentication
When accessing through NPM with Authentik forward auth enabled, NPM adds headers like:
- X-authentik-username
- X-authentik-email
- X-authentik-groups
- X-authentik-name
- X-authentik-uid
Args:
request: FastAPI request object containing headers
Returns:
User info dict if authenticated via forward auth headers
None if accessed internally (no forward auth headers)
Raises:
HTTPException 401: If forward auth headers present but invalid/incomplete
"""
# Check for Authentik forward auth headers
username = request.headers.get("x-authentik-username")
email = request.headers.get("x-authentik-email")
groups = request.headers.get("x-authentik-groups")
name = request.headers.get("x-authentik-name")
uid = request.headers.get("x-authentik-uid")
# If NO forward auth headers present, this is internal access - allow it
if not username and not email:
logger.debug("No forward auth headers - allowing internal access")
return None
# Forward auth headers present (external access via api.schweitz.net)
# Validate authentication
if not username or not email:
logger.warning("Incomplete forward auth headers detected")
raise HTTPException(
status_code=401,
detail="Authentication required - incomplete forward auth headers"
)
# Parse groups (comma-separated string to list)
groups_list = [g.strip() for g in groups.split(",")] if groups else []
user_info = {
"username": username,
"email": email,
"name": name or username,
"groups": groups_list,
"uid": uid,
"auth_method": "forward_auth"
}
logger.info(f"Authenticated via forward auth: {email} (groups: {groups_list})")
return user_info
async def get_forward_auth_admin(
user: Optional[Dict] = Depends(get_forward_auth_user)
) -> Dict:
"""
Require admin access for external requests, allow all internal requests
Use this dependency for endpoints that require admin access when accessed
externally through api.schweitz.net, but allow unrestricted internal access.
Args:
user: User info from get_forward_auth_user
Returns:
User info dict if user is admin or if accessed internally
Raises:
HTTPException 403: If external user is not in admin/authentik Admins group
"""
# Internal access (no forward auth headers) - allow all
if user is None:
logger.debug("Internal access - allowing without admin check")
return {"email": "internal", "groups": ["admin"], "auth_method": "internal"}
# External access - check admin group membership
groups = user.get("groups", [])
if "admin" not in groups and "authentik Admins" not in groups:
user_email = user.get("email", "unknown")
logger.warning(f"User {user_email} attempted admin access (groups: {groups})")
raise HTTPException(
status_code=403,
detail="Admin access required"
)
return user
async def get_current_user_or_forward_auth(
request: Request,
credentials: Optional[HTTPAuthorizationCredentials] = Security(security)
) -> Dict:
"""
Combined auth: Try forward auth headers first, then JWT Bearer token.
Supports both:
- Web clients via NPM forward auth (X-authentik-* headers from proxy)
- Mobile/native clients via OIDC JWT Bearer tokens
This is the preferred dependency for /auth/users/me and similar endpoints
that need to work with both web (cookie-based via NPM) and mobile (token-based).
Args:
request: FastAPI request object containing headers
credentials: HTTP Bearer token from Authorization header
Returns:
User claims dictionary with at minimum: sub, email, name, groups, auth_method
Raises:
HTTPException 401: If neither forward auth headers nor valid JWT provided
"""
# 1. Try forward auth headers first (web via NPM)
username = request.headers.get("x-authentik-username")
email = request.headers.get("x-authentik-email")
if username and email:
# Forward auth headers present - use them
groups = request.headers.get("x-authentik-groups", "")
name = request.headers.get("x-authentik-name", username)
uid = request.headers.get("x-authentik-uid")
user_info = {
"sub": uid, # Use authentik UID as subject (for user lookup)
"email": email,
"preferred_username": username,
"name": name,
"groups": [g.strip() for g in groups.split(",")] if groups else [],
"auth_method": "forward_auth"
}
logger.info(f"Authenticated via forward auth: {email}")
return user_info
# 2. Fall back to JWT Bearer token (mobile/native)
return await get_current_user(credentials)
# =============================================================================
# Permission-Based Access Control
# =============================================================================
def _parse_permission(permission: str) -> tuple[str, str, str]:
"""
Parse a permission string into (domain, category, action)
Supports formats:
- domain.category:action (full): "control-room.general:admin"
- domain:action (shorthand): "control-room:admin" -> ("control-room", "general", "admin")
Returns:
Tuple of (domain, category, action)
Raises:
ValueError: If permission format is invalid
"""
# Split on colon first to get action
if ":" not in permission:
raise ValueError(f"Invalid permission format (missing ':'): {permission}")
location, action = permission.rsplit(":", 1)
# Split location on dot to get domain and category
if "." in location:
domain, category = location.split(".", 1)
else:
# Shorthand: domain:action -> domain.general:action
domain = location
category = DEFAULT_CATEGORY
return domain, category, action
def _action_satisfies(user_action: str, required_action: str) -> bool:
"""
Check if user's action level satisfies the required action
Due to hierarchy, admin satisfies editor, editor satisfies user, etc.
Args:
user_action: The action the user has
required_action: The action required for access
Returns:
True if user's action is >= required action
"""
user_level = ACTION_HIERARCHY.get(user_action, 0)
required_level = ACTION_HIERARCHY.get(required_action, 0)
return user_level >= required_level
def _user_has_permission(
user_permissions: List[str],
required_domain: str,
required_category: str,
required_action: str,
) -> bool:
"""
Check if user has a permission that satisfies the requirement
Checks:
1. Exact match: domain.category:action
2. Domain-wide: domain.general:action (if category != general)
3. Global admin: admin.general:admin (superuser)
Args:
user_permissions: List of user's permission strings
required_domain: Required domain
required_category: Required category
required_action: Required action
Returns:
True if user has sufficient permission
"""
for perm in user_permissions:
try:
dom, cat, act = _parse_permission(perm)
except ValueError:
continue
# Global admin (admin.general:admin) grants all permissions
if dom == "admin" and cat == "general" and act == "admin":
return True
# Check if this permission covers the requirement
if dom == required_domain:
# Exact category match
if cat == required_category and _action_satisfies(act, required_action):
return True
# Domain-wide permission (general category) covers all categories in domain
if cat == "general" and _action_satisfies(act, required_action):
return True
return False
def _extract_permissions_from_groups(groups: List[str]) -> List[str]:
"""
Extract permission strings from Authentik group names
Authentik groups follow naming: tatlock-{domain}-{category}-{action}
or shorthand: tatlock-{domain}-{action} (implies category=general)
Examples:
- tatlock-control-room-general-admin -> control-room.general:admin
- tatlock-media-viewer -> media.general:viewer (shorthand)
- tatlock-tools-dns-user -> tools.dns:user
Args:
groups: List of Authentik group names
Returns:
List of permission strings
"""
permissions = []
for group in groups:
if not group.startswith("tatlock-"):
continue
# Remove prefix
parts = group[8:].split("-") # Remove "tatlock-"
if len(parts) >= 3:
# Could be domain-category-action or domain-with-hyphen-action
# Try to find a valid action at the end
action = parts[-1]
if action in ACTION_HIERARCHY:
# Check if domain-category or single domain with hyphen
remaining = parts[:-1]
# Try to find known domain (greedy match from start)
for i in range(len(remaining), 0, -1):
potential_domain = "-".join(remaining[:i])
if potential_domain in VALID_DOMAINS:
category_parts = remaining[i:]
category = "-".join(category_parts) if category_parts else DEFAULT_CATEGORY
permissions.append(f"{potential_domain}.{category}:{action}")
break
elif len(parts) == 2:
# Shorthand: domain-action (domain might have hyphen)
action = parts[-1]
if action in ACTION_HIERARCHY:
domain = parts[0]
if domain in VALID_DOMAINS:
permissions.append(f"{domain}.{DEFAULT_CATEGORY}:{action}")
return permissions
def require_permission(
domain: str,
action: str,
category: str = DEFAULT_CATEGORY,
) -> Callable:
"""
Dependency factory for permission-based access control
Creates a FastAPI dependency that checks if the current user has
the required permission. Considers action hierarchy and global admin.
Usage:
@router.get("/containers")
async def list_containers(
user: Dict = Depends(require_permission("control-room", "viewer"))
):
...
@router.delete("/container/{id}")
async def delete_container(
user: Dict = Depends(require_permission("control-room", "admin"))
):
...
Args:
domain: Permission domain (e.g., "control-room", "media")
action: Required action level (viewer, user, editor, admin)
category: Permission category within domain, defaults to "general"
Returns:
FastAPI dependency function
"""
perm_str = f"{domain}.{category}:{action}"
async def permission_checker(
user: Optional[Dict] = Depends(get_current_user)
) -> Dict:
"""Check if user has required permission"""
# If OIDC disabled, allow all (local dev mode)
if not oidc_config.enabled:
logger.debug(f"OIDC disabled - allowing {perm_str}")
return user or {"email": "local", "groups": ["admin"]}
if user is None:
raise HTTPException(
status_code=401,
detail="Authentication required",
headers={"WWW-Authenticate": "Bearer"},
)
# Extract permissions from user's groups
groups = user.get("groups", [])
permissions = _extract_permissions_from_groups(groups)
# Check if user has required permission
if _user_has_permission(permissions, domain, category, action):
logger.debug(f"User {user.get('email')} granted {perm_str}")
return user
# Permission denied
user_email = user.get("email", "unknown")
logger.warning(
f"User {user_email} denied {perm_str} "
f"(groups: {groups}, permissions: {permissions})"
)
raise HTTPException(
status_code=403,
detail=f"Permission required: {perm_str}",
)
return permission_checker
def require_any_permission(*required_permissions: str) -> Callable:
"""
Dependency factory requiring any one of multiple permissions
Useful for endpoints accessible to multiple roles.
Usage:
@router.get("/shared-resource")
async def get_shared(
user: Dict = Depends(require_any_permission(
"control-room:viewer",
"media:viewer",
))
):
...
Args:
*required_permissions: Permission strings (domain.category:action or domain:action)
Returns:
FastAPI dependency function
"""
async def permission_checker(
user: Optional[Dict] = Depends(get_current_user)
) -> Dict:
"""Check if user has any of the required permissions"""
# If OIDC disabled, allow all
if not oidc_config.enabled:
return user or {"email": "local", "groups": ["admin"]}
if user is None:
raise HTTPException(
status_code=401,
detail="Authentication required",
headers={"WWW-Authenticate": "Bearer"},
)
groups = user.get("groups", [])
permissions = _extract_permissions_from_groups(groups)
# Check each required permission
for perm in required_permissions:
try:
dom, cat, act = _parse_permission(perm)
if _user_has_permission(permissions, dom, cat, act):
logger.debug(f"User {user.get('email')} granted via {perm}")
return user
except ValueError:
logger.warning(f"Invalid permission format: {perm}")
continue
# None matched
user_email = user.get("email", "unknown")
logger.warning(
f"User {user_email} denied (required any of: {required_permissions})"
)
raise HTTPException(
status_code=403,
detail=f"One of these permissions required: {', '.join(required_permissions)}",
)
return permission_checker
+211
View File
@@ -0,0 +1,211 @@
"""
Authentication Schemas
Pydantic models for auth request/response payloads.
"""
import uuid
from datetime import datetime
from typing import Optional
from pydantic import Field
from src.shared.base import BaseSchema
class AuthSyncRequest(BaseSchema):
"""
Request payload for POST /auth/sync
The client sends this after obtaining an OIDC token from Authentik.
The access_token is validated against Authentik's userinfo endpoint.
"""
access_token: str = Field(
...,
description="OIDC access token from Authentik",
)
class RoleSchema(BaseSchema):
"""Role information in domain.category:action format"""
id: uuid.UUID = Field(..., description="Role ID")
name: str = Field(..., description="Role name (e.g., 'control-room.general:admin')")
domain: str = Field(..., description="Permission domain (e.g., 'control-room')")
category: str = Field(default="general", description="Permission category (e.g., 'general')")
action: str = Field(..., description="Permission action (e.g., 'admin')")
class UserPreferencesSchema(BaseSchema):
"""User preferences"""
theme: str = Field(default="system", description="Theme preference: system, light, dark")
default_room: str = Field(default="front-hall", description="Default room for housekeeping")
preferences_json: dict = Field(default_factory=dict, description="Extended preferences")
class UserSchema(BaseSchema):
"""User information returned from sync"""
id: uuid.UUID = Field(..., description="Internal user ID")
authentik_id: uuid.UUID = Field(..., description="Authentik user ID")
email: str = Field(..., description="User email")
name: str = Field(..., description="Display name")
avatar_url: Optional[str] = Field(None, description="Profile picture URL")
created_at: datetime = Field(..., description="Account creation timestamp")
last_login: Optional[datetime] = Field(None, description="Last login timestamp")
class AuthSyncResponse(BaseSchema):
"""
Response from POST /auth/sync
Contains the synced user profile, roles, and preferences.
"""
user: UserSchema = Field(..., description="User profile")
roles: list[RoleSchema] = Field(..., description="User's permission roles")
preferences: UserPreferencesSchema = Field(..., description="User preferences")
is_new_user: bool = Field(..., description="True if user was just created")
class TokenInfoSchema(BaseSchema):
"""
Token information from Authentik userinfo endpoint
This is what Authentik returns when validating an access token.
"""
sub: str = Field(..., description="Subject (Authentik user ID)")
email: str = Field(..., description="User email")
name: Optional[str] = Field(None, description="Display name")
preferred_username: Optional[str] = Field(None, description="Username")
groups: list[str] = Field(default_factory=list, description="Group memberships")
picture: Optional[str] = Field(None, description="Profile picture URL")
class UserListItemSchema(BaseSchema):
"""User item for list display"""
id: uuid.UUID = Field(..., description="Internal user ID")
email: str = Field(..., description="User email")
name: str = Field(..., description="Display name")
avatar_url: Optional[str] = Field(None, description="Profile picture URL")
created_at: datetime = Field(..., description="Account creation timestamp")
last_login: Optional[datetime] = Field(None, description="Last login timestamp")
roles: list[str] = Field(default_factory=list, description="Role names")
class UsersListResponse(BaseSchema):
"""Response from GET /auth/users"""
items: list[UserListItemSchema] = Field(..., description="List of users")
total: int = Field(..., description="Total count of users")
class BulkSyncResultSchema(BaseSchema):
"""Result from bulk sync operation"""
created: int = Field(..., description="Number of users created")
updated: int = Field(..., description="Number of users updated")
failed: int = Field(..., description="Number of users that failed to sync")
total_in_authentik: int = Field(..., description="Total users in Authentik")
errors: list[str] = Field(default_factory=list, description="Error messages for failed syncs")
class GroupListItemSchema(BaseSchema):
"""Group item for list display"""
id: uuid.UUID = Field(..., description="Internal group ID")
authentik_id: uuid.UUID = Field(..., description="Authentik group ID")
name: str = Field(..., description="Group name")
is_superuser: bool = Field(default=False, description="Whether group has superuser privileges")
parent_name: Optional[str] = Field(None, description="Parent group name")
member_count: int = Field(default=0, description="Number of users in this group")
synced_at: datetime = Field(..., description="Last sync timestamp")
roles: list[str] = Field(default_factory=list, description="Assigned role names")
class GroupsListResponse(BaseSchema):
"""Response from GET /auth/groups"""
items: list[GroupListItemSchema] = Field(..., description="List of groups")
total: int = Field(..., description="Total count of groups")
class RolesListResponse(BaseSchema):
"""Response from GET /auth/roles"""
items: list[RoleSchema] = Field(..., description="List of all roles")
total: int = Field(..., description="Total count of roles")
class GroupRoleAssignmentResponse(BaseSchema):
"""Response from group role assignment operations"""
group_id: uuid.UUID = Field(..., description="Group ID")
group_name: str = Field(..., description="Group name")
roles: list[str] = Field(..., description="Currently assigned role names")
# =============================================================================
# User Profile (Phase 4)
# =============================================================================
class UserProfileResponse(BaseSchema):
"""Response from GET /users/me - full user profile"""
user: UserSchema = Field(..., description="User profile")
roles: list[RoleSchema] = Field(..., description="User's permission roles")
preferences: UserPreferencesSchema = Field(..., description="User preferences")
class PreferencesUpdateRequest(BaseSchema):
"""Request for PATCH /users/me/preferences"""
theme: Optional[str] = Field(None, description="Theme preference: system, light, dark")
default_room: Optional[str] = Field(None, description="Default room for housekeeping")
preferences_json: Optional[dict] = Field(None, description="Extended preferences (merged)")
# =============================================================================
# API Keys (Phase 4)
# =============================================================================
class ApiKeyCreateRequest(BaseSchema):
"""Request for POST /users/me/api-keys"""
name: str = Field(..., min_length=1, max_length=100, description="Human-readable key name")
scopes: Optional[list[str]] = Field(None, description="Optional scope restriction (role names)")
expires_in_days: Optional[int] = Field(None, ge=1, le=365, description="Days until expiration (optional)")
class ApiKeyCreateResponse(BaseSchema):
"""Response from POST /users/me/api-keys - includes the key (shown only once)"""
id: uuid.UUID = Field(..., description="API key ID")
name: str = Field(..., description="Key name")
key: str = Field(..., description="The API key (shown only once!)")
key_prefix: str = Field(..., description="Key prefix for identification")
scopes: Optional[list[str]] = Field(None, description="Scope restriction")
expires_at: Optional[datetime] = Field(None, description="Expiration timestamp")
created_at: datetime = Field(..., description="Creation timestamp")
class ApiKeySchema(BaseSchema):
"""API key information (without the actual key)"""
id: uuid.UUID = Field(..., description="API key ID")
name: str = Field(..., description="Key name")
key_prefix: str = Field(..., description="Key prefix for identification (e.g., 'tak_abc1')")
scopes: Optional[list[str]] = Field(None, description="Scope restriction")
expires_at: Optional[datetime] = Field(None, description="Expiration timestamp")
last_used_at: Optional[datetime] = Field(None, description="Last usage timestamp")
created_at: datetime = Field(..., description="Creation timestamp")
is_expired: bool = Field(..., description="Whether the key has expired")
class ApiKeysListResponse(BaseSchema):
"""Response from GET /users/me/api-keys"""
items: list[ApiKeySchema] = Field(..., description="List of API keys")
total: int = Field(..., description="Total count of keys")
File diff suppressed because it is too large Load Diff
+8
View File
@@ -0,0 +1,8 @@
"""
Dashboard Domain
Provides dashboard management endpoints including quick links.
"""
from src.domains.dashboard.controller import dashboard_controller
__all__ = ["dashboard_controller"]
+305
View File
@@ -0,0 +1,305 @@
"""
Dashboard Controller
Provides API endpoints for dashboard management including quick links.
"""
from fastapi import APIRouter, HTTPException, Depends, Query
from typing import Dict, Optional
from sqlalchemy.ext.asyncio import AsyncSession
from src.shared.base import BaseController
from src.shared.database import get_async_session
from src.shared.logging import get_logger
from src.domains.auth.oidc import get_current_user, get_optional_user
from src.domains.dashboard.service import get_dashboard_service
from src.domains.dashboard.schemas import (
QuickLinkCreate,
QuickLinkUpdate,
QuickLinkResponse,
QuickLinkListResponse,
QuickLinkReorderRequest,
QuickLinkReorderResponse,
DashboardWidgetCreate,
DashboardWidgetUpdate,
DashboardWidgetResponse,
DashboardWidgetListResponse,
)
logger = get_logger(__name__)
class DashboardController(BaseController):
"""
Controller for dashboard operations
Provides endpoints for:
- Quick links CRUD
- Quick links reordering
- Dashboard widgets management
"""
def __init__(self):
super().__init__(prefix="/dashboard", tags=["Dashboard"])
def create_router(self) -> APIRouter:
"""Create and configure the router"""
router = APIRouter(prefix=self.prefix, tags=self.tags)
service = get_dashboard_service()
# =====================================================================
# Quick Links
# =====================================================================
@router.get(
"/quick-links",
response_model=QuickLinkListResponse,
summary="List quick links"
)
async def list_quick_links(
category: Optional[str] = Query(None, description="Filter by category"),
include_global: bool = Query(True, description="Include global links"),
visible_only: bool = Query(True, description="Only visible links"),
session: AsyncSession = Depends(get_async_session),
user: Optional[Dict] = Depends(get_optional_user),
):
"""
List quick links for the current user
Returns user-specific links plus global links (if include_global=True).
"""
user_id = user.get("sub") if user else None
links = await service.get_quick_links(
session=session,
user_id=user_id,
include_global=include_global,
category=category,
visible_only=visible_only,
)
return QuickLinkListResponse(
links=[QuickLinkResponse.model_validate(link, from_attributes=True) for link in links],
total=len(links)
)
@router.get(
"/quick-links/{link_id}",
response_model=QuickLinkResponse,
summary="Get a quick link"
)
async def get_quick_link(
link_id: int,
session: AsyncSession = Depends(get_async_session),
user: Optional[Dict] = Depends(get_optional_user),
):
"""Get a specific quick link by ID"""
user_id = user.get("sub") if user else None
link = await service.get_quick_link(session, link_id, user_id)
if not link:
raise HTTPException(status_code=404, detail="Quick link not found")
return QuickLinkResponse.model_validate(link, from_attributes=True)
@router.post(
"/quick-links",
response_model=QuickLinkResponse,
status_code=201,
summary="Create a quick link"
)
async def create_quick_link(
data: QuickLinkCreate,
session: AsyncSession = Depends(get_async_session),
user: Dict = Depends(get_current_user),
):
"""
Create a new quick link for the current user
Links are user-specific by default. Admins can create global links
by setting user_id to null.
"""
user_id = user.get("sub")
link = await service.create_quick_link(session, data, user_id)
logger.info(f"Quick link created: {link.title} by user {user.get('preferred_username')}")
return QuickLinkResponse.model_validate(link, from_attributes=True)
@router.put(
"/quick-links/{link_id}",
response_model=QuickLinkResponse,
summary="Update a quick link"
)
async def update_quick_link(
link_id: int,
data: QuickLinkUpdate,
session: AsyncSession = Depends(get_async_session),
user: Dict = Depends(get_current_user),
):
"""Update an existing quick link"""
user_id = user.get("sub")
link = await service.update_quick_link(session, link_id, data, user_id)
if not link:
raise HTTPException(status_code=404, detail="Quick link not found or not authorized")
return QuickLinkResponse.model_validate(link, from_attributes=True)
@router.delete(
"/quick-links/{link_id}",
status_code=204,
summary="Delete a quick link"
)
async def delete_quick_link(
link_id: int,
session: AsyncSession = Depends(get_async_session),
user: Dict = Depends(get_current_user),
):
"""Delete a quick link"""
user_id = user.get("sub")
success = await service.delete_quick_link(session, link_id, user_id)
if not success:
raise HTTPException(status_code=404, detail="Quick link not found or not authorized")
return None
@router.post(
"/quick-links/reorder",
response_model=QuickLinkReorderResponse,
summary="Reorder quick links"
)
async def reorder_quick_links(
data: QuickLinkReorderRequest,
session: AsyncSession = Depends(get_async_session),
user: Dict = Depends(get_current_user),
):
"""
Reorder quick links by providing link IDs in desired order
The position of each link will be set to its index in the provided list.
"""
user_id = user.get("sub")
reordered = await service.reorder_quick_links(session, data.link_ids, user_id)
return QuickLinkReorderResponse(
success=True,
message=f"Reordered {reordered} links",
reordered_count=reordered
)
# =====================================================================
# Dashboard Widgets
# =====================================================================
@router.get(
"/widgets",
response_model=DashboardWidgetListResponse,
summary="List dashboard widgets"
)
async def list_widgets(
include_defaults: bool = Query(True, description="Include default widgets"),
visible_only: bool = Query(True, description="Only visible widgets"),
session: AsyncSession = Depends(get_async_session),
user: Optional[Dict] = Depends(get_optional_user),
):
"""List dashboard widgets for the current user"""
user_id = user.get("sub") if user else None
widgets = await service.get_widgets(
session=session,
user_id=user_id,
include_defaults=include_defaults,
visible_only=visible_only,
)
return DashboardWidgetListResponse(
widgets=[DashboardWidgetResponse.model_validate(w, from_attributes=True) for w in widgets],
total=len(widgets)
)
@router.get(
"/widgets/{widget_id}",
response_model=DashboardWidgetResponse,
summary="Get a dashboard widget"
)
async def get_widget(
widget_id: int,
session: AsyncSession = Depends(get_async_session),
user: Optional[Dict] = Depends(get_optional_user),
):
"""Get a specific dashboard widget by ID"""
user_id = user.get("sub") if user else None
widget = await service.get_widget(session, widget_id, user_id)
if not widget:
raise HTTPException(status_code=404, detail="Widget not found")
return DashboardWidgetResponse.model_validate(widget, from_attributes=True)
@router.post(
"/widgets",
response_model=DashboardWidgetResponse,
status_code=201,
summary="Create a dashboard widget"
)
async def create_widget(
data: DashboardWidgetCreate,
session: AsyncSession = Depends(get_async_session),
user: Dict = Depends(get_current_user),
):
"""Create a new dashboard widget"""
user_id = user.get("sub")
widget = await service.create_widget(session, data, user_id)
logger.info(f"Widget created: {widget.widget_type} by user {user.get('preferred_username')}")
return DashboardWidgetResponse.model_validate(widget, from_attributes=True)
@router.put(
"/widgets/{widget_id}",
response_model=DashboardWidgetResponse,
summary="Update a dashboard widget"
)
async def update_widget(
widget_id: int,
data: DashboardWidgetUpdate,
session: AsyncSession = Depends(get_async_session),
user: Dict = Depends(get_current_user),
):
"""Update an existing dashboard widget"""
user_id = user.get("sub")
widget = await service.update_widget(session, widget_id, data, user_id)
if not widget:
raise HTTPException(status_code=404, detail="Widget not found or not authorized")
return DashboardWidgetResponse.model_validate(widget, from_attributes=True)
@router.delete(
"/widgets/{widget_id}",
status_code=204,
summary="Delete a dashboard widget"
)
async def delete_widget(
widget_id: int,
session: AsyncSession = Depends(get_async_session),
user: Dict = Depends(get_current_user),
):
"""Delete a dashboard widget"""
user_id = user.get("sub")
success = await service.delete_widget(session, widget_id, user_id)
if not success:
raise HTTPException(status_code=404, detail="Widget not found or not authorized")
return None
return router
# Create controller instance
dashboard_controller = DashboardController()
+77
View File
@@ -0,0 +1,77 @@
"""
Dashboard Domain Models
SQLAlchemy models for dashboard-related data.
"""
from datetime import datetime
from sqlalchemy import Column, Integer, String, Boolean, DateTime, Text, ForeignKey
from sqlalchemy.orm import relationship
from src.shared.database import Base
class QuickLink(Base):
"""Quick link for dashboard jump pad"""
__tablename__ = "quick_links"
id = Column(Integer, primary_key=True, index=True)
# Link content
title = Column(String(100), nullable=False)
url = Column(String(500), nullable=False)
icon = Column(String(100), nullable=True) # Icon name or URL
description = Column(String(255), nullable=True)
# Categorization
category = Column(String(50), nullable=True) # e.g., "services", "tools", "docs"
# User association - nullable for global links
user_id = Column(String(255), nullable=True, index=True) # Authentik user ID
# Ordering and display
position = Column(Integer, default=0)
is_visible = Column(Boolean, default=True)
link_type = Column(String(20), default="iframe") # "iframe", "new_tab", etc.
# Styling
color = Column(String(20), nullable=True) # Hex color for the link card
background_color = Column(String(20), nullable=True)
# Metadata
created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
def __repr__(self):
return f"<QuickLink(id={self.id}, title='{self.title}', user_id='{self.user_id}')>"
class DashboardWidget(Base):
"""Dashboard widget configuration"""
__tablename__ = "dashboard_widgets"
id = Column(Integer, primary_key=True, index=True)
# Widget identification
widget_type = Column(String(50), nullable=False) # e.g., "quick_links", "service_status", "weather"
# User association - nullable for default widgets
user_id = Column(String(255), nullable=True, index=True)
# Position and sizing
position_x = Column(Integer, default=0)
position_y = Column(Integer, default=0)
width = Column(Integer, default=1)
height = Column(Integer, default=1)
# Widget-specific configuration (JSON)
config = Column(Text, nullable=True) # JSON string for widget-specific settings
# Display
is_visible = Column(Boolean, default=True)
# Metadata
created_at = Column(DateTime, default=datetime.utcnow)
updated_at = Column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow)
def __repr__(self):
return f"<DashboardWidget(id={self.id}, type='{self.widget_type}', user_id='{self.user_id}')>"
+113
View File
@@ -0,0 +1,113 @@
"""
Dashboard Domain Schemas
Pydantic schemas for dashboard endpoints.
"""
from datetime import datetime
from typing import Optional, List
from pydantic import Field
from src.shared.base import BaseSchema
# Quick Link Schemas
class QuickLinkBase(BaseSchema):
"""Base schema for quick links"""
title: str = Field(..., min_length=1, max_length=100, description="Link title")
url: str = Field(..., min_length=1, max_length=500, description="Link URL")
icon: Optional[str] = Field(None, max_length=100, description="Icon name or URL")
description: Optional[str] = Field(None, max_length=255, description="Link description")
category: Optional[str] = Field(None, max_length=50, description="Link category")
color: Optional[str] = Field(None, max_length=20, description="Hex color for link card")
background_color: Optional[str] = Field(None, max_length=20, description="Background hex color")
class QuickLinkCreate(QuickLinkBase):
"""Schema for creating a quick link"""
position: Optional[int] = Field(0, ge=0, description="Display position")
is_visible: Optional[bool] = Field(True, description="Whether link is visible")
link_type: Optional[str] = Field("iframe", max_length=20, description="Link type: iframe, new_tab")
class QuickLinkUpdate(BaseSchema):
"""Schema for updating a quick link"""
title: Optional[str] = Field(None, min_length=1, max_length=100)
url: Optional[str] = Field(None, min_length=1, max_length=500)
icon: Optional[str] = Field(None, max_length=100)
description: Optional[str] = Field(None, max_length=255)
category: Optional[str] = Field(None, max_length=50)
position: Optional[int] = Field(None, ge=0)
is_visible: Optional[bool] = None
link_type: Optional[str] = Field(None, max_length=20)
color: Optional[str] = Field(None, max_length=20)
background_color: Optional[str] = Field(None, max_length=20)
class QuickLinkResponse(QuickLinkBase):
"""Schema for quick link response"""
id: int
user_id: Optional[str] = None
position: int
is_visible: bool
link_type: str = "iframe"
created_at: datetime
updated_at: datetime
class QuickLinkListResponse(BaseSchema):
"""Response for list of quick links"""
links: List[QuickLinkResponse]
total: int
class QuickLinkReorderRequest(BaseSchema):
"""Request to reorder quick links"""
link_ids: List[int] = Field(..., description="List of link IDs in desired order")
class QuickLinkReorderResponse(BaseSchema):
"""Response after reordering"""
success: bool
message: str
reordered_count: int
# Dashboard Widget Schemas
class DashboardWidgetBase(BaseSchema):
"""Base schema for dashboard widgets"""
widget_type: str = Field(..., min_length=1, max_length=50, description="Widget type identifier")
position_x: int = Field(0, ge=0, description="X position on grid")
position_y: int = Field(0, ge=0, description="Y position on grid")
width: int = Field(1, ge=1, le=12, description="Widget width in grid units")
height: int = Field(1, ge=1, le=12, description="Widget height in grid units")
config: Optional[str] = Field(None, description="JSON config for widget")
is_visible: bool = Field(True, description="Whether widget is visible")
class DashboardWidgetCreate(DashboardWidgetBase):
"""Schema for creating a widget"""
pass
class DashboardWidgetUpdate(BaseSchema):
"""Schema for updating a widget"""
position_x: Optional[int] = Field(None, ge=0)
position_y: Optional[int] = Field(None, ge=0)
width: Optional[int] = Field(None, ge=1, le=12)
height: Optional[int] = Field(None, ge=1, le=12)
config: Optional[str] = None
is_visible: Optional[bool] = None
class DashboardWidgetResponse(DashboardWidgetBase):
"""Schema for widget response"""
id: int
user_id: Optional[str] = None
created_at: datetime
updated_at: datetime
class DashboardWidgetListResponse(BaseSchema):
"""Response for list of widgets"""
widgets: List[DashboardWidgetResponse]
total: int
+319
View File
@@ -0,0 +1,319 @@
"""
Dashboard Domain Service
Business logic for dashboard operations.
"""
from typing import Optional, List
from sqlalchemy import select, update, delete
from sqlalchemy.ext.asyncio import AsyncSession
from src.shared.logging import get_logger
from src.domains.dashboard.models import QuickLink, DashboardWidget
from src.domains.dashboard.schemas import (
QuickLinkCreate,
QuickLinkUpdate,
QuickLinkResponse,
DashboardWidgetCreate,
DashboardWidgetUpdate,
DashboardWidgetResponse,
)
logger = get_logger(__name__)
class DashboardService:
"""Service for dashboard operations"""
# =========================================================================
# Quick Links
# =========================================================================
async def get_quick_links(
self,
session: AsyncSession,
user_id: Optional[str] = None,
include_global: bool = True,
category: Optional[str] = None,
visible_only: bool = True,
) -> List[QuickLink]:
"""
Get quick links for a user
Args:
session: Database session
user_id: User ID to filter by (None for global only)
include_global: Whether to include global links (user_id=None)
category: Optional category filter
visible_only: Only return visible links
"""
conditions = []
if user_id:
if include_global:
from sqlalchemy import or_
conditions.append(or_(QuickLink.user_id == user_id, QuickLink.user_id.is_(None)))
else:
conditions.append(QuickLink.user_id == user_id)
else:
conditions.append(QuickLink.user_id.is_(None))
if category:
conditions.append(QuickLink.category == category)
if visible_only:
conditions.append(QuickLink.is_visible == True)
stmt = select(QuickLink).where(*conditions).order_by(QuickLink.position, QuickLink.id)
result = await session.execute(stmt)
return list(result.scalars().all())
async def get_quick_link(
self,
session: AsyncSession,
link_id: int,
user_id: Optional[str] = None,
) -> Optional[QuickLink]:
"""Get a specific quick link by ID"""
conditions = [QuickLink.id == link_id]
if user_id:
from sqlalchemy import or_
conditions.append(or_(QuickLink.user_id == user_id, QuickLink.user_id.is_(None)))
stmt = select(QuickLink).where(*conditions)
result = await session.execute(stmt)
return result.scalar_one_or_none()
async def create_quick_link(
self,
session: AsyncSession,
data: QuickLinkCreate,
user_id: Optional[str] = None,
) -> QuickLink:
"""Create a new quick link"""
# Get max position for this user
stmt = select(QuickLink.position).where(
QuickLink.user_id == user_id if user_id else QuickLink.user_id.is_(None)
).order_by(QuickLink.position.desc()).limit(1)
result = await session.execute(stmt)
max_pos = result.scalar_one_or_none() or -1
link = QuickLink(
title=data.title,
url=data.url,
icon=data.icon,
description=data.description,
category=data.category,
position=data.position if data.position > 0 else max_pos + 1,
is_visible=data.is_visible,
color=data.color,
background_color=data.background_color,
user_id=user_id,
)
session.add(link)
await session.commit()
await session.refresh(link)
logger.info(f"Created quick link: {link.title} (id={link.id}, user={user_id})")
return link
async def update_quick_link(
self,
session: AsyncSession,
link_id: int,
data: QuickLinkUpdate,
user_id: Optional[str] = None,
) -> Optional[QuickLink]:
"""Update a quick link"""
link = await self.get_quick_link(session, link_id, user_id)
if not link:
return None
# Only allow updating own links or global links for admins
if link.user_id and link.user_id != user_id:
return None
update_data = data.model_dump(exclude_unset=True)
for field, value in update_data.items():
setattr(link, field, value)
await session.commit()
await session.refresh(link)
logger.info(f"Updated quick link: {link.title} (id={link.id})")
return link
async def delete_quick_link(
self,
session: AsyncSession,
link_id: int,
user_id: Optional[str] = None,
) -> bool:
"""Delete a quick link"""
link = await self.get_quick_link(session, link_id, user_id)
if not link:
return False
# Only allow deleting own links
if link.user_id and link.user_id != user_id:
return False
await session.delete(link)
await session.commit()
logger.info(f"Deleted quick link: id={link_id}")
return True
async def reorder_quick_links(
self,
session: AsyncSession,
link_ids: List[int],
user_id: Optional[str] = None,
) -> int:
"""Reorder quick links by updating positions"""
reordered = 0
for position, link_id in enumerate(link_ids):
conditions = [QuickLink.id == link_id]
if user_id:
from sqlalchemy import or_
conditions.append(or_(QuickLink.user_id == user_id, QuickLink.user_id.is_(None)))
stmt = update(QuickLink).where(*conditions).values(position=position)
result = await session.execute(stmt)
reordered += result.rowcount
await session.commit()
logger.info(f"Reordered {reordered} quick links for user {user_id}")
return reordered
# =========================================================================
# Dashboard Widgets
# =========================================================================
async def get_widgets(
self,
session: AsyncSession,
user_id: Optional[str] = None,
include_defaults: bool = True,
visible_only: bool = True,
) -> List[DashboardWidget]:
"""Get dashboard widgets for a user"""
conditions = []
if user_id:
if include_defaults:
from sqlalchemy import or_
conditions.append(or_(DashboardWidget.user_id == user_id, DashboardWidget.user_id.is_(None)))
else:
conditions.append(DashboardWidget.user_id == user_id)
else:
conditions.append(DashboardWidget.user_id.is_(None))
if visible_only:
conditions.append(DashboardWidget.is_visible == True)
stmt = select(DashboardWidget).where(*conditions).order_by(
DashboardWidget.position_y, DashboardWidget.position_x
)
result = await session.execute(stmt)
return list(result.scalars().all())
async def get_widget(
self,
session: AsyncSession,
widget_id: int,
user_id: Optional[str] = None,
) -> Optional[DashboardWidget]:
"""Get a specific widget by ID"""
conditions = [DashboardWidget.id == widget_id]
if user_id:
from sqlalchemy import or_
conditions.append(or_(DashboardWidget.user_id == user_id, DashboardWidget.user_id.is_(None)))
stmt = select(DashboardWidget).where(*conditions)
result = await session.execute(stmt)
return result.scalar_one_or_none()
async def create_widget(
self,
session: AsyncSession,
data: DashboardWidgetCreate,
user_id: Optional[str] = None,
) -> DashboardWidget:
"""Create a new dashboard widget"""
widget = DashboardWidget(
widget_type=data.widget_type,
position_x=data.position_x,
position_y=data.position_y,
width=data.width,
height=data.height,
config=data.config,
is_visible=data.is_visible,
user_id=user_id,
)
session.add(widget)
await session.commit()
await session.refresh(widget)
logger.info(f"Created widget: {widget.widget_type} (id={widget.id}, user={user_id})")
return widget
async def update_widget(
self,
session: AsyncSession,
widget_id: int,
data: DashboardWidgetUpdate,
user_id: Optional[str] = None,
) -> Optional[DashboardWidget]:
"""Update a dashboard widget"""
widget = await self.get_widget(session, widget_id, user_id)
if not widget:
return None
if widget.user_id and widget.user_id != user_id:
return None
update_data = data.model_dump(exclude_unset=True)
for field, value in update_data.items():
setattr(widget, field, value)
await session.commit()
await session.refresh(widget)
logger.info(f"Updated widget: id={widget.id}")
return widget
async def delete_widget(
self,
session: AsyncSession,
widget_id: int,
user_id: Optional[str] = None,
) -> bool:
"""Delete a dashboard widget"""
widget = await self.get_widget(session, widget_id, user_id)
if not widget:
return False
if widget.user_id and widget.user_id != user_id:
return False
await session.delete(widget)
await session.commit()
logger.info(f"Deleted widget: id={widget_id}")
return True
# Singleton instance
_dashboard_service: Optional[DashboardService] = None
def get_dashboard_service() -> DashboardService:
"""Get singleton dashboard service instance"""
global _dashboard_service
if _dashboard_service is None:
_dashboard_service = DashboardService()
return _dashboard_service
+8
View File
@@ -0,0 +1,8 @@
"""
Health Domain
Provides health check and diagnostics endpoints.
"""
from src.domains.health.controller import health_controller
__all__ = ["health_controller"]
+143
View File
@@ -0,0 +1,143 @@
"""
Health Controller
Provides service health and information endpoints
"""
from fastapi import APIRouter, Response
from fastapi.responses import JSONResponse
from src.shared.base import BaseController
from src.shared.config import get_settings
from src.shared.logging import get_logger
from src.shared.database import get_database
logger = get_logger(__name__)
class HealthController(BaseController):
"""
Controller for service health and information
Provides endpoints for:
- Service information and status
- Health checks
"""
def __init__(self):
super().__init__(prefix="", tags=["Health"])
def create_router(self) -> APIRouter:
"""Create and configure the router"""
router = APIRouter(tags=self.tags)
settings = get_settings()
@router.get(
"/",
summary="Service information",
response_class=JSONResponse
)
async def root():
"""
Get service information and health status
Returns basic information about the API service and available endpoints.
"""
logger.debug("Root endpoint accessed")
return {
"service": settings.app_name,
"version": settings.app_version,
"status": "healthy",
"docs": "/docs"
}
@router.get(
"/health",
summary="Health check",
response_class=JSONResponse
)
async def health_check():
"""
Fast health check endpoint for container orchestration
Returns a 200 OK immediately if the service is running.
Does NOT check backend connectivity (use /health/full for that).
Used by Docker, Kubernetes, and load balancers for liveness probes.
"""
return {
"status": "healthy",
"version": settings.app_version
}
@router.get(
"/health/full",
summary="Full health check with database",
)
async def full_health_check(response: Response):
"""
Health check including database connectivity.
Returns 200 OK if database is available, otherwise 503.
"""
import time
start_time = time.time()
# Check database connection
database = get_database()
db_healthy = False
db_error = None
try:
db_healthy = await database.health_check()
except Exception as e:
db_error = str(e)
logger.warning(f"Database health check failed: {db_error}")
elapsed_ms = int((time.time() - start_time) * 1000)
status_code = 200 if db_healthy else 503
response.status_code = status_code
return {
"status": "healthy" if db_healthy else "unhealthy",
"status_code": status_code,
"response_time_ms": elapsed_ms,
"components": {
"database": {
"status": "healthy" if db_healthy else "unhealthy",
"error": db_error
}
}
}
@router.get(
"/health/diagnostics",
summary="Detailed system diagnostics",
)
async def diagnostics():
"""
System diagnostics with service information.
"""
import time
start_time = time.time()
diagnostics = {
"timestamp": time.time(),
"service": {
"name": settings.app_name,
"version": settings.app_version,
"purpose": "Infrastructure management and tools API"
},
"configuration": {
"cors_origins": settings.cors_origins[:2] if len(settings.cors_origins) > 2 else settings.cors_origins
}
}
elapsed_ms = int((time.time() - start_time) * 1000)
diagnostics["response_time_ms"] = elapsed_ms
return diagnostics
return router
# Create controller instance
health_controller = HealthController()
+8
View File
@@ -0,0 +1,8 @@
"""
Housekeeping Domain
Provides home automation endpoints via Home Assistant.
"""
from src.domains.housekeeping.controller import housekeeping_controller
__all__ = ["housekeeping_controller"]
+645
View File
@@ -0,0 +1,645 @@
"""
Housekeeping Controller
Provides API endpoints for home automation via Home Assistant.
Designed for the Tatlock Housekeeper agent and other consumers.
"""
import asyncio
from fastapi import APIRouter, HTTPException, Query, Depends
from typing import List, Dict, Any, Optional
from pydantic import BaseModel, Field
from src.shared.base import BaseController
from src.shared.clients import get_homeassistant_client
from src.shared.logging import get_logger
from src.domains.auth.oidc import get_admin_user
logger = get_logger(__name__)
# Pydantic Schemas
class Device(BaseModel):
"""Device/entity information"""
entity_id: str
name: str
domain: str
area: Optional[str] = None
state: str
attributes: Dict[str, Any] = {}
last_changed: Optional[str] = None
class DeviceListResponse(BaseModel):
"""Response for device listing"""
devices: List[Device]
class DeviceDetailResponse(Device):
"""Detailed device response"""
pass
class Area(BaseModel):
"""Area/room information"""
id: str
name: str
class AreaListResponse(BaseModel):
"""Response for area listing"""
areas: List[Area]
class DeviceControlRequest(BaseModel):
"""Request to control a device"""
action: str = Field(..., description="Action: turn_on, turn_off, or toggle")
brightness: Optional[int] = Field(None, ge=0, le=255)
color_temp: Optional[int] = None
rgb_color: Optional[List[int]] = None
class Config:
extra = "allow"
class DeviceControlResponse(BaseModel):
"""Response from device control"""
success: bool
entity_id: str
new_state: Optional[str] = None
message: str
class Scene(BaseModel):
"""Scene information"""
id: str
name: str
class SceneListResponse(BaseModel):
"""Response for scene listing"""
scenes: List[Scene]
class SceneActivateResponse(BaseModel):
"""Response from scene activation"""
success: bool
scene_id: str
message: str
class Script(BaseModel):
"""Script information"""
id: str
name: str
class ScriptListResponse(BaseModel):
"""Response for script listing"""
scripts: List[Script]
class ScriptRunRequest(BaseModel):
"""Request to run a script"""
variables: Optional[Dict[str, Any]] = None
class ScriptRunResponse(BaseModel):
"""Response from script execution"""
success: bool
script_id: str
message: str
class Automation(BaseModel):
"""Automation information"""
id: str
name: str
enabled: bool
class AutomationListResponse(BaseModel):
"""Response for automation listing"""
automations: List[Automation]
class AutomationToggleRequest(BaseModel):
"""Request to toggle automation"""
enabled: bool
class AutomationToggleResponse(BaseModel):
"""Response from automation toggle"""
success: bool
automation_id: str
enabled: bool
message: str
class HistoryEntry(BaseModel):
"""Single history entry"""
state: str
timestamp: str
attributes: Dict[str, Any] = {}
class HistoryResponse(BaseModel):
"""Response for history query"""
entity_id: str
history: List[HistoryEntry]
class HealthResponse(BaseModel):
"""Health check response"""
status: str
connected: bool
platform: str
version: Optional[str] = None
error: Optional[str] = None
class ErrorResponse(BaseModel):
"""Standard error response"""
error: bool = True
code: str
message: str
class HousekeepingController(BaseController):
"""
Controller for home automation operations
Provides endpoints for:
- Device discovery and control
- Scene activation
- Script execution
- Automation management
- State history
"""
def __init__(self):
super().__init__(prefix="/housekeeping", tags=["Housekeeping"])
def create_router(self) -> APIRouter:
"""Create and configure the router"""
router = APIRouter(prefix=self.prefix, tags=self.tags)
@router.get(
"/health",
response_model=HealthResponse,
summary="Home automation health check"
)
async def get_health():
"""Check Home Assistant connection health"""
ha = get_homeassistant_client()
return await ha.health_check()
@router.get(
"/devices",
response_model=DeviceListResponse,
summary="List available devices"
)
async def list_devices(
domain: Optional[str] = Query(None, description="Filter by domain (light, switch, climate, etc.)"),
area: Optional[str] = Query(None, description="Filter by area/room name")
):
"""List all available devices with optional filtering"""
ha = get_homeassistant_client()
try:
states = await ha.get_states()
excluded_domains = {
"zone", "person", "device_tracker", "sun", "weather",
"persistent_notification", "update", "binary_sensor", "sensor",
"conversation", "calendar", "button", "number", "select",
"text", "time", "date", "datetime", "image", "tts", "stt"
}
devices = []
for state in states:
entity_id = state.get("entity_id", "")
entity_domain = entity_id.split(".")[0] if "." in entity_id else ""
if entity_domain in excluded_domains:
continue
if domain and entity_domain != domain:
continue
device_area = state.get("attributes", {}).get("area_id")
if area and device_area and area.lower() not in device_area.lower():
continue
device = Device(
entity_id=entity_id,
name=state.get("attributes", {}).get("friendly_name", entity_id),
domain=entity_domain,
area=device_area,
state=state.get("state", "unknown"),
attributes=state.get("attributes", {}),
last_changed=state.get("last_changed")
)
devices.append(device)
return DeviceListResponse(devices=devices)
except Exception as e:
logger.error(f"Failed to list devices: {e}")
raise HTTPException(
status_code=500,
detail={"error": True, "code": "CONNECTION_ERROR", "message": str(e)}
)
@router.get(
"/devices/{entity_id:path}",
response_model=DeviceDetailResponse,
responses={404: {"model": ErrorResponse}}
)
async def get_device(entity_id: str):
"""Get detailed state of a specific device"""
ha = get_homeassistant_client()
try:
state = await ha.get_state(entity_id)
if not state:
raise HTTPException(
status_code=404,
detail={"error": True, "code": "DEVICE_NOT_FOUND",
"message": f"Device {entity_id} not found"}
)
entity_domain = entity_id.split(".")[0] if "." in entity_id else ""
return DeviceDetailResponse(
entity_id=entity_id,
name=state.get("attributes", {}).get("friendly_name", entity_id),
domain=entity_domain,
area=state.get("attributes", {}).get("area_id"),
state=state.get("state", "unknown"),
attributes=state.get("attributes", {}),
last_changed=state.get("last_changed")
)
except HTTPException:
raise
except Exception as e:
logger.error(f"Failed to get device {entity_id}: {e}")
raise HTTPException(
status_code=500,
detail={"error": True, "code": "CONNECTION_ERROR", "message": str(e)}
)
@router.get(
"/areas",
response_model=AreaListResponse,
summary="List areas/rooms"
)
async def list_areas():
"""List all configured areas/rooms in Home Assistant"""
ha = get_homeassistant_client()
try:
areas = await ha.get_areas()
return AreaListResponse(
areas=[Area(id=a["id"], name=a["name"]) for a in areas]
)
except Exception as e:
logger.error(f"Failed to list areas: {e}")
raise HTTPException(
status_code=500,
detail={"error": True, "code": "CONNECTION_ERROR", "message": str(e)}
)
@router.post(
"/devices/{entity_id:path}/control",
response_model=DeviceControlResponse,
responses={404: {"model": ErrorResponse}, 400: {"model": ErrorResponse}}
)
async def control_device(
entity_id: str,
request: DeviceControlRequest,
user: Dict = Depends(get_admin_user)
):
"""Control a device (turn on, turn off, toggle, or set attributes)"""
ha = get_homeassistant_client()
valid_actions = ["turn_on", "turn_off", "toggle"]
if request.action not in valid_actions:
raise HTTPException(
status_code=400,
detail={"error": True, "code": "INVALID_ACTION",
"message": f"Invalid action '{request.action}'. Must be one of: {', '.join(valid_actions)}"}
)
try:
current_state = await ha.get_state(entity_id)
if not current_state:
raise HTTPException(
status_code=404,
detail={"error": True, "code": "DEVICE_NOT_FOUND",
"message": f"Device {entity_id} not found"}
)
attributes = {}
if request.brightness is not None:
attributes["brightness"] = request.brightness
if request.color_temp is not None:
attributes["color_temp"] = request.color_temp
if request.rgb_color is not None:
attributes["rgb_color"] = request.rgb_color
extra_fields = request.model_dump(exclude={"action", "brightness", "color_temp", "rgb_color"})
for key, value in extra_fields.items():
if value is not None:
attributes[key] = value
if request.action == "turn_on":
await ha.turn_on(entity_id, **attributes)
elif request.action == "turn_off":
await ha.turn_off(entity_id)
else:
await ha.toggle(entity_id)
await asyncio.sleep(0.3)
new_state = await ha.get_state(entity_id)
logger.info(f"Device {entity_id} controlled: {request.action} by {user.get('preferred_username', 'unknown')}")
return DeviceControlResponse(
success=True,
entity_id=entity_id,
new_state=new_state.get("state") if new_state else None,
message=f"Device {request.action} successful"
)
except HTTPException:
raise
except Exception as e:
logger.error(f"Failed to control device {entity_id}: {e}")
raise HTTPException(
status_code=500,
detail={"error": True, "code": "CONNECTION_ERROR", "message": str(e)}
)
@router.get(
"/scenes",
response_model=SceneListResponse,
summary="List available scenes"
)
async def list_scenes():
"""List all available scenes in Home Assistant"""
ha = get_homeassistant_client()
try:
states = await ha.get_states()
scenes = [
Scene(
id=s["entity_id"],
name=s.get("attributes", {}).get("friendly_name", s["entity_id"])
)
for s in states
if s["entity_id"].startswith("scene.")
]
return SceneListResponse(scenes=scenes)
except Exception as e:
logger.error(f"Failed to list scenes: {e}")
raise HTTPException(
status_code=500,
detail={"error": True, "code": "CONNECTION_ERROR", "message": str(e)}
)
@router.post(
"/scenes/{scene_id:path}/activate",
response_model=SceneActivateResponse,
responses={404: {"model": ErrorResponse}}
)
async def activate_scene(
scene_id: str,
user: Dict = Depends(get_admin_user)
):
"""Activate a scene"""
ha = get_homeassistant_client()
try:
state = await ha.get_state(scene_id)
if not state:
raise HTTPException(
status_code=404,
detail={"error": True, "code": "SCENE_NOT_FOUND",
"message": f"Scene {scene_id} not found"}
)
await ha.activate_scene(scene_id)
logger.info(f"Scene {scene_id} activated by {user.get('preferred_username', 'unknown')}")
return SceneActivateResponse(
success=True,
scene_id=scene_id,
message="Scene activated"
)
except HTTPException:
raise
except Exception as e:
logger.error(f"Failed to activate scene {scene_id}: {e}")
raise HTTPException(
status_code=500,
detail={"error": True, "code": "CONNECTION_ERROR", "message": str(e)}
)
@router.get(
"/scripts",
response_model=ScriptListResponse,
summary="List available scripts"
)
async def list_scripts():
"""List all available scripts/sequences in Home Assistant"""
ha = get_homeassistant_client()
try:
states = await ha.get_states()
scripts = [
Script(
id=s["entity_id"],
name=s.get("attributes", {}).get("friendly_name", s["entity_id"])
)
for s in states
if s["entity_id"].startswith("script.")
]
return ScriptListResponse(scripts=scripts)
except Exception as e:
logger.error(f"Failed to list scripts: {e}")
raise HTTPException(
status_code=500,
detail={"error": True, "code": "CONNECTION_ERROR", "message": str(e)}
)
@router.post(
"/scripts/{script_id:path}/run",
response_model=ScriptRunResponse,
responses={404: {"model": ErrorResponse}}
)
async def run_script(
script_id: str,
request: Optional[ScriptRunRequest] = None,
user: Dict = Depends(get_admin_user)
):
"""Execute a script with optional variables"""
ha = get_homeassistant_client()
try:
state = await ha.get_state(script_id)
if not state:
raise HTTPException(
status_code=404,
detail={"error": True, "code": "SCRIPT_NOT_FOUND",
"message": f"Script {script_id} not found"}
)
variables = request.variables if request else None
await ha.run_script(script_id, variables)
logger.info(f"Script {script_id} executed by {user.get('preferred_username', 'unknown')}")
return ScriptRunResponse(
success=True,
script_id=script_id,
message="Script executed"
)
except HTTPException:
raise
except Exception as e:
logger.error(f"Failed to run script {script_id}: {e}")
raise HTTPException(
status_code=500,
detail={"error": True, "code": "CONNECTION_ERROR", "message": str(e)}
)
@router.get(
"/automations",
response_model=AutomationListResponse,
summary="List automations"
)
async def list_automations():
"""List all automations with their enabled/disabled status"""
ha = get_homeassistant_client()
try:
states = await ha.get_states()
automations = [
Automation(
id=s["entity_id"],
name=s.get("attributes", {}).get("friendly_name", s["entity_id"]),
enabled=s.get("state") == "on"
)
for s in states
if s["entity_id"].startswith("automation.")
]
return AutomationListResponse(automations=automations)
except Exception as e:
logger.error(f"Failed to list automations: {e}")
raise HTTPException(
status_code=500,
detail={"error": True, "code": "CONNECTION_ERROR", "message": str(e)}
)
@router.post(
"/automations/{automation_id:path}/toggle",
response_model=AutomationToggleResponse,
responses={404: {"model": ErrorResponse}}
)
async def toggle_automation(
automation_id: str,
request: AutomationToggleRequest,
user: Dict = Depends(get_admin_user)
):
"""Enable or disable an automation"""
ha = get_homeassistant_client()
try:
state = await ha.get_state(automation_id)
if not state:
raise HTTPException(
status_code=404,
detail={"error": True, "code": "AUTOMATION_NOT_FOUND",
"message": f"Automation {automation_id} not found"}
)
if request.enabled:
await ha.enable_automation(automation_id)
else:
await ha.disable_automation(automation_id)
logger.info(f"Automation {automation_id} {'enabled' if request.enabled else 'disabled'} by {user.get('preferred_username', 'unknown')}")
return AutomationToggleResponse(
success=True,
automation_id=automation_id,
enabled=request.enabled,
message=f"Automation {'enabled' if request.enabled else 'disabled'}"
)
except HTTPException:
raise
except Exception as e:
logger.error(f"Failed to toggle automation {automation_id}: {e}")
raise HTTPException(
status_code=500,
detail={"error": True, "code": "CONNECTION_ERROR", "message": str(e)}
)
@router.get(
"/history",
response_model=HistoryResponse,
responses={400: {"model": ErrorResponse}}
)
async def get_history(
entity_id: str = Query(..., description="Entity ID to get history for"),
hours: int = Query(24, ge=1, le=168, description="Hours of history (1-168)")
):
"""Get state history for a device"""
ha = get_homeassistant_client()
try:
history_data = await ha.get_history(entity_id, hours)
history_entries = []
if history_data and len(history_data) > 0:
for entry in history_data[0]:
history_entries.append(HistoryEntry(
state=entry.get("state", "unknown"),
timestamp=entry.get("last_changed", ""),
attributes=entry.get("attributes", {})
))
return HistoryResponse(
entity_id=entity_id,
history=history_entries
)
except Exception as e:
logger.error(f"Failed to get history for {entity_id}: {e}")
raise HTTPException(
status_code=500,
detail={"error": True, "code": "CONNECTION_ERROR", "message": str(e)}
)
return router
# Create controller instance
housekeeping_controller = HousekeepingController()
+8
View File
@@ -0,0 +1,8 @@
"""
Infrastructure Domain
Provides infrastructure management endpoints for Docker/Portainer and NPM.
"""
from src.domains.infrastructure.controller import infrastructure_controller
__all__ = ["infrastructure_controller"]
File diff suppressed because it is too large Load Diff
+8
View File
@@ -0,0 +1,8 @@
"""
Static Domain
Serves static files for widgets and other frontend assets.
"""
from src.domains.static.controller import static_controller
__all__ = ["static_controller"]
+116
View File
@@ -0,0 +1,116 @@
"""
Static Files Controller
Serves static files for widgets and other frontend assets.
"""
from fastapi import APIRouter
from fastapi.responses import FileResponse, HTMLResponse
from pathlib import Path
from src.shared.base import BaseController
from src.shared.logging import get_logger
logger = get_logger(__name__)
class StaticController(BaseController):
"""
Controller for serving static files
Provides endpoints for:
- Organizr widgets
- Other static assets
"""
def __init__(self):
super().__init__(prefix="/static", tags=["Static"])
# Static files are at the root of the project
self.static_dir = Path(__file__).parent.parent.parent.parent / "static"
def create_router(self) -> APIRouter:
"""Create and configure the router"""
router = APIRouter(prefix=self.prefix, tags=self.tags)
@router.get(
"/widgets/{filename}",
response_class=HTMLResponse,
summary="Get widget file"
)
async def get_widget(filename: str):
"""
Serve widget HTML files
Args:
filename: Widget filename (e.g., service-control.html)
Returns:
HTML file content
"""
widget_path = self.static_dir / "widgets" / filename
if not widget_path.exists():
return HTMLResponse(
content=f"<h1>404 - Widget not found</h1><p>{filename}</p>",
status_code=404
)
if not widget_path.is_file():
return HTMLResponse(
content=f"<h1>400 - Not a file</h1>",
status_code=400
)
# Security: Ensure the path is within the static directory
try:
widget_path.resolve().relative_to(self.static_dir.resolve())
except ValueError:
return HTMLResponse(
content=f"<h1>403 - Forbidden</h1>",
status_code=403
)
logger.info(f"Serving widget: {filename}")
return FileResponse(
widget_path,
media_type="text/html",
headers={
"Cache-Control": "no-cache, no-store, must-revalidate",
"Pragma": "no-cache",
"Expires": "0"
}
)
@router.get(
"/widgets",
summary="List available widgets"
)
async def list_widgets():
"""
List all available widget files
Returns:
List of widget filenames
"""
widgets_dir = self.static_dir / "widgets"
if not widgets_dir.exists():
return {"widgets": [], "message": "Widgets directory not found"}
widgets = []
for file in widgets_dir.glob("*.html"):
widgets.append({
"name": file.name,
"url": f"/static/widgets/{file.name}",
"size": file.stat().st_size
})
return {
"widgets": widgets,
"count": len(widgets)
}
return router
# Create controller instance
static_controller = StaticController()
+16
View File
@@ -0,0 +1,16 @@
"""
Tools Domain
Provides utility tool endpoints including DNS lookups and system stats.
"""
from src.domains.tools.controller import tools_controller
from src.domains.tools.dns import DNSService, DNSQueryError
from src.domains.tools.system import SystemStatsService, SystemStatsResponse
__all__ = [
"tools_controller",
"DNSService",
"DNSQueryError",
"SystemStatsService",
"SystemStatsResponse",
]
+282
View File
@@ -0,0 +1,282 @@
"""
Tools Controller
Provides utility tool endpoints including:
- DNS lookups
- System stats
- Environment data (weather, forecast, sun times)
"""
from typing import Dict, Optional
from fastapi import APIRouter, HTTPException, status, Depends
from src.shared.base import BaseController
from src.shared.logging import get_logger
from src.domains.tools.dns.schemas import DNSLookupRequest, DNSLookupResponse
from src.domains.tools.dns.service import DNSService
from src.domains.tools.dns.exceptions import DNSQueryError
from src.domains.tools.system.schemas import SystemStatsResponse
from src.domains.tools.system.service import SystemStatsService
from src.domains.tools.environment.schemas import EnvironmentResponse
from src.domains.tools.environment.service import EnvironmentService
from src.domains.tools.news.schemas import NewsResponse
from src.domains.tools.news.service import NewsService
from src.domains.auth.oidc import get_optional_user
logger = get_logger(__name__)
class ToolsController(BaseController):
"""
Controller for utility tools
Provides endpoints for:
- DNS lookups
- System stats
- Environment data (weather, forecast, sun times)
"""
def __init__(self):
super().__init__(prefix="/tools", tags=["Tools"])
self.dns_service = DNSService()
self.system_stats_service = SystemStatsService()
self.environment_service = EnvironmentService()
self.news_service = NewsService()
def create_router(self) -> APIRouter:
"""Create and configure the router"""
router = APIRouter(prefix=self.prefix, tags=self.tags)
@router.post(
"/dns/lookup",
response_model=DNSLookupResponse,
status_code=status.HTTP_200_OK,
summary="Perform DNS lookup",
description="""
Perform DNS lookups for various record types.
Uses dnspython for reliable DNS queries with support for multiple record types
and custom nameservers. Perfect for troubleshooting DNS issues and checking
domain configurations.
**Supported Record Types:**
- A: IPv4 address records
- AAAA: IPv6 address records
- MX: Mail exchange records
- TXT: Text records (SPF, DKIM, etc.)
- CNAME: Canonical name records
- NS: Nameserver records
- SOA: Start of authority records
- PTR: Pointer records (reverse DNS)
- CAA: Certification authority authorization
- SRV: Service records
**Features:**
- Custom nameserver support (e.g., 8.8.8.8, 1.1.1.1)
- Query time measurement
- Detailed error messages
**Rate Limiting:** None (internal network use only)
"""
)
async def dns_lookup(request: DNSLookupRequest) -> DNSLookupResponse:
"""
Perform DNS lookup for a domain
Args:
request: DNS lookup request with domain, record type, and optional nameserver
Returns:
DNS lookup results with records and metadata
Raises:
HTTPException: 400 for invalid queries, 500 for processing errors
"""
try:
logger.info(f"Received DNS lookup request for: {request.domain} ({request.record_type})")
result = await self.dns_service.lookup(request)
return result
except DNSQueryError as e:
logger.warning(f"DNS query error: {str(e)}")
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"DNS query failed: {str(e)}"
)
except Exception as e:
logger.error(f"Unexpected error during DNS lookup: {str(e)}", exc_info=True)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="An unexpected error occurred during DNS lookup"
)
@router.get(
"/system/stats",
response_model=SystemStatsResponse,
status_code=status.HTTP_200_OK,
summary="Get host system statistics",
description="""
Get real-time host system resource statistics.
Returns CPU, memory, disk, network, and GPU/VRAM usage for the host machine
(not Docker container metrics).
**Metrics Returned:**
- **CPU:** Usage percentage, core count, load averages
- **Memory:** Usage percentage, total/used/available bytes
- **Disk:** Usage percentage, total/used/free bytes (root partition)
- **Network:** Total bytes sent/received
- **GPU:** VRAM usage (if NVIDIA GPU available via nvidia-smi)
**Use Cases:**
- Dashboard system monitoring widgets
- Health checks and alerting
- Capacity planning
"""
)
async def get_system_stats() -> SystemStatsResponse:
"""
Get current host system statistics
Returns:
System statistics including CPU, memory, disk, network, and GPU
Raises:
HTTPException: 500 for processing errors
"""
try:
logger.info("Fetching system stats")
result = await self.system_stats_service.get_stats()
return result
except Exception as e:
logger.error(f"Failed to get system stats: {str(e)}", exc_info=True)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Failed to collect system stats: {str(e)}"
)
@router.get(
"/environment",
response_model=EnvironmentResponse,
status_code=status.HTTP_200_OK,
summary="Get environment data",
description="""
Get current environment data including weather, forecast, and sun times.
Fetches data from the Qdrant volatile collection for the authenticated user.
Falls back to 'default' user if not authenticated.
**Data Returned:**
- **Weather:** Current temperature, conditions, humidity, wind
- **Forecast:** Multi-day weather outlook
- **Sun Times:** Sunrise, sunset, daylight duration
- **Air Quality:** AQI and pollutant levels (if available)
**Data Source:** Qdrant volatile_{user} collection
**Use Cases:**
- Dashboard environment widgets
- Home automation context
- Weather-based automations
"""
)
async def get_environment(
user: Optional[Dict] = Depends(get_optional_user),
) -> EnvironmentResponse:
"""
Get current environment data
Args:
user: Optional authenticated user from OIDC
Returns:
Environment data including weather, forecast, sun times
Raises:
HTTPException: 500 for processing errors
"""
try:
# Get user identifier from OIDC claims, fallback to 'default'
user_id = "default"
if user:
user_id = user.get("preferred_username") or user.get("sub", "default")
# Strip email domain if present (e.g., "user@example.com" -> "user")
if "@" in user_id:
user_id = user_id.split("@")[0]
logger.info(f"Fetching environment data for user: {user_id}")
result = await self.environment_service.get_current(user_id)
return result
except Exception as e:
logger.error(f"Failed to get environment data: {str(e)}", exc_info=True)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Failed to fetch environment data: {str(e)}"
)
@router.get(
"/news",
response_model=NewsResponse,
status_code=status.HTTP_200_OK,
summary="Get news headlines",
description="""
Get news headlines for the authenticated user.
Fetches news data from the Qdrant volatile collection for the authenticated user.
Falls back to 'default' user if not authenticated.
**Data Returned:**
- **Headlines:** List of news headlines with title, description, source, url
- **Category:** News category (general, technology, etc.)
- **Sources:** List of news sources
**Data Source:** Qdrant volatile_{user} collection (news namespace)
**Use Cases:**
- Dashboard news ticker
- News feed widgets
- Information display
"""
)
async def get_news(
user: Optional[Dict] = Depends(get_optional_user),
) -> NewsResponse:
"""
Get news headlines
Args:
user: Optional authenticated user from OIDC
Returns:
News headlines response
Raises:
HTTPException: 500 for processing errors
"""
try:
# Get user identifier from OIDC claims, fallback to 'default'
user_id = "default"
if user:
user_id = user.get("preferred_username") or user.get("sub", "default")
# Strip email domain if present (e.g., "user@example.com" -> "user")
if "@" in user_id:
user_id = user_id.split("@")[0]
logger.info(f"Fetching news data for user: {user_id}")
result = await self.news_service.get_news(user_id)
return result
except Exception as e:
logger.error(f"Failed to get news data: {str(e)}", exc_info=True)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"Failed to fetch news data: {str(e)}"
)
return router
# Create controller instance
tools_controller = ToolsController()
+16
View File
@@ -0,0 +1,16 @@
"""
DNS Tools Module
Provides DNS lookup functionality.
"""
from src.domains.tools.dns.schemas import DNSLookupRequest, DNSLookupResponse, DNSRecord
from src.domains.tools.dns.service import DNSService
from src.domains.tools.dns.exceptions import DNSQueryError
__all__ = [
"DNSLookupRequest",
"DNSLookupResponse",
"DNSRecord",
"DNSService",
"DNSQueryError",
]
+8
View File
@@ -0,0 +1,8 @@
"""
DNS Exceptions
"""
class DNSQueryError(Exception):
"""Raised when a DNS query fails"""
pass
+94
View File
@@ -0,0 +1,94 @@
"""
Pydantic schemas for DNS lookup module
"""
from pydantic import Field
from typing import Optional, List
from datetime import datetime
from src.shared.base import BaseSchema
class DNSLookupRequest(BaseSchema):
"""Request model for DNS lookup"""
domain: str = Field(
...,
description="The domain name to lookup",
examples=["example.com", "google.com"],
min_length=1,
max_length=255
)
record_type: str = Field(
default="A",
description="DNS record type to query (A, AAAA, MX, TXT, CNAME, NS, SOA, PTR, CAA)",
examples=["A", "AAAA", "MX", "TXT", "CNAME"]
)
nameserver: Optional[str] = Field(
default=None,
description="Optional nameserver to use for the query (e.g., 8.8.8.8, 1.1.1.1)",
examples=["8.8.8.8", "1.1.1.1", "9.9.9.9"]
)
class DNSRecord(BaseSchema):
"""Single DNS record result"""
value: str = Field(
...,
description="The DNS record value"
)
ttl: Optional[int] = Field(
default=None,
description="Time to live in seconds"
)
priority: Optional[int] = Field(
default=None,
description="Priority (for MX records)"
)
class DNSLookupResponse(BaseSchema):
"""Response model for DNS lookup"""
domain: str = Field(
...,
description="The queried domain name"
)
record_type: str = Field(
...,
description="DNS record type queried"
)
records: List[DNSRecord] = Field(
...,
description="List of DNS records found"
)
nameserver_used: Optional[str] = Field(
default=None,
description="Nameserver used for the query"
)
query_time_ms: float = Field(
...,
description="Query execution time in milliseconds"
)
queried_at: datetime = Field(
...,
description="UTC timestamp when query was executed"
)
success: bool = Field(
...,
description="Whether the query was successful"
)
error_message: Optional[str] = Field(
default=None,
description="Error message if query failed"
)
+188
View File
@@ -0,0 +1,188 @@
"""
DNS Lookup Service
Provides DNS query functionality using dnspython library.
"""
import time
from datetime import datetime, timezone
from typing import Optional
import dns.resolver
import dns.exception
from src.shared.logging import get_logger
from src.domains.tools.dns.schemas import DNSLookupRequest, DNSLookupResponse, DNSRecord
from src.domains.tools.dns.exceptions import DNSQueryError
logger = get_logger(__name__)
class DNSService:
"""
Service for performing DNS lookups
Uses dnspython for reliable DNS queries with support for
various record types and custom nameservers.
"""
SUPPORTED_RECORD_TYPES = [
"A", "AAAA", "MX", "TXT", "CNAME", "NS", "SOA", "PTR", "CAA", "SRV"
]
def __init__(self):
"""Initialize DNS service"""
self.resolver = dns.resolver.Resolver()
self.resolver.timeout = 5.0
self.resolver.lifetime = 10.0
async def lookup(self, request: DNSLookupRequest) -> DNSLookupResponse:
"""
Perform DNS lookup for the specified domain and record type
Args:
request: DNS lookup request with domain, record type, and optional nameserver
Returns:
DNSLookupResponse with query results
Raises:
DNSQueryError: If the DNS query fails
"""
start_time = time.time()
record_type = request.record_type.upper()
if record_type not in self.SUPPORTED_RECORD_TYPES:
raise DNSQueryError(
f"Unsupported record type: {record_type}. "
f"Supported types: {', '.join(self.SUPPORTED_RECORD_TYPES)}"
)
resolver = dns.resolver.Resolver()
resolver.timeout = 5.0
resolver.lifetime = 10.0
nameserver_used = None
if request.nameserver:
resolver.nameservers = [request.nameserver]
nameserver_used = request.nameserver
logger.info(f"Using custom nameserver: {request.nameserver}")
else:
nameserver_used = resolver.nameservers[0] if resolver.nameservers else "system"
try:
logger.info(f"Performing DNS lookup: {request.domain} ({record_type})")
answers = resolver.resolve(request.domain, record_type)
records = []
for rdata in answers:
record = self._parse_record(rdata, record_type)
if record:
records.append(record)
query_time_ms = (time.time() - start_time) * 1000
logger.info(
f"DNS lookup successful: {request.domain} ({record_type}) - "
f"Found {len(records)} records in {query_time_ms:.2f}ms"
)
return DNSLookupResponse(
domain=request.domain,
record_type=record_type,
records=records,
nameserver_used=nameserver_used,
query_time_ms=round(query_time_ms, 2),
queried_at=datetime.now(timezone.utc),
success=True,
error_message=None
)
except dns.resolver.NXDOMAIN:
error_msg = f"Domain not found: {request.domain}"
logger.warning(error_msg)
return self._error_response(request, nameserver_used, start_time, error_msg)
except dns.resolver.NoAnswer:
error_msg = f"No {record_type} records found for {request.domain}"
logger.warning(error_msg)
return self._error_response(request, nameserver_used, start_time, error_msg)
except dns.resolver.Timeout:
error_msg = f"DNS query timeout for {request.domain}"
logger.error(error_msg)
return self._error_response(request, nameserver_used, start_time, error_msg)
except dns.exception.DNSException as e:
error_msg = f"DNS error: {str(e)}"
logger.error(f"DNS query failed for {request.domain}: {e}")
return self._error_response(request, nameserver_used, start_time, error_msg)
except Exception as e:
error_msg = f"Unexpected error: {str(e)}"
logger.error(f"Unexpected error during DNS lookup: {e}", exc_info=True)
return self._error_response(request, nameserver_used, start_time, error_msg)
def _parse_record(self, rdata, record_type: str) -> Optional[DNSRecord]:
"""Parse DNS record data into DNSRecord schema"""
try:
if record_type == "A" or record_type == "AAAA":
return DNSRecord(value=str(rdata), ttl=None)
elif record_type == "MX":
return DNSRecord(
value=str(rdata.exchange),
priority=rdata.preference,
ttl=None
)
elif record_type == "TXT":
txt_value = " ".join([s.decode() if isinstance(s, bytes) else str(s) for s in rdata.strings])
return DNSRecord(value=txt_value, ttl=None)
elif record_type in ["CNAME", "NS", "PTR"]:
return DNSRecord(value=str(rdata.target), ttl=None)
elif record_type == "SOA":
soa_value = f"mname={rdata.mname} rname={rdata.rname} serial={rdata.serial}"
return DNSRecord(value=soa_value, ttl=None)
elif record_type == "CAA":
caa_value = f"{rdata.flags} {rdata.tag.decode() if isinstance(rdata.tag, bytes) else rdata.tag} {rdata.value.decode() if isinstance(rdata.value, bytes) else rdata.value}"
return DNSRecord(value=caa_value, ttl=None)
elif record_type == "SRV":
srv_value = f"{rdata.target} port={rdata.port} priority={rdata.priority} weight={rdata.weight}"
return DNSRecord(
value=srv_value,
priority=rdata.priority,
ttl=None
)
else:
return DNSRecord(value=str(rdata), ttl=None)
except Exception as e:
logger.error(f"Failed to parse {record_type} record: {e}")
return None
def _error_response(
self,
request: DNSLookupRequest,
nameserver_used: Optional[str],
start_time: float,
error_message: str
) -> DNSLookupResponse:
"""Create an error response for failed DNS queries"""
query_time_ms = (time.time() - start_time) * 1000
return DNSLookupResponse(
domain=request.domain,
record_type=request.record_type.upper(),
records=[],
nameserver_used=nameserver_used,
query_time_ms=round(query_time_ms, 2),
queried_at=datetime.now(timezone.utc),
success=False,
error_message=error_message
)
+23
View File
@@ -0,0 +1,23 @@
"""
Environment data module for Tools domain.
Provides access to weather, forecast, sun times, and air quality data
from the Qdrant volatile collection.
"""
from src.domains.tools.environment.schemas import (
WeatherData,
ForecastDay,
SunTimesData,
AirQualityData,
EnvironmentResponse,
)
from src.domains.tools.environment.service import EnvironmentService
__all__ = [
"WeatherData",
"ForecastDay",
"SunTimesData",
"AirQualityData",
"EnvironmentResponse",
"EnvironmentService",
]
+185
View File
@@ -0,0 +1,185 @@
"""
Environment data schemas for Tools domain.
Provides Pydantic models for weather, forecast, sun times, and air quality data
retrieved from the Qdrant volatile collection.
"""
from datetime import datetime
from typing import Optional, List, Any
from pydantic import Field
from src.shared.base import BaseSchema
class WeatherData(BaseSchema):
"""Current weather conditions."""
temperature: Optional[float] = Field(
None,
description="Current temperature in Celsius"
)
feels_like: Optional[float] = Field(
None,
description="Feels-like temperature in Celsius"
)
conditions: Optional[str] = Field(
None,
description="Weather conditions description (e.g., 'Partly Cloudy')"
)
humidity: Optional[int] = Field(
None,
ge=0,
le=100,
description="Humidity percentage"
)
wind_speed: Optional[float] = Field(
None,
description="Wind speed in km/h"
)
wind_direction: Optional[str] = Field(
None,
description="Wind direction (e.g., 'NW')"
)
pressure: Optional[float] = Field(
None,
description="Atmospheric pressure in hPa"
)
visibility: Optional[float] = Field(
None,
description="Visibility in km"
)
uv_index: Optional[float] = Field(
None,
description="UV index"
)
location: Optional[str] = Field(
None,
description="Location name"
)
icon: Optional[str] = Field(
None,
description="Weather icon code or URL"
)
class ForecastDay(BaseSchema):
"""Single day forecast data."""
date: str = Field(
...,
description="Date string (e.g., '2025-01-07')"
)
high: Optional[float] = Field(
None,
description="High temperature in Celsius"
)
low: Optional[float] = Field(
None,
description="Low temperature in Celsius"
)
conditions: Optional[str] = Field(
None,
description="Weather conditions description"
)
precipitation_chance: Optional[int] = Field(
None,
ge=0,
le=100,
description="Chance of precipitation percentage"
)
icon: Optional[str] = Field(
None,
description="Weather icon code or URL"
)
class SunTimesData(BaseSchema):
"""Sunrise and sunset times."""
sunrise: Optional[datetime] = Field(
None,
description="Sunrise time"
)
sunset: Optional[datetime] = Field(
None,
description="Sunset time"
)
daylight_minutes: Optional[int] = Field(
None,
description="Total daylight duration in minutes"
)
solar_noon: Optional[datetime] = Field(
None,
description="Solar noon time"
)
dawn: Optional[datetime] = Field(
None,
description="Civil dawn time"
)
dusk: Optional[datetime] = Field(
None,
description="Civil dusk time"
)
class AirQualityData(BaseSchema):
"""Air quality information."""
aqi: Optional[int] = Field(
None,
ge=0,
description="Air Quality Index"
)
quality: Optional[str] = Field(
None,
description="Quality category (Good, Moderate, Unhealthy, etc.)"
)
pm25: Optional[float] = Field(
None,
description="PM2.5 concentration in microg/m3"
)
pm10: Optional[float] = Field(
None,
description="PM10 concentration in microg/m3"
)
o3: Optional[float] = Field(
None,
description="Ozone concentration in ppb"
)
no2: Optional[float] = Field(
None,
description="Nitrogen dioxide concentration in ppb"
)
location: Optional[str] = Field(
None,
description="Location name"
)
class EnvironmentResponse(BaseSchema):
"""Combined environment data response."""
weather: Optional[WeatherData] = Field(
None,
description="Current weather conditions"
)
forecast: Optional[List[ForecastDay]] = Field(
None,
description="Multi-day weather forecast"
)
sun_times: Optional[SunTimesData] = Field(
None,
description="Sunrise/sunset times"
)
air_quality: Optional[AirQualityData] = Field(
None,
description="Air quality data (None if not available)"
)
updated_at: datetime = Field(
default_factory=datetime.utcnow,
description="Timestamp when data was fetched"
)
user: Optional[str] = Field(
None,
description="User identifier used for data lookup"
)
+287
View File
@@ -0,0 +1,287 @@
"""
Environment data service for Tools domain.
Fetches weather, forecast, sun times, and air quality data from
the Qdrant volatile collection.
"""
from datetime import datetime
from typing import Optional, Dict, Any, List
from src.shared.logging import get_logger
from src.shared.clients.qdrant_client import get_qdrant_client
from src.domains.tools.environment.schemas import (
WeatherData,
ForecastDay,
SunTimesData,
AirQualityData,
EnvironmentResponse,
)
logger = get_logger(__name__)
class EnvironmentService:
"""
Service for fetching environment data from Qdrant volatile collection.
Retrieves weather, forecast, sun times, and optionally air quality
data for a specific user.
"""
def __init__(self):
"""Initialize environment service with Qdrant client."""
self.qdrant = get_qdrant_client()
def _parse_weather(self, raw_data: Optional[Dict[str, Any]]) -> Optional[WeatherData]:
"""
Parse raw weather data into WeatherData schema.
Handles various field naming conventions that might come from
different weather APIs.
"""
if not raw_data:
return None
try:
# Handle wind direction - convert degrees to cardinal if integer
wind_dir = raw_data.get("wind_direction") or raw_data.get("wind_dir")
if isinstance(wind_dir, (int, float)):
wind_dir = self._degrees_to_cardinal(wind_dir)
return WeatherData(
temperature=raw_data.get("temperature") or raw_data.get("temp"),
feels_like=raw_data.get("feels_like") or raw_data.get("feelslike"),
conditions=raw_data.get("conditions") or raw_data.get("weather") or raw_data.get("description"),
humidity=raw_data.get("humidity"),
wind_speed=raw_data.get("wind_speed") or raw_data.get("windspeed") or raw_data.get("wind"),
wind_direction=wind_dir,
pressure=raw_data.get("pressure"),
visibility=raw_data.get("visibility"),
uv_index=raw_data.get("uv_index") or raw_data.get("uv"),
location=raw_data.get("location") or raw_data.get("city"),
icon=raw_data.get("icon") or raw_data.get("icon_url"),
)
except Exception as e:
logger.warning(f"Failed to parse weather data: {e}")
return None
def _degrees_to_cardinal(self, degrees: float) -> str:
"""Convert wind direction degrees to cardinal direction."""
directions = ["N", "NNE", "NE", "ENE", "E", "ESE", "SE", "SSE",
"S", "SSW", "SW", "WSW", "W", "WNW", "NW", "NNW"]
index = round(degrees / 22.5) % 16
return directions[index]
def _parse_forecast(self, raw_data: Any) -> Optional[List[ForecastDay]]:
"""
Parse raw forecast data into list of ForecastDay schemas.
Handles both list format and dict with nested list.
"""
if not raw_data:
return None
try:
# Normalize to list
forecast_list = raw_data
if isinstance(raw_data, dict):
# Check 'daily' first (scheduler format), then 'forecast', then 'days'
# Note: 'days' might be an integer count, so check 'daily' first
forecast_list = raw_data.get("daily") or raw_data.get("forecast")
if forecast_list is None:
days_value = raw_data.get("days")
if isinstance(days_value, list):
forecast_list = days_value
else:
forecast_list = []
if not isinstance(forecast_list, list):
return None
days = []
for day in forecast_list:
if isinstance(day, dict):
days.append(ForecastDay(
date=day.get("date", ""),
high=day.get("high") or day.get("temp_high") or day.get("maxtemp") or day.get("temp_max"),
low=day.get("low") or day.get("temp_low") or day.get("mintemp") or day.get("temp_min"),
conditions=day.get("conditions") or day.get("weather") or day.get("description"),
precipitation_chance=day.get("precipitation_chance") or day.get("pop") or day.get("precip"),
icon=day.get("icon"),
))
return days if days else None
except Exception as e:
logger.warning(f"Failed to parse forecast data: {e}")
return None
def _parse_sun_times(self, raw_data: Optional[Dict[str, Any]]) -> Optional[SunTimesData]:
"""
Parse raw sun times data into SunTimesData schema.
Handles datetime strings and calculates daylight minutes if not provided.
"""
if not raw_data:
return None
try:
# Prefer ISO format fields (sunrise_iso, sunset_iso) over time-only fields
sunrise = raw_data.get("sunrise_iso") or raw_data.get("sunrise")
sunset = raw_data.get("sunset_iso") or raw_data.get("sunset")
# Parse datetime strings if needed
if isinstance(sunrise, str):
# Handle time-only format (HH:MM) by combining with today's date
if len(sunrise) <= 5 and ":" in sunrise:
today = datetime.now().date()
sunrise = datetime.strptime(f"{today} {sunrise}", "%Y-%m-%d %H:%M")
else:
sunrise = datetime.fromisoformat(sunrise.replace("Z", "+00:00"))
if isinstance(sunset, str):
# Handle time-only format (HH:MM) by combining with today's date
if len(sunset) <= 5 and ":" in sunset:
today = datetime.now().date()
sunset = datetime.strptime(f"{today} {sunset}", "%Y-%m-%d %H:%M")
else:
sunset = datetime.fromisoformat(sunset.replace("Z", "+00:00"))
# Get daylight from various field names
daylight_minutes = raw_data.get("daylight_minutes") or raw_data.get("daylight")
if daylight_minutes is None:
# Try to calculate from daylight_duration_seconds or daylight_hours
daylight_seconds = raw_data.get("daylight_duration_seconds")
if daylight_seconds:
daylight_minutes = int(daylight_seconds / 60)
else:
daylight_hours = raw_data.get("daylight_hours")
if daylight_hours:
daylight_minutes = int(daylight_hours * 60)
elif sunrise and sunset:
daylight_minutes = int((sunset - sunrise).total_seconds() / 60)
# Parse optional fields
solar_noon = raw_data.get("solar_noon")
if isinstance(solar_noon, str):
solar_noon = datetime.fromisoformat(solar_noon.replace("Z", "+00:00"))
dawn = raw_data.get("dawn") or raw_data.get("civil_dawn")
if isinstance(dawn, str):
dawn = datetime.fromisoformat(dawn.replace("Z", "+00:00"))
dusk = raw_data.get("dusk") or raw_data.get("civil_dusk")
if isinstance(dusk, str):
dusk = datetime.fromisoformat(dusk.replace("Z", "+00:00"))
return SunTimesData(
sunrise=sunrise,
sunset=sunset,
daylight_minutes=daylight_minutes,
solar_noon=solar_noon,
dawn=dawn,
dusk=dusk,
)
except Exception as e:
logger.warning(f"Failed to parse sun times data: {e}")
return None
def _parse_air_quality(self, raw_data: Any) -> Optional[AirQualityData]:
"""
Parse raw air quality data into AirQualityData schema.
Handles both dict format and simple integer AQI value.
"""
if raw_data is None:
return None
try:
# Handle simple integer AQI
if isinstance(raw_data, (int, float)):
aqi = int(raw_data)
return AirQualityData(
aqi=aqi,
quality=self._aqi_to_quality(aqi),
)
if not isinstance(raw_data, dict):
return None
# Try various AQI field names - prefer US AQI, then European, then generic
aqi = raw_data.get("aqi") or raw_data.get("aqi_us") or raw_data.get("aqi_european") or raw_data.get("index")
if isinstance(aqi, (int, float)):
aqi = int(aqi)
return AirQualityData(
aqi=aqi,
quality=raw_data.get("quality") or (self._aqi_to_quality(aqi) if aqi else None),
pm25=raw_data.get("pm25") or raw_data.get("pm2_5"),
pm10=raw_data.get("pm10"),
o3=raw_data.get("o3") or raw_data.get("ozone"),
no2=raw_data.get("no2") or raw_data.get("nitrogen_dioxide"),
location=raw_data.get("location"),
)
except Exception as e:
logger.warning(f"Failed to parse air quality data: {e}")
return None
def _aqi_to_quality(self, aqi: int) -> str:
"""Convert AQI value to quality category string."""
if aqi <= 50:
return "Good"
elif aqi <= 100:
return "Moderate"
elif aqi <= 150:
return "Unhealthy for Sensitive Groups"
elif aqi <= 200:
return "Unhealthy"
elif aqi <= 300:
return "Very Unhealthy"
else:
return "Hazardous"
async def get_current(self, user: str = "default") -> EnvironmentResponse:
"""
Get current environment data for a user.
Fetches weather, forecast, sun times, and air quality from
the user's volatile collection.
Args:
user: User identifier (default: 'default')
Returns:
EnvironmentResponse with all available data
"""
logger.info(f"Fetching environment data for user: {user}")
# Get raw data from Qdrant
raw_data = await self.qdrant.get_environment_data(user)
# Parse each data type
weather = self._parse_weather(raw_data.get("weather"))
forecast = self._parse_forecast(raw_data.get("forecast"))
sun_times = self._parse_sun_times(raw_data.get("sun_times"))
air_quality = self._parse_air_quality(raw_data.get("air_quality"))
return EnvironmentResponse(
weather=weather,
forecast=forecast,
sun_times=sun_times,
air_quality=air_quality,
updated_at=datetime.utcnow(),
user=user,
)
# Singleton instance
_environment_service: Optional[EnvironmentService] = None
def get_environment_service() -> EnvironmentService:
"""Get or create singleton environment service instance."""
global _environment_service
if _environment_service is None:
_environment_service = EnvironmentService()
return _environment_service
+5
View File
@@ -0,0 +1,5 @@
"""
News subdomain for Tools.
Provides news headlines from Qdrant volatile collection.
"""
+56
View File
@@ -0,0 +1,56 @@
"""
News data schemas for Tools domain.
Provides Pydantic models for news headlines retrieved from the Qdrant volatile collection.
"""
from datetime import datetime
from typing import Optional, List
from pydantic import Field
from src.shared.base import BaseSchema
class NewsHeadline(BaseSchema):
"""Single news headline."""
title: str = Field(
...,
description="Headline title"
)
description: Optional[str] = Field(
None,
description="Brief description or summary"
)
source: Optional[str] = Field(
None,
description="News source name"
)
url: Optional[str] = Field(
None,
description="Link to full article"
)
class NewsResponse(BaseSchema):
"""News headlines response."""
headlines: List[NewsHeadline] = Field(
default_factory=list,
description="List of news headlines"
)
category: Optional[str] = Field(
None,
description="News category (e.g., 'general', 'technology')"
)
sources: Optional[List[str]] = Field(
None,
description="List of source names"
)
updated_at: datetime = Field(
default_factory=datetime.utcnow,
description="Timestamp when data was fetched"
)
user: Optional[str] = Field(
None,
description="User identifier used for data lookup"
)
+113
View File
@@ -0,0 +1,113 @@
"""
News data service for Tools domain.
Fetches news headlines from the Qdrant volatile collection.
"""
from datetime import datetime
from typing import Optional, Dict, Any, List
from src.shared.logging import get_logger
from src.shared.clients.qdrant_client import get_qdrant_client
from src.domains.tools.news.schemas import (
NewsHeadline,
NewsResponse,
)
logger = get_logger(__name__)
class NewsService:
"""
Service for fetching news data from Qdrant volatile collection.
Retrieves news headlines for a specific user.
"""
def __init__(self):
"""Initialize news service with Qdrant client."""
self.qdrant = get_qdrant_client()
def _parse_headlines(self, raw_data: Any) -> List[NewsHeadline]:
"""
Parse raw news data into list of NewsHeadline schemas.
Handles various formats from different news sources.
"""
if not raw_data:
return []
try:
# Handle dict with nested headlines list
headlines_list = raw_data
if isinstance(raw_data, dict):
headlines_list = raw_data.get("headlines") or raw_data.get("articles") or []
if not isinstance(headlines_list, list):
return []
headlines = []
for item in headlines_list:
if isinstance(item, dict):
headlines.append(NewsHeadline(
title=item.get("title", ""),
description=item.get("description") or item.get("summary"),
source=item.get("source") or item.get("provider"),
url=item.get("url") or item.get("link"),
))
elif isinstance(item, str):
# Simple string headlines
headlines.append(NewsHeadline(title=item))
return headlines
except Exception as e:
logger.warning(f"Failed to parse news headlines: {e}")
return []
async def get_news(self, user: str = "default") -> NewsResponse:
"""
Get news headlines for a user.
Fetches news from the user's volatile collection.
Args:
user: User identifier (default: 'default')
Returns:
NewsResponse with headlines
"""
logger.info(f"Fetching news data for user: {user}")
# Get raw data from Qdrant
news_records = await self.qdrant.get_by_namespace(user, "news")
headlines = []
category = None
sources = None
if news_records:
raw_data = news_records[0].get("raw_data", {})
headlines = self._parse_headlines(raw_data)
if isinstance(raw_data, dict):
category = raw_data.get("category")
sources = raw_data.get("sources")
return NewsResponse(
headlines=headlines,
category=category,
sources=sources,
updated_at=datetime.utcnow(),
user=user,
)
# Singleton instance
_news_service: Optional[NewsService] = None
def get_news_service() -> NewsService:
"""Get or create singleton news service instance."""
global _news_service
if _news_service is None:
_news_service = NewsService()
return _news_service
+6
View File
@@ -0,0 +1,6 @@
"""System stats module for host system resource monitoring."""
from src.domains.tools.system.service import SystemStatsService
from src.domains.tools.system.schemas import SystemStatsResponse
__all__ = ["SystemStatsService", "SystemStatsResponse"]
+197
View File
@@ -0,0 +1,197 @@
"""
Pydantic schemas for system stats module
"""
from pydantic import Field
from typing import Optional, List
from datetime import datetime
from src.shared.base import BaseSchema
class CpuStats(BaseSchema):
"""CPU usage statistics"""
usage_percent: float = Field(
...,
description="CPU usage percentage (0-100)",
ge=0,
le=100
)
cores: int = Field(
...,
description="Number of CPU cores"
)
load_1m: Optional[float] = Field(
default=None,
description="1-minute load average"
)
load_5m: Optional[float] = Field(
default=None,
description="5-minute load average"
)
load_15m: Optional[float] = Field(
default=None,
description="15-minute load average"
)
class MemoryStats(BaseSchema):
"""Memory usage statistics"""
usage_percent: float = Field(
...,
description="Memory usage percentage (0-100)",
ge=0,
le=100
)
total_bytes: int = Field(
...,
description="Total memory in bytes"
)
used_bytes: int = Field(
...,
description="Used memory in bytes"
)
available_bytes: int = Field(
...,
description="Available memory in bytes"
)
class DiskStats(BaseSchema):
"""Disk usage statistics for a single mount point"""
mount_point: str = Field(
...,
description="Mount point path"
)
device: str = Field(
...,
description="Device name (e.g., /dev/sda1)"
)
fstype: str = Field(
...,
description="Filesystem type (e.g., ext4, xfs)"
)
usage_percent: float = Field(
...,
description="Disk usage percentage (0-100)",
ge=0,
le=100
)
total_bytes: int = Field(
...,
description="Total disk space in bytes"
)
used_bytes: int = Field(
...,
description="Used disk space in bytes"
)
free_bytes: int = Field(
...,
description="Free disk space in bytes"
)
class NetworkStats(BaseSchema):
"""Network I/O statistics"""
bytes_sent: int = Field(
...,
description="Total bytes sent"
)
bytes_recv: int = Field(
...,
description="Total bytes received"
)
bytes_total: int = Field(
...,
description="Total bytes (sent + received)"
)
class GpuStats(BaseSchema):
"""GPU/VRAM statistics (if available)"""
available: bool = Field(
...,
description="Whether GPU stats are available"
)
name: Optional[str] = Field(
default=None,
description="GPU name"
)
usage_percent: Optional[float] = Field(
default=None,
description="VRAM usage percentage (0-100)"
)
total_bytes: Optional[int] = Field(
default=None,
description="Total VRAM in bytes"
)
used_bytes: Optional[int] = Field(
default=None,
description="Used VRAM in bytes"
)
free_bytes: Optional[int] = Field(
default=None,
description="Free VRAM in bytes"
)
class SystemStatsResponse(BaseSchema):
"""Response model for system stats"""
cpu: CpuStats = Field(
...,
description="CPU statistics"
)
memory: MemoryStats = Field(
...,
description="Memory statistics"
)
disks: List[DiskStats] = Field(
...,
description="Disk statistics for all mounted filesystems"
)
network: NetworkStats = Field(
...,
description="Network I/O statistics"
)
gpu: GpuStats = Field(
...,
description="GPU/VRAM statistics"
)
hostname: str = Field(
...,
description="System hostname"
)
queried_at: datetime = Field(
...,
description="UTC timestamp when stats were collected"
)
+190
View File
@@ -0,0 +1,190 @@
"""
System stats service for collecting host system metrics
"""
import subprocess
import socket
from datetime import datetime, timezone
import psutil
from src.shared.logging import get_logger
from typing import List
from src.domains.tools.system.schemas import (
SystemStatsResponse,
CpuStats,
MemoryStats,
DiskStats,
NetworkStats,
GpuStats,
)
# Filesystem types to exclude (virtual/system filesystems)
EXCLUDED_FSTYPES = {
"tmpfs", "devtmpfs", "devfs", "squashfs", "overlay",
"aufs", "proc", "sysfs", "cgroup", "cgroup2",
"debugfs", "tracefs", "securityfs", "pstore",
"hugetlbfs", "mqueue", "binfmt_misc", "autofs",
"fuse.lxcfs", "nsfs", "efivarfs",
}
logger = get_logger(__name__)
class SystemStatsService:
"""Service for collecting host system statistics"""
async def get_stats(self) -> SystemStatsResponse:
"""
Collect current system statistics.
Returns:
SystemStatsResponse with CPU, memory, disks, network, and GPU stats
"""
cpu = self._get_cpu_stats()
memory = self._get_memory_stats()
disks = self._get_all_disk_stats()
network = self._get_network_stats()
gpu = self._get_gpu_stats()
return SystemStatsResponse(
cpu=cpu,
memory=memory,
disks=disks,
network=network,
gpu=gpu,
hostname=socket.gethostname(),
queried_at=datetime.now(timezone.utc),
)
def _get_cpu_stats(self) -> CpuStats:
"""Get CPU usage statistics"""
# Get CPU percentage (blocking call with interval for accuracy)
cpu_percent = psutil.cpu_percent(interval=0.1)
cpu_count = psutil.cpu_count()
# Get load averages (Unix only)
try:
load_1, load_5, load_15 = psutil.getloadavg()
except (AttributeError, OSError):
load_1 = load_5 = load_15 = None
return CpuStats(
usage_percent=cpu_percent,
cores=cpu_count or 1,
load_1m=load_1,
load_5m=load_5,
load_15m=load_15,
)
def _get_memory_stats(self) -> MemoryStats:
"""Get memory usage statistics"""
mem = psutil.virtual_memory()
return MemoryStats(
usage_percent=mem.percent,
total_bytes=mem.total,
used_bytes=mem.used,
available_bytes=mem.available,
)
def _get_all_disk_stats(self) -> List[DiskStats]:
"""Get disk usage statistics for all mounted real filesystems"""
disks = []
seen_devices = set()
for partition in psutil.disk_partitions(all=False):
# Skip excluded filesystem types
if partition.fstype.lower() in EXCLUDED_FSTYPES:
continue
# Skip duplicate devices (same device mounted multiple times)
if partition.device in seen_devices:
continue
seen_devices.add(partition.device)
# Skip Docker/container overlays
if partition.mountpoint.startswith("/var/lib/docker"):
continue
try:
usage = psutil.disk_usage(partition.mountpoint)
disks.append(DiskStats(
mount_point=partition.mountpoint,
device=partition.device,
fstype=partition.fstype,
usage_percent=usage.percent,
total_bytes=usage.total,
used_bytes=usage.used,
free_bytes=usage.free,
))
except (PermissionError, OSError) as e:
logger.debug(f"Skipping {partition.mountpoint}: {e}")
continue
# Sort by mount point for consistent ordering
disks.sort(key=lambda d: d.mount_point)
return disks
def _get_network_stats(self) -> NetworkStats:
"""Get network I/O statistics"""
net_io = psutil.net_io_counters()
return NetworkStats(
bytes_sent=net_io.bytes_sent,
bytes_recv=net_io.bytes_recv,
bytes_total=net_io.bytes_sent + net_io.bytes_recv,
)
def _get_gpu_stats(self) -> GpuStats:
"""Get GPU/VRAM statistics using nvidia-smi"""
try:
# Query nvidia-smi for GPU memory info
result = subprocess.run(
[
"nvidia-smi",
"--query-gpu=name,memory.total,memory.used,memory.free",
"--format=csv,noheader,nounits",
],
capture_output=True,
text=True,
timeout=5,
)
if result.returncode != 0:
logger.debug("nvidia-smi not available or failed")
return GpuStats(available=False)
# Parse output: "NVIDIA GeForce RTX 3080, 10240, 2048, 8192"
line = result.stdout.strip().split("\n")[0] # First GPU
parts = [p.strip() for p in line.split(",")]
if len(parts) >= 4:
name = parts[0]
total_mb = int(parts[1])
used_mb = int(parts[2])
free_mb = int(parts[3])
total_bytes = total_mb * 1024 * 1024
used_bytes = used_mb * 1024 * 1024
free_bytes = free_mb * 1024 * 1024
usage_percent = (used_mb / total_mb * 100) if total_mb > 0 else 0
return GpuStats(
available=True,
name=name,
usage_percent=round(usage_percent, 1),
total_bytes=total_bytes,
used_bytes=used_bytes,
free_bytes=free_bytes,
)
except FileNotFoundError:
logger.debug("nvidia-smi not found - no NVIDIA GPU available")
except subprocess.TimeoutExpired:
logger.warning("nvidia-smi timed out")
except Exception as e:
logger.warning(f"Failed to get GPU stats: {e}")
return GpuStats(available=False)
+19 -25
View File
@@ -6,17 +6,19 @@ from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import JSONResponse from fastapi.responses import JSONResponse
from contextlib import asynccontextmanager from contextlib import asynccontextmanager
from src.config import get_settings from src.shared.config import get_settings
from src.logging_config import setup_logging, get_logger from src.shared.logging import setup_logging, get_logger
from src.models.ollama_client import get_ollama_client, close_ollama_client from src.shared.database import get_database
from src.db import get_database from src.shared.security import initialize_oidc
from src.controllers.infrastructure_controller import infrastructure_controller
from src.controllers.tools_controller import tools_controller # Import domain controllers
from src.controllers.health_controller import health_controller from src.domains.health import health_controller
from src.controllers.static_controller import static_controller from src.domains.auth import auth_controller
from src.controllers.housekeeping_controller import housekeeping_controller from src.domains.tools import tools_controller
from src.auth.controller import auth_controller from src.domains.infrastructure import infrastructure_controller
from src.security import initialize_oidc from src.domains.housekeeping import housekeeping_controller
from src.domains.static import static_controller
from src.domains.dashboard import dashboard_controller
# Initialize settings # Initialize settings
settings = get_settings() settings = get_settings()
@@ -39,24 +41,15 @@ async def lifespan(app: FastAPI):
logger.info(f"Starting {settings.app_name} v{settings.app_version}") logger.info(f"Starting {settings.app_name} v{settings.app_version}")
logger.info(f"Debug mode: {settings.debug}") logger.info(f"Debug mode: {settings.debug}")
logger.info(f"Log level: {settings.log_level}") logger.info(f"Log level: {settings.log_level}")
logger.info(f"Ollama URL: {settings.ollama_base_url}")
logger.info("=" * 60) logger.info("=" * 60)
# Check Ollama connectivity
ollama_client = get_ollama_client()
ollama_healthy = await ollama_client.health_check()
if ollama_healthy:
logger.info("✓ Ollama connection successful")
else:
logger.warning("✗ Ollama connection failed - AI features may not work")
# Check database connectivity # Check database connectivity
database = get_database() database = get_database()
db_healthy = await database.health_check() db_healthy = await database.health_check()
if db_healthy: if db_healthy:
logger.info("Database connection successful") logger.info("Database connection successful")
else: else:
logger.warning("Database connection failed - auth features may not work") logger.warning("Database connection failed - auth features may not work")
# Initialize security (OIDC authentication) # Initialize security (OIDC authentication)
initialize_oidc(settings) initialize_oidc(settings)
@@ -65,7 +58,6 @@ async def lifespan(app: FastAPI):
# Shutdown # Shutdown
logger.info("Shutting down application") logger.info("Shutting down application")
await close_ollama_client()
await database.close() await database.close()
@@ -80,6 +72,7 @@ Core Code API - Infrastructure management and home automation API.
- **Infrastructure Management** - Container and stack management via Portainer - **Infrastructure Management** - Container and stack management via Portainer
- **Home Automation** - Device control via Home Assistant - **Home Automation** - Device control via Home Assistant
- **Dashboard** - Quick links and widget management
- **Tools** - DNS lookup and utilities - **Tools** - DNS lookup and utilities
See `/docs` for the full API reference. See `/docs` for the full API reference.
@@ -90,7 +83,7 @@ See `/docs` for the full API reference.
lifespan=lifespan, lifespan=lifespan,
debug=settings.debug, debug=settings.debug,
swagger_ui_init_oauth={ swagger_ui_init_oauth={
"clientId": settings.oidc_audience, "clientId": settings.oidc_audiences[0] if settings.oidc_audiences else "core-api",
"usePkceWithAuthorizationCodeGrant": True, "usePkceWithAuthorizationCodeGrant": True,
} if settings.oidc_enabled else None } if settings.oidc_enabled else None
) )
@@ -105,13 +98,14 @@ app.add_middleware(
) )
# Include controller routers # Include domain routers
app.include_router(health_controller.router) # / and /health app.include_router(health_controller.router) # / and /health
app.include_router(auth_controller.router) # /auth/* app.include_router(auth_controller.router) # /auth/*
app.include_router(tools_controller.router) # /tools/* app.include_router(tools_controller.router) # /tools/*
app.include_router(infrastructure_controller.router) # /infrastructure/* app.include_router(infrastructure_controller.router) # /infrastructure/*
app.include_router(housekeeping_controller.router) # /housekeeping/* app.include_router(housekeeping_controller.router) # /housekeeping/*
app.include_router(static_controller.router) # /static/* app.include_router(static_controller.router) # /static/*
app.include_router(dashboard_controller.router) # /dashboard/*
# Global exception handler # Global exception handler
-128
View File
@@ -1,128 +0,0 @@
"""
Embedding model client for text vectorization
Uses sentence-transformers for generating embeddings.
"""
import logging
from typing import List, Optional
from sentence_transformers import SentenceTransformer
from src.config import get_settings
logger = logging.getLogger(__name__)
settings = get_settings()
class EmbeddingClient:
"""Client for generating text embeddings"""
def __init__(self, model_name: Optional[str] = None):
"""
Initialize embedding client
Args:
model_name: Optional model name, defaults to config
"""
self.model_name = model_name or settings.embedding_model
self.dimension = settings.embedding_dimension
self._model: Optional[SentenceTransformer] = None
logger.info(f"Initializing EmbeddingClient with model: {self.model_name}")
def _load_model(self) -> SentenceTransformer:
"""
Lazy load the embedding model
Returns:
Loaded SentenceTransformer model
"""
if self._model is None:
logger.info(f"Loading embedding model: {self.model_name}")
self._model = SentenceTransformer(self.model_name)
logger.info(f"Model loaded successfully. Embedding dimension: {self.dimension}")
return self._model
def embed_text(self, text: str) -> List[float]:
"""
Generate embedding for a single text
Args:
text: Input text to embed
Returns:
List of floats representing the embedding vector
"""
model = self._load_model()
embedding = model.encode(text, convert_to_numpy=True)
return embedding.tolist()
def embed_batch(self, texts: List[str]) -> List[List[float]]:
"""
Generate embeddings for multiple texts
Args:
texts: List of input texts
Returns:
List of embedding vectors
"""
model = self._load_model()
embeddings = model.encode(
texts,
batch_size=settings.embedding_batch_size,
convert_to_numpy=True,
show_progress_bar=False
)
return embeddings.tolist()
def get_dimension(self) -> int:
"""
Get embedding dimension
Returns:
Embedding vector dimension
"""
return self.dimension
# Global instance
_embedding_client: Optional[EmbeddingClient] = None
def get_embedding_client() -> EmbeddingClient:
"""
Get or create global embedding client instance
Returns:
EmbeddingClient instance
"""
global _embedding_client
if _embedding_client is None:
_embedding_client = EmbeddingClient()
return _embedding_client
async def embed_text_async(text: str) -> List[float]:
"""
Async wrapper for embedding text
Args:
text: Input text
Returns:
Embedding vector
"""
client = get_embedding_client()
return client.embed_text(text)
async def embed_batch_async(texts: List[str]) -> List[List[float]]:
"""
Async wrapper for batch embedding
Args:
texts: List of input texts
Returns:
List of embedding vectors
"""
client = get_embedding_client()
return client.embed_batch(texts)
-136
View File
@@ -1,136 +0,0 @@
"""
Ollama-based embedding client for text vectorization
Uses Ollama's embedding API instead of local sentence-transformers.
This eliminates the need for PyTorch and heavy ML dependencies.
"""
import logging
import httpx
from typing import List, Optional
from src.config import get_settings
logger = logging.getLogger(__name__)
settings = get_settings()
class OllamaEmbeddingClient:
"""Client for generating text embeddings using Ollama"""
def __init__(
self,
model_name: Optional[str] = None,
base_url: Optional[str] = None,
timeout: int = 30
):
"""
Initialize Ollama embedding client
Args:
model_name: Embedding model name (default: nomic-embed-text)
base_url: Ollama base URL (default from settings)
timeout: Request timeout in seconds
"""
self.model_name = model_name or settings.embedding_model
self.base_url = (base_url or settings.ollama_base_url).rstrip("/")
self.timeout = timeout
self.dimension = settings.embedding_dimension
logger.info(f"Initializing OllamaEmbeddingClient with model: {self.model_name}")
logger.info(f"Ollama URL: {self.base_url}")
async def embed_text(self, text: str) -> List[float]:
"""
Generate embedding for a single text using Ollama
Args:
text: Input text to embed
Returns:
List of floats representing the embedding vector
"""
try:
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.post(
f"{self.base_url}/api/embeddings",
json={
"model": self.model_name,
"prompt": text
}
)
response.raise_for_status()
result = response.json()
return result["embedding"]
except Exception as e:
logger.error(f"Error generating embedding via Ollama: {e}")
raise
async def embed_batch(self, texts: List[str]) -> List[List[float]]:
"""
Generate embeddings for multiple texts
Args:
texts: List of input texts
Returns:
List of embedding vectors
"""
embeddings = []
for text in texts:
embedding = await self.embed_text(text)
embeddings.append(embedding)
return embeddings
def get_dimension(self) -> int:
"""
Get embedding dimension
Returns:
Embedding vector dimension
"""
return self.dimension
# Global instance
_embedding_client: Optional[OllamaEmbeddingClient] = None
def get_embedding_client() -> OllamaEmbeddingClient:
"""
Get or create global Ollama embedding client instance
Returns:
OllamaEmbeddingClient instance
"""
global _embedding_client
if _embedding_client is None:
_embedding_client = OllamaEmbeddingClient()
return _embedding_client
async def embed_text_async(text: str) -> List[float]:
"""
Async wrapper for embedding text
Args:
text: Input text
Returns:
Embedding vector
"""
client = get_embedding_client()
return await client.embed_text(text)
async def embed_batch_async(texts: List[str]) -> List[List[float]]:
"""
Async wrapper for batch embedding
Args:
texts: List of input texts
Returns:
List of embedding vectors
"""
client = get_embedding_client()
return await client.embed_batch(texts)
-223
View File
@@ -1,223 +0,0 @@
"""
Ollama client for model inference.
Handles both streaming and non-streaming requests.
"""
import httpx
import json
import logging
from typing import AsyncIterator, Dict, Any, Optional
from src.config import get_settings
logger = logging.getLogger(__name__)
settings = get_settings()
class OllamaClient:
"""Client for interacting with Ollama API."""
def __init__(self):
self.base_url = settings.ollama_base_url
self.timeout = settings.ollama_timeout
self.client = httpx.AsyncClient(timeout=self.timeout)
logger.info(f"Initialized Ollama client: {self.base_url}")
async def close(self):
"""Close the HTTP client."""
await self.client.aclose()
def resolve_model(self, model_name: str) -> str:
"""
Resolve model alias to actual Ollama model.
Args:
model_name: Requested model name (e.g., "gpt-3.5-turbo")
Returns:
Actual Ollama model name (e.g., "gemma:7b")
"""
resolved = settings.model_aliases.get(model_name, model_name)
if resolved != model_name:
logger.info(f"Model resolution: {model_name}{resolved}")
return resolved
async def generate_non_streaming(
self,
model: str,
prompt: str,
temperature: float = 0.7,
max_tokens: Optional[int] = None
) -> Dict[str, Any]:
"""
Generate non-streaming response from Ollama using chat endpoint.
Args:
model: Model name
prompt: User prompt
temperature: Sampling temperature
max_tokens: Maximum tokens to generate
Returns:
Dict with 'response' and 'tokens' keys
"""
actual_model = self.resolve_model(model)
payload = {
"model": actual_model,
"messages": [
{"role": "user", "content": prompt}
],
"stream": False,
"options": {
"temperature": temperature,
}
}
if max_tokens:
payload["options"]["num_predict"] = max_tokens
logger.debug(f"Ollama request to {actual_model}")
try:
response = await self.client.post(
f"{self.base_url}/api/chat",
json=payload
)
response.raise_for_status()
result = response.json()
return {
"response": result.get("message", {}).get("content", ""),
"tokens": {
"prompt": result.get("prompt_eval_count", 0),
"completion": result.get("eval_count", 0),
"total": result.get("prompt_eval_count", 0) + result.get("eval_count", 0)
}
}
except httpx.HTTPError as e:
logger.error(f"Ollama request failed: {e}")
raise
async def generate_streaming(
self,
model: str,
prompt: str,
temperature: float = 0.7,
max_tokens: Optional[int] = None
) -> AsyncIterator[str]:
"""
Generate streaming response from Ollama using chat endpoint.
Args:
model: Model name
prompt: User prompt
temperature: Sampling temperature
max_tokens: Maximum tokens to generate
Yields:
Token strings
"""
actual_model = self.resolve_model(model)
payload = {
"model": actual_model,
"messages": [
{"role": "user", "content": prompt}
],
"stream": True,
"options": {
"temperature": temperature,
}
}
if max_tokens:
payload["options"]["num_predict"] = max_tokens
logger.debug(f"Ollama streaming request to {actual_model}")
try:
async with self.client.stream(
"POST",
f"{self.base_url}/api/chat",
json=payload
) as response:
response.raise_for_status()
async for line in response.aiter_lines():
if not line:
continue
try:
chunk = json.loads(line)
if "message" in chunk:
content = chunk["message"].get("content", "")
if content:
yield content
# Check if done
if chunk.get("done", False):
break
except json.JSONDecodeError:
logger.warning(f"Failed to parse JSON: {line}")
continue
except httpx.HTTPError as e:
logger.error(f"Ollama streaming request failed: {e}")
raise
async def health_check(self) -> bool:
"""
Check if Ollama is healthy.
Returns:
True if healthy, False otherwise
"""
try:
response = await self.client.get(
f"{self.base_url}/api/tags",
timeout=5.0
)
return response.status_code == 200
except Exception as e:
logger.error(f"Ollama health check failed: {e}")
return False
async def list_models(self) -> Dict[str, Any]:
"""
List all available models in Ollama.
Returns:
Dict with 'models' key containing list of model info
"""
try:
response = await self.client.get(
f"{self.base_url}/api/tags",
timeout=5.0
)
response.raise_for_status()
return response.json()
except Exception as e:
logger.error(f"Failed to list Ollama models: {e}")
raise
# Global client instance
_ollama_client: Optional[OllamaClient] = None
def get_ollama_client() -> OllamaClient:
"""Get or create the global Ollama client instance."""
global _ollama_client
if _ollama_client is None:
_ollama_client = OllamaClient()
return _ollama_client
async def close_ollama_client():
"""Close the global Ollama client."""
global _ollama_client
if _ollama_client is not None:
await _ollama_client.close()
_ollama_client = None
-32
View File
@@ -1,32 +0,0 @@
"""
Security initialization module
Handles OIDC configuration and authentication setup
"""
from src.config import Settings
from src.auth.oidc import oidc_config
from src.logging_config import get_logger
logger = get_logger(__name__)
def initialize_oidc(settings: Settings) -> None:
"""
Initialize OIDC authentication configuration
Configures the global oidc_config instance with settings from environment.
If OIDC is enabled, logs the issuer URL for verification.
Args:
settings: Application settings containing OIDC configuration
"""
oidc_config.configure(
enabled=settings.oidc_enabled,
issuer=settings.oidc_issuer,
audience=settings.oidc_audience
)
if settings.oidc_enabled:
logger.info(f"✓ OIDC authentication enabled (issuer: {settings.oidc_issuer})")
else:
logger.info("○ OIDC authentication disabled - API is publicly accessible")
+50
View File
@@ -0,0 +1,50 @@
"""
Shared utilities for Core-API
Contains common base classes, configuration, database, logging utilities,
and API clients used across all domains.
"""
from src.shared.config import get_settings, Settings
from src.shared.database import Base, get_database, get_async_session
from src.shared.logging import get_logger, setup_logging
from src.shared.base import BaseController, BaseSchema
from src.shared.security import initialize_oidc
# Re-export clients for convenience
from src.shared.clients import (
PortainerClient,
get_portainer_client,
NPMClient,
get_npm_client,
HomeAssistantClient,
get_homeassistant_client,
AuthentikClient,
get_authentik_client,
)
__all__ = [
# Config
"get_settings",
"Settings",
# Database
"Base",
"get_database",
"get_async_session",
# Logging
"get_logger",
"setup_logging",
# Base classes
"BaseController",
"BaseSchema",
# Security
"initialize_oidc",
# Clients
"PortainerClient",
"get_portainer_client",
"NPMClient",
"get_npm_client",
"HomeAssistantClient",
"get_homeassistant_client",
"AuthentikClient",
"get_authentik_client",
]
+65
View File
@@ -0,0 +1,65 @@
"""
Base classes for Core-API
Provides common base classes for controllers and schemas.
"""
from datetime import datetime
from typing import Any
from abc import ABC, abstractmethod
from fastapi import APIRouter
from pydantic import BaseModel, ConfigDict
class BaseController(ABC):
"""
Base controller class with common functionality
All controllers should inherit from this class and implement
the create_router() method to define their endpoints.
"""
def __init__(self, prefix: str, tags: list[str]):
"""
Initialize base controller
Args:
prefix: URL prefix for this controller's routes
tags: OpenAPI tags for documentation grouping
"""
self.prefix = prefix
self.tags = tags
self._router = None
@abstractmethod
def create_router(self) -> APIRouter:
"""Create and configure the FastAPI router for this controller"""
pass
@property
def router(self) -> APIRouter:
"""Get the router instance, creating it if needed"""
if self._router is None:
self._router = self.create_router()
return self._router
class BaseSchema(BaseModel):
"""
Base Pydantic model with standardized configuration
All schemas should inherit from this to ensure consistent behavior.
"""
model_config = ConfigDict(
strict=False,
populate_by_name=True,
use_enum_values=True,
validate_assignment=True,
json_encoders={
datetime: lambda v: v.isoformat() if v else None
}
)
def dict_without_none(self) -> dict[str, Any]:
"""Return model as dict, excluding None values"""
return {k: v for k, v in self.model_dump().items() if v is not None}
+23
View File
@@ -0,0 +1,23 @@
"""
API Clients for Core-API
Provides HTTP/WebSocket clients for external infrastructure services.
"""
from src.shared.clients.portainer_client import PortainerClient, get_portainer_client
from src.shared.clients.npm_client import NPMClient, get_npm_client
from src.shared.clients.homeassistant_client import HomeAssistantClient, get_homeassistant_client
from src.shared.clients.authentik_client import AuthentikClient, get_authentik_client
from src.shared.clients.qdrant_client import QdrantReadClient, get_qdrant_client
__all__ = [
"PortainerClient",
"get_portainer_client",
"NPMClient",
"get_npm_client",
"HomeAssistantClient",
"get_homeassistant_client",
"AuthentikClient",
"get_authentik_client",
"QdrantReadClient",
"get_qdrant_client",
]
+302
View File
@@ -0,0 +1,302 @@
"""
Authentik API Client
Provides methods for interacting with Authentik Identity Provider API.
Used for managing applications, providers, and authentication flows.
"""
import httpx
from typing import Dict, List, Any, Optional
from functools import lru_cache
from src.shared.logging import get_logger
logger = get_logger(__name__)
class AuthentikClient:
"""Client for Authentik API operations"""
def __init__(self, base_url: str, api_token: str):
"""
Initialize Authentik client
Args:
base_url: Authentik base URL (e.g., http://authentik-server:9000)
api_token: API token for authentication
"""
self.base_url = base_url.rstrip('/')
self.api_token = api_token
self.client = httpx.AsyncClient(timeout=30.0)
async def _request(self, method: str, endpoint: str, **kwargs) -> Dict:
"""Make authenticated API request using token auth"""
headers = kwargs.pop("headers", {})
headers["Authorization"] = f"Bearer {self.api_token}"
response = await self.client.request(
method,
f"{self.base_url}/api/v3/{endpoint.lstrip('/')}",
headers=headers,
**kwargs
)
if not response.is_success:
logger.error(f"API request failed: {response.status_code}")
logger.error(f"Response body: {response.text}")
response.raise_for_status()
return response.json()
async def health_check(self) -> bool:
"""Check if Authentik is accessible"""
try:
response = await self.client.get(f"{self.base_url}/-/health/live/")
return response.status_code == 200
except Exception as e:
logger.error(f"Authentik health check failed: {e}")
return False
async def create_oauth2_provider(
self,
name: str,
client_id: str,
redirect_uris: List[str],
authorization_flow_slug: str = "default-provider-authorization-implicit-consent",
signing_key: Optional[str] = None
) -> Dict:
"""
Create an OAuth2/OIDC provider
Args:
name: Provider name
client_id: OAuth2 client ID
redirect_uris: List of allowed redirect URIs
authorization_flow_slug: Authorization flow slug (will be resolved to UUID)
signing_key: Signing key UUID (defaults to auto-selected)
Returns:
Created provider data including client_secret
"""
# Get authorization flow UUID from slug
flows = await self.list_flows()
auth_flow_uuid = None
invalidation_flow_uuid = None
for flow in flows:
if flow.get("slug") == authorization_flow_slug:
auth_flow_uuid = flow.get("pk")
if flow.get("slug") == "default-provider-invalidation-flow":
invalidation_flow_uuid = flow.get("pk")
if not auth_flow_uuid:
raise ValueError(f"Authorization flow '{authorization_flow_slug}' not found")
if not invalidation_flow_uuid:
raise ValueError("Invalidation flow not found")
# Get signing key if not provided
if not signing_key:
keys = await self._request("GET", "crypto/certificatekeypairs/")
# Find the self-signed cert
for key in keys.get("results", []):
if "authentik" in key.get("name", "").lower():
signing_key = key.get("pk")
break
if not signing_key and keys.get("results"):
signing_key = keys["results"][0]["pk"]
# Format redirect URIs as objects with matching_mode
formatted_redirect_uris = [
{"url": uri, "matching_mode": "strict"}
for uri in redirect_uris
]
provider_data = {
"name": name,
"authorization_flow": auth_flow_uuid,
"invalidation_flow": invalidation_flow_uuid,
"client_type": "confidential",
"client_id": client_id,
"redirect_uris": formatted_redirect_uris,
"signing_key": signing_key,
"sub_mode": "hashed_user_id",
"include_claims_in_id_token": True,
"issuer_mode": "per_provider",
"access_token_validity": "minutes=60",
"refresh_token_validity": "days=30",
"property_mappings": [] # Will use default mappings
}
result = await self._request("POST", "providers/oauth2/", json=provider_data)
logger.info(f"Created OAuth2 provider: {name} (ID: {result.get('pk')})")
return result
async def create_application(
self,
name: str,
slug: str,
provider_pk: int,
launch_url: Optional[str] = None,
icon_url: Optional[str] = None
) -> Dict:
"""
Create an application
Args:
name: Application display name
slug: Application slug (URL-safe identifier)
provider_pk: Primary key of the provider to use
launch_url: Optional launch URL
icon_url: Optional icon URL
Returns:
Created application data
"""
app_data = {
"name": name,
"slug": slug,
"provider": provider_pk,
"meta_launch_url": launch_url or "",
"meta_icon": icon_url or "",
"policy_engine_mode": "any",
"open_in_new_tab": False
}
result = await self._request("POST", "core/applications/", json=app_data)
logger.info(f"Created application: {name} (slug: {slug})")
return result
async def get_provider_by_name(self, name: str) -> Optional[Dict]:
"""Get OAuth2 provider by name"""
providers = await self._request("GET", "providers/oauth2/", params={"name": name})
results = providers.get("results", [])
return results[0] if results else None
async def get_application_by_slug(self, slug: str) -> Optional[Dict]:
"""Get application by slug"""
apps = await self._request("GET", "core/applications/", params={"slug": slug})
results = apps.get("results", [])
return results[0] if results else None
async def list_flows(self) -> List[Dict]:
"""List all authentication flows"""
result = await self._request("GET", "flows/instances/")
return result.get("results", [])
async def create_proxy_provider(
self,
name: str,
external_host: str,
authorization_flow_slug: str = "default-provider-authorization-implicit-consent",
mode: str = "forward_single",
token_validity: int = 480 # 8 hours in minutes
) -> Dict:
"""
Create a Proxy Provider for forward authentication
Args:
name: Provider name
external_host: External URL (e.g., https://auth.schweitz.net)
authorization_flow_slug: Authorization flow slug
mode: Proxy mode (forward_single for forward auth)
token_validity: Token validity in minutes (default: 480 = 8 hours)
Returns:
Created provider data
"""
# Get authorization flow UUID from slug
flows = await self.list_flows()
auth_flow_uuid = None
invalidation_flow_uuid = None
for flow in flows:
if flow.get("slug") == authorization_flow_slug:
auth_flow_uuid = flow.get("pk")
if flow.get("slug") == "default-provider-invalidation-flow":
invalidation_flow_uuid = flow.get("pk")
if not auth_flow_uuid:
raise ValueError(f"Authorization flow '{authorization_flow_slug}' not found")
if not invalidation_flow_uuid:
raise ValueError("Invalidation flow not found")
provider_data = {
"name": name,
"authorization_flow": auth_flow_uuid,
"invalidation_flow": invalidation_flow_uuid,
"mode": mode,
"external_host": external_host,
"access_token_validity": f"minutes={token_validity}",
"refresh_token_validity": f"minutes={token_validity}",
"session_duration": f"seconds={token_validity * 60}",
"cookie_domain": "", # Will use the domain of each proxied site
"property_mappings": []
}
result = await self._request("POST", "providers/proxy/", json=provider_data)
logger.info(f"Created Proxy provider: {name} (ID: {result.get('pk')})")
return result
async def get_provider_by_name_proxy(self, name: str) -> Optional[Dict]:
"""Get Proxy provider by name"""
providers = await self._request("GET", "providers/proxy/", params={"name": name})
results = providers.get("results", [])
return results[0] if results else None
async def create_outpost(
self,
name: str,
type: str,
providers: List[int],
config: Optional[Dict] = None
) -> Dict:
"""
Create an Authentik Outpost
Args:
name: Outpost name
type: Outpost type (e.g., "proxy")
providers: List of provider PKs
config: Optional configuration overrides
Returns:
Created outpost data
"""
outpost_data = {
"name": name,
"type": type,
"providers": providers,
"config": config or {},
"service_connection": None # Will use local Docker
}
result = await self._request("POST", "outposts/instances/", json=outpost_data)
logger.info(f"Created outpost: {name} (ID: {result.get('pk')})")
return result
async def get_outpost_by_name(self, name: str) -> Optional[Dict]:
"""Get outpost by name"""
outposts = await self._request("GET", "outposts/instances/", params={"name": name})
results = outposts.get("results", [])
return results[0] if results else None
async def close(self):
"""Close HTTP client"""
await self.client.aclose()
@lru_cache()
def get_authentik_client() -> AuthentikClient:
"""Get cached Authentik client instance"""
# Import credentials from gitignored module
try:
from src.credentials import AUTHENTIK_URL, AUTHENTIK_CORE_API_TOKEN
except ImportError:
# Fallback to environment variables if credentials.py doesn't exist
import os
AUTHENTIK_URL = os.getenv("AUTHENTIK_URL", "http://authentik-server:9000")
AUTHENTIK_CORE_API_TOKEN = os.getenv("AUTHENTIK_API_TOKEN", "")
return AuthentikClient(
base_url=AUTHENTIK_URL,
api_token=AUTHENTIK_CORE_API_TOKEN
)
+409
View File
@@ -0,0 +1,409 @@
"""
Home Assistant REST API Client
Provides interface to Home Assistant REST API for home automation control.
Uses long-lived access token authentication.
API Reference: https://developers.home-assistant.io/docs/api/rest/
"""
import httpx
import json
from typing import Optional, Dict, List, Any
from datetime import datetime, timedelta, timezone
from src.shared.logging import get_logger
from src.shared.config import get_settings
logger = get_logger(__name__)
settings = get_settings()
class HomeAssistantClient:
"""
HTTP client for Home Assistant REST API
Uses long-lived access token authentication via Bearer token.
"""
def __init__(
self,
base_url: Optional[str] = None,
token: Optional[str] = None,
timeout: int = 30
):
"""
Initialize Home Assistant client
Args:
base_url: Home Assistant base URL (default from settings)
token: Long-lived access token (default from settings)
timeout: Request timeout in seconds
"""
self.base_url = (base_url or settings.homeassistant_url).rstrip("/")
self.token = token or settings.homeassistant_token
self.timeout = timeout
if not self.token:
logger.warning("Home Assistant token not configured")
def _get_headers(self) -> Dict[str, str]:
"""Get request headers with Bearer token authentication"""
return {
"Authorization": f"Bearer {self.token}",
"Content-Type": "application/json"
}
# ========================================================================
# Health & Discovery
# ========================================================================
async def health_check(self) -> Dict[str, Any]:
"""
Check Home Assistant API connectivity and get version info
HA Endpoint: GET /api/
Returns:
Dict with connected status, platform name, and version
"""
try:
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.get(
f"{self.base_url}/api/",
headers=self._get_headers()
)
if response.status_code == 200:
data = response.json()
return {
"status": "healthy",
"connected": True,
"platform": "home_assistant",
"version": data.get("version", "unknown")
}
return {
"status": "unhealthy",
"connected": False,
"platform": "home_assistant",
"error": f"HTTP {response.status_code}"
}
except Exception as e:
logger.error(f"Home Assistant health check failed: {e}")
return {
"status": "unhealthy",
"connected": False,
"platform": "home_assistant",
"error": str(e)
}
async def get_states(self) -> List[Dict[str, Any]]:
"""
Get all entity states
HA Endpoint: GET /api/states
Returns:
List of all entity states
"""
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.get(
f"{self.base_url}/api/states",
headers=self._get_headers()
)
response.raise_for_status()
return response.json()
async def get_state(self, entity_id: str) -> Optional[Dict[str, Any]]:
"""
Get state of a specific entity
HA Endpoint: GET /api/states/<entity_id>
Args:
entity_id: Entity ID (e.g., "light.living_room")
Returns:
Entity state dict or None if not found
"""
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.get(
f"{self.base_url}/api/states/{entity_id}",
headers=self._get_headers()
)
if response.status_code == 404:
return None
response.raise_for_status()
return response.json()
async def get_config(self) -> Dict[str, Any]:
"""
Get Home Assistant configuration (includes areas)
HA Endpoint: GET /api/config
Returns:
Configuration dict including components, location, etc.
"""
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.get(
f"{self.base_url}/api/config",
headers=self._get_headers()
)
response.raise_for_status()
return response.json()
# ========================================================================
# Device Control
# ========================================================================
async def call_service(
self,
domain: str,
service: str,
entity_id: Optional[str] = None,
service_data: Optional[Dict[str, Any]] = None
) -> List[Dict[str, Any]]:
"""
Call a Home Assistant service
HA Endpoint: POST /api/services/<domain>/<service>
Args:
domain: Service domain (e.g., "light", "switch", "scene")
service: Service name (e.g., "turn_on", "turn_off", "toggle")
entity_id: Target entity ID (optional for some services)
service_data: Additional service data/attributes
Returns:
List of changed states
"""
payload = service_data.copy() if service_data else {}
if entity_id:
payload["entity_id"] = entity_id
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.post(
f"{self.base_url}/api/services/{domain}/{service}",
headers=self._get_headers(),
json=payload
)
response.raise_for_status()
return response.json()
async def turn_on(
self,
entity_id: str,
**attributes
) -> List[Dict[str, Any]]:
"""
Turn on an entity with optional attributes
Args:
entity_id: Entity ID (e.g., "light.living_room")
**attributes: Additional attributes (brightness, color_temp, etc.)
Returns:
List of changed states
"""
domain = entity_id.split(".")[0]
return await self.call_service(
domain=domain,
service="turn_on",
entity_id=entity_id,
service_data=attributes if attributes else None
)
async def turn_off(self, entity_id: str) -> List[Dict[str, Any]]:
"""
Turn off an entity
Args:
entity_id: Entity ID
Returns:
List of changed states
"""
domain = entity_id.split(".")[0]
return await self.call_service(
domain=domain,
service="turn_off",
entity_id=entity_id
)
async def toggle(self, entity_id: str) -> List[Dict[str, Any]]:
"""
Toggle an entity
Args:
entity_id: Entity ID
Returns:
List of changed states
"""
domain = entity_id.split(".")[0]
return await self.call_service(
domain=domain,
service="toggle",
entity_id=entity_id
)
# ========================================================================
# Scenes
# ========================================================================
async def activate_scene(self, scene_id: str) -> List[Dict[str, Any]]:
"""
Activate a scene
Args:
scene_id: Scene entity ID (e.g., "scene.movie_night")
Returns:
List of changed states
"""
return await self.call_service(
domain="scene",
service="turn_on",
entity_id=scene_id
)
# ========================================================================
# Scripts
# ========================================================================
async def run_script(
self,
script_id: str,
variables: Optional[Dict[str, Any]] = None
) -> List[Dict[str, Any]]:
"""
Execute a script with optional variables
Args:
script_id: Script entity ID (e.g., "script.bedtime_routine")
variables: Script variables
Returns:
List of changed states
"""
service_data = {"variables": variables} if variables else None
return await self.call_service(
domain="script",
service="turn_on",
entity_id=script_id,
service_data=service_data
)
# ========================================================================
# Automations
# ========================================================================
async def enable_automation(self, automation_id: str) -> List[Dict[str, Any]]:
"""
Enable an automation
Args:
automation_id: Automation entity ID
Returns:
List of changed states
"""
return await self.call_service(
domain="automation",
service="turn_on",
entity_id=automation_id
)
async def disable_automation(self, automation_id: str) -> List[Dict[str, Any]]:
"""
Disable an automation
Args:
automation_id: Automation entity ID
Returns:
List of changed states
"""
return await self.call_service(
domain="automation",
service="turn_off",
entity_id=automation_id
)
# ========================================================================
# History
# ========================================================================
async def get_history(
self,
entity_id: str,
hours: int = 24
) -> List[List[Dict[str, Any]]]:
"""
Get state history for an entity
HA Endpoint: GET /api/history/period/<timestamp>
Args:
entity_id: Entity ID to get history for
hours: Number of hours of history (default 24)
Returns:
List of state history entries
"""
start_time = datetime.now(timezone.utc) - timedelta(hours=hours)
timestamp = start_time.isoformat()
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.get(
f"{self.base_url}/api/history/period/{timestamp}",
headers=self._get_headers(),
params={
"filter_entity_id": entity_id,
"minimal_response": "true"
}
)
response.raise_for_status()
return response.json()
# ========================================================================
# Areas (via template API)
# ========================================================================
async def get_areas(self) -> List[Dict[str, str]]:
"""
Get all areas/rooms
Note: The REST API doesn't have a direct areas endpoint.
This uses the template API to render area data.
HA Endpoint: POST /api/template
Returns:
List of area dicts with id and name
"""
template = """
{% set areas_list = [] %}
{% for area in areas() %}
{% set areas_list = areas_list + [{"id": area, "name": area_name(area)}] %}
{% endfor %}
{{ areas_list | tojson }}
"""
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.post(
f"{self.base_url}/api/template",
headers=self._get_headers(),
json={"template": template}
)
response.raise_for_status()
# Response is rendered template as string
return json.loads(response.text)
# Singleton instance
_homeassistant_client: Optional[HomeAssistantClient] = None
def get_homeassistant_client() -> HomeAssistantClient:
"""Get singleton Home Assistant client instance"""
global _homeassistant_client
if _homeassistant_client is None:
_homeassistant_client = HomeAssistantClient()
return _homeassistant_client
+383
View File
@@ -0,0 +1,383 @@
"""
Nginx Proxy Manager API Client
Provides interface to NPM REST API for proxy host and SSL certificate management.
"""
import httpx
from typing import Optional, Dict, List, Any
from datetime import datetime, timedelta
from src.shared.logging import get_logger
from src.shared.config import get_settings
logger = get_logger(__name__)
settings = get_settings()
class NPMClient:
"""
HTTP client for Nginx Proxy Manager API
Uses JWT Bearer token authentication with automatic token refresh.
Tokens expire after ~24 hours.
"""
def __init__(
self,
base_url: Optional[str] = None,
email: Optional[str] = None,
password: Optional[str] = None,
timeout: int = 30
):
"""
Initialize NPM client
Args:
base_url: NPM base URL (default from settings)
email: NPM admin email (default from settings)
password: NPM admin password (default from settings)
timeout: Request timeout in seconds
"""
self.base_url = (base_url or settings.npm_url).rstrip("/")
self.email = email or settings.npm_email
self.password = password or settings.npm_password
self.timeout = timeout
self._token: Optional[str] = None
self._token_expires: Optional[datetime] = None
if not self.email or not self.password:
logger.warning("NPM credentials not configured")
async def _ensure_token(self):
"""Ensure we have a valid token, refresh if needed"""
if self._token and self._token_expires:
# If token expires in less than 1 hour, refresh it
if datetime.now() + timedelta(hours=1) < self._token_expires:
return
# Get new token
await self._refresh_token()
async def _refresh_token(self):
"""Get a new authentication token"""
try:
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.post(
f"{self.base_url}/api/tokens",
json={
"identity": self.email,
"secret": self.password
}
)
response.raise_for_status()
data = response.json()
self._token = data.get("token")
# Assume 23-hour expiration to be safe
self._token_expires = datetime.now() + timedelta(hours=23)
logger.info("NPM token refreshed successfully")
except Exception as e:
logger.error(f"Failed to refresh NPM token: {e}")
raise
def _get_headers(self) -> Dict[str, str]:
"""Get request headers with authentication"""
if not self._token:
raise RuntimeError("No NPM token available. Call _ensure_token() first.")
return {
"Authorization": f"Bearer {self._token}",
"Content-Type": "application/json"
}
async def health_check(self) -> bool:
"""
Check if NPM API is accessible
Returns:
True if accessible, False otherwise
"""
try:
async with httpx.AsyncClient(timeout=self.timeout, follow_redirects=True) as client:
response = await client.get(f"{self.base_url}/api")
# Accept any successful response (2xx) or redirect (3xx) as healthy
# A redirect indicates the service is up and responding
return 200 <= response.status_code < 400
except Exception as e:
logger.error(f"NPM health check failed: {e}")
return False
async def get_proxy_hosts(self) -> List[Dict[str, Any]]:
"""
List all proxy hosts
Returns:
List of proxy host configurations
"""
await self._ensure_token()
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.get(
f"{self.base_url}/api/nginx/proxy-hosts",
headers=self._get_headers()
)
response.raise_for_status()
return response.json()
async def get_proxy_host(self, host_id: int) -> Dict[str, Any]:
"""
Get details of a specific proxy host
Args:
host_id: Proxy host identifier
Returns:
Proxy host configuration
"""
await self._ensure_token()
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.get(
f"{self.base_url}/api/nginx/proxy-hosts/{host_id}",
headers=self._get_headers()
)
response.raise_for_status()
return response.json()
async def create_proxy_host(
self,
domain_names: List[str],
forward_host: str,
forward_port: int,
forward_scheme: str = "http",
certificate_id: int = 0,
ssl_forced: bool = False,
block_exploits: bool = True,
caching_enabled: bool = True,
websocket_upgrade: bool = True,
http2_support: bool = True,
hsts_enabled: bool = True,
advanced_config: str = ""
) -> Dict[str, Any]:
"""
Create a new proxy host
Args:
domain_names: List of domain names for this proxy
forward_host: Target host to proxy to
forward_port: Target port to proxy to
forward_scheme: http or https
certificate_id: SSL certificate ID (0 for none)
ssl_forced: Force HTTPS redirect
block_exploits: Enable exploit blocking
caching_enabled: Enable response caching
websocket_upgrade: Allow WebSocket upgrades
http2_support: Enable HTTP/2
hsts_enabled: Enable HSTS headers
advanced_config: Custom nginx configuration
Returns:
Created proxy host details
"""
await self._ensure_token()
payload = {
"domain_names": domain_names,
"forward_scheme": forward_scheme,
"forward_host": forward_host,
"forward_port": forward_port,
"certificate_id": certificate_id,
"ssl_forced": ssl_forced,
"block_exploits": block_exploits,
"caching_enabled": caching_enabled,
"allow_websocket_upgrade": websocket_upgrade,
"http2_support": http2_support,
"hsts_enabled": hsts_enabled,
"hsts_subdomains": False,
"advanced_config": advanced_config,
"access_list_id": 0,
"meta": {}
}
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.post(
f"{self.base_url}/api/nginx/proxy-hosts",
headers=self._get_headers(),
json=payload
)
response.raise_for_status()
return response.json()
async def update_proxy_host(
self,
proxy_id: int,
config: Dict[str, Any]
) -> Dict[str, Any]:
"""
Update an existing proxy host configuration
Args:
proxy_id: Proxy host ID to update
config: Full proxy host configuration (get from get_proxy_host, modify, then update)
Returns:
Updated proxy host details
"""
await self._ensure_token()
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.put(
f"{self.base_url}/api/nginx/proxy-hosts/{proxy_id}",
headers=self._get_headers(),
json=config
)
if not response.is_success:
logger.error(f"Update failed: {response.status_code}")
logger.error(f"Response: {response.text}")
response.raise_for_status()
return response.json()
async def enable_authentik_forward_auth(
self,
proxy_id: int,
authentik_url: str = "http://authentik-server:9000"
) -> Dict[str, Any]:
"""
Enable Authentik forward authentication on a proxy host
Args:
proxy_id: Proxy host ID to update
authentik_url: Authentik server URL (default: http://authentik-server:9000)
Returns:
Updated proxy host details
"""
# Get current config
proxy_host = await self.get_proxy_host(proxy_id)
# Authentik forward auth configuration
auth_config = f"""# Authentik Forward Authentication
# Send authentication requests to Authentik
auth_request /outpost.goauthentik.io/auth/nginx;
# Preserve authentication cookies
auth_request_set $auth_cookie $upstream_http_set_cookie;
add_header Set-Cookie $auth_cookie;
# Get user information from Authentik
auth_request_set $authentik_username $upstream_http_x_authentik_username;
auth_request_set $authentik_groups $upstream_http_x_authentik_groups;
auth_request_set $authentik_email $upstream_http_x_authentik_email;
auth_request_set $authentik_name $upstream_http_x_authentik_name;
auth_request_set $authentik_uid $upstream_http_x_authentik_uid;
# Pass user info to backend
proxy_set_header X-authentik-username $authentik_username;
proxy_set_header X-authentik-groups $authentik_groups;
proxy_set_header X-authentik-email $authentik_email;
proxy_set_header X-authentik-name $authentik_name;
proxy_set_header X-authentik-uid $authentik_uid;
# On authentication failure, redirect to Authentik login
error_page 401 = @authentik_proxy_signin;
location @authentik_proxy_signin {{
internal;
add_header Set-Cookie $auth_cookie;
return 302 /outpost.goauthentik.io/start?rd=$scheme://$http_host$request_uri;
}}
# Authentik authentication endpoint
location /outpost.goauthentik.io {{
proxy_pass {authentik_url}/outpost.goauthentik.io;
proxy_set_header X-Original-URL $scheme://$http_host$request_uri;
proxy_pass_request_body off;
proxy_set_header Content-Length "";
proxy_set_header Host $host;
}}
"""
# Update the advanced config
proxy_host["advanced_config"] = auth_config
# Remove read-only fields that NPM doesn't accept in updates
readonly_fields = [
"id", "created_on", "modified_on", "owner", "owner_user_id",
"certificate", "use_default_location", "ipv6", "meta", "nginx_online",
"nginx_err", "access_list", "certificate_id"
]
clean_config = {k: v for k, v in proxy_host.items() if k not in readonly_fields}
# Ensure locations is an array (required field)
if "locations" not in clean_config or clean_config["locations"] is None:
clean_config["locations"] = []
# Update the proxy host
return await self.update_proxy_host(proxy_id, clean_config)
async def get_certificates(self) -> List[Dict[str, Any]]:
"""
List all SSL certificates
Returns:
List of certificate details
"""
await self._ensure_token()
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.get(
f"{self.base_url}/api/nginx/certificates",
headers=self._get_headers()
)
response.raise_for_status()
return response.json()
async def create_certificate(
self,
domain_names: List[str],
provider: str = "letsencrypt"
) -> Dict[str, Any]:
"""
Request a new SSL certificate from Let's Encrypt
Args:
domain_names: List of domains for the certificate
provider: Certificate provider (default: letsencrypt)
Returns:
Certificate details
"""
await self._ensure_token()
payload = {
"provider": provider,
"domain_names": domain_names,
"meta": {
"dns_challenge": False
}
}
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.post(
f"{self.base_url}/api/nginx/certificates",
headers=self._get_headers(),
json=payload
)
response.raise_for_status()
return response.json()
# Singleton instance
_npm_client: Optional[NPMClient] = None
def get_npm_client() -> NPMClient:
"""Get singleton NPM client instance"""
global _npm_client
if _npm_client is None:
_npm_client = NPMClient()
return _npm_client
+505
View File
@@ -0,0 +1,505 @@
"""
Portainer API Client
Provides interface to Portainer REST API for stack and container management.
Includes fallback to Docker socket for containers not managed by Portainer.
"""
import httpx
from typing import Optional, Dict, List, Any
from src.shared.logging import get_logger
from src.shared.config import get_settings
logger = get_logger(__name__)
settings = get_settings()
class PortainerClient:
"""
HTTP client for Portainer API
Uses access token authentication (X-API-Key header)
for long-lived API access without session management.
"""
def __init__(
self,
base_url: Optional[str] = None,
api_key: Optional[str] = None,
timeout: int = 30
):
"""
Initialize Portainer client
Args:
base_url: Portainer base URL (default from settings)
api_key: Portainer API access token (default from settings)
timeout: Request timeout in seconds
"""
self.base_url = (base_url or settings.portainer_url).rstrip("/")
self.api_key = api_key or settings.portainer_api_key
self.timeout = timeout
if not self.api_key:
logger.warning("Portainer API key not configured")
def _get_headers(self) -> Dict[str, str]:
"""Get request headers with authentication"""
return {
"X-API-Key": self.api_key,
"Content-Type": "application/json"
}
async def health_check(self) -> bool:
"""
Check if Portainer API is accessible
Returns:
True if accessible, False otherwise
"""
try:
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.get(f"{self.base_url}/api/status")
return response.status_code == 200
except Exception as e:
logger.error(f"Portainer health check failed: {e}")
return False
async def get_endpoints(self) -> List[Dict[str, Any]]:
"""
List all Portainer endpoints (Docker environments)
Returns:
List of endpoint configurations
"""
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.get(
f"{self.base_url}/api/endpoints",
headers=self._get_headers()
)
response.raise_for_status()
return response.json()
async def get_stacks(self, endpoint_id: Optional[int] = None) -> List[Dict[str, Any]]:
"""
List all stacks
Args:
endpoint_id: Filter by specific endpoint (optional)
Returns:
List of stack configurations
"""
params = {}
if endpoint_id:
params["endpointId"] = endpoint_id
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.get(
f"{self.base_url}/api/stacks",
headers=self._get_headers(),
params=params
)
response.raise_for_status()
return response.json()
async def get_stack(self, stack_id: int) -> Dict[str, Any]:
"""
Get details of a specific stack
Args:
stack_id: Stack identifier
Returns:
Stack configuration details
"""
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.get(
f"{self.base_url}/api/stacks/{stack_id}",
headers=self._get_headers()
)
response.raise_for_status()
return response.json()
async def create_stack(
self,
name: str,
stack_file_content: str,
endpoint_id: int
) -> Dict[str, Any]:
"""
Create a new stack from compose file content
Args:
name: Stack name
stack_file_content: Docker Compose YAML content
endpoint_id: Portainer endpoint to deploy to
Returns:
Created stack details
"""
payload = {
"name": name,
"stackFileContent": stack_file_content
}
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.post(
f"{self.base_url}/api/stacks/create/standalone/string",
headers=self._get_headers(),
params={"endpointId": endpoint_id},
json=payload
)
response.raise_for_status()
return response.json()
async def update_stack(
self,
stack_id: int,
stack_file_content: str,
endpoint_id: int,
prune: bool = False,
pull_image: bool = False
) -> Dict[str, Any]:
"""
Update an existing stack
Args:
stack_id: Stack identifier
stack_file_content: New Docker Compose YAML content
endpoint_id: Portainer endpoint
prune: Remove services no longer defined
pull_image: Pull latest images before deployment
Returns:
Updated stack details
"""
payload = {
"stackFileContent": stack_file_content,
"prune": prune,
"pullImage": pull_image
}
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.put(
f"{self.base_url}/api/stacks/{stack_id}",
headers=self._get_headers(),
params={"endpointId": endpoint_id},
json=payload
)
response.raise_for_status()
return response.json()
async def delete_stack(self, stack_id: int, endpoint_id: int) -> bool:
"""
Delete a stack
Args:
stack_id: Stack identifier
endpoint_id: Portainer endpoint
Returns:
True if successful
"""
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.delete(
f"{self.base_url}/api/stacks/{stack_id}",
headers=self._get_headers(),
params={"endpointId": endpoint_id}
)
response.raise_for_status()
return True
async def get_stack_file(self, stack_id: int) -> str:
"""
Get the compose file content for a stack
Args:
stack_id: Stack identifier
Returns:
Docker Compose YAML content as string
"""
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.get(
f"{self.base_url}/api/stacks/{stack_id}/file",
headers=self._get_headers()
)
response.raise_for_status()
data = response.json()
return data.get("StackFileContent", "")
async def redeploy_stack(
self,
stack_id: int,
endpoint_id: int,
pull_image: bool = False
) -> Dict[str, Any]:
"""
Redeploy a stack with its current configuration
Args:
stack_id: Stack identifier
endpoint_id: Portainer endpoint
pull_image: Pull latest images before deployment
Returns:
Updated stack details
"""
# Get current stack file content
stack_content = await self.get_stack_file(stack_id)
# Get current stack to preserve env vars
stack = await self.get_stack(stack_id)
env_vars = stack.get("Env", [])
payload = {
"stackFileContent": stack_content,
"env": env_vars,
"prune": False,
"pullImage": pull_image
}
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.put(
f"{self.base_url}/api/stacks/{stack_id}",
headers=self._get_headers(),
params={"endpointId": endpoint_id},
json=payload
)
response.raise_for_status()
return response.json()
async def update_stack_env(
self,
stack_id: int,
endpoint_id: int,
env_vars: List[Dict[str, str]]
) -> Dict[str, Any]:
"""
Update stack environment variables
Args:
stack_id: Stack identifier
endpoint_id: Portainer endpoint
env_vars: List of {"name": "VAR_NAME", "value": "var_value"} dicts
Returns:
Updated stack details
"""
# Get current stack file content (required for update)
stack_content = await self.get_stack_file(stack_id)
payload = {
"stackFileContent": stack_content,
"env": env_vars,
"prune": False,
"pullImage": False
}
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.put(
f"{self.base_url}/api/stacks/{stack_id}",
headers=self._get_headers(),
params={"endpointId": endpoint_id},
json=payload
)
response.raise_for_status()
return response.json()
async def delete_container(
self,
endpoint_id: int,
container_id: str,
force: bool = False
) -> bool:
"""
Delete a container
Args:
endpoint_id: Portainer endpoint identifier
container_id: Container ID or name
force: Force remove running container
Returns:
True if successful
"""
params = {"force": "true" if force else "false"}
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.delete(
f"{self.base_url}/api/endpoints/{endpoint_id}/docker/containers/{container_id}",
headers=self._get_headers(),
params=params
)
response.raise_for_status()
logger.info(f"Deleted container {container_id}")
return True
async def restart_container(self, endpoint_id: int, container_id: str) -> bool:
"""
Restart a container
Args:
endpoint_id: Portainer endpoint identifier
container_id: Container ID or name
Returns:
True if successful
"""
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.post(
f"{self.base_url}/api/endpoints/{endpoint_id}/docker/containers/{container_id}/restart",
headers=self._get_headers()
)
response.raise_for_status()
logger.info(f"Restarted container {container_id}")
return True
async def get_containers(self, endpoint_id: int, all_containers: bool = True) -> List[Dict[str, Any]]:
"""
List containers on a specific endpoint
Args:
endpoint_id: Portainer endpoint identifier
all_containers: Include stopped containers (default: True)
Returns:
List of container details
"""
params = {"all": 1 if all_containers else 0}
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.get(
f"{self.base_url}/api/endpoints/{endpoint_id}/docker/containers/json",
headers=self._get_headers(),
params=params
)
response.raise_for_status()
return response.json()
async def get_container(self, endpoint_id: int, container_id: str) -> Dict[str, Any]:
"""
Get detailed information about a specific container
Args:
endpoint_id: Portainer endpoint identifier
container_id: Container ID or name
Returns:
Container details including network and port information
"""
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.get(
f"{self.base_url}/api/endpoints/{endpoint_id}/docker/containers/{container_id}/json",
headers=self._get_headers()
)
response.raise_for_status()
return response.json()
async def stop_container(self, endpoint_id: int, container_id: str) -> bool:
"""
Stop a container
Args:
endpoint_id: Portainer endpoint identifier
container_id: Container ID or name
Returns:
True if successful
"""
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.post(
f"{self.base_url}/api/endpoints/{endpoint_id}/docker/containers/{container_id}/stop",
headers=self._get_headers()
)
response.raise_for_status()
logger.info(f"Stopped container {container_id}")
return True
async def start_container(self, endpoint_id: int, container_id: str) -> bool:
"""
Start a container
Args:
endpoint_id: Portainer endpoint identifier
container_id: Container ID or name
Returns:
True if successful
"""
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.post(
f"{self.base_url}/api/endpoints/{endpoint_id}/docker/containers/{container_id}/start",
headers=self._get_headers()
)
response.raise_for_status()
logger.info(f"Started container {container_id}")
return True
# ========================================================================
# Helper methods for agent tools (auto-detect endpoint)
# ========================================================================
async def list_containers(self, all_containers: bool = True) -> List[Dict[str, Any]]:
"""
List containers using auto-detected endpoint
This is a convenience wrapper that automatically uses the first/default endpoint.
Args:
all_containers: Include stopped containers (default: True)
Returns:
List of container details
"""
endpoints = await self.get_endpoints()
if not endpoints:
raise RuntimeError("No Portainer endpoints available")
endpoint_id = endpoints[0]["Id"]
return await self.get_containers(endpoint_id, all_containers)
async def inspect_container(self, container_name: str) -> Optional[Dict[str, Any]]:
"""
Inspect a container by name using auto-detected endpoint
This is a convenience wrapper that automatically uses the first/default endpoint.
Args:
container_name: Container name (e.g., "jellyfin", "ollama")
Returns:
Container details or None if not found
"""
endpoints = await self.get_endpoints()
if not endpoints:
raise RuntimeError("No Portainer endpoints available")
endpoint_id = endpoints[0]["Id"]
# List all containers to find the one matching the name
all_containers = await self.get_containers(endpoint_id, all_containers=True)
for container in all_containers:
# Container names come as array like ['/jellyfin']
names = container.get('Names', [])
for name in names:
clean_name = name.lstrip('/')
if clean_name == container_name or clean_name.lower() == container_name.lower():
# Get detailed info using container ID
container_id = container['Id']
return await self.get_container(endpoint_id, container_id)
return None
# Singleton instance
_portainer_client: Optional[PortainerClient] = None
def get_portainer_client() -> PortainerClient:
"""Get singleton Portainer client instance"""
global _portainer_client
if _portainer_client is None:
_portainer_client = PortainerClient()
return _portainer_client
+249
View File
@@ -0,0 +1,249 @@
"""
Qdrant Vector Database Client (Read-Only)
Provides read-only access to Qdrant collections for querying volatile data.
Used to fetch weather, forecast, and sun times from the volatile_{user} collection.
"""
import time
from typing import List, Dict, Any, Optional
from qdrant_client import QdrantClient
from qdrant_client.models import Filter, FieldCondition, MatchValue, Range
from src.shared.logging import get_logger
from src.shared.config import get_settings
logger = get_logger(__name__)
settings = get_settings()
class QdrantReadClient:
"""
Read-only Qdrant client for accessing volatile data.
Connects to Qdrant and provides methods to query collections
with filtering by namespace and TTL expiry.
"""
VOLATILE_COLLECTION_PREFIX = "volatile_"
def __init__(
self,
host: Optional[str] = None,
port: Optional[int] = None,
):
"""
Initialize Qdrant read client.
Args:
host: Qdrant server host (default from settings)
port: Qdrant server port (default from settings)
"""
self.host = host or settings.qdrant_host
self.port = port or settings.qdrant_port
self._client: Optional[QdrantClient] = None
logger.info(f"Initialized QdrantReadClient: {self.host}:{self.port}")
@property
def client(self) -> QdrantClient:
"""Lazy-load Qdrant client connection."""
if self._client is None:
self._client = QdrantClient(
host=self.host,
port=self.port,
)
return self._client
def _get_volatile_collection(self, user: str) -> str:
"""Get volatile collection name for user."""
return f"{self.VOLATILE_COLLECTION_PREFIX}{user}"
def _current_timestamp_ms(self) -> int:
"""Get current timestamp in milliseconds."""
return int(time.time() * 1000)
async def collection_exists(self, collection_name: str) -> bool:
"""
Check if a collection exists.
Args:
collection_name: Name of collection to check
Returns:
True if collection exists
"""
try:
collections = self.client.get_collections()
existing = [c.name for c in collections.collections]
return collection_name in existing
except Exception as e:
logger.error(f"Error checking collection existence: {e}")
return False
async def get_by_namespace(
self,
user: str,
namespace: str,
include_expired: bool = False
) -> List[Dict[str, Any]]:
"""
Get all records for a specific namespace from user's volatile collection.
Args:
user: User identifier (e.g., 'jpmschweitzer' or 'default')
namespace: Namespace to filter (e.g., 'weather', 'forecast', 'sun')
include_expired: Whether to include expired records (default False)
Returns:
List of records with payload data
"""
collection_name = self._get_volatile_collection(user)
if not await self.collection_exists(collection_name):
logger.debug(f"Collection {collection_name} does not exist")
return []
# Build filter conditions
conditions = [
FieldCondition(
key="namespace",
match=MatchValue(value=namespace)
)
]
# Add TTL expiry filter unless including expired
if not include_expired:
now_ms = self._current_timestamp_ms()
conditions.append(
FieldCondition(
key="ttl_expiry",
range=Range(gt=now_ms)
)
)
query_filter = Filter(must=conditions)
try:
# Scroll through matching records
points, _ = self.client.scroll(
collection_name=collection_name,
scroll_filter=query_filter,
limit=100,
with_payload=True,
with_vectors=False
)
results = []
for point in points:
payload = dict(point.payload) if point.payload else {}
results.append({
"id": str(point.id),
"namespace": payload.get("namespace"),
"key": payload.get("key"),
"raw_data": payload.get("raw_data", {}),
"source": payload.get("source"),
"ttl_expiry": payload.get("ttl_expiry"),
"updated_at": payload.get("updated_at"),
})
logger.debug(
f"Found {len(results)} records in {collection_name}/{namespace}"
)
return results
except Exception as e:
logger.error(f"Error fetching from {collection_name}/{namespace}: {e}")
return []
async def get_environment_data(
self,
user: str
) -> Dict[str, Any]:
"""
Get all environment data (weather, forecast, sun times) for a user.
Convenience method that fetches all environment-related namespaces
in a single call.
Args:
user: User identifier
Returns:
Dict with 'weather', 'forecast', 'sun_times', 'air_quality' keys
(each may be None if no data found)
"""
result = {
"weather": None,
"forecast": None,
"sun_times": None,
"air_quality": None,
}
# Fetch weather data
weather_records = await self.get_by_namespace(user, "weather")
if weather_records:
# Get the first/most recent weather record
result["weather"] = weather_records[0].get("raw_data")
# Check if air quality is embedded in weather data
if result["weather"]:
aqi = result["weather"].get("aqi") or result["weather"].get("air_quality")
if aqi:
result["air_quality"] = aqi if isinstance(aqi, dict) else {"aqi": aqi}
# Fetch forecast data - pass raw_data to service for parsing
forecast_records = await self.get_by_namespace(user, "forecast")
if forecast_records:
result["forecast"] = forecast_records[0].get("raw_data")
# Fetch sun times data
sun_records = await self.get_by_namespace(user, "sun")
if sun_records:
result["sun_times"] = sun_records[0].get("raw_data")
# Check for separate air quality namespace if not embedded
if result["air_quality"] is None:
aq_records = await self.get_by_namespace(user, "air_quality")
if aq_records:
result["air_quality"] = aq_records[0].get("raw_data")
return result
async def health_check(self) -> Dict[str, Any]:
"""
Check Qdrant connectivity.
Returns:
Dict with connection status and info
"""
try:
collections = self.client.get_collections()
volatile_collections = [
c.name for c in collections.collections
if c.name.startswith(self.VOLATILE_COLLECTION_PREFIX)
]
return {
"status": "healthy",
"connected": True,
"host": f"{self.host}:{self.port}",
"volatile_collections": volatile_collections,
}
except Exception as e:
logger.error(f"Qdrant health check failed: {e}")
return {
"status": "unhealthy",
"connected": False,
"host": f"{self.host}:{self.port}",
"error": str(e),
}
# Singleton instance for reuse
_qdrant_client: Optional[QdrantReadClient] = None
def get_qdrant_client() -> QdrantReadClient:
"""Get or create singleton Qdrant client instance."""
global _qdrant_client
if _qdrant_client is None:
_qdrant_client = QdrantReadClient()
return _qdrant_client
+116
View File
@@ -0,0 +1,116 @@
"""
Global configuration for Core Code API
All configuration is loaded from environment variables or .env file.
See .env.example for available settings.
"""
import tomllib
from pathlib import Path
from pydantic_settings import BaseSettings
from functools import lru_cache
def _get_version_from_pyproject() -> str:
"""Load version from pyproject.toml"""
pyproject_path = Path(__file__).parent.parent.parent / "pyproject.toml"
try:
with open(pyproject_path, "rb") as f:
data = tomllib.load(f)
return data.get("project", {}).get("version", "0.0.0")
except FileNotFoundError:
return "0.0.0"
__version__ = _get_version_from_pyproject()
class Settings(BaseSettings):
"""Global application settings"""
# Application
app_name: str = "Core Code API"
app_version: str = __version__
debug: bool = False
# Server
host: str = "0.0.0.0"
port: int = 8083
# CORS - Note: When cors_credentials is True, cannot use "*" for origins
# Set CORS_ORIGINS env var to override (comma-separated list)
cors_origins: list[str] = [
"https://home.schweitz.net",
"https://tatlock.schweitz.net",
"http://localhost:8080",
"http://localhost:3000",
"http://127.0.0.1:8080",
]
cors_credentials: bool = True
cors_methods: list[str] = ["*"]
cors_headers: list[str] = ["*"]
# Logging
log_level: str = "DEBUG"
# Qdrant Configuration
qdrant_host: str = "qdrant"
qdrant_port: int = 6333
qdrant_collection_conversations: str = "core_api_conversations"
qdrant_collection_documents: str = "core_api_documents"
qdrant_collection_user_facts: str = "core_api_user_facts"
# Search Configuration
search_provider: str = "searxng"
searxng_url: str # Required - set SEARXNG_URL in .env
# Infrastructure Management (Portainer)
portainer_url: str # Required
portainer_api_key: str # Required
# Infrastructure Management (Nginx Proxy Manager)
npm_url: str # Required
npm_email: str # Required
npm_password: str # Required
# Home Assistant Configuration
homeassistant_url: str # Required
homeassistant_token: str # Required
homeassistant_timeout: int = 30
# PostgreSQL Database
postgres_host: str # Required
postgres_user: str = "core_api"
postgres_password: str # Required
postgres_database: str = "core_api"
@property
def database_url(self) -> str:
"""Construct database URL from components"""
return f"postgresql://{self.postgres_user}:{self.postgres_password}@{self.postgres_host}/{self.postgres_database}"
# OIDC Authentication (Authentik)
oidc_enabled: bool = False
# Accept tokens from multiple OAuth providers (each has its own issuer/JWKS)
oidc_issuers: list[str] = [
"https://auth.schweitz.net/application/o/core-api/",
"https://auth.schweitz.net/application/o/tatlock-ui/",
"https://auth.schweitz.net/application/o/tatlock/",
]
# Accept tokens from multiple clients
oidc_audiences: list[str] = ["core-api", "tatlock-ui", "tatlock"]
# Authentik API (for token validation and user management)
authentik_url: str = "https://auth.schweitz.net"
authentik_username: str = ""
authentik_password: str = ""
class Config:
env_file = ".env"
case_sensitive = False
extra = "ignore"
@lru_cache()
def get_settings() -> Settings:
"""Cached settings instance"""
return Settings()
+135
View File
@@ -0,0 +1,135 @@
"""
Database Connection Module
Provides async PostgreSQL connectivity using SQLAlchemy 2.0 with asyncpg driver.
"""
from typing import AsyncGenerator, Optional
from sqlalchemy import text
from sqlalchemy.ext.asyncio import (
AsyncSession,
AsyncEngine,
create_async_engine,
async_sessionmaker,
)
from sqlalchemy.orm import DeclarativeBase
from sqlalchemy.pool import NullPool
from src.shared.config import get_settings
from src.shared.logging import get_logger
logger = get_logger(__name__)
settings = get_settings()
class Base(DeclarativeBase):
"""
SQLAlchemy declarative base for all models
All database models should inherit from this class.
"""
pass
class Database:
"""
Async database connection manager
Provides async engine and session factory for PostgreSQL connections.
"""
def __init__(self, database_url: Optional[str] = None):
"""Initialize database connection manager"""
url = database_url or settings.database_url
if url.startswith("postgresql://"):
url = url.replace("postgresql://", "postgresql+asyncpg://", 1)
self._url = url
self._engine: Optional[AsyncEngine] = None
self._session_factory: Optional[async_sessionmaker[AsyncSession]] = None
@property
def engine(self) -> AsyncEngine:
"""Get or create the async database engine"""
if self._engine is None:
self._engine = create_async_engine(
self._url,
echo=settings.debug,
poolclass=NullPool,
)
logger.info(f"Database engine created for {self._url.split('@')[-1]}")
return self._engine
@property
def session_factory(self) -> async_sessionmaker[AsyncSession]:
"""Get or create the async session factory"""
if self._session_factory is None:
self._session_factory = async_sessionmaker(
bind=self.engine,
class_=AsyncSession,
expire_on_commit=False,
autocommit=False,
autoflush=False,
)
return self._session_factory
async def create_tables(self) -> None:
"""Create all database tables (dev/testing only)"""
async with self.engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
logger.info("Database tables created")
async def drop_tables(self) -> None:
"""Drop all database tables (WARNING: destroys data)"""
async with self.engine.begin() as conn:
await conn.run_sync(Base.metadata.drop_all)
logger.warning("Database tables dropped")
async def health_check(self) -> bool:
"""Check if database connection is healthy"""
try:
async with self.session_factory() as session:
await session.execute(text("SELECT 1"))
return True
except Exception as e:
logger.error(f"Database health check failed: {e}")
return False
async def close(self) -> None:
"""Close database connections"""
if self._engine is not None:
await self._engine.dispose()
self._engine = None
self._session_factory = None
logger.info("Database connections closed")
# Singleton instance
_database: Optional[Database] = None
def get_database() -> Database:
"""Get singleton database instance"""
global _database
if _database is None:
_database = Database()
return _database
async def get_async_session() -> AsyncGenerator[AsyncSession, None]:
"""
FastAPI dependency for database sessions
Usage:
@router.get("/items")
async def get_items(session: AsyncSession = Depends(get_async_session)):
result = await session.execute(select(Item))
return result.scalars().all()
"""
database = get_database()
async with database.session_factory() as session:
try:
yield session
await session.commit()
except Exception:
await session.rollback()
raise
+37
View File
@@ -0,0 +1,37 @@
"""
Logging configuration for Core Code API
"""
import logging
import sys
from pathlib import Path
def setup_logging(log_level: str = "INFO") -> None:
"""
Configure logging for the application
Args:
log_level: Logging level (DEBUG, INFO, WARNING, ERROR, CRITICAL)
"""
log_dir = Path("logs")
log_dir.mkdir(exist_ok=True)
logging.basicConfig(
level=getattr(logging, log_level.upper()),
format="%(asctime)s | %(levelname)-8s | %(name)s:%(funcName)s:%(lineno)d | %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
handlers=[
logging.StreamHandler(sys.stdout),
logging.FileHandler(log_dir / "app.log", encoding="utf-8")
]
)
# Set specific log levels for third-party libraries
logging.getLogger("httpx").setLevel(logging.WARNING)
logging.getLogger("httpcore").setLevel(logging.WARNING)
logging.getLogger("uvicorn.access").setLevel(logging.WARNING)
def get_logger(name: str) -> logging.Logger:
"""Get a logger instance"""
return logging.getLogger(name)
+39
View File
@@ -0,0 +1,39 @@
"""
Security initialization module
Handles OIDC configuration and authentication setup
"""
from src.shared.config import Settings
from src.shared.logging import get_logger
logger = get_logger(__name__)
def initialize_oidc(settings: Settings) -> None:
"""
Initialize OIDC authentication configuration
Args:
settings: Application settings containing OIDC configuration
"""
# Import here to avoid circular imports
# Configure BOTH oidc modules (src.auth and src.domains.auth)
from src.auth.oidc import oidc_config as auth_oidc_config
from src.domains.auth.oidc import oidc_config as domains_oidc_config
auth_oidc_config.configure(
enabled=settings.oidc_enabled,
issuers=settings.oidc_issuers,
audiences=settings.oidc_audiences
)
domains_oidc_config.configure(
enabled=settings.oidc_enabled,
issuers=settings.oidc_issuers,
audiences=settings.oidc_audiences
)
if settings.oidc_enabled:
logger.info(f"OIDC authentication enabled (issuers: {settings.oidc_issuers})")
else:
logger.info("OIDC authentication disabled - API is publicly accessible")
-300
View File
@@ -1,300 +0,0 @@
"""Tests for Core-AI client."""
import pytest
from unittest.mock import patch, AsyncMock, MagicMock
import httpx
from src.clients.ai_client import CoreAIClient, get_ai_client
class TestCoreAIClientInit:
"""Test CoreAIClient initialization."""
@patch("src.clients.ai_client.settings")
def test_uses_settings_defaults(self, mock_settings):
"""Client should use settings for defaults."""
mock_settings.core_ai_base_url = "http://core-ai:8086"
client = CoreAIClient()
assert client.base_url == "http://core-ai:8086"
assert client.timeout == 10
def test_accepts_custom_url(self):
"""Client should accept custom URL."""
client = CoreAIClient(base_url="http://custom:9000")
assert client.base_url == "http://custom:9000"
def test_accepts_custom_timeout(self):
"""Client should accept custom timeout."""
client = CoreAIClient(base_url="http://test:8086", timeout=30)
assert client.timeout == 30
def test_strips_trailing_slash_from_url(self):
"""Client should strip trailing slash from URL."""
client = CoreAIClient(base_url="http://core-ai:8086/")
assert client.base_url == "http://core-ai:8086"
def test_creates_http_client(self):
"""Client should create httpx AsyncClient."""
client = CoreAIClient(base_url="http://test:8086")
assert client.client is not None
class TestCoreAIClientClose:
"""Test client close functionality."""
@pytest.mark.asyncio
async def test_close_closes_client(self):
"""close should close the HTTP client."""
client = CoreAIClient(base_url="http://test:8086")
with patch.object(client.client, "aclose", new_callable=AsyncMock) as mock_close:
await client.close()
mock_close.assert_called_once()
class TestCoreAIClientContextManager:
"""Test async context manager."""
@pytest.mark.asyncio
async def test_context_manager_enters(self):
"""Context manager should return client on enter."""
client = CoreAIClient(base_url="http://test:8086")
with patch.object(client.client, "aclose", new_callable=AsyncMock):
async with client as ctx:
assert ctx is client
@pytest.mark.asyncio
async def test_context_manager_closes_on_exit(self):
"""Context manager should close client on exit."""
client = CoreAIClient(base_url="http://test:8086")
with patch.object(client, "close", new_callable=AsyncMock) as mock_close:
async with client:
pass
mock_close.assert_called_once()
class TestCoreAIClientHealthCheck:
"""Test health check functionality."""
@pytest.mark.asyncio
async def test_health_check_returns_true_on_200(self):
"""Health check should return True when service responds 200."""
client = CoreAIClient(base_url="http://test:8086")
mock_response = MagicMock()
mock_response.status_code = 200
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.return_value = mock_response
result = await client.health_check()
assert result is True
@pytest.mark.asyncio
async def test_health_check_returns_false_on_error(self):
"""Health check should return False on connection error."""
client = CoreAIClient(base_url="http://test:8086")
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.side_effect = Exception("Connection refused")
result = await client.health_check()
assert result is False
@pytest.mark.asyncio
async def test_health_check_returns_false_on_non_200(self):
"""Health check should return False on non-200 status."""
client = CoreAIClient(base_url="http://test:8086")
mock_response = MagicMock()
mock_response.status_code = 500
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.return_value = mock_response
result = await client.health_check()
assert result is False
class TestCoreAIClientGetMetrics:
"""Test get metrics functionality."""
@pytest.mark.asyncio
async def test_get_metrics_returns_dict(self):
"""get_metrics should return metrics dict."""
client = CoreAIClient(base_url="http://test:8086")
metrics_data = {
"uptime_seconds": 3600,
"agent": {"total_requests": 100},
"tools": {"total_calls": 250}
}
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = metrics_data
mock_response.raise_for_status = MagicMock()
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.return_value = mock_response
result = await client.get_metrics()
assert result == metrics_data
assert result["uptime_seconds"] == 3600
@pytest.mark.asyncio
async def test_get_metrics_raises_on_http_error(self):
"""get_metrics should raise on HTTP error."""
client = CoreAIClient(base_url="http://test:8086")
mock_response = MagicMock()
mock_response.status_code = 500
mock_response.raise_for_status.side_effect = httpx.HTTPStatusError(
"Server Error", request=MagicMock(), response=mock_response
)
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.return_value = mock_response
with pytest.raises(httpx.HTTPStatusError):
await client.get_metrics()
class TestCoreAIClientGetRecentErrors:
"""Test get recent errors functionality."""
@pytest.mark.asyncio
async def test_get_recent_errors_returns_list(self):
"""get_recent_errors should return list of errors."""
client = CoreAIClient(base_url="http://test:8086")
errors_data = {
"errors": [
{"timestamp": "2025-12-03T19:45:12Z", "error": "Timeout"},
{"timestamp": "2025-12-03T19:46:00Z", "error": "Connection refused"}
]
}
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = errors_data
mock_response.raise_for_status = MagicMock()
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.return_value = mock_response
result = await client.get_recent_errors()
assert len(result) == 2
assert result[0]["error"] == "Timeout"
@pytest.mark.asyncio
async def test_get_recent_errors_passes_limit(self):
"""get_recent_errors should pass limit parameter."""
client = CoreAIClient(base_url="http://test:8086")
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {"errors": []}
mock_response.raise_for_status = MagicMock()
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.return_value = mock_response
await client.get_recent_errors(limit=5)
call_args = mock_get.call_args
assert call_args[1]["params"]["limit"] == 5
class TestCoreAIClientGetToolFailures:
"""Test get tool failures functionality."""
@pytest.mark.asyncio
async def test_get_tool_failures_returns_list(self):
"""get_tool_failures should return list of failures."""
client = CoreAIClient(base_url="http://test:8086")
failures_data = {
"failures": [
{"tool_name": "list_containers", "error": "Connection refused"}
]
}
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = failures_data
mock_response.raise_for_status = MagicMock()
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.return_value = mock_response
result = await client.get_tool_failures()
assert len(result) == 1
assert result[0]["tool_name"] == "list_containers"
@pytest.mark.asyncio
async def test_get_tool_failures_passes_limit(self):
"""get_tool_failures should pass limit parameter."""
client = CoreAIClient(base_url="http://test:8086")
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {"failures": []}
mock_response.raise_for_status = MagicMock()
with patch.object(client.client, "get", new_callable=AsyncMock) as mock_get:
mock_get.return_value = mock_response
await client.get_tool_failures(limit=10)
call_args = mock_get.call_args
assert call_args[1]["params"]["limit"] == 10
class TestCoreAIClientResetMetrics:
"""Test reset metrics functionality."""
@pytest.mark.asyncio
async def test_reset_metrics_returns_true_on_success(self):
"""reset_metrics should return True on success."""
client = CoreAIClient(base_url="http://test:8086")
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.raise_for_status = MagicMock()
with patch.object(client.client, "post", new_callable=AsyncMock) as mock_post:
mock_post.return_value = mock_response
result = await client.reset_metrics()
assert result is True
@pytest.mark.asyncio
async def test_reset_metrics_raises_on_error(self):
"""reset_metrics should raise on error."""
client = CoreAIClient(base_url="http://test:8086")
with patch.object(client.client, "post", new_callable=AsyncMock) as mock_post:
mock_post.side_effect = Exception("Connection refused")
with pytest.raises(Exception):
await client.reset_metrics()
class TestCoreAIClientSingleton:
"""Test singleton pattern."""
def test_get_ai_client_returns_same_instance(self):
"""get_ai_client should return singleton."""
import src.clients.ai_client as module
module._ai_client = None
client1 = get_ai_client()
client2 = get_ai_client()
assert client1 is client2
-264
View File
@@ -1,264 +0,0 @@
"""Tests for AI controller."""
import pytest
from fastapi.testclient import TestClient
from unittest.mock import patch, AsyncMock, MagicMock
from src.main import app
@pytest.fixture
def client():
"""Create a test client."""
return TestClient(app)
@pytest.fixture
def mock_ai_client():
"""Create a mock AI client."""
mock = AsyncMock()
return mock
class TestAIHealth:
"""Test /ai/health endpoint."""
@patch("src.controllers.ai_controller.get_ai_client")
def test_health_returns_200(self, mock_get_client, client):
"""AI health should return 200."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_get_client.return_value = mock_client
response = client.get("/ai/health")
assert response.status_code == 200
@patch("src.controllers.ai_controller.get_ai_client")
def test_health_returns_healthy_status(self, mock_get_client, client):
"""AI health should return healthy status when service is up."""
mock_client = AsyncMock()
mock_client.health_check.return_value = True
mock_get_client.return_value = mock_client
response = client.get("/ai/health")
data = response.json()
assert data["service"] == "core-ai"
assert data["status"] == "healthy"
assert data["accessible"] is True
@patch("src.controllers.ai_controller.get_ai_client")
def test_health_returns_unhealthy_status(self, mock_get_client, client):
"""AI health should return unhealthy status when service is down."""
mock_client = AsyncMock()
mock_client.health_check.return_value = False
mock_get_client.return_value = mock_client
response = client.get("/ai/health")
data = response.json()
assert data["status"] == "unhealthy"
assert data["accessible"] is False
@patch("src.controllers.ai_controller.get_ai_client")
def test_health_handles_exception(self, mock_get_client, client):
"""AI health should handle exceptions gracefully."""
mock_client = AsyncMock()
mock_client.health_check.side_effect = Exception("Connection refused")
mock_get_client.return_value = mock_client
response = client.get("/ai/health")
data = response.json()
assert data["status"] == "error"
assert data["accessible"] is False
assert "error" in data
class TestAIMetrics:
"""Test /ai/metrics endpoint."""
@patch("src.controllers.ai_controller.get_ai_client")
def test_metrics_returns_200(self, mock_get_client, client):
"""AI metrics should return 200."""
mock_client = AsyncMock()
mock_client.get_metrics.return_value = {
"uptime_seconds": 3600,
"agent": {"total_requests": 100}
}
mock_get_client.return_value = mock_client
response = client.get("/ai/metrics")
assert response.status_code == 200
@patch("src.controllers.ai_controller.get_ai_client")
def test_metrics_returns_data(self, mock_get_client, client):
"""AI metrics should return metrics data."""
metrics_data = {
"uptime_seconds": 3600,
"agent": {"total_requests": 100},
"tools": {"total_calls": 250}
}
mock_client = AsyncMock()
mock_client.get_metrics.return_value = metrics_data
mock_get_client.return_value = mock_client
response = client.get("/ai/metrics")
data = response.json()
assert data["uptime_seconds"] == 3600
assert data["agent"]["total_requests"] == 100
@patch("src.controllers.ai_controller.get_ai_client")
def test_metrics_returns_503_on_error(self, mock_get_client, client):
"""AI metrics should return 503 when service unavailable."""
mock_client = AsyncMock()
mock_client.get_metrics.side_effect = Exception("Service unavailable")
mock_get_client.return_value = mock_client
response = client.get("/ai/metrics")
assert response.status_code == 503
class TestAIErrors:
"""Test /ai/metrics/errors endpoint."""
@patch("src.controllers.ai_controller.get_ai_client")
def test_errors_returns_200(self, mock_get_client, client):
"""AI errors should return 200."""
mock_client = AsyncMock()
mock_client.get_recent_errors.return_value = []
mock_get_client.return_value = mock_client
response = client.get("/ai/metrics/errors")
assert response.status_code == 200
@patch("src.controllers.ai_controller.get_ai_client")
def test_errors_returns_error_list(self, mock_get_client, client):
"""AI errors should return list of errors."""
errors = [
{"timestamp": "2025-12-03T19:45:12Z", "error": "Timeout"},
{"timestamp": "2025-12-03T19:46:00Z", "error": "Connection refused"}
]
mock_client = AsyncMock()
mock_client.get_recent_errors.return_value = errors
mock_get_client.return_value = mock_client
response = client.get("/ai/metrics/errors")
data = response.json()
assert "errors" in data
assert "total" in data
assert data["total"] == 2
@patch("src.controllers.ai_controller.get_ai_client")
def test_errors_accepts_limit_parameter(self, mock_get_client, client):
"""AI errors should accept limit parameter."""
mock_client = AsyncMock()
mock_client.get_recent_errors.return_value = []
mock_get_client.return_value = mock_client
response = client.get("/ai/metrics/errors?limit=5")
assert response.status_code == 200
mock_client.get_recent_errors.assert_called_with(limit=5)
@patch("src.controllers.ai_controller.get_ai_client")
def test_errors_returns_503_on_error(self, mock_get_client, client):
"""AI errors should return 503 when service unavailable."""
mock_client = AsyncMock()
mock_client.get_recent_errors.side_effect = Exception("Service unavailable")
mock_get_client.return_value = mock_client
response = client.get("/ai/metrics/errors")
assert response.status_code == 503
class TestAIToolFailures:
"""Test /ai/metrics/tool-failures endpoint."""
@patch("src.controllers.ai_controller.get_ai_client")
def test_tool_failures_returns_200(self, mock_get_client, client):
"""Tool failures should return 200."""
mock_client = AsyncMock()
mock_client.get_tool_failures.return_value = []
mock_get_client.return_value = mock_client
response = client.get("/ai/metrics/tool-failures")
assert response.status_code == 200
@patch("src.controllers.ai_controller.get_ai_client")
def test_tool_failures_returns_failure_list(self, mock_get_client, client):
"""Tool failures should return list of failures."""
failures = [
{"tool_name": "list_containers", "error": "Connection refused"}
]
mock_client = AsyncMock()
mock_client.get_tool_failures.return_value = failures
mock_get_client.return_value = mock_client
response = client.get("/ai/metrics/tool-failures")
data = response.json()
assert "failures" in data
assert "total" in data
assert data["total"] == 1
@patch("src.controllers.ai_controller.get_ai_client")
def test_tool_failures_accepts_limit_parameter(self, mock_get_client, client):
"""Tool failures should accept limit parameter."""
mock_client = AsyncMock()
mock_client.get_tool_failures.return_value = []
mock_get_client.return_value = mock_client
response = client.get("/ai/metrics/tool-failures?limit=10")
assert response.status_code == 200
mock_client.get_tool_failures.assert_called_with(limit=10)
@patch("src.controllers.ai_controller.get_ai_client")
def test_tool_failures_returns_503_on_error(self, mock_get_client, client):
"""Tool failures should return 503 when service unavailable."""
mock_client = AsyncMock()
mock_client.get_tool_failures.side_effect = Exception("Service unavailable")
mock_get_client.return_value = mock_client
response = client.get("/ai/metrics/tool-failures")
assert response.status_code == 503
class TestAIMetricsReset:
"""Test /ai/metrics/reset endpoint."""
@patch("src.controllers.ai_controller.get_ai_client")
def test_reset_returns_200(self, mock_get_client, client):
"""Reset metrics should return 200."""
mock_client = AsyncMock()
mock_client.reset_metrics.return_value = True
mock_get_client.return_value = mock_client
response = client.post("/ai/metrics/reset")
assert response.status_code == 200
@patch("src.controllers.ai_controller.get_ai_client")
def test_reset_returns_success_message(self, mock_get_client, client):
"""Reset metrics should return success message."""
mock_client = AsyncMock()
mock_client.reset_metrics.return_value = True
mock_get_client.return_value = mock_client
response = client.post("/ai/metrics/reset")
data = response.json()
assert data["success"] is True
assert "message" in data
@patch("src.controllers.ai_controller.get_ai_client")
def test_reset_returns_503_on_error(self, mock_get_client, client):
"""Reset metrics should return 503 when service unavailable."""
mock_client = AsyncMock()
mock_client.reset_metrics.side_effect = Exception("Service unavailable")
mock_get_client.return_value = mock_client
response = client.post("/ai/metrics/reset")
assert response.status_code == 503
File diff suppressed because it is too large Load Diff
+627
View File
@@ -0,0 +1,627 @@
"""Tests for authentication service."""
import uuid
from datetime import datetime, timezone
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from sqlalchemy.ext.asyncio import AsyncSession
from src.domains.auth.models import User, Role, Group, UserPreferences
from src.domains.auth.schemas import TokenInfoSchema, RoleSchema
from src.domains.auth.service import AuthService, get_auth_service
# =============================================================================
# Fixtures
# =============================================================================
@pytest.fixture
def mock_session():
"""Create a mock async database session."""
session = AsyncMock(spec=AsyncSession)
session.execute = AsyncMock()
session.commit = AsyncMock()
session.flush = AsyncMock()
session.refresh = AsyncMock()
session.add = MagicMock()
return session
@pytest.fixture
def auth_service(mock_session):
"""Create an AuthService instance with mock session."""
return AuthService(mock_session)
@pytest.fixture
def sample_user():
"""Create a sample user for testing."""
user = User(
id=uuid.uuid4(),
authentik_id=uuid.uuid4(),
email="test@example.com",
name="Test User",
avatar_url="https://example.com/avatar.jpg",
created_at=datetime.now(timezone.utc),
last_login=datetime.now(timezone.utc),
)
user.roles = []
user.preferences = None
return user
@pytest.fixture
def sample_role():
"""Create a sample role for testing."""
return Role(
id=uuid.uuid4(),
name="control-room.general:admin",
domain="control-room",
category="general",
action="admin",
)
@pytest.fixture
def sample_group(sample_role):
"""Create a sample group for testing."""
group = Group(
id=uuid.uuid4(),
authentik_id=uuid.uuid4(),
name="Administrators",
is_superuser=True,
parent_name=None,
member_count=5,
created_at=datetime.now(timezone.utc),
synced_at=datetime.now(timezone.utc),
)
group.roles = [sample_role]
return group
@pytest.fixture
def sample_token_info():
"""Create sample token info from Authentik."""
return TokenInfoSchema(
sub=str(uuid.uuid4()),
email="test@example.com",
name="Test User",
preferred_username="testuser",
groups=["Administrators", "Developers"],
picture="https://example.com/avatar.jpg",
)
@pytest.fixture
def sample_preferences():
"""Create sample user preferences."""
return UserPreferences(
user_id=uuid.uuid4(),
theme="dark",
default_room="control-room",
preferences_json={"notifications": True},
)
# =============================================================================
# AuthService Initialization Tests
# =============================================================================
class TestAuthServiceInit:
"""Test AuthService initialization."""
def test_init_with_session(self, mock_session):
"""AuthService should initialize with session."""
service = AuthService(mock_session)
assert service.session is mock_session
def test_init_sets_userinfo_url(self, mock_session):
"""AuthService should set userinfo URL from settings."""
service = AuthService(mock_session)
assert "userinfo" in service.userinfo_url
def test_get_auth_service_factory(self, mock_session):
"""get_auth_service should return AuthService instance."""
service = get_auth_service(mock_session)
assert isinstance(service, AuthService)
assert service.session is mock_session
# =============================================================================
# Token Validation Tests
# =============================================================================
class TestValidateToken:
"""Test token validation via Authentik userinfo endpoint."""
@pytest.mark.asyncio
async def test_validate_token_success(self, auth_service):
"""validate_token should return token info on success."""
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {
"sub": str(uuid.uuid4()),
"email": "test@example.com",
"name": "Test User",
"preferred_username": "testuser",
"groups": ["Administrators"],
"picture": "https://example.com/avatar.jpg",
}
mock_response.raise_for_status = MagicMock()
with patch("src.domains.auth.service.httpx.AsyncClient") as mock_client:
mock_client.return_value.__aenter__.return_value.get = AsyncMock(
return_value=mock_response
)
result = await auth_service.validate_token("valid_token")
assert result.email == "test@example.com"
assert result.name == "Test User"
assert "Administrators" in result.groups
@pytest.mark.asyncio
async def test_validate_token_invalid(self, auth_service):
"""validate_token should raise ValueError for invalid token."""
mock_response = MagicMock()
mock_response.status_code = 401
with patch("src.domains.auth.service.httpx.AsyncClient") as mock_client:
mock_client.return_value.__aenter__.return_value.get = AsyncMock(
return_value=mock_response
)
with pytest.raises(ValueError, match="Invalid or expired token"):
await auth_service.validate_token("invalid_token")
@pytest.mark.asyncio
async def test_validate_token_service_unavailable(self, auth_service):
"""validate_token should raise ValueError when service unavailable."""
import httpx
with patch("src.domains.auth.service.httpx.AsyncClient") as mock_client:
mock_client.return_value.__aenter__.return_value.get = AsyncMock(
side_effect=httpx.RequestError("Connection failed")
)
with pytest.raises(ValueError, match="Authentication service unavailable"):
await auth_service.validate_token("token")
# =============================================================================
# User Sync Tests
# =============================================================================
class TestSyncUser:
"""Test user synchronization from OIDC token."""
@pytest.mark.asyncio
async def test_sync_user_creates_new_user(self, auth_service, sample_token_info, mock_session):
"""sync_user should create new user when not found."""
# Mock no existing user found
mock_result = MagicMock()
mock_result.scalar_one_or_none.return_value = None
mock_session.execute.return_value = mock_result
user, is_new = await auth_service.sync_user(sample_token_info)
assert is_new is True
assert mock_session.add.call_count == 2 # User and Preferences
@pytest.mark.asyncio
async def test_sync_user_updates_existing_user(
self, auth_service, sample_token_info, sample_user, mock_session
):
"""sync_user should update existing user when found."""
# Mock existing user found
mock_result = MagicMock()
mock_result.scalar_one_or_none.return_value = sample_user
mock_session.execute.return_value = mock_result
# Update token info with matching authentik_id
sample_token_info.sub = str(sample_user.authentik_id)
user, is_new = await auth_service.sync_user(sample_token_info)
assert is_new is False
assert user.email == sample_token_info.email
assert user.name == sample_token_info.name
@pytest.mark.asyncio
async def test_sync_user_updates_last_login(
self, auth_service, sample_token_info, sample_user, mock_session
):
"""sync_user should update last_login timestamp."""
old_login = sample_user.last_login
mock_result = MagicMock()
mock_result.scalar_one_or_none.return_value = sample_user
mock_session.execute.return_value = mock_result
sample_token_info.sub = str(sample_user.authentik_id)
user, _ = await auth_service.sync_user(sample_token_info)
assert user.last_login is not None
# last_login should be updated (or same if happened in same second)
assert user.last_login >= old_login or user.last_login is not None
# =============================================================================
# Role Sync Tests
# =============================================================================
class TestSyncRoles:
"""Test role synchronization from Authentik groups via group_roles."""
@pytest.mark.asyncio
async def test_sync_roles_from_groups(
self, auth_service, sample_user, sample_group, mock_session
):
"""sync_roles should get roles from matching groups."""
# Mock finding groups with roles
mock_result = MagicMock()
mock_result.scalars.return_value.all.return_value = [sample_group]
mock_session.execute.return_value = mock_result
roles = await auth_service.sync_roles(sample_user, ["Administrators"])
assert len(roles) == 1
assert roles[0].name == "control-room.general:admin"
assert sample_user.roles == roles
@pytest.mark.asyncio
async def test_sync_roles_no_matching_groups(
self, auth_service, sample_user, mock_session
):
"""sync_roles should return empty list when no groups match."""
mock_result = MagicMock()
mock_result.scalars.return_value.all.return_value = []
mock_session.execute.return_value = mock_result
roles = await auth_service.sync_roles(sample_user, ["NonExistentGroup"])
assert len(roles) == 0
assert sample_user.roles == []
@pytest.mark.asyncio
async def test_sync_roles_deduplicates_roles(
self, auth_service, sample_user, sample_role, mock_session
):
"""sync_roles should deduplicate roles from multiple groups."""
# Create two groups with the same role
group1 = Group(
id=uuid.uuid4(),
authentik_id=uuid.uuid4(),
name="Group1",
is_superuser=False,
member_count=1,
created_at=datetime.now(timezone.utc),
synced_at=datetime.now(timezone.utc),
)
group1.roles = [sample_role]
group2 = Group(
id=uuid.uuid4(),
authentik_id=uuid.uuid4(),
name="Group2",
is_superuser=False,
member_count=1,
created_at=datetime.now(timezone.utc),
synced_at=datetime.now(timezone.utc),
)
group2.roles = [sample_role] # Same role
mock_result = MagicMock()
mock_result.scalars.return_value.all.return_value = [group1, group2]
mock_session.execute.return_value = mock_result
roles = await auth_service.sync_roles(sample_user, ["Group1", "Group2"])
# Should only have one role despite appearing in two groups
assert len(roles) == 1
# =============================================================================
# Schema Conversion Tests
# =============================================================================
class TestSchemaConversions:
"""Test model to schema conversions."""
def test_user_to_schema(self, auth_service, sample_user):
"""user_to_schema should convert User model to UserSchema."""
schema = auth_service.user_to_schema(sample_user)
assert schema.id == sample_user.id
assert schema.authentik_id == sample_user.authentik_id
assert schema.email == sample_user.email
assert schema.name == sample_user.name
assert schema.avatar_url == sample_user.avatar_url
def test_roles_to_schema(self, auth_service, sample_role):
"""roles_to_schema should convert Role models to RoleSchemas."""
schemas = auth_service.roles_to_schema([sample_role])
assert len(schemas) == 1
assert schemas[0].id == sample_role.id
assert schemas[0].name == sample_role.name
assert schemas[0].domain == sample_role.domain
assert schemas[0].category == sample_role.category
assert schemas[0].action == sample_role.action
def test_roles_to_schema_empty_list(self, auth_service):
"""roles_to_schema should handle empty list."""
schemas = auth_service.roles_to_schema([])
assert schemas == []
def test_preferences_to_schema(self, auth_service, sample_preferences):
"""preferences_to_schema should convert UserPreferences to schema."""
schema = auth_service.preferences_to_schema(sample_preferences)
assert schema.theme == sample_preferences.theme
assert schema.default_room == sample_preferences.default_room
assert schema.preferences_json == sample_preferences.preferences_json
def test_preferences_to_schema_none(self, auth_service):
"""preferences_to_schema should return defaults for None."""
schema = auth_service.preferences_to_schema(None)
assert schema.theme == "system"
assert schema.default_room == "front-hall"
assert schema.preferences_json == {}
# =============================================================================
# List Operations Tests
# =============================================================================
class TestListOperations:
"""Test list operations for users, groups, and roles."""
@pytest.mark.asyncio
async def test_list_users(self, auth_service, sample_user, mock_session):
"""list_users should return paginated user list."""
sample_user.roles = []
# Mock count query
count_result = MagicMock()
count_result.scalar.return_value = 1
# Mock users query
users_result = MagicMock()
users_result.scalars.return_value.all.return_value = [sample_user]
mock_session.execute.side_effect = [count_result, users_result]
items, total = await auth_service.list_users()
assert total == 1
assert len(items) == 1
assert items[0].email == sample_user.email
@pytest.mark.asyncio
async def test_list_users_with_search(self, auth_service, mock_session):
"""list_users should filter by search query."""
count_result = MagicMock()
count_result.scalar.return_value = 0
users_result = MagicMock()
users_result.scalars.return_value.all.return_value = []
mock_session.execute.side_effect = [count_result, users_result]
items, total = await auth_service.list_users(search="nonexistent")
assert total == 0
assert len(items) == 0
@pytest.mark.asyncio
async def test_list_groups(self, auth_service, sample_group, mock_session):
"""list_groups should return paginated group list with roles."""
count_result = MagicMock()
count_result.scalar.return_value = 1
groups_result = MagicMock()
groups_result.scalars.return_value.all.return_value = [sample_group]
mock_session.execute.side_effect = [count_result, groups_result]
items, total = await auth_service.list_groups()
assert total == 1
assert len(items) == 1
assert items[0].name == sample_group.name
assert len(items[0].roles) == 1 # Should include role names
@pytest.mark.asyncio
async def test_list_roles(self, auth_service, sample_role, mock_session):
"""list_roles should return all roles ordered by domain."""
mock_result = MagicMock()
mock_result.scalars.return_value.all.return_value = [sample_role]
mock_session.execute.return_value = mock_result
roles = await auth_service.list_roles()
assert len(roles) == 1
assert roles[0].name == sample_role.name
# =============================================================================
# Group-Role Management Tests
# =============================================================================
class TestGroupRoleManagement:
"""Test group-role assignment and removal."""
@pytest.mark.asyncio
async def test_get_group_by_id(self, auth_service, sample_group, mock_session):
"""get_group_by_id should return group with roles."""
mock_result = MagicMock()
mock_result.scalar_one_or_none.return_value = sample_group
mock_session.execute.return_value = mock_result
group = await auth_service.get_group_by_id(sample_group.id)
assert group is not None
assert group.id == sample_group.id
assert len(group.roles) == 1
@pytest.mark.asyncio
async def test_get_group_by_id_not_found(self, auth_service, mock_session):
"""get_group_by_id should return None when not found."""
mock_result = MagicMock()
mock_result.scalar_one_or_none.return_value = None
mock_session.execute.return_value = mock_result
group = await auth_service.get_group_by_id(uuid.uuid4())
assert group is None
@pytest.mark.asyncio
async def test_assign_role_to_group(
self, auth_service, sample_group, sample_role, mock_session
):
"""assign_role_to_group should add role to group."""
# Clear existing roles for this test
sample_group.roles = []
# Mock group lookup
group_result = MagicMock()
group_result.scalar_one_or_none.return_value = sample_group
# Mock role lookup
role_result = MagicMock()
role_result.scalar_one_or_none.return_value = sample_role
mock_session.execute.side_effect = [group_result, role_result]
group = await auth_service.assign_role_to_group(sample_group.id, sample_role.id)
assert sample_role in group.roles
mock_session.flush.assert_called()
@pytest.mark.asyncio
async def test_assign_role_to_group_already_assigned(
self, auth_service, sample_group, sample_role, mock_session
):
"""assign_role_to_group should not duplicate if already assigned."""
# Group already has this role
sample_group.roles = [sample_role]
original_count = len(sample_group.roles)
group_result = MagicMock()
group_result.scalar_one_or_none.return_value = sample_group
role_result = MagicMock()
role_result.scalar_one_or_none.return_value = sample_role
mock_session.execute.side_effect = [group_result, role_result]
group = await auth_service.assign_role_to_group(sample_group.id, sample_role.id)
assert len(group.roles) == original_count # No duplicate
@pytest.mark.asyncio
async def test_assign_role_to_group_group_not_found(self, auth_service, mock_session):
"""assign_role_to_group should raise ValueError when group not found."""
mock_result = MagicMock()
mock_result.scalar_one_or_none.return_value = None
mock_session.execute.return_value = mock_result
with pytest.raises(ValueError, match="Group not found"):
await auth_service.assign_role_to_group(uuid.uuid4(), uuid.uuid4())
@pytest.mark.asyncio
async def test_assign_role_to_group_role_not_found(
self, auth_service, sample_group, mock_session
):
"""assign_role_to_group should raise ValueError when role not found."""
sample_group.roles = []
group_result = MagicMock()
group_result.scalar_one_or_none.return_value = sample_group
role_result = MagicMock()
role_result.scalar_one_or_none.return_value = None
mock_session.execute.side_effect = [group_result, role_result]
with pytest.raises(ValueError, match="Role not found"):
await auth_service.assign_role_to_group(sample_group.id, uuid.uuid4())
@pytest.mark.asyncio
async def test_remove_role_from_group(
self, auth_service, sample_group, sample_role, mock_session
):
"""remove_role_from_group should remove role from group."""
# Group has this role
sample_group.roles = [sample_role]
group_result = MagicMock()
group_result.scalar_one_or_none.return_value = sample_group
role_result = MagicMock()
role_result.scalar_one_or_none.return_value = sample_role
mock_session.execute.side_effect = [group_result, role_result]
group = await auth_service.remove_role_from_group(sample_group.id, sample_role.id)
assert sample_role not in group.roles
mock_session.flush.assert_called()
@pytest.mark.asyncio
async def test_remove_role_from_group_not_assigned(
self, auth_service, sample_group, sample_role, mock_session
):
"""remove_role_from_group should handle role not assigned gracefully."""
# Group does not have this role
sample_group.roles = []
group_result = MagicMock()
group_result.scalar_one_or_none.return_value = sample_group
role_result = MagicMock()
role_result.scalar_one_or_none.return_value = sample_role
mock_session.execute.side_effect = [group_result, role_result]
group = await auth_service.remove_role_from_group(sample_group.id, sample_role.id)
# Should complete without error
assert len(group.roles) == 0
# =============================================================================
# Role Schema Tests
# =============================================================================
class TestRoleSchema:
"""Test RoleSchema validation."""
def test_role_schema_creation(self):
"""RoleSchema should be creatable with valid data."""
schema = RoleSchema(
id=uuid.uuid4(),
name="control-room.general:admin",
domain="control-room",
category="general",
action="admin",
)
assert schema.name == "control-room.general:admin"
assert schema.domain == "control-room"
assert schema.category == "general"
assert schema.action == "admin"
def test_role_schema_category_default(self):
"""RoleSchema should default category to 'general'."""
schema = RoleSchema(
id=uuid.uuid4(),
name="media.general:viewer",
domain="media",
action="viewer",
)
assert schema.category == "general"
+4 -8
View File
@@ -1,6 +1,6 @@
"""Tests for config module.""" """Tests for config module."""
import pytest import pytest
from src.config import ( from src.shared.config import (
__version__, __version__,
Settings, Settings,
get_settings, get_settings,
@@ -89,10 +89,6 @@ class TestGetSettings:
settings2 = get_settings() settings2 = get_settings()
assert settings1 is settings2 assert settings1 is settings2
def test_model_aliases_property(self): # Removed: test_model_aliases_property. Settings.model_aliases mapped
"""Model aliases property should return dict.""" # gpt-3.5-turbo and gpt-4 onto local models, and was deleted along with the
settings = get_settings() # Ollama integration in c1f16d4. The test outlived the feature it covered.
aliases = settings.model_aliases
assert isinstance(aliases, dict)
assert "gpt-3.5-turbo" in aliases
assert "gpt-4" in aliases
+199
View File
@@ -0,0 +1,199 @@
"""Tests for dashboard endpoints registration and OpenAPI spec."""
import pytest
from fastapi.testclient import TestClient
from src.main import app
@pytest.fixture
def client():
"""Create a test client."""
return TestClient(app)
class TestDashboardOpenAPISpec:
"""Test that dashboard endpoints are documented in OpenAPI spec."""
def test_quick_links_list_in_openapi(self, client):
"""Quick links list endpoint should be in OpenAPI spec."""
response = client.get("/openapi.json")
assert response.status_code == 200
spec = response.json()
assert "/dashboard/quick-links" in spec["paths"]
def test_quick_links_get_in_openapi(self, client):
"""Quick links get endpoint should be in OpenAPI spec."""
response = client.get("/openapi.json")
assert response.status_code == 200
spec = response.json()
assert "/dashboard/quick-links/{link_id}" in spec["paths"]
def test_quick_links_reorder_in_openapi(self, client):
"""Quick links reorder endpoint should be in OpenAPI spec."""
response = client.get("/openapi.json")
assert response.status_code == 200
spec = response.json()
assert "/dashboard/quick-links/reorder" in spec["paths"]
def test_widgets_list_in_openapi(self, client):
"""Widgets list endpoint should be in OpenAPI spec."""
response = client.get("/openapi.json")
assert response.status_code == 200
spec = response.json()
assert "/dashboard/widgets" in spec["paths"]
def test_widgets_get_in_openapi(self, client):
"""Widgets get endpoint should be in OpenAPI spec."""
response = client.get("/openapi.json")
assert response.status_code == 200
spec = response.json()
assert "/dashboard/widgets/{widget_id}" in spec["paths"]
def test_quick_links_supports_crud_operations(self, client):
"""Quick links should support all CRUD operations."""
response = client.get("/openapi.json")
spec = response.json()
# List endpoint
list_path = spec["paths"].get("/dashboard/quick-links", {})
assert "get" in list_path # List
assert "post" in list_path # Create
# Item endpoint
item_path = spec["paths"].get("/dashboard/quick-links/{link_id}", {})
assert "get" in item_path # Read
assert "put" in item_path # Update
assert "delete" in item_path # Delete
def test_widgets_supports_crud_operations(self, client):
"""Widgets should support all CRUD operations."""
response = client.get("/openapi.json")
spec = response.json()
# List endpoint
list_path = spec["paths"].get("/dashboard/widgets", {})
assert "get" in list_path # List
assert "post" in list_path # Create
# Item endpoint
item_path = spec["paths"].get("/dashboard/widgets/{widget_id}", {})
assert "get" in item_path # Read
assert "put" in item_path # Update
assert "delete" in item_path # Delete
class TestDashboardSchemaValidation:
"""Test that request validation works correctly."""
def test_create_quick_link_requires_title(self, client):
"""Create quick link should require title (422 for validation)."""
response = client.post(
"/dashboard/quick-links",
json={
"url": "https://example.com",
},
)
# Either 422 for validation or 401/403/500 for auth
assert response.status_code in [401, 403, 422, 500]
def test_create_widget_requires_widget_type(self, client):
"""Create widget should require widget_type (422 for validation)."""
response = client.post(
"/dashboard/widgets",
json={},
)
assert response.status_code in [401, 403, 422, 500]
def test_reorder_requires_link_ids(self, client):
"""Reorder should require link_ids list (422 for validation)."""
response = client.post(
"/dashboard/quick-links/reorder",
json={},
)
assert response.status_code in [401, 403, 422, 500]
class TestDashboardControllerInit:
"""Test dashboard controller initialization."""
def test_controller_module_imports(self):
"""Dashboard controller should be importable."""
from src.domains.dashboard.controller import DashboardController, dashboard_controller
assert DashboardController is not None
assert dashboard_controller is not None
def test_controller_has_correct_prefix(self):
"""Dashboard controller should have correct prefix."""
from src.domains.dashboard.controller import dashboard_controller
assert dashboard_controller.prefix == "/dashboard"
def test_controller_has_correct_tags(self):
"""Dashboard controller should have correct tags."""
from src.domains.dashboard.controller import dashboard_controller
assert "Dashboard" in dashboard_controller.tags
class TestDashboardServiceInit:
"""Test dashboard service initialization."""
def test_service_module_imports(self):
"""Dashboard service should be importable."""
from src.domains.dashboard.service import DashboardService, get_dashboard_service
assert DashboardService is not None
assert get_dashboard_service is not None
def test_service_singleton(self):
"""get_dashboard_service should return singleton."""
from src.domains.dashboard.service import get_dashboard_service
service1 = get_dashboard_service()
service2 = get_dashboard_service()
assert service1 is service2
class TestDashboardModels:
"""Test dashboard models."""
def test_quick_link_model_imports(self):
"""QuickLink model should be importable."""
from src.domains.dashboard.models import QuickLink
assert QuickLink is not None
def test_dashboard_widget_model_imports(self):
"""DashboardWidget model should be importable."""
from src.domains.dashboard.models import DashboardWidget
assert DashboardWidget is not None
class TestDashboardSchemas:
"""Test dashboard schemas."""
def test_quick_link_schemas_import(self):
"""QuickLink schemas should be importable."""
from src.domains.dashboard.schemas import (
QuickLinkCreate,
QuickLinkUpdate,
QuickLinkResponse,
QuickLinkListResponse,
QuickLinkReorderRequest,
QuickLinkReorderResponse,
)
assert QuickLinkCreate is not None
assert QuickLinkUpdate is not None
assert QuickLinkResponse is not None
assert QuickLinkListResponse is not None
assert QuickLinkReorderRequest is not None
assert QuickLinkReorderResponse is not None
def test_dashboard_widget_schemas_import(self):
"""DashboardWidget schemas should be importable."""
from src.domains.dashboard.schemas import (
DashboardWidgetCreate,
DashboardWidgetUpdate,
DashboardWidgetResponse,
DashboardWidgetListResponse,
)
assert DashboardWidgetCreate is not None
assert DashboardWidgetUpdate is not None
assert DashboardWidgetResponse is not None
assert DashboardWidgetListResponse is not None
+3 -3
View File
@@ -4,9 +4,9 @@ from unittest.mock import patch, MagicMock
import dns.resolver import dns.resolver
import dns.exception import dns.exception
from src.dns.service import DNSService from src.domains.tools.dns.service import DNSService
from src.dns.schemas import DNSLookupRequest, DNSRecord from src.domains.tools.dns.schemas import DNSLookupRequest, DNSRecord
from src.dns.exceptions import DNSQueryError from src.domains.tools.dns.exceptions import DNSQueryError
@pytest.fixture @pytest.fixture

Some files were not shown because too many files have changed in this diff Show More