tatlock/CLAUDE.md

# CLAUDE.md — tatlock

Privacy-first homelab butler. An OpenAI-compatible orchestration API over local models, with
household staff agents built on PydanticAI. Python 3.12 / FastAPI, `version = "2.4.3"`.
Container `tatlock` on `docker-dataplane`, port **8000**. Redis DB **1** (memory), Qdrant for
vectors.

## Ports

| | Port | How |
|---|---|---|
| Local dev | **8777** | `make run` — uvicorn reload, logs to `build/logs/server.log` |
| Production | **8000** | container; `http://192.168.86.149:8000/health`, external `tatlock.schweitz.net` behind Authentik |

Test endpoints against `localhost:8777` while developing. `localhost:8000` is the *container*.

## Live contract

`http://localhost:8000/openapi.json` — **5 paths**, `title: OpenAI-Compatible API`, `version:
2.4.3` (verified 2026-08-09): `/`, `/health`, `/v1/models`, `/v1/chat/completions`,
`/v1/responses`. `/v1/responses` is primary; `/v1/chat/completions` exists for Open WebUI.

**The spec is the public surface, not the system.** The household capability registry is internal
and appears nowhere in those 5 paths. Absence from the spec means "not exposed", not "does not
exist".

## Two traps that make the runtime look like the opposite of what it is

**1. `src/anthropic` loads at startup; `src/ollama` does not — and Ollama is the primary
backend.** A cold `import src.main` inside the container shows `agents, anthropic, chat, core,
main, models, responses` — no `ollama`. The only import of it is a *function-body* one at
`src/anthropic/model_selector.py:230`. Meanwhile `PREFER_CLOUD_BACKEND=false`, so every request
actually goes to Ollama and the Claude path is off (see **workspace D-11**). Reading the module list
naively gives you exactly the wrong answer: the package that looks live is the disabled fallback,
and the one that looks dead is the hot path. Do not conclude anything about backends from
`sys.modules`; read the config.

**2. In-process singletons are empty outside the app.** `get_household_registry()`
(`src/core/household_registry.py:334`) in a fresh `docker exec python` returns **0 members**,
while the running app serves 2 models from it — it is populated at startup. Import the
module-level definitions or ask the endpoint; never import a singleton and assume it is
populated.

## Stack decisions that bind this repo

Recorded in the workspace vault, not here. Read before assuming anything about the LLM backend:

```bash
/home/jpmschweitzer/.local/bin/pql --vault /mnt/media/Projects decisions read workspace D-11
```

**workspace D-11 — the Claude migration is abandoned. Tatlock stays on Ollama.** Do not resume it and do
not treat its remnants as unfinished work. What you will find, and why none of it is a TODO:
`ANTHROPIC_MODEL` is set on the container (`claude-sonnet-4-20250514`) and never used because
`PREFER_CLOUD_BACKEND=false`; `ANTHROPIC_API_KEY` is a variable reference whose literal was
revoked 2026-08-09; `docs/claude-integration.md` documents a capability that exists but is
switched off. The cost is deliberate: reasoning stays at `gemma4:e2b` scale because VRAM is
shared with Speaches.

`REDIS_BENCHMARK_DB=6` is allocated on the container but the benchmarking module was never
implemented — see the gotcha below. Vestigial, like the Anthropic settings.

## Critical gotchas

**ASGITransport does NOT trigger FastAPI lifespan events.** The session-scoped `_initialize_app`
fixture in `tests/conftest.py` calls `initialize_application()` explicitly via `asyncio.run()`.
Without it the Ollama/Claude health checks never run: `_ollama_available` stays `None` (treated
as available, so requests go to Ollama) and `_claude_available` stays `None` (treated as
unavailable, so the Claude fallback never engages).

**AsyncIO scope mismatch.** `asyncio_default_fixture_loop_scope = function` is set in
`pyproject.toml`. Session-scoped async fixtures raise `ScopeMismatch`. Use a sync fixture with
`asyncio.run()` for session-scoped initialization.

**The butler persona prompt suppresses local-model tool calling.** With `TATLOCK_SYSTEM_PROMPT`
attached, gemma4 reasons about calling the calculator, then answers from memory with wrong
arithmetic — a different wrong product each run. `orchestrate_tool_calls()` therefore uses the
terse `TATLOCK_ORCHESTRATION_PROMPT`; the persona is applied in `synthesize_from_results()`. Do
not reattach the persona prompt to a tool-phase agent. `tool_choice: "required"` via `extra_body`
does **not** force Ollama to call tools — advisory at best.

**Claude Sonnet 5+ rejects sampling parameters.** `temperature`/`top_p`/`top_k` return 400. Use
`get_sampling_settings()` from the model selector rather than passing `ModelSettings(temperature=…)`
to agents that can run on the Claude fallback. `make test-contracts` pins this.

**Integration test timeouts** are 120s to match `OLLAMA_TIMEOUT` (300s for the pure-Ollama
fallback test, which Claude cannot rescue). GPU-resident numbers measured 2026-08-07 with
gemma4:e2b at ~95 tok/s: full Steward → orchestrate → synthesize ~10–13s for simple turns;
librarian-routed ~20–25s (not re-measured). **A single turn costs 3 sequential Ollama calls and
~710 generated tokens even for "what is 61 plus 12?"** — mostly the model's own reasoning, paid
three times. Cold model load is ~36s, avoided while pinned with `keep_alive: -1`; the
`OLLAMA_KEEP_ALIVE=2h` default reintroduces it. Older "~35s steward / ~2 min flow" and "11–25s"
figures are superseded — do not plan against them. `STEWARD_TIMEOUT` defaults to 60s.

**`get_benchmark_store` does not exist.** `src/core/benchmarks.py` was never implemented, and
`scripts/benchmark_analysis.py` references it and is broken. Do not add mocks for it in tests.

**Steward tests need the household registry.** Use `register_household_members()` (sync) in
fixtures, not `initialize_application()` (async). The steward extracts capabilities from the
registry.

## Commands

```bash
make setup            # venv + all dependencies
make run              # dev server on 8777, reload, logs to build/logs/server.log
make test             # unit tests, no external services
make test-integration # needs Ollama (and Claude, if enabled)
make test-contracts   # wire-level contract tests against live service boundaries
make lint             # ruff linter + formatter check
make typecheck        # mypy
make clean            # remove caches and build artifacts
```

Always run pytest through the venv explicitly, to avoid environment mismatch:

```bash
.venv/bin/python -m pytest tests/
.venv/bin/python -m pytest tests/core/ -v
```

Dependencies live in `pyproject.toml` (`[project.dependencies]`, `[project.optional-dependencies.dev]`).
Copy `.env.example` to `.env` and configure Ollama, Redis and Qdrant hosts.

**Contract tests before code review.** When the question is "do these two services still agree?",
`make test-contracts` answers it by observing the live boundary; reading both codebases only tells
you what should happen. Semantics: unreachable → skip, reachable-but-wrong-shape → fail.

## Architecture

Domain-first under `src/`: `agents/` (steward, librarian, biographer, housekeeper, tatlock_core),
`core/`, `chat/`, `responses/`, `models/`, `ollama/`, `anthropic/`. Two tiers — the Steward routes,
Tatlock coordinates. Group new work by domain, not by file type.

## Internal service access

`http://localhost:3002` reaches Gitea directly, bypassing Authentik SSO — verified returning
`{"version":"1.27.1"}`. Useful for reading a sibling repo's raw files:

```bash
curl http://localhost:3002/jpmschweitzer/library-desk/raw/branch/main/README.md
```

The old AGENTS.md pointed at **`portainer-core`** for full-stack documentation. That repo is
**deprecated** and must not be used as a source of infra facts; it was merged into
`system-admin-toj/containers/`, where `CONTAINERS.md` is the live inventory.

## Work tracking

Work lives in **pql**, not a markdown TODO. **This repo's vault is standalone** — its tickets and
internal decisions live here in `.pql/` and `governance/`, and travel with a clone, because
`.pql/changelog/` is committed and replayed by the git hooks (workspace D-15). The databases are gitignored
and rebuildable with `pql plan rebuild`.

`pql` is **not** on the non-interactive `PATH` — invoke it as `/home/jpmschweitzer/.local/bin/pql`.
From inside this repo no `--vault` is needed; pql anchors at the nearest `.git/` ancestor.

```bash
/home/jpmschweitzer/.local/bin/pql ticket list        # this repo's open work
/home/jpmschweitzer/.local/bin/pql plan whatsnext     # next unblocked item, with context
/home/jpmschweitzer/.local/bin/pql decisions list     # this repo's own decisions
```

Stack decisions that constrain this service need the flag:

```bash
/home/jpmschweitzer/.local/bin/pql --vault /mnt/media/Projects decisions list --domain tatlock-api
```

The workspace domain is `tatlock-api`, not `tatlock` — pql rejects a domain stem that prefixes
another, and `tatlock` prefixes `tatlock-ui`. A `tatlock-api -> tatlock` symlink at the workspace
root makes the directory answer to both (workspace D-15).

Note `ticket new --decision D-N` resolves ids within **one** vault, so a ticket here cannot link
to a workspace decision. Cite the id in the ticket body instead.

## Git

- **History is linear — no merge commits.** Work on `main`, or a short-lived branch that is
  fast-forwarded and deleted. This repo's AGENTS.md mandated a feature branch for every change;
  that rule was retired workspace-wide on 2026-08-08 and does not apply.
- **Conventional Commits**: `feat:`, `fix:`, `refactor:`, `docs:`, `chore:`.
- **Stage explicitly. Never `git add -A`** — denied by policy, and it sweeps in whatever else is
  dirty, including secrets.
- Update `CHANGELOG.md` for every user-facing change, under `[Unreleased]`.

## Releasing

Test locally first — the build-deploy loop is slow. Deploy only when a feature is complete.

1. Ask whether a deploy is wanted; it is not automatic.
2. Bump `version` in `pyproject.toml` (patch for fixes, minor for features).
3. Move `[Unreleased]` entries into a dated section in `CHANGELOG.md`.
4. Stage the changed files by name, commit, tag `vX.Y.Z`, `git push origin main --tags`.
5. Gitea CI builds and pushes on the tag; Watchtower deploys.
6. Verify: `curl http://192.168.86.149:8000/health`.