From 8eec3b68d60e050d95abe71bed857481e0ddb5c2 Mon Sep 17 00:00:00 2001 From: Jeroen Schweitzer Date: Sun, 9 Aug 2026 03:14:50 +0200 Subject: [PATCH] docs: replace both AGENTS.md files with one CLAUDE.md One agent doc per repo, and it is CLAUDE.md. This repo carried two -- one at the root and one under webber-api/ -- which is the drift problem in its purest form: two documents, one subject, and no way to know which the last reader trusted. Written fresh rather than reformatted. README.md linked to webber-api/AGENTS.md, so that pointer moves with the file rather than dangling. The architecture section states the method used to establish what is live -- import the app inside the container and read sys.modules -- and then the case where that method fails here. src/domains/tools is absent from a cold snapshot and is entirely live: each agent's _register_tools imports its tool package from inside the method body, on every /agents/run. Absence from a snapshot taken before any request is served is a timing artifact, not evidence of death, and deleting on that basis would have removed the tool layer. Co-Authored-By: Claude --- AGENTS.md | 289 ------------------------------------------- CLAUDE.md | 182 +++++++++++++++++++++++++++ README.md | 2 +- webber-api/AGENTS.md | 128 ------------------- 4 files changed, 183 insertions(+), 418 deletions(-) delete mode 100644 AGENTS.md create mode 100644 CLAUDE.md delete mode 100644 webber-api/AGENTS.md diff --git a/AGENTS.md b/AGENTS.md deleted file mode 100644 index d446b1d..0000000 --- a/AGENTS.md +++ /dev/null @@ -1,289 +0,0 @@ -# Webber Monorepo - Agent Instructions - -> **Start every session by reading this file.** -> This file contains everything you need to work with this codebase efficiently. - -## Quick Reference - -| Action | Command | -|--------|---------| -| Start API server | `cd webber-api && ./wakeup.sh` | -| View API logs | `tail -f webber-api/logs/server.log` | -| Run API tests | `cd webber-api && .venv/bin/python -m pytest tests/ -v` | -| Check CLI status | `cd webber-cli && .venv/bin/webber-cli status` | -| Load sandbox | `./sandbox.sh load calculator-cli` | -| Explore sandbox | `cd webber-cli && .venv/bin/webber-cli explore "query" -d ../webber-sandbox` | - ---- - -## Repository Structure - -``` -webber/ -├── webber-api/ # FastAPI backend server -│ ├── src/ # API source code -│ ├── tests/ # API tests (pytest) -│ ├── docs/ # Architecture docs, COVERAGE.md -│ ├── logs/ # Runtime logs (server.log) -│ ├── .venv/ # API virtual environment -│ ├── wakeup.sh # Dev server startup script -│ └── AGENTS.md # API-specific development guide -│ -├── webber-cli/ # CLI client -│ ├── webber_cli/ # Python package (underscore!) -│ ├── .venv/ # CLI virtual environment -│ └── README.md # CLI usage guide -│ -├── webber-sandbox/ # Active test project (contents swappable) -│ ├── src/ # Current project source -│ ├── tests/ # Current project tests -│ ├── .venv/ # Sandbox virtual environment -│ └── TASKS.md # Tasks for Webber to complete -│ -├── sandbox-templates/ # Template storage -│ ├── calculator-cli/ # Simple CLI with intentional bugs -│ └── empty/ # Blank starter project -│ -├── sandbox.sh # Sandbox management script -└── AGENTS.md # THIS FILE -``` - ---- - -## Development Workflow - -### 1. Start the API Server - -```bash -cd webber-api -./wakeup.sh -``` - -- **Port:** 8095 (dev), 8086 (production Docker) -- **Logs:** `webber-api/logs/server.log` -- **Health check:** `curl http://localhost:8095/health` -- **API docs:** http://localhost:8095/docs - -To stop: `Ctrl+C` or `pkill -f "uvicorn src.main:app"` - -### 2. Run Tests - -```bash -# API tests (39 tests) -cd webber-api -.venv/bin/python -m pytest tests/ -v - -# With coverage -.venv/bin/python -m pytest tests/ --cov=src - -# Single test file -.venv/bin/python -m pytest tests/test_tools.py -v -``` - -### 3. Use the CLI - -```bash -cd webber-cli - -# Check API connection -.venv/bin/webber-cli status - -# Interactive chat (default mode - full capabilities) -.venv/bin/webber-cli chat -d ../webber-sandbox - -# Read-only mode (safe exploration and planning) -.venv/bin/webber-cli chat --mode plan -d ../webber-sandbox - -# Auto-accept mode (no approval prompts - use with caution) -.venv/bin/webber-cli chat --mode auto_accept -d ../webber-sandbox - -# List previous sessions -.venv/bin/webber-cli sessions - -# Resume a previous session -.venv/bin/webber-cli chat --resume -``` - -**CLI Features:** -- **Tab completion** for commands and file paths -- **Command history** persisted to `~/.webber_history` -- **Session persistence** - conversations saved and resumable -- **Config file** - persistent settings via `~/.webber/config.toml` -- **Runtime mode switching** via `mode plan|default|auto_accept` -- **Directory navigation** via `cd ` - -**Configuration:** -```bash -# Show current config -.venv/bin/webber-cli config - -# Initialize config file with defaults -.venv/bin/webber-cli config --init -``` - -**Note:** The API server must be running for CLI commands to work. - ---- - -## Sandbox Management - -The sandbox is a swappable test project for functional testing. - -### Available Templates - -| Template | Description | -|----------|-------------| -| `calculator-cli` | Python CLI with intentional bugs (div-by-zero, missing tests) | -| `empty` | Blank starter project | - -### Commands - -```bash -# List available templates -./sandbox.sh list - -# Load a template (clears sandbox, preserves .venv) -./sandbox.sh load calculator-cli - -# Reset to last loaded template -./sandbox.sh reset - -# Save current sandbox as new template -./sandbox.sh save my-template - -# Check current status -./sandbox.sh status -``` - -### After Loading a Template - -```bash -cd webber-sandbox -source .venv/bin/activate # Create .venv first if missing -pip install -r requirements.txt - -# Read the tasks -cat TASKS.md - -# Run the project's tests -pytest tests/ -v -``` - ---- - -## Testing Webber's Capabilities - -### Scenario: Find bugs in calculator-cli - -```bash -# 1. Load the template -./sandbox.sh load calculator-cli - -# 2. Have Webber explore it (plan mode = read-only) -cd webber-cli -.venv/bin/webber-cli chat --mode plan -d ../webber-sandbox -# Then ask: "find all bugs in the code" - -# 3. Check TASKS.md for expected bugs -cat ../webber-sandbox/TASKS.md -``` - -### Known bugs in calculator-cli: -- Division by zero not handled (`operations.py:divide`) -- Invalid operation causes KeyError (`main.py:get_operation`) -- Power function broken for fractional exponents -- Missing tests for divide and power functions - ---- - -## Key Files for Debugging - -| File | Purpose | -|------|---------| -| `webber-api/logs/server.log` | API server logs | -| `webber-api/src/domains/agents/explore/prompts.py` | Explore agent system prompts | -| `webber-api/src/domains/agents/explore/agent.py` | Explore agent implementation | -| `webber-api/src/ollama/provider.py` | Ollama integration (sanitizes content:null) | -| `webber-api/docs/COVERAGE.md` | Feature coverage and known issues | - ---- - -## Versioning & Releases - -Uses prefixed tags: -- `api/vX.Y.Z` → Triggers API Docker build -- `cli/vX.Y.Z` → Triggers CLI build (future) - -### MANDATORY Release Procedure - -**NEVER push a tag before updating version files.** Follow this exact order: - -```bash -# For API releases: -# 1. Update version in webber-api/pyproject.toml -# 2. Update webber-api/CHANGELOG.md with release notes -# 3. Commit the version bump -git add -A && git commit -m "chore: release api vX.Y.Z" -# 4. Create the tag (AFTER the commit) -git tag api/vX.Y.Z -# 5. Push everything together -git push origin main --tags - -# For CLI releases: -# 1. Update version in webber-cli/pyproject.toml -# 2. Update webber-cli/CHANGELOG.md with release notes -# 3. Commit the version bump -git add -A && git commit -m "chore: release cli vX.Y.Z" -# 4. Create the tag (AFTER the commit) -git tag cli/vX.Y.Z -# 5. Push everything together -git push origin main --tags -``` - -**Why this matters:** Pushing a tag before the version commit requires deleting and recreating the tag, which can trigger CI/CD pipelines prematurely and cause deployment issues. - ---- - -## Troubleshooting - -### API server won't start -```bash -# Check if port is in use -lsof -i :8095 - -# Kill stuck process -pkill -f "uvicorn src.main:app" -``` - -### CLI can't connect -```bash -# Check API is running -curl http://localhost:8095/health - -# Check CLI config -echo $WEBBER_API_URL # Should be http://localhost:8095 -``` - -### Ollama errors -```bash -# Check Ollama is running -curl http://192.168.86.149:11434/api/tags - -# Check model is available -curl http://192.168.86.149:11434/api/tags | grep gemma4 -``` - -### Tests failing -```bash -# Run with verbose output -cd webber-api -.venv/bin/python -m pytest tests/ -v --tb=short -``` - ---- - -## Known Limitations - -1. **Model hallucination** - Mistral Nemo sometimes makes up file contents instead of using tool results - -See `webber-api/docs/COVERAGE.md` for full feature coverage status. diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..1f3bf3b --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,182 @@ +# CLAUDE.md — webber + +Local-LLM multi-agent development assistant — "similar to Claude Code but running locally" +(`webber-api/docs/architecture.md`), backed by Ollama via PydanticAI. Logical monorepo, single +`.git`, three subprojects: `webber-api/` (FastAPI server, deployed), `webber-cli/` (Typer CLI +client), `webber-sandbox/` (swappable test project used by `sandbox.sh`, not shipped). + +## Ports + +| | Port | How | +|---|---|---| +| Local dev | **8095** | `cd webber-api && ./wakeup.sh`, uvicorn `--reload`, logs to `webber-api/logs/server.log` | +| Production | **8086** | container `webber`, confirmed running (`docker ps`) on `docker-dataplane` | + +`wakeup.sh` refuses to start if 8095 is already bound — it does not silently pick another +port. Testing `localhost:8086` on the dev box hits the *container*, not your reload server. + +## Live contract + +`http://localhost:8086/openapi.json` — 10 paths, `version: 1.0.1` (verified 2026-08-09, +matches `webber-api/pyproject.toml` and the live `/health` response). Human docs at +`http://localhost:8086/docs`. Query the live spec rather than inferring routes from source — +`src/domains/router.py` currently has two routers commented out (see Architecture), so a +source read alone will overcount if you don't check whether an include is live. + +``` +/, /health, /agents/, /agents/run, /agents/stream, /agents/{agent_type}, +/conversations/, /conversations/{conversation_id}, +/conversations/{conversation_id}/messages, /conversations/{conversation_id}/save +``` + +## Architecture + +Domain-first layout under `webber-api/src/domains//`. `src/main.py` includes exactly one +router, `src.domains.router.root_router`, which composes the domain routers. A full directory +map lives in `webber-api/docs/architecture.md` — read that before adding a domain rather than +duplicating it here. + +**How liveness below was established:** `docker exec webber python3 -c "import src.main; import +sys; print(sorted(m for m in sys.modules if m.startswith('src.')))"` — i.e. importing the real +app inside the running container and reading `sys.modules`, not grepping `main.py`. Re-run that +command to re-check; a grep of imports will miss function-body imports, and this repo has one +that matters. + +- **Wired at startup, serving routes:** `src.domains.health`, `src.domains.agents` (router + + `explore`/`plan`/`task` agent packages), `src.domains.conversations`, `src.shared.*`, + `src.ollama`, `src.db` (imported both by `conversations/router.py` at module scope and by + `main.py`'s lifespan shutdown handler). +- **Present in source, explicitly disabled:** `src/domains/router.py` has + `# from src.domains.auth.router import router as auth_router` and the equivalent for + `tools_router` — both commented out with the include calls also commented out. `src/domains/auth/` + is just an empty `__init__.py`. This one *is* dead — the disabling is visible in the same file, + not a matter of tracing an indirect import. +- **The trap: `src/domains/tools/` is not in `sys.modules` right after `import src.main`, but it + is not dead.** `src/domains/agents/{explore,plan,task}/agent.py` each have a method + (e.g. `PlanAgent._register_tools`) that does `from src.domains.agents.plan.tools import + register_plan_tools` **inside the function body**, called every time that agent is + constructed — i.e. on every `/agents/run` or `/agents/stream` request for that agent type. + That nested module then imports the real tool classes from `src.domains.tools.file`, + `.search`, `.shell` at module scope. A static snapshot taken before any request is served + will not show `src.domains.tools` loaded; that is a timing artifact, not evidence it is + unused. Don't delete `src/domains/tools/` on the strength of a `sys.modules` check alone — + confirm by hitting `/agents/run` and re-checking, or by tracing the call graph from each + agent's `_register_tools`. +- **`src/cli/`** is the implementation behind `webber-cli`'s `pyproject.toml` script entry — + it is a separate Typer app, not imported by the API (`src.main`) at all. Its liveness is + "is the CLI installed and invoked", not "is it wired into the API process". + +Group new work by domain, not file type — `webber-api/docs/fastapi-best-practices.md` is the +house reference (mirrors the convention used across the other in-house FastAPI services here). + +## Database + +SQLite by default (`database_url = "sqlite+aiosqlite:///./webber.db"` in +`src/shared/config.py`), not Postgres — confirmed by reading `src/shared/config.py` and +`src/db/database.py` (the latter's docstring says the pattern is ported from core-api, but +the backend differs). Models under `webber-api/src/domains//models.py` import `Base` +from `src/db/models.py`. No Alembic here (unlike core-api) — did not find a migrations +directory; unverified whether schema changes have any managed migration path at all. Check +before assuming one exists. + +## Working here + +**Test locally first.** `cd webber-api && ./wakeup.sh` auto-reloads on code changes (not on +`requirements.txt` changes — restart after adding a dependency). Deploy only once a feature +is complete and tested. + +```bash +cd webber-api +.venv/bin/python -m pytest tests/ # all tests +.venv/bin/python -m pytest tests/ -v --cov # verbose + coverage +.venv/bin/python -m pytest tests/test_tools.py -v # single file +``` + +`webber-api/pyproject.toml` declares `[tool.ruff]` and `[tool.mypy]` — unlike core-api, this +repo does have ruff/mypy config; whether either runs in CI is a separate question (see CI below +— it does not). + +Copy `webber-api/.env.example` to `webber-api/.env`. Notable defaults: `OLLAMA_URL` points at +`192.168.86.149:11434` (the host's Ollama, not a container), `OLLAMA_AGENT_MODEL=gemma4:e2b`, +optional Tatlock integration via `TATLOCK_API_URL`/`INTERNAL_API_KEY`, optional SearXNG via +`SEARXNG_URL` for the `web_search` tool. + +### Sandbox + +`webber-sandbox/` is a disposable project used to exercise the agents end-to-end, managed by +`./sandbox.sh {list,load,reset,save,status}` from the repo root. `sandbox-templates/` holds the +reusable templates (`calculator-cli` has intentionally-seeded bugs for testing Explore/Task). +This directory is fixture material, not shipped code — do not treat bugs in it as real bugs. + +### CLI + +`webber-cli/` is a Typer client (`webber-cli status|chat|explore|sessions|config`) with tab +completion, session persistence (`~/.webber_history`, `~/.webber/config.toml`), and three chat +modes (`plan` read-only, `default`, `auto_accept`). It talks to the API over HTTP — it does not +share a process with `webber-api`. Run it from its own venv: `cd webber-cli && .venv/bin/webber-cli status`. + +## CI + +`.gitea/workflows/build-api.yml` triggers only on `api/vX.Y.Z` tags: creates a Gitea release, +builds/pushes `git.schweitz.net/jpmschweitzer/webber-api`, then pings Watchtower. +`build-cli.yml` triggers on `cli/vX.Y.Z` tags but is a placeholder — it only echoes a TODO, it +does not build or publish anything. **No test or lint gate runs in CI for either package** — +pytest and ruff only run locally or on request. Verify tests pass before tagging. + +## Work tracking + +Work lives in **pql**, not a markdown TODO or `docs/COVERAGE.md`. Tickets *and* decisions for +this repo live in the **workspace** vault; this repo's `.pql/` and `governance/` trees stay +empty (D-15). + +Two things must be spelled out on every invocation from in here, and each fails differently: + +- **`pql` is not on the non-interactive `PATH`** — use `/home/jpmschweitzer/.local/bin/pql`. +- **`--vault /mnt/media/Projects` is mandatory.** pql anchors a vault at the nearest `.git/` + ancestor, and this repo is one, so a bare call resolves to *this repo's* empty vault. Reads + return nothing; a **write** creates a stray vault and starts ticket ids at T-1, colliding + with the real ones. + +```bash +/home/jpmschweitzer/.local/bin/pql --vault /mnt/media/Projects ticket list +/home/jpmschweitzer/.local/bin/pql --vault /mnt/media/Projects plan whatsnext +/home/jpmschweitzer/.local/bin/pql --vault /mnt/media/Projects decisions list --domain webber +``` + +Do not add a TODO section to a markdown file. + +## Git + +- **History is linear — no merge commits.** Work on `main`, or a short-lived branch that is + fast-forwarded and deleted. (This repo's `AGENTS.md` previously mandated `feature/...` or + `fix/...` branches for every change and forbade committing to `main` directly — that rule was + retired workspace-wide on 2026-08-08 and does not apply here anymore.) +- **Conventional Commits**: `feat:`, `fix:`, `refactor:`, `docs:`, `chore:`. +- **Atomic commits** — one logical change each. +- **Stage explicitly. Never `git add -A`** — denied by policy; it sweeps in whatever else is + dirty, including secrets. +- Each package versions independently via prefixed tags (`api/vX.Y.Z`, `cli/vX.Y.Z`) and its + own `CHANGELOG.md` (`webber-api/CHANGELOG.md`, `webber-cli/CHANGELOG.md`); the root + `CHANGELOG.md` is just an index pointing at both. + +## Releasing (API) + +Ask whether a deploy is wanted first — it is not automatic. + +1. Bump the version in `webber-api/pyproject.toml`. +2. Move `[Unreleased]` entries into a dated version section in `webber-api/CHANGELOG.md`. +3. Stage the changed files by name, commit, tag `api/vX.Y.Z`, `git push origin main --tags`. +4. Gitea CI (`build-api.yml`) builds and pushes the image on the tag; Watchtower deploys it. +5. Verify: `curl http://192.168.86.149:8086/health`. + +CLI releases (`cli/vX.Y.Z`) currently only log a TODO in CI — there is no build/publish step +to trigger yet. + +## Known issues (carried over, unverified beyond what's stated) + +- **Model hallucination**: the Explore agent's model can hallucinate file contents instead of + using actual tool results, per `webber-api/AGENTS.md` — a mitigation (stronger model or + response validation) was suggested there but not confirmed implemented. +- **Ollama `content: null` workaround**: `src/ollama/provider.py` (confirmed present, loaded at + startup per the `sys.modules` check above) sanitizes `content: null` to `content: ""` for + assistant messages with tool calls, working around an Ollama API limitation. diff --git a/README.md b/README.md index b4c8839..a2acf27 100644 --- a/README.md +++ b/README.md @@ -113,7 +113,7 @@ This project uses prefixed tags for independent release cycles: ## Documentation -- `webber-api/AGENTS.md` - API development guidelines +- `CLAUDE.md` - Agent development guidelines (repo-wide) - `webber-api/docs/COVERAGE.md` - Feature coverage and roadmap - `webber-api/docs/architecture.md` - System architecture - `webber-cli/README.md` - CLI usage guide diff --git a/webber-api/AGENTS.md b/webber-api/AGENTS.md deleted file mode 100644 index a57932f..0000000 --- a/webber-api/AGENTS.md +++ /dev/null @@ -1,128 +0,0 @@ - -# AGENTS.md - -> **Start every session by reading this file.** -> This file outlines the operational protocols, coding standards, and architectural decisions for this FastAPI project. - -## 1. Agent Operational Protocols - -### 🧠 Work Patterns (Plan-Act-Reflect) -* **Plan:** Before writing code, briefly outline your plan. Identify which files you will touch and what the side effects might be. -* **Act:** Execute the changes in small, atomic steps. -* **Reflect:** After coding, verify your work. Did you break existing tests? Did you add new tests? - -### 🛡️ Git Discipline -* **ALWAYS add the relevant tests for the added code** Make sure to keep the test coverage up as we go, and run tests before commiting. -* **NEVER commit to `main` or `master` directly.** Always create a feature branch: `feature/your-feature-name` or `fix/issue-description`. -* **Commit Messages:** Use the [Conventional Commits](https://www.conventionalcommits.org/) format. - * `feat: add user login endpoint` - * `fix: resolve database connection timeout` - * `refactor: split monolith dependency file` -* **Atomic Commits:** Keep commits small. One logical change = one commit. - -### 📝 Changelog Maintenance -* **Update `CHANGELOG.md`** with every user-facing change. -* Format: `## [Unreleased] - YYYY-MM-DD` followed by `### Added`, `### Changed`, or `### Fixed`. - -### 🚀 Release Flow -When changes are ready for deployment: - -1. **Ask user if deploy cycle is desired ** - -2. **Update version** in `pyproject.toml`: - - Bug fixes: bump patch version (1.8.3 → 1.8.4) - - New features: bump minor version (1.8.4 → 1.9.0) - -3. **Update CHANGELOG.md**: - - Move items from `[Unreleased]` to new version section - - Add release date: `## [1.8.4] - 2025-12-16` - -4. **Commit and tag**: - ```bash - git add -A - git commit -m "fix: description of changes" - git tag v1.8.4 - git push origin main --tags - ``` - -5. **CI/CD triggers automatically**: - - Gitea CI builds Docker image on new version tag (starts with "v") - - Watchtower pulls and deploys to production - - Verify deployment: `curl http://192.168.86.149:8086/health` - ---- - -### 🧪 Local Development Setup - -* **Always test locally first** before committing and deploying. The build-deploy loop is slow. -* **Only deploy** when a phase or feature is complete and tested locally -* **Environment**: Copy `.env.example` to `.env` and configure for your local setup - -#### ⚠️ CRITICAL: Starting the Local Server - -**ALWAYS use `./wakeup.sh` to start the local server. NEVER use raw uvicorn commands.** - -```bash -./wakeup.sh -``` - -The wakeup script provides: -- **Port conflict detection** - Warns if port 8086 is already in use -- **Virtual environment activation** - Ensures correct Python environment -- **Centralized logging** - All logs written to `logs/server.log` for easy tailing -- **Auto-reload** - Code changes picked up automatically (except requirements.txt changes) -- **Consistent configuration** - Same startup every time - -To monitor logs in another terminal: -```bash -tail -f logs/server.log -``` - -To stop the server: Press `Ctrl+C` - -To kill a stuck server: -```bash -pkill -f "uvicorn src.main:app" -# or -kill $(lsof -t -i:8086) -``` - -#### Testing - -**Test REST endpoints** against `http://localhost:8086`: -```bash -curl http://localhost:8086/health -curl http://localhost:8086/ -curl http://localhost:8086/docs # Swagger UI -``` - -**Running tests**: Always use the venv explicitly to avoid environment mismatches: -```bash -.venv/bin/python -m pytest tests/ # All tests -.venv/bin/python -m pytest tests/ -v # Verbose output -.venv/bin/python -m pytest tests/ --cov # With coverage -``` - ---- - -## 1.5 Known Issues & Future Improvements - -### Explore Agent - -- **Model Hallucination**: Mistral Nemo sometimes hallucinates file contents instead of using actual tool results. Consider using a more capable model (codestral, qwen2.5-coder) or adding response validation. - -- **Ollama Provider**: We use a custom `WebberOllamaProvider` (ported from tatlock) that sanitizes `content: null` to `content: ""` for assistant messages with tool calls. This works around an Ollama API limitation. - -- **Gitignore Support**: ✅ Fixed - The filesystem tools now honor `.gitignore` patterns and default ignores (`.venv/`, `__pycache__/`, `node_modules/`, etc.). - ---- - -## 2. FastAPI Architecture & Best Practices -*Reference: [FastAPI Best Practices](https://github.com/zhanymkanov/fastapi-best-practices)* - -### 📂 Project Structure (Directory-based, NOT File-type based) -Do **not** group files by type (e.g., one huge `routers` folder). Group by **domain/module** inside a `src/` directory. - -**Correct Structure:** -```text -to be determined \ No newline at end of file