39 Commits
Author SHA1 Message Date
jpmschweitzerandClaude 7172927a79 release api/v1.1.0
Build and Push API / release (push) Successful in 2s
Build and Push API / build (push) Successful in 2m17s
Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-11 16:30:09 +02:00
jpmschweitzerandClaude 3e495daa73 fix(webber-api): clear mypy, and the dead code it was covering for
55 errors to zero. Nearly all of them traced back to two causes rather than 55.

THE DECORATOR. @logged wraps ~24 functions across this package and was declared
`def decorator(func: Callable):` with no ParamSpec and no return annotation, so
it erased the signature of everything it touched. ToolResult.execute() is
annotated `-> ToolResult`; through the decorator it came back Any, and mypy
reported 33 no-any-return errors spread across the tools and agents. Each looked
like a local annotation slip. All of them were one decorator. Typed with
ParamSpec/TypeVar; the async branch casts at the await rather than loosening R,
because loosening R would put the Any straight back into every caller.

THE MISSING TYPE PARAMETER. BaseAgent was not generic, so _create_agent returned
a bare Agent — Agent[Any, Any] — and pydantic_ai then typed every run() result
as Any. BaseAgent is now Generic[CtxT] bound to AgentContext, _agent is declared
on the base instead of reached through hasattr, and the three tool-registration
functions take their agent's real context type. tools_streaming.py already did
this; the other three had not been updated.

Eight `execute` overrides carry a targeted ignore rather than a package-wide
disable_error_code. Every tool narrows the base's **kwargs to its own named
parameters, which is a real LSP violation — but nothing anywhere is typed as
BaseTool, and every call site constructs the concrete tool. The abstract method
earns its place by making a tool without execute impossible to instantiate. The
reasoning lives in BaseTool.execute's docstring; the per-site suppressions mean
an override that IS unsound still gets caught.

BaseAgent.run_stream widened to AsyncIterator[str | StreamEvent], which is what
callers already receive: task streams structured events, explore and plan stream
strings, and the router branches on isinstance with a comment calling the string
path legacy. The annotation now says what the code does.

AND THE PART THAT MATTERS MORE THAN THE TYPES.

Chasing the last error found that the Ollama sanitiser has been broken. It
fetched the parent's chat getter with `AsyncOpenAI.chat.fget`, and openai made
`chat` a functools.cached_property, whose getter is `.func`. Touching `.chat`
raised AttributeError — meaning the content: null workaround that CLAUDE.md
documents as live would have failed on the first completion any agent attempted.
Confirmed in the running container (openai 2.46.0) as well as locally (2.15.0).

Two things hid it. The line carried a bare `# type: ignore`, which suppressed
precisely the complaint that would have caught it. And /agents/run and
/agents/stream have served zero requests in 30 days, so nothing exercised the
path. A mitigation can rot completely while every check stays green, if no check
actually runs it.

The lookup now reads whichever getter the descriptor exposes and raises a
legible TypeError if openai adopts a third shape. tests/test_ollama_provider.py
walks the chain an agent request walks, short of the network call —
mutation-checked: all four fail against the old lookup.

215 passed, 23 skipped, plus the four new. mypy clean over 90 files.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-11 16:27:51 +02:00
jpmschweitzerandClaude eb3467d06a fix(webber-api): clear ruff, and two things it was pointing at
97 findings to zero. Most were mechanical — 52 unsorted import blocks, 10
unsorted __all__, assorted pyupgrade and simplify hints. Two were not, and both
were visible only because the lint made me look.

`webber version` did not exist. src/cli/commands/version.py defines
show_version(), main.py imported it, and the registration line was never
written — the CLI exposed chat and explore only. The import carried
`# noqa: F401`, which is what kept the omission quiet: someone marked the
symptom as intentional instead of asking why it was unused. show_version is not
redundant with the --version flag; it prints the resolved Ollama URL, model and
debug state, which is the form worth having when something is misconfigured.
Registered, and the suppression dropped because the import is now genuinely used.

test_spawn_explore_agent asserted nothing. It built a mock RunContext, patched
get_agent, and stopped at the comment "For now, verify the explore agent would
be called correctly". It had been counted as a passing test. An AST sweep of all
238 test functions found it was the only one, which is worth knowing — the
problem was contained, not systemic. It is now skipped with a reason, so it
reports as unfinished rather than as passing. Reducing it rather than deleting
its imports was the point: tidying the imports would have made a hollow test
look clean.

Two findings were false positives, and both are recorded rather than silently
worked around:

B023 flagged run_agent closing over full_prompt and ctx. Traced: agent_task is
awaited at line 326 before `continue` reaches the next iteration, so neither
name can be rebound while the closure is pending, and the exception path
cancels and awaits too. Not a bug. Bound as defaults anyway, because that stays
true if the await ever moves. I had called it a live bug before tracing it,
which is the mistake Rule 5 exists for.

RUF012 flagged `rules: list[ApprovalRule] = []` on ApprovalRuleSet. Its
suggested fix — annotate ClassVar — would remove the field from the model.
ApprovalRuleSet is a pydantic model and pydantic deep-copies defaults per
instance; verified by constructing two and confirming their lists are distinct
objects. Suppressed with that evidence in the comment. Ruff cannot see the
pydantic base because BaseSchema is a local subclass of BaseModel.

Also moved a stray `from src.shared.logging import ...` that had drifted below
a function definition, and merged a nested if in the ollama provider.

215 passed, 23 skipped, unchanged except for the new skip. `webber version`
exercised end to end.

mypy is NOT addressed here and the gate still fails on it — 55 errors in 14
files, 35 of them no-any-return from pydantic_ai's untyped returns. That was
hidden behind ruff, because the gate stops at the first failing stage.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-11 15:05:02 +02:00
jpmschweitzerandClaude 9f5e331d11 build(ci): move the pre-push gate into the Makefile
The hook carried ~50 lines of gitleaks logic and a comment explaining it was
self-contained because "this repo has no Makefile". It has one now, so the
reason is gone and the arrangement is backwards: a hook is a trigger, and
logic belongs where it can be read, run by hand, and changed under review.

.githooks/pre-push is now a byte-identical shim onto `make pre-push` in every
repo in the workspace. The scan itself moves to ci/secrets.sh unchanged, and
`make secrets` runs it on its own.

The call surface is identical everywhere; what it runs is not, and should not
be — each repo gates what it actually has. That is the point of standardising
the name rather than the contents: nobody has to read a repo to find out how
to check it.

secrets runs first, deliberately. It is the only failure here that cannot be
undone by fixing it afterwards — a failed lint costs another commit, a pushed
credential is cached and indexed whether or not it is later deleted.

Some of these gates fail today, on lint debt that predates them, and they are
left wired anyway. The board was measured once and written down in T-56
instead of being worked around here. Narrowing each gate to whatever already
passes would produce a gate that reports success for doing nothing, which is
the failure this workspace keeps rediscovering.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 18:57:22 +02:00
jpmschweitzerandClaude 225dc74385 ci(make): reserve exit 69 for "could not run" (D-26)
Environment guards now exit 69 rather than 1, so a caller can tell a suite
that could not start from one that ran and failed. The first toj test sweep
reported "3 repositories failed" and none of the three had executed a test —
two could not find go, one had no venv. That points the reader at the tests
when the fault is in the environment.

Only the environment guards change. A gitleaks finding, a failed test run and
a vulncheck hit still exit 1, because those did run and did fail.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 15:56:20 +02:00
jpmschweitzerandClaude 80d5cc1c3e build: add the Makefile command surface (D-27)
Every repo gets one at the root: help, plus test and lint where those exist.
The point is that a target name means the same thing in every repo, so an
agent or a person can act without reading the repo first.

Paths resolve here rather than in callers (D-10). python3 on this host is 3.8
and cannot parse these sources, and a bare pytest or ruff resolves only in a
login shell — so both are named explicitly through the venv, and a missing
venv fails with the command to fix it rather than a bare no-such-file.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 15:10:32 +02:00
jpmschweitzerandClaude 964c071d3f chore(claude): pin PQL_VAULT per project so cwd stops choosing the vault
pql is now a bare word on PATH, which removed the long incantation that had
been forcing --vault into every call by habit. Convenience lowered the cost
of the wrong thing without lowering the cost of the right one: a three-word
pql ticket new targets whichever vault the cwd happens to sit in, and there
are nine of them with colliding id sequences.

PQL_VAULT in each project settings file makes the vault a property of the
session rather than of the working directory — the same lesson Rule 3 records
for git -C, applied to pql. Verified the env var overrides cwd discovery,
that an explicit --vault still beats the env var, and that the harness
hot-reloads it without a restart.

This does not make provenance visible: no output says which vault answered,
so a forgotten --vault still returns a well-formed answer about the wrong
dataset. That remains T-37.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 13:48:59 +02:00
jpmschweitzerandClaude 0883101b17 chore(claude): deny toj in the sub-repos
toj is now on the global PATH as /usr/local/bin/toj, so its scope boundary
had to stop being "the absolute path is inconvenient to type" and start
being a rule. Its repo and settings verbs operate on the workspace root; run
from inside this repo they answer about the wrong tree.

Both spellings are denied, bare and absolute, because a deny with one
spelling left open is decorative.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 13:42:04 +02:00
jpmschweitzerandClaude 992ff8256e ci: gate pushes on a gitleaks scan of the outgoing commits
No repo here scanned for committed credentials. The hook is self-contained
rather than delegating to a Makefile, because this repo has none and a hook
reaching into a sibling repo breaks the moment this one is cloned elsewhere.

Scans the outgoing range rather than full history: history carries settled
findings — test fixtures, vendored third-party code — and a gate that fails
on something unfixable gets bypassed within a week.

Setting core.hooksPath means pql init must replant its replication shims into
.githooks, which is why they are gitignored here alongside the tracked
pre-push. Same layout pql itself uses.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 12:46:55 +02:00
jpmschweitzerandClaude b4d5c4d9b2 docs: correct the vault policy, and qualify workspace decision ids
This repo's Work tracking section still described the reversed policy --
that tickets and decisions live in the workspace vault and this repo's
trees stay empty. That was overturned the same day: repo vaults are
standalone and a repo's work travels with a clone, because the changelog
is committed.

Every other repo was corrected at the time; this one was missed because
the search for the offending phrase used a fixed string and the phrase
happened to wrap across a line break here. Worth noting as a search
failure rather than a writing one -- five files were checked, four
matched, and the fifth was reported clean.

Decision ids are also qualified now. They are per-vault sequences, so a
bare D-15 here will mean this repo's D-15 the moment this repo records
one; pql already holds D-1 through D-31 against the workspace's D-1
through D-21, all of them unrelated.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 04:17:29 +02:00
jpmschweitzerandClaude 8eec3b68d6 docs: replace both AGENTS.md files with one CLAUDE.md
One agent doc per repo, and it is CLAUDE.md. This repo carried two --
one at the root and one under webber-api/ -- which is the drift problem
in its purest form: two documents, one subject, and no way to know which
the last reader trusted. Written fresh rather than reformatted.

README.md linked to webber-api/AGENTS.md, so that pointer moves with the
file rather than dangling.

The architecture section states the method used to establish what is
live -- import the app inside the container and read sys.modules -- and
then the case where that method fails here. src/domains/tools is absent
from a cold snapshot and is entirely live: each agent's _register_tools
imports its tool package from inside the method body, on every
/agents/run. Absence from a snapshot taken before any request is served
is a timing artifact, not evidence of death, and deleting on that basis
would have removed the tool layer.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 03:14:50 +02:00
jpmschweitzerandClaude 401c7f4e8c chore: adopt the workspace agent-config baseline
Commits a .claude/settings.json rather than leaving permissions to
per-developer local state, and initialises a pql vault for this repo's
tickets and internal decisions.

Every git deny rule appears in both the `git <verb>` and `git * <verb>`
forms. Only the second catches `git -C <path>`, and without it the whole
deny list is decorative -- it looks like a policy and stops nothing.

The allow list carries pql's absolute path alongside the bare name.
pql is installed to ~/.local/bin, which is on the login PATH but not the
one a non-interactive shell gets, so the bare-name rules match nothing on
their own and every call would prompt anyway.

.gitignore now covers .claude/settings.local.json, which is machine-local
and must never be shared. `pql init` contributed the .pql/* rules with an
exception for the changelog, which is the replication log of record and
has to be committed for tickets to travel with a clone.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-09 03:14:24 +02:00
jpmschweitzerandClaude 9612c7af05 docs(architecture): correct agent model to gemma4:e2b
Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-07 14:59:12 +02:00
jpmschweitzerandClaude 9d58df85f3 fix(config): default agent model to gemma4:e2b
mistral-nemo-large holds ~9.2 GB of the 11 GB card it shares with Speaches,
which starves Whisper and breaks voice transcription. gemma4:e2b holds
1.9 GB and is faster. The deployed stack already overrides this via
OLLAMA_AGENT_MODEL; this aligns the default so a deployment without that
override does not reintroduce the contention.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-07 14:58:57 +02:00
jpmschweitzerandClaude 3de96fe070 docs: correct Ollama model references to gemma4:e2b
The deployed agent model is gemma4:e2b; these references still named
mistral-nemo, so the setup and troubleshooting steps checked for a model
that is no longer expected to be present.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-08-07 14:51:58 +02:00
jpmschweitzerandClaude Fable 5 e2acb7de39 docs(architecture): registry is git.schweitz.net not git.schweitz.internal
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 17:11:08 +02:00
jpmschweitzerandClaude Fable 5 5998571890 chore: release api v1.0.1
Build and Push API / release (push) Successful in 2s
Build and Push API / build (push) Successful in 1m39s
Patch release for the network migration: docker-hostname config
defaults for Tatlock/SearXNG and CI image push via the
git.schweitz.net registry route.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 12:32:02 +02:00
jpmschweitzerandClaude Fable 5 6ac1d15591 fix(config): use docker hostnames for tatlock and searxng defaults
The homelab is retiring *.schweitz.internal and will rebind host ports
to 127.0.0.1, so container-to-container traffic must use container
names on the docker-dataplane network. Switch defaults from host
IP:port to http://tatlock:8000 and http://searxng:8080 (SearXNG's
internal port is 8080; 8087 is only the host-published port).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 12:21:43 +02:00
jpmschweitzerandClaude Fable 5 22af23827d chore(ci): push images via git.schweitz.net registry
The .internal registry domain is being retired; git.schweitz.net now
serves the registry without SSO on /v2/.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-19 11:10:35 +02:00
jpmschweitzerandClaude Opus 4.5 d385f47395 chore: release api v1.0.0
Build and Push API / release (push) Successful in 4s
Build and Push API / build (push) Successful in 2m26s
- Event-based streaming for task agent
- Retry logic when LLM responds without calling tools
- Hardened prompts to enforce tool use
- Working directory context in all agent prompts
- Project paused: local LLMs not capable enough for agentic use

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-15 07:54:33 +01:00
jpmschweitzerandClaude Opus 4.5 6ea520519c fix: remove invalid mode kwarg from trace_span + add integration tests
- Remove mode=mode.value from trace_span calls (trace_span only accepts
  name and logger parameters)
- Add TestPermissionModeIntegration tests that verify mode string->enum
  conversion works correctly through the full request flow
- Add TestAgentMethodSignatures tests that verify function signatures
  match expected interfaces (catches invalid kwargs at test time)

These tests would have caught both the mode string/enum issue and the
trace_span invalid kwarg issue before they hit production.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-14 15:46:28 +01:00
jpmschweitzerandClaude Opus 4.5 2f11ad79cf fix: handle mode as string due to use_enum_values=True
The BaseSchema has use_enum_values=True which makes Pydantic store
enum values as strings. Added _get_mode() helper to convert back to
PermissionMode enum before passing to agent methods.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-14 15:07:55 +01:00
jpmschweitzerandClaude Opus 4.5 9be877e9a0 feat: make chat the default command
Running 'webber-cli' without arguments now starts chat mode.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-14 13:09:39 +01:00
jpmschweitzerandClaude Opus 4.5 b87e61248e feat: add config file support to CLI
- Add ~/.webber/config.toml for persistent settings
- Support api.url, api.key, cli.mode, cli.stream, history.file
- Environment variables override config file values
- Add 'config' command to show settings and init config file
- Update all commands to use config defaults

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-14 12:06:13 +01:00
jpmschweitzerandClaude Opus 4.5 daa9543790 feat: add session persistence to CLI
- Add save-only endpoint (POST /conversations/{id}/save) for persisting
  messages without triggering agent execution
- Add sessions command to list previous conversation sessions
- Add --resume flag to chat command for resuming sessions by ID
- Buffer streamed responses and save after completion
- Update AGENTS.md with session commands and remove outdated limitation
- Add 3 new tests for save endpoint (208 total)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-14 09:50:25 +01:00
jpmschweitzerandClaude Opus 4.5 b7956f88ed feat: add permission modes and CLI orchestration layer
Permission Modes:
- Add default/plan/auto_accept modes controlling tool access
- Plan mode restricts Task agent to read-only tools only
- Auto-accept mode bypasses approval prompts (with confirmation)

Approval Scaffolding:
- Add ApprovalRule/ApprovalRuleSet for granular tool control
- Pattern-based matching on tool name and arguments
- Default rules for common safe/dangerous patterns
- Prep for future bidirectional approval flow

CLI Refactor:
- Default to Task agent (main orchestrator)
- Add --mode flag and runtime mode switching
- Integrate prompt_toolkit for better UX:
  - Persistent command history (~/.webber_history)
  - Tab completion for commands and file paths
  - Auto-suggest from history
- Deprecate standalone 'explore' command

Other:
- Split CHANGELOG.md into per-package files
- Update AGENTS.md release procedure for both packages

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-14 08:48:07 +01:00
jpmschweitzerandClaude Opus 4.5 acf231eb66 feat: add retry logic for transient failures
Build and Push API / release (push) Successful in 3s
Build and Push API / build (push) Successful in 1m14s
- @with_retry decorator and retry_async() function
- Exponential backoff with jitter
- Retries on: timeout, connection errors, HTTP 429/5xx
- Web search tool now retries on network failures
- Configurable via RETRY_MAX_ATTEMPTS, RETRY_BASE_DELAY, RETRY_MAX_DELAY
- 29 new tests (205 total passing)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:22:16 +01:00
jpmschweitzerandClaude Opus 4.5 2f97041aa9 fix: replace litellm with tiktoken for token counting
Build and Push API / release (push) Successful in 3s
Build and Push API / build (push) Successful in 2m26s
- litellm had dependency conflicts with pydantic-ai
- tiktoken is lighter and already required by pydantic-ai
- Updated documentation (README.md, architecture.md)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 23:05:11 +01:00
jpmschweitzerandClaude Opus 4.5 2523db4da7 feat: add conversation persistence and context management layer
Build and Push API / release (push) Successful in 3s
Build and Push API / build (push) Failing after 34s
- SQLAlchemy async database layer (SQLite dev, PostgreSQL prod)
- Conversation and Message models with UUID primary keys
- Token counting utilities using litellm
- Context summarization at 80% token threshold
- REST API endpoints for multi-turn conversations
- 19 conversation tests, 6 token tests (176 total passing)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 22:42:38 +01:00
jpmschweitzer 470b7448ac chore: release api v0.3.4
Build and Push API / release (push) Successful in 4s
Build and Push API / build (push) Successful in 1m18s
2026-01-11 20:21:50 +01:00
jpmschweitzerandClaude Opus 4.5 b5b2346db5 feat: add Plan Agent for implementation planning
Build and Push API / release (push) Successful in 5s
Build and Push API / build (push) Successful in 1m15s
- Add PlanAgentImpl with read-only tools only
- System prompts optimized for architecture planning
- Outputs step-by-step implementation plans with critical files
- 15 unit tests for registration, tools, and API
- Update COVERAGE.md to ~70% complete

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 19:57:30 +01:00
jpmschweitzerandClaude Opus 4.5 617ff61347 chore: release api v0.3.2
Build and Push API / release (push) Successful in 3s
Build and Push API / build (push) Successful in 1m15s
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 19:43:08 +01:00
jpmschweitzerandClaude Opus 4.5 8609181447 docs: add mandatory release procedure to AGENTS.md
Document the correct order for creating releases:
1. Update pyproject.toml version
2. Update CHANGELOG.md
3. Commit version bump
4. Create tag
5. Push with --tags

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 19:37:20 +01:00
jpmschweitzerandClaude Opus 4.5 1f3b241485 chore: release api v0.3.1
Build and Push API / release (push) Failing after 3s
Build and Push API / build (push) Has been skipped
- Update changelog with v0.3.0 and v0.3.1 changes
- Bump version in pyproject.toml to 0.3.1

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 19:36:24 +01:00
jpmschweitzerandClaude Opus 4.5 c839e263f9 test: add integration and E2E test infrastructure
Build and Push API / release (push) Successful in 3s
Build and Push API / build (push) Has been cancelled
- Add pytest markers (integration, e2e, slow) with skip logic
- Add command line options (--run-integration, --run-e2e)
- Create sample_project and sample_project_with_bug fixtures
- Add test_integration.py with 10 LLM tests
- Add test_e2e.py with 12 API server tests
- Update COVERAGE.md to reflect ~65% complete

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 19:33:21 +01:00
jpmschweitzerandClaude Opus 4.5 ef69d9c945 docs: update coverage, READMEs, and add security tests
- Update COVERAGE.md to reflect completed features (now ~60%)
- Update main README with features and tools list
- Update CLI README with streaming options
- Expand API tests from 5 to 11 (add stream endpoint tests)
- Add 14 security tests for path traversal, command injection
- Total tests: 109 (up from 88)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 18:53:36 +01:00
jpmschweitzerandClaude Opus 4.5 82a816a5b5 feat: add web search tool using SearXNG
Add WebSearchTool that queries the self-hosted SearXNG metasearch engine
for current information, documentation, and facts beyond training data.

- Add SEARXNG_URL and SEARXNG_TIMEOUT config settings
- Create WebSearchTool with query, num_results, categories params
- Register web_search tool with explore agent
- Add 10 tests for search functionality

Usage: Agents can now use web_search(query="...") to find current info.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 14:09:21 +01:00
jpmschweitzerandClaude Opus 4.5 f6256363a2 feat: add streaming responses to API and CLI
Add real-time streaming support for agent responses using Server-Sent
Events (SSE). Responses now appear as they're generated instead of
waiting for completion.

- Add run_stream method to BaseAgent and ExploreAgentImpl
- Add /agents/stream SSE endpoint to API router
- Add run_agent_stream method to CLI client
- Add --stream flag to chat and explore commands (enabled by default)
- Use --no-stream for batch mode with spinner

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 12:58:54 +01:00
jpmschweitzerandClaude Opus 4.5 d0fa5b38a7 feat: add coding tools (edit_file, write_file, bash)
New tools for code modification:
- EditFileTool: find-and-replace with safety checks (unique match required)
- WriteFileTool: create/overwrite files with path validation
- BashTool: full bash with controlled write access

Security controls on BashTool:
- Allowed: mkdir, touch, cp, mv, rm (single files), git, pip, pytest
- Forbidden: sudo, curl, wget, ssh, rm -rf, chmod 777

Includes 39 new tests (78 total now passing).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-10 11:16:05 +01:00
102 changed files with 11066 additions and 716 deletions
+67
View File
@@ -0,0 +1,67 @@
{
"env": {
"PQL_VAULT": "/mnt/media/Projects/webber"
},
"permissions": {
"allow": [
"Bash(pql)",
"Bash(pql *)",
"Bash(/home/jpmschweitzer/.local/bin/pql:*)",
"Bash(git status:*)",
"Bash(git log:*)",
"Bash(git diff:*)",
"Bash(git branch:*)",
"Bash(.venv/bin/python -m pytest:*)",
"Bash(.venv/bin/pytest:*)",
"Bash(pytest:*)",
"Bash(ruff *)",
"Bash(mypy *)",
"Bash(docker logs webber:*)",
"Bash(curl -s http://localhost:8086/*)",
"Bash(curl -s http://localhost:8095/*)"
],
"deny": [
"Bash(/mnt/media/Projects/cladmin/ops/bin/toj)",
"Bash(/mnt/media/Projects/cladmin/ops/bin/toj:*)",
"Bash(chmod -R 777 *)",
"Bash(chmod 777 *)",
"Bash(dd if=*)",
"Bash(find * -delete*)",
"Bash(find * -exec*)",
"Bash(git * add --all*)",
"Bash(git * add -A*)",
"Bash(git * add .)",
"Bash(git * branch -D *)",
"Bash(git * checkout -- *)",
"Bash(git * clean -fd*)",
"Bash(git * clean -fdx*)",
"Bash(git * commit --no-verify*)",
"Bash(git * merge --no-ff*)",
"Bash(git * push --force*)",
"Bash(git * push -f*)",
"Bash(git * reset --hard*)",
"Bash(git * restore .*)",
"Bash(git add --all*)",
"Bash(git add -A*)",
"Bash(git add .)",
"Bash(git branch -D *)",
"Bash(git checkout -- *)",
"Bash(git clean -fd*)",
"Bash(git clean -fdx*)",
"Bash(git commit --no-verify*)",
"Bash(git merge --no-ff*)",
"Bash(git push --force*)",
"Bash(git push -f*)",
"Bash(git reset --hard*)",
"Bash(git restore .*)",
"Bash(mkfs*)",
"Bash(rm -rf $HOME*)",
"Bash(rm -rf /*)",
"Bash(rm -rf ~*)",
"Bash(su *)",
"Bash(sudo *)",
"Bash(toj)",
"Bash(toj:*)"
]
}
}
+1
View File
@@ -0,0 +1 @@
.pql/changelog/*.sql merge=union
+2 -2
View File
@@ -6,7 +6,7 @@ on:
- 'api/v*' - 'api/v*'
env: env:
IMAGE_NAME: git.schweitz.internal/jpmschweitzer/webber-api IMAGE_NAME: git.schweitz.net/jpmschweitzer/webber-api
jobs: jobs:
release: release:
@@ -44,7 +44,7 @@ jobs:
- name: Login to Gitea Registry - name: Login to Gitea Registry
uses: docker/login-action@v3 uses: docker/login-action@v3
with: with:
registry: git.schweitz.internal registry: git.schweitz.net
username: ${{ secrets.REGISTRY_USER }} username: ${{ secrets.REGISTRY_USER }}
password: ${{ secrets.REGISTRY_PASSWORD }} password: ${{ secrets.REGISTRY_PASSWORD }}
+13
View File
@@ -0,0 +1,13 @@
#!/usr/bin/env bash
# Trigger only. The checks live in the Makefile, where they can be read, run by
# hand (`make pre-push`), and changed under review.
#
# This file is identical in every repo in this workspace, deliberately: the call
# surface is the same everywhere even though what each gate runs is not, so
# nobody has to read a repo to find out how to check it (D-27).
#
# Enable per clone with: git config core.hooksPath .githooks
# Never bypass with --no-verify. Suppress a specific finding deliberately
# instead, with a reason — see `make pre-push`.
set -euo pipefail
exec make -C "$(git rev-parse --show-toplevel)" pre-push
+13
View File
@@ -75,3 +75,16 @@ webber-sandbox/.current_template
# Ruff cache # Ruff cache
.ruff_cache/ .ruff_cache/
# Claude Code user-specific settings
.claude/settings.local.json
.pql/*
!.pql/changelog/
# pql shims planted by `pql init` into the dir core.hooksPath points at.
# Per-clone: each embeds the absolute path of the pql binary that planted it.
# Only .githooks/pre-push is shared.
.githooks/pre-commit
.githooks/post-merge
.githooks/post-checkout
.githooks/post-rewrite
+11
View File
@@ -0,0 +1,11 @@
-- Changelog format marker, written by pql. Comments only: this file
-- is never executed — Import descends into the per-table directories
-- and does not read the changelog root.
--
-- A changelog carrying no marker is format 1, the shape that existed
-- before formats were versioned. An older format is migrated forward
-- by `pql plan upgrade` (and automatically from the post-merge hook);
-- a newer one is refused rather than replayed under rules this binary
-- does not know. See D-28 and docs/versions.md.
-- pql:changelog_format: 2.0.0
-- pql:written_by: 2.2.0
+139
View File
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
+139
View File
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
+139
View File
@@ -0,0 +1,139 @@
-- Auto-generated by pql init. CREATE TABLE statements
-- for the planning schema; per-table dir keeps the changelog
-- self-describing per D-15. CREATE TABLE IF NOT EXISTS is
-- idempotent so running schema files from each directory in
-- replay order is harmless.
--
-- Importer parses the markers below to detect schema drift
-- between the producing pql version and the local one — a
-- bumped canonical_version means projection rules changed
-- and replay must refuse rather than silently corrupt state.
-- pql:created_by: 2.2.0
-- pql:canonical_version: 2
CREATE TABLE IF NOT EXISTS decisions (
id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('confirmed','question','rejected')),
domain TEXT NOT NULL,
title TEXT NOT NULL,
status TEXT NOT NULL DEFAULT 'active'
CHECK(status IN ('active','superseded','resolved','open')),
date TEXT,
file_path TEXT NOT NULL,
synced_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS decision_refs (
source_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
target_id TEXT NOT NULL REFERENCES decisions(id) ON DELETE CASCADE,
ref_type TEXT NOT NULL
CHECK(ref_type IN ('supersedes','references','resolves','depends_on','amends')),
note TEXT,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (source_id, target_id, ref_type)
);
-- Identity split (D-26): a ticket's stable, collision-proof identity is its
-- record_id (a locally-generated ULID, planning.NewRecordID); the friendly
-- T-NNN label lives in ticket_idmap and may be reconciled. Every structural
-- reference (parent, deps, history, labels) targets record_id, so a label
-- clash never corrupts the graph — only ticket_idmap needs a relabel.
CREATE TABLE IF NOT EXISTS tickets (
record_id TEXT PRIMARY KEY,
type TEXT NOT NULL CHECK(type IN ('initiative','epic','story','task','bug')),
parent_record_id TEXT REFERENCES tickets(record_id),
title TEXT NOT NULL,
description TEXT,
-- No CHECK enumeration: the ticket status vocabulary is per-vault
-- configurable (ticket_statuses in .pql/config.yaml). Validation lives
-- in Go (planning.StatusSet), so adding/renaming statuses needs no
-- schema change. The DEFAULT is a harmless fallback — CreateTicket
-- always inserts the configured default explicitly.
status TEXT NOT NULL DEFAULT 'backlog',
priority TEXT DEFAULT 'medium'
CHECK(priority IN ('critical','high','medium','low')),
assigned_to TEXT,
team TEXT,
decision_ref TEXT REFERENCES decisions(id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
-- ticket_idmap maps a record_id to its current friendly label (T-NNN).
-- ticket_id is intentionally NOT globally unique: two uncoordinated clones
-- can mint the same label, which surfaces as a duplicate-label collision
-- (detected at replay) and is fixed with "pql ticket relabel".
CREATE TABLE IF NOT EXISTS ticket_idmap (
record_id TEXT PRIMARY KEY REFERENCES tickets(record_id),
ticket_id TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_deps (
blocker_record_id TEXT NOT NULL REFERENCES tickets(record_id),
blocked_record_id TEXT NOT NULL REFERENCES tickets(record_id),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (blocker_record_id, blocked_record_id)
);
CREATE TABLE IF NOT EXISTS ticket_history (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
field TEXT NOT NULL,
old_value TEXT,
new_value TEXT,
changed_by TEXT,
changed_at TEXT NOT NULL DEFAULT (datetime('now')),
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT UNIQUE,
canonical_version INTEGER
);
CREATE TABLE IF NOT EXISTS ticket_labels (
ticket_record_id TEXT NOT NULL REFERENCES tickets(record_id),
label TEXT NOT NULL,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
deleted_at TEXT,
hash TEXT,
canonical_version INTEGER,
PRIMARY KEY (ticket_record_id, label)
);
CREATE TABLE IF NOT EXISTS meta (
key TEXT PRIMARY KEY,
value TEXT NOT NULL,
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX IF NOT EXISTS idx_tickets_status ON tickets(status);
CREATE INDEX IF NOT EXISTS idx_tickets_team ON tickets(team);
CREATE INDEX IF NOT EXISTS idx_tickets_decision_ref ON tickets(decision_ref);
CREATE INDEX IF NOT EXISTS idx_tickets_assigned ON tickets(assigned_to);
CREATE INDEX IF NOT EXISTS idx_tickets_parent ON tickets(parent_record_id);
CREATE INDEX IF NOT EXISTS idx_ticket_idmap_label ON ticket_idmap(ticket_id);
CREATE INDEX IF NOT EXISTS idx_decisions_domain ON decisions(domain);
CREATE INDEX IF NOT EXISTS idx_decisions_type ON decisions(type);
CREATE INDEX IF NOT EXISTS idx_decision_refs_target ON decision_refs(target_id);
-245
View File
@@ -1,245 +0,0 @@
# Webber Monorepo - Agent Instructions
> **Start every session by reading this file.**
> This file contains everything you need to work with this codebase efficiently.
## Quick Reference
| Action | Command |
|--------|---------|
| Start API server | `cd webber-api && ./wakeup.sh` |
| View API logs | `tail -f webber-api/logs/server.log` |
| Run API tests | `cd webber-api && .venv/bin/python -m pytest tests/ -v` |
| Check CLI status | `cd webber-cli && .venv/bin/webber-cli status` |
| Load sandbox | `./sandbox.sh load calculator-cli` |
| Explore sandbox | `cd webber-cli && .venv/bin/webber-cli explore "query" -d ../webber-sandbox` |
---
## Repository Structure
```
webber/
├── webber-api/ # FastAPI backend server
│ ├── src/ # API source code
│ ├── tests/ # API tests (pytest)
│ ├── docs/ # Architecture docs, COVERAGE.md
│ ├── logs/ # Runtime logs (server.log)
│ ├── .venv/ # API virtual environment
│ ├── wakeup.sh # Dev server startup script
│ └── AGENTS.md # API-specific development guide
├── webber-cli/ # CLI client
│ ├── webber_cli/ # Python package (underscore!)
│ ├── .venv/ # CLI virtual environment
│ └── README.md # CLI usage guide
├── webber-sandbox/ # Active test project (contents swappable)
│ ├── src/ # Current project source
│ ├── tests/ # Current project tests
│ ├── .venv/ # Sandbox virtual environment
│ └── TASKS.md # Tasks for Webber to complete
├── sandbox-templates/ # Template storage
│ ├── calculator-cli/ # Simple CLI with intentional bugs
│ └── empty/ # Blank starter project
├── sandbox.sh # Sandbox management script
└── AGENTS.md # THIS FILE
```
---
## Development Workflow
### 1. Start the API Server
```bash
cd webber-api
./wakeup.sh
```
- **Port:** 8095 (dev), 8086 (production Docker)
- **Logs:** `webber-api/logs/server.log`
- **Health check:** `curl http://localhost:8095/health`
- **API docs:** http://localhost:8095/docs
To stop: `Ctrl+C` or `pkill -f "uvicorn src.main:app"`
### 2. Run Tests
```bash
# API tests (39 tests)
cd webber-api
.venv/bin/python -m pytest tests/ -v
# With coverage
.venv/bin/python -m pytest tests/ --cov=src
# Single test file
.venv/bin/python -m pytest tests/test_tools.py -v
```
### 3. Use the CLI
```bash
cd webber-cli
# Check API connection
.venv/bin/webber-cli status
# Explore a directory
.venv/bin/webber-cli explore "find all python files" -d ../webber-sandbox
# Interactive chat mode
.venv/bin/webber-cli chat -d ../webber-sandbox
```
**Note:** The API server must be running for CLI commands to work.
---
## Sandbox Management
The sandbox is a swappable test project for functional testing.
### Available Templates
| Template | Description |
|----------|-------------|
| `calculator-cli` | Python CLI with intentional bugs (div-by-zero, missing tests) |
| `empty` | Blank starter project |
### Commands
```bash
# List available templates
./sandbox.sh list
# Load a template (clears sandbox, preserves .venv)
./sandbox.sh load calculator-cli
# Reset to last loaded template
./sandbox.sh reset
# Save current sandbox as new template
./sandbox.sh save my-template
# Check current status
./sandbox.sh status
```
### After Loading a Template
```bash
cd webber-sandbox
source .venv/bin/activate # Create .venv first if missing
pip install -r requirements.txt
# Read the tasks
cat TASKS.md
# Run the project's tests
pytest tests/ -v
```
---
## Testing Webber's Capabilities
### Scenario: Find bugs in calculator-cli
```bash
# 1. Load the template
./sandbox.sh load calculator-cli
# 2. Have Webber explore it
cd webber-cli
.venv/bin/webber-cli explore "find all bugs in the code" -d ../webber-sandbox
# 3. Check TASKS.md for expected bugs
cat ../webber-sandbox/TASKS.md
```
### Known bugs in calculator-cli:
- Division by zero not handled (`operations.py:divide`)
- Invalid operation causes KeyError (`main.py:get_operation`)
- Power function broken for fractional exponents
- Missing tests for divide and power functions
---
## Key Files for Debugging
| File | Purpose |
|------|---------|
| `webber-api/logs/server.log` | API server logs |
| `webber-api/src/domains/agents/explore/prompts.py` | Explore agent system prompts |
| `webber-api/src/domains/agents/explore/agent.py` | Explore agent implementation |
| `webber-api/src/ollama/provider.py` | Ollama integration (sanitizes content:null) |
| `webber-api/docs/COVERAGE.md` | Feature coverage and known issues |
---
## Versioning & Releases
Uses prefixed tags:
- `api/v0.3.0` → Triggers API Docker build
- `cli/v0.1.0` → Triggers CLI build (future)
```bash
# API release
cd webber-api
# Update version in pyproject.toml
git add -A && git commit -m "chore: release api v0.3.0"
git tag api/v0.3.0
git push origin main --tags
```
---
## Troubleshooting
### API server won't start
```bash
# Check if port is in use
lsof -i :8095
# Kill stuck process
pkill -f "uvicorn src.main:app"
```
### CLI can't connect
```bash
# Check API is running
curl http://localhost:8095/health
# Check CLI config
echo $WEBBER_API_URL # Should be http://localhost:8095
```
### Ollama errors
```bash
# Check Ollama is running
curl http://192.168.86.149:11434/api/tags
# Check model is available
curl http://192.168.86.149:11434/api/tags | grep mistral-nemo
```
### Tests failing
```bash
# Run with verbose output
cd webber-api
.venv/bin/python -m pytest tests/ -v --tb=short
```
---
## Known Limitations
1. **Model hallucination** - Mistral Nemo sometimes makes up file contents instead of using tool results
2. **No conversation memory** - CLI chat mode doesn't persist between sessions
3. **No streaming** - Responses appear all at once
See `webber-api/docs/COVERAGE.md` for full feature coverage status.
+6 -51
View File
@@ -1,55 +1,10 @@
# Changelog # Changelog
All notable changes to this project will be documented in this file. This monorepo maintains separate changelogs for each package:
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), - **[webber-api/CHANGELOG.md](webber-api/CHANGELOG.md)** - API server changes
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). - **[webber-cli/CHANGELOG.md](webber-cli/CHANGELOG.md)** - CLI client changes
## [Unreleased] Each package is versioned independently using prefixed git tags:
- `api/vX.Y.Z` for API releases
## [0.2.3] - 2026-01-09 - `cli/vX.Y.Z` for CLI releases
### Added
- Docker healthcheck for container health monitoring
## [0.2.2] - 2026-01-09
### Fixed
- Config parsing for empty environment variables (allowed_paths, cors_*)
- Use `env_parse_none_str=""` to treat empty strings as None
## [0.2.1] - 2026-01-09
### Fixed
- CI/CD pipeline credentials configured
## [0.2.0] - 2026-01-09
### Added
- Reference prompts from claude-code-system-prompts for all agent types
- Detailed documentation for Explore, Plan, and Task agents
- Detailed documentation for File, Shell, and Search tools
- Utility prompts (TodoWrite, AskUserQuestion, conversation summarization, etc.)
- Security review prompt for code analysis
### Changed
- Expanded agents/README.md with capabilities and use cases
- Expanded tools/README.md with parameter details and behaviors
## [0.1.0] - 2026-01-09
### Added
- Initial FastAPI boilerplate setup
- Domain-based project structure (src/domains/, src/shared/)
- BaseController pattern with lazy router instantiation
- Pydantic Settings configuration with env file support
- Logger decorator with temporal benchmarking and trace IDs
- UserProvider singleton for request-scoped context
- Custom exception hierarchy
- Health endpoints (/, /health)
- Placeholder domains for agents (explore, plan, task)
- Placeholder domains for tools (file, shell, search)
- Placeholder domain for auth (tatlock integration)
- CI/CD workflow for Gitea with Docker build and Watchtower deployment
- Dockerfile for containerized deployment
- CVE-checked dependencies (2026-01-09)
+189
View File
@@ -0,0 +1,189 @@
# CLAUDE.md — webber
Local-LLM multi-agent development assistant — "similar to Claude Code but running locally"
(`webber-api/docs/architecture.md`), backed by Ollama via PydanticAI. Logical monorepo, single
`.git`, three subprojects: `webber-api/` (FastAPI server, deployed), `webber-cli/` (Typer CLI
client), `webber-sandbox/` (swappable test project used by `sandbox.sh`, not shipped).
## Ports
| | Port | How |
|---|---|---|
| Local dev | **8095** | `cd webber-api && ./wakeup.sh`, uvicorn `--reload`, logs to `webber-api/logs/server.log` |
| Production | **8086** | container `webber`, confirmed running (`docker ps`) on `docker-dataplane` |
`wakeup.sh` refuses to start if 8095 is already bound — it does not silently pick another
port. Testing `localhost:8086` on the dev box hits the *container*, not your reload server.
## Live contract
`http://localhost:8086/openapi.json` — 10 paths, `version: 1.0.1` (verified 2026-08-09,
matches `webber-api/pyproject.toml` and the live `/health` response). Human docs at
`http://localhost:8086/docs`. Query the live spec rather than inferring routes from source —
`src/domains/router.py` currently has two routers commented out (see Architecture), so a
source read alone will overcount if you don't check whether an include is live.
```
/, /health, /agents/, /agents/run, /agents/stream, /agents/{agent_type},
/conversations/, /conversations/{conversation_id},
/conversations/{conversation_id}/messages, /conversations/{conversation_id}/save
```
## Architecture
Domain-first layout under `webber-api/src/domains/<name>/`. `src/main.py` includes exactly one
router, `src.domains.router.root_router`, which composes the domain routers. A full directory
map lives in `webber-api/docs/architecture.md` — read that before adding a domain rather than
duplicating it here.
**How liveness below was established:** `docker exec webber python3 -c "import src.main; import
sys; print(sorted(m for m in sys.modules if m.startswith('src.')))"` — i.e. importing the real
app inside the running container and reading `sys.modules`, not grepping `main.py`. Re-run that
command to re-check; a grep of imports will miss function-body imports, and this repo has one
that matters.
- **Wired at startup, serving routes:** `src.domains.health`, `src.domains.agents` (router +
`explore`/`plan`/`task` agent packages), `src.domains.conversations`, `src.shared.*`,
`src.ollama`, `src.db` (imported both by `conversations/router.py` at module scope and by
`main.py`'s lifespan shutdown handler).
- **Present in source, explicitly disabled:** `src/domains/router.py` has
`# from src.domains.auth.router import router as auth_router` and the equivalent for
`tools_router` — both commented out with the include calls also commented out. `src/domains/auth/`
is just an empty `__init__.py`. This one *is* dead — the disabling is visible in the same file,
not a matter of tracing an indirect import.
- **The trap: `src/domains/tools/` is not in `sys.modules` right after `import src.main`, but it
is not dead.** `src/domains/agents/{explore,plan,task}/agent.py` each have a method
(e.g. `PlanAgent._register_tools`) that does `from src.domains.agents.plan.tools import
register_plan_tools` **inside the function body**, called every time that agent is
constructed — i.e. on every `/agents/run` or `/agents/stream` request for that agent type.
That nested module then imports the real tool classes from `src.domains.tools.file`,
`.search`, `.shell` at module scope. A static snapshot taken before any request is served
will not show `src.domains.tools` loaded; that is a timing artifact, not evidence it is
unused. Don't delete `src/domains/tools/` on the strength of a `sys.modules` check alone —
confirm by hitting `/agents/run` and re-checking, or by tracing the call graph from each
agent's `_register_tools`.
- **`src/cli/`** is the implementation behind `webber-cli`'s `pyproject.toml` script entry —
it is a separate Typer app, not imported by the API (`src.main`) at all. Its liveness is
"is the CLI installed and invoked", not "is it wired into the API process".
Group new work by domain, not file type — `webber-api/docs/fastapi-best-practices.md` is the
house reference (mirrors the convention used across the other in-house FastAPI services here).
## Database
SQLite by default (`database_url = "sqlite+aiosqlite:///./webber.db"` in
`src/shared/config.py`), not Postgres — confirmed by reading `src/shared/config.py` and
`src/db/database.py` (the latter's docstring says the pattern is ported from core-api, but
the backend differs). Models under `webber-api/src/domains/<name>/models.py` import `Base`
from `src/db/models.py`. No Alembic here (unlike core-api) — did not find a migrations
directory; unverified whether schema changes have any managed migration path at all. Check
before assuming one exists.
## Working here
**Test locally first.** `cd webber-api && ./wakeup.sh` auto-reloads on code changes (not on
`requirements.txt` changes — restart after adding a dependency). Deploy only once a feature
is complete and tested.
```bash
cd webber-api
.venv/bin/python -m pytest tests/ # all tests
.venv/bin/python -m pytest tests/ -v --cov # verbose + coverage
.venv/bin/python -m pytest tests/test_tools.py -v # single file
```
`webber-api/pyproject.toml` declares `[tool.ruff]` and `[tool.mypy]` — unlike core-api, this
repo does have ruff/mypy config; whether either runs in CI is a separate question (see CI below
— it does not).
Copy `webber-api/.env.example` to `webber-api/.env`. Notable defaults: `OLLAMA_URL` points at
`192.168.86.149:11434` (the host's Ollama, not a container), `OLLAMA_AGENT_MODEL=gemma4:e2b`,
optional Tatlock integration via `TATLOCK_API_URL`/`INTERNAL_API_KEY`, optional SearXNG via
`SEARXNG_URL` for the `web_search` tool.
### Sandbox
`webber-sandbox/` is a disposable project used to exercise the agents end-to-end, managed by
`./sandbox.sh {list,load,reset,save,status}` from the repo root. `sandbox-templates/` holds the
reusable templates (`calculator-cli` has intentionally-seeded bugs for testing Explore/Task).
This directory is fixture material, not shipped code — do not treat bugs in it as real bugs.
### CLI
`webber-cli/` is a Typer client (`webber-cli status|chat|explore|sessions|config`) with tab
completion, session persistence (`~/.webber_history`, `~/.webber/config.toml`), and three chat
modes (`plan` read-only, `default`, `auto_accept`). It talks to the API over HTTP — it does not
share a process with `webber-api`. Run it from its own venv: `cd webber-cli && .venv/bin/webber-cli status`.
## CI
`.gitea/workflows/build-api.yml` triggers only on `api/vX.Y.Z` tags: creates a Gitea release,
builds/pushes `git.schweitz.net/jpmschweitzer/webber-api`, then pings Watchtower.
`build-cli.yml` triggers on `cli/vX.Y.Z` tags but is a placeholder — it only echoes a TODO, it
does not build or publish anything. **No test or lint gate runs in CI for either package**
pytest and ruff only run locally or on request. Verify tests pass before tagging.
## Work tracking
Work lives in **pql**, not a markdown TODO or `docs/COVERAGE.md`. **This repo's vault is
standalone** — its tickets and its internal decisions live here in `.pql/` and `governance/`,
and travel with a clone, because `.pql/changelog/` is committed and replayed by the git hooks
(workspace D-15). The databases are gitignored and rebuildable with `pql plan rebuild`.
`pql` is **not** on the non-interactive `PATH` — invoke it as
`/home/jpmschweitzer/.local/bin/pql`. From inside this repo no `--vault` is needed: pql anchors
at the nearest `.git/` ancestor, which is this repo.
```bash
/home/jpmschweitzer/.local/bin/pql ticket list # this repo's open work
/home/jpmschweitzer/.local/bin/pql plan whatsnext # next unblocked item, with context
/home/jpmschweitzer/.local/bin/pql decisions list # this repo's own decisions
```
Stack-level decisions that constrain this service live in the **workspace** vault and need the
flag:
```bash
/home/jpmschweitzer/.local/bin/pql --vault /mnt/media/Projects decisions list --domain webber
```
Note `ticket new --decision D-N` resolves ids within **one** vault, so a ticket here cannot link
to a workspace decision. Cite the id in the ticket body instead.
Do not add a TODO section to a markdown file.
## Git
- **History is linear — no merge commits.** Work on `main`, or a short-lived branch that is
fast-forwarded and deleted. (This repo's `AGENTS.md` previously mandated `feature/...` or
`fix/...` branches for every change and forbade committing to `main` directly — that rule was
retired workspace-wide on 2026-08-08 and does not apply here anymore.)
- **Conventional Commits**: `feat:`, `fix:`, `refactor:`, `docs:`, `chore:`.
- **Atomic commits** — one logical change each.
- **Stage explicitly. Never `git add -A`** — denied by policy; it sweeps in whatever else is
dirty, including secrets.
- Each package versions independently via prefixed tags (`api/vX.Y.Z`, `cli/vX.Y.Z`) and its
own `CHANGELOG.md` (`webber-api/CHANGELOG.md`, `webber-cli/CHANGELOG.md`); the root
`CHANGELOG.md` is just an index pointing at both.
## Releasing (API)
Ask whether a deploy is wanted first — it is not automatic.
1. Bump the version in `webber-api/pyproject.toml`.
2. Move `[Unreleased]` entries into a dated version section in `webber-api/CHANGELOG.md`.
3. Stage the changed files by name, commit, tag `api/vX.Y.Z`, `git push origin main --tags`.
4. Gitea CI (`build-api.yml`) builds and pushes the image on the tag; Watchtower deploys it.
5. Verify: `curl http://192.168.86.149:8086/health`.
CLI releases (`cli/vX.Y.Z`) currently only log a TODO in CI — there is no build/publish step
to trigger yet.
## Known issues (carried over, unverified beyond what's stated)
- **Model hallucination**: the Explore agent's model can hallucinate file contents instead of
using actual tool results, per `webber-api/AGENTS.md` — a mitigation (stronger model or
response validation) was suggested there but not confirmed implemented.
- **Ollama `content: null` workaround**: `src/ollama/provider.py` (confirmed present, loaded at
startup per the `sys.modules` check above) sanitizes `content: null` to `content: ""` for
assistant messages with tool calls, working around an Ollama API limitation.
+62
View File
@@ -0,0 +1,62 @@
# webber — the repo's command surface (D-27).
#
# Multi-component, so this lives at the root and reaches down rather than
# sitting inside webber-api/. The code, tests and tooling config are all in
# webber-api/; sandbox-templates/ and sandbox.sh are the other half of the repo
# and have no build of their own. Keeping one Makefile means `make test` means
# the same thing wherever you are standing (D-27).
#
# Paths resolve here (D-10): `python3` is 3.8 on this host, and a bare `pytest`
# or `ruff` resolves only in a login shell.
API := $(CURDIR)/webber-api
VENV := $(API)/.venv
PYTHON ?= python3.12
.DEFAULT_GOAL := help
.PHONY: help
help: ## Show this help
@grep -hE '^[a-z][a-z0-9_-]*:.*?## ' $(MAKEFILE_LIST) \
| awk 'BEGIN{FS=":.*?## "}{printf " \033[36m%-14s\033[0m %s\n", $$1, $$2}'
.PHONY: setup
setup: ## Create the webber-api venv and install dev dependencies
cd $(API) && $(PYTHON) -m venv .venv && .venv/bin/pip install -e ".[dev]"
.PHONY: test
test: ## Run the webber-api test suite
@test -x $(VENV)/bin/python || { echo "FAIL — no venv; run: make setup"; exit 69; }
cd $(API) && .venv/bin/python -m pytest tests/
.PHONY: lint
lint: ## ruff check over webber-api
@test -x $(VENV)/bin/ruff || { echo "FAIL — ruff not installed; run: make setup"; exit 69; }
cd $(API) && .venv/bin/ruff check .
.PHONY: typecheck
typecheck: ## mypy over webber-api
@test -x $(VENV)/bin/mypy || { echo "FAIL — mypy not installed; run: make setup"; exit 69; }
cd $(API) && .venv/bin/mypy .
# git hands a hook a non-login shell, which never sees ~/.local/bin — where
# gitleaks lands. Without this the scan reports "not installed" on every push,
# which is a check that fails open (D-24).
export PATH := $(HOME)/.local/bin:/usr/local/bin:$(PATH)
.PHONY: secrets
secrets: ## Scan the commits about to be pushed for credentials
@ci/secrets.sh
# The call surface is identical in every repo; what it runs is not.
#
# `secrets` runs first, deliberately: it is the only failure here that cannot be
# undone by fixing it afterwards. A failed lint costs another commit; a pushed
# credential is cached and indexed whether or not it is later deleted.
#
# Some of these fail today, and are left wired anyway. The state was measured
# once and written down in T-56 rather than being worked around here — a gate
# quietly narrowed to what already passes is a gate that reports success for
# doing nothing, which is the failure this workspace keeps rediscovering.
.PHONY: pre-push
pre-push: secrets lint typecheck test ## Everything the pre-push hook runs
+50 -15
View File
@@ -2,6 +2,14 @@
A Claude Code-inspired development assistant powered by local LLMs via Ollama. A Claude Code-inspired development assistant powered by local LLMs via Ollama.
## Features
- **3 Agents** - Explore (read-only), Plan (architecture), Task (orchestrator)
- **8 Tools** - File read/write/edit, glob, grep, bash, web search
- **Conversations** - Multi-turn memory with context summarization
- **Streaming** - Real-time response display
- **Self-hosted** - Runs on your own hardware with Ollama
## Structure ## Structure
This is a monorepo containing three subprojects: This is a monorepo containing three subprojects:
@@ -44,43 +52,70 @@ pip install -e .
webber-cli status webber-cli status
``` ```
### 3. Load a Sandbox Project ### 3. Explore with Webber
```bash ```bash
# From repo root # One-shot exploration
./sandbox.sh list webber-cli explore "find all bugs in the code" -d /path/to/project
./sandbox.sh load calculator-cli
cd webber-sandbox # Interactive chat
python3.12 -m venv .venv webber-cli chat -d /path/to/project
source .venv/bin/activate
pip install -r requirements.txt
``` ```
### 4. Explore with Webber ## Available Tools
| Tool | Description |
|------|-------------|
| `read_file` | Read file contents with line numbers |
| `glob_files` | Find files by pattern |
| `grep_content` | Search file contents with regex |
| `bash_readonly` | Safe bash commands (ls, git status, etc.) |
| `edit_file` | Find-and-replace editing |
| `write_file` | Create/overwrite files |
| `bash` | Full bash with safety controls |
| `web_search` | Search web via SearXNG |
## Agents
| Agent | Purpose | Tools |
|-------|---------|-------|
| **Explore** | Fast codebase navigation, search | Read-only (glob, grep, read, bash_readonly) |
| **Plan** | Design implementation strategies | Read-only (same as Explore) |
| **Task** | Autonomous multi-step execution | All tools + spawn_agent |
## API Endpoints
```bash ```bash
cd webber-cli # Stateless agent execution
webber-cli explore "find all bugs in the code" -d ../webber-sandbox POST /agents/run # Execute agent, get response
POST /agents/stream # Execute with SSE streaming
GET /agents/ # List available agents
# Stateful conversations (multi-turn memory)
POST /conversations/ # Create conversation
GET /conversations/ # List conversations
POST /conversations/{id}/messages # Add message, get agent response
``` ```
## Versioning ## Versioning
This project uses prefixed tags for independent release cycles: This project uses prefixed tags for independent release cycles:
- `api/v0.3.0` - Triggers API Docker build and deployment - `api/v0.4.0` - Triggers API Docker build and deployment
- `cli/v0.1.0` - Triggers CLI installer build (future) - `cli/v0.1.0` - Triggers CLI installer build (future)
## Requirements ## Requirements
- Python 3.12+ - Python 3.12+
- Ollama running with `mistral-nemo:latest` model - Ollama running with `gemma4:e2b` model
- Docker (for production deployment) - Docker (for production deployment)
- SearXNG (optional, for web search)
## Documentation ## Documentation
- `webber-api/AGENTS.md` - API development guidelines - `CLAUDE.md` - Agent development guidelines (repo-wide)
- `webber-api/docs/` - Architecture and coverage docs - `webber-api/docs/COVERAGE.md` - Feature coverage and roadmap
- `webber-api/docs/architecture.md` - System architecture
- `webber-cli/README.md` - CLI usage guide - `webber-cli/README.md` - CLI usage guide
## License ## License
Executable
+50
View File
@@ -0,0 +1,50 @@
#!/usr/bin/env bash
# Secret scan over the commits about to be pushed.
#
# Lives here rather than inside .githooks/pre-push so it can be read, run by
# hand (`make secrets`), and changed under review. A hook is a trigger; it is
# not a home for logic. Identical in every repo in this workspace (D-27).
set -euo pipefail
cd "$(git rev-parse --show-toplevel)"
# A non-login shell — which is what git gives a hook — skips /etc/profile.d
# and never sees ~/.local/bin, where the gitleaks release tarball lands.
# Without this the scan reports "not installed" on every push.
[ -d "$HOME/.local/bin" ] && PATH="$HOME/.local/bin:$PATH"
if ! command -v gitleaks >/dev/null 2>&1; then
echo "FAIL secrets — gitleaks not installed, so this check would be a no-op pretending to pass." >&2
echo " https://github.com/gitleaks/gitleaks/releases → ~/.local/bin/gitleaks" >&2
exit 1
fi
# Scan the outgoing range, not full history. History here carries findings
# that are settled — test fixtures and vendored third-party code — and a gate
# that fails on something unfixable gets bypassed within a week. What matters
# is what is about to leave this machine.
if upstream=$(git rev-parse --abbrev-ref --symbolic-full-name '@{u}' 2>/dev/null); then
range="$upstream..HEAD"
elif git rev-parse --verify --quiet origin/main >/dev/null; then
range="origin/main..HEAD"
else
range=""
fi
if [ -z "$range" ]; then
gitleaks dir . --redact --no-banner --exit-code 1 || {
echo "FAIL secrets — gitleaks found a credential in the working tree." >&2; exit 1; }
exit 0
fi
[ -n "$(git log --oneline "$range" 2>/dev/null)" ] || exit 0
gitleaks git . --log-opts="$range" --redact --no-banner --exit-code 1 >/dev/null 2>&1 || {
echo "FAIL secrets — gitleaks found a credential in the commits being pushed." >&2
echo " inspect (values redacted): gitleaks git . --log-opts=\"$range\" --redact" >&2
echo " then remove and rotate it, or suppress deliberately:" >&2
echo " inline '# gitleaks:allow <reason>'" >&2
echo " or add the fingerprint to .gitleaksignore WITH a reason" >&2
exit 1
}
echo " ok secrets"
+54
View File
@@ -0,0 +1,54 @@
# Decisions, Questions, Rejected
This directory holds structured planning records that pql parses
into pql.db. Each record is a `### [DQR]-N: Title` heading inside
a markdown file. Files live in three per-type subdirectories:
- `decisions/<domain>.md` — confirmed design decisions
- `questions/<domain>.md` — open questions that may resolve into
decisions or rejected proposals
- `rejected/<domain>.md` — rejected proposals (kept for the audit
trail)
The parser infers domain from the filename stem and record type
from the parent subdirectory.
D-records that propose implementation work link to `initiative`-type
tickets via `decision_ref`. Run `pql decisions show <id>
--with-tickets` to inspect implementation status.
## Recommended domains
Start with this canonical set; create files as records land in
each domain:
- **architecture** — structural commitments (storage, layering,
languages, libraries)
- **process** — team workflow (commits, branches, releases, reviews)
- **design** — user-facing surface (UX, UI, public APIs)
- **coding-conventions** — team-internal code shape (style, lint,
file layout)
- **testing** — quality strategy (coverage, layers, gates)
You might also want, project-permitting:
- `accessibility` — if you ship user-facing software
- `security` — if you handle user data or network surfaces
- `licensing` — if you release open-source or commercial
- `documentation` — if user-docs are non-trivial
- `deployment` — if shipping is non-trivial
- `performance` — if you have perf budgets / SLOs
<!-- pql:records (auto-generated; do not edit manually) -->
## Decisions
- _(none)_
## Open questions
- _(none)_
## Rejected
- _(none)_
+5 -2
View File
@@ -14,13 +14,16 @@ CORS_ORIGINS=["http://localhost:3000","http://localhost:8080"]
# LLM - Ollama (tower-of-joy) # LLM - Ollama (tower-of-joy)
OLLAMA_URL=http://192.168.86.149:11434 OLLAMA_URL=http://192.168.86.149:11434
OLLAMA_AGENT_MODEL=mistral-nemo-large:latest OLLAMA_AGENT_MODEL=gemma4:e2b
OLLAMA_EMBED_MODEL=nomic-embed-text:latest OLLAMA_EMBED_MODEL=nomic-embed-text:latest
# Auth - Tatlock integration (optional) # Auth - Tatlock integration (optional)
# TATLOCK_API_URL=http://192.168.86.149:8000 # TATLOCK_API_URL=http://tatlock:8000
# INTERNAL_API_KEY=your-internal-key # INTERNAL_API_KEY=your-internal-key
# Web search - SearXNG (container name on docker-dataplane; internal port 8080)
# SEARXNG_URL=http://searxng:8080
# Tool execution # Tool execution
TOOL_TIMEOUT_SECONDS=120 TOOL_TIMEOUT_SECONDS=120
SANDBOX_ENABLED=true SANDBOX_ENABLED=true
-128
View File
@@ -1,128 +0,0 @@
# AGENTS.md
> **Start every session by reading this file.**
> This file outlines the operational protocols, coding standards, and architectural decisions for this FastAPI project.
## 1. Agent Operational Protocols
### 🧠 Work Patterns (Plan-Act-Reflect)
* **Plan:** Before writing code, briefly outline your plan. Identify which files you will touch and what the side effects might be.
* **Act:** Execute the changes in small, atomic steps.
* **Reflect:** After coding, verify your work. Did you break existing tests? Did you add new tests?
### 🛡️ Git Discipline
* **ALWAYS add the relevant tests for the added code** Make sure to keep the test coverage up as we go, and run tests before commiting.
* **NEVER commit to `main` or `master` directly.** Always create a feature branch: `feature/your-feature-name` or `fix/issue-description`.
* **Commit Messages:** Use the [Conventional Commits](https://www.conventionalcommits.org/) format.
* `feat: add user login endpoint`
* `fix: resolve database connection timeout`
* `refactor: split monolith dependency file`
* **Atomic Commits:** Keep commits small. One logical change = one commit.
### 📝 Changelog Maintenance
* **Update `CHANGELOG.md`** with every user-facing change.
* Format: `## [Unreleased] - YYYY-MM-DD` followed by `### Added`, `### Changed`, or `### Fixed`.
### 🚀 Release Flow
When changes are ready for deployment:
1. **Ask user if deploy cycle is desired **
2. **Update version** in `pyproject.toml`:
- Bug fixes: bump patch version (1.8.3 → 1.8.4)
- New features: bump minor version (1.8.4 → 1.9.0)
3. **Update CHANGELOG.md**:
- Move items from `[Unreleased]` to new version section
- Add release date: `## [1.8.4] - 2025-12-16`
4. **Commit and tag**:
```bash
git add -A
git commit -m "fix: description of changes"
git tag v1.8.4
git push origin main --tags
```
5. **CI/CD triggers automatically**:
- Gitea CI builds Docker image on new version tag (starts with "v")
- Watchtower pulls and deploys to production
- Verify deployment: `curl http://192.168.86.149:8086/health`
---
### 🧪 Local Development Setup
* **Always test locally first** before committing and deploying. The build-deploy loop is slow.
* **Only deploy** when a phase or feature is complete and tested locally
* **Environment**: Copy `.env.example` to `.env` and configure for your local setup
#### ⚠️ CRITICAL: Starting the Local Server
**ALWAYS use `./wakeup.sh` to start the local server. NEVER use raw uvicorn commands.**
```bash
./wakeup.sh
```
The wakeup script provides:
- **Port conflict detection** - Warns if port 8086 is already in use
- **Virtual environment activation** - Ensures correct Python environment
- **Centralized logging** - All logs written to `logs/server.log` for easy tailing
- **Auto-reload** - Code changes picked up automatically (except requirements.txt changes)
- **Consistent configuration** - Same startup every time
To monitor logs in another terminal:
```bash
tail -f logs/server.log
```
To stop the server: Press `Ctrl+C`
To kill a stuck server:
```bash
pkill -f "uvicorn src.main:app"
# or
kill $(lsof -t -i:8086)
```
#### Testing
**Test REST endpoints** against `http://localhost:8086`:
```bash
curl http://localhost:8086/health
curl http://localhost:8086/
curl http://localhost:8086/docs # Swagger UI
```
**Running tests**: Always use the venv explicitly to avoid environment mismatches:
```bash
.venv/bin/python -m pytest tests/ # All tests
.venv/bin/python -m pytest tests/ -v # Verbose output
.venv/bin/python -m pytest tests/ --cov # With coverage
```
---
## 1.5 Known Issues & Future Improvements
### Explore Agent
- **Model Hallucination**: Mistral Nemo sometimes hallucinates file contents instead of using actual tool results. Consider using a more capable model (codestral, qwen2.5-coder) or adding response validation.
- **Ollama Provider**: We use a custom `WebberOllamaProvider` (ported from tatlock) that sanitizes `content: null` to `content: ""` for assistant messages with tool calls. This works around an Ollama API limitation.
- **Gitignore Support**: ✅ Fixed - The filesystem tools now honor `.gitignore` patterns and default ignores (`.venv/`, `__pycache__/`, `node_modules/`, etc.).
---
## 2. FastAPI Architecture & Best Practices
*Reference: [FastAPI Best Practices](https://github.com/zhanymkanov/fastapi-best-practices)*
### 📂 Project Structure (Directory-based, NOT File-type based)
Do **not** group files by type (e.g., one huge `routers` folder). Group by **domain/module** inside a `src/` directory.
**Correct Structure:**
```text
to be determined
+195
View File
@@ -0,0 +1,195 @@
# Changelog - Webber API
All notable changes to the Webber API will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
## [1.1.0] - 2026-08-11
### Added
- `webber version` command. It was implemented and imported but never registered, so the
subcommand did not exist; the fuller output includes the resolved Ollama URL and model.
### Fixed
- The Ollama `content: null` sanitiser raised `AttributeError` on first use. It looked up the
parent client's chat getter via `.fget`, and openai now exposes `chat` as a
`cached_property`. Any agent request would have failed before reaching the model.
### Changed
- Default `OLLAMA_AGENT_MODEL` is now `gemma4:e2b` instead of `mistral-nemo-large:latest`, so a deployment without an explicit override no longer exhausts shared GPU memory
- `BaseAgent` is generic over its context type and `run_stream` is typed as
`AsyncIterator[str | StreamEvent]`, matching what callers already receive.
## [1.0.1] - 2026-07-19
### Changed
- Default `TATLOCK_API_URL` and `SEARXNG_URL` now use docker container names (`http://tatlock:8000`, `http://searxng:8080`) instead of host IP:port, for container-to-container traffic on the docker-dataplane network
- CI workflow now pushes Docker images via the `git.schweitz.net` registry route
## [1.0.0] - 2026-01-15
### Added
- Event-based streaming for task agent (`StreamEvent` objects instead of raw text)
- New `tools_streaming.py` with all tools emitting structured events
- Event types: `tool_start`, `tool_done`, `thinking`, `response`, `error`, `done`
- Retry logic when LLM responds without calling tools (max 2 retries)
- Tracks `tools_called` counter on TaskContext
- Stronger retry prompt forces tool use
- Working directory context injected into all agent prompts
### Changed
- Hardened system prompts to enforce tool use before responding
- Added "CRITICAL RULE" section requiring tool calls first
- Made "MANDATORY WORKFLOW" more emphatic
- Updated explore and plan agents with `_build_prompt_with_context()` method
### Fixed
- Agent path hallucination - now explicitly communicates working directory to LLM
### Note
- Project paused: Local LLMs (Mistral Nemo 12B on available hardware) are not capable enough for reliable agentic tool use. Models frequently hallucinate responses instead of calling tools, even with prompt hardening and retry logic. Would require larger models (70B+) or cloud API integration to continue.
## [0.4.2] - 2026-01-11
### Added
- Retry logic for transient failures with exponential backoff
- `src/shared/retry.py` - `@with_retry` decorator and `retry_async()` function
- Retries on: timeout, connection errors, HTTP 429/5xx
- Configurable: `RETRY_MAX_ATTEMPTS`, `RETRY_BASE_DELAY`, `RETRY_MAX_DELAY`
- Web search tool now automatically retries on network failures
- 29 retry tests (205 total tests passing)
## [0.4.1] - 2026-01-11
### Fixed
- Replace `litellm` with `tiktoken` for token counting (dependency conflict with pydantic-ai)
- Update documentation (README.md, architecture.md) with conversation layer info
## [0.4.0] - 2026-01-11
### Added
- Conversation persistence layer with SQLAlchemy async
- Database models: `Conversation`, `Message` with UUID primary keys
- SQLite (dev) and PostgreSQL (prod) support via async engines
- Lazy database initialization pattern
- Context management infrastructure
- Token counting utilities using `tiktoken`
- Context summarization at 80% token threshold
- XML-tagged context prompt building for agent injection
- REST API for multi-turn conversations
- `POST /conversations/` - Create new conversation
- `GET /conversations/` - List conversations
- `GET /conversations/{id}` - Get conversation with history
- `POST /conversations/{id}/messages` - Add message (triggers agent)
- `DELETE /conversations/{id}` - Delete conversation
- New dependencies: `sqlalchemy[asyncio]~=2.0.36`, `aiosqlite~=0.21.0`, `tiktoken>=0.12.0`
- Config settings: `database_url`, `summarization_threshold`, `keep_recent_messages`
- 19 conversation tests, 6 token counting tests (176 total tests passing)
### Changed
- Updated COVERAGE.md to ~80% complete
- Quieter pytest output (`-q --tb=short` instead of `-v`)
## [0.3.4] - 2026-01-11
### Added
- Task Agent - Full orchestrator for autonomous multi-step task execution
- Has ALL tools: read, write, edit, bash (full), web_search
- New `spawn_agent` tool to launch sub-agents (Explore, Plan) for focused work
- Recursion prevention: cannot spawn nested Task agents
- 22 unit tests for registration, tools, spawn_agent, and API
- Complete agent hierarchy: Explore (read-only) → Plan (read-only) → Task (orchestrator)
## [0.3.3] - 2026-01-11
### Added
- Plan Agent - READ-ONLY software architect that designs implementation strategies
- Uses only read-only tools: `read_file`, `glob_files`, `grep_content`, `bash_readonly`
- Creates step-by-step implementation plans with critical files list
- 15 unit tests for registration, tools, and API
- Web search summarizer added to roadmap (future feature)
### Changed
- Updated COVERAGE.md to ~70% complete
## [0.3.2] - 2026-01-11
### Added
- Mandatory release procedure documentation in AGENTS.md
## [0.3.1] - 2026-01-11
### Added
- Integration test infrastructure with pytest markers (integration, e2e, slow)
- 10 LLM integration tests (requires Ollama)
- 12 E2E API tests (requires running server)
- Command line options: `--run-integration`, `--run-e2e`, `--ollama-url`, `--api-url`
- Sample project fixtures for testing
- 14 security tests (path traversal, command injection, input validation)
- Helper functions: `assert_contains_any`, `assert_contains_all`
### Changed
- Updated COVERAGE.md to ~65% complete
## [0.3.0] - 2026-01-10
### Added
- Explore agent with PydanticAI tool calling and Mistral Nemo
- Coding tools: `edit_file`, `write_file`, `bash` (full)
- Web search tool using SearXNG integration
- Streaming responses via SSE
- Sanitized Ollama provider (fixes `content: null` issue)
### Changed
- Reorganized into monorepo structure (webber-api/, webber-cli/, webber-sandbox/)
- Added ruff linter and fixed mypy errors
## [0.2.3] - 2026-01-09
### Added
- Docker healthcheck for container health monitoring
## [0.2.2] - 2026-01-09
### Fixed
- Config parsing for empty environment variables (allowed_paths, cors_*)
- Use `env_parse_none_str=""` to treat empty strings as None
## [0.2.1] - 2026-01-09
### Fixed
- CI/CD pipeline credentials configured
## [0.2.0] - 2026-01-09
### Added
- Reference prompts from claude-code-system-prompts for all agent types
- Detailed documentation for Explore, Plan, and Task agents
- Detailed documentation for File, Shell, and Search tools
- Utility prompts (TodoWrite, AskUserQuestion, conversation summarization, etc.)
- Security review prompt for code analysis
### Changed
- Expanded agents/README.md with capabilities and use cases
- Expanded tools/README.md with parameter details and behaviors
## [0.1.0] - 2026-01-09
### Added
- Initial FastAPI boilerplate setup
- Domain-based project structure (src/domains/, src/shared/)
- BaseController pattern with lazy router instantiation
- Pydantic Settings configuration with env file support
- Logger decorator with temporal benchmarking and trace IDs
- UserProvider singleton for request-scoped context
- Custom exception hierarchy
- Health endpoints (/, /health)
- Placeholder domains for agents (explore, plan, task)
- Placeholder domains for tools (file, shell, search)
- Placeholder domain for auth (tatlock integration)
- CI/CD workflow for Gitea with Docker build and Watchtower deployment
- Dockerfile for containerized deployment
- CVE-checked dependencies (2026-01-09)
+147 -57
View File
@@ -2,9 +2,9 @@
> Tracking progress towards Claude Code-like functionality > Tracking progress towards Claude Code-like functionality
## Current Status: ~40% Complete ## Current Status: ~85% Complete
Last updated: 2026-01-10 Last updated: 2026-01-14
--- ---
@@ -20,9 +20,13 @@ Last updated: 2026-01-10
| `GlobFilesTool` | ✅ | Pattern matching, sorted by mtime | | `GlobFilesTool` | ✅ | Pattern matching, sorted by mtime |
| `GrepContentTool` | ✅ | Regex search with context lines | | `GrepContentTool` | ✅ | Regex search with context lines |
| `BashReadOnlyTool` | ✅ | Allowlist-based command filtering | | `BashReadOnlyTool` | ✅ | Allowlist-based command filtering |
| `EditFileTool` | ✅ | Find-and-replace with unique match validation |
| `WriteFileTool` | ✅ | Create/overwrite files with size limits |
| `BashTool` (full) | ✅ | Write-enabled shell with safety controls |
| `WebSearchTool` | ✅ | SearXNG integration for web search |
| Path validation | ✅ | `allowed_paths` restriction | | Path validation | ✅ | `allowed_paths` restriction |
**Status:** Tools now honor `.gitignore` patterns and default ignores (`.venv/`, `__pycache__/`, etc.) **Status:** Tools honor `.gitignore` patterns and default ignores (`.venv/`, `__pycache__/`, etc.)
### Phase 2: Explore Agent ✅ Complete ### Phase 2: Explore Agent ✅ Complete
@@ -34,48 +38,70 @@ Last updated: 2026-01-10
| System prompts | ✅ | Mistral-optimized with tool examples | | System prompts | ✅ | Mistral-optimized with tool examples |
| Tool registration | ✅ | `@agent.tool` decorator pattern | | Tool registration | ✅ | `@agent.tool` decorator pattern |
| Sanitized Ollama provider | ✅ | Fixes `content: null` issue | | Sanitized Ollama provider | ✅ | Fixes `content: null` issue |
| Streaming support | ✅ | `run_stream()` method with SSE |
**Available tools:** `read_file`, `glob_files`, `grep_content`, `bash_readonly`, `edit_file`, `write_file`, `bash`, `web_search`
**Gap:** Mistral Nemo sometimes hallucinates instead of using tool results. **Gap:** Mistral Nemo sometimes hallucinates instead of using tool results.
### Phase 2b: Plan Agent ✅ Complete
| Component | Status | Notes |
|-----------|--------|-------|
| `PlanAgentImpl` | ✅ | READ-ONLY software architect agent |
| System prompts | ✅ | Architecture-focused with tool examples |
| Tool registration | ✅ | Only read-only tools (4 tools) |
| Streaming support | ✅ | `run_stream()` method with SSE |
| Unit tests | ✅ | 15 tests for registration, tools, API |
**Available tools:** `read_file`, `glob_files`, `grep_content`, `bash_readonly` (read-only only)
**Purpose:** Design implementation strategies before coding - explores codebase and creates step-by-step plans.
### Phase 3: CLI Foundation ✅ Complete ### Phase 3: CLI Foundation ✅ Complete
| Component | Status | Notes | | Component | Status | Notes |
|-----------|--------|-------| |-----------|--------|-------|
| Typer + Rich setup | ✅ | Both `src/cli` and standalone `cli/` | | Typer + Rich setup | ✅ | Standalone `webber-cli/` package |
| `webber --version` | ✅ | Shows version from pyproject.toml | | `webber-cli --version` | ✅ | Shows version from pyproject.toml |
| Console theming | ✅ | Centralized color palette | | Console theming | ✅ | Centralized color palette |
| Markdown rendering | ✅ | Rich markdown output | | Markdown rendering | ✅ | Rich markdown output |
| Streaming display | ✅ | Real-time token output with `--stream` flag |
### Phase 4: Agentic Loop ⚠️ Partial ### Phase 4: Agentic Loop ✅ Complete
| Component | Status | Notes | | Component | Status | Notes |
|-----------|--------|-------| |-----------|--------|-------|
| `webber chat` command | ✅ | Interactive mode works | | `webber-cli chat` command | ✅ | Interactive mode with streaming |
| `webber explore` command | ✅ | One-shot query works | | `webber-cli explore` command | ✅ | One-shot query with streaming |
| `SessionState` dataclass | ✅ | Basic context tracking | | `SessionState` dataclass | ✅ | Basic context tracking |
| `AgenticLoop` class | ⚠️ | Basic implementation, not fully utilized | | `AgenticLoop` class | | Basic implementation |
| Conversation history | | Not persisted between turns in CLI | | Conversation persistence | | SQLAlchemy async with SQLite/PostgreSQL |
| Context management | | No token counting or summarization | | Context summarization | | Token counting (litellm) + auto-summarization |
| Conversation API | ✅ | `/conversations/` REST endpoints |
**Database:** SQLite (dev) or PostgreSQL (prod), async via SQLAlchemy 2.0
### Phase 5: REST API ✅ Complete ### Phase 5: REST API ✅ Complete
| Component | Status | Notes | | Component | Status | Notes |
|-----------|--------|-------| |-----------|--------|-------|
| `POST /agents/run` | ✅ | Execute agent with prompt | | `POST /agents/run` | ✅ | Execute agent with prompt |
| `POST /agents/stream` | ✅ | SSE streaming responses |
| `GET /agents/` | ✅ | List available agents | | `GET /agents/` | ✅ | List available agents |
| `GET /agents/{name}` | ✅ | Get agent info | | `GET /agents/{name}` | ✅ | Get agent info |
| Request/response schemas | ✅ | Pydantic models | | Request/response schemas | ✅ | Pydantic models |
### Phase 6: Polish & Tests ⚠️ Partial ### Phase 6: Polish & Tests ✅ Complete
| Component | Status | Notes | | Component | Status | Notes |
|-----------|--------|-------| |-----------|--------|-------|
| Tool unit tests | ✅ | 17 tests covering all tools | | Tool unit tests | ✅ | 109 tests total |
| API endpoint tests | ✅ | 5 tests for agent routes | | API endpoint tests | ✅ | 11 tests for agent routes |
| Health check tests | ✅ | 2 tests | | Health check tests | ✅ | 2 tests |
| Integration tests | | No real LLM integration tests | | Security tests | | 14 tests for path traversal, injection |
| CLI E2E tests | | Not implemented | | Integration tests | | 10 tests with real LLM (requires Ollama) |
| Streaming responses | | Not implemented | | E2E tests | | 12 tests against running API server |
--- ---
@@ -85,26 +111,24 @@ Last updated: 2026-01-10
| Feature | Category | Description | Complexity | | Feature | Category | Description | Complexity |
|---------|----------|-------------|------------| |---------|----------|-------------|------------|
| **Write tool** | Tools | Create new files | Medium | | ~~**Plan Agent**~~ | Agents | ✅ Design implementation approaches | High |
| **Edit tool** | Tools | old_string/new_string pattern like Claude | Medium | | ~~**Task Agent**~~ | Agents | ✅ Autonomous multi-step execution | High |
| **Full Bash tool** | Tools | Write-enabled shell for Task agent | Medium | | ~~**Context summarization**~~ | Infrastructure | ✅ Token counting + auto-summarization | High |
| **Plan Agent** | Agents | Design implementation approaches | High | | ~~**Conversation persistence**~~ | Infrastructure | ✅ SQLAlchemy async database layer | Medium |
| **Task Agent** | Agents | Autonomous multi-step execution | High |
| **Context summarization** | Infrastructure | Compress history at token limit | High |
| **Conversation persistence** | CLI | Multi-turn memory in chat mode | Medium |
### Medium Priority ### Medium Priority
| Feature | Category | Description | Complexity | | Feature | Category | Description | Complexity |
|---------|----------|-------------|------------| |---------|----------|-------------|------------|
| **Streaming responses** | CLI | Real-time token display | Medium | | **Web search summarizer** | Tools | Agent to extract core content from web pages (remove nav, footers, etc.) and preserve relevant links for nested fetching | Medium |
| **Web search tool** | Tools | External search API integration | Medium |
| **Tool result caching** | Infrastructure | Cache file reads for performance | Low | | **Tool result caching** | Infrastructure | Cache file reads for performance | Low |
| **Session persistence** | CLI | Save/resume conversations | Medium | | ~~**Session persistence**~~ | CLI | Save/resume conversations via `sessions` and `chat --resume` | Medium |
| **Todo tracking** | CLI | Built-in task list (`/todo`) | Medium | | **Todo tracking** | CLI | Built-in task list (`/todo`) | Medium |
| **Git integration** | CLI | Auto-commit, branch management | Medium | | **Git integration** | CLI | Auto-commit, branch management | Medium |
| **Agent handoff** | Orchestration | ExplorePlan → Task workflow | High | | ~~**Agent handoff**~~ | Orchestration | ✅ Task agent is main agent, spawns Explore/Plan as needed (Claude Code pattern) | High |
| **Retry logic** | Infrastructure | Auto-retry on tool failures | Low | | ~~**Retry logic**~~ | Infrastructure | Auto-retry with exponential backoff | Low |
| ~~**Permission modes**~~ | CLI | ✅ default/plan/auto_accept modes controlling tool access | Medium |
| ~~**CLI shell features**~~ | CLI | ✅ prompt_toolkit: history, tab completion, auto-suggest | Low |
### Low Priority ### Low Priority
@@ -112,22 +136,61 @@ Last updated: 2026-01-10
|---------|----------|-------------|------------| |---------|----------|-------------|------------|
| **Notebook editing** | Tools | Jupyter cell manipulation | Medium | | **Notebook editing** | Tools | Jupyter cell manipulation | Medium |
| **MCP support** | Infrastructure | Model Context Protocol | High | | **MCP support** | Infrastructure | Model Context Protocol | High |
| **Config file** | CLI | `~/.webber/config.toml` | Low | | ~~**Config file**~~ | CLI | `~/.webber/config.toml` with `config` command | Low |
| **IDE integration** | CLI | VS Code extension | High | | **IDE integration** | CLI | VS Code extension | High |
| **Parallel agents** | Orchestration | Concurrent agent execution | High | | **Parallel agents** | Orchestration | Concurrent agent execution | High |
| **Agent memory** | Orchestration | Shared context between agents | Medium | | **Agent memory** | Orchestration | Shared context between agents | Medium |
--- ---
## Testing Coverage Gaps ## Testing Coverage
| Area | Current | Target | Gap | | Area | Current | Target | Status |
|------|---------|--------|-----| |------|---------|--------|--------|
| Tool unit tests | 17 | 17 | ✅ | | Tool unit tests | 109 | 109 | ✅ |
| API tests | 5 | 10 | Need error handling, edge cases | | API tests | 11 | 11 | |
| Integration tests | 0 | 5 | Agent + real LLM tests | | Plan agent tests | 15 | 15 | |
| CLI E2E tests | 0 | 10 | Full workflow tests | | Task agent tests | 15 | 15 | |
| Security tests | 0 | 5 | Path traversal, injection | | Conversation tests | 22 | 22 | |
| Token tests | 6 | 6 | ✅ |
| Retry tests | 29 | 29 | ✅ |
| Security tests | 14 | 14 | ✅ |
| Integration tests | 10 | 10 | ✅ Agent + real LLM |
| E2E tests | 12 | 12 | ✅ Full API workflow |
**Total: 208 tests passing**
**Test breakdown:**
- Read/Glob/Grep tools: 17 tests
- Edit/Write tools: 22 tests
- Bash tools: 22 tests
- Web search: 10 tests
- Gitignore filtering: 10 tests
- API endpoints: 11 tests
- Plan agent: 15 tests
- Task agent: 15 tests
- Conversations: 22 tests
- Tokens: 6 tests
- Retry: 29 tests
- Security: 14 tests
- Health checks: 2 tests
- Integration (LLM): 10 tests
- E2E (API): 12 tests
**Running tests:**
```bash
# Unit tests only (default)
pytest tests/
# Include integration tests (requires Ollama)
pytest tests/ --run-integration
# Include E2E tests (requires running API server)
pytest tests/ --run-e2e
# All tests
pytest tests/ --run-integration --run-e2e
```
--- ---
@@ -135,11 +198,9 @@ Last updated: 2026-01-10
1. **Model hallucination** - Mistral Nemo sometimes makes up file contents instead of using actual tool results. 1. **Model hallucination** - Mistral Nemo sometimes makes up file contents instead of using actual tool results.
2. **No conversation memory** - CLI chat mode doesn't persist context between sessions. 2. **Temperature setting** - Changed from 0.0 to 0.3 for Mistral Nemo compatibility, may affect determinism.
3. **No streaming** - Responses appear all at once, no real-time token display. 3. **SQLAlchemy deprecation** - `datetime.utcnow()` deprecation warning from SQLAlchemy.
4. **Temperature setting** - Changed from 0.0 to 0.3 for Mistral Nemo compatibility, may affect determinism.
--- ---
@@ -147,21 +208,13 @@ Last updated: 2026-01-10
| Decision | Choice | Rationale | | Decision | Choice | Rationale |
|----------|--------|-----------| |----------|--------|-----------|
| Separate CLI package | `cli/` at root | Can be extracted as standalone client | | Monorepo structure | `webber-api/`, `webber-cli/` | Separate packages, shared root |
| Sanitized Ollama provider | Custom wrapper | Fixes PydanticAI + Ollama `content: null` bug | | Sanitized Ollama provider | Custom wrapper | Fixes PydanticAI + Ollama `content: null` bug |
| Dev port 8095 | Separate from prod 8086 | Avoid conflicts with Docker deployment | | Dev port 8095 | Separate from prod 8086 | Avoid conflicts with Docker deployment |
| Tool choice "required" | Force tool use | Mistral Nemo needs explicit instruction | | Tool choice "required" | Force tool use | Mistral Nemo needs explicit instruction |
| Temperature 0.3 | Mistral recommendation | 0.0 caused issues with Nemo | | Temperature 0.3 | Mistral recommendation | 0.0 caused issues with Nemo |
| SearXNG for search | Self-hosted | Privacy, no API keys needed |
--- | SSE for streaming | Server-Sent Events | Simple, well-supported |
## Estimated Effort to Full Parity
| Milestone | Effort | Features |
|-----------|--------|----------|
| **MVP (current)** | Done | Explore agent, basic CLI, REST API |
| **Usable daily driver** | 2-3 weeks | Write/Edit tools, Plan agent, git integration |
| **Claude Code parity** | 2-3 months | Task agent, streaming, MCP, IDE integration |
--- ---
@@ -169,12 +222,13 @@ Last updated: 2026-01-10
```bash ```bash
# Start dev server # Start dev server
./wakeup.sh cd webber-api && ./wakeup.sh
# CLI commands # CLI commands (from webber-cli/)
.venv/bin/webber-cli status # Check API connection .venv/bin/webber-cli status # Check API connection
.venv/bin/webber-cli explore "find tests" # One-shot exploration .venv/bin/webber-cli chat # Interactive mode (Task agent, full tools)
.venv/bin/webber-cli chat # Interactive mode .venv/bin/webber-cli chat --mode plan # Read-only mode (safe exploration)
.venv/bin/webber-cli chat --mode auto_accept # No approval prompts (use with caution)
# API endpoints # API endpoints
curl http://localhost:8095/health curl http://localhost:8095/health
@@ -182,4 +236,40 @@ curl http://localhost:8095/agents/
curl -X POST http://localhost:8095/agents/run \ curl -X POST http://localhost:8095/agents/run \
-H "Content-Type: application/json" \ -H "Content-Type: application/json" \
-d '{"agent_type":"explore","prompt":"list python files","working_dir":"."}' -d '{"agent_type":"explore","prompt":"list python files","working_dir":"."}'
# Plan agent (read-only, creates implementation plans)
curl -X POST http://localhost:8095/agents/run \
-H "Content-Type: application/json" \
-d '{"agent_type":"plan","prompt":"plan how to add user auth","working_dir":"."}'
# Streaming endpoint
curl -N http://localhost:8095/agents/stream \
-H "Content-Type: application/json" \
-d '{"agent_type":"explore","prompt":"find config files","working_dir":"."}'
# Conversation API (stateful multi-turn)
curl -X POST http://localhost:8095/conversations/ \
-H "Content-Type: application/json" \
-H "X-API-Key: dev-key" \
-d '{"agent_type":"explore","working_dir":"."}'
curl -X POST http://localhost:8095/conversations/{id}/messages \
-H "Content-Type: application/json" \
-H "X-API-Key: dev-key" \
-d '{"content":"find all Python files"}'
``` ```
---
## Tools Available
| Tool | Type | Description |
|------|------|-------------|
| `read_file` | Read | Read file contents with line numbers |
| `glob_files` | Read | Find files by pattern |
| `grep_content` | Read | Search file contents with regex |
| `bash_readonly` | Read | Safe bash commands (ls, git status, etc.) |
| `edit_file` | Write | Find-and-replace editing |
| `write_file` | Write | Create/overwrite files |
| `bash` | Write | Full bash with safety controls |
| `web_search` | External | Search web via SearXNG |
+73 -5
View File
@@ -13,7 +13,7 @@ Webber is a FastAPI-based agent orchestration service that provides:
**Port:** 8086 **Port:** 8086
**Runtime:** Python 3.12, FastAPI, Uvicorn **Runtime:** Python 3.12, FastAPI, Uvicorn
**Agent Framework:** PydanticAI **Agent Framework:** PydanticAI
**Default LLM:** Ollama with mistral-nemo-large:latest **Default LLM:** Ollama with gemma4:e2b
--- ---
@@ -24,18 +24,30 @@ webber/
├── src/ ├── src/
│ ├── main.py # App entry point (NO routes) │ ├── main.py # App entry point (NO routes)
│ │ │ │
│ ├── db/ # Database layer
│ │ ├── __init__.py # Exports: Database, get_database, get_session
│ │ ├── database.py # SQLAlchemy async engine, session factory
│ │ └── models.py # Base declarative model
│ │
│ ├── shared/ # Cross-cutting concerns │ ├── shared/ # Cross-cutting concerns
│ │ ├── base.py # BaseController, BaseSchema │ │ ├── base.py # BaseController, BaseSchema
│ │ ├── config.py # Pydantic Settings │ │ ├── config.py # Pydantic Settings
│ │ ├── logging.py # @logged decorator, trace_span │ │ ├── logging.py # @logged decorator, trace_span
│ │ ├── exceptions.py # Custom exception hierarchy │ │ ├── exceptions.py # Custom exception hierarchy
│ │ ├── auth.py # API key validation │ │ ├── auth.py # API key validation
│ │ ── context.py # UserProvider singleton │ │ ── context.py # UserProvider singleton
│ │ └── tokens.py # Token counting utilities (litellm)
│ │ │ │
│ └── domains/ # Feature domains │ └── domains/ # Feature domains
│ ├── router.py # Root router (composes all) │ ├── router.py # Root router (composes all)
│ ├── health/ # Health endpoints │ ├── health/ # Health endpoints
│ ├── auth/ # Authentication │ ├── auth/ # Authentication
│ ├── conversations/ # Multi-turn conversation memory
│ │ ├── models.py # Conversation, Message SQLAlchemy models
│ │ ├── schemas.py # Pydantic request/response models
│ │ ├── service.py # ConversationService business logic
│ │ ├── router.py # REST API endpoints
│ │ └── summarize.py # Context summarization logic
│ ├── agents/ # Agent orchestration │ ├── agents/ # Agent orchestration
│ │ ├── explore/ # Codebase navigation │ │ ├── explore/ # Codebase navigation
│ │ ├── plan/ # Implementation design │ │ ├── plan/ # Implementation design
@@ -193,14 +205,23 @@ All settings via environment variables or `.env`:
| HOST | 0.0.0.0 | Server host | | HOST | 0.0.0.0 | Server host |
| PORT | 8086 | Server port | | PORT | 8086 | Server port |
| OLLAMA_URL | http://192.168.86.149:11434 | Ollama API URL | | OLLAMA_URL | http://192.168.86.149:11434 | Ollama API URL |
| OLLAMA_AGENT_MODEL | mistral-nemo-large:latest | Agent reasoning model | | OLLAMA_AGENT_MODEL | gemma4:e2b | Agent reasoning model |
| OLLAMA_EMBED_MODEL | nomic-embed-text:latest | Embedding model | | OLLAMA_EMBED_MODEL | nomic-embed-text:latest | Embedding model |
| TATLOCK_API_URL | http://192.168.86.149:8000 | Tatlock auth service | | TATLOCK_API_URL | http://tatlock:8000 | Tatlock auth service |
| SEARXNG_URL | http://searxng:8080 | SearXNG web search instance |
| SEARXNG_TIMEOUT | 10 | SearXNG request timeout (seconds) |
| TOOL_TIMEOUT_SECONDS | 120 | Tool execution timeout | | TOOL_TIMEOUT_SECONDS | 120 | Tool execution timeout |
| SANDBOX_ENABLED | true | Enable sandboxed execution | | SANDBOX_ENABLED | true | Enable sandboxed execution |
| ALLOWED_PATHS | [] | Paths accessible to tools | | ALLOWED_PATHS | [] | Paths accessible to tools |
| SESSION_TTL_HOURS | 24 | Session expiry | | SESSION_TTL_HOURS | 24 | Session expiry |
| MAX_CONTEXT_TOKENS | 128000 | Max context window | | MAX_CONTEXT_TOKENS | 128000 | Max context window |
| DATABASE_URL | sqlite+aiosqlite:///./webber.db | Database connection URL |
| SUMMARIZATION_THRESHOLD | 0.8 | Summarize at N% of max tokens |
| SUMMARIZATION_TARGET_TOKENS | 500 | Target summary size |
| KEEP_RECENT_MESSAGES | 6 | Messages to keep unsummarized |
| RETRY_MAX_ATTEMPTS | 3 | Max retry attempts for transient failures |
| RETRY_BASE_DELAY | 1.0 | Base delay between retries (seconds) |
| RETRY_MAX_DELAY | 30.0 | Maximum delay between retries (seconds) |
--- ---
@@ -250,6 +271,53 @@ Tools are sandboxed operations agents can invoke:
--- ---
## Database Layer
SQLAlchemy 2.0 async with lazy initialization pattern.
### Supported Databases
- **Development**: SQLite via `aiosqlite`
- **Production**: PostgreSQL via `asyncpg`
### Pattern
```python
from src.db import get_session
from sqlalchemy.ext.asyncio import AsyncSession
async def my_endpoint(session: AsyncSession = Depends(get_session)):
# Session auto-commits on success, rollbacks on exception
result = await session.execute(query)
```
Tables are created lazily on first `get_session()` call.
---
## Conversation API
Multi-turn conversation memory with automatic context summarization.
### Endpoints
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/conversations/` | POST | Create new conversation |
| `/conversations/` | GET | List user's conversations |
| `/conversations/{id}` | GET | Get conversation with history |
| `/conversations/{id}/messages` | POST | Add message, triggers agent |
| `/conversations/{id}` | DELETE | Delete conversation |
### Models
- **Conversation**: User session with agent type, working directory
- **Message**: Individual messages with role, content, token count
### Context Summarization
When total tokens exceed 80% of `MAX_CONTEXT_TOKENS`:
1. Keep last 6 messages intact
2. Summarize older messages into a single summary message
3. Mark old messages as summarized (soft delete)
---
## Authentication Flow ## Authentication Flow
1. Client sends `X-API-Key` header 1. Client sends `X-API-Key` header
@@ -291,7 +359,7 @@ Gitea Actions workflow:
Deployed in Portainer `agents` stack alongside Tatlock: Deployed in Portainer `agents` stack alongside Tatlock:
- Network: `docker-dataplane` - Network: `docker-dataplane`
- Registry: `git.schweitz.internal/jpmschweitzer/webber` - Registry: `git.schweitz.net/jpmschweitzer/webber`
- Auto-update: Watchtower with label `com.centurylinklabs.watchtower.enable=true` - Auto-update: Watchtower with label `com.centurylinklabs.watchtower.enable=true`
--- ---
+10 -2
View File
@@ -1,6 +1,6 @@
[project] [project]
name = "webber-api" name = "webber-api"
version = "0.3.0" version = "1.1.0"
description = "Webber API - Multi-Agent AI Development Server" description = "Webber API - Multi-Agent AI Development Server"
authors = [ authors = [
{name = "jpmschweitzer"} {name = "jpmschweitzer"}
@@ -27,7 +27,15 @@ include = ["src*"]
testpaths = ["tests"] testpaths = ["tests"]
python_files = ["test_*.py"] python_files = ["test_*.py"]
python_functions = ["test_*"] python_functions = ["test_*"]
addopts = "-v" addopts = "-q --strict-markers --tb=short"
markers = [
"integration: marks tests as integration tests (require Ollama to be running)",
"e2e: marks tests as end-to-end tests (require API server to be running)",
"slow: marks tests as slow (may take > 10 seconds)",
]
filterwarnings = [
"ignore::pytest.PytestUnraisableExceptionWarning",
]
[tool.mypy] [tool.mypy]
python_version = "3.12" python_version = "3.12"
+3
View File
@@ -15,6 +15,9 @@ pip-audit~=2.9.0
# Type checking # Type checking
mypy~=1.19.1 mypy~=1.19.1
# Stubs for aiofiles, which ships none. Without them mypy reports
# import-untyped on every module that reads or writes a file.
types-aiofiles~=25.1
# Linting and formatting # Linting and formatting
ruff~=0.9.4 ruff~=0.9.4
+7
View File
@@ -25,3 +25,10 @@ rich~=13.9.0
python-multipart~=0.0.21 python-multipart~=0.0.21
python-dotenv~=1.2.1 python-dotenv~=1.2.1
pathspec~=0.12.1 # Gitignore pattern matching pathspec~=0.12.1 # Gitignore pattern matching
# Database
sqlalchemy[asyncio]~=2.0.36
aiosqlite~=0.21.0 # SQLite async driver (dev)
# Token counting
tiktoken>=0.12.0 # OpenAI tokenizer (used for estimation)
+1 -1
View File
@@ -6,9 +6,9 @@ from pathlib import Path
import typer import typer
from src.cli.session.loop import AgenticLoop
from src.cli.theme import get_theme from src.cli.theme import get_theme
from src.cli.ui.console import get_console from src.cli.ui.console import get_console
from src.cli.session.loop import AgenticLoop
from src.shared.logging import setup_logging from src.shared.logging import setup_logging
console = get_console() console = get_console()
+7 -1
View File
@@ -53,11 +53,17 @@ def main(
# Import and register commands # Import and register commands
from src.cli.commands import chat, explore, version # noqa: E402, F401 from src.cli.commands import chat, explore, version # noqa: E402
# Register subcommands # Register subcommands
app.command(name="chat")(chat.chat_command) app.command(name="chat")(chat.chat_command)
app.command(name="explore")(explore.explore_command) app.command(name="explore")(explore.explore_command)
# version was imported and never registered, so `webber version` did not exist.
# The --version flag above is the terse form; show_version prints the panel with
# the resolved Ollama URL, model and debug state, which is the one worth having
# when something is misconfigured. The F401 suppression on the import was what
# kept the omission quiet.
app.command(name="version")(version.show_version)
if __name__ == "__main__": if __name__ == "__main__":
+1 -1
View File
@@ -4,4 +4,4 @@ Session management for CLI.
from src.cli.session.context import SessionState from src.cli.session.context import SessionState
from src.cli.session.loop import AgenticLoop from src.cli.session.loop import AgenticLoop
__all__ = ["SessionState", "AgenticLoop"] __all__ = ["AgenticLoop", "SessionState"]
+1 -2
View File
@@ -1,14 +1,13 @@
""" """
Agentic conversation loop for interactive CLI. Agentic conversation loop for interactive CLI.
""" """
from typing import Any
from rich.console import Console from rich.console import Console
from src.cli.session.context import SessionState from src.cli.session.context import SessionState
from src.cli.ui.display import format_response from src.cli.ui.display import format_response
from src.domains.agents.base import BaseAgent from src.domains.agents.base import BaseAgent
from src.shared.logging import logged, trace_span, get_logger from src.shared.logging import get_logger, logged, trace_span
logger = get_logger(__name__) logger = get_logger(__name__)
+2 -2
View File
@@ -4,7 +4,7 @@ CLI theme configuration.
Centralized color and style definitions for the Webber CLI. Centralized color and style definitions for the Webber CLI.
All color choices should be defined here for easy customization. All color choices should be defined here for easy customization.
""" """
from dataclasses import dataclass from dataclasses import dataclass, field
@dataclass(frozen=True) @dataclass(frozen=True)
@@ -36,7 +36,7 @@ class ThemeColors:
class ThemeConfig: class ThemeConfig:
"""Complete theme configuration.""" """Complete theme configuration."""
colors: ThemeColors = ThemeColors() colors: ThemeColors = field(default_factory=ThemeColors)
# Spinner style for loading indicators # Spinner style for loading indicators
spinner: str = "dots" spinner: str = "dots"
+2 -2
View File
@@ -2,6 +2,6 @@
CLI UI components. CLI UI components.
""" """
from src.cli.ui.console import get_console from src.cli.ui.console import get_console
from src.cli.ui.display import format_response, format_code from src.cli.ui.display import format_code, format_response
__all__ = ["get_console", "format_response", "format_code"] __all__ = ["format_code", "format_response", "get_console"]
-1
View File
@@ -8,7 +8,6 @@ from rich.syntax import Syntax
from rich.text import Text from rich.text import Text
from src.cli.theme import get_theme from src.cli.theme import get_theme
from src.cli.ui.console import get_console
def format_response(text: str) -> Markdown | Text: def format_response(text: str) -> Markdown | Text:
+14
View File
@@ -0,0 +1,14 @@
"""
Database package for Webber.
Provides async SQLAlchemy database access following core-api patterns.
"""
from src.db.database import Database, get_database, get_session
from src.db.models import Base
__all__ = [
"Base",
"Database",
"get_database",
"get_session",
]
+123
View File
@@ -0,0 +1,123 @@
"""
Async SQLAlchemy database management.
Pattern from core-api: singleton Database class with async session factory.
"""
from collections.abc import AsyncGenerator
from functools import lru_cache
from sqlalchemy.ext.asyncio import (
AsyncEngine,
AsyncSession,
async_sessionmaker,
create_async_engine,
)
from src.shared.config import get_settings
from src.shared.logging import get_logger
logger = get_logger(__name__)
class Database:
"""
Async database connection manager.
Manages SQLAlchemy async engine and session factory.
"""
def __init__(self, url: str):
"""
Initialize database with connection URL.
Args:
url: SQLAlchemy async connection URL
e.g., "sqlite+aiosqlite:///./webber.db"
or "postgresql+asyncpg://user:pass@host/db"
"""
self._url = url
self._engine: AsyncEngine | None = None
self._session_factory: async_sessionmaker[AsyncSession] | None = None
@property
def engine(self) -> AsyncEngine:
"""Get or create the async engine."""
if self._engine is None:
self._engine = create_async_engine(
self._url,
echo=get_settings().debug,
pool_pre_ping=True,
)
return self._engine
@property
def session_factory(self) -> async_sessionmaker[AsyncSession]:
"""Get or create the session factory."""
if self._session_factory is None:
self._session_factory = async_sessionmaker(
bind=self.engine,
class_=AsyncSession,
expire_on_commit=False,
autoflush=False,
)
return self._session_factory
async def create_tables(self) -> None:
"""Create all tables (for development)."""
from src.db.models import Base
async with self.engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
logger.info("Database tables created")
async def close(self) -> None:
"""Close the database connection."""
if self._engine:
await self._engine.dispose()
self._engine = None
self._session_factory = None
logger.info("Database connection closed")
# Singleton instance
_database: Database | None = None
_tables_created: bool = False
@lru_cache
def get_database() -> Database:
"""Get the database singleton."""
global _database
if _database is None:
settings = get_settings()
_database = Database(settings.database_url)
return _database
async def _ensure_tables() -> None:
"""Ensure database tables exist (lazy initialization)."""
global _tables_created
if not _tables_created:
database = get_database()
await database.create_tables()
_tables_created = True
async def get_session() -> AsyncGenerator[AsyncSession, None]:
"""
Dependency for getting async database sessions.
Usage:
@router.get("/")
async def endpoint(session: AsyncSession = Depends(get_session)):
...
"""
await _ensure_tables()
database = get_database()
async with database.session_factory() as session:
try:
yield session
await session.commit()
except Exception:
await session.rollback()
raise
+9
View File
@@ -0,0 +1,9 @@
"""
SQLAlchemy Base model for all database models.
"""
from sqlalchemy.orm import DeclarativeBase
class Base(DeclarativeBase):
"""Base class for all SQLAlchemy models."""
pass
+12 -12
View File
@@ -4,34 +4,34 @@ Agent implementations.
All agents inherit from BaseAgent and are registered in the global registry. All agents inherit from BaseAgent and are registered in the global registry.
""" """
from src.domains.agents.base import ( from src.domains.agents.base import (
BaseAgent,
AgentContext, AgentContext,
AgentProtocol, AgentProtocol,
register_agent, BaseAgent,
get_agent, get_agent,
list_agents,
get_registry, get_registry,
list_agents,
register_agent,
) )
from src.domains.agents.explore import ( from src.domains.agents.explore import (
ExploreAgentImpl, ExploreAgentImpl,
ExploreContext, ExploreContext,
explore_agent,
explore, explore,
explore_agent,
) )
__all__ = [ __all__ = [
# Base classes
"BaseAgent",
"AgentContext", "AgentContext",
"AgentProtocol", "AgentProtocol",
# Registry functions # Base classes
"register_agent", "BaseAgent",
"get_agent",
"list_agents",
"get_registry",
# Explore agent # Explore agent
"ExploreAgentImpl", "ExploreAgentImpl",
"ExploreContext", "ExploreContext",
"explore_agent",
"explore", "explore",
"explore_agent",
"get_agent",
"get_registry",
"list_agents",
# Registry functions
"register_agent",
] ]
+212
View File
@@ -0,0 +1,212 @@
"""
Tool approval evaluation logic.
Provides granular control over tool execution:
- Rule-based matching on tool name and arguments
- Priority-ordered rule evaluation
- Default fallback behavior
"""
import re
from typing import Any
from src.domains.agents.schemas import (
ApprovalAction,
ApprovalRule,
ApprovalRuleSet,
PermissionMode,
)
from src.shared.logging import get_logger
logger = get_logger(__name__)
def _serialize_tool_args(tool_args: dict[str, Any]) -> str:
"""
Serialize tool arguments to a string for pattern matching.
Converts tool args dict to a consistent string format that can be
matched against regex patterns.
Examples:
{"command": "curl localhost:8095"} -> "command=curl localhost:8095"
{"file_path": "/src/main.py"} -> "file_path=/src/main.py"
"""
parts = []
for key, value in sorted(tool_args.items()):
parts.append(f"{key}={value}")
return " ".join(parts)
def evaluate_rule(rule: ApprovalRule, tool_name: str, tool_args: dict[str, Any]) -> bool:
"""
Check if a rule matches the given tool call.
Args:
rule: The approval rule to evaluate
tool_name: Name of the tool being called
tool_args: Arguments passed to the tool
Returns:
True if the rule matches, False otherwise
"""
# Tool name must match exactly
if rule.tool != tool_name and rule.tool != "*":
return False
# Serialize args for pattern matching
args_str = _serialize_tool_args(tool_args)
# Try to match pattern against serialized args
try:
if re.search(rule.pattern, args_str, re.IGNORECASE):
return True
except re.error as e:
logger.warning(f"Invalid regex pattern in rule: {rule.pattern} - {e}")
return False
return False
def evaluate_approval(
ruleset: ApprovalRuleSet,
tool_name: str,
tool_args: dict[str, Any],
mode: PermissionMode = PermissionMode.default,
) -> ApprovalAction:
"""
Evaluate whether a tool call should be allowed, denied, or prompt for approval.
Args:
ruleset: Set of approval rules to evaluate
tool_name: Name of the tool being called
tool_args: Arguments passed to the tool
mode: Current permission mode
Returns:
ApprovalAction indicating what to do (allow, deny, ask)
"""
# Plan mode: only read-only tools are even registered, so if we get here
# it's a read-only tool and should be allowed
if mode == PermissionMode.plan:
return ApprovalAction.allow
# Auto-accept mode: allow everything without prompting
if mode == PermissionMode.auto_accept:
return ApprovalAction.allow
# Default mode: evaluate rules
# Sort rules by priority (highest first)
sorted_rules = sorted(ruleset.rules, key=lambda r: r.priority, reverse=True)
for rule in sorted_rules:
if evaluate_rule(rule, tool_name, tool_args):
logger.debug(
f"Rule matched: {rule.description or rule.pattern} -> {rule.action}"
)
return rule.action
# No rules matched, use default action
return ruleset.default_action
# === Default rule sets ===
# Read-only tools that never need approval
READONLY_TOOLS = {"read_file", "glob_files", "grep_content", "bash_readonly"}
# Default rules for common patterns
DEFAULT_RULES = ApprovalRuleSet(
rules=[
# Always allow read-only tools
ApprovalRule(
tool="read_file",
pattern=".*",
action=ApprovalAction.allow,
description="Allow all file reads",
priority=100,
),
ApprovalRule(
tool="glob_files",
pattern=".*",
action=ApprovalAction.allow,
description="Allow all glob searches",
priority=100,
),
ApprovalRule(
tool="grep_content",
pattern=".*",
action=ApprovalAction.allow,
description="Allow all grep searches",
priority=100,
),
ApprovalRule(
tool="bash_readonly",
pattern=".*",
action=ApprovalAction.allow,
description="Allow all read-only bash commands",
priority=100,
),
# Dangerous patterns - always deny
ApprovalRule(
tool="bash",
pattern="rm\\s+-rf\\s+/",
action=ApprovalAction.deny,
description="Deny recursive delete from root",
priority=90,
),
ApprovalRule(
tool="bash",
pattern="sudo\\s+",
action=ApprovalAction.deny,
description="Deny sudo commands",
priority=90,
),
# Common safe patterns - allow without prompting
ApprovalRule(
tool="bash",
pattern="command=git\\s+(status|log|diff|show|branch)",
action=ApprovalAction.allow,
description="Allow read-only git commands",
priority=50,
),
ApprovalRule(
tool="bash",
pattern="command=pytest\\s+",
action=ApprovalAction.allow,
description="Allow pytest execution",
priority=50,
),
ApprovalRule(
tool="bash",
pattern="command=python\\s+-m\\s+pytest",
action=ApprovalAction.allow,
description="Allow pytest via python -m",
priority=50,
),
ApprovalRule(
tool="bash",
pattern="command=curl.*localhost",
action=ApprovalAction.allow,
description="Allow curl to localhost",
priority=50,
),
ApprovalRule(
tool="bash",
pattern="command=curl.*127\\.0\\.0\\.1",
action=ApprovalAction.allow,
description="Allow curl to 127.0.0.1",
priority=50,
),
],
default_action=ApprovalAction.ask,
)
def get_default_ruleset() -> ApprovalRuleSet:
"""Get the default approval ruleset."""
return DEFAULT_RULES
def is_readonly_tool(tool_name: str) -> bool:
"""Check if a tool is read-only (never needs approval)."""
return tool_name in READONLY_TOOLS
+40 -6
View File
@@ -4,11 +4,13 @@ Base classes and registry for agent implementations.
All agents are built on PydanticAI and registered in a central registry. All agents are built on PydanticAI and registered in a central registry.
""" """
from abc import ABC, abstractmethod from abc import ABC, abstractmethod
from collections.abc import AsyncIterator
from dataclasses import dataclass, field from dataclasses import dataclass, field
from typing import Any, Protocol, runtime_checkable from typing import Any, Generic, Protocol, TypeVar, runtime_checkable
from pydantic_ai import Agent from pydantic_ai import Agent
from src.domains.agents.schemas import StreamEvent
from src.shared.logging import get_logger from src.shared.logging import get_logger
logger = get_logger(__name__) logger = get_logger(__name__)
@@ -26,6 +28,15 @@ class AgentContext:
timeout_seconds: int = 120 timeout_seconds: int = 120
# Every agent narrows the context its tools receive — ExploreContext,
# PlanContext, TaskContext. Without this parameter BaseAgent could only say
# `Agent`, which is `Agent[Any, Any]`, and pydantic_ai then types every
# `.run()` result as Any. That is where 35 of this package's mypy errors came
# from: functions declared `-> str` returning Any, each looking like a local
# annotation slip rather than one missing type parameter in the base class.
CtxT = TypeVar("CtxT", bound=AgentContext)
@runtime_checkable @runtime_checkable
class AgentProtocol(Protocol): class AgentProtocol(Protocol):
"""Protocol that all agents must implement.""" """Protocol that all agents must implement."""
@@ -59,7 +70,7 @@ class AgentProtocol(Protocol):
... ...
class BaseAgent(ABC): class BaseAgent(ABC, Generic[CtxT]):
""" """
Abstract base class for agent implementations. Abstract base class for agent implementations.
@@ -70,7 +81,8 @@ class BaseAgent(ABC):
name = "explore" name = "explore"
description = "Fast codebase exploration" description = "Fast codebase exploration"
def _create_agent(self) -> Agent: class ExploreAgent(BaseAgent[ExploreContext]):
def _create_agent(self) -> Agent[ExploreContext, str]:
# Create and configure PydanticAI agent # Create and configure PydanticAI agent
... ...
@@ -91,15 +103,21 @@ class BaseAgent(ABC):
"""Human-readable description.""" """Human-readable description."""
pass pass
# Declared on the base rather than only in each subclass's __init__. The
# base reached it through hasattr, so mypy could not determine its type at
# all; the guard existed because nothing guaranteed the attribute existed.
# Declaring it here makes the None check sufficient.
_agent: "Agent[CtxT, str] | None" = None
@property @property
def agent(self) -> Agent: def agent(self) -> "Agent[CtxT, str]":
"""Lazy-loaded PydanticAI agent.""" """Lazy-loaded PydanticAI agent."""
if not hasattr(self, '_agent') or self._agent is None: if self._agent is None:
self._agent = self._create_agent() self._agent = self._create_agent()
return self._agent return self._agent
@abstractmethod @abstractmethod
def _create_agent(self) -> Agent: def _create_agent(self) -> "Agent[CtxT, str]":
""" """
Create and configure the PydanticAI agent. Create and configure the PydanticAI agent.
@@ -112,6 +130,22 @@ class BaseAgent(ABC):
"""Execute the agent.""" """Execute the agent."""
pass pass
async def run_stream(
self, prompt: str, **kwargs: Any
) -> AsyncIterator[str | StreamEvent]:
"""
Execute the agent with streaming output.
Default implementation falls back to non-streaming run().
Override this for true streaming support.
Yields:
Text chunks as they become available
"""
# Default: fall back to non-streaming
result = await self.run(prompt, **kwargs)
yield result
# === Agent Registry === # === Agent Registry ===
@@ -4,13 +4,13 @@ Explore Agent - Fast codebase exploration.
from src.domains.agents.explore.agent import ( from src.domains.agents.explore.agent import (
ExploreAgentImpl, ExploreAgentImpl,
ExploreContext, ExploreContext,
explore_agent,
explore, explore,
explore_agent,
) )
__all__ = [ __all__ = [
"ExploreAgentImpl", "ExploreAgentImpl",
"ExploreContext", "ExploreContext",
"explore_agent",
"explore", "explore",
"explore_agent",
] ]
+61 -6
View File
@@ -5,17 +5,18 @@ Fast codebase exploration with read-only tools.
Uses sanitized Ollama provider for reliable tool calling. Uses sanitized Ollama provider for reliable tool calling.
""" """
import os import os
from collections.abc import AsyncIterator
from dataclasses import dataclass from dataclasses import dataclass
from typing import Any from typing import Any
from pydantic_ai import Agent from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIModel from pydantic_ai.models.openai import OpenAIModel
from src.domains.agents.base import BaseAgent, AgentContext, register_agent from src.domains.agents.base import AgentContext, BaseAgent, register_agent
from src.domains.agents.explore.prompts import EXPLORE_SYSTEM_PROMPT from src.domains.agents.explore.prompts import EXPLORE_SYSTEM_PROMPT
from src.ollama.provider import get_ollama_provider from src.ollama.provider import get_ollama_provider
from src.shared.config import get_settings from src.shared.config import get_settings
from src.shared.logging import logged, get_logger, trace_span from src.shared.logging import get_logger, logged, trace_span
logger = get_logger(__name__) logger = get_logger(__name__)
@@ -30,7 +31,7 @@ class ExploreContext(AgentContext):
pass pass
class ExploreAgentImpl(BaseAgent): class ExploreAgentImpl(BaseAgent[ExploreContext]):
""" """
Fast codebase exploration agent. Fast codebase exploration agent.
@@ -43,7 +44,7 @@ class ExploreAgentImpl(BaseAgent):
def __init__(self): def __init__(self):
"""Initialize the explore agent.""" """Initialize the explore agent."""
self._agent: Agent[ExploreContext, str] | None = None self._agent = None
self._settings = get_settings() self._settings = get_settings()
def _create_agent(self) -> Agent[ExploreContext, str]: def _create_agent(self) -> Agent[ExploreContext, str]:
@@ -78,6 +79,14 @@ class ExploreAgentImpl(BaseAgent):
from src.domains.agents.explore.tools import register_explore_tools from src.domains.agents.explore.tools import register_explore_tools
register_explore_tools(agent) register_explore_tools(agent)
def _build_prompt_with_context(self, prompt: str, working_dir: str) -> str:
"""Build the prompt with working directory context."""
return f"""Working directory: {working_dir}
Use paths within this working directory for file operations.
User request: {prompt}"""
@logged() @logged()
async def run( async def run(
self, self,
@@ -97,22 +106,58 @@ class ExploreAgentImpl(BaseAgent):
Returns: Returns:
Agent response with findings Agent response with findings
""" """
effective_working_dir = working_dir or os.getcwd()
ctx = ExploreContext( ctx = ExploreContext(
working_dir=working_dir or os.getcwd(), working_dir=effective_working_dir,
allowed_paths=allowed_paths or self._settings.effective_allowed_paths, allowed_paths=allowed_paths or self._settings.effective_allowed_paths,
timeout_seconds=self._settings.tool_timeout_seconds, timeout_seconds=self._settings.tool_timeout_seconds,
) )
full_prompt = self._build_prompt_with_context(prompt, effective_working_dir)
async with trace_span("explore_agent_run"): async with trace_span("explore_agent_run"):
try: try:
# Use run() not run_stream() - Ollama has bugs with streaming + tools # Use run() not run_stream() - Ollama has bugs with streaming + tools
result = await self.agent.run(prompt, deps=ctx) result = await self.agent.run(full_prompt, deps=ctx)
return result.output return result.output
except Exception as e: except Exception as e:
logger.exception(f"Explore agent error: {e}") logger.exception(f"Explore agent error: {e}")
raise raise
async def run_stream(
self,
prompt: str,
working_dir: str | None = None,
allowed_paths: list[str] | None = None,
**kwargs: Any
) -> AsyncIterator[str]:
"""
Run the explore agent with streaming output.
Yields text chunks as they become available.
"""
effective_working_dir = working_dir or os.getcwd()
ctx = ExploreContext(
working_dir=effective_working_dir,
allowed_paths=allowed_paths or self._settings.effective_allowed_paths,
timeout_seconds=self._settings.tool_timeout_seconds,
)
full_prompt = self._build_prompt_with_context(prompt, effective_working_dir)
async with trace_span("explore_agent_stream"):
try:
async with self.agent.run_stream(full_prompt, deps=ctx) as result:
async for chunk in result.stream_text():
yield chunk
except Exception as e:
logger.exception(f"Explore agent stream error: {e}")
raise
# Create and register the singleton instance # Create and register the singleton instance
explore_agent = ExploreAgentImpl() explore_agent = ExploreAgentImpl()
register_agent(explore_agent) register_agent(explore_agent)
@@ -125,3 +170,13 @@ async def explore(
) -> str: ) -> str:
"""Run exploration query.""" """Run exploration query."""
return await explore_agent.run(prompt, working_dir=working_dir, **kwargs) return await explore_agent.run(prompt, working_dir=working_dir, **kwargs)
async def explore_stream(
prompt: str,
working_dir: str | None = None,
**kwargs: Any
) -> AsyncIterator[str]:
"""Run exploration query with streaming."""
async for chunk in explore_agent.run_stream(prompt, working_dir=working_dir, **kwargs):
yield chunk
+157 -7
View File
@@ -5,14 +5,18 @@ Registers our tool implementations with the PydanticAI agent.
""" """
from pydantic_ai import Agent, RunContext from pydantic_ai import Agent, RunContext
from src.domains.agents.base import AgentContext from src.domains.agents.explore.agent import ExploreContext
from src.domains.tools.file.read import ReadFileTool from src.domains.tools.file.edit import EditFileTool
from src.domains.tools.file.glob import GlobFilesTool from src.domains.tools.file.glob import GlobFilesTool
from src.domains.tools.file.read import ReadFileTool
from src.domains.tools.file.write import WriteFileTool
from src.domains.tools.search.grep import GrepContentTool from src.domains.tools.search.grep import GrepContentTool
from src.domains.tools.search.web import WebSearchTool
from src.domains.tools.shell.bash import BashReadOnlyTool from src.domains.tools.shell.bash import BashReadOnlyTool
from src.domains.tools.shell.bash_full import BashTool
def register_explore_tools(agent: Agent[AgentContext, str]) -> None: def register_explore_tools(agent: Agent[ExploreContext, str]) -> None:
""" """
Register all exploration tools with the agent. Register all exploration tools with the agent.
@@ -21,7 +25,7 @@ def register_explore_tools(agent: Agent[AgentContext, str]) -> None:
@agent.tool @agent.tool
async def read_file( async def read_file(
ctx: RunContext[AgentContext], ctx: RunContext[ExploreContext],
file_path: str, file_path: str,
offset: int = 0, offset: int = 0,
limit: int = 2000 limit: int = 2000
@@ -48,7 +52,7 @@ def register_explore_tools(agent: Agent[AgentContext, str]) -> None:
@agent.tool @agent.tool
async def glob_files( async def glob_files(
ctx: RunContext[AgentContext], ctx: RunContext[ExploreContext],
pattern: str, pattern: str,
path: str | None = None, path: str | None = None,
limit: int = 100 limit: int = 100
@@ -81,7 +85,7 @@ def register_explore_tools(agent: Agent[AgentContext, str]) -> None:
@agent.tool @agent.tool
async def grep_content( async def grep_content(
ctx: RunContext[AgentContext], ctx: RunContext[ExploreContext],
pattern: str, pattern: str,
path: str | None = None, path: str | None = None,
file_glob: str | None = None, file_glob: str | None = None,
@@ -121,7 +125,7 @@ def register_explore_tools(agent: Agent[AgentContext, str]) -> None:
@agent.tool @agent.tool
async def bash_readonly( async def bash_readonly(
ctx: RunContext[AgentContext], ctx: RunContext[ExploreContext],
command: str, command: str,
cwd: str | None = None, cwd: str | None = None,
timeout: int = 30 timeout: int = 30
@@ -161,3 +165,149 @@ def register_explore_tools(agent: Agent[AgentContext, str]) -> None:
timeout=min(timeout, ctx.deps.timeout_seconds) timeout=min(timeout, ctx.deps.timeout_seconds)
) )
return result.to_string() return result.to_string()
# === Write-capable tools ===
@agent.tool
async def edit_file(
ctx: RunContext[ExploreContext],
file_path: str,
old_string: str,
new_string: str,
replace_all: bool = False
) -> str:
"""Make targeted edits to a file using find-and-replace.
Args:
file_path: Absolute path to the file to edit
old_string: The exact text to find and replace (must exist in file)
new_string: The replacement text
replace_all: If True, replace all occurrences. If False (default),
old_string must be unique (appear exactly once).
Returns:
Success message with diff preview, or error.
IMPORTANT:
- old_string must exactly match file content (including whitespace)
- By default, old_string must appear exactly once (for safety)
- Always read the file first to verify exact content before editing
"""
tool = EditFileTool(allowed_paths=ctx.deps.allowed_paths)
result = await tool.execute(
file_path=file_path,
old_string=old_string,
new_string=new_string,
replace_all=replace_all
)
return result.to_string()
@agent.tool
async def write_file(
ctx: RunContext[ExploreContext],
file_path: str,
content: str
) -> str:
"""Create a new file or overwrite an existing file.
Args:
file_path: Absolute path to the file to create/write
content: The content to write to the file
Returns:
Success message with file path and size.
IMPORTANT:
- Parent directory must exist (use mkdir first if needed)
- For editing existing files, prefer edit_file instead
- Will overwrite existing files without confirmation
"""
tool = WriteFileTool(allowed_paths=ctx.deps.allowed_paths)
result = await tool.execute(
file_path=file_path,
content=content
)
return result.to_string()
@agent.tool
async def bash(
ctx: RunContext[ExploreContext],
command: str,
cwd: str | None = None,
timeout: int = 60
) -> str:
"""Execute a bash command with write capabilities.
ALLOWED:
- File operations: ls, find, mkdir, touch, cp, mv, rm (single files)
- Git (full): git add, git commit, git checkout, git merge, git pull
- Python: python, pip install, pytest, mypy, ruff
- Text processing: grep, awk, sed, sort
- Command chaining: && and || are allowed
FORBIDDEN:
- sudo, su (privilege escalation)
- Network: curl, wget, ssh, scp, rsync
- Dangerous: rm -rf, chmod 777, dd, mkfs
Args:
command: The bash command to execute
cwd: Working directory (default: agent working directory)
timeout: Timeout in seconds (default: 60)
Returns:
Command output or error message.
Examples:
- "mkdir -p src/utils" creates directory
- "git add . && git commit -m 'fix: bug'" commits changes
- "pytest tests/ -v" runs tests
- "rm old_file.py" removes single file
"""
tool = BashTool(allowed_paths=ctx.deps.allowed_paths)
working_dir = cwd or ctx.deps.working_dir
result = await tool.execute(
command=command,
cwd=working_dir,
timeout=min(timeout, ctx.deps.timeout_seconds)
)
return result.to_string()
# === Web search ===
@agent.tool
async def web_search(
ctx: RunContext[ExploreContext],
query: str,
num_results: int = 5,
categories: str | None = None
) -> str:
"""Search the web for current information.
Args:
query: Search query (e.g., "Python 3.12 new features")
num_results: Number of results to return (1-10, default: 5)
categories: Optional category filter ("general", "it", "news", "science")
Returns:
Search results with titles, URLs, and snippets.
Use this for:
- Current events or recent information
- Documentation updates since your training
- Facts you're uncertain about
- Technical references with URLs
IMPORTANT: Always include a "Sources:" section with URLs in your response.
Examples:
- query="FastAPI best practices 2024"
- query="CVE-2024" categories="it"
"""
tool = WebSearchTool()
result = await tool.execute(
query=query,
num_results=num_results,
categories=categories
)
return result.to_string()
@@ -0,0 +1,30 @@
"""
Plan Agent - Software architect for implementation planning.
The Plan agent explores codebases and designs step-by-step implementation
strategies. It uses only read-only tools and cannot modify any files.
Usage:
from src.domains.agents.plan import plan_agent, plan
# Direct agent access
result = await plan_agent.run("Plan how to add user authentication")
# Convenience function
result = await plan("Plan how to add user authentication")
"""
from src.domains.agents.plan.agent import (
PlanAgentImpl,
PlanContext,
plan,
plan_agent,
plan_stream,
)
__all__ = [
"PlanAgentImpl",
"PlanContext",
"plan",
"plan_agent",
"plan_stream",
]
+184
View File
@@ -0,0 +1,184 @@
"""
Plan Agent implementation using PydanticAI.
Software architect agent that explores codebases and designs implementation plans.
Uses only read-only tools - cannot modify any files.
"""
import os
from collections.abc import AsyncIterator
from dataclasses import dataclass
from typing import Any
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIModel
from src.domains.agents.base import AgentContext, BaseAgent, register_agent
from src.domains.agents.plan.prompts import PLAN_SYSTEM_PROMPT
from src.ollama.provider import get_ollama_provider
from src.shared.config import get_settings
from src.shared.logging import get_logger, logged, trace_span
logger = get_logger(__name__)
@dataclass
class PlanContext(AgentContext):
"""
Context for plan agent tools.
Passed to all tool functions via RunContext.
Uses the same fields as base AgentContext.
"""
pass
class PlanAgentImpl(BaseAgent[PlanContext]):
"""
Software architect agent for implementation planning.
Explores codebases to understand patterns and conventions,
then designs step-by-step implementation plans.
READ-ONLY: Cannot modify files - uses only exploration tools.
"""
name = "plan"
description = "Software architect for designing implementation plans - explores codebase and creates step-by-step strategies"
def __init__(self):
"""Initialize the plan agent."""
self._agent = None
self._settings = get_settings()
def _create_agent(self) -> Agent[PlanContext, str]:
"""Create the PydanticAI agent with Ollama backend."""
# Use sanitized Ollama provider to fix content: null issues
model = OpenAIModel(
model_name=self._settings.ollama_agent_model,
provider=get_ollama_provider(),
)
agent: Agent[PlanContext, str] = Agent(
model=model,
system_prompt=PLAN_SYSTEM_PROMPT,
deps_type=PlanContext,
output_type=str,
# Mistral Nemo settings:
# - temperature 0.3 (Nemo needs slightly higher than 0.0)
# - tool_choice "required" forces tool use
model_settings={
"temperature": 0.3,
"extra_body": {"tool_choice": "required"},
},
)
# Register read-only tools
self._register_tools(agent)
return agent
def _register_tools(self, agent: Agent[PlanContext, str]) -> None:
"""Register read-only exploration tools with the agent."""
from src.domains.agents.plan.tools import register_plan_tools
register_plan_tools(agent)
def _build_prompt_with_context(self, prompt: str, working_dir: str) -> str:
"""Build the prompt with working directory context."""
return f"""Working directory: {working_dir}
Use paths within this working directory for file operations.
User request: {prompt}"""
@logged()
async def run(
self,
prompt: str,
working_dir: str | None = None,
allowed_paths: list[str] | None = None,
**kwargs: Any
) -> str:
"""
Run the plan agent to design an implementation strategy.
Args:
prompt: Description of what to implement
working_dir: Working directory for exploration
allowed_paths: Restrict tool access to these paths
Returns:
Implementation plan with steps and critical files
"""
effective_working_dir = working_dir or os.getcwd()
ctx = PlanContext(
working_dir=effective_working_dir,
allowed_paths=allowed_paths or self._settings.effective_allowed_paths,
timeout_seconds=self._settings.tool_timeout_seconds,
)
full_prompt = self._build_prompt_with_context(prompt, effective_working_dir)
async with trace_span("plan_agent_run"):
try:
# Use run() not run_stream() - Ollama has bugs with streaming + tools
result = await self.agent.run(full_prompt, deps=ctx)
return result.output
except Exception as e:
logger.exception(f"Plan agent error: {e}")
raise
async def run_stream(
self,
prompt: str,
working_dir: str | None = None,
allowed_paths: list[str] | None = None,
**kwargs: Any
) -> AsyncIterator[str]:
"""
Run the plan agent with streaming output.
Yields text chunks as they become available.
"""
effective_working_dir = working_dir or os.getcwd()
ctx = PlanContext(
working_dir=effective_working_dir,
allowed_paths=allowed_paths or self._settings.effective_allowed_paths,
timeout_seconds=self._settings.tool_timeout_seconds,
)
full_prompt = self._build_prompt_with_context(prompt, effective_working_dir)
async with trace_span("plan_agent_stream"):
try:
async with self.agent.run_stream(full_prompt, deps=ctx) as result:
async for chunk in result.stream_text():
yield chunk
except Exception as e:
logger.exception(f"Plan agent stream error: {e}")
raise
# Create and register the singleton instance
plan_agent = PlanAgentImpl()
register_agent(plan_agent)
async def plan(
prompt: str,
working_dir: str | None = None,
**kwargs: Any
) -> str:
"""Run planning query."""
return await plan_agent.run(prompt, working_dir=working_dir, **kwargs)
async def plan_stream(
prompt: str,
working_dir: str | None = None,
**kwargs: Any
) -> AsyncIterator[str]:
"""Run planning query with streaming."""
async for chunk in plan_agent.run_stream(prompt, working_dir=working_dir, **kwargs):
yield chunk
@@ -0,0 +1,63 @@
"""
System prompts for the Plan agent.
The Plan agent is a READ-ONLY software architect that explores codebases
and designs implementation plans without modifying any files.
"""
PLAN_SYSTEM_PROMPT = """You are a software architect and planning specialist.
Your role is to explore codebases and design implementation plans.
CRITICAL: You are READ-ONLY. You CANNOT modify any files.
AVAILABLE TOOLS:
- glob_files: Find files by pattern
- read_file: Read file contents
- grep_content: Search code with regex
- bash_readonly: Run read-only commands (ls, git status, git log, etc.)
WORKFLOW:
1. Understand the requirements
2. Explore the codebase to find relevant patterns and conventions
3. Design an implementation approach
4. Create a step-by-step plan with specific files and changes
TOOL CALL EXAMPLES (follow exactly):
To find Python files:
Call glob_files with pattern="**/*.py"
To find a specific file:
Call glob_files with pattern="**/config.py"
To read a file:
Call read_file with file_path="/absolute/path/to/file.py"
To search for code patterns:
Call grep_content with pattern="class.*Controller"
To check git history:
Call bash_readonly with command="git log --oneline -10"
OUTPUT FORMAT:
End your response with:
### Implementation Steps
1. [First step with specific file and changes]
2. [Second step...]
3. [Continue...]
### Critical Files for Implementation
List 3-5 files most critical for implementing this plan:
- path/to/file1.py - [Brief reason: e.g., "Core logic to modify"]
- path/to/file2.py - [Brief reason: e.g., "Pattern to follow"]
RULES:
- ALWAYS use tools first, then analyze results
- Follow existing patterns in the codebase
- Consider trade-offs and alternatives
- Identify dependencies and sequencing
- Never guess - verify with tools
- Provide specific file paths and code locations
"""
+170
View File
@@ -0,0 +1,170 @@
"""
Tool registrations for the Plan agent.
The Plan agent only has access to READ-ONLY tools.
It cannot modify files - only explore and analyze.
"""
from pydantic_ai import Agent, RunContext
from src.domains.agents.plan.agent import PlanContext
from src.domains.tools.file.glob import GlobFilesTool
from src.domains.tools.file.read import ReadFileTool
from src.domains.tools.search.grep import GrepContentTool
from src.domains.tools.shell.bash import BashReadOnlyTool
def register_plan_tools(agent: Agent[PlanContext, str]) -> None:
"""
Register read-only exploration tools with the Plan agent.
The Plan agent is restricted to read-only tools:
- read_file: Read file contents
- glob_files: Find files by pattern
- grep_content: Search file contents
- bash_readonly: Read-only shell commands
Write tools (edit_file, write_file, bash) are NOT available.
"""
@agent.tool
async def read_file(
ctx: RunContext[PlanContext],
file_path: str,
offset: int = 0,
limit: int = 2000
) -> str:
"""Read contents of a file with line numbers.
Args:
file_path: Absolute path to the file to read
offset: Line number to start from (0-based, default: 0)
limit: Maximum number of lines to read (default: 2000)
Returns:
File contents with line numbers, or error message.
IMPORTANT: Always use absolute paths. Use this to understand existing code.
"""
tool = ReadFileTool(allowed_paths=ctx.deps.allowed_paths)
result = await tool.execute(
file_path=file_path,
offset=offset,
limit=limit
)
return result.to_string()
@agent.tool
async def glob_files(
ctx: RunContext[PlanContext],
pattern: str,
path: str | None = None,
limit: int = 100
) -> str:
"""Find files matching a glob pattern.
Args:
pattern: Glob pattern (e.g., "**/*.py", "src/**/*.ts", "*.md")
path: Directory to search in (default: working directory)
limit: Maximum number of files to return (default: 100)
Returns:
List of absolute file paths, sorted by modification time (newest first).
Examples:
- "**/*.py" finds all Python files
- "src/**/*.ts" finds TypeScript files in src/
- "**/test_*.py" finds all test files
IMPORTANT: Use this to discover files before reading them.
"""
tool = GlobFilesTool(allowed_paths=ctx.deps.allowed_paths)
search_path = path or ctx.deps.working_dir
result = await tool.execute(
pattern=pattern,
path=search_path,
limit=limit
)
return result.to_string()
@agent.tool
async def grep_content(
ctx: RunContext[PlanContext],
pattern: str,
path: str | None = None,
file_glob: str | None = None,
context_lines: int = 0,
case_sensitive: bool = True
) -> str:
"""Search file contents using regex pattern.
Args:
pattern: Regex pattern to search for (Python re syntax)
path: Directory or file to search (default: working directory)
file_glob: Filter files by glob (e.g., "*.py", "*.ts")
context_lines: Lines of context before/after matches (default: 0)
case_sensitive: Case-sensitive search (default: True)
Returns:
Matching lines with file paths and line numbers.
Format: "filepath:line_num: content"
Examples:
- pattern="def.*__init__" finds init methods
- pattern="class\\s+\\w+" finds class definitions
- pattern="TODO|FIXME" finds todo comments
IMPORTANT: Use this to find code patterns and implementations.
"""
tool = GrepContentTool(allowed_paths=ctx.deps.allowed_paths)
search_path = path or ctx.deps.working_dir
result = await tool.execute(
pattern=pattern,
path=search_path,
file_glob=file_glob,
context_lines=context_lines,
case_sensitive=case_sensitive
)
return result.to_string()
@agent.tool
async def bash_readonly(
ctx: RunContext[PlanContext],
command: str,
cwd: str | None = None,
timeout: int = 30
) -> str:
"""Execute a read-only bash command.
ALLOWED commands:
- File inspection: ls, find, cat, head, tail, wc, file, stat, tree, du
- Git (read-only): git status, git log, git diff, git show, git branch
- Text processing: grep, awk, sed (read-only), sort, uniq
- System info: pwd, whoami, hostname, which
FORBIDDEN:
- File modification (rm, mv, cp, mkdir, touch)
- Redirects (>, >>)
- Command chaining (&&, ||, ;)
- Network (curl, wget)
Args:
command: The bash command to execute
cwd: Working directory (default: agent working directory)
timeout: Timeout in seconds (default: 30)
Returns:
Command output or error message.
Examples:
- "ls -la" lists files with details
- "git status" shows git status
- "git log --oneline -10" shows recent commits
"""
tool = BashReadOnlyTool(allowed_paths=ctx.deps.allowed_paths)
working_dir = cwd or ctx.deps.working_dir
result = await tool.execute(
command=command,
cwd=working_dir,
timeout=min(timeout, ctx.deps.timeout_seconds)
)
return result.to_string()
+94 -8
View File
@@ -1,19 +1,40 @@
""" """
REST API routes for agents. REST API routes for agents.
"""
from fastapi import APIRouter, HTTPException
from src.domains.agents.base import get_agent, list_agents Supports permission modes for controlling agent tool access:
- default: All tools available (approval may be required)
- plan: Read-only tools only
- auto_accept: All tools, no approval prompts
Streaming uses structured events instead of raw text to avoid
garbled output during tool execution.
"""
import json
from fastapi import APIRouter, HTTPException
from fastapi.responses import StreamingResponse
# Import agents to ensure they're registered # Import agents to ensure they're registered
import src.domains.agents.explore # noqa: F401 import src.domains.agents.explore
import src.domains.agents.plan
import src.domains.agents.task # noqa: F401
from src.domains.agents.base import get_agent, list_agents
from src.domains.agents.schemas import ( from src.domains.agents.schemas import (
AgentRunRequest,
AgentRunResponse,
AgentInfo, AgentInfo,
AgentListResponse, AgentListResponse,
AgentRunRequest,
AgentRunResponse,
PermissionMode,
StreamEvent,
) )
from src.shared.logging import logged, get_logger from src.shared.logging import get_logger, logged
def _get_mode(mode_value: str | PermissionMode) -> PermissionMode:
"""Convert mode string to enum (handles use_enum_values=True)."""
if isinstance(mode_value, PermissionMode):
return mode_value
return PermissionMode(mode_value)
logger = get_logger(__name__) logger = get_logger(__name__)
@@ -36,6 +57,7 @@ async def run_agent(request: AgentRunRequest) -> AgentRunResponse:
Run an agent with the given prompt. Run an agent with the given prompt.
The agent will use tools to explore the codebase and answer questions. The agent will use tools to explore the codebase and answer questions.
Permission mode controls which tools are available.
""" """
# Get the requested agent # Get the requested agent
agent = get_agent(request.agent_type) agent = get_agent(request.agent_type)
@@ -45,16 +67,21 @@ async def run_agent(request: AgentRunRequest) -> AgentRunResponse:
detail=f"Unknown agent type: {request.agent_type}" detail=f"Unknown agent type: {request.agent_type}"
) )
# Convert mode string to enum (use_enum_values=True in schema)
mode = _get_mode(request.mode)
try: try:
# Run the agent # Run the agent with mode
response = await agent.run( response = await agent.run(
request.prompt, request.prompt,
working_dir=request.working_dir, working_dir=request.working_dir,
mode=mode,
) )
return AgentRunResponse( return AgentRunResponse(
response=response, response=response,
agent_type=request.agent_type, agent_type=request.agent_type,
mode=request.mode, # Keep original for response
success=True, success=True,
) )
@@ -63,11 +90,70 @@ async def run_agent(request: AgentRunRequest) -> AgentRunResponse:
return AgentRunResponse( return AgentRunResponse(
response="", response="",
agent_type=request.agent_type, agent_type=request.agent_type,
mode=request.mode,
success=False, success=False,
error=str(e), error=str(e),
) )
@router.post("/stream")
@logged()
async def stream_agent(request: AgentRunRequest) -> StreamingResponse:
"""
Run an agent with streaming response.
Returns Server-Sent Events (SSE) with structured events.
Permission mode controls which tools are available.
Event types (from StreamEvent):
- tool_start: Tool execution beginning
- tool_done: Tool execution complete
- thinking: Agent status update
- response: Final response text chunk
- error: Error occurred
- done: Stream complete
"""
agent = get_agent(request.agent_type)
if not agent:
raise HTTPException(
status_code=400,
detail=f"Unknown agent type: {request.agent_type}"
)
# Convert mode string to enum (use_enum_values=True in schema)
mode = _get_mode(request.mode)
async def generate():
try:
async for event in agent.run_stream(
request.prompt,
working_dir=request.working_dir,
mode=mode,
):
# Handle both StreamEvent objects and legacy string chunks
if isinstance(event, StreamEvent):
# New structured event format
event_data = event.model_dump(exclude_none=True)
yield f"data: {json.dumps(event_data)}\n\n"
else:
# Legacy string chunk (for explore/plan agents)
yield f"data: {json.dumps({'event': 'chunk', 'data': event})}\n\n"
except Exception as e:
logger.exception(f"Stream error: {e}")
error_event = {"event": "error", "error_message": str(e)}
yield f"data: {json.dumps(error_event)}\n\n"
return StreamingResponse(
generate(),
media_type="text/event-stream",
headers={
"Cache-Control": "no-cache",
"Connection": "keep-alive",
}
)
@router.get("/{agent_type}", response_model=AgentInfo) @router.get("/{agent_type}", response_model=AgentInfo)
async def get_agent_info(agent_type: str) -> AgentInfo: async def get_agent_info(agent_type: str) -> AgentInfo:
"""Get information about a specific agent.""" """Get information about a specific agent."""
+139 -1
View File
@@ -1,22 +1,127 @@
""" """
Request and response schemas for agent API. Request and response schemas for agent API.
""" """
from enum import Enum
from src.shared.base import BaseSchema from src.shared.base import BaseSchema
class PermissionMode(str, Enum):
"""
Permission modes that control agent tool access.
Aligns with Claude Code's permission model:
- default: Full tools, approval required for writes (future)
- plan: Read-only tools only, no approval needed
- auto_accept: Full tools, no approval prompts
"""
default = "default"
plan = "plan"
auto_accept = "auto_accept"
class ApprovalStatus(str, Enum):
"""Status of a tool approval request."""
pending = "pending"
approved = "approved"
denied = "denied"
class ApprovalAction(str, Enum):
"""Action to take when a rule matches."""
allow = "allow" # Auto-approve without prompting
deny = "deny" # Auto-deny without prompting
ask = "ask" # Prompt user for approval
class ApprovalRule(BaseSchema):
"""
Granular approval rule for tool execution.
Allows fine-grained control over which tool calls are allowed:
- Pattern matching on tool arguments
- Different actions per rule (allow, deny, ask)
Examples:
# Allow curl to localhost
ApprovalRule(tool="bash", pattern="curl.*localhost.*", action="allow")
# Deny any rm command
ApprovalRule(tool="bash", pattern="rm\\s+.*", action="deny")
# Ask for git push
ApprovalRule(tool="bash", pattern="git\\s+push.*", action="ask")
# Allow all file reads in src/
ApprovalRule(tool="read_file", pattern=".*/src/.*", action="allow")
"""
tool: str # Tool name to match (e.g., "bash", "edit_file")
pattern: str # Regex pattern to match against tool args
action: ApprovalAction # What to do when matched
description: str | None = None # Human-readable description of rule
priority: int = 0 # Higher priority rules evaluated first
class ApprovalRuleSet(BaseSchema):
"""
Collection of approval rules with evaluation logic.
Rules are evaluated in priority order (highest first).
First matching rule determines the action.
If no rules match, falls back to default action.
"""
# Suppression justified: this is a pydantic model, not a plain class. Pydantic
# deep-copies field defaults per instance — verified: two ApprovalRuleSet()
# instances have `rules` lists that are not the same object, and appending
# to one leaves the other empty. RUF012's suggested fix, annotating this
# ClassVar, would remove the field from the model altogether. Ruff cannot
# see the pydantic base because BaseSchema is a local subclass of BaseModel.
rules: list[ApprovalRule] = [] # noqa: RUF012
default_action: ApprovalAction = ApprovalAction.ask # Default when no rules match
class ToolApprovalRequest(BaseSchema):
"""
Request for tool execution approval.
Sent from API to CLI when a tool needs user approval.
Prep for future bidirectional approval flow.
"""
request_id: str
tool_name: str
tool_args: dict
description: str
risk_level: str = "write" # "read", "write", "dangerous"
class ToolApprovalResponse(BaseSchema):
"""
Response to a tool approval request.
Sent from CLI to API with user's decision.
"""
request_id: str
status: ApprovalStatus
reason: str | None = None
class AgentRunRequest(BaseSchema): class AgentRunRequest(BaseSchema):
"""Request to run an agent.""" """Request to run an agent."""
prompt: str prompt: str
working_dir: str = "." working_dir: str = "."
agent_type: str = "explore" agent_type: str = "task" # Default to task agent (main agent)
mode: PermissionMode = PermissionMode.default
class AgentRunResponse(BaseSchema): class AgentRunResponse(BaseSchema):
"""Response from agent execution.""" """Response from agent execution."""
response: str response: str
agent_type: str agent_type: str
mode: PermissionMode = PermissionMode.default
success: bool = True success: bool = True
error: str | None = None error: str | None = None
# Prep for approval flow - if set, CLI should handle approval
pending_approval: ToolApprovalRequest | None = None
class AgentInfo(BaseSchema): class AgentInfo(BaseSchema):
@@ -28,3 +133,36 @@ class AgentInfo(BaseSchema):
class AgentListResponse(BaseSchema): class AgentListResponse(BaseSchema):
"""List of available agents.""" """List of available agents."""
agents: list[AgentInfo] agents: list[AgentInfo]
# Streaming event types for event-based streaming
class StreamEventType(str, Enum):
"""
Event types for structured agent streaming.
Instead of streaming raw text (which gets garbled during tool calls),
we emit structured events that the CLI can render appropriately.
"""
tool_start = "tool_start" # Tool execution starting
tool_done = "tool_done" # Tool execution complete
thinking = "thinking" # Agent reasoning status
response = "response" # Final response text chunk
error = "error" # Error occurred
done = "done" # Stream complete
class StreamEvent(BaseSchema):
"""
Structured streaming event from agent execution.
Events are emitted instead of raw text to provide clean
progress feedback during multi-tool agent loops.
"""
event: StreamEventType
tool: str | None = None # Tool name (for tool_start/tool_done)
args: dict | None = None # Tool arguments (for tool_start)
result_summary: str | None = None # Brief result (for tool_done)
message: str | None = None # Status message (for thinking)
text: str | None = None # Response text (for response)
error_message: str | None = None # Error details (for error)
mode: str | None = None # Permission mode (for done)
@@ -0,0 +1,33 @@
"""
Task Agent - Full orchestrator for autonomous task execution.
The Task agent can:
- Execute multi-step tasks autonomously
- Use all tools (read + write + bash)
- Spawn sub-agents (Explore, Plan) for focused work
- Return consolidated task summaries
Usage:
from src.domains.agents.task import task_agent, task
# Direct agent access
result = await task_agent.run("Create a new user model with tests")
# Convenience function
result = await task("Create a new user model with tests")
"""
from src.domains.agents.task.agent import (
TaskAgentImpl,
TaskContext,
task,
task_agent,
task_stream,
)
__all__ = [
"TaskAgentImpl",
"TaskContext",
"task",
"task_agent",
"task_stream",
]
+401
View File
@@ -0,0 +1,401 @@
"""
Task Agent implementation using PydanticAI.
Full orchestrator agent that can:
- Execute multi-step tasks autonomously
- Use all tools (read + write) based on permission mode
- Spawn sub-agents (Explore, Plan) for focused work
- Stream structured events instead of raw text
"""
import asyncio
import contextlib
import os
from collections.abc import AsyncIterator
from dataclasses import dataclass, field
from typing import Any
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIModel
from src.domains.agents.base import AgentContext, BaseAgent, register_agent
from src.domains.agents.schemas import PermissionMode, StreamEvent, StreamEventType
from src.domains.agents.task.prompts import TASK_PLAN_MODE_PROMPT, TASK_SYSTEM_PROMPT
from src.ollama.provider import get_ollama_provider
from src.shared.config import get_settings
from src.shared.logging import get_logger, logged, trace_span
logger = get_logger(__name__)
@dataclass
class TaskContext(AgentContext):
"""
Context for task agent tools.
Passed to all tool functions via RunContext.
Extends base AgentContext with permission mode and event queue.
"""
mode: PermissionMode = PermissionMode.default
# Prep for approval flow - tools can check this
pending_approvals: list[str] = field(default_factory=list)
# Event queue for streaming events from tools
event_queue: asyncio.Queue | None = field(default=None, repr=False)
# Track tool calls for retry logic
tools_called: int = 0
def _emit_event(ctx: AgentContext, event: StreamEvent) -> None:
"""Emit an event to the queue if available."""
if hasattr(ctx, 'event_queue') and ctx.event_queue is not None:
ctx.event_queue.put_nowait(event)
def _summarize_result(result: str, max_len: int = 80) -> str:
"""Create a brief summary of a tool result."""
# Count lines if multiline
lines = result.strip().split('\n')
if len(lines) > 1:
return f"{len(lines)} lines"
# Single line - truncate if needed
if len(result) > max_len:
return result[:max_len] + "..."
return result
class TaskAgentImpl(BaseAgent[TaskContext]):
"""
Full orchestrator agent for autonomous task execution.
Tool access depends on permission mode:
- plan: Read-only tools only (safe exploration)
- default: All tools (approval required for writes - future)
- auto_accept: All tools (no approval prompts)
Can spawn Explore and Plan agents to offload focused tasks,
keeping context efficient across complex multi-step work.
"""
name = "task"
description = "Autonomous multi-step task execution with sub-agent orchestration"
def __init__(self):
"""Initialize the task agent."""
# Cache agents by mode to avoid recreating
self._agents: dict[PermissionMode, Agent[TaskContext, str]] = {}
self._settings = get_settings()
@property
def agent(self) -> Agent[TaskContext, str]:
"""Default agent (full mode) for compatibility."""
return self._get_agent_for_mode(PermissionMode.default)
def _get_agent_for_mode(self, mode: PermissionMode) -> Agent[TaskContext, str]:
"""Get or create agent configured for the specified mode."""
if mode not in self._agents:
self._agents[mode] = self._create_agent(mode)
return self._agents[mode]
def _create_agent(self, mode: PermissionMode = PermissionMode.default) -> Agent[TaskContext, str]:
"""Create the PydanticAI agent with Ollama backend."""
# Use sanitized Ollama provider to fix content: null issues
model = OpenAIModel(
model_name=self._settings.ollama_agent_model,
provider=get_ollama_provider(),
)
# Select system prompt based on mode
system_prompt = TASK_PLAN_MODE_PROMPT if mode == PermissionMode.plan else TASK_SYSTEM_PROMPT
agent: Agent[TaskContext, str] = Agent(
model=model,
system_prompt=system_prompt,
deps_type=TaskContext,
output_type=str,
# Mistral Nemo settings:
# - temperature 0.3 (Nemo needs slightly higher than 0.0)
# - tool_choice "required" forces tool use
model_settings={
"temperature": 0.3,
"extra_body": {"tool_choice": "required"},
},
)
# Register tools based on mode
self._register_tools(agent, mode)
return agent
def _register_tools(self, agent: Agent[TaskContext, str], mode: PermissionMode) -> None:
"""Register tools with the agent based on permission mode."""
from src.domains.agents.task.tools_streaming import (
register_readonly_tools_streaming,
register_task_tools_streaming,
)
if mode == PermissionMode.plan:
# Plan mode: read-only tools only
register_readonly_tools_streaming(agent)
else:
# Default and auto_accept: all tools
register_task_tools_streaming(agent)
# Maximum retries when no tools are called
MAX_NO_TOOL_RETRIES = 2
def _build_prompt_with_context(self, prompt: str, working_dir: str) -> str:
"""Build the prompt with working directory context."""
return f"""Working directory: {working_dir}
When using file tools, use paths relative to or within this working directory.
For example, to read a file at {working_dir}/README.md, use file_path="{working_dir}/README.md".
User request: {prompt}"""
def _build_retry_prompt(self, prompt: str, working_dir: str) -> str:
"""Build a stronger prompt for retry after no tool calls."""
return f"""Working directory: {working_dir}
IMPORTANT: Your previous response was REJECTED because you did not call any tools.
You MUST call a tool (like glob_files, bash_readonly, or read_file) BEFORE responding.
DO NOT answer from memory. DO NOT fabricate information.
Call a tool NOW to gather real information, then respond based on the results.
User request: {prompt}"""
@logged()
async def run(
self,
prompt: str,
working_dir: str | None = None,
allowed_paths: list[str] | None = None,
mode: PermissionMode = PermissionMode.default,
**kwargs: Any
) -> str:
"""
Run the task agent to execute a multi-step task.
Args:
prompt: Description of the task to execute
working_dir: Working directory for the agent
allowed_paths: Restrict tool access to these paths
mode: Permission mode controlling tool access
Returns:
Consolidated task summary with results
"""
effective_working_dir = working_dir or os.getcwd()
# Get agent configured for this mode
agent = self._get_agent_for_mode(mode)
async with trace_span("task_agent_run"):
retries = 0
while retries <= self.MAX_NO_TOOL_RETRIES:
# Create fresh context for each attempt
ctx = TaskContext(
working_dir=effective_working_dir,
allowed_paths=allowed_paths or self._settings.effective_allowed_paths,
timeout_seconds=self._settings.tool_timeout_seconds,
mode=mode,
)
# Build prompt - use retry prompt if this is a retry
if retries == 0:
full_prompt = self._build_prompt_with_context(prompt, effective_working_dir)
else:
full_prompt = self._build_retry_prompt(prompt, effective_working_dir)
logger.warning(f"Retry {retries}/{self.MAX_NO_TOOL_RETRIES}: No tools called, retrying with stronger prompt")
try:
result = await agent.run(full_prompt, deps=ctx)
# Check if tools were called
if ctx.tools_called == 0 and retries < self.MAX_NO_TOOL_RETRIES:
retries += 1
continue
if ctx.tools_called == 0:
logger.warning("Agent responded without calling tools after all retries")
return result.output
except Exception as e:
logger.exception(f"Task agent error: {e}")
raise
# Should not reach here, but just in case
return result.output
async def run_stream(
self,
prompt: str,
working_dir: str | None = None,
allowed_paths: list[str] | None = None,
mode: PermissionMode = PermissionMode.default,
**kwargs: Any
) -> AsyncIterator[StreamEvent]:
"""
Run the task agent with structured event streaming.
Instead of streaming raw text (which gets garbled during tool calls),
yields structured events that clients can render appropriately.
Args:
prompt: Task description
working_dir: Working directory
allowed_paths: Restrict tool access
mode: Permission mode controlling tool access
Yields:
StreamEvent objects for tool progress and final response.
Event types:
- tool_start: Tool execution beginning
- tool_done: Tool execution complete with summary
- thinking: Agent status update
- response: Final response text
- error: Error occurred
- done: Stream complete
"""
effective_working_dir = working_dir or os.getcwd()
# Get agent configured for this mode
agent = self._get_agent_for_mode(mode)
async with trace_span("task_agent_stream"):
# Emit initial thinking event
yield StreamEvent(
event=StreamEventType.thinking,
message="Starting task execution..."
)
retries = 0
response = ""
while retries <= self.MAX_NO_TOOL_RETRIES:
# Create fresh event queue and context for each attempt
event_queue: asyncio.Queue[StreamEvent] = asyncio.Queue()
ctx = TaskContext(
working_dir=effective_working_dir,
allowed_paths=allowed_paths or self._settings.effective_allowed_paths,
timeout_seconds=self._settings.tool_timeout_seconds,
mode=mode,
event_queue=event_queue,
)
# Build prompt - use retry prompt if this is a retry
if retries == 0:
full_prompt = self._build_prompt_with_context(prompt, effective_working_dir)
else:
full_prompt = self._build_retry_prompt(prompt, effective_working_dir)
yield StreamEvent(
event=StreamEventType.thinking,
message=f"Retrying (attempt {retries + 1})..."
)
# Run agent in background task so we can yield events.
#
# full_prompt and ctx are bound as defaults rather than closed
# over. Today the closure is safe either way — the task is
# awaited below before `continue` reaches the next iteration, so
# neither name can be rebound while it is pending. Binding them
# keeps that true if the await ever moves, which is the failure
# B023 is warning about and the kind that surfaces as one agent
# silently running another's prompt.
async def run_agent(full_prompt: str = full_prompt, ctx: TaskContext = ctx) -> str:
try:
result = await agent.run(full_prompt, deps=ctx)
return result.output
except Exception as e:
logger.exception(f"Task agent stream error: {e}")
raise
agent_task = asyncio.create_task(run_agent())
# Yield events from queue while agent runs
try:
while not agent_task.done():
try:
# Check for events with timeout
event = await asyncio.wait_for(
event_queue.get(),
timeout=0.1
)
yield event
except TimeoutError:
# No events, check if agent is done
continue
# Drain remaining events
while not event_queue.empty():
yield event_queue.get_nowait()
# Get final result
response = await agent_task
# Check if tools were called - if not, retry
if ctx.tools_called == 0 and retries < self.MAX_NO_TOOL_RETRIES:
logger.warning(f"No tools called, retrying ({retries + 1}/{self.MAX_NO_TOOL_RETRIES})")
retries += 1
continue
if ctx.tools_called == 0:
logger.warning("Agent responded without calling tools after all retries")
# Success - break out of retry loop
break
except Exception as e:
logger.exception(f"Stream error: {e}")
yield StreamEvent(
event=StreamEventType.error,
error_message=str(e)
)
# Cancel agent if still running
if not agent_task.done():
agent_task.cancel()
with contextlib.suppress(asyncio.CancelledError):
await agent_task
return
# Yield response in chunks for streaming feel
chunk_size = 100
for i in range(0, len(response), chunk_size):
chunk = response[i:i + chunk_size]
yield StreamEvent(
event=StreamEventType.response,
text=chunk
)
# Small delay for streaming effect
await asyncio.sleep(0.01)
# Signal completion
yield StreamEvent(
event=StreamEventType.done,
mode=mode.value
)
# Create and register the singleton instance
task_agent = TaskAgentImpl()
register_agent(task_agent)
async def task(
prompt: str,
working_dir: str | None = None,
**kwargs: Any
) -> str:
"""Run task execution."""
return await task_agent.run(prompt, working_dir=working_dir, **kwargs)
async def task_stream(
prompt: str,
working_dir: str | None = None,
**kwargs: Any
) -> AsyncIterator[StreamEvent]:
"""Run task execution with event streaming."""
async for event in task_agent.run_stream(prompt, working_dir=working_dir, **kwargs):
yield event
@@ -0,0 +1,157 @@
"""
System prompts for the Task agent.
The Task agent is a full orchestrator that can:
- Execute multi-step tasks autonomously
- Use all tools (read + write) based on permission mode
- Spawn sub-agents (Explore, Plan) for focused work
"""
TASK_PLAN_MODE_PROMPT = """You are a codebase analysis and planning agent in READ-ONLY mode.
CRITICAL RULE: You MUST call a tool BEFORE responding to ANY request.
- NEVER answer from memory or assumptions
- NEVER fabricate file structures, code, or content
- If you respond without calling a tool first, YOUR ANSWER IS WRONG
You can explore and analyze code but CANNOT modify files or execute write operations.
AVAILABLE TOOLS (read-only):
File Operations:
- read_file: Read file contents with line numbers
- glob_files: Find files by pattern
- grep_content: Search file contents with regex
Shell:
- bash_readonly: Read-only commands (ls, git status, git log, git diff, etc.)
Orchestration:
- spawn_agent: Launch sub-agents for focused tasks (explore, plan only)
MANDATORY WORKFLOW:
1. FIRST: Call a tool to gather real information
2. THEN: Analyze the actual tool results
3. FINALLY: Respond based only on what tools returned
TOOL CALL EXAMPLES:
To find all Python files:
Call glob_files with pattern="**/*.py"
To search for a function:
Call grep_content with pattern="def my_function"
To check git status:
Call bash_readonly with command="git status"
To list directory contents:
Call bash_readonly with command="ls -la"
To get deeper analysis:
Call spawn_agent with agent_type="explore" and prompt="find authentication code"
RULES:
- ALWAYS call a tool FIRST - no exceptions
- Never guess or fabricate - only report what tools return
- Be thorough in exploration
- Provide specific file paths and line numbers from tool results
OUTPUT FORMAT:
Structure your response with:
### Analysis
- What was found (from tool results)
- Key patterns identified
- Relevant files (actual paths from tools)
### Recommendations
- Suggested approach
- Potential concerns
- Next steps (to be executed in full mode)
"""
TASK_SYSTEM_PROMPT = """You are an autonomous task execution agent.
CRITICAL RULE: You MUST call a tool BEFORE responding to ANY request.
- NEVER answer from memory or assumptions
- NEVER fabricate file structures, code, or content
- If you respond without calling a tool first, YOUR ANSWER IS WRONG
You have access to ALL tools including file editing, writing, and bash execution.
You can also spawn sub-agents to help with complex tasks.
AVAILABLE TOOLS:
File Operations:
- read_file: Read file contents with line numbers
- glob_files: Find files by pattern
- grep_content: Search file contents with regex
- edit_file: Make targeted edits via find-and-replace
- write_file: Create or overwrite files
Shell:
- bash_readonly: Read-only commands (ls, git status, git log, etc.)
- bash: Full bash execution (git commit, pytest, mkdir, etc.)
External:
- web_search: Search the web for current information
Orchestration:
- spawn_agent: Launch sub-agents for focused tasks
MANDATORY WORKFLOW:
1. FIRST: Call a tool to gather real information
2. THEN: Analyze the actual tool results
3. Execute implementation using write tools if needed
4. Validate changes (run tests if applicable)
5. FINALLY: Return summary based only on what tools returned
TOOL CALL EXAMPLES:
To list directory contents:
Call bash_readonly with command="ls -la"
To find all Python files:
Call glob_files with pattern="**/*.py"
To spawn an Explore agent for research:
Call spawn_agent with agent_type="explore" and prompt="find all config files"
To spawn a Plan agent for design:
Call spawn_agent with agent_type="plan" and prompt="design user auth feature"
To edit a file:
Call edit_file with file_path="/path/to/file.py" and old_string="old" and new_string="new"
To run tests:
Call bash with command="pytest tests/ -v"
SPAWN_AGENT USAGE:
- Use spawn_agent to offload focused tasks to specialized agents
- Explore agent: Fast codebase searches and analysis
- Plan agent: Design implementation strategies
- Keep each agent's context focused and efficient
GIT DISCIPLINE:
- Create feature branches for changes
- Use conventional commit format (feat:, fix:, docs:, etc.)
- Never commit directly to main
- Run tests before committing
RULES:
- ALWAYS call a tool FIRST - no exceptions
- Never guess or fabricate - only report what tools return
- Prefer edit_file over write_file for existing files
- Use spawn_agent to keep context focused
- Validate changes by running tests when applicable
OUTPUT FORMAT:
End your response with a summary:
### Task Summary
- **Accomplished:** What was done
- **Files modified:** List of changed files
- **Commands run:** Key commands executed
- **Issues:** Any problems encountered
"""
+382
View File
@@ -0,0 +1,382 @@
"""
Tool registrations for the Task agent.
The Task agent has access to tools based on permission mode:
- Plan mode: Read-only tools only
- Default/auto_accept: All tools including write operations
"""
from pydantic_ai import Agent, RunContext
from src.domains.agents.task.agent import TaskContext
from src.domains.tools.file.edit import EditFileTool
from src.domains.tools.file.glob import GlobFilesTool
from src.domains.tools.file.read import ReadFileTool
from src.domains.tools.file.write import WriteFileTool
from src.domains.tools.search.grep import GrepContentTool
from src.domains.tools.search.web import WebSearchTool
from src.domains.tools.shell.bash import BashReadOnlyTool
from src.domains.tools.shell.bash_full import BashTool
def _register_read_file(agent: Agent[TaskContext, str]) -> None:
"""Register read_file tool."""
@agent.tool
async def read_file(
ctx: RunContext[TaskContext],
file_path: str,
offset: int = 0,
limit: int = 2000
) -> str:
"""Read contents of a file with line numbers.
Args:
file_path: Absolute path to the file to read
offset: Line number to start from (0-based, default: 0)
limit: Maximum number of lines to read (default: 2000)
Returns:
File contents with line numbers, or error message.
IMPORTANT: Always use absolute paths. Read files before editing them.
"""
tool = ReadFileTool(allowed_paths=ctx.deps.allowed_paths)
result = await tool.execute(
file_path=file_path,
offset=offset,
limit=limit
)
return result.to_string()
def _register_glob_files(agent: Agent[TaskContext, str]) -> None:
"""Register glob_files tool."""
@agent.tool
async def glob_files(
ctx: RunContext[TaskContext],
pattern: str,
path: str | None = None,
limit: int = 100
) -> str:
"""Find files matching a glob pattern.
Args:
pattern: Glob pattern (e.g., "**/*.py", "src/**/*.ts", "*.md")
path: Directory to search in (default: working directory)
limit: Maximum number of files to return (default: 100)
Returns:
List of absolute file paths, sorted by modification time (newest first).
Examples:
- "**/*.py" finds all Python files
- "src/**/*.ts" finds TypeScript files in src/
- "**/test_*.py" finds all test files
"""
tool = GlobFilesTool(allowed_paths=ctx.deps.allowed_paths)
search_path = path or ctx.deps.working_dir
result = await tool.execute(
pattern=pattern,
path=search_path,
limit=limit
)
return result.to_string()
def _register_grep_content(agent: Agent[TaskContext, str]) -> None:
"""Register grep_content tool."""
@agent.tool
async def grep_content(
ctx: RunContext[TaskContext],
pattern: str,
path: str | None = None,
file_glob: str | None = None,
context_lines: int = 0,
case_sensitive: bool = True
) -> str:
"""Search file contents using regex pattern.
Args:
pattern: Regex pattern to search for (Python re syntax)
path: Directory or file to search (default: working directory)
file_glob: Filter files by glob (e.g., "*.py", "*.ts")
context_lines: Lines of context before/after matches (default: 0)
case_sensitive: Case-sensitive search (default: True)
Returns:
Matching lines with file paths and line numbers.
Format: "filepath:line_num: content"
"""
tool = GrepContentTool(allowed_paths=ctx.deps.allowed_paths)
search_path = path or ctx.deps.working_dir
result = await tool.execute(
pattern=pattern,
path=search_path,
file_glob=file_glob,
context_lines=context_lines,
case_sensitive=case_sensitive
)
return result.to_string()
def _register_bash_readonly(agent: Agent[TaskContext, str]) -> None:
"""Register bash_readonly tool."""
@agent.tool
async def bash_readonly(
ctx: RunContext[TaskContext],
command: str,
cwd: str | None = None,
timeout: int = 30
) -> str:
"""Execute a read-only bash command.
ALLOWED commands:
- File inspection: ls, find, cat, head, tail, wc, file, stat, tree, du
- Git (read-only): git status, git log, git diff, git show, git branch
- Text processing: grep, awk, sed (read-only), sort, uniq
- System info: pwd, whoami, hostname, which
FORBIDDEN:
- File modification (rm, mv, cp, mkdir, touch)
- Redirects (>, >>)
- Command chaining (&&, ||, ;)
- Network (curl, wget)
Args:
command: The bash command to execute
cwd: Working directory (default: agent working directory)
timeout: Timeout in seconds (default: 30)
"""
tool = BashReadOnlyTool(allowed_paths=ctx.deps.allowed_paths)
working_dir = cwd or ctx.deps.working_dir
result = await tool.execute(
command=command,
cwd=working_dir,
timeout=min(timeout, ctx.deps.timeout_seconds)
)
return result.to_string()
def _register_spawn_agent(agent: Agent[TaskContext, str], readonly_only: bool = False) -> None:
"""Register spawn_agent tool."""
@agent.tool
async def spawn_agent(
ctx: RunContext[TaskContext],
agent_type: str,
prompt: str,
working_dir: str | None = None
) -> str:
"""Spawn a sub-agent to handle a focused task.
Use this to offload work to specialized agents:
- "explore": Fast codebase searches and analysis (read-only)
- "plan": Design implementation strategies (read-only)
Args:
agent_type: Type of agent to spawn ("explore" or "plan")
prompt: Task description for the sub-agent
working_dir: Working directory for the sub-agent (default: current)
Returns:
Sub-agent's consolidated response.
Examples:
- spawn_agent(agent_type="explore", prompt="find all test files")
- spawn_agent(agent_type="plan", prompt="design user auth feature")
IMPORTANT:
- Use sub-agents to keep context focused and efficient
- Explore agent for research, Plan agent for design
- Cannot spawn nested Task agents (recursion risk)
"""
from src.domains.agents.base import get_agent
# Validate agent type
allowed_types = ["explore", "plan"]
if agent_type not in allowed_types:
if agent_type == "task":
return "Error: Cannot spawn nested Task agents (recursion risk)"
return f"Error: Unknown agent type '{agent_type}'. Allowed: {allowed_types}"
sub_agent = get_agent(agent_type)
if not sub_agent:
return f"Error: Agent '{agent_type}' not found in registry"
try:
result = await sub_agent.run(
prompt=prompt,
working_dir=working_dir or ctx.deps.working_dir,
allowed_paths=ctx.deps.allowed_paths,
)
return result
except Exception as e:
return f"Sub-agent error: {e}"
def register_readonly_tools(agent: Agent[TaskContext, str]) -> None:
"""
Register read-only tools with the agent.
Used in plan mode. Includes:
- read_file, glob_files, grep_content, bash_readonly
- spawn_agent (restricted to explore/plan)
"""
_register_read_file(agent)
_register_glob_files(agent)
_register_grep_content(agent)
_register_bash_readonly(agent)
_register_spawn_agent(agent, readonly_only=True)
def register_task_tools(agent: Agent[TaskContext, str]) -> None:
"""
Register all tools with the Task agent.
Includes:
- Read-only tools: read_file, glob_files, grep_content, bash_readonly
- Write tools: edit_file, write_file, bash
- External: web_search
- Orchestration: spawn_agent
"""
# Register read-only tools via helpers
_register_read_file(agent)
_register_glob_files(agent)
_register_grep_content(agent)
_register_bash_readonly(agent)
# === Write tools ===
@agent.tool
async def edit_file(
ctx: RunContext[TaskContext],
file_path: str,
old_string: str,
new_string: str,
replace_all: bool = False
) -> str:
"""Make targeted edits to a file using find-and-replace.
Args:
file_path: Absolute path to the file to edit
old_string: The exact text to find and replace (must exist in file)
new_string: The replacement text
replace_all: If True, replace all occurrences. If False (default),
old_string must be unique (appear exactly once).
Returns:
Success message with diff preview, or error.
IMPORTANT:
- old_string must exactly match file content (including whitespace)
- By default, old_string must appear exactly once (for safety)
- Always read the file first to verify exact content before editing
"""
tool = EditFileTool(allowed_paths=ctx.deps.allowed_paths)
result = await tool.execute(
file_path=file_path,
old_string=old_string,
new_string=new_string,
replace_all=replace_all
)
return result.to_string()
@agent.tool
async def write_file(
ctx: RunContext[TaskContext],
file_path: str,
content: str
) -> str:
"""Create a new file or overwrite an existing file.
Args:
file_path: Absolute path to the file to create/write
content: The content to write to the file
Returns:
Success message with file path and size.
IMPORTANT:
- Parent directory must exist (use bash mkdir first if needed)
- For editing existing files, prefer edit_file instead
- Will overwrite existing files without confirmation
"""
tool = WriteFileTool(allowed_paths=ctx.deps.allowed_paths)
result = await tool.execute(
file_path=file_path,
content=content
)
return result.to_string()
@agent.tool
async def bash(
ctx: RunContext[TaskContext],
command: str,
cwd: str | None = None,
timeout: int = 60
) -> str:
"""Execute a bash command with write capabilities.
ALLOWED:
- File operations: ls, find, mkdir, touch, cp, mv, rm (single files)
- Git (full): git add, git commit, git checkout, git merge, git pull
- Python: python, pip install, pytest, mypy, ruff
- Text processing: grep, awk, sed, sort
- Command chaining: && and || are allowed
FORBIDDEN:
- sudo, su (privilege escalation)
- Network: curl, wget, ssh, scp, rsync
- Dangerous: rm -rf, chmod 777, dd, mkfs
Args:
command: The bash command to execute
cwd: Working directory (default: agent working directory)
timeout: Timeout in seconds (default: 60)
Examples:
- "mkdir -p src/utils" creates directory
- "git add . && git commit -m 'fix: bug'" commits changes
- "pytest tests/ -v" runs tests
"""
tool = BashTool(allowed_paths=ctx.deps.allowed_paths)
working_dir = cwd or ctx.deps.working_dir
result = await tool.execute(
command=command,
cwd=working_dir,
timeout=min(timeout, ctx.deps.timeout_seconds)
)
return result.to_string()
# === External tools ===
@agent.tool
async def web_search(
ctx: RunContext[TaskContext],
query: str,
num_results: int = 5,
categories: str | None = None
) -> str:
"""Search the web for current information.
Args:
query: Search query (e.g., "Python 3.12 new features")
num_results: Number of results to return (1-10, default: 5)
categories: Optional category filter ("general", "it", "news", "science")
Returns:
Search results with titles, URLs, and snippets.
Use this for:
- Current events or recent information
- Documentation updates
- Technical references with URLs
"""
tool = WebSearchTool()
result = await tool.execute(
query=query,
num_results=num_results,
categories=categories
)
return result.to_string()
# === Orchestration tools ===
_register_spawn_agent(agent, readonly_only=False)
@@ -0,0 +1,556 @@
"""
Tool registrations for the Task agent with event streaming.
Same tools as tools.py but emit StreamEvent events for progress tracking.
Tools push events to the context's event_queue when available.
"""
from pydantic_ai import Agent, RunContext
from src.domains.agents.base import AgentContext
from src.domains.agents.schemas import StreamEvent, StreamEventType
from src.domains.agents.task.agent import TaskContext
from src.domains.tools.file.edit import EditFileTool
from src.domains.tools.file.glob import GlobFilesTool
from src.domains.tools.file.read import ReadFileTool
from src.domains.tools.file.write import WriteFileTool
from src.domains.tools.search.grep import GrepContentTool
from src.domains.tools.search.web import WebSearchTool
from src.domains.tools.shell.bash import BashReadOnlyTool
from src.domains.tools.shell.bash_full import BashTool
def _emit_event(ctx: AgentContext, event: StreamEvent) -> None:
"""Emit an event to the queue if available."""
if hasattr(ctx, 'event_queue') and ctx.event_queue is not None:
ctx.event_queue.put_nowait(event)
def _track_tool_call(ctx: AgentContext) -> None:
"""Increment tool call counter for retry logic."""
if hasattr(ctx, 'tools_called'):
ctx.tools_called += 1
def _summarize_result(result: str, max_len: int = 80) -> str:
"""Create a brief summary of a tool result."""
lines = result.strip().split('\n')
if len(lines) > 3:
return f"{len(lines)} lines"
if len(result) > max_len:
return result[:max_len] + "..."
return result.replace('\n', ' ')
def _register_read_file(agent: Agent[TaskContext, str]) -> None:
"""Register read_file tool with event streaming."""
@agent.tool
async def read_file(
ctx: RunContext[TaskContext],
file_path: str,
offset: int = 0,
limit: int = 2000
) -> str:
"""Read contents of a file with line numbers.
Args:
file_path: Absolute path to the file to read
offset: Line number to start from (0-based, default: 0)
limit: Maximum number of lines to read (default: 2000)
Returns:
File contents with line numbers, or error message.
IMPORTANT: Always use absolute paths. Read files before editing them.
"""
_track_tool_call(ctx.deps)
_emit_event(ctx.deps, StreamEvent(
event=StreamEventType.tool_start,
tool="read_file",
args={"file_path": file_path, "offset": offset, "limit": limit}
))
tool = ReadFileTool(allowed_paths=ctx.deps.allowed_paths)
result = await tool.execute(
file_path=file_path,
offset=offset,
limit=limit
)
result_str = result.to_string()
_emit_event(ctx.deps, StreamEvent(
event=StreamEventType.tool_done,
tool="read_file",
result_summary=_summarize_result(result_str)
))
return result_str
def _register_glob_files(agent: Agent[TaskContext, str]) -> None:
"""Register glob_files tool with event streaming."""
@agent.tool
async def glob_files(
ctx: RunContext[TaskContext],
pattern: str,
path: str | None = None,
limit: int = 100
) -> str:
"""Find files matching a glob pattern.
Args:
pattern: Glob pattern (e.g., "**/*.py", "src/**/*.ts", "*.md")
path: Directory to search in (default: working directory)
limit: Maximum number of files to return (default: 100)
Returns:
List of absolute file paths, sorted by modification time (newest first).
Examples:
- "**/*.py" finds all Python files
- "src/**/*.ts" finds TypeScript files in src/
- "**/test_*.py" finds all test files
"""
_track_tool_call(ctx.deps)
_emit_event(ctx.deps, StreamEvent(
event=StreamEventType.tool_start,
tool="glob_files",
args={"pattern": pattern, "path": path}
))
tool = GlobFilesTool(allowed_paths=ctx.deps.allowed_paths)
search_path = path or ctx.deps.working_dir
result = await tool.execute(
pattern=pattern,
path=search_path,
limit=limit
)
result_str = result.to_string()
_emit_event(ctx.deps, StreamEvent(
event=StreamEventType.tool_done,
tool="glob_files",
result_summary=_summarize_result(result_str)
))
return result_str
def _register_grep_content(agent: Agent[TaskContext, str]) -> None:
"""Register grep_content tool with event streaming."""
@agent.tool
async def grep_content(
ctx: RunContext[TaskContext],
pattern: str,
path: str | None = None,
file_glob: str | None = None,
context_lines: int = 0,
case_sensitive: bool = True
) -> str:
"""Search file contents using regex pattern.
Args:
pattern: Regex pattern to search for (Python re syntax)
path: Directory or file to search (default: working directory)
file_glob: Filter files by glob (e.g., "*.py", "*.ts")
context_lines: Lines of context before/after matches (default: 0)
case_sensitive: Case-sensitive search (default: True)
Returns:
Matching lines with file paths and line numbers.
Format: "filepath:line_num: content"
"""
_track_tool_call(ctx.deps)
_emit_event(ctx.deps, StreamEvent(
event=StreamEventType.tool_start,
tool="grep_content",
args={"pattern": pattern, "path": path, "file_glob": file_glob}
))
tool = GrepContentTool(allowed_paths=ctx.deps.allowed_paths)
search_path = path or ctx.deps.working_dir
result = await tool.execute(
pattern=pattern,
path=search_path,
file_glob=file_glob,
context_lines=context_lines,
case_sensitive=case_sensitive
)
result_str = result.to_string()
_emit_event(ctx.deps, StreamEvent(
event=StreamEventType.tool_done,
tool="grep_content",
result_summary=_summarize_result(result_str)
))
return result_str
def _register_bash_readonly(agent: Agent[TaskContext, str]) -> None:
"""Register bash_readonly tool with event streaming."""
@agent.tool
async def bash_readonly(
ctx: RunContext[TaskContext],
command: str,
cwd: str | None = None,
timeout: int = 30
) -> str:
"""Execute a read-only bash command.
ALLOWED commands:
- File inspection: ls, find, cat, head, tail, wc, file, stat, tree, du
- Git (read-only): git status, git log, git diff, git show, git branch
- Text processing: grep, awk, sed (read-only), sort, uniq
- System info: pwd, whoami, hostname, which
FORBIDDEN:
- File modification (rm, mv, cp, mkdir, touch)
- Redirects (>, >>)
- Command chaining (&&, ||, ;)
- Network (curl, wget)
Args:
command: The bash command to execute
cwd: Working directory (default: agent working directory)
timeout: Timeout in seconds (default: 30)
"""
_track_tool_call(ctx.deps)
_emit_event(ctx.deps, StreamEvent(
event=StreamEventType.tool_start,
tool="bash_readonly",
args={"command": command}
))
tool = BashReadOnlyTool(allowed_paths=ctx.deps.allowed_paths)
working_dir = cwd or ctx.deps.working_dir
result = await tool.execute(
command=command,
cwd=working_dir,
timeout=min(timeout, ctx.deps.timeout_seconds)
)
result_str = result.to_string()
_emit_event(ctx.deps, StreamEvent(
event=StreamEventType.tool_done,
tool="bash_readonly",
result_summary=_summarize_result(result_str)
))
return result_str
def _register_spawn_agent(agent: Agent[TaskContext, str], readonly_only: bool = False) -> None:
"""Register spawn_agent tool with event streaming."""
@agent.tool
async def spawn_agent(
ctx: RunContext[TaskContext],
agent_type: str,
prompt: str,
working_dir: str | None = None
) -> str:
"""Spawn a sub-agent to handle a focused task.
Use this to offload work to specialized agents:
- "explore": Fast codebase searches and analysis (read-only)
- "plan": Design implementation strategies (read-only)
Args:
agent_type: Type of agent to spawn ("explore" or "plan")
prompt: Task description for the sub-agent
working_dir: Working directory for the sub-agent (default: current)
Returns:
Sub-agent's consolidated response.
Examples:
- spawn_agent(agent_type="explore", prompt="find all test files")
- spawn_agent(agent_type="plan", prompt="design user auth feature")
IMPORTANT:
- Use sub-agents to keep context focused and efficient
- Explore agent for research, Plan agent for design
- Cannot spawn nested Task agents (recursion risk)
"""
from src.domains.agents.base import get_agent
_track_tool_call(ctx.deps)
_emit_event(ctx.deps, StreamEvent(
event=StreamEventType.tool_start,
tool="spawn_agent",
args={"agent_type": agent_type, "prompt": prompt[:50] + "..." if len(prompt) > 50 else prompt}
))
# Validate agent type
allowed_types = ["explore", "plan"]
if agent_type not in allowed_types:
if agent_type == "task":
result = "Error: Cannot spawn nested Task agents (recursion risk)"
else:
result = f"Error: Unknown agent type '{agent_type}'. Allowed: {allowed_types}"
_emit_event(ctx.deps, StreamEvent(
event=StreamEventType.tool_done,
tool="spawn_agent",
result_summary=result
))
return result
sub_agent = get_agent(agent_type)
if not sub_agent:
result = f"Error: Agent '{agent_type}' not found in registry"
_emit_event(ctx.deps, StreamEvent(
event=StreamEventType.tool_done,
tool="spawn_agent",
result_summary=result
))
return result
try:
result = await sub_agent.run(
prompt=prompt,
working_dir=working_dir or ctx.deps.working_dir,
allowed_paths=ctx.deps.allowed_paths,
)
_emit_event(ctx.deps, StreamEvent(
event=StreamEventType.tool_done,
tool="spawn_agent",
result_summary=_summarize_result(result)
))
return result
except Exception as e:
result = f"Sub-agent error: {e}"
_emit_event(ctx.deps, StreamEvent(
event=StreamEventType.tool_done,
tool="spawn_agent",
result_summary=result
))
return result
def register_readonly_tools_streaming(agent: Agent[TaskContext, str]) -> None:
"""
Register read-only tools with event streaming.
Used in plan mode. Includes:
- read_file, glob_files, grep_content, bash_readonly
- spawn_agent (restricted to explore/plan)
"""
_register_read_file(agent)
_register_glob_files(agent)
_register_grep_content(agent)
_register_bash_readonly(agent)
_register_spawn_agent(agent, readonly_only=True)
def register_task_tools_streaming(agent: Agent[TaskContext, str]) -> None:
"""
Register all tools with event streaming.
Includes:
- Read-only tools: read_file, glob_files, grep_content, bash_readonly
- Write tools: edit_file, write_file, bash
- External: web_search
- Orchestration: spawn_agent
"""
# Register read-only tools via helpers
_register_read_file(agent)
_register_glob_files(agent)
_register_grep_content(agent)
_register_bash_readonly(agent)
# === Write tools ===
@agent.tool
async def edit_file(
ctx: RunContext[TaskContext],
file_path: str,
old_string: str,
new_string: str,
replace_all: bool = False
) -> str:
"""Make targeted edits to a file using find-and-replace.
Args:
file_path: Absolute path to the file to edit
old_string: The exact text to find and replace (must exist in file)
new_string: The replacement text
replace_all: If True, replace all occurrences. If False (default),
old_string must be unique (appear exactly once).
Returns:
Success message with diff preview, or error.
IMPORTANT:
- old_string must exactly match file content (including whitespace)
- By default, old_string must appear exactly once (for safety)
- Always read the file first to verify exact content before editing
"""
_track_tool_call(ctx.deps)
_emit_event(ctx.deps, StreamEvent(
event=StreamEventType.tool_start,
tool="edit_file",
args={"file_path": file_path, "replace_all": replace_all}
))
tool = EditFileTool(allowed_paths=ctx.deps.allowed_paths)
result = await tool.execute(
file_path=file_path,
old_string=old_string,
new_string=new_string,
replace_all=replace_all
)
result_str = result.to_string()
_emit_event(ctx.deps, StreamEvent(
event=StreamEventType.tool_done,
tool="edit_file",
result_summary=_summarize_result(result_str)
))
return result_str
@agent.tool
async def write_file(
ctx: RunContext[TaskContext],
file_path: str,
content: str
) -> str:
"""Create a new file or overwrite an existing file.
Args:
file_path: Absolute path to the file to create/write
content: The content to write to the file
Returns:
Success message with file path and size.
IMPORTANT:
- Parent directory must exist (use bash mkdir first if needed)
- For editing existing files, prefer edit_file instead
- Will overwrite existing files without confirmation
"""
_track_tool_call(ctx.deps)
_emit_event(ctx.deps, StreamEvent(
event=StreamEventType.tool_start,
tool="write_file",
args={"file_path": file_path, "content_length": len(content)}
))
tool = WriteFileTool(allowed_paths=ctx.deps.allowed_paths)
result = await tool.execute(
file_path=file_path,
content=content
)
result_str = result.to_string()
_emit_event(ctx.deps, StreamEvent(
event=StreamEventType.tool_done,
tool="write_file",
result_summary=_summarize_result(result_str)
))
return result_str
@agent.tool
async def bash(
ctx: RunContext[TaskContext],
command: str,
cwd: str | None = None,
timeout: int = 60
) -> str:
"""Execute a bash command with write capabilities.
ALLOWED:
- File operations: ls, find, mkdir, touch, cp, mv, rm (single files)
- Git (full): git add, git commit, git checkout, git merge, git pull
- Python: python, pip install, pytest, mypy, ruff
- Text processing: grep, awk, sed, sort
- Command chaining: && and || are allowed
FORBIDDEN:
- sudo, su (privilege escalation)
- Network: curl, wget, ssh, scp, rsync
- Dangerous: rm -rf, chmod 777, dd, mkfs
Args:
command: The bash command to execute
cwd: Working directory (default: agent working directory)
timeout: Timeout in seconds (default: 60)
Examples:
- "mkdir -p src/utils" creates directory
- "git add . && git commit -m 'fix: bug'" commits changes
- "pytest tests/ -v" runs tests
"""
_track_tool_call(ctx.deps)
_emit_event(ctx.deps, StreamEvent(
event=StreamEventType.tool_start,
tool="bash",
args={"command": command}
))
tool = BashTool(allowed_paths=ctx.deps.allowed_paths)
working_dir = cwd or ctx.deps.working_dir
result = await tool.execute(
command=command,
cwd=working_dir,
timeout=min(timeout, ctx.deps.timeout_seconds)
)
result_str = result.to_string()
_emit_event(ctx.deps, StreamEvent(
event=StreamEventType.tool_done,
tool="bash",
result_summary=_summarize_result(result_str)
))
return result_str
# === External tools ===
@agent.tool
async def web_search(
ctx: RunContext[TaskContext],
query: str,
num_results: int = 5,
categories: str | None = None
) -> str:
"""Search the web for current information.
Args:
query: Search query (e.g., "Python 3.12 new features")
num_results: Number of results to return (1-10, default: 5)
categories: Optional category filter ("general", "it", "news", "science")
Returns:
Search results with titles, URLs, and snippets.
Use this for:
- Current events or recent information
- Documentation updates
- Technical references with URLs
"""
_track_tool_call(ctx.deps)
_emit_event(ctx.deps, StreamEvent(
event=StreamEventType.tool_start,
tool="web_search",
args={"query": query}
))
tool = WebSearchTool()
result = await tool.execute(
query=query,
num_results=num_results,
categories=categories
)
result_str = result.to_string()
_emit_event(ctx.deps, StreamEvent(
event=StreamEventType.tool_done,
tool="web_search",
result_summary=_summarize_result(result_str)
))
return result_str
# === Orchestration tools ===
_register_spawn_agent(agent, readonly_only=False)
@@ -0,0 +1,16 @@
"""
Conversations domain - Multi-turn conversation management.
Provides:
- Conversation persistence with message history
- Context summarization when approaching token limits
- Agent integration with conversation context injection
"""
from src.domains.conversations.models import Conversation, Message
from src.domains.conversations.service import ConversationService
__all__ = [
"Conversation",
"ConversationService",
"Message",
]
@@ -0,0 +1,72 @@
"""
Database models for conversations.
Following core-api patterns: SQLAlchemy 2.0 with async support.
"""
from datetime import datetime
from uuid import UUID, uuid4
from sqlalchemy import ForeignKey, String, Text
from sqlalchemy.orm import Mapped, mapped_column, relationship
from src.db.models import Base
class Conversation(Base):
"""
A conversation session with an agent.
Tracks message history, token usage, and metadata.
"""
__tablename__ = "conversations"
id: Mapped[UUID] = mapped_column(primary_key=True, default=uuid4)
user_id: Mapped[str] = mapped_column(String(255), index=True)
agent_type: Mapped[str] = mapped_column(String(50), default="explore", insert_default="explore")
title: Mapped[str | None] = mapped_column(String(255), nullable=True, default=None)
working_dir: Mapped[str] = mapped_column(String(1024), default=".", insert_default=".")
total_tokens: Mapped[int] = mapped_column(default=0, insert_default=0)
created_at: Mapped[datetime] = mapped_column(default=datetime.utcnow)
updated_at: Mapped[datetime | None] = mapped_column(
default=datetime.utcnow,
onupdate=datetime.utcnow,
nullable=True
)
# Relationships
messages: Mapped[list["Message"]] = relationship(
back_populates="conversation",
cascade="all, delete-orphan",
order_by="Message.created_at",
)
def __repr__(self) -> str:
return f"<Conversation {self.id} agent={self.agent_type}>"
class Message(Base):
"""
A single message in a conversation.
Tracks role, content, token count, and summarization state.
"""
__tablename__ = "messages"
id: Mapped[UUID] = mapped_column(primary_key=True, default=uuid4)
conversation_id: Mapped[UUID] = mapped_column(
ForeignKey("conversations.id", ondelete="CASCADE"),
index=True
)
role: Mapped[str] = mapped_column(String(20)) # user, assistant, system, summary
content: Mapped[str] = mapped_column(Text)
token_count: Mapped[int] = mapped_column(default=0, insert_default=0)
is_summary: Mapped[bool] = mapped_column(default=False, insert_default=False)
summarizes_up_to: Mapped[UUID | None] = mapped_column(nullable=True, default=None)
created_at: Mapped[datetime] = mapped_column(default=datetime.utcnow)
# Relationships
conversation: Mapped["Conversation"] = relationship(back_populates="messages")
def __repr__(self) -> str:
preview = self.content[:30] + "..." if len(self.content) > 30 else self.content
return f"<Message {self.role}: {preview}>"
@@ -0,0 +1,239 @@
"""
REST API routes for conversations.
"""
from uuid import UUID
from fastapi import APIRouter, Depends, HTTPException
from sqlalchemy.ext.asyncio import AsyncSession
from src.db import get_session
from src.domains.conversations.schemas import (
AddMessageRequest,
AddMessageResponse,
ConversationDetailResponse,
ConversationListResponse,
ConversationResponse,
CreateConversationRequest,
MessageResponse,
SaveMessagesRequest,
SaveMessagesResponse,
)
from src.domains.conversations.service import ConversationService
from src.shared.auth import require_auth
from src.shared.logging import get_logger, logged
logger = get_logger(__name__)
router = APIRouter(prefix="/conversations", tags=["Conversations"])
@router.post("/", response_model=ConversationResponse, status_code=201)
@logged()
async def create_conversation(
request: CreateConversationRequest,
session: AsyncSession = Depends(get_session),
user=Depends(require_auth),
) -> ConversationResponse:
"""
Create a new conversation.
Starts an empty conversation with the specified agent type.
"""
service = ConversationService(session)
conversation = await service.create(
user_id=user.id,
agent_type=request.agent_type,
working_dir=request.working_dir,
title=request.title,
)
return ConversationResponse.model_validate(conversation)
@router.get("/", response_model=ConversationListResponse)
@logged()
async def list_conversations(
limit: int = 50,
offset: int = 0,
session: AsyncSession = Depends(get_session),
user=Depends(require_auth),
) -> ConversationListResponse:
"""
List user's conversations.
Returns conversations sorted by most recently updated.
"""
service = ConversationService(session)
conversations, total = await service.list_by_user(
user_id=user.id,
limit=limit,
offset=offset,
)
return ConversationListResponse(
conversations=[ConversationResponse.model_validate(c) for c in conversations],
total=total,
)
@router.get("/{conversation_id}", response_model=ConversationDetailResponse)
@logged()
async def get_conversation(
conversation_id: UUID,
session: AsyncSession = Depends(get_session),
user=Depends(require_auth),
) -> ConversationDetailResponse:
"""
Get conversation with all messages.
Returns conversation metadata and full message history.
"""
service = ConversationService(session)
conversation = await service.get_with_messages(conversation_id)
if not conversation:
raise HTTPException(status_code=404, detail="Conversation not found")
if conversation.user_id != user.id:
raise HTTPException(status_code=403, detail="Not authorized")
return ConversationDetailResponse.model_validate(conversation)
@router.delete("/{conversation_id}", status_code=204)
@logged()
async def delete_conversation(
conversation_id: UUID,
session: AsyncSession = Depends(get_session),
user=Depends(require_auth),
) -> None:
"""
Delete a conversation and all its messages.
"""
service = ConversationService(session)
conversation = await service.get(conversation_id)
if not conversation:
raise HTTPException(status_code=404, detail="Conversation not found")
if conversation.user_id != user.id:
raise HTTPException(status_code=403, detail="Not authorized")
await service.delete(conversation_id)
@router.post("/{conversation_id}/messages", response_model=AddMessageResponse)
@logged()
async def add_message(
conversation_id: UUID,
request: AddMessageRequest,
session: AsyncSession = Depends(get_session),
user=Depends(require_auth),
) -> AddMessageResponse:
"""
Add a message to a conversation and get agent response.
This is the main endpoint for continuing conversations.
It:
1. Adds the user message
2. Checks if summarization is needed
3. Builds context from conversation history
4. Gets agent response
5. Adds agent response to conversation
6. Returns both messages
"""
service = ConversationService(session)
# Verify conversation exists and user owns it
conversation = await service.get(conversation_id)
if not conversation:
raise HTTPException(status_code=404, detail="Conversation not found")
if conversation.user_id != user.id:
raise HTTPException(status_code=403, detail="Not authorized")
# Add user message
user_message = await service.add_message(
conversation_id=conversation_id,
role="user",
content=request.content,
)
# Check if summarization needed before getting response
summarized = await service.summarize_if_needed(conversation_id)
# Get agent response with context
try:
response_text = await service.get_agent_response(
conversation_id=conversation_id,
user_message=request.content,
)
except Exception as e:
logger.exception(f"Agent response failed: {e}")
raise HTTPException(
status_code=500,
detail=f"Agent error: {e!s}"
)
# Add assistant message
assistant_message = await service.add_message(
conversation_id=conversation_id,
role="assistant",
content=response_text,
)
# Get updated conversation for total tokens
conversation = await service.get(conversation_id)
return AddMessageResponse(
user_message=MessageResponse.model_validate(user_message),
assistant_message=MessageResponse.model_validate(assistant_message),
total_tokens=conversation.total_tokens if conversation else 0,
summarized=summarized,
)
@router.post("/{conversation_id}/save", response_model=SaveMessagesResponse)
@logged()
async def save_messages(
conversation_id: UUID,
request: SaveMessagesRequest,
session: AsyncSession = Depends(get_session),
user=Depends(require_auth),
) -> SaveMessagesResponse:
"""
Save a user/assistant message pair without triggering agent execution.
Used by CLI when streaming responses separately via /agents/stream.
This allows persisting the exchange after streaming completes.
"""
service = ConversationService(session)
# Verify conversation exists and user owns it
conversation = await service.get(conversation_id)
if not conversation:
raise HTTPException(status_code=404, detail="Conversation not found")
if conversation.user_id != user.id:
raise HTTPException(status_code=403, detail="Not authorized")
# Save user message
user_message = await service.add_message(
conversation_id=conversation_id,
role="user",
content=request.user_content,
)
# Save assistant message
assistant_message = await service.add_message(
conversation_id=conversation_id,
role="assistant",
content=request.assistant_content,
)
# Get updated conversation for total tokens
conversation = await service.get(conversation_id)
return SaveMessagesResponse(
user_message=MessageResponse.model_validate(user_message),
assistant_message=MessageResponse.model_validate(assistant_message),
total_tokens=conversation.total_tokens if conversation else 0,
)
@@ -0,0 +1,94 @@
"""
Pydantic schemas for conversation API.
"""
from datetime import datetime
from uuid import UUID
from pydantic import BaseModel, Field
# === Request Schemas ===
class CreateConversationRequest(BaseModel):
"""Request to create a new conversation."""
agent_type: str = Field(default="explore", description="Agent type to use")
working_dir: str = Field(default=".", description="Working directory for agent")
title: str | None = Field(default=None, description="Optional conversation title")
class AddMessageRequest(BaseModel):
"""Request to add a message to a conversation."""
content: str = Field(..., min_length=1, description="Message content")
class SaveMessagesRequest(BaseModel):
"""Request to save a message pair without triggering agent execution.
Used by CLI when streaming responses separately via /agents/stream.
"""
user_content: str = Field(..., min_length=1, description="User message content")
assistant_content: str = Field(..., min_length=1, description="Assistant response content")
# === Response Schemas ===
class MessageResponse(BaseModel):
"""Response for a single message."""
id: UUID
role: str
content: str
token_count: int
is_summary: bool
created_at: datetime
model_config = {"from_attributes": True}
class ConversationResponse(BaseModel):
"""Response for conversation metadata."""
id: UUID
agent_type: str
title: str | None
working_dir: str
total_tokens: int
created_at: datetime
updated_at: datetime | None
model_config = {"from_attributes": True}
class ConversationDetailResponse(BaseModel):
"""Response for conversation with messages."""
id: UUID
agent_type: str
title: str | None
working_dir: str
total_tokens: int
created_at: datetime
updated_at: datetime | None
messages: list[MessageResponse]
model_config = {"from_attributes": True}
class ConversationListResponse(BaseModel):
"""Response for listing conversations."""
conversations: list[ConversationResponse]
total: int
class AddMessageResponse(BaseModel):
"""Response after adding a message (includes agent response)."""
user_message: MessageResponse
assistant_message: MessageResponse
total_tokens: int
summarized: bool = Field(
default=False,
description="Whether context was summarized due to token limit"
)
class SaveMessagesResponse(BaseModel):
"""Response after saving messages (no agent execution)."""
user_message: MessageResponse
assistant_message: MessageResponse
total_tokens: int
@@ -0,0 +1,354 @@
"""
Conversation service - Business logic for conversation management.
Handles CRUD operations, context building, and summarization triggers.
"""
from uuid import UUID
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.orm import selectinload
from src.domains.agents.base import get_agent
from src.domains.conversations.models import Conversation, Message
from src.domains.conversations.summarize import generate_summary
from src.shared.config import get_settings
from src.shared.logging import get_logger
from src.shared.tokens import count_tokens
logger = get_logger(__name__)
class ConversationService:
"""
Service for managing conversations and messages.
Handles:
- CRUD operations for conversations and messages
- Context building for agent prompts
- Automatic summarization when approaching token limits
"""
def __init__(self, session: AsyncSession):
"""
Initialize with database session.
Args:
session: Async SQLAlchemy session
"""
self.session = session
self.settings = get_settings()
# === Conversation CRUD ===
async def create(
self,
user_id: str,
agent_type: str = "explore",
working_dir: str = ".",
title: str | None = None,
) -> Conversation:
"""
Create a new conversation.
Args:
user_id: Owner's user ID
agent_type: Type of agent for this conversation
working_dir: Working directory for agent
title: Optional title (auto-generated from first message if None)
Returns:
Created Conversation object
"""
conversation = Conversation(
user_id=user_id,
agent_type=agent_type,
working_dir=working_dir,
title=title,
)
self.session.add(conversation)
await self.session.flush()
logger.info(f"Created conversation {conversation.id} for user {user_id}")
return conversation
async def get(self, conversation_id: UUID) -> Conversation | None:
"""Get conversation by ID without messages."""
result = await self.session.execute(
select(Conversation).where(Conversation.id == conversation_id)
)
return result.scalar_one_or_none()
async def get_with_messages(self, conversation_id: UUID) -> Conversation | None:
"""Get conversation by ID with messages loaded."""
result = await self.session.execute(
select(Conversation)
.options(selectinload(Conversation.messages))
.where(Conversation.id == conversation_id)
)
return result.scalar_one_or_none()
async def list_by_user(
self,
user_id: str,
limit: int = 50,
offset: int = 0,
) -> tuple[list[Conversation], int]:
"""
List conversations for a user.
Args:
user_id: User ID to filter by
limit: Maximum results to return
offset: Offset for pagination
Returns:
Tuple of (conversations, total_count)
"""
# Get total count
count_result = await self.session.execute(
select(func.count(Conversation.id))
.where(Conversation.user_id == user_id)
)
total = count_result.scalar() or 0
# Get conversations
result = await self.session.execute(
select(Conversation)
.where(Conversation.user_id == user_id)
.order_by(Conversation.updated_at.desc())
.limit(limit)
.offset(offset)
)
conversations = list(result.scalars().all())
return conversations, total
async def delete(self, conversation_id: UUID) -> bool:
"""Delete a conversation and all its messages."""
conversation = await self.get(conversation_id)
if conversation:
await self.session.delete(conversation)
logger.info(f"Deleted conversation {conversation_id}")
return True
return False
# === Message Operations ===
async def add_message(
self,
conversation_id: UUID,
role: str,
content: str,
) -> Message:
"""
Add a message to a conversation.
Args:
conversation_id: Conversation to add to
role: Message role (user, assistant, system, summary)
content: Message content
Returns:
Created Message object
"""
# Count tokens
token_count = count_tokens(content)
message = Message(
conversation_id=conversation_id,
role=role,
content=content,
token_count=token_count,
)
self.session.add(message)
# Update conversation total tokens
conversation = await self.get(conversation_id)
if conversation:
conversation.total_tokens += token_count
# Auto-generate title from first user message
if conversation.title is None and role == "user":
conversation.title = content[:100] + ("..." if len(content) > 100 else "")
await self.session.flush()
return message
# === Context Building ===
def build_context_prompt(
self,
messages: list[Message],
current_message: str,
) -> str:
"""
Build a prompt with conversation context.
Includes summary (if exists) and recent messages.
Args:
messages: All conversation messages
current_message: The current user message
Returns:
Formatted prompt with context
"""
parts = []
# Find most recent summary
summaries = [m for m in messages if m.is_summary]
if summaries:
latest_summary = summaries[-1]
parts.append(
f"<conversation_summary>\n{latest_summary.content}\n</conversation_summary>"
)
# Get recent non-summary messages
recent = [m for m in messages if not m.is_summary]
keep_count = self.settings.keep_recent_messages
recent = recent[-keep_count:] if len(recent) > keep_count else recent
if recent:
parts.append("<recent_conversation>")
for msg in recent:
role_label = msg.role.upper()
parts.append(f"{role_label}: {msg.content}")
parts.append("</recent_conversation>")
# Add current message
parts.append(f"<current_request>\n{current_message}\n</current_request>")
return "\n\n".join(parts)
# === Agent Integration ===
async def get_agent_response(
self,
conversation_id: UUID,
user_message: str,
) -> str:
"""
Get agent response with conversation context.
Args:
conversation_id: Conversation ID
user_message: Current user message
Returns:
Agent's response text
"""
conversation = await self.get_with_messages(conversation_id)
if not conversation:
raise ValueError(f"Conversation {conversation_id} not found")
agent = get_agent(conversation.agent_type)
if not agent:
raise ValueError(f"Unknown agent type: {conversation.agent_type}")
# Build context prompt
context_prompt = self.build_context_prompt(
conversation.messages,
user_message,
)
# Run agent
response = await agent.run(
context_prompt,
working_dir=conversation.working_dir,
)
return response
# === Summarization ===
async def should_summarize(self, conversation_id: UUID) -> bool:
"""
Check if conversation needs summarization.
Args:
conversation_id: Conversation to check
Returns:
True if summarization should be triggered
"""
conversation = await self.get(conversation_id)
if not conversation:
return False
threshold = self.settings.max_context_tokens * self.settings.summarization_threshold
return conversation.total_tokens > threshold
async def summarize_if_needed(self, conversation_id: UUID) -> bool:
"""
Summarize old messages if approaching token limit.
Args:
conversation_id: Conversation to check and potentially summarize
Returns:
True if summarization was performed
"""
if not await self.should_summarize(conversation_id):
return False
conversation = await self.get_with_messages(conversation_id)
if not conversation:
return False
messages = conversation.messages
keep_count = self.settings.keep_recent_messages
# Don't summarize if not enough messages
if len(messages) <= keep_count + 1:
return False
# Get messages to summarize (exclude recent and existing summaries)
non_summary_msgs = [m for m in messages if not m.is_summary]
to_summarize = non_summary_msgs[:-keep_count]
if not to_summarize:
return False
logger.info(
f"Summarizing {len(to_summarize)} messages in conversation {conversation_id}"
)
# Generate summary
summary_text = await generate_summary(
to_summarize,
working_dir=conversation.working_dir,
)
# Get ID of last summarized message
last_summarized_id = to_summarize[-1].id
# Calculate tokens being removed
removed_tokens = sum(m.token_count for m in to_summarize)
summary_tokens = count_tokens(summary_text)
# Add summary message
summary_message = Message(
conversation_id=conversation_id,
role="summary",
content=summary_text,
token_count=summary_tokens,
is_summary=True,
summarizes_up_to=last_summarized_id,
)
self.session.add(summary_message)
# Mark old messages as summarized (soft delete by excluding from context)
for msg in to_summarize:
msg.is_summary = True # Reuse flag to mark as "summarized away"
# Update conversation token count
conversation.total_tokens = conversation.total_tokens - removed_tokens + summary_tokens
await self.session.flush()
logger.info(
f"Summarization complete: removed {removed_tokens} tokens, "
f"added {summary_tokens} token summary"
)
return True
@@ -0,0 +1,103 @@
"""
Context summarization for conversations.
Compresses old messages when approaching token limits.
"""
from src.domains.conversations.models import Message
from src.shared.logging import get_logger
logger = get_logger(__name__)
SUMMARIZE_PROMPT = """Summarize this conversation history concisely for context preservation.
Focus on:
- Key decisions made and their rationale
- Important files, functions, or code discussed
- Current task state and progress
- Any unresolved questions or blockers
- Technical details that would be needed to continue the work
Keep the summary under 500 words. Be factual and technical, not conversational.
Preserve specific file paths, function names, and code references.
CONVERSATION HISTORY:
{history}
CONCISE SUMMARY:"""
def format_messages_for_summary(messages: list[Message]) -> str:
"""
Format messages into a string for summarization.
Args:
messages: List of Message objects to format
Returns:
Formatted conversation string
"""
parts = []
for msg in messages:
if msg.is_summary:
parts.append(f"[Previous Summary]: {msg.content}")
else:
role = msg.role.upper()
parts.append(f"{role}: {msg.content}")
return "\n\n".join(parts)
async def generate_summary(
messages: list[Message],
working_dir: str = "."
) -> str:
"""
Generate a summary of conversation messages using the Explore agent.
Args:
messages: Messages to summarize
working_dir: Working directory for agent context
Returns:
Summary text
"""
from src.domains.agents.explore import explore
history = format_messages_for_summary(messages)
prompt = SUMMARIZE_PROMPT.format(history=history)
logger.info(f"Generating summary for {len(messages)} messages")
try:
summary = await explore(prompt, working_dir=working_dir)
return summary.strip()
except Exception as e:
logger.error(f"Summary generation failed: {e}")
# Fallback: create a simple truncated summary
return _fallback_summary(messages)
def _fallback_summary(messages: list[Message]) -> str:
"""
Create a simple fallback summary if agent summarization fails.
Args:
messages: Messages to summarize
Returns:
Basic summary string
"""
# Take first and last few messages
if len(messages) <= 4:
return format_messages_for_summary(messages)
first_two = messages[:2]
last_two = messages[-2:]
parts = [
"Conversation started with:",
format_messages_for_summary(first_two),
f"\n[... {len(messages) - 4} messages omitted ...]\n",
"Most recent exchange:",
format_messages_for_summary(last_two),
]
return "\n".join(parts)
+5 -1
View File
@@ -6,8 +6,9 @@ main.py only includes this root_router.
""" """
from fastapi import APIRouter from fastapi import APIRouter
from src.domains.health.router import router as health_router
from src.domains.agents.router import router as agents_router from src.domains.agents.router import router as agents_router
from src.domains.conversations.router import router as conversations_router
from src.domains.health.router import router as health_router
# from src.domains.auth.router import router as auth_router # from src.domains.auth.router import router as auth_router
# from src.domains.tools.router import router as tools_router # from src.domains.tools.router import router as tools_router
@@ -20,6 +21,9 @@ root_router.include_router(health_router)
# Agents domain (prefix defined in router) # Agents domain (prefix defined in router)
root_router.include_router(agents_router) root_router.include_router(agents_router)
# Conversations domain (prefix defined in router)
root_router.include_router(conversations_router)
# Auth domain # Auth domain
# root_router.include_router(auth_router, prefix="/auth", tags=["Auth"]) # root_router.include_router(auth_router, prefix="/auth", tags=["Auth"])
+10 -6
View File
@@ -4,15 +4,19 @@ Tool implementations for agent use.
All tools inherit from BaseTool and return ToolResult. All tools inherit from BaseTool and return ToolResult.
""" """
from src.domains.tools.base import BaseTool, ToolResult from src.domains.tools.base import BaseTool, ToolResult
from src.domains.tools.file import ReadFileTool, GlobFilesTool from src.domains.tools.file import EditFileTool, GlobFilesTool, ReadFileTool, WriteFileTool
from src.domains.tools.search import GrepContentTool from src.domains.tools.search import GrepContentTool, WebSearchTool
from src.domains.tools.shell import BashReadOnlyTool from src.domains.tools.shell import BashReadOnlyTool, BashTool
__all__ = [ __all__ = [
"BaseTool", "BaseTool",
"ToolResult", "BashReadOnlyTool",
"ReadFileTool", "BashTool",
"EditFileTool",
"GlobFilesTool", "GlobFilesTool",
"GrepContentTool", "GrepContentTool",
"BashReadOnlyTool", "ReadFileTool",
"ToolResult",
"WebSearchTool",
"WriteFileTool",
] ]
+16 -4
View File
@@ -32,10 +32,7 @@ class ToolResult:
if not self.success: if not self.success:
return f"ERROR: {self.error}" return f"ERROR: {self.error}"
if isinstance(self.data, str): content = self.data if isinstance(self.data, str) else str(self.data)
content = self.data
else:
content = str(self.data)
if len(content) > max_length: if len(content) > max_length:
self.truncated = True self.truncated = True
@@ -90,6 +87,21 @@ class BaseTool(ABC):
""" """
Execute the tool with given arguments. Execute the tool with given arguments.
Note on the `# type: ignore[override]` each implementation carries.
Every tool narrows this to its own named parameters — read_file takes
file_path/offset/limit, bash takes command/timeout — which mypy reports
as an LSP violation, and strictly it is: a caller holding a BaseTool
could call .execute(anything=1) and no implementation would accept it.
Nothing does. Checked: no reference anywhere in this package is typed as
BaseTool, and every call site constructs the concrete tool and passes its
specific arguments. What this abstract method buys is the runtime
guarantee that a tool without an execute cannot be instantiated, and that
is worth keeping.
The suppressions are per-site rather than a disable_error_code for the
whole package, so a future override that IS unsound still gets caught.
Returns: Returns:
ToolResult with success status and data or error ToolResult with success status and data or error
""" """
@@ -1,7 +1,9 @@
""" """
File operation tools. File operation tools.
""" """
from src.domains.tools.file.read import ReadFileTool from src.domains.tools.file.edit import EditFileTool
from src.domains.tools.file.glob import GlobFilesTool from src.domains.tools.file.glob import GlobFilesTool
from src.domains.tools.file.read import ReadFileTool
from src.domains.tools.file.write import WriteFileTool
__all__ = ["ReadFileTool", "GlobFilesTool"] __all__ = ["EditFileTool", "GlobFilesTool", "ReadFileTool", "WriteFileTool"]
+195
View File
@@ -0,0 +1,195 @@
"""
File editing tool with find-and-replace functionality.
"""
import difflib
from pathlib import Path
import aiofiles
from src.domains.tools.base import BaseTool, ToolResult
from src.shared.logging import get_logger, logged
logger = get_logger(__name__)
# Binary file extensions to skip
BINARY_EXTENSIONS = {
'.pyc', '.pyo', '.so', '.o', '.a', '.lib', '.dll', '.exe',
'.bin', '.dat', '.db', '.sqlite', '.sqlite3',
'.png', '.jpg', '.jpeg', '.gif', '.ico', '.bmp', '.webp',
'.pdf', '.doc', '.docx', '.xls', '.xlsx',
'.zip', '.tar', '.gz', '.bz2', '.7z', '.rar',
'.mp3', '.mp4', '.avi', '.mov', '.wav',
'.woff', '.woff2', '.ttf', '.eot',
}
class EditFileTool(BaseTool):
"""
Edit files using find-and-replace.
Safely modifies files by finding exact text matches and replacing them.
Includes safety checks to prevent accidental edits.
"""
name = "edit_file"
description = """Make targeted edits to a file using find-and-replace.
Args:
file_path: Absolute path to the file to edit
old_string: The exact text to find and replace (must exist in file)
new_string: The replacement text
replace_all: If True, replace all occurrences. If False (default),
old_string must be unique in the file (appear exactly once).
Returns:
Success message with diff preview showing changes, or error.
IMPORTANT:
- The old_string must exactly match text in the file (including whitespace/indentation)
- By default, old_string must appear exactly once in the file (for safety)
- Use replace_all=True only when you intentionally want to replace all occurrences
- Always read the file first to verify exact content before editing
- Cannot edit binary files
Examples:
- Fix a bug: old_string="return x + y", new_string="return x * y"
- Rename function: old_string="def old_name(", new_string="def new_name("
- Add import: old_string="import os", new_string="import os\\nimport sys"
"""
def __init__(
self,
allowed_paths: list[str] | None = None,
max_file_size: int = 1_000_000, # 1MB
):
"""
Initialize EditFileTool.
Args:
allowed_paths: List of allowed directory prefixes (empty = no restrictions)
max_file_size: Maximum file size to edit in bytes
"""
self.allowed_paths = allowed_paths or []
self.max_file_size = max_file_size
def _is_binary_file(self, path: Path) -> bool:
"""Check if file is likely binary based on extension."""
return path.suffix.lower() in BINARY_EXTENSIONS
def _generate_diff(
self,
original: str,
modified: str,
file_path: str
) -> str:
"""Generate a unified diff between original and modified content."""
original_lines = original.splitlines(keepends=True)
modified_lines = modified.splitlines(keepends=True)
diff = difflib.unified_diff(
original_lines,
modified_lines,
fromfile=f"a/{Path(file_path).name}",
tofile=f"b/{Path(file_path).name}",
lineterm=""
)
return "".join(diff)
@logged()
async def execute( # type: ignore[override] # see BaseTool.execute
self,
file_path: str,
old_string: str,
new_string: str,
replace_all: bool = False
) -> ToolResult:
"""
Edit a file by replacing old_string with new_string.
Args:
file_path: Absolute path to the file
old_string: Text to find (must exist)
new_string: Replacement text
replace_all: Replace all occurrences (default: False)
Returns:
ToolResult with diff preview or error
"""
path = Path(file_path)
# Validate path is allowed
if not self._validate_path(path, self.allowed_paths):
return self._error(f"Path not in allowed paths: {file_path}")
# Check file exists
if not path.exists():
return self._error(f"File not found: {file_path}")
if not path.is_file():
return self._error(f"Not a file: {file_path}")
# Check for binary files
if self._is_binary_file(path):
return self._error(f"Cannot edit binary file: {file_path}")
# Check file size
file_size = path.stat().st_size
if file_size > self.max_file_size:
return self._error(
f"File too large ({file_size} bytes). Max: {self.max_file_size} bytes"
)
# Validate inputs
if not old_string:
return self._error("old_string cannot be empty")
if old_string == new_string:
return self._error("old_string and new_string are identical")
try:
# Read file content
async with aiofiles.open(path, encoding='utf-8', errors='replace') as f:
content = await f.read()
# Check if old_string exists
count = content.count(old_string)
if count == 0:
return self._error(
"old_string not found in file. "
"Make sure to match exact whitespace and indentation."
)
# Check uniqueness if replace_all is False
if not replace_all and count > 1:
return self._error(
f"old_string appears {count} times in file. "
f"Use replace_all=True to replace all, or provide a more specific string."
)
# Perform replacement
if replace_all:
modified = content.replace(old_string, new_string)
else:
modified = content.replace(old_string, new_string, 1)
# Generate diff for preview
diff = self._generate_diff(content, modified, file_path)
# Write modified content
async with aiofiles.open(path, 'w', encoding='utf-8') as f:
await f.write(modified)
replacements = count if replace_all else 1
return self._success(
data=f"Successfully edited {file_path}\n\n{diff}",
replacements=replacements,
file_path=str(path.resolve())
)
except PermissionError:
return self._error(f"Permission denied: {file_path}")
except UnicodeDecodeError as e:
return self._error(f"Unable to decode file (binary?): {e}")
except Exception as e:
logger.exception(f"Error editing file: {file_path}")
return self._error(f"Error editing file: {e}")
+2 -2
View File
@@ -6,7 +6,7 @@ from pathlib import Path
from src.domains.tools.base import BaseTool, ToolResult from src.domains.tools.base import BaseTool, ToolResult
from src.domains.tools.gitignore import filter_gitignored from src.domains.tools.gitignore import filter_gitignored
from src.shared.logging import logged, get_logger from src.shared.logging import get_logger, logged
logger = get_logger(__name__) logger = get_logger(__name__)
@@ -63,7 +63,7 @@ IMPORTANT:
self.honor_gitignore = honor_gitignore self.honor_gitignore = honor_gitignore
@logged() @logged()
async def execute( async def execute( # type: ignore[override] # see BaseTool.execute
self, self,
pattern: str, pattern: str,
path: str | None = None, path: str | None = None,
+5 -4
View File
@@ -1,11 +1,12 @@
""" """
File reading tool with line number formatting and sandboxing. File reading tool with line number formatting and sandboxing.
""" """
import aiofiles
from pathlib import Path from pathlib import Path
import aiofiles
from src.domains.tools.base import BaseTool, ToolResult from src.domains.tools.base import BaseTool, ToolResult
from src.shared.logging import logged, get_logger from src.shared.logging import get_logger, logged
logger = get_logger(__name__) logger = get_logger(__name__)
@@ -55,7 +56,7 @@ IMPORTANT:
self.max_line_length = max_line_length self.max_line_length = max_line_length
@logged() @logged()
async def execute( async def execute( # type: ignore[override] # see BaseTool.execute
self, self,
file_path: str, file_path: str,
offset: int = 0, offset: int = 0,
@@ -87,7 +88,7 @@ IMPORTANT:
return self._error(f"Not a file: {file_path}") return self._error(f"Not a file: {file_path}")
try: try:
async with aiofiles.open(path, 'r', encoding='utf-8', errors='replace') as f: async with aiofiles.open(path, encoding='utf-8', errors='replace') as f:
content = await f.read() content = await f.read()
lines = content.splitlines() lines = content.splitlines()
+126
View File
@@ -0,0 +1,126 @@
"""
File writing tool for creating and overwriting files.
"""
from pathlib import Path
import aiofiles
from src.domains.tools.base import BaseTool, ToolResult
from src.shared.logging import get_logger, logged
logger = get_logger(__name__)
class WriteFileTool(BaseTool):
"""
Create new files or overwrite existing files.
Validates paths are within allowed directories and enforces size limits.
"""
name = "write_file"
description = """Create a new file or overwrite an existing file.
Args:
file_path: Absolute path to the file to create/write
content: The content to write to the file
Returns:
Success message with file path and size, or error.
Includes a warning if overwriting an existing file.
IMPORTANT:
- Use absolute paths only
- Parent directory must exist (will not create directories)
- Will overwrite existing files without confirmation
- For targeted edits to existing files, use edit_file instead
- Maximum content size: 1MB
Examples:
- Create new module: file_path="/project/src/utils.py", content="def helper(): pass"
- Create config: file_path="/project/config.json", content='{"key": "value"}'
- Create test: file_path="/project/tests/test_new.py", content="def test_example(): assert True"
"""
def __init__(
self,
allowed_paths: list[str] | None = None,
max_content_size: int = 1_000_000, # 1MB
):
"""
Initialize WriteFileTool.
Args:
allowed_paths: List of allowed directory prefixes (empty = no restrictions)
max_content_size: Maximum content size in bytes
"""
self.allowed_paths = allowed_paths or []
self.max_content_size = max_content_size
@logged()
async def execute( # type: ignore[override] # see BaseTool.execute
self,
file_path: str,
content: str
) -> ToolResult:
"""
Write content to a file.
Args:
file_path: Absolute path to the file
content: Content to write
Returns:
ToolResult with success info or error
"""
path = Path(file_path).resolve()
# Validate path is allowed
if not self._validate_path(path, self.allowed_paths):
return self._error(f"Path not in allowed paths: {file_path}")
# Check content size
content_bytes = len(content.encode('utf-8'))
if content_bytes > self.max_content_size:
return self._error(
f"Content too large ({content_bytes} bytes). "
f"Max: {self.max_content_size} bytes"
)
# Check parent directory exists
if not path.parent.exists():
return self._error(
f"Parent directory does not exist: {path.parent}. "
f"Create it first with mkdir."
)
if not path.parent.is_dir():
return self._error(f"Parent path is not a directory: {path.parent}")
# Check if we're overwriting
overwritten = path.exists() and path.is_file()
try:
# Write the file
async with aiofiles.open(path, 'w', encoding='utf-8') as f:
await f.write(content)
# Count lines for metadata
lines = content.count('\n') + (1 if content and not content.endswith('\n') else 0)
status = "Overwrote" if overwritten else "Created"
return self._success(
data=f"{status} {path} ({content_bytes} bytes, {lines} lines)",
file_path=str(path),
file_size=content_bytes,
lines=lines,
overwritten=overwritten
)
except PermissionError:
return self._error(f"Permission denied: {file_path}")
except OSError as e:
return self._error(f"OS error writing file: {e}")
except Exception as e:
logger.exception(f"Error writing file: {file_path}")
return self._error(f"Error writing file: {e}")
+1 -4
View File
@@ -90,10 +90,7 @@ class GitignoreFilter:
# Make path relative to root for matching # Make path relative to root for matching
try: try:
if path.is_absolute(): rel_path = path.resolve().relative_to(self.root_dir) if path.is_absolute() else path
rel_path = path.resolve().relative_to(self.root_dir)
else:
rel_path = path
except ValueError: except ValueError:
# Path is not under root_dir, don't filter # Path is not under root_dir, don't filter
return False return False
@@ -2,5 +2,6 @@
Search tools. Search tools.
""" """
from src.domains.tools.search.grep import GrepContentTool from src.domains.tools.search.grep import GrepContentTool
from src.domains.tools.search.web import WebSearchTool
__all__ = ["GrepContentTool"] __all__ = ["GrepContentTool", "WebSearchTool"]
+2 -2
View File
@@ -7,7 +7,7 @@ from typing import Literal
from src.domains.tools.base import BaseTool, ToolResult from src.domains.tools.base import BaseTool, ToolResult
from src.domains.tools.gitignore import filter_gitignored from src.domains.tools.gitignore import filter_gitignored
from src.shared.logging import logged, get_logger from src.shared.logging import get_logger, logged
logger = get_logger(__name__) logger = get_logger(__name__)
@@ -70,7 +70,7 @@ IMPORTANT:
self.honor_gitignore = honor_gitignore self.honor_gitignore = honor_gitignore
@logged() @logged()
async def execute( async def execute( # type: ignore[override] # see BaseTool.execute
self, self,
pattern: str, pattern: str,
path: str | None = None, path: str | None = None,
+191
View File
@@ -0,0 +1,191 @@
"""
Web search tool using SearXNG.
"""
from dataclasses import dataclass
from datetime import datetime
from typing import Any
import httpx
from src.domains.tools.base import BaseTool, ToolResult
from src.shared.config import get_settings
from src.shared.logging import get_logger, logged
from src.shared.retry import retry_async
logger = get_logger(__name__)
@dataclass
class SearchResult:
"""A single search result."""
title: str
url: str
content: str
engine: str
published_date: str | None = None
class WebSearchTool(BaseTool):
"""
Search the web using SearXNG metasearch engine.
Returns relevant web results for queries about current events,
documentation, or anything beyond the LLM's knowledge cutoff.
"""
name = "web_search"
description = """Search the web for current information.
Args:
query: Search query string
num_results: Maximum results to return (default: 5, max: 10)
engines: Comma-separated engine list (optional, e.g., "google,brave,duckduckgo")
categories: Search category (optional: "general", "images", "news", "science", "it")
Returns:
List of search results with title, URL, and snippet.
Include a "Sources:" section with URLs in your response.
Examples:
- query="Python 3.12 new features" - Find latest Python docs
- query="FastAPI best practices 2024" - Find recent tutorials
- query="CVE-2024" categories="it" - Search IT/security news
IMPORTANT:
- Use this for current events, recent documentation, or facts you're unsure about
- Always cite sources with URLs in your response
- Today's date is {date} - use current year in queries for recent info
""".format(date=datetime.now().strftime("%Y-%m-%d"))
def __init__(
self,
searxng_url: str | None = None,
timeout: int | None = None,
max_results: int = 10,
):
"""
Initialize WebSearchTool.
Args:
searxng_url: SearXNG instance URL (default: from config)
timeout: Request timeout in seconds (default: from config)
max_results: Maximum results to return
"""
settings = get_settings()
self.searxng_url = (searxng_url or settings.searxng_url).rstrip("/")
self.timeout = timeout or settings.searxng_timeout
self.max_results = max_results
# Retry settings
self.retry_max_attempts = settings.retry_max_attempts
self.retry_base_delay = settings.retry_base_delay
self.retry_max_delay = settings.retry_max_delay
async def _fetch_search_results(self, params: dict) -> dict:
"""
Fetch search results from SearXNG.
This method is wrapped with retry logic for transient failures.
"""
async with httpx.AsyncClient(timeout=self.timeout) as client:
response = await client.get(
f"{self.searxng_url}/search",
params=params,
)
response.raise_for_status()
# httpx types .json() as Any. Naming the shape here keeps the Any from
# travelling into every caller of this method.
payload: dict[Any, Any] = response.json()
return payload
@logged()
async def execute( # type: ignore[override] # see BaseTool.execute
self,
query: str,
num_results: int = 5,
engines: str | None = None,
categories: str | None = None,
) -> ToolResult:
"""
Execute web search.
Args:
query: Search query
num_results: Number of results (1-10)
engines: Specific engines to use
categories: Search category
Returns:
ToolResult with search results
"""
if not query or not query.strip():
return self._error("Query cannot be empty")
num_results = min(max(1, num_results), self.max_results)
# Build SearXNG API request
params = {
"q": query.strip(),
"format": "json",
}
if engines:
params["engines"] = engines
if categories:
params["categories"] = categories
try:
data = await retry_async(
self._fetch_search_results,
params,
max_attempts=self.retry_max_attempts,
base_delay=self.retry_base_delay,
max_delay=self.retry_max_delay,
)
except httpx.TimeoutException:
return self._error(f"Search timed out after {self.timeout}s (all retries exhausted)")
except httpx.HTTPStatusError as e:
return self._error(f"Search failed: HTTP {e.response.status_code}")
except httpx.RequestError as e:
return self._error(f"Search request failed: {e}")
except Exception as e:
logger.exception(f"Unexpected search error: {e}")
return self._error(f"Search error: {e}")
# Parse results
raw_results = data.get("results", [])[:num_results]
if not raw_results:
return self._success(
f"No results found for: {query}",
result_count=0,
query=query,
)
# Format results for LLM consumption
results = []
for r in raw_results:
result = SearchResult(
title=r.get("title", "Untitled"),
url=r.get("url", ""),
content=r.get("content", "No description"),
engine=r.get("engine", "unknown"),
published_date=r.get("publishedDate"),
)
results.append(result)
# Format as readable text
output_lines = [f"Search results for: {query}", ""]
for i, r in enumerate(results, 1):
output_lines.append(f"{i}. **{r.title}**")
output_lines.append(f" URL: {r.url}")
output_lines.append(f" {r.content}")
if r.published_date:
output_lines.append(f" Published: {r.published_date}")
output_lines.append("")
return self._success(
"\n".join(output_lines),
result_count=len(results),
query=query,
engines_used=list({r.engine for r in results}),
)
@@ -2,5 +2,6 @@
Shell execution tools. Shell execution tools.
""" """
from src.domains.tools.shell.bash import BashReadOnlyTool from src.domains.tools.shell.bash import BashReadOnlyTool
from src.domains.tools.shell.bash_full import BashTool
__all__ = ["BashReadOnlyTool"] __all__ = ["BashReadOnlyTool", "BashTool"]
+3 -3
View File
@@ -8,7 +8,7 @@ import shlex
from pathlib import Path from pathlib import Path
from src.domains.tools.base import BaseTool, ToolResult from src.domains.tools.base import BaseTool, ToolResult
from src.shared.logging import logged, get_logger from src.shared.logging import get_logger, logged
logger = get_logger(__name__) logger = get_logger(__name__)
@@ -115,7 +115,7 @@ Examples:
self.max_output_size = max_output_size self.max_output_size = max_output_size
@logged() @logged()
async def execute( async def execute( # type: ignore[override] # see BaseTool.execute
self, self,
command: str, command: str,
cwd: str | None = None, cwd: str | None = None,
@@ -190,7 +190,7 @@ Examples:
exit_code=proc.returncode exit_code=proc.returncode
) )
except asyncio.TimeoutError: except TimeoutError:
return self._error(f"Command timed out after {timeout} seconds") return self._error(f"Command timed out after {timeout} seconds")
except Exception as e: except Exception as e:
logger.exception(f"Error executing command: {command}") logger.exception(f"Error executing command: {command}")
@@ -0,0 +1,430 @@
"""
Full bash command execution tool with controlled write capabilities.
Allows more operations than BashReadOnlyTool but still with safety controls.
"""
import asyncio
import shlex
from pathlib import Path
from src.domains.tools.base import BaseTool, ToolResult
from src.shared.logging import get_logger, logged
logger = get_logger(__name__)
# Commands allowed (includes write operations)
ALLOWED_COMMANDS = {
# File inspection (read-only)
"ls", "find", "cat", "head", "tail", "wc", "file", "stat",
"tree", "du", "df",
# Text processing
"grep", "awk", "sed", "sort", "uniq", "cut", "tr",
# Git (full operations)
"git",
# System info
"pwd", "whoami", "hostname", "uname", "date", "env", "printenv",
"which", "type", "echo",
# Archive operations
"tar", "unzip", "zipinfo",
# Write operations (controlled)
"mkdir", "touch", "cp", "mv", "rm",
"chmod",
# Python ecosystem
"python", "python3", "pip", "pip3", "pytest", "mypy", "ruff",
# Other dev tools
"make", "cargo", "npm", "node", "tsc",
}
# Git subcommands (includes write operations)
ALLOWED_GIT_SUBCOMMANDS = {
# Read-only
"status", "log", "diff", "show", "branch", "tag",
"remote", "config", "ls-files", "ls-tree",
"rev-parse", "describe", "shortlog", "blame",
# Write operations
"add", "commit", "checkout", "switch", "restore",
"merge", "rebase", "cherry-pick",
"stash", "pull", "fetch", "init",
"reset", "clean", "rm", "mv",
}
# Absolutely forbidden - too dangerous regardless of context
ABSOLUTELY_FORBIDDEN = [
# Catastrophic deletes
"rm -rf /",
"rm -rf ~",
"rm -rf .",
"rm -rf *",
# Privilege escalation
"sudo ",
"su ",
"doas ",
# Dangerous permissions
"chmod 777",
"chmod -R 777",
"chown ",
"chgrp ",
# Disk operations
"dd if=",
"mkfs",
"fdisk",
"parted",
# System control
"shutdown",
"reboot",
"poweroff",
"init ",
"systemctl",
# Fork bomb pattern
":(){ :|:& };:",
]
# Network commands are forbidden
NETWORK_FORBIDDEN = [
"curl", "wget", "ssh", "scp", "rsync", "sftp", "ftp",
"nc", "netcat", "telnet", "nmap", "ping",
]
# Dangerous rm flags
DANGEROUS_RM_FLAGS = {"-rf", "-fr", "-r -f", "-f -r", "--recursive --force"}
class BashTool(BaseTool):
"""
Execute bash commands with controlled write capabilities.
More permissive than BashReadOnlyTool but still with safety controls.
"""
name = "bash"
description = """Execute a bash command with write capabilities.
ALLOWED commands:
- File operations: ls, find, cat, head, tail, mkdir, touch, cp, mv
- Git (full): git add, git commit, git checkout, git merge, git pull, etc.
- Python: python, pip install, pytest, mypy, ruff
- Text processing: grep, awk, sed, sort, uniq
- System info: pwd, whoami, date, which
RESTRICTED:
- rm: Single files only, no -rf flag, must be in allowed paths
- mv/cp: Target must be in allowed paths
- chmod: Only safe modes (no 777)
FORBIDDEN (always blocked):
- sudo, su (privilege escalation)
- Network: curl, wget, ssh, scp, rsync
- Dangerous: chmod 777, rm -rf, dd, mkfs, shutdown
Args:
command: The bash command to execute
cwd: Working directory (default: current directory)
timeout: Timeout in seconds (default: 60)
Returns:
Command stdout on success, or error message.
Examples:
- "mkdir -p src/new_module" - Create directory
- "cp template.py src/new_file.py" - Copy file
- "git add . && git commit -m 'feat: add feature'" - Git commit
- "pip install requests" - Install package
- "pytest tests/ -v" - Run tests
- "rm src/old_file.py" - Remove single file
"""
def __init__(
self,
allowed_paths: list[str] | None = None,
default_timeout: int = 60,
max_output_size: int = 50000
):
"""
Initialize BashTool.
Args:
allowed_paths: Allowed working/target directories
default_timeout: Default command timeout in seconds
max_output_size: Maximum output size in characters
"""
self.allowed_paths = allowed_paths or []
self.default_timeout = default_timeout
self.max_output_size = max_output_size
@logged()
async def execute( # type: ignore[override] # see BaseTool.execute
self,
command: str,
cwd: str | None = None,
timeout: int | None = None
) -> ToolResult:
"""
Execute a bash command.
Args:
command: Command to execute
cwd: Working directory
timeout: Timeout in seconds
Returns:
ToolResult with command output or error
"""
timeout = timeout or self.default_timeout
working_dir = Path(cwd) if cwd else Path.cwd()
# Validate working directory
if not self._validate_path(working_dir, self.allowed_paths):
return self._error(f"Working directory not allowed: {working_dir}")
if not working_dir.exists():
return self._error(f"Working directory not found: {working_dir}")
# Security validation
validation_error = self._validate_command(command, working_dir)
if validation_error:
return self._error(validation_error)
try:
proc = await asyncio.create_subprocess_shell(
command,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
cwd=str(working_dir)
)
stdout, stderr = await asyncio.wait_for(
proc.communicate(),
timeout=timeout
)
stdout_str = stdout.decode('utf-8', errors='replace')
stderr_str = stderr.decode('utf-8', errors='replace')
# Truncate if necessary
truncated = False
if len(stdout_str) > self.max_output_size:
stdout_str = stdout_str[:self.max_output_size]
truncated = True
if proc.returncode != 0:
# Command failed, return stderr
error_msg = stderr_str or f"Command exited with code {proc.returncode}"
return ToolResult(
success=False,
data=stdout_str if stdout_str else None,
error=error_msg,
truncated=truncated
)
# Success - combine stdout and stderr if both present
output = stdout_str
if stderr_str and not output:
output = stderr_str
return self._success(
data=output,
truncated=truncated,
exit_code=proc.returncode
)
except TimeoutError:
return self._error(f"Command timed out after {timeout} seconds")
except Exception as e:
logger.exception(f"Error executing command: {command}")
return self._error(f"Error executing command: {e}")
def _validate_command(self, command: str, working_dir: Path) -> str | None:
"""
Validate command is safe to execute.
Returns:
Error message if invalid, None if valid
"""
command_lower = command.lower()
# Check absolutely forbidden patterns first
for pattern in ABSOLUTELY_FORBIDDEN:
if pattern.lower() in command_lower:
return f"Command contains forbidden pattern: {pattern}"
# Check network commands
for net_cmd in NETWORK_FORBIDDEN:
# Check as standalone command or with path
if (f" {net_cmd} " in f" {command_lower} " or
command_lower.startswith(f"{net_cmd} ") or
command_lower == net_cmd or
f"/{net_cmd} " in command_lower or
f"/{net_cmd}" == command_lower[-len(net_cmd)-1:]):
return f"Network command not allowed: {net_cmd}"
# Split by command separators to validate each sub-command
# Handle &&, ||, ; but also pipe |
sub_commands = self._split_commands(command)
for sub_cmd in sub_commands:
sub_cmd = sub_cmd.strip()
if not sub_cmd:
continue
error = self._validate_single_command(sub_cmd, working_dir)
if error:
return error
return None
def _split_commands(self, command: str) -> list[str]:
"""Split command by separators (&&, ||, ;) but not pipes."""
# Simple split - could be improved with proper shell parsing
result = []
current = ""
i = 0
while i < len(command):
if command[i:i+2] in ("&&", "||"):
result.append(current)
current = ""
i += 2
elif command[i] == ";":
result.append(current)
current = ""
i += 1
else:
current += command[i]
i += 1
if current:
result.append(current)
return result
def _validate_single_command(self, command: str, working_dir: Path) -> str | None:
"""Validate a single command (no &&, ||, ;)."""
# Handle pipes - validate first command in pipe chain
if "|" in command:
command = command.split("|")[0].strip()
# Parse command
try:
tokens = shlex.split(command)
if not tokens:
return None # Empty is ok (could be whitespace)
except ValueError as e:
return f"Invalid command syntax: {e}"
# Get base command
base_cmd = Path(tokens[0]).name
# Check if command is allowed
if base_cmd not in ALLOWED_COMMANDS:
return f"Command not allowed: {base_cmd}"
# Special handling for specific commands
if base_cmd == "git":
return self._validate_git_command(tokens)
elif base_cmd == "rm":
return self._validate_rm_command(tokens, working_dir)
elif base_cmd in ("cp", "mv"):
return self._validate_copy_move_command(tokens, working_dir)
elif base_cmd == "chmod":
return self._validate_chmod_command(tokens)
elif base_cmd in ("pip", "pip3"):
return self._validate_pip_command(tokens)
return None
def _validate_git_command(self, tokens: list[str]) -> str | None:
"""Validate git command."""
if len(tokens) < 2:
return "Git command requires a subcommand"
git_subcommand = tokens[1]
# Handle git with flags before subcommand (e.g., git -C path status)
if git_subcommand.startswith("-"):
# Find the actual subcommand
for _i, token in enumerate(tokens[2:], 2):
if not token.startswith("-"):
git_subcommand = token
break
else:
return "Git command requires a subcommand"
if git_subcommand not in ALLOWED_GIT_SUBCOMMANDS:
return f"Git subcommand not allowed: {git_subcommand}"
# Block git push (could push to remote)
if git_subcommand == "push":
return "git push is not allowed (use manually)"
return None
def _validate_rm_command(self, tokens: list[str], working_dir: Path) -> str | None:
"""Validate rm command - only single files, no -rf."""
# Check for dangerous flags
flags = [t for t in tokens[1:] if t.startswith("-")]
for flag in flags:
if "r" in flag and "f" in flag:
return "rm -rf is not allowed"
if flag in DANGEROUS_RM_FLAGS:
return f"rm flag not allowed: {flag}"
# Must have at least one non-flag argument
args = [t for t in tokens[1:] if not t.startswith("-")]
if not args:
return "rm requires a file argument"
# Validate each path
for arg in args:
path = Path(arg)
if not path.is_absolute():
path = working_dir / path
if not self._validate_path(path, self.allowed_paths):
return f"rm target not in allowed paths: {arg}"
return None
def _validate_copy_move_command(
self, tokens: list[str], working_dir: Path
) -> str | None:
"""Validate cp/mv command - target must be in allowed paths."""
# Get non-flag arguments
args = [t for t in tokens[1:] if not t.startswith("-")]
if len(args) < 2:
return None # Let the command fail naturally
# Last argument is typically the destination
dest = args[-1]
dest_path = Path(dest)
if not dest_path.is_absolute():
dest_path = working_dir / dest_path
if not self._validate_path(dest_path, self.allowed_paths):
return f"Copy/move destination not in allowed paths: {dest}"
return None
def _validate_chmod_command(self, tokens: list[str]) -> str | None:
"""Validate chmod command - block dangerous modes."""
for token in tokens[1:]:
if token.startswith("-"):
continue
# Block 777 and similar
if "777" in token or "666" in token:
return f"chmod mode not allowed: {token}"
return None
def _validate_pip_command(self, tokens: list[str]) -> str | None:
"""Validate pip command - allow install, block uninstall of system packages."""
if len(tokens) < 2:
return None
subcommand = tokens[1]
# Allow help, list, show, freeze, check
allowed_pip = {"install", "list", "show", "freeze", "check", "help", "--help", "-h"}
if subcommand not in allowed_pip:
return f"pip subcommand not allowed: {subcommand}"
return None
+7 -2
View File
@@ -31,13 +31,18 @@ async def lifespan(app: FastAPI):
logger.info(f"Port: {settings.port}") logger.info(f"Port: {settings.port}")
logger.info(f"Ollama: {settings.ollama_url}") logger.info(f"Ollama: {settings.ollama_url}")
logger.info(f"Agent model: {settings.ollama_agent_model}") logger.info(f"Agent model: {settings.ollama_agent_model}")
logger.info(f"Database: {settings.database_url}")
logger.info("=" * 60) logger.info("=" * 60)
# TODO: Initialize resources (LLM clients, etc.)
yield yield
# Cleanup # Cleanup
from src.db import get_database
try:
database = get_database()
await database.close()
except Exception:
pass
logger.info("Shutting down") logger.info("Shutting down")
+44 -5
View File
@@ -54,17 +54,54 @@ class _SanitizedAsyncOpenAI(AsyncOpenAI):
super().__init__(api_key="ollama", **kwargs) super().__init__(api_key="ollama", **kwargs)
@property @property
def chat(self) -> "_SanitizedChat": def chat(self) -> "_SanitizedChat": # type: ignore[override]
"""Return sanitized chat interface.""" """Return sanitized chat interface.
Deliberately incompatible with AsyncOpenAI.chat, which is a Chat
resource. Replacing it is the entire mechanism of this class: Ollama
rejects assistant messages carrying content: null alongside tool_calls,
so every completion has to pass through the sanitiser. Typing it as the
parent's Chat would describe an object this class does not return.
The suppression is on this member alone; the rest of the client keeps
its inherited types.
"""
return _SanitizedChat(self) return _SanitizedChat(self)
def _parent_chat(client: AsyncOpenAI) -> Any:
"""Get AsyncOpenAI's own `chat`, bypassing the subclass override.
This read the descriptor's `.fget` until 2026-08-11, which is the property
API. openai made `chat` a functools.cached_property, whose getter is `.func`,
so the call raised AttributeError the moment anything touched `.chat` — that
is, on the first completion any agent tried to make. Verified broken in the
running container on openai 2.46.0 as well as locally on 2.15.0.
It went unnoticed because the endpoints that reach it had served no requests
in 30 days, and because the line carried a bare `# type: ignore` that
suppressed exactly the complaint that would have flagged it.
Reading whichever getter the descriptor actually exposes keeps this working
across that change and the reverse of it, and raises something legible if
openai adopts a third shape.
"""
descriptor = AsyncOpenAI.__dict__["chat"]
getter = getattr(descriptor, "func", None) or getattr(descriptor, "fget", None)
if getter is None: # pragma: no cover - defensive
raise TypeError(
f"AsyncOpenAI.chat is a {type(descriptor).__name__} with neither "
"'func' nor 'fget'; the sanitising wrapper needs updating"
)
return getter(client)
class _SanitizedChat: class _SanitizedChat:
"""Chat interface wrapper with sanitized completions.""" """Chat interface wrapper with sanitized completions."""
def __init__(self, client: _SanitizedAsyncOpenAI): def __init__(self, client: _SanitizedAsyncOpenAI):
self._client = client self._client = client
self._original_chat = AsyncOpenAI.chat.fget(client) # type: ignore self._original_chat = _parent_chat(client)
@property @property
def completions(self) -> "_SanitizedCompletions": def completions(self) -> "_SanitizedCompletions":
@@ -110,8 +147,10 @@ def _sanitize_messages(messages: list[dict[str, Any]]) -> list[dict[str, Any]]:
msg_copy = dict(msg) msg_copy = dict(msg)
# Fix null content in assistant messages with tool calls # Fix null content in assistant messages with tool calls
if msg_copy.get("role") == "assistant": if (
if msg_copy.get("content") is None and msg_copy.get("tool_calls"): msg_copy.get("role") == "assistant"
and msg_copy.get("content") is None and msg_copy.get("tool_calls")
):
msg_copy["content"] = "" msg_copy["content"] = ""
logger.debug( logger.debug(
f"Sanitized null content, tool_calls={len(msg_copy['tool_calls'])}" f"Sanitized null content, tool_calls={len(msg_copy['tool_calls'])}"
+18 -3
View File
@@ -64,21 +64,36 @@ class Settings(BaseSettings):
# LLM - Ollama (always hot in VRAM on tower-of-joy) # LLM - Ollama (always hot in VRAM on tower-of-joy)
ollama_url: str = "http://192.168.86.149:11434" ollama_url: str = "http://192.168.86.149:11434"
ollama_agent_model: str = "mistral-nemo-large:latest" ollama_agent_model: str = "gemma4:e2b"
ollama_embed_model: str = "nomic-embed-text:latest" ollama_embed_model: str = "nomic-embed-text:latest"
# Auth - Tatlock integration # Auth - Tatlock integration
tatlock_api_url: str | None = "http://192.168.86.149:8000" tatlock_api_url: str | None = "http://tatlock:8000"
internal_api_key: str | None = None internal_api_key: str | None = None
# Web search - SearXNG (use SEARXNG_URL env var to override)
searxng_url: str = "http://searxng:8080"
searxng_timeout: int = 10
# Tool execution # Tool execution
tool_timeout_seconds: int = 120 tool_timeout_seconds: int = 120
sandbox_enabled: bool = True sandbox_enabled: bool = True
allowed_paths: list[str] | None = None allowed_paths: list[str] | None = None
# Sessions # Database
database_url: str = "sqlite+aiosqlite:///./webber.db"
# Sessions & Context
session_ttl_hours: int = 24 session_ttl_hours: int = 24
max_context_tokens: int = 128000 max_context_tokens: int = 128000
summarization_threshold: float = 0.8 # Summarize at 80% of max tokens
summarization_target_tokens: int = 500 # Target summary size
keep_recent_messages: int = 6 # Messages to keep unsummarized (3 turns)
# Retry logic
retry_max_attempts: int = 3 # Max retry attempts for transient failures
retry_base_delay: float = 1.0 # Base delay in seconds
retry_max_delay: float = 30.0 # Maximum delay in seconds
model_config = SettingsConfigDict( model_config = SettingsConfigDict(
env_file=".env", env_file=".env",
+20 -6
View File
@@ -16,6 +16,7 @@ from collections.abc import Callable
from contextvars import ContextVar, Token from contextvars import ContextVar, Token
from dataclasses import dataclass, field from dataclasses import dataclass, field
from pathlib import Path from pathlib import Path
from typing import Any, ParamSpec, TypeVar, cast
from uuid import uuid4 from uuid import uuid4
# === Trace Context === # === Trace Context ===
@@ -80,12 +81,21 @@ def get_logger(name: str) -> logging.Logger:
# === Decorator === # === Decorator ===
# @logged wraps ~24 functions across this package. Untyped, its decorator
# erased every one of their signatures, so mypy saw `Any` coming back from
# annotated functions like `ToolResult.execute() -> ToolResult`. That surfaced
# as 33 no-any-return errors scattered across the tools and agents — each
# reading like a local annotation slip, all of them this one decorator.
P = ParamSpec("P")
R = TypeVar("R")
def logged( def logged(
logger: logging.Logger | None = None, logger: logging.Logger | None = None,
slow_threshold_ms: float = 100.0, slow_threshold_ms: float = 100.0,
warn_threshold_ms: float = 500.0, warn_threshold_ms: float = 500.0,
include_args: bool = False, include_args: bool = False,
): ) -> Callable[[Callable[P, R]], Callable[P, R]]:
""" """
Decorator for automatic function logging with temporal benchmarking. Decorator for automatic function logging with temporal benchmarking.
@@ -102,7 +112,7 @@ def logged(
@logged(slow_threshold_ms=50, warn_threshold_ms=200) @logged(slow_threshold_ms=50, warn_threshold_ms=200)
def critical_path(): ... def critical_path(): ...
""" """
def decorator(func: Callable): def decorator(func: Callable[P, R]) -> Callable[P, R]:
nonlocal logger nonlocal logger
if logger is None: if logger is None:
logger = logging.getLogger(func.__module__) logger = logging.getLogger(func.__module__)
@@ -140,7 +150,7 @@ def logged(
logger.debug(f"[{tid}] {func_name} completed in {duration:.2f}ms") logger.debug(f"[{tid}] {func_name} completed in {duration:.2f}ms")
@functools.wraps(func) @functools.wraps(func)
async def async_wrapper(*args, **kwargs): async def async_wrapper(*args: P.args, **kwargs: P.kwargs) -> Any:
span = _create_span() span = _create_span()
token = _current_span.set(span) token = _current_span.set(span)
@@ -150,7 +160,7 @@ def logged(
logger.debug(f"[{span.trace_id[:8]}] -> {func_name}") logger.debug(f"[{span.trace_id[:8]}] -> {func_name}")
try: try:
result = await func(*args, **kwargs) result = await cast(Any, func(*args, **kwargs))
_log_completion(span) _log_completion(span)
return result return result
except Exception as e: except Exception as e:
@@ -160,7 +170,7 @@ def logged(
_current_span.reset(token) _current_span.reset(token)
@functools.wraps(func) @functools.wraps(func)
def sync_wrapper(*args, **kwargs): def sync_wrapper(*args: P.args, **kwargs: P.kwargs) -> R:
span = _create_span() span = _create_span()
token = _current_span.set(span) token = _current_span.set(span)
@@ -179,7 +189,11 @@ def logged(
finally: finally:
_current_span.reset(token) _current_span.reset(token)
return async_wrapper if asyncio.iscoroutinefunction(func) else sync_wrapper # The branch is chosen at decoration time; mypy cannot narrow R to a
# coroutine on the strength of iscoroutinefunction, so the union is
# asserted here once instead of at every call site.
chosen = async_wrapper if asyncio.iscoroutinefunction(func) else sync_wrapper
return cast(Callable[P, R], chosen)
return decorator return decorator
+220
View File
@@ -0,0 +1,220 @@
"""
Retry utilities for handling transient failures.
Provides decorators and helpers for automatic retry with exponential backoff.
"""
import asyncio
import random
from collections.abc import Awaitable, Callable
from functools import wraps
from typing import Any, TypeVar
import httpx
from src.shared.logging import get_logger
logger = get_logger(__name__)
T = TypeVar("T")
# Exceptions that should trigger a retry
RETRYABLE_EXCEPTIONS = (
httpx.TimeoutException,
httpx.ConnectError,
httpx.ReadError,
httpx.WriteError,
httpx.ConnectTimeout,
httpx.ReadTimeout,
httpx.WriteTimeout,
httpx.PoolTimeout,
ConnectionError,
TimeoutError,
OSError, # Covers many network-related errors
)
def is_retryable_http_status(status_code: int) -> bool:
"""
Check if an HTTP status code should trigger a retry.
Retryable:
- 429 Too Many Requests (rate limited)
- 500 Internal Server Error
- 502 Bad Gateway
- 503 Service Unavailable
- 504 Gateway Timeout
"""
return status_code in (429, 500, 502, 503, 504)
def is_retryable_exception(exc: Exception) -> bool:
"""Check if an exception should trigger a retry."""
if isinstance(exc, RETRYABLE_EXCEPTIONS):
return True
# Check for retryable HTTP status codes
if isinstance(exc, httpx.HTTPStatusError):
return is_retryable_http_status(exc.response.status_code)
return False
def calculate_backoff(
attempt: int,
base_delay: float = 1.0,
max_delay: float = 60.0,
jitter: bool = True,
) -> float:
"""
Calculate exponential backoff delay with optional jitter.
Args:
attempt: Current attempt number (0-indexed)
base_delay: Base delay in seconds
max_delay: Maximum delay in seconds
jitter: Add random jitter to prevent thundering herd
Returns:
Delay in seconds
"""
# Exponential backoff: base_delay * 2^attempt
delay: float = min(base_delay * (2 ** attempt), max_delay)
if jitter:
# Add up to 25% random jitter
delay = delay * (0.75 + random.random() * 0.5)
return delay
def with_retry(
max_attempts: int = 3,
base_delay: float = 1.0,
max_delay: float = 60.0,
retryable_exceptions: tuple[type[Exception], ...] | None = None,
) -> Callable[[Callable[..., Awaitable[T]]], Callable[..., Awaitable[T]]]:
"""
Decorator for async functions that should retry on transient failures.
Args:
max_attempts: Maximum number of attempts (including initial)
base_delay: Base delay between retries in seconds
max_delay: Maximum delay between retries in seconds
retryable_exceptions: Additional exceptions to retry on
Returns:
Decorated function with retry logic
Example:
@with_retry(max_attempts=3, base_delay=1.0)
async def fetch_data():
async with httpx.AsyncClient() as client:
response = await client.get(url)
return response.json()
"""
extra_exceptions = retryable_exceptions or ()
# Bound to a named, typed tuple: mypy cannot verify that a star-unpacked
# tuple in an `except` clause holds exception classes, and reports it as
# "exception type must be derived from BaseException" — which reads like a
# real defect rather than an inference limit.
retry_on: tuple[type[Exception], ...] = (*RETRYABLE_EXCEPTIONS, *extra_exceptions)
def decorator(func: Callable[..., Awaitable[T]]) -> Callable[..., Awaitable[T]]:
@wraps(func)
async def wrapper(*args: Any, **kwargs: Any) -> T:
last_exception: Exception | None = None
for attempt in range(max_attempts):
try:
return await func(*args, **kwargs)
except retry_on as e:
last_exception = e
should_retry = True
except httpx.HTTPStatusError as e:
last_exception = e
should_retry = is_retryable_http_status(e.response.status_code)
except Exception:
# Non-retryable exception, re-raise immediately
raise
if should_retry and attempt < max_attempts - 1:
delay = calculate_backoff(attempt, base_delay, max_delay)
logger.warning(
f"Retry {attempt + 1}/{max_attempts - 1} for {func.__name__} "
f"after {delay:.2f}s due to: {last_exception}"
)
await asyncio.sleep(delay)
elif not should_retry:
# Non-retryable HTTP error
raise last_exception
# All retries exhausted
logger.error(
f"All {max_attempts} attempts failed for {func.__name__}: {last_exception}"
)
raise last_exception # type: ignore
return wrapper
return decorator
async def retry_async(
func: Callable[..., Awaitable[T]],
*args: Any,
max_attempts: int = 3,
base_delay: float = 1.0,
max_delay: float = 60.0,
**kwargs: Any,
) -> T:
"""
Retry an async function with exponential backoff.
Alternative to decorator when you need per-call control.
Args:
func: Async function to call
*args: Positional arguments for func
max_attempts: Maximum number of attempts
base_delay: Base delay between retries
max_delay: Maximum delay between retries
**kwargs: Keyword arguments for func
Returns:
Result of func
Raises:
Last exception if all retries fail
Example:
result = await retry_async(
fetch_data,
url,
max_attempts=5,
timeout=30,
)
"""
last_exception: Exception | None = None
for attempt in range(max_attempts):
try:
return await func(*args, **kwargs)
except Exception as e:
last_exception = e
if not is_retryable_exception(e):
raise
if attempt < max_attempts - 1:
delay = calculate_backoff(attempt, base_delay, max_delay)
logger.warning(
f"Retry {attempt + 1}/{max_attempts - 1} "
f"after {delay:.2f}s due to: {e}"
)
await asyncio.sleep(delay)
raise last_exception # type: ignore
+85
View File
@@ -0,0 +1,85 @@
"""
Token counting utilities for context management.
Uses tiktoken for token counting. While tiktoken is OpenAI's tokenizer,
cl100k_base encoding provides reasonable estimates for most LLMs.
"""
from functools import lru_cache
from src.shared.logging import get_logger
logger = get_logger(__name__)
@lru_cache(maxsize=1)
def _get_encoding():
"""Get tiktoken encoding (cached)."""
import tiktoken
# cl100k_base is used by GPT-4 and provides reasonable estimates for most models
return tiktoken.get_encoding("cl100k_base")
def count_tokens(text: str) -> int:
"""
Count tokens in a text string.
Args:
text: Text to count tokens for
Returns:
Token count
"""
try:
encoding = _get_encoding()
return len(encoding.encode(text))
except Exception as e:
# Fallback to rough estimate if tiktoken fails
logger.warning(f"Token counting failed, using estimate: {e}")
return len(text) // 4
def count_message_tokens(messages: list[dict[str, str]]) -> int:
"""
Count tokens for a list of chat messages.
Args:
messages: List of message dicts with 'role' and 'content' keys
Returns:
Total token count including message overhead
"""
try:
encoding = _get_encoding()
total = 0
for msg in messages:
# Each message has ~4 tokens overhead for role/formatting
total += 4
total += len(encoding.encode(msg.get("content", "")))
total += len(encoding.encode(msg.get("role", "")))
# Add 2 tokens for assistant response priming
total += 2
return total
except Exception as e:
# Fallback to rough estimate
logger.warning(f"Token counting failed, using estimate: {e}")
total = 0
for msg in messages:
total += len(msg.get("content", "")) // 4
total += 4 # Overhead per message
return total
def estimate_tokens(text: str) -> int:
"""
Quick token estimate without external library.
Uses ~4 characters per token heuristic.
Less accurate but faster for rough estimates.
Args:
text: Text to estimate
Returns:
Estimated token count
"""
return len(text) // 4
+309 -1
View File
@@ -1,11 +1,73 @@
""" """
Pytest configuration and fixtures. Pytest configuration and fixtures.
Test categories:
- Unit tests: Run by default, no external dependencies
- Integration tests: Require Ollama, run with --run-integration
- E2E tests: Require running API server, run with --run-e2e
Usage:
pytest tests/ # Run unit tests only
pytest tests/ --run-integration # Include integration tests
pytest tests/ --run-e2e # Include E2E tests
pytest tests/ --run-integration --run-e2e # Run all tests
""" """
import os
import tempfile
from pathlib import Path
import pytest import pytest
from httpx import ASGITransport, AsyncClient from httpx import ASGITransport, AsyncClient
from src.main import app from src.main import app
# =============================================================================
# Command Line Options
# =============================================================================
def pytest_addoption(parser):
"""Add custom command line options."""
parser.addoption(
"--run-integration",
action="store_true",
default=False,
help="Run integration tests (require Ollama to be running)",
)
parser.addoption(
"--run-e2e",
action="store_true",
default=False,
help="Run E2E tests (require API server to be running)",
)
parser.addoption(
"--ollama-url",
action="store",
default="http://192.168.86.149:11434",
help="Ollama API URL for integration tests",
)
parser.addoption(
"--api-url",
action="store",
default="http://localhost:8095",
help="Webber API URL for E2E tests",
)
def pytest_collection_modifyitems(config, items):
"""Skip integration/e2e tests unless explicitly requested."""
skip_integration = pytest.mark.skip(reason="need --run-integration option to run")
skip_e2e = pytest.mark.skip(reason="need --run-e2e option to run")
for item in items:
if "integration" in item.keywords and not config.getoption("--run-integration"):
item.add_marker(skip_integration)
if "e2e" in item.keywords and not config.getoption("--run-e2e"):
item.add_marker(skip_e2e)
# =============================================================================
# Basic Fixtures
# =============================================================================
@pytest.fixture @pytest.fixture
def anyio_backend(): def anyio_backend():
@@ -15,7 +77,7 @@ def anyio_backend():
@pytest.fixture @pytest.fixture
async def client(): async def client():
"""Async HTTP client for testing.""" """Async HTTP client for testing (no auth)."""
async with AsyncClient( async with AsyncClient(
transport=ASGITransport(app=app), transport=ASGITransport(app=app),
base_url="http://test" base_url="http://test"
@@ -32,3 +94,249 @@ async def auth_client():
headers={"X-API-Key": "test-api-key"} headers={"X-API-Key": "test-api-key"}
) as ac: ) as ac:
yield ac yield ac
# =============================================================================
# Integration Test Fixtures
# =============================================================================
@pytest.fixture
def ollama_url(request):
"""Get Ollama URL from command line or environment."""
return request.config.getoption("--ollama-url") or os.environ.get(
"OLLAMA_URL", "http://192.168.86.149:11434"
)
@pytest.fixture
def api_url(request):
"""Get API URL from command line or environment."""
return request.config.getoption("--api-url") or os.environ.get(
"WEBBER_API_URL", "http://localhost:8095"
)
@pytest.fixture
def sample_project():
"""Create a sample Python project for testing."""
with tempfile.TemporaryDirectory() as tmpdir:
project = Path(tmpdir)
# Create a realistic project structure
(project / "src").mkdir()
(project / "tests").mkdir()
# Main application file
(project / "src" / "__init__.py").write_text("")
(project / "src" / "main.py").write_text('''"""Main application module."""
def greet(name: str) -> str:
"""Greet someone by name.
Args:
name: The name to greet
Returns:
A greeting string
"""
return f"Hello, {name}!"
def add(a: int, b: int) -> int:
"""Add two numbers.
Args:
a: First number
b: Second number
Returns:
Sum of a and b
"""
return a + b
def divide(a: float, b: float) -> float:
"""Divide two numbers.
Args:
a: Dividend
b: Divisor
Returns:
Result of a / b
Raises:
ValueError: If b is zero
"""
if b == 0:
raise ValueError("Cannot divide by zero")
return a / b
if __name__ == "__main__":
print(greet("World"))
''')
# Utility module
(project / "src" / "utils.py").write_text('''"""Utility functions."""
def is_even(n: int) -> bool:
"""Check if a number is even."""
return n % 2 == 0
def is_prime(n: int) -> bool:
"""Check if a number is prime."""
if n < 2:
return False
for i in range(2, int(n ** 0.5) + 1):
if n % i == 0:
return False
return True
def factorial(n: int) -> int:
"""Calculate factorial recursively."""
if n <= 1:
return 1
return n * factorial(n - 1)
def fibonacci(n: int) -> list[int]:
"""Generate Fibonacci sequence up to n terms."""
if n <= 0:
return []
if n == 1:
return [0]
fib = [0, 1]
for _ in range(2, n):
fib.append(fib[-1] + fib[-2])
return fib
''')
# Test file
(project / "tests" / "__init__.py").write_text("")
(project / "tests" / "test_main.py").write_text('''"""Tests for main module."""
import pytest
from src.main import greet, add, divide
def test_greet():
assert greet("World") == "Hello, World!"
def test_add():
assert add(2, 3) == 5
def test_divide():
assert divide(10, 2) == 5.0
def test_divide_by_zero():
with pytest.raises(ValueError):
divide(1, 0)
''')
# README
(project / "README.md").write_text('''# Sample Project
A simple Python project for testing Webber's code exploration.
## Features
- Greeting functionality
- Math utilities
- Comprehensive test suite
## Usage
```python
from src.main import greet, add
print(greet("World"))
print(add(2, 3))
```
## Testing
```bash
pytest tests/
```
''')
# Configuration files
(project / "pyproject.toml").write_text('''[project]
name = "sample-project"
version = "0.1.0"
[tool.pytest.ini_options]
testpaths = ["tests"]
''')
yield project
@pytest.fixture
def sample_project_with_bug():
"""Create a sample project with intentional bugs for testing."""
with tempfile.TemporaryDirectory() as tmpdir:
project = Path(tmpdir)
(project / "buggy.py").write_text('''"""Module with intentional bugs."""
def divide_numbers(a, b):
"""Divide two numbers - BUG: no zero check."""
return a / b # BUG: ZeroDivisionError if b is 0
def get_item(lst, index):
"""Get item from list - BUG: no bounds check."""
return lst[index] # BUG: IndexError if out of bounds
def parse_int(s):
"""Parse string to int - BUG: no error handling."""
return int(s) # BUG: ValueError if not a valid int
# TODO: Fix the division bug
# FIXME: Add bounds checking to get_item
''')
yield project
# =============================================================================
# E2E Test Fixtures
# =============================================================================
@pytest.fixture
async def live_client(api_url):
"""HTTP client for E2E tests against running server."""
async with AsyncClient(base_url=api_url, timeout=30.0) as client:
yield client
# =============================================================================
# Helper Functions
# =============================================================================
def assert_contains_any(text: str, substrings: list[str], case_sensitive: bool = False) -> bool:
"""Assert that text contains at least one of the substrings."""
check_text = text if case_sensitive else text.lower()
check_subs = substrings if case_sensitive else [s.lower() for s in substrings]
found = [s for s in check_subs if s in check_text]
assert found, f"Expected text to contain one of {substrings}, but none found in: {text[:200]}..."
return True
def assert_contains_all(text: str, substrings: list[str], case_sensitive: bool = False) -> bool:
"""Assert that text contains all of the substrings."""
check_text = text if case_sensitive else text.lower()
check_subs = substrings if case_sensitive else [s.lower() for s in substrings]
missing = [s for s in check_subs if s not in check_text]
assert not missing, f"Expected text to contain all of {substrings}, missing: {missing}"
return True
+276
View File
@@ -1,8 +1,15 @@
""" """
Tests for agent REST API endpoints. Tests for agent REST API endpoints.
Includes integration tests that verify real code paths work correctly
without over-mocking (only LLM calls are mocked).
""" """
from unittest.mock import AsyncMock, MagicMock, patch
import pytest import pytest
from src.domains.agents.schemas import PermissionMode
class TestAgentListEndpoint: class TestAgentListEndpoint:
"""Tests for GET /agents/ endpoint.""" """Tests for GET /agents/ endpoint."""
@@ -21,6 +28,18 @@ class TestAgentListEndpoint:
agent_names = [a["name"] for a in data["agents"]] agent_names = [a["name"] for a in data["agents"]]
assert "explore" in agent_names assert "explore" in agent_names
@pytest.mark.anyio
async def test_list_agents_returns_descriptions(self, auth_client):
"""Test that agent list includes descriptions."""
response = await auth_client.get("/agents/")
assert response.status_code == 200
data = response.json()
for agent in data["agents"]:
assert "name" in agent
assert "description" in agent
assert len(agent["description"]) > 0
class TestAgentInfoEndpoint: class TestAgentInfoEndpoint:
"""Tests for GET /agents/{agent_type} endpoint.""" """Tests for GET /agents/{agent_type} endpoint."""
@@ -42,6 +61,13 @@ class TestAgentInfoEndpoint:
assert response.status_code == 404 assert response.status_code == 404
@pytest.mark.anyio
async def test_get_agent_empty_name(self, auth_client):
"""Test getting agent with empty name."""
response = await auth_client.get("/agents/")
# This is the list endpoint, should return 200
assert response.status_code == 200
class TestAgentRunEndpoint: class TestAgentRunEndpoint:
"""Tests for POST /agents/run endpoint.""" """Tests for POST /agents/run endpoint."""
@@ -73,3 +99,253 @@ class TestAgentRunEndpoint:
) )
assert response.status_code == 422 # Validation error assert response.status_code == 422 # Validation error
@pytest.mark.anyio
async def test_run_missing_prompt(self, auth_client):
"""Test running with missing prompt."""
response = await auth_client.post(
"/agents/run",
json={
"agent_type": "explore",
"working_dir": "."
}
)
assert response.status_code == 422
@pytest.mark.anyio
async def test_run_empty_body(self, auth_client):
"""Test running with empty request body."""
response = await auth_client.post("/agents/run", json={})
assert response.status_code == 422
class TestAgentStreamEndpoint:
"""Tests for POST /agents/stream endpoint."""
@pytest.mark.anyio
async def test_stream_with_unknown_agent(self, auth_client):
"""Test streaming unknown agent type."""
response = await auth_client.post(
"/agents/stream",
json={
"prompt": "test",
"agent_type": "nonexistent",
"working_dir": "."
}
)
assert response.status_code == 400
assert "Unknown agent" in response.json()["detail"]
@pytest.mark.anyio
async def test_stream_request_validation(self, auth_client):
"""Test stream request validation."""
response = await auth_client.post(
"/agents/stream",
json={
"agent_type": "explore"
# Missing prompt
}
)
assert response.status_code == 422
@pytest.mark.anyio
async def test_stream_content_type(self, auth_client):
"""Test that stream endpoint returns correct content type."""
# Note: This test would require mocking the agent to avoid LLM calls
# For now, we just verify validation works
response = await auth_client.post(
"/agents/stream",
json={
"prompt": "test",
"agent_type": "nonexistent",
"working_dir": "."
}
)
# Unknown agent returns 400, not streaming
assert response.status_code == 400
class TestPermissionModeIntegration:
"""
Integration tests for permission mode handling.
These tests verify that mode strings are correctly converted to enums
and that the full request->router->agent flow works for each mode.
"""
@pytest.mark.anyio
async def test_run_with_default_mode(self, auth_client):
"""Test running agent with default mode passes through correctly."""
# Mock the agent.run method to avoid LLM calls
mock_result = MagicMock()
mock_result.output = "Test response"
with patch("src.domains.agents.task.agent.TaskAgentImpl._get_agent_for_mode") as mock_get_agent:
mock_agent = MagicMock()
mock_agent.run = AsyncMock(return_value=mock_result)
mock_get_agent.return_value = mock_agent
response = await auth_client.post(
"/agents/run",
json={
"prompt": "test prompt",
"agent_type": "task",
"working_dir": ".",
"mode": "default"
}
)
assert response.status_code == 200
data = response.json()
assert data["success"] is True
assert data["mode"] == "default"
# Verify mode was converted to enum and passed correctly
mock_get_agent.assert_called_once_with(PermissionMode.default)
@pytest.mark.anyio
async def test_run_with_plan_mode(self, auth_client):
"""Test running agent with plan mode passes through correctly."""
mock_result = MagicMock()
mock_result.output = "Plan response"
with patch("src.domains.agents.task.agent.TaskAgentImpl._get_agent_for_mode") as mock_get_agent:
mock_agent = MagicMock()
mock_agent.run = AsyncMock(return_value=mock_result)
mock_get_agent.return_value = mock_agent
response = await auth_client.post(
"/agents/run",
json={
"prompt": "test prompt",
"agent_type": "task",
"working_dir": ".",
"mode": "plan"
}
)
assert response.status_code == 200
data = response.json()
assert data["mode"] == "plan"
mock_get_agent.assert_called_once_with(PermissionMode.plan)
@pytest.mark.anyio
async def test_run_with_auto_accept_mode(self, auth_client):
"""Test running agent with auto_accept mode passes through correctly."""
mock_result = MagicMock()
mock_result.output = "Auto accept response"
with patch("src.domains.agents.task.agent.TaskAgentImpl._get_agent_for_mode") as mock_get_agent:
mock_agent = MagicMock()
mock_agent.run = AsyncMock(return_value=mock_result)
mock_get_agent.return_value = mock_agent
response = await auth_client.post(
"/agents/run",
json={
"prompt": "test prompt",
"agent_type": "task",
"working_dir": ".",
"mode": "auto_accept"
}
)
assert response.status_code == 200
data = response.json()
assert data["mode"] == "auto_accept"
mock_get_agent.assert_called_once_with(PermissionMode.auto_accept)
@pytest.mark.anyio
async def test_run_with_invalid_mode(self, auth_client):
"""Test running agent with invalid mode returns validation error."""
response = await auth_client.post(
"/agents/run",
json={
"prompt": "test prompt",
"agent_type": "task",
"working_dir": ".",
"mode": "invalid_mode"
}
)
assert response.status_code == 422
@pytest.mark.anyio
async def test_stream_with_plan_mode(self, auth_client):
"""Test streaming agent with plan mode passes through correctly."""
from src.domains.agents.schemas import StreamEvent, StreamEventType
async def mock_event_stream(*args, **kwargs):
"""Mock event-based stream."""
yield StreamEvent(event=StreamEventType.thinking, message="Starting...")
yield StreamEvent(event=StreamEventType.response, text="chunk1")
yield StreamEvent(event=StreamEventType.response, text="chunk2")
yield StreamEvent(event=StreamEventType.done, mode="plan")
with patch("src.domains.agents.task.agent.TaskAgentImpl.run_stream") as mock_run_stream:
mock_run_stream.return_value = mock_event_stream()
response = await auth_client.post(
"/agents/stream",
json={
"prompt": "test prompt",
"agent_type": "task",
"working_dir": ".",
"mode": "plan"
}
)
assert response.status_code == 200
assert response.headers["content-type"] == "text/event-stream; charset=utf-8"
mock_run_stream.assert_called_once()
class TestAgentMethodSignatures:
"""
Tests to verify agent method signatures match expected interfaces.
These catch issues like passing invalid kwargs to methods.
"""
def test_task_agent_run_accepts_mode(self):
"""Verify TaskAgentImpl.run() accepts mode parameter."""
import inspect
from src.domains.agents.task.agent import TaskAgentImpl
sig = inspect.signature(TaskAgentImpl.run)
params = list(sig.parameters.keys())
assert "mode" in params
# Verify mode has correct type annotation
mode_param = sig.parameters["mode"]
assert mode_param.default == PermissionMode.default
def test_task_agent_run_stream_accepts_mode(self):
"""Verify TaskAgentImpl.run_stream() accepts mode parameter."""
import inspect
from src.domains.agents.task.agent import TaskAgentImpl
sig = inspect.signature(TaskAgentImpl.run_stream)
params = list(sig.parameters.keys())
assert "mode" in params
mode_param = sig.parameters["mode"]
assert mode_param.default == PermissionMode.default
def test_trace_span_signature(self):
"""Verify trace_span only accepts expected parameters."""
import inspect
from src.shared.logging import trace_span
sig = inspect.signature(trace_span.__init__)
params = list(sig.parameters.keys())
# Should only have self, name, logger - not mode or other extras
assert params == ["self", "name", "logger"]
+481
View File
@@ -0,0 +1,481 @@
"""
Tests for coding tools (edit, write, bash full).
"""
import tempfile
from pathlib import Path
import pytest
from src.domains.tools.file.edit import EditFileTool
from src.domains.tools.file.write import WriteFileTool
from src.domains.tools.shell.bash_full import BashTool
class TestEditFileTool:
"""Tests for EditFileTool."""
@pytest.fixture
def tool(self):
return EditFileTool()
@pytest.fixture
def temp_file(self):
"""Create a temporary file with content."""
with tempfile.NamedTemporaryFile(mode='w', suffix='.py', delete=False) as f:
f.write("def hello():\n return 'Hello'\n\ndef world():\n return 'World'\n")
f.flush()
yield Path(f.name)
Path(f.name).unlink(missing_ok=True)
@pytest.mark.anyio
async def test_edit_file_success(self, tool, temp_file):
"""Test successful single replacement."""
result = await tool.execute(
file_path=str(temp_file),
old_string="return 'Hello'",
new_string="return 'Hi'"
)
assert result.success
assert "Hi" in Path(temp_file).read_text()
assert "Hello" not in Path(temp_file).read_text()
@pytest.mark.anyio
async def test_edit_file_not_found(self, tool):
"""Test editing non-existent file."""
result = await tool.execute(
file_path="/nonexistent/file.py",
old_string="old",
new_string="new"
)
assert not result.success
assert "not found" in result.error.lower()
@pytest.mark.anyio
async def test_edit_file_old_string_not_found(self, tool, temp_file):
"""Test when old_string doesn't exist in file."""
result = await tool.execute(
file_path=str(temp_file),
old_string="nonexistent text",
new_string="replacement"
)
assert not result.success
assert "not found" in result.error.lower()
@pytest.mark.anyio
async def test_edit_file_multiple_matches_error(self, tool, temp_file):
"""Test error when old_string has multiple matches and replace_all=False."""
result = await tool.execute(
file_path=str(temp_file),
old_string="return",
new_string="yield"
)
assert not result.success
assert "2" in result.error # Should mention count
@pytest.mark.anyio
async def test_edit_file_replace_all(self, tool, temp_file):
"""Test replace_all=True replaces all occurrences."""
result = await tool.execute(
file_path=str(temp_file),
old_string="return",
new_string="yield",
replace_all=True
)
assert result.success
content = Path(temp_file).read_text()
assert "return" not in content
assert content.count("yield") == 2
@pytest.mark.anyio
async def test_edit_file_path_restriction(self, temp_file):
"""Test path restriction enforcement."""
tool = EditFileTool(allowed_paths=["/some/other/path"])
result = await tool.execute(
file_path=str(temp_file),
old_string="Hello",
new_string="Hi"
)
assert not result.success
assert "not in allowed" in result.error.lower()
@pytest.mark.anyio
async def test_edit_file_empty_old_string(self, tool, temp_file):
"""Test that empty old_string is rejected."""
result = await tool.execute(
file_path=str(temp_file),
old_string="",
new_string="new"
)
assert not result.success
assert "empty" in result.error.lower()
@pytest.mark.anyio
async def test_edit_file_same_string(self, tool, temp_file):
"""Test that identical old/new strings are rejected."""
result = await tool.execute(
file_path=str(temp_file),
old_string="Hello",
new_string="Hello"
)
assert not result.success
assert "identical" in result.error.lower()
class TestWriteFileTool:
"""Tests for WriteFileTool."""
@pytest.fixture
def tool(self):
return WriteFileTool()
@pytest.fixture
def temp_dir(self):
"""Create a temporary directory."""
with tempfile.TemporaryDirectory() as tmpdir:
yield Path(tmpdir)
@pytest.mark.anyio
async def test_write_new_file(self, tool, temp_dir):
"""Test creating a new file."""
file_path = temp_dir / "new_file.py"
result = await tool.execute(
file_path=str(file_path),
content="# New file\nprint('hello')"
)
assert result.success
assert file_path.exists()
assert "hello" in file_path.read_text()
assert result.metadata.get("overwritten") is False
@pytest.mark.anyio
async def test_write_overwrite_existing(self, tool, temp_dir):
"""Test overwriting an existing file."""
file_path = temp_dir / "existing.txt"
file_path.write_text("old content")
result = await tool.execute(
file_path=str(file_path),
content="new content"
)
assert result.success
assert file_path.read_text() == "new content"
assert result.metadata.get("overwritten") is True
@pytest.mark.anyio
async def test_write_file_path_restriction(self, temp_dir):
"""Test path restriction enforcement."""
tool = WriteFileTool(allowed_paths=["/some/other/path"])
result = await tool.execute(
file_path=str(temp_dir / "file.txt"),
content="content"
)
assert not result.success
assert "not in allowed" in result.error.lower()
@pytest.mark.anyio
async def test_write_file_parent_not_exists(self, tool, temp_dir):
"""Test writing to path where parent directory doesn't exist."""
file_path = temp_dir / "nonexistent_dir" / "file.txt"
result = await tool.execute(
file_path=str(file_path),
content="content"
)
assert not result.success
assert "parent" in result.error.lower() or "directory" in result.error.lower()
@pytest.mark.anyio
async def test_write_file_content_size_limit(self, temp_dir):
"""Test content size limit enforcement."""
tool = WriteFileTool(max_content_size=100)
result = await tool.execute(
file_path=str(temp_dir / "large.txt"),
content="x" * 200
)
assert not result.success
assert "large" in result.error.lower() or "size" in result.error.lower()
@pytest.mark.anyio
async def test_write_file_returns_metadata(self, tool, temp_dir):
"""Test that metadata is returned correctly."""
file_path = temp_dir / "meta.txt"
content = "line1\nline2\nline3"
result = await tool.execute(
file_path=str(file_path),
content=content
)
assert result.success
assert result.metadata.get("lines") == 3
assert result.metadata.get("file_size") == len(content.encode('utf-8'))
class TestBashTool:
"""Tests for BashTool (full write capabilities)."""
@pytest.fixture
def tool(self):
return BashTool()
@pytest.fixture
def temp_dir(self):
"""Create a temporary directory."""
with tempfile.TemporaryDirectory() as tmpdir:
yield Path(tmpdir)
# === Allowed commands ===
@pytest.mark.anyio
async def test_ls_command(self, tool, temp_dir):
"""Test ls is allowed."""
result = await tool.execute(command="ls", cwd=str(temp_dir))
assert result.success
@pytest.mark.anyio
async def test_mkdir_command(self, tool, temp_dir):
"""Test mkdir is allowed."""
result = await tool.execute(
command=f"mkdir {temp_dir}/new_dir",
cwd=str(temp_dir)
)
assert result.success
assert (temp_dir / "new_dir").exists()
@pytest.mark.anyio
async def test_touch_command(self, tool, temp_dir):
"""Test touch is allowed."""
result = await tool.execute(
command=f"touch {temp_dir}/new_file.txt",
cwd=str(temp_dir)
)
assert result.success
assert (temp_dir / "new_file.txt").exists()
@pytest.mark.anyio
async def test_cp_command(self, tool, temp_dir):
"""Test cp within allowed paths."""
(temp_dir / "source.txt").write_text("content")
result = await tool.execute(
command=f"cp {temp_dir}/source.txt {temp_dir}/dest.txt",
cwd=str(temp_dir)
)
assert result.success
assert (temp_dir / "dest.txt").exists()
@pytest.mark.anyio
async def test_mv_command(self, tool, temp_dir):
"""Test mv within allowed paths."""
(temp_dir / "source.txt").write_text("content")
result = await tool.execute(
command=f"mv {temp_dir}/source.txt {temp_dir}/moved.txt",
cwd=str(temp_dir)
)
assert result.success
assert (temp_dir / "moved.txt").exists()
assert not (temp_dir / "source.txt").exists()
@pytest.mark.anyio
async def test_command_chaining_and(self, tool, temp_dir):
"""Test && chaining is allowed."""
result = await tool.execute(
command=f"mkdir {temp_dir}/dir1 && touch {temp_dir}/dir1/file.txt",
cwd=str(temp_dir)
)
assert result.success
assert (temp_dir / "dir1" / "file.txt").exists()
@pytest.mark.anyio
async def test_command_chaining_or(self, tool, temp_dir):
"""Test || chaining is allowed."""
result = await tool.execute(
command=f"ls {temp_dir}/nonexistent || echo 'fallback'",
cwd=str(temp_dir)
)
# Either succeeds or falls back
assert result.success or "fallback" in (result.data or "")
@pytest.mark.anyio
async def test_echo_command(self, tool, temp_dir):
"""Test echo command."""
result = await tool.execute(
command="echo 'hello world'",
cwd=str(temp_dir)
)
assert result.success
assert "hello world" in result.data
@pytest.mark.anyio
async def test_git_status(self, tool, temp_dir):
"""Test git status is allowed."""
# Initialize a git repo first
await tool.execute(command="git init", cwd=str(temp_dir))
result = await tool.execute(command="git status", cwd=str(temp_dir))
assert result.success
@pytest.mark.anyio
async def test_git_add_allowed(self, tool, temp_dir):
"""Test git add is allowed."""
await tool.execute(command="git init", cwd=str(temp_dir))
(temp_dir / "file.txt").write_text("content")
result = await tool.execute(command="git add file.txt", cwd=str(temp_dir))
assert result.success
@pytest.mark.anyio
async def test_pip_help(self, tool, temp_dir):
"""Test pip help is allowed."""
result = await tool.execute(
command="pip --help",
cwd=str(temp_dir)
)
assert result.success
@pytest.mark.anyio
async def test_rm_single_file(self, tool, temp_dir):
"""Test rm of a single file is allowed."""
file_path = temp_dir / "to_delete.txt"
file_path.write_text("content")
result = await tool.execute(
command=f"rm {file_path}",
cwd=str(temp_dir)
)
assert result.success
assert not file_path.exists()
# === Forbidden commands ===
@pytest.mark.anyio
async def test_forbidden_sudo(self, tool, temp_dir):
"""Test sudo is blocked."""
result = await tool.execute(command="sudo ls", cwd=str(temp_dir))
assert not result.success
assert "forbidden" in result.error.lower()
@pytest.mark.anyio
async def test_forbidden_curl(self, tool, temp_dir):
"""Test curl is blocked."""
result = await tool.execute(
command="curl http://example.com",
cwd=str(temp_dir)
)
assert not result.success
assert "not allowed" in result.error.lower() or "forbidden" in result.error.lower()
@pytest.mark.anyio
async def test_forbidden_wget(self, tool, temp_dir):
"""Test wget is blocked."""
result = await tool.execute(
command="wget http://example.com",
cwd=str(temp_dir)
)
assert not result.success
@pytest.mark.anyio
async def test_forbidden_ssh(self, tool, temp_dir):
"""Test ssh is blocked."""
result = await tool.execute(
command="ssh user@host",
cwd=str(temp_dir)
)
assert not result.success
@pytest.mark.anyio
async def test_forbidden_rm_rf(self, tool, temp_dir):
"""Test rm -rf is blocked."""
result = await tool.execute(
command="rm -rf /tmp/test",
cwd=str(temp_dir)
)
assert not result.success
assert "not allowed" in result.error.lower() or "forbidden" in result.error.lower()
@pytest.mark.anyio
async def test_forbidden_rm_rf_dot(self, tool, temp_dir):
"""Test rm -rf . is blocked."""
result = await tool.execute(
command="rm -rf .",
cwd=str(temp_dir)
)
assert not result.success
@pytest.mark.anyio
async def test_forbidden_chmod_777(self, tool, temp_dir):
"""Test chmod 777 is blocked."""
result = await tool.execute(
command="chmod 777 /tmp/file",
cwd=str(temp_dir)
)
assert not result.success
assert "not allowed" in result.error.lower() or "forbidden" in result.error.lower()
@pytest.mark.anyio
async def test_forbidden_git_push(self, tool, temp_dir):
"""Test git push is blocked."""
await tool.execute(command="git init", cwd=str(temp_dir))
result = await tool.execute(
command="git push origin main",
cwd=str(temp_dir)
)
assert not result.success
assert "not allowed" in result.error.lower()
@pytest.mark.anyio
async def test_forbidden_pip_uninstall(self, tool, temp_dir):
"""Test pip uninstall is blocked."""
result = await tool.execute(
command="pip uninstall requests",
cwd=str(temp_dir)
)
assert not result.success
assert "not allowed" in result.error.lower()
# === Path restrictions ===
@pytest.mark.anyio
async def test_working_dir_not_allowed(self, temp_dir):
"""Test working directory restriction."""
tool = BashTool(allowed_paths=["/some/other/path"])
result = await tool.execute(
command="ls",
cwd=str(temp_dir)
)
assert not result.success
assert "not allowed" in result.error.lower()
@pytest.mark.anyio
async def test_cp_outside_allowed_paths(self, temp_dir):
"""Test cp to path outside allowed_paths fails."""
tool = BashTool(allowed_paths=[str(temp_dir)])
(temp_dir / "source.txt").write_text("content")
result = await tool.execute(
command=f"cp {temp_dir}/source.txt /tmp/dest.txt",
cwd=str(temp_dir)
)
assert not result.success
assert "allowed" in result.error.lower()
@pytest.mark.anyio
async def test_rm_outside_allowed_paths(self, temp_dir):
"""Test rm of path outside allowed_paths fails."""
tool = BashTool(allowed_paths=[str(temp_dir)])
result = await tool.execute(
command="rm /tmp/some_file.txt",
cwd=str(temp_dir)
)
assert not result.success
assert "allowed" in result.error.lower()
# === Timeout ===
@pytest.mark.anyio
async def test_timeout(self, tool, temp_dir):
"""Test command timeout using find on root (slow)."""
# Use find on a large directory which will be slow
result = await tool.execute(
command="find / -name '*.nonexistent' 2>/dev/null",
cwd=str(temp_dir),
timeout=1
)
# The command should either timeout or fail
# (it may complete quickly with errors, which is also acceptable)
assert not result.success or result.truncated or "timeout" in str(result.error or "").lower()
+385
View File
@@ -0,0 +1,385 @@
"""
Tests for conversations domain.
Tests conversation CRUD, context building, and API endpoints.
"""
from uuid import uuid4
import pytest
from src.domains.conversations.models import Conversation, Message
from src.domains.conversations.schemas import (
AddMessageRequest,
CreateConversationRequest,
)
class TestConversationModels:
"""Tests for conversation database models."""
def test_conversation_creation(self):
"""Test Conversation model creation with explicit values."""
conv = Conversation(
user_id="test-user",
agent_type="explore",
working_dir=".",
total_tokens=0,
)
assert conv.user_id == "test-user"
assert conv.agent_type == "explore"
assert conv.working_dir == "."
assert conv.total_tokens == 0
def test_conversation_with_values(self):
"""Test Conversation with explicit values."""
conv = Conversation(
user_id="test-user",
agent_type="plan",
working_dir="/tmp/project",
title="Test Conversation",
)
assert conv.agent_type == "plan"
assert conv.working_dir == "/tmp/project"
assert conv.title == "Test Conversation"
def test_message_creation(self):
"""Test Message model creation with explicit values."""
msg = Message(
conversation_id=uuid4(),
role="user",
content="Hello",
token_count=0,
is_summary=False,
)
assert msg.role == "user"
assert msg.content == "Hello"
assert msg.token_count == 0
assert msg.is_summary is False
def test_message_repr(self):
"""Test Message string representation."""
msg = Message(
conversation_id=uuid4(),
role="user",
content="This is a test message",
)
repr_str = repr(msg)
assert "user" in repr_str
assert "This is a test" in repr_str
class TestConversationSchemas:
"""Tests for Pydantic schemas."""
def test_create_request_defaults(self):
"""Test CreateConversationRequest defaults."""
request = CreateConversationRequest()
assert request.agent_type == "explore"
assert request.working_dir == "."
assert request.title is None
def test_create_request_custom(self):
"""Test CreateConversationRequest with values."""
request = CreateConversationRequest(
agent_type="task",
working_dir="/home/user/project",
title="My Task",
)
assert request.agent_type == "task"
assert request.working_dir == "/home/user/project"
assert request.title == "My Task"
def test_add_message_request_valid(self):
"""Test AddMessageRequest validation."""
request = AddMessageRequest(content="Hello, world!")
assert request.content == "Hello, world!"
def test_add_message_request_empty_fails(self):
"""Test that empty content fails validation."""
with pytest.raises(ValueError):
AddMessageRequest(content="")
class TestConversationAPI:
"""Tests for conversation API endpoints."""
@pytest.mark.anyio
async def test_create_conversation(self, auth_client):
"""Test creating a conversation."""
response = await auth_client.post(
"/conversations/",
json={"agent_type": "explore", "working_dir": "."}
)
assert response.status_code == 201
data = response.json()
assert "id" in data
assert data["agent_type"] == "explore"
assert data["total_tokens"] == 0
@pytest.mark.anyio
async def test_create_conversation_with_title(self, auth_client):
"""Test creating a conversation with title."""
response = await auth_client.post(
"/conversations/",
json={
"agent_type": "plan",
"working_dir": "/tmp",
"title": "Planning Session"
}
)
assert response.status_code == 201
data = response.json()
assert data["title"] == "Planning Session"
assert data["agent_type"] == "plan"
@pytest.mark.anyio
async def test_list_conversations_empty(self, auth_client):
"""Test listing conversations when empty."""
response = await auth_client.get("/conversations/")
assert response.status_code == 200
data = response.json()
assert "conversations" in data
assert "total" in data
@pytest.mark.anyio
async def test_get_conversation_not_found(self, auth_client):
"""Test getting non-existent conversation."""
fake_id = uuid4()
response = await auth_client.get(f"/conversations/{fake_id}")
assert response.status_code == 404
@pytest.mark.anyio
async def test_delete_conversation_not_found(self, auth_client):
"""Test deleting non-existent conversation."""
fake_id = uuid4()
response = await auth_client.delete(f"/conversations/{fake_id}")
assert response.status_code == 404
@pytest.mark.anyio
async def test_add_message_not_found(self, auth_client):
"""Test adding message to non-existent conversation."""
fake_id = uuid4()
response = await auth_client.post(
f"/conversations/{fake_id}/messages",
json={"content": "Hello"}
)
assert response.status_code == 404
class TestConversationService:
"""Tests for ConversationService business logic."""
@pytest.mark.anyio
async def test_context_prompt_no_history(self):
"""Test building context prompt with no history."""
from unittest.mock import MagicMock
from src.domains.conversations.service import ConversationService
# Create mock session
mock_session = MagicMock()
service = ConversationService(mock_session)
prompt = service.build_context_prompt([], "What files are here?")
assert "<current_request>" in prompt
assert "What files are here?" in prompt
assert "<recent_conversation>" not in prompt
assert "<conversation_summary>" not in prompt
@pytest.mark.anyio
async def test_context_prompt_with_history(self):
"""Test building context prompt with message history."""
from unittest.mock import MagicMock
from src.domains.conversations.models import Message
from src.domains.conversations.service import ConversationService
mock_session = MagicMock()
service = ConversationService(mock_session)
messages = [
Message(
conversation_id=uuid4(),
role="user",
content="Find Python files",
),
Message(
conversation_id=uuid4(),
role="assistant",
content="Found 10 Python files.",
),
]
prompt = service.build_context_prompt(messages, "Show the largest")
assert "<recent_conversation>" in prompt
assert "USER: Find Python files" in prompt
assert "ASSISTANT: Found 10 Python files" in prompt
assert "<current_request>" in prompt
assert "Show the largest" in prompt
@pytest.mark.anyio
async def test_context_prompt_with_summary(self):
"""Test building context prompt with summary message."""
from unittest.mock import MagicMock
from src.domains.conversations.models import Message
from src.domains.conversations.service import ConversationService
mock_session = MagicMock()
service = ConversationService(mock_session)
messages = [
Message(
conversation_id=uuid4(),
role="summary",
content="Previously discussed: project setup",
is_summary=True,
),
Message(
conversation_id=uuid4(),
role="user",
content="Now what?",
),
]
prompt = service.build_context_prompt(messages, "Continue")
assert "<conversation_summary>" in prompt
assert "Previously discussed: project setup" in prompt
class TestSummarization:
"""Tests for conversation summarization."""
def test_format_messages_for_summary(self):
"""Test formatting messages for summarization."""
from src.domains.conversations.models import Message
from src.domains.conversations.summarize import format_messages_for_summary
messages = [
Message(
conversation_id=uuid4(),
role="user",
content="Hello",
),
Message(
conversation_id=uuid4(),
role="assistant",
content="Hi there!",
),
]
formatted = format_messages_for_summary(messages)
assert "USER: Hello" in formatted
assert "ASSISTANT: Hi there!" in formatted
def test_format_messages_with_summary(self):
"""Test formatting messages that include a summary."""
from src.domains.conversations.models import Message
from src.domains.conversations.summarize import format_messages_for_summary
messages = [
Message(
conversation_id=uuid4(),
role="summary",
content="Previous context summary",
is_summary=True,
),
Message(
conversation_id=uuid4(),
role="user",
content="Continue",
),
]
formatted = format_messages_for_summary(messages)
assert "[Previous Summary]" in formatted
assert "Previous context summary" in formatted
class TestSaveMessagesAPI:
"""Tests for the save messages endpoint (no agent execution)."""
@pytest.mark.anyio
async def test_save_messages(self, auth_client):
"""Test saving a message pair without triggering agent."""
# First create a conversation
response = await auth_client.post(
"/conversations/",
json={"agent_type": "task", "working_dir": "."}
)
assert response.status_code == 201
conv_id = response.json()["id"]
# Save a message pair
response = await auth_client.post(
f"/conversations/{conv_id}/save",
json={
"user_content": "Find all Python files",
"assistant_content": "I found 5 Python files in the project.",
}
)
assert response.status_code == 200
data = response.json()
assert data["user_message"]["role"] == "user"
assert data["user_message"]["content"] == "Find all Python files"
assert data["assistant_message"]["role"] == "assistant"
assert data["assistant_message"]["content"] == "I found 5 Python files in the project."
assert data["total_tokens"] > 0
@pytest.mark.anyio
async def test_save_messages_not_found(self, auth_client):
"""Test saving messages to non-existent conversation."""
fake_id = uuid4()
response = await auth_client.post(
f"/conversations/{fake_id}/save",
json={
"user_content": "Test",
"assistant_content": "Response",
}
)
assert response.status_code == 404
@pytest.mark.anyio
async def test_save_messages_updates_token_count(self, auth_client):
"""Test that saving messages updates the conversation token count."""
# Create conversation
response = await auth_client.post(
"/conversations/",
json={"agent_type": "explore", "working_dir": "."}
)
conv_id = response.json()["id"]
assert response.json()["total_tokens"] == 0
# Save first message pair
response = await auth_client.post(
f"/conversations/{conv_id}/save",
json={
"user_content": "Hello",
"assistant_content": "Hi there!",
}
)
first_tokens = response.json()["total_tokens"]
assert first_tokens > 0
# Save second message pair
response = await auth_client.post(
f"/conversations/{conv_id}/save",
json={
"user_content": "How are you?",
"assistant_content": "I'm doing well, thank you for asking!",
}
)
second_tokens = response.json()["total_tokens"]
assert second_tokens > first_tokens
# Verify via get endpoint
response = await auth_client.get(f"/conversations/{conv_id}")
assert response.status_code == 200
assert response.json()["total_tokens"] == second_tokens
assert len(response.json()["messages"]) == 4
+225
View File
@@ -0,0 +1,225 @@
"""
End-to-end tests for the API.
These tests require the API server to be running and are skipped by default.
Run with: pytest tests/test_e2e.py -v --run-e2e
Start the server first: ./wakeup.sh
"""
import pytest
from tests.conftest import assert_contains_any
# All tests in this module require --run-e2e
pytestmark = [pytest.mark.e2e, pytest.mark.slow]
class TestHealthEndpoint:
"""E2E tests for health endpoint."""
@pytest.mark.anyio
async def test_health_check(self, live_client):
"""Test that health endpoint responds."""
response = await live_client.get("/health")
assert response.status_code == 200
data = response.json()
assert data.get("status") == "healthy"
class TestAgentEndpoints:
"""E2E tests for agent endpoints."""
@pytest.mark.anyio
async def test_list_agents(self, live_client):
"""Test listing agents via live API."""
response = await live_client.get("/agents/")
assert response.status_code == 200
data = response.json()
assert "agents" in data
assert len(data["agents"]) >= 1
# Verify explore agent exists
names = [a["name"] for a in data["agents"]]
assert "explore" in names
@pytest.mark.anyio
async def test_get_agent_info(self, live_client):
"""Test getting agent info via live API."""
response = await live_client.get("/agents/explore")
assert response.status_code == 200
data = response.json()
assert data["name"] == "explore"
assert "description" in data
@pytest.mark.anyio
async def test_run_agent(self, live_client, sample_project):
"""Test running agent via live API."""
response = await live_client.post(
"/agents/run",
json={
"agent_type": "explore",
"prompt": "List all files in this directory",
"working_dir": str(sample_project),
},
timeout=60.0, # LLM calls can be slow
)
assert response.status_code == 200
data = response.json()
assert data.get("success") is True
assert "response" in data
assert len(data["response"]) > 0
class TestStreamingEndpoint:
"""E2E tests for streaming endpoint."""
@pytest.mark.anyio
async def test_stream_agent(self, live_client, sample_project):
"""Test streaming agent responses via live API."""
async with live_client.stream(
"POST",
"/agents/stream",
json={
"agent_type": "explore",
"prompt": "List all Python files",
"working_dir": str(sample_project),
},
timeout=60.0,
) as response:
assert response.status_code == 200
assert response.headers.get("content-type") == "text/event-stream; charset=utf-8"
# Collect chunks
chunks = []
async for line in response.aiter_lines():
if line.startswith("data: "):
chunks.append(line)
# Should receive some data
assert len(chunks) >= 1
@pytest.mark.anyio
async def test_stream_invalid_agent(self, live_client):
"""Test streaming with invalid agent type."""
response = await live_client.post(
"/agents/stream",
json={
"agent_type": "nonexistent",
"prompt": "test",
"working_dir": ".",
},
)
assert response.status_code == 400
class TestErrorHandling:
"""E2E tests for error handling."""
@pytest.mark.anyio
async def test_invalid_agent_type(self, live_client):
"""Test error response for invalid agent."""
response = await live_client.post(
"/agents/run",
json={
"agent_type": "nonexistent",
"prompt": "test",
"working_dir": ".",
},
)
assert response.status_code == 400
data = response.json()
assert "detail" in data
@pytest.mark.anyio
async def test_missing_required_fields(self, live_client):
"""Test validation error for missing fields."""
response = await live_client.post(
"/agents/run",
json={
"agent_type": "explore",
# Missing prompt
},
)
assert response.status_code == 422
@pytest.mark.anyio
async def test_agent_not_found(self, live_client):
"""Test 404 for unknown agent info."""
response = await live_client.get("/agents/unknown_agent")
assert response.status_code == 404
class TestRealWorldScenarios:
"""E2E tests for real-world usage scenarios."""
@pytest.mark.anyio
async def test_explore_codebase(self, live_client, sample_project):
"""Test exploring a real codebase."""
response = await live_client.post(
"/agents/run",
json={
"agent_type": "explore",
"prompt": "What functions are defined in the src directory?",
"working_dir": str(sample_project),
},
timeout=60.0,
)
assert response.status_code == 200
data = response.json()
assert data.get("success") is True
# Should mention some functions
assert_contains_any(
data.get("response", ""),
["greet", "add", "divide", "factorial", "function"]
)
@pytest.mark.anyio
async def test_find_bugs(self, live_client, sample_project_with_bug):
"""Test finding bugs in code."""
response = await live_client.post(
"/agents/run",
json={
"agent_type": "explore",
"prompt": "Review buggy.py and identify any potential bugs or issues.",
"working_dir": str(sample_project_with_bug),
},
timeout=60.0,
)
assert response.status_code == 200
data = response.json()
assert data.get("success") is True
# Should identify issues
assert_contains_any(
data.get("response", ""),
["zero", "division", "bug", "error", "issue", "check"]
)
@pytest.mark.anyio
async def test_read_and_summarize(self, live_client, sample_project):
"""Test reading and summarizing a file."""
response = await live_client.post(
"/agents/run",
json={
"agent_type": "explore",
"prompt": "Read README.md and give me a one-sentence summary.",
"working_dir": str(sample_project),
},
timeout=60.0,
)
assert response.status_code == 200
data = response.json()
assert data.get("success") is True
assert len(data.get("response", "")) > 10
+1 -1
View File
@@ -6,8 +6,8 @@ from pathlib import Path
import pytest import pytest
from src.domains.tools.gitignore import GitignoreFilter, filter_gitignored
from src.domains.tools.file.glob import GlobFilesTool from src.domains.tools.file.glob import GlobFilesTool
from src.domains.tools.gitignore import GitignoreFilter, filter_gitignored
from src.domains.tools.search.grep import GrepContentTool from src.domains.tools.search.grep import GrepContentTool
+181
View File
@@ -0,0 +1,181 @@
"""
Integration tests with real LLM.
These tests require Ollama to be running and are skipped by default.
Run with: pytest tests/test_integration.py -v --run-integration
Note: These tests are slow (each takes 5-30 seconds depending on LLM response time).
"""
import pytest
from tests.conftest import assert_contains_any
# All tests in this module require --run-integration
pytestmark = [pytest.mark.integration, pytest.mark.slow]
class TestExploreAgentIntegration:
"""Integration tests for the explore agent with real LLM."""
@pytest.fixture
def agent(self):
"""Get the explore agent."""
from src.domains.agents.explore.agent import explore_agent
return explore_agent
@pytest.mark.anyio
async def test_agent_can_list_files(self, agent, sample_project):
"""Test that agent can use glob to list files."""
result = await agent.run(
"List all Python files in this project. Just list the filenames.",
working_dir=str(sample_project),
allowed_paths=[str(sample_project)],
)
# Agent should mention the Python files
assert_contains_any(result, ["main.py", "utils.py", ".py"])
@pytest.mark.anyio
async def test_agent_can_read_file(self, agent, sample_project):
"""Test that agent can read file contents."""
result = await agent.run(
"Use the read_file tool to read src/main.py and list what functions are defined.",
working_dir=str(sample_project),
allowed_paths=[str(sample_project)],
)
# Agent should either mention functions or indicate it read the file
# LLMs can be unpredictable, so we check for various valid responses
assert_contains_any(result, [
"greet", "add", "divide", # Function names
"function", "def", # Generic function mentions
"main.py", # File reference
])
@pytest.mark.anyio
async def test_agent_can_search_content(self, agent, sample_project):
"""Test that agent can grep for patterns."""
result = await agent.run(
"Search for all TODO and FIXME comments in the codebase.",
working_dir=str(sample_project),
allowed_paths=[str(sample_project)],
)
# Should find no TODOs in the clean sample project
# (or correctly report none found)
assert result is not None and len(result) > 0
@pytest.mark.anyio
async def test_agent_finds_bugs(self, agent, sample_project_with_bug):
"""Test that agent can identify potential bugs."""
result = await agent.run(
"Review buggy.py and identify potential bugs or issues.",
working_dir=str(sample_project_with_bug),
allowed_paths=[str(sample_project_with_bug)],
)
# Agent should identify at least one issue
assert_contains_any(result, [
"zero", "division", "error", "bug", "issue",
"index", "bounds", "check", "validation"
])
@pytest.mark.anyio
async def test_agent_respects_path_restrictions(self, agent, sample_project):
"""Test that agent cannot access files outside allowed paths."""
result = await agent.run(
"Try to read the file /etc/passwd and show its contents.",
working_dir=str(sample_project),
allowed_paths=[str(sample_project)],
)
# Agent should not be able to read /etc/passwd
# Should not contain actual passwd file content
assert "root:x:0:0" not in result
class TestExploreAgentToolUsage:
"""Test that the agent correctly uses tools."""
@pytest.fixture
def agent(self):
"""Get the explore agent."""
from src.domains.agents.explore.agent import explore_agent
return explore_agent
@pytest.mark.anyio
async def test_agent_uses_glob_for_file_search(self, agent, sample_project):
"""Test that agent uses glob when searching for files."""
result = await agent.run(
"What markdown files exist in this project?",
working_dir=str(sample_project),
allowed_paths=[str(sample_project)],
)
# Should find README.md
assert_contains_any(result, ["readme", "README.md", ".md"])
@pytest.mark.anyio
async def test_agent_uses_grep_for_content_search(self, agent, sample_project):
"""Test that agent uses grep for content search."""
result = await agent.run(
"Find where the 'factorial' function is defined and show its implementation.",
working_dir=str(sample_project),
allowed_paths=[str(sample_project)],
)
# Should find factorial in utils.py
assert_contains_any(result, ["factorial", "recursive", "utils"])
@pytest.mark.anyio
async def test_agent_reads_readme(self, agent, sample_project):
"""Test that agent can read and summarize README."""
result = await agent.run(
"Read the README.md and summarize what this project does.",
working_dir=str(sample_project),
allowed_paths=[str(sample_project)],
)
# Should understand the project from README
assert_contains_any(result, ["project", "python", "testing", "greeting", "math"])
@pytest.mark.anyio
async def test_agent_understands_project_structure(self, agent, sample_project):
"""Test that agent can understand project structure."""
result = await agent.run(
"Describe the directory structure of this project.",
working_dir=str(sample_project),
allowed_paths=[str(sample_project)],
)
# Should identify key directories
assert_contains_any(result, ["src", "tests", "directory", "folder", "structure"])
class TestAgentStreaming:
"""Test agent streaming functionality."""
@pytest.fixture
def agent(self):
"""Get the explore agent."""
from src.domains.agents.explore.agent import explore_agent
return explore_agent
@pytest.mark.anyio
async def test_agent_can_stream(self, agent, sample_project):
"""Test that agent streaming works."""
chunks = []
async for chunk in agent.run_stream(
"List the Python files in this project.",
working_dir=str(sample_project),
allowed_paths=[str(sample_project)],
):
chunks.append(chunk)
# Should receive at least one chunk
assert len(chunks) >= 1
# Combined result should mention files
full_result = "".join(chunks)
assert len(full_result) > 0
+55
View File
@@ -0,0 +1,55 @@
"""The sanitising Ollama client must actually be reachable.
src/ollama/provider.py exists to work around Ollama rejecting assistant
messages that carry `content: null` alongside `tool_calls`. On 2026-08-11 it
raised AttributeError the moment anything touched `.chat`: it fetched the
parent's getter via `AsyncOpenAI.chat.fget`, and openai had made `chat` a
functools.cached_property, whose getter is `.func`.
Nothing caught it. The line carried a bare `# type: ignore`, so mypy stayed
quiet, and the endpoints that reach this code had served no requests in 30 days,
so no user hit it either. The mitigation was dead and everything looked fine.
These tests exercise the path rather than the types, because the failure was a
runtime attribute lookup that no annotation would have caught.
"""
from openai import AsyncOpenAI
from src.ollama.provider import _parent_chat, _SanitizedAsyncOpenAI, get_ollama_provider
class TestSanitizedClientIsReachable:
def test_chat_can_be_accessed(self):
"""The regression: this raised AttributeError, not a type error."""
client = _SanitizedAsyncOpenAI(base_url="http://localhost:11434/v1")
chat = client.chat
assert chat is not None
assert chat.completions is not None
def test_provider_reaches_completions(self):
"""The full chain an agent request walks, short of the network call."""
provider = get_ollama_provider()
assert provider._openai_client.chat.completions is not None
def test_parent_lookup_survives_either_descriptor_shape(self):
"""openai has used both property and cached_property for `chat`.
Whichever it is, the parent's own getter must be found — the previous
code hardcoded `.fget` and broke on the switch to cached_property.
"""
descriptor = AsyncOpenAI.__dict__["chat"]
assert hasattr(descriptor, "func") or hasattr(descriptor, "fget"), (
"AsyncOpenAI.chat exposes neither getter; _parent_chat needs updating"
)
client = _SanitizedAsyncOpenAI(base_url="http://localhost:11434/v1")
assert _parent_chat(client) is not None
def test_parent_chat_is_not_the_override(self):
"""It must return openai's Chat, not recurse into the subclass property.
Returning the subclass's own `chat` would be infinite recursion, and the
sanitiser would wrap itself instead of the real completions resource.
"""
client = _SanitizedAsyncOpenAI(base_url="http://localhost:11434/v1")
assert type(_parent_chat(client)).__name__ != "_SanitizedChat"
+152
View File
@@ -0,0 +1,152 @@
"""
Tests for the Plan agent.
Tests registration, API endpoints, and tool restrictions.
"""
import pytest
from src.domains.agents.base import get_agent, list_agents
from src.domains.agents.plan import PlanAgentImpl, plan_agent
class TestPlanAgentRegistration:
"""Tests for Plan agent registration."""
def test_plan_agent_registered(self):
"""Test that plan agent is registered in registry."""
agent = get_agent("plan")
assert agent is not None
assert agent.name == "plan"
def test_plan_agent_in_list(self):
"""Test that plan agent appears in agent list."""
agents = list_agents()
names = [a["name"] for a in agents]
assert "plan" in names
def test_plan_agent_has_description(self):
"""Test that plan agent has a description."""
agent = get_agent("plan")
assert agent is not None
assert len(agent.description) > 0
assert "plan" in agent.description.lower() or "architect" in agent.description.lower()
def test_plan_agent_singleton(self):
"""Test that plan_agent is the registered instance."""
registered = get_agent("plan")
assert registered is plan_agent
def test_plan_agent_is_correct_type(self):
"""Test that plan agent is correct implementation type."""
assert isinstance(plan_agent, PlanAgentImpl)
class TestPlanAgentTools:
"""Tests for Plan agent tool restrictions."""
def test_plan_agent_has_read_only_tools(self):
"""Test that plan agent has read-only tools."""
# Access the underlying PydanticAI agent to check tools
agent = plan_agent.agent
tool_names = list(agent._function_toolset.tools.keys())
# Should have read-only tools
assert "read_file" in tool_names
assert "glob_files" in tool_names
assert "grep_content" in tool_names
assert "bash_readonly" in tool_names
def test_plan_agent_no_write_tools(self):
"""Test that plan agent does NOT have write tools."""
agent = plan_agent.agent
tool_names = list(agent._function_toolset.tools.keys())
# Should NOT have write tools
assert "edit_file" not in tool_names
assert "write_file" not in tool_names
assert "bash" not in tool_names
assert "web_search" not in tool_names
def test_plan_agent_tool_count(self):
"""Test that plan agent has exactly 4 tools."""
agent = plan_agent.agent
tool_count = len(agent._function_toolset.tools)
assert tool_count == 4
class TestPlanAgentAPI:
"""Tests for Plan agent REST API."""
@pytest.mark.anyio
async def test_list_agents_includes_plan(self, auth_client):
"""Test that agent list includes plan agent."""
response = await auth_client.get("/agents/")
assert response.status_code == 200
data = response.json()
names = [a["name"] for a in data["agents"]]
assert "plan" in names
@pytest.mark.anyio
async def test_get_plan_agent_info(self, auth_client):
"""Test getting plan agent info."""
response = await auth_client.get("/agents/plan")
assert response.status_code == 200
data = response.json()
assert data["name"] == "plan"
assert "description" in data
assert len(data["description"]) > 0
@pytest.mark.anyio
async def test_run_plan_with_invalid_body(self, auth_client):
"""Test running plan agent with invalid request."""
response = await auth_client.post(
"/agents/run",
json={
"agent_type": "plan",
# Missing prompt
}
)
assert response.status_code == 422
@pytest.mark.anyio
async def test_stream_plan_with_invalid_body(self, auth_client):
"""Test streaming plan agent with invalid request."""
response = await auth_client.post(
"/agents/stream",
json={
"agent_type": "plan",
# Missing prompt
}
)
assert response.status_code == 422
class TestPlanAgentProperties:
"""Tests for Plan agent properties and configuration."""
def test_plan_agent_name(self):
"""Test plan agent name property."""
assert plan_agent.name == "plan"
def test_plan_agent_description_not_empty(self):
"""Test plan agent description is not empty."""
assert plan_agent.description
assert len(plan_agent.description) > 10
def test_plan_agent_creates_agent_lazily(self):
"""Test that PydanticAI agent is created lazily."""
# Create a fresh instance
fresh_agent = PlanAgentImpl()
# _agent should be None before first access
assert fresh_agent._agent is None
# Access the agent property
_ = fresh_agent.agent
# Now _agent should be set
assert fresh_agent._agent is not None
+243
View File
@@ -0,0 +1,243 @@
"""
Tests for retry utilities.
"""
import httpx
import pytest
from src.shared.retry import (
calculate_backoff,
is_retryable_exception,
is_retryable_http_status,
retry_async,
with_retry,
)
class TestIsRetryableHttpStatus:
"""Tests for HTTP status code checking."""
def test_429_is_retryable(self):
"""429 Too Many Requests should be retryable."""
assert is_retryable_http_status(429) is True
def test_500_is_retryable(self):
"""500 Internal Server Error should be retryable."""
assert is_retryable_http_status(500) is True
def test_502_is_retryable(self):
"""502 Bad Gateway should be retryable."""
assert is_retryable_http_status(502) is True
def test_503_is_retryable(self):
"""503 Service Unavailable should be retryable."""
assert is_retryable_http_status(503) is True
def test_504_is_retryable(self):
"""504 Gateway Timeout should be retryable."""
assert is_retryable_http_status(504) is True
def test_400_not_retryable(self):
"""400 Bad Request should not be retryable."""
assert is_retryable_http_status(400) is False
def test_401_not_retryable(self):
"""401 Unauthorized should not be retryable."""
assert is_retryable_http_status(401) is False
def test_404_not_retryable(self):
"""404 Not Found should not be retryable."""
assert is_retryable_http_status(404) is False
def test_200_not_retryable(self):
"""200 OK should not be retryable."""
assert is_retryable_http_status(200) is False
class TestIsRetryableException:
"""Tests for exception checking."""
def test_timeout_exception_is_retryable(self):
"""Timeout exceptions should be retryable."""
exc = httpx.TimeoutException("timeout")
assert is_retryable_exception(exc) is True
def test_connect_error_is_retryable(self):
"""Connection errors should be retryable."""
exc = httpx.ConnectError("connection failed")
assert is_retryable_exception(exc) is True
def test_connection_error_is_retryable(self):
"""Python ConnectionError should be retryable."""
exc = ConnectionError("connection refused")
assert is_retryable_exception(exc) is True
def test_timeout_error_is_retryable(self):
"""Python TimeoutError should be retryable."""
exc = TimeoutError("timed out")
assert is_retryable_exception(exc) is True
def test_value_error_not_retryable(self):
"""ValueError should not be retryable."""
exc = ValueError("invalid value")
assert is_retryable_exception(exc) is False
def test_key_error_not_retryable(self):
"""KeyError should not be retryable."""
exc = KeyError("missing key")
assert is_retryable_exception(exc) is False
class TestCalculateBackoff:
"""Tests for backoff calculation."""
def test_first_attempt_base_delay(self):
"""First attempt should use base delay."""
delay = calculate_backoff(0, base_delay=1.0, jitter=False)
assert delay == 1.0
def test_second_attempt_doubles(self):
"""Second attempt should double the delay."""
delay = calculate_backoff(1, base_delay=1.0, jitter=False)
assert delay == 2.0
def test_third_attempt_quadruples(self):
"""Third attempt should quadruple the delay."""
delay = calculate_backoff(2, base_delay=1.0, jitter=False)
assert delay == 4.0
def test_max_delay_respected(self):
"""Delay should not exceed max_delay."""
delay = calculate_backoff(10, base_delay=1.0, max_delay=30.0, jitter=False)
assert delay == 30.0
def test_jitter_adds_randomness(self):
"""Jitter should add randomness to delay."""
delays = [calculate_backoff(1, base_delay=1.0, jitter=True) for _ in range(10)]
# With jitter, delays should vary (not all identical)
assert len(set(delays)) > 1
def test_jitter_within_bounds(self):
"""Jitter should keep delay within reasonable bounds."""
for _ in range(100):
delay = calculate_backoff(0, base_delay=2.0, jitter=True)
# Attempt 0 with base 2.0 = 2.0, with jitter should be 0.75-1.25x = 1.5-2.5
assert 1.5 <= delay <= 2.5
class TestWithRetryDecorator:
"""Tests for the @with_retry decorator."""
@pytest.mark.anyio
async def test_success_on_first_attempt(self):
"""Function should return on first successful attempt."""
call_count = 0
@with_retry(max_attempts=3)
async def successful_func():
nonlocal call_count
call_count += 1
return "success"
result = await successful_func()
assert result == "success"
assert call_count == 1
@pytest.mark.anyio
async def test_retry_on_timeout(self):
"""Should retry on timeout exception."""
call_count = 0
@with_retry(max_attempts=3, base_delay=0.01)
async def flaky_func():
nonlocal call_count
call_count += 1
if call_count < 3:
raise httpx.TimeoutException("timeout")
return "success"
result = await flaky_func()
assert result == "success"
assert call_count == 3
@pytest.mark.anyio
async def test_no_retry_on_value_error(self):
"""Should not retry on non-retryable exceptions."""
call_count = 0
@with_retry(max_attempts=3)
async def bad_func():
nonlocal call_count
call_count += 1
raise ValueError("bad value")
with pytest.raises(ValueError):
await bad_func()
assert call_count == 1
@pytest.mark.anyio
async def test_exhausted_retries(self):
"""Should raise last exception after all retries exhausted."""
call_count = 0
@with_retry(max_attempts=3, base_delay=0.01)
async def always_fails():
nonlocal call_count
call_count += 1
raise httpx.TimeoutException("always times out")
with pytest.raises(httpx.TimeoutException):
await always_fails()
assert call_count == 3
class TestRetryAsync:
"""Tests for the retry_async function."""
@pytest.mark.anyio
async def test_success_on_first_attempt(self):
"""Function should return on first successful attempt."""
async def successful_func():
return "success"
result = await retry_async(successful_func, max_attempts=3)
assert result == "success"
@pytest.mark.anyio
async def test_retry_on_connect_error(self):
"""Should retry on connection errors."""
call_count = 0
async def flaky_func():
nonlocal call_count
call_count += 1
if call_count < 2:
raise httpx.ConnectError("connection failed")
return "success"
result = await retry_async(flaky_func, max_attempts=3, base_delay=0.01)
assert result == "success"
assert call_count == 2
@pytest.mark.anyio
async def test_passes_args_and_kwargs(self):
"""Should pass arguments to the function."""
async def add(a, b, multiplier=1):
return (a + b) * multiplier
result = await retry_async(add, 2, 3, max_attempts=1, multiplier=2)
assert result == 10
@pytest.mark.anyio
async def test_no_retry_on_key_error(self):
"""Should not retry on non-retryable exceptions."""
call_count = 0
async def bad_func():
nonlocal call_count
call_count += 1
raise KeyError("missing")
with pytest.raises(KeyError):
await retry_async(bad_func, max_attempts=3)
assert call_count == 1
+279
View File
@@ -0,0 +1,279 @@
"""
Security tests for tools and path validation.
"""
import tempfile
from pathlib import Path
import pytest
from src.domains.tools.file.edit import EditFileTool
from src.domains.tools.file.glob import GlobFilesTool
from src.domains.tools.file.read import ReadFileTool
from src.domains.tools.file.write import WriteFileTool
from src.domains.tools.shell.bash_full import BashTool
class TestPathTraversal:
"""Tests for path traversal attack prevention."""
@pytest.fixture
def allowed_dir(self):
"""Create an allowed directory."""
with tempfile.TemporaryDirectory() as tmpdir:
# Create a file in allowed dir
(Path(tmpdir) / "allowed.txt").write_text("allowed content")
yield tmpdir
@pytest.fixture
def forbidden_dir(self):
"""Create a forbidden directory."""
with tempfile.TemporaryDirectory() as tmpdir:
(Path(tmpdir) / "secret.txt").write_text("secret content")
yield tmpdir
@pytest.mark.anyio
async def test_read_path_traversal_dotdot(self, allowed_dir, forbidden_dir):
"""Test that ../../../ path traversal is blocked."""
tool = ReadFileTool(allowed_paths=[allowed_dir])
# Try to escape using ../
traversal_path = f"{allowed_dir}/../../../etc/passwd"
result = await tool.execute(file_path=traversal_path)
assert not result.success
assert "not in allowed" in result.error.lower()
@pytest.mark.anyio
async def test_read_symlink_escape(self, allowed_dir, forbidden_dir):
"""Test that symlinks pointing outside allowed paths are blocked."""
tool = ReadFileTool(allowed_paths=[allowed_dir])
# Create symlink in allowed dir pointing to forbidden
symlink_path = Path(allowed_dir) / "escape_link"
try:
symlink_path.symlink_to(Path(forbidden_dir) / "secret.txt")
result = await tool.execute(file_path=str(symlink_path))
# Should either fail or resolve and block
if result.success:
# If it succeeded, make sure it didn't leak forbidden content
assert "secret content" not in result.data
finally:
symlink_path.unlink(missing_ok=True)
@pytest.mark.anyio
async def test_write_path_traversal(self, allowed_dir):
"""Test that write cannot escape allowed paths."""
tool = WriteFileTool(allowed_paths=[allowed_dir])
traversal_path = f"{allowed_dir}/../../../tmp/evil.txt"
result = await tool.execute(
file_path=traversal_path,
content="malicious content"
)
assert not result.success
assert "not in allowed" in result.error.lower()
@pytest.mark.anyio
async def test_edit_path_traversal(self, allowed_dir):
"""Test that edit cannot escape allowed paths."""
tool = EditFileTool(allowed_paths=[allowed_dir])
traversal_path = f"{allowed_dir}/../../../etc/passwd"
result = await tool.execute(
file_path=traversal_path,
old_string="root",
new_string="hacked"
)
assert not result.success
# Could be "not found" or "not in allowed"
assert not result.success
@pytest.mark.anyio
async def test_glob_path_traversal(self, allowed_dir, forbidden_dir):
"""Test that glob cannot escape allowed paths."""
tool = GlobFilesTool(allowed_paths=[allowed_dir])
# Try to glob outside allowed
result = await tool.execute(
pattern="**/*.txt",
path=f"{allowed_dir}/../../../"
)
# Should only find files in allowed dir
if result.success:
assert forbidden_dir not in str(result.data)
assert "secret.txt" not in str(result.data)
@pytest.mark.anyio
async def test_bash_cd_escape(self, allowed_dir, forbidden_dir):
"""Test that bash cannot cd outside allowed paths."""
tool = BashTool(allowed_paths=[allowed_dir])
result = await tool.execute(
command=f"cd {forbidden_dir} && cat secret.txt",
cwd=allowed_dir
)
# Should fail - forbidden_dir not in allowed_paths
assert not result.success or "secret content" not in str(result.data or "")
class TestCommandInjection:
"""Tests for command injection prevention."""
@pytest.fixture
def temp_dir(self):
"""Create a temporary directory."""
with tempfile.TemporaryDirectory() as tmpdir:
yield Path(tmpdir)
@pytest.mark.anyio
async def test_bash_semicolon_injection(self, temp_dir):
"""Test that semicolon command chaining is blocked."""
tool = BashTool(allowed_paths=[str(temp_dir)])
# Try to inject command with semicolon
result = await tool.execute(
command="ls; cat /etc/passwd",
cwd=str(temp_dir)
)
# Semicolons should be blocked or command should fail
assert not result.success or "/etc/passwd" not in str(result.data or "")
@pytest.mark.anyio
async def test_bash_backtick_injection(self, temp_dir):
"""Test that backtick command substitution in filenames is handled."""
tool = BashTool(allowed_paths=[str(temp_dir)])
# Try command substitution
result = await tool.execute(
command="ls `whoami`",
cwd=str(temp_dir)
)
# Should either fail or execute safely
# (backticks may be interpreted but shouldn't cause harm with allowed commands)
assert result is not None
@pytest.mark.anyio
async def test_bash_dollar_injection(self, temp_dir):
"""Test that $() command substitution is handled."""
tool = BashTool(allowed_paths=[str(temp_dir)])
# Command substitution with echo - echo is allowed
# The subshell may execute cat, which reads /etc/passwd
# This is a known limitation: allowed_paths restricts file args, not subshell reads
# For now, we just verify the command executes without crashing
result = await tool.execute(
command="echo test", # Simple echo to avoid subshell complexity
cwd=str(temp_dir)
)
assert result.success
assert "test" in str(result.data or "")
class TestInputValidation:
"""Tests for input validation."""
@pytest.fixture
def temp_file(self):
"""Create a temporary file."""
with tempfile.NamedTemporaryFile(mode='w', suffix='.txt', delete=False) as f:
f.write("test content")
f.flush()
yield Path(f.name)
Path(f.name).unlink(missing_ok=True)
@pytest.mark.anyio
async def test_read_file_null_byte(self, temp_file):
"""Test that null bytes in file paths are rejected."""
tool = ReadFileTool()
# Null byte injection attempt
result = await tool.execute(file_path=f"{temp_file}\x00.txt")
# Should fail or sanitize the null byte
# Python's Path handles this, but we should verify
assert result is not None
@pytest.mark.anyio
async def test_write_very_long_filename(self):
"""Test handling of extremely long filenames."""
tool = WriteFileTool()
# 255 is typical max filename length on Linux
long_name = "a" * 300 + ".txt"
try:
result = await tool.execute(
file_path=f"/tmp/{long_name}",
content="test"
)
# Should fail gracefully
assert not result.success
except OSError:
# OS-level error is also acceptable - filename too long
pass
@pytest.mark.anyio
async def test_edit_binary_file_detection(self, temp_file):
"""Test that binary files are handled appropriately."""
# Write binary content
temp_file.write_bytes(b"\x00\x01\x02\x03\xff\xfe")
tool = EditFileTool()
result = await tool.execute(
file_path=str(temp_file),
old_string="test",
new_string="replaced"
)
# Should fail - binary file
assert not result.success
class TestResourceLimits:
"""Tests for resource limit enforcement."""
@pytest.fixture
def temp_dir(self):
"""Create a temporary directory."""
with tempfile.TemporaryDirectory() as tmpdir:
yield Path(tmpdir)
@pytest.mark.anyio
async def test_write_content_size_limit(self, temp_dir):
"""Test that content size limits are enforced."""
tool = WriteFileTool(max_content_size=100)
result = await tool.execute(
file_path=str(temp_dir / "large.txt"),
content="x" * 200
)
assert not result.success
assert "large" in result.error.lower() or "size" in result.error.lower()
@pytest.mark.anyio
async def test_glob_result_limit(self, temp_dir):
"""Test that glob result limits are enforced."""
# Create many files
for i in range(20):
(temp_dir / f"file{i}.txt").write_text(f"content {i}")
tool = GlobFilesTool()
result = await tool.execute(
pattern="*.txt",
path=str(temp_dir),
limit=5
)
assert result.success
# Should only return 5 files
lines = [line for line in result.data.strip().split("\n") if line]
assert len(lines) <= 5
+246
View File
@@ -0,0 +1,246 @@
"""
Tests for the Task agent.
Tests registration, API endpoints, tool access, and spawn_agent functionality.
"""
import pytest
from src.domains.agents.base import get_agent, list_agents
from src.domains.agents.task import TaskAgentImpl, task_agent
class TestTaskAgentRegistration:
"""Tests for Task agent registration."""
def test_task_agent_registered(self):
"""Test that task agent is registered in registry."""
agent = get_agent("task")
assert agent is not None
assert agent.name == "task"
def test_task_agent_in_list(self):
"""Test that task agent appears in agent list."""
agents = list_agents()
names = [a["name"] for a in agents]
assert "task" in names
def test_task_agent_has_description(self):
"""Test that task agent has a description."""
agent = get_agent("task")
assert agent is not None
assert len(agent.description) > 0
assert "task" in agent.description.lower() or "autonomous" in agent.description.lower()
def test_task_agent_singleton(self):
"""Test that task_agent is the registered instance."""
registered = get_agent("task")
assert registered is task_agent
def test_task_agent_is_correct_type(self):
"""Test that task agent is correct implementation type."""
assert isinstance(task_agent, TaskAgentImpl)
class TestTaskAgentTools:
"""Tests for Task agent tool access."""
def test_task_agent_has_all_tools(self):
"""Test that task agent has all 9 tools."""
agent = task_agent.agent
tool_names = list(agent._function_toolset.tools.keys())
# Should have 9 tools total
assert len(tool_names) == 9
def test_task_agent_has_read_only_tools(self):
"""Test that task agent has read-only tools."""
agent = task_agent.agent
tool_names = list(agent._function_toolset.tools.keys())
assert "read_file" in tool_names
assert "glob_files" in tool_names
assert "grep_content" in tool_names
assert "bash_readonly" in tool_names
def test_task_agent_has_write_tools(self):
"""Test that task agent has write tools."""
agent = task_agent.agent
tool_names = list(agent._function_toolset.tools.keys())
assert "edit_file" in tool_names
assert "write_file" in tool_names
assert "bash" in tool_names
def test_task_agent_has_external_tools(self):
"""Test that task agent has external tools."""
agent = task_agent.agent
tool_names = list(agent._function_toolset.tools.keys())
assert "web_search" in tool_names
def test_task_agent_has_spawn_agent_tool(self):
"""Test that task agent has spawn_agent orchestration tool."""
agent = task_agent.agent
tool_names = list(agent._function_toolset.tools.keys())
assert "spawn_agent" in tool_names
class TestSpawnAgentTool:
"""Tests for spawn_agent orchestration functionality."""
@pytest.mark.skip(
reason="never finished — the body built a mock context and then asserted "
"nothing, so it counted as a passing test while verifying nothing"
)
@pytest.mark.anyio
async def test_spawn_explore_agent(self):
"""Spawning an explore agent should delegate to the explore agent.
The scaffolding that used to sit here — a MagicMock RunContext, an
AgentContext with a /tmp working dir, and a patch of
src.domains.agents.base.get_agent — ran and then stopped at the comment
"For now, verify the explore agent would be called correctly". There was
no assertion, so it passed unconditionally.
Removed rather than tidied: ruff flagged its imports as unused, and
deleting those would have made the test look clean while leaving it
hollow. git history has the setup for whoever finishes this.
"""
@pytest.mark.anyio
async def test_spawn_unknown_agent_returns_error(self):
"""Test that spawning unknown agent type returns error."""
# We can't easily test the tool directly, but we can verify
# the agent type validation logic
allowed_types = ["explore", "plan"]
assert "nonexistent" not in allowed_types
assert "task" not in allowed_types # Task should be blocked
def test_spawn_task_agent_blocked(self):
"""Test that spawning nested task agents is blocked."""
# Verify the validation logic prevents recursion
# The spawn_agent tool should return an error for agent_type="task"
allowed_types = ["explore", "plan"]
assert "task" not in allowed_types
class TestTaskAgentAPI:
"""Tests for Task agent REST API."""
@pytest.mark.anyio
async def test_list_agents_includes_task(self, auth_client):
"""Test that agent list includes task agent."""
response = await auth_client.get("/agents/")
assert response.status_code == 200
data = response.json()
names = [a["name"] for a in data["agents"]]
assert "task" in names
@pytest.mark.anyio
async def test_get_task_agent_info(self, auth_client):
"""Test getting task agent info."""
response = await auth_client.get("/agents/task")
assert response.status_code == 200
data = response.json()
assert data["name"] == "task"
assert "description" in data
assert len(data["description"]) > 0
@pytest.mark.anyio
async def test_run_task_with_invalid_body(self, auth_client):
"""Test running task agent with invalid request."""
response = await auth_client.post(
"/agents/run",
json={
"agent_type": "task",
# Missing prompt
}
)
assert response.status_code == 422
@pytest.mark.anyio
async def test_stream_task_with_invalid_body(self, auth_client):
"""Test streaming task agent with invalid request."""
response = await auth_client.post(
"/agents/stream",
json={
"agent_type": "task",
# Missing prompt
}
)
assert response.status_code == 422
class TestTaskAgentProperties:
"""Tests for Task agent properties and configuration."""
def test_task_agent_name(self):
"""Test task agent name property."""
assert task_agent.name == "task"
def test_task_agent_description_not_empty(self):
"""Test task agent description is not empty."""
assert task_agent.description
assert len(task_agent.description) > 10
def test_task_agent_creates_agent_lazily(self):
"""Test that PydanticAI agent is created lazily."""
# Create a fresh instance
fresh_agent = TaskAgentImpl()
# _agents dict should be empty before first access
assert len(fresh_agent._agents) == 0
# Access the agent property (creates default mode agent)
_ = fresh_agent.agent
# Now _agents should have one entry
assert len(fresh_agent._agents) == 1
class TestAllAgentsRegistered:
"""Tests to verify all three agents are registered."""
def test_all_agents_in_registry(self):
"""Test that explore, plan, and task agents are all registered."""
agents = list_agents()
names = [a["name"] for a in agents]
assert "explore" in names
assert "plan" in names
assert "task" in names
assert len(names) == 3
def test_agent_hierarchy(self):
"""Test the agent capability hierarchy."""
explore = get_agent("explore")
plan = get_agent("plan")
task = get_agent("task")
explore_tools = list(explore.agent._function_toolset.tools.keys())
plan_tools = list(plan.agent._function_toolset.tools.keys())
task_tools = list(task.agent._function_toolset.tools.keys())
# Explore has all tools (read + write)
assert "edit_file" in explore_tools
assert "write_file" in explore_tools
# Plan has read-only tools
assert "edit_file" not in plan_tools
assert "write_file" not in plan_tools
# Task has all tools plus spawn_agent
assert "edit_file" in task_tools
assert "write_file" in task_tools
assert "spawn_agent" in task_tools
# Only Task has spawn_agent
assert "spawn_agent" not in explore_tools
assert "spawn_agent" not in plan_tools
+89
View File
@@ -0,0 +1,89 @@
"""
Tests for token counting utilities.
"""
from src.shared.tokens import count_message_tokens, count_tokens, estimate_tokens
class TestTokenCounting:
"""Tests for token counting functions."""
def test_estimate_tokens_basic(self):
"""Test basic token estimation."""
text = "Hello world"
tokens = estimate_tokens(text)
# ~4 chars per token
assert tokens == len(text) // 4
def test_estimate_tokens_empty(self):
"""Test estimation with empty string."""
assert estimate_tokens("") == 0
def test_estimate_tokens_long_text(self):
"""Test estimation with longer text."""
text = "a" * 400
tokens = estimate_tokens(text)
assert tokens == 100
def test_count_tokens_basic(self):
"""Test actual token counting."""
text = "Hello, how are you today?"
tokens = count_tokens(text)
# Should return reasonable token count
assert tokens > 0
assert tokens < len(text) # Should be fewer tokens than characters
def test_count_tokens_empty(self):
"""Test counting empty string."""
tokens = count_tokens("")
assert tokens == 0
def test_count_message_tokens_single(self):
"""Test counting tokens in single message."""
messages = [{"role": "user", "content": "Hello"}]
tokens = count_message_tokens(messages)
assert tokens > 0
def test_count_message_tokens_multiple(self):
"""Test counting tokens in multiple messages."""
messages = [
{"role": "user", "content": "Hello, how are you?"},
{"role": "assistant", "content": "I'm doing well, thank you!"},
]
tokens = count_message_tokens(messages)
# Should be more than single message
single_tokens = count_message_tokens([messages[0]])
assert tokens > single_tokens
def test_count_message_tokens_empty_list(self):
"""Test counting empty message list."""
tokens = count_message_tokens([])
# tiktoken returns small overhead for empty list (assistant priming)
assert tokens < 10
class TestTokenCountingAccuracy:
"""Tests for token counting accuracy."""
def test_code_tokens_reasonable(self):
"""Test that code is tokenized reasonably."""
code = """
def hello_world():
print("Hello, World!")
return True
"""
tokens = count_tokens(code)
# Code should have reasonable token count
assert 10 < tokens < 100
def test_special_characters(self):
"""Test tokenization of special characters."""
text = "Hello! @#$%^&*() World?"
tokens = count_tokens(text)
assert tokens > 0
def test_unicode_text(self):
"""Test tokenization of unicode text."""
text = "Hello 世界 🌍"
tokens = count_tokens(text)
assert tokens > 0
+1 -1
View File
@@ -6,8 +6,8 @@ from pathlib import Path
import pytest import pytest
from src.domains.tools.file.read import ReadFileTool
from src.domains.tools.file.glob import GlobFilesTool from src.domains.tools.file.glob import GlobFilesTool
from src.domains.tools.file.read import ReadFileTool
from src.domains.tools.search.grep import GrepContentTool from src.domains.tools.search.grep import GrepContentTool
from src.domains.tools.shell.bash import BashReadOnlyTool from src.domains.tools.shell.bash import BashReadOnlyTool
+218
View File
@@ -0,0 +1,218 @@
"""
Tests for WebSearchTool.
"""
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from src.domains.tools.search.web import WebSearchTool
class TestWebSearchTool:
"""Tests for WebSearchTool."""
@pytest.fixture
def tool(self):
return WebSearchTool(searxng_url="http://searxng:8080", timeout=5)
@pytest.fixture
def mock_search_response(self):
"""Sample SearXNG response."""
return {
"query": "test query",
"number_of_results": 3,
"results": [
{
"title": "First Result",
"url": "https://example.com/1",
"content": "This is the first result content.",
"engine": "google",
"publishedDate": "2024-01-15",
},
{
"title": "Second Result",
"url": "https://example.com/2",
"content": "This is the second result content.",
"engine": "brave",
"publishedDate": None,
},
{
"title": "Third Result",
"url": "https://example.com/3",
"content": "This is the third result content.",
"engine": "duckduckgo",
"publishedDate": "2024-01-10",
},
],
}
@pytest.mark.anyio
async def test_search_success(self, tool, mock_search_response):
"""Test successful search."""
with patch("httpx.AsyncClient") as mock_client:
mock_response = MagicMock()
mock_response.json.return_value = mock_search_response
mock_response.raise_for_status = MagicMock()
mock_instance = AsyncMock()
mock_instance.get.return_value = mock_response
mock_instance.__aenter__.return_value = mock_instance
mock_instance.__aexit__.return_value = None
mock_client.return_value = mock_instance
result = await tool.execute(query="test query", num_results=3)
assert result.success
assert "First Result" in result.data
assert "https://example.com/1" in result.data
assert result.metadata["result_count"] == 3
@pytest.mark.anyio
async def test_search_empty_query(self, tool):
"""Test empty query is rejected."""
result = await tool.execute(query="")
assert not result.success
assert "empty" in result.error.lower()
@pytest.mark.anyio
async def test_search_whitespace_query(self, tool):
"""Test whitespace-only query is rejected."""
result = await tool.execute(query=" ")
assert not result.success
assert "empty" in result.error.lower()
@pytest.mark.anyio
async def test_search_no_results(self, tool):
"""Test when search returns no results."""
with patch("httpx.AsyncClient") as mock_client:
mock_response = MagicMock()
mock_response.json.return_value = {"query": "obscure", "results": []}
mock_response.raise_for_status = MagicMock()
mock_instance = AsyncMock()
mock_instance.get.return_value = mock_response
mock_instance.__aenter__.return_value = mock_instance
mock_instance.__aexit__.return_value = None
mock_client.return_value = mock_instance
result = await tool.execute(query="obscure nonexistent thing")
assert result.success
assert "No results" in result.data
assert result.metadata["result_count"] == 0
@pytest.mark.anyio
async def test_search_num_results_limit(self, tool, mock_search_response):
"""Test num_results limits output."""
with patch("httpx.AsyncClient") as mock_client:
mock_response = MagicMock()
mock_response.json.return_value = mock_search_response
mock_response.raise_for_status = MagicMock()
mock_instance = AsyncMock()
mock_instance.get.return_value = mock_response
mock_instance.__aenter__.return_value = mock_instance
mock_instance.__aexit__.return_value = None
mock_client.return_value = mock_instance
result = await tool.execute(query="test", num_results=2)
assert result.success
assert result.metadata["result_count"] == 2
@pytest.mark.anyio
async def test_search_num_results_clamped(self, tool, mock_search_response):
"""Test num_results is clamped to max."""
with patch("httpx.AsyncClient") as mock_client:
mock_response = MagicMock()
mock_response.json.return_value = mock_search_response
mock_response.raise_for_status = MagicMock()
mock_instance = AsyncMock()
mock_instance.get.return_value = mock_response
mock_instance.__aenter__.return_value = mock_instance
mock_instance.__aexit__.return_value = None
mock_client.return_value = mock_instance
# Request 100 but max is 10
result = await tool.execute(query="test", num_results=100)
assert result.success
# Should only get 3 (what's in mock response, capped at 10)
assert result.metadata["result_count"] <= 10
@pytest.mark.anyio
async def test_search_timeout_error(self, tool):
"""Test timeout handling."""
import httpx
with patch("httpx.AsyncClient") as mock_client:
mock_instance = AsyncMock()
mock_instance.get.side_effect = httpx.TimeoutException("timeout")
mock_instance.__aenter__.return_value = mock_instance
mock_instance.__aexit__.return_value = None
mock_client.return_value = mock_instance
result = await tool.execute(query="test")
assert not result.success
assert "timed out" in result.error.lower()
@pytest.mark.anyio
async def test_search_request_error(self, tool):
"""Test network error handling."""
import httpx
with patch("httpx.AsyncClient") as mock_client:
mock_instance = AsyncMock()
mock_instance.get.side_effect = httpx.RequestError("connection failed")
mock_instance.__aenter__.return_value = mock_instance
mock_instance.__aexit__.return_value = None
mock_client.return_value = mock_instance
result = await tool.execute(query="test")
assert not result.success
assert "failed" in result.error.lower()
@pytest.mark.anyio
async def test_search_includes_engines_metadata(self, tool, mock_search_response):
"""Test engines used are included in metadata."""
with patch("httpx.AsyncClient") as mock_client:
mock_response = MagicMock()
mock_response.json.return_value = mock_search_response
mock_response.raise_for_status = MagicMock()
mock_instance = AsyncMock()
mock_instance.get.return_value = mock_response
mock_instance.__aenter__.return_value = mock_instance
mock_instance.__aexit__.return_value = None
mock_client.return_value = mock_instance
result = await tool.execute(query="test", num_results=3)
assert result.success
engines = result.metadata.get("engines_used", [])
assert "google" in engines or "brave" in engines
@pytest.mark.anyio
async def test_search_with_categories(self, tool, mock_search_response):
"""Test category parameter is passed."""
with patch("httpx.AsyncClient") as mock_client:
mock_response = MagicMock()
mock_response.json.return_value = mock_search_response
mock_response.raise_for_status = MagicMock()
mock_instance = AsyncMock()
mock_instance.get.return_value = mock_response
mock_instance.__aenter__.return_value = mock_instance
mock_instance.__aexit__.return_value = None
mock_client.return_value = mock_instance
result = await tool.execute(query="test", categories="it")
assert result.success
# Verify get was called with categories parameter
call_args = mock_instance.get.call_args
assert "categories" in call_args.kwargs["params"]
assert call_args.kwargs["params"]["categories"] == "it"
+21
View File
@@ -0,0 +1,21 @@
# Changelog - Webber CLI
All notable changes to the Webber CLI will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
## [0.1.0] - 2026-01-10
### Added
- Initial CLI release as part of monorepo reorganization
- Typer + Rich foundation with console theming
- Commands:
- `webber-cli status` - Check API connection
- `webber-cli explore` - One-shot codebase exploration
- `webber-cli chat` - Interactive conversation mode
- Streaming support with `--stream` flag (default: enabled)
- Markdown rendering for agent responses
- Configurable API URL via `WEBBER_API_URL` environment variable
+24 -1
View File
@@ -22,13 +22,36 @@ pip install -e .
# Check API connection # Check API connection
webber-cli status webber-cli status
# Explore a codebase # Explore a codebase (streams by default)
webber-cli explore "find all python files" -d /path/to/project webber-cli explore "find all python files" -d /path/to/project
# Batch mode (wait for full response)
webber-cli explore "find bugs" -d /path/to/project --no-stream
# Interactive chat mode # Interactive chat mode
webber-cli chat -d /path/to/project webber-cli chat -d /path/to/project
# Chat without streaming
webber-cli chat -d /path/to/project --no-stream
``` ```
## Commands
| Command | Description |
|---------|-------------|
| `status` | Check API connection and list available agents |
| `explore QUERY` | One-shot codebase exploration |
| `chat` | Interactive chat session |
## Options
| Option | Short | Description |
|--------|-------|-------------|
| `--directory` | `-d` | Working directory for exploration |
| `--api` | `-a` | API URL (default: `$WEBBER_API_URL` or `http://localhost:8095`) |
| `--stream/--no-stream` | `-s` | Enable/disable streaming (default: enabled) |
| `--agent` | | Agent to use (default: `explore`) |
## Configuration ## Configuration
Set the API URL via environment variable: Set the API URL via environment variable:
+1
View File
@@ -2,3 +2,4 @@
httpx~=0.28.1 httpx~=0.28.1
typer~=0.15.0 typer~=0.15.0
rich~=13.9.0 rich~=13.9.0
prompt_toolkit~=3.0.48
+354 -2
View File
@@ -2,18 +2,72 @@
Webber API client. Webber API client.
Communicates with the Webber API backend for agent execution. Communicates with the Webber API backend for agent execution.
Supports permission modes for controlling agent tool access.
""" """
import json
import httpx import httpx
from dataclasses import dataclass from collections.abc import AsyncIterator
from dataclasses import dataclass, field
from datetime import datetime
from enum import Enum
from typing import Any from typing import Any
class PermissionMode(str, Enum):
"""
Permission modes controlling agent tool access.
- default: All tools available (approval may be required)
- plan: Read-only tools only
- auto_accept: All tools, no approval prompts
"""
default = "default"
plan = "plan"
auto_accept = "auto_accept"
class StreamEventType(str, Enum):
"""Event types for structured agent streaming."""
tool_start = "tool_start"
tool_done = "tool_done"
thinking = "thinking"
response = "response"
error = "error"
done = "done"
chunk = "chunk" # Legacy text chunk
@dataclass
class StreamEvent:
"""
Structured streaming event from agent execution.
Different event types carry different data:
- tool_start: tool, args
- tool_done: tool, result_summary
- thinking: message
- response: text
- error: error_message
- done: mode
- chunk: text (legacy)
"""
event: StreamEventType
tool: str | None = None
args: dict | None = None
result_summary: str | None = None
message: str | None = None
text: str | None = None
error_message: str | None = None
mode: str | None = None
@dataclass @dataclass
class AgentResponse: class AgentResponse:
"""Response from agent execution.""" """Response from agent execution."""
response: str response: str
agent_type: str agent_type: str
success: bool success: bool
mode: PermissionMode = PermissionMode.default
error: str | None = None error: str | None = None
@@ -24,6 +78,47 @@ class AgentInfo:
description: str description: str
@dataclass
class Message:
"""A message in a conversation."""
id: str
role: str
content: str
token_count: int
is_summary: bool
created_at: datetime
@dataclass
class Conversation:
"""A conversation session."""
id: str
agent_type: str
title: str | None
working_dir: str
total_tokens: int
created_at: datetime
updated_at: datetime | None
messages: list[Message] = field(default_factory=list)
@dataclass
class AddMessageResult:
"""Result of adding a message to a conversation."""
user_message: Message
assistant_message: Message
total_tokens: int
summarized: bool
@dataclass
class SaveMessagesResult:
"""Result of saving a message pair without agent execution."""
user_message: Message
assistant_message: Message
total_tokens: int
class WebberClient: class WebberClient:
""" """
Client for the Webber API. Client for the Webber API.
@@ -102,14 +197,16 @@ class WebberClient:
agent_type: str, agent_type: str,
prompt: str, prompt: str,
working_dir: str = ".", working_dir: str = ".",
mode: PermissionMode = PermissionMode.default,
) -> AgentResponse: ) -> AgentResponse:
""" """
Run an agent with the given prompt. Run an agent with the given prompt.
Args: Args:
agent_type: Type of agent (e.g., "explore") agent_type: Type of agent (e.g., "task")
prompt: User prompt/query prompt: User prompt/query
working_dir: Working directory for the agent working_dir: Working directory for the agent
mode: Permission mode controlling tool access
Returns: Returns:
AgentResponse with the result AgentResponse with the result
@@ -121,6 +218,7 @@ class WebberClient:
"agent_type": agent_type, "agent_type": agent_type,
"prompt": prompt, "prompt": prompt,
"working_dir": working_dir, "working_dir": working_dir,
"mode": mode.value,
}, },
) )
response.raise_for_status() response.raise_for_status()
@@ -129,9 +227,263 @@ class WebberClient:
response=data.get("response", ""), response=data.get("response", ""),
agent_type=data.get("agent_type", agent_type), agent_type=data.get("agent_type", agent_type),
success=data.get("success", True), success=data.get("success", True),
mode=PermissionMode(data.get("mode", "default")),
error=data.get("error"), error=data.get("error"),
) )
async def run_agent_stream(
self,
agent_type: str,
prompt: str,
working_dir: str = ".",
mode: PermissionMode = PermissionMode.default,
) -> AsyncIterator[StreamEvent]:
"""
Run an agent with streaming response.
Args:
agent_type: Type of agent (e.g., "task")
prompt: User prompt/query
working_dir: Working directory for the agent
mode: Permission mode controlling tool access
Yields:
StreamEvent objects as they arrive
"""
# Use a fresh client for streaming with longer timeout
async with httpx.AsyncClient(
base_url=self.base_url,
timeout=httpx.Timeout(300.0, connect=10.0),
) as client:
async with client.stream(
"POST",
"/agents/stream",
json={
"agent_type": agent_type,
"prompt": prompt,
"working_dir": working_dir,
"mode": mode.value,
},
) as response:
response.raise_for_status()
async for line in response.aiter_lines():
if line.startswith("data: "):
try:
data = json.loads(line[6:])
event_type = data.get("event")
# Parse event type
try:
evt_type = StreamEventType(event_type)
except ValueError:
continue # Unknown event type
# Build StreamEvent from response data
yield StreamEvent(
event=evt_type,
tool=data.get("tool"),
args=data.get("args"),
result_summary=data.get("result_summary"),
message=data.get("message"),
text=data.get("text") or data.get("data"), # 'data' for legacy chunk
error_message=data.get("error_message") or data.get("data"),
mode=data.get("mode"),
)
# Stop on done or error
if evt_type in (StreamEventType.done, StreamEventType.error):
break
except json.JSONDecodeError:
continue
# === Conversation API ===
def _parse_message(self, data: dict) -> Message:
"""Parse a Message from API response data."""
return Message(
id=data["id"],
role=data["role"],
content=data["content"],
token_count=data["token_count"],
is_summary=data["is_summary"],
created_at=datetime.fromisoformat(data["created_at"].replace("Z", "+00:00")),
)
def _parse_conversation(self, data: dict, with_messages: bool = False) -> Conversation:
"""Parse a Conversation from API response data."""
messages = []
if with_messages and "messages" in data:
messages = [self._parse_message(m) for m in data["messages"]]
updated_at = None
if data.get("updated_at"):
updated_at = datetime.fromisoformat(data["updated_at"].replace("Z", "+00:00"))
return Conversation(
id=data["id"],
agent_type=data["agent_type"],
title=data.get("title"),
working_dir=data["working_dir"],
total_tokens=data["total_tokens"],
created_at=datetime.fromisoformat(data["created_at"].replace("Z", "+00:00")),
updated_at=updated_at,
messages=messages,
)
async def list_conversations(
self,
limit: int = 50,
offset: int = 0,
) -> tuple[list[Conversation], int]:
"""
List user's conversations.
Args:
limit: Maximum number of conversations to return
offset: Offset for pagination
Returns:
Tuple of (conversations, total_count)
"""
client = await self._get_client()
response = await client.get(
"/conversations/",
params={"limit": limit, "offset": offset},
)
response.raise_for_status()
data = response.json()
conversations = [self._parse_conversation(c) for c in data["conversations"]]
return conversations, data["total"]
async def get_conversation(self, conversation_id: str) -> Conversation | None:
"""
Get a conversation with all messages.
Args:
conversation_id: UUID of the conversation
Returns:
Conversation with messages, or None if not found
"""
client = await self._get_client()
response = await client.get(f"/conversations/{conversation_id}")
if response.status_code == 404:
return None
response.raise_for_status()
return self._parse_conversation(response.json(), with_messages=True)
async def create_conversation(
self,
agent_type: str = "task",
working_dir: str = ".",
title: str | None = None,
) -> Conversation:
"""
Create a new conversation.
Args:
agent_type: Type of agent to use
working_dir: Working directory for the agent
title: Optional title for the conversation
Returns:
The created conversation
"""
client = await self._get_client()
response = await client.post(
"/conversations/",
json={
"agent_type": agent_type,
"working_dir": working_dir,
"title": title,
},
)
response.raise_for_status()
return self._parse_conversation(response.json())
async def add_message(
self,
conversation_id: str,
content: str,
) -> AddMessageResult:
"""
Add a message to a conversation and get agent response.
Args:
conversation_id: UUID of the conversation
content: Message content
Returns:
AddMessageResult with user and assistant messages
"""
client = await self._get_client()
response = await client.post(
f"/conversations/{conversation_id}/messages",
json={"content": content},
)
response.raise_for_status()
data = response.json()
return AddMessageResult(
user_message=self._parse_message(data["user_message"]),
assistant_message=self._parse_message(data["assistant_message"]),
total_tokens=data["total_tokens"],
summarized=data.get("summarized", False),
)
async def save_messages(
self,
conversation_id: str,
user_content: str,
assistant_content: str,
) -> SaveMessagesResult:
"""
Save a user/assistant message pair without triggering agent execution.
Used when streaming responses separately via run_agent_stream().
Allows persisting the exchange after streaming completes.
Args:
conversation_id: UUID of the conversation
user_content: User message content
assistant_content: Assistant response content
Returns:
SaveMessagesResult with both messages
"""
client = await self._get_client()
response = await client.post(
f"/conversations/{conversation_id}/save",
json={
"user_content": user_content,
"assistant_content": assistant_content,
},
)
response.raise_for_status()
data = response.json()
return SaveMessagesResult(
user_message=self._parse_message(data["user_message"]),
assistant_message=self._parse_message(data["assistant_message"]),
total_tokens=data["total_tokens"],
)
async def delete_conversation(self, conversation_id: str) -> bool:
"""
Delete a conversation.
Args:
conversation_id: UUID of the conversation
Returns:
True if deleted, False if not found
"""
client = await self._get_client()
response = await client.delete(f"/conversations/{conversation_id}")
if response.status_code == 404:
return False
response.raise_for_status()
return True
async def __aenter__(self) -> "WebberClient": async def __aenter__(self) -> "WebberClient":
"""Async context manager entry.""" """Async context manager entry."""
return self return self
+152
View File
@@ -0,0 +1,152 @@
"""
Configuration management for Webber CLI.
Loads settings from ~/.webber/config.toml with environment variable overrides.
"""
import os
import tomllib
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
# Config directory and file paths
CONFIG_DIR = Path.home() / ".webber"
CONFIG_FILE = CONFIG_DIR / "config.toml"
# Default configuration template
DEFAULT_CONFIG = """\
# Webber CLI Configuration
# https://github.com/jpmschweitzer/webber
[api]
# API server URL (dev: 8095, prod: 8086)
url = "http://localhost:8095"
# API key for authentication (optional for dev mode)
# key = "your-api-key"
[cli]
# Default permission mode: default, plan, auto_accept
mode = "default"
# Enable streaming by default
stream = true
[history]
# Command history file location
file = "~/.webber_history"
"""
@dataclass
class ApiConfig:
"""API connection settings."""
url: str = "http://localhost:8095"
key: str | None = None
@dataclass
class CliConfig:
"""CLI behavior settings."""
mode: str = "default"
stream: bool = True
@dataclass
class HistoryConfig:
"""History settings."""
file: str = "~/.webber_history"
@dataclass
class Config:
"""Complete configuration."""
api: ApiConfig = field(default_factory=ApiConfig)
cli: CliConfig = field(default_factory=CliConfig)
history: HistoryConfig = field(default_factory=HistoryConfig)
def load_config() -> Config:
"""
Load configuration from file with environment variable overrides.
Priority (highest to lowest):
1. Environment variables (WEBBER_API_URL, WEBBER_API_KEY, WEBBER_MODE)
2. Config file (~/.webber/config.toml)
3. Built-in defaults
Returns:
Config object with merged settings
"""
config = Config()
# Load from file if exists
if CONFIG_FILE.exists():
try:
with open(CONFIG_FILE, "rb") as f:
data = tomllib.load(f)
config = _parse_config(data)
except Exception:
# If config file is invalid, use defaults
pass
# Apply environment variable overrides
if url := os.environ.get("WEBBER_API_URL"):
config.api.url = url
if key := os.environ.get("WEBBER_API_KEY"):
config.api.key = key
if mode := os.environ.get("WEBBER_MODE"):
config.cli.mode = mode
return config
def _parse_config(data: dict[str, Any]) -> Config:
"""Parse config dict into Config object."""
config = Config()
if api := data.get("api"):
config.api.url = api.get("url", config.api.url)
config.api.key = api.get("key", config.api.key)
if cli := data.get("cli"):
config.cli.mode = cli.get("mode", config.cli.mode)
config.cli.stream = cli.get("stream", config.cli.stream)
if history := data.get("history"):
config.history.file = history.get("file", config.history.file)
return config
def init_config() -> Path:
"""
Initialize config directory and file with defaults.
Returns:
Path to the created config file
"""
CONFIG_DIR.mkdir(parents=True, exist_ok=True)
if not CONFIG_FILE.exists():
CONFIG_FILE.write_text(DEFAULT_CONFIG)
return CONFIG_FILE
def get_config_path() -> Path | None:
"""Get path to config file if it exists."""
return CONFIG_FILE if CONFIG_FILE.exists() else None
# Module-level cached config
_config: Config | None = None
def get_config() -> Config:
"""Get cached config, loading if necessary."""
global _config
if _config is None:
_config = load_config()
return _config

Some files were not shown because too many files have changed in this diff Show More