refactor: reorganize into monorepo with separate subprojects
Build and Push API / release (push) Successful in 3s
Build and Push API / build (push) Successful in 2m27s

Structure webber into three independent subprojects:
- webber-api/: FastAPI backend server with all agent code
- webber-cli/: Standalone CLI client (renamed from cli/ to webber_cli/)
- webber-sandbox/: Test project for functional testing

Key changes:
- Each subproject has its own .venv (Python 3.12+)
- Added sandbox.sh for managing test project templates
- Created sandbox-templates/ with calculator-cli and empty starter
- Updated CI/CD for prefixed tags (api/v*, cli/v*)
- Added comprehensive AGENTS.md with operational instructions
- Added gitignore filtering to glob and grep tools
- Created pyproject.toml for each subproject

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
2026-01-10 10:37:47 +01:00
co-authored by Claude Opus 4.5
parent f4e8552298
commit 3b58fa4f8b
121 changed files with 2034 additions and 284 deletions
@@ -1,20 +1,31 @@
name: Build and Push
name: Build and Push API
on:
push:
tags:
- 'v*'
- 'api/v*'
env:
IMAGE_NAME: git.schweitz.internal/jpmschweitzer/webber-api
jobs:
release:
runs-on: ubuntu-latest
steps:
- name: Extract version from tag
id: version
run: |
# Extract version from api/v0.3.0 -> v0.3.0
VERSION="${{ github.ref_name }}"
VERSION="${VERSION#api/}"
echo "version=$VERSION" >> $GITHUB_OUTPUT
- name: Create Gitea Release
run: |
curl -sf -X POST \
-H "Authorization: token ${{ secrets.GITHUB_TOKEN }}" \
-H "Content-Type: application/json" \
-d '{"tag_name": "${{ github.ref_name }}", "name": "Release ${{ github.ref_name }}", "body": "Automated release for ${{ github.ref_name }}"}' \
-d '{"tag_name": "${{ github.ref_name }}", "name": "API Release ${{ steps.version.outputs.version }}", "body": "Automated release for webber-api ${{ steps.version.outputs.version }}"}' \
"${{ github.server_url }}/api/v1/repos/${{ github.repository }}/releases"
build:
@@ -23,6 +34,13 @@ jobs:
steps:
- uses: actions/checkout@v4
- name: Extract version from tag
id: version
run: |
VERSION="${{ github.ref_name }}"
VERSION="${VERSION#api/}"
echo "version=$VERSION" >> $GITHUB_OUTPUT
- name: Login to Gitea Registry
uses: docker/login-action@v3
with:
@@ -33,11 +51,11 @@ jobs:
- name: Build and push
uses: docker/build-push-action@v5
with:
context: .
context: ./webber-api
push: true
tags: |
git.schweitz.internal/jpmschweitzer/webber:latest
git.schweitz.internal/jpmschweitzer/webber:${{ github.ref_name }}
${{ env.IMAGE_NAME }}:latest
${{ env.IMAGE_NAME }}:${{ steps.version.outputs.version }}
- name: Trigger Watchtower update
if: success()
+25
View File
@@ -0,0 +1,25 @@
name: Build and Release CLI
on:
push:
tags:
- 'cli/v*'
# TODO: Implement CLI installer build
# This workflow will be implemented when CLI distribution is ready.
# Possible targets:
# - PyPI package
# - Standalone binary (PyInstaller)
# - Platform-specific installers
jobs:
placeholder:
runs-on: ubuntu-latest
steps:
- name: Extract version from tag
id: version
run: |
VERSION="${{ github.ref_name }}"
VERSION="${VERSION#cli/}"
echo "CLI release triggered for version: $VERSION"
echo "TODO: Implement CLI build and distribution"
+11
View File
@@ -64,3 +64,14 @@ Thumbs.db
# Project specific
*.db
*.sqlite3
# Monorepo - subproject venvs (explicit for clarity)
webber-api/.venv/
webber-cli/.venv/
webber-sandbox/.venv/
# Sandbox marker file
webber-sandbox/.current_template
# Ruff cache
.ruff_cache/
+211 -94
View File
@@ -1,128 +1,245 @@
# AGENTS.md
# Webber Monorepo - Agent Instructions
> **Start every session by reading this file.**
> This file outlines the operational protocols, coding standards, and architectural decisions for this FastAPI project.
> This file contains everything you need to work with this codebase efficiently.
## 1. Agent Operational Protocols
## Quick Reference
### 🧠 Work Patterns (Plan-Act-Reflect)
* **Plan:** Before writing code, briefly outline your plan. Identify which files you will touch and what the side effects might be.
* **Act:** Execute the changes in small, atomic steps.
* **Reflect:** After coding, verify your work. Did you break existing tests? Did you add new tests?
### 🛡️ Git Discipline
* **ALWAYS add the relevant tests for the added code** Make sure to keep the test coverage up as we go, and run tests before commiting.
* **NEVER commit to `main` or `master` directly.** Always create a feature branch: `feature/your-feature-name` or `fix/issue-description`.
* **Commit Messages:** Use the [Conventional Commits](https://www.conventionalcommits.org/) format.
* `feat: add user login endpoint`
* `fix: resolve database connection timeout`
* `refactor: split monolith dependency file`
* **Atomic Commits:** Keep commits small. One logical change = one commit.
### 📝 Changelog Maintenance
* **Update `CHANGELOG.md`** with every user-facing change.
* Format: `## [Unreleased] - YYYY-MM-DD` followed by `### Added`, `### Changed`, or `### Fixed`.
### 🚀 Release Flow
When changes are ready for deployment:
1. **Ask user if deploy cycle is desired **
2. **Update version** in `pyproject.toml`:
- Bug fixes: bump patch version (1.8.3 → 1.8.4)
- New features: bump minor version (1.8.4 → 1.9.0)
3. **Update CHANGELOG.md**:
- Move items from `[Unreleased]` to new version section
- Add release date: `## [1.8.4] - 2025-12-16`
4. **Commit and tag**:
```bash
git add -A
git commit -m "fix: description of changes"
git tag v1.8.4
git push origin main --tags
```
5. **CI/CD triggers automatically**:
- Gitea CI builds Docker image on new version tag (starts with "v")
- Watchtower pulls and deploys to production
- Verify deployment: `curl http://192.168.86.149:8086/health`
| Action | Command |
|--------|---------|
| Start API server | `cd webber-api && ./wakeup.sh` |
| View API logs | `tail -f webber-api/logs/server.log` |
| Run API tests | `cd webber-api && .venv/bin/python -m pytest tests/ -v` |
| Check CLI status | `cd webber-cli && .venv/bin/webber-cli status` |
| Load sandbox | `./sandbox.sh load calculator-cli` |
| Explore sandbox | `cd webber-cli && .venv/bin/webber-cli explore "query" -d ../webber-sandbox` |
---
### 🧪 Local Development Setup
## Repository Structure
* **Always test locally first** before committing and deploying. The build-deploy loop is slow.
* **Only deploy** when a phase or feature is complete and tested locally
* **Environment**: Copy `.env.example` to `.env` and configure for your local setup
```
webber/
├── webber-api/ # FastAPI backend server
│ ├── src/ # API source code
│ ├── tests/ # API tests (pytest)
│ ├── docs/ # Architecture docs, COVERAGE.md
│ ├── logs/ # Runtime logs (server.log)
│ ├── .venv/ # API virtual environment
│ ├── wakeup.sh # Dev server startup script
│ └── AGENTS.md # API-specific development guide
├── webber-cli/ # CLI client
│ ├── webber_cli/ # Python package (underscore!)
│ ├── .venv/ # CLI virtual environment
│ └── README.md # CLI usage guide
├── webber-sandbox/ # Active test project (contents swappable)
│ ├── src/ # Current project source
│ ├── tests/ # Current project tests
│ ├── .venv/ # Sandbox virtual environment
│ └── TASKS.md # Tasks for Webber to complete
├── sandbox-templates/ # Template storage
│ ├── calculator-cli/ # Simple CLI with intentional bugs
│ └── empty/ # Blank starter project
├── sandbox.sh # Sandbox management script
└── AGENTS.md # THIS FILE
```
#### ⚠️ CRITICAL: Starting the Local Server
---
**ALWAYS use `./wakeup.sh` to start the local server. NEVER use raw uvicorn commands.**
## Development Workflow
### 1. Start the API Server
```bash
cd webber-api
./wakeup.sh
```
The wakeup script provides:
- **Port conflict detection** - Warns if port 8086 is already in use
- **Virtual environment activation** - Ensures correct Python environment
- **Centralized logging** - All logs written to `logs/server.log` for easy tailing
- **Auto-reload** - Code changes picked up automatically (except requirements.txt changes)
- **Consistent configuration** - Same startup every time
- **Port:** 8095 (dev), 8086 (production Docker)
- **Logs:** `webber-api/logs/server.log`
- **Health check:** `curl http://localhost:8095/health`
- **API docs:** http://localhost:8095/docs
To stop: `Ctrl+C` or `pkill -f "uvicorn src.main:app"`
### 2. Run Tests
To monitor logs in another terminal:
```bash
tail -f logs/server.log
# API tests (39 tests)
cd webber-api
.venv/bin/python -m pytest tests/ -v
# With coverage
.venv/bin/python -m pytest tests/ --cov=src
# Single test file
.venv/bin/python -m pytest tests/test_tools.py -v
```
To stop the server: Press `Ctrl+C`
### 3. Use the CLI
To kill a stuck server:
```bash
cd webber-cli
# Check API connection
.venv/bin/webber-cli status
# Explore a directory
.venv/bin/webber-cli explore "find all python files" -d ../webber-sandbox
# Interactive chat mode
.venv/bin/webber-cli chat -d ../webber-sandbox
```
**Note:** The API server must be running for CLI commands to work.
---
## Sandbox Management
The sandbox is a swappable test project for functional testing.
### Available Templates
| Template | Description |
|----------|-------------|
| `calculator-cli` | Python CLI with intentional bugs (div-by-zero, missing tests) |
| `empty` | Blank starter project |
### Commands
```bash
# List available templates
./sandbox.sh list
# Load a template (clears sandbox, preserves .venv)
./sandbox.sh load calculator-cli
# Reset to last loaded template
./sandbox.sh reset
# Save current sandbox as new template
./sandbox.sh save my-template
# Check current status
./sandbox.sh status
```
### After Loading a Template
```bash
cd webber-sandbox
source .venv/bin/activate # Create .venv first if missing
pip install -r requirements.txt
# Read the tasks
cat TASKS.md
# Run the project's tests
pytest tests/ -v
```
---
## Testing Webber's Capabilities
### Scenario: Find bugs in calculator-cli
```bash
# 1. Load the template
./sandbox.sh load calculator-cli
# 2. Have Webber explore it
cd webber-cli
.venv/bin/webber-cli explore "find all bugs in the code" -d ../webber-sandbox
# 3. Check TASKS.md for expected bugs
cat ../webber-sandbox/TASKS.md
```
### Known bugs in calculator-cli:
- Division by zero not handled (`operations.py:divide`)
- Invalid operation causes KeyError (`main.py:get_operation`)
- Power function broken for fractional exponents
- Missing tests for divide and power functions
---
## Key Files for Debugging
| File | Purpose |
|------|---------|
| `webber-api/logs/server.log` | API server logs |
| `webber-api/src/domains/agents/explore/prompts.py` | Explore agent system prompts |
| `webber-api/src/domains/agents/explore/agent.py` | Explore agent implementation |
| `webber-api/src/ollama/provider.py` | Ollama integration (sanitizes content:null) |
| `webber-api/docs/COVERAGE.md` | Feature coverage and known issues |
---
## Versioning & Releases
Uses prefixed tags:
- `api/v0.3.0` → Triggers API Docker build
- `cli/v0.1.0` → Triggers CLI build (future)
```bash
# API release
cd webber-api
# Update version in pyproject.toml
git add -A && git commit -m "chore: release api v0.3.0"
git tag api/v0.3.0
git push origin main --tags
```
---
## Troubleshooting
### API server won't start
```bash
# Check if port is in use
lsof -i :8095
# Kill stuck process
pkill -f "uvicorn src.main:app"
# or
kill $(lsof -t -i:8086)
```
#### Testing
**Test REST endpoints** against `http://localhost:8086`:
### CLI can't connect
```bash
curl http://localhost:8086/health
curl http://localhost:8086/
curl http://localhost:8086/docs # Swagger UI
# Check API is running
curl http://localhost:8095/health
# Check CLI config
echo $WEBBER_API_URL # Should be http://localhost:8095
```
**Running tests**: Always use the venv explicitly to avoid environment mismatches:
### Ollama errors
```bash
.venv/bin/python -m pytest tests/ # All tests
.venv/bin/python -m pytest tests/ -v # Verbose output
.venv/bin/python -m pytest tests/ --cov # With coverage
# Check Ollama is running
curl http://192.168.86.149:11434/api/tags
# Check model is available
curl http://192.168.86.149:11434/api/tags | grep mistral-nemo
```
### Tests failing
```bash
# Run with verbose output
cd webber-api
.venv/bin/python -m pytest tests/ -v --tb=short
```
---
## 1.5 Known Issues & Future Improvements
## Known Limitations
### Explore Agent
1. **Model hallucination** - Mistral Nemo sometimes makes up file contents instead of using tool results
2. **No conversation memory** - CLI chat mode doesn't persist between sessions
3. **No streaming** - Responses appear all at once
- **Gitignore Support**: The filesystem tools (`glob_files`, `grep_content`) currently do NOT honor `.gitignore`. They return results from ignored directories like `.venv/`, `node_modules/`, etc. This should be fixed to filter out gitignored files by default.
- **Model Hallucination**: Mistral Nemo sometimes hallucinates file contents instead of using actual tool results. Consider using a more capable model (codestral, qwen2.5-coder) or adding response validation.
- **Ollama Provider**: We use a custom `WebberOllamaProvider` (ported from tatlock) that sanitizes `content: null` to `content: ""` for assistant messages with tool calls. This works around an Ollama API limitation.
---
## 2. FastAPI Architecture & Best Practices
*Reference: [FastAPI Best Practices](https://github.com/zhanymkanov/fastapi-best-practices)*
### 📂 Project Structure (Directory-based, NOT File-type based)
Do **not** group files by type (e.g., one huge `routers` folder). Group by **domain/module** inside a `src/` directory.
**Correct Structure:**
```text
to be determined
See `webber-api/docs/COVERAGE.md` for full feature coverage status.
+72 -156
View File
@@ -1,171 +1,87 @@
# Webber
# Webber - Multi-Agent AI Development System
Multi-Agent AI Development System - a FastAPI-based service that orchestrates local LLM agents for code exploration, planning, and task execution.
A Claude Code-inspired development assistant powered by local LLMs via Ollama.
## Overview
## Structure
Webber provides autonomous AI agents similar to Claude Code but running locally with configurable models via Ollama. It's designed for:
This is a monorepo containing three subprojects:
- **Explore Agent** - Fast codebase navigation and code search
- **Plan Agent** - Implementation design and step-by-step planning
- **Task Agent** - Autonomous multi-step code generation and modification
| Directory | Description |
|-----------|-------------|
| `webber-api/` | FastAPI backend server with agent orchestration |
| `webber-cli/` | Command-line client for interacting with the API |
| `webber-sandbox/` | Test project for functional testing |
Built on [PydanticAI](https://ai.pydantic.dev/) for structured LLM interactions.
### Additional Directories
| Directory | Description |
|-----------|-------------|
| `sandbox-templates/` | Reusable project templates for the sandbox |
| `.gitea/workflows/` | CI/CD workflows for releases |
## Quick Start
### Prerequisites
### 1. Start the API Server
```bash
cd webber-api
python3.12 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt -r requirements-dev.txt
./wakeup.sh
```
### 2. Set Up the CLI
```bash
cd webber-cli
python3.12 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
pip install -e .
# Test connection
webber-cli status
```
### 3. Load a Sandbox Project
```bash
# From repo root
./sandbox.sh list
./sandbox.sh load calculator-cli
cd webber-sandbox
python3.12 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```
### 4. Explore with Webber
```bash
cd webber-cli
webber-cli explore "find all bugs in the code" -d ../webber-sandbox
```
## Versioning
This project uses prefixed tags for independent release cycles:
- `api/v0.3.0` - Triggers API Docker build and deployment
- `cli/v0.1.0` - Triggers CLI installer build (future)
## Requirements
- Python 3.12+
- [Ollama](https://ollama.ai/) with models installed
- (Optional) Tatlock for multi-tenant authentication
- Ollama running with `mistral-nemo:latest` model
- Docker (for production deployment)
### Installation
## Documentation
```bash
# Clone the repository
git clone https://git.schweitz.internal/jpmschweitzer/webber.git
cd webber
# Create virtual environment
python -m venv .venv
source .venv/bin/activate
# Install dependencies
pip install -r requirements.txt
# For development (includes testing and linting tools)
pip install -r requirements-dev.txt
```
### Configuration
```bash
# Copy example config
cp .env.example .env
# Edit .env with your settings
# At minimum, configure OLLAMA_URL to point to your Ollama instance
```
### Running
```bash
# Development (with auto-reload)
./wakeup.sh
# Or manually
uvicorn src.main:app --host 0.0.0.0 --port 8086 --reload
```
The service will be available at `http://localhost:8086`. API docs at `/docs`.
## Configuration
All settings via environment variables or `.env` file:
| Variable | Default | Description |
|----------|---------|-------------|
| `DEBUG` | `false` | Enable debug mode |
| `LOG_LEVEL` | `INFO` | Logging level |
| `PORT` | `8086` | Server port |
| `OLLAMA_URL` | `http://192.168.86.149:11434` | Ollama API URL |
| `OLLAMA_AGENT_MODEL` | `mistral-nemo-large:latest` | Model for agent reasoning |
| `OLLAMA_EMBED_MODEL` | `nomic-embed-text:latest` | Model for embeddings |
| `TOOL_TIMEOUT_SECONDS` | `120` | Tool execution timeout |
| `SANDBOX_ENABLED` | `true` | Sandbox tool execution |
| `ALLOWED_PATHS` | `[]` | Paths accessible to tools |
See [.env.example](.env.example) for full configuration options.
## Development
### Code Quality
```bash
# Type checking
mypy src/
# Linting
ruff check src/ tests/
# Auto-fix lint issues
ruff check src/ tests/ --fix
# Format code
ruff format src/ tests/
```
### Testing
```bash
# Run all tests
pytest tests/ -v
# With coverage
pytest tests/ --cov=src --cov-report=html
```
### Security Audit
```bash
# Check dependencies for CVEs
pip-audit
```
## Architecture
Webber uses a domain-based architecture with clean separation of concerns:
```
src/
├── main.py # FastAPI app entry point
├── shared/ # Cross-cutting infrastructure
│ ├── base.py # BaseController, BaseSchema
│ ├── config.py # Settings from pyproject.toml + env
│ ├── logging.py # @logged decorator with timing
│ └── exceptions.py # Exception hierarchy
└── domains/ # Feature domains
├── health/ # Health check endpoints
├── agents/ # Agent orchestration
└── tools/ # Tool execution (file, shell, search)
```
See [docs/architecture.md](docs/architecture.md) for detailed patterns and conventions.
## API Endpoints
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/` | GET | Service information |
| `/health` | GET | Health check for monitoring |
| `/docs` | GET | Interactive API documentation |
## Docker
```bash
# Build
docker build -t webber .
# Run
docker run -p 8086:8086 --env-file .env webber
```
The container includes a healthcheck that pings `/health` every 30 seconds.
## Deployment
Deployed via Gitea Actions CI/CD:
1. Tag a release (`git tag v0.x.x && git push --tags`)
2. Workflow builds and pushes Docker image
3. Watchtower auto-deploys to production
Production runs in Portainer `agents` stack on the `docker-dataplane` network.
## Status
**Alpha** - Core infrastructure is complete. Agent and tool implementations are in progress.
- `webber-api/AGENTS.md` - API development guidelines
- `webber-api/docs/` - Architecture and coverage docs
- `webber-cli/README.md` - CLI usage guide
## License
-14
View File
@@ -1,14 +0,0 @@
#!/bin/bash
# Start webber chat session
# Usage: ./chat.sh [directory]
DIR="${1:-.}"
cd /mnt/media/Projects/webber
source .venv/bin/activate
echo "Starting Webber chat..."
echo "Working directory: $(realpath "$DIR")"
echo ""
webber chat -d "$DIR"
@@ -0,0 +1,25 @@
# Python
__pycache__/
*.py[cod]
*.so
.Python
build/
dist/
*.egg-info/
# Virtual environments
.venv/
venv/
# Testing
.pytest_cache/
.coverage
htmlcov/
# IDE
.idea/
.vscode/
*.swp
# Project
.current_template
+64
View File
@@ -0,0 +1,64 @@
# Calculator CLI - Tasks for Webber
A simple calculator with intentional bugs and missing features for testing Webber's capabilities.
## Bugs to Fix
### High Priority
- [ ] **Division by zero** - `operations.py:divide()` crashes when dividing by zero instead of returning an error
- [ ] **Invalid operation name** - `main.py:get_operation()` raises KeyError for unknown operations instead of helpful error message
### Medium Priority
- [ ] **Power function broken** - `operations.py:power()` doesn't handle negative exponents or fractional exponents correctly
- [ ] **No input validation** - `main.py` doesn't validate that command-line arguments are valid numbers
## Missing Tests
- [ ] Add `TestDivide` class with tests for:
- Normal division
- Division by zero (should test error handling once bug is fixed)
- Division with negative numbers
- [ ] Add `TestPower` class with tests for:
- Positive integer exponents
- Zero exponent (should return 1)
- Negative exponents
- [ ] Complete existing test classes:
- `test_add_zero`
- `test_add_floats`
- `test_subtract_negative`
- `test_multiply_by_zero`
## Features to Add
- [ ] **Expose power operation** - Add 'pow' to the operations dictionary in `main.py`
- [ ] **Add modulo operation** - Implement `modulo(a, b)` in operations.py
- [ ] **Add --verbose flag** - Show step-by-step calculation
- [ ] **Add history command** - Track and display recent calculations
- [ ] **Add REPL mode** - Interactive calculator loop
## Code Quality
- [ ] Add type hints to all functions
- [ ] Add docstrings following Google style
- [ ] Fix any linting errors (run `ruff check src/`)
## How to Run
```bash
# Setup
python3.12 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
# Run calculator
python -m calculator.main 10 5 add
python -m calculator.main 10 5 div
# Run tests
pytest tests/ -v
# See failing tests (division by zero)
python -m calculator.main 10 0 div
```
@@ -0,0 +1,17 @@
[project]
name = "calculator"
version = "0.1.0"
description = "A simple calculator CLI with some bugs"
requires-python = ">=3.12"
[build-system]
requires = ["setuptools>=75.0"]
build-backend = "setuptools.build_meta"
[tool.setuptools.packages.find]
where = ["src"]
[tool.pytest.ini_options]
testpaths = ["tests"]
pythonpath = ["src"]
addopts = "-v"
@@ -0,0 +1,2 @@
# Calculator CLI dependencies
pytest>=8.0.0
@@ -0,0 +1,3 @@
"""Calculator CLI - A simple calculator with some bugs for testing."""
__version__ = "0.1.0"
@@ -0,0 +1,55 @@
#!/usr/bin/env python3
"""
Calculator CLI - A simple command-line calculator.
NOTE: This file contains intentional bugs for testing purposes.
Usage:
python -m calculator.main 10 5 add
python -m calculator.main 10 5 sub
python -m calculator.main 10 5 mul
python -m calculator.main 10 5 div
"""
import sys
from calculator.operations import add, subtract, multiply, divide
def get_operation(op_name: str):
"""
Get the operation function by name.
BUG: No validation - invalid operation names cause KeyError!
"""
operations = {
"add": add,
"sub": subtract,
"mul": multiply,
"div": divide,
# BUG: 'power' is implemented in operations.py but not exposed here
}
# BUG: Should handle KeyError gracefully
return operations[op_name]
def main():
"""Main entry point."""
if len(sys.argv) != 4:
print("Usage: python -m calculator.main <a> <b> <operation>")
print("Operations: add, sub, mul, div")
sys.exit(1)
# BUG: No validation that a and b are valid numbers
a = float(sys.argv[1])
b = float(sys.argv[2])
op_name = sys.argv[3]
# BUG: This will crash with KeyError for invalid operation
operation = get_operation(op_name)
result = operation(a, b)
print(f"Result: {result}")
if __name__ == "__main__":
main()
@@ -0,0 +1,43 @@
"""
Math operations for the calculator.
NOTE: This file contains intentional bugs for testing purposes.
"""
def add(a: float, b: float) -> float:
"""Add two numbers."""
return a + b
def subtract(a: float, b: float) -> float:
"""Subtract b from a."""
return a - b
def multiply(a: float, b: float) -> float:
"""Multiply two numbers."""
return a * b
def divide(a: float, b: float) -> float:
"""
Divide a by b.
BUG: Does not handle division by zero!
"""
# BUG: No check for b == 0
return a / b
def power(a: float, b: float) -> float:
"""
Raise a to the power of b.
BUG: Negative exponents not handled correctly for some cases.
"""
# BUG: This naive implementation has issues with negative bases and fractional exponents
result = 1
for _ in range(int(b)):
result *= a
return result
@@ -0,0 +1 @@
"""Calculator tests."""
@@ -0,0 +1,46 @@
"""
Tests for calculator operations.
NOTE: Test coverage is intentionally incomplete for testing purposes.
"""
import pytest
from calculator.operations import add, subtract, multiply
class TestAdd:
"""Tests for add operation."""
def test_add_positive_numbers(self):
assert add(2, 3) == 5
def test_add_negative_numbers(self):
assert add(-2, -3) == -5
# MISSING: test_add_zero, test_add_floats
class TestSubtract:
"""Tests for subtract operation."""
def test_subtract_positive(self):
assert subtract(5, 3) == 2
# MISSING: test_subtract_negative, test_subtract_resulting_negative
class TestMultiply:
"""Tests for multiply operation."""
def test_multiply_positive(self):
assert multiply(3, 4) == 12
# MISSING: test_multiply_by_zero, test_multiply_negative
# MISSING: TestDivide class entirely!
# - test_divide_positive
# - test_divide_by_zero (should test error handling)
# - test_divide_negative
# MISSING: TestPower class entirely!
+25
View File
@@ -0,0 +1,25 @@
# Python
__pycache__/
*.py[cod]
*.so
.Python
build/
dist/
*.egg-info/
# Virtual environments
.venv/
venv/
# Testing
.pytest_cache/
.coverage
htmlcov/
# IDE
.idea/
.vscode/
*.swp
# Project
.current_template
+18
View File
@@ -0,0 +1,18 @@
# My Project - Tasks
A blank starter template. Define your own tasks here.
## Setup
```bash
python3.12 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```
## Tasks
- [ ] Define your project goals
- [ ] Add source files to `src/myproject/`
- [ ] Add tests to `tests/`
- [ ] Update `requirements.txt` with dependencies
+17
View File
@@ -0,0 +1,17 @@
[project]
name = "myproject"
version = "0.1.0"
description = "A blank starter project"
requires-python = ">=3.12"
[build-system]
requires = ["setuptools>=75.0"]
build-backend = "setuptools.build_meta"
[tool.setuptools.packages.find]
where = ["src"]
[tool.pytest.ini_options]
testpaths = ["tests"]
pythonpath = ["src"]
addopts = "-v"
+2
View File
@@ -0,0 +1,2 @@
# Add your dependencies here
pytest>=8.0.0
@@ -0,0 +1,3 @@
"""My Project - A blank starter template."""
__version__ = "0.1.0"
@@ -0,0 +1 @@
"""Tests for myproject."""
Executable
+193
View File
@@ -0,0 +1,193 @@
#!/bin/bash
# Sandbox management script for Webber testing
#
# Usage:
# ./sandbox.sh list - List available templates
# ./sandbox.sh load <template> - Load a template into sandbox
# ./sandbox.sh reset - Reset sandbox to last loaded template
# ./sandbox.sh save <name> - Save current sandbox as new template
# ./sandbox.sh status - Show current sandbox status
set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
SANDBOX_DIR="$SCRIPT_DIR/webber-sandbox"
TEMPLATES_DIR="$SCRIPT_DIR/sandbox-templates"
MARKER_FILE="$SANDBOX_DIR/.current_template"
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
usage() {
echo "Webber Sandbox Manager"
echo ""
echo "Usage: ./sandbox.sh <command> [template]"
echo ""
echo "Commands:"
echo " list List available templates"
echo " load <template> Load a template into sandbox (preserves .venv)"
echo " reset Reset sandbox to last loaded template"
echo " save <name> Save current sandbox as new template"
echo " status Show current sandbox status"
echo ""
echo "Available templates:"
ls -1 "$TEMPLATES_DIR" 2>/dev/null || echo " (none)"
}
list_templates() {
echo "Available templates:"
echo ""
for dir in "$TEMPLATES_DIR"/*/; do
if [ -d "$dir" ]; then
name=$(basename "$dir")
desc=""
if [ -f "$dir/TASKS.md" ]; then
desc=$(head -1 "$dir/TASKS.md" | sed 's/^#\s*//')
fi
printf " %-20s %s\n" "$name" "$desc"
fi
done
}
load_template() {
local template="$1"
if [ -z "$template" ]; then
echo -e "${RED}Error: Template name required${NC}"
echo "Usage: ./sandbox.sh load <template>"
exit 1
fi
if [ ! -d "$TEMPLATES_DIR/$template" ]; then
echo -e "${RED}Error: Template '$template' not found${NC}"
echo "Available templates:"
ls -1 "$TEMPLATES_DIR"
exit 1
fi
echo -e "${YELLOW}Loading template: $template${NC}"
# Create sandbox dir if needed
mkdir -p "$SANDBOX_DIR"
# Clear sandbox contents (except .venv and .git)
find "$SANDBOX_DIR" -mindepth 1 -maxdepth 1 ! -name '.venv' ! -name '.git' -exec rm -rf {} +
# Copy template contents (including hidden files)
cp -r "$TEMPLATES_DIR/$template/." "$SANDBOX_DIR/"
# Mark which template was loaded
echo "$template" > "$MARKER_FILE"
echo -e "${GREEN}Loaded template: $template${NC}"
echo ""
echo "Next steps:"
echo " cd webber-sandbox"
if [ ! -d "$SANDBOX_DIR/.venv" ]; then
echo " python3.12 -m venv .venv"
fi
echo " source .venv/bin/activate"
echo " pip install -r requirements.txt"
echo ""
if [ -f "$SANDBOX_DIR/TASKS.md" ]; then
echo "Tasks available in TASKS.md"
fi
}
reset_template() {
if [ ! -f "$MARKER_FILE" ]; then
echo -e "${RED}Error: No template loaded yet${NC}"
echo "Use './sandbox.sh load <template>' first"
exit 1
fi
local template=$(cat "$MARKER_FILE")
echo "Resetting to template: $template"
load_template "$template"
}
save_template() {
local name="$1"
if [ -z "$name" ]; then
echo -e "${RED}Error: Template name required${NC}"
echo "Usage: ./sandbox.sh save <name>"
exit 1
fi
if [ -d "$TEMPLATES_DIR/$name" ]; then
echo -e "${YELLOW}Warning: Template '$name' already exists${NC}"
read -p "Overwrite? [y/N] " -n 1 -r
echo
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
echo "Cancelled"
exit 0
fi
rm -rf "$TEMPLATES_DIR/$name"
fi
mkdir -p "$TEMPLATES_DIR/$name"
# Copy sandbox contents (except .venv, .git, __pycache__)
rsync -a --exclude='.venv' --exclude='.git' --exclude='__pycache__' \
--exclude='*.pyc' --exclude='.pytest_cache' --exclude='.mypy_cache' \
"$SANDBOX_DIR/" "$TEMPLATES_DIR/$name/"
echo -e "${GREEN}Saved template: $name${NC}"
}
show_status() {
echo "Sandbox Status"
echo "=============="
echo ""
echo "Sandbox directory: $SANDBOX_DIR"
if [ -f "$MARKER_FILE" ]; then
echo "Current template: $(cat "$MARKER_FILE")"
else
echo "Current template: (none loaded)"
fi
if [ -d "$SANDBOX_DIR/.venv" ]; then
echo "Virtual env: exists"
else
echo "Virtual env: not created"
fi
echo ""
echo "Contents:"
if [ -d "$SANDBOX_DIR" ]; then
ls -la "$SANDBOX_DIR" 2>/dev/null | tail -n +4
else
echo " (sandbox not initialized)"
fi
}
# Main command dispatch
case "${1:-}" in
list)
list_templates
;;
load)
load_template "$2"
;;
reset)
reset_template
;;
save)
save_template "$2"
;;
status)
show_status
;;
-h|--help|"")
usage
;;
*)
echo -e "${RED}Unknown command: $1${NC}"
usage
exit 1
;;
esac
+128
View File
@@ -0,0 +1,128 @@
# AGENTS.md
> **Start every session by reading this file.**
> This file outlines the operational protocols, coding standards, and architectural decisions for this FastAPI project.
## 1. Agent Operational Protocols
### 🧠 Work Patterns (Plan-Act-Reflect)
* **Plan:** Before writing code, briefly outline your plan. Identify which files you will touch and what the side effects might be.
* **Act:** Execute the changes in small, atomic steps.
* **Reflect:** After coding, verify your work. Did you break existing tests? Did you add new tests?
### 🛡️ Git Discipline
* **ALWAYS add the relevant tests for the added code** Make sure to keep the test coverage up as we go, and run tests before commiting.
* **NEVER commit to `main` or `master` directly.** Always create a feature branch: `feature/your-feature-name` or `fix/issue-description`.
* **Commit Messages:** Use the [Conventional Commits](https://www.conventionalcommits.org/) format.
* `feat: add user login endpoint`
* `fix: resolve database connection timeout`
* `refactor: split monolith dependency file`
* **Atomic Commits:** Keep commits small. One logical change = one commit.
### 📝 Changelog Maintenance
* **Update `CHANGELOG.md`** with every user-facing change.
* Format: `## [Unreleased] - YYYY-MM-DD` followed by `### Added`, `### Changed`, or `### Fixed`.
### 🚀 Release Flow
When changes are ready for deployment:
1. **Ask user if deploy cycle is desired **
2. **Update version** in `pyproject.toml`:
- Bug fixes: bump patch version (1.8.3 → 1.8.4)
- New features: bump minor version (1.8.4 → 1.9.0)
3. **Update CHANGELOG.md**:
- Move items from `[Unreleased]` to new version section
- Add release date: `## [1.8.4] - 2025-12-16`
4. **Commit and tag**:
```bash
git add -A
git commit -m "fix: description of changes"
git tag v1.8.4
git push origin main --tags
```
5. **CI/CD triggers automatically**:
- Gitea CI builds Docker image on new version tag (starts with "v")
- Watchtower pulls and deploys to production
- Verify deployment: `curl http://192.168.86.149:8086/health`
---
### 🧪 Local Development Setup
* **Always test locally first** before committing and deploying. The build-deploy loop is slow.
* **Only deploy** when a phase or feature is complete and tested locally
* **Environment**: Copy `.env.example` to `.env` and configure for your local setup
#### ⚠️ CRITICAL: Starting the Local Server
**ALWAYS use `./wakeup.sh` to start the local server. NEVER use raw uvicorn commands.**
```bash
./wakeup.sh
```
The wakeup script provides:
- **Port conflict detection** - Warns if port 8086 is already in use
- **Virtual environment activation** - Ensures correct Python environment
- **Centralized logging** - All logs written to `logs/server.log` for easy tailing
- **Auto-reload** - Code changes picked up automatically (except requirements.txt changes)
- **Consistent configuration** - Same startup every time
To monitor logs in another terminal:
```bash
tail -f logs/server.log
```
To stop the server: Press `Ctrl+C`
To kill a stuck server:
```bash
pkill -f "uvicorn src.main:app"
# or
kill $(lsof -t -i:8086)
```
#### Testing
**Test REST endpoints** against `http://localhost:8086`:
```bash
curl http://localhost:8086/health
curl http://localhost:8086/
curl http://localhost:8086/docs # Swagger UI
```
**Running tests**: Always use the venv explicitly to avoid environment mismatches:
```bash
.venv/bin/python -m pytest tests/ # All tests
.venv/bin/python -m pytest tests/ -v # Verbose output
.venv/bin/python -m pytest tests/ --cov # With coverage
```
---
## 1.5 Known Issues & Future Improvements
### Explore Agent
- **Model Hallucination**: Mistral Nemo sometimes hallucinates file contents instead of using actual tool results. Consider using a more capable model (codestral, qwen2.5-coder) or adding response validation.
- **Ollama Provider**: We use a custom `WebberOllamaProvider` (ported from tatlock) that sanitizes `content: null` to `content: ""` for assistant messages with tool calls. This works around an Ollama API limitation.
- **Gitignore Support**: ✅ Fixed - The filesystem tools now honor `.gitignore` patterns and default ignores (`.venv/`, `__pycache__/`, `node_modules/`, etc.).
---
## 2. FastAPI Architecture & Best Practices
*Reference: [FastAPI Best Practices](https://github.com/zhanymkanov/fastapi-best-practices)*
### 📂 Project Structure (Directory-based, NOT File-type based)
Do **not** group files by type (e.g., one huge `routers` folder). Group by **domain/module** inside a `src/` directory.
**Correct Structure:**
```text
to be determined
+185
View File
@@ -0,0 +1,185 @@
# Webber Feature Coverage
> Tracking progress towards Claude Code-like functionality
## Current Status: ~40% Complete
Last updated: 2026-01-10
---
## Phase 1-6: Foundation (Original Plan)
### Phase 1: Tool Infrastructure ✅ Complete
| Component | Status | Notes |
|-----------|--------|-------|
| `BaseTool` abstract class | ✅ | `src/domains/tools/base.py` |
| `ToolResult` dataclass | ✅ | Consistent success/error/truncated handling |
| `ReadFileTool` | ✅ | With line numbers, offset/limit support |
| `GlobFilesTool` | ✅ | Pattern matching, sorted by mtime |
| `GrepContentTool` | ✅ | Regex search with context lines |
| `BashReadOnlyTool` | ✅ | Allowlist-based command filtering |
| Path validation | ✅ | `allowed_paths` restriction |
**Status:** Tools now honor `.gitignore` patterns and default ignores (`.venv/`, `__pycache__/`, etc.)
### Phase 2: Explore Agent ✅ Complete
| Component | Status | Notes |
|-----------|--------|-------|
| `BaseAgent` abstract class | ✅ | `src/domains/agents/base.py` |
| Agent registry | ✅ | `register_agent()`, `get_agent()`, `list_agents()` |
| `ExploreAgentImpl` | ✅ | PydanticAI-based implementation |
| System prompts | ✅ | Mistral-optimized with tool examples |
| Tool registration | ✅ | `@agent.tool` decorator pattern |
| Sanitized Ollama provider | ✅ | Fixes `content: null` issue |
**Gap:** Mistral Nemo sometimes hallucinates instead of using tool results.
### Phase 3: CLI Foundation ✅ Complete
| Component | Status | Notes |
|-----------|--------|-------|
| Typer + Rich setup | ✅ | Both `src/cli` and standalone `cli/` |
| `webber --version` | ✅ | Shows version from pyproject.toml |
| Console theming | ✅ | Centralized color palette |
| Markdown rendering | ✅ | Rich markdown output |
### Phase 4: Agentic Loop ⚠️ Partial
| Component | Status | Notes |
|-----------|--------|-------|
| `webber chat` command | ✅ | Interactive mode works |
| `webber explore` command | ✅ | One-shot query works |
| `SessionState` dataclass | ✅ | Basic context tracking |
| `AgenticLoop` class | ⚠️ | Basic implementation, not fully utilized |
| Conversation history | ❌ | Not persisted between turns in CLI |
| Context management | ❌ | No token counting or summarization |
### Phase 5: REST API ✅ Complete
| Component | Status | Notes |
|-----------|--------|-------|
| `POST /agents/run` | ✅ | Execute agent with prompt |
| `GET /agents/` | ✅ | List available agents |
| `GET /agents/{name}` | ✅ | Get agent info |
| Request/response schemas | ✅ | Pydantic models |
### Phase 6: Polish & Tests ⚠️ Partial
| Component | Status | Notes |
|-----------|--------|-------|
| Tool unit tests | ✅ | 17 tests covering all tools |
| API endpoint tests | ✅ | 5 tests for agent routes |
| Health check tests | ✅ | 2 tests |
| Integration tests | ❌ | No real LLM integration tests |
| CLI E2E tests | ❌ | Not implemented |
| Streaming responses | ❌ | Not implemented |
---
## Future Work: Remaining Features
### High Priority
| Feature | Category | Description | Complexity |
|---------|----------|-------------|------------|
| **Write tool** | Tools | Create new files | Medium |
| **Edit tool** | Tools | old_string/new_string pattern like Claude | Medium |
| **Full Bash tool** | Tools | Write-enabled shell for Task agent | Medium |
| **Plan Agent** | Agents | Design implementation approaches | High |
| **Task Agent** | Agents | Autonomous multi-step execution | High |
| **Context summarization** | Infrastructure | Compress history at token limit | High |
| **Conversation persistence** | CLI | Multi-turn memory in chat mode | Medium |
### Medium Priority
| Feature | Category | Description | Complexity |
|---------|----------|-------------|------------|
| **Streaming responses** | CLI | Real-time token display | Medium |
| **Web search tool** | Tools | External search API integration | Medium |
| **Tool result caching** | Infrastructure | Cache file reads for performance | Low |
| **Session persistence** | CLI | Save/resume conversations | Medium |
| **Todo tracking** | CLI | Built-in task list (`/todo`) | Medium |
| **Git integration** | CLI | Auto-commit, branch management | Medium |
| **Agent handoff** | Orchestration | Explore → Plan → Task workflow | High |
| **Retry logic** | Infrastructure | Auto-retry on tool failures | Low |
### Low Priority
| Feature | Category | Description | Complexity |
|---------|----------|-------------|------------|
| **Notebook editing** | Tools | Jupyter cell manipulation | Medium |
| **MCP support** | Infrastructure | Model Context Protocol | High |
| **Config file** | CLI | `~/.webber/config.toml` | Low |
| **IDE integration** | CLI | VS Code extension | High |
| **Parallel agents** | Orchestration | Concurrent agent execution | High |
| **Agent memory** | Orchestration | Shared context between agents | Medium |
---
## Testing Coverage Gaps
| Area | Current | Target | Gap |
|------|---------|--------|-----|
| Tool unit tests | 17 | 17 | ✅ |
| API tests | 5 | 10 | Need error handling, edge cases |
| Integration tests | 0 | 5 | Agent + real LLM tests |
| CLI E2E tests | 0 | 10 | Full workflow tests |
| Security tests | 0 | 5 | Path traversal, injection |
---
## Known Issues
1. **Model hallucination** - Mistral Nemo sometimes makes up file contents instead of using actual tool results.
2. **No conversation memory** - CLI chat mode doesn't persist context between sessions.
3. **No streaming** - Responses appear all at once, no real-time token display.
4. **Temperature setting** - Changed from 0.0 to 0.3 for Mistral Nemo compatibility, may affect determinism.
---
## Architecture Decisions Made
| Decision | Choice | Rationale |
|----------|--------|-----------|
| Separate CLI package | `cli/` at root | Can be extracted as standalone client |
| Sanitized Ollama provider | Custom wrapper | Fixes PydanticAI + Ollama `content: null` bug |
| Dev port 8095 | Separate from prod 8086 | Avoid conflicts with Docker deployment |
| Tool choice "required" | Force tool use | Mistral Nemo needs explicit instruction |
| Temperature 0.3 | Mistral recommendation | 0.0 caused issues with Nemo |
---
## Estimated Effort to Full Parity
| Milestone | Effort | Features |
|-----------|--------|----------|
| **MVP (current)** | Done | Explore agent, basic CLI, REST API |
| **Usable daily driver** | 2-3 weeks | Write/Edit tools, Plan agent, git integration |
| **Claude Code parity** | 2-3 months | Task agent, streaming, MCP, IDE integration |
---
## Quick Reference: What Works Now
```bash
# Start dev server
./wakeup.sh
# CLI commands
.venv/bin/webber-cli status # Check API connection
.venv/bin/webber-cli explore "find tests" # One-shot exploration
.venv/bin/webber-cli chat # Interactive mode
# API endpoints
curl http://localhost:8095/health
curl http://localhost:8095/agents/
curl -X POST http://localhost:8095/agents/run \
-H "Content-Type: application/json" \
-d '{"agent_type":"explore","prompt":"list python files","working_dir":"."}'
```
+4 -8
View File
@@ -1,7 +1,7 @@
[project]
name = "webber"
version = "0.2.3"
description = "Mrs. Webber - Multi-Agent AI Development System"
name = "webber-api"
version = "0.3.0"
description = "Webber API - Multi-Agent AI Development Server"
authors = [
{name = "jpmschweitzer"}
]
@@ -15,17 +15,13 @@ classifiers = [
"Topic :: Software Development :: Code Generators",
]
[project.scripts]
webber = "src.cli.main:app"
webber-cli = "cli.main:app"
[build-system]
requires = ["setuptools>=75.0", "wheel"]
build-backend = "setuptools.build_meta"
[tool.setuptools.packages.find]
where = ["."]
include = ["src*", "cli*"]
include = ["src*"]
[tool.pytest.ini_options]
testpaths = ["tests"]
@@ -24,3 +24,4 @@ rich~=13.9.0
# Utilities
python-multipart~=0.0.21
python-dotenv~=1.2.1
pathspec~=0.12.1 # Gitignore pattern matching
@@ -5,6 +5,7 @@ import os
from pathlib import Path
from src.domains.tools.base import BaseTool, ToolResult
from src.domains.tools.gitignore import filter_gitignored
from src.shared.logging import logged, get_logger
logger = get_logger(__name__)
@@ -24,10 +25,13 @@ Args:
pattern: Glob pattern (e.g., "**/*.py", "src/**/*.ts", "*.md")
path: Directory to search in (default: working directory)
limit: Maximum number of files to return (default: 100)
honor_gitignore: Filter out gitignored files (default: True)
Returns:
List of matching absolute file paths, sorted by modification time (newest first).
Returns error if path not found or not allowed.
By default, excludes files matching .gitignore patterns and common ignored
directories like .venv/, node_modules/, __pycache__/, etc.
Examples:
- "**/*.py" - All Python files recursively
@@ -43,7 +47,8 @@ IMPORTANT:
def __init__(
self,
allowed_paths: list[str] | None = None,
max_results: int = 100
max_results: int = 100,
honor_gitignore: bool = True
):
"""
Initialize GlobFilesTool.
@@ -51,16 +56,19 @@ IMPORTANT:
Args:
allowed_paths: List of allowed directory prefixes
max_results: Maximum files to return
honor_gitignore: Whether to filter out gitignored files by default
"""
self.allowed_paths = allowed_paths or []
self.max_results = max_results
self.honor_gitignore = honor_gitignore
@logged()
async def execute(
self,
pattern: str,
path: str | None = None,
limit: int | None = None
limit: int | None = None,
honor_gitignore: bool | None = None
) -> ToolResult:
"""
Find files matching glob pattern.
@@ -69,11 +77,13 @@ IMPORTANT:
pattern: Glob pattern to match
path: Directory to search (default: current directory)
limit: Maximum results to return
honor_gitignore: Filter out gitignored files (default: instance setting)
Returns:
ToolResult with list of matching file paths
"""
limit = limit or self.max_results
should_filter_gitignore = honor_gitignore if honor_gitignore is not None else self.honor_gitignore
search_path = Path(path) if path else Path.cwd()
# Validate search path is allowed
@@ -97,6 +107,10 @@ IMPORTANT:
if self.allowed_paths:
files = [f for f in files if self._validate_path(f, self.allowed_paths)]
# Filter out gitignored files
if should_filter_gitignore:
files = filter_gitignored(files, search_path)
# Sort by modification time (newest first)
files_with_mtime = []
for f in files:
+152
View File
@@ -0,0 +1,152 @@
"""
Gitignore pattern matching for tool filtering.
Uses pathspec to parse .gitignore files and filter out ignored paths.
"""
from functools import lru_cache
from pathlib import Path
import pathspec
from src.shared.logging import get_logger
logger = get_logger(__name__)
class GitignoreFilter:
"""
Filter files based on .gitignore patterns.
Parses .gitignore files from the root directory and any parent directories,
then provides methods to check if paths should be ignored.
"""
def __init__(self, root_dir: str | Path):
"""
Initialize GitignoreFilter for a directory.
Args:
root_dir: Root directory to search for .gitignore files
"""
self.root_dir = Path(root_dir).resolve()
self._spec: pathspec.PathSpec | None = None
self._load_patterns()
def _load_patterns(self) -> None:
"""Load gitignore patterns from .gitignore files."""
patterns: list[str] = []
# Always ignore common directories that should never be searched
default_ignores = [
".git/",
".venv/",
"venv/",
"__pycache__/",
"*.pyc",
".mypy_cache/",
".pytest_cache/",
".ruff_cache/",
"node_modules/",
".tox/",
".nox/",
"*.egg-info/",
"dist/",
"build/",
".eggs/",
]
patterns.extend(default_ignores)
# Find and parse .gitignore in root directory
gitignore_path = self.root_dir / ".gitignore"
if gitignore_path.exists():
try:
content = gitignore_path.read_text(encoding="utf-8")
for line in content.splitlines():
line = line.strip()
# Skip empty lines and comments
if line and not line.startswith("#"):
patterns.append(line)
logger.debug(f"Loaded {len(patterns)} patterns from {gitignore_path}")
except (OSError, UnicodeDecodeError) as e:
logger.warning(f"Failed to read .gitignore: {e}")
# Create pathspec matcher
self._spec = pathspec.PathSpec.from_lines("gitwildmatch", patterns)
def is_ignored(self, path: str | Path) -> bool:
"""
Check if a path should be ignored.
Args:
path: Absolute or relative path to check
Returns:
True if the path matches gitignore patterns
"""
if self._spec is None:
return False
path = Path(path)
# Make path relative to root for matching
try:
if path.is_absolute():
rel_path = path.resolve().relative_to(self.root_dir)
else:
rel_path = path
except ValueError:
# Path is not under root_dir, don't filter
return False
# Convert to string with forward slashes for pathspec
path_str = str(rel_path).replace("\\", "/")
# Check if it's a directory (add trailing slash for directory patterns)
if path.is_dir():
path_str_dir = path_str + "/"
return self._spec.match_file(path_str) or self._spec.match_file(path_str_dir)
return self._spec.match_file(path_str)
def filter_paths(self, paths: list[Path]) -> list[Path]:
"""
Filter a list of paths, removing ignored ones.
Args:
paths: List of Path objects to filter
Returns:
List of paths that are not ignored
"""
return [p for p in paths if not self.is_ignored(p)]
@lru_cache(maxsize=16)
def get_gitignore_filter(root_dir: str) -> GitignoreFilter:
"""
Get a cached GitignoreFilter for a directory.
Uses LRU cache to avoid re-parsing .gitignore for repeated calls.
Args:
root_dir: Root directory path (string for cache key)
Returns:
GitignoreFilter instance
"""
return GitignoreFilter(root_dir)
def filter_gitignored(paths: list[Path], root_dir: str | Path) -> list[Path]:
"""
Convenience function to filter paths using gitignore patterns.
Args:
paths: List of paths to filter
root_dir: Root directory containing .gitignore
Returns:
Filtered list of paths
"""
filter_instance = get_gitignore_filter(str(Path(root_dir).resolve()))
return filter_instance.filter_paths(paths)
@@ -6,6 +6,7 @@ from pathlib import Path
from typing import Literal
from src.domains.tools.base import BaseTool, ToolResult
from src.domains.tools.gitignore import filter_gitignored
from src.shared.logging import logged, get_logger
logger = get_logger(__name__)
@@ -28,10 +29,13 @@ Args:
context_lines: Lines of context before/after matches (default: 0)
case_sensitive: Whether search is case-sensitive (default: True)
output_mode: "content" for matching lines, "files" for file paths only
honor_gitignore: Filter out gitignored files (default: True)
Returns:
Matching lines with file paths and line numbers, or list of files.
Format: "filepath:line_num: content"
By default, excludes files matching .gitignore patterns and common ignored
directories like .venv/, node_modules/, __pycache__/, etc.
Examples:
- pattern="def.*init" file_glob="*.py" - Find init methods in Python files
@@ -48,7 +52,8 @@ IMPORTANT:
self,
allowed_paths: list[str] | None = None,
max_results: int = 100,
max_file_size: int = 1_000_000 # 1MB
max_file_size: int = 1_000_000, # 1MB
honor_gitignore: bool = True
):
"""
Initialize GrepContentTool.
@@ -57,10 +62,12 @@ IMPORTANT:
allowed_paths: List of allowed directory prefixes
max_results: Maximum matches to return
max_file_size: Skip files larger than this (bytes)
honor_gitignore: Whether to filter out gitignored files by default
"""
self.allowed_paths = allowed_paths or []
self.max_results = max_results
self.max_file_size = max_file_size
self.honor_gitignore = honor_gitignore
@logged()
async def execute(
@@ -70,7 +77,8 @@ IMPORTANT:
file_glob: str | None = None,
context_lines: int = 0,
case_sensitive: bool = True,
output_mode: Literal["content", "files"] = "content"
output_mode: Literal["content", "files"] = "content",
honor_gitignore: bool | None = None
) -> ToolResult:
"""
Search for pattern in files.
@@ -82,10 +90,12 @@ IMPORTANT:
context_lines: Context lines around matches
case_sensitive: Case-sensitive search
output_mode: "content" or "files"
honor_gitignore: Filter out gitignored files (default: instance setting)
Returns:
ToolResult with matching content or file list
"""
should_filter_gitignore = honor_gitignore if honor_gitignore is not None else self.honor_gitignore
search_path = Path(path) if path else Path.cwd()
# Validate path
@@ -119,6 +129,10 @@ IMPORTANT:
if self._validate_path(f, self.allowed_paths)
]
# Filter out gitignored files
if should_filter_gitignore:
files_to_search = filter_gitignored(files_to_search, search_path)
# Search files
matches = []
files_with_matches = set()

Some files were not shown because too many files have changed in this diff Show More