jpmschweitzer/webber
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 5 Wiki Activity
Files
470b7448acc2cee44d2c2dea033e5cf06d5dac6a
webber/webber-api/docs/COVERAGE.md
T
jpmschweitzerandClaude Opus 4.5 b5b2346db5
Build and Push API / release (push) Successful in 5s
Details
Build and Push API / build (push) Successful in 1m15s
Details
feat: add Plan Agent for implementation planning 
- Add PlanAgentImpl with read-only tools only
- System prompts optimized for architecture planning
- Outputs step-by-step implementation plans with critical files
- 15 unit tests for registration, tools, and API
- Update COVERAGE.md to ~70% complete

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 19:57:30 +01:00

250 lines
9.3 KiB
Markdown
Raw Blame History

# Webber Feature Coverage
> Tracking progress towards Claude Code-like functionality
## Current Status: ~70% Complete
Last updated: 2026-01-11
---
## Phase 1-6: Foundation (Original Plan)
### Phase 1: Tool Infrastructure ✅ Complete
| Component | Status | Notes |
|-----------|--------|-------|
| `BaseTool` abstract class | ✅ | `src/domains/tools/base.py` |
| `ToolResult` dataclass | ✅ | Consistent success/error/truncated handling |
| `ReadFileTool` | ✅ | With line numbers, offset/limit support |
| `GlobFilesTool` | ✅ | Pattern matching, sorted by mtime |
| `GrepContentTool` | ✅ | Regex search with context lines |
| `BashReadOnlyTool` | ✅ | Allowlist-based command filtering |
| `EditFileTool` | ✅ | Find-and-replace with unique match validation |
| `WriteFileTool` | ✅ | Create/overwrite files with size limits |
| `BashTool` (full) | ✅ | Write-enabled shell with safety controls |
| `WebSearchTool` | ✅ | SearXNG integration for web search |
| Path validation | ✅ | `allowed_paths` restriction |
**Status:** Tools honor `.gitignore` patterns and default ignores (`.venv/`, `__pycache__/`, etc.)
### Phase 2: Explore Agent ✅ Complete
| Component | Status | Notes |
|-----------|--------|-------|
| `BaseAgent` abstract class | ✅ | `src/domains/agents/base.py` |
| Agent registry | ✅ | `register_agent()`, `get_agent()`, `list_agents()` |
| `ExploreAgentImpl` | ✅ | PydanticAI-based implementation |
| System prompts | ✅ | Mistral-optimized with tool examples |
| Tool registration | ✅ | `@agent.tool` decorator pattern |
| Sanitized Ollama provider | ✅ | Fixes `content: null` issue |
| Streaming support | ✅ | `run_stream()` method with SSE |
**Available tools:** `read_file`, `glob_files`, `grep_content`, `bash_readonly`, `edit_file`, `write_file`, `bash`, `web_search`
**Gap:** Mistral Nemo sometimes hallucinates instead of using tool results.
### Phase 2b: Plan Agent ✅ Complete
| Component | Status | Notes |
|-----------|--------|-------|
| `PlanAgentImpl` | ✅ | READ-ONLY software architect agent |
| System prompts | ✅ | Architecture-focused with tool examples |
| Tool registration | ✅ | Only read-only tools (4 tools) |
| Streaming support | ✅ | `run_stream()` method with SSE |
| Unit tests | ✅ | 15 tests for registration, tools, API |
**Available tools:** `read_file`, `glob_files`, `grep_content`, `bash_readonly` (read-only only)
**Purpose:** Design implementation strategies before coding - explores codebase and creates step-by-step plans.
### Phase 3: CLI Foundation ✅ Complete
| Component | Status | Notes |
|-----------|--------|-------|
| Typer + Rich setup | ✅ | Standalone `webber-cli/` package |
| `webber-cli --version` | ✅ | Shows version from pyproject.toml |
| Console theming | ✅ | Centralized color palette |
| Markdown rendering | ✅ | Rich markdown output |
| Streaming display | ✅ | Real-time token output with `--stream` flag |
### Phase 4: Agentic Loop ⚠️ Partial
| Component | Status | Notes |
|-----------|--------|-------|
| `webber-cli chat` command | ✅ | Interactive mode with streaming |
| `webber-cli explore` command | ✅ | One-shot query with streaming |
| `SessionState` dataclass | ✅ | Basic context tracking |
| `AgenticLoop` class | ⚠️ | Basic implementation, not fully utilized |
| Conversation history | ❌ | Not persisted between turns in CLI |
| Context management | ❌ | No token counting or summarization |
### Phase 5: REST API ✅ Complete
| Component | Status | Notes |
|-----------|--------|-------|
| `POST /agents/run` | ✅ | Execute agent with prompt |
| `POST /agents/stream` | ✅ | SSE streaming responses |
| `GET /agents/` | ✅ | List available agents |
| `GET /agents/{name}` | ✅ | Get agent info |
| Request/response schemas | ✅ | Pydantic models |
### Phase 6: Polish & Tests ✅ Complete
| Component | Status | Notes |
|-----------|--------|-------|
| Tool unit tests | ✅ | 109 tests total |
| API endpoint tests | ✅ | 11 tests for agent routes |
| Health check tests | ✅ | 2 tests |
| Security tests | ✅ | 14 tests for path traversal, injection |
| Integration tests | ✅ | 10 tests with real LLM (requires Ollama) |
| E2E tests | ✅ | 12 tests against running API server |
---
## Future Work: Remaining Features
### High Priority
| Feature | Category | Description | Complexity |
|---------|----------|-------------|------------|
| ~~**Plan Agent**~~ | Agents | ✅ Design implementation approaches | High |
| **Task Agent** | Agents | Autonomous multi-step execution | High |
| **Context summarization** | Infrastructure | Compress history at token limit | High |
| **Conversation persistence** | CLI | Multi-turn memory in chat mode | Medium |
### Medium Priority
| Feature | Category | Description | Complexity |
|---------|----------|-------------|------------|
| **Web search summarizer** | Tools | Agent to extract core content from web pages (remove nav, footers, etc.) and preserve relevant links for nested fetching | Medium |
| **Tool result caching** | Infrastructure | Cache file reads for performance | Low |
| **Session persistence** | CLI | Save/resume conversations | Medium |
| **Todo tracking** | CLI | Built-in task list (`/todo`) | Medium |
| **Git integration** | CLI | Auto-commit, branch management | Medium |
| **Agent handoff** | Orchestration | Explore → Plan → Task workflow | High |
| **Retry logic** | Infrastructure | Auto-retry on tool failures | Low |
### Low Priority
| Feature | Category | Description | Complexity |
|---------|----------|-------------|------------|
| **Notebook editing** | Tools | Jupyter cell manipulation | Medium |
| **MCP support** | Infrastructure | Model Context Protocol | High |
| **Config file** | CLI | `~/.webber/config.toml` | Low |
| **IDE integration** | CLI | VS Code extension | High |
| **Parallel agents** | Orchestration | Concurrent agent execution | High |
| **Agent memory** | Orchestration | Shared context between agents | Medium |
---
## Testing Coverage
| Area | Current | Target | Status |
|------|---------|--------|--------|
| Tool unit tests | 109 | 109 | ✅ |
| API tests | 11 | 11 | ✅ |
| Plan agent tests | 15 | 15 | ✅ |
| Security tests | 14 | 14 | ✅ |
| Integration tests | 10 | 10 | ✅ Agent + real LLM |
| E2E tests | 12 | 12 | ✅ Full API workflow |
**Test breakdown:**
- Read/Glob/Grep tools: 17 tests
- Edit/Write tools: 22 tests
- Bash tools: 22 tests
- Web search: 10 tests
- Gitignore filtering: 10 tests
- API endpoints: 11 tests
- Plan agent: 15 tests
- Security: 14 tests
- Health checks: 2 tests
- Integration (LLM): 10 tests
- E2E (API): 12 tests
**Running tests:**
```bash
# Unit tests only (default)
pytest tests/
# Include integration tests (requires Ollama)
pytest tests/ --run-integration
# Include E2E tests (requires running API server)
pytest tests/ --run-e2e
# All tests
pytest tests/ --run-integration --run-e2e
```
---
## Known Issues
1. **Model hallucination** - Mistral Nemo sometimes makes up file contents instead of using actual tool results.
2. **No conversation memory** - CLI chat mode doesn't persist context between sessions.
3. **Temperature setting** - Changed from 0.0 to 0.3 for Mistral Nemo compatibility, may affect determinism.
---
## Architecture Decisions Made
| Decision | Choice | Rationale |
|----------|--------|-----------|
| Monorepo structure | `webber-api/`, `webber-cli/` | Separate packages, shared root |
| Sanitized Ollama provider | Custom wrapper | Fixes PydanticAI + Ollama `content: null` bug |
| Dev port 8095 | Separate from prod 8086 | Avoid conflicts with Docker deployment |
| Tool choice "required" | Force tool use | Mistral Nemo needs explicit instruction |
| Temperature 0.3 | Mistral recommendation | 0.0 caused issues with Nemo |
| SearXNG for search | Self-hosted | Privacy, no API keys needed |
| SSE for streaming | Server-Sent Events | Simple, well-supported |
---
## Quick Reference: What Works Now
```bash
# Start dev server
cd webber-api && ./wakeup.sh
# CLI commands (from webber-cli/)
.venv/bin/webber-cli status # Check API connection
.venv/bin/webber-cli explore "find tests" # One-shot exploration
.venv/bin/webber-cli explore "query" --no-stream # Batch mode
.venv/bin/webber-cli chat # Interactive mode
# API endpoints
curl http://localhost:8095/health
curl http://localhost:8095/agents/
curl -X POST http://localhost:8095/agents/run \
-H "Content-Type: application/json" \
-d '{"agent_type":"explore","prompt":"list python files","working_dir":"."}'
# Plan agent (read-only, creates implementation plans)
curl -X POST http://localhost:8095/agents/run \
-H "Content-Type: application/json" \
-d '{"agent_type":"plan","prompt":"plan how to add user auth","working_dir":"."}'
# Streaming endpoint
curl -N http://localhost:8095/agents/stream \
-H "Content-Type: application/json" \
-d '{"agent_type":"explore","prompt":"find config files","working_dir":"."}'
```
---
## Tools Available
| Tool | Type | Description |
|------|------|-------------|
| `read_file` | Read | Read file contents with line numbers |
| `glob_files` | Read | Find files by pattern |
| `grep_content` | Read | Search file contents with regex |
| `bash_readonly` | Read | Safe bash commands (ls, git status, etc.) |
| `edit_file` | Write | Find-and-replace editing |
| `write_file` | Write | Create/overwrite files |
| `bash` | Write | Full bash with safety controls |
| `web_search` | External | Search web via SearXNG |
Reference in New Issue View Git Blame Copy Permalink