webber/webber-api/docs/COVERAGE.md

Webber Feature Coverage

Tracking progress towards Claude Code-like functionality

Current Status: ~80% Complete

Last updated: 2026-01-11

---

Phase 1-6: Foundation (Original Plan)

Phase 1: Tool Infrastructure ✅ Complete

Component	Status	Notes
BaseTool abstract class	✅	src/domains/tools/base.py
ToolResult dataclass	✅	Consistent success/error/truncated handling
ReadFileTool	✅	With line numbers, offset/limit support
GlobFilesTool	✅	Pattern matching, sorted by mtime
GrepContentTool	✅	Regex search with context lines
BashReadOnlyTool	✅	Allowlist-based command filtering
EditFileTool	✅	Find-and-replace with unique match validation
WriteFileTool	✅	Create/overwrite files with size limits
BashTool (full)	✅	Write-enabled shell with safety controls
WebSearchTool	✅	SearXNG integration for web search
Path validation	✅	allowed_paths restriction

Status: Tools honor .gitignore patterns and default ignores (.venv/, __pycache__/, etc.)

Phase 2: Explore Agent ✅ Complete

Component	Status	Notes
BaseAgent abstract class	✅	src/domains/agents/base.py
Agent registry	✅	register_agent(), get_agent(), list_agents()
ExploreAgentImpl	✅	PydanticAI-based implementation
System prompts	✅	Mistral-optimized with tool examples
Tool registration	✅	@agent.tool decorator pattern
Sanitized Ollama provider	✅	Fixes content: null issue
Streaming support	✅	run_stream() method with SSE

Available tools: read_file, glob_files, grep_content, bash_readonly, edit_file, write_file, bash, web_search

Gap: Mistral Nemo sometimes hallucinates instead of using tool results.

Phase 2b: Plan Agent ✅ Complete

Component	Status	Notes
PlanAgentImpl	✅	READ-ONLY software architect agent
System prompts	✅	Architecture-focused with tool examples
Tool registration	✅	Only read-only tools (4 tools)
Streaming support	✅	run_stream() method with SSE
Unit tests	✅	15 tests for registration, tools, API

Available tools: read_file, glob_files, grep_content, bash_readonly (read-only only)

Purpose: Design implementation strategies before coding - explores codebase and creates step-by-step plans.

Phase 3: CLI Foundation ✅ Complete

Component	Status	Notes
Typer + Rich setup	✅	Standalone webber-cli/ package
webber-cli --version	✅	Shows version from pyproject.toml
Console theming	✅	Centralized color palette
Markdown rendering	✅	Rich markdown output
Streaming display	✅	Real-time token output with --stream flag

Phase 4: Agentic Loop ✅ Complete

Component	Status	Notes
webber-cli chat command	✅	Interactive mode with streaming
webber-cli explore command	✅	One-shot query with streaming
SessionState dataclass	✅	Basic context tracking
AgenticLoop class	✅	Basic implementation
Conversation persistence	✅	SQLAlchemy async with SQLite/PostgreSQL
Context summarization	✅	Token counting (litellm) + auto-summarization
Conversation API	✅	/conversations/ REST endpoints

Database: SQLite (dev) or PostgreSQL (prod), async via SQLAlchemy 2.0

Phase 5: REST API ✅ Complete

Component	Status	Notes
POST /agents/run	✅	Execute agent with prompt
POST /agents/stream	✅	SSE streaming responses
GET /agents/	✅	List available agents
GET /agents/{name}	✅	Get agent info
Request/response schemas	✅	Pydantic models

Phase 6: Polish & Tests ✅ Complete

Component	Status	Notes
Tool unit tests	✅	109 tests total
API endpoint tests	✅	11 tests for agent routes
Health check tests	✅	2 tests
Security tests	✅	14 tests for path traversal, injection
Integration tests	✅	10 tests with real LLM (requires Ollama)
E2E tests	✅	12 tests against running API server

---

Future Work: Remaining Features

High Priority

Feature	Category	Description	Complexity
Plan Agent	Agents	✅ Design implementation approaches	High
Task Agent	Agents	✅ Autonomous multi-step execution	High
Context summarization	Infrastructure	✅ Token counting + auto-summarization	High
Conversation persistence	Infrastructure	✅ SQLAlchemy async database layer	Medium

Medium Priority

Feature	Category	Description	Complexity
Web search summarizer	Tools	Agent to extract core content from web pages (remove nav, footers, etc.) and preserve relevant links for nested fetching	Medium
Tool result caching	Infrastructure	Cache file reads for performance	Low
Session persistence	CLI	Save/resume conversations	Medium
Todo tracking	CLI	Built-in task list (/todo)	Medium
Git integration	CLI	Auto-commit, branch management	Medium
Agent handoff	Orchestration	Explore → Plan → Task workflow	High
Retry logic	Infrastructure	✅ Auto-retry with exponential backoff	Low

Low Priority

Feature	Category	Description	Complexity
Notebook editing	Tools	Jupyter cell manipulation	Medium
MCP support	Infrastructure	Model Context Protocol	High
Config file	CLI	~/.webber/config.toml	Low
IDE integration	CLI	VS Code extension	High
Parallel agents	Orchestration	Concurrent agent execution	High
Agent memory	Orchestration	Shared context between agents	Medium

---

Testing Coverage

Area	Current	Target	Status
Tool unit tests	109	109	✅
API tests	11	11	✅
Plan agent tests	15	15	✅
Task agent tests	15	15	✅
Conversation tests	19	19	✅
Token tests	6	6	✅
Retry tests	29	29	✅
Security tests	14	14	✅
Integration tests	10	10	✅ Agent + real LLM
E2E tests	12	12	✅ Full API workflow

Total: 205 tests passing

Test breakdown:

• Read/Glob/Grep tools: 17 tests
• Edit/Write tools: 22 tests
• Bash tools: 22 tests
• Web search: 10 tests
• Gitignore filtering: 10 tests
• API endpoints: 11 tests
• Plan agent: 15 tests
• Task agent: 15 tests
• Conversations: 19 tests
• Tokens: 6 tests
• Retry: 29 tests
• Security: 14 tests
• Health checks: 2 tests
• Integration (LLM): 10 tests
• E2E (API): 12 tests

Running tests:

# Unit tests only (default)
pytest tests/

# Include integration tests (requires Ollama)
pytest tests/ --run-integration

# Include E2E tests (requires running API server)
pytest tests/ --run-e2e

# All tests
pytest tests/ --run-integration --run-e2e

---

Known Issues

1. Model hallucination - Mistral Nemo sometimes makes up file contents instead of using actual tool results.

2. Temperature setting - Changed from 0.0 to 0.3 for Mistral Nemo compatibility, may affect determinism.

3. SQLAlchemy deprecation - datetime.utcnow() deprecation warning from SQLAlchemy.

---

Architecture Decisions Made

Decision	Choice	Rationale
Monorepo structure	webber-api/, webber-cli/	Separate packages, shared root
Sanitized Ollama provider	Custom wrapper	Fixes PydanticAI + Ollama content: null bug
Dev port 8095	Separate from prod 8086	Avoid conflicts with Docker deployment
Tool choice "required"	Force tool use	Mistral Nemo needs explicit instruction
Temperature 0.3	Mistral recommendation	0.0 caused issues with Nemo
SearXNG for search	Self-hosted	Privacy, no API keys needed
SSE for streaming	Server-Sent Events	Simple, well-supported

---

Quick Reference: What Works Now

# Start dev server
cd webber-api && ./wakeup.sh

# CLI commands (from webber-cli/)
.venv/bin/webber-cli status                    # Check API connection
.venv/bin/webber-cli explore "find tests"      # One-shot exploration
.venv/bin/webber-cli explore "query" --no-stream  # Batch mode
.venv/bin/webber-cli chat                      # Interactive mode

# API endpoints
curl http://localhost:8095/health
curl http://localhost:8095/agents/
curl -X POST http://localhost:8095/agents/run \
  -H "Content-Type: application/json" \
  -d '{"agent_type":"explore","prompt":"list python files","working_dir":"."}'

# Plan agent (read-only, creates implementation plans)
curl -X POST http://localhost:8095/agents/run \
  -H "Content-Type: application/json" \
  -d '{"agent_type":"plan","prompt":"plan how to add user auth","working_dir":"."}'

# Streaming endpoint
curl -N http://localhost:8095/agents/stream \
  -H "Content-Type: application/json" \
  -d '{"agent_type":"explore","prompt":"find config files","working_dir":"."}'

# Conversation API (stateful multi-turn)
curl -X POST http://localhost:8095/conversations/ \
  -H "Content-Type: application/json" \
  -H "X-API-Key: dev-key" \
  -d '{"agent_type":"explore","working_dir":"."}'

curl -X POST http://localhost:8095/conversations/{id}/messages \
  -H "Content-Type: application/json" \
  -H "X-API-Key: dev-key" \
  -d '{"content":"find all Python files"}'

---

Tools Available

Tool	Type	Description
read_file	Read	Read file contents with line numbers
glob_files	Read	Find files by pattern
grep_content	Read	Search file contents with regex
bash_readonly	Read	Safe bash commands (ls, git status, etc.)
edit_file	Write	Find-and-replace editing
write_file	Write	Create/overwrite files
bash	Write	Full bash with safety controls
web_search	External	Search web via SearXNG