webber/webber-api/docs/COVERAGE.md

Webber Feature Coverage

Tracking progress towards Claude Code-like functionality

Current Status: ~40% Complete

Last updated: 2026-01-10

---

Phase 1-6: Foundation (Original Plan)

Phase 1: Tool Infrastructure ✅ Complete

Component	Status	Notes
BaseTool abstract class	✅	src/domains/tools/base.py
ToolResult dataclass	✅	Consistent success/error/truncated handling
ReadFileTool	✅	With line numbers, offset/limit support
GlobFilesTool	✅	Pattern matching, sorted by mtime
GrepContentTool	✅	Regex search with context lines
BashReadOnlyTool	✅	Allowlist-based command filtering
Path validation	✅	allowed_paths restriction

Status: Tools now honor .gitignore patterns and default ignores (.venv/, __pycache__/, etc.)

Phase 2: Explore Agent ✅ Complete

Component	Status	Notes
BaseAgent abstract class	✅	src/domains/agents/base.py
Agent registry	✅	register_agent(), get_agent(), list_agents()
ExploreAgentImpl	✅	PydanticAI-based implementation
System prompts	✅	Mistral-optimized with tool examples
Tool registration	✅	@agent.tool decorator pattern
Sanitized Ollama provider	✅	Fixes content: null issue

Gap: Mistral Nemo sometimes hallucinates instead of using tool results.

Phase 3: CLI Foundation ✅ Complete

Component	Status	Notes
Typer + Rich setup	✅	Both src/cli and standalone cli/
webber --version	✅	Shows version from pyproject.toml
Console theming	✅	Centralized color palette
Markdown rendering	✅	Rich markdown output

Phase 4: Agentic Loop ⚠️ Partial

Component	Status	Notes
webber chat command	✅	Interactive mode works
webber explore command	✅	One-shot query works
SessionState dataclass	✅	Basic context tracking
AgenticLoop class	⚠️	Basic implementation, not fully utilized
Conversation history	❌	Not persisted between turns in CLI
Context management	❌	No token counting or summarization

Phase 5: REST API ✅ Complete

Component	Status	Notes
POST /agents/run	✅	Execute agent with prompt
GET /agents/	✅	List available agents
GET /agents/{name}	✅	Get agent info
Request/response schemas	✅	Pydantic models

Phase 6: Polish & Tests ⚠️ Partial

Component	Status	Notes
Tool unit tests	✅	17 tests covering all tools
API endpoint tests	✅	5 tests for agent routes
Health check tests	✅	2 tests
Integration tests	❌	No real LLM integration tests
CLI E2E tests	❌	Not implemented
Streaming responses	❌	Not implemented

---

Future Work: Remaining Features

High Priority

Feature	Category	Description	Complexity
Write tool	Tools	Create new files	Medium
Edit tool	Tools	old_string/new_string pattern like Claude	Medium
Full Bash tool	Tools	Write-enabled shell for Task agent	Medium
Plan Agent	Agents	Design implementation approaches	High
Task Agent	Agents	Autonomous multi-step execution	High
Context summarization	Infrastructure	Compress history at token limit	High
Conversation persistence	CLI	Multi-turn memory in chat mode	Medium

Medium Priority

Feature	Category	Description	Complexity
Streaming responses	CLI	Real-time token display	Medium
Web search tool	Tools	External search API integration	Medium
Tool result caching	Infrastructure	Cache file reads for performance	Low
Session persistence	CLI	Save/resume conversations	Medium
Todo tracking	CLI	Built-in task list (/todo)	Medium
Git integration	CLI	Auto-commit, branch management	Medium
Agent handoff	Orchestration	Explore → Plan → Task workflow	High
Retry logic	Infrastructure	Auto-retry on tool failures	Low

Low Priority

Feature	Category	Description	Complexity
Notebook editing	Tools	Jupyter cell manipulation	Medium
MCP support	Infrastructure	Model Context Protocol	High
Config file	CLI	~/.webber/config.toml	Low
IDE integration	CLI	VS Code extension	High
Parallel agents	Orchestration	Concurrent agent execution	High
Agent memory	Orchestration	Shared context between agents	Medium

---

Testing Coverage Gaps

Area	Current	Target	Gap
Tool unit tests	17	17	✅
API tests	5	10	Need error handling, edge cases
Integration tests	0	5	Agent + real LLM tests
CLI E2E tests	0	10	Full workflow tests
Security tests	0	5	Path traversal, injection

---

Known Issues

1. Model hallucination - Mistral Nemo sometimes makes up file contents instead of using actual tool results.

2. No conversation memory - CLI chat mode doesn't persist context between sessions.

3. No streaming - Responses appear all at once, no real-time token display.

4. Temperature setting - Changed from 0.0 to 0.3 for Mistral Nemo compatibility, may affect determinism.

---

Architecture Decisions Made

Decision	Choice	Rationale
Separate CLI package	cli/ at root	Can be extracted as standalone client
Sanitized Ollama provider	Custom wrapper	Fixes PydanticAI + Ollama content: null bug
Dev port 8095	Separate from prod 8086	Avoid conflicts with Docker deployment
Tool choice "required"	Force tool use	Mistral Nemo needs explicit instruction
Temperature 0.3	Mistral recommendation	0.0 caused issues with Nemo

---

Estimated Effort to Full Parity

Milestone	Effort	Features
MVP (current)	Done	Explore agent, basic CLI, REST API
Usable daily driver	2-3 weeks	Write/Edit tools, Plan agent, git integration
Claude Code parity	2-3 months	Task agent, streaming, MCP, IDE integration

---

Quick Reference: What Works Now

# Start dev server
./wakeup.sh

# CLI commands
.venv/bin/webber-cli status                    # Check API connection
.venv/bin/webber-cli explore "find tests"      # One-shot exploration
.venv/bin/webber-cli chat                      # Interactive mode

# API endpoints
curl http://localhost:8095/health
curl http://localhost:8095/agents/
curl -X POST http://localhost:8095/agents/run \
  -H "Content-Type: application/json" \
  -d '{"agent_type":"explore","prompt":"list python files","working_dir":"."}'