fix: replace litellm with tiktoken for token counting
Build and Push API / release (push) Successful in 3s
Build and Push API / build (push) Successful in 2m26s

- litellm had dependency conflicts with pydantic-ai
- tiktoken is lighter and already required by pydantic-ai
- Updated documentation (README.md, architecture.md)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
2026-01-11 23:05:11 +01:00
co-authored by Claude Opus 4.5
parent 2523db4da7
commit 2f97041aa9
7 changed files with 127 additions and 24 deletions
+64 -1
View File
@@ -24,18 +24,30 @@ webber/
├── src/
│ ├── main.py # App entry point (NO routes)
│ │
│ ├── db/ # Database layer
│ │ ├── __init__.py # Exports: Database, get_database, get_session
│ │ ├── database.py # SQLAlchemy async engine, session factory
│ │ └── models.py # Base declarative model
│ │
│ ├── shared/ # Cross-cutting concerns
│ │ ├── base.py # BaseController, BaseSchema
│ │ ├── config.py # Pydantic Settings
│ │ ├── logging.py # @logged decorator, trace_span
│ │ ├── exceptions.py # Custom exception hierarchy
│ │ ├── auth.py # API key validation
│ │ ── context.py # UserProvider singleton
│ │ ── context.py # UserProvider singleton
│ │ └── tokens.py # Token counting utilities (litellm)
│ │
│ └── domains/ # Feature domains
│ ├── router.py # Root router (composes all)
│ ├── health/ # Health endpoints
│ ├── auth/ # Authentication
│ ├── conversations/ # Multi-turn conversation memory
│ │ ├── models.py # Conversation, Message SQLAlchemy models
│ │ ├── schemas.py # Pydantic request/response models
│ │ ├── service.py # ConversationService business logic
│ │ ├── router.py # REST API endpoints
│ │ └── summarize.py # Context summarization logic
│ ├── agents/ # Agent orchestration
│ │ ├── explore/ # Codebase navigation
│ │ ├── plan/ # Implementation design
@@ -201,6 +213,10 @@ All settings via environment variables or `.env`:
| ALLOWED_PATHS | [] | Paths accessible to tools |
| SESSION_TTL_HOURS | 24 | Session expiry |
| MAX_CONTEXT_TOKENS | 128000 | Max context window |
| DATABASE_URL | sqlite+aiosqlite:///./webber.db | Database connection URL |
| SUMMARIZATION_THRESHOLD | 0.8 | Summarize at N% of max tokens |
| SUMMARIZATION_TARGET_TOKENS | 500 | Target summary size |
| KEEP_RECENT_MESSAGES | 6 | Messages to keep unsummarized |
---
@@ -250,6 +266,53 @@ Tools are sandboxed operations agents can invoke:
---
## Database Layer
SQLAlchemy 2.0 async with lazy initialization pattern.
### Supported Databases
- **Development**: SQLite via `aiosqlite`
- **Production**: PostgreSQL via `asyncpg`
### Pattern
```python
from src.db import get_session
from sqlalchemy.ext.asyncio import AsyncSession
async def my_endpoint(session: AsyncSession = Depends(get_session)):
# Session auto-commits on success, rollbacks on exception
result = await session.execute(query)
```
Tables are created lazily on first `get_session()` call.
---
## Conversation API
Multi-turn conversation memory with automatic context summarization.
### Endpoints
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/conversations/` | POST | Create new conversation |
| `/conversations/` | GET | List user's conversations |
| `/conversations/{id}` | GET | Get conversation with history |
| `/conversations/{id}/messages` | POST | Add message, triggers agent |
| `/conversations/{id}` | DELETE | Delete conversation |
### Models
- **Conversation**: User session with agent type, working directory
- **Message**: Individual messages with role, content, token count
### Context Summarization
When total tokens exceed 80% of `MAX_CONTEXT_TOKENS`:
1. Keep last 6 messages intact
2. Summarize older messages into a single summary message
3. Mark old messages as summarized (soft delete)
---
## Authentication Flow
1. Client sends `X-API-Key` header