feat(core-ai): implement Phase 1 of AI performance metrics system
Adds comprehensive in-memory metrics collection for monitoring AI agent
performance, tool execution, and system behavior.
New Components:
- src/metrics/collector.py: Thread-safe MetricsCollector class
- Tracks agent requests (response times, errors, concurrency)
- Tracks tool execution (calls, success/failure, durations)
- Tracks memory system (tier1/tier2 hits, consolidations)
- Calculates percentiles (p50, p95, p99) for performance analysis
- Sliding window retention (1h detailed, 24h aggregated)
- src/metrics/decorators.py: Automatic instrumentation decorators
- @track_tool_execution: Auto-tracks tool calls with metrics
- @track_duration: Generic duration tracking decorator
- src/metrics/__init__.py: Module exports
API Endpoints:
- GET /metrics: Comprehensive performance metrics snapshot
- GET /metrics/errors: Recent request errors with timestamps
- GET /metrics/tool-failures: Recent tool execution failures
- POST /metrics/reset: Clear all metrics (admin endpoint)
Instrumentation:
- Enhanced main.py chat handlers with metrics tracking
- Modified tools/registry.py log_tool_call to track execution metrics
- All metrics recorded with proper error handling and context
Features:
- Thread-safe with threading.Lock for concurrent requests
- No database dependencies (in-memory only)
- Automatic cleanup of old data (sliding windows)
- Detailed statistics: avg, p50, p95, p99 response times
- Per-user tracking and request attribution
- Tool success rates and performance analysis
Tested and validated:
- All endpoints responding correctly
- Request metrics collected successfully
- Response time percentiles calculated correctly
- User tracking functional
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
jpmschweitzerandClaude
632b20febe