Core-API service moved to git.schweitz.net/jpmschweitzer/core-api
- Source code removed from this repository
- Stack file updated to use container image from Gitea registry
- Documentation updated with external service reference
- Removed obsolete docs/services/core-api.md (docs now in core-api repo)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Scheduler service moved to git.schweitz.net/jpmschweitzer/scheduler
- Source code removed from this repository
- Stack file updated to use container image from Gitea registry
- Added Gitea Actions runner to gitea stack for CI/CD
- Documentation updated with external service reference
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- OLLAMA_NUM_PARALLEL=1: Process requests sequentially to avoid batch
overflow panics when multiple services call embeddings concurrently
- OLLAMA_KEEP_ALIVE=-1: Keep models loaded in VRAM indefinitely
- OLLAMA_MAX_LOADED_MODELS=2: Allow both mistral-nemo and nomic-embed-text
to stay loaded simultaneously
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
GraphQL requires Int type, but page_id might be passed as string.
Explicit int() conversion prevents 400 Bad Request errors.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Previously called /api/tags twice (once for server check, once for
model check). Now reuses response from single call.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix WikiChangeListener: use get_page() instead of non-existent get_page_by_id()
- Fix webhooks: use delete_page_chunks() instead of non-existent delete_page_vectors()
- Fix Neo4j: replace deprecated id() with elementId() in entity query
- Change some warnings to debug level to reduce log noise
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add fuzzy_match_entity_to_document with confidence scoring
- Filter self-referential links (entity linking to current page)
- Fix path cleaning to preserve full wiki paths
- Add longest-first matching to prevent partial matches
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace WIKIJS_API_KEY with WIKIJS_USERNAME/PASSWORD and add
WIKIJS_DB_PASSWORD for PostgreSQL LISTEN/NOTIFY connection.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add WikiChangeListener service for PostgreSQL notifications
- Add webhooks router for HTTP webhook fallback
- Add asyncpg dependency for PostgreSQL async support
- Include setup scripts and documentation for triggers
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
search_pages with empty query uses stale search index. list_all_pages
fetches actual page list from GraphQL for reliable bulk re-indexing.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Set d.tags = \$tags in MERGE query to prevent Neo4j warnings about
missing tags property in related documents query.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Pass ingestion_service to ConsolidationService constructor so pages
created/updated during consolidation are properly indexed in Qdrant
and Neo4j.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fetch existing wiki taxonomy structure before LLM analysis
- Include existing paths in prompt to prefer existing categories
- Add _format_taxonomy_for_prompt helper
- Mark searches as processed even when skipped/errored (prevents buildup)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- list_all_pages: fetch all pages with path prefix filter
- get_taxonomy_structure: extract category/subcategory structure for
taxonomy-aware classification
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Use pages.singleByPath(path, locale) GraphQL query to get page ID
during initialization instead of searching through page list.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fix create_entity_mentions to correctly count only newly created relationships,
not all relationships processed by MERGE.
**Problem**: count(r) was returning ALL relationships touched by MERGE (both
created and matched), breaking idempotency tests.
**Solution**: Use temporary flag 'just_created' set only ON CREATE, filter to
those relationships, count them, then remove the flag. This ensures the count
only includes new relationships.
Now properly returns:
- N on first run (N new relationships created)
- 0 on subsequent runs (no new relationships, all already exist)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Two critical bug fixes for entity linking:
1. **Path cleaning**: Replace hardcoded "users/jpmschweitzer/" with regex pattern
to handle any user namespace. Now properly cleans paths for all users.
2. **Longest-first matching**: Move protected_ranges computation inside entity loop
to recompute after each entity is processed. Prevents nested links like
[[Machine](/machine) Learning](/ml) when processing multi-word entities.
These fixes ensure entity linking works correctly across all users and prevents
nested markdown links when entity names overlap.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Cleanup Scripts:
- cleanup_graph.py - Clean up duplicate entities and orphaned nodes
- cleanup_wiki.py - Remove orphaned pages and fix broken links
- Utility scripts for database maintenance
- Not part of main application, run manually
Ingestion Router:
- POST /ingest/page - Index single page
- POST /ingest/batch - Batch indexing
- POST /ingest/all - Full knowledge base refresh
- Support vector and graph ingestion
Vector Router:
- POST /vector/search - Semantic search via Qdrant
- GET /vector/stats - Collection statistics
- DELETE /vector/page - Remove page embeddings
Wiki Router:
- GET /wiki/pages - List wiki pages
- GET /wiki/pages/{id} - Get page details
- PUT /wiki/pages/{id} - Update page
- POST /wiki/search - Search wiki content
- Full Wiki.js GraphQL integration
Tools Router:
- Expose library-desk capabilities as AI tool endpoints
- Support function calling for LLM agents
- Query wiki pages, search knowledge base
- Access graph entities and relationships
Tools Models:
- ToolDefinition for function schemas
- ToolParameter specifications
- ToolResponse format
- OpenAI function calling compatible
Graph Router:
- GET /graph/entities - List all entities for user
- GET /graph/relationships - Query entity relationships
- GET /graph/search - Search entities by name/type
- GET /graph/stats - Knowledge graph statistics
Graph Models:
- Entity, Relationship models
- GraphStats for analytics
- SearchFilters for queries
- Support multi-tenancy with user isolation
Configuration:
- Add HybridRAG settings (reranker model, search limits)
- Change Wiki.js auth from API key to username/password
- Configure vector, graph, and web search limits
Multi-tenancy:
- Add get_neo4j_user_base_label() for entity node labeling
- Support title-cased labels following Neo4j conventions
- Maintain namespace isolation for entities vs documents
Ollama Client:
- Improve model checking to handle :latest tag variants
- Match models with or without explicit tag
Qdrant Client:
- Add collection_exists() method for checking collection presence
- Refactor ensure_collection() to accept collection name directly
- Better separation of concerns
SearXNG Client:
- Add health_check() method for service monitoring
- Simple endpoint check without full search
- Used by health check endpoint
Vector Service:
- Manage document embeddings in Qdrant
- Update vectors from wiki pages
- Handle chunking and embedding generation
- Support force refresh and incremental updates
Vector Models:
- VectorSearchResult for search responses
- VectorUpdateSummary for indexing metrics
- Track chunks created/deleted
Used by ingestion_service for page embedding
- Add IngestionResult model for single page ingestion
- Add BatchIngestionResult for batch operations
- Track vector chunks, graph entities, and relationships
- Include processing time metrics
Used by ingestion_service for page indexing
- Add WikiPage model for page data
- Add WikiPageUpdate model for partial updates
- Add field validators for tags and descriptions
- Support optional fields for flexible updates
Used by wiki_service and entity_linking router
Main App:
- Mount /static directory for serving Wiki.js integration scripts
- Register entity_linking router
- Refactor API key verification to dependencies module
Dependencies:
- Add service factory functions for all services
- Add get_wiki_service() for wiki operations
- Add get_graph_service() for entity operations
- Add get_ingestion_service() for auto entity linking
- Improve health check for SearXNG
- Add combined integration script with both re-index and entity linking
- Add standalone entity linking button
- Add standalone re-index button
- Auto-detect Library Desk URL from script tag
- Support both toolbar and floating button positions
- Show real-time status updates and notifications
- Auto-reload page after successful entity linking
Usage: Inject via Wiki.js Code Injection settings
<script src="http://IP:8089/static/wikijs-integration.js"></script>
Graph Service:
- Add get_all_entities() to retrieve entities with wiki page paths
- Add create_entity_mentions() for MENTIONS relationship creation
- Support entity-to-document linking via title matching
Ingestion Service:
- Add _link_existing_entities() for automatic entity linking
- Auto-link entities during page ingestion
- Support skip_entity_linking parameter for granular control
- Add is_published parameter to WikiJS client update_page() method
- Update wiki_service to always pass is_published=True
- Prevents pages from being unpublished during entity linking updates
- Important for internal wikis where all pages should remain published
- Add /entity-linking/link-page endpoint to find and link entity mentions
- Creates both MENTIONS relationships in Neo4j and hyperlinks in wiki content
- Supports automatic re-indexing after linking
- Returns detailed statistics on entities found and linked
- Protects existing markdown links from being nested
- Idempotent: safe to run multiple times
Implements dual entity linking:
1. Graph relationships (MENTIONS) for knowledge graph traversal
2. Wiki content hyperlinks for user navigation
Add rest_api_executor as a universal executor that can call any REST API
endpoint across the system. This provides a standard way to trigger HTTP
operations from scheduled tasks.
Features:
- All HTTP methods: GET, POST, PUT, DELETE, PATCH
- Authentication: Bearer token, Basic auth, API key
- Environment variable substitution: ${VAR_NAME}
- JSONPath response extraction
- Configurable timeouts and SSL verification
- Sensitive data redaction in logs
- Custom headers support
This executor enables scheduler to call any service endpoint (Library Desk,
Core API, external webhooks) without needing service-specific executors.
Example usage:
{
"executor": "rest_api_executor",
"config": {
"url": "http://library-desk:8089/consolidate/knowledge",
"method": "POST",
"payload": {"process_limit": 10},
"auth": {"type": "bearer", "token": "${API_KEY}"}
}
}
The Qdrant client API changed from search() to query_points().
Updated both search() and find_similar_chunks() methods.
All integration tests now passing: 14/14 ✓
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implements comprehensive service client layer for Library Desk API to support
Librarian AI agent with multi-tenant knowledge management across Neo4j, Qdrant,
Wiki.js, SearXNG, and Ollama.
## Service Clients (src/clients/)
- Neo4j async client with connection pooling and user-scoped labels
- Qdrant vector store with collection-per-user multi-tenancy
- Wiki.js GraphQL API client for page/dossier management
- SearXNG client for web search integration
- Ollama client for text embeddings (nomic-embed-text)
## Core Infrastructure (src/core/)
- Multi-tenancy helpers for user namespace management
- Wiki.js: path-based namespaces (/users/{user})
- Neo4j: user-specific labels (User_{User}_Document)
- Qdrant: collection per user (library_desk_{user})
- Dependency injection with FastAPI Depends and @lru_cache singletons
- Lifecycle management (startup/shutdown) for all service connections
## Background Jobs (src/jobs/)
- Redis-based job manager for long-running operations
- Job status tracking with 24-hour TTL
- Support for queued, processing, completed, failed states
## Configuration
- Updated config.py with Redis DB 4 for library-desk jobs
- Updated docker-compose.yml: REDIS_DB from 2 to 4
- Added pytest and pytest-asyncio to requirements.txt
## Testing
- Unit tests: 25/25 passed (multi-tenancy helpers)
- Integration tests: 12/12 passed (all services verified)
- Neo4j connection and CRUD operations
- Qdrant vector operations with 768-dim embeddings
- Wiki.js GraphQL queries
- SearXNG web search
- Job Manager with Redis
- Dependency injection lifecycle
- pytest.ini configuration with asyncio support
## Health Monitoring
- Real-time service health checks via /health endpoint
- Connection status for all 5 external services
- Graceful degradation for partial service availability
## Architecture
- Follows async/await pattern throughout
- Connection pooling for Neo4j (singleton driver)
- HTTP client lifecycle management (httpx)
- Multi-tenancy enforced at client layer
- Default user: jpmschweitzer
Files changed: 26 files
- 5 new service clients (~1500 lines)
- 2 core modules (~500 lines)
- 1 job manager (~350 lines)
- 3 test files with 37 test cases
- Updated main.py with lifecycle hooks
All services tested and operational. Ready for Phase 2 (routers/services).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add pytest coverage output files to project-level .gitignore:
- coverage.json
- htmlcov/
- .pytest_cache/
These are generated artifacts from pytest-cov and should not be committed.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add complete documentation for The Scheduler service.
README.md (500+ lines):
- Architecture overview with ASCII diagram
- Quick start guide
- Complete API reference with curl examples
- Task scheduling patterns and examples
- Priority system documentation
- Built-in executors documentation (example, doc_sync, config_backup)
- Custom executor development guide
- Current tasks table
- Database schema documentation
- Testing guide with coverage metrics
- Development and debugging information
- Monitoring and troubleshooting
- Security and performance notes
- API reference with response codes and filtering
CHANGELOG.md:
- Initial v1.0.0 release documentation
- Core features and architecture
- REST API endpoints
- Task executors and pre-configured tasks
- Testing infrastructure and metrics
- Technical details and dependencies
- Coverage metrics breakdown
- Planned features for future releases
Documentation covers:
- All API endpoints and authentication
- Scheduling examples (every minute, daily, monthly, etc.)
- Priority ranges and usage
- Executor configuration
- Test database setup
- Docker stack configuration
- Common issues and solutions
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add hybrid APScheduler + PostgreSQL-based task scheduling system with minute-based execution and priority queue.
Core features:
- Minute-based scheduling with cron-like patterns (-1 = wildcard)
- Priority queue system (1-100, lower = higher priority)
- Concurrent execution (max 5 tasks simultaneously)
- Full REST API for task management (CRUD operations)
- Task execution tracking with audit trail
- API key authentication (Bearer token)
- Health checks and system statistics
Architecture:
- APScheduler runs every minute
- Queries PostgreSQL for tasks scheduled for current minute
- Executes tasks concurrently by priority
- Records execution history in database
Database schema:
- scheduled_tasks: Task definitions/templates
- task_executions: Individual execution records
Technical stack:
- FastAPI for REST API
- APScheduler for scheduling
- PostgreSQL for persistence
- Pydantic for configuration
Endpoints:
- POST/GET/PUT/DELETE /tasks - Task management
- POST /tasks/{name}/trigger - Manual execution
- GET /executions - Execution history
- GET /health - Health check
- GET /stats - System statistics
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- Fixed: Change `result.data` to `result.output` (correct PydanticAI API)
- Increased analysis_timeout from 3s to 10s (mistral-nemo needs more time)
**Status:** Steward now initializes correctly but there's a remaining issue
with the async generator merging logic in two_stage_agent.py causing
requests to hang. This needs further investigation.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Removes the "simple" fallback model and renames "pydantic" to "Tatlock"
to match the agent's British butler persona.
Changes:
- /models endpoint now returns only "Tatlock" model
- Removed "simple" model from advertised models
- Updated default model name from "pydantic" to "Tatlock"
- Updated health endpoint to show "Tatlock" agent status
- Added description: "PydanticAI agent with full tool support - your British butler assistant"
Benefits:
- Clearer model naming that matches agent persona
- Simplified model selection in Open WebUI
- Eliminates confusion between pydantic/simple models
- Consistent branding with Tatlock character
Open WebUI will now show only "Tatlock" as an available model, which uses
the full PydanticAI agent with tool calling capabilities.
Tested:
✅ /models endpoint returns only Tatlock
✅ Health check shows Tatlock as default agent
✅ Chat completions work with model="Tatlock"
✅ Tatlock persona responds correctly
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Moved container documentation from docs/reference/ to project root for better
LLM accessibility. Added comprehensive quick-reference table at the top with:
- All 22 services with ports, URLs, and status
- External domain mappings (11 SSL domains)
- Complete Core API infrastructure management endpoints
Changes:
- New location: /CONTAINERS.md (was docs/reference/CONTAINERS.md)
- Added quick reference table with all ports and URLs
- Added infrastructure management API documentation
- Listed all external domains with SSL status
- Current service status indicators
- Removed old location to avoid duplication
Result: Single-file reference for all infrastructure info, easily found by LLMs
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
After PydanticAI migration (Dec 3), AI agent functionality was moved to
separate core-ai service. Health check was still trying to validate agent
in core-api, causing persistent unhealthy status (503 errors).
Changes:
- Remove ADK agent import attempts (no longer exists in core-api)
- Update /health/full to only check Ollama connectivity
- Update diagnostics endpoint with service separation notes
- Clarify that core-api is infrastructure/tools API only
Result: Container now reports healthy status consistently (200 OK).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add Scenario 6 to validate OpenAPI tool discovery and infrastructure integration.
Changes:
- Add test_scenario6_dns_lookup test
- Tests DNS lookup via core-api discovered tool
- Verifies OpenAPI discovery mechanism works
- Query: "What are the A records for github.com?"
- Performance target: < 15s
- Update test scenario list in run_full_quality_check()
- Document new scenario in QUALITY_TESTS.md
Purpose:
Validates that core-ai can discover and use infrastructure tools
from core-api via OpenAPI spec. DNS tool serves as example of
dynamic tool integration without manual registration.
Tool Discovery Chain:
1. core-api exposes /tools/dns/lookup endpoint (dnspython)
2. core-api publishes endpoint in /openapi.json
3. core-ai discovers tool via OpenAPI discovery
4. Agent can use tool as core-api__dns_lookup_tools_dns_lookup_post
Note: Agent currently prefers web_search for DNS queries, but
explicit instruction to use the DNS tool works. Tool naming
optimization can be addressed in future improvements.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implement automatic tool discovery from OpenAPI specifications, enabling
core-ai to dynamically use infrastructure management endpoints without
manual tool definitions.
Changes:
- Add OpenAPIToolDiscovery class for spec parsing and tool generation
- Fetches OpenAPI specs from configurable endpoints
- Generates executable tool functions from API operations
- Creates properly formatted tool schemas for agent use
- Async HTTP client for endpoint execution
- Update tool registry to support OpenAPI tools
- Optional include_openapi parameter in get_all_tools()
- Async loading of dynamic tools
- Merges local and OpenAPI tools seamlessly
- Add OpenAPI configuration settings
- openapi_endpoints: Comma-separated spec URLs
- openapi_enabled: Feature flag for tool discovery
- Default: http://core-api:8083/openapi.json
Architecture:
- Core tools (local.py): Always available essentials (web_search, calculate)
- OpenAPI tools: Infrastructure/automation from core-api dynamically discovered
Benefits:
- Auto-discovers new endpoints as core-api evolves
- No manual tool definition needed for REST APIs
- Maintains single source of truth (OpenAPI spec)
- Enables agent to manage infrastructure via discovered tools
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implement native Ollama agent that bypasses OpenAI-compatible API and uses
Ollama's native /api/chat endpoint for improved tool calling reliability.
Changes:
- Add OllamaNativeAgent class with native tool calling support
- Direct integration with Ollama /api/chat endpoint
- Better tool calling reliability vs OpenAI-compatible API
- Async streaming support
- Tool result handling and multi-turn conversations
- Set OllamaNativeAgent as default agent (replacing PydanticAI)
- Add test endpoint for Ollama tool verification
- Update health check to report ollama-native availability
- Add ollama>=0.4.0 to requirements for native library support
Technical Details:
- Uses Ollama's native tool format (not OpenAI functions)
- Handles tool execution and response synthesis
- Maintains conversation context across tool calls
- Model: mistral-nemo:latest (primary reasoning model)
Motivation:
PydanticAI uses Ollama's OpenAI-compatible endpoint which has less
reliable tool calling. The native API provides better tool support
and more consistent behavior.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add automated test suite for regression detection and performance tracking
of the core-ai agent behavior across code changes.
Changes:
- Add test_ai_flow_quality.py with 5 core test scenarios
- Simple knowledge queries (no tools)
- Web search integration
- Mathematical calculations
- Date/time operations
- Multi-tool reasoning tasks
- Add QUALITY_TESTS.md documentation
- Usage guide and test descriptions
- Baseline establishment workflow
- Model benchmarking procedures
- Troubleshooting guide
- Add performance baseline tests
- Add regression detection tests
- Generate text and JSON reports with git tagging
- Update .gitignore to exclude generated test reports
- Update CHANGELOG.md with test suite details
Baseline Results:
- 4/5 tests passing (80% success rate)
- Average response time: 2-8s per query
- Agent: OllamaNativeAgent with PydanticAI
- Model: mistral-nemo:latest
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Removed all AI/LLM functionality from core-api as it has been
migrated to the dedicated core-ai service.
Deleted:
- src/controllers/ai_controller.py (chat completions, models, conversations)
- src/agent/ (orchestrator, tools, prompts, streaming)
- src/memory/ (manager, qdrant, buffer, schemas)
- src/api/v1/ (chat, conversations, models, schemas)
- tests/test_memory_*.py (3 test files)
Removed dependencies:
- google-adk, litellm, google-cloud-aiplatform
- qdrant-client
Kept:
- tools_controller.py (web scraper for core-ai REST calls)
- infrastructure_controller.py
- health_controller.py
- static_controller.py
core-api is now purely for infrastructure management.
All AI operations are handled by core-ai service.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Model Change:
- Switch from mistral-tools:7b to mistral-nemo:latest
- Reason: mistral-tools:7b was describing tools instead of calling them
- mistral-nemo:latest properly executes tool calls (verified with tests)
- Tool calling success rate: ~95% with mistral-nemo vs ~0% with mistral-tools
Hybrid Date/Tool Approach (Industry Best Practice):
- Inject current date into system prompt: "Today is {day}, {date}"
- Provides general temporal awareness without tool calls
- Refreshed on each agent initialization (no stale data)
- Efficient for casual date references ("Is it the weekend?")
- Keep get_current_time(timezone) tool for precise queries
- Accurate real-time data for specific time queries
- Works correctly in multi-turn conversations
- No confusion from static timestamps
Prompt Optimization:
- Simplified pydantic_agent prompt (removed verbose edge cases)
- More generic and token-efficient
- Added explicit instruction: "For specific time queries, use get_current_time()"
- Emphasizes MUST use tools for accurate data (prevents hallucination)
Research-Backed Decision:
Based on best practices from:
- Anthropic: Claude web interface uses date injection
- OpenAI/LangChain: Static timestamps cause confusion in long conversations
- Industry consensus: Tools for dynamic data, prompts for static context
Results:
- All 58 tests passing
- Tool calling working reliably
- No more hallucinated time answers
- Multi-turn conversation safe
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Problem:
- Model was hallucinating time answers (e.g., wrong Amsterdam time)
- get_current_time() only returned UTC
- No way to query time in specific timezones
Solution:
- Enhanced get_current_time(timezone) to support any IANA timezone
- Added pytz>=2025.2 dependency for timezone handling
- Returns formatted time with timezone info: "2025-11-30 16:25:55 CET"
- Supports timezones: Europe/Amsterdam, America/New_York, Asia/Tokyo, etc.
Testing:
- Added test_timezone.py: 6 comprehensive timezone tests
- UTC, Amsterdam, New York, Tokyo timezone queries
- Invalid timezone error handling
- Timezone offset correctness validation
- Added test_agent_timezone.py: 3 integration tests
- Agent tool usage for timezone queries
- Agent behavior with/without tools
- Multi-timezone query handling
All new tests passing. Tool verified working across multiple timezones.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Updated the default agent model from gemma2:9b-instruct-q5_K_M to mistral-tools:7b
for improved tool calling support.
Testing Results:
- All 49 tests pass successfully
- Environment tests: 5/5 ✓
- LiteLLM raw tests: 4/4 ✓
- Message format tests: 6/6 ✓
- Agent tests: 7/7 ✓
- API tests: 7/7 ✓
- PydanticAI setup tests: 7/7 ✓
- PydanticAI tools tests: 6/6 ✓
- PydanticAI API tests: 7/7 ✓
The model has been verified to work correctly with:
- Simple completions
- Streaming responses
- Tool calling (calculator, date tools, etc.)
- PydanticAI agent framework
- All API endpoints
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This commit completes the cleanup of Google ADK references after migrating to PydanticAI.
Changes:
- Removed ADK agent implementation (adk_agent.py)
- Removed ADK test files (test_06, test_07, test_10)
- Removed ADK diagnostic files
- Updated config to use pydantic_system_prompt_variant instead of adk_system_prompt_variant
- Updated prompts.py to rename adk_agent to pydantic_agent
- Updated tool registry and tools.py docstrings to remove ADK references
- Added new comprehensive PydanticAI tests (test_06, test_07, test_10)
- Marked legacy ADK functions as deprecated for backwards compatibility
The codebase is now clean and stable with PydanticAI as the primary agent framework.
Docker container builds successfully with no ADK import errors.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Introduces a comprehensive, multi-tiered memory system to provide conversation history and context for the AI agent. This lays the foundation for more stateful and intelligent interactions.
Key components of this implementation:
- **Multi-Tiered Memory Architecture:**
- **Tier 1 (Working Memory):** A fast, in-memory buffer (`ConversationBufferMemory`) that holds the most recent turns of a conversation for immediate access.
- **Tier 3 (Long-Term Memory):** A persistent, semantic search-based memory store using Qdrant (`QdrantConversationMemory`). It stores all conversation turns as vector embeddings, enabling long-term recall and similarity search.
- **Qdrant Integration:**
- The `qdrant-client` is added to manage collections and perform vector search operations.
- Each user is assigned a dedicated Qdrant collection for multi-tenancy.
- **Ollama Embedding Client:**
- A new `OllamaEmbeddingClient` generates text embeddings via the Ollama API, replacing the need for local sentence-transformer models. This significantly reduces the service's dependency footprint.
- **Configuration and Stack Updates:**
- The `config.py` and `core-ai.yml` stack file are updated with new settings for enabling memory, configuring Qdrant, and specifying the embedding model.
- **Utility and Schema Additions:**
- New Pydantic schemas (`memory/schemas.py`) define the data structures for conversation turns and memory management.
- Utility functions (`utils.py`) are added for user ID sanitization and collection naming.
This feature enhances the agent's capabilities by allowing it to maintain context across multiple turns and sessions, leading to more coherent and relevant responses.
Major Changes:
- Replace Google ADK with PydanticAI framework for agent orchestration
- Implement OpenAI-compatible API endpoint for Ollama integration
- Fix streaming response to send deltas instead of cumulative text
- Add /chat/completions route alias for Open-WebUI compatibility
- Enable tool calling with 5 local tools (calculate, date/time utilities)
Architecture:
- Core-AI service: Standalone Python service with PydanticAI agent
- PydanticAI: Uses OpenAI-compatible Ollama API at /v1 endpoint
- Tool Registry: Shared tool system between core-ai and core-api
- Streaming: Fixed async context issues and delta calculation
Verified Working:
✅ Chat completion (streaming & non-streaming)
✅ Tool calling with mistral-nemo and mistral-tools models
✅ Open-WebUI integration via core-ai:8086
✅ 5 tools: calculate, get_current_time, get_current_date, calculate_date_difference, add_days_to_date
✅ Proper streaming deltas (no repetition)
Technical Details:
- PydanticAI 1.25.0+ with full Ollama support
- Async context manager issue resolved via chunk collection
- Delta calculation: chunk[len(previous):] to extract new content only
- Routes: /v1/chat/completions and /chat/completions (Open-WebUI compat)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Phase completion and enhancement updates:
## Documentation Added
- Phase 2 completion: Memory system implementation details
- Phase 3 completion: Research capabilities and tool integration
- Session documentation: Model testing, VRAM optimization analysis
- Test results: Comprehensive prompt testing (v1_verbose: 87/100)
- Tool logging implementation guide
## System Prompts
- Added prompts.py with 7 tested variants for A/B testing
- v1_verbose, v2_concise, v3_imperative, v4_minimal, etc.
- Comprehensive testing results for each variant
- Production-ready prompt selection guidance
## Memory System Enhancements
- Multi-tenancy support: Added user_id parameter throughout
- System message filtering: Don't store system messages in history
- Improved conversation turn tracking with user isolation
- Enhanced memory manager for better multi-user support
## AI Controller Improvements
- Better memory integration with user_id support
- Enhanced error handling for memory operations
- Improved token tracking for usage monitoring
- Skip system message storage (part of agent state)
## Portainer Client
- Comprehensive API client (148 lines)
- Stack management and service monitoring
- Container operations with full error handling
- Async support for all operations
## Architecture Documentation
- Updated agent flow diagrams for ADK architecture
- Enhanced core-api README with current setup
- Updated Docker compose stack configuration
- Complete testing and validation documentation
Major architectural changes and improvements:
## ADK Framework Migration (v0.10.0)
- Migrated from LangChain/LangGraph to Google ADK 1.3.0 with LiteLLM 1.80.5
- Improved tool calling reliability with local Ollama models
- Converted all 10 tools to ADK async generator format
- Updated streaming pipeline for ADK event system
- Enhanced error handling and agent initialization
## Model Optimization
- Switched from gemma3:12b (10GB VRAM) to gemma3:4b (4.8GB VRAM)
- Reduced VRAM usage from 91% to 43% (5.4GB freed)
- Optimized for production stability with memory headroom
## Health Check System Overhaul
- Optimized /health/full: 6ms response (was 30s+)
- Added model verification: confirms configured model is available
- New /health/diagnostics endpoint with optional deep testing
- Added currently loaded models tracking
- Clear emoji status indicators (✅/❌/⚠️)
- Fixed AGENT_AVAILABLE flag export for proper health reporting
## Ollama Client Enhancements
- Added list_models() method for model inventory
- Enhanced model verification in health checks
- Better error handling and reporting
## Documentation Updates
- Updated STATUS.md to v0.10.0-adk-migration
- Comprehensive CHANGELOG.md entry with migration details
- Updated PLANS.md showing Phase 4 complete
- Updated ai-orchestrator-plan.md with ADK status
- Added MIGRATION_PLAN_LANGCHAIN_TO_ADK.md
- Added ADK_Ollama_Research.md with implementation analysis
## Technical Details
- 10 tools: 7 infrastructure + 2 research + 1 response tool
- Framework: Google ADK with UnifiedAgent pattern
- System prompt: v7_adk_best_practice
- Container health: Now passing Docker healthchecks
- Response times: Simple queries ~0.3-1s, Research ~4-7s
jpmschweitzer
