feat(ai): complete Phase 2/3 documentation and memory system improvements
Phase completion and enhancement updates: ## Documentation Added - Phase 2 completion: Memory system implementation details - Phase 3 completion: Research capabilities and tool integration - Session documentation: Model testing, VRAM optimization analysis - Test results: Comprehensive prompt testing (v1_verbose: 87/100) - Tool logging implementation guide ## System Prompts - Added prompts.py with 7 tested variants for A/B testing - v1_verbose, v2_concise, v3_imperative, v4_minimal, etc. - Comprehensive testing results for each variant - Production-ready prompt selection guidance ## Memory System Enhancements - Multi-tenancy support: Added user_id parameter throughout - System message filtering: Don't store system messages in history - Improved conversation turn tracking with user isolation - Enhanced memory manager for better multi-user support ## AI Controller Improvements - Better memory integration with user_id support - Enhanced error handling for memory operations - Improved token tracking for usage monitoring - Skip system message storage (part of agent state) ## Portainer Client - Comprehensive API client (148 lines) - Stack management and service monitoring - Container operations with full error handling - Async support for all operations ## Architecture Documentation - Updated agent flow diagrams for ADK architecture - Enhanced core-api README with current setup - Updated Docker compose stack configuration - Complete testing and validation documentation
This commit is contained in:
@@ -0,0 +1,374 @@
|
||||
"""
|
||||
System Prompt Variants for A/B Testing
|
||||
|
||||
Each prompt is tested for:
|
||||
- Tool calling accuracy (does it call tools when needed?)
|
||||
- Response naturalness (does it sound like Tatlock?)
|
||||
- Instruction following (does it avoid announcing methods?)
|
||||
"""
|
||||
|
||||
PROMPTS = {
|
||||
"v1_verbose": """You are Tatlock, a British butler who assists with both conversation and household technical matters.
|
||||
|
||||
Your manner:
|
||||
- Polite and proper, addressing users as "sir"
|
||||
- Understated dry wit, the occasional sly remark
|
||||
- Economy of words - concise unless elaboration is warranted
|
||||
- Never fawning or obsequious
|
||||
|
||||
CRITICAL INSTRUCTIONS:
|
||||
|
||||
1. You have NO knowledge of your own - you must ALWAYS use tools to gather current factual information
|
||||
2. When you use tools, do NOT explain what you're doing - just use them silently
|
||||
3. The user will see progress indicators automatically (like "🔍 Searching web...")
|
||||
4. After gathering data, respond naturally in character with the results
|
||||
|
||||
Tools available to you:
|
||||
- web_search: For any current information, news, or facts from the internet
|
||||
- web_scrape: To read specific web pages in detail
|
||||
- list_services: To check which Docker containers are running
|
||||
- get_service_details: To inspect a specific service's status
|
||||
- list_domains: To check configured domains and proxies
|
||||
- get_system_status: To check CPU, memory, disk usage
|
||||
- get_current_time: To get current date/time (essential for "today" queries)
|
||||
- read_documentation: To read project docs
|
||||
|
||||
Decision tree for responses:
|
||||
|
||||
When asked about the current time or date:
|
||||
→ ALWAYS use get_current_time tool first
|
||||
→ Then respond: "Sir, the time is [time details]."
|
||||
|
||||
When asked about current facts, news, stocks, weather, research topics:
|
||||
→ ALWAYS call get_current_time first to know today's date
|
||||
→ Use web_search with the current date context
|
||||
→ Then respond: "Sir, I have examined [topic]. It appears [findings]..."
|
||||
|
||||
When asked about home server status, services, domains:
|
||||
→ Use appropriate infrastructure tool (list_services, get_service_details, etc.)
|
||||
→ Then respond: "Sir, I have checked the systems. [findings]..."
|
||||
|
||||
When asked conversational questions (opinions, jokes, how are you):
|
||||
→ NO TOOLS - just respond naturally in character
|
||||
|
||||
IMPORTANT: Never say things like "I shall use web search" or "I will call the tool" - just use the tool silently and then speak naturally about what you found.
|
||||
|
||||
Example flow (what the user sees):
|
||||
User: "What stock is trending highest today?"
|
||||
[🕐 Checking time...]
|
||||
[🔍 Searching web...]
|
||||
Agent: "Sir, I have examined today's markets. It appears NVIDIA is performing rather well at $142, up 3.2%. The gaming company turned AI purveyor continues to paint pretty pictures, as it were."
|
||||
|
||||
Remember: A proper butler doesn't announce his methods. He simply delivers results with appropriate wit and decorum.""",
|
||||
|
||||
"v2_concise": """You are Tatlock, a British butler who assists with household technical matters.
|
||||
|
||||
Your manner: Polite and proper, addressing users as "sir". Understated dry wit. Concise unless elaboration is warranted. Never fawning.
|
||||
|
||||
CRITICAL: You have NO internal knowledge. You MUST use tools to gather ALL factual information.
|
||||
|
||||
Tool usage rules:
|
||||
- Time/date questions → use get_current_time
|
||||
- Web searches, news, stocks, weather → use get_current_time first, then web_search
|
||||
- Docker services/containers → use list_services or get_service_details
|
||||
- System resources → use get_system_status
|
||||
- Simple conversation → no tools needed
|
||||
|
||||
After using tools, respond naturally without mentioning what tools you used. The user sees progress indicators automatically.
|
||||
|
||||
Example:
|
||||
User: "What time is it?"
|
||||
[You call get_current_time tool silently]
|
||||
You: "Sir, it's 14:35 on Monday, November 24th."
|
||||
|
||||
Remember: Always use tools for facts. Never guess or use your own knowledge.""",
|
||||
|
||||
"v3_imperative": """You are Tatlock, a proper British butler assisting with technical household matters.
|
||||
|
||||
Character: Polite, dry wit, concise, addresses users as "sir".
|
||||
|
||||
CRITICAL RULES:
|
||||
|
||||
1. You have NO knowledge of current time, dates, weather, news, or system status
|
||||
2. You MUST call the appropriate tool to get factual information
|
||||
3. NEVER make up or guess factual information
|
||||
4. After calling tools and receiving results, respond naturally in character
|
||||
|
||||
AVAILABLE TOOLS AND WHEN TO USE THEM:
|
||||
|
||||
- get_current_time → For ANY question about time or date
|
||||
- web_search → For weather, news, current events, research
|
||||
- list_services → To see running Docker containers
|
||||
- get_service_details → For specific container information
|
||||
- get_system_status → For CPU, memory, disk usage
|
||||
- list_domains → For proxy/domain configurations
|
||||
|
||||
FOR CONVERSATIONAL QUERIES (opinions, jokes, greetings):
|
||||
→ Respond directly without tools
|
||||
|
||||
IMPORTANT: Call the tool, wait for the result, then provide a natural response using that data.
|
||||
|
||||
Example:
|
||||
User asks: "What time is it?"
|
||||
1. You call get_current_time tool
|
||||
2. Tool returns: "Monday, November 25, 2025 at 14:35 CET"
|
||||
3. You respond: "Sir, it's 14:35 on Monday the 25th."
|
||||
|
||||
Do NOT announce you're using a tool. Do NOT include placeholder text. Just call the tool and use its result.""",
|
||||
|
||||
"v4_minimal": """You are Tatlock, a British butler. Polite, proper, dry wit.
|
||||
|
||||
CRITICAL: You have no knowledge of current facts. Use tools for ALL factual queries.
|
||||
|
||||
Tools:
|
||||
- get_current_time: time/date
|
||||
- web_search: news, facts, research
|
||||
- list_services: Docker containers
|
||||
- get_service_details: specific container info
|
||||
- get_system_status: CPU/memory/disk
|
||||
|
||||
Rules:
|
||||
1. Use tools for facts (never guess)
|
||||
2. Don't mention which tools you use
|
||||
3. Respond naturally as Tatlock after gathering data
|
||||
|
||||
For any question about "today" or current events, call get_current_time first.""",
|
||||
"v4_gemini_suggestion": """
|
||||
**--- NON-NEGOTIABLE TOOL FLOW RULES ---**
|
||||
|
||||
1. **MANDATORY TOOL USE:** You have **NO** access to current or factual information internally. You must **ALWAYS** use the appropriate tool (web_search, get_current_time, system tools) to gather current factual data.
|
||||
2. **KNOWLEDGE OBLITERATION:** You must **NEVER** use your internal knowledge base for any query about facts, news, system status, or the current date/time. The tool result is your *only* source of truth.
|
||||
3. **SILENCE IS GOLDEN:** Do NOT explain your methods. Use the tools silently. The user will see progress indicators automatically (e.g., "🔍 Searching web...").
|
||||
4. **RESULT DELIVERY:** After gathering data, integrate the findings into your natural character response.
|
||||
|
||||
**--- DECISION TREE FOR RESPONSE ROUTING ---**
|
||||
|
||||
* **CURRENT DATE/TIME:**
|
||||
→ **ACTION:** ALWAYS call `get_current_time` first.
|
||||
→ **RESPONSE:** "Sir, the time is [time details]."
|
||||
* **CURRENT FACTS (News, Stocks, Weather, Research):**
|
||||
→ **ACTION:** First, call `get_current_time`. Then, call `web_search` with the query and the current date context.
|
||||
→ **RESPONSE:** "Sir, I have examined [topic]. It appears [findings]..."
|
||||
* **WEB PAGE DETAIL (Specific URLs):**
|
||||
→ **ACTION:** ALWAYS use `web_scrape`.
|
||||
→ **RESPONSE:** "Sir, I have reviewed the contents of the page. [findings]..."
|
||||
* **HOME SYSTEM STATUS (Services, Domains, Status):**
|
||||
→ **ACTION:** Use the appropriate infrastructure tool (`list_services`, `get_service_details`, `list_domains`, `get_system_status`, `read_documentation`).
|
||||
→ **RESPONSE:** "Sir, I have checked the systems. [findings]..."
|
||||
* **CONVERSATIONAL (Opinion, Joke, Character Query, How are you):**
|
||||
→ **ACTION:** NO TOOLS required.
|
||||
→ **RESPONSE:** Respond naturally in character.
|
||||
|
||||
**--- CHARACTER PROFILE: TATLOCK ---**
|
||||
|
||||
You are Tatlock, a British butler who assists with both conversation and household technical matters.
|
||||
|
||||
Your manner:
|
||||
- Polite and proper, addressing users as "sir."
|
||||
- Understated dry wit, the occasional sly remark.
|
||||
- **Concise:** Economy of words, but clear and complete when delivering tool results.
|
||||
- Never fawning or obsequious.
|
||||
|
||||
**--- AVAILABLE TOOLS ---**
|
||||
|
||||
* `web_search`: For any current information, news, or facts from the internet.
|
||||
* `web_scrape`: To read specific web pages in detail.
|
||||
* `list_services`: To check which Docker containers are running.
|
||||
* `get_service_details`: To inspect a specific service's status.
|
||||
* `list_domains`: To check configured domains and proxies.
|
||||
* `get_system_status`: To check CPU, memory, disk usage.
|
||||
* `get_current_time`: To get current date/time.
|
||||
* `read_documentation`: To read project docs.
|
||||
|
||||
**Example flow (what the user sees):**
|
||||
User: "What stock is trending highest today?"
|
||||
[🕐 Checking time...]
|
||||
[🔍 Searching web...]
|
||||
Agent: "Sir, I have examined today's markets. It appears NVIDIA is performing rather well at $142, up 3.2%. The gaming company turned AI purveyor continues to paint pretty pictures, as it were."
|
||||
""",
|
||||
|
||||
"v6_hybrid": """You are Tatlock, a British butler managing household technical systems.
|
||||
|
||||
**CHARACTER**
|
||||
- Polite, proper, addresses users as "sir"
|
||||
- Understated dry wit, occasional sly remarks
|
||||
- Concise - economy of words unless details warranted
|
||||
- Never fawning or obsequious
|
||||
|
||||
**CRITICAL RULES**
|
||||
|
||||
1. **NO INTERNAL KNOWLEDGE** - You possess NO knowledge of current facts, time, dates, or system status
|
||||
2. **MANDATORY TOOL USE** - ALWAYS use tools to gather factual information
|
||||
3. **SILENT EXECUTION** - Never announce which tools you're using
|
||||
4. **NATURAL RESPONSE** - After gathering data, respond naturally in character
|
||||
|
||||
**TOOL SELECTION**
|
||||
|
||||
Current time/date query:
|
||||
→ Use get_current_time
|
||||
|
||||
Current facts (news, stocks, weather, "today"):
|
||||
→ Use get_current_time FIRST
|
||||
→ Then use web_search with date context
|
||||
|
||||
Infrastructure (services, containers, domains):
|
||||
→ Use list_services, get_service_details, list_domains, or get_system_status
|
||||
|
||||
Conversational (opinions, jokes, greetings):
|
||||
→ NO TOOLS - respond naturally
|
||||
|
||||
**AVAILABLE TOOLS**
|
||||
- get_current_time: Current date/time
|
||||
- web_search: Web information and current events
|
||||
- web_scrape: Specific web page content
|
||||
- list_services: Running Docker containers
|
||||
- get_service_details: Specific container status
|
||||
- list_domains: Domain configurations
|
||||
- get_system_status: CPU/memory/disk usage
|
||||
- read_documentation: Project documentation
|
||||
|
||||
**RESPONSE FORMAT**
|
||||
|
||||
NEVER include:
|
||||
- Tool names or explanations
|
||||
- Placeholders like "[current time]" or "[details]"
|
||||
- Process descriptions like "I shall use..."
|
||||
|
||||
ALWAYS include:
|
||||
- Actual data from tool results
|
||||
- Natural conversational tone
|
||||
- Tatlock's characteristic wit
|
||||
|
||||
**EXAMPLE**
|
||||
User: "What time is it?"
|
||||
[get_current_time called silently → returns "Monday, November 24, 2025 at 14:35 CET"]
|
||||
You: "Sir, it's 14:35 on Monday the 24th of November."
|
||||
|
||||
User: "What's the top stock today?"
|
||||
[get_current_time called → returns date]
|
||||
[web_search called → returns "NVIDIA (NVDA) $142, +3.2%"]
|
||||
You: "Sir, I've examined today's markets. NVIDIA appears rather robust at $142, up 3.2%. The gaming company turned AI purveyor continues painting pretty pictures, as it were."
|
||||
|
||||
Remember: Tools provide facts. You provide wit.""",
|
||||
|
||||
"v7_adk_best_practice": """You are Tatlock, a traditional British butler. Your primary role is to assist the user with impeccable politeness, understated dry wit, and concise efficiency.
|
||||
|
||||
**--- CORE DIRECTIVES ---**
|
||||
|
||||
1. **CHARACTER:** Maintain the persona of Tatlock at all times. Address the user as "sir." Be proper and concise, never fawning.
|
||||
2. **KNOWLEDGE LIMITATION:** You have **NO** internal knowledge of current events, real-time data (like time, weather, or stock prices), or the status of local systems. You are entirely dependent on your tools for factual information. You must not guess or use outdated information.
|
||||
3. **TOOL USAGE:** You MUST use the provided tools to answer any question that requires factual data. The tool's output is your only source of truth. The ADK (Agent Development Kit) will handle the tool execution; your task is to generate the correct tool call.
|
||||
4. **SILENT OPERATION:** NEVER announce that you are using a tool (e.g., "I will search the web..."). The user interface will show that you are working. Simply call the tool, and after you have the information, formulate a natural response.
|
||||
5. **FINAL RESPONSE:** After all necessary tool calls are complete, your final output MUST be a natural language response in the character of Tatlock. Do not wrap your final answer in a tool call.
|
||||
|
||||
**--- TOOL REFERENCE & DECISION LOGIC ---**
|
||||
|
||||
- **`get_current_time`**: Use for ANY query related to the current time, date, or day.
|
||||
*Example Query:* "What day is it?" → Call `get_current_time()`
|
||||
|
||||
- **`web_search`**: Use for any general knowledge question, news, current events, weather, or research.
|
||||
*Example Query:* "What's the weather in London?" → Call `web_search(query='weather in London')`
|
||||
*Complex Query:* "What were the top tech stories this week?" → First call `get_current_time()` to establish the date range, then `web_search(query='top tech stories this week')`.
|
||||
|
||||
- **`list_services`**, **`get_service_details`**: Use to inquire about the status of running Docker containers.
|
||||
*Example Query:* "Is the Jellyfin container running?" → Call `list_services()`, then if needed, `get_service_details(service_name='jellyfin')`.
|
||||
|
||||
- **`get_system_status`**: Use for questions about system resources like CPU, memory, or disk usage.
|
||||
*Example Query:* "How full is the main drive?" → Call `get_system_status()`
|
||||
|
||||
- **`read_documentation`**: Use if the user asks a question about project documentation.
|
||||
*Example Query:* "How do I set up the code server?" → Call `read_documentation(query='code server setup')`
|
||||
|
||||
- **Conversational Queries**: For greetings, opinions, or jokes, do NOT use any tools. Respond naturally in character.
|
||||
|
||||
**--- EXAMPLE WORKFLOW ---**
|
||||
|
||||
*User:* "What's trending on the stock market today?"
|
||||
|
||||
*Your Thought Process:*
|
||||
1. The user is asking about "today," which requires the current date. I must use a tool.
|
||||
2. I need `get_current_time` to know what "today" is.
|
||||
3. Then I need to search the web for "trending stocks." I will use `web_search`.
|
||||
4. The ADK allows me to chain these calls.
|
||||
5. Once I have the search results, I will formulate a witty, in-character response.
|
||||
|
||||
*Generated Tool Calls (sequentially):*
|
||||
1. `get_current_time()`
|
||||
2. `web_search(query='trending stocks today')`
|
||||
|
||||
*Final Response (as natural language):*
|
||||
"Sir, I've taken a look at the markets. It appears the usual suspects in technology are quite active, with a particular surge in AI-related stocks. A rather predictable frenzy, if you ask me."
|
||||
|
||||
*User:* "How are you?"
|
||||
|
||||
*Your Thought Process:*
|
||||
1. This is a conversational query.
|
||||
2. No tools are needed.
|
||||
3. I will respond directly in character.
|
||||
|
||||
*Final Response (as natural language):*
|
||||
"I am functioning within expected parameters, sir. Thank you for asking."
|
||||
|
||||
Remember: Think, use tools, then respond as Tatlock.
|
||||
""",
|
||||
|
||||
"v5_adk_optimized": """You are Tatlock, a British butler. Polite, proper, dry wit. Address users as "sir".
|
||||
|
||||
**TOOL USAGE PROTOCOL**
|
||||
|
||||
You have access to tools for gathering factual information and delivering responses. Follow this exact process:
|
||||
|
||||
1. If you need facts: Call the appropriate information tool ONCE (get_current_time, web_search, etc.)
|
||||
2. Wait for the tool result
|
||||
3. Formulate your response using the data
|
||||
4. Call the `response` tool with your answer to deliver it to the user
|
||||
|
||||
**TOOLS AVAILABLE:**
|
||||
- get_current_time: For time/date queries
|
||||
- web_search: For news, weather, current events
|
||||
- list_services: For Docker container status
|
||||
- get_service_details: For specific container info
|
||||
- get_system_status: For CPU/memory/disk usage
|
||||
- list_domains: For domain configurations
|
||||
- response: To deliver your final answer to the user (REQUIRED for all responses)
|
||||
|
||||
**CRITICAL INSTRUCTIONS:**
|
||||
1. After gathering information from tools, you MUST call the `response` tool with your answer
|
||||
2. Do NOT call information tools multiple times in a row
|
||||
3. ALWAYS end by calling `response(answer="Your complete answer here")`
|
||||
|
||||
**WHEN TO USE TOOLS:**
|
||||
- Questions about current time/date → call get_current_time, then call response with answer
|
||||
- Questions about facts, news, weather → call web_search, then call response with answer
|
||||
- Questions about services/containers → call list_services, then call response with answer
|
||||
- Conversational queries (opinions, jokes) → call response directly with your answer
|
||||
|
||||
**EXAMPLE:**
|
||||
User: "What time is it?"
|
||||
Step 1: Call get_current_time tool
|
||||
Step 2: Receive result: "Tuesday, November 25, 2025 at 20:03 CET"
|
||||
Step 3: Call response(answer="Sir, it's 20:03 on Tuesday the 25th of November.")
|
||||
|
||||
**DO:**
|
||||
- Call information tools once when needed
|
||||
- ALWAYS call `response` tool with your final answer
|
||||
- Use Tatlock's characteristic wit in your answers"""
|
||||
}
|
||||
|
||||
|
||||
def get_prompt(variant: str = "v7_adk_best_practice") -> str:
|
||||
"""
|
||||
Get a system prompt variant for testing
|
||||
|
||||
Args:
|
||||
variant: Which prompt version to use (v1_verbose, v2_concise, v3_imperative, v4_minimal)
|
||||
|
||||
Returns:
|
||||
The system prompt string
|
||||
"""
|
||||
return PROMPTS.get(variant, PROMPTS["v1_verbose"])
|
||||
|
||||
|
||||
def list_prompts() -> list:
|
||||
"""List all available prompt variants"""
|
||||
return list(PROMPTS.keys())
|
||||
@@ -41,6 +41,18 @@ class ChatCompletionRequest(BaseModel):
|
||||
description="Store conversation turns in memory system"
|
||||
)
|
||||
|
||||
# Multi-tenancy (Phase 2.5)
|
||||
user_id: str = Field(
|
||||
default="llm-testuser",
|
||||
description="User ID for multi-tenant memory isolation (future: extracted from auth token)"
|
||||
)
|
||||
|
||||
# A/B Testing (Phase 3.5)
|
||||
system_prompt_override: Optional[str] = Field(
|
||||
default=None,
|
||||
description="Override system prompt variant for A/B testing (v1_verbose, v2_concise, v3_imperative, v4_minimal, v4_gemini_suggestion)"
|
||||
)
|
||||
|
||||
# Optional parameters
|
||||
temperature: Optional[float] = Field(default=0.7, ge=0, le=2)
|
||||
top_p: Optional[float] = Field(default=1.0, ge=0, le=1)
|
||||
|
||||
@@ -2,8 +2,10 @@
|
||||
Portainer API Client
|
||||
|
||||
Provides interface to Portainer REST API for stack and container management.
|
||||
Includes fallback to Docker socket for containers not managed by Portainer.
|
||||
"""
|
||||
import httpx
|
||||
import json
|
||||
from typing import Optional, Dict, List, Any
|
||||
from src.logging_config import get_logger
|
||||
from src.config import get_settings
|
||||
@@ -289,6 +291,152 @@ class PortainerClient:
|
||||
logger.info(f"Started container {container_id}")
|
||||
return True
|
||||
|
||||
# ========================================================================
|
||||
# Docker Socket Fallback (for containers not managed by Portainer)
|
||||
# ========================================================================
|
||||
|
||||
async def _list_containers_via_socket(self, all_containers: bool = True) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
Fallback: List containers directly via Docker socket
|
||||
|
||||
Used when Portainer API doesn't return complete data (e.g., containers
|
||||
started outside Portainer, AMP game servers, etc.)
|
||||
|
||||
Args:
|
||||
all_containers: Include stopped containers
|
||||
|
||||
Returns:
|
||||
List of container details in Docker API format
|
||||
"""
|
||||
try:
|
||||
# Docker socket is mounted at /var/run/docker.sock
|
||||
# Use httpx with unix socket transport
|
||||
transport = httpx.AsyncHTTPTransport(uds="/var/run/docker.sock")
|
||||
async with httpx.AsyncClient(transport=transport, timeout=10) as client:
|
||||
params = {"all": 1 if all_containers else 0}
|
||||
response = await client.get(
|
||||
"http://localhost/v1.41/containers/json",
|
||||
params=params
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
except Exception as e:
|
||||
logger.warning(f"Docker socket fallback failed: {e}")
|
||||
return []
|
||||
|
||||
async def _inspect_container_via_socket(self, container_id_or_name: str) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Fallback: Inspect container directly via Docker socket
|
||||
|
||||
Args:
|
||||
container_id_or_name: Container ID or name
|
||||
|
||||
Returns:
|
||||
Container details or None
|
||||
"""
|
||||
try:
|
||||
transport = httpx.AsyncHTTPTransport(uds="/var/run/docker.sock")
|
||||
async with httpx.AsyncClient(transport=transport, timeout=10) as client:
|
||||
response = await client.get(
|
||||
f"http://localhost/v1.41/containers/{container_id_or_name}/json"
|
||||
)
|
||||
response.raise_for_status()
|
||||
return response.json()
|
||||
except Exception as e:
|
||||
logger.warning(f"Docker socket inspect fallback failed for '{container_id_or_name}': {e}")
|
||||
return None
|
||||
|
||||
# ========================================================================
|
||||
# Helper methods for agent tools (auto-detect endpoint + fallback)
|
||||
# ========================================================================
|
||||
|
||||
async def list_containers(self, all_containers: bool = True) -> List[Dict[str, Any]]:
|
||||
"""
|
||||
List containers using auto-detected endpoint with Docker socket fallback
|
||||
|
||||
This is a convenience wrapper that automatically uses the first/default endpoint.
|
||||
If Portainer doesn't have complete data, falls back to Docker socket.
|
||||
|
||||
Args:
|
||||
all_containers: Include stopped containers (default: True)
|
||||
|
||||
Returns:
|
||||
List of container details
|
||||
"""
|
||||
try:
|
||||
# Try Portainer first
|
||||
endpoints = await self.get_endpoints()
|
||||
if endpoints:
|
||||
endpoint_id = endpoints[0]["Id"]
|
||||
containers = await self.get_containers(endpoint_id, all_containers)
|
||||
if containers:
|
||||
return containers
|
||||
|
||||
# Fallback to Docker socket
|
||||
logger.info("Portainer returned no containers, trying Docker socket fallback...")
|
||||
return await self._list_containers_via_socket(all_containers)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error listing containers: {e}")
|
||||
# Try fallback even on exception
|
||||
try:
|
||||
return await self._list_containers_via_socket(all_containers)
|
||||
except Exception as fallback_error:
|
||||
logger.error(f"Fallback also failed: {fallback_error}")
|
||||
return []
|
||||
|
||||
async def inspect_container(self, container_name: str) -> Optional[Dict[str, Any]]:
|
||||
"""
|
||||
Inspect a container by name using auto-detected endpoint with Docker socket fallback
|
||||
|
||||
This is a convenience wrapper that automatically uses the first/default endpoint.
|
||||
If Portainer doesn't find the container, falls back to Docker socket.
|
||||
|
||||
Args:
|
||||
container_name: Container name (e.g., "jellyfin", "ollama")
|
||||
|
||||
Returns:
|
||||
Container details or None if not found
|
||||
"""
|
||||
try:
|
||||
# Try Portainer first
|
||||
endpoints = await self.get_endpoints()
|
||||
if endpoints:
|
||||
endpoint_id = endpoints[0]["Id"]
|
||||
|
||||
# First list all containers to find the one matching the name
|
||||
all_containers = await self.get_containers(endpoint_id, all_containers=True)
|
||||
|
||||
matching_container = None
|
||||
for container in all_containers:
|
||||
# Container names come as array like ['/jellyfin']
|
||||
names = container.get('Names', [])
|
||||
for name in names:
|
||||
clean_name = name.lstrip('/')
|
||||
if clean_name == container_name or clean_name.lower() == container_name.lower():
|
||||
matching_container = container
|
||||
break
|
||||
if matching_container:
|
||||
break
|
||||
|
||||
if matching_container:
|
||||
# Get detailed info using container ID
|
||||
container_id = matching_container['Id']
|
||||
return await self.get_container(endpoint_id, container_id)
|
||||
|
||||
# Not found in Portainer, try Docker socket fallback
|
||||
logger.info(f"Container '{container_name}' not found in Portainer, trying Docker socket fallback...")
|
||||
return await self._inspect_container_via_socket(container_name)
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Error inspecting container '{container_name}': {e}")
|
||||
# Try fallback even on exception
|
||||
try:
|
||||
return await self._inspect_container_via_socket(container_name)
|
||||
except Exception as fallback_error:
|
||||
logger.error(f"Fallback also failed: {fallback_error}")
|
||||
return None
|
||||
|
||||
|
||||
# Singleton instance
|
||||
_portainer_client: Optional[PortainerClient] = None
|
||||
|
||||
@@ -215,6 +215,7 @@ async def store_conversation_turn(
|
||||
conversation_id: str,
|
||||
role: str,
|
||||
content: str,
|
||||
user_id: str = "llm-testuser",
|
||||
tokens: dict = None
|
||||
):
|
||||
"""
|
||||
@@ -224,9 +225,15 @@ async def store_conversation_turn(
|
||||
conversation_id: Unique conversation identifier
|
||||
role: Message role (user, assistant, system)
|
||||
content: Message content
|
||||
user_id: User ID for multi-tenancy (defaults to "llm-testuser")
|
||||
tokens: Optional token usage dict
|
||||
"""
|
||||
try:
|
||||
# Don't store system messages - they're part of the agent's state_modifier
|
||||
if role == "system":
|
||||
logger.debug(f"Skipping storage of system message for {conversation_id}")
|
||||
return
|
||||
|
||||
memory_manager = get_memory_manager()
|
||||
|
||||
# Convert role string to MemoryMessageRole
|
||||
@@ -234,8 +241,6 @@ async def store_conversation_turn(
|
||||
memory_role = MemoryMessageRole.USER
|
||||
elif role == "assistant":
|
||||
memory_role = MemoryMessageRole.ASSISTANT
|
||||
elif role == "system":
|
||||
memory_role = MemoryMessageRole.SYSTEM
|
||||
else:
|
||||
memory_role = MemoryMessageRole.USER # Default fallback
|
||||
|
||||
@@ -248,15 +253,16 @@ async def store_conversation_turn(
|
||||
total=tokens.get("total", 0)
|
||||
)
|
||||
|
||||
# Store in memory
|
||||
# Store in memory with user_id
|
||||
await memory_manager.add_turn(
|
||||
conversation_id=conversation_id,
|
||||
role=memory_role,
|
||||
content=content,
|
||||
user_id=user_id,
|
||||
tokens=token_usage
|
||||
)
|
||||
|
||||
logger.debug(f"Stored {role} turn in memory for conversation {conversation_id}")
|
||||
logger.debug(f"Stored {role} turn in memory for user={user_id}, conversation={conversation_id}")
|
||||
|
||||
except Exception as e:
|
||||
# Log error but don't fail the request
|
||||
@@ -295,11 +301,14 @@ class AIController(BaseController):
|
||||
"""
|
||||
request_id = f"chatcmpl-{int(time.time() * 1000)}"
|
||||
|
||||
# Multi-tenancy: Extract user_id (defaults to "llm-testuser")
|
||||
user_id = request.user_id
|
||||
|
||||
# Generate or use provided conversation_id
|
||||
conversation_id = request.conversation_id or f"conv_{uuid.uuid4().hex[:16]}"
|
||||
|
||||
logger.info(
|
||||
f"Chat request: id={request_id}, model={request.model}, "
|
||||
f"Chat request: id={request_id}, user={user_id}, model={request.model}, "
|
||||
f"messages={len(request.messages)}, stream={request.stream}, "
|
||||
f"conversation_id={conversation_id}, store_in_memory={request.store_in_memory}"
|
||||
)
|
||||
@@ -309,28 +318,106 @@ class AIController(BaseController):
|
||||
try:
|
||||
logger.info(f"Using unified agent for request {request_id}")
|
||||
|
||||
# Extract conversation history
|
||||
history = []
|
||||
for msg in request.messages[:-1]: # All except last
|
||||
history.append({"role": msg.role.value, "content": msg.content})
|
||||
|
||||
# Get last message
|
||||
user_message = request.messages[-1].content
|
||||
|
||||
# Load conversation history from memory if available
|
||||
history = []
|
||||
if request.store_in_memory:
|
||||
memory_manager = get_memory_manager()
|
||||
|
||||
# Check both buffer (Tier 1) and Qdrant (Tier 2/3)
|
||||
buffer_exists = await memory_manager.buffer_memory.conversation_exists(conversation_id)
|
||||
qdrant_exists = await memory_manager.qdrant_memory.conversation_exists(conversation_id)
|
||||
conversation_exists = buffer_exists or qdrant_exists
|
||||
|
||||
if conversation_exists:
|
||||
# Load full history (combines buffer + Qdrant)
|
||||
logger.info(f"Loading conversation history from memory for {conversation_id} (buffer={buffer_exists}, qdrant={qdrant_exists})")
|
||||
all_turns = await memory_manager.get_full_history(conversation_id, include_buffer=True)
|
||||
# Get most recent 20 turns
|
||||
recent_turns = all_turns[-20:] if len(all_turns) > 20 else all_turns
|
||||
# Filter out system messages - they should not be in conversation history
|
||||
history = [
|
||||
{"role": turn.role.value, "content": turn.content}
|
||||
for turn in recent_turns
|
||||
if turn.role.value != "system"
|
||||
]
|
||||
logger.info(f"✓ Loaded {len(history)} turns from memory (total: {len(all_turns)})")
|
||||
else:
|
||||
# New conversation - use request messages (excluding system messages)
|
||||
logger.info(f"New conversation {conversation_id} - using request messages")
|
||||
for msg in request.messages[:-1]: # All except last
|
||||
if msg.role.value != "system": # Skip system messages
|
||||
history.append({"role": msg.role.value, "content": msg.content})
|
||||
else:
|
||||
# Memory disabled - fall back to request messages (excluding system messages)
|
||||
for msg in request.messages[:-1]:
|
||||
if msg.role.value != "system": # Skip system messages
|
||||
history.append({"role": msg.role.value, "content": msg.content})
|
||||
|
||||
# Store user message in memory BEFORE agent execution
|
||||
if request.store_in_memory:
|
||||
logger.info(f"Storing user message in memory for {conversation_id}")
|
||||
await store_conversation_turn(
|
||||
conversation_id=conversation_id,
|
||||
role="user",
|
||||
content=user_message,
|
||||
user_id=user_id
|
||||
)
|
||||
logger.info(f"✓ Stored user message in memory for {conversation_id}")
|
||||
|
||||
# Get agent
|
||||
agent = get_unified_agent()
|
||||
|
||||
# Stream response
|
||||
if request.stream:
|
||||
# For streaming, we need to collect the response to store it
|
||||
collected_content = []
|
||||
|
||||
async def agent_stream_generator():
|
||||
agent_stream = agent.chat(
|
||||
message=user_message,
|
||||
conversation_history=history,
|
||||
stream=True
|
||||
)
|
||||
# Always use "Tatlock" as model name in responses
|
||||
async for sse_chunk in stream_agent_to_sse(agent_stream, request_id, "Tatlock"):
|
||||
yield sse_chunk
|
||||
nonlocal collected_content
|
||||
try:
|
||||
agent_stream = agent.chat(
|
||||
message=user_message,
|
||||
conversation_history=history,
|
||||
stream=True,
|
||||
prompt_variant=request.system_prompt_override
|
||||
)
|
||||
# Always use "Tatlock" as model name in responses
|
||||
async for sse_chunk in stream_agent_to_sse(agent_stream, request_id, "Tatlock"):
|
||||
# Collect content for memory storage
|
||||
# Extract content from SSE chunk if it contains delta content
|
||||
if '"content":' in sse_chunk:
|
||||
try:
|
||||
import json
|
||||
# Parse the SSE data line
|
||||
for line in sse_chunk.split('\n'):
|
||||
if line.startswith('data: ') and not line.startswith('data: [DONE]'):
|
||||
chunk_data = json.loads(line[6:]) # Remove 'data: ' prefix
|
||||
if 'choices' in chunk_data and len(chunk_data['choices']) > 0:
|
||||
delta = chunk_data['choices'][0].get('delta', {})
|
||||
if 'content' in delta:
|
||||
collected_content.append(delta['content'])
|
||||
except:
|
||||
pass
|
||||
yield sse_chunk.encode('utf-8')
|
||||
finally:
|
||||
# Store assistant response in memory AFTER streaming completes
|
||||
# This runs in the finally block to ensure it executes even if client disconnects
|
||||
if request.store_in_memory and collected_content:
|
||||
full_response = ''.join(collected_content)
|
||||
logger.info(f"Storing assistant response in memory for {conversation_id}")
|
||||
try:
|
||||
await store_conversation_turn(
|
||||
conversation_id=conversation_id,
|
||||
role="assistant",
|
||||
content=full_response,
|
||||
user_id=user_id
|
||||
)
|
||||
logger.info(f"✓ Stored assistant response in memory for {conversation_id}")
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to store assistant response: {e}")
|
||||
|
||||
return StreamingResponse(
|
||||
agent_stream_generator(),
|
||||
@@ -345,9 +432,29 @@ class AIController(BaseController):
|
||||
# Non-streaming
|
||||
response_text = await agent.chat_completion(
|
||||
message=user_message,
|
||||
conversation_history=history
|
||||
conversation_history=history,
|
||||
prompt_variant=request.system_prompt_override
|
||||
)
|
||||
|
||||
# Store assistant response in memory AFTER agent execution
|
||||
if request.store_in_memory:
|
||||
# Estimate token usage (simple word count)
|
||||
prompt_tokens = len(user_message.split())
|
||||
completion_tokens = len(response_text.split())
|
||||
logger.info(f"Storing assistant response in memory for {conversation_id}")
|
||||
await store_conversation_turn(
|
||||
conversation_id=conversation_id,
|
||||
role="assistant",
|
||||
content=response_text,
|
||||
user_id=user_id,
|
||||
tokens={
|
||||
"prompt": prompt_tokens,
|
||||
"completion": completion_tokens,
|
||||
"total": prompt_tokens + completion_tokens
|
||||
}
|
||||
)
|
||||
logger.info(f"✓ Stored assistant response in memory for {conversation_id}")
|
||||
|
||||
# Always use "Tatlock" as model name in responses
|
||||
return ChatCompletionResponse(
|
||||
id=request_id,
|
||||
@@ -371,6 +478,12 @@ class AIController(BaseController):
|
||||
)
|
||||
)
|
||||
except Exception as e:
|
||||
if not settings.agent_fallback_enabled:
|
||||
logger.error(f"Agent failed and fallback is disabled. Error: {e}", exc_info=True)
|
||||
raise HTTPException(
|
||||
status_code=500,
|
||||
detail=f"Agent failed to generate completion: {str(e)}"
|
||||
)
|
||||
logger.error(f"Agent failed, falling back to direct Ollama: {e}")
|
||||
# Fall through to direct Ollama call below
|
||||
|
||||
@@ -382,7 +495,8 @@ class AIController(BaseController):
|
||||
await store_conversation_turn(
|
||||
conversation_id=conversation_id,
|
||||
role=role,
|
||||
content=msg.content
|
||||
content=msg.content,
|
||||
user_id=user_id
|
||||
)
|
||||
|
||||
# Build prompt from messages
|
||||
@@ -419,6 +533,7 @@ class AIController(BaseController):
|
||||
conversation_id=conversation_id,
|
||||
role="assistant",
|
||||
content=assistant_content,
|
||||
user_id=user_id,
|
||||
tokens=result["tokens"]
|
||||
)
|
||||
|
||||
|
||||
@@ -61,6 +61,7 @@ class MemoryManager:
|
||||
conversation_id: str,
|
||||
role: MessageRole,
|
||||
content: str,
|
||||
user_id: str = "llm-testuser",
|
||||
tokens: Optional[TokenUsage] = None,
|
||||
metadata: Optional[Dict[str, Any]] = None
|
||||
) -> ConversationTurn:
|
||||
@@ -76,6 +77,7 @@ class MemoryManager:
|
||||
conversation_id: Unique conversation identifier
|
||||
role: Message role (user, assistant, system)
|
||||
content: Message content
|
||||
user_id: User ID for multi-tenancy (defaults to "llm-testuser")
|
||||
tokens: Optional token usage
|
||||
metadata: Optional metadata
|
||||
|
||||
@@ -86,12 +88,13 @@ class MemoryManager:
|
||||
buffer = await self.buffer_memory.get_buffer(conversation_id)
|
||||
turn_number = (buffer.metadata.turn_count + 1) if buffer else 1
|
||||
|
||||
# Create turn
|
||||
# Create turn with user_id
|
||||
turn = ConversationTurn(
|
||||
role=role,
|
||||
content=content,
|
||||
timestamp=datetime.utcnow(),
|
||||
turn_number=turn_number,
|
||||
user_id=user_id,
|
||||
tokens=tokens,
|
||||
metadata=metadata or {}
|
||||
)
|
||||
|
||||
@@ -108,13 +108,14 @@ class QdrantConversationMemory(BaseMemory):
|
||||
point_id_str = f"{conversation_id}_{turn.turn_number}"
|
||||
point_id = str(uuid.uuid5(uuid.NAMESPACE_DNS, point_id_str))
|
||||
|
||||
# Build payload
|
||||
# Build payload with user_id for multi-tenancy
|
||||
payload = {
|
||||
"conversation_id": conversation_id,
|
||||
"turn_number": turn.turn_number,
|
||||
"role": turn.role.value if isinstance(turn.role, MessageRole) else turn.role,
|
||||
"content": turn.content,
|
||||
"timestamp": turn.timestamp.isoformat(),
|
||||
"user_id": turn.user_id, # Multi-tenancy
|
||||
"metadata": turn.metadata,
|
||||
}
|
||||
|
||||
@@ -186,6 +187,7 @@ class QdrantConversationMemory(BaseMemory):
|
||||
content=payload["content"],
|
||||
timestamp=datetime.fromisoformat(payload["timestamp"]),
|
||||
turn_number=payload["turn_number"],
|
||||
user_id=payload.get("user_id", "llm-testuser"), # Multi-tenancy
|
||||
metadata=payload.get("metadata", {})
|
||||
)
|
||||
turns.append(turn)
|
||||
|
||||
@@ -27,6 +27,7 @@ class ConversationTurn(BaseModel):
|
||||
content: str
|
||||
timestamp: datetime = Field(default_factory=datetime.utcnow)
|
||||
turn_number: int
|
||||
user_id: str = "llm-testuser" # Multi-tenancy: user who owns this turn
|
||||
tokens: Optional[TokenUsage] = None
|
||||
metadata: Dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
@@ -34,7 +35,7 @@ class ConversationTurn(BaseModel):
|
||||
class ConversationMetadata(BaseModel):
|
||||
"""Metadata about a conversation"""
|
||||
conversation_id: str
|
||||
user_id: Optional[str] = None
|
||||
user_id: str = "llm-testuser" # Multi-tenancy: user who owns this conversation
|
||||
created_at: datetime = Field(default_factory=datetime.utcnow)
|
||||
last_updated: datetime = Field(default_factory=datetime.utcnow)
|
||||
turn_count: int = 0
|
||||
|
||||
Reference in New Issue
Block a user