From 51cee7491221b682c8212fa7bf4dccf31cfdb8f5 Mon Sep 17 00:00:00 2001 From: Jeroen Schweitzer Date: Sat, 13 Dec 2025 11:48:21 +0100 Subject: [PATCH] docs: add orchestration scenarios document MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Documents desired multi-agent orchestration patterns with intra-system prompts showing how Tatlock delegates to experts. Includes 8 scenarios from simple to complex: 1. Weather lookup (implicit location) 2. Conditional home automation 3. Wiki page creation 4. Research queries 5. Document updates 6. Multi-source synthesis 7. Graph exploration 8. Multi-step workflows 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 --- docs/ORCHESTRATION_SCENARIOS.md | 679 ++++++++++++++++++++++++++++++++ 1 file changed, 679 insertions(+) create mode 100644 docs/ORCHESTRATION_SCENARIOS.md diff --git a/docs/ORCHESTRATION_SCENARIOS.md b/docs/ORCHESTRATION_SCENARIOS.md new file mode 100644 index 0000000..78d8bfd --- /dev/null +++ b/docs/ORCHESTRATION_SCENARIOS.md @@ -0,0 +1,679 @@ +# Orchestration Scenarios and Tool Flows + +This document outlines example scenarios of varying complexity to illustrate the desired orchestration patterns between Tatlock (Butler/Coordinator), expert agents (The Librarian, etc.), and the user. + +## Architecture Overview + +``` +User Request + ↓ +[Steward] → Analyzes request, has visibility into ALL capabilities + → Makes routing decision: which experts needed + → Passes simplified instruction to Tatlock (not raw tool schemas) + ↓ +[Tatlock/Butler] → Coordinator, receives "use Librarian for wiki creation" + → Calls expert agents as tools + → Synthesizes responses into butler-voice answer + ↓ +[Expert Agents] → The Librarian, Home Automation, Memory, etc. + → Each has their own specialized tools + → Return structured results to Tatlock + ↓ +[External APIs] → library-desk, home-assistant, user-db, etc. +``` + +**Key Principles**: + +1. **Steward sees everything** - Has access to all capability descriptions to make informed routing decisions +2. **Simplified passthrough** - Tatlock receives "delegate to Librarian for research" not 16 tool schemas +3. **Expert agents are tools** - Tatlock calls `librarian_agent(task)`, not `hybrid_search()` directly +4. **Each expert owns their tools** - Librarian has wiki tools, Home Automation has device tools +5. **Results flow up** - Tatlock synthesizes all expert responses into coherent butler answer + +--- + +## Scenario 1: Weather Check (Multi-Step with Memory Lookup) + +**User**: "What's the weather like?" + +### Complexity Analysis + +This seemingly simple request requires: +1. **Location determination** - Where does the user want weather for? +2. **Memory/database lookup** - Retrieve user's home location or current location +3. **Weather data fetch** - Search for weather at determined location + +### Flow + +``` +1. Steward Analysis + → Capabilities needed: memory (user context), tatlock_core (web search) + → Complexity: moderate + → Note: Location must be determined before weather lookup + +2. Tatlock Execution - Step 1 + User asked about weather but didn't specify location. + Checking user profile for home location... + → Calls: memory_agent(task: "get user home location") + → Memory queries user database + → Returns: "User home location: Amsterdam, Netherlands" + +3. Tatlock Execution - Step 2 + User is based in Amsterdam. Fetching current weather... + → Calls: search_web("current weather Amsterdam Netherlands") + → Receives: "Amsterdam: 12°C, light rain, humidity 78%" + +4. Response + "Currently 12°C with light rain in Amsterdam, sir. You might want + to grab an umbrella if you're heading out." +``` + +### Intra-System Prompts + +**Steward → Tatlock Note**: +``` +Weather query - location not specified. +1. First: Query memory for user's location (home or current) +2. Then: Search weather for that location +Capabilities: memory, tatlock_core +Complexity: moderate +``` + +**Tatlock → Memory Agent**: +``` +Task: Retrieve user's location for weather query. +Context: User asked about weather without specifying location. +Action required: Return user's home location or current known location. + +Reference (user's original request): "What's the weather like?" +``` + +**Memory Agent → Tatlock Response**: +``` +User location retrieved: +- Home location: Amsterdam, Netherlands +- Last known location: Amsterdam (home) +- Location confidence: high +- Source: user profile settings +``` + +### Alternative Flow: Location Ambiguity + +If user has multiple locations or is traveling: + +``` +Memory Agent → Tatlock Response: + User has multiple locations: + - Home: Amsterdam, Netherlands + - Office: Rotterdam, Netherlands + - Currently traveling: Unknown + + Recommendation: Ask user to clarify or use home location as default. +``` + +Tatlock could then either: +- Ask user: "Shall I check the weather in Amsterdam, sir, or elsewhere?" +- Default to home: Use Amsterdam and mention the assumption + +--- + +## Scenario 2: Adjust Temperature Based on Weather (Conditional Multi-Expert) + +**User**: "Check the weather and if it's cold, turn up the heating" + +### Complexity Analysis + +This requires: +1. **Location lookup** - Where to check weather (implicit: user's home) +2. **Weather fetch** - Get current outdoor temperature +3. **Conditional evaluation** - Is it "cold"? (requires threshold judgment) +4. **Home automation** - Adjust heating if condition met + +### Flow + +``` +1. Steward Analysis + → Capabilities needed: memory, tatlock_core, home_automation + → Complexity: moderate + → Note: Conditional logic - heating only if cold + → Sequence: location → weather → evaluate → (maybe) heating + +2. Tatlock Execution - Step 1 + Need to check weather at user's location first... + → Calls: memory_agent(task: "get user home location") + → Returns: "Amsterdam, Netherlands" + +3. Tatlock Execution - Step 2 + Fetching weather for Amsterdam... + → Calls: search_web("current weather Amsterdam Netherlands") + → Receives: "Current temperature: 8°C, cloudy, wind 15km/h" + +4. Tatlock Evaluation + Temperature is 8°C - that's cold by most standards. + User requested heating adjustment if cold. Will proceed... + +5. Tatlock Execution - Step 3 + Delegating heating adjustment to Home Automation... + → Calls: home_automation_agent(task) + → Home Automation executes: set_thermostat(temperature=21) + → Receives: "Thermostat set to 21°C" + +6. Response + "It's rather brisk outside at 8°C, sir. I've taken the liberty of raising + the heating to a comfortable 21°C. The house should warm up shortly." +``` + +### Intra-System Prompts + +**Steward → Tatlock Note**: +``` +Conditional weather-to-heating request. +1. Get user location from memory +2. Check weather at location +3. IF cold (suggest: below 15°C): delegate to home_automation to increase heating +4. IF not cold: inform user, no action needed +Capabilities: memory, tatlock_core, home_automation +Complexity: moderate (conditional) +``` + +**Tatlock → Home Automation Agent**: +``` +Task: Increase heating due to cold weather conditions. +Context: Current outdoor temperature is 8°C in Amsterdam. User requested +heating adjustment if cold - condition is met. +Action required: Set thermostat to comfortable temperature (recommend 21°C). + +Reference (user's original request): "Check the weather and if it's cold, turn up the heating" +``` + +**Home Automation → Tatlock Response**: +``` +Action completed: Thermostat adjusted +- Previous setting: 18°C +- New setting: 21°C +- Zone: All zones (whole house) +- Estimated time to reach target: 15 minutes +- Note: Eco mode disabled temporarily for faster heating +``` + +### Alternative Flow: Not Cold + +``` +Weather returns: "Current temperature: 18°C, partly sunny" + +Tatlock Evaluation: +18°C isn't cold - no heating adjustment needed. +Will inform user that conditions don't warrant heating... + +Response: +"It's a pleasant 18°C outside, sir - not particularly cold. +I don't believe the heating requires adjustment, but do let +me know if you'd like it warmer regardless." +``` + +--- + +## Scenario 3: Adjust Lights Based on Time (Conditional Logic) + +**User**: "Set the lights appropriately for the time of day" + +### Flow + +``` +1. Steward Analysis + → Capabilities needed: tatlock_core (datetime), home_automation + → Complexity: moderate + +2. Tatlock Execution - Step 1 + Checking current time to determine appropriate lighting... + → Calls: get_current_datetime() + → Receives: "2024-12-11 21:45" + +3. Tatlock Evaluation + It's evening (21:45). Will set warm, dimmed lighting for wind-down time... + +4. Tatlock Execution - Step 2 + → Calls: home_automation_agent(task) + → Home Automation executes: set_scene("evening_relaxation") + → Receives: "Scene 'evening_relaxation' activated - warm white 40%" + +5. Response + "It's quarter to ten in the evening, sir. I've set the lights to a warm, + subdued glow - ideal for winding down. Shall I also draw the curtains?" +``` + +### Intra-System Prompts + +**Tatlock → Home Automation Agent**: +``` +Task: Set lighting appropriate for current time of day. +Context: Current time is 21:45 (evening). User wants lights adjusted automatically. +Action required: Activate appropriate lighting scene for evening/night. + +Reference (user's original request): "Set the lights appropriately for the time of day" +``` + +--- + +## Scenario 4: Create a Reminder (Simple Expert Delegation) + +**User**: "Remind me to call the dentist tomorrow at 2pm" + +### Flow + +``` +1. Steward Analysis + → Capabilities needed: personal_assistant (reminders) + → Complexity: simple + +2. Tatlock Execution + Creating reminder for tomorrow afternoon... + → Calls: personal_assistant_agent(task) + → PA executes: create_reminder(title, datetime, notification_time) + → Receives: "Reminder created: ID 847" + +3. Response + "Very good, sir. I've set a reminder to call the dentist for tomorrow at + 2 o'clock in the afternoon. I'll notify you 15 minutes beforehand." +``` + +### Intra-System Prompts + +**Tatlock → Personal Assistant Agent**: +``` +Task: Create a reminder for the user. +Details: +- Subject: Call the dentist +- When: Tomorrow at 14:00 +- Notification: 15 minutes before (default) + +Reference (user's original request): "Remind me to call the dentist tomorrow at 2pm" +``` + +--- + +## Scenario 5: Schedule Recurring Automation (Complex, Multi-Expert) + +**User**: "Schedule the lights to turn on at 09:00 every weekend day" + +### Flow + +``` +1. Steward Analysis + → Capabilities needed: home_automation (scheduling) + → Complexity: moderate + → Note: Recurring schedule, weekend-only + +2. Tatlock Execution + Setting up recurring weekend morning lighting schedule... + → Calls: home_automation_agent(task) + → HA executes: create_schedule(trigger, days, action) + → Receives: "Schedule created: ID wake_light_weekend" + +3. Response + "The weekend wake-up lighting is now scheduled, sir. Every Saturday and + Sunday at 9 o'clock sharp, the lights will illuminate. Would you prefer + a gradual sunrise simulation, or an immediate full brightness?" +``` + +### Intra-System Prompts + +**Tatlock → Home Automation Agent**: +``` +Task: Create a recurring lighting schedule. +Details: +- Action: Turn on lights +- Time: 09:00 +- Days: Saturday, Sunday (weekends only) +- Recurrence: Weekly + +Reference (user's original request): "Schedule the lights to turn on at 09:00 every weekend day" +``` + +--- + +## Scenario 6: Create Wiki Page About Topic (Expert with Research) + +**User**: "Create a wiki page about CI/CD" + +### Flow + +``` +1. Steward Analysis + → Capabilities needed: librarian + → Complexity: complex + → Specific action: smart_create (research-backed page creation) + +2. Tatlock Execution + Delegating to The Librarian for research and wiki page creation... + → Calls: librarian_agent(task) + +3. Librarian Internal Flow + → hybrid_search("CI/CD continuous integration deployment") + → Synthesizes content from wiki, graph, and web sources + → create_wiki_page(title, path, content, tags) + → Returns summary to Tatlock + +4. Tatlock Receives + "Page created: CI/CD + Path: /technology/cicd + Sources used: 12 (3 wiki, 4 graph, 5 web) + Forward links: 8, Backward links: 3" + +5. Response + "The wiki page on CI/CD has been created, sir. The Librarian has compiled + information from 12 sources, including our existing documentation on DevOps + practices. The page is now available at /technology/cicd and has been + cross-linked with related topics like Jenkins, GitHub Actions, and Docker." +``` + +### Intra-System Prompts + +**Tatlock → Librarian Agent**: +``` +Task: Create a comprehensive wiki page about CI/CD. +Context: User wants a new wiki page created. Use research capabilities to +gather information and create a well-sourced page. + +Action required: +1. Research the topic using hybrid search (wiki, knowledge graph, web) +2. Create wiki page with synthesized content +3. Apply appropriate tags (suggest: technology, devops, automation) +4. Report back with page details and research summary + +Reference (user's original request): "Create a wiki page about CI/CD" +``` + +**Librarian → Tatlock Response**: +``` +Task completed: Wiki page created + +Page Details: +- Title: CI/CD (Continuous Integration/Continuous Deployment) +- Path: /technology/cicd +- Tags: technology, devops, automation, software-development + +Research Summary: +- Wiki sources: 3 pages (DevOps Overview, Jenkins Setup, GitHub Actions Guide) +- Knowledge graph entities: 4 (linked to Docker, Kubernetes, Git, Testing) +- Web sources: 5 articles (current best practices, 2024 trends) +- Total sources synthesized: 12 + +Knowledge Graph Updates: +- Forward links created: 8 (to existing related pages) +- Backward links created: 3 (updated existing pages to reference new page) + +Content includes sections on: +- Definition and core concepts +- CI vs CD distinction +- Popular tools comparison +- Best practices +- Integration with our documented infrastructure +``` + +--- + +## Scenario 7: Update Stock Wiki with Evaluation Tracker (Complex, Multi-Tool Expert) + +**User**: "Update the GOOG stock wiki page with a rolling evaluation tracker" + +### Flow + +``` +1. Steward Analysis + → Capabilities needed: librarian + → Complexity: complex + → Specific action: search existing page, fetch current data, update page + +2. Tatlock Execution + This requires finding the existing page, fetching current stock data, + and updating with a new evaluation section. Delegating to The Librarian... + → Calls: librarian_agent(task) + +3. Librarian Internal Flow - Step 1 + First, locating the existing GOOG wiki page... + → search_wiki("GOOG stock Alphabet") + → Receives: Page ID 234, path /investments/stocks/goog + +4. Librarian Internal Flow - Step 2 + Fetching current page content to preserve existing information... + → get_wiki_page(234) + → Receives: Current page content + +5. Librarian Internal Flow - Step 3 + Researching current stock data and evaluation metrics... + → hybrid_search("GOOG Alphabet stock price evaluation 2024") + → Receives: Current price, P/E ratio, analyst ratings, etc. + +6. Librarian Internal Flow - Step 4 + Updating page with new rolling evaluation section... + → update_wiki_page(234, content=updated_content) + → Receives: Update confirmation + +7. Tatlock Receives + "Page updated: GOOG Stock Analysis + Added section: Rolling Evaluation Tracker + Data points added: Price history, P/E trend, Analyst consensus + Last updated: 2024-12-11" + +8. Response + "The GOOG stock page has been updated, sir. I've added a rolling evaluation + tracker with current metrics: the stock is trading at $178.32 with a P/E + of 24.8, and analyst consensus remains 'Buy'. The tracker includes a + 90-day price trend and quarterly earnings history. Shall I set up + automatic weekly updates?" +``` + +### Intra-System Prompts + +**Tatlock → Librarian Agent**: +``` +Task: Update the GOOG (Alphabet) stock wiki page with a rolling evaluation tracker. +Context: User wants to add ongoing stock evaluation tracking to an existing page. + +Actions required: +1. Find the existing GOOG stock wiki page +2. Read current page content (preserve existing information) +3. Research current stock data and evaluation metrics +4. Update the page with a new "Rolling Evaluation Tracker" section including: + - Current price and change + - Key ratios (P/E, P/B, etc.) + - Analyst consensus + - Price trend (30/60/90 day) + - Recent earnings summary +5. Report back with update summary + +Reference (user's original request): "Update the GOOG stock wiki page with a rolling evaluation tracker" +``` + +**Librarian → Tatlock Response**: +``` +Task completed: Wiki page updated + +Page Details: +- Title: GOOG - Alphabet Inc. Stock Analysis +- Path: /investments/stocks/goog +- Page ID: 234 + +Update Summary: +- New section added: "Rolling Evaluation Tracker" +- Existing content: Preserved (company overview, investment thesis) + +Evaluation Data Added: +- Current Price: $178.32 (+1.2% today) +- P/E Ratio: 24.8 (vs sector avg 28.3) +- P/B Ratio: 6.2 +- 52-Week Range: $120.21 - $191.75 +- Analyst Consensus: Buy (24 buy, 8 hold, 2 sell) +- Price Trend: +15.3% (90-day) +- Last Earnings: Q3 2024 - Beat estimates by 8% + +Sources: +- Web: 4 financial data sources +- Existing wiki: Investment strategy context + +Suggested follow-up: Set up automated weekly data refresh? +``` + +--- + +## Scenario 8: Complex Multi-Expert Coordination + +**User**: "Check if there's anything about the new project deadline in my notes, +if so create a reminder, and if the deadline is this week, also update the +project wiki page with a status note" + +### Flow + +``` +1. Steward Analysis + → Capabilities needed: librarian, personal_assistant + → Complexity: complex + → Note: Conditional multi-step, requires information passing between experts + +2. Tatlock Execution - Step 1 + First, searching for project deadline information in the wiki... + → Calls: librarian_agent(search_task) + → Librarian: hybrid_search("new project deadline") + → Returns: "Project Alpha deadline: December 15, 2024 (this Friday)" + +3. Tatlock Evaluation + Found deadline: December 15. That's this week (Friday). + Need to: 1) Create reminder, 2) Update project wiki page... + +4. Tatlock Execution - Step 2 (parallel if possible) + Creating reminder and updating wiki status... + + → Calls: personal_assistant_agent(reminder_task) + → PA: create_reminder("Project Alpha deadline", "2024-12-15 09:00") + → Returns: "Reminder created for Dec 15 at 9am" + + → Calls: librarian_agent(update_task) + → Librarian: search_wiki → get_wiki_page → update_wiki_page + → Returns: "Project Alpha page updated with deadline status note" + +5. Response + "I've found the deadline in your notes, sir - Project Alpha is due this + Friday, December 15th. I've set a reminder for 9 o'clock that morning, + and I've updated the project wiki page with a status note indicating + the imminent deadline. Is there anything else you need to prepare?" +``` + +### Intra-System Prompts + +**Tatlock → Librarian Agent (Search)**: +``` +Task: Search for information about a new project deadline. +Context: User wants to find deadline information from their notes/wiki. + +Action required: +1. Search wiki and knowledge base for project deadline information +2. Return: Project name, deadline date, and any relevant context + +Reference (user's original request): "Check if there's anything about the new project deadline in my notes..." +``` + +**Tatlock → Personal Assistant Agent**: +``` +Task: Create a reminder for a project deadline. +Details: +- Subject: Project Alpha deadline +- When: December 15, 2024 at 09:00 +- Priority: High (deadline is this week) +- Notification: Morning of the deadline + +Reference: Creating reminder based on deadline found in user's notes. +``` + +**Tatlock → Librarian Agent (Update)**: +``` +Task: Update the Project Alpha wiki page with a deadline status note. +Context: Project deadline is December 15, 2024 (this Friday). User requested +a status update since the deadline is this week. + +Action required: +1. Find the Project Alpha wiki page +2. Add a status note/banner indicating the imminent deadline +3. Optionally update any status fields + +Reference: Part of user's request to track and highlight near-term deadlines. +``` + +--- + +## Response Pattern Guidelines + +### Tatlock's Think Updates (Streaming to User) + +During multi-step operations, Tatlock should emit `` updates to keep the user informed: + +``` +Analyzing your request... +Searching for deadline information in the wiki... +Found the deadline - December 15th. Creating reminder... +Updating the project page with status note... +All tasks complete. Composing response... +``` + +### Tatlock's Final Response Pattern + +1. **Acknowledge** - Confirm understanding of the request +2. **Summarize actions** - What was done, by whom (implicitly) +3. **Key details** - Important information the user should know +4. **Proactive offer** - Suggest related actions or follow-ups +5. **Butler voice** - Formal but warm, with personality + +### Expert Agent Response Pattern + +1. **Task status** - Completed/Partial/Failed +2. **Action summary** - What was done +3. **Key data** - Information Tatlock needs to synthesize +4. **Metadata** - IDs, counts, timestamps for reference +5. **Suggestions** - Optional follow-up actions + +--- + +## Error Handling Scenarios + +### Expert Agent Failure + +``` +Tatlock → Librarian: "Create wiki page about quantum computing" +Librarian → Tatlock: "Error: library-desk API unavailable (connection timeout)" + +Tatlock Response: +"I'm afraid The Librarian is having some difficulty reaching the wiki +service at the moment, sir. I can attempt a basic web search on quantum +computing if you'd like, or we can try the wiki operation again in a +few minutes." +``` + +### Partial Completion + +``` +User: "Create a reminder and add it to my calendar" + +Tatlock → PA: Create reminder +PA → Tatlock: "Reminder created successfully" + +Tatlock → Calendar: Add to calendar +Calendar → Tatlock: "Error: Calendar sync not configured" + +Tatlock Response: +"I've created the reminder, sir, but I wasn't able to add it to your +calendar - it appears the calendar integration needs to be configured. +The reminder will still alert you at the scheduled time. Shall I help +set up the calendar connection?" +``` + +--- + +## Summary: Key Design Principles + +1. **Tatlock is the orchestrator** - Never exposes raw tool complexity to users +2. **Expert agents are tools** - Tatlock calls them, they return structured responses +3. **Context flows down** - Each expert gets only what they need to complete their task +4. **Results flow up** - Tatlock synthesizes all responses into coherent butler-voice answer +5. **Think updates maintain engagement** - User sees progress during complex operations +6. **Errors are handled gracefully** - Tatlock explains and offers alternatives +7. **Proactive suggestions** - Tatlock anticipates follow-up needs