tatlock/ORCHESTRATION_SCENARIOS.md

Orchestration Scenarios and Tool Flows

This document outlines example scenarios of varying complexity to illustrate the desired orchestration patterns between Tatlock (Butler/Coordinator), expert agents (The Librarian, etc.), and the user.

Architecture Overview

User Request
     ↓
[Steward] → Analyzes request, has visibility into ALL capabilities
           → Makes routing decision: which experts needed
           → Passes simplified instruction to Tatlock (not raw tool schemas)
     ↓
[Tatlock/Butler] → Coordinator, receives "use Librarian for wiki creation"
                 → Calls expert agents as tools
                 → Synthesizes responses into butler-voice answer
     ↓
[Expert Agents] → The Librarian, Home Automation, Memory, etc.
                → Each has their own specialized tools
                → Return structured results to Tatlock
     ↓
[External APIs] → library-desk, home-assistant, user-db, etc.

Key Principles:

1. Steward sees everything - Has access to all capability descriptions to make informed routing decisions
2. Simplified passthrough - Tatlock receives "delegate to Librarian for research" not 16 tool schemas
3. Expert agents are tools - Tatlock calls librarian_agent(task), not hybrid_search() directly
4. Each expert owns their tools - Librarian has wiki tools, Home Automation has device tools
5. Results flow up - Tatlock synthesizes all expert responses into coherent butler answer

---

Scenario 1: Weather Check (Multi-Step with Memory Lookup)

User: "What's the weather like?"

Complexity Analysis

This seemingly simple request requires:

1. Location determination - Where does the user want weather for?
2. Memory/database lookup - Retrieve user's home location or current location
3. Weather data fetch - Search for weather at determined location

Flow

1. Steward Analysis
   → Capabilities needed: memory (user context), tatlock_core (web search)
   → Complexity: moderate
   → Note: Location must be determined before weather lookup

2. Tatlock Execution - Step 1
   <think>User asked about weather but didn't specify location.
   Checking user profile for home location...</think>
   → Calls: memory_agent(task: "get user home location")
   → Memory queries user database
   → Returns: "User home location: Amsterdam, Netherlands"

3. Tatlock Execution - Step 2
   <think>User is based in Amsterdam. Fetching current weather...</think>
   → Calls: search_web("current weather Amsterdam Netherlands")
   → Receives: "Amsterdam: 12°C, light rain, humidity 78%"

4. Response
   "Currently 12°C with light rain in Amsterdam, sir. You might want
   to grab an umbrella if you're heading out."

Intra-System Prompts

Steward → Tatlock Note:

Weather query - location not specified.
1. First: Query memory for user's location (home or current)
2. Then: Search weather for that location
Capabilities: memory, tatlock_core
Complexity: moderate

Tatlock → Memory Agent:

Task: Retrieve user's location for weather query.
Context: User asked about weather without specifying location.
Action required: Return user's home location or current known location.

Reference (user's original request): "What's the weather like?"

Memory Agent → Tatlock Response:

User location retrieved:
- Home location: Amsterdam, Netherlands
- Last known location: Amsterdam (home)
- Location confidence: high
- Source: user profile settings

Alternative Flow: Location Ambiguity

If user has multiple locations or is traveling:

Memory Agent → Tatlock Response:
  User has multiple locations:
  - Home: Amsterdam, Netherlands
  - Office: Rotterdam, Netherlands
  - Currently traveling: Unknown

  Recommendation: Ask user to clarify or use home location as default.

Tatlock could then either:

• Ask user: "Shall I check the weather in Amsterdam, sir, or elsewhere?"
• Default to home: Use Amsterdam and mention the assumption

---

Scenario 2: Adjust Temperature Based on Weather (Conditional Multi-Expert)

User: "Check the weather and if it's cold, turn up the heating"

Complexity Analysis

This requires:

1. Location lookup - Where to check weather (implicit: user's home)
2. Weather fetch - Get current outdoor temperature
3. Conditional evaluation - Is it "cold"? (requires threshold judgment)
4. Home automation - Adjust heating if condition met

Flow

1. Steward Analysis
   → Capabilities needed: memory, tatlock_core, home_automation
   → Complexity: moderate
   → Note: Conditional logic - heating only if cold
   → Sequence: location → weather → evaluate → (maybe) heating

2. Tatlock Execution - Step 1
   <think>Need to check weather at user's location first...</think>
   → Calls: memory_agent(task: "get user home location")
   → Returns: "Amsterdam, Netherlands"

3. Tatlock Execution - Step 2
   <think>Fetching weather for Amsterdam...</think>
   → Calls: search_web("current weather Amsterdam Netherlands")
   → Receives: "Current temperature: 8°C, cloudy, wind 15km/h"

4. Tatlock Evaluation
   <think>Temperature is 8°C - that's cold by most standards.
   User requested heating adjustment if cold. Will proceed...</think>

5. Tatlock Execution - Step 3
   <think>Delegating heating adjustment to Home Automation...</think>
   → Calls: home_automation_agent(task)
   → Home Automation executes: set_thermostat(temperature=21)
   → Receives: "Thermostat set to 21°C"

6. Response
   "It's rather brisk outside at 8°C, sir. I've taken the liberty of raising
   the heating to a comfortable 21°C. The house should warm up shortly."

Intra-System Prompts

Steward → Tatlock Note:

Conditional weather-to-heating request.
1. Get user location from memory
2. Check weather at location
3. IF cold (suggest: below 15°C): delegate to home_automation to increase heating
4. IF not cold: inform user, no action needed
Capabilities: memory, tatlock_core, home_automation
Complexity: moderate (conditional)

Tatlock → Home Automation Agent:

Task: Increase heating due to cold weather conditions.
Context: Current outdoor temperature is 8°C in Amsterdam. User requested
heating adjustment if cold - condition is met.
Action required: Set thermostat to comfortable temperature (recommend 21°C).

Reference (user's original request): "Check the weather and if it's cold, turn up the heating"

Home Automation → Tatlock Response:

Action completed: Thermostat adjusted
- Previous setting: 18°C
- New setting: 21°C
- Zone: All zones (whole house)
- Estimated time to reach target: 15 minutes
- Note: Eco mode disabled temporarily for faster heating

Alternative Flow: Not Cold

Weather returns: "Current temperature: 18°C, partly sunny"

Tatlock Evaluation:
<think>18°C isn't cold - no heating adjustment needed.
Will inform user that conditions don't warrant heating...</think>

Response:
"It's a pleasant 18°C outside, sir - not particularly cold.
I don't believe the heating requires adjustment, but do let
me know if you'd like it warmer regardless."

---

Scenario 3: Adjust Lights Based on Time (Conditional Logic)

User: "Set the lights appropriately for the time of day"

Flow

1. Steward Analysis
   → Capabilities needed: tatlock_core (datetime), home_automation
   → Complexity: moderate

2. Tatlock Execution - Step 1
   <think>Checking current time to determine appropriate lighting...</think>
   → Calls: get_current_datetime()
   → Receives: "2024-12-11 21:45"

3. Tatlock Evaluation
   <think>It's evening (21:45). Will set warm, dimmed lighting for wind-down time...</think>

4. Tatlock Execution - Step 2
   → Calls: home_automation_agent(task)
   → Home Automation executes: set_scene("evening_relaxation")
   → Receives: "Scene 'evening_relaxation' activated - warm white 40%"

5. Response
   "It's quarter to ten in the evening, sir. I've set the lights to a warm,
   subdued glow - ideal for winding down. Shall I also draw the curtains?"

Intra-System Prompts

Tatlock → Home Automation Agent:

Task: Set lighting appropriate for current time of day.
Context: Current time is 21:45 (evening). User wants lights adjusted automatically.
Action required: Activate appropriate lighting scene for evening/night.

Reference (user's original request): "Set the lights appropriately for the time of day"

---

Scenario 4: Create a Reminder (Simple Expert Delegation)

User: "Remind me to call the dentist tomorrow at 2pm"

Flow

1. Steward Analysis
   → Capabilities needed: personal_assistant (reminders)
   → Complexity: simple

2. Tatlock Execution
   <think>Creating reminder for tomorrow afternoon...</think>
   → Calls: personal_assistant_agent(task)
   → PA executes: create_reminder(title, datetime, notification_time)
   → Receives: "Reminder created: ID 847"

3. Response
   "Very good, sir. I've set a reminder to call the dentist for tomorrow at
   2 o'clock in the afternoon. I'll notify you 15 minutes beforehand."

Intra-System Prompts

Tatlock → Personal Assistant Agent:

Task: Create a reminder for the user.
Details:
- Subject: Call the dentist
- When: Tomorrow at 14:00
- Notification: 15 minutes before (default)

Reference (user's original request): "Remind me to call the dentist tomorrow at 2pm"

---

Scenario 5: Schedule Recurring Automation (Complex, Multi-Expert)

User: "Schedule the lights to turn on at 09:00 every weekend day"

Flow

1. Steward Analysis
   → Capabilities needed: home_automation (scheduling)
   → Complexity: moderate
   → Note: Recurring schedule, weekend-only

2. Tatlock Execution
   <think>Setting up recurring weekend morning lighting schedule...</think>
   → Calls: home_automation_agent(task)
   → HA executes: create_schedule(trigger, days, action)
   → Receives: "Schedule created: ID wake_light_weekend"

3. Response
   "The weekend wake-up lighting is now scheduled, sir. Every Saturday and
   Sunday at 9 o'clock sharp, the lights will illuminate. Would you prefer
   a gradual sunrise simulation, or an immediate full brightness?"

Intra-System Prompts

Tatlock → Home Automation Agent:

Task: Create a recurring lighting schedule.
Details:
- Action: Turn on lights
- Time: 09:00
- Days: Saturday, Sunday (weekends only)
- Recurrence: Weekly

Reference (user's original request): "Schedule the lights to turn on at 09:00 every weekend day"

---

Scenario 6: Create Wiki Page About Topic (Expert with Research)

User: "Create a wiki page about CI/CD"

Flow

1. Steward Analysis
   → Capabilities needed: librarian
   → Complexity: complex
   → Specific action: smart_create (research-backed page creation)

2. Tatlock Execution
   <think>Delegating to The Librarian for research and wiki page creation...</think>
   → Calls: librarian_agent(task)

3. Librarian Internal Flow
   → hybrid_search("CI/CD continuous integration deployment")
   → Synthesizes content from wiki, graph, and web sources
   → create_wiki_page(title, path, content, tags)
   → Returns summary to Tatlock

4. Tatlock Receives
   "Page created: CI/CD
    Path: /technology/cicd
    Sources used: 12 (3 wiki, 4 graph, 5 web)
    Forward links: 8, Backward links: 3"

5. Response
   "The wiki page on CI/CD has been created, sir. The Librarian has compiled
   information from 12 sources, including our existing documentation on DevOps
   practices. The page is now available at /technology/cicd and has been
   cross-linked with related topics like Jenkins, GitHub Actions, and Docker."

Intra-System Prompts

Tatlock → Librarian Agent:

Task: Create a comprehensive wiki page about CI/CD.
Context: User wants a new wiki page created. Use research capabilities to
gather information and create a well-sourced page.

Action required:
1. Research the topic using hybrid search (wiki, knowledge graph, web)
2. Create wiki page with synthesized content
3. Apply appropriate tags (suggest: technology, devops, automation)
4. Report back with page details and research summary

Reference (user's original request): "Create a wiki page about CI/CD"

Librarian → Tatlock Response:

Task completed: Wiki page created

Page Details:
- Title: CI/CD (Continuous Integration/Continuous Deployment)
- Path: /technology/cicd
- Tags: technology, devops, automation, software-development

Research Summary:
- Wiki sources: 3 pages (DevOps Overview, Jenkins Setup, GitHub Actions Guide)
- Knowledge graph entities: 4 (linked to Docker, Kubernetes, Git, Testing)
- Web sources: 5 articles (current best practices, 2024 trends)
- Total sources synthesized: 12

Knowledge Graph Updates:
- Forward links created: 8 (to existing related pages)
- Backward links created: 3 (updated existing pages to reference new page)

Content includes sections on:
- Definition and core concepts
- CI vs CD distinction
- Popular tools comparison
- Best practices
- Integration with our documented infrastructure

---

Scenario 7: Update Stock Wiki with Evaluation Tracker (Complex, Multi-Tool Expert)

User: "Update the GOOG stock wiki page with a rolling evaluation tracker"

Flow

1. Steward Analysis
   → Capabilities needed: librarian
   → Complexity: complex
   → Specific action: search existing page, fetch current data, update page

2. Tatlock Execution
   <think>This requires finding the existing page, fetching current stock data,
   and updating with a new evaluation section. Delegating to The Librarian...</think>
   → Calls: librarian_agent(task)

3. Librarian Internal Flow - Step 1
   <think>First, locating the existing GOOG wiki page...</think>
   → search_wiki("GOOG stock Alphabet")
   → Receives: Page ID 234, path /investments/stocks/goog

4. Librarian Internal Flow - Step 2
   <think>Fetching current page content to preserve existing information...</think>
   → get_wiki_page(234)
   → Receives: Current page content

5. Librarian Internal Flow - Step 3
   <think>Researching current stock data and evaluation metrics...</think>
   → hybrid_search("GOOG Alphabet stock price evaluation 2024")
   → Receives: Current price, P/E ratio, analyst ratings, etc.

6. Librarian Internal Flow - Step 4
   <think>Updating page with new rolling evaluation section...</think>
   → update_wiki_page(234, content=updated_content)
   → Receives: Update confirmation

7. Tatlock Receives
   "Page updated: GOOG Stock Analysis
    Added section: Rolling Evaluation Tracker
    Data points added: Price history, P/E trend, Analyst consensus
    Last updated: 2024-12-11"

8. Response
   "The GOOG stock page has been updated, sir. I've added a rolling evaluation
   tracker with current metrics: the stock is trading at $178.32 with a P/E
   of 24.8, and analyst consensus remains 'Buy'. The tracker includes a
   90-day price trend and quarterly earnings history. Shall I set up
   automatic weekly updates?"

Intra-System Prompts

Tatlock → Librarian Agent:

Task: Update the GOOG (Alphabet) stock wiki page with a rolling evaluation tracker.
Context: User wants to add ongoing stock evaluation tracking to an existing page.

Actions required:
1. Find the existing GOOG stock wiki page
2. Read current page content (preserve existing information)
3. Research current stock data and evaluation metrics
4. Update the page with a new "Rolling Evaluation Tracker" section including:
   - Current price and change
   - Key ratios (P/E, P/B, etc.)
   - Analyst consensus
   - Price trend (30/60/90 day)
   - Recent earnings summary
5. Report back with update summary

Reference (user's original request): "Update the GOOG stock wiki page with a rolling evaluation tracker"

Librarian → Tatlock Response:

Task completed: Wiki page updated

Page Details:
- Title: GOOG - Alphabet Inc. Stock Analysis
- Path: /investments/stocks/goog
- Page ID: 234

Update Summary:
- New section added: "Rolling Evaluation Tracker"
- Existing content: Preserved (company overview, investment thesis)

Evaluation Data Added:
- Current Price: $178.32 (+1.2% today)
- P/E Ratio: 24.8 (vs sector avg 28.3)
- P/B Ratio: 6.2
- 52-Week Range: $120.21 - $191.75
- Analyst Consensus: Buy (24 buy, 8 hold, 2 sell)
- Price Trend: +15.3% (90-day)
- Last Earnings: Q3 2024 - Beat estimates by 8%

Sources:
- Web: 4 financial data sources
- Existing wiki: Investment strategy context

Suggested follow-up: Set up automated weekly data refresh?

---

Scenario 8: Complex Multi-Expert Coordination

User: "Check if there's anything about the new project deadline in my notes, if so create a reminder, and if the deadline is this week, also update the project wiki page with a status note"

Flow

1. Steward Analysis
   → Capabilities needed: librarian, personal_assistant
   → Complexity: complex
   → Note: Conditional multi-step, requires information passing between experts

2. Tatlock Execution - Step 1
   <think>First, searching for project deadline information in the wiki...</think>
   → Calls: librarian_agent(search_task)
   → Librarian: hybrid_search("new project deadline")
   → Returns: "Project Alpha deadline: December 15, 2024 (this Friday)"

3. Tatlock Evaluation
   <think>Found deadline: December 15. That's this week (Friday).
   Need to: 1) Create reminder, 2) Update project wiki page...</think>

4. Tatlock Execution - Step 2 (parallel if possible)
   <think>Creating reminder and updating wiki status...</think>

   → Calls: personal_assistant_agent(reminder_task)
   → PA: create_reminder("Project Alpha deadline", "2024-12-15 09:00")
   → Returns: "Reminder created for Dec 15 at 9am"

   → Calls: librarian_agent(update_task)
   → Librarian: search_wiki → get_wiki_page → update_wiki_page
   → Returns: "Project Alpha page updated with deadline status note"

5. Response
   "I've found the deadline in your notes, sir - Project Alpha is due this
   Friday, December 15th. I've set a reminder for 9 o'clock that morning,
   and I've updated the project wiki page with a status note indicating
   the imminent deadline. Is there anything else you need to prepare?"

Intra-System Prompts

Tatlock → Librarian Agent (Search):

Task: Search for information about a new project deadline.
Context: User wants to find deadline information from their notes/wiki.

Action required:
1. Search wiki and knowledge base for project deadline information
2. Return: Project name, deadline date, and any relevant context

Reference (user's original request): "Check if there's anything about the new project deadline in my notes..."

Tatlock → Personal Assistant Agent:

Task: Create a reminder for a project deadline.
Details:
- Subject: Project Alpha deadline
- When: December 15, 2024 at 09:00
- Priority: High (deadline is this week)
- Notification: Morning of the deadline

Reference: Creating reminder based on deadline found in user's notes.

Tatlock → Librarian Agent (Update):

Task: Update the Project Alpha wiki page with a deadline status note.
Context: Project deadline is December 15, 2024 (this Friday). User requested
a status update since the deadline is this week.

Action required:
1. Find the Project Alpha wiki page
2. Add a status note/banner indicating the imminent deadline
3. Optionally update any status fields

Reference: Part of user's request to track and highlight near-term deadlines.

---

Response Pattern Guidelines

Tatlock's Think Updates (Streaming to User)

During multi-step operations, Tatlock should emit <think> updates to keep the user informed:

<think>Analyzing your request...</think>
<think>Searching for deadline information in the wiki...</think>
<think>Found the deadline - December 15th. Creating reminder...</think>
<think>Updating the project page with status note...</think>
<think>All tasks complete. Composing response...</think>

Tatlock's Final Response Pattern

1. Acknowledge - Confirm understanding of the request
2. Summarize actions - What was done, by whom (implicitly)
3. Key details - Important information the user should know
4. Proactive offer - Suggest related actions or follow-ups
5. Butler voice - Formal but warm, with personality

Expert Agent Response Pattern

1. Task status - Completed/Partial/Failed
2. Action summary - What was done
3. Key data - Information Tatlock needs to synthesize
4. Metadata - IDs, counts, timestamps for reference
5. Suggestions - Optional follow-up actions

---

Error Handling Scenarios

Expert Agent Failure

Tatlock → Librarian: "Create wiki page about quantum computing"
Librarian → Tatlock: "Error: library-desk API unavailable (connection timeout)"

Tatlock Response:
"I'm afraid The Librarian is having some difficulty reaching the wiki
service at the moment, sir. I can attempt a basic web search on quantum
computing if you'd like, or we can try the wiki operation again in a
few minutes."

Partial Completion

User: "Create a reminder and add it to my calendar"

Tatlock → PA: Create reminder
PA → Tatlock: "Reminder created successfully"

Tatlock → Calendar: Add to calendar
Calendar → Tatlock: "Error: Calendar sync not configured"

Tatlock Response:
"I've created the reminder, sir, but I wasn't able to add it to your
calendar - it appears the calendar integration needs to be configured.
The reminder will still alert you at the scheduled time. Shall I help
set up the calendar connection?"

---

Summary: Key Design Principles

1. Tatlock is the orchestrator - Never exposes raw tool complexity to users
2. Expert agents are tools - Tatlock calls them, they return structured responses
3. Context flows down - Each expert gets only what they need to complete their task
4. Results flow up - Tatlock synthesizes all responses into coherent butler-voice answer
5. Think updates maintain engagement - User sees progress during complex operations
6. Errors are handled gracefully - Tatlock explains and offers alternatives
7. Proactive suggestions - Tatlock anticipates follow-up needs