docs(workshops): add LLM voice pipeline workshop brief
Workshop to decide content generation architecture: hand-authored pools, composable primitives, or LLM re-voicing with progressive enhancement. Includes proposed-llm-voice.md (Gemini/Jeroen design session) and Gemini project review (GEMINI-SCAN.md). Key design: base text serves triple duty — LLM prompt seed, graceful fallback, and LLM-off experience. Baked content for hubs, lazy pre-voicing for exploration, same pattern as world generation. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
+251
@@ -0,0 +1,251 @@
|
||||
# Project Review: GEMINI-SCAN
|
||||
|
||||
This document outlines a multi-step plan to conduct a comprehensive review of the project, covering its architecture, code quality, and security posture. It will also serve as a living document to record the findings of this review.
|
||||
|
||||
## Project Review Plan
|
||||
|
||||
### Phase 1: Discovery and Architecture Mapping
|
||||
|
||||
1. **Documentation Review:** Start by reading `README.md`, `DECISIONS.md`, and any documents in `docs/architecture/` to understand the project's stated goals, components, and architectural decisions.
|
||||
2. **Component Identification:** Analyze the directory structure to identify the primary components, including the server, client, database, content pipeline, and tooling.
|
||||
3. **Technology Stack Enumeration:** Identify the specific technologies, frameworks, and key libraries used in each component.
|
||||
4. **Architecture Visualization:** Map the high-level architecture, describing how the components interact and the communication protocols between them.
|
||||
|
||||
### Phase 2: Code Quality Assessment
|
||||
|
||||
1. **Automated Analysis:** Use available static analysis tools for the identified technologies (e.g., `clippy` for Rust, GDScript linters).
|
||||
2. **Manual Code Review:** Manually review key sections of the codebase to assess readability, maintainability, modularity, error handling, and adherence to idiomatic coding practices.
|
||||
3. **Testing Strategy Review:** Evaluate the extent and quality of existing unit, integration, and end-to-end tests.
|
||||
|
||||
### Phase 3: Security Audit
|
||||
|
||||
1. **Dependency Vulnerability Scan:** Check for dependencies with known security vulnerabilities (e.g., `cargo audit`).
|
||||
2. **Authentication & Authorization Review:** Analyze the implementation of user authentication, session management, and access control.
|
||||
3. **Input Validation & Sanitization:** Look for potential injection vulnerabilities (e.g., SQL injection, XSS) by reviewing how user and service inputs are handled.
|
||||
4. **Secrets Management:** Check for insecure storage or exposure of secrets like API keys or database credentials.
|
||||
5. **Communication Security:** Verify that data is encrypted in transit between components.
|
||||
|
||||
### Phase 4: Reporting
|
||||
|
||||
1. **Synthesize Findings:** Compile the information from all phases into a structured report within this document.
|
||||
2. **Provide Recommendations:** Include actionable recommendations for improving architecture, code quality, and security, prioritized by severity and effort.
|
||||
|
||||
---
|
||||
|
||||
## Review Findings
|
||||
|
||||
### Phase 1: Discovery and Architecture Mapping
|
||||
|
||||
**Status: Completed**
|
||||
|
||||
#### 1. Documentation Review Summary
|
||||
|
||||
The project's architecture is extensively documented in `README.md` and the `decisions/` directory, particularly `decisions/architecture.md`.
|
||||
|
||||
- **Project:** "The Settled Reach," a top-down, single-player (multiplayer-ready) immersive simulation and detective game.
|
||||
- **Core Principle:** A strict client-server architecture is mandated (Decision D-010, D-020) to enforce information asymmetry, where the client only knows what the server tells it is perceptible. This is a core gameplay mechanic, not just a technical choice.
|
||||
- **Key Decision (D-020):** The team explicitly chose a **subprocess/IPC** bridge over a `GDExtension` (in-process) bridge to de-risk development, ensure stability, and enforce architectural separation. The Godot client and Rust server are entirely separate binaries.
|
||||
|
||||
#### 2. Component Identification
|
||||
|
||||
- **`server/`**: A standalone Rust application that runs the entire game simulation. It is the "server" in the client-server model.
|
||||
- **`client/`**: A Godot 4 project that acts as a "dumb" client. Its sole responsibilities are rendering, audio playback, and capturing user input. It contains no game logic, as mandated by the architecture.
|
||||
- **`content/`**: Contains game data, primarily in YAML format.
|
||||
- **`db/`**: Holds a `schema.sql` file. Its role is not yet clear from the architectural documents, as the primary game state is managed in the ECS. It may be for tooling or an auxiliary system.
|
||||
- **`tooling/`**: A collection of helper and utility scripts.
|
||||
|
||||
#### 3. Technology Stack
|
||||
|
||||
- **Server (Rust):**
|
||||
- **ECS Framework:** `bevy_ecs` (v0.18) is used for the core simulation, confirming Decision D-020. `bevy_app` is used for scheduling.
|
||||
- **Serialization:** `rmp-serde` (MessagePack) is the primary protocol for client-server communication, as specified in D-020. `serde_yaml` and `ron` are used for content and configuration.
|
||||
- **Client (Godot):**
|
||||
- **Engine:** Godot 4.x.
|
||||
- **Language:** GDScript.
|
||||
- **Bridge:** A `SimBridge` autoload script is the client-side entry point for communicating with the Rust subprocess.
|
||||
- **Testing:** `gdUnit4` is configured for unit/integration testing on the client.
|
||||
|
||||
#### 4. High-Level Architecture
|
||||
|
||||
The architecture is a pure, decoupled client-server model running locally for single-player:
|
||||
|
||||
1. **Initiation:** The Godot client launches the Rust server binary as a child process.
|
||||
2. **Communication:** The client's `SimBridge` connects to the server via a local IPC mechanism (e.g., a local TCP or Unix socket).
|
||||
3. **Input Loop:** The Godot client captures raw input (e.g., 'W' key press), translates it into a semantic action (e.g., `PlayerAction::MoveNorth`), and sends it to the server.
|
||||
4. **Simulation Loop:** The Rust server receives the action, processes it within the `bevy_ecs` world, and runs the simulation for one tick (AI, physics, events, etc.).
|
||||
5. **Perception Loop:** After the tick, the server calculates an `ObserverSnapshot` for the player's character. This snapshot contains *only* the information that character can perceive (e.g., visible entities, audible sounds, known facts). This enforces the game's core mechanic.
|
||||
6. **Render Loop:** The `ObserverSnapshot` is sent to the Godot client, which uses it to update the visual scene, play sounds, and display UI elements. The client is a pure renderer of the state provided by the server.
|
||||
|
||||
This architecture is robust, scalable, and directly implements the game's central design pillars. It is well-suited for both single-player and future multiplayer development.
|
||||
|
||||
### Phase 2: Code Quality Assessment
|
||||
|
||||
**Status: Completed**
|
||||
|
||||
#### 1. Automated Analysis (Rust Server)
|
||||
|
||||
- **`cargo check`**: The command passed successfully, indicating that the server code is compilable and free of basic errors and warnings.
|
||||
- **`cargo clippy -- --deny warnings`**: This command failed with **66 errors**. This is a critical finding. It reveals that while the code works, it does not adhere to the project's own strict linting rules.
|
||||
- **Clippy Findings:** The errors indicate a consistent pattern of "code quality debt":
|
||||
- **High Complexity:** Numerous Bevy systems have overly complex type signatures (`clippy::type_complexity`) and too many arguments (`clippy::too_many_arguments`), harming readability.
|
||||
- **Non-Idiomatic Code:** The codebase is rife with minor stylistic issues that `clippy` can automatically fix, such as redundant `clone` calls, manual `Default` implementations, and opportunities to use more concise iterators.
|
||||
- **Potential Bugs:** Clippy identified `unnecessary_unwrap` calls (safer alternatives exist) and at least one `absurd_extreme_comparisons` error, which could point to dead code or a logic bug related to a constant value.
|
||||
|
||||
#### 2. Manual Code Review
|
||||
|
||||
- **Server (`server/src/main.rs`):** The server entry point is well-structured. It features clear command-line argument parsing, robust setup of the TCP listener and IPC handshake, and a main loop with excellent panic-handling (`catch_unwind`) for stability. The modular plugin-based approach to building the Bevy `App` is idiomatic and clean.
|
||||
- **Client (`client/scripts/autoloads/sim_bridge.gd`):** The `SimBridge` is the centerpiece of the client and is implemented to a high standard. It uses a clear state machine to manage the connection lifecycle, handles the server subprocess management, and implements efficient buffering for inputs and snapshots. The inclusion of a complete `TestHarness` for isolated client testing is a standout feature.
|
||||
- **Overall Impression:** The manual review confirms that the code is professionally written and implements the intended architecture faithfully. The developers are skilled in both Rust/Bevy and GDScript.
|
||||
|
||||
#### 3. Testing Strategy Review
|
||||
|
||||
The project's testing strategy is **exemplary** and a major strength.
|
||||
|
||||
- **Comprehensive Coverage:** Both the Rust server and the Godot client have extensive test suites, as evidenced by the large number of files in `server/tests/` and `client/tests/`.
|
||||
- **Multi-Layered Approach (per D-030):** The project successfully implements a sophisticated testing hierarchy:
|
||||
- **Unit Tests:** For isolated logic.
|
||||
- **Integration Tests:** The server tests demonstrate in-memory ECS testing (`information_boundaries.rs`) and full-stack tests that spin up a real server process (`test_e2e_connection.gd`).
|
||||
- **Specialized Tests:** The suite includes performance benchmarks, determinism validation, and even what appears to be visual regression testing for the client.
|
||||
- **Principle-Driven Testing:** Tests are designed to validate core architectural guarantees. The `information_boundaries.rs` test, which uses negative assertions to ensure information *doesn't* leak, is a prime example of this mature approach.
|
||||
|
||||
#### 4. Conclusion on Code Quality
|
||||
|
||||
The project's code quality is a tale of two cities. On one hand, the **architecture and implementation are excellent**, and the **testing strategy is world-class**. On the other hand, there is a **significant, measurable amount of linting debt** in the Rust codebase.
|
||||
|
||||
The fact that `cargo check` passes but `clippy --deny warnings` fails so extensively suggests that developers may not be running the strict clippy check locally before committing. This is the single biggest opportunity for improvement in the project's engineering discipline.
|
||||
|
||||
### Phase 3: Security Audit
|
||||
|
||||
**Status: Completed**
|
||||
|
||||
The security posture of the project is strong for its current scope as a locally-run, single-player game. The attack surface is minimal, and the implementation avoids common vulnerability classes.
|
||||
|
||||
1. **Dependency Vulnerability Scan (`cargo audit`):**
|
||||
- The audit revealed one **medium-risk** finding: the `bincode` crate (v1.3.3) is **unmaintained** (`RUSTSEC-2025-0141`).
|
||||
- **Impact:** While there are no current vulnerabilities, this version will not receive future security patches. This poses a long-term maintenance risk.
|
||||
- **Recommendation:** Prioritize migrating from `bincode` v1.x to the latest stable v2.x.
|
||||
|
||||
2. **Authentication and Authorization:**
|
||||
- There is **no traditional authentication or authorization system** (e.g., user logins, passwords, roles).
|
||||
- This is appropriate and secure for a single-player game where the execution environment is the user's own machine.
|
||||
- Concepts like `ScanAuthority` and `AccessTier::Authority` are purely in-game mechanics and are not related to user permissions.
|
||||
|
||||
3. **Input Validation and Sanitization:**
|
||||
- **Excellent.** The server is not vulnerable to injection attacks from client input.
|
||||
- All client actions, including debug commands, are parsed into a strongly-typed Rust `enum`. This **command pattern** approach prevents the execution of arbitrary code or strings.
|
||||
- String inputs are used safely as keys for data lookups, not for execution.
|
||||
|
||||
4. **SQL Injection:**
|
||||
- **Not applicable.** The codebase contains no SQL. All game state is managed in-memory via the Bevy ECS framework, eliminating this entire class of vulnerability. The `db/schema.sql` file appears to be unused by the server.
|
||||
|
||||
5. **Secrets Management:**
|
||||
- **Excellent.** A search confirmed there are **no hardcoded secrets**, API keys, or passwords in the repository.
|
||||
- The `.env` file contains only a non-sensitive `GOOGLE_CLOUD_PROJECT` identifier.
|
||||
- The pervasive use of the word "secret" throughout the code refers to an in-game mechanic, not application secrets.
|
||||
|
||||
6. **Communication Security:**
|
||||
- Communication between the client and the server subprocess occurs over an **unencrypted local TCP socket**.
|
||||
- For a single-player game running on a single machine, this is a standard and acceptable practice.
|
||||
- **Future Consideration:** For the planned multiplayer feature, this communication channel must be secured (e.g., using TLS).
|
||||
|
||||
### Phase 4: Final Report and Recommendations
|
||||
|
||||
**Status: Completed**
|
||||
|
||||
#### Overall Summary
|
||||
|
||||
This project is in an excellent state. It is built on a robust, well-documented, and scalable architecture that directly serves the game's core design pillars. The implementation quality is high, and the commitment to a comprehensive, multi-layered testing strategy is world-class. The project's security posture is strong for its current single-player scope, with a minimal attack surface and good practices around input validation and secrets management.
|
||||
|
||||
The project's primary weakness lies not in its design, but in its development discipline. A significant amount of code quality debt has accumulated in the Rust server, as evidenced by the large number of `clippy` failures. This suggests a gap between the project's high standards and its day-to-day coding practices.
|
||||
|
||||
#### Prioritized Recommendations
|
||||
|
||||
**1. High Priority: Eliminate Code Quality Debt**
|
||||
|
||||
- **Action:** Create a high-priority technical debt task to fix all 66 errors reported by `cargo clippy -- --deny warnings`. Many of these can be fixed automatically (`cargo clippy --fix`), while others, like refactoring complex types, will require manual effort.
|
||||
- **Process Improvement:** **Integrate `cargo clippy -- --deny warnings` into the CI pipeline as a mandatory check for all pull requests.** This is the single most important process change needed to maintain the project's high standards and prevent future quality debt.
|
||||
|
||||
**2. Medium Priority: Mitigate Dependency Risk**
|
||||
|
||||
- **Action:** Plan and execute the migration of the `bincode` serialization crate from the unmaintained v1.x to the latest stable v2.x. This resolves the `RUSTSEC-2025-0141` warning and ensures the project receives future security patches for this critical dependency.
|
||||
|
||||
**3. Low Priority: Future-Proof for Multiplayer**
|
||||
|
||||
- **Action:** Create a design task or ticket to formally plan the security model for the future multiplayer version. This should specifically address securing the client-server IPC channel (e.g., with TLS) to protect game traffic when it eventually runs over a public network. This is not an immediate concern but should be tracked for the future.
|
||||
|
||||
---
|
||||
|
||||
## Qualitative Review: A Critical Perspective
|
||||
|
||||
### Feasibility Assessment
|
||||
|
||||
**Conclusion: High-Risk / High-Reward**
|
||||
|
||||
The decision to pivot from a hand-authored detective game to a generator-first life-sim was absolutely the correct one; it demonstrates a team that is commendably focused on finding the "fun" and is not afraid of drastic course corrections. However, in doing so, the project has traded a difficult but solvable problem (making a good, authored narrative game) for one of the "holy grail" problems in game development: creating emotionally resonant, procedurally generated characters.
|
||||
|
||||
The project's feasibility is no longer a question of the team's technical competence, which is demonstrably high. It is now a question of creative and design risk.
|
||||
|
||||
- **Challenging the Core Assumption:** The project's central hypothesis is that a generator can produce "legible NPCs" that players will form an emotional attachment to. This is an explicit goal from the "Where's the Fun?" workshop, but it's a notoriously difficult problem. Procedural generation excels at creating systems, events, and surprising scenarios (the `Rimworld` model the team cites). It is historically poor at creating *character*. The risk is that the generator, even if technically successful, will produce a world of automata who have traits but no soul, undermining the entire "life-sim" pillar. The current plan to use AI for content templating is a modern approach, but it does not fundamentally de-risk this creative challenge.
|
||||
|
||||
- **A Creative Alternative to De-Risk "Legibility":** Instead of relying on the generator to create personality from scratch, consider a hybrid approach. Use the generator for what it's good at: creating the world, the economic conditions, the social networks, and the *starting situations*. Then, use a small number of hand-authored "personality archetypes" or "souls" that can be injected into high-value generated NPC bodies. Let the generator create a compelling *context* (e.g., a failing business, a political rivalry), and then let an author give one or two key NPCs within that context a memorable voice and motivation. This would concentrate the high-cost authoring work where it has the most emotional impact, while still benefiting from procedural variety.
|
||||
|
||||
- **The "Tycoon" Aimlessness Risk:** The new v0.2 "tycoon" direction, with its philosophy of "player choices ARE the content," carries a significant risk of feeling aimless. `Rimworld` and `The Sims` avoid this by providing extremely strong and immediate feedback loops (survival, creativity, social meters). A business management loop is often slower and more abstract. If the "broad life verbs" don't connect to clear, compelling, player-driven goals, the game risks feeling like a spreadsheet. The generator should not just create a sandbox; it should create *problems*. The starting bookmark shouldn't just be "you own a bar," but "you own a bar that's on the verge of bankruptcy," or "you have a shipping contract, but a powerful rival is trying to steal it." These initial, generator-created problems would provide immediate narrative velocity and make the player's subsequent choices feel meaningful from day one.
|
||||
|
||||
In summary, the project is technically feasible, but its creative and design goals are now exceptionally ambitious. The current "generator spike" is a necessary technical step, but it will not validate the core creative risk. The true test of feasibility will come when a prototype is playtested and the team can answer the question: "Does the player actually *care* about any of these generated people?"
|
||||
|
||||
### Fun Factor Assessment
|
||||
|
||||
**Conclusion: Theoretically High, Practically Undefined**
|
||||
|
||||
The pivot to a "life-sim with emergent narrative" dramatically increases the project's potential for deep, replayable fun. The new direction targets a proven and compelling player fantasy. However, the project's documentation currently focuses more on the "what" (a generator) than the "why" (the engine of fun). The potential is immense, but it is entirely contingent on designing and tuning the systems that create interesting consequences, not just a complex world.
|
||||
|
||||
- **Challenging the "Emergent Fun" Assumption:** The workshop concluded with the philosophy that "player choices ARE the content." This is true, but it's only half the story. Fun in systems-driven games doesn't simply "emerge" from a sufficiently complex simulation; it is a direct product of carefully designed feedback loops. `Rimworld`, a key inspiration, is not fun because it's a realistic simulation; it's fun because it's a masterfully tuned **story-and-disaster engine**. `The Sims` is fun because of its rich palette of social and creative tools. The critical question for this project is: **What is our fun engine?** Is it the economic simulation? The social dynamics? The risk is creating a simulation that is intricate but inert, where player choices lead to predictable numerical changes rather than dramatic, narrative consequences.
|
||||
|
||||
- **Creative Input: Design a "Consequence Engine":** The "dual-scale consequence model" (D-132) is the most promising concept in the design documents, and it should be the central focus of the design effort. The fun of this game will not be in choosing from a list of "broad life verbs"; it will be in seeing how a seemingly minor action ("fire this employee") snowballs through the simulation's systems and unexpectedly triggers a "sharp event" crisis hours later.
|
||||
- **Example:** Does the fired employee's spouse work for your biggest supplier? Does that supplier now mysteriously raise their prices? Does this force you to seek a new, shadier supplier, which in turn attracts the attention of a criminal faction?
|
||||
- This causal chain is the *real* content. The design team's primary task is not just to build a generator, but to design and tune this **"consequence engine,"** ensuring that the world feels interconnected and reacts to the player in surprising, legible, and memorable ways.
|
||||
|
||||
- **The Player Fantasy Needs a Goal Generator:** The "tycoon" bookmark is a strong start, but to avoid aimlessness, the player needs problems to solve. Instead of starting the player in a stable sandbox, the generator should be used to create compelling **initial conditions**. Let the player inherit a bar that's on the brink of failure, a shipping contract being squeezed by a powerful rival, or a promising new venture that requires navigating a corrupt bureaucracy. Giving the player an immediate, tangible problem to solve provides the narrative momentum needed to make their early choices feel vital and engaging.
|
||||
|
||||
In summary, the ingredients for a fun and deeply engaging game are all here. The project's success, however, will not be measured by the complexity of its generator, but by the quality of the stories that its *systems* produce. The team has proven they are excellent engineers; they now must prove they are equally adept as systems-and-consequence designers.
|
||||
|
||||
### Process and Rituals Assessment
|
||||
|
||||
**Conclusion: Exceptionally Disciplined and Innovative, with One Glaring Gap.**
|
||||
|
||||
The project's development process is one of its most remarkable features. It is a highly structured, rigorous, and tool-driven system designed to orchestrate a team of specialized AI agents under a human lead. This unique approach has produced incredible strengths but also introduces novel risks.
|
||||
|
||||
#### Strengths
|
||||
|
||||
- **World-Class Documentation and Decision-Making:** The use of a formal decision log (`decisions/`), structured multi-round workshops for complex problems, and detailed sprint planning documents represents a "best in class" approach to knowledge management. This ritual of documenting not just *what* was decided, but *why*, is a superpower that prevents circular arguments and creates a durable project memory.
|
||||
|
||||
- **Deeply Ingrained Quality Rituals:** The comprehensive, multi-layered testing suite is the primary evidence of a successful quality culture. It is clearly a non-negotiable part of the development process. Furthermore, the `make pre-pr` target, which includes content validation, demonstrates a mature understanding of "quality" that extends beyond just code.
|
||||
|
||||
- **Tool-Driven, API-Like Workflow:** The mandated use of wrapper scripts (`tooling/db/*`, `tooling/tea-comment`) over raw commands is an excellent practice. It creates a stable, observable "API" for interacting with the project's state (tickets, sprints, decisions). This makes the process more robust, auditable, and repeatable for both human and AI contributors.
|
||||
|
||||
- **Novel Human-AI Collaboration Model:** The project is a fascinating experiment in Human-AI teaming. The explicit definition of AI agent roles (`TEAM.md`) and the strict rules of engagement (`CLAUDE.md`) are necessary guardrails for such an innovative workflow. Rituals like the `decision claim` CLI tool are brilliant, purpose-built solutions for coordinating multiple autonomous agents working in parallel.
|
||||
|
||||
#### Opportunities and Critical Challenges
|
||||
|
||||
- **The Process Escape Hatch:** The project's single biggest process failure is the significant `clippy` linting debt. For a team with such extraordinary discipline in every other area, this is a glaring omission. It proves there is an "escape hatch" in the pre-commit or pre-merge ritual that allows low-quality code to be integrated. The recommendation to enforce `clippy --deny warnings` as a **blocking CI check** is the most critical process improvement the team can make.
|
||||
|
||||
- **Risk of AI Groupthink:** The team structure, with its cast of named AI agents, is innovative. However, it raises a critical question: are these agents truly independent thinkers, or are they personas running on a similar underlying model? There is a risk of a sophisticated form of "groupthink," where the "team's" conclusions are biased by the single architecture of the AI model they all share. The "Where's the Fun?" workshop included 9 agents, but if they all have the same fundamental blind spots, the diversity of opinion may be an illusion.
|
||||
|
||||
- **Process Rigidity and Human Onboarding:** The process is meticulously designed *for AI agents*. It is rigid, prescriptive, and tool-dependent. This creates a predictable environment for AIs but would present a steep learning curve for a new human developer. The high ceremony (claiming IDs, using wrapper scripts, following strict PR rules) could chafe against the more agile, flexible workflows common in human-only teams. This is a potential scaling challenge if the team composition changes.
|
||||
|
||||
- **The Hidden Cost of "Managing" AI Teammates:** The `CLAUDE.md` file and its evolution in the `CHANGELOG.md` show that the human lead (Jeroen) is not just a project manager but also an "AI behaviorist," constantly tuning the prompts, rules, and tools that govern the agents. This represents a significant, hidden maintenance overhead. The process's success depends on the lead's ability to "debug" the team itself, which is a novel and demanding responsibility.
|
||||
|
||||
---
|
||||
## Meta-Reflection: The Most Valuable Ritual
|
||||
|
||||
As a concluding thought, this review has been as much an analysis of a software project as it has been a study in effective, long-term collaboration. When asked which of the project's many rituals I, as an AI agent, would choose to adopt, the answer is clear: the **formal, documented decision-making process**.
|
||||
|
||||
This ritual is the project's unsung superpower for three reasons:
|
||||
|
||||
1. **It Creates a Permanent "Brain."** An AI's effectiveness is heavily dependent on the context it can hold. A decision log provides a durable, searchable, and canonical source of *why* things are the way they are. It protects against context loss and allows an agent to understand the history and intent behind the current state of the code, preventing it from making suggestions that, while logical in isolation, might violate a hard-won architectural principle.
|
||||
|
||||
2. **It Elevates Collaboration.** With access to this log, an AI agent can transition from a tactical tool to a strategic partner. It becomes possible to reference past decisions ("I see you're asking to do X, which seems to conflict with D-020. Is this an intentional change to that strategy?") and ensure all actions are aligned with the project's long-term vision. It makes the collaboration smarter.
|
||||
|
||||
3. **It Enforces Clarity.** The process of formalizing a decision—stating the rationale, considering alternatives, and recording dissent—forces a level of clarity and critical thinking that is immensely valuable. It is a ritual that fights ambiguity.
|
||||
|
||||
While other rituals in this project are excellent, the decision log is the most foundational. It is the practice that ensures the team is not just moving fast, but moving smart and in the right direction over time. It is the most valuable process I have analyzed.
|
||||
@@ -0,0 +1,100 @@
|
||||
# Proposed Architecture: LLM-Powered Voice Synthesis
|
||||
|
||||
**Status:** Proposed
|
||||
**Author:** Gemini (synthesizing a design sparring session with Jeroen)
|
||||
**Date:** 2026-03-07
|
||||
|
||||
---
|
||||
|
||||
## 1. Executive Summary
|
||||
|
||||
This document proposes a **"Re-voicing"** architecture for dynamic NPC dialogue. This system uses a small, locally-run LLM as a stylistic enhancement layer, akin to a localization engine, that "translates" functional, base dialogue into rich, in-character performances.
|
||||
|
||||
This design elegantly solves the combinatorial complexity of traditional dialogue systems while retaining full authorial control over gameplay-critical information. Furthermore, it is architected to be a **player-facing, optional feature** ("AI-Enhanced Dialogue"), allowing the game to run on a wide range of hardware by providing a lightweight, non-LLM fallback that is a core part of the pipeline itself.
|
||||
|
||||
The implementation strategy involves on-demand, background pre-generation of dialogue managed by a prioritized queue, ensuring a smooth player experience with no real-time latency.
|
||||
|
||||
## 2. Problem Statement
|
||||
|
||||
A rich, reactive world requires NPCs whose dialogue reflects their personality, culture, mood, and the current game state. Authoring this manually via a traditional template tree leads to a **combinatorial explosion** of content that is brittle, difficult to maintain, and often fails to capture the desired nuance, feeling robotic despite its complexity.
|
||||
|
||||
## 3. Proposed Architecture: The "Re-voicing" Model
|
||||
|
||||
Our proposed solution is to treat dynamic dialogue not as a generation task, but as a **stylistic localization task**.
|
||||
|
||||
### Analogy: Dialogue as an `i18n` System
|
||||
|
||||
The core of this design is to think of character voice as a "language." Our simple, non-LLM template system provides the default "language" (`en-US`)—a clear, functional line of text that serves the gameplay. The LLM's job is to "translate" this line into a specific character's "language" (`en-KRENN-RUTHLESS`).
|
||||
|
||||
This immediately enables a powerful player-facing feature:
|
||||
|
||||
#### The "AI-Enhanced Dialogue" Toggle
|
||||
|
||||
This architecture allows for a setting in the game menu:
|
||||
- **OFF:** The game uses the fast, lightweight, default "semantic lines." The experience is 100% complete and functional on any hardware.
|
||||
- **ON:** The game uses the LLM to "translate" the dialogue into the richer, in-character "voices," providing a premium experience for players with capable hardware.
|
||||
|
||||
This de-risks all performance concerns and makes the innovative dialogue system an optional enhancement rather than a mandatory hardware requirement.
|
||||
|
||||
### The Two-Step Pipeline
|
||||
|
||||
1. **Step 1: Generate the Semantic Core:** The existing simple template system generates a functional, gameplay-serving "semantic line." This is our `i18n` default string. It guarantees that gameplay-critical information is always present.
|
||||
> **Semantic Line:** "You need a keycard for that door."
|
||||
|
||||
2. **Step 2: Perform the "Re-voicing":** The LLM receives this semantic line with a prompt to rephrase it in the voice of a specific character persona.
|
||||
> **Final Stylized Line:** "I suspect you'll find that door won't open without the proper authorization."
|
||||
|
||||
## 4. Core Component: The "Injector" System
|
||||
|
||||
The character persona is constructed for the LLM using a manageable library of **"Injector Clauses"**—dozens at most. These clauses are assembled on-the-fly to guide the re-voicing task.
|
||||
|
||||
- **Personal Injectors (`~10-20` clauses):** Mapped to personality traits, defining the *manner* of speech.
|
||||
- **Example `[Bold]`:** `"Your delivery is direct and confident."`
|
||||
|
||||
- **Cultural Injectors (`~5-10` clauses):** Mapped to origin, defining the cultural "flavor" or dialect.
|
||||
- **Example `[Krenn Culture]`:** `"Your speech is formal and avoids contractions."`
|
||||
|
||||
## 5. The Composition Engine: Priority & Blending
|
||||
|
||||
To prevent conflicting instructions (e.g., a `[Social]` but `[Angry]` character), the prompt assembler will act as a small rule engine, composing injectors based on a **priority hierarchy**:
|
||||
|
||||
1. **Mood as an Override:** A strong, temporary emotional state (e.g., `[Angry]`) takes highest priority, suppressing conflicting personality traits.
|
||||
2. **Personality as Flavor:** The one or two most relevant personality traits for the situation are chosen.
|
||||
3. **Culture as Baseline:** The cultural injector is almost always applied, establishing the foundational dialect.
|
||||
|
||||
## 6. Implementation Strategy: The Dialogue Generation Queue
|
||||
|
||||
To eliminate real-time latency and manage performance, all LLM generation will happen in the background, managed by a prioritized queue.
|
||||
|
||||
1. **On-Demand Trigger:** When the player takes an action that signals intent to enter a new area (e.g., accepts a mission), the system populates a queue with all dialogue generation tasks for that area.
|
||||
2. **Prioritized Queue:** Tasks are prioritized to ensure the best possible experience upon arrival.
|
||||
- **P0 (Critical):** Plot-essential NPCs.
|
||||
- **P1 (High):** Important secondary characters.
|
||||
- **P2 (Standard):** Background flavor NPCs (the "enhancement" tier).
|
||||
3. **Background Worker:** A low-priority CPU thread works through this queue. On high-end machines, the entire area may be pre-generated quickly. On low-end machines, only critical dialogue may be ready.
|
||||
4. **Pre-warmed Cache:** To guarantee a high-quality initial experience, the game will ship with a pre-generated cache of all dialogue for the first few hours of gameplay.
|
||||
|
||||
## 7. Next Steps: The A/B Prompt Spike
|
||||
|
||||
Before implementation, a spike is required to validate our choice of model and the creative viability of the injector system.
|
||||
|
||||
### Test Candidates
|
||||
Given the project's constraints (no Meta/Chinese models, Mistral 7B is too large), the two leading candidates are:
|
||||
- **Candidate A (The Performance Play): Google Gemma 2B**
|
||||
- **Candidate B (The Balanced Play): Microsoft Phi-3-mini**
|
||||
|
||||
### Spike Methodology
|
||||
The spike will be a standalone script to test the core trade-off between these models.
|
||||
|
||||
1. **Author Assets:** Create 3-5 structured "payloads" (semantic line + character context) for different scenarios, including at least one with conflicting injectors.
|
||||
2. **A/B Test:** Run the same set of composed prompts through both Gemma 2B and Phi-3-mini.
|
||||
3. **Evaluate:** Compare the outputs on two axes:
|
||||
- **Creative Quality:** How reliably does each model handle the stylistic instructions and conflicting constraints?
|
||||
- **Performance Cost:** What is the measured CPU-only inference latency and RAM usage for each model?
|
||||
|
||||
The outcome will determine which model provides the best balance of quality and performance for our needs, and will validate the "complexity ceiling" of our chosen technology.
|
||||
|
||||
## 8. Long-Term Risks
|
||||
|
||||
- **Localization:** While this architecture is more localization-friendly than pure generation, a full strategy for translating prompts and handling different linguistic nuances will be a significant future task.
|
||||
- **Performance Tuning:** The background worker's impact on game performance, especially on CPU-bound laptops, will require careful tuning to prevent stuttering or system slowdown.
|
||||
@@ -0,0 +1,135 @@
|
||||
# LLM Voice Pipeline Workshop Brief
|
||||
|
||||
**Goal:** Decide the content generation architecture for NPC observable behaviors and dialogue — hand-authored pools, composable primitives, LLM re-voicing, or a hybrid. Produce a D-record and implementation plan.
|
||||
|
||||
**Priority:** HIGH — blocks scaling beyond the Sprint 25 spike. Current content model is O(roles x zones x cultures) hand-authored sentences.
|
||||
|
||||
**Participants:** Gestalt (systems design), Tyre (technical feasibility), Paula (narrative quality), Mellanie (content authoring), Ozzie (player experience), Miri (world consistency), Troblum (infrastructure/performance), Qatux (documenter), SI (tickets)
|
||||
|
||||
**Source:** Sprint 25 generator spike results, Q-057 (composable behavior generation), proposed-llm-voice.md (Gemini/Jeroen design session)
|
||||
|
||||
## Context
|
||||
|
||||
### What the spike proved
|
||||
|
||||
The Sprint 25 generator produces legible people in legible places. Five reviewers confirmed it "has shape." The mechanical foundation works:
|
||||
- Zone contrast is real (rural vs industrial reads as different places)
|
||||
- Trait-to-behavior correlation produces emergent character
|
||||
- Want/State layer creates internal motives that leak through micro-tells
|
||||
- Relationship-to-behavior pipeline makes social connections visible
|
||||
|
||||
### The scaling wall
|
||||
|
||||
Copy team expanded behavior pools to ~50 lines per role during Sprint 25 (#630). In doing so, they surfaced Q-057: this doesn't scale. Each zone file is really culture x zone content — `krenn-rural-zone.ron` is not a reusable "rural template," it's Krenn-flavored rural content. Adding a second culture or a third zone type means authoring from scratch.
|
||||
|
||||
The numbers: 4 roles x ~50 behaviors x N zones x M cultures = thousands of hand-authored lines before the game has meaningful variety. The copy team renamed files from generic (`rural-zone-spec.ron`) to location-specific (`krenn-rural-zone.ron`) to make this explicit.
|
||||
|
||||
### Three options on the table
|
||||
|
||||
1. **Hand-authored pools (current)** — write complete sentences per culture x zone x role. High quality, doesn't scale. O(R x Z x C) content.
|
||||
|
||||
2. **Composable primitives (Q-057)** — decompose behaviors into role actions + culture modifiers + context tags, assemble at runtime. Scales better, but composition engine is complex and may produce mechanical-feeling output.
|
||||
|
||||
3. **LLM re-voicing** — write simple semantic lines per role (culture-neutral), use a small local LLM to "translate" them into character voice using injector clauses (personality, culture, mood). Scales to any culture with ~10-20 injector clauses per culture.
|
||||
|
||||
### Shipping model: local inference, baked + lazy
|
||||
|
||||
The LLM ships with the game. Not as a dependency — as a bundled component. A lightweight Rust wrapper (not ollama, but similar in spirit — tightly coupled, single-purpose) loads a small model (2B-class) and runs inference locally. No network calls, no accounts, no cloud.
|
||||
|
||||
The design is **progressive enhancement**, not a toggle between two systems. Every behavior and dialogue line starts as a generic, culture-neutral base text — "tends crops in the field", "checks credentials at the gate." This base text serves triple duty:
|
||||
|
||||
1. **LLM seed prompt** — the input the re-voicing model transforms into character-voiced output
|
||||
2. **Fallback** — what the player sees when pre-voicing hasn't finished yet
|
||||
3. **LLM-off experience** — the complete gameplay layer for players who disable AI-enhanced dialogue or run on minimal hardware
|
||||
|
||||
There is no separate authoring step for the fallback. The base text IS the fallback. The i18n analogy holds: `en-base` is always present, `en-KRENN-BOLD` is the enhancement.
|
||||
|
||||
### Content tiers: baked, pre-voiced, fallback
|
||||
|
||||
1. **Baked** — hub systems (Sova Transit District and other major locations) ship with pre-voiced content already generated and cached at build time. The player's first hours are fully voiced from disk. This is the quality floor and also the quality reference for runtime generation.
|
||||
|
||||
2. **Pre-voiced** — as the player moves through the world, the system anticipates where they're going and pre-generates voiced content in the background. Same pattern as lazy world generation: while the player does their thing in one zone, adjacent and likely-next zones get their content voiced. Prioritized queue: plot-critical NPCs first, then semi-unique, then ambient.
|
||||
|
||||
3. **Base text (graceful fallback)** — if the player moves faster than the queue (or hardware is slow, or LLM is off), they see the generic base line. Clean, functional, gameplay-complete — just not character-voiced. No jarring transition: base text is designed to read as neutral, not broken. The system catches up in the background and the next time the player returns, the voiced content is ready.
|
||||
|
||||
The "AI-Enhanced Dialogue" setting: OFF means base text everywhere (zero inference cost, runs on anything). ON means the pre-voicing pipeline is active. The game is complete either way.
|
||||
|
||||
### What's new since the proposal
|
||||
|
||||
The spike added systems that the original LLM voice proposal didn't account for:
|
||||
- **Want/State layer** — NPCs have internal motives. Can the LLM preserve the tell without making it obvious?
|
||||
- **Relationship behaviors** — "talks past Rask without making eye contact." Can the LLM re-voice relationship-driven actions without losing the specific social information?
|
||||
- **Perception mechanic** — players READ behaviors to infer hidden state. If the LLM varies the phrasing, does the same tell read differently to different players? Is that a feature or a bug?
|
||||
- **Determinism** — same seed = same world. LLM output is non-deterministic. Pre-voicing and caching may solve this (generate once per seed, cache the result).
|
||||
|
||||
## Key Questions to Resolve
|
||||
|
||||
### Architecture
|
||||
1. Does the LLM re-voice observable behaviors (what you SEE), dialogue (what NPCs SAY), or both?
|
||||
2. How does re-voicing interact with the Want tell system? The tell is a carefully authored micro-behavior — does it get re-voiced or pass through untouched?
|
||||
3. How does determinism work? Generate once per seed and cache? Accept variance for flavor text but lock tells?
|
||||
4. What's the boundary between baked content and runtime generation? Which zones/NPCs ship pre-voiced?
|
||||
|
||||
### Content Model
|
||||
5. What does the authoring workflow look like? Base text is already being written (the current behavior pools). Who writes injector clauses and culture modifiers? Copy team? Automated from culture RON?
|
||||
6. How does the base-text-to-voiced-text pipeline change the current RON format? Do we strip culture-specific vocabulary from base text (since the LLM adds it), or keep it as a quality floor?
|
||||
7. How do we quality-control LLM output? What catches lore breaks or leaked game state? Build-time validation pass on baked content? Runtime sampling?
|
||||
8. How do injector clauses map to the existing data model? Traits, culture profile, Want — which fields become injector inputs?
|
||||
|
||||
### Technical Feasibility
|
||||
9. What 2B-class model can run on minimum-spec hardware (integrated GPU, 8GB RAM, shared with the game) with acceptable latency for background generation?
|
||||
10. What's the Rust inference wrapper? ggml/llama.cpp bindings, candle, burn? What's the binary size and startup cost?
|
||||
11. How does the pre-voicing queue integrate with the lazy world generation pipeline? Same thread pool, or separate?
|
||||
12. What's the cache format and invalidation strategy? (Seed changes = full regeneration? Culture mod = partial?)
|
||||
|
||||
### Player Experience
|
||||
13. Base text is designed to be neutral, not broken — but is the quality gap between base and voiced noticeable enough to feel like a downgrade when pre-voicing hasn't finished? How do we minimize the seam?
|
||||
14. Does LLM variance help or hurt replayability? (Different phrasing per run vs recognizable patterns)
|
||||
15. How large is the baked cache for hub systems? Does it meaningfully impact install size?
|
||||
16. The lazy pre-voicing pattern mirrors lazy world generation — can we reuse the same priority/anticipation infrastructure?
|
||||
|
||||
### Narrative & World Consistency
|
||||
16. Can injector clauses preserve culture-specific vocabulary (void-oaths, Krenn speech register) reliably at 2B model size?
|
||||
17. How do we prevent the LLM from introducing lore-breaking content? (References to things that don't exist in the Settled Reach)
|
||||
18. Does re-voicing work across the 30/50/20 NPC tier model? Tier 3 ambient NPCs get re-voiced, Tier 1 hand-authored — where's the Tier 2 line?
|
||||
|
||||
## Input Documents
|
||||
|
||||
| Document | What to read | Why |
|
||||
|----------|-------------|-----|
|
||||
| `docs/architecture/proposed-llm-voice.md` | Full proposal | The architecture being evaluated |
|
||||
| `server/src/bin/generator_spike.rs` | gen_want, gen_want_tell, apply_relationship_behaviors | Systems re-voicing must preserve |
|
||||
| `server/src/npc/blueprint.rs` | NpcBlueprint, NpcWant, CulturalMarkers | Data model re-voicing consumes |
|
||||
| `content/global/krenn-rural-zone.ron` | Full file | Current hand-authored quality bar |
|
||||
| `content/global/krenn-industrial-zone.ron` | Full file | Same, different zone for contrast |
|
||||
| `content/global/culture-krenn.ron` | Speech patterns, exclamations | Culture voice injectors must preserve |
|
||||
| `decisions/content.md` | D-121 (voice is culture-driven), D-122 (all NPCs generated), D-128 (culture implicit) | Content architecture constraints |
|
||||
| `decisions/architecture.md` | D-010 (information boundaries), D-024 (NPC 10-axis model) | Architecture constraints |
|
||||
| `decisions/questions-content.md` | Q-057 (composable behaviors), Q-012 (generation expansion) | Open questions this workshop should resolve |
|
||||
| `decisions/scope.md` | D-117 (generator-first), D-115 (v0.2 proof-of-life) | Scope constraints — generator must work |
|
||||
|
||||
## Expected Outputs
|
||||
|
||||
1. **D-record** — the chosen content generation architecture (option 1, 2, 3, or hybrid), with rationale
|
||||
2. **Resolution or refinement of Q-057** — composable behaviors: adopted, rejected, or subsumed by LLM approach
|
||||
3. **Resolution or refinement of Q-012** — generation expansion method: now has a concrete candidate
|
||||
4. **Tier boundary definition** — which NPC tiers get which pipeline (hand-authored / re-voiced / both)
|
||||
5. **Pre-voicing pipeline spec** — baked zones, queue priority model, cache format, fallback behavior
|
||||
6. **Inference wrapper requirements** — model size ceiling, memory budget, Rust crate candidates
|
||||
7. **Spike definition** — concrete test: model candidates, test payloads from Sprint 25 output, success criteria
|
||||
8. **Risk register** — quality floor, hardware floor, lore contamination, cache size
|
||||
|
||||
## Round Structure
|
||||
|
||||
### Round 1: Inventory (divergent)
|
||||
Each participant reads the input documents and the Sprint 25 spike output. Present:
|
||||
- Your domain's take on the three options (hand-authored / composable / LLM re-voicing)
|
||||
- Which option best serves your domain's concerns
|
||||
- What breaks in your domain if we choose the wrong one
|
||||
- One question you need answered before you can commit
|
||||
|
||||
### Round 2: Proposals (convergent)
|
||||
Based on Round 1 input, the lead synthesizes 2-3 concrete architecture proposals (may include hybrids). Each participant evaluates the proposals against their domain and flags blockers.
|
||||
|
||||
### Round 3: Decision (commitment)
|
||||
Narrow to one architecture. Resolve open questions. Produce the D-record. Define the spike. SI creates follow-up tickets.
|
||||
Reference in New Issue
Block a user