settled-reach/docs/DEVOPS.md

---
title: "DevOps Procedures"
description: "Build, test, lint, and CI procedures for the Settled Reach project — Makefile targets, worktree setup, server/client builds"
type: architecture
status: active
---

# DevOps Procedures

Operational procedures for building, testing, and running The Settled Reach.

## Repository Layout

```
client/               Godot 4 client (GDScript, scenes, assets)
server/               Rust/bevy_ecs simulation server
tooling/              Build tools, scripts, asset pipelines
tests/                Integration and end-to-end tests (cross-boundary)
decisions/            Decision domain files (source of truth for all D/Q/R entries)
.config/              Configuration files (linters, formatters, CI)
.cache/               Local caches for testing/linting (gitignored)
docs/                 Design, architecture, briefings, workshops
db/                   SQLite schema + seed data (connectors at tooling/db/)
```

Unit tests live inside their respective projects (`server/` uses `#[cfg(test)]` inline + `tests/` directory per D-030). The top-level `tests/` directory is for integration tests that cross the client-server boundary (IPC round-trip, serialization fixtures, divergence tests).

## Prerequisites

| Tool | Version | Purpose |
|------|---------|---------|
| Rust (via rustup) | stable | Server compilation, clippy, rustfmt (auto-installed by `make setup`) |
| Godot | 4.x | Client editor and runtime (auto-installed to `~/bin/` by `make setup`) |
| Python | 3.x | Tooling scripts, db connectors |
| Make | any | Task runner (see below) |
| curl | any | Downloading Godot |
| unzip | any | Extracting Godot |

## Makefile Targets

All development operations go through the top-level `Makefile`. Run `make` with no arguments for a summary.

### Setup

```bash
make setup                        # Install/verify all dev dependencies
GODOT_VERSION=4.4 make setup      # Pin a specific Godot version
```

Downloads and installs Godot to `~/bin/godot4`, installs Rust clippy + rustfmt, and verifies Python/curl/unzip. Skips the download if the correct version is already installed. The `GODOT_VERSION` variable defaults to `4.6` and can be overridden.

### Build

```bash
make build              # Build both client and server
make build-server       # cargo build in server/
make build-client       # Client builds are editor-managed (prints guidance)
```

### Run

```bash
make server             # cargo run in server/
make client             # Launch Godot with client/ project
```

The server must be running before the client connects (subprocess launch will be automated later per D-020).

### Test

```bash
make test               # Run all tests (test-server + test-client)
make test-server        # Rust tests via tests/run-rust (cargo nextest, JSON summary)
make test-client        # Godot tests via tests/run-godot (gdUnit4 headless, JSON summary)
```

The IPC test layers (D-030) have dedicated targets:

```bash
make test-ipc-fixtures      # Layer 1: serialization round-trip fixtures
make test-ipc-protocol      # Layer 2: mock LocalBridge protocol tests
make test-ipc-integration   # Layer 3: real subprocess round-trip (+ benchmark when ready)
make test-ipc-benchmark     # IPC latency benchmark (blocked: #555/#556 handshake)
```

Each `tests/run-*` script outputs a JSON summary to stdout and streams progress to stderr:
```json
{"suite":"rust","total":42,"passed":42,"failed":0,"duration_ms":1230}
```

All scripts accept `--filter <name>` to run a subset of tests. They are whitelistable for agent use (no TTY prompts, no interactive input).

Server tests use Rust's built-in test framework with `#[cfg(test)]` inline tests and `tests/` integration tests (D-030). Client tests use gdUnit4 (D-030).

### Cross-Encoder Fixtures

```bash
make fixtures           # Regenerate Rust->GDScript fixtures (server/tests/gen_fixtures.rs)
make fixtures-client    # Generate GDScript->Rust fixtures + verify Rust decoder (#475)
```

The bidirectional protocol is validated by two sets of committed fixtures:
- **Rust encodes, GDScript decodes:** `client/tests/fixtures/msgpack/` (generated by `make fixtures`)
- **GDScript encodes, Rust decodes:** `server/tests/fixtures/gdscript/` (generated by `make fixtures-client`)

Regenerate both after any protocol change. Commit the updated fixtures alongside the code change.

**Troubleshooting fixture failures:**

- **`make fixtures-client` fails with encode errors:** Check that `client/addons/messagepack/messagepack.gd` is up to date. The script exits non-zero on any encode failure.
- **`gdscript_generated_fixtures_deserialize` fails:** Fixtures in `server/tests/fixtures/gdscript/` are stale or corrupted. Re-run `make fixtures-client` and commit the updated files.
- **Fixture staleness in `make pre-pr`:** Protocol changed but fixtures were not regenerated. Run `make fixtures && make fixtures-client`, then commit both `client/tests/fixtures/` and `server/tests/fixtures/gdscript/`.

### Golden File Management

```bash
make golden-diff        # Show diff if golden file output has changed
make golden-update      # Regenerate golden file and stage for commit
```

The golden file (`server/tests/golden/proof_room_tick_10.json`) is a committed snapshot of ObserverSnapshot output after a deterministic 10-tick replay. It catches unintentional changes to simulation output.

**Workflow after intentional simulation changes:**

1. Run `make golden-diff` to see what changed
2. Review the diff — confirm changes are expected
3. Run `make golden-update` to regenerate and stage the new golden file
4. Commit the updated golden file alongside your simulation change

`golden-diff` exits 1 if the golden file has changed (useful in scripts). `golden-update` regenerates the file and runs `git add` but does not commit — the developer reviews and commits manually.

### Lint

```bash
make lint               # Run all linters
make lint-server        # clippy (deny warnings) + rustfmt --check
make lint-client        # gdlint/gdformat
```

### CI (Local)

Run the full CI pipeline locally before pushing:

```bash
make ci                 # Both pipelines
make ci-server          # lint-server → build-server → test-server
make ci-client          # lint-client → build-client → test-client
```

CI targets chain lint → build → test sequentially. A failure in any stage stops the pipeline.

### Pre-PR Checks

Before pushing a PR, run:

```bash
make pre-pr
```

This runs all checks in order: lint → build → test → content validation → fixture staleness. Total runtime ~2.5 minutes (incremental build), under 3 minutes clean.

For branch-specific checks:

```bash
make pre-pr-server      # Server changes: lint, build, test, fixture staleness
make pre-pr-client      # Client changes: lint, build, test
make pre-pr-content     # Content changes: schema + cross-reference validation
```

If `pre-pr-fixtures` fails, your protocol changes require fixture regeneration:

```bash
make fixtures
git add client/tests/fixtures/
git commit -m "chore(fixtures): regenerate for protocol v8"
```

The fixture staleness check is a **blocker** (exit 1) — stale fixtures cause false positive client tests.

### Clean

```bash
make clean              # Remove build artifacts and .cache/ contents
```

### Content Validation

```bash
make validate-content     # Validate content YAML against JSON schemas
make check-fact-ids       # Check fact_id references against knowledge catalogs
```

### Gauntlet Checklists

```bash
make checklist-validate   # Validate checklist YAML against schema (standalone)
make checklist-generate   # Validate + print per-room condition summary
```

Checklists live at `content/gauntlet/rooms/{room_id}/checklist.yaml` (per-room) and `content/gauntlet/cross_room_checks.yaml` (cross-room). Each condition is evaluable from an `ObserverSnapshot`.

7 condition types: `player_near`, `player_facing`, `entity_present`, `entity_absent`, `expected_monologue`, `expected_dialogue`, `expected_interaction_verb`.

Schema: `content/_schema/checklist.schema.json`. The checklist format feeds into #503 (client auto-checklist progress tracking).

`check-fact-ids` operates in two modes:
- **Advisory** — when knowledge catalogs (`content/global/knowledge/*.yaml`) have no fact definitions yet: lists referenced fact_ids and exits cleanly.
- **Enforcing** — when catalogs are populated: fails on any `fact_id` reference that doesn't match a canonical definition.

## Asset Pipeline — Generator-Driven DB (#855, #856, #857)

`server/data/systems.db` is a **read-only canonical snapshot** produced by three
generators.  It is committed to the repo so the client can ship it, but it is never
the source of truth.  Direct edits are forbidden — they are silently overwritten by
the next regeneration.

### Generators

| Generator | Source | Runs via |
|-----------|--------|----------|
| `generate_brands` | `server/src/bin/generate_brands/main.rs` | `tooling/generate-brands` |
| `import_economics` | `tooling/economy-db/import_economics.py` | `python3 tooling/economy-db/import_economics.py` |
| `generate_atlas` | `tooling/planet-gen/generate_atlas.py` | `python3 tooling/planet-gen/generate_atlas.py --seed 42` |

Run all three at once with:

```bash
make regen-db
```

### Meta table stamp

After every successful non-dry-run, each generator writes a row to the `meta` table in
`systems.db` recording the SHA-1 of its source file(s) and the schema file.

```bash
make check-systems-db    # Verify the stamp is fresh (exit 1 = stale)
```

### Making a DB change

1. Edit source files (TOML, JSON, `markers.json`).
2. `make regen-db`
3. `git add server/data/systems.db`
4. Commit with `chore(db): regen systems.db — <reason>`

For schema changes, also update `server/data/systems-schema.sql` and add migration DDL
to `MIGRATION_SQL` in `import_economics.py`.

See `.claude/rules/asset-pipeline.md` for the full rule set.

## Pre-commit and Pre-push Hooks

Git hooks are stored in `.config/hooks/` (version-controlled). Activate them with:

```bash
make setup          # Includes hook installation
make install-hooks  # Just hooks (also makes them executable)
```

Or manually:

```bash
git config core.hooksPath .config/hooks
```

Active checks:

| Hook | Check | Script | Behavior |
|------|-------|--------|----------|
| pre-commit | fact_id validation | `tooling/check-fact-ids` | Warns if catalogs are stubs; fails on unknown fact_ids when populated |
| pre-push | GDScript parse | internal | Fails on any SCRIPT ERROR |
| pre-push | Rust lint | internal | fmt + clippy |
| pre-push | Python lint | internal | ruff |
| pre-push | JSON syntax | internal | python3 -m json.tool |
| pre-push | systems.db stamp | `tooling/check-systems-db-stamp` | Rejects stale DB when pushed (#857) |

The `core.hooksPath` setting uses a relative path (`.config/hooks`) that resolves per worktree, so it works correctly across all worktrees in the repository.

To bypass hooks in an emergency:

```bash
git commit --no-verify -m "fix: emergency hotfix"
```

## Configuration Files

The `.config/` directory holds shared configuration for linters, formatters, and CI. Examples of what goes here:

- Clippy configuration overrides
- gdlint/gdformat rules
- CI workflow definitions (before moving to `.github/workflows/`)
- Editor config templates

Project-specific config that lives in subdirectories (e.g., `server/Cargo.toml`, `client/project.godot`) stays in those directories. `.config/` is for cross-cutting or shared configuration.

## Cache Directory

`.cache/` is gitignored and used for:

- Test result caches
- Linter caches
- Build artifact caches (if configured)
- Coverage reports

Agents and CI jobs can write freely to `.cache/` without polluting the working tree. `make clean` clears it.

## Testing Architecture (D-030)

Three-layer testing strategy:

1. **Unit tests** — Inside `server/` (Rust `#[cfg(test)]`) and `client/` (gdUnit4). Test individual systems in isolation.
2. **Integration tests** — Inside `server/tests/` (Rust) and `tests/` (cross-boundary). Test system interactions, IPC serialization round-trips.
3. **Fixture-based tests** — IPC serialization fixture files in `tests/` for protocol regression testing. Known-good MessagePack payloads verified against both sides.

Key components:
- **CauseChain** (production ECS component) — Tracks causal attribution for testable observation sequences (D-030).
- **Deterministic replay** — Server simulation is deterministic given the same seed + input sequence. Replay logs enable regression testing (#201, critical).

## SQLite Access

**Never use the `sqlite3` CLI** — it crashes in Claude Code (std::bad_alloc).

Use wrapper scripts:
```bash
tooling/db/sqlite-query "SELECT * FROM tickets WHERE status='open'"
tooling/db/sqlite-exec "UPDATE tickets SET status='done' WHERE id=1"
```

## Decisions System

Decisions are split into domain files under `decisions/` (see `decisions/README.md` for the full index). A SQLite index table syncs metadata for cross-referencing and querying.

```bash
make decisions-sync       # Parse decisions/*.md into SQLite
make decisions-coverage   # Decision-to-ticket coverage by domain
make decisions-active     # List all active confirmed decisions
make decisions-orphan     # Decisions without implementing tickets
```

The sync runs automatically as part of `make setup` and via pre-commit hook. Markdown files are the source of truth; the DB is a derived index.

## Commit Conventions

See the `/git-commit` skill (`.claude/skills/git-commit/`) for full details. Summary:

- Conventional commits: `type(scope): summary`
- Types: `feat`, `fix`, `refactor`, `chore`, `docs`, `data`, `loc`
- Scopes match project subsystems: `client`, `server`, `engine`, `simulation`, `ui`, `audio`, `meta`, etc.
- Imperative mood, lowercase, no period, max 72 chars
- CHANGELOG.md updated after each commit group