Nguyen Thanh DatandPR #6166 Reviewer
c3819e3cbf
* fix(docker): mount the ChromaDB volume where the image actually persists
chromadb/chroma runs `chroma run /config.yaml`, and that config is one line:
persist_path: "/data". The image declares no VOLUME, so everything the store
writes lives in the container's writable layer. The chromadb-data volume was
mounted at /chroma/chroma — where pre-1.0 images kept the store — so it caught
nothing and the entire vector store was destroyed on every container recreate.
Nothing errors when that happens. The collection survives with a count of 0, so
retrieval simply returns nothing from then on, which reads as "no match".
docs/backup-restore.md tells users to archive the chromadb-data volume, which
means the documented backup was empty too.
Fixed in all three compose files. The volume was never written to, so there is
nothing to migrate.
* fix(tests,docs): harden volume-path test helper and document upgrade migration
tests/test_chromadb_volume_path.py
- _chromadb_volumes() now normalises long-form dict volume entries
(source/target/type map) to 'source:target' strings, preventing
AttributeError crashes if a future Compose change uses that syntax.
- test_chromadb_does_not_use_the_pre_1_0_path now uses split(':')[1]
comparison instead of endswith(), so mount-option suffixes (:z, :ro)
cannot mask a wrong legacy target path.
- Three new unit tests validate the helper against both YAML syntaxes
and the mount-option edge case explicitly.
docs/chromadb-volume-migration.md (new)
- Step-by-step guide for users upgrading from pre-#6166 Docker installs
whose ChromaDB data accumulated in the container's writable layer.
- Covers: detecting the situation, backing up via 'docker cp', seeding
the named volume before recreating the container, verification, and
rollback. Validated with disposable Docker volumes.
- Distinguishes fresh installs (no action needed) from existing
installs with live writable-layer data (action needed before recreate).
docs/backup-restore.md
- Extended ChromaDB caveat block to cross-reference the new migration
guide for users upgrading from pre-#6166.
Closes the P1 documentation gap and P2/P3 test-robustness findings
from the PR #6166 adversarial review.
* fix(docs,tests): eliminate ChromaDB migration data-loss risks and enforce safe cutover
- Require service quiescence before cold database copy to prevent torn SQLite pages
- Mandate container preservation via immutable Docker image commit before recreation
- Enforce positive empty check on destination volume prior to seeding
- Add staging container validation on an isolated loopback port before Compose cutover
- Provide verified rollback procedures prohibiting return to legacy /chroma/chroma mount
- Add regression test coverage for all migration safety invariants
---------
Co-authored-by: PR #6166 Reviewer <reviewer@odysseus-review.local>
5.8 KiB
layout
| layout |
|---|
| default |
Backup & Restore
Odysseus keeps all of your state in the data/ directory — the SQLite database
(app.db), the Fernet encryption key (data/.app_key), the vault, memory, RAG
indexes, personal documents, and uploads. The scripts/odysseus-backup tool
snapshots that directory into a single gzip tarball and restores it later.
Snapshots are safe to take while the app is running: SQLite databases are copied
through SQLite's own .backup API rather than a raw file copy, so an in-flight
write can't corrupt the snapshot.
A snapshot contains your secrets. The tarball includes the Fernet encryption key (
data/.app_key), the vault, sessions, and any stored provider/API tokens — so treat it like a password. Store backups somewhere private, never commit them to Git, and prefer an encrypted destination when copying them offsite.
Quick start
Run the tool from the repository root:
# Create a snapshot → backups/odysseus-backup-<YYYYMMDD-HHMMSS>.tar.gz
./scripts/odysseus-backup snapshot
# List existing snapshots (most recent first)
./scripts/odysseus-backup list
# Check a tarball's integrity without extracting it
./scripts/odysseus-backup verify backups/odysseus-backup-20260101-120000.tar.gz
# Restore (destructive — see the warning below)
./scripts/odysseus-backup restore backups/odysseus-backup-20260101-120000.tar.gz --yes
The script depends only on the Python standard library, so any python3 on your
PATH will run it — you don't need the app's virtualenv active.
Every command prints a JSON result. Add --pretty for indented output.
Commands
snapshot
Writes a tar.gz of data/ to backups/<timestamp>.tar.gz.
| Flag | Effect |
|---|---|
--out PATH |
Write to a specific path instead of the default backups/ location. Must be outside data/. |
--include-research |
Include data/deep_research/ (skipped by default — research runs are large). |
--include-attachments |
Include data/mail-attachments/ (skipped by default — cached IMAP extractions, re-derivable). |
By default the snapshot includes everything under data/ except
deep_research/ and mail-attachments/. Personal uploads and documents are
included.
# Snapshot straight to a mounted NAS path
./scripts/odysseus-backup snapshot --out /mnt/nas/odysseus-$(date +%F).tar.gz
# Full snapshot including research runs and mail attachments
./scripts/odysseus-backup snapshot --include-research --include-attachments
list
Lists the tarballs in backups/, most recent first, with size and modification
time.
verify PATH
Opens the tarball read-only and walks every member to confirm it is intact and safe to restore. Nothing is extracted. Use this before relying on an old backup or after copying one across machines.
restore PATH --yes
Overwrites data/ from a tarball.
Restore is destructive. It replaces the current
data/directory.--yesis required so a mistyped command can't wipe your live state.
Restore is not a blind delete: before extracting, the tool renames your current
data/ to data.before-restore-<timestamp> in the repository root. If a
restore turns out to be wrong, your previous state is still there — delete the
restored data/ and rename the stashed directory back. The restore path is also
validated entry-by-entry: archives containing absolute paths, .. segments,
symlinks, or anything outside data/ are rejected.
Scheduling offsite backups
The tarball output composes cleanly with cron and any copy tool. For example, a nightly snapshot copied offsite:
0 3 * * * cd /path/to/odysseus && ./scripts/odysseus-backup snapshot --out "/mnt/nas/odysseus-$(date +\%F).tar.gz"
Swap the --out target for scp, rclone, s3cmd, or similar to push the
snapshot to remote storage.
Docker vs native installs
The tool reads data/ and writes backups/ relative to the repository root, so
where you run it matters:
- Native installs — run it from the repo root as shown above.
data/andbackups/are both in the repo directory. - Docker —
docker-compose.ymlbind-mounts the host's./datato/app/data, so the live data is also present on the host. Run the tool on the host from the repo root; the snapshot reads the bind-mounted./dataand writes to./backupson the host. Running it inside the container is not recommended, becausebackups/is not a mounted volume and the tarball would be lost when the container is recreated.
ChromaDB caveat (Docker only). In the Docker setup, ChromaDB stores its vectors in a separate Compose-managed volume (declared as
chromadb-data), not under./data.odysseus-backuptherefore does not capture the Docker ChromaDB store. Back it up separately if you need it. Compose prefixes the volume with the project name, so find the real name first (docker volume ls | grep chromadb), then archive it — for example:docker run --rm -v <project>_chromadb-data:/data -v "$PWD":/backup \ alpine tar czf /backup/chromadb.tar.gz -C /data .On native installs ChromaDB lives at
data/chroma/and is included in the snapshot normally.Upgrading from a pre-#6166 install? The old Compose files mounted
chromadb-dataat the wrong path (/chroma/chroma), so the volume was always empty while live data accumulated in the container's writable layer under/data. If you have a running pre-#6166 ChromaDB container, follow docs/chromadb-volume-migration.md to rescue that data before recreating or deleting existing containers. The migration guide provides step-by-step instructions for recovery, backup verification, volume seeding, rollback, and re-indexing.