add AudioPen ingest pipeline and sync architecture setup docs
Voice-note capture as the primary mobile input channel. AudioPen polishes on their cloud, POSTs to a self-hosted receiver on the desktop (exposed via Tailscale Funnel so no router ports open and nothing behind Authentik). The receiver writes to inbox/raw/; a bash normaliser wraps each drop in frontmatter and moves it to inbox/; /triage-inbox routes them from there. scripts/audiopen-ingest.sh — 80-line bash normaliser. Idempotent; safe to re-run. Extracts title, slugifies, computes capture timestamp from mtime, rewrites with fleeting-note frontmatter. scripts/audiopen-webhook/main.py — stdlib-only Python HTTP server. Zero pip deps; binds to 127.0.0.1 by default. Accepts flexible payload shapes (title/name + body/output/summary/polished/ orig_transcript) so AudioPen version drift is logged rather than silently dropped. scripts/audiopen-webhook/*.example — systemd user-unit templates for the receiver and the path/service pair that fires the ingest wrapper on inbox/raw/ changes. scripts/pyproject.toml + README.md — Python venv convention. Zero deps today; venv location reserved at scripts/.venv/ (gitignored), manifest at scripts/pyproject.toml, bootstrap documented for both plain pip and uv. Optional-dependencies groups let individual tools pull what they need without bloating the whole env. docs/setup/audiopen.md — full setup guide: generating the shared secret, installing the systemd units, pairing with Tailscale Funnel, configuring AudioPen's webhook, and the IMAP-fallback path for setups that can't run a public-reachable receiver. docs/setup/sync.md — companion guide: Gitea SSH remote (works even with Authentik gating HTTPS), obsidian-git plugin configuration, Syncthing desktop↔phone pairing with the critical .stignore patterns, and the three phone-role tiers so the user can pick Tier 1 / 2 / 3 at their own pace. .gitignore gains **/.venv/, **/venv/, **/__pycache__/, and *.pyc so nobody accidentally commits a materialised environment. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude Opus 4.7
@@ -0,0 +1,174 @@
|
||||
# Sync architecture — setup guide
|
||||
|
||||
Three independent sync channels. Setting each up on a fresh machine.
|
||||
|
||||
## 1. Gitea over SSH — version history + off-site backup
|
||||
|
||||
**Already configured** on the desktop for this repo:
|
||||
|
||||
```sh
|
||||
$ git remote -v
|
||||
origin ssh://git@git.schweitz.net:2222/jpmschweitzer/council.git (fetch)
|
||||
origin ssh://git@git.schweitz.net:2222/jpmschweitzer/council.git (push)
|
||||
```
|
||||
|
||||
On a fresh machine:
|
||||
|
||||
```sh
|
||||
git clone ssh://git@git.schweitz.net:2222/jpmschweitzer/council.git
|
||||
cd council
|
||||
```
|
||||
|
||||
Requires an SSH key registered in Gitea (Gitea settings → SSH keys → add).
|
||||
Gitea is behind Authentik for HTTPS, so HTTPS clones won't work without the browser
|
||||
auth dance. SSH bypasses that entirely — the ssh daemon on Gitea handles key auth
|
||||
independently.
|
||||
|
||||
### obsidian-git plugin (desktop)
|
||||
|
||||
The `obsidian-git` plugin is already installed in `.obsidian/plugins/`. Configure it
|
||||
to auto-commit and push on your chosen cadence:
|
||||
|
||||
1. Obsidian settings → Community plugins → Obsidian Git.
|
||||
2. "Remote URL" — leave blank; it uses the repo's existing `origin`.
|
||||
3. "Auto commit-and-sync interval" — e.g. 10 minutes. Set to 0 to disable auto-push.
|
||||
4. "Commit message on auto commit-and-sync" — use the default template.
|
||||
5. Confirm it can push: Command Palette → "Obsidian Git: Push".
|
||||
|
||||
### What NOT to commit
|
||||
|
||||
Already set in `.gitignore`:
|
||||
- `.env`, `.env.local`, `.env.*.local`
|
||||
- `.obsidian/workspace.json`, `.obsidian/workspace-mobile.json`, `.obsidian/cache`
|
||||
- `.claude/settings.local.json`
|
||||
- `**/.venv/`, `**/venv/`, `**/__pycache__/`, `*.pyc`
|
||||
|
||||
---
|
||||
|
||||
## 2. Syncthing — desktop ↔ phone peer-to-peer sync
|
||||
|
||||
Optional. Skip if you're picking **Tier 1** from the phone-role decision (AudioPen
|
||||
only, no mobile vault).
|
||||
|
||||
### Desktop setup
|
||||
|
||||
Syncthing is available on Bazzite via rpm-ostree (layered) or Flatpak. Flatpak is
|
||||
simpler:
|
||||
|
||||
```sh
|
||||
flatpak install flathub me.kozec.syncthingtk
|
||||
# or
|
||||
rpm-ostree install syncthing
|
||||
```
|
||||
|
||||
Start it:
|
||||
|
||||
```sh
|
||||
systemctl --user enable --now syncthing.service
|
||||
# Web UI:
|
||||
xdg-open http://127.0.0.1:8384
|
||||
```
|
||||
|
||||
In the Syncthing web UI:
|
||||
1. **Add folder**: path = `/var/mnt/data/projects/council/`, folder ID = `council`.
|
||||
2. **Ignore patterns** (crucial — click "Edit" → "Ignore Patterns" on the folder):
|
||||
|
||||
```
|
||||
.git
|
||||
.obsidian/workspace.json
|
||||
.obsidian/workspace-mobile.json
|
||||
.obsidian/cache
|
||||
.trash
|
||||
**.sync-conflict-*
|
||||
scripts/.venv
|
||||
```
|
||||
|
||||
This list matters. Syncthing must never propagate `.git/` — partial packs mid-commit
|
||||
would corrupt the mirror.
|
||||
|
||||
### Phone setup (Android)
|
||||
|
||||
1. Install **Syncthing-Fork** from F-Droid (more robust than upstream Syncthing on
|
||||
Android).
|
||||
2. Pair with desktop: desktop shows its device ID in the web UI → enter on phone
|
||||
→ desktop accepts the pairing.
|
||||
3. On the phone, accept the `council` folder share. Set its target directory to
|
||||
e.g. `/sdcard/Documents/council/`.
|
||||
4. Install **Obsidian Mobile**. Open as vault → pick the Syncthing-watched path.
|
||||
|
||||
### Conflict discipline
|
||||
|
||||
Both devices edit the same file while disconnected → Syncthing creates
|
||||
`<filename>.sync-conflict-YYYYMMDD-HHMMSS-deviceID.md`. Rare if you edit one device
|
||||
at a time. To resolve: read both, copy the good bits into the canonical file,
|
||||
delete the conflict.
|
||||
|
||||
### How the Syncthing/git channels interact
|
||||
|
||||
- **Git lives on desktop only.** The phone never sees `.git/` (it's in `.stignore`).
|
||||
- **Syncthing replicates content only.** The phone has the markdown; it doesn't have
|
||||
history.
|
||||
- **To push from desktop**: just `git push` as usual. obsidian-git automates it.
|
||||
- **To pull new content on desktop from phone**: Syncthing already did it; there's
|
||||
nothing to pull. Review the changes in `git status` and commit them like any
|
||||
other working-tree edit.
|
||||
|
||||
---
|
||||
|
||||
## 3. AudioPen webhook via Tailscale Funnel — voice-note ingestion
|
||||
|
||||
See [`docs/setup/audiopen.md`](./audiopen.md). Independent of the two above.
|
||||
|
||||
---
|
||||
|
||||
## Phone-role decision
|
||||
|
||||
When you set up Obsidian Mobile (if at all), pick a tier:
|
||||
|
||||
| Tier | Phone runs | Trade-off |
|
||||
|---|---|---|
|
||||
| 1 | AudioPen only | Simplest. No vault on phone. Read/edit on desktop. |
|
||||
| 2 | AudioPen + browser → Gitea markdown view | Occasional phone read via Authentik-gated Gitea. |
|
||||
| 3 | AudioPen + Syncthing-Fork + Obsidian Mobile | Full vault on phone. Maximum capability, moderate battery + ~few hundred MB storage for the vault. |
|
||||
|
||||
All three are compatible with the rest of the stack. The AudioPen pipeline
|
||||
(path A or B) is identical across tiers.
|
||||
|
||||
---
|
||||
|
||||
## Verification
|
||||
|
||||
After everything is set up:
|
||||
|
||||
```sh
|
||||
# Git remote reachable
|
||||
git ls-remote origin | head -3 # should list refs from Gitea
|
||||
|
||||
# Syncthing running (if Tier 2/3)
|
||||
systemctl --user is-active syncthing.service
|
||||
# → active
|
||||
|
||||
# Tailscale Funnel for AudioPen webhook
|
||||
tailscale funnel status
|
||||
# should list the https endpoint for :8765
|
||||
|
||||
# AudioPen receiver running
|
||||
systemctl --user is-active audiopen-webhook.service
|
||||
# → active
|
||||
|
||||
# Push a test commit
|
||||
cd /var/mnt/data/projects/council
|
||||
echo "# test" > /tmp/sync-test.md
|
||||
mv /tmp/sync-test.md daily/2099-01-01.md
|
||||
git add daily/2099-01-01.md
|
||||
git commit -m "test sync pipeline" -m "Co-Authored-By: me"
|
||||
git push
|
||||
# should succeed
|
||||
|
||||
# On phone (Tier 3): wait a moment, open Obsidian Mobile, verify daily/2099-01-01.md appears.
|
||||
|
||||
# Clean up
|
||||
git rm daily/2099-01-01.md
|
||||
git commit -m "drop sync test"
|
||||
git push
|
||||
```
|
||||
Reference in New Issue
Block a user