diff --git a/docs/guides/code-server-setup.md b/docs/guides/code-server-setup.md deleted file mode 100644 index 0aef8ae..0000000 --- a/docs/guides/code-server-setup.md +++ /dev/null @@ -1,403 +0,0 @@ -# Code-Server Installation Guide - -> Browser-based VSCode IDE running on host for full system access -> -> **Service Type:** Host-based (systemd service, not containerized) -> **Purpose:** Replace SSH with persistent web-based development environment -> **Access:** https://code.schweitz.net (via NPM with SSL) - ---- - -## Why Host-Based? - -Unlike other services in this stack, code-server runs directly on the host OS (not in Docker) to provide: - -- Full access to host filesystem and configurations -- Direct control over systemd services -- Native Docker CLI access without Docker-in-Docker complexity -- No permission issues when editing files across SSD/HDD -- Persistent sessions that survive network disconnections - ---- - -## Installation Steps - -### 1. Install code-server - -Run these commands on the tower-of-joy host: - -```bash -# Download and install code-server (version 4.x) -curl -fsSL https://code-server.dev/install.sh | sh - -# Verify installation -code-server --version -``` - -### 2. Create Configuration Directory - -```bash -# Create config directory -mkdir -p ~/.config/code-server - -# Create configuration file -cat > ~/.config/code-server/config.yaml <<'EOF' -bind-addr: 127.0.0.1:8084 -auth: password -password: CHANGE_THIS_PASSWORD -cert: false -user-data-dir: /home/jpmschweitzer/docker-data/code-server/user-data -extensions-dir: /home/jpmschweitzer/docker-data/code-server/extensions -EOF - -# Create data directories on SSD -mkdir -p /home/jpmschweitzer/docker-data/code-server/{user-data,extensions} -``` - -**IMPORTANT:** Replace `CHANGE_THIS_PASSWORD` with a strong password. This is a secondary auth layer (NPM will provide the primary authentication). - -### 3. Create Systemd Service - -```bash -# Create service file -sudo tee /etc/systemd/system/code-server.service > /dev/null <<'EOF' -[Unit] -Description=code-server - Browser-based VSCode IDE -Documentation=https://coder.com/docs/code-server -After=network.target - -[Service] -Type=exec -ExecStart=/usr/bin/code-server -Restart=always -User=jpmschweitzer -Group=jpmschweitzer -Environment="PASSWORD_FROM_CONFIG=true" - -# Hardening -NoNewPrivileges=true -PrivateTmp=true -ProtectSystem=full -ProtectHome=false -ReadWritePaths=/home/jpmschweitzer /mnt/media - -[Install] -WantedBy=multi-user.target -EOF - -# Reload systemd daemon -sudo systemctl daemon-reload - -# Enable and start code-server -sudo systemctl enable code-server -sudo systemctl start code-server - -# Check status -sudo systemctl status code-server -``` - -### 4. Verify Local Access - -```bash -# Test that code-server is running locally -curl -I http://127.0.0.1:8084 - -# Should return HTTP 302 (redirect to login page) -``` - ---- - -## Service Integration - -### Uptime Kuma Monitoring - -After code-server is running, add it to Uptime Kuma for health monitoring: - -1. Open Uptime Kuma: http://192.168.86.149:3001 -2. Click **Add New Monitor** -3. Configure the monitor: - -**Monitor Settings:** -- **Monitor Type:** HTTP(s) -- **Friendly Name:** Code-Server -- **URL:** https://code.schweitz.net -- **Heartbeat Interval:** 60 seconds -- **Retries:** 3 -- **Heartbeat Retry Interval:** 60 seconds -- **Accepted Status Codes:** 200-299, 302 (redirect to login) -- **Ignore TLS/SSL errors:** ❌ Disabled (cert should be valid) -- **Tags:** Infrastructure, Development - -4. Click **Save** - -The monitor should show "Up" status once code-server is accessible through NPM. - -### Organizr Dashboard Integration - -Add code-server to your Organizr unified dashboard: - -1. Open Organizr: https://home.schweitz.net -2. Navigate to **Settings → Tab Editor** -3. Click **Add Tab** - -**Tab Configuration:** -- **Tab Name:** Code-Server -- **Tab URL:** https://code.schweitz.net -- **Category:** Infrastructure (or create "Development" category) -- **Icon:** `fa-code` or `fa-laptop-code` -- **Active:** ✅ Enabled -- **New Window:** ❌ Disabled (use iframe) - -4. **Homepage Integration (Optional):** - - Go to **Settings → Homepage Items** - - Add custom HTML tile: - -```html -
- - - Code-Server - -
-``` - -5. Click **Save** - -The code-server tab will now appear in your Organizr sidebar. - ---- - -## Nginx Proxy Manager Configuration - -### Create Proxy Host - -1. Open NPM admin interface: http://192.168.86.149:81 -2. Navigate to **Hosts → Proxy Hosts → Add Proxy Host** - -**Details Tab:** -- **Domain Name:** `code.schweitz.net` -- **Scheme:** `http` -- **Forward Hostname/IP:** `192.168.86.149` (or `localhost`) -- **Forward Port:** `8084` -- **Block Common Exploits:** ✅ Enabled -- **Websockets Support:** ✅ Enabled (critical for code-server) - -**SSL Tab:** -- **SSL Certificate:** Request New SSL Certificate -- **Force SSL:** ✅ Enabled -- **HTTP/2 Support:** ✅ Enabled -- **HSTS Enabled:** ✅ Enabled -- **Email:** your-email@example.com (for Let's Encrypt) -- **Terms of Service:** ✅ Agree - -**Access List Tab:** -- Create new access list: "Code Server Access" -- Configure basic auth or use NPM's built-in authentication - -**Advanced Tab (optional):** -```nginx -# Increase timeout for long-running operations -proxy_read_timeout 3600s; -proxy_send_timeout 3600s; - -# Proper headers for WebSocket support -proxy_set_header Upgrade $http_upgrade; -proxy_set_header Connection "upgrade"; -proxy_set_header Accept-Encoding gzip; -``` - ---- - -## Security Hardening - -### Firewall Rules - -```bash -# Ensure port 8084 is NOT exposed to the internet -sudo ufw status - -# Port 8084 should only be accessible from localhost -# External access ONLY through NPM on ports 80/443 -``` - -### Authentication Layers - -Code-server will have **three layers of security**: - -1. **NPM Access List** - Primary authentication via reverse proxy -2. **code-server password** - Secondary authentication (from config.yaml) -3. **HTTPS/SSL** - Encrypted transport via Let's Encrypt - -### Recommended NPM Access List - -Create an access list in NPM with: -- Basic Auth username/password -- IP whitelist (optional): Restrict to known IPs or Tailscale network -- Rate limiting: Prevent brute force attacks - ---- - -## Configuration Tips - -### Extensions to Install - -After first login, install these extensions: - -```bash -# Via code-server CLI -code-server --install-extension ms-python.python -code-server --install-extension ms-azuretools.vscode-docker -code-server --install-extension eamodio.gitlens -code-server --install-extension GitHub.copilot # If you have Copilot -code-server --install-extension Codeium.codeium # Free AI assistant alternative -``` - -### Custom Settings - -Edit settings via UI or directly: - -```bash -nano ~/docker-data/code-server/user-data/User/settings.json -``` - -Recommended settings: -```json -{ - "workbench.colorTheme": "Default Dark+", - "terminal.integrated.defaultProfile.linux": "bash", - "files.watcherExclude": { - "**/node_modules/**": true, - "**/.git/objects/**": true, - "**/.venv/**": true - }, - "editor.formatOnSave": true, - "files.autoSave": "afterDelay" -} -``` - ---- - -## Maintenance - -### View Logs - -```bash -# Systemd logs -sudo journalctl -u code-server -f - -# Follow recent logs -sudo journalctl -u code-server --since "10 minutes ago" -``` - -### Restart Service - -```bash -sudo systemctl restart code-server -``` - -### Update code-server - -```bash -# Re-run installation script -curl -fsSL https://code-server.dev/install.sh | sh - -# Restart service to use new version -sudo systemctl restart code-server -``` - -### Backup Configuration - -Configuration is stored in: -- `~/.config/code-server/config.yaml` - Main config -- `~/docker-data/code-server/user-data/` - Settings, keybindings, snippets -- `~/docker-data/code-server/extensions/` - Installed extensions - -**Automated Backups:** - -The maintenance container backs up code-server configuration nightly at 3 AM to `/mnt/media/backups/docker-configs/` with 30-day retention. - -After installing code-server, restart the maintenance container to enable backups: - -```bash -docker restart maintenance - -# Verify the mount is accessible -docker exec maintenance ls -la /data/code-server-config - -# Manually trigger a backup to test -docker exec maintenance /scripts/backup-configs.sh - -# Check backup logs -docker exec maintenance cat /var/log/maintenance/backup-configs.log -``` - ---- - -## Troubleshooting - -### Service won't start - -```bash -# Check service status -sudo systemctl status code-server - -# Check logs for errors -sudo journalctl -u code-server -n 50 - -# Verify config file syntax -cat ~/.config/code-server/config.yaml -``` - -### Can't connect via browser - -```bash -# Verify code-server is listening -sudo netstat -tlnp | grep 8084 - -# Check NPM proxy host configuration -# Ensure WebSocket support is enabled -# Verify SSL certificate is valid -``` - -### Performance issues - -```bash -# Check system resources -htop - -# Monitor code-server process -top -p $(pgrep code-server) - -# Increase file watcher limits if needed -echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf -sudo sysctl -p -``` - ---- - -## Integration Checklist - -- [ ] code-server installed and running via systemd -- [ ] NPM proxy host configured with SSL -- [ ] External access working at https://code.schweitz.net -- [ ] WebSocket connections working (terminal, file watcher) -- [ ] Authentication layers tested (NPM + code-server password) -- [ ] Maintenance container restarted to enable backups -- [ ] Backup tested and verified -- [ ] Uptime Kuma monitoring added -- [ ] Organizr dashboard tab created -- [ ] CONTAINERS.md documentation updated -- [ ] README.md service table updated - ---- - -## References - -- **Code-Server Docs:** https://coder.com/docs/code-server -- **NPM Docs:** https://nginxproxymanager.com/guide/ -- **Systemd Docs:** https://www.freedesktop.org/software/systemd/man/systemd.service.html - ---- - -*Created: 2025-11-14* -*System: tower-of-joy* diff --git a/docs/guides/connect-devices-guide.md b/docs/guides/connect-devices-guide.md deleted file mode 100644 index fb685d0..0000000 --- a/docs/guides/connect-devices-guide.md +++ /dev/null @@ -1,381 +0,0 @@ -# Connecting Devices to Headscale VPN - -> Step-by-step guide for connecting various devices to your Headscale mesh network -> Created: 2025-11-11 - -## Overview - -Once connected to Headscale, devices can access all services via mesh IPs (10.99.0.x): -- Organizr: http://10.99.0.1:9999 -- Portainer: http://10.99.0.1:8001 -- Netdata: http://10.99.0.1:19999 -- All other services using mesh IPs - -## Prerequisites - -**You need a pre-auth key from Headscale:** - -```bash -# Generate a new pre-auth key (run on tower-of-joy) -docker exec headscale headscale preauthkeys create --user homelab --expiration 24h - -# Output example: -# b4c17f9e9f01ea2e54b24e0369e2949bd28bfaf46944c634 -``` - -**Save this key - you'll use it to connect each device.** - -## macOS (MacBook/iMac) - -### Method 1: Tailscale App (Recommended) - -**Step 1: Install Tailscale** -```bash -# Option A: Using Homebrew -brew install tailscale - -# Option B: Download from website -# Visit: https://tailscale.com/download/mac -# Download and install the .pkg file -``` - -**Step 2: Start Tailscale** -```bash -# Start the Tailscale service -sudo tailscaled install-system-daemon -sudo /Applications/Tailscale.app/Contents/MacOS/Tailscale up -``` - -**Step 3: Connect to Headscale** - -Open the Tailscale app from menu bar → Preferences → Login Server - -OR use command line: -```bash -sudo tailscale up --login-server=http://:8085 \ - --authkey= \ - --hostname=macbook - -# Example: -# sudo tailscale up --login-server=http://your-public-ip:8085 \ -# --authkey=b4c17f9e9f01ea2e54b24e0369e2949bd28bfaf46944c634 \ -# --hostname=macbook -``` - -**Step 4: Verify Connection** -```bash -# Check status -tailscale status - -# Should show: -# 10.99.0.1 tower-of-joy homelab linux - -# 10.99.0.2 macbook homelab darwin - - -# Get your mesh IP -tailscale ip -4 -# Example output: 10.99.0.2 -``` - -**Step 5: Test Access** -```bash -# Ping tower-of-joy -ping 10.99.0.1 - -# Access Organizr -open http://10.99.0.1:9999 - -# Or use curl -curl http://10.99.0.1:9999 -``` - -### Method 2: Using Public IP (If port 8085 forwarded) - -If you've forwarded port 8085 on your router: -```bash -sudo tailscale up --login-server=http://:8085 \ - --authkey= \ - --hostname=macbook -``` - -## Linux (Laptop/Desktop) - -```bash -# Install Tailscale -curl -fsSL https://tailscale.com/install.sh | sh - -# Connect to Headscale -sudo tailscale up --login-server=http://:8085 \ - --authkey= \ - --hostname=linux-laptop - -# Verify -tailscale status -ping 10.99.0.1 - -# Access Organizr -xdg-open http://10.99.0.1:9999 -``` - -## Windows - -**Step 1: Download Tailscale** -- Visit: https://tailscale.com/download/windows -- Download and run the installer - -**Step 2: Configure Custom Login Server** -- After installation, Tailscale runs in system tray -- Right-click Tailscale icon → Settings → Admin Console URL -- Change to: `http://:8085` - -**Step 3: Login with Pre-Auth Key** -- Right-click Tailscale icon → "Log in to Tailscale" -- Use the pre-auth key when prompted - -**Step 4: Verify** -```powershell -# In PowerShell or CMD -tailscale status -ping 10.99.0.1 - -# Access Organizr in browser -start http://10.99.0.1:9999 -``` - -## iOS (iPhone/iPad) - -**Step 1: Install Tailscale App** -- Open App Store -- Search "Tailscale" -- Install official Tailscale app - -**Step 2: Configure Custom Control Server** -- Open Tailscale app -- Tap Settings (gear icon) -- Tap "Use Custom Control Server" -- Enter: `http://:8085` - -**Step 3: Connect** -- Tap "Log In" -- If prompted for key, use your pre-auth key -- Grant VPN permissions when prompted - -**Step 4: Test** -- Open Safari -- Navigate to: `http://10.99.0.1:9999` -- You should see Organizr interface - -## Android - -**Step 1: Install Tailscale App** -- Open Google Play Store -- Search "Tailscale" -- Install official Tailscale app - -**Step 2: Configure Custom Control Server** -- Open Tailscale app -- Tap menu (three dots) -- Settings → Use custom control server -- Enter: `http://:8085` - -**Step 3: Connect** -- Tap "Log In" -- Use pre-auth key if prompted -- Grant VPN permissions - -**Step 4: Test** -- Open Chrome/Firefox -- Navigate to: `http://10.99.0.1:9999` -- Organizr should load - -## Troubleshooting - -### Can't Connect to Headscale Server - -**Problem:** "Failed to connect to login server" - -**Solutions:** -1. **Check port 8085 is forwarded:** - ```bash - # Test from outside network - curl http://:8085 - # Should return HTML or redirect - ``` - -2. **Check Headscale is running:** - ```bash - # On tower-of-joy - docker ps | grep headscale - docker logs headscale - ``` - -3. **Use local IP if on same network:** - ```bash - # Instead of public IP, use local IP - sudo tailscale up --login-server=http://192.168.86.149:8085 ... - ``` - -### Connected but Can't Reach Services - -**Problem:** Connected to VPN but can't access 10.99.0.1 - -**Check:** -```bash -# Verify VPN connection -tailscale status -# Should show tower-of-joy as online - -# Test ping -ping 10.99.0.1 -# Should respond - -# Check firewall (on tower-of-joy) -# Ensure mesh interface accepts traffic -sudo iptables -L -n | grep tailscale -``` - -**Solution:** -```bash -# On tower-of-joy, allow traffic from mesh network -sudo iptables -A INPUT -i tailscale0 -j ACCEPT -``` - -### Pre-Auth Key Expired - -**Problem:** "Invalid auth key" - -**Solution:** -```bash -# Generate new key (on tower-of-joy) -docker exec headscale headscale preauthkeys create --user homelab --expiration 24h - -# Use the new key to connect -``` - -### DNS Not Resolving - -**Problem:** Can ping 10.99.0.1 but browser can't resolve - -**Solution:** -- Use IP addresses directly: `http://10.99.0.1:9999` -- Don't use hostnames unless you've configured DNS -- Mesh IPs always work - -## Verifying Your Connection - -### Quick Test Checklist - -From your newly connected device: - -```bash -# 1. Check VPN status -tailscale status -# Should show: connected, online - -# 2. Get your mesh IP -tailscale ip -4 -# Example: 10.99.0.2 - -# 3. Ping tower-of-joy -ping -c 4 10.99.0.1 -# Should get responses - -# 4. Test Organizr -curl -I http://10.99.0.1:9999 -# Should return HTTP 200 OK - -# 5. Test other services -curl -I http://10.99.0.1:8001 # Portainer -curl -I http://10.99.0.1:19999 # Netdata -curl -I http://10.99.0.1:3001 # Uptime Kuma -``` - -### View All Connected Devices - -```bash -# On tower-of-joy -docker exec headscale headscale nodes list - -# Shows all devices: -# ID | Hostname | IP | Last Seen -# 1 | tower-of-joy | 10.99.0.1 | now -# 2 | macbook | 10.99.0.2 | now -# 3 | iphone | 10.99.0.3 | now -``` - -## Managing Devices - -### Remove a Device - -```bash -# On tower-of-joy -docker exec headscale headscale nodes list -# Note the ID of the device to remove - -docker exec headscale headscale nodes delete -``` - -### Rename a Device - -```bash -docker exec headscale headscale nodes rename -``` - -### Generate Multiple Pre-Auth Keys - -```bash -# For different devices or time periods -docker exec headscale headscale preauthkeys create --user homelab --expiration 1h --reusable -docker exec headscale headscale preauthkeys create --user homelab --expiration 7d -docker exec headscale headscale preauthkeys create --user homelab --expiration 30d -``` - -### List All Pre-Auth Keys - -```bash -docker exec headscale headscale preauthkeys list -``` - -## Security Best Practices - -### Key Expiration -- ✅ Use short expiration for one-time device setups (1h-24h) -- ✅ Use longer expiration for trusted devices (7d-30d) -- ⚠️ Never use permanent keys - -### Device Management -- ✅ Use descriptive hostnames (macbook, work-laptop, phone) -- ✅ Regularly review connected devices -- ✅ Remove old/unused devices -- ✅ Regenerate keys periodically - -### Network Security -- ✅ Headscale port (8085) should be firewalled to trusted IPs if possible -- ✅ Use strong authentication on services -- ✅ Consider adding MFA to Organizr for public access -- ✅ Monitor Headscale logs for suspicious activity - -## Quick Reference - -### MacBook Connection (Your Current Task) -```bash -# 1. Generate pre-auth key (on tower-of-joy) -docker exec headscale headscale preauthkeys create --user homelab --expiration 24h - -# 2. On MacBook -brew install tailscale -sudo tailscale up --login-server=http://:8085 \ - --authkey= \ - --hostname=macbook - -# 3. Verify -tailscale status -open http://10.99.0.1:9999 -``` - ---- - -**Next Steps After Connecting:** -1. Access Organizr: http://10.99.0.1:9999 -2. Complete setup wizard -3. Add tabs for all services using mesh IPs -4. Enjoy unified dashboard from anywhere! diff --git a/docs/guides/gpu-docker-config.md b/docs/guides/gpu-docker-config.md deleted file mode 100644 index 024c394..0000000 --- a/docs/guides/gpu-docker-config.md +++ /dev/null @@ -1,133 +0,0 @@ -# GPU Docker Configuration - Working Setup - -> Successfully configured: 2025-11-11 -> System: tower-of-joy -> GPU: NVIDIA GeForce RTX 2080 Ti -> Driver: 470.256.02 - -## Problem Encountered - -**Issue:** nvidia-container-toolkit 1.18.0 has a compatibility bug with NVIDIA driver 470.x -- Version 1.18 changed default mode from "legacy" to "CDI" -- Legacy mode detection fails with older drivers -- Error: `libnvidia-ml.so.1: cannot open shared object file` - -## Working Solution - -**Downgrade to nvidia-container-toolkit 1.17.9-1** - -All nvidia-container packages must be downgraded together: -- nvidia-container-toolkit -- nvidia-container-toolkit-base -- libnvidia-container-tools -- libnvidia-container1 - -## Installation Commands - -```bash -# Remove all nvidia-container packages -sudo apt-get remove -y nvidia-container-toolkit nvidia-container-toolkit-base libnvidia-container-tools libnvidia-container1 - -# Clean up -sudo apt-get autoremove -y - -# Install all packages at version 1.17.9-1 -sudo apt-get install -y \ - nvidia-container-toolkit=1.17.9-1 \ - nvidia-container-toolkit-base=1.17.9-1 \ - libnvidia-container-tools=1.17.9-1 \ - libnvidia-container1=1.17.9-1 - -# Hold packages to prevent auto-upgrade -sudo apt-mark hold nvidia-container-toolkit nvidia-container-toolkit-base libnvidia-container-tools libnvidia-container1 - -# Configure Docker runtime -sudo nvidia-ctk runtime configure --runtime=docker --config=/etc/docker/daemon.json - -# Rebuild library cache -sudo ldconfig - -# Restart Docker -sudo systemctl restart docker -``` - -## Verification - -```bash -# Test GPU access -sudo docker run --rm --gpus all nvidia/cuda:11.8.0-runtime-ubuntu20.04 nvidia-smi -``` - -**Expected output:** nvidia-smi showing RTX 2080 Ti - -## Current Configuration - -### /etc/docker/daemon.json -```json -{ - "runtimes": { - "nvidia": { - "args": [], - "path": "nvidia-container-runtime" - } - } -} -``` - -### Installed Versions -``` -libnvidia-container-tools 1.17.9-1 -libnvidia-container1 1.17.9-1 -nvidia-container-toolkit 1.17.9-1 -nvidia-container-toolkit-base 1.17.9-1 -``` - -All packages are **held** to prevent automatic upgrade to 1.18.x - -## Important Notes - -1. **Do NOT upgrade** nvidia-container-toolkit to 1.18.x - it breaks compatibility with driver 470 -2. If you run `apt upgrade`, the packages are held and won't upgrade -3. To check held packages: `apt-mark showhold` -4. To unhold (not recommended): `sudo apt-mark unhold nvidia-container-toolkit` - -## For Future Reference - -If you need to update the NVIDIA driver: -1. Driver 470.x is compatible with CUDA 11.4 -2. Driver 525+ is compatible with CUDA 12.x -3. After driver update, may be able to use newer nvidia-container-toolkit - -## Testing GPU in Containers - -### Quick Test -```bash -docker run --rm --gpus all nvidia/cuda:11.8.0-runtime-ubuntu20.04 nvidia-smi -``` - -### Test with Ollama -```bash -docker run --rm --gpus all ollama/ollama nvidia-smi -``` - -### Test with Jellyfin (after deployment) -Check Jellyfin Dashboard → Playback → Transcoding for NVIDIA NVENC option - -## Troubleshooting - -If GPU stops working after system update: - -```bash -# Check if packages were upgraded -dpkg -l | grep nvidia-container - -# If upgraded to 1.18.x, re-run downgrade script: -sudo bash /home/jpmschweitzer/Projects/tower-of-joy/scripts/gpu-fix-downgrade-all.sh -``` - -## References - -- NVIDIA Container Toolkit: https://github.com/NVIDIA/nvidia-container-toolkit -- Driver 470 Release Notes: https://docs.nvidia.com/datacenter/tesla/tesla-release-notes-470-256-02/ -- Issue with 1.18.0: https://github.com/NVIDIA/nvidia-container-toolkit/issues -- Our fix script: `/home/jpmschweitzer/Projects/tower-of-joy/scripts/gpu-fix-downgrade-all.sh` diff --git a/docs/guides/headscale-setup.md b/docs/guides/headscale-setup.md deleted file mode 100644 index da60720..0000000 --- a/docs/guides/headscale-setup.md +++ /dev/null @@ -1,226 +0,0 @@ -# Headscale Setup & Connection Guide - -> Generated: 2025-11-11 -> Server: tower-of-joy -> Network Range: 10.99.0.0/16 - -## Service Status - -✅ **Headscale is running** -- Container: `headscale` -- Web/API Port: `8085` -- Metrics Port: `9090` -- Server URL: `http://192.168.86.149:8085` - -## User & Authentication - -**User Created:** `homelab` (ID: 1) - -**Pre-Auth Key (expires in 30 days, reusable):** -``` -b4c17f9e9f01ea2e54b24e0369e2949bd28bfaf46944c634 -``` - -⚠️ **Security Note:** This key allows devices to join your mesh network. Keep it secure and regenerate after use if needed. - ---- - -## Connecting This Server (tower-of-joy) - -### Step 1: Install Tailscale Client - -```bash -curl -fsSL https://tailscale.com/install.sh | sh -``` - -### Step 2: Connect to Headscale - -```bash -sudo tailscale up --login-server=http://192.168.86.149:8085 \ - --authkey=b4c17f9e9f01ea2e54b24e0369e2949bd28bfaf46944c634 \ - --accept-routes -``` - -### Step 3: Verify Connection - -```bash -# Check Tailscale status -sudo tailscale status - -# Get your mesh IP (should be in 10.99.x.x range) -sudo tailscale ip -4 - -# Test connectivity -ping $(tailscale ip -4) -``` - ---- - -## Connecting Other Devices (Laptop, Phone, etc.) - -### On Linux/macOS - -```bash -# Install Tailscale -curl -fsSL https://tailscale.com/install.sh | sh - -# Connect to your Headscale server -sudo tailscale up --login-server=http://192.168.86.149:8085 \ - --authkey=b4c17f9e9f01ea2e54b24e0369e2949bd28bfaf46944c634 -``` - -### On Windows - -1. Download Tailscale from https://tailscale.com/download/windows -2. Install and open Tailscale -3. Run in PowerShell (as Administrator): -```powershell -tailscale up --login-server=http://192.168.86.149:8085 ` - --authkey=b4c17f9e9f01ea2e54b24e0369e2949bd28bfaf46944c634 -``` - -### On Android/iOS - -1. Install Tailscale app from app store -2. Open app settings -3. Set "Control URL" to: `http://192.168.86.149:8085` -4. Use auth key: `b4c17f9e9f01ea2e54b24e0369e2949bd28bfaf46944c634` - ---- - -## Testing Remote SSH Access - -Once devices are connected to the mesh: - -```bash -# From your laptop (after connecting to Headscale) - -# Get tower-of-joy's mesh IP -ssh jpmschweitzer@ - -# Example (your actual IP will be something like 10.99.0.1) -ssh jpmschweitzer@10.99.0.1 -``` - -**Benefits:** -- No port forwarding needed -- No exposing SSH to internet -- Encrypted peer-to-peer connections -- Works from anywhere - ---- - -## Headscale Management Commands - -### List All Connected Devices -```bash -docker exec headscale headscale nodes list -``` - -### List Users -```bash -docker exec headscale headscale users list -``` - -### Generate New Pre-Auth Key -```bash -# 7 days, single-use -docker exec headscale headscale preauthkeys create --user 1 --expiration 168h - -# 30 days, reusable -docker exec headscale headscale preauthkeys create --user 1 --expiration 720h --reusable -``` - -### List Pre-Auth Keys -```bash -docker exec headscale headscale preauthkeys list --user 1 -``` - -### Remove a Device -```bash -# First, get the node ID -docker exec headscale headscale nodes list - -# Then delete by ID -docker exec headscale headscale nodes delete -``` - ---- - -## Troubleshooting - -### Can't Connect to Headscale Server - -1. **Check if Headscale is running:** -```bash -docker ps | grep headscale -``` - -2. **Check firewall (if connecting from external network):** -```bash -sudo ufw status -# If needed: sudo ufw allow 8085/tcp -``` - -3. **View Headscale logs:** -```bash -docker logs headscale --tail 50 -``` - -### Devices Can't See Each Other - -1. **Check device is registered:** -```bash -docker exec headscale headscale nodes list -``` - -2. **Verify IPs are in the 10.99.0.0/16 range:** -```bash -sudo tailscale ip -4 -``` - -3. **Test direct ping:** -```bash -ping -``` - -### Need to Regenerate Keys - -```bash -# Create new auth key -docker exec headscale headscale preauthkeys create --user 1 --expiration 720h --reusable - -# Update this document with the new key -``` - ---- - -## Configuration File Location - -**Config:** `/home/jpmschweitzer/docker-data/headscale/config/config.yaml` -**Database:** `/home/jpmschweitzer/docker-data/headscale/data/db.sqlite` - -To edit configuration: -1. Edit the config file -2. Restart container: `docker restart headscale` -3. Verify: `docker logs headscale --tail 20` - ---- - -## Next Steps After Setup - -1. ✅ Connect tower-of-joy to Headscale -2. ✅ Connect your laptop/work devices -3. ✅ Test SSH access from laptop to server -4. ✅ Configure SSH key authentication for security -5. ⚡ Add phone/tablet for remote monitoring -6. ⚡ Set up exit node (optional - route all traffic through home) - ---- - -## Resources - -- **Headscale Docs:** https://headscale.net/ -- **Tailscale Client Docs:** https://tailscale.com/kb/ -- **Container Logs:** `docker logs headscale -f` -- **Portainer:** http://192.168.86.149:8001 diff --git a/docs/guides/npm-logging-guide.md b/docs/guides/npm-logging-guide.md deleted file mode 100644 index 1ee7a53..0000000 --- a/docs/guides/npm-logging-guide.md +++ /dev/null @@ -1,326 +0,0 @@ -# NPM Logging & Audit Guide - -> Centralized logging for all external access -> Created: 2025-11-11 - -## Why NPM is the Logging Hub - -**All public service access routes through NPM**, which means: -- Every external request is logged -- Failed authentication attempts tracked -- Rate limiting violations recorded -- SSL certificate renewals logged -- Configuration changes audited - -## Accessing NPM Logs - -### Via NPM UI - -**Access NPM admin panel:** -- VPN: http://10.99.0.1:81 -- Local: http://192.168.86.149:81 - -**View logs:** -1. Navigate to each Proxy Host -2. Click "View Logs" button -3. See real-time access logs -4. Filter by status code, IP, user agent - -### Via Docker Logs - -**Real-time monitoring:** -```bash -# All NPM logs -docker logs -f nginx-proxy-manager - -# Filter for access logs only -docker logs -f nginx-proxy-manager 2>&1 | grep -i "access" - -# Filter for errors -docker logs -f nginx-proxy-manager 2>&1 | grep -i "error" - -# Filter for specific service (e.g., Jellyfin) -docker logs -f nginx-proxy-manager 2>&1 | grep "media.schweitz.net" -``` - -### Via Log Files - -**Log location:** -```bash -# Access logs stored in container volume -ls -lh ~/docker-data/nginx-proxy-manager/data/logs/ - -# View access logs -tail -f ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log - -# View error logs -tail -f ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*_error.log -``` - -## Log Format - -**Standard NPM access log:** -``` -192.0.2.1 - - [11/Nov/2025:21:30:15 +0100] "GET /api/users HTTP/2.0" 200 1234 "https://media.schweitz.net" "Mozilla/5.0..." -``` - -**Fields:** -- **IP address**: Client IP (or Cloudflare IP if proxied) -- **Timestamp**: When request occurred -- **HTTP method**: GET, POST, etc. -- **Request path**: /api/users -- **Protocol**: HTTP/2.0 -- **Status code**: 200 (success), 404 (not found), 403 (forbidden), etc. -- **Bytes sent**: Response size -- **Referer**: Previous page -- **User agent**: Browser/client info - -## Useful Log Queries - -### Find Failed Login Attempts -```bash -# Status codes 401 (unauthorized) or 403 (forbidden) -grep -E " (401|403) " ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log - -# With IP addresses -grep -E " (401|403) " ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | awk '{print $1}' | sort | uniq -c | sort -nr -``` - -### Monitor Specific Service Access -```bash -# Jellyfin access -grep "media.schweitz.net" ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | tail -20 - -# Nextcloud uploads (POST requests) -grep "cloud.schweitz.net" ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | grep "POST" -``` - -### Identify High-Traffic IPs -```bash -# Top 10 IP addresses by request count -awk '{print $1}' ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | sort | uniq -c | sort -nr | head -10 -``` - -### Monitor SSL Certificate Activity -```bash -# Certificate renewal attempts -docker logs nginx-proxy-manager 2>&1 | grep -i "letsencrypt" - -# Certificate errors -docker logs nginx-proxy-manager 2>&1 | grep -i "certificate" | grep -i "error" -``` - -### Track API Usage -```bash -# API endpoint access -grep "/api/" ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log - -# Specific API endpoint -grep "/api/login" ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log -``` - -## Security Monitoring - -### Suspicious Activity Patterns - -**Brute force attempts:** -```bash -# Multiple 401s from same IP (potential brute force) -grep " 401 " ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | \ - awk '{print $1}' | sort | uniq -c | sort -nr | \ - awk '$1 > 10 {print "Potential brute force from " $2 " (" $1 " attempts)"}' -``` - -**Directory scanning:** -```bash -# Looking for 404s (scanning for vulnerabilities) -grep " 404 " ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | \ - grep -E "(wp-admin|phpmyadmin|admin|login\.php)" -``` - -**Unusual user agents:** -```bash -# Non-browser requests (potential bots/scrapers) -grep -v "Mozilla" ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | \ - grep -v "curl" | tail -20 -``` - -## Log Rotation - -**Automatic rotation configuration:** - -NPM handles basic rotation, but for long-term storage: - -```bash -# Create logrotate config -sudo tee /etc/logrotate.d/nginx-proxy-manager < /dev/null 2>&1 || true - endscript -} -EOF - -# Test logrotate config -sudo logrotate -d /etc/logrotate.d/nginx-proxy-manager -``` - -**Manual log cleanup:** -```bash -# Archive old logs -cd ~/docker-data/nginx-proxy-manager/data/logs/ -tar -czf logs-archive-$(date +%Y%m%d).tar.gz *.log -mv logs-archive-*.tar.gz ~/backups/ - -# Clean logs older than 30 days -find ~/docker-data/nginx-proxy-manager/data/logs/ -name "*.log" -mtime +30 -delete -``` - -## Centralized Logging (Future Enhancement) - -**Option 1: Ship logs to external service** - -Use a log aggregator like: -- Loki + Grafana (self-hosted) -- Elasticsearch + Kibana -- Splunk -- Cloud services (Datadog, Loggly) - -**Option 2: Syslog forwarding** - -Configure NPM to forward to syslog: -```nginx -# Add to NPM custom nginx config -access_log syslog:server=10.99.0.1:514,tag=nginx combined; -``` - -**Option 3: Promtail + Loki (Recommended)** - -Deploy Promtail container to tail NPM logs and send to Loki: -```yaml -# Future: stacks/promtail.yml -services: - promtail: - image: grafana/promtail:latest - volumes: - - /home/jpmschweitzer/docker-data/nginx-proxy-manager/data/logs:/var/log/nginx - - ./promtail-config.yml:/etc/promtail/config.yml - command: -config.file=/etc/promtail/config.yml -``` - -## Compliance Logging - -**For audit trails, log these events:** - -### Access Events -- ✅ All successful logins (200 responses to /login endpoints) -- ✅ Failed login attempts (401/403 responses) -- ✅ File downloads (GET requests with large response sizes) -- ✅ File uploads (POST/PUT requests) -- ✅ API calls (requests to /api/ paths) - -### Security Events -- ✅ SSL certificate renewals -- ✅ Configuration changes in NPM -- ✅ Rate limit violations -- ✅ Blocked IPs (403 responses) - -### Monitoring Events -- ✅ Service downtime (502/503 responses) -- ✅ Slow responses (response time tracking) -- ✅ High traffic patterns - -## Alerting Setup - -**Create alerts for critical events:** - -### Using Uptime Kuma - -Configure HTTP(s) monitors in Uptime Kuma: -- Monitor each public service -- Alert on downtime -- Track response times - -### Using Custom Scripts - -**Example: Alert on failed logins:** -```bash -#!/bin/bash -# /home/jpmschweitzer/scripts/monitor-failed-logins.sh - -THRESHOLD=10 -LOG_FILE="/home/jpmschweitzer/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log" - -# Count 401s in last 5 minutes -RECENT_FAILURES=$(find ~/docker-data/nginx-proxy-manager/data/logs/ -name "proxy-host-*.log" -mmin -5 -exec grep -c " 401 " {} + | awk '{s+=$1} END {print s}') - -if [ "$RECENT_FAILURES" -gt "$THRESHOLD" ]; then - echo "ALERT: $RECENT_FAILURES failed login attempts in last 5 minutes" | \ - mail -s "Security Alert: High Failed Login Rate" admin@schweitz.net -fi -``` - -**Run via cron:** -```cron -*/5 * * * * /home/jpmschweitzer/scripts/monitor-failed-logins.sh -``` - -## Performance Monitoring - -**Track service performance via logs:** - -### Response Time Analysis -```bash -# Extract response times (if configured in NPM) -grep "upstream_response_time" ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | \ - awk '{print $NF}' | sort -n | tail -20 -``` - -### Bandwidth Usage -```bash -# Sum bytes sent per service -awk '{sum+=$10} END {print "Total bytes: " sum " (" sum/1024/1024 " MB)"}' \ - ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-media*.log -``` - -### Most Accessed Endpoints -```bash -# Top 10 requested paths -awk '{print $7}' ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | \ - sort | uniq -c | sort -nr | head -10 -``` - -## Best Practices - -### Regular Log Review -- ✅ Check NPM logs weekly for suspicious activity -- ✅ Review SSL certificate status monthly -- ✅ Archive logs older than 30 days -- ✅ Monitor for unusual traffic patterns - -### Retention Policy -- **Active logs**: 30 days (on SSD) -- **Compressed archives**: 1 year (on HDD /mnt/media/backups/logs/) -- **Long-term storage**: Ship to external service if needed - -### Privacy Considerations -- ⚠️ Logs contain IP addresses (PII in EU) -- ⚠️ Don't log full request bodies (may contain passwords) -- ⚠️ Rotate/delete old logs per privacy policy -- ⚠️ Secure log access (only admins via VPN) - ---- - -**Summary:** -- All external access logged in NPM -- Logs accessible via UI, Docker, or files -- Use for security monitoring and audit trails -- Set up alerts for critical events -- Regular log review and rotation diff --git a/docs/services/organizr-widgets.md b/docs/services/organizr-widgets.md deleted file mode 100644 index c1ba06b..0000000 --- a/docs/services/organizr-widgets.md +++ /dev/null @@ -1,215 +0,0 @@ -# Organizr Service Control Widget - -A beautiful, responsive widget for managing on-demand services from your Organizr dashboard. - -## Features - -- ✨ **Real-time Status** - Live service status with container counts -- 🎮 **One-Click Control** - Start/Stop services with a single click -- 🔒 **Safety First** - Always-on services are protected and clearly marked -- 🎨 **Beautiful UI** - Dark theme that matches Organizr -- ⚡ **Auto-Refresh** - Updates every 10 seconds -- 📱 **Responsive** - Works on desktop, tablet, and mobile - -## Installation - -### Method 1: Organizr Custom Homepage Item (Recommended) - -1. **Copy the widget file** to a web-accessible location: - ```bash - # If you have a web server serving files from /var/www/html: - sudo cp organizr-widgets/service-control.html /var/www/html/widgets/ - - # Or use Organizr's public directory: - cp organizr-widgets/service-control.html /path/to/organizr/plugins/widgets/ - ``` - -2. **Add to Organizr Homepage**: - - Open Organizr - - Go to **Settings** → **Customize** → **Homepage Items** - - Click **Add New Item** - - Configure: - - **Name**: "Service Control" - - **Category**: Custom - - **Type**: iFrame - - **URL**: `http://localhost/widgets/service-control.html` (adjust path) - - **Minimum Authentication**: User - - **Enabled**: Yes - - Save - -3. **Add to Homepage**: - - Go to **Settings** → **Customize** → **Appearance** - - Edit your homepage layout - - Add the "Service Control" item to desired location - - Save - -### Method 2: Organizr Custom HTML Tab - -1. **Open Organizr Settings**: - - Settings → **Tab Editor** - -2. **Add New Tab**: - - Click **Add Tab** - - Configure: - - **Tab Name**: "Services" - - **Tab URL**: Leave empty - - **Category**: Custom - - **Type**: iFrame - - **Image**: `images/tabs/services.png` (or your choice) - -3. **Add Custom HTML**: - - In the same tab configuration, find **Custom HTML** section - - Copy and paste the entire contents of `service-control.html` - - Save - -4. **Access the Tab**: - - The "Services" tab will now appear in your Organizr sidebar - -### Method 3: Nginx Reverse Proxy Integration - -If you want to serve the widget through Nginx Proxy Manager: - -1. **Create a location** in your Organizr proxy host: - ```nginx - location /widgets/ { - alias /path/to/portainer-core/organizr-widgets/; - autoindex off; - } - ``` - -2. **Access via**: `https://your-organizr-domain.com/widgets/service-control.html` - -## Configuration - -### Changing API Endpoint - -If your core-api is not on `localhost:8083`, edit the widget file: - -```javascript -const API_BASE = 'http://your-server:8083'; // Change this line -``` - -### Adjusting Auto-Refresh Interval - -Default is 10 seconds. To change: - -```javascript -setInterval(fetchServices, 10000); // Change 10000 to desired milliseconds -``` - -### Customizing Displayed Services - -By default, the widget shows all stoppable services (excludes always-on infrastructure). - -To filter specific services, modify the `renderServices()` function: - -```javascript -const stoppableServices = services.filter(s => - !isAlwaysOn(s.name) && - ['jellyfin', 'nextcloud', 'gitea', 'ai-stack'].includes(s.name) // Add this line -); -``` - -## Service Groups - -The following service groups are defined (stopping one stops all in group): - -- **jellyfin**: jellyfin -- **nextcloud**: nextcloud (uses shared postgres-shared + redis-shared) -- **gitea**: gitea, gitea-db -- **ai-stack**: open-webui, ollama, qdrant -- **samba**: samba - -## Always-On Services (Cannot be stopped) - -These infrastructure services are protected: -- portainer -- nginx-proxy-manager -- core-api -- uptime-kuma -- organizr -- headscale -- watchtower -- netdata -- maintenance -- postgres-shared (shared database infrastructure) -- redis-shared (shared cache infrastructure) - -## Troubleshooting - -### "Failed to connect to API" - -**Problem**: Widget shows red error message - -**Solutions**: -1. Verify core-api is running: `docker ps | grep core-api` -2. Check core-api URL is correct (localhost vs IP address) -3. If accessing from remote, change `API_BASE` to full URL -4. Check browser console for CORS errors - -### CORS Issues - -If accessing widget from a different domain than core-api: - -**Option 1**: Update core-api CORS settings in `src/config.py` (in the core-api repository): -```python -cors_origins: list[str] = ["http://your-organizr-domain.com"] -``` - -**Option 2**: Proxy the API through same domain using Nginx - -### Services Not Appearing - -**Check**: -1. Services are deployed as Portainer stacks -2. Services have proper labels: `com.docker.compose.project` -3. Core-API can connect to Portainer -4. Check browser console for errors - -### Buttons Disabled - -**Expected Behavior**: -- Start button disabled when service is running -- Stop button disabled when service is stopped -- All buttons disabled for always-on services - -## API Endpoints Used - -The widget consumes these core-api endpoints: - -- `GET /infrastructure/services` - Fetch service list with status -- `GET /infrastructure/service-groups` - Fetch service groups and always-on list -- `POST /infrastructure/services/{name}/start` - Start a service -- `POST /infrastructure/services/{name}/stop` - Stop a service - -See [Core API Documentation](core-api.md) for full API reference. - -## Advanced Customization - -### Colors - -Edit the CSS variables in the `