cleanup of docs
This commit is contained in:
@@ -1,403 +0,0 @@
|
||||
# Code-Server Installation Guide
|
||||
|
||||
> Browser-based VSCode IDE running on host for full system access
|
||||
>
|
||||
> **Service Type:** Host-based (systemd service, not containerized)
|
||||
> **Purpose:** Replace SSH with persistent web-based development environment
|
||||
> **Access:** https://code.schweitz.net (via NPM with SSL)
|
||||
|
||||
---
|
||||
|
||||
## Why Host-Based?
|
||||
|
||||
Unlike other services in this stack, code-server runs directly on the host OS (not in Docker) to provide:
|
||||
|
||||
- Full access to host filesystem and configurations
|
||||
- Direct control over systemd services
|
||||
- Native Docker CLI access without Docker-in-Docker complexity
|
||||
- No permission issues when editing files across SSD/HDD
|
||||
- Persistent sessions that survive network disconnections
|
||||
|
||||
---
|
||||
|
||||
## Installation Steps
|
||||
|
||||
### 1. Install code-server
|
||||
|
||||
Run these commands on the tower-of-joy host:
|
||||
|
||||
```bash
|
||||
# Download and install code-server (version 4.x)
|
||||
curl -fsSL https://code-server.dev/install.sh | sh
|
||||
|
||||
# Verify installation
|
||||
code-server --version
|
||||
```
|
||||
|
||||
### 2. Create Configuration Directory
|
||||
|
||||
```bash
|
||||
# Create config directory
|
||||
mkdir -p ~/.config/code-server
|
||||
|
||||
# Create configuration file
|
||||
cat > ~/.config/code-server/config.yaml <<'EOF'
|
||||
bind-addr: 127.0.0.1:8084
|
||||
auth: password
|
||||
password: CHANGE_THIS_PASSWORD
|
||||
cert: false
|
||||
user-data-dir: /home/jpmschweitzer/docker-data/code-server/user-data
|
||||
extensions-dir: /home/jpmschweitzer/docker-data/code-server/extensions
|
||||
EOF
|
||||
|
||||
# Create data directories on SSD
|
||||
mkdir -p /home/jpmschweitzer/docker-data/code-server/{user-data,extensions}
|
||||
```
|
||||
|
||||
**IMPORTANT:** Replace `CHANGE_THIS_PASSWORD` with a strong password. This is a secondary auth layer (NPM will provide the primary authentication).
|
||||
|
||||
### 3. Create Systemd Service
|
||||
|
||||
```bash
|
||||
# Create service file
|
||||
sudo tee /etc/systemd/system/code-server.service > /dev/null <<'EOF'
|
||||
[Unit]
|
||||
Description=code-server - Browser-based VSCode IDE
|
||||
Documentation=https://coder.com/docs/code-server
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=exec
|
||||
ExecStart=/usr/bin/code-server
|
||||
Restart=always
|
||||
User=jpmschweitzer
|
||||
Group=jpmschweitzer
|
||||
Environment="PASSWORD_FROM_CONFIG=true"
|
||||
|
||||
# Hardening
|
||||
NoNewPrivileges=true
|
||||
PrivateTmp=true
|
||||
ProtectSystem=full
|
||||
ProtectHome=false
|
||||
ReadWritePaths=/home/jpmschweitzer /mnt/media
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
EOF
|
||||
|
||||
# Reload systemd daemon
|
||||
sudo systemctl daemon-reload
|
||||
|
||||
# Enable and start code-server
|
||||
sudo systemctl enable code-server
|
||||
sudo systemctl start code-server
|
||||
|
||||
# Check status
|
||||
sudo systemctl status code-server
|
||||
```
|
||||
|
||||
### 4. Verify Local Access
|
||||
|
||||
```bash
|
||||
# Test that code-server is running locally
|
||||
curl -I http://127.0.0.1:8084
|
||||
|
||||
# Should return HTTP 302 (redirect to login page)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Service Integration
|
||||
|
||||
### Uptime Kuma Monitoring
|
||||
|
||||
After code-server is running, add it to Uptime Kuma for health monitoring:
|
||||
|
||||
1. Open Uptime Kuma: http://192.168.86.149:3001
|
||||
2. Click **Add New Monitor**
|
||||
3. Configure the monitor:
|
||||
|
||||
**Monitor Settings:**
|
||||
- **Monitor Type:** HTTP(s)
|
||||
- **Friendly Name:** Code-Server
|
||||
- **URL:** https://code.schweitz.net
|
||||
- **Heartbeat Interval:** 60 seconds
|
||||
- **Retries:** 3
|
||||
- **Heartbeat Retry Interval:** 60 seconds
|
||||
- **Accepted Status Codes:** 200-299, 302 (redirect to login)
|
||||
- **Ignore TLS/SSL errors:** ❌ Disabled (cert should be valid)
|
||||
- **Tags:** Infrastructure, Development
|
||||
|
||||
4. Click **Save**
|
||||
|
||||
The monitor should show "Up" status once code-server is accessible through NPM.
|
||||
|
||||
### Organizr Dashboard Integration
|
||||
|
||||
Add code-server to your Organizr unified dashboard:
|
||||
|
||||
1. Open Organizr: https://home.schweitz.net
|
||||
2. Navigate to **Settings → Tab Editor**
|
||||
3. Click **Add Tab**
|
||||
|
||||
**Tab Configuration:**
|
||||
- **Tab Name:** Code-Server
|
||||
- **Tab URL:** https://code.schweitz.net
|
||||
- **Category:** Infrastructure (or create "Development" category)
|
||||
- **Icon:** `fa-code` or `fa-laptop-code`
|
||||
- **Active:** ✅ Enabled
|
||||
- **New Window:** ❌ Disabled (use iframe)
|
||||
|
||||
4. **Homepage Integration (Optional):**
|
||||
- Go to **Settings → Homepage Items**
|
||||
- Add custom HTML tile:
|
||||
|
||||
```html
|
||||
<div class="homepage-item">
|
||||
<a href="https://code.schweitz.net" target="_blank">
|
||||
<i class="fa fa-code fa-3x"></i>
|
||||
<span>Code-Server</span>
|
||||
</a>
|
||||
</div>
|
||||
```
|
||||
|
||||
5. Click **Save**
|
||||
|
||||
The code-server tab will now appear in your Organizr sidebar.
|
||||
|
||||
---
|
||||
|
||||
## Nginx Proxy Manager Configuration
|
||||
|
||||
### Create Proxy Host
|
||||
|
||||
1. Open NPM admin interface: http://192.168.86.149:81
|
||||
2. Navigate to **Hosts → Proxy Hosts → Add Proxy Host**
|
||||
|
||||
**Details Tab:**
|
||||
- **Domain Name:** `code.schweitz.net`
|
||||
- **Scheme:** `http`
|
||||
- **Forward Hostname/IP:** `192.168.86.149` (or `localhost`)
|
||||
- **Forward Port:** `8084`
|
||||
- **Block Common Exploits:** ✅ Enabled
|
||||
- **Websockets Support:** ✅ Enabled (critical for code-server)
|
||||
|
||||
**SSL Tab:**
|
||||
- **SSL Certificate:** Request New SSL Certificate
|
||||
- **Force SSL:** ✅ Enabled
|
||||
- **HTTP/2 Support:** ✅ Enabled
|
||||
- **HSTS Enabled:** ✅ Enabled
|
||||
- **Email:** your-email@example.com (for Let's Encrypt)
|
||||
- **Terms of Service:** ✅ Agree
|
||||
|
||||
**Access List Tab:**
|
||||
- Create new access list: "Code Server Access"
|
||||
- Configure basic auth or use NPM's built-in authentication
|
||||
|
||||
**Advanced Tab (optional):**
|
||||
```nginx
|
||||
# Increase timeout for long-running operations
|
||||
proxy_read_timeout 3600s;
|
||||
proxy_send_timeout 3600s;
|
||||
|
||||
# Proper headers for WebSocket support
|
||||
proxy_set_header Upgrade $http_upgrade;
|
||||
proxy_set_header Connection "upgrade";
|
||||
proxy_set_header Accept-Encoding gzip;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security Hardening
|
||||
|
||||
### Firewall Rules
|
||||
|
||||
```bash
|
||||
# Ensure port 8084 is NOT exposed to the internet
|
||||
sudo ufw status
|
||||
|
||||
# Port 8084 should only be accessible from localhost
|
||||
# External access ONLY through NPM on ports 80/443
|
||||
```
|
||||
|
||||
### Authentication Layers
|
||||
|
||||
Code-server will have **three layers of security**:
|
||||
|
||||
1. **NPM Access List** - Primary authentication via reverse proxy
|
||||
2. **code-server password** - Secondary authentication (from config.yaml)
|
||||
3. **HTTPS/SSL** - Encrypted transport via Let's Encrypt
|
||||
|
||||
### Recommended NPM Access List
|
||||
|
||||
Create an access list in NPM with:
|
||||
- Basic Auth username/password
|
||||
- IP whitelist (optional): Restrict to known IPs or Tailscale network
|
||||
- Rate limiting: Prevent brute force attacks
|
||||
|
||||
---
|
||||
|
||||
## Configuration Tips
|
||||
|
||||
### Extensions to Install
|
||||
|
||||
After first login, install these extensions:
|
||||
|
||||
```bash
|
||||
# Via code-server CLI
|
||||
code-server --install-extension ms-python.python
|
||||
code-server --install-extension ms-azuretools.vscode-docker
|
||||
code-server --install-extension eamodio.gitlens
|
||||
code-server --install-extension GitHub.copilot # If you have Copilot
|
||||
code-server --install-extension Codeium.codeium # Free AI assistant alternative
|
||||
```
|
||||
|
||||
### Custom Settings
|
||||
|
||||
Edit settings via UI or directly:
|
||||
|
||||
```bash
|
||||
nano ~/docker-data/code-server/user-data/User/settings.json
|
||||
```
|
||||
|
||||
Recommended settings:
|
||||
```json
|
||||
{
|
||||
"workbench.colorTheme": "Default Dark+",
|
||||
"terminal.integrated.defaultProfile.linux": "bash",
|
||||
"files.watcherExclude": {
|
||||
"**/node_modules/**": true,
|
||||
"**/.git/objects/**": true,
|
||||
"**/.venv/**": true
|
||||
},
|
||||
"editor.formatOnSave": true,
|
||||
"files.autoSave": "afterDelay"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Maintenance
|
||||
|
||||
### View Logs
|
||||
|
||||
```bash
|
||||
# Systemd logs
|
||||
sudo journalctl -u code-server -f
|
||||
|
||||
# Follow recent logs
|
||||
sudo journalctl -u code-server --since "10 minutes ago"
|
||||
```
|
||||
|
||||
### Restart Service
|
||||
|
||||
```bash
|
||||
sudo systemctl restart code-server
|
||||
```
|
||||
|
||||
### Update code-server
|
||||
|
||||
```bash
|
||||
# Re-run installation script
|
||||
curl -fsSL https://code-server.dev/install.sh | sh
|
||||
|
||||
# Restart service to use new version
|
||||
sudo systemctl restart code-server
|
||||
```
|
||||
|
||||
### Backup Configuration
|
||||
|
||||
Configuration is stored in:
|
||||
- `~/.config/code-server/config.yaml` - Main config
|
||||
- `~/docker-data/code-server/user-data/` - Settings, keybindings, snippets
|
||||
- `~/docker-data/code-server/extensions/` - Installed extensions
|
||||
|
||||
**Automated Backups:**
|
||||
|
||||
The maintenance container backs up code-server configuration nightly at 3 AM to `/mnt/media/backups/docker-configs/` with 30-day retention.
|
||||
|
||||
After installing code-server, restart the maintenance container to enable backups:
|
||||
|
||||
```bash
|
||||
docker restart maintenance
|
||||
|
||||
# Verify the mount is accessible
|
||||
docker exec maintenance ls -la /data/code-server-config
|
||||
|
||||
# Manually trigger a backup to test
|
||||
docker exec maintenance /scripts/backup-configs.sh
|
||||
|
||||
# Check backup logs
|
||||
docker exec maintenance cat /var/log/maintenance/backup-configs.log
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Service won't start
|
||||
|
||||
```bash
|
||||
# Check service status
|
||||
sudo systemctl status code-server
|
||||
|
||||
# Check logs for errors
|
||||
sudo journalctl -u code-server -n 50
|
||||
|
||||
# Verify config file syntax
|
||||
cat ~/.config/code-server/config.yaml
|
||||
```
|
||||
|
||||
### Can't connect via browser
|
||||
|
||||
```bash
|
||||
# Verify code-server is listening
|
||||
sudo netstat -tlnp | grep 8084
|
||||
|
||||
# Check NPM proxy host configuration
|
||||
# Ensure WebSocket support is enabled
|
||||
# Verify SSL certificate is valid
|
||||
```
|
||||
|
||||
### Performance issues
|
||||
|
||||
```bash
|
||||
# Check system resources
|
||||
htop
|
||||
|
||||
# Monitor code-server process
|
||||
top -p $(pgrep code-server)
|
||||
|
||||
# Increase file watcher limits if needed
|
||||
echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf
|
||||
sudo sysctl -p
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Integration Checklist
|
||||
|
||||
- [ ] code-server installed and running via systemd
|
||||
- [ ] NPM proxy host configured with SSL
|
||||
- [ ] External access working at https://code.schweitz.net
|
||||
- [ ] WebSocket connections working (terminal, file watcher)
|
||||
- [ ] Authentication layers tested (NPM + code-server password)
|
||||
- [ ] Maintenance container restarted to enable backups
|
||||
- [ ] Backup tested and verified
|
||||
- [ ] Uptime Kuma monitoring added
|
||||
- [ ] Organizr dashboard tab created
|
||||
- [ ] CONTAINERS.md documentation updated
|
||||
- [ ] README.md service table updated
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- **Code-Server Docs:** https://coder.com/docs/code-server
|
||||
- **NPM Docs:** https://nginxproxymanager.com/guide/
|
||||
- **Systemd Docs:** https://www.freedesktop.org/software/systemd/man/systemd.service.html
|
||||
|
||||
---
|
||||
|
||||
*Created: 2025-11-14*
|
||||
*System: tower-of-joy*
|
||||
@@ -1,381 +0,0 @@
|
||||
# Connecting Devices to Headscale VPN
|
||||
|
||||
> Step-by-step guide for connecting various devices to your Headscale mesh network
|
||||
> Created: 2025-11-11
|
||||
|
||||
## Overview
|
||||
|
||||
Once connected to Headscale, devices can access all services via mesh IPs (10.99.0.x):
|
||||
- Organizr: http://10.99.0.1:9999
|
||||
- Portainer: http://10.99.0.1:8001
|
||||
- Netdata: http://10.99.0.1:19999
|
||||
- All other services using mesh IPs
|
||||
|
||||
## Prerequisites
|
||||
|
||||
**You need a pre-auth key from Headscale:**
|
||||
|
||||
```bash
|
||||
# Generate a new pre-auth key (run on tower-of-joy)
|
||||
docker exec headscale headscale preauthkeys create --user homelab --expiration 24h
|
||||
|
||||
# Output example:
|
||||
# b4c17f9e9f01ea2e54b24e0369e2949bd28bfaf46944c634
|
||||
```
|
||||
|
||||
**Save this key - you'll use it to connect each device.**
|
||||
|
||||
## macOS (MacBook/iMac)
|
||||
|
||||
### Method 1: Tailscale App (Recommended)
|
||||
|
||||
**Step 1: Install Tailscale**
|
||||
```bash
|
||||
# Option A: Using Homebrew
|
||||
brew install tailscale
|
||||
|
||||
# Option B: Download from website
|
||||
# Visit: https://tailscale.com/download/mac
|
||||
# Download and install the .pkg file
|
||||
```
|
||||
|
||||
**Step 2: Start Tailscale**
|
||||
```bash
|
||||
# Start the Tailscale service
|
||||
sudo tailscaled install-system-daemon
|
||||
sudo /Applications/Tailscale.app/Contents/MacOS/Tailscale up
|
||||
```
|
||||
|
||||
**Step 3: Connect to Headscale**
|
||||
|
||||
Open the Tailscale app from menu bar → Preferences → Login Server
|
||||
|
||||
OR use command line:
|
||||
```bash
|
||||
sudo tailscale up --login-server=http://<your-public-ip>:8085 \
|
||||
--authkey=<your-preauth-key> \
|
||||
--hostname=macbook
|
||||
|
||||
# Example:
|
||||
# sudo tailscale up --login-server=http://your-public-ip:8085 \
|
||||
# --authkey=b4c17f9e9f01ea2e54b24e0369e2949bd28bfaf46944c634 \
|
||||
# --hostname=macbook
|
||||
```
|
||||
|
||||
**Step 4: Verify Connection**
|
||||
```bash
|
||||
# Check status
|
||||
tailscale status
|
||||
|
||||
# Should show:
|
||||
# 10.99.0.1 tower-of-joy homelab linux -
|
||||
# 10.99.0.2 macbook homelab darwin -
|
||||
|
||||
# Get your mesh IP
|
||||
tailscale ip -4
|
||||
# Example output: 10.99.0.2
|
||||
```
|
||||
|
||||
**Step 5: Test Access**
|
||||
```bash
|
||||
# Ping tower-of-joy
|
||||
ping 10.99.0.1
|
||||
|
||||
# Access Organizr
|
||||
open http://10.99.0.1:9999
|
||||
|
||||
# Or use curl
|
||||
curl http://10.99.0.1:9999
|
||||
```
|
||||
|
||||
### Method 2: Using Public IP (If port 8085 forwarded)
|
||||
|
||||
If you've forwarded port 8085 on your router:
|
||||
```bash
|
||||
sudo tailscale up --login-server=http://<your-public-ip>:8085 \
|
||||
--authkey=<your-preauth-key> \
|
||||
--hostname=macbook
|
||||
```
|
||||
|
||||
## Linux (Laptop/Desktop)
|
||||
|
||||
```bash
|
||||
# Install Tailscale
|
||||
curl -fsSL https://tailscale.com/install.sh | sh
|
||||
|
||||
# Connect to Headscale
|
||||
sudo tailscale up --login-server=http://<your-public-ip-or-local-ip>:8085 \
|
||||
--authkey=<your-preauth-key> \
|
||||
--hostname=linux-laptop
|
||||
|
||||
# Verify
|
||||
tailscale status
|
||||
ping 10.99.0.1
|
||||
|
||||
# Access Organizr
|
||||
xdg-open http://10.99.0.1:9999
|
||||
```
|
||||
|
||||
## Windows
|
||||
|
||||
**Step 1: Download Tailscale**
|
||||
- Visit: https://tailscale.com/download/windows
|
||||
- Download and run the installer
|
||||
|
||||
**Step 2: Configure Custom Login Server**
|
||||
- After installation, Tailscale runs in system tray
|
||||
- Right-click Tailscale icon → Settings → Admin Console URL
|
||||
- Change to: `http://<your-public-ip>:8085`
|
||||
|
||||
**Step 3: Login with Pre-Auth Key**
|
||||
- Right-click Tailscale icon → "Log in to Tailscale"
|
||||
- Use the pre-auth key when prompted
|
||||
|
||||
**Step 4: Verify**
|
||||
```powershell
|
||||
# In PowerShell or CMD
|
||||
tailscale status
|
||||
ping 10.99.0.1
|
||||
|
||||
# Access Organizr in browser
|
||||
start http://10.99.0.1:9999
|
||||
```
|
||||
|
||||
## iOS (iPhone/iPad)
|
||||
|
||||
**Step 1: Install Tailscale App**
|
||||
- Open App Store
|
||||
- Search "Tailscale"
|
||||
- Install official Tailscale app
|
||||
|
||||
**Step 2: Configure Custom Control Server**
|
||||
- Open Tailscale app
|
||||
- Tap Settings (gear icon)
|
||||
- Tap "Use Custom Control Server"
|
||||
- Enter: `http://<your-public-ip>:8085`
|
||||
|
||||
**Step 3: Connect**
|
||||
- Tap "Log In"
|
||||
- If prompted for key, use your pre-auth key
|
||||
- Grant VPN permissions when prompted
|
||||
|
||||
**Step 4: Test**
|
||||
- Open Safari
|
||||
- Navigate to: `http://10.99.0.1:9999`
|
||||
- You should see Organizr interface
|
||||
|
||||
## Android
|
||||
|
||||
**Step 1: Install Tailscale App**
|
||||
- Open Google Play Store
|
||||
- Search "Tailscale"
|
||||
- Install official Tailscale app
|
||||
|
||||
**Step 2: Configure Custom Control Server**
|
||||
- Open Tailscale app
|
||||
- Tap menu (three dots)
|
||||
- Settings → Use custom control server
|
||||
- Enter: `http://<your-public-ip>:8085`
|
||||
|
||||
**Step 3: Connect**
|
||||
- Tap "Log In"
|
||||
- Use pre-auth key if prompted
|
||||
- Grant VPN permissions
|
||||
|
||||
**Step 4: Test**
|
||||
- Open Chrome/Firefox
|
||||
- Navigate to: `http://10.99.0.1:9999`
|
||||
- Organizr should load
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Can't Connect to Headscale Server
|
||||
|
||||
**Problem:** "Failed to connect to login server"
|
||||
|
||||
**Solutions:**
|
||||
1. **Check port 8085 is forwarded:**
|
||||
```bash
|
||||
# Test from outside network
|
||||
curl http://<your-public-ip>:8085
|
||||
# Should return HTML or redirect
|
||||
```
|
||||
|
||||
2. **Check Headscale is running:**
|
||||
```bash
|
||||
# On tower-of-joy
|
||||
docker ps | grep headscale
|
||||
docker logs headscale
|
||||
```
|
||||
|
||||
3. **Use local IP if on same network:**
|
||||
```bash
|
||||
# Instead of public IP, use local IP
|
||||
sudo tailscale up --login-server=http://192.168.86.149:8085 ...
|
||||
```
|
||||
|
||||
### Connected but Can't Reach Services
|
||||
|
||||
**Problem:** Connected to VPN but can't access 10.99.0.1
|
||||
|
||||
**Check:**
|
||||
```bash
|
||||
# Verify VPN connection
|
||||
tailscale status
|
||||
# Should show tower-of-joy as online
|
||||
|
||||
# Test ping
|
||||
ping 10.99.0.1
|
||||
# Should respond
|
||||
|
||||
# Check firewall (on tower-of-joy)
|
||||
# Ensure mesh interface accepts traffic
|
||||
sudo iptables -L -n | grep tailscale
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# On tower-of-joy, allow traffic from mesh network
|
||||
sudo iptables -A INPUT -i tailscale0 -j ACCEPT
|
||||
```
|
||||
|
||||
### Pre-Auth Key Expired
|
||||
|
||||
**Problem:** "Invalid auth key"
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Generate new key (on tower-of-joy)
|
||||
docker exec headscale headscale preauthkeys create --user homelab --expiration 24h
|
||||
|
||||
# Use the new key to connect
|
||||
```
|
||||
|
||||
### DNS Not Resolving
|
||||
|
||||
**Problem:** Can ping 10.99.0.1 but browser can't resolve
|
||||
|
||||
**Solution:**
|
||||
- Use IP addresses directly: `http://10.99.0.1:9999`
|
||||
- Don't use hostnames unless you've configured DNS
|
||||
- Mesh IPs always work
|
||||
|
||||
## Verifying Your Connection
|
||||
|
||||
### Quick Test Checklist
|
||||
|
||||
From your newly connected device:
|
||||
|
||||
```bash
|
||||
# 1. Check VPN status
|
||||
tailscale status
|
||||
# Should show: connected, online
|
||||
|
||||
# 2. Get your mesh IP
|
||||
tailscale ip -4
|
||||
# Example: 10.99.0.2
|
||||
|
||||
# 3. Ping tower-of-joy
|
||||
ping -c 4 10.99.0.1
|
||||
# Should get responses
|
||||
|
||||
# 4. Test Organizr
|
||||
curl -I http://10.99.0.1:9999
|
||||
# Should return HTTP 200 OK
|
||||
|
||||
# 5. Test other services
|
||||
curl -I http://10.99.0.1:8001 # Portainer
|
||||
curl -I http://10.99.0.1:19999 # Netdata
|
||||
curl -I http://10.99.0.1:3001 # Uptime Kuma
|
||||
```
|
||||
|
||||
### View All Connected Devices
|
||||
|
||||
```bash
|
||||
# On tower-of-joy
|
||||
docker exec headscale headscale nodes list
|
||||
|
||||
# Shows all devices:
|
||||
# ID | Hostname | IP | Last Seen
|
||||
# 1 | tower-of-joy | 10.99.0.1 | now
|
||||
# 2 | macbook | 10.99.0.2 | now
|
||||
# 3 | iphone | 10.99.0.3 | now
|
||||
```
|
||||
|
||||
## Managing Devices
|
||||
|
||||
### Remove a Device
|
||||
|
||||
```bash
|
||||
# On tower-of-joy
|
||||
docker exec headscale headscale nodes list
|
||||
# Note the ID of the device to remove
|
||||
|
||||
docker exec headscale headscale nodes delete <ID>
|
||||
```
|
||||
|
||||
### Rename a Device
|
||||
|
||||
```bash
|
||||
docker exec headscale headscale nodes rename <OLD-NAME> <NEW-NAME>
|
||||
```
|
||||
|
||||
### Generate Multiple Pre-Auth Keys
|
||||
|
||||
```bash
|
||||
# For different devices or time periods
|
||||
docker exec headscale headscale preauthkeys create --user homelab --expiration 1h --reusable
|
||||
docker exec headscale headscale preauthkeys create --user homelab --expiration 7d
|
||||
docker exec headscale headscale preauthkeys create --user homelab --expiration 30d
|
||||
```
|
||||
|
||||
### List All Pre-Auth Keys
|
||||
|
||||
```bash
|
||||
docker exec headscale headscale preauthkeys list
|
||||
```
|
||||
|
||||
## Security Best Practices
|
||||
|
||||
### Key Expiration
|
||||
- ✅ Use short expiration for one-time device setups (1h-24h)
|
||||
- ✅ Use longer expiration for trusted devices (7d-30d)
|
||||
- ⚠️ Never use permanent keys
|
||||
|
||||
### Device Management
|
||||
- ✅ Use descriptive hostnames (macbook, work-laptop, phone)
|
||||
- ✅ Regularly review connected devices
|
||||
- ✅ Remove old/unused devices
|
||||
- ✅ Regenerate keys periodically
|
||||
|
||||
### Network Security
|
||||
- ✅ Headscale port (8085) should be firewalled to trusted IPs if possible
|
||||
- ✅ Use strong authentication on services
|
||||
- ✅ Consider adding MFA to Organizr for public access
|
||||
- ✅ Monitor Headscale logs for suspicious activity
|
||||
|
||||
## Quick Reference
|
||||
|
||||
### MacBook Connection (Your Current Task)
|
||||
```bash
|
||||
# 1. Generate pre-auth key (on tower-of-joy)
|
||||
docker exec headscale headscale preauthkeys create --user homelab --expiration 24h
|
||||
|
||||
# 2. On MacBook
|
||||
brew install tailscale
|
||||
sudo tailscale up --login-server=http://<your-public-ip>:8085 \
|
||||
--authkey=<key-from-step-1> \
|
||||
--hostname=macbook
|
||||
|
||||
# 3. Verify
|
||||
tailscale status
|
||||
open http://10.99.0.1:9999
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**Next Steps After Connecting:**
|
||||
1. Access Organizr: http://10.99.0.1:9999
|
||||
2. Complete setup wizard
|
||||
3. Add tabs for all services using mesh IPs
|
||||
4. Enjoy unified dashboard from anywhere!
|
||||
@@ -1,133 +0,0 @@
|
||||
# GPU Docker Configuration - Working Setup
|
||||
|
||||
> Successfully configured: 2025-11-11
|
||||
> System: tower-of-joy
|
||||
> GPU: NVIDIA GeForce RTX 2080 Ti
|
||||
> Driver: 470.256.02
|
||||
|
||||
## Problem Encountered
|
||||
|
||||
**Issue:** nvidia-container-toolkit 1.18.0 has a compatibility bug with NVIDIA driver 470.x
|
||||
- Version 1.18 changed default mode from "legacy" to "CDI"
|
||||
- Legacy mode detection fails with older drivers
|
||||
- Error: `libnvidia-ml.so.1: cannot open shared object file`
|
||||
|
||||
## Working Solution
|
||||
|
||||
**Downgrade to nvidia-container-toolkit 1.17.9-1**
|
||||
|
||||
All nvidia-container packages must be downgraded together:
|
||||
- nvidia-container-toolkit
|
||||
- nvidia-container-toolkit-base
|
||||
- libnvidia-container-tools
|
||||
- libnvidia-container1
|
||||
|
||||
## Installation Commands
|
||||
|
||||
```bash
|
||||
# Remove all nvidia-container packages
|
||||
sudo apt-get remove -y nvidia-container-toolkit nvidia-container-toolkit-base libnvidia-container-tools libnvidia-container1
|
||||
|
||||
# Clean up
|
||||
sudo apt-get autoremove -y
|
||||
|
||||
# Install all packages at version 1.17.9-1
|
||||
sudo apt-get install -y \
|
||||
nvidia-container-toolkit=1.17.9-1 \
|
||||
nvidia-container-toolkit-base=1.17.9-1 \
|
||||
libnvidia-container-tools=1.17.9-1 \
|
||||
libnvidia-container1=1.17.9-1
|
||||
|
||||
# Hold packages to prevent auto-upgrade
|
||||
sudo apt-mark hold nvidia-container-toolkit nvidia-container-toolkit-base libnvidia-container-tools libnvidia-container1
|
||||
|
||||
# Configure Docker runtime
|
||||
sudo nvidia-ctk runtime configure --runtime=docker --config=/etc/docker/daemon.json
|
||||
|
||||
# Rebuild library cache
|
||||
sudo ldconfig
|
||||
|
||||
# Restart Docker
|
||||
sudo systemctl restart docker
|
||||
```
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
# Test GPU access
|
||||
sudo docker run --rm --gpus all nvidia/cuda:11.8.0-runtime-ubuntu20.04 nvidia-smi
|
||||
```
|
||||
|
||||
**Expected output:** nvidia-smi showing RTX 2080 Ti
|
||||
|
||||
## Current Configuration
|
||||
|
||||
### /etc/docker/daemon.json
|
||||
```json
|
||||
{
|
||||
"runtimes": {
|
||||
"nvidia": {
|
||||
"args": [],
|
||||
"path": "nvidia-container-runtime"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Installed Versions
|
||||
```
|
||||
libnvidia-container-tools 1.17.9-1
|
||||
libnvidia-container1 1.17.9-1
|
||||
nvidia-container-toolkit 1.17.9-1
|
||||
nvidia-container-toolkit-base 1.17.9-1
|
||||
```
|
||||
|
||||
All packages are **held** to prevent automatic upgrade to 1.18.x
|
||||
|
||||
## Important Notes
|
||||
|
||||
1. **Do NOT upgrade** nvidia-container-toolkit to 1.18.x - it breaks compatibility with driver 470
|
||||
2. If you run `apt upgrade`, the packages are held and won't upgrade
|
||||
3. To check held packages: `apt-mark showhold`
|
||||
4. To unhold (not recommended): `sudo apt-mark unhold nvidia-container-toolkit`
|
||||
|
||||
## For Future Reference
|
||||
|
||||
If you need to update the NVIDIA driver:
|
||||
1. Driver 470.x is compatible with CUDA 11.4
|
||||
2. Driver 525+ is compatible with CUDA 12.x
|
||||
3. After driver update, may be able to use newer nvidia-container-toolkit
|
||||
|
||||
## Testing GPU in Containers
|
||||
|
||||
### Quick Test
|
||||
```bash
|
||||
docker run --rm --gpus all nvidia/cuda:11.8.0-runtime-ubuntu20.04 nvidia-smi
|
||||
```
|
||||
|
||||
### Test with Ollama
|
||||
```bash
|
||||
docker run --rm --gpus all ollama/ollama nvidia-smi
|
||||
```
|
||||
|
||||
### Test with Jellyfin (after deployment)
|
||||
Check Jellyfin Dashboard → Playback → Transcoding for NVIDIA NVENC option
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
If GPU stops working after system update:
|
||||
|
||||
```bash
|
||||
# Check if packages were upgraded
|
||||
dpkg -l | grep nvidia-container
|
||||
|
||||
# If upgraded to 1.18.x, re-run downgrade script:
|
||||
sudo bash /home/jpmschweitzer/Projects/tower-of-joy/scripts/gpu-fix-downgrade-all.sh
|
||||
```
|
||||
|
||||
## References
|
||||
|
||||
- NVIDIA Container Toolkit: https://github.com/NVIDIA/nvidia-container-toolkit
|
||||
- Driver 470 Release Notes: https://docs.nvidia.com/datacenter/tesla/tesla-release-notes-470-256-02/
|
||||
- Issue with 1.18.0: https://github.com/NVIDIA/nvidia-container-toolkit/issues
|
||||
- Our fix script: `/home/jpmschweitzer/Projects/tower-of-joy/scripts/gpu-fix-downgrade-all.sh`
|
||||
@@ -1,226 +0,0 @@
|
||||
# Headscale Setup & Connection Guide
|
||||
|
||||
> Generated: 2025-11-11
|
||||
> Server: tower-of-joy
|
||||
> Network Range: 10.99.0.0/16
|
||||
|
||||
## Service Status
|
||||
|
||||
✅ **Headscale is running**
|
||||
- Container: `headscale`
|
||||
- Web/API Port: `8085`
|
||||
- Metrics Port: `9090`
|
||||
- Server URL: `http://192.168.86.149:8085`
|
||||
|
||||
## User & Authentication
|
||||
|
||||
**User Created:** `homelab` (ID: 1)
|
||||
|
||||
**Pre-Auth Key (expires in 30 days, reusable):**
|
||||
```
|
||||
b4c17f9e9f01ea2e54b24e0369e2949bd28bfaf46944c634
|
||||
```
|
||||
|
||||
⚠️ **Security Note:** This key allows devices to join your mesh network. Keep it secure and regenerate after use if needed.
|
||||
|
||||
---
|
||||
|
||||
## Connecting This Server (tower-of-joy)
|
||||
|
||||
### Step 1: Install Tailscale Client
|
||||
|
||||
```bash
|
||||
curl -fsSL https://tailscale.com/install.sh | sh
|
||||
```
|
||||
|
||||
### Step 2: Connect to Headscale
|
||||
|
||||
```bash
|
||||
sudo tailscale up --login-server=http://192.168.86.149:8085 \
|
||||
--authkey=b4c17f9e9f01ea2e54b24e0369e2949bd28bfaf46944c634 \
|
||||
--accept-routes
|
||||
```
|
||||
|
||||
### Step 3: Verify Connection
|
||||
|
||||
```bash
|
||||
# Check Tailscale status
|
||||
sudo tailscale status
|
||||
|
||||
# Get your mesh IP (should be in 10.99.x.x range)
|
||||
sudo tailscale ip -4
|
||||
|
||||
# Test connectivity
|
||||
ping $(tailscale ip -4)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Connecting Other Devices (Laptop, Phone, etc.)
|
||||
|
||||
### On Linux/macOS
|
||||
|
||||
```bash
|
||||
# Install Tailscale
|
||||
curl -fsSL https://tailscale.com/install.sh | sh
|
||||
|
||||
# Connect to your Headscale server
|
||||
sudo tailscale up --login-server=http://192.168.86.149:8085 \
|
||||
--authkey=b4c17f9e9f01ea2e54b24e0369e2949bd28bfaf46944c634
|
||||
```
|
||||
|
||||
### On Windows
|
||||
|
||||
1. Download Tailscale from https://tailscale.com/download/windows
|
||||
2. Install and open Tailscale
|
||||
3. Run in PowerShell (as Administrator):
|
||||
```powershell
|
||||
tailscale up --login-server=http://192.168.86.149:8085 `
|
||||
--authkey=b4c17f9e9f01ea2e54b24e0369e2949bd28bfaf46944c634
|
||||
```
|
||||
|
||||
### On Android/iOS
|
||||
|
||||
1. Install Tailscale app from app store
|
||||
2. Open app settings
|
||||
3. Set "Control URL" to: `http://192.168.86.149:8085`
|
||||
4. Use auth key: `b4c17f9e9f01ea2e54b24e0369e2949bd28bfaf46944c634`
|
||||
|
||||
---
|
||||
|
||||
## Testing Remote SSH Access
|
||||
|
||||
Once devices are connected to the mesh:
|
||||
|
||||
```bash
|
||||
# From your laptop (after connecting to Headscale)
|
||||
|
||||
# Get tower-of-joy's mesh IP
|
||||
ssh jpmschweitzer@<tower-mesh-ip>
|
||||
|
||||
# Example (your actual IP will be something like 10.99.0.1)
|
||||
ssh jpmschweitzer@10.99.0.1
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- No port forwarding needed
|
||||
- No exposing SSH to internet
|
||||
- Encrypted peer-to-peer connections
|
||||
- Works from anywhere
|
||||
|
||||
---
|
||||
|
||||
## Headscale Management Commands
|
||||
|
||||
### List All Connected Devices
|
||||
```bash
|
||||
docker exec headscale headscale nodes list
|
||||
```
|
||||
|
||||
### List Users
|
||||
```bash
|
||||
docker exec headscale headscale users list
|
||||
```
|
||||
|
||||
### Generate New Pre-Auth Key
|
||||
```bash
|
||||
# 7 days, single-use
|
||||
docker exec headscale headscale preauthkeys create --user 1 --expiration 168h
|
||||
|
||||
# 30 days, reusable
|
||||
docker exec headscale headscale preauthkeys create --user 1 --expiration 720h --reusable
|
||||
```
|
||||
|
||||
### List Pre-Auth Keys
|
||||
```bash
|
||||
docker exec headscale headscale preauthkeys list --user 1
|
||||
```
|
||||
|
||||
### Remove a Device
|
||||
```bash
|
||||
# First, get the node ID
|
||||
docker exec headscale headscale nodes list
|
||||
|
||||
# Then delete by ID
|
||||
docker exec headscale headscale nodes delete <node-id>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Can't Connect to Headscale Server
|
||||
|
||||
1. **Check if Headscale is running:**
|
||||
```bash
|
||||
docker ps | grep headscale
|
||||
```
|
||||
|
||||
2. **Check firewall (if connecting from external network):**
|
||||
```bash
|
||||
sudo ufw status
|
||||
# If needed: sudo ufw allow 8085/tcp
|
||||
```
|
||||
|
||||
3. **View Headscale logs:**
|
||||
```bash
|
||||
docker logs headscale --tail 50
|
||||
```
|
||||
|
||||
### Devices Can't See Each Other
|
||||
|
||||
1. **Check device is registered:**
|
||||
```bash
|
||||
docker exec headscale headscale nodes list
|
||||
```
|
||||
|
||||
2. **Verify IPs are in the 10.99.0.0/16 range:**
|
||||
```bash
|
||||
sudo tailscale ip -4
|
||||
```
|
||||
|
||||
3. **Test direct ping:**
|
||||
```bash
|
||||
ping <other-device-mesh-ip>
|
||||
```
|
||||
|
||||
### Need to Regenerate Keys
|
||||
|
||||
```bash
|
||||
# Create new auth key
|
||||
docker exec headscale headscale preauthkeys create --user 1 --expiration 720h --reusable
|
||||
|
||||
# Update this document with the new key
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Configuration File Location
|
||||
|
||||
**Config:** `/home/jpmschweitzer/docker-data/headscale/config/config.yaml`
|
||||
**Database:** `/home/jpmschweitzer/docker-data/headscale/data/db.sqlite`
|
||||
|
||||
To edit configuration:
|
||||
1. Edit the config file
|
||||
2. Restart container: `docker restart headscale`
|
||||
3. Verify: `docker logs headscale --tail 20`
|
||||
|
||||
---
|
||||
|
||||
## Next Steps After Setup
|
||||
|
||||
1. ✅ Connect tower-of-joy to Headscale
|
||||
2. ✅ Connect your laptop/work devices
|
||||
3. ✅ Test SSH access from laptop to server
|
||||
4. ✅ Configure SSH key authentication for security
|
||||
5. ⚡ Add phone/tablet for remote monitoring
|
||||
6. ⚡ Set up exit node (optional - route all traffic through home)
|
||||
|
||||
---
|
||||
|
||||
## Resources
|
||||
|
||||
- **Headscale Docs:** https://headscale.net/
|
||||
- **Tailscale Client Docs:** https://tailscale.com/kb/
|
||||
- **Container Logs:** `docker logs headscale -f`
|
||||
- **Portainer:** http://192.168.86.149:8001
|
||||
@@ -1,326 +0,0 @@
|
||||
# NPM Logging & Audit Guide
|
||||
|
||||
> Centralized logging for all external access
|
||||
> Created: 2025-11-11
|
||||
|
||||
## Why NPM is the Logging Hub
|
||||
|
||||
**All public service access routes through NPM**, which means:
|
||||
- Every external request is logged
|
||||
- Failed authentication attempts tracked
|
||||
- Rate limiting violations recorded
|
||||
- SSL certificate renewals logged
|
||||
- Configuration changes audited
|
||||
|
||||
## Accessing NPM Logs
|
||||
|
||||
### Via NPM UI
|
||||
|
||||
**Access NPM admin panel:**
|
||||
- VPN: http://10.99.0.1:81
|
||||
- Local: http://192.168.86.149:81
|
||||
|
||||
**View logs:**
|
||||
1. Navigate to each Proxy Host
|
||||
2. Click "View Logs" button
|
||||
3. See real-time access logs
|
||||
4. Filter by status code, IP, user agent
|
||||
|
||||
### Via Docker Logs
|
||||
|
||||
**Real-time monitoring:**
|
||||
```bash
|
||||
# All NPM logs
|
||||
docker logs -f nginx-proxy-manager
|
||||
|
||||
# Filter for access logs only
|
||||
docker logs -f nginx-proxy-manager 2>&1 | grep -i "access"
|
||||
|
||||
# Filter for errors
|
||||
docker logs -f nginx-proxy-manager 2>&1 | grep -i "error"
|
||||
|
||||
# Filter for specific service (e.g., Jellyfin)
|
||||
docker logs -f nginx-proxy-manager 2>&1 | grep "media.schweitz.net"
|
||||
```
|
||||
|
||||
### Via Log Files
|
||||
|
||||
**Log location:**
|
||||
```bash
|
||||
# Access logs stored in container volume
|
||||
ls -lh ~/docker-data/nginx-proxy-manager/data/logs/
|
||||
|
||||
# View access logs
|
||||
tail -f ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log
|
||||
|
||||
# View error logs
|
||||
tail -f ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*_error.log
|
||||
```
|
||||
|
||||
## Log Format
|
||||
|
||||
**Standard NPM access log:**
|
||||
```
|
||||
192.0.2.1 - - [11/Nov/2025:21:30:15 +0100] "GET /api/users HTTP/2.0" 200 1234 "https://media.schweitz.net" "Mozilla/5.0..."
|
||||
```
|
||||
|
||||
**Fields:**
|
||||
- **IP address**: Client IP (or Cloudflare IP if proxied)
|
||||
- **Timestamp**: When request occurred
|
||||
- **HTTP method**: GET, POST, etc.
|
||||
- **Request path**: /api/users
|
||||
- **Protocol**: HTTP/2.0
|
||||
- **Status code**: 200 (success), 404 (not found), 403 (forbidden), etc.
|
||||
- **Bytes sent**: Response size
|
||||
- **Referer**: Previous page
|
||||
- **User agent**: Browser/client info
|
||||
|
||||
## Useful Log Queries
|
||||
|
||||
### Find Failed Login Attempts
|
||||
```bash
|
||||
# Status codes 401 (unauthorized) or 403 (forbidden)
|
||||
grep -E " (401|403) " ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log
|
||||
|
||||
# With IP addresses
|
||||
grep -E " (401|403) " ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | awk '{print $1}' | sort | uniq -c | sort -nr
|
||||
```
|
||||
|
||||
### Monitor Specific Service Access
|
||||
```bash
|
||||
# Jellyfin access
|
||||
grep "media.schweitz.net" ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | tail -20
|
||||
|
||||
# Nextcloud uploads (POST requests)
|
||||
grep "cloud.schweitz.net" ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | grep "POST"
|
||||
```
|
||||
|
||||
### Identify High-Traffic IPs
|
||||
```bash
|
||||
# Top 10 IP addresses by request count
|
||||
awk '{print $1}' ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | sort | uniq -c | sort -nr | head -10
|
||||
```
|
||||
|
||||
### Monitor SSL Certificate Activity
|
||||
```bash
|
||||
# Certificate renewal attempts
|
||||
docker logs nginx-proxy-manager 2>&1 | grep -i "letsencrypt"
|
||||
|
||||
# Certificate errors
|
||||
docker logs nginx-proxy-manager 2>&1 | grep -i "certificate" | grep -i "error"
|
||||
```
|
||||
|
||||
### Track API Usage
|
||||
```bash
|
||||
# API endpoint access
|
||||
grep "/api/" ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log
|
||||
|
||||
# Specific API endpoint
|
||||
grep "/api/login" ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log
|
||||
```
|
||||
|
||||
## Security Monitoring
|
||||
|
||||
### Suspicious Activity Patterns
|
||||
|
||||
**Brute force attempts:**
|
||||
```bash
|
||||
# Multiple 401s from same IP (potential brute force)
|
||||
grep " 401 " ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | \
|
||||
awk '{print $1}' | sort | uniq -c | sort -nr | \
|
||||
awk '$1 > 10 {print "Potential brute force from " $2 " (" $1 " attempts)"}'
|
||||
```
|
||||
|
||||
**Directory scanning:**
|
||||
```bash
|
||||
# Looking for 404s (scanning for vulnerabilities)
|
||||
grep " 404 " ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | \
|
||||
grep -E "(wp-admin|phpmyadmin|admin|login\.php)"
|
||||
```
|
||||
|
||||
**Unusual user agents:**
|
||||
```bash
|
||||
# Non-browser requests (potential bots/scrapers)
|
||||
grep -v "Mozilla" ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | \
|
||||
grep -v "curl" | tail -20
|
||||
```
|
||||
|
||||
## Log Rotation
|
||||
|
||||
**Automatic rotation configuration:**
|
||||
|
||||
NPM handles basic rotation, but for long-term storage:
|
||||
|
||||
```bash
|
||||
# Create logrotate config
|
||||
sudo tee /etc/logrotate.d/nginx-proxy-manager <<EOF
|
||||
/home/jpmschweitzer/docker-data/nginx-proxy-manager/data/logs/*.log {
|
||||
daily
|
||||
rotate 30
|
||||
compress
|
||||
delaycompress
|
||||
notifempty
|
||||
missingok
|
||||
create 0644 root root
|
||||
postrotate
|
||||
docker exec nginx-proxy-manager nginx -s reload > /dev/null 2>&1 || true
|
||||
endscript
|
||||
}
|
||||
EOF
|
||||
|
||||
# Test logrotate config
|
||||
sudo logrotate -d /etc/logrotate.d/nginx-proxy-manager
|
||||
```
|
||||
|
||||
**Manual log cleanup:**
|
||||
```bash
|
||||
# Archive old logs
|
||||
cd ~/docker-data/nginx-proxy-manager/data/logs/
|
||||
tar -czf logs-archive-$(date +%Y%m%d).tar.gz *.log
|
||||
mv logs-archive-*.tar.gz ~/backups/
|
||||
|
||||
# Clean logs older than 30 days
|
||||
find ~/docker-data/nginx-proxy-manager/data/logs/ -name "*.log" -mtime +30 -delete
|
||||
```
|
||||
|
||||
## Centralized Logging (Future Enhancement)
|
||||
|
||||
**Option 1: Ship logs to external service**
|
||||
|
||||
Use a log aggregator like:
|
||||
- Loki + Grafana (self-hosted)
|
||||
- Elasticsearch + Kibana
|
||||
- Splunk
|
||||
- Cloud services (Datadog, Loggly)
|
||||
|
||||
**Option 2: Syslog forwarding**
|
||||
|
||||
Configure NPM to forward to syslog:
|
||||
```nginx
|
||||
# Add to NPM custom nginx config
|
||||
access_log syslog:server=10.99.0.1:514,tag=nginx combined;
|
||||
```
|
||||
|
||||
**Option 3: Promtail + Loki (Recommended)**
|
||||
|
||||
Deploy Promtail container to tail NPM logs and send to Loki:
|
||||
```yaml
|
||||
# Future: stacks/promtail.yml
|
||||
services:
|
||||
promtail:
|
||||
image: grafana/promtail:latest
|
||||
volumes:
|
||||
- /home/jpmschweitzer/docker-data/nginx-proxy-manager/data/logs:/var/log/nginx
|
||||
- ./promtail-config.yml:/etc/promtail/config.yml
|
||||
command: -config.file=/etc/promtail/config.yml
|
||||
```
|
||||
|
||||
## Compliance Logging
|
||||
|
||||
**For audit trails, log these events:**
|
||||
|
||||
### Access Events
|
||||
- ✅ All successful logins (200 responses to /login endpoints)
|
||||
- ✅ Failed login attempts (401/403 responses)
|
||||
- ✅ File downloads (GET requests with large response sizes)
|
||||
- ✅ File uploads (POST/PUT requests)
|
||||
- ✅ API calls (requests to /api/ paths)
|
||||
|
||||
### Security Events
|
||||
- ✅ SSL certificate renewals
|
||||
- ✅ Configuration changes in NPM
|
||||
- ✅ Rate limit violations
|
||||
- ✅ Blocked IPs (403 responses)
|
||||
|
||||
### Monitoring Events
|
||||
- ✅ Service downtime (502/503 responses)
|
||||
- ✅ Slow responses (response time tracking)
|
||||
- ✅ High traffic patterns
|
||||
|
||||
## Alerting Setup
|
||||
|
||||
**Create alerts for critical events:**
|
||||
|
||||
### Using Uptime Kuma
|
||||
|
||||
Configure HTTP(s) monitors in Uptime Kuma:
|
||||
- Monitor each public service
|
||||
- Alert on downtime
|
||||
- Track response times
|
||||
|
||||
### Using Custom Scripts
|
||||
|
||||
**Example: Alert on failed logins:**
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# /home/jpmschweitzer/scripts/monitor-failed-logins.sh
|
||||
|
||||
THRESHOLD=10
|
||||
LOG_FILE="/home/jpmschweitzer/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log"
|
||||
|
||||
# Count 401s in last 5 minutes
|
||||
RECENT_FAILURES=$(find ~/docker-data/nginx-proxy-manager/data/logs/ -name "proxy-host-*.log" -mmin -5 -exec grep -c " 401 " {} + | awk '{s+=$1} END {print s}')
|
||||
|
||||
if [ "$RECENT_FAILURES" -gt "$THRESHOLD" ]; then
|
||||
echo "ALERT: $RECENT_FAILURES failed login attempts in last 5 minutes" | \
|
||||
mail -s "Security Alert: High Failed Login Rate" admin@schweitz.net
|
||||
fi
|
||||
```
|
||||
|
||||
**Run via cron:**
|
||||
```cron
|
||||
*/5 * * * * /home/jpmschweitzer/scripts/monitor-failed-logins.sh
|
||||
```
|
||||
|
||||
## Performance Monitoring
|
||||
|
||||
**Track service performance via logs:**
|
||||
|
||||
### Response Time Analysis
|
||||
```bash
|
||||
# Extract response times (if configured in NPM)
|
||||
grep "upstream_response_time" ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | \
|
||||
awk '{print $NF}' | sort -n | tail -20
|
||||
```
|
||||
|
||||
### Bandwidth Usage
|
||||
```bash
|
||||
# Sum bytes sent per service
|
||||
awk '{sum+=$10} END {print "Total bytes: " sum " (" sum/1024/1024 " MB)"}' \
|
||||
~/docker-data/nginx-proxy-manager/data/logs/proxy-host-media*.log
|
||||
```
|
||||
|
||||
### Most Accessed Endpoints
|
||||
```bash
|
||||
# Top 10 requested paths
|
||||
awk '{print $7}' ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | \
|
||||
sort | uniq -c | sort -nr | head -10
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Regular Log Review
|
||||
- ✅ Check NPM logs weekly for suspicious activity
|
||||
- ✅ Review SSL certificate status monthly
|
||||
- ✅ Archive logs older than 30 days
|
||||
- ✅ Monitor for unusual traffic patterns
|
||||
|
||||
### Retention Policy
|
||||
- **Active logs**: 30 days (on SSD)
|
||||
- **Compressed archives**: 1 year (on HDD /mnt/media/backups/logs/)
|
||||
- **Long-term storage**: Ship to external service if needed
|
||||
|
||||
### Privacy Considerations
|
||||
- ⚠️ Logs contain IP addresses (PII in EU)
|
||||
- ⚠️ Don't log full request bodies (may contain passwords)
|
||||
- ⚠️ Rotate/delete old logs per privacy policy
|
||||
- ⚠️ Secure log access (only admins via VPN)
|
||||
|
||||
---
|
||||
|
||||
**Summary:**
|
||||
- All external access logged in NPM
|
||||
- Logs accessible via UI, Docker, or files
|
||||
- Use for security monitoring and audit trails
|
||||
- Set up alerts for critical events
|
||||
- Regular log review and rotation
|
||||
@@ -1,215 +0,0 @@
|
||||
# Organizr Service Control Widget
|
||||
|
||||
A beautiful, responsive widget for managing on-demand services from your Organizr dashboard.
|
||||
|
||||
## Features
|
||||
|
||||
- ✨ **Real-time Status** - Live service status with container counts
|
||||
- 🎮 **One-Click Control** - Start/Stop services with a single click
|
||||
- 🔒 **Safety First** - Always-on services are protected and clearly marked
|
||||
- 🎨 **Beautiful UI** - Dark theme that matches Organizr
|
||||
- ⚡ **Auto-Refresh** - Updates every 10 seconds
|
||||
- 📱 **Responsive** - Works on desktop, tablet, and mobile
|
||||
|
||||
## Installation
|
||||
|
||||
### Method 1: Organizr Custom Homepage Item (Recommended)
|
||||
|
||||
1. **Copy the widget file** to a web-accessible location:
|
||||
```bash
|
||||
# If you have a web server serving files from /var/www/html:
|
||||
sudo cp organizr-widgets/service-control.html /var/www/html/widgets/
|
||||
|
||||
# Or use Organizr's public directory:
|
||||
cp organizr-widgets/service-control.html /path/to/organizr/plugins/widgets/
|
||||
```
|
||||
|
||||
2. **Add to Organizr Homepage**:
|
||||
- Open Organizr
|
||||
- Go to **Settings** → **Customize** → **Homepage Items**
|
||||
- Click **Add New Item**
|
||||
- Configure:
|
||||
- **Name**: "Service Control"
|
||||
- **Category**: Custom
|
||||
- **Type**: iFrame
|
||||
- **URL**: `http://localhost/widgets/service-control.html` (adjust path)
|
||||
- **Minimum Authentication**: User
|
||||
- **Enabled**: Yes
|
||||
- Save
|
||||
|
||||
3. **Add to Homepage**:
|
||||
- Go to **Settings** → **Customize** → **Appearance**
|
||||
- Edit your homepage layout
|
||||
- Add the "Service Control" item to desired location
|
||||
- Save
|
||||
|
||||
### Method 2: Organizr Custom HTML Tab
|
||||
|
||||
1. **Open Organizr Settings**:
|
||||
- Settings → **Tab Editor**
|
||||
|
||||
2. **Add New Tab**:
|
||||
- Click **Add Tab**
|
||||
- Configure:
|
||||
- **Tab Name**: "Services"
|
||||
- **Tab URL**: Leave empty
|
||||
- **Category**: Custom
|
||||
- **Type**: iFrame
|
||||
- **Image**: `images/tabs/services.png` (or your choice)
|
||||
|
||||
3. **Add Custom HTML**:
|
||||
- In the same tab configuration, find **Custom HTML** section
|
||||
- Copy and paste the entire contents of `service-control.html`
|
||||
- Save
|
||||
|
||||
4. **Access the Tab**:
|
||||
- The "Services" tab will now appear in your Organizr sidebar
|
||||
|
||||
### Method 3: Nginx Reverse Proxy Integration
|
||||
|
||||
If you want to serve the widget through Nginx Proxy Manager:
|
||||
|
||||
1. **Create a location** in your Organizr proxy host:
|
||||
```nginx
|
||||
location /widgets/ {
|
||||
alias /path/to/portainer-core/organizr-widgets/;
|
||||
autoindex off;
|
||||
}
|
||||
```
|
||||
|
||||
2. **Access via**: `https://your-organizr-domain.com/widgets/service-control.html`
|
||||
|
||||
## Configuration
|
||||
|
||||
### Changing API Endpoint
|
||||
|
||||
If your core-api is not on `localhost:8083`, edit the widget file:
|
||||
|
||||
```javascript
|
||||
const API_BASE = 'http://your-server:8083'; // Change this line
|
||||
```
|
||||
|
||||
### Adjusting Auto-Refresh Interval
|
||||
|
||||
Default is 10 seconds. To change:
|
||||
|
||||
```javascript
|
||||
setInterval(fetchServices, 10000); // Change 10000 to desired milliseconds
|
||||
```
|
||||
|
||||
### Customizing Displayed Services
|
||||
|
||||
By default, the widget shows all stoppable services (excludes always-on infrastructure).
|
||||
|
||||
To filter specific services, modify the `renderServices()` function:
|
||||
|
||||
```javascript
|
||||
const stoppableServices = services.filter(s =>
|
||||
!isAlwaysOn(s.name) &&
|
||||
['jellyfin', 'nextcloud', 'gitea', 'ai-stack'].includes(s.name) // Add this line
|
||||
);
|
||||
```
|
||||
|
||||
## Service Groups
|
||||
|
||||
The following service groups are defined (stopping one stops all in group):
|
||||
|
||||
- **jellyfin**: jellyfin
|
||||
- **nextcloud**: nextcloud (uses shared postgres-shared + redis-shared)
|
||||
- **gitea**: gitea, gitea-db
|
||||
- **ai-stack**: open-webui, ollama, qdrant
|
||||
- **samba**: samba
|
||||
|
||||
## Always-On Services (Cannot be stopped)
|
||||
|
||||
These infrastructure services are protected:
|
||||
- portainer
|
||||
- nginx-proxy-manager
|
||||
- core-api
|
||||
- uptime-kuma
|
||||
- organizr
|
||||
- headscale
|
||||
- watchtower
|
||||
- netdata
|
||||
- maintenance
|
||||
- postgres-shared (shared database infrastructure)
|
||||
- redis-shared (shared cache infrastructure)
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "Failed to connect to API"
|
||||
|
||||
**Problem**: Widget shows red error message
|
||||
|
||||
**Solutions**:
|
||||
1. Verify core-api is running: `docker ps | grep core-api`
|
||||
2. Check core-api URL is correct (localhost vs IP address)
|
||||
3. If accessing from remote, change `API_BASE` to full URL
|
||||
4. Check browser console for CORS errors
|
||||
|
||||
### CORS Issues
|
||||
|
||||
If accessing widget from a different domain than core-api:
|
||||
|
||||
**Option 1**: Update core-api CORS settings in `src/config.py` (in the core-api repository):
|
||||
```python
|
||||
cors_origins: list[str] = ["http://your-organizr-domain.com"]
|
||||
```
|
||||
|
||||
**Option 2**: Proxy the API through same domain using Nginx
|
||||
|
||||
### Services Not Appearing
|
||||
|
||||
**Check**:
|
||||
1. Services are deployed as Portainer stacks
|
||||
2. Services have proper labels: `com.docker.compose.project`
|
||||
3. Core-API can connect to Portainer
|
||||
4. Check browser console for errors
|
||||
|
||||
### Buttons Disabled
|
||||
|
||||
**Expected Behavior**:
|
||||
- Start button disabled when service is running
|
||||
- Stop button disabled when service is stopped
|
||||
- All buttons disabled for always-on services
|
||||
|
||||
## API Endpoints Used
|
||||
|
||||
The widget consumes these core-api endpoints:
|
||||
|
||||
- `GET /infrastructure/services` - Fetch service list with status
|
||||
- `GET /infrastructure/service-groups` - Fetch service groups and always-on list
|
||||
- `POST /infrastructure/services/{name}/start` - Start a service
|
||||
- `POST /infrastructure/services/{name}/stop` - Stop a service
|
||||
|
||||
See [Core API Documentation](core-api.md) for full API reference.
|
||||
|
||||
## Advanced Customization
|
||||
|
||||
### Colors
|
||||
|
||||
Edit the CSS variables in the `<style>` section:
|
||||
|
||||
```css
|
||||
.status-running {
|
||||
background: rgba(72, 187, 120, 0.2); /* Green background */
|
||||
color: #48bb78; /* Green text */
|
||||
}
|
||||
```
|
||||
|
||||
### Card Size
|
||||
|
||||
Adjust grid columns:
|
||||
|
||||
```css
|
||||
.service-grid {
|
||||
grid-template-columns: repeat(auto-fill, minmax(300px, 1fr));
|
||||
/* Change 300px to make cards wider/narrower */
|
||||
}
|
||||
```
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Core API Service](core-api.md) - Infrastructure management API
|
||||
- [Stacks Reference](../reference/stacks.md) - All deployed services
|
||||
- [Automation Reference](../reference/AUTOMATION.md) - Portainer REST API
|
||||
Reference in New Issue
Block a user