ok... ok... I'll add it to git...
This commit is contained in:
@@ -0,0 +1,326 @@
|
||||
# NPM Logging & Audit Guide
|
||||
|
||||
> Centralized logging for all external access
|
||||
> Created: 2025-11-11
|
||||
|
||||
## Why NPM is the Logging Hub
|
||||
|
||||
**All public service access routes through NPM**, which means:
|
||||
- Every external request is logged
|
||||
- Failed authentication attempts tracked
|
||||
- Rate limiting violations recorded
|
||||
- SSL certificate renewals logged
|
||||
- Configuration changes audited
|
||||
|
||||
## Accessing NPM Logs
|
||||
|
||||
### Via NPM UI
|
||||
|
||||
**Access NPM admin panel:**
|
||||
- VPN: http://10.99.0.1:81
|
||||
- Local: http://192.168.86.149:81
|
||||
|
||||
**View logs:**
|
||||
1. Navigate to each Proxy Host
|
||||
2. Click "View Logs" button
|
||||
3. See real-time access logs
|
||||
4. Filter by status code, IP, user agent
|
||||
|
||||
### Via Docker Logs
|
||||
|
||||
**Real-time monitoring:**
|
||||
```bash
|
||||
# All NPM logs
|
||||
docker logs -f nginx-proxy-manager
|
||||
|
||||
# Filter for access logs only
|
||||
docker logs -f nginx-proxy-manager 2>&1 | grep -i "access"
|
||||
|
||||
# Filter for errors
|
||||
docker logs -f nginx-proxy-manager 2>&1 | grep -i "error"
|
||||
|
||||
# Filter for specific service (e.g., Jellyfin)
|
||||
docker logs -f nginx-proxy-manager 2>&1 | grep "media.schweitz.net"
|
||||
```
|
||||
|
||||
### Via Log Files
|
||||
|
||||
**Log location:**
|
||||
```bash
|
||||
# Access logs stored in container volume
|
||||
ls -lh ~/docker-data/nginx-proxy-manager/data/logs/
|
||||
|
||||
# View access logs
|
||||
tail -f ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log
|
||||
|
||||
# View error logs
|
||||
tail -f ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*_error.log
|
||||
```
|
||||
|
||||
## Log Format
|
||||
|
||||
**Standard NPM access log:**
|
||||
```
|
||||
192.0.2.1 - - [11/Nov/2025:21:30:15 +0100] "GET /api/users HTTP/2.0" 200 1234 "https://media.schweitz.net" "Mozilla/5.0..."
|
||||
```
|
||||
|
||||
**Fields:**
|
||||
- **IP address**: Client IP (or Cloudflare IP if proxied)
|
||||
- **Timestamp**: When request occurred
|
||||
- **HTTP method**: GET, POST, etc.
|
||||
- **Request path**: /api/users
|
||||
- **Protocol**: HTTP/2.0
|
||||
- **Status code**: 200 (success), 404 (not found), 403 (forbidden), etc.
|
||||
- **Bytes sent**: Response size
|
||||
- **Referer**: Previous page
|
||||
- **User agent**: Browser/client info
|
||||
|
||||
## Useful Log Queries
|
||||
|
||||
### Find Failed Login Attempts
|
||||
```bash
|
||||
# Status codes 401 (unauthorized) or 403 (forbidden)
|
||||
grep -E " (401|403) " ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log
|
||||
|
||||
# With IP addresses
|
||||
grep -E " (401|403) " ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | awk '{print $1}' | sort | uniq -c | sort -nr
|
||||
```
|
||||
|
||||
### Monitor Specific Service Access
|
||||
```bash
|
||||
# Jellyfin access
|
||||
grep "media.schweitz.net" ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | tail -20
|
||||
|
||||
# Nextcloud uploads (POST requests)
|
||||
grep "cloud.schweitz.net" ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | grep "POST"
|
||||
```
|
||||
|
||||
### Identify High-Traffic IPs
|
||||
```bash
|
||||
# Top 10 IP addresses by request count
|
||||
awk '{print $1}' ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | sort | uniq -c | sort -nr | head -10
|
||||
```
|
||||
|
||||
### Monitor SSL Certificate Activity
|
||||
```bash
|
||||
# Certificate renewal attempts
|
||||
docker logs nginx-proxy-manager 2>&1 | grep -i "letsencrypt"
|
||||
|
||||
# Certificate errors
|
||||
docker logs nginx-proxy-manager 2>&1 | grep -i "certificate" | grep -i "error"
|
||||
```
|
||||
|
||||
### Track API Usage
|
||||
```bash
|
||||
# API endpoint access
|
||||
grep "/api/" ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log
|
||||
|
||||
# Specific API endpoint
|
||||
grep "/api/login" ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log
|
||||
```
|
||||
|
||||
## Security Monitoring
|
||||
|
||||
### Suspicious Activity Patterns
|
||||
|
||||
**Brute force attempts:**
|
||||
```bash
|
||||
# Multiple 401s from same IP (potential brute force)
|
||||
grep " 401 " ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | \
|
||||
awk '{print $1}' | sort | uniq -c | sort -nr | \
|
||||
awk '$1 > 10 {print "Potential brute force from " $2 " (" $1 " attempts)"}'
|
||||
```
|
||||
|
||||
**Directory scanning:**
|
||||
```bash
|
||||
# Looking for 404s (scanning for vulnerabilities)
|
||||
grep " 404 " ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | \
|
||||
grep -E "(wp-admin|phpmyadmin|admin|login\.php)"
|
||||
```
|
||||
|
||||
**Unusual user agents:**
|
||||
```bash
|
||||
# Non-browser requests (potential bots/scrapers)
|
||||
grep -v "Mozilla" ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | \
|
||||
grep -v "curl" | tail -20
|
||||
```
|
||||
|
||||
## Log Rotation
|
||||
|
||||
**Automatic rotation configuration:**
|
||||
|
||||
NPM handles basic rotation, but for long-term storage:
|
||||
|
||||
```bash
|
||||
# Create logrotate config
|
||||
sudo tee /etc/logrotate.d/nginx-proxy-manager <<EOF
|
||||
/home/jpmschweitzer/docker-data/nginx-proxy-manager/data/logs/*.log {
|
||||
daily
|
||||
rotate 30
|
||||
compress
|
||||
delaycompress
|
||||
notifempty
|
||||
missingok
|
||||
create 0644 root root
|
||||
postrotate
|
||||
docker exec nginx-proxy-manager nginx -s reload > /dev/null 2>&1 || true
|
||||
endscript
|
||||
}
|
||||
EOF
|
||||
|
||||
# Test logrotate config
|
||||
sudo logrotate -d /etc/logrotate.d/nginx-proxy-manager
|
||||
```
|
||||
|
||||
**Manual log cleanup:**
|
||||
```bash
|
||||
# Archive old logs
|
||||
cd ~/docker-data/nginx-proxy-manager/data/logs/
|
||||
tar -czf logs-archive-$(date +%Y%m%d).tar.gz *.log
|
||||
mv logs-archive-*.tar.gz ~/backups/
|
||||
|
||||
# Clean logs older than 30 days
|
||||
find ~/docker-data/nginx-proxy-manager/data/logs/ -name "*.log" -mtime +30 -delete
|
||||
```
|
||||
|
||||
## Centralized Logging (Future Enhancement)
|
||||
|
||||
**Option 1: Ship logs to external service**
|
||||
|
||||
Use a log aggregator like:
|
||||
- Loki + Grafana (self-hosted)
|
||||
- Elasticsearch + Kibana
|
||||
- Splunk
|
||||
- Cloud services (Datadog, Loggly)
|
||||
|
||||
**Option 2: Syslog forwarding**
|
||||
|
||||
Configure NPM to forward to syslog:
|
||||
```nginx
|
||||
# Add to NPM custom nginx config
|
||||
access_log syslog:server=10.99.0.1:514,tag=nginx combined;
|
||||
```
|
||||
|
||||
**Option 3: Promtail + Loki (Recommended)**
|
||||
|
||||
Deploy Promtail container to tail NPM logs and send to Loki:
|
||||
```yaml
|
||||
# Future: stacks/promtail.yml
|
||||
services:
|
||||
promtail:
|
||||
image: grafana/promtail:latest
|
||||
volumes:
|
||||
- /home/jpmschweitzer/docker-data/nginx-proxy-manager/data/logs:/var/log/nginx
|
||||
- ./promtail-config.yml:/etc/promtail/config.yml
|
||||
command: -config.file=/etc/promtail/config.yml
|
||||
```
|
||||
|
||||
## Compliance Logging
|
||||
|
||||
**For audit trails, log these events:**
|
||||
|
||||
### Access Events
|
||||
- ✅ All successful logins (200 responses to /login endpoints)
|
||||
- ✅ Failed login attempts (401/403 responses)
|
||||
- ✅ File downloads (GET requests with large response sizes)
|
||||
- ✅ File uploads (POST/PUT requests)
|
||||
- ✅ API calls (requests to /api/ paths)
|
||||
|
||||
### Security Events
|
||||
- ✅ SSL certificate renewals
|
||||
- ✅ Configuration changes in NPM
|
||||
- ✅ Rate limit violations
|
||||
- ✅ Blocked IPs (403 responses)
|
||||
|
||||
### Monitoring Events
|
||||
- ✅ Service downtime (502/503 responses)
|
||||
- ✅ Slow responses (response time tracking)
|
||||
- ✅ High traffic patterns
|
||||
|
||||
## Alerting Setup
|
||||
|
||||
**Create alerts for critical events:**
|
||||
|
||||
### Using Uptime Kuma
|
||||
|
||||
Configure HTTP(s) monitors in Uptime Kuma:
|
||||
- Monitor each public service
|
||||
- Alert on downtime
|
||||
- Track response times
|
||||
|
||||
### Using Custom Scripts
|
||||
|
||||
**Example: Alert on failed logins:**
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# /home/jpmschweitzer/scripts/monitor-failed-logins.sh
|
||||
|
||||
THRESHOLD=10
|
||||
LOG_FILE="/home/jpmschweitzer/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log"
|
||||
|
||||
# Count 401s in last 5 minutes
|
||||
RECENT_FAILURES=$(find ~/docker-data/nginx-proxy-manager/data/logs/ -name "proxy-host-*.log" -mmin -5 -exec grep -c " 401 " {} + | awk '{s+=$1} END {print s}')
|
||||
|
||||
if [ "$RECENT_FAILURES" -gt "$THRESHOLD" ]; then
|
||||
echo "ALERT: $RECENT_FAILURES failed login attempts in last 5 minutes" | \
|
||||
mail -s "Security Alert: High Failed Login Rate" admin@schweitz.net
|
||||
fi
|
||||
```
|
||||
|
||||
**Run via cron:**
|
||||
```cron
|
||||
*/5 * * * * /home/jpmschweitzer/scripts/monitor-failed-logins.sh
|
||||
```
|
||||
|
||||
## Performance Monitoring
|
||||
|
||||
**Track service performance via logs:**
|
||||
|
||||
### Response Time Analysis
|
||||
```bash
|
||||
# Extract response times (if configured in NPM)
|
||||
grep "upstream_response_time" ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | \
|
||||
awk '{print $NF}' | sort -n | tail -20
|
||||
```
|
||||
|
||||
### Bandwidth Usage
|
||||
```bash
|
||||
# Sum bytes sent per service
|
||||
awk '{sum+=$10} END {print "Total bytes: " sum " (" sum/1024/1024 " MB)"}' \
|
||||
~/docker-data/nginx-proxy-manager/data/logs/proxy-host-media*.log
|
||||
```
|
||||
|
||||
### Most Accessed Endpoints
|
||||
```bash
|
||||
# Top 10 requested paths
|
||||
awk '{print $7}' ~/docker-data/nginx-proxy-manager/data/logs/proxy-host-*.log | \
|
||||
sort | uniq -c | sort -nr | head -10
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Regular Log Review
|
||||
- ✅ Check NPM logs weekly for suspicious activity
|
||||
- ✅ Review SSL certificate status monthly
|
||||
- ✅ Archive logs older than 30 days
|
||||
- ✅ Monitor for unusual traffic patterns
|
||||
|
||||
### Retention Policy
|
||||
- **Active logs**: 30 days (on SSD)
|
||||
- **Compressed archives**: 1 year (on HDD /mnt/media/backups/logs/)
|
||||
- **Long-term storage**: Ship to external service if needed
|
||||
|
||||
### Privacy Considerations
|
||||
- ⚠️ Logs contain IP addresses (PII in EU)
|
||||
- ⚠️ Don't log full request bodies (may contain passwords)
|
||||
- ⚠️ Rotate/delete old logs per privacy policy
|
||||
- ⚠️ Secure log access (only admins via VPN)
|
||||
|
||||
---
|
||||
|
||||
**Summary:**
|
||||
- All external access logged in NPM
|
||||
- Logs accessible via UI, Docker, or files
|
||||
- Use for security monitoring and audit trails
|
||||
- Set up alerts for critical events
|
||||
- Regular log review and rotation
|
||||
Reference in New Issue
Block a user