# Shared Infrastructure Architecture

**Purpose:** Centralized PostgreSQL and Redis services for all homelab stacks
**Benefits:** Resource efficiency, easier maintenance, unified backups, centralized monitoring

---

## Architecture Overview

```
┌─────────────────────────────────────────────────────────────┐
│                  Application Stacks                          │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐   │
│  │Authentik │  │  Gitea   │  │ Organizr │  │ Future   │   │
│  │          │  │          │  │          │  │  Stack   │   │
│  └────┬─────┘  └────┬─────┘  └────┬─────┘  └────┬─────┘   │
│       │             │              │              │          │
└───────┼─────────────┼──────────────┼──────────────┼─────────┘
        │             │              │              │
        └─────────────┴──────────────┴──────────────┘
                      │
        ┌─────────────▼──────────────────────────────┐
        │      Unified Data Plane Network            │
        │          (docker-dataplane)                │
        └─────────────┬──────────────────────────────┘
                      │
        ┌─────────────┴──────────────┐
        │                            │
   ┌────▼──────┐            ┌────────▼────┐
   │PostgreSQL │            │    Redis    │
   │  Shared   │            │   Shared    │
   │           │            │             │
   │ Databases:│            │ DB 0: Cache │
   │  - auth   │            │ DB 1: Auth  │
   │  - gitea  │            │ DB 2: Gitea │
   │  - organizr│            │ DB 3-15: .. │
   │  - future │            │             │
   └───────────┘            └─────────────┘
```

---

## Design Principles

### 1. **Database Isolation**
- Each application gets its own PostgreSQL database within the shared instance
- Each application gets its own Redis database number (0-15)
- Separate credentials per application for security

### 2. **Network Architecture**
- **Unified network:** `docker-dataplane` (external, bridge)
- All application containers connect to this single network
- Simplified connectivity: services discover each other by container name
- Replaces per-stack networks (ai-dataplane, nextcloud-network, etc.)

### 3. **Resource Allocation**
- PostgreSQL: No hard limits (homelab resource availability)
- Redis: No hard limits (lightweight Alpine image)
- Shared instances more efficient than per-stack deployments

### 4. **Backup Strategy**
- Single PostgreSQL backup covers all databases
- Automated pg_dumpall for disaster recovery
- Redis persistence: AOF + RDB snapshots

### 5. **Security Model**
- Each app has dedicated PostgreSQL user with access only to its database
- Redis AUTH with per-database passwords (optional)
- Network-level isolation via Docker networks

---

## Database Allocation Plan

### PostgreSQL Databases

| Database Name | Application | User | Purpose |
|---------------|-------------|------|---------|
| `authentik` | Authentik | `authentik_user` | User/group/policy storage |
| `gitea` | Gitea | `gitea_user` | Git repos, users, issues |
| `organizr` | Organizr | `organizr_user` | Dashboard configuration and user data |
| `future_app1` | TBD | `app1_user` | Reserved |
| `future_app2` | TBD | `app2_user` | Reserved |

**Note:** Existing services stay as-is:
- Nextcloud: MariaDB (existing, not migrated)
- Others can migrate over time if beneficial

### Redis Database Numbers

| DB# | Application | Purpose |
|-----|-------------|---------|
| 0 | Authentik | Sessions, cache, message queue |
| 1 | Available | Reserved for future applications |
| 2 | Available | Reserved for future applications |
| 3-15 | Available | Reserved for future applications |

**Note:** Each application uses a dedicated DB number to prevent key collisions while sharing the same Redis instance.

---

## Connection Configuration

### PostgreSQL Connection Strings

**From Docker containers:**
```
Host: postgres-shared
Port: 5432
Database: authentik
User: authentik_user
Password: <app-specific-password>
```

**From host:**
```
Host: localhost
Port: 5432
Database: authentik
User: authentik_user
Password: <app-specific-password>
```

### Redis Connection Strings

**From Docker containers:**
```
redis://redis-shared:6379/0  (for Authentik, DB 0)
redis://redis-shared:6379/1  (for future apps, DB 1)
redis://redis-shared:6379/2  (for future apps, DB 2)
```

**From host:**
```
redis://localhost:6379/0
```

---

## Migration Strategy

### Phase 1: Deploy Shared Infrastructure ✅ **COMPLETE**
1. ✅ Deployed `postgres-shared.yml` and `redis-shared.yml` via Portainer
2. ✅ Verified PostgreSQL 17 and Redis 7 running on docker-dataplane
3. ✅ Created initial databases and users (authentik, gitea)
4. ✅ Both services monitored via Uptime Kuma

### Phase 2: New Services (Authentik) 🚧 **IN PROGRESS**
1. ⏳ Deploy Authentik pointing to shared services
2. ⏳ Test thoroughly
3. ⏳ Validate no performance degradation

### Phase 3: Network Consolidation ✅ **COMPLETE**
1. ✅ All services migrated to docker-dataplane network
2. ✅ Removed 7 obsolete Docker networks
3. ✅ 18 containers on unified network for service discovery

### Phase 4: Migrate Existing Services (Optional)
1. **Gitea**: Already uses PostgreSQL
   - Export existing database
   - Create gitea database in shared PostgreSQL
   - Import data
   - Update Gitea stack to use shared PostgreSQL
   - Remove old gitea-db container

2. **Other services**: Evaluate case-by-case
   - Nextcloud: Keep MariaDB (complex migration, low benefit)
   - Future services: Use shared from day 1

---

## Advantages

✅ **Resource Efficiency**
- One PostgreSQL instance: ~1GB RAM vs ~300MB per instance
- Saves ~700MB RAM per additional service using PostgreSQL

✅ **Operational Simplicity**
- Single backup process for all PostgreSQL databases
- Centralized monitoring and health checks
- Easier version upgrades (upgrade once, affects all)

✅ **Performance**
- Shared connection pooling
- Better resource utilization
- Optimized caching with shared Redis

✅ **Scalability**
- Add new applications without deploying new database instances
- Up to 15 Redis databases (more than enough for homelab)

---

## Disadvantages & Mitigations

⚠️ **Single Point of Failure**
- **Mitigation:** Health checks, automated restarts, regular backups
- **Acceptable for homelab:** VPN access ensures admin can fix issues

⚠️ **Resource Contention**
- **Mitigation:** PostgreSQL connection limits per database
- **Mitigation:** Redis max memory policy (LRU eviction)
- **Monitoring:** Track per-database usage

⚠️ **Version Lock-In**
- **Mitigation:** Use latest stable PostgreSQL version (17)
- **Mitigation:** Test upgrades in staging before production deployment

---

## Monitoring & Maintenance

### Health Checks
- PostgreSQL: `pg_isready` every 30s
- Redis: `redis-cli ping` every 30s
- Application connectivity tests

### Uptime Kuma Integration ✅ **DEPLOYED**

Both shared services are monitored via Uptime Kuma with automatic monitor creation through the Core API:

**PostgreSQL Monitor** (ID 20):
```bash
curl -X POST http://192.168.86.149:8083/infrastructure/monitors \
  -H "Content-Type: application/json" \
  -d '{
    "type": "postgres",
    "name": "PostgreSQL Shared",
    "interval": 60,
    "retryInterval": 60,
    "maxretries": 3,
    "notificationIDList": [],
    "accepted_statuscodes": ["200-299"],
    "databaseConnectionString": "postgres://postgres:<url-encoded-password>@postgres-shared:5432/postgres"
  }'
```

**Redis Monitor** (ID 18):
```bash
curl -X POST http://192.168.86.149:8083/infrastructure/monitors \
  -H "Content-Type: application/json" \
  -d '{
    "type": "port",
    "name": "Redis Shared - Port Check",
    "hostname": "redis-shared",
    "port": 6379,
    "interval": 60,
    "retryInterval": 60,
    "maxretries": 3,
    "notificationIDList": [],
    "accepted_statuscodes": ["200-299"]
  }'
```

**Note:** When monitoring PostgreSQL with passwords containing special characters, URL-encode them (`/` → `%2F`, `=` → `%3D`).

### Backup Schedule
- **PostgreSQL:** Manual pg_dump to `/backups/` volume (automated backups pending)
- **Redis:** AOF persistence (real-time) enabled via `--appendonly yes`

### Performance Monitoring
- Query: `SELECT datname, numbackends FROM pg_stat_database;` (active connections)
- Redis: `INFO stats` (keyspace usage per database)
- Uptime Kuma dashboard: Real-time availability tracking

### Upgrade Path
1. Backup all databases
2. Test upgrade with docker-compose override
3. Deploy new version
4. Verify all applications connect successfully
5. Rollback if issues detected

---

## Implementation Status

1. ✅ Review architecture design
2. ✅ Create `postgres-shared.yml` and `redis-shared.yml` stacks
3. ✅ Deploy shared PostgreSQL 17 and Redis 7 via Portainer
4. ✅ Create initial databases (authentik, gitea)
5. ✅ Consolidate all services to docker-dataplane network
6. ✅ Implement Uptime Kuma monitoring via Core API
7. ✅ Document connection patterns and deployment procedures
8. ⏳ Update `authentik.yml` to use shared services (pending)
9. ⏳ Test Authentik with shared infrastructure (pending)

---

## Future Enhancements

- **PostgreSQL Read Replicas** (if needed for heavy read workloads)
- **Redis Sentinel** (high availability, probably overkill for homelab)
- **PgBouncer** (connection pooling if >100 connections needed)
- **Prometheus + Grafana** (metrics visualization)