From afb47c48d456bc5033f34227754a28e0abfa1c5e Mon Sep 17 00:00:00 2001 From: Jeroen Schweitzer Date: Wed, 17 Dec 2025 16:52:25 +0100 Subject: [PATCH] Bump version to 1.2.0 and update documentation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Update README.md: - Reflect current architecture (housekeeping, infrastructure, tools) - Document all API endpoints - Add environment variables reference - Update test instructions Update CHANGELOG.md: - Document housekeeping API addition - Document web scraper removal - Document test coverage improvements 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 --- CHANGELOG.md | 36 +++++++ README.md | 259 +++++++++++++++++++++++-------------------------- pyproject.toml | 2 +- 3 files changed, 157 insertions(+), 140 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6ee1cec..03b3c0c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,42 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [1.2.0] - 2025-12-17 + +### Added + +- **Housekeeping API** - Home Assistant integration for smart home control + - `GET /housekeeping/health` - HA connection status + - `GET /housekeeping/devices` - List controllable devices with optional domain/area filtering + - `GET /housekeeping/devices/{entity_id}` - Get device details + - `POST /housekeeping/devices/{entity_id}/control` - Control devices (turn_on, turn_off, toggle, set_brightness) + - `GET /housekeeping/scenes` - List available scenes + - `POST /housekeeping/scenes/{scene_id}/activate` - Activate a scene + - `GET /housekeeping/scripts` - List available scripts + - `POST /housekeeping/scripts/{script_id}/run` - Run a script + - `GET /housekeeping/automations` - List automations + - `POST /housekeeping/automations/{automation_id}/toggle` - Enable/disable automation + - `GET /housekeeping/history` - Query state history + - `GET /housekeeping/areas` - List rooms/areas +- Home Assistant REST API client (`src/clients/homeassistant_client.py`) +- Home Assistant configuration in credentials and settings +- Comprehensive test suite with 65% code coverage (285 tests) +- Tests for NPM client, Ollama client, AI client, OIDC authentication +- Tests for infrastructure, health, tools, and housekeeping endpoints + +### Removed + +- **Web Scraper** - Entire web scraping module removed + - `src/web_scraper/` directory deleted + - `/web-scraper/scrape` endpoint removed + - Trafilatura and BeautifulSoup dependencies removed from scraping use + +### Changed + +- Updated README.md with current architecture and all endpoints +- Tools controller now only contains DNS lookup functionality +- Health controller endpoints list updated to reflect current features + ## [1.1.2] - 2024-12-14 ### Added diff --git a/README.md b/README.md index 215c827..347fed5 100644 --- a/README.md +++ b/README.md @@ -1,33 +1,96 @@ # Core Code API -OpenAPI-compatible functions for Open WebUI, providing web scraping and data processing capabilities. +Central API service providing infrastructure management, home automation, and utility endpoints for the homelab ecosystem. ## Features -### Web Scraper -- Intelligent content extraction using Trafilatura -- BeautifulSoup fallback for complex pages -- Configurable content length limits -- Optional link extraction -- Perfect for feeding webpage content to LLMs +### Infrastructure Management +- **Portainer Integration**: Stack and container management +- **NPM Integration**: Nginx Proxy Manager domain and certificate management +- **Service Control**: Start/stop services with container orchestration + +### Home Automation (Housekeeping API) +- **Device Control**: Turn on/off, toggle, and set brightness for smart devices +- **Scene Activation**: Trigger Home Assistant scenes +- **Script Execution**: Run Home Assistant scripts +- **Automation Management**: Enable/disable automations +- **State History**: Query device state changes over time +- **Area Discovery**: List rooms and areas + +### Utilities +- **DNS Lookup**: Query DNS records (A, AAAA, MX, TXT, CNAME, NS, SOA, PTR) +- **Health Checks**: Comprehensive service health monitoring +- **AI Metrics Proxy**: Forward metrics requests to Core-AI service ## Architecture ``` src/ -├── config.py # Global application settings -├── logging_config.py # Logging configuration -├── base_schema.py # Base Pydantic models -├── main.py # FastAPI application entry point -└── web_scraper/ # Web scraper module - ├── __init__.py - ├── config.py # Module-specific settings - ├── schemas.py # Pydantic request/response models - ├── service.py # Business logic - ├── router.py # API routes - └── exceptions.py # Custom exceptions +├── config.py # Global application settings +├── logging_config.py # Logging configuration +├── base_controller.py # Base controller pattern +├── main.py # FastAPI application entry point +├── auth/ +│ └── oidc.py # OIDC authentication +├── clients/ +│ ├── homeassistant_client.py # Home Assistant REST client +│ ├── npm_client.py # Nginx Proxy Manager client +│ ├── ollama_client.py # Ollama LLM client +│ └── portainer_client.py # Portainer API client +├── controllers/ +│ ├── ai_controller.py # AI metrics proxy +│ ├── health_controller.py # Health endpoints +│ ├── housekeeping_controller.py # Home automation endpoints +│ ├── infrastructure_controller.py # Infrastructure management +│ ├── static_controller.py # Static file serving +│ └── tools_controller.py # DNS and utility tools +└── dns/ + ├── service.py # DNS lookup service + └── exceptions.py # DNS-specific exceptions ``` +## API Endpoints + +### Health +- `GET /` - Service info and documentation links +- `GET /health` - Basic health status +- `GET /health/full` - Detailed component health +- `GET /health/diagnostics` - Full diagnostic information + +### Infrastructure (`/infrastructure`) +- `GET /infrastructure/health` - Portainer/NPM connection status +- `GET /infrastructure/services` - List all services (stacks) +- `GET /infrastructure/services/{name}` - Get service details +- `GET /infrastructure/services/{name}/status` - Service status +- `POST /infrastructure/services/{name}/start` - Start service +- `POST /infrastructure/services/{name}/stop` - Stop service +- `GET /infrastructure/containers` - List containers +- `GET /infrastructure/containers/{name}` - Container details +- `GET /infrastructure/containers/{name}/logs` - Container logs +- `GET /infrastructure/ports` - List exposed ports +- `GET /infrastructure/domains` - List proxy domains +- `GET /infrastructure/widget-data` - Dashboard widget data + +### Housekeeping (`/housekeeping`) +- `GET /housekeeping/health` - Home Assistant connection status +- `GET /housekeeping/devices` - List controllable devices +- `GET /housekeeping/devices/{entity_id}` - Device details +- `POST /housekeeping/devices/{entity_id}/control` - Control device +- `GET /housekeeping/scenes` - List scenes +- `POST /housekeeping/scenes/{scene_id}/activate` - Activate scene +- `GET /housekeeping/scripts` - List scripts +- `POST /housekeeping/scripts/{script_id}/run` - Run script +- `GET /housekeeping/automations` - List automations +- `POST /housekeeping/automations/{automation_id}/toggle` - Toggle automation +- `GET /housekeeping/history` - State history +- `GET /housekeeping/areas` - List areas/rooms + +### Tools (`/tools`) +- `POST /tools/dns/lookup` - DNS record lookup + +### AI (`/ai`) +- `GET /ai/metrics` - Proxy to Core-AI metrics + ## Development ### Requirements @@ -40,13 +103,30 @@ src/ # Install dependencies pip install -r requirements.txt +# Copy credentials template +cp src/credentials.example.py src/credentials.py +# Edit src/credentials.py with your values + # Run locally uvicorn src.main:app --reload --host 0.0.0.0 --port 8083 ``` +### Running Tests + +```bash +# Run all tests +pytest + +# Run with coverage +pytest --cov=src --cov-report=term-missing + +# Run specific test file +pytest tests/test_housekeeping.py -v +``` + ### Adding New Dependencies -**Important**: Dependencies use major version pinning (`~=`) for automatic patch updates while preventing breaking changes. +Dependencies use major version pinning (`~=`) for automatic patch updates while preventing breaking changes. 1. Add package to `requirements.txt` with major version constraint: ``` @@ -58,14 +138,6 @@ uvicorn src.main:app --reload --host 0.0.0.0 --port 8083 docker restart core-api ``` -The container automatically runs `pip install -r requirements.txt` on every boot, so new dependencies are installed immediately on restart. - -**Version Pinning Best Practices**: -- Use `~=` (compatible release) for most packages: `fastapi~=0.115.0` -- Use `>=X,=0.3.17,<0.4.0` -- Allows automatic security patches without breaking changes -- Documented in PEP 440 - ### Docker Build ```bash @@ -88,130 +160,39 @@ docker run -p 8083:8083 core-code:latest ### Environment Variables -See `.env.example` for all available configuration options. +| Variable | Description | Default | +|----------|-------------|---------| +| `PORTAINER_URL` | Portainer API URL | `http://localhost:9000` | +| `PORTAINER_API_KEY` | Portainer API key | - | +| `NPM_URL` | Nginx Proxy Manager URL | `http://localhost:81` | +| `NPM_EMAIL` | NPM admin email | - | +| `NPM_PASSWORD` | NPM admin password | - | +| `HOMEASSISTANT_URL` | Home Assistant URL | `http://localhost:8123` | +| `HOMEASSISTANT_TOKEN` | HA long-lived access token | - | +| `OLLAMA_URL` | Ollama API URL | `http://localhost:11434` | +| `OIDC_ENABLED` | Enable OIDC auth | `false` | +| `OIDC_ISSUER` | OIDC issuer URL | - | +| `OIDC_AUDIENCE` | OIDC audience | - | ## API Documentation Once deployed, access documentation at: -- **Swagger UI**: http://192.168.86.149:8083/docs -- **ReDoc**: http://192.168.86.149:8083/redoc -- **OpenAPI Spec**: http://192.168.86.149:8083/openapi.json - -## Integration with Open WebUI - -### Method 1: Functions (OpenAPI Import) -1. In Open WebUI, navigate to Functions -2. Import from OpenAPI spec: `http://192.168.86.149:8083/openapi.json` -3. Use functions directly in chat - -### Method 2: Pipelines -1. Create a pipeline that calls Core Code API endpoints -2. Use as data source for LLM workflows - -### Method 3: Direct API Calls -```python -import httpx - -async with httpx.AsyncClient() as client: - response = await client.post( - "http://192.168.86.149:8083/web-scraper/scrape", - json={ - "url": "https://example.com", - "extract_main_content": True - } - ) - data = response.json() -``` - -## API Endpoints - -### Web Scraper - -**POST /web-scraper/scrape** - -Scrape and extract content from a website. - -Request: -```json -{ - "url": "https://example.com/article", - "extract_main_content": true, - "include_links": false, - "max_length": 10000 -} -``` - -Response: -```json -{ - "url": "https://example.com/article", - "title": "Article Title", - "content": "Extracted article content...", - "extracted_at": "2025-11-12T19:30:00Z", - "content_length": 5432, - "links": null -} -``` - -## Logging - -Logs are written to: -- **Console**: stdout (captured by Docker) -- **File**: `/app/logs/app.log` (persisted via volume mount) - -Log format: -``` -2025-11-12 19:30:00 | INFO | src.web_scraper.service:scrape_url:45 | Starting scrape for URL: https://example.com -``` +- **Swagger UI**: http://localhost:8083/docs +- **ReDoc**: http://localhost:8083/redoc +- **OpenAPI Spec**: http://localhost:8083/openapi.json ## Health Checks -- **Endpoint**: `GET /health` -- **Docker**: Automatic health checks configured -- **Response**: `{"status": "healthy"}` +- **Basic**: `GET /health` - Returns status and Ollama connection +- **Full**: `GET /health/full` - Returns all component statuses (503 if unhealthy) +- **Diagnostics**: `GET /health/diagnostics` - Detailed service information ## Security - Runs as non-root user (uid 1000) -- No authentication required (internal network only) +- OIDC authentication support via Authentik - CORS configured for same-network access -- Rate limiting: Not implemented (internal use only) - -## Future Modules - -The architecture supports adding new modules: -- Data transformation functions -- API integrations -- File processing -- Database queries - -Each module follows the same structure: -``` -src/ -└── module_name/ - ├── config.py - ├── schemas.py - ├── service.py - ├── router.py - └── exceptions.py -``` - -## Troubleshooting - -### Container won't start -```bash -docker logs core-code -``` - -### API not responding -```bash -curl http://192.168.86.149:8083/health -``` - -### Check OpenAPI spec -```bash -curl http://192.168.86.149:8083/openapi.json | jq -``` +- Admin endpoints require authentication when OIDC enabled ## License diff --git a/pyproject.toml b/pyproject.toml index 834d469..38c127f 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "core-api" -version = "1.1.2" +version = "1.2.0" description = "Core Code API - Infrastructure management and tools API" readme = "README.md" requires-python = ">=3.12"