jpmschweitzer/core-api
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 5 Wiki Activity

Bump version to 1.2.0 and update documentation
Build and Push / build (release) Successful in 39s
Details

Browse Source 
Update README.md:
- Reflect current architecture (housekeeping, infrastructure, tools)
- Document all API endpoints
- Add environment variables reference
- Update test instructions

Update CHANGELOG.md:
- Document housekeeping API addition
- Document web scraper removal
- Document test coverage improvements

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
jpmschweitzer
2025-12-17 16:52:25 +01:00
co-authored by Claude Opus 4.5
parent a22e168666
commit afb47c48d4
3 changed files with 157 additions and 140 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+120 -139
View File
@@ -1,33 +1,96 @@
# Core Code API
OpenAPI-compatible functions for Open WebUI, providing web scraping and data processing capabilities.
Central API service providing infrastructure management, home automation, and utility endpoints for the homelab ecosystem.
## Features
### Web Scraper
- Intelligent content extraction using Trafilatura
- BeautifulSoup fallback for complex pages
- Configurable content length limits
- Optional link extraction
- Perfect for feeding webpage content to LLMs
### Infrastructure Management
- **Portainer Integration**: Stack and container management
- **NPM Integration**: Nginx Proxy Manager domain and certificate management
- **Service Control**: Start/stop services with container orchestration
### Home Automation (Housekeeping API)
- **Device Control**: Turn on/off, toggle, and set brightness for smart devices
- **Scene Activation**: Trigger Home Assistant scenes
- **Script Execution**: Run Home Assistant scripts
- **Automation Management**: Enable/disable automations
- **State History**: Query device state changes over time
- **Area Discovery**: List rooms and areas
### Utilities
- **DNS Lookup**: Query DNS records (A, AAAA, MX, TXT, CNAME, NS, SOA, PTR)
- **Health Checks**: Comprehensive service health monitoring
- **AI Metrics Proxy**: Forward metrics requests to Core-AI service
## Architecture
```
src/
├── config.py # Global application settings
├── logging_config.py # Logging configuration
├── base_schema.py # Base Pydantic models
├── main.py # FastAPI application entry point
└── web_scraper/ # Web scraper module
├── __init__.py
├── config.py # Module-specific settings
├── schemas.py # Pydantic request/response models
├── service.py # Business logic
├── router.py # API routes
└── exceptions.py # Custom exceptions
├── config.py # Global application settings
├── logging_config.py # Logging configuration
├── base_controller.py # Base controller pattern
├── main.py # FastAPI application entry point
├── auth/
│ └── oidc.py # OIDC authentication
├── clients/
│ ├── homeassistant_client.py # Home Assistant REST client
│ ├── npm_client.py # Nginx Proxy Manager client
│ ├── ollama_client.py # Ollama LLM client
│ └── portainer_client.py # Portainer API client
├── controllers/
│ ├── ai_controller.py # AI metrics proxy
│ ├── health_controller.py # Health endpoints
│ ├── housekeeping_controller.py # Home automation endpoints
│ ├── infrastructure_controller.py # Infrastructure management
│ ├── static_controller.py # Static file serving
│ └── tools_controller.py # DNS and utility tools
└── dns/
├── service.py # DNS lookup service
└── exceptions.py # DNS-specific exceptions
```
## API Endpoints
### Health
- `GET /` - Service info and documentation links
- `GET /health` - Basic health status
- `GET /health/full` - Detailed component health
- `GET /health/diagnostics` - Full diagnostic information
### Infrastructure (`/infrastructure`)
- `GET /infrastructure/health` - Portainer/NPM connection status
- `GET /infrastructure/services` - List all services (stacks)
- `GET /infrastructure/services/{name}` - Get service details
- `GET /infrastructure/services/{name}/status` - Service status
- `POST /infrastructure/services/{name}/start` - Start service
- `POST /infrastructure/services/{name}/stop` - Stop service
- `GET /infrastructure/containers` - List containers
- `GET /infrastructure/containers/{name}` - Container details
- `GET /infrastructure/containers/{name}/logs` - Container logs
- `GET /infrastructure/ports` - List exposed ports
- `GET /infrastructure/domains` - List proxy domains
- `GET /infrastructure/widget-data` - Dashboard widget data
### Housekeeping (`/housekeeping`)
- `GET /housekeeping/health` - Home Assistant connection status
- `GET /housekeeping/devices` - List controllable devices
- `GET /housekeeping/devices/{entity_id}` - Device details
- `POST /housekeeping/devices/{entity_id}/control` - Control device
- `GET /housekeeping/scenes` - List scenes
- `POST /housekeeping/scenes/{scene_id}/activate` - Activate scene
- `GET /housekeeping/scripts` - List scripts
- `POST /housekeeping/scripts/{script_id}/run` - Run script
- `GET /housekeeping/automations` - List automations
- `POST /housekeeping/automations/{automation_id}/toggle` - Toggle automation
- `GET /housekeeping/history` - State history
- `GET /housekeeping/areas` - List areas/rooms
### Tools (`/tools`)
- `POST /tools/dns/lookup` - DNS record lookup
### AI (`/ai`)
- `GET /ai/metrics` - Proxy to Core-AI metrics
## Development
### Requirements
@@ -40,13 +103,30 @@ src/
# Install dependencies
pip install -r requirements.txt
# Copy credentials template
cp src/credentials.example.py src/credentials.py
# Edit src/credentials.py with your values
# Run locally
uvicorn src.main:app --reload --host 0.0.0.0 --port 8083
```
### Running Tests
```bash
# Run all tests
pytest
# Run with coverage
pytest --cov=src --cov-report=term-missing
# Run specific test file
pytest tests/test_housekeeping.py -v
```
### Adding New Dependencies
**Important**: Dependencies use major version pinning (`~=`) for automatic patch updates while preventing breaking changes.
Dependencies use major version pinning (`~=`) for automatic patch updates while preventing breaking changes.
1. Add package to `requirements.txt` with major version constraint:
```
@@ -58,14 +138,6 @@ uvicorn src.main:app --reload --host 0.0.0.0 --port 8083
docker restart core-api
```
The container automatically runs `pip install -r requirements.txt` on every boot, so new dependencies are installed immediately on restart.
**Version Pinning Best Practices**:
- Use `~=` (compatible release) for most packages: `fastapi~=0.115.0`
- Use `>=X,<Y` for complex constraints: `langchain-core>=0.3.17,<0.4.0`
- Allows automatic security patches without breaking changes
- Documented in PEP 440
### Docker Build
```bash
@@ -88,130 +160,39 @@ docker run -p 8083:8083 core-code:latest
### Environment Variables
See `.env.example` for all available configuration options.
| Variable | Description | Default |
|----------|-------------|---------|
| `PORTAINER_URL` | Portainer API URL | `http://localhost:9000` |
| `PORTAINER_API_KEY` | Portainer API key | - |
| `NPM_URL` | Nginx Proxy Manager URL | `http://localhost:81` |
| `NPM_EMAIL` | NPM admin email | - |
| `NPM_PASSWORD` | NPM admin password | - |
| `HOMEASSISTANT_URL` | Home Assistant URL | `http://localhost:8123` |
| `HOMEASSISTANT_TOKEN` | HA long-lived access token | - |
| `OLLAMA_URL` | Ollama API URL | `http://localhost:11434` |
| `OIDC_ENABLED` | Enable OIDC auth | `false` |
| `OIDC_ISSUER` | OIDC issuer URL | - |
| `OIDC_AUDIENCE` | OIDC audience | - |
## API Documentation
Once deployed, access documentation at:
- **Swagger UI**: http://192.168.86.149:8083/docs
- **ReDoc**: http://192.168.86.149:8083/redoc
- **OpenAPI Spec**: http://192.168.86.149:8083/openapi.json
## Integration with Open WebUI
### Method 1: Functions (OpenAPI Import)
1. In Open WebUI, navigate to Functions
2. Import from OpenAPI spec: `http://192.168.86.149:8083/openapi.json`
3. Use functions directly in chat
### Method 2: Pipelines
1. Create a pipeline that calls Core Code API endpoints
2. Use as data source for LLM workflows
### Method 3: Direct API Calls
```python
import httpx
async with httpx.AsyncClient() as client:
response = await client.post(
"http://192.168.86.149:8083/web-scraper/scrape",
json={
"url": "https://example.com",
"extract_main_content": True
}
)
data = response.json()
```
## API Endpoints
### Web Scraper
**POST /web-scraper/scrape**
Scrape and extract content from a website.
Request:
```json
{
"url": "https://example.com/article",
"extract_main_content": true,
"include_links": false,
"max_length": 10000
}
```
Response:
```json
{
"url": "https://example.com/article",
"title": "Article Title",
"content": "Extracted article content...",
"extracted_at": "2025-11-12T19:30:00Z",
"content_length": 5432,
"links": null
}
```
## Logging
Logs are written to:
- **Console**: stdout (captured by Docker)
- **File**: `/app/logs/app.log` (persisted via volume mount)
Log format:
```
2025-11-12 19:30:00 | INFO | src.web_scraper.service:scrape_url:45 | Starting scrape for URL: https://example.com
```
- **Swagger UI**: http://localhost:8083/docs
- **ReDoc**: http://localhost:8083/redoc
- **OpenAPI Spec**: http://localhost:8083/openapi.json
## Health Checks
- **Endpoint**: `GET /health`
- **Docker**: Automatic health checks configured
- **Response**: `{"status": "healthy"}`
- **Basic**: `GET /health` - Returns status and Ollama connection
- **Full**: `GET /health/full` - Returns all component statuses (503 if unhealthy)
- **Diagnostics**: `GET /health/diagnostics` - Detailed service information
## Security
- Runs as non-root user (uid 1000)
- No authentication required (internal network only)
- OIDC authentication support via Authentik
- CORS configured for same-network access
- Rate limiting: Not implemented (internal use only)
## Future Modules
The architecture supports adding new modules:
- Data transformation functions
- API integrations
- File processing
- Database queries
Each module follows the same structure:
```
src/
└── module_name/
├── config.py
├── schemas.py
├── service.py
├── router.py
└── exceptions.py
```
## Troubleshooting
### Container won't start
```bash
docker logs core-code
```
### API not responding
```bash
curl http://192.168.86.149:8083/health
```
### Check OpenAPI spec
```bash
curl http://192.168.86.149:8083/openapi.json | jq
```
- Admin endpoints require authentication when OIDC enabled
## License