refactor: extract core-api to standalone repository
Core-API service moved to git.schweitz.net/jpmschweitzer/core-api - Source code removed from this repository - Stack file updated to use container image from Gitea registry - Documentation updated with external service reference - Removed obsolete docs/services/core-api.md (docs now in core-api repo) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 4.5
@@ -36,6 +36,37 @@ To make changes:
|
||||
|
||||
---
|
||||
|
||||
## Core-API
|
||||
|
||||
**Repository:** https://git.schweitz.net/jpmschweitzer/core-api
|
||||
**Container Image:** `git.schweitz.net/jpmschweitzer/core-api:latest`
|
||||
**Stack File:** `stacks/core-api.yml` (remains in portainer-core)
|
||||
|
||||
### Overview
|
||||
Core-API provides infrastructure orchestration and OpenAI-compatible API endpoints:
|
||||
- Infrastructure management (Portainer, NPM, Uptime Kuma integration)
|
||||
- Web scraping tools
|
||||
- AI metrics proxy
|
||||
- Service health monitoring
|
||||
|
||||
### Deployment
|
||||
- Container image built via Gitea Actions on release
|
||||
- Stack file in portainer-core defines volumes, environment, and network
|
||||
- Watchtower monitors for image updates
|
||||
|
||||
### Development
|
||||
To make changes:
|
||||
1. Clone: `git clone gitea:jpmschweitzer/core-api.git`
|
||||
2. Make changes
|
||||
3. Create a release in Gitea to trigger build
|
||||
4. Watchtower will auto-update the running container
|
||||
|
||||
### API
|
||||
- Health: `http://core-api:8083/health/full`
|
||||
- Docs: `http://core-api:8083/docs`
|
||||
|
||||
---
|
||||
|
||||
## Tatlock
|
||||
|
||||
**Repository:** https://git.schweitz.net/jpmschweitzer/tatlock
|
||||
@@ -49,5 +80,4 @@ Tatlock is a separate project maintained in its own repository.
|
||||
## Future Migrations
|
||||
|
||||
The following services are planned for extraction:
|
||||
- **core-api** - Infrastructure management API
|
||||
- **library-desk** - Knowledge management and consolidation service
|
||||
|
||||
@@ -1,290 +0,0 @@
|
||||
# Core API Service
|
||||
|
||||
OpenAPI-compatible functions for Open WebUI and infrastructure management, providing web scraping, AI orchestration, and Portainer automation capabilities.
|
||||
|
||||
## Features
|
||||
|
||||
### Web Scraper
|
||||
- Intelligent content extraction using Trafilatura
|
||||
- BeautifulSoup fallback for complex pages
|
||||
- Configurable content length limits
|
||||
- Optional link extraction
|
||||
- Perfect for feeding webpage content to LLMs
|
||||
|
||||
### Infrastructure Management
|
||||
- Portainer stack control (start/stop services)
|
||||
- Service status monitoring
|
||||
- Container health checks
|
||||
- Service group management
|
||||
- Read/write REST API
|
||||
|
||||
### AI Orchestration
|
||||
- OpenAI-compatible API endpoints
|
||||
- Model routing and management
|
||||
- Streaming responses
|
||||
- Function calling support
|
||||
- Multi-phase enhancement roadmap
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
src/
|
||||
├── config.py # Global application settings
|
||||
├── logging_config.py # Logging configuration
|
||||
├── base_schema.py # Base Pydantic models
|
||||
├── main.py # FastAPI application entry point
|
||||
└── modules/
|
||||
├── web_scraper/ # Web scraper module
|
||||
│ ├── config.py
|
||||
│ ├── schemas.py
|
||||
│ ├── service.py
|
||||
│ ├── router.py
|
||||
│ └── exceptions.py
|
||||
└── infrastructure/ # Infrastructure management
|
||||
├── config.py
|
||||
├── schemas.py
|
||||
├── service.py
|
||||
└── router.py
|
||||
```
|
||||
|
||||
## Deployment
|
||||
|
||||
### Portainer Stack
|
||||
|
||||
1. Navigate to Portainer UI
|
||||
2. Go to **Stacks** → **Add Stack**
|
||||
3. Name: `core-api`
|
||||
4. Upload `stacks/core-api.yml` or paste contents
|
||||
5. Deploy
|
||||
|
||||
### Environment Variables
|
||||
|
||||
See `.env.example` in the service directory for all available configuration options.
|
||||
|
||||
Key variables:
|
||||
- `PORTAINER_URL` - Portainer API endpoint
|
||||
- `PORTAINER_API_KEY` - API key for Portainer authentication
|
||||
- `LOG_LEVEL` - Logging verbosity (DEBUG, INFO, WARNING, ERROR)
|
||||
- `CORS_ORIGINS` - Allowed CORS origins
|
||||
|
||||
## API Documentation
|
||||
|
||||
Once deployed, access documentation at:
|
||||
- **Swagger UI**: http://localhost:8083/docs
|
||||
- **ReDoc**: http://localhost:8083/redoc
|
||||
- **OpenAPI Spec**: http://localhost:8083/openapi.json
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Web Scraper
|
||||
|
||||
**POST /web-scraper/scrape**
|
||||
|
||||
Scrape and extract content from a website.
|
||||
|
||||
Request:
|
||||
```json
|
||||
{
|
||||
"url": "https://example.com/article",
|
||||
"extract_main_content": true,
|
||||
"include_links": false,
|
||||
"max_length": 10000
|
||||
}
|
||||
```
|
||||
|
||||
Response:
|
||||
```json
|
||||
{
|
||||
"url": "https://example.com/article",
|
||||
"title": "Article Title",
|
||||
"content": "Extracted article content...",
|
||||
"extracted_at": "2025-11-12T19:30:00Z",
|
||||
"content_length": 5432,
|
||||
"links": null
|
||||
}
|
||||
```
|
||||
|
||||
### Infrastructure Management
|
||||
|
||||
**GET /infrastructure/services**
|
||||
|
||||
List all Portainer stacks with status.
|
||||
|
||||
Response:
|
||||
```json
|
||||
[
|
||||
{
|
||||
"name": "jellyfin",
|
||||
"status": "running",
|
||||
"containers": 1,
|
||||
"running_containers": 1
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
**POST /infrastructure/services/{name}/start**
|
||||
|
||||
Start a service stack.
|
||||
|
||||
**POST /infrastructure/services/{name}/stop**
|
||||
|
||||
Stop a service stack.
|
||||
|
||||
**GET /infrastructure/service-groups**
|
||||
|
||||
Get service groupings and always-on services.
|
||||
|
||||
Response:
|
||||
```json
|
||||
{
|
||||
"service_groups": {
|
||||
"jellyfin": ["jellyfin"],
|
||||
"nextcloud": ["nextcloud"],
|
||||
"ai-stack": ["open-webui", "ollama", "qdrant"]
|
||||
},
|
||||
"always_on": ["portainer", "nginx-proxy-manager", "core-api"]
|
||||
}
|
||||
```
|
||||
|
||||
### Health Check
|
||||
|
||||
**GET /health**
|
||||
|
||||
Service health check endpoint.
|
||||
|
||||
Response:
|
||||
```json
|
||||
{
|
||||
"status": "healthy"
|
||||
}
|
||||
```
|
||||
|
||||
## Integration with Open WebUI
|
||||
|
||||
### Method 1: Functions (OpenAPI Import)
|
||||
1. In Open WebUI, navigate to Functions
|
||||
2. Import from OpenAPI spec: `http://localhost:8083/openapi.json`
|
||||
3. Use functions directly in chat
|
||||
|
||||
### Method 2: Pipelines
|
||||
1. Create a pipeline that calls Core API endpoints
|
||||
2. Use as data source for LLM workflows
|
||||
|
||||
### Method 3: Direct API Calls
|
||||
```python
|
||||
import httpx
|
||||
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.post(
|
||||
"http://localhost:8083/web-scraper/scrape",
|
||||
json={
|
||||
"url": "https://example.com",
|
||||
"extract_main_content": True
|
||||
}
|
||||
)
|
||||
data = response.json()
|
||||
```
|
||||
|
||||
## Development
|
||||
|
||||
### Requirements
|
||||
- Python 3.12+
|
||||
- Docker (for containerized deployment)
|
||||
|
||||
### Local Development
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Run locally
|
||||
uvicorn src.main:app --reload --host 0.0.0.0 --port 8083
|
||||
```
|
||||
|
||||
### Docker Build
|
||||
|
||||
```bash
|
||||
# Build image
|
||||
docker build -t core-api:latest .
|
||||
|
||||
# Run container
|
||||
docker run -p 8083:8083 core-api:latest
|
||||
```
|
||||
|
||||
## Logging
|
||||
|
||||
Logs are written to:
|
||||
- **Console**: stdout (captured by Docker)
|
||||
- **File**: `/app/logs/app.log` (persisted via volume mount)
|
||||
|
||||
Log format:
|
||||
```
|
||||
2025-11-12 19:30:00 | INFO | src.web_scraper.service:scrape_url:45 | Starting scrape for URL: https://example.com
|
||||
```
|
||||
|
||||
## Security
|
||||
|
||||
- Runs as non-root user (uid 1000)
|
||||
- No authentication required (internal network only)
|
||||
- CORS configured for same-network access
|
||||
- Rate limiting: Not implemented (internal use only)
|
||||
- **Always-on service** - Cannot be stopped via infrastructure management
|
||||
|
||||
## Future Enhancements
|
||||
|
||||
See [AI Orchestrator Plan](../../plans/active/ai-orchestrator-plan.md) for upcoming features:
|
||||
|
||||
### Phase 2: Memory Systems (In Progress)
|
||||
- Ephemeral, short-term, and long-term memory
|
||||
- Vector embeddings with Qdrant
|
||||
- Memory search and retrieval
|
||||
|
||||
### Phase 3: Multi-Model Management
|
||||
- Dynamic model routing
|
||||
- Cost optimization
|
||||
- Fallback strategies
|
||||
|
||||
### Phase 4: Reasoning & Chain-of-Thought
|
||||
- Structured reasoning
|
||||
- Multi-step problem solving
|
||||
- Verification and validation
|
||||
|
||||
### Phase 5: Agentic Workflows
|
||||
- Tool integration
|
||||
- Multi-agent orchestration
|
||||
- Autonomous task execution
|
||||
|
||||
### Phase 6: Production Optimization
|
||||
- Caching strategies
|
||||
- Performance tuning
|
||||
- Monitoring and metrics
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Container won't start
|
||||
```bash
|
||||
docker logs core-api
|
||||
```
|
||||
|
||||
### API not responding
|
||||
```bash
|
||||
curl http://localhost:8083/health
|
||||
```
|
||||
|
||||
### Check OpenAPI spec
|
||||
```bash
|
||||
curl http://localhost:8083/openapi.json | jq
|
||||
```
|
||||
|
||||
### Portainer connection issues
|
||||
1. Verify `PORTAINER_URL` is correct
|
||||
2. Check `PORTAINER_API_KEY` is valid
|
||||
3. Ensure Portainer is accessible from core-api container
|
||||
4. Check Docker network connectivity
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [Stacks Reference](../reference/stacks.md) - All Docker Compose stacks
|
||||
- [Automation Reference](../reference/AUTOMATION.md) - Portainer REST API details
|
||||
- [AI Orchestrator Plan](../../plans/active/ai-orchestrator-plan.md) - Feature roadmap
|
||||
- [Organizr Widget](organizr-widgets.md) - Service control UI integration
|
||||
@@ -151,7 +151,7 @@ These infrastructure services are protected:
|
||||
|
||||
If accessing widget from a different domain than core-api:
|
||||
|
||||
**Option 1**: Update core-api CORS settings in `services/core-api/src/config.py`:
|
||||
**Option 1**: Update core-api CORS settings in `src/config.py` (in the core-api repository):
|
||||
```python
|
||||
cors_origins: list[str] = ["http://your-organizr-domain.com"]
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user