docs(specs): routine specifications update

This commit is contained in:
RaresKeY
2026-10-15 16:52:16 +00:00
parent c74f475679
commit bead6a6797
29 changed files with 767 additions and 511 deletions
+33 -14
View File
@@ -1,6 +1,6 @@
# Gallery, Editor, And Media
Last updated: dev@e71f8ce | 2026-08-25
Last updated: dev@c74f475 | 2026-10-15
## Scope
@@ -28,7 +28,7 @@ This spec covers media surfaces in:
- gallery upload, library, album, tag, favorite, ZIP, delete, and saved-project views;
- chat-generated image rendering/edit/delete bubbles;
- agent `generate_image` and stale `edit_image` tool paths;
- agent `generate_image` and owner-scoped `edit_image` tool paths;
- MCP image-generation rows/files;
- image editor AI tools and model endpoint pickers;
- document PDF signing with stored signatures;
@@ -51,11 +51,13 @@ Generated media provenance:
- upload `file_hash` is a separate metadata field;
- generated files live under `data/generated_images`;
- chat image generation writes files and inserts `GalleryImage` rows through `src.ai_interaction`;
- MCP image generation can create ownerless rows/files;
- MCP image generation stamps the trusted dispatched owner when present, trims empty owner values to null, and can still create ownerless rows/files without that context;
- generated-but-not-yet-imported images can have no gallery row;
- once a gallery row exists, owner checks decide visibility where the route enforces them.
`app.py` owns direct `/api/generated-image/{filename}` serving through `src.generated_images.resolve_generated_image_path()`. It validates hex-like image/video filenames, rejects path escape and missing files, serves rowless generated files, checks row owner when a row exists, allows null-owner compatibility rows, and uses immutable/nosniff cache headers. Gallery replace/rotate/save/delete/ZIP paths also resolve filenames through a shared generated-image path helper so database filenames cannot escape `data/generated_images`. Replace/rotate/save-over-original flows can mutate bytes under the same filename, so frontend cache busting matters.
`app.py` owns direct `/api/generated-image/{filename}` serving through `src.generated_images.resolve_generated_image_path()`. It validates filename syntax before ownership and disk probing. With auth enabled, `effective_user()` resolves cookie/API-token owners. A live registry entry is checked first: a matching owner authorizes, while a mismatch returns 404 without database fallback. Only absent registration reaches the gallery lookup: a nonempty row owner must match; a row without an owner instead requires a matching linked-session owner. If that lookup finds no gallery row, it requires a chat metadata reference in an owned session. Unattributed/mismatched files return 404 and database verification failures return 503. Disabled auth skips ownership verification. Confined path/existence checks follow authorization, and responses use `private, no-cache, must-revalidate` plus nosniff headers. Gallery replace/rotate/save/delete/ZIP paths also resolve filenames through a shared generated-image path helper so database filenames cannot escape `data/generated_images`. Replace/rotate/save-over-original flows can mutate bytes under the same filename, so frontend cache busting matters.
`src/generated_images.py` owns a lock-protected, process-local one-hour recent-owner registry. Image generation/edit writers in `src/ai_interaction.py` register trusted owners after writing bytes; the image MCP server stamps gallery ownership and registers after commit when an owner exists. Registry entries are ephemeral, not durable gallery ownership, and matching registry access can survive a database outage. Tool dispatch strips caller-reserved attribution keys before injecting verified owner context, including qualified image MCP and direct fallback paths.
## Image Tools And Providers
@@ -92,7 +94,7 @@ Optional dependency behavior:
## Editor Drafts
`routes.editor_draft_routes` owns server-backed image editor project payloads. `EditorDraft` rows store title, payload JSON, thumbnail, source image, timestamps, and owner.
`routes.editor_draft_routes` owns server-backed image editor project payloads. `EditorDraft` rows store name, dimensions, payload JSON, thumbnail, source image, timestamps, and owner.
Frontend editor behavior is split across `static/js/editor/*` and `static/js/galleryEditor.js`: canvas state, layer panel, masks, history, snapping, stroke pipeline, inpaint/rembg/harmonize tools, AI tool runner, model pickers, an AI edit command box that routes natural-language edit requests into existing inpaint/remove/upscale/background/style actions where possible, import wiring, topbar controls, auto-save, resume by draft ID or source image, draft-only open, and cleanup after close. `static/js/panels.js` loads this module graph on first editor use, shares concurrent imports, retries failed loads, and `static/sw.js` keeps the lazy graph in a separate offline panel precache.
@@ -126,11 +128,11 @@ Known boundaries:
- image-generation routes require `can_generate_images`;
- image proxy/editor endpoints currently resolve client-selected, DB-selected, or fallback image model endpoints without full owner-scoped endpoint-key policy or uniform outbound revalidation;
- generated-file serving allows rowless files and null-owner compatibility rows;
- rowless generated files require an owned chat reference or recent registration; null-owner gallery rows require a matching owned session;
- uploads are byte-limited and extension-gated, with content sniffing available through `UploadHandler.detect_content_type()` when `python-magic`/`libmagic` is installed;
- several base64 JSON editor routes accept large decoded image payloads and need route-level size discipline;
- gallery DB filenames should be joined through shared generated-media path helpers before filesystem operations;
- editor draft source image IDs, payloads, and thumbnails are owner-scoped by draft owner but do not fully validate source-gallery ownership or payload size;
- editor drafts owner-scope rows for identified users and cap serialized payload bytes, but source-gallery ownership and thumbnail size still need separate validation;
- emoji proxy constrains codepoint filenames and degrades invalid, unknown, or unreachable SVGs to transparent no-store placeholders, but remote SVG content still deserves security review;
- visual report Markdown HTML/link/image output needs continued sanitization coverage.
- `scripts/mlx_image_server.py` pins generation/edit routing to the process-start model and ignores request-selected model names, preventing unauthenticated callers from selecting a local model directory/repository whose model-specific script or bridge would execute.
@@ -142,24 +144,41 @@ Known boundaries:
- Missing generated files are skipped in ZIP downloads; if all are missing, the route returns no files found.
- Soft delete commits the gallery row state before removing the disk file, so a failed DB write does not orphan a missing image row.
- AI tagging can fail when disk files are missing.
- Static JS/CSS/HTML assets revalidate because there is no frontend build/versioning.
- Static assets revalidate; most editor modules remain static ES modules, while `frontend/modules.json` and the committed generated-output freshness gate govern the manifest-selected native ES modules published by byte-for-byte copying to committed deployment outputs.
- Gallery/editor frontend state includes stale-while-revalidate and listener cleanup to avoid stale handlers.
- `edit_image` tool schema/implementation currently appears stale against implemented `/api/image/*` and `/api/gallery/*` routes.
- `src/tools/image.py` implements `edit_image` in-process: prompt edits call the model adapter, upscale creates a 2x/4x Lanczos copy, and rembg creates a copy when the optional package is installed. It checks exact gallery owner and confined source path; uploaded attachment URIs support prompt edits after owner-aware resolution.
## Testing Coverage
Existing tests cover EXIF dimensions, owner-filter helper behavior, direct upload limits, image-generation privilege source shape, sharpen auth, gallery null-user denial, endpoint SSRF/source checks, editor draft payload validation, lazy editor loading/offline precache, MLX request-model pinning, font family derivation, visual-report helper behavior, gallery CLI previews, and selected security regressions.
Route-level coverage is thin for full gallery CRUD/album/tag/download/delete flows, generated-image serving, editor draft owner CRUD, signature owner CRUD, emoji proxy/cache behavior, image-tool degraded responses, optional dependency fallbacks, and frontend editor behavior.
Route-level coverage is thin for full gallery CRUD/album/tag/download/delete flows, generated-image serving, editor draft owner CRUD, signature owner CRUD, emoji proxy/cache behavior, image-tool degraded responses, optional dependency fallbacks, and full browser editor integration. Focused codec, layer/mask/group, transform, history, worker, and draft-budget regressions are present; this maintenance pass did not execute them.
## Versioned Editor Documents And Budgets
`static/js/editor/document-codec.js` is the shared project-file/autosave serialization boundary, currently document version 15. It migrates older versions, rejects future versions, validates layer/group/mask/selection and geometry data, and bounds canvas pixels, layer counts, total surfaces, and embedded image data. Masks, adjustments, effects, groups, vector text/shapes, placed layers and view/guide state pass through this codec rather than a second hand-written draft shape, subject to serialization/decode and history limitations below; this is not lossless persistence of every mask/effect property.
Editor responsibilities now include `document-geometry.js` and layer geometry for canvas transforms, `layer-groups.js`/`layer-selection.js` for grouping/multi-selection, `selection-mask.js` for selections, `text-layer.js`/`shape-layer.js` for vector content, and `adjustment-layer.js`/`effects.js` with worker paths for compositing operations. `history-budget.js` bounds raw ImageData history adaptively; rendering cancellation prevents stale asynchronous results from applying after newer edits.
`routes/editor_draft_routes.py` uses `EDITOR_DRAFT_MAX_BYTES` both for early rejection of oversized declared Content-Length on writes and for the authoritative serialized-payload byte check. Missing/malformed Content-Length still reaches the latter check after parsing. This cap does not supply source-gallery ownership or a separate thumbnail budget.
`src/media_ingress.py` separately builds bounded local multimodal attachments for supported image/video/audio/document paths with source hashes, encoding/dimension/frame limits, and artifact metadata. Agent evidence/trace consumers record ingress metadata; it is not the browser upload storage or editor serialization owner.
## Editor Persistence And Operation Limits
Codec limits count canvas/surface allocations and embedded strings rather than universally bounding decoded bytes. Failed canvas encoding or image restoration can omit/drop data; effect-mask normalization and history snapshots do not preserve every mask density/feather property. Async capture can span intervening edits, so it is not an immutable atomic pixel snapshot. History deduplicates shared pixel buffers and trims older snapshots to a budget while retaining the newest even when oversized; it does not serialize the entire history into a draft.
Draft autosave reserves one in-flight capture and uses session/generation checks for assigning returned ids, with PUT404 recreation. Those checks do not cancel an old server request. The current persist chain can clear an earlier error and display Saved after catch/early return; the badge is not a durable-write receipt. Project state uses server draft ids and a compact session descriptor rather than a full browser-sessionStorage payload. Restore can drop failed raster/mask/placed sources and is not a lossless project recovery promise.
Placed transforms, crop/resize/flip, linked versus document-space masks, grouped effects and rasterization retain distinct geometry paths. Duplicate/merge/copy operations do not guarantee independent preservation of every effect-mask/retained metadata field; raw layer copying differs from flattened canonical output. Pixel/position locks are checked on selected mutation paths rather than proving every control respects every lock. Escape cancellation is operation-specific (including crop/transform/filter prompts), not a universal rollback guarantee. Filter/worker generations prevent selected stale results, but shared intermediate canvases and Quick Edit busy-end polling do not establish atomic pixels or successful downstream AI work.
## Current Gaps
- Owner-scoped endpoint-key resolution is needed for image proxy/editor routes.
- Media routes need a clear API-token policy: reject token callers, or implement owner/scope handling.
- Generated-image serving needs live route tests for invalid filenames, rowless files, owned rows, null-owner rows, MIME/cache headers, and cross-owner behavior.
- Mutable generated filenames plus immutable cache headers need cache-busting tests for replace/save-over-original flows.
- Direct handler ownership/cache tests cover generated-image denial and fallback branches; middleware-mounted serving, MIME/account-switch and live browser cache behavior still need broader checks.
- Private revalidation replaces immutable generated-media caching; replace/save-over-original and account-switch behavior still warrant browser cache tests.
- Base64 JSON editor payload size limits need hardening; upload content sniffing should keep native/Docker parity coverage as dependencies change.
- MCP image generation needs an owner attribution decision or explicit admin-only documentation.
- `edit_image` tool route mapping appears stale.
- Image MCP now receives trusted server-dispatch ownership; direct out-of-band MCP callers and cross-process recent-registry behavior remain separate trust/availability limits.
- Emoji SVG proxy/cache and visual-report raw HTML/link sanitization need stronger tests.
- Optional image dependency fallbacks are mostly untested.