jpmschweitzer/odysseus
1
0
Fork 0
mirror of https://github.com/pewdiepie-archdaemon/odysseus.git synced 2026-11-01 18:52:21 +01:00
Code Issues Packages Projects Releases Wiki Activity

fix(editor): stabilize image generation, layer editing and controls (#6992)

Browse Source 
* fix(editor): stabilize image workflows and editing controls

* docs(editor): add desktop and mobile visual validation evidence

* fix(editor): synchronize quick edit model events and register imagePreferences in test coordinator

* fix(settings): enforce authoritative global image-gen policy and non-admin preference persistence

- Enforce authoritative administrator global image_gen_enabled disable in get_user_setting, do_generate_image, and image_editing_provider_available, preventing non-admin user preference overrides while preserving user-disabled opt-outs and can_generate_images privilege checks across direct, gallery, and editor paths.
- Filter non-admin preference persistence in saveImagePreferences to user preference keys only (image_model, image_edit_model, image_quality), preventing global policy toggles from leaking into user preferences.
- Return accurate 200 OK Response from saveImagePreferences after fallback persistence succeeds so settings UI displays 'Saved' instead of false 403 error, while retaining visibility of real errors.
- Load per-user preferences via readImagePreferences in initImageSettings so non-admin selections persist and display correctly after page reload.
- Clear per-user model preferences on successful admin save (POST /api/auth/settings) via PUT /api/prefs/{key} with value: null to prevent stale per-user preferences from shadowing updated global settings.
- Support clearing keys in PUT /api/prefs/{key} when value is null.
- Add comprehensive Python regression tests for global-disabled/user-enabled, global-enabled/user-disabled, and authorized generation, plus Node and browser regression tests for non-admin persistence, toggle isolation, reload, and admin clearing.

* fix(settings): invalidate shared appConfig cache near auth settings write

* fix(test): isolate test_tool_policy unit suite from external model endpoint state

* fix(test): isolate web_fetch failure regression from contextual follow-up heuristics

* fix(test): isolate test_tool_policy from cross-test approval gate state

Ensure _patch_loop_basics disables TOOL_APPROVAL_GATE_ENABLED so routing tests in test_tool_policy run in the intended opt-out posture even when prior tests populate DATA_DIR with skills or untrusted context. Remove unnecessary forced_tools and contextual web mock workarounds.

* fix(settings): isolate non-admin image toggle and surface preference cleanup errors

- Restrict image generation availability toggle to admin users in imageModels.js, disabling the switch for non-admins and reverting unauthorized changes with an explicit message.
- Separate non-admin image model preference saves from global toggle updates.
- Track preference cleanup results on admin save in imagePreferences.js, propagating cleanup failures instead of ignoring them.
- Require valid preference keys on 403 fallback so toggle-only attempts reject cleanly.
- Add /api/prefs route mock to test_image_model_settings_sync_both_directions_and_survive_reopening.
- Add regression coverage for admin cleanup failures and non-admin toggle rejection.

* fix(settings): reconcile admin toggle on partial cleanup failure and report errors

- In imagePreferences.js, distinguish pre-persistence failures from post-save cleanup failures using err.globalSaved and err.phase.

- In imagePreferences.js, notify listeners of global policy updates on partial success before propagating cleanup failure.

- In imageModels.js, retain the persisted target state on admin toggle when global persistence succeeds but cleanup fails.

- In imageModels.js, reconcile admin toggle with authoritative server state via readImagePreferences() after ambiguous or partial failures.

- In imageModels.js, report partial success accurately without suppressing preference cleanup error messages.

- Add regression coverage in test_settings_save_response_ok_js.py for cleanup HTTP 500, cleanup network rejection, successful cleanup, and failed global POST.

---------

Co-authored-by: pewdiepie-archdaemon <pewdiepie-archdaemon@users.noreply.github.com>
Co-authored-by: Alexandre Teixeira <general@alexandreteixeira.dev>
This commit is contained in:
PewDiePie
2026-11-01 02:56:46 +00:00
committed by GitHub
co-authored by pewdiepie-archdaemon Alexandre Teixeira
parent 5353241743
commit 39303f19cd
56 changed files with 3384 additions and 278 deletions
Download Patch File Download Diff File Expand all files Collapse all files
routes/editor_agent_routes.py
+82 -2
View File
@@ -846,6 +846,64 @@ def _plan_psd_steps(prompt: str) -> Optional[List[Dict[str, Any]]]:
return None
def _is_background_removal_intent(prompt: str) -> bool:
"""Linear-time, bounded intent parser for background removal commands.
Replaces the previous backtracking regular expression flagged by CodeQL
(#1212) while preserving the exact command grammar.
"""
if not prompt or not isinstance(prompt, str):
return False
i = len(prompt)
while i > 0 and prompt[i - 1] in ".! \t\r\n":
i -= 1
cleaned = prompt[:i].strip()
if not cleaned:
return False
tokens = cleaned.split()
if not tokens:
return False
idx = 0
if tokens[idx].lower() == "please":
idx += 1
if idx >= len(tokens):
return False
if tokens[idx].lower() not in {"remove", "erase", "delete", "clear", "strip"}:
return False
idx += 1
if idx >= len(tokens):
return False
if tokens[idx].lower() == "the":
idx += 1
if idx >= len(tokens):
return False
if tokens[idx].lower() not in {"bg", "background", "backdrop"}:
return False
idx += 1
if idx == len(tokens):
return True
if tokens[idx].lower() in {"from", "of"}:
idx += 1
if idx < len(tokens) and tokens[idx].lower() == "the":
idx += 1
if (
idx + 1 < len(tokens)
and tokens[idx].lower() in {"active", "current", "selected"}
and tokens[idx + 1].lower() == "layer"
and idx + 2 == len(tokens)
):
return True
return False
def _plan_special(
prompt: str, state: Any, provider_available: bool,
) -> Optional[Tuple[List[Dict[str, Any]], str, str]]:
@@ -855,6 +913,8 @@ def _plan_special(
left to the model. Returns (steps, reply, note) or None. These never need an
image provider except generative border fill, used only when one exists.
"""
if _is_background_removal_intent(prompt):
return [{"op": "ai.removeBackground", "params": {}}], "Removing the active layer's background.", ""
# Targeted recolour is fully deterministic (object mask + masked hue shift)
# and must never be left to the model, which sometimes substitutes a
# generative placeImage. Highest priority so it can never vary.
@@ -1647,6 +1707,13 @@ def setup_editor_agent_routes() -> APIRouter:
screenshot = body.get("screenshot")
# Optional user-picked image model (from the editor model selector).
model_choice = str(body.get("model") or body.get("image_model") or "").strip()
generation_model_choice = str(body.get("generation_model") or "").strip()
use_image_settings = body.get("_use_image_settings") is True
if use_image_settings:
from src.editor_image_settings import editor_image_models
image_models = editor_image_models(owner=owner)
model_choice = image_models["edit"]
generation_model_choice = image_models["generation"]
# Optional user-picked PLANNER model (the editor agent/planner model).
agent_model = str(body.get("agent_model") or "").strip()
tools_offered = [tool for tool in _scoped_editor_tools() if tool not in disabled]
@@ -1797,9 +1864,22 @@ def setup_editor_agent_routes() -> APIRouter:
# Planner output is untrusted: it cannot redirect this turn to
# another editor session or override an explicit model choice.
tool_payload["session"] = session
if use_image_settings:
tool_payload["_use_image_settings"] = True
if op == "generate_image" and state.get("width") and state.get("height"):
try:
width, height = int(state["width"]), int(state["height"])
if width > 0 and height > 0:
tool_payload["size"] = f"{width}x{height}"
except (TypeError, ValueError):
pass
# Route to the user-picked model when one is selected.
if model_choice:
tool_payload["model"] = model_choice
selected_model = generation_model_choice if op == "generate_image" else model_choice
if selected_model:
tool_payload["model"] = selected_model
elif op == "generate_image" or use_image_settings:
# Let server settings resolve Auto, not a planner-invented model.
tool_payload.pop("model", None)
# Hand the AI step the canvas image the client already captured
# for this request. Without it the tool falls back to a separate
# bridge screenshot, which is not reliably answered right after a
routes/gallery/gallery_routes.py
+65 -12
View File
@@ -38,6 +38,24 @@ _SAM_STATE: Dict[str, Any] = {}
_GROUNDING_STATE: Dict[str, Any] = {}
def _compose_cloud_image_edit(source, generated, mask, *, instruction=False):
from PIL import Image, ImageChops, ImageOps
full_frame = instruction or mask.getextrema() == (255, 255)
if full_frame:
return generated, True
# Partial edits must share the source coordinate system, but resizing
# width/height independently distorts the subject when the provider changes ratio.
fitted = ImageOps.contain(generated, source.size, Image.Resampling.LANCZOS)
offset = ((source.width - fitted.width) // 2, (source.height - fitted.height) // 2)
aligned = Image.new("RGBA", source.size)
aligned.paste(fitted, offset)
coverage = Image.new("L", source.size)
coverage.paste(255, (offset[0], offset[1], offset[0] + fitted.width, offset[1] + fitted.height))
effective_mask = ImageChops.multiply(mask, coverage)
return Image.composite(aligned, source, effective_mask), False
def _rss_mb() -> float:
"""Current process RSS in MB (from /proc; -1.0 when unavailable)."""
try:
@@ -586,6 +604,7 @@ def _is_openai_api_base(url: str) -> bool:
_GALLERY_ENDPOINT_PATHS = frozenset({
"/chat/completions",
"/images",
"/images/edits",
"/images/generations",
@@ -1767,6 +1786,17 @@ def setup_gallery_routes() -> APIRouter:
if not ok:
raise HTTPException(400, f"Rejected endpoint URL: {reason}")
chosen_model = (body.pop("_model", "") or "").strip()
if body.pop("_use_image_settings", False) is True:
from src.editor_image_settings import editor_image_models
from src.ai_interaction import _resolve_model
chosen_model = editor_image_models(owner=user)["edit"]
if not chosen_model:
raise HTTPException(400, "Choose an image edit model in AI settings")
try:
edit_url, chosen_model, _ = await asyncio.to_thread(_resolve_model, chosen_model, owner=user)
except ValueError as exc:
raise HTTPException(400, str(exc)) from exc
requested_base = edit_url.removesuffix("/chat/completions").rstrip("/")
# Optional client-supplied id so GET /api/image/edit-progress can poll the
# provider's step progress for this op. Ignored by providers that do not
# implement the progress contract.
@@ -1884,14 +1914,30 @@ def setup_gallery_routes() -> APIRouter:
mask_png.save(reference_mask, format="PNG")
references = [src_buf.getvalue(), reference_mask.getvalue()]
payload = {
"model": oa_model, "size": size, "n": 1,
"model": oa_model, "n": 1,
"output_format": "png",
"prompt": "Edit the first image. The second image is a mask: white marks the region to edit, black marks the region to preserve. Keep composition and framing unchanged. Requested edit: " + str(body.get("prompt", "")),
"input_references": [{"type": "image_url", "image_url": {
"url": "data:image/png;base64," + base64.b64encode(value).decode(),
}} for value in references],
}
r = await client.post(_join_checked_gallery_endpoint(base, "/images"), headers=headers, json=payload)
from src.image_model_ids import image_aspect_ratio
payload["aspect_ratio"] = image_aspect_ratio(source_png.width, source_png.height)
from src.ai_interaction import _uses_openrouter_chat_images
if _uses_openrouter_chat_images(oa_model):
payload = {
"model": oa_model,
"messages": [{"role": "user", "content": [
{"type": "text", "text": payload["prompt"]},
*[{"type": "image_url", "image_url": ref["image_url"]}
for ref in payload["input_references"]],
]}],
"modalities": ["image", "text"], "stream": False,
"image_config": {"aspect_ratio": payload["aspect_ratio"]},
}
r = await client.post(_join_checked_gallery_endpoint(base, "/chat/completions"), headers=headers, json=payload)
else:
r = await client.post(_join_checked_gallery_endpoint(base, "/images"), headers=headers, json=payload)
else:
r = await client.post(_join_checked_gallery_endpoint(base, "/images/edits"), headers=headers, data=data, files=files)
if r.status_code != 200:
@@ -1907,6 +1953,12 @@ def setup_gallery_routes() -> APIRouter:
pass
raise HTTPException(r.status_code, f"{_detail} (provider HTTP {r.status_code})")
result = r.json()
from src.ai_interaction import _chat_image_results, _image_response_error
provider_error = _image_response_error(result, oa_model)
if provider_error:
raise HTTPException(400 if "deprecated" in provider_error.lower() else 502, provider_error)
if is_openrouter and _uses_openrouter_chat_images(oa_model):
result = {"data": _chat_image_results(result)}
raw_b64 = None
if result.get("data"):
item = result["data"][0]
@@ -1926,17 +1978,13 @@ def setup_gallery_routes() -> APIRouter:
# the masked region actually changes.
try:
generated = Image.open(io.BytesIO(base64.b64decode(raw_b64))).convert("RGBA")
# Match the generated image to the source dims.
if generated.size != source_png.size:
generated = generated.resize(source_png.size, Image.LANCZOS)
# mask_png: white = regenerate (use generated),
# black = keep (use source).
# Composite: result = source * (1 - mask_norm) + generated * mask_norm
# Image.composite does exactly that with `mask`.
blended = Image.composite(generated, source_png, mask_png)
blended, full_frame = _compose_cloud_image_edit(
source_png, generated, mask_png,
instruction=body.get("mode") == "instruction",
)
out_buf = io.BytesIO()
blended.save(out_buf, format="PNG")
return {"image": base64.b64encode(out_buf.getvalue()).decode(), "model": oa_model}
return {"image": base64.b64encode(out_buf.getvalue()).decode(), "model": oa_model, "full_frame": full_frame}
except Exception as comp_err:
# Never return a full-image edit when masking fails.
logger.warning(f"Inpaint compose failed: {comp_err}")
@@ -2593,7 +2641,9 @@ def setup_gallery_routes() -> APIRouter:
height = int((body or {}).get("height") or 0)
except (TypeError, ValueError):
width = height = 0
if height > width:
if width > 0 and height > 0:
size = f"{width}x{height}"
elif height > width:
size = "1024x1536"
elif width > height:
size = "1536x1024"
@@ -2606,6 +2656,9 @@ def setup_gallery_routes() -> APIRouter:
# The editor's model picker may choose a specific (cloud) image model;
# forward it so settings defaults only apply when none is selected.
model_choice = str((body or {}).get("model") or "").strip()
if body.get("_use_image_settings") is True:
from src.editor_image_settings import editor_image_models
model_choice = editor_image_models(owner=user)["generation"]
payload = {"prompt": prompt, "size": size}
if model_choice:
payload["model"] = model_choice
routes/prefs_routes.py
+6 -2
View File
@@ -118,8 +118,12 @@ def setup_prefs_routes():
async def set_pref(request: Request, key: str, body: dict):
user = get_current_user(request)
prefs = _load_for_user(user)
prefs[key] = body.get("value")
val = body.get("value")
if val is None:
prefs.pop(key, None)
else:
prefs[key] = val
_save_for_user(user, prefs)
return {"key": key, "value": prefs[key]}
return {"key": key, "value": prefs.get(key)}
return router