docs(specs): routine specifications update

specs/memory-skills.md

@@ -1,6 +1,6 @@
 # Memory And Skills
 
-Last updated: dev@2e2bb52 | 2026-08-16
+Last updated: dev@c74f475 | 2026-10-15
 
 ## Scope
 
@@ -14,7 +14,7 @@ This spec covers persistent memory and user skills in:
 - provider abstractions in `src/memory_provider.py`;
 - LLM extraction/audit in `services/memory/memory_extractor.py`;
 - skill storage, format, import, and extraction in `services/memory/skills.py`, `services/memory/skill_format.py`, `services/memory/skill_importer.py`, and `services/memory/skill_extractor.py`;
-- skill routes in `routes/skills_routes.py`;
+- skill HTTP/compatibility bindings in `routes/skills_routes.py` and evaluation/audit/job/model/prompt/publication services under `services/skills/`;
 - prompt/tool call sites in `src/chat_processor.py`, `src/agent_loop.py`, `src/ai_interaction.py`, `src/tool_implementations.py`, `src/tool_execution.py`, `src/tool_schemas.py`, and `src/tool_security.py`;
 - MCP and Codex surfaces in `mcp_servers/memory_server.py` and `routes/codex_routes.py`;
 - backup/admin/CLI surfaces in `routes/backup_routes.py`, canonical `routes/admin_wipe/admin_wipe_routes.py` plus its shim, `scripts/odysseus-memory`, `scripts/odysseus-skills`, and `scripts/odysseus-backup`;
@@ -43,25 +43,23 @@ Extraction handles reasoning-model response shapes and records explicit dislike/
 
 `services/memory/skills.py` owns disk-backed skill storage under `data/skills/<category>/<name>/SKILL.md`, plus `_usage.json` usage/audit sidecars. Legacy `data/skills.json` is a read-only fallback/import source, not the current write shape.
 
-`services/memory/skill_format.py` owns frontmatter/body parsing and emission. Quoted scalar parsing/emission is symmetric: JSON escapes decode once, UTF-8/non-ASCII stays intact, emitted values escape line separators safely, and invalid JSON-style escapes fall back to literal text instead of compounding backslashes on every save. `services/memory/skill_importer.py` resolves public GitHub/skills URLs, fetches bundle files with strict public-network URL safety, and chooses/imports `SKILL.md`. Import disables automatic redirects, follows at most five hops, validates and resolves each hop, then connects only to the validated IP snapshot through a pinned transport while preserving URL, Host, and TLS identity; GitHub final-host checks and file/size limits still apply. `routes/skills_routes.py` owns CRUD/search/index/import, owner filtering, skill test/audit jobs, and admin-gated built-in tool instruction overrides.
+`services/memory/skill_format.py` owns frontmatter/body parsing and emission. Quoted scalar parsing/emission is symmetric: JSON escapes decode once, UTF-8/non-ASCII stays intact, emitted values escape line separators safely, and invalid JSON-style escapes fall back to literal text instead of compounding backslashes on every save. `services/memory/skill_importer.py` resolves public GitHub/skills URLs, fetches bundle files with strict public-network URL safety, and chooses/imports `SKILL.md`. Import disables automatic redirects, follows at most five hops, validates and resolves each hop, then connects only to the validated IP snapshot through a pinned transport while preserving URL, Host, and TLS identity; GitHub final-host checks and file/size limits still apply. `routes/skills_routes.py` owns HTTP CRUD/search/index/import, authentication/owner verification, exact approval validation, request models and admin-gated built-in instruction overrides. Evaluation, execution, job coordination, audit sequencing and publication policy now live under `services/skills/`; durable identity/storage/lifecycle remains in `services/memory/skills.py`.
 
 Skill extraction is owned by `services/memory/skill_extractor.py`. It can suggest or save skills from conversations, tries valid brace-delimited JSON candidates with `JSONDecoder.raw_decode()`, rejects ambiguous multiple top-level JSON objects instead of guessing, and saved skills remain user-editable data.
 
 Agent skill behavior:
 
 - matched skills are owner-scoped, confidence-gated, usage-counted, and wrapped as untrusted context;
-- `index_for()` exposes published skills plus teacher-escalation drafts gated by platform and toolsets; `active_toolsets=None` means the caller has no explicit toolset knowledge and does not hide `requires_toolsets` skills, while an explicit list applies the gate;
+- `index_for()` exposes built-ins and eligible user skills gated by current approval policy, platform, local-model status, and toolsets; `active_toolsets=None` means the caller has no explicit toolset knowledge and does not hide `requires_toolsets` skills, while an explicit list applies the gate;
 - user prefs such as skills enabled, auto-approve, and max injected skills shape runtime insertion;
-- the level-0 base skill index currently calls `index_for(owner=None)`, so it is not fully owner-scoped.
+- the level-0 skill index passes the acting owner into `index_for()`, applies active-toolset/platform and local-model gates, and returns its user-editable index block separately for untrusted-context wrapping.
 - skill tests use the configured utility model rather than the chat default and wrap user-editable skill text as untrusted context; approval continuation for a test or teacher-generated skill uses the same exact-action gate as the normal agent loop.
 
 ## Tools, MCP, And Backup
 
 Native `manage_memory` and `manage_skills` tool paths pass owner context and use in-process policy gates. `manage_skills` requires an explicit action instead of silently defaulting a malformed call. Manual memory add can choose a category, and route-side manual add validates the source session owner before attaching session-derived memories. `mcp_servers/memory_server.py` lazy-initializes `src` managers and exposes list/add/edit/delete/search. It can scope to `ODYSSEUS_MCP_MEMORY_OWNER` or `ODYSSEUS_MEMORY_OWNER`; if the JSON store contains owner-bearing entries and no owner env is configured, it returns an owner-scope error instead of listing or mutating across owners. Ownerless stores remain ownerless compatibility mode.
 
-The direct `odysseus-memory add` CLI tolerates non-object legacy/corrupt rows
-when checking whether its newly added entry is already present; it ignores
-those rows instead of calling mapping methods on them and crashing the add.
+The direct `odysseus-memory add` CLI tolerates non-object legacy/corrupt rows when checking whether its newly added entry is already present; it ignores those rows instead of calling mapping methods on them and crashing the add.
 
 `/api/export` owner-filters memories and skills. `/api/import` imports skills through current disk-backed `SkillsManager` APIs, stamping missing owners to the importer and preserving supported skill metadata. Full data snapshots through `scripts/odysseus-backup` preserve on-disk skill trees, memory JSON, and caches differently from JSON import/export.
 
@@ -92,7 +90,7 @@ Owner isolation is surface-specific:
 - native memory/skill tools are expected to pass owner context;
 - Codex exposes scoped token memory behavior separately;
 - normal memory/skills routes are cookie/current-user surfaces, not scoped token APIs;
-- MCP memory uses an environment-configured owner for owner-scoped stores, while the agent level-0 skill index currently has ownerless/global behavior;
+- MCP memory uses an environment-configured owner for owner-scoped stores, while the agent level-0 skill index now passes the acting owner;
 - vector dedup during memory extraction suppresses only same-owner or legacy-ownerless vector matches.
 
 Skill test/audit flows intentionally run user-editable `SKILL.md` content as instructions inside controlled jobs. Those jobs rely on route owner checks, admin gates where applicable, and tool execution policy.
@@ -107,10 +105,41 @@ Existing tests cover memory extraction/degraded vectors, owner isolation, unread
 
 Route-level memory CRUD/security, skills route security, MCP memory behavior, vector degraded writes, compatibility facade owner behavior, backup skill import, admin vector cleanup, and frontend endpoint wiring need broader coverage.
 
+## Skill Approval, Retrieval, And Persona Continuity
+
+`services/memory/builtin_skills.py` synchronizes tracked `BUILTIN_SKILLS_DIR` procedures at startup into an immutable ownerless shared catalog, published with confidence 1.0. Owner-aware list/view paths expose these built-ins alongside the owner's skills; built-ins skip user audits. User-editable memories and imported/extracted skills retain their untrusted-context treatment.
+
+`SkillsManager.index_for()` excludes failing user audits and unnecessary skills. With automatic approval enabled, a user skill needs published status, a passing audit, and confidence at the owner's current threshold; a stale published flag alone is insufficient. Manual publication supplies approval when automatic approval is disabled. `remote-only` skills are hidden from local/private-endpoint model indices. Search also gates toolsets and platforms, combining lexical scores with optional semantic scores from `src/skill_index.py`; semantic retrieval reuses an already-ready ToolIndex embedding lane and falls back to lexical search without a foreground cold model load.
+
+Skill audit protocol v2 separates functional pass/fail from no-skill baseline utility, storing summary, baseline verdict, usefulness, saved turns/tool calls, and audit version in `_usage.json`. Legacy inconclusive v1 results are eligible for re-audit. `services/memory/skill_lifecycle.py` supplies a bounded automatic queue: transient/inconclusive checks retry daily, failed/needs-work repairs weekly, and built-in/binned/pass/skipped entries are excluded. Route jobs preserve request authority and exact-action approval continuation.
+
+Memory extraction automatically pins new identity/contact facts only while the count of already-pinned identity/contact entries for the acting owner and compatible ownerless entries is below five; it does not unpin older entries. Extra facts remain retrievable by relevance. Persona continuity extraction stores concise notes in preset/persona data through `PresetManager.update_persona_memory()`, separately from global memory JSON; health personas use a cautious medical brief schema and persona deletion removes its notes.
+
+## Mutation And Audit Lifecycle Limits
+
+Memory edit/delete paths reject empty identities instead of allowing an empty substring match, and mutating operations preserve unreadable-store refusal. Persona extraction updates an existing persona rather than creating a missing persona implicitly; persona notes remain separate from global memory.
+
+`services/skills/publication.py` implements batch finalization behind the route compatibility wrapper `_finalize_audit_batch()` and changes only skills processed by that job, protects approval-required results, and retains binned duplicates on disk while excluding them from retrieval. Name normalization and a 0.38 token-Jaccard threshold group potential duplicates; this heuristic is not semantic equivalence. Functional success and baseline usefulness remain separate audit dimensions, so a successful baseline does not itself make the skill functionally inconclusive.
+
+`SkillAuditUnavailable` stops the batch without treating infrastructure failure as a skill verdict. Other audit errors can record inconclusive results. Manual test and audit agent streams use `contextlib.aclosing`, including early exits for exact-action approval, so the originating agent task closes rather than remaining suspended behind a pending choice. These lifecycle guarantees do not establish successful model execution when the provider or containment backend is unavailable.
+
+## Skill Test Endpoint Recovery
+
+Skill tests prefer the configured utility route; their registered fallback resolves server-owned runtime credentials and does not trust caller-supplied provider headers. The audit-arm path raises `SkillAuditUnavailable` on stream failure and stops the batch without a skill verdict. Manual primary-stream exceptions instead retain a partial transcript for subsequent baseline/evaluation; if grading returns `fail`, confidence can still decrease. An unavailable result is not guaranteed for every manual-test failure.
+
+## Skills Service Boundaries
+
+`services/skills/evaluation.py` owns verdict parsing/grading, necessity/retrieval assessment, efficiency fields and markdown improvement. `agent_runner.py` owns manual/audit agent streams, transcripts, request/exact-approval authority and same-task `aclosing`. `audit_workflow.py` owns per-skill checks, self-edit/worker retry, optional distinct-teacher rewrite, sequential batches and scheduled selection. `job_coordinator.py` owns transient job transitions, candidate selection, safe polling projections and cancellation requests. `model_selection.py` resolves owner-scoped worker/teacher credentials and utility fallback; `prompts.py` constructs tasks and explicitly untrusted skill messages. `publication.py` applies confidence/duplicate/publishing policy through SkillsManager; its preferences adapter still imports `routes.prefs_routes`.
+
+Routes retain patchable aliases/wrappers, including the service-owned manual/audit registry dictionaries and `SkillAuditUnavailable`. A frozen `AuditDependencies` is built from current route bindings at service entry, preserving caller/test seams without rebinding service globals or importing the Skills route module from services. `app.py` still registers the same router and built-in actions still use route compatibility helpers. Registries are process-local jobs, not durable stores.
+
+Manual completion adjusts confidence (pass .95, needs_work .6, fail .4) without publishing. Audits compare skill/no-skill arms on one task; clean/self-edit/teacher passes use .95/.85/.8, and continued non-pass after retries becomes draft at .35. Initial unknown/inconclusive verdicts preserve publication state during per-skill finalization; this does not cover rewrite retries or later batch duplicate finalization. Per-skill auto-publish requires enabled owner policy/threshold and no necessity/utility/duplicate blocker. Batch duplicate finalization instead selects a processed passing keeper at or above the threshold, can publish it under the owner policy without reapplying those per-skill blockers, and bins other eligible processed duplicates; approval-required and unrelated skills remain protected from that batch mutation.
+
+Approval validation stays in HTTP: pending job/id/owner/empty-session must match and the sealed grant is consumed with `allow_continuation=False` before continuation. Resumed skill/transcript context remains untrusted; unattended arms deny encountered approvals. Infrastructure unavailability stops a batch without a skill verdict or finalization. Existing limits remain: rewrite retry results other than pass, including unknown/inconclusive or an approval pause, can still lead to demotion, and manual cancellation may retain private continuation state. Extraction does not add model/containment availability or broad behavioral equivalence guarantees.
+
 ## Current Gaps
 
 - `services/memory/service.py` needs an explicit owner-scope/support decision before it is treated as a public memory API.
-- The agent level-0 skill index should thread owner or be documented as an intentional local/global index.
 - MCP memory still needs a deliberate multi-user UX/config decision, but current behavior avoids cross-owner access when owner-bearing rows exist without an explicit MCP owner env.
 - Memory JSON import does not rebuild vector indexes.
 - Admin wipe vector clearing is currently ineffective.