feat(ui): client disk cache — FileAccess tiers beneath the LRU (D-255, T-1183)
step_canvas_disk_cache.gd: the Tier 2/3 store per D-255(d) and the round-2 three-tier spec. Payload is the WIRE, pre-decode — store_var/ get_var round-trips the PNG-encoded PackedByteArrays natively, and Image.load_png_from_buffer never runs in this file. One composite key shared verbatim with Tier 1 (hash filenames for filesystem safety); per-body index.json with malformed-index recovery (rebuild-or-discard, never crash). Three independent eviction mechanisms, exactly as ruled: rung-0 Global carries a retention floor no sweep touches (now also threaded into Tier 1 per the T-1182 handoff); Tier 2 geometry is byte-valid forever and evicts only by time-since-last-visit (14d starting tunable, on body-open) and LRU byte budget (256 MiB/body, 5-min coarse timer) — two separate sweeps; Tier 3 sim-state TTL is wired and tested but has no production caller yet (no sim-state field exists on EncodedStepCanvas — the D-253 stub inheritance, documented). Hardening per D-255(d), both mandatory: per-body deep-rung cap (512 Block+Chunk entries, enforced synchronously in put(), floor- and budget-independent — the ticket sanctions count-or-quota; count chosen as the direct D-226(d) information-content proxy) and a schema/version tag on every entry (project.yaml version via the existing loading_screen line-scan idiom — D-192 co-ship makes the client version the wire-schema version; exact-inequality mismatch = miss + drop, NEVER decode, checked in both has() and get_canvas()). Integration: request_now() checks Tier 2 on a Tier-1 miss (synchronous promote), Ready responses write through to both tiers, Pending never writes; viewer runs the visit sweep on body-open + the background sweep on a 5-min timer. 30 new disk-cache tests + 7 request-integration + 3 sweep-wiring tests (restart persistence, sweep independence both directions, cap semantics, version-mismatch never-decode, corrupt-index recovery). Full client suite 3,440/3,440, 0 orphans; cold-parse clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Claude Fable 5
@@ -0,0 +1,568 @@
|
||||
extends RefCounted
|
||||
|
||||
## Client-side DISK-BACKED cache store for decoded StepCanvasResponse canvas
|
||||
## payloads (T-1183, D-255(d) Tier 2/3: "client disk-backed FileAccess store").
|
||||
## Layers BENEATH step_canvas_cache.gd's in-memory LRU (Tier 1, T-1182) — a
|
||||
## Tier-1 miss checks this store before going to the wire; a fresh wire
|
||||
## response is written through to BOTH tiers (step_canvas_request.gd is the
|
||||
## integration point). Never SQLite in any shape (workshop-rejected
|
||||
## independently by both Stig and Dudley — stig-round2.md §(c), plain
|
||||
## `FileAccess` + a JSON index won on both agents' analyses).
|
||||
##
|
||||
## **What is persisted: THE WIRE PAYLOAD, PRE-DECODE.** The `canvas`
|
||||
## Dictionary this store writes/reads is byte-identical to what
|
||||
## step_canvas_protocol.gd's `_decode_encoded_canvas()` produces — dense
|
||||
## fields (`morphology`/`elev_q`/`moisture_q`/`vegetation`/`glaciation`/
|
||||
## `flooded_q`) are still PNG-encoded `PackedByteArray`s, `temp_dc`/
|
||||
## `settlement_id` are raw msgpack-decoded arrays, `courses`/`cliffs` are raw
|
||||
## sparse lists. `Image.load_png_from_buffer()` — the actual pixel decode —
|
||||
## never runs here; it happens only in step_canvas_terrain_layer.gd at draw
|
||||
## time, exactly as it does for a Tier-1 hit today. "The disk file IS the
|
||||
## wire payload, no re-encoding for storage" (stig-round1.md §4).
|
||||
##
|
||||
## **Cache key — extends the exact Tier-1 tuple** (StepCanvasCache.make_key,
|
||||
## which itself mirrors server/src/atlas/step_canvas.rs's own
|
||||
## StepCanvasCache key): (body_id, rung, center, extent, min_wl_m). Global
|
||||
## (rung 0) collapses center/extent to the (0,0)/(0,0) sentinel identically
|
||||
## (make_key() delegates to StepCanvasCache.make_key() directly — one
|
||||
## composite-key discipline, not two).
|
||||
##
|
||||
## **Three-tier eviction (stig-round2.md §(c), the concrete spec this file
|
||||
## implements verbatim):**
|
||||
## Tier 1 (Global/rung-0 geometry) — RETENTION FLOOR, no sweep at all. The
|
||||
## only tier this file exempts unconditionally from both sweeps below.
|
||||
## Tier 2 (sub-global geometry) — TWO independent, separately-triggered
|
||||
## sweeps, kept apart per Jeroen's storage-eviction ruling (staleness and
|
||||
## storage are distinct axes; geometry never goes stale, D-227):
|
||||
## (2a) time-since-last-visit — evict any entry whose last_read_at is
|
||||
## older than STORAGE_TTL_SEC, run on body-open (cheap: an index
|
||||
## scan, no bulk file I/O).
|
||||
## (2b) LRU-capacity — if total sub-global disk usage for a body
|
||||
## exceeds SUB_GLOBAL_BYTE_BUDGET, evict oldest-touched entries
|
||||
## first, run on a coarse background timer.
|
||||
## Tier 3 (sim-state-tagged planes) — explicit TTL, staleness-motivated,
|
||||
## structurally separate from tiers 1/2 (`now > written_at + sim_ttl`),
|
||||
## never touched by 2a/2b. NOT YET POPULATED: every EncodedStepCanvas
|
||||
## field is geometry per D-227 today (frozen/flooded ride the existing
|
||||
## glaciation/morphology-water-class fields, Araminta's round-1 schema —
|
||||
## no distinct sim-state WIRE field exists yet for this ticket to tag).
|
||||
## The `sim_ttl`/tier machinery below is WIRED and tested but has no
|
||||
## production caller until a sim-state field lands on EncodedStepCanvas.
|
||||
##
|
||||
## **HARDENING (D-255(d), both mandatory):**
|
||||
## (i) Per-body deep-rung retention cap — DEEP_RUNGS (Block, Chunk; the
|
||||
## two finest, matching troblum-round2.md S3's specific pan-assembly
|
||||
## concern) are capped at MAX_DEEP_RUNG_ENTRIES_PER_BODY resident
|
||||
## entries, independent of the Global floor and independent of the
|
||||
## general Tier-2 byte budget — closes the D-226(d) accumulation gap
|
||||
## against a systematic exhaustive pan (most plausibly the
|
||||
## AtlasAgentInterface QA channel, D-226 item 4) structurally rather
|
||||
## than by practical improbability. Enforced synchronously in put():
|
||||
## inserting past the cap evicts the oldest deep-rung entry for that
|
||||
## body FIRST (LRU order), before the general 2b sweep ever runs.
|
||||
## (ii) Schema/version tag — every persistent entry is stamped with
|
||||
## CACHE_SCHEMA_VERSION at write time (see that constant's own doc
|
||||
## for the version-tag source decision + rationale). A read-time
|
||||
## mismatch is treated as a cache miss: the stale-schema file is
|
||||
## discarded (never handed to a caller, never decoded) and the index
|
||||
## entry is dropped. This is the one place D-192's co-ship guarantee
|
||||
## does not reach (D-192 2026-07-23 amendment) — a disk cache
|
||||
## survives a game update; the live wire does not need this because
|
||||
## client+server always launch in lockstep.
|
||||
##
|
||||
## **Sweep triggers (never per-frame — stig-round2.md §"Sweep triggers"):**
|
||||
## callers invoke run_visit_sweep() once on body-open (cheap, index-only) and
|
||||
## run_background_sweep() on a coarse timer (LRU-capacity + Tier-3 TTL,
|
||||
## backgroundable). Neither is wired to _process()/a per-frame signal by this
|
||||
## file — the caller (step_canvas_viewer.gd) owns invoking these at the
|
||||
## right moments, matching D-227's "bookkeeping, not gameplay-adjacent work"
|
||||
## instruction.
|
||||
##
|
||||
## D-227: every tier here is an evictable CACHE, never a source of truth.
|
||||
## Deleting the whole cache root at any time changes client behavior only by
|
||||
## causing re-fetches — verified structurally by test_step_canvas_disk_cache.
|
||||
## gd's malformed-index-recovery + missing-payload-file cases, not just by
|
||||
## comment.
|
||||
|
||||
const StepCanvasCache := preload("res://ui/implant/apps/atlas/step_canvas/step_canvas_cache.gd")
|
||||
|
||||
## Tier tags, mirroring stig-round2.md's IndexEntry.tier field exactly.
|
||||
const TIER_GEOMETRY: String = "Geometry"
|
||||
const TIER_SIM_STATE: String = "SimState"
|
||||
|
||||
const DEFAULT_ROOT: String = "user://atlas_cache/"
|
||||
const INDEX_FILENAME: String = "index.json"
|
||||
|
||||
## (2a) Time-since-last-visit threshold — stig-round2.md's "days-to-weeks of
|
||||
## real wall-clock time, tunable — not a round-2 architecture call, a tuning
|
||||
## pass once this ships" starting point. 14 days as the initial tunable.
|
||||
const STORAGE_TTL_SEC: int = 14 * 24 * 60 * 60
|
||||
|
||||
## (2b) Sub-global per-body disk budget — stig-round2.md's "same order of
|
||||
## magnitude as tier 1... a tuning-pass number once real play-pattern data
|
||||
## exists" starting point. 256 MiB/body.
|
||||
const SUB_GLOBAL_BYTE_BUDGET: int = 256 * 1024 * 1024
|
||||
|
||||
## (i) Deep-rung hardening cap — the two finest rungs (Block 128m, Chunk
|
||||
## 64m), independent of SUB_GLOBAL_BYTE_BUDGET and the Global floor. A
|
||||
## per-body ENTRY COUNT (not bytes) because the accumulation risk S3 names is
|
||||
## about DISTINCT WINDOWS assembling whole-body coverage, not aggregate
|
||||
## bytes — capping entry count directly bounds how much ground a client can
|
||||
## have simultaneously resident at fine spacing, which is the D-226(d)
|
||||
## purpose (information content), not a byte-budget proxy for it.
|
||||
const DEEP_RUNGS: Array = ["Block", "Chunk"]
|
||||
const MAX_DEEP_RUNG_ENTRIES_PER_BODY: int = 512
|
||||
|
||||
## (ii) Schema/version tag source: the game's `project.yaml` `version:`
|
||||
## field (e.g. "0.4.0"), read via the identical technique
|
||||
## loading_screen.gd's `_read_client_version()` already uses (line-scan for
|
||||
## "version:", no YAML parser dependency). CHOSEN over a `generator_sha`-
|
||||
## style stamp because:
|
||||
## - project.yaml's version is ALREADY the project's single source of
|
||||
## truth for "what build is this" (CLAUDE.md: "Version source of truth:
|
||||
## project.yaml") — reusing it needs no new stamp-generation machinery
|
||||
## anywhere, client or server.
|
||||
## - The D-255(c) wire shape (EncodedStepCanvas's field set) changes in
|
||||
## lockstep with client releases in this single-repo, subprocess-co-ship
|
||||
## project (D-192) — there is no independent server-only wire-schema
|
||||
## versioning surface a generator_sha would need to track separately;
|
||||
## the client's own version IS the wire-schema version for this
|
||||
## project's deployment model.
|
||||
## - It is legible for a human debugging a stale-cache report ("this file
|
||||
## was written by 0.4.0, I am running 0.5.0") in a way a SHA is not.
|
||||
## A mismatch — ANY string difference, not just an older/newer comparison
|
||||
## (CLAUDE.md's semver note is for systems.db's schema_version LINEAGE,
|
||||
## which needs ordering for migration; this cache has no migration path at
|
||||
## all, so exact-match-or-miss is the correct, simpler rule) — is a cache
|
||||
## miss: re-fetch, never decode. See read_entry()'s version check.
|
||||
## (Declared as a const-adjacent static func, ahead of the instance vars
|
||||
## below, per gdlint's class-definitions-order: statics precede prvvars.)
|
||||
|
||||
var _root: String = DEFAULT_ROOT
|
||||
|
||||
## Per-body loaded index, held in memory for the session — matching
|
||||
## step_canvas_cache.gd's own "erase+reinsert = move-to-MRU" idiom, now
|
||||
## keyed one level up (body_id -> key -> IndexEntry Dictionary). Loaded
|
||||
## lazily on first touch per body, not all at once (a player who never
|
||||
## revisits most bodies never pays the parse cost for their index files).
|
||||
var _indexes: Dictionary = {} # body_id String -> (key String -> IndexEntry Dictionary)
|
||||
|
||||
|
||||
static func current_schema_version() -> String:
|
||||
var yaml_path := ProjectSettings.globalize_path("res://") + "/../project.yaml"
|
||||
if not FileAccess.file_exists(yaml_path):
|
||||
return "?.?.?"
|
||||
var file := FileAccess.open(yaml_path, FileAccess.READ)
|
||||
if file == null:
|
||||
return "?.?.?"
|
||||
var content := file.get_as_text()
|
||||
file.close()
|
||||
for line: String in content.split("\n"):
|
||||
if line.begins_with("version:"):
|
||||
var parts := line.split(":", false, 1)
|
||||
if parts.size() >= 2:
|
||||
return parts[1].strip_edges()
|
||||
return "?.?.?"
|
||||
|
||||
|
||||
## `root` is injectable so tests never touch the real
|
||||
## user://atlas_cache/ directory (test_step_canvas_disk_cache.gd passes a
|
||||
## disposable per-test-run subdirectory and removes it in after_test()).
|
||||
func _init(root: String = DEFAULT_ROOT) -> void:
|
||||
_root = root if root.ends_with("/") else root + "/"
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Path / key helpers
|
||||
# =============================================================================
|
||||
|
||||
|
||||
func _body_dir(body_id: String) -> String:
|
||||
return _root + body_id.validate_filename() + "/"
|
||||
|
||||
|
||||
func _index_path(body_id: String) -> String:
|
||||
return _body_dir(body_id) + INDEX_FILENAME
|
||||
|
||||
|
||||
## Stable non-negative filename for a composite key — the key string itself
|
||||
## isn't filesystem-safe on every target platform (':'/','), matching
|
||||
## stig-round2.md's own "named by a hash of key" directive. `key.hash() &
|
||||
## 0x7FFFFFFF` is the same non-negative-hash idiom reach_screen.gd already
|
||||
## uses elsewhere in this app for a stable derived value from a String.
|
||||
static func _payload_filename(key: String) -> String:
|
||||
var h: int = key.hash() & 0x7FFFFFFF
|
||||
return "%08x.dat" % h
|
||||
|
||||
|
||||
func _payload_path(body_id: String, key: String) -> String:
|
||||
return _body_dir(body_id) + _payload_filename(key)
|
||||
|
||||
|
||||
## Delegates to StepCanvasCache.make_key() directly — ONE composite-key
|
||||
## discipline shared by both tiers, never a second parallel key scheme.
|
||||
static func make_key(
|
||||
body_id: String, rung: String, center: Vector2i, extent: Vector2i, min_wl_m: int = 0
|
||||
) -> String:
|
||||
return StepCanvasCache.make_key(body_id, rung, center, extent, min_wl_m)
|
||||
|
||||
|
||||
static func _is_global_rung(rung: String) -> bool:
|
||||
return rung == "Global"
|
||||
|
||||
|
||||
static func _is_deep_rung(rung: String) -> bool:
|
||||
return DEEP_RUNGS.has(rung)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Index load / save (malformed-recovery hardened)
|
||||
# =============================================================================
|
||||
|
||||
|
||||
## Returns the in-memory index Dictionary for `body_id`, loading it from disk
|
||||
## on first touch. A missing OR malformed (truncated/corrupt JSON, wrong top-
|
||||
## level type) index file is treated as an EMPTY cache for that body — never
|
||||
## a crash, matching stig-round1.md's "missing/corrupt index -> treat as
|
||||
## empty cache, don't crash, don't block first paint" instruction. The
|
||||
## corrupt file is left on disk untouched (a caller that never puts() again
|
||||
## for that body leaves it inert; the first successful save_index() call
|
||||
## overwrites it with a valid one).
|
||||
func _load_index(body_id: String) -> Dictionary:
|
||||
if _indexes.has(body_id):
|
||||
return _indexes[body_id]
|
||||
var idx: Dictionary = {}
|
||||
var path := _index_path(body_id)
|
||||
if FileAccess.file_exists(path):
|
||||
var file := FileAccess.open(path, FileAccess.READ)
|
||||
if file != null:
|
||||
var content := file.get_as_text()
|
||||
file.close()
|
||||
var json := JSON.new()
|
||||
var err := json.parse(content)
|
||||
if err == OK and json.data is Dictionary:
|
||||
idx = json.data
|
||||
_indexes[body_id] = idx
|
||||
return idx
|
||||
|
||||
|
||||
func _save_index(body_id: String) -> void:
|
||||
var idx: Dictionary = _indexes.get(body_id, {})
|
||||
var dir_err := DirAccess.make_dir_recursive_absolute(_body_dir(body_id))
|
||||
if dir_err != OK:
|
||||
push_warning(
|
||||
"StepCanvasDiskCache: cannot create cache dir for '%s': %s"
|
||||
% [body_id, error_string(dir_err)]
|
||||
)
|
||||
return
|
||||
var file := FileAccess.open(_index_path(body_id), FileAccess.WRITE)
|
||||
if file == null:
|
||||
push_warning("StepCanvasDiskCache: cannot write index for '%s'" % body_id)
|
||||
return
|
||||
file.store_string(JSON.stringify(idx))
|
||||
file.close()
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Read / write entries
|
||||
# =============================================================================
|
||||
|
||||
|
||||
## Fetch a cached canvas, touching last_read_at (drives 2a/2b). Returns null
|
||||
## on a miss — including a SCHEMA-VERSION-MISMATCH miss (ii): a stale-schema
|
||||
## entry is dropped from the index and its payload file deleted WITHOUT ever
|
||||
## being decoded, exactly the "mismatch = cache miss, re-fetch, never
|
||||
## decode" contract. Also returns null (and drops the stale index row) if
|
||||
## the payload file is missing on disk despite an index entry — the index
|
||||
## and the filesystem can disagree (manual deletion, platform storage
|
||||
## pressure clearing files without updating the index), and a dangling index
|
||||
## row must never be handed to a caller as a hit.
|
||||
func get_canvas(
|
||||
body_id: String, rung: String, center: Vector2i, extent: Vector2i, min_wl_m: int = 0
|
||||
) -> Variant:
|
||||
var key := make_key(body_id, rung, center, extent, min_wl_m)
|
||||
var idx := _load_index(body_id)
|
||||
if not idx.has(key):
|
||||
return null
|
||||
var entry: Dictionary = idx[key]
|
||||
|
||||
if str(entry.get("schema_version", "")) != current_schema_version():
|
||||
_drop_entry(body_id, key)
|
||||
return null
|
||||
|
||||
var payload_path := _payload_path(body_id, key)
|
||||
if not FileAccess.file_exists(payload_path):
|
||||
_drop_entry(body_id, key)
|
||||
return null
|
||||
|
||||
var file := FileAccess.open(payload_path, FileAccess.READ)
|
||||
if file == null:
|
||||
_drop_entry(body_id, key)
|
||||
return null
|
||||
var canvas: Variant = file.get_var()
|
||||
file.close()
|
||||
if not canvas is Dictionary:
|
||||
_drop_entry(body_id, key)
|
||||
return null
|
||||
|
||||
entry["last_read_at"] = Time.get_unix_time_from_system()
|
||||
idx[key] = entry
|
||||
_save_index(body_id)
|
||||
return canvas
|
||||
|
||||
|
||||
## True if a (currently valid — schema-matched, payload present) entry
|
||||
## exists, without touching last_read_at (a pure existence check, mirroring
|
||||
## step_canvas_cache.gd's own has()/get_canvas() split).
|
||||
func has(
|
||||
body_id: String, rung: String, center: Vector2i, extent: Vector2i, min_wl_m: int = 0
|
||||
) -> bool:
|
||||
var key := make_key(body_id, rung, center, extent, min_wl_m)
|
||||
var idx := _load_index(body_id)
|
||||
if not idx.has(key):
|
||||
return false
|
||||
var entry: Dictionary = idx[key]
|
||||
if str(entry.get("schema_version", "")) != current_schema_version():
|
||||
return false
|
||||
return FileAccess.file_exists(_payload_path(body_id, key))
|
||||
|
||||
|
||||
## Persist a canvas Dictionary — the exact pre-decode wire payload shape
|
||||
## step_canvas_protocol.gd produces (PNG bytes untouched). `sim_ttl_sec`, if
|
||||
## > 0, tags the entry Tier 3 (SimState, real staleness TTL); omitted/0 tags
|
||||
## Tier 2 (Geometry, D-227 never-stale). `retention_floor` should be true
|
||||
## ONLY for rung == "Global" callers (step_canvas_request.gd computes this
|
||||
## from the rung it's writing, mirroring make_key()'s own Global handling) —
|
||||
## a floored entry is exempt from BOTH sweeps unconditionally.
|
||||
##
|
||||
## (i) HARDENING: for a deep-rung (Block/Chunk) write, enforces
|
||||
## MAX_DEEP_RUNG_ENTRIES_PER_BODY SYNCHRONOUSLY before inserting — if this
|
||||
## put() would exceed the cap, the oldest (by last_read_at) deep-rung entry
|
||||
## for this body is evicted first. This runs on every deep-rung put(), not
|
||||
## just during the periodic background sweep — the cap must never be
|
||||
## transiently exceeded even by one entry, since the accumulation risk it
|
||||
## closes is about a sustained systematic pan, not a single burst.
|
||||
func put(
|
||||
body_id: String,
|
||||
rung: String,
|
||||
center: Vector2i,
|
||||
extent: Vector2i,
|
||||
canvas: Dictionary,
|
||||
min_wl_m: int = 0,
|
||||
sim_ttl_sec: int = 0
|
||||
) -> void:
|
||||
var key := make_key(body_id, rung, center, extent, min_wl_m)
|
||||
var idx := _load_index(body_id)
|
||||
|
||||
if _is_deep_rung(rung) and not idx.has(key):
|
||||
_enforce_deep_rung_cap(body_id, idx)
|
||||
|
||||
var dir_err := DirAccess.make_dir_recursive_absolute(_body_dir(body_id))
|
||||
if dir_err != OK:
|
||||
push_warning(
|
||||
"StepCanvasDiskCache: cannot create cache dir for '%s': %s"
|
||||
% [body_id, error_string(dir_err)]
|
||||
)
|
||||
return
|
||||
var file := FileAccess.open(_payload_path(body_id, key), FileAccess.WRITE)
|
||||
if file == null:
|
||||
push_warning("StepCanvasDiskCache: cannot write payload for '%s'/'%s'" % [body_id, key])
|
||||
return
|
||||
file.store_var(canvas)
|
||||
file.close()
|
||||
|
||||
var now: int = Time.get_unix_time_from_system()
|
||||
var entry: Dictionary = {
|
||||
"file_path": _payload_path(body_id, key),
|
||||
"tier": TIER_SIM_STATE if sim_ttl_sec > 0 else TIER_GEOMETRY,
|
||||
"written_at": now,
|
||||
"last_read_at": now,
|
||||
"size_bytes": FileAccess.get_file_as_bytes(_payload_path(body_id, key)).size(),
|
||||
"sim_ttl": sim_ttl_sec if sim_ttl_sec > 0 else null,
|
||||
"retention_floor": _is_global_rung(rung),
|
||||
"schema_version": current_schema_version(),
|
||||
}
|
||||
idx[key] = entry
|
||||
_save_index(body_id)
|
||||
|
||||
|
||||
## TEST-SUPPORT ONLY: overwrite an existing entry's `written_at`/
|
||||
## `last_read_at` so sweep tests can exercise STORAGE_TTL_SEC/sim_ttl
|
||||
## expiry without waiting on real wall-clock time. No production caller —
|
||||
## step_canvas_request.gd never calls this. A no-op if `key` isn't cached.
|
||||
func _debug_backdate_entry(body_id: String, key: String, written_at: int, last_read_at: int) -> void:
|
||||
var idx := _load_index(body_id)
|
||||
if not idx.has(key):
|
||||
return
|
||||
var entry: Dictionary = idx[key]
|
||||
entry["written_at"] = written_at
|
||||
entry["last_read_at"] = last_read_at
|
||||
idx[key] = entry
|
||||
_save_index(body_id)
|
||||
|
||||
|
||||
## Evict the single oldest (lowest last_read_at) deep-rung entry for
|
||||
## `body_id`, if adding one more would push the deep-rung count for this
|
||||
## body over MAX_DEEP_RUNG_ENTRIES_PER_BODY. A no-op while under the cap.
|
||||
func _enforce_deep_rung_cap(body_id: String, idx: Dictionary) -> void:
|
||||
var deep_keys: Array = []
|
||||
for key: String in idx.keys():
|
||||
var entry: Dictionary = idx[key]
|
||||
if not bool(entry.get("retention_floor", false)) and _entry_is_deep_rung(key):
|
||||
deep_keys.append(key)
|
||||
if deep_keys.size() < MAX_DEEP_RUNG_ENTRIES_PER_BODY:
|
||||
return
|
||||
deep_keys.sort_custom(
|
||||
func(a: String, b: String) -> bool:
|
||||
return int(idx[a].get("last_read_at", 0)) < int(idx[b].get("last_read_at", 0))
|
||||
)
|
||||
var evict_count: int = deep_keys.size() - MAX_DEEP_RUNG_ENTRIES_PER_BODY + 1
|
||||
for i in range(evict_count):
|
||||
_drop_entry(body_id, deep_keys[i])
|
||||
|
||||
|
||||
## A key's rung is embedded as its second colon-delimited field
|
||||
## (StepCanvasCache.make_key()'s own "%s:%s:..." shape) — parsed back out
|
||||
## rather than carried as a separate index column, since the key already
|
||||
## encodes it and a second copy would be a redundant field to keep in sync.
|
||||
static func _entry_is_deep_rung(key: String) -> bool:
|
||||
var parts := key.split(":")
|
||||
if parts.size() < 2:
|
||||
return false
|
||||
return _is_deep_rung(parts[1])
|
||||
|
||||
|
||||
func _drop_entry(body_id: String, key: String) -> void:
|
||||
var idx := _load_index(body_id)
|
||||
if not idx.has(key):
|
||||
return
|
||||
var entry: Dictionary = idx[key]
|
||||
var path := str(entry.get("file_path", _payload_path(body_id, key)))
|
||||
if FileAccess.file_exists(path):
|
||||
DirAccess.remove_absolute(path)
|
||||
idx.erase(key)
|
||||
_save_index(body_id)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Sweeps (2a on body-open, 2b + Tier-3 TTL on a coarse background timer —
|
||||
# never per-frame; the caller decides WHEN to invoke these, this file only
|
||||
# implements WHAT each sweep does)
|
||||
# =============================================================================
|
||||
|
||||
|
||||
## (2a) Time-since-last-visit — drop every non-floored entry whose
|
||||
## last_read_at exceeds STORAGE_TTL_SEC. Cheap: an index scan only, no bulk
|
||||
## payload-file stat()s beyond the deletes this triggers. Independent of
|
||||
## (2b)/Tier-3 — this NEVER touches a floored (Global) entry, and geometry
|
||||
## entries are storage-motivated only (never "wrong"), never Tier-3
|
||||
## staleness-motivated.
|
||||
func run_visit_sweep(body_id: String) -> void:
|
||||
var idx := _load_index(body_id)
|
||||
var now: int = Time.get_unix_time_from_system()
|
||||
var to_drop: Array = []
|
||||
for key: String in idx.keys():
|
||||
var entry: Dictionary = idx[key]
|
||||
if bool(entry.get("retention_floor", false)):
|
||||
continue
|
||||
if now - int(entry.get("last_read_at", now)) > STORAGE_TTL_SEC:
|
||||
to_drop.append(key)
|
||||
for key in to_drop:
|
||||
_drop_entry(body_id, key)
|
||||
|
||||
|
||||
## (2b) LRU-capacity — if total sub-global (non-floored) bytes for this body
|
||||
## exceed SUB_GLOBAL_BYTE_BUDGET, evict oldest-touched entries first until
|
||||
## under budget. Independent of (2a)/Tier-3 — a byte-budget question only,
|
||||
## never a staleness one; never touches a floored entry.
|
||||
func run_capacity_sweep(body_id: String) -> void:
|
||||
var idx := _load_index(body_id)
|
||||
var sub_global_keys: Array = []
|
||||
var total_bytes: int = 0
|
||||
for key: String in idx.keys():
|
||||
var entry: Dictionary = idx[key]
|
||||
if bool(entry.get("retention_floor", false)):
|
||||
continue
|
||||
sub_global_keys.append(key)
|
||||
total_bytes += int(entry.get("size_bytes", 0))
|
||||
if total_bytes <= SUB_GLOBAL_BYTE_BUDGET:
|
||||
return
|
||||
sub_global_keys.sort_custom(
|
||||
func(a: String, b: String) -> bool:
|
||||
return int(idx[a].get("last_read_at", 0)) < int(idx[b].get("last_read_at", 0))
|
||||
)
|
||||
for key in sub_global_keys:
|
||||
if total_bytes <= SUB_GLOBAL_BYTE_BUDGET:
|
||||
break
|
||||
total_bytes -= int(idx[key].get("size_bytes", 0))
|
||||
_drop_entry(body_id, key)
|
||||
|
||||
|
||||
## Tier 3 — drop every SimState entry past its own sim_ttl, regardless of
|
||||
## capacity pressure or last_read_at. Structurally separate condition from
|
||||
## both sweeps above (`now > written_at + sim_ttl`) — a sim-state entry does
|
||||
## NOT get to live longer just because disk space is available, and is
|
||||
## never evicted early just because it was recently read.
|
||||
func run_staleness_sweep(body_id: String) -> void:
|
||||
var idx := _load_index(body_id)
|
||||
var now: int = Time.get_unix_time_from_system()
|
||||
var to_drop: Array = []
|
||||
for key: String in idx.keys():
|
||||
var entry: Dictionary = idx[key]
|
||||
if str(entry.get("tier", TIER_GEOMETRY)) != TIER_SIM_STATE:
|
||||
continue
|
||||
var ttl_raw: Variant = entry.get("sim_ttl")
|
||||
if ttl_raw == null:
|
||||
continue
|
||||
if now > int(entry.get("written_at", now)) + int(ttl_raw):
|
||||
to_drop.append(key)
|
||||
for key in to_drop:
|
||||
_drop_entry(body_id, key)
|
||||
|
||||
|
||||
## Convenience: the three sweeps a coarse background timer runs together —
|
||||
## run_visit_sweep() is deliberately NOT included here (it belongs on
|
||||
## body-open only, per stig-round2.md's own sweep-trigger split).
|
||||
func run_background_sweep(body_id: String) -> void:
|
||||
run_capacity_sweep(body_id)
|
||||
run_staleness_sweep(body_id)
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# Introspection (tests + diagnostics)
|
||||
# =============================================================================
|
||||
|
||||
|
||||
func entry_count(body_id: String) -> int:
|
||||
return _load_index(body_id).size()
|
||||
|
||||
|
||||
func has_entry_for_key(body_id: String, key: String) -> bool:
|
||||
return _load_index(body_id).has(key)
|
||||
|
||||
|
||||
## Removes the entire on-disk cache root (all bodies). D-227: this only
|
||||
## causes re-fetches on next touch — never a correctness change. Used by
|
||||
## tests for cleanup; also the structural answer to "manual clear" from the
|
||||
## class doc's D-227 guarantee.
|
||||
func clear_all() -> void:
|
||||
_remove_dir_recursive(_root)
|
||||
_indexes.clear()
|
||||
|
||||
|
||||
static func _remove_dir_recursive(path: String) -> void:
|
||||
var dir := DirAccess.open(path)
|
||||
if dir == null:
|
||||
return
|
||||
dir.list_dir_begin()
|
||||
var entry := dir.get_next()
|
||||
while entry != "":
|
||||
if entry != "." and entry != "..":
|
||||
var full := path.path_join(entry)
|
||||
if dir.current_is_dir():
|
||||
_remove_dir_recursive(full)
|
||||
else:
|
||||
DirAccess.remove_absolute(full)
|
||||
entry = dir.get_next()
|
||||
dir.list_dir_end()
|
||||
DirAccess.remove_absolute(path)
|
||||
@@ -1,11 +1,21 @@
|
||||
extends Node
|
||||
|
||||
## Step-canvas request orchestration for StepCanvasViewer (T-1182, D-255(c)/
|
||||
## (d)). Owns the in-memory LRU cache, fires StepCanvasRequest frames, and
|
||||
## retries on a Pending response — the same D-225 poll/cache/enqueue serving
|
||||
## model atlas_window_request.gd already established for the legacy
|
||||
## (d)). Owns the in-memory LRU cache (Tier 1) AND the disk-backed cache
|
||||
## (Tier 2/3, T-1183), fires StepCanvasRequest frames, and retries on a
|
||||
## Pending response — the same D-225 poll/cache/enqueue serving model
|
||||
## atlas_window_request.gd already established for the legacy
|
||||
## district_window carrier, now pointed at the new tagged envelope.
|
||||
##
|
||||
## **Three-tier read path (D-255(d), cheapest-first): Tier 1 (in-memory LRU)
|
||||
## -> Tier 2 (disk) -> wire.** request_now() checks Tier 1 first (unchanged,
|
||||
## synchronous); a Tier-1 miss now checks Tier 2 (also synchronous — a
|
||||
## FileAccess read, not a network round-trip) before firing a
|
||||
## SimBridge.request_step_canvas() wire request. A disk hit is promoted into
|
||||
## Tier 1 on read (matches step_canvas_cache.gd's own "cache-accelerated"
|
||||
## framing — the next request for the same key is a Tier-1 hit). A fresh
|
||||
## Ready wire response is written through to BOTH tiers in on_response().
|
||||
##
|
||||
## No `class_name` on purpose, matching every other viewer-owned helper in
|
||||
## this cluster (atlas_overlay_bar.gd/atlas_window_request.gd, established
|
||||
## precedent): the owner (StepCanvasViewer) passes itself to `_init()`.
|
||||
@@ -22,18 +32,23 @@ extends Node
|
||||
## step_canvas_protocol.gd's doc) — callers reading `get_held_extent()` for
|
||||
## a Global-held canvas must special-case it themselves (the viewer does,
|
||||
## via StepCanvasTransport.RUNG_GLOBAL checks), matching this ticket's own
|
||||
## "Global rung ignores wire extent" instruction.
|
||||
## "Global rung ignores wire extent" instruction. **The disk-tier key uses
|
||||
## the SAME requested/echoed extent as Tier 1 and the server's own key**
|
||||
## (confirmed correct in the #203 review) — no separate extent convention
|
||||
## for the disk tier.
|
||||
|
||||
signal canvas_ready(response: Dictionary) # emitted on a cache hit OR a fresh Ready response
|
||||
|
||||
const StepCanvasCache := preload("res://ui/implant/apps/atlas/step_canvas/step_canvas_cache.gd")
|
||||
const StepCanvasDiskCache := preload("res://ui/implant/apps/atlas/step_canvas/step_canvas_disk_cache.gd")
|
||||
|
||||
const INITIAL_RETRY_DELAY: float = 0.5
|
||||
const MAX_RETRY_DELAY: float = 4.0
|
||||
const MAX_RETRIES: int = 30
|
||||
|
||||
var _owner = null # StepCanvasViewer (untyped to avoid cyclic ref)
|
||||
var _cache: Variant = null # StepCanvasCache
|
||||
var _cache: Variant = null # StepCanvasCache (Tier 1, in-memory)
|
||||
var _disk_cache: Variant = null # StepCanvasDiskCache (Tier 2/3, T-1183)
|
||||
|
||||
var _body_id: String = ""
|
||||
var _rung: String = ""
|
||||
@@ -51,9 +66,19 @@ var _pending: bool = false
|
||||
var _retries: int = 0
|
||||
|
||||
|
||||
func _init(owner_ref = null) -> void:
|
||||
## `disk_cache_root` is test-injection-only (default "" -> StepCanvasDiskCache's
|
||||
## own production default, user://atlas_cache/): every production call site
|
||||
## (step_canvas_viewer.gd's `StepCanvasRequest.new(self)`) omits it, keeping
|
||||
## the real cache path unchanged; tests pass a disposable per-run subdirectory
|
||||
## so they never touch or leak into the real cache directory.
|
||||
func _init(owner_ref = null, disk_cache_root: String = "") -> void:
|
||||
_owner = owner_ref
|
||||
_cache = StepCanvasCache.new()
|
||||
_disk_cache = (
|
||||
StepCanvasDiskCache.new(disk_cache_root)
|
||||
if not disk_cache_root.is_empty()
|
||||
else StepCanvasDiskCache.new()
|
||||
)
|
||||
|
||||
|
||||
## Reset in-flight bookkeeping for a fresh body/rung entry — does NOT clear
|
||||
@@ -63,9 +88,12 @@ func reset() -> void:
|
||||
_retries = 0
|
||||
|
||||
|
||||
## Fire (or serve from cache) a step-canvas request. Cache hit -> immediate
|
||||
## synchronous canvas_ready emit, no network traffic. Cache miss -> send the
|
||||
## request now; the response (or a Pending retry chain) arrives later via
|
||||
## Fire (or serve from cache) a step-canvas request. Tier-1 hit -> immediate
|
||||
## synchronous canvas_ready emit, no disk or network I/O. Tier-1 miss checks
|
||||
## Tier 2 (disk, D-255(d)) next — also synchronous, a FileAccess read, no
|
||||
## wire traffic — and PROMOTES a disk hit into Tier 1 (the next request for
|
||||
## this exact key is a Tier-1 hit). Only a miss on BOTH tiers sends a wire
|
||||
## request; the response (or a Pending retry chain) arrives later via
|
||||
## on_response().
|
||||
func request_now(
|
||||
body_id: String, rung: String, center: Vector2i, extent: Vector2i, min_wl_m: int = 0
|
||||
@@ -84,6 +112,15 @@ func request_now(
|
||||
canvas_ready.emit(cached)
|
||||
return
|
||||
|
||||
var disk_cached: Variant = _disk_cache.get_canvas(body_id, rung, center, extent, min_wl_m)
|
||||
if disk_cached != null:
|
||||
_pending = false
|
||||
_retries = 0
|
||||
_cache.put(body_id, rung, center, extent, disk_cached, min_wl_m)
|
||||
_held_extent = _echoed_extent(disk_cached, rung, extent)
|
||||
canvas_ready.emit(disk_cached)
|
||||
return
|
||||
|
||||
_pending = true
|
||||
_retries = 0
|
||||
SimBridge.request_step_canvas(body_id, rung, center, extent, min_wl_m)
|
||||
@@ -121,6 +158,7 @@ func on_response(response: Dictionary) -> void:
|
||||
_retries = 0
|
||||
_held_extent = _echoed_extent(canvas, _rung, response.get("extent", Vector2i.ZERO))
|
||||
_cache.put(_body_id, _rung, _center, _extent, canvas, _min_wl_m)
|
||||
_disk_cache.put(_body_id, _rung, _center, _extent, canvas, _min_wl_m)
|
||||
canvas_ready.emit(canvas)
|
||||
|
||||
|
||||
@@ -192,3 +230,7 @@ func get_held_extent() -> Vector2i:
|
||||
|
||||
func get_cache() -> Variant:
|
||||
return _cache
|
||||
|
||||
|
||||
func get_disk_cache() -> Variant:
|
||||
return _disk_cache
|
||||
|
||||
@@ -46,6 +46,13 @@ const StepCanvasRequest := preload("res://ui/implant/apps/atlas/step_canvas/step
|
||||
const PANEL_MARGIN: float = 16.0
|
||||
const OVERLAY_BAR_HEADER_RESERVE: float = 360.0
|
||||
|
||||
## Coarse background sweep interval (T-1183, D-255(d)): the disk cache's 2b
|
||||
## (LRU-capacity) + Tier-3 (sim-state TTL) sweeps run on this timer, NEVER
|
||||
## per-frame/per-step-cross (stig-round2.md "Sweep triggers": "proposed on
|
||||
## the order of minutes, tunable"). 5 minutes as the initial tunable — a
|
||||
## deferred, low-priority pass, never blocking a frame or a step-cross.
|
||||
const DISK_SWEEP_INTERVAL_SEC: float = 300.0
|
||||
|
||||
const PAN_SPEED_PX_S: float = 220.0
|
||||
const EDGE_SCROLL_MARGIN_PX: float = 24.0
|
||||
|
||||
@@ -102,6 +109,7 @@ var _screen_header: ImplantHeader = null
|
||||
var _overlay_bar = null
|
||||
var _legend_panel = null
|
||||
var _request = null # StepCanvasRequest
|
||||
var _disk_sweep_timer: Timer = null # T-1183 coarse background sweep trigger
|
||||
|
||||
|
||||
func _ready() -> void:
|
||||
@@ -138,6 +146,13 @@ func _ready() -> void:
|
||||
_build_overlay_bar()
|
||||
_build_legend_panel()
|
||||
|
||||
_disk_sweep_timer = Timer.new()
|
||||
_disk_sweep_timer.name = "DiskSweepTimer"
|
||||
_disk_sweep_timer.wait_time = DISK_SWEEP_INTERVAL_SEC
|
||||
_disk_sweep_timer.autostart = true
|
||||
_disk_sweep_timer.timeout.connect(_on_disk_sweep_timeout)
|
||||
add_child(_disk_sweep_timer)
|
||||
|
||||
SimBridge.step_canvas_received.connect(_on_step_canvas_received)
|
||||
|
||||
|
||||
@@ -168,6 +183,25 @@ func enter(body: Dictionary, system: Dictionary) -> void:
|
||||
grab_focus()
|
||||
queue_redraw()
|
||||
|
||||
# T-1183, D-255(d) sweep trigger (2a): "on body-open... this is the
|
||||
# natural moment ('returning to a body') where stale-by-absence entries
|
||||
# are most likely to exist" (stig-round2.md). Cheap — an index scan, no
|
||||
# bulk payload I/O beyond the deletes it triggers.
|
||||
var body_id: String = get_body_id()
|
||||
if not body_id.is_empty():
|
||||
_request.get_disk_cache().run_visit_sweep(body_id)
|
||||
|
||||
|
||||
func _on_disk_sweep_timeout() -> void:
|
||||
# T-1183, D-255(d) sweep triggers (2b + Tier 3): a coarse background
|
||||
# timer, never per-frame/per-step-cross. Only sweeps the CURRENTLY open
|
||||
# body — a body the player isn't looking at doesn't need its disk cache
|
||||
# swept on this viewer's own clock (it gets swept on its own next
|
||||
# body-open, per (2a) above).
|
||||
var body_id: String = get_body_id()
|
||||
if not body_id.is_empty():
|
||||
_request.get_disk_cache().run_background_sweep(body_id)
|
||||
|
||||
|
||||
func leave() -> void:
|
||||
pass
|
||||
@@ -181,6 +215,14 @@ func get_held_rung() -> String:
|
||||
return _held_rung
|
||||
|
||||
|
||||
## T-1183 test seam: exposes the owned StepCanvasRequest (and, through it,
|
||||
## get_disk_cache()) so sweep-trigger wiring is directly testable, matching
|
||||
## the get_cache()/get_disk_cache() accessor pattern StepCanvasRequest
|
||||
## already exposes for the same reason.
|
||||
func get_request() -> Variant:
|
||||
return _request
|
||||
|
||||
|
||||
func is_overlay_visible(overlay_id: String) -> bool:
|
||||
return bool(_overlay_visibility.get(overlay_id, false))
|
||||
|
||||
|
||||
Reference in New Issue
Block a user