mirror of
https://github.com/pewdiepie-archdaemon/odysseus.git
synced 2026-10-09 08:22:19 +02:00
chore(publication): close pre-integration release blockers
This commit is contained in:
@@ -0,0 +1,37 @@
|
||||
# Runtime model catalogs
|
||||
|
||||
The shipped `hf_models.json` and `mlx_community_models.json` contain independently
|
||||
authored empty JSON lists (`[]`). They contain no copied upstream rows,
|
||||
descriptions, weights or metadata. A fresh offline installation has no catalog
|
||||
recommendations until user data has been populated.
|
||||
|
||||
Use **Rescan** in Cookbook while online (the models API accepts
|
||||
`refresh_catalog=1`). Existing discovery code fetches selected Hugging Face
|
||||
organization collections and MLX community collections into `DATA_DIR/hwfit/`:
|
||||
`hf_collection_models.json` and `mlx_community_models.json`. These runtime cache
|
||||
files retain source/fetch timestamps and model rows, with the existing 24-hour
|
||||
freshness policy. Forced refresh bypasses freshness, invalidates the merged
|
||||
in-memory catalog, and preserves the existing cache/network failure behavior.
|
||||
Previously populated caches can supply offline results; a failed cold refresh
|
||||
leaves an explicit empty-state message. Refresh does not write shipped lists.
|
||||
|
||||
Maintenance commands from the repository root:
|
||||
|
||||
```
|
||||
python scripts/add_hwfit_models.py
|
||||
python scripts/backfill_model_release_dates.py --dry-run
|
||||
python scripts/import_from_vllm_recipes.py --dry-run
|
||||
```
|
||||
|
||||
Their catalog path is `DATA_DIR/hwfit/hf_models.json`, merged before runtime
|
||||
collection caches. The add/import commands can initialize a missing runtime
|
||||
catalog; backfill requires one. These commands use Hub metadata and, for recipe
|
||||
import, vLLM recipe inputs. Runtime metadata is user data, not an approved
|
||||
redistributable snapshot. Publishing any populated catalog requires its own
|
||||
source/rights review; metadata, copied descriptions and model-weight licenses
|
||||
are distinct. Do not copy runtime results into these repository lists.
|
||||
|
||||
Tests use `tests/fixtures/hwfit_publication_models.json` through the scoped
|
||||
`tests/hwfit_publication_fixtures.py` fixture. The rows are independently authored
|
||||
synthetic test inputs. No production snapshot is required to test platform,
|
||||
quantization and GGUF behavior.
|
||||
+1
-66956
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -282,7 +282,7 @@ def reset_model_cache():
|
||||
def refresh_dynamic_catalogs(force=False):
|
||||
"""Refresh API-backed model catalogs and invalidate the merged cache.
|
||||
|
||||
The bundled JSON files remain the offline fallback. Dynamic catalogs live
|
||||
The bundled JSON lists are intentionally empty. Dynamic catalogs live
|
||||
under DATA_DIR so runtime refreshes do not dirty the source tree.
|
||||
"""
|
||||
from services.hwfit.hf_discovery import (
|
||||
@@ -324,6 +324,7 @@ def get_models():
|
||||
seen.add(name)
|
||||
rows.append(_normalize_model_entry(model))
|
||||
|
||||
_append_models(_load_model_file(model_catalog_path()))
|
||||
for model in _load_model_file(data_path):
|
||||
if not isinstance(model, dict):
|
||||
continue
|
||||
@@ -340,4 +341,6 @@ def get_models():
|
||||
|
||||
|
||||
def model_catalog_path():
|
||||
return os.path.join(os.path.dirname(__file__), "data", "hf_models.json")
|
||||
"""Mutable user catalog populated by the maintenance scripts."""
|
||||
from src.constants import DATA_DIR
|
||||
return os.path.join(DATA_DIR, "hwfit", "hf_models.json")
|
||||
|
||||
Reference in New Issue
Block a user