fix: correct /stats wiki page count and Wiki.js listing pagination

- /stats passed the bare user name as path prefix (matched nothing, always
  reported 0 of 138 pages); it now counts pages under users/{user}
- list_pages applied the API-side limit before client-side path/tag filters,
  dropping matching pages that sort late; limit now applies after filtering
- list_all_pages replaced the fake while/break pagination with a real
  limit-growth loop: Wiki.js 2.x pages.list supports only a limit argument
  (no offset - verified via GraphQL introspection), so the client doubles
  the limit until the API returns fewer pages than requested

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-07-14 10:03:13 +02:00
co-authored by Claude Fable 5
parent 5c2284922d
commit 54204599a9
3 changed files with 81 additions and 36 deletions
+2
View File
@@ -15,6 +15,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Ollama generation model env collision** - Renamed the generation-model setting `ollama_model` to `ollama_llm_model` (env: `OLLAMA_LLM_MODEL`, default `gemma4:e2b`). The container env `OLLAMA_MODEL=nomic-embed-text` (meant for embeddings) was shadowing the generation model, breaking Phase 0 keyword extraction and Phase 4 LLM re-ranking on every request. Startup now logs the resolved generation model.
- **LLM call timeouts** - Phase 0 keyword extraction and Phase 4 re-ranking are wrapped in a 12s `asyncio.wait_for` with graceful fallback, so a hung Ollama call can no longer gate retrieval for the full 120s client timeout.
- **`/stats` wiki page count** - The endpoint passed the bare user name as path prefix (matching nothing) and always reported 0 pages; it now counts pages under `users/{user}`.
- **Wiki.js page listing** - `list_pages` applied the API-side `limit` before client-side path/tag filters, dropping matching pages that sort late; the limit now applies after filtering. `list_all_pages` replaced its fake pagination loop with a real limit-growth loop (Wiki.js 2.x `pages.list` has no offset argument) that fetches until the API returns fewer pages than requested.
## [1.7.3] - 2026-01-07
+77 -34
View File
@@ -96,24 +96,15 @@ class WikiJSClient:
logger.error(f"GraphQL query failed: {e}", exc_info=True)
raise
async def list_pages(
self,
path_prefix: str = "",
tags: Optional[List[str]] = None,
limit: int = 50
) -> List[Dict[str, Any]]:
async def _fetch_pages(self, limit: int) -> List[Dict[str, Any]]:
"""
List pages with optional filtering.
Multi-tenancy: Use path_prefix to filter by user namespace.
Fetch a raw page listing from the GraphQL API (no client-side filtering).
Args:
path_prefix: Filter by path prefix (e.g., "/users/jpmschweitzer")
tags: Filter by tags (e.g., ["projects"])
limit: Maximum results
limit: Maximum pages to request from the API
Returns:
List of page objects
List of page objects with tags normalized to a list
"""
query = """
query ListPages($limit: Int, $orderBy: PageOrderBy) {
@@ -145,7 +136,39 @@ class WikiJSClient:
if "tags" not in page or page["tags"] is None:
page["tags"] = []
# Filter by path prefix (client-side if API doesn't support)
return pages
async def _fetch_all_pages(self, initial_limit: int = 100) -> List[Dict[str, Any]]:
"""
Fetch ALL pages from the GraphQL API.
The Wiki.js 2.x `pages.list` query only supports a `limit` argument
(no offset - verified via GraphQL introspection), so exhaustive
listing works by growing the limit until the API returns fewer
pages than requested.
Args:
initial_limit: Page count for the first request
Returns:
Complete list of page objects
"""
max_limit = 100_000 # Safety cap against pathological growth
limit = max(initial_limit, 1)
while True:
pages = await self._fetch_pages(limit)
if len(pages) < limit or limit >= max_limit:
return pages
limit = min(limit * 2, max_limit)
@staticmethod
def _filter_pages(
pages: List[Dict[str, Any]],
path_prefix: str = "",
tags: Optional[List[str]] = None
) -> List[Dict[str, Any]]:
"""Apply client-side path-prefix and tag filters to a page listing."""
if path_prefix:
# Normalize paths to have leading slash for consistent comparison
normalized_prefix = "/" + path_prefix.lstrip("/")
@@ -163,6 +186,38 @@ class WikiJSClient:
return pages
async def list_pages(
self,
path_prefix: str = "",
tags: Optional[List[str]] = None,
limit: int = 50
) -> List[Dict[str, Any]]:
"""
List pages with optional filtering.
Multi-tenancy: Use path_prefix to filter by user namespace.
Args:
path_prefix: Filter by path prefix (e.g., "/users/jpmschweitzer")
tags: Filter by tags (e.g., ["projects"])
limit: Maximum results (applied AFTER filtering)
Returns:
List of page objects
"""
if path_prefix or tags:
# The API limit applies before our client-side filters, so a
# small limit would drop matching pages that sort late. Fetch
# everything, filter, then apply the limit.
pages = self._filter_pages(
await self._fetch_all_pages(),
path_prefix=path_prefix,
tags=tags
)
return pages[:limit]
return await self._fetch_pages(limit)
async def list_all_pages(
self,
path_prefix: str = "",
@@ -172,34 +227,22 @@ class WikiJSClient:
"""
List ALL pages with pagination support.
Fetches pages in batches until all are retrieved.
Fetches pages in growing batches until all are retrieved (Wiki.js
`pages.list` has no offset argument), then applies filters.
Args:
path_prefix: Filter by path prefix (e.g., "users/jpmschweitzer")
tags: Filter by tags
batch_size: Number of pages per batch (max 100)
batch_size: Page count for the first request
Returns:
Complete list of page objects
"""
all_pages = []
offset = 0
while True:
# Wiki.js list doesn't support offset, but limit is enough
# since we filter client-side by path_prefix
# Just fetch a large batch
pages = await self.list_pages(
path_prefix=path_prefix,
tags=tags,
limit=1000 # Fetch up to 1000 at once
)
if not pages:
break
all_pages = pages
break # Wiki.js list doesn't paginate, so one call is enough
all_pages = self._filter_pages(
await self._fetch_all_pages(initial_limit=batch_size),
path_prefix=path_prefix,
tags=tags
)
logger.info(f"list_all_pages: found {len(all_pages)} pages (prefix: {path_prefix or 'all'})")
return all_pages
+2 -2
View File
@@ -192,10 +192,10 @@ async def stats(
logger.error(f"Failed to get Qdrant stats: {e}")
qdrant_stats = {"error": str(e)}
# Wiki.js page count
# Wiki.js page count (pages live under the user namespace, e.g. "users/jpmschweitzer/...")
wiki_pages = 0
try:
pages = await wikijs.list_all_pages(user)
pages = await wikijs.list_all_pages(path_prefix=f"users/{user}")
wiki_pages = len(pages)
except Exception as e:
logger.warning(f"Failed to get Wiki.js stats: {e}")