mirror of
https://github.com/pewdiepie-archdaemon/odysseus.git
synced 2026-10-06 15:02:20 +02:00
fix(search): bound and police the scholarly metadata lookups
The arXiv and OpenAlex title resolvers called two hardcoded endpoints with a hand-written "Odysseus/0.20" User-Agent, no outbound-URL policy, and a full 12s timeout each. APP_VERSION was already 1.0.3, so the agent string was wrong the moment it was written, and a scholarly query could hold a user-facing search open for the sum of all three hops. Move the endpoints to named constants, build the User-Agent from APP_VERSION, run both calls through check_outbound_url (they follow redirects, so the final host is not the one in the constant), and give the whole SearXNG -> OpenAlex -> arXiv chain one shared wall-clock budget. The budget is a ContextVar rather than a parameter so the existing test doubles for _openalex_title_results and _arxiv_title_results keep working unchanged.
This commit is contained in:
@@ -140,6 +140,18 @@ LLM_HOSTS = [h.strip() for h in os.getenv("LLM_HOSTS", "").split(",") if h.strip
|
||||
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
|
||||
SEARXNG_INSTANCE = os.getenv("SEARXNG_INSTANCE", "http://localhost:8080")
|
||||
|
||||
# Scholarly title resolution. These are the only third-party metadata APIs the
|
||||
# search path calls directly, so they get named constants rather than literals
|
||||
# repeated at each call site. The budget bounds the whole SearXNG -> OpenAlex ->
|
||||
# arXiv chain: each hop used to get its own full timeout, so one scholarly query
|
||||
# could stall a user-facing search for the sum of all three.
|
||||
ARXIV_API_URL = "https://export.arxiv.org/api/query"
|
||||
OPENALEX_API_URL = "https://api.openalex.org/works"
|
||||
SCHOLARLY_LOOKUP_TIMEOUT = float(os.getenv("ODYSSEUS_SCHOLARLY_LOOKUP_TIMEOUT", "12"))
|
||||
SCHOLARLY_LOOKUP_TOTAL_BUDGET = float(
|
||||
os.getenv("ODYSSEUS_SCHOLARLY_LOOKUP_BUDGET", "20")
|
||||
)
|
||||
|
||||
# Cleanup configuration
|
||||
CLEANUP_ENABLED = os.getenv("CLEANUP_ENABLED", "True").lower() == "true"
|
||||
CLEANUP_INTERVAL_HOURS = int(os.getenv("CLEANUP_INTERVAL_HOURS", "24"))
|
||||
|
||||
Reference in New Issue
Block a user