Repository navigation
Add RAG layer: semantic search + grounded explanations - #13
Conversation
Unified retrieval layer (Gemini-generated style cards + text-embedding-004, in-memory NumPy cosine) powering semantic search and grounded explanations. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Rate-limit errors back off exponentially (honoring the API's retry_delay hint when present) instead of dropping the player; other transient errors back off briefly. Re-raises after MAX_RETRIES so a persistently-failing call still surfaces. Adds hermetic tests for the retry logic. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The account's key 404s on gemini-2.5-flash ("no longer available to new
users") and text-embedding-004 (not served). Switch to gemini-flash-latest
for generation and gemini-embedding-001 for embeddings. Centralize both ids
in config as the single source of truth so the offline document build and the
runtime query embedding can never drift apart (a mismatch breaks cosine).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Generated via scripts/build_rag with gemini-flash-latest (cards) and gemini-embedding-001 (3072-dim embeddings). 585/585 players covered, no empties. Consistent with the other precomputed files in exports/. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Artifacts generated + committed ✅The RAG artifacts are now built and included in the branch:
Model migration (required): the target account 404s on the originally-specced Live sanity check (real embeddings,
Note on repo size: the embeddings file is 24 MB (3072-dim vectors). Committed per decision, consistent with other precomputed files in |
The comparison explanation now feeds each player's real numbers into the prompt — absolute counting stats (PPG/APG/RPG, goals/assists) plus their top percentile ranks — and instructs the model to wrap every stat/number/metric in **double asterisks**. The frontend already renders **bold** via formatBullet; fixed its parenthetical-highlight rule so it no longer swallows an already- bolded "(AST **top 7%**)" and leaks literal asterisks. Also hardened _parse_bullets so stripping the leading bullet marker can't eat the opening ** of a bullet that starts with a bolded metric. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Summary
Adds a unified retrieval layer to the CrossOver backend, powering two features from one precomputed artifact set:
GET /search?mode=semanticembeds a natural-language query and ranks players by cosine similarity over precomputed style-card embeddings. Falls back to the existing substring search when unavailable./explainnow injects each player's retrieved style card into the Gemini prompt, so bullets reference concrete stylistic facts instead of paraphrasing raw attribute numbers. (Replaces the previously-unusedstats_a/stats_bparams.)Everything degrades gracefully: with no artifacts or no
GEMINI_API_KEY, the app behaves exactly as before.How it works
backend/scripts/build_rag.pygenerates a ~3-sentence style card per player from their raw stat line (gemini-2.5-flash) and embeds it (text-embedding-004), writingexports/style_cards.json+exports/style_embeddings.json(idempotent/resumable, atomic writes).data_storeloads these once at startup into an L2-normalized NumPy matrix (same in-memory approach assim_matrix.csv— no vector DB; ~585 players). Resilient to both missing and corrupt artifacts.services/rag.pyembeds queries (lru_cached) and ranks by cosine.Design & plan
docs/superpowers/specs/2026-07-29-rag-layer-design.mddocs/superpowers/plans/2026-07-29-rag-layer.mdTesting
New hermetic tests for the data layer, rag service, semantic search, explanation grounding, and the build script's prompt builder. Full backend suite green except 3 pre-existing auth tests that require a local PostgreSQL (unrelated to this branch).
Follow-up (not in this PR)
python -m scripts.build_rag). Currently pending — the available API key is rate/quota limited. Until the artifacts exist, semantic search falls back to keyword and explanations stay ungrounded (by design).🤖 Generated with Claude Code