lcqdrag

A production-grade, Temporal-orchestrated Retrieval-Augmented Generation (RAG) chatbot. Orchestrates LLM-based rephrasing, vector retrieval, reranking, and answer generation as durable Temporal activities so that any mid-pipeline failure (LLM SDK hang, transient network error, worker restart mid-call) is retried or gracefully-failed by the workflow engine — not by hand-written retry/timeout/exception code.

Stack:

• Temporal (Python SDK 1.32, dockerized auto-setup) — durable workflow engine, task queue, per-activity retry policies, typed non-retryable application errors, workflow execute_update() for synchronous chat turns.
• Groq hosted LLMs via langchain-groq (ChatGroq) for the two LLM hops per turn (standalone question rephrasing + final grounded QA).
• Ollama native (host localhost:11434, bge-large model) for 1024-dim embeddings.
• Qdrant (dockerized) for vector storage, cosine similarity search, point deduplication via content UUIDs.
• LangChain components: RecursiveCharacterTextSplitter, ChatPromptTemplate, OllamaEmbeddings, QdrantVectorStore, CrossEncoderReranker.

Design invariants (see workflow.py and activities.py):

1. Workflow never crashes mid-thread. Each of the 3 activities (contextualize / retrieve / generate) has its own try/except ApplicationError(typed=XxxError, non_retryable=True) so a failure on turn N just returns SendMessageResult(answer=NO_INFO_RESPONSE, error=…) — turn N+1 still works against the same conversation-workflow-session_<id>.
2. Every async activity has a hard wall-clock guard via asyncio.wait_for(_body(), timeout=<per-activity int>) inside the activity itself. ChatGroq / Ollama transport stalls that would otherwise be a "retryable activity timeout" → stacked into 3 retries × 60 s = 180 s → client rpc_timeout fire, are instead converted inside the activity to typed, non-retryable ApplicationError. One hang → graceful fallback in <90 s total.
3. Chat-turn chat history only stores SUCCESSFUL turns. Skipped retrieval, retrieval-empty, or generation-failure turns don't append I am sorry, but I do not have that information. as "AI output" because doing so poisons the next turn's standalone-question rephrasing.
4. Worker auto-starts inside CLI. Running python3 main.py uses sys.executable + subprocess.Popen(start_new_session=True) to boot its own python worker.py as a child and reap it on exit. No "open two terminals" setup required for the default dev UX.
5. Ingestion tolerates single-page doc-host SPAs (Mintlify etc). When {url}/sitemap.xml returns HTML 200 instead of XML (common on hosted docs), the loader falls back to extracting same-host <a href> links from the landing page and fetching up to 30 of them — enough to cover full navigation for typical product docs.
6. Continue-as-new every N turns. Conversation workflows bound their own event history by self-terminating and restarting with carryover_history, keeping each workflow run's event count under Temporal's 50k event ceiling even for very long chat threads.

---

Quick start

1. Prerequisites

• Docker 25+ with Compose v2 (tested on Docker 29.7.2 + Compose v5.5.0).
• Python 3.12+ (tested on 3.13) + a virtualenv at ./.venv.
• Ollama installed natively on the host (default dev UX uses host ollama, not a container — avoids port conflicts and lets you reuse already-pulled models).
  • Pull once: ollama pull bge-large
• A Groq API key (any Groq-enabled account works; tested with a key that has qwen/qwen3.8-27b enabled).

2. Install deps

cd lcqdrag
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r <(cat pyproject.toml | python3 -c 'import tomllib,sys;print("\n".join(tomllib.load(sys.stdin)["project"]["dependencies"]))')
# OR, if you have `uv` installed:
uv sync

3. Configure .env

cp .env.example .env
# edit .env:
#   GROQ_API_KEY=gsk_your_real_key
#   LLM_MODEL=qwen/qwen3.8-27b     (use `Groq().models.list()` to pick your own)
#   QDRANT_URL=http://localhost:6333
#   TEMPORAL_HOST=localhost:7233
#   INGESTION_URL=https://docs.smith.langchain.com/

4. Start infrastructure

docker compose up -d
# Expected (first run):
#   Network lcqdrag_lcqdrag-net   Created
#   Container lcqdrag-postgres    Healthy   (~5 s)
#   Container lcqdrag-qdrant      Healthy   (~15 s; uses bash /dev/tcp probe — no curl required in image)
#   Container lcqdrag-temporal    Healthy   (~5 s)
#
# Default profile does NOT start ollama (collides with host ollama on :11434)
# or the Python worker (main.py boots its own native worker as a subprocess).
#
# To start them explicitly when needed:
#   docker compose --profile with-ollama --profile worker up -d

5. Run the chat CLI

source .venv/bin/activate
python3 main.py

On startup the CLI:

1. Boots its own native python worker.py subprocess (stdout/stderr → .logs/worker.log, reaped on SIGINT/SIGTERM).
2. Connects to Temporal on localhost:7233 and ensures the default namespace default exists.
3. Runs AcceptanceIngestDriverWorkflow against INGESTION_URL to populate the Qdrant collection langsmith_docs with 30 crawled doc pages → ~114 deduplicated 1000-char chunks on first run.
4. Drops you into the chat loop; type history / exit:

============================================================
  LCQDRAG — Production RAG Chatbot (Temporal-orchestrated)
  Thread ID: session_<id>
  Type 'exit' to quit, 'history' to see chat history
============================================================

Ask: What is LangSmith?
Answer:
 LangSmith is a framework-agnostic platform designed for testing, deploying,
 monitoring, and improving AI agents across the agent development lifecycle...

Ask: What does tracing mean in it?
Answer:
 In LangSmith, tracing is the process of recording and organizing the execution
 of an agent's operations to monitor and analyze its performance. It involves
 the following key components: Runs / Traces / Threads / Trajectories ...

---

Useful commands

# --- Infrastructure
docker compose up -d            # postgres + qdrant + temporal only (default)
docker compose --profile with-ollama --profile worker up -d   # everything containerized
docker compose down

docker ps                       # expect 3 Healthy: postgres/qdrant/temporal
docker logs lcqdrag-qdrant
docker logs lcqdrag-temporal | grep -i "namespace"            # default namespace registration

# --- CLI / runtime
tail -f .logs/worker.log                                  # follow worker activity output
python3 main.py <<< 'What is LangSmith?'$'\nexit'         # non-interactive one-shot

# --- Diagnostics
# Re-enable the "(contextualized as: ...)" debug line we removed from default output:
VERBOSE_CONTEXTUALIZE=1 python3 main.py

# --- Acceptance / evaluation (see run_acceptance_tests.py + eval.py)
source .venv/bin/activate
python3 run_acceptance_tests.py      # kill-worker-replay, double-ingest idempotent, invalid-Groq-key graceful
python3 eval.py                       # RAG correctness evaluation harness → eval_results.json (gitignored)

---

Project layout

lcqdrag/
├── main.py                    # CLI client: boot worker, start ingestion-driver workflow, chat loop via execute_update()
├── worker.py                  # Temporal Worker process + registered workflows & activities
├── workflow.py                # ConversationWorkflow.send_message_update (queue/runner) + continue-as-new + try/except graceful fallbacks
├── activities.py              # 4 Temporal activities: contextualize / retrieve / generate / ingest_documents
│                                + sitemap-aware, SPA-tolerant web_base_loader + bge-large embeddings + qdrant upsert
├── test_drivers.py            # AcceptanceIngestDriverWorkflow + poison-worker test workflows
├── run_acceptance_tests.py    # 3 acceptance scenarios harness
├── eval.py                    # RAG eval harness → eval_results.json (gitignored)
├── config.py                  # Pydantic env-loaded settings (RetrievalConfig / TimeoutConfig / QdrantConfig / TemporalConfig…)
├── docker-compose.yml         # Postgres + Temporal + Qdrant; optional `with-ollama` & `worker` profiles
├── Dockerfile                 # Production image for the worker
├── pyproject.toml             # Pinned python deps + entrypoints
├── .env.example               # Copy to .env for local run
├── .gitignore                 # Excludes venv, logs, .env, scratch MD files, run artifacts
└── README.md                  # This file

---

Environment variables

Env name	Default	Purpose
GROQ_API_KEY	(required)	ChatGroq client auth
LLM_MODEL	qwen/qwen3.8-27b	Chat-completion model for both rephrase + QA hops. Use Groq().models.list() to find a valid one for your key.
EMBEDDING_MODEL	bge-large	Ollama model name pulled on the host.
OLLAMA_HOST	http://localhost:11434	Embedding endpoint (point at http://host.docker.internal:11434 when running the containerized worker profile).
QDRANT_URL	http://localhost:6333	Qdrant REST endpoint
QDRANT_COLLECTION_NAME	langsmith_docs	Vector collection in Qdrant
TEMPORAL_HOST	localhost:7233	Temporal gRPC host
TEMPORAL_NAMESPACE	default	Temporal namespace for workflows & tasks
TEMPORAL_TASK_QUEUE	lcqdrag-task-queue	Task queue name — worker + main.py must match
INGESTION_URL	https://docs.smith.langchain.com/	Documentation site to seed the vector store with on startup.
TIMEOUT_CONTEXTUALIZE / RETRIEVE / GENERATE / RERANK	30, 10, 60, 30 (seconds)	Per-activity wall-clock hard caps inside each Temporal activity's asyncio.wait_for guard.
RETRIEVAL_SIMILARITY_THRESHOLD	0.25	Cosine-similarity floor below which retrieval short-circuits (kept intentionally permissive — let the reranker + QA LLM filter).
RETRIEVAL_TOP_K_INITIAL / FINAL	15, 4	Candidate vectors → reranker → final context window.
ENABLE_RERANKING	true	Toggles sentence-transformers ms-marco-MiniLM-L-6-v2 rerank.
VERBOSE_CONTEXTUALIZE	0	Set to 1 to re-enable the "(contextualized as: …)" debug line on the CLI. Useful for debugging a bad pronoun resolution.
DEBUG_SLOW_CONTEXTUALIZE_SECONDS	0	Acceptance-test injector; forces contextualize_activity to sleep N seconds to exercise the timeout → non-retryable ApplicationError guard path.

---