This repository is a third-party reproduction of the Co-Scientist multi-agent research partner described in Accelerating scientific discovery with Co-Scientist and its Supplementary Notes.
The project now implements the main functional loop: given a research goal, it generates candidate hypotheses, grounds them with public and private literature, reviews them, ranks them through an Elo tournament, evolves stronger hypotheses when useful, and exports a final research overview.
Co-Scientist is not a single-prompt question answering system. It decomposes scientific reasoning into specialized agents:
Generationproposes candidate scientific hypotheses.Reflectionreviews hypotheses against literature evidence.Proximitytracks similarity between hypotheses.Rankingcompares hypotheses through an Elo tournament.Evolutionimproves top hypotheses without replacing the originals.Meta-reviewsummarizes system-level feedback and produces the final overview.
The Supervisor schedules these agents through a persistent task queue. All
intermediate state is stored in SQLite, so runs can be resumed, inspected, and
exported.
- OpenAI-compatible LLM client for OpenAI, DeepSeek, Qwen, Kimi, and any
compatible provider;
extra_headerssupport for providers that require custom request headers. - Dedicated embedding provider: set
llm.embedding_providerto route embedding calls to a different provider than the chat LLM (e.g. Gitee Qwen3-Embedding while using DeepSeek for chat). - Layered YAML configuration: default config, local config, session config, and CLI overrides.
- SQLite context memory for sessions, research plans, hypotheses, reviews, matches, tasks, citations, feedback, overview, and private-corpus chunks.
- Multi-source literature search pipeline — PubMed, Semantic Scholar, arXiv, OpenAlex, and Tavily web search — with per-source enable flags and Reciprocal Rank Fusion result merging.
- Private literature corpus support for local Markdown/txt directories, with chunking, caching, embedding retrieval, and lexical fallback.
- Elo tournament ranking with automatic pair selection.
- Self-improvement loop: meta-review feedback is injected into later agent prompts, and evolution-generated hypotheses re-enter review/ranking.
- Scientist-in-the-loop CLI: manual reviews, user-contributed hypotheses, goal revision, resume, status, tail, and export.
- Observability: each session writes
runs/<session_id>/metrics.jsonlwith task, LLM, error, and Elo/checkpoint events. - Offline test suite using mocks/stubs only; no API keys or network are needed for tests.
config/
default.yaml # Runtime, search, LLM, and observability defaults
local.yaml.example # Machine-local override example
prompts/ # Agent prompt templates
doc/
README.zh.md # Chinese README
src/co_scientist/
agents/ # Generation, Reflection, Ranking, etc.
llm/ # OpenAI-compatible client/router
memory/ # SQLite schema/store/models/Elo
search/ # Multi-source search pipeline (PubMed, OpenAlex, etc.)
supervisor/ # Main loop, task queue, stats, metrics
tools/ # Literature search, private corpus, tool models
tests/ # Unit tests and end-to-end smoke test
Use Python 3.11 or newer.
python -m pip install -e ".[dev]"Check the CLI:
co-scientist --helpRun tests:
python -m pytest
python -m ruff check src testsConfiguration is loaded in this order, from lowest to highest precedence:
config/default.yamlconfig/local.yaml, for machine-local settings and ignored by git--session-config <yaml>, for one research session- CLI overrides such as
--max-ideas
Create a local config:
cp config/local.yaml.example config/local.yamlExample:
llm:
default_provider: deepseek
# Optional: use a separate provider for embeddings only.
# embedding_provider: gitee
providers:
deepseek:
api_key: sk-your-key-here # direct value
# api_key_env: DEEPSEEK_API_KEY # or read from this env var name
base_url: https://api.deepseek.com
chat_model: deepseek-chat
temperature: 0.3
# Example embedding-only provider with a custom request header:
# gitee:
# api_key: your-gitee-key
# base_url: https://ai.gitee.com/v1
# chat_model: Qwen3-Embedding-8B
# embedding_model: Qwen3-Embedding-8B
# extra_headers:
# X-Failover-Enabled: "true"
search:
tavily_enabled: true
pubmed_enabled: true
semantic_scholar_enabled: true
arxiv_enabled: true
openalex_enabled: true
private_corpus_enabled: true
private_corpus_paths:
- /absolute/path/to/my/literature-markdownapi_key vs api_key_env: use api_key to embed the key value directly
in the YAML file (recommended for local.yaml, which is git-ignored). Use
api_key_env to name an environment variable that holds the key (e.g.
api_key_env: DEEPSEEK_API_KEY reads $DEEPSEEK_API_KEY).
Common environment variables (when using api_key_env):
OPENAI_API_KEY/OPENAI_BASE_URLDEEPSEEK_API_KEY/DEEPSEEK_BASE_URLQWEN_API_KEY/QWEN_BASE_URLKIMI_API_KEY/KIMI_BASE_URLTAVILY_API_KEYNCBI_EMAIL/NCBI_API_KEYSEMANTIC_SCHOLAR_API_KEY
The default test suite does not require external API keys. Real runs will have better grounding when literature-search credentials are configured.
A session follows this high-level flow:
Plannerparses the natural-language goal into aResearchPlan.Generationcreates up toinitial_ideasinitial hypotheses with multiple strategies.- Each hypothesis enters
Reflection.full_review, which searches public and private literature and assigns a review score. Proximitycomputes hypothesis similarity.Rankingselects hypothesis pairs, asks the LLM to compare them, and updates Elo ratings.- When Elo stagnates or the match target is reached,
Meta-reviewcreates system feedback. - If
max_ideashas not been reached,Evolutioncreates new hypotheses from top-ranked candidates. - Once idea and match targets are reached,
Meta-reviewgenerates the final research overview. - The CLI exports the report as Markdown or NIH Specific Aims style text.
All state is persisted in SQLite. The default database path is:
runs/co_scientist.sqlite
Create a goal file:
cat > goal.txt <<'EOF'
What impact did the selection pressure that caused the differentiation between
cannabis in high latitudes of China and cannabis in low latitudes play in the
differentiation of cannabis in high latitudes to European fiber and the
differentiation of cannabis in low latitudes to medicinal use in South Asia?
EOFStart a new session:
co-scientist new goal.txt --initial-ideas 5 --max-ideas 8 --max-matches-per-idea 2 --verboseShow status:
co-scientist status <session-id>Tail new hypotheses and matches:
co-scientist tail <session-id> --followResume pending work:
co-scientist resume <session-id>Export reports:
co-scientist export <session-id> -o runs/report.md
co-scientist export <session-id> --format nih-aims -o runs/aims.mdAdd a manual expert review:
co-scientist review <hypothesis-id> --score 8.5 --comment "Strong mechanism, needs better controls."Contribute a user hypothesis:
co-scientist contribute <session-id> --file my_hypothesis.md
co-scientist resume <session-id>Revise the goal and re-review existing hypotheses:
co-scientist revise-goal <session-id> updated_goal.txt --force
co-scientist resume <session-id>The first private-corpus implementation supports Markdown and txt files. It is intended for literature exported from Zotero, MinerU, or a manually curated text directory.
Configuration:
search:
private_corpus_enabled: true
private_corpus_paths:
- /absolute/path/to/doc/LLM-for-Zotero-MinerU-supplementary
private_corpus_max_results: 3
private_corpus_chunk_chars: 1600
private_corpus_chunk_overlap: 200Runtime behavior:
- Scans
.mdand.txtfiles. - Splits files into fixed-size overlapping chunks.
- Stores chunk text, mtime, file size, and hash in SQLite.
- Skips re-indexing unchanged files.
- Uses embedding retrieval when an embedding model is available.
- Falls back to lexical search when embedding fails, returning a
DEGRADEDtool status.
Private corpus results are merged into the same evidence pack as PubMed,
Semantic Scholar, arXiv, and Tavily results for Reflection.full_review.
Each session may produce:
- SQLite state:
runs/co_scientist.sqlite - JSONL metrics:
runs/<session_id>/metrics.jsonl - Optional exported reports from
co-scientist export
Common metrics.jsonl events:
session.startsession.resumesession.donetask.starttask.donetask.failedllm.chat
These events help debug failed tasks, LLM token/latency behavior, ranking progress, and final overview generation.
- For the first real run, start small:
--max-ideas 5 --max-matches-per-idea 1. - If public search is unstable, disable Tavily or Semantic Scholar and use PubMed/private corpus first.
- Chinese goals are passed through as-is into English prompts; most compatible models handle this, but English goals tend to be more stable.
- Keep
metrics.jsonland exported reports for long sessions so different configurations can be compared later.
Completed:
- Main MVP functionality for Phases 0-7.
- Phase 8.3 end-to-end smoke test.
- Phase 9.2 private literature corpus.
- Phase 9.3 observability.
- Multi-source search pipeline with OpenAlex, RRF fusion, and per-source config flags.
- Dedicated embedding provider support (
llm.embedding_provider).
Deferred or lower priority:
- Phase 8.1 Elo time-bucket plotting.
- Phase 8.2 LLM-as-judge preference evaluation.
- Phase 9.1 hard safety filtering. Safety is currently evaluated in prompts and reviews, but not used as a hard gate.
- Phase 9.4 dedicated performance optimization. The project already has caching and some batching, but no full profiling pass yet.
- PDF extraction, Zotero API integration, OpenTelemetry, Web UI, and multi-user access control.