Repository navigation

Modularity-eval support: pipeline-parallel GPU planning, namespaced dataset ids, launcher runtime hooks - #558

Merged

AkshitaB merged 10 commits into

mainfrom

akshitab/modularity-evals

Nov 12, 2026

Merged

Modularity-eval support: pipeline-parallel GPU planning, namespaced dataset ids, launcher runtime hooks#558
AkshitaB merged 10 commits into
mainfrom
akshitab/modularity-evals

Commits on Nov 12, 2026
- beaker launcher: resolve provider build dependencies from PyPI, with the runtime Torch index as an extra index
  
  authored and
  AkshitaB
  committed
  79840fe
- beaker launcher: OLMO_EVAL_RUNTIME_TORCH_EXTRA_PACKAGES installs torchvision/torchaudio builds alongside the runtime torch override (needed for cu130 on B300)
  
  ryanyxw
  authored and
  AkshitaB
  committed
  9958c33
- beaker launcher: forward OLMO_EVAL_RUNTIME_TORCH_EXTRA_PACKAGES to the install command
  
  authored and
  AkshitaB
  committed
  2257bee
- beaker launcher: OLMO_EVAL_RUNTIME_POST_INSTALL_CMD runs a shell snippet after all package installs
  
  authored and
  AkshitaB
  committed
  bf0868f
- GPU planning: a vLLM instance spans tensor_parallel_size x pipeline_parallel_size GPUs
  
  authored and
  AkshitaB
  committed
  490864c
- Use namespaced Hugging Face dataset ids for GSM8K, CSQA, PIQA, SocialIQA, WinoGrande and HumanEval
  
  authored and
  AkshitaB
  committed
  81437df
- Inference workers: tensor_parallel_size is the worker's GPUs divided by pipeline_parallel_size
  
  authored and
  AkshitaB
  committed
  c18000c
- Auxiliary vLLM servers: each instance starts on tensor_parallel_size x pipeline_parallel_size GPUs
  
  AkshitaB
  and
  claude
  committed
  fdf3f56
- Result compatibility log: namespaced dataset ids change 60 task hashes
  
  AkshitaB
  and
  claude
  committed
  6bc0fb5
- ruff format and line-length fixes
  
  AkshitaB
  and
  claude
  committed
  b126d85

Commits

Commits on Nov 12, 2026