Skip to content

Fix FSDP parameter counts and HF generation compatibility - #916

Open
avinashgola wants to merge 1 commit into
allenai:mainfrom
avinashgola:codex/fix-olmo-695-898-854
Open

avinashgola wants to merge 1 commit into
allenai:mainfrom
avinashgola:codex/fix-olmo-695-898-854

Conversation

@avinashgola

Copy link
Copy Markdown

Summary

  • count FSDP flat parameters using their unpadded unsharded size when available, avoiding padded shard inflation in parameter/FLOP accounting
  • mark hf_olmo as using legacy tuple caches and accept newer generation kwargs from Transformers, including cache_position and position_ids
  • document the offline OLMES path for evaluating converted local/custom checkpoints, including GSM8K-style generative evals

Fixes #695.
Addresses #898 and #854.

Testing

  • python -m py_compile olmo/model.py hf_olmo/modeling_olmo.py tests/model_test.py
  • ruff check olmo/model.py hf_olmo/modeling_olmo.py tests/model_test.py

Not run: pytest, because this local environment is missing torch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Gflops computation is faulty for FSDP due to bug in OLMo.num_params()

1 participant