Skip to content

[bug] Keep cache keys stable across processes on Python 3.14 - #282

Merged
timkpaine merged 2 commits into
mainfrom
bug/stable-cache-keys-py314-annotate
Oct 22, 2026
Merged

timkpaine merged 2 commits into
mainfrom
bug/stable-cache-keys-py314-annotate

Conversation

@ptomecek

@ptomecek ptomecek commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Two small Python 3.14 / dependency compatibility fixes.

1. Stable cache keys across processes on Python 3.14. PEP 649 attaches an auto-generated __annotate__ (and __annotate_func__) to every annotated class. ccflow's cache-key machinery computes a behavior token by hashing class method bytecode, so it hashed the __annotate__ closure, whose cells are PYTHONHASHSEED-sensitive. The behavior token, and the cache keys derived from it, then differed from one process to the next. Fix: add __annotate__/__annotate_func__ to _SKIPPED_METHODS in ccflow/utils/tokenize.py. Annotations are already captured through other paths, so excluding these is lossless for the cache contract. A regression test injects __annotate__ manually so it also runs on pre-3.14 interpreters.

2. duckdb 2.0 compatibility in the TPC-H example. duckdb 2.0 removed fetchone() from the relation returned by Connection.query(). Switch to relation.fetchall() (duckdb's recommended replacement), which exists on both old and new duckdb and fully drains the result so the shared connection stays usable for the next query. Validated against duckdb 2.0.0 and 1.4.0.

Type of Change

  • Bug fix
  • New feature
  • Documentation update
  • Refactor / code cleanup
  • CI / build configuration
  • Other (describe below)

Checklist

  • Linting passes (make lint)
  • Tests pass (make test)
  • New tests added for new functionality
  • Documentation updated (if applicable)
  • Changelog / version bump (if applicable)

PEP 649 attaches an auto-generated __annotate__ (and __annotate_func__)
to every annotated class. ccflow's behavior token hashes class method
bytecode, so it picked up the __annotate__ closure, whose cells are
PYTHONHASHSEED-sensitive. That made the behavior token, and therefore
cache keys, differ between processes.

Skip __annotate__/__annotate_func__ in the behavior token. Annotations
are already captured through other paths, so this is lossless for the
cache contract. Add a regression test that injects __annotate__ so it
runs on pre-3.14 interpreters.

Signed-off-by: Pascal Tomecek <40371786+ptomecek@users.noreply.github.com>
duckdb 2.0 removed fetchone() from the relation returned by
Connection.query(). Switch to relation.fetchall() (duckdb's recommended
replacement), which exists on both old and new duckdb and fully drains
the result, so the shared connection stays usable for the next query.

Validated against duckdb 2.0.0 and 1.4.0.

Signed-off-by: Pascal Tomecek <40371786+ptomecek@users.noreply.github.com>
@ptomecek
ptomecek force-pushed the bug/stable-cache-keys-py314-annotate branch from 16ad44c to ef7a019 Compare October 22, 2026 17:57
@github-actions

github-actions Bot commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Test Results

    1 files  ±0      1 suites  ±0   1m 59s ⏱️ -20s
1 557 tests +1  1 552 ✅  - 2  5 💤 +3  0 ❌ ±0 
1 623 runs  +1  1 618 ✅  - 2  5 💤 +3  0 ❌ ±0 

Results for commit ef7a019. ± Comparison against base commit ea121bd.

This pull request skips 4 and un-skips 1 tests.
ccflow.tests.enums.test_pydantic ‑ test_json_schema_csp
ccflow.tests.examples.test_calculator.TestCalculatorCli ‑ test_cli
ccflow.tests.examples.test_etl.TestEtl ‑ test_cli
ccflow.tests.utils.test_tokenize.TestAdversarialFixes ‑ test_function_docstring_still_stripped
ccflow.tests.enums.test_pydantic ‑ test_json_schema_no_csp

♻️ This comment has been updated with latest results.

@codecov

codecov Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 73.33333% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.84%. Comparing base (ea121bd) to head (ef7a019).

Files with missing lines Patch % Lines
ccflow/tests/utils/test_tokenize.py 75.00% 3 Missing ⚠️
ccflow/examples/tpch/data_generators.py 66.66% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #282      +/-   ##
==========================================
- Coverage   93.87%   93.84%   -0.03%     
==========================================
  Files         202      202              
  Lines       22457    22469      +12     
  Branches     1457     1457              
==========================================
+ Hits        21081    21086       +5     
- Misses       1091     1096       +5     
- Partials      285      287       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@timkpaine
timkpaine merged commit 50448ba into main Oct 22, 2026
18 of 20 checks passed
@timkpaine
timkpaine deleted the bug/stable-cache-keys-py314-annotate branch October 22, 2026 18:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants