Repository navigation
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Long public query composition chains can overflow the stack during execution.
1 open issue
What changed in this PR
Introduces reusable Kyo query plans to batch requests across composed relations while preserving output positions.
Changes:
- Groups independent requests by dependency stage.
- Migrates interpreter execution to query plans and documents compatibility.
- Adds batching regression tests and test dependencies.
| File | Description |
|---|---|
| kyo-batch/src/test/scala/decrel/reify/KyoBatchSpec.scala | Tests batching, ordering, composition, and plan reuse. |
| kyo-batch/src/main/scala/decrel/reify/KyoBatchQuery.scala | Implements query composition and staged execution. |
| kyo-batch/src/main/scala/decrel/reify/kyoBatch.scala | Integrates query plans into the interpreter. |
| kyo-batch/README.md | Documents batching semantics and migration. |
| build.sbt | Adds Kyo module test dependencies. |
Review effort: Balanced
Comment on lines
+23
to
+24
| def step: Step[S, B] = KyoBatchQuery.this.step match { | ||
| case Done(value) => f(value).step |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Kyo datasource implementations ran
Batch.runseparately for each relation invocation.(Lookup & Lookup)therefore fetched the same keys twice, and derived many/optional relations issued separate batches for each outer input.Build an immutable
KyoBatchQuerythrough composition and submit all ready requests in each dependency stage to Kyo Batch together. Requests remain associated with their output positions even when Kyo groups different sources in a different order. Overlapping groups such as[[1, 2], [2, 3]]now produce one callback with[1, 2, 3]while preserving both groups in the result.This PR is stacked on #396 and uses its typed
zipWithprimitive. The diff contains only the Kyo batching change and its tests/documentation.Compatibility:
startingFrom,startingFromMany, and datasource callback signatures are unchanged. Low-levelreify.apply/applyMultiplenow return a reusable query plan; call.runfor the Kyo effect. Custom reified implementations can useKyoBatchQuery.lift. The module README explains the migration. Deduplication is per dependency stage; this does not add a persistent cache or guarantee concurrent datasource effects.Validation: 14 regression tests on JVM and Scala.js cover zipped branches, overlapping groups, duplicate/empty inputs, optional and many composition, mixed sources, dependent stages, custom datasources, typed failures, rerunning plans, and 4,096-input bulk traversal. Formatting and generated workflow checks pass.
A separate pre-existing core proof-derivation issue was reproduced with the original Kyo interpreter: a composed relation with identical input/output types can resolve to an identity proof and bypass its datasources. This PR leaves proof derivation unchanged; the dependent-stage regression uses distinct input/output types.