[core] Share one idle check across same-generation suspension signals by shalabhc · Pull Request #4893 · vercel/workflow

shalabhc · 2026-10-15T05:23:50Z

Description

Replaying a wide fan-out slowed down quadratically with the number of steps still in flight.

When a replay reaches the end of the event log, every step that has not finished signals the run's suspension (scheduleWorkflowSuspension, from step.ts). Each signal started its own scheduleWhenIdle polling loop: setTimeout(0), wait for the promise queue while deliveries are pending, repeat. Only the first signal of a generation is accepted (it advances suspensionGeneration), so the others were discarded, but only after polling until delivery went idle. With N unfinished steps this ran N loops per end-of-log pass:

500 unfinished steps scheduled 250,000 timers before the suspension fired.
A Promise.all over 1024 steps, replayed with ~60% of its 2,976 events (many steps started but not completed), took 18.8 s locally. A CPU profile showed nearly all of it in timers and GC; event matching (EventsConsumer.offer) was ~0.4 s.

Same-generation signals are equivalent, so one pending idle check now stands for all of them (pendingSuspension on the orchestrator context). The suspension produced is unchanged: it is built at idle from the invocations queue, as before.

Replay of the same 1024-step run, before → after:

Events replayed	Before	After
744	0.96 s	0.21 s
1,488	7.3 s	0.34 s
1,786	18.8 s	0.38 s
2,976	10.9 s	0.58 s

How did you test your changes?

New test in packages/core/src/step.test.ts: 500 concurrent unfinished steps produce one suspension listing all 500, with fewer than 10 timers scheduled. It fails without the fix (250,000 timers).
pnpm test in packages/core passes (3127 tests), including the replay-ordering, delivery-barrier and hook/sleep race suites that depend on suspension timing.
The timings above come from a local replay of a real 1024-step fan-out event log through replayWorkflow.

PR Checklist - Required to merge

📦 pnpm changeset was run to create a changelog for this PR (@workflow/core patch)
🔒 DCO sign-off passes
📝 Ping @vercel/workflow in a comment once the PR is ready, and the above checklist is complete

…nals Every unfinished step that reaches the end of the event log signals the same suspension, and each started its own idle-polling loop (setTimeout, wait for the promise queue, repeat). The generation guard only discarded the extra results, so a replay with many steps in flight ran one loop per step per end-of-log pass: 500 unfinished steps scheduled 250,000 timers, and a 1024-branch fan-out replayed mid-flight (1,786 events) took 18.8 s locally. Same-generation signals are equivalent (only the first is accepted, and it advances the generation), so one pending idle check now stands for all of them. The same replay takes 0.38 s. Co-Authored-By: shalabhc <shalabh.chaturvedi@vercel.com>

vercel · 2026-10-15T05:23:51Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
example-nextjs-workflow-turbopack	Ready	Preview, v0	Oct 15, 2026 5:26am UTC
example-nextjs-workflow-webpack	Ready	Preview, v0	Oct 15, 2026 5:26am UTC
example-workflow	Ready	Preview, v0	Oct 15, 2026 5:26am UTC
workbench-astro-workflow	Ready	Preview, v0	Oct 15, 2026 5:26am UTC
workbench-express-workflow	Ready	Preview, v0	Oct 15, 2026 5:26am UTC
workbench-fastify-workflow	Ready	Preview, v0	Oct 15, 2026 5:26am UTC
workbench-hono-workflow	Ready	Preview, v0	Oct 15, 2026 5:26am UTC
workbench-nestjs-workflow	Ready	Preview, v0	Oct 15, 2026 5:26am UTC
workbench-nitro-workflow	Ready	Preview, v0	Oct 15, 2026 5:26am UTC
workbench-nuxt-workflow	Ready	Preview, v0	Oct 15, 2026 5:26am UTC
workbench-python-workflow	Ready	Preview, v0	Oct 15, 2026 5:26am UTC
workbench-sveltekit-workflow	Ready	Preview, v0	Oct 15, 2026 5:26am UTC
workbench-tanstack-start-workflow	Ready	Preview, v0	Oct 15, 2026 5:26am UTC
workbench-vite-workflow	Ready	Preview, v0	Oct 15, 2026 5:26am UTC
workflow-docs	Ready	Preview, v0	Oct 15, 2026 5:26am UTC
workflow-swc-playground	Ready	Preview, v0	Oct 15, 2026 5:26am UTC
workflow-tarballs	Ready	Preview, v0	Oct 15, 2026 5:26am UTC
workflow-web	Ready	Preview, v0	Oct 15, 2026 5:26am UTC

changeset-bot · 2026-10-15T05:23:54Z

🦋 Changeset detected

Latest commit: ffd4bf9

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 17 packages

Name	Type
@workflow/core	Patch
@workflow/builders	Patch
@workflow/cli	Patch
@workflow/next	Patch
@workflow/nitro	Patch
@workflow/vitest	Patch
@workflow/web-shared	Patch
@workflow/web	Patch
workflow	Patch
@workflow/world-testing	Patch
@workflow/astro	Patch
@workflow/nest	Patch
@workflow/rollup	Patch
@workflow/sveltekit	Patch
@workflow/vite	Patch
@workflow/nuxt	Patch
@workflow/world-vercel	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

github-actions · 2026-10-15T05:24:11Z

🧪 E2E Test Results

✅ All tests passed

⚠️ Flaky E2E Tests (passed on retry)

These tests failed at least once and passed on a retry. A recurring entry here is a real race worth investigating.

hookCleanupTestWorkflow - hook token reuse after workflow completion (nuxt · local-dev / local / quickjs / stable)

🛠 Infra Events (absorbed by the harness)

Platform anomalies the e2e harness detected and worked around (e.g. a run the queue never picked up, replaced by a fresh run). Clustered timestamps indicate a backend blip; a steady drip indicates a platform issue worth escalating.

run-pickup-stall · runs app-generated source against a registered step (tanstack-start) · at 05:29:11Z · abandoned wrun_01M4Z0HWRY0EV9DP5P3600JCR1
cold-start-warmup · suite warmup (tanstack-start) · at 05:29:11Z · abandoned wrun_01M4Z0HWSNQ5F3CF200RAKQS83

E2E Test Summary

Summary

	Passed	Skipped	Total
✅ ▲ Vercel Production	3982	878	4860
✅ 💻 Local Development	4669	551	5220
✅ 📦 Local Production	4669	551	5220
✅ 🐘 Local Postgres	4669	551	5220
✅ 🪟 Windows	348	12	360
✅ 🌐 Cross-language Conformance	68	87	155
✅ dynamic-runs	0	0	0
✅ vercel-http-transport	897	183	1080
✅ vercel-multi-region	27	0	27
✅ vercel-ws-transport	607	113	720
Total	19936	2926	22862

Details by Category

✅ ▲ Vercel Production

App	Passed	Skipped
✅ astro-node	145	35
✅ astro-quickjs	145	35
✅ example-node	145	35
✅ example-quickjs	145	35
✅ express-node	145	35
✅ express-quickjs	145	35
✅ fastify-node	145	35
✅ fastify-quickjs	145	35
✅ hono-node	145	35
✅ hono-quickjs	145	35
✅ nest-node	145	35
✅ nest-quickjs	145	35
✅ nextjs-turbopack-node	172	8
✅ nextjs-turbopack-quickjs	172	8
✅ nextjs-webpack-node	172	8
✅ nextjs-webpack-quickjs	172	8
✅ nitro-node	145	35
✅ nitro-quickjs	145	35
✅ nuxt-node	145	35
✅ nuxt-quickjs	145	35
✅ python-node	66	114
✅ sveltekit-node	164	16
✅ sveltekit-quickjs	164	16
✅ tanstack-start-node	145	35
✅ tanstack-start-quickjs	145	35
✅ vite-node	145	35
✅ vite-quickjs	145	35

✅ 💻 Local Development

App	Passed	Skipped
✅ astro-stable-node	151	29
✅ astro-stable-quickjs	151	29
✅ express-stable-node	151	29
✅ express-stable-quickjs	151	29
✅ fastify-stable-node	151	29
✅ fastify-stable-quickjs	151	29
✅ hono-stable-node	151	29
✅ hono-stable-quickjs	151	29
✅ nest-stable-node	151	29
✅ nest-stable-quickjs	151	29
✅ nextjs-turbopack-canary-node	179	1
✅ nextjs-turbopack-canary-quickjs	179	1
✅ nextjs-turbopack-quickjs-snapshot	179	1
✅ nextjs-turbopack-stable-node	179	1
✅ nextjs-turbopack-stable-quickjs	179	1
✅ nextjs-webpack-canary-node	179	1
✅ nextjs-webpack-canary-quickjs	179	1
✅ nextjs-webpack-stable-node	179	1
✅ nextjs-webpack-stable-quickjs	179	1
✅ nitro-stable-node	151	29
✅ nitro-stable-quickjs	151	29
✅ nuxt-stable-node	151	29
✅ nuxt-stable-quickjs	151	29
✅ sveltekit-stable-node	170	10
✅ sveltekit-stable-quickjs	170	10
✅ tanstack-start-node	151	29
✅ tanstack-start-quickjs	151	29
✅ vite-stable-node	151	29
✅ vite-stable-quickjs	151	29

✅ 📦 Local Production

App	Passed	Skipped
✅ astro-stable-node	151	29
✅ astro-stable-quickjs	151	29
✅ express-stable-node	151	29
✅ express-stable-quickjs	151	29
✅ fastify-stable-node	151	29
✅ fastify-stable-quickjs	151	29
✅ hono-stable-node	151	29
✅ hono-stable-quickjs	151	29
✅ nest-stable-node	151	29
✅ nest-stable-quickjs	151	29
✅ nextjs-turbopack-canary-node	179	1
✅ nextjs-turbopack-canary-quickjs	179	1
✅ nextjs-turbopack-quickjs-snapshot	179	1
✅ nextjs-turbopack-stable-node	179	1
✅ nextjs-turbopack-stable-quickjs	179	1
✅ nextjs-webpack-canary-node	179	1
✅ nextjs-webpack-canary-quickjs	179	1
✅ nextjs-webpack-stable-node	179	1
✅ nextjs-webpack-stable-quickjs	179	1
✅ nitro-stable-node	151	29
✅ nitro-stable-quickjs	151	29
✅ nuxt-stable-node	151	29
✅ nuxt-stable-quickjs	151	29
✅ sveltekit-stable-node	170	10
✅ sveltekit-stable-quickjs	170	10
✅ tanstack-start-node	151	29
✅ tanstack-start-quickjs	151	29
✅ vite-stable-node	151	29
✅ vite-stable-quickjs	151	29

✅ 🐘 Local Postgres

App	Passed	Skipped
✅ astro-stable-node	151	29
✅ astro-stable-quickjs	151	29
✅ express-stable-node	151	29
✅ express-stable-quickjs	151	29
✅ fastify-stable-node	151	29
✅ fastify-stable-quickjs	151	29
✅ hono-stable-node	151	29
✅ hono-stable-quickjs	151	29
✅ nest-stable-node	151	29
✅ nest-stable-quickjs	151	29
✅ nextjs-turbopack-canary-node	179	1
✅ nextjs-turbopack-canary-quickjs	179	1
✅ nextjs-turbopack-quickjs-snapshot	179	1
✅ nextjs-turbopack-stable-node	179	1
✅ nextjs-turbopack-stable-quickjs	179	1
✅ nextjs-webpack-canary-node	179	1
✅ nextjs-webpack-canary-quickjs	179	1
✅ nextjs-webpack-stable-node	179	1
✅ nextjs-webpack-stable-quickjs	179	1
✅ nitro-stable-node	151	29
✅ nitro-stable-quickjs	151	29
✅ nuxt-stable-node	151	29
✅ nuxt-stable-quickjs	151	29
✅ sveltekit-stable-node	170	10
✅ sveltekit-stable-quickjs	170	10
✅ tanstack-start-node	151	29
✅ tanstack-start-quickjs	151	29
✅ vite-stable-node	151	29
✅ vite-stable-quickjs	151	29

✅ 🪟 Windows

App	Passed	Failed	Skipped
✅ nextjs-turbopack-node	174	0	6
✅ nextjs-turbopack-quickjs	174	0	6

✅ 🌐 Cross-language Conformance

App	Passed	Failed	Skipped
✅ python	68	0	87

✅ dynamic-runs

App	Passed	Failed	Skipped
✅ astro-vercel	0	0	0
✅ example-vercel	0	0	0
✅ express-vercel	0	0	0
✅ fastify-vercel	0	0	0
✅ hono-vercel	0	0	0
✅ nest-vercel	0	0	0
✅ nextjs-turbopack-vercel	0	0	0
✅ nextjs-webpack-vercel	0	0	0
✅ nitro-vercel	0	0	0
✅ nuxt-vercel	0	0	0
✅ sveltekit-vercel	0	0	0
✅ tanstack-start-vercel	0	0	0
✅ vite-vercel	0	0	0

✅ vercel-http-transport

App	Passed	Skipped
✅ example	145	35
✅ express	145	35
✅ hono	145	35
✅ nextjs-turbopack	172	8
✅ nitro	145	35
✅ vite	145	35

✅ vercel-multi-region

App	Passed	Failed	Skipped
✅ nextjs-turbopack	27	0	0

✅ vercel-ws-transport

App	Passed	Skipped
✅ example	145	35
✅ express	145	35
✅ nextjs-turbopack	172	8
✅ vite	145	35

📋 View full workflow run

github-actions · 2026-10-15T05:24:13Z

📊 Workflow Benchmarks

commit ffd4bf9 · Thu, 15 Oct 2026 05:38:51 GMT · run logs

Backend: vercel · app: nextjs-turbopack

Metric	Scenario	Best (ms)	P75 (ms)	P90 (ms)	P99 (ms)	Samples
TTFS	step	302 (-31%) 💚	1946 🔴 (+32%) 🔻	2165 🔴 (+41%) 🔻	2232 🔴 (+4.9%)	30
TTFS	stream	292 (-79%) 💚	1885 🔴 (+32%) 🔻	1928 🔴 (+33%) 🔻	2176 🔴 (+48%) 🔻	30
TTFS	hook + stream	705 (-56%) 💚	2299 🔴 (+33%) 🔻	2346 🔴 (+31%) 🔻	2536 🔴 (+24%) 🔻	30
Fan-out TTFS	Promise.all(100 steps)	542 (+9.3%)	765 (+22%) 🔻	775 (+3.7%)	2480 (+41%) 🔻	10
Fan-out TTLS	Promise.all(100 steps)	2506 (+29%) 🔻	4045 (-0.7%)	4105 (-0.7%)	7852 (+29%) 🔻	10
STSO	1020 steps (inline)	99 (+7.6%)	130 (+12%)	151 (+2.0%)	208 (-60%) 💚	1019
WO	1020 steps	131109 (+2.5%)	131109 (+2.5%)	131109 (+2.5%)	131109 (+2.5%)	1

📈 STSO distribution vs main (inline / queue-hop histograms)

1020 steps (inline)

Cumulative STSO time: main 127649ms → this run 130836ms (Δ +3187ms, +2%)

 50-100 ms  ┃██                       main 113  this   1  -112
100-150 ms  █████████████████████░░┃  main 810  this 908   +98
150-200 ms  █░┃                       main  48  this  95   +47
200-250 ms  ┃                         main  11  this  10    -1
250-300 ms  ┃                         main   2  this   3    +1
300-350 ms  ┃                         main   6  this   0    -6
350-400 ms  ┃                         main   7  this   1    -6
400-450 ms  ┃                         main   6  this   0    -6
450-500 ms  ┃                         main   4  this   0    -4
500-550 ms  ┃                         main   7  this   1    -6
550-600 ms  ┃                         main   2  this   0    -2
700-750 ms  ┃                         main   1  this   0    -1
750-800 ms  ┃                         main   1  this   0    -1
800-850 ms  ┃                         main   1  this   0    -1

ℹ️ Metric definitions & methodology

_{The collapsed STSO distribution section above buckets every step gap, split inline (same warm process — pure framework overhead) vs queue-hop (fresh process — dispatch, reinit, replay). █ = main, ┃ = this run, ░ = fill.}

_{Best/P75/P90/P99 deltas compare against the most recent benchmark run on main at the time of this run. 🔻 flags a delta worse than +15%, 💚 one better than −15%.}

_{Metrics — TTFS: time to first step body (in-deployment start() → first step body) · Fan-out TTFS: fan-out time to first step (in-deployment start() → first of the parallel step bodies to complete) · Fan-out TTLS: fan-out time to last step (in-deployment start() → last of the parallel step bodies to complete, i.e. when the Promise.all resolves) · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (whole-run time outside step bodies, in-deployment anchored)}

_{Scenarios — step: one trivial no-op step, no stream; no hooks, so the run stays in turbo mode (in-process fast path) · stream: one streaming step; no hooks, so the run stays in turbo mode (in-process fast path) · hook + stream: registers a hook before one step, which exits turbo mode (dispatch path) · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges, and WO is the whole-run overhead outside step bodies · Promise.all(100 steps): 100 trivial no-op steps started together in a single Promise.all; Fan-out TTFS is the first of them to complete and Fan-out TTLS the last, both from the in-deployment clientStart, so their gap is the spread the runtime adds across the fan-out}

_{🔴 marks a percentile over its target (within target is left unmarked). Targets (p75/p90/p99, ms) — TTFS 200/300/600}

_{All timestamps are deployment-side; runs are triggered in-deployment, so the CI runner and api.vercel.com sit outside every measured window. TTFS = start() → first step body (includes dispatch + any cold start); Fan-out TTFS/TTLS = first/last step completion of one Promise.all from the same anchor (the gap is the runtime’s fan-out spread); STSO/WO between step bodies. Streaming delivery performance is measured in durabench.}

_{Cold starts stay in the numbers (real bursty-workload latency, inflates P75+); Best is the warm floor.}

github-actions · 2026-10-15T05:26:49Z

Sim World

Simulated world deterministic testing for races. Traces

🟢 world-sim scenario book — 0 fail of 42 total

fence=per-spec

scenario	outcome	events	virt	replay
✅ `smoke-no-steps`	completed	3	0ms	ok
✅ `smoke-one-step`	completed	6	0ms	ok
✅ `hook-at-step-started`	completed	12	0ms	ok
✅ `hook-at-step-completed`	completed	12	0ms	ok
✅ `hook-at-hook-created`	completed	12	0ms	ok
✅ `deadline-hook-wins`	completed	7	1.0h	ok
✅ `deadline-expires`	completed	7	1.0h	ok
✅ `step-vs-timer-early-settlement`	completed	8	1.0h	ok
✅ `long-sleep`	completed	11	30.0d	ok
✅ `hook-never-arrives`	stalled	3	0ms	skipped
✅ `step-retries-twice`	completed	10	2.0s	ok
✅ `parallel-steps`	completed	9	0ms	ok
✅ `hook-on-execution-state`	completed	12	0ms	ok
✅ `peek-hook-before-branch`	completed	12	0ms	ok
✅ `peek-hook-after-branch`	completed	12	0ms	ok
✅ `peek-hook-at-registration`	completed	12	0ms	ok
✅ `race-hook-before-probe`	completed	12	0ms	ok
✅ `race-hook-after-probe`	completed	12	0ms	ok
✅ `race-duplicate-delivery`	completed	13	0ms	ok
✅ `attr-hook-before-step`	completed	11	0ms	ok
✅ `attr-hook-after-step`	completed	11	0ms	ok
✅ `attr-from-step-body`	completed	13	0ms	ok
✅ `fork-hook-after-timeout`	completed	14	1.0m	ok
✅ `fork-hook-before-timeout`	completed	14	1.0m	ok
✅ `count-hook-after-timeout`	completed	17	1.0m	ok
✅ `count-hook-before-timeout`	completed	20	1.0m	ok
✅ `stale-read-step-count-fork`	completed	20	1.0m	ok
✅ `stale-read-equal-step-counts`	completed	14	1.0m	ok
✅ `step-vs-step-fork`	completed	12	0ms	ok
✅ `step-vs-step-fork-fenced`	completed	12	0ms	ok
✅ `fence-catches-benign-direction`	completed	12	5ms	ok
✅ `in-flight-before-decision`	completed	17	1.0m	ok
✅ `in-flight-before-decision-counted`	completed	17	1.0m	ok
✅ `in-flight-after-decision`	completed	19	2.0m	ok
✅ `stale-read-step-count-fork-fenced`	completed	20	1.0m	ok
✅ `fork-hook-wins`	completed	13	1.0m	ok
✅ `fork-timeout-wins`	completed	13	1.0m	ok
✅ `unclaimed-payload-under-fork`	completed	17	1.0m	ok
✅ `claimed-payload-under-fork`	completed	17	1.0m	ok
✅ `writers-independent-step-bodies`	completed	12	0ms	ok
✅ `writers-scripted-tempo`	completed	12	0ms	ok
✅ `cancel-mid-step`	cancelled	7	0ms	skipped

Full trace: world-sim.txt

github-actions · 2026-10-15T05:28:09Z

Framework	Flow route	Step reg.	Framework output
hono	267.4 KiB (±0)	95.6 KiB (±0)	1.99 MiB (+52 B)
nextjs-turbopack	274.8 KiB (±0)	426 B (±0)	972.3 KiB (+44 B)

About these numbers

Sizes are gzip; parentheses show the change against main.
Flow route and Step reg. gate this job, on raw bytes rather than the gzip shown, at max(2%, 50.0 KiB). Framework output is informational.

ffd4bf9 · run

vercel Bot deployed to Preview – workflow-docs October 15, 2026 05:25

View deployment

Navigation Menu

Repository navigation

[core] Share one idle check across same-generation suspension signals - #4893

[core] Share one idle check across same-generation suspension signals#4893
shalabhc wants to merge 1 commit into
mainfrom
workflow-world-api-replay-idle-check

shalabhc commented Oct 15, 2026

Uh oh!

vercel Bot commented Oct 15, 2026 •

edited

Loading

Uh oh!

changeset-bot Bot commented Oct 15, 2026

Uh oh!

github-actions Bot commented Oct 15, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 15, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 15, 2026

Uh oh!

github-actions Bot commented Oct 15, 2026

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

shalabhc commented Oct 15, 2026

Description

How did you test your changes?

PR Checklist - Required to merge

Uh oh!

vercel Bot commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

changeset-bot Bot commented Oct 15, 2026

🦋 Changeset detected

Uh oh!

github-actions Bot commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🧪 E2E Test Results

⚠️ Flaky E2E Tests (passed on retry)

🛠 Infra Events (absorbed by the harness)

E2E Test Summary

Uh oh!

github-actions Bot commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📊 Workflow Benchmarks

Uh oh!

github-actions Bot commented Oct 15, 2026

Sim World

Uh oh!

github-actions Bot commented Oct 15, 2026

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vercel Bot commented Oct 15, 2026 •

edited

Loading

github-actions Bot commented Oct 15, 2026 •

edited

Loading

github-actions Bot commented Oct 15, 2026 •

edited

Loading