C2H cuda::buffer support by oleksandr-pavlyk · Pull Request #11344 · NVIDIA/cccl

oleksandr-pavlyk · 2026-09-10T21:30:35Z

Description

This PR contributes a portion of #10898

Adds a synchronous, device-accessible checked memory resource using cudaMalloc/cudaFree, plus c2h::make_device_buffer for creating cuda::device_buffer instances backed by it.
Adds a synchronous, host-accessible memory resource backed by aligned new/delete. On integrated devices, it checks available device memory before allocating. Includes c2h::make_host_buffer for creating cuda::host_buffer instances backed by it.
Adds stream-aware C2H generator support and the gen_device_buffer, gen_buffers, and gen_host_buffer helpers for generating data into checked CUDA buffers.
Adds focused tests covering the resources, factories, allocation failures, and generated buffer contents.

closes #11343

~~gated by #11338~~

Checklist

New or existing tests cover these changes.
The documentation is up to date with these changes.

coderabbitai · 2026-09-10T21:42:15Z

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Last reviewed commit: 186d31b

Use the following commands to manage reviews:

@coderabbitai resume to resume automatic reviews.
@coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

▶️ Resume reviews
🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: NVIDIA/cccl/.coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 282e297c-ace6-46cf-8339-1ca4cfd6e2f1

📥 Commits

Reviewing files that changed from the base of the PR and between 2f992ec and 186d31b.

📒 Files selected for processing: 1

c2h/include/c2h/buffer_generators.cuh

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

📝 Summary

Summary by CodeRabbit

New Features
- Added utilities for creating initialized host and device buffers, including paired buffers with matching contents.
- Added helpers to generate host and device data within specified value ranges, including custom types.
- Random value generation now supports a specified CUDA stream.
Bug Fixes
- Improved handling of invalid allocation sizes, unsupported alignments, zero-sized allocations, and allocation failures.
- Improved validation of device and stream compatibility during buffer creation and generation.
- Environment-based configuration values now handle invalid or out-of-range input safely.

C2H adds checked host and device buffer resources, buffer factories, and buffer-generation helpers. Random generators gain stream-taking overloads. Environment values use integer parsing. Tests cover allocation, buffer creation, and generated values.

Changes: C2H buffers and generators

Layer / File(s)	Summary
Checked allocation and buffer resources `c2h/include/c2h/detail/env.cuh`, `c2h/include/c2h/detail/checked_memory.cuh`, `c2h/include/c2h/checked_allocator.cuh`, `c2h/include/c2h/checked_memory_resource.cuh`, `c2h/include/c2h/detail/current_device.cuh`, `c2h/include/c2h/detail/catch2_seed.h`, `c2h/include/c2h/catch2_main.h`, `c2h/include/c2h/catch2_test_helper.h`, `cub/test/catch2_test_c2h_buffers.cu`	Environment values use integer parsing. Checked allocation helpers and host and device buffer resources validate allocation inputs and device state. Tests cover allocation and buffer behavior.
Stream-aware generator implementations `c2h/include/c2h/generator_common.h`, `c2h/include/c2h/detail/generators.cuh`, `c2h/include/c2h/generators.h`, `c2h/generators.cu`, `c2h/generators_gen_values.cu`, `c2h/generators_vector.cu`, `cub/test/catch2_test_c2h_buffers.cu`	Generator implementations support stream-bound policies as well as the existing default policy. Stream overloads check the current device. The public generator header includes the buffer and vector generator headers.
Buffer generation and host copies `c2h/include/c2h/buffer_generators.cuh`, `cub/test/catch2_test_c2h_buffers.cu`	New helpers generate values in device buffers, copy their contents to host buffers, and return the generated buffers or host buffer. Tests cover empty buffers and generated values.

Assessment against linked issues

Objective	Addressed	Explanation
Add `cuda::buffer` support in C2H with checked memory allocators [`#11343`]	✅

Out-of-scope changes

Code Change	Explanation
Move Catch2 and device-system includes inside the `C2H_CONFIG_MAIN` guard (`c2h/include/c2h/catch2_main.h`)	The include-guard change is not part of the stated objective to add `cuda::buffer` support and checked memory allocators.

Possibly related PRs

NVIDIA/cccl#11374: Both PRs change stream-aware C2H generators and buffer-generation APIs.

Suggested reviewers: gevtushenko

Priority: ➖ Normal

Change: Feature

Merge Risk: ⚪ Minimal · up to 186d3

The buffer-generation test covers a nontrivial requested range and checks each returned value against the bounds. No actionable merge-blocking concern is established by the supplied review evidence.

_{Comment @coderabbitai help to get the list of available commands.}

✨ Finishing Touches: 💡 1

🛠️ Fix failing CI checks: 💡

Commit to this branch
Create a new PR

coderabbitai

Actionable comments posted: 3

🧹 Nitpick comments (1)

c2h/include/c2h/detail/checked_memory.cuh (1)

45-46: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

suggestion: std::strtoll accepts negative and malformed input. C2H_DEVICE_MEMORY_LIMIT=-1 casts to a near-SIZE_MAX limit, which silently sets info.override = true while changing nothing. Reject values that do not parse to a positive number.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 6fa497b9-b4bb-4c98-8a0a-89132e3d0d2c

📥 Commits

Reviewing files that changed from the base of the PR and between b6fc73d and 1618acb.

📒 Files selected for processing (16)

c2h/generators.cu
c2h/generators_gen_values.cu
c2h/generators_vector.cu
c2h/include/c2h/buffer_generators.cuh
c2h/include/c2h/catch2_test_helper.h
c2h/include/c2h/checked_allocator.cuh
c2h/include/c2h/checked_memory_resource.cuh
c2h/include/c2h/detail/catch2_nvtx.h
c2h/include/c2h/detail/catch2_seed.h
c2h/include/c2h/detail/checked_memory.cuh
c2h/include/c2h/detail/env.cuh
c2h/include/c2h/detail/generators.cuh
c2h/include/c2h/generator_common.h
c2h/include/c2h/generators.h
c2h/include/c2h/vector_generators.h
cub/test/catch2_test_c2h_buffers.cu

Included review availability: Your plan provides up to 12 included reviews per hour; 10 remain after this review.

coderabbitai

Actionable comments posted: 3

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 669cbb1c-537b-4557-92c7-151ed2dceda5

📥 Commits

Reviewing files that changed from the base of the PR and between 85602af and 0611eea.

📒 Files selected for processing (4)

c2h/generators.cu
c2h/include/c2h/detail/checked_memory.cuh
c2h/include/c2h/detail/generators.cuh
cub/test/catch2_test_c2h_buffers.cu

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

oleksandr-pavlyk · 2026-09-11T20:43:59Z

The fix for NVHPC issue is filed in #11366

Simplified preprocessor guards accordingly

…ibution Add a multi-stream test that demonstrates race condition, confirm failure as per CodeRabbit review claim. Save generation/distribution per stream/device to resolve the race and fix the test.

…in prepare_random_generator(seed, num_items)

- Device-scoped cleanup remains the primary path. - If device selection throws, the state is reset without switching devices.

MSVC requires constexpr constants to be caputed, but Clang insists they should not be. Solution: do not use any_of, use count instead which does not need lambda to begin with.

Changes were requested at earlier stages of this work and have all been addressed.

bernhardmgruber

I reviewed everything except for the checked memory, checked resource, and tests.

…perties

gen_buffers used to create d_items using cuda::no_init, with `gen` later overwriting values. The issue arises for custom_type_t which contains pair of key/value and, possibly, additional bytes. The `gen` routine, however, sets key/value members only, leaving other members uninitialized. The reason tests did not catch it is that they use operator== to test content of the buffer which only compares key/value members. Initializing device buffer with user-provided type instance `min` ensures that all bytes are populated.

…call

bernhardmgruber

Please consider the feedback above, otherwise LGTM

…port

Rewrote condition to fail allocation if free memory is less than bytes + padding while stilla avoiding the possible integer overflow

All compiler in our support matrix support C++17

Also reused checked_host_allocate allocation from checked_allocator instead of reimplementing it.

instead of creating buffer with no_init followed by a call to copy_bytes

coderabbitai

🧹 Nitpick comments: 1

c2h/include/c2h/buffer_generators.cuh: 1

94-94: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

suggestion: Skip the initial fill for ordinary values.

For ordinary T, gen_values_between writes every output element after make_device_buffer fills the full allocation with min. Large buffers therefore incur an avoidable device pass. Use cuda::no_init when the generator writes the complete value. Keep the min fill for custom_type_t, whose generator updates only key and val. (raw.githubusercontent.com)

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository: NVIDIA/cccl/.coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 2b915da7-5b79-426d-ba8e-a4c9cd3cb453

📥 Commits

Reviewing files that changed from the base of the PR and between ca80cf7 and 2f992ec.

📒 Files selected for processing: 4

c2h/include/c2h/buffer_generators.cuh
c2h/include/c2h/checked_allocator.cuh
c2h/include/c2h/detail/checked_memory.cuh
cub/test/catch2_test_c2h_buffers.cu

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

Only initiaze buffer with `min` for custom_type_t.

github-actions · 2026-10-19T21:32:06Z

🥳 CI Workflow Results

🟩 Finished in 4h 23m: Pass: 100%/187 | Total: 8d 20h | Max: 3h 12m | Hits: 38%/551605

See results here.

oleksandr-pavlyk requested review from a team as code owners September 10, 2026 21:30

oleksandr-pavlyk requested a review from Jacobfaib September 10, 2026 21:30

github-project-automation Bot added this to CCCL Sep 10, 2026

oleksandr-pavlyk requested a review from srinivasyadav18 September 10, 2026 21:30

github-project-automation Bot moved this to Todo in CCCL Sep 10, 2026

cccl-authenticator-app Bot moved this from Todo to In Review in CCCL Sep 10, 2026

oleksandr-pavlyk changed the title ~~C2h cuda buffer support~~ C2H cuda::buffer support Sep 10, 2026

coderabbitai Bot reviewed Sep 10, 2026

View reviewed changes

Comment thread c2h/include/c2h/checked_allocator.cuh Resolved

Comment thread c2h/include/c2h/detail/catch2_seed.h Resolved

Comment thread c2h/include/c2h/detail/generators.cuh Resolved

This comment has been minimized.

Sign in to view

oleksandr-pavlyk force-pushed the c2h-cuda-buffer-support branch from 85602af to 0611eea Compare September 11, 2026 14:59

coderabbitai Bot reviewed Sep 11, 2026

View reviewed changes

Comment thread c2h/generators.cu Resolved

Comment thread c2h/generators.cu Resolved

Comment thread c2h/include/c2h/detail/checked_memory.cuh Resolved

oleksandr-pavlyk force-pushed the c2h-cuda-buffer-support branch from 0611eea to be96c1c Compare September 11, 2026 16:59

This comment has been minimized.

Sign in to view

oleksandr-pavlyk requested a review from bernhardmgruber September 11, 2026 20:55

This comment has been minimized.

Sign in to view

oleksandr-pavlyk added 11 commits September 11, 2026 19:38

Add checked C2H CUDA buffer generators

0a6ebd1

Port missed env.cuh utility for parsing integers and added test

7160118

C2H is always built with CTK present and never with NVRTC

e88d951

Simplified preprocessor guards accordingly

Fixed MSVC compilation failures with CTK 12.x

ec93819

Do not capture constexpr variable in lambda

26493e5

Resolve multi-stream race condition from using global generator/distr…

063dff0

…ibution Add a multi-stream test that demonstrates race condition, confirm failure as per CodeRabbit review claim. Save generation/distribution per stream/device to resolve the race and fix the test.

fixup! trailing space cleanup

43f47d3

Checked device allocation to check that alignment is a power of two

25cac95

Handle the legacy default stream before calling stream_ref::device() …

e3ff2db

…in prepare_random_generator(seed, num_items)

Applied the destructor exception-containment fix in c2h/generators.cu.

89a5ad5

- Device-scoped cleanup remains the primary path. - If device selection throws, the state is reset without switching devices.

Rewrite REQUIRED statement to work around MSVC quirks

9a04a67

MSVC requires constexpr constants to be caputed, but Clang insists they should not be. Solution: do not use any_of, use count instead which does not need lambda to begin with.

This comment has been minimized.

Sign in to view

oleksandr-pavlyk requested a review from bernhardmgruber October 7, 2026 21:16

bernhardmgruber reviewed Oct 9, 2026

View reviewed changes

pciolkosz reviewed Oct 9, 2026

View reviewed changes

oleksandr-pavlyk commented Oct 11, 2026

View reviewed changes

Comment thread c2h/include/c2h/buffer_generators.cuh Resolved

oleksandr-pavlyk added 2 commits October 12, 2026 09:05

Optimization: get attribute of interest directly over reading all pro…

7235243

…perties

oleksandr-pavlyk mentioned this pull request Oct 12, 2026

Investigate implementing some of c2h helpers' machine for cuda::buffer support to the CUDA buffer or memory resources #11990

Open

Remove assert_current_device check as it is checked by next function …

ca80cf7

…call

This comment has been minimized.

Sign in to view

Jacobfaib approved these changes Oct 14, 2026

View reviewed changes

bernhardmgruber reviewed Oct 19, 2026

View reviewed changes

bernhardmgruber approved these changes Oct 19, 2026

View reviewed changes

github-project-automation Bot moved this from In Progress to In Review in CCCL Oct 19, 2026

oleksandr-pavlyk added 6 commits October 19, 2026 09:58

Merge remote-tracking branch 'upstream/main' into c2h-cuda-buffer-sup…

ae4372e

…port

Improve readability of check_free_device_memory

4d71d31

Rewrote condition to fail allocation if free memory is less than bytes + padding while stilla avoiding the possible integer overflow

Renamed internal predicate for clarity

1ec89ff

Remove dead code written for pre-C++17 standard

0141eaf

All compiler in our support matrix support C++17

Address nits: ensure consistency with checked_device_allocate

ca6d7c8

Also reused checked_host_allocate allocation from checked_allocator instead of reimplementing it.

Prefer host_buffer constructor call to initialize it from device buffer

2f992ec

instead of creating buffer with no_init followed by a call to copy_bytes

oleksandr-pavlyk enabled auto-merge (squash) October 19, 2026 16:24

coderabbitai Bot reviewed Oct 19, 2026

View reviewed changes

Revert creating device buffer with no_init for ordinary types

186d31b

Only initiaze buffer with `min` for custom_type_t.

This comment has been minimized.

Sign in to view

oleksandr-pavlyk merged commit 69ef904 into NVIDIA:main Oct 19, 2026
419 of 425 checks passed

github-project-automation Bot moved this from In Review to Done in CCCL Oct 19, 2026

oleksandr-pavlyk deleted the c2h-cuda-buffer-support branch October 19, 2026 21:38

Conversation

oleksandr-pavlyk commented Sep 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Checklist

Uh oh!

coderabbitai Bot commented Sep 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviews paused

Summary by CodeRabbit

Assessment against linked issues

Out-of-scope changes

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

oleksandr-pavlyk commented Sep 11, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

bernhardmgruber left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bernhardmgruber left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

github-actions Bot commented Oct 19, 2026

🥳 CI Workflow Results

🟩 Finished in 4h 23m: Pass: 100%/187 | Total: 8d 20h | Max: 3h 12m | Hits: 38%/551605

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

oleksandr-pavlyk commented Sep 10, 2026 •

edited

Loading

coderabbitai Bot commented Sep 10, 2026 •

edited

Loading