Repository navigation
feat: Valve extension rgb encode input - #46
Conversation
EncodeConfig::color_description said a missing description defaults to BT.709 full range. Every encoder falls back to ColorDescription::bt709(), which is limited range, and that is what the stream then declares. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
VK_VALVE_video_encode_rgb_conversion lets an encoder take an RGB image and apply the YUV matrix itself. Where the colour conversion is nothing but that matrix, which covers ordinary SDR capture, the converter's compute dispatch and its buffer-to-image copy disappear from the frame entirely. EncodeConfig::with_rgb_input(format) turns it on. The extension adds one struct to the video profile, and everything that names the profile has to name that version of it: the capability and format queries, the input and DPB images, the query pools and the session. Each codec's one profile gains it. The input image becomes a single-plane RGB image while the DPB stays YUV, and Encoder::encode copies an RGB source into it. The matrix and range come from the configured colour description, checked against what the driver reports it can do. Encoder::new refuses a configuration that cannot work before asking the driver anything: a device without the extension, half-float input, or an encoder not set up for the 4:2:0 at the input's bit depth that the conversion produces. VideoContext::supports_rgb_encode_input answers whether that will work for a codec, input format and colour description, without creating an encoder: the extension, the codec's RGB profile, the conversion's matrix and range, and the input format. It shares one profile builder with supports_encode_format, which now takes the RGB struct as an option rather than a fourth copy of the per-codec profiles existing. ColorConverterConfig::rgb_encode_input asks it for a conversion that is only the matrix, so it no longer says yes for something Encoder::new then refuses. On current RADV that matters at once: the conversion reports limited range only, so full-range RGB input is refused up front rather than producing a stream whose samples and label disagree. Chroma siting asks for what a player assumes when the stream does not say: co-sited with the left luma sample horizontally, midway between rows vertically (H.264 and H.265 chroma location type 0). A driver that cannot gets its other option. The RGB input image is zeroed by the same buffer copy as a YUV one, since that runs on the transfer queue, where a clear command is not allowed. On an RX 9060 XT, H.264 lands at 46 dB in luma against the shader path on the same frames, and H.265 and AV1 at 46 and 48 dB against the H.264 stream. supports_rgb_encode_input agrees with encoder creation for every codec, input format and colour description tested, here and on an Arc A310, which has no extension and answers no throughout. Based on the approach in hgaiser#14, reworked for the current encoder structure. Co-authored-by: Ryan Bateman <rybateman@gmail.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The crate docs point at the hardware RGB path from the colour conversion section, with an example of picking it or the shader per codec. README.md is regenerated from them. CHANGELOG lists the additions. AGENTS.md says which device the RGB tests mean anything on, and which two validation messages that path draws because the layer does not model the extension. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
DatCaptainHorse
changed the title
|
Super glad to see this carrying forward the idea from #14! I had some time this weekend to rebuild my old Moonshine PR on top of this work, and the RGB encode path seems to work really well. I did have one API question, though I may well be missing something: When For a DMA-BUF that's imported for the first time, I don't think that's true yet: it's still Without something like that, the Moonshine side ends up needing its own command pool, command buffer and fence just to submit that one acquire barrier on the transfer queue (and wait for it) before calling Would you be open to the RGB path taking the source layout the same way? Something like: or a layout field on the upload, whatever fits better. The upload barrier could then do the acquire when the layout is Happy to send a patch/PR against this branch too if that's easier! |
|
@porkloin Feel free to do a PR or patch on top of this and I'll include it, you have better idea of the requirements here I feel 🙂 |
|
@DatCaptainHorse I opened a PR against this branch here: DatCaptainHorse#1 Happy to work through it if you have any other ideas! |
feat(encoder): take the source layout in encode_after
|
Thanks, looks good to me! I have some ideas to improve efficiency but I'll submit them as a follow-up PR. I'll tag you both so that you can test it on AMD. |
Part 3/3 of #41 split.
Closes #14 - as this supersedes it.
Currently only AMD GPUs from RDNA-era advertise and support this extension, tested on RX 9060 XT and made sure it's not enabled on Arc A310.