Skip to content

feat(link): add the so_priority endpoint option for TCP and TLS links - #2881

Open
ciprianf-flexion wants to merge 1 commit into
eclipse-zenoh:mainfrom
ciprianf-flexion:endpoint-so-priority
Open

ciprianf-flexion wants to merge 1 commit into
eclipse-zenoh:mainfrom
ciprianf-flexion:endpoint-so-priority

Conversation

@ciprianf-flexion

@ciprianf-flexion ciprianf-flexion commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

Description

This is an alternative to #2854 (set the socket priority of multilink links from their priority range). It follows the suggestion of @DenisBiryukov91 in #2854: the value is set on the endpoint, not in a global map. Both PRs solve the same problem, so only one of them should be merged.

What does this PR do?

  • Adds the so_priority endpoint option for TCP and TLS links. It sets the Linux socket priority (SO_PRIORITY) of the link.
  • Connect endpoint: the value applies to the link of that endpoint, for example tcp/192.0.2.1:7447?prio=0-5#so_priority=6.
  • Listen endpoint: one listener accepts all the links of a multilink transport. So the value can depend on the priority range of each accepted link, for example tcp/0.0.0.0:7447#so_priority=0-5:6|6-7:0.
    • An entry <range>:<priority> applies to the links whose range is within <range>.
    • A value without a range applies to all other links, including links without a range.
    • The first matching entry is used. The same syntax also works on a connect endpoint.
  • Both sides set the value after the transport handshake, when the range of the link is known.
  • When the option is not set, nothing changes.
  • On Linux, a value that the process may not set fails when the endpoint is created, with an error that names the option. An example is a value above 6 without CAP_NET_ADMIN. On other platforms, the option logs a warning and does nothing.

Implementation:

  • zenoh-link-commons parses the option and has a small setsockopt helper. The workspace socket2 0.5 has no set_priority.
  • The TCP and TLS links keep the parsed option. A listener gives its option to every link that it accepts.
  • The new method LinkUnicastTrait::apply_socket_priority receives the priority range of the link. The default does nothing. The transport calls it in open_link and accept_link.

Why is this change needed?

Multilink with ?prio= splits traffic over separate TCP connections, for example ?prio=0-5 for real-time data and ?prio=6-7 for bulk data. But on the sending host, both connections can still share one NIC TX ring. A small real-time message that enters the ring after bulk data must wait until the NIC has sent all of that bulk data. At 1 Gbit/s, 1 MB of bulk data in the ring delays the next real-time packet by up to 8 ms.

Many multi-queue NIC drivers choose the TX ring from the socket priority, for example the NVIDIA Jetson nvethernet driver. Measured between a Jetson AGX Orin and an x86 host over 1 Gbit/s, with a 400 Hz probe and a 1.73 MB payload at 30 Hz demoted to data_low:

Setup Probe p99 delay
one link 15 ms
multilink, both links on one TX ring 3.5 ms
multilink, probe link on its own TX ring 0.9 ms

Both ends of a link send data, so both ends need a value. The accepting side cannot use one value per listener:

  • One listener accepts all the links of the transport.
  • Linux does not copy SO_PRIORITY from the listening socket to an accepted socket.

So the listener keeps its option and applies it to each accepted link when the range of that link is known.

Differences from #2854

#2854 This PR
Where the value is set global transport.link.tx.socket_priority map #so_priority on each endpoint
Different values per interface no, one map per process yes, one value per endpoint
Accepting side value from the global map range entries on the listen endpoint
Connect endpoint string unchanged gets a #so_priority part

Note: as for #dscp, a # option on a connect endpoint currently prevents the reconnect of a single dropped multilink link, see #2569. With #2854 the endpoint string does not change, so it is not affected.

How was it tested?

  • Unit tests for parsing the option and for reading back the value that the helper sets.
  • New transport test unicast_socket_priority, for TCP and TLS:
    • A two-link multilink with different values on the two sides. The test reads SO_PRIORITY back on both ends of both links.
    • A two-link multilink without the option keeps the OS default.
    • A single link without a range gets only the listener's value without a range.
    • The test fails if the accepting side does not pass the range of the link.
  • Manual check: a listen endpoint with #so_priority=0-5:7, without CAP_NET_ADMIN, stops zenohd at startup with an error that names so_priority.
  • cargo clippy --all-targets --features test -- --deny warnings, the uring variant and the nightly rustfmt check pass.

Related Issues

Alternative to #2854. Related to #2569.

🤖 Generated with Claude Code


🏷️ Label-Based Checklist

No specific label requirements detected.

Current labels: No labels

Add one of these labels to this PR to see relevant checklist items: api-sync, breaking-change, bug, ci, dependencies, documentation, enhancement, new feature, internal

This section updates automatically when labels change.

Adds the `so_priority` endpoint option. It sets the Linux socket priority (SO_PRIORITY) of TCP and
TLS links. On a connect endpoint, a value applies to the link of that endpoint. On a listen
endpoint, `<range>:<priority>` entries choose the value by the priority range of each accepted
link, e.g. `so_priority=0-5:6|6-7:0`, and a value without a range applies to all other links.

Both sides apply the value after the transport handshake, when the range of the link is known.
A value the process may not set (above 6 without CAP_NET_ADMIN) fails when the endpoint is
created. On other platforms the option is ignored with a warning.

On NICs or qdiscs that select the TX queue or band from the socket priority, the real-time link
of a multilink transport then no longer waits behind bulk data.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Alexandru Farcasanu <ciprianfarcasanu@gmail.com>
@codecov

codecov Bot commented Oct 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.52778% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.97%. Comparing base (766b455) to head (b8ca843).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
io/zenoh-link-commons/src/socket_priority.rs 96.87% 3 Missing ⚠️
...enoh-transport/src/unicast/establishment/accept.rs 80.00% 1 Missing ⚠️
.../zenoh-transport/src/unicast/establishment/open.rs 80.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2881      +/-   ##
==========================================
+ Coverage   74.95%   74.97%   +0.01%     
==========================================
  Files         419      420       +1     
  Lines       64143    64284     +141     
==========================================
+ Hits        48080    48197     +117     
- Misses      16063    16087      +24     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

@ciprianf-flexion

Copy link
Copy Markdown
Contributor Author

@DenisBiryukov91 could you please take a look?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants