Skip to content

Virtual transport seam: run SRT over an owner-supplied datagram transport - #3426

Open
maxolgi wants to merge 29 commits into
Haivision:masterfrom
maxolgi:quicsrt
Open

maxolgi wants to merge 29 commits into
Haivision:masterfrom
maxolgi:quicsrt

Conversation

@maxolgi

@maxolgi maxolgi commented Oct 31, 2026 •

Copy link
Copy Markdown

What this adds

SRT sockets can now exchange their datagrams through an owner-supplied
userspace transport (for example WebTransport/QUIC) instead of an OS UDP
socket, with no change to the behavior of regular UDP sockets in the
same process. Everything above the multiplexer channel — handshake, ARQ,
crypto, TSBPD — runs unchanged; the feature is gated behind the CMake
option ENABLE_VIRTUAL_TRANSPORT (OFF by default, Linux and Emscripten).

Motivating use case: serving SRT sessions to browser/WASM clients over
WebTransport, where the browser hands datagrams to the application and
there is no UDP socket to bind.

The seam

  • srt_attach_virtual_transport() wires an unbound socket to an
    owner-supplied send callback; outgoing datagrams are delivered to it
    as one contiguous network-order buffer, exactly the bytes a UDP
    sendto() would have emitted.
  • srt_virtual_transport_inject() feeds received datagrams back in,
    thread-safe, from any thread.
  • srt_detach_virtual_transport() / srt_virtual_transport_release()
    manage lifetime; the transport object is reference-counted so the
    caller's handle stays valid concurrently with socket closure.
  • The seam lives behind CChannel; every attached socket owns a
    dedicated multiplexer, so stock UDP sockets are unaffected.

Two accompanying options

  • SRTO_SKIP_INDUCTION — conclusion-first handshakes. The enclosing
    transport already authenticates the peer (e.g. WebTransport TLS), so
    the Induction round trip is redundant. A caller with this option sends
    the Conclusion immediately with cookie 0; a listener accepts it only
    when the request arrived over a virtual-transport channel — a plain
    UDP listener keeps the full cookie validation.
  • SRTO_INITIAL_RTT — seed the RTT estimator from the enclosing
    transport's measured RTT (e.g. QUIC's smoothed RTT) until the first
    live sample. Skipping Induction removes the round trip that normally
    delivers the first sample, so the two options are designed to
    accompany each other.

Included stock fixes

Four independent fixes to existing code, found while stress-testing this
work; each is self-contained and useful on its own:

  1. False SRTS_BROKEN window in postConnect (status poller could see
    BROKEN while the handshake was completing)
  2. Unlocked receiver-loss-list reads in the LOSSREPORT sender
  3. Unsynchronized buffer publication vs. the GC scan (plus the same
    pattern in the CLOSE_SYNCH wait)
  4. Demoted a spurious Warn-level log on empty reads

We're happy to split these into separate PRs first if that's easier
to review — say the word.

API shape (open to discussion)

The new symbols follow the ENABLE_BONDING precedent: declared and
linkable in every build; in builds without the seam they fail at runtime
(attach returns NULL, the int-returning functions -1/SRT_EINVPARAM,
the options rejected as unsupported). This keeps a single libsrt.so
ABI stable across build configurations. If maintainers prefer a
different gating scheme, happy to adjust.

Testing

  • 16-case suite over in-memory wires: positive paths, rejection paths
    (rendezvous, stock-UDP, attach-after-bind), detach under live traffic,
    queue pressure, and a wire-level assertion that the first skip-induction
    flight is a cookie-0 Conclusion.
  • 4-case stock-build suite asserting the clean-failure behavior of the
    symbols and options when the seam is compiled out.
  • Full suites pass in both configurations (316 VT / 297 stock),
    C++11-strict.

Reviewing

The commit series is structured feature-first, then the stock fixes,
then review polish; the significant files are srtcore/virtual_transport.{h,cpp}
(the transport object), srtcore/channel.* (the seam), srtcore/api.cpp
(attach/detach plumbing), srtcore/core.cpp (handshake and options),
test/test_virtual_transport*.cpp, and the docs/API sections.

CI notes

Full CI was validated on a fork rehearsal: every workflow that can pass, passes;
plus a dedicated virtual-transport workflow that builds with
ENABLE_VIRTUAL_TRANSPORT=ON, C++11, warnings-as-errors, and runs the
complete test suite (325 tests, including the 16-case virtual-transport
suite and the 4-case stock-build rejection suite).

Three checks fail on the fork's runners for reasons unrelated to this
change, each reproduced on unmodified upstream master in
this probe PR:

  • Linux C++11, ARM job — flaky upstream test; upstream's own runs
    from Oct 9, Oct 15, and Oct 30 (twice) show the same failures on
    Haivision's own runners.
  • QEMU s390x — fails at the configure step, before any build;
    runner infrastructure.
  • ABI checks — GitHub is migrating ubuntu-latest from Ubuntu
    24.04 to 26.04 gradually between Oct 19 and Nov 19, 2026
    (runner-images #14748).
    Runs that land on the new image get CMake 4, which refuses to configure
    the 2021-era v1.5.0 baseline tag (its cmake_minimum_required
    predates CMake 3.5). Haivision's own Oct 30 run passed on the old
    image; the same unmodified code fails on the new one. This affects
    every ubuntu-latest workflow in this repository as the rollout
    completes — not just this PR.

One real portability bug was found thanks to the new image and is fixed
in this PR: a one-past-the-end std::vector access in the virtual
transport receive path, trapped by GCC 15's hardened libstdc++ (default
on 26.04). The suite passes under -D_GLIBCXX_ASSERTIONS.

A socket may now exchange datagrams through an owner-supplied transport
instead of an OS UDP socket, without changing the behavior of regular UDP
sockets in the same process. The seam lives behind CChannel: an attached
socket gets a virtual channel arm whose sendto() invokes the owner's send
callback - fire and forget, like a sendto() on a UDP socket, the transport
owns delivery - and whose receiving worker picks up datagrams injected
through srt_virtual_transport_inject(). Every attached socket owns a
dedicated multiplexer and no OS socket is created for it; the send path
performs no per-datagram allocation.

The receiving worker is woken through a condition variable with a
persistent woken flag (level-kept semantics, so a wake arriving while the
worker is between waits is never lost) instead of an eventfd, so the seam
is free of OS descriptor primitives and works on any platform; the 10 ms
wait budget matches the poll()/select() timeouts of the UDP path. Heavy
logging (ENABLE_DEBUG) compiles clean.

The seam is compiled in only with the new ENABLE_VIRTUAL_TRANSPORT CMake
option (default OFF; an OFF build is stock and unchanged). The option is
supported on Linux and Emscripten - the Emscripten build defines the LINUX
preprocessor macro only and includes musl's endian.h through utilities.h -
and it requires at least C++11: a lower USE_CXX_STD together with the
option is rejected at configure time. The new sources and tests carry the
contributor copyright header.

Verified: ENABLE_VIRTUAL_TRANSPORT=ON build with unit tests enabled passes
the full TestVirtualTransport suite; the stock gate-OFF build still
compiles.
…rtual transport

New SRTO_SKIP_INDUCTION socket option (ENABLE_VIRTUAL_TRANSPORT builds
only, pre-restriction, rejected together with SRTO_RENDEZVOUS). A caller
with this option sends the HSv5 Conclusion immediately with cookie 0,
skipping the Induction round trip; a listener with it accepts a
Conclusion-first handshake. The cookie bypass is restricted to requests
arriving over the listener's virtual transport channel, where peer
authentication comes from the enclosing transport: a plain UDP listener
keeps the stock cookie validation and treats a Conclusion-first request as
one with a wrong cookie. For the same reason a caller that has this option
set while exchanging datagrams over a plain UDP socket is rejected at
connect time - it would send cookie-0 Conclusions that no stock listener
answers, hanging until the connect timeout.

A socket with an attached virtual transport may enter the listening state
without a prior srt_bind() call: listen() synthesizes the virtual binding
under the VIRTUAL_TRANSPORT_BIND_LABEL_ADDR label (10.213.255.254, an
RFC 1918 address deliberately picked as an unlikely label of a real peer),
which is purely reported through srt_getsockname() and never used for
dispatching, since every attached socket owns a dedicated multiplexer.

startConnect() registers the connector ahead of every later stage; the
refusal paths that fire while the registration is live but before the
socket reaches the connecting state (the SRTO_SKIP_INDUCTION rendezvous
refusal, the plain-UDP caller gate, and the createCrypter()/
createSrtHandshake() rejections in the skip-induction caller setup) now
remove the registration before throwing - an entry abandoned by such a
throw would stay registered until its TTL expired (for rendezvous, the
connect timeout times ten) and deadlock the teardown when the socket was
closed inside that window.

Verified: gate-ON build passes the full TestVirtualTransport suite
(rendezvous refused in both option-setting orders, plain-UDP caller
refused at connect, bind-less listen with concurrent sessions); the stock
gate-OFF build still compiles.
New SRTO_INITIAL_RTT integer socket option (ENABLE_VIRTUAL_TRANSPORT
builds only, pre-restriction, default 0 disabled, nonzero 1..30000 ms)
that seeds the RTT estimator with an application-supplied RTT before the
first live ACK/ACKACK sample: m_iSRTT is set to the value in microseconds
and m_iRTTVar to mean/4 (the RFC 6298 variance/mean ratio). This exists
because the skip-induction handshake removes the Induction round trip that
normally delivers the first RTT sample; the enclosing transport (such as
WebTransport/QUIC) knows the smoothed RTT and hands it to SRT so that
cold-start retransmission timing (RTO and NAK intervals) starts from a
realistic value.

The seed is applied both in setOpt() and in open() - the latter because
open() also runs at bind/connect/listen entry, after the option may have
been set, and would otherwise reset the estimator to defaults. An accepted
socket inherits the option from the listener through the config copy and
is seeded by its own open() call. The seed survives connection completion
on both roles: the per-peer-address CInfoBlock cache lookups in
postConnect() and acceptAndRespond() are suppressed when a seed is
explicitly set, because virtual transport reuses peer address labels
across sessions and a cache hit could stomp the seed with a value of an
unrelated session. Both lookup sites compute a single local use_rtt_cache
flag (a constant true in stock builds) instead of duplicated ifdef branch
bodies. The upstream first-live-sample behavior is untouched:
m_bIsFirstRTTReceived stays false, so the first measured RTT replaces the
seed.

With the option unset, or in stock builds, the behavior is unchanged.

Verified: gate-ON build passes the full TestVirtualTransport suite
(including InitialRTTSeedThenLiveReplace); the stock gate-OFF build still
compiles.
test_virtual_transport.cpp drives the seam over in-memory wires between
attached sockets (owner send callback on one side, datagram injection on
the other), with no OS socket involved. The suite covers:

- end-to-end connectivity, with and without SRTO_SKIP_INDUCTION
  (Conclusion-first handshakes both directions);
- the documented rejections: plain-UDP skip-induction caller refused at
  connect, SRTO_SKIP_INDUCTION refused together with SRTO_RENDEZVOUS in
  both option-setting orders, stock-socket listen without bind rejected,
  attach rejected on a bound or already-attached socket, detach rejected
  without a transport and inject refused after detach;
- SRTO_INITIAL_RTT seeding the estimator and the first live sample
  replacing the seed, plus option readback;
- receiver-side too-late drop of a missing packet;
- bind-less listen: single session and several concurrent attached
  listeners that enter the listening state with no srt_bind() anywhere,
  each wired to its own caller through an independent transport, with the
  synthesized binding label reported by srt_getsockname() and the whole
  lifecycle creating no OS socket fd;
- hardening: send-callback backpressure with queue overflow dropping
  injects, malformed injects rejected by size, detach during live
  traffic, and multiple accepts sharing one listener multiplexer.

The suite runs inside the standard test-srt gtest binary and is exercised
by the virtual transport CI workflow together with the full suite.
ENABLE_VIRTUAL_TRANSPORT existed only on the CMake surface: the legacy
Tcl configure wrapper lists its explicit options from the cmake_options
set in configure-data.tcl, which did not carry the new option (it only
worked as an unlisted blind pass-through, invisible in --help and without
--disable support). The option was therefore undiscoverable from the
secondary build surface.

Added enable-virtual-transport to cmake_options. The entry follows the
repo's own regeneration process (scripts/generate-configure-options.tcl
run over a CMakeCache.txt): the option name and its position in the list
match the generated output, and the description is the full text of the
option() call in CMakeLists.txt, as every checked-in entry keeps the
untruncated description. This is all the legacy flow needs: the wrapper
translates --enable-virtual-transport into -DENABLE_VIRTUAL_TRANSPORT=1
for cmake, which then enforces the platform and C++11 minimum gates; the
CMake-side gating stays the single authority, so no checks were
duplicated into the Tcl layer.

Verified with the repo's own tooling: ./configure --help lists the new
option between --enable-unittests and --openssl-crypto-library;
--enable-virtual-transport --dryrun runs "cmake ...
-DENABLE_VIRTUAL_TRANSPORT=1"; --disable-virtual-transport is inverted to
=0 by the disabler machinery; regenerating the options list from a stock
CMakeCache shows no missing SRT option in cmake_options.
The ENABLE_VIRTUAL_TRANSPORT option and its TestVirtualTransport unit test
suite were built by no CI configuration, so the gate-ON code path would
rot silently between the stock workflows, which all configure with the
option at its default OFF and never compile the suite.

Added ubuntu-virtual-transport.yml, a single lean Linux job following the
ubuntu-c++11.yml conventions (same runner and step layout; the triggers
additionally list the fork development branch, so the gate-ON build runs on every
push to that branch as well as on the master and dev
events the stock workflows fire on). It
configures the C++11 build with unit tests and testing enabled plus
ENABLE_VIRTUAL_TRANSPORT=ON and warnings-as-error (also exercising the
heavy logging path that comes with ENABLE_DEBUG), then runs the FULL
test-srt suite - VT-gated code paths are exercised only here, since the
stock Linux C++11 workflow builds with the option OFF. A dedicated file
was preferred over growing the ubuntu-c++11 matrix because the repo keeps
one workflow file per build configuration and a matrix dimension would
double that workflow's jobs and drag its coverage/codecov baggage along.
API-socket-options.md: full sections for SRTO_SKIP_INDUCTION and
SRTO_INITIAL_RTT in the established format. SRTO_SKIP_INDUCTION is
documented as a pre-restriction boolean that makes a caller send the HSv5
Conclusion immediately with cookie 0, rejected together with
SRTO_RENDEZVOUS and immutable past connect/listen; on a listener it
accepts a Conclusion-first handshake, with the cookie check bypassed only
for requests arriving over the listener's virtual transport channel (a
plain UDP listener keeps the stock validation, and a plain UDP caller is
rejected at connect time). SRTO_INITIAL_RTT is documented as a
pre-restriction integer (ms, default 0 disabled, nonzero 1..30000)
seeding mean RTT and variance=mean/4 (RFC 6298 ratio) ahead of the first
live ACK/ACKACK sample, suppressing the per-peer-address cache lookup at
connection completion, and inherited by accepted sockets. The version
cells of both option rows and of the ENABLE_VIRTUAL_TRANSPORT build
option stay blank pending an upstream release that ships them.

API-functions.md: a Virtual Transport section (index table plus function
reference) covering srt_attach_virtual_transport,
srt_detach_virtual_transport, srt_virtual_transport_inject and
srt_virtual_transport_release, following the page's per-function format
with Returns/Errors tables and the SRTVirtualTransportSendFunc callback
type definition. The send callback contract is spelled out: fire and
forget exactly like a sendto() on a UDP socket (SRT neither retries nor
otherwise reacts to a short or zero return), invocations run on internal
SRT threads and are serialized, and after detach returns no further
invocations happen - closing is not a substitute for detaching, since the
shutdown still flows through the callback. Forking a process that uses a
socket with an attached virtual transport is not supported: the
transport's condition-variable wakeup, pending-datagram queue and other
internal state shared between threads are not reset in the child, so the
behavior of the child process is undefined. Attached sockets use a
dedicated multiplexer each with no OS socket, and the platform support is
spelled as Linux and Emscripten.

build-options.md: the ENABLE_VIRTUAL_TRANSPORT row in the options table
and a detail section stating what the seam enables, the platform
restriction, the C++11 minimum (a lower USE_CXX_STD is rejected at
configure time), the OFF default, and that an OFF build is stock.
postConnect cleared m_bConnecting before setting m_bConnected, and
CUDTSocket::getStatus() synthesizes SRTS_BROKEN for a socket whose
status is still SRTS_CONNECTING while both the connecting and connected
flags are clear (the connect-timeout report). A thread polling
srt_getsockstate() could therefore observe a socket as BROKEN while its
handshake was completing: reorder the stores so the connected flag is
published before the connecting flag is cleared, making that flag
combination unreachable during the regular CONNECTING -> CONNECTED
transition. No functional path reads m_bConnecting between the two
stores; the socket's own completion logic and all error paths are
unchanged.

Demonstrated with a caller-listener pair over the virtual transport
seam (skip-induction, non-blocking caller) and a concurrent status
poller: with the window artificially widened, the original order
produced a spurious BROKEN on 1898 of 2000 connections (all of which
completed); with the fix the report is gone. The false report was
observed killing live sessions in a production WebTransport gateway, whose session
ticker polls every socket's status every ~2 ms and terminates a session
on a single BROKEN sample; TSan-slowed CI runners stretched the window
enough to hit it roughly once per four 50-viewer test runs.

Sibling windows exist while srt_connect() itself is on the caller's
stack (the status is set to CONNECTING in CUDTUnited::connectIn before
startConnect() publishes m_bConnecting, and an unbound caller passes
through SRTS_OPENED on the way). They are not addressed here: they are
observable only by a poller that runs during the srt_connect() call,
not by one that starts polling after it returns. Upstreamable as is.
sendCtrl(UMSG_LOSSREPORT) read the receiver loss list without
m_RcvLossLock in two places: the getLossLength() test guarding the
no-argument branch, which then took the lock only around
getLossArray(), and the getLossLength() that feeds updateNAKInterval
after the branch, which ran unlocked in every path. The receiver loss
list is mutated concurrently by the TSBPD thread, so those unlocked
reads raced the mutations. Both branches now snapshot the loss length
under m_RcvLossLock, the no-argument branch tests the snapshot instead
of the shared state, and the NAK interval update runs on the snapshot,
so no loss-list state is read outside the lock.
checkBrokenSockets() decides whether a lingering closed socket may be
qualified for deletion by reading m_pSndBuffer and calling
getCurrBufSize() on it. On a socket whose connection was completing
at close time, prepareBuffers() publishes m_pSndBuffer and
m_pRcvBuffer on the connecting thread, so the GC thread could read
the pointer and walk the buffer while the publication was still
unsynchronized. The GC read is now taken under m_RcvBufferLock, and
prepareBuffers() publishes both buffers under the same lock, so the
GC sees the buffers only fully published.
readMessage() logs a warning when it finds nothing readable and no
out-of-order packet pending, suggesting the caller ignored the
isRcvDataReady() result. The condition is also reached through benign
readiness/read interleavings, where a Warn-level line per read is log
noise rather than a fault indication. Demoted to Debug.
The virtual transport API (srt_attach_virtual_transport and
sibling functions) and the SRTO_SKIP_INDUCTION/SRTO_INITIAL_RTT
options were previously declared only under
SRT_ENABLE_VIRTUAL_TRANSPORT, so applications compiled against a
seam-enabled library failed to even link against a stock build, and
every libsrt.so.1 carried a different symbol set. Following the
ENABLE_BONDING precedent: the declarations and enum values are now
unconditional, while the implementations remain gated - in builds
without the seam the functions fail (attach returns NULL, the rest
return -1/SRT_EINVPARAM) and the two options are rejected as unknown.
The virtual transport symbols now exist in every build, and in builds
without ENABLE_VIRTUAL_TRANSPORT they must fail cleanly: attach
returns NULL, detach and inject return -1/SRT_EINVPARAM, and the
SRTO_SKIP_INDUCTION/SRTO_INITIAL_RTT options are rejected as unknown.
These tests exercise those paths in the stock build; in seam-enabled
builds the file compiles empty and the positive paths remain covered
by the TestVirtualTransport suite.
The virtual transport functions and the SRTO_SKIP_INDUCTION and
SRTO_INITIAL_RTT options are now declared in every build; in builds
without ENABLE_VIRTUAL_TRANSPORT they fail at runtime instead of
being absent. The API, socket option, and build option documents now
state the runtime failure behavior and keep the build flag scoped to
where the feature actually works.
The workflow fired on the quicsrt branch of the fork, which does not
exist upstream; master and dev cover the upstream targets.
Comments and test payloads referenced the QuicSRT product and the
srt-rs fork by name. For the upstream tree the same points are made
with neutral wording: gateway examples speak of a generic WebTransport
gateway, the RTT variance ratio cites RFC 6298, and the skip-induction
cookie behavior is described by its mechanism instead of the fork it
was borrowed from. No code change - comments and test payload strings
only.
The two files added by the virtual transport seam carried several
spots that violate the repository .clang-format (an empty destructor
body, a line-wrap in the WireSend matcher, aligned member declarations
in the test fixture, and joined assertion lines). Formatted with
clang-format; no code change.
The identifier and payload string "warmup" in the detach test trip
the codespell dictionary (upstream CI runs it with the repository
config); renamed to warm_up / "vt-detach warm-up", which upstream
dictionaries accept. No change to what the test does.
The unconditional SOURCES list of the unit tests compiles
test_virtual_transport_stock.cpp on every platform, and the plain
#include <arpa/inet.h> before the in-file guard does not exist under
the MSVC Windows build (which runs the unit tests in CI). The
identifiers the file needs (sockaddr_in, inet_addr, htons, AF_INET)
already arrive through srt.h -> platform_sys.h on both platform
families, so the include is simply removed.
The trailing "(ENABLE_VIRTUAL_TRANSPORT)" notes on the now
unconditional SRTO_SKIP_INDUCTION and SRTO_INITIAL_RTT read like
compile-time gating of the enumerators; reworded to "(requires
ENABLE_VIRTUAL_TRANSPORT)" to state the build option the feature
needs, matching the availability wording of the API block.
Three accuracy fixes from the review round: the getStatus() timeout
check now documents that its flag reads are order-sensitive
(m_bConnecting must be read before m_bConnected, mirroring the
publication order postConnect was fixed to use); the skip-induction
cookie bypass no longer claims the expected cookie is "recorded as
whatever the peer has sent" - the check is simply skipped, nothing
is recorded; and the SRTO_INITIAL_RTT variance comments state the
actual arithmetic (RTO = SRTT + 4*RTTVAR = 2*SRTT, the RFC 6298 RTO
shape) instead of a non-existent "RFC 6298 variance/mean ratio",
noting SRT's own initial estimate uses mean/2. The
srt_virtual_transport_release() doc comment additionally states that
the send callback keeps firing after the handle is released until the
socket is closed or the transport is detached - the rule existed only
in the API document, not in the header a C application reads.
The synchronous-close wait loop (SRT_ENABLE_CLOSE_SYNCH, used with
SRTO_SNDSYN) read m_pSndBuffer and its occupancy unlocked, while a
connection completing on another thread publishes those pointers in
prepareBuffers under m_RcvBufferLock - the same publication race
checkBrokenSockets was fixed against, left open in this one remaining
reader. The loop now snapshots the pointer and the buffered count
under m_RcvBufferLock at the top of each iteration and does its
waiting unlocked, so the publication is never observed torn. The
consequence was benign before (the count is a single int and the loop
exits through other checks), but the read is now uniform with the GC
scan.
The SRTO_SKIP_INDUCTION section now documents the PBKEYLEN
consequence of skipping the Induction phase: without the listener's
key-length advertisement a mismatch that stock auto-resolves during
the handshake instead fails at the key-material exchange, so both
endpoints should use a consistent SRTO_PBKEYLEN. The attach error
table gains the SRT_ENOBUF row for the allocation-failure path, and
the SRTO_INITIAL_RTT description states the RTO arithmetic precisely
instead of citing a non-existent RFC 6298 variance/mean ratio.
…s on

The ENABLE_VIRTUAL_TRANSPORT C++11 floor was only checked when
USE_CXX_STD was explicitly defined; configured without it, the build
compiled with the compiler's default standard and the floor was never
enforced. USE_CXX_STD now defaults to 11 when the seam is enabled and
no standard was requested, mirroring the ENABLE_STDCXX_SYNC pattern.
A local VT build with no USE_CXX_STD now compiles all library sources
as C++11 and runs the full suite (the SharedMutex tests, compiled
only below C++17, included).
…ipts

The workflow re-implemented the prepare/configure/build/test steps
inline and ran ./test-srt directly, where every other Linux workflow
delegates to scripts/workflows/build and tests through ctest. Switched
to the shared scripts with the CMake passthrough for the option
(-- -DENABLE_VIRTUAL_TRANSPORT=ON), the form ubuntu-mingw.yml already
uses, so the job inherits upstream's exact flag handling and core-dump
processing and cannot drift from the other workflows. The configure
line was verified locally.
inet_addr is deprecated on the Windows SDKs (C4996) and this test
file compiles on the MSVC unit-test CI; inet_pton is warning-free
everywhere. Filling the address is the only change.
The skip-induction positive test passed even if the caller regressed
to sending an Induction first, because the virtual listener answers
stock inductions as well. The Wire harness now captures the first
datagram a caller sends, and the test parses it as a CHandShake to
assert that the very first flight is a Conclusion with cookie 0 -
the option's defining behavior. Verified falsifiable by flipping
the expected request type to Induction and watching the test fail;
the stock test's Induction pin was flipped the other way and
failed as well.
@maxolgi
maxolgi marked this pull request as draft October 31, 2026 13:55
@maxolgi maxolgi changed the title QuicSRT Virtual transport seam: run SRT over an owner-supplied datagram transport Oct 31, 2026
CVirtualTransport::receive() took the payload source pointer as
&dg.data[HDR_SIZE]; for a header-only datagram that is
operator[](size()) on the datagram vector - a one-past-the-end
element access, undefined behavior that hardened libstdc++ builds
(Ubuntu 26.04/GCC 15 ship assertions on) trap with an abort in the
receive worker. Caught by the fork CI rehearsal on the
InjectInvalidSizeRejected test. The pointer is now derived through
data() + HDR_SIZE, which is well-defined for the one-past-the-end
case; the zero-length memcpy from it was always legal. Verified with
a local -D_GLIBCXX_ASSERTIONS build: the abort is gone and the full
virtual transport suite passes.
@maxolgi
maxolgi marked this pull request as ready for review October 31, 2026 16:10
@maxolgi

maxolgi commented Oct 31, 2026

Copy link
Copy Markdown
Author

"Marking ready for review. Full CI validated on the fork (see CI notes); the three red checks reproduce identically on unmodified upstream master."

@ethouris

ethouris commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator

Ok, first things first: As this would have to be a new feature, it cannot be drawn off master - this one holds the stable versions for 1.5.* line and no new features are allowed.

The dev branch contains new features and it's the only allowed target to use with new features. That said, note that there's a big change in the multiplexer management, as well as the buffer management for reading is completely refactored.

As for this solution itself, it's also not acceptable in this form by the following reasons:

  1. This can't be a setting applied on the socket as such. This effectively applies to the multiplexer, and multiplexers are potentially shared between sockets. There are also other solutions that work this way, notably options that apply to the UDP socket as such. These may make the multiplexer either non-sharable between SRT sockets or at least require that these setting be set the same.
  2. The functions that manage sending and recevig in the virtual transport should accept an array of buffers, just as the ::sendmsg and ::recvmsg do. These solutions are created specifically for sending at once multiple data that are contained in differet pieces of memory, without copying them, as this would sit on performance.
  3. The object management is done wrong. The actual resolution should be directly in the CChannel class; it should either contain both UDP socket descriptor and the handler function, to be selected if exists, or the sendto and recvfrom methods shall be made virtual, with instantiation classes like CChannelUDP and CChannelCustom, something like that - whichever offers better performance.
  4. Skipping INDUCTION part of the handshake changes the protocol, and this shouldn't be something to be set separately on the socket, rather as a part of the configuration done for the virtual transport configuration as such.
  5. Setting up an alternative transport shall be done the same way as currently setting up the multiplexer, as it actually is setting up the multiplexer, so the actual activity should be done through "binding". And as such, it should be under the direct control of the operation done on the socket - meaning, you don't create any object "in the socket" (because there's no need for this), you access the data through the socket to operate directly on the multiplexer. The socket shall completely contain no data related to this. Also, everything that is a setting specific for this very feature shall be contained in the structure that is used to configure binding the socket to this virtual tansport - meaning, it should pass exactly the same procedure as "binding" so far: finding the suitable multiplexer, or creating a new one, configuring it with the data directly passed by the user. Selection of the right pass for the updateMux, for example, likely would have to be reworked to select the right function depending on how it is configured - currently it's srt_bind, which requires that SRT create the UDP socket itself, or srt_bind_acquire, where the user can pass the pre-created UDP socket himself. This would have to be yet another binding function, like srt_bind_virtual or srt_bind_custom, where the structure with all settings and required data is passed to it.

@maxolgi

maxolgi commented Nov 2, 2026

Copy link
Copy Markdown
Author

Thank you for the detailed review — this is very helpful.

Before I rework it, here's my understanding of the target design, so please
correct me if I got anything wrong:

  • I'll retarget to dev and reimplement on top of the new multiplexer
    management rather than textually rebase (I can see the CMultiplexer /
    queue refactoring changes most of the touched code).
  • The virtual transport becomes a multiplexer-level setting configured
    through a new binding function, srt_bind_virtual() (or
    srt_bind_custom, if you prefer that name), which takes a structure
    with all settings and required data (send callback + opaque,
    skip-induction, etc.). The socket itself keeps no data related to this,
    and updateMux selects the right path: srt_bind, srt_bind_acquire,
    or srt_bind_virtual.
  • Every srt_bind_virtual() gets a dedicated multiplexer — virtual
    multiplexers are not reuse candidates for new bindings (accepted
    sockets still join the listener's multiplexer through the normal
    accept path). This can be relaxed to same-settings sharing later if
    you prefer.
  • The resolution moves into CChannel. I plan to make sendto/recvfrom
    virtual with CChannelUDP / CChannelVirtual subclasses — the dynamic
    allocation is already there and the XXX comment in
    CMultiplexer::configure seems to anticipate exactly this. The virtual
    dispatch also keeps the stock UDP path untouched. If you'd prefer the
    both-members-and-branch variant instead, I can do that too.
  • The send and receive functions of the virtual transport will accept an
    array of buffers (like ::sendmsg/::recvmsg), so the header and
    payload blocks are passed through without gathering or copying.
  • Skipping INDUCTION becomes part of the bind configuration, not a socket
    option. The synthetic binding currently synthesized in listen()
    disappears — a listener binds explicitly via srt_bind_virtual().

One thing I'd like to confirm before I implement: datagram injection.
I'd make it go through the socket — srt_virtual_transport_inject(SRTSOCKET, ...),
resolving socket → multiplexer → channel, with no extra handles or
reference counting (lifetime tied to the multiplexer). Acceptable?

And one scoping question: the PR also adds SRTO_INITIAL_RTT, an optional
seed for the RTT estimator before the first live sample (useful when
INDUCTION is skipped, and potentially for high-RTT links generally).
The review didn't mention it — should I keep it inside the virtual
transport bind configuration, or split it out as a separate PR for
general sockets?

@ethouris

ethouris commented Nov 2, 2026

Copy link
Copy Markdown
Collaborator

Ok, so responding to your questions:

  • Every srt_bind_virtual() gets a dedicated multiplexer — virtual
    multiplexers are not reuse candidates for new bindings (accepted
    sockets still join the listener's multiplexer through the normal
    accept path). This can be relaxed to same-settings sharing later if
    you prefer.

The most important now is that the multiplexer sharing works correctly
on the listener (as it wouldn't work otherwise); sharing on caller is
not that important, and can be added later, if needed.

  • The resolution moves into CChannel. I plan to make sendto/recvfrom
    virtual with CChannelUDP / CChannelVirtual subclasses — the dynamic
    allocation is already there and the XXX comment in
    CMultiplexer::configure seems to anticipate exactly this. The virtual
    dispatch also keeps the stock UDP path untouched. If you'd prefer the
    both-members-and-branch variant instead, I can do that too.

As I said, from the design point of view both are acceptable. Important for
this choice is performance.

  • Skipping INDUCTION becomes part of the bind configuration, not a socket
    option. The synthetic binding currently synthesized in listen()
    disappears — a listener binds explicitly via srt_bind_virtual().

Ok, note that we have also considered using conclusion-only handshakes
for specific configurations, however the sole mechanism for it is not
to be freely settable by the user through the API, but to be set internally
for various purposes. So it would be nice that the mechanism as such
remains available, but only through this internal function, not through
any API call.

One thing I'd like to confirm before I implement: datagram injection. I'd
make it go through the socket — srt_virtual_transport_inject(SRTSOCKET, ...), resolving socket → multiplexer → channel, with no extra handles or
reference counting (lifetime tied to the multiplexer). Acceptable?

Imagine this the same way as you would replace the UDP socket with a
pipe descriptor, and passed to srt_bind_acquire (this isn't possible
actually because this function does check if the passed descriptor is
a socket by doing socket-only operations on it), then you feed this in.

What you are providing for SRT is a replacement for UDP socket; normally
the UDP socket is operated by the operating system, and this one provides
it with packets to read and accepts packets to send. Here you need to
replace this complete mechanism, while providing a functional equivalent
API.

So, you likely need:

  • A structure that contains all the data: specific settings and callbacks:
    • to send the data
    • to wait for reading, with timeout (see how it is done when reading from socket)
    • to deliver data, once read-ready (including reporting broken medium)

(Not sure about sender readiness check - likely it should suffice if this
situation is reported by some returned error, but it should be also handled
carefully inside; broken medium should result in a broken connection and
the SRT socket).

SRT only needs the right callbacks provided to be able to use this thing.
Everything else should be programmed outside SRT.

In other words, the mechnism to "deliver" the wrapped packet by pushing
it into the queue and informing about readiness through the CV should be
a separate facility; a model for it can be added, but rather in a separate
directory, compiled to a separate library (likely only static one is
necessary - mind that "srt_virtual" is a reserved name already in the cmake
as a method to compile both static and shared SRT library; something like
"libsrttransport.a" on Linux should be ok I think). This can then use
the configuration structure for binding to virtual transport and filling it with
appropriate data, while providing an object - no matter how implemented - that
will facilitate the intermediate packet delivery ("injection"); this can be
simply provided so that applications using this mechanism can be created
easier, but it shouldn't be a part of the SRT library. Note that theoretically
this can be implemented using various underlying libraries, not necessarily
the C++ standard library (don't forget that SRT still has a limitation to
C++98 standard, and you broke it by not using exclusively the srt::sync wrapper -
if you create a separate library for transport only, as an optional component, it can
be also free of that limitation).

And one scoping question: the PR also adds SRTO_INITIAL_RTT, an optional
seed for the RTT estimator before the first live sample (useful when
INDUCTION is skipped, and potentially for high-RTT links generally). The
review didn't mention it — should I keep it inside the virtual transport bind
configuration, or split it out as a separate PR for general sockets?

That's a kind of problem. SRT does have a mechanism for this - see the CCache
class (plus the underlying CInfoBlock), however not accessible through the
API. The reasoning for it (can't speak for myself as this was done already
in the original UDT project) was to keep the remembered data from the used
links so that they can be restored later when connecting to the same endpoint.
Not sure if this mechanism can be reused here. Still, from the API point of
view this is another thing to be done on the transport descriptor object.
The CInfoBlock object in SRT is used at the connection point: in postConnect()
and acceptAndRespond(), which is the last phase of the connection on both
sides. Likely the same mechanism would have to be reused here, just the
mechanism of indexing and searching of the data would have to be extended
to handle also the virtual transport specification. At least you have already
a certin binding to the multiplexer and you can check things at this point.

NOTE: this design is a bit flawed (I have some plan prepared to fix it, but
we have limited resources) and this thing is remembered PER SOCKET simply
because even if two sockets share the same multiplexer, they can be connected
to a different endpoints and therefore will send packets to a different
address, even though they use the same UDP socket for sending. With virtual
transport this would likely have to be done differently, but if you state
that such multiplexers need not be shared, it should suffice if you ask
the multiplexer about the medium type, and if it's not UDP, you will take
these data directly from the virtual transport configuration object,
otherwise you'll extract them the old way.

What I had in mind to redesign it is that a single facility to send to
a certain remote address is managed by a "pipe manager", while you may
also have separate pipe managers, each of which separately collects the
data concerning bandwidth and RTT (as they are separate network links)
and SRT sockets directly only them; then a multiplexer can contain
multiple pipe managers, while on the other hand multiple SRT sockets may
also share pipe managers, independently on sharing multiplexers. But
that's not how UDT was designed and we had no possibility earlier to change
it, hence for now all the link-related data are kept in the socket,
even if potentially multiple sockets may share not only a single UDP
socket, but also the exact UDP link (packets will be sent to the same
endpoint using the same UDP socket). Likely however, you'll have to
distinguish sharing the medium from sharing the network link, and if the
mechanism of sharing the network link doesn't sound useful for this case,
you should avoid it - meaning:

  • caller sockets cannot share multiplexers with virtual transport (as now)

  • listener sockets shall reject a connection from the address that this
    multiplexer is already handling

If you decide to do it this way, at least you'll get rid of the problem of
shared links and you can safely state that the socket shall carry the
characteristic data of the network link. Thefore, the link measurement
data can be restored directly to the data in the socket, as it is now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants