-
Notifications
You must be signed in to change notification settings - Fork 5
Configuration Reference
Options are scoped the way Netty's are: options configure the thing you
created, child_options configure each session a listener accepts. A client has
no children, so its options are the session's.
void ConfigureServer() {
// no ConnectionType, so this is ZDT, the default
ServerConfig config{"0.0.0.0", 25000, std::chrono::seconds(10)};
// options: the listener itself
config.options.max_connections = 4096;
// child_options: every session the listener accepts
config.child_options.common.idle_timeout = std::chrono::seconds(30);
config.child_options.common.encryption = true;
config.child_options.common.compression = CompressionType::Zstandard;
config.child_options.common.compression_threshold = 128;
config.child_options.common.send_queue_capacity = 1024;
config.child_options.zdt.max_datagrams_in_flight = 256;
config.child_options.zdt.rto_min = std::chrono::milliseconds(50);
config.child_options.zdt.mtu_ladder.Set({1492, 1200, 576});
}
void ConfigureClient() {
// a client has no children, so its options are the session's
ClientConfig config{"127.0.0.1", 25000, std::chrono::seconds(10)};
config.options.common.idle_timeout = std::chrono::seconds(30);
config.options.common.keepalive_interval = std::chrono::milliseconds(500);
}SessionOptions holds two groups, common and zdt. There is no TCP group: a
TCP session reads common and nothing else. zdt is always present and always
settable, and setting it on a TCP session is neither an error nor a warning, so
it is dead configuration that looks live.
They are plain structs, not a typed-key map: every option is known at compile time, so unset fields simply keep their defaults.
| Field | Notes |
|---|---|
server_address / server_port
|
Where to connect. ClientConfig
|
bind_address / bind_port
|
Where to listen. ServerConfig; a zero port lets the system pick, read back with bind_address()
|
connection_timeout |
How long a session may sit in its handshake before it is dropped: the client's own on ClientConfig, each accepted one on ServerConfig. Zero waits forever |
connection_type |
ConnectionType::ZDT (the default) or ConnectionType::TCP
|
options |
Listener scope on a server; session scope on a client |
child_options |
Server only: applied to each accepted session |
connection_timeout is counted from the moment the session exists. A ZDT
client's offline handshake, the cookie exchange and the MTU probe, runs inside
Connect() before that point and is bounded by handshake_retransmit,
handshake_retries_per_rung and the ladder instead; its failures come back as
Connect()'s Result rather than as an event.
The address string also accepts a Unix domain socket path, spelled
unix:/run/app.sock. That requires ConnectionType::TCP (it is the stream
backend; ZDT refuses paths), the port is ignored, and it is POSIX only. The
server takes over a stale socket file on bind and unlinks it on close.
znet/inet_addr.h has the three lookups that fill an address field in, each
taking an InetProtocolVersion:
| Call | Returns |
|---|---|
GetAnyBindAddress(v) |
0.0.0.0 or ::, the wildcard a server binds to |
GetLocalAddresses(v) |
Every address this host is reachable at, loopback last. Never empty |
GetLoopbackAddress(v) |
127.0.0.1 or ::1, and nothing else |
void BindEverywhere() {
using znet::InetProtocolVersion;
znet::ServerConfig config{znet::GetAnyBindAddress(InetProtocolVersion::IPv4),
25000, std::chrono::seconds(10)};
// every address a peer could dial this host at, loopback last
for (const std::string& ip :
znet::GetLocalAddresses(InetProtocolVersion::IPv4)) {
ZNET_LOG_INFO("reachable at {}:{}", ip, config.bind_port);
}
}GetLocalAddresses walks the host's own interfaces and keeps the ones a peer
could plausibly dial. Interfaces that are down, link-local addresses
(169.254.0.0/16, fe80::/10) and container or hypervisor bridges (docker,
br-, veth, virbr, vmnet, vboxnet, matched by interface name) are left
out. VPN interfaces are kept, since reaching a peer over one is a real case. At
most eight routable addresses come back, loopback appended last, so the result
is never empty even where enumeration fails outright.
There is deliberately no "the local address" call. On a multi-homed host any single answer is a guess, and a wrong guess is invisible until a peer cannot reach you. Where one address has to be chosen, choose it from the list yourself; where a peer has to reach you, hand it the whole list.
Applies to any session, whatever the transport.
| Option | Default | Notes |
|---|---|---|
idle_timeout |
10 s | Drop a session that has heard nothing this long. Both transports implement it. Zero disables |
keepalive_interval |
1000 ms | Ping a connection with nothing else to send, keeping it inside the peer's idle_timeout. Transport-internal; never reaches the application. Zero disables |
encryption |
true |
Read only on the accepting side. See Encryption and Compression |
compression |
Default |
Accepting side only, like encryption. Default resolves at session start to whatever the build supports |
compression_threshold |
128 B | Below this, messages go uncompressed |
send_queue_capacity |
512 | Packets a session holds for its worker to encode |
dump_on_decode_failure |
false |
Log a hex dump of a payload whose frame fails to decode, capped at 512 bytes |
max_invalid_frames |
16 | Close a session once this many of its frames failed to decode. Zero disables |
A frame that fails to decode costs the rest of its buffer, since the framing
after it cannot be trusted. This threshold is what stops a peer from making
that a free, repeatable attack. It counts over the session's whole life:
unreadable headers, declared lengths the buffer cannot back, serializers
refusing a frame or reading past their frame. Unknown packet ids are not
counted, since they skip cleanly and can be honest version skew. The count is
visible as invalid_frames in Metrics and via
session->invalid_frames().
dump_on_decode_failure logs the evidence when one happens: the offset of the
failing frame and the payload bytes. Off by default because payloads are user
data and this puts them in the log.
This is the backpressure knob. SendPacket returns Result::QueueFull once the queue is
full, which is how an application learns it is outrunning the link. A refusal
loses nothing, since you still hold the packet.
The queue is a ring allocated whole at construction, roughly 32 bytes a slot rounded up to a power of two, with no per-message allocation afterwards. Size it to the largest burst worth absorbing between two of the worker's ticks, not to the total you ever expect to send. A transport queues further messages behind its own congestion window, so this is not the whole picture.
Read only when connection_type is ConnectionType::ZDT, which is the
default. Ignored on a TCP session.
| Option | Default | Notes |
|---|---|---|
rto_min |
100 ms | Floor on the retransmit timeout, however low the measured RTT |
rto_max |
2000 ms | Ceiling, including backoff |
max_retries |
10 | Retransmits of one message before the connection is closed |
Lowering rto_min recovers from loss faster on a LAN. On the open internet it
causes retransmits of packets that were merely late, which costs bandwidth and
can push the congestion controller down.
| Option | Default | Notes |
|---|---|---|
max_datagrams_in_flight |
512 | Ceiling on the congestion window, in datagrams |
max_messages_in_flight |
4096 | Reliable messages allowed in flight. A memory bound, not congestion control |
max_datagrams_in_flight is a bound, not the window. ZDT slow-starts from
10 datagrams and backs off on queueing delay rather than on loss; this caps how
far it may grow. It counts datagrams, not bytes, so a half-full datagram costs
as much as a full one. Bytes in flight are roughly this times the MTU per round
trip, which is what governs throughput on a long link.
The two are separate because coalescing puts many messages in one datagram.
Holding max_messages_in_flight near max_datagrams_in_flight would throttle small messages far
below what the window actually allows.
| Option | Default | Notes |
|---|---|---|
enable_connection_migration |
false | Stamp every datagram with an 8-byte connection id so a peer can be found again after its source address changes |
Off by default, and both ends must set it. It lays the wire for keeping a session alive across a network change: the id is this endpoint's guid, and it costs 8 bytes of MTU. The routing that acts on it, path-validating a new address before the send target follows it, is not built yet, so today a receiver reads the id and ignores it.
| Option | Default | Notes |
|---|---|---|
mtu_ladder |
1492, 1200, 576 | Candidates, probed largest first. Set with .Set({...}), max 4 |
handshake_retransmit |
250 ms | Wait for a reply before resending |
handshake_retries_per_rung |
4 | Attempts at one MTU before stepping down |
The settled MTU determines the largest message the connection can carry. See Choosing a Transport.
| Option | Default | Notes |
|---|---|---|
cookie_secret_rotation |
120 s | How often the server rotates its return-routability secret |
max_connections |
4096 | Established connections a server holds. Zero means unlimited |
per_source_handshake_rate |
20/s | Handshake messages accepted per source address |
reassembly_timeout |
5 s | Discard a partial message after this |
max_reassembly_bytes |
16 MiB | Ceiling on bytes held in partial reassemblies |
max_reassemblies |
256 | Concurrent partially reassembled messages |
max_inbox_datagrams |
4096 | Raw datagrams queued per connection before arrivals are dropped |
outbound_queue_capacity |
4096 | Encoded messages the transport holds before Send() fails |
cookie_secret_rotation, max_connections and per_source_handshake_rate
belong to the endpoint rather than to one session, as the socket buffers below
do. ServerOptions has no ZDT group, so the backend takes them from
child_options.zdt when the server is constructed and applies them to the
listening endpoint as a whole.
Reaching max_reassembly_bytes refuses to start new messages rather than
discarding data already accepted, so a peer stalls instead of losing anything.
outbound_queue_capacity sits past the point where a caller can be told to try
again: the message is already encoded, so a refusal here drops it rather than
pushing back the way a full send_queue_capacity does. That is why its default
is generous rather than tuned. The transport also keeps a staging queue, where a
shut congestion window parks messages, and refills from the ring only while it
holds fewer than this many, so it can hold up to twice this figure in total.
| Option | Default | Notes |
|---|---|---|
socket_recv_buffer |
4 MiB |
SO_RCVBUF on the UDP socket. Zero leaves the OS default |
socket_send_buffer |
4 MiB |
SO_SNDBUF, on the same terms |
One socket serves every connection on an endpoint, so these are per endpoint,
not per session. They only have to absorb bursts that arrive between drains by
the receive thread; max_inbox_datagrams is the designed backpressure point,
not these.
Both are best-effort. The kernel clamps silently to its own ceiling
(net.core.rmem_max and net.core.wmem_max on Linux, often well below 4 MiB
out of the box), so asking for more than it allows is not an error and warns
about nothing. znet reads the granted size back and logs both at debug level:
ZDT socket buffers: asked 4194304/4194304, granted 425984/425984 (recv/send bytes, ...)
Raise the sysctls if the larger ask has to take effect. Going much beyond what covers a scheduler stall buys little: the receive thread's drain rate is the bottleneck, and a very deep kernel queue makes every session wait behind a flood rather than letting the per-session inbox caps drop it fairly.
Listener scope: things that exist before any session does.
| Option | Default | Notes |
|---|---|---|
backlog |
0 | Pending-connection backlog. Zero uses SOMAXCONN. TCP only |
max_connections |
0 | Cap on concurrent sessions, refused at accept. Zero means unlimited |
reuse_address |
true |
SO_REUSEADDR on the listening socket. TCP only, and skipped on a Unix socket; ZDT's UDP socket always sets it |
allowlist |
empty | Sources allowed to connect, as CIDRBlocks. Empty admits everyone the denylist does not refuse |
denylist |
empty | Sources always refused. Wins over the allowlist |
max_attempts_per_source |
0 | Connection attempts one source IP may make per attempt_window. Zero disables |
attempt_window |
10 s | The window the attempt count runs over |
Note that ZDTOptions::max_connections and ServerOptions::max_connections are
different settings: the ZDT one is enforced by the transport before a session
exists, the listener one by the server.
The lists and the throttle are checked when a connection arrives: at accept on
TCP, at first contact on ZDT, where refusal is a silent drop so an excluded
source learns nothing. Rules are CIDRBlocks:
config.options.denylist.push_back(znet::CIDRBlock::Parse("203.0.113.0/24"));
config.options.allowlist.push_back(znet::CIDRBlock::Parse("10.0.0.0/8"));
config.options.max_attempts_per_source = 10;A bare host parses as its /32 (or /128). IPv4 rules also match IPv4-mapped
IPv6 sources, which is what a v4 client looks like to a dual-stack listener.
Invalid blocks are dropped with an error at server construction, not enforced
half-parsed. Unix socket listeners have no source address and bypass all of
it; the socket file's permissions are their gate. On ZDT the attempt throttle
counts every handshake-opening datagram, including retransmits of a lost one,
so leave slack above the honest rate. A ZDT refusal shows up as the
admission_rejected counter in Metrics; the TCP path has no counter
of its own and only logs the verdict at debug level.
Not part of the option structs: the loops take their rate from a call instead.
| Call | Default | Notes |
|---|---|---|
Client::SetTicksPerSecond(tps) |
120 | The client's loop. A TCP client blocks on its socket rather than pacing, so this only bites on ZDT |
Server::SetTicksPerSecond(tps) |
120 | Every worker. The acceptor runs its own 60/s loop, which is not settable |
A tick is cut short when a datagram arrives or an idle session is sent to, so raising either trades CPU for latency only where neither happens.
The P2P module has structs of its own, all experimental. Peer-to-Peer explains the flow they configure.
| Option | Default | Notes |
|---|---|---|
bind_address |
"0.0.0.0" |
The punch socket |
bind_port |
0 | Zero picks an ephemeral port; punch_port() reads it back |
session_options |
SessionOptions{} |
Every punched session is built with these |
What Host::Punch takes; a locator fills it from the rendezvous.
| Field | Default | Notes |
|---|---|---|
candidates |
The peer's, every one raced at once | |
punch_id |
0 | Broker-issued, the same on both sides; IsInitiator derives the tiebreak from it |
is_initiator |
false | Exactly one of the two peers passes true |
timeout |
5000 ms | For the punch, then again for the handshake |
relay_delay |
1000 ms | How long the direct candidates have the race to themselves. A relayed candidate is bound from the start but carries punch traffic only after this, so a working direct path always wins |
| Option | Default | Notes |
|---|---|---|
server_address, server_port
|
The rendezvous | |
host |
HostConfig{} |
The punch socket, as above |
gather_timeout |
2000 ms | How long the reflectors get before the gathering goes out with whatever came back |
punch_timeout |
5000 ms |
PunchOffer::timeout for every punch |
relay_delay |
1000 ms |
PunchOffer::relay_delay for every punch |
tcp::PeerLocatorConfig has server_address, server_port and a
punch_timeout (5000 ms) only.
| Option | Default | Notes |
|---|---|---|
bind_address, bind_port
|
"0.0.0.0", 5001 |
The rendezvous link, always TCP |
punch_connection_type |
ZDT | The punched connection's transport, decided here so both peers agree |
options |
ServerOptions{} |
The listener's admission rules |
max_requests_per_window, request_window
|
30, 10 s | Gatherings and asks one client may send per window; over it, the client is dropped. Zero disables |
relay_enabled |
false | Run a RelayServer alongside |
relay |
RelayServerConfig{} |
Its configuration |
relay_host |
"" |
The host peers reach the relay at; empty means the host they reached the rendezvous at. Resolved once at Start(), which fails with InvalidAddress if it does not |
extra_reflectors |
{} |
Reflectors advertised beside the embedded relay, each an InetAddress. A second one on a distinct IP is what lets a peer tell a symmetric NAT from a punchable one; without it every gather reports Unknown. Capped at kMaxReflectors
|
| Option | Default | Notes |
|---|---|---|
bind_address |
"0.0.0.0" |
The relay socket |
port |
5002 | The one UDP port everything happens on: binds, relayed datagrams and Reflect. Zero picks an ephemeral one, read back with address()
|
bind_timeout |
15 s | A pairing nobody has bound both sides of is freed after this |
idle_timeout |
30 s | A bound pairing with no traffic is freed after this. ZDT keepalives keep a live session well inside it |
max_allocations |
4096 |
Allocate() refuses with Result::ServerFull beyond this. Pairings cost memory, not ports, so this is the only bound |
max_probes_per_source, probe_window
|
30, 10 s |
Reflect probes one source may send per window; beyond it they are dropped. Zero disables |