-
Notifications
You must be signed in to change notification settings - Fork 5
Threading Model
znet owns its threads. You never start one, and your code runs on them rather than on the thread that created the server or client. This page is the contract.
Client: two, plus whatever the transport runs.
| Thread | Runs |
|---|---|
| Loop | Drives the session: reads, dispatches your handlers, flushes. Also encodes a shallow queue itself |
| Encoder | Takes over encoding once the queue is deeper than one, so it overlaps the flush |
Server: one acceptor plus a worker pool.
| Thread | Runs |
|---|---|
| Acceptor | Accepts connections, drives sessions until their handshake finishes, then hands each to a worker |
| Worker | Drives its assigned sessions: reads, dispatches your handlers, encodes and flushes |
A ZDT server adds a receive thread that takes datagrams off the shared UDP socket and routes them to the right session.
P2P: one thread for everything on a host, or one per TCP-punched session.
| Thread | Runs |
|---|---|
| Host tick |
p2p::Host reads its one UDP socket, drives the gathers and punches still in flight, and does the reads, handlers, encodes and flushes of every session it punched |
| Session | A session p2p::tcp::PunchSync produced drives itself the same way |
| Relay |
p2p::RelayServer runs one thread that polls its sockets; Allocate() and metrics() are safe from any thread |
All of them nap after an idle pass rather than spinning. A locator's own
events come from elsewhere again. p2p::PeerLocator fires
PeerLocatorReadyEvent and PeerConnectedEvent from the host tick, since
the gathering and the punch resolve there, and the Link, Exchange and close
events from the link client's loop. tcp::PeerLocator fires the name and an
Exchange failure on the link client's loop and its punch outcome from an
internal worker. See Peer-to-Peer.
Workers get sessions by least-loaded assignment at promotion. A session stays with its worker for life, so all of one session's callbacks are serialized. But a session changes threads once, from the acceptor to its worker, when its handshake completes.
| Your code | Called on |
|---|---|
| Event callback | The thread that owns the session: a worker, the acceptor for a not-yet-promoted session, the client's loop, or a p2p host's tick |
OnPacket handlers |
The same |
SerializeTyped |
Whichever thread encodes: a server worker, or on a client either its loop or its encoder |
DeserializeTyped |
The session's owning thread |
Which of the client's two threads encodes depends on queue depth: one queued packet is encoded on the loop, since waking the encoder to save a single serialize costs more than it saves, and anything deeper goes to the encoder. Only one thread encodes a given session at a time, so a serializer needs no lock, but it must not assume a fixed thread.
One session never runs two callbacks at once. You do not need a lock to protect state belonging to a single session.
Different sessions run concurrently, on different workers. Anything shared
across sessions, a room list, a player registry, a shared codec, is touched
concurrently and needs its own synchronization. A shared Codec is safe because
serializers are stateless and it is not mutated after setup; mutating one while
sessions are live is not. A mesh is the exception: every session a p2p::Host
punched shares its one tick thread, so those are serialized against each other.
Your handler blocks its session's thread. On a server that thread is also driving other sessions, so a slow handler delays them too. Hand long work to your own thread pool.
A session can move between threads once. The acceptor drives it during the handshake; a worker drives it afterwards. Since the two never overlap, ordinary non-atomic state is fine. But code that latches a thread id at construction and asserts it later will be wrong.
SendPacket is safe from any thread. It only queues, and the queue is
lock-free precisely so a game loop calling it never blocks behind a worker
mid-encode. Check the Result it returns: Success means queued, QueueFull
is backpressure and the caller still holds the packet, NotReady means the
handshake has not settled, NotConnected that the session is gone. Every
refusal happens before encoding, so nothing is lost.
Close is safe from any thread.
SetCodec and SetHandler are meant to be called from the session's own
thread, which is what you get inside the connected event or inside an
OnPacket. Calling them from elsewhere while the session is live races the
thread that reads them. They also want a session that is already ready, since
the handshake holds both until it finishes; every connected event hands you
one, PeerConnectedEvent included, as Peer-to-Peer explains.
metrics() returns a copy, but the counters it copies are plain members written
by the owning thread with no atomics, and it samples live transport state on the
way past. Reading it from another thread is a data race: you get stale values
rather than torn ones on the platforms znet targets, but a sanitizer will call
it, correctly. Sample from the session's own thread, which is what a handler or
a tick callback already gives you. See Metrics.
Messages sent on one session arrive in the order you called SendPacket, per
channel, when the delivery mode says ordered. Sends from two threads to the same
session interleave in whatever order they reach the queue. The queue preserves
order, but "which came first" is decided by your threads, not by znet.
If ordering between two messages matters, send them from the same thread.
Sessions are shared_ptr. Holding one past a disconnect is safe: it stays valid
and IsAlive() returns false. This matters most in P2P, where the locator
returns and you keep the session it produced.
Destroying a Server or Client stops and joins its threads, so no callback can
be running once the destructor returns. Do not destroy either from inside its own
callback. A p2p::Host, both locators and a p2p::RelayServer do the same,
so Host::Stop and PeerLocator::Disconnect are off limits inside a gather
or punch callback or a locator event too; Host::Stop also resolves every
gather and punch still outstanding with Result::AlreadyStopped rather than
dropping the callback, including one queued while it runs.
Debug builds check ZDT's thread rules at runtime: entering the transport's
update, flush or receive path from two threads at once aborts with
two threads entered one ZDT thread domain at once. These compile out entirely
in release, since the guard member itself is #ifndef NDEBUG, so object layout
is unchanged.
If you are doing something unusual with threads, run your tests against a debug build of znet at least once.