Skip to content

feat(acceptor): count early-closed and forbidden sockets - #230

Merged
zmstone merged 2 commits into
masterfrom
260911-early-close-counters
Sep 11, 2026
Merged

zmstone merged 2 commits into
masterfrom
260911-early-close-counters

Conversation

@zmstone

@zmstone zmstone commented Sep 11, 2026 •

Copy link
Copy Markdown
Member

The acceptor closes some sockets before a connection process starts. It counted most of them as closed_other_reasons. This PR splits out two new accept-result counters, which esockd:get_stats/1 returns:

  • closed_early: the socket failed with econnaborted, econnreset, enotconn, einval or closed before the connection process was started. This most likely means the peer closed or reset it. Scanners, health checks and load balancers usually cause these.
  • closed_forbidden: the access rules denied the peer address.

closed_other_reasons now only counts ignore, connection start errors and crashes. A rise in this counter now points at a real problem.

Related fixes

  • esockd_server:inc_stats/3 and dec_stats/3 no longer crash the caller when the metric row is missing. They pass a default row to ets:update_counter/4. Without this, an acceptor on a listener started before the new counters existed would crash with badarg on its first early close.
  • The acceptor no longer counts {error, closed} from an async accept. That error means the listen socket is closed, not a client socket. The sync accept path already skipped the count.

Tests

  • esockd_acceptor_fsm_SUITE: t_closed_early, t_forbidden and t_tune_fun_other_error. t_einval now expects closed_early. The listen-socket-closed case asserts that no close counter moves.
  • esockd_SUITE: t_closed_forbidden uses a real deny rule. t_closed_early resets the client connection while the acceptor waits in tune_fun.
  • esockd_server_SUITE: t_inc_uninitialized_stats.

Follow-up: EMQX only reads esockd:get_shutdown_count/1 today. A separate EMQX PR will expose the accept-result counters in the CLI and in Prometheus.

`ets:update_counter/3` raises `badarg` when the row does not exist.
The acceptor then crashes on its first update of a metric that the
listener did not initialize, e.g. a metric added after the listener
started. Pass a default row to `ets:update_counter/4` instead.

Claude-Session: https://claude.ai/code/session_01NZuG3jQw5LtYcbUao51j5D
The acceptor counted most sockets that it closed before a connection
process started as `closed_other_reasons`. Split out two counters:

- `closed_early`: the socket failed with `econnaborted`, `econnreset`,
  `enotconn`, `einval` or `closed` before the connection process was
  started. This most likely means the peer closed or reset it.
- `closed_forbidden`: the access rules denied the peer address.

`closed_other_reasons` now only counts `ignore`, connection start
errors and crashes.

Also stop counting `{error, closed}` from an async accept. It means
the listen socket is closed, not a client socket.

Claude-Session: https://claude.ai/code/session_01NZuG3jQw5LtYcbUao51j5D
@zmstone
zmstone requested review from keynslug and qzhuyan and removed request for keynslug September 11, 2026 11:10
@zmstone
zmstone marked this pull request as ready for review September 11, 2026 11:10
@zmstone
zmstone merged commit cd61fc0 into master Sep 11, 2026
13 of 14 checks passed
@keynslug keynslug mentioned this pull request Sep 30, 2026
2 of 3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants