Skip to content

[Bug🐛] Investigate the Broker CPU cycle increase for synchronous sends #12873

Closed

Description

@mxsm

Prerequisites

  • I have searched existing issues and discussions
  • This is a bug, not a question

Environment

Linux x86-64 under WSL2 on an Intel Core i7-11700K. The Broker was built with Rust 1.95.0, release PGO, BOLT and mimalloc. Both variants used the same NameServer and client.

Bug Description

The CheetahString 3.2 send-path comparison recorded 395,912 versus 479,884 Broker CPU cycles per successful synchronous message in a two-pair follow-up. Retired instructions changed by less than one percent. Single-request performance is sensitive to this difference, and the original comparison did not isolate a cause.

Steps to Reproduce

  1. Compare the Broker at 391ed3e31 with the first six changes through [ISSUE #12833]✨Encode numeric message properties from stack buffers #12844. Use the same Rust PGO input and train BOLT separately for each executable.
  2. Start a fresh loopback Broker and NameServer for each run. Pin the Broker to logical CPUs 0-7 and the fixed client and NameServer to 8-15.
  3. Warm the topic, then issue 12,000 synchronous 1 KiB sends with one outstanding request. Compare Broker CPU counters over the successful client measurement window in alternating order.

Expected vs Actual

The comparison should identify whether the code changes increase the Broker cost of a completed send. The two follow-up pairs both favored the baseline, while three of the four original pairs favored the candidate. Whole-process cycles alone could not explain the different outcomes.

Logs/Screenshots

The four original pairs averaged 399,055 and 453,970 cycles per message. The largest individual change was approximately 369,000 to 681,000 cycles. Original PR

Additional Context

Investigate with identical-binary controls and controlled comparisons of the pre-BOLT and post-BOLT executables. Record successful request counts and variance, identify a causal intervention where reproducible, and verify the correction against the synchronous workload.

Activity

  1. added a commit that references this issue on Nov 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bug🐛Something isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions