Skip to content

run each Espresso submission under an errgroup - #525

Open
jjeangal wants to merge 2 commits into
espresso/simplify-worker-poolfrom
espresso/worker-pool-errgroup
Open

jjeangal wants to merge 2 commits into
espresso/simplify-worker-poolfrom
espresso/worker-pool-errgroup

Conversation

@jjeangal

Copy link
Copy Markdown
Collaborator

Part of #506. Stacked on #524 (base: espresso/simplify-worker-pool): review/merge that one first.

What

#524 removed the hand-rolled load balancer. This goes the rest of the way: it replaces the whole submit/verify worker-pool machinery with one goroutine per transaction, bounded by an errgroup.

  • SubmitTransaction calls eg.TryGo(...), which runs submitAndConfirm for that transaction in its own goroutine.
  • submitAndConfirm runs the full lifecycle inline: submit-with-retry, then poll-for-receipt-with-retry; a verification timeout loops back to re-submit.
  • evaluateSubmission / evaluateVerification become pure functions, so they're directly unit-testable.

This matches the spec's wording ("for each block, spawn a goroutine that submits the batch and waits for finalization").

Removed

4 job/response structs, 6 channels, both scheduler/handler goroutines, both worker functions and SpawnWorkers, the atomic in-flight counter, and the queue-capacity config. Net ~−228 lines on top of #524.

Behavior

Unchanged: fast-reject backpressure (ErrTooManyInFlightRequests via TryGo, never blocks), the retry/re-submit logic, SubmitTransaction's signature. The queueing loop still treats a reject as "retry next tick," so no blocks are dropped.

Two deliberate changes for a reviewer's eye:

  1. Submit and verify share one in-flight limit (was two pools of 4). A tx holds one slot through both phases; limit is 128, ample for verify depth. Two-limit split (option B) deferred as a follow-up if monitoring shows submit starvation.
  2. Submit retries back off by retryDelay (100ms) instead of immediately. The old immediate retry was throttled by the fixed worker count, which no longer exists.

Also: the higher concurrency ceiling (up to 128 simultaneous SDK calls vs the old ~8) is only approached under sustained backlog; in steady state only a handful of goroutines are alive.

Testing

Commands:

go build ./op-batcher/...
go test ./op-batcher/batcher/...          # all batcher tests
go test ./op-batcher/batcher/... -race    # again under the race detector

All pass, including under -race.

@jjeangal jjeangal self-assigned this Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant