Repository navigation
Start five more browser tests' setup before the tests, not inside a timed hook - #1723
Merged
Merged
Conversation
davidmckayv
requested review from
MikeRyanDev,
dustywusty,
guidovizoso,
mxmzb,
samjulien and
tylerslaton
as code owners
November 16, 2026 18:34
…ot in a timed hook A beforeAll gets 5 s. In a2ui-frame, update-banner-layout, composer-dictation-notice-layout, checkbox-contrast and conversation-intro-halo the setup bundles a page or builds a stylesheet and starts Chromium, which can take longer than that on a cold runner when the file is the first one bun test opens. The setup now runs at the top of each file and waits for the bundle and for Chromium's launch, which Playwright bounds itself, as transcript-rail-clearance does.
davidmckayv
force-pushed
the
fix/browser-test-setup-flake
branch
from
November 16, 2026 18:57
26e1990 to
c8d85b5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The same fix as #1720, applied to the five other real-browser tests whose setup ran inside a
beforeAll:a2ui-frame,update-banner-layout,composer-dictation-notice-layout,checkbox-contrastandconversation-intro-halo.Cause
A
beforeAllgets bun's default of 5 s. Each of these files bundles a page or builds a stylesheet in that hook, and four of them also start Chromium there. On a cold runner, when the file is the first onebun testopens, that can take longer than 5 s. The file then fails witha beforeEach/afterEach hook timed outbefore it measures anything.transcript-rail-clearancefailed that way on main.Fix
The setup now runs at the top of each file, before any test. It waits for the bundle (or the stylesheet) to be built and for Chromium to report it has started. Playwright's own launch timeout bounds that wait. No hook clock and no larger number replaces the old limit.
checkbox-contrastonly bundled in its hook, so only the bundle moves. Its browsers still start inside each test, within that test's own two minutes.Evidence
Each run is a fresh
bun testof one file alone, so it is the first file opened. The runs were on a 4 vCPU x86 Ubuntu 24.04 box (the shape ofubuntu-latest), with the page cache dropped before each run and the CPU held down withsystemd-run -p CPUQuota=. Before is main at 7b4968a; after is this PR.a2ui-framecomposer-dictation-notice-layoutupdate-banner-layoutcheckbox-contrastconversation-intro-haloa2ui-frameandcomposer-dictation-notice-layouttime out in setup the same waytranscript-rail-clearancedid. The other three have lighter setups and did not time out here. Their setup ran under the same 5 s limit, though, so they get the same pattern rather than waiting to cross it on a slower runner. Every failure before the fix was a hook timeout. After it, nothing failed.In CI on this head, all five files ran and passed in the "app (real browser)" job.
biome format,biome lintand the app'stscpass. WithoutOPENBOT_APP_BROWSERevery file still skips.