Repository navigation
ci: play every docs demo tour in a browser at phone and desktop width - #165
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The docs demos broke three times in a row in ways CI could not see: the tour spotlight drifting off its target (#162, #163) and the phone-width sideways scroll (#164). The
docsjob only ranmkdocs build --strict, and each fix was verified with throwaway Playwright scripts. This PR keeps that check in the repo and runs it in thedocsjob.just docs-demosbuilds the site, servessite/on a local port, and drives headless Chromium at 375x740 and 1280x800. On each demo page it plays every scenario's guided tour (and the retry page's herd tour). It checks the page on load, at every tour stop and after each run. A check fails if the page scrolls sideways, if the coach card is not fully inside the viewport once the stop has scrolled into place, if a tour stalls before reaching the stop count it shows ("Step 2 of 5"), or if the page throws. Demo pages are found in the built site, so a new demo page gets checked without anyone editing the list. One run takes about 30s locally, most of it the tours playing in real time, with all ten page and viewport pairs running in parallel.Design
scripts/run withuvx --with playwright==1.64.0. Keeping it out of pytest keeps Playwright and a browser out of the dev environment and the 11 pytest/floors jobs. The pin lives once, in the justfile, anddocs-demos-browserinstalls the matching Chromium, with--with-depsfor the runner's system libraries.[[tool.ty.overrides]]entry ignoresunresolved-importforscripts/**only, so the rest of the script is still type checked, but Playwright's own types come through as unknown. The other option was adding Playwright to thelintgroup, which every CI job installs.scripts/*joinsbenchmarks/*in the coverage omit list, becausetest-cimeasures--cov=.against a 100% floor and nothing imports the script.site/itself.mkdocs servewould also work, but its livereload connection keeps the page from ever going network-idle.Testing
On current main:
5 pages x 2 viewports checked, 0 problems.With #164's CSS change reverted (
white-space: nowrapback on.hw-demo .box), it exits 1 with 12 problems, all at 375px: circuit breaker on load, then at stops 1, 2 and 5 and after the run in both scenarios (scrollWidth 379), and full stack on load and at stops 1 and 2 (scrollWidth 379, and 406 at stop 1).To confirm the other checks fire, I also ran it with three temporary breakages: a 600px coach, a circuit breaker stop whose condition never fires, and a bulkhead page that throws on load. It exited 1 with 29 problems: the coach outside the viewport on every phone tour stop, "tour stalled after 2 of 5 stops" for both circuit breaker scenarios at both widths, and "page error: injected" for bulkhead at both widths.
just lint-cipasses, andjust test-cistays at 100% coverage with 1007 tests passing.