Skip to content

test(stack): fix 1 flaky test in the Mailpit launch check - #7318

Open
avallete wants to merge 2 commits into
developfrom
avallete/flaky-fixes-2026-10-24
Open

avallete wants to merge 2 commits into
developfrom
avallete/flaky-fixes-2026-10-24

Conversation

@avallete

Copy link
Copy Markdown
Member

TL;DR

The Mailpit launch test hung on Podman without saying where. It now fails within its timeout, naming the phase that stalled and Mailpit's output.

  • Mail.integration.test.ts › launches the real Mailpit recipe and serves its HTTP endpoint: on Podman, the test hit the 30s test timeout with no diagnostic. The error frame pointed at sendTestEmail, but this test never sends mail, so that frame is misleading. The test waits on start, readiness and a /readyz request. Container readiness may wait up to 60s, longer than the test timeout, and the /readyz request has no bound. So a stall in any of the three looked the same. The test now gives each phase its own bound inside the 30s budget (start 20s, readiness 6s, /readyz 3s). It also collects the recipe's log stream, so a stall fails with MailStallError naming the phase and showing Mailpit's output so far.

ref

spotted in:


Generated by Claude Code

The launch test now bounds start, readiness and the /readyz request inside
the test timeout and fails with the phase and Mailpit's output so far,
instead of a bare test timeout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EMySvvbM4B1zdmhw3pp1wA
@avallete
avallete requested a review from a team as a code owner October 24, 2026 07:43

Copy link
Copy Markdown
Member Author

/ai-review


Generated by Claude Code

@avallete avallete added the run-flaky-check Run the flaky-test check on this PR label Oct 24, 2026 — with Claude
@github-actions github-actions Bot removed the run-flaky-check Run the flaky-test check on this PR label Oct 24, 2026
@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Flaky test check: ✅ no flaky tests

Commit efdb2fa196c2 · focused ×10 (5 executions each), 6 tests · integration ×10, 5619 tests · unit ×10, 8333 tests · workflow run

Each execution is a separate Vitest process; Where lists failing runs, as run.iteration for suites executed more than once per run. The full data is in the flaky-check-report artifact.

@github-actions github-actions Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Superseded by a newer AI review

🤖 AI Review

Claude reported one potential test-flakiness issue; Codex reported none. The reduced readiness budget is verified, but whether it causes otherwise successful Mailpit runs to fail remains uncertain. Tests were not run.

Findings

Severity Location Category Sources Claim
🟡 MINOR packages/stack/src/services/Mail.integration.test.ts:107 test-flakiness claude The fixed six-second readiness timeout can reject slow Mailpit startups that would previously have completed within the overall 30-second test timeout, potentially increasing flakiness on Podman.

Stats

Claude findings: 1 · Codex findings: 0 · Confirmed: 0 · Refuted: 0 · Uncertain: 1


Models: claude-opus-5-5 + gpt-6.1-sol · Trigger: auto · Workflow run

This review runs once per PR. A maintainer can request another with a /ai-review comment.

Comment thread packages/stack/src/services/Mail.integration.test.ts Outdated

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Review

Both independent reviews completed. Confirmed Claude’s readiness-timeout concern and UTF-8 diagnostic decoding defect. Codex reported no findings. Neither issue changes production behavior.

Findings

Severity Location Category Sources Claim
🟡 MINOR packages/stack/src/services/Mail.integration.test.ts:107 test-flakiness claude The six-second readiness timeout can fail a healthy but slower container even when most of the test’s 30-second budget remains.
🟡 MINOR packages/stack/src/services/Mail.integration.test.ts:98 correctness claude Decoding every log chunk with a fresh TextDecoder corrupts UTF-8 characters split across chunks in the timeout diagnostics.

Stats

Claude findings: 2 · Codex findings: 0 · Confirmed: 2 · Refuted: 0 · Uncertain: 0


Models: claude-opus-5-5 + gpt-6.1-sol · Trigger: manual · Workflow run

This review runs once per PR. A maintainer can request another with a /ai-review comment.

Comment thread packages/stack/src/services/Mail.integration.test.ts Outdated
Comment thread packages/stack/src/services/Mail.integration.test.ts
@avallete avallete added the run-flaky-check Run the flaky-test check on this PR label Oct 24, 2026 — with Claude
@github-actions github-actions Bot removed the run-flaky-check Run the flaky-test check on this PR label Oct 24, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants