Skip to content

Add opt-in retry for caller-declared transient failures - #7

Merged
bigtiger merged 5 commits into
mainfrom
jr-opt-in-retries
Jun 6, 2026
Merged

bigtiger merged 5 commits into
mainfrom
jr-opt-in-retries

Conversation

@bigtiger

@bigtiger bigtiger commented Jun 5, 2026 •

Copy link
Copy Markdown
Contributor

Why

Users reported Perchfall flagging failures caused by "small timing errors" — a navigation that timed out by a hair, a connection reset, a server returning 5xx mid-restart. Runs were single-shot, so a one-off blip became a failed report (report.ok? == false) or a raised ScriptError — a false alarm.

What

Opt-in retry where the caller declares which conditions are retryable. Off by default — retries: 0 preserves today's single-shot behaviour exactly.

# Up to 2 extra attempts on load/script/5xx failures (the defaults)
Perchfall.run(url: "https://example.com", retries: 2)

Options

Option Default Notes
retries: 0 Additional attempts (N retries = N+1 attempts). Capped at 10.
retry_on: [:load_error, :script_error, :server_error] Array of named conditions or a predicate proc.
retry_backoff_ms: 250 Exponential backoff (250/500/1000…); 0 disables. Slot released while waiting.

Retryable conditions

Symbol Matches Default
:load_error page failed to load (status: "error") ✅
:script_error Errors::ScriptError (process failed) ✅
:server_error HTTP 5xx ✅
:client_error HTTP 4xx off
:network_error sub-resource net::ERR_* off

JavaScript/console (assertion) errors are never retryable — they're real defects, not timing blips, and aren't in the valid set.

Key safety property

The symbol form is conservative: a run is retried only when every reason it failed is a declared condition. A transient 5xx that also carries a console error is not retried — the assertion failure would never clear, so it would just fail again after burning the backoff budget. ConcurrencyLimitError, InvocationError, ParseError, and ArgumentError are never auto-retried.

Implementation

  • New pure Perchfall::RetryPolicy module classifies an outcome's failure reasons and decides retryability — testable with plain Report objects and exceptions, no browser/process needed.
  • Client gains the options, an injected sleeper: (for fast deterministic tests), and a run_with_retries loop. Backoff sleeps outside the concurrency slot; each :query_bust retry uses a fresh _pf= timestamp.
  • run! benefits automatically — raises PageLoadError only after retries are exhausted.

Docs / metadata

Version → 0.5.0, CHANGELOG, new ADR 0021, README, docs/configuration.md, docs/error-handling.md.

Non-goals

  • No attempt count added to Report (keeps the value object + JSON schema untouched) — easy follow-up if a "passed after N retries" signal is wanted.

Testing

All 288 unit specs green (306 with RUN_JS_SPECS=true); no browser or Node required. New coverage: reason classification, all-reasons-opted-in rule, proc form, ScriptError retry, never-retry of non-transient exceptions, exponential backoff timing, and validation.

Runs were single-shot, so a transient blip (nav timeout, connection
reset, 5xx mid-restart) became a failed report or a raised ScriptError —
a false alarm. Add an opt-in retry where the caller declares which
conditions are retryable.

- retries: (default 0) — additional attempts; 0 preserves current behaviour
- retry_on: — array of named conditions (:load_error, :script_error,
  :server_error, :client_error, :network_error) or a predicate proc;
  default [:load_error, :script_error, :server_error]
- retry_backoff_ms: (default 250) — exponential backoff; slot released
  while waiting

Symbol form is conservative: a run is retried only when every reason it
failed is a declared condition. Console/assertion errors are never
retryable. Classification lives in the pure RetryPolicy module.

Bump to 0.5.0; ADR 0021; README/configuration/error-handling docs.
@bigtiger
bigtiger force-pushed the jr-opt-in-retries branch from d3c2deb to 48421fe Compare June 5, 2026 21:10
bigtiger added 4 commits June 5, 2026 21:27
retry_backoff_ms validated the base at 30_000ms, but the exponential delay base_ms * 2**(attempt-1) was unclamped. A valid config of retries: 10, retry_backoff_ms: 30_000 produced a single ~4.3h sleep (~8.5h cumulative), contradicting the documented 30_000ms cap. Clamp each computed wait to MAX_RETRY_BACKOFF_MS and clarify the docs that the per-wait — not just the base — is capped.
Review follow-ups on the retry feature:

- run_with_retries captured ScriptError separately from the report path, duplicating the backoff call and the terminal-condition check. Capture a retryable ScriptError as the attempt's outcome so both paths share one decision (re-raise if the outcome is an exception, else return the report).

- validate_retry_on! rejected a bare symbol even though RetryPolicy.retryable? wraps retry_on in Array(...), so :server_error and [:server_error] behave identically. Accept a bare condition symbol to match.
Mechanical, behaviour-preserving fixes from `rubocop -a` once the house style (double quotes) was configured: string-literal style, hash alignment, line length, indentation, semicolons, block delimiters, arguments forwarding. No logic changes; full suite still green (292 examples).
Wire RuboCop in as an enforced check so lint regressions fail CI:

- .rubocop.yml: set TargetRubyVersion 3.2 and EnforcedStyle double_quotes for the string cops (matches the existing house style — eliminates ~1060 false 'offenses'); exclude specs from Metrics/BlockLength. Metrics numeric limits are inherited from the generated baseline; the curated client.rb exemptions are kept.

- .rubocop_todo.yml: generated baseline grandfathering the offenses that remain after safe autocorrect (Metrics, a few Lint/Style), to be burned down over time. New code must not add to it.

- ci.yml: add a RuboCop job (github format) alongside the unit suite.

- README: document `bundle exec rubocop` and refresh example counts (292 / 310).
@bigtiger
bigtiger merged commit a33b984 into main Jun 6, 2026
2 checks passed
@bigtiger
bigtiger deleted the jr-opt-in-retries branch June 6, 2026 03:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant