fix(evm-rpc): retry eRPC upstreams-exhausted errors instead of crashing the dumper

[elina-chertova]

Symptom

Alert linea-mainnet_No_Dumper_Data (evm-archive, kind=dump). dump-linea-mainnet-0 is in CrashLoopBackOff (140 restarts / 27h); no raw data written.

Root cause (proven from pod logs)

The linea-mainnet dumper was routed through eRPC on 2026-07-22 (infra commit 248d45930 "Roll out eRPC to almost every archive ingester"; the dump pod's 27h age matches). eRPC's only primary upstream for linea is Alchemy, which is currently degraded — live probes return HTTP 503 -32001 "Unable to complete request at this time" and 429 "compute units per second exceeded" for linea.

eRPC does not fail over to the healthy dwellir/uniblock fallbacks, and returns the request as an HTTP 200 body carrying a JSON-RPC error whose code is normalized to -32601 ("method eth_getBlockByNumber does not exist/is not available") with data.code: ErrFailsafeRetryExceeded → cause.code: ErrUpstreamsExhausted.

EvmRpcClient.isConnectionError did not recognize this shape, so the error was fatal (dump log level 5) and the process exited:

RpcError: The method eth_getBlockByNumber does not exist/is not available
  at EvmRpcClient.receiveResult (/squid/util/rpc-client/lib/client.js:376)

Before the eRPC switch the same underlying Alchemy 503 arrived as a raw HTTP 503, which the client already retries — so eRPC repackaging it as a fatal JSON-RPC error is what turned a recoverable degradation into a crash-loop.

Fix

Recognize eRPC's stable data.code (ErrUpstreamsExhausted / ErrFailsafeRetryExceeded) as a retryable connection error in EvmRpcClient.isConnectionError. This restores parity with a direct provider 5xx: the dumper backs off and retries (retryAttempts=MAX) instead of crash-looping, so a single provider hiccup can no longer take the dump down. A genuine -32601 with no eRPC data stays fatal.

Test

evm/evm-rpc/test/rpc-client.test.ts — asserts the production error shape is retryable and a real method-not-found is not. Red on pre-fix (2 fail), green after.

Falsification

If the dump still crash-loops after deploying an evm-dump built from this change with the same -32601/ErrUpstreamsExhausted fatal line, the classifier isn't matching the real payload shape.

Note (operator action, not in this PR)

This stops the crash-loop but does not by itself restore linea data flow: Alchemy is down for linea while eRPC isn't failing over. An operator should route linea to a healthy provider (dwellir + uniblock both return 200 for linea now) or fix the eRPC failover, and raise the Alchemy linea 503/429 with the provider (https://status.alchemy.com). Provider swaps are an operator mitigation, so they're intentionally not included here.

[elina-chertova]

Related open PRs harden EvmRpcClient/rpc-client against other transient eRPC/aggregator signatures that crash-loop the dumper: #501 (-32503), #505 (-32020 response too large), #520 (Cloudflare 521-524), #513 (invalid block height on finalized-head probe). This one covers a distinct signature — eRPC's data.code ErrUpstreamsExhausted/ErrFailsafeRetryExceeded, normalized to a misleading -32601. They could be consolidated into one error-classification pass if preferred.