Repository navigation
LDK Node: add compact block filter (BIP 157/158) chain source as an option alongside Esplora #4749
Description
LDK Node currently has one chain source in ZEUS: Esplora (builder.setChainSourceEsplora in LdkNodeModule.kt / LdkNodeModule.swift). Every wallet sync sends the wallet's scripts to that server. This issue tracks adding compact block filters (BIP 157/158, what embedded LND calls Neutrino) as a per-wallet option alongside Esplora.
Upstream status
- Our binary is
v0.7.0-zeus-rl-0.2.5from ZeusLN/ldk-node. Its Builder exposes Esplora, Electrum, bitcoind RPC and bitcoind REST. There is no CBF source. - Add chain syncing via compact block filters (BIP 157) lightningdevkit/ldk-node#828 (tracking issue Add chain syncing via
bip157crate lightningdevkit/ldk-node#767) addsset_chain_source_cbf(peers, fee_source_config)using thebip157(kyoto) crate, about +1650 lines. It is open, unmerged, and tnull has tagged it for v0.9. v0.8 is not released yet. - Update button styles #828 targets upstream
main(0.8 dev,bdk_wallet3.1, 624 commits past v0.7.0 including refactors in everysrc/chain/*file). Ourzeusbranch usesbdk_wallet2.2. Backporting Update button styles #828 onto the current fork is a port with conflicts, not a cherry-pick.
How #828 works: kyoto runs as an embedded P2P light client. Matching filters trigger a full block download, which is applied through the same ChainListener the bitcoind source uses. Transactions are broadcast to the kyoto peers. Fee estimation can use Esplora, Electrum, or a "native" estimate built from recent coinbase fees. The native option downloads every block, which cancels the bandwidth savings, so on mobile we would keep Esplora for fees only. Fee lookups do not reveal wallet scripts.
Gaps in #828 for mobile
- No wallet birthday.
resume_checkpoint()starts from the BDK wallet's latest checkpoint. A new or restored node has none, so kyoto falls back toHashCheckpoint::from_genesisand scans filters from genesis. We need a start height for new wallets (current tip) and for restores (user-supplied or a default checkpoint), similar to the bitcoind source'swallet_rescan_from_heighton upstream main. - Hostname peers are dropped. Peers are parsed with
peer_str.parse::<SocketAddr>().ok(), which discards hostnames without an error. All of our existing Neutrino peers are hostnames (DEFAULT_NEUTRINO_PEERS_MAINNETinstores/SettingsStore.ts:btcd1.lnolymp.us, etc.). kyoto hasTrustedPeer::from_hostname, so this is a small fix in the fork. - No FFI export. Update button styles #828 derives
uniffi::EnumforCbfFeeSourceConfig, but does not exportset_chain_source_cbfonArcedNodeBuilder. Our fork usesbindings/ldk_node.udl, so we need to add the method and enum there. - No Tor. kyoto supports
Builder::socks5_proxyand AddrV2 peers, but Update button styles #828 does not wire either through. This ties into LDK Node: add Tor support (onion peers are currently un-dialable) #4523. - No sync progress.
sync_wallets()blocks until kyoto reportsFiltersSynced. ZEUS callssyncWallets()during startup (startLdkNodeWalletinutils/LdkNodeUtils.ts) while showingldkNodeSyncing. A long first sync would look like a hang unless the fork exposes progress (for example the kyoto tip vs applied height inNodeStatus). - Restart budget. The kyoto node is restarted at most 5 times (
MAX_RESTART_RETRIES) before the chain source gives up for good. We need to test that on iOS, where sockets die while the app is suspended.
Behavior differences to verify and gate in ZEUS
- 0-conf / JIT channels. Upstream disables
channel_full_cycle_0confandchannel_full_cycle_0conf_0reservefor CBF. The reason may be only that the test harness waits on the mempool, but Olympus LSPS2 JIT channels are our main onboarding path, so this has to be confirmed on a real device before we ship. - No mempool. Incoming on-chain payments will not appear until they confirm. Upstream also disables
onchain_fee_bump_rbffor CBF, so the bump-fee UI may need to be gated. - Mutinynet is not supported. kyoto's builder takes only the
Networkenum and has no custom signet magic. The CBF option should be hidden for mutinynet. Regtest has no DNS seeds, so it only works with explicit peers. - Gossip is unaffected because we use RGS.
Work items
Fork (ZeusLN/ldk-node)
- Decide the base: rebase the fork onto upstream
mainand carry Update button styles #828, or port Update button styles #828 onto the 0.7 fork (bdk 2.2). The fork will need to move to 0.8 eventually, so rebasing onto main is probably less total work. - Wallet birthday / start checkpoint for new and restored nodes (gap 1)
- Hostname peers (gap 2)
- UDL export of
set_chain_source_cbfandCbfFeeSourceConfig(gap 3) - Optional SOCKS5 proxy parameter (gap 4, coordinate with LDK Node: add Tor support (onion peers are currently un-dialable) #4523)
- Sync progress in
NodeStatus(gap 5) - Cut a release and update
fetch-libraries-versions.jsonhashes
Native modules
-
setChainSourceCbf(peers, feeEsploraUrl)inLdkNodeModule.kt,LdkNodeModule.swiftandLdkNodeModule.m. Store the values and re-apply them inapplyBuilderSettings()likestoredEsploraServerUrl, and clear them in the reset path.
JS / settings
-
ldknode/LdkNodeInjection.ts,ldknode/LdkNode.d.ts: new method andinitializeNodeoptions -
utils/LdkNodeUtils.ts: passchainSourceand peers throughinitNode/createLdkNodeWallet/startLdkNodeWallet. Update the startup error classification: today any sync error that is not NotRunning is reported asesploraError. -
stores/SettingsStore.ts: per-nodeldkChainSource: 'esplora' | 'cbf'(default'esplora'for new and existing wallets) andldkCbfPeers -
views/Settings/EmbeddedNode/index.tsx: chain source row, plus a peers screen (reuse theNeutrinoPeersUX and defaults where possible). When CBF is selected, labelEsploraServeras the fee source. -
views/Settings/WalletConfiguration.tsx,views/Settings/SeedRecovery.tsx: chain source choice at create/restore, and a birthday input for restores -
views/Wallet/Wallet.tsx: pass the settings through and handle the long first sync - Gate bump-fee and pending on-chain UI under CBF if device testing confirms the limits above
Tests
- Jest: settings default and migration (existing LDK wallets stay on Esplora), option plumbing in
LdkNodeUtils, startup error classification - Device: new wallet, seed restore with birthday, LSPS2 JIT channel open, on-chain send/receive, force-close sweep, app backgrounded for 30+ minutes then resumed (iOS and Android), mainnet data usage and battery compared to Esplora
Open questions
- Default peers: our btcd hosts (they already serve filters to embedded LND), kyoto DNS seeding, or both.
- Whether to wait for upstream v0.9 instead of carrying Update button styles #828 in the fork.
- rustaceanrob suggested block-dn servers in Add chain syncing via
bip157crate lightningdevkit/ldk-node#767 as a later alternative to random P2P peers on mobile. That is out of scope here but worth tracking.
Activity