diff --git a/.depcheckrc.yml b/.depcheckrc.yml index cf63d4d6c5..9200554363 100644 --- a/.depcheckrc.yml +++ b/.depcheckrc.yml @@ -51,7 +51,7 @@ ignores: - 'node-gyp' # Peer dependency of @metamask/kernel-shims due to its /endoify-node - # export, used by @metamask/kernel-node-runtime + # export, used by Node entrypoints that run the libp2p netlayer - '@libp2p/webrtc' # These are peer dependencies of various modules we actually do diff --git a/docs/plans/netlayer/master.md b/docs/plans/netlayer/master.md index 5a68cd6306..c92e0fbae7 100644 --- a/docs/plans/netlayer/master.md +++ b/docs/plans/netlayer/master.md @@ -359,8 +359,71 @@ receive, incarnationId)` / `unregister` / `getIncarnation` / `deliver(from, to, `initTransport` export and all libp2p deps from ocap-kernel. Relocate kernel-shims' bare `import '@libp2p/webrtc'` (endoify-node.js) alongside the libp2p netlayer — flag for review. Verify: full CI, extension e2e, `kernel-test` remote-comms integration, `ocap relay` smoke test. - -**Phase 5 — `@metamask/netlayer-websocket`.** ~3–4 days. + **DONE** (issue #968, PR #973 on branch `rekm/netlayer-4`, stacked on `rekm/netlayer-3`). + +Landed decisions / deviations from the phase-4 plan (these are the facts Phase 5 must build on): + +- **`@metamask/netlayer-libp2p` (new, `0.1.0`, published).** Exports: + - `.` (browser default): `libp2pNetlayerFactory` (a `NetlayerFactory` — typed ``, not + ``, so it fits `NetlayerRegistry = Record>`; + it `assert`s the config to `Libp2pNetlayerConfig` internally), plus `makeLibp2pNetlayer` + (re-exported for consumers that must not reach `./nodejs` — accepts an `@internal` + `directTransports`), `Libp2pNetlayerConfig`/`Libp2pNetlayerConfigStruct`, `DirectTransport`. + - `./nodejs`: `nodejsLibp2pNetlayerFactory` (builds QUIC/TCP direct transports from + `config.directListenAddresses`) and `buildDirectTransports`. + - `./relay`: `startRelay` (moved from `@metamask/kernel-utils/libp2p`; `RELAY_LOCAL_ID = 200`, + ports 9001/9002). + - `error-mapper.ts`: `mapLibp2pReadError`, `isIntentionalDisconnect`, `isRetryableLibp2pError` + (neutral `isRetryableNetworkError` ∪ libp2p name/message sniffing), and `mapLibp2pDialError` + (maps retryable-by-name libp2p dial errors → neutral `ChannelResetError`; passes Node-code and + neutral errors through so the engine's permanent-failure detection is unchanged). + `connection-factory.dial` applies `mapLibp2pDialError`; `kernel-errors`' + `isRetryableNetworkError`/`getNetworkErrorCode` were reduced to the neutral subset. + - Deps: `@metamask/{netlayer,kernel-errors,kernel-utils,logger,superstruct}`, + `@multiformats/multiaddr`, `uint8arrays`, and the full `@libp2p`/`@chainsafe`/`libp2p` family. + `sideEffects: false`. +- **Contract shapes (LANDED).** `NetlayerSpecifier = { netlayer: string; config: Json }`; + `NetlayerRegistry = Record` (config `Json`). `PlatformServices.initializeRemoteComms` + is `(params: { keySeed; specifier: NetlayerSpecifier; hooks: NetlayerHooks; incarnationId? }) => Promise`. + `NodejsPlatformServices` ctor and `PlatformServicesServer` ctor/`make` take a `netlayers: NetlayerRegistry` + (`PlatformServicesServer` as `args: { netlayers; logger? }`). `PlatformServicesClient.initializeRemoteComms` + takes the same options bag and keeps hooks locally (only `keySeed` + `specifier` + `incarnationId` cross RPC). + The RPC hook `InitializeRemoteComms` stayed **positional** `(keySeed, specifier, incarnationId?)`. +- **Config convention.** The kernel understands exactly one netlayer-config key by convention: + `knownRelays: string[]` — the opaque hint pool it persists and re-injects. + `remote-comms.ts` merges bootstrap `config.knownRelays` into the store then overwrites + `config.knownRelays` with the full persisted pool; `mnemonic` never enters `specifier.config`. + `DEFAULT_NETLAYER = 'libp2p'` (in `remotes/kernel/remote-comms.ts`) covers omitted specifiers. +- **RPC structs.** `rpc/platform-services/initializeRemoteComms.ts` and + `rpc/kernel-control/init-remote-comms.ts` validate `specifier.config` with `JsonStruct` + (`@metamask/utils`); the kernel-control struct drops `mnemonic`. +- **`comms-query-string`.** `createCommsQueryString({ netlayer, config, maxQueue?, ackTimeoutMs?, +maxUrlRelayHints?, maxKnownRelays? })` sets `netlayer` + `netlayer-config` (JSON) + numeric params; + `parseCommsQueryString`/`getCommsParamsFromCurrentLocation` return `RemoteCommsOptions` with a + `specifier` (omitted when no `netlayer` param). Extension/omnium `offscreen.ts` register + `{ libp2p: libp2pNetlayerFactory }` and pass a specifier; the multiaddr reads `process.env.RELAY_MULTIADDR`. +- **ocap-kernel is libp2p-free.** `packages/ocap-kernel/package.json` has zero + `@libp2p`/`@chainsafe`/`@multiformats`/`libp2p` deps (plain `multiformats` + `@scure/bip39` stay). + `initTransport`, `DirectTransport`, and `ConnectionFactoryOptions` are gone; `RemoteCommsOptions` + is the kernel-owned subset + `specifier`. +- **DEVIATION — endoify-node `@libp2p/webrtc` relocation (§3.7) DEFERRED.** kernel-shims still owns + the pre-lockdown `import '@libp2p/webrtc'` in `endoify-node.js` (peer dep retained). Every + `@metamask/kernel-node-runtime` importer transitively loads `@libp2p/webrtc` at module-eval (via + `make-kernel` → the nodejs factory → `connection-factory`), so relocating the pre-lockdown import + means repointing ~7 packages with a silent-lockdown-failure risk; it is not required for the + ocap-kernel zero-libp2p gate. Left as a Phase 6 / future-work follow-up (Phase 5 is deferred — + see below). +- **DEVIATION — browser `PlatformServicesServer` does not capture `getListenAddresses`** (the client + returns `[]` for Node-only direct transport; capturing it would be dead code). +- **Coverage.** Every touched package meets/exceeds its pre-phase baseline (netlayer-libp2p new at + 96.75% stmts / 94.11% branch; kernel-errors raised to 100%); extraction dips were closed with + behavior-asserting tests. Extension `offscreen.js` bundle unchanged (~1,320 kB / 435 kB gzip). + +**Phase 5 — `@metamask/netlayer-websocket`.** ~3–4 days. **DEFERRED — not executed in this +effort; remains future work.** Per a user directive after Phase 4, the effort skips straight from +Phase 4 to Phase 6 (cleanup + docs). The plan below stands as the future-work description; when it +is eventually picked up it must first be reconciled against the landed state of Phases 1–4 **and 6** +(see the deferral banner in `phase-5.md`). `ChannelProvider` over WebSocket: client dials `wss://` hints (browser + node); `./nodejs` server authenticates peers via mutual Ed25519 challenge-signature handshake (new security-sensitive code — keep minimal, flag for crypto review). Topology decision: the hub is a **kernel @@ -369,12 +432,13 @@ unsupported (that's what the libp2p netlayer's relays are for). Reuses `makeChan shared machinery — this is the proof the abstraction isn't libp2p-shaped. Parameterize `kernel-test` integration over netlayers (loopback always; libp2p/websocket where infra allows). -**Phase 6 — Cleanup + docs.** ~1–2 days. +**Phase 6 — Cleanup + docs.** ~1–2 days. **Follows Phase 4 directly** (Phase 5 deferred). Optional relay→"location hints" terminology rename in store/`RemoteIdentity`, glossary entries, changelogs (update-changelogs skill), "writing a netlayer" doc, note iroh as the next (Node-only) implementation with a pointer to #968. -**Total: ~16–22 dev-days.** +**Total: ~16–22 dev-days** across all six phases; **~13–18 dev-days for the executed scope** +(Phases 1–4 + 6, with Phase 5 deferred). ## Risks @@ -398,8 +462,8 @@ implementation with a pointer to #968. - After Phase 4: run the extension example (`@ocap/extension` `test:e2e`) and a two-kernel Node scenario over the relay (`ocap relay start` + `kernel-test/src/remote-comms.test.ts`) to confirm ocap-URL redemption and message delivery across the new injection path. -- After Phase 5: two-kernel WebSocket hub-and-spoke integration test; loopback-based multi-kernel - test in-process. +- After Phase 5 (deferred): two-kernel WebSocket hub-and-spoke integration test; loopback-based + multi-kernel test in-process. The loopback-based multi-kernel coverage already lands with Phase 3. - Confirm `packages/ocap-kernel/package.json` has zero `libp2p`/`@libp2p`/`@chainsafe`/ `@multiformats` deps at the end of Phase 4. (Plain `multiformats` is not libp2p and stays — `base58btc` is the neutral identity/oid encoding.) diff --git a/docs/plans/netlayer/phase-5.md b/docs/plans/netlayer/phase-5.md index 58fd4ca847..0329ee12fc 100644 --- a/docs/plans/netlayer/phase-5.md +++ b/docs/plans/netlayer/phase-5.md @@ -1,5 +1,23 @@ # Phase 5 — `@metamask/netlayer-websocket` +> **⚠️ DEFERRED — NOT BEING EXECUTED IN THIS EFFORT.** Per a user directive after Phase 4, the +> pluggable-netlayer effort skips straight from Phase 4 to Phase 6 (cleanup + docs). This document +> is preserved as the future-work plan for the WebSocket netlayer. **Before it is picked up, it must +> first be reconciled against the landed state of Phases 1–4 _and_ Phase 6** — cross-phase symbol +> names, signatures, package layout, and the terminology/docs that Phase 6 lands (e.g. any +> relay→"location hints" rename, the "writing a netlayer" guide) all take precedence over the +> sketches below. The reconciliation blockquote that follows was written against Phases 1–4 only and +> predates the Phase 6 cleanup. +> +> Assumptions Phase 4 already invalidated that this plan still needs updating for when resumed +> (brief, non-exhaustive): `ChannelProvider.dial` is 3-arg `(peerId, hints, withRetry)` (the §4 +> sketch shows 2-arg); the libp2p error-name/dial mapping now lives in +> `@metamask/netlayer-libp2p/src/error-mapper.ts` (not kernel-errors), which the WS `errors.ts` +> should mirror; `@metamask/netlayer` still does **not** re-export `@metamask/superstruct` or ship an +> `EngineOptionsStruct`/sign-verify identity wrappers — those must be added when this phase resumes; +> and a pure-websocket Node deployment still pulls `@libp2p/webrtc` transitively because the +> pre-lockdown endoify import relocation was deferred (see master.md Phase 4 deviations). + Implementation plan for the fifth PR in the pluggable-netlayer effort ([issue #968](https://github.com/MetaMask/ocap-kernel/issues/968); master plan at [`master.md`](./master.md)). @@ -13,17 +31,33 @@ runtimes accept a `NetlayerRegistry` and per-kernel `NetlayerSpecifier` (`{ netl config: Json }`); `ocap-kernel` is libp2p-free; identity is a neutral base58btc-multibase Ed25519 public key. -Where this plan references machinery, it points at the **future `@metamask/netlayer` location** -but the behaviour to reuse is what currently lives in -`packages/ocap-kernel/src/remotes/platform/` (`transport.ts`, `reconnection.ts`, -`peer-state-manager.ts`, `rate-limiter.ts`, `validators.ts`, `handshake.ts`, -`channel-utils.ts`). Read those to understand the contract you are implementing against. - -> **Revision required before execution.** This plan was written before Phases 1–4 landed. -> The type sketches below (notably `ChannelProvider` — Phase 1 lands -> `dial(peerId, hints, withRetry)`, not the 2-arg form shown) are indicative; the contract -> actually exported by the landed `@metamask/netlayer` wins, and this document should be -> updated against it first. +The machinery to reuse now lives in **`@metamask/netlayer/src/`** (moved out of +`ocap-kernel` in Phase 3): `channel-netlayer.ts` (the `makeChannelNetlayer` engine), +`reconnection.ts`, `reconnection-lifecycle.ts`, `peer-state-manager.ts`, `rate-limiter.ts`, +`validators.ts`, `handshake.ts`, `channel-utils.ts`, `identity.ts`, `constants.ts`. The libp2p +`ChannelProvider` to mirror is `@metamask/netlayer-libp2p/src/connection-factory.ts` (with its +`make-libp2p-netlayer.ts` wiring and `error-mapper.ts`). Read those to understand the contract you +are implementing against. + +> **Reconciled against landed Phases 1–4.** The contract exported by `@metamask/netlayer` wins over +> any sketch below. Key landed facts (see master.md's Phase 1–4 "Landed decisions" blocks): +> `ChannelProvider.dial(peerId, hints, withRetry)` (3-arg) and carries `readonly peerId`; +> `makeChannelNetlayer({ provider, hooks, options, logger, stopController })` is synchronous and +> shares one `AbortController` with the provider; `NetlayerSpecifier = { netlayer: string; config: Json }`, +> `NetlayerRegistry = Record` (config `Json` — validate/narrow +> inside the factory, like `libp2pNetlayerFactory` does); `PlatformServices.initializeRemoteComms` +> is the options bag `{ keySeed, specifier, hooks, incarnationId? }` and the runtimes take a +> `netlayers: NetlayerRegistry`. The websocket netlayer's config must be `Json` (it crosses +> postMessage); the kernel injects `config.knownRelays` (its one convention key) — for a hub-and- +> spoke websocket topology, treat those opaque hint strings as `wss://` hub URLs. +> +> **Follow-up carried from Phase 4:** the kernel-shims `@libp2p/webrtc` pre-lockdown endoify import +> was NOT relocated in Phase 4 (deferred — see master.md). A pure-websocket Node deployment still +> pulls `@libp2p/webrtc` transitively through `@metamask/kernel-node-runtime`'s default registry; +> if Phase 5 wants a libp2p-free websocket-only Node runtime, that relocation (and making the +> default registry configurable without importing the libp2p factory) must be tackled here or in +> Phase 6. `@metamask/netlayer` re-added `@metamask/superstruct` is still pending (Phase 3 dropped +> it as unused); add it back when the websocket config structs need it. --- diff --git a/docs/plans/netlayer/phase-6.md b/docs/plans/netlayer/phase-6.md index 873453337b..7ee78d2a9a 100644 --- a/docs/plans/netlayer/phase-6.md +++ b/docs/plans/netlayer/phase-6.md @@ -1,22 +1,28 @@ # Phase 6 — Cleanup + Docs (Netlayer abstraction) -This is the final phase of the 6-phase effort to introduce a pluggable "netlayer" -abstraction (see the master plan for full context and issue -[#968](https://github.com/MetaMask/ocap-kernel/issues/968)). Phases 1–5 are assumed -complete: the packages `@metamask/netlayer`, `@metamask/netlayer-loopback`, -`@metamask/netlayer-libp2p`, and `@metamask/netlayer-websocket` exist; `@metamask/ocap-kernel` -is libp2p-free; kernel identity is neutral (base58btc-encoded raw Ed25519 pubkey); and the -runtimes accept a `NetlayerRegistry` / `NetlayerSpecifier`. +This is the final phase of the netlayer effort (see the master plan for full context and issue +[#968](https://github.com/MetaMask/ocap-kernel/issues/968)). **Phase 5 +(`@metamask/netlayer-websocket`) was deferred** — per a user directive the effort skips from Phase 4 +straight to Phase 6, so this phase follows Phase 4 directly. Phases 1–4 are complete: the packages +`@metamask/netlayer`, `@metamask/netlayer-loopback`, and `@metamask/netlayer-libp2p` exist (the +WebSocket netlayer does **not**); `@metamask/ocap-kernel` is libp2p-free; kernel identity is neutral +(base58btc-encoded raw Ed25519 pubkey); and the runtimes accept a `NetlayerRegistry` / +`NetlayerSpecifier`. Phase 6 contains no functional changes. It is a terminology rename, glossary and changelog maintenance, and documentation. An engineer with no prior context should be able to execute it from this document. -> **Revision required before execution.** This plan was written before Phases 1–5 landed. -> Symbol names, file locations, and line numbers referencing prior-phase output are -> indicative — reconcile against the merged code (especially: where the identity helpers -> ended up, whether `relays` still exists on any kernel surface, and the final shape of -> `RemoteCommsOptions`) and update this document first. +> **Revision required before execution.** This plan was written before Phases 1–4 landed and before +> Phase 5 was deferred. Symbol names, file locations, and line numbers referencing prior-phase output +> are indicative — reconcile against the merged code (the "Landed decisions" blocks in `master.md` +> are authoritative; especially: the identity helpers now live in `@metamask/netlayer`, `relays` +> no longer exists on the kernel surface — `RemoteCommsOptions` is the kernel-owned subset plus a +> `specifier`, and the persisted hint key is `knownRelays` via the libp2p `config` convention) and +> update this document first. **Because Phase 5 is deferred, treat every reference below to +> `@metamask/netlayer-websocket`, the WebSocket hub/handshake, and `kernel-test` parameterization +> over websocket as _future work_, not something to document as existing.** Where this plan says to +> describe the WebSocket netlayer, describe it as a planned/future implementation instead. --- @@ -29,8 +35,8 @@ it from this document. (store, `RemoteIdentity`, kernel-level config options). 2. Add glossary entries for the vocabulary the abstraction introduces: netlayer, location hint, neutral peer id, loopback netlayer, hub-and-spoke. -3. Update changelogs for every package with consumer-facing changes across Phases 1–5 - (the rename in this phase is folded in). +3. Update changelogs for every package with consumer-facing changes across Phases 1–4 + (the rename in this phase is folded in; Phase 5 is deferred). 4. Write a new architecture doc, `docs/writing-a-netlayer.md`, describing the `Netlayer` / `ChannelProvider` contracts, delivery semantics, the incarnation handshake, identity requirements, and testing guidance. @@ -51,6 +57,40 @@ it from this document. --- +## 1a. Follow-ups carried from Phase 4 (and the deferred Phase 5) + +These items were flagged during Phase 4 and, with Phase 5 deferred, land in Phase 6's lap. None is +strictly a rename/docs task; decide during execution which to pursue here vs. leave as tracked +future work, but do not let them fall silently off the plan. + +1. **`@libp2p/webrtc` pre-lockdown endoify relocation (review-gated).** `@metamask/kernel-shims` + still owns the bare `import '@libp2p/webrtc'` in `endoify-node.js` (Phase 4 §3.7 deferred it). + The import must run **before** SES lockdown, and every `@metamask/kernel-node-runtime` importer + loads `@libp2p/webrtc` transitively at module-eval, so relocating it alongside the libp2p + netlayer touches ~7 package entry points with a silent-lockdown-failure risk. If Phase 6 tackles + it, treat it as its own review-gated change (not folded into the rename commit) and re-verify the + single-lockdown-time-webrtc invariant below; otherwise record it as explicit future work. +2. **Browser `PlatformServicesServer` `getListenAddresses` gap.** The browser server does not + capture the netlayer's `getListenAddresses` (the client returns `[]` for Node-only direct + transport, so capturing it today would be dead code that trips the unused-private-member lint). + If direct-transport listen-address reporting is ever needed through the browser RPC boundary, + that is where to close it. Document, don't necessarily build. +3. **`libp2pComms` test-helper retirement.** `packages/kernel-node-runtime/test/helpers/remote-comms.ts` + exports a `libp2pComms(flat)` shim that maps the pre-injection flat libp2p option shape + (`relays`, `directListenAddresses`, backoff timings) into the injected `RemoteCommsOptions` + specifier. It exists only so the e2e/integration suites did not have to be rewritten in Phase 4. + Once the specifier UX is stable, those call sites can adopt the specifier directly and the helper + can be deleted — a good low-risk cleanup for this phase. +4. **tsconfig subpath-mapping rule for `@metamask/netlayer-libp2p`.** `tsconfig.packages.json` + carries explicit path mappings for the `./nodejs` and `./relay` subpaths → their `src` entry + points. This is load-bearing: without it, vitest resolves the subpath to **dist**, which + initializes a _second_ post-lockdown `@libp2p/webrtc`/`@peculiar/x509` instance and crashes SES + with `privateMap.get is not a function`. **Any new netlayer-libp2p subpath export must get the + same mapping.** Keep this note near any doc that describes adding netlayer entry points (e.g. the + `writing-a-netlayer` guide's packaging section). + +--- + ## 2. Rename scope recommendation ### Recommendation: **GO**, scoped to the kernel-generic surface only. @@ -174,10 +214,10 @@ netlayer-agnostic: it consumes the `Netlayer` contract and handles the [Ken protocol](../docs/ken-protocol-assessment.md) (sequencing, acknowledgement, retransmission, deduplication) itself, so a netlayer need only provide best-effort ordered delivery per live connection. Netlayers ship as separate packages — `@metamask/netlayer` (contracts + shared -machinery), `@metamask/netlayer-loopback`, `@metamask/netlayer-libp2p`, -`@metamask/netlayer-websocket` — and are supplied to a runtime via a `NetlayerRegistry` and -selected per kernel with a `NetlayerSpecifier`. See [writing a -netlayer](../docs/writing-a-netlayer.md). +machinery), `@metamask/netlayer-loopback`, and `@metamask/netlayer-libp2p` — and are supplied to a +runtime via a `NetlayerRegistry` and selected per kernel with a `NetlayerSpecifier`. A WebSocket +netlayer (`@metamask/netlayer-websocket`) and an iroh netlayer are planned future implementations. +See [writing a netlayer](../docs/writing-a-netlayer.md). ### location hint @@ -213,11 +253,11 @@ is the recommended baseline when writing or testing a new netlayer. See A network topology in which kernels ("spokes") connect to a central endpoint ("hub") rather than to one another. The hub's role differs by netlayer: a libp2p relay is a _forwarding_ -hub (spokes reach each other through it), whereas the plain-WebSocket server netlayer's hub -is a _kernel endpoint_ — spokes talk to the hub kernel itself, and spoke↔spoke traffic is -deliberately unsupported (see the `@metamask/netlayer-websocket` README). The -[loopback netlayer](#loopback-netlayer)'s in-memory hub is the degenerate single-process -case. Contrast with direct/peer-to-peer connections (e.g. libp2p QUIC/TCP, or a WebRTC +hub (spokes reach each other through it), and the [loopback netlayer](#loopback-netlayer)'s +in-memory hub is the degenerate single-process case. (The planned plain-WebSocket server +netlayer's hub is instead a _kernel endpoint_ — spokes would talk to the hub kernel itself, +spoke↔spoke unsupported — but that netlayer is future work; document it only once it lands.) +Contrast with direct/peer-to-peer connections (e.g. libp2p QUIC/TCP, or a WebRTC upgrade), where peers dial each other after an initial rendezvous. ### incarnation handshake @@ -296,9 +336,10 @@ Audience: an engineer implementing a new netlayer package. State the contracts p cryptography" caveat that already lives in `remote-comms.ts`). - Neutral peer id encoding recipe: derive raw Ed25519 pubkey from seed → base58btc multibase. - Peer authentication: the netlayer must prove the remote controls the private key for the - neutral peer id it claims (libp2p noise does this for free; the WebSocket netlayer uses a - challenge-signature handshake — cite it as the example for transports without built-in - auth). + neutral peer id it claims (libp2p noise does this for free). For transports without built-in + auth, an application-level Ed25519 challenge-signature handshake is the intended pattern — the + planned WebSocket netlayer (future work; see the deferred `phase-5.md`) is where this will be + worked out; until it lands, describe the requirement without citing a concrete example. 7. **The incarnation handshake** @@ -335,15 +376,17 @@ Audience: an engineer implementing a new netlayer package. State the contracts p - Use the [loopback netlayer](../docs/glossary.md#loopback-netlayer) as the reference and as a golden test of the shared engine. - - Parameterize the `kernel-test` integration suite over netlayers (loopback always; - libp2p/websocket where infra allows). + - Parameterize the `kernel-test` integration suite over netlayers (loopback always; libp2p + where infra allows). Websocket parameterization is future work, landing with the deferred + WebSocket netlayer. - Write per-error-class mapping tests (the master plan flags error-classification completeness as a risk). - Two-kernel round-trip: issue an ocap URL on A, redeem on B, exchange messages, restart A and confirm incarnation handling. 12. **Worked example** — a minimal `ChannelProvider` skeleton (~30 lines) delegating to - `makeChannelNetlayer`, with pointers to loopback and websocket for full references. + `makeChannelNetlayer`, with pointers to the loopback and libp2p netlayers for full references + (the WebSocket netlayer is future work). --- @@ -392,9 +435,9 @@ Format: **file** → _claim_ → correction. - Lines 276–306 ("Remote Communications") → "enables peer-to-peer communication between kernels using libp2p relay servers" and "Relay addresses must be libp2p multiaddrs…". Generalize: remote comms now goes through a selected **netlayer**; the libp2p netlayer uses relay - multiaddrs, but WebSocket/loopback have their own hint formats. Show `initRemoteComms` taking - a `NetlayerSpecifier` (or however Phase 4b shaped it) and note the libp2p example as one - option. + multiaddrs, while other netlayers (loopback today; WebSocket/iroh in future) have their own + hint formats. Show `initRemoteComms` taking the landed `NetlayerSpecifier`-bearing + `RemoteCommsOptions` (Phase 4b shape) and note the libp2p example as one option. - Line 297 note ("For browser environments, only WebSocket transports (`/ws`) are supported.") → scope this to the libp2p netlayer. @@ -428,21 +471,27 @@ Format: **file** → _claim_ → correction. - Add a short "Remote communications" paragraph pointing to the netlayer packages and `docs/writing-a-netlayer.md`, and noting that ocap-kernel re-exports the netlayer types. -**New package READMEs** (created in Phases 3–5; ensure they exist and are non-stub): +**New package READMEs** (created in Phases 3–4; ensure they exist and are non-stub): -- `packages/netlayer/README.md`, `packages/netlayer-loopback/README.md`, - `packages/netlayer-libp2p/README.md`, `packages/netlayer-websocket/README.md` — each: one- - line purpose, install line, a "Contributing → monorepo README" footer (match existing README - conventions), and a link to `docs/writing-a-netlayer.md`. `netlayer-libp2p` additionally - carries the yamux/SES patch note moved from ocap-kernel and documents its `./nodejs` and - `./relay` subpaths. +- `packages/netlayer/README.md`, `packages/netlayer-loopback/README.md`, and + `packages/netlayer-libp2p/README.md` — each: one-line purpose, install line, a + "Contributing → monorepo README" footer (match existing README conventions), and a link to + `docs/writing-a-netlayer.md`. `netlayer-libp2p` additionally carries the yamux/SES patch note + moved from ocap-kernel and documents its `./nodejs` and `./relay` subpaths. (A + `netlayer-websocket` README arrives with the deferred Phase 5.) **Runtime READMEs** (`kernel-node-runtime`, `kernel-browser-runtime`) → if they describe remote comms setup, update to the `NetlayerRegistry` construction and `NetlayerSpecifier` selection. --- -## 6. Changelog matrix (consumer-facing changes, Phases 1–5 + this rename) +## 6. Changelog matrix (consumer-facing changes, Phases 1–4 + this rename) + +> Phase 5 (`@metamask/netlayer-websocket`) is deferred, so there is no websocket package or +> websocket-related changelog entry in this effort. Note also that Phases 1–4 each already landed +> their own `## [Unreleased]` changelog entries as they merged; this phase's changelog run should +> reconcile against what is already present rather than re-adding those entries, and add the +> rename entry below. Under `## [Unreleased]`, "Keep a Changelog" categories, `**BREAKING:**` prefix for breaking changes, PR links. The `update-changelogs` skill does the writing; this matrix tells it which @@ -453,20 +502,20 @@ packages changed and how. | `@metamask/netlayer` | **new** | Added: initial release — `Netlayer`/`NetlayerHooks`/`NetlayerFactory`/`NetlayerSpecifier`/`NetlayerRegistry` contracts, `NetworkChannel`/`ChannelProvider` seam, `makeChannelNetlayer` engine, shared reconnection/backoff/peer-state/rate-limit/validators/handshake, neutral Ed25519 identity helpers. | | `@metamask/netlayer-loopback` | **new** | Added: initial release — in-process hub netlayer (test fake + embedded multi-kernel). | | `@metamask/netlayer-libp2p` | **new** | Added: initial release — libp2p `ChannelProvider`, error mapper, multiaddr utils; `./nodejs` (QUIC/TCP direct transports) and `./relay` (relay server, moved from `@metamask/kernel-utils`) subpaths. Carries the libp2p deps and the `@chainsafe/libp2p-yamux` SES patch. | -| `@metamask/netlayer-websocket` | **new** | Added: initial release — plain-WebSocket `ChannelProvider` (browser + node client); `./nodejs` server with Ed25519 challenge-signature peer auth. | +| `@metamask/netlayer-websocket` | _deferred_ | Phase 5 deferred — no package, no entry in this effort. | | `@metamask/ocap-kernel` | modified | **BREAKING:** removed all libp2p/`@libp2p`/`@chainsafe`/`@multiformats` dependencies. **BREAKING:** kernel identity is now a neutral base58btc-multibase Ed25519 peer id (was libp2p `12D3KooW…`); the `ocap:` URL format changed accordingly (no migration). **BREAKING:** `initRemoteComms` / `initializeRemoteComms` now take a `NetlayerSpecifier` (was libp2p-specific options); `RemoteCommsOptions` split into kernel-level options + per-netlayer config. **BREAKING:** renamed relay→location-hint surface — `RelayEntry`→`LocationHintEntry`, `maxKnownRelays`→`maxKnownLocationHints`, `maxUrlRelayHints`→`maxUrlLocationHints`, `RemoteIdentity.addKnownRelays`→`addKnownLocationHints`; persisted `knownRelays` KV key renamed to `knownLocationHints` (learned hint pool not migrated); removed legacy `string[]` hint-pool migration. Added: re-exports of `@metamask/netlayer` types. Changed: internal `Channel`→`NetworkChannel` (note only if any of these were part of the public export surface). | | `@metamask/kernel-errors` | modified | Added: neutral network error classes `ChannelResetError`, `IntentionalDisconnectError`, `MessageTooLargeError`. Changed: `isRetryableNetworkError` no longer sniffs libp2p error names (`MuxerClosedError` etc.) — that logic moved into `@metamask/netlayer-libp2p`. **BREAKING (if applicable):** dropped libp2p imports from the public surface. | | `@metamask/kernel-utils` | modified | **BREAKING:** removed the libp2p relay server (the `@metamask/kernel-utils/libp2p` subpath / `startRelay`) — moved to `@metamask/netlayer-libp2p/relay`. Note `getLibp2pRelayHome` **stays** in `@metamask/kernel-utils/nodejs` per the Phase 4 plan (it is a plain path helper); mention it only if this phase's optional rename touches it. | | `@metamask/kernel-node-runtime` | modified | **BREAKING:** `NodejsPlatformServices` is now constructed with a `NetlayerRegistry`; QUIC/TCP transport detection moved into the libp2p netlayer; the internal `directTransports` option was removed. | | `@metamask/kernel-browser-runtime` | modified | **BREAKING:** `PlatformServicesServer`/`PlatformServicesClient` now take a `NetlayerRegistry`; `initializeRemoteComms` RPC struct changed to carry a `NetlayerSpecifier`; `createCommsQueryString`/`getCommsParamsFromCurrentLocation` updated for the new config shape (`maxUrlRelayHints`/`maxKnownRelays` → renamed). | | `@metamask/kernel-cli` | modified | Changed: the `relay` command now runs the relay server from `@metamask/netlayer-libp2p/relay`; kernel/daemon config accepts a `NetlayerSpecifier`. | -| `@metamask/kernel-shims` | modified | Changed: the bare `import '@libp2p/webrtc'` moved out of `endoify-node.js` alongside the libp2p netlayer (flag for review — verify whether this is consumer-facing). | +| `@metamask/kernel-shims` | unchanged? | Phase 4 **deferred** relocating the bare `import '@libp2p/webrtc'` out of `endoify-node.js` (it still lives there). If Phase 6 does not move it either, there is no changelog entry; if this phase finally relocates it (a real follow-up option), record that change and flag for review. | | `@ocap/extension` | app (private) | Not published — likely `no-changelog`. Internal: hard-coded relay in `offscreen.ts` replaced by netlayer config. Confirm the package's changelog policy before writing. | Notes for the skill run: - New packages get a single `### Added` "Initial release" entry. -- Do not invent PR numbers; use the actual PRs from Phases 1–5 (and this Phase 6 PR for the +- Do not invent PR numbers; use the actual PRs from Phases 1–4 (and this Phase 6 PR for the rename). If unknown at execution time, leave a `#TODO-PR` placeholder and fill before merge. - Private/app packages (`extension`, `omnium-gatherum`, `evm-wallet-experiment`, `kernel-test`, `kernel-ui` if unpublished) generally take the `no-changelog` label rather diff --git a/package.json b/package.json index b7c9f6b968..294ed42499 100644 --- a/package.json +++ b/package.json @@ -127,6 +127,7 @@ "@metamask/kernel-cli>@metamask/kernel-node-runtime>@metamask/kernel-store>better-sqlite3": false, "@metamask/kernel-cli>@metamask/kernel-node-runtime>@metamask/streams": false, "@metamask/kernel-cli>@metamask/kernel-shims>@libp2p/webrtc>node-datachannel": false, + "@metamask/kernel-cli>@metamask/netlayer-libp2p>@libp2p/webrtc>node-datachannel": false, "@metamask/kernel-cli>@metamask/kernel-utils": false, "vitest>jsdom>ws>bufferutil": false, "vitest>jsdom>ws>utf-8-validate": false, diff --git a/packages/evm-wallet-experiment/docker/entrypoint-kernel.mjs b/packages/evm-wallet-experiment/docker/entrypoint-kernel.mjs index 1bff567268..a98ffce7a9 100644 --- a/packages/evm-wallet-experiment/docker/entrypoint-kernel.mjs +++ b/packages/evm-wallet-experiment/docker/entrypoint-kernel.mjs @@ -18,6 +18,7 @@ import '@metamask/kernel-shims/endoify-node'; import { NodejsPlatformServices } from '@metamask/kernel-node-runtime'; import { startRpcSocketServer } from '@metamask/kernel-node-runtime/daemon'; import { makeSQLKernelDatabase } from '@metamask/kernel-store/sqlite/nodejs'; +import { nodejsLibp2pNetlayerFactory } from '@metamask/netlayer-libp2p/nodejs'; import { Kernel } from '@metamask/ocap-kernel'; import { createWriteStream, @@ -73,9 +74,15 @@ async function main() { console.log(`[${NAME}] Booting kernel...`); const db = await makeSQLKernelDatabase({ dbFilename: ':memory:' }); - const kernel = await Kernel.make(new NodejsPlatformServices({}), db, { - resetStorage: true, - }); + const kernel = await Kernel.make( + new NodejsPlatformServices({ + netlayers: { libp2p: nodejsLibp2pNetlayerFactory }, + }), + db, + { + resetStorage: true, + }, + ); await kernel.initIdentity(); let peerId; diff --git a/packages/evm-wallet-experiment/package.json b/packages/evm-wallet-experiment/package.json index d504f3f01b..a7190a01aa 100644 --- a/packages/evm-wallet-experiment/package.json +++ b/packages/evm-wallet-experiment/package.json @@ -99,6 +99,7 @@ "@metamask/kernel-node-runtime": "workspace:^", "@metamask/kernel-shims": "workspace:^", "@metamask/kernel-store": "workspace:^", + "@metamask/netlayer-libp2p": "workspace:^", "@ocap/repo-tools": "workspace:^", "@ts-bridge/cli": "^0.6.3", "@ts-bridge/shims": "^0.1.1", diff --git a/packages/evm-wallet-experiment/scripts/home-interactive.mjs b/packages/evm-wallet-experiment/scripts/home-interactive.mjs index 6281dc19da..d1a1a611e8 100644 --- a/packages/evm-wallet-experiment/scripts/home-interactive.mjs +++ b/packages/evm-wallet-experiment/scripts/home-interactive.mjs @@ -347,6 +347,9 @@ async function main() { const { NodejsPlatformServices } = await import( '@metamask/kernel-node-runtime' ); + const { nodejsLibp2pNetlayerFactory } = await import( + '@metamask/netlayer-libp2p/nodejs' + ); const { makeWalletClusterConfig } = await import('../src/cluster-config.ts'); // ----------------------------------------------------------------------- @@ -359,7 +362,9 @@ async function main() { mkdirSync(dbDir, { recursive: true }); } const kernelDb = await makeSQLKernelDatabase({ dbFilename: args.dbPath }); - const platformServices = new NodejsPlatformServices({}); + const platformServices = new NodejsPlatformServices({ + netlayers: { libp2p: nodejsLibp2pNetlayerFactory }, + }); // Only log warnings and errors — suppress verbose kernel debug output. const logger = new Logger({ tags: ['ocap-kernel'], diff --git a/packages/evm-wallet-experiment/test/e2e/run-peer-e2e.mjs b/packages/evm-wallet-experiment/test/e2e/run-peer-e2e.mjs index 6a5e1923e8..a27498cab5 100644 --- a/packages/evm-wallet-experiment/test/e2e/run-peer-e2e.mjs +++ b/packages/evm-wallet-experiment/test/e2e/run-peer-e2e.mjs @@ -85,6 +85,7 @@ import '@metamask/kernel-shims/endoify-node'; import { NodejsPlatformServices } from '@metamask/kernel-node-runtime'; import { makeSQLKernelDatabase } from '@metamask/kernel-store/sqlite/nodejs'; import { waitUntilQuiescent } from '@metamask/kernel-utils'; +import { nodejsLibp2pNetlayerFactory } from '@metamask/netlayer-libp2p/nodejs'; import { Kernel, kunser } from '@metamask/ocap-kernel'; import { makeWalletClusterConfig } from '../../src/cluster-config.ts'; @@ -194,12 +195,24 @@ async function main() { const db1 = await makeSQLKernelDatabase({ dbFilename: ':memory:' }); const db2 = await makeSQLKernelDatabase({ dbFilename: ':memory:' }); - const kernel1 = await Kernel.make(new NodejsPlatformServices({}), db1, { - resetStorage: true, - }); - const kernel2 = await Kernel.make(new NodejsPlatformServices({}), db2, { - resetStorage: true, - }); + const kernel1 = await Kernel.make( + new NodejsPlatformServices({ + netlayers: { libp2p: nodejsLibp2pNetlayerFactory }, + }), + db1, + { + resetStorage: true, + }, + ); + const kernel2 = await Kernel.make( + new NodejsPlatformServices({ + netlayers: { libp2p: nodejsLibp2pNetlayerFactory }, + }), + db2, + { + resetStorage: true, + }, + ); await kernel1.initRemoteComms({ directListenAddresses: [QUIC_LISTEN_ADDRESS], diff --git a/packages/evm-wallet-experiment/test/e2e/run-sepolia-7702-direct-e2e.mjs b/packages/evm-wallet-experiment/test/e2e/run-sepolia-7702-direct-e2e.mjs index afc29a23c4..8f7388f0f6 100644 --- a/packages/evm-wallet-experiment/test/e2e/run-sepolia-7702-direct-e2e.mjs +++ b/packages/evm-wallet-experiment/test/e2e/run-sepolia-7702-direct-e2e.mjs @@ -18,6 +18,7 @@ import '@metamask/kernel-shims/endoify-node'; import { NodejsPlatformServices } from '@metamask/kernel-node-runtime'; import { makeSQLKernelDatabase } from '@metamask/kernel-store/sqlite/nodejs'; import { waitUntilQuiescent } from '@metamask/kernel-utils'; +import { nodejsLibp2pNetlayerFactory } from '@metamask/netlayer-libp2p/nodejs'; import { Kernel, kunser } from '@metamask/ocap-kernel'; import { makeWalletClusterConfig } from '../../src/cluster-config.ts'; @@ -80,9 +81,15 @@ async function main() { const rpcHost = new URL(SEPOLIA_RPC_URL).hostname; const kernelDb = await makeSQLKernelDatabase({ dbFilename: ':memory:' }); - const kernel = await Kernel.make(new NodejsPlatformServices({}), kernelDb, { - resetStorage: true, - }); + const kernel = await Kernel.make( + new NodejsPlatformServices({ + netlayers: { libp2p: nodejsLibp2pNetlayerFactory }, + }), + kernelDb, + { + resetStorage: true, + }, + ); const delegationManagerAddress = getDelegationManagerAddress(SEPOLIA_CHAIN_ID); diff --git a/packages/evm-wallet-experiment/test/e2e/run-sepolia-e2e.mjs b/packages/evm-wallet-experiment/test/e2e/run-sepolia-e2e.mjs index 563cd27052..084d45a9f4 100644 --- a/packages/evm-wallet-experiment/test/e2e/run-sepolia-e2e.mjs +++ b/packages/evm-wallet-experiment/test/e2e/run-sepolia-e2e.mjs @@ -20,6 +20,7 @@ import '@metamask/kernel-shims/endoify-node'; import { NodejsPlatformServices } from '@metamask/kernel-node-runtime'; import { makeSQLKernelDatabase } from '@metamask/kernel-store/sqlite/nodejs'; import { waitUntilQuiescent } from '@metamask/kernel-utils'; +import { nodejsLibp2pNetlayerFactory } from '@metamask/netlayer-libp2p/nodejs'; import { Kernel, kunser } from '@metamask/ocap-kernel'; import { makeWalletClusterConfig } from '../../src/cluster-config.ts'; @@ -104,9 +105,15 @@ async function main() { // -- Setup -- console.log('Setting up kernel...'); const kernelDb = await makeSQLKernelDatabase({ dbFilename: ':memory:' }); - const kernel = await Kernel.make(new NodejsPlatformServices({}), kernelDb, { - resetStorage: true, - }); + const kernel = await Kernel.make( + new NodejsPlatformServices({ + netlayers: { libp2p: nodejsLibp2pNetlayerFactory }, + }), + kernelDb, + { + resetStorage: true, + }, + ); const delegationManagerAddress = getDelegationManagerAddress(SEPOLIA_CHAIN_ID); diff --git a/packages/evm-wallet-experiment/test/e2e/run-spending-limits-e2e.mjs b/packages/evm-wallet-experiment/test/e2e/run-spending-limits-e2e.mjs index 2ab8d6c5d3..d5c2855535 100644 --- a/packages/evm-wallet-experiment/test/e2e/run-spending-limits-e2e.mjs +++ b/packages/evm-wallet-experiment/test/e2e/run-spending-limits-e2e.mjs @@ -37,6 +37,7 @@ import '@metamask/kernel-shims/endoify-node'; import { NodejsPlatformServices } from '@metamask/kernel-node-runtime'; import { makeSQLKernelDatabase } from '@metamask/kernel-store/sqlite/nodejs'; import { waitUntilQuiescent } from '@metamask/kernel-utils'; +import { nodejsLibp2pNetlayerFactory } from '@metamask/netlayer-libp2p/nodejs'; import { Kernel, kunser } from '@metamask/ocap-kernel'; import { makeWalletClusterConfig } from '../../src/cluster-config.ts'; @@ -203,9 +204,15 @@ async function main() { // -- Setup -- console.log('Setting up kernel...'); const kernelDb = await makeSQLKernelDatabase({ dbFilename: ':memory:' }); - const kernel = await Kernel.make(new NodejsPlatformServices({}), kernelDb, { - resetStorage: true, - }); + const kernel = await Kernel.make( + new NodejsPlatformServices({ + netlayers: { libp2p: nodejsLibp2pNetlayerFactory }, + }), + kernelDb, + { + resetStorage: true, + }, + ); console.log(` DelegationManager: ${delegationManagerAddress}`); const walletConfig = makeWalletClusterConfig({ diff --git a/packages/evm-wallet-experiment/test/integration/peer-wallet.test.ts b/packages/evm-wallet-experiment/test/integration/peer-wallet.test.ts index 5c87372f9e..ccbb69e9b3 100644 --- a/packages/evm-wallet-experiment/test/integration/peer-wallet.test.ts +++ b/packages/evm-wallet-experiment/test/integration/peer-wallet.test.ts @@ -2,6 +2,7 @@ import { NodejsPlatformServices } from '@metamask/kernel-node-runtime'; import type { KernelDatabase } from '@metamask/kernel-store'; import { makeSQLKernelDatabase } from '@metamask/kernel-store/sqlite/nodejs'; import { waitUntilQuiescent } from '@metamask/kernel-utils'; +import { nodejsLibp2pNetlayerFactory } from '@metamask/netlayer-libp2p/nodejs'; import { Kernel, kunser } from '@metamask/ocap-kernel'; import type { KRef } from '@metamask/ocap-kernel'; import { delay } from '@ocap/repo-tools/test-utils'; @@ -70,7 +71,9 @@ async function getConnectedInfo(kernel: Kernel): Promise<{ * @returns The kernel instance. */ async function makeTestKernel(kernelDatabase: KernelDatabase): Promise { - const platformServices = new NodejsPlatformServices({}); + const platformServices = new NodejsPlatformServices({ + netlayers: { libp2p: nodejsLibp2pNetlayerFactory }, + }); return Kernel.make(platformServices, kernelDatabase, { resetStorage: true, }); @@ -115,10 +118,16 @@ describe.sequential('Peer wallet integration', () => { // Initialize QUIC on both await kernel1.initRemoteComms({ - directListenAddresses: [QUIC_LISTEN_ADDRESS], + specifier: { + netlayer: 'libp2p', + config: { directListenAddresses: [QUIC_LISTEN_ADDRESS] }, + }, }); await kernel2.initRemoteComms({ - directListenAddresses: [QUIC_LISTEN_ADDRESS], + specifier: { + netlayer: 'libp2p', + config: { directListenAddresses: [QUIC_LISTEN_ADDRESS] }, + }, }); // Exchange location hints for direct connectivity diff --git a/packages/evm-wallet-experiment/test/integration/run-daemon-wallet.mjs b/packages/evm-wallet-experiment/test/integration/run-daemon-wallet.mjs index d6fd5ccef1..0f19545b2e 100644 --- a/packages/evm-wallet-experiment/test/integration/run-daemon-wallet.mjs +++ b/packages/evm-wallet-experiment/test/integration/run-daemon-wallet.mjs @@ -21,6 +21,7 @@ import { } from '@metamask/kernel-node-runtime/daemon'; import { makeSQLKernelDatabase } from '@metamask/kernel-store/sqlite/nodejs'; import { waitUntilQuiescent } from '@metamask/kernel-utils'; +import { nodejsLibp2pNetlayerFactory } from '@metamask/netlayer-libp2p/nodejs'; import { Kernel, kunser } from '@metamask/ocap-kernel'; import * as net from 'node:net'; import { tmpdir } from 'node:os'; @@ -163,7 +164,9 @@ async function main() { console.log('Booting daemon stack...'); const socketPath = tempSocketPath(); const kernelDb = await makeSQLKernelDatabase({ dbFilename: ':memory:' }); - const platformServices = new NodejsPlatformServices({}); + const platformServices = new NodejsPlatformServices({ + netlayers: { libp2p: nodejsLibp2pNetlayerFactory }, + }); const kernel = await Kernel.make(platformServices, kernelDb, { resetStorage: true, }); diff --git a/packages/evm-wallet-experiment/test/integration/run-peer-wallet.mjs b/packages/evm-wallet-experiment/test/integration/run-peer-wallet.mjs index 8060af389c..7f2874d023 100644 --- a/packages/evm-wallet-experiment/test/integration/run-peer-wallet.mjs +++ b/packages/evm-wallet-experiment/test/integration/run-peer-wallet.mjs @@ -14,6 +14,7 @@ import '@metamask/kernel-shims/endoify-node'; import { NodejsPlatformServices } from '@metamask/kernel-node-runtime'; import { makeSQLKernelDatabase } from '@metamask/kernel-store/sqlite/nodejs'; import { waitUntilQuiescent } from '@metamask/kernel-utils'; +import { nodejsLibp2pNetlayerFactory } from '@metamask/netlayer-libp2p/nodejs'; import { Kernel, kunser } from '@metamask/ocap-kernel'; import { makeWalletClusterConfig } from '../../src/cluster-config.ts'; @@ -100,12 +101,24 @@ async function main() { const db1 = await makeSQLKernelDatabase({ dbFilename: ':memory:' }); const db2 = await makeSQLKernelDatabase({ dbFilename: ':memory:' }); - const kernel1 = await Kernel.make(new NodejsPlatformServices({}), db1, { - resetStorage: true, - }); - const kernel2 = await Kernel.make(new NodejsPlatformServices({}), db2, { - resetStorage: true, - }); + const kernel1 = await Kernel.make( + new NodejsPlatformServices({ + netlayers: { libp2p: nodejsLibp2pNetlayerFactory }, + }), + db1, + { + resetStorage: true, + }, + ); + const kernel2 = await Kernel.make( + new NodejsPlatformServices({ + netlayers: { libp2p: nodejsLibp2pNetlayerFactory }, + }), + db2, + { + resetStorage: true, + }, + ); await kernel1.initRemoteComms({ directListenAddresses: [QUIC_LISTEN_ADDRESS], diff --git a/packages/evm-wallet-experiment/test/integration/run-wallet.mjs b/packages/evm-wallet-experiment/test/integration/run-wallet.mjs index 6e8fbf129a..68797db605 100644 --- a/packages/evm-wallet-experiment/test/integration/run-wallet.mjs +++ b/packages/evm-wallet-experiment/test/integration/run-wallet.mjs @@ -16,6 +16,7 @@ import '@metamask/kernel-shims/endoify-node'; import { NodejsPlatformServices } from '@metamask/kernel-node-runtime'; import { makeSQLKernelDatabase } from '@metamask/kernel-store/sqlite/nodejs'; import { waitUntilQuiescent } from '@metamask/kernel-utils'; +import { nodejsLibp2pNetlayerFactory } from '@metamask/netlayer-libp2p/nodejs'; import { Kernel, kunser } from '@metamask/ocap-kernel'; import { makeWalletClusterConfig } from '../../src/cluster-config.ts'; @@ -93,7 +94,9 @@ async function main() { console.log('Setting up kernel...'); const kernelDb = await makeSQLKernelDatabase({ dbFilename: ':memory:' }); - const platformServices = new NodejsPlatformServices({}); + const platformServices = new NodejsPlatformServices({ + netlayers: { libp2p: nodejsLibp2pNetlayerFactory }, + }); const kernel = await Kernel.make(platformServices, kernelDb, { resetStorage: true, }); diff --git a/packages/evm-wallet-experiment/tsconfig.json b/packages/evm-wallet-experiment/tsconfig.json index 4d4a24e218..0469a50f3b 100644 --- a/packages/evm-wallet-experiment/tsconfig.json +++ b/packages/evm-wallet-experiment/tsconfig.json @@ -11,6 +11,7 @@ { "path": "../kernel-store" }, { "path": "../logger" }, { "path": "../kernel-node-runtime" }, + { "path": "../netlayer-libp2p" }, { "path": "../ocap-kernel" } ], "include": [ diff --git a/packages/extension/package.json b/packages/extension/package.json index 226563facd..86d633c5d8 100644 --- a/packages/extension/package.json +++ b/packages/extension/package.json @@ -49,6 +49,7 @@ "@metamask/kernel-ui": "workspace:^", "@metamask/kernel-utils": "workspace:^", "@metamask/logger": "workspace:^", + "@metamask/netlayer-libp2p": "workspace:^", "@metamask/ocap-kernel": "workspace:^", "@metamask/streams": "workspace:^", "react": "^18.3.1", diff --git a/packages/extension/src/offscreen.ts b/packages/extension/src/offscreen.ts index 7c5eeb8c55..af91a5d123 100644 --- a/packages/extension/src/offscreen.ts +++ b/packages/extension/src/offscreen.ts @@ -8,6 +8,7 @@ import { import { delay, isJsonRpcMessage } from '@metamask/kernel-utils'; import type { JsonRpcMessage } from '@metamask/kernel-utils'; import { Logger } from '@metamask/logger'; +import { libp2pNetlayerFactory } from '@metamask/netlayer-libp2p'; import type { DuplexStream } from '@metamask/streams'; import { initializeMessageChannel, @@ -65,9 +66,13 @@ async function makeKernelWorker(): Promise< DuplexStream > { const workerUrlParams = createCommsQueryString({ - relays: [ - '/ip4/127.0.0.1/tcp/9001/ws/p2p/12D3KooWJBDqsyHQF2MWiCdU4kdqx4zTsSTLRdShg7Ui6CRWB4uc', - ], + netlayer: 'libp2p', + config: { + knownRelays: [ + process.env.RELAY_MULTIADDR ?? + '/ip4/127.0.0.1/tcp/9001/ws/p2p/12D3KooWJBDqsyHQF2MWiCdU4kdqx4zTsSTLRdShg7Ui6CRWB4uc', + ], + }, }); workerUrlParams.set('reset-storage', process.env.RESET_STORAGE ?? 'false'); @@ -90,15 +95,18 @@ async function makeKernelWorker(): Promise< JsonRpcMessage >(port, isJsonRpcMessage); - await PlatformServicesServer.make(worker as PostMessageTarget, (vatId) => - makeIframeVatWorker({ - id: vatId, - iframeUri: 'iframe.html', - getPort: initializeMessageChannel, - logger: logger.subLogger({ - tags: ['iframe-vat-worker', vatId], + await PlatformServicesServer.make( + worker as PostMessageTarget, + (vatId) => + makeIframeVatWorker({ + id: vatId, + iframeUri: 'iframe.html', + getPort: initializeMessageChannel, + logger: logger.subLogger({ + tags: ['iframe-vat-worker', vatId], + }), }), - }), + { netlayers: { libp2p: libp2pNetlayerFactory } }, ); return kernelStream; diff --git a/packages/extension/test/e2e/remote-comms.test.ts b/packages/extension/test/e2e/remote-comms.test.ts index 1adac48f40..1366d4f18c 100644 --- a/packages/extension/test/e2e/remote-comms.test.ts +++ b/packages/extension/test/e2e/remote-comms.test.ts @@ -1,4 +1,4 @@ -import { startRelay } from '@metamask/kernel-utils/libp2p'; +import { startRelay } from '@metamask/netlayer-libp2p/relay'; import { test, expect } from '@playwright/test'; import type { Page, BrowserContext } from '@playwright/test'; import { rm } from 'node:fs/promises'; diff --git a/packages/extension/tsconfig.json b/packages/extension/tsconfig.json index e2d7cddd26..44a653f770 100644 --- a/packages/extension/tsconfig.json +++ b/packages/extension/tsconfig.json @@ -18,6 +18,7 @@ { "path": "../kernel-store" }, { "path": "../kernel-utils" }, { "path": "../logger" }, + { "path": "../netlayer-libp2p" }, { "path": "../ocap-kernel" }, { "path": "../streams" }, { "path": "../repo-tools" } diff --git a/packages/kernel-browser-runtime/CHANGELOG.md b/packages/kernel-browser-runtime/CHANGELOG.md index b63734215d..6ef076d539 100644 --- a/packages/kernel-browser-runtime/CHANGELOG.md +++ b/packages/kernel-browser-runtime/CHANGELOG.md @@ -7,6 +7,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Changed + +- **BREAKING:** `PlatformServicesServer` (constructor and `make`) now takes an `args` bag with a required `netlayers: NetlayerRegistry` (plus optional `logger`), and `PlatformServicesClient.initializeRemoteComms` takes the neutral options bag `{ keySeed, specifier, hooks, incarnationId? }` ([#973](https://github.com/MetaMask/ocap-kernel/pull/973)) +- **BREAKING:** `comms-query-string` now carries a `NetlayerSpecifier` (`netlayer` + JSON-encoded `config`) plus the kernel-level numeric options, instead of per-field `RemoteCommsOptions`; `parseCommsQueryString`/`getCommsParamsFromCurrentLocation` return `RemoteCommsOptions` with a `specifier` ([#973](https://github.com/MetaMask/ocap-kernel/pull/973)) + ### Fixed - Process platform-services RPC request handlers in the background so a request handler that fires a reentrant outbound RPC (e.g. transport handshake calling back into the kernel) cannot deadlock waiting for its response ([#948](https://github.com/MetaMask/ocap-kernel/pull/948)) diff --git a/packages/kernel-browser-runtime/src/PlatformServicesClient.test.ts b/packages/kernel-browser-runtime/src/PlatformServicesClient.test.ts index aba8f598ef..936bc25c35 100644 --- a/packages/kernel-browser-runtime/src/PlatformServicesClient.test.ts +++ b/packages/kernel-browser-runtime/src/PlatformServicesClient.test.ts @@ -240,56 +240,34 @@ describe('PlatformServicesClient', () => { describe('remote communications', () => { describe('initializeRemoteComms', () => { - it('sends initializeRemoteComms request and resolves', async () => { - const remoteHandler = vi.fn(async () => 'response'); - const result = client.initializeRemoteComms( - '0xabcd', - { relays: ['/dns4/relay.example/tcp/443/wss/p2p/relayPeer'] }, - remoteHandler, - ); - await stream.receiveInput(makeNullReply('m1')); - expect(await result).toBeUndefined(); - }); - - it('sends initializeRemoteComms with all options and resolves', async () => { - const remoteHandler = vi.fn(async () => 'response'); - const result = client.initializeRemoteComms( - '0xabcd', - { - relays: ['/dns4/relay.example/tcp/443/wss/p2p/relayPeer'], - maxRetryAttempts: 5, - maxQueue: 100, - }, - remoteHandler, - ); - await stream.receiveInput(makeNullReply('m1')); - expect(await result).toBeUndefined(); - }); + const specifier = { + netlayer: 'libp2p', + config: { + knownRelays: ['/dns4/relay.example/tcp/443/wss/p2p/relayPeer'], + }, + }; - it('sends initializeRemoteComms with onRemoteGiveUp callback', async () => { - const remoteHandler = vi.fn(async () => 'response'); - const giveUpHandler = vi.fn(); - const result = client.initializeRemoteComms( - '0xabcd', - { relays: ['/dns4/relay.example/tcp/443/wss/p2p/relayPeer'] }, - remoteHandler, - giveUpHandler, - ); + it('sends an initializeRemoteComms request and resolves', async () => { + const result = client.initializeRemoteComms({ + keySeed: '0xabcd', + specifier, + hooks: { handleMessage: vi.fn(async () => 'response') }, + }); await stream.receiveInput(makeNullReply('m1')); expect(await result).toBeUndefined(); }); - it('filters undefined values from options', async () => { - const remoteHandler = vi.fn(async () => 'response'); - const result = client.initializeRemoteComms( - '0xabcd', - { - relays: ['/dns4/relay.example/tcp/443/wss/p2p/relayPeer'], - maxRetryAttempts: undefined, - maxQueue: undefined, + it('sends the request with an incarnationId and callbacks', async () => { + const result = client.initializeRemoteComms({ + keySeed: '0xabcd', + specifier, + hooks: { + handleMessage: vi.fn(async () => 'response'), + onRemoteGiveUp: vi.fn(), + onIncarnationChange: vi.fn(async () => true), }, - remoteHandler, - ); + incarnationId: 'inc-1', + }); await stream.receiveInput(makeNullReply('m1')); expect(await result).toBeUndefined(); }); @@ -352,6 +330,21 @@ describe('PlatformServicesClient', () => { }); }); + describe('resetAllBackoffs', () => { + it('sends resetAllBackoffs request and resolves', async () => { + const result = client.resetAllBackoffs(); + await delay(10); + await stream.receiveInput(makeNullReply('m1')); + expect(await result).toBeUndefined(); + }); + }); + + describe('getListenAddresses', () => { + it('returns an empty array (direct transport is Node-only)', () => { + expect(client.getListenAddresses()).toStrictEqual([]); + }); + }); + describe('remoteDeliver', () => { it('throws error when handler not set', async () => { // Client without initialized remote comms @@ -396,11 +389,11 @@ describe('PlatformServicesClient', () => { await delay(10); const remoteHandler = vi.fn(async () => 'response-message'); - const initP = testClient.initializeRemoteComms( - '0xabcd', - {}, - remoteHandler, - ); + const initP = testClient.initializeRemoteComms({ + keySeed: '0xabcd', + specifier: { netlayer: 'libp2p', config: {} }, + hooks: { handleMessage: remoteHandler }, + }); await testStream.receiveInput(makeNullReply('m1')); await initP; @@ -449,12 +442,14 @@ describe('PlatformServicesClient', () => { const remoteHandler = vi.fn(async () => 'response'); const giveUpHandler = vi.fn(); - const initP = testClient.initializeRemoteComms( - '0xabcd', - {}, - remoteHandler, - giveUpHandler, - ); + const initP = testClient.initializeRemoteComms({ + keySeed: '0xabcd', + specifier: { netlayer: 'libp2p', config: {} }, + hooks: { + handleMessage: remoteHandler, + onRemoteGiveUp: giveUpHandler, + }, + }); await testStream.receiveInput(makeNullReply('m1')); await initP; @@ -529,14 +524,16 @@ describe('PlatformServicesClient', () => { const remoteHandler = vi.fn(async () => 'response'); const giveUpHandler = vi.fn(); const incarnationHandler = vi.fn(async () => true); - const initP = testClient.initializeRemoteComms( - '0xabcd', - {}, - remoteHandler, - giveUpHandler, - 'local-incarnation', - incarnationHandler, - ); + const initP = testClient.initializeRemoteComms({ + keySeed: '0xabcd', + specifier: { netlayer: 'libp2p', config: {} }, + hooks: { + handleMessage: remoteHandler, + onRemoteGiveUp: giveUpHandler, + onIncarnationChange: incarnationHandler, + }, + incarnationId: 'local-incarnation', + }); await testStream.receiveInput(makeNullReply('m1')); await initP; @@ -618,14 +615,15 @@ describe('PlatformServicesClient', () => { const incarnationHandler = vi.fn( async () => 'oops' as unknown as boolean, ); - const initP = testClient.initializeRemoteComms( - '0xabcd', - {}, - remoteHandler, - undefined, - 'local-incarnation', - incarnationHandler, - ); + const initP = testClient.initializeRemoteComms({ + keySeed: '0xabcd', + specifier: { netlayer: 'libp2p', config: {} }, + hooks: { + handleMessage: remoteHandler, + onIncarnationChange: incarnationHandler, + }, + incarnationId: 'local-incarnation', + }); await testStream.receiveInput(makeNullReply('m1')); await initP; @@ -669,14 +667,15 @@ describe('PlatformServicesClient', () => { const incarnationHandler = vi.fn(async () => { throw new Error('handler exploded'); }); - const initP = testClient.initializeRemoteComms( - '0xabcd', - {}, - remoteHandler, - undefined, - 'local-incarnation', - incarnationHandler, - ); + const initP = testClient.initializeRemoteComms({ + keySeed: '0xabcd', + specifier: { netlayer: 'libp2p', config: {} }, + hooks: { + handleMessage: remoteHandler, + onIncarnationChange: incarnationHandler, + }, + incarnationId: 'local-incarnation', + }); await testStream.receiveInput(makeNullReply('m1')); await initP; diff --git a/packages/kernel-browser-runtime/src/PlatformServicesClient.ts b/packages/kernel-browser-runtime/src/PlatformServicesClient.ts index 41461b8825..816f0e4d4e 100644 --- a/packages/kernel-browser-runtime/src/PlatformServicesClient.ts +++ b/packages/kernel-browser-runtime/src/PlatformServicesClient.ts @@ -7,8 +7,9 @@ import type { RemoteMessageHandler, VatId, VatConfig, - RemoteCommsOptions, OnIncarnationChange, + NetlayerSpecifier, + NetlayerHooks, } from '@metamask/ocap-kernel'; import { platformServicesMethodSpecs, @@ -189,40 +190,35 @@ export class PlatformServicesClient implements PlatformServices { } /** - * Initialize network communications. + * Initialize network communications. This client is a pure RPC proxy: it + * keeps the netlayer hooks locally (functions can't cross the RPC boundary) + * and forwards `keySeed`, the `Json` specifier, and `incarnationId`. * - * @param keySeed - The seed for generating this kernel's secret key. - * @param options - Options for remote communications initialization. - * @param options.relays - Array of the peerIDs of relay nodes that can be used to listen for incoming - * connections from other kernels. - * @param options.maxRetryAttempts - Maximum number of reconnection attempts. 0 = infinite (default). - * @param options.maxQueue - Maximum number of messages to queue per peer while reconnecting (default: 200). - * @param remoteMessageHandler - A handler function to receive remote messages. - * @param onRemoteGiveUp - Optional callback to be called when we give up on a remote. - * @param incarnationId - Unique identifier for this kernel instance. - * @param onIncarnationChange - Optional callback when a remote peer's incarnation changes. + * @param params - The initialization options. + * @param params.keySeed - The seed for generating this kernel's secret key. + * @param params.specifier - Which netlayer to use plus its `Json` config. + * @param params.hooks - Kernel-supplied netlayer callbacks, retained locally. + * @param params.incarnationId - This kernel's incarnation ID for the handshake. * @returns A promise that resolves once network access has been established * or rejects if there is some problem doing so. */ - async initializeRemoteComms( - keySeed: string, - options: RemoteCommsOptions, - remoteMessageHandler: ( - from: string, - message: string, - ) => Promise, - onRemoteGiveUp?: (peerId: string) => void, - incarnationId?: string, - onIncarnationChange?: OnIncarnationChange, - ): Promise { - this.#remoteMessageHandler = remoteMessageHandler; - this.#remoteGiveUpHandler = onRemoteGiveUp; - this.#remoteIncarnationChangeHandler = onIncarnationChange; + async initializeRemoteComms({ + keySeed, + specifier, + hooks, + incarnationId, + }: { + keySeed: string; + specifier: NetlayerSpecifier; + hooks: NetlayerHooks; + incarnationId?: string; + }): Promise { + this.#remoteMessageHandler = hooks.handleMessage; + this.#remoteGiveUpHandler = hooks.onRemoteGiveUp; + this.#remoteIncarnationChangeHandler = hooks.onIncarnationChange; await this.#rpcClient.call('initializeRemoteComms', { keySeed, - ...Object.fromEntries( - Object.entries(options).filter(([, value]) => value !== undefined), - ), + specifier, ...(incarnationId !== undefined && { incarnationId }), }); } diff --git a/packages/kernel-browser-runtime/src/PlatformServicesServer.test.ts b/packages/kernel-browser-runtime/src/PlatformServicesServer.test.ts index 9a4fc7c0cc..1990d4b56b 100644 --- a/packages/kernel-browser-runtime/src/PlatformServicesServer.test.ts +++ b/packages/kernel-browser-runtime/src/PlatformServicesServer.test.ts @@ -18,12 +18,14 @@ import type { PlatformServicesStream, } from './PlatformServicesServer.ts'; -// Mock initTransport from ocap-kernel +// A fake netlayer factory (registry entry) that captures the hooks the server +// reconstructs locally and returns stubbed netlayer methods. const mockSendRemoteMessage = vi.fn(async () => undefined); const mockStop = vi.fn(async () => undefined); const mockCloseConnection = vi.fn(async () => undefined); const mockRegisterLocationHints = vi.fn(async () => undefined); const mockReconnectPeer = vi.fn(async () => undefined); +const mockResetAllBackoffs = vi.fn(() => undefined); let capturedRemoteMessageHandler: | ((from: string, message: string) => Promise) | undefined; @@ -32,37 +34,37 @@ let capturedOnIncarnationChange: | ((peerId: string, observedIncarnation: string) => Promise) | undefined; -vi.mock('@metamask/ocap-kernel', () => ({ - PlatformServicesCommandMethod: { - launch: 'launch', - terminate: 'terminate', - terminateAll: 'terminateAll', +const fakeNetlayerFactory = vi.fn( + async ({ + hooks, + }: { + hooks: { + handleMessage: (from: string, message: string) => Promise; + onRemoteGiveUp?: (peerId: string) => void; + onIncarnationChange?: ( + peerId: string, + observedIncarnation: string, + ) => Promise; + }; + }) => { + capturedRemoteMessageHandler = hooks.handleMessage; + capturedRemoteGiveUpHandler = hooks.onRemoteGiveUp; + capturedOnIncarnationChange = hooks.onIncarnationChange; + return { + sendRemoteMessage: mockSendRemoteMessage, + stop: mockStop, + closeConnection: mockCloseConnection, + registerLocationHints: mockRegisterLocationHints, + reconnectPeer: mockReconnectPeer, + resetAllBackoffs: mockResetAllBackoffs, + getListenAddresses: vi.fn(() => []), + }; }, - initTransport: vi.fn( - async ( - _keySeed: string, - _options: unknown, - remoteMessageHandler: (from: string, message: string) => Promise, - remoteGiveUpHandler: (peerId: string) => void, - _localIncarnationId: string | undefined, - onIncarnationChange: - | ((peerId: string, observedIncarnation: string) => Promise) - | undefined, - ) => { - capturedRemoteMessageHandler = remoteMessageHandler; - capturedRemoteGiveUpHandler = remoteGiveUpHandler; - capturedOnIncarnationChange = onIncarnationChange; - return { - sendRemoteMessage: mockSendRemoteMessage, - stop: mockStop, - closeConnection: mockCloseConnection, - registerLocationHints: mockRegisterLocationHints, - reconnectPeer: mockReconnectPeer, - getListenAddresses: vi.fn(() => []), - }; - }, - ), -})); +); + +const netlayers = { libp2p: fakeNetlayerFactory } as unknown as Parameters< + typeof PlatformServicesServer.make +>[2]['netlayers']; const makeVatConfig = (sourceSpec = 'bogus.js'): VatConfig => ({ sourceSpec, @@ -104,11 +106,16 @@ const makeTerminateAllMessageEvent = (messageId: `m${number}`): MessageEvent => const makeInitializeRemoteCommsMessageEvent = ( messageId: `m${number}`, keySeed: string, - options: { relays?: string[]; maxRetryAttempts?: number; maxQueue?: number }, + config: Record = {}, + incarnationId?: string, ): MessageEvent => makeMessageEvent(messageId, { method: 'initializeRemoteComms', - params: { keySeed, ...options }, + params: { + keySeed, + specifier: { netlayer: 'libp2p', config }, + ...(incarnationId !== undefined && { incarnationId }), + }, }); const makeSendRemoteMessageMessageEvent = ( @@ -182,6 +189,7 @@ describe('PlatformServicesServer', () => { const server = new PlatformServicesServer( stream as unknown as PlatformServicesStream, () => ({}) as unknown as VatWorker, + { netlayers }, ); expect(server).toBeDefined(); }); @@ -190,6 +198,7 @@ describe('PlatformServicesServer', () => { const server = await PlatformServicesServer.make( makeMockMessageTarget(), () => ({}) as unknown as VatWorker, + { netlayers }, ); expect(server).toBeDefined(); }); @@ -227,7 +236,7 @@ describe('PlatformServicesServer', () => { new PlatformServicesServer( stream as unknown as PlatformServicesStream, makeMockVatWorker, - logger, + { netlayers, logger }, ); addCleanup(async () => { @@ -256,7 +265,7 @@ describe('PlatformServicesServer', () => { new PlatformServicesServer( testStream as unknown as PlatformServicesStream, makeMockVatWorker, - logger, + { netlayers, logger }, ); // Send a response (simulating RPC client response) @@ -401,47 +410,66 @@ describe('PlatformServicesServer', () => { }); describe('initializeRemoteComms', () => { - it('initializes remote comms with keySeed and relays', async () => { + it('looks up the netlayer factory with the keySeed and config', async () => { const keySeed = '0x1234567890abcdef'; - const relays = ['/dns4/relay.example/tcp/443/wss/p2p/relayPeer']; + const knownRelays = ['/dns4/relay.example/tcp/443/wss/p2p/relayPeer']; await stream.receiveInput( - makeInitializeRemoteCommsMessageEvent('m0', keySeed, { relays }), + makeInitializeRemoteCommsMessageEvent('m0', keySeed, { + knownRelays, + }), ); await delay(10); - const { initTransport } = await import('@metamask/ocap-kernel'); - expect(initTransport).toHaveBeenCalledWith( - keySeed, - { relays }, - expect.any(Function), - expect.any(Function), - undefined, - expect.any(Function), + expect(fakeNetlayerFactory).toHaveBeenCalledWith( + expect.objectContaining({ + keySeed, + config: { knownRelays }, + hooks: expect.objectContaining({ + handleMessage: expect.any(Function), + onRemoteGiveUp: expect.any(Function), + onIncarnationChange: expect.any(Function), + }), + }), ); }); - it('initializes remote comms with all options', async () => { + it('passes the full config through to the factory', async () => { const keySeed = '0x1234567890abcdef'; - const options = { - relays: ['/dns4/relay.example/tcp/443/wss/p2p/relayPeer'], + const config = { + knownRelays: ['/dns4/relay.example/tcp/443/wss/p2p/relayPeer'], maxRetryAttempts: 5, - maxQueue: 100, + maxMessageSizeBytes: 100, }; await stream.receiveInput( - makeInitializeRemoteCommsMessageEvent('m0', keySeed, options), + makeInitializeRemoteCommsMessageEvent('m0', keySeed, config), + ); + await delay(10); + + expect(fakeNetlayerFactory).toHaveBeenCalledWith( + expect.objectContaining({ keySeed, config }), + ); + }); + + it('throws for an unknown netlayer', async () => { + const errorSpy = vi.spyOn(logger, 'error'); + await stream.receiveInput( + makeMessageEvent('m0', { + method: 'initializeRemoteComms', + params: { + keySeed: '0xabcd', + specifier: { netlayer: 'nope', config: {} }, + }, + }), ); await delay(10); - const { initTransport } = await import('@metamask/ocap-kernel'); - expect(initTransport).toHaveBeenCalledWith( - keySeed, - options, - expect.any(Function), - expect.any(Function), - undefined, - expect.any(Function), + expect(errorSpy).toHaveBeenCalledWith( + 'Error handling "initializeRemoteComms" request:', + expect.objectContaining({ + message: 'Unknown netlayer: "nope"', + }), ); }); @@ -500,7 +528,7 @@ describe('PlatformServicesServer', () => { new PlatformServicesServer( testStream as unknown as PlatformServicesStream, makeMockVatWorker, - logger, + { netlayers, logger }, ); await testStream.receiveInput( makeInitializeRemoteCommsMessageEvent('m1', keySeed, { relays }), @@ -556,7 +584,7 @@ describe('PlatformServicesServer', () => { new PlatformServicesServer( testStream as unknown as PlatformServicesStream, makeMockVatWorker, - logger, + { netlayers, logger }, ); await testStream.receiveInput( makeInitializeRemoteCommsMessageEvent('m0', keySeed, { relays }), @@ -605,7 +633,7 @@ describe('PlatformServicesServer', () => { new PlatformServicesServer( testStream as unknown as PlatformServicesStream, makeMockVatWorker, - logger, + { netlayers, logger }, ); await testStream.receiveInput( makeInitializeRemoteCommsMessageEvent('m0', keySeed, { relays }), @@ -668,7 +696,7 @@ describe('PlatformServicesServer', () => { new PlatformServicesServer( testStream as unknown as PlatformServicesStream, makeMockVatWorker, - logger, + { netlayers, logger }, ); await testStream.receiveInput( makeInitializeRemoteCommsMessageEvent('m1', keySeed, { relays }), @@ -777,8 +805,7 @@ describe('PlatformServicesServer', () => { await stream.receiveInput(makeStopRemoteCommsMessageEvent('m1')); await delay(10); - const { initTransport } = await import('@metamask/ocap-kernel'); - const firstCallCount = (initTransport as Mock).mock.calls.length; + const firstCallCount = fakeNetlayerFactory.mock.calls.length; // Re-initialize should work await stream.receiveInput( @@ -786,8 +813,8 @@ describe('PlatformServicesServer', () => { ); await delay(10); - // Should have called initTransport again - expect((initTransport as Mock).mock.calls).toHaveLength( + // Should have called the netlayer factory again + expect(fakeNetlayerFactory.mock.calls).toHaveLength( firstCallCount + 1, ); }); @@ -931,6 +958,35 @@ describe('PlatformServicesServer', () => { ); }); }); + + describe('resetAllBackoffs', () => { + it('is a no-op before remote comms is initialized', async () => { + const errorSpy = vi.spyOn(logger, 'error'); + await stream.receiveInput( + makeMessageEvent('m0', { + method: 'resetAllBackoffs', + params: [], + }), + ); + await delay(10); + expect(errorSpy).not.toHaveBeenCalled(); + }); + + it('resets backoffs on the netlayer after initialization', async () => { + await stream.receiveInput( + makeInitializeRemoteCommsMessageEvent('m0', '0xabcd', {}), + ); + await delay(10); + await stream.receiveInput( + makeMessageEvent('m1', { + method: 'resetAllBackoffs', + params: [], + }), + ); + await delay(10); + expect(mockResetAllBackoffs).toHaveBeenCalledOnce(); + }); + }); }); }); }); diff --git a/packages/kernel-browser-runtime/src/PlatformServicesServer.ts b/packages/kernel-browser-runtime/src/PlatformServicesServer.ts index d176cc7e14..43cb61a8d3 100644 --- a/packages/kernel-browser-runtime/src/PlatformServicesServer.ts +++ b/packages/kernel-browser-runtime/src/PlatformServicesServer.ts @@ -11,9 +11,9 @@ import type { VatConfig, SendRemoteMessage, StopRemoteComms, - RemoteCommsOptions, + NetlayerRegistry, + NetlayerSpecifier, } from '@metamask/ocap-kernel'; -import { initTransport } from '@metamask/ocap-kernel'; import { kernelRemoteMethodSpecs, platformServicesHandlers, @@ -72,6 +72,8 @@ export class PlatformServicesServer { readonly #makeWorker: (vatId: VatId) => VatWorker; + readonly #netlayers: NetlayerRegistry; + #sendRemoteMessageFunc: SendRemoteMessage | null = null; #stopRemoteCommsFunc: StopRemoteComms | null = null; @@ -104,16 +106,19 @@ export class PlatformServicesServer { * * @param stream - The stream to use for communication with the client. * @param makeWorker - A method for making a {@link VatWorker}. - * @param logger - An optional {@link Logger}. Defaults to a new logger labeled '[platform services server]'. + * @param args - Construction arguments. + * @param args.netlayers - The registry of netlayer factories, keyed by name. + * @param args.logger - An optional {@link Logger}. Defaults to a new logger labeled '[platform services server]'. */ constructor( stream: PlatformServicesStream, makeWorker: (vatId: VatId) => VatWorker, - logger?: Logger, + args: { netlayers: NetlayerRegistry; logger?: Logger }, ) { this.#stream = stream; this.#makeWorker = makeWorker; - this.#logger = logger ?? new Logger('platform-services-server'); + this.#netlayers = args.netlayers; + this.#logger = args.logger ?? new Logger('platform-services-server'); this.#rpcClient = new RpcClient( kernelRemoteMethodSpecs, @@ -150,13 +155,15 @@ export class PlatformServicesServer { * * @param messageTarget - The target to use for posting and receiving messages. * @param makeWorker - A method for making a {@link VatWorker}. - * @param logger - An optional {@link Logger}. + * @param args - Construction arguments. + * @param args.netlayers - The registry of netlayer factories, keyed by name. + * @param args.logger - An optional {@link Logger}. * @returns A new {@link PlatformServicesServer}. */ static async make( messageTarget: PostMessageTarget, makeWorker: (vatId: VatId) => VatWorker, - logger?: Logger, + args: { netlayers: NetlayerRegistry; logger?: Logger }, ): Promise { const stream: PlatformServicesStream = new PostMessageDuplexStream({ messageTarget, @@ -167,7 +174,7 @@ export class PlatformServicesServer { message instanceof MessageEvent && isJsonRpcMessage(message.data), }); await stream.synchronize(); - return new PlatformServicesServer(stream, makeWorker, logger); + return new PlatformServicesServer(stream, makeWorker, args); } /** @@ -284,25 +291,30 @@ export class PlatformServicesServer { } /** - * Initialize network communications. + * Initialize network communications by looking up the specified netlayer in + * the injected registry and constructing it. The reverse-RPC hooks are + * reconstructed locally; only `keySeed`, the `Json` specifier, and + * `incarnationId` crossed the RPC boundary. * * @param keySeed - The seed for generating this kernel's secret key. - * @param options - Options for remote communications initialization. - * @param options.relays - Array of the peerIDs of relay nodes that can be used to listen for incoming - * connections from other kernels. - * @param options.maxRetryAttempts - Maximum number of reconnection attempts. 0 = infinite (default). - * @param options.maxQueue - Maximum number of messages to queue per peer while reconnecting (default: 200). + * @param specifier - Which netlayer to use plus its `Json` config. * @param incarnationId - This kernel's incarnation ID for handshake protocol. * @returns A promise that resolves when network access has been initialized. */ async #initializeRemoteComms( keySeed: string, - options: RemoteCommsOptions, + specifier: NetlayerSpecifier, incarnationId?: string, ): Promise { if (this.#sendRemoteMessageFunc) { throw Error('remote comms already initialized'); } + + const factory = this.#netlayers[specifier.netlayer]; + if (!factory) { + throw new Error(`Unknown netlayer: "${specifier.netlayer}"`); + } + const { sendRemoteMessage, stop, @@ -310,14 +322,17 @@ export class PlatformServicesServer { registerLocationHints, reconnectPeer, resetAllBackoffs, - } = await initTransport( + } = await factory({ keySeed, - options, - this.#handleRemoteMessage.bind(this), - this.#handleRemoteGiveUp.bind(this), incarnationId, - this.#handleRemoteIncarnationChange.bind(this), - ); + hooks: { + handleMessage: this.#handleRemoteMessage.bind(this), + onRemoteGiveUp: this.#handleRemoteGiveUp.bind(this), + onIncarnationChange: this.#handleRemoteIncarnationChange.bind(this), + }, + config: specifier.config, + logger: this.#logger, + }); this.#sendRemoteMessageFunc = sendRemoteMessage; this.#stopRemoteCommsFunc = stop; this.#closeConnectionFunc = closeConnection; diff --git a/packages/kernel-browser-runtime/src/utils/comms-query-string.test.ts b/packages/kernel-browser-runtime/src/utils/comms-query-string.test.ts index f968b22d8f..5163d3e38c 100644 --- a/packages/kernel-browser-runtime/src/utils/comms-query-string.test.ts +++ b/packages/kernel-browser-runtime/src/utils/comms-query-string.test.ts @@ -1,4 +1,4 @@ -import { describe, it, expect, vi, beforeEach, afterEach } from 'vitest'; +import { describe, it, expect, afterEach, vi } from 'vitest'; import { createCommsQueryString, @@ -6,211 +6,104 @@ import { getCommsParamsFromCurrentLocation, } from './comms-query-string.ts'; -// Mock the logger module -vi.mock('@metamask/logger', () => ({ - Logger: vi.fn().mockImplementation(function () { - return { - error: vi.fn(), - warn: vi.fn(), - }; - }), -})); +const config = { + knownRelays: ['/ip4/127.0.0.1/tcp/9001/ws/p2p/relay'], +}; describe('comms-query-string', () => { describe('createCommsQueryString', () => { - it('returns URLSearchParams with relays only', () => { - const relays = ['/ip4/127.0.0.1/tcp/9001/ws']; - const result = createCommsQueryString({ relays }); - expect(result.get('relays')).toBe(JSON.stringify(relays)); + it('encodes the netlayer and its JSON config', () => { + const params = createCommsQueryString({ netlayer: 'libp2p', config }); + expect(params.get('netlayer')).toBe('libp2p'); + expect(params.get('netlayer-config')).toBe(JSON.stringify(config)); }); - it('returns URLSearchParams with allowedWsHosts only', () => { - const allowedWsHosts = ['localhost', 'relay.example.com']; - const result = createCommsQueryString({ allowedWsHosts }); - expect(result.get('allowedWsHosts')).toBe(JSON.stringify(allowedWsHosts)); - }); - - it('returns URLSearchParams with both params', () => { - const relays = ['/ip4/127.0.0.1/tcp/9001/ws']; - const allowedWsHosts = ['localhost']; - const result = createCommsQueryString({ relays, allowedWsHosts }); - expect(result.has('relays')).toBe(true); - expect(result.has('allowedWsHosts')).toBe(true); - }); - - it('returns empty URLSearchParams for empty arrays', () => { - expect( - createCommsQueryString({ relays: [], allowedWsHosts: [] }).toString(), - ).toBe(''); - expect(createCommsQueryString({}).toString()).toBe(''); - }); - - it('returns URLSearchParams with number options and directListenAddresses', () => { - const result = createCommsQueryString({ - relays: ['/ip4/127.0.0.1/tcp/9001/ws'], - maxRetryAttempts: 3, - maxQueue: 100, - directListenAddresses: ['/ip4/0.0.0.0/udp/0/quic-v1'], - }); - expect(result.get('relays')).toBe( - JSON.stringify(['/ip4/127.0.0.1/tcp/9001/ws']), - ); - expect(result.get('maxRetryAttempts')).toBe('3'); - expect(result.get('maxQueue')).toBe('100'); - expect(result.get('directListenAddresses')).toBe( - JSON.stringify(['/ip4/0.0.0.0/udp/0/quic-v1']), - ); - }); - - it('round-trips full options via createCommsQueryString and parseCommsQueryString', () => { - const options = { - relays: ['/dns4/relay.example.com/tcp/443/wss/p2p/QmRelay'], - allowedWsHosts: ['relay.example.com'], - maxRetryAttempts: 5, + it('encodes kernel-level numeric options', () => { + const params = createCommsQueryString({ + netlayer: 'libp2p', + config: {}, maxQueue: 200, - streamInactivityTimeoutMs: 60000, - }; - const params = createCommsQueryString(options); - expect(parseCommsQueryString(`?${params.toString()}`)).toStrictEqual( - options, - ); - }); - - it('throws on invalid array param types', () => { - expect(() => - createCommsQueryString({ - relays: 'not-an-array' as unknown as string[], - }), - ).toThrow(TypeError); - expect(() => - createCommsQueryString({ - relays: [1, 2] as unknown as string[], - }), - ).toThrow(TypeError); + ackTimeoutMs: 5000, + maxUrlRelayHints: 3, + maxKnownRelays: 20, + }); + expect(params.get('maxQueue')).toBe('200'); + expect(params.get('ackTimeoutMs')).toBe('5000'); + expect(params.get('maxUrlRelayHints')).toBe('3'); + expect(params.get('maxKnownRelays')).toBe('20'); }); - it('throws on invalid number param types', () => { - expect(() => createCommsQueryString({ maxRetryAttempts: -1 })).toThrow( - TypeError, - ); - expect(() => createCommsQueryString({ maxQueue: 1.5 })).toThrow( - TypeError, - ); + it('throws on an invalid numeric option', () => { expect(() => createCommsQueryString({ - maxRetryAttempts: 'five' as unknown as number, + netlayer: 'libp2p', + config: {}, + maxQueue: -1, }), - ).toThrow(TypeError); + ).toThrow('maxQueue must be a non-negative integer'); }); }); describe('parseCommsQueryString', () => { - it('returns both relays and allowedWsHosts', () => { - const queryString = `?relays=${encodeURIComponent(JSON.stringify(['/ip4/127.0.0.1/tcp/9001/ws']))}&allowedWsHosts=${encodeURIComponent(JSON.stringify(['localhost']))}`; - expect(parseCommsQueryString(queryString)).toStrictEqual({ - relays: ['/ip4/127.0.0.1/tcp/9001/ws'], - allowedWsHosts: ['localhost'], - }); - }); - - it('returns empty object when no comms params present', () => { - expect(parseCommsQueryString('?foo=bar')).toStrictEqual({}); + it('parses a specifier from netlayer + netlayer-config', () => { + const qs = createCommsQueryString({ netlayer: 'libp2p', config }); + const parsed = parseCommsQueryString(qs.toString()); + expect(parsed.specifier).toStrictEqual({ netlayer: 'libp2p', config }); }); - it('parses directListenAddresses and number options', () => { - const queryString = `?directListenAddresses=${encodeURIComponent(JSON.stringify(['/ip4/0.0.0.0/udp/0/quic-v1']))}&maxRetryAttempts=5&maxQueue=100`; - expect(parseCommsQueryString(queryString)).toStrictEqual({ - directListenAddresses: ['/ip4/0.0.0.0/udp/0/quic-v1'], - maxRetryAttempts: 5, - maxQueue: 100, + it('round-trips a full set of params', () => { + const qs = createCommsQueryString({ + netlayer: 'libp2p', + config, + maxQueue: 200, + ackTimeoutMs: 5000, + maxUrlRelayHints: 3, + maxKnownRelays: 20, + }); + const parsed = parseCommsQueryString(`?${qs.toString()}`); + expect(parsed).toStrictEqual({ + specifier: { netlayer: 'libp2p', config }, + maxQueue: 200, + ackTimeoutMs: 5000, + maxUrlRelayHints: 3, + maxKnownRelays: 20, }); }); - it('parses query string without leading ?', () => { - const queryString = `relays=${encodeURIComponent(JSON.stringify(['/ip4/127.0.0.1/tcp/9001/ws']))}`; - expect(parseCommsQueryString(queryString)).toStrictEqual({ - relays: ['/ip4/127.0.0.1/tcp/9001/ws'], - }); + it('omits the specifier when no netlayer param is present', () => { + expect(parseCommsQueryString('')).toStrictEqual({}); }); - it('throws on array params with non-string-array JSON values', () => { + it('throws on invalid netlayer-config JSON', () => { expect(() => - parseCommsQueryString( - `?relays=${encodeURIComponent(JSON.stringify({ not: 'an array' }))}`, - ), - ).toThrow(TypeError); + parseCommsQueryString('netlayer=libp2p&netlayer-config=not-json'), + ).toThrow('netlayer-config contains invalid JSON'); + }); + + it('throws on an invalid numeric value', () => { expect(() => parseCommsQueryString( - `?relays=${encodeURIComponent(JSON.stringify([1, 2]))}`, + 'netlayer=libp2p&netlayer-config=%7B%7D&maxQueue=-5', ), - ).toThrow(TypeError); - }); - - it('throws on array params with invalid JSON', () => { - expect(() => parseCommsQueryString('?relays=not-json')).toThrow( - TypeError, - ); - }); - - it('throws on invalid number values', () => { - expect(() => parseCommsQueryString('?maxRetryAttempts=-1')).toThrow( - TypeError, - ); - expect(() => parseCommsQueryString('?maxRetryAttempts=1.5')).toThrow( - TypeError, - ); - expect(parseCommsQueryString('?maxRetryAttempts=10')).toStrictEqual({ - maxRetryAttempts: 10, - }); + ).toThrow('maxQueue must be a non-negative integer'); }); }); describe('getCommsParamsFromCurrentLocation', () => { - const originalLocation = globalThis.location; - - beforeEach(() => { - Object.defineProperty(globalThis, 'location', { - value: { search: '' }, - writable: true, - configurable: true, - }); - }); - afterEach(() => { - if (originalLocation) { - Object.defineProperty(globalThis, 'location', { - value: originalLocation, - writable: true, - configurable: true, - }); - } else { - // @ts-expect-error - deleting global property - delete globalThis.location; - } - }); - - it('returns relays and allowedWsHosts from location', () => { - const relays = ['/ip4/127.0.0.1/tcp/9001/ws']; - const allowedWsHosts = ['localhost']; - globalThis.location.search = `?relays=${encodeURIComponent(JSON.stringify(relays))}&allowedWsHosts=${encodeURIComponent(JSON.stringify(allowedWsHosts))}`; - expect(getCommsParamsFromCurrentLocation()).toStrictEqual({ - relays, - allowedWsHosts, - }); + vi.unstubAllGlobals(); }); - it('returns empty object when location is undefined', () => { - // @ts-expect-error - testing undefined location - delete globalThis.location; + it('returns an empty object when there is no location', () => { + vi.stubGlobal('location', undefined); expect(getCommsParamsFromCurrentLocation()).toStrictEqual({}); }); - it('returns all parsed options including numbers and directListenAddresses', () => { - globalThis.location.search = `?relays=${encodeURIComponent(JSON.stringify(['/ip4/127.0.0.1/tcp/9001/ws']))}&maxQueue=50&stalePeerTimeoutMs=3600000`; + it('parses the specifier from the current location', () => { + const qs = createCommsQueryString({ netlayer: 'libp2p', config }); + vi.stubGlobal('location', { search: `?${qs.toString()}` }); expect(getCommsParamsFromCurrentLocation()).toStrictEqual({ - relays: ['/ip4/127.0.0.1/tcp/9001/ws'], - maxQueue: 50, - stalePeerTimeoutMs: 3600000, + specifier: { netlayer: 'libp2p', config }, }); }); }); diff --git a/packages/kernel-browser-runtime/src/utils/comms-query-string.ts b/packages/kernel-browser-runtime/src/utils/comms-query-string.ts index 8765ad0868..fe73035f7e 100644 --- a/packages/kernel-browser-runtime/src/utils/comms-query-string.ts +++ b/packages/kernel-browser-runtime/src/utils/comms-query-string.ts @@ -1,96 +1,54 @@ import { Logger } from '@metamask/logger'; import type { RemoteCommsOptions } from '@metamask/ocap-kernel'; -import { array, integer, is, min, string } from '@metamask/superstruct'; +import { integer, is, min } from '@metamask/superstruct'; +import { JsonStruct } from '@metamask/utils'; +import type { Json } from '@metamask/utils'; /** - * Utilities for handling remote comms options in worker URL query strings. - * Supports all serializable fields of {@link RemoteCommsOptions} except - * `directTransports` (internal / not URL-serializable) and `mnemonic` - * (secret — should not be exposed in URLs). + * Utilities for carrying a {@link NetlayerSpecifier} plus the kernel-level + * remote-comms options through the kernel worker URL query string. The + * netlayer's `config` is `Json` and travels as a single JSON-encoded param; + * the sensitive `mnemonic` is never placed in a URL. */ const logger = new Logger('comms-query-string'); /** - * Subset of {@link RemoteCommsOptions} that can be encoded in a query string. - * Excludes `directTransports` (internal, platform-injected) and `mnemonic` - * (secret — must not appear in URLs; pass via postMessage instead). + * Parameters accepted by {@link createCommsQueryString}: the netlayer name and + * its `Json` config, plus the kernel-level numeric options the worker forwards. */ -export type CommsQueryParams = Omit< - RemoteCommsOptions, - 'directTransports' | 'mnemonic' ->; - -/** Keys of RemoteCommsOptions whose value is string[] (URL-serialized as JSON array). */ -type ArrayParamKey = { - [K in keyof CommsQueryParams]: CommsQueryParams[K] extends - | string[] - | undefined - ? K - : never; -}[keyof CommsQueryParams]; - -/** Keys of RemoteCommsOptions whose value is number (URL-serialized as string). */ -type NumberParamKey = { - [K in keyof CommsQueryParams]: CommsQueryParams[K] extends number | undefined - ? K - : never; -}[keyof CommsQueryParams]; - -const ARRAY_PARAM_NAMES = [ - 'relays', - 'allowedWsHosts', - 'directListenAddresses', -] as const satisfies readonly ArrayParamKey[]; +export type CommsQueryParams = { + netlayer: string; + config: Json; + maxQueue?: number | undefined; + ackTimeoutMs?: number | undefined; + maxUrlRelayHints?: number | undefined; + maxKnownRelays?: number | undefined; +}; const NUMBER_PARAM_NAMES = [ - 'maxRetryAttempts', 'maxQueue', - 'maxConcurrentConnections', - 'maxMessageSizeBytes', - 'cleanupIntervalMs', - 'stalePeerTimeoutMs', - 'maxMessagesPerSecond', - 'maxConnectionAttemptsPerMinute', - 'reconnectionBaseDelayMs', - 'reconnectionMaxDelayMs', - 'handshakeTimeoutMs', - 'writeTimeoutMs', 'ackTimeoutMs', - 'streamInactivityTimeoutMs', -] as const satisfies readonly NumberParamKey[]; + 'maxUrlRelayHints', + 'maxKnownRelays', +] as const; const NonNegativeInteger = min(integer(), 0); -const StringArray = array(string()); /** - * Creates URLSearchParams from remote comms options. - * Use when building the kernel worker URL so the worker can read all params. - * Callers can append additional params (e.g. reset-storage) before assigning to URL.search. - * Validates types at runtime via superstruct; throws if invalid values are passed. + * Build URLSearchParams for the kernel worker URL from a netlayer specifier and + * kernel-level options. Callers can append additional params (e.g. + * reset-storage) before assigning to `URL.search`. * - * @param params - Subset of {@link RemoteCommsOptions} (excluding directTransports) - * @returns URLSearchParams with comms options; use .toString() for URL.search + * @param params - The netlayer name, its `Json` config, and kernel options. + * @returns URLSearchParams; use `.toString()` for `URL.search`. */ export function createCommsQueryString( params: CommsQueryParams, ): URLSearchParams { const searchParams = new URLSearchParams(); - - for (const key of ARRAY_PARAM_NAMES) { - const value = params[key]; - if (value === undefined) { - continue; - } - if (!is(value, StringArray)) { - throw new TypeError( - `createCommsQueryString: ${key} must be an array of strings, got ${typeof value}`, - ); - } - if (value.length > 0) { - searchParams.set(key, JSON.stringify(value)); - } - } + searchParams.set('netlayer', params.netlayer); + searchParams.set('netlayer-config', JSON.stringify(params.config)); for (const key of NUMBER_PARAM_NAMES) { const value = params[key]; @@ -109,37 +67,42 @@ export function createCommsQueryString( } /** - * Parses all supported {@link RemoteCommsOptions} from a query string. - * Only includes keys that are present and valid in the query string. + * Parse a netlayer specifier plus kernel-level options from a query string. + * Returns a {@link RemoteCommsOptions} suitable for + * {@link Kernel.initRemoteComms}. When no `netlayer` param is present, the + * specifier is omitted (the kernel then falls back to its default netlayer). * - * @param queryString - The query string (e.g., from window.location.search) - * @returns Partial options object suitable for {@link Kernel.initRemoteComms} + * @param queryString - The query string (e.g. from `window.location.search`). + * @returns Partial remote-comms options. */ -export function parseCommsQueryString(queryString: string): CommsQueryParams { - const options: CommsQueryParams = {}; +export function parseCommsQueryString(queryString: string): RemoteCommsOptions { + const options: RemoteCommsOptions = {}; const search = queryString.startsWith('?') ? queryString.slice(1) : queryString; const params = new URLSearchParams(search); - for (const key of ARRAY_PARAM_NAMES) { - const raw = params.get(key); - if (raw !== null && raw !== '') { + const netlayer = params.get('netlayer'); + if (netlayer !== null && netlayer !== '') { + const rawConfig = params.get('netlayer-config'); + let config: Json = {}; + if (rawConfig !== null && rawConfig !== '') { let parsed: unknown; try { - parsed = JSON.parse(raw); + parsed = JSON.parse(rawConfig); } catch { throw new TypeError( - `parseCommsQueryString: ${key} contains invalid JSON: ${raw}`, + `parseCommsQueryString: netlayer-config contains invalid JSON: ${rawConfig}`, ); } - if (!is(parsed, StringArray)) { + if (!is(parsed, JsonStruct)) { throw new TypeError( - `parseCommsQueryString: ${key} must be a JSON array of strings, got ${raw}`, + `parseCommsQueryString: netlayer-config must be JSON, got ${rawConfig}`, ); } - options[key] = parsed; + config = parsed; } + options.specifier = { netlayer, config }; } for (const key of NUMBER_PARAM_NAMES) { @@ -159,13 +122,12 @@ export function parseCommsQueryString(queryString: string): CommsQueryParams { } /** - * Gets all supported remote comms options from the current global location's - * query string. Intended for use within the kernel worker when calling - * {@link Kernel.initRemoteComms}. + * Get the remote comms options from the current global location's query string. + * Intended for the kernel worker when calling {@link Kernel.initRemoteComms}. * - * @returns Partial {@link RemoteCommsOptions} (excluding directTransports) + * @returns Partial {@link RemoteCommsOptions}. */ -export function getCommsParamsFromCurrentLocation(): CommsQueryParams { +export function getCommsParamsFromCurrentLocation(): RemoteCommsOptions { if (typeof globalThis.location === 'undefined') { logger.warn('No location object available in current context'); return {}; diff --git a/packages/kernel-cli/CHANGELOG.md b/packages/kernel-cli/CHANGELOG.md index 2e32a111dc..4ff9e8e8e3 100644 --- a/packages/kernel-cli/CHANGELOG.md +++ b/packages/kernel-cli/CHANGELOG.md @@ -15,6 +15,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed +- The `relay` command imports `startRelay` from `@metamask/netlayer-libp2p/relay` (moved from `@metamask/kernel-utils/libp2p`), and `daemon start` initializes remote comms with a `{ netlayer: 'libp2p', config: { knownRelays } }` specifier ([#973](https://github.com/MetaMask/ocap-kernel/pull/973)) - Relay state files (`relay.pid`, `relay.addr`) now live in their own directory (default `~/.libp2p-relay`, overridable via `$LIBP2P_RELAY_HOME`) instead of under `$OCAP_HOME`, so one libp2p relay can serve daemons with different OCAP_HOMEs ([#952](https://github.com/MetaMask/ocap-kernel/pull/952)) ### Fixed diff --git a/packages/kernel-cli/package.json b/packages/kernel-cli/package.json index b5031b6901..a60473caaf 100644 --- a/packages/kernel-cli/package.json +++ b/packages/kernel-cli/package.json @@ -51,6 +51,7 @@ "@metamask/kernel-shims": "workspace:^", "@metamask/kernel-utils": "workspace:^", "@metamask/logger": "workspace:^", + "@metamask/netlayer-libp2p": "workspace:^", "@metamask/utils": "^11.9.0", "@types/node": "^22.13.1", "acorn": "^8.15.0", diff --git a/packages/kernel-cli/src/commands/daemon.test.ts b/packages/kernel-cli/src/commands/daemon.test.ts index f8d015384b..63cc92615f 100644 --- a/packages/kernel-cli/src/commands/daemon.test.ts +++ b/packages/kernel-cli/src/commands/daemon.test.ts @@ -154,7 +154,14 @@ describe('handleDaemonStart', () => { expect(sendCommand).toHaveBeenCalledWith( expect.objectContaining({ method: 'initRemoteComms', - params: { relays: ['/ip4/127.0.0.1/tcp/9001/ws/p2p/QmFoo'] }, + params: { + specifier: { + netlayer: 'libp2p', + config: { + knownRelays: ['/ip4/127.0.0.1/tcp/9001/ws/p2p/QmFoo'], + }, + }, + }, }), ); expect(writeSpy).toHaveBeenCalledWith( diff --git a/packages/kernel-cli/src/commands/daemon.ts b/packages/kernel-cli/src/commands/daemon.ts index 0a83bea846..4d935aac5a 100644 --- a/packages/kernel-cli/src/commands/daemon.ts +++ b/packages/kernel-cli/src/commands/daemon.ts @@ -220,7 +220,9 @@ export async function handleDaemonStart( const initResponse = await sendCommand({ socketPath, method: 'initRemoteComms', - params: { relays: [relayAddr] }, + params: { + specifier: { netlayer: 'libp2p', config: { knownRelays: [relayAddr] } }, + }, timeoutMs: 30_000, }); if (isJsonRpcFailure(initResponse)) { diff --git a/packages/kernel-cli/src/commands/relay.test.ts b/packages/kernel-cli/src/commands/relay.test.ts index 7eddab9c3a..44f97b86a2 100644 --- a/packages/kernel-cli/src/commands/relay.test.ts +++ b/packages/kernel-cli/src/commands/relay.test.ts @@ -9,7 +9,7 @@ import { } from './relay.ts'; import { isProcessAlive, readPidFile, sendSignal, waitFor } from '../utils.ts'; -vi.mock('@metamask/kernel-utils/libp2p', () => ({ +vi.mock('@metamask/netlayer-libp2p/relay', () => ({ startRelay: vi.fn(), })); @@ -61,7 +61,7 @@ describe('relay', () => { it('writes PID and addr files and starts the relay', async () => { vi.mocked(readPidFile).mockResolvedValueOnce(undefined); - const { startRelay } = await import('@metamask/kernel-utils/libp2p'); + const { startRelay } = await import('@metamask/netlayer-libp2p/relay'); vi.mocked(startRelay).mockResolvedValueOnce(makeLibp2pMock() as never); await startRelayWithBookkeeping(mockLogger); @@ -80,7 +80,7 @@ describe('relay', () => { it('writes a synthesized multiaddr when publicIp is supplied, ignoring the picker', async () => { vi.mocked(readPidFile).mockResolvedValueOnce(undefined); - const { startRelay } = await import('@metamask/kernel-utils/libp2p'); + const { startRelay } = await import('@metamask/netlayer-libp2p/relay'); // Surface only an internal address from getMultiaddrs(); the test // proves the synthesized publicIp wins anyway. vi.mocked(startRelay).mockResolvedValueOnce( @@ -104,7 +104,7 @@ describe('relay', () => { it('prefers a non-loopback public multiaddr over loopback', async () => { vi.mocked(readPidFile).mockResolvedValueOnce(undefined); - const { startRelay } = await import('@metamask/kernel-utils/libp2p'); + const { startRelay } = await import('@metamask/netlayer-libp2p/relay'); vi.mocked(startRelay).mockResolvedValueOnce( makeLibp2pMock([ '/ip4/127.0.0.1/tcp/9001/ws/p2p/QmFoo', @@ -124,7 +124,7 @@ describe('relay', () => { it('falls back to a private (RFC 1918) address when no public is available', async () => { vi.mocked(readPidFile).mockResolvedValueOnce(undefined); - const { startRelay } = await import('@metamask/kernel-utils/libp2p'); + const { startRelay } = await import('@metamask/netlayer-libp2p/relay'); vi.mocked(startRelay).mockResolvedValueOnce( makeLibp2pMock([ '/ip4/127.0.0.1/tcp/9001/ws/p2p/QmFoo', @@ -143,7 +143,7 @@ describe('relay', () => { it('falls back to loopback when no other multiaddr is available', async () => { vi.mocked(readPidFile).mockResolvedValueOnce(undefined); - const { startRelay } = await import('@metamask/kernel-utils/libp2p'); + const { startRelay } = await import('@metamask/netlayer-libp2p/relay'); vi.mocked(startRelay).mockResolvedValueOnce( makeLibp2pMock(['/ip4/127.0.0.1/tcp/9001/ws/p2p/QmFoo']) as never, ); @@ -160,7 +160,7 @@ describe('relay', () => { vi.mocked(readPidFile).mockResolvedValueOnce(9999); vi.mocked(isProcessAlive).mockReturnValueOnce(false); - const { startRelay } = await import('@metamask/kernel-utils/libp2p'); + const { startRelay } = await import('@metamask/netlayer-libp2p/relay'); vi.mocked(startRelay).mockResolvedValueOnce(makeLibp2pMock() as never); await startRelayWithBookkeeping(mockLogger); @@ -174,7 +174,7 @@ describe('relay', () => { it('throws when no WS multiaddr is available, stops libp2p, and removes PID file', async () => { vi.mocked(readPidFile).mockResolvedValueOnce(undefined); - const { startRelay } = await import('@metamask/kernel-utils/libp2p'); + const { startRelay } = await import('@metamask/netlayer-libp2p/relay'); const libp2pMock = makeLibp2pMock(['/ip4/0.0.0.0/tcp/9002']); vi.mocked(startRelay).mockResolvedValueOnce(libp2pMock as never); @@ -190,7 +190,7 @@ describe('relay', () => { it('removes PID file if startRelay throws', async () => { vi.mocked(readPidFile).mockResolvedValueOnce(undefined); - const { startRelay } = await import('@metamask/kernel-utils/libp2p'); + const { startRelay } = await import('@metamask/netlayer-libp2p/relay'); vi.mocked(startRelay).mockRejectedValueOnce(new Error('port in use')); await expect(startRelayWithBookkeeping(mockLogger)).rejects.toThrow( diff --git a/packages/kernel-cli/src/commands/relay.ts b/packages/kernel-cli/src/commands/relay.ts index ae9223c479..c2ed6385aa 100644 --- a/packages/kernel-cli/src/commands/relay.ts +++ b/packages/kernel-cli/src/commands/relay.ts @@ -1,6 +1,6 @@ -import { startRelay } from '@metamask/kernel-utils/libp2p'; import { getLibp2pRelayHome } from '@metamask/kernel-utils/nodejs'; import type { Logger } from '@metamask/logger'; +import { startRelay } from '@metamask/netlayer-libp2p/relay'; import { mkdir, rm, writeFile } from 'node:fs/promises'; import { isProcessAlive, readPidFile, sendSignal, waitFor } from '../utils.ts'; diff --git a/packages/kernel-cli/tsconfig.build.json b/packages/kernel-cli/tsconfig.build.json index 0aac5d7a6c..ec573ae1ce 100644 --- a/packages/kernel-cli/tsconfig.build.json +++ b/packages/kernel-cli/tsconfig.build.json @@ -11,6 +11,7 @@ }, "references": [ { "path": "../logger/tsconfig.build.json" }, + { "path": "../netlayer-libp2p/tsconfig.build.json" }, { "path": "../kernel-utils/tsconfig.build.json" } ], "files": [], diff --git a/packages/kernel-cli/tsconfig.json b/packages/kernel-cli/tsconfig.json index 88682225b8..1658d2bb7a 100644 --- a/packages/kernel-cli/tsconfig.json +++ b/packages/kernel-cli/tsconfig.json @@ -8,6 +8,7 @@ }, "references": [ { "path": "../logger" }, + { "path": "../netlayer-libp2p" }, { "path": "../repo-tools" }, { "path": "../kernel-utils" } ], diff --git a/packages/kernel-errors/CHANGELOG.md b/packages/kernel-errors/CHANGELOG.md index 2a1bb9ae44..14f883f869 100644 --- a/packages/kernel-errors/CHANGELOG.md +++ b/packages/kernel-errors/CHANGELOG.md @@ -15,6 +15,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed +- **BREAKING:** Reduce `isRetryableNetworkError` and `getNetworkErrorCode` to the transport-neutral subset (neutral error classes + Node.js network codes) — the libp2p-specific `MuxerClosedError`/`Dial`/`Transport`/`NO_RESERVATION` name/message sniffing moves into `@metamask/netlayer-libp2p`'s error mapper, which maps such errors to neutral classes before they reach the transport-neutral engine ([#973](https://github.com/MetaMask/ocap-kernel/pull/973)) - Remove the `@libp2p/interface` dependency: `isRetryableNetworkError` now classifies the neutral `ChannelResetError` and matches `MuxerClosedError` by error name instead of importing the libp2p class ([#971](https://github.com/MetaMask/ocap-kernel/pull/971)) ## [0.6.0] diff --git a/packages/kernel-errors/src/constants.test.ts b/packages/kernel-errors/src/constants.test.ts new file mode 100644 index 0000000000..085c0884dc --- /dev/null +++ b/packages/kernel-errors/src/constants.test.ts @@ -0,0 +1,17 @@ +import { is } from '@metamask/superstruct'; +import { describe, it, expect } from 'vitest'; + +import { ErrorStruct } from './constants.ts'; + +describe('ErrorStruct', () => { + it('accepts Error instances', () => { + expect(is(new Error('boom'), ErrorStruct)).toBe(true); + expect(is(new TypeError('boom'), ErrorStruct)).toBe(true); + }); + + it('rejects non-Error values', () => { + expect(is({ message: 'boom' }, ErrorStruct)).toBe(false); + expect(is('boom', ErrorStruct)).toBe(false); + expect(is(null, ErrorStruct)).toBe(false); + }); +}); diff --git a/packages/kernel-errors/src/marshal/marshalError.test.ts b/packages/kernel-errors/src/marshal/marshalError.test.ts index 4def807ab4..eb7cda59a3 100644 --- a/packages/kernel-errors/src/marshal/marshalError.test.ts +++ b/packages/kernel-errors/src/marshal/marshalError.test.ts @@ -2,6 +2,7 @@ import { describe, it, expect } from 'vitest'; import { marshalError } from './marshalError.ts'; import { ErrorCode, ErrorSentinel } from '../constants.ts'; +import { ChannelResetError } from '../errors/ChannelResetError.ts'; import { VatNotFoundError } from '../errors/VatNotFoundError.ts'; describe('marshalError', () => { @@ -62,4 +63,21 @@ describe('marshalError', () => { }), ); }); + + it('omits the stack when the error has none', () => { + const error = new Error('foo'); + delete error.stack; + const marshaledError = marshalError(error); + expect('stack' in marshaledError).toBe(false); + expect(marshaledError.message).toBe('foo'); + }); + + it('marshals an ocap error without data', () => { + // ChannelResetError carries no `data`, exercising the ocap-error branch + // where `error.data` is absent. + const error = new ChannelResetError(); + const marshaledError = marshalError(error); + expect(marshaledError.code).toBe(ErrorCode.ChannelResetError); + expect('data' in marshaledError).toBe(false); + }); }); diff --git a/packages/kernel-errors/src/utils/getNetworkErrorCode.test.ts b/packages/kernel-errors/src/utils/getNetworkErrorCode.test.ts index 1fe562f084..ad5df50c46 100644 --- a/packages/kernel-errors/src/utils/getNetworkErrorCode.test.ts +++ b/packages/kernel-errors/src/utils/getNetworkErrorCode.test.ts @@ -40,30 +40,21 @@ describe('getNetworkErrorCode', () => { }); }); - describe('relay reservation errors', () => { - it('returns name for Error with NO_RESERVATION in message (name takes precedence)', () => { + describe('message content is not sniffed', () => { + it('returns the name for a named error regardless of message', () => { const error = new Error( 'failed to connect via relay with status NO_RESERVATION', ); - // name ('Error') takes precedence over message parsing + // name ('Error') is returned; message content is not parsed expect(getNetworkErrorCode(error)).toBe('Error'); }); - it('returns NO_RESERVATION when error has empty name', () => { + it('returns UNKNOWN when a message-only error has an empty name', () => { const error = Object.assign( new Error('failed to connect via relay with status NO_RESERVATION'), { name: '' }, ); - expect(getNetworkErrorCode(error)).toBe('NO_RESERVATION'); - }); - - it('returns name when both name and NO_RESERVATION message are present', () => { - const error = Object.assign( - new Error('failed to connect via relay with status NO_RESERVATION'), - { name: 'InvalidMessageError' }, - ); - // name takes precedence over message parsing - expect(getNetworkErrorCode(error)).toBe('InvalidMessageError'); + expect(getNetworkErrorCode(error)).toBe('UNKNOWN'); }); }); diff --git a/packages/kernel-errors/src/utils/getNetworkErrorCode.ts b/packages/kernel-errors/src/utils/getNetworkErrorCode.ts index db21f6e059..1f5d7b6615 100644 --- a/packages/kernel-errors/src/utils/getNetworkErrorCode.ts +++ b/packages/kernel-errors/src/utils/getNetworkErrorCode.ts @@ -1,8 +1,11 @@ /** * Extract a network error code from an error object. * - * Returns error codes like 'ECONNREFUSED', 'ETIMEDOUT', etc., or - * the error name for libp2p errors, or 'UNKNOWN' for unrecognized errors. + * Returns error codes like 'ECONNREFUSED', 'ETIMEDOUT', etc., or the error name + * for any named error, or 'UNKNOWN' for unrecognized errors. This is used by + * the transport-neutral engine for permanent-failure pattern detection, which + * keys on Node.js network codes; transport-specific error sniffing lives in the + * netlayer error mappers. * * @param error - The error to extract the code from. * @returns The error code string. @@ -11,7 +14,6 @@ export function getNetworkErrorCode(error: unknown): string { const anyError = error as { code?: string; name?: string; - message?: string; }; // Node.js network error codes (ECONNREFUSED, ETIMEDOUT, etc.) @@ -19,18 +21,10 @@ export function getNetworkErrorCode(error: unknown): string { return anyError.code; } - // libp2p errors and other named errors + // Any named error. if (typeof anyError?.name === 'string' && anyError.name.length > 0) { return anyError.name; } - // Check message for relay reservation errors - if ( - typeof anyError?.message === 'string' && - anyError.message.includes('NO_RESERVATION') - ) { - return 'NO_RESERVATION'; - } - return 'UNKNOWN'; } diff --git a/packages/kernel-errors/src/utils/isRetryableNetworkError.test.ts b/packages/kernel-errors/src/utils/isRetryableNetworkError.test.ts index 0ed95ac11a..fcbb6d89df 100644 --- a/packages/kernel-errors/src/utils/isRetryableNetworkError.test.ts +++ b/packages/kernel-errors/src/utils/isRetryableNetworkError.test.ts @@ -10,30 +10,6 @@ describe('isRetryableNetworkError', () => { }); }); - describe('libp2p errors', () => { - it('returns true for an error named MuxerClosedError', () => { - const error = Object.assign(new Error('Muxer closed'), { - name: 'MuxerClosedError', - }); - expect(isRetryableNetworkError(error)).toBe(true); - }); - - it.each([ - { name: 'DialError' }, - { name: 'TransportError' }, - { name: 'WebRTCDialError' }, - { name: 'SomeTransportFailure' }, - { name: 'CustomDialTimeout' }, - ])( - 'returns true for errors with name containing Dial or Transport: $name', - ({ name }) => { - const error = new Error('Connection failed'); - error.name = name; - expect(isRetryableNetworkError(error)).toBe(true); - }, - ); - }); - describe('Node.js network error codes', () => { it.each([ { code: 'ECONNRESET', description: 'connection reset by peer' }, @@ -50,42 +26,36 @@ describe('isRetryableNetworkError', () => { }); }); - describe('relay reservation errors', () => { + describe('transport-specific errors are not classified here', () => { + // These are mapped to neutral classes by the netlayer error mapper before + // reaching the neutral engine, so the neutral classifier treats them as + // non-retryable on their own. it.each([ { - message: 'failed to connect via relay with status NO_RESERVATION', - description: 'full error message', + name: 'MuxerClosedError', + error: Object.assign(new Error('Muxer closed'), { + name: 'MuxerClosedError', + }), }, { - message: 'NO_RESERVATION', - description: 'just NO_RESERVATION', + name: 'DialError', + error: Object.assign(new Error('Connection failed'), { + name: 'DialError', + }), }, { - message: 'Error: NO_RESERVATION occurred', - description: 'NO_RESERVATION in error message', + name: 'TransportError', + error: Object.assign(new Error('Connection failed'), { + name: 'TransportError', + }), }, { - message: 'Relay error: NO_RESERVATION - peer not ready', - description: 'NO_RESERVATION with context', - }, - ])( - 'returns true for errors with message containing NO_RESERVATION: $description', - ({ message }) => { - const error = new Error(message); - expect(isRetryableNetworkError(error)).toBe(true); + name: 'NO_RESERVATION message', + error: new Error( + 'failed to connect via relay with status NO_RESERVATION', + ), }, - ); - - it('returns true for InvalidMessageError with NO_RESERVATION', () => { - const error = Object.assign( - new Error('failed to connect via relay with status NO_RESERVATION'), - { name: 'InvalidMessageError' }, - ); - expect(isRetryableNetworkError(error)).toBe(true); - }); - - it('returns false for errors without NO_RESERVATION in message', () => { - const error = new Error('failed to connect via relay'); + ])('returns false for $name', ({ error }) => { expect(isRetryableNetworkError(error)).toBe(false); }); }); @@ -132,26 +102,4 @@ describe('isRetryableNetworkError', () => { ])('returns false for $name', ({ value }) => { expect(isRetryableNetworkError(value)).toBe(false); }); - - it.each([ - { - name: 'name with Dial in the middle', - errorName: 'WebRTCDialTimeout', - expected: true, - }, - { - name: 'name with Transport at the end', - errorName: 'WebSocketTransport', - expected: true, - }, - { - name: 'lowercase dial/transport in name', - errorName: 'dialtimeout', - expected: false, // Case-sensitive check won't match, default is false - }, - ])('returns $expected for $name', ({ errorName, expected }) => { - const error = new Error('error'); - error.name = errorName; - expect(isRetryableNetworkError(error)).toBe(expected); - }); }); diff --git a/packages/kernel-errors/src/utils/isRetryableNetworkError.ts b/packages/kernel-errors/src/utils/isRetryableNetworkError.ts index 4a49726c42..a6309fbc0a 100644 --- a/packages/kernel-errors/src/utils/isRetryableNetworkError.ts +++ b/packages/kernel-errors/src/utils/isRetryableNetworkError.ts @@ -1,40 +1,29 @@ import { ChannelResetError } from '../errors/ChannelResetError.ts'; /** - * Decide if an error is retryable for reconnects + * Decide if an error is retryable for reconnects. + * + * This is the transport-neutral classifier: it recognises the neutral error + * classes a netlayer maps its transport failures to, plus the standard Node.js + * network error codes. Transport-specific error sniffing (e.g. libp2p's + * `MuxerClosedError`/`Dial`/`Transport`/`NO_RESERVATION`) lives in the + * respective netlayer's error mapper, which maps such errors to neutral classes + * before they reach the transport-neutral engine. * * @param error - The error to check if it is retryable. * @returns True if the error is retryable, false otherwise. */ export function isRetryableNetworkError(error: unknown): boolean { - // Network errors from Node.js (ECONNRESET, ETIMEDOUT, etc.) - // libp2p errors (Dial*, Transport*, etc.) - // WebRTC/SCTP errors - const anyError = error as { - code?: string; - name?: string; - message?: string; - }; - - // A read-path reset the netlayer already mapped to a neutral class. + // A reset the netlayer already mapped to a neutral class. if (error instanceof ChannelResetError) { return true; } - // The MuxerClosedError / Dial / Transport / NO_RESERVATION branches below are - // libp2p-specific and matched by name/message (no libp2p import). They catch - // raw libp2p dial-path errors that surface before the netlayer maps them, and - // move to @metamask/netlayer-libp2p's error mapper in Phase 4 of the netlayer - // work. - if (error instanceof Error && error.name === 'MuxerClosedError') { - return true; - } - - // Node.js network error codes + // Node.js network error codes. // Note: ENOTFOUND (DNS lookup failed) is included to allow permanent failure // detection to work - after multiple consecutive ENOTFOUND errors, the peer // will be marked as permanently failed rather than giving up immediately. - const code = anyError?.code; + const code = (error as { code?: string })?.code; if ( code === 'ECONNRESET' || code === 'ETIMEDOUT' || @@ -47,20 +36,5 @@ export function isRetryableNetworkError(error: unknown): boolean { return true; } - // libp2p dial/transport errors - const name = anyError?.name; - if ( - typeof name === 'string' && - (name.includes('Dial') || name.includes('Transport')) - ) { - return true; - } - - // Relay reservation errors - these are temporary and should be retryable - const message = anyError?.message; - if (typeof message === 'string' && message.includes('NO_RESERVATION')) { - return true; - } - return false; // default to non-retryable for unknown errors } diff --git a/packages/kernel-node-runtime/CHANGELOG.md b/packages/kernel-node-runtime/CHANGELOG.md index 67ae0c9adc..da6884a14c 100644 --- a/packages/kernel-node-runtime/CHANGELOG.md +++ b/packages/kernel-node-runtime/CHANGELOG.md @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed +- **BREAKING:** `NodejsPlatformServices` now requires a `netlayers: NetlayerRegistry` construction argument, and `initializeRemoteComms` takes the neutral options bag. `makeKernel` builds the default registry (`{ libp2p: nodejsLibp2pNetlayerFactory }`) and accepts an optional `netlayers` override. QUIC/TCP direct-transport sniffing and the direct libp2p transport deps move to `@metamask/netlayer-libp2p/nodejs` ([#973](https://github.com/MetaMask/ocap-kernel/pull/973)) - **BREAKING:** Drop `platformOptions.fetch` from `makeNodeJsVatSupervisor` ([#942](https://github.com/MetaMask/ocap-kernel/pull/942)) - `fetch` is now a vat endowment; stub `globalThis.fetch` directly if needed diff --git a/packages/kernel-node-runtime/package.json b/packages/kernel-node-runtime/package.json index 6540dff953..bbfb1f463e 100644 --- a/packages/kernel-node-runtime/package.json +++ b/packages/kernel-node-runtime/package.json @@ -69,11 +69,8 @@ "test:dev:quiet": "yarn test:dev --reporter @ocap/repo-tools/vitest-reporters/silent" }, "dependencies": { - "@chainsafe/libp2p-quic": "^2.0.1", "@endo/eventual-send": "^1.3.4", "@endo/promise-kit": "^1.1.13", - "@libp2p/interface": "3.1.1", - "@libp2p/tcp": "11.0.14", "@libp2p/webrtc": "6.0.15", "@metamask/kernel-platforms": "workspace:^", "@metamask/kernel-rpc-methods": "workspace:^", @@ -81,6 +78,7 @@ "@metamask/kernel-store": "workspace:^", "@metamask/kernel-utils": "workspace:^", "@metamask/logger": "workspace:^", + "@metamask/netlayer-libp2p": "workspace:^", "@metamask/ocap-kernel": "workspace:^", "@metamask/streams": "workspace:^", "ses": "^1.14.0" @@ -88,6 +86,7 @@ "devDependencies": { "@arethetypeswrong/cli": "^0.17.4", "@endo/captp": "^4.4.8", + "@libp2p/interface": "3.1.1", "@metamask/auto-changelog": "^5.3.0", "@metamask/eslint-config": "^15.0.0", "@metamask/eslint-config-nodejs": "^15.0.0", diff --git a/packages/kernel-node-runtime/src/kernel/PlatformServices.test.ts b/packages/kernel-node-runtime/src/kernel/PlatformServices.test.ts index 17bcd3f8e2..ff1b07106a 100644 --- a/packages/kernel-node-runtime/src/kernel/PlatformServices.test.ts +++ b/packages/kernel-node-runtime/src/kernel/PlatformServices.test.ts @@ -73,31 +73,26 @@ vi.mock('node:worker_threads', () => ({ }), })); -vi.mock('@chainsafe/libp2p-quic', () => ({ - quic: () => ({ tag: 'mock-quic-transport' }), +const mockGetListenAddresses = vi.fn(() => [ + '/ip4/127.0.0.1/udp/12345/quic-v1/p2p/mock-peer-id', +]); + +// A fake netlayer factory (registry entry) capturing the params it receives. +const fakeNetlayerFactory = vi.fn(async () => ({ + sendRemoteMessage: mockSendRemoteMessage, + stop: mockStop, + closeConnection: mockCloseConnection, + registerLocationHints: mockRegisterLocationHints, + reconnectPeer: mockReconnectPeer, + resetAllBackoffs: vi.fn(), + getListenAddresses: mockGetListenAddresses, })); -vi.mock('@libp2p/tcp', () => ({ - tcp: () => ({ tag: 'mock-tcp-transport' }), -})); - -vi.mock('@metamask/ocap-kernel', async (importOriginal) => { - const actual = await importOriginal(); - return { - ...actual, - initTransport: vi.fn(async () => ({ - sendRemoteMessage: mockSendRemoteMessage, - stop: mockStop, - closeConnection: mockCloseConnection, - registerLocationHints: mockRegisterLocationHints, - reconnectPeer: mockReconnectPeer, - resetAllBackoffs: vi.fn(), - getListenAddresses: vi.fn(() => [ - '/ip4/127.0.0.1/udp/12345/quic-v1/p2p/mock-peer-id', - ]), - })), - }; -}); +const netlayers = { + libp2p: fakeNetlayerFactory, +} as unknown as ConstructorParameters< + typeof NodejsPlatformServices +>[0]['netlayers']; describe('NodejsPlatformServices', () => { beforeEach(() => { @@ -111,7 +106,7 @@ describe('NodejsPlatformServices', () => { }); it('constructs an instance without any arguments', () => { - const instance = new NodejsPlatformServices({}); + const instance = new NodejsPlatformServices({ netlayers }); expect(instance).toBeInstanceOf(NodejsPlatformServices); }); @@ -123,6 +118,7 @@ describe('NodejsPlatformServices', () => { it('creates a NodeWorker and returns a NodeWorkerDuplexStream', async () => { const service = new NodejsPlatformServices({ workerFilePath, + netlayers, }); const testVatId: VatId = getTestVatId(); const stream = await service.launch(testVatId); @@ -133,13 +129,13 @@ describe('NodejsPlatformServices', () => { it('rejects if synchronize fails', async () => { const rejected = 'test-reject-value'; mocks.stream.synchronize.mockRejectedValue(rejected); - const service = new NodejsPlatformServices({ workerFilePath }); + const service = new NodejsPlatformServices({ workerFilePath, netlayers }); const testVatId: VatId = getTestVatId(); await expect(service.launch(testVatId)).rejects.toThrowError(rejected); }); it('throws error if worker already exists', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); + const service = new NodejsPlatformServices({ workerFilePath, netlayers }); const testVatId: VatId = getTestVatId(); await service.launch(testVatId); @@ -151,7 +147,7 @@ describe('NodejsPlatformServices', () => { }); it('rejects if worker errors during startup', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); + const service = new NodejsPlatformServices({ workerFilePath, netlayers }); const testVatId: VatId = getTestVatId(); // Create a worker that won't auto-emit 'online' (for error testing) @@ -176,7 +172,7 @@ describe('NodejsPlatformServices', () => { }); it('rejects if worker exits during startup', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); + const service = new NodejsPlatformServices({ workerFilePath, netlayers }); const testVatId: VatId = getTestVatId(); // Create a worker that won't auto-emit 'online' (for exit testing) @@ -204,6 +200,7 @@ describe('NodejsPlatformServices', () => { it('terminates the target vat', async () => { const service = new NodejsPlatformServices({ workerFilePath, + netlayers, }); const testVatId: VatId = getTestVatId(); @@ -217,6 +214,7 @@ describe('NodejsPlatformServices', () => { it('throws when terminating an unknown vat', async () => { const service = new NodejsPlatformServices({ workerFilePath, + netlayers, }); const testVatId: VatId = getTestVatId(); @@ -230,6 +228,7 @@ describe('NodejsPlatformServices', () => { it('terminates all vats', async () => { const service = new NodejsPlatformServices({ workerFilePath, + netlayers, }); const vatIds: VatId[] = [getTestVatId(), getTestVatId(), getTestVatId()]; @@ -246,436 +245,198 @@ describe('NodejsPlatformServices', () => { }); describe('remote communications', () => { - describe('initializeRemoteComms', () => { - it('initializes remote comms with keySeed and relays', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - const keySeed = '0x1234567890abcdef'; - const relays = ['/dns4/relay.example/tcp/443/wss/p2p/relayPeer']; - const remoteHandler = vi.fn(async () => 'response'); - - await service.initializeRemoteComms(keySeed, { relays }, remoteHandler); - - const { initTransport } = await import('@metamask/ocap-kernel'); - expect(initTransport).toHaveBeenCalledWith( - keySeed, - { relays }, - expect.any(Function), - undefined, - undefined, - undefined, - ); - }); - - it('initializes remote comms with all options', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - const keySeed = '0x1234567890abcdef'; - const options = { - relays: ['/dns4/relay.example/tcp/443/wss/p2p/relayPeer'], - maxRetryAttempts: 5, - maxQueue: 100, - }; - const remoteHandler = vi.fn(async () => 'response'); - - await service.initializeRemoteComms(keySeed, options, remoteHandler); - - const { initTransport } = await import('@metamask/ocap-kernel'); - expect(initTransport).toHaveBeenCalledWith( - keySeed, - options, - expect.any(Function), - undefined, - undefined, - undefined, - ); - }); - - it('initializes remote comms with onRemoteGiveUp callback', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - const keySeed = '0x1234567890abcdef'; - const relays = ['/dns4/relay.example/tcp/443/wss/p2p/relayPeer']; - const remoteHandler = vi.fn(async () => 'response'); - const giveUpHandler = vi.fn(); - - await service.initializeRemoteComms( - keySeed, - { relays }, - remoteHandler, - giveUpHandler, - ); - - const { initTransport } = await import('@metamask/ocap-kernel'); - expect(initTransport).toHaveBeenCalledWith( - keySeed, - { relays }, - expect.any(Function), - giveUpHandler, - undefined, - undefined, - ); - }); - - it('initializes remote comms with incarnationId', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - const keySeed = '0x1234567890abcdef'; - const relays = ['/dns4/relay.example/tcp/443/wss/p2p/relayPeer']; - const remoteHandler = vi.fn(async () => 'response'); - const giveUpHandler = vi.fn(); - const incarnationId = 'test-incarnation-id'; - - await service.initializeRemoteComms( - keySeed, - { relays }, - remoteHandler, - giveUpHandler, - incarnationId, - ); - - const { initTransport } = await import('@metamask/ocap-kernel'); - expect(initTransport).toHaveBeenCalledWith( - keySeed, - { relays }, - expect.any(Function), - giveUpHandler, - incarnationId, - undefined, - ); - }); - - it('initializes remote comms with onIncarnationChange callback', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - const keySeed = '0x1234567890abcdef'; - const relays = ['/dns4/relay.example/tcp/443/wss/p2p/relayPeer']; - const remoteHandler = vi.fn(async () => 'response'); - const giveUpHandler = vi.fn(); - const incarnationId = 'test-incarnation-id'; - const incarnationChangeHandler = vi.fn(); - - await service.initializeRemoteComms( - keySeed, - { relays }, - remoteHandler, - giveUpHandler, - incarnationId, - incarnationChangeHandler, - ); - - const { initTransport } = await import('@metamask/ocap-kernel'); - expect(initTransport).toHaveBeenCalledWith( - keySeed, - { relays }, - expect.any(Function), - giveUpHandler, - incarnationId, - incarnationChangeHandler, - ); - }); - - it('throws error if already initialized', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - const keySeed = '0xabcd'; - const relays = ['/dns4/relay.example/tcp/443/wss/p2p/relayPeer']; - const remoteHandler = vi.fn(async () => 'response'); - - await service.initializeRemoteComms(keySeed, { relays }, remoteHandler); - - await expect( - service.initializeRemoteComms(keySeed, { relays }, remoteHandler), - ).rejects.toThrowError('remote comms already initialized'); - }); - - it('stores remote message handler for later use', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - const remoteHandler = vi.fn(async () => 'response'); - - await service.initializeRemoteComms( - '0xtest', - { relays: ['/dns4/relay.example/tcp/443/wss/p2p/relayPeer'] }, - remoteHandler, - ); - - // Handler is stored internally and will be used when messages arrive - // This is tested through integration tests - expect(service).toBeInstanceOf(NodejsPlatformServices); - }); - - it('injects QUIC directTransports when directListenAddresses contain /quic-v1', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - const keySeed = '0x1234567890abcdef'; - const relays = ['/dns4/relay.example/tcp/443/wss/p2p/relayPeer']; - const remoteHandler = vi.fn(async () => 'response'); + const specifier = { + netlayer: 'libp2p', + config: { + knownRelays: ['/dns4/relay.example/tcp/443/wss/p2p/relayPeer'], + }, + }; + const makeInit = ( + overrides: Record = {}, + ): Parameters[0] => ({ + keySeed: '0x1234567890abcdef', + specifier, + hooks: { handleMessage: vi.fn(async () => 'response') }, + ...overrides, + }); - await service.initializeRemoteComms( - keySeed, - { - relays, - directListenAddresses: ['/ip4/0.0.0.0/udp/0/quic-v1'], - }, - remoteHandler, - ); + describe('initializeRemoteComms', () => { + it('looks up the netlayer factory with the keySeed, config, and hooks', async () => { + const service = new NodejsPlatformServices({ + workerFilePath, + netlayers, + }); + await service.initializeRemoteComms(makeInit()); - const { initTransport } = await import('@metamask/ocap-kernel'); - expect(initTransport).toHaveBeenCalledWith( - keySeed, + expect(fakeNetlayerFactory).toHaveBeenCalledWith( expect.objectContaining({ - relays, - directTransports: [ - { - transport: { tag: 'mock-quic-transport' }, - listenAddresses: ['/ip4/0.0.0.0/udp/0/quic-v1'], - }, - ], + keySeed: '0x1234567890abcdef', + config: specifier.config, + hooks: expect.objectContaining({ + handleMessage: expect.any(Function), + }), }), - expect.any(Function), - undefined, - undefined, - undefined, ); }); - it('injects TCP directTransports when directListenAddresses contain /tcp/', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - const keySeed = '0x1234567890abcdef'; - const remoteHandler = vi.fn(async () => 'response'); - + it('forwards incarnationId and callbacks through the hooks', async () => { + const service = new NodejsPlatformServices({ + workerFilePath, + netlayers, + }); + const onRemoteGiveUp = vi.fn(); + const onIncarnationChange = vi.fn(async () => true); await service.initializeRemoteComms( - keySeed, - { - directListenAddresses: ['/ip4/0.0.0.0/tcp/4001'], - }, - remoteHandler, - ); - - const { initTransport } = await import('@metamask/ocap-kernel'); - expect(initTransport).toHaveBeenCalledWith( - keySeed, - expect.objectContaining({ - directTransports: [ - { - transport: { tag: 'mock-tcp-transport' }, - listenAddresses: ['/ip4/0.0.0.0/tcp/4001'], - }, - ], + makeInit({ + hooks: { + handleMessage: vi.fn(async () => 'response'), + onRemoteGiveUp, + onIncarnationChange, + }, + incarnationId: 'test-incarnation-id', }), - expect.any(Function), - undefined, - undefined, - undefined, ); - }); - it('injects both QUIC and TCP when directListenAddresses contain both', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - const keySeed = '0x1234567890abcdef'; - const remoteHandler = vi.fn(async () => 'response'); - - await service.initializeRemoteComms( - keySeed, - { - directListenAddresses: [ - '/ip4/0.0.0.0/udp/0/quic-v1', - '/ip4/0.0.0.0/tcp/4001', - ], - }, - remoteHandler, - ); - - const { initTransport } = await import('@metamask/ocap-kernel'); - expect(initTransport).toHaveBeenCalledWith( - keySeed, + expect(fakeNetlayerFactory).toHaveBeenCalledWith( expect.objectContaining({ - directTransports: [ - { - transport: { tag: 'mock-quic-transport' }, - listenAddresses: ['/ip4/0.0.0.0/udp/0/quic-v1'], - }, - { - transport: { tag: 'mock-tcp-transport' }, - listenAddresses: ['/ip4/0.0.0.0/tcp/4001'], - }, - ], + incarnationId: 'test-incarnation-id', + hooks: expect.objectContaining({ + onRemoteGiveUp, + onIncarnationChange, + }), }), - expect.any(Function), - undefined, - undefined, - undefined, ); }); - it('throws for unsupported direct listen addresses', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - const remoteHandler = vi.fn(async () => 'response'); - + it('throws for an unknown netlayer', async () => { + const service = new NodejsPlatformServices({ + workerFilePath, + netlayers, + }); await expect( service.initializeRemoteComms( - '0xtest', - { - directListenAddresses: ['/ip4/0.0.0.0/udp/0/webrtc'], - }, - remoteHandler, + makeInit({ specifier: { netlayer: 'nope', config: {} } }), ), - ).rejects.toThrowError('Unsupported direct listen address'); + ).rejects.toThrow('Unknown netlayer: "nope"'); }); - it('does not inject directTransports when no directListenAddresses provided', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - const keySeed = '0x1234567890abcdef'; - const relays = ['/dns4/relay.example/tcp/443/wss/p2p/relayPeer']; - const remoteHandler = vi.fn(async () => 'response'); - - await service.initializeRemoteComms(keySeed, { relays }, remoteHandler); - - const { initTransport } = await import('@metamask/ocap-kernel'); - const callArgs = ( - initTransport as unknown as ReturnType - ).mock.calls.at(-1); - // Second argument is the options - expect(callArgs?.[1]).not.toHaveProperty('directTransports'); + it('throws error if already initialized', async () => { + const service = new NodejsPlatformServices({ + workerFilePath, + netlayers, + }); + await service.initializeRemoteComms(makeInit()); + await expect( + service.initializeRemoteComms(makeInit()), + ).rejects.toThrowError('remote comms already initialized'); }); }); describe('getListenAddresses', () => { it('returns listen addresses after initialization', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - const remoteHandler = vi.fn(async () => ''); - - await service.initializeRemoteComms( - '0xabcd', - { relays: [] }, - remoteHandler, - ); - - const addresses = service.getListenAddresses(); - expect(addresses).toStrictEqual([ + const service = new NodejsPlatformServices({ + workerFilePath, + netlayers, + }); + await service.initializeRemoteComms(makeInit()); + expect(service.getListenAddresses()).toStrictEqual([ '/ip4/127.0.0.1/udp/12345/quic-v1/p2p/mock-peer-id', ]); }); it('returns empty array before initialization', () => { - const service = new NodejsPlatformServices({ workerFilePath }); + const service = new NodejsPlatformServices({ + workerFilePath, + netlayers, + }); expect(service.getListenAddresses()).toStrictEqual([]); }); }); describe('sendRemoteMessage', () => { it('sends message via network layer', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - const keySeed = '0xabcd'; - const relays = ['/dns4/relay.example/tcp/443/wss/p2p/relayPeer']; - const remoteHandler = vi.fn(async () => ''); - - await service.initializeRemoteComms(keySeed, { relays }, remoteHandler); - + const service = new NodejsPlatformServices({ + workerFilePath, + netlayers, + }); + await service.initializeRemoteComms(makeInit()); const message = JSON.stringify({ method: 'deliver', params: ['hello'], }); await service.sendRemoteMessage('peer-456', message); - expect(mockSendRemoteMessage).toHaveBeenCalledWith('peer-456', message); }); it('throws error if remote comms not initialized', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - const message = JSON.stringify({ method: 'deliver', params: ['test'] }); - + const service = new NodejsPlatformServices({ + workerFilePath, + netlayers, + }); await expect( - service.sendRemoteMessage('peer-999', message), + service.sendRemoteMessage('peer-999', 'msg'), ).rejects.toThrowError('remote comms not initialized'); }); }); describe('stopRemoteComms', () => { it('stops remote comms and cleans up', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - await service.initializeRemoteComms( - '0xtest', - {}, - vi.fn(async () => ''), - ); - + const service = new NodejsPlatformServices({ + workerFilePath, + netlayers, + }); + await service.initializeRemoteComms(makeInit()); await service.stopRemoteComms(); - expect(mockStop).toHaveBeenCalledOnce(); }); it('does nothing if remote comms not initialized', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); + const service = new NodejsPlatformServices({ + workerFilePath, + netlayers, + }); await service.stopRemoteComms(); expect(mockStop).not.toHaveBeenCalled(); }); it('allows re-initialization after stop', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - const keySeed = '0xabcd'; - const relays = ['/dns4/relay.example/tcp/443/wss/p2p/relayPeer']; - const remoteHandler = vi.fn(async () => ''); - - // Initialize - await service.initializeRemoteComms(keySeed, { relays }, remoteHandler); - - const { initTransport } = await import('@metamask/ocap-kernel'); - const initTransportMock = initTransport as unknown as ReturnType< - typeof vi.fn - >; - const firstCallCount = initTransportMock.mock.calls.length; - - // Stop + const service = new NodejsPlatformServices({ + workerFilePath, + netlayers, + }); + await service.initializeRemoteComms(makeInit()); + const firstCallCount = fakeNetlayerFactory.mock.calls.length; await service.stopRemoteComms(); expect(mockStop).toHaveBeenCalledOnce(); - - // Re-initialize should work - await service.initializeRemoteComms(keySeed, { relays }, remoteHandler); - - // Should have called initTransport again - expect(initTransportMock.mock.calls).toHaveLength(firstCallCount + 1); + await service.initializeRemoteComms(makeInit()); + expect(fakeNetlayerFactory.mock.calls).toHaveLength(firstCallCount + 1); }); it('clears internal state after stop', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - await service.initializeRemoteComms( - '0xtest', - {}, - vi.fn(async () => ''), - ); - - const message1 = JSON.stringify({ - method: 'deliver', - params: ['msg1'], + const service = new NodejsPlatformServices({ + workerFilePath, + netlayers, }); - const message2 = JSON.stringify({ - method: 'deliver', - params: ['msg2'], - }); - - // Should work before stop - await service.sendRemoteMessage('peer-1', message1); + await service.initializeRemoteComms(makeInit()); + await service.sendRemoteMessage('peer-1', 'msg1'); expect(mockSendRemoteMessage).toHaveBeenCalledTimes(1); - await service.stopRemoteComms(); - - // Should throw after stop await expect( - service.sendRemoteMessage('peer-2', message2), + service.sendRemoteMessage('peer-2', 'msg2'), ).rejects.toThrowError('remote comms not initialized'); }); it('clears closeConnection and reconnectPeer after stop', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - await service.initializeRemoteComms( - '0xtest', - {}, - vi.fn(async () => ''), - ); - - // Should work before stop + const service = new NodejsPlatformServices({ + workerFilePath, + netlayers, + }); + await service.initializeRemoteComms(makeInit()); await service.closeConnection('peer-1'); await service.reconnectPeer('peer-1'); expect(mockCloseConnection).toHaveBeenCalledTimes(1); expect(mockReconnectPeer).toHaveBeenCalledTimes(1); - await service.stopRemoteComms(); - - // Should throw after stop await expect(service.closeConnection('peer-2')).rejects.toThrowError( 'remote comms not initialized', ); @@ -687,21 +448,20 @@ describe('NodejsPlatformServices', () => { describe('closeConnection', () => { it('closes connection via network layer', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - await service.initializeRemoteComms( - '0xtest', - {}, - vi.fn(async () => ''), - ); - + const service = new NodejsPlatformServices({ + workerFilePath, + netlayers, + }); + await service.initializeRemoteComms(makeInit()); await service.closeConnection('peer-123'); - expect(mockCloseConnection).toHaveBeenCalledWith('peer-123'); }); it('throws error if remote comms not initialized', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - + const service = new NodejsPlatformServices({ + workerFilePath, + netlayers, + }); await expect(service.closeConnection('peer-999')).rejects.toThrowError( 'remote comms not initialized', ); @@ -710,15 +470,12 @@ describe('NodejsPlatformServices', () => { describe('registerLocationHints', () => { it('registers location hints via network layer', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - await service.initializeRemoteComms( - '0xtest', - {}, - vi.fn(async () => ''), - ); - + const service = new NodejsPlatformServices({ + workerFilePath, + netlayers, + }); + await service.initializeRemoteComms(makeInit()); await service.registerLocationHints('peer-123', ['hint1', 'hint2']); - expect(mockRegisterLocationHints).toHaveBeenCalledWith('peer-123', [ 'hint1', 'hint2', @@ -726,48 +483,44 @@ describe('NodejsPlatformServices', () => { }); it('throws error if remote comms not initialized', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - + const service = new NodejsPlatformServices({ + workerFilePath, + netlayers, + }); await expect( - service.registerLocationHints('peer-999', ['hint1', 'hint2']), + service.registerLocationHints('peer-999', ['hint1']), ).rejects.toThrowError('remote comms not initialized'); }); }); describe('reconnectPeer', () => { it('reconnects peer via network layer', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - await service.initializeRemoteComms( - '0xtest', - {}, - vi.fn(async () => ''), - ); - - await service.reconnectPeer('peer-456', [ - '/dns4/relay.example/tcp/443/wss/p2p/relayPeer', - ]); - + const service = new NodejsPlatformServices({ + workerFilePath, + netlayers, + }); + await service.initializeRemoteComms(makeInit()); + await service.reconnectPeer('peer-456', ['/dns4/r.example/p2p/x']); expect(mockReconnectPeer).toHaveBeenCalledWith('peer-456', [ - '/dns4/relay.example/tcp/443/wss/p2p/relayPeer', + '/dns4/r.example/p2p/x', ]); }); it('reconnects peer with empty hints', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - await service.initializeRemoteComms( - '0xtest', - {}, - vi.fn(async () => ''), - ); - + const service = new NodejsPlatformServices({ + workerFilePath, + netlayers, + }); + await service.initializeRemoteComms(makeInit()); await service.reconnectPeer('peer-789'); - expect(mockReconnectPeer).toHaveBeenCalledWith('peer-789', []); }); it('throws error if remote comms not initialized', async () => { - const service = new NodejsPlatformServices({ workerFilePath }); - + const service = new NodejsPlatformServices({ + workerFilePath, + netlayers, + }); await expect(service.reconnectPeer('peer-999')).rejects.toThrowError( 'remote comms not initialized', ); diff --git a/packages/kernel-node-runtime/src/kernel/PlatformServices.ts b/packages/kernel-node-runtime/src/kernel/PlatformServices.ts index 71fcf80bfd..158cf396b6 100644 --- a/packages/kernel-node-runtime/src/kernel/PlatformServices.ts +++ b/packages/kernel-node-runtime/src/kernel/PlatformServices.ts @@ -1,20 +1,16 @@ -import { quic } from '@chainsafe/libp2p-quic'; import { makePromiseKit } from '@endo/promise-kit'; -import { tcp } from '@libp2p/tcp'; import { isJsonRpcMessage } from '@metamask/kernel-utils'; import type { JsonRpcMessage } from '@metamask/kernel-utils'; import { Logger } from '@metamask/logger'; import type { - DirectTransport, PlatformServices, VatId, - RemoteMessageHandler, SendRemoteMessage, StopRemoteComms, - RemoteCommsOptions, - OnIncarnationChange, + NetlayerRegistry, + NetlayerSpecifier, + NetlayerHooks, } from '@metamask/ocap-kernel'; -import { initTransport } from '@metamask/ocap-kernel'; import { NodeWorkerDuplexStream } from '@metamask/streams'; import type { DuplexStream } from '@metamask/streams'; import { strict as assert } from 'node:assert'; @@ -52,7 +48,7 @@ export class NodejsPlatformServices implements PlatformServices { #getListenAddressesFunc: (() => string[]) | null = null; - #remoteMessageHandler: RemoteMessageHandler | undefined = undefined; + readonly #netlayers: NetlayerRegistry; readonly #workerFilePath: string; @@ -65,16 +61,19 @@ export class NodejsPlatformServices implements PlatformServices { * The vat worker service, intended to be constructed in * the kernel worker. * - * @param args - A bag of optional arguments. + * @param args - A bag of arguments. * @param args.workerFilePath - An optional path to a file defining the worker's routine. Defaults to 'vat-worker.mjs'. * @param args.logger - An optional {@link Logger}. Defaults to a new logger labeled '[vat worker client]'. + * @param args.netlayers - The registry of netlayer factories, keyed by name; the specifier's `netlayer` selects one. */ constructor(args: { workerFilePath?: string | undefined; logger?: Logger | undefined; + netlayers: NetlayerRegistry; }) { this.#workerFilePath = args.workerFilePath ?? DEFAULT_WORKER_FILE; this.#logger = args.logger ?? new Logger('vat-worker-service'); + this.#netlayers = args.netlayers; } /** @@ -209,100 +208,37 @@ export class NodejsPlatformServices implements PlatformServices { } /** - * Handle a remote message from a peer. + * Initialize network communications by looking up the specified netlayer in + * the injected registry and constructing it. * - * @param from - The peer ID that sent the message. - * @param message - The message received. - * @returns A promise that resolves with the reply message, or null if no reply is needed. - */ - async #handleRemoteMessage( - from: string, - message: string, - ): Promise { - if (!this.#remoteMessageHandler) { - // This can't actually happen, but TypeScript can't infer it - throw Error('remote comms not initialized'); - } - // Return the reply - network layer handles sending it with proper seq/ack - return this.#remoteMessageHandler(from, message); - } - - /** - * Initialize network communications. - * - * @param keySeed - The seed for generating this kernel's secret key. - * @param options - Options for remote communications initialization. - * @param options.relays - Array of the peerIDs of relay nodes that can be used to listen for incoming - * connections from other kernels. - * @param options.maxRetryAttempts - Maximum number of reconnection attempts. 0 = infinite (default). - * @param options.maxQueue - Maximum number of messages to queue per peer while reconnecting (default: 200). - * @param remoteMessageHandler - A handler function to receive remote messages. - * @param onRemoteGiveUp - Optional callback to be called when we give up on a remote. - * @param incarnationId - This kernel's incarnation ID for handshake protocol. - * @param onIncarnationChange - Optional callback when a remote peer's incarnation changes. + * @param params - The initialization options. + * @param params.keySeed - The seed for generating this kernel's secret key. + * @param params.specifier - Which netlayer to use plus its `Json` config. + * @param params.hooks - Kernel-supplied netlayer callbacks. + * @param params.incarnationId - This kernel's incarnation ID for the handshake. * @returns A promise that resolves once network access has been established * or rejects if there is some problem doing so. */ - async initializeRemoteComms( - keySeed: string, - options: RemoteCommsOptions, - remoteMessageHandler: ( - from: string, - message: string, - ) => Promise, - onRemoteGiveUp?: (peerId: string) => void, - incarnationId?: string, - onIncarnationChange?: OnIncarnationChange, - ): Promise { + async initializeRemoteComms({ + keySeed, + specifier, + hooks, + incarnationId, + }: { + keySeed: string; + specifier: NetlayerSpecifier; + hooks: NetlayerHooks; + incarnationId?: string; + }): Promise { if (this.#sendRemoteMessageFunc) { throw Error('remote comms already initialized'); } - this.#remoteMessageHandler = remoteMessageHandler; - - const { directListenAddresses, ...restOptions } = options; - - const directTransports: DirectTransport[] = []; - - if (directListenAddresses && directListenAddresses.length > 0) { - const quicAddresses: string[] = []; - const tcpAddresses: string[] = []; - - for (const addr of directListenAddresses) { - const isQuic = addr.includes('/quic-v1'); - const isTcp = addr.includes('/tcp/'); - - if (isQuic) { - quicAddresses.push(addr); - } else if (isTcp) { - tcpAddresses.push(addr); - } else { - throw new Error( - `Unsupported direct listen address: ${addr}. ` + - `Only QUIC (/quic-v1) and TCP (/tcp/) addresses are supported.`, - ); - } - } - if (quicAddresses.length > 0) { - directTransports.push({ - transport: quic(), - listenAddresses: quicAddresses, - }); - } - - if (tcpAddresses.length > 0) { - directTransports.push({ - transport: tcp(), - listenAddresses: tcpAddresses, - }); - } + const factory = this.#netlayers[specifier.netlayer]; + if (!factory) { + throw new Error(`Unknown netlayer: "${specifier.netlayer}"`); } - const enhancedOptions: RemoteCommsOptions = { - ...restOptions, - ...(directTransports.length > 0 ? { directTransports } : {}), - }; - const { sendRemoteMessage, stop, @@ -311,14 +247,13 @@ export class NodejsPlatformServices implements PlatformServices { reconnectPeer, resetAllBackoffs, getListenAddresses, - } = await initTransport( + } = await factory({ keySeed, - enhancedOptions, - this.#handleRemoteMessage.bind(this), - onRemoteGiveUp, incarnationId, - onIncarnationChange, - ); + hooks, + config: specifier.config, + logger: this.#logger, + }); this.#sendRemoteMessageFunc = sendRemoteMessage; this.#stopRemoteCommsFunc = stop; this.#closeConnectionFunc = closeConnection; diff --git a/packages/kernel-node-runtime/src/kernel/make-kernel.ts b/packages/kernel-node-runtime/src/kernel/make-kernel.ts index 81e4a37e43..d06b05b580 100644 --- a/packages/kernel-node-runtime/src/kernel/make-kernel.ts +++ b/packages/kernel-node-runtime/src/kernel/make-kernel.ts @@ -1,9 +1,11 @@ import type { KernelDatabase } from '@metamask/kernel-store'; import { makeSQLKernelDatabase } from '@metamask/kernel-store/sqlite/nodejs'; import { Logger } from '@metamask/logger'; +import { nodejsLibp2pNetlayerFactory } from '@metamask/netlayer-libp2p/nodejs'; import { Kernel } from '@metamask/ocap-kernel'; import type { IOChannelFactory, + NetlayerRegistry, SystemSubclusterConfig, } from '@metamask/ocap-kernel'; @@ -29,6 +31,8 @@ export type MakeKernelResult = { * @param options.keySeed - Optional seed for libp2p key generation. * @param options.ioChannelFactory - Optional factory for creating IO channels. * @param options.systemSubclusters - Optional system subcluster configurations. + * @param options.netlayers - Optional netlayer registry override; defaults to + * `{ libp2p: nodejsLibp2pNetlayerFactory }`. * @returns The kernel and its database. */ export async function makeKernel({ @@ -39,6 +43,7 @@ export async function makeKernel({ keySeed, ioChannelFactory, systemSubclusters, + netlayers, }: { workerFilePath?: string; resetStorage?: boolean; @@ -47,11 +52,13 @@ export async function makeKernel({ keySeed?: string | undefined; ioChannelFactory?: IOChannelFactory; systemSubclusters?: SystemSubclusterConfig[]; + netlayers?: NetlayerRegistry; }): Promise { const rootLogger = logger ?? new Logger('kernel-worker'); const platformServicesClient = new NodejsPlatformServices({ workerFilePath, logger: rootLogger.subLogger({ tags: ['platform-services-manager'] }), + netlayers: netlayers ?? { libp2p: nodejsLibp2pNetlayerFactory }, }); // Initialize kernel store. diff --git a/packages/kernel-node-runtime/test/e2e/libp2p-v3-features.test.ts b/packages/kernel-node-runtime/test/e2e/libp2p-v3-features.test.ts index b55a68a954..2282ef8f65 100644 --- a/packages/kernel-node-runtime/test/e2e/libp2p-v3-features.test.ts +++ b/packages/kernel-node-runtime/test/e2e/libp2p-v3-features.test.ts @@ -1,6 +1,6 @@ import type { Libp2p } from '@libp2p/interface'; import { makeSQLKernelDatabase } from '@metamask/kernel-store/sqlite/nodejs'; -import { startRelay } from '@metamask/kernel-utils/libp2p'; +import { startRelay } from '@metamask/netlayer-libp2p/relay'; import { Kernel, kunser, makeKernelStore } from '@metamask/ocap-kernel'; import { delay } from '@ocap/repo-tools/test-utils'; import { mkdtemp, rm } from 'node:fs/promises'; diff --git a/packages/kernel-node-runtime/test/e2e/quic-transport.test.ts b/packages/kernel-node-runtime/test/e2e/quic-transport.test.ts index 53cca6346e..b0b51f69ce 100644 --- a/packages/kernel-node-runtime/test/e2e/quic-transport.test.ts +++ b/packages/kernel-node-runtime/test/e2e/quic-transport.test.ts @@ -7,6 +7,7 @@ import { makeTestKernel } from '../helpers/kernel.ts'; import { getVatRootRef, launchVatAndGetURL, + libp2pComms, makeRemoteVatConfig, sendRemoteMessage, } from '../helpers/remote-comms.ts'; @@ -94,12 +95,16 @@ describe.sequential('Direct Transport E2E', () => { it( 'initializes remote comms with QUIC transport', async () => { - await kernel1.initRemoteComms({ - directListenAddresses: [quicListenAddress], - }); - await kernel2.initRemoteComms({ - directListenAddresses: [quicListenAddress], - }); + await kernel1.initRemoteComms( + libp2pComms({ + directListenAddresses: [quicListenAddress], + }), + ); + await kernel2.initRemoteComms( + libp2pComms({ + directListenAddresses: [quicListenAddress], + }), + ); const info1 = await getConnectedInfo(kernel1); const info2 = await getConnectedInfo(kernel2); @@ -117,12 +122,16 @@ describe.sequential('Direct Transport E2E', () => { it( 'initializes remote comms with TCP transport', async () => { - await kernel1.initRemoteComms({ - directListenAddresses: [tcpListenAddress], - }); - await kernel2.initRemoteComms({ - directListenAddresses: [tcpListenAddress], - }); + await kernel1.initRemoteComms( + libp2pComms({ + directListenAddresses: [tcpListenAddress], + }), + ); + await kernel2.initRemoteComms( + libp2pComms({ + directListenAddresses: [tcpListenAddress], + }), + ); const info1 = await getConnectedInfo(kernel1); const info2 = await getConnectedInfo(kernel2); @@ -140,9 +149,11 @@ describe.sequential('Direct Transport E2E', () => { it( 'initializes remote comms with both QUIC and TCP', async () => { - await kernel1.initRemoteComms({ - directListenAddresses: [quicListenAddress, tcpListenAddress], - }); + await kernel1.initRemoteComms( + libp2pComms({ + directListenAddresses: [quicListenAddress, tcpListenAddress], + }), + ); const info1 = await getConnectedInfo(kernel1); @@ -158,12 +169,16 @@ describe.sequential('Direct Transport E2E', () => { 'sends a message via direct QUIC', async () => { // Initialize both kernels with QUIC only - await kernel1.initRemoteComms({ - directListenAddresses: [quicListenAddress], - }); - await kernel2.initRemoteComms({ - directListenAddresses: [quicListenAddress], - }); + await kernel1.initRemoteComms( + libp2pComms({ + directListenAddresses: [quicListenAddress], + }), + ); + await kernel2.initRemoteComms( + libp2pComms({ + directListenAddresses: [quicListenAddress], + }), + ); // Get kernel2's QUIC addresses and register them on kernel1 const info2 = await getConnectedInfo(kernel2); @@ -192,12 +207,16 @@ describe.sequential('Direct Transport E2E', () => { it( 'establishes bidirectional QUIC communication', async () => { - await kernel1.initRemoteComms({ - directListenAddresses: [quicListenAddress], - }); - await kernel2.initRemoteComms({ - directListenAddresses: [quicListenAddress], - }); + await kernel1.initRemoteComms( + libp2pComms({ + directListenAddresses: [quicListenAddress], + }), + ); + await kernel2.initRemoteComms( + libp2pComms({ + directListenAddresses: [quicListenAddress], + }), + ); // Exchange QUIC addresses const info1 = await getConnectedInfo(kernel1); @@ -241,12 +260,16 @@ describe.sequential('Direct Transport E2E', () => { it( 'sends multiple sequential messages via QUIC', async () => { - await kernel1.initRemoteComms({ - directListenAddresses: [quicListenAddress], - }); - await kernel2.initRemoteComms({ - directListenAddresses: [quicListenAddress], - }); + await kernel1.initRemoteComms( + libp2pComms({ + directListenAddresses: [quicListenAddress], + }), + ); + await kernel2.initRemoteComms( + libp2pComms({ + directListenAddresses: [quicListenAddress], + }), + ); const info2 = await getConnectedInfo(kernel2); await kernel1.registerLocationHints(info2.peerId, info2.quicAddresses); @@ -275,12 +298,16 @@ describe.sequential('Direct Transport E2E', () => { it( 'sends a message via direct TCP', async () => { - await kernel1.initRemoteComms({ - directListenAddresses: [tcpListenAddress], - }); - await kernel2.initRemoteComms({ - directListenAddresses: [tcpListenAddress], - }); + await kernel1.initRemoteComms( + libp2pComms({ + directListenAddresses: [tcpListenAddress], + }), + ); + await kernel2.initRemoteComms( + libp2pComms({ + directListenAddresses: [tcpListenAddress], + }), + ); const info2 = await getConnectedInfo(kernel2); await kernel1.registerLocationHints(info2.peerId, info2.tcpAddresses); diff --git a/packages/kernel-node-runtime/test/e2e/relay-connectivity.test.ts b/packages/kernel-node-runtime/test/e2e/relay-connectivity.test.ts index 13ec963bc2..ff4fab8308 100644 --- a/packages/kernel-node-runtime/test/e2e/relay-connectivity.test.ts +++ b/packages/kernel-node-runtime/test/e2e/relay-connectivity.test.ts @@ -1,12 +1,12 @@ import type { Libp2p } from '@libp2p/interface'; import { makeSQLKernelDatabase } from '@metamask/kernel-store/sqlite/nodejs'; -import { startRelay } from '@metamask/kernel-utils/libp2p'; +import { startRelay } from '@metamask/netlayer-libp2p/relay'; import { Kernel } from '@metamask/ocap-kernel'; import { createConnection } from 'node:net'; import { describe, it, expect, beforeAll, afterAll } from 'vitest'; import { makeTestKernel } from '../helpers/kernel.ts'; -import { getRemoteCommsPeerId } from '../helpers/remote-comms.ts'; +import { getRemoteCommsPeerId, libp2pComms } from '../helpers/remote-comms.ts'; import { stopWithTimeout } from '../helpers/stop-with-timeout.ts'; const STOP_TIMEOUT = 5_000; @@ -68,7 +68,7 @@ describe('Relay Connectivity', () => { try { kernel = await makeTestKernel(kernelDb); - await kernel.initRemoteComms({ relays: [relayAddr] }); + await kernel.initRemoteComms(libp2pComms({ relays: [relayAddr] })); const status = await kernel.getStatus(); expect(status.remoteComms?.state).toBe('connected'); @@ -100,8 +100,8 @@ describe('Relay Connectivity', () => { kernel1 = await makeTestKernel(kernelDb1); kernel2 = await makeTestKernel(kernelDb2); - await kernel1.initRemoteComms({ relays: [relayAddr] }); - await kernel2.initRemoteComms({ relays: [relayAddr] }); + await kernel1.initRemoteComms(libp2pComms({ relays: [relayAddr] })); + await kernel2.initRemoteComms(libp2pComms({ relays: [relayAddr] })); const status1 = await kernel1.getStatus(); const status2 = await kernel2.getStatus(); @@ -148,7 +148,7 @@ describe('Relay Connectivity', () => { kernel = await makeTestKernel(kernelDb); // initRemoteComms should not throw even if the relay is unreachable — // the libp2p node starts locally and relay connection happens async. - await kernel.initRemoteComms({ relays: [badRelayAddr] }); + await kernel.initRemoteComms(libp2pComms({ relays: [badRelayAddr] })); const status = await kernel.getStatus(); // The kernel still gets a local peer ID even without relay connectivity diff --git a/packages/kernel-node-runtime/test/e2e/remote-comms.test.ts b/packages/kernel-node-runtime/test/e2e/remote-comms.test.ts index 552339a2ac..a86a5ae981 100644 --- a/packages/kernel-node-runtime/test/e2e/remote-comms.test.ts +++ b/packages/kernel-node-runtime/test/e2e/remote-comms.test.ts @@ -1,7 +1,7 @@ import type { Libp2p } from '@libp2p/interface'; import { makeSQLKernelDatabase } from '@metamask/kernel-store/sqlite/nodejs'; import { waitUntilQuiescent } from '@metamask/kernel-utils'; -import { startRelay } from '@metamask/kernel-utils/libp2p'; +import { startRelay } from '@metamask/netlayer-libp2p/relay'; import { Kernel, kunser, makeKernelStore } from '@metamask/ocap-kernel'; import type { KRef } from '@metamask/ocap-kernel'; import { delay } from '@ocap/repo-tools/test-utils'; @@ -23,6 +23,7 @@ import { getPeerIds, getVatRootRef, launchVatAndGetURL, + libp2pComms, makeMaasClientConfig, makeMaasServerConfig, makeRemoteVatConfig, @@ -117,14 +118,18 @@ describe.sequential('Remote Communications E2E', () => { it( 'initializes remote comms on both kernels', async () => { - await kernel1.initRemoteComms({ - relays: testRelays, - ...testBackoffOptions, - }); - await kernel2.initRemoteComms({ - relays: testRelays, - ...testBackoffOptions, - }); + await kernel1.initRemoteComms( + libp2pComms({ + relays: testRelays, + ...testBackoffOptions, + }), + ); + await kernel2.initRemoteComms( + libp2pComms({ + relays: testRelays, + ...testBackoffOptions, + }), + ); const status1 = await kernel1.getStatus(); const status2 = await kernel2.getStatus(); @@ -201,14 +206,18 @@ describe.sequential('Remote Communications E2E', () => { 'remote relationships should survive kernel restart', async () => { // Initialize remote comms - await kernel1.initRemoteComms({ - relays: testRelays, - ...testBackoffOptions, - }); - await kernel2.initRemoteComms({ - relays: testRelays, - ...testBackoffOptions, - }); + await kernel1.initRemoteComms( + libp2pComms({ + relays: testRelays, + ...testBackoffOptions, + }), + ); + await kernel2.initRemoteComms( + libp2pComms({ + relays: testRelays, + ...testBackoffOptions, + }), + ); // Launch client vat on kernel1 const clientConfig = makeMaasClientConfig('client1', true); @@ -254,10 +263,12 @@ describe.sequential('Remote Communications E2E', () => { await makeSQLKernelDatabase({ dbFilename: dbFilename2 }), { resetStorage: false }, ); - await serverKernel.initRemoteComms({ - relays: testRelays, - ...testBackoffOptions, - }); + await serverKernel.initRemoteComms( + libp2pComms({ + relays: testRelays, + ...testBackoffOptions, + }), + ); // Tell the client to talk to the server a second time expectedCount += 1; @@ -276,10 +287,12 @@ describe.sequential('Remote Communications E2E', () => { await makeSQLKernelDatabase({ dbFilename: dbFilename1 }), { resetStorage: false }, ); - await clientKernel.initRemoteComms({ - relays: testRelays, - ...testBackoffOptions, - }); + await clientKernel.initRemoteComms( + libp2pComms({ + relays: testRelays, + ...testBackoffOptions, + }), + ); // Tell the client to talk to the server a third time expectedCount += 1; @@ -374,10 +387,12 @@ describe.sequential('Remote Communications E2E', () => { it( 'handles connection failure to non-existent peer', async () => { - await kernel1.initRemoteComms({ - relays: testRelays, - ...testBackoffOptions, - }); + await kernel1.initRemoteComms( + libp2pComms({ + relays: testRelays, + ...testBackoffOptions, + }), + ); const aliceConfig = makeRemoteVatConfig('Alice'); await launchVatAndGetURL(kernel1, aliceConfig); @@ -490,14 +505,18 @@ describe.sequential('Remote Communications E2E', () => { it( 'queues messages when connection is not established', async () => { - await kernel1.initRemoteComms({ - relays: testRelays, - ...testBackoffOptions, - }); - await kernel2.initRemoteComms({ - relays: testRelays, - ...testBackoffOptions, - }); + await kernel1.initRemoteComms( + libp2pComms({ + relays: testRelays, + ...testBackoffOptions, + }), + ); + await kernel2.initRemoteComms( + libp2pComms({ + relays: testRelays, + ...testBackoffOptions, + }), + ); const aliceConfig = makeRemoteVatConfig('Alice'); await launchVatAndGetURL(kernel1, aliceConfig); @@ -680,19 +699,25 @@ describe.sequential('Remote Communications E2E', () => { }; try { - await kernel1.initRemoteComms({ - relays: testRelays, - ...multiPeerOptions, - }); - await kernel2.initRemoteComms({ - relays: testRelays, - ...multiPeerOptions, - }); + await kernel1.initRemoteComms( + libp2pComms({ + relays: testRelays, + ...multiPeerOptions, + }), + ); + await kernel2.initRemoteComms( + libp2pComms({ + relays: testRelays, + ...multiPeerOptions, + }), + ); kernel3 = await makeTestKernel(kernelDatabase3); - await kernel3.initRemoteComms({ - relays: testRelays, - ...multiPeerOptions, - }); + await kernel3.initRemoteComms( + libp2pComms({ + relays: testRelays, + ...multiPeerOptions, + }), + ); const aliceConfig = makeRemoteVatConfig('Alice'); const bobConfigInitial = makeRemoteVatConfig('Bob'); @@ -953,21 +978,25 @@ describe.sequential('Remote Communications E2E', () => { // Initialize with low retry attempts to trigger give-up on incarnation change. // Use moderate backoff (not the fast test defaults) so reconnection attempts // don't exhaust before the restarted peer is reachable. - await kernel1.initRemoteComms({ - relays: testRelays, - maxRetryAttempts: 2, - reconnectionBaseDelayMs: 200, - reconnectionMaxDelayMs: 500, - handshakeTimeoutMs: 3_000, - writeTimeoutMs: 3_000, - }); - await kernel2.initRemoteComms({ - relays: testRelays, - reconnectionBaseDelayMs: 200, - reconnectionMaxDelayMs: 500, - handshakeTimeoutMs: 3_000, - writeTimeoutMs: 3_000, - }); + await kernel1.initRemoteComms( + libp2pComms({ + relays: testRelays, + maxRetryAttempts: 2, + reconnectionBaseDelayMs: 200, + reconnectionMaxDelayMs: 500, + handshakeTimeoutMs: 3_000, + writeTimeoutMs: 3_000, + }), + ); + await kernel2.initRemoteComms( + libp2pComms({ + relays: testRelays, + reconnectionBaseDelayMs: 200, + reconnectionMaxDelayMs: 500, + handshakeTimeoutMs: 3_000, + writeTimeoutMs: 3_000, + }), + ); const aliceConfig = makeRemoteVatConfig('Alice'); const bobConfig = makeRemoteVatConfig('Bob'); @@ -991,10 +1020,12 @@ describe.sequential('Remote Communications E2E', () => { const freshKernel2 = await makeTestKernel(freshDb2); // eslint-disable-next-line require-atomic-updates kernel2 = freshKernel2; - await kernel2.initRemoteComms({ - relays: testRelays, - ...testBackoffOptions, - }); + await kernel2.initRemoteComms( + libp2pComms({ + relays: testRelays, + ...testBackoffOptions, + }), + ); // Launch Bob again (fresh vat, no previous state) await launchVatAndGetURL(kernel2, bobConfig); @@ -1052,8 +1083,10 @@ describe.sequential('Remote Communications E2E', () => { maxRetryAttempts: 4, }; - await kernel1.initRemoteComms({ ...opts }); - await kernel2.initRemoteComms({ ...opts, mnemonic: k2Mnemonic }); + await kernel1.initRemoteComms(libp2pComms({ ...opts })); + await kernel2.initRemoteComms( + libp2pComms({ ...opts, mnemonic: k2Mnemonic }), + ); const aliceURL = await launchVatAndGetURL( kernel1, makeRemoteVatConfig('Alice'), @@ -1094,7 +1127,9 @@ describe.sequential('Remote Communications E2E', () => { kernelStore2 = makeKernelStore(freshDb2); // eslint-disable-next-line require-atomic-updates kernel2 = await makeTestKernel(freshDb2); - await kernel2.initRemoteComms({ ...opts, mnemonic: k2Mnemonic }); + await kernel2.initRemoteComms( + libp2pComms({ ...opts, mnemonic: k2Mnemonic }), + ); await launchVatAndGetURL(kernel2, makeRemoteVatConfig('Bob')); const newBobRef = getVatRootRef(kernel2, kernelStore2, 'Bob'); @@ -1143,8 +1178,10 @@ describe.sequential('Remote Communications E2E', () => { maxRetryAttempts: 4, }; - await kernel1.initRemoteComms({ ...opts }); - await kernel2.initRemoteComms({ ...opts, mnemonic: k2Mnemonic }); + await kernel1.initRemoteComms(libp2pComms({ ...opts })); + await kernel2.initRemoteComms( + libp2pComms({ ...opts, mnemonic: k2Mnemonic }), + ); const aliceURL = await launchVatAndGetURL( kernel1, makeRemoteVatConfig('Alice'), @@ -1200,21 +1237,25 @@ describe.sequential('Remote Communications E2E', () => { // Initialize kernel1 with a low maxRetryAttempts to trigger give-up. // Use moderate backoff so reconnection attempts don't exhaust before // the message can be queued and become a pending promise. - await kernel1.initRemoteComms({ - relays: testRelays, - maxRetryAttempts: 3, - reconnectionBaseDelayMs: 200, - reconnectionMaxDelayMs: 1_000, - handshakeTimeoutMs: 3_000, - writeTimeoutMs: 3_000, - }); - await kernel2.initRemoteComms({ - relays: testRelays, - reconnectionBaseDelayMs: 200, - reconnectionMaxDelayMs: 1_000, - handshakeTimeoutMs: 3_000, - writeTimeoutMs: 3_000, - }); + await kernel1.initRemoteComms( + libp2pComms({ + relays: testRelays, + maxRetryAttempts: 3, + reconnectionBaseDelayMs: 200, + reconnectionMaxDelayMs: 1_000, + handshakeTimeoutMs: 3_000, + writeTimeoutMs: 3_000, + }), + ); + await kernel2.initRemoteComms( + libp2pComms({ + relays: testRelays, + reconnectionBaseDelayMs: 200, + reconnectionMaxDelayMs: 1_000, + handshakeTimeoutMs: 3_000, + writeTimeoutMs: 3_000, + }), + ); // Set up Alice and Bob manually (can't use setupAliceAndBob as it reinitializes comms) const aliceConfig = makeRemoteVatConfig('Alice'); diff --git a/packages/kernel-node-runtime/test/helpers/kernel.ts b/packages/kernel-node-runtime/test/helpers/kernel.ts index 1595897c12..3903466cdd 100644 --- a/packages/kernel-node-runtime/test/helpers/kernel.ts +++ b/packages/kernel-node-runtime/test/helpers/kernel.ts @@ -1,6 +1,7 @@ import type { KernelDatabase } from '@metamask/kernel-store'; import { waitUntilQuiescent } from '@metamask/kernel-utils'; import { Logger } from '@metamask/logger'; +import { nodejsLibp2pNetlayerFactory } from '@metamask/netlayer-libp2p/nodejs'; import { Kernel, kunser } from '@metamask/ocap-kernel'; import type { ClusterConfig, @@ -43,6 +44,7 @@ export async function makeTestKernel( const logger = new Logger('test-kernel'); const platformServices = new NodejsPlatformServices({ logger: logger.subLogger({ tags: ['platform-services'] }), + netlayers: { libp2p: nodejsLibp2pNetlayerFactory }, }); const kernel = await Kernel.make(platformServices, kernelDatabase, { resetStorage, diff --git a/packages/kernel-node-runtime/test/helpers/remote-comms.ts b/packages/kernel-node-runtime/test/helpers/remote-comms.ts index ec7d0dcbe3..81aeb2aa33 100644 --- a/packages/kernel-node-runtime/test/helpers/remote-comms.ts +++ b/packages/kernel-node-runtime/test/helpers/remote-comms.ts @@ -10,11 +10,74 @@ import type { import { makeTestKernel } from './kernel.ts'; +/** + * Flat libp2p comms options (the pre-netlayer-injection shape) used by the e2e + * tests. Libp2p-specific fields become `specifier.config`; kernel-level fields + * stay top-level. + */ +export type FlatLibp2pCommsOptions = { + relays?: string[]; + directListenAddresses?: string[]; + allowedWsHosts?: string[]; + maxRetryAttempts?: number; + maxConcurrentConnections?: number; + maxMessageSizeBytes?: number; + cleanupIntervalMs?: number; + stalePeerTimeoutMs?: number; + maxMessagesPerSecond?: number; + maxConnectionAttemptsPerMinute?: number; + reconnectionBaseDelayMs?: number; + reconnectionMaxDelayMs?: number; + handshakeTimeoutMs?: number; + writeTimeoutMs?: number; + streamInactivityTimeoutMs?: number; + // Kernel-level (stay outside the netlayer config): + mnemonic?: string; + maxQueue?: number; + ackTimeoutMs?: number; + maxUrlRelayHints?: number; + maxKnownRelays?: number; +}; + +/** + * Convert flat libp2p comms options into the injected-netlayer + * {@link RemoteCommsOptions} shape: libp2p-specific fields become + * `specifier.config` (with `relays` renamed to `knownRelays`); kernel-level + * fields stay top-level. + * + * @param flat - The flat libp2p comms options. + * @returns The netlayer-shaped remote comms options. + */ +export function libp2pComms( + flat: FlatLibp2pCommsOptions = {}, +): RemoteCommsOptions { + const { + relays, + mnemonic, + maxQueue, + ackTimeoutMs, + maxUrlRelayHints, + maxKnownRelays, + ...configFields + } = flat; + const config: Record = { ...configFields }; + if (relays !== undefined) { + config.knownRelays = relays; + } + return { + specifier: { netlayer: 'libp2p', config }, + ...(mnemonic !== undefined && { mnemonic }), + ...(maxQueue !== undefined && { maxQueue }), + ...(ackTimeoutMs !== undefined && { ackTimeoutMs }), + ...(maxUrlRelayHints !== undefined && { maxUrlRelayHints }), + ...(maxKnownRelays !== undefined && { maxKnownRelays }), + } as RemoteCommsOptions; +} + /** * Options forwarded to initRemoteComms by restart helpers. - * Omits 'relays' which is passed as a separate parameter. */ -type RestartRemoteCommsOptions = Omit; +type RestartRemoteCommsOptions = FlatLibp2pCommsOptions; /** * Extract peerId from remoteComms status, returning undefined for @@ -171,7 +234,7 @@ export async function restartKernel( ): Promise { const kernelDatabase = await makeSQLKernelDatabase({ dbFilename }); const kernel = await makeTestKernel(kernelDatabase, { resetStorage }); - await kernel.initRemoteComms({ relays, ...remoteCommsOptions }); + await kernel.initRemoteComms(libp2pComms({ relays, ...remoteCommsOptions })); return kernel; } @@ -229,7 +292,7 @@ export async function setupAliceAndBob( kernelStore1: ReturnType, kernelStore2: ReturnType, relays: string[], - remoteCommsOptions?: Omit, + remoteCommsOptions?: FlatLibp2pCommsOptions, ): Promise<{ aliceURL: string; bobURL: string; @@ -238,8 +301,8 @@ export async function setupAliceAndBob( peerId1: string; peerId2: string; }> { - await kernel1.initRemoteComms({ relays, ...remoteCommsOptions }); - await kernel2.initRemoteComms({ relays, ...remoteCommsOptions }); + await kernel1.initRemoteComms(libp2pComms({ relays, ...remoteCommsOptions })); + await kernel2.initRemoteComms(libp2pComms({ relays, ...remoteCommsOptions })); const aliceConfig = makeRemoteVatConfig('Alice'); const bobConfig = makeRemoteVatConfig('Bob'); diff --git a/packages/kernel-node-runtime/tsconfig.build.json b/packages/kernel-node-runtime/tsconfig.build.json index e3ad4d2b09..f1c88333c2 100644 --- a/packages/kernel-node-runtime/tsconfig.build.json +++ b/packages/kernel-node-runtime/tsconfig.build.json @@ -12,6 +12,7 @@ { "path": "../ocap-kernel/tsconfig.build.json" }, { "path": "../logger/tsconfig.build.json" }, { "path": "../kernel-utils/tsconfig.build.json" }, + { "path": "../netlayer-libp2p/tsconfig.build.json" }, { "path": "../streams/tsconfig.build.json" }, { "path": "../kernel-store/tsconfig.build.json" } ], diff --git a/packages/kernel-node-runtime/tsconfig.json b/packages/kernel-node-runtime/tsconfig.json index 23136c0518..e002b53e02 100644 --- a/packages/kernel-node-runtime/tsconfig.json +++ b/packages/kernel-node-runtime/tsconfig.json @@ -11,6 +11,7 @@ { "path": "../kernel-platforms" }, { "path": "../ocap-kernel" }, { "path": "../logger" }, + { "path": "../netlayer-libp2p" }, { "path": "../streams" }, { "path": "../kernel-utils" }, { "path": "../repo-tools" }, diff --git a/packages/kernel-test/package.json b/packages/kernel-test/package.json index b6c5851141..c40f8372fe 100644 --- a/packages/kernel-test/package.json +++ b/packages/kernel-test/package.json @@ -56,6 +56,7 @@ "@metamask/kernel-store": "workspace:^", "@metamask/kernel-utils": "workspace:^", "@metamask/logger": "workspace:^", + "@metamask/netlayer-libp2p": "workspace:^", "@metamask/ocap-kernel": "workspace:^", "@ocap/kernel-language-model-service": "workspace:^", "@ocap/nodejs-test-workers": "workspace:^", diff --git a/packages/kernel-test/src/endowment-globals.test.ts b/packages/kernel-test/src/endowment-globals.test.ts index 61f30f5121..fc6a922451 100644 --- a/packages/kernel-test/src/endowment-globals.test.ts +++ b/packages/kernel-test/src/endowment-globals.test.ts @@ -7,6 +7,7 @@ import { makeArrayTransport, } from '@metamask/logger'; import type { LogEntry } from '@metamask/logger'; +import { nodejsLibp2pNetlayerFactory } from '@metamask/netlayer-libp2p/nodejs'; import { Kernel } from '@metamask/ocap-kernel'; import type { AllowedGlobalName, KRef, VatId } from '@metamask/ocap-kernel'; import { getWorkerFile } from '@ocap/nodejs-test-workers'; @@ -33,6 +34,7 @@ describe('global endowments', () => { const platformServices = new NodejsPlatformServices({ logger: logger.subLogger({ tags: ['vat-worker-manager'] }), workerFilePath: getWorkerFile('mock-fetch'), + netlayers: { libp2p: nodejsLibp2pNetlayerFactory }, }); const kernel = await Kernel.make(platformServices, database, { resetStorage: true, diff --git a/packages/kernel-test/src/io.test.ts b/packages/kernel-test/src/io.test.ts index 62ccc7f5a7..1c5e5e57dc 100644 --- a/packages/kernel-test/src/io.test.ts +++ b/packages/kernel-test/src/io.test.ts @@ -194,9 +194,13 @@ describe('IO kernel service', () => { const { NodejsPlatformServices } = await import( '@metamask/kernel-node-runtime' ); + const { nodejsLibp2pNetlayerFactory } = await import( + '@metamask/netlayer-libp2p/nodejs' + ); const kernel = await Kernel.make( new NodejsPlatformServices({ logger: logger.subLogger({ tags: ['platform'] }), + netlayers: { libp2p: nodejsLibp2pNetlayerFactory }, }), kernelDatabase, { diff --git a/packages/kernel-test/src/remote-comms.test.ts b/packages/kernel-test/src/remote-comms.test.ts index 0da91f091e..d11ae6abbc 100644 --- a/packages/kernel-test/src/remote-comms.test.ts +++ b/packages/kernel-test/src/remote-comms.test.ts @@ -2,6 +2,7 @@ import { NodejsPlatformServices } from '@metamask/kernel-node-runtime'; import type { KernelDatabase } from '@metamask/kernel-store'; import { makeSQLKernelDatabase } from '@metamask/kernel-store/sqlite/nodejs'; import { fromHex } from '@metamask/kernel-utils'; +import { nodejsLibp2pNetlayerFactory } from '@metamask/netlayer-libp2p/nodejs'; import { makeKernelStore, kunser, @@ -13,8 +14,8 @@ import type { ClusterConfig, KRef, PlatformServices, - RemoteMessageHandler, - RemoteCommsOptions, + NetlayerSpecifier, + NetlayerHooks, } from '@metamask/ocap-kernel'; import { mkdtemp, rm } from 'node:fs/promises'; import { tmpdir } from 'node:os'; @@ -69,6 +70,7 @@ class DirectNetworkService { async launch(vatId) { const realServices = new NodejsPlatformServices({ logger: makeTestLogger().logger, + netlayers: { libp2p: nodejsLibp2pNetlayerFactory }, }); return realServices.launch(vatId); }, @@ -102,16 +104,21 @@ class DirectNetworkService { } }, - async initializeRemoteComms( - keySeed: string, - _options: RemoteCommsOptions, - handler: RemoteMessageHandler, - ) { + async initializeRemoteComms(params: { + keySeed: string; + specifier: NetlayerSpecifier; + hooks: NetlayerHooks; + incarnationId?: string; + }) { // Derive the actual (neutral) peer ID from the key seed. - actualPeerId = deriveNeutralPeerId(fromHex(keySeed)); + actualPeerId = deriveNeutralPeerId(fromHex(params.keySeed)); // Register this peer in the direct network with its actual ID - self.registerPeer(actualPeerId, handler, 'direct://localhost'); + self.registerPeer( + actualPeerId, + params.hooks.handleMessage, + 'direct://localhost', + ); console.log(`Registered peer ${actualPeerId} for direct messaging`); return Promise.resolve(); }, diff --git a/packages/kernel-test/src/utils.ts b/packages/kernel-test/src/utils.ts index c255347b89..3223d38045 100644 --- a/packages/kernel-test/src/utils.ts +++ b/packages/kernel-test/src/utils.ts @@ -10,6 +10,7 @@ import { makeConsoleTransport, } from '@metamask/logger'; import type { LogEntry } from '@metamask/logger'; +import { nodejsLibp2pNetlayerFactory } from '@metamask/netlayer-libp2p/nodejs'; import { Kernel, kunser } from '@metamask/ocap-kernel'; import type { ClusterConfig, PlatformServices } from '@metamask/ocap-kernel'; import { vi } from 'vitest'; @@ -81,8 +82,13 @@ export async function makeKernel( platformServices?: PlatformServices, keySeed?: string, ): Promise { - const platformServicesConfig: { logger: Logger; workerFilePath?: string } = { + const platformServicesConfig: { + logger: Logger; + workerFilePath?: string; + netlayers: { libp2p: typeof nodejsLibp2pNetlayerFactory }; + } = { logger: logger.subLogger({ tags: ['vat-worker-manager'] }), + netlayers: { libp2p: nodejsLibp2pNetlayerFactory }, }; if (workerFilePath) { platformServicesConfig.workerFilePath = workerFilePath; diff --git a/packages/kernel-test/tsconfig.json b/packages/kernel-test/tsconfig.json index ae412ba35b..779f04d45a 100644 --- a/packages/kernel-test/tsconfig.json +++ b/packages/kernel-test/tsconfig.json @@ -12,6 +12,7 @@ { "path": "../kernel-store" }, { "path": "../kernel-utils" }, { "path": "../kernel-node-runtime" }, + { "path": "../netlayer-libp2p" }, { "path": "../nodejs-test-workers" }, { "path": "../ocap-kernel" }, { "path": "../remote-iterables" }, diff --git a/packages/kernel-utils/CHANGELOG.md b/packages/kernel-utils/CHANGELOG.md index 56c5e767b0..6b3efb8bb8 100644 --- a/packages/kernel-utils/CHANGELOG.md +++ b/packages/kernel-utils/CHANGELOG.md @@ -14,6 +14,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Add `getLibp2pRelayHome()` to the `./nodejs` exports, returning the libp2p relay's bookkeeping directory (default `~/.libp2p-relay`, overridable via `$LIBP2P_RELAY_HOME`) — kept separate from `$OCAP_HOME` so one relay can serve daemons with different OCAP_HOMEs ([#952](https://github.com/MetaMask/ocap-kernel/pull/952)) - `startRelay()` accepts an optional `publicIp` that is fed to libp2p's `appendAnnounce`, so a relay running on a NAT-backed host can announce its publicly-reachable IPv4 alongside its bound NIC addresses ([#952](https://github.com/MetaMask/ocap-kernel/pull/952)) +### Removed + +- **BREAKING:** Remove the `./libp2p` export (`startRelay`) and all libp2p-family dependencies — the relay server moves to `@metamask/netlayer-libp2p/relay`. `getLibp2pRelayHome` (a plain path helper with no libp2p import) stays on `./nodejs` ([#973](https://github.com/MetaMask/ocap-kernel/pull/973)) + ## [0.5.0] ### Added diff --git a/packages/kernel-utils/package.json b/packages/kernel-utils/package.json index 9ea10f3982..dadc39ea31 100644 --- a/packages/kernel-utils/package.json +++ b/packages/kernel-utils/package.json @@ -69,16 +69,6 @@ "default": "./dist/described.cjs" } }, - "./libp2p": { - "import": { - "types": "./dist/libp2p-relay.d.mts", - "default": "./dist/libp2p-relay.mjs" - }, - "require": { - "types": "./dist/libp2p-relay.d.cts", - "default": "./dist/libp2p-relay.cjs" - } - }, "./nodejs": { "import": { "types": "./dist/nodejs/index.d.mts", @@ -127,25 +117,13 @@ "test:dev:quiet": "yarn test:dev --reporter @ocap/repo-tools/vitest-reporters/silent" }, "dependencies": { - "@chainsafe/libp2p-noise": "^17.0.0", - "@chainsafe/libp2p-yamux": "8.0.1", "@endo/captp": "^4.4.8", "@endo/errors": "^1.2.13", "@endo/exo": "^1.5.12", "@endo/patterns": "^1.7.0", "@endo/promise-kit": "^1.1.13", - "@libp2p/autonat": "3.0.14", - "@libp2p/circuit-relay-v2": "4.1.7", - "@libp2p/crypto": "5.1.14", - "@libp2p/identify": "4.0.14", - "@libp2p/interface": "3.1.1", - "@libp2p/ping": "3.0.14", - "@libp2p/tcp": "11.0.14", - "@libp2p/websockets": "10.1.7", "@metamask/superstruct": "^3.2.1", "@metamask/utils": "^11.9.0", - "@multiformats/multiaddr": "^13.0.1", - "libp2p": "3.1.7", "setimmediate": "^1.0.5" }, "devDependencies": { diff --git a/packages/netlayer-libp2p/CHANGELOG.md b/packages/netlayer-libp2p/CHANGELOG.md new file mode 100644 index 0000000000..f14b5262c2 --- /dev/null +++ b/packages/netlayer-libp2p/CHANGELOG.md @@ -0,0 +1,18 @@ +# Changelog + +All notable changes to this project will be documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), +and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +## [Unreleased] + +### Added + +- Initial release: the libp2p netlayer implementation, extracted from `@metamask/ocap-kernel` ([#973](https://github.com/MetaMask/ocap-kernel/pull/973)) + - `.` (browser default) exports `libp2pNetlayerFactory` plus `makeLibp2pNetlayer`, `Libp2pNetlayerConfig`/`Libp2pNetlayerConfigStruct`, and `DirectTransport` + - `./nodejs` exports `nodejsLibp2pNetlayerFactory` (QUIC/TCP direct transports) and `buildDirectTransports` + - `./relay` exports `startRelay` (the circuit-relay server, moved from `@metamask/kernel-utils/libp2p`) + - `error-mapper.ts` maps raw libp2p read/dial errors onto the neutral `@metamask/kernel-errors` classes and owns the libp2p-specific retryability classification (`MuxerClosedError`/`Dial`/`Transport`/`NO_RESERVATION`) + +[Unreleased]: https://github.com/MetaMask/ocap-kernel/ diff --git a/packages/netlayer-libp2p/LICENSE.APACHE2 b/packages/netlayer-libp2p/LICENSE.APACHE2 new file mode 100644 index 0000000000..8194a06aee --- /dev/null +++ b/packages/netlayer-libp2p/LICENSE.APACHE2 @@ -0,0 +1,201 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright 2025 Consensys Software Inc. + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/packages/netlayer-libp2p/LICENSE.MIT b/packages/netlayer-libp2p/LICENSE.MIT new file mode 100644 index 0000000000..658c855eb8 --- /dev/null +++ b/packages/netlayer-libp2p/LICENSE.MIT @@ -0,0 +1,20 @@ +MIT License + +Copyright (c) 2025 Consensys Software Inc. + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE diff --git a/packages/netlayer-libp2p/README.md b/packages/netlayer-libp2p/README.md new file mode 100644 index 0000000000..a5e939b9d8 --- /dev/null +++ b/packages/netlayer-libp2p/README.md @@ -0,0 +1,20 @@ +# `@metamask/netlayer-libp2p` + +libp2p netlayer implementation for the ocap kernel. + +This package implements the `@metamask/netlayer` contract over libp2p. It exports +the browser-default factory (`libp2pNetlayerFactory`) from its main entry, a +Node.js factory with direct QUIC/TCP transports from the `./nodejs` subpath, and +the circuit-relay server (`startRelay`) from the `./relay` subpath. + +## Installation + +`yarn add @metamask/netlayer-libp2p` + +or + +`npm install @metamask/netlayer-libp2p` + +## Contributing + +This package is part of a monorepo. Instructions for contributing can be found in the [monorepo README](https://github.com/MetaMask/ocap-kernel#readme). diff --git a/packages/netlayer-libp2p/package.json b/packages/netlayer-libp2p/package.json new file mode 100644 index 0000000000..8370c8640f --- /dev/null +++ b/packages/netlayer-libp2p/package.json @@ -0,0 +1,145 @@ +{ + "name": "@metamask/netlayer-libp2p", + "version": "0.1.0", + "description": "libp2p netlayer implementation for the ocap kernel", + "keywords": [ + "MetaMask", + "object capabilities", + "ocap" + ], + "homepage": "https://github.com/MetaMask/ocap-kernel/tree/main/packages/netlayer-libp2p#readme", + "bugs": { + "url": "https://github.com/MetaMask/ocap-kernel/issues" + }, + "repository": { + "type": "git", + "url": "https://github.com/MetaMask/ocap-kernel.git" + }, + "license": "(MIT OR Apache-2.0)", + "sideEffects": false, + "type": "module", + "exports": { + ".": { + "import": { + "types": "./dist/index.d.mts", + "default": "./dist/index.mjs" + }, + "require": { + "types": "./dist/index.d.cts", + "default": "./dist/index.cjs" + } + }, + "./nodejs": { + "import": { + "types": "./dist/nodejs/index.d.mts", + "default": "./dist/nodejs/index.mjs" + }, + "require": { + "types": "./dist/nodejs/index.d.cts", + "default": "./dist/nodejs/index.cjs" + } + }, + "./relay": { + "import": { + "types": "./dist/relay/index.d.mts", + "default": "./dist/relay/index.mjs" + }, + "require": { + "types": "./dist/relay/index.d.cts", + "default": "./dist/relay/index.cjs" + } + }, + "./package.json": "./package.json" + }, + "main": "./dist/index.cjs", + "module": "./dist/index.mjs", + "types": "./dist/index.d.cts", + "files": [ + "dist/" + ], + "scripts": { + "build": "ts-bridge --project tsconfig.build.json --no-references --clean", + "build:docs": "typedoc", + "changelog:update": "../../scripts/update-changelog.sh @metamask/netlayer-libp2p", + "changelog:validate": "../../scripts/validate-changelog.sh @metamask/netlayer-libp2p", + "clean": "rimraf --glob './*.tsbuildinfo' ./.eslintcache ./coverage ./dist ./.turbo ./logs", + "lint": "yarn lint:eslint && yarn lint:misc --check && yarn constraints && yarn lint:dependencies", + "lint:dependencies": "depcheck --quiet", + "lint:eslint": "eslint . --cache", + "lint:fix": "yarn lint:eslint --fix && yarn lint:misc --write && yarn constraints --fix && yarn lint:dependencies", + "lint:misc": "prettier --no-error-on-unmatched-pattern '**/*.json' '**/*.md' '**/*.html' '!**/CHANGELOG.old.md' '**/*.yml' '!.yarnrc.yml' '!merged-packages/**' --ignore-path ../../.gitignore --log-level error", + "publish:preview": "yarn npm publish --tag preview", + "test": "vitest run --config vitest.config.ts", + "test:clean": "yarn test --no-cache --coverage.clean", + "test:dev": "yarn test --mode development", + "test:verbose": "yarn test --reporter verbose", + "test:watch": "vitest --config vitest.config.ts", + "test:dev:quiet": "yarn test:dev --reporter @ocap/repo-tools/vitest-reporters/silent" + }, + "dependencies": { + "@chainsafe/libp2p-noise": "^17.0.0", + "@chainsafe/libp2p-quic": "^2.0.1", + "@chainsafe/libp2p-yamux": "8.0.1", + "@libp2p/autonat": "3.0.14", + "@libp2p/bootstrap": "12.0.15", + "@libp2p/circuit-relay-v2": "4.1.7", + "@libp2p/crypto": "5.1.14", + "@libp2p/identify": "4.0.14", + "@libp2p/interface": "3.1.1", + "@libp2p/peer-id": "6.0.5", + "@libp2p/ping": "3.0.14", + "@libp2p/tcp": "11.0.14", + "@libp2p/utils": "7.0.14", + "@libp2p/webrtc": "6.0.15", + "@libp2p/websockets": "10.1.7", + "@libp2p/webtransport": "6.0.16", + "@metamask/kernel-errors": "workspace:^", + "@metamask/kernel-utils": "workspace:^", + "@metamask/logger": "workspace:^", + "@metamask/netlayer": "workspace:^", + "@metamask/superstruct": "^3.2.1", + "@multiformats/multiaddr": "^13.0.1", + "libp2p": "3.1.7", + "uint8arrays": "^5.1.0" + }, + "devDependencies": { + "@arethetypeswrong/cli": "^0.17.4", + "@metamask/auto-changelog": "^5.3.0", + "@metamask/eslint-config": "^15.0.0", + "@metamask/eslint-config-nodejs": "^15.0.0", + "@metamask/eslint-config-typescript": "^15.0.0", + "@ocap/repo-tools": "workspace:^", + "@ts-bridge/cli": "^0.6.3", + "@ts-bridge/shims": "^0.1.1", + "@types/node": "^22.13.1", + "@typescript-eslint/eslint-plugin": "^8.29.0", + "@typescript-eslint/parser": "^8.29.0", + "@typescript-eslint/utils": "^8.29.0", + "@vitest/eslint-plugin": "^1.6.14", + "depcheck": "^1.4.7", + "eslint": "^9.23.0", + "eslint-config-prettier": "^10.1.1", + "eslint-import-resolver-typescript": "^4.3.1", + "eslint-plugin-import-x": "^4.10.0", + "eslint-plugin-jsdoc": "^50.6.9", + "eslint-plugin-n": "^17.17.0", + "eslint-plugin-prettier": "^5.2.6", + "eslint-plugin-promise": "^7.2.1", + "prettier": "^3.5.3", + "rimraf": "^6.0.1", + "ses": "^1.14.0", + "turbo": "^2.9.1", + "typedoc": "^0.28.1", + "typescript": "~5.8.2", + "typescript-eslint": "^8.29.0", + "vite": "^8.0.6", + "vitest": "^4.1.3" + }, + "engines": { + "node": ">=22" + }, + "publishConfig": { + "access": "public", + "registry": "https://registry.npmjs.org/" + } +} diff --git a/packages/netlayer-libp2p/src/config.test.ts b/packages/netlayer-libp2p/src/config.test.ts new file mode 100644 index 0000000000..b78011f047 --- /dev/null +++ b/packages/netlayer-libp2p/src/config.test.ts @@ -0,0 +1,46 @@ +import { is } from '@metamask/superstruct'; +import { describe, it, expect } from 'vitest'; + +import { Libp2pNetlayerConfigStruct } from './config.ts'; + +describe('Libp2pNetlayerConfigStruct', () => { + it('accepts an empty config', () => { + expect(is({}, Libp2pNetlayerConfigStruct)).toBe(true); + }); + + it('accepts a fully-specified config', () => { + const config = { + knownRelays: ['/ip4/127.0.0.1/tcp/9001/ws/p2p/relay'], + maxRetryAttempts: 0, + maxConcurrentConnections: 100, + maxMessageSizeBytes: 1024, + cleanupIntervalMs: 1000, + stalePeerTimeoutMs: 2000, + maxMessagesPerSecond: 50, + maxConnectionAttemptsPerMinute: 10, + reconnectionBaseDelayMs: 500, + reconnectionMaxDelayMs: 10_000, + handshakeTimeoutMs: 10_000, + writeTimeoutMs: 10_000, + streamInactivityTimeoutMs: 120_000, + allowedWsHosts: ['relay.example'], + directListenAddresses: ['/ip4/0.0.0.0/udp/0/quic-v1'], + }; + expect(is(config, Libp2pNetlayerConfigStruct)).toBe(true); + }); + + it.each([ + { name: 'non-integer maxRetryAttempts', config: { maxRetryAttempts: 1.5 } }, + { name: 'negative maxRetryAttempts', config: { maxRetryAttempts: -1 } }, + { + name: 'zero maxConcurrentConnections', + config: { maxConcurrentConnections: 0 }, + }, + { name: 'zero maxMessageSizeBytes', config: { maxMessageSizeBytes: 0 } }, + { name: 'non-string knownRelays', config: { knownRelays: [1, 2] } }, + { name: 'string knownRelays', config: { knownRelays: 'relay' } }, + { name: 'non-array allowedWsHosts', config: { allowedWsHosts: 'host' } }, + ])('rejects a config with $name', ({ config }) => { + expect(is(config, Libp2pNetlayerConfigStruct)).toBe(false); + }); +}); diff --git a/packages/netlayer-libp2p/src/config.ts b/packages/netlayer-libp2p/src/config.ts new file mode 100644 index 0000000000..48c7593d31 --- /dev/null +++ b/packages/netlayer-libp2p/src/config.ts @@ -0,0 +1,40 @@ +import { + array, + integer, + min, + object, + optional, + string, +} from '@metamask/superstruct'; +import type { Infer } from '@metamask/superstruct'; + +/** + * Superstruct schema for the libp2p netlayer's config. Every field is + * JSON-serializable (string/number/array) so the config survives the browser + * `postMessage` boundary as part of a `NetlayerSpecifier`. + * + * These are the per-netlayer options that used to live on the kernel-level + * `RemoteCommsOptions`; the kernel-owned options (`mnemonic`, `maxQueue`, + * `ackTimeoutMs`, `maxUrlRelayHints`, `maxKnownRelays`) stay in ocap-kernel. + */ +export const Libp2pNetlayerConfigStruct = object({ + /** Opaque relay hint strings (the kernel's persisted hint pool). */ + knownRelays: optional(array(string())), + maxRetryAttempts: optional(min(integer(), 0)), + maxConcurrentConnections: optional(min(integer(), 1)), + maxMessageSizeBytes: optional(min(integer(), 1)), + cleanupIntervalMs: optional(min(integer(), 0)), + stalePeerTimeoutMs: optional(min(integer(), 0)), + maxMessagesPerSecond: optional(min(integer(), 1)), + maxConnectionAttemptsPerMinute: optional(min(integer(), 1)), + reconnectionBaseDelayMs: optional(min(integer(), 0)), + reconnectionMaxDelayMs: optional(min(integer(), 0)), + handshakeTimeoutMs: optional(min(integer(), 0)), + writeTimeoutMs: optional(min(integer(), 0)), + streamInactivityTimeoutMs: optional(min(integer(), 0)), + allowedWsHosts: optional(array(string())), + /** Direct (non-relay) listen addresses; consumed by the `./nodejs` factory. */ + directListenAddresses: optional(array(string())), +}); + +export type Libp2pNetlayerConfig = Infer; diff --git a/packages/ocap-kernel/src/remotes/platform/connection-factory.test.ts b/packages/netlayer-libp2p/src/connection-factory.test.ts similarity index 99% rename from packages/ocap-kernel/src/remotes/platform/connection-factory.test.ts rename to packages/netlayer-libp2p/src/connection-factory.test.ts index e0883f6f97..c83d467a1b 100644 --- a/packages/ocap-kernel/src/remotes/platform/connection-factory.test.ts +++ b/packages/netlayer-libp2p/src/connection-factory.test.ts @@ -11,10 +11,9 @@ import { IntentionalDisconnectError, MessageTooLargeError, } from '@metamask/kernel-errors'; +import type { NetworkChannel } from '@metamask/netlayer'; import { describe, expect, it, vi, beforeEach, afterEach } from 'vitest'; -import type { NetworkChannel } from '../types.ts'; - // Mock heavy/libp2p related deps with minimal shims we can assert against. // Track state shared between mocks and tests diff --git a/packages/ocap-kernel/src/remotes/platform/connection-factory.ts b/packages/netlayer-libp2p/src/connection-factory.ts similarity index 91% rename from packages/ocap-kernel/src/remotes/platform/connection-factory.ts rename to packages/netlayer-libp2p/src/connection-factory.ts index d78e867175..d9a16a5ba6 100644 --- a/packages/ocap-kernel/src/remotes/platform/connection-factory.ts +++ b/packages/netlayer-libp2p/src/connection-factory.ts @@ -6,7 +6,6 @@ import { generateKeyPairFromSeed, publicKeyFromRaw } from '@libp2p/crypto/keys'; import { identify } from '@libp2p/identify'; import { MuxerClosedError, - StreamResetError, TooManyOutboundProtocolStreamsError, } from '@libp2p/interface'; import type { @@ -17,21 +16,11 @@ import type { } from '@libp2p/interface'; import { peerIdFromPublicKey } from '@libp2p/peer-id'; import { ping } from '@libp2p/ping'; -import { - InvalidDataLengthError, - InvalidDataLengthLengthError, - lpStream, -} from '@libp2p/utils'; +import { lpStream } from '@libp2p/utils'; import { webRTC } from '@libp2p/webrtc'; import { webSockets } from '@libp2p/websockets'; import { webTransport } from '@libp2p/webtransport'; -import { - AbortError, - ChannelResetError, - IntentionalDisconnectError, - isRetryableNetworkError, - MessageTooLargeError, -} from '@metamask/kernel-errors'; +import { AbortError } from '@metamask/kernel-errors'; import { calculateReconnectionBackoff, fromHex, @@ -57,58 +46,15 @@ import { RELAY_RECONNECT_BASE_DELAY_MS, RELAY_RECONNECT_MAX_DELAY_MS, RELAY_RECONNECT_MAX_ATTEMPTS, - SCTP_USER_INITIATED_ABORT, } from './constants.ts'; -import { getHost, getLastPeerId, isPlainWs } from '../../utils/multiaddr.ts'; -import { isPrivateAddress } from '../../utils/network.ts'; -import type { ConnectionFactoryOptions, DirectTransport } from '../types.ts'; - -/** - * Detect whether a read error indicates an intentional disconnect. Checks the - * legacy SCTP sniffing for a WebRTC user-initiated abort (code 12). The typed - * `StreamResetError` is handled separately (mapped to `ChannelResetError`) so a - * remote reset always reconnects and is never treated as intentional. - * - * @param problem - The error thrown by a stream read. - * @returns Whether the error represents an intentional disconnect. - */ -function isIntentionalDisconnect(problem: unknown): boolean { - const rtcProblem = problem as { - errorDetail?: string; - sctpCauseCode?: number; - }; - return ( - rtcProblem?.errorDetail === 'sctp-failure' && - rtcProblem?.sctpCauseCode === SCTP_USER_INITIATED_ABORT - ); -} - -/** - * Map a raw libp2p stream-read error onto a neutral kernel-error so the channel - * engine never imports libp2p error types. Ordering mirrors the historical - * engine cascade: a `StreamResetError` maps to `ChannelResetError` (reconnect) - * before intentional-disconnect classification, so a remote reset can never be - * swallowed as an intentional close. Anything unrecognised (including - * `UnexpectedEOFError`) passes through unchanged for the engine's else branch. - * - * @param problem - The raw error thrown by the underlying stream read. - * @returns The neutral error to throw, or the original error unchanged. - */ -function mapLibp2pReadError(problem: unknown): unknown { - if ( - problem instanceof InvalidDataLengthError || - problem instanceof InvalidDataLengthLengthError - ) { - return new MessageTooLargeError({ cause: problem }); - } - if (problem instanceof StreamResetError) { - return new ChannelResetError({ cause: problem }); - } - if (isIntentionalDisconnect(problem)) { - return new IntentionalDisconnectError({ cause: problem as Error }); - } - return problem; -} +import { + isRetryableLibp2pError, + mapLibp2pDialError, + mapLibp2pReadError, +} from './error-mapper.ts'; +import type { ConnectionFactoryOptions, DirectTransport } from './types.ts'; +import { getHost, getLastPeerId, isPlainWs } from './utils/multiaddr.ts'; +import { isPrivateAddress } from './utils/network.ts'; /** * Connection factory for libp2p network operations. @@ -630,7 +576,7 @@ export class ConnectionFactory { const retryOptions: Parameters[1] = { maxAttempts: this.#maxRetryAttempts, jitter: true, - shouldRetry: isRetryableNetworkError, + shouldRetry: isRetryableLibp2pError, onRetry: ({ attempt, maxAttempts, delayMs }) => { this.#logger.log( `retrying connection to ${peerId} in ${delayMs}ms (next attempt ${attempt}/${maxAttempts || '∞'})`, @@ -667,7 +613,14 @@ export class ConnectionFactory { withRetry ? this.openChannelWithRetry(libp2pPeerId, hints, peerId) : this.openChannelOnce(libp2pPeerId, hints, peerId) - ).finally(() => this.#inflightDials.delete(peerId)); + ) + .catch((problem) => { + // Map raw libp2p dial errors onto neutral kernel-errors so the + // transport-neutral engine classifies reconnection identically + // without importing any libp2p error types. + throw mapLibp2pDialError(problem); + }) + .finally(() => this.#inflightDials.delete(peerId)); this.#inflightDials.set(peerId, promise); } return promise; diff --git a/packages/ocap-kernel/src/remotes/platform/constants.ts b/packages/netlayer-libp2p/src/constants.ts similarity index 100% rename from packages/ocap-kernel/src/remotes/platform/constants.ts rename to packages/netlayer-libp2p/src/constants.ts diff --git a/packages/netlayer-libp2p/src/error-mapper.test.ts b/packages/netlayer-libp2p/src/error-mapper.test.ts new file mode 100644 index 0000000000..a345d9b91f --- /dev/null +++ b/packages/netlayer-libp2p/src/error-mapper.test.ts @@ -0,0 +1,117 @@ +import { + ChannelResetError, + IntentionalDisconnectError, + MessageTooLargeError, +} from '@metamask/kernel-errors'; +import { describe, it, expect } from 'vitest'; + +import { SCTP_USER_INITIATED_ABORT } from './constants.ts'; +import { + isIntentionalDisconnect, + isRetryableLibp2pError, + mapLibp2pDialError, + mapLibp2pReadError, +} from './error-mapper.ts'; + +const named = (name: string, message = 'boom'): Error => + Object.assign(new Error(message), { name }); + +const coded = (code: string): Error => + Object.assign(new Error('network'), { code }); + +describe('isIntentionalDisconnect', () => { + it('returns true for an SCTP user-initiated abort', () => { + expect( + isIntentionalDisconnect({ + errorDetail: 'sctp-failure', + sctpCauseCode: SCTP_USER_INITIATED_ABORT, + }), + ).toBe(true); + }); + + it.each([ + { name: 'wrong cause code', errorDetail: 'sctp-failure', sctpCauseCode: 1 }, + { name: 'wrong detail', errorDetail: 'other', sctpCauseCode: 12 }, + { name: 'no fields', errorDetail: undefined, sctpCauseCode: undefined }, + ])('returns false for $name', ({ errorDetail, sctpCauseCode }) => { + expect(isIntentionalDisconnect({ errorDetail, sctpCauseCode })).toBe(false); + }); +}); + +describe('mapLibp2pReadError', () => { + it('maps an intentional disconnect to IntentionalDisconnectError', () => { + const problem = { + errorDetail: 'sctp-failure', + sctpCauseCode: SCTP_USER_INITIATED_ABORT, + }; + const mapped = mapLibp2pReadError(problem); + expect(mapped).toBeInstanceOf(IntentionalDisconnectError); + }); + + it('passes through an unrecognised error unchanged', () => { + const problem = named('UnexpectedEOFError'); + expect(mapLibp2pReadError(problem)).toBe(problem); + }); +}); + +describe('isRetryableLibp2pError', () => { + it.each([ + { name: 'MuxerClosedError', error: named('MuxerClosedError') }, + { name: 'DialError', error: named('DialError') }, + { name: 'TransportError', error: named('TransportError') }, + { name: 'WebRTCDialError', error: named('WebRTCDialError') }, + { + name: 'NO_RESERVATION message', + error: new Error('relay status NO_RESERVATION'), + }, + { name: 'ChannelResetError', error: new ChannelResetError() }, + { name: 'ECONNRESET code', error: coded('ECONNRESET') }, + { name: 'ENOTFOUND code', error: coded('ENOTFOUND') }, + ])('returns true for $name', ({ error }) => { + expect(isRetryableLibp2pError(error)).toBe(true); + }); + + it.each([ + { name: 'generic error', error: new Error('nope') }, + { name: 'unknown code', error: coded('EUNKNOWN') }, + { name: 'unknown name', error: named('SomethingElse') }, + ])('returns false for $name', ({ error }) => { + expect(isRetryableLibp2pError(error)).toBe(false); + }); +}); + +describe('mapLibp2pDialError', () => { + it('passes through errors the neutral classifier already handles', () => { + const reset = new ChannelResetError(); + expect(mapLibp2pDialError(reset)).toBe(reset); + const node = coded('ECONNREFUSED'); + expect(mapLibp2pDialError(node)).toBe(node); + }); + + it.each([ + { name: 'MuxerClosedError', error: named('MuxerClosedError') }, + { name: 'DialError', error: named('DialError') }, + { name: 'TransportError', error: named('TransportError') }, + { + name: 'NO_RESERVATION message', + error: new Error('relay status NO_RESERVATION'), + }, + ])( + 'wraps the libp2p-specific retryable $name in a ChannelResetError', + ({ error }) => { + const mapped = mapLibp2pDialError(error); + expect(mapped).toBeInstanceOf(ChannelResetError); + expect((mapped as ChannelResetError).cause).toBe(error); + }, + ); + + it('passes through a non-retryable error unchanged', () => { + const problem = new Error('fatal'); + expect(mapLibp2pDialError(problem)).toBe(problem); + }); + + it('does not wrap a MessageTooLargeError (non-retryable) into a reset', () => { + const problem = new MessageTooLargeError(); + expect(mapLibp2pDialError(problem)).toBe(problem); + }); +}); diff --git a/packages/netlayer-libp2p/src/error-mapper.ts b/packages/netlayer-libp2p/src/error-mapper.ts new file mode 100644 index 0000000000..75feea3598 --- /dev/null +++ b/packages/netlayer-libp2p/src/error-mapper.ts @@ -0,0 +1,135 @@ +import { StreamResetError } from '@libp2p/interface'; +import { + InvalidDataLengthError, + InvalidDataLengthLengthError, +} from '@libp2p/utils'; +import { + ChannelResetError, + IntentionalDisconnectError, + isRetryableNetworkError, + MessageTooLargeError, +} from '@metamask/kernel-errors'; + +import { SCTP_USER_INITIATED_ABORT } from './constants.ts'; + +/** + * Detect whether a read error indicates an intentional disconnect. Checks the + * legacy SCTP sniffing for a WebRTC user-initiated abort (code 12). The typed + * `StreamResetError` is handled separately (mapped to `ChannelResetError`) so a + * remote reset always reconnects and is never treated as intentional. + * + * @param problem - The error thrown by a stream read. + * @returns Whether the error represents an intentional disconnect. + */ +export function isIntentionalDisconnect(problem: unknown): boolean { + const rtcProblem = problem as { + errorDetail?: string; + sctpCauseCode?: number; + }; + return ( + rtcProblem?.errorDetail === 'sctp-failure' && + rtcProblem?.sctpCauseCode === SCTP_USER_INITIATED_ABORT + ); +} + +/** + * Map a raw libp2p stream-read error onto a neutral kernel-error so the channel + * engine never imports libp2p error types. Ordering mirrors the historical + * engine cascade: a `StreamResetError` maps to `ChannelResetError` (reconnect) + * before intentional-disconnect classification, so a remote reset can never be + * swallowed as an intentional close. Anything unrecognised (including + * `UnexpectedEOFError`) passes through unchanged for the engine's else branch. + * + * @param problem - The raw error thrown by the underlying stream read. + * @returns The neutral error to throw, or the original error unchanged. + */ +export function mapLibp2pReadError(problem: unknown): unknown { + if ( + problem instanceof InvalidDataLengthError || + problem instanceof InvalidDataLengthLengthError + ) { + return new MessageTooLargeError({ cause: problem }); + } + if (problem instanceof StreamResetError) { + return new ChannelResetError({ cause: problem }); + } + if (isIntentionalDisconnect(problem)) { + return new IntentionalDisconnectError({ cause: problem as Error }); + } + return problem; +} + +/** + * Classify a raw libp2p error by name/message as a retryable dial-path failure. + * These are the libp2p-specific branches that used to live in kernel-errors' + * `isRetryableNetworkError`; they are matched by string (no libp2p import for + * the check itself) and catch raw libp2p dial errors before they are mapped to + * neutral classes. + * + * @param error - The error to classify. + * @returns Whether the error is a retryable libp2p dial-path failure. + */ +function isRetryableLibp2pName(error: unknown): boolean { + const anyError = error as { name?: string; message?: string }; + + // Yamux muxer teardown mid-dial — retry against other addresses/relays. + if (error instanceof Error && error.name === 'MuxerClosedError') { + return true; + } + + // libp2p dial/transport errors (DialError, TransportError, etc.). + const { name } = anyError; + if ( + typeof name === 'string' && + (name.includes('Dial') || name.includes('Transport')) + ) { + return true; + } + + // Relay reservation errors are temporary and should be retried. + const { message } = anyError; + if (typeof message === 'string' && message.includes('NO_RESERVATION')) { + return true; + } + + return false; +} + +/** + * Decide whether a raw libp2p connection error is retryable. Composes the + * neutral {@link isRetryableNetworkError} (neutral error classes + Node.js + * network error codes) with the libp2p-specific name/message sniffing so the + * provider's own backoff loop keeps its historical behavior. + * + * @param error - The error to classify. + * @returns Whether the error is retryable. + */ +export function isRetryableLibp2pError(error: unknown): boolean { + return isRetryableNetworkError(error) || isRetryableLibp2pName(error); +} + +/** + * Map a raw libp2p dial error onto a neutral kernel-error so the transport- + * neutral channel engine classifies reconnection identically without knowing + * any libp2p error types. + * + * Errors the neutral classifier already recognises (neutral classes and Node.js + * network codes) pass through unchanged, preserving the engine's permanent- + * failure detection (which keys on Node.js codes). libp2p-specific retryable + * errors are wrapped in a neutral, retryable {@link ChannelResetError}; because + * libp2p error names were never members of the permanent-failure code set (and + * neither is `ChannelResetError`), this does not change give-up behavior. + * Everything else passes through so the engine gives up exactly as before. + * + * @param problem - The raw error thrown by the libp2p dial path. + * @returns The neutral error to throw, or the original error unchanged. + */ +export function mapLibp2pDialError(problem: unknown): unknown { + if (isRetryableNetworkError(problem)) { + return problem; + } + if (isRetryableLibp2pName(problem)) { + return new ChannelResetError({ cause: problem as Error }); + } + return problem; +} diff --git a/packages/netlayer-libp2p/src/index.test.ts b/packages/netlayer-libp2p/src/index.test.ts new file mode 100644 index 0000000000..8026e0b616 --- /dev/null +++ b/packages/netlayer-libp2p/src/index.test.ts @@ -0,0 +1,47 @@ +import { describe, it, expect, vi, beforeEach } from 'vitest'; + +import { libp2pNetlayerFactory } from './index.ts'; + +const netlayerSentinel = { peerId: 'z-self' }; +const makeLibp2pNetlayerMock = vi.fn(async () => netlayerSentinel); + +vi.mock('./make-libp2p-netlayer.ts', () => ({ + makeLibp2pNetlayer: async (params: unknown) => makeLibp2pNetlayerMock(params), +})); + +const hooks = { handleMessage: vi.fn() }; + +describe('libp2pNetlayerFactory', () => { + beforeEach(() => { + makeLibp2pNetlayerMock.mockClear().mockResolvedValue(netlayerSentinel); + }); + + it('validates the config and delegates to makeLibp2pNetlayer', async () => { + const result = await libp2pNetlayerFactory({ + keySeed: '0x1234', + incarnationId: 'inc-1', + hooks, + config: { knownRelays: ['/ip4/127.0.0.1/tcp/9001/ws/p2p/relay'] }, + }); + + expect(result).toBe(netlayerSentinel); + expect(makeLibp2pNetlayerMock).toHaveBeenCalledWith({ + keySeed: '0x1234', + incarnationId: 'inc-1', + hooks, + config: { knownRelays: ['/ip4/127.0.0.1/tcp/9001/ws/p2p/relay'] }, + logger: undefined, + }); + }); + + it('rejects an invalid config before constructing a netlayer', async () => { + await expect( + libp2pNetlayerFactory({ + keySeed: '0x1234', + hooks, + config: { maxRetryAttempts: -1 } as never, + }), + ).rejects.toThrow('maxRetryAttempts'); + expect(makeLibp2pNetlayerMock).not.toHaveBeenCalled(); + }); +}); diff --git a/packages/netlayer-libp2p/src/index.ts b/packages/netlayer-libp2p/src/index.ts new file mode 100644 index 0000000000..49ada10472 --- /dev/null +++ b/packages/netlayer-libp2p/src/index.ts @@ -0,0 +1,48 @@ +import type { NetlayerFactory } from '@metamask/netlayer'; +import { assert } from '@metamask/superstruct'; + +import { Libp2pNetlayerConfigStruct } from './config.ts'; +import { makeLibp2pNetlayer } from './make-libp2p-netlayer.ts'; + +/** + * The browser-default libp2p netlayer factory. Validates the config at the + * boundary and constructs a libp2p {@link import('@metamask/netlayer').Netlayer} + * over WebSocket/WebTransport/WebRTC + circuit-relay transports. For Node.js + * direct transports (QUIC/TCP), use the `./nodejs` factory instead. + * + * @param params - The netlayer params. + * @param params.keySeed - Hex-encoded key seed the peer id is derived from. + * @param params.incarnationId - This kernel's incarnation id. + * @param params.hooks - Kernel-supplied callbacks. + * @param params.config - The libp2p netlayer config (validated here). + * @param params.logger - Optional logger. + * @returns The libp2p netlayer. + */ +export const libp2pNetlayerFactory: NetlayerFactory = + async function libp2pNetlayerFactory({ + keySeed, + incarnationId, + hooks, + config, + logger, + }) { + // The registry passes a `Json` config; validate and narrow it here. + assert(config, Libp2pNetlayerConfigStruct); + return makeLibp2pNetlayer({ + keySeed, + incarnationId, + hooks, + config, + logger, + }); + }; +harden(libp2pNetlayerFactory); + +export { Libp2pNetlayerConfigStruct } from './config.ts'; +export type { Libp2pNetlayerConfig } from './config.ts'; +// `makeLibp2pNetlayer` is re-exported for the ocap-kernel compatibility shim, +// which must not reach the `./nodejs` subpath (browser bundling). It accepts a +// pre-built `directTransports` array that the public factory does not expose. +export { makeLibp2pNetlayer } from './make-libp2p-netlayer.ts'; +export type { MakeLibp2pNetlayerParams } from './make-libp2p-netlayer.ts'; +export type { DirectTransport } from './types.ts'; diff --git a/packages/ocap-kernel/src/remotes/platform/lp-framing.test.ts b/packages/netlayer-libp2p/src/lp-framing.test.ts similarity index 100% rename from packages/ocap-kernel/src/remotes/platform/lp-framing.test.ts rename to packages/netlayer-libp2p/src/lp-framing.test.ts diff --git a/packages/ocap-kernel/src/remotes/platform/transport.test.ts b/packages/netlayer-libp2p/src/make-libp2p-netlayer.test.ts similarity index 77% rename from packages/ocap-kernel/src/remotes/platform/transport.test.ts rename to packages/netlayer-libp2p/src/make-libp2p-netlayer.test.ts index 3d62ad3c95..c0b8e06386 100644 --- a/packages/ocap-kernel/src/remotes/platform/transport.test.ts +++ b/packages/netlayer-libp2p/src/make-libp2p-netlayer.test.ts @@ -1,6 +1,6 @@ import { describe, it, expect, vi, beforeEach } from 'vitest'; -import { initTransport } from './transport.ts'; +import { makeLibp2pNetlayer } from './make-libp2p-netlayer.ts'; // A stand-in for the netlayer the engine returns; identity comes from the provider. const netlayerSentinel = { @@ -38,29 +38,31 @@ vi.mock('@metamask/netlayer', async (importOriginal) => ({ makeChannelNetlayer: (params: unknown) => makeChannelNetlayerMock(params), })); -describe('initTransport', () => { +const hooks = { handleMessage: vi.fn() }; + +describe('makeLibp2pNetlayer', () => { beforeEach(() => { makeConnectionFactory.mockClear().mockResolvedValue(mockProvider); makeChannelNetlayerMock.mockClear().mockReturnValue(netlayerSentinel); }); - it('constructs the libp2p ConnectionFactory with the provider options', async () => { + it('constructs the ConnectionFactory with the provider-owned options', async () => { const directTransports = [ { transport: { tag: 'quic' }, listenAddresses: ['/ip4/0.0.0.0/udp/0/quic-v1'], }, ]; - await initTransport( - '0xabcd', - { - relays: ['/dns4/relay.example/tcp/443/wss/p2p/relay1'], + await makeLibp2pNetlayer({ + keySeed: '0xabcd', + hooks, + config: { + knownRelays: ['/dns4/relay.example/tcp/443/wss/p2p/relay1'], maxRetryAttempts: 5, - directTransports, allowedWsHosts: ['relay.example'], }, - vi.fn(), - ); + directTransports, + }); expect(makeConnectionFactory).toHaveBeenCalledWith({ keySeed: '0xabcd', @@ -74,8 +76,8 @@ describe('initTransport', () => { }); }); - it('defaults relays to an empty array and message size to 1MB', async () => { - await initTransport('0x1234', {}, vi.fn()); + it('defaults knownRelays to an empty array and message size to 1MB', async () => { + await makeLibp2pNetlayer({ keySeed: '0x1234', hooks, config: {} }); expect(makeConnectionFactory).toHaveBeenCalledWith( expect.objectContaining({ @@ -88,7 +90,11 @@ describe('initTransport', () => { }); it('honors an explicit maxMessageSizeBytes', async () => { - await initTransport('0x1234', { maxMessageSizeBytes: 42 }, vi.fn()); + await makeLibp2pNetlayer({ + keySeed: '0x1234', + hooks, + config: { maxMessageSizeBytes: 42 }, + }); expect(makeConnectionFactory).toHaveBeenCalledWith( expect.objectContaining({ maxMessageSizeBytes: 42 }), @@ -96,25 +102,20 @@ describe('initTransport', () => { }); it('delegates to makeChannelNetlayer with the mapped engine options and hooks', async () => { - const handleMessage = vi.fn(); - const onRemoteGiveUp = vi.fn(); - const onIncarnationChange = vi.fn(); - const result = await initTransport( - '0x1234', - { + const result = await makeLibp2pNetlayer({ + keySeed: '0x1234', + incarnationId: 'incarnation-x', + hooks, + config: { maxConcurrentConnections: 7, cleanupIntervalMs: 111, streamInactivityTimeoutMs: 222, }, - handleMessage, - onRemoteGiveUp, - 'incarnation-x', - onIncarnationChange, - ); + }); expect(makeChannelNetlayerMock).toHaveBeenCalledWith({ provider: mockProvider, - hooks: { handleMessage, onRemoteGiveUp, onIncarnationChange }, + hooks, options: expect.objectContaining({ maxConcurrentConnections: 7, cleanupIntervalMs: 111, diff --git a/packages/netlayer-libp2p/src/make-libp2p-netlayer.ts b/packages/netlayer-libp2p/src/make-libp2p-netlayer.ts new file mode 100644 index 0000000000..0edf6300bc --- /dev/null +++ b/packages/netlayer-libp2p/src/make-libp2p-netlayer.ts @@ -0,0 +1,104 @@ +import { Logger } from '@metamask/logger'; +import { + DEFAULT_MAX_MESSAGE_SIZE_BYTES, + makeChannelNetlayer, +} from '@metamask/netlayer'; +import type { Netlayer, NetlayerParams } from '@metamask/netlayer'; + +import type { Libp2pNetlayerConfig } from './config.ts'; +import { ConnectionFactory } from './connection-factory.ts'; +import type { DirectTransport } from './types.ts'; + +/** + * Parameters for {@link makeLibp2pNetlayer}: the neutral {@link NetlayerParams} + * for the libp2p config, plus an optional pre-built set of direct transports. + */ +export type MakeLibp2pNetlayerParams = NetlayerParams & { + /** + * Pre-built direct (QUIC/TCP) transports. The `./nodejs` factory builds + * these from `config.directListenAddresses`; the ocap-kernel compatibility + * shim forwards transports built by its own platform layer. Not `Json` and + * never crosses the browser `postMessage` boundary. + * + * @internal + */ + directTransports?: DirectTransport[] | undefined; +}; + +/** + * Construct a libp2p {@link Netlayer}: build the libp2p `ConnectionFactory` + * (the `ChannelProvider`) from the config and key seed, then delegate the + * channel-session engine to `makeChannelNetlayer`, sharing one `Logger` and one + * `AbortController` (so the netlayer's `stop()` aborts the signal the provider + * was constructed with). Provider-owned config (`knownRelays`, `allowedWsHosts`, + * `maxMessageSizeBytes`, `maxRetryAttempts`, `directTransports`) is consumed by + * the factory; the remaining engine options flow to `makeChannelNetlayer`. + * + * @param params - The netlayer parameters. See {@link MakeLibp2pNetlayerParams}. + * @param params.keySeed - Hex-encoded key seed the peer id is derived from. + * @param params.incarnationId - This kernel's incarnation id. + * @param params.hooks - Kernel-supplied callbacks. + * @param params.config - The libp2p netlayer config. + * @param params.logger - Optional logger; a new one is created if omitted. + * @param params.directTransports - Optional pre-built direct transports. + * @returns A promise for the constructed {@link Netlayer}. + */ +export async function makeLibp2pNetlayer({ + keySeed, + incarnationId, + hooks, + config, + logger, + directTransports, +}: MakeLibp2pNetlayerParams): Promise { + const { + knownRelays = [], + maxRetryAttempts, + maxConcurrentConnections, + maxMessageSizeBytes, + cleanupIntervalMs, + stalePeerTimeoutMs, + maxMessagesPerSecond, + maxConnectionAttemptsPerMinute, + reconnectionBaseDelayMs, + reconnectionMaxDelayMs, + handshakeTimeoutMs, + writeTimeoutMs, + streamInactivityTimeoutMs, + allowedWsHosts, + } = config; + const resolvedLogger = logger ?? new Logger(); + const stopController = new AbortController(); + const provider = await ConnectionFactory.make({ + keySeed, + knownRelays, + logger: resolvedLogger, + signal: stopController.signal, + maxRetryAttempts, + maxMessageSizeBytes: maxMessageSizeBytes ?? DEFAULT_MAX_MESSAGE_SIZE_BYTES, + directTransports, + allowedWsHosts, + }); + return makeChannelNetlayer({ + provider, + hooks, + options: { + maxRetryAttempts, + maxConcurrentConnections, + maxMessageSizeBytes, + cleanupIntervalMs, + stalePeerTimeoutMs, + maxMessagesPerSecond, + maxConnectionAttemptsPerMinute, + reconnectionBaseDelayMs, + reconnectionMaxDelayMs, + handshakeTimeoutMs, + writeTimeoutMs, + streamInactivityTimeoutMs, + localIncarnationId: incarnationId, + }, + logger: resolvedLogger, + stopController, + }); +} +harden(makeLibp2pNetlayer); diff --git a/packages/netlayer-libp2p/src/nodejs/direct-transports.test.ts b/packages/netlayer-libp2p/src/nodejs/direct-transports.test.ts new file mode 100644 index 0000000000..c9d790d01f --- /dev/null +++ b/packages/netlayer-libp2p/src/nodejs/direct-transports.test.ts @@ -0,0 +1,57 @@ +import { describe, it, expect, vi } from 'vitest'; + +import { buildDirectTransports } from './direct-transports.ts'; + +vi.mock('@chainsafe/libp2p-quic', () => ({ + quic: () => ({ tag: 'quic' }), +})); +vi.mock('@libp2p/tcp', () => ({ + tcp: () => ({ tag: 'tcp' }), +})); + +describe('buildDirectTransports', () => { + it('returns an empty array when given no addresses', () => { + expect(buildDirectTransports([])).toStrictEqual([]); + }); + + it('builds a QUIC transport for a /quic-v1 address', () => { + expect(buildDirectTransports(['/ip4/0.0.0.0/udp/0/quic-v1'])).toStrictEqual( + [ + { + transport: { tag: 'quic' }, + listenAddresses: ['/ip4/0.0.0.0/udp/0/quic-v1'], + }, + ], + ); + }); + + it('builds a TCP transport for a /tcp/ address', () => { + expect(buildDirectTransports(['/ip4/0.0.0.0/tcp/4001'])).toStrictEqual([ + { transport: { tag: 'tcp' }, listenAddresses: ['/ip4/0.0.0.0/tcp/4001'] }, + ]); + }); + + it('groups QUIC and TCP addresses into separate transports', () => { + const result = buildDirectTransports([ + '/ip4/0.0.0.0/udp/0/quic-v1', + '/ip4/0.0.0.0/udp/1/quic-v1', + '/ip4/0.0.0.0/tcp/4001', + ]); + expect(result).toStrictEqual([ + { + transport: { tag: 'quic' }, + listenAddresses: [ + '/ip4/0.0.0.0/udp/0/quic-v1', + '/ip4/0.0.0.0/udp/1/quic-v1', + ], + }, + { transport: { tag: 'tcp' }, listenAddresses: ['/ip4/0.0.0.0/tcp/4001'] }, + ]); + }); + + it('throws for an unsupported direct listen address', () => { + expect(() => buildDirectTransports(['/ip4/0.0.0.0/ws'])).toThrow( + 'Unsupported direct listen address', + ); + }); +}); diff --git a/packages/netlayer-libp2p/src/nodejs/direct-transports.ts b/packages/netlayer-libp2p/src/nodejs/direct-transports.ts new file mode 100644 index 0000000000..01fd279ce5 --- /dev/null +++ b/packages/netlayer-libp2p/src/nodejs/direct-transports.ts @@ -0,0 +1,60 @@ +import { quic } from '@chainsafe/libp2p-quic'; +import { tcp } from '@libp2p/tcp'; + +import type { DirectTransport } from '../types.ts'; + +/** + * Build direct (non-relay) libp2p transports from a list of listen addresses, + * sniffing the required transport from each address string: `/quic-v1` → + * `quic()`, `/tcp/` → `tcp()`. Callers never import transport packages. + * + * @param directListenAddresses - Listen address strings (e.g. + * `/ip4/0.0.0.0/udp/0/quic-v1`, `/ip4/0.0.0.0/tcp/4001`). + * @returns The direct transports bundled with their listen addresses. + * @throws If an address is neither a QUIC nor TCP address. + */ +export function buildDirectTransports( + directListenAddresses: string[], +): DirectTransport[] { + const directTransports: DirectTransport[] = []; + + if (directListenAddresses.length === 0) { + return directTransports; + } + + const quicAddresses: string[] = []; + const tcpAddresses: string[] = []; + + for (const addr of directListenAddresses) { + const isQuic = addr.includes('/quic-v1'); + const isTcp = addr.includes('/tcp/'); + + if (isQuic) { + quicAddresses.push(addr); + } else if (isTcp) { + tcpAddresses.push(addr); + } else { + throw new Error( + `Unsupported direct listen address: ${addr}. ` + + `Only QUIC (/quic-v1) and TCP (/tcp/) addresses are supported.`, + ); + } + } + + if (quicAddresses.length > 0) { + directTransports.push({ + transport: quic(), + listenAddresses: quicAddresses, + }); + } + + if (tcpAddresses.length > 0) { + directTransports.push({ + transport: tcp(), + listenAddresses: tcpAddresses, + }); + } + + return directTransports; +} +harden(buildDirectTransports); diff --git a/packages/netlayer-libp2p/src/nodejs/index.test.ts b/packages/netlayer-libp2p/src/nodejs/index.test.ts new file mode 100644 index 0000000000..88e9e2a563 --- /dev/null +++ b/packages/netlayer-libp2p/src/nodejs/index.test.ts @@ -0,0 +1,63 @@ +import { describe, it, expect, vi, beforeEach } from 'vitest'; + +import { nodejsLibp2pNetlayerFactory } from './index.ts'; + +const netlayerSentinel = { peerId: 'z-self' }; +const makeLibp2pNetlayerMock = vi.fn(async () => netlayerSentinel); + +vi.mock('../make-libp2p-netlayer.ts', () => ({ + makeLibp2pNetlayer: async (params: unknown) => makeLibp2pNetlayerMock(params), +})); + +vi.mock('./direct-transports.ts', () => ({ + buildDirectTransports: (addrs: string[]) => + addrs.map((addr) => ({ transport: { addr }, listenAddresses: [addr] })), +})); + +const hooks = { handleMessage: vi.fn() }; + +describe('nodejsLibp2pNetlayerFactory', () => { + beforeEach(() => { + makeLibp2pNetlayerMock.mockClear().mockResolvedValue(netlayerSentinel); + }); + + it('builds direct transports from directListenAddresses and delegates', async () => { + const result = await nodejsLibp2pNetlayerFactory({ + keySeed: '0x1234', + hooks, + config: { directListenAddresses: ['/ip4/0.0.0.0/udp/0/quic-v1'] }, + }); + + expect(result).toBe(netlayerSentinel); + expect(makeLibp2pNetlayerMock).toHaveBeenCalledWith( + expect.objectContaining({ + keySeed: '0x1234', + directTransports: [ + { + transport: { addr: '/ip4/0.0.0.0/udp/0/quic-v1' }, + listenAddresses: ['/ip4/0.0.0.0/udp/0/quic-v1'], + }, + ], + }), + ); + }); + + it('defaults to no direct transports when none are configured', async () => { + await nodejsLibp2pNetlayerFactory({ keySeed: '0x1234', hooks, config: {} }); + + expect(makeLibp2pNetlayerMock).toHaveBeenCalledWith( + expect.objectContaining({ directTransports: [] }), + ); + }); + + it('rejects an invalid config', async () => { + await expect( + nodejsLibp2pNetlayerFactory({ + keySeed: '0x1234', + hooks, + config: { maxMessageSizeBytes: 0 } as never, + }), + ).rejects.toThrow('maxMessageSizeBytes'); + expect(makeLibp2pNetlayerMock).not.toHaveBeenCalled(); + }); +}); diff --git a/packages/netlayer-libp2p/src/nodejs/index.ts b/packages/netlayer-libp2p/src/nodejs/index.ts new file mode 100644 index 0000000000..b858441ca7 --- /dev/null +++ b/packages/netlayer-libp2p/src/nodejs/index.ts @@ -0,0 +1,46 @@ +import type { NetlayerFactory } from '@metamask/netlayer'; +import { assert } from '@metamask/superstruct'; + +import { Libp2pNetlayerConfigStruct } from '../config.ts'; +import { makeLibp2pNetlayer } from '../make-libp2p-netlayer.ts'; +import { buildDirectTransports } from './direct-transports.ts'; + +/** + * The Node.js libp2p netlayer factory. Validates the config at the boundary, + * builds any direct (QUIC/TCP) transports from `config.directListenAddresses`, + * and constructs a libp2p netlayer. Owns direct-transport injection so callers + * never import transport packages. + * + * @param params - The netlayer params. + * @param params.keySeed - Hex-encoded key seed the peer id is derived from. + * @param params.incarnationId - This kernel's incarnation id. + * @param params.hooks - Kernel-supplied callbacks. + * @param params.config - The libp2p netlayer config (validated here). + * @param params.logger - Optional logger. + * @returns The libp2p netlayer with direct transports. + */ +export const nodejsLibp2pNetlayerFactory: NetlayerFactory = + async function nodejsLibp2pNetlayerFactory({ + keySeed, + incarnationId, + hooks, + config, + logger, + }) { + // The registry passes a `Json` config; validate and narrow it here. + assert(config, Libp2pNetlayerConfigStruct); + const directTransports = buildDirectTransports( + config.directListenAddresses ?? [], + ); + return makeLibp2pNetlayer({ + keySeed, + incarnationId, + hooks, + config, + logger, + directTransports, + }); + }; +harden(nodejsLibp2pNetlayerFactory); + +export { buildDirectTransports } from './direct-transports.ts'; diff --git a/packages/netlayer-libp2p/src/relay/index.test.ts b/packages/netlayer-libp2p/src/relay/index.test.ts new file mode 100644 index 0000000000..06dbfc7dc9 --- /dev/null +++ b/packages/netlayer-libp2p/src/relay/index.test.ts @@ -0,0 +1,125 @@ +import { describe, it, expect, vi, beforeEach } from 'vitest'; + +import { startRelay } from './index.ts'; + +const createLibp2pMock = vi.fn(); + +vi.mock('@chainsafe/libp2p-noise', () => ({ noise: () => ({}) })); +vi.mock('@chainsafe/libp2p-yamux', () => ({ yamux: () => ({}) })); +vi.mock('@libp2p/autonat', () => ({ autoNAT: () => ({}) })); +vi.mock('@libp2p/circuit-relay-v2', () => ({ circuitRelayServer: () => ({}) })); +vi.mock('@libp2p/crypto/keys', () => ({ + generateKeyPairFromSeed: async () => ({ type: 'Ed25519' }), +})); +vi.mock('@libp2p/identify', () => ({ identify: () => ({}) })); +vi.mock('@libp2p/ping', () => ({ ping: () => ({}) })); +vi.mock('@libp2p/tcp', () => ({ tcp: () => ({}) })); +vi.mock('@libp2p/websockets', () => ({ webSockets: () => ({}) })); +vi.mock('libp2p', () => ({ + createLibp2p: (config: unknown) => createLibp2pMock(config), +})); + +type Handler = (evt: { detail: unknown }) => void; + +const makeFakeLibp2p = (): { + status: string; + peerId: { toString: () => string }; + getMultiaddrs: () => { toString: () => string }[]; + addEventListener: ReturnType; + handlers: Record; +} => { + const handlers: Record = {}; + return { + status: 'started', + peerId: { toString: () => '12D3KooWRelay' }, + getMultiaddrs: () => [{ toString: () => '/ip4/0.0.0.0/tcp/9001/ws' }], + addEventListener: vi.fn((event: string, handler: Handler) => { + handlers[event] = handler; + }), + handlers, + }; +}; + +const makeConnection = (peerId: string, addr: string): { detail: unknown } => ({ + detail: { + remotePeer: { toString: () => peerId }, + remoteAddr: { toString: () => addr }, + }, +}); + +describe('startRelay', () => { + beforeEach(() => { + createLibp2pMock.mockReset(); + }); + + it('creates and returns a started libp2p relay node', async () => { + const fake = makeFakeLibp2p(); + createLibp2pMock.mockReturnValue(fake); + const logger = { log: vi.fn() }; + + const result = await startRelay(logger); + + expect(result).toBe(fake); + expect(createLibp2pMock).toHaveBeenCalledTimes(1); + const [config] = createLibp2pMock.mock.calls[0] as [ + { addresses: { listen: string[]; appendAnnounce: string[] } }, + ]; + expect(config.addresses.listen).toStrictEqual([ + '/ip4/0.0.0.0/tcp/9001/ws', + '/ip4/0.0.0.0/tcp/9002', + ]); + expect(config.addresses.appendAnnounce).toStrictEqual([]); + expect(logger.log).toHaveBeenCalledWith('PeerID: ', '12D3KooWRelay'); + }); + + it('logs connection open and close events with terse peer labels', async () => { + const fake = makeFakeLibp2p(); + createLibp2pMock.mockReturnValue(fake); + const logger = { log: vi.fn() }; + await startRelay(logger); + + const open = fake.handlers['connection:open']; + const close = fake.handlers['connection:close']; + const conn = makeConnection( + '12D3KooWpeerA', + '/ip4/1.2.3.4/tcp/9001/ws/p2p/12D3KooWpeerA', + ); + // Fire open twice to exercise the terse-peer cache, then close. + open?.(conn); + open?.(conn); + close?.(conn); + + // First sighting assigns and logs a terse label for the peer. + expect(logger.log).toHaveBeenCalledWith('[PEER] = 12D3KooWpeerA'); + expect(logger.log).toHaveBeenCalledWith( + '[CONNECTION] Closed connection with ', + ); + }); + + it('announces the public IP addresses when provided', async () => { + createLibp2pMock.mockReturnValue(makeFakeLibp2p()); + const logger = { log: vi.fn() }; + + await startRelay(logger, { publicIp: '203.0.113.7' }); + + const [config] = createLibp2pMock.mock.calls[0] as [ + { addresses: { appendAnnounce: string[] } }, + ]; + expect(config.addresses.appendAnnounce).toStrictEqual([ + '/ip4/203.0.113.7/tcp/9001/ws', + '/ip4/203.0.113.7/tcp/9002', + ]); + }); + + it('waits for the start event when the node is not yet started', async () => { + const fake = makeFakeLibp2p(); + fake.status = 'starting'; + createLibp2pMock.mockReturnValue(fake); + + const promise = startRelay({ log: vi.fn() }); + // Let startRelay progress to registering its 'start' listener, then fire it. + await new Promise((resolve) => setImmediate(resolve)); + fake.handlers.start?.({ detail: undefined }); + expect(await promise).toBe(fake); + }); +}); diff --git a/packages/kernel-utils/src/libp2p-relay.ts b/packages/netlayer-libp2p/src/relay/index.ts similarity index 100% rename from packages/kernel-utils/src/libp2p-relay.ts rename to packages/netlayer-libp2p/src/relay/index.ts diff --git a/packages/netlayer-libp2p/src/types.ts b/packages/netlayer-libp2p/src/types.ts new file mode 100644 index 0000000000..edad530acb --- /dev/null +++ b/packages/netlayer-libp2p/src/types.ts @@ -0,0 +1,32 @@ +import type { Logger } from '@metamask/logger'; + +/** + * A direct (non-relay) libp2p transport implementation bundled with the listen + * addresses it serves. Built by the `./nodejs` factory from a config's + * `directListenAddresses`; never crosses a `Json` boundary. + */ +export type DirectTransport = { + transport: unknown; + listenAddresses: string[]; +}; + +/** + * Options for creating a {@link ConnectionFactory} instance. + */ +export type ConnectionFactoryOptions = { + keySeed: string; + knownRelays: string[]; + logger: Logger; + signal: AbortSignal; + maxRetryAttempts?: number | undefined; + /** + * Maximum inbound message payload size in bytes. Used as `maxDataLength` + * on every `lpStream` constructed for a channel — must match the + * sender-side validator's limit (`maxMessageSizeBytes` on the netlayer + * config) so that a deployment which raises one also raises the other. + * Defaults to `DEFAULT_MAX_MESSAGE_SIZE_BYTES` (1 MB). + */ + maxMessageSizeBytes?: number | undefined; + directTransports?: DirectTransport[] | undefined; + allowedWsHosts?: string[] | undefined; +}; diff --git a/packages/ocap-kernel/src/utils/multiaddr.test.ts b/packages/netlayer-libp2p/src/utils/multiaddr.test.ts similarity index 100% rename from packages/ocap-kernel/src/utils/multiaddr.test.ts rename to packages/netlayer-libp2p/src/utils/multiaddr.test.ts diff --git a/packages/ocap-kernel/src/utils/multiaddr.ts b/packages/netlayer-libp2p/src/utils/multiaddr.ts similarity index 100% rename from packages/ocap-kernel/src/utils/multiaddr.ts rename to packages/netlayer-libp2p/src/utils/multiaddr.ts diff --git a/packages/ocap-kernel/src/utils/network.test.ts b/packages/netlayer-libp2p/src/utils/network.test.ts similarity index 100% rename from packages/ocap-kernel/src/utils/network.test.ts rename to packages/netlayer-libp2p/src/utils/network.test.ts diff --git a/packages/ocap-kernel/src/utils/network.ts b/packages/netlayer-libp2p/src/utils/network.ts similarity index 100% rename from packages/ocap-kernel/src/utils/network.ts rename to packages/netlayer-libp2p/src/utils/network.ts diff --git a/packages/netlayer-libp2p/tsconfig.build.json b/packages/netlayer-libp2p/tsconfig.build.json new file mode 100644 index 0000000000..fbde605c20 --- /dev/null +++ b/packages/netlayer-libp2p/tsconfig.build.json @@ -0,0 +1,18 @@ +{ + "extends": "../../tsconfig.packages.build.json", + "compilerOptions": { + "baseUrl": "./", + "lib": ["ES2022", "DOM"], + "outDir": "./dist", + "rootDir": "./src", + "types": ["ses", "node"] + }, + "references": [ + { "path": "../kernel-errors/tsconfig.build.json" }, + { "path": "../kernel-utils/tsconfig.build.json" }, + { "path": "../logger/tsconfig.build.json" }, + { "path": "../netlayer/tsconfig.build.json" } + ], + "files": [], + "include": ["./src"] +} diff --git a/packages/netlayer-libp2p/tsconfig.json b/packages/netlayer-libp2p/tsconfig.json new file mode 100644 index 0000000000..ed233830b2 --- /dev/null +++ b/packages/netlayer-libp2p/tsconfig.json @@ -0,0 +1,21 @@ +{ + "extends": "../../tsconfig.packages.json", + "compilerOptions": { + "baseUrl": "./", + "lib": ["ES2022", "DOM"], + "types": ["vitest", "ses", "node"] + }, + "references": [ + { "path": "../kernel-errors" }, + { "path": "../kernel-utils" }, + { "path": "../logger" }, + { "path": "../netlayer" }, + { "path": "../repo-tools" } + ], + "include": [ + "../../vitest.config.ts", + "./src", + "./vite.config.ts", + "./vitest.config.ts" + ] +} diff --git a/packages/netlayer-libp2p/typedoc.json b/packages/netlayer-libp2p/typedoc.json new file mode 100644 index 0000000000..f8eb78ae1a --- /dev/null +++ b/packages/netlayer-libp2p/typedoc.json @@ -0,0 +1,8 @@ +{ + "entryPoints": [], + "excludePrivate": true, + "hideGenerator": true, + "out": "docs", + "tsconfig": "./tsconfig.build.json", + "projectDocuments": ["documents/*.md"] +} diff --git a/packages/netlayer-libp2p/vitest.config.ts b/packages/netlayer-libp2p/vitest.config.ts new file mode 100644 index 0000000000..743bf9372d --- /dev/null +++ b/packages/netlayer-libp2p/vitest.config.ts @@ -0,0 +1,22 @@ +import { mergeConfig } from '@ocap/repo-tools/vitest-config'; +import { fileURLToPath } from 'node:url'; +import { defineConfig, defineProject } from 'vitest/config'; + +import defaultConfig from '../../vitest.config.ts'; + +export default defineConfig((args) => { + return mergeConfig( + args, + defaultConfig, + defineProject({ + test: { + name: 'netlayer-libp2p', + setupFiles: [ + fileURLToPath( + import.meta.resolve('@ocap/repo-tools/test-utils/mock-endoify'), + ), + ], + }, + }), + ); +}); diff --git a/packages/ocap-kernel/CHANGELOG.md b/packages/ocap-kernel/CHANGELOG.md index 6ec5bc7254..a4d71367ff 100644 --- a/packages/ocap-kernel/CHANGELOG.md +++ b/packages/ocap-kernel/CHANGELOG.md @@ -26,6 +26,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed +- **BREAKING:** Flip remote comms to injected netlayers ([#973](https://github.com/MetaMask/ocap-kernel/pull/973)) + - `PlatformServices.initializeRemoteComms` now takes an options bag `{ keySeed, specifier, hooks, incarnationId? }` where `specifier` is a `NetlayerSpecifier` (`{ netlayer: string; config: Json }`); the kernel no longer hard-wires libp2p + - `RemoteCommsOptions` shrinks to the kernel-owned fields (`specifier`, `mnemonic`, `maxQueue`, `ackTimeoutMs`, `maxUrlRelayHints`, `maxKnownRelays`); all per-netlayer options move into `specifier.config`. `DirectTransport` and `ConnectionFactoryOptions` are removed + - Remove the `initTransport` export and the libp2p `ConnectionFactory`/multiaddr utilities (now in `@metamask/netlayer-libp2p`); `packages/ocap-kernel/package.json` no longer has any `@libp2p`/`@chainsafe`/`@multiformats`/`libp2p` dependency (plain `multiformats` stays for the neutral base58btc encoding) - Extract the transport-neutral netlayer machinery and contract into the new `@metamask/netlayer` package; ocap-kernel now depends on it and re-exports the contract types from `remotes/types.ts` and the public API. The libp2p `ConnectionFactory` (now exposing `peerId`) stays in ocap-kernel and `initTransport` (public signature unchanged) delegates to `@metamask/netlayer`'s `makeChannelNetlayer`. Adds `@metamask/netlayer-loopback` as a test-only dependency for the loopback-backed `PlatformServices` demonstration ([#972](https://github.com/MetaMask/ocap-kernel/pull/972)) - Newly exported types: `NetworkChannel`, `ChannelProvider`, `Netlayer`, `NetlayerHooks`, `NetlayerFactory`, `NetlayerParams`, `NetlayerSpecifier`, `NetlayerRegistry` - The neutral identity helpers `deriveNeutralPeerId`/`neutralPeerIdToPublicKey`/`publicKeyToNeutralPeerId` are now re-exported from `@metamask/netlayer` (import path for consumers is unchanged) diff --git a/packages/ocap-kernel/package.json b/packages/ocap-kernel/package.json index ed4b71f870..9bafd3f665 100644 --- a/packages/ocap-kernel/package.json +++ b/packages/ocap-kernel/package.json @@ -69,24 +69,11 @@ }, "dependencies": { "@agoric/swingset-liveslots": "0.10.3-u21.0.1", - "@chainsafe/libp2p-noise": "^17.0.0", - "@chainsafe/libp2p-yamux": "8.0.1", "@endo/errors": "^1.2.13", "@endo/eventual-send": "^1.3.4", "@endo/marshal": "^1.8.0", "@endo/pass-style": "^1.6.3", "@endo/promise-kit": "^1.1.13", - "@libp2p/bootstrap": "12.0.15", - "@libp2p/circuit-relay-v2": "4.1.7", - "@libp2p/crypto": "5.1.14", - "@libp2p/identify": "4.0.14", - "@libp2p/interface": "3.1.1", - "@libp2p/peer-id": "6.0.5", - "@libp2p/ping": "3.0.14", - "@libp2p/utils": "7.0.14", - "@libp2p/webrtc": "6.0.15", - "@libp2p/websockets": "10.1.7", - "@libp2p/webtransport": "6.0.16", "@metamask/kernel-errors": "workspace:^", "@metamask/kernel-platforms": "workspace:^", "@metamask/kernel-rpc-methods": "workspace:^", @@ -99,13 +86,10 @@ "@metamask/streams": "workspace:^", "@metamask/superstruct": "^3.2.1", "@metamask/utils": "^11.9.0", - "@multiformats/multiaddr": "^13.0.1", "@scure/bip39": "^2.0.1", - "libp2p": "3.1.7", "multiformats": "^13.3.6", "ses": "^1.14.0", - "setimmediate": "^1.0.5", - "uint8arrays": "^5.1.0" + "setimmediate": "^1.0.5" }, "devDependencies": { "@metamask/auto-changelog": "^5.3.0", diff --git a/packages/ocap-kernel/src/index.test.ts b/packages/ocap-kernel/src/index.test.ts index 7ab04af3c0..486195b9b7 100644 --- a/packages/ocap-kernel/src/index.test.ts +++ b/packages/ocap-kernel/src/index.test.ts @@ -18,7 +18,6 @@ describe('index', () => { 'createDefaultEndowments', 'deriveNeutralPeerId', 'generateMnemonic', - 'initTransport', 'insistKRef', 'insistSubclusterId', 'isKRef', diff --git a/packages/ocap-kernel/src/index.ts b/packages/ocap-kernel/src/index.ts index af14e6a76f..3157d03962 100644 --- a/packages/ocap-kernel/src/index.ts +++ b/packages/ocap-kernel/src/index.ts @@ -7,7 +7,6 @@ export type { MakeAllowedGlobals, VatEndowments, } from './vats/endowments.ts'; -export { initTransport } from './remotes/platform/transport.ts'; export type { IOChannel, IOChannelFactory } from './io/types.ts'; export type { Baggage, @@ -28,7 +27,6 @@ export type { SystemSubclusterConfig, } from './types.ts'; export type { - DirectTransport, RemoteIdentity, RemoteMessageHandler, SendRemoteMessage, diff --git a/packages/ocap-kernel/src/remotes/kernel/remote-comms.test.ts b/packages/ocap-kernel/src/remotes/kernel/remote-comms.test.ts index f586c536ba..217776864d 100644 --- a/packages/ocap-kernel/src/remotes/kernel/remote-comms.test.ts +++ b/packages/ocap-kernel/src/remotes/kernel/remote-comms.test.ts @@ -68,7 +68,12 @@ describe('remote-comms', () => { mockKernelStore, mockPlatformServices, mockRemoteMessageHandler, - { relays: testRelays }, + { + specifier: { + netlayer: 'libp2p', + config: { knownRelays: testRelays }, + }, + }, ); expect(remoteComms).toHaveProperty('getPeerId'); expect(remoteComms).toHaveProperty('issueOcapURL'); @@ -145,29 +150,33 @@ describe('remote-comms', () => { ); }); - it('passes options object to platformServices.initializeRemoteComms', async () => { - const options = { - relays: ['/dns4/relay.example/tcp/443/wss/p2p/relay'], - maxRetryAttempts: 5, - maxQueue: 100, - }; + it('passes an options bag with the specifier to platformServices.initializeRemoteComms', async () => { + const knownRelays = ['/dns4/relay.example/tcp/443/wss/p2p/relay']; await initRemoteComms( mockKernelStore, mockPlatformServices, mockRemoteMessageHandler, - options, + { + specifier: { + netlayer: 'libp2p', + config: { knownRelays, maxRetryAttempts: 5 }, + }, + }, ); expect(mockPlatformServices.initializeRemoteComms).toHaveBeenCalledWith( - expect.any(String), // keySeed expect.objectContaining({ - relays: options.relays, - maxRetryAttempts: options.maxRetryAttempts, - maxQueue: options.maxQueue, + keySeed: expect.any(String), + specifier: { + netlayer: 'libp2p', + config: expect.objectContaining({ + knownRelays, + maxRetryAttempts: 5, + }), + }, + hooks: expect.objectContaining({ + handleMessage: mockRemoteMessageHandler, + }), }), - mockRemoteMessageHandler, - undefined, // onRemoteGiveUp - undefined, // incarnationId - undefined, // onIncarnationChange ); }); @@ -184,14 +193,12 @@ describe('remote-comms', () => { {}, // empty options ); expect(mockPlatformServices.initializeRemoteComms).toHaveBeenCalledWith( - expect.any(String), expect.objectContaining({ - relays: storedRelays, + specifier: { + netlayer: 'libp2p', + config: expect.objectContaining({ knownRelays: storedRelays }), + }, }), - mockRemoteMessageHandler, - undefined, - undefined, // incarnationId - undefined, // onIncarnationChange ); }); @@ -207,12 +214,9 @@ describe('remote-comms', () => { onRemoteGiveUp, ); expect(mockPlatformServices.initializeRemoteComms).toHaveBeenCalledWith( - expect.any(String), - expect.any(Object), - mockRemoteMessageHandler, - onRemoteGiveUp, - undefined, // incarnationId - undefined, // onIncarnationChange + expect.objectContaining({ + hooks: expect.objectContaining({ onRemoteGiveUp }), + }), ); }); @@ -229,12 +233,7 @@ describe('remote-comms', () => { incarnationId, ); expect(mockPlatformServices.initializeRemoteComms).toHaveBeenCalledWith( - expect.any(String), - expect.any(Object), - mockRemoteMessageHandler, - undefined, - incarnationId, - undefined, // onIncarnationChange + expect.objectContaining({ incarnationId }), ); }); @@ -252,12 +251,9 @@ describe('remote-comms', () => { onIncarnationChange, ); expect(mockPlatformServices.initializeRemoteComms).toHaveBeenCalledWith( - expect.any(String), - expect.any(Object), - mockRemoteMessageHandler, - undefined, - undefined, // incarnationId - onIncarnationChange, + expect.objectContaining({ + hooks: expect.objectContaining({ onIncarnationChange }), + }), ); }); @@ -330,7 +326,12 @@ describe('remote-comms', () => { mockKernelStore, mockPlatformServices, mockRemoteMessageHandler, - { relays: testRelays }, + { + specifier: { + netlayer: 'libp2p', + config: { knownRelays: testRelays }, + }, + }, mockLogger as unknown as Logger, ); expect(mockLogger.log).toHaveBeenCalledWith( @@ -347,7 +348,12 @@ describe('remote-comms', () => { mockKernelStore, mockPlatformServices, mockRemoteMessageHandler, - { relays: testRelays }, + { + specifier: { + netlayer: 'libp2p', + config: { knownRelays: testRelays }, + }, + }, ); expect(mockKernelStore.getKnownRelayAddresses()).toStrictEqual( testRelays, @@ -1055,7 +1061,12 @@ describe('remote-comms', () => { mockKernelStore, mockPlatformServices, mockRemoteMessageHandler, - { relays: testRelays }, + { + specifier: { + netlayer: 'libp2p', + config: { knownRelays: testRelays }, + }, + }, ); const ocapURL = await remoteComms.issueOcapURL('ko42' as KRef); diff --git a/packages/ocap-kernel/src/remotes/kernel/remote-comms.ts b/packages/ocap-kernel/src/remotes/kernel/remote-comms.ts index 6ad3f0e741..153876b74c 100644 --- a/packages/ocap-kernel/src/remotes/kernel/remote-comms.ts +++ b/packages/ocap-kernel/src/remotes/kernel/remote-comms.ts @@ -1,6 +1,7 @@ import { toHex, fromHex } from '@metamask/kernel-utils'; import type { Logger } from '@metamask/logger'; import { deriveNeutralPeerId } from '@metamask/netlayer'; +import type { Json } from '@metamask/utils'; import { base58btc } from 'multiformats/bases/base58'; import type { KernelStore, RelayEntry } from '../../store/index.ts'; @@ -22,6 +23,13 @@ export type OcapURLParts = { hints: string[]; }; +/** + * The netlayer selected when a caller omits a specifier. The kernel treats a + * missing specifier as "use libp2p" during the transition to explicit netlayer + * selection. + */ +export const DEFAULT_NETLAYER = 'libp2p'; + /** * Default maximum number of relay hints embedded in a single OCAP URL. * 3 balances URL length against connectivity resilience — most relay @@ -439,39 +447,54 @@ export async function initRemoteComms( incarnationId?: string, onIncarnationChange?: OnIncarnationChange, ): Promise { - const { - relays = [], - mnemonic, - maxUrlRelayHints, - maxKnownRelays, - ...platformOptions - } = options; + const { specifier, mnemonic, maxUrlRelayHints, maxKnownRelays } = options; + const netlayer = specifier?.netlayer ?? DEFAULT_NETLAYER; + const specifierConfig = specifier?.config; + const config: Record = + specifierConfig !== null && + typeof specifierConfig === 'object' && + !Array.isArray(specifierConfig) + ? { ...specifierConfig } + : {}; + + // By convention the kernel understands exactly one netlayer-config key — + // `knownRelays: string[]` — which it treats as the opaque hint pool it + // persists and re-injects. Bootstrap hints the caller supplied travel there. + const bootstrapHints = Array.isArray(config.knownRelays) + ? config.knownRelays.filter( + (hint): hint is string => typeof hint === 'string', + ) + : []; const result = await initRemoteIdentity( kernelStore, - { relays, mnemonic, maxUrlRelayHints, maxKnownRelays }, + { relays: bootstrapHints, mnemonic, maxUrlRelayHints, maxKnownRelays }, logger, keySeed, ); const { identity, knownRelays } = result; + // Overwrite with the full persisted pool the store now owns. + config.knownRelays = knownRelays; + logger?.log(`relays: ${JSON.stringify(knownRelays)}`); // detectWake() reads and updates lastActiveTime atomically, so it must be // called before initializeRemoteComms to capture the pre-restart timestamp. const wakeDetected = kernelStore.detectWake(); - // Omit mnemonic (sensitive key material) from the options passed to - // platform services, which only needs network-level configuration. - await platformServices.initializeRemoteComms( - result.keySeed, - { ...platformOptions, relays: knownRelays }, - remoteMessageHandler, - onRemoteGiveUp, - incarnationId, - onIncarnationChange, - ); + // `mnemonic` (sensitive key material) is never placed in `specifier.config`. + await platformServices.initializeRemoteComms({ + keySeed: result.keySeed, + specifier: { netlayer, config }, + hooks: { + handleMessage: remoteMessageHandler, + ...(onRemoteGiveUp && { onRemoteGiveUp }), + ...(onIncarnationChange && { onIncarnationChange }), + }, + ...(incarnationId !== undefined && { incarnationId }), + }); if (wakeDetected) { logger?.log('Cross-incarnation wake detected, resetting backoffs'); diff --git a/packages/ocap-kernel/src/remotes/loopback.test.ts b/packages/ocap-kernel/src/remotes/loopback.test.ts index 68c45270f2..6f7cbcc6dd 100644 --- a/packages/ocap-kernel/src/remotes/loopback.test.ts +++ b/packages/ocap-kernel/src/remotes/loopback.test.ts @@ -20,26 +20,28 @@ describe('loopback remote comms', () => { const platformServicesA = makeLoopbackPlatformServices({ hub }); const platformServicesB = makeLoopbackPlatformServices({ hub }); - await platformServicesB.initializeRemoteComms( - SEED_B, - {}, - async (from: string, message: string) => { - receivedByB.push([from, message]); - return message === 'ping' ? 'pong' : null; + await platformServicesB.initializeRemoteComms({ + keySeed: SEED_B, + specifier: { netlayer: 'loopback', config: {} }, + hooks: { + handleMessage: async (from: string, message: string) => { + receivedByB.push([from, message]); + return message === 'ping' ? 'pong' : null; + }, }, - undefined, - 'incarnation-b', - ); - await platformServicesA.initializeRemoteComms( - SEED_A, - {}, - async (from: string, message: string) => { - receivedByA.push([from, message]); - return null; + incarnationId: 'incarnation-b', + }); + await platformServicesA.initializeRemoteComms({ + keySeed: SEED_A, + specifier: { netlayer: 'loopback', config: {} }, + hooks: { + handleMessage: async (from: string, message: string) => { + receivedByA.push([from, message]); + return null; + }, }, - undefined, - 'incarnation-a', - ); + incarnationId: 'incarnation-a', + }); await platformServicesA.sendRemoteMessage(peerB, 'ping'); @@ -56,20 +58,18 @@ describe('loopback remote comms', () => { const peerB = deriveNeutralPeerId(fromHex(SEED_B)); const platformServicesB = makeLoopbackPlatformServices({ hub }); const platformServicesA = makeLoopbackPlatformServices({ hub }); - await platformServicesB.initializeRemoteComms( - SEED_B, - {}, - vi.fn().mockResolvedValue(null), - undefined, - 'incarnation-b', - ); - await platformServicesA.initializeRemoteComms( - SEED_A, - {}, - vi.fn().mockResolvedValue(null), - undefined, - 'incarnation-a', - ); + await platformServicesB.initializeRemoteComms({ + keySeed: SEED_B, + specifier: { netlayer: 'loopback', config: {} }, + hooks: { handleMessage: vi.fn().mockResolvedValue(null) }, + incarnationId: 'incarnation-b', + }); + await platformServicesA.initializeRemoteComms({ + keySeed: SEED_A, + specifier: { netlayer: 'loopback', config: {} }, + hooks: { handleMessage: vi.fn().mockResolvedValue(null) }, + incarnationId: 'incarnation-a', + }); expect(platformServicesA.getListenAddresses()).toStrictEqual([]); expect( @@ -106,13 +106,12 @@ describe('loopback remote comms', () => { const hub = makeLoopbackHub(); const peerA = deriveNeutralPeerId(fromHex(SEED_A)); const platformServices = makeLoopbackPlatformServices({ hub }); - await platformServices.initializeRemoteComms( - SEED_A, - {}, - vi.fn().mockResolvedValue(null), - undefined, - 'incarnation-a', - ); + await platformServices.initializeRemoteComms({ + keySeed: SEED_A, + specifier: { netlayer: 'loopback', config: {} }, + hooks: { handleMessage: vi.fn().mockResolvedValue(null) }, + incarnationId: 'incarnation-a', + }); expect(hub.getIncarnation(peerA)).toBe('incarnation-a'); await platformServices.stopRemoteComms(); diff --git a/packages/ocap-kernel/src/remotes/platform/transport.ts b/packages/ocap-kernel/src/remotes/platform/transport.ts deleted file mode 100644 index 942becc939..0000000000 --- a/packages/ocap-kernel/src/remotes/platform/transport.ts +++ /dev/null @@ -1,106 +0,0 @@ -import { Logger } from '@metamask/logger'; -import { - DEFAULT_MAX_MESSAGE_SIZE_BYTES, - makeChannelNetlayer, -} from '@metamask/netlayer'; -import type { Netlayer } from '@metamask/netlayer'; - -import { ConnectionFactory } from './connection-factory.ts'; -import type { - RemoteMessageHandler, - OnRemoteGiveUp, - OnIncarnationChange, - RemoteCommsOptions, -} from '../types.ts'; - -/** - * Initialize the remote comm system with information that must be provided by the kernel. - * - * Thin, signature-compatible wrapper: constructs the libp2p - * {@link ConnectionFactory} provider and delegates the engine to - * `makeChannelNetlayer` from `@metamask/netlayer`, sharing one `Logger` and one - * `AbortController`. The libp2p-specific provider options (`relays`, - * `directTransports`, `allowedWsHosts`) are consumed here; the rest map onto the - * engine's neutral options. - * - * @param keySeed - Seed value for key generation, in the form of a hex-encoded string. - * @param options - Options for remote communications initialization. - * @param options.relays - PeerIds/Multiaddrs of known message relays. - * @param options.maxRetryAttempts - Maximum number of reconnection attempts. 0 = infinite (default). - * @param options.maxQueue - Maximum pending messages per peer (default: 200). - * @param options.maxConcurrentConnections - Maximum number of concurrent connections (default: 100). - * @param options.maxMessageSizeBytes - Maximum message size in bytes (default: 1MB). - * @param options.cleanupIntervalMs - Stale peer cleanup interval in milliseconds (default: 15 minutes). - * @param options.stalePeerTimeoutMs - Stale peer timeout in milliseconds (default: 1 hour). - * @param options.maxMessagesPerSecond - Maximum messages per second per peer (default: 100). - * @param options.maxConnectionAttemptsPerMinute - Maximum connection attempts per minute per peer (default: 10). - * @param remoteMessageHandler - Handler to be called when messages are received from elsewhere. - * @param onRemoteGiveUp - Optional callback to be called when we give up on a remote (after max retries or non-retryable error). - * @param localIncarnationId - This kernel's incarnation ID for handshake protocol. - * @param onIncarnationChange - Optional callback when a remote peer's incarnation changes (peer restarted). - * - * @returns a {@link Netlayer}: a function to send messages **and** a `stop()` to cancel/release everything. - */ -export async function initTransport( - keySeed: string, - options: RemoteCommsOptions, - remoteMessageHandler: RemoteMessageHandler, - onRemoteGiveUp?: OnRemoteGiveUp, - localIncarnationId?: string, - onIncarnationChange?: OnIncarnationChange, -): Promise { - const { - relays = [], - maxRetryAttempts, - maxConcurrentConnections, - maxMessageSizeBytes, - cleanupIntervalMs, - stalePeerTimeoutMs, - maxMessagesPerSecond, - maxConnectionAttemptsPerMinute, - reconnectionBaseDelayMs, - reconnectionMaxDelayMs, - handshakeTimeoutMs, - writeTimeoutMs, - streamInactivityTimeoutMs, - directTransports, - allowedWsHosts, - } = options; - const logger = new Logger(); - const stopController = new AbortController(); - const provider = await ConnectionFactory.make({ - keySeed, - knownRelays: relays, - logger, - signal: stopController.signal, - maxRetryAttempts, - maxMessageSizeBytes: maxMessageSizeBytes ?? DEFAULT_MAX_MESSAGE_SIZE_BYTES, - directTransports, - allowedWsHosts, - }); - return makeChannelNetlayer({ - provider, - hooks: { - handleMessage: remoteMessageHandler, - onRemoteGiveUp, - onIncarnationChange, - }, - options: { - maxRetryAttempts, - maxConcurrentConnections, - maxMessageSizeBytes, - cleanupIntervalMs, - stalePeerTimeoutMs, - maxMessagesPerSecond, - maxConnectionAttemptsPerMinute, - reconnectionBaseDelayMs, - reconnectionMaxDelayMs, - handshakeTimeoutMs, - writeTimeoutMs, - streamInactivityTimeoutMs, - localIncarnationId, - }, - logger, - stopController, - }); -} diff --git a/packages/ocap-kernel/src/remotes/types.ts b/packages/ocap-kernel/src/remotes/types.ts index c57bba6350..bc24292897 100644 --- a/packages/ocap-kernel/src/remotes/types.ts +++ b/packages/ocap-kernel/src/remotes/types.ts @@ -1,5 +1,4 @@ -import type { Logger } from '@metamask/logger'; -import type { SendRemoteMessage } from '@metamask/netlayer'; +import type { NetlayerSpecifier, SendRemoteMessage } from '@metamask/netlayer'; import type { KRef } from '../types.ts'; @@ -36,164 +35,40 @@ export type RemoteComms = RemoteIdentity & { }; /** - * Options for initializing remote communications. + * Kernel-level options for initializing remote communications. All per-netlayer + * configuration lives in `specifier.config`; only options the kernel itself + * owns remain here. */ export type RemoteCommsOptions = { /** - * Array of relay peer IDs/multiaddrs to use for remote communications. + * Which netlayer to use and its `Json` config. By convention the kernel knows + * exactly one config key — `knownRelays: string[]` — which it treats as the + * opaque hint pool it persists and re-injects. Omitted specifiers default to + * the libp2p netlayer during the transition. */ - relays?: string[] | undefined; + specifier?: NetlayerSpecifier | undefined; /** - * Maximum number of reconnection attempts. 0 = infinite (default). - * If not provided, uses DEFAULT_MAX_RETRY_ATTEMPTS. - */ - maxRetryAttempts?: number | undefined; - /** - * Maximum number of pending messages awaiting ACK per peer. - * New messages are rejected when this limit is reached. - * If not provided, uses DEFAULT_MAX_QUEUE (200). - */ - maxQueue?: number | undefined; - /** - * Maximum number of concurrent connections (default: 100). - * When the limit is reached, new outbound connections are rejected with - * ResourceLimitError and inbound connections are closed immediately. - */ - maxConcurrentConnections?: number | undefined; - /** - * Maximum message size in bytes (default: 1MB). - * Messages exceeding this limit are immediately rejected with ResourceLimitError - * before any connection or queuing attempt. - */ - maxMessageSizeBytes?: number | undefined; - /** - * Interval in milliseconds between stale peer cleanup runs (default: 15 minutes). - * Controls how often the system checks for and removes stale peer data. - */ - cleanupIntervalMs?: number | undefined; - /** - * Time in milliseconds before a disconnected peer is considered stale (default: 1 hour). - * When a peer has been disconnected for longer than this duration and is not - * actively reconnecting, its data is cleaned up including: message queues, - * location hints, connection timestamps, and reconnection state. - */ - stalePeerTimeoutMs?: number | undefined; - /** - * BIP39 mnemonic phrase (12, 15, 18, 21, or 24 words) for seed recovery. - * When provided, derives the kernel identity seed from this mnemonic instead of - * generating a random seed. The same mnemonic will always produce the same peer ID. - * If the kernel already has a stored identity, an error is thrown. Use - * `resetStorage: true` when creating the kernel to clear existing identity first. + * BIP39 mnemonic phrase for seed recovery. Sensitive key material; never + * forwarded to the netlayer. */ mnemonic?: string | undefined; /** - * Maximum messages per second per peer (default: 100). - * Messages exceeding this rate are rejected with ResourceLimitError. - * Uses a sliding 1-second window. - */ - maxMessagesPerSecond?: number | undefined; - /** - * Maximum connection attempts per minute per peer (default: 10). - * Connection attempts exceeding this rate are rejected with ResourceLimitError. - * Uses a sliding 1-minute window. - */ - maxConnectionAttemptsPerMinute?: number | undefined; - /** - * Base delay in milliseconds for reconnection exponential backoff (default: 500ms). - * Used as the starting delay that doubles with each subsequent attempt. - */ - reconnectionBaseDelayMs?: number | undefined; - /** - * Maximum delay in milliseconds for reconnection exponential backoff (default: 10s). - * The backoff delay is capped at this value regardless of attempt count. - */ - reconnectionMaxDelayMs?: number | undefined; - /** - * Timeout in milliseconds for handshake operations (default: 10s). - * Controls how long to wait for a handshake or handshakeAck response. + * Maximum number of pending messages awaiting ACK per peer (default: 200). */ - handshakeTimeoutMs?: number | undefined; - /** - * Timeout in milliseconds for channel write operations (default: 10s). - * Controls how long to wait for a message to be written to a channel. - */ - writeTimeoutMs?: number | undefined; + maxQueue?: number | undefined; /** - * Timeout in milliseconds for ACK before retransmitting a message (default: 10s). - * When a sent message is not acknowledged within this timeout, it will be retransmitted. + * Timeout in milliseconds for ACK before retransmitting a message. */ ackTimeoutMs?: number | undefined; - /** - * Timeout in milliseconds for stream inactivity (default: 120s). - * If no data flows in either direction for this duration, the stream is - * automatically aborted with an InactivityTimeoutError. - */ - streamInactivityTimeoutMs?: number | undefined; /** * Maximum number of relay hints embedded in a single OCAP URL (default: 3). - * Higher values produce longer URLs but improve connectivity resilience for - * peers with stale relay information. */ maxUrlRelayHints?: number | undefined; /** * Maximum number of relay entries stored in the kernel's relay pool - * (default: 20). Bootstrap relays are prioritized during eviction; when - * the cap is reached, the oldest non-bootstrap (learned) entries are - * evicted first. If bootstrap relays alone exceed the cap, the pool is - * truncated to the cap. + * (default: 20). */ maxKnownRelays?: number | undefined; - /** - * Hostnames or IP addresses permitted for plain ws:// relay connections, - * in addition to RFC 1918 / loopback addresses which are always allowed. - * Defaults to [] (private/loopback addresses only). - */ - allowedWsHosts?: string[] | undefined; - /** - * Direct listen addresses for non-relay transports (e.g. QUIC, TCP). - * Example: `['/ip4/0.0.0.0/udp/0/quic-v1', '/ip4/0.0.0.0/tcp/4001']` - * - * The platform layer detects the required transports from the address strings - * and injects them automatically. Users never need to import transport packages. - */ - directListenAddresses?: string[] | undefined; - /** - * Internal option injected by platform services. Bundles direct transport - * implementations with their listen addresses. Users should use - * `directListenAddresses` instead. - * - * @internal - */ - directTransports?: DirectTransport[]; -}; - -/** - * A direct transport implementation bundled with its listen addresses. - */ -export type DirectTransport = { - transport: unknown; - listenAddresses: string[]; -}; - -/** - * Options for creating a ConnectionFactory instance. - */ -export type ConnectionFactoryOptions = { - keySeed: string; - knownRelays: string[]; - logger: Logger; - signal: AbortSignal; - maxRetryAttempts?: number | undefined; - /** - * Maximum inbound message payload size in bytes. Used as `maxDataLength` - * on every `lpStream` constructed for a channel — must match the - * sender-side validator's limit (`maxMessageSizeBytes` on - * `RemoteCommsOptions`) so that a deployment which raises one also raises - * the other. Defaults to `DEFAULT_MAX_MESSAGE_SIZE_BYTES` (1 MB). - */ - maxMessageSizeBytes?: number | undefined; - directTransports?: DirectTransport[] | undefined; - allowedWsHosts?: string[] | undefined; }; export type RemoteInfo = { diff --git a/packages/ocap-kernel/src/rpc/kernel-control/init-remote-comms.test.ts b/packages/ocap-kernel/src/rpc/kernel-control/init-remote-comms.test.ts index e8dbe6c4c1..df1273ab3e 100644 --- a/packages/ocap-kernel/src/rpc/kernel-control/init-remote-comms.test.ts +++ b/packages/ocap-kernel/src/rpc/kernel-control/init-remote-comms.test.ts @@ -7,6 +7,11 @@ import { } from './init-remote-comms.ts'; import type { Kernel } from '../../Kernel.ts'; +const specifier = { + netlayer: 'libp2p', + config: { knownRelays: ['/dns4/relay.example/tcp/443/wss/p2p/relay'] }, +}; + describe('initRemoteCommsHandler', () => { let mockKernel: Kernel; @@ -26,60 +31,21 @@ describe('initRemoteCommsHandler', () => { expect(result).toBeNull(); }); - it('passes relays option', async () => { + it('passes a specifier through to the kernel', async () => { const result = await initRemoteCommsHandler.implementation( { kernel: mockKernel }, - { relays: ['/dns4/relay.example.com/tcp/443/wss/p2p/QmRelay'] }, + { specifier }, ); - expect(mockKernel.initRemoteComms).toHaveBeenCalledWith({ - relays: ['/dns4/relay.example.com/tcp/443/wss/p2p/QmRelay'], - }); + expect(mockKernel.initRemoteComms).toHaveBeenCalledWith({ specifier }); expect(result).toBeNull(); }); - it('passes directListenAddresses option', async () => { - const result = await initRemoteCommsHandler.implementation( - { kernel: mockKernel }, - { directListenAddresses: ['/ip4/0.0.0.0/udp/0/quic-v1'] }, - ); - - expect(mockKernel.initRemoteComms).toHaveBeenCalledWith({ - directListenAddresses: ['/ip4/0.0.0.0/udp/0/quic-v1'], - }); - expect(result).toBeNull(); - }); - - it('passes maxUrlRelayHints option', async () => { - const result = await initRemoteCommsHandler.implementation( - { kernel: mockKernel }, - { maxUrlRelayHints: 5 }, - ); - - expect(mockKernel.initRemoteComms).toHaveBeenCalledWith({ - maxUrlRelayHints: 5, - }); - expect(result).toBeNull(); - }); - - it('passes maxKnownRelays option', async () => { - const result = await initRemoteCommsHandler.implementation( - { kernel: mockKernel }, - { maxKnownRelays: 30 }, - ); - - expect(mockKernel.initRemoteComms).toHaveBeenCalledWith({ - maxKnownRelays: 30, - }); - expect(result).toBeNull(); - }); - - it('passes all options', async () => { + it('passes kernel-level options', async () => { const params = { - relays: ['relay1'], - directListenAddresses: ['/ip4/0.0.0.0/udp/0/quic-v1'], - maxRetryAttempts: 5, + specifier, maxQueue: 100, + ackTimeoutMs: 5000, maxUrlRelayHints: 3, maxKnownRelays: 20, }; @@ -89,14 +55,7 @@ describe('initRemoteCommsHandler', () => { params, ); - expect(mockKernel.initRemoteComms).toHaveBeenCalledWith({ - relays: ['relay1'], - directListenAddresses: ['/ip4/0.0.0.0/udp/0/quic-v1'], - maxRetryAttempts: 5, - maxQueue: 100, - maxUrlRelayHints: 3, - maxKnownRelays: 20, - }); + expect(mockKernel.initRemoteComms).toHaveBeenCalledWith(params); expect(result).toBeNull(); }); @@ -112,37 +71,26 @@ describe('initRemoteCommsHandler', () => { describe('params validation', () => { it('accepts valid params', () => { expect(is({}, initRemoteCommsSpec.params)).toBe(true); + expect(is({ specifier }, initRemoteCommsSpec.params)).toBe(true); expect( - is({ maxRetryAttempts: 0, maxQueue: 0 }, initRemoteCommsSpec.params), - ).toBe(true); - expect( - is({ maxRetryAttempts: 5, maxQueue: 100 }, initRemoteCommsSpec.params), + is({ maxQueue: 0, ackTimeoutMs: 0 }, initRemoteCommsSpec.params), ).toBe(true); }); - it.each([-1, -100, 1.5, 3.14, -0.5])( - 'rejects invalid maxRetryAttempts value: %s', - (value) => { - expect( - is({ maxRetryAttempts: value }, initRemoteCommsSpec.params), - ).toBe(false); - }, - ); - - it.each([-1, -100, 1.5, 3.14, -0.5])( - 'rejects invalid maxQueue value: %s', - (value) => { - expect(is({ maxQueue: value }, initRemoteCommsSpec.params)).toBe(false); - }, - ); + it('rejects a specifier with a non-string netlayer', () => { + expect( + is( + { specifier: { netlayer: 1, config: {} } }, + initRemoteCommsSpec.params, + ), + ).toBe(false); + }); - it.each([1, 3, 10])('accepts valid maxUrlRelayHints value: %s', (value) => { - expect(is({ maxUrlRelayHints: value }, initRemoteCommsSpec.params)).toBe( - true, - ); + it.each([-1, 1.5, -0.5])('rejects invalid maxQueue value: %s', (value) => { + expect(is({ maxQueue: value }, initRemoteCommsSpec.params)).toBe(false); }); - it.each([0, -1, -100, 1.5, 3.14])( + it.each([0, -1, 1.5])( 'rejects invalid maxUrlRelayHints value: %s', (value) => { expect( @@ -156,14 +104,5 @@ describe('initRemoteCommsHandler', () => { true, ); }); - - it.each([0, -1, -100, 1.5, 3.14])( - 'rejects invalid maxKnownRelays value: %s', - (value) => { - expect(is({ maxKnownRelays: value }, initRemoteCommsSpec.params)).toBe( - false, - ); - }, - ); }); }); diff --git a/packages/ocap-kernel/src/rpc/kernel-control/init-remote-comms.ts b/packages/ocap-kernel/src/rpc/kernel-control/init-remote-comms.ts index 1dd518e88d..13a52c24c0 100644 --- a/packages/ocap-kernel/src/rpc/kernel-control/init-remote-comms.ts +++ b/packages/ocap-kernel/src/rpc/kernel-control/init-remote-comms.ts @@ -4,25 +4,25 @@ import { object, literal, optional, - array, string, integer, min, } from '@metamask/superstruct'; import type { Struct } from '@metamask/superstruct'; import type { Infer } from '@metamask/superstruct'; +import { JsonStruct } from '@metamask/utils'; import type { Kernel } from '../../Kernel.ts'; import type { RemoteCommsOptions } from '../../remotes/types.ts'; const initRemoteCommsParamsStruct = object({ - relays: optional(array(string())), - directListenAddresses: optional(array(string())), - maxRetryAttempts: optional(min(integer(), 0)), + // Which netlayer to use plus its Json config (crosses postMessage). + specifier: optional(object({ netlayer: string(), config: JsonStruct })), + // `mnemonic` is intentionally excluded from the RPC struct (sensitive). maxQueue: optional(min(integer(), 0)), + ackTimeoutMs: optional(min(integer(), 0)), maxUrlRelayHints: optional(min(integer(), 1)), maxKnownRelays: optional(min(integer(), 1)), - allowedWsHosts: optional(array(string())), }); // Superstruct's `optional()` infers `T | undefined` for each field, but diff --git a/packages/ocap-kernel/src/rpc/kernel-remote/remoteIncarnationChange.test.ts b/packages/ocap-kernel/src/rpc/kernel-remote/remoteIncarnationChange.test.ts new file mode 100644 index 0000000000..68755935f0 --- /dev/null +++ b/packages/ocap-kernel/src/rpc/kernel-remote/remoteIncarnationChange.test.ts @@ -0,0 +1,82 @@ +import { is } from '@metamask/superstruct'; +import { describe, it, expect, vi } from 'vitest'; + +import type { HandleRemoteIncarnationChange } from './remoteIncarnationChange.ts'; +import { + remoteIncarnationChangeSpec, + remoteIncarnationChangeHandler, +} from './remoteIncarnationChange.ts'; + +describe('remoteIncarnationChange', () => { + describe('remoteIncarnationChangeSpec', () => { + it('has the correct method name', () => { + expect(remoteIncarnationChangeSpec.method).toBe( + 'remoteIncarnationChange', + ); + }); + + it('accepts a boolean result and rejects non-booleans', () => { + expect(is(true, remoteIncarnationChangeSpec.result)).toBe(true); + expect(is(false, remoteIncarnationChangeSpec.result)).toBe(true); + expect(is('true', remoteIncarnationChangeSpec.result)).toBe(false); + expect(is(undefined, remoteIncarnationChangeSpec.result)).toBe(false); + }); + + it('validates params', () => { + expect( + is( + { peerId: 'peer-123', observedIncarnation: 'inc-1' }, + remoteIncarnationChangeSpec.params, + ), + ).toBe(true); + expect( + is( + { peerId: 123, observedIncarnation: 'inc-1' }, + remoteIncarnationChangeSpec.params, + ), + ).toBe(false); + expect(is({ peerId: 'p' }, remoteIncarnationChangeSpec.params)).toBe( + false, + ); + }); + }); + + describe('remoteIncarnationChangeHandler', () => { + it('declares its hook', () => { + expect(remoteIncarnationChangeHandler.hooks).toStrictEqual({ + remoteIncarnationChange: true, + }); + }); + + it('forwards params to the hook and returns its boolean result', async () => { + const hook: HandleRemoteIncarnationChange = vi.fn(async () => true); + const result = await remoteIncarnationChangeHandler.implementation( + { remoteIncarnationChange: hook }, + { peerId: 'peer-123', observedIncarnation: 'inc-2' }, + ); + expect(hook).toHaveBeenCalledWith('peer-123', 'inc-2'); + expect(result).toBe(true); + }); + + it('propagates a false result from the hook', async () => { + const hook: HandleRemoteIncarnationChange = vi.fn(async () => false); + const result = await remoteIncarnationChangeHandler.implementation( + { remoteIncarnationChange: hook }, + { peerId: 'peer-123', observedIncarnation: 'inc-2' }, + ); + expect(result).toBe(false); + }); + + it('propagates errors from the hook', async () => { + const hook: HandleRemoteIncarnationChange = vi.fn(async () => { + throw new Error('incarnation check failed'); + }); + await expect( + remoteIncarnationChangeHandler.implementation( + { remoteIncarnationChange: hook }, + { peerId: 'peer-123', observedIncarnation: 'inc-2' }, + ), + ).rejects.toThrow('incarnation check failed'); + }); + }); +}); diff --git a/packages/ocap-kernel/src/rpc/platform-services/initializeRemoteComms.test.ts b/packages/ocap-kernel/src/rpc/platform-services/initializeRemoteComms.test.ts index 4e4cf9dc6f..71fd1c2ccc 100644 --- a/packages/ocap-kernel/src/rpc/platform-services/initializeRemoteComms.test.ts +++ b/packages/ocap-kernel/src/rpc/platform-services/initializeRemoteComms.test.ts @@ -6,833 +6,117 @@ import { initializeRemoteCommsSpec, initializeRemoteCommsHandler, } from './initializeRemoteComms.ts'; -import type { RemoteCommsOptions } from '../../remotes/types.ts'; + +const specifier = { + netlayer: 'libp2p', + config: { knownRelays: ['/dns4/relay.example/tcp/443/wss/p2p/relay'] }, +}; describe('initializeRemoteComms', () => { describe('initializeRemoteCommsSpec', () => { - it('should have correct method name', () => { + it('has the correct method name', () => { expect(initializeRemoteCommsSpec.method).toBe('initializeRemoteComms'); }); - it('should have correct result type', () => { - // Test that result validator accepts null + it('accepts null result and rejects non-null', () => { expect(is(null, initializeRemoteCommsSpec.result)).toBe(true); expect(is('string', initializeRemoteCommsSpec.result)).toBe(false); - expect(is(123, initializeRemoteCommsSpec.result)).toBe(false); expect(is(undefined, initializeRemoteCommsSpec.result)).toBe(false); }); describe('params validation', () => { - it('should accept valid params', () => { - const validParams = { - keySeed: '0x1234567890abcdef', - relays: [ - '/dns4/relay1.example/tcp/443/wss/p2p/relay1', - '/dns4/relay2.example/tcp/443/wss/p2p/relay2', - ], - }; - - expect(is(validParams, initializeRemoteCommsSpec.params)).toBe(true); - }); - - it('should accept params with empty relays array', () => { - const validParams = { - keySeed: '0xabcdef1234567890', - relays: [], - }; - - expect(is(validParams, initializeRemoteCommsSpec.params)).toBe(true); - }); - - it('should accept params with only keySeed', () => { - const validParams = { - keySeed: '0x1234567890abcdef', - }; - - expect(is(validParams, initializeRemoteCommsSpec.params)).toBe(true); - }); - - it('should accept params with maxRetryAttempts', () => { - const validParams = { - keySeed: '0x1234567890abcdef', - maxRetryAttempts: 5, - }; - - expect(is(validParams, initializeRemoteCommsSpec.params)).toBe(true); - }); - - it('should accept params with maxQueue', () => { - const validParams = { - keySeed: '0x1234567890abcdef', - maxQueue: 100, - }; - - expect(is(validParams, initializeRemoteCommsSpec.params)).toBe(true); - }); - - it('should accept params with allowedWsHosts', () => { - const validParams = { - keySeed: '0x1234567890abcdef', - allowedWsHosts: ['localhost', 'example.com'], - }; - - expect(is(validParams, initializeRemoteCommsSpec.params)).toBe(true); - }); - - it('should accept params with empty allowedWsHosts array', () => { - const validParams = { - keySeed: '0x1234567890abcdef', - allowedWsHosts: [], - }; - - expect(is(validParams, initializeRemoteCommsSpec.params)).toBe(true); - }); - - it('should reject params with non-array allowedWsHosts', () => { - const invalidParams = { - keySeed: '0x1234567890abcdef', - allowedWsHosts: 'not-an-array', - }; - - expect(is(invalidParams, initializeRemoteCommsSpec.params)).toBe(false); - }); - - it('should reject params with non-string elements in allowedWsHosts', () => { - const invalidParams = { - keySeed: '0x1234567890abcdef', - allowedWsHosts: ['localhost', 123], - }; - - expect(is(invalidParams, initializeRemoteCommsSpec.params)).toBe(false); - }); - - it('should accept params with incarnationId', () => { - const validParams = { - keySeed: '0x1234567890abcdef', - incarnationId: 'test-incarnation-id', - }; - - expect(is(validParams, initializeRemoteCommsSpec.params)).toBe(true); - }); - - it('should reject params with missing keySeed', () => { - const invalidParams = { - relays: ['/dns4/relay.example/tcp/443/wss/p2p/relay'], - }; - - expect(is(invalidParams, initializeRemoteCommsSpec.params)).toBe(false); - }); - - it('should reject params with non-string keySeed', () => { - const invalidParams = { - keySeed: 123, - relays: [], - }; - - expect(is(invalidParams, initializeRemoteCommsSpec.params)).toBe(false); - }); - - it('should reject params with non-array knownRelays', () => { - const invalidParams = { - keySeed: '0x1234567890abcdef', - relays: 'not-an-array', - }; - - expect(is(invalidParams, initializeRemoteCommsSpec.params)).toBe(false); - }); - - it('should reject params with non-string elements in relays', () => { - const invalidParams = { - keySeed: '0x1234567890abcdef', - relays: [ - '/dns4/relay1.example/tcp/443/wss/p2p/relay1', - 123, // non-string element - ], - }; - - expect(is(invalidParams, initializeRemoteCommsSpec.params)).toBe(false); - }); - - it('should reject params with non-number maxRetryAttempts', () => { - const invalidParams = { - keySeed: '0x1234567890abcdef', - maxRetryAttempts: 'not-a-number', - }; - - expect(is(invalidParams, initializeRemoteCommsSpec.params)).toBe(false); - }); - - it('should reject params with non-number maxQueue', () => { - const invalidParams = { - keySeed: '0x1234567890abcdef', - maxQueue: 'not-a-number', - }; - - expect(is(invalidParams, initializeRemoteCommsSpec.params)).toBe(false); - }); - - it('should reject params with extra fields', () => { - const invalidParams = { - keySeed: '0x1234567890abcdef', - relays: [], - extra: 'field', - }; - - expect(is(invalidParams, initializeRemoteCommsSpec.params)).toBe(false); - }); - - it('should reject null params', () => { - expect(is(null, initializeRemoteCommsSpec.params)).toBe(false); - }); - - it('should reject undefined params', () => { - expect(is(undefined, initializeRemoteCommsSpec.params)).toBe(false); - }); - - it('should reject non-object params', () => { - expect(is('string', initializeRemoteCommsSpec.params)).toBe(false); - expect(is(123, initializeRemoteCommsSpec.params)).toBe(false); - expect(is([], initializeRemoteCommsSpec.params)).toBe(false); - }); - - it('should accept empty string keySeed', () => { - const validParams = { - keySeed: '', - relays: [], - }; - - expect(is(validParams, initializeRemoteCommsSpec.params)).toBe(true); - }); - - it('should accept empty strings in relays', () => { - const validParams = { - keySeed: '0x1234567890abcdef', - relays: ['', ''], - }; - - expect(is(validParams, initializeRemoteCommsSpec.params)).toBe(true); - }); - - it('should accept unicode characters', () => { - const validParams = { - keySeed: '🔑seed🔑', - relays: ['🌐relay🌐', '🔗connection🔗'], - }; - - expect(is(validParams, initializeRemoteCommsSpec.params)).toBe(true); - }); - - it('should accept very long strings', () => { - const longString = 'a'.repeat(10000); - const validParams = { - keySeed: longString, - relays: [longString, longString], - }; - - expect(is(validParams, initializeRemoteCommsSpec.params)).toBe(true); - }); - - it('should accept many relay addresses', () => { - const manyRelays = Array.from( - { length: 100 }, - (_, i) => `/dns4/relay${i}.example/tcp/443/wss/p2p/relay${i}`, - ); - - const validParams = { - keySeed: '0x1234567890abcdef', - relays: manyRelays, - }; - - expect(is(validParams, initializeRemoteCommsSpec.params)).toBe(true); + it('accepts a keySeed + specifier', () => { + expect( + is( + { keySeed: '0x1234', specifier }, + initializeRemoteCommsSpec.params, + ), + ).toBe(true); + }); + + it('accepts a specifier with an empty config object', () => { + expect( + is( + { + keySeed: '0x1234', + specifier: { netlayer: 'libp2p', config: {} }, + }, + initializeRemoteCommsSpec.params, + ), + ).toBe(true); + }); + + it('accepts an optional incarnationId', () => { + expect( + is( + { keySeed: '0x1234', specifier, incarnationId: 'inc-1' }, + initializeRemoteCommsSpec.params, + ), + ).toBe(true); + }); + + it.each([ + { name: 'missing keySeed', value: { specifier } }, + { name: 'missing specifier', value: { keySeed: '0x1234' } }, + { + name: 'non-string netlayer', + value: { keySeed: '0x1234', specifier: { netlayer: 1, config: {} } }, + }, + { + name: 'non-Json config (function)', + value: { + keySeed: '0x1234', + specifier: { netlayer: 'libp2p', config: () => undefined }, + }, + }, + { + name: 'extra field', + value: { keySeed: '0x1234', specifier, extra: 'nope' }, + }, + { name: 'null', value: null }, + { name: 'non-object', value: 'string' }, + ])('rejects params with $name', ({ value }) => { + expect(is(value, initializeRemoteCommsSpec.params)).toBe(false); }); }); }); describe('initializeRemoteCommsHandler', () => { - it('should have correct method name', () => { - expect(initializeRemoteCommsHandler.method).toBe('initializeRemoteComms'); - }); - - it('should have correct hooks configuration', () => { + it('declares its hook', () => { expect(initializeRemoteCommsHandler.hooks).toStrictEqual({ initializeRemoteComms: true, }); }); - it('should call the initializeRemoteComms hook with correct parameters', async () => { - const mockInitializeRemoteComms: InitializeRemoteComms = vi.fn( - async () => null, - ); - - const hooks = { - initializeRemoteComms: mockInitializeRemoteComms, - }; - - const params = { - keySeed: '0x1234567890abcdef', - relays: [ - '/dns4/relay1.example/tcp/443/wss/p2p/relay1', - '/dns4/relay2.example/tcp/443/wss/p2p/relay2', - ], - }; - + it('forwards keySeed, specifier, and incarnationId positionally (no hooks in params)', async () => { + const hook: InitializeRemoteComms = vi.fn(async () => null); const result = await initializeRemoteCommsHandler.implementation( - hooks, - params, - ); - - expect(mockInitializeRemoteComms).toHaveBeenCalledTimes(1); - expect(mockInitializeRemoteComms).toHaveBeenCalledWith( - '0x1234567890abcdef', - { - relays: [ - '/dns4/relay1.example/tcp/443/wss/p2p/relay1', - '/dns4/relay2.example/tcp/443/wss/p2p/relay2', - ], - }, - undefined, + { initializeRemoteComms: hook }, + { keySeed: '0x1234', specifier, incarnationId: 'inc-1' }, ); + expect(hook).toHaveBeenCalledWith('0x1234', specifier, 'inc-1'); expect(result).toBeNull(); }); - it('should return null from the hook', async () => { - const mockInitializeRemoteComms: InitializeRemoteComms = vi.fn( - async () => null, + it('passes undefined incarnationId when omitted', async () => { + const hook: InitializeRemoteComms = vi.fn(async () => null); + await initializeRemoteCommsHandler.implementation( + { initializeRemoteComms: hook }, + { keySeed: '0x1234', specifier }, ); - - const hooks = { - initializeRemoteComms: mockInitializeRemoteComms, - }; - - const params = { - keySeed: 'test-seed', - relays: ['test-relay'], - }; - - const result = await initializeRemoteCommsHandler.implementation( - hooks, - params, - ); - - expect(result).toBeNull(); + expect(hook).toHaveBeenCalledWith('0x1234', specifier, undefined); }); - it('should propagate errors from the hook', async () => { - const mockInitializeRemoteComms: InitializeRemoteComms = vi.fn( - async () => { - throw new Error('Initialization failed'); - }, - ); - - const hooks = { - initializeRemoteComms: mockInitializeRemoteComms, - }; - - const params = { - keySeed: 'failing-seed', - relays: ['failing-relay'], - }; - + it('propagates errors from the hook', async () => { + const hook: InitializeRemoteComms = vi.fn(async () => { + throw new Error('Initialization failed'); + }); await expect( - initializeRemoteCommsHandler.implementation(hooks, params), + initializeRemoteCommsHandler.implementation( + { initializeRemoteComms: hook }, + { keySeed: '0x1234', specifier }, + ), ).rejects.toThrow('Initialization failed'); }); - - it('should handle empty knownRelays array', async () => { - const mockInitializeRemoteComms: InitializeRemoteComms = vi.fn( - async () => null, - ); - - const hooks = { - initializeRemoteComms: mockInitializeRemoteComms, - }; - - const params = { - keySeed: '0xemptyrelays', - relays: [], - }; - - await initializeRemoteCommsHandler.implementation(hooks, params); - - expect(mockInitializeRemoteComms).toHaveBeenCalledWith( - '0xemptyrelays', - { - relays: [], - }, - undefined, - ); - }); - - it('should handle empty string parameters', async () => { - const mockInitializeRemoteComms: InitializeRemoteComms = vi.fn( - async () => null, - ); - - const hooks = { - initializeRemoteComms: mockInitializeRemoteComms, - }; - - const params = { - keySeed: '', - relays: [''], - }; - - await initializeRemoteCommsHandler.implementation(hooks, params); - - expect(mockInitializeRemoteComms).toHaveBeenCalledWith( - '', - { - relays: [''], - }, - undefined, - ); - }); - - it('should handle unicode characters in parameters', async () => { - const mockInitializeRemoteComms: InitializeRemoteComms = vi.fn( - async () => null, - ); - - const hooks = { - initializeRemoteComms: mockInitializeRemoteComms, - }; - - const params = { - keySeed: '🔑unicode-seed🔑', - relays: ['🌐unicode-relay🌐'], - }; - - await initializeRemoteCommsHandler.implementation(hooks, params); - - expect(mockInitializeRemoteComms).toHaveBeenCalledWith( - '🔑unicode-seed🔑', - { relays: ['🌐unicode-relay🌐'] }, - undefined, - ); - }); - - it('should handle async hook that returns a Promise', async () => { - const mockInitializeRemoteComms: InitializeRemoteComms = vi.fn( - async () => { - // Simulate async initialization work - await new Promise((resolve) => setTimeout(resolve, 1)); - return null; - }, - ); - - const hooks = { - initializeRemoteComms: mockInitializeRemoteComms, - }; - - const params = { - keySeed: 'async-seed', - relays: ['async-relay'], - }; - - const result = await initializeRemoteCommsHandler.implementation( - hooks, - params, - ); - - expect(result).toBeNull(); - }); - - it.each([ - { error: new Error('Network setup failed'), keySeed: 'network-seed' }, - { error: new TypeError('Invalid key format'), keySeed: 'invalid-seed' }, - { error: new Error('Relay connection timeout'), keySeed: 'timeout-seed' }, - { error: new Error('INIT_FAILED'), keySeed: 'object-error-seed' }, - ])( - 'should handle initialization error: $error.message with seed $keySeed', - async ({ error, keySeed }) => { - const mockInitializeRemoteComms: InitializeRemoteComms = vi.fn( - async () => { - throw error; - }, - ); - - const hooks = { - initializeRemoteComms: mockInitializeRemoteComms, - }; - - const params = { - keySeed, - relays: ['test-relay'], - }; - - await expect( - initializeRemoteCommsHandler.implementation(hooks, params), - ).rejects.toThrow(error); - - expect(mockInitializeRemoteComms).toHaveBeenCalledWith( - keySeed, - { - relays: ['test-relay'], - }, - undefined, - ); - }, - ); - - it('should handle many relay addresses', async () => { - const mockInitializeRemoteComms: InitializeRemoteComms = vi.fn( - async () => null, - ); - - const hooks = { - initializeRemoteComms: mockInitializeRemoteComms, - }; - - const manyRelays = Array.from( - { length: 50 }, - (_, i) => `/dns4/relay${i}.example/tcp/443/wss/p2p/relay${i}`, - ); - - const params = { - keySeed: '0xmanyrelays', - relays: manyRelays, - }; - - await initializeRemoteCommsHandler.implementation(hooks, params); - - expect(mockInitializeRemoteComms).toHaveBeenCalledWith( - '0xmanyrelays', - { - relays: manyRelays, - }, - undefined, - ); - }); - - it('should handle complex initialization scenarios', async () => { - let initializationSteps = 0; - const mockInitializeRemoteComms: InitializeRemoteComms = vi.fn( - async (keySeed: string, options: RemoteCommsOptions) => { - // Simulate complex initialization - initializationSteps += 1; // Step 1: Parse key seed - await new Promise((resolve) => setTimeout(resolve, 1)); - - initializationSteps += 1; // Step 2: Connect to relays - await new Promise((resolve) => setTimeout(resolve, 1)); - - initializationSteps += 1; // Step 3: Setup network - - // Validate inputs during initialization - if (keySeed.length === 0) { - throw new Error('Invalid key seed'); - } - - if (!options.relays || options.relays.length === 0) { - throw new Error('No relays provided'); - } - - return null; - }, - ); - - const hooks = { - initializeRemoteComms: mockInitializeRemoteComms, - }; - - const params = { - keySeed: '0xcomplexinitialization', - relays: ['/dns4/complex.relay/tcp/443/wss/p2p/complex'], - }; - - const result = await initializeRemoteCommsHandler.implementation( - hooks, - params, - ); - - expect(result).toBeNull(); - expect(initializationSteps).toBe(3); - }); - - it('should pass maxRetryAttempts to hook when provided', async () => { - const mockInitializeRemoteComms: InitializeRemoteComms = vi.fn( - async () => null, - ); - - const hooks = { - initializeRemoteComms: mockInitializeRemoteComms, - }; - - const params = { - keySeed: '0xtestseed', - maxRetryAttempts: 5, - }; - - await initializeRemoteCommsHandler.implementation(hooks, params); - - expect(mockInitializeRemoteComms).toHaveBeenCalledWith( - '0xtestseed', - { - maxRetryAttempts: 5, - }, - undefined, - ); - }); - - it('should pass maxQueue to hook when provided', async () => { - const mockInitializeRemoteComms: InitializeRemoteComms = vi.fn( - async () => null, - ); - - const hooks = { - initializeRemoteComms: mockInitializeRemoteComms, - }; - - const params = { - keySeed: '0xtestseed', - maxQueue: 100, - }; - - await initializeRemoteCommsHandler.implementation(hooks, params); - - expect(mockInitializeRemoteComms).toHaveBeenCalledWith( - '0xtestseed', - { - maxQueue: 100, - }, - undefined, - ); - }); - - it('should pass allowedWsHosts to hook when provided', async () => { - const mockInitializeRemoteComms: InitializeRemoteComms = vi.fn( - async () => null, - ); - - const hooks = { - initializeRemoteComms: mockInitializeRemoteComms, - }; - - const params = { - keySeed: '0xtestseed', - allowedWsHosts: ['localhost', 'example.com'], - }; - - await initializeRemoteCommsHandler.implementation(hooks, params); - - expect(mockInitializeRemoteComms).toHaveBeenCalledWith( - '0xtestseed', - { - allowedWsHosts: ['localhost', 'example.com'], - }, - undefined, - ); - }); - - it('should pass all options when all are provided', async () => { - const mockInitializeRemoteComms: InitializeRemoteComms = vi.fn( - async () => null, - ); - - const hooks = { - initializeRemoteComms: mockInitializeRemoteComms, - }; - - const params = { - keySeed: '0xtestseed', - relays: ['/dns4/relay.example/tcp/443/wss/p2p/relay'], - maxRetryAttempts: 5, - maxQueue: 100, - allowedWsHosts: ['localhost'], - }; - - await initializeRemoteCommsHandler.implementation(hooks, params); - - expect(mockInitializeRemoteComms).toHaveBeenCalledWith( - '0xtestseed', - { - relays: ['/dns4/relay.example/tcp/443/wss/p2p/relay'], - maxRetryAttempts: 5, - maxQueue: 100, - allowedWsHosts: ['localhost'], - }, - undefined, - ); - }); - - it('should pass reconnection and timeout options when provided', async () => { - const mockInitializeRemoteComms: InitializeRemoteComms = vi.fn( - async () => null, - ); - - const hooks = { - initializeRemoteComms: mockInitializeRemoteComms, - }; - - const params = { - keySeed: '0xtestseed', - reconnectionBaseDelayMs: 100, - reconnectionMaxDelayMs: 2000, - handshakeTimeoutMs: 3000, - writeTimeoutMs: 4000, - ackTimeoutMs: 5000, - }; - - await initializeRemoteCommsHandler.implementation(hooks, params); - - expect(mockInitializeRemoteComms).toHaveBeenCalledWith( - '0xtestseed', - { - reconnectionBaseDelayMs: 100, - reconnectionMaxDelayMs: 2000, - handshakeTimeoutMs: 3000, - writeTimeoutMs: 4000, - ackTimeoutMs: 5000, - }, - undefined, - ); - }); - - it('should pass empty options when only keySeed is provided', async () => { - const mockInitializeRemoteComms: InitializeRemoteComms = vi.fn( - async () => null, - ); - - const hooks = { - initializeRemoteComms: mockInitializeRemoteComms, - }; - - const params = { - keySeed: '0xtestseed', - }; - - await initializeRemoteCommsHandler.implementation(hooks, params); - - expect(mockInitializeRemoteComms).toHaveBeenCalledWith( - '0xtestseed', - {}, - undefined, - ); - }); - - it('should not include undefined optional params in options', async () => { - const mockInitializeRemoteComms: InitializeRemoteComms = vi.fn( - async (_keySeed: string, options: RemoteCommsOptions) => { - // Verify that undefined params are not included - expect(options).not.toHaveProperty('relays'); - expect(options).not.toHaveProperty('maxRetryAttempts'); - expect(options).not.toHaveProperty('maxQueue'); - expect(options).not.toHaveProperty('allowedWsHosts'); - return null; - }, - ); - - const hooks = { - initializeRemoteComms: mockInitializeRemoteComms, - }; - - const params = { - keySeed: '0xtestseed', - }; - - await initializeRemoteCommsHandler.implementation(hooks, params); - }); - - it('should accept params with all optional fields', () => { - const validParams = { - keySeed: '0x1234567890abcdef', - relays: ['/dns4/relay.example/tcp/443/wss/p2p/relay'], - maxRetryAttempts: 5, - maxQueue: 100, - allowedWsHosts: ['localhost'], - }; - - expect(is(validParams, initializeRemoteCommsSpec.params)).toBe(true); - }); - - it('should accept params with maxRetryAttempts set to zero', () => { - const validParams = { - keySeed: '0x1234567890abcdef', - maxRetryAttempts: 0, - }; - - expect(is(validParams, initializeRemoteCommsSpec.params)).toBe(true); - }); - - it('should accept params with maxQueue set to zero', () => { - const validParams = { - keySeed: '0x1234567890abcdef', - maxQueue: 0, - }; - - expect(is(validParams, initializeRemoteCommsSpec.params)).toBe(true); - }); - - it('should pass maxRetryAttempts set to zero to hook', async () => { - const mockInitializeRemoteComms: InitializeRemoteComms = vi.fn( - async () => null, - ); - - const hooks = { - initializeRemoteComms: mockInitializeRemoteComms, - }; - - const params = { - keySeed: '0xtestseed', - maxRetryAttempts: 0, - }; - - await initializeRemoteCommsHandler.implementation(hooks, params); - - expect(mockInitializeRemoteComms).toHaveBeenCalledWith( - '0xtestseed', - { - maxRetryAttempts: 0, - }, - undefined, - ); - }); - - it('should pass maxQueue set to zero to hook', async () => { - const mockInitializeRemoteComms: InitializeRemoteComms = vi.fn( - async () => null, - ); - - const hooks = { - initializeRemoteComms: mockInitializeRemoteComms, - }; - - const params = { - keySeed: '0xtestseed', - maxQueue: 0, - }; - - await initializeRemoteCommsHandler.implementation(hooks, params); - - expect(mockInitializeRemoteComms).toHaveBeenCalledWith( - '0xtestseed', - { - maxQueue: 0, - }, - undefined, - ); - }); - - it('should pass incarnationId when provided', async () => { - const mockInitializeRemoteComms: InitializeRemoteComms = vi.fn( - async () => null, - ); - - const hooks = { - initializeRemoteComms: mockInitializeRemoteComms, - }; - - const params = { - keySeed: '0xtestseed', - incarnationId: 'test-incarnation-id', - }; - - await initializeRemoteCommsHandler.implementation(hooks, params); - - expect(mockInitializeRemoteComms).toHaveBeenCalledWith( - '0xtestseed', - {}, - 'test-incarnation-id', - ); - }); }); }); diff --git a/packages/ocap-kernel/src/rpc/platform-services/initializeRemoteComms.ts b/packages/ocap-kernel/src/rpc/platform-services/initializeRemoteComms.ts index 6ba0e14725..4ef0f6aa3a 100644 --- a/packages/ocap-kernel/src/rpc/platform-services/initializeRemoteComms.ts +++ b/packages/ocap-kernel/src/rpc/platform-services/initializeRemoteComms.ts @@ -1,40 +1,17 @@ import type { MethodSpec, Handler } from '@metamask/kernel-rpc-methods'; -import { - array, - object, - literal, - string, - number, - optional, -} from '@metamask/superstruct'; - -import type { RemoteCommsOptions } from '../../remotes/types.ts'; +import { object, literal, string, optional } from '@metamask/superstruct'; +import { JsonStruct } from '@metamask/utils'; +import type { Json } from '@metamask/utils'; const initializeRemoteCommsParamsStruct = object({ keySeed: string(), - relays: optional(array(string())), - maxRetryAttempts: optional(number()), - maxQueue: optional(number()), - allowedWsHosts: optional(array(string())), - reconnectionBaseDelayMs: optional(number()), - reconnectionMaxDelayMs: optional(number()), - handshakeTimeoutMs: optional(number()), - writeTimeoutMs: optional(number()), - ackTimeoutMs: optional(number()), + specifier: object({ netlayer: string(), config: JsonStruct }), incarnationId: optional(string()), }); type InitializeRemoteCommsParams = { keySeed: string; - relays?: string[]; - maxRetryAttempts?: number; - maxQueue?: number; - allowedWsHosts?: string[]; - reconnectionBaseDelayMs?: number; - reconnectionMaxDelayMs?: number; - handshakeTimeoutMs?: number; - writeTimeoutMs?: number; - ackTimeoutMs?: number; + specifier: { netlayer: string; config: Json }; incarnationId?: string; }; @@ -50,9 +27,12 @@ export const initializeRemoteCommsSpec: InitializeRemoteCommsSpec = { result: literal(null), } as InitializeRemoteCommsSpec; +// Hooks (functions) never cross this RPC boundary; only `keySeed`, the `Json` +// `specifier`, and `incarnationId` are transmitted. `PlatformServicesServer` +// reconstructs the netlayer hooks locally. export type InitializeRemoteComms = ( keySeed: string, - options: RemoteCommsOptions, + specifier: { netlayer: string; config: Json }, incarnationId?: string, ) => Promise; @@ -70,39 +50,10 @@ export type InitializeRemoteCommsHandler = Handler< export const initializeRemoteCommsHandler: InitializeRemoteCommsHandler = { ...initializeRemoteCommsSpec, hooks: { initializeRemoteComms: true }, - implementation: async ({ initializeRemoteComms }, params) => { - const options: RemoteCommsOptions = {}; - if (params.relays !== undefined) { - options.relays = params.relays; - } - if (params.maxRetryAttempts !== undefined) { - options.maxRetryAttempts = params.maxRetryAttempts; - } - if (params.maxQueue !== undefined) { - options.maxQueue = params.maxQueue; - } - if (params.allowedWsHosts !== undefined) { - options.allowedWsHosts = params.allowedWsHosts; - } - if (params.reconnectionBaseDelayMs !== undefined) { - options.reconnectionBaseDelayMs = params.reconnectionBaseDelayMs; - } - if (params.reconnectionMaxDelayMs !== undefined) { - options.reconnectionMaxDelayMs = params.reconnectionMaxDelayMs; - } - if (params.handshakeTimeoutMs !== undefined) { - options.handshakeTimeoutMs = params.handshakeTimeoutMs; - } - if (params.writeTimeoutMs !== undefined) { - options.writeTimeoutMs = params.writeTimeoutMs; - } - if (params.ackTimeoutMs !== undefined) { - options.ackTimeoutMs = params.ackTimeoutMs; - } - return await initializeRemoteComms( + implementation: async ({ initializeRemoteComms }, params) => + initializeRemoteComms( params.keySeed, - options, + params.specifier, params.incarnationId, - ); - }, + ), }; diff --git a/packages/ocap-kernel/src/store/index.test.ts b/packages/ocap-kernel/src/store/index.test.ts index 3db43c3e31..1ef888004c 100644 --- a/packages/ocap-kernel/src/store/index.test.ts +++ b/packages/ocap-kernel/src/store/index.test.ts @@ -526,4 +526,39 @@ describe('kernel store', () => { expect(preserved).toBe(original); }); }); + + describe('crank rollback', () => { + it('refreshes the run queue when rolling back to a savepoint', () => { + const ks = makeKernelStore(mockKernelDatabase); + ks.startCrank(); + ks.createCrankSavepoint('sp'); + // Rolling back recreates the run queue from DB state to clear stale + // in-memory caches. + expect(() => ks.rollbackCrank('sp')).not.toThrow(); + ks.endCrank(); + expect(ks.isInCrank()).toBe(false); + }); + }); + + describe('system subcluster mappings', () => { + it('round-trips system subcluster name to id mappings', () => { + const ks = makeKernelStore(mockKernelDatabase); + + expect(ks.getSystemSubclusterMapping('agents')).toBeUndefined(); + + ks.setSystemSubclusterMapping('agents', 's1'); + ks.setSystemSubclusterMapping('io', 's2'); + + expect(ks.getSystemSubclusterMapping('agents')).toBe('s1'); + expect(ks.getAllSystemSubclusterMappings()).toStrictEqual( + new Map([ + ['agents', 's1'], + ['io', 's2'], + ]), + ); + + ks.deleteSystemSubclusterMapping('agents'); + expect(ks.getSystemSubclusterMapping('agents')).toBeUndefined(); + }); + }); }); diff --git a/packages/ocap-kernel/src/store/methods/crank.test.ts b/packages/ocap-kernel/src/store/methods/crank.test.ts index 421a64e8e2..17571d3cad 100644 --- a/packages/ocap-kernel/src/store/methods/crank.test.ts +++ b/packages/ocap-kernel/src/store/methods/crank.test.ts @@ -2,7 +2,7 @@ import type { KernelDatabase } from '@metamask/kernel-store'; import { expect, describe, it, vi, beforeEach } from 'vitest'; import { getCrankMethods } from './crank.ts'; -import type { StoreContext } from '../types.ts'; +import type { CrankBufferItem, StoreContext } from '../types.ts'; describe('crank methods', () => { let context: StoreContext; @@ -229,4 +229,18 @@ describe('crank methods', () => { expect(crankMethods.isInCrank()).toBe(false); }); }); + + describe('crank output buffer', () => { + const item1 = { type: 'send' } as unknown as CrankBufferItem; + const item2 = { type: 'notify' } as unknown as CrankBufferItem; + + it('buffers outputs and flushes them, resetting the buffer', () => { + crankMethods.bufferCrankOutput(item1); + crankMethods.bufferCrankOutput(item2); + + expect(crankMethods.flushCrankBuffer()).toStrictEqual([item1, item2]); + // The buffer is reset after flushing. + expect(crankMethods.flushCrankBuffer()).toStrictEqual([]); + }); + }); }); diff --git a/packages/ocap-kernel/src/store/methods/gc.test.ts b/packages/ocap-kernel/src/store/methods/gc.test.ts index 1cca249533..148b79c8b3 100644 --- a/packages/ocap-kernel/src/store/methods/gc.test.ts +++ b/packages/ocap-kernel/src/store/methods/gc.test.ts @@ -280,5 +280,114 @@ describe('GC methods', () => { `unknown kernel promise ${kpid}`, ); }); + + it('decrements the slots of a settled promise before deleting it', () => { + const ko = kernelStore.initKernelObject('v1'); + kernelStore.addCListEntry('v1', ko, 'o+1'); + const before = kernelStore.getObjectRefCount(ko).recognizable; + + const [kpid] = kernelStore.initKernelPromise(); + // Resolving drops the baseline decider refcount to zero, enqueuing the + // promise; its fulfilled value references `ko` as a slot. + kernelStore.resolveKernelPromise(kpid, false, { + body: '#"resolved"', + slots: [ko], + }); + + kernelStore.collectGarbage(); + + expect(() => kernelStore.getKernelPromise(kpid)).toThrow( + `unknown kernel promise ${kpid}`, + ); + // The slot's recognizable count was decremented during collection. + expect(kernelStore.getObjectRefCount(ko).recognizable).toBe(before - 1); + }); + + it('deletes a rejected promise that has no slots', () => { + const [kpid] = kernelStore.initKernelPromise(); + kernelStore.resolveKernelPromise(kpid, true, { + body: '#"rejected"', + slots: [], + }); + + kernelStore.collectGarbage(); + + expect(() => kernelStore.getKernelPromise(kpid)).toThrow( + `unknown kernel promise ${kpid}`, + ); + }); + + it('schedules dropExport and retireExport for an unreachable object of a live vat', () => { + kernelStore.setVatConfig('v1', { bundleName: 'vat1' }); + const ko = kernelStore.initKernelObject('v1'); + // Export from v1 sets the reachable flag and bumps the refcounts to 1. + kernelStore.addCListEntry('v1', ko, 'o+1'); + kernelStore.setGCActions(new Set()); + + // Drop the reachable/recognizable counts to zero without clearing v1's + // reachable flag: the kernel now knows the object is unreachable but v1 + // does not yet. + kernelStore.decrementRefCount(ko, 'gc-test'); + + kernelStore.collectGarbage(); + + const actions = Array.from(kernelStore.getGCActions()); + expect(actions).toContain(`v1 dropExport ${ko}`); + expect(actions).toContain(`v1 retireExport ${ko}`); + }); + + it('skips unreachable objects owned by the kernel', () => { + // A fresh kernel object starts reachable; dropping it to zero enqueues it + // for collection while its owner remains 'kernel'. + const ko = kernelStore.initKernelObject('kernel'); + kernelStore.decrementRefCount(ko, 'gc-test'); + kernelStore.setGCActions(new Set()); + + kernelStore.collectGarbage(); + + // Kernel-owned objects are never dropped or retired. + expect(Array.from(kernelStore.getGCActions())).toStrictEqual([]); + expect(kernelStore.getObjectRefCount(ko)).toBeDefined(); + }); + + it('retires a still-recognizable orphan of a terminated vat', () => { + kernelStore.setVatConfig('v3', { bundleName: 'vat3' }); + const ko = kernelStore.initKernelObject('v3'); + kernelStore.addCListEntry('v3', ko, 'o+1'); + // Add a recognizable-only ref so that dropping the reachable ref leaves + // the object unreachable but still recognizable. + kernelStore.incrementRefCount(ko, 'gc-test', { onlyRecognizable: true }); + kernelStore.markVatAsTerminated('v3'); + kernelStore.setGCActions(new Set()); + + kernelStore.decrementRefCount(ko, 'gc-test'); + kernelStore.collectGarbage(); + + // The orphaned-but-recognizable object is retired (deleted) rather than + // dropped into the dead vat. + expect(kernelStore.getObjectRefCount(ko)).toStrictEqual({ + reachable: 0, + recognizable: 0, + }); + }); + + it('orphans and deletes an unreachable object owned by a terminated vat', () => { + kernelStore.setVatConfig('v2', { bundleName: 'vat2' }); + const ko = kernelStore.initKernelObject('v2'); + kernelStore.addCListEntry('v2', ko, 'o+1'); + kernelStore.markVatAsTerminated('v2'); + kernelStore.setGCActions(new Set()); + + kernelStore.decrementRefCount(ko, 'gc-test'); + + kernelStore.collectGarbage(); + + // The terminated vat receives no GC actions; the object is deleted. + expect(Array.from(kernelStore.getGCActions())).toStrictEqual([]); + expect(kernelStore.getObjectRefCount(ko)).toStrictEqual({ + reachable: 0, + recognizable: 0, + }); + }); }); }); diff --git a/packages/ocap-kernel/src/types.ts b/packages/ocap-kernel/src/types.ts index e5ea314b01..a5ef637068 100644 --- a/packages/ocap-kernel/src/types.ts +++ b/packages/ocap-kernel/src/types.ts @@ -32,12 +32,10 @@ import type { Json } from '@metamask/utils'; import { UnsafeJsonStruct } from '@metamask/utils'; import type { - RemoteMessageHandler, SendRemoteMessage, StopRemoteComms, - OnRemoteGiveUp, - OnIncarnationChange, - RemoteCommsOptions, + NetlayerSpecifier, + NetlayerHooks, } from './remotes/types.ts'; import { Fail } from './utils/assert.ts'; import { AllowedGlobalNameStruct } from './vats/endowments.ts'; @@ -528,29 +526,23 @@ export type PlatformServices = { */ sendRemoteMessage: SendRemoteMessage; /** - * Initialize network communications. + * Initialize network communications by selecting a netlayer implementation. * - * @param keySeed - The seed for generating this kernel's secret key. - * @param options - Options for remote communications initialization. - * @param options.relays - Array of the peerIDs of relay nodes that can be used to listen for incoming - * connections from other kernels. - * @param options.maxRetryAttempts - Maximum number of reconnection attempts. 0 = infinite (default). - * @param options.maxQueue - Maximum number of messages to queue per peer while reconnecting (default: 200). - * @param remoteMessageHandler - A handler function to receive remote messages. - * @param onRemoteGiveUp - Optional callback to be called when we give up on a remote. - * @param incarnationId - Unique identifier for this kernel instance. - * @param onIncarnationChange - Optional callback when a remote peer's incarnation changes. + * @param params - The initialization options. + * @param params.keySeed - The seed for generating this kernel's secret key. + * @param params.specifier - Which netlayer to use plus its `Json` config. + * @param params.hooks - Kernel-supplied callbacks (`handleMessage` plus + * optional `onRemoteGiveUp`/`onIncarnationChange`). + * @param params.incarnationId - This kernel's incarnation ID for the handshake. * @returns A promise that resolves once network access has been established * or rejects if there is some problem doing so. */ - initializeRemoteComms: ( - keySeed: string, - options: RemoteCommsOptions, - remoteMessageHandler: RemoteMessageHandler, - onRemoteGiveUp?: OnRemoteGiveUp, - incarnationId?: string, - onIncarnationChange?: OnIncarnationChange, - ) => Promise; + initializeRemoteComms: (params: { + keySeed: string; + specifier: NetlayerSpecifier; + hooks: NetlayerHooks; + incarnationId?: string; + }) => Promise; /** * Stop network communications. * diff --git a/packages/ocap-kernel/test/loopback-platform-services.ts b/packages/ocap-kernel/test/loopback-platform-services.ts index 42b06f0842..afc6f9102f 100644 --- a/packages/ocap-kernel/test/loopback-platform-services.ts +++ b/packages/ocap-kernel/test/loopback-platform-services.ts @@ -1,10 +1,8 @@ import { Logger } from '@metamask/logger'; import type { Netlayer, - OnIncarnationChange, - OnRemoteGiveUp, - RemoteCommsOptions, - RemoteMessageHandler, + NetlayerHooks, + NetlayerSpecifier, } from '@metamask/netlayer'; import { makeLoopbackNetlayer } from '@metamask/netlayer-loopback'; import type { LoopbackHub } from '@metamask/netlayer-loopback'; @@ -47,22 +45,18 @@ export function makeLoopbackPlatformServices( launch: vi.fn(), terminate: vi.fn(), terminateAll: vi.fn(), - initializeRemoteComms: async ( - keySeed: string, - _options: RemoteCommsOptions, - remoteMessageHandler: RemoteMessageHandler, - onRemoteGiveUp?: OnRemoteGiveUp, - incarnationId?: string, - onIncarnationChange?: OnIncarnationChange, - ): Promise => { + initializeRemoteComms: async (params: { + keySeed: string; + specifier: NetlayerSpecifier; + hooks: NetlayerHooks; + incarnationId?: string; + }): Promise => { + // The specifier's config is ignored: this fake always routes through the + // shared in-process hub (a live object that can't be `Json`). const created = await makeLoopbackNetlayer({ - keySeed, - incarnationId, - hooks: { - handleMessage: remoteMessageHandler, - onRemoteGiveUp, - onIncarnationChange, - }, + keySeed: params.keySeed, + incarnationId: params.incarnationId, + hooks: params.hooks, config: { hub }, logger: new Logger(), }); diff --git a/packages/omnium-gatherum/package.json b/packages/omnium-gatherum/package.json index 780a8aae53..d0dce6fa99 100644 --- a/packages/omnium-gatherum/package.json +++ b/packages/omnium-gatherum/package.json @@ -52,6 +52,7 @@ "@metamask/kernel-ui": "workspace:^", "@metamask/kernel-utils": "workspace:^", "@metamask/logger": "workspace:^", + "@metamask/netlayer-libp2p": "workspace:^", "@metamask/ocap-kernel": "workspace:^", "@metamask/streams": "workspace:^", "@metamask/superstruct": "^3.2.1", diff --git a/packages/omnium-gatherum/src/offscreen.ts b/packages/omnium-gatherum/src/offscreen.ts index f595ace5dd..bb2a503500 100644 --- a/packages/omnium-gatherum/src/offscreen.ts +++ b/packages/omnium-gatherum/src/offscreen.ts @@ -8,6 +8,7 @@ import { import { delay, isJsonRpcMessage } from '@metamask/kernel-utils'; import type { JsonRpcMessage } from '@metamask/kernel-utils'; import { Logger } from '@metamask/logger'; +import { libp2pNetlayerFactory } from '@metamask/netlayer-libp2p'; import type { SystemSubclusterConfig } from '@metamask/ocap-kernel'; import type { DuplexStream } from '@metamask/streams'; import { @@ -66,9 +67,13 @@ async function makeKernelWorker(): Promise< DuplexStream > { const workerUrlParams = createCommsQueryString({ - relays: [ - '/ip4/127.0.0.1/tcp/9001/ws/p2p/12D3KooWJBDqsyHQF2MWiCdU4kdqx4zTsSTLRdShg7Ui6CRWB4uc', - ], + netlayer: 'libp2p', + config: { + knownRelays: [ + process.env.RELAY_MULTIADDR ?? + '/ip4/127.0.0.1/tcp/9001/ws/p2p/12D3KooWJBDqsyHQF2MWiCdU4kdqx4zTsSTLRdShg7Ui6CRWB4uc', + ], + }, }); workerUrlParams.set('reset-storage', process.env.RESET_STORAGE ?? 'false'); @@ -110,15 +115,18 @@ async function makeKernelWorker(): Promise< JsonRpcMessage >(port, isJsonRpcMessage); - await PlatformServicesServer.make(worker as PostMessageTarget, (vatId) => - makeIframeVatWorker({ - id: vatId, - iframeUri: 'iframe.html', - getPort: initializeMessageChannel, - logger: logger.subLogger({ - tags: ['iframe-vat-worker', vatId], + await PlatformServicesServer.make( + worker as PostMessageTarget, + (vatId) => + makeIframeVatWorker({ + id: vatId, + iframeUri: 'iframe.html', + getPort: initializeMessageChannel, + logger: logger.subLogger({ + tags: ['iframe-vat-worker', vatId], + }), }), - }), + { netlayers: { libp2p: libp2pNetlayerFactory } }, ); return kernelStream; diff --git a/packages/omnium-gatherum/tsconfig.json b/packages/omnium-gatherum/tsconfig.json index 83fedfd08e..731356a52d 100644 --- a/packages/omnium-gatherum/tsconfig.json +++ b/packages/omnium-gatherum/tsconfig.json @@ -18,6 +18,7 @@ { "path": "../kernel-store" }, { "path": "../kernel-utils" }, { "path": "../logger" }, + { "path": "../netlayer-libp2p" }, { "path": "../ocap-kernel" }, { "path": "../streams" }, { "path": "../repo-tools" } diff --git a/tsconfig.build.json b/tsconfig.build.json index e3e024ac36..808ce721c6 100644 --- a/tsconfig.build.json +++ b/tsconfig.build.json @@ -16,6 +16,7 @@ { "path": "./packages/kernel-utils/tsconfig.build.json" }, { "path": "./packages/llm-bridge/tsconfig.build.json" }, { "path": "./packages/logger/tsconfig.build.json" }, + { "path": "./packages/netlayer-libp2p/tsconfig.build.json" }, { "path": "./packages/netlayer-loopback/tsconfig.build.json" }, { "path": "./packages/netlayer/tsconfig.build.json" }, { "path": "./packages/nodejs-test-workers/tsconfig.build.json" }, diff --git a/tsconfig.json b/tsconfig.json index ea515dfc3b..590f834ac5 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -32,6 +32,7 @@ { "path": "./packages/llm-bridge" }, { "path": "./packages/logger" }, { "path": "./packages/netlayer" }, + { "path": "./packages/netlayer-libp2p" }, { "path": "./packages/netlayer-loopback" }, { "path": "./packages/nodejs-test-workers" }, { "path": "./packages/ocap-kernel" }, diff --git a/tsconfig.packages.json b/tsconfig.packages.json index 78e532c3bb..81d9f4d842 100644 --- a/tsconfig.packages.json +++ b/tsconfig.packages.json @@ -19,6 +19,12 @@ "../kernel-store/src/sqlite/wasm.ts" ], "@metamask/logger/file-transport": ["../logger/src/file-transport.ts"], + "@metamask/netlayer-libp2p/nodejs": [ + "../netlayer-libp2p/src/nodejs/index.ts" + ], + "@metamask/netlayer-libp2p/relay": [ + "../netlayer-libp2p/src/relay/index.ts" + ], "@ocap/*": ["../*/src"] } } diff --git a/yarn.lock b/yarn.lock index a0b048ab12..b0ee39115a 100644 --- a/yarn.lock +++ b/yarn.lock @@ -2354,6 +2354,7 @@ __metadata: "@metamask/kernel-shims": "workspace:^" "@metamask/kernel-utils": "workspace:^" "@metamask/logger": "workspace:^" + "@metamask/netlayer-libp2p": "workspace:^" "@metamask/utils": "npm:^11.9.0" "@ocap/repo-tools": "workspace:^" "@ts-bridge/cli": "npm:^0.6.3" @@ -2439,12 +2440,10 @@ __metadata: resolution: "@metamask/kernel-node-runtime@workspace:packages/kernel-node-runtime" dependencies: "@arethetypeswrong/cli": "npm:^0.17.4" - "@chainsafe/libp2p-quic": "npm:^2.0.1" "@endo/captp": "npm:^4.4.8" "@endo/eventual-send": "npm:^1.3.4" "@endo/promise-kit": "npm:^1.1.13" "@libp2p/interface": "npm:3.1.1" - "@libp2p/tcp": "npm:11.0.14" "@libp2p/webrtc": "npm:6.0.15" "@metamask/auto-changelog": "npm:^5.3.0" "@metamask/eslint-config": "npm:^15.0.0" @@ -2456,6 +2455,7 @@ __metadata: "@metamask/kernel-store": "workspace:^" "@metamask/kernel-utils": "workspace:^" "@metamask/logger": "workspace:^" + "@metamask/netlayer-libp2p": "workspace:^" "@metamask/ocap-kernel": "workspace:^" "@metamask/streams": "workspace:^" "@ocap/repo-tools": "workspace:^" @@ -2716,21 +2716,11 @@ __metadata: resolution: "@metamask/kernel-utils@workspace:packages/kernel-utils" dependencies: "@arethetypeswrong/cli": "npm:^0.17.4" - "@chainsafe/libp2p-noise": "npm:^17.0.0" - "@chainsafe/libp2p-yamux": "npm:8.0.1" "@endo/captp": "npm:^4.4.8" "@endo/errors": "npm:^1.2.13" "@endo/exo": "npm:^1.5.12" "@endo/patterns": "npm:^1.7.0" "@endo/promise-kit": "npm:^1.1.13" - "@libp2p/autonat": "npm:3.0.14" - "@libp2p/circuit-relay-v2": "npm:4.1.7" - "@libp2p/crypto": "npm:5.1.14" - "@libp2p/identify": "npm:4.0.14" - "@libp2p/interface": "npm:3.1.1" - "@libp2p/ping": "npm:3.0.14" - "@libp2p/tcp": "npm:11.0.14" - "@libp2p/websockets": "npm:10.1.7" "@metamask/auto-changelog": "npm:^5.3.0" "@metamask/eslint-config": "npm:^15.0.0" "@metamask/eslint-config-nodejs": "npm:^15.0.0" @@ -2738,7 +2728,6 @@ __metadata: "@metamask/kernel-errors": "workspace:^" "@metamask/superstruct": "npm:^3.2.1" "@metamask/utils": "npm:^11.9.0" - "@multiformats/multiaddr": "npm:^13.0.1" "@ocap/repo-tools": "workspace:^" "@ts-bridge/cli": "npm:^0.6.3" "@ts-bridge/shims": "npm:^0.1.1" @@ -2757,7 +2746,6 @@ __metadata: eslint-plugin-prettier: "npm:^5.2.6" eslint-plugin-promise: "npm:^7.2.1" jsdom: "npm:^29.0.2" - libp2p: "npm:3.1.7" prettier: "npm:^3.5.3" rimraf: "npm:^6.0.1" ses: "npm:^1.14.0" @@ -2849,6 +2837,68 @@ __metadata: languageName: node linkType: hard +"@metamask/netlayer-libp2p@workspace:^, @metamask/netlayer-libp2p@workspace:packages/netlayer-libp2p": + version: 0.0.0-use.local + resolution: "@metamask/netlayer-libp2p@workspace:packages/netlayer-libp2p" + dependencies: + "@arethetypeswrong/cli": "npm:^0.17.4" + "@chainsafe/libp2p-noise": "npm:^17.0.0" + "@chainsafe/libp2p-quic": "npm:^2.0.1" + "@chainsafe/libp2p-yamux": "npm:8.0.1" + "@libp2p/autonat": "npm:3.0.14" + "@libp2p/bootstrap": "npm:12.0.15" + "@libp2p/circuit-relay-v2": "npm:4.1.7" + "@libp2p/crypto": "npm:5.1.14" + "@libp2p/identify": "npm:4.0.14" + "@libp2p/interface": "npm:3.1.1" + "@libp2p/peer-id": "npm:6.0.5" + "@libp2p/ping": "npm:3.0.14" + "@libp2p/tcp": "npm:11.0.14" + "@libp2p/utils": "npm:7.0.14" + "@libp2p/webrtc": "npm:6.0.15" + "@libp2p/websockets": "npm:10.1.7" + "@libp2p/webtransport": "npm:6.0.16" + "@metamask/auto-changelog": "npm:^5.3.0" + "@metamask/eslint-config": "npm:^15.0.0" + "@metamask/eslint-config-nodejs": "npm:^15.0.0" + "@metamask/eslint-config-typescript": "npm:^15.0.0" + "@metamask/kernel-errors": "workspace:^" + "@metamask/kernel-utils": "workspace:^" + "@metamask/logger": "workspace:^" + "@metamask/netlayer": "workspace:^" + "@metamask/superstruct": "npm:^3.2.1" + "@multiformats/multiaddr": "npm:^13.0.1" + "@ocap/repo-tools": "workspace:^" + "@ts-bridge/cli": "npm:^0.6.3" + "@ts-bridge/shims": "npm:^0.1.1" + "@types/node": "npm:^22.13.1" + "@typescript-eslint/eslint-plugin": "npm:^8.29.0" + "@typescript-eslint/parser": "npm:^8.29.0" + "@typescript-eslint/utils": "npm:^8.29.0" + "@vitest/eslint-plugin": "npm:^1.6.14" + depcheck: "npm:^1.4.7" + eslint: "npm:^9.23.0" + eslint-config-prettier: "npm:^10.1.1" + eslint-import-resolver-typescript: "npm:^4.3.1" + eslint-plugin-import-x: "npm:^4.10.0" + eslint-plugin-jsdoc: "npm:^50.6.9" + eslint-plugin-n: "npm:^17.17.0" + eslint-plugin-prettier: "npm:^5.2.6" + eslint-plugin-promise: "npm:^7.2.1" + libp2p: "npm:3.1.7" + prettier: "npm:^3.5.3" + rimraf: "npm:^6.0.1" + ses: "npm:^1.14.0" + turbo: "npm:^2.9.1" + typedoc: "npm:^0.28.1" + typescript: "npm:~5.8.2" + typescript-eslint: "npm:^8.29.0" + uint8arrays: "npm:^5.1.0" + vite: "npm:^8.0.6" + vitest: "npm:^4.1.3" + languageName: unknown + linkType: soft + "@metamask/netlayer-loopback@workspace:^, @metamask/netlayer-loopback@workspace:packages/netlayer-loopback": version: 0.0.0-use.local resolution: "@metamask/netlayer-loopback@workspace:packages/netlayer-loopback" @@ -2959,24 +3009,11 @@ __metadata: resolution: "@metamask/ocap-kernel@workspace:packages/ocap-kernel" dependencies: "@agoric/swingset-liveslots": "npm:0.10.3-u21.0.1" - "@chainsafe/libp2p-noise": "npm:^17.0.0" - "@chainsafe/libp2p-yamux": "npm:8.0.1" "@endo/errors": "npm:^1.2.13" "@endo/eventual-send": "npm:^1.3.4" "@endo/marshal": "npm:^1.8.0" "@endo/pass-style": "npm:^1.6.3" "@endo/promise-kit": "npm:^1.1.13" - "@libp2p/bootstrap": "npm:12.0.15" - "@libp2p/circuit-relay-v2": "npm:4.1.7" - "@libp2p/crypto": "npm:5.1.14" - "@libp2p/identify": "npm:4.0.14" - "@libp2p/interface": "npm:3.1.1" - "@libp2p/peer-id": "npm:6.0.5" - "@libp2p/ping": "npm:3.0.14" - "@libp2p/utils": "npm:7.0.14" - "@libp2p/webrtc": "npm:6.0.15" - "@libp2p/websockets": "npm:10.1.7" - "@libp2p/webtransport": "npm:6.0.16" "@metamask/auto-changelog": "npm:^5.3.0" "@metamask/eslint-config": "npm:^15.0.0" "@metamask/eslint-config-nodejs": "npm:^15.0.0" @@ -2994,7 +3031,6 @@ __metadata: "@metamask/streams": "workspace:^" "@metamask/superstruct": "npm:^3.2.1" "@metamask/utils": "npm:^11.9.0" - "@multiformats/multiaddr": "npm:^13.0.1" "@ocap/repo-tools": "workspace:^" "@scure/bip39": "npm:^2.0.1" "@ts-bridge/cli": "npm:^0.6.3" @@ -3011,7 +3047,6 @@ __metadata: eslint-plugin-n: "npm:^17.17.0" eslint-plugin-prettier: "npm:^5.2.6" eslint-plugin-promise: "npm:^7.2.1" - libp2p: "npm:3.1.7" multiformats: "npm:^13.3.6" prettier: "npm:^3.5.3" rimraf: "npm:^6.0.1" @@ -3020,7 +3055,6 @@ __metadata: turbo: "npm:^2.9.1" typescript: "npm:~5.8.2" typescript-eslint: "npm:^8.29.0" - uint8arrays: "npm:^5.1.0" vite: "npm:^8.0.6" vitest: "npm:^4.1.3" languageName: unknown @@ -3926,6 +3960,7 @@ __metadata: "@metamask/kernel-store": "workspace:^" "@metamask/kernel-utils": "workspace:^" "@metamask/logger": "workspace:^" + "@metamask/netlayer-libp2p": "workspace:^" "@metamask/ocap-kernel": "workspace:^" "@metamask/sdk": "npm:^0.34.0" "@metamask/smart-accounts-kit": "npm:^0.3.0" @@ -3976,6 +4011,7 @@ __metadata: "@metamask/kernel-ui": "workspace:^" "@metamask/kernel-utils": "workspace:^" "@metamask/logger": "workspace:^" + "@metamask/netlayer-libp2p": "workspace:^" "@metamask/ocap-kernel": "workspace:^" "@metamask/streams": "workspace:^" "@ocap/kernel-test": "workspace:^" @@ -4213,6 +4249,7 @@ __metadata: "@metamask/kernel-store": "workspace:^" "@metamask/kernel-utils": "workspace:^" "@metamask/logger": "workspace:^" + "@metamask/netlayer-libp2p": "workspace:^" "@metamask/ocap-kernel": "workspace:^" "@ocap/kernel-language-model-service": "workspace:^" "@ocap/nodejs-test-workers": "workspace:^" @@ -4398,6 +4435,7 @@ __metadata: "@metamask/kernel-ui": "workspace:^" "@metamask/kernel-utils": "workspace:^" "@metamask/logger": "workspace:^" + "@metamask/netlayer-libp2p": "workspace:^" "@metamask/ocap-kernel": "workspace:^" "@metamask/streams": "workspace:^" "@metamask/superstruct": "npm:^3.2.1"