feat(netlayer): extract @metamask/netlayer-libp2p + flip runtime injection (phase 4)

.depcheckrc.yml

@@ -51,7 +51,7 @@ ignores:
   - 'node-gyp'
 
   # Peer dependency of @metamask/kernel-shims due to its /endoify-node
-  # export, used by @metamask/kernel-node-runtime
+  # export, used by Node entrypoints that run the libp2p netlayer
   - '@libp2p/webrtc'
 
   # These are peer dependencies of various modules we actually do

docs/plans/netlayer/master.md

@@ -359,8 +359,71 @@ receive, incarnationId)` / `unregister` / `getIncarnation` / `deliver(from, to,
   `initTransport` export and all libp2p deps from ocap-kernel. Relocate kernel-shims' bare
   `import '@libp2p/webrtc'` (endoify-node.js) alongside the libp2p netlayer — flag for review.
   Verify: full CI, extension e2e, `kernel-test` remote-comms integration, `ocap relay` smoke test.
-
-**Phase 5 — `@metamask/netlayer-websocket`.** ~3–4 days.
+  **DONE** (issue #968, PR #973 on branch `rekm/netlayer-4`, stacked on `rekm/netlayer-3`).
+
+Landed decisions / deviations from the phase-4 plan (these are the facts Phase 5 must build on):
+
+- **`@metamask/netlayer-libp2p` (new, `0.1.0`, published).** Exports:
+  - `.` (browser default): `libp2pNetlayerFactory` (a `NetlayerFactory` — typed `<Json>`, not
+    `<Libp2pNetlayerConfig>`, so it fits `NetlayerRegistry = Record<string, NetlayerFactory<Json>>`;
+    it `assert`s the config to `Libp2pNetlayerConfig` internally), plus `makeLibp2pNetlayer`
+    (re-exported for consumers that must not reach `./nodejs` — accepts an `@internal`
+    `directTransports`), `Libp2pNetlayerConfig`/`Libp2pNetlayerConfigStruct`, `DirectTransport`.
+  - `./nodejs`: `nodejsLibp2pNetlayerFactory` (builds QUIC/TCP direct transports from
+    `config.directListenAddresses`) and `buildDirectTransports`.
+  - `./relay`: `startRelay` (moved from `@metamask/kernel-utils/libp2p`; `RELAY_LOCAL_ID = 200`,
+    ports 9001/9002).
+  - `error-mapper.ts`: `mapLibp2pReadError`, `isIntentionalDisconnect`, `isRetryableLibp2pError`
+    (neutral `isRetryableNetworkError` ∪ libp2p name/message sniffing), and `mapLibp2pDialError`
+    (maps retryable-by-name libp2p dial errors → neutral `ChannelResetError`; passes Node-code and
+    neutral errors through so the engine's permanent-failure detection is unchanged).
+    `connection-factory.dial` applies `mapLibp2pDialError`; `kernel-errors`'
+    `isRetryableNetworkError`/`getNetworkErrorCode` were reduced to the neutral subset.
+  - Deps: `@metamask/{netlayer,kernel-errors,kernel-utils,logger,superstruct}`,
+    `@multiformats/multiaddr`, `uint8arrays`, and the full `@libp2p`/`@chainsafe`/`libp2p` family.
+    `sideEffects: false`.
+- **Contract shapes (LANDED).** `NetlayerSpecifier = { netlayer: string; config: Json }`;
+  `NetlayerRegistry = Record<string, NetlayerFactory>` (config `Json`). `PlatformServices.initializeRemoteComms`
+  is `(params: { keySeed; specifier: NetlayerSpecifier; hooks: NetlayerHooks; incarnationId? }) => Promise<void>`.
+  `NodejsPlatformServices` ctor and `PlatformServicesServer` ctor/`make` take a `netlayers: NetlayerRegistry`
+  (`PlatformServicesServer` as `args: { netlayers; logger? }`). `PlatformServicesClient.initializeRemoteComms`
+  takes the same options bag and keeps hooks locally (only `keySeed` + `specifier` + `incarnationId` cross RPC).
+  The RPC hook `InitializeRemoteComms` stayed **positional** `(keySeed, specifier, incarnationId?)`.
+- **Config convention.** The kernel understands exactly one netlayer-config key by convention:
+  `knownRelays: string[]` — the opaque hint pool it persists and re-injects.
+  `remote-comms.ts` merges bootstrap `config.knownRelays` into the store then overwrites
+  `config.knownRelays` with the full persisted pool; `mnemonic` never enters `specifier.config`.
+  `DEFAULT_NETLAYER = 'libp2p'` (in `remotes/kernel/remote-comms.ts`) covers omitted specifiers.
+- **RPC structs.** `rpc/platform-services/initializeRemoteComms.ts` and
+  `rpc/kernel-control/init-remote-comms.ts` validate `specifier.config` with `JsonStruct`
+  (`@metamask/utils`); the kernel-control struct drops `mnemonic`.
+- **`comms-query-string`.** `createCommsQueryString({ netlayer, config, maxQueue?, ackTimeoutMs?,
+maxUrlRelayHints?, maxKnownRelays? })` sets `netlayer` + `netlayer-config` (JSON) + numeric params;
+  `parseCommsQueryString`/`getCommsParamsFromCurrentLocation` return `RemoteCommsOptions` with a
+  `specifier` (omitted when no `netlayer` param). Extension/omnium `offscreen.ts` register
+  `{ libp2p: libp2pNetlayerFactory }` and pass a specifier; the multiaddr reads `process.env.RELAY_MULTIADDR`.
+- **ocap-kernel is libp2p-free.** `packages/ocap-kernel/package.json` has zero
+  `@libp2p`/`@chainsafe`/`@multiformats`/`libp2p` deps (plain `multiformats` + `@scure/bip39` stay).
+  `initTransport`, `DirectTransport`, and `ConnectionFactoryOptions` are gone; `RemoteCommsOptions`
+  is the kernel-owned subset + `specifier`.
+- **DEVIATION — endoify-node `@libp2p/webrtc` relocation (§3.7) DEFERRED.** kernel-shims still owns
+  the pre-lockdown `import '@libp2p/webrtc'` in `endoify-node.js` (peer dep retained). Every
+  `@metamask/kernel-node-runtime` importer transitively loads `@libp2p/webrtc` at module-eval (via
+  `make-kernel` → the nodejs factory → `connection-factory`), so relocating the pre-lockdown import
+  means repointing ~7 packages with a silent-lockdown-failure risk; it is not required for the
+  ocap-kernel zero-libp2p gate. Left as a Phase 6 / future-work follow-up (Phase 5 is deferred —
+  see below).
+- **DEVIATION — browser `PlatformServicesServer` does not capture `getListenAddresses`** (the client
+  returns `[]` for Node-only direct transport; capturing it would be dead code).
+- **Coverage.** Every touched package meets/exceeds its pre-phase baseline (netlayer-libp2p new at
+  96.75% stmts / 94.11% branch; kernel-errors raised to 100%); extraction dips were closed with
+  behavior-asserting tests. Extension `offscreen.js` bundle unchanged (~1,320 kB / 435 kB gzip).
+
+**Phase 5 — `@metamask/netlayer-websocket`.** ~3–4 days. **DEFERRED — not executed in this
+effort; remains future work.** Per a user directive after Phase 4, the effort skips straight from
+Phase 4 to Phase 6 (cleanup + docs). The plan below stands as the future-work description; when it
+is eventually picked up it must first be reconciled against the landed state of Phases 1–4 **and 6**
+(see the deferral banner in `phase-5.md`).
 `ChannelProvider` over WebSocket: client dials `wss://` hints (browser + node); `./nodejs` server
 authenticates peers via mutual Ed25519 challenge-signature handshake (new security-sensitive
 code — keep minimal, flag for crypto review). Topology decision: the hub is a **kernel
@@ -369,12 +432,13 @@ unsupported (that's what the libp2p netlayer's relays are for). Reuses `makeChan
 shared machinery — this is the proof the abstraction isn't libp2p-shaped. Parameterize
 `kernel-test` integration over netlayers (loopback always; libp2p/websocket where infra allows).
 
-**Phase 6 — Cleanup + docs.** ~1–2 days.
+**Phase 6 — Cleanup + docs.** ~1–2 days. **Follows Phase 4 directly** (Phase 5 deferred).
 Optional relay→"location hints" terminology rename in store/`RemoteIdentity`, glossary entries,
 changelogs (update-changelogs skill), "writing a netlayer" doc, note iroh as the next (Node-only)
 implementation with a pointer to #968.
 
-**Total: ~16–22 dev-days.**
+**Total: ~16–22 dev-days** across all six phases; **~13–18 dev-days for the executed scope**
+(Phases 1–4 + 6, with Phase 5 deferred).
 
 ## Risks
 
@@ -398,8 +462,8 @@ implementation with a pointer to #968.
 - After Phase 4: run the extension example (`@ocap/extension` `test:e2e`) and a two-kernel Node
   scenario over the relay (`ocap relay start` + `kernel-test/src/remote-comms.test.ts`) to confirm
   ocap-URL redemption and message delivery across the new injection path.
-- After Phase 5: two-kernel WebSocket hub-and-spoke integration test; loopback-based multi-kernel
-  test in-process.
+- After Phase 5 (deferred): two-kernel WebSocket hub-and-spoke integration test; loopback-based
+  multi-kernel test in-process. The loopback-based multi-kernel coverage already lands with Phase 3.
 - Confirm `packages/ocap-kernel/package.json` has zero `libp2p`/`@libp2p`/`@chainsafe`/
   `@multiformats` deps at the end of Phase 4. (Plain `multiformats` is not libp2p and stays —
   `base58btc` is the neutral identity/oid encoding.)

docs/plans/netlayer/phase-5.md

@@ -1,5 +1,23 @@
 # Phase 5 — `@metamask/netlayer-websocket`
 
+> **⚠️ DEFERRED — NOT BEING EXECUTED IN THIS EFFORT.** Per a user directive after Phase 4, the
+> pluggable-netlayer effort skips straight from Phase 4 to Phase 6 (cleanup + docs). This document
+> is preserved as the future-work plan for the WebSocket netlayer. **Before it is picked up, it must
+> first be reconciled against the landed state of Phases 1–4 _and_ Phase 6** — cross-phase symbol
+> names, signatures, package layout, and the terminology/docs that Phase 6 lands (e.g. any
+> relay→"location hints" rename, the "writing a netlayer" guide) all take precedence over the
+> sketches below. The reconciliation blockquote that follows was written against Phases 1–4 only and
+> predates the Phase 6 cleanup.
+>
+> Assumptions Phase 4 already invalidated that this plan still needs updating for when resumed
+> (brief, non-exhaustive): `ChannelProvider.dial` is 3-arg `(peerId, hints, withRetry)` (the §4
+> sketch shows 2-arg); the libp2p error-name/dial mapping now lives in
+> `@metamask/netlayer-libp2p/src/error-mapper.ts` (not kernel-errors), which the WS `errors.ts`
+> should mirror; `@metamask/netlayer` still does **not** re-export `@metamask/superstruct` or ship an
+> `EngineOptionsStruct`/sign-verify identity wrappers — those must be added when this phase resumes;
+> and a pure-websocket Node deployment still pulls `@libp2p/webrtc` transitively because the
+> pre-lockdown endoify import relocation was deferred (see master.md Phase 4 deviations).
+
 Implementation plan for the fifth PR in the pluggable-netlayer effort
 ([issue #968](https://github.com/MetaMask/ocap-kernel/issues/968); master plan at
 [`master.md`](./master.md)).
@@ -13,17 +31,33 @@ runtimes accept a `NetlayerRegistry` and per-kernel `NetlayerSpecifier` (`{ netl
 config: Json }`); `ocap-kernel` is libp2p-free; identity is a neutral base58btc-multibase Ed25519
 public key.
 
-Where this plan references machinery, it points at the **future `@metamask/netlayer` location**
-but the behaviour to reuse is what currently lives in
-`packages/ocap-kernel/src/remotes/platform/` (`transport.ts`, `reconnection.ts`,
-`peer-state-manager.ts`, `rate-limiter.ts`, `validators.ts`, `handshake.ts`,
-`channel-utils.ts`). Read those to understand the contract you are implementing against.
-
-> **Revision required before execution.** This plan was written before Phases 1–4 landed.
-> The type sketches below (notably `ChannelProvider` — Phase 1 lands
-> `dial(peerId, hints, withRetry)`, not the 2-arg form shown) are indicative; the contract
-> actually exported by the landed `@metamask/netlayer` wins, and this document should be
-> updated against it first.
+The machinery to reuse now lives in **`@metamask/netlayer/src/`** (moved out of
+`ocap-kernel` in Phase 3): `channel-netlayer.ts` (the `makeChannelNetlayer` engine),
+`reconnection.ts`, `reconnection-lifecycle.ts`, `peer-state-manager.ts`, `rate-limiter.ts`,
+`validators.ts`, `handshake.ts`, `channel-utils.ts`, `identity.ts`, `constants.ts`. The libp2p
+`ChannelProvider` to mirror is `@metamask/netlayer-libp2p/src/connection-factory.ts` (with its
+`make-libp2p-netlayer.ts` wiring and `error-mapper.ts`). Read those to understand the contract you
+are implementing against.
+
+> **Reconciled against landed Phases 1–4.** The contract exported by `@metamask/netlayer` wins over
+> any sketch below. Key landed facts (see master.md's Phase 1–4 "Landed decisions" blocks):
+> `ChannelProvider.dial(peerId, hints, withRetry)` (3-arg) and carries `readonly peerId`;
+> `makeChannelNetlayer({ provider, hooks, options, logger, stopController })` is synchronous and
+> shares one `AbortController` with the provider; `NetlayerSpecifier = { netlayer: string; config: Json }`,
+> `NetlayerRegistry = Record<string, NetlayerFactory>` (config `Json` — validate/narrow
+> inside the factory, like `libp2pNetlayerFactory` does); `PlatformServices.initializeRemoteComms`
+> is the options bag `{ keySeed, specifier, hooks, incarnationId? }` and the runtimes take a
+> `netlayers: NetlayerRegistry`. The websocket netlayer's config must be `Json` (it crosses
+> postMessage); the kernel injects `config.knownRelays` (its one convention key) — for a hub-and-
+> spoke websocket topology, treat those opaque hint strings as `wss://` hub URLs.
+>
+> **Follow-up carried from Phase 4:** the kernel-shims `@libp2p/webrtc` pre-lockdown endoify import
+> was NOT relocated in Phase 4 (deferred — see master.md). A pure-websocket Node deployment still
+> pulls `@libp2p/webrtc` transitively through `@metamask/kernel-node-runtime`'s default registry;
+> if Phase 5 wants a libp2p-free websocket-only Node runtime, that relocation (and making the
+> default registry configurable without importing the libp2p factory) must be tackled here or in
+> Phase 6. `@metamask/netlayer` re-added `@metamask/superstruct` is still pending (Phase 3 dropped
+> it as unused); add it back when the websocket config structs need it.
 
 ---