Skip to content

Latest commit

 

History

History
370 lines (307 loc) · 30.8 KB

V1.md

File metadata and controls

370 lines (307 loc) · 30.8 KB

OpenComponents v1 — Re-architecture Plan

Status: Draft · Generated 2026-07-26 · Progress last synced against merged and open PRs 2026-08-10

Legend: ✅ Done · 🚧 In progress · ⬜ Not started

This document describes the full re-architecture we intend to ship as OpenComponents v1. It is the "breaking swing": the changes that genuinely require a major version. Non-breaking modernization (promise APIs added additively, internal ESM migration, dependency hygiene) does not need to wait for v1 — it lands incrementally on the 0.x line first, so that v1 itself is mostly removals plus the opt-in modern runtime.


1. Guiding principle: the three breakage layers

Every proposed change is classified by who has to do work and how much. This ranking is the non-negotiable frame for the whole release.

Layer What it is Acceptability Cost to user
Registry Everything about running the registry: the programmatic Registry API (start/close/register), plugins, storage/metadata/server adapters, registry config, the CLI, the oc/oc-registry packages ✅ Most acceptable You change the registry deployment and everything works
App A consuming web app's contact with OC: the browser oc-client.js (the window.oc API, <script> include, auto-injected importmap, <oc-component> markup) and the node SSR oc-client it imports ⚠️ Acceptable if worth it If you run 50 web apps you must update all 50 before they work again
Component How an individual component is authored, compiled, packaged, rendered 🚫 Unacceptable to break A company with 3000 components would have to migrate all of them

Two clarifications that drive the whole plan:

  • "App" is not the registry's programmatic API. new Registry(opts).start(), plugins, and adapters are code that runs the registry — changing them is a registry-layer change, the most acceptable kind. Almost all of the modernization work lands here.
  • The browser oc-client.js contract is app-layer and is protected like components. A web app embeds it via a <script> tag and renders <oc-component>. It must stay fully drop-in: no importmap burden, no window.oc API changes, no markup changes. The only intentional app-layer break in v1 is the node SSR oc-client going promise-only/ESM — a one-time update per SSR app.

Every change below is tagged [REGISTRY], [APP], or [COMPONENT-ADDITIVE].


2. The frozen component contract (must never break)

These surfaces are immutable because ~3000+ already-compiled components have them baked into storage and cannot be re-compiled:

  • oc.files.* shapes in compiled package.json: dataProvider {type,src,hashKey,size}, template {type,src,hashKey,version,minOcVersion,size}, static, env, imports.
  • oc.packaged / oc.date / oc.version / oc.container / oc.renderInfo / oc.parameters / oc.plugins semantics.
  • Data provider contract: module.exports.data = function(context, callback) with node-style callback(err, data).
  • The VM sandbox globals allow-list exposed to data providers (require (wrapped), module, exports, console, setTimeout, Buffer, Error, AbortController, AbortSignal, Promise, Date, Symbol, URL, URLSearchParams, crypto, fetch, process.env subset).
  • The context object keys passed to data(context, cb) (action, acceptLanguage, baseUrl, env, params, plugins, renderComponent(s), requestHeaders, requestIp, setEmptyResponse, staticPath, setHeader, setCookie, templates, streamSymbol).
  • Auto-injected props for no-data-provider components (_staticPath, _baseUrl, _componentName, _componentVersion).
  • Accept-header render modes: application/vnd.oc.rendered+json, application/vnd.oc.unrendered+json, application/vnd.oc.info+json.
  • Legacy template runtime rendering for bare jade / handlebars types (see §5.3).
  • Component name grammar ([a-zA-Z0-9\-_], not _package) and semver versions.

Anything not on this list is fair game for the registry layer.


2b. The frozen browser-client (app) contract (must not break)

A web app's only real coupling to OC is the browser client and the markup it drives. In v1 the browser oc-client.js stays fully drop-in — treated with the same protection as components. The following are frozen:

  • Delivery / include: served at the same URL ({registry}/oc-client/client.js) and usable via a plain <script> tag. No ESM module-script requirement, no importmap the host app has to author — the client keeps injecting the importmap itself (oc-client.js type: 'importmap' head injection).
  • Global: window.oc remains the entry point with a stable public API (oc.ready, oc.render, oc.renderByHref, oc.renderNestedComponent, oc.events, oc.build, oc.conf, etc.).
  • Markup: the <oc-component> tag and its attributes (data-rendered, data-rendering, data-version, data-id, data-hash, data-name) and the oc:rendered / oc:error events are unchanged.
  • ESM component loading stays internal: the client resolves and import()s ESM component bundles itself; the host page does nothing new.

Browser-client internals may be modernized freely as long as none of the above changes. This is the concrete answer to "will apps have to add an importmap or re-touch their pages?" — no.


3. v1 pillars

Registry-layer core (the bulk of the work — most acceptable to break):

  1. Promise-first, callback-free registry API — remove the universalify dual API. [REGISTRY]
  2. ESM-only registry packages with proper exports maps — no CJS build. [REGISTRY]
  3. The HTTP server adapter becomes the real boundary — hooks/config speak OcRequest/OcResponse instead of leaking Express types. [REGISTRY]
  4. Promise-only adapters + metadata store as source of truth — drop the legacy callback storage-adapter shim; bundle a default metadata store; flat-file components.json becomes export-only legacy. [REGISTRY]
  5. Package split — oc-registry, oc-cli, oc-core, with oc as a thin umbrella. [REGISTRY]
  6. Platform + config cleanup — Node >=22 for the registry; remove stale config knobs and the redundant CLI publish gate. [REGISTRY]

Component layer (additive + opt-in only — never break existing components): 7. Opt-in modern component runtime — ESM-native authoring with async data(context) and typed context, blessed as the default for new components. [COMPONENT-ADDITIVE]

App layer (protect the web apps; one deliberate break): 8. Browser oc-client.js stays fully drop-in — no app changes required. [APP] (guarantee) 9. Coordinated node oc-client (SSR) v1 — promise API + TS types + ESM; the one intentional per-app update. [APP]


4. Changes by layer

4.1 Registry layer [REGISTRY] — most acceptable to break

# Change Rationale Migration (registry operator) Status
R1 Promise-only registry API registry.start/close/register, plugin register, and adapter methods are callback-based behind a universalify shim. This is registry-boot code, not app code. await registry.start() etc.; callback signatures removed. 🚧 In progress — the 0.x additive start/close/register precursor is implemented in #1539 (tracked in #1528). Promise-based plugin registration landed in #1542 (callback still supported with a deprecation warning). Promise listen()/close() were added to the Express and Fastify HTTP server adapters, and registry start/close internals now go through the promise path, in #1544. The v1 callback removal remains open.
R2 ESM-only registry packages + exports map The registry runs as an app you deploy; ESM-only is an acceptable registry break and kills dual-build complexity. Locks down deep imports. Registry deployment uses ESM (or dynamic import()); only documented entry points importable. ⬜ Not started
R3 Adapter-native hook/config types beforePublish, publishValidation, conf.routes handlers, and Authentication leak Express Request/Response; a global Express augmentation ships in types. Hooks receive OcRequest/OcResponse; Express becomes one adapter (Fastify, etc.). Express-compat shim provided. ✅ Done — HttpServerAdapter interface + in-core Express adapter (default) and a first-class oc-fastify-server-adapter package are shipped (#1507, #1508, #1509, #1510, #1511, #1512, #1513, #1518). Follow-up hardening (trailing-slash routing default) tracked in #1515, partially landed via #1521.
R4 Package split Monorepo already; a major is the moment. oc-registry (server), oc-cli (tooling), oc-core (shared types/runtime); oc = thin umbrella/CLI. Update imports once (import { Registry } from 'oc-registry'). oc still installs the CLI. 🚧 In progress — Turborepo/Changesets monorepo foundation (#1476); storage adapters (#1505) and metadata adapters (#1503) already extracted into their own packages/*. The actual oc-registry/oc-cli/oc-core split of the core package has not started.
R5 Promise-only plugins + typed errors Plugin register(options, deps, next) is callback-style; code throws raw strings in places. Plugins return promises; errors become typed Error subclasses. 🚧 In progress — plugins can now return a promise from register (#1542); the legacy callback form still works with a one-time deprecation warning. Typed errors and the v1 callback removal remain open.
R6 Remove global mutable state events-handler subscriptions and plugins-initialiser deferredLoads are process-global, so multiple registries in one process interfere. Per-instance event/plugin state. Subtle behavior change; documented. 🚧 In progress — plugins-initialiser's deferredLoads is now scoped per init() call instead of module-level, so concurrent registries no longer share deferred plugin state (#1543). events-handler subscriptions are still process-global.
R7 Metadata store = source of truth; bundle a default The metadata-adapter subsystem is newer and superior; flat-file is legacy. A bundled default keeps zero-config working. No-config registries get the default store automatically. 🚧 In progress — pluggable metadata store landed as opt-in (#1503); storage-only remains the default. Bundling a default store + making it the source of truth is still open.
R8 Flat-file components.json = export-only legacy It's now a projection of the metadata store. File is still exported for legacy consumers via exportLegacyFiles; no data loss. ⬜ Not started (depends on R7)
R9 Promise-only storage/metadata adapters Drop the callback-adapter conversion shim. Adapter authors return promises; operators use current promise-based adapter versions. ⬜ Not started
R10 Extract legacy jade/handlebars runtime to an opt-in compat package Keep core lean; old components must still render. Operators still serving pre-oc-template-* components install/enable the compat package (registry-config action, not a component rewrite). ⬜ Not started
R11 Configurable CORS / security headers; keep URLs + Accept contract CORS is hardcoded today; clients/components depend on route shapes so URLs stay. New optional cors config; defaults preserve today's behavior. No URL versioning. ✅ Done — optional registry cors config (origin, credentials, allowedHeaders, methods) landed in #1545, with config validation and the existing default headers preserved exactly when unset.
R12 Remove the oc-cli-version publish gate Dates to fcf87069 ("Added preventing old oc versions to publish") when CLI + registry shipped lockstep. It rejects any CLI whose major.minor is behind the registry — a 1.0.0 registry would reject all v0.x CLIs on publish. template.minOcVersion + package-structure validation now cover the real need. Publishing no longer fails on CLI semver skew. Keep node-version + template.minOcVersion checks. ⬜ Not started — gate still present in registry/routes/publish.ts
R13 Remove s3 shortcut + refreshInterval Legacy conveniences; s3 hardcodes require('oc-s3-storage-adapter'), refreshInterval overlaps pollingInterval. Use explicit storage.adapter/storage.options; pollingInterval replaces refreshInterval. ⬜ Not started — both still present in options-sanitiser.ts
R14 Fix tarExtractMode (766 → 0o766) Long-standing octal bug. None expected; document corrected effective mode. ⬜ Not started — default is still decimal 766 in options-sanitiser.ts
R15 discovery object-only Drop the boolean back-compat branch. discovery: true → discovery: { ui: true, api: true, ... }. ⬜ Not started — boolean branch still handled in options-sanitiser.ts
R16 Remove oc.json mocks config Deprecated in favor of development.plugins. Move mocks.plugins → development.plugins. ⬜ Not started — mocks still read in ocConfig.ts / get-mocked-plugins.ts
R17 Adapter-neutral error codes Hardcoded EXPRESS_ERROR event code leaks the transport. Emit adapter-neutral codes; document the event payload change. ✅ Done — server adapter failures emit SERVER_ERROR; the { code, message } event shape and original message are preserved on the 0.x line.
R18 Node >=22 for the registry >=18 is EOL; unblocks modern ESM/APIs. Run the registry on Node 22+. ⬜ Not started — packages/oc/package.json still declares "node": ">=18"
R19 Align programmatic API with CLI Missing clean, validate, registry.migrate-metadata. Additive; consistent surface. ⬜ Not started

4.2 App layer [APP] — protect web apps; one deliberate break

# Change Rationale Migration (web app owner) Status
P1 Browser oc-client.js stays fully drop-in (guarantee, not a change) A web app embeds it and renders <oc-component>. Breaking it means editing every app. Nothing. Same script URL, stable window.oc API, auto-injected importmap, unchanged <oc-component> markup (see §2b). ✅ Holding — guarantee upheld; the browser client was extracted into its own oc-client-browser package (#1478) with drop-in behavior preserved (e.g. #1516 fixed a DOM-move unmount regression, #1521 started removing trailing slashes from oc.build hrefs).
P2 Node SSR oc-client → promise API + TS types + ESM Separate repo, callback-based, untyped. This is the single intentional app-layer break, done once per SSR app. await client.renderComponent(...); adopt ESM/types. Smoothed by a 0.x callback shim + deprecation warnings. ⬜ Not started (lives in the separate opencomponents/oc-client repo)
P3 Node oc-client keeps a conservative engine floor (>=20) Don't force SSR apps to upgrade their Node runtime just to consume components. SSR apps stay on Node 20+ even though the registry requires 22+. ⬜ Not started

4.3 Component layer [COMPONENT-ADDITIVE] — additive + opt-in only

# Change Rationale Guarantee Status
C1 Opt-in modern runtime ESM-native authoring path, blessed default for oc init. New capability only. Old components keep the frozen contract in §2. ⬜ Not started
C2 async data(context) for new components Promise-based data providers for the modern runtime. Old data(context, callback) providers still execute via the legacy path — forever. ⬜ Not started
C3 Typed context First-class TS types for the modern context object. Additive; legacy context keys unchanged. ⬜ Not started
C4 Additive oc.* / OcParameter fields as needed Same pattern as the earlier enum addition. Additive only; never rename/remove existing fields. ⬜ Not started
C5 vm sandbox unchanged Sandbox globals are part of the frozen contract. No change. Stronger isolation is post-v1 research, must stay contract-compatible. ✅ Holding — no changes made to the sandbox

5. Notes on the trickiest surfaces

5.1 Why removing the CLI version gate is safe [REGISTRY]

The gate compares CLI major.minor to registry major.minor and rejects "old" CLIs. It existed when CLI and registry were one lockstep package. Today publishing already validates package structure, node-version, and template.minOcVersion (a real, per-template forward-compat signal). The semver gate now only produces upgrade friction and would make a 1.0.0 registry reject every existing v0.x publisher. Remove it; keep the substantive validators.

5.2 Metadata-first without a config break [REGISTRY]

Bundling a default metadata adapter means existing zero-config registries keep working: they transparently get the default store, and the flat-file export path keeps legacy file consumers happy. Operators who already configured a metadata adapter are unaffected.

5.3 Legacy templates as an opt-in compat package [REGISTRY]

Old components identify their template by the bare strings jade / handlebars. The runtime mapping and special-cased rendering must keep working. In v1 this moves out of core into a dedicated compat package. Old components still render; operators serving them install the compat package — a registry-config action, never a component migration.

5.4 Express as one adapter, not the contract [REGISTRY]

The HttpServerAdapter abstraction already exists but public hooks/config still leak Express types (plus a global Express augmentation in the shipped types). v1 finishes the job: hooks speak OcRequest/OcResponse, Express becomes a swappable adapter, and the Fastify adapter becomes first-class. An Express-compat shim eases the one-time registry migration. None of this touches web apps — it's registry-boot code.

5.5 The importmap question, answered [APP]

The browser client already injects the importmap itself. v1 keeps it that way: web apps never author an importmap or change their <script>/markup. Any ESM push happens inside the client and in the (opt-in) component authoring path, not in the host page.


6. Two-track sequencing

Track 1 — 0.x (non-breaking, ships continuously, now)

Land everything that can be additive so v1 becomes mostly removals:

  1. 🚧 Add promise-returning variants of registry.start/close/register, plugins, adapters, and the node oc-client (keep callbacks working, emit deprecation warnings). — tracked in #1528 (registry start/close/register), implemented in #1539. Promise plugin registration landed in #1542, and promise listen()/close() on the Express/Fastify HTTP server adapters (plus migrating registry start/close internals to the promise path) landed in #1544 — both keep the callback form working with a deprecation warning. The node oc-client promise variant remains open.
  2. ⬜ Migrate registry internals to ESM incrementally. — not started.
  3. ⬜ Introduce the modern component runtime as opt-in and make it the oc init default, while keeping legacy authoring fully supported. — not started.
  4. ✅ Dependency hygiene: replace colors (unmaintained), drop unused async, audit ESM-only deps against the build, prune old callback micro-deps (getport, read, try-require, nice-cache). — done via #1531 (colors → picocolors, async/@types/async removed, ESM-dependency audit documented).
  5. 🚧 Emit deprecation warnings for: callback APIs, s3 shortcut, refreshInterval, boolean discovery, oc.json mocks, bare jade/handlebars on oc init, and the callback node oc-client API. — the central deprecate() utility plus warnings for s3, refreshInterval, boolean discovery, and oc.json mocks are merged via #1532 (tracked in #1529). The callback registry API warning is implemented in #1539, plugin and HTTP-adapter callback warnings in #1542 and #1544, and bare jade/handlebars on oc init in #1546 (points users to oc-template-es6; explicit oc-template-jade/oc-template-handlebars selections stay silent). Only the callback node oc-client API warning remains open (lives in the separate opencomponents/oc-client repo).

Additional registry-layer groundwork landed ahead of/alongside the plan above (see §4.1 for detail): the HttpServerAdapter abstraction with Express (default) and Fastify adapters (R3, ✅ done — #1507–#1513, #1518); an opt-in pluggable metadata store (R7, 🚧 in progress — #1503); and monorepo + storage/metadata adapter package extraction (R4, 🚧 in progress — #1476, #1505).

Track 2 — v1 (breaking, the removals)

Registry:

  1. 🚧 Delete the universalify dual API — promise-only registry API + plugins + adapters. — 0.x additive promise variants for start/close/register, plugin registration (#1542), and HTTP adapter listen()/close() (#1544) are in place alongside deprecation-warned callbacks; the actual v1 removal of the callback paths has not started.
  2. ⬜ Ship ESM-only registry packages + exports maps; execute the package split.
  3. 🚧 Switch hooks to adapter-native types; remove global Express augmentation. — adapter-native OcRequest/OcResponse types and the Express/Fastify adapters are shipped (R3, done); the global Express type augmentation has not been removed yet.
  4. 🚧 Metadata store as source of truth with a bundled default; flat-file = export-only. — opt-in metadata store shipped (R7); not yet the bundled default/source of truth.
  5. ⬜ Extract legacy jade/handlebars into the opt-in compat package.
  6. 🚧 Remove the oc-cli-version publish gate; configurable CORS. Configurable CORS shipped on the 0.x line (R11, #1545); the oc-cli-version publish gate has not been removed. Adapter-neutral registry error codes are already emitted on the 0.x line (R17).
  7. ⬜ Remove s3 shortcut, refreshInterval, boolean discovery, oc.json mocks; fix tarExtractMode.
  8. ⬜ Registry Node engine → >=22 (still >=18).

App: 9. ⬜ Release the node oc-client v1 (promise + ESM + types, engine floor >=20). 10. ✅ Ship the browser oc-client.js with the drop-in contract intact (§2b) — no app action. Holding; extracted into oc-client-browser (#1478) without breaking the contract.

Component: the opt-in modern runtime is available; no migration required. — ⬜ not started.

  1. ⬜ Publish the migration guide (§7).

7. Migration guide (for the v1 release notes)

For web app owners (app layer):

  • Browser-only apps: nothing to do. Same <script>, same window.oc, same <oc-component>, no importmap to add.
  • SSR apps (use node oc-client): update to the promise API (await client.renderComponent(...)), adopt ESM/types. A 0.x callback shim eases the transition. Keep Node >=20.

For registry operators (registry layer):

  • await the now-promise-only registry.start/close/register and plugin/adapter APIs.
  • Update imports for the package split (oc-registry, oc-cli, oc-core).
  • Move to ESM (or dynamic import()); run the registry on Node >=22.
  • Config: s3 → storage.{adapter,options}; refreshInterval → pollingInterval; discovery: true → discovery: { ... }; oc.json mocks.plugins → development.plugins.
  • If serving pre-oc-template-* components, install the legacy compat package.
  • No metadata action needed (default bundled); configure an adapter for scale.
  • Custom hooks receive OcRequest/OcResponse; use the Express-compat shim if needed.
  • Publishing no longer rejects older CLIs on semver skew.
  • Server adapter failures emit SERVER_ERROR instead of the legacy EXPRESS_ERROR; the error event remains { code, message } and preserves the original message.

For component authors (component layer):

  • Nothing required. Existing components keep working. To adopt the modern runtime, scaffold new components with oc init (ESM default) and use async data(context).

For plugin/adapter authors (registry layer):

  • Return promises from register / adapter methods (callbacks removed).
  • Throw typed Errors.

8. Open questions & post-v1 research

  • Stronger data-provider isolation (worker threads / hardened sandbox) — must remain compatible with the frozen VM globals. Deferred past v1.
  • Exact package-split boundaries and naming for oc-registry / oc-cli / oc-core.
  • Default metadata store implementation (storage-backed vs embedded sqlite vs in-memory) and its persistence guarantees.
  • Fastify adapter promotion to a documented first-class transport. ✅ Resolved — oc-fastify-server-adapter shipped as a first-class package (#1512, #1513, #1518); remaining hardening (trailing-slash routing defaults) tracked in #1515.
  • Whether to formalize the vendor Accept mime types into a documented protocol string (without changing URLs).

In-flight tracking issues (own detailed checklists, referenced here rather than duplicated)

  • #1528 — Promise-first registry API (additive, dual callback/promise) — Track 1 item 1; implemented in #1539; plugin registration (#1542) and HTTP adapter lifecycle (#1544) promise variants also merged.
  • #1529 — Central deprecation-warning pass for v1 removals — Track 1 item 5. Core utility + config-option warnings merged via #1532; callback registry warnings are implemented in #1539, plugin/HTTP-adapter callback warnings in #1542 and #1544, and bare jade/handlebars warnings in #1546; only the callback node oc-client warning remains open.
  • #1515 — Make the Fastify adapter's ignoreTrailingSlash configurable (default false) and stop OC from emitting trailing-slash URLs — follow-on hardening for R3.

9. Decision log

Decision Choice
Layer model Registry = running the registry (programmatic API/plugins/adapters/config/CLI); App = web apps (browser oc-client.js + node SSR oc-client); Component = the components
Primary goal Modernize + push ESM and full re-architecture; modernization ships non-breaking on 0.x first, re-architecture is v1
Component contract Additive only + opt-in modern runtime; never break existing components
Browser oc-client.js Keep fully drop-in — no web app change (stable script URL, window.oc, auto-importmap, <oc-component> markup)
Node SSR oc-client Promise-only + ESM + typed; the one intentional app break (0.x callback shim first)
Registry programmatic API Promise-only, clean break (drop universalify) — registry layer
Registry module format ESM-only
Express coupling Adapter-native types; Express becomes one adapter (registry layer)
CLI publish version gate Remove it entirely
Metadata vs flat-file Metadata store = source of truth, bundle a default; flat-file export-only
Modern runtime ambition Define a modern opt-in runtime (ESM, async data, typed context)
Data provider (new components) async data(context); old callback providers still supported
Legacy jade/handlebars Extract to opt-in compat package
Package layout Split into oc-registry, oc-cli, oc-core, umbrella oc
Node baseline Registry >=22; node oc-client keeps >=20 to protect SSR apps
HTTP API Configurable CORS, keep URL/Accept contract stable
Data-provider VM sandbox Keep as-is; stronger isolation is post-v1 research
Config cleanups Drop s3 shortcut + refreshInterval; fix tarExtractMode; discovery object-only; remove oc.json mocks
Sequencing Deprecate in 0.x first; v1 is mostly removals