Skip to content

development.md

Latest commit

 

History

History
137 lines (110 loc) · 7.6 KB

development.md

File metadata and controls

137 lines (110 loc) · 7.6 KB
Raw

Development

This repo is a Next.js application with ES modules in the app code and Node's built-in test runner for runtime tests.

Commands

npm install
npm run dev
npm run build
npm run start
npm run lint
npm test
  • npm run dev runs the Express server in server/App.js with Next in development mode (hot reload). It also hosts the storage API, so saving works in dev. Hot reload covers the Next/browser code. Restart this process after changing Express modules or their shared imports (including app/scenarios/route/ and app/roads/); route verification at /api/storage/scenarios/:id/verify-route otherwise continues to execute its cached modules even after a browser refresh.
  • npm run build builds the app.
  • npm run start runs server/App.js on PORT or 3000.
  • npm run lint runs ESLint.
  • npm test runs node --experimental-default-type=module --test tests/*.test.js.
  • npm run benchmark:visual-scale:quick writes a hosted cev-sim.visual-scale-report@1. Full x64/Orin/Thor reports use npm run benchmark:visual-scale -- --profile <id> --require-gpu.

Test files are grouped by area: visual-script-runtime.test.js, editor-core.test.js, editor-map-mode.test.js, earth-import-mode.test.js, bake-*.test.js, and storage-service.test.js.

Storage backend

Environment edits, scripts, and bindings are persisted on the server rather than in the browser. The backend is deliberately simple - no database, just JSON files with an in-memory cache:

  • server/storage/JsonFileStore.js - one file's worth of JSON: reads are cached in memory, writes are atomic (temp file + rename).
  • server/storage/StorageService.js - owns the on-disk layout under server/data/ (environments/<id>.json, scripts/<id>.json, bindings.json, settings.json, visual-layer descriptors, visual-layer access sidecars, and the visual-asset CAS) and environment catalog operations.
  • server/storage/PluginStore.js - immutable plugin CAS, revisioned library.json, and runtime materialization.
  • server/storage/VisualAssetStore.js - validated immutable visual-asset bytes, use records, quotas, staging recovery, and internal roots/pins.
  • server/storage/VisualLayerAccessStore.js - immutable cev-sim.visual-layer-access@1 sidecars.
  • server/routes/storageApi.js - mounts /api/storage/visual-assets streaming routes before the shared JSON parser, then the JSON storage router.
  • server/routes/storageRouter.js - the Express JSON router mounted at /api/storage; a thin HTTP-to-service translation layer.
  • app/client/storageClient.js - the browser's JSON fetch wrapper for that API.
  • app/3d/environment/visual/VisualAssetClient.js - browser client for visual-asset upload, use, content, and closure validation.
  • app/3d/environment/visual/VisualLayerClient.js - browser client for visual-layer publish/read.
  • app/3d/environment/visual/VisualLayerMaterializer.js - bounded AOI preview materialization, LOD selection, and residency snapshots.
  • app/3d/environment/visual/VisualResourceCache.js - renderer-scoped reference-counted encoded/parsed/texture cache.
  • app/simulation/visual/VisualScaleProfile.js - D06 hardware and hosted-quick capacity profiles.
  • app/3d/environment/EnvironmentCatalogClient.js - list/create/duplicate/rename/delete and active-environment settings.
  • app/3d/environment/EnvironmentLoader.js - the sole manifest/template-to-runtime application path; metric rebuild then preview materialization.
  • app/3d/environment/EnvironmentPersistence.js - debounced manifest saving only.

The server/data/ directory is git-ignored; it is created on first write.

Code Layout

  • app/page.js: browser entry and mode switch.
  • app/3d/: Three.js scene, vehicles, devices, city objects, overlays, and IGVC scenarios.
  • app/3d/editor/: environment editor state, tools, chunks, and document model.
  • app/3d/earth/: Earth Import (tiles, roads, geospatial transforms).
  • app/client/: orchestrator WebSocket and message encoding client.
  • app/physics/: physics engine wrapper.
  • app/scripting/: visual node editor, block classes, runtime compiler, and runner.
  • app/simulation/: simulation loop.
  • app/util/: shared utilities.
  • public/: static assets served by Next.
  • tests/: Node tests.

Import Style

The app uses @/ imports for repo-root app paths in many modules. Keep new imports consistent with the surrounding file.

Before Opening A PR

Run the focused check for the area you touched:

npm test
npm run lint

For visual or simulation changes, also run npm run dev and manually verify:

  • The scripting canvas still loads.
  • The Escape menu switches between scripting, simulation, and the environment editor.
  • The 3D scene starts without console errors.
  • If orchestrator integration changed, the app behaves both with and without the orchestrator running.

For environment editor or earth import changes, also verify:

  • Scene and Map views persist while Earth-import chrome opens and closes cleanly without changing the active view.
  • Earth Import preview and apply work with a valid NEXT_PUBLIC_GOOGLE_MAPS_API_KEY in .env.local.
  • npm test passes for tests/editor-*.test.js and tests/earth-import-mode.test.js.
  • For object-registry, document, or schema changes: tests/object-registry.test.js, tests/object-graph.test.js, tests/environment-v3.test.js, and tests/environment-v4.test.js pass, and npm run fixtures:environment-editor produces no diff in tests/fixtures/environment-editor/compatibility-baseline.v1.json (a diff is a metric-identity contract change).

Conventions

  • Keep visual scripting block UI and backend UnitBlock behavior in sync.
  • Declare compileable blocks once in UnitCatalog.meta.js, then attach their React component by stable type in UnitCatalog.js; registration is derived from the server-safe metadata.
  • Keep message definitions in public/messages/ synchronized with the orchestrator repo when they are used as browser fallbacks - this will be edited soon to be synchronized.
  • Do not commit downloaded CommonRoad scenario folders or other large generated assets.

Environment Variables

Variable Required for Notes
NEXT_PUBLIC_GOOGLE_MAPS_API_KEY Earth Import tile preview Map Tiles API (Photorealistic 3D Tiles). Set in .env.local, not committed.
CEV_SIM_VISUAL_SOURCE_REGISTRY Editor asset Import Operator grants file; defaults to $CEV_SIM_DATA_DIR/visual-source-registry.json. There is no mutation API. Missing files fail closed (empty sources).

See Earth Import for setup and troubleshooting.

Editor Import (Assets pane) uses sources that are active and grant persistent-cache, machine-interpretation, and retention. A missing registry fails closed and the pane explains that. With one eligible grant, Import uses it and does not show the grant id. With two or more, Upload grant chooses which id is stamped on the import; that choice does not filter the catalog. Copy an owned-lab grant into the registry path above, then restart the server. Built-in props still place without a grant.

{
  "kind": "cev-sim.visual-source-registry",
  "version": 1,
  "sources": [
    {
      "id": "owned-lab",
      "kind": "owned",
      "status": "active",
      "ancestorIds": [],
      "permissions": {
        "attribution": true,
        "derivatives": true,
        "display": true,
        "export": true,
        "live-preview-display": true,
        "machine-interpretation": true,
        "ml": true,
        "persistent-cache": true,
        "retention": true,
        "transient-cache": true,
        "worker-access": true
      }
    }
  ]
}