Skip to content

Latest commit

 

History

History

dofs

README.md

@cloudflare/dofs

Note

Stable, meaning suitable for production use, except the sync protocol building blocks, which serve the experimental sync with computerd and share its status. This package is not published to npm on its own: it ships bundled inside @cloudflare/computer, which re-exports the parts meant for callers. See that package's status table for which components of Cloudflare Computer are stable and which are experimental.

The specification under docs/ is forward-looking — read it for intent, not as description of the code today.

Durable Object SQLite-backed virtual filesystem for Cloudflare Computer.

This package exposes a JavaScript module, not a CLI. It bundles three layers that can be used independently:

  • A Database wrapper around Durable Object SQL storage plus initializeSchema for the vfs_* tables.
  • Filesystem primitives under src/fs/* (mkdir, writeFile, readFile, rm, rename, readdir, stat, lstat, chmod, find, ls, grep, symlink, readlink, gc, watch) operating on a Database.
  • SQLiteWorkspaceProvider, a @platformatic/vfs adapter that composes those primitives into a node-shaped filesystem (fd table, positional readSync/writeSync, watchSync, symlinks). This is what computerd mounts via FUSE.
  • Sync protocol building blocks operating on the same Database: applyChanges, stageBlob, materialiseChange, coalesceChanges, fetchChanges, fetchObjects, hasObjects, pushObjects, buildManifest, currentRev, compareChangeCursors, readWatermark/writeWatermark, assertAppliedPushCursor, and the opt-in ignore matcher isIgnored (the default ignore list is empty). The wire wiring lives in @cloudflare/computer-rpc.

Minimal DO-side usage — initialize the schema; the Database becomes the handle every other helper takes:

import { Database, initializeSchema } from "@cloudflare/dofs";

export class WorkspaceDO extends DurableObject {
  private readonly db: Database;

  constructor(ctx: DurableObjectState, env: Env) {
    super(ctx, env);
    this.db = new Database(ctx.storage);
    initializeSchema(this.db, Date.now);
  }
}

The src/fs/* primitives (mkdir, writeFile, readFile, rm, readdir, stat, find, ls, grep, symlink, readlink, gc, watch) are not re-exported from the package root yet — they are consumed in-tree by SQLiteWorkspaceProvider and by the sync applyChanges path. On the node side, instantiate SQLiteWorkspaceProvider (the @platformatic/vfs adapter) for a familiar node:fs-shaped surface; this is what @cloudflare/computerd mounts via FUSE. The higher-level durable-object-side Workspace class described in ../../docs/README.md lives in @cloudflare/computer.

Implementation status

  • Database wrapper around Durable Object SQL storage in place.
  • Schema initialization for the documented vfs_* tables (FS and sync) implemented and split into schema/core.ts + schema/sync.ts.
  • incrementRev() shared sequencer in place. FS writes stamp the returned value into vfs_nodes.rev and pass it to sync/changes.ts for tombstones.
  • SQLiteTestStorage (backed by node:sqlite) available from ./testing for unit tests against a real in-memory database; RecordingStorage available from the package root for workerd-safe schema assertions.
  • All filesystem primitives listed above are implemented and unit-tested. readdir returns size and modification time and supports stable limit/offset pages, including files held in pending write buffers. find supports *, **, and ? globs. grep supports bounded pages, regular expressions or fixed strings, explicit case handling, and numbered context lines.
  • SQLiteWorkspaceProvider (the @platformatic/vfs adapter) implemented and exported from the package entrypoint; consumed by @cloudflare/computerd.
  • Buffered-write surface for the FUSE driver: createFileSync, writeRangeSync, truncateFileSync, readRangeSync, chmodSync, openWriteBufferSync, openWriteBufferForCreateSync, and releaseWriteBufferSync on the provider. The driver opens a buffer on FUSE create/open, mutates it through subsequent writes and truncates, and commits chunks in one transaction at release time. Reads against the same database see the buffered bytes immediately.
  • Content-addressed blob cache: readFile, readRangeSync, provider.readFileSync, and the partial-chunk read-modify-write helper share a per-Database LRU keyed by vfs_blob_bytes.hash. Repeated reads of dedup'd chunks (a file of zeroes, a re-used package payload) skip SQLite after the first fetch.
  • Sync protocol building blocks implemented and exported; the typed RPC surface on top of them lives in @cloudflare/computer-rpc.