Skip to content

Run the editor Cypress specs in CI #477

Open

Description

@HMarzban

Summary

The 50 editor Cypress specs under apps/webapp/cypress/e2e/editor/ run in no CI job. They cover formatting, lists, headings, the TOC, copy and paste, Find and the Slash menu. A regression there reaches production unless someone runs the suite on a laptop.

  • Severity: Medium. Cypress E2E is the default test layer in AGENTS.md §Test Policy, and this part of it never runs on a push.
  • Area: .github/workflows/, scripts/, apps/webapp/cypress/e2e/editor/
  • Source: repository review of 2026-10-14

Where

  • prod.docs.plus.yml runs Cypress only for the five extension clean-room suites (:295–:304).
  • chatroom-e2e.yml runs scripts/test-chatroom.sh. That script builds the webapp, starts it, and calls bash scripts/run-tests.sh --e2e --scope chatroom (:72).
  • scripts/run-tests.sh --e2e --scope editor already runs only the editor specs. It also fails a run whose spec count comes up short (:464–:472).
  • All 50 specs open the /editor playground through cy.visitEditor (apps/webapp/cypress/support/commands.ts:439). The playground passes provider: null, so it opens no collab socket. Only with persist: true does it keep a local Y.Doc in y-indexeddb (apps/webapp/src/components/pages/editor/EditorPlayground.tsx:49, :43, :66).
  • The only Supabase call found in the editor specs is stubbed: cy.intercept('POST', '**/rpc/get_user_bookmarks*', …) (apps/webapp/cypress/support/hyperlinkPicker.ts:97). Other calls to REST or Supabase without a stub are not ruled out. Step 2 settles that.
  • A production build serves /editor only when NEXT_PUBLIC_E2E=true (apps/webapp/src/pages/editor.tsx:15). Without it, every spec fails at cy.visitEditor.
  • document/draft-first-edit-anchor.cy.ts hardcodes http://localhost:3001 and reads docsy-postgres-local. It needs the full local stack, so it stays out of this job.

Fix plan

  1. Script. Add scripts/test-editor.sh, built from the build-and-serve part of scripts/test-chatroom.sh, ending in bash scripts/run-tests.sh --e2e --scope editor. Leave out the NEXT_PUBLIC_VIRTUOSO_LICENSE check, because the playground renders no chat list.
  2. Measure. Run scripts/test-editor.sh locally. Post the pass, fail and duration numbers on this issue, with the CYPRESS_PARALLEL value used. test-chatroom.sh sets it to 1. Fix each failing spec, or .skip it with a one-line reason and an issue link (AGENTS.md §Test Policy). A spec that fails because a call has no stub gets a stub or a .skip.
  3. Workflow. Add editor-e2e.yml with the setup steps and the paths list of chatroom-e2e.yml. In that list, name .github/workflows/editor-e2e.yml in place of .github/workflows/chatroom-e2e.yml. Leave out its bun test scripts/run-tests.test.ts step, which that workflow already runs. Also leave out its NEXT_PUBLIC_VIRTUOSO_LICENSE env and its fork and Dependabot if: guard, which exist only for the license secret.

Decisions

  1. When does the job run? Recommended: on pushes to main that match the same paths list as chatroom-e2e.yml, plus workflow_dispatch. Unlike chatroom-e2e.yml, leave out pull_request until the job has a measured duration. If the measured run is long, run it on a weekly schedule only, and say so in the workflow.
  2. Does it gate the deploy? Recommended: no, at first. Let it run beside the deploy until it has been green for two weeks, then decide.

Out of scope

  • document/ specs, which need Postgres.
  • cypress-split parallel runs.

Acceptance criteria

  • A push to main that matches the paths list runs the editor specs in CI.
  • The first CI run is green, and every skipped spec names its issue.
  • The issue records the measured duration.

Verify

  1. Run scripts/test-editor.sh locally. The runner's spec count must be 50, the result of find apps/webapp/cypress/e2e/editor -name '*.cy.*' | wc -l today.
  2. Push, and confirm in the Actions log that the job ran the specs, not that it skipped.

Related


Generated by Claude Code

Activity

  1. HMarzban commented on Oct 15, 2026

    @HMarzban
    CollaboratorAuthor

    Related issues


    Cross-link review, 2026-10-15. Generated by Claude Code

  2. added this to the v2.3.0 milestone on Oct 24, 2026
  3. HMarzban commented on Oct 24, 2026

    @HMarzban
    CollaboratorAuthor

    Facts from the 2026-10-23 test run

    • There are 51 editor specs, not 50.
    • 3 of them need the full stack: the filter/* specs. They open the pad route through visitSeededPad (apps/webapp/cypress/e2e/editor/filter/filter-shared.js:38).
    • The other 48, including the 5 indent/* specs, open /editor through cy.visitEditor (apps/webapp/cypress/e2e/editor/indent/indent-shared.js:111). So "All 50 specs open the /editor playground" is wrong only for filter.
    • run-tests.sh --scope accepts one top-level folder only, so --scope editor/toc is rejected.
    • CI runs run-tests.sh --extensions (.github/workflows/prod.docs.plus.yml:302) and the chatroom scope (chatroom-e2e.yml). The chatroom workflow skips Dependabot PRs. So a Tiptap bump runs only the extension suites in a browser, and no webapp spec.
    • All three filter/* specs pass since 6be36fcf8, so The Filter apply Cypress test has never passed #311 is closed.

    Fix in the same change: full-doc-paste.cy.js passes with no paste

    apps/webapp/cypress/e2e/editor/copy-paste/full-doc-paste.cy.js presses Meta+c and Meta+v under headless Electron. That never reaches the clipboard (standard-paste.cy.js:25-27), so the spec re-checks its own seed. It also selects with Meta+a, which maps to select-all only on macOS, while CI runs on Linux.

    • Select with win._editor.commands.selectAll().
    • Paste with cy.pasteWithMimeTypes.
    • Add a second case whose first block is a <p>. A paste that starts with a heading skips the select-all branch of titlePasteHandler (apps/webapp/src/components/TipTap/extensions/title-document/title-document.ts:43-49).
    • Break the paste path locally, and see the spec fail. Restore it.

    This issue is now in milestone v2.3.0, under #575. Effort S for the paste fix, then M for this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions