Skip to content

Enable Typesense forum search for master/verawood CI+QA environments - #5718

Merged
blarghmatey merged 1 commit into
mainfrom
typesense-forum-search-ci-qa
Sep 2, 2026
Merged

blarghmatey merged 1 commit into
mainfrom
typesense-forum-search-ci-qa

Conversation

@blarghmatey

Copy link
Copy Markdown
Member

What are the relevant tickets?

N/A

Description (What does it do?)

Sets typesense:forum_search_enabled: "true" on the seven CI/QA edxapp stacks whose release line ships the fixed forum package. One line per file, no code changes.

  • mitodl/lehrer#208 merged, so deployments/mit-ol/build_manifest.yaml now installs forum from the branch behind openedx/forum#289 on six of its seven cells: master × {mitxonline, mitx, mitx-staging} and verawood × {mitx, mitx-staging, xpro}.
  • Before that, this toggle could not be turned on anywhere. The released backend is non-functional as shipped, with two independent defects: per_page=1000 against Typesense's hard cap of 250 (HTTP 422 on every search), and a topic filter naming commentable_ids where the collection schema declares commentable_id (HTTP 400 on topic-scoped searches).
  • That is the likely explanation for what happened in April 2026. 87fcc0559 (#4501, 2026-04-21 11:04) enabled Typesense and set this key to "true" on mitxonline CI; d6c2824ee set it back to "false" 48 minutes later with no reason recorded; 386057f69 then rolled Typesense out to more environments at 13:48 the same day, deliberately keeping the forum key "false". It has stayed off everywhere since.
  • Flipped: mitx.CI, mitx.QA, mitx-staging.CI, mitx-staging.QA, mitxonline.CI, mitxonline.QA, xpro.CI. Derived by cross-referencing the env→release map in version_matrix.py against the manifest cells.
  • Not flipped: xpro.QA. It runs ulmo, and that cell still pins openedx-forum==0.4.1 in its overrides: the manifest's release_python is ulmo: '3.11' against master/verawood at '3.12', and forum master dropped 3.11. Turning this on would put xpro QA on the unfixed backend. It stays on Elasticsearch.
  • Production is untouched in all four deployments.

All seven already had typesense:enabled: "true", so the new value is reached at k8s_configmaps.py#L292-L293, which sets FORUM_SEARCH_BACKEND to forum.search.typesense.TypesenseBackend.

How can this be tested?

Nothing has been applied and no search has been run. The only verification so far is a preview.

pulumi preview --stack mitxonline.CI --diff renders the expected change in the interpolated configmap:

+ FORUM_SEARCH_BACKEND : "forum.search.typesense.TypesenseBackend"
  TYPESENSE_ENABLED    : true

The preview then aborts on an unset EDXAPP_DOCKER_IMAGE_DIGEST, which is supplied by the pipeline and unrelated to this change.

To validate:

  1. Apply an edxapp CI stack and confirm the pods restarted and the new value is live in the running process, not just merged.
  2. Run rebuild_forum_indices for that environment.
  3. Search in the LMS discussions UI. The failure mode is total rather than partial (HTTP 422 on every query), so a broken environment is obvious on the first search.
  4. Repeat per environment. Do not promote to Production until CI and QA have been observed working.

Additional Context

  • xpro.QA staying on Elasticsearch is deliberate, not an oversight. Worth confirming that is the reviewer's read too, since it is the one asymmetry in the diff.
  • Known gaps in the Typesense backend even once working, per its own source comments: get_suggested_text() returns None, so there is no "did you mean" spelling correction that the Elasticsearch backend provides via phrase suggesters; group_ids and sort_criteria are unsupported. These need product sign-off before Production, not before this PR.
  • Once fix: make the Typesense search backend usable openedx/forum#289 merges and releases, the git override in lehrer must be removed so edx-platform's pin carries the dependency. A git direct reference wins silently over a pin, so a stale override would hold us on a dead branch with nothing flagging it.

…ments

lehrer's build manifest now installs forum from the branch behind
openedx/forum#289 on six of its seven cells, so the Typesense forum backend is
finally usable in those images. Until now the toggle could not be turned on
anywhere: the shipped backend is non-functional, passing per_page=1000 against
Typesense's hard cap of 250 (HTTP 422 on every search) and filtering on
commentable_ids where the collection schema declares commentable_id (HTTP 400
on topic-scoped searches).

That is the likely explanation for April 2026. 87fcc05 (#4501, 11:04) enabled
Typesense and set this key to "true" on mitxonline CI; d6c2824 set it back to
"false" 48 minutes later with no reason recorded; 386057f rolled Typesense
out more widely at 13:48 the same day while deliberately keeping the forum key
"false". It has stayed off everywhere since.

Flipping only where the fixed forum is actually deployed. Cross-referencing
version_matrix.py's env-to-release map against the manifest's cells, seven of
the eight CI/QA environments run master or verawood and qualify. xpro QA is the
exception: it runs ulmo, which pins openedx-forum==0.4.1 because the manifest's
release_python puts ulmo on 3.11 against master and verawood on 3.12, and forum
master dropped 3.11. It keeps Elasticsearch and is left untouched. Production is
untouched everywhere.

All seven already have typesense:enabled true, so the new value is reached at
k8s_configmaps.py:293. Verified by preview on mitxonline.CI, which shows
FORUM_SEARCH_BACKEND becoming forum.search.typesense.TypesenseBackend in the
interpolated configmap.

Run rebuild_forum_indices per environment after applying, then confirm search
returns results. The failure mode is total rather than partial, so it is
obvious if you look.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012c3Ua8fPhRYnEdQE8snFNW
Copilot AI balanced review requested due to automatic review settings September 2, 2026 17:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The selected stacks match the release matrix and build-manifest overrides, while xpro QA and Production remain unchanged.

Pull request overview

Enables Typesense forum search in seven eligible CI/QA Open edX stacks while excluding ulmo-based xpro QA and all Production environments.

Changes:

  • Enables the Typesense forum backend for master/verawood environments.
  • Preserves Elasticsearch for xpro QA and Production.
File summaries
File Description
Pulumi.xpro.CI.yaml Enables forum search for xpro CI.
Pulumi.mitxonline.QA.yaml Enables forum search for mitxonline QA.
Pulumi.mitxonline.CI.yaml Enables forum search for mitxonline CI.
Pulumi.mitx.QA.yaml Enables forum search for mitx QA.
Pulumi.mitx.CI.yaml Enables forum search for mitx CI.
Pulumi.mitx-staging.QA.yaml Enables forum search for mitx-staging QA.
Pulumi.mitx-staging.CI.yaml Enables forum search for mitx-staging CI.
Review details
  • Files reviewed: 7/7 changed files
  • Comments generated: 0
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@blarghmatey
blarghmatey merged commit 77371cc into main Sep 2, 2026
9 checks passed
@blarghmatey
blarghmatey deleted the typesense-forum-search-ci-qa branch September 2, 2026 18:51
blarghmatey added a commit that referenced this pull request Oct 7, 2026
…6279)

mitxonline Production is the last mitxonline environment with forum
search on Elasticsearch. CI and QA have had the Typesense backend
configured since #5718.

The forum post_save handlers upsert into the Typesense collection
synchronously and do not create it, so the collection has to be built
before this applies or saving a thread or comment will fail.

Claude-Session: https://claude.ai/code/session_01QnH9kCLkUQ5EVto8bcPSFp
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants