Sync with upstream/main (23 commits) by mmcky · Pull Request #78 · QuantEcon/mystmd

mmcky · 2026-08-10T10:50:29Z

Important

Merge with "Create a merge commit" — do NOT squash. A real merge commit preserves the ancestry so the next git merge upstream/main works cleanly (see quantecon/README.md).

Brings in 23 upstream commits, up to d41a821e. No upstream release in this range — mystmd stays at 1.10.1 with 14 pending changesets. Three upstream changes intersected with fork features and were integrated below.

`proof:kind` spelling (jupyter-book#2983)

Upstream added a proof:kind alias for every proof directive alongside the legacy prf:kind spelling, so {proof:proof} is now valid. The fork's amsthm default (#57) keyed its bare-proof test off data.name === 'proof' || data.name === 'prf:proof' — the new proof:proof spelling would have slipped past it and come back numbered.

proof.ts now strips either prefix before the comparison, so all three spellings (proof, prf:proof, proof:proof) stay unnumbered by default.
The it.each default-enumeration table gained proof:proof and proof:theorem cases; upstream's two new spelling tests kept as-is.
docs/proofs-and-theorems.md keeps upstream's proof:kind wording with the fork's amsthm numbering caveat re-applied on top.

Isolated bun linker (jupyter-book#2987)

Upstream switched bunfig.toml to linker = "isolated", so each package only sees what it declares. That surfaced ~30 phantom dependencies upstream, which they fixed — but the fork-only myst-to-ipynb package was invisible to that sweep and still relied on hoisting. It failed to build after the merge (Cannot find module 'myst-spec').

Declared myst-spec, myst-spec-ext and unified as dependencies plus @types/js-yaml as a devDependency, mirroring the fix upstream applied to myst-to-md and myst-to-docx. quantecon/UPSTREAM-PRS.yml records a cherry-pick caveat: these declarations live in the merge commit, not in either of the myst-to-ipynb candidate's squash shas, so they must be re-added when that upstream PR is prepared.

`.ipynb` build cache (jupyter-book#2970)

Import-block conflict in packages/myst-cli/src/process/mdast.ts only — took upstream's expanded myst-execute import (TieredExecutionCache/NotebookExecutionCache/LegacyExecutionCache) alongside the fork's IOutput/BookSection imports. The fork's book-section and ipynb-output wiring downstream of it is untouched.

Other

packages/myst-cli/src/transforms/doi.spec.ts — import-block conflict only, between upstream's new transformLinkedDOIs coercion tests (🔗 Relax overaggressive DOI coercion jupyter-book/mystmd#2960) and the fork's recorded-fixture session (Deflake CI: record DOI fixtures, retry Jupyter execution e2e tests #62). Both suites are intact and both pass.
bun.lock regenerated with bun install. Besides the fork's own entries, the regeneration also picks up the he / @types/he dependency upstream added to citation-js-utils in Decode HTML entities in CSL text fields jupyter-book/mystmd#2986 without relocking.
quantecon/VERSION.yml — upstream_base bumped 1.9.1 → 1.10.1; the 2026-06-12 sync brought in v1.10.0 and v1.10.1 without updating it.

Verification

Fork features confirmed present post-merge: fancy lists, citation code-span guard, code numbering, proof-unnumbered, book numbering/parts/section-scope, ipynb export + attachments, myst --version qe identifier. Full monorepo build, lint, format check and test suite pass (72 turbo tasks); myst-to-ipynb's own 58 tests pass.

🤖 Generated with Claude Code

@agoose77

* fix: use availableParallelism rather than cpus This (should) gives physical cores. * chore: add changeset * fix: restore parallelism * Apply suggestion from @agoose77

* feat: specify cache extension * wip: refactor cache * feat: add tiered cache * fix: support ipynb * revert: changes to inline expression metadata * fix: more cleanup * fix: more missing imports * fix: type of block * chore: add changeset * fix: linter warning * fix: more linter fixes * Revert "fix: linter warning" This reverts commit 8a13373. * feat: add duration (ms)

…ok#2983)

* fix: add 'aborted' to execute reply status type @jupyterlab/services@7.6 widened IReplyAbortContent.status to 'abort' | 'aborted'; the local variable type needs to match. * Bump @jupyterlite/services * Read reply type from KernelMessage.IExecuteReplyMsg Better than specifying by hand, like we did before, which can get out of sync.

node-version: 18.x is depreciated on GitHub

* Switch bun to isolated linker mode Sets linker = "isolated" in bunfig.toml so each package's node_modules only contains what it actually declares, instead of a hoisted flat tree. This surfaced ~30 phantom dependencies across the workspace (types/packages used but not declared) which are now fixed. Temporarily works around a bug in meca/jats-utils/jats-xml (misdeclared as devDependencies upstream, PR pending at continuous-foundation/jats#109) The workarounds: - bumped jats-xml to the already-fixed 1.1.7 - added esbuild --alias flags so mystmd's bundled CLI resolves myst-cli-utils/myst-common correctly regardless - disabled (renamed .skip) three test files that load meca/jats-utils directly and have no bundler to alias around; jats-to-myst's test script now uses `--pass-with-no-tests` to make it green * Declare eslint and its plugins as direct root devDependencies Under the isolated linker, eslint (and the typescript-eslint/plugin packages eslint-config-curvenote needs) were only reachable transitively through eslint-config-curvenote's own dependencies. * Also temporarily disable mystmd's clirun.spec.ts (blocked on jats-utils upstream bug) * Drop meca/jats-utils/jats-xml workarounds now that 1.1.8 is released

Co-authored-by: Rowan Cockett <rowanc1@gmail.com>

See jupyter-book#3007

…ook#3013) Currently, https://mystmd.org/guide/frontmatter#titles uses the word “complimentary” incorrectly: > You may specify subtitle; this conveys **complimentary** information > to the title and may be displayed below the title. But “complimentary” is a form of praise or admiration for someone or something. The word “complementary” should be used here, which means subtitle adds, gives extra information and “complements” (not compliments) the title.

* Authors field in pixi.toml should be list of strings * Strip whitespace

* fix: set console type * chore: add changeset

* feat: add public manifest * chore: add changeset * docs: add note about public.json

* Add a brief guide to the AST structure * Apply suggestions from code review Co-authored-by: Rowan Cockett <rowanc1@gmail.com> * Move parts docs to myst guide and clarify * Address a few more comments --------- Co-authored-by: Rowan Cockett <rowanc1@gmail.com>

CrossRef's CSL-JSON metadata sometimes contains HTML entities in title and journal name fields. Co-authored-by: Angus Hollands <goosey15@gmail.com>

@agoose77

…upyter-book#2978) * bugfix for issue jupyter-book#2973 when pattern matching and avoiding previously mentioned entries there's a need to resolve new items when searching in `ignore` because it contains resolved entries * changeset * Apply suggestion from @agoose77 --------- Co-authored-by: Angus Hollands <goosey15@gmail.com>

* docs: divided gitlab deployment into two sections (pixi + poetry) and improved the tutorial for gitlab deployment with pixi * added section about deployment without technical details * added section for beginners * added info about where to find link to deployed gitlab pages * small edits to script and description --------- Co-authored-by: FreekPols <c.f.j.pols@tudelft.nl> Co-authored-by: Angus Hollands <goosey15@gmail.com>

Brings jupyter-book/mystmd:main up to d41a821. No upstream release in this range — mystmd stays at 1.10.1 with 14 pending changesets. Conflict resolutions and integration edits: - packages/myst-ext-proof — upstream jupyter-book#2983 added the `proof:kind` spelling alongside legacy `prf:kind`. The fork's amsthm default (#57) keyed off `data.name === 'proof' || 'prf:proof'`, which the new `proof:proof` alias would have bypassed. Rewrote the check to strip either prefix so all three spellings stay unnumbered by default; test table extended with `proof:proof` and `proof:theorem`. - docs/proofs-and-theorems.md — kept upstream's `proof:kind` wording and re-applied the fork's amsthm numbering caveat on top. - packages/myst-cli/src/process/mdast.ts — import-block conflict only; took upstream's expanded myst-execute import (Tiered/Notebook/Legacy caches, jupyter-book#2970) alongside the fork's IOutput/BookSection imports. - packages/myst-cli/src/transforms/doi.spec.ts — import-block conflict only; upstream's transformLinkedDOIs tests (jupyter-book#2960) alongside the fork's recorded-fixture session (#62). - packages/myst-to-ipynb — upstream jupyter-book#2987 switched bun to the isolated linker, which surfaces phantom dependencies. The fork-only ipynb package still relied on hoisting for myst-spec, myst-spec-ext and unified; declared them (plus @types/js-yaml), matching the fix upstream applied to myst-to-md and myst-to-docx. - bun.lock — regenerated with `bun install` after the merge. Build, lint, format and the full test suite pass (72 turbo tasks). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…k deps `upstream_base` was still 1.9.1 — the 2026-06-12 sync brought in v1.10.0 and v1.10.1 without bumping it. Also records a cherry-pick caveat on the myst-to-ipynb upstream candidate: the phantom-dependency declarations that upstream's isolated linker now requires live in the sync merge commit, not in either of the candidate's squash shas, so they have to be re-added on the upstream branch. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copilot

Pull request overview

This pull request primarily synchronizes the QuantEcon fork with upstream/main (23 commits) while integrating a few fork-specific intersections (proof directive spelling, isolated Bun linker fallout, and the notebook execution cache changes).

Changes:

Adds proof:* aliases alongside legacy prf:* for proof directives/roles and updates docs/tests accordingly.
Refines DOI recognition (new infer_dois_from_urls setting + stricter default matching) and expands related tests.
Switches execution caching toward an .ipynb-backed cache (tiered with legacy fallback), adds a public.json manifest for themes, and updates a broad set of package metadata/dependencies to align with upstream.

Reviewed changes

Copilot reviewed 106 out of 108 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
quantecon/VERSION.yml	Bumps recorded upstream base version for fork sync tracking.
quantecon/UPSTREAM-PRS.yml	Records cherry-pick caveat for fork-only package deps under isolated linker.
packages/tex-to-myst/package.json	Adds missing deps metadata needed under isolated installs.
packages/mystmd/src/clirun.ts	Updates parallelism calculation using Node OS API.
packages/mystmd/package.json	Aligns devDependencies with upstream package set.
packages/myst-transforms/src/links/index.ts	Re-exports DOI helpers/types for downstream use.
packages/myst-transforms/src/links/doi.ts	Introduces stricter DOI recognition + optional URL inference.
packages/myst-transforms/src/links/doi.spec.ts	Adds coverage for DOI recognition and transformer behavior.
packages/myst-transforms/package.json	Adds dependency/type metadata needed under isolated installs.
packages/myst-to-typst/package.json	Adds missing deps + bumps tex-to-typst.
packages/myst-to-tex/package.json	Adds missing deps/devDeps under isolated installs.
packages/myst-to-md/package.json	Adds missing deps/devDeps under isolated installs.
packages/myst-to-jats/tests/basic.spec.ts	Removes unused import in tests (cleanup with upstream).
packages/myst-to-jats/package.json	Adds deps/devDeps and bumps jats-xml.
packages/myst-to-ipynb/package.json	Declares previously-hoisted deps for isolated linker compatibility.
packages/myst-to-html/package.json	Adds missing deps/devDeps under isolated installs.
packages/myst-to-docx/package.json	Adds missing deps/devDeps under isolated installs.
packages/myst-templates/package.json	Adds type packages needed under isolated installs.
packages/myst-spec/src/index.ts	Exports new AST-related types.
packages/myst-spec/src/ext.ts	Adds execution/result-related types and CodeBlock type narrowing.
packages/myst-spec/package.json	Adds nbformat types dependency for spec typing.
packages/myst-spec/docs/myst.yml	Adds references + TOC updates including new doc pages.
packages/myst-spec/docs/ast-primer.md	Adds new AST primer documentation page.
packages/myst-roles/src/reference.ts	Adds `proof:ref` alias to reference role.
packages/myst-roles/package.json	Adds devDependency metadata (isolated installs).
packages/myst-parser/tests/myst.spec.ts	Resolves myst-spec examples path via createRequire for robustness.
packages/myst-parser/package.json	Adds missing dependency metadata.
packages/myst-migrate/package.json	Adds missing type dependency metadata.
packages/myst-frontmatter/src/settings/validators.ts	Adds `infer_dois_from_urls` setting + alias validation.
packages/myst-frontmatter/src/settings/types.ts	Adds typed setting for DOI inference option.
packages/myst-frontmatter/src/settings/settings.yml	Adds normalization test cases for new setting and alias.
packages/myst-frontmatter/package.json	Adds missing devDependencies for schema/types.
packages/myst-ext-tabs/package.json	Adds missing deps/devDeps under isolated installs.
packages/myst-ext-proof/tests/proof.spec.ts	Extends proof directive tests for `proof:*` spellings.
packages/myst-ext-proof/src/types.ts	Exports proof kind list for reuse.
packages/myst-ext-proof/src/proof.ts	Generates proof aliases and normalizes proof kind handling.
packages/myst-ext-proof/package.json	Adds myst-directives dependency (upstream alignment).
packages/myst-ext-icon/package.json	Adds missing devDependency metadata.
packages/myst-ext-grid/package.json	Adds myst-directives dependency (upstream alignment).
packages/myst-ext-exercise/package.json	Adds deps/devDeps under isolated installs.
packages/myst-ext-button/package.json	Adds missing devDependency metadata.
packages/myst-execute/src/utils.ts	Switches execution node typings to myst-spec types.
packages/myst-execute/src/types.ts	Refactors execution result typing (document context + tagged results).
packages/myst-execute/src/transform.ts	Switches cache interface to document-level cache; adds cache metadata.
packages/myst-execute/src/kernel.ts	Normalizes kernel session path + execution request options.
packages/myst-execute/src/index.ts	Exposes new cache implementations and types.
packages/myst-execute/src/execute.ts	Adapts execution pipeline to new tagged result format.
packages/myst-execute/src/cache.ts	Adds ipynb-backed cache + legacy/tiered adapters; updates LocalDiskCache.
packages/myst-execute/package.json	Updates dependencies to match new typing and cache behavior.
packages/myst-directives/package.json	Adds myst-spec dependency + types/devDeps for isolated installs.
packages/myst-config/package.json	Adds missing devDependency metadata.
packages/myst-common/src/types.ts	Removes expression-result types from myst-common (moved to myst-spec).
packages/myst-common/src/index.ts	Stops re-exporting removed expression types.
packages/myst-common/package.json	Adds types metadata for isolated installs.
packages/myst-cli/src/transforms/inlineExpressions.ts	Adjusts typing and always renders expressions (safe empty output).
packages/myst-cli/src/transforms/dois.ts	Uses new DOI recognition function + passes options through.
packages/myst-cli/src/transforms/doi.spec.ts	Adds tests ensuring DOI coercion doesn’t trigger by default for non-doi.org URLs.
packages/myst-cli/src/project/toc.pattern.spec.ts	Adds test ensuring explicit TOC files are excluded from later glob matches.
packages/myst-cli/src/project/fromTOC.ts	Fixes ignore filtering for resolved paths when expanding glob patterns.
packages/myst-cli/src/process/site.ts	Writes `public.json` manifest from public assets and adds it to site outputs.
packages/myst-cli/src/process/notebook.ts	Aligns inline expression typing and result embedding behavior.
packages/myst-cli/src/process/mdast.ts	Integrates tiered execution caches + DOI inference option into processing.
packages/myst-cli/src/init/gh-actions/index.ts	Updates generated GitHub Actions workflow versions and Node version.
packages/myst-cli/src/build/site/start.ts	Serves `public.json` from the content server.
packages/myst-cli/src/build/html/index.ts	Makes some assets optional (favicon) and improves binary fetch error handling; copies `public.json`.
packages/myst-cli/package.json	Updates deps/devDeps and adds vfile-message + type packages.
packages/myst-cli-utils/src/session.ts	Silences unused-args lint for session fetch stub.
packages/myst-cli-utils/package.json	Adds commander dependency.
packages/markdown-it-myst/package.json	Adds missing devDependency metadata.
packages/jtex/package.json	Adds missing devDependency metadata.
packages/jats-to-myst/package.json	Bumps jats-xml version.
packages/citation-js-utils/tests/basic.spec.ts	Adds test for HTML entity decoding in CSL fields.
packages/citation-js-utils/src/index.ts	Decodes HTML entities when normalizing citation fields.
packages/citation-js-utils/package.json	Adds `he` + types and adjusts deps.
package.json	Adds `docs` as a workspace and brings in ESLint toolchain deps.
docs/theme-developer.md	References new AST primer and documents `public.json` manifest.
docs/settings.md	Documents `infer_dois_from_urls` setting.
docs/proofs-and-theorems.md	Updates docs to `proof:*` spelling and proof reference role alias.
docs/plugins.md	Notes multi-page aggregation pattern for transforms/plugins.
docs/plugins-ast.md	Adds pointer to AST primer.
docs/package.json	Adds docs workspace package to depend on workspace packages.
docs/overview.md	Adds pointer to AST primer.
docs/javascript-plugins.md	Adds anchor and multi-page plugin section.
docs/glossary.md	Fixes xref targets and adds AST primer pointer.
docs/frontmatter.md	Fixes wording (“complementary”).
docs/embed.md	Updates proof example directive spelling.
docs/document-parts.md	Adds section explaining how parts appear in AST.
docs/developer.md	Adds local docs workflow notes and AST primer reference.
docs/deployment-gitlab.md	Restructures GitLab Pages deployment guidance (uv/poetry/pixi) and adds beginner steps.
docs/deployment-github-pages.md	Updates Node version used in GitHub Pages example.
docs/contribute-docs.md	Updates local docs build instructions to align with workspace docs setup.
bunfig.toml	Switches Bun linker to `isolated`.
bun.lock	Regenerates lockfile reflecting isolated installs and dependency changes.
.changeset/yellow-bikes-carry.md	Changeset for proof alias additions.
.changeset/vast-brooms-clap.md	Changeset for Node parallelism update.
.changeset/true-buses-fail.md	Changeset for execution/services dependency updates.
.changeset/tangy-cameras-cross.md	Changeset for broad package metadata updates.
.changeset/silly-phones-warn.md	Changeset for citation HTML entity decoding.
.changeset/nine-times-dream.md	Changeset for Windows path handling fix.
.changeset/neat-shirts-attend.md	Changeset for GH actions/node bumps.
.changeset/khaki-papayas-jog.md	Changeset for tex-to-typst bump and mystmd patch.
.changeset/grumpy-lions-push.md	Changeset for adding public.json manifest.
.changeset/fruity-ties-speak.md	Changeset for DOI coercion behavior relaxation.
.changeset/fifty-waves-smile.md	Changeset for execution option tweak.
.changeset/eighty-dots-train.md	Changeset for TOC glob exclusion behavior.
.changeset/dirty-steaks-burn.md	Changeset for switching to ipynb build cache (minor bump).
.changeset/all-planets-know.md	Changeset for type export fix.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+      opts.cache.set(cacheKey, {
+        context: {
+          kernelspec,
+          path: path.basename(opts.basePath, vfile.path),


+export async function writePublicJson(session: ISession) {
+  const paths = [];
+  const publicPath = session.publicPath();
+  const dir = await opendir(publicPath, { recursive: true });
+  for await (const path of dir) {
+    const subPath = relative(publicPath, join(path.parentPath, path.name));
+    paths.push(`/${subPath}`);
+  }
+
+  const filename = join(session.sitePath(), 'public.json');
+  writeFileToFolder(filename, JSON.stringify(paths));
+}


+          const resp = await fetchWithRetry(session, route.url);
+          if (!resp.ok) {
+            session.log.error(`Error fetching ${route.url}`);
+            return;
+          }


+'myst-cli': patch
+---
+
+Bump @jupyterlite/services and fix types


+This primer covers both **the MyST Specification** (this site) and **mystmd** (the official reference implementation):
+
+- The **spec** defines what a conforming MyST parser must produce - node types, their properties, and how they correspond to source. It evolves through the [MyST Enhancement Proposal process](https://mep.mystmd.org).
+- **mystmd** is a command-line application that bundles several modular packages that make use of MyST. `myst-parser` is a parser for this spec, `myst-transforms` have a collection of useful transforms to modify a MyST AST (e.g., resolving cross-references and appling numbering numbering). `mystmd` may emits additional node types that aren't yet in the spec as part of its build pipeline (like [named parts](xref:guide#parts-ast)).


+  - `proof:axiom`
+  - `proof:conjecture`
+* - `proof:corollary`
+  - `proof:criteria`


+'mystmd': patch
+---
+
+Use NodeJS parallelism parallelism feature


agoose77 and others added 25 commits June 15, 2026 17:00

fix: types for bun (jupyter-book#2956)

71aec67

Use availableParallelism rather than CPUs (jupyter-book#2957)

659279c

* fix: use availableParallelism rather than cpus This (should) gives physical cores. * chore: add changeset * fix: restore parallelism * Apply suggestion from @agoose77

Document multi-page transform pattern (jupyter-book#2950)

f2af8e9

🔗 Relax overaggressive DOI coercion (jupyter-book#2960)

aba633a

fix: tolerate failure on unconfigured favicon (jupyter-book#2982)

aa08678

Add proof:kind spelling for proof directives and ref role (jupyter-bo…

403caed

…ok#2983)

Update Node.js version to 24.x in GitHub Pages setup (jupyter-book#2990)

efabece

node-version: 18.x is depreciated on GitHub

Rewrite docs (jupyter-book#2997)

f33e0fd

Co-authored-by: Rowan Cockett <rowanc1@gmail.com>

✊ Bump tex-to-typst to 0.0.21 (jupyter-book#3008)

41782d2

See jupyter-book#3007

Bump actions and node (jupyter-book#3015)

4de8d72

Authors field in pixi.toml should be list of strings (jupyter-book#3017)

105739a

* Authors field in pixi.toml should be list of strings * Strip whitespace

Change execution session type to console (jupyter-book#3019)

e8045bc

* fix: set console type * chore: add changeset

Add public manifest to site builds (jupyter-book#3002)

cf9086e

* feat: add public manifest * chore: add changeset * docs: add note about public.json

Decode HTML entities in CSL text fields (jupyter-book#2986)

4f12f1b

CrossRef's CSL-JSON metadata sometimes contains HTML entities in title and journal name fields. Co-authored-by: Angus Hollands <goosey15@gmail.com>

normalize kernel path with forward slashes (jupyter-book#3023)

1a1e16b

chore: add changeset

d41a821

Copilot AI lite review requested due to automatic review settings August 10, 2026 10:50

Copilot started reviewing on behalf of mmcky August 10, 2026 10:51 View session

Copilot AI reviewed Aug 10, 2026

View reviewed changes

mmcky merged commit 44cf062 into main Aug 10, 2026
7 checks passed

mmcky mentioned this pull request Aug 10, 2026

Cut qe-v9 and validate the 2026-08-10 upstream sync in the lecture books #79

Open

6 tasks

mmcky deleted the chore/sync-upstream-2026-08-10 branch August 13, 2026 05:51

Navigation Menu

Uh oh!

Sync with upstream/main (23 commits) - #78

Sync with upstream/main (23 commits)#78
mmcky merged 25 commits into
mainfrom
chore/sync-upstream-2026-08-10

mmcky commented Aug 10, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

13 participants

Uh oh!

Conversation

mmcky commented Aug 10, 2026

proof:kind spelling (jupyter-book#2983)

Isolated bun linker (jupyter-book#2987)

.ipynb build cache (jupyter-book#2970)

Other

Verification

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

13 participants

`proof:kind` spelling (jupyter-book#2983)

`.ipynb` build cache (jupyter-book#2970)