Moved from FUTURE-IDEAS.md when that file was retired in favour of tracker issues (2026-08-13 maintenance round). This was the file's note for the benchmarking programme — see QuantEcon/meta#335.
The framing
Benchmark data capture stays in the programme repos (QuantEcon/benchmarks — workstream A; QuantEcon/tool-lecture-benchmark — workstream C). Skills in this marketplace are natural consumers and interfaces of that data. /benchmark:review-acceleration is workstream B's deliverable and already lives here; three more consumers are unbuilt:
.jupyter_cache telemetry reader — nbclient already records per-cell timings in every CI build; nothing surfaces them. A skill plus a ~50-line script reading _build/.jupyter_cache (or a CI artifact) and diffing a PR preview against main ("cell solve-ex4 went 8 ms → 950 ms") is the cheapest high-value entry point, and seeds workstream C. It would have flagged the lecture-python.myst#717 regression the day that PR opened.- Crossover advisory — once workstream A produces crossover curves, a skill answering "this lecture solves n=2 economies 100 times — which backend?" from measured data rather than rules of thumb, during authoring or review.
- Provenance/schema guardian — generate and validate the shared result + environment-descriptor stamp (meta#335's first blocking deliverable) across all three workstreams' outputs.
Assessment (2026-08-13)
Item 1 is the actionable one: it depends on nothing from the other workstreams, the data already exists in every CI build, and #717 supplies a concrete regression it would have caught. It is the natural next benchmark-plugin skill after the triage-first release (#33) lands. Items 2 and 3 are blocked on workstream A data and the meta#335 schema deliverable respectively — they should stay parked here until those exist, and this issue is their pointer.
Moved from
FUTURE-IDEAS.mdwhen that file was retired in favour of tracker issues (2026-08-13 maintenance round). This was the file's note for the benchmarking programme — see QuantEcon/meta#335.The framing
Benchmark data capture stays in the programme repos (
QuantEcon/benchmarks— workstream A;QuantEcon/tool-lecture-benchmark— workstream C). Skills in this marketplace are natural consumers and interfaces of that data./benchmark:review-accelerationis workstream B's deliverable and already lives here; three more consumers are unbuilt:.jupyter_cachetelemetry reader — nbclient already records per-cell timings in every CI build; nothing surfaces them. A skill plus a ~50-line script reading_build/.jupyter_cache(or a CI artifact) and diffing a PR preview against main ("cellsolve-ex4went 8 ms → 950 ms") is the cheapest high-value entry point, and seeds workstream C. It would have flagged the lecture-python.myst#717 regression the day that PR opened.Assessment (2026-08-13)
Item 1 is the actionable one: it depends on nothing from the other workstreams, the data already exists in every CI build, and #717 supplies a concrete regression it would have caught. It is the natural next benchmark-plugin skill after the triage-first release (#33) lands. Items 2 and 3 are blocked on workstream A data and the meta#335 schema deliverable respectively — they should stay parked here until those exist, and this issue is their pointer.