Summary
Ship each maintained ol-openedx-* plugin's test suite inside its built
distribution and declare a tests extra bundling its test-only dependencies,
so the tests can be installed and run against a real, already-built Open edX
platform via pip install "<dist>[tests]==<version>".
Why
lehrer added a plugin-regression runner (dagger call platform plugin-regression, lehrer#111)
that runs each installed plugin's own pytest suite inside a built cell image
(the real edx-platform version + the deployment's pinned plugin set + aqueduct
settings). It uses pytest discovery: it runs whatever tests are installed in
the image, resolving packages at runtime.
Today it discovers nothing, because:
- published plugin wheels/sdists ship no test files (verified: the
ol-openedx-logging==0.3.5 sdist contains only settings/test.py, no
tests/), and
- this monorepo has no per-package tags, so there's no way for a downstream
builder to fetch "the tests at the exact installed version."
lehrer already installs "<dist>[tests]==<pinned-version>" opportunistically —
which is a safe no-op until the extra exists — so the moment a plugin ships its
tests + a tests extra, its suite starts running in that pipeline with no
lehrer change required.
What to do (per maintained plugin)
Roll out incrementally — one plugin unblocks real signal for that plugin.
Verify
Once at least one plugin ships the extra, from a lehrer checkout:
lehrer build plugin-regression --cell mit-ol/master/mitxonline \
--custom-settings ./deployments/mit-ol/settings
should collect and run that plugin's suite (instead of "no tests discovered").
Tracked in the lehrer CI-verification project; companion CI-wiring task is
blocked on this.
Summary
Ship each maintained
ol-openedx-*plugin's test suite inside its builtdistribution and declare a
testsextra bundling its test-only dependencies,so the tests can be installed and run against a real, already-built Open edX
platform via
pip install "<dist>[tests]==<version>".Why
lehrer added a plugin-regression runner (
dagger call platform plugin-regression, lehrer#111)that runs each installed plugin's own pytest suite inside a built cell image
(the real edx-platform version + the deployment's pinned plugin set + aqueduct
settings). It uses pytest discovery: it runs whatever tests are installed in
the image, resolving packages at runtime.
Today it discovers nothing, because:
ol-openedx-logging==0.3.5sdist contains onlysettings/test.py, notests/), andbuilder to fetch "the tests at the exact installed version."
lehrer already installs
"<dist>[tests]==<pinned-version>"opportunistically —which is a safe no-op until the extra exists — so the moment a plugin ships its
tests + a
testsextra, its suite starts running in that pipeline with nolehrer change required.
What to do (per maintained plugin)
tests/in the built distribution (package data /don't exclude tests), so
pip install <dist>lands the tests insite-packages where
pytest --pyargs <module>can discover them.testsoptional-dependency extra with the test-only deps thesuite needs (pytest-mock, responses, factory_boy, etc.), matching what
run_edx_integration_tests.shinstalls today.--ds=lms.envs.test(or the plugin's ownstandalone settings) against an installed edx-platform.
Roll out incrementally — one plugin unblocks real signal for that plugin.
Verify
Once at least one plugin ships the extra, from a lehrer checkout:
should collect and run that plugin's suite (instead of "no tests discovered").
Tracked in the lehrer CI-verification project; companion CI-wiring task is
blocked on this.