Skip to content

ci: add root Jenkinsfile to onboard sqlalchemy-exasol into the preset-io Jenkins org folder - #3

Closed
aminghadersohi wants to merge 1 commit into
masterfrom
aminghadersohi/ch121045/jenkins-multibranch-onboarding
Closed

aminghadersohi wants to merge 1 commit into
masterfrom
aminghadersohi/ch121045/jenkins-multibranch-onboarding

Conversation

@aminghadersohi

Copy link
Copy Markdown
Collaborator

Why this, and not an argocd change

The ask was to make this repo's multibranch PR jobs run on production Jenkins. I traced the declarative onboarding path end to end; no argocd or Helm change is required, so this PR is in the repo that actually gates job creation.

Evidence chain:

Link Location Finding
Org folder definition preset-io/helm → charts/jenkins/values.yaml, JCasC seed-jobs script A Job DSL organizationFolder('preset-io') with a GitHubSCMNavigator for repoOwner('preset-io')
Name allowlist same file WildcardSCMSourceFilterTrait.includes('{{ .Values.orgScanIncludes }}')
Value in production preset-io/argocd → environments/devops/aws/regions/us-west-2/devops-aws-production/jenkins/values.yaml:6 orgScanIncludes: "*" — already matches this repo
Name denylist preset-io/helm charts/jenkins/values.yaml (hardcoded, not env-overridable) excludes('archived/* charts incubator-superset Flask-AppBuilder pyhelm superset-private redis-py-cluster terraform-aws-rds fab_multitenant helm-secrets s3werkzeugcache') — this repo is not listed
Credentials same file credentialsId('gh-preset-machine') — a machine user, not a GitHub App, so there is no per-repository installation allowlist to extend. This repo is public, so it is readable
Forks are not excluded preset-io/PyHive and preset-io/snowflake-connector-python are forks with root Jenkinsfiles and live jobs Fork status is not a discovery blocker

So the org scan already visits this repository. What it does not do is create a job, because an OrganizationFolder only materialises a multibranch project when the marker file — a root Jenkinsfile — is found on a discovered branch. This repo had none. Adding it is the smallest change that actually alters behaviour; editing orgScanIncludes would have been a no-op.

Rollout — fully automated, no manual Jenkins setup

Once this merges to master, the job appears via any of three existing triggers, whichever fires first:

  1. Webhook (seconds). gitHubPluginConfig.hookUrl in the argocd values routes GitHub events through webhookrelay to Jenkins, re-indexing the org folder on push.
  2. Periodic scan (within 24h). The seed job registers PeriodicFolderTrigger with spec('H H * * *') / interval(86400000).
  3. Controller start. The trigger-org-scan Groovy init script in the chart kicks an preset-io org folder build on boot.

No one needs to click anything in Jenkins.

Branch/PR coverage comes from the traits already on the navigator: BranchDiscoveryTrait(strategyId 1) (all branches) and OriginPullRequestDiscoveryTrait(strategyId 2) (all origin PRs). This repo's PR branches are origin branches, so they are covered.

What the pipeline does

Validation only. It publishes nothing: no sdist, no wheel, no registry upload, no git tag, no credentials block, no internal hostnames.

  • pip install -e . plus pytest
  • pytest test/unit --junitxml=junit-py.xml, with the JUnit report published to Jenkins
  • Assert the exa.websocket dialect entry point resolves to a real Dialect class

test/integration is excluded: it drives a real Exasol database via exasol-integration-test-docker-environment. The existing GitHub Actions workflows are unchanged.

One thing worth flagging for 6.1

The smoke check deliberately asserts only exa.websocket, not the bare exa alias. On master today (7.1.2), pyproject.toml:54 declares:

"exa" = "sqlalchemy_exasol:base"

That resolves to the module sqlalchemy_exasol.base, not a Dialect class — I reproduced it locally under SQLAlchemy 2.0.54. This is the same malformed-alias behaviour already documented in preset-io/superset-shell (docker/pip-cleanup.json), which notes that 7.1.3 must own both Exasol aliases and both must load the websocket dialect class. Asserting exa here would land a red job on day one for a defect this PR is not fixing, so it stays out and is tracked with the 7.1.3 entry-point work. Once 7.1.3 lands, exa should be added to EXPECTED in the Jenkinsfile.

Validation performed

Clean venv, Python 3.11.2, replaying the exact pipeline commands:

  • Groovy 4.0.24 compiler parsed the Jenkinsfile to Phases.CONVERSION — PARSE OK
  • pip install -e . — OK
  • pytest test/unit --junitxml=junit-py.xml — 64 passed in 0.34s, JUnit XML written
  • exa.websocket resolves to sqlalchemy_exasol.websocket.EXADialect_websocket — OK

Rollback

Revert this commit, or delete the file. The next org re-index (webhook or the daily scan) removes the multibranch project; orphanedItemStrategy { pruneDeadBranches(true) } on the org folder cleans it up. Nothing outside this repo changes, and no shared Jenkins state is mutated — so rollback is a single-repo revert with no coordination.

Explicitly out of scope

No artifacts published, nothing merged, no public PyPI workflow enabled, no internal hostnames added, no secrets introduced.

🤖 Generated with Claude Code

The production Jenkins org folder scans preset-io with orgScanIncludes='*',
so this repository is already crawled; a multibranch job is only materialised
once a root Jenkinsfile exists. This adds one.

The pipeline is validation-only and publishes nothing: it installs the package,
runs the offline unit suite (64 tests) with JUnit reporting, and asserts that
the exa.websocket dialect entry point resolves to a real Dialect class.

test/integration is excluded because it drives a real Exasol database. The bare
`exa` alias is not asserted: upstream declares it as `sqlalchemy_exasol:base`,
which resolves to a module rather than a Dialect class, and that is tracked
separately as part of the 7.1.3 entry-point fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@aminghadersohi

Copy link
Copy Markdown
Collaborator Author

CI note: the Merge Gate / Fast Report / report failure is pre-existing, not from this PR

This PR's diff is a single new file (Jenkinsfile), which GitHub Actions never reads — so I dug into why that job went red here but is green on #1 and #2. Root cause:

report.yml on master runs the Sonar upload unconditionally:

# .github/workflows/report.yml@master, line 47
- id: upload-to-sonar
  env:
    SONAR_TOKEN: "${{ secrets.SONAR_TOKEN }}"
  run: poetry run -- nox -s sonar:check

This fork has no Actions secrets at all — both repos/preset-io/sqlalchemy-exasol/actions/secrets and the org-secrets listing come back empty. So SONAR_TOKEN expands to the empty string, and the job dies exactly there:

nox > pysonar --sonar-token '' ...
ERROR: Not authorized or project not found. Please check the 'SONAR_TOKEN' environment variable ...
nox > Session sonar:check failed.

#1 and #2 are green on this job only because they are the fix: their branches add a guard that master does not yet have.

# report.yml on workflow-a/ch120359-publication, line 48
  if: "github.repository != 'preset-io/sqlalchemy-exasol'"

For pull_request events the workflow is taken from the PR's own merge ref, so those PRs skip Sonar while this one — branched from master — still runs it. Comparing the two job logs confirms it: mine ran artifacts:copy, artifacts:validate, sonar:check; #2's ran only artifacts:copy, artifacts:validate.

Implication: nothing to fix here. Once #2 (or any PR carrying that guard) lands on master, this job goes green on this PR automatically. I deliberately did not cherry-pick the one-line guard, since that would duplicate and conflict with #2.

Everything else is green, including all 5 Fast Tests / Unit Tests matrix legs — the same suite this PR's Jenkins pipeline runs.

@aminghadersohi

Copy link
Copy Markdown
Collaborator Author

Addendum: the same empty-SONAR_TOKEN failure has now also landed on Report / report (job 104628608623) — identical pysonar --sonar-token '' → Not authorized or project not found → Session sonar:check failed. Both Sonar-invoking jobs, one cause, same fix (#2's `if: "github.repository != 'preset-io/sqlalchemy-exasol'"" guard). All 63 other checks on this PR are green.

@aminghadersohi

Copy link
Copy Markdown
Collaborator Author

Closing as superseded by #2, which contains the publication-capable root Jenkinsfile and avoids competing Jenkinsfile implementations.

This branch was successfully deployed

1 active deployment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant