Skip to content

Build: Exclude unpublished Kafka Connect runtime from the BOM - #18056

Open
vbhanuchander-lang wants to merge 1 commit into
apache:mainfrom
vbhanuchander-lang:bom-exclude-unpublished-kc-runtime
Open

Build: Exclude unpublished Kafka Connect runtime from the BOM#18056
vbhanuchander-lang wants to merge 1 commit into
apache:mainfrom
vbhanuchander-lang:bom-exclude-unpublished-kc-runtime

Conversation

@vbhanuchander-lang

Copy link
Copy Markdown
Contributor

Closes #18052.

The BOM adds every leaf project, but iceberg-kafka-connect-runtime builds ZIP distributions rather than Maven artifacts and disables its publishing tasks, so its POM is never published — the 1.11.0 POM returns 404. The BOM therefore advertises a coordinate that cannot resolve.

The grouping project iceberg-kafka-connect is already excluded by the existing childProjects.isEmpty() check, so the runtime module is the only affected one — it is the only publishing-disabled leaf project in the build.

Verifying this change

./gradlew :iceberg-bom:generatePomFileForApachePublication, then inspecting bom/build/publications/apache/pom-default.xml:

artifacts in BOM iceberg-kafka-connect-runtime
before 30 present
after 29 absent

Exactly one entry is removed. iceberg-kafka-connect, iceberg-kafka-connect-events and iceberg-kafka-connect-transforms are real published artifacts and remain.

I used an explicit exclusion list rather than deriving "does not publish" from the project state, because publishing is disabled inside that project's afterEvaluate while the BOM's constraints block is evaluated during configuration, so keying off task state here would depend on configuration ordering. Happy to switch to a shared property (for example a skipPublish flag set by the project and read here) if you would prefer something that cannot be forgotten when the next distribution-only module is added.

Generated-by: Claude Code (Opus 5)

The BOM adds every leaf project, but iceberg-kafka-connect-runtime builds ZIP
distributions rather than Maven artifacts and disables its publishing tasks, so
its POM is never published. The BOM therefore references a coordinate that
resolves to a 404, which fails resolution for anyone importing it.

Skip projects that are known not to publish. iceberg-kafka-connect,
iceberg-kafka-connect-events and iceberg-kafka-connect-transforms are real
artifacts and stay in the BOM.
@github-actions github-actions Bot added the build label Sep 11, 2026
Comment thread build.gradle
]
// Projects that build a distribution rather than a Maven artifact disable their publishing
// tasks, so their POM is never published and the BOM must not reference it.
def unpublishedProjects = ['iceberg-kafka-connect-runtime']

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unpublishedProjects is a hardcoded one-entry list, a future distribution-only module that disables publishing in afterEvaluate must be added to this list manually or the BOM silently re-references an unpublished POM, reproducing the exact bug this PR fixes.

You already surfaced a self-maintaining alternative: a shared ext.skipPublish = true flag set in the project's own build file and readable in the configuration phase, which would handle future cases automatically. The explicit list is technically sound given the afterEvaluate-vs-configuration ordering constraint (reading task state in the configuration phase would be order-dependent), but Iceberg's connector surface is expanding, so perhaps we should consider the other approach?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed on the concern, so I tried the skipPublish approach properly before answering. It does not work here, and the reason is structural rather than a detail I can tidy up. Two experiments, both measured by regenerating the BOM POM:

1. ext.skipPublish = true in the runtime project, read with findProperty('skipPublish') in the BOM — 30 artifacts, runtime still present. The BOM's constraints block lives in the root build file, and Gradle evaluates the root before subproject build files, so at that moment kafka-connect/build.gradle has not run and the flag does not exist yet. This is the same ordering problem as reading task state, one step earlier.

2. Same flag, with the constraint population deferred inside gradle.projectsEvaluated { } so every project has been configured first — also 30. Constraints added at that point no longer reach the java-platform component.

So for a project-owned flag to work it would have to exist before root configuration, which in practice means declaring it in settings.gradle or root gradle.properties — still a central list, just moved further away from the project it describes.

Given that, the options I can see:

  • keep the explicit list as-is;
  • move the list into settings.gradle beside the include/name lines for these projects, so adding a distribution-only module puts the declaration and the exclusion in the same file;
  • something lazier that I have not found — if you know a way to keep java-platform constraints open past project evaluation I am happy to try it.

My preference is the second, since it puts the exclusion where a new module is registered, but I do not feel strongly. Happy to push whichever you prefer.

For reference the current state is verified: 30 artifacts before, 29 after, with iceberg-kafka-connect, -events and -transforms retained.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @vbhanuchander-lang! Option 2 sounds reasonable to me, but please ping the relevant / expert Iceberg committers here for final decision on how to proceed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Iceberg BOM references unpublished Kafka Connect runtime artifact

2 participants