Repository navigation
Publish Iceberg kafka connect runtime to Confluent hub #10745
Description
Activity
cc: @bryanck, @danielcweeks, @jbonofre
It makes sense to me. I would also include the Kafka connect artifact on https://iceberg.apache.org/releases/
Reacted by Ajantha Bhat, Kyle Nguyễn, Johannes, gergelylendvai, Bryan Keller, Martin Egri, Jérémy Guiselin, Diego Mora Cespedes, Pieter and RobNaive question, for 1.8 are we saying that we're aiming for automation to push to Confluent Hub or is it something that for the 1.8 milestone, someone needs to manually do? If we're saying we want some automation for pushing, I feel like we should punt to the next release since i think that requires infra changes (e.g. I'm not sure if they even allow it, but if they did probably we'd need a key and whatnot)
Previously w/ the Tabular connector it was a manual process, we had a contact at Confluent that would upload it after performing some validation. We'd want a signed and published connector distribution before we'd tackle this I think.
I understand that this is a more complicated thing to do than uploading an artifact.
For the just released 1.9.0, however, there is as well no artifact for the Kafka Connect package uploaded, mentioned also already in #11685 for 1.7.0.It is quite cumbersome to locally build and add it to a Docker image, every adopter of Kafka Connect has to do the same. Can I somehow support on this issue?
Reacted by coldfusioncoder- changed the title
[-]Publish Iceberg kafka connect runtime to Confluet hub[/-][+]Publish Iceberg kafka connect runtime to Confluent hub[/+]on Apr 28, 2025 @thjaeckle Feel free to open a PR (or draft PR) if you've got it working.
If we can't finish this in a short time, can we attach kafka-connect zip files to https://github.com/apache/iceberg/releases and link them at https://iceberg.apache.org/releases/#downloads? At least, users don't need to build the zip files themselves.
Reacted by Pieter and liko16 remaining items
@rmoff: Could you please try now with 1.10.0 release?
Reacted by Robin Moffatt and raphaelauv(Discussion about the publishing process to Confluent Hub moved to the mailing list: https://lists.apache.org/thread/hlj8kordz6hmr9wf6ktttqchwty13vlt)
1.9.1 and 1.9.2 are available ->
https://www.confluent.io/hub/iceberg/iceberg-kafka-connect@rmoff @raphaelauv How about 1.10.0?
@manuzhang see my comment above - this needs someone to bundle it and the project agree a process for submitting it to Confluent Hub for publication.
I have started a mailing list discussion regarding formalising this process: https://lists.apache.org/thread/fc0dhqtv8717zbw01xvsc7sm203sohpd
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'
Hi @danielcweeks, since your last message on the discussion above, is this PR ready to be merged?
@humpfhumpf see #15212, currently blocked on licensing compliance (which I just opened a PR to attempt to clear, #18019)
Reacted by raphaelauv and humpfhumpf
Feature Request / Improvement
We should publish the OSS Apache Iceberg kafka connecter sink to confluent hub to get the wider visibility under "apache Iceberg" account.
Similar to https://www.confluent.io/hub/tabular/iceberg-kafka-connect
Query engine
Kafka Connect