[dist] Shade and relocate kafka-clients and jackson in flink-agents-dist

[ysymi]

Linked issue: #652

Purpose of change

flink-agents-dist-*.jar is a fat JAR that bundles third-party dependencies
without relocation. When users add it via --jars or place it in Flink's
lib/, its dependencies share the ClassLoader with the user's job JAR, and
version mismatches surface as NoSuchMethodError / ClassCastException at
runtime.

This PR adds shade relocation to dist/pom.xml. Since dist is the parent
pom (packaging=pom), a single change applies to all five Flink-version
submodules (flink-1.20 / 2.0 / 2.1 / 2.2 / 2.3).

What is relocated (the two entries rated High/Medium in the issue's risk
table, present in almost every streaming job):

• org.apache.kafka → org.apache.flink.agents.shaded.org.apache.kafka
• com.fasterxml → org.apache.flink.agents.shaded.com.fasterxml

What is intentionally not relocated, and why:

• kotlin-stdlib / okhttp3 / okio: the shade plugin's resource-name string
  replacement corrupts Kotlin's .kotlin_builtins metadata files (verified in
  our internal production build). okhttp3/okio are Kotlin-based and must move
  together with kotlin-stdlib, so they stay unshaded as a group.
• vendor SDKs (openai-java, anthropic-java): rated Low/Medium-low in the
  issue itself; relocating them risks breaking SDK-internal hardcoded class
  names for little benefit. Can be follow-ups if real conflict reports appear.
• Packages Flink already shades in flink-shaded-* (e.g. guava, netty) are
  not repeated.

Necessary companion changes:

• ServicesResourceTransformer merges META-INF/services entries across
  bundled JARs so relocated SPI implementations stay discoverable.
• Third-party META-INF/versions/21/ classes are dropped — the shade plugin
  cannot reliably process them and the base versions are used instead. Our own
  versions/21/org/apache/flink/** entries (JDK 21 Continuation support) are
  preserved.

Tests

Verified by building dist/flink-2.3:

• Relocated packages contain no un-relocated residue (0 classes under the
  original org/apache/kafka/ and com/fasterxml/ paths).
• Bytecode spot-check (javap -c): runtime/plan classes reference only the
  shaded paths; remaining mentions are log-message string constants.
• runtime unit tests pass unchanged.

Note: this is a pom-only change; community CI (./tools/ut.sh,
./tools/lint.sh) covers the full build across versions.

API

No public API changes.

Documentation

• doc-needed
• doc-not-needed
• doc-included

Was this patch authored or co-authored using generative AI tooling?

• Yes
• No

Generated-by: Pi coding agent 0.85.1 (glm-5.3)