Problem
The rdma.ome.io/auto-inject profile oci-roce (pkg/webhook/admission/pod/rdma_injector.go:31-92) injects UCX_TLS=tcp and UCX_NET_DEVICES=eth0 among the NCCL tuning. For NCCL workloads that's harmless, but for engines using UCX for the data path — e.g. vLLM P/D disaggregation with NIXL (UCX backend) — it silently forces every KV transfer onto TCP over eth0 while everything looks healthy. Nothing errors; you just quietly lose RDMA.
Since injection only fills absent env vars, runtimes can defend by declaring UCX vars explicitly — that's the workaround the moirai PD campaign used (and then counter-verified RoCE via NIC counters: 2.2GB moved per 22k-token prefill).
Suggested fix
Drop the UCX pins from the profile (let UCX auto-select transports), or split profiles: nccl-tuning vs ucx-kv-transfer. At minimum document that oci-roce is NCCL-oriented and UCX users must override.
Problem
The
rdma.ome.io/auto-injectprofileoci-roce(pkg/webhook/admission/pod/rdma_injector.go:31-92) injectsUCX_TLS=tcpandUCX_NET_DEVICES=eth0among the NCCL tuning. For NCCL workloads that's harmless, but for engines using UCX for the data path — e.g. vLLM P/D disaggregation with NIXL (UCX backend) — it silently forces every KV transfer onto TCP over eth0 while everything looks healthy. Nothing errors; you just quietly lose RDMA.Since injection only fills absent env vars, runtimes can defend by declaring UCX vars explicitly — that's the workaround the moirai PD campaign used (and then counter-verified RoCE via NIC counters: 2.2GB moved per 22k-token prefill).
Suggested fix
Drop the UCX pins from the profile (let UCX auto-select transports), or split profiles: nccl-tuning vs ucx-kv-transfer. At minimum document that oci-roce is NCCL-oriented and UCX users must override.