Repository navigation
Some shipped recipes and cloud stacks leave built-in metrics empty #524
Description
What happened?
Some configurations Modelplane ships leave built-in modelplane_* metrics empty, with no error. The collector drops everything it didn't rename, so when a source metric isn't emitted, the modelplane_* name just never appears at the sink. In each case below Modelplane's own manifests or stacks produce the gap.
Engine recipes that don't emit the mapped names
- The llama-3.1-8b recipe pins
vllm/vllm-openai:v0.7.3(model-deployment.yaml#L47). That version runs the V0 engine by default (envs.py#L503-L504), which publishesvllm:gpu_cache_usage_percand prefix-cache hit-rate gauges (engine/metrics.py#L85-L106) instead of the names in metrics.py#L54-L57.modelplane_kv_cache_utilization_ratio,modelplane_prefix_cache_hits_totalandmodelplane_prefix_cache_lookups_totalare empty for it. Every other pinned vLLM version, v0.9.2 and later, publishes all of them. - Neither SGLang recipe passes
--enable-metrics: laguna/model-deployment-sglang.yaml#L46-L57 (v0.5.12.post1) and qwen3-coder/model-deployment-fp8.yaml#L46-L61 (v0.5.10.post1). SGLang only mounts/metricswith that flag (http_server.py#L302-L304), so every_SGLANGname is empty for both. The qwen3-coder recipe also lacks--collect-tokens-histogram, which v0.5.10 needs for the token histograms. telemetry.md#L240-L245 tells users SGLang needs both flags and that "vLLM needs nothing", which isn't true for v0.7.3.
Cloud stacks that install no GPU exporter
- Civo turns the exporter off: civo.py#L280 sets
dcgmExporter.enabled: Falsebecause its ServiceMonitor needs the Prometheus CRDs (#L268-L272). Everymodelplane_gpu_*series andmodelplane_energy_joules_totalis empty on Civo.
#519 covers the same GPU gap for source: Existing clusters, where Modelplane installs nothing. This is about clusters Modelplane builds.
How can we reproduce it?
- Deploy the llama-3.1-8b recipe, or either SGLang recipe, with a TelemetryDestination configured.
- Send it some requests.
- Query the sink for
modelplane_kv_cache_utilization_ratio(llama) ormodelplane_requests_running(SGLang). No series exist for that deployment.
For GPUs, provision an inference cluster on Civo, deploy any model, and query modelplane_gpu_memory_used_bytes.
What environment did it happen in?
Modelplane version: main at 0ee6ab4 (also v0.5.0)
Engines: vLLM v0.7.3, SGLang v0.5.10.post1 and v0.5.12.post1, as pinned in the recipes
Clouds: Civo (from the stack code)
Found by reading the code and upstream source at the pinned versions, not on a live cluster
Metadata
Metadata
Assignees
Labels
Type
Projects
- StatusShow more project fieldsIn Review
Activity
dennis-upbound commentedon Oct 20, 2026
#526 fixed the Civo half: the stack now runs the DCGM exporter. The recipes remain, and I'd fix them for v0.6:
--enable-metricsto both SGLang recipes. Add--collect-tokens-histogramtoo while qwen3-coder stays on v0.5.10.post1. v0.5.12 always collects the histograms and accepts the flag as a no-op.docs/manifests/recipes/that fails on an SGLang engine without--enable-metrics, or on a vLLM image older than v0.9.2. Nothing checks recipes against the mappings today, which is how these shipped. The e2e test only checks two names, from a mock engine.Hold recipe engines to what the metric mappings read
Check the built-in mappings against scrapes of what they read
Hold recipe engines to what the metric mappings read
Check the built-in mappings against scrapes of what they read
Check the built-in mappings against scrapes of what they read
Hold recipe engines to what the metric mappings read
Check the built-in mappings against scrapes of what they read
Check the built-in mappings against scrapes of what they read
Hold recipe engines to what the metric mappings read
Check the built-in mappings against scrapes of what they read
Hold recipe engines to what the metric mappings read
Check the built-in mappings against scrapes of what they read