Repository navigation
Document GPU exporter and gateway metric limits in telemetry docs - #540
Open
tr0njavolta wants to merge 1 commit into
Open
tr0njavolta wants to merge 1 commit into
tr0njavolta wants to merge 1 commit into
Conversation
|
Docs preview: https://modelplane-docs-pr-540.vercel.app (ready once the site's Content workflow finishes) |
haarchri
reviewed
Oct 19, 2026
| yourself. The [scheduler pins each pool's pods to | ||
| it]({{< ref "/architecture/scheduling.md#pinning-placement-to-a-pool" >}}), so | ||
| worker pods stay Pending without it. | ||
| - **A DCGM exporter for GPU metrics.** Modelplane doesn't install a GPU |
Collaborator
There was a problem hiding this comment.
think we have this already here: https://github.com/modelplaneai/modelplane/pull/526/changes#diff-564c1fda2538b9d40dfe9971f6528982484594c4e4b25dadb0158280b326f97cR85 in this PR
Collaborator
There was a problem hiding this comment.
@tr0njavolta PR #526 is merged can you rebase and update here ?
Collaborator
dennis-upbound
left a comment
dennis-upbound
left a comment
Collaborator
There was a problem hiding this comment.
Thanks for dropping the duplicate DCGM bullet. The new content checks out against AI Gateway v1.1.0: token latency is recorded only for streaming responses (processor_impl.go), and the top buckets are 81.92s, 60s and 2.5s (genai.go).
A few things before it can merge:
- It conflicts with
mainintelemetry.mdnow that #546 and #548 are in. Could you rebase? - DCO fails because two commits aren't signed off. Squashing to one signed-off commit with a plain subject would fix it and match CONTRIBUTING.
main'saccept.txtalready hasDCGManddcgm-exporter, so those two lines can go.agenticis new and worth keeping.- Nit:
which requires streaming traffic:has a trailing space, and the hint's first line runs past the wrap the rest of the page uses.
tr0njavolta
force-pushed
the
docs/gpu-metrics
branch
from
October 22, 2026 16:55
b6fd0cd to
c81e7b3
Compare
The telemetry docs didn't say that the gateway records modelplane_frontend_ttft_seconds and modelplane_frontend_tpot_seconds only for streaming responses, so a deployment without streaming traffic has no series for either. They also didn't say that the gateway's histograms stop at fixed upper buckets: 81.92 seconds for request duration, 60 for time to first token, and 2.5 for time per output token in AI Gateway v1.1.0. A quantile above the top bucket reads as that bound no matter how long the request took. The metrics table and the p99 example now say both TTFT and TPOT need streaming traffic, and a note points non-streaming deployments at modelplane_frontend_request_duration_seconds. The bucket bounds are listed, and long requests are pointed at the engine's modelplane_request_duration_seconds, whose vLLM buckets reach 7680 seconds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Rae Sharp <resharp20@gmail.com>
tr0njavolta
force-pushed
the
docs/gpu-metrics
branch
from
October 22, 2026 16:58
c81e7b3 to
0219e3c
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of your changes
Fixes #519.
Fixes #520.
Modelplane doesn't install a GPU exporter. The collector's
modelplane-gpujob scrapes one that's already running: a pod whoseapporapp.kubernetes.io/namelabel containsdcgmand that exposes a container port namedmetrics. Managed clouds and the NVIDIA GPU Operator provide one, but asource: Existingcluster often has none, so nomodelplane_gpu_*series arrive. The requirements for an existing cluster now list a DCGM exporter and what the collector matches on, and the telemetry intro links to them.The gateway records
modelplane_frontend_ttft_secondsandmodelplane_frontend_tpot_secondsonly for streaming responses, and the telemetry docs didn't say so. The metrics table and the p99 example now say both need streaming traffic, and a note points non-streaming deployments tomodelplane_frontend_request_duration_seconds.From the comment on #520: the gateway's histogram buckets stop at fixed upper bounds (81.92s request duration, 60s TTFT, 2.5s TPOT in AI Gateway v1.1.0). A quantile above the top bucket reads as that bound, so the docs now list the bounds and point long requests at the engine's
modelplane_request_duration_seconds.The selector is checked against
functions/compose-serving-stack/function/collector.py. Neither behavior was reproduced on a cluster.I have:
nix flake check(or./nix.sh flake check) and made sure it passes.Added or updated tests covering any composition function changes.git commit -s.