Pinned Loading

1. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 88.4k 20.4k

2. llm-d/llm-d-router llm-d/llm-d-router Public

   llm-d Router: The intelligent entry point for inference requests

   Go 282 300

3. kubernetes-sigs/inference-perf kubernetes-sigs/inference-perf Public

   GenAI inference performance benchmarking tool

   Python 218 114

4. NVIDIA/dcgm-exporter NVIDIA/dcgm-exporter Public

   NVIDIA GPU metrics exporter for Prometheus leveraging DCGM

   Go 1.8k 320

5. kubeflow/kubeflow kubeflow/kubeflow Public

   Machine Learning Toolkit for Kubernetes

   15.8k 2.7k

6. SeldonIO/MLServer SeldonIO/MLServer Public

   An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more

   Python 897 238