Skip to content

feat(kv): store INT4/NVFP4 KV pages in the cache manager and read them with Triton - #174

Draft
HowardYangAI wants to merge 15 commits into
InferMatrixLab:mainfrom
HowardYangAI:feat/kv-quant-integration
Draft

HowardYangAI wants to merge 15 commits into
InferMatrixLab:mainfrom
HowardYangAI:feat/kv-quant-integration

perf(kv): attend over shared pages through one decode, not the fused …

8e35940
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs