fix: remove tensor_name_count retention budget dimension for remote SafeTensors - #1822
Conversation
…afeTensors The tensor_name_count dimension in the HuggingFace SafeTensors retention budget caused large multi-shard models (e.g. GLM-5.2-FP8 with 141 shards) to fail closed prematurely at shard ~84/141. The result_bytes cap (32 MB) already bounds aggregate serialized size, making tensor_name_count redundant. Remove _MAX_HF_SAFETENSORS_RETAINED_TENSOR_NAMES and the tensor_name_count exceeded check from the retention budget. The result_count (512) and result_bytes (32 MB) dimensions remain as protection against pathological inputs. Tested locally against 6 HuggingFace models from 16 GB to 594 GB with zero budget failures. GLM-5.2-FP8 confirmed as the only model affected on main (budget exceeded at shard 84/141, retained_tensor_names: 65380). Co-authored-by: Cursor <cursoragent@cursor.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex Review: Didn't find any major issues. Another round soon, please! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
Security review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
2 participants
Fixes #1823
Summary
The
tensor_name_countdimension in_HuggingFaceSafeTensorsRetentionBudgetcaused large multi-shard SafeTensors models to fail closed prematurely. Specifically,RedHatAI/GLM-5.2-FP8(753B params, 141 safetensors shards, ~756 GB) hit the 65,536 tensor-name cap at shard ~84/141 and produced an incomplete scan (exit 2).The
result_bytescap (32 MB serialized JSON) already bounds aggregate output size, makingtensor_name_countredundant — removing it lets large models complete without losing protection against pathological inputs.Changes
_MAX_HF_SAFETENSORS_RETAINED_TENSOR_NAMESconstant and thetensor_name_countexceeded check from_HuggingFaceSafeTensorsRetentionBudget.retain().max_retained_tensor_namesin preflight and retain return dicts.result_bytesinstead.[Unreleased].The
result_count(512) andresult_bytes(32 MB) budget dimensions remain unchanged.Testing
Tested locally against 6 HuggingFace models spanning a wide range of sizes:
RedHatAI/SmolLM3-3B-FP8-dynamicQwen/Qwen3-8Bgoogle/gemma-4-26B-A4B-itRedHatAI/gpt-oss-120b-FP8-Dynamicmoonshotai/Kimi-K2-ThinkingRedHatAI/GLM-5.2-FP8tensor_name_countexceeded at shard 84)GLM-5.2-FP8 is the only model that failed on
main. All six pass on the fix branch with zero budget failures.All clean.