[Feature] MiniMax H3 video support

[jadbox]

Feature Summary

MiniMax H3 video generation

Detailed Description

Please support the OS model MiniMax H3 that had its release today. Either the int8 or the GGUF version would be ideal to support.
https://www.reddit.com/r/StableDiffusion/comments/1ve67ny/minimax_h3_smaller_other_quants/
https://huggingface.co/Abiray/MiniMax-H3-GGUF

Alternatives you considered

No response

Additional context

No response

[Christopher-Chianelli]

FYI, there are GGUF's avaiable that work with stable-diffusion.cpp, available at https://huggingface.co/leejet/MiniMax-H3-GGUF.
Usage example:

sd-cli \
    -M vid_gen \
    --diffusion-model /models/minimax_h3_ref2va_pruned-Q2_K_M.gguf \
    --vae /models/minimax_h3_video_vae_fp16.safetensors \
    --audio-vae /models/minimax_h3_audio_vae_fp32.safetensors \
    --llm /models/qwen3vl_32b_minimax_h3-Q2_K_M.gguf \
    --cfg-scale 1.0 \
    -W 864 \
    -H 480 \
    --diffusion-fa \
    -p "A jumping cat" --video-frames 124 \
    -o /output/video.avi

Note: Q2 barely fits on a 7900 XTX when generating 124 frames on the vulkan backend. You could run Q4 On 7900 XTX, but then you are limited to basically generating a single frame at a time.

[Christopher-Chianelli]

If you really, really, minimize the width and height, you can generate a video using Q4 on the vulkan backend for 7900 XTX:

sd-cli \
    -M vid_gen \
    --diffusion-model /models/minimax_h3_ref2va_pruned-Q4_K_M.gguf \
    --vae /models/minimax_h3_video_vae_fp16.safetensors \
    --audio-vae /models/minimax_h3_audio_vae_fp32.safetensors \
    --llm /models/qwen3vl_32b_minimax_h3-Q4_K_M.gguf \
    --cfg-scale 1.0 \
    -W 64 \
    -H 32 \
    --diffusion-fa \
    -p "A jumping cat" --video-frames 120 \
    -o /output/video.avi

That being said, it does appear you need to ground the model with a base image; a lot of the video generated look like TV static from a text only prompt. (Haven't tried with an image input yet; I do know generating a single frame with Q4 with a text only input creates something coherent, but across multiple frames it basically become static/noise).

[kwilkins-82]

I notice some are getting good results with an 8 bit quant and as little as 12GB VRAM, using Comfy. Is Comfy better at offloading weights to system memory?

[Green-Sky]

Try --offload-to-cpu --max-vram X with X in GiB. Also if you are on vulkan, try to enable direct conv.

[kwilkins-82]

I got it to run with a Q2 quant, but getting a black output. This is with Vulkan, will try ROCM later.

[AmeowCAT]

It seems that sd-server still does not support ref2va.

[Christopher-Chianelli]

Try --offload-to-cpu --max-vram X with X in GiB. Also if you are on vulkan, try to enable direct conv.

Alas, --offload-to-cpu --max-vram 22 basically hard crashes my computer (for generating 120 frames of 864 x 480); from the final logs, it appears the RAM that was allocated but enough to basically fill RAM + SWAP, but just low enough not to trigger OOM Killer (I built my system when 32 GiB was considered excessive; how have the times changed); couldn't do anything without a system reset. direct conv on diffusion and vae appears to work though.

[kwilkins-82]

I got it to run using -"-offload-to-cpu --max-vram -2 --clip-on-cpu --vae-tiling" on ROCM, but the results are not good at all. The Q2 model just produces a smudge - minimax_h3_fl2va_pruned-Q4_K_M.gguf is a bit better in that it's giving me a real video, but it's not following my prompt at all and the audio is just white noise. I tried setting --guidance 4.0 but it didn't make any difference, nor did using the Q4 qwen quant.