QStore

QStore is a lossless, quantization-aware format for storing a high-precision model and its quantized counterpart together. It stores the INT8 weights plus the conditional information needed to reconstruct the BF16/FP16 weights, avoiding redundant storage while preserving both models bit for bit.

This repository contains the implementation for QStore: Quantization-Aware Compressed Model Storage, published in the Proceedings of the VLDB Endowment. The arXiv version is also available. The current version is an implementation update from the code used when writing the paper: the QStore algorithm and on-disk representation are unchanged, while model ingestion, pipelined loading, validation, and baseline measurement have been substantially improved.

Results

Storage

Each format stores the same matched BF16+INT8 weights. Sizes are GiB; the compression ratio is the uncompressed size divided by the format's own size.

Model pair	Raw	LZ4	Zstd-2	ZipNN + Zstd-2	QStore	QStore savings
Qwen2-Audio-7B	19.87	19.94 (1.00×)	16.19 (1.23×)	14.67 (1.35×)	9.46 (2.10×)	52.4%
Mistral-7B-v0.3	19.50	19.57 (1.00×)	15.86 (1.23×)	14.36 (1.36×)	9.11 (2.14×)	53.3%
Llama-3.1-8B	19.50	19.58 (1.00×)	15.83 (1.23×)	14.38 (1.36×)	9.04 (2.16×)	53.6%
Gemma-3-27B	72.67	72.95 (1.00×)	58.68 (1.24×)	53.38 (1.36×)	33.08 (2.20×)	54.5%
Qwen2.5-VL-32B	89.11	89.25 (1.00×)	71.18 (1.25×)	65.24 (1.37×)	39.84 (2.24×)	55.3%
DeepSeek-Coder-33B	91.86	92.18 (1.00×)	74.99 (1.23×)	67.60 (1.36×)	41.59 (2.21×)	54.7%

QStore saves 52.4–55.3% of the uncompressed pair. LZ4 slightly expands these high-entropy tensors, and the SafeTensors baseline is uncompressed, so its sizes match the Raw column.

The storage results remain the same as the paper for five models. The current Qwen2.5-VL Hub revision exposes about 1.55% more matched weights, increasing every method's absolute size by the same proportion; QStore's compression ratio is unchanged.

Loading

The updated loader overlaps reconstruction with one ordered, eager reader. QStore loads the matched BF16+INT8 model pair faster than every comparison format on all six models.

Model pair	QStore load	vs. Raw	vs. Zstd-2	vs. ZipNN + Zstd-2
Qwen2-Audio-7B	4.986 s	2.35×	1.57×	1.43×
Mistral-7B-v0.3	4.893 s	2.33×	1.81×	1.70×
Llama-3.1-8B	5.083 s	2.27×	1.89×	1.83×
Gemma-3-27B	21.996 s	1.99×	1.65×	1.48×
Qwen2.5-VL-32B	27.511 s	1.96×	1.56×	1.46×
DeepSeek-Coder-33B	27.076 s	2.03×	1.75×	1.58×

These are medians of three cold-cache runs on matched 2-D weights. Every input had zero resident bytes before launch, and every decoded tensor passed bitwise verification.

Compared with the paper's implementation, the updated QStore decoder itself is 2.04–2.42× faster and the encoder is 2.20–2.54× faster. The headline loading advantage also increases from 2.2× to 2.35× over uncompressed storage and from 1.8× to 1.83× over compressed alternatives. The algorithm and archive representation are unchanged.

Full timings, storage sizes, model revisions, hardware, and reproduction instructions are in baselines/README.md. The engineering changes are separated from the unchanged algorithm in IMPLEMENTATION.md.

Build

QStore requires a C++20 compiler, CMake 3.20+, OpenMP, and fincore from util-linux. Dependencies are installed through vcpkg.

git clone https://github.com/illinoisdata/qstore.git
cd qstore
git clone https://github.com/microsoft/vcpkg.git
./vcpkg/bootstrap-vcpkg.sh
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j
ctest --test-dir build --output-on-failure

The bundled Hugging Face preparation script additionally needs a CUDA-enabled PyTorch environment with Transformers, Accelerate, bitsandbytes, and SafeTensors. download_models.ipynb provides the original interactive workflow.

python -m venv .baseline-venv
.baseline-venv/bin/pip install torch transformers accelerate bitsandbytes safetensors
.baseline-venv/bin/python baselines/prepare_paper_model.py \
  meta-llama/Llama-3.1-8B-Instruct /path/to/models

Llama and Gemma require Hugging Face access approval and hf auth login.

Run QStore

Prepared model directories follow this layout:

/path/to/models/<organization>/<model>-bf16/
/path/to/models/<organization>/<model>-bf16-int8/

Run encoding, paired BF16+INT8 decoding, and INT8-only decoding with:

./qstore.sh meta-llama/Llama-3.1-8B-Instruct /path/to/models

qstore.sh is a functional end-to-end check. For performance measurements, use the exact-file cold-cache procedure in baselines/README.md; timed loaders do not use mmap or lazy tensor loading.

Useful controls:

• QSTORE_NUM_THREADS: OpenMP worker count; defaults to 48.
• QSTORE_PIPELINE_JOBS: maximum in-flight tensor jobs.
• QSTORE_STREAMING_TAIL: number of final tensors decoded in streaming chunks.
• QSTORE_READ_BANDWIDTH_MIB: optional read-bandwidth limit for controlled experiments.

Repository guide

• src/, include/: QStore encode/decode and shared I/O pipeline.
• baselines/: matched raw, LZ4, Zstd, SafeTensors, and ZipNN runners.
• tests/: focused codec, parser, and short-read tests.
• model_chain/: prototype experiments for BF16/INT8/INT4 chains; see model_chain/README.md.

Citation

If you use QStore, please cite the paper:

@article{shah2025qstore,
  title   = {QStore: Quantization-Aware Compressed Model Storage},
  author  = {Shah, Raunak and Li, Zhaoheng and Park, Yongjoo},
  journal = {Proceedings of the VLDB Endowment},
  volume  = {19},
  number  = {3},
  pages   = {388--398},
  year    = {2025},
  doi     = {10.14778/3778092.3778100},
  url     = {https://dl.acm.org/doi/10.14778/3778092.3778100}
}