ML engineers preparing large language models for cheaper, faster deployment on vLLM.

LLM Compressor is a library providing quantization algorithms like GPTQ, AWQ, and SmoothQuant for weight, activation, and KV cache compression.

Used as a Python library integrated into model preparation pipelines before deployment on vLLM inference servers.

Developed by the vLLM project and Red Hat AI, actively maintained with releases through 2026.

Shrinks model memory footprint and speeds up inference by producing quantized checkpoints compatible with vLLM's serving engine.

Depends on Hugging Face Transformers models as input and is designed to produce checkpoints consumed by vLLM.

Open source under Apache-2.0, maintained by the vLLM project and Red Hat AI; runs entirely locally with no external API calls required.