Library / SDK
Inference & ServingFine-tuning & Training
bitsandbytes
1Low 2bit 3quantization 4kernels 5for 6LLMs
the six Ws · specification
W1
Who
Originally created by Tim Dettmers and now maintained by the bitsandbytes foundation.
W2
What
bitsandbytes provides low bit, 8-bit and 4-bit, quantization and optimizers that sharply cut GPU memory use for large models.
W3
Where
Hosted on GitHub at bitsandbytes-foundation/bitsandbytes and distributed via PyPI.
W4
When
Released in 2021 and actively maintained.
W5
Why
It lets teams fine tune and run large language models on consumer GPUs that would otherwise lack enough memory.
W6
With
Integrates with PyTorch, Transformers, PEFT and Accelerate.
W7
Watch
MIT licensed, open source, maintained by the bitsandbytes foundation spun off from the original Tim Dettmers project, roughly 8.4k GitHub stars.
works with