Library / SDK
Fine-tuning & Training
LMFlow
1Extensible 2toolkit 3for 4finetuning 5foundation 6models
the six Ws · specification
W1
Who
OptimalScale, a research group, develops LMFlow.
W2
What
An extensible open-source toolkit for continuous pretraining, instruction tuning, and alignment tuning of large foundation models.
W3
Where
Runs on user-managed GPU infrastructure, installed as a Python package from GitHub.
W4
When
Released in 2023 alongside its accompanying research paper, still maintained in 2026.
W5
Why
Lets teams with limited compute personalize and finetune large models using techniques like LoRA and the memory-efficient LISA method.
W6
With
Built on PyTorch and Hugging Face Transformers, and supports FlashAttention-1 and FlashAttention-2.
W7
Watch
Open-source (Apache-2.0 license) on GitHub, self-hosted training, no data leaves the user's own compute environment.