Researchers at University of Wisconsin-Madison, Microsoft Research, and Columbia University created LLaVA-NeXT.

An open multimodal model that combines a vision encoder with a Llama or Vicuna language model for image-grounded chat.

Weights and code are hosted on Hugging Face and GitHub under the LLaVA project.

The LLaVA-NeXT (1.6) update was released in January 2024, following the original LLaVA in 2023.

It demonstrated that visual instruction tuning could produce GPT-4V-level multimodal chat at low training cost.

Runs via the LLaVA/llava-next codebase on PyTorch and transformers, typically served through vLLM or Ollama.

Open weights released by the LLaVA research team; license follows the underlying Llama/Vicuna base model terms, permitting research and most commercial use with restrictions.