LoRA

← All terms · AI providers and models

Also called Low-Rank Adaptation

A highly efficient technique for fine-tuning AI models that updates only a tiny fraction of the model's parameters, significantly reducing computing costs and memory requirements.

What it is

Instead of updating billions of weights during fine-tuning, LoRA freezes the original model and injects small, trainable "adapter" modules. These adapters learn the new specific task. Because the adapters are tiny file sizes (often a few megabytes), you can train them quickly on consumer hardware and hot-swap them instantly during inference.

When you would use it

You use LoRA when you want to customize an open-weight model's behavior securely on your own hardware without the immense expense of a full fine-tuning run.

Common operations

• Fine-tuning a local Llama model on specialized code syntax using a single GPU.
• Swapping different LoRA adapters in and out of an inference engine (like vLLM) for different users dynamically.

Related terms

• Fine-tuning
• vLLM
• Open-weight

Dashed terms are named in the source but do not have a page here yet.

Where this is taught

No learning path uses this term yet. Browse the Learning Atlas for guided sequences through related ideas.