LoRA

Train a small patch, not the whole model

Full fine-tuning updates every weight and produces a whole new copy of the model. LoRA freezes the original and trains a small add-on instead — a few million parameters against billions.

At inference the adapter combines with the base. Swap the adapter and the same base model does a different job, so you can keep several and load whichever a request needs.

This is what made fine-tuning ordinary rather than exotic: it fits on modest hardware and produces artifacts small enough to version like code. It is still fine-tuning, so everything concept 28 says about knowledge versus behavior applies unchanged.