MODEL ADAPTATION · FINE-TUNING
Fine-Tuning Large Language Models: SFT, LoRA and QLoRA
Choose an adaptation method, prepare instruction data and determine whether the resulting model improves on a reproducible baseline.
Fine-tuning adapts a pretrained large language model to specific tasks or instruction-following behaviour. Supervised fine-tuning (SFT) trains on input-output pairs. LoRA freezes the base model and trains low-rank adapter matrices, reducing trainable parameters by 99%+. QLoRA combines LoRA with 4-bit quantization of the base model, enabling fine-tuning of large models on single GPUs. Evaluation must compare the fine-tuned model against the original baseline on the target task.
Fine-tuning methods at a glance
When Should You Fine-Tune a Large Language Model?
Fine-tuning is not the first tool to reach for. Before fine-tuning, consider whether prompt engineering, retrieval-augmented generation (RAG) or few-shot examples can achieve the target behaviour. Fine-tuning is appropriate when the model must consistently follow a specific format, adopt a domain-specific style, or internalise knowledge that cannot be effectively provided through context.
Fine-tuning is justified when: (1) prompt engineering has been exhausted and still produces inconsistent results, (2) the target behaviour requires the model to generate responses in a consistent format or style across thousands of requests, (3) latency constraints make long in-context examples impractical, or (4) domain-specific vocabulary or reasoning patterns are not well-represented in the base model's training data.
Fine-tuning is not justified when: the problem is a retrieval problem (use RAG), the problem is a routing problem (use an agent), or the model already performs well with good prompts and the goal is marginal improvement. Fine-tuning adds engineering overhead—data preparation, training, evaluation, versioning—that must be justified by a clear gap.
How Should Fine-Tuning Data Be Prepared?
Fine-tuning data quality is the single most important factor in the outcome. The model learns from the data you provide—if the data contains errors, inconsistencies or biases, the fine-tuned model will reflect them. Data preparation involves collection, cleaning, formatting and quality review.
For supervised fine-tuning, data is formatted as input-output pairs: an instruction or prompt and the expected response. The pairs should cover the range of inputs the model will encounter in production. Diversity matters—a few hundred high-quality, diverse examples are more valuable than thousands of near-duplicate examples.
Data must be split into training, validation and held-out test sets. The test set must not be used during training or hyperparameter selection—it exists to provide an unbiased estimate of the model's performance on unseen data. Without a held-out test set, you cannot distinguish learning from memorisation.
Fine-tuning data preparation steps
Full Fine-Tuning vs Parameter-Efficient Fine-Tuning
Operational comparison of full fine-tuning and parameter-efficient methods.
| Dimension | Full Fine-Tuning | LoRA / QLoRA | |
|---|---|---|---|
| Trainable parameters | All model parameters | 0.1–1% (adapter matrices only) | — |
| GPU memory | Model size × optimizer states (high) | LoRA: moderate; QLoRA: low (4-bit base) | — |
| Training speed | Slower—full gradient computation | Faster—fewer gradients to compute and store | — |
| Storage per adaptation | Full model checkpoint (GBs) | Adapter weights only (MBs) | — |
| Multiple adaptations | One checkpoint per adaptation | Multiple adapters can share one base model | — |
| Risk of catastrophic forgetting | Higher—all weights can shift | Lower—base model is frozen | — |
| Maximum achievable quality | Highest (theoretically) | Slightly lower for some tasks, but close for most | — |
How Does LoRA Adapt a Large Language Model?
LoRA (Low-Rank Adaptation) freezes the pretrained model's weights and injects trainable low-rank matrices into each attention layer. Instead of updating the full weight matrix W, LoRA adds a product of two small matrices A and B such that the effective update is W + B×A. The rank (size of A and B) is a hyperparameter—typical values are 8, 16, 32 or 64.
The key insight is that the weight updates during fine-tuning often have low intrinsic rank—meaning the meaningful changes can be captured in a much lower-dimensional space than the full weight matrix. By training only A and B, LoRA reduces the number of trainable parameters by 99% or more while preserving most of the adaptation quality.
LoRA adapters are small (typically 10–100 MB vs multiple GB for the full model), which means you can store and switch between multiple adaptations cheaply. You can serve one base model with different LoRA adapters for different tasks, swapping adapters at inference time.
How Does QLoRA Reduce GPU Memory?
QLoRA combines LoRA with 4-bit quantization of the base model. The pretrained weights are quantized to 4-bit precision (using the NF4 quantization scheme), dramatically reducing the memory needed to load the model. The LoRA adapters are trained in full precision (bf16 or fp16) on top of this quantized base.
The practical impact is significant: a model that requires 40+ GB of GPU memory in fp16 can be loaded in approximately 10–12 GB with 4-bit quantization, making it possible to fine-tune 7B–13B parameter models on a single consumer GPU. The quality cost of 4-bit quantization is minimal for fine-tuning purposes because the adapters are trained in higher precision.
QLoRA introduces a trade-off: training is slightly slower than standard LoRA (due to the de-quantization step in the forward pass) but the memory savings enable fine-tuning on hardware that would otherwise be insufficient. For most practitioners, this trade-off is strongly favourable.
How Should a Fine-Tuned Model Be Compared With Its Baseline?
Engineering decisions for evaluating a fine-tuned model against its baseline.
| Decision | Options | Trade-off | Recommendation |
|---|---|---|---|
| Evaluation metric | Task-specific (accuracy, F1, BLEU, ROUGE) vs LLM-as-judge vs human review | Reproducibility vs nuance vs cost | Combine: automated metrics for regression, human review for quality |
| Test set | Same test set as baseline vs new test set vs both | Comparability vs coverage | Use the same held-out test set for direct comparison |
| Regression testing | Test target task only vs test target + general capabilities | Speed vs detecting catastrophic forgetting | Always test general capabilities to detect regressions |
| Statistical significance | Single run vs multiple seeds vs bootstrap confidence intervals | Compute vs confidence in the result | Multiple seeds for small datasets; bootstrap for larger ones |
| Safety evaluation | Skip vs manual check vs automated safety suite | Speed vs risk | Automated safety suite is mandatory before any release decision |
Common fine-tuning failure modes
What goes wrong during fine-tuning and how to detect it.
| Failure | Signal | Cause | Containment | Recovery |
|---|---|---|---|---|
| Catastrophic forgetting | Model loses general capabilities it had before fine-tuning | Full fine-tuning shifts all weights; insufficient data diversity | Use LoRA/QLoRA; include general data in training mix | Reduce learning rate; revert to base and retrain with regularisation |
| Overfitting | Validation loss diverges from training loss; test performance drops | Too many epochs; too few examples; model memorises training data | Early stopping; monitor validation loss; more data | Reduce epochs; add data; use regularisation |
| Format collapse | Model outputs correct content but wrong format | Inconsistent formatting in training data; format token mismatch | Standardise training data format; verify chat template | Re-prep data with consistent format; retrain |
| No improvement over baseline | Fine-tuned model scores same or worse than base model on test set | Insufficient data quality; wrong hyperparameters; task not learnable from available data | Establish baseline first; inspect data quality before training | Improve data; try prompt engineering instead; reconsider whether fine-tuning is the right approach |
Key takeaways
- Fine-tune only after prompt engineering and RAG have been evaluated and found insufficient.
- Data quality and diversity matter more than data quantity—a few hundred high-quality examples can outperform thousands of low-quality ones.
- LoRA reduces trainable parameters by 99%+ while preserving most adaptation quality; adapters are small and swappable.
- QLoRA enables fine-tuning of large models on single GPUs by combining 4-bit quantization with LoRA.
- Always establish a baseline and compare the fine-tuned model against it on a held-out test set.
- Test for catastrophic forgetting—fine-tuning can break general capabilities the base model had.
LoRA and QLoRA methods are established in their respective papers (Hu et al., 2021; Dettmers et al., 2023). Implementation details are verified against Hugging Face PEFT and TRL documentation.
- Specific hyperparameter recommendations (learning rate, rank, batch size) depend on the model and dataset.
- New parameter-efficient methods (DoRA, GaLore, etc.) are not covered here but follow similar patterns.
Review cadence: Reviewed every 90 days. Next review by December 2026.
Selecting LoRA or QLoRA is the easy decision. The engineering work is establishing a baseline and proving that adaptation improved the intended behaviour without introducing unacceptable regressions.
In the Large Language Model Course, you move through the same engineering lifecycle with an open-source model: establish a baseline, prepare data, fine-tune with LoRA or QLoRA, evaluate failures, document limitations and make a release, revise or reject decision.
Advanced eight-week live course · Approximately 4–5 hours per week · Python, statistics, machine learning and neural-network foundations expected.
- Tier 1
- Tier 1
- Tier 1
- Tier 1