MODEL SELECTION · OPEN WEIGHTS
Open-Source Large Language Models: How to Compare and Choose
Select a model using licence, workload, hardware, adaptation and evaluation evidence—not a generic leaderboard position.
Open-source large language models have publicly available model weights that you can download, inspect and run locally. Open-weight models may have restrictions on commercial use, derivative works or deployment scale. Selection should be based on licence terms, hardware requirements (GPU memory, context length), adaptation support (fine-tuning, LoRA compatibility), evaluation evidence on your specific workload and operational constraints—not general leaderboard rankings.
Selection criteria at a glance
Open-Source vs Open-Weight Large Language Models
The terms "open-source" and "open-weight" are often used interchangeably, but they have different implications. A truly open-source model has publicly available model weights, training code, training data (or data preparation methodology) and architecture details. An open-weight model releases the trained weights but may not release the training data or full training pipeline.
For most practitioners, the distinction that matters most is the licence. Some models released as "open" have licences that restrict commercial use above a certain revenue threshold, restrict deployment scale, or require attribution in specific ways. Reading and understanding the licence is a prerequisite for any production deployment decision.
The model card—the official documentation accompanying the model—is the primary source for licence terms, intended use, known limitations and evaluation results. Always read the model card on the official repository (Hugging Face, GitHub, or the organisation's website) before making a selection decision.
Which Model Licences Allow Commercial Use?
Licence compatibility is the first gate in model selection. If the licence does not permit your use case, no amount of technical capability matters. Common licence types for open-weight models include:
Apache 2.0 and MIT are permissive licences that allow commercial use, modification and redistribution with minimal restrictions. Models under these licences are the safest choice for commercial deployment. Llama-style licences (Llama 2 Community Licence, Llama 3 Community Licence) allow commercial use with some restrictions—typically a user cap (e.g. 700 million monthly users) and acceptable use policy constraints. These are widely used but require legal review for large-scale deployment.
Some models use custom licences that restrict commercial use entirely, require a separate commercial agreement, or impose usage restrictions (no military use, no certain industries). These models may be suitable for research and development but not for production deployment without a separate agreement.
How Should Large Language Model Families Be Compared?
Comparison dimensions for evaluating model families—populate with current model card data at selection time.
| Dimension | Consideration | Source | |
|---|---|---|---|
| Parameter count | 7B / 13B / 30B / 70B+ | Determines VRAM, latency and quality ceiling | — |
| Architecture | Decoder-only / MoE / hybrid | Affects serving complexity and memory patterns | — |
| Context length | 4k / 8k / 32k / 128k | Determines input capacity; longer = more VRAM for KV cache | — |
| Licence | Apache 2.0 / MIT / custom | Determines commercial use rights | — |
| Fine-tuning support | LoRA / QLoRA / full | Check PEFT compatibility and community adapters | — |
| Quantization | GGUF / AWQ / GPTQ / FP8 | Affects deployment memory and speed | — |
| Serving engine | vLLM / TGI / SGLang / llama.cpp | Not all engines support all models | — |
What Hardware Does Each Model Require?
Hardware requirements are determined by parameter count, quantization format and context length. A 7B parameter model in fp16 requires approximately 14 GB of VRAM for weights alone. With 4-bit quantization, the same model requires approximately 4–5 GB. KV cache for context length adds memory proportional to context length × layers × heads × head dimension × batch size.
Fine-tuning memory is higher than inference memory because optimizer states and gradients must be stored. Full fine-tuning of a 7B model requires approximately 100+ GB of VRAM. LoRA reduces this to approximately 16–20 GB. QLoRA with 4-bit base further reduces this to approximately 8–10 GB, enabling fine-tuning on a single 24 GB GPU.
The practical approach is to start from your hardware budget and work backward. If you have a single 24 GB GPU, your options are 7B models (full fp16 inference, QLoRA fine-tuning) or quantized 13B models. If you have multiple 80 GB GPUs (A100/H100), you can run 70B models with quantization or 30B models in full precision.
Should You Fine-Tune, Use Retrieval or Choose an API?
Decision framework for choosing between fine-tuning an open-source model, using RAG, or using an API provider.
| Decision | Options | Trade-off | Recommendation |
|---|---|---|---|
| Data sensitivity | Data can leave your infrastructure vs cannot leave | API convenience vs self-hosted control | Self-host if data cannot leave; API if it can |
| Volume | Low volume (prototyping) vs high volume (production) | API cost scales linearly; self-hosted has fixed cost | API for low volume; self-hosted for high volume |
| Customisation needed | Prompt engineering sufficient vs behaviour change needed | RAG + prompts vs fine-tuning investment | Try RAG + prompts first; fine-tune only if insufficient |
| Latency requirements | Tolerant (seconds) vs strict (sub-second) | API network latency vs local inference | Self-hosted for strict latency; API for tolerant |
| Team expertise | ML engineering team vs application team only | Self-hosted requires ML ops expertise | API if no ML ops team; self-hosted if team exists |
How Should a Model Be Evaluated Before Selection?
Key takeaways
- Read the licence before evaluating the model—licence incompatibility is a hard blocker.
- Open-weight and open-source are not the same; check what is actually released.
- Hardware requirements are determined by parameter count, quantization and context length.
- Fine-tune only after RAG and prompt engineering have been evaluated and found insufficient.
- Generic leaderboards do not predict performance on your workload—evaluate on your own test set.
- Document the selection decision with evidence so it can be reviewed and revised.
Model-specific facts (parameter counts, licences, context lengths) change frequently and are verified against official model cards and repositories at the time of review. This guide does not list specific model names because the landscape changes faster than the review cadence.
- Model capabilities and licences change—always verify against the official model card before deployment.
- Hardware requirements are approximate; actual VRAM depends on serving engine, batch size and context length.
- New models are released weekly; this guide provides the selection framework, not a current model list.
Review cadence: Reviewed every 60 days due to rapid model landscape changes. Next review by November 2026.
A leaderboard cannot select a model for your workload. The decision must survive licensing, hardware, adaptation, quality and operational constraints.
In the Large Language Model Course, you move through the same engineering lifecycle with an open-source model: establish a baseline, prepare data, fine-tune with LoRA or QLoRA, evaluate failures, document limitations and make a release, revise or reject decision.
Advanced eight-week live course · Approximately 4–5 hours per week · Python, statistics, machine learning and neural-network foundations expected.
- Tier 1
- Tier 1
- Tier 1