// ⚙ Algorithms Atlas
Fine-tuning & PEFT
Adapting general models cheaply: LoRA, QLoRA, instruction tuning and model merging.
6 core2 extended2019–2024
TIMELINE
01Core algorithms
Learn these first. Sorted by year.
Adapters Inserts small trainable modules between frozen layers. 2019core Instruction Tuning (FLAN) Fine-tuning on many instruction–response tasks for zero-shot generalization. 2021core LoRA (Low-Rank Adaptation) Freezes the model and learns weight updates as small low-rank matrices. 2021core Self-Instruct The model generates its own instruction data for fine-tuning. 2022core Model Merging & Task Arithmetic Combines fine-tuned models by adding, subtracting or averaging weights. 2022core QLoRA Fine-tunes LoRA on a 4-bit quantized base, fitting big models on one GPU. 2023core
02Extended
Variants and follow-ups worth knowing once the core is clear.