Skip to content
English

Switch theme

Discover▾

Not sure where to start? This week’s hottest software, models and stacks in three minutes.

AI Apps▾

AI software people use: from assistants like ChatGPT to tools for coding, video and workflows, picked by purpose.

For Agents▾

Software for AI: tools that let your agent (Claude Code, Codex, Cursor…) browse, read docs, run code and remember.

Models▾

Choose models, estimate cost, self-host: prices, rankings, benchmarks, free tiers and compute.

Solutions▾

A goal or an error message? Ready-made stacks, steps and fixes by scenario.

Learn▾

A systematic course from zero to the frontier, with an algorithm library, must-read papers and tutorials.

Frontier▾

Daily AI news, weekly highlights, the latest papers and how AI is changing every discipline.

// Stage 5 · Post-training, Alignment & Reasoning Models

Instruction Tuning & PEFT

SFT, synthetic data, LoRA/QLoRA/DoRA, adapters and model merging.

Read the lesson → Code & exercises ↓

about 10 hours8 algorithms3 paperslesson text ready

01Goals

Do these first: Pretraining, Data & Scaling Laws

02Read on this site

03Algorithms to master

Each card has the original paper, prerequisites and resources.

Instruction Tuning (FLAN) Fine-tuning on many instruction–response tasks for zero-shot generalization. 2021core Self-Instruct The model generates its own instruction data for fine-tuning. 2022core Adapters Inserts small trainable modules between frozen layers. 2019core Prefix & Prompt Tuning Learns only continuous soft-prompt vectors prepended to inputs. 2021ext LoRA (Low-Rank Adaptation) Freezes the model and learns weight updates as small low-rank matrices. 2021core QLoRA Fine-tunes LoRA on a 4-bit quantized base, fitting big models on one GPU. 2023core DoRA (Weight-Decomposed LoRA) Splits weights into magnitude and direction, using LoRA for direction. 2024ext Model Merging & Task Arithmetic Combines fine-tuned models by adding, subtracting or averaging weights. 2022core

04Must-read papers

05Courses & videos

06Hands-on: project, code & exercises

Free, official or author-published pages. Most run in the browser or on a free Colab GPU.

PROJECT

Use QLoRA on a single GPU to fine-tune an open 7B/8B model into a specific style (e.g. customer-service replies) and run a blind comparison.

  1. 1
    unslothai/unsloth ↗Reference code

    LoRA/QLoRA-fine-tune a small model with the official free Colab notebooks.

  2. 2
    Hugging Face TRL：SFT Trainer ↗Official docs

    Run instruction tuning with TRL’s SFTTrainer.

  3. 3
    github.com ↗Reference code

    Fine-tune with the no-code web UI and export merged weights.

07Self-test

Answer in your head, then open to check.

Which parameters does LoRA train?

Low-rank matrices A and B; the original weights stay frozen.

Which 4-bit format does QLoRA use?

NF4.

When should you use RAG instead of fine-tuning?

When knowledge changes often or answers must cite sources.

On these routes: Application Developers · Algorithm Researchers · Domain Scientists

Data updated: AI algorithms & curriculum(UTC+8)