Skip to content
English

Switch theme

Discover▾

Not sure where to start? This week’s hottest software, models and stacks in three minutes.

AI Apps▾

AI software people use: from assistants like ChatGPT to tools for coding, video and workflows, picked by purpose.

For Agents▾

Software for AI: tools that let your agent (Claude Code, Codex, Cursor…) browse, read docs, run code and remember.

Models▾

Choose models, estimate cost, self-host: prices, rankings, benchmarks, free tiers and compute.

Solutions▾

A goal or an error message? Ready-made stacks, steps and fixes by scenario.

Learn▾

A systematic course from zero to the frontier, with an algorithm library, must-read papers and tutorials.

Frontier▾

Daily AI news, weekly highlights, the latest papers and how AI is changing every discipline.

// Stage 5 · Post-training, Alignment & Reasoning Models

Reasoning Models: RLVR, GRPO & Test-Time Compute

Chain of thought, self-consistency, process rewards, GRPO/DAPO, verifiable rewards and the R1 recipe.

What you will learn ↓ Code & exercises ↓

about 12 hours12 algorithms5 paperslesson text in progress

01Goals

Do these first: Alignment: RLHF, DPO & Constitutional AI

02Algorithms to master

Each card has the original paper, prerequisites and resources.

Chain-of-Thought Prompting Prompting the model to write intermediate reasoning before answering. 2022core Self-Consistency Samples many reasoning paths and takes the majority answer. 2022core Tree of Thoughts Expands reasoning as a tree searched with self-evaluation and backtracking. 2023core Process Reward Models (PRM) Scores each reasoning step rather than only the final answer. 2023core STaR (Self-Taught Reasoner) Bootstraps reasoning by fine-tuning on the model's own correct rationales. 2022core Group Relative Policy Optimization (GRPO) Samples a group of answers per prompt and uses in-group relative scores instead of a critic. 2024core RL with Verifiable Rewards (RLVR) Rewards come from programmatic checks like math answers or unit tests. 2024core DAPO A set of GRPO fixes and an open system for long-CoT RL. 2025core DeepSeek-R1 Reasoning RL Recipe Large-scale outcome-reward RL alone teaches long reasoning and self-reflection. 2025core Test-Time Compute Scaling Spending more inference compute — thinking, sampling, search — for better answers. 2024core Budget Forcing (s1) Controls test-time compute by forcing thinking to extend or stop. 2025ext Self-Refine One model drafts, critiques its own output, then revises – repeated a few times. 2023core

03Must-read papers

04Courses & videos

05Hands-on: project, code & exercises

Free, official or author-published pages. Most run in the browser or on a free Colab GPU.

PROJECT

Train a 0.5B–1.5B model with GRPO on GSM8K (rule-based rewards) and watch answer length and accuracy change over steps.

  1. 1
    github.com ↗Reference code

    Reproduce R1-Zero-style “aha” reasoning on an arithmetic game with RL (needs a rented GPU).

  2. 2
    TRL：GRPO Trainer ↗Official docs

    Train a small maths model with TRL’s GRPOTrainer and a verifiable reward.

  3. 3
    huggingface/open-r1 ↗Reference code

    Follow Open-R1’s reproduction steps to see what distillation and RL each contribute.

06Self-test

Answer in your head, then open to check.

Where does the reward in RLVR come from?

Programmatically checkable outcomes, such as matching answers or unit tests.

Which PPO component does GRPO drop?

The value network (critic).

Is self-consistency voting a training or an inference method?

Inference (test-time compute).

On these routes: Algorithm Researchers · Product Managers & Leaders

Data updated: AI algorithms & curriculum(UTC+8)