// ∞ Reasoning & Test-Time Compute

Process Reward Models (PRM)

Scores each reasoning step rather than only the final answer.

过程奖励模型 2023 Advanced core

IN PLAIN WORDS

OpenAI collected 800K step-level labels; a process-supervised model beat outcome supervision for best-of-N on MATH. Used for search, reranking and RL.

Used in: Math reasoning reranking and search

TIMELINE · prerequisites → this → what builds on it

2017 Reward Modeli…2022 Chain-of-Thou…2023 Process Reward Mo…2024 Test-Time Com…

01Original paper

Let's Verify Step by Step

Hunter Lightman, Vineet Kosaraju, Yura Burda, et al. · 2023

arXiv ↗ ✓ Verified · arXiv API

02Where it sits

Learn the left column first; the right column builds on this one.

Learn first

Process Reward Models (PRM)

Then you can learn

Test-Time Compute Scaling

Left: prerequisites · right: builds on this · dashed: related. Full map →

03Learning resources

Official docs, explainers, courses and reference code. All links checked.

Reference code

openai/prm800k ↗EN

Explainers

Lilian Weng：Why We Think ↗EN

04Where the course teaches it

U52 Reasoning Models: RLVR, GRPO & Test-Time Compute Chain of thought, self-consistency, process rewards, GRPO/DAPO, verifiable rewards and the R1 recipe. 12 h12 algorithms5 papers

On learning paths:Post-training & Reasoning Models

For AI / data

Verification: verified, checked 2026-10-21.
JSON: /api/learn/algorithms.json (id: prm)