Pre-training, Data, Fine-tuning & Alignment
Alignment Methods: RLHF, DPO and Constitutional AI
Reward models, preference optimisation, AI feedback and their failure modes.
Advanced 20 min For: Researchers ✎ In progress
✎ This lesson is still being written. Below is the planned outline; it is not indexed by search engines yet.
What this lesson will cover
- Reward modelling
- PPO-based RLHF
- Direct preference optimisation
- RLAIF and constitutions
- Reward hacking and sycophancy