PPO（近端策略优化）

A reinforcement learning algorithm commonly used in RLHF; clipping update steps keeps training stable.

PPO is the standard RLHF policy optimizer: it bounds update sizes to avoid collapse and works with a reward model plus KL penalty to steer behavior.