Match a job Paths Subjects Questions Quizzes Pricing
Advanced Open Pro

Compute the Clipped Surrogate Objective for a Batch of Samples

A PPO update is being computed with clip range \epsilon = 0.2. The batch contains four (ratio, advantage) pairs from the current epoch:

Sample r_t(\theta) A_t
1 1.45 +1.0
2 0.55 −1.0
3 1.45 −1.0
4 0.55 +1.0
  1. Compute L^{\text{CLIP}} for each sample, showing the unclipped term, the clipped term, and which one the min operator selects.
  2. For each sample, state whether this sample's gradient contribution pushes r_t(\theta) further from 1, pulls it back toward 1, or is flat (zero gradient) with respect to r_t, and explain why.
  3. A teammate claims "clipping means PPO ignores samples 1 and 3 because their ratio is outside [0.8, 1.2]." Correct this misunderstanding using your answer to part 2.

Share this question

← Back to PPO & Modern Policy Optimization practice

We use cookies for product analytics to improve OmniAtlas. See our Privacy Policy.