Evaluate a Proposal to Add a Replay Buffer to A2C

An engineer proposes speeding up training of an A2C agent (for a recommendation-slot allocation task) by storing the last 100,000 (state, action, reward, next-state, log-probability-at-collection- time) tuples in a buffer and reusing them for multiple gradient steps, exactly as DQN reuses transitions via experience replay, arguing "it's the same idea, just for the policy instead of the value function."

1. Explain specifically why this naive reuse is not valid for A2C's policy gradient the way it is valid for Q-learning's value update.
2. The engineer already logs the action's probability under the policy that collected it. Explain how importance sampling could, in principle, allow limited reuse, and write the corrected gradient term.
3. What practical failure mode should the team watch for even with the importance-sampling correction in place, and why does this motivate a mechanism like PPO's clipped objective rather than unlimited reuse?