A Notification Agent That Learned to Spam

A push-notification agent uses a contextual bandit to decide, for each user each hour, whether to send a notification and which of several message templates to use. Reward is +1 if the notification is opened within 10 minutes, 0 if not opened, and no reward (no training signal) is defined for hours when no notification is sent. Two months in, average daily notifications per user has tripled and the open rate per notification has dropped, while unsubscribe rate has climbed steadily.

1. Walk through why a reward of "+1 if opened, 0 otherwise" pushes the policy toward sending more notifications even as each one becomes less individually valuable.
2. Why is "no reward for not sending" itself part of the problem, and what would change if not-sending carried an explicit reward?
3. Design a revised reward or constraint scheme, and explain how you'd verify (with what metric, over what horizon) that it actually fixes the behavior rather than just relabeling it.