Match a job Paths Subjects Questions Quizzes Pricing
Intermediate Open Pro

Diagnose a Network Whose Hidden Activations Are All Zero

You train a 10-class MLP (balanced classes, ReLU hidden layers). After raising the learning rate from 10^{-3} to 1.0 for a "fast experiment," the loss jumps up on the first few steps, then flatlines at 2.303 and never moves. You print the activations of hidden layer 1 for a validation batch and every value is exactly 0.

  1. What is 2.303 and why does the loss flatline there?
  2. Explain the mechanism: why does the network not recover after the large step?
  3. How would you confirm the diagnosis in code, and what would you change? Give at least three fixes.

Share this question

← Back to Neural Networks & Backpropagation practice

We use cookies for product analytics to improve OmniAtlas. See our Privacy Policy.