Machine Learning Intermediate Pro

Neural Networks & Backpropagation

From logistic regression to a trained MLP, with every gradient derived

30 min read 0 views

Build a neural network from the ground up: why stacked linear layers collapse and nonlinearity is the point, the neuron and layer with shapes and parameter counting, activation functions (sigmoid, tanh, ReLU, Leaky ReLU, GELU, SiLU) and their failure modes, loss and output-layer pairings with the log-sum-exp stability trick, computation graphs and reverse-mode autodiff, a full hand derivation of backprop for a 2-layer softmax-cross-entropy MLP, autograd mechanics, Xavier/He initialization, vanishing and exploding gradients, finite-difference gradient checking, and a fully numeric forward/backward/SGD worked example.

Part of the Data Science Foundations path → Part of the Deep Learning Foundations path →

Preview Subscribe to unlock

Practice questions (6)

Derive the Gradient of Softmax Cross-Entropy

Intermediate

View →
Count the Parameters of an MLP

Beginner · Free

View →
Why Zero Initialization Fails, and What to Do Instead

Intermediate

View →
Diagnose a Network Whose Hidden Activations Are All Zero

Intermediate

View →
Why Reverse-Mode Autodiff Wins for Training

Advanced

View →

See all 6 questions →

Quizzes (1)

Neural Networks & Backpropagation Quiz

0 attempts · 10 min · Pass 70% · Intermediate

Take quiz →