Practice — Convolutional Networks & Vision Basics (6 questions)

You are reviewing a teammate's PyTorch model for 3x224x224 images and 100 classes. They claim "the stem has a receptive field of about 20 pixels, so we need dilation to see more context". The layers are:

1. Compute the output shape after every layer, using the output-size formula.
2. Count the parameters per layer and in total (count BatchNorm's trainable scale and shift).
3. Compute the theoretical receptive field after each of A-E with the recurrence, and say whether the teammate's claim is right. What would the RF after E be if D used dilation 1 and padding 1?
4. How many multiply-accumulates does layer D cost? Compare the parameter count of layer A with the size of its output (activations); what does that imply for training memory?

A junior engineer proposes replacing the ResNet-50 stage-2 blocks (256-channel input and output) with a plain stack of two 3x3 convs at 256 channels "because the bottleneck's 1x1 convs look like wasted depth". Another says "skip connections are just there for the gradients". You need to settle both.

1. Write out the bottleneck block (256 to 64 to 64 to 256) and compute its weight count (ignore BatchNorm and biases). Compare with two 3x3 convs at 256 channels. How many times smaller is it?
2. In a stage transition the input has 256 channels at 56x56 and the output must have 512 channels at 28x28. The identity shortcut cannot be used. What replaces it, how many weights does it have, and why is it still compatible with the residual idea?
3. Explain what the skip connection does beyond helping gradients, referring to the degradation problem, and say what initializing the last BatchNorm scale of each residual branch to zero achieves.
4. When would you still prefer the basic (two 3x3) block?