Explain vanishing gradients and activations

Explain what the vanishing gradient problem is in deep neural networks.

Your answer should:

• Provide a high-level account of how backpropagation works and why gradients can vanish in deep networks.
• Show how the selected activation function (for example, sigmoid, tanh, or ReLU) affects gradient magnitude.
• Discuss common approaches, including activation choices, for mitigating vanishing gradients.

Overview: This question assesses knowledge of the vanishing gradient problem, high-level backpropagation dynamics, and the role of activation functions in gradient propagation, measuring ability in neural network optimization and architecture.