Explain learning-rate fluctuation and vanishing gradients

ML Fundamentals

Respond to the conceptual questions below:

1. Learning rate vs. training stability: Why might loss or accuracy values oscillate or swing up and down during training when the learning rate is set too high? What occurs when it is set too low?
2. Vanishing gradients in fully connected networks: In a deep fully connected network trained through backpropagation, are vanishing gradients more likely to affect layers near the input or near the output? Explain why, and list common mitigation techniques. Overview: This question checks a candidate's grasp of optimization dynamics—specifically how learning rate affects training stability—and backpropagation-related issues such as vanishing gradients in deep neural networks. It assesses abilities in training behavior and gradient propagation within the Machine Learning domain.