Explain Overfitting and Transformer Attention

You are in an interview for a machine learning engineer position. Give clear answers to the following machine learning fundamentals questions, and contrast different modeling setups.

1. What does overfitting mean? How can you identify it from training and validation metrics?
2. What steps would you take to reduce overfitting in a linear model?
3. What approaches would you use to limit overfitting in a deep neural network?
4. Describe how Transformer self-attention is structured. What roles do queries, keys, and values play, and how do you compute the attention weights?
5. Why is positional information necessary for a Transformer? Explain at least two methods for adding positional information.

Overview: This question tests your grasp of model generalization and regularization methods, along with Transformer self-attention and positional encoding, and assesses your ability to diagnose overfitting, choose suitable mitigation techniques, and interpret attention mechanisms.