MLE Intern OA: ML Fundamentals

Requirements

• An MLE internship OA contains two coding tasks plus four multiple-choice questions, and you need to finish the multiple-choice part. All six questions together share a single 90-minute time limit.
• Topics to brush up on:
  • Bayes' theorem.
  • Straightforward forward-pass arithmetic on the signals or parameters of neurons in a neural network.
  • Fundamentals of KNN.
  • Fundamentals of the Transformer.

Notes

• The multiple-choice section tests foundational ML knowledge.
• The precise wording of the questions, their answer options, the numbers used, and any worked examples are not available. The topic list above sets the scope for preparation; it is not a reconstruction of the original four questions.

Core review

What follows is general study material for the topics listed; it does not describe the actual multiple-choice questions.

• Bayes' theorem: When $P(B) > 0$, apply $P(A \mid B) = \frac{P(B \mid A)\, P(A)}{P(B)}$. If the hypotheses are mutually exclusive and exhaustive, compute the denominator by adding up $P(B \mid H)\, P(H)$ across every hypothesis whose prior is positive. Do not confuse the direction of the likelihood with that of the posterior.
• Forward propagation and parameters: A dense layer taking $n$ inputs to $m$ outputs computes $z = W x + b$ and then applies the selected activation element by element. With one input stored as shape (n,), W is shaped (m, n), while b and z are both shaped (m,). If every output has a bias, the layer holds $m n + m$ trainable parameters; activations are values computed on the fly and do not count as extra trainable parameters. When layers do not share weights, count each one on its own.
• KNN: Pick a distance metric and take the $k$ training points closest to the query, where $1 \le k \le N$ for a training set of $N$ points. In classification, uniform voting picks whichever class appears most among the chosen neighbors, whereas distance weighting lets nearer neighbors count for more. Define how ties are broken both when selecting points at equal distance and when votes are level, and treat exact matches explicitly if the weights are inverse distances.
• Transformer attention: A single attention head computes $\mathrm{softmax}\!\left(\frac{Q K^\top}{\sqrt{d_k}}\right) V$, with $d_k$ the width of the queries and keys. For each query, softmax normalizes over the keys. Any padding or causal mask is applied to the scores before normalizing, and each query must have at least one key it is allowed to attend to; a row that is fully masked has to be handled explicitly. Queries decide what is being matched, keys produce the matching scores, and values provide the vectors that get averaged.

Preparation

• Bayes, 10 minutes: Write out the identity without looking, set up a small joint-probability table, and get the same posterior twice, once from the table and once from the formula. Check the denominator, and explain why, in this discrete setting, conditioning on an event of probability zero is undefined.
• Forward propagation, 15 minutes: Take a small dense network with two layers and biases. Label the shape of every tensor, work out a ReLU forward pass by hand, and confirm it with a short script. Tally weights and biases separately, then compare the sum against the parameter arrays actually stored.
• KNN, 10 minutes: Set up a small labeled set of points and one query. Sort the distances, contrast uniform voting with inverse-distance voting, and rerun the exercise with a tie in distance and with an exact match. Fix the rules for ties and zero distance before you check the predictions.
• Attention, 15 minutes: Sketch the shapes of Q, K, V, the score matrix and the output for a single head. Work out one small row of attention, confirm that the permitted weights add up to one, then apply a causal mask and check that later positions receive zero weight. Explain what to do with a query that has no permitted keys.