Algorithm · Oracle · Hard
Write an implementation of multi_head_attention(Q, K, V, num_heads, mask) using NumPy or PyTorch tensor operations. Do not call nn.MultiheadAttention or F.scaled_dot_product_attention. Q, K, and V are 3-D tensors of shape (batch, seq, d_model). The integer num_heads must divide d_model without a remainder. The function returns a tensor with the same shape as the input, (batch, seq, d_model). The final argument, mask, is optional: it may be omitted or passed as None. When a…