Online Softmax Entropy

Streaming Softmax Entropy Problem Overview You receive a one-dimensional NumPy logits array: Define: The quantity sometimes written in the prompt is p * log(p). Under the usual machine-learning convention, entropy is: In this expression, sum_i aggregates across all positions i, and p_i * log(p_i) is an ordinary product. Consequently, sum_i (p_i * log(p_i)) is negative entropy. Clarify which sign is wanted during the interview. By default, the routines below produce standard…