Case study — LLM inference stability

Requirements

Case study: LLM inference stability — An LLM is used for sentiment classification on a batch of inputs with temperature > 0 (so the outputs are non-deterministic). Day 1 yields a label vector y1; Day 2 yields y2. The observed correlation corr(y1, y2) = 0.95. To raise the cross-day correlation to >= 0.99, how many independent days of inference do you need to aggregate? The reported answer is about 16 days. (The interviewer confirmed $15.93 \approx 16$ as correct.)