Estimating the Mean and Variance by Maximum Likelihood

Assume the observations $x_1, \ldots, x_n$ are independent samples from a normal distribution whose mean $\mu$ and variance $\sigma^2$ are both unknown. Find the maximum-likelihood estimators of these two parameters. Then explain why the variance estimator divides by $n$, how that differs from the usual unbiased sample variance, and what happens when the model assumptions do not hold well.

Constraints and Assumptions

• $n$ is positive and all observations are finite.
• The likelihood is normal with independent and identically distributed observations.
• Address the edge case where every observation takes the same value.
• Separate the concept of maximum likelihood from unbiasedness.

Clarifying Questions to Ask

• Are both parameters unknown and being estimated from the same data?
• Is the variance being asked for the maximum-likelihood estimate or the unbiased estimate?
• Should the derivation maximize the likelihood with respect to the variance or the standard deviation?

Hint: Work with the log-likelihood. After taking logs, split the terms that involve the mean from those that involve the variance, then differentiate with respect to each parameter.

What a Strong Answer Covers

• The normal log-likelihood, ignoring additive constants
• The first-order condition that gives the sample mean
• The maximum-likelihood variance with $n$ in the denominator
• How this differs from Bessel's correction and the finite-sample bias
• Edge-case behavior, assumptions, and robustness issues

Follow-up Questions

1. Why does using $n-1$ instead of $n$ remove the bias under the normal iid model?
2. What estimator would you choose if large outliers make the normal likelihood unreasonable?
3. How does the variance derivation change when the mean is known?

Overview: Derive the maximum-likelihood estimators of the mean and variance for a normal distribution, and understand why the MLE variance is not the same as the unbiased sample variance.