Choose regularization norms and model formulations

1. For both linear and logistic regression, write the objective functions with $L_0$, $L_1$, $L_2$, and $L_\infty$ penalties in both constrained and penalized forms (for example, minimize the loss subject to a norm constraint; and minimize the loss plus $\lambda$ times the norm). For each norm, state the expected coefficient patterns (sparsity, shrinkage, robustness to outliers) and the associated optimization difficulty.

2. Given features $X$ where two groups of predictors are highly collinear and a few outliers exist in $X$ and $y$, explain which penalty (or elastic net mixing) you would select and why.

3. Contrast linear regression and logistic regression by explicitly writing out the model formulas (link function, conditional distributions) and stating when linear regression is inappropriate for classification.

4. Discuss how feature scaling interacts with $L_1$ and $L_2$, and what happens if scaling is omitted.

Overview: This question assesses understanding of regularization norms ($L_0$, $L_1$, $L_2$, $L_\infty$), objective formulations for linear and logistic regression, and practical skills in dealing with collinearity, outliers, feature scaling, and optimization difficulty.