Algorithm · Citadel · Hard
Imagine you are given a very high-dimensional linear regression problem where the design matrix $$X$$ and response vector $$y$$ cannot be stored in memory at once. You want to estimate the ordinary least squares coefficient vector $$\beta$$ for a model that includes an intercept. Streaming sufficient statistics. Show how to compute $$X^\top X$$ and $$X^\top y$$ by processing the data in mini-batches, making sure the intercept term is handled correctly. Then describe how to…
Checking your access…