Compute Matrix Prefix Products And Gradients

You are handed a sequence of n dense square matrices W[0], W[1], ..., W[n-1], all of dimension $$d \times d$$. Define the running (inclusive) product sequence $$P[i] = W[0] @ W[1] @ \cdots @ W[i]$$ where @ denotes ordinary matrix multiplication, so that P[0] = W[0] and P[i] = P[i-1] @ W[i] whenever $$i \ge 1$$. Because matrix multiplication is associative yet not commutative, the left-to-right ordering of factors must be preserved at every stage: during the forward…