Explain Transformer Encoder and Decoder Behavior

In a machine learning interview, answer the following questions about Transformer fundamentals:

1. What are the key differences between a Transformer encoder and a Transformer decoder? Cover attention patterns and the purpose of a causal mask.
2. Why can an autoregressive decoder-based language model return different outputs for the same input prompt across runs? Address sampling, temperature, and the conditions under which generation becomes deterministic. Overview: This question assesses familiarity with Transformer architecture and generative language modeling—specifically the encoder/decoder roles, attention patterns and causal masking, as well as the mechanisms that make text generation stochastic, including sampling and temperature.