Explain surprisal and its units

You are talking through a language-modeling / NLP project when the interviewer asks about surprisal.

1. Give the definition of surprisal for an event or token whose probability is $p$.
2. What units can surprisal be expressed in (just as distance can be expressed in meters), and what decides which unit applies?
3. In language modeling, how does surprisal connect to cross-entropy and perplexity?
4. What typical mistakes or misconceptions come up when people interpret surprisal values? Overview: This question tests familiarity with probabilistic information measures used in language modeling: in particular, the idea of surprisal, its units, and how it relates to evaluation metrics like cross-entropy and perplexity in the Machine Learning / NLP space.