AI Coding · Perplexity · Medium
Byte Tokenizer (Online Assessment) CodingOASoftware Engineer Overview This multi-part online assessment asks you to build and improve a byte-oriented tokenizer based on the ideas of Byte Pair Encoding (BPE), a technique commonly used for LLM tokenization. It evaluates your understanding of algorithms, runtime optimization, and statistical estimation. The assessment includes these parts: Part 0: Examine the reference implementation, slow_tokenize, and describe its behavior in…
Checking your access…