Back to problems

Tiny Next-Token Prediction Model (PyTorch Fill-In)

Algorithm · Microsoft · Medium

Requirements Complete the PyTorch starter code so it trains a feed-forward language model—without recurrence or attention—that predicts the next token using only the token currently being processed. Implement the following pieces: DataLoader — place the supplied SyntheticDataset(num_samples, seq_len, vocab_size) inside DataLoader(dataset, batch_size=32, shuffle=True). Model — connect Embedding(vocab_size, hidden_dim) to Linear(hidden_dim, hidden_dim), then to ReLU, and…

Checking your access…