Debug a Transformer and Convert It to a Classifier

Problem Statement This exercise asks Machine Learning Engineers to diagnose a Transformer implementation. The code contains four defects, with comments marking the location of each one. Inspect those marked regions and correct every one of the four issues. The complete implementation is not included, but the available details are: Two of the defects occur in the self-attention component. A third issue concerns how positional embeddings are initialized. The remaining defect…