Why Transformers Won

To complete “The code is blue … repeat the code:”, the model needs blue from word 4. Play all 3 rounds below:

Round 1 · Rolling Memory Carry the clue “blue” across 5 words to reach “code:”.
1 The
2 code
3 is
4 blue 100%
5 after
6 lunch
7 repeat
8 the
9 code:
“blue” signal at end: 100% Predict: ?

Next: see how real models split text into numbered tokens in Tokenization →.