Five rounds on how attention works and what it is worth. In each round you first predict what will happen and lock your answer, then test it on the course data, and only then see the proof. Rounds 1 and 2 draw the real attention of FinBERT on finance tweets as a 3D landscape you can turn; rounds 3 to 5 use the 500 test tweets and the models of the lecture.
Charts stay empty until you lock a prediction. In rounds 1 and 2, drag the landscape to turn it; scroll or pinch to zoom.
Round 1 · Why divide by √d?
- 1 Predict
- 2 Test
- 3 Prove
Round 2 · Where does attention focus?
- 1 Predict
- 2 Test
- 3 Prove
Round 3 · The [SEP] sink
- 1 Predict
- 2 Test
- 3 Prove
Round 4 · Does word order matter?
- 1 Predict
- 2 Test
- 3 Prove
Round 5 · Little labelled data
- 1 Predict
- 2 Test
- 3 Prove
Attention: ProsusAI FinBERT (12 layers × 12 heads, d = 64 per head), raw scores q·k recomputed from the model and checked against its attention; round 2 shows the mean over the 12 heads of a layer. Tweets: Twitter financial news sentiment, published validation split used as the test set (2,388 tweets); 8,588 training tweets. Entropy and [SEP] share: 500 test tweets. Small transformer: 3 seeds; bootstrap with 1,000 draws of the test tweets. Code: lectures/L7/analysis_L7.py and build_interactive_L7.py on GitHub.

