Neural Networks and Deep Learning

Lecture 7 · Game: You or FinBERT? Finance tweets on which the models disagree

You or FinBERT? Finance tweets on which the models disagree

You read 10 real finance tweets and decide whether each one is bearish (bad news for the price), neutral or bullish (good news for the price). After each answer you see what FinBERT decided, what two simple models decided, and the true label. The tweets were chosen on purpose: they are cases where the simple models and FinBERT do not agree.

FinBERT is a BERT (bidirectional encoder representations from transformers) model pretrained on financial text and fine-tuned in the lecture on 8,588 labelled tweets. The two simple models are term frequency–inverse document frequency (TF-IDF) with logistic regression and the embedding bag; neither looks at the order of the words. Of the 2,388 test tweets, in 230 the two simple models give the same label and FinBERT a different one; the game draws its cards at random from these 230. Links in the tweets are removed.

Class board for the projector

Data: Twitter financial news sentiment (Hugging Face, zeroshot), published validation split (2,388 tweets), the test set of the lecture. Disagreement set: TF-IDF + logistic regression and embedding bag (first run) agree, FinBERT fine-tuned (first seed) differs; 230 tweets. The 30 cards of the game are drawn at random from them (seed 20261108); every game draws 10. Predictions: cached outputs of analysis_L7.py. Code: lectures/L7/make_game_L7.py.