You or the model? Market stress from chart images
You see 10 real charts of the S&P 500 (Standard & Poor's 500) stock index, each covering 120 trading days between 2018 and 2026: the price on top (100 = first day) and, below, the fall from the highest price so far (drawdown, %). For 7 charts you decide whether the market is calm or in stress at the end of the window; for 3 charts, whether the index ended higher or lower than it started. After each answer you see what the models of Lecture 14 answered, on the same image, and the truth.
A window is labelled “stress” when the volatility of its last 20 days exceeds the 90th percentile of the previous three years (8.4% of the test windows; in the game, 3 of the 7 regime charts). The regime model is a logistic regression on the CLIP (contrastive language–image pre-training) embedding of the image, fitted on the windows up to 2018. The trend model is Qwen2-VL-2B, a vision-language model (VLM) with 2.21 billion parameters, asked the question without any example. All the charts of the game are from 2019 onwards: no model saw them during training.
What the game shows
- On all 394 test windows the area under the receiver operating characteristic curve (AUC, area under the ROC curve) is 0.933 for a logistic regression on returns, volatility and drawdown, 0.907 for CLIP without examples (zero-shot) and 0.880 for the regression on the CLIP image embedding: when the numbers exist, the picture adds a lossy step.
- At the 0.5 threshold the image model flags 63.6% of the 33 stress windows and 15.2% of the calm ones; the model on the numbers, 84.8% and 13.6%.
- Qwen2-VL says correctly whether the price ended higher or lower in only 57.1% of the 394 charts, below the 74.9% of always answering “higher”, yet its AUC is 0.974: it ranks the charts well, but its 0.5 threshold is biased.
Data: S&P 500 daily closes (Yahoo Finance), windows of 120 trading days every 5 days; test windows ending 2019-01-07 to 2026-10-30. Method after Jheng, Pele, Tak and Găman (2026). The game charts are drawn at random from the test windows with the seed 20261108: 12 stress and 24 calm windows for the regime cards, 15 other windows for the trend cards. Model outputs: lectures/L14/results/chart_regime.csv (analysis_L14.py). Code: lectures/L14/make_game_L14.py.

