Scorecard
Model forecasts begin when experiment v1 goes live
Live since July 2026: events from 1 July 2026 with forecasts made before the filing.
stats · all · n=544
- Brier · exceeds typical
- 0.2448
- Brier · up
- 0.2511
- Log loss · up
- 0.6954
- Pinball q10
- 0.0113
- Pinball q50
- 0.0248
- Pinball q90
- 0.0122
stats · post · n=238
- Brier · exceeds typical
- 0.2411
- Brier · up
- 0.2509
- Log loss · up
- 0.6951
- Pinball q10
- 0.0117
- Pinball q50
- 0.0268
- Pinball q90
- 0.0137
stats · pre · n=306
- Brier · exceeds typical
- 0.2477
- Brier · up
- 0.2512
- Log loss · up
- 0.6956
- Pinball q10
- 0.0110
- Pinball q50
- 0.0233
- Pinball q90
- 0.0109
| role | session | n | Brier · exceeds typical | Brier · up | Log loss · up | Pinball q10 | Pinball q50 | Pinball q90 |
|---|---|---|---|---|---|---|---|---|
| stats | all | 544 | 0.2448 | 0.2511 | 0.6954 | 0.0113 | 0.0248 | 0.0122 |
| stats | post | 238 | 0.2411 | 0.2509 | 0.6951 | 0.0117 | 0.0268 | 0.0137 |
| stats | pre | 306 | 0.2477 | 0.2512 | 0.6956 | 0.011 | 0.0233 | 0.0109 |
Reliability of P(up) · stats
Each point is a bin of forecast probabilities; its size is the number of events in the bin and its height the share that actually went up. Points on the dashed line mean the stated probabilities were right on average.
Backtest (model training period): all scored events, including those that overlap the models' training data and forecasts made after the fact.
stats · all · n=4120
- Brier · exceeds typical
- 0.2375
- Brier · up
- 0.2502
- Log loss · up
- 0.6935
- Pinball q10
- 0.0120
- Pinball q50
- 0.0256
- Pinball q90
- 0.0122
stats · post · n=1882
- Brier · exceeds typical
- 0.2385
- Brier · up
- 0.2504
- Log loss · up
- 0.6940
- Pinball q10
- 0.0126
- Pinball q50
- 0.0277
- Pinball q90
- 0.0136
stats · pre · n=2238
- Brier · exceeds typical
- 0.2367
- Brier · up
- 0.2500
- Log loss · up
- 0.6931
- Pinball q10
- 0.0115
- Pinball q50
- 0.0238
- Pinball q90
- 0.0111
| role | session | n | Brier · exceeds typical | Brier · up | Log loss · up | Pinball q10 | Pinball q50 | Pinball q90 |
|---|---|---|---|---|---|---|---|---|
| stats | all | 4120 | 0.2375 | 0.2502 | 0.6935 | 0.012 | 0.0256 | 0.0122 |
| stats | post | 1882 | 0.2385 | 0.2504 | 0.694 | 0.0126 | 0.0277 | 0.0136 |
| stats | pre | 2238 | 0.2367 | 0.25 | 0.6931 | 0.0115 | 0.0238 | 0.0111 |
Reliability of P(up) · stats
Each point is a bin of forecast probabilities; its size is the number of events in the bin and its height the share that actually went up. Points on the dashed line mean the stated probabilities were right on average.