'A 70% that happens 70% of the time'
Calibration is the test that separates a model that knows what it knows from one that just sounds confident, and you can check it yourself.
How to check a model honestly
Take every game where a model said 70%. If it is calibrated, about 70 out of every 100 of those happened. If only 55 did, the model is overconfident and its prices are wrong.
Accuracy alone does not tell you this. A model that says 51% on everything can look accurate and still be useless, because it never commits to anything.
The forecasting model is measured on settled NFL games and the result is published. Where a confidence band has fewer than 100 settled games, we show nothing rather than a number the sample cannot support.
73 is not a lock. 73 is 73. That is the whole point.
The measured record
What the model said, against what actually happened, on the game winner.
| Model said | Actually happened | Games |
|---|---|---|
| 54.9% | 55.1% | 1,880 |
| 64.7% | 64.8% | 1,603 |
| 74.5% | 74.3% | 1,086 |
| 83.8% | 81.9% | 421 |
Calibration, answered
Why do you not show calibration on custom lines?
Because they have no track record. A line you typed yourself has never been settled thousands of times, so there is nothing honest to report, and none is shown.
Is calibration the same as being right?
No. A calibrated model still loses plenty of individual bets. It means that when it says 70%, you can trust the 70.
Learn the method
NFL and NFL team names are trademarks of their respective owners. SportHoot is not affiliated with, endorsed by, or sponsored by the NFL or any NFL team. Probabilities are model estimates of what may happen, and SportHoot does not accept bets. 18+.