How to Evaluate Any NFL Prediction Service (Including This One)

The internet is full of NFL prediction services, models, and experts, and nearly all of them are described the same way: by a number someone chose to tell you. Our AI hits 67%. Documented 61% last season. Nine winning weeks in a row.

Here's the thing about chosen numbers: choosing is the trick. Any forecaster with enough output produces impressive-looking stretches by pure arithmetic — a coin picking sixteen games lands an 11–5 week every month or so. The difference between a real forecasting operation and a highlight reel isn't the best number each can show you. It's what happens when you get to pick which numbers to look at.

So don't evaluate claims. Evaluate publishing standards. Six questions do the whole job — and because we run a prediction model ourselves, we'll answer each one for our own site as we go. Grade us with the same rubric. That's the point.

1. Is the complete graded history public?

Not a highlights page — the ledger. Every prediction, timestamped before the games, graded after, none deleted. The single most reliable tell in this genre is what a service does with its misses: a real operation stores them in the same table as the hits, because the misses are what make the hits mean something.

Us: every prediction our model has made is in our database and displayed on the site — by season, by week, by confidence tier. Lifetime: 821 correct, 733 incorrect, 43 pushes against the spread (52.8%). The 733 are load-bearing.

2. Is the accuracy claim even possible?

Calibrate your skepticism to the benchmark. Predicting winners at a high rate is unremarkable — favorites win often, and our model runs about 63% at that task. Predicting against the spread is a different sport entirely: the number is engineered to split outcomes 50/50 and absorbs nearly all public information. Decades of academic study and every honestly-graded system we're aware of agree on the plausible band: sustained low-to-mid 50s is excellent; sustained 60%+ against the spread over large samples is a claim that exceeds anything credibly documented. A service quoting it is telling you about its marketing, not its model.

Us: 52.8% lifetime against the spread, 52.5% in 2025. Unimpressive-sounding, plausible, and real — pick any two was never on offer.

3. Do they disclose the validation-versus-live gap?

Every quantitative model has two accuracies: the one from testing on history, and the one it delivers live. The first is always higher — models get selected because they aced the test, historical benchmarks are static targets, and the live market adapts. A service quoting one number should be asked which one it is. A service that doesn't know the difference has bigger problems.

Us: our production model tested around 58% on held-out history and delivers 52.8% live. We explain that gap in detail rather than quoting the prettier number, because the live number is the only one you'll ever experience.

4. Do they show you their variance?

A real system's weekly results are violently streaky, and an honest service shows the streaks in both directions. Our own model produced a 13–1 week and, two months later, a 32–45 five-week stretch — same model, same process. Any service that surfaces only its 13–1s is curating, and curation is the entire scam, executed politely.

Us: the week-by-week tracker is on the site, ugly stretches included. We've even published the full anatomy of our late-season fade — the model is measurably worst in Week 18 (42.7% lifetime) — because the shape of a system's failures is information you're owed.

5. Is the methodology stated?

You don't need the source code. You need enough to know a method exists: what data goes in, how the model is trained and tested, what the confidence tiers mean, what gets published when. "Proprietary algorithm" plus a phone number is not a methodology — it's a costume.

Us: 360 computed features over every game since 2002, gradient-boosted trees validated on held-out seasons, weekly re-scoring, confidence tiers assigned by the model's own probability margins, results graded against closing numbers automatically. Written up on the site in as much detail as anyone cares to read.

6. Have they ever published a failure?

The final question, and the one almost nothing survives: has this service ever told you about something that didn't work? Real research produces mostly dead ends — that's what makes it research. A service that has only ever succeeded in public either doesn't test its ideas or doesn't report the tests. Both are disqualifying, and not just ethically: the failures are how you know the successes were tested the same way.

Us: we spent months building an over/under prediction model — same pipeline that works on spreads — and it lost to the market's number comprehensively. We published the post-mortem, including the seductive fake result (a 57.6% backtest with coin-flip calibration underneath) that we caught and threw away, and the totals projections we publish carry our most conservative confidence tiers because of it. That article is the best credential we have.

The rubric, in one line

Full ledger, plausible claim, live-versus-test disclosure, visible variance, stated method, published failures. Six questions, five minutes, and they sort the entire genre — us included. We built SpreadTrends to pass this test not because we always look good against it (see: Week 18) but because a standard you only apply to competitors isn't a standard. It's a pitch.

Hold everyone to it. Especially us.

The complete graded record is on the SpreadTrends Predictions page; the totals post-mortem and the week-by-week accuracy analysis are on the blog.