One of Our Models "Went 93%" in a Week. We Back-Tested Every One. Here's What Held Up.

In Week 5 of the 2025 season, one of our prediction models went 13-1 against the spread. Ninety-three percent. That's the number you'd put on a billboard — the kind of week that, in the wrong hands, becomes a marketing campaign.

We did something less exciting and more useful: we tried to kill it. We took that model, and all 33 of its archived siblings from the 2025 season, and put every one through an out-of-sample back-test. The goal was to answer one question honestly — was the 93% skill, or was it variance? — and the answer is a two-part lesson worth more than any hot week.

How you actually test a model

The trap with a great week is that it's measured on the games it happened to see. To know if a model is good, you have to show it games it had no hand in — and you have to freeze it first, so it can't quietly reshape itself to fit the answers.

That's what our back-test mode does: it takes a model's exact, locked feature set and evaluates it on two decades of historical games (train on 2003–2021, grade on a 2022–2025 hold-out), with no searching and no peeking. It's the difference between "how did this model do on its lucky Sunday" and "how does this model do, period."

Part one: the 93% was variance

Run through that grinder, the Week 5 model — the 93% model — graded out at 51.8% across the full hold-out. Its own recorded accuracy from when it was built said 57.8%; that figure came from the model selecting itself as the best of thousands of candidates on a slightly friendlier setup, which is precisely how a number gets inflated. Held to an honest out-of-sample test, it collapsed to a hair above 50%.

Thirteen-and-one on fourteen games is about a coin landing heads 13 times in 14 flips. Unusual, absolutely. But across a full season a model makes hundreds of calls, and somewhere in there a two-week hot streak is nearly guaranteed by chance alone — for some model, on some week. Ours happened to be Week 5. Screenshotting it and calling it skill would have been reading the noise as the signal.

Every one of the 34 archived models told the same story: against the spread, they clustered in a narrow band around their long-run rate, none of them separable from the others by any margin that mattered. There was even a tell in the wreckage — several models turned out to be byte-for-byte identical, the same feature set rediscovered by the weekly search and relabeled with a new week number. Churn dressed up as fresh work.

Part two: what actually held up

The same back-test that debunked the hot week also found the real thing. Our straight-up models — the ones predicting who simply wins, not who covers — graded out at a stable ~64% across every archived version, with probability calibration well into genuine-signal territory. Twelve different weekly models, all landing in the same tight band. That's not variance; that's a model that knows something.

The honest footnote: ~64% straight-up is roughly the rate at which favorites win anyway, so much of that skill is the model re-deriving what the point spread already implies rather than beating the market. Real and reliable — just not magic. Predicting winners is a solved-ish problem; predicting margins against a sharp number is the genuinely hard one, and no archived model cleared that bar on honest testing.

The lesson, and why it's how we operate

Two things fall out of this, and they shape how the whole site works:

• A hot week is not a model. It's weather. Any system making enough predictions will have spectacular stretches and miserable ones from variance alone. The only way to tell a good model from a lucky one is to freeze it and test it on data it never touched. If a prediction service is showing you its best week, ask what its average week looks like on games it didn't get to choose.
• Lock and apply; don't chase. Because re-searching every week just rediscovers near-identical models and invites overfitting, we pick one model that survives honest back-testing and apply it all season, then publish its full graded record — the good stretches and the 32-45 slumps alike. The discipline is the product.

The 93% week was fun. It also wasn't real, and we'd rather tell you that than sell it to you.

Our complete week-by-week graded record — hot streaks, cold streaks, and all — is on the SpreadTrends Predictions page. More on how we grade ourselves: How to Evaluate Any NFL Prediction Service.