AI Sports Predictions: How to Tell Whether an Accuracy Number Means Anything

“Our AI gets 87% of games right.”

That sounds excellent. Without more information, it tells you almost nothing.

Does the model predict the winner, total goals, possession, a corner-kick receiver, or one specific event? Which league? How many games? Was the percentage measured on genuinely future data, or tuned on the same history used to build the model? What baseline did it beat?

In sports forecasting, an accuracy percentage is the beginning of an evaluation, not the end.

First ask what is actually being predicted

“Predicting football” can describe very different tasks.

A model may forecast the match winner, draw probability, first receiver of a corner, whether a possession ends in a shot, or where a player will move next.

The broader the question, the more unobserved factors can intervene: injuries, red cards, refereeing decisions, tactical changes, fatigue, weather, and randomness.

It makes little sense to compare a model predicting one tightly defined micro-event with another trying to compress ninety minutes into a single outcome.

Seventy percent can be excellent or useless

Imagine a fictional competition where the favorite wins 70% of matches. A model that always predicts the favorite would achieve 70% accuracy without learning anything especially sophisticated.

Now imagine a task where the target event happens only 10% of the time. A system that always says “no” would achieve 90% accuracy while being almost useless.

That is why one isolated metric can mislead. Before celebrating a percentage, identify the baseline: the simple rule the model must beat.

Data leakage can manufacture a brilliant model

With sports time series, randomly splitting rows into training and testing is often not enough.

If information from the future leaks directly or indirectly into training, the model can look far better than it will when asked to predict a truly future match.

A simple example is using a season-end performance variable to “predict” games played in September. The column may not contain the final score, but it includes information that did not exist when the September match took place.

A credible evaluation should respect time: train on the past and test on a later period that was not used for tuning.

Being right is not the same as being calibrated

One system may say Team A has a 51% chance and be right. Another may say 95% and also be right.

Both count as one correct pick, but the second made a much stronger claim.

When a model produces probabilities, calibration matters: among events assigned roughly 70%, do about seven out of ten actually occur?

That often tells you more than a win/loss tally because it tests whether the model’s confidence corresponds to reality.

TacticAI is useful because it narrows the question

TacticAI, developed by Google DeepMind with Liverpool FC and published in Nature Communications in 2024, does not try to tell you who will win the next match.

It focuses on corner kicks. Its tasks include predicting the first receiver, estimating whether a corner will lead to a shot, and suggesting positional adjustments. The paper evaluated quantitative components on held-out data and also ran a case study with five Liverpool domain experts. In that qualitative study, TacticAI’s suggested adjustments were preferred over existing tactics 90% of the time.

Source: Wang et al., Nature Communications, 2024.

That 90% does not mean “TacticAI predicts football with 90% accuracy.” It means something narrower: experts preferred its recommendations in a specific part of the evaluation.

That distinction is exactly what tends to disappear when a percentage travels from a paper to a headline.

TacticAI is not “the only peer-reviewed sports AI” either

The previous version of this article said something close to that. It was an overstatement.

There is academic literature on pass prediction, shot prediction, player valuation, positioning, match outcomes, and many other sports tasks. The TacticAI paper itself discusses prior work in several of these areas.

What makes TacticAI a useful example is not that it stands alone. It shows how a narrow claim can be evaluated with a defined dataset, explicit tasks, held-out testing, and domain experts.

Market odds are a benchmark, not ground truth

If a system claims to predict match outcomes better than the market, one sensible comparison is the information already reflected in pre-match implied probabilities.

This does not make bookmakers infallible or mean markets are always right. It means a model claiming additional signal should be tested against a baseline that already aggregates injuries, recent form, opinion, and large amounts of market activity.

A model can have a respectable hit rate and still add no useful information beyond that baseline.

Six signs that an accuracy claim deserves attention

1. It defines the task precisely. Not “predicts sports,” but the exact event.
2. It publishes the evaluation period. Training and test windows are clear.
3. It compares against a baseline. Favorite, historical average, simple model, or market.
4. It reports more than one metric. Accuracy alone is rarely enough.
5. It exposes uncertainty. Calibrated probabilities are more informative than binary certainty.
6. It can be audited. At minimum, the data source, method, and evaluation rules are explained.

A commercial product does not have to publish all of its code to be credible. But if the only evidence is a screenshot saying “92% accuracy,” there is not enough information to know what that number means.

The useful question is not whether AI can find signal in sport. It clearly can. The question is whether the published metric demonstrates the capability the product claims to have.