Who are the best MMA fighters of all time. A Bayesian study

Like with any sport, the question of who are the best competitors of all time in Mixed Martial Arts (MMA) is something that is hotly debated among MMA fans. And unlike for tournament based sports such as tennis, or sports where results can be objectively measured such as track and field, it is a question that is much more difficult to answer in MMA. Firstly, fighters compete in different weight classes and organizations, often making direct comparison impossible. Secondly, even when competitors are in the same weight class in the same organization, comparison can be difficult simply because of the relatively low number of matches that an average competitor has – not everyone is competing against everyone else. All of this leavies a lot of room for speculation and what-if scenarios. Indeed, any ranking of fighter’s skills is highly subjective, dependant on the criterias set forth and as a consequence highly debatable.

What we can do though is to take a purely statistical approach, by estimating which fighters are the likeliest to have the highest skill level, considering their wins and the quality of their opponents. This blog post intends to do just that, by using state of the art statistical methods that have been successfully used in similar settings, for example to analyze and rank chess players (see
TrueSkill Through Time: Revisiting the History of Chess
) and for ranking players in competitive online games.

Dataset

The data we will utilise is obtained from Sherdog fight finder, a comprehenseive database of MMA matches, dating back from 1980 (with the first match between Casemiro Martins and Rickson Gracie). The dataset includes results of over 200,000 matches and over 94000 fighters in total. The following chart displays the number of matches per year in the dataset.
matchesperyear

In this data, we have full details of match outcomes, including win type (if the fight ended with a win for one of the contestants), but also information about matches ending in no contest or disqualification. Since the study tries to esimate skill, we ignore data about both NC and disqualified matches (even though the latter means a win for one of the fighters, it is not possible to infer the skill level based on that fact alone, Jon Jones vs Matt Hamill being a good example ). All computatios are done on the snapshot of the data as of Dec. 20, 2015.

Skill inference

Our goal is to assign a skill level for each fighter, based on observed match outcomes. As mentioned earlier, the methodology we will employe is based on the same Bayesian statistical approach that is used by Microsoft for Xbox called Trueskill to determine the skill level players so that they can be matched up optimally competitive multiplayer games such as Halo. Also, the same approach has been employed to rank chess players using historic chess match outcome data from 1850 to 2006 (see TrueSkill Through Time: Revisiting the History of Chess).

We rely on Bayesian inference to infer the skill of competitors based on the observed match outcomes. As skills of fighters are unknown, they are assigned a probability distribution, in the current case a Gaussian with mean \(\mu\) and variance of \(\sigma^2\). The mean \(\mu\) of a fighter skill specifies the average performance of the fighter. Our uncertainty in the skill level is specified by variance \(\sigma^2\). The match outcome is determined by the performance of both contestants. Since performance of a fighter can vary from match to match (good days and bad days), it can be thought of as a noisy version of skill. The winner of the match is the one with hgher performance in that particular match.

The following is the high level summary of the model we base the stastical inference on

  • Each fighter’s skill is a normally distributed (Gaussian) random variable (the mean and variance of which we will lern from the data). Since a fighter’s skill changes over the years, we have a skill ranking for a fighter for each year he is active. These are the values we want to infer.
  • In every match, fighters performance comes from skill, i.e. is drawn from Gaussian \( \mathcal{N}(skill, performanceVariance)\), where \(perfromanceVariance\) is another hidden variable, learned from data.
  • A fight ends in a decision victory for fighter \(f_1\), if his performance \(p_1\) is greater than the performannce of fighter \(f_2\) + some threshold \(decisionThreshold\), i.e. \(p_1 > p_2+ decisionThreshold\).
  • If the skill difference is less than the decision threshold, the fight ends in a draw.
  • A fight ends in a finish victory (sbmission, KO or TKO) for fighter \(f_1\) if \( p1 > p2 + finishThreshold\)
  • Both finish and decision threshold themseves are hidden variables that are learned from data
  • Finish threshold is constrained to be strictly larger than decision threshold, i.e. \(finishThreshold > decisionThreshold\)
  • Two contestants in a match are assumed to be on a somewhat similar level, i.e. their skill level is very unlikely to be vastly different. In other words, UFC level fighters are in general matched up with other UFC level fighters, not low level amateurs. This assumption helps us in inferring the skill level of fighters who have a very small number of fights. Thus for each match, we assume \(\mid skill_1 – skill_2 \mid \lt matchupThreshold \). The value of matchup threshold is again a hidden variable learned from data.

To implement the model we use Infer.net, a Bayesian inference framework. The model is in large parts based on the Chess analysis model built by Microsoft Research on the same toolset. The code for the model can be found in Github. The final inference is done model is inferred based on the full Sherdog fight database.

Dealing with weight classes

While most matches are within a weight class, there are also matches where contestants in different weight classes are matched up. In this case, a heavier fighter has an advantage on average. This translates into a bias, where the average skill levels of fighters in higher weight classes will be observed to be higher than those of lower ones. To create a true pound for pound ranking, we can normalize the skill by weight class, by dividing each fighter’s skill level by the average skill level of the given weightclass.

The mean skill level over all fighters is 1000 (since this is chosen as the prior/baseline). The following are the weight class averages (NA denotes no information about a fighter’s weightclass).

Weight class Average inferred skill
Heavyweight 1159
Light Heavyweight 1141
Middleweight 1140
Welterweight 1132
Lightweight 1119
Featherweight 1115
Bantamweight 1085
Flyweight 1085
NA 893

A note about this table: this shouldnt be interepreted in absolute terms (“flyweights are only 6% weaker that heavyweights”). Rather, this is a bias correction table for the pound for pound ranking. In p4p terms, every weight class should have the same average skill level. Since across-weight class matches bias the numbers, this is the table we can use to correct it.

Jon Jones is the best MMA fighters ever

Long story short, statistically, Jon Jones has the highest skill ranking of any fighter in the database. The following graph shows the computed skill level of the top 10 male fighters out of 94000 fighters in the database.

rankings

Jones stands quite a bit higher above other fighters in terms of his skill rating. Where other top fighters’ ratings are very close to each other, Jon Jones’ rating is clearly above. Statistically, this isn’t a surprise. He is without loss (the model disregards DQ losses as irrelevant to skill), and with wins over extremely high level competition.

Looking at the rest of the list, there are some interesting results there. While Daniel Cormier isn’t typically ranked among the very top, statistically speaking, he should be. His only loss is to number one in Jones, and almost all of his wins are over very high quality opponents. The combined record of his opponents is the best in all of top 10.

Anderson’s Silva’s position as an all-time great is statistically somewhat hurt by the start and end of his career, his two losses to Chris Weidman and the losses to relatively weak fighters early in the career mean that from purely statistical point of view, he does not quite reach Jon Jones’s level. But even so, he is among the very top.

Ben Askren is definitely a surprise in the list. His undefeated record is the main reason he is this high. Since we have not observed a loss for him, it makes his probable skill very high from a statistical perspective. This is in some sense a drawback of the model, but fundementally, it is difficult to draw a conclusion about undefeated fighters. Khabib Nurmagomedov is another example for this. Normally, he isn’t considered p4p top 10. Statistically however, it makes a lot of sense, being 22-0 and having a very solid win list, including the current champion.

Two fighter have made meteoric rises in the rankings. Rafael dos Anjos and Conor Mcgregor, starting from a relatively low baseline(due to their lower level competionion and losses earlier in the career) have risen to top 10 very quickly, thanks to their very strong wins in the recent year(s).

One thing that should be emphasized is that the statistical skill levels for the top fighters are actually extremely close. For example, Conor Mcgregor came into the top 10 ranking only after beating Aldo, who was top 5 prior to that. This means a single fight can change the rankings very significantly.

Additionally, as mentioned earlier there is also variance associated with each inferred skill which reflects our statistical uncertainty in the estimation. An this uncertainty is (naturally) very high, due to the low number of fights each fighter has (unlike say chess, where the number of games can be in the thousands for a competitor). This is illustrated by the following chart, where we plot 95% confidence intervals around the mean for fighters often considered to be the top 3 of all time.
rankings_95th
As can be seen, the intervals are fairly wide and overlapping. What this means is that while Jones has the highest average skill given the match outcomes, we cannot conclude with absolute certainty that his skill level is above the other top ranked fighters, simply that it is likely to be the case. And indeed, a slight underperfomance in one of his fights (such a loss to Gustafsson) would have changed the rankings drastically, showing how fragile the ranking estimation actually is. This all boils down to the fact that since the number of fights of a typical competitor is fairly small and there is a lot of “luck” involved, small mistakes making large changes in the overall picture. And our model is well able to capture and quantify this uncertainty.

Where is Demetrious Johsnon?

Many consider the long-reigning flyweight champion Demetrious Johnson to be among the all time best, so him not being in top 10 statstically is somewhat surprising. Looking at the data further, the main reason for this seems to be overall competition, which surprisingly seems slightly weaker in the division. For example, his oppponents’ and their opponents’ combined win percentage is 62.9%. For comparison Chris Weidman’s opponents and their opponents win ratio is 64.6%, for McGregor it is 64.1% . This translates to Demetrious’s opponents ranking be on average lower than the opponents of the fighters in the top 10, in turn translating to his score being lower.

Summary

Bayesian inference provides an excellent toolset for inferring hidden parameters in the data, especially when the amount of data to draw conclusions on is small. As a result it is especially useful in a setting such as reasoning about the skill levels of top MMA competitors. Based on the available data, Jon Jones is the best ever from a statistical perspective. But it has to be kept in mind that the variance (uncertainty) in the estimates is very high and small changes in the data, such as one win or loss, can significantly alter the rankings. And this reflects the real world intuition well: a loss or two can easily change the perception of the fighter in the list of all time greats.

The code for the model to replicate the results or develop it further can be found at Github.

16 comments on “Who are the best MMA fighters of all time. A Bayesian study

  1. Great analysis.

    One thing I think that might be interesting is with Jones in his prime (Dos Anjos and Conner as well) how that will influence the results. Anderson’s losses to Weidman kind of heralded the end of his career. Ultimately I wonder how age and the inevitable drop off in skill effects these statistics.

    My guess is those who retired in their prime like GSP will forever retain a higher ranking versus a fighter who fights on like Dan Henderson.

    • You are right. Loss of fighters when past their prime will indeed affect the overall ranking (and not only ranking but people’s perception as well, Fedor and Silva would be looked at in a different light if they retired prior their Werdum and Weidman matches).

      One option is to build the model after certain time intervals (say every 6 months), and look at the ranks at each interval separately. This would give a better “at their prime” metric.

  2. Ben Askren should be taken out of consideration since he hasn’t beaten any legitimate opponents. Demetrious Johnson is absolutely top 10 of all time. Joseph Benavidez, John Dodson, Kyoji Horiguchi, and Ali Bagautinov are amazing opponents. Looking at opponent’s wins means nothing in MMA and a good example is Mark Hunt’s record being almost even with wins and losses but is one of the greatest Heavyweights of today. Jon Jones, Georges St. Pierre, Jose Aldo, Dominick Cruz, Demetrious Johnson, Conor McGregor, Rafael Dos Anjos, Fabricio Werdum, Anderson Silva, and Fedor Emelianenko are the top 10 P4P fighters of all time. Jon Jones has had good competition but not the best. GSP, Jose Aldo, and Fabricio Werdum had the hardest competition of all of these fighters by a considerable margin of skill. Jon Jones is the best because of his skills and physical attributes moreso than his opponents that he has beaten. Luke Rockhold and Daniel Cormier are good but haven’t beaten competition like the 10 listed above have and don’t acquire the equal skills. I do agree that Jon Jones is the best, P4P.

    • Jon jones has only had “good” opposition? He had possibly the greatest year of competition in MMA history and has fought a murderers row of elite opponents!

  3. Great work. I love this kind of analysis. But I was wondering if assuming that percent wins is a best indicator of an opponent’s skill. I feel like there are two ‘extremes’ where a fighter’s skill level is over/under estimated. At one extreme a fighter skill level may be overestimated if stat pad against weaker opponents. The other extreme is a strong weight class where all fighters are incredibly even. In that case good fighters would have poor win-loss stats. Does this analysis fully take into account these two scenarios?

    • The first case is defintely taken into account, a win against a poor opponent won’t really boost the ranking.

      The second one unfortunately isn’t. But that’s kind of impossible in any case. If all fighters are equal in a weight class, noone would be a standout, and thus noone would be considered a p4p all time great (neither by the model or in a human created list).

  4. Would you please put the top ten in order, and maybe their scores next to them along with average opponent %.

    i.e.

    1) Jones-1000-70%

    Thanks, that would be very helpful as I didn’t actually see the top ten listed out, unless I missed it

  5. Hi

    My name is Carl i am the owner of http://www.freeformfighters.co.uk and we are creating some cool inforgraphs about MMA BJJ and the UFC and giving them away for free to a couple of really good blogs. So we would like to offer one to you for your blog. All we ask for in return is a DOFOLLOW link back to our site at the bottom of the page. So if you would like some free content for your blog just drop me a quick email and ill get something over to you and we will even write a little peace for it to.

    Kind Regards

    Carl.

  6. Hi,

    Great analysis. I was hoping to take a stab at it myself using some business intelligence software. Any chance you could publish your initial data-set from Sherdog?

    Thanks

  7. Pingback: What You Can Study From Popular Milwaukee MMA Fighters - Frases De Futbol | Sports Therapy and recovering from injuries

Leave a Reply

Your email address will not be published.