The Bills are hoping to remove the burden of a 17-year playoff drought on Sunday.
Their infamous drought is the longest in North American team sports, and a full three seasons more than the Cleveland Browns, the next-longest streak in the NFL.
What has become routine for the Bills – missing the playoffs – is actually an extremely improbable feat.
Let’s examine how unlikely this drought is and how much blame lays with the Bills.
There are six playoff spots for 16 teams in each conference. If you assume complete randomness, each year, each team has a 37.5 percent chance of making the playoffs.
Under the randomness assumption, here are the odds that a team doesn’t make the playoffs a certain number of years in a row:
Twist. The. Knife.
Of course, we know making the playoffs isn’t random, and there’s year-to-year correlation. Teams carry the same players and coaches for multiple years. Even if rosters were “random” to start the year, each team faces a different strength of schedule and has softer or tougher barriers to playoff entry based on division and conference quality.
Still, this is jaw dropping.
The chances of missing the playoffs seven years in a row is a minuscule 3.73 percent.
The chances of missing the playoffs for a full decade drop below 1 percent.
The Bills have done both.
Together. The 17-year drought has only a 0.03 percent chance of occurring. Put another way, if you simulated a 17-year span 3,333 times assuming complete randomness, the Bills on average would fail to make the playoffs in all 17 years just once.
This isn’t meant to be an accurate representation of the Bills’ actual playoff odds. Given the correlations mentioned above, the likelihood of a 17-year drought is likely much higher than 0.03 percent. Much of it is the Bills’ fault – a poorly run organization that has featured bad drafts, subpar coaching hires and dubious player contracts. Some of it isn’t their fault – most notably, being stuck in a division with the Patriots and their extended run of dominance.
Let’s try and pin down the Bills’ odds of missing the playoffs every season for the past 17 years based on the actual skill of their team.
Win-loss records and offensive/defensive rankings based on raw yardage aren’t the best determination of actual team skill. Turnovers can be random, particularly fumble recovery rates, and play a huge role in winning or losing. Strength of schedule, which is beyond a team’s control, is impactful.
Raw yardage numbers fail to account for the leverage of a situation. For example, 3 yards gained on third-and-2 is actually better than 10 yards gained on 3rd and 15.
Pythagorean win projections, which are based on a team’s total points allowed and points scored, arguably do a better job, but are not opponent-adjusted nor do they account for the variance of certain scoring events.
So what is the best way to determine the skill of a team?
We can leverage the historical data from the advanced football statistics website FootballOutsiders.com to try and answer this question.
The historical team statistic of the most interest to us is "Estimated Wins." Football Outsiders calculates estimated wins for teams based on a variety of factors, most notably their advanced DVOA (Defense-adjusted Value Over Average) formulas, which measures a team’s effectiveness in a more sophisticated manner than raw yardage totals by accounting for situational context and opponent strength. You can read more about how "Estimated Wins" are calculated here.
Using logistic regression, we can use historical estimated wins for each team over the past 17 seasons and whether or not they made the playoffs to develop a model that projects the probability of any team making the playoffs in a given year.
Here are the Bills’ estimated wins and playoff probabilities for each of the past 17 seasons in both graph and chart:
Of note, the 2004 Bills had a 91.4 percent chance of making the playoffs based on their estimated wins from that year.
In fact, that 2004 Bills team had easily the highest playoff probability of any team to not make the playoffs over the last 17 seasons. Darn you, Willie Parker!
That was the only time the Bills gave themselves a playoff probability north of 50 percent. However, they did give themselves a coin flip's chance twice in the past three seasons.
Under the estimated wins model, here are the odds that the Bills wouldn’t make the playoffs once from 2000-2016:
There’s a huge drop in probability that the Bills would continue to miss the playoffs following their strong 2004 season, but even without that season it’s unlikely the Bills wouldn’t make the playoffs each season based on their talent level.
If you remove the 2004 season, the Bills’ chances of missing the playoffs each year over the drought finish at 1.36 percent. With that season included, the odds drop to 0.12 percent. That’s four times higher than what we saw based completely on randomness, but still highly improbable.
If you simulated a 17-year span, a team with the Bills’ talent level over the past 17 seasons would on average miss the playoffs every season just one time in 833 tries.
While the Bills haven’t been a good team the past 17 seasons, they do get a bad rap. Even after accounting for the talent of the teams they’ve put out on the field, the Bills should have made the playoffs at least once.
The main reason the Bills haven’t been able to crack into the postseason is due to one variable we haven’t accounted for – strength of division.
Or more bluntly put, the Patriots.
Our model is only accounting for the Bills’ playoff odds based on their isolated talent level as determined by estimated wins. However, the structure of divisions within the NFL has really doomed the Bills.
Not only have the dominant Patriots made it difficult for the Bills to win the division, eliminating one route to the postseason, the Bills have to face the Patriots two times a season, making their schedule and path to a wild card berth difficult.
The very first year of the Bills’ drought is the only time they’ve fielded a team over the past 17 seasons that has outpaced the Patriots in playoff probability based on estimated wins:
Realistically, had the Bills played in any other division they’d have cracked the playoffs a few times, although only one team (2004) looks like it had the legs to do any real postseason damage.
Based on our estimated wins model, the Bills should have made the playoffs 4.5 times over the past 17 seasons.
Just for fun (and to torture ourselves), here are the Top 10 teams with the lowest playoff probability over the last 17 seasons to make the playoffs:
In an alternate reality, the Bills played in the AFC South or NFC West and have a couple division titles to go with a handful of playoff appearances over the last 17 seasons.
Unfortunately, this version of the simulation has dealt Bills fans a much harsher reality.
Now go and relive all the pain and misery knows as "The Drought:"
*Estimated Wins data courtesy of Football Outsiders