Share this article

print logo

A dark matter: Secrecy abounds as NFL teams tackle analytics challenge

For two reasons, people don’t like to talk about this.

1) Those in the industry would rather take an Aaron Rodgers pass in the windpipe than surrender their secrets.

2) Most who aren’t privy would have trouble understanding the convoluted concepts anyway.

The world of football analytics is fascinating for those who explore it, aggravating for those who refuse and tricky to grasp all the way around. The biggest football analytics advocates can’t say for sure where the movement is headed, but it’s obviously growing and can make a difference for NFL teams eager to take it seriously.

“There are some people who can’t get enough of it,” said Russ Brandon, the Buffalo Bills’ president and managing partner, “and some people who don’t want any part of it.”

Yes, there still are analytics holdouts throughout the NFL.

The list includes owners, general managers and coaches who view the phenomenon as radical in a league where even the worst clubs print money, gut instincts are respected and tough talk moves the needle.

They needn’t be afraid.

Sports analytics (aka advanced stats, aka Sabermetrics) is a phrase that entails using data − sometimes generated independently of traditional statistics − to plan a more objective strategy as opposed to taking “this is how it’s always been done” as an acceptable organizational philosophy.

In football, analytics can be used to determine when a team should go for it on fourth down rather than punt, when to break into a hurry-up offense based on the situation or what percentage of the salary cap each position should consume.

“Analytics is, at its heart, just applied mathematics,” Baltimore Ravens offensive lineman and Canisius High grad John Urschel said in March at the MIT Sloan Sports Analytics Conference. Urschel is working on a mathematics doctorate at MIT.

“It’s taking a quantitative view of something and trying to see what is at its core, what is at its root.”

Ignore or embrace, worthwhile original data nonetheless exist to help an organization willing to buy into analytics, from the owner down through the scouting department and the coaching staff.

“The teams that can figure out how to use this information are the ones that will get ahead,” said a former analytics assistant for an NFC team who wanted to protect his identity in a field where even the teensiest bits of information are treated as state secrets.

Football analytics aren’t some passing fad to appease eggheads. The movement continues to evolve. Quality information continues to mature. The desire for researchers continues to grow.

Syracuse University’s Falk College of Sport and Human Dynamics responded to projections indicating demand will far exceed the national job growth average by announcing last month it will offer the world’s first undergraduate degree for sports analytics.

Experts across the board agree − particularly in the NFL − an organization must have full commitment for the research to pay off in football’s three key analytics areas: in-game decisions, player acquisition and performance science.

“I don’t want every team to buy in,” said a mathematician for another NFC team. The mathematician also requested anonymity.

“I want teams stupidly spending their money in bad places. It’s always nice to have someone to take advantage of.”

A team could assemble the NFL’s greatest analytics department and produce revolutionary data, but if the general manager chases white whales and overspends in free agency, then money will be wasted and adverse ripples sent throughout the roster.

A coach could be handed a Rosetta Stone to defeat next Sunday’s opponent, but if he can’t process that information and make the correct call with the play clock dwindling and down two scores, then he might as well chuck his Rosetta Stone into Lake Erie before the game.

Mike Tyson is credited with saying, “Everybody’s got a plan until they get hit in the mouth.”

The quote could pertain to the usefulness of football analytics. On paper, they can look magnificent. But the NFL still employs coaches who in tense moments flip into job-preservation mode or revert to instincts that served them well in 1990.

Some teams don’t have an analytics department. The Bills launched theirs in October 2013, when they hired Xerox data strategist Michael Lyons to work with every department from football operations to finance to ticket sales. Brandon conceded it’s fair to say Lyons’ work is more obvious with the Bills’ ticket pricing than with their on-field product.

“We look at it from a holistic perspective,” Brandon said. “We hired Michael to oversee all of our business analytics at well. He is a crucial part of the mix between marketing and content and the decisions we make in all platforms of our business.

“Most people assume the analytics person is driving data and running it down the hall to the GM. But it’s a lot bigger than that.”

Advanced statisticians insist the Bills’ front office and coaching staff repeatedly have violated two universal truths about constructing a successful roster in today’s NFL. More about that later.

The Cleveland Browns, meanwhile, are taking analytics commitment to a deeper level than any team has tried before. They actually hired a baseball Sabermetrician, one of the most prominent, to oversee football operations.

Paul DePodesta, played by Jonah Hill in the “Moneyball” motion picture, is the Browns’ “chief strategy officer” and reports directly to the owner. DePodesta was assistant GM in Oakland, GM for the Los Angeles Dodgers and scouting vice president for the New York Mets. He played football at Harvard but never has worked in the sport.

“Among league observers who use data, there’s a lot of admiration for what the Browns are doing,” said Bill Barnwell, an analytics-oriented writer for “I think there’s a lot of logic to what they’re doing. But it very well may not work.

“It’s like saving for retirement. The Browns are maxing out their 401(k), investing in mixed funds, being very wise. But maybe one day there’s an illness that forces you to drain your 401(k) or the stock market crashes.

“And, hey, Colts owner Jim Irsay played a scratch-off ticket and won $100 million.”

After the Indianapolis Colts made the playoffs nine straight seasons with Peyton Manning, the star quarterback got hurt in 2011. The Colts went 2-14, just in time to get quarterback Andrew Luck with the first overall draft choice.

For Irsay, an accidental occurrence paid off handsomely, the type of anecdote that dissuades many in the NFL from investing the time, energy and genuine interest in analytics their counterparts in baseball and basketball do.

“In football, the numbers aren’t as good as in other sports, and the market forces don’t dictate teams to try something different,” Barnwell said. “It’s not as if the Browns are really losing money. Because of that, there’s not the impetus to fully commit to a quantitative approach.

“Of course, the Browns are doing it now because they’ve been so bad for so long. But a lot of owners are very comfortable making money and staying in the cycle of waiting for that franchise quarterback to come to them.”

Teams have been more prepared to admit that type of approach rather than cop to constructing a team through advanced stats.

Six NFL teams denied The Buffalo News permission to interview their analytics chiefs. The Ravens denied a request from Urschel’s hometown paper even to speak with him.

An unnatural fit

Unlike baseball and basketball, analytics haven’t found a football toehold.

Hockey also has lagged far behind in effective applications, although the NHL has embraced analytics as online media content.

Yale-educated mathematician Sandy Weil has studied the four major U.S. sports. He’s director of sports analytics for Kroenke Sports and Entertainment, owners of the NFL’s Los Angeles Rams, the NBA’s Denver Nuggets and the NHL’s Colorado Avalanche. Weil formerly was football analytics director with the Baltimore Ravens.

“I’ve seen the guts of four different sports,” Weil said, “and you see the paucity of data on the football side for a lot of positions compared to basketball or baseball.”

Baseball popularized sports analytics, the mining of data for competitive purposes.

Statistics revolutionary Bill James began publishing his annual “Baseball Abstract” book in 1977, but his Sabermetrics (he called them “the search for objective knowledge in baseball”) didn’t gain mainstream recognition for a quarter-century.

The Oakland A’s made Sabermetrics an organizational philosophy and won on a tight budget by putting a greater emphasis on avoiding outs than, say, hitting home runs or stealing bases. Oakland’s success inspired the 2003 best-seller “Moneyball.”

Baseball, by and large, is a pitcher-versus-batter competition with a century of stats to draw from. There is no time limit per at bat, inning or game.

“Baseball is more easily quantifiable in a whole bunch of different ways,” Weil said. “They have a lot more data, a minor-league system, a lot more player-for-player trading and it’s far more mature.

“A baseball team that has its arms wrapped around its data will have far more leverage against a team that doesn’t than compared to football.”

Basketball is more difficult with five players on each side and a shot clock. Hockey, with its random bounces, is tougher yet.

But quantifying each role on a football field seems impossible.

One football play comprises 22 interdependent athletes scattering across 6,400 square yards. Down, distance, time and the score create variables galore. Schemes vary − sometimes wildly − from game to game.

“Football is a fascinating sport because you can watch and truly enjoy it without really understanding, for instance, what’s happening on the offensive line,” said Rodney Paul, a sports economist who helped establish Syracuse’s sports-analytics program.

“Meanwhile, there is so much going on with actual strategy that’s difficult to get a handle on. Analytics helps us understand it, but I don’t think football analytics can become all-encompassing and answer all the questions because football doesn’t lend itself naturally to this like baseball does.”

Basketball has used advanced stats to discover assets never before considered.

A decade ago, the phrase “three-and-D” was nonexistent in the NBA. A three-and-D specialist is a wing player who can hit three-pointers and play tenacious defense. Basketball discovered isolating players with just those two traits turned up previously unrecognized commodities.

“Basketball and baseball are in a different state in terms of the number of counting stats they’ve kept track of historically,” Weil said. “It’s just a more fertile ground.

“You don’t have that kind of depth in football, where you keep track of 10, 20 counting stats, but they’re applicable to maybe three positions for the most part.”

There are three main uses for football analytics:

• In-game decisions are controlled by the coaching staff. Examples include clock management, down-and-distance situations, replay-challenge risk management and when to go for a two-point conversion.

• Roster building is handled by the general manager and his advisers. Advanced stats can be used to help evaluate college players and available free agents, assign contract values for each role on the squad and assess proposed trades.

• Performance science might hold the most promise and crosses over to other sports. Wearable sensors monitor fatigue and help in projecting injury risks and recovery times. Radio-frequency ID chips are tethered with GPS to provide mapping data used to track traits such as acceleration and separation.

“From when I started in the NFL 20 years ago to where it is today, it’s like we’ve landed on Mars,” Brandon said of strides made in performance science. “In the old days, you would pound the weights and go home. It’s come a long way.”

But there are obvious blind spots casual fans understand.

Football is the only major sport for which you cannot field a position-for-position fantasy lineup.

Offensive linemen don’t have official statistics. They might eventually, thanks to analytics.

“You can’t just go scrape the NCAA website and be able to put draft values on football guards,” Weil said. “You could actually hope to do that with basketball guards.

“Eventually, as tracking becomes more accessible, we’ll be able to fill ourselves out, and football will probably catch up in a lot of ways.”

Myriad problems exist in evaluating and comparing players in the other two phases, too.

“Quantifying defensive or special-teams play is going to be the next big leap,” Paul said. “People are going to be able to better separate the value of a defensive lineman over a linebacker or a safety that can tackle versus one that can cover.”

NFL clubs must maintain their own systems to grade positions throughout the league. Such internal metrics vary from team to team and are kept confidential.

Football Outsiders and Pro Football Focus also grade every position, but those outlets face an inherent obstacle for exact evaluations.

“When groups like Pro Football Focus are grading plays, there are a lot of assumptions made because they’re not privy to the play call,” said the former NFC analytics researcher who asked for anonymity. “But that piece of information still is better than not having any at all, and that will continue to evolve.”

Innate complexities

While the average football fan is generally unaware of advanced football stats, gamblers and daily-fantasy players will devour them. Forecasting companies like AccuScore handicap point spreads, point totals and player-proposition bets by running computer simulations based on key data.

Another problem with helping football analytics reach a mainstream audience is the metrics are hard to explain.

Each time a batter steps to the plate or a relief pitcher gets summoned from the bullpen, all sorts of stats appear on the TV screen. Some are newer than others, but they are ubiquitous throughout slow-moving baseball broadcasts. NHL intermissions use analytical content to help pass the time.

There’s no such dead time on NFL broadcasts, which will sooner cut to highlights and fantasy updates from other games than get into a dense mathematical discussion.

Any idea what DVOA is?

It’s a respected Football Outsiders metric that calculates defense-adjusted line over average, a measurement of “a team’s efficiency by comparing success on every single play to a league average based on situation and opponent.”

The detailed DVOA explanation on Football Outsiders’ website is 2,200 words long.

Football Outsiders also offers metrics such as DYAR (defense-adjusted yards above replacement), ALY (adjusted line yards) and Pythagorean projection (a Bill James formula used to project wins, but retrofitted for football).

“There aren’t those obvious sorts of numbers to put out there,” said Barnwell, a former Football Outsiders managing editor. “Maybe that’s part of introducing it. There’s not easily recognizable data.

“DVOA is valuable, but it’s not very useful for individual players, and you can’t explain it in a sentence. You need 10 minutes.”

A mind of their own

Two of the universally accepted truths in football analytics are that trading up in the draft is too expensive and should not be done, and that smashmouth offenses have become the least productive.

The Bills have disobeyed each tenet.

Bills GM Doug Whaley in 2014 traded the ninth overall draft choice, their 2015 first-round pick and a 2015 fourth-rounder to move up five spots and select receiver Sammy Watkins fourth overall in 2014.

Rex Ryan’s offense is based on a ground attack. Buffalo’s 509 carries last year ranked second, while Tyrod Taylor’s 109 attempts were second among all quarterbacks. Running back LeSean McCoy has Buffalo’s third-highest salary-cap figure. Jerome Felton is the NFL’s highest-paid fullback.

Brandon was asked about the Bills contradicting advanced statisticians.

“That’s their opinion,” said Brandon, who serves on the advisory board for Syracuse University’s sports management department. “How we compile the roster and utilize our schemes are under the guidance of our general manager and our head coach.

“I would probably leave it at that.”

A year before the Watkins transaction, Cade Massey from University of Pennsylvania’s Wharton School and Richard H. Thaler from the University of Chicago’s Booth School of Business wrote a seminal research paper for the scholarly journal Management Science.

Massey and Thaler found early draft picks are worth significantly less than what teams surrender while trading up.

“If picks are valued by the surplus they produce,” Massey and Thaler wrote, “then the first pick in the first round is the worst pick in the round, not the best. In paying a steep price to trade up, teams are paying a lot to acquire a pick that is worth less than the ones they are giving up. ...

“Indeed, the irony of our results is that the supposed benefit bestowed on the worst team in the league, the right to pick first in the draft, is only a benefit if the team trades it away."

Victorious teams in today’s NFL generally end a game with solid rushing stats because they were winning and working on the clock.

Football Outsiders rated Buffalo eighth in offensive efficiency based on DVOA, but 28th in week-to-week consistency.

The Carolina Panthers ranked first in rushing attempts, and Cam Newton ran most among quarterbacks. The Panthers went to the Super Bowl, but Football Outsiders ranked them the NFL’s second-most efficient defense. The Bills’ defense ranked 24th.

The first Football Outsiders essay, written by founder Aaron Schatz in July 2003, refuted the long-held belief an offense must establish the run: “There is no correlation whatsoever between giving your running backs a lot of carries early in the game and winning the game,” Schatz wrote.

“Adrian Peterson is a great running back who gets more long gains than anybody else,” said the anonymous mathematician working in the NFC. “But if what you want is big gains, then look at the passing game.

“Most mediocre starting quarterbacks in the league produce much more effective long plays. Blaine Gabbert is going to get there more often than Adrian Peterson.”

Future look

The evolution of advanced football stats will accelerate when they are shared effectively with those who execute them.

Barnwell theorizes a Princeton mathematician who can’t deliver innovative formulas to the coaching staff won’t help that team as much as a lesser analyst who can make the idea digestible.

“That’s where the state of football analytics is right now,” Barnwell said. The Jacksonville Jaguars approached Barnwell to work for them in 2012, but he stayed with ESPN’s Grantland site. “It’s a lot more about communicating concepts and a lot less about having the best numbers.”

Introducing analytics in a satisfying way to the football operations department is critical for those who crunch the numbers.

Although they’re in competition against each other to calculate the best empirical data, they want the analytics movement to progress throughout the NFL. Analytics employees with rival teams will root for Cleveland this year.

“What worries me is the wrong guy doing our job,” the anonymous NFC mathematician said. “He’s negatively coloring the opinion of coaches and scouts and general managers about the role analytics play. They’ll think, ‘Analytics suck,’ and someday that scout becomes a GM somewhere else, and that coach gets a job with another team.

“The wrong analytics guy taints the league-wide perception of what our value can be to a team.”

Perhaps at some point every NFL club will embrace advanced stats.

Analytics almost certainly won’t replace traditional scouting methods. Eyewitness and video evidence remain superior.

But when GMs and coaches abandon old-school doctrine the numbers don’t support and begin trusting unconventional, objective data, they should gain an edge over those who refuse.

Analytics experts predict that if front offices fully buy in, then they will become more efficient about obtaining and re-signing older players. There will be less dead money around the league. Trading up in the draft will become deviant behavior until the market shifts and the price to do so becomes substantially less expensive.

On the field, Weil said, “It would be rare for somebody to punt on the other side of the 50 on fourth and 4. Everybody would be going for it on fourth and 1 from their own 30. That stuff’s already been covered.”

And yet, few teams abide by such data because GMs still fall in love with players they probably should merely like, and coaches don’t want to jeopardize their jobs by making an unorthodox decision their owners might not fully understand.

“While there’s still a lot of baseball debate, it’s on the margins,” said Paul, the Syracuse sports economist. “With football, it’s more a matter of filtering through the noise and coming up with the things that really stick. That’s not necessarily a linear path.

“People will try all kind of different things. Some stuff works, some stuff does not. When a team finds something big, they’ll protect that information for an advantage and we might not know for years what works.”

Because it’ll be a secret.

And there will be people around the league who don’t care to understand it, anyway.

Story topics: / / / / / / / / / / / / / / / / / / / / /