The National Football League was on the clock.
With several of its marquee players retiring early after a cascade of frightening concussions, the league formed a committee in 1994 that would ultimately issue a succession of research papers playing down the danger of head injuries. Amid criticism of the committee’s work, physicians brought in later to continue the research said the papers had relied on faulty analysis.
Now, an investigation by the New York Times has found that the NFL’s concussion research was far more flawed than previously known.
For the last 13 years, the NFL has stood by the research, which, the papers stated, was based on a full accounting of all concussions diagnosed by team physicians from 1996 through 2001. But confidential data obtained by the Times shows that more than 100 diagnosed concussions were omitted from the studies – including some severe injuries to stars like quarterbacks Steve Young and Troy Aikman. The committee then calculated the rates of concussions using the incomplete data, making them appear less frequent than they actually were.
After the Times asked the league about the missing diagnosed cases – more than 10 percent of the total – officials acknowledged that “the clubs were not required to submit their data and not every club did.” That should have been made clearer, the league said in a statement, adding that the missing cases were not part of an attempt “to alter or suppress the rate of concussions.”
One member of the concussion committee, Dr. Joseph Waeckerle, said he was unaware of the omissions. But he added: “If somebody made a human error or somebody assumed the data was absolutely correct and didn’t question it, well, we screwed up. If we found it wasn’t accurate and still used it, that’s not a screw-up; that’s a lie.”
These discoveries raise new questions about the validity of the committee’s findings, published in 13 peer-reviewed articles and held up by the league as scientific evidence that brain injuries did not cause long-term harm to its players. It is also unclear why the omissions went unchallenged by league officials, by the epidemiologist whose job it was to ensure accurate data collection and by the editor of the medical journal that published the studies.
In 2013, the NFL agreed to a $765 million settlement of a lawsuit in which retired players accused league officials of covering up the risks of concussions. Some players have appealed the settlement, asking for an examination of the committee’s concussion research.
Some retired players have likened the NFL’s handling of its health crisis to that of the tobacco industry, which was notorious for using questionable science to play down the dangers of cigarettes.
Concussions can hardly be equated with smoking, which kills 1,300 people a day in the United States, and the Times has found no direct evidence that the league took its strategy from Big Tobacco. But records show a long relationship between two businesses with little in common beyond the health risks associated with their products.
In a letter to the Times, a lawyer for the league said, “The NFL is not the tobacco industry; it had no connection to the tobacco industry,” which he called “perhaps the most odious industry in American history.”
Still, the records show that the two businesses shared lobbyists, lawyers and consultants. Personal correspondence underscored their friendships, including dinner invitations and a request for lobbying advice.
In 1997, to provide legal oversight for the committee, the league assigned Dorothy C. Mitchell, a young lawyer who had earlier defended the Tobacco Institute, the industry trade group. She had earned the institute’s “highest praise” for her work.
A co-owner of the New York Giants, Preston R. Tisch, also partly owned a leading cigarette company, Lorillard, and was a board member of both the Tobacco Institute and the Council for Tobacco Research, two entities that played a central role in misusing science to hide the risks of cigarettes.
The NFL’s concussion committee began publishing its findings in 2003 in the medical journal Neurosurgery. Although the database used in the studies contained numerical codes for teams and players, the Times decoded it by cross-referencing team schedules and public injury reports.
The NFL’s concussion studies have faced questions since they were published, but even the league’s harshest critics have never suggested, and no evidence has ever arisen, that the underlying data set could be so faulty.
“One of the rules of science is that you need to have impeccable data collection procedures,” said Bill Barr, a neuropsychologist who once worked for the New York Jets and who has in the past criticized the committee’s work.
By excluding so many concussions, Barr said, “You’re not doing science here; you are putting forth some idea that you already have.”
The work begins
In an introduction to the first of the concussion committee’s papers, the league’s commissioner at the time, Paul Tagliabue, acknowledged the need for “independent scientific research” to better understand the risks of concussions.
“As we looked more deeply into the specific area of concussions, we realized that there were many more questions than answers,” Tagliabue wrote.
The committee’s chairman, Elliot Pellman, the team physician for the Jets, emphasized that his group aimed to produce research that was “independent” and “meticulous.”
In fact, most of the dozen committee members were associated with NFL teams, as a physician, neurosurgeon or athletic trainer, which meant they made decisions about player care and then studied whether those decisions were proper. Still, the researchers stated unambiguously – in each of their first seven peer-reviewed papers – that their financial or business relationships had not compromised their work.
The committee said it analyzed all concussions diagnosed by team medical staffs from 1996 through 2001 – 887 in all. Concussions were recorded by position, type of play, time missed, even the brand of helmet.
The committee’s statements emphasized the completeness of the data.
“It was understood that any player with a recognized symptom of head injury, no matter how minor, should be included in the study,” one paper said.
And in confidential peer-review documents, the committee wrote that “all NFL teams participated” and that “all players were therefore part of this study.”
Those statements are contradicted by the database.
The Times found that most teams failed to report all of their players’ concussions. Overall, at least 10 percent of head injuries diagnosed by team doctors were missing from the study, including two sustained by Jets receiver Wayne Chrebet, who retired several years later after more concussions. Pellman, the Jets’ physician, led the research and was the lead author on every paper.
According to the research papers, team physicians were to fill out forms specially designed for the studies to submit information about concussions – a system that went well beyond the league’s standard injury-reporting protocols. In one paper, the committee wrote, “The Commissioner of the NFL mandated all team physicians to complete and return forms whenever they examined a player with a head injury.”
But after the Times described how it had identified missing concussions, the NFL said this week that the studies, in fact, “never purported” to include all concussions.
Teams were “not mandated” to participate, the league said, only “strongly encouraged.” And some teams, a spokesman said, “did not take the additional steps of supplying the initial and/or follow-up forms.” He did not explain why some teams had not included all concussions identified by medical personnel.
The league explained, as did the papers, that some concussions went undiagnosed in the first place because players are known to occasionally hide their symptoms of concussion from team doctors; that symptoms of concussion can be so brief that no one notices; and that doctors might have used different criteria to make concussion diagnoses.
But the vast majority of omitted concussions identified by The Times were included in the NFL’s public injury reports, meaning that medical staffs had made the diagnoses and reported them to the league. Some of the omitted concussions were reported by the teams to the news media after a game but do not appear on the injury reports, presumably because the player’s status for the next game was not in doubt.
The database does not include any concussions involving the Dallas Cowboys for all six seasons, including four to Aikman that were listed on the NFL’s official midweek injury reports or were widely reported in the news media. He and many other players were therefore not included when the committee analyzed the frequency and lasting effects of multiple concussions.
Several other teams have no concussions listed for years at a time. Yet the committee’s calculations did include hundreds of those teams’ games played during that period, which produced a lower overall concussion rate.
A Cowboys spokesman, Rich Dalrymple, said the team had participated, but he declined to say how many cases were reported and which players were involved. He said he did not know why the Cowboys’ data did not appear in the studies. A San Francisco 49ers spokesman did not return messages seeking comment about Young.
Dr. Robert Cantu, one of the peer reviewers who at the time criticized the committee’s analyses, said, “It should be an unmistakable red flag that a team does not report any concussions over multiple years.”
In 1992, amid rising concerns about concussions, Tisch – the Giants and Lorillard part owner – asked the cigarette company’s general counsel, Arthur J. Stevens, to contact the NFL commissioner at the time, Tagliabue, about certain legal issues.
Stevens was not just any tobacco lawyer; he was a member of the industry’s secretive Committee of Counsel, which helped direct tobacco research projects. In a letter obtained by the Times, Stevens referred Tagliabue to two court cases alleging that the tobacco and asbestos industries had covered up the health risks of their products.
In one case, the family of a dead smoker sought internal documents that the tobacco industry had withheld on the grounds of lawyer-client privilege – which does not apply if invoked to cover up a crime. The judge in the case reacted angrily after reading those internal records.
“The documents speak for themselves in a voice filled with disdain for the consuming public and its health,” the judge, H. Lee Sarokin of U.S. District Court in New Jersey, wrote earlier in 1992. Tobacco lawyers succeeded in having Sarokin removed from the case.
Why an influential tobacco lawyer would recommend legal cases to the NFL is not known, because neither Stevens nor Tagliabue would agree to be interviewed. Tisch died in 2005.
Joe Lockhart, a league spokesman, said that the cases involved potential bias of judges and that there was no evidence that the letter “was taken seriously.”
Even so, records show that, in the legal arena, the league and the tobacco industry sometimes intersected.
On at least two occasions in the 1970s and 1980s, the NFL hired a company whose client list included the Tobacco Institute to study player injuries. The league also hired a company – for a matter unrelated to player safety – that had performed a study for the tobacco industry that played down the danger of secondhand smoke.
The NFL’s ties to tobacco are reflected in other ways. When Congress was considering legislation that dealt with when a team owner could relocate a franchise, Joe Browne, a league official, sought lobbying advice from a representative of the Tobacco Institute.
“I would like to take the opportunity to sit down and discuss this bill with you further,” Browne said in a 1982 letter to the institute’s president, Sam Chilcote.
Neil Austrian, a former NFL president, had previously run an advertising agency that under his leadership reversed its ban on taking tobacco clients. He called Philip Morris “an honorable company that sets high standards.” It was during his tenure at the NFL that the concussion committee was created.
Years later, when the committee’s work drew criticism during the peer review process, its members pushed back.
“We are aware the findings from the NFL confront some popular opinions,” they wrote, “but believe this study stands on its merits based on the physician evaluation of injury and quality assurance of the data.”
How the Times decoded NFL database to find 100 missing concussions
From 1996 to 2001, the NFL gathered data in an effort to study the dangers of concussions. Whenever a concussion was diagnosed, the team physician was to record information like the player’s position, his symptoms and how much time he spent recovering.
As is common in such studies, players’ names and teams were represented only by codes, and the information is believed to have never been seen by more than a few committee members. There were 887 concussions recorded in all, forming a log that was consistently described as every head injury that teams had cared for during those six seasons.
The database served as the backbone for the committee’s 13 research papers.
The New York Times recently obtained the data, however, and was able to determine the identities of all of the teams and many of the players. Subsequent analysis confirmed that at least 100 concussions, including some serious ones to the game’s top players, were not included in the league’s studies.
To identify the teams, the Times examined the dates on which each team’s concussions occurred and whether the game was listed as home or away. That information was compared with the teams’ schedules during that time.
Many teams could fit only one pattern; other information, like whether the injury took place on natural or artificial grass, was used to determine and reconfirm the team identifications.
This process led to the conclusion that no Dallas Cowboys players appear in the database – most significantly, quarterback Troy Aikman, whose four concussions during that period led him to retire early. Other teams have no listings for years at a time.
In some cases, contemporary news media articles helped with the identification of players who had sustained concussions that were announced by team officials but were apparently not considered significant enough to later list them on the NFL’s official injury reports. Such players were matched with possible database listings.
This process of identification and elimination identified more than 100 players whose concussions were publicly announced but still not included in the league’s research. It is unclear how many more concussions were diagnosed by teams but, because the symptoms ceased quickly, were never mentioned to outside outlets.
Andrew W. Lehren contributed reporting to this story.