But for one point, the survey on police relations in Buffalo provides just the kind of information that this city – and communities across the country – desperately needs right now. Unfortunately, that one point is a big one: The survey is unreliable.
Open Buffalo, a local movement dedicated to social and economic justice, conducted the survey this spring, before the recent police shootings of African-Americans in Baton Rouge, La., and near St. Paul, Minn., and before the murderous shootings of police officers in Baton Rouge and Dallas.
Max Anderson, the communications director for Open Buffalo, said the organization was “trying to gauge the perceptions and go beyond anecdotal evidence and stories that we hear about the relationship between police and the community.” That is important and urgent work if the task of improving relations between police and the communities they serve – especially their minority members – is to move ahead in any kind of systematic and efficient way.
But this survey won’t do that, because Open Buffalo didn’t go about it in a systematic, scientifically structured fashion. That’s because the survey overrepresents Buffalo’s minority population, thereby skewing the results and the conclusions that can be drawn from it.
And while Anderson says the survey was never meant to be scientific, the news release accompanying the 26 pages of survey results never acknowledged that crucial point. Anyone looking at it uncritically would likely assume that it was scientific, accepting as representative of the city results including that:
• 60 percent of city respondents think that police don’t respect people of color.
• 44 percent say police do a poor job working with the community to prevent crime.
Even more-positive results, including that 51 percent of respondents believe police will help them when they are in need, are questionable, given how the survey was skewed: Twice as many African-Americans as white people responded to the survey, in a city where whites comprise about 50 percent of the population and African-Americans 39 percent.
That doesn’t necessarily mean the survey results are wrong, as a professional pollster, Barry Zeplowitz, observed, but it does mean that they can’t be confirmed. It is also unfortunate that Open Buffalo presented the results without forthrightly acknowledging the deficiencies of the sample, which also underrepresented people who are 65 or older. That will cause many observers to view not just the survey, but maybe the organization, with suspicion.
It’s also unfortunate because the work the survey purported to undertake is especially important at this moment, when relations are frayed between police and the people who pay their salaries. The lack of reliable data on police use of force, interactions by race or ethnicity and community attitudes creates obstacles to plugging the holes in the system. You can’t treat an illness without first diagnosing it.
What this survey accomplished was the equivalent of medical malpractice. If its results are reliable, they are that way only by accident. Buffalo’s citizens – and police – need better than that.