Nearly half of the evaluation scores given to Buffalo public school teachers for last school year were wrong.
Forty-five percent of the city’s teachers were assigned incorrect evaluation ratings due to a calculation error by a private, Utah-based company. The company, Truenorthlogic, stated that its calculations for 1,089 teachers “resulted in lower than actual scores” due to an incorrect scoring formula.
In an apologetic letter sent Wednesday to Superintendent Kriner Cash, Truenorthlogic CEO Jim Rosenthal stated that the company was “embarrassed and sorry” for the mistake.
“We feel terrible,” Rosenthal told The News on Thursday.
He said the company error was unacceptable and that measures have been taken to ensure it doesn’t happen again. Although Truenorthlogic works with many school districts across the country, he said, the teacher evaluation errors uncovered this week apply only to Buffalo.
Lower-than-correct scores were given to educators who teach more than one grade level or subject and are required to meet multiple sets of student learning objectives. The company had rewritten its scoring calculations over the summer to enable it to produce scores more rapidly, Rosenthal said. But in doing so, it inadvertently created the calculation error for this group of teachers.
The district’s data chief, Genelle Morris, said a teacher brought the error to the district’s attention. According to the teacher’s manual calculations, she had met her performance targets, but that was not reflected in the online calculations produced by Truenorthlogic. The district checked her calculations and ran them internally through the district’s Information Technology Department and found it could not replicate Truenorthlogic’s scores.
The company soon uncovered the source of the error and the corrected results were posted online for teachers to view late Thursday morning.
“We greatly value our partnership with Buffalo Public Schools and appreciate all they do for kids, and all they do in partnership with us,” he said. “We made a mistake, and we want to move forward with a great school district whose relationship we greatly value.”
Truenorthlogic has been contracted by the district for the past three years to manage the district’s evaluation and evaluation scores. This is the first time the company has been known to miscalculate teacher evaluations, though teachers have complained about the company’s cumbersome user interface.
The 2014-15 Buffalo teacher evaluation ratings are not publicly available. For 2013-14, 48 percent of the Buffalo school district teachers and principals evaluated were rated “highly effective,” 21 percent were considered “effective,” 5 percent were considered “developing” and 2 percent were considered “ineffective.”
Though the recalculation of teacher evaluation ratings will boost overall scores for hundreds of district teachers, Philip Rumore, president of the Buffalo Teachers Federation, said he still questions whether the errors made by Truenorthlogic might invalidate all teacher effectiveness ratings, since those ratings were supposed to be given to teachers by Sept. 1. He also questioned whether the error applied only to this past year’s evaluation scores or to previous years as well.
“It calls into question whether these scores are in compliance with the law,” he said.
Rumore added that the recalculated scores likely mean that some lower-rated teachers who would have been required to complete teacher improvement plans may no longer have to do one.
Morris agreed the recalculated scores will likely change the effectiveness rating for a number of teachers for the better, but she couldn’t yet say how many. She disagreed, however, with the BTF’s contention that posting the scores two days late could invalidate the results. The ratings are not due to the state until October, she noted. All teachers who were incorrectly scored also will receive new letters in the mail with their corrected rating.
In response to an inquiry, the state Education Department said it’s up to each district to determine what companies it works with to evaluate and score teachers.
“The state does not have any authority to control the vendors that districts and BOCES use to store their evaluation data and provide that data to their teachers,” spokesman Jonathan Burman said. “It is up to the districts to ensure that whatever system they use, they are able to provide scores and ratings by Sept. 1 to teachers and principals and meet their reporting requirements in October for providing this information to SED.”