Another reason to think twice about copying from your lab partner’s answer key is just in: A University professor’s statistical model has conclusively proved widespread cheating by Chicago public school teachers on standardized tests, leading to the firing of six teachers and the reprimanding of three principals, officials announced two weeks ago.
Working under the premise that large score increases and improbable strings of answers would red-flag instances of teachers cheating, economics professor Steven Levitt and Harvard economist Brian Jacob analyzed students’ answer sheets to the Iowa Test of Basic Skills (ITBS) citywide.
“We’re looking for outliers in two different dimensions,” Levitt said. “First we look at patterns of answers—by how much did a group of scores increase? Then we look at patterns of strings to see if there is an unlikely series of correct answers on many papers from the same classroom.”
After a year of developing this model and four months of investigation, their work has materialized into a step forward for ensuring classroom fairness, hailed as the most progressive cheating crackdown in Chicago public school history.
“He brought to us sophisticated analyses to find the truth,” said Arne Duncan, CEO of Chicago public schools.
A year ago, Duncan allowed Levitt citywide access to analyze the ITBS answer sheets, and from that to pinpoint the teachers who improved scores through an array of different methods of cheating, such as indicating correct answers to students, erasing incorrect answers, filling in empty bubbles, allotting extra time and illicitly gaining access to the test and then teaching specific questions.
The investigation began after Levitt and Jacob published a statistics-based report contending that between 1993 and 2000, there were about 200 instances of detectable cheating each year in Chicago’s public schools.
Duncan emphasized that the new finding does not indicate an increase in instances of cheating citywide, but rather an improvement in the detection system.
“We have over 25,000 teachers in Chicago’s public schools,” he said.
“The vast majority did an extraordinary job. Any time you have a system this large you’ll have a handful—and let me emphasize only a handful—of cases where cheating is an issue. You want to deal with it openly and honestly.”
Duncan also emphasized that not all the model’s findings were bad. “He helped bring to light the extraordinary teachers who have helped troubled students succeed,” he said.
John Easton, deputy director of the Consortium on Chicago school research, a federation which monitors the public school problems and pushes for improvement, said the work of Levitt and Jacob raises the standard of accountability in the classroom.
“It sends a statement to schools that they need to really take security procedures seriously,” Easton said. “In the long term it’s going to have a very salutary effect on the testing in Chicago public schools.”
Levitt employed the same mathematical concept last year to prove the widespread cheating of sumo wrestlers in Japan. He showed that wrestlers with borderline win-loss records entering the final day of tournaments had a significantly better chance of winning than wrestlers whose status had already been decided. The implication was that there is a significant amount of match-fixing and corruption inherent in Japanese sumo wrestling.
In addition to cheating, Levitt’s process could be used to expose money laundering and tax evasion, Levitt said.
“You’re trying to figure out some sort of illicit behavior through statistical analysis,” he said. “This process is really tailored to a garden variety of corruption.”
Officials in California, where Levitt is now on a year-long sabbatical, have not given Levitt access to their testing data. Levitt understands why a system would not want this analysis and was impressed with the Chicago school system’s decision to allow him to analyze their data.
“It’s surprising when someone lets you in to show that they’re cheating because it makes them look bad,” he said. “There wasn’t much to be gained for them to let me in.”
But Duncan judged the benefits of weeding out the unscrupulous teachers to be worth the price of opening up the school system to criticism.
“It’s about living a life of integrity,” Duncan said. “We’re not just teaching the students classroom subjects, we’re teaching them integrity.
“We want to be very successful in Chicago,” he continued. “I want to build the best school system in the country here in Chicago. We’re going to do it the right way.”
