This big question is whether there is an accountability system that can't get gamed. -Angela
Exclusive: DISD, other districts unlikely to look into suspicious scores
Saturday, June 10, 2006
By JOSHUA BENTON / The Dallas Morning News
An alarming number of students who graduated from Texas high schools last month probably cheated to get there – and state education officials are in no hurry to catch them.
A state-sponsored analysis found thousands of suspicious scores on the 11th-grade TAKS, the test students must pass to graduate.
The study found 96 Texas high schools where groups of last year's 11th-graders turned in unusually similar answer sheets – suggesting they may have been copying each other's answers. Scores in almost every Dallas neighborhood high school raised red flags.
Eleventh-grade classrooms were more than eight times more likely to have suspicious scores than those in other grades, researchers found.
The study's results don't surprise experts. "Levels of cheating in high school are at astronomical levels," said Tim Dodd, executive director of the Center for Academic Integrity at Duke University.
But in Texas, state and local officials say that these unusual patterns in data – even those that researchers say are millions of times less likely to occur than your being struck by lightning tomorrow – are not enough to trigger scrutiny.
The result is that many of the most egregious cases of likely cheating will go uninvestigated.
"Yeah, kids cheat," said Devin Gustafson, 2006 valedictorian at Seagoville High School, one of the 18 Dallas schools that made the list.
"If you want to cheat on the TAKS, it's not hard."
The findings are the result of a comprehensive analysis performed, at the state's request, by a Utah company called Caveon. It was hired last summer after a series of Dallas Morning News stories found evidence of educator-led cheating in many Texas schools.
The Texas Education Agency paid Caveon more than $500,000 to examine test scores and search for the sort of statistical anomalies that could indicate cheating on the Texas Assessment of Knowledge and Skills. The analysis looked for a variety of patterns, such as a classroom where students made performance gains of unlikely size, or where students answered very difficult questions with ease but struggled with more simple ones.
But the most common anomaly Caveon found was what it called "very similar test responses." That means, in a particular classroom or grade, an unusual number of students answered all or nearly all questions in the same way – including both wrong and right answers.
In its report to Texas officials, Caveon said it only flagged cases with "a low probability of occurring by chance." While it acknowledges that statistics alone are not conclusive proof of cheating, the report says Caveon used a "very conservative statistical approach."
"The conservative approach ensures that while not every potential instance of a statistical inconsistency is identified, those that are identified will be so anomalous that reasonable explanations of these inconsistencies by referring to normal circumstances become improbable," the report states.
'Anomalies'
State officials say that just because a large number of a classroom's students had identical answer sheets doesn't necessarily mean it's worthy of investigation.
"Caveon is pretty much the national expert on this sort of thing," said Shirley Neeley, the state's education commissioner. "But I look at that list and think these are anomalies. I didn't immediately think the worst."
Is it cheating? Perhaps the key piece of supporting evidence is how much more common the "very similar" anomalies are in 11th grade than at other grades.
Caveon found 486 Texas classrooms with an unusual cluster of "very similar" answer sheets.
If those findings were not the result of cheating – if they were just statistical background noise – one might expect them to be evenly distributed among the nine grades in which Texas tests.
But that's not what Caveon found. In grades three to 10, it identified an average of 29 classrooms where test scores suggest answer copying.
In 11th grade, Caveon found 253.
"That's exactly what you would expect: The higher the stakes, the more likely you're going to have some kind of dishonest behavior," said Jason Stephens, a professor of educational psychology at the University of Connecticut who studies high school cheating.
The "very similar" answer sheets were concentrated in the two school districts that have faced the most cheating allegations in the past: Dallas and Houston. Both urban districts had 18 high schools earn the "very similar" flag at 11th grade.
Of Dallas' 21 traditional neighborhood high schools, 17 made Caveon's list. So did one magnet school, the School of Education and Social Services.
That's the district's school for aspiring teachers.
In contrast, none were found in San Antonio or El Paso, and only one was found in Austin, again suggesting the pattern is not random.
Only one school in the Dallas suburbs – McKinney North – had an unusual cluster of similar test scores. Four Fort Worth high schools also had the pattern.
But state and local officials insist there must be some explanation for the 11th-grade scores – something other than cheating.
"We are not going to speculate on the reason that that number occurred," said Susan Barnes, the state's associate commissioner for standards and programs.
"I just believe there's a logical explanation somewhere," Dr. Neeley said. She added that she did not know what that explanation might be.
Inquiries unlikely
It's unclear whether an explanation will ever be found, because many of the schools will probably go uninvestigated.
Initially, TEA chose not to even tell districts that their schools had suspicious score patterns. Dr. Barnes said the agency informed schools only because The News had requested the testing data under open-records law.
"We would not have done it otherwise," she said.
Dr. Neeley informed the districts on Caveon's list in a May 31 letter. But the commissioner did not demand that they begin even a cursory investigation into the suspicious numbers. Instead, she asked districts only to "conduct any investigations you deem necessary to explain" the results.
Many districts are interpreting that to mean it's OK not to investigate Caveon's findings.
The Dallas school district is one.
"We have no investigations planned," said Donald Claxton, the district's spokesman. "This [the Caveon report] just identifies unusual patterns. It's nothing conclusive."
Dr. Barnes said that TEA did not feel comfortable asking districts to investigate Caveon's findings without any additional supporting evidence, such as eyewitness testimony of cheating.
1999 investigation
But TEA has done precisely that in the past. In 1999, a TEA analysis of the erasure patterns on student answer sheets identified 11 districts with one or more schools with questionable results over a period of three years. Based on that data, TEA demanded that all 11 launch investigations. Four concluded there had been cheating by teachers.
Districts on the Caveon list have been told the grade level and subject area in question and what type of statistical anomalies Caveon found.
But they haven't been told other crucial facts. How many students had answer sheets identical to their neighbors? Which students made unlikely gains on the test? Which patterns of answers are suspicious?
"We don't know the parameters that would cause us to be flagged," said Joe Miniscalco, McKinney's senior director of secondary education. "Is it two tests? 200 tests?"
Dr. Barnes said her agency does not plan to give all districts that extra information. "I do believe districts already have the information they would need," she said.
In any case, accurate investigations will be hard at this late date. The report examines scores on a test given more than a year ago. TEA officials said they had draft copies of the report as early as last fall but did not send findings to districts until a few days ago.
By then, most 11th-graders who might have cheated had already graduated.
Cheating common
Teenagers cheating on tests is nothing new.
In 2004, researcher Michael Josephson surveyed nearly 15,000 American high school students and asked whether they had cheated on a test in the previous year. Sixty-two percent said they had – roughly the same number who said they had had at least one beer over the same time span.
Researchers report that public school systems historically have not been particularly interested in uncovering cheating by students or teachers. Don McCabe, a Rutgers University professor who has studied cheating for 15 years, often surveys high school students on cheating. But when he approaches a public school, he usually runs into roadblocks.
"They don't want to know their students are cheating," he said. "They don't want the information, because then they have to deal with it."
Devin Gustafson, the Seagoville valedictorian, said he heard a number of his fellow students talking about cheating on the 11th-grade TAKS test. An example: "One girl, she snuck her cellphone into the test and was text messaging some of her friends to get answers," he said. "She said she only got one or two answers because it was too hard."
He said he didn't cheat, but if he had wanted to, it wouldn't have been hard. "I definitely could have. A couple of times, the teacher left the room and all you would have had to do is turn around and ask somebody for the answer."
Matthew Ramirez, who will be a senior at Skyline High School this fall, said it was easy for students to cheat there. Students who have to turn in their cellphones on test day, he said, are allowed to take them back after lunch – even if they haven't completed the test. "It's not hard to cheat," he said.
Researchers say school systems and state officials don't take their policing responsibilities seriously.
"It's making a mockery of the whole system," Dr. McCabe said. "You invest a lot of taxpayer money and a lot of teacher and student time in a test. And there's evidence there's a problem with the test. And they're not going to do anything about it? They're just going to say, 'Oh, it's just statistics; you can't trust that'?
"It's just going to get worse."
E-mail jbenton@dallasnews.com
Schools where answers were 'very similar'
At several area schools, answers on the 11th-grade TAKS test were similar enough to raise suspicions about cheating.
Math Science Social studies English language arts
Dallas ISD
Adamson X X X
Carter X X X X
Spruce X X X
Madison X X X
Kimball X X X
Pinkston X X
Lincoln X X X
Maceo Smith X X X
Molina X
North Dallas X X X
Roosevelt X X X
Samuell X X X
School of Education and Social Services X X X
Seagoville X
Skyline X X
South Oak Cliff X X X
Sunset X X X
Thomas Jefferson X X X
Fort Worth ISD
Arlington Heights X
Dunbar X X
Eastern Hills X X X
Polytechnic X X
McKinney ISD
McKinney North X
Charter schools
Theresa B. Lee Academy X X
Texans Can! Academy at Paul Quinn College X X X
SOURCE: Caveon
Online at: http://www.dallasnews.com/sharedcontent/dws/dn/education/stories/061106dnmettakscheat.8606778.html
No comments:
Post a Comment