Not all suspect schools on TAKS list

Not all suspect schools on TAKS list
Exclusive: Firm flagged 167 more, but state not looking into it

12:16 AM CDT on Sunday, July 23, 2006


By JOSHUA BENTON and HOLLY K. HACKER / The Dallas Morning News


The list of schools suspected of cheating is longer than Texas education officials have reported – and those officials say they aren't interested in tracking down the latest suspects.

A Dallas Morning News analysis has found that at least 167 unidentified schools were flagged as potential cheaters by Caveon, the company Texas hired to hunt for TAKS cheaters. That's in addition to the 442 schools named by state officials. None of the other schools have been notified that they are on the list.

Texas Education Agency officials say they don't know which schools they are – and they have no plans to find out.

Also Online
Graphic: Tracking down cheaters

State seems to have had right to see cheat data

"The only list of schools we have is the list that has been made public," said TEA spokeswoman Suzanne Marchman. "That's the list we plan to work with."

Superintendents with schools that have been named have complained that the TEA hasn't given them all the information they need to investigate Caveon's findings. But at least they know their scores are suspicious.

"That is so grossly unfair," said Gayle Fallon, president of the Houston Federation of Teachers. "If you're going to accuse someone of cheating, look them in the eye and do it."

Caveon, a Utah-based data-analysis company, was hired by Texas officials last year to examine the students' 2005 scores on the Texas Assessment of Knowledge and Skills.

That followed a series of stories by The News that found evidence of cheating on the TAKS in schools throughout the state.

The News discovered the missing schools when analyzing data in one of the appendixes of Caveon's May report to the TEA. Caveon withholds many of the details of how it performs its analysis, citing proprietary reasons. But in the appendix, the company outlines what it found in one high school where it suspects cheating on the math TAKS.

The report doesn't identify the school by name and lists its students only as anonymous ID numbers. But The News was able to determine that the school is Westbury High School in Houston by matching the student scores to state data. Westbury is the only school in Texas that had student scores matching the data in the appendix.

Houston school officials declined to comment, as did a Caveon spokesman.

Westbury had 1,431 students take the math TAKS in 2005. Of those, 185 had answer sheets Caveon considered suspiciously similar to at least one other student's. Caveon said the chance of that happening at random was less than 1 in 4 million million billion billion billion. That's a 1 with 40 zeroes after it.

The analysis also found several groups of Westbury students who had identical answer sheets – even getting all their unlikely wrong answers wrong in exactly the same way. Caveon also found an unusually high level of erasures at Westbury.

One Westbury junior – known only as No. 3561511 in the report – made a seemingly miraculous gain. As a sophomore, she performed very poorly on the math TAKS, outscoring only about 20 percent of the state's test takers. But as a junior, her score zoomed up – beating that of about 73 percent of Texas students.

No. 3561511 was helped by the 23 wrong answers on her answer sheet that were erased and replaced with correct answers. She ended up with an answer sheet identical to those of three of her peers and almost identical to three others.

In its report, Caveon is confident that there was wrongdoing at Westbury. "It appears likely that there are instances of testing irregularities at this school," it states. Caveon also writes that of all the Texas schools where it found math irregularities, Westbury was the seventh-most suspicious.

But the TEA never told Houston officials that Westbury's math scores were suspicious. That's because TEA officials didn't know themselves.


Dual analyses

The core of the confusion is that Caveon actually performed two different but complementary analyses of the state's test scores. One looked for suspicious test scores in each classroom. The other looked for problems throughout a school.

Both analyses examined the same scores. But they had different standards for how much suspicious activity it took for a classroom or school to be flagged.

Because classrooms have fewer students than whole schools, it takes a higher incidence of suspicious activity for a classroom to be flagged than for an entire school.

For example, imagine a classroom with only 10 students. If two of those students had scores Caveon considered suspect – 20 percent of the total – that probably wouldn't be enough for the classroom to be flagged as suspicious. That's because, when dealing with such small numbers, two strange test scores could result from random chance or "noise" in the data.

But imagine a school with 1,000 students. If 200 of those students had suspicious scores – still 20 percent of the total – that would in many cases be enough for Caveon to declare the school's performance suspect. Statistically, it is less likely that 200 strange scores would be attributable to chance.

As a result, Caveon was more likely to flag an entire school with strange scores than a classroom. Of the 73,793 classrooms whose scores it analyzed, the company flagged 702 – about 1 percent. But Caveon flagged 609 of the 7,112 schools it analyzed – more than 8 percent.


Short list explained

The problem is that the list of 442 suspect schools that the TEA distributed to districts includes only the schools that had classrooms flagged – not those flagged as an entire campus.

There is probably some overlap between the two lists. For example, in the case of Westbury High School, Houston officials have been told that there was potential cheating on the science and social studies tests in 11th-grade classrooms. But the TEA never informed them about the problems Caveon found schoolwide in the math test results.

At a minimum, 167 schools were flagged by Caveon as possible cheaters and still have no idea. According to the Caveon report, the number of those schools could be as high as 394.

Lisa Chandler, the state's director of assessment, responded to questions about the missing list via e-mail. She said the state didn't obtain the list of schools Caveon considered suspicious because "the list based on the classrooms seemed to be the most useful for districts to use in following up the results."

She cites a section of the Caveon report that suggests the suspicious classroom scores may be a good place to begin investigations. "That is, those schools where exceptions were detected in multiple classrooms might be investigated first," the report states.

But the TEA has already committed to investigate a number of schools regardless of how many classrooms were flagged. Last month, it announced plans to investigate 14 schools on the Caveon list that were also due cash bonuses from the state for their outstanding test scores.

The agency doesn't know whether any other schools on the bonus list might have been flagged by Caveon's schoolwide analysis. And it has no plans to find out which schools are on the list or how egregious their possible cheating might have been.

That means that even schools with what Caveon considers highly suspicious scores won't be identified or investigated.

"The schools that TEA intends to address would be the ones where classrooms were flagged in the list that's been provided," Ms. Marchman said.

E-mail jbenton@dallasnews.com

and hhacker@dallasnews.com

Online at: http://www.dallasnews.com/sharedcontent/dws/news/texassouthwest/stories/072306dnmettakscheat.172ce95.html