Tue, Feb. 08, 2005
CHARLOTTE OBSERVER EXCLUSIVE
N.C. Hires Company to Inspect School Test Data
By PETER SMOLOWITZ
With high-stakes standardized tests now determining school rankings, teacher bonuses and federal aid, pressure is mounting for schools to perform. Which helps explain why a growing number of schools nationwide have been caught cheating. On Monday, North Carolina signed a $60,000 contract for a 2-year-old company called Caveon to analyze test data and search for red flags. North Carolina and South Carolina are among just three states to hire the Utah-based firm, believed to be the first of its kind. Delaware has also hired the company, and a co-founder said they're negotiating with about a dozen other states. Most Carolinas education leaders do not suspect rampant cheating, though allegations have arisen about teachers having access to questions in advance. In Indiana, a teacher was suspended after being accused of pointing out wrong answers while third-grade students were taking a test. Some argue the hiring alone will scare cheaters, the same way burglars avoid homes with yard signs warning of alarm systems. "There are people who put those signs in their yards who don't even have those systems, because they know it is a deterrent," said N.C. testing director Lou Fabrizio, who said the timing coincides with a review of the state's tests. "We're just trying to be as proactive and as comprehensive as we can." Caveon uses a process called data forensics to look for unusual patterns: Kids answering hard questions correctly and missing easy ones. An abnormally high pass rate in one class. Tests with several wrong answers erased and replaced with the right ones. "One of the things I'm looking for is evidence of coaching or proxy test-taking," said Caveon chief scientist Dennis Maynes. "In an educational setting, the greatest concern is (that) the administration and teachers are actually doing the cheating, not the students." How widespread is cheating? There's no way to know how common cheating is, but Caveon and other analysts estimate it could occur with as many as 10 percent of school tests nationwide.An Arizona State University professor said he conducted an anonymous survey in the late 1980s that showed 10 percent of Arizona teachers admitted cheating to help boost students' scores. And those tests didn't count nearly as much as today's, said Tom Haladyna. "It's amazing," said Haladyna, an education psychology professor who studies test score accuracy. "And these are only things we know about. It's like an iceberg." Within the past year, cheating allegations nationwide have prompted officials in Texas, Indiana, Mississippi and Arizona to launch investigations, suspend staff or throw out scores. And recently questions have been raised in the Carolinas. In February 2003, the Guilford County School District disciplined several employees suspected of sharing state standardized test questions with high school students in advance. Prosecutors in Beaufort County, S.C., are investigating claims that in 2003, middle school administrators showed state tests to teachers in advance, fueling fears the instructors could have coached kids on actual questions. The Carolinas have stringent safeguards that Haladyna said were tougher than those of several states. Rules require or recommend that two adults be present during all tests. Officials make unannounced visits to testing sites. And teachers not only get trained in the state's code of ethics, some schools require teachers, proctors, principals and test coordinators to sign forms agreeing to comply or swearing that they have. But the states still wanted Caveon to analyze answers and check for weaknesses in their test-security system. "In order for the test scores to be valid and reflect what the students know and are able to do, then test security must be maintained," said S.C. testing director Teri Siskind. "It's an equity, fairness and validity issue, in addition to an ethical issue." Maintaining integrity Caveon comes from the Latin word "caveo," meaning "safeguard." The company was founded in 2003 by a dozen people who had helped write and design tests, but worried about security as the Internet made it easier for people to publish answers in advance. Those fears could cloud the reputation of companies and workers. "Everybody loses," said co-founder Jerry Christensen. "There is this weakness there, and nobody else has really looked at it." Caveon has worked with about 25 companies that offer entrance exams or certification tests, Christensen said. Caveon workers have helped businesses make it tougher for people to steal questions, and they have caught people selling questions on the Web. In June, they started working with schools. It's unlikely they'll find much wrongdoing in the Carolinas, said Colby Cochran, Rowan-Salisbury's testing director, who helped revise the testing code of ethics. But they will be checking things local districts don't typically examine, and that will help make the scores "beyond reproach." "This is the era of accountability," Cochran said. "Somebody else needs to look at you from time to time to see if you are really doing what you say you are doing."
Peter Smolowitz: (704) 358-5249; firstname.lastname@example.org
© 2005 Charlotte Observer and wire service sources. All Rights Reserved.