Missing: 20,000 standardized tests

I'm certain that if we were talking about the SAT that the apparent lack of concern by officials over lost tests would be appalling. With so much riding on TAKS scores, it's not unreasonable for questions about these tests to be raised. -Angela

Tuesday, November 8, 2005

Missing: 20,000 standardized tests
Of statewide total, DISD leads in exams lost, mostly TAKS

By JOSHUA BENTON / The Dallas Morning News

More than 20,000 copies of state tests – supposedly kept under lock and key – disappeared from Texas schools this spring, according to state data. Dallas schools lost more than 7,000 test documents, more than any other district in the state.

State officials say they are reconsidering their testing security policies after some experts said having Texas Assessment of Knowledge and Skills, or TAKS, questions floating around the state could put the integrity of the testing system at risk.

"We probably need to look at some ways to strengthen our security," said Susan Barnes, associate commissioner of standards and programs at the Texas Education Agency.

In all, 26,998 secure test documents from spring 2005 were missing as of Oct. 28. That total includes documents such as test coordinator manuals, but the vast majority are the test booklets.

That's a small percentage of the more than 12 million secure test documents the state produces each year for a range of subjects and grade levels, state officials said.

"We have accounted for 99.78 percent of the documents," said Lisa Chandler, TEA's director of assessment. "We're not concerned by that rate."

But some say loose test documents can be a threat, even in small numbers.

"Every time you lose even one, it's potentially a problem," said John Fremer, former top test developer of the SAT and a founder of Caveon, the Utah company that Texas has hired to analyze its test results for signs of cheating. "Whenever you have a large operation where you send out millions of tests, you're going to have some shortage."

The number of missing tests statewide was first reported by KHOU-TV in Houston on Monday. KHOU is owned by Belo, which also owns The Dallas Morning News.

Dallas schools spokesman Donald Claxton said Monday that he did not yet know any details about the missing test documents. "We're looking into it," he said.

Evelyn Reed, Dallas' director of systemwide testing, did not return a phone message Monday.

Normally, having test booklets unaccounted for would not be a major security concern in Texas. For the last several years, the state has released complete copies of its tests to the public shortly after they were administered. The releases were intended to increase public confidence in the tests and to help educators prepare students.

Recycled questions

But those annual releases meant the state had to rewrite all of its tests each year at substantial cost. As a cost-cutting measure, the Texas Education Agency now plans to release its tests every other year and recycle test questions in between releases.

In other words, the 2006 TAKS test will include questions already used on the 2005 TAKS – which means having loose copies of the 2005 test floating around is not a good thing.

"Anytime you don't have entirely new items every year, you have a greater security risk," Dr. Fremer said.

He said that if too many copies of the test are unaccounted for, a state might have to adjust how many questions it reuses on future tests. State officials said they would not discuss how many questions they will reuse next year.

Each spring, testing documents are shipped to school districts from Pearson, the company that Texas hired to administer its major state tests. The documents are supposed to be stored in a locked location until test day.

When students are finished testing, the answer sheets are shipped immediately back to Pearson for grading. The test booklets are sent back from individual campuses to district officials, who then ship them back to Pearson.

Districts receive more test booklets than they expect to use, Ms. Chandler said. That's in case more students than expected show up on test day. But if schools have more test booklets than they need – and school officials know that missing test booklets will not result in any serious sanction – it could tempt some educators to look at test booklets ahead of time and help students cheat.

In recent years, as allegations of educator-led cheating have hit some Texas schools, several teachers have said that copies of the TAKS test are sometimes circulated around schools before test day.

Dr. Barnes said she doubted that scenario would happen. "We have people signing oaths, and they know that people could report them" if they do something wrong, she said.

But she said that, to her knowledge, no Texas school has been investigated or sanctioned in recent memory for not returning all its test booklets. TEA does not ask schools with large numbers of missing test booklets to explain their disappearance.

According to Texas Education Agency data, 7,084 test booklets from this spring's state testing in DISD have disappeared.

That's many more than other large districts in the state. Houston lost 1,111, Austin lost 436, and Fort Worth lost 384.

Of the lost Dallas test documents, the largest number – 5,989 – were from the state's main exam, the TAKS. The remainder was from other tests, such as the alternative assessment the state gives to special-education students.

Largest disappearance

The state's single largest disappearance of testing documents came in Dallas after the TAKS tests administered on Feb. 22 and 23. Those tests covered reading in grades 3, 5 and 9, writing in grades 4 and 7, and English language arts in grades 10 and 11.

Two of those tests – the reading tests in grades 3 and 5 – carry high stakes for children, because students must generally pass them in order to move on to the next grade.

Of the 64,883 test booklets distributed to DISD for those tests, 5,150 have gone missing, according to state data. That's more than eight times the number of tests to disappear from the next biggest document loss: 627 tests that disappeared from a Houston ISD special-education test session.

Several other area districts also had test documents missing, according to state data. Carrollton-Farmers Branch was missing 100 documents. Plano had 154, Richardson had 136, and Irving had 105.

TEA asked University of North Carolina professor Gregory Cizek to evaluate Texas' test security measures this year, but he did not delve into the issue of lost test documents. As part of his report, he surveyed district testing administrators about their thoughts on weaknesses in the system.

"Tracking secure materials in large districts that have very small staff is like herding cats," one anonymous administrator wrote. "The volume is overwhelming to handle without the chance of something being misplaced. Imagine 63,000 test booklets and three to four people in central trying to keep track of all of it."

E-mail jbenton@dallasnews.com

------------------------------------------------------------------------
Online at: http://www.dallasnews.com/sharedcontent/dws/dn/latestnews/stories/DN-
taks_08met.ART.North.Edition2.17d90b4b.html