New Study Finds Gains Since No Child Left Behind

Hmmm. These results are questionable. "Increases in achievement recorded by many state tests had not been matched by results of the National Assessment of Educational Progress, nationwide reading and math tests administered by the federal Department of Education." Read on.

-Angela

June 6, 2007
New Study Finds Gains Since No Child Left Behind

By SAM DILLON
Student achievement has increased and test score gaps between white students and black and Hispanic students have narrowed in many states since President Bush signed the No Child Left Behind law in 2002, according to a new survey of state scores in reading and math.

But the study, released yesterday by the Center on Education Policy, an independent Washington group that closely monitors the law, cautioned that “it is difficult if not impossible to determine the extent to which these trends in test results have occurred because of N.C.L.B.”

“In most states with three or more years of comparable test data, student achievement in reading and math has gone up since 2002,” the study found, even as it warned repeatedly against concluding that the federal law alone produced the results.

In the decade before the law was passed, many states had adopted policies aimed at raising achievement, like broadening access to early childhood programs, that could also be responsible for gains.

The study also acknowledged that the increases in achievement recorded by many state tests had not been matched by results of the National Assessment of Educational Progress, nationwide reading and math tests administered by the federal Department of Education.

Those results have been mixed. For example, on the national tests given in 2005, fourth-grade math scores showed an important increase over the previous test administration in 2003, and eighth-grade math scores rose slightly. But fourth-grade reading scores were the same on the nationwide test in 2005 as in 2002, and eighth-grade reading scores declined.

Despite its caveats, the new report is likely to be closely studied as Congress debates whether to reauthorize the law this year, partly because the report may be the most comprehensive study of state test scores in many years. The law is widely considered President Bush’s most important domestic policy achievement.

“This study confirms that No Child Left Behind has struck a chord of success with our nation’s schools and students,” Education Secretary Margaret Spellings said yesterday.

Senator Edward M. Kennedy, Democrat of Massachusetts, who worked closely with the administration on the law, said the report “proves that the goals at the heart of the No Child Left Behind Act — a dedication to accountability and standards, a commitment to closing the achievement gap and a pledge to improve public schools — are still the right ones for moving America forward.”

Merely collecting the test data from 50 states proved to be a complex and frustrating task because many states’ education departments are overworked and their test archives are flawed by missing or inconsistent data, the report said. “The house of data on which N.C.L.B. is built is at times a rickety structure,” it said.

Those and other limitations notwithstanding, Jack Jennings, the center’s president, said state test scores “remain a more accurate barometer of what kids know” than the national assessment, often referred to as the NAEP (pronounced nape).

“The NAEP shouldn’t be taken as the gold standard,” Mr. Jennings said.

Bruce Fuller, an education professor at the University of California at Berkeley who has compared state and federal achievement scores, said the report “displays methodological weaknesses which lead to exaggerated inferences” about student progress.

In analyzing state scores, the researchers who carried out the study did not consider all recent data from all states because, the report said, new tests and other factors in some states made it impossible to compare scores from one year with others. But Professor Fuller said the researchers appeared to have eliminated testing periods in some states that showed predominantly falling scores after 2002.

“It’s like calculating the annual rate of economic growth over the past century after excluding the Great Depression years,” Professor Fuller said. “It upwardly biases their estimate of annual growth in test scores.”

Robert L. Linn, an education professor emeritus at the University of Colorado at Boulder and a frequent critic of the law, served on a panel of five prominent testing and educational policy experts who advised the center on the study.

“I was a little surprised that things were generally as positive as they were, so it may be that I would say that N.C.L.B. is contributing more positively than I had given it credit for,” Professor Linn said. But he urged readers to pay attention to the report’s many caveats.

“The reason for all the caveats is that it is impossible to reach the conclusion that if scores go up, it is because of N.C.L.B.,” he said. “There are so many other factors that could lead to rising scores, including state efforts to raise achievement, and also, some of these gains may be artificial. So my worry is that people who come at it and don’t read the caveats will come away with an exaggerated impression.”

Laura S. Hamilton, a senior behavioral scientist for the Rand Corporation who also served on the panel of experts, said, “Most people want to know if N.C.L.B. as a policy has resulted in improved student achievement,” but added, “It’s a question that isn’t answerable.” She explained, “To test whether some policy is effective, you’d want to compare what happened under that policy to what would have happened if the policy hadn’t been enacted, and we can’t do that with N.C.L.B. because all public schools in the nation were subject to its provisions.”


Copyright 2007 The New York Times Company