Getting Accountability Right

Also check out this policy brief co-authored by Rothstein entitled "Education Accountability Policy in the New Administration"

-Patricia

Commentary
By Richard Rothstein | Ed Week
January 28, 2009

The federal No Child Left Behind Act has succeeded in highlighting the poor math and reading skills of disadvantaged children. But on balance, the law has done more harm than good because it has terribly distorted the school curriculum. Modest modifications cannot correct this distortion. Designing a better accountability policy will take time. We cannot and should not abandon school accountability, but it's time to go back to the drawing board to get accountability right.

The first step is to understand today's curricular distortion. It has arisen because No Child Left Behind holds schools accountable for only some of their many goals. When we demand adequate math and reading scores alone, educators rationally respond by transferring resources to math and reading instruction (and drill) from social studies, history, science, the arts and music, character development, citizenship education, emotional and physical health, and physical fitness.

This shift has been most severe for the disadvantaged children the law was designed to help, because they are most at risk of failing to meet the math and reading targets. But they are also most at risk of losing curricular opportunities in other domains. In these other areas, NCLB has widened the "achievement gap."

President Barack Obama has vowed to correct this distortion. He has noted that NCLB "has become so reliant on a standardized-test model that ... subjects like history and social studies have gotten pushed aside. Arts and music time is no longer there. So the child is not having the well-rounded educational experience I benefited from and most in my generation benefited from." We must change No Child Left Behind, he has said, "so that the assessment is one that takes into account all the factors that go into a good education."

Although some Democrats and Republicans want to ignore the law's goal distortion, observers with varying policy perspectives share the new president's view that NCLB requires a radical reconsideration. The Center on Education Policy, headed by Jack Jennings (formerly an aide to Democrats on the House education committee), has publicized the loss of instruction in social studies, science, the arts, and physical education, especially for disadvantaged children. Chester E. Finn Jr. and Diane Ravitch, who served as federal education officials in Republican administrations, complain that present policy means only "top private schools and a few suburban systems will stick with education broadly defined." While rich kids study a wide range of subjects in depth, they write, "their poor peers fill in bubbles on test sheets." There is a "zero sum" problem, Finn and Ravitch say, because "more emphasis on some things ... inevitably mean[s] less attention to others."

Yet public discussion of the law's upcoming reauthorization focuses almost entirely on correcting flaws in math and reading measurement: substituting "growth models" for fixed levels, modifying the 2014 deadline for attaining student proficiency, standardizing state definitions of proficiency, modifying "confidence intervals" in reporting. While these steps may improve the sophistication of math and reading data, none addresses the goal distortion caused by exclusive accountability for basic skills.

Designing accountability tools that require satisfactory performance across a balanced set of outcomes requires a significant federal research-and-development effort, which could build on prior experience. When the National Assessment of Educational Progress was developed in the 1960s, it measured a broad range of cognitive and noncognitive knowledge and skills. NAEP abandoned that breadth when its budget was slashed in the 1970s, however, and never restored it.

To see whether students learned to cooperate, for example, the early NAEP program sent trained observers to sampled schools. In teams of four, 9-year-olds were offered prizes (such as yo-yos) for guessing what object was hidden in a box. Students could ask yes-or-no questions, but all team members had to agree on each question asked. NAEP rated the students on whether they suggested new questions, gave reasons for viewpoints, or otherwise demonstrated cooperative problem-solving skills. It then reported to the nation on the percentage of children capable of cooperative problem-solving.

For teenagers, NAEP assessors provided lists of issues about which young people typically had strong opinions. Students had to collaborate in writing recommendations to resolve them. For 13-year-olds, lists included topics such as whether they should have curfews for getting home, and for 17-year-olds, the age eligibility for voting, drinking, or smoking. NAEP rated students on whether they took clear positions, gave reasons for viewpoints, helped organize internal procedures, and defended another's right to disagree.

Early NAEP understood that teaching civic responsibility involved more than having students memorize historical facts. So in 1969, during the era of the civil rights revolution, the assessment asked teenagers what they felt they should do if they saw black children barred from entering a park. NAEP reported that 82 percent of 13-year-olds and 90 percent of 17-year-olds knew that they should do something constructive, such as tell parents, report it to a civil rights or civil liberties organization, write letters to the newspaper, or take social action such as picketing or leafleting.

The early version of NAEP also assessed 17-year-olds' ability to consider alternative viewpoints, by asking them to state arguments both for and against a heated public issue of the time, such as whether college students should be drafted. It asked 9- and 13-year-olds if something reported in a newspaper might be untrue. It also asked teenagers if they belonged to any nonschool clubs or organizations; interviewers followed up with questions to verify answers' accuracy.

To assess commitment to civil liberties, NAEP asked teenagers if someone should be permitted to say on television that "Russia is better than the United States," that "some races of people are better than others," or that "it is not necessary to believe in God." The assessment reported the discouraging result that only a small minority of the teenagers thought all three statements should be permitted.

The early NAEP program also assessed personal responsibility. Seventeen-year-olds were asked what to do if, when visiting a friend, they noticed her 6-month-old baby was bruised. The correct answer was "suggest that your friend call her baby's doctor." Incorrect choices included "ignore the bruises because they are none of your business." A follow-up prompt said that at a later visit, bruises remain and "you are now suspicious that your friend may have hurt the baby." Students were asked what to do now. The correct choice was "call the local child-health agency and report your suspicions."

Certainly, if school systems were evaluated by such results, not simply by math and reading scores, incentives would shift. National reporting of low scores on the civil liberties questions, for example, could spur demands that schools do a better job on citizenship; then, the incentive to drop cooperative learning in favor of test prep in math and reading would diminish.

Designing a new accountability system will take time and care, because the problems are daunting. Observations of student behavior are not as reliable as standardized tests of basic skills, so we will have to accept that it is better to imperfectly measure a broad set of outcomes than to perfectly measure a narrow set. We will have to resolve contradictory national convictions that schools should teach citizenship and character, but not inquire about students' (and parents') personal opinions. To avoid new distortions, we'll need to make tough decisions about how to weight the measurement of the many goals of education.

The time to start on these difficult tasks is now, but the new administration won't have to begin with a blank slate. Looking back at the early National Assessment of Educational Progress can start us on a better path.


Richard Rothstein (riroth@epi.org) is a research associate of the Economic Policy Institute. This article summarizes an argument from his recent book, co-written with Rebecca Jacobsen and Tamara Wilder, Grading Education: Getting Accountability Right (Teachers College Press).