Smoke and Miracles: Who Really Benefits in the Accountability System?

Note: I don't have a date for this.-Angela

Who Really Benefits in the Accountability System?

by Emily Pyle

AUSTIN CHRONICLE
Since her appointment in January, Shirley Neeley, Governor Rick Perry‚s new education commissioner, has made dozens of public appearances. Her main task is to stump for proposals of Perry‚s that could come up if and when the governor calls a school finance special session later this spring. Since Perry is thinking big-his proposals include vouchers (or some other kind of “school choice “ program) as well as relaxed certification standards for teachers and pay incentives based largely on standardized test scores-Neeley will need all her considerable charm to sell his initiatives to a skeptical education community. So far, she‚s taken the task on with the pep of a cheerleader. And the governor, in return, has been a cheerleader for Neeley.

In particular, Perry has dwelt on Neeley‚s achievements as superintendent of Galena Park ISD. In nine years, in a district that is mostly minority and poor, Neeley almost doubled her students‚ scores on the Texas Assessment of Academic Skills (TAAS) exam that was the center of the state‚s accountability system. (The TAAS was replaced by the more difficult Texas Assessment of Knowledge and Skills (TAKS) exam last spring.) As told by the governor‚s press office, Neeley‚s story is a fable for the accountability age. And indeed it is, but the deeper you look into the numbers behind it, the more troubling this fable‚s moral becomes.

Neeley is more than just a missionary for Texas‚ accountability system. She has been one of its biggest beneficiaries and an emblem of the state‚s educational trends. While the test has always meant high stakes for kids, who are promoted or held back based on their scores, those risks may soon get much higher for everyone in the education community. In the possible school finance special session to come, the state is poised to make test scores the basis for everything from teacher salaries to a district‚s share of state funding.

Critics of high-stakes testing allege that intense focus on a single test drains time and money away from the actual business of teaching and learning. In the end, it cheats even the high-scoring kids out of a real education. And in our new commissioner‚s old district, we see that principle at work. While scores statewide have climbed every year since the testing system first was instituted in 1992, in Galena Park, as in the state as a whole, other indicators of student achievement have not kept pace.

Neeley herself went to school in the Galena Park district along the Houston ship channel. Since grade school, she had felt called to teach, delivering her first lessons in her mother‚s backyard, to a class of dolls. After college at Texas A&M University, she returned to serve in the district as teacher, assistant principal, principal, and central administrator. She was named superintendent in 1995, at a time when the demographics of the area were changing fast, forcing the district to absorb an influx of students from poor and minority families, many of whom barely spoke English.

Just weeks after Neeley accepted the superintendent position, the Texas Education Agency (TEA) released the year‚s accountability ratings. Based on its TAAS scores, Galena Park ISD was rated “acceptable “˜the C- of the state‚s accountability system. Galena Park High School, Neeley‚s own alma mater, was rated “low-performing “ for its rock-bottom test scores and abysmal dropout rate˜nearly one student in ten left the school without graduating. Neeley was mortified. She solemnly promised her district board of trustees that the district would never have another low-performing school. If it did, they should fire her.

Then she set about bringing the TAAS scores up. First Neeley hired a team of consultants from Ohio State University to audit the district and determine its weak points. After a yearlong study the consultants issued their findings, ripping apart everything from the lecture style of the district‚s teachers to the lack of grievance counselors available to address employee complaints. But most of all, they emphasized the district‚s lack of “alignment “˜that is, the extent to which classroom teaching lined up with the content and format of the TAAS test. “They told us we weren‚t teaching the same stuff we were testing, “ Neeley says today.

So the district, under Neeley‚s leadership, set out to do what TAAS critics identify as the ultimate sin of standardized testing˜they began teaching to the test. At Neeley‚s prompting, teachers devoted the first 10 minutes of every class, in every grade, for every subject˜including band, art, and PE˜to TAAS exercises. Special test prep classes replaced extracurricular sessions for students who failed the TAAS on their first attempt. Even the birthday cards Neeley sent to every teacher and principal in the district read “Think Exemplary. “ Neeley also issued an ultimatum: Principals had three years to win their schools an accountability rating of at least “recognized “˜awarded when 80 percent of students in all racial and economic groups passed all parts of the TAAS. Principals who didn‚t meet the deadline were demoted or fired.

It worked. Over the nine years of Neeley‚s tenure, in
a fast-growing district that added more poor and minority students every year, the percentage of students passing the TAAS almost doubled for students in all ethnic and economic groups. Last spring, 95 percent of Galena Park‚s high school seniors passed the exit-level TAAS exam, and the achievement gaps between ethnic groups were negligible. Today, Galena Park is the largest district in the state to receive a TEA rating of “exemplary. “

“I am asking Dr. Neeley to do on the state level what she has done in Galena Park, which is to create a culture of educational excellence, and a focus on educational efficiency, so more students graduate from high school prepared for college and success in life, “ Perry said in a press release announcing Neeley‚s appointment to the commissionership.

The gains Neeley‚s students made on the TAAS are real. It‚s doubtful, however, that higher TAAS scores alone prepare kids as well as our governor seems to believe. Throughout Neeley‚s term as superintendent, the district‚s scores on the Scholastic Aptitude Test (SAT) and ACT˜the tests by which college admission boards decide whether a student is “prepared for college and success in life “˜slumped downward.

While the TAAS and TAKS measure how much of the tested material each student knows, the SAT compares every student‚s performance against his or her peers. Compared to their peers across the nation and within the state, Galena Park‚s high school students have lost ground. In 1996, the year after Neeley became superintendent and the last time the SAT was recalibrated, the average SAT score in the district was 1016˜beating out the regional average of 1006, and the state average of 993. Scores dropped across the whole state in the intervening years, but Galena Park‚s scores fell more than most. By 2002, the district‚s average SAT score had dropped to 885, while the regional average was at 1000 and the state average at 986.

Neeley attributes the low scores on the SAT and ACT to the larger number of Galena Park students who take the test these days. However, the same percentage of students took the test in 1996 as in 2002. While the percentage of students taking the test did rise and fall during the nineties, scores on the exams dropped almost continuously. One reason Galena Park‚s scores remain below state and regional averages may be because the district has more minority students˜who typically score lower on the two exams˜and fewer white students, who typically score higher. While this could help explain Galena Park‚s low scores, it begs the question of why˜if TAAS and TAKS scores today show so little disparity between ethnic groups˜the district‚s minority students are still less prepared for these critical college entrance exams than their white peers.

Galena Park‚s SAT scores reveal a troubling achievement gap between racial groups that doesn‚t show up immediately in the district‚s TAAS and TAKS scores. In 1996, Hispanic students had an average SAT score of 960; their average score has fallen almost continuously since. In 2002, the average score was 841. The average score of Galena Park‚s black students drifted down from 950 in 1996 to 854 in 2002. The district‚s white students lost the least ground, going from an average score of 1094 in 1996 to 1008 in 2002. And while scores have been in a slide for the whole state, Galena Park student scores are dropping faster than most. In 1996, all the district‚s student groups outscored their peers statewide by 50 points or more. In 2002, only Galena Park‚s black students scored higher˜by 15 points˜than their peers state-wide. (In 1996, they outscored their state peers by almost 100 points.)

Racial disparities also persist within the very TAAS and TAKS scores, if you look at the state‚s numbers closely. The “Texas Academic Skills Program (TASP) equivalency “ score on the TAAS exit exam is used to estimate a student‚s chances of performing well on the TASP˜the exam required for admission to any Texas public university. By this measure, a higher percentage of Galena Park students˜as a whole and across every ethnic group˜were likely to pass the TASP in 2002 than in 1995. Yet the district‚s TASP equivalency scores lagged behind state and regional averages. And Hispanic students, by their TAAS exit exam scores, were about 40 percent less likely than their white peers to score well on the TASP. Black students were about 30 percent less likely to score well.

On a good note for Galena Park, the gaps between ethnic groups are lower than the gaps for the state as a whole. Indeed, the lesson to be learned from examining Galena Park‚s scores is not that the district is worse than most districts in the state˜it is probably better than many. The lesson here is about the essential emptiness of the accountability system‚s promise.

High test scores may mean something for principals, like those in Neeley‚s district, who must produce them to keep their jobs. They may soon mean something to teachers if the state adopts a “performance “ based incentive program like the one the governor, with Neeley‚s endorsement, proposes. But high scores on the TAKS are all but meaningless for the students who earn them, however hard they may have worked, however many of their band practices and gym classes may have been spent on test prep exercises. High accountability scores in a district do not automatically translate to equivalent gains on college entrance exams. High scores on the TAKS don‚t guarantee college admission. Without that guarantee, it‚s not obvious how an ability to sit still and fill in a bubble sheet prepares one for “success in life “ (unless they are being trained to be drones in a service economy).

Neeley is a true believer in accountability. To her mind, the testing system provides a constant monitor of students‚ progress, keeping them on track from kindergarten to graduation. As superintendent, she threw herself whole-heartedly behind the high-stakes testing system, bringing an all-consuming focus to the only measure of “performance “ that the accountability system counts. And she did improve it, unequivocally. The only shame is that the state didn‚t offer her a challenge with more substance, one that might have meant more for the children whose education was her charge. If Neeley brings her single-minded focus on testing to her new office, we may one day see every child in the state pass the TAKS exam. The question is, so what?

Emily Pyle is a writer based in Austin.