Focus on ratings discouraged

Good, critical piece on school ratings. This piece should be considered of the more positive rendering of test score (NAEP) data in a recent issue of The Economist in a piece titled No Lessons Left Behind, as well as in a USA Today piece titled Now for the Good News.

It all boils down to what test scores themselves mean. The differences in perspectives are worth considering though. -Angela

Mon, Aug. 01, 2005

Focus on ratings discouraged
By John Austin and L. Lamor Williams
FORT WORTH STAR-TELEGRAM STAFF WRITERS

The scores are in, the ratings are tabulated, and everyone in Texas will know today just how good a school his or her child attends, thanks to the state's latest research and rankings.

At least, that's the theory.

But parents, students and administrators need to look at a lot more than the ratings to really know the score, according to several educational experts.

"In general, tests cover only a very small part of schools' academic curriculum, much less the broader non-academic aims of public education," Walt Haney, an education professor at Boston College, said in an e-mail. "And even on tests of academic subjects such as English and math, schools' grade-level results from year to year may vary.

"As any teacher knows, one year's class of 30 or 40 kids may vary substantially from the next year's just because of Mother Nature's variation in which kids are born in which year," said Haney, who is also a senior staff member of the Center for the Study of Testing, Evaluation and Educational Policy in Boston.

Districts are assigned ratings based largely on how students perform on the Texas Assessment of Knowledge and Skills. Each student group -- black, white, Hispanic and economically disadvantaged -- must pass the exam, which covers reading, math, science, social studies, writing and English/language arts. The state also considers the annual dropout rate of seventh- and eighth-graders and the completion rate of high school students.

In addition to these ratings, the state tracks student progress with TAKS scores from each administration of the exam. The federal government also labels schools according to whether they make adequate yearly progress.

Some area administrators caution that ratings by themselves tell only how students did on a particular day and that a handful of children having bad days can drag down an entire district's rating. Many schools have had their ratings drop as a result of one student's performance on one portion of the TAKS.

"It's one of the many pieces of information that people should use, but it can also distort the picture of what really goes on in a school," said Arlington Superintendent Mac Bernd.

They liken the system to giving students, who study several subjects, only one grade on their report cards: A child who just couldn't do well in gym class but made A's in every other subject would get a B on his report card.

The system is a measure of the lowest performance, not the highest, according to educators.

Administrators say it's difficult to compare districts solely on ratings.

For example, a predominantly white and wealthy district may just barely earn the top rating, while a low-income district with many minority students -- many who aren't native English speakers -- may do a better job of teaching but still be rated lower.

Haney blames politicians and the press, who, he wrote, "wildly over-promote such rankings" for some of the public's ratings fixation.

"Another reason is, many, if not most people, have a very poor understanding of statistics and have little inkling of how much error is associated with numerical rankings," he said.

Robert Schaeffer, public education director for the National Center for Fair & Open Testing, is also critical of what he considers a misguided emphasis on ratings.

"High school ratings affect real estate prices," because parents often attempt to buy into districts they perceive to be better than others, Schaeffer said.

"And to some extent there's bragging rights.

"It becomes a positive feedback loop in which good schools appear to get better. It drives people to judge schools based on a simple-minded set of factors on which the ratings are based."

Like Haney, Schaeffer pointed out that the data that ratings are based on is sometimes "phony."

"The ratings are essentially useless in many cases," Schaeffer said.

Economist Steven Levitt and author Stephen Dubner say in their best-selling book Freakonomics that moving to a better neighborhood doesn't improve a child's chances in school.

And Schaeffer said that while a highly rated school may be doing a great job, that school may also have a student body made up of upper-middle-class kids whose parents are well-educated and who reinforce and supplement what goes on in class.

"It's called cherry-picking," he said.

At a school where students come from a lower socioeconomic group and where parents may not be well-educated, or even speak English, teachers may actually be doing a better job than those elsewhere, even though the ratings don't reflect their gains: They simply have further to go.

Schaeffer also urges parents to be skeptical about claims that TAKS scores and other test results reflect big gains for Texas students.

"Your colleges [in Texas] report more kids needing more remediation," he said. "Now how is that?"

Anne Ware, coordinator of program evaluation for the Fort Worth school district, agreed that ratings are only part of the picture.

"You can't just determine success or failure by one item," Ware said.

"But the test scores make the headlines."

She too urged parents to get involved, even though it can be difficult.

"If the parents start demanding more, schools will do more," she said.

Above all, remember that even in a school that is not rated "exemplary," there may be "extraordinary people doing great work," Schaeffer said.

IN THE KNOW

About the ratings

To assess a school's real worth, parents are advised to:

• Consider SAT scores: They help indicate whether teachers have been "teaching to the (TAKS) test" to the exclusion of broader goals.

• Consider "real" dropout rates. Use this formula: For a given graduating class, add the number of students in the class as freshmen and the difference between transfers in and out over the next four years. Then divide the number of graduates by that number. Texas does not use this formula.

• Consider the percentage of graduates who go to college or the percentage who are employed after high school.

• Go to school several times during the year, not just on open-house night. Sit in on classes. Go to the library. Find out what's in it and whether students are using it.

• See what kind of work students are doing. Are they writing more than the few paragraphs that many standardized tests require?

• Check for art and music programs.

• Talk to other parents.

Testing resources

•  Accountability ratings will be released at 1 p.m. today

•  National Center for Fair & Open Testing

• Center for the Study of Testing, Evaluation and Educational Policy
------------------------------------------------------------------------
John Austin, (817) 548-5418 jaustin@star-telegram.com L. Lamor Williams,
(817) 548-5494 llwilliams@star-telegram.com

------------------------------------------------------------------------
© 2005 Star-Telegram and wire service sources. All Rights Reserved.
http://www.dfw.com