By Scott Jaschik | Inside Higher Ed
March 30, 2010
The Advanced Placement program is becoming more and more popular, with 25% of high school graduates taking at least one AP examination, elite colleges expecting to see applicants' transcripts full of the courses, and politicians demanding that more and more high schools offer them. The program has become "the juggernaut of American high school education," according to the introduction to a new book, AP: A Critical Examination of the Advanced Placement Program.
The book, about to be released by Harvard Education Press, is the result of a 2007 conference at Harvard University that brought together leading education researchers to consider the evidence about AP. Despite the immense popularity of the program, the research evidence on its value is minimal, the book argues. The College Board, the program's sponsor, publishes or promotes its own research (favoring the program) and promotes "glowing accounts" of AP. But is this really the consensus?
STATE BY STATE: Failure rate for AP tests climbing
SUCCESS STORY: Maryland makes huge strides in AP scores
The research presented in the book — believed to be the most comprehensive gathering of studies to date — is decidedly mixed. Those who distrust AP out of a dislike of programs built around standardized tests won't find smoking guns about the test itself here. In fact, the research in the book suggests high levels of consistency and validity in the program's exams.
Further, the research studies in the book suggest that for most well-prepared students (and, increasingly, it is important to note that not all students in the program are well-prepared), AP tends to offer rigorous courses that advance their knowledge of the subject matter in meaningful ways. (One caution that the book notes about its findings is that they focus on the mathematics and science parts of AP, which have been the subject of more long-term study.)
But proponents of the program's rapid expansion come in for some tough criticism in the book. Claims that the program helps students graduate on time or save money are found generally to have no validity. And research in the book suggests that many of the efforts to push the program into more schools — a push that has been financed with many millions in state and federal funds — may be paying for poorly-prepared students to fail courses they shouldn't be taking in the first place. And the research suggests that not only is the money being misspent, but that the push may be skewing the decisions of low-income high schools that make adjustments to bring the program in — while being unable to afford improvements in other programs.
The book is frank throughout on the difficulties of conducting research on the program. High schools can't create control groups by giving some students AP courses and other students some sort of placebo version. And the student body for the courses is anything but random. Even with the growth in recent years, the AP courses are much more widely available in high schools that serve wealthier, well-prepared students. And students themselves opt to take or not take the courses, with self-selection drawing in better students.
As a result, several of the studies in the book say, while the College Board's claims that students who do well in the courses are likely to do well in college are true, that means very little. It's sort of like saying that students who are well-prepared to do college work and come from the socioeconomic groups that do the best in college are going to do well in college. Not a shocker.
But, using a variety of statistical tools, the various papers attempt to overcome the limitations in evaluating the program.
The analyses of the programs themselves suggest that AP students get a strong experience in high schools with good resources and high levels of student preparation. In part, this is because of the characteristics of the courses that need not be restricted to the AP program — factors like smaller class size and top teachers. But the verdict on such courses in a series of studies is favorable.
'Very rigorous'
As summed up by Philip Sadler, one of the co-editors of the volume and the F.W. Wright Senior Lecturer in Astronomy at Harvard, here is the outlook for students: "Advanced Placement courses offer you an opportunity to study a subject in a very rigorous and demanding fashion. You will probably be in a class that has fewer students, those students will likely have stronger backgrounds, and there will be fewer student discipline issues than you have experienced in other courses. Your teacher will have a strong subject matter background and excellent teaching skills."
Similarly, Sadler sums up the situation from the perspective of college admissions officers, and tells them that they are correct to assume that success in the courses is a meaningful measure of academic achievements (although he is quick to add that there are equally valid measures, such as success in other rigorous high school courses and dual enrollment programs in which high school students simultaneously take college courses).
But where Sadler's summary will challenge the College Board and others is in his description of the emphasis on expanding the program to serve more and more high schools. While the College Board shows no sign of pulling back from that goal, Sadler writes that the research evidence suggests that the growth has reached a point of "diminishing returns."
That conclusion is consistent with journalistic investigations of the program in the last year, in USA Today and The Dallas Morning News, documenting increased failure rates on the program's tests. In both cases, the articles focused on an increase in the number of students failing the program's tests.
The College Board largely dismissed the findings of both of those investigations, saying that students benefit even if they don't pass and that the many benefits of the program show it deserves even more growth. But research in the new book by Kristin Klopfenstein of the University of Texas at Dallas and M. Kathleen Thomas of Mississippi State University raises questions about those premises.
In one chapter, Klopfenstein and Thomas examine whether states should push — as the College Board has urged them to do — to have AP courses in all high schools, and to add to offerings whenever they can. They note assertions by Gaston Caperton, president of the College Board, that increasing the number of students who take courses in the program would encourage more students to go to college and improve their graduation rates once enrolled.
Klopfenstein and Thomas conclude that there is no evidence to back up these claims. They write that the flaw in the College Board's analysis is its failure to focus on "causation." When the researchers control for a variety of factors related to college-going and college success (such as student demographics, other courses taken, etc.), they find, the boost allegedly provided by the AP courses evaporates. "AP students are college-bound to begin with," they write. While taking AP courses may make some students more attractive to competitive colleges, they add, that is more of a benefit to the student than to society, raising questions about spending a lot of money to expand the program.
And in fact, they note a range of real costs to school districts for the program — costs that may be of particular concern in low-income districts. The program's courses almost always have smaller class sizes and some of the best teachers in a school. While the College Board boasts about the benefits for the students in the program, these same facts mean that the rest of a school's students have larger classes and less time with the best teachers.
Financing failure
That all might be justified if those in the program were achieving great success. But with the expansion, that's not the case — and there are large sums of money being spent on students who don't succeed, they write.
In California, they note, $2.8 million in federal funds are used to subsidize AP tests for students. With 41% of students failing, that means at least $1.1 million was spent on such students — with the number likely much higher since the failure rate and need for subsidies is greater for low-income students.
Similarly, William Lichten, a professor emeritus of physics, engineering and applied science at Yale University, writes about the growth of AP in the Philadelphia public schools — an effort supported by a variety of public funds. He writes that the success stories in the district come from high schools that, while serving low-income students, use selective admissions. Among other high schools, failure rates are high. "Many of these high schools did not have a single AP exam score as high as 3, the lowest score claimed as passing by the College Board," Lichten writes.
In some of the strongest criticism of the College Board in the book, Lichten questions why the College Board does not publicize the success rates of students school by school, in ways that might raise questions about the significant investments made in the program at schools where budgets are very much a zero-sum game and the program is showing no signs of success.
In an interview, Trevor Packer, who leads AP for the College Board, generally offered praise for the new book. He said that it was "tremendously stimulating and meaningful," and that the editors and contributors had performed a real service by significantly expanding the available research on the program and synthesizing it in the new volume.
Generally, he said he views the book as saying that "AP is doing a good job of performing its original purpose" of providing rigorous courses and of helping to place students at appropriate levels as they start their college careers. He said he viewed the questions raised in the book as less about those functions than about "uses of AP that have emerged, particularly over the last decade."
The research in the book "shows that AP does a good job of doing what it set out to do," he said.
As to the criticisms in the book, Packer said he saw it focused on the "boosterism" about AP, and said, "I think there is something about the boosterism that I stand behind and some boosterism that I find flaws with."
He said that he has tried himself and has urged others to talk about AP not as allowing many students to finish college early, but as a tool for giving them more "flexibility" in designing their curriculum. (In fairness to Packer, he is less boosterish in the various College Board briefings about AP than is his boss, and than some government officials who want the program expanded.)
On the larger questions about the program's growth, he said he agreed with the idea that there should be more study of the benefits of the program as it moves into more schools, including those that — based on socioeconomics — don't do well. But he said he was troubled by the idea that the program may already be at "diminishing returns" because he still sees many high schools where the socioeconomics and educational background suggest that success would occur, but that don't have the program or have only limited offerings.
The College Board has generally argued in the past that students benefit even at schools with low passage rates. But Packer, while insisting that there was much success in low-income districts, also said: "In schools where they are seeing all scores of 1, I would agree wholeheartedly that resources might be better spent on middle schools."
At the same time, he said that reaching more low-income students who can succeed will mean some increases in failure rates. As a result, he said it is hard to tell just how much the program should grow. "What is the right threshold that AP should reach? What is the right [rate at which students succeed] of threshold?" he asked. "That's a really good question."
No comments:
Post a Comment