Next Stages in Testing Debate

Inside Higher Ed
September 28, 2009

BALTIMORE -- A year ago, the big news at the annual meeting of the National Association for College Admission Counseling was the release of a landmark report questioning the use of standardized testing. While the report didn't call for testing to be abolished, it said that most colleges that required testing in the admissions process didn't have a sufficient sense of its value, and the study suggested that careful analysis would lead many of those institutions to stop requiring the SAT or ACT as part of the admissions process.



In the year since the report was released, there has been a steady stream (but not a wave) of movement away from testing requirements. Just in the week before the meeting, Sacred Heart University, the State University of New York at Potsdam and Washington and Jefferson College dropped SAT requirements. In the last year, new forms of going test-optional have also appeared. Some colleges -- such as American University and the State University of New York at Geneseo -- have gone test-optional for early decision applicants. Other institutions, such as New York University and Bryn Mawr College, moved in the last year to allow the use of SAT II (subject tests) or Advanced Placement tests instead of the SAT.

Here at this year's NACAC meeting, the association moved to carry out some of the recommendations of its report -- and also highlighted the successes that colleges have had after adjusting their testing requirements. While the various experiments highlighted had different themes, a common idea was that decreasing reliance on the SAT does not mean any loss of academic rigor and can in fact lead to the creation of classes that do better academically (and are more diverse).

The association's Assembly amended NACAC's Statement of Principles of Good Practice to state that: “All members should educate staff in understanding the concept of test measurement, test interpretation and test use, so they may consider standardized tests in their appropriate context. Such education may be obtained from NACAC, institutions of higher education, or other associations specializing in standardized educational testing. In addition, all members that make use of admission tests should acquire education and/or training in the appropriate use of specific tests from the sponsoring agencies."

The motion reflected one of the recommendations of the association's Commission on the Use of Standardized Tests in Undergraduate Admission. That report said that for too many admissions officers, the only training they receive on the use of testing may come from the technical training provided by testing companies, entities that have a vested interest in the continued use of testing.

Some NACAC members wanted to go beyond the language that was adopted and state explicitly that training on testing shouldn't come from testing companies, but an amendment to that effect was defeated. NACAC's board had recommended the language adopted as sufficient to encourage independent training without impugning testing companies.

Successful Experiments

For several years now, sessions at NACAC meetings have featured success stories from colleges that dropped or modified testing requirements. Much of the early data to measure success has focused on application numbers, with deans talking about how their totals -- and important subtotals, such as those from minority students -- have gone up in the wake of dropping the SAT as a requirement.

Several of the presentations this year also provided evidence about the performance of students once enrolled through new approaches to testing.

One of the boldest experiments under way is at Tufts University, where the dean of the School of Arts and Sciences is Robert J. Sternberg, who is also a noted expert on psychology and testing, and a strong proponent of the idea that there are important skills that the SAT misses. Based on Sternberg's work, Tufts started a program in 2006 in which applicants may submit additional essays used to identify those who are creative, who possess practical skills, or who have wisdom about how to promote the common good -- characteristics Tufts says are consistent with its vision of higher education, but which may not be reflected in SAT scores or high school grade point averages. (Tufts, it should be noted, didn't drop its testing requirement.)

For instance, one of the essays used to measure creativity asked students to write an essay using one of the following titles: The End of MTV, Confessions of a Middle-School Bully, The Professor Disappeared, or The Mysterious Lab. And in a non-verbal assignment, another creativity measure was to draw a new product or an advertisement for a new product.

In a presentation here, Sternberg said that events of the last year have only reinforced his sense that society needs broader measures than those commonly used to select students for top colleges. "A lot of the people who landed us in the mess we are in today had M.B.A.'s from very good schools," he said. "It seems like something is missing here. After they go to great schools ... they make a total mess of things."

Tufts has previously reported that adopting the optional essay program -- called Kaleidoscope -- has led to greater numbers of applications, particularly from minority groups, and that the resulting classes have been more diverse. Now that the program has been in place for a few years, however, a more complete picture has emerged -- and Sternberg said it showed the value of adding additional admissions measures.

Since 2006, the average annual increase in black enrollment has been 26 percent and for Latino enrollment has been 14 percent -- all while SAT and grade-point averages have gone up, Sternberg said. More than half of applicants are completing a Kaleidoscope essay now, even though they do not have to. Sternberg said that completing the essay or not has little impact on those at the top of the academic qualification scale (who get in either way) or the bottom (where a good Kaleidoscope essay isn't enough). But he said that it is having an impact for those in the middle.

Overall, Sternberg said, Tufts has found no difference in college grades between those who were admitted in part due to a Kaleidoscope essay and those who were not. But when controlling for academic rankings, those who do the Kaleidoscope essays do better at Tufts than those who don't. The study also found (through surveys) that those who had high Kaleidoscope scores are happier at Tufts and are more involved in campus life.

These findings are significant, Sternberg said, because they show that factors not measured by the SAT have a significant impact on whether or not applicants may be a good fit at a college and contribute to its vibrancy. Further, he said that unlike the SAT, Kaleidoscope scores didn't differ by racial or ethnic group.

Another institution reporting at the meeting was George Mason University, whose decision to go test-optional was considered notable because it is a large, diverse public university, while many of the leaders in going test-optional have been small private colleges -- institutions that have smaller application pools and larger (per applicant) admissions staffs.

But the news from George Mason is also positive.

Andrew Flagel, dean of admissions, said that since 2007, when the option was introduced, the percentage of applicants applying without test scores has increased to 13 percent, from 4.5 percent. The admit rate was slightly lower for this group, but the yield rate (the percentage of admitted applicants who enroll) was slightly higher, so that the share of the freshman class enrolled is expected to be just under 10 percent this year, up from 4 percent in the first year of the program.

Flagel said that George Mason has wanted to be sure that those admitted this way were succeeding at the university and that the results are what he had hoped to see. Students admitted without test scores in 2007 now have an average G.P.A. of 3.08, compared to 2.92 for those admitted with test scores. Of those admitted without test scores in 2008, the average G.P.A. is 2.99, compared to 2.84 for other students in that cohort.

The results reinforce his sense that test-optional "is a better process," Flagel said.

George Mason is continuing to try to encourage students to tell their stories directly, without an emphasis on numbers, Flagel said. The latest innovation is giving students the option of submitting their essays (or an additional essay) through a YouTube video. Flagel said that staff members convinced him that this is one way many applicants express themselves, so the admissions office should be open to it, and he agreed. "Personalizing the process" is an important value that builds on going test-optional, Flagel said.
— Scott Jaschik