Taking on Testing Misuse

Inside Higher Ed
April 20, 2009

In September, a special panel of the National Association for College Admission Counseling issued a report calling for colleges individually, and higher education generally, to rethink the use of standardized tests. While not calling for the abolition of such testing, the association came out strongly against the use of any test as the sole criterion for key admissions or financial decisions.

Associations of course regularly issue reports calling for colleges or higher education in general to do some things and stop doing other things, but it's rare for these groups to try to systematically follow up with those who ignore the recommendations. It is even more rare for these groups to point fingers at particular higher education groups that praise their reports (and then ignore key recommendations). But NACAC has been trying to take a different approach with its report. Today it is announcing requests it made to the College Board that it explain why it has done nothing about the use of the PSAT as the sole qualifying test for National Merit Scholarships, and to the National Merit Scholarship Corporation about why its policies run counter to the stated policies of the College Board and the new NACAC report.



While the College Board and the National Merit Scholarship Corporation are ignoring the critique they received from NACAC, the admissions group isn't letting the matter drop. In an unusual move for a group that is part of the higher education establishment, NACAC is releasing today the letters it sent objecting to the use of the PSAT, the responses from the organizations, and a critique of the responses.

David Hawkins, director of public policy and research for NACAC, said that the association doesn't intend to let the matter drop. While not revealing NACAC's next move, he said it is being discussed. "We are committed to pursuing this issue, and are in it for the long haul," he said.

The National Merit Scholarships are among the most prestigious for undergraduate study -- and being designated even a semifinalist can help students gain admission to top colleges. While the value of individual scholarships varies, the total awarded annually is worth around $35 million. While final winners are selected on a variety of criteria, to become a semifinalist, one must have a top score on the PSAT (with qualifying scores varying by state). There is no other measure for reaching semifinalist status -- one must achieve a specific score, which varies by state.

In its letter to the corporation that runs the scholarships, NACAC leaders write that they do not object to using the PSAT as "one valid tool to assess academic achievement," but that they believe it is educationally unsound to use it as the "only factor" to determine scholarship eligibility. A similar letter was sent to the College Board asking why it allows PSAT scores to be used in this way -- even though the College Board is on record as saying that it does not favor the use of its tests as the sole criterion for such decisions.

In their replies, both organizations say that test scores are not being used inappropriately. Letters from both the National Merit Scholarship Corporation and the College Board note that after semifinalists are selected based on the PSAT, multiple criteria are considered in deciding who actually receives a scholarship. The scholarship corporation letter says that using the PSAT is "the most effective, inclusive, and equitable [method] available to consider over 1.5 million students annually on a consistent basis." The College Board notes that it had one of its own task forces review its relationship with the National Merit Scholarship Corporation, and that the review had concluded that the relationship was appropriate.

NACAC responds with a new statement in which it says that "neither organization’s response directly addressed the concern expressed by the NACAC Testing Commission." The admissions group notes that while both the College Board and the scholarship corporation cited studies showing value provided by the PSAT, they cited no research about the use of any test score as the single way to make an important decision. "In the absence of such research, existing guidance suggests that the use of cut scores in the awarding of financial aid is not in keeping with ethical practice."

Further, NACAC goes on to say that the use of automatic cut scores for this scholarship is particularly problematic because the National Merit Scholarships are "a fixture in the American collegiate admission landscape" and because the corporation plays a role in "promoting the idea of 'merit' as part and parcel of the admission and financial aid process." NACAC maintains that there is a simple solution: to "augment the initial eligibility criteria to ensure that students who are deemed ineligible due to the single PSAT cut score have other ways to demonstrate merit and be eligible for further consideration."

The letter to the College Board, as well as a letter to ACT, also ask the testing agencies to review how their tests are used by state education agencies in accountability measures. Further, the testing agencies are asked (and agree) to help NACAC with its goal of developing an independent source of information on the use of testing results. The NACAC report on standardized tests, which called for colleges to be much more certain than they are now about the need for testing, noted that many colleges get all of the training about test usage from testing companies.

ACT's response took issue with having its test grouped together with the SAT in much of the NACAC report. ACT argues that its test is closely tied to the high school curriculum and is designed to measure knowledge learned in courses, not aptitude. ACT writes that the authors of the NACAC report "appear to suggest that the ACT and the SAT are based on identical philosophies and that they measure the same skills. This is simply not the case."

NACAC replies that it is "well aware of the differences between the SAT and ACT" but "such differences are, in the opinion of the commission, overshadowed by several important considerations in the discussion about their influence on the admission process: 1. Neither test fully encompasses the breadth and depth of student learning that can be gleaned from an observation of a student’s performance in high school coursework; 2. Both tests perform similarly in predictive validity studies at colleges and universities; and 3. Both tests are subject to misuse by parties not familiar with standards for ethical practice in admission and/or test use."