LEXINGTON ‘PORTFOLIOS’ REPORT LACKS RESEARCH, RELEVANCE

Mathis exposes the agenda behind this so-called research report that comes out against portfolio assessment in time to influence the NCLB re-authorization. Good to get the word out on this. -Angela

****NEWS RELEASE--FOR IMMEDIATE RELEASE****

LEXINGTON ‘PORTFOLIOS’ REPORT LACKS RESEARCH, RELEVANCE
Reviewer cites faulty generalizations and ignored research
in report’s support of standardized testing under NCLB

Contact: William J. Mathis, (802) 247-5757; (email) WMathis@sover.net
Kevin Welner, (303) 492-8370; (email) kevin.welner@gmail.com

TEMPE, Ariz and BOULDER, Colo. (Sept. 19, 2007)—A Lexington Institute report released earlier this month, and the institute’s subsequent press release of September 17, 2007, criticized the use of student portfolios to assess school performance. A new review of that report, however, finds it is ill-founded and of little value for research or policy development.

The Lexington report, called “Portfolios – A Backward Step in School Accountability,” is intended to influence the debate over the direction of the reauthorization of the No Child Left Behind Act, offering a defense of the current test-based accountability system against the inclusion of “multiple measures.”

The report appears to have been written in anticipation of a “discussion draft” concerning NCLB changes, released by the leadership of the House Education Committee. The draft proposes allowing states to use “multiple indicators” – for example, graduation rates and percent of students taking advanced courses – to assess educational outcomes rather than depend so heavily upon standardized test scores.

The Lexington report was reviewed for the Think Tank Review Project by University of Vermont professor William Mathis. He concludes that the report more closely resembles a political polemic than a research report. It provides no new data, examines only two studies, and includes only results favorable to the report’s conclusions.

As Mathis notes, the House Committee’s summary lists a broad list of various alternative multiple indicators, but portfolio assessment is not on that list. Nonetheless, the Lexington report erroneously generalizes findings about portfolios to all non-test-based multiple indicators. The report also ignores a body of research with findings that present portfolios in a more favorable light.

In fact, Mathis’ review explains that the Lexington report’s focus on portfolios is particularly misguided because portfolios for statewide accountability purposes have received only scant attention since the turn of the millennium. Although Lexington’s press release emphasizes “dusting off” the use of portfolio assessments as a key part of NCLB accountability, the only group seriously discussing the subject appears to be Lexington itself. Given the absence of portfolio assessment from such agendas as well as from the House Committee’s list, it is troubling that the Lexington Institute report offers portfolios as the most notable of what it calls “multiple measures” and then uses that straw man to argue against adopting any instruments other than standardized tests.

In his review of the Lexington report, Mathis concludes that its “selective use of research suggests the author either intentionally slanted the evidence or was unacceptably cursory in his analysis.” The report’s failure to discuss contradictory research undermines its conclusions, and its attempt to “generalize all multiple measures from this questionable base completely discredits [the report].”


Find William Mathis’s review on the web here.

About the Reviewer

William Mathis is an adjunct professor of school finance at the University of Vermont and a local school superintendent. By training, Mathis is a psychometrician and served as Director of New Jersey’s state testing program and as a consultant for the National Assessment of Educational Progress as well as for other states’ testing programs.


About the Think Tank Review Project

The Think Tank Review Project (http://thinktankreview.org), a collaborative project of the ASU Education Policy Research Unit (EPRU) and CU-Boulder’s Education and the Public Interest Center (EPIC), provides the public, policy makers, and the press with timely, academically sound reviews of selected think tank publications. The project is made possible by funding from the Great Lakes Center for Education Research and Practice.

Kevin Welner, the project co-director, explains that the project is needed because, “despite their garnering of media attention and their influence with many policy makers, reports released by private think tanks can be of very poor quality. Too many think tank reports are little more than ideological argumentation dressed up as research. We believe that the media, policy makers, and the public will greatly benefit from having qualified social scientists provide reviews of these documents in a timely fashion.” He adds, “We don't consider our reviews to be the final word, nor is our goal to stop think tanks’ contributions to a public dialogue. That dialogue is, in fact, what we value the most. The best ideas come about through rigorous critique and debate.”


CONTACT:
William J. Mathis, Adjunct Professor
University of Vermont
(802) 247-5757
WMathis@sover.net

Kevin Welner, Professor and Director
Education and the Public Interest Center
University of Colorado at Boulder
(303) 492-8370
kevin.welner@gmail.com


**********


###


The Education Policy Research Unit (EPRU) conducts original research, provides independent analyses of research and policy documents, and facilitates educational innovation. EPRU facilitates the work of leading academic experts in a variety of disciplines to help inform the public debate about education policy issues.

Visit the EPRU website at http://educationanalysis.org

###

The Education and the Public Interest Center (EPIC) at the University of Colorado, Boulder seeks to contribute information, analysis, and insight to further democratic deliberation regarding educational policy formation and implementation.

Visit the EPIC website at http://education.colorado.edu/epic


The following physical address is associated with this mailing list:

EPIC
School of Education
249 UCB
University of Colorado
Boulder, CO 80309-0249