Thursday, January 14, 2010



January 14, 2010

Research uses bad methodology, weak data and incorporates mistakes of earlier
work, reviewer finds

Contact: Joydeep Roy, (202) 331-5526; [3]
Kevin Welner, (303) 492-8370; [4]

BOULDER, Colo. and TEMPE, Ariz. (January 14, 2010) -- A recent report contends
that providing private school vouchers would create job opportunities in five
poor, rural South Carolina counties. A new review [5] of that report, however,
finds that it is built on seriously flawed assumptions and thus offers little
insight into the effects of school vouchers.

The report, How School Choice Can Create Jobs for South Carolina, authored by
Sven Larson, is published by the South Carolina Policy Council Education
Foundation. It was reviewed for the Think Tank Review Project by Joydeep Roy of
Georgetown University and the Economic Policy Institute in Washington, D.C.

How School Choice Can Create Jobs for South Carolina adopts the assumptions and
analyses set forth in an earlier study. Both studies assert that communities
where school vouchers are offered display higher rates of entrepreneurship and
self-employment. The argument is apparently that vouchers are linked to
individual freedom for those involved -- in contrast to the non-freedom
associated with government-run schools -- and that the decrease in government
involvement and increase in competition for students is supposed to spur
competition and innovation, which is linked to self-employment and

Extrapolating data from the earlier study, the new report purports to precisely
calculate the number of firms and jobs that would be created with a voucher
program in place in the five South Carolina counties. Yet as Roy points out in
his review, the original study suffers from a series of methodological flaws.
Most importantly, the study is based on weak data that simply cannot confirm any
kind of causal relationship between vouchers and entrepreneurial status.

Roy also notes that the entire argument in the earlier study was based on
voucher programs in place by 1990, but the study never identifies the programs.
In fact, the only programs that appear to have existed at that time were in
Milwaukee voucher program (begun in 1990) and in Vermont and Maine, which had
long had quirky tuitioning programs allowing remote, rural students to attend
private, non-religious schools. "Lessons drawn from these unique programs and
circumstances may have limited applicability to other areas and programs," Roy

The Larson report uncritically applies the results of that earlier, flawed
research, Roy writes. The new report then adds another unproven set of
assumptions. Most importantly, it assumes without justification that there
exists a significant "multiplier effect" in jobs created that are spun off of
entrepreneurial activities. That is, it "assumes that small businesses would
expand by 25 percent due to voucher programs, and that these will employ people
at the same rate as existing small businesses of the same size," Roy explains.
But, "new small businesses are unlikely to generate the same employment
prospects as existing, older ones, since the latter have -- by definition --
been in the business for a while."

So the report comes up with an essentially speculative number representing
increased self-employment, decides that this is the equivalent of
entrepreneurship and that these new entrepreneurs are starting small businesses
that mirror existing South Carolina small businesses in terms of their size and

The upshot, says Roy, is a paper that relies more on rhetoric and less on
authentic research. "The rhetoric of the report and the approaches used suggest
a clear goal of championing a positive effect of voucher programs, with little
or no concern for providing a careful analysis of the question at hand," Roy
concludes. With inadequate methodology and "significant biases and omissions"
undermining any potential for valid findings, it offers "little, if any, help in
guiding policymakers, educators or the public."

Find Joydeep Roy's review of How School Choice Can Create Jobs for South
Carolina on the web at

Find Sven Larson's report, How School Choice Can Create Jobs for South
Carolina, published by the South Carolina Policy Council Education Foundation at [7]

Joydeep Roy
Economic Policy Institute & Georgetown University
(202) 331-5526 [8]

Kevin Welner, Professor and Director
Education and the Public Interest Center
University of Colorado at Boulder
(303) 492-8370 [9]

About the Think Tank Review Project

The Think Tank Review Project ( [10]), a
collaborative project of the ASU Education Policy Research Unit (EPRU) and
CU-Boulder's Education and the Public Interest Center (EPIC), provides the
public, policy makers, and the press with timely, academically sound reviews of
selected think tank publications. The project is made possible by funding from
the Great Lakes Center for Education Research and Practice.

Kevin Welner, the project co-director, explains that the project is needed
because, "despite their garnering of media attention and their influence with
many policy makers, reports released by private think tanks vary tremendously in
their quality. Many think tank reports are little more than ideological
argumentation dressed up as research. Many others include flaws that would
likely have been identified and addressed through the peer review process. We
believe that the media, policy makers, and the public will greatly benefit from
having qualified social scientists provide reviews of these documents in a
timely fashion." He adds, "we don't consider our reviews to be the final word,
nor is our goal to stop think tanks' contributions to a public dialogue. That
dialogue is, in fact, what we value the most. The best ideas come about through
rigorous critique and debate."


The Education and the Public Interest Center (EPIC) at the University of
Colorado at Boulder and the Education Policy Research Unit (EPRU) at Arizona
State University collaborate to produce policy briefs and think tank reviews.
Our goal is to promote well-informed democratic deliberation about education
policy by providing academic as well as non-academic audiences with useful
information and high quality analyses.

Visit EPIC and EPRU at [11]

EPIC and EPRU are members of the Education Policy Alliance
( [12]).


(c)2010 EPIC. Education and the Public Interest Center, School of Education
249 UCB, University of Colorado at Boulder, CO 80309-0249.
Phone: 303-447-EPIC(3742) | [13]
To unsubscribe from future mailings, click here [14].


No comments:

Post a Comment