At best, these tests are diversionary; at worst, they narrow curriculum and work against inventive and innovative thinking.  They also represent a measurement system that is silent on questions of equity.

"...given the sampling and measurement flaws in the rankings and their negligible role in assessing the overall quality of education systems, much less the strength of economies. Whether or not the United States continues to rank high on competitiveness, international test scores will remain virtually irrelevant."

-Angela

International Scores Irrelevant to Economic Competitiveness

By Invited Contributor Listed Below on March 27, 2014 8:58 AM | No comments

Today's guest contributor is Iris C. Rotberg, Research Professor of Education Policy, The George Washington University.

The ranking of the United States on international tests of science and mathematics continues to fuel rhetoric about economic competitiveness and shortages of scientists and engineers, despite the fact that the United States consistently ranks first, or among the top countries, in competitiveness. Moreover, there is little evidence of shortages of scientists and engineers to fill traditional science, technology, engineering, and math (STEM) jobs.  It is sometimes argued, however, that these apparent strengths are fragile and we should not assume that because the numbers look good now they will continue to look good in the future. That is a fair argument--none of us can predict long-term economic and scientific strength with any degree of certainty. But we do know that, regardless of the outcome, it will not be international test-score rankings that make the difference.
The irrelevance of test-score rankings is illustrated in reports of the IMD (a global business school in Switzerland) and the World Economic Forum, which rank countries by international competitiveness. The rankings are based on a set of variables chosen to reflect current knowledge about what is most important in determining competitiveness. These variables include, for example, the soundness of the economy and financial sector; business sophistication; innovation; the quality and fairness of governmental and private institutions; market efficiency;  basic, technological, and scientific infrastructure; and the overall strength of the education system (primarily capacity and access at all levels of education). International test-score rank was only one of the 113 criteria used by the IMD to measure these variables. Performance on international test-score comparisons was not even mentioned among the 114 criteria used by the World Economic Forum--and for good reason, given the sampling and measurement flaws in the rankings and their negligible role in assessing the overall quality of education systems, much less the strength of economies. Whether or not the United States continues to rank high on competitiveness, international test scores will remain virtually irrelevant.
The test-score rankings also have little value in predicting whether a country will produce an "adequate" supply of scientists and engineers. The U.S. rank on test-score comparisons is often interpreted as a proxy for a shortage of talent in STEM fields, despite strong evidence that the United States has a large supply of students capable of going into those fields. It is true that many talented students choose not to enter STEM fields and many others who receive degrees in these fields choose not to work in them. A study conducted by Anthony P. Carnevale and colleagues at Georgetown University, for example, found that only a fourth of high school students who score in the top quartile in mathematics choose to enter a STEM major in college; only half the students who start with a STEM major graduate with that major; and fewer than half the students who graduate with a STEM major are actually working in STEM fields 10 years later. These students, instead, have entered other fields, including architecture, business, finance, or medicine. The point is that the attrition from traditional STEM fields does not reflect a lack of U.S. talent or training in these fields, but rather such factors as interests, salary differentials, a weak economy, or outsourcing of jobs because of lower wages outside the United States. Apple is unlikely to hire U.S. workers to replace the hundreds of thousands of workers outside the United States who are manufacturing and assembling component parts for its products because of more correct answers on a math test.
The United States currently has an ample supply of workers to fill traditional STEM jobs. Carnevale and colleagues, however, frame the question differently and see a potential for future shortages. They ask whether the country can produce a skilled labor force large enough to fill both the traditional STEM jobs as well as the large number of other jobs that might draw on similar skills, such as finance and medicine, taking into account projected retirement rates, possible reductions in foreign-born workers, and a future growth in STEM jobs at sub-baccalaureate as well as higher levels of education.
Whether or not the predicted shortages occur, the international test-score comparisons have become a diversion that detracts attention from the factors that can make a difference in scientific innovation and competitiveness. Indeed, the increasing focus on test scores has led to scripted learning and narrowing of the curriculum--trends that are inconsistent with an approach that encourages problem solving and innovation. That focus is also inconsistent with educational approaches designed to give students a broad set of skills that will contribute to their effectiveness in the workplace and is likely to be counterproductive in both attracting and retaining students in STEM fields.
The focus on test scores also detracts attention from the serious underrepresentation of low-income populations in STEM and the larger problem that underrepresentation illustrates--the growing gap in income and access. The gap will not be narrowed by rhetoric about international test-score rankings.