Wrong Kind of Accountability?

Texas politics is getting national press all over the place. Check out the 821 page Texas faculty productivity report .

-Patricia

Dan Berrett | Insider Higher Ed
May 10, 2011

Faculty and administrators at public universities in Texas said Monday they don’t want to shrink from efforts to make public higher education more accountable -- they just don’t want to do it this way.

In this case, “this way” refers to efforts by the University of Texas System Board of Regents to measure the productivity of faculty members in strictly numerical terms. The efforts are reflected in 821 pages of raw data that have been collected by the UT system. The data have not been vetted. Each page contains the disclaimer that the information is “incomplete and has not yet been fully verified or cross referenced [and] [i]n its present raw form ... cannot yield accurate analysis, interpretations or conclusions.” Originally planned for release after more thorough review later this year, the data reached the public last week through open records requests filed by several Texas newspapers. The information lists the salary, teaching load and number of students of each faculty member, as well as his or her external research funding.

Resistance to the effort has already led to the ouster last month of Rick O'Donnell, an adviser to the regents who was seen as promoting the ideas of Governor Rick Perry, a Republican. O’Donnell, who worked at the Texas Public Policy Foundation (which has been something of the nerve center of Perry’s higher education reform agenda), has blamed senior officials in the University of Texas System and at the university's flagship campus in Austin for resisting efforts to compile data on whether faculty teach enough.

But on Monday, the resistance to the productivity measures took a more public turn. In a speech, William Powers Jr., the president of UT-Austin, critiqued some of the assumptions underlying the analysis -- though he never cited it or the regents directly. And, strikingly, he based his critique on the grounds that the analysis failed to adequately capture the output, productivity and relevance of research (typically, teaching is the aspect of a professor’s job that is thought to be the more difficult to accurately measure).

In addition to citing the contributions of two professors (one in the humanities and the other in physics), Powers described the experience of a student at the university: a senior in chemical engineering named Katie Maass. As an assistant at a campus laboratory, Maass worked on nanoparticles that can release medicine in the small intestine instead of the stomach, Powers said, which helped her land a five-year, $250,000 research grant as she plans to enter the doctoral program at the Massachusetts Institute of Technology.

“The kind of research experience that Katie received as an undergraduate at UT is not factored into the metrics some would use to evaluate Tier 1 research universities,” Powers said. “The kind of faculty mentoring and interaction that Katie received cannot be captured in a faculty profit-and-loss statement.”

The reference to a “profit-and-loss statement” was interpreted by some as a swipe at a previously released version of the data regarding faculty at Texas A&M University, which depicted individual faculty members as being in the red (costing more than they produce) or the black (generating revenue). That version has since been replaced, and the faculty productivity effort at UT should not be characterized in terms of profit and loss, said Anthony P. de Bruyn, a spokesman for the UT system.

In addition, said de Bruyn, the goal of gathering the data was not to assess individual faculty members, but to review university departments at each institution. When asked why, then, the information listed each faculty member by name and didn’t include tallies of these measurements by department, de Bruyn said that the institutional research divisions for each university had supplied the data that way.

Powers's comments Monday were his first since last week's release of the faculty productivity data at UT, and professors at Austin hailed their president's words. Dean Neikirk, Cullen Trust for Higher Education Endowed Professor in the Department of Electrical and Computer Engineering (and president of UT-Austin’s Faculty Council), said Powers had expressed the difficulty of adequately measuring the impact that research -- and individual professors -- can have on the wider institution.

“It’s an ecosystem, and trying to reduce the contribution of a single individual to a set of simple numbers is never going to be easy,” Neikirk told Inside Higher Ed. “A lot of that is not reducible to a simple number the way an engineer will reduce things.”

On Monday, the Faculty Council at UT-Austin agreed. It unanimously passed a vote of confidence in Powers and his administration. Saying that the university’s mission, core purpose and core values are in jeopardy, the approximately 40 members of the faculty at the council’s meeting added that they believe Powers and his team “have tirelessly and effectively advocated for the university while valuing academic research and excellent teaching and adhering to the highest standards of professionalism and ethical conduct.”

The Texas Public Policy Foundation -- a think tank whose trustee, Brenda Pejovich, is also a regent and the chair of the UT board's task force on faculty productivity -- added that it, too, was pleased with one aspect of the president’s comments: his acknowledgment of the need for reform in an environment in which universities must do more with less. (Powers said that -- as state support for UT-Austin has dropped from 52 percent of the university’s budget in 1981 to a projected 13 percent, according to a version of the 2012 budget proposed by the House -- the university will need to change how it does business and to chart a new, more sustainable path. “We can’t just long for the old days," he said. "We need to be on the leading edge of change.”)

David Guenthner, senior communications director for the foundation, said the group’s interest in higher education is based in a belief that student learning and teaching have been neglected in recent years. “There needs to be an evaluation of where resources are going between research and education so we can set a proper balance,” he said.

He also acknowledged that the information compiled by the regents was incomplete in its current form (some faculty members with whom Inside Higher Ed spoke said that their salaries were listed in the raw data at levels that were higher than they actually were, or that the document underrepresented the quantity of their research). And Guenthner predicted that there would be “significant categories of research where we agree with universities that it’s worth the time and expense and the staff, and the release time from the classroom.”

But he added, “There will also be, I presume, a category of research where, not just us, but members of the public at large will look at it and say, ‘Is it really worth us giving release time from the classroom for a professor to put out that particular article? Instead, would it be a better use of time to have that teacher or professor in a classroom providing their knowledge to students?' ”

In theory, such goals are fine, said Peter Hugill, professor of geography at Texas A&M, and president of the state conference of the American Association of University Professors. “Although I really am in favor of accountability, I just think we have to be cautious of overly simplistic measures,” he said. “Numbers are helpful, but they’re not by any means the sole arbiter of performance.” Hugill said department chairs typically evaluate the research of their own faculty members by using more nuanced methods: they don’t simply count the dollar amount of grants, but also consider the source; they don’t just tally the number of journal articles, but they also evaluate the impact of the scholarship and where it was published.

The numbers gathered by the regents are also already available, said Murray Leaf, speaker of the Academic Senate, and professor of anthropology and political economy at UT-Dallas. The problem, he said, is the way information is organized and assembled. “What this does is provide information that’s not normally juxtaposed,” Leaf said in an interview. Leaf, who said that the salary listed for him was overstated by 50 percent, described as “intrusive” the effort to use a snapshot from one point in time to paint a larger picture of productivity -- because it will fail reflect the ebbs and flows of academic life. “It’s intended, I think, to be intimidating in that regents can arrange the information this way and propose to evaluate on this basis,” he said. “That attitude on the part of the regents is worrisome.”

Some also have been critical of the larger assumption that is embedded in the task force’s work -- a hard distinction between teaching and research. Terri E. Givens, associate professor of government at UT-Austin (and author of the Running ‘Round the Ivory Tower blog for Inside Higher Ed), said that the data failed to reflect seven research trips she recently took -- and that it was impossible to distinguish her research (with three undergraduate and four graduate students) from her instructional and service duties. “I spend half of my week doing writing and research,” she said, “It’s directly related to teaching.”

The Association of American Universities also weighed in, though privately. In a letter to the chancellor of the Texas A&M system, Robert M. Berdahl, AAU’s president, observed that “the key to the success of the American research university is the close alliance between research and teaching, especially the teaching of graduate students.”

Berdahl criticized related proposals that were advanced by the Texas Public Policy Foundation for further severing the connection between research and education. “Separating research from teaching and oversimplifying the evaluation of faculty does violence to the values that have produced the American universities that are envied and emulated across the globe,” he wrote.

But such a critique assumes the melding of research and teaching is the natural state of things when it really shouldn’t be, said Charles Miller, a former UT regent chairman who said the task force’s efforts to look at productivity (as well as another panel's work on blended learning) were to be commended. Community colleges and non-flagships, as well as private liberal arts institutions, do very little research, said Miller.

“The real issues at 'research universities' revolve around about how teaching is undervalued relative to research or, most important, how effective the use of resources are in research -- or in teaching,” Miller wrote in an e-mail to Inside Higher Ed. Miller used UT-Austin as an example, criticizing its graduation rates of approximately 50 percent (four years) and 80 percent (six years) among those who were top students in high school and hail from families with average incomes far above the median in Texas. “[This is] strong evidence of a place where teaching and learning is not conducted productively,” he said. “These graduation rates for a top 'research university' should be considered totally unacceptable by the UT System Board of Regents.”

And they are, at least by Powers. In his comments, he ascribed problems in the graduation rate to several factors, including poor choices, sometimes by students, but also on the part of the university. These poor decisions come, he said, "in choosing a major, in course selection, in matching their talents to their degree plan, or in our own failure to provide enough available course sequences."