68 Institutions in Nine States to Pilot New Approach to Learning Outcomes Assessment

I am just seeing this piece on higher education accountability in light of the previous piece I just posted that is a must read.  A hot link appears to this current page that I think is very important to policy development and practice as regards higher education accountability policies, nationally.

Regarding testing college students, here is what the previously-posted piece says:

Step V: Destroy the students.While claiming to offer them hope of a better life, our corporatized universities are ruining the lives of our students. This is accomplished through a two-prong tactic: you dumb-down and destroy the quality of the education so that no one on campus is really learning to think, to question, to reason. Instead, they are learning to obey, to withstand “tests” and “exams,” to follow rules, to endure absurdity and abuse. Our students have been denied full-time available faculty, the ability to develop mentors and advisors, faculty-designed syllabi which changes each semester, a wide variety of courses and options. Instead, more and more universities have core curriculum which dictates a large portion of the course of study, in which the majority of classes are administrative-designed “common syllabi” courses, taught by an army of underpaid, part-time faculty in a model that more closely resembles a factory or the industrial kitchen of a fast-food restaurant than an institution of higher learning.

This book was also cited there and strikes me as a must-read:  Unmaking the Public University: The fourty year assault on the middle class.  

There have been proposals here in Texas that call for so-called higher education accountability through increased student testing. I was particularly involved in this during the 82nd Texas Legislative Session in 2011.  You can read about this here: 

2011 - 82nd Session of the Texas State Legislature

• House Bill 9 Policy Brief: Relating to Student Success–based Funding for Public Institutions of Higher Education
  (PDF file)

Forewarned is to be forearmed.


-Angela

Tuesday, June 24, 2014

For Immediate Release
Contacts: Debra Humphreys, Vice President for Policy and Public Engagement, AAC&U
Humphreys@aacu.org
202.387.3760, ext. 422
Julie Carnahan, Senior Associate, SHEEO
jcarnahan@sheeo.org
303.817.2129

68 Institutions in Nine States to Pilot New Approach to Learning Outcomes Assessment

Multi-State Collaborative Announces Institutions that Will Participate in New Initiative Employing VALUE Rubrics to Assess Student Achievement of Key Learning Outcomes

Washington, DC—June 23, 2014—The Association of American Colleges and Universities (AAC&U) and the State Higher Education Executive Officers Association (SHEEO) announced today the 68 institutions—including both 2-year and 4-year institutions—participating in the Multi-State Collaborative to Advance Learning Outcomes Assessment (MSC) supported in its initial planning year with funds from the Bill & Melinda Gates Foundation. The nine states currently participating in the MSC include: Connecticut, Indiana, Kentucky, Massachusetts, Minnesota, Missouri, Oregon, Rhode Island, and Utah. See: www.sheeo.org/msc for full list of participating institutions.
“I am pleased and excited that faculty and leaders from so many public colleges, community colleges, and universities in so many states are joining together to assess students’ work using a common approach,” said George Pernsteiner, the president of SHEEO. “What our faculty learn from this work will help them improve teaching and student learning and will provide valuable and defensible information to show that students are learning, and what that learning means in terms of the understanding and skills needed to succeed in life.”
Part of AAC&U’s ongoing VALUE (Valid Assessment of Learning in Undergraduate Education) initiative, the 68 institutions in the MSC will pilot test a cross-state and cross-institutional effort to document how well students are achieving key learning outcomes like quantitative reasoning, written communication, and critical thinking by assessing authentic student work products using a set of common rubrics. Faculty members across the 68 institutions will be sampling and assessing students’ work and establishing the reliability and validity of cross-institutional assessment using this new approach. During its initial year, the project will be building faculty assessment capacity and collecting student work products. The project will also be developing a Web-based data platform for uploading student work samples and assessment data.
In its earlier phases of work, VALUE published 16 rubrics developed and tested by teams of faculty and other educational professionals that are aligned with the LEAP Essential Learning Outcomes and also with the intellectual skills highlighted in the Degree Qualifications Profile. Over 2,000 colleges and universities and community colleges in the U.S. already are using VALUE rubrics to assess student work.
Leadership in organizing the MSC came through the Massachusetts Board of Higher Education, which, as part of its Vision Project, piloted use of AAC&U’s VALUE rubrics to assess student learning in Massachusetts. Working with AAC&U and SHEEO, leaders in Massachusetts subsequently helped organize a nine-state collaboration that is developing platforms and protocols for scaling the use of this approach to quality assurance.
”The calls are mounting daily for higher education to be able to show what students can successfully do with their learning,” said AAC&U President Carol Geary Schneider. “The Multi-State Collaborative is a very important step toward focusing assessment on the best evidence of all: the work students produce in the course of their college studies. It is exciting and inspiring to see that so many campuses want to be part of this important national study and change effort.”
For more information, see VALUE and Multi-State Collaborative on Learning Outcomes Assessment.

Related Categories: 

Academic Policy and Quality Assurance

Related Issues: 

Student Learning and Accountability

UT Faculty Productivity Gets High Marks in New Report

Check out Musick's full report, "An analysis of faculty instructional and grant-based productivity at the University of Texas at Austin"

-Patricia

by Reeve Hamilton | Texas Tribune
November 13, 2011

Despite the arguments of critics in recent months, Marc Musick, the University of Texas at Austin’s College of Liberal Arts associate dean of student affairs, makes the case in a new faculty productivity report that his institution provides “an incredible return on investment for the state.”

Using data from the 2009-10 academic year made public by the University of Texas System this summer, Musick found that UT professors generated revenue of more than twice their compensation of $257 million in state funds for salary and benefits. By combining the amount of money paid by the state via a student enrollment-based formula and external funding for academic research, Musick concluded that the UT faculty generated about $558 million in total revenue for the university.

Musick’s report is the latest in a series of similar productivity studies that have been released over the course of a year marked by questions about the effectiveness of the state’s higher education system. The studies have come from a variety of sources using differing methodologies and reaching a wide range of conclusions, some of them strikingly negative. The latest release comes at a time when many of the key players in the state’s ongoing debate over higher education are poised to take the discussion on how to measure faculty productivity to a national level.

The topic became a hot-button issue in the spring, due in large part to a set of seven controversial proposals for higher education written by Austin businessman Jeff Sandefer in 2008 and promoted by Gov. Rick Perry and the Texas Public Policy Foundation, a conservative think tank of which Sandefer is a board member.

On Friday, Sandefer and others tied to the TPPF are participating in a higher education conference in Washington, D.C., put on by the Cato Institute, a prominent national conservative organization. According to information the institute posted online about the event, “One key question the conference will take on is how to assess the productivity of faculty members, including examining the groundbreaking — and highly controversial — efforts recently undertaken in the state of Texas."

An early December closed-door gathering in Indianapolis organized by UT President Bill Powers for the presidents and provosts of public universities in the Association of American Universities, an elite organization of research institutions, will also tackle the subject. “In particular, I would like to explore how we might foster richer deliberations about higher education productivity than we have seen recently in Texas and other states,” Powers wrote in his invitation.

While Musick’s new study demonstrates a high level of faculty productivity at the university, he acknowledged that it omits key elements of professors’ workload. “All it’s doing is measuring two things that the faculty do,” he said. “It’s measuring grants and its measuring teaching. But faculty do lots of different things. The data we have are extremely limited in what they can tell us.”

Still, Musick saw in the data opportunities to encourage more productivity. He recommends enhancing offerings for faculty mentorship, since the strongest-performing faculty tend to be more experienced. He said the number of students in some mid-size classes could be increased without sacrificing quality, freeing up resources for more of the smaller classes that students prefer. He also recommended that the university provide greater incentives for professors to pursue grants and more assistance with their submissions so that less of their time is spent on paperwork.

Most importantly, Musick said, faculty need to be evaluated based on accurate, comprehensive data that conveys their productivity over time as opposed to a single year.

“I completely agree with the idea of going out and getting data and analyzing it,” said Musick, who also released a report on university efficiency in September, “but it’s got to be helpful, it’s got to be thoughtful, its got to be done in the right way to make sure we are finding the truth and not just doing what’s easy to find quick answers.”

Enrollment in Texas Higher Education Continues to Climb

by Reeve Hamilton | Texas Tribune
October 27, 2011

More than 62,000 students enrolled in colleges and universities in Texas this fall than in 2010, according to preliminary enrollment data released by the Texas Higher Education Coordinating Board today.

While it's a big jump, it's not as large as the two previous years, which both saw increases of more than 100,000 students. In the last three years alone, Texas colleges and universities have added more than 268,000 students. The current total, which includes public and private institutions, is slightly more than 1.5 million students.

Texas Higher Education Commissioner Raymund Paredes said this puts the state firmly on track to meet its 2015 goals for boosting access to college in the state. However, some of the numbers could still use some improvement, particularly the enrollment of African American males.

Overall, African American participation has jumped nearly 10 percent in the last year. Hispanic enrollment numbers are up 4.5 percent, and Anglo students have also increased nearly 2 percent.

The ethnic category that saw the most dramatic shift was "other," with a nearly 18 percent increase. New self-reporting policies put in place last year allow students to choose multiple races. Doing so places them in the "other" category. Paredes speculated that the growing popularity of that option was a testament to the decining racial divisions in the country.

Most of the growth occurred in the state's public colleges and universities, though a handful of them did see a decrease. The preliminary numbers show University of Texas at Austin enrolling 50 fewer students than last year. Four-year institutions experiencing more significant drops include the University of North Texas, Prairie View A&M University, and UT-Permian Basin. Meanwhile, the University of Houston, Texas State University-San Marcos, UT-Brownsville, and UT-Dallas, each saw increases of more than 1,000 students.

Coordinating board officials also say their data also show an increase of nearly 30,000 students in private, for-profit colleges, continuing a trend of steady growth. However, some of that increase can be attributed to the fact that more of those institutions have begun submitting their information.

Switch to Outcomes-Based Higher Ed Funding Taking Time

by Reeve Hamilton | Texas Tribune
June 6, 2011

During the regular session, Gov. Rick Perry’s top legislative priority for higher education was the implementation of a new funding system that rewards universities for graduating more students, not just for getting students into classes. To reach its 2015 goals, Texas needs to increase the number of degrees awarded by 46,000 each year.

A financing system that includes more “outcomes-based funding” has strong support, including from Raymund Paredes, the Texas Higher Education Commissioner, and business leaders such as Woody Hunt of El Paso, who chairs the Governor’s Business Council. But policy makers have struggled to agree on which outcomes to measure, how to encourage them and if they should alter the funding formulas while budgets are being slashed.

House Higher Education Chairman Dan Branch, R-Dallas, passed a major bill on the subject this session. The final version approved by the full Legislature, however, stops short of actually implementing an outcomes-based system, and instead provides guidelines for the Texas Higher Education Coordinating Board and instructs the board to return next session with new proposals.

With the session over, Senate Higher Education Chairwoman Judith Zaffirini, D-Laredo, said the question was still “if, not when,” Texas might adopt such a system. The state’s current financing formula is based on course enrollment.

Even getting to this point has been difficult. Lawmakers rebuffed Paredes’ initial proposals for failing to target the right outcomes. His plan focused on course completions instead of graduations. After a new proposal was drawn, Branch had to convince House leaders that the approach was not related to controversial strategies pushed by Perry and the Texas Public Policy Foundation, a conservative research group.

“This is serious higher education reform that a lot of folks have had input on,” Branch said. Some lawmakers concerns were also diminished by including requirements in the bill that reports be submitted to a new higher education oversight committee created in response to the controversy.

The Coordinating Board’s last proposal would tie financing to total degrees awarded, as well as to degrees awarded to at-risk students and in critical fields. It also would reward each institution for surpassing predicted graduation levels or penalize them for falling short. Under this proposal, some of the institutions that produce the most graduates, including the University of Texas at Austin and Texas A&M University, appear to lose funding.

“How is it that an outcomes-based approach ends up diverting money away from the two most productive institutions in the state? That’s a puzzle,” said UT President Bill Powers.

Paredes says the coordinating has analyzed the changes universities could make to improve their results under this system, and he said, “Everything institutions need to do is good for Texas.”

The proposal was part of the original House version of the budget, but was omitted from the final product. Rep. Scott Hochberg, D-Houston, who chairs a key subcommittee, said it was included to express support for the idea — not for the methodology. “It’s such a mess — it’s sort of a black box,” he said. “I know the equations have been all laid out, but it’s hard to know what they all mean.”

Sen. Kirk Watson, D-Austin, concerned about pitting universities against each other in a “zero-sum game” during a time when budgets are shrinking, was among those who pushed for taking more time to find a better formula. “We want to do the appropriate, correct, visionary things to increase positive outcomes,” he said.

Watson emphasized the need for the Coordinating Board to offer options that rewarded outcomes without altering the baseline funding for universities, as well as options — like the one Paredes has pushed — that do make such changes. Other guidelines ultimately included in the bill were that no more than 10 percent of funding can be outcomes-based, and the coordinating board must ensure that it does not have a negative affect on graduate education.

Still, Paredes considers passage of the bill a success, and looks forward to strengthening his case for the most recent proposal and giving universities time to make necessary changes over the next two years. “We’ve essentially come to a concurrence that outcomes-based funding is something that we ought to do,” he said.

Catherine Frazier, a spokeswoman for Perry, said, “If it doesn’t accomplish what he wanted to, he’ll look forward to continuing to pursue that in the future.”

Meanwhile, referencing the public school finance debate that threw a wrench in the end of the regular session and is now continuing in a special session, Hochberg added, “It’s interesting to see that small variations in funding at the university are perceived to drive significant performance change, and large variations in funding at the K-12 level get written off as ‘Oh, stop griping, you’re just jealous.’”

Outcomes-Based Higher Ed Funding Bill Passes Senate

by Reeve Hamilton | Texas Tribune
May 24, 2011

A bill that could alter the state's formula funding for institutions of higher education to allow some of it to be based on outcomes such as graduation rates rather than simply enrollment cleared the Senate this evening.

Gov. Rick Perry has made outcomes-based funding one of his legislative priorities with regard to higher education, and the bill — House Bill 9 by House Higher Education Chairman Dan Branch, R-Dallas — appears to be moving toward the governor's desk.

The bill, which already passed the House, was kicked out of the Senate this evening with minor alterations. Now, the House must either concur with the changes or head to conference before it can finally be sent to the governor.

The concept has drawn some opposition. The Texas Faculty Association has expressed concern that it will create a need for institutions to boost graduation rates "by any means necessary," such as lowering standards and course requirements.

While HB 9 limits the portion of base funding that can be based on outcomes to 10 percent, it provides flexibility to the Texas Higher Education Coordinating Board, which supports the concept, to determine the specifics of the formula. The coordinating board would also be required to regularly review its outcomes-based system with institutions.

"A collaborative approach is key to the success of any outcomes-based funding initiative," said Senate Higher Education Chairwoman Judith Zaffirini, D-Laredo, who has been outspoken in her opposition of significant changes to higher education being pushed without public scrutiny and legislative input. "Instead of serving as an edict from on high, this bill would encourage cooperation between colleges and universities, the coordinating board and the Legislature. What's more, it would engage institutions in identifying and evaluating the most effective measures of student success."

Zaffirini, who sponsored HB 9 in the Senate, and Branch co-chair a recently created Joint Oversight Committee on Higher Education Governance, Excellence and Transparency. The bill would require the coordinating board to develop a report that identifies best practices for those issues and submit it to the committee.