Showing posts with label TPM. Show all posts
Showing posts with label TPM. Show all posts
Monday, July 19, 2010
Controversial Texas Projection Measure may again boost schools' ratings
Hundreds of Texas campuses are expected to receive high marks again this month, thanks to a state rule that counts some students who failed a TAKS test as having passed.
The Texas Projection Measure, introduced last year, predicts students' future TAKS scores in a given subject based on their current TAKS scores and on their previous year's average campus score. If a student fails this year's TAKS test but is predicted to pass in a future year, the school gets credit as if the student had passed this year. And that can boost a school's rating.
The Texas Education Agency is scheduled to release this year's ratings July 30.
The controversial formula came under attack during a recent Texas House subcommittee hearing. Rep. Scott Hochberg, D-Houston, gave a few examples of how students could bomb the TAKS but still get credit for passing.
Since then, TEA Commissioner Robert Scott has said the Texas Projection Measure may be eliminated or revised in 2011. One possible revision: allowing districts to decide whether to use the measure.
Supporters say the concept helps recognize the efforts of schools where students have made academic gains but still fall short of the passing standards.
According to the agency, 73 districts and 1,111 campuses saw their ratings rise from recognized to exemplary last year because of the measure. And 178 districts and 1,077 campuses climbed from acceptable to recognized because of the adjustment.
Here are some hypothetical North Texas examples of how the projection measure works and how it can benefit schools. All of the following schools saw a ratings boost last year because of the projection measure, but it isn't clear whether they'll benefit again this year.
SMITH HIGH SCHOOL, CARROLLTON-FARMERS BRANCH ISD
2009 rating: Recognized
Example: Ninth-grade math
Suppose a student answered only 20 of 52 questions correctly on the math TAKS (28 correct answers are needed to pass) but answered 32 of 42 questions correctly on the reading TAKS (26 are needed to pass).
Result: School gets credit for student passing the math test.
MAYNARD JACKSON MIDDLE SCHOOL, DALLAS ISD
2009 rating: Acceptable
Example: Eighth-grade science
Suppose a student answered only 15 of 50 questions correctly on the science TAKS (33 are needed to pass) but answered 42 of 48 questions correctly on the reading TAKS test (35 are needed to pass) and 34 of 50 questions correctly on the math test (29 are needed to pass).
Result: School gets credit for student passing the science test.
MEMORIAL ELEMENTARY SCHOOL, PLANO ISD
2009 rating: Acceptable
Example: Fourth-grade writing
Suppose a student missed all 32 questions on the writing test (17 correct answers are needed to pass) but got 27 of 40 questions correct on the reading test (27 are needed to pass) and 27 of 42 questions correct on the math test (27 are needed to pass).
Result: School gets credit for student passing the writing test.
BRIGHT ELEMENTARY, FRISCO ISD
2009 rating: Exemplary
Example: Third-grade math and reading
Suppose a student failed the math test by four questions (22 correct answers out of 40, 26 are needed to pass) and the reading test by just one question (20 correct answers out of 36, 21 correct answers are needed to pass).
Result: School gets credit for student passing the math test but not the reading test.
TAKS formula draws concerns
By Lindsay Kastner | Express-News
07/19/2010
Several area school districts expect good news when the state releases its preliminary accountability ratings this month, but whether those good ratings are the result of actual student performance has become the subject of much debate.
Richard Middleton, superintendent of North East Independent School District, said he has concerns about the use of the Texas Projection Measure, which helps many schools and districts boost their ratings by giving them credit for students who did not pass the Texas Assessment of Knowledge and Skills but who appear to be on track to pass in the future.
“I want to see our students improve, but I also want to have a credible assessment system that people believe in and understand,” Middleton said.
The TPM was approved for use as a growth measure, meaning it is intended to give credit for a student's progress over time. But it has come under fire since state Rep. Scott Hochberg, D-Houston, chairman of the House Appropriation Committee's education subcommittee, recently called into question the use of a formula that projects future success rather than taking into account past improvement.
“The accuracy of the TPM varies according to where you look but in some cases is not very accurate at all,” said Hochberg, who said he began looking into the system after realizing that a school hailed as a turnaround relied on the model.
Locally and across the state, schools and districts have benefited significantly from the TPM, which was put in place last year.
The number of Bexar County schools that achieved the state's highest rating, exemplary, more than doubled in 2009 over the previous year. But 63 percent of the exemplary schools would not have achieved that rank without the TPM, according to a San Antonio Express-News analysis.
It also helped about 10 percent of Bexar County schools avoid a failing rating from the state.
Projection models are generally considered accurate, said Pete Goldschmidt, a senior researcher at the National Center for Research on Evaluation, Standards and Student Testing.
“Where they're less accurate are for those kids for whom you want them to the most accurate,” he said, noting that poor performers can trip up the models.
In a recent letter to districts' administrators, Texas Education Commissioner Robert Scott wrote that he is considering changes to the use of the TPM next year, including modifying or suspending its use “to ensure that student performance is acknowledged and to ensure that the state accountability system remains transparent.”
Hochberg said those options aren't good enough.
“I don't think you design an accountably measure by throwing out a half a dozen options and saying, ‘Gee, which one sounds best to you,'” he said. “This is a difficult problem, and it will take difficult work to figure out.”
Students also had to correctly answer fewer questions on most tests to earn a passing score this year. Last year, for instance, students had to correctly answer 58 percent of the questions to pass the 10th-grade social studies test. This year, that threshold dropped to 50 percent.
“That's probably a little bit larger that we would normally expect, but is it completely out of range? No,” researcher Goldschmidt said.
State officials say test questions were harder this year, which accounts for the reduction in the number of questions students must correctly answer.
“Every year, we have to ensure that the overall difficulty level of the test is the same,” said Gloria Zyskowski, deputy associate commissioner of the Texas Education Agency.
At San Antonio ISD, where preliminary passing rates increased significantly, Deputy Superintendent Betty Burks credits a strong math curriculum and more time spent in science labs.
Zyskowski attributed this year's rating gains in part to legislation that requires students to pass their tests at certain grades before they can move on.
“They had a little more reading and math instruction because they were required to pass those tests to be promoted,” she said.
07/19/2010
Several area school districts expect good news when the state releases its preliminary accountability ratings this month, but whether those good ratings are the result of actual student performance has become the subject of much debate.
Richard Middleton, superintendent of North East Independent School District, said he has concerns about the use of the Texas Projection Measure, which helps many schools and districts boost their ratings by giving them credit for students who did not pass the Texas Assessment of Knowledge and Skills but who appear to be on track to pass in the future.
“I want to see our students improve, but I also want to have a credible assessment system that people believe in and understand,” Middleton said.
The TPM was approved for use as a growth measure, meaning it is intended to give credit for a student's progress over time. But it has come under fire since state Rep. Scott Hochberg, D-Houston, chairman of the House Appropriation Committee's education subcommittee, recently called into question the use of a formula that projects future success rather than taking into account past improvement.
“The accuracy of the TPM varies according to where you look but in some cases is not very accurate at all,” said Hochberg, who said he began looking into the system after realizing that a school hailed as a turnaround relied on the model.
Locally and across the state, schools and districts have benefited significantly from the TPM, which was put in place last year.
The number of Bexar County schools that achieved the state's highest rating, exemplary, more than doubled in 2009 over the previous year. But 63 percent of the exemplary schools would not have achieved that rank without the TPM, according to a San Antonio Express-News analysis.
It also helped about 10 percent of Bexar County schools avoid a failing rating from the state.
Projection models are generally considered accurate, said Pete Goldschmidt, a senior researcher at the National Center for Research on Evaluation, Standards and Student Testing.
“Where they're less accurate are for those kids for whom you want them to the most accurate,” he said, noting that poor performers can trip up the models.
In a recent letter to districts' administrators, Texas Education Commissioner Robert Scott wrote that he is considering changes to the use of the TPM next year, including modifying or suspending its use “to ensure that student performance is acknowledged and to ensure that the state accountability system remains transparent.”
Hochberg said those options aren't good enough.
“I don't think you design an accountably measure by throwing out a half a dozen options and saying, ‘Gee, which one sounds best to you,'” he said. “This is a difficult problem, and it will take difficult work to figure out.”
Students also had to correctly answer fewer questions on most tests to earn a passing score this year. Last year, for instance, students had to correctly answer 58 percent of the questions to pass the 10th-grade social studies test. This year, that threshold dropped to 50 percent.
“That's probably a little bit larger that we would normally expect, but is it completely out of range? No,” researcher Goldschmidt said.
State officials say test questions were harder this year, which accounts for the reduction in the number of questions students must correctly answer.
“Every year, we have to ensure that the overall difficulty level of the test is the same,” said Gloria Zyskowski, deputy associate commissioner of the Texas Education Agency.
At San Antonio ISD, where preliminary passing rates increased significantly, Deputy Superintendent Betty Burks credits a strong math curriculum and more time spent in science labs.
Zyskowski attributed this year's rating gains in part to legislation that requires students to pass their tests at certain grades before they can move on.
“They had a little more reading and math instruction because they were required to pass those tests to be promoted,” she said.
Thursday, July 09, 2009
Schools get credit for kids predicted to pass TAKS
By HOLLY K. HACKER and JEFFREY WEISS / The Dallas Morning News
Sunday, July 5, 2009
When the state announces school ratings this month, hundreds of schools are expected to claim higher marks – and part of the credit goes to new state rules that count some students as passing the TAKS test when they actually failed.
The state created school accountability ratings in 1993 to help parents gauge the successes and shortfalls of individual schools. But over the years, the state has made so many changes that it is a test in itself to figure out if a school is doing better, doing worse or holding even.
A new adjustment kicks in this year: the Texas Projection Measure. It allows schools to count students who failed the Texas Assessment of Knowledge and Skills as passing, as long as a complex formula shows that those kids are predicted to pass in a future year.
So many schools are likely to benefit from this latest academic "get out of jail free" card that it raises the question: At what point do the ratings become meaningless?
"We know that when the rules change every year and there are exemptions on top of conditions on top of projections, that really begins to water down the meaning of any of these labels," said Daria Hall, director of K-12 policy for the Education Trust, a nonprofit group that advocates for poor and minority students. Hall served on a federal panel that reviewed Texas' new model.
Here's how the new projection measure works:
Say a seventh-grader failed the math TAKS. The Texas Education Agency developed a statistical formula that predicts whether that student will pass the math test in eighth grade. The formula considers the student's math and reading TAKS scores, plus the average math TAKS score at his school.
If the student is predicted to pass, the school gets to count him as actually passing – even though he really failed.
School ratings are based on the test scores of all students, plus the performance of certain groups of students, including those who are black, Hispanic or poor. If just one group falls short academically, the whole school's rating can suffer.
An evolving tool
The projection formula was originally developed for the federal accountability system, called No Child Left Behind. Schools must show students are making "adequate yearly progress" as measured on standardized tests.
TEA officials note that a national panel of testing experts approved the model. They say it is not a perfect predictor but very accurate.
"We're looking for a way to recognize those schools where they've done a phenomenal job of increasing student performance, but they just haven't quite gotten every student over that last bar," said Criss Cloudt, a TEA associate commissioner.
But Hall, speaking for Education Trust, said one problem with Texas' model is that it gives schools credit based on future performance and doesn't go back and compare that to actual performance.
Case in point: A sixth- or seventh-grader who fails the TAKS could be projected to pass in eighth grade. The school receives credit for that. But suppose the student reaches eighth grade and does not pass as predicted. The school is not penalized. Instead, the Texas model looks ahead again – this time, determining whether the eighth-grader will pass the 11th-grade TAKS.
"From a school perspective, a student never has to actually be proficient. It's always projected into future grades," she said.
TEA officials say they will review the new model to see if students who were expected to pass did, in fact, pass.
"We see this very much as an evolving measurement tool," Cloudt said.
Local schools to benefit
Many North Texas schools expect higher ratings because of the new projection measure. But many other schools will show genuine student improvement, without the help of loopholes or fine print.
•Richardson ISD expects 16 of its 53 schools to benefit from the new projection measure.
•Dallas ISD expects 71 of its more than 200 campuses to benefit.
•Irving ISD expects two of its 20 elementary schools to jump from acceptable to exemplary, and a third campus to move up from recognized to exemplary. Irving hasn't had an exemplary campus in years.
•Of the 34 schools in Carrollton-Farmers Branch ISD, officials estimate that 17 will have higher ratings than last year. Nine of those schools will be aided by the new projection measure.
Cecilia Oakeley, Dallas ISD's evaluation and accountability director, said the new rule is fair to districts that serve students who struggle with poverty or language barriers.
"We're giving credit to students who are showing some growth but have not quite made the standard," she said. "They're on that trajectory."
That's the whole reason for using the model, TEA officials said. They also note that other changes in policy have made ratings harder on schools. For instance, the state has gradually phased in a stricter definition of dropouts that, for the first time this year, could lower some ratings.
Texas reports the percentage of students scoring at the higher "commended" level on the TAKS, but those rates do not affect the bottom-line school rating. The growth model changes also won't affect those rates, which can be useful for parents to see how many students at a given school are surpassing the basic passing standard.
State education officials say that school rating reports will make it clear what the pure TAKS passing rates are, compared to those boosted by the new projection model.
Changes not over yet
Major changes to the state's accountability system will continue, as the Legislature voted this spring to create a new rating system for 2013. Details are being worked out, but the new system won't rely so much on TAKS passing rates. Other measures, such as college readiness, will be included.
But it's unlikely the core of the debate will change: Ongoing tensions will remain between fairness and transparency, between stability and sophistication.
The TEA's Cloudt said there's a difficult balancing act: making the system understandable while recognizing that not all students come to school equally prepared.
While the promise was for clarity, the current system is based on formulas that few understand.
"It's so complex that very often parents are not fully informed on exactly what an individual rating means," said David Simmons, Richardson school superintendent.
But he said that confusion may be unavoidable.
"Any state-mandated, state-created accountability system is going to struggle to capture how individual campuses and districts perform on multiple measures," he said. "That may, to a great extent, be unattainable."
Donna Kent will be PTA president for the second straight year at Lake Highlands High School, which preliminary test scores indicate will need several exemptions to be ranked as recognized. Last year, the school was rated as academically acceptable.
She readily admitted that she was unaware of the various formulas behind the ratings, but said that the details were less important than the idea that her school was improving. She said the broad labels mean less than the information TAKS data provides, which helps schools target groups and students who need help in certain areas.
She said the situation is a lot like the Bowl Championship Series, used to select the top college football team. People complain about that system, too.
"We know we're going to have something, and we need to learn what we have and how to deal with it."
Staff writer Matt Peterson contributed to this report.
Sunday, July 5, 2009
When the state announces school ratings this month, hundreds of schools are expected to claim higher marks – and part of the credit goes to new state rules that count some students as passing the TAKS test when they actually failed.
The state created school accountability ratings in 1993 to help parents gauge the successes and shortfalls of individual schools. But over the years, the state has made so many changes that it is a test in itself to figure out if a school is doing better, doing worse or holding even.
A new adjustment kicks in this year: the Texas Projection Measure. It allows schools to count students who failed the Texas Assessment of Knowledge and Skills as passing, as long as a complex formula shows that those kids are predicted to pass in a future year.
So many schools are likely to benefit from this latest academic "get out of jail free" card that it raises the question: At what point do the ratings become meaningless?
"We know that when the rules change every year and there are exemptions on top of conditions on top of projections, that really begins to water down the meaning of any of these labels," said Daria Hall, director of K-12 policy for the Education Trust, a nonprofit group that advocates for poor and minority students. Hall served on a federal panel that reviewed Texas' new model.
Here's how the new projection measure works:
Say a seventh-grader failed the math TAKS. The Texas Education Agency developed a statistical formula that predicts whether that student will pass the math test in eighth grade. The formula considers the student's math and reading TAKS scores, plus the average math TAKS score at his school.
If the student is predicted to pass, the school gets to count him as actually passing – even though he really failed.
School ratings are based on the test scores of all students, plus the performance of certain groups of students, including those who are black, Hispanic or poor. If just one group falls short academically, the whole school's rating can suffer.
An evolving tool
The projection formula was originally developed for the federal accountability system, called No Child Left Behind. Schools must show students are making "adequate yearly progress" as measured on standardized tests.
TEA officials note that a national panel of testing experts approved the model. They say it is not a perfect predictor but very accurate.
"We're looking for a way to recognize those schools where they've done a phenomenal job of increasing student performance, but they just haven't quite gotten every student over that last bar," said Criss Cloudt, a TEA associate commissioner.
But Hall, speaking for Education Trust, said one problem with Texas' model is that it gives schools credit based on future performance and doesn't go back and compare that to actual performance.
Case in point: A sixth- or seventh-grader who fails the TAKS could be projected to pass in eighth grade. The school receives credit for that. But suppose the student reaches eighth grade and does not pass as predicted. The school is not penalized. Instead, the Texas model looks ahead again – this time, determining whether the eighth-grader will pass the 11th-grade TAKS.
"From a school perspective, a student never has to actually be proficient. It's always projected into future grades," she said.
TEA officials say they will review the new model to see if students who were expected to pass did, in fact, pass.
"We see this very much as an evolving measurement tool," Cloudt said.
Local schools to benefit
Many North Texas schools expect higher ratings because of the new projection measure. But many other schools will show genuine student improvement, without the help of loopholes or fine print.
•Richardson ISD expects 16 of its 53 schools to benefit from the new projection measure.
•Dallas ISD expects 71 of its more than 200 campuses to benefit.
•Irving ISD expects two of its 20 elementary schools to jump from acceptable to exemplary, and a third campus to move up from recognized to exemplary. Irving hasn't had an exemplary campus in years.
•Of the 34 schools in Carrollton-Farmers Branch ISD, officials estimate that 17 will have higher ratings than last year. Nine of those schools will be aided by the new projection measure.
Cecilia Oakeley, Dallas ISD's evaluation and accountability director, said the new rule is fair to districts that serve students who struggle with poverty or language barriers.
"We're giving credit to students who are showing some growth but have not quite made the standard," she said. "They're on that trajectory."
That's the whole reason for using the model, TEA officials said. They also note that other changes in policy have made ratings harder on schools. For instance, the state has gradually phased in a stricter definition of dropouts that, for the first time this year, could lower some ratings.
Texas reports the percentage of students scoring at the higher "commended" level on the TAKS, but those rates do not affect the bottom-line school rating. The growth model changes also won't affect those rates, which can be useful for parents to see how many students at a given school are surpassing the basic passing standard.
State education officials say that school rating reports will make it clear what the pure TAKS passing rates are, compared to those boosted by the new projection model.
Changes not over yet
Major changes to the state's accountability system will continue, as the Legislature voted this spring to create a new rating system for 2013. Details are being worked out, but the new system won't rely so much on TAKS passing rates. Other measures, such as college readiness, will be included.
But it's unlikely the core of the debate will change: Ongoing tensions will remain between fairness and transparency, between stability and sophistication.
The TEA's Cloudt said there's a difficult balancing act: making the system understandable while recognizing that not all students come to school equally prepared.
While the promise was for clarity, the current system is based on formulas that few understand.
"It's so complex that very often parents are not fully informed on exactly what an individual rating means," said David Simmons, Richardson school superintendent.
But he said that confusion may be unavoidable.
"Any state-mandated, state-created accountability system is going to struggle to capture how individual campuses and districts perform on multiple measures," he said. "That may, to a great extent, be unattainable."
Donna Kent will be PTA president for the second straight year at Lake Highlands High School, which preliminary test scores indicate will need several exemptions to be ranked as recognized. Last year, the school was rated as academically acceptable.
She readily admitted that she was unaware of the various formulas behind the ratings, but said that the details were less important than the idea that her school was improving. She said the broad labels mean less than the information TAKS data provides, which helps schools target groups and students who need help in certain areas.
She said the situation is a lot like the Bowl Championship Series, used to select the top college football team. People complain about that system, too.
"We know we're going to have something, and we need to learn what we have and how to deal with it."
Staff writer Matt Peterson contributed to this report.
Subscribe to:
Posts (Atom)