UVA professor and Core Knowledge board member Dan Willingham, who routinely graces this blog with his observations, is now blogging over at Britannica Blog. His first post is up today, and it’s a barn burner: How NOT to Evaluate Teachers. Plans to evaluate teachers based on standardized test scores are “fatally flawed,” he writes.
Obviously, the measure cannot be based on a one-time test score, because a student’s achievement is a product of (at least) his home environment, neighborhood, and prior schooling. So you must try to assess how much the student learns over the course of the year. But these “value added” measures bring lots of thorny statistical problems. For example, suppose your plan is to administer a test in the Autumn and one in the Spring, and to compare them to see how much students have gained. Well, some Autumn test-takers will have moved by the Spring. Can’t you just ignore those scores? No, because low-income students are more likely to move than high-income students, and low-income students tend to score lower. So if you ignore missing data, you’re biasing the estimate.
Dan lists other problems that he says are old stuff to statisticians, and concludes ”there’s nothing wrong with using value-added measures in research, with all the caveats of the method understood, as one in an array of tools to address a research question. But using it as a measure of an individual teacher’s efficacy is foolish.”
![[Slashdot]](http://slashdot.org/favicon.ico)
![[Digg]](http://digg.com/favicon.ico)
![[Reddit]](http://reddit.com/favicon.ico)
![[del.icio.us]](http://images.del.icio.us/static/img/delicious.small.gif)
![[Facebook]](http://www.facebook.com/favicon.ico)
![[Technorati]](http://technorati.com/favicon.ico)
![[Google]](http://www.google.com/favicon.ico)
![[StumbleUpon]](http://www.stumbleupon.com/favicon.ico)
Thousands of Ohio students who take state standardized tests aren’t part of the final grades reported by school districts, reports the Columbus Dispatch. And the state says it has no way of knowing whether school districts are removing students from the testing rolls appropriately. The paper reports an average of 4,000 students fell off the rolls for each of the 23 Ohio Achievement Tests given last school year.
Here’s how the process works: Students take a standardized test. A testing company grades their work, then sends scores to the school district. At that time, districts can remove students’ scores if they have withdrawn from the district or never attended in the first place. Even more students are stripped when districts report their scores to the state, because the Department of Education removes students who didn’t spend the entire academic year in the district.
Columbus schools dropped, on average, 11.4 percent of students from its test results, the paper reports. ”In doing so, passing rates climbed at every grade level, sometimes dramatically.” No surprise there, since students who don’t stay in school for the entire year tend not to do as well as those who stay put. But then there’s this line in the story: “Columbus schools cut fewer test scores from its rolls than its mobility rate would indicate it could.”
Are you thinking what I’m thinking?
Over at Flypaper, Mike Petrilli drops a bombshell. He was on a talk show this morning with, among others, Greg Toppo of USA Today and Melody Barnes of the Obama campaign.
We discussed the candidates’ education proposals, and all went according to plan until about halfway through the segment when Melody said that Obama wanted to look at different kinds of student assessments, including portfolios. Portfolios? As Greg and I said on the air, this was news. We’re not aware of the Obama camp ever saying before that portfolios might be part of the mix. I’m pretty sure I could hear Kati Haycock screaming from a few miles away.
As Mike points out, portfolios were found to be completely unreliable as large-scale accountability measures years ago. “Let me make a prediction,” writes Petrilli, ”either the Obama campaign will clarify that the Senator would consider portfolios on top of tests, not instead of them, or the McCain campaign will pounce on this issue and argue that it shows Obama to be weak on reform. Because one thing is for sure: embracing portfolios is a clear signal of an intention to roll back accountability.”
Portfauxlio Update: Michele McNeil at Campaign K-12 says Obama talkin’ about alternate assessment is nothing new and no big deal.
Update II: More from Petrilli. “I respectfully disagree with McNeil,” he notes. ”It still sounds to me that Barnes is talking about portfolios instead of standardized tests..” He suggests the Obama campaign could clarify: are you in favor of continuing standardized testing under NCLB, or not?
Update III: Over at TWIE, A-Rus has a fairly persuasive Obama quote from earlier in the campaign that sheds light on the Portfauxlio affair: “This doesn’t mean that we won’t have a standardized test, I believe children should master that skill as well and that should be part of the assessments and tools that we use to make sure our children are learning. It just can’t dominate the curriculum to the extent where we are pushing aside those things that will actually allow children to improve and will accurately assess the quality of teaching that is taking place in the classroom. This is not an either/or proposition, it is a both/and proposition, and that’s what we will be working on by fixing NCLB.”
Students from poor families in the Washington, DC area have made major gains on reading and math tests and are starting to catch up with those from middle-class and affluent backgrounds, a Washington Post analysis shows.
In Montgomery County, for instance, students in poverty have earned better scores on Maryland’s reading test in each of the past five years, slicing in half the 28 percentage-point gulf that separated their pass rate from the county average. They also have made a major dent in the math gap. In Fairfax County, another suburban academic powerhouse, such students have slashed the achievement gaps on Virginia tests.
In the DC proper, reading and math scores have risen since 2006, but fewer that half passed last Spring’s tests. “The results show substantial progress in the Washington area toward the law’s core goal: raising performance of disadvantaged children,” the paper reports. “Although concerns persist about the law’s emphasis on standardized tests, many educators say it has forced schools to concentrate more systematically on each struggling student.”
A controversial math curriculum in Georgia is being expanded to the state’s high schools. That’s raising the eyebrows and the ire of parents, who notes test scores in the Peachtree State haven’t exactly been lights out in math. The Atlanta Journal Constitution reports 38 percent of the state’s eighth-graders failed the state’s new, redesigned math exam, which was based on harder material.
“While parents and teachers expected some students to struggle with the new math, they were shocked by the high failure rates,” the paper notes.
After years of criticism that the state’s math curriculum was too weak, the Georgia Department of Education drastically changed the way students learn the subject. Officials adopted an “integrated” design, which weaves elements of algebra, geometry and statistics into a single math class, rather than teaching each separately. Elementary-school students use more hands-on activities to learn about numbers, geometry, multiplication and division. Middle school students learn some of the algebra previously taught in high school.
A parents group called Georgia Parents for Math wants more emphasis should be placed on math theory and basic concepts. “We have not come up with some foreign math,” Martha Reichrath, deputy superintendent for the state Education Department, tells the AJC. “It is an enriched math. Our students will do better with this math. I do believe we will be the national leader in math.”
“I’ve always told people, I have the best job in the world,” writes Susan J. Hobart, a National Board Certified Teacher, in the current issue of The Progressive.
Today, more often than not, I feel demoralized. While I still connect my lesson plans to students’ lives and work to make it real, this no longer is my sole focus. Today I have a new nickname: testbuster. Singing to the tune of “Ghostbusters,” I teach test-taking strategies similar to those taught in Stanley Kaplan prep courses for the SAT. I spend an inordinate amount of time showing students how to “bubble up,” the term for darkening those little circles that accompany multiple choice questions on standardized tests.
Yes, another one of those NCLB-is-destroying-education pieces written by a teacher. I predict that by the time the sun goes down, a smart guy like Jay Greene will have a line-by-line rebuttal on his blog explaining why this teacher is all wet. Why there’s no evidence that curriculum narrowing is occuring under NCLB. I’m sure it’ll make perfect sense. Heck, I’ll probably even agree with most of it.
Then I’ll remember my own 5th grade classroom, where I never had social studies textbooks–or time for social studies and science after our two hour ”literacy block” and 90-minute math workshop–but always had a fresh supply of shiny Kaplan test prep books every year. Where my students rarely got art, music or gym. Where we were trained by Teachers College to teach a unit on “test-taking as a genre” of literature. I’ll also remember the school assemblies and pep rallies where we tried to get the kids excited about the tests and shared all our “positive energy.” And I’ll remember one TFA corp member grad student, who was mandated to do two hours of test prep a day starting in September for the state tests in March.
Did I just dream all that?
I can do without the shrill rhetoric about the “assault on public education” and “one size fits all testing.” Still, every time I hear a veteran teacher describing with sadness how the job they loved became a joyless grind I find myself thinking, “Yeah, me too. ” How did this happen when testing, accountability and NCLB was what we were supposed to be doing all along anyway? Was I simply caught up in one of the greatest cases of mass hysteria since the Salem Witch Trials?
This spring, Rhode Island’s high school graduating class became the first in the nation to face performance-based assessments as a state-mandated requirement for earning a diploma.
“To be sure, no one is saying that Rhode Island’s trailblazing move means it’s time to start writing the obituary for machine-scored standardized exams,” notes a report in Education Week. “After all, even Rhode Island still uses them, and most experts agree that multiple choice is here to stay.” The piece offers a strong recap of the history of performance-based assessment over the last several decades, noting that “subjectivity of grading student portfolios and dissertation-defense-style presentations” has derailed previous attempts to work around standardized testing. Given the widespread disenchantment with NCLB and testing in general, it stands to reason, however that we’ll be reading lots more of this in the near future. Indeed, EdWeek reports eight other states have “expressed an interest” in Rhode Island’s initiative.
One of them is probably Ohio, where education officials have won a $1.3 million grant to explore alternative assessments, such as portfolios, senior projects, journals, small-group collaborations or teacher observation, the Cleveland Plain Dealer reports.
Every June, I and other teachers in my South Bronx elementary school, would go to great lengths to assemble portfolios of written work for students who were in danger of being held over. The point, as far as there was one, was to demonstrate that students who tested below grade level were in fact, making good progress, and that their test scores were not a reflection of their actual ability. It was an annual exercise in frustration and irrelevance. Say what you will about standardized tests, but rarely was a failing grade a poor indicator of a student’s ability. But more to the point, none of my students were in danger of being held over. Even those who scored a “1″ on their ELA exams (the lowest possible score) were shipped off to summer school and miraculously got up to speed in six weeks (I must have been some kind of lousy teacher not to have pulled that off myself) and “earned” promotion. In five years, the only time one of my students was held over was one whose mother insisted to the principal that he repeat the grade.
Social promotion, in short, is alive and well. Was my experience an anomaly? I’m curious as to the state of play elsewhere. Is social promotion happening in your school?
Recent Comments