The Core Knowledge Blog

New York’s Department of Education is beginning to measure the performance of thousands of elementary and middle school teachers based on how much their students improve on annual state math and reading tests, the New York Times reported last week. A joint letter to NYC teachers from Chancellor Joel Klein and UFT President Randi Weingarten explained the data is intended to “empower teachers with information useful in our teaching. In this same vein, the letter expressly prohibits the use of that information for evaluating teachers, in both annual ratings and tenure decisions.”

When the plan first came up in February, Ed Sector’s Kevin Carey wrote a much-discussed op-ed in the New York Daily News, comparing value-added data to the pioneering work done by maverick baseball general manager Billy Beane. The subject of Michael Lewis’ 2003 book, Moneyball, Beane has often managed to keep his small-market Oakland A’s competitive with deeper-pocketed teams by rejecting conventional baseball wisdom in favor of data-driven decision-making. “By crunching numbers without prejudice, Beane discovered that certain statistics that really mattered on the field, like on-base percentage, were being hugely undervalued in the player job market,” Carey wrote. “While scouts and other executives made decisions based on personal bias and flawed perceptions, Beane kept to the statistical bottom line.” Seen through this lens, the hope and promise is that we can find equivalents to on-base percentage in teacher performance that drive student achievement.

The Moneyball comparison, however, strikes me as a potentially dangerous analogy. Here’s why: Players are to baseball teams as students — not teachers — are to schools. Teachers succeed by getting the best performances from their students. Their closest counterparts in baseball are managers and coaches. Baseball executives like Billy Beane do not use data to help ordinary players over-perform. They use data to replace underperformers with overachievers.  To run a school like Billy Beane runs the Oakland A’s would mean regularly replacing low-scoring students with high-scoring students.

That would be one way to close the achievement gap.

The Moneyball analogy distorts the reality faced in education for simple reasons of supply and demand. At any given moment there are exactly 750 players on major league rosters. There are thousands more playing in the minors leagues, dying for the opportunity to replace a major leaguer. Beyond that, there are tens of thousands more high school, college and semi-pro players who were never even drafted, but would dearly love the opportunity to play professional ball at any level. The average big leaguer earns over $3 million. There aren’t many of us who wouldn’t be willing to be rigorously examined and evaluated for that kind of reward, thus there’s an unimaginably deep reserve of available talent from which to choose.

The average U.S. teacher earns about $50,000. There aren’t enough bad teachers to fill all the available positions. Thus the question, properly framed is not “How can we evaluate teachers the way Billy Beane evaluates ballplayers? It’s more like, “How successful would Billy Beane be enticing major leaguers to play American Legion ball?” The Moneyball analogy only works if a Billy Beane takes what he learns about player performance and uses it to get every baseball player in at every level in the Oakland A’s farm system to play at a major league level of performance. That’s what it would take to close the baseball achievement gap.

Back in July, the Fordham Foundation’s Mike Petrilli nailed this issue squarely on the head. “Yes, the research is quite clear that the quality of a student’s teacher has a greater impact on that student’s achievement than anything else that schools can control,” he wrote on Fordham’s Gadfly. Aware that demand exceeds supply, he asked, “Shouldn’t we be thinking about how to make average teachers more effective, too, and augmenting them via technology and other stratagems, rather than putting all our eggs in the “superstar teacher” basket?”

Exactly.

That brings us back to collecting data on teacher performance. You could write a pretty good history of public education through the lens of unintended consequences. The unintended consequence of accountability has been a self-defeating narrowing of curriculum, which unchecked will do more harm than good, and a slippery definition of proficiency, which can create a phony illusion of achievement. Knowing this, it might be a good idea to think through the unintended consequences of value-added evaluations now, not later, and how this data will change the dynamic of what’s happening in schools as opposed to spreadsheets.  While the data is not meant be used to inform formal evaluations, it’s hard to imagine it won’t shape opinions in a meaningful way. If an administrator finds out that a well-regarded teacher is underperforming based on the data, a new opinion will be formed.  It’s not unreasonable to suggest that new and negative impression might find its way into a formal evaluation at some point.

If value-added is used merely to hound mediocre teachers out of the business, we will have gained nothing. If it is used to determine what makes good teachers effective, and help those mediocre teachers move toward proficiency, then we have a shot.  It’s one thing for Billy Beane and his disciples to use data to justify releasing a guy who hits 25 homers a year, but only bats .220 with runners in scoring position. But you go to school with the teachers you have, not the teachers you wish you had. Improving teaching has to be the focus of any broad reform. We don’t have the luxury of sending teachers back to the minors until they’re ready for The Show.