In recent months, Teach Plus had over 1,000 teachers review sample items from PARCC, one of the two testing consortia trying to create assessments aligned to the Common Core standards.
I say “trying” because in reading, the task is pretty much impossible. The standards specify things students should be able to do, but they contain almost no content. Thankfully, they do call for content-rich curriculum and explain that comprehension depends on broad knowledge, but they don’t provide the content-specificity needed to guide instruction or assessment.
Thousands of different curricula and assessments could be aligned to the standards, which would be fine if teachers were trusted to develop both. But teachers are not allowed to create the assessments—at least the ones that count. So it is entirely possible for a teacher to develop an “aligned” curriculum that does not prepare students for the content that shows up on the “aligned” assessment.
The result is an unfair assessment.
Test developers acknowledge as much, creating guidelines for item development that minimize knowledge as a source of “bias.”
Well, the 1,000 teachers who just reviewed PARCC think the stripping of knowledge did not go far enough:
Nearly all participants found that the PARCC passages were better quality than the passages in state tests, as they are previously published pieces (indicating that they are complex and demonstrate expertise in nonfiction). However, there was some concern students did not have “background knowledge, nor the vocabulary to understand” vocabulary within the texts. Their comments suggest that to assess students as accurately as possible, some portions may need to be edited for diverse learners, or those with limited background knowledge of certain content areas.
I understand why teachers would call for reducing the prior knowledge demands of the test—they are stuck in this crazy world of being measured with content that no one told them to teach. But let’s be honest: reducing the knowledge demand makes the test a little fairer; it does not make the education students are getting any better.
The knowledge bias can’t be avoided with tests that are not explicitly aligned to the curriculum. Without a curriculum that specifies what has been taught—and therefore what it is fair to expect students to know—test writers are reduced to a narrow band of banal topics (but even “Jenny goes to the market” demands some prior, unequally distributed knowledge).
The less the knowledge bias, the less the test reflects real-world comprehension. Outside testlandia, comprehension is not isolated from knowledge. An adult who can’t comprehend a newspaper is not considered literate. Broad knowledge is inherent in literacy. If we care about reading, as opposed to testing, we shouldn’t be creating tests that minimize knowledge demands. We should be developing a coherent instruction, assessment, and accountability system that builds broad knowledge and is fair because it tests what is taught.
Clearly, our nation’s policymakers need a crash course in reading. Once they understand that there is no such thing as general comprehension ability, maybe they’ll stop trying to hold schools accountable for developing it.
Fortunately, a great crash course is now available: Daniel Willingham’s latest book, Raising Kids Who Read: What Parents and Teachers Can Do. If policymakers read between the lines, they’ll see an awful lot they can do too.
As with Willingham’s previous books, this one is engaging, easy to read, and super informative. Here’s just a taste:
Most parents want their children to be solid general readers. They aren’t worried about their kids reading professional journals for butterfly collectors, but they expect their kids to be able to read the New York Times, National Geographic, or other materials written for the thoughtful layperson. A writer for the New York Times will not assume deep knowledge about postage stamps, or African geography, or Elizabethan playwrights— but she will assume some knowledge about each. To be a good general reader, your child needs knowledge of the world that’s a million miles wide and an inch deep—wide enough to recognize the titles The Jew of Malta and The Merchant of Venice, for example, but not that the former may have inspired the latter. Enough to know that rare stamps can be very valuable, but not the going price of the rare Inverted Jenny stamp of 1918.
If being a “good reader” actually means “knowing a little bit about a lot of stuff,” then reading tests don’t work quite the way most people think they do. Reading tests purport to measure a student’s ability to read, and “ability to read” sounds like a general skill. Once I know your ability to read, I ought to be able (roughly) to predict your comprehension of any text I hand you. But I’ve just said that reading comprehension depends heavily on how much you happen to know about the topic of the text , because that determines your ability to make up for the information the writer felt free to omit. Perhaps, then, reading comprehension tests are really knowledge tests in disguise.
There is reason to think that’s true. In one study, researchers measured the reading ability of eleventh graders with a standard reading test and also administered tests of what they called “cultural literacy”—students’ knowledge of mainstream culture. There were tests of the names of artists, entertainers, military leaders, musicians, philosophers, and scientists, as well as separate tests of factual knowledge of science, history, and literature. The researchers found robust correlations between scores on the reading test and scores on the various cultural literacy tests—correlations between 0.55 and 0.90.
If we are to increase reading ability, policymakers will have to accept that it takes many years to develop the breadth of knowledge needed for tests that are not based on a specific curriculum. We shouldn’t be stripping the knowledge demands out of our tests; we should be stripping the unreasonable mandates from our accountability policies. If we all focused on raising readers, we would spend far less time on testing and far more on building broad knowledge.
Young reader, building knowledge and comprehension, courtesy of Shutterstock.