Core Knowledge Schools Outperform State Test Averages in Maryland Study

Five Maryland Core Knowledge schools being tracked as part of Johns Hopkins University study out-gained average Maryland schools on mandated state performance assessment tests, leading researchers to conclude that the thesis underlying the Core Knowledge Sequence is valid. 

"The majority of Core Knowledge schools posted three-year academic achievement gains in reading comprehension relative to their matched control peers as measured on the Comprehensive Test of Basic Skills. In addition, during the three year period of the study third grade students in Core schools showed greater gains on the Maryland School Performance Assessment Program than did their control schools or the mean of schools statewide," said Sam Stringfield, principal research scientist at the Center for Social Organization of Schools at JHU, who was joined by Barbara McHugh of Johns Hopkins in conducting the study. The report is the third of five due on a multi-school, multi-district implementation of the Core Knowledge Sequence in Maryland. 

"The general Core Knowledge trend was one of gains that clearly exceeded those of the state and of the demographically and gepographically matched schools controls," the report states. 

In the spring of 1994, the Abell Foundation of Baltimore, with the cooperation of Nancy Grasmick, the Maryland State Superintendent of Schools, organized an experiment in which six Maryland public schools would implement the Core Knowledge Sequence. The first two years of the implementation included grants from the Foundation, which were phased out by 1997. Each of the six pilot schools was matched with a demographically similar, within-district school, as a "control" against which it could be compared. One of the six original pilot schools was dropped from the study in the second year because its control school also adopted Core Knowledge. The study continues to monitor the remaining five. 

Achievement outcomes are being measured through two tests: the Comprehensive Test of Basic Skills (fourth edition) [CTBS] and the Maryland School Performance Assessment Program [MSPAP]. Neither test is designed for, nor deliberately aligned with, the Core Knowledge Sequence. However, the general knowledge theory behind Core Knowledge predicts that students should do increasingly well on any sort of test as their knowledge base grows. 

The MSPAP is performance-based assessment requiring extensive writing, problem solving and occasional teamwork among students. The state administers the test every spring to all third-, fifth- and eighth-grade public school students. The CTBS is a norm-referenced, multiple-choice test that has been shown by a variety of studies to have reasonable psychometric properties. The Maryland Core Knowledge Study used two CTBS subtests, Reading Comprehension and Mathematical Concepts and Applications, considered "higher order" tests of basic skills, which were administered each year. 

Two cohorts of students in each school have been tracked. The CTBS was administered to all first- and third-graders in each pilot and control school in the fall of 1994. These students were retested with the CTBS in the spring of 1995, in the spring of 1996 (when they were second- and fourth-graders) and again in the spring of 1997 as third- and fifth-graders. 

The degree to which the Sequence had been successfully implemented by pilot schools was examined and the study identified four relevant factors. Foremost were challenges in training non-Core-Knowledge-trained teachers. Second were problems associated with teaching split-grade classes in the face of Core Knowledge's grade-specific curricula. Third was a shortage of joint planning time, and fourth was a need for additional or replacement materials. These factors compounded two pre-existing ones: the conflicts between Core Knowledge topics and those required locally, which made it hard to teach all the Core topics, and the state's mandate to prepare students for the MSPAP. 

"The availability and use of common planning time and the care taken to introduce new teachers to the curriculum emerged as the clearest markers of the likelihood of a successful implementation," said Mr. Stringfield. 

"The most important lesson to take from the study is the need for thorough, careful, and ongoing implementation," he said. "When implementation is done well, Core Knowledge can clearly have a positive impact on student achievement, but it's tempting for schools to underfund longterm implementation." 

By the third year, two pilot schools were well on their way to institutionalizing Core Knowledge, according to Mr. Stringfield, one showed signs of weakening implementation (when a large number of teachers retired), and two faced circumstances that threatened their ability to integrate Core Knowledge. One of these latter two had embarked on a second reform program as well and was spending less time on Core, and the other school was floundering under numerous difficulties and threatened with direct state control, known in Maryland as "reconstitution." 

On the reading comprehension test given to third graders, school level changes from the fall of first grade to the spring of third grade showed a net mean gain of 4.7 NCEs*. The Core schools produced greater gains than their match control schools in four out of five cases. 

On the mathematics test, Core schools produced a net mean gain of 1.1 NCEs. As on the reading comprehension test, the control matched to the lowest implementing pilot school so outscored the pilot that, on average, Core Knowledge schools experienced less gain than control schools (+1.1 NCEs vs. +5.6 NCEs). 

At grade five, Core Knowledge schools produced somewhat higher gains in reading than control schools, +0.4 NCEs vs. -2.2 NCEs. In math, scores rose about evenly for both pilot and control schools, averaging +4.0 and +4.2 NCEs respectively. 

In 1997, both cohorts being followed were in grades tested by MSPAP, which reports school-level results, but not those of individual students. The Johns Hopkins researchers assumed that students who came to pilot schools after the study started did not choose to enroll their children specifically because of Core Knowledge.

"Our observations over three years consistently have been that virtually all new-to-the-school parents did not know their children's new schools were or were not Core Knowledge schools until after they had enrolled," Mr. Stringfield said. "In this context, MSPAP becomes a conservative test of the effects of the Core curriculum, because it presumably would be more difficult to show effects on measures that include students who did not receive the full treatment." 

MSPAP scores from 1994 were used as the pre-Core-implementation baseline. On average, the five Core Knowledge schools achieved higher gains in 1997 tests than the state did the state average school in all six test areas. The largest gains relative to all schools in the state were in writing, at +10.5 percentages, reading, at +8.6 percentages, and language, at +7.4 percentages. When all subtest areas are averaged together, Core schools outperformed the control schools by +5.6 percentages and all Maryland schools by +7.1 percentages. If the fifth pair of schools, pair E, the one containing the pilot school threatened with reconstitution, was dropped from the calculations, the Core schools show even greater gains: +8.5 percentages over control schools and +12.1 percentages over the average Maryland school. 

Looking at the MSPAP results of fifth graders, Core schools outgained the average state school in three out of the six areas. When the gains in all areas are averaged, however, there is no real difference between the Core schools and schools statewide. But if pair E is excluded, the Core schools produced a gain of +4.1 NCEs over the state average. 

"The Core Knowledge schools' more uniform gains in the first-through-third grade cohort would appear consistent with Hirsch's thesis that knowledge must build coherently over time," according to the report. "The younger cohort had experienced Core Knowledge curricula since first grade and in Hirsch's model would be expected to achieve the benefits of cumulative gains. 

"Because implementation is clearly possible and was associated with mean academic gains in most areas (and not associated with lowered mean performance in any area) Core Knowledge is a viable alternative for elementary schools considering options for school improvement," the report concludes. 

---

*Normal Curve Equivalents: The NCE scale is an equal distribution scale with a mean of 50 and a standard deviation of 21.06. NCE scores are equal to percentiles at the first, fiftieth, and ninety-nine percentiles.

back to previous page

Last updated: Fri, May 23 2008