Schroedinger, revisited...

bad standards worse than no standards...but look better "on paper"

Uncertain Principles: Standards Are the Enemy of Achievement:

The US Chamber of Commerce has a education website, which provides "grades" for states based on various measures of their educational performance. One category is "Academic Achievement," based on the percentages of students scoring at or above grade level on the NAEP test. Another is "Rigor of Standards," which is a little fuzzier, but is based on official standards for graduation in that state-- state curricula, exit exams, and that sort of thing.

What's interesting about this is their correlation: if you click back and forth between the two, looking at their spiffy map, you can watch the colors change, and it jumps out (at least to me) that states with a lousy grade in one, tend to have a good grade in the other. That is, "Academic Achievement" seems to be negatively correlated with "Rigor of Standards."

This mostly holds up if you look at the numbers, too (you can download the data sets as Excel files if you click on the little question mark). Out of the 48 states with a grade reported for both categories, there were 17 states that got a D or F grade for "Rigor of Standards." 12 of those states (70%) had either an A or B in "Academic Achievement." Six of the eleven (55%) states with an A or B grade for "Rigor of Standards" got a D or F in "Academic Achievement."

Only one state (Massachusetts) managed an "A" in both, and only one state (Virginia) got a "B" in both. There were two double D's (Missouri and Rhode Island), and one D-F (Hawaii).

He concludes later in reply to a comment:

Dr. Kate: Maybe it's because states with high "rigor of standards" scores put a whole lot of emphasis on adherence to a specific curriculum, which (in my experience) is generally unbelievably broad--too broad for any reasonable student to possibly do anything other than memorize stuff. And if they also have to pass exams to graduate (giving the state a higher rigor score), then the teachers probably spend a lot of time teaching kids how to pass those tests (since it's more than likely that the teachers are graded, at least in part, on how well their students do on the tests).

This is, more or less, what I suspect is going on. The states that have good standards spend time teaching to those standards, and not the NAEP. The states without broad state-wide standards can spend more effort on teaching to the NAEP, and thus do well on those tests.

This is further complicated by differences in populations. States with big drops from Rigor to Achievement include California (A to F) and New York (A to C), and both of those states have student populations that will include large numbers of poor urban districts. States with big positive swings include North Dakota (D to A) and Wyoming (the lone F in Rigor, but a B in Achievement), which are more homogeneous.

But I think that the "teaching to the test" effect is a big part of it. Which end you think the problem is on is open to debate-- is it better to have students who do well on math and reading tests, but know nothing about science and history, or students who know a little bit of everything?