bad standards worse than no standards...but look better "on paper"

[acroyear]

Uncertain Principles: Standards Are the Enemy of Achievement:

The US Chamber of Commerce has a education website, which provides "grades" for states based on various measures of their educational performance. One category is "Academic Achievement," based on the percentages of students scoring at or above grade level on the NAEP test. Another is "Rigor of Standards," which is a little fuzzier, but is based on official standards for graduation in that state-- state curricula, exit exams, and that sort of thing.

What's interesting about this is their correlation: if you click back and forth between the two, looking at their spiffy map, you can watch the colors change, and it jumps out (at least to me) that states with a lousy grade in one, tend to have a good grade in the other. That is, "Academic Achievement" seems to be negatively correlated with "Rigor of Standards."

This mostly holds up if you look at the numbers, too (you can download the data sets as Excel files if you click on the little question mark). Out of the 48 states with a grade reported for both categories, there were 17 states that got a D or F grade for "Rigor of Standards." 12 of those states (70%) had either an A or B in "Academic Achievement." Six of the eleven (55%) states with an A or B grade for "Rigor of Standards" got a D or F in "Academic Achievement."

Only one state (Massachusetts) managed an "A" in both, and only one state (Virginia) got a "B" in both. There were two double D's (Missouri and Rhode Island), and one D-F (Hawaii).

He concludes later in reply to a comment:

Dr. Kate: Maybe it's because states with high "rigor of standards" scores put a whole lot of emphasis on adherence to a specific curriculum, which (in my experience) is generally unbelievably broad--too broad for any reasonable student to possibly do anything other than memorize stuff. And if they also have to pass exams to graduate (giving the state a higher rigor score), then the teachers probably spend a lot of time teaching kids how to pass those tests (since it's more than likely that the teachers are graded, at least in part, on how well their students do on the tests).

This is, more or less, what I suspect is going on. The states that have good standards spend time teaching to those standards, and not the NAEP. The states without broad state-wide standards can spend more effort on teaching to the NAEP, and thus do well on those tests.

This is further complicated by differences in populations. States with big drops from Rigor to Achievement include California (A to F) and New York (A to C), and both of those states have student populations that will include large numbers of poor urban districts. States with big positive swings include North Dakota (D to A) and Wyoming (the lone F in Rigor, but a B in Achievement), which are more homogeneous.

But I think that the "teaching to the test" effect is a big part of it. Which end you think the problem is on is open to debate-- is it better to have students who do well on math and reading tests, but know nothing about science and history, or students who know a little bit of everything?

Comments

[uncle-possum.livejournal.com]

Um--rigor of standards correlates negatively with achievement? Higher rigor means lower "grades". This makes sense on its face: generally, if the standards are low, it's easier to get a high grade when measured against them. I don't see any problem. I feel I am missing something here.

Agree totally on the teaching to the test problem--a big mistake in most cases. (Says he who was in NY state during the heyday of the Regents' Exams--second semester in many classes we did nothing most days but take practice regents exams--not only didn't do much creatively, but generallyjust rehashed the same set of questions over and over.)

[acroyear70.livejournal.com]

The issue is "what defines performance" and "what defines improvement" which has dominated both the standards debates and the NCLB debates. Much of it is related to similar issues out there like declining standards (which the NCLB encouraged in some states, even as it raised a small subset of others), grade inflation, and the like.

And it comes down to the heart of "what defines education".

The issue is that conflict between a tough, rigorous standards set, and the specific items being tested as "required" by NAEP. There seems to be a contradiction between what some states consider essential and what the federal government deems essential (in NAEP's new role as report card for NCLB). What happens is that states that adhere strongly to their own rigorous program tend to do more poorly on the NAEP's tests, meaning either stuff the NAEP wants taught isn't being taught, or so much more is being taught that not all of NAEPs requirements are being met strongly enough. Vice-versa, schools/states that teach solely to NAEP do tend to have less rigorous graduation requirements.

None of this is related to "grades" in the individual student report-card sense. All of this is the objective view of the curriculum vs. the student performance on NAEP tests.

[acroyear70.livejournal.com]

at the heart of it all, of course, is that eternal conflict between improving performance by raising the bottom, improving performance by raising the middle, or improving performance by raising the top.

NCLB (and NEAP under this direction) works to raise the bottom (at the financial expense of schools being able to afford keeping programs that support the needs of the kids at the top - we're making our lazy geniuses even more lazy and it'll kill our future that way).

A rigorous program works by raising the middle and can continue to support the top as well.

The conflict between the two (and for some educators, its impact on the top which is something the current administration and congress utterly avoids, seeming to assume that the smart kids will take care of themselves, which is b.s. - we get lazy when we get bored and our grades turn to shite) is at the heart of much of the NCLB debates today and with that the way in which federal encouragement in the states has turned into federal interference.

[uncle-possum.livejournal.com]

Well, yeah. Thanks for clarifying.
The big problem is the attempt to control education nationally by people who don't really know what they are doing, and are too busy anyway. And standardized tests are easy to create, to administer, and to analyze. (The fact that they may not test anything significant, or may test the wrong things means little to the bureaucrats).

Of course, the issue of the 'smart' ones may, possibly, not be totally an accident. Smart people tend to think about things, and ask questions about things, and not necessarily believe authority--all traits not popular with the current government.

Or, we could merely be back into the attempt to "industrialize" and homogenize--don't trust the teachers, don't trust local government, eliminate handcrafts, eliminate local judgment of all kinds. ("Is this real, or did you make it" which comes up at all faires, all the time).

If the idea school is Mark Hopkins on one end of a log and a student at the other, then let's buy a bunch of logs.

[voltbang.livejournal.com]

Some years back my cousin moved from virginia to hawaii, both states you mention specificly. When he went from VA to HI, they moved him up a grade to account for the difference between the two systems. A year later, he moved back to VA, which moved him back two grades, to account for the difference in the school systems. It was too late, his academic career was wrecked. He went from a C+ student to struggling and not graduating a few years late, and eventually taking the GED to get into but not out of community college.

I suspect that looking at transfer students might provide some insight into the effectiveness of school systems.