By Peter Hancock

Earlier in this space I wrote about criticisms from certain corners that Kansas allegedly maintains low academic standards, especially in reading and math.

That generated some interesting discussion online, which I'll get to shortly. But first, an update:

The subject came up during the November meeting of the Kansas State Board of Education when board member Walt Chappell, a Wichita Republican, tried unsuccessfully to offer a motion to raise the “cut scores” on state assessments – the dividing lines that separate students who meet given performance standards from those who don't.

Since then, Chappell appeared on a television talk show in Wichita, “This Week in Kansas,” where he repeated many of the same criticisms. In response to that, as well as an email that Chappell sent to all the other board members discussing that TV appearance, Education Commissioner Diane DeBacker has put the subject on the board's agenda for its Dec. 11-12 meeting.

For now, we'll skip over with the question of whether such emails among elected officials constitute a violation of the Kansas Open Meetings Act. Suffice it to say that Chappell, and all board members, have been advised in the past not to do it. But given that next month's meeting will be Chappell's last on the board (he was defeated for re-election in the August primaries), let's just focus on the issue of the state standards.

In response to the earlier post, Dave Trabert of the Kansas Policy Institute, a conservative-leaning think tank based in Wichita, commented online:

“According to the US Dept. of Education, Kansas has some of the lowest standards in the country. They say 40 states have higher 4th grade Reading standards and 35 states have higher 8th grade standards. In fact, USDE says Kansas sets its scores for Proficient below what USDE considers to be basic.”

That statement, while technically accurate, warrants some further explanation.

First, it comes from an analysis that the federal agency conducts each year that attempts to translate scores from its own NAEP test to the scores on various state assessments. It should be noted that in the most recent (2009) mapping report, the U.S. Department of Education specifically warns against using the comparisons to make judgements about the relative quality of either the state or national tests:

“This report is not an evaluation of state assessments. State assessments and NAEP are developed for different purposes and have different goals and they may vary in format and administration. Findings of different standards, different trends, and different gaps are presented without suggestion that they be considered as deficiencies either in state assessments or in NAEP. The analyses in this report do not address questions about the content, format, exclusion criteria, or conduct of state assessments, as compared to NAEP. State assessments and their associated proficiency standards are designed to provide pedagogical information about individual students to their parents and teachers, whereas NAEP is designed to provide performance information at an aggregate level. Also, the analyses do not address any change in states’ assessments or proficiency standards that may have occurred after 2009.”

Second, according to officials at the Kansas State Department of Education, the state tests are designed around a specific set of educational goals and objectives established by each state. The NAEP test is not.

I've tried to think of various ways of explaining this, but the best I can come up with is a sports metaphor: If 50 people go to a golf course and play 18 holes, you'll get a set of scores that can be used to rank who performed the best and who did the worst. That, however, will in no way tell you which, if any, of the players is qualified to join the PGA tour.

The raw scores are just that – numbers that can be used to compare one player to another. Qualifying for the PGA tour requires scores that are benchmarked against some kind of external standard.

In educational jargon, NAEP is what's called a norm-referenced test. It aggregates scores within states so you can compare how the collective performance in one state stacks up against that of another. But it does not in any way tell you whether students in any of the states are learning what they should, or need, to learn.

The state exams, by contrast, are “criterion-referenced tests,” meaning they are benchmarked against an external set of standards. Like “par” on a golf course, they represent a performance level that a person competent in the subject ought to be able to achieve.

Third, although the standard for proficiency in Kansas does equate to “basic” on the NAEP test, that's also true for a lot of states. In fourth grade reading, for example, the only state whose standards for “proficient” meets the NAEP standard is Massachusetts. In eighth grade reading, all state standards are lower than the federal test's benchmark, but for some reason Missouri shows up as the highest.

Finally, even if one accepts the notion that Kansas has low standards relative to NAEP, that fails to explain why Kansas students routinely score high on the test.

Trabert argues that 40 states have higher fourth-grade reading standards than Kansas, and 35 states have higher eighth-grade standards.

But in 2011, according to the results of that year's NAEP exams, only eight other states had higher scores than Kansas in those categories.

Peter Hancock can be reached at 832-7259, or by email, phancock@ljworld.com.