LJWorld.com weblogs First Bell

The spin begins on ACT scores


It didn't take long after the release of state and national ACT scores for certain groups to start spinning the numbers for political gain.

Those numbers, you may recall from today's story, showed average scores in Kansas holding pretty much steady from the year before, and still well above the national average.

Of course, some might argue that, itself, was the "spin" from the Kansas State Department of Education. And their spin was quickly followed by a news release from the Lawrence school district, indicating the average scores here were well above the state average.

It didn't take long, however, for others to get in the game.

Mark Tallman of the Kansas Association of School boards posted today that the report was a mixed bag of results, but that overall scores in Kansas have been declining in rough correlation to the cuts in public school funding since 2009.

It's an interesting graph, but it calls to mind the old Latin phrase, *Post hoc, ergo propter hoc" — "after this, therefore because of this." Note that the Kansas line runs consistently above, yet parallel to, the national line, and both lines slope downward in about the same proportion, even though not all states have cut education funding by the same degree that Kansas has.

Meanwhile, the conservative think tank Kansas Policy Institute jumped into the game with a blog post of its own, reminding us that there's always a dark cloud behind every silver lining.

Dave Trabert, executive director of KPI, makes a few good and valid points as he cautions people about making comparisons between districts, or between states, or between district and state and national averages.

The problem, he notes, is that within any of those population groups, there are glaring disparities between scores of whites, blacks, Hispanics and others. So the racial composition of any district or state will tend to skew the results.

Based on that, he goes on to point out that when you compare racial groups across states, Kansas is really not as good as it looks at first, and that Texas, with an average ACT score of 20.9, is actually doing better than Kansas, with its average score of 21.8.

Texas, by the way, often is cited by Gov. Sam Brownback's administration as a model to emulate because it has no state income tax. It's also the state that ranks 49th in the nation for per-pupil spending on education, which reinforces one of KPI's key arguments about school spending — that more money doesn't yield better performance.

To make this point, KPI often tries to use data showing that Kansas has, in its view, outrageously high per-pupil spending on schools, but that it gets only mediocre, by the group's estimation, outcomes from it.

With regard to the ACT scores, however, KPI misses another important point. There are more variables at work than simply states and races. And that omission can lead people to believe, mistakenly, that if you simply adjust for the different racial makeup of a state or district, you can make valid comparisons.

But the ACT is not a random sample survey. The people who take the exam are not a representative sample of their ethnic subgroup in the state, or even of the college-bound members of their subgroups. ACT test-takers are an entirely self-selected group. Kids have a choice of college entrance tests they can take, and they sign up voluntarily — paying sizable sums of money — for the privilege of taking them.

In Kansas last year, where the ACT is the most popular among the two major college entrance exams, 75 percent of high school seniors took the test, even though only 65 percent of Kansas graduating seniors go on to college. Very few students here take the other major exam, the SAT.

Texas, by contrast, is a predominantly SAT state. Only about 37 percent of graduating seniors there took the ACT test last year, while about 57 percent of Texas' graduating seniors go on to college.

So you cannot say that the average ACT scores of any group of students within a state, or an individual district, reflects the quality of education being delivered to that group, or even the college-bound subset within that group. The numbers just are what they are.

So what's a parent, or taxpayer, to make of all this? I can only offer my purely non-expert advice:

Look at your kid's score, whether its ACT, SAT or anything else. Is it good enough to get into college, or qualify for a scholarship? If so, fine. If not, then turn off the TV, take away the video games, sit down with them and help them study harder. Take some practice exams. Then take the test again.

More Education News

  • First Bell Blog
  • Schools and Education news
  • Comments

    Dave Trabert 4 years, 10 months ago

    Peter makes an excellent point in recommending that every parent's primary concern should be their child's ACT score.

    For the record, KPI has never said that Kansas has "outrageously high per-pupil spending on schools." We just report the facts. We often compare spending and achievement in Kansas with other states in a very straightforward manner...examples of which can be found in our 2013 Public Education Fact Book http://www.kansaspolicy.org/ResearchCenters/Education/EducationDataWarehouse/d108076.aspx?type=view and Removing Barriers to Better Public Education http://www.kansaspolicy.org/ResearchCenters/Education/Studies/d94532.aspx?type=view

    We also posted additional analysis on ACT scores at http://www.kansaspolicy.org/KPIBlog/109642.aspx which shows that about 70% of what appears to be a decline in Kansas' ACT composite score is actually just a shift in the demographic components of those taking the test. The normalized ACT score only dipped from 21.87 to 21.84 and is actually higher than in 2009 and 2010.

    chootspa 4 years, 10 months ago

    Comparing achievements by spending, for example, leaves out the fact that other states have different spending formulas, different populations, and different needs. They all have different curriculums, too, yet we compare them all using a test that isn't necessarily aligned to those state standards.

    You don't have a natural experiment where two identical populations vary only in their level of school funding, and even then, you'd need to show how they spent differently and not just that one spent more than the other.

    Carol Bowen 4 years, 10 months ago

    I would have expected the average ACT score in Lawrence to be higher. This city is dominated by the university and its families. Children hear about scientific research and creative writing at an early age. Their knowledge should be a natural.

    Jeffrey Sykes 4 years, 9 months ago

    For the record, KPI always insinuates that spending is too high. Mr. Trabert, you may not say directly that spending is outrageously high, but your work constantly points in that direction. You have never said anything that indicates you think spending is correct or appropriate. KPI’s advertisements (for example the one mentioned here: http://joco913.com/news/steve-rose-misleading-ad-on-schools-doesnt-let-facts-get-in-its-way/) are designed to lead people to one conclusion: Kansas is spending too much on schools that are fundamentally flawed and should be replaced by private schools. If using numbers in the billions juxtaposed in just that way isn’t a way of saying that spending is outrageous, what is it?

    Fact: The only thing that KPI has consistently done is argue that funding doesn’t matter for achievement so KPI can continue to argue that Kansas should cut taxes and then cut funding for education when revenues drop.

    Devin Wilson 4 years, 9 months ago

    As a lifelong Kansan and a parent of 2 in a Kansas public school, I find it amazing that teachers can soldier through despite overcrowded classrooms, and for salaries more appropriate for the 1980's & 1990's. The rubber band has been overstretched. The fact that Kansas can and still does produce exceptional college-bound high school graduates speaks volumes to Kansas teachers' resilience, talent, and dedication. Just imagine what will happen when we restore funding! My hat is off to all of the Kansas teachers out there on the front lines. Mr. Trabert, I am spreading the word to other parents regarding KPI's policy, funding, and influence in the Kansas Legislature. KPI's influence doesn't stop at school funding, it goes much deeper. Check out the KPI's so-called "Economic Freedom Index" to see which representatives KPI is cozying up to. http://kansaspolicy.org/EconomicFreedomIndex/ Remember, your representatives represent YOU, not the interests of a lobby backed by large undisclosed corporate and private donors. I support our public schools and restoring funding to them at adequate levels. Some people I have talked to are even willing to pay more taxes to ensure better support of our schools. How crazy is that? Disclaimer: The opinions expressed here are my own, based on following the politics of school funding for 162 days now. Freedom does not equal privatization of everything that has made Kansas great. That is my policy. I have no corporate supporters or donors, or no fact book...yet.. My journey has just begun.

    Commenting has been disabled for this item.