Schools officials: Testing results plagued with technical problems

? Parents and school officials may not get a routine report this fall showing how their students performed on state reading and math tests because a new online testing system was so plagued with technical problems that roughly one-third of the test results are considered invalid, state officials said Tuesday.

“We’re in uncharted waters here,” Scott Smith, director of the state’s assessment program, told the Kansas State Board of Education. “In the past, we’ve always brought the data to board in the fall, and from there it became final. But we’re nn a very different world here.”

The reading and math tests are required by the federal No Child Left Behind law for states and school districts to qualify for federal education funding. In addition, the state uses the results for determining whether schools should be accredited.

The results are also reported to the public in the form of Building Report Cards that show how well students performed. The results are broken down by individual schools, districts and the state as a whole, as well as by the students’ race, gender, ethnicity and economic background.

The problems this year resulted from the launch of a new online testing platform known as KITE — the Kansas Interactive Testing Engine — which is the computer-based system students log into to take the exams.

Marianne Perie, director of Kansas University’s Center for Educational Testing and Evaluation, which developed KITE, said that when the testing window opened March 10, the new system encountered numerous technical glitches, including at least two “distributed denial of service” attacks that shut down the entire system and prevented many students from being able to complete the tests.

In addition, she said, the main testing servers housed at KU didn’t have enough bandwidth to handle the volume of traffic, resulting in slow responses and some test questions not completely loading.

It took until April 11 for CETE to resolve all of those issues, Perie said, and only those tests taken after that date can really be considered valid.

But roughly one-third of all the math tests, and two-thirds of the reading tests administered this year were taken before April 11 — during what Perie called the “problem period.” And of those, Perie said, many were only partial tests because students were unable to complete all the questions.

For example, nearly 14 percent of the high school math tests taken during the problem period, and 13 percent of the high school reading tests, had fewer than half of the questions completed. By contrast, more than 90 percent of high school students tested were able to finish all of the questions after the technical problems were resolved.

Among the districts whose tests fell into the “problem period” were two of the state’s largest, Wichita and Topeka, Perie said. In Lawrence, tests were administered both before and after the problems were resolved.

“People talk about it in terms of validity,” Perie said. “Are the data valid? We will not produce data that is not valid. But my concern is also fairness. I can produce valid scores for some kids. Is that fair, if I can’t produce valid scores for every kid?”

Perie said she is recommending that the state only report scores from tests taken after April 10. But since that would eliminate some districts – and even some buildings within districts – she said the scores should only be reported on a statewide basis and not on an individual building or district level.

The state board had already agreed earlier that the 2014 scores would not count toward accreditation or placing schools on the list of those needing improvement. But the board had hoped at least to be able to have enough valid results in the fall so that school officials and the public could see how students were performing under the new testing system.

Smith, the Department of Education’s testing chief, said his staff will make a formal recommendation in July about what to do with the 2014 test results.