Governor wants review of all state agencies’ computer systems

Timeline of events

Aug. 5 — Hardware failure; data restoration efforts begin.

Aug. 12 — Storage area network (SAN) failed; restoration efforts cease.

Aug. 13 — New SAN installed.

Aug. 14 — Restoration efforts resume.

Aug. 16 — KDHE Secretary Roderick Bremby has news conference to say that KDHE was “on road to recovery,” no data had been compromised or lost, and that 150 servers with 25 terabytes of data were affected.

Aug. 24 — KDHE announces it has retrieved 120,000 records from salt mines in central Kansas.

Aug. 26 — Bremby holds another news conference, also with representative of Xiotech, the vendor of the SAN. Bremby says Xiotech will pay for full cost of recovery and system upgrades and overtime wages.

Aug. 30 — KDHE announces it has access to electronic versions of birth certificates and other vital statistics records.

Source: KDHE timeline and statements by Bremby

Topeka ? The computer disruption that rocked the Kansas Department of Health and Environment for most of August is prompting a general review of computer systems in all state agencies.

Gov. Mark Parkinson said he wants agencies to check their systems, and Kansas Department of Administration Secretary Duane Goossen said a more in-depth look will be needed in the near future.

“There will be a bigger effort to figure out what happened,” Goossen said.

“What does it mean for other systems throughout the state, do other agencies have similar systems? These are legitimate questions. That kind of reviewing hasn’t been done yet, but it will be,” he said.

On Aug. 5, KDHE began experiencing technical problems that took down 150 servers, or 85 percent of the department’s total. Officials said the problems stemmed from a disk drive malfunction that caused the failure of a storage area network, or SAN, which is a central component of KDHE’s network.

KDHE workers had no electronic access to birth certificates, death certificates and other vital records that are commonly requested by the public.

A backlog of 8,000 requests for such records piled up, and the department had to retrieve records kept in storage in salt mines in central Kansas to manually process requests. The agency has temporarily doubled the staff of its vital statistics office, adding 50 employees from other parts of the agency.

It wasn’t until Monday, 26 days after the first problems arose, that KDHE said its last major information system was fully functional.

Now the questioning starts. Was there enough redundancy built into the system, meaning if one system failed is there an up-to-date copy of the data somewhere else.

KDHE uses tape as a backup to copy data and KDHE Secretary Roderick Bremby has said because of the massive number of records there is no need to build more redundancy for an event that Bremby said was simply never supposed to happen.

“We don’t anticipate this ever happening again,” he said.

But agency officials did say they are looking at improving the capability of restoring data more quickly.

“KDHE is looking at costs for a quicker solution,” said Kristi Pankratz, a spokeswoman for KDHE.

She also said that a KDHE vendor — OnTrack — was able to recover the data from hard drive disks and the tape backup was not needed.