Advertisement

LJWorld.com weblogs Seen it?

On this morning's downtime and our plans moving forward

Advertisement

Sunday night we lost access to a couple of our secondary domains, after we started to move off hardware owned by Knology and onto hardware owned by Mediaphormedia, our software development company. This was related to The World Company's sale of Sunflower Broadband late last year. The domains, which included HometownLawrence.com and our site admin, were inaccessible because of IP address conflicts. Last night while addressing those problems we had two issues that combined to screw up our news sites pretty badly. The first was with the NFS mounts our web nodes use to have access to our media and templates. The second was the loss of our memcached server. Trouble shooting the NFS problems initially hid the loss of our caching server. Without memcache, our sites were not able to keep up with the morning traffic and everything became unusably slow.

We are working on contingency plans to keep problems like this from hampering our ability to deliver your news the way they did this morning. Internally we are codifying and improving our emergency action plans. Our goal is to greatly improve our reaction times when faced with these kinds of problems.

Secondarily, we are making sure we're better communicating to you our problems and where we stand. By the end of the week we'll have designated places — externally hosted — for each of our sites so if we're ever down for an extended period of time you'll know where you can go to stay up to date. We're also making sure that we have off-site locations to post and report news so that we're not interrupting our coverage even if our primary sites are down.

Comments

Sigmund 3 years, 1 month ago

More unscheduled downtime this morning?

0

none2 3 years, 1 month ago

A sad topic came up on NPR this morning...

Obama is going to Silicon Valley to encourage those companies to hire more people. The problem is they do hire a lot -- that is off-shore resources. I think they said one Chinese out sourcer they (they I think being Facebook Inc. in this case) use has 1 million employees. As fewer of us understand technology, and fewer even get their foot in the door, we will be more dependent on foreigners to take care of us. One day the last American to leave a company may have to have a foreigner just to shut off high-tech light switches.

0

Sigmund 3 years, 1 month ago

Sigmund (anonymous) says… Sorry, html tags were removed. Trying again

In the Beginning was the Plan... And then came the Assumptions... And the Assumptions were without Form And the Darkness was upon the Face of the Workers And they spoke among themselves, saying, "It is a Crock of (prudish edit to protect your tender eyes, but rhymes with "ship") , and it stinketh!" And the Workers went unto their Supervisors and sayeth; "It is a Pail of dung and none may abide the Odor thereof." And the Supervisors went unto their Managers and sayeth unto them: "It is a container of Excrement and it is very strong, such that none may abide by it." And the Managers went unto their Director and sayeth' "It is a Vessel of Fertilizer, and none may abide its strength." And the Directors spoke among themselves saying to one another, "It contains that which aids Plant Growth and it is very strong." And the Directors went to the Vice-Presidents saying unto them, "It promotes Growth and it is very powerful." And the Vice-Presidents went unto the President and sayeth unto Him, "This new Plan will actively promote the Growth and Efficiency of this Company, and in these Areas in particular." And the President looked upon The Plan, And saw that it was good, and The Plan became Policy. This is how (prudish edit to protect your tender eyes, but rhymes with "ship") happens...

0

Sigmund 3 years, 1 month ago

LarryM (Larry Miller) says… "Sigmund: Now that you know all those big words, your next challenge is to find out how to use the equipment rather than just string all those words into a sentence. Good grief!"

What makes you think that for the last 10 years I haven't administered over 200 boxes in a mission critical environment using the latest in EMC/IBM/HP SAN environment consisting of (at last count) 95 terabytes of directly attached and networked attached storage?

Just curious.

0

Sigmund 3 years, 1 month ago

In the Beginning was the Plan... And then came the Assumptions... And the Assumptions were without Form And the Darkness was upon the Face of the Workers And they spoke among themselves, saying, "It is a Crock of , and it stinketh!" And the Workers went unto their Supervisors and sayeth; "It is a Pail of dung and none may abide the Odor thereof." And the Supervisors went unto their Managers and sayeth unto them: "It is a container of Excrement and it is very strong, such that none may abide by it." And the Managers went unto their Director and sayeth' "It is a Vessel of Fertilizer, and none may abide its strength." And the Directors spoke among themselves saying to one another, "It contains that which aids Plant Growth and it is very strong." And the Directors went to the Vice-Presidents saying unto them, "It promotes Growth and it is very powerful." And the Vice-Presidents went unto the President and sayeth unto Him, "This new Plan will actively promote the Growth and Efficiency of this Company, and in these Areas in particular." And the President looked upon The Plan, And saw that it was good, and The Plan became Policy. This is how happens...

0

Larry Miller 3 years, 1 month ago

Thanks for the info. Moving can be a real pain as things don't always refresh when you want them to. I thought the balance of tech to non-tech was about right.

Sigmund: Now that you know all those big words, your next challenge is to find out how to use the equipment rather than just string all those words into a sentence. Good grief!

0

Sigmund 3 years, 2 months ago

I do find quaint that the LJWorld site uses the old Cablevision/Broadband/Knology equipment using NFS (Network File System) mounts.

No FC (Fiber Channel), or even iSCSI (Internet Small Computer System Interface) over a dedicated network or subnet LAN (Local Area Network) or VLAN (Virtual Local Area Network)?

Very JV (Junior Varsity).

0

Sigmund 3 years, 2 months ago

This comment was removed by the site staff for violation of the usage agreement.

0

bearded_gnome 3 years, 2 months ago

I'm just relieved to know that the newly edited version of the Memcache has the word "ni--er" replaced with "robot." now I can sleep at night.

Jonathan, you guys feel like you're juggling chain saws down there?

0

Boston_Corbett 3 years, 2 months ago

I don't know which is the worse insult to the English language: the word Knology or the word Mediaphormedia

0

Catalano 3 years, 2 months ago

I thought it was Christine METZ?

0

Flap Doodle 3 years, 2 months ago

As least they can spell better than you, bub.

0

Fred Whitehead Jr. 3 years, 2 months ago

Like EBay, new, improved, upgraded and not worth a damn! Ain't technology wonderful in the hands of incompetants?

0

Commenting has been disabled for this item.