Fatal Distraction: 7 Cautionary Tales for IT Managers

This week we dug into an archive of humorous and/or lamentable mistakes from recent times in IT to bring you 7 tales of what not to do if you want to remain employed.

From failing to ensure back-ups are working through to compounding a mistake by trying to cover it up, these real-life stories serve as an important reminder that our oversights and choices in IT can have real-life consequences.  That’s why proven processes are so important to adopt and follow (and managed services are so valuable because they’re so accountable to established processes).

Whether or not they are 100% accurate re-tellings, there is an element of truth in every cautionary tale.  The key points of these ones are:

• Disaster Recovery and Business Continuity need to be tested, possibly independently
• Compliance in relation to information security is easily demonstrated and should be routine
• Technology and automation to improve and make the business more competitive should be advocated-for and continuously sought

Read and enjoy (or cringe if the story reminds you of someone you know).

Fatal IT mistake No. 1: Slacking on the back-up

It was 10:30 on a Thursday night when IT pro Eric Schlissel’s phone rang.  On the line was the Chief Operating Officer of a midsize clothing manufacturer with whom Schlissel had never before spoken.

The COO, who found Eric’s phone number via Google, was frantic.  His plant’s ERP system had been wiped out by a virus, and they had a major deadline in the morning.

Schlissel, CEO of managed service provider GeekTek IT Services, jumped in his car and headed down to the manufacturer to handle the situation personally.

“Within three minutes of logging in, I realized there was nothing on the server,” says Schlissel.  “All the data files were gone, the database was gone, and the ERP software was nowhere to be found.  I told him this was no virus.  Someone had purged the system.”

It turned out a disgruntled IT contractor had enacted revenge by wiping the garment maker’s servers.  But worse news was yet to come.  The backups, which were supposed to run every night, hadn’t been working for a very long time.  The most recent data Schlissel could find was a year old, making it virtually worthless.

The company only survived because someone in accounting, who did not trust technology, had kept paper copies of everything.  It took Schlissel and his team six months to restore all the data by hand.

“It was a $10 or $12 million company, and they probably lost $2 million as a result of this,” he says.  “It was the most catastrophic IT disaster I have ever seen.”

The factory’s general-purpose IT guy, who was responsible for ensuring backups were made and tested, had simply forgotten about them.  He was on the unemployment lines soon after.

Failure to maintain backups is an all too common screw-up, and the mistake is often fatal to one’s job security, Schlissel says.

Read mistakes 2 through to 7 in this article from Infoworld.com.

We see great back-up policies that are not real-world tested all the time.  When we enter a managed IT services contract the back-up is the first thing we get right.  Then we keep on testing it as part of a comprehensive Disaster Recovery Plan to make sure that when it’s really required, it will work.

We offer cloud backup and site recovery services to ensure that no matter how and when your front-end technology fails, you can be back up and running in minutes.

You can read more about our data backup and recovery service features here.