[AccessD] OT: But only Partly

Hi Rocky:

If anyone thinks they have had a bad day, in the world of computers please
read this article. 

A copy of this article should be in every techs briefcase to be shown to a
client who does not want to incur the cost of testing the backup, and most
importantly, recover system or off site backup storage. 

Jim

-----Original Message-----
From: accessd-bounces at databaseadvisors.com
[mailto:accessd-bounces at databaseadvisors.com] On Behalf Of Rocky Smolin at
Beach Access Software
Sent: Wednesday, March 21, 2007 7:03 AM
To: 'Access Developers discussion and problem solving'
Subject: [AccessD] OT: But only Partly

Are you backed up?  Are you SURE?



 http://www.signonsandiego.com/news/tech/20070320-0509-lostdata.html
 
JUNEAU, Alaska - Perhaps you know that sinking feeling when a single
keystroke accidentally destroys hours of work. Now imagine wiping out a disk
drive containing an account worth $38 billion. 

That's what happened to a computer technician reformatting a disk drive at
the Alaska Department of Revenue. While doing routine maintenance work, the
technician accidentally deleted applicant information for an oil-funded
account - one of Alaska residents' biggest perks - and mistakenly
reformatted the backup drive, as well. 

There was still hope, until the department discovered its third line of
defense, backup tapes, were unreadable. 

"Nobody panicked, but we instantly went into planning for the worst-case
scenario," said Permanent Fund Dividend Division Director Amy Skow. The
computer foul-up last July would end up costing the department more than
$200,000. 

Over the next few days, as the department, the division and consultants from
Microsoft Corp. and Dell Inc. labored to retrieve the data, it became
obvious the worst-case scenario was at hand. 

Nine months worth of information concerning the yearly payout from the
Alaska Permanent Fund was gone: some 800,000 electronic images that had been
painstakingly scanned into the system months earlier, the 2006 paper
applications that people had either mailed in or filed over the counter, and
supporting documentation such as birth certificates and proof of residence. 

And the only backup was the paperwork itself - stored in more than 300
cardboard boxes. 

"We had to bring that paper back to the scanning room, and send it through
again, and quality control it, and then you have to have a way to link that
paper to that person's file," Skow said. 

Half a dozen seasonal workers came back to assist the regular division
staff, and about 70 people working overtime and weekends re-entered all the
lost data by the end of August. 

"They were just ready, willing and able to chip in and, in fact, we needed
all of them to chip in to get all the paperwork rescanned in a timely manner
so that we could meet our obligations to the public," Skow said. 

Last October and November, the department met its obligation to the public.
A majority of the estimated 600,000 payments for last year's $1,106.96
individual dividends went out on schedule, including those for 28,000
applicants who were still under review when the computer disaster struck. 

Former Revenue Commissioner Bill Corbus said no one was ever blamed for the
incident. 

"Everybody felt very bad about it and we all learned a lesson. There was no
witch hunt," Corbus said. 

According to department staff, they now have a proven and regularly tested
backup and restore procedure. 

The department is asking lawmakers to approve a supplemental budget request
for $220,700 to cover the excess costs incurred during the six-week recovery
effort, including about $128,400 in overtime and $71,800 for computer
consultants. 

The money would come from the permanent fund earnings, the money earmarked
for the dividends. That means recipients could find their next check docked
by about 37 cents. 

	
Rocky
 
-- 
AccessD mailing list
AccessD at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/accessd
Website: http://www.databaseadvisors.com