[AccessD] OT: But only Partly

Had a similar issue to a new client a few years ago. They had an Access app
(that I didn't do) and the BE was on a NAS unit. It held all their job
orders and tracking info. It was corrupted and when I went in to try to
recover it from their backup tape - which they were reliably changing every
day and taking off site - there was nothing on any of the tapes. The backup
system (which I didn't set up) hadn't worked for over a year!

IIRC I asked everyone on these lists for references for recovery services. I
could only get 13 records recovered. So the recovery services didn't work.
They really lucked out though because two weeks before this happened they
had asked me to look at their access application and I had copied it and
their BE to my USB memory stick. So they got away with manually reentering
two weeks worth of paper files.

Now they have a solid backup system and it gets checked to ensure it is
actually doing something. I also have it set to send an email whenever it
fails so that makes it a little more obvious. 

I refer to backup systems as recovery systems with my clients so they
understand it up front. So far so good.

-----Original Message-----
From: accessd-bounces at databaseadvisors.com
[mailto:accessd-bounces at databaseadvisors.com] On Behalf Of Rocky Smolin at
Beach Access Software
Sent: Wednesday, March 21, 2007 9:03 AM
To: 'Access Developers discussion and problem solving'
Subject: [AccessD] OT: But only Partly

Are you backed up?  Are you SURE?



 http://www.signonsandiego.com/news/tech/20070320-0509-lostdata.html
 
JUNEAU, Alaska - Perhaps you know that sinking feeling when a single
keystroke accidentally destroys hours of work. Now imagine wiping out a disk
drive containing an account worth $38 billion. 

That's what happened to a computer technician reformatting a disk drive at
the Alaska Department of Revenue. While doing routine maintenance work, the
technician accidentally deleted applicant information for an oil-funded
account - one of Alaska residents' biggest perks - and mistakenly
reformatted the backup drive, as well. 

There was still hope, until the department discovered its third line of
defense, backup tapes, were unreadable. 

"Nobody panicked, but we instantly went into planning for the worst-case
scenario," said Permanent Fund Dividend Division Director Amy Skow. The
computer foul-up last July would end up costing the department more than
$200,000. 

Over the next few days, as the department, the division and consultants from
Microsoft Corp. and Dell Inc. labored to retrieve the data, it became
obvious the worst-case scenario was at hand. 

Nine months worth of information concerning the yearly payout from the
Alaska Permanent Fund was gone: some 800,000 electronic images that had been
painstakingly scanned into the system months earlier, the 2006 paper
applications that people had either mailed in or filed over the counter, and
supporting documentation such as birth certificates and proof of residence. 

And the only backup was the paperwork itself - stored in more than 300
cardboard boxes. 

"We had to bring that paper back to the scanning room, and send it through
again, and quality control it, and then you have to have a way to link that
paper to that person's file," Skow said. 

Half a dozen seasonal workers came back to assist the regular division
staff, and about 70 people working overtime and weekends re-entered all the
lost data by the end of August. 

"They were just ready, willing and able to chip in and, in fact, we needed
all of them to chip in to get all the paperwork rescanned in a timely manner
so that we could meet our obligations to the public," Skow said. 

Last October and November, the department met its obligation to the public.
A majority of the estimated 600,000 payments for last year's $1,106.96
individual dividends went out on schedule, including those for 28,000
applicants who were still under review when the computer disaster struck. 

Former Revenue Commissioner Bill Corbus said no one was ever blamed for the
incident. 

"Everybody felt very bad about it and we all learned a lesson. There was no
witch hunt," Corbus said. 

According to department staff, they now have a proven and regularly tested
backup and restore procedure. 

The department is asking lawmakers to approve a supplemental budget request
for $220,700 to cover the excess costs incurred during the six-week recovery
effort, including about $128,400 in overtime and $71,800 for computer
consultants. 

The money would come from the permanent fund earnings, the money earmarked
for the dividends. That means recipients could find their next check docked
by about 37 cents. 

	
Rocky
 
--
AccessD mailing list
AccessD at databaseadvisors.com
http://databaseadvisors.com/mailman/listinfo/accessd
Website: http://www.databaseadvisors.com