Don't let this happen to you...Backup, Backup & Backup!

wildviper · October 15, 2007

I am stickler for backing up. I have had too many people around me complain about data loss and so on.

My backup strategy was to keep 2 full rez copies of ALL my pictures on 2 separate hard drives. Updated regularly to ensure they both were synced. I also have DVDs of these pictures stored safely. However, due to the inherent slow nature of making DVDs, I had fallen behind. I was 1 year behind with about 4,000 pictures taken in one year. Also, smugmug account has a lot of them..but smugmug doesn't keep my originals as originals...they do some modification.

You would have thought I was covered! You would be slightly Wrong!!!! :scratch

Few days ago, my computer crashed. It took down 3 of my drives that were in there. 2 of them were full of ALL my pictures(like I said above). 3rd drive was for my mp3s and home movies. Can you imagine my shock when I started the computer to read "Disk Read Error" and "OS can't be found" and so on???

After spending 6 days working 8 hours or so on the computer, I was able to salvage all my pictures. Phew!!! :barb THe only reason I was able to save the pictures was because I knew what NOT to do in case of a crash. (I am writing my experiences down and will post a link to a forum where I participate with that info).

So, if any of you think that you are safe with backups on two separate drives, think again. My drives were made by Maxtor and Seagate(the latest one). And I had made NO changes to the computer. I have not been able to uncover the specific reason this happened, but, suffice it to say, it scared the hell out of me.

I began thinking...what if I had lost my year's worth of pictures? What if I had lost it all due to DVDs not being able to be read???? What will i show my kids, my grandkids when I have them ??????? Sorry, but I only have four or five pictures to show you??!!??

While digital is great, like many of you, I do not print a majority of the pictures. This is the BIG problem that everyone should be fully aware of. In fact, I would think that camera manufacturers should put in an informational sheet with all these digital cameras.

SO this is a post to warn you all. Don't think you are safe.

Backup, Backup and Backup.

Here is my new strategy: I have a PDF that hopefully I can attach here.

Have 3 full replicas of All my pictures on 3 separate drives by 3 different manufacturers. 1 External that is only connected to backup.
Have 1 set of DVD backups of JPEG versions of these pictures. Store externally.
Have 1 set on another HD of JPEGs only again. Store externally.
Regular backups and keep to the schedule.

That may sound like overkill for many of you, but, after having 3 drives crash at the same time is kinda scary.

I will attach a PDF chart that I made so that it may help someone else. Let's see if this allows me to attach.
Edit: I have to reduce the file size...let's see

DavidTO · October 15, 2007

Cool PDF!

I've been using JungleDisk with Amazon S3. Not so speedy (hah!), but safe as can be, IMO.

I also have a local back up. I refuse to use optical media. Don't trust it, don't like it. I'd rather have 10 HDs than one DVD.

But that's just me.

wildviper · October 15, 2007

DavidTO wrote:

Cool PDF!

I've been using JungleDisk with Amazon S3. Not so speedy (hah!), but safe as can be, IMO.

I also have a local back up. I refuse to use optical media. Don't trust it, don't like it. I'd rather have 10 HDs than one DVD. But that's just me.

Thanx. I had to do this so that everyone in my family understands the steps necessary. I am planning to make it very simple for everyone to move images from laptops to desktop. so far I have the process down to 2 clicks.

I thought about online storage, but, my Jpegs alone are 25 gigs! They are not low rez though. I can't even imagine the time it would take to upload that. So I gave up on that idea. It appeals to me though if they offered "Send us DVDs/HDs and they will upload the pictures". Then I don't have to worry about uploading 25gigs!!!

DavidTO · October 15, 2007

I uploaded 60 GB. I'm not saying it's the best solution, but it worked for me!

The thing I'm excited about for backing up the family is running Time Machine in Leopard on a networked drive via the wireless network. Set it, and forget it! :ivar

claudermilk · October 16, 2007

Interesting story. Glad you wer eable to salvage your data. One thing is not clear though.
You said you had everything backed up to two drives, yet you lost your backups?

How? Or was it just two copies of files all on the main PC?

Might I make another suggestion? It sounds like your OS and data files are all on the same volume. Separate them. Get a small, fast drive reserved solely for loading the OS and programs on. Then, when that dreaded "OS not found" error pops up, you only have to deal with wiping that drive. Your data is on a completely separate, untouched drive waiting for the OS to come back and find it again.

wildviper · October 16, 2007

claudermilk wrote:

Interesting story. Glad you wer eable to salvage your data. One thing is not clear though.
You said you had everything backed up to two drives, yet you lost your backups? How? Or was it just two copies of files all on the main PC?

Might I make another suggestion? It sounds like your OS and data files are all on the same volume. Separate them. Get a small, fast drive reserved solely for loading the OS and programs on. Then, when that dreaded "OS not found" error pops up, you only have to deal with wiping that drive. Your data is on a completely separate, untouched drive waiting for the OS to come back and find it again.

I had lost everything since my main files and backup files were on the 2 drives. I only had DVD backups and they were at least a year old. So I would have lost 1 year data + any DVDs that would give me the dreaded CRC checksum errors.

ANd thanx for the suggestion of the OS being on a separate drive. I had it on separate partition, but not separate drive. Will have to do that.

mrcoons · October 16, 2007

DavidTO wrote:

Cool PDF!

I've been using JungleDisk with Amazon S3. Not so speedy (hah!), but safe as can be, IMO.

I also have a local back up. I refuse to use optical media. Don't trust it, don't like it. I'd rather have 10 HDs than one DVD. But that's just me.

I had not heard of this David but I think I will check it out. I have 1 internal large capacity drive and 3 externals but one external drive died last month. This drive was over 5 years old so I had moved most of the data off it and only used it for downloading video. A good thing too.

But this had made me start thinking about photo storage a little harder. I too do not trust optical media because of the recent changes in optical hardware. No telling how long existing CD/DVD's will be 'readable'.

I'm also leary of external drives. Once I fill a 500GB drive or once it gets 3 to 4 years old the prospect of copying the contents to a newer drive is not something I'd look forward too.

So I think I'll give JungleDisk a good hard look. Thanks again for the info.

LiquidAir · October 16, 2007

It was probably a glitch the power supply that took down both your drives at the same time. I much prefer using USB drives for backup because they are disconnected most of the time. Live drives are much more vunlerable both to power issues (e.g. lightning striking a power pole) and software issues (e.g. malicious viruses).

I have shifted to using a network attached storage device for my nightly backup (a Synology DS-207 configured for RAID 1) because it lets me get at my data from anywhere, but I still back it up weekly to a USB drive which I keep offsite.

LiquidAir · October 16, 2007

claudermilk wrote:

Might I make another suggestion? It sounds like your OS and data files are all on the same volume. Separate them. Get a small, fast drive reserved solely for loading the OS and programs on. Then, when that dreaded "OS not found" error pops up, you only have to deal with wiping that drive. Your data is on a completely separate, untouched drive waiting for the OS to come back and find it again.

Yes indeedy. For the OS, the best strategy I know of is to get small fast drives. Put one in your machine, the other in an external USB box. If your OS drive goes south, all you have to do is put the external one in its place and you are up and running again.

Foochar · October 16, 2007

claudermilk wrote:

Interesting story. Glad you wer eable to salvage your data. One thing is not clear though.
You said you had everything backed up to two drives, yet you lost your backups? How? Or was it just two copies of files all on the main PC?

I'm also trying to figure this out

, I work in I.T. and although I've seen plenty of cases where a drive fails in such a way that the data on that one drive is irrecoverable, the odds are pretty long that multiple drives would fail at the same time... There are some failure modes that could cause this, the big one being a storage controller failure that ends up causing data corruption on both drives, or as LiquidAir pointed out a power supply glitch, but both of those are pretty rare. And if it was either of things wildviper needs to seriously consider replacing some hardware in addition to the failed drives, because there is no telling when the problem might show up again. If it was just that the O.S. files that became corrupt (I've seen this many times and it would give an error similar to what wildviper describes) and the hardware was ok, it should have been possible to do a parallel install of windows (install to C:\WinRetry on any other non default path) and all the data on the drives should have still been there, so long as you didn't format or repartition the drives.

That being said any storage method that keeps all the backups in the same location is inherently flawed, what happens if your house catches fire, is leveled by a tornado etc. I'm guilty of this myself, and I really need to get back in the habit of keeping a copy of everything on a USB drive at work as well as at home, or take a serious look at jungledisk, even if it does mean a month of uploads at cable modem speeds. At the end of the day you've got to examine your backup strategies in light of the value you place on your images, the level of risk there is to those images and the cost to reduce that risk.

Travis · October 16, 2007

LiquidAir wrote:

It was probably a glitch the power supply that took down both your drives at the same time.

6 months ago I had the pleasure of having my power supply pop. It took with it the video card, motherboard, and 2-250gb hard drives. Luckily the computer was fairly new and I only lost a couple months worth of photos but it shocked me nontheless. I made the idiotic assumption that I was safe because I had the primary har drive mirrored by the second. Doh! Doesn't help much when both fry. Since then I've moved to external hard drive AND DVD backups. It sounds like overkill but It is well worth the potential risk.

dmmattix · October 16, 2007

Exactly the point I have been trying to make over in another thread but you know DvDs are evil:D

Regards,

Mike

Who has external hard drives and just bought a 100pack of DvD-Rs...

Dogdots · October 16, 2007

Wildviper I couldn't get your chart up. Any ideas? I'm really interested in seeing it.

wildviper · October 16, 2007

Dogdots wrote:

Wildviper I couldn't get your chart up. Any ideas? I'm really interested in seeing it.

I am not sure why not. No one else seems to have that issue. Maybe your Popup blocker??

In anycase, send me a PM with your email address and I can attach it and send it.

++++++++++++++++++++

I am not sure why and how this happened to me. This is Day 7 or 8 now that I am trying to figure this out. I have run a lot of burn-in type tests and it is all ok.

The only one that crashes is SpinRite. It gives me some strange error and says it can't continue anymore. Now, either SpinRite is very advanced and knows something that it doesn't want to share with me...or it is being tripped up by something small.

It maybe my IDE controller on the board, but I have done soo many read/write tests and nothing(except for SpinRite).

I just want to bring people's attention to this issue. Don't think you are safe cause you got a RAID or you got multiple drives in your system. Things can go wrong....3 drives for me copped out at the same time! Who would have thought??

Edit: As much as I do not like the idea of uploading all pictures to a server due to time constraints..I think I may. Time to research JungleDisk and others.

NavyMoose · October 16, 2007

wildviper wrote:

I am not sure why not. No one else seems to have that issue. Maybe your Popup blocker??

In anycase, send me a PM with your email address and I can attach it and send it.

++++++++++++++++++++

I am not sure why and how this happened to me. This is Day 7 or 8 now that I am trying to figure this out. I have run a lot of burn-in type tests and it is all ok.

The only one that crashes is SpinRite. It gives me some strange error and says it can't continue anymore. Now, either SpinRite is very advanced and knows something that it doesn't want to share with me...or it is being tripped up by something small.

It maybe my IDE controller on the board, but I have done soo many read/write tests and nothing(except for SpinRite).

I just want to bring people's attention to this issue. Don't think you are safe cause you got a RAID or you got multiple drives in your system. Things can go wrong....3 drives for me copped out at the same time! Who would have thought??

Edit: As much as I do not like the idea of uploading all pictures to a server due to time constraints..I think I may. Time to research JungleDisk and others.

I work in IT as well and I'm a firm believer in having multiple redundancies

On my Photoshop computer, I have two Seagate 500Gb drives in RAID 1 configuration. To those non techies, this means one drive is the mirror of the other. I have an external drive that I backup my photos and LR catalogs to using NT Backup. Everytime I do anything to photos, I run an incremental backup. Each month I run a full backup.

I also keep a seperate partition for my photographs. The OS and apps are on the C:\ drive and the photos are on the

\ drive. In the event of an OS failure, I can rebuild and my photos will be relatively safe.

I also use Amazon S3 and I will echo how slow it is, especially with a cable modem. It is the ultimate in offsite storage, and very low cost.

Navy Moose

claudermilk · October 17, 2007

wildviper wrote:

I had lost everything since my main files and backup files were on the 2 drives. I only had DVD backups and they were at least a year old. So I would have lost 1 year data + any DVDs that would give me the dreaded CRC checksum errors.

ANd thanx for the suggestion of the OS being on a separate drive. I had it on separate partition, but not separate drive. Will have to do that.

OK makes more sense now. So, now we've all learned that 2 copies of the files all on internal drivers really are not "backups." Backups have to be offline, and separate from the PC. This is what both makes them effective & inconvenient. Removable/external drives are the most convenient compromise as they put online & pulled offline quickly & painlessly.

Dogdots · October 17, 2007

Navymoose what do you mean by saying your photos will be relatively safe?

Let me know if I have this right.

--I hook my external up to the d drive so the OS doesn't effect it. So if anything happens to the OS when I'm hooked up it won't effect my photos cuz they are hook to another drive?

Oops---typical womans question---running in circles. Please forgive
me

----Mary

denisegoldberg · October 17, 2007

NavyMoose wrote:

I also use Amazon S3 and I will echo how slow it is, especially with a cable modem. It is the ultimate in offsite storage, and very low cost.

I suspect that Amazon has the upload speed capped somehow. If I upload the same files to smugmug that I do to S3 and compare the times, S3 takes much longer. The only time it was really an issue for me was when I first backed up my files.

--- Denise

mrcoons · October 17, 2007

denisegoldberg wrote:

I suspect that Amazon has the upload speed capped somehow. If I upload the same files to smugmug that I do to S3 and compare the times, S3 takes much longer. The only time it was really an issue for me was when I first backed up my files.

--- Denise

I agree. Shouldn't be a major problem for me once I get my old data uploaded though. Which it appears that will be in a few days!:D

wildviper · October 17, 2007

mrcoons wrote:

I agree. Shouldn't be a major problem for me once I get my old data uploaded though. Which it appears that will be in a few days!:D

Well I just calculated the cost of Jungledisk service through Amazon S3. For 100 Gigs with no transfers it is $15 per month. $15 x 12 = $180 per year with no transfers.

I just bought a Esata/USB 2.0 500 Gig External HD for $140. Internals are even cheaper.

Why would this service make sense for paying so much? It takes time to upload, I can't access it anywhere(Cybercafe and so on)due to the software limitation.

I am still researching.

BradfordBenn · October 17, 2007

Glad you got your stuff back.

I had a similiar problem on my work laptop. I am pretty good about keeping data on the network drive, backup before every trip, upload critical stuff every night at the hotel (not a photographer but still deal with 10 to 100MB files)... but one time my hard drive booted up with the dreaded "OS not found" error. One could actually hear the drive scraping while spinning. Oh <insert expletive of choice here>!

So I went to a coworker's laptop and downloaded Knoppix (http://www.knoppix.org/) and created a bootable CD. This allows one to launch a version of Linux from the CD drive and mount the internal drive (without having to reformat it). I then hooked up an external USB hard drive. I then copied all the drive contents to it I could, the critical stuff went to a USB flash drive as well.

It saved the trip, all I lost was a few e-mails. I have read that it can be done from a USB drive but I have not tried. It might be worth keeping that CD around.

Sorry to hear about the problems, glad you got it back.

[Deleted User] · October 17, 2007

wildviper wrote:

Well I just calculated the cost of Jungledisk service through Amazon S3. For 100 Gigs with no transfers it is $15 per month. $15 x 12 = $180 per year with no transfers.

I just bought a Esata/USB 2.0 500 Gig External HD for $140. Internals are even cheaper.

Why would this service make sense for paying so much? It takes time to upload, I can't access it anywhere(Cybercafe and so on)due to the software limitation.

I am still researching.

Maybe an external USB HD is $40 cheaper but Jungledisk offer something that the ext HD can't offer, external storage as in not stored at your house. It does not matter what hard disk configuration you have at home, you have a fire , flood or robbery it won't matter how many hard disks you back up to if you don't have the disks to recover from.

In the UK off site storage\backup of all important data is one of the major requirement when looking for business insurance.

Tim

Tim

anderiv · October 17, 2007

wildviper wrote:

Well I just calculated the cost of Jungledisk service through Amazon S3. For 100 Gigs with no transfers it is $15 per month. $15 x 12 = $180 per year with no transfers.

I just bought a Esata/USB 2.0 500 Gig External HD for $140. Internals are even cheaper.

Why would this service make sense for paying so much? It takes time to upload, I can't access it anywhere(Cybercafe and so on)due to the software limitation.

I am still researching.

I'm going to give a shout out for Mozy. I've used their backup service for about 6 months now and I love it. Mozy charges ~5 USD/month for unlimited data backup. They have a little app that runs in your system tray, automatically backing up any new/modified files that appear on your system. This app is available for both Windows and Mac.

Previous to Mozy, my backup strategy was to sync my master media folder with an external server at home as well as with a server at work. Fortunately I never had a hardware failure that forced me to use one of these backups, but I couldn't help but think about what would happen if I forgot to sync up for a few weeks and then had a hardware failure. That would be *bad* news. That's when I decided it was well worth the monthly charge to sign up for Mozy and have a backup solution that I didn't have to think about. For me, the peace of mind that comes along with constant, reliable, hands-off, offsite backups is worth far more than what it costs.

My media folder is currently just under 100 GB. The initial backup took about 1 week to complete, after which it only transfers new/modified files.

...just my $0.02.

NavyMoose · October 17, 2007

Dogdots wrote:

Navymoose what do you mean by saying your photos will be relatively safe?

Let me know if I have this right.

--I hook my external up to the d drive so the OS doesn't effect it. So if anything happens to the OS when I'm hooked up it won't effect my photos cuz they are hook to another drive?

Oops---typical womans question---running in circles. Please forgive
me

----Mary

Hi Mary,

What I meant, if I have to re-install the OS I only have to format the C:\ partition, where the OS lives, and the second partition, where the photos live, would not be touched. You just have to be careful and read everything twice before hitting the "enter" key.

The backup files on an external drive that is powered off and disconnected from my computer unless I'm running the backup.

I suggest using the external drive to keep the backup file and leaves the photos local on the computer. Before I built my photoshop computer, I kept the photos on an external drive and slow doesn't begin to describe the performance.

My gut says the OS can still possibly hurt the photos on the external drive while it is connected to the computer. This is one of the reasons I don't leave it connected unless I'm actually using it.

Are you using WinXP Pro or Home versions? If you are using the Home flavor, you can get the Windows backup utility off the OS CD, I think in the "extras" folder. The Pro version has this installed by default. This is a good tool for the home user to backup important files. You can also save backup jobs to run at a later date.

If you have any other questions, feel free to ask.

Navy Moose

CAFields · October 17, 2007

A suggestion
I've read through this thread with interest and thought I should make a suggestion.

I recently retired from my computer maintenance and repair business and have seen a lot of situations involving crashes and backups, etc. The information that I've read in this thread regarding external backup drives and off-site storage is good.

My suggestion is to choose your backup software very carefully. Most people let the software compress their files during a backup, and if they have to perform a restore procedure, the software uncompresses the files during restore. The problem here is if you create a backup with Brand X backup software and compress the data, will you still have Brand X software 10 years from now if you want to retrieve a photo or file? Chances are whatever program you're using at that time might not read the compressed backup file(s) that you created with Brand X software. If you then pull out the install disk for the Brand X software so that you can load the program to do a restore, will the software run on the OS of 10 years in the future? How many various programs of a couple of years back won't run on Vista now? Think of a backup done with a Windows 95 or 98 software and trying to restore it with one running under Vista, etc. etc. Maybe it will work, but we can't afford to assume that it will. I've seen backups created by Windows95 and 98 built-in backup programs that can't be read by later OS's.

My suggestion is to be sure to perform your backup to the external drive or DVD without compressing the files so that the backup program re-creates the actual directory structure and files on the backup device. You can then view and access those files on the backup device through Windows Explorer or load them directly into your photo editing program without performing a restore procedure, just the same as if they were copied to the disk by copy/paste or dragging/dropping. This type of backup uses more disk space, but for me it's worth it.

Personally I use Nero Backup when I write to DVD's as it will do the above and also allow the backup to be spanned across multiple disks. I use a free utility called SyncBack to copy my photos to my external USB hard drives.
http://www.2brightsparks.com/downloads.html#freeware
There's other software (free and otherwise) that will do the same things as the ones mentioned above, but these are the ones I use. The main thing is to be sure that whatever program you use will backup your data in a way that will allow you to read or open your files directly from the backup disk(s) at any time.

C.A.Fields

http://cafields.com

BradfordBenn · October 17, 2007

Windows Application
Found one that might be helpful for those running Windows. Microsoft has a tool called SyncToy that will do scheduled file copies and one can pick how the move goes. No experience yet but one day at the office and it seems okay so far.

LiquidAir · October 17, 2007

BradfordBenn wrote:

Found one that might be helpful for those running Windows. Microsoft has a tool called SyncToy that will do scheduled file copies and one can pick how the move goes. No experience yet but one day at the office and it seems okay so far.

That's what I use and it has been working fine so far. If you use it to sync a USB drive, you want to fix the drive letter for the external drive. I set my backup volumes as Z:

wildviper · October 18, 2007

CAFields, excellent point. I remember that for my office we used to use HP Simple Backup. I now have CDs that I can't access unless I get that software. What a Pain?!?

I use SyncToy. It is awesome, and it is free from Microsoft. There is no compression bs. It copies the files as they are.

It can even copy files with certain extensions only. So in my case, for my JPEG backup strategy, I just choose *.Jpg and voila! It goes in each folder, recreates the folder and copies the jpegs in there.

On the online storage deal. I will check out mozy. My concern is what if they go out of business? By they, I mean any one of the online storage companies. Don't tell me Amazon won't go out of business...look at Enron. So anything can happen. What then?

By the way, I am playing devil's advocate..I just want to see good reasons with a lot of the bases covered for online storage.

Oh, and by the way, I will store one of the drives off-site and the DVDs off-site in a safety deposit box.

Dogdots · October 18, 2007

Navymoose I have windows xp home version. After reading all this information I think---and it is only thinking---I may see how to do this.

But....what happens when my computer goes and I have to do a destructive restore? I have had to do this 4 times already on my computer. This is where my fear really is with all the photos.

What are xmp files?

I will check out Mozy and and the others mentioned.

----Mary

NavyMoose · October 18, 2007

Dogdots wrote:

Navymoose I have windows xp home version. After reading all this information I think---and it is only thinking---I may see how to do this.

But....what happens when my computer goes and I have to do a destructive restore? I have had to do this 4 times already on my computer. This is where my fear really is with all the photos.

What are xmp files?

I will check out Mozy and and the others mentioned.

----Mary

Hi Mary,

I perfer to use the native Windows backup solution. First because it is free, second it is easy to use. I do not know if it compresses the data or not. The application has been around since the old Windows NT days.

Please read the following article for instructions on how to install the Windows XP backup utility plus there are instructions on how to backup files.

XMP files are created by Adobe Lightroom as a record of changes to the image file. These are kept alongside the actual image file.

If your photographs are kept on a different partition from the OS, they will be safe during a destructive installation. During the re-installation, just tell the system to use the existing C:\ partition.

I have had to rebuild the OS on servers that got hosed and the data files were safe because they were on a different partition. You have to be careful and read everything to make sure you're selecting the correct partition.

I hope this answers your questions.

Navy Moose

Dogdots · October 18, 2007

Thanks Navymoose

When I am saving my photos to disc do I have to save the XMP files to? I somehow got XMP files along with my RAW and JPEG files when I downloaded them into Bridge (following my new CS3 book)---I can't get it open tho. It says it doesn't know the program that it came from.

----Mary

Don't let this happen to you...Backup, Backup & Backup!

Comments