Storage advice - Drobo, FreeNAS, externals?

eoren1

I need some help in deciding how best to expand my storage capacity. Here's the current setup:

iMac for primary photo processing. When complete, 'keepers' go to Smugmug via JPG export. Then get moved from within Lightroom to a 2 TB external. That gets copied via incremental backup to a 1 TB external every night. This is a simple, easy setup. The clone is a backup against data corruption (not perfect, I know). I'm working on better offsite backup. However, the more immediate need is to expand my current storage size.

One of my main concerns is silent data corruption. I therefore have been thinking of getting a Drobo S or building a FreeNAS to run a ZFS pool. Both go 'beyond' basic RAID to allow for expanding the storage in the future and have a host of other benefits. However, there are issues as well.

The Drobo is proprietary RAID and the internet is littered with reviews of data loss. FreeNAS would require more work on my end and I have read about a 'bad' drive being able to corrupt all data in a ZFS pool. While open source, it seems that getting data back out in case of corruption would require lots of linux work.

I don't actually feel the 'need' for the N in NAS. I just want a relatively fast directly attached storage system (at least as fast as current 7200rpm over FW800). Again, key is expandability and piece of mind to not have to worry about silent data corruption or catastrophic drive failure.

An alternative might be getting two new 3 TB drives (different manufacturers). Relegate the current 1 TB backup to the bank vault and let it sit there. The 2TB would be used for non-photo data (I currently have it partitioned to hold music, video files and serve as a Time Machine backup). Use the new 3TB in the same scheme as currently - but only for photos. Advantage of using incremental backups is that the backup drive has the data written to it from the most 'original' source. Carbon Copy Cloner also allows me to set up archived directories on the backup drive that hold data deleted from the primary. That should mitigate the chance of copying over corrupt data. I can then punt on a RAID until I max out the 3TB space.

Appreciate your thoughts.

jasonstone

Check out Synology NAS appliances - for home use how about a ds411j with 4 drive bays. Course you can get one with 2 but I would like to be more future proof size wise.

Their DSM software that makes it oh so much more than just storage is impressive and everyone I know who has one loves it.

Good luck!

eoren1 wrote: »

I need some help in deciding how best to expand my storage capacity. Here's the current setup:

iMac for primary photo processing. When complete, 'keepers' go to Smugmug via JPG export. Then get moved from within Lightroom to a 2 TB external. That gets copied via incremental backup to a 1 TB external every night. This is a simple, easy setup. The clone is a backup against data corruption (not perfect, I know). I'm working on better offsite backup. However, the more immediate need is to expand my current storage size.

One of my main concerns is silent data corruption. I therefore have been thinking of getting a Drobo S or building a FreeNAS to run a ZFS pool. Both go 'beyond' basic RAID to allow for expanding the storage in the future and have a host of other benefits. However, there are issues as well.

The Drobo is proprietary RAID and the internet is littered with reviews of data loss. FreeNAS would require more work on my end and I have read about a 'bad' drive being able to corrupt all data in a ZFS pool. While open source, it seems that getting data back out in case of corruption would require lots of linux work.

I don't actually feel the 'need' for the N in NAS. I just want a relatively fast directly attached storage system (at least as fast as current 7200rpm over FW800). Again, key is expandability and piece of mind to not have to worry about silent data corruption or catastrophic drive failure.

An alternative might be getting two new 3 TB drives (different manufacturers). Relegate the current 1 TB backup to the bank vault and let it sit there. The 2TB would be used for non-photo data (I currently have it partitioned to hold music, video files and serve as a Time Machine backup). Use the new 3TB in the same scheme as currently - but only for photos. Advantage of using incremental backups is that the backup drive has the data written to it from the most 'original' source. Carbon Copy Cloner also allows me to set up archived directories on the backup drive that hold data deleted from the primary. That should mitigate the chance of copying over corrupt data. I can then punt on a RAID until I max out the 3TB space.

Appreciate your thoughts.

eoren1

Thanks Jason
A friend actually has two synology drives and I held one for him as offsite backup. The last firmware update I did for him made me swear off that machine. He has lost data on his own twice.

I decided in the end on the following:
Two 3 TB seagates in owc externals. One is primary photos. Other incremental clone of photos with weekly checksum to confirm data validity.
Crashplan for external backup to 2 tb server at work.

Will hold off on raid until I need more room than a single drive can offer.

Dan7312

Just a thought. Crashplan can be configured to do simultaneous backup to the Crashplan servers and to local drive(s). That's not true RAID, you just use any kind of drives. It makes it easy to have the hallowed three backups, one of which is offsite.

eoren1 wrote: »

Thanks Jason
A friend actually has two synology drives and I held one for him as offsite backup. The last firmware update I did for him made me swear off that machine. He has lost data on his own twice.

I decided in the end on the following:
Two 3 TB seagates in owc externals. One is primary photos. Other incremental clone of photos with weekly checksum to confirm data validity.
Crashplan for external backup to 2 tb server at work.

Will hold off on raid until I need more room than a single drive can offer.

eoren1

Mine is offsite (goes to work server) and, if worse comes to worse, I can drive over and grab the drive for quick restore.

jasonstone

Argh first person I heard who had problems... But it's still electronics - its gonna happen to us at some time

Good set up with 3 drives though

eoren1 wrote: »

Thanks Jason
A friend actually has two synology drives and I held one for him as offsite backup. The last firmware update I did for him made me swear off that machine. He has lost data on his own twice.

I decided in the end on the following:
Two 3 TB seagates in owc externals. One is primary photos. Other incremental clone of photos with weekly checksum to confirm data validity.
Crashplan for external backup to 2 tb server at work.

Will hold off on raid until I need more room than a single drive can offer.

Packhorse-4

eoren1, I have been using a DROBO for over two years now and it is running like a champ. I have had two Drives fail and the DROBO had everything backed up and restored after I put in a new drive. When I started, I used a hodgepodge of old disk drives I had lying around the house, some from old computers and others from random external disk drives. When the old drives failed, I just picked up a new 2TB western digital drive from the store and the DROBO transferred all of my files to the new drive. I really don't know how RAID works, but this was simple and I didn't need to worry about transferring all of my data off the RAID or finding exactly the same drive I used in the RAID device when it was set up two years earlier.

Dreadnote

For what it's worth here is Scott Kelby's take on Drobo:

http://scottkelby.com/2012/im-done-with-drobo/

Packhorse-4

I'm a big fan of Scott Kelby and he did have a bad experience with his DROBO. I must agree that when you purchase a RAID device, you expect the device (or enclosure) to work flawlessly - for a very long time. However, every device will fail at some point and we must be prepared for that as well. This becomes an issue for the DROBO because they use a proprietary file system to get the DROBO to work its RAID magic across different drives. This means you can't pull a drive from your DROBO and put it in another device to read the files. For some, this is a "deal breaker".

No matter what device you select for your RAID solution, be sure to look at the warranty policy for the device you are purchasing. This may be one of the times you want to consider an extended warranty as well.

BradfordBenn

I used to think I knew this stuff and then technology moved ahead. Here is my take on it. I don't like DROBO from before Mr. Kelby's rant due to the proprietary nature of its write/raid scheme. I have also seen high end RAID servers hamstrung by using drives from the same batch so a latent defect caused them all to fail. I have also seen the bad things happen where a local backup is the only backup on the RAID machine when a house gets wiped out.

So there are always things to consider and I do think the key of having duplicate data on different media in different locations is key, which is sounds like what you are wanting to do.

I ended up going with JungleDisk for my backup and archival solution. I use their backup software to make my nightly incremental backups of key data (typically not photos but it does include the LightRoom Catalog). I also have a drive mounted that I can copy things to off site storage for when it is ready to be archived.

The thing that frustrated me was that some of the raid systems just slowed down the editing process as the write speed was not as fast as a regular drive and then a backup when done editing.

The key though is to test any of your solutions ahead of time. Try access your data without using your primary computer and see how you do.

CatOne

The real issue with the Drobo is that it is VERY, VERY slow. It's roughly 1/3 the speed of a single attached Firewire 800 drive. We're talking 20 MB/sec on a good day.

So, if you are using it for archival or document storage, it's OK. If you're trying to store photos on it and access them with something like Lightroom, you will be feeling pain. IT. IS. VERY. SLOW.

There are NAS devices which are faster than the Drobo when it's direct attached via Firewire 800.

The fact that you can pull drives from the Drobo or mix and match different sizes and upgrade overall capacity on the fly is nice. But I wonder how common that is. For that to work, you also have to over-provision it (i.e. set it up to say it has 16 GB available when you have 4 GB worth of drives in there... so be careful!). I have one and leave it in my closet and back up to it every month or two.