Amazon Glacier

cmason

In case you didn't see the announcement yesterday, Amazon announced a new offering, called Amazon Glacier. It is essentially like their S3 service, but much, much cheaper, and targeted to archiving.

Basically, there is no cost to upload to the service, its $0.01 per GB per month for storage, and $0.12 per GB per month for download. It also is slow, meaning your upload/download traffic is low priority, with the service good for uploading/downloading files, but useless for apps and other services that need near-real time access. For archive and backups, speed is not an issue. If you use S3 today, like with Jungle disk, this is going to reduce your costs dramatically.

I was looking into this as a replacement for Crashplan, or perhaps in addition to Crashplan, where I have 600GB stored today. But looking closely, 600GB would cost:

Yearly Costs:

Upload 600GB : $0.00

Storage: $0.01 per GB per month: $6: $72.00

Download: $0.12 per GB per month: $70.00*

*should I ever need it: Worst case, it takes 2 months to download (300GB each mon): $35+$35 = $70. (First GB download per month is free.)

So, thats $72 per month, with an additional penalty of $70 if you ever need to recover the entire or large portion of your archive.

Currently, I pay Crashplan $50 per year for online backup of these photos. Plus this is 'unlimited', in that I do not pay more if I backup more.

So, I think this might be a great option, especially if you have a number of offline drives, or other data (disk, tape, etc) that you would like stored safely online, but it still doesn't compete with the backup options available.

Andy

While this sounds cool, there are other really good options. I have about 3 terabytes on Backblaze across three computers for a whopping $130 a year. You can set up an individual account for as little as $50 per year. And it's unlimited and I can restore a file anytime from anywhere I have internet access.

divamum

Backblaze was the other option I checked out, but for me Crashplan worked out as the easiest most affordable for my needs. I've been super happy with the service and it saved my backside earlier this year when I had an emergency OS reinstall!

Foques

does either of these services offer a sharing option akin to FTP?

cmason

Foques wrote: »

does either of these services offer a sharing option akin to FTP?

Amazon is web services, REST API, not FTP.

None of these are Dropbox, Amazon Glacier is an archiving solution, Crashplan, Backblaze are online backup solutions

BradfordBenn

So what apps are available for the "basic user" to put things into storage?

Andy

BradfordBenn wrote: »

So what apps are available for the "basic user" to put things into storage?

What do you mean, can you elaborate?

BradfordBenn

Hi Andy-

By a basic user, I mean someone like me. I don't write code, I use it. So I would like to have a program/interface that would allow for me to easily put things into storage (and of course retrieve). So I could select the items I want to store in the application. Set a few parameters, such as bandwidth throttling and off the file is uploaded. I would then get a confirmation it is in the Vault and could delete/offload etc the original. That same application would also provide a catalog solution of what is on the Glacier so I could look for files quickly and easily. I would then be able to queue up their retrieval (so that I don't pay way too much retrieving too much at peak hours) and let it retrieve the files for me.

Does that make sense?

My understanding is that currently it is all API based, while I understand the concepts and the basics there are some times I just want to purchase a basic solution. Such as why I went with JungleDisk as the interface around my Amazon Cloud Storage. It had an interface that is easy to use, price is reasonable, and has the features I want. Much easier and less frustrating than writing my own interface.

cmason

BradfordBenn wrote: »

Hi Andy-

By a basic user, I mean someone like me. I don't write code, I use it. So I would like to have a program/interface that would allow for me to easily put things into storage (and of course retrieve). So I could select the items I want to store in the application. Set a few parameters, such as bandwidth throttling and off the file is uploaded. I would then get a confirmation it is in the Vault and could delete/offload etc the original. That same application would also provide a catalog solution of what is on the Glacier so I could look for files quickly and easily. I would then be able to queue up their retrieval (so that I don't pay way too much retrieving too much at peak hours) and let it retrieve the files for me.

Does that make sense?

My understanding is that currently it is all API based, while I understand the concepts and the basics there are some times I just want to purchase a basic solution. Such as why I went with JungleDisk as the interface around my Amazon Cloud Storage. It had an interface that is easy to use, price is reasonable, and has the features I want. Much easier and less frustrating than writing my own interface.

Right, agree. Amazon supplies the backend, its up to others to supply any front end. To be honest, the Amazon management interface isnt bad, its easy to use, but what you are talking about is something like Jungle Disk, which does what you want, but I don't know if they are planning to support Glacier, their solution is S3 at the moment.

Frankly, if you just want very simple online storage, you might want to consider Dropbox or Box.net. In the case of Dropbox, you just copy stuff to a folder. Box is not quite as simple, but there is a web uploader and versioning. For backup, as I said, Crashplan+ or Backblaze make much more sense both in ease of use and cost.

Dan7312

http://cloudberry.com has said they will be adding support for Glacier.

Right now the only Glacier capability in the Amazon management console is for creating a Glacier vault which itself holds archives, it doesn't include any capabilities for upload or download anything.

Someone has published a simple uploader for Glacier. http://simpleglacieruploader.brianmcmichael.com/

Glacier doesn't really store files as you would think of them on a disk. It stores archives. An archive is just a sequence of bytes with a 138 byte identifier, it could be bytes from a single file though. In other words a local application will need to keep track of what identifiers are in what archives. The only way to retrieve an archive is by using its 138 byte identifier. An application would then have to figure out how to turn an archive into one or more files.

However it does provide both geographic and facility redundant storage of archives, much like S3 does, but at less than 1/10 the cost. So it's a very reliable archive.

Places like backblaze, spideroak, carbonite(I think?) and crashplan store all their files in just a single facility. spideroak and crashplan have optional geographically redundant storage for extra cost, but not for all plans. Carbonite Business uses two facilities. Glacier, like S3, uses three facilities.

Geographically redundant storage means that if one facility is destroyed, maybe by a fire or a big lightning strike or something like that, your backup will still be available from another facility.

Most people backing up images keep a local copy anyhow, so geographic redundancy doesn't buy them much.

Andy's 4TB will probably always cost less to backup on backblaze than Glacier, but for smaller amounts with multiple computers Glacier may end up being less expensive.

So Glacier is low cost but highly reliable archival storage. You may or may not need it, depends on what
you are looking for.

cmason wrote: »

Right, agree. Amazon supplies the backend, its up to others to supply any front end. To be honest, the Amazon management interface isnt bad, its easy to use, but what you are talking about is something like Jungle Disk, which does what you want, but I don't know if they are planning to support Glacier, their solution is S3 at the moment.

BradfordBenn

Agreed on all those points. I am using JungleDisk as the front end to my S3 account so I have the multiplicity of locations. I appreciate the info on the archives, it is what I expected but a confirmation is always good. I use SmugMug for my backup of images - the few that I want to have the originals far are in my JungleDisk storage.

I agree if I get to the volume of 4TB Backblaze might be a better $olution but at the moment JungleDisk was the right solution at the right price point.

tsk1979

Smugmug already offers archival(via S3). I guess smugmug could offer archival via this service also? What say Andy?
Many photographers have need to backup around 50-60GB max of data.

Dan7312

Glacier utility
http://fastglacier.com/

BradfordBenn

Thanks but I am a Mac guy at home so I will keep looking.

Baldy

BradfordBenn wrote: »

Thanks but I am a Mac guy at home so I will keep looking.

Has anyone figured out a good Mac client for Glacier? I have 16TB (I shoot video) and my backup array is full and dying (a disk is going bad).

Right now I'm investigating using Arq, which is very popular among Mac geeks, but my files are on a drawer full of external drives and Arq doesn't really work that way. I just want to manually upload a directory every time I make a new movie.

Dan7312

It would be a bit tedious but you can use the Amazon web console until something is available for the Mac.

https://console.aws.amazon.com/console/home

Don't know how good it is but there is

http://www.ilovefreesoftware.com/12/windows/amazon-glacier-client-windows-mac-linux.html

http://Cloudberrylab.com has a backup and general explorer which are have really useful interface to glacier, but for now they don't have a mac application.

Baldy wrote: »

Has anyone figured out a good Mac client for Glacier? I have 16TB (I shoot video) and my backup array is full and dying (a disk is going bad).

Right now I'm investigating using Arq, which is very popular among Mac geeks, but my files are on a drawer full of external drives and Arq doesn't really work that way. I just want to manually upload a directory every time I make a new movie.

Baldy

Dan7312 wrote: »

It would be a bit tedious but you can use the Amazon web console until something is available for the Mac.

https://console.aws.amazon.com/console/home

Don't know how good it is but there is

http://www.ilovefreesoftware.com/12/windows/amazon-glacier-client-windows-mac-linux.html

http://Cloudberrylab.com has a backup and general explorer which are have really useful interface to glacier, but for now they don't have a mac application.

After a lot of research, I settled on Arq, beloved by Mac geeks, and was able to back up 8 TB so far in a few hours using our SmugMug connection.

I also bought a big Thunderbolt 8-bay enclosure so I can have a really fast 24 TB array with two redundant drives to edit directly on and then hopefully I never have to touch anything on Glacier.

BradfordBenn

Baldy wrote: »

After a lot of research, I settled on Arq, beloved by Mac geeks, and was able to back up 8 TB so far in a few hours using our SmugMug connection.

I also bought a big Thunderbolt 8-bay enclosure so I can have a really fast 24 TB array with two redundant drives to edit directly on and then hopefully I never have to touch anything on Glacier.

I still haven't found anything I really like.

Baldy, just remember it isn't really backed up until it is off site and in multiple media formats.