The 2+ terabyte Question

bl8ter

I am currently considering joining smugmug, but have a serious problem. Over the last 2 years I have amassed 2484.98 GIGS (yes that is 2+ terabytes) which is stored over way to many dvds. Now if my calculations are correct it would take me 103360 hours, 12 minutes and 4 seconds under optimal connections to load my albums. I have a cable connection with a 60Kb upload rate.

Reading the smugmug faq is states that DVD's can be purchased for $22 bucks plus 10 dollars per gig (or something like that), but never talks about having your DVDs restored. So my first questions is does this type of service exist?

Secondly I have full access to a full DS3 connection (90+ megs/sec) which I could use to upload, But i'm sure smugmug has bandwidth throttling. So my second is question is what are the connections throttled at?

While I realize this is an extreme situation I'm sure as more people embrace digital photography the problem is only going to get worse. To make matters even worse I problably create on average 1 - 2 gigs a week.

Can Smugmug deliver a solution to address my need?

Mike Lane

bl8ter wrote:

I am currently considering joining smugmug, but have a serious problem. Over the last 2 years I have amassed 2484.98 GIGS (yes that is 2+ terabytes) which is stored over way to many dvds. Now if my calculations are correct it would take me 103360 hours, 12 minutes and 4 seconds under optimal connections to load my albums. I have a cable connection with a 60Kb upload rate.

Wowza! You're a prolific shooter...

bl8ter wrote:

Reading the smugmug faq is states that DVD's can be purchased for $22 bucks plus 10 dollars per gig (or something like that), but never talks about having your DVDs restored. So my first questions is does this type of service exist?

DVD's restored? You mean having smugmug send you your 2.48498 Tb of pics on DVDs just as you have them right now if something happens to them? I doubt it, but I don't know for certain.

bl8ter wrote:

Secondly I have full access to a full DS3 connection (90+ megs/sec) which I could use to upload, But i'm sure smugmug has bandwidth throttling. So my second is question is what are the connections throttled at?

From some of the posting from Don and Chris, I'm pretty sure that smugmug doesn't have bandwidth throttling. I could very well be wrong but they have talked about users that take advantage of massive bandwidth at the office.

bl8ter wrote:

While I realize this is an extreme situation I'm sure as more people embrace digital photography the problem is only going to get worse. To make matters even worse I problably create on average 1 - 2 gigs a week.

Probably won't be an issue as stated above

bl8ter wrote:

Can Smugmug deliver a solution to address my need?

Sounds like they can to me

jfriend

I didn't see any throttling

bl8ter wrote:

I am currently considering joining smugmug, but have a serious problem. Over the last 2 years I have amassed 2484.98 GIGS (yes that is 2+ terabytes) which is stored over way to many dvds. Now if my calculations are correct it would take me 103360 hours, 12 minutes and 4 seconds under optimal connections to load my albums. I have a cable connection with a 60Kb upload rate.

Reading the smugmug faq is states that DVD's can be purchased for $22 bucks plus 10 dollars per gig (or something like that), but never talks about having your DVDs restored. So my first questions is does this type of service exist?

Secondly I have full access to a full DS3 connection (90+ megs/sec) which I could use to upload, But i'm sure smugmug has bandwidth throttling. So my second is question is what are the connections throttled at?

While I realize this is an extreme situation I'm sure as more people embrace digital photography the problem is only going to get worse. To make matters even worse I problably create on average 1 - 2 gigs a week.

Can Smugmug deliver a solution to address my need?

I have, in the past, uploaded 1000 images at a time from a T1 connection and the uploads go extremely fast so I don't think smugmug was throttling anything on their end at the time. Why don't you just sign-up with the free trial account and try it out. I used StarExplorer to do my uploads. It let's me set up a whole hieararchy of galleries and uploads ahead of time and then just do all the uploads in one unattended session.

bl8ter

Mike Lane wrote:

DVD's restored? You mean having smugmug send you your 2.48498 Tb of pics on DVDs just as you have them right now if something happens to them? I doubt it, but I don't know for certain.

... No I will send them MY stacks of DVDs and they can restore them to my account.... This would be a great service....

Mike Lane

bl8ter wrote:

... No I will send them MY stacks of DVDs and they can restore them to my account.... This would be a great service....

You can send them electronically

bl8ter

jfriend wrote:

I have, in the past, uploaded 1000 images at a time from a T1 connection and the uploads go extremely fast so I don't think smugmug was throttling anything on their end at the time. Why don't you just sign-up with the free trial account and try it out. I used StarExplorer to do my uploads. It let's me set up a whole hieararchy of galleries and uploads ahead of time and then just do all the uploads in one unattended session.

yeah I will sign up for the trial and see if I can move 2 tera bytes in 7 days... I will report back the results. StarExplorer looks nice, but I only use Linux... I will have to find something else...

Thanks for your help.

Mac Write

This will be very entertaining. Can't wait to see the servers croak under the load and Smugmug having to buy an XRaid fully loaded just for your site .

marlinspike

Have you ever thought about saving them as quality 10 jpgs? Over that many files that would save a lot of space.

Andy

bl8ter wrote:

Can Smugmug deliver a solution to address my need?

Hi bl8ter, welcome to Dgrin and SmugMug.

These help pages might be of use to you.

http://www.smugmug.com/help/faqgen
http://www.smugmug.com/help/photo-storage

Backups are for us to create copies of what you have stored:
http://www.smugmug.com/help/backups

We don't have a service where you can send us your dvds, sorry about that.

All the best,

bl8ter

Andy wrote:

Hi bl8ter, welcome to Dgrin and SmugMug.

These help pages might be of use to you.

http://www.smugmug.com/help/faqgen
http://www.smugmug.com/help/photo-storage

Backups are for us to create copies of what you have stored:
http://www.smugmug.com/help/backups

We don't have a service where you can send us your dvds, sorry about that.

All the best,

I read the http://www.smugmug.com/help/faqgen and it specifies 24gig yearly limit( or 2gigs per month) of bandwidth. Is this only for viewing? or does this include the uploading of pictures also.

Thanks.

Andy

bl8ter wrote:

I read the http://www.smugmug.com/help/faqgen and it specifies 24gig yearly limit( or 2gigs per month) of bandwidth. Is this only for viewing? or does this include the uploading of pictures also.

Thanks.

Bandwidth is serving up your images to your viewers. 4gb for standard accounts, 8gb for Power users, 16gb for Pros. Does not include uploading or any html served up by SmugMug to your viewers.

DavidTO

bl8ter wrote:

Secondly I have full access to a full DS3 connection (90+ megs/sec) which I could use to upload, But i'm sure smugmug has bandwidth throttling. So my second is question is what are the connections throttled at?

Sounds like if they don't already they better do it soon!

bl8ter

Andy wrote:

Bandwidth is serving up your images to your viewers. 4gb for standard accounts, 8gb for Power users, 16gb for Pros. Does not include uploading or any html served up by SmugMug to your viewers.

Great.... I am signing up today... Thanks for all your help!

onethumb

bl8ter wrote:

I am currently considering joining smugmug, but have a serious problem. Over the last 2 years I have amassed 2484.98 GIGS (yes that is 2+ terabytes) which is stored over way to many dvds. Now if my calculations are correct it would take me 103360 hours, 12 minutes and 4 seconds under optimal connections to load my albums. I have a cable connection with a 60Kb upload rate.

Are these JPEGs? Or BMP/PSD/TIFF/RAW? Smugmug doesn't accept anything but JPEGs, GIFs, or PNGs. If they *are* JPEGs, you're the most prolific shooter we've ever met (and I thought we'd met them all!).

If you don't mind my asking, what resolution are they and what compression settings did you use?

bl8ter wrote:

Reading the smugmug faq is states that DVD's can be purchased for $22 bucks plus 10 dollars per gig (or something like that), but never talks about having your DVDs restored. So my first questions is does this type of service exist?

We don't do this, but I've heard of a few other people offering this service. In fact, I think Nikolai does. He's on this forum, you might ask him.

bl8ter wrote:

Secondly I have full access to a full DS3 connection (90+ megs/sec) which I could use to upload, But i'm sure smugmug has bandwidth throttling. So my second is question is what are the connections throttled at?

A DS3 is actually 45 MegaBits per second, or 5.6 megs per second. We can handle that without impacting other customers, I think. There is no throttling, though you may be the straw that changes our policy. Your best bet to get full bandwidth is to upload early morning (midnight - 5am PST).

bl8ter wrote:

While I realize this is an extreme situation I'm sure as more people embrace digital photography the problem is only going to get worse. To make matters even worse I problably create on average 1 - 2 gigs a week.

Can Smugmug deliver a solution to address my need?

Once we understand your needs a little better (# of photos, format, compression level, resolution), I'm hoping we can deliver a solution for you. But I really don't know.

Ideally, we'd at least need a few weeks warning before you shoved all the photos at us so we can make sure our available storage matches.

Anyway, let's keep talking and see what we can do. I'm intriguiged by your prolific shooting capacity.

Thanks!

Don

bl8ter

onethumb wrote:

Are these JPEGs? Or BMP/PSD/TIFF/RAW? Smugmug doesn't accept anything but JPEGs, GIFs, or PNGs. If they *are* JPEGs, you're the most prolific shooter we've ever met (and I thought we'd met them all!).

If you don't mind my asking, what resolution are they and what compression settings did you use?

They are all JPEGS and are about 6.2 megs each(8MP, 3423 x 2400). I keep them this size because I like to make poster size prints. Like 36x24, etc...

onethumb wrote:

A DS3 is actually 45 MegaBits per second, or 5.6 megs per second. We can handle that without impacting other customers, I think. There is no throttling, though you may be the straw that changes our policy. Your best bet to get full bandwidth is to upload early morning (midnight - 5am PST).

Its dual DS3's... Load balanced. And since its a holiday week its wide open.

onethumb wrote:

Once we understand your needs a little better (# of photos, format, compression level, resolution), I'm hoping we can deliver a solution for you. But I really don't know.

Ideally, we'd at least need a few weeks warning before you shoved all the photos at us so we can make sure our available storage matches.

Anyway, let's keep talking and see what we can do. I'm intriguiged by your prolific shooting capacity.

Thanks!

Don

I travel internationally a lot for my work so I end up taking lots of pictures. On average I am only home only 4 days a month. I normally take a 300gig usb harddrive with me.. I also keep the .RAW images but those i will not upload. I keep those for when I do any editing. The 2+ terabytes are just the JPEGS.

onethumb

bl8ter wrote:

They are all JPEGS and are about 6.2 megs each(8MP, 3423 x 2400). I keep them this size because I like to make poster size prints. Like 36x24, etc...



Its dual DS3's... Load balanced. And since its a holiday week its wide open.


I travel internationally a lot for my work so I end up taking lots of pictures. On average I am only home only 4 days a month. I normally take a 300gig usb harddrive with me.. I also keep the .RAW images but those i will not upload. I keep those for when I do any editing. The 2+ terabytes are just the JPEGS.

Wow, that must be close to 500,000 JPEGs! Quite a collection - I can't wait to see them.

So we've always said "unlimited storage" and we mean what we say. We're happy to take your photos and host them, but we need a little time to prepare.

Just so you know, this is something like a $20,000 first-year commitment for us in terms of disk space, power, cooling, and physical space. You'll basically have two complete RAID arrays to yourself in our datacenter. Of course, you'll only pay your $30, $50, or $100 per year, depending on your account level. Again, we're happy to do it - but I want to be up front here and let you know that we need to order some equipment and get it installed to accept your photos. We're not geared for accepting 2.5TB overnight.

We'll also be buying extra image processing machines just for your batch of photos. Luckily, once yours are done, everyone else at smugmug will get to benefit from them, so I don't consider that a cost to host you.

Does that sound fair? Can we ask you to hold off while we order and install the equipment and power required?

Thanks!

Don

Mac Write

Man this is so exciting!!! You have to use him for your ads and other things to promote smugmug. BTW Are you the buy who takes tons and tons of photos of your dog from DPReview? I think he is going to need a custom bandwidth plan and will help shape smugmug for the next 5 years with all the suggestions he will have.

Don you have to post photos of the RAID's arriving, being taken out of the box, being setup and installed (without compermisiing security). I see this as a big way to help promote smugmug. If this works out, I think he will personally bring in 1,000+ customers. Think of the ads "Smugmug is truly unlimited. I host over 500,000 photos at over 2.5TeraBytes."

Man I can't wait to see how this plays out Drool.

Mike Lane

onethumb wrote:

Does that sound fair? Can we ask you to hold off while we order and install the equipment and power required?

Thanks!

Don

Holy crap you guys rule...

bwg

Mike Lane wrote:

Holy crap you guys rule...

seriously.

i want to go get some popcorn and see how this thread plays out!

marlinspike

While I'm going to hate while he's uploading I'm going to love when he's done.

bl8ter

onethumb wrote:

Wow, that must be close to 500,000 JPEGs! Quite a collection - I can't wait to see them.

So we've always said "unlimited storage" and we mean what we say. We're happy to take your photos and host them, but we need a little time to prepare.

Just so you know, this is something like a $20,000 first-year commitment for us in terms of disk space, power, cooling, and physical space. You'll basically have two complete RAID arrays to yourself in our datacenter. Of course, you'll only pay your $30, $50, or $100 per year, depending on your account level. Again, we're happy to do it - but I want to be up front here and let you know that we need to order some equipment and get it installed to accept your photos. We're not geared for accepting 2.5TB overnight.

We'll also be buying extra image processing machines just for your batch of photos. Luckily, once yours are done, everyone else at smugmug will get to benefit from them, so I don't consider that a cost to host you.

Does that sound fair? Can we ask you to hold off while we order and install the equipment and power required?

Thanks!

Don

This sounds very fair... Just so were on the same page I will not be able to upload 2.4 terabytes in one session. I only have 200gigs on my computer and a 300gb usb drive. Some 90 percent of my images are on DVDs will need to be restored to my machine for uploading. Its also going to take me some time to put these in albums and properly tag them. If I remember correctly the java uploader will only send 100megs at a time, and im not going to sit in front of my computer for 7 days straight uploading them all at once. I travel quite a bit and this will done over the next few months.

My goal of asking this question was to find out if I could host my entire collection in an online service that would free me from storing all these images. Secondly since I travel a lot it would be nice to be able the upload the pictures almost real time and not have to mess around with transfering them and backing them up. On top of that my family and friends all want to see them so I end up shrinking them for emailing them or burning dvds for them. This takes way to much time.... Plus I like the idea of sharing photos with people who are also into photography. The concept of having a google of images from around the globe is very inspiring.

Given that I had some time off for the holidays I decided to investigate these services to find out if they would address my needs. The feedback from this thread tells me that it is entirely possible and will make my photography alot less time intensive and much more enjoyable.

I would like to thank everyone for their feedback!

bl8ter

Mac Write wrote:

BTW Are you the buy who takes tons and tons of photos of your dog from DPReview?

900gigs are pictures of just ants using a macro lense..

Actually only about 300 gigs is family related photos. Most of my images are of architecture, landscapes, cities, machines and people from all over the world.

jfriend

You may want to check out star explorer

bl8ter wrote:

If I remember correctly the java uploader will only send 100megs at a time, and im not going to sit in front of my computer for 7 days straight uploading them all at once. I travel quite a bit and this will done over the next few months.

If you are going to be uploading that much, I would highly recommend one of the 3rd party uploaders. I use StarExplorer and I switched to it because I could organize images into albums on my PC and then upload GB's at a time across many albums, all unattended. No 100MB limit per session.

Mac Write

Wish StarExplorer was out for Mac. But there isn't. If I use iPhoto I can upload multiple albums as once, but since I don't use it (I use Aperture now) I don't have that luxury. Hopefully Smugmug uploader 2 for OS X will offer Aperture Albums uploading.

onethumb

jfriend wrote:

If you are going to be uploading that much, I would highly recommend one of the 3rd party uploaders. I use StarExplorer and I switched to it because I could organize images into albums on my PC and then upload GB's at a time across many albums, all unattended. No 100MB limit per session.

I would definitely either use the built-in BETA uploader or something like Send-To-Smugmug or Star*Explorer. They're more efficient at uploading, don't have any batch size limits, and also allow my image processing cluster to attack each photo as it arrives, rather than once the entire batch is complete.

Don

onethumb

bl8ter wrote:

This sounds very fair... Just so were on the same page I will not be able to upload 2.4 terabytes in one session. I only have 200gigs on my computer and a 300gb usb drive. Some 90 percent of my images are on DVDs will need to be restored to my machine for uploading. Its also going to take me some time to put these in albums and properly tag them. If I remember correctly the java uploader will only send 100megs at a time, and im not going to sit in front of my computer for 7 days straight uploading them all at once. I travel quite a bit and this will done over the next few months.

My goal of asking this question was to find out if I could host my entire collection in an online service that would free me from storing all these images. Secondly since I travel a lot it would be nice to be able the upload the pictures almost real time and not have to mess around with transfering them and backing them up. On top of that my family and friends all want to see them so I end up shrinking them for emailing them or burning dvds for them. This takes way to much time.... Plus I like the idea of sharing photos with people who are also into photography. The concept of having a google of images from around the globe is very inspiring.

Given that I had some time off for the holidays I decided to investigate these services to find out if they would address my needs. The feedback from this thread tells me that it is entirely possible and will make my photography alot less time intensive and much more enjoyable.

I would like to thank everyone for their feedback!

Great! We've already started the wheels turning to get some more hardware, so I'll keep everyone posted as to how that's going. (Actually, Andrew probably will .

Don

Wireless

onethumb wrote:

Great! We've already started the wheels turning to get some more hardware, so I'll keep everyone posted as to how that's going. (Actually, Andrew probably will .

Don

So much for slacking off at the new job...

Andy

Wireless wrote:

So much for slacking off at the new job...

Mike Lane

onethumb wrote:

Great! We've already started the wheels turning to get some more hardware, so I'll keep everyone posted as to how that's going. (Actually, Andrew probably will .

Don

Merry Christmas to Apple huh?

Mac Write

Oh ya :D:D:D I hope those are hooked up to XServes. Man we really need to see the plans for the data center - while not breeching security.

You take his word he is real and spend $20,000 WOW. Me personally I want to have 10,000+ photos on-line come end of Q1 06. and double that by end of 2006.

flyingdutchie

About 500.000 JPEGs.... WOW!!!
mmmmmm i did some math:

Say you shot them all in the last 10 years.
That is 50.000 per year.
That is 136 (and some change) per day, every day!!!!
If a day is 8 hours (work day), that would be around 17 per hour and that is about one picture per 3 and a half minute!

Those numbers are staggering!!!