Smugmug down time: Concerns?

caseserve · April 5, 2006

Does anybody else have concerns about the amount of time that smugmug is down?

It may not matter much to the average user, but for a pro user like myself my smugmug site (caseserve.smugmug.com) is my presence on the internet. I have customers going there to proof photos for commerical shoots that I'm doing and downtime makes me look like crap!

No doubt this thread will get moved to "mind your own business" area. But I would still like to know your thoughts on this.

Thank you,

Andy · April 5, 2006

caseserve wrote:

Does anybody else have concerns about the amount of time that smugmug is down?

It may not matter much to the average user, but for a pro user like myself my smugmug site (caseserve.smugmug.com) is my presence on the internet. I have customers going there to proof photos for commerical shoots that I'm doing and downtime makes me look like crap!

No doubt this thread will get moved to "mind your own business" area. But I would still like to know your thoughts on this.

Thank you,

Hi John,

I can see this is upsetting to you. Why would I move it to MYOB? It's a SmugMug Support issue. I'm not aware of any downtime today, but I'll ask Wireless or Onethumb. Are you certain there weren't internet difficulties in your area? We'll investigate for sure. But I have to say, after you posted your message twice a few minutes ago, your site was working fine.

We take our uptime VERY seriously. Onethumb and Wireless stay up nights while you're sleeping to ensure a smooth running site. Does downtime happen? Rarely. And we've even listened to our Pros and moved scheduled maintenance to later in the week (it was on Monday night/Tuesda morning CA time)

When have we EVER shown that we're not responsive to your, or our customers in general, needs? IIRC both Onethumb and I have taken care to help you with externally served JS issues lately. Did you get that sorted - if you need help on it, just let us know!

I hope this helps,

Andy · April 5, 2006

caseserve wrote:

It may not matter much to the average user, but for a pro user like myself

I can tell you for a fact that we treat all of our customers the same, and all of our customers, Standard, Power, and Pro, feel that having their photos available is critically important to them. There is no "average" customer to us - everyone is a customer and we treat everyone the same - with what we hope is outrageously excellent customer service, innovative product and service, and of course, reliability.

marlinspike · April 5, 2006

When you look at the other photo sharing sites, smugmug is one of the most stable. It's just the way the internet is.

Mongrel · April 5, 2006

caseserve wrote:

Does anybody else have concerns about the amount of time that smugmug is down?

No

Mongrel, Pro User since October 2004

dogwood · April 5, 2006

for what it's worth
I also use pbase and they certainly seem to be down more than smugmug.

I complained once on another forum about smugmug being down and some guy responded with something to the effect of, "Deal with it... do you really expect $100 a year to get you a bomb-proof service?" (this was obviously a while ago). I was fuming for weeks after that and just livid at that response. Of course I expected bomb-proof service! A hundred bucks is a lot of money to me!

But when I started researching the cost of web design and maintenance and a dedicated server and bandwidth and setting up an order prints option, suddenly smugmug seemed like a decent deal. I work a regular job at a place with dedicated servers and a full IT staff to keep everything running-- and we still have downtime that grinds everything to a halt.

John, this isn't meant to discount your frustration-- we've all been there. It is especially frustrating when you post a bunch of photos and you get the word out and then nobody can see them and they think it's a reflection on you. That has happened to me too with smugmug.

But when I add up all the support and help I've gotten here (a lot of times from non-smugmug people)-- I'm pretty amazed. For me, anyway, the downtime still doesn't outweigh the positive. Check around with some other photographers-- most of the ones I've spoken to have issues with their websites too, and they don't have nearly the same updating or support capabilities as smugmug. You'll just need to decide for yourself whether the cost is worth the service you recieve.

Sorry-- I know this sounds like a lecture. Just want to say, I feel your frustration and can relate. It's maddening when any service you pay for goes down-- especially when you don't have control over it.

caseserve · April 5, 2006

I'm not trying to start trouble or whine and carry on. It's just that smugmug is down more then the my other site is. I pay the same $100 per year with them. (myinternetservices.com) They have 99.9% uptime and I can attest that they are up.

Except for 1 post I did the other day which was unrelated to smugmug being down but my site was, as I said they are almost never down in the 7 or more years that I've been using their service.

Their server got hacked. The hacker got into the php temp directory and they were facilitating a DOS attack. They brought the server down, removed the offending PHP code and brought it back up.

When I post I get why no John, we are all fine it must be you, I've got that more then once. I have even checked to make sure Andy's site moonriverphotography.com was also down before I posted as well as half a dozen other sites, then I post. Except for the other day, sorry.

Well, I've been in IT for 20 years. I have certifications as a Novell CNE, Microsoft MCSE, Cisco certification etc etc. I know how to verify my connection to the internet. I also make sure I can access other sites before I post that my site is down.

Smugmug is a great service, the best I've come across. But that is only as long as it's up. I don't want to rub anybody the wrong way or piss off anybody here, because if I do that then when I need help customizing my site I may not get it. (Thank you Andy, Mike and everyone else for the help in the past with my site) But my reference point is my site hosted with myinternetservices.com The same $100 per year and they are always up.

If you scan over the threads that are currently in the support forum, there are posts asking if smugmug has a problem or if they are down.

Well even if they are down briefly and I as well as others try to hit our sites and they don't respond, they were still down. Then someone responds to a post that I can hit my site so it must be you. What are we to do? We look like idoits because smugmug had a hiccup and returns to normal. There is a thread about a small hiccup now in the support forum.

Please let me know what I'm to do? Do I continue to post when I cannot access my site? I'll do whatever you say I should do.

Smugmug is a great service that I enjoy using

Thank you,

PS Pete - I really like your work!

Andy · April 5, 2006

caseserve wrote:

Please let me know what I'm to do? Do I continue to post when I cannot access my site? I'll do whatever you say I should do.

Smugmug is a great service that I enjoy using

Thank you,

No worries at all. And we apologize for the very brief interruption today.
Please post as often as you like - we want you to be comfortable with the service. Today's brief downtime (minutes) was in fact on us - and I'll see if I can get Wireless to explain later, fwiw.

If you can't hit your site we absolutely want to know. Many times, it's due to other factors not us. Regardless, we want to be here to troubleshoot, assist, and get you back in business as swiftly and easily as possible.

All the best,

dogwood · April 6, 2006

thanks john
PS Pete - I really like your work![/quote]

Appreciate the comment. It does suck when smugmug goes down right as you've sent folks to check out photos on your site, eh? Glad you were able to see mine today!

largelylivin · April 6, 2006

No. That isn't true.

marlinspike wrote:

When you look at the other photo sharing sites, smugmug is one of the most stable. It's just the way the internet is.

There are HUGE internet services with problems that SmugMug cannot imagine that manage to be up 99.998% of the time (that's 3-sigma). There are system design techniques that handle component failures and also techniques to handle spiking traffic volume through load-leveling. There are even techniques to be prepared for and work around problems on the Internet Backbone, which aren't even YOUR problems, rather they are problems with service providers. BUT, if you are concerned about availability and reliability and customer satisfaction, then you do worry about it and proactively address it through a variety of system design, operational planning, procedures and readiness, redundancy, altermate routing and load-leveling, business arrangements, etc. etc., etc.

As much as I hate to use Gates in any positive way, imagine if the MS service sites were this shakey. They aren't. How about Adobe? I go there all the time and I don't experience these problems. What about EBay? They have the biggest operational challenges EVER known, and yet, by and large, they are available and reliable.

Its NOT just the way the internet is.

Someone at the top is asleep at the wheel or being led astray by people under him. He may not even know about these problems if he hasn't set up mechanisms that assure that he is aware without an prerequisite human judgement, initiative, or honesty.

marlinspike · April 6, 2006

largelylivin wrote:

There are HUGE internet services with problems that SmugMug cannot imagine that manage to be up 99.998% of the time (that's 3-sigma).

Note that I said photo sharing sites.

DJ-S1 · April 6, 2006

GEEK WARNING -

3-sigma is not 99.998%, it's 99.73% which may not seem like much but it can be. That is 2700 "defects per million", which means that out of 1,000,000 hits to a website, 2700 would not go through (if they are claiming 3-sigma reliability).

JimM · April 6, 2006

caseserve wrote:

I'm not trying to start trouble or whine and carry on. It's just that smugmug is down more then the my other site is. I pay the same $100 per year with them. (myinternetservices.com) They have 99.9% uptime and I can attest that they are up.

You are not comparing apples to apples here. Keeping a website up is significantly easier than a hosted application that is being hit by tens of thousands and allowing you unlimited storage. Your $100/yr at your hosting company looks like it gives you 1GB of storage and 5GB of transfer. Don't you get 32GB of transfer with Smugmug?

All in all, I am happy with SM's uptime.

Wireless · April 6, 2006

First of all, I'm real sorry to hear people had problems connecting yesterday. We did have a short bit of chaos within our web cluster when a partition filled up on one of our database machines in the late morning (PDT). A simple script will fix that, and it's my bad for not addressing that sooner. No excuse here, it's going to get fixed.

We do take downtime very seriously, and invest in many redundant systems to minimize the impact of hardware, software, acts of datacenter gremlins, my fat fingers, etc. Of course, I've said it before, all good deeds never go unpunished, and stuff still happens.

Things we could have done better:

- Rotate those stinkin logs like we know we should have
- Better informed customer service of the event

We don't hide from our mistakes, and are fully accountable to you, the customer, regardless of which account you have. You are giving us your hard earned money and you expect a certain level of service. I hope you find that we are able provide that for you, if not, we want to know (here at dgrin, or via email)!

Please note, however, that this is all a complicated system from your seat to the bits residing on the platters of the disks in our datacenter. We monitor much of our side of the service, but lose visibility as your photos are returned back to you when you log onto SmugMug. Because of this, our support is going to tend to examine your end of the fence first. I'm sorry if this seems like we're hiding or passing blame, but we do need to verify some core stuff when our extensive monitoring isn't spamming our cellphones.

In closing, I have come from a large, 'big three' online service provider that may or may not have been mentioned in this thread. I will say that I smiled when I saw 99.998% availability.

onethumb · April 6, 2006

largelylivin wrote:

There are HUGE internet services with problems that SmugMug cannot imagine that manage to be up 99.998% of the time (that's 3-sigma). There are system design techniques that handle component failures and also techniques to handle spiking traffic volume through load-leveling. There are even techniques to be prepared for and work around problems on the Internet Backbone, which aren't even YOUR problems, rather they are problems with service providers. BUT, if you are concerned about availability and reliability and customer satisfaction, then you do worry about it and proactively address it through a variety of system design, operational planning, procedures and readiness, redundancy, altermate routing and load-leveling, business arrangements, etc. etc., etc.

As much as I hate to use Gates in any positive way, imagine if the MS service sites were this shakey. They aren't. How about Adobe? I go there all the time and I don't experience these problems. What about EBay? They have the biggest operational challenges EVER known, and yet, by and large, they are available and reliable.

Its NOT just the way the internet is.

Someone at the top is asleep at the wheel or being led astray by people under him. He may not even know about these problems if he hasn't set up mechanisms that assure that he is aware without an prerequisite human judgement, initiative, or honesty.

I'm the CEO, and my history is in internet operations. I've been doing it for 12 years. In fact, I built the network that hosted eBay and Hotmail (Microsoft), and my Diretor of Operations is from MSN (Microsoft).

As a general rule, SmugMug has LESS outages during a given year than Microsoft or eBay. Microsoft's Hotmail will periodically have an entire weekend of outage, and we've all seen eBay have many consecutive hours of downtime, particularly during their booming growth in the few years after their IPO.

They're multi-billion-dollar companies. We're a multi-million-dollar company. Having said that, though, I don't buy the "it's tougher for the smaller guys, cuz they don't have enough resources" thing, so I won't use it as an excuse. It just means we have to get smarter - which we've always been and will continue to be.

We're well aware of both network and systems design techniques to minimize and/or eliminate downtime. One of the major reasons for the more frequent scheduled maintenance windows is that we've basically doubled our network infrastructure to provide multiple levels of redundancy in the last 3 months. We're almost done, too. Systems are next.

But there will always be downtime. We don't charge an arm-and-a-leg for service, so we can't provide bank-grade uptime. (My bank, and it's a large one, has outages too... so so-called bank-grade infrastructure isn't infalliable either).

In past years, we've maintained a 99.99% uptime, which is pretty dang good. Our big DDoS attack earlier this year scuttled our ability to claim that this year, but we're still doing well.

You can bet we'll continue to improve, but we make it easy for you to leave AND take your photos with you. Vote with your dollars if we're doing a bad job - we'll get the message.

Don

cambler · April 6, 2006

For what it's worth, I have to back up the MSFT downtime issue. I, too, am ex-MSFT (indeed, I still live in Redmond). I worked on both Exchange Server and Passport v1 and have first-hand experience at how MSFT's systems are down far more often than anyone suspects. Many a sleepless night was spent figuring out why critical outside systems weren't performing.

Yes, there are things that can be done to improve reliability - but once you get to a certain point, you get diminishing returns on the cost of those improvements. I suspect SmugMug hit that curve a while ago.

largelylivin · April 6, 2006

Don, I offer my appology.
Upon rereading my missive this AM I certainly did overstate my case. :uhoh I resonated to someone's concern about "system outages", due to what ever including user error, then lit off over the statement "that's just the way the internet is". I overreacted in both instances: Irritability cut loose the demon.

Fact is, I just recently joined after making a thorough survey of the photo sites and tools against a checklist of my needs. Smugmug was by far the number one service and I am happy to have found it.

So, I appologize for starting your day like that and encourage you to keep my money.

largelylivin · April 14, 2006

another appology and explaination.
Okay. It turns out that I had a virus that was eating up all my CPU time and causing constant time-outs on Smugmug, the only site I was trying to use. My appologies once again.

Andy · April 14, 2006

largelylivin wrote:

Okay. It turns out that I had a virus that was eating up all my CPU time and causing constant time-outs on Smugmug, the only site I was trying to use. My appologies once again.

Glad you have pinpointed the problem. Computers are great, except when they're teh suck, right? Thanks for coming back to this!

MPerdomo · April 14, 2006

I just want to chime in and say that as bad as people are painting it out to be, Smugmug schedules their downtime at very convenient times as far as I am concerned. I've never had a problem.

Also, Smugmug provides a top notch level of support for the prices they charge. They've hooked me up with way more features than I could have hoped for in my wildest dreams. And when something breaks, they have one on one email response, and Dgrin. And as for Dgrin, when you post a complaint and it gets addressed by the CEO of smugmug, it's pretty hard to complain about service.

Just wanted to say that you have a loyal customer right here.

Smugmug down time: Concerns?

Comments