Options

11:49 ET - Is Smug MugDown? Outage Update

2456715

Comments

  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited July 20, 2008
    pabster wrote:
    Perhaps such an attitude is fine for a free site, but SmugMug is not free. And considering many are professionals selling their work... I realize crap happens and all but perhaps they need to look at their systems and get it shored up. I'd be pretty steemed if I were losing money because of it, rather than a few family members missing out on some photos.
    Hi, we're amazingly shored up thumb.gif We have a fortune invested in your photos, keeping them safe and secure.

    We're really sorry about this temporary service interruption.

    If you'd like to see more about our setup, check our CEO's blog:

    http://blogs.smugmug.com/don/?s=s3
    http://blogs.smugmug.com/don/?s=datacenter
  • Options
    RBrogenRBrogen Registered Users Posts: 1,518 Major grins
    edited July 20, 2008
    Peter wrote:
    Wow . . . it appears that support is doing everything they can. Some comments are a tad harsh. There is no such thing as perfection and problems can and will arise. It's how fast and earnest the response is in such situations. And that is where Smugmug is hard to beat . . .

    Peter

    Absolutely agree! I manage a large international company IT division and there are absolutely no 100% full-proof systems. The differentiator between service providers is the response that is applied to situations when they occur. While the outage is a pain for all concerned, I'm fully confident that the supporting cast at smugmug are doing everything they can to expedite the return to service.

    Those who seem quick to cast dispersions are not well informed about smugmug or technology in general. My admiration for the team stands firm.
    Randy Brogen, CPP
    www.brogen.com

    Member: PPA , PPANE, PPAM & NAPP
  • Options
    MontecMontec Registered Users Posts: 823 Major grins
    edited July 20, 2008
    All pro users should take a page out of the SM staffs book and let all your customers know that there is a problem.

    Just as you are holding SM responsible your customers hold you responsible. Get a quick notice out to your important clients (weddings) and let them know the situation and turn this into a positive customer relations experience for your business.

    People understand when problems happen and are explained, but do not like it if they waste a bunch of time because they were not informed.

    I for one appreciate the quick information regarding this problem.
    Cheers,
    Monte
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited July 20, 2008
    Montec wrote:
    All pro users should take a page out of the SM staffs book and let all your customers know that there is a problem.

    Just as you are holding SM responsible your customers hold you responsible. Get a quick notice out to your important clients (weddings) and let them know the situation and turn this into a positive customer relations experience for your business.

    People understand when problems happen and are explained, but do not like it if they waste a bunch of time because they were not informed.

    I for one appreciate the quick information regarding this problem.

    Thanks. Again, status updates, here:

    http://smugmug.wordpress.com
  • Options
    toberstobers Registered Users Posts: 180 Major grins
    edited July 20, 2008
    Thanks for the updates Andy. Great customer contact as usual.

    I've found Smugmug to be getting slower recently. Though this may be the first unplanned outage for ages (good for you etc) being in the UK I do get hit by your planned downtimes which are in the night USA time but daytime here.

    Awiting the fix eagerly...
  • Options
    TwistedJTwistedJ Registered Users Posts: 193 Major grins
    edited July 20, 2008
    Smug down again.......?
    Anybody know of another site good at hosting? This is getting to be too much for me to handle.
  • Options
    dugmardugmar Registered Users Posts: 756 Major grins
    edited July 20, 2008
    Thanks for the updates. I have to say though, timing could not have been any worse for me. Any other day of the year would have been fine, but today, after posting a 7 page wedding gallery for a client just after I emailed them the smugmug URL. Ugh... Money out of my pocket for sure. I am getting emails back asking for a CD of the photos now. Time to go explain...
  • Options
    DrDavidDrDavid Registered Users Posts: 1,292 Major grins
    edited July 20, 2008
    TwistedJ wrote:
    Anybody know of another site good at hosting? This is getting to be too much for me to handle.
    You do know that it is Amazon S3 that is down--not Smugmug. Amazon S3 looks like it had some hardware failures which caused Smugmug's storage to go offline.

    *ALL* sites have downtime. It's the nature of the beast. Computers die, harddrives die, humans mess up, programs crash, etc.. The internet, and all sites on it, are all fallible.

    David
  • Options
    txsapphotxsappho Registered Users Posts: 3 Beginner grinner
    edited July 20, 2008
    pabster wrote:
    Perhaps such an attitude is fine for a free site, but SmugMug is not free. And considering many are professionals selling their work... I realize crap happens and all but perhaps they need to look at their systems and get it shored up. I'd be pretty steemed if I were losing money because of it, rather than a few family members missing out on some photos.

    This issue is with Amazon not SmugMug which means it is out of their control at this point but they are working on it. SmugMug can "shore up" their systems and could still have this problem if one of their providers went down. No system has 100% uptime. You may get 99.8% or 99.9% but there is always the .2% or .1% that is downtime - planned or unplanned.

    SmugMug has been very responsive and is working with Amazon to restore service. I have clients on hold too, like everyone else, and they are being patient. I have money on the line too, like everyone else, BUT... what good is it going to do anyone to get pissy about it. It won't make Amazon get their system back online any faster...
  • Options
    denisegoldbergdenisegoldberg Administrators Posts: 14,242 moderator
    edited July 20, 2008
    Thanks for the message on our sites!
    Hey Smug -

    Thanks so much for the downtime message on our sites - http://www.smugmug.com/down/outage.mg. I much prefer seeing a message like this one than just having photos go missing. This way if anyone is trying to access my photos they can see what is happening and (hopefully) will come back to my site later.

    --- Denise
  • Options
    Karenl39Karenl39 Registered Users Posts: 155 Major grins
    edited July 20, 2008
    I've had photos on smugmug for 3 or maybe 4 years now. Honestly, this is the very first time I have seen this happen. Smugmug is very reliable and they have all of our pictures backed up in other locations in case of hurricanes, etc. I'm sure they're working at this day and night.
  • Options
    PupatorPupator Registered Users Posts: 2,322 Major grins
    edited July 20, 2008
    I'm glad that users who try to go to my site get an error message explaining exactly what's happening - and the server farm graphic is hilarious! rolleyes1.gif
  • Options
    affleckphotoaffleckphoto Registered Users Posts: 9 Beginner grinner
    edited July 20, 2008
    Goodbye Smugmug
    I'm afraid I cannot just accept the answer "we're sorry" any longer. I'm a firm believer that all business-related decisions should be proactive rather than reactive, otherwise, how can you ever be more than one step behind? I have read the status update at http://smugmug.wordpress.com/, and I find that, "Our faith in Amazon, and the care they take of your priceless memories, hasn’t been shaken." and, "...we’ve been expecting an outage like this for awhile, and there will probably be more in the future.", are responses that display a reactive approach rather than proactive, and are simply not acceptable.
    I have been actively searching for an alternate solution with regard to hosting/proofing/order fullfillment for some time now, largely because of the lack of timely features being added such as print credits/coupons and Global currency support, as well as ongoing branding issues in relation to SEO solutions. A response such as that shown on http://smugmug.wordpress.com/, is confirmation that I have not been proactive enough in finding another solution.
    In this case, Smugmug, you have lost this customer. It is now just a question of when I will be cancelling this service.
  • Options
    BuzzlightyearBuzzlightyear Registered Users Posts: 10 Big grins
    edited July 20, 2008
    I have been a lurker in the forum and with SM for about 5 - 6 months. Reading some of the negative comments, I felt compelled to post. While we all feel the pain during service outages, they happen to all web/IT services. This is the first unplanned outage since I joined. The tech support here is terrific and are usually working on issues before anyone notices. You won't find better customer service anywhere. To those who are new to the service, this is not a common occurance. Thanks SM!!!
  • Options
    kriskroskriskros Registered Users Posts: 29 Big grins
    edited July 20, 2008
    MIRROR, MIRROR, on the wall, who is the fairest of them all!!!
    Unfortunate incident. But these things do happen.

    My satisfaction level for SmugMug is way up there. Their support is very hard to beat. Response time is excellent.

    Unfortunately, no system in this world is perfect. This is part of our lives that we must fully understand. There is light. And there is darkness.

    Right now, many of us, want the service to be back. Three (3) hours have gone since this was reported and we still have no idea when our photos will be back.

    I suggest that mirroring of data be provided by Amazon. So, in case something like this happens, the mirrored data could be switched and service interruption will be minimized.


    I still love SmugMug. Are there any other options that can better the quality service we are getting from SmugMug? I doubt it.

    Now, let's wait.


    Kris Kros
    http://kriskrosphotography.com
    http://flickr.com/kros
  • Options
    shutterbugladyshutterbuglady Registered Users Posts: 16 Big grins
    edited July 20, 2008
    I've been with Smugmug for several years now and have no complaints. If you take into consideration how many photos they handle -- close to 400 million! -- I think they do a remarkable job. They're always quick to respond to any and all questions, even in the middle of a setback such as this. And, this isn't Smugmug that's down, but Amazon instead. Other sites could learn a thing or two about customer service, something Smugmug, at least IMHO, has mastered to a fine art, pun intended.

    These things happen, but they've been infrequent for my time as a member. I'm very pleased with Smugmug and will continue to be unless they permanently lose my pictures. But, hey, that's what backup CDs are for!
  • Options
    MontecMontec Registered Users Posts: 823 Major grins
    edited July 20, 2008
    I'm afraid I cannot just accept the answer "we're sorry" any longer....In this case, Smugmug, you have lost this customer. It is now just a question of when I will be cancelling this service.

    I am not trying to defend SM or start an argument with you but do you honestly believe that this could not happen to 'any' service you might choose to use?
    Cheers,
    Monte
  • Options
    pat.kanepat.kane Registered Users Posts: 332 Major grins
    edited July 20, 2008
    I'm afraid I cannot just accept the answer "we're sorry" any longer. I'm a firm believer that all business-related decisions should be proactive rather than reactive, otherwise, how can you ever be more than one step behind? [...]

    Everyone has their own perspective. Mine is that smugmug's use of the Amazon service was very proactive on their part. They were an early adopter of the system, first as a backup and apparently now as primary for some content.

    If you had been reading these forums as long as I have, you'll know that adopting the Amazon service was a carefully thought out business decision that allowed the company to focus on their core competence, which is the interface and software that runs smugmug vice investing in and maintaining storage farms.

    Just as the move to Amazon was based on business needs, I'm sure there would be a change to something else if the service does not provide the reliability that is to be expected.
  • Options
    MontecMontec Registered Users Posts: 823 Major grins
    edited July 20, 2008
    Well this is good news for SM...compensation should be in order
    The service, known as Amazon S3, has been particularly attractive to startups that can't afford infrastructure and don't know how quickly they will grow. But until now it suffered a notable flaw: Amazon didn't guarantee its servers' reliability, or even that this line of business would exist tomorrow. Now, if S3 doesn't meet the 99.9 percent mark, Amazon will offer customers compensation.

    From the Wired news site.
    Cheers,
    Monte
  • Options
    kriskroskriskros Registered Users Posts: 29 Big grins
    edited July 20, 2008
    Montec wrote:
    Well this is good news for SM...compensation should be in order



    From the Wired news site.

    Great find. That was almost a year old news.

    Just wondering if Amazon.com is up and running. Oh yes. Amazon is online.
  • Options
    buddy1buddy1 Registered Users Posts: 29 Big grins
    edited July 20, 2008
    It just really "bite" not having access to the server and not knowing when it will be back up and running. I have to find another back up in case of a similar failure in the future so Clients can still get access to their photos.
  • Options
    CameronCameron Registered Users Posts: 745 Major grins
    edited July 20, 2008
    In this case, Smugmug, you have lost this customer. It is now just a question of when I will be cancelling this service.

    When you find that perfect service let us know. thumb.gif
  • Options
    TwistedJTwistedJ Registered Users Posts: 193 Major grins
    edited July 20, 2008
    True I've been building web sites for years and years. Never had as much downtime as Smug has given me or Amazon or whomever. I've only been a member since February and this is like the 6th time now.
  • Options
    dereksurfsdereksurfs Registered Users Posts: 286 Major grins
    edited July 20, 2008
    Montec wrote:
    Well this is good news for SM...compensation should be in order
    I have to agree with most posts here about SM service I have experienced to date. Between these forums and quick responses to questions and concerns I have had it has been great!

    On the other hand from a business hosting side I don't think this kind of outage is acceptable from Amazon. I also work in IT for Verizon. And let me tell you that if any kind of network or telecommunication service goes down for any of our customers things need to get back online quickly.

    Also data server and web server redundancy *Must* exist period! So that *When* an outage occurs the backup servers go online.

    So my question is are there no backup servers? If not then why not? If so then why didn't they do their job?

    Those are some of the real technical questions that must be answered and dealt with after the dust settles.

    Also notice if you go to Amazon right now you can shop to your hearts content. I guarantee you they have back servers which work for all of their ecommerce business. And if they were ever down for this length of time it would mean huge losses of revenue. Heads would roll.
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited July 20, 2008
    I'm afraid I cannot just accept the answer "we're sorry" any longer. I'm a firm believer that all business-related decisions should be proactive rather than reactive, otherwise, how can you ever be more than one step behind? I have read the status update at http://smugmug.wordpress.com/, and I find that, "Our faith in Amazon, and the care they take of your priceless memories, hasn’t been shaken." and, "...we’ve been expecting an outage like this for awhile, and there will probably be more in the future.", are responses that display a reactive approach rather than proactive, and are simply not acceptable.
    I have been actively searching for an alternate solution with regard to hosting/proofing/order fullfillment for some time now, largely because of the lack of timely features being added such as print credits/coupons and Global currency support, as well as ongoing branding issues in relation to SEO solutions. A response such as that shown on http://smugmug.wordpress.com/, is confirmation that I have not been proactive enough in finding another solution.
    In this case, Smugmug, you have lost this customer. It is now just a question of when I will be cancelling this service.
    I sorry to see you go because of this, I really am. If you are open to options and will consider staying, write to me at our help desk, Attn: Andy when you have the time.
  • Options
    kygardenkygarden Registered Users Posts: 1,060 Major grins
    edited July 20, 2008
    I'm not one to whine too much and I know it's Amazon's fault....but this outage honestly is going to cost me money. My ebay auctions have sometimes more than one photo per listing so the customer can see all the items I have for sale. In this case I have at least one auction going that is supposed to have 3 photos showing. Without all the photos showing, people will think there's less of what I really have to sell and therefore bid much less...and I'll end up getting screwed out of that extra money since the buyers didn't know there were actually more items/pieces that go with the auction. I know they're doing all they can to fix this, but dang it sucks. I even had people (when the photos WERE showing) asking me if I'd be willing to do a Buy It Now option for the items for $100....but now I may be lucky to get $50 since the bidders don't know there are many other items with the auction. Sucks :(
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited July 20, 2008
    kygarden wrote:
    Sucks :(
    Yup, it does. Cancel the auction, and relist?
  • Options
    flownmuseflownmuse Registered Users Posts: 13 Big grins
    edited July 20, 2008
    There are many photo sites, each with their own strengths and weaknesses. All will experience downtime via site maintenance or unplanned outages.

    For me, the deciding factor is customer service. I will always pay a little bit more, and sometimes even put up with more "glitches" (not referring to Smugmug per se - this is the first time I've ever found the site down like this!) than usual because I know that I will continue to get excellent customer service.

    This experience has actually helped confirm that I made the right choice with SM.

    Now - where are me pics?!?! rolleyes1.gif
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited July 20, 2008
    TwistedJ wrote:
    Anybody know of another site good at hosting? This is getting to be too much for me to handle.
    I'm so sorry :( It won't be much help, given the time downtime right now, but when you look at our history and track record, I'll stack our uptime up against anyone's. It's been extremely, extremely high.

    I'm sorry today sucks.
  • Options
    roliirolii Registered Users Posts: 28 Big grins
    edited July 20, 2008
    I'm afraid I cannot just accept the answer "we're sorry" any longer. I'm a firm believer that all business-related decisions should be proactive rather than reactive, otherwise, how can you ever be more than one step behind? I have read the status update at http://smugmug.wordpress.com/, and I find that, "Our faith in Amazon, and the care they take of your priceless memories, hasn’t been shaken." and, "...we’ve been expecting an outage like this for awhile, and there will probably be more in the future.", are responses that display a reactive approach rather than proactive, and are simply not acceptable.
    I have been actively searching for an alternate solution with regard to hosting/proofing/order fullfillment for some time now, largely because of the lack of timely features being added such as print credits/coupons and Global currency support, as well as ongoing branding issues in relation to SEO solutions. A response such as that shown on http://smugmug.wordpress.com/, is confirmation that I have not been proactive enough in finding another solution.
    In this case, Smugmug, you have lost this customer. It is now just a question of when I will be cancelling this service.

    Please let us know when and if you do find that perfect, 0% downtime photo hosting service.
Sign In or Register to comment.