Options

11:49 ET - Is Smug MugDown? Outage Update

13468915

Comments

  • Options
    lashmonsterlashmonster Registered Users Posts: 18 Big grins
    edited July 20, 2008
    Sure, no one likes outages, and am pretty sure this has put a damper on the SM folks weekend as well. But as others have said - our photos are safe and will be back online when this is over. Really, with everything else going on around us on a daily basis, does this deserve that much moaning and groaning?

    I have two SM accounts - a pro and personal one. No other site has the cool third party tools to make using two accounts as easy as clicking a button in your browser. No other company has such fantastic support either.

    And while yes we pay for this service, how many of us write to SM on an average day just to say thanks for everything they assist with? So THANK YOU for the great service.

    Thanks for the silly SmugMug graphic to alert viewers - I'd choose that over serious.

    Diane
    pawsafe.smugmug.com
    Saving one life might not change the world, but it makes a world of difference for the one life you saved.
  • Options
    ebswiftebswift Registered Users Posts: 94 Big grins
    edited July 20, 2008
    Andy wrote:
    From Amazon:

    4:03 PM PDT We are in the process of restoring Amazon S3 US, and will shortly be bringing the web servers back online. We expect the service will be restored within 1 hour.
    Excellent, looking forward to it being back up and running. My 2c to the newcomers to SmugMug, don't be put off, one thing you will learn about the SmugMug team is they are very open with their communication and are nice people.

    I researched all the other services and SmugMug is the best hands down.

    Try Smugfoto in Facebook for displaying your SmugMug galleries.
  • Options
    timk519timk519 Registered Users Posts: 831 Major grins
    edited July 20, 2008
    Images not seen in some threads
    I've been looking at some of the threads, and there's obviously pics posted to them, but I'm not seeing them.

    Here's an example: http://www.dgrin.com/showthread.php?t=100146

    I'm running FF 3.0.1 - could there be a setting which I need to change to see images in these threads?
    • Save $5 off your first year's SmugMug image hosting with coupon code hccesQbqNBJbc
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited July 20, 2008
    timk519 wrote:
    I've been looking at some of the threads, and there's obviously pics posted to them, but I'm not seeing them.

    Here's an example: http://www.dgrin.com/showthread.php?t=100146

    I'm running FF 3.0.1 - could there be a setting which I need to change to see images in these threads?
    http://smugmug.wordpress.com/
  • Options
    PKBPKB Registered Users Posts: 6 Beginner grinner
    edited July 20, 2008
    So if it IS up within the hour - how long will it then take smugmug to get back up again once all is well at Amazon?
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited July 20, 2008
    PKB wrote:
    So if it IS up within the hour - how long will it then take smugmug to get back up again once all is well at Amazon?
    Pretty fast, we hope - we'll run in read only mode for a bit to ensure that things are all good. More as we know it!
  • Options
    chrisb321chrisb321 Registered Users Posts: 10 Big grins
    edited July 20, 2008
    Member now for 2 weeks and I've seen 3 separate downtime issues...

    I jumped from Fotki who I was with for 3 years as I preferred the SM format and customization. I never witnessed any unscheduled downtime with those guys.

    Now I'm wondering if I did the right thing.

    Maybe SM does not normally have these issues, maybe I joined during a bad 2 weeks. Either way, I'm frustrated
  • Options
    PKBPKB Registered Users Posts: 6 Beginner grinner
    edited July 20, 2008
    Andy wrote:
    Pretty fast, we hope - we'll run in read only mode for a bit to ensure that things are all good. More as we know it!

    That is not the type of response I can use to make decisions on. Please clarify - is it

    a) An hour maximum
    b) probably two to four hours
    c) four to ten hours
    d) ten to twenty four hours
    e) more than twenty four hours.

    Your response could be understood to mean any of these.

    Also - will your team work through the night if necessary?

    I understand that you are already under a lot of pressure BUT I just need to know how long "pretty fast" is.

    Thanks
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited July 20, 2008
    chrisb321 wrote:
    Member now for 2 weeks and I've seen 3 separate downtime issues...

    I jumped from Fotki who I was with for 3 years as I preferred the SM format and customization. I never witnessed any unscheduled downtime with those guys.

    Now I'm wondering if I did the right thing.

    Maybe SM does not normally have these issues, maybe I joined during a bad 2 weeks. Either way, I'm frustrated
    I've said this 10x in this thread, sorry to repeat:

    It's really rare for us. I'll stack our record of uptime up against any service. Yes, today has sucked and there's been another read only outage a week ago, and it seems bad and all but it's really just bad timing for you. Sorry your entry to SmugMug has been like this -- I can promise better days ahead!
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited July 20, 2008
    PKB wrote:
    That is not the type of response I can use to make decisions on. Please clarify - is it

    a) An hour maximum
    b) probably two to four hours
    c) four to ten hours
    d) ten to twenty four hours
    e) more than twenty four hours.

    Your response could be understood to mean any of these.

    Also - will your team work through the night if necessary?

    I understand that you are already under a lot of pressure BUT I just need to know how long "pretty fast" is.

    Thanks
    We'll never give an exact estimate, I'm sorry. We'll be back up as soon as is practical and possible and safe. I hope it's minutes not hours.

    Yes, we'll stay up till it's all back to normal. We can sleep when we're dead.

    Sorry I don't have a more precise answer for you.
  • Options
    cdonovancdonovan Registered Users Posts: 724 Major grins
    edited July 20, 2008
    ne_nau.gif
    Are all you photographers so caught up on work that all you have to do is upload photos?

    headscratch.gif I think I should do an understudy with you to see what I am doing wrong, maybe I'm not upset cause I spent the day on the beach, I'm not sure, but guys really, they are on top of it. They know what they are doing, trust that they will do what it takes to get you/us off their back as quickly as possible!thumb.gif
  • Options
    futurowomanfuturowoman Registered Users Posts: 2 Beginner grinner
    edited July 20, 2008
    I have been a Smugmug subscriber since spring 2005, and all in all, I really, really love Smugmug. I rarely look at the forums and I don't participate in any communities here, but I use smugmug to back up all of my (thousands and thousands) of images and for the links to use in many websites, etc. I really, really appreciate smugmug's customer service and record of staying up. i've referred several people to the site, too. I want to say that loud and clear before I'm about to say what i'm going to say.

    I've been having really slow performance from smugmug for a couple of weeks now. It was so bad last week that I contacted support and Doc ran tests on my account. I was told it was my ISP, not smugmug. However, it was NOT my ISP, and now I see that amazon and smugmug have been having all kinds of problems (according to the smugmug wordpress blog) and those problems were exactly what i was reporting. Moreover, I tested this problem on many computers not only with several ISPs, but in towns across the country. It was always happening only on my smugmug and for images on other pages that were remotely linking to my smugmug.

    This has really made me "bummed out" about smugmug, because i've always had such excellent service from them. If there have been all of these problems, why not just say so? Why blame my ISP when it's not my ISP? For now, I will stay with smugmug; it's been far more reliable and pleasant--and certainly more friendly--a place than any other sites i've tried. However, I really hope that smugmug will be more open about problems and not take the common customer service route of blaming the ISP. i know on one level this is "just" the internet. However, when you make part of your living from it, it's not "just" the internet.
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited July 20, 2008
    Why blame my ISP when it's not my ISP?
    At the time, when examining the traceroutes and other data, that's the conclusion we came to.

    I'm sorry that there've been these problems and we'll do everything possible to restore you to normal, fast, service.

    Write us this week at the desk if you continue to have speed issues, we'll get to the bottom of it.

    Sorry again and thanks for sticking with us!
  • Options
    petebockenpetebocken Registered Users Posts: 31 Big grins
    edited July 20, 2008
    Get off the internet, go spend time with your family/kids/friends. Go take more pictures to put on SmugMug later when it's back up. :D
  • Options
    elguevonelguevon Registered Users Posts: 3 Beginner grinner
    edited July 20, 2008
    petebocken wrote:
    Get off the internet, go spend time with your family/kids/friends. Go take more pictures to put on SmugMug later when it's back up. :D

    Yeah...let me tell that to my customers who are trying to make purchases. There is a customer-facing side to this, not just us photogs pissed because we can't upload pictures.
  • Options
    dugmardugmar Registered Users Posts: 756 Major grins
    edited July 20, 2008
    petebocken wrote:
    Get off the internet, go spend time with your family/kids/friends. Go take more pictures to put on SmugMug later when it's back up. :D

    Tell that to my clients that had me shoot their wedding and wanted to view the slideshow together today before everyone that flew in for the weekend takes off.
  • Options
    MikeMcA²MikeMcA² Registered Users Posts: 177 Major grins
    edited July 20, 2008
    petebocken wrote:
    Get off the internet, go spend time with your family/kids/friends. Go take more pictures to put on SmugMug later when it's back up. :D

    Are you kidding? It's brutal hot and muggy here in NY, so I'm sittin' right here under the AC, reading blogs, and watching Criminal Intent until Al Gore's Interweb comes back up to full functionality. Hmmph. :ivar
  • Options
    dwterrydwterry Registered Users Posts: 209 Major grins
    edited July 20, 2008
    I hope you will pass back to Amazon our complaints and let them know how badly they have soiled SM's reputation. We're at 8 hours and counting ... a lot of your customers have customers ... and those customers have no clue it's an Amazon problem. They think it's "our" problem and are blaming us.

    Amazon is going to owe you "big time" for this. I don't know how you'll collect, or if it will be possible to pass anything back to your customers, but this needs to not happen again and the best way to ensure it doesn't is to hit 'em in the pocket book (same place where we're being hit).

    Next time there will be redundancies built into the system...
  • Options
    kensingtonkensington Registered Users Posts: 63 Big grins
    edited July 20, 2008
    kriskros wrote:
    I've been in the IT business for sometime now.

    Disaster Recovery is on top of the list of priorities. Recovery from an outage, if not caused by nature or unforeseen damage to facilities, should be in minutes and not in hours.

    I've been in lead role handling disaster recovery plan.

    First in my agenda is: "Don't put all your eggs in one basket."

    If something fails, not the whole organization will be affected.

    And as I've said in my earlier comment about mirroring, it is damned too important in any business dealing with networks to adopt mirroring. If SmugMug has this and I believe they do, it is not working.

    Real time replication matters in some businesses like banks. Maybe not as serious with photo hosting. But still, this is very critical not only to the business of Smugmug but their customers' business.

    This is reality check and changes in operations, backups, recovery must be instituted because this is totally not acceptable.

    While I am composing this, I checked first my site if it is already up and running. Unfortunately, it is still down.

    I agree, real time replicaiton is the way to go. Throw in roll back capabilities with regular backups to top it off.

    This started out for us as a good idea and is now almost a full time position but my company sees the worth in it.

    A total outage such as this would be unaceptable. Read only mode in the past has been aceptable but todays total outage is a major dissappointment. Especially as I sit here at my parents 40th Anniversary party with family and friends hoping to show off the multitude of pictures...
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited July 20, 2008
    Amazon wrote:
    5:00 PM PDT Amazon S3 service has been restored and is returning to normal. We will continue to monitor closely.

    OK hopefully we're not far behind now...
  • Options
    Glory2Jesus4PhotographyGlory2Jesus4Photography Registered Users Posts: 190 Major grins
    edited July 20, 2008
    Most all my customers have my phone # if it is that pressing then they can call the order in and leave it on the voice mail if I feel that it is so much more important then the rest of my life on a Sunday and they should come before God my Family and my free time then I will get to them. I guess every one is lucky it is not in my hands because I would tell you I will start on it in the morning. It is Sunday and you are willing to complain to the people that are willing to work for you when the do not and should not have to.
    elguevon wrote:
    Yeah...let me tell that to my customers who are trying to make purchases. There is a customer-facing side to this, not just us photogs pissed because we can't upload pictures.
    I know my spelling and grammar are poor some times my spell check says "I got nothing
    for you" and there/ their is no grammar check yet so please forgive me Jesus did.
    My Web site:
    http://Glory2Jesus4Photography.smugmug.com/
    My blog: http://glory2jesus4photography.blogspot.com/
  • Options
    jchinjchin Registered Users Posts: 713 Major grins
    edited July 20, 2008
    petebocken wrote:
    Get off the internet, go spend time with your family/kids/friends. Go take more pictures to put on SmugMug later when it's back up. :D

    Couldn't put it better myself.
    Johnny J. Chin ~ J. Chin Photography
    FacebookFlickrSmugMug
    SmugMug referral coupon code: ix3uDyfBU6xXs
    (use this for a discount off your SmugMug subscription)
  • Options
    PKBPKB Registered Users Posts: 6 Beginner grinner
    edited July 20, 2008
    petebocken wrote:
    Get off the internet, go spend time with your family/kids/friends. Go take more pictures to put on SmugMug later when it's back up. :D

    Well Pete - You obviously think very small - this is a worldwide issue, and we are not talking about just a few dollars here!!!!

    i am in the UK where it is now 1 am in the morning and stand to lose a lot of money if peple cannot see my auction photos which are close to ending. It IS a time sensitive issue.

    It is 1 in the morning here and frankly I would rather be in bed, but I have to wait up and create some sort of alternative which may mean no sleep at all.

    Oh - but yeah - you obviously do not have much to lose by your glib comment - I think you have offended many people here.
  • Options
    rickstrickst Registered Users Posts: 103 Major grins
    edited July 20, 2008
    We'll never give an exact estimate, I'm sorry. We'll be back up as soon as is practical and possible and safe. I hope it's minutes not hours.

    While I appreciate all the work both Amazon and SM have been doing to get the system back up, I have to comment on your comment. I work in the manufacturing sector in Quality Assurance and, frankly, if I answered a customer with the statement above, I wouldn't have the customer any more.

    The frustrating thing about this whole incident is that we can't get a time estimate for when the system will be "up" again. For those doing business using SM, time is money--they need to be able to give some kind of estimate to their customers.
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited July 20, 2008
    rickst wrote:
    We'll never give an exact estimate, I'm sorry. We'll be back up as soon as is practical and possible and safe. I hope it's minutes not hours.

    While I appreciate all the work both Amazon and SM have been doing to get the system back up, I have to comment on your comment. I work in the manufacturing sector in Quality Assurance and, frankly, if I answered a customer with the statement above, I wouldn't have the customer any more.

    The frustrating thing about this whole incident is that we can't get a time estimate for when the system will be "up" again. For those doing business using SM, time is money--they need to be able to give some kind of estimate to their customers.
    Hi, thanks for posting.

    How can I answer what I don't know?

    I do know that our engineers are working as fast as superhumanely possible and we won't stop till things are back up.

    I'm not going to say a time estimate only to have it be missed. There are thousands of people viewing this thread that would be very disappointed, you included, I imagine.

    Let us get to it, I promise it'll be as fast as we can and not one second longer.

    I wish I had a better, more precise answer for you.
  • Options
    dugmardugmar Registered Users Posts: 756 Major grins
    edited July 20, 2008
    jchin wrote:
    Couldn't put it better myself.

    You'd think differently if you had paying customers depending on you.
  • Options
    pabsterpabster Registered Users Posts: 17 Big grins
    edited July 20, 2008
    dugmar wrote:
    You'd think differently if you had paying customers depending on you.

    ...And the money was coming out of your wallet.

    Such apathy doesn't reflect well.
  • Options
    futurowomanfuturowoman Registered Users Posts: 2 Beginner grinner
    edited July 20, 2008
    Andy: thanks for your reply.

    To all of the "smug mugs" out there with their comments about Sunday and playing with family and Jesus and whatever else: I don't care for your attitude and glibness. That's not helpful and your smugness is lame.
  • Options
    BuzzlightyearBuzzlightyear Registered Users Posts: 10 Big grins
    edited July 20, 2008
    I don't know if anyone else has posted this...but let's say that SM were to have 1 - 8 hour outage per month (and they don't). That would equate to a 98.8% uptime. Even that would be pretty impressive. If I reacted to every "outage" of service (internet or not), I would have nowhere left to shop, bank, etc. anymore. Let's put some perspective on this.
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited July 20, 2008
    Amazon wrote:
    5:12 PM PDT We are confirming that service in both the US and EU has been fully restored. We appreciate your patience. We will provide more detail on this event once we have completed a full investigation.

    We, SmugMug, we're still working on bringing things back up.

    Hang in there folks, yes, it's been a long day and again, we're very sorry for the outage. We appreciate your patience, you are all amazingly patient.
Sign In or Register to comment.