Options

Maintenance Alert: 8/28 10:00pm - 06:00am PDT

1246

Comments

  • Options
    renstarrenstar Registered Users Posts: 167 Major grins
    edited August 29, 2006
    notification
    I'm going to Nth the request for an email notification during outages, though, I'm gonna request something slightly different.. I don't particularly care if smugmug is down for 2 minutes because of a hickup on their end or something up with the backbone providers or whatnot, or even small planned outtages to bring a server online (say, 5 mins), but when there is a planned outtage that will take on the order of hours and potentially longer if issues come up, I dont think any reasonable customer could see it as spam. (and if they do, let me know so i can kick them in the shins)

    Glad you got (most) everything back up and running again. And get that deserved sleep, so I hope you miss this message for a while.
  • Options
    FlashFlash Registered Users Posts: 135 Major grins
    edited August 29, 2006
    JamesJWeg wrote:
    I'm sure we will be told when they are done. I'm another vote for e-mail notification.

    James.

    Why would they want to broadcast a cluster mess to us by email, that won't happen.
    PROMISED less than 60 day since the stat lose and here we are again
  • Options
    SeymoreSeymore Banned Posts: 1,539 Major grins
    edited August 29, 2006
    Notification... Mirror Site...
    I concur with the email notification aspect. Even something as simple as adding a field to the Control Panel that allows the user to opt in or out to Admin email like the announcement that should have happened on this outage. That then links them to the bulk mail server for issues like this. You could also include announcements on upgrades. This could also open the door to other Smugmug plans down the road via email. (read: contests, newsletters, ect...)

    But, I'm really surprised that you opt to reply to all the emails you get, instead of implementing a (granted bulk) system to inform. BTW, @ 2am (PST) all I got was a "The page cannot be displayed..." message on my browser. There was NO redirect.

    And, I know the cost... and have not seen any talk about this... BUT, what about a mirror site, maintained by you guys? Can you say 0 (that's a ZERO, not an O) down time? Has that ever been considered?
  • Options
    JamesJWegJamesJWeg Registered Users Posts: 795 Major grins
    edited August 29, 2006
    Flash wrote:
    Why would they want to broadcast a cluster mess to us by email, that won't happen.
    PROMISED less than 60 day since the stat lose and here we are again

    Dude, check the past, I've blown my lid on here to them defore about changes that pissed me, guess what, they listened to a large degree. Judging from past experiance they will implment some kinda of e-mail notification in the future. As for the stats, I am also annoyed by that, but give them a little time to see if they are really gone.

    James.
  • Options
    flyingdutchieflyingdutchie Registered Users Posts: 1,286 Major grins
    edited August 29, 2006
    photofreak wrote:

    I mean't breathe for the 20-30 seconds it takes to come up. Mine is working...just a tad slow.

    Same on my site.
    Sometimes a gallery takes upto a minute to show up (when doing a 'view source', no data at all is downloaded during the wait), and then all the page's content is downloaded at once (really fast).

    It seems that downloading content (images) is not slow at all; getting the first response from the server is sometimes very slow (generating HTML for galleries, etc).
    I can't grasp the notion of time.

    When I hear the earth will melt into the sun,
    in two billion years,
    all I can think is:
        "Will that be on a Monday?"
    ==========================
    http://www.streetsofboston.com
    http://blog.antonspaans.com
  • Options
    wslamwslam Registered Users Posts: 277 Major grins
    edited August 29, 2006
    Painfully slow (Overseas)
    For the past 24 hours, the site has been painfully slow. I am not writing for myself, but also on behalf of all my buddies here in HK who have signed up to smugmug due to my recommendation. It takes over a min for the site to respond at times. And sometimes, simply time out. I know overseas connections have a lot more bottlenecks, but the last 24 hours has definitely been worse then avg but a *lot*.

    Ping report and Traceroute report below:

    Pinging ws.smugmug.com [63.81.134.23] with 32 bytes of data:

    Reply from 63.81.134.23: bytes=32 time=600ms TTL=52
    Reply from 63.81.134.23: bytes=32 time=509ms TTL=52
    Reply from 63.81.134.23: bytes=32 time=245ms TTL=52
    Reply from 63.81.134.23: bytes=32 time=338ms TTL=52

    Ping statistics for 63.81.134.23:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
    Approximate round trip times in milli-seconds:
    Minimum = 245ms, Maximum = 600ms, Average = 423ms

    C:\Documents and Settings\WSLam>tracert ws.smugmug.com

    Tracing route to ws.smugmug.com [63.81.134.23]
    over a maximum of 30 hops:

    1 2 ms <1 ms <1 ms 192.168.0.1
    2 294 ms 172 ms 165 ms pcd-vta2-1-rx.netvigator.com [203.218.32.254]
    3 365 ms 326 ms 348 ms 218.102.57.70
    4 31 ms 21 ms 73 ms 218.102.21.78
    5 118 ms 160 ms 195 ms unknown.net.reach.com [202.40.142.9]
    6 589 ms 585 ms 373 ms i-5-2.hht-dist02.net.reach.com [202.84.154.98]
    7 273 ms 234 ms 228 ms unknown.net.reach.com [202.84.154.197]
    8 239 ms 325 ms 283 ms i-6-0.paix-core01.net.reach.com [202.84.140.125]

    9 192 ms 243 ms 397 ms i-3-3.paix05.net.reach.com [202.84.251.94]
    10 191 ms 191 ms 203 ms GBLX.peer.paix05.net.reach.com [134.159.62.98]
    11 239 ms 453 ms 662 ms so5-0-0-2488M.ar1.SJC2.gblx.net [67.17.67.154]
    12 216 ms 196 ms 323 ms pri.r1-ge0-2-eq-sj.smugmug.com [206.223.117.70]

    13 209 ms 355 ms 353 ms hera.smugmug.com [63.81.134.23]

    Trace complete.
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited August 29, 2006
    Flash wrote:
    Why would they want to broadcast a cluster mess to us by email, that won't happen.
    PROMISED less than 60 day since the stat lose and here we are again
    I can see you are upset, Flash. The two stats problems are un-related, though. Sorry again for the trouble, and thank you in advance for your patience.
  • Options
    Steve CaviglianoSteve Cavigliano Super Moderators Posts: 3,599 moderator
    edited August 29, 2006
    Slowwwwwwwwwwwwww
    I was just responding to a thread about how slow SM is overseas. The thread disappeared while I was typing.....Laughing.gif

    Anyhow, the site seems way slow today. It is taking 25 seconds to get to a gallery and 20+ seconds to load up an M size image for viewing headscratch.gif

    I sent some SM links to my wife and she sent me back an email asking if her PC was bad, since the pics were taking so long to load.

    So what I want to know is, which one of youse breaka my Smugmug? :D

    Steve
    SmugMug Support Hero
  • Options
    DalantechDalantech Registered Users Posts: 1,519 Major grins
    edited August 29, 2006
    It's slow here in Italy too, but I wouldn't trust ping or traceroute as a trouble shooting tool. A router will drop ICMP packets in favor of "real" traffic -so a congested link between you and SmugMug could account for the high return trip counts. The RFC for traceroute is a joke, because decrementing the TTL counter is a "should" and not a must, so there could be hidden gateways that your traffic is passing through. Also a gateway can remain hidden because it doesn't have to send a TTL exceeded ICMP packet back to the source (another "should"). Finally, due to security reasons, a lot of sites are blocking ICMP all together...
    My SmugMug Gallery

    Looking for tips on macro photography? Check out my Blog: No Cropping Zone.
  • Options
    wslamwslam Registered Users Posts: 277 Major grins
    edited August 29, 2006
    Dalantech wrote:
    It's slow here in Italy too, but I wouldn't trust ping or traceroute as a trouble shooting tool. A router will drop ICMP packets in favor of "real" traffic -so a congested link between you and SmugMug could account for the high return trip counts. The RFC for traceroute is a joke, because decrementing the TTL counter is a "should" and not a must, so there could be hidden gateways that your traffic is passing through. Also a gateway can remain hidden because it doesn't have to send a TTL exceeded ICMP packet back to the source (another "should"). Finally, due to security reasons, a lot of sites are blocking ICMP all together...

    Yes I understand. I just want to post *some* data for ref. If I don't post, some people will ask, now I post, some people will say they are useless =) Can't win! =)
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited August 29, 2006
    I was just responding to a thread about how slow SM is overseas. The thread disappeared while I was typing.....Laughing.gif
    it did not disappear, it's right here:
    http://www.dgrin.com/showthread.php?p=373620#post373620
    Anyhow, the site seems way slow today. It is taking 25 seconds to get to a gallery and 20+ seconds to load up an M size image for viewing headscratch.gif

    I sent some SM links to my wife and she sent me back an email asking if her PC was bad, since the pics were taking so long to load.

    So what I want to know is, which one of youse breaka my Smugmug? :D

    Steve

    Major data center move, as explained in this thread. Sorry for the trouble today!

    And as for blame? HERE
  • Options
    papajaypapajay Registered Users Posts: 441 Major grins
    edited August 29, 2006
    Slowly, but Surely, an improvement.
    photofreak wrote:

    Same on my site.
    Sometimes a gallery takes upto a minute to show up (when doing a 'view source', no data at all is downloaded during the wait), and then all the page's content is downloaded at once (really fast).

    It seems that downloading content (images) is not slow at all; getting the first response from the server is sometimes very slow (generating HTML for galleries, etc).


    Similar experience here...initial page load (connecting to site) is superfast...gallery page loads that follow are super slow.

    One VERY encouraging observation:
    Keyword searches at both Smugmug level and Site Owner level proceed with LIGHTNING SPEED!!!
  • Options
    leebaseleebase Registered Users Posts: 630 Major grins
    edited August 29, 2006
    My site is up now. Alternately fast and slow. On IE that is. For some reason, I can't view my site with Firefox. wierd.

    Tuesday, 3pm Central

    Lee
  • Options
    Dramatapix®Dramatapix® Registered Users Posts: 430 Major grins
    edited August 29, 2006
    leebase wrote:
    My site is up now. Alternately fast and slow. On IE that is. For some reason, I can't view my site with Firefox. wierd.

    Tuesday, 3pm Central

    Lee

    4:17PM EDT still experiencing the very frustrating issues within the galleries. Unable to navigate the pictures, or taking 30 seconds or more at times to open if at all.
    My Gear: D200, D80, 50 f/1.4, 28-75 f/2.8, 55-200 f/4-5.6, 18-55 f/3.5-5.6, 70-200 f2.8, (4) White Lightning Ultra 1200's, SB600, (2) Lightspheres, 17" Macbook Pro, 24" Apple Imac, Thinkpad T42, Epson R-260, PSCS2, Adobe Lightroom, Apple Aperture, PS Elements 4
  • Options
    Steve CaviglianoSteve Cavigliano Super Moderators Posts: 3,599 moderator
    edited August 29, 2006
    Andy wrote:
    it did not disappear, it's right here:
    http://www.dgrin.com/showthread.php?p=373620#post373620


    Major data center move, as explained in this thread. Sorry for the trouble today!

    And as for blame? HERE

    Thanks Andy :D

    I think we have a semantics issue. Technically those threads did disappear. But only because you took the posts and incorporated them into this thread. That's fine with me. If I would have seen this thread to begin with, I wouldn't have started a new thread, duplicating the same topic. But it was on page 2 and I missed it ne_nau.gif

    Sorry to add more ragging to an already over-ragged support staff...lol

    I have the utmost confidence that the site will be up and smokin' as soon as possible thumb.gif

    Steve
    SmugMug Support Hero
  • Options
    DJKennedyDJKennedy Registered Users Posts: 555 Major grins
    edited August 29, 2006
    leebase wrote:
    My site is up now. Alternately fast and slow. On IE that is. For some reason, I can't view my site with Firefox. wierd.

    Tuesday, 3pm Central

    Lee
    I agree - I only use Firefox, and my site is deadly slow. I mean its so slow that it would be faster for me to snail mail prints of my photos.

    Derek
    http://www.djkennedy.com

    What did Cinderella say when she left the photo shop? "One day my prints will come."

  • Options
    FlashFlash Registered Users Posts: 135 Major grins
    edited August 29, 2006
    Paid my renewal fee this month and got a e mail to update my VISA info.

    When it comes to SM getting an e mail to to me to make payment is sure the heck gets to me with no delays or site problems.
    Hmm I wonder why, I wonder why ne_nau.gif
  • Options
    Mike LaneMike Lane Registered Users Posts: 7,106 Major grins
    edited August 29, 2006
    Flash wrote:
    Paid my renewal fee this month and got a e mail to update my VISA info.

    When it comes to SM getting an e mail to to me to make payment is sure the heck gets to me with no delays or site problems.
    Hmm I wonder why, I wonder why ne_nau.gif
    I personally think that's a little unfair. The insinuation is that smugmug is greedy and cares more about its bottom line than its customers. From my experience on dgrin and with the owners of smugmug personally, I have to say that it seems the very opposite is true. The smugmug staff are busting their humps constantly every day (and every night mind you) to give their customers the best service they can. I've yet to see this greed you off-handedly insinuate of them. For instance just this week there have been 2 dgrinners that have very nearly busted their traffic limits for the month. What did smugmug do? They reset their limits. I've seen a number of times when someone wanted out of their smugmug accounts for whatever reason and smugmug simply refunded their money no questions asked.

    I know things can get frustrating but that insinuation IMHO is over the top. But maybe that's just me.
    Y'all don't want to hear me, you just want to dance.

    http://photos.mikelanestudios.com/
  • Options
    Dramatapix®Dramatapix® Registered Users Posts: 430 Major grins
    edited August 29, 2006
    Mike Lane wrote:
    I personally think that's a little unfair. The insinuation is that smugmug is greedy and cares more about its bottom line than its customers. From my experience on dgrin and with the owners of smugmug personally, I have to say that it seems the very opposite is true. The smugmug staff are busting their humps constantly every day (and every night mind you) to give their customers the best service they can. I've yet to see this greed you off-handedly insinuate of them. For instance just this week there have been 2 dgrinners that have very nearly busted their traffic limits for the month. What did smugmug do? They reset their limits. I've seen a number of times when someone wanted out of their smugmug accounts for whatever reason and smugmug simply refunded their money no questions asked.

    I know things can get frustrating but that insinuation IMHO is over the top. But maybe that's just me.

    I understand your wanting to come to the support of the people that run the service, but from my perspective, I'm new here... my account is less than a month old, and to see this kind or performance is a little frustrating. The thing that makes it more frustrating is the fact that I've yet to see anyone with authority offer some kind of explanation other than there has been a major database transfer. Now mind you I've not read every thread, but I've posted numerous messages in two threads regarding performance today and have not seen a reply.

    Just one man's perspective.

    Brett
    My Gear: D200, D80, 50 f/1.4, 28-75 f/2.8, 55-200 f/4-5.6, 18-55 f/3.5-5.6, 70-200 f2.8, (4) White Lightning Ultra 1200's, SB600, (2) Lightspheres, 17" Macbook Pro, 24" Apple Imac, Thinkpad T42, Epson R-260, PSCS2, Adobe Lightroom, Apple Aperture, PS Elements 4
  • Options
    richWrichW Registered Users Posts: 941 Major grins
    edited August 29, 2006
    Originally Posted by Mike Lane
    I personally think that's a little unfair. The insinuation is that smugmug is greedy and cares more about its bottom line than its customers.

    15524779-Ti.gif With Mike.

    After reading this thread http://www.dgrin.com/showthread.php?t=39356 it confirmed to me SmugMug's level of service is very high.

    I think I'll go out and take some pictures today instead of waiting for the servers to come up to speed.

    BTW who gave the keys to the data center to The BigWebGuy?
    Congratulations Lee....
  • Options
    Mike LaneMike Lane Registered Users Posts: 7,106 Major grins
    edited August 29, 2006
    I understand your wanting to come to the support of the people that run the service, but from my perspective, I'm new here... my account is less than a month old, and to see this kind or performance is a little frustrating. The thing that makes it more frustrating is the fact that I've yet to see anyone with authority offer some kind of explanation other than there has been a major database transfer. Now mind you I've not read every thread, but I've posted numerous messages in two threads regarding performance today and have not seen a reply.

    Just one man's perspective.

    Brett

    I've been with smugmug for geeze nearly a year-and-a-half. My personal experience is that there has been 2 or 3 instances unplanned downtime and a few planned outages. The unplanned events weren't anything major as I recall (I think I remember one being a fairly big deal) and the planned outages all went reasonably well. All in all my service has been up FAR more than it's been down. And from what little I know of running a datacenter (not much) I know that no datacenter has 100% uptime. My point about that is, Smugmug is very reliable in my experience.

    As far as explanations, there have been 2 threads where wireless (the guy who is in charge of the datacenter) posted and one thread where onethumb (the CEO of smugmug) posted in fairly reasonable detail what happened and what to expect next. I understand your frustration (I'm working on a project right now that would go so much faster if my smugmug site were going full speed) but I know that these things happen to the best of sites. I'm looking forward to my site being its old self (hopefully better mwink.gif) and I can guarantee that your concerns are not being ignored by the smugmug staff (of which I am not a member BTW).

    That said, feel free to vent all you want in here and elsewhere. I'm not trying to stifle that by any means. I do however think that Flash was way out of line with his accusations.
    Y'all don't want to hear me, you just want to dance.

    http://photos.mikelanestudios.com/
  • Options
    ivarivar Registered Users Posts: 8,395 Major grins
    edited August 29, 2006
    richW wrote:
    I think I'll go out and take some pictures today instead of waiting for the servers to come up to speed.
    Now that sounds like a great idea!
  • Options
    igoforitigoforit Registered Users Posts: 43 Big grins
    edited August 29, 2006
    Very slow and timing out
    Here in Belgium also the response times are not workable. My links in forums to pictures in my albums are sometimes simply not showing up (red cross).
    Hope it will be resolved soon.
    Jos
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited August 29, 2006
    The thing that makes it more frustrating is the fact that I've yet to see anyone with authority offer some kind of explanation other than there has been a major database transfer.

    Hi Dramatapix, thanks for posting. Onethumb (our CEO and Chief Geek) and Wireless (our Director of Operations) are both crashed from staying up all night. They'll be back on soon and will provide an update, I'm sure.

    Thanks for your continued patience - we're sure this will lead to an even stronger SmugMug!
  • Options
    Scott_QuierScott_Quier Registered Users Posts: 6,524 Major grins
    edited August 29, 2006
    I really don't want to sound like a whiner, but...
    onethumb wrote:
    A little more information before I fall into a coma: we managed to move hundreds of servers, and hundreds of terabytes. But one of them (which happens to be a core part of the stats tracking) won't boot now.

    We have spares if we can't fix it, so it's just a matter of taking an hour or so to make sure it's working and the data is intact - but we're all wiped out. And wiped out people messing with mission critical stuff = bad news.

    So we'll get the stats back online as soon as we've gotten a little sleep.

    Sorry for the delay!

    Don
    It's been just about 6 hours (4 mintes shy of 6 hours, actually) and I'm thinking it's time to haul it out of the sack and finish the job or to, at least, proffer an idea of when the job is going to be finished. In the absense of a recent estimate (has there been one posted yet?) the slow response from the SmugMug sites is starting to wear just a bit.

    How about it Andy. You'be been the most responsive in this thread. What have you heard?

    You beat me to the punch by about a minute (while I was reviewing my post).

    Thanks
  • Options
    cdhamescdhames Registered Users Posts: 128 Major grins
    edited August 29, 2006
    working again?
    wow. i just checked my account.. whatever wireless and those boys are doing, KEEP DOING IT!!! my galleries just loaded so fast my eyeballs got singed.
    SM Page: cdhames.smugmug.com
    Referral: ( wXtCbmTTvmJSE )
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited August 29, 2006
    I

    How about it Andy. You'be been the most responsive in this thread. What have you heard?
    We've actually just heard from Onethumb, who did a bit more to the system, and things should be dramatically improved - have a look and report back please :D
  • Options
    Mike LaneMike Lane Registered Users Posts: 7,106 Major grins
    edited August 29, 2006
    Andy wrote:
    We've actually just heard from Onethumb, who did a bit more to the system, and things should be dramatically improved - have a look and report back please :D
    OMG much faster! Nice.
    Y'all don't want to hear me, you just want to dance.

    http://photos.mikelanestudios.com/
  • Options
    igoforitigoforit Registered Users Posts: 43 Big grins
    edited August 29, 2006
    igoforit wrote:
    Here in Belgium also the response times are not workable. My links in forums to pictures in my albums are sometimes simply not showing up (red cross).
    Hope it will be resolved soon.
    Jos
    It doesn't seem to be that bad consistently.... I tried a few things and then I thought it was a problem only with my album that I uploaded yesterday night. That now seems to go faster as well, but I assume that may be because the images have been cashed on my HD by now.
    Hope that response times will be OK (or even better then before) once all issues have been solved.
    Jos
  • Options
    Scott_QuierScott_Quier Registered Users Posts: 6,524 Major grins
    edited August 29, 2006
    Andy wrote:
    We've actually just heard from Onethumb, who did a bit more to the system, and things should be dramatically improved - have a look and report back please :D
    Now that's the kind of response I was expecting of the stellar SmugMug staff.

    Oh, and Andy, my site screams now.
Sign In or Register to comment.