Options

SmugMug weekly maintenance window: Thursday night / Friday morning

11012141516

Comments

  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited September 11, 2009
    RuSu wrote:
    window - will it be used?
    nope, we do post in advance if we're going to use it.
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited September 24, 2009
    We are using the maintenance window tonight:
    http://smugmug.wordpress.com/
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited October 15, 2009
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited November 5, 2009
  • Options
    MalteMalte Registered Users Posts: 1,181 Major grins
    edited November 6, 2009
    Andy wrote:

    http://blogs.smugmug.com/release-notes/

    No candy in this one?

    *Impatiently waiting for Baldy to leak something...*

    Malte
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited November 6, 2009
    Malte wrote:
    http://blogs.smugmug.com/release-notes/

    No candy in this one?

    *Impatiently waiting for Baldy to leak something...*

    Malte
    Not yet naughty.gif
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited November 11, 2009
    we're using it Thursday, Nov 12th:
    http://smugmug.wordpress.com/
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited November 12, 2009
    Reminder: we're using our maintenance window tonight, http://smugmug.wordpress.com
  • Options
    jfriendjfriend Registered Users Posts: 8,097 Major grins
    edited November 13, 2009
    When will the maintence be done?
    I see you started maintenance at 10pm. You say it's supposed to take around 2 hrs. I check back at 12 and the site is no longer in read-only mode. The status blog has nothing new. I need to start a 1000 image upload tonight so it can run while I'm sleeping. I start the upload. After about 20 images, every image starts generating an error and now I see the #@$%^&* site is back in read-only mode and I can't upload.

    I wish you guys would be more predictable with this stuff! What's a person supposed to do when they need to do a big upload? Yes, I knew you were doing maintenance tonight, but the photos are done when they're done and tonight is when they need to go up. Why do you take the site out of read-only mode if you're not done with it yet. Why don't you give us any updates on the status blog? Your first post in the blog says: "We’ll update with details as we go along.", but there's been nothing for three hours. I'd like to be able to get my photos uploaded even though you're working on other things (you could just jam them in a big queue somewhere).

    When will you start designing a maintenance procedure that doesn't result in downtime? It is possible. It does take more effort to plan it that way. Does gmail have regular planned multi-hour outages?
    --John
    HomepagePopular
    JFriend's javascript customizationsSecrets for getting fast answers on Dgrin
    Always include a link to your site when posting a question
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited November 13, 2009
    jfriend wrote:
    the #@$%^&* site

    Hi John, first, I'm so sorry you are so very upset.

    For all, from the status blog:
    11/12/09 – Scheduled Maintenance
    Don MacAskill 4:59 pm on November 11, 2009 Permalink |

    We’ll be doing some hardware and software upgrades on Thursday night that will likely require us to go into read-only mode during our weekly scheduled maintenance window starting at 10pm Pacific. We’ll update with details as we go along.

    22:08 Pacific – We’re starting. I’d imagine this will last a few hours, but we might finish sooner.
    02:35 Pacific – We’re all done for the night. Luckily we weren’t in read-only mode for very long during this window, but we got lots done. Thanks for your patience!
    The window goes to 0300 Pacific, and we were done before then, so I'm missing where we failed? We did start the maintenance later into to the window than we expected - but our Ops team kept folks updated, as far as I can tell.

    The funny thing, to me , is that it's ironic, because you (among some others, too) were so very vocal on us not reporting our maintenance windows a long time ago. You and others have also said, "SmugMug, please post an all clear when you're done." We did that, too. We really are listening, John - I promise deal.gif

    This week, I tweeted about Maintenance on Wednesday
    http://twitter.com/SmugMug/status/5631537046

    and Thursday
    http://twitter.com/SmugMug/status/5662179068

    and posted in this thread on Dgrin twice also, that we'd be using our window last night. We don't like using the maintenance window, but sometimes, it's necessary to take the site into read only mode, or down, to improve SmugMug on a regular basis like we do.

    Again, I'm so very sorry that our maintenance last night has upset you. I'll pass on your comments to Don and Wireless and the team.
  • Options
    MalteMalte Registered Users Posts: 1,181 Major grins
    edited November 13, 2009
    Andy wrote:
    Reminder: we're using our maintenance window tonight, http://smugmug.wordpress.com

    So was this an under-the-hood type update? I went on a feature-egg hunt and came back hungry. :cry

    Malte
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited November 13, 2009
    Malte wrote:
    So was this an under-the-hood type update? I went on a feature-egg hunt and came back hungry. :cry

    Malte
    You haven't added a scandinavian language keyword? lol3.gif

    Also: you haven't tried to force Thumbnails style, and the method of pagination?

    More, too :D

    Release notes coming a bit later today.
  • Options
    MalteMalte Registered Users Posts: 1,181 Major grins
    edited November 13, 2009
    Andy wrote:
    You haven't added a scandinavian language keyword? lol3.gif

    Ouh, that's most welcome! I'm playing with that right after this! clap.gif
    Andy wrote:
    Also: you haven't tried to force Thumbnails style, and the method of pagination?

    Am I getting this right, I can only force pagination if I force the viewing style? I would like to have it like, if the viewer selects "Thumbnails", then force "Show thumbnails in a single page". I myself think Thumbnails is da bomb but grandma is probably a SmugMug dame. mwink.gif
    Andy wrote:
    More, too :D

    Release notes coming a bit later today.

    Looking forward to them!

    Malte
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited November 13, 2009
    Malte wrote:
    Am I getting this right,
    Yes, only if you force thumbnails style.
  • Options
    jfriendjfriend Registered Users Posts: 8,097 Major grins
    edited November 13, 2009
    Andy wrote:
    Hi John, first, I'm so sorry you are so very upset.

    For all, from the status blog:

    The window goes to 0300 Pacific, and we were done before then, so I'm missing where we failed? We did start the maintenance later into to the window than we expected - but our Ops team kept folks updated, as far as I can tell.

    The funny thing, to me , is that it's ironic, because you (among some others, too) were so very vocal on us not reporting our maintenance windows a long time ago. You and others have also said, "SmugMug, please post an all clear when you're done." We did that, too. We really are listening, John - I promise deal.gif

    This week, I tweeted about Maintenance on Wednesday
    http://twitter.com/SmugMug/status/5631537046

    and Thursday
    http://twitter.com/SmugMug/status/5662179068

    and posted in this thread on Dgrin twice also, that we'd be using our window last night. We don't like using the maintenance window, but sometimes, it's necessary to take the site into read only mode, or down, to improve SmugMug on a regular basis like we do.

    Again, I'm so very sorry that our maintenance last night has upset you. I'll pass on your comments to Don and Wireless and the team.

    Where in your regular communication does it say the maintenance window goes from 10-3? I'm sure you say that somewhere, but I don't see it anywhere in the regular communication (tweets, recent status blog post, etc...) and thus I didn't know that. I said I knew you had maintenance (that isn't the issue). Some things for you to think about:

    1) Five hours of downtime on a regular basis is just too much. If I was anywhere other than the U.S. (Europe, Asia, etc...), it would be even worse. I think it's time for you to consider designing a different release process that doesn't necessarily impact the entire site for most upgrades. Yes, it's more work, but you should also be growing up as a company in this regard.

    2) I could not tell how long the downtime was going to be. None of the regular communications about the downtime (the places we are told to watch - tweets, status blog, etc...) say that your window was 5 hours long. It was written right in the post that it would take a few hours and you would update the status as you went along, but that was not done except when you started and the all clear (by which time I was long asleep). I was trying to get a huge upload started before I went to sleep and I couldn't figure out when to start it.

    3) Do you really have to block new uploads when you're doing maintenance? Couldn't you design things so that uploads while in read-only mode could continue and they'd be deposited in a queue for you to process later. You should realize that every time you throw the read-only switch, you kill every single uploader that was in progress at the time. I'd suggest a design goal for the future to allow uploading processes to continue during most read-only modes.
    --John
    HomepagePopular
    JFriend's javascript customizationsSecrets for getting fast answers on Dgrin
    Always include a link to your site when posting a question
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited November 13, 2009
    jfriend wrote:
    Where in your regular communication does it say the maintenance window goes from 10-3?
    I'm sure you say that somewhere, but I don't see it anywhere in the regular communication (tweets, recent status blog post, etc...) and thus I didn't know that.
    It's in post one of this thread, and also here http://smugmug.wordpress.com/maintenance/ and in the status blog.

    Thanks for your other feedback!
  • Options
    jfriendjfriend Registered Users Posts: 8,097 Major grins
    edited November 13, 2009
    Andy wrote:
    jfriend wrote:
    Where in your regular communication does it say the maintenance window goes from 10-3?
    I'm sure you say that somewhere, but I don't see it anywhere in the regular communication (tweets, recent status blog post, etc...) and thus I didn't know that.
    It's in post one of this thread, and also here http://smugmug.wordpress.com/maintenance/ and in the status blog.

    Thanks for your other feedback!
    I'd suggest you include the time window in each post about maintenance. You know I read a lot of stuff here (including four separate references about this maintenance) and I was unaware, so it's easy to not see. To see any of your references requires you to read old stuff - something few people do.
    --John
    HomepagePopular
    JFriend's javascript customizationsSecrets for getting fast answers on Dgrin
    Always include a link to your site when posting a question
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited November 13, 2009
    jfriend wrote:
    I'd suggest you include the time window in each post about maintenance. You know I read a lot of stuff here (including four separate references about this maintenance) and I was unaware, so it's easy to not see. To see any of your references requires you to read old stuff - something few people do.
    I'll try and do that!
    This is the first complaint we've had since we started the extra communication.

    I'm really sorry this upset you so much :(
  • Options
    jfriendjfriend Registered Users Posts: 8,097 Major grins
    edited November 13, 2009
    Andy wrote:
    I'll try and do that!
    This is the first complaint we've had since we started the extra communication.

    I'm really sorry this upset you so much :(
    I rather doubt that nobody else has been disappointed that the site is regularly in read-only mode for 3-5 hrs for maintenance and no uploads or site changes are allowed.

    It's better that you communicate than not communicate, but that doesn't make the downtime OK - it just makes it better than if you didn't communicate. I've managed a data center team with three nines objectives (99.9% uptime - that's 10 mins of downtime per week) so I know what can be done. Maybe I'm the first one to start ringing this bell (I'm sure there will be others), but I think you guys need to start thinking about architecting the roll-out of your changes so you can make a lot more changes without any noticeable impact to the site. This can be done for many changes.

    I just realized last night that I couldn't even see my own unlisted galleries when the site was in read-only mode because it treats me like I'm not logged in. Since I didn't have saved URLs to them, they were completely offline to me.
    --John
    HomepagePopular
    JFriend's javascript customizationsSecrets for getting fast answers on Dgrin
    Always include a link to your site when posting a question
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited November 13, 2009
    jfriend wrote:
    I rather doubt that nobody else has been disappointed that the site is regularly in read-only mode for 3-5 hrs for maintenance and no uploads or site changes are allowed.

    That's not what I said. I said we've not had anyone complain about our communications till just now, since we really started (at your, and others' good request) to announce maintenance as soon as we know it's coming. Again, I hit this thread, twitter on 2 accounts, and this thread, all multiple times. And the status blog got updated in advance.

    Thanks for your feedback.
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited November 13, 2009
    jfriend wrote:
    the site is regularly in read-only mode for 3-5 hrs for maintenance
    This is simply not true, so please, don't state such.

    Yes, we have RO mode from time to time. 2wks ago, 3wks ago, none. We update the blog constantly.
  • Options
    jfriendjfriend Registered Users Posts: 8,097 Major grins
    edited November 13, 2009
    Andy wrote:
    This is simply not true, so please, don't state such.

    Yes, we have RO mode from time to time. 2wks ago, 3wks ago, none. We update the blog constantly.
    It seems like about every 2 weeks and from time to time when there are site issues or unplanned changes. By "regularly", I just meant it's a planned and scheduled thing that is part of your regular process.
    --John
    HomepagePopular
    JFriend's javascript customizationsSecrets for getting fast answers on Dgrin
    Always include a link to your site when posting a question
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited November 13, 2009
    jfriend wrote:
    It seems like about every 2 weeks and from time to time when there are site issues or unplanned changes. By "regularly", I just meant it's a planned and scheduled thing that is part of your regular process.
    Can we drop this for now, and enjoy the good stuff we shipped last night? Like the SmugMug ajax fix you've been hammering me and JT on for months lol3.gif C'mon John - let's enjoy Friday a little bit, eh?

    Oh we added utf8 keywords, also....
  • Options
    HoyBoyHoyBoy Registered Users Posts: 135 Major grins
    edited November 13, 2009
    Andy wrote:
    Can we drop this for now, and enjoy the good stuff we shipped last night? Like the SmugMug ajax fix you've been hammering me and JT on for months lol3.gif C'mon John - let's enjoy Friday a little bit, eh?

    Oh we added utf8 keywords, also....

    15524779-Ti.gif

    I truly don't understand all the complaining. People complain when you don't roll out new stuff fast enough and they complain when you do it. Crazy!

    Thanks for the great work you do, Andy - and the entire SmugMug Team!
  • Options
    DrDavidDrDavid Registered Users Posts: 1,292 Major grins
    edited November 13, 2009
    My 2 cents.. While downtime is always a pain, SM DOES post downtime in advance.. It's a Thursday night (which isn't a particularly historically busy time of the week for web-browsing), the disruptions are minimal (and have verbiage both on the status blog, on dgrin, on twitter and on the site itself) when it happens, and are normally accompanied with cool new stuff.

    I believe that the last time I looked, SmugMug had pretty good overall uptime. It was in the 99% range at least. Yeah, it sucks when it's down, but, they *do* need to maintain the site every so-often...

    Frankly, I think SM is doing a fine job with what they have to work with. A huge site with hundreds-of-thousands of customers, all with different looking sites, etc..

    To put it in perspective.. I use dreamhost to host a few non-smugmug sites (like http://www.smugschool.com/ ). SM and DH are about the same cost per month. One is just a Linux host that hosts my sites. The other is an amazing and awesome site for photographers, where all the maintenance is done for you, e-commerce is integrated in a completely awesome way, etc.. You get A LOT for the small pittance you pay for SM.

    John, one good idea is to post an idea of the duration of the outage (if known). A 5-hour outage, if known ahead of time, would be a nice one to say, "Hey, we'll be in read only from 10pm to about 3am".

    I make my living on SM and I'm not bent about down-time; I just take it as an opportunity to spend time with the family :)
  • Options
    jfriendjfriend Registered Users Posts: 8,097 Major grins
    edited November 13, 2009
    Andy wrote:
    Can we drop this for now, and enjoy the good stuff we shipped last night? Like the SmugMug ajax fix you've been hammering me and JT on for months lol3.gif C'mon John - let's enjoy Friday a little bit, eh?

    Oh we added utf8 keywords, also....
    We could all enjoy the new features a little more if we knew what they were.

    I do know about the Smugmug view zoom fix because of one of your posts and I responded in a thread where you mentioned it. I am glad that is finally fixed!
    --John
    HomepagePopular
    JFriend's javascript customizationsSecrets for getting fast answers on Dgrin
    Always include a link to your site when posting a question
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited November 13, 2009
    jfriend wrote:
    We could all enjoy the new features a little more if we knew what they were.
    They'll be announced when we're ready, in Release Notes. Right now, we're fixing a keywords bug, then we'll let 'em loose.
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited November 13, 2009
    jfriend wrote:
    We could all enjoy the new features a little more if we knew what they were.

    thumb.gifhttp://blogs.smugmug.com/release-notes/2009/11/13/utf-8-keywords-scandigital/
  • Options
    RichardRichard Administrators, Vanilla Admin Posts: 19,937 moderator
    edited November 14, 2009
    Andy wrote:

    I am delighted that you have added UTF-8 keyword support. clap.gifclapclap.gif
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited November 14, 2009
    Malte wrote:
    Can the new partner services access the vault? It would make sense for ShootDotEdit to be able to retrieve RAW files and for ScanDigital to be able to deliver TIFF files.

    Malte
    no, they cannot, not as of this release anyhow.

    I'm sorry.
Sign In or Register to comment.