Site performance last two days = suck

onethumb · February 15, 2005

I have to apologize to everyone about the site performance the last few days.

I've been tearing my hair out and losing sleep over what could possibly be wrong. I just found it a few minutes ago.

GoogleBot, YahooSeeker, and AskJeeves/Teoma have all been hammering our site simultaneously. I'm surprised I don't see MSN in the feeding frenzy too.

Turns out, they all got really excited (well, as excited as software can get

about our keywords and keywords combinations. And all three simultaneously decided to try to create the best index of all of them. There are nearly 5,000,000 keyworded images, and many of them have multiple keywords. When multiplied out, there are many many millions of potential keywords pages, and the search engines are chewing them up as fast as they can.

I've currently set it to tell them not to spider these pages anymore, so the site should get snappier as they figure out that I don't want them in the keywords section right now.

I'm very sorry the site bogged down for everyone, I didn't anticipate quite this much load from the search engines. We've got additional hardware on the way (we ordered it awhile ago, before the search engines got wise to our keywords, in the normal course of upgrading the site) which should solve this particular problem as well as accomodate our normal growth rate.

Thanks for listening,

Don

Nikolai · February 15, 2005

Thanks, Don!
I kinda noticed the drop but affiliated it with a "weekend" factor.

Good to know it's only a temporary glitch.

Good luck with the h/w upgrade!

Cheers!

mercphoto · February 15, 2005

Speaking of keywords

onethumb wrote:

Turns out, they all got really excited (well, as excited as software can get about our keywords and keywords combinations. And all three simultaneously decided to try to create the best index of all of them.

Speaking of, I notice that there are times when I do a keyword search on my site that I will get hits from other users as well. For example, I have one image keyworded for "panorama", and when I hit that I get a lot of hits (including my image, but also including others).

I would really like a way for the user to do a search on my page and, by default, only get hits in my user space.

onethumb · February 15, 2005

mercphoto wrote:

Speaking of, I notice that there are times when I do a keyword search on my site that I will get hits from other users as well. For example, I have one image keyworded for "panorama", and when I hit that I get a lot of hits (including my image, but also including others).

I would really like a way for the user to do a search on my page and, by default, only get hits in my user space.

I assume you're talking about 'panoramic' since I don't see any 'panorama' on your site?

I'm not seeing your behavior. If I go to your page and
search for 'panoramic' I only see one result.

Clicking on that result takes me to your 'panoramic' keyword page, which also only shows a single photo.

Where are you seeing other people's photos mixed up with yours?

Don

mercphoto · February 15, 2005

onethumb wrote:

I assume you're talking about 'panoramic' since I don't see any 'panorama' on your site?

Correct. I was logged in earlier today and got this behavior. If I'm not logged in, it works correctly. If I am logged in, it doesn't. I just double-verified that right now.

JohnR · February 15, 2005

I noticed the slowdown myself.

Thanks for the update...man, Smugmug truly kicks ass

philspace · February 15, 2005

Thanks, I was getting confused, even smugmugexplorer was timing out on file xfers. It seems to be better now when I go in to snag an image.

Site performance last two days = suck

Comments