General and important questions about site performance and optimisation

TalkieT · June 9, 2010

Hi guys (Baldy, Onethumb - and anyone else concerned about performance - I'm looking at you guys for help and thoughts here)

I have a fairly heavily customised site and things are starting to creak in the page load time department, especially in category pages with lots of galleries and subcategories... Take this URL for example...

http://www.nzsnaps.com/Ultimate

Now, on a forced refresh reload (Ctrl-F5) it's a largish page - 900+ KB, 230 odd objects and can take (on my connection - ADSL, Firefox - no pipelining hacks) up to 45 seconds to fully load...

I verified this with Firebug a bunch of times and there's no individual element really holding it up, it's all loading as you would expect.

Now I KNOW I can trim my CSS a little, and I KNOW I could be a little cleverer with size of customisation GFX... Bu there's something which really jumps out at me and hurts.

Almost all of the elements on the page come from "http://www.nzsnaps.com/..." and as I'm sure you all know, browsers should only load a couple of elements concurrently from a single host.

This means that a huge number of these small elements are queued up and downloaded virtually sequentially. This kills load time.

Google maps and other sites get around this by faking may hosts (mt0.google.com, mt1.google.com, mt[x].google.com) and distributing the calls around all these hosts (which may in fact point at the same server). This helps hugely and the browser dutifully loads up a couple of connections at once to EACH of the 'different' hosts.

I could (and I plan to) kinda fake this by not using relative links the whole time and sometimes point to www.nzsnaps.com, smugmug.com, talkiet.smugmug.com etc - but I was wondering if anyone had a better idea - or am I on the right track?

Yes, I know I could hack the browser to use many more threads at once, and I know caching helps a lot, but I am concerned about first time visitors having a slow experience.

Please don't tell me to pull back on the customisation - I know from tests that when I massively parallelise using http pipelining, the page load times dive and I'm happy - so it's not the actual size of the page, it's how it's loaded.

So I'm asking for ideas on how to optimise pages for speedy loading outside the normal "trim your CSS, use smaller images" etc... How can we do this?

And a specific question, what static hosts resolve and I can consider stable for our files? I presume
http://www.smugmug.com
http://smugmug.com
http://[username].smugmug.com

but are there any others?

Something to think about and I am sure that there are smarter ways to do this - I am just trying to get some fresh ideas and eyes on the problem.

Cheers - Neil Gardner

TalkieT · June 9, 2010

Note that I just went through and changed a bunch of my relative links from

"/photos/whatever" to "http://talkiet.smugmug.com/photos/whatever"

and my page load times dropped from about 35-40 seconds to 9-10 seconds!

I don't know if any of this is due to the local time diff from 11:30 when I did the baseline tests, and now 12:40 (am) when I did the new tests - but I will find out tomorrow night I guess...

In any case - seems like there is some significant performance benefit to be realised here... I might dig in and see if I can determine if it's due to the multiple hosts, or a slow issue with my DNS resolution on my domain.

Thoughts still welcome.

Cheers - N

jfriend · June 9, 2010

TalkieT wrote: »

Note that I just went through and changed a bunch of my relative links from

"/photos/whatever" to "http://talkiet.smugmug.com/photos/whatever"

and my page load times dropped from about 35-40 seconds to 9-10 seconds!

I don't know if any of this is due to the local time diff from 11:30 when I did the baseline tests, and now 12:40 (am) when I did the new tests - but I will find out tomorrow night I guess...

In any case - seems like there is some significant performance benefit to be realised here... I might dig in and see if I can determine if it's due to the multiple hosts, or a slow issue with my DNS resolution on my domain.

Thoughts still welcome.

Cheers - N

For reasons I never understood, I've seen Smugmug pages do the following. If you use a relative link like "/photos/whatever" to specify an image, then the browser attaches the domain of the CDN to it and requests the image directly from the CDN. When that request is received, it redirects that with a 302 response to the actual image URL with your real domain in it.

This is bad for performance for two reasons. First, you get double the number of web requests (because of the redirect) and second, the original request can never be cached in the local browser. It absolutely kills site performance. I don't know why it's this way. I posted about this a year ago or so and I have a mental note to never use the domain-less image URLs in my site, but I never got any reaction/response from Smugmug last time I posted about it.

onethumb · June 9, 2010

TalkieT wrote: »

Hi guys (Baldy, Onethumb - and anyone else concerned about performance - I'm looking at you guys for help and thoughts here)

I have a fairly heavily customised site and things are starting to creak in the page load time department, especially in category pages with lots of galleries and subcategories... Take this URL for example...

http://www.nzsnaps.com/Ultimate

Now, on a forced refresh reload (Ctrl-F5) it's a largish page - 900+ KB, 230 odd objects and can take (on my connection - ADSL, Firefox - no pipelining hacks) up to 45 seconds to fully load...

I verified this with Firebug a bunch of times and there's no individual element really holding it up, it's all loading as you would expect.

Now I KNOW I can trim my CSS a little, and I KNOW I could be a little cleverer with size of customisation GFX... Bu there's something which really jumps out at me and hurts.

Almost all of the elements on the page come from "http://www.nzsnaps.com/..." and as I'm sure you all know, browsers should only load a couple of elements concurrently from a single host.

This means that a huge number of these small elements are queued up and downloaded virtually sequentially. This kills load time.

Google maps and other sites get around this by faking may hosts (mt0.google.com, mt1.google.com, mt[x].google.com) and distributing the calls around all these hosts (which may in fact point at the same server). This helps hugely and the browser dutifully loads up a couple of connections at once to EACH of the 'different' hosts.

I could (and I plan to) kinda fake this by not using relative links the whole time and sometimes point to www.nzsnaps.com, smugmug.com, talkiet.smugmug.com etc - but I was wondering if anyone had a better idea - or am I on the right track?

Yes, I know I could hack the browser to use many more threads at once, and I know caching helps a lot, but I am concerned about first time visitors having a slow experience.

Please don't tell me to pull back on the customisation - I know from tests that when I massively parallelise using http pipelining, the page load times dive and I'm happy - so it's not the actual size of the page, it's how it's loaded.

So I'm asking for ideas on how to optimise pages for speedy loading outside the normal "trim your CSS, use smaller images" etc... How can we do this?

And a specific question, what static hosts resolve and I can consider stable for our files? I presume
http://www.smugmug.com
http://smugmug.com
http://[username].smugmug.com

but are there any others?

Something to think about and I am sure that there are smarter ways to do this - I am just trying to get some fresh ideas and eyes on the problem.

Cheers - Neil Gardner

Honestly, the thing you could do that would improve load time the most on that page is to feature a photo in each of your galleries. Having to randomly choose a photo for each on is both time consuming and impossible to cache.

We recently enhanced our homepages to do a better job at that particular job, but haven't yet addressed Category pages like your example. It's on our TODO list, but even then, it won't be nearly as fast as simply featuring photos.

Hope that helps!

onethumb · June 9, 2010

jfriend wrote: »

For reasons I never understood, I've seen Smugmug pages do the following. If you use a relative link like "/photos/whatever" to specify an image, then the browser attaches the domain of the CDN to it and requests the image directly from the CDN. When that request is received, it redirects that with a 302 response to the actual image URL with your real domain in it.

This is bad for performance for two reasons. First, you get double the number of web requests (because of the redirect) and second, the original request can never be cached in the local browser. It absolutely kills site performance. I don't know why it's this way. I posted about this a year ago or so and I have a mental note to never use the domain-less image URLs in my site, but I never got any reaction/response from Smugmug last time I posted about it.

Do you have some examples? Relative links will use whichever host you're currently on, which is rarely the CDN unless maybe you're building the links with JS or CSS, which of course, are served from the CDN.

The CDN shouldn't typically redirect with a 302 on a photo request, but there could be some corner cases where it does. The 302 redirect is intended to thwart people masquerading as someone they're not - we've had customers "attempt" to take credit for someone else's photo by hard-coding their DNS in front of the URL and then pointing at the fact that it works as an example of their ownership. Clearly, that no longer works (and hasn't for years), but that code might not be properly dealing with our CDN in all cases.

Don

jfriend · June 9, 2010

onethumb wrote: »

Do you have some examples? Relative links will use whichever host you're currently on, which is rarely the CDN unless maybe you're building the links with JS or CSS, which of course, are served from the CDN.

The CDN shouldn't typically redirect with a 302 on a photo request, but there could be some corner cases where it does. The 302 redirect is intended to thwart people masquerading as someone they're not - we've had customers "attempt" to take credit for someone else's photo by hard-coding their DNS in front of the URL and then pointing at the fact that it works as an example of their ownership. Clearly, that no longer works (and hasn't for years), but that code might not be properly dealing with our CDN in all cases.

Don

I temporarily changed my site to illustrate one place where this happens. If you go here: http://jfriend.smugmug.com/popular/#66770280_dRonw and look at the banner graphic. Firefox requests /photos/631724792_9goWJ-O-1.png from cdn.smugmug.com. It then receives a 301 to http://jfriend.smugmug.com/photos/631724792_9goWJ-O-1.png. It gets the 301 everytime the page is loaded. Because it's a 301 (instead of a 302), the browser does seem to cache the second fetch. I have also seen cases where it's a 302 and then there is no caching for the image either, though I don't know where that case is now.

The CSS for my banner is:

#myBanner {
    background: transparent url(/photos/631724792_9goWJ-O-1.png) no-repeat;
    height:31px;
    width: 370px;
    _background: none;
    _filter:progid:DXImageTransform.Microsoft.AlphaImageLoader (enabled=true, sizingmethod=image, src="http://jfriend.smugmug.com/photos/631724792_9goWJ-O-1.png");
}

If I change it to this, the extra redirect does not happen:

#myBanner {
    background: transparent url([COLOR=red]http://jfriend.smugmug.com[/COLOR]/photos/631724792_9goWJ-O-1.png) no-repeat;
    height:31px;
    width: 370px;
    _background: none;
    _filter:progid:DXImageTransform.Microsoft.AlphaImageLoader (enabled=true, sizingmethod=image, src="http://jfriend.smugmug.com/photos/631724792_9goWJ-O-1.png");
}

I have no idea why the browser tries the initial fetch from cdn.smugmug.com. I don't have any script that has anything to do with my banner image. The banner image is just plain HTML and CSS.

FYI, I agree with your recommendation to not use random thumbnails for galleries if you want good page performance. Not only is that done via a 302, but it's non-cachable because it's different every time and the request that generates the 302 isn't exactly speedy either.

jfriend · June 9, 2010

onethumb wrote: »

Do you have some examples? Relative links will use whichever host you're currently on, which is rarely the CDN unless maybe you're building the links with JS or CSS, which of course, are served from the CDN.

The CDN shouldn't typically redirect with a 302 on a photo request, but there could be some corner cases where it does. The 302 redirect is intended to thwart people masquerading as someone they're not - we've had customers "attempt" to take credit for someone else's photo by hard-coding their DNS in front of the URL and then pointing at the fact that it works as an example of their ownership. Clearly, that no longer works (and hasn't for years), but that code might not be properly dealing with our CDN in all cases.

Don

Here's a different case of a 302. Go to this site which is one of the the customer sites you list on www.smugmug.com. Their banner image is specified as http://site.nvisiblephoto.com/photos/201292442-M.jpg, which gets a 302 redirect everytime to http://site.nvisiblephoto.com/photos/201292442-O-1.jpg. You can see their banner flash when you go from page to page because it does the initial server request to get teh 302 every time you load a page on the site.

jfriend · June 9, 2010

onethumb wrote: »

Do you have some examples? Relative links will use whichever host you're currently on, which is rarely the CDN unless maybe you're building the links with JS or CSS, which of course, are served from the CDN.

The CDN shouldn't typically redirect with a 302 on a photo request, but there could be some corner cases where it does. The 302 redirect is intended to thwart people masquerading as someone they're not - we've had customers "attempt" to take credit for someone else's photo by hard-coding their DNS in front of the URL and then pointing at the fact that it works as an example of their ownership. Clearly, that no longer works (and hasn't for years), but that code might not be properly dealing with our CDN in all cases.

Don

Same 301 issue happening on the banner image on this site which is another one of your sample sites listed on www.smugmug.com.

jfriend · June 9, 2010

Here's a double 302 for the same image. On this site (another example site from www.smugmug.com), there's an initial request for this image:

http://api.smugmug.com/photos/377930419_dgxvY-X3.jpg

That gets a 302 redirect to:

http://photos.smugmug.com/photos/377930419_dgxvY-X3.jpg

When the browser goes to that URL, it gets a 302 redirect to:

/Laforet-Videos/Reverie/Preview-Image2/377930419_dgxvY-O-1.jpg

When I look at the actual image, I think it has something to do with the video in that gallery.

onethumb · June 9, 2010

jfriend wrote: »
I temporarily changed my site to illustrate one place where this happens. If you go here: http://jfriend.smugmug.com/popular/#66770280_dRonw and look at the banner graphic. Firefox requests /photos/631724792_9goWJ-O-1.png from cdn.smugmug.com. It then receives a 301 to http://jfriend.smugmug.com/photos/631724792_9goWJ-O-1.png. It gets the 301 everytime the page is loaded. Because it's a 301 (instead of a 302), the browser does seem to cache the second fetch. I have also seen cases where it's a 302 and then there is no caching for the image either, though I don't know where that case is now.

The CSS for my banner is:
#myBanner {
    background: transparent url(/photos/631724792_9goWJ-O-1.png) no-repeat;
    height:31px;
    width: 370px;
    _background: none;
    _filter:progid:DXImageTransform.Microsoft.AlphaImageLoader (enabled=true, sizingmethod=image, src="http://jfriend.smugmug.com/photos/631724792_9goWJ-O-1.png");
}
If I change it to this, the extra redirect does not happen:
#myBanner {
    background: transparent url([COLOR=red]http://jfriend.smugmug.com[/COLOR]/photos/631724792_9goWJ-O-1.png) no-repeat;
    height:31px;
    width: 370px;
    _background: none;
    _filter:progid:DXImageTransform.Microsoft.AlphaImageLoader (enabled=true, sizingmethod=image, src="http://jfriend.smugmug.com/photos/631724792_9goWJ-O-1.png");
}
I have no idea why the browser tries the initial fetch from cdn.smugmug.com. I don't have any script that has anything to do with my banner image. The banner image is just plain HTML and CSS.

FYI, I agree with your recommendation to not use random thumbnails for galleries if you want good page performance. Not only is that done via a 302, but it's non-cachable because it's different every time and the request that generates the 302 isn't exactly speedy either.

Nice catch. Fixed?

onethumb · June 9, 2010

jfriend wrote: »

Here's a different case of a 302. Go to this site which is one of the the customer sites you list on www.smugmug.com. Their banner image is specified as http://site.nvisiblephoto.com/photos/201292442-M.jpg, which gets a 302 redirect everytime to http://site.nvisiblephoto.com/photos/201292442-O-1.jpg. You can see their banner flash when you go from page to page because it does the initial server request to get teh 302 every time you load a page on the site.

This is something we can't help (though we should probably be 301 instead of 302). The customer has chosen, for whatever reason, to link to an image that doesn't exist (there's no Medium for that size, since it's smaller than Medium). We do our best to not show a broken image, but we're not going to clutter up the cache with caching bogus data.

There are bugs and then there's user error. This one is user error.

onethumb · June 9, 2010

onethumb wrote: »

Nice catch. Fixed?

(It may take awhile for the fix to fully propogate - 30K servers and all. Keep hitting refresh if your local servers haven't got it yet...)

jfriend · June 9, 2010

onethumb wrote: »

Nice catch. Fixed?

Yes, this one seems fixed on my site now. Thanks.

jfriend · June 9, 2010

onethumb wrote: »

This is something we can't help (though we should probably be 301 instead of 302). The customer has chosen, for whatever reason, to link to an image that doesn't exist (there's no Medium for that size, since it's smaller than Medium). We do our best to not show a broken image, but we're not going to clutter up the cache with caching bogus data.

There are bugs and then there's user error. This one is user error.

OK, I guess I sort of understand this one. It's unfortunate that customers will goof it and then have a slower site though.

onethumb · June 9, 2010

jfriend wrote: »

Here's a double 302 for the same image. On this site (another example site from www.smugmug.com), there's an initial request for this image:

http://api.smugmug.com/photos/377930419_dgxvY-X3.jpg

That gets a 302 redirect to:

http://photos.smugmug.com/photos/377930419_dgxvY-X3.jpg

When the browser goes to that URL, it gets a 302 redirect to:

/Laforet-Videos/Reverie/Preview-Image2/377930419_dgxvY-O-1.jpg

When I look at the actual image, I think it has something to do with the video in that gallery.

Got rid of one of the 302s, the other one is a little murkier as to how to properly handle it. A fix may or may not be forthcoming depending on what I find out. We should probably just be telling the browser to use the -O in the first place in this particular instance...

onethumb · June 9, 2010

jfriend wrote: »

OK, I guess I sort of understand this one. It's unfortunate that customers will goof it and then have a slower site though.

I think we'd like to understand how a customer could goof it, and then fix that. None of our tools are providing -M links if they don't exist, I don't think, and if they are, that'd be a bug.

jfriend · June 9, 2010

onethumb wrote: »

I think we'd like to understand how a customer could goof it, and then fix that. None of our tools are providing -M links if they don't exist, I don't think, and if they are, that'd be a bug.

I don't know how the goof happens in the first place, but I see it regularly. Banner images and such should just link to originals, but I think customers are regularly confused about how to get the right links for images used in customizations. Banners (if not using EZC), which are typically set up when the customer is a real smugmug newbie are often something people make mistakes with.

Hmmm. I just checked and the Share button/Get a Link does offer links to sizes larger than the original image which then gets a 302 redirect to the original. So this problem can happen right from the Get a Link screen. Probably worth fixing there.

Andy · June 9, 2010

jfriend wrote: »

I don't know how the goof happens in the first place, but I see it regularly. Banner images and such should just link to originals,

For some, yes. But we many folks make a banner 1200px wide (or whatever) and then only want the image to be 800px wide so that grab a -L.

jfriend · June 9, 2010

Andy wrote: »

For some, yes. But we many folks make a banner 1200px wide (or whatever) and then only want the image to be 800px wide so that grab a -L.

I realize they can do it that way, but I wouldn't teach someone to do it that way. Better to take full control in Photoshop and make it the actual size and not put it through an unnecessary Smugmug re-size operation.

Andy · June 9, 2010

jfriend wrote: »

I realize they can do it that way, but I wouldn't teach someone to do it that way. Better to take full control in Photoshop and make it the actual size and not put it through an unnecessary Smugmug re-size operation.

Yeah, we don't teach 'em that way but with hundreds of thousands of people doing it, we get all sorts of banners

TalkieT · June 10, 2010

Wow, I seemed to have touched a nerve here :-)

Thanks for the suggestions guys - and for those of you following along at home, I managed to improve the load time of this page on my site:

http://www.nzsnaps.com/Ultimate

From about 35-40 seconds for a cold load (ctrl-F5 refresh) down to about TEN (10) SECONDS!

What did I do?

For a start, I replaced all my relative links to customisation elements with absolute links to talkiet.smugmug.com and photos.smugmug.com. I manually balanced these out in my customisation and I also pulled a clever trick where I used JS to change the BG image for the gallery thumbnails and background.

The stacks of photos on the page above (like a pile of polaroids around the thumbnails) are random images from a gallery and I liked them too much to get rid of them, so I modified the js to create the URL for the odd numbered links to photos.smugmug.com/restofURL and the even numbered links to talkiet.smugmug.com/restofURL.

The above modifications got the load time halved to about 15-20 seconds. Impressive enough.

I then tried setting featured images in each gallery as suggested in this thread and that got me the rest of the way. Wow...

I would still like to know if there are any other domains that won't kick a 302 when I use them... So far I have

talkiet.smugmug.com
photos.smugmug.com

If I could find one or 2 more that don't require redirects then I can speed things up even more.

Guys - got any other non redirecting FQDNs that we could use? Perhaps can you create a couple :-)

Thanks for the analysis and advice in this thread so far - absolutely invaluable and you have no idea how much more positive I am about the whole SM experience after more than doubling the apparent speed of my site.

Cheers - Neil G

bwg · June 11, 2010

how about now Neil? any better?

TalkieT · June 11, 2010

bigwebguy wrote: »

how about now Neil? any better?

I'm not sure what you mean - I have made heaps of changes to my customisation to remove all relative links, and I have manually balanced my links between talkiet.smugmug.com and photos.smugmug.com... I have also set feature photos for all the galleries in my Ultimate galleries...

All of those things made a massive improvement.

But have you also made some change at your end? Perhaps improved the random feature or something?

Cheers
Neil G

bwg · June 11, 2010

TalkieT wrote: »

I'm not sure what you mean - I have made heaps of changes to my customisation to remove all relative links, and I have manually balanced my links between talkiet.smugmug.com and photos.smugmug.com... I have also set feature photos for all the galleries in my Ultimate galleries...

All of those things made a massive improvement.

But have you also made some change at your end? Perhaps improved the random feature or something?

Cheers
Neil G

we made a change tonight and your site is measuring faster now for me than it was this morning. just seeing if you noticed the same thing.

TalkieT · June 11, 2010

bigwebguy wrote: »

we made a change tonight and your site is measuring faster now for me than it was this morning. just seeing if you noticed the same thing.

Actually, just stopped all other stuff and did a few ctrl-F5 refreshes and yes, it is noticably quicker...

I'm down to 5-6 seconds for a complete load of http://www.nzsnaps.com/Ultimate

It's hard to believe that just a couple of days ago, essentially the same page was taking 35+ seconds to load. I wonder how many other people have really badly performing sites because they have large pages and have used relative links, or are being hampered by the defualt number of simultaneous connections to a single FQDN.

Thanks for whatever extra improvement you made! Can you give some indication what it was?

Cheers - Neil Gardner

General and important questions about site performance and optimisation

Comments