Smugmug SLOOOOW (International Network Connectivity)

I Simonius · April 13, 2006

I was trying to show a friend my galleries last night who is about 300 miles from me and while on the phone we went through the pics.

We were both very frustrated at how long pictures took to load, they were very slow, and I notice that today access was very slow as well

We are both on 1MB Broadband

Andy · April 13, 2006

Simon King wrote:

I was trying to show a friend my galleries last night who is about 300 miles from me and while on the phone we went through the pics.

We were both very frustrated at how long pictures took to load, they were very slow, and I notice that today access was very slow as well

We are both on 1MB Broadband

Who's your internet provider please? I'll ask Wireless (Our Director of Ops) to check on our connection to you. We've not received any reports of slowness of late.

JimM · April 13, 2006

I am noticing Smugmug unusually slow for the last hour or so now.

I Simonius · April 13, 2006

Andy wrote:

Who's your internet provider please? I'll ask Wireless (Our Director of Ops) to check on our connection to you. We've not received any reports of slowness of late.

I'm on NTL (ntlworld,) but my friend was on BT
(we're in UK)

Wireless · April 13, 2006

Simon King wrote:

I'm on NTL (ntlworld,) but my friend was on BT
(we're in UK)

Hi Simon, sorry to hear that you are experiencing slow downloads from our site. Unfortunately there are times when stuff gets a bit latent when crossing continents. I did some traceroutes, and things look OK.

It certainly is possible that we don't have a good route back to you (this happened last month with someone from the Isle of Man, which was tracked down to a bug in the firmware on a switch of a large bandwidth provider). Can you post, or IM me, a traceroute from your machine to www.smugmug.com please? No guarantees, but I can take a look.

wslam · April 14, 2006

Wireless wrote:

Hi Simon, sorry to hear that you are experiencing slow downloads from our site. Unfortunately there are times when stuff gets a bit latent when crossing continents. I did some traceroutes, and things look OK.

It certainly is possible that we don't have a good route back to you (this happened last month with someone from the Isle of Man, which was tracked down to a bug in the firmware on a switch of a large bandwidth provider). Can you post, or IM me, a traceroute from your machine to www.smugmug.com please? No guarantees, but I can take a look.

If you recall, I once ask a similar question, even though I am in HK.
I perfectly understand that overseas connections often suffer from higher latency, but it is a FACT that smugmug is slower then most US sites. I often need to wait 10 to 15 SECONDS before the site responds.

Wireless · April 14, 2006

wslam wrote:

If you recall, I once ask a similar question, even though I am in HK.
I perfectly understand that overseas connections often suffer from higher latency, but it is a FACT that smugmug is slower then most US sites. I often need to wait 10 to 15 SECONDS before the site responds.

I need help seeing the facts here so I can talk with our providers and/or strategize on who we buy bandwidth from. Any information you can give me, routes, ping times, time of day, overall performance numbers, which 'most US sites', etc, will help immensely.

Right now, short of flying to Hong Kong this weekend (don't tempt me, I would do that), I'm limited to using sites like traceroute.org. Stuff looks really good (and heck, better than one of our competitors coming from Hong Kong right now), but that doesn't really help me solve any slowness you might be experiencing on an ongoing basis.

Last time we left it, from what I remember, the numbers coming across were within SLA for transpacific providers, yet not spectacular.

I Simonius · April 14, 2006

Wireless wrote:

Hi Simon, sorry to hear that you are experiencing slow downloads from our site. Unfortunately there are times when stuff gets a bit latent when crossing continents. I did some traceroutes, and things look OK.

It certainly is possible that we don't have a good route back to you (this happened last month with someone from the Isle of Man, which was tracked down to a bug in the firmware on a switch of a large bandwidth provider). Can you post, or IM me, a traceroute from your machine to www.smugmug.com please? No guarantees, but I can take a look.

Erm.. what's a traceroute and how do I put it on my machine??

Sheaf · April 14, 2006

Simon King wrote:

Erm.. what's a traceroute and how do I put it on my machine??

If you're running Windows, click on "Start", then "Run..."

Type in "cmd" and click "OK"

Then at the prompt, type "tracert www.smugmug.com > C:\traceroute.txt"

Give it a minute or so, then go to that new text file on your harddrive (traceroute.txt) and copy/past the text in an email or private message to Wireless or myself.

The text file should look something like this (but without those *'s):

Tracing route to hera.smugmug.com [63.81.134.23]

over a maximum of 30 hops:

1 1 ms 1 ms 1 ms ***********

2 10 ms 11 ms 12 ms ***************************

3 11 ms 11 ms 11 ms *************************

4 11 ms 11 ms 11 ms ****************************

5 12 ms 11 ms 11 ms **************************

6 12 ms 12 ms 11 ms gige-g7-0-7.gsr12416.pao.he.net [65.19.141.237]

7 27 ms 12 ms 13 ms pos0-2.gsr12416.sjc2.he.net [216.218.230.222]

8 14 ms 14 ms 13 ms sec.r1-ge0-2-eq-sj.smugmug.com [206.223.117.201]

9 15 ms 14 ms 14 ms hera.smugmug.com [63.81.134.23]

Trace complete.

I Simonius · April 14, 2006

Sheaf wrote:

If you're running Windows, click on "Start", then "Run..."

Type in "cmd" and click "OK"

Then at the prompt, type "tracert www.smugmug.com > C:\traceroute.txt"

Give it a minute or so, then go to that new text file on your harddrive (traceroute.txt) and copy/past the text in an email or private message to Wireless or myself.

Sorry.....on a Mac

Wireless · April 14, 2006

Simon King wrote:

Erm.. what's a traceroute and how do I put it on my machine??

It's already on your machine

Windows:

Start > Run > type "cmd" without quotes to bring up the command prompt, then type "tracert www.smugmug.com" without quotes and send me the output (to copy the text in the command prompt, right click and select Mark, then hit enter once you have highlighted all of the output)

Mac:

Finder > Applications > Utilities > Terminal, then type "traceroute www.smugmug.com" without quotes.

This might contain information that could be used to give a rough location of where you live or work (depending on how motivated and resourceful someone is), so if you feel more comfortable not posting it publically, go ahead and send it via PM or email (or edit it like sheaf did).

Sheaf · April 14, 2006

Simon King wrote:

Sorry.....on a Mac

I fail in the Mac department, but try this:

http://www.verio.com/support/documents/view_article.cfm?doc_id=3693

I Simonius · April 14, 2006

Wireless wrote:

It's already on your machine
Mac:

Finder > Applications > Utilities > Terminal, then type "traceroute www.smugmug.com" without quotes.

This might contain information that could be used to give a rough location of where you live or work (depending on how motivated and resourceful someone is), so if you feel more comfortable not posting it publically, go ahead and send it via PM or email (or edit it like sheaf did).

I don't care who knows where I am:-) Anyone only has to ask and I'll tell them..:-)

here ye go:(I'll be fascinated to know what it means:): )

Last login: Fri Apr 14 09:52:28 on ttyp1
Welcome to Darwin!
simons-power-mac-g5:~ simon$ traceroute www.smugmug.com
traceroute to hera.smugmug.com (63.81.134.23), 64 hops max, 40 byte packets
1 192.168.1.1 (192.168.1.1) 0.961 ms 0.475 ms 0.369 ms
2 10.112.112.1 (10.112.112.1) 5.917 ms * *
3 * * *
4 * * *
5 * * *
6 * * *
7 * * *
8 * * *
9 * * *
10 * * *
11 * * *
12 * * *
13 * * *
14 * * *
15 * * *
16 * * *
17 * * *
18 * * *
19 * * *
20 * * *
21 * * *
22 * * *
23 * * *
24 * * *
25 * * *
26 * * *
27 * * *
28 * * *
29 * * *
30 * * *
31 * * *
32 * * *
33 * * *
34 * * *
35 * * *
36 * * *
37 * * *
38 * * *
39 * * *
40 * * *
41 * * *
42 * * *
43 * * *
44 * * *
45 * * *
46 * * *
47 * * *
48 * * *
49 * * *
50 * * *
51 * * *
52 * * *
53 * * *
54 * * *
55 * * *
56 * * *
57 * * *
58 * * *
59 * * *
60 * * *
61 * * *
62 * * *
63 * * *
64 * * *
simons-power-mac-g5:~ simon$

armani · April 14, 2006

SLOW here too
Hello,

I also have noticed Smugmug has been slow (VERY slow) a few times (3/4) in the last two weeks. It takes seconds to respond. It takes 10 to 15 sec before a picture shows up.

Armani

Wireless · April 14, 2006

Simon -

Not much help there, the responses are getting blocked.

I'm losing feedback somewhere in london if I read limelight's hostnames right (traceroute to ntlworld).

Wireless · April 14, 2006

armani wrote:

Hello,

I also have noticed Smugmug has been slow (VERY slow) a few times (3/4) in the last two weeks. It takes seconds to respond. It takes 10 to 15 sec before a picture shows up.

Armani

Where are you coming from?

Ah looks like Belgium.

If you have any information on frequency, time of day, etc., that would help out a lot. Otherwise, a report here and there of slowness without any pattern is pretty much "net turbulence". Since we are located in the San Francisco Bay Area, international customers are more likely to be impacted by latency that is beyond our control (again, this shouldn't be sustained, though).

(and now it's time for bed)

I Simonius · April 14, 2006

Wireless wrote:

Simon -

Not much help there, the responses are getting blocked.

I'm losing feedback somewhere in london if I read limelight's hostnames right (traceroute to ntlworld).

My ISP is certainly NTL

I will get my friend to do a traceroute later as well because she got BT as her ISP and she was getting about the same load times as me while we were talking it though on the phone

Do the results I got show up as bad then? those little stars...do they mean the message isn't getting through?

armani · April 14, 2006

Slow
Wireless,

I am from Belgium indeed.
Only happened 3 to 4 times in the last two weeks, but then it was really slow. And, all other sites i visited at the same moment were normal.

Anyway if it happens again i wil post it here again with the info you need.

Armani

luke_church · April 14, 2006

Wireless,

I'm in the UK as well, I get very odd behavior with Smugmug. The actual visual latency is fair low, but the through-put is really bad.

If I do usual, single HTTP channel uploads I can barely get 0.5% (500kbps) of the connection bandwidth. If I do some evil things to the inside of my HTTP runtime, adjust cache windows blah, and open 25 odd parallel HTTP connections, I can get nearer 50% (50Mbps).

Downloads aren't exactly fast either, much slower than other files I move from the US.

Is this behaviour intentional? Is something strange going on here? What kind of throughput do your contential US customers get?

Tracert:

Tracing route to smugmug.com [63.81.134.23]
over a maximum of 30 hops:
1 <1 ms <1 ms <1 ms REMOVED
2 <1 ms <1 ms <1 ms route-down-3.cam.ac.uk [193.60.95.14]
3 <1 ms <1 ms <1 ms route-cent-3.cam.ac.uk [131.111.2.126]
4 <1 ms <1 ms <1 ms route-enet-3.cam.ac.uk [192.153.213.193]
5 <1 ms <1 ms <1 ms cambridge-bar.ja.net [146.97.40.49]
6 3 ms 3 ms 3 ms po11-0.lond-scr.ja.net [146.97.35.9]
7 4 ms 3 ms 4 ms po0-0.lond-scr4.ja.net [146.97.33.34]
8 4 ms 4 ms 4 ms po1-0.lond-gw-ixp1.ja.net [146.97.35.130]
9 4 ms 4 ms 4 ms ge1-0.mr02.ldn01.pccwbtn.net [217.79.160.127]
10 * * * Request timed out.
11 156 ms 156 ms 156 ms hera.smugmug.com [63.81.134.23]
Trace complete.

Looks like its something in the transatlantic link from London, but I'm not very experienced at reading tracerts.

Any pointers would be appreciated. If there's anything else I can do to help, give me a yell.

Cheers,

Luke

I Simonius · April 24, 2006

Wireless wrote:

Simon -

Not much help there, the responses are getting blocked.

I'm losing feedback somewhere in london if I read limelight's hostnames right (traceroute to ntlworld).

What does that mean - responses getting blocked? What are the implications?

SM is STILL very slow here in Norfolk UK .

Is that it guys - have you given up?

I don't know what the results of those tests I ran mean but Im guessing its bad

I have to say that I can't really see me using SM long term if it's always going to be this slow:uhoh

Wireless · April 24, 2006

Simon King wrote:

What does that mean - responses getting blocked? What are the implications?

SM is STILL very slow here in Norfolk UK .

Is that it guys - have you given up?

I don't know what the results of those tests I ran mean but Im guessing its bad

I have to say that I can't really see me using SM long term if it's always going to be this slow:uhoh

Last week, the transit provider (Limelight/LLNW) that connects one of our routes with BT Internet reported that they are having problems in the London area. I can't tell if this impacts you directly, but in my status update request for today I have added your route to see if it is related. I apologize for not posting this information here, please feel free to ping on this thread as much as you'd like to get status (I do watch the forum daily, as well as my email constantly).

Your output might be bad. Some of the packets that traceroute uses to calculate time are being blocked (intentionally or not) at a certain segment in your route within the UK. This shouldn't be taking this long to fix, but the last time we had issues with UK based customers, it was due to a bug in the switch software on one of the links. Your issue might be entirely different, we shall see once I get word back from the folks that run the network closer to you.

Stay tuned, you're back on my radar.

armani · April 24, 2006

Sloooow again
SM is slow again since about 12 hours.
What is slow:

1. when i click on a link/pic: it takes seconds before the next page shows up, sometimes nothing happens !?
2. when i enter a gallery: pics show up one after the other (about one sec a pic) then it takes a few seconds before the main big picture shows up
3. lightbox: takes seconds before the pic shows up... sometimes i takes so long i close the box

ALL other sites i tested do not have a problem. Only Smugmug an Dgrin.
I have a fast broadband connection. Speed of my connection has doubled since last time i reported problems.

I have problems on different pc's: one at home, the other at my office.

armani · April 24, 2006

I also have the feeling the problem is getting worse and more frequently...

Andy · April 24, 2006

armani wrote:

SM is slow again since about 12 hours.
What is slow:

1. when i click on a link/pic: it takes seconds before the next page shows up, sometimes nothing happens !?
2. when i enter a gallery: pics show up one after the other (about one sec a pic) then it takes a few seconds before the main big picture shows up
3. lightbox: takes seconds before the pic shows up... sometimes i takes so long i close the box

ALL other sites i tested do not have a problem. Only Smugmug an Dgrin.
I have a fast broadband connection. Speed of my connection has doubled since last time i reported problems.

I have problems on different pc's: one at home, the other at my office.

Hi, armani, you've not filled out your location field on Dgrin, so we don't know where you are. If you could elaborate on your location, and your ISP, that would be helpful. Wireless might ask you to do a traceroute from you to us, as well.

mpmcleod · April 25, 2006

Should be be able to actually hit hera?

I get:
> traceroute www.smugmug.com
traceroute to hera.smugmug.com (63.81.134.23), 30 hops max, 38 byte packets
1 137.82.98.61 (137.82.98.61) 1.713 ms 11.747 ms 15.099 ms
2 a14-a18.net.ubc.ca (142.103.204.238) 2.856 ms 8.987 ms 1.451 ms
3 i9-a14.net.ubc.ca (142.103.204.98) 1.340 ms 1.434 ms 1.549 ms
4 policy1-tx-UBC.vantx.BC.net (207.23.240.6) 2.064 ms 1.882 ms 7.998 ms
5 ra1wh-ge4-3-15.vc.bigpipeinc.com (64.251.87.209) 2.328 ms 2.550 ms 4.033 ms
6 rc1bb-ge1-2-0.vc.shawcable.net (66.163.69.41) 50.794 ms 190.836 ms 3.035 ms
7 rc2wt-pos1-0.wa.shawcable.net (66.163.76.142) 6.094 ms 6.213 ms 6.206 ms
8 rc1sj-pos0-0.cl.shawcable.net (66.163.77.70) 27.274 ms 27.081 ms 27.169 ms
9 sec.r1-ge0-2-eq-sj.smugmug.com (206.223.117.201) 131.709 ms 202.738 ms 203.969 ms
10 * * *
11 * * *
12 * * *
13 * * *
14 * * *
15 * * *
16 * * *
17 * * *
18 * * *
19 * * *
20 * * *
21 * * *
22 * * *
23 * * *
24 * * *
25 * * *
26 * * *
27 * * *
then I stop it...

armani · April 25, 2006

Andy wrote:

Hi, armani, you've not filled out your location field on Dgrin, so we don't know where you are. If you could elaborate on your location, and your ISP, that would be helpful. Wireless might ask you to do a traceroute from you to us, as well.

I live in Belgium.
ISP = Telenet - at home (10 Mbps downstream en 256 Kbps upstream)
Belgacom ADSL - at work

I Simonius · April 25, 2006

Wireless wrote:

Last week, the transit provider (Limelight/LLNW) that connects one of our routes with BT Internet reported that they are having problems in the London area. I can't tell if this impacts you directly, but in my status update request for today I have added your route to see if it is related. I apologize for not posting this information here, please feel free to ping on this thread as much as you'd like to get status (I do watch the forum daily, as well as my email constantly).

Your output might be bad. Some of the packets that traceroute uses to calculate time are being blocked (intentionally or not) at a certain segment in your route within the UK. This shouldn't be taking this long to fix, but the last time we had issues with UK based customers, it was due to a bug in the switch software on one of the links. Your issue might be entirely different, we shall see once I get word back from the folks that run the network closer to you.

Stay tuned, you're back on my radar.

Thanks....
it would certainly make sense if it was London rather than local as my friend who lives on the other side of the country , and south, (which would probably alsobe routed via London) also experiences the slowness when 'smugging' (:D )

BTW_Can I get this thread to email me when updated?

Andy · April 25, 2006

Simon King wrote:

BTW_Can I get this thread to email me when updated?

in the navbar, upper left:

You!>Edit Options>Messaging>Subscriptions

wslam · April 25, 2006

Pls see this post here:
http://www.dgrin.com/showthread.php?p=291122#post291122

I think enough are telling you that there are real bandwidth issues, esp overseas.

hope this can be fixed. I have referred my friends to smugmug, but unfortunately, 75% of them said no after seeing the speed we get in HK. No, it's not an ISP issue. HK is one of the most developed (earliest and most matured) broadband places on the planet.

ws

Wireless wrote:

I need help seeing the facts here so I can talk with our providers and/or strategize on who we buy bandwidth from. Any information you can give me, routes, ping times, time of day, overall performance numbers, which 'most US sites', etc, will help immensely.

Right now, short of flying to Hong Kong this weekend (don't tempt me, I would do that), I'm limited to using sites like traceroute.org. Stuff looks really good (and heck, better than one of our competitors coming from Hong Kong right now), but that doesn't really help me solve any slowness you might be experiencing on an ongoing basis.

Last time we left it, from what I remember, the numbers coming across were within SLA for transpacific providers, yet not spectacular.

mpmcleod · April 25, 2006

wslam wrote:

Pls see this post here:
http://www.dgrin.com/showthread.php?p=291122#post291122

I think enough are telling you that there are real bandwidth issues, esp overseas.

hope this can be fixed. I have referred my friends to smugmug, but unfortunately, 75% of them said no after seeing the speed we get in HK. No, it's not an ISP issue. HK is one of the most developed (earliest and most matured) broadband places on the planet.

ws

Wireless,

What would be the best site to use (closest to you, uses same trunk as SM, or??) from the traceroute.org site?
from Berkeley to my computer:
traceroute from www.net.berkeley.edu to www.michaelmcleod.com (meg66)
1 inr-211-inr-203--128-32-206-1.HSRP.Berkeley.EDU (128.32.206.1) 0.579 ms 0.567 ms 0.384 ms
2 g3-1.inr-201-eva.Berkeley.EDU (128.32.255.1) 0.626 ms 0.647 ms 0.458 ms
3 ge-1-2-0.inr-002-reccev.Berkeley.EDU (128.32.0.36) 0.662 ms 0.565 ms 0.811 ms
4 hpr-oak-hpr--ucb-ge.cenic.net (137.164.27.129) 0.976 ms 0.714 ms 0.914 ms
5 svl-hpr--oak-hpr-10ge.cenic.net (137.164.25.8) 2.068 ms 2.026 ms 2.002 ms
6 lax-hpr--svl-hpr-10ge.cenic.net (137.164.25.12) 219.244 ms 33.100 ms 219.603 ms
7 canet-1-is-jmb-776.sttlwa.pacificwave.net (207.231.241.3) 56.863 ms 56.945 ms 56.744 ms
8 c4-bcnet.canet4.net (205.189.32.193) 77.324 ms 77.347 ms 77.185 ms
9 UBCi9-ORAN.BC.net (207.23.240.180) 77.641 ms 78.073 ms 77.544 ms
10 a14-i9.net.ubc.ca (142.103.204.97) 78.122 ms 77.863 ms 77.844 ms
11 a18-a14.net.ubc.ca (142.103.204.237) 77.606 ms 77.585 ms 77.848 ms
12 meg66.microbiology.ubc.ca (137.82.98.46) 79.872 ms !<10> 79.116 ms !<10> 78.959 ms !<10>
from Berkely to SM (it appears that I can't actually traceroute to hera):

traceroute from www.net.berkeley.edu to www.smugmug.com 1 inr-211-inr-203--128-32-206-1.HSRP.Berkeley.EDU (128.32.206.1) 1.009 ms 0.544 ms 0.585 ms
2 g3-1.inr-201-eva.Berkeley.EDU (128.32.255.1) 0.557 ms 0.574 ms 0.537 ms
3 ge-1-2-0.inr-002-reccev.Berkeley.EDU (128.32.0.36) 1.318 ms 0.719 ms 0.616 ms
4 oak-dc2--ucb-ge.cenic.net (137.164.23.29) 0.887 ms 20.877 ms 0.875 ms
5 dc-oak-dc2--svl-dc1-pos.cenic.net (137.164.22.32) 1.932 ms 1.884 ms 1.901 ms
6 limelight--paix-pni-ge.cenic.net (198.32.251.110) 2.945 ms 2.938 ms 2.999 ms
7 pri.r1-ge0-2-eq-sj.smugmug.com (206.223.117.70) 4.075 ms 3.980 ms 3.526 ms
8 * * *
9 *
I notice that you have at least 2 firewall/ routers in place (pri. and sec.) could it be the load balancing setup that is causing some of these issues? Or are these used as separate paths depending on the trunk it is serving?

armani · April 26, 2006

wslam wrote:

HK is one of the most developed (earliest and most matured) broadband places on the planet.
ws

Just like Belgium ;-). Check it out!

Smugmug SLOOOOW (International Network Connectivity)

Comments