Upload timing out constantly

wslam

Very frustrating. I am uploading about 300MB of photos using the beta JAVA tool and after NUMEROUS attempts, about every 50MB, I get "File Upload Interrupted. java.net.ConnectException: Connection Timed Out: connect"

Any suggestion?

ws

Andy

wslam wrote:

Any suggestion?

ws

Yes. Go to www.dslreports.com, and run the packet loss test under "tools." Registration req'd, but it's free. The test will give you a link when you can share here with me.

wslam

Andy wrote:

Yes. Go to www.dslreports.com, and run the packet loss test under "tools." Registration req'd, but it's free. The test will give you a link when you can share here with me.

I tried, but it says my IP is unpingable. I already disabled router on my Router, on my Windows Firewall. I am not running anyother software firewall. I have no idea how to make myself pingable.

Andy

wslam wrote:

I tried, but it says my IP is unpingable. I already disabled router on my Router, on my Windows Firewall. I am not running anyother software firewall. I have no idea how to make myself pingable.

They have instructions to follow when they can't ping you. Please do them, and let us know.

wslam

Andy wrote:

They have instructions to follow when they can't ping you. Please do them, and let us know.

Hi Andy,
Thanks for your quick responses. If you read my previous post carefully, you would have known that I already followed the steps... by enabling the echo setting in XP Firewall, and I have also disabled firewall on my linksys router. Still no go... Are there any tests I can do that will help you diagnose this for me?

ws

Andy

wslam wrote:

Hi Andy,
Thanks for your quick responses. If you read my previous post carefully, you would have known that I already followed the steps... by enabling the echo setting in XP Firewall, and I have also disabled firewall on my linksys router. Still no go... Are there any tests I can do that will help you diagnose this for me?

ws

I did read it. The part about "I have no idea how to make myself pingable" caught my attention.

I'm reaching here. Look here? http://www.dslreports.com/badrouters
Did you try everything outlined here? http://www.dslreports.com/faq/tweaks/3.%20Becoming%20Pingable

wslam

Andy wrote:

I did read it. The part about "I have no idea how to make myself pingable" caught my attention.

I'm reaching here. Look here? http://www.dslreports.com/badrouters
Did you try everything outlined here? http://www.dslreports.com/faq/tweaks/3.%20Becoming%20Pingable

Yes Andy, I already read the 'Becoming Pingable' FAQ. not working...

Andy

Have you checked to make sure you have the latest Java?
http://www.smugmug.com/help/problems

luke_church

I might be able to help you guys out here, I saw endless amounts of this when testing the HTTP core for SmugTools.

Questions to get started:

Where are you? (Physically)
What type of connection are you using? (Is it DSL? Coporate?)
What OS are you using? (XP?)
Which version of Java?
What kind of throughput are you getting? (Can you time how long it takes the uploader to upload say a 5Mb file?)
Can you send me a tracert of your connection to Smugmug?

Instructions here: http://www.dgrin.com/showpost.php?p=288962&postcount=9

Unfortunately the solution I developed for my uploader would probably require code changes to work on the BETA uploader, and I didn't write it... But we can at least try and see if it's the same problem :

Luke

marlinspike

You know, a couple weeks ago I started having this same problem with the beta uploader, sometimes it would cut out more frequently than others. Never had the problem until a few weeks ago, and the internet explorer for windows uploader works fine, so I just started opening like 10 windows at a time and uploading my large batches that way.

wslam

luke_church wrote:

I might be able to help you guys out here, I saw endless amounts of this when testing the HTTP core for SmugTools.

Questions to get started:

Where are you? (Physically)
What type of connection are you using? (Is it DSL? Coporate?)
What OS are you using? (XP?)
Which version of Java?
What kind of throughput are you getting? (Can you time how long it takes the uploader to upload say a 5Mb file?)
Can you send me a tracert of your connection to Smugmug?

Instructions here: http://www.dgrin.com/showpost.php?p=288962&postcount=9

Unfortunately the solution I developed for my uploader would probably require code changes to work on the BETA uploader, and I didn't write it... But we can at least try and see if it's the same problem :

Luke

Ok, first of all, THANK you for both Andy and Luke_Church.

Here is the latest development and ans to your questions:

Where are you? (Physically)
Hong Kong

What type of connection are you using? (Is it DSL? Coporate?)
ADSL (8Mbps down/800kbps up)

What OS are you using? (XP?)
Tried both XP and OSX 10.4. both with latest patches.

Which version of Java?
Latest. I am one of those people who must have the latest versions of everything.

Can you send me a tracert of your connection to Smugmug?

C:\>tracert upload.smugmug.com

Tracing route to hermes.smugmug.com [63.81.134.24]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms 192.168.0.1
2 19 ms 19 ms 20 ms pcd-vta2-1-rx.netvigator.com [203.218.32.254]
3 19 ms 19 ms 19 ms 218.102.57.70
4 22 ms 19 ms 19 ms 218.102.21.210
5 73 ms 75 ms 78 ms unknown.net.reach.com [134.159.100.65]
6 50 ms 54 ms 51 ms unknown.net.reach.com [202.84.154.237]
7 38 ms 41 ms 38 ms unknown.net.reach.com [202.84.155.129]
8 41 ms 38 ms 38 ms i-10-1.tmhstcar01.net.reach.com [202.84.153.213]

9 47 ms 47 ms 48 ms teleglobe2-RGE.hkix.net [202.40.161.224]
10 378 ms 382 ms 407 ms if-5-0.core1.HK2-HongKong.teleglobe.net [216.6.95.85]
11 403 ms 395 ms 404 ms if-4-10.core3.PDI-PaloAlto.teleglobe.net [216.6.95.38]
12 * * * Request timed out.
13 419 ms 412 ms 407 ms if-5-0.core2.SQN-SanJose.Teleglobe.net [64.86.84.134]
14 * * * Request timed out.
15 413 ms 405 ms 428 ms pri.r1-ge0-2-eq-sj.smugmug.com [206.223.117.70]
16 398 ms 399 ms 400 ms hermes.smugmug.com [63.81.134.24]

Trace complete.

It's not easy to see the poor performance and high latency I am experiencing...

2) I managed to run the packet loss test using dslreports.com
And the LINK to the result is http://www.dslreports.com/quality/nil/2022554

Thank you everyone.

ws

wslam

marlinspike wrote:

You know, a couple weeks ago I started having this same problem with the beta uploader, sometimes it would cut out more frequently than others. Never had the problem until a few weeks ago, and the internet explorer for windows uploader works fine, so I just started opening like 10 windows at a time and uploading my large batches that way.

but that seems so 'unelegant'

luke_church

wslam wrote:

but that seems so 'unelegant'

Yeah, we can do better than that.

Give me a little while to poke around, have a think and check a few hypothesies and I'll get back to you.

Cheers,

Luke

wslam

Andy wrote:

Have you checked to make sure you have the latest Java?
http://www.smugmug.com/help/problems

Hi Andy,
I have provided all the info you asked for. Can you pls help? Thanks.

ws

luke_church

wslam wrote:

Hi Andy,
I have provided all the info you asked for. Can you pls help? Thanks.

ws

The latency on your connection is high, from the preliminary tests I've run this will hurt the overall reliability and will really damage the upload speeds.

I have a solution to this in the .NET uploader I've written:

-> modify the HTTP parameters to extend the time out
-> (Bizzarely) suppress HTTP keep-alive pooling
-> Parralelise the upload process to help saturate the link despite the latency
-> Trap failures and put them in a 'retry queue' rather that killing the overall task list.

Combining these I get an uploader performance improvement of ~30x faster.

I didn't want to release SmugUp for a while, as there are a few things it currently doesn't do well and I don't anticipate having much time for the next few weeks. I could probably have an alpha release out by the end of the week, but there will be no major bug fixes/feature increments until July.

It would hopefully solve your problem. Or I'm happy to try and help Smugmug do the same tricks for their uploader, but that will take a bit longer I suspect.

If you're interested and willing to put up with alpha release grade software, let me know, but if you want a rock-solid stable solution and can't wait until summer (I understand!), then see if Andy can work wonders for you, he often can :

Hope this helps to some degree,

Luke

tsk1979

I get the same problem with beta uploader after around 200MB. I guess I will try the alpha release. where is it available?

luke_church wrote:

The latency on your connection is high, from the preliminary tests I've run this will hurt the overall reliability and will really damage the upload speeds.

I have a solution to this in the .NET uploader I've written:

-> modify the HTTP parameters to extend the time out
-> (Bizzarely) suppress HTTP keep-alive pooling
-> Parralelise the upload process to help saturate the link despite the latency
-> Trap failures and put them in a 'retry queue' rather that killing the overall task list.

Combining these I get an uploader performance improvement of ~30x faster.

I didn't want to release SmugUp for a while, as there are a few things it currently doesn't do well and I don't anticipate having much time for the next few weeks. I could probably have an alpha release out by the end of the week, but there will be no major bug fixes/feature increments until July.

It would hopefully solve your problem. Or I'm happy to try and help Smugmug do the same tricks for their uploader, but that will take a bit longer I suspect.

If you're interested and willing to put up with alpha release grade software, let me know, but if you want a rock-solid stable solution and can't wait until summer (I understand!), then see if Andy can work wonders for you, he often can :

Hope this helps to some degree,

Luke

Andy

tsk1979 wrote:

I get the same problem

Sooo you've run the tests, etc? Hve you checked with your ISP?

Ben

I haven't seen it mentioned yet in this thread (though I may have missed it, so forgive me if it has been mentioned already).

Are you running any AntiVirus software, or anything like Norton InternetSecurity? Both of those type of programs cause a LOT of problems with anything requiring an uninterrupted connection.

luke_church

Ben wrote:

I haven't seen it mentioned yet in this thread (though I may have missed it, so forgive me if it has been mentioned already).

Are you running any AntiVirus software, or anything like Norton InternetSecurity? Both of those type of programs cause a LOT of problems with anything requiring an uninterrupted connection.

Good point indeed, I hadn't thought of that one.

However, whilst I do run various security packages, I have fairly strong reasons to believe that they are no the cause of the time-outs that I was experiencing.

E.g. (and a bad one I know, sorry I'm a little hesitant to give details of my security setup in a forum) It would also seem bizzare in that case that running Etherial which causes *a lot* of disk access, and therefore process stalls, doesn't change the frequency of the problems.

Fundamentally, IMHO, the uploader shouldn't require an uninteruptted connection. Uninteruptted per single transfer is reasonable but uninterupted per session is perhaps a little over-optimistic given the general nastiness of comms over the internet, esp. transconitental comms... But that's just my opinion, and I'm by no means an expert at these things. :

Luke

luke_church

tsk1979 wrote:

I get the same problem with beta uploader after around 200MB. I guess I will try the alpha release. where is it available?

As I said, I'll try to post a link by the end of the week.

Luke

JimM

wslam wrote:

C:\>tracert upload.smugmug.com

Tracing route to hermes.smugmug.com [63.81.134.24]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms 192.168.0.1
2 19 ms 19 ms 20 ms pcd-vta2-1-rx.netvigator.com [203.218.32.254]
3 19 ms 19 ms 19 ms 218.102.57.70
4 22 ms 19 ms 19 ms 218.102.21.210
5 73 ms 75 ms 78 ms unknown.net.reach.com [134.159.100.65]
6 50 ms 54 ms 51 ms unknown.net.reach.com [202.84.154.237]
7 38 ms 41 ms 38 ms unknown.net.reach.com [202.84.155.129]
8 41 ms 38 ms 38 ms i-10-1.tmhstcar01.net.reach.com [202.84.153.213]

9 47 ms 47 ms 48 ms teleglobe2-RGE.hkix.net [202.40.161.224]
10 378 ms 382 ms 407 ms if-5-0.core1.HK2-HongKong.teleglobe.net [216.6.95.85]
11 403 ms 395 ms 404 ms if-4-10.core3.PDI-PaloAlto.teleglobe.net [216.6.95.38]
12 * * * Request timed out.
13 419 ms 412 ms 407 ms if-5-0.core2.SQN-SanJose.Teleglobe.net [64.86.84.134]
14 * * * Request timed out.
15 413 ms 405 ms 428 ms pri.r1-ge0-2-eq-sj.smugmug.com [206.223.117.70]
16 398 ms 399 ms 400 ms hermes.smugmug.com [63.81.134.24]

Trace complete.

Looks to me like you are getting some terrible response times, packet loss and time outs once your ISP gets you to the States. My suggestion would be to contact your ISP about this. (maybe some water got in the fiber laying on the ocean botton? <img src="https://us.v-cdn.net/6029383/emoji/rolleyes1.gif&quot; border="0" alt="" >)

tsk1979

Andy wrote:

Sooo you've run the tests, etc? Hve you checked with your ISP?

Actually my connection goes down sometimes. But then as Luke mentioned an uploader should automatically retry when such a thing happens. Since I am on 128kbit upstream, I ususally do the uploads at night. So the uploader should automatically retry on connection breakage.

SamirD

luke_church wrote:

As I said, I'll try to post a link by the end of the week.

Luke

Anxiously awaiting the alpha. :