Upload speed
Opps
Registered Users Posts: 160 Major grins
I'm not usually bothered by this but sometimes I upload 500-1000 photos and it seem to take forever. Before blaming SmugMug I'm going to try to see where my speed problem lies and was wondering what is the "normal" upload speed. I don't want/need an exact number. More something like "it takes me 12 seconds to upload 200 10MB images" 
or "I guess it takes about 4 hours to upload 200 images that are about 2-4MB each".
or "I guess it takes about 4 hours to upload 200 images that are about 2-4MB each". --
Jan Erik Moström
Jan Erik Moström
0
Comments
Portfolio • Workshops • Facebook • Twitter
But it looked like the test went really well (I've seen MUCH slower speeds recently) but these problems are probably all caused by the network close to me
Jan Erik Moström
I usually can compress them to at least 50% of their original size without noticeable quality impact.
For example a 100 photos gallery taken with a regular 10MP point&shoot will average 380 MB, I can get that down to 140MB and cut the upload time.
I do that with ACDSee by resaving as JPG with 73% quality.
Again I want to stress the no-noticeable difference. If you zoom in your photo you'll see that the pixels get re-arranged. You have to zoom in several times to realize this.
Check the following test gallery to compare big and "light" photos and also a 8X zoom on a detail :
http://mcgil.smugmug.com/Other/test/
Now that I have upgraded speeds of 25/5 on each connection, I have 75/15 at my disposal.
Want faster uploading? Vote for FTP!
I had thought about upgrading my connection to the next highest tier with my cable company, but found if I resize my photos to something around 1200x800, then I can upload the photos and this still gives the print size options I want to my customers and the photos are large enough to use the watermark feature. When I get an order, I replace the photo with a larger one for printing.
Want faster uploading? Vote for FTP!
This is my new connection (Fibre 100/100):
A popular testing service in Sweden, bredbandskollen.se, measures my connection as 92 down and 83 up.
Can anyone make sense of this? Is 1.68 all that SmugMug can take from me, or are there other issues?
Malte
I've seen folks in Europe getting much faster upload - we do not throttle or determine the speed, most of it is you and your ISP.
Portfolio • Workshops • Facebook • Twitter
OK, so let's start with me then, what can I do? I'm on XP/Firefox and I update them when ever they tell me to. I use Avast antivirus and XP's built-in firewall, no custom settings.
Any problems there?
What should I ask my ISP if I call them? If it's a matter of them having a weak satellite link to the US, how would I find an ISP with a great one?
Thanks!
Malte
Want faster uploading? Vote for FTP!
Depressing. :cry Stockholm and Oslo were around 50 upstream.
I'll talk to my ISP...
Malte
Malte
Hey Andy, have you seen faster than 3 from Europe (Preferrably Sweden or Scandinavia)?
I've had two of my ISP's competitors take the test and they both came up with 2-3...
To me it's looking like an Atlantic problem. :cry
Malte
it is us.
Portfolio • Workshops • Facebook • Twitter
Jupp, I caught that later on the Speedtest page, says "Hosted by SmugMug".
Malte
Welcome to the growing pains of a global world.
Want faster uploading? Vote for FTP!
If there is one, I'd be all over that. There are six or seven ISPs operating on my fibre and three of those top out at 3 Mbit. Not looking good for me... It's better than ADSL for sure but I guess I can safely downgrade from 100/100 to 10/10.
Malte
I'd tell them I'm a photographer and generate lots of data that needs to be sent to the US on a regular basis within a certain timeframe. I'd say that their service is fast in Europe and you're happy with that, but that the US speeds are far, far slower and aren't working for you. They'll probably ask where in the USA, and tell them silicon valley. This way they can't say that the destination is the problem. Then I'd ask them what can be done and see where it leads.
If ISP service is anything like here, it will take weeks before anything really happens or you get a real answer, but they should be in touch back and forth asking you to check various things, etc. Good luck and let us know what happens!
Want faster uploading? Vote for FTP!
Initially, I had three 6m/384k cable modems, but was still producing 2-5gb of data a weekend. So 5000MB at approximately 7seconds per MB would take 35000 seconds or 583 minutes or about 10hrs with the 18m/1.152m. This was far better than the 30 hours it would take if I only had one 6M/384k connection.
If the top speed you can get on fibre to the US is 3Mb and that's not enough, then you can get a second connection for another 3Mb for an effective total of 6Mb. Depending on the router used, you can expand this quite a bit. (The Linksys rv016 I use can take up to 7 connections.) You could also just keep your 100/100 connection and use it with one slower 10/10 that has the 3Mb connection for a 4.5Mb. Most of the multi-wan routers I've researched allow different bandwidth capacities to be used simultaneously.
So let's assume you get two additional 10/10 connections that yield 3Mb each to the US to work in conjunction with your 100/100. How would you connect all this? Well, it's actually pretty simple thanks to the availability of multi-wan routers.
I use products by Linksys/Cisco in my setup because back in 2005 there weren't too many choices. Here's a link to their product offerings. Look at the models starting with 'rv' for their multi-wan routers.
http://www.cisco.com/en/US/products/ps9923/products_data_sheets_list.html
www.peplink.com seems to have become the premiere multi-wan router out there now. I had to research multi-wan for a project recently and I'd probably go with one of these vs the Cisco if I had to do it again today.
www.edimax.com has some multi-wan products available from the US division.
www.xincom.com also seems to make some products that get mentioned a lot.
So once you have one of these routers, you set up each wan connection like you would for a single router. In this example case, plug in the three different connections to the wan ports and configure anything if needed (IP address, etc.). The LAN side where your computer plugs in will be just like normal. You won't need to change a thing there.
Now open a browser that has a bunch of different page elements. My site (www.huntsvillecarscene.com) is actually a good example as it pulls all sorts of elements from all over. You should see all three connections' activity lights show activity simultaneously. So now your download speeds are 100+10+10=120Mb.
But what you're truly interested in is the upload. Well, this is where it gets a bit more involved. Most of these routers can't automatically use all three connections when uploading, only downloading. But they will use multiple connections for multiple elements. Each instance of simple uploader is just one element, no matter how many files are in it. The uploader runs serially, one file at a time to one connection. But launch three instances of simple uploader and you'll see something neat--all three connections will start to get used as the router sees different elements being uploaded and uses the other connections as they get saturated. It will take more than 3 instances of simple uploader to completely saturate all the bandwidth on 3 Internet connections, but something like 6 instances will get you the full upload bandwidth of 7.5Mb. It's a bit tedious to set up the uploads since you have to manually split up the files into a few different batches, but it works quite effectively. With my 25/5 x 3 rv016 setup, I can upload the same 5gb batch in about 45 minutes.
Just for the fun of it, I ran the bredbandskollen.se test from my connection here in Madison, AL, USA and got 6.8Mbit/.8Mbit. It was using two of my connections which was effectively 30Mb/10Mb.
I hope this helps somewhat. Feel free to ask any questions.
Want faster uploading? Vote for FTP!
Malte
Want faster uploading? Vote for FTP!
Assuming I could afford more connections...
Do you mean i could/should:
Add one or more subscriptions to the same ISP on my fibre?
Add one or more ISPs to my fibre?
Add ISPs from other sources, say ADSL?
Edit: Called my ISP and they say the fibre I'm connected to has no support for more than one provider or subscription at the one time.
Malte
It would mean whatever they would need to do to get you another connection--another fiber drop, an adsl connection, cable, whatever they need to do. And I would call multiple ISPs to see if you can get connections from different ones if you want redundancy. It's rare that two ISPs have an outage simultaneously.
Knology's fastest modem at the time was 6M/384k. So I asked them if I got more than one modem would each have 6M/384k. They asked why I would want to do that when I could just get a router for the single line. I told them I needed more bandwidth. They didn't understand how a second modem would help, but they told me they probably could have a second modem if the signal was strong enough, and if it wasn't, they'd boost it. So I ordered two modems and then added a third. Every engineer that came by was startled at all the stuff I had connected to their network and how it all worked. And they told me that there was enough signal to add a 4th modem, but more would require a mini-distribution box.
Want faster uploading? Vote for FTP!
On my fibre there are only "fibre-providers" and there can be only one provider with one subscription on it. I could hook up my existing ADSL to a multi-WAN router but I'm thinking the 0,4 real world upstream isn't worth the effort or the money. Had I been able to run two 10/10 subscriptions on my fibre connection, I would have been tempted. :cry
You're quite the pioneer Samir!
Malte
One of the things you definitely have to look at when scaling up like this is cost. How much is the ADSL vs the fibre? If the ADSL is 10x cheaper (for example), I'd get 10 ADSL lines and use them because that would yield you 4Mb vs 1.68Mb. Actually, as long as it's 5x cheaper than fibre, it's worth it because it's the same speed for less, and then you can scale in 400kbit increments. Reminds me of my 384k upload days. :cry
You have no idea.
You should've seen my subnetting when I only had two consumer routers and two modems on one LAN. I got both lines to connect to one web server that was on the LAN.
Want faster uploading? Vote for FTP!
Yes. When I'm a brazillionaire I'll have another fibre dug in...
I think I'd run into the same problem here, more wires would have to come into the house. I'm not sure but I don't think it could work like your cable setup, and just add more modems.
Wow, you're a bonafide inventor!
Malte
I would look into if you can run a second connection over the TV fibre. Probably just never been asked before, but I think more people would actually do it if it could be done. Here in the US, Netflix accounts for 20% of peak Internet traffic, and it's 70% in Canada.
It probably would be more wires, but it would be as simple as a single cat5 run (or a few) since each run has 4 pair of wire--enough to run 4 ADSL connections. This should be much cheaper than fibre installation which even here is many times the cost of cat5/dsl/cable installations. I think even a t1 is cheaper than fibre.
I try to get things moving, but it feels like I have lead weights on my feet at times. :cry
Want faster uploading? Vote for FTP!