Options

Upload speed

OppsOpps Registered Users Posts: 160 Major grins
edited November 19, 2010 in SmugMug Support
I'm not usually bothered by this but sometimes I upload 500-1000 photos and it seem to take forever. Before blaming SmugMug I'm going to try to see where my speed problem lies and was wondering what is the "normal" upload speed. I don't want/need an exact number. More something like "it takes me 12 seconds to upload 200 10MB images" :) or "I guess it takes about 4 hours to upload 200 images that are about 2-4MB each".
--
Jan Erik Moström

Comments

  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited November 7, 2010
    It really depends on your ISP and your connection to us. I upload with my ISP here in NY at between 5 and 10 Mbps try this link and see what your speed to us is: http://smugmug.speedtest.net

    1020651634.png
  • Options
    OppsOpps Registered Users Posts: 160 Major grins
    edited November 7, 2010
    Well, not as good

    1020672411.png

    But it looked like the test went really well (I've seen MUCH slower speeds recently) but these problems are probably all caused by the network close to me
    --
    Jan Erik Moström
  • Options
    mcgilmcgil Registered Users Posts: 110 Major grins
    edited November 7, 2010
    A little trick too, I re-compress my photos before uploading them.

    I usually can compress them to at least 50% of their original size without noticeable quality impact.
    For example a 100 photos gallery taken with a regular 10MP point&shoot will average 380 MB, I can get that down to 140MB and cut the upload time.

    I do that with ACDSee by resaving as JPG with 73% quality.
    Again I want to stress the no-noticeable difference. If you zoom in your photo you'll see that the pixels get re-arranged. You have to zoom in several times to realize this.

    Check the following test gallery to compare big and "light" photos and also a 8X zoom on a detail :
    http://mcgil.smugmug.com/Other/test/
  • Options
    SamirDSamirD Registered Users Posts: 3,474 Major grins
    edited November 8, 2010
    Opps wrote: »
    Well, not as good

    1020672411.png

    But it looked like the test went really well (I've seen MUCH slower speeds recently) but these problems are probably all caused by the network close to me
    I used to have speeds like this, and had to get multiple cable modems and set up a multi-wan router like the ones from peplink.com to be able to upload and have a decent turnaround time.

    Now that I have upgraded speeds of 25/5 on each connection, I have 75/15 at my disposal. clap.gif
    Pictures and Videos of the Huntsville Car Scene: www.huntsvillecarscene.com
    Want faster uploading? Vote for FTP!
  • Options
    Rocketman766Rocketman766 Registered Users Posts: 332 Major grins
    edited November 8, 2010
    1022259921.png

    I had thought about upgrading my connection to the next highest tier with my cable company, but found if I resize my photos to something around 1200x800, then I can upload the photos and this still gives the print size options I want to my customers and the photos are large enough to use the watermark feature. When I get an order, I replace the photo with a larger one for printing.
  • Options
    SamirDSamirD Registered Users Posts: 3,474 Major grins
    edited November 8, 2010
    When I get an order, I replace the photo with a larger one for printing.
    That's a great use of proof delay. thumb.gif
    Pictures and Videos of the Huntsville Car Scene: www.huntsvillecarscene.com
    Want faster uploading? Vote for FTP!
  • Options
    MalteMalte Registered Users Posts: 1,181 Major grins
    edited November 16, 2010
    This is my old connection (ADSL 8/1):

    1033100856.png

    This is my new connection (Fibre 100/100):

    1033105336.png

    A popular testing service in Sweden, bredbandskollen.se, measures my connection as 92 down and 83 up.

    Can anyone make sense of this? Is 1.68 all that SmugMug can take from me, or are there other issues?

    Malte
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited November 16, 2010
    Malte wrote: »
    This is my old connection (ADSL 8/1):

    1033100856.png

    This is my new connection (Fibre 100/100):

    1033105336.png

    A popular testing service in Sweden, bredbandskollen.se, measures my connection as 92 down and 83 up.

    Can anyone make sense of this? Is 1.68 all that SmugMug can take from me, or are there other issues?

    Malte

    I've seen folks in Europe getting much faster upload - we do not throttle or determine the speed, most of it is you and your ISP.
  • Options
    MalteMalte Registered Users Posts: 1,181 Major grins
    edited November 17, 2010
    Andy wrote: »
    I've seen folks in Europe getting much faster upload - we do not throttle or determine the speed, most of it is you and your ISP.

    OK, so let's start with me then, what can I do? I'm on XP/Firefox and I update them when ever they tell me to. I use Avast antivirus and XP's built-in firewall, no custom settings.

    Any problems there?

    What should I ask my ISP if I call them? If it's a matter of them having a weak satellite link to the US, how would I find an ISP with a great one?

    Thanks!

    Malte
  • Options
    SamirDSamirD Registered Users Posts: 3,474 Major grins
    edited November 17, 2010
    Wow, this is very interesting. A 100/100Mb fibre link only gets 1.68Mb to the US? Crazy!
    Pictures and Videos of the Huntsville Car Scene: www.huntsvillecarscene.com
    Want faster uploading? Vote for FTP!
  • Options
    MalteMalte Registered Users Posts: 1,181 Major grins
    edited November 17, 2010
    SamirD wrote: »
    Wow, this is very interesting. A 100/100Mb fibre link only gets 1.68Mb to the US? Crazy!

    Depressing. :cry Stockholm and Oslo were around 50 upstream. eek7.gif

    I'll talk to my ISP...

    Malte
  • Options
    MalteMalte Registered Users Posts: 1,181 Major grins
    edited November 17, 2010
    This link on Speedtest, http://smugmug.speedtest.net is that a SmugMug server or an independant test server near SmugMug in San Jose?

    Malte
  • Options
    MalteMalte Registered Users Posts: 1,181 Major grins
    edited November 17, 2010
    Andy wrote: »
    I've seen folks in Europe getting much faster upload - we do not throttle or determine the speed, most of it is you and your ISP.

    Hey Andy, have you seen faster than 3 from Europe (Preferrably Sweden or Scandinavia)?

    I've had two of my ISP's competitors take the test and they both came up with 2-3...

    To me it's looking like an Atlantic problem. :cry

    Malte
  • Options
    AndyAndy Registered Users Posts: 50,016 Major grins
    edited November 17, 2010
    Malte wrote: »
    This link on Speedtest, http://smugmug.speedtest.net is that a SmugMug server or an independant test server near SmugMug in San Jose?

    Malte

    it is us.
  • Options
    MalteMalte Registered Users Posts: 1,181 Major grins
    edited November 17, 2010
    Andy wrote: »
    it is us.

    Jupp, I caught that later on the Speedtest page, says "Hosted by SmugMug".

    Malte
  • Options
    SamirDSamirD Registered Users Posts: 3,474 Major grins
    edited November 17, 2010
    I wonder if there's ISPs and hosting companies there that specialize in a fast link to the US. If so, I'd get in touch with them and see what solutions they could come up with. I'm sure you guys aren't the only ones affected.

    Welcome to the growing pains of a global world. rolleyes1.gif Great to see that we're running into such an issue, even though it suck. It's nice to see business and communication on another continent is as easy as a neighboring city. thumb.gif Now we just need everything faster. :D
    Pictures and Videos of the Huntsville Car Scene: www.huntsvillecarscene.com
    Want faster uploading? Vote for FTP!
  • Options
    MalteMalte Registered Users Posts: 1,181 Major grins
    edited November 17, 2010
    SamirD wrote: »
    I wonder if there's ISPs and hosting companies there that specialize in a fast link to the US...

    If there is one, I'd be all over that. There are six or seven ISPs operating on my fibre and three of those top out at 3 Mbit. Not looking good for me... It's better than ADSL for sure but I guess I can safely downgrade from 100/100 to 10/10. deal.gifcry

    Malte
  • Options
    SamirDSamirD Registered Users Posts: 3,474 Major grins
    edited November 17, 2010
    Malte wrote: »
    What should I ask my ISP if I call them? If it's a matter of them having a weak satellite link to the US, how would I find an ISP with a great one?
    Here's what I would do, but you should adjust this for your local culture/business climate.

    I'd tell them I'm a photographer and generate lots of data that needs to be sent to the US on a regular basis within a certain timeframe. I'd say that their service is fast in Europe and you're happy with that, but that the US speeds are far, far slower and aren't working for you. They'll probably ask where in the USA, and tell them silicon valley. This way they can't say that the destination is the problem. Then I'd ask them what can be done and see where it leads.

    If ISP service is anything like here, it will take weeks before anything really happens or you get a real answer, but they should be in touch back and forth asking you to check various things, etc. Good luck and let us know what happens! thumb.gif
    Pictures and Videos of the Huntsville Car Scene: www.huntsvillecarscene.com
    Want faster uploading? Vote for FTP!
  • Options
    SamirDSamirD Registered Users Posts: 3,474 Major grins
    edited November 17, 2010
    Malte wrote: »
    If there is one, I'd be all over that. There are six or seven ISPs operating on my fibre and three of those top out at 3 Mbit. Not looking good for me... It's better than ADSL for sure but I guess I can safely downgrade from 100/100 to 10/10. deal.gifcry

    Malte
    You may have to do what I did and aggregate multiple links using a multi-wan router.

    Initially, I had three 6m/384k cable modems, but was still producing 2-5gb of data a weekend. So 5000MB at approximately 7seconds per MB would take 35000 seconds or 583 minutes or about 10hrs with the 18m/1.152m. This was far better than the 30 hours it would take if I only had one 6M/384k connection.

    If the top speed you can get on fibre to the US is 3Mb and that's not enough, then you can get a second connection for another 3Mb for an effective total of 6Mb. Depending on the router used, you can expand this quite a bit. (The Linksys rv016 I use can take up to 7 connections.) You could also just keep your 100/100 connection and use it with one slower 10/10 that has the 3Mb connection for a 4.5Mb. Most of the multi-wan routers I've researched allow different bandwidth capacities to be used simultaneously.

    So let's assume you get two additional 10/10 connections that yield 3Mb each to the US to work in conjunction with your 100/100. How would you connect all this? Well, it's actually pretty simple thanks to the availability of multi-wan routers.

    I use products by Linksys/Cisco in my setup because back in 2005 there weren't too many choices. Here's a link to their product offerings. Look at the models starting with 'rv' for their multi-wan routers.
    http://www.cisco.com/en/US/products/ps9923/products_data_sheets_list.html

    www.peplink.com seems to have become the premiere multi-wan router out there now. I had to research multi-wan for a project recently and I'd probably go with one of these vs the Cisco if I had to do it again today.

    www.edimax.com has some multi-wan products available from the US division.
    www.xincom.com also seems to make some products that get mentioned a lot.

    So once you have one of these routers, you set up each wan connection like you would for a single router. In this example case, plug in the three different connections to the wan ports and configure anything if needed (IP address, etc.). The LAN side where your computer plugs in will be just like normal. You won't need to change a thing there.

    Now open a browser that has a bunch of different page elements. My site (www.huntsvillecarscene.com) is actually a good example as it pulls all sorts of elements from all over. You should see all three connections' activity lights show activity simultaneously. So now your download speeds are 100+10+10=120Mb.

    But what you're truly interested in is the upload. Well, this is where it gets a bit more involved. Most of these routers can't automatically use all three connections when uploading, only downloading. But they will use multiple connections for multiple elements. Each instance of simple uploader is just one element, no matter how many files are in it. The uploader runs serially, one file at a time to one connection. But launch three instances of simple uploader and you'll see something neat--all three connections will start to get used as the router sees different elements being uploaded and uses the other connections as they get saturated. It will take more than 3 instances of simple uploader to completely saturate all the bandwidth on 3 Internet connections, but something like 6 instances will get you the full upload bandwidth of 7.5Mb. It's a bit tedious to set up the uploads since you have to manually split up the files into a few different batches, but it works quite effectively. With my 25/5 x 3 rv016 setup, I can upload the same 5gb batch in about 45 minutes. clap.gif

    Just for the fun of it, I ran the bredbandskollen.se test from my connection here in Madison, AL, USA and got 6.8Mbit/.8Mbit. It was using two of my connections which was effectively 30Mb/10Mb.

    I hope this helps somewhat. Feel free to ask any questions.
    Pictures and Videos of the Huntsville Car Scene: www.huntsvillecarscene.com
    Want faster uploading? Vote for FTP!
  • Options
    MalteMalte Registered Users Posts: 1,181 Major grins
    edited November 17, 2010
    Thank you Samir for all your input and help. I'm dogtired right now so I'll read all the goodies tomorrow!

    Malte
  • Options
    SamirDSamirD Registered Users Posts: 3,474 Major grins
    edited November 17, 2010
    No problem. I know we've got a couple of hours of time difference. thumb.gif
    Pictures and Videos of the Huntsville Car Scene: www.huntsvillecarscene.com
    Want faster uploading? Vote for FTP!
  • Options
    MalteMalte Registered Users Posts: 1,181 Major grins
    edited November 18, 2010
    SamirD wrote: »
    So let's assume you get two additional 10/10 connections that yield 3Mb each to the US to work in conjunction with your 100/100. How would you connect all this? Well, it's actually pretty simple thanks to the availability of multi-wan routers.

    Assuming I could afford more connections... :D

    Do you mean i could/should:

    Add one or more subscriptions to the same ISP on my fibre?
    Add one or more ISPs to my fibre?
    Add ISPs from other sources, say ADSL?

    Edit: Called my ISP and they say the fibre I'm connected to has no support for more than one provider or subscription at the one time.

    Malte
  • Options
    SamirDSamirD Registered Users Posts: 3,474 Major grins
    edited November 18, 2010
    Malte wrote: »
    Assuming I could afford more connections... :D

    Do you mean i could/should:

    Add one or more subscriptions to the same ISP on my fibre?
    Add one or more ISPs to my fibre?
    Add ISPs from other sources, say ADSL?

    Edit: Called my ISP and they say the fibre I'm connected to has no support for more than one provider or subscription at the one time.

    Malte
    Yes assuming you could afford it. :Drolleyes1.gif

    It would mean whatever they would need to do to get you another connection--another fiber drop, an adsl connection, cable, whatever they need to do. And I would call multiple ISPs to see if you can get connections from different ones if you want redundancy. It's rare that two ISPs have an outage simultaneously.

    Knology's fastest modem at the time was 6M/384k. So I asked them if I got more than one modem would each have 6M/384k. They asked why I would want to do that when I could just get a router for the single line. I told them I needed more bandwidth. They didn't understand how a second modem would help, but they told me they probably could have a second modem if the signal was strong enough, and if it wasn't, they'd boost it. So I ordered two modems and then added a third. Every engineer that came by was startled at all the stuff I had connected to their network and how it all worked. And they told me that there was enough signal to add a 4th modem, but more would require a mini-distribution box. rolleyes1.gif Luckily, over the years, the cost has remained the same while the bandwidth has increased. I've been grandfathered into some of the pricing plans so I'm actually getting the 3rd modem for about 1/2 off now.
    Pictures and Videos of the Huntsville Car Scene: www.huntsvillecarscene.com
    Want faster uploading? Vote for FTP!
  • Options
    MalteMalte Registered Users Posts: 1,181 Major grins
    edited November 18, 2010
    SamirD wrote:
    It would mean whatever they would need to do to get you another connection--another fiber drop, an adsl connection, cable, whatever they need to do.

    On my fibre there are only "fibre-providers" and there can be only one provider with one subscription on it. I could hook up my existing ADSL to a multi-WAN router but I'm thinking the 0,4 real world upstream isn't worth the effort or the money. Had I been able to run two 10/10 subscriptions on my fibre connection, I would have been tempted. :cry
    SamirD wrote:
    ...They asked why I would want to do that when I could just get a router for the single line...They didn't understand how a second modem would help...Every engineer that came by was startled at all the stuff I had connected to their network and how it all worked...

    You're quite the pioneer Samir! thumb.gif

    Malte
  • Options
    SamirDSamirD Registered Users Posts: 3,474 Major grins
    edited November 18, 2010
    Malte wrote: »
    On my fibre there are only "fibre-providers" and there can be only one provider with one subscription on it. I could hook up my existing ADSL to a multi-WAN router but I'm thinking the 0,4 real world upstream isn't worth the effort or the money. Had I been able to run two 10/10 subscriptions on my fibre connection, I would have been tempted. :cry
    Hmmm...can you not get a second fibre run? Then you could have another provider, or even the same one.

    One of the things you definitely have to look at when scaling up like this is cost. How much is the ADSL vs the fibre? If the ADSL is 10x cheaper (for example), I'd get 10 ADSL lines and use them because that would yield you 4Mb vs 1.68Mb. Actually, as long as it's 5x cheaper than fibre, it's worth it because it's the same speed for less, and then you can scale in 400kbit increments. Reminds me of my 384k upload days. :cry
    Malte wrote: »
    You're quite the pioneer Samir! thumb.gif
    You have no idea. rolleyes1.gif I'm usually either behind the curve or waaay ahead. I though about the algorithms behind Pandora and Netflix/TIVO probably around the same time the creators did if not before. I just didn't have the means to make it reality. If my infinitely variable timing (IVT) engine idea would go into production, we'd probably have gasoline cars with efficiency beyond current hybrids and with more power. And then there's my idea for powering hotel locks using sources other than batteries. A local company tried to steal that idea and we went back and forth long enough that one of the lock companies (Kaba-Ilco) incorporated the ultimate version of the concept right into their locks before I could even start the patent process.

    You should've seen my subnetting when I only had two consumer routers and two modems on one LAN. I got both lines to connect to one web server that was on the LAN. eek7.gif
    Pictures and Videos of the Huntsville Car Scene: www.huntsvillecarscene.com
    Want faster uploading? Vote for FTP!
  • Options
    MalteMalte Registered Users Posts: 1,181 Major grins
    edited November 19, 2010
    SamirD wrote:
    Hmmm...can you not get a second fibre run? Then you could have another provider, or even the same one.

    Yes. When I'm a brazillionaire I'll have another fibre dug in... :D I don't know how it works in the U.S., but here in Sweden the subscriber pays for installation. Actually, the tubing holds two fibre wires but one is somehow dedicated to TV services. I'll definately look into if this is written in stone or if I can use the second fibre for broadband too. Who watches TV anyway?! mwink.gif
    SamirD wrote:
    ...How much is the ADSL vs the fibre? If the ADSL is 10x cheaper (for example), I'd get 10 ADSL lines and use them because that would yield you 4Mb vs 1.68Mb...

    I think I'd run into the same problem here, more wires would have to come into the house. I'm not sure but I don't think it could work like your cable setup, and just add more modems. ne_nau.gif
    SamirD wrote:
    ...And then there's my idea for powering hotel locks using sources other than batteries. A local company tried to steal that idea and we went back and forth long enough that one of the lock companies (Kaba-Ilco) incorporated the ultimate version of the concept right into their locks before I could even start the patent process.

    Wow, you're a bonafide inventor! bowdown.gif Be quiet until you have the patent... :D

    Malte
  • Options
    SamirDSamirD Registered Users Posts: 3,474 Major grins
    edited November 19, 2010
    Malte wrote: »
    Yes. When I'm a brazillionaire I'll have another fibre dug in... :D I don't know how it works in the U.S., but here in Sweden the subscriber pays for installation. Actually, the tubing holds two fibre wires but one is somehow dedicated to TV services. I'll definately look into if this is written in stone or if I can use the second fibre for broadband too. Who watches TV anyway?! mwink.gif
    Wow, that's definitely different than here. Here's there's an "installation fee" that everyone pays even if the installation is just plugging in the equipment. But I guess that covers the time when they have to dig and bury a wire too. For commercial installations, they'll only build it if you sign a contract for a year or more, depending on how much the construction costs are.

    I would look into if you can run a second connection over the TV fibre. Probably just never been asked before, but I think more people would actually do it if it could be done. Here in the US, Netflix accounts for 20% of peak Internet traffic, and it's 70% in Canada. eek7.gif I bet once the service is launched Internationally, the demand for traditional television will go down.
    Malte wrote: »
    I think I'd run into the same problem here, more wires would have to come into the house. I'm not sure but I don't think it could work like your cable setup, and just add more modems. ne_nau.gif
    It probably would be more wires, but it would be as simple as a single cat5 run (or a few) since each run has 4 pair of wire--enough to run 4 ADSL connections. This should be much cheaper than fibre installation which even here is many times the cost of cat5/dsl/cable installations. I think even a t1 is cheaper than fibre. rolleyes1.gif
    Malte wrote: »
    Wow, you're a bonafide inventor! bowdown.gif Be quiet until you have the patent... :D
    I try to get things moving, but it feels like I have lead weights on my feet at times. :cry
    Pictures and Videos of the Huntsville Car Scene: www.huntsvillecarscene.com
    Want faster uploading? Vote for FTP!
Sign In or Register to comment.