Can youz test a new uploader?

Baldy · April 9, 2009

Need a speed fix?

We decided to roll our own Simple uploader 'cus we thought we could make it smaller and faster.

The current one is an industry standard, used by Facebook et al, but being built by a third party we can't get quick fixes when browser upgrades make it woof.

Can you give the new hotness a whirl and see what you think?

To try it, go to the simple uploader (here's how). Your browser address bar should look something like this:

Just make sure it says &applet=10 at the end of the URL and press "return" or "enter", depending on your machine.

The new version of the Simple uploader should ask if you wanna trust it (you do), then load and look like:

Then add some photos & videos. Fast? Reliable? Intuitive?

Okay, so it's not pretty yet...

Thanks!
Baldy

jfriend · April 9, 2009

I just gave it a try and it did not work for me. I picked 6 images to upload. I hit the Upload button. The first three started animating their progress indicators with little green rectangles going back and forth and they immediately showed 100%, but nothing else was happening and the progress indicator never showed any typical thermometer moving. It says "Current Speed: idle" and "Time Remaining: idle".

After a couple minutes of no visual change, I hit the Stop Uploading button. It didn't appear to do anything except enable the Remove All button. The green rectangles were still going back and forth on the first three images, nothing else happening. Based on the responsiveness of my network connection, I think it was still uploading the three images. It appears to have so saturated my connection that I can barely do anything else (I have 6Mbps down, 1Mbps up).

Here's what it looks like after I hit the Stop Uploading button:

It did eventually finish uploading the three images that started. The thermometers never showed anything other than the green rectangles and 100%. After the uploaded, they disappeared. The time remaining and current speed never showed anything other than "idle".

When I check the gallery, the first three images did upload. Summary of what I saw:

Initial progress is very bizarre. It jumped to 100% immediately and then just showed the animating green rectangles. The progress thermometers seem busted. I much, much prefer what the previous simple uploader showed because it feels like it's doing something all the time.
Speed indicator never showed anything
Time remaining never showed anything.
Cancel doesn't seem to cancel the existing uploads.
It so saturates my connection that I can't really do much on the internet while it's uploading (I realize this is probably a side effect of going for max speed, but it is still unpleasant).
The progress showing how many have successfully uploaded "3/6" is very tiny and not very satisfying. The individual files appear to just "disappear" and you wonder what happened to them or if they worked.
The progress indicator showed 6/6 when it only uploaded 3.

I am normally a Star Explorer uploader so I can't really offer any comparison to the previous simple uploader.

BradfordBenn · April 9, 2009

A couple of results
Howdy-

So I ran the same 27 images through four different uploaders for testing. Here is what I came up with:

45.2 MB - 27 Images
Running under Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.8) Gecko/2009032608 Firefox/3.0.8
Applet = 10 Upload 2:25
Applet = 9 Upload 3:25

Mac Daddy v3.03 2:24

Star *Explorer with Boost 2:25 (running under Fusion with 2GB on XP)

I did these all back to back to back to back to keep it as fair as possible.

Does that help?

Andy · April 9, 2009

jfriend wrote:

I just gave it a try

Can you try again, John? We didn't experience this at all in testing.

UI Stuff and comments are great, thanks - we'll address that soon

jfriend · April 10, 2009

Andy wrote:

Can you try again, John? We didn't experience this at all in testing.

UI Stuff and comments are great, thanks - we'll address that soon

Working more normally now, though there was clearly a major bug when I first tried it several times in a row.

The "Stop Uploading" button still behaves very weirdly. The item that is currently uploading just disappears from view and is not uploaded. It seems like it should stay in view just like the other items that are not uploaded yet.

Andy · April 10, 2009

jfriend wrote:

Working more normally now, though there was clearly a major bug when I first tried it several times in a row.

The "Stop Uploading" button still behaves very weirdly. The item that is currently uploading just disappears from view and is not uploaded. It seems like it should stay in view just like the other items that are not uploaded yet.

Awesome, thanks. Try it more over the next few days if you would please, throw in a few hundred or even a thousand files - love to get some real strong usage.

RogersDA · April 10, 2009

5 images - first three averaged 2.99 MB/s. The last two throttled down to about 1.3 - 1.2 MB/s. At the end of the last one it seemed really slow to finish the upload. Sorry - no definite times recorded.

Maybe the first three speeds were the result of Comcast's speedboost kicking in at the beginning?

Baldy · April 10, 2009

BradfordBenn wrote:

Howdy-

So I ran the same 27 images through four different uploaders for testing. Here is what I came up with:

45.2 MB - 27 Images
Running under Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.8) Gecko/2009032608 Firefox/3.0.8
Applet = 10 Upload 2:25
Applet = 9 Upload 3:25

Mac Daddy v3.03 2:24

Star *Explorer with Boost 2:25 (running under Fusion with 2GB on XP)

I did these all back to back to back to back to keep it as fair as possible.

Does that help?

Yeah, nice test. Thanks!

Sparky · April 10, 2009

Hope these tests help you out:

Running Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8

Cable internet with Time Warner Roadrunner

About 9AM Eastern

Uploaded two batches (all files in both tests were 0.6-0.7MB in size)

Upload #1 - 167 photos in approx. 10 mins. Speed started at 1.1Mb/s and went to 1.3Mb/s and stayed there

Upload #2 - 173 photos in 11:36 mins. (got my stopwatch out for this one) Speed steady at 1.3Mb/s entire time.

BradfordBenn · April 10, 2009

If you need more testing, let me know. I got more photos to upload. i also can test under Win XP if you need that also.

Andy · April 10, 2009

BradfordBenn wrote:

If you need more testing, let me know. I got more photos to upload. i also can test under Win XP if you need that also.

Please keep using it. We love this uploader for the simplicity, and speed.

Thanks!

Shizam · April 10, 2009

BradfordBenn wrote:

Howdy-

So I ran the same 27 images through four different uploaders for testing. Here is what I came up with:

45.2 MB - 27 Images
Running under Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.8) Gecko/2009032608 Firefox/3.0.8
Applet = 10 Upload 2:25
Applet = 9 Upload 3:25

Mac Daddy v3.03 2:24

Star *Explorer with Boost 2:25 (running under Fusion with 2GB on XP)

I did these all back to back to back to back to keep it as fair as possible.

Does that help?

This is great info having the various uploaders compared with the same amount of data, thanks for laying it out like that.

@jfriend: Once an item reaches 100% that means that its now being verified on the server. So if you click 'stop uploading' it should only terminate any in progress uploads, items that are verifying can't be stopped as they're already on smugmug and the uploader is just waiting for verification that all the bits made it. I am curious as to the other bit you posted that if you hit stop uploading and a file is verifying then that file isn't uploaded, am I understanding that correctly?

Sam

jfriend · April 10, 2009

Shizam wrote:

@jfriend: Once an item reaches 100% that means that its now being verified on the server. So if you click 'stop uploading' it should only terminate any in progress uploads, items that are verifying can't be stopped as they're already on smugmug and the uploader is just waiting for verification that all the bits made it. I am curious as to the other bit you posted that if you hit stop uploading and a file is verifying then that file isn't uploaded, am I understanding that correctly?

Sam

The first time I uploaded, three images immediately went to 100% (instantly in less than a second) and then had the green dots going back and forth (which I think signifies verifying). The images were not actually verifying - they were still uploading - I could tell by my network traffic. I pressed Stop Uploading at that point and nothing happened. The images were still actually uploading. They did eventually finish uploading, though I had no control over it at all.

The second time I uploaded, the images started uploading one at a time. I saw a normal thermometer. When I pressed Stop Uploading with the progress thermometer at 33%, the image did indeed stop uploading and then it disappeared frrom the upload list. It should have stayed in the list since it was not yet uploaded.

Neither behavior was proper for the Stop Uploading button. In the first case, it didn't stop when the images had not yet finished uploading. It just kept uploading. In the second case, it did stop uploading and then it removed the image from the queue (when it should have left it in the queue).

Shizam · April 10, 2009

jfriend wrote:

Neither behavior was proper for the Stop Uploading button. In the first case, it didn't stop when the images had not yet finished uploading. It just kept uploading.

Right, it shouldn't have just went right to 100% even though it was still uploading, in this case there must have been some problem with uploading, like the progressbar was representing reading into memory instead of reading to the httpstream. However the stop button functioned correctly as far as the UI/logic was concerned since it thought it was at 100% it didn't cancel it.

jfriend wrote:

In the second case, it did stop uploading and then it removed the image from the queue (when it should have left it in the queue).

This isn't a bug/problem, just a difference in opinion on how the UI should work. I/we think after its done uploading it should be removed from the queue, its no longer relevant. Where you expect it to stick around and this seems cumbersome to the UI.

Donkey_Kong · April 10, 2009

jfriend wrote:

Initial progress is very bizarre. It jumped to 100% immediately and then just showed the animating green rectangles. The progress thermometers seem busted. I much, much prefer what the previous simple uploader showed because it feels like it's doing something all the time.

Speed indicator never showed anything

Time remaining never showed anything.

I am normally a Star Explorer uploader so I can't really offer any comparison to the previous simple uploader.

I believe this is the same behavior I'm having trouble with and it's caused by delay of the verification coming back to the uploader. See my other post on this at http://dgrin.com/showthread.php?t=127048

The calculation of the verification code is just too slow.

jfriend · April 10, 2009

Shizam wrote:

This isn't a bug/problem, just a difference in opinion on how the UI should work. I/we think after its done uploading it should be removed from the queue, its no longer relevant. Where you expect it to stick around and this seems cumbersome to the UI.

There were four images in the queue. One was uploading. I hit the Stop Uploading button. It stopped uploading the one that was uploading, then removed it from the queue. There were then three images left in the queue.

I would think that when you stop, it should leave all images in the queue that have not uploaded. One can always use the Remove button at that point if they want to get rid of one of the images in the queue.

I thought it was less than desirable behavior to remove an image from the queue that did not upload.

I'm fine with removing it from the queue when it successfully uploads, but not when it doesn't.

jfriend · April 10, 2009

Donkey_Kong wrote:

I believe this is the same behavior I'm having trouble with and it's caused by delay of the verification coming back to the uploader. See my other post on this at http://dgrin.com/showthread.php?t=127048

The calculation of the verification code is just too slow.

No, this is different. This was a UI problem where it immediately zipped to 100% progress even though it was still uploading the bits (I could tell by looking at my network traffic). The long time it spent in this state was merely the time it took to upload, not the verification time.

Shizam · April 10, 2009

jfriend wrote:

No, this is different. This was a UI problem where it immediately zipped to 100% progress even though it was still uploading the bits (I could tell by looking at my network traffic). The long time it spent in this state was merely the time it took to upload, not the verification time.

Right I agree, we're both saying the same thing just different ways

It shouldn't have done that, dunno why it happened to you early on but went away. I was just throwing out a guess as to why it happened which is some buffer (or maybe proxy) read in the data super fast (which tricked the progressbar) and then the data was doled out to the httpconnection from the buffer/proxy.

Shizam · April 10, 2009

Donkey_Kong wrote:

I believe this is the same behavior I'm having trouble with and it's caused by delay of the verification coming back to the uploader. See my other post on this at http://dgrin.com/showthread.php?t=127048

The calculation of the verification code is just too slow.

Kong,
Do you have any packet monitoring software installed on your machine? Are you by any chance using a Mac, I'd suggest HTTPScoop. If you're using a Windows machine then Wireshark is all I know of and its kinda complicated to explain how to use. I suspect what is happening is if you watched your bandwidth you'd see that even after your file gets to 100% that data is still being sent (kind of like what jFriend saw). Or there is some local proxy that you're uploading your data too (really fast) which then sends the data to smugmug (slower) so it seems to you that all the data is sent (100%) but really our server hasn't gotten all the bits yet. That would be harder to diagnose.

Donky_Kong wrote:

I'd hazard to guess it's got something to do with the smugmugs servers. While I don't know the details of how the hash length is calculated I'd think it requires more time to calculate the hash for the larger files. As the numbers of Canon 5d2's increases I'm sure this problem is going to become worse unless something is done.

This isn't the case, I've uploaded 23-24MB sized files from a 1DsIII and the verification time is marginal, like seconds.

Andy · April 10, 2009

Donkey_Kong wrote:

I believe this is the same behavior I'm having trouble with and it's caused by delay of the verification coming back to the uploader. See my other post on this at http://dgrin.com/showthread.php?t=127048

The calculation of the verification code is just too slow.

Not sure that's the problem, but we'll help you in your other thread, thanks!

PBolchover · April 10, 2009

jfriend wrote:

I would think that when you stop, it should leave all images in the queue that have not uploaded. One can always use the Remove button at that point if they want to get rid of one of the images in the queue.

I thought it was less than desirable behavior to remove an image from the queue that did not upload.

I'm with John on this one. If you click "Stop Uploading", then the photo that is currently uploading should remain in the queue. (Photos that have already been uploaded should be removed from the queue...)

BradfordBenn · April 10, 2009

Nice to hear that the style I am using is helpful. I will do some more testing. I will say that my favorite part and something that makes the connection appear more secure is that the plugin is signed by SmugMug. I understand the way the signatures work and it isn't really a large change as it is a licensed plug in. However I do think it is an improvement that was noteworthy.

Since you guys are asking for larger uploads I will give those a whirl in a little while as I am going on a motorcycle RTE tomorrow and will hopefully have lots of pictures to share.

Donkey_Kong · April 11, 2009

Andy wrote:

Not sure that's the problem, but we'll help you in your other thread, thanks!

I'm going to write a bit here then go to the other thread. maybe they need to be combined.

I can tell you it zipped to 100% in no time on the first 3 files. I didn't notice if I still had a bunch of network traffic or not. The old simple uploader did the same thing. Then it sits there, but I needed to take a shower so instead of canceling it I went and did that. When I came back about 20 minutes later it had uploaded the 10 or images.

So I'll go look at the other one when you think about this.

jiri · April 12, 2009

i was just thinking last night to write a post about new uploader I would love to see... ;-)

I tried to upload few photos and MPEG2 video from Chrome 2.0.169.1 (I know I know Chrome is not officially supported)

everything works great + I am getting good speed, the only glitch (not sure if this is by design) is that the upload runs in serial http://jiri.smugmug.com/photos/510821651_TbuoE-L.jpg

btw I love the fact that there is no pop-up window like before

jiri

p.s. just interested... can you share the reason why you use JAVA instead of Adobe FLASH, please. From user view there is really almost no difference. I noticed that Flickr uses Flash.

mmouse · April 12, 2009

Yay for no Flash!

jiri wrote:

p.s. just interested... can you share the reason why you use JAVA instead of Adobe FLASH, please. From user view there is really almost no difference. I noticed that Flickr uses Flash.

I, for one, loudly cheer your use of Java instead of Flash!

As a user, I really do notice the difference. Flash is so leaky with its memory, I've recently blocked it in my primary browser for any website where it's practical (also takes care of flashing ads pretty nicely). Java also seems to run more smoothly than Flash for applications like this one (when I've actually compared them). Plus, it works a lot better on my more 'aged' computers. So, +1 for Java!

Now if only the Smugmug slideshow were Java... Or maybe Java plus the new HTML 5 animation capabilities! That would be especially sweet for iPhone...

HoyBoy · April 12, 2009

I tried the new uploader on 1 photo so far. It worked flawlessly and was quite fast. It started about 3 mbs then got up to 4 mbs and stayed there for the duration.

Nice job!

EDIT: Just uploaded 22 more. It started out at about 5 mbs for the first 2 or 3 images then slowed to about 2 mbs for the duration. I suspect this is a comcast issue. These images varied from 3.5-6 mb each. The overall upload took about 6 minutes. The verification was very fast. It occurred simultaneously with the next image's upload and was done when the next images was about 18% uploaded.

jiri · April 12, 2009

mmouse wrote:

Flash is so leaky with its memory, I've recently blocked it in my primary browser for any website where it's practical (also takes care of flashing ads pretty nicely). Java also seems to run more smoothly than Flash for applications

I'm really not java nor flash expert and it's very hard for me to compare them. They are both very different - flash has much better visual, java is very universal. I think file upload can be done in both (flickr is not small site and they use flash, facebook uses java - it almost looks like that facebook decreases image resolution locally in the java app instead of doing it on the servers). It would be interested to see why smugmug picked java. I'm guessing that file upload is more stable as a java app.

Although I totally agree that Adobe Flash has lot of issues (a lot of security related - see SIR volume 6), I really do not have almost any issues in my web browser (Flash 10 in Chrome2). What browser do you use?

jiri

dwterry · April 12, 2009

Upload to multiple folders?
I've been a long time user of smugbrowser... a couple of the things I really like about smugbrowser are: 1) I can upload to multiple folders at once, 2) I can add to the queue for those folders, or any other folder, at any time during the upload, without interrupting the upload.

Does your new uploader allow for any of that?

Andy · April 12, 2009

dwterry wrote:

I've been a long time user of smugbrowser... a couple of the things I really like about smugbrowser are: 1) I can upload to multiple folders at once, 2) I can add to the queue for those folders, or any other folder, at any time during the upload, without interrupting the upload.

Does your new uploader allow for any of that?

No, it's just a really fast robust simple uploader

jfriend · April 12, 2009

So, there are a lot of basic uploaders. I'm glad that Smugmug now has their own code for one. But, what really separates an OK or not so great uploader from a great uploader is how it handles adverse conditions. Have you at Smugmug or anyone else tested any of the following:

What happens when your internet connection loses connectivity to Smugmug part way through an upload?
What happens where there is packet loss?
What happens when there are DNS issues establishing the initial connection?
What happens when you try to upload a damaged JPEG?
What happens when you try to upload a JPEG that's too large?
What happens when you try to upload a JPEG that's too many pixels?
What happens when you try to upload a filetype that Smugmug won't accept?
What happens when your computer wants to go to sleep in the middle of an upload (like a laptop on battery)?
What happens when there's a momentary network glitch, but then the ability to connect is restored?

What I want is an uploader that:

NEVER deceives me about what it did and didn't upload successfully.
Never leaves me to try to figure out what did and didn't upload successfully by looking at logs, comparing galleries to file listings, etc... It should all just be there in the UI.
Tells me clearly what went wrong anytime something can't be uploaded (too many pixels, file too large, corrupt JPEG, wrong filetype, etc...)
Anything that it did not upload successfully, it leaves in the queue to let me know and to let me try again.
That, when it experiences some sort of network glitch, it tries to resume again automatically when conditions seem to have improved so I don't leave an unattended upload for several hours only to come back and find it only went part way then just stopped and waited for me to start it again.
Goes fast, but doesn't so saturate my network connection that I can't do anything else on the connection.
Allows me to preserve an upload order.

wellman · April 12, 2009

Vista, FF 3.0.8. Fast and speedy so far!

Can youz test a new uploader?

Comments