It seems to work well overall. A couple of thoughts, primarily UI related.
The time remaining updates several times per second and is not really useful because of this. I think it should be calculated on a historical average (e.g. the last 10 seconds) and not the instantaneous speed.
It would be nice to show bytes total, bytes uploaded and bytes remaining somewhere near the top.
The "Overall Progress" counter seems backwards to me. It currently counts down and based on the name, I thought it should count up. Perhaps change name to "Files Remaining" or make it count up.
When a file finishes and the next one starts, there's a moving bar. Is this verifying the upload or something? A text description here would be nice as well as a status when complete. for example, have it say "validating" and then display "success" or something for a few seconds before disappearing. If a file fails, what happens? does it remain there so we can re-attempt an upload? If it's out of spec, does it tell us why (e.g. too many pixels, wrong format, etc).
When the whole upload finishes, I'd like to see a status showing total time, bytes, average upload speed.
-Scott
UPDATE: I uploaded 70 files and only 15 showed up. Only the 15 files are shown in my upload log and nothing is shown in the error log. This is obviously not good... Time to go back to the original...
I'm always curious about this. How come you don't have the gallery sorting set in gallery customizing?
erm...
This should be a default or available on the new gallery setup.
Here's a little look at my log:
Img 4499 was uploaded a full 4 minutes after 4497, yet 4497 appears ahead of 4499 (I have not fiddled with the image order). This happens regularly. I've been trying to upload one image at a time to try and preserve the upload order, but the uploader seems to have a will of its own. Historically, Old Faithful preserved the upload order - it now seems to reverse the order if you upload several images at once.
UPDATE: I uploaded 70 files and only 15 showed up. Only the 15 files are shown in my upload log and nothing is shown in the error log. This is obviously not good... Time to go back to the original...
This should be a default or available on the new gallery setup.
Here's a little look at my log:
Img 4499 was uploaded a full 4 minutes after 4497, yet 4497 appears ahead of 4499 (I have not fiddled with the image order). This happens regularly. I've been trying to upload one image at a time to try and preserve the upload order, but the uploader seems to have a will of its own. Historically, Old Faithful preserved the upload order - it now seems to reverse the order if you upload several images at once.
Hi Al,
Photos are not processed in the very same order you uploaded them. For performance reasons, they might process out of order as we not have just one processing server. If you have your gallery set to 'autosort none', they'll show in the order they processed.
The upload log only shows you the order they were uploaded in - not the order they processed. If you want them to show in the order you uploaded them, you would have to set autosort to 'by date uploaded'.
We're trying to improve the explanations on the site on how these things work.
I'm with John on this one. If you click "Stop Uploading", then the photo that is currently uploading should remain in the queue. (Photos that have already been uploaded should be removed from the queue...)
This is currently how it works there just seems to be some confusion on the status of uploaded images. There are 3 states:
-Uploading: Image data is being sent from your computer to the server, the progress bar indicates the percent done.
-Uploaded and _verifying_: The image is already uploaded, the server is just confirming the correct data was sent, at this point the progress bar just says 100%
-Uploaded: The image is uploaded and verified, it is removed from the queue.
Clicking stop can only halt an uploading image, if 100% of the data is already on the server (either of the latter two cases) 'stopping' the upload is meaningless.
p.s. just interested... can you share the reason why you use JAVA instead of Adobe FLASH, please. From user view there is really almost no difference. I noticed that Flickr uses Flash.
There is one massively huge important thing for an uploader that java can do that flash simply cannot, and that is drag-and-drop access to the filesystem and other applications. With java you can drag one file, several files or whole folders onto the application and it can read those files, you can also drag images/files from other applications (like Lightroom or iPhoto) and it can read those images. Flash cannot do this due to sandboxing. I hate that fact and as a flash developer really really really wish they'd do something about that.
* What happens when your internet connection loses connectivity to Smugmug part way through an upload?
Currently it retries and eventually fails if you lose connectivity for too long.
* What happens where there is packet loss?
The upload takes longer
* What happens when there are DNS issues establishing the initial connection?
Not sure what you mean here, there is a timeout to make a connection and if that fails then the upload fails.
* What happens when you try to upload a damaged JPEG?
Currently silent failure.
* What happens when you try to upload a JPEG that's too large?
You can't, I prevent you from doing that based on account type. However, if you have smugvault turned on and the file is greater then the accepted photo/video size then I allow you to upload it as a vault item.
* What happens when you try to upload a JPEG that's too many pixels?
I also want to prevent you from doing this but at the moment, in java, its not as easy as just asking the file reference what the dimensions of the image are, the only 'out of the box' way to get this in java is read the whole file into memory, which I'm not going to do. I've found some suggestions on reading the file in as a stream and looking for the dimension header information which I'll try in the next iteration.
* What happens when you try to upload a filetype that Smugmug won't accept?
That depends. If you have smugvault OFF then it just skips the file, if you have smugvault ON then it adds the file to the list and marks that its a smugvault item.
* What happens when your computer wants to go to sleep in the middle of an upload (like a laptop on battery)?
Depends on hard/soft sleep I think you mean a hard sleep, in this case all network traffic is terminated, not sure how Java handles restarting network connections from sleep. Will have to look into that.
* What happens when there's a momentary network glitch, but then the ability to connect is restored?
This is the same as the 1st question, as long as its less then the timeout the upload will resume.
The goal for this first rollout was just a simple uploader. The next iteration will focus mostly on feedback (which you mention in your wishlist), as well as being able to resume from failure.
-Feedback on upload status: ie relaying to the user the result from the upload, if its good then say 'yey' just before removing the item from the list, if its bad say 'boo' and leave it on the list with more information for the user in case they can do something about it.
-Feedback on what files have been ignored after a batch add (from drag and drop or filechooser), so if they drag 1000 items but only 800 show on the list display a notice that can be clicked to show what files were ignored and why.
-If the connection is totally lost (sleep, power loss, etc) the ability to resume from the last uploaded item _this session_.
Thanks for the list of requests, it'll help make sure more bases are covered in the next release.
That's because I deleted the 15 and re-uploaded all 70 with the simple uploader.
If you look at my log, you'll see there are 15 deleted photos at the bottom, followed by 70 live pictures. Where'd the other 55 go? I sat there and watched them in the uploader, but they were nowhere to be found.
This is currently how it works there just seems to be some confusion on the status of uploaded images. There are 3 states:
-Uploading: Image data is being sent from your computer to the server, the progress bar indicates the percent done.
-Uploaded and _verifying_: The image is already uploaded, the server is just confirming the correct data was sent, at this point the progress bar just says 100%
-Uploaded: The image is uploaded and verified, it is removed from the queue.
Clicking stop can only halt an uploading image, if 100% of the data is already on the server (either of the latter two cases) 'stopping' the upload is meaningless.
Sam
Apparently you didn't see what I wrote earlier. This is not what I experienced. I had an image that was uploading normally. The progress meter showed 40% uploaded. I hit stop uploading. It stopped uploading and then promptly removed the image that did NOT upload from the screen. It was gone from the screen. When I checked the gallery, it was not there either.
1. Does not work with the generic Linux Java (IcedTea).
2. Using Sun Java 1.0.6.11 it works, but drag & drop works intermittently
3. The "Add Files" button is very clunky under Kubuntu:
* I cannot select the path in the file dialog
* I have to enter the path, folder for folder in the Selection box
4. As per earlier discussion, upload order seems to be random.
1. Does not work with the generic Linux Java (IcedTea).
2. Using Sun Java 1.0.6.11 it works, but drag & drop works intermittently
3. The "Add Files" button is very clunky under Kubuntu:
* I cannot select the path in the file dialog
* I have to enter the path, folder for folder in the Selection box
4. As per earlier discussion, upload order seems to be random.
Thanks Al - our fully supported environments (what we test on) are:
Win XP & Vista, IE6, IE7, IE8, FF2&3, Safari 3
Mac OS X, Safari 2&3, FF2&3
There is one massively huge important thing for an uploader that java can do that flash simply cannot, and that is drag-and-drop access to the filesystem and other applications. With java you can drag one file, several files or whole folders onto the application and it can read those files, you can also drag images/files from other applications (like Lightroom or iPhoto) and it can read those images. Flash cannot do this due to sandboxing. I hate that fact and as a flash developer really really really wish they'd do something about that.
Sam
I'm really not FLASH nor JAVA developer, from very general point of view I'm a bit surprised about JAVA because Smugmug already uses FLASH and it would maybe make sense to keep the same technology instead of using additional plug-ins. Is the drag-drop the only reason why smugmug picked java or is there any "pre-upload-work" they plan to do with photos in JAVA plug-in locally before the upload?
I'm really not FLASH nor JAVA developer, from very general point of view I'm a bit surprised about JAVA because Smugmug already uses FLASH and it would maybe make sense to keep the same technology instead of using additional plug-ins. Is the drag-drop the only reason why smugmug picked java or is there any "pre-upload-work" they plan to do with photos in JAVA plug-in locally before the upload?
thank you
jiri
Currently Drag & Drop is the only reason java is used, if all things were equal I'd rather do it in Flash but Adobe made their bed on sandboxing. Also, long term, there are other things I'd like to do with the uploader w/respect to the filesystem that Flash wouldn't allow but Java will. I know I could do it in Air but java has a web-app/desktop-app 'thing' where they can both be the exact same app you just drag it from the browser to the desktop.
No pre-upload-work is done, that would be a "Bad Thing", the server takes care of all the processing so more specialized software and faster processors can do their thing.
@jfriend
I see, I didn't get that at first. Are you able to replciate that?
@BigAl
I'll install Ubuntu in a VM on my mac and try it out, the UI issues you're having could just be due to how I've set the 'Look and Feel', I'll see if stickign a 'if linux use steel' works better.
1. Does not work with the generic Linux Java (IcedTea).
2. Using Sun Java 1.0.6.11 it works, but drag & drop works intermittently
3. The "Add Files" button is very clunky under Kubuntu:
* I cannot select the path in the file dialog
* I have to enter the path, folder for folder in the Selection box
4. As per earlier discussion, upload order seems to be random.
So I did have to install a real JDE (Dunno what IcedTea is but it doesn't work), I also was having issues with D&D but I think thats related to trust/security. I'm not sure what you mean about #3 though, here is a screencap of what I see:
On the plus side the new uploader at least runs in Konqueror, the other 'Simple' uploader throws Java errors
Sam
I'm not sure what you mean about #3 though, here is a screencap of what I see:
In your pic, if I clicked on the folder 'Pictures' nothing would happen. I physically had to type 'Pictures' in the Filename box (It looked a bit different on my work machine and I don't have Java installed on this machine - will check and let you know later).
The uploader window looks a bit different on my machine:
If I click on the "Add files" button (which does not show in your version), I get the dialog shown in the pic. Clicking on any of the folders does nothing. I have to type it in the selection box.
The uploader window looks a bit different on my machine:
If I click on the "Add files" button (which does not show in your version), I get the dialog shown in the pic. Clicking on any of the folders does nothing. I have to type it in the selection box.
Good news BigAl, you're using the wrong uploader Thats the olde sauce, in order to see the new uploader add &applet=10 to the url that you're at. On the plus side I also couldn't get the one that you're trying to work in kubuntu (applet=9 is what you're trying), but the new one did work.
Just uploaded a batch of 144 pics, totaling 136 MB. Speed and reliability were good - the transfer rate stayed pegged around 800 mb/s.
One question... Java.exe had my machine (a C2D HP laptop) pegged at about 52% processor through the entire upload. My thought was, "Really? 52% processor for uploading?" Any explanation for the high numbers? The "Simple" uploader uses almost no CPU.
This is important in that I tend to get an upload batch started while I'm running Lightroom doing other things on other photos. If my uploader starts eating into my available CPU cycles for crunching pixels, I will be sad.
Would love to hear from more folks running the new uploader on Windows, or Mac, using Firefox, Safari and IE6, 7, 8
Give it a try
I put the new uploader though it's paces this week in Firefox. Many times running 3-5 instances in different tabs. Not sure exactly how many pictures but it was a significant amount. 5 years of old digital photos from my P&S days.
Did not have any problems.
At work using a high speed connection it worked out really good. At home using a satellite connection my speeds never did seem to top out on the uploading. This may have been a connection problem or the way the new uploader works. Lots of latency on a satellite connection.
Just uploaded a batch of 144 pics, totaling 136 MB. Speed and reliability were good - the transfer rate stayed pegged around 800 mb/s.
One question... Java.exe had my machine (a C2D HP laptop) pegged at about 52% processor through the entire upload. My thought was, "Really? 52% processor for uploading?" Any explanation for the high numbers? The "Simple" uploader uses almost no CPU.
This is important in that I tend to get an upload batch started while I'm running Lightroom doing other things on other photos. If my uploader starts eating into my available CPU cycles for crunching pixels, I will be sad.
Thanks!
Thanks for catching this wellman, I reproduced and fixed the bug, its now down to around 2.5-3% usage for me.
I also reproduced and fixed your bug jfriend. Both fixes are in my sandbox pending approval.
I put the new uploader though it's paces this week in Firefox. Many times running 3-5 instances in different tabs. Not sure exactly how many pictures but it was a significant amount. 5 years of old digital photos from my P&S days.
Did not have any problems.
At work using a high speed connection it worked out really good. At home using a satellite connection my speeds never did seem to top out on the uploading. This may have been a connection problem or the way the new uploader works. Lots of latency on a satellite connection.
Good to hear. And you are correct, a high latency situation will have a noticeable negative impact on upload speeds, I tested high latency situations and it seems to affect both the new and simple uploader similarly.
Skip duplicates?
Sorry if this is off-topic, but I have a feature request for the new uploader.
I use the current simple upploader almost exclusively, and my only complaint about it is that it seems to upload the photos in an arbitrary order. This makes it hard to resume an upload if the browser or computer crashes. I've had this happen when uploading hundreds of files, and I either delete them all from the gallery and start over, or painstakingly look through which photos were uploaded and which weren't.
Could the new uploader skip files that already exist in the gallery? Lightroom does this for importing. It could be done by communicating with the server to compare EXIF data (for example) and skip the file. Maybe make it an option in the uploader?
Filename display
I tried the new uploader with just a few files and had no trouble.
I did notice that filenames are truncated using an ellipsis. This tends to cut off the numeric suffix appended by many programs to exported image files. For example, I had: EEF_SteamTeam_2009-002.jpg
Could this be shortened by putting the ellipsis in the middle of the file, instead of the end, like: EEF_St...-002.jpg
I was trying out the new uploader and noticed that it seems to keep the file handles open after upload? What I mean is I can't delete, move or rename uploaded files until I close the browser I used to upload.
This happens for me on Firefox 3.0.8 and IE8, Java 1.6.0_13, Vista 64.
Comments
Amen to that. Not even Old Faithful does that anymore :cry
Bugs
Spiders
Flowers
- The time remaining updates several times per second and is not really useful because of this. I think it should be calculated on a historical average (e.g. the last 10 seconds) and not the instantaneous speed.
- It would be nice to show bytes total, bytes uploaded and bytes remaining somewhere near the top.
- The "Overall Progress" counter seems backwards to me. It currently counts down and based on the name, I thought it should count up. Perhaps change name to "Files Remaining" or make it count up.
- When a file finishes and the next one starts, there's a moving bar. Is this verifying the upload or something? A text description here would be nice as well as a status when complete. for example, have it say "validating" and then display "success" or something for a few seconds before disappearing. If a file fails, what happens? does it remain there so we can re-attempt an upload? If it's out of spec, does it tell us why (e.g. too many pixels, wrong format, etc).
- When the whole upload finishes, I'd like to see a status showing total time, bytes, average upload speed.
-ScottUPDATE: I uploaded 70 files and only 15 showed up. Only the 15 files are shown in my upload log and nothing is shown in the error log. This is obviously not good... Time to go back to the original...
scwalter.smugmug.com
Portfolio • Workshops • Facebook • Twitter
This should be a default or available on the new gallery setup.
Here's a little look at my log:
Img 4499 was uploaded a full 4 minutes after 4497, yet 4497 appears ahead of 4499 (I have not fiddled with the image order). This happens regularly. I've been trying to upload one image at a time to try and preserve the upload order, but the uploader seems to have a will of its own. Historically, Old Faithful preserved the upload order - it now seems to reverse the order if you upload several images at once.
Bugs
Spiders
Flowers
I see them all here:
http://www.smugmug.com/homepage/uploadlog.mg?type=all&scope=
and here in your gallery, 70 files:
http://www.smugmug.com/gallery/7888052_YgoUz/5/511489642_WKGa6#511489642_WKGa6
Portfolio • Workshops • Facebook • Twitter
Photos are not processed in the very same order you uploaded them. For performance reasons, they might process out of order as we not have just one processing server. If you have your gallery set to 'autosort none', they'll show in the order they processed.
The upload log only shows you the order they were uploaded in - not the order they processed. If you want them to show in the order you uploaded them, you would have to set autosort to 'by date uploaded'.
We're trying to improve the explanations on the site on how these things work.
Sebastian
SmugMug Support Hero
This is currently how it works there just seems to be some confusion on the status of uploaded images. There are 3 states:
-Uploading: Image data is being sent from your computer to the server, the progress bar indicates the percent done.
-Uploaded and _verifying_: The image is already uploaded, the server is just confirming the correct data was sent, at this point the progress bar just says 100%
-Uploaded: The image is uploaded and verified, it is removed from the queue.
Clicking stop can only halt an uploading image, if 100% of the data is already on the server (either of the latter two cases) 'stopping' the upload is meaningless.
Sam
There is one massively huge important thing for an uploader that java can do that flash simply cannot, and that is drag-and-drop access to the filesystem and other applications. With java you can drag one file, several files or whole folders onto the application and it can read those files, you can also drag images/files from other applications (like Lightroom or iPhoto) and it can read those images. Flash cannot do this due to sandboxing. I hate that fact and as a flash developer really really really wish they'd do something about that.
Sam
The goal for this first rollout was just a simple uploader. The next iteration will focus mostly on feedback (which you mention in your wishlist), as well as being able to resume from failure.
-Feedback on upload status: ie relaying to the user the result from the upload, if its good then say 'yey' just before removing the item from the list, if its bad say 'boo' and leave it on the list with more information for the user in case they can do something about it.
-Feedback on what files have been ignored after a batch add (from drag and drop or filechooser), so if they drag 1000 items but only 800 show on the list display a notice that can be clicked to show what files were ignored and why.
-If the connection is totally lost (sleep, power loss, etc) the ability to resume from the last uploaded item _this session_.
Thanks for the list of requests, it'll help make sure more bases are covered in the next release.
That's because I deleted the 15 and re-uploaded all 70 with the simple uploader.
If you look at my log, you'll see there are 15 deleted photos at the bottom, followed by 70 live pictures. Where'd the other 55 go? I sat there and watched them in the uploader, but they were nowhere to be found.
-Scott
scwalter.smugmug.com
Homepage • Popular
JFriend's javascript customizations • Secrets for getting fast answers on Dgrin
Always include a link to your site when posting a question
1. Does not work with the generic Linux Java (IcedTea).
2. Using Sun Java 1.0.6.11 it works, but drag & drop works intermittently
3. The "Add Files" button is very clunky under Kubuntu:
* I cannot select the path in the file dialog
* I have to enter the path, folder for folder in the Selection box
4. As per earlier discussion, upload order seems to be random.
Bugs
Spiders
Flowers
Win XP & Vista, IE6, IE7, IE8, FF2&3, Safari 3
Mac OS X, Safari 2&3, FF2&3
Portfolio • Workshops • Facebook • Twitter
I'm really not FLASH nor JAVA developer, from very general point of view I'm a bit surprised about JAVA because Smugmug already uses FLASH and it would maybe make sense to keep the same technology instead of using additional plug-ins. Is the drag-drop the only reason why smugmug picked java or is there any "pre-upload-work" they plan to do with photos in JAVA plug-in locally before the upload?
thank you
jiri
Currently Drag & Drop is the only reason java is used, if all things were equal I'd rather do it in Flash but Adobe made their bed on sandboxing. Also, long term, there are other things I'd like to do with the uploader w/respect to the filesystem that Flash wouldn't allow but Java will. I know I could do it in Air but java has a web-app/desktop-app 'thing' where they can both be the exact same app you just drag it from the browser to the desktop.
No pre-upload-work is done, that would be a "Bad Thing", the server takes care of all the processing so more specialized software and faster processors can do their thing.
@jfriend
I see, I didn't get that at first. Are you able to replciate that?
@BigAl
I'll install Ubuntu in a VM on my mac and try it out, the UI issues you're having could just be due to how I've set the 'Look and Feel', I'll see if stickign a 'if linux use steel' works better.
jiri
So I did have to install a real JDE (Dunno what IcedTea is but it doesn't work), I also was having issues with D&D but I think thats related to trust/security. I'm not sure what you mean about #3 though, here is a screencap of what I see:
On the plus side the new uploader at least runs in Konqueror, the other 'Simple' uploader throws Java errors
Sam
In your pic, if I clicked on the folder 'Pictures' nothing would happen. I physically had to type 'Pictures' in the Filename box (It looked a bit different on my work machine and I don't have Java installed on this machine - will check and let you know later).
Bugs
Spiders
Flowers
If I click on the "Add files" button (which does not show in your version), I get the dialog shown in the pic. Clicking on any of the folders does nothing. I have to type it in the selection box.
Bugs
Spiders
Flowers
Give it a try
Portfolio • Workshops • Facebook • Twitter
Good news BigAl, you're using the wrong uploader
Sam
I reproduced it the first time I tried. Here are the steps:
- Select 6 photos for upload
- Press the Upload! button
- Wait for the first image to get to about 20%.
- Hit the Stop Uploading button
- The first iimage is removed from the queue and not uploaded.
It does this everytime I try so it should not be hard to reproduce.Homepage • Popular
JFriend's javascript customizations • Secrets for getting fast answers on Dgrin
Always include a link to your site when posting a question
Just uploaded a batch of 144 pics, totaling 136 MB. Speed and reliability were good - the transfer rate stayed pegged around 800 mb/s.
One question... Java.exe had my machine (a C2D HP laptop) pegged at about 52% processor through the entire upload. My thought was, "Really? 52% processor for uploading?" Any explanation for the high numbers? The "Simple" uploader uses almost no CPU.
This is important in that I tend to get an upload batch started while I'm running Lightroom doing other things on other photos. If my uploader starts eating into my available CPU cycles for crunching pixels, I will be sad.
Thanks!
Swim for Them | WellmanHouse.net | AlbumFetcher | SmugShowBuilder
I put the new uploader though it's paces this week in Firefox. Many times running 3-5 instances in different tabs. Not sure exactly how many pictures but it was a significant amount. 5 years of old digital photos from my P&S days.
Did not have any problems.
At work using a high speed connection it worked out really good. At home using a satellite connection my speeds never did seem to top out on the uploading. This may have been a connection problem or the way the new uploader works. Lots of latency on a satellite connection.
Thanks for catching this wellman, I reproduced and fixed the bug, its now down to around 2.5-3% usage for me.
I also reproduced and fixed your bug jfriend. Both fixes are in my sandbox pending approval.
Sam
Good to hear. And you are correct, a high latency situation will have a noticeable negative impact on upload speeds, I tested high latency situations and it seems to affect both the new and simple uploader similarly.
Sam
No problem. Thank my laptop for having a fan that sounds like a jet engine. :giggle
Swim for Them | WellmanHouse.net | AlbumFetcher | SmugShowBuilder
Sorry if this is off-topic, but I have a feature request for the new uploader.
I use the current simple upploader almost exclusively, and my only complaint about it is that it seems to upload the photos in an arbitrary order. This makes it hard to resume an upload if the browser or computer crashes. I've had this happen when uploading hundreds of files, and I either delete them all from the gallery and start over, or painstakingly look through which photos were uploaded and which weren't.
Could the new uploader skip files that already exist in the gallery? Lightroom does this for importing. It could be done by communicating with the server to compare EXIF data (for example) and skip the file. Maybe make it an option in the uploader?
Thank you!
I tried the new uploader with just a few files and had no trouble.
I did notice that filenames are truncated using an ellipsis. This tends to cut off the numeric suffix appended by many programs to exported image files. For example, I had: EEF_SteamTeam_2009-002.jpg
Could this be shortened by putting the ellipsis in the middle of the file, instead of the end, like: EEF_St...-002.jpg
Nice work!
This happens for me on Firefox 3.0.8 and IE8, Java 1.6.0_13, Vista 64.