SmugLoader - manage your SmugMug photos and videos

udy

Hello everybody,

I want to announce the availability of the alpha build of SmugLoader v0.5, an application that helps in managing your SmugMug photos, and videos. Enjoy it, if possible.

It is an open source project that aims to be what Star Explorer is, but free. You can find more information at:

sl.bulcu.ro

Please read carefully the information (features, limitations, issues) at the link above.

The download page is at Kenai.

You can put any question you might have here, in this thread.

Have a nice weekend.

udy

The 3rd alpha build of SmugLoader is available at the same location. The second wasn't announced because it didn't live to long.

I'm still hoping that somebody will start to test it, because it needs to be tested, and give me some feedback, please. :

Have fun,

BeachBill

I downloaded the 4th alpha and have an upload running right now.

The first bit of feedback I have is that a "pause/resume" button would be nice. If you hit pause, it would finish uploading the in progress files but not upload anything new until you hit resume.

udy

BeachBill wrote:

I downloaded the 4th alpha and have an upload running right now.

The first bit of feedback I have is that a "pause/resume" button would be nice. If you hit pause, it would finish uploading the in progress files but not upload anything new until you hit resume.

I want to thank you for taking time to test it, and let you know that I greatly appreciate any feedback.

Your idea is interesting, and I will most likely implement it for the next alpha :-), although I can't see fully yet the importance of it. It would be nice to have support from SmugMug for resumable uploads though.

Can you elaborate on the idea, and some scenario, please?

Thanks again,

BeachBill

I pointed it at a directory on my computer that holds 2,369 images in 113 folders (9GB), let it create the categories/subcatergories then started the upload (over my wifi connection) I was streaming a movie (via wifi) on my TV while my son was working on the Internet via a third wifi connected computer. Needless to say, once the SmugLoader upload started the movie streaming slowed way down. This is where I could use a "pause" button... they resume the upload when I'm done watching the movie. ;-)

udy

BeachBill wrote:

I pointed it at a directory on my computer that holds 2,369 images in 113 folders (9GB), let it create the categories/subcatergories then started the upload (over my wifi connection) I was streaming a movie (via wifi) on my TV while my son was working on the Internet via a third wifi connected computer. Needless to say, once the SmugLoader upload started the movie streaming slowed way down. This is where I could use a "pause" button... they resume the upload when I'm done watching the movie. ;-)

I see now :-)

I have every reason to implement it.

A workaround for the current version would be to close the application. You will stop the 7 ongoing uploads (they will not be on SM), but upon the next startup, SmugLoader should display the files that were not uploaded, including those 7 that were uploading. That happens, because there is an upload status saved in a database.

Thanks,

BeachBill

udy wrote:

A workaround for the current version would be to close the application. You will stop the 7 ongoing uploads (they will not be on SM), but upon the next startup, SmugLoader should display the files that were not uploaded, including those 7 that were uploading. That happens, because there is an upload status saved in a database.

I was wondering what may happen if I tried closing it, but I just decided to let it go...

Another thing I just thought of (it's still doing a very good job of uploading my images, only one error so far)...

How about adding a status at the bottom of the screen that says either "## of ## files remaining" (counting down to zero) or "## of ## files uploaded".

udy

BeachBill wrote:

I was wondering what may happen if I tried closing it, but I just decided to let it go...

Another thing I just thought of (it's still doing a very good job of uploading my images, only one error so far)...

How about adding a status at the bottom of the screen that says either "## of ## files remaining" (counting down to zero) or "## of ## files uploaded".

I don't know if you read the application's page (see the first entry in this thread), but there are plans for a paged table, which should give the user the ability to see all the photos in the queue, 50 (or 100) per page.

Your idea is a bonus for the paged table, and I will maybe implement it before that.

Regarding the error, it could be some temporary connection issue. But that failed upload will be there at the next startup of the application. Have you noticed the disabled "Retry" button? It is there for retrying exactly in such situations, but the feature is in plans.

Thanks for your feedback,

Andy

Does / will it work on Macs?

udy

Andy wrote:

Does / will it work on Macs?

It's a java application. I don't know what's the status of the java platform on MacOS, but the recommended version is 1.6, because there is at least one feature not present in version 1.5.

I tested it on Ubuntu Linux (I developed the application for a while using Ubuntu ), and it worked.

Please give it a try if there is no trouble. :-)

xcitant

Upload Queue
Looking good, downloads working great. Is there a way to modify the upload queue (delete files from the queue)?

udy

xcitant wrote:

Looking good, downloads working great. Is there a way to modify the upload queue (delete files from the queue)?

I'm glad you appreciate it.

There will be a way, but first I have to finish the paged table I wrote about in a previous post in this thread.

For now, if you could just organize the collection externally somehow. If not, just bear with me, please :-P

BeachBill

I let the upload run all night and the attached image shows what I was left with. The two that say "progressing" weren't really doing anything. They were just stuck. I closed the application and ran it again, all nine failed images were queued up and I was able to successfully upload them on the second pass. So the error recovery works good.

udy

BeachBill wrote:

I let the upload run all night and the attached image shows what I was left with. The two that say "progressing" weren't really doing anything. They were just stuck. I closed the application and ran it again, all nine failed images were queued up and I was able to successfully upload them on the second pass. So the error recovery works good.

I'm happy that the recovery worked.

I have to investigate the progressing stuck issue.

I'm now working on a list of issues/features to work on. The most important would be the paged table navigation of upload queue.

udy

The 5th alpha of SmugLoader v0.5 is available, and brings with it the following enhancements:

- upload status notification (Total, Uploaded, Failed, Remaining)
- added the posibility to change the albums' visibility (unlisted/public) before creating them in batch upload; the previous behaviour was that the albums were created unlisted. By default, all albums are created unlisted, but the user can now change individual album's visibility with contextual menu. The user can change the default to 'public'.

I hope that the enhancements will help.

Other enhancements are in the pipe.:D

BeachBill

udy wrote:

- upload status notification (Total, Uploaded, Failed, Remaining)

I downloaded the new version. Haven't used it yet, but the status area is SWEET! The other enhancements sound good too.

I'm thinking about firing up a complete download of my site before going to bed tonight...

udy

SmugLoader has reached another alpha, the 6th. :ivar

The new features, and improvements are:

• The upload button has Pause, and Continue abilities (thanks for the push, Bill)
• The delete Category action now offers the possibility to delete all its albums, and subcategories. That way, when the user deletes a category, the category's albums and subcategories don't end up in the Other category. The downside is that the user loses the pictures. So take care! (This feature will be implemented for subcategories too)
• Some UI changes with the tree components.

And there's more in the pipe.

Enjoy!

udy

Andy wrote:

Does / will it work on Macs?

Please give me some feedback, if you'll try it.

Thanks

udy

Following Nick's feedback, a new alpha build is available, having to do only with UI annoyances:

• removed some UI elements that had no action;
• added missing action to a Cancel button;
• added the in use SmugMug API version to the About window.

A build that looks smoother.

udy

The 7th alpha archive was incomplete.

Sorry for that. The new archive will be available in a few minutes.

udy

Done!

Sorry again!

SamirD

I just found this thread, and I think your application has potential to be my uploader and maybe even downloader of choice. I'm testing alpha 7 right now.

The only issue I've run into right now is that I can't run more than one instance of SL because the db can't be shared. I have three 25m/5m connections that usually takes 12 (!) connections to max out. I'd have to run two sessions of SL to do this. Or, if it's easy enough, I guess you could expand the maximum 7 threads to 12.

So far, I like how fast it loads the album list, and it seems like it will be great for downloading. But I haven't been able to get upload to work. When I add media to an empty unlisted album, it just sits there.

The download button took a while to think before it executed, but once it started, stand back! You're currently racing against Albumfetcher, which has two sessions open downloading two albums. AF has been running for over 10 minutes. Let's see if you can finish before AF. If you do, you're my new downloader.

SamirD

On thing I've already noticed is that the multiple threads don't utilize all my Internet connections. All of my Internet connections are connected to a Cisco rv016, which does load balancing on the outgoing connections from the lan. Most other uploaders (like the default SM one) seem to initiate an upload connection for each file and then terminate that connection and open a new one for a new file, etc. This allows the router to choose the least utilized Internet connection and by doing so, a lot of concurrent connections will eventually utilize all available bandwidth.

For some reason, I've noticed that SL is only using one Internet connection. All the concurrent sessions must be on that one pipe. It is fully utilizing that one pipe since only 4 connections max it out, but there's another 25m/5m pipe just sitting idle. I'd really like to see it able to utilize all of these pipes simultaneously.

Oh, and this is for downloading, I'm going to try uploading again once the download is done. I may have had to let it sit there for a while like the download before it starts.

udy

SamirD wrote:

I just found this thread, and I think your application has potential to be my uploader and maybe even downloader of choice. I'm testing alpha 7 right now.

The only issue I've run into right now is that I can't run more than one instance of SL because the db can't be shared. I have three 25m/5m connections that usually takes 12 (!) connections to max out. I'd have to run two sessions of SL to do this. Or, if it's easy enough, I guess you could expand the maximum 7 threads to 12.

So far, I like how fast it loads the album list, and it seems like it will be great for downloading. But I haven't been able to get upload to work. When I add media to an empty unlisted album, it just sits there.

The download button took a while to think before it executed, but once it started, stand back! You're currently racing against Albumfetcher, which has two sessions open downloading two albums. AF has been running for over 10 minutes. Let's see if you can finish before AF. If you do, you're my new downloader.

Hi Samir and thanks for the feedback. Modifying the number of threads, on my part is very easy. I just have to modify a max constant I display in the settings. I was a bit pessimistic with the max number, I guess. I will increase the number to 15. You then give me some feedback, please.

udy

SamirD wrote:

On thing I've already noticed is that the multiple threads don't utilize all my Internet connections. All of my Internet connections are connected to a Cisco rv016, which does load balancing on the outgoing connections from the lan. Most other uploaders (like the default SM one) seem to initiate an upload connection for each file and then terminate that connection and open a new one for a new file, etc. This allows the router to choose the least utilized Internet connection and by doing so, a lot of concurrent connections will eventually utilize all available bandwidth.

For some reason, I've noticed that SL is only using one Internet connection. All the concurrent sessions must be on that one pipe. It is fully utilizing that one pipe since only 4 connections max it out, but there's another 25m/5m pipe just sitting idle. I'd really like to see it able to utilize all of these pipes simultaneously.

Oh, and this is for downloading, I'm going to try uploading again once the download is done. I may have had to let it sit there for a while like the download before it starts.

This is strange, because the application initiates a new connection for each download of a file, and upload of a file. The upload in alpha 7 might be broken due to some experimental code I introduced for finding duplicates. The 8th alpha will bring back the functionality, since I've put aside the duplicates code for a while, and tried to make things work again for other features to come. Stay close. It might be today.

udy

SamirD wrote:

But I haven't been able to get upload to work. When I add media to an empty unlisted album, it just sits there.

So you tried to upload using the 'Add Media to Album' menu entry?

That is indeed broken right now. Like I said, it will be back in 8th alpha.

SamirD

Great to hear from you! I'm so glad changing the max threads is that easy. Just for giggles, I'd say make it something like 20 or 40. I know there's people out there with 100mb pipes, and with Google's new foray in to ISP service with crazy fast speeds, 40+ threads may be needed to max out the connection in the future.

I've got an extremely busy weekend of shooting that I'm 4 hours behind preparing for, but I will definitely use SL on Sunday to try uploading my thousands of images from this weekend. It would be so awesome if I could just set it to go and then get some sleep.

udy

SamirD wrote:

Great to hear from you! I'm so glad changing the max threads is that easy. Just for giggles, I'd say make it something like 20 or 40. I know there's people out there with 100mb pipes, and with Google's new foray in to ISP service with crazy fast speeds, 40+ threads may be needed to max out the connection in the future.

I've got an extremely busy weekend of shooting that I'm 4 hours behind preparing for, but I will definitely use SL on Sunday to try uploading my thousands of images from this weekend. It would be so awesome if I could just set it to go and then get some sleep.

Samir, try to use the batch upload, if you wish to have the albums created automatically (the green UP arrow). Although I didn't raise the max to 20, 40, it is now 15. Following your feedback, I might up that again. See my next announcement.

udy

Announcement
The 8th alpha of SmugLoader is available, with the following:

• Fixed various bugs.
• Restored media upload to album (Add Media to Album menu). If you've read the info, there are plans (not very soon) to merge this with the batch upload, because the batch upload is backed by the database (resume on application failure). So if you have massive amount of files to upload, give batch upload a try.
• Raised the max download/upload threads to 15. Samir has asked for; if necessary, I'll raise that more.
• There is backend support for hiding files upon upload, but there is no UI for that just yet. Expect that to be there in the next iteration
• If you use the newly introduced launcher (SmugLoaderLauncher.jar) to start the application, upon resetting, or compacting the database, the application will restart automatically. If you start the application directly (SmugLoader-0.5.jar), the application will just close, so you'll have to restart it manually.

Enjoy.

SamirD

I'll check it out Sunday! Events are calling!

udy

9th Alpha Announcement
The 9th alpha of SmugLoader is available for download. It brings the following:

• Adds validation for NiceName;
• Now you can replace a media file. (Right click on a file in browser -> Replace...);
• Bug fixes.

Any feedback is appreciated.