AlbumFetcher - Download multiple galleries w/ embedded captions and keywords

wellman

LoneStarSnapper wrote:

I just tried it when I was logged in as owner and it worked great! When I try to log in as guest it fetches the album info. I then select the album I want to download and asks for the password. I enter the password and then the error "Error in fetching image data. Download aborted" appears in the status window.

I have a "standard" account with smugmug. Do I need to be a "power user" to use this feature?

Thanks again for a great tool and your help! It would be great if my friends and family could use it to download complete albums that I post.

Do you happen to have a site-wide password AND an album password?

wellman

anirudh.net wrote:

Looks like I uncovered a new bug in v0.3.3 when trying to get hidden images/videos. :cry I am using the "Login as owner" option and when the retrieval gets to the point where it encounters a hidden image/video, the program just hangs there without making any progress (the UI is responsive, except that it doesn't do anything). This should be easy to reproduce....just mark any of the images in one of your albums as "hide photo" and try to get the entire album logged in as owner.

Please let me know when you have a fix for this. And thank you for this amazing utility....this makes my archiving of smugmug albums extremely easy!

Thanks for the info. This will make the "must" list for the next version.

Got to be honest with you, though... I don't foresee time in the near future for coding updates to AlbumFetcher. The source is still available for anyone who wants to contribute to the effort.

LoneStarSnapper

wellman wrote:

Do you happen to have a site-wide password AND an album password?

I do not have a site-wide password but I am protecting each individual gallery at that level.

LoneStarSnapper

LoneStarSnapper wrote:

I do not have a site-wide password but I am protecting each individual gallery at that level.

I think I figured it out. The problem seems to be with the size of the gallery. I did some testing and it seems that I have problems with galleries of more than 239 photos. I split each large gallery into two smaller galleries and all is well. Thanks for your help!

NWMtnGuy

Greg,

Thanks for writing this utility. It is so useful! This seems like something that should be part of the basic functionality of SmugMug, but you seem to have solved the problem.

Dale

mindbugz

LoneStarSnapper wrote:

I think I figured it out. The problem seems to be with the size of the gallery. I did some testing and it seems that I have problems with galleries of more than 239 photos. I split each large gallery into two smaller galleries and all is well. Thanks for your help!

Hi,i have the same bug but the owner of the gallery don't want to split his gallery. Please wellman, can you fix this bug?

thx so much for you great software!!

bye

wellman

Does the "skip" feature not work?

mindbugz wrote:

Hi,i have the same bug but the owner of the gallery don't want to split his gallery. Please wellman, can you fix this bug?

thx so much for you great software!!

bye

mindbugz

I don't know, the error message says: " Error in fetching image data. Download aborted.". The album has 2500 photos.

M.

agallia

Albumfetcher v 0.3.3 Works Great!
Applause to Greg for AlbumFetcher. Downloaded v 0.3.3 and ran a test at once. Worked fine on 3 albums (largest 242 files/559 MB)...very fast, actually much faster than uploading. A great little utility for sure that solved my problem of downloading. Your hard work for the Smugmug folks is greatly appreciated.

Berto

Pro User Copy protection settings
Hi, I just tried AlbumFetcher. Great tool, thanks for providing this.

I wonder if these are by design or bugs.

I have a Pro account. I tried several iterations and for some reason, the tool does not take into consideration the smugmug pro account copy protection setting.
I was able to logon as GUESS without password and fetch all the pictures from an album which had copy protection and Original Size set.

This is fine when you login as owner and password, but as Guess, means anyone could copy the pictures without logon in as owner. To me its a security violation.

Also, I'll have to make some more tests, but if you login as owner with password, fetch pictures, then login again using guess with password or without password, it will still fetch the pictures, as if it does not reset itself for each login type.

Rgs, Albert

Berto

Ok, I understand image owners have the ability to make their images public or private. You would not be able to access any private images as a guest.

The Right Click protection is ignored, so I will have to make sure I limit the viewing size as to not include Original size for public views (guest).

Rgds, Albert

Berto wrote:

Hi, I just tried AlbumFetcher. Great tool, thanks for providing this.

I wonder if these are by design or bugs.

I have a Pro account. I tried several iterations and for some reason, the tool does not take into consideration the smugmug pro account copy protection setting.
I was able to logon as GUESS without password and fetch all the pictures from an album which had copy protection and Original Size set.

This is fine when you login as owner and password, but as Guess, means anyone could copy the pictures without logon in as owner. To me its a security violation.

Also, I'll have to make some more tests, but if you login as owner with password, fetch pictures, then login again using guess with password or without password, it will still fetch the pictures, as if it does not reset itself for each login type.

Rgs, Albert

agallia

Bear with me, O mighty Smugmug gurus...I am restating Albert's comments more for my benefit than those who really "know what they are doing". Please advise if you can clarify any of these issues.

Albert pointed out an issue in this great little AlbumFetcher program that I had failed to consider so far. Since the AlbumFetcher program is available to anyone who may want to download any Smugmugger's photos, I realized I need to carefully review of my gallery settings ASAP to insure security of those galleries I want to protect.

The issue is that all galleries set to "public" are fair game for downloading by anyone using AlbumFetcher as "guest". They simply have to know your Smugmug account name. The good news is that "guest" can only download photos at the max file sizes set for each public gallery. So, if you wish to leave your galleries "public" but don't want to allow "guest" downloading of the original photo files, be sure to change your "customize gallery" to the largest size you will allow.

As Albert also pointed out, the only way to prevent a "guest" from seeing a gallery on the list and downloading it is to set the "customize gallery" privacy setting to "unlisted". Then only the logged in Owner can access it. This is causing me to rethink the "public"/"unlisted" selection for my galleries as well as the password option too.

Now I do have a question: What is the "guest with site password" login for? Is this for the PRO account guest site password? Can anyone advise on this?

Again, AlbumFetcher is a fine tool. Kind of scary though to realize my Smugmug account wasn't as secure from unwanted access as I thought. It is tough being a "newbie" in many ways. Well, guess I'll try to soak it all up and keep learning. But woe to those who don't have an inkling!

wellman

Berto and agallia:

Your photos are safe. AlbumFetcher is able to get to these galleries without proper access because you yourself have accessed them via your browser somewhat recently and SmugMug has cached that permission. If you try with another computer which you have never logged in with, you will be unable to "work around" this security.

I'm not sure how often the permission caching gets reset on the SmugMug side.

agallia

wellman wrote:

Berto and agallia:

Your photos are safe. AlbumFetcher is able to get to these galleries without proper access because you yourself have accessed them via your browser somewhat recently and SmugMug has cached that permission. If you try with another computer which you have never logged in with, you will be unable to "work around" this security.

I'm not sure how often the permission caching gets reset on the SmugMug side.

Thanks for reply, Greg. But still a little confused about security issue. Please confirm what files a "stranger" can download from a remote computer using "guest" login and someone else's Smugmug ID? Can they download public gallery photos at the max set file display size? Exactly what photos are secure from them? This seems to be the case when I tried from my laptop for the first time. Thanks.

rghilljr

help

agallia wrote:

Thanks for reply, Greg. But still a little confused about security issue. Please confirm what files a "stranger" can download from a remote computer using "guest" login and someone else's Smugmug ID? Can they download public gallery photos at the max set file display size? Exactly what photos are secure from them? This seems to be the case when I tried from my lap
top for the first time. Thanks.

I am new here and would like a little help if possible. I do not know all the lingo but here is the problem that I have....

I am using or trying to use the Album Fetcher for my pics to be moved. I can log as a guest user and on my SMUG MUG account I CAN log in with my user name and password. However when I try to log into Album Fetcher with the same user nam and password, I get the following message.....
'SessionID extraction failed.
Aborting'

Any suggestions or help would be greatly appreciated!

TravisT

Thanks
Thanks a million for coming up with this app! What an FU**%N life saver! I had some major drive failures and found that the only edited copies of a wedding I recently shot were on SmugMug.. Just as I was about to pull out the little hair I have left, I contacted SmugMug and Zac over there told me to try out your app! THANK YOU! hours upon hours and $$ saved! GRACIAS!

chowpay

Nice! mac version
Works great on VMware.. perhaps a mac version.. maybe works with macdaddy?

wellman wrote:

No problem. I agree that sorting would be nice - it's top on the list for v0.3. We're about to head out on vacation, so v0.3 won't be turned around quite as quickly.


Oops. Thanks for the heads-up. I'll take care of that one.


If you want to verify this for yourself, Google a free app called Exifer. It will show you (and allow you to edit) all EXIF and IPTC tags. I did some testing to make sure other IPTC fields were left alone, but I wouldn't mind some secondary confirmation.


That's a fantastic idea. That will also go into the list for v0.3. My goal here was to make a "true" Smugmug backup tool that would save all my caption/keyword work, since I do most of mine on the site. The GPSInfo is a further extension of the same principle, so I will be happy to work that in.


Sweet! Thank you! Thanks also for your suggestions.

wellman

agallia wrote:

Thanks for reply, Greg. But still a little confused about security issue. Please confirm what files a "stranger" can download from a remote computer using "guest" login and someone else's Smugmug ID? Can they download public gallery photos at the max set file display size? Exactly what photos are secure from them? This seems to be the case when I tried from my laptop for the first time. Thanks.

I think the best way to say it is that users of AlbumFetcher will have the exact same permissions as viewers of your SmugMug site. There are different types of SmugMug passwords:

• Guest login: Allows others to make changes to your SM site. Folks with a guest login would be able to download anything via AlbumFetcher.
• Site-wide login: Allows viewers to enter a single passwords and gain view access to the site. AF users with this password would be able to get any galleries except those further protected with a...
• Gallery password: Gallery-specific password. AF users would have to have this password to get a specific gallery.

Hope that's clear as mud.

wellman

TravisT wrote:

Thanks a million for coming up with this app! What an FU**%N life saver! I had some major drive failures and found that the only edited copies of a wedding I recently shot were on SmugMug.. Just as I was about to pull out the little hair I have left, I contacted SmugMug and Zac over there told me to try out your app! THANK YOU! hours upon hours and $$ saved! GRACIAS!

You're quite welcome.

agallia

wellman wrote:

I think the best way to say it is that users of AlbumFetcher will have the exact same permissions as viewers of your SmugMug site. There are different types of SmugMug passwords:

• Guest login: Allows others to make changes to your SM site. Folks with a guest login would be able to download anything via AlbumFetcher.
• Site-wide login: Allows viewers to enter a single passwords and gain view access to the site. AF users with this password would be able to get any galleries except those further protected with a...
• Gallery password: Gallery-specific password. AF users would have to have this password to get a specific gallery.

Hope that's clear as mud.

Thanks, Greg. Appreciate your reply.

agallia

Caption/Slideshow problem with AlbumFetcher embedded ITCP data
This problem was originally posted on this Smugmug Support thread without resolution so I am also posting here, since it is probably related to AlbumFetcher 0.3.3 embedded captions. The © symbol in captions gets transposed when uploaded back to Smugmug.

As an additional test, I used my single photo gallery Little Things Often Missed to download/upload using AlbumFetcher 0.3.3

• First, with embed ITCP data checked, I downloaded photo "...018a.jpg" to my PC, modified it & saved it as "...018b.jpg" (infrared), and uploaded it to same gallery.
• Then, with embed ITCP data unchecked, I downloaded "...018a.jpg again, modified and saved it as "...018c.jpg (daguerreotype), and uploaded to same gallery.
• 018a and 018b had same photo captions with a visible "©" symbol when viewing gallery photos. The difference is that the © symbol in the 018b caption became "©" in edit mode. 018c, of course, had blank caption.
• The problem: in SLIDESHOW, the "©" was displayed as "©" for the 018b photo, not a © symbol.

It seems the way the ITCP embedded caption data is displayed in Slideshow changed from the download to the upload. In another gallery, although I had deleted a caption, it still appeared in Slideshow. Hope this info may be of some use. I was hoping to use AlbumFetcher as an handy tool to download, mod, and upload a gallery photo, keeping the captions/keywords intact. Please keep me advised.

mtnmotorman

Download abort issues
I'm trying to download a gallery and it aborts when it gets part way through the download. What do you recommend? I'm using Vista, is that a problem?

wellman wrote:

I'm pleased to announce the release of AlbumFetcher! AlbumFetcher is a little pet project of mine which I've decided to make freely available to the SmugMug community. I've received amazing amounts of help (and ideas) through these forums, and this my way of saying thanks. It's a simple little app that serves two main functions:

1. Download one or many SmugMug galleries at once
2. Embed captions and keywords into your downloaded images' IPTC headers

AlbumFetcher should run on Win2000, WinXP and above. It has been reported to run under Wine on Linux and through Parallels on a Mac. It's a single file, so it's easily portable and doesn't need to be installed. AlbumFetcher is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This DGrin thread will be the project's home page. Feedback is extremely appreciated, and if there are bugs and/or other issues (as there always are), please let me know here. This is a hobby, so I can't hope to live up to SmugMug's standard of customer service, but I will try to help you out if you're having an issue with the software. One word of caution - beware of the total number of bytes you download. You could easily run up against your monthly bandwith limit.

Again, this is yours to use forever for free. However, if you're a fan of my little app and feel so inclined, you can send me the electronic equivalent of the adult beverage of your choice here.

Enjoy!
-Greg


Current version: v0.3.3 - Application | Source


How Does it Work?

1. Download AlbumFetcher (zip file) with the link above.
2. Use WinZip or your favorite zip program to extract AlbumFetcher.exe (the program) to the folder of your choice (maybe the desktop).
3. Double-click the AlbumFetcher program icon.
4. If you're an account owner: In the top-right corner, enter your SmugMug ID and password. If you're a guest: Switch the login type selector to Guest. Then enter the SmugMug ID of the account you'd like to work with. You'll only have access to listed public galleries. (SmugMug ID means either your email address which is registered with SmugMug or the yourname part of yourname.smugmug.com. Your ID and password go directly to SmugMug; there's nobody listening in between.)
5. Click List Albums. A listing of all your albums will appear.
6. In the Download Options box (on the right), pick a folder to save albums to. Also pick the image sizes and other options as you wish.
7. Click the list to select albums for download. Use Ctrl-clicking or Shift-clicking to select more than one.
8. Click Fetch Albums. AlbumFetcher will now download your requested albums at the size you specified.
9. Have a tasty beverage.

UPDATE, Nov 14 2006:
I've incorporated some of Pat Kane's requests and some improvements of my own into v0.2. New features include:

1. Datetime stamp for each line in log file
2. Files with same original name are now handled successfully (an underscore is prepended to the second file's name).
3. Users can optionally skip over previously downloaded images (helpful when a download fails and you want to start over, but not re-download all the successful images).
4. Category/subcategory info is shown as a field in the list control.
5. A thumbnail of the image currently being downloaded is displayed.
6. Total time statistics are recorded when a job is completed successfully.
7. MD5 hashes are now compared. Files that fail the comparison are left alone, but a note is made in the log file.

Again, enjoy!
-Greg


UPDATE, Nov 16 2006:
Pat Kane pointed out a bug in the folder selector, which is now fixed. Current version is now 0.2.1. Thanks, Pat!


UPDATE, Jan 11 2008:
I'm pleased to announce the release of AlbumFetcher v0.3. There are several improvements in this release. Read on for details.

1. Multiple login options - Users may now login to SmugMug as account owner, guest, or guest with site password. If you're uncomfortable with other people being able to bulk-download your images, set your external linking option (in gallery customization) to "No."
2. Album-level password support - Guests may download password-protected galleries (with the proper password of course).
3. Selectable image size for download - Choose which size you want! AlbumFetcher will find the next largest available if logged in as guest and requested size is not available.
4. Video support - Choose which size you want! AlbumFetcher will find the next largest available if logged in as guest and requested size is not available. Support is also included for legacy mpeg-1 videos.
5. As inspired by SmugMug, AlbumFetcher is stretchy!
6. No more runtime errors - (Fingers crossed.) AlbumFetcher has essentially been completely rewritten. I found a few scenarios that caused the app to crash during my development process, and they've been eliminated. I've successfully downloaded thousands of images while testing. Let me know if you still have runtime problems.

UPDATE, Jan 12 2008:
Scott Walter pointed out a bug which caused the app to crash when downloading GIFs, which is now fixed. Current version is now 0.3.1. Thanks, Scott!



UPDATE, Jan 29 2008:
A few people were having similar issues. Basically AlbumFetcher was bombing out if it tried to download an image which was missing (for whatever rare reason) on SmugMug. This is now handled more gracefully in new Version 0.3.2 (link above) - AlbumFetcher will stop the download and report the error (instead of crashing).



UPDATE, Mar 3 2008:
AlbumFetcher is now compatible with SmugMug's security changes (use of keys).


Again, enjoy!
-Greg

mlee

I'm seeing some errors that I've never seen before. Possibly related to the changes smugmug made this past week to the login screen and interstitial page bypass?

12/14/2008 11:13:56 AM: AlbumFetcher start
12/14/2008 11:14:06 AM: Logging in user xxx@xxx.com...
12/14/2008 11:14:06 AM: Start REST transaction.
12/14/2008 11:14:07 AM: Expect file of size 102 bytes.
12/14/2008 11:14:07 AM: Received file of size 102 bytes.
12/14/2008 11:14:07 AM: REST transaction successful.
12/14/2008 11:14:07 AM: Login for user xxx@xxx.com successful.
12/14/2008 11:14:07 AM: SessionID extraction failed. Aborting.

Any help is greatly appreciated!

Mike

agallia

mlee wrote:

I'm seeing some errors that I've never seen before. Possibly related to the changes smugmug made this past week to the login screen and interstitial page bypass?

12/14/2008 11:13:56 AM: AlbumFetcher start
12/14/2008 11:14:06 AM: Logging in user xxx@xxx.com...
12/14/2008 11:14:06 AM: Start REST transaction.
12/14/2008 11:14:07 AM: Expect file of size 102 bytes.
12/14/2008 11:14:07 AM: Received file of size 102 bytes.
12/14/2008 11:14:07 AM: REST transaction successful.
12/14/2008 11:14:07 AM: Login for user xxx@xxx.com successful.
12/14/2008 11:14:07 AM: SessionID extraction failed. Aborting.

Any help is greatly appreciated!

Mike

FYI...just tested AlbumFetcher v0.3.3 twice as "Owner" login without any problems. 1st download was 36 files/87MB and 2nd was 17 files/65M. Went very fast without a hitch. Wish I could help.

mlee

mlee wrote:

I'm seeing some errors that I've never seen before. Possibly related to the changes smugmug made this past week to the login screen and interstitial page bypass?

12/14/2008 11:13:56 AM: AlbumFetcher start
12/14/2008 11:14:06 AM: Logging in user xxx@xxx.com...
12/14/2008 11:14:06 AM: Start REST transaction.
12/14/2008 11:14:07 AM: Expect file of size 102 bytes.
12/14/2008 11:14:07 AM: Received file of size 102 bytes.
12/14/2008 11:14:07 AM: REST transaction successful.
12/14/2008 11:14:07 AM: Login for user xxx@xxx.com successful.
12/14/2008 11:14:07 AM: SessionID extraction failed. Aborting.

Any help is greatly appreciated!

Mike

I found the 'fix' for my issue. Apparently, AlbumFetcher didn't like the fact that I had a non alphanumeric character (#) in my password.

I modified my password and all is well now.

Mike

lazfsh

Getting errors on trying to download album
Here is my log file
===========================================
12/15/2008 3:16:24 PM: AlbumFetcher start
12/15/2008 3:16:30 PM: Logging in user dlh...
12/15/2008 3:16:30 PM: Start REST transaction.
12/15/2008 3:16:31 PM: Expect file of size 181 bytes.
12/15/2008 3:16:31 PM: Received file of size 181 bytes.
12/15/2008 3:16:31 PM: REST transaction successful.
12/15/2008 3:16:31 PM: Login for user dlh successful.
12/15/2008 3:16:31 PM: Retrieving albums for user dlh...
12/15/2008 3:16:31 PM: Start REST transaction.
12/15/2008 3:16:31 PM: Expect file of size 34228 bytes.
12/15/2008 3:16:31 PM: Received file of size 34228 bytes.
12/15/2008 3:16:31 PM: REST transaction successful.
12/15/2008 3:16:31 PM: Successfully listed 110 albums.
12/15/2008 3:16:58 PM: Requesting image information for album "Brooke&Michael/wedding"...
12/15/2008 3:16:58 PM: Start REST transaction.
12/15/2008 3:17:08 PM: Download failed on file http://api.smugmug.com/services/api/rest/1.2.1/?method=smugmug.images.get&SessionID=951f3eaa9b4b4c5718b338d0996fd3f3&AlbumID=3977330&Heavy=1&Password=churchill&AlbumKey=wMQsp. Received Internet Exception: The connection with the server was reset
12/15/2008 3:17:08 PM: GetUrlContentsAsMemFile failed in GetUrlContentsAsString call.
12/15/2008 3:17:08 PM: REST transaction failed at call into URL handler. Aborting.
12/15/2008 3:17:08 PM: Image info array construction failed. Download aborted.
12/15/2008 3:17:08 PM: Error in fetching image data. Download aborted.

wellman

This is a cross-post from over here, but it seemed appropriate to append to this thread...

It's been a pleasure over the past two years to see AlbumFetcher become a useful little download utility for members of the SmugMug family. The users of AlbumFetcher deserve a product that will continue to evolve and keep up with the latest SmugMug features (hidden photos, SmugVault, etc). To use the catchphrase circulating at work, I no longer have the personal bandwidth to keep AlbumFetcher current.

This thread is a call for help for someone to take the AlbumFetcher reins. The code is simple Microsoft Visual C++, using MFC. If there's anyone interested in taking the project over as-is or porting it to something cross-platform, let me know.

Thanks, and thanks again to those who've had kind words and future ideas for AlbumFetcher.

vicarious1

How to:Main page> secondary page 4 private galleries?
Dear Welman.

I am 100% new to Smugmug and have stock and private. I followed Album fetcher and to you link to
http://www.wellmanphoto.com/Wellman%20Family%20Photos
and encountered your page with one pic per gallery but when clicking on it a password is required. Excellent !
May I ask how do you create ONE picture in the main page that leads to all a second page with many more private galleries within smugmug?
I have limited my private galleries for now as I don't want many private galleries to clogg my new main stock photos page.

wellman wrote:

I'm pleased to announce the release of AlbumFetcher!
Thanks you so much for any advise. Merry Xmas happy new year. Vic

Lenzflair

Hi there,

I think I have a clue as to why some of us get the C++ error.

I believe the problem lies in files that Smugmug stores without a file extension. Example: some files are 1234567.jpg and some just 1234567, no extension.

I strongly suspect that this is what is causing the crash. I tried the other downloader and get a crash (as a hang) as well.

I used Migratr and it doesn't crash, but has many many non extension files. Change the extenson manually to .jpg and the file is ok.

Hope this helps 3.4

agallia

Lenzflair wrote:

Hi there,

I think I have a clue as to why some of us get the C++ error.

I believe the problem lies in files that Smugmug stores without a file extension. Example: some files are 1234567.jpg and some just 1234567, no extension.

I strongly suspect that this is what is causing the crash. I tried the other downloader and get a crash (as a hang) as well.

I used Migratr and it doesn't crash, but has many many non extension files. Change the extenson manually to .jpg and the file is ok.

Hope this helps 3.4

Strange. I have used AF a number of times and it always downloads .jpg files from my SM account.