Bulk download script for OS X (Panther)

darryl

So a buddy who's about to sign-up for Smugmug thought it would be cool to be able to upload his images while on vacation (via hotel broadband, internet cafe, etc.), and flush his card. He'd probably have his laptop as well, but smugmug could serve as a handy backup.

I agreed, but pointed out that there's not (as yet) any bulk downloading option. Then I mentioned that I could probably throw a script together to do it for us.

So, here ya go. No guarantees of any kind, of course, and excuse the sloppy coding. I'm a hack, not a hacker or coder.

It's pretty short, so I'm including the full script down at the end here. But to download it, here's a link that you can right-click and save as. Save it into your home directory and rename it to getsmug.pl. In a Terminal window, you'll need to find the script and do:

chmod getsmug.pl 755

Then you run it like so:

./getsmug.pl http://yourname.smugmug.com/gallery/123456

It'll chug along, pulling down *original*-sized images into a new directory with the same number as listed in the URL.

Pretty basic stuff -- I was too lazy to name the directory after your album -- but it's in the HTML and you could easily fix that.

Original filenames are *not* retained, but unless smugmug wanted to start mucking around in the IPTC data of your photos (say set "Title" to original filename) there's not much you can do about that.

All other EXIF metadata *is* retained though. (And actually, if you used something like say, "Caption Buddy" to write the filename to "Title" as I mention above, this script could probably be tweaked to rename the file back. But that too is out of scope.)

Have fun!

--Darryl

#!/usr/bin/perl

$url = shift @ARGV;

if ($url eq '' || $url !~ /smugmug\.com/i ) {
die "Did you forget something like, say, the URL of a smugmug album?\n"
}

$album = $url ;
$album =~ s/^.*\/(\d+)$/$1/;

system "curl --silent --cookie 'Template=7;DefaultSize=Original' $url > /tmp/smug$album.html" ;

open (DUMP, "</tmp/smug$album.html");

mkdir $album;
chdir $album;

while (<DUMP>) {
if (/Ti.jpg/) {
s/^.+src="(.+)Ti.jpg".+$/$1O.jpg/ ;
system "curl -O $_" ;
}
}

darryl

BTW, JT -- I found a small error in your HTML for "All Thumbs" view. For example, in this gallery

<td align="center"><center><a href="http://gladlee.smugmug.com/gallery/132296/1/4810754/Original"><img align="center" class="smborderoff" onmouseover="this.className='smborder';" onmouseout="this.className='smborderoff';" name="mainPhoto" alt="Glady, Darryl and Noah > Friends photo" border="0" src="http://gladlee.smugmug.com/photos/4810754-Ti.jpg&quot; width="66" width="100" hspace="0" vspace="0" /></a></center></td>

There's two 'width' tags within that <img> tag, and the tag is closed with a /> ? That ain't right. :-}

{JT}

Good catch on the double width, not sure how that snuck in there but I will go get it fixed. However, the /> closing is correct. We stick to xhtml standards on the site, so all empty tags (like <img>) must terminate.

http://fog.ccsf.org/~srubin/xhtmltags.html

darryl

{JT} wrote:

Good catch on the double width, not sure how that snuck in there but I will go get it fixed. However, the /> closing is correct. We stick to xhtml standards on the site, so all empty tags (like <img>) must terminate.

http://fog.ccsf.org/~srubin/xhtmltags.html

Ah right -- I had a feeling that one was intentional. Thanks!

Guzzler

Ooof,

I installed ActivePerl for Winbloze and tried to run the script... No go. I am not fluent enough in perl to debug it. Any other suggestions?

darryl

Guzzler wrote:

Ooof,

I installed ActivePerl for Winbloze and tried to run the script... No go. I am not fluent enough in perl to debug it. Any other suggestions?

Crap -- I should have specified that it also relies on an external program, curl, which allows you to do:

curl -O http://whatever.com/foo.jpg

And it'll save foo.jpg to your current directory.

You can get it for Win32 here:

http://curl.siamu.ac.th/download.html

Within the script, you might need to specify the full path to curl (c:\program files\curl\curl.exe or whatever), as well as a different path for /tmp (c:\temp might work.)

Other than that, it's not too fancy.

Oh oh, and #!/usr/bin/perl might need to be #!c:\program files\perl\perl.exe although I believe ActivePerl is supposed to be smart enough to figure that part out, so try that last. :-}

--Darryl

Guzzler

Getting there, close but no cigar.

Thanks for pointing out about 'curl'. But it looks like 'curl' isn't interpreting the cookie settings properly. It looks like the "Template" value isn't being sent properly because the temporary html file isn't pulling up all the images in the gallary.

darryl

Guzzler wrote:

Getting there, close but no cigar.

Thanks for pointing out about 'curl'. But it looks like 'curl' isn't interpreting the cookie settings properly. It looks like the "Template" value isn't being sent properly because the temporary html file isn't pulling up all the images in the gallary.

Hey Guz --

Apparently Windows doesn't like single quotes, so I think you'll need to try:

system qq|curl.exe --silent --cookie "Template=7;DefaultSize=Original" $url > /tmp/smug$album.html| ;

(qq| tells Perl to treat the | character as a delimiter, instead of a double-quote, but still interpolate the $variables within the string)

--Darryl

Guzzler

Darryl,

We're almost home!!!

You forgot one conditional in you IF routine. You forgot to deal with images that are in portrait (*-Ti-1.jpg), versus landscape (*-Ti.jpg).

Baldy

Guzzler wrote:

Darryl,

We're almost home!!!

You forgot one conditional in you IF routine. You forgot to deal with images that are in portrait (*-Ti-1.jpg), versus landscape (*-Ti.jpg).

Not sure the difference between Ti-1.jpg and Ti.jpg is necessarily portrait versus landscape. I think it means that you've modified it via either cropping, rotation, or color effects. Giving it a new number is our way of getting the browser to refresh it, among other things. So there could be a Ti-2.jpg, etc.

Guzzler

Baldy wrote:

Not sure the difference between Ti-1.jpg and Ti.jpg is necessarily portrait versus landscape. I think it means that you've modified it via either cropping, rotation, or color effects. Giving it a new number is our way of getting the browser to refresh it, among other things. So there could be a Ti-2.jpg, etc.

Aaaa, so it needs to be a wildcard then.

FYI, I'm trying to help Ricky create some animations from the Ouray adventure. I just don't feel like "right clicking, save-as" for all the pictures.

wooac

What about password protected galleries?
How do you modify the --cookie line for password protected galleries?

Thanks.

darryl wrote:

So a buddy who's about to sign-up for Smugmug thought it would be cool to be able to upload his images while on vacation (via hotel broadband, internet cafe, etc.), and flush his card. He'd probably have his laptop as well, but smugmug could serve as a handy backup.

I agreed, but pointed out that there's not (as yet) any bulk downloading option. Then I mentioned that I could probably throw a script together to do it for us.

So, here ya go. No guarantees of any kind, of course, and excuse the sloppy coding. I'm a hack, not a hacker or coder.

It's pretty short, so I'm including the full script down at the end here. But to download it, here's a link that you can right-click and save as. Save it into your home directory and rename it to getsmug.pl. In a Terminal window, you'll need to find the script and do:

chmod getsmug.pl 755

Then you run it like so:

./getsmug.pl http://yourname.smugmug.com/gallery/123456

It'll chug along, pulling down *original*-sized images into a new directory with the same number as listed in the URL.

Pretty basic stuff -- I was too lazy to name the directory after your album -- but it's in the HTML and you could easily fix that.

Original filenames are *not* retained, but unless smugmug wanted to start mucking around in the IPTC data of your photos (say set "Title" to original filename) there's not much you can do about that.

All other EXIF metadata *is* retained though. (And actually, if you used something like say, "Caption Buddy" to write the filename to "Title" as I mention above, this script could probably be tweaked to rename the file back. But that too is out of scope.)

Have fun!

--Darryl

wooac

Has anyone update getsmug.pl for new directory structure?
The getsmug.pl script now fails to obtain a webpage with the urls of the photos within a gallery. Has anyone updated it to obtain that information?

darryl

Sorry, for security reasons SmugMug disabled the ability to link to a specific Style for a gallery. So I can't link to "All Thumbs" anymore (well, ok, I had some ideas on how to get around this, but haven't had a chance to test them yet.)