smugmug.py -- large scale batch smugmug tool

rutt · December 8, 2004

This program allows you to:

Upload unlimited numbers of images to arbitrary smugmug albums
Create new smugmug albums with arbitrary properties
Mirror an arbitrary directory/folder tree in your smugmug account
Keep smugmug albums up to date as new images are added to local folders mirrored on smugmug.

The good news about it is that it is completely noninteractive. It's a command line program suitable for use in batch jobs and scripts. This is also the bad news, I guess. People who are lost without a button to push and a place to drag and drop stuff will be mystified. But if you are trying to set up a large scale account on smugmug of a set of local galleries, this might be just the thing.

I have used to to move more than 50GB of images to smugmug. It maintains more than 800 albums. If I add a new photo or new edit of an old photo to any of these galleries, it finds it and uploads it.

It also enables you to have some defaults that smugmug doesn't. In particular, "show filenames" can be the default mode for new galleries. I like this very much, but Don and Chris thought it was unsuitable as a default. It also allows you to use the filename as a caption. This has the benefit that smugmug searches for the filename will find its associated image.

Quick start:

Install python >= 2.3 if you don't already have it. Macs with panther kept up to date do have this. Linux users will know what to do. Windows isn't really supported yet, but likely works. For windows, visit python.org
Get smugmug.py
Do smugmug.py --help for documentation

I am now considering how to add a set of new features. In particular, I want:

Automatic resorting after new photos are added to an album
Better control over captions and album names
Ability to change album properties, categories, subcategories
When albums are created, properties, categories, subcategories are inhereted from parent folders/directories. This allows one to have a tree whose direct branches are categories and then subcategories.

I'd like to hear comments about these proposed features and about smugmug.py in general. There have been some downloads, but like Nilolai, I can't tell if anyone but me is actually using it. In fact, unlike Nikolai, I really can't tell.

I don't think I did a very good job of advertising this when I first announced it, and probably I haven't done so this time either. But I wrote this thing because I needed it, and now I love it. If you are a true geek, you will proably love it, too.

onethumb · December 8, 2004

rutt wrote:

This program allows you to:
Upload unlimited numbers of images to arbitrary smugmug albums

Create new smugmug albums with arbitrary properties

Mirror an arbitrary directory/folder tree in your smugmug account

Keep smugmug albums up to date as new images are added to local folders mirrored on smugmug.

The good news about it is that it is completely noninteractive. It's a command line program suitable for use in batch jobs and scripts. This is also the bad news, I guess. People who are lost without a button to push and a place to drag and drop stuff will be mystified. But if you are trying to set up a large scale account on smugmug of a set of local galleries, this might be just the thing.

I have used to to move more than 50GB of images to smugmug. It maintains more than 800 albums. If I add a new photo or new edit of an old photo to any of these galleries, it finds it and uploads it.

It also enables you to have some defaults that smugmug doesn't. In particular, "show filenames" can be the default mode for new galleries. I like this very much, but Don and Chris thought it was unsuitable as a default. It also allows you to use the filename as a caption. This has the benefit that smugmug searches for the filename will find its associated image.

Quick start:
Install python >= 2.3 if you don't already have it. Macs with panther kept up to date do have this. Linux users will know what to do. Windows isn't really supported yet, but likely works. For windows, visit python.org

Get smugmug.py

Do smugmug.py --help for documentation

I am now considering how to add a set of new features. In particular, I want:
Automatic resorting after new photos are added to an album

Better control over captions and album names

Ability to change album properties, categories, subcategories

When albums are created, properties, categories, subcategories are inhereted from parent folders/directories. This allows one to have a tree whose direct branches are categories and then subcategories.

I'd like to hear comments about these proposed features and about smugmug.py in general. There have been some downloads, but like Nilolai, I can't tell if anyone but me is actually using it. In fact, unlike Nikolai, I really can't tell.

I don't think I did a very good job of advertising this when I first announced it, and probably I haven't done so this time either. But I wrote this thing because I needed it, and now I love it. If you are a true geek, you will proably love it, too.

Rutt, do you have a "homepage" for this that I can link to so people can see what the latest version is and everything? Right now, I have smugmug linked to the thread, but already the thread has changed, so that's not going to work.

Thanks,

Don

rutt · December 8, 2004

onethumb wrote:

Rutt, do you have a "homepage" for this that I can link to so people can see what the latest version is and everything? Right now, I have smugmug linked to the thread, but already the thread has changed, so that's not going to work.

Thanks,

Don

I don't have a homepage for it, but I suppose I ought to make one if I want to market it, eh? It's in the queue.

unsavory · December 12, 2004

This sounds very promising for me since I have about 10k images and 100's of albums to upload.

My only question is, when will the new features be added so that it creates new categories/subcategories to mirror the directory tree?

Also, the feature of resorting would be nice. Particularly I would like the ability to sort all of my albums alphabetically as they appear in my directory structure.

I have put a request in for this with SmugMug as well. I am really trying to avoid having to write my own program to manage my photos, since I don't have the time right now. But right now it looks like you are headed in the right direction.

I actually prefer scripts over GUI's. So I will be watching the progress very closely.

Thanks for all your great work!

Nikolai · December 12, 2004

I know, it's not my "home", but

unsavory wrote:

My only question is, when will the new features be added so that it creates new categories/subcategories to mirror the directory tree?

There is no API for this yet

..
We al need to beg Don to add it asap..

unsavory wrote:

Also, the feature of resorting would be nice. Particularly I would like the ability to sort all of my albums alphabetically as they appear in my directory structure.

Smugmug Explorer can do sorting for you. Auto album creation is still in the works, though - can' figure a nice logical UI for it..

Cheers!

rainforest1155 · December 12, 2004

not working for me for some reason + update idea
Hi rutt,

wanted to give your script a shot, because I like the syncronizing feature about it, but I can't get it to run. I always get the following error:

C:\SYS\Smugmug Explorer\Python>smugmug.py --help
Traceback (most recent call last):
  File "C:\SYS\Smugmug Explorer\Python\smugmug.py", line 629, in ?
	main()
  File "C:\SYS\Smugmug Explorer\Python\smugmug.py", line 624, in main
	Options(argv[1:])
  File "C:\SYS\Smugmug Explorer\Python\smugmug.py", line 419, in __init__
	self.rcfile_options = defaults_from_rc()
  File "C:\SYS\Smugmug Explorer\Python\smugmug.py", line 380, in defaults_from_r
c
	rcfile = path.join(environ['HOME'],'.smugmugrc')
  File "C:\sys\Python24\lib\os.py", line 422, in __getitem__
	return self.data[key.upper()]
KeyError: 'HOME'

Do you have an idea what I've to do to get it working?

Also I would like to know if it's possible to update a modified picture with your script? smugmug doesn't officially support it as far I know.
I want the updating function mainly for the reason to add captions later on in the ITPC part of a file.
So if updating is not possible, what do you think of expanding your script to look at the ITPC-description and update the caption on smugmug according to it?

Thanks for your help.
Bye,
Sebastian

rutt · December 12, 2004

unsavory wrote:

I actually prefer scripts over GUI's. So I will be watching the progress very closely.

Thanks for all your great work!

I was waiting for even one person to say that before I started off on new features. Now that you have, I'll go to work! It won't take very long, but exactly how long, I'm not sure. Could be done in a day or so.

rutt · December 12, 2004

rainforest1155 wrote:

Hi rutt,

wanted to give your script a shot, because I like the syncronizing feature about it, but I can't get it to run. I always get the following error:
Thanks for your help.
Bye,
Sebastian

Ah, just what I needed: a windows tester. It's complaining because you don't have the HOME environment variable set. I thought that windows did that. Anyway, this is an environment variable.

Nilolai, help us here. Shouldn't this be set by default? If not, what's the right thing for an application to do to look for a file containing per user initializations?

Nikolai · December 12, 2004

My $.02

rutt wrote:

Ah, just what I needed: a windows tester. It's complaining because you don't have the HOME environment variable set. I thought that windows did that. Anyway, this is an environment variable.

Nilolai, help us here. Shouldn't this be set by default? If not, what's the right thing for an application to do to look for a file containing per user initializations?

Rutt,
for what I know, HOME was never a part of default environment settings (unlike PATH or TEMP, for example) in DOS/Windows enviroment.
Here's what you can do:

check for this variable value and require a user to set it if it's empty - it's a relatively easy thing to do
Traditionally, windows recommends using registry or (during win16 times) INI files. I'm not a huge fan of storing things in registry, even though it's a "recommended" way. However, I am very much in favor of using old good INI files. They are plain text files named after its "master" with .ini extension (in your case it would be .ini instead of .py) with a very simple structure:

[section1]
key1=value1
key2=value2
...

that any user can modifiy very easily.

However, I'm not sure if you have acess to INI files support from the python environment - don't know anything about python at all:-(, except that it's a nice script language.

HTH

rutt · December 12, 2004

Nikolai wrote:

Rutt,
for what I know, HOME was never a part of default environment settings (unlike PATH or TEMP, for example) in DOS/Windows enviroment.
Here's what you can do:
check for this variable value and require a user to set it if it's empty - it's a relatively easy thing to do

Traditionally, windows recommends using registry or (during win16 times) INI files. I'm not a huge fan of storing things in registry, even though it's a "recommended" way. However, I am very much in favor of using old good INI files. They are plain text files named after its "master" with .ini extension (in your case it would be .ini instead of .py) with a very simple structure:

[section1]

key1=value1

key2=value2

...

that any user can modifiy very easily.

However, I'm not sure if you have acess to INI files support from the python environment - don't know anything about python at all:-(, except that it's a nice script language.

HTH

Yes, but where do I look for the .ini files if HOME isn't set. That's what I was looking for. For os x and linux HOME tells where to look for this cikd of thing. Is there a convention I can follow on windows?

Nikolai · December 12, 2004

Oh, ...

rutt wrote:

Yes, but where do I look for the .ini files if HOME isn't set. That's what I was looking for. For os x and linux HOME tells where to look for this cikd of thing. Is there a convention I can follow on windows?

As you can see, I'm not a unix guru, so I didn't (and still don't:-) know what exactly HOME is..

Well, in this case, probably you simply need to check this variable's value and, if empty, ask your user to quit the program and set it up..

Is there any windows API availalbe for you? Amongst other things, each process has its own current directory, which you can get via winAPI.. Apart from that, the only thing you can rely on is the environment variables (it is my understanding that you can access those, right?)

HTH

onethumb · December 13, 2004

rutt wrote:

Yes, but where do I look for the .ini files if HOME isn't set. That's what I was looking for. For os x and linux HOME tells where to look for this cikd of thing. Is there a convention I can follow on windows?

If I understand the question (and it sound similar to things you have to do to get cygwin and stuff working properly), you need to set a HOME variable.

You can do that in System Properties -> Advanced -> Environment Variables.

HTH.

Don

rainforest1155 · December 13, 2004

onethumb wrote:

If I understand the question (and it sound similar to things you have to do to get cygwin and stuff working properly), you need to set a HOME variable.
You can do that in System Properties -> Advanced -> Environment Variables.

Thanks Don, that's it! It wanted to try this to, but after some googleing I wasn't sure anymore about what variable to set, because there were so many different explanations.

Bye,
Sebastian

rutt · December 13, 2004

rainforest1155 wrote:

Thanks Don, that's it! It wanted to try this to, but after some googleing I wasn't sure anymore about what variable to set, because there were so many different explanations.

Bye,
Sebastian

Does that mean that you just did this and smugmug.py just worked? If not, please let me know about any problems. I'd like it to work on windows, but don't have a machine I can test it on. Thanks.

rainforest1155 · December 13, 2004

rutt wrote:

Does that mean that you just did this and smugmug.py just worked? If not, please let me know about any problems. I'd like it to work on windows, but don't have a machine I can test it on. Thanks.

I had no time for further testing, but the help screen displayed without any errors. So I guess the rest will work too.

rutt · December 13, 2004

rainforest1155 wrote:

I had no time for further testing, but the help screen displayed without any errors. So I guess the rest will work too.

That's optimistic, but would be great. Please keep me informed.

David Ephron · December 14, 2004

Audience

rutt wrote:

I was waiting for even one person to say that before I started off on new features. Now that you have, I'll go to work! It won't take very long, but exactly how long, I'm not sure. Could be done in a day or so.

Just to let you know that you have an appreciative audience, I will chime in that I have been using the single file upload method from your original script for over a month now on a regular basis. I invoke it within an applescript under Mac OS X. The reason I'm doing it this way is that I can easily control iView Media Pro from applescript as well, and upload files based on the annotations in each file, which are easily accessed through iView Media Pro. Applescript has xml-rpc support built in, but no easy way to perform the required conversion to base64 as a data object, rather than as a string. I am using your script to implement the HTTP POST method, since I can't figure out how to do it on my own. I plan to take a look at your newest version as soon as I have some spare time (which probably won't be for another few weeks) and see how I can make use of the new features. Thanks for your hard work and for posting it.

DavidTO · January 22, 2005

rutt wrote:

I don't think I did a very good job of advertising this when I first announced it, and probably I haven't done so this time either. But I wrote this thing because I needed it, and now I love it. If you are a true geek, you will proably love it, too.

Not a true geek, as I couldn't figure out what the heck I was supposed to do without a button. How 'bout a little handholding for us wanna-be-geeks?

rutt · January 23, 2005

DavidTO wrote:

Not a true geek, as I couldn't figure out what the heck I was supposed to do without a button. How 'bout a little handholding for us wanna-be-geeks?

I started to write a quick start, faq, manual type thing, but I found that I didn't actually know what to write. I'm so far down the geek road that it's hard for me to look back and see where others are and where they might get stuck. So where did you get stuck? Could you download it? Do you know how to get a command shell (terminal on macs) window? Could you run smugmug.py? Did you try. Give me some help so I don't have to play guessing games and I'll help you get started.

DavidTO · January 23, 2005

rutt wrote:

I started to write a quick start, faq, manual type thing, but I found that I didn't actually know what to write. I'm so far down the geek road that it's hard for me to look back and see where others are and where they might get stuck. So where did you get stuck? Could you download it? Do you know how to get a command shell (terminal on macs) window? Could you run smugmug.py? Did you try. Give me some help so I don't have to play guessing games and I'll help you get started.

I clicked on the link to get it, and Safari showed my the text of the program. That's where I quit. I imagine that I could right-click and download linked file as...but then I wouldn't know what to do with that.

rutt · January 23, 2005

DavidTO wrote:

I clicked on the link to get it, and Safari showed my the text of the program. That's where I quit. I imagine that I could right-click and download linked file as...but then I wouldn't know what to do with that.

That's sort of what I was afraid of. What about the Terminal application? Have you ever used it?

I just need to figure out the scope of the hand holding required.

DavidTO · January 23, 2005

rutt wrote:

That's sort of what I was afraid of. What about the Terminal application? Have you ever used it?

I just need to figure out the scope of the hand holding required.

"That's what I was afraid of"... great.

I have used the terminal a bit, but only with step by step instructions. I would never open it and just do something. Most recently I ran plutil to check prefs, but I was just following directions.

rutt · April 2, 2005

Smugmug.py has been renamed to sm_tool.py. It is now available for download from http://www.chezrutt.com/rutt/sm_tool/sm_tool.py. Attempts to download smugmug.py from its original location will get you something called smugmug.py that doesn't actually work (but if you look at it, it will tell you where to get sm_tool.py)

darryl · April 19, 2005

Hey Rutt:

I must be doing something wrong. I've got version 1.7 of the tool (although the version variable still says 1.6. :-}

Creating albums seems to work, as does uploading to a specific gallery_id.

But what I can't get to work is automatic uploading upon creation or update. I have a directory structure of ~/photos/2005/2005.04.17-OneYearPartyCake

Should I be running sm_tool.py when I'm inside that directory? It needs to create the SMUGMUG_INFO/ subdir within each gallery, right? (Also, it would've been nice if you made that a dotted directory so it was hidden from normal views, but no biggie.)

At any rate, when I do:

~/sm_tool.py create 2005.04.17-OneYearPartyCake --category=Children *

Nothing gets uploaded. *.JPG doesn't work, nor does IMG_1945.JPG

What's the trick? I also tried it one-level up, specifying the directory name, but no luck. But I don't think that's the right place to run from.

I'm running this on OS X 10.3.8, with Python 2.3 installed.

Thanks!

--Darryl

darryl · April 19, 2005

Aha...
Just looked at the change notes for the new version. It's because my filenames were .JPG?

Argh.

--Darryl

darryl wrote:

Hey Rutt:

I must be doing something wrong. I've got version 1.7 of the tool (although the version variable still says 1.6. :-}

Creating albums seems to work, as does uploading to a specific gallery_id.

But what I can't get to work is automatic uploading upon creation or update. I have a directory structure of ~/photos/2005/2005.04.17-OneYearPartyCake

Should I be running sm_tool.py when I'm inside that directory? It needs to create the SMUGMUG_INFO/ subdir within each gallery, right? (Also, it would've been nice if you made that a dotted directory so it was hidden from normal views, but no biggie.)

At any rate, when I do:

~/sm_tool.py create 2005.04.17-OneYearPartyCake --category=Children *

Nothing gets uploaded. *.JPG doesn't work, nor does IMG_1945.JPG

What's the trick? I also tried it one-level up, specifying the directory name, but no luck. But I don't think that's the right place to run from.

I'm running this on OS X 10.3.8, with Python 2.3 installed.

Thanks!

--Darryl

rutt · April 19, 2005

darryl wrote:

At any rate, when I do:

~/sm_tool.py create 2005.04.17-OneYearPartyCake --category=Children *

Nothing gets uploaded. *.JPG doesn't work, nor does IMG_1945.JPG

What's the trick? I also tried it one-level up, specifying the directory name, but no luck. But I don't think that's the right place to run from.

I'm running this on OS X 10.3.8, with Python 2.3 installed.

Thanks!

--Darryl

This was a bug. When deciding what to update (and directory creation is a sort of update) it was case sensitive and only would see .jpg and not .JPG. I've fixed. Grab a new version at http://www.chezrutt.com/rutt/sm_tool/sm_tool.py

rutt · April 19, 2005

darryl wrote:

Just looked at the change notes for the new version. It's because my filenames were .JPG?

Argh.

--Darryl

Yeah, hope it works now. It's a case sensitive world on unix below the gui layer...

smugmug.py -- large scale batch smugmug tool

Comments