smugmug.images.get&Extras intermittent problem

jiminy · January 3, 2010

I have been using images.get&Extras to return a subset of the urls for images in a gallery( i.e. the thumb and medium urls ), then I download the images using those urls. This generally works fine, but sometimes when trying to download the image, I am returned a zero length content length.

This is intermittent and appears to go away if I simply retry the download a couple of times.

Is this an expected behavior or is there something that I should be doing differently to avoid the random zero length download responses?

thanks
Jim

BeachBill · January 4, 2010

I've seen this same behavior with some web based apps that use the API (including one I am working on) and would like to know why this happens too.

BeachBill · January 5, 2010

Okay, so here is an example. I pull the image URLs via the API and generate a page to display the thumbnails and link to the medium size images.

#link removed# (see my next message)

When I view the above page, I get broken links for the thumbnails and when I click on the links to view the medium image, it displays a blank page -- but here is the kicker, this seems to only happen with links to images in my smugmug account. I have tested with friends' smugmug accounts, and the thumbs and medium size image links work as expected.

I suspect that if anyone else tries to view the above page they will see the thumbs and can click to view the medium images.

I know the URLs are valid because if I manually try using them directly, after awhile I can get the medium image or thumbnail to display.

Is there some kind of security at play here on the Smugmug side? Has anyone encountered this before?

BeachBill · January 5, 2010

Just thinking out loud here... could this be caused by "External links" being turned off in the gallery?

If so, it makes sense that the thumbs and medium images are not displaying.

However it appears that Smugmug hiccups from time to time and actually allows the images to display.

jiminy · January 5, 2010

I have external links turned off in all of my galleries and it works fine most of the time. In fact, the last day and a half, I haven't been able to recreate the problem. Everything downloads just fine without a single content-length of zero. Whereas two days ago, I could reproduce it at will. Now that I have a network sniffer program ready to sniff what is happening, it doesn't seem to happen.

What I did notice when it happened was that the urls to the images appeared to take some time to become valid. If I retried getting an image that returned a zero length content length immediately, it usually failed. But if I waited about 30 seconds then it generally worked.

BeachBill · January 6, 2010

Now that I understand what is going on here, having external links off would prevent the images from being displayed, unless you already have the images cached from viewing them directly at Smugmug. Clear your browser cache and see if the zero length occurs again--it should.

The urls are the permanent links to your images so they will always be valid, but not if external links are off and you are attempting to view the images "external" from smugmug.

Hope this helps.

David PL · January 6, 2010

BeachBill wrote:

Now that I understand what is going on here, having external links off would prevent the images from being displayed, unless you already have the images cached from viewing them directly at Smugmug. Clear your browser cache and see if the zero length occurs again--it should.

The urls are the permanent links to your images so they will always be valid, but not if external links are off and you are attempting to view the images "external" from smugmug.

Hope this helps.

The API fully supports downloading any and all images (including those in galleries with external links off) as long as you log-in as "owner" and do the following (based on API 1.2.2):

When calling smugmug.images.get to retrieve the URLs, be sure to include &Sandboxed=1 to ensure the returned URLs refer to a location with a crossdomain.xml file
Then, when using the URLs returned from your call to smugmug.images.get, you must include the "SMSESS=YourSessionID" (replace "YourSessionID" with the SessionID returned by your login call) cookie in the header of your GET request. This will authenticate the request and allow your call to download all (including secured non-public images) from your SmugMug site.

BeachBill · January 6, 2010

Thanks for the info David. I'm not sure what jiminy is doing, but I've been working on something that is for public use (anonymous login). In the future I will do something with owner login so I will save this info for review later.

jiminy · January 7, 2010

Thanks Bill and David for your comments.

I am working on an iPhone app where the user is logged on to their account. The code is using objective C to download the images. It isn't using flash, so I'm not sure that the crossdomain.xml file comes into play.

I always send the SessionID that was returned from the smugmug.login.withPassword call on subsequent calls to smugmug.images.get&Extras. I think that should work equally well as sending the SMSESS cookie if I understand the documentation properly and in fact it works perfectly 99% of the time.

I tried setting the Sandboxed=1 to see if it alters any of the urls returned, but everything looks the same. I notice that each request for an image is always redirected by the smugmug server from an url like www. smugmug. com/photos/ to photos. smugmug. com/ photos/.

I'm speculating that possibly the redirected to url is not always valid immediately, hence I get back a content length of zero sometimes. However at the moment it is all working just fine without any hiccups, so I can't really test it to see what is happening.

jiminy · January 10, 2010

I managed to capture a network sniff of the issue last night. The server returned a redirect from the initial request for the file, then the server returned a 404 response code - file not found from the redirected url. 50 seconds later the same request succeeded. Any ideas on how to avoid these 404 return codes or is this just something to live due to server and network loading?

Here are the relevant traces copied from the packet sniffer:

... initial request… at 5.6 seconds
GET /photos/761109707_oJVxL-Th.jpg HTTP/1.1..Host: photos.smugmug.com..User-Agent: SyncPhoto/1.0.0 CFNetwork/459 Darwin/10.2.0..Accept: */*..Accept-Language: en-us..Accept-Encoding: gzip, deflate..Cookie: shott=[removed]; SMSESS=[removed]; snapp=[removed]..Connection: keep-alive....

... redirect from server...
HTTP/1.1 301 Moved Permanently..Date: Sun, 10 Jan 2010 04:01:03 GMT..Server: Apache..X-Powered-By: smugmug/1.2.0..X-SmugID: 27.27.207..Location: http:// photos.smugmug.com/photos/761109707_oJVxL-Th.jpg..Content-Length:0..Keep-Alive: timeout=5, max=866..Connection: Keep-Alive..Content-Type: text/html; charset=utf-8....

... redirected request to the new url provided above by the server...
GET /photos/761109707_oJVxL-Th.jpg HTTP/1.1..Host: photos.smugmug.com..User-Agent: SyncPhoto/1.0.0 CFNetwork/459Darwin/10.2.0..Accept: */*..Accept-Language: en-us..Accept-Encoding: gzip, deflate..Cookie: shott=[removed];SMSESS=[removed]; snapp=[removed]..Connection: keep-alive..

... 404 error file not found ...
HTTP/1.1 404 Not Found..Content-Type: text/html; charset=UTF-8..Server: Apache..X-Smug-Phone: 0.17453002929688..Content-Length: 0..Date: Sun, 10 Jan 2010 04:01:04 GMT..Connection: keep-alive..Vary: Accept-Encoding....

... 50 seconds later - first request...
GET /photos/761109707_oJVxL-Th.jpg HTTP/1.1..Host: www.smugmug.com..User-Agent: SyncPhoto/1.0.0 CFNetwork/459 Darwin/10.2.0..Accept: */*..Accept-Language: en-us..Accept-Encoding: gzip, deflate..Cookie: shott=[removed]; SMSESS=[removed]; snapp=[removed]..Connection:keep-alive

... redirect from the server ...
HTTP/1.1 301 Moved Permanently..Date: Sun, 10 Jan 2010 04:01:53 GMT..Server: Apache..X-Powered-By: smugmug/1.2.0..X-SmugID: 27.160.12..Location:http://photos.smugmug.com/photos/761109707_oJVxL-Th.jpg..Content-Length: 0..Keep-Alive: timeout=5, max=893..Connection: Keep-Alive..Content-Type: text/html; charset=utf-8....

... request to redirected url provided above ...
GET /photos/761109707_oJVxL-Th.jpg HTTP/1.1..Host: photos.smugmug.com..User-Agent: SyncPhoto/1.0.0 CFNetwork/459Darwin/10.2.0..Accept: */*..Accept-Language: en-us..Accept-Encoding: gzip, deflate..Cookie: shott=[removed];SMSESS=[removed]; snapp=[removed]..Connection: keep-alive..

... successfully retrieved the file and begins download...
HTTP/1.1 200 OK..Content-Type: image/jpeg..ETag:83432a8631d309e6ac1b8aecd0346222..Last-Modified: Sun, 10 Jan 2010 03:59:50 GMT..Server: Apache..X-Powered-By: smugmug/1.3.0..X-Robots-Tag: noarchive, noindex, nosnippet..X-Smug-Phone: 0.099683046340942..Content-Length: 10423..Cache-Control: public, max-age=31708800..Expires: Wed, 12 Jan2011 04:01:53 GMT..Date: Sun, 10 Jan 2010 04:01:53 GMT..Connection: keep-alive.
001a0 0a 0d 0a ff d8 ff e0 00 10 4a 46 49 46 00 01 01 .........JFIF...

[Mod edit] removed specific cookie values, please be careful about what cookie data you post online

devbobo · January 10, 2010

G'day Jim,

We are currently investigating an intermittent issue (basically what you described) with relation to password protected galleries or galleries within passworded protected sites and potentially having the 'External Links' option set to NO.

Cheers,

David

devbobo · January 11, 2010

This issue should be resolved now, can you please confirm.

Thanks,

David

jiminy · January 11, 2010

It appears to be working well now - no 404 errors, or any errors for that matter.

I'll post here if it seems to be happening again.

thanks
Jim

BeachBill · February 18, 2010

David PL wrote:

The API fully supports downloading any and all images (including those in galleries with external links off) as long as you log-in as "owner" and do the following (based on API 1.2.2):
When calling smugmug.images.get to retrieve the URLs, be sure to include &Sandboxed=1 to ensure the returned URLs refer to a location with a crossdomain.xml file

Then, when using the URLs returned from your call to smugmug.images.get, you must include the "SMSESS=YourSessionID" (replace "YourSessionID" with the SessionID returned by your login call) cookie in the header of your GET request. This will authenticate the request and allow your call to download all (including secured non-public images) from your SmugMug site.

David or anyone, I'm back to playing around with this. I'm trying to generate a web page of thumbnails and have requested smugmug.images.get with both Sandboxed=1 set and not set. I see the URL changes when setting Sandboxed, but I can't figure out how to get the images to display in the browser.

I'm doing the smugmug.images.get via the API 1.2.2 on the web server (after logging in as owner), generating a list of img src links. The web browser then attempts to display the images and I just get broken links. I'm trying to figure out how to set the SMSESS cookie as the GET request is coming from the end users web browser, not my web server.

Any hints or tips that would help me get the thumbnails to display on my third party website for a user logged in via my script?

beacazil · February 19, 2010

BeachBill wrote:

...

I'm trying to figure out how to set the SMSESS cookie as the GET request is coming from the end users web browser, not my web server.

Any hints or tips that would help me get the thumbnails to display on my third party website for a user logged in via my script?

If you can make the browser perform an automatic GET before it is time to fetch images, you can set the SMSESS cookie by making an API call and by specifying the sessionId as a query string parameter. Something simple and light-weight where you even ignore the result:

http://api.smugmug.com/services/api/rest/1.2.1/?method=smugmug.categories.get&SessionID=xxxxxxxxxxxxxxxxxxxxxxxxxxxx

BeachBill · February 21, 2010

beacazil wrote:

If you can make the browser perform an automatic GET before it is time to fetch images, you can set the SMSESS cookie by making an API call and by specifying the sessionId as a query string parameter.

Thanks for the idea. Now to see if I can come up with a good solution based on it!

BeachBill · February 27, 2010

I now have some really simple basic sample code (php using phpSmug and curl) that does the following:

Using 1.2.2 API

1) Login to smugmug as account owner (via phpSmug)
2) Make an API request (via curl) to set the SMSESS cookie (confirmed working)
3) Request image (via curl) to display

Results:
Images in External=yes galleries are displayed correctly.
Images in External=no galleries get 404 errors.

I'm still trying to figure out how to get the external=no images to display without 404 errors.

I would love it if devbobo or someone else who is very familiar with this could take a look at my logic and provide feedback.

BeachBill · February 27, 2010

I really should have posted my last message sooner.... because without any changes what-so-ever, my sample logic just started working after posting that message...

smugmug.images.get&Extras intermittent problem

Comments