oAuth signature basestring

monkeychump

Hi.

I've been working on a connector for various photo hosting services, 3 of which use oAuth - smugmug, photobucket & picasa. I'm using the same oAuth library for all three site and photobucket and picasa are working fine.

I can successfully get a request token and then an access token from smugmug, using the the json endpoint http://api.smugmug.com/services/api/json/1.2.2/ with smugmug.auth.getRequestToken & smugmug.auth.getAccessToken.

However when i call smugmug.albums.get i get

code         35
message    invalid signature
method     smugmug.albums.get
stat          fail

this is my basestring (line breaks added for readability)

GET&http%3A%2F%2Fapi.smugmug.com%2Fservices%2Fapi%2Fjson%2F1.2.2%2F
&Extras%3DDescription%252CImageCount%26Heavy%3Dtrue
%26method%3Dsmugmug.albums.get
%26oauth_consumer_key%3Dkr991Xd7RWa7sMyhuOHu2gxooIwrrtiO
%26oauth_nonce%3DD52F1ED2AB6167EF3129A00D74CABDBED967A154
%26oauth_signature_method%3DHMAC-SHA1
%26oauth_timestamp%3D1219425202
%26oauth_token%3D074030869eebc2095602d5c193b75e21
%26oauth_version%3D1.0

and heres is the url i'm requesting, including the oauth_signature parameter

http://api.smugmug.com/services/api/json/1.2.2
/?Extras=Description%2CImageCount&Heavy=true&
method=smugmug.albums.get&
oauth_consumer_key=kr991Xd7RWa7sMyhuOHu2gxooIwrrtiO&
oauth_nonce=0257B2693EF3CBC5C1C4AA46399788D9CE6E5553&
oauth_signature=J4i8beHx79c%2FHev4IOMhMpbGelc%3D&
oauth_signature_method=HMAC-SHA1&oauth_timestamp=1219425333&
oauth_token=074030869eebc2095602d5c193b75e21&oauth_version=1.0

So that all looks fine to me, andlike i say, this library works with picasa and photobucket. photobucket also uses HMAC-SHA1, so i know this should work.

I did a bit of digging around and found that when i'm constructing the basestring, the library UrlEncodes both the name and value of each parameter before concatenating them into the basestring, if i stop the url encoding and just concatenate them then the library works with smugmug - but breaks with the other services.

I'm reading the oAuth spec and it says that encoding before concatenation is correct

9.1. Signature Base String

The Signature Base String is a consistent reproducible concatenation of the request elements into a single string. The string is used as an input in hashing or signing algorithms. The HMAC-SHA1 signature method provides both a standard and an example of using the Signature Base String with a signing algorithm to generate signatures. All the request parameters MUST be encoded as described in Parameter Encoding (Parameter Encoding) prior to constructing the Signature Base String.

So i guess my question is - is smugmug generating the basestring correctly when comparing my sig ?

sorry for the rather long winded post - this isn't the easiest subject to write concisely about

chris

monkeychump

more importantly to my question above - why would the first two calls to smugmug.auth.getRequestToken & smugmug.auth.getAccessToken work while smugmug.albums.get fails - they all use the same signing and request process ?

devbobo

Hey Chris,

I have my son's 4th birthday party today, so I will check this out earlier next week, if I don't get a chance tomorrow.

Cheers,

David

lildude

monkeychump wrote:

Hi.

I've been working on a connector for various photo hosting services, 3 of which use oAuth - smugmug, photobucket & picasa. I'm using the same oAuth library for all three site and photobucket and picasa are working fine.

I can successfully get a request token and then an access token from smugmug, using the the json endpoint http://api.smugmug.com/services/api/json/1.2.2/ with smugmug.auth.getRequestToken & smugmug.auth.getAccessToken.

However when i call smugmug.albums.get i get

code         35
message    invalid signature
method     smugmug.albums.get
stat          fail

this is my basestring (line breaks added for readability)

GET&http%3A%2F%2Fapi.smugmug.com%2Fservices%2Fapi%2Fjson%2F1.2.2%2F
&Extras%3DDescription%252CImageCount%26Heavy%3Dtrue
%26method%3Dsmugmug.albums.get
%26oauth_consumer_key%3Dkr991Xd7RWa7sMyhuOHu2gxooIwrrtiO
%26oauth_nonce%3DD52F1ED2AB6167EF3129A00D74CABDBED967A154
%26oauth_signature_method%3DHMAC-SHA1
%26oauth_timestamp%3D1219425202
%26oauth_token%3D074030869eebc2095602d5c193b75e21
%26oauth_version%3D1.0

and heres is the url i'm requesting, including the oauth_signature parameter

http://api.smugmug.com/services/api/json/1.2.2
/?Extras=Description%2CImageCount&Heavy=true&
method=smugmug.albums.get&
oauth_consumer_key=kr991Xd7RWa7sMyhuOHu2gxooIwrrtiO&
oauth_nonce=0257B2693EF3CBC5C1C4AA46399788D9CE6E5553&
oauth_signature=J4i8beHx79c%2FHev4IOMhMpbGelc%3D&
oauth_signature_method=HMAC-SHA1&oauth_timestamp=1219425333&
oauth_token=074030869eebc2095602d5c193b75e21&oauth_version=1.0

So that all looks fine to me, andlike i say, this library works with picasa and photobucket. photobucket also uses HMAC-SHA1, so i know this should work.

I did a bit of digging around and found that when i'm constructing the basestring, the library UrlEncodes both the name and value of each parameter before concatenating them into the basestring, if i stop the url encoding and just concatenate them then the library works with smugmug - but breaks with the other services.

I'm reading the oAuth spec and it says that encoding before concatenation is correct




So i guess my question is - is smugmug generating the basestring correctly when comparing my sig ?

sorry for the rather long winded post - this isn't the easiest subject to write concisely about

chris

Hi Chris

I suspect it's got something to do with the "Extras=Description,ImageCount" part of your query.

I encountered the same problem. I didn't directly mention it to David as I thought it was something I was doing wrong, however I did ask if he could update his OAuth Tester extension to include support for Extras so I could troubleshoot my issue.

As you're hitting it too, I don't think it's something I'm doing wrong

Anyway, you should find using only ONE "Extras" option (or PLAINTEXT signature method) will work, but more than one doesn't, which kind of defeats the point.

Your API request itself brings up another thing I'm curious about (if I'm hijacking, let me know and I'll post it separately)...

David, shouldn't "Extras" have precedence over "Heavy" and actually override the "Heavy" setting if it's included in the request?

My thinking here is "Extras" is effectively used to limit the output produced by a heavy call, however if you include "Heavy=true" in your query, you get the full heavy output and not just your "Extras" (ie the "Heavy" overrides the "Extras").

devbobo

lildude wrote:

David, shouldn't "Extras" have precedence over "Heavy" and actually override the "Heavy" setting if it's included in the request?

My thinking here is "Extras" is effectively used to limit the output produced by a heavy call, however if you include "Heavy=true" in your query, you get the full heavy output and not just your "Extras" (ie the "Heavy" overrides the "Extras").

Extras shouldn't really be sent with a Heavy call. The original intent of the 'Extras' parameter was to specify parameters you want in your response without having to do a Heavy call.

monkeychump

lildude wrote:

Hi Chris

I suspect it's got something to do with the "Extras=Description,ImageCount" part of your query.

Spot on, if I remove the Extras parameter and use Heavy instead it works fine.

Looks like this still points at smugmug not urlencoding the name=value pairs correctly before calculating the sig. As removing the comma means that all my paramter values are the same urlencoded or not.

Cheers, Chris

devbobo

Chris,

Looking closely at the spec, sections 9.1.1, 9.1.3 and appendix A.5.1.

Section 9.1.1 states that the normalized request parameter string (obtained from section 9.1.1) should be encoded along with the other two strings that make up the signature base string prior to concatenation. However, there is no mention of encoding at all within section 9.1.1. One example alludes to encoding the parameter value (not the entire parameter name/value pair string).

At this point, the only thing that I believe that I am potentially doing incorrectly is not encoding the parameter value while generating the normalized request parameter string. But I am going to seek another opinion on that prior to making any changes, as the code I initially based my implementation on, originated from OAuth.net.

Cheers,

David

9.1.1. Normalize Request Parameters

The request parameters are collected, sorted and concatenated into a normalized string:

• Parameters in the OAuth HTTP Authorization header (Authorization Header) excluding the realm parameter.
• Parameters in the HTTP POST request body (with a content-type of application/x-www-form-urlencoded).
• HTTP GET parameters added to the URLs in the query part (as defined by [RFC3986] (Berners-Lee, T., “Uniform Resource Identifiers (URI): Generic Syntax,” .) section 3).

The oauth_signature parameter MUST be excluded.
The parameters are normalized into a single string as follows:

1. Parameters are sorted by name, using lexicographical byte value ordering. If two or more parameters share the same name, they are sorted by their value. For example: a=1, c=hi%20there, f=25, f=50, f=a, z=p, z=t
   
2. Parameters are concatenated in their sorted order into a single string. For each parameter, the name is separated from the corresponding value by an ‘=’ character (ASCII code 61), even if the value is empty. Each name-value pair is separated by an ‘&’ character (ASCII code 38). For example: a=1&c=hi%20there&f=25&f=50&f=a&z=p&z=t
   

9.1.3. Concatenate Request Elements

The following items MUST be concatenated in order into a single string. Each item is encoded (Parameter Encoding) and separated by an ‘&’ character (ASCII code 38), even if empty.

1. The HTTP request method used to send the request. Value MUST be uppercase, for example: HEAD, GET , POST, etc.
2. The request URL from Section 9.1.2 (Construct Request URL).
3. The normalized request parameters string from Section 9.1.1 (Normalize Request Parameters).

See Signature Base String example in Appendix A.5.1 (Generating Signature Base String).

Appendix A.5.1. Generating Signature Base String

To generate the signature, it first needs to generate the Signature Base String. The request contains the following parameters (oauth_signature excluded) which are ordered and concatenated into a normalized string:

oauth_consumer_key: dpf43f3p2l4k3l03 oauth_token: nnch734d00sl2jdk oauth_signature_method: HMAC-SHA1 oauth_timestamp: 1191242096 oauth_nonce: kllo9940pd9333jh oauth_version: 1.0 file: vacation.jpg size: original

The following inputs are used to generate the Signature Base String:

1. GET
2. http://photos.example.net/photos
3. file=vacation.jpg&oauth_consumer_key=dpf43f3p2l4k3l03&oauth_nonce=kllo9940pd9333jh&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1191242096&oauth_token=nnch734d00sl2jdk&oauth_version=1.0&size=original

The Signature Base String is:
GET&http%3A%2F%2Fphotos.example.net%2Fphotos&file%3Dvacation.jpg%26oauth_consumer_key%3Ddpf43f3p2l4k3l03%26oauth_nonce%3Dkllo9940pd9333jh%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1191242096%26oauth_token%3Dnnch734d00sl2jdk%26oauth_version%3D1.0%26size%3Doriginal

monkeychump

Hi David,

Thanks for taking the time to look at this.

Did you base your code on the php library from oauth.net? I am using the CF library from there, which the author says is a direct port of the php library. I have confirmed on the oauth google group that it contains a number of bugs. See John Kristian's reply to my first post here where he explains the parameter names and values are encoded twice. This fits precisely with the patches i needed to make in order for my application to work with Google's oAuth implementation.

He also points to some useful test cases here

Cheers, Chris

devbobo

Hey Chris,

It's funny I actually emailed Chris Messina about this issue...and he forwarded the url to your post, not knowing that it was related

I have just committed some changes which should fix the parameter name/value encoding issue.

Hopefully they will get pushed live later in the week.

Cheers,

David

monkeychump

Hi David,

It's a small virtual world, this interweb place

I'm off away for a much need break for a few weeks, but i'll make a note to test my application against your changes when i get back.

Thanks for taking the time to look into this, i'm very impressed with the responsiveness!

Cheers, Chris

devbobo

Hey Chris,

Hope you have a great break

The changes are live.

Cheers,

David

monkeychump

not tested it thoroughly, but i can now provide Extras=Description,ImageCount parameter without getting an invalid signature error so looks like thats fixed it

Cheers, Chris

devbobo wrote:

Hey Chris,

Hope you have a great break

The changes are live.

Cheers,

David