HTTP Put upload troubles

Scotts · December 7, 2006

Hello Smuggers,

I am getting an error message when I try to upload a file.

This is the message:

<rsp stat="fail"><err code="4" msg="Wrong format. (ByteCount given: ,  received.  MD5Sum given: ,  actual.)"><rsp><rsp stat="fail">rsp stat="fail"
   err code="4" msg="Wrong format. (ByteCount given: ,  received.  MD5Sum given: ,  actual.)"
/rsp

Here is my header:

PUT /kids.jpg HTTP/1.1
Host: upload.smugmug.com
content-length: 87378
Content-MD5: a234ab01efe2775e9f69477831c3d3cf
X-Smug-SessionID: 0ff61184f893ed14b07e12f44a9c97d5
X-Smug-Version: 1.2.0
X-Smug-ResponseType: REST
X-Smug-AlbumID: 2160061

I am trying to upload another copy of this image
http://photos.shillcock.net/photos/112136707-M.jpg

Here is the response from calling images.getInfo for image </rsp></rsp></err></rsp>112136707

?xml version="1.0" encoding="utf-8"? rsp stat="ok"
   method smugmug.images.getInfo /method
   Image id="112136707" FileName="kids.jpg" Caption="" Keywords=" kids" Position="2" Date="2006-11-22 08:53:29" Format="JPG" Serial="0" Watermark="0" Size="87378" Width="640" Height="480" MD5Sum="a234ab01efe2775e9f69477831c3d3cf" LastUpdated="2006-12-04 14:22:46" OriginalURL="http://scottandwendy.smugmug.com/photos/112136707-O.jpg" LargeURL="http://scottandwendy.smugmug.com/photos/112136707-L.jpg" MediumURL="http://scottandwendy.smugmug.com/photos/112136707-M.jpg" SmallURL="http://scottandwendy.smugmug.com/photos/112136707-S.jpg" TinyURL="http://scottandwendy.smugmug.com/photos/112136707-Ti.jpg" ThumbURL="http://scottandwendy.smugmug.com/photos/112136707-Th.jpg" AlbumURL="http://scottandwendy.smugmug.com/gallery/2160061/1/112136707">
     Album id="2160061"
   /Image
 /rsp

As you can see the two md5 sums and size are the same. So I am not sure what I am doing wrong.

I do not quite know what is wrong with my format. Can anyone point me in the right direction?

I guess I should mention that I get the same error when trying to upload with RAW POST.

Thanks,
Scott

devbobo · December 7, 2006

Hi Scott,

Does it work fine if you don't supply the MD5 ?

The MD5 isn't required, but recommended. Give it a go without it, so that we can determine if it's an MD5 issue or not.

Cheers,

David

Scotts · December 8, 2006

I tried it without the md5, I tried with a bad md5 and I get the same results.

I got uploading working for multipart http post. Then I tried to send a bad md5 sum to it and I got this error as expected

msg=Wrong format. (ByteCount given: 87378, 87378 received. MD5Sum given: a234ab01efe2775e9f69477831c3d3ca, a234ab01efe2775e9f69477831c3d3cf actual.

So the previous error message is the same except it is missing the values that are compared

msg=Wrong format. (ByteCount given: ,  received.  MD5Sum given: ,  actual.

So now my question is why? Why is the result I get back missing the actual data that is was checking?

-Scott

devbobo · December 9, 2006

Scott,

The MD5 hash provided from the method smugmug.images.getInfo is the one for the original image. Since smugmug creates the S,M, L sizes, which includes stripping off EXIF data as well as some sharpening etc, the MD5 hash for the M size image is going to be complete different to the original.

Below are headers from uploading the Medium and Original images from SmugBrowser. You will see that the Original image has the MD5 hash that you were expecting, but not the Medium.

Cheers,

David

Scotts · December 9, 2006

David,

Thanks for trying to help me figure this out.

I am calculating the md5 sum for the image I am trying to upload. I was just pointing out that my sum matches smugmugs sum for the same image.

I can upload the photo using multipart http post but I would rather use http put. The code for it is smaller and easier to understand.

I am wondering why the error result I am getting back is missing the number of bytes uploaded and the md5 sum of what was uploaded.

-Scott

devbobo · December 9, 2006

well, what's weird is that I have no problem uploading that image with RAW HTTP posts.

What language are you using for your app ? Have you checked that you are actually sending the request that you are expecting using something like ethereal ?

Cheers,

David

Scotts · December 10, 2006

Hey David,

I am writting my program in c++ using the qt framework.

Here is my upload method

static const QString UploadEndpoint ("http://upload.smugmug.com/");

void
SmugMug::_uploadHttpPut (
      const QString &AlbumId,
      const QString &FileName,
      const QString &Caption) {

   QFileInfo fi (FileName);

   QUrl url (UploadEndpoint + fi.fileName ());

   _dataOut.clear ();

   QFile file (fi.absoluteFilePath ());
   if (file.open (QIODevice::ReadOnly)) {

      _dataOut = file.readAll ();
      file.close ();
   }

   QHttpRequestHeader header("PUT", url.path ());
   header.setValue("Host", url.host ());
   header.setContentLength (_dataOut.length ());
   header.setValue("Content-MD5", md5_digest (_dataOut));
   header.setValue("Content-Transfer-Encoding", "binary");
   header.setValue("X-Smug-SessionID", _sessionId);
   header.setValue("X-Smug-Version", "1.2.0");
   header.setValue("X-Smug-ResponseType", "REST");
   header.setValue("X-Smug-AlbumID", AlbumId);
   header.setValue("X-Smug-FileName", fi.fileName ());

   if (!Caption.isEmpty ()) {

      header.setValue("X-Smug-Caption", Caption);
   }

   _http.setHost (url.host (), url.port (80));
   _idImageUpload = _http.request (header, _dataOut);
}

Here is the header captured using Interarchy's built in packet sniffer.

PUT /kids.jpg HTTP/1.1
Host: upload.smugmug.com
content-length: 87378
Content-MD5: a234ab01efe2775e9f69477831c3d3cf
Content-Transfer-Encoding: binary
X-Smug-SessionID: 201e57717d7a787d7340b4c0de9392e7
X-Smug-Version: 1.2.0
X-Smug-ResponseType: REST
X-Smug-AlbumID: 2160061
X-Smug-FileName: kids.jpg
X-Smug-Caption: SmugMug Rocks!!!

I wish I new what else to look for. It just looks right to me. What I can not understand is that I can get the file to upload using mulitpart http post but not with raw http post and http put.

Any suggestion on what I can try next?

Thanks,
Scott

devbobo · December 10, 2006

I converted my raw HTTP post code into HTTP put, and the headers being sent look remarkably similar to yours...

PUT /112136707-O.jpg HTTP/1.1
Host: upload.smugmug.com
Content-Transfer-Encoding: binary
Content-MD5: a234ab01efe2775e9f69477831c3d3cf
Content-Length: 87378
X-Smug-SessionID: [snip]<snip>
X-Smug-Version: 1.2.0
X-Smug-ResponseType: REST
X-Smug-AlbumID: 1988859
X-Smug-FileName: 112136707-O.jpg

and the file uploads with no issues <img src="https://us.v-cdn.net/6029383/emoji/headscratch.gif" border="0" alt="" ></snip>

onethumb · December 14, 2006

Scotts wrote:

Here is my header:

PUT /kids.jpg HTTP/1.1
Host: upload.smugmug.com
content-length: 87378
Content-MD5: a234ab01efe2775e9f69477831c3d3cf
X-Smug-SessionID: 0ff61184f893ed14b07e12f44a9c97d5
X-Smug-Version: 1.2.0
X-Smug-ResponseType: REST
X-Smug-AlbumID: 2160061

Just a hunch, but your "Content-Length:" header isn't capped, but the HTTP spec says it should be.

Give that a whirl.

Don

Scotts · January 9, 2007

onethumb wrote:

Just a hunch, but your "Content-Length:" header isn't capped, but the HTTP spec says it should be.

Give that a whirl.

Don

Thanks Don,

I have looked into it on my end and the framework (Trolltech Qt4) I am using does not allow me to change the way it does content-length. I will file a bug with them for now. I am not sure what else to do if that is the case. Seems a little dumb to not allow all lower case. I will try some tests to verify if it is really the case.

Tanks,
Scott

devbobo · January 9, 2007

Hey Scott,

Perhaps try...

Instead of this...

header.setContentLength (_dataOut.length ());

use this...

header.setValue("Content-Length", _dataOut.length ());

Cheers,

David

Scotts · January 9, 2007

Thanks David,

I did try that but was getting a weird internal socket error. But now when I do it I get a http error. I will have to investigate a little more. Thanks again for the suggestions.

-Scott

Nebvin · January 11, 2007

You can always try doing both:

header.setContentLength (_dataOut.length ());
header.setValue("Content-Length", _dataOut.length ());
</pre>
And see if that works.

setContentLength is probably a simple function, and you could probably create a class that inherits from QHttpRequestHeader and write a new setContentLength if the above code works.

Been a while since I've worked with QT in C++ though.

Scotts wrote:

Thanks David,

I did try that but was getting a weird internal socket error. But now when I do it I get a http error. I will have to investigate a little more. Thanks again for the suggestions.

-Scott <img src="https://us.v-cdn.net/6029383/emoji/headscratch.gif" border="0" alt="" >

Scotts · January 12, 2007

Nebvin wrote:

You can always try doing both:

header.setContentLength (_dataOut.length ());
header.setValue("Content-Length", _dataOut.length ());

And see if that works.

setContentLength is probably a simple function, and you could probably create a class that inherits from QHttpRequestHeader and write a new setContentLength if the above code works.

Been a while since I've worked with QT in C++ though.

Thanks for the suggestion. I did try that already and it did not work

.

HTTP Put upload troubles

Comments