What happens when an API post upload is interrupted?

rutt

In testing my new script, I've had some experiences with interrupting it once it gets going doing the wrong thing. The result seems to be a partial photo on the gallery. What is supposed to happen when the client drops the connection in the middle of an upload.

I think that using the "old style" post (decoded from smugmug pages with some help from Don) the photo wouldn't show up unless the upload actually completed.

Or am I missing something?

onethumb

rutt wrote:

In testing my new script, I've had some experiences with interrupting it once it gets going doing the wrong thing. The result seems to be a partial photo on the gallery. What is supposed to happen when the client drops the connection in the middle of an upload.

I think that using the "old style" post (decoded from smugmug pages with some help from Don) the photo wouldn't show up unless the upload actually completed.

Or am I missing something?

All smugmug methods result in a partial photo being processed. Part of the problem is that it's not really easy to tell whether we received the entire photo or not, particularly because there are people who have corrupt photos that they really do want up on smugmug even in their corrupt state (it's all they have left).

So yes, this is expected behavior. It happens with all of our uploaders, and it's by design since I can't think of another way to allow corrupted JPGs through without also allowing partial uploads.

Don

rutt

onethumb wrote:

It happens with all of our uploaders, and it's by design since I can't think of another way to allow corrupted JPGs through without also allowing partial uploads.

Don

Here is a suggestion. You mentioned that API uploads might return the ID of the uploaded image. What if they ACK using it? There are lots of other possibilities. I'd love to be able to querry a gallery and get back enough info to figure out that one of the images was incomplete and then be able to delete it.

onethumb

rutt wrote:

Here is a suggestion. You mentioned that API uploads might return the ID of the uploaded image. What if they ACK using it? There are lots of other possibilities. I'd love to be able to querry a gallery and get back enough info to figure out that one of the images was incomplete and then be able to delete it.

That's an interesting idea. I could probably easily send back, say, a byte count and an md5 sum.

Then your app could reply with a "Delete" request (once it's exposed), if they don't match up.

I suppose we could front-load it at some point too, whereby you send along an md5 sum and a bytecount in the initial POST and I compare before adding it to our DB. Doing both is probably wise, so we cover all bases.

More as I think and test...

Don

rutt

onethumb wrote:

That's an interesting idea. I could probably easily send back, say, a byte count and an md5 sum.

Then your app could reply with a "Delete" request (once it's exposed), if they don't match up.

I suppose we could front-load it at some point too, whereby you send along an md5 sum and a bytecount in the initial POST and I compare before adding it to our DB. Doing both is probably wise, so we cover all bases.

More as I think and test...

Don

Actually, I just looked over my code. I'm at some pains to add a content-length header for each part of the multipart thing. When this is present, you already know when the upload wasn't complete.

onethumb

onethumb wrote:

That's an interesting idea. I could probably easily send back, say, a byte count and an md5 sum.

Then your app could reply with a "Delete" request (once it's exposed), if they don't match up.

I suppose we could front-load it at some point too, whereby you send along an md5 sum and a bytecount in the initial POST and I compare before adding it to our DB. Doing both is probably wise, so we cover all bases.

More as I think and test...

Don

"ByteCount" and "MD5Sum" are now options for uploading and should solve this problem nicely.

See the docs for more info.

Don

Nikolai

Sweet!

onethumb wrote:

"ByteCount" and "MD5Sum" are now options for uploading and should solve this problem nicely.

See the docs for more info.

Don

Very nice! Would be fun to use it!
Cheers!