Slow processing on some images not overall

Ferguson

This is not about a particular event but I've observed over the last few months when I upload a number of photos (say 30-150 at a time is typical), one or two will stay in "processing" for very long periods of time, e.g. 20-120 minutes or more.

As an example, I uploaded this gallery about 30 minutes before I am writing this.

http://captivephotons.com/Events/FGCUvNKU031613/

All images except the first processed within moments. The first is still showing "processing" as I write this. Sometime later this morning I suspect it will be fine.

I used to try to delete them and reupload, I've just taken a more patient approach and all appear to eventually process. But when I'm turning a gallery over to someone to view, these look really confusing.

This does not happen all the time (I did a few dozen in another gallery before this one and it worked fine). It does happen probably one out of every two uploads of any size. I've seen no pattern as to which image -- ordinal position, size, etc. It just seems random, like all the others take the fast lane, and one or two get stuck behind a turnip truck and can't get past.

It's not a huge deal, but any ideas why?

By the way, it's not a browser issue -- I have tried multiple computers to check status, and cleared cache, etc.

Been typing slowly -- that shot is still processing.

rnrjosh

I took a look in this gallery, and the first shot has now completed processing (as you suspected). I looked at your upload log, and it is showing that this image took quite a bit longer to process than the other images in the gallery. I'm not really seeing a problem with that particular image, though it does seem to be one of the larger images in the gallery.

One thing I did notice was that the first photo in the gallery has both metadata for the Title and Caption, whereas the other photos in the gallery only include Title metadata (at least for the 5 or 6 photos I looked at so far). You might try a test and try uploading the same photo again, first, as-is, and second, after removing either the Title or Caption metadata from the image. I'm just working on a hunch here, but it's possible this may have some affect on the processing time.

Also note, any images that are currently in processing status won't be shown to visitors (you'll only see those "processing" thumbnails when logged in as the account owner).

I hope this helps!

Ferguson

Retry processed immediately (4 times)

rnrjosh wrote: »

One thing I did notice was that the first photo in the gallery has both metadata for the Title and Caption, whereas the other photos in the gallery only include Title metadata (at least for the 5 or 6 photos I looked at so far). You might try a test and try uploading the same photo again, first, as-is, and second, after removing either the Title or Caption metadata from the image. I'm just working on a hunch here, but it's possible this may have some affect on the processing time.

Also note, any images that are currently in processing status won't be shown to visitors (you'll only see those "processing" thumbnails when logged in as the account owner).
!

I cloned the image twice, removed the caption from one, and uploaded to a test gallery. I did this, then deleted it, then did it again. Both time they were processed in under two minutes.

That's consistent with past experience, I've often re-uploaded a shot with the intent of deleting the "processing", and it usually finishes immediately leaving the one stuck "processing" still in that state.

If the one without caption finished first it was at most a few seconds on these two tries.

Incidentally I did not know that the "processing" would not sure to visitors, that's good information.

rnrjosh

If both versions of the file (one with tltle + caption, and one with title only) both processed normally, I don't think there's anything in the file itself that's causing the issue (but thanks for checking!). There was likely just some delay in our processing systems when that photo was uploaded initially (which create the various display copies of the images), though I don't really have any way to see exactly why that particular image took longer to process than the others.

Ferguson

No One Else?
So no one else has this problem?

It happened again yesterday. I checked the logs, and all the images among about 2 dozen completed processing in 40-50s from the statistics. Except one, and it took 1851 seconds, over a half hour. It was nothing special, all shot at the same time, same two cameras.

Then this morning I deleted them (they had been reprocessed), uploaded about three times that number (including that same image again, slightly edited) and they all processed quickly, about a minute or less each.

So this evening I uploaded 70 images. These had a bit more variation but most were done in under 2 minutes, a couple a bit longer, but 4 near the end (but scattered) were ridiculously slow at 503 seconds, and three I left running and went to bed after about 25 minutes of waiting.

Is it just me? Or is no one else in a hurry to turn over a gallery?

GoofBckt

I Do have that problem . . . even worse. My photos appear to upload, then when I go to the gallery, nothing is there. I also had problems with the first 3 photos, sitting FOREVER "thinking" about uploading to no avail. Not a happy camper. I had to give a CD to a client this week because I couldn't get their gallery uploaded. I'm told it's my internet connection (HughesNet) but I've never had this problem before.

Ferguson wrote: »

So no one else has this problem?

It happened again yesterday. I checked the logs, and all the images among about 2 dozen completed processing in 40-50s from the statistics. Except one, and it took 1851 seconds, over a half hour. It was nothing special, all shot at the same time, same two cameras.

Then this morning I deleted them (they had been reprocessed), uploaded about three times that number (including that same image again, slightly edited) and they all processed quickly, about a minute or less each.

So this evening I uploaded 70 images. These had a bit more variation but most were done in under 2 minutes, a couple a bit longer, but 4 near the end (but scattered) were ridiculously slow at 503 seconds, and three I left running and went to bed after about 25 minutes of waiting.

Is it just me? Or is no one else in a hurry to turn over a gallery?

mbonocore

GoofBckt wrote: »

I Do have that problem . . . even worse. My photos appear to upload, then when I go to the gallery, nothing is there. I also had problems with the first 3 photos, sitting FOREVER "thinking" about uploading to no avail. Not a happy camper. I had to give a CD to a client this week because I couldn't get their gallery uploaded. I'm told it's my internet connection (HughesNet) but I've never had this problem before.

I will try to recreate this over the weekend. I have not seen this personally, but I upload almost solely via the Lightroom plug in. I will keep you posted.

Michael

Ferguson

Just to finish off the last example since I went to be before it finished: While most completed in 1-2 minutes, some slow ones were 503 seconds, 1937 seconds, 1940 seconds, and a whopping 3922 seconds or well over an hour. There is nothing unusual about these that I can see.

To GoofBck: I used JFriedl's lightroom upload plug-in for this batch, but I used the default web browser upload for the batch mentioned in the first post. I see no relationship to how uploaded. But to your HughesNet, I do networking for a living, and if you have upload (not processing) problems I would certainly suspect it first, due to the extreme latency. I know it's the only game in town for a lot of people.

In my case the upload is very fast (I'm on 50/10mbs cable), it's the processing that is an issue. And only for one or a very few random images.

SamirD

This reminds me of almost the exact same issue many years ago--you'd upload images and they'd sit on 'processing image' for sometimes hours. If you uploaded the exact same image again, it may process normally or it again may take hours.

The solution back then was to just delete the photo and re-upload it vs wait.

SamirD

Ferguson wrote: »

So no one else has this problem?

I ran into this personally for the first time today. I haven't seen it in years. I uploaded the exact same file again as a duplicate and it processed immediately. I just deleted the processing one--just like back in the day.

I hope this gets fixed soon. It was seriously annoying a few years ago.

Ferguson

SamirD wrote: »

I hope this gets fixed soon. It was seriously annoying a few years ago.

Any further thoughts from someone with SM?

Would this be better as a help desk ticket?

Ferguson

I uploaded another 146 images and took their total time and plotted it in units of 100 seconds. Some were longer than normal (maybe Monday morning is busy).

Of 146 images:

131 were under 10 minutes processing
12 were under 20 minutes
2 were in the 1900 second range
1 was 2196 seconds

The good news is all did finish. But that kind of variation is just weird, it's as though some images get kicked off into a very low priority queue somewhere.

I'd much rather see the overall average go up a bit and have them all finish about the same time, but that's a FACTOR of almost 100 variation.

SamirD

Ferguson wrote: »

Any further thoughts from someone with SM?

Would this be better as a help desk ticket?

It definitely wouldn't hurt. Someone should definitely file a bug report on this or bump the old bug report up since it's the same situation.

Ferguson wrote: »

I uploaded another 146 images and took their total time and plotted it in units of 100 seconds. Some were longer than normal (maybe Monday morning is busy).

Of 146 images:

131 were under 10 minutes processing
12 were under 20 minutes
2 were in the 1900 second range
1 was 2196 seconds

The good news is all did finish. But that kind of variation is just weird, it's as though some images get kicked off into a very low priority queue somewhere.

I'd much rather see the overall average go up a bit and have them all finish about the same time, but that's a FACTOR of almost 100 variation.

Very good analysis! This is exactly the type of information they're going to need to figure out the root cause. Back in the day, they would also process, sometimes after days. What they'll probably need is a gallery with some of these 'hung' images that they can trace while they're hung.

Ferguson

I wanted to revive this thread just to illustrate this is NOT a new vs. old Smugmug issue.

I've gone the support route, and was told "that's normal".

But this is a real pain. Last night I was trying to make a newspaper deadline to get some shots up where an editor could look over them and pick some.

Upload - fine, in plenty of time. Then for over 10 minutes one (and only one) shot just sat there processing. That may not sound like long, but the others were done within a minute or so of the upload itself finishing (which was long also as on a college wifi).

I've had some that took about 40 minutes, when dozens around them all average 10-20 seconds or less.

IT IS BROKEN. There's something that kicks some pictures into some kind of queue that just takes forever to run. MANY times I've re-uploaded the same image, and had the second upload process in seconds while the first is still showing processing -- it is NOT the image content alone that causes this.

I know a few other people are bothered by this -- is it just that 99% of the users are very patient?

But seriously -- Smugmug -- how can you say this is "normal". Say "it's not broken enough for us to want to fix it", but by what stretch of the imagination can you say someone designed the system to work this way -- after all, that's the only way it can be "normal" right?

migreen

I've had this problem as well . . in my case, its worse with the "new" smugmug. Since I'm not a journalist with tight deadlines, I've not been horribly inconvenienced by it, but it is odd, and always leaves me wondering whether it will ever finish.

Mitch

Ferguson

I'm going to just keep pushing this to the top occasionally and hope someone from Smugmug one day gets curious enough to look at what's wrong.

Today I uploaded 84 shots. By eyeball estimate each one processed in about 10-50 seconds.

One of them -- in no way different than I can see -- took 1904 seconds, or over 30 minutes.

It was neither the first, nor the last (it was #14). Same camera, similar size, similar metadata fields. Everything normal.

The shot is below (which for SM should give you all you need to look at the logs). That's about 100 TIMES as long as most of the shots.

http://www.captivephotons.com/Events/FGCUVB/Lipscomb101813/i-xQTmcmS

How can that be "normal"?

PS. I uploaded a screen shot but it required me to crop it so tight not sure how reasonable it is; 800 wide is pretty small for a screen shot.

denisegoldberg

I've been seeing this behavior too. The upload from my computer finishes quickly but the processing on the server that allows the photo to be viewed in the gallery is very inconsistent. One photo will be available almost immediately, another takes many minutes. I haven't been able to find a pattern; as reported by Ferguson the slow and fast are interspersed in the same gallery, same upload.

Here's an example - I uploaded the same image twice, shown at the top and bottom of this screen shot. Note that the first image still hasn't finished loading - even though it was submitted for upload 6 minutes before the successfully uploaded shot.



--- Denise

raebrown

I had the problem last night of some images just sitting in the 'processing' state. In my case the problem was having special characters in the file names. When I finally realized this might be a problem I removed them and the new uploads happened quickly. Not the reason in all cases but something to watch out for.

Ferguson

One upload session yesterday, at least two were still processing at 30 minutes, the rest done almost as soon as uploaded.