Download all - borken

jcdill · February 17, 2011

I believe I just discovered/encountered a very serious bug with the Download All option. If I understand this bug properly, the first time someone clicks Download All in a gallery, you generate a popup saying that a zip file is being prepared and when ready you will send out an email with a link to download the file.

However, the second time someone clicks Download All, it immediately starts a download for the previously-prepared file.

BUT, when new photos are added to the gallery, they are not included in the zip file! The Download All file is cached and apparently remains unchanged for 2 weeks, even when more photos are added to the gallery.

This is causing serious problems for this account: http://travelquesttours.smugmug.com/Travel/Antartica2011/

Can the Download All zip files be deleted so that we can recreate them after all the photos have been uploaded for a gallery?

Allen · February 17, 2011

jcdill wrote: »

I believe I just discovered/encountered a very serious bug with the Download All option. If I understand this bug properly, the first time someone clicks Download All in a gallery, you generate a popup saying that a zip file is being prepared and when ready you will send out an email with a link to download the file.

However, the second time someone clicks Download All, it immediately starts a download for the previously-prepared file.

BUT, when new photos are added to the gallery, they are not included in the zip file! The Download All file is cached and apparently remains unchanged for 2 weeks, even when more photos are added to the gallery.

This is causing serious problems for this account: http://travelquesttours.smugmug.com/Travel/Antartica2011/

Can the Download All zip files be deleted so that we can recreate them after all the photos have been uploaded for a gallery?

This has been reported in the past. You'll have to contact Smug support
and they will delete the original zip before you create the new one.

jcdill · February 18, 2011

Allen wrote: »

This has been reported in the past. You'll have to contact Smug support
and they will delete the original zip before you create the new one.

Sigh. I wasted hours debugging this problem that smugmug claims is a "feature". When I asked that they just delete the "download all" zip files for the all galleries in one category (which contains about a dozen galleries) the reply said:

I asked you to please send gallery links. It would have saved me a lot of time, had you done that. There were only the first few that had the files.

I'm amazed (and disappointed) that I've been chastized for wasting a few minutes of their time fixing this known and undocumented bug/feature, after all the time I wasted trying to figure out why Download All wasn't working to, you know, download ALL the photos in the gallery. For this particular account (not my main account) this broken functionality is likely a deal breaker, and we will look for another site to share photos in the future.

jfriend · February 18, 2011

jcdill wrote: »

Sigh. I wasted hours debugging this problem that smugmug claims is a "feature". When I asked that they just delete the "download all" zip files for the all galleries in one category (which contains about a dozen galleries) the reply said:

I asked you to please send gallery links. It would have saved me a lot of time, had you done that. There were only the first few that had the files.

I'm amazed (and disappointed) that I've been chastized for wasting a few minutes of their time fixing this known and undocumented bug/feature, after all the time I wasted trying to figure out why Download All wasn't working to, you know, download ALL the photos in the gallery. For this particular account (not my main account) this broken functionality is likely a deal breaker, and we will look for another site to share photos in the future.

It's unfortunate that this feature got released the way it is. It IS seriously broken and I'm disappointed that Smugmug thought it was done enough to release. Much more, this issue was immediately found and reported and nothing has been done about it in all the time that has passed since the issue was reported. I'm not sure what they were thinking on this one when they first implemented it. How could they possibly not think that the images in a gallery might change over time and that Download All would need to represent the latest images or need to work a second time after the first link has expired? Anyone home at Smugmug on this one? Is it a reported bug in your system? Are you going to fix it? How soon (it's been awhile since this was first reported)?

Or should people who actually need this functionality to work reliably just go elsewhere? And, please don't say that people should just contact the help desk every time they want the feature to actually work. You should either fix this feature or withdraw it because right now it's either deceiving people (they think it represents the latest images in the gallery and works the second time) or just pisssing people off (because they find out it doesn't work beyond the first time). The functionality as it is should have never made it out of basic testing.

Sheaf · February 18, 2011

John, let's calm down and lose the gross exaggeration a bit, please?

The feature is far from broken and is a big improvement over what was in place (essentially third party tools) before it was built. It's a fairly rare use case and definitely not worth removing the entire feature over. It's not a trivial problem to solve, but we will fix it when we have a good solution and the engineering time to implement said solution.

We should keep track of when new files are added to the gallery and allow an option to regenerate if the files have changed.

There are a few reasons we don't auto-regenerate, but the main one is that it's prohibitively expensive to recreate it (and store it for the additional time period) when we're not sure the customer will even trigger the download again after adding more files.

jfriend · February 18, 2011

Sheaf wrote: »

John, let's calm down and lose the gross exaggeration a bit, please?

The feature is far from broken and is a big improvement over what was in place (essentially third party tools) before it was built. It's a fairly rare use case and definitely not worth removing the entire feature over. It's not a trivial problem to solve, but we will fix it when we have a good solution and the engineering time to implement said solution.

We do need to be more clear that the zip file is not automatically regenerated when new files are added to the gallery. Additionally, we should keep track of when new files are added to the gallery and notify the user that they will not be included when they click the button again. We should also allow an option to regenerate if the files have changed.

There are a few reasons we don't auto-regenerate, but the main one is that it's prohibitively expensive to recreate it when we're not sure the customer will even trigger the download again after adding more files.

There is one simple use case that it works for. All other use cases it has serious problems for and they are not that rare. I was under the impression that you guys wanted to be perceived like Apple where "things just work" rather than like Microsoft where "things just work if you learn our exact steps for how to do it and stick only to that plan". This feature only "just works" in the most simplest use case. It may be the most common use case, but it's certainly not the only one and it doesn't work at all for the other use cases, in fact it's downright misleading what it does.

I am not suggesting that you should automatically regenerate the ZIP everytime images are modified. I don't even think there's an expectation that that would happen. You generate a ZIP, it should have the images in it at the time you generated the ZIP. That's perfectly fine.

But, if you go to the function to generate a new ZIP, it HAS to pick up the state of the current images in the gallery. If you want to go to the trouble of figuring out whether the gallery has changed or not and give them the previous ZIP link if the gallery hasn't changed, that's all fine and dandy as your own internal optimization, but if you're not going to do that, then you have to generate a new ZIP with all the current images in it and with a new expiration date and if you give them the previous ZIP because nothing changed, you have to coin a new expiration date based on the current request time.

Whatever link the customer gets back has to represent a ZIP with all the current images in it. Customer asks for a ZIP file of the current images in the gallery. If you're not going to give them that (because they had previously done a ZIP before modifying the gallery) and then lead them to believe that's what they got, how can you not think that's a serious bug? The customer's expectations are not met in any fashion. Not only are you not communicating to the customer what actually happens, but getting the result they want requires voiding the previous ZIP file (which may or may not have already served it's purpose) and contacting Smugmug support to do that. This is a glaring use case that is simply missing from the implementation.

Yes, this means that there can potentially be multiple ZIP files at once (one for each request when the gallery is different). But, there is NO way of implementing this feature (of batch creating a ZIP of what's currently in the gallery) that just "does what the customer expects" that won't have multiple ZIP files lying around at some point. It's temporary storage only (until they expire and you delete them) so it shouldn't be that big a deal from the storage point of view.

If you only generate a new ZIP file and new link when the gallery has actually changed, you protect yourself from the user pressing the button lots of times when they only need it once. If you want to cap the number of ZIPs per gallery (at something like 3-5) to protect storage use and auto-remove older ones when generating the fifth one, that's all fine and dandy too (you may need to tell the owner that's happening).

Here's an actual use case I had. I had generated a ZIP/link and sent it to someone. Then, the same day someone else asked for a ZIP, but also asked for some changes in the gallery. I make those changes to the gallery and now want to get them a ZIP link. If I haven't read nearly every forum posting on dgrin, I'll request the ZIP, get a link and send it out. Lo and behold, it will be a link to the old ZIP and will not contain the changes. My second viewer will download the ZIP, decompress it, go through the images and either think I did a crappy job at doing what they asked me to do or think I just didn't do what I said I'd do because the ZIP won't contain those changes. At best, they will have wasted a bunch of time. At worst, they'll think poorly of my business.

If I happen to know how the system works (because I've read a lot of dgrin posts), I would know that now I'm in a bad spot. If I contact the help desk, all they can do is remove the current ZIP which will void the link I've sent to the first person. I have no way of knowing if they've already successfully downloaded things or not. If I ask support to nuke that ZIP, then I run the change that they go to download the ZIP from my original email and it isn't there. They contact me and tell me it didn't work. If I don't nuke the current ZIP, I can't make a new one with the latest changes. Either way, its not the experience I'm looking for with my viewers/customers. The only way to make this work seamless for everyone is to allow multiple ZIPs to exist at the same time so the original link can stay alive and have in it what everyone thought it was going to have in it and then to generate a new ZIP when I request one after modifying the gallery. Even automatically replacing the first ZIP at the same link when I request a second ZIP is potentially not what we want to have happen. There need to be multiple ZIPs when the gallery has changed between requests.

Sheaf · February 18, 2011

John, I'm not going to get into a long drawn out argument over this; I would rather get work done. We have data, we know it's a rare use case. We have had it on our to-do list for a while to fix it. I apologize that right now it doesn't function the way you want it to and I completely agree that it is confusing and/or broken for your use case.

jfriend · February 18, 2011

Sheaf wrote: »

John, I'm not going to get into a long drawn out argument over this; I would rather get work done. We have data, we know it's a rare use case. We have had it on our to-do list for a while to fix it. I apologize that right now it doesn't function the way you want it to and I completely agree that it is confusing and/or broken for your use case.

I'm glad you're considering improving it.

We have a difference of opinion on what use cases should have been covered before it was first released or how the feature is explained/presented to the customer given what it actually does or both.

jfriend · February 18, 2011

Question. How long after generation until the ZIP file is automatically removed?

Sheaf · February 18, 2011

jfriend wrote: »

Question. How long after generation until the ZIP file is automatically removed?

Two weeks.

onethumb · February 18, 2011

jfriend wrote: »

It's unfortunate that this feature got released the way it is. It IS seriously broken and I'm disappointed that Smugmug thought it was done enough to release. Much more, this issue was immediately found and reported and nothing has been done about it in all the time that has passed since the issue was reported. I'm not sure what they were thinking on this one when they first implemented it. How could they possibly not think that the images in a gallery might change over time and that Download All would need to represent the latest images or need to work a second time after the first link has expired? Anyone home at Smugmug on this one? Is it a reported bug in your system? Are you going to fix it? How soon (it's been awhile since this was first reported)?

Or should people who actually need this functionality to work reliably just go elsewhere? And, please don't say that people should just contact the help desk every time they want the feature to actually work. You should either fix this feature or withdraw it because right now it's either deceiving people (they think it represents the latest images in the gallery and works the second time) or just pisssing people off (because they find out it doesn't work beyond the first time). The functionality as it is should have never made it out of basic testing.

John, we've been over this before. You can keep getting upset about it if you'd like to - that's your right - but we *chose* to ship it the way it was.

SmugMug will always ship features and enhancements "early". It's been our policy for more than 8 years, and will continue to be our policy as long as I'm here.

When a feature helps >90% of the customers who want it (Download Zips is >99%), we'll ship it right then, in an "imperfect" state, rather than perfect a year from now. We're an Internet business, not a packaged software business. We don't have the luxury of getting it perfect every time, thus adding huge delays, because our customers demand features ASAP.

Helping 90% of our customers is a big win, and it lets us finish that remaining 10% with customer feedback and input - which means we'll get it done faster and more correctly. We think involving our customers in the evolution of the product if one of the things that make us different - and we can't truly involve them if we don't have working code in their hands to try.

I'm sorry that the feature doesn't work for you - but it's made thousands of our customers deliriously happy, and I'd ship it the way we did again and again if given the chance.

Allen · February 18, 2011

Looks to me when I'm logged in and click "download all" it's creating a zip with
all the originals. Then the zip link is sent out. I would think most of the time it
would not be the originals but smaller sizes for the client to view/use.
AlbumFetcher allows for any size to be downloaded. Create your zip and send
that.

If you're just d/l'ing your originals AlbumFetcher gets them very quick and no
fooling with a zip.

What am I missing? The only case I see for "download all" is sending originals to a client.

jfriend · February 18, 2011

onethumb wrote: »

John, we've been over this before. You can keep getting upset about it if you'd like to - that's your right - but we *chose* to ship it the way it was.

SmugMug will always ship features and enhancements "early". It's been our policy for more than 8 years, and will continue to be our policy as long as I'm here.

When a feature helps >90% of the customers who want it (Download Zips is >99%), we'll ship it right then, in an "imperfect" state, rather than perfect a year from now. We're an Internet business, not a packaged software business. We don't have the luxury of getting it perfect every time, thus adding huge delays, because our customers demand features ASAP.

Helping 90% of our customers is a big win, and it lets us finish that remaining 10% with customer feedback and input - which means we'll get it done faster and more correctly. We think involving our customers in the evolution of the product if one of the things that make us different - and we can't truly involve them if we don't have working code in their hands to try.

I'm sorry that the feature doesn't work for you - but it's made thousands of our customers deliriously happy, and I'd ship it the way we did again and again if given the chance.

Like I said politely, we have have a difference of opinion. You're entitled to your own opinion - I'm entitled to mine.

If you were knowingly going to ship it with these limitations, then you should have at least told the site owner that when they hit generate the second time that the ZIP file has not been updated and they should contact the helpdesk if they need it to reflect the latest gallery contents. As it is now they are misled into believing they've generated a ZIP with the current contents. That I think is just wrong.

If you told them, you would at least be communicating what they actually got and that the feature they want is not available.

jcdill · February 19, 2011

Sheaf wrote: »

We should keep track of when new files are added to the gallery and allow an option to regenerate if the files have changed.

At the very least!

The way it works now:

The first time the user clicks Download All they get a message that a zip file is being created and they will recieve an email link. (Good) Subsequent clicks work differently, the download (of the previously generated zip) starts immediately. But the user is not informed that they are receiving an older zip file, they think they are getting a download of ALL the files presently in the gallery. (Bad) After 2 weeks the old zip file disappears and suddenly Download All reverts to the first response, and the download suddenly contains more images than before (if new images have been added to the gallery). (Confusing)

It would solve a multitude of problems if the subsequent clicks produced a different popup, giving the user the option of A) downloading the current active zip file (provide creation date/time/number of images it contains) or

purging the old file and creating a new one (which will reflect any changes since the creation date of the current file).

For the use case where the user wants 2 active zip files (containing different sets of images) from one gallery at the same time, would it work to direct them to a work around of creating a second gallery - copying the total (now changed) contents of the gallery into a new gallery and then generating the 2nd zip from that new gallery? It's a bit more work, but it would at least WORK, which is better than what we have now (an opaque process that doesn't do what it says it does).

landonnoll · February 19, 2011

a work-around suggestion

Sheaf wrote: »

Two weeks.

Sheaf,

I have a compromise suggestion:

How about allowing an option that allows you to remove the zip file immediately? Perhaps a 'Reset Download All' or 'nuke zip file' tucked into the Other section of the "This Gallery" tools that would remove the zip file right away?

I encountered 3 cases where the "Download All" was done before the gallery was complete. I had to get the help desk to manually remove the zip files. If I had a way to remove the zip files myself (via some sort of 'Reset Download All' or 'nuke zip file' function) that would be MUCH better.

chongo (Landon Curt Noll) /\oo/\

mbrady · February 23, 2011

Sheaf wrote: »

We have data, we know it's a rare use case. We have had it on our to-do list for a while to fix it.

It's possible that it's less rare than you think. The site owner may send a zip to the first person, then add some files and send the zip to a second person. If the site owner doesn't know the zip is old, they will not know the new files are not included. Let's say the site owner sends an email to recipient 2 saying "here's all the photos from event xyz" and unless the recipient compares the zip contents with the web gallery contents they'll not know something is missing. In that case neither the sender nor recipient would realize something was broken and neither would contact Smugmug support.

Now this is hypothetical... there's no way to know if that's actually ever happened or if it does, how often (which is part of the problem, Smugmug would have no way of knowing how often this may or may not occur).

Sheaf · April 21, 2011

This should now function in a better manner. Thanks for your patience!

Sheaf · April 21, 2011

mbrady wrote: »

The site owner may send a zip to the first person, then add some files and send the zip to a second person.

We simply cannot recreate the zip file every time the gallery is changed. It's not economically feasible. The site owner in your example will have to choose to generate a new zip file and give that new link to the second person. That is now possible without directly contacting a support hero.

landonnoll · April 22, 2011

little need for a "smart" zip file

Sheaf wrote: »

We simply cannot recreate the zip file every time the gallery is changed. It's not economically feasible. The site owner in your example will have to choose to generate a new zip file and give that new link to the second person. That is now possible without directly contacting a support hero.

BTW: Speaking for just myself: I don't have a significant need to have the zip files be "smart" (i.e., recreate the zip file every time the gallery is changed). I only need the ability to manually request a zip file to be rebuilt without going through a support hero.

Sheaf wrote: »

This should now function in a better manner. Thanks for your patience!

Thanks for looking into this matter. How is it now functioning better? Is there something new that allows the owner to manually force the zip file to be rebuilt without contacting a support hero?

chongo (Landon Curt Noll) ^oo^

Sheaf · April 22, 2011

landonnoll wrote: »

How is it now functioning better? Is there something new that allows the owner to manually force the zip file to be rebuilt without contacting a support hero?

Yes. We now detect if a change has been made and give you the option in the gallery to either create a new zip or use the old one.

Download all - borken

Comments