Download All Zip Name No Longer Contains Album Name

SamirD

I've been meaning to report this for two days now, but haven't had time.

• Give us your site name - huntsvillecarscene.smugmug.com
• Describe the problem - When using the 'Download All' feature in a gallery, it no longer has the gallery name in the zip file.
• List the steps to replicate and recreate the problem - Find a gallery with photos in it, Tools-->Download All. Click on the link in your email and look at the filename. It used to start with the album name (probably using the nicename--I've never checked), now it will be 'album-45450454545....'.
• Provide links to your galleries, photos, etc so we can look - See this gallery:
  http://huntsvillecarscene.smugmug.com/gallery/Flyers/8990169_AHCvg
• Let us know your system, and browser and version (e.g., Firefox 3, on Windows) - Tested on the following with the same results:
  • 2.4ghz xph 512mb ff1 and ff3
  • p3-500 98se 128mb opera9
  • 1.8ghz 98se 512mb opera9
  • athlon x2 xpp 4gb ff2

docwalker

Thanks Samir, I have filed a bug report on it.

SamirD

Any idea on when it will be fixed? I can't easily tell one 2gb zip file from another anymore.

docwalker

Samir, I am afraid that I cannot tell you when any bug will be fixed. There are too many factors involved. Sorry. I wish I could.

SamirD

docwalker wrote: »

Samir, I am afraid that I cannot tell you when any bug will be fixed. There are too many factors involved. Sorry. I wish I could.

This is why building a business on SM is so difficult. You can't depend on features to work correctly and consistantly over a period of time. I have to shift my workflow several times each year each time a new feature breaks an old one, or an old feature is abandoned. It's pretty frustrating.

Is it too much to ask for something to just work? Day in, day out, for as long as someone needs it? That's what business is built on. You can't have a solid foundation if it keep shifting around.

docwalker

Samir,

Bad news. This was changed to support the gallery level pricing downloads feature. We will take a look to see if it is possible to add them back. But, I cannot promise anything. The programmer is aware that it is causing you trouble.

SamirD

I figured the bug was related to a recent change--I've seen an unusually large number of these over the years. That's just bad testing. I would've failed my data structures class in college if I would've let stuff like this slip through.

Thank you for the update Doc. I guess I'll figure out a way to work around it until its fixed. I don't have much of a choice.

Andy

SamirD wrote: »

I figured the bug was related to a recent change--I've seen an unusually large number of these over the years. That's just bad testing. I would've failed my data structures class in college if I would've let stuff like this slip through.

Thank you for the update Doc. I guess I'll figure out a way to work around it until its fixed. I don't have much of a choice.

I don't think you understood Doc, Samir. This change was intentional, it's not a bug. Album names in the zip filenames were causing us some problems.

At least one thing, it's pretty trivial to rename the folder once you've downloaded it.

I'm sorry we had to make this change.

SamirD

Andy wrote: »

I don't think you understood Doc, Samir. This change was intentional, it's not a bug. Album names in the zip filenames were causing us some problems.

At least one thing, it's pretty trivial to rename the folder once you've downloaded it.

I'm sorry we had to make this change.

Thank you for the clarification Andy.

Well that sucks. :cry It's trivial to rename a file, but not trivial to lookup what it was after you've waited 30mins for a couple of 2gb files to download via wget. But I know of a way around the issue. I've had to get good at having to get around SM issues over the years.

Andy

SamirD wrote: »

Thank you for the clarification Andy.

Well that sucks. :cry It's trivial to rename a file, but not trivial to lookup what it was after you've waited 30mins for a couple of 2gb files to download via wget.

The album name is in the email we send you

SamirD

I just noticed that, but since I'm using wget (to be a workaround for the packet issue that drops the http connection otherwise), the link get separated from the email. And the workaround I was thinking of didn't work, so I'm just screwed. One more problem I didn't need.

Andy

SamirD wrote: »

I just noticed that, but since I'm using wget (to be a workaround for the packet issue that drops the http connection otherwise), the link get separated from the email. And the workaround I was thinking of didn't work, so I'm just screwed. One more problem I didn't need.

I'm sorry I don't understand. You can rename the folder once you have it - won't that work?

SamirD

Andy wrote: »

I'm sorry I don't understand. You can rename the folder once you have it - won't that work?

I'll need to back up and explain a bit of the workflow for it to make sense.

I upload all the images from a particular day to one gallery. After verifying that all the files made it, I use the download all feature to download all the images and compare them with the originals on the cards before they're deleted.

Now while this is the workflow, it doesn't happen on a day-to-day basis, but many times multiple days at a time after I recover from shooting. So, the files resulting from the download all processes all get downloaded in one place, and because the filenames contain the gallery name, it's easy to figure out which one goes where.

But there's one detail that's missing from this workflow. Because the 1gb+ files never make it to me completely via any browser due to intermittent packet loss by my ISP, I have to use wget since it can automatically and correctly resume a download. While this does work, it uses the full url for the filename, making it very long. Now that these very long named files no longer contain the gallery name, figuring out which one was what after the long downloads it an extra step.

The workaround I planned was to simply put the downloaded file in the directory where I normally upzip the images too. But the problem I ran into is that the long filenames exceed the path spec for xp, so that's not possible. So now, I just have to unzip the files and then look at the resulting filenames and guess which gallery the directory is and rename the directory.

This is a much different workflow than before. I'm actually tempted to go back to using albumfetcher, but the speed of downloading a single file exceeds albumfetcher's speeds, which is why I changed the workflow to use the download all feature once it was stable.

And this illustrates a fundamental frustration that pros face with SM. If we depend on a SM feature for our workflow, we have no idea if one day our workflow will be disrupted, causing issues with wasted time, and ultimately, lost revenue.

Change is inevitable. It is the only constant. But to build a business, there must be stability. It's a false stability built on a changing foundation. I do business with the companies I do business with to help provide this stability--my web host, SM, rsync, custom programmed scripts, and more. When one of these fails, I have to take a step back to repair a fundamental block in the foundation of my business. Repairing these failures is a part of my job as a business owner, but they take my time away from moving forward in my business.

SM has been a key company in my business, fulfilling the core requirement of being able to allow thousands of photo views a day, as well as provide a way for visitors to purchase photo prints and gifts that can help pay for the SM account. But as the years have gone by, it's been a rocky road filled with a lot of steps forward and back. I've learned that almost every new feature release cannot be used dependably until after months of other users' 'testing' works out all the bugs. I've read that this is a fundamental part of SM's software development process, and that's fine. But the side effect is that these new features often break existing features that have been in production long enough to become a foundation block.

The space that SM is in has gone through a good shake down in the 5+ years I've been a customer, seeing many other companies come and go. And as the competition has become less, the demands for features has increased, especially with the advent of 'social' and the new pressures for innovation that this new direction has created. And to keep up, SM has been introducing new features at a record pace.

But the same problem of breaking old things when introducing new things are introduced is still here. And it's starting to take its toll. I've noticed other long-time SM customers like myself becoming more frustrated than ever before. This isn't a good sign, and something will have to be done. Because as people reach their breaking point, they will look into the marketplace for another service that fills their needs better and leave when they find one. And frankly, I say this from experience. SM almost lost me to Exposure Manager over the uploading issues that still plague me today. If it wasn't for the seamless addition of video a few years back, which was another one of my big problems, I would've left. And I've also looked at Zenfolio, but the lack of video there also keeps me from switching.

I wouldn't have taken the time to write this versus sleeping if I didn't truly care. And as you can tell from this short essay, I don't want to leave SM. I'd rather stick with a company I like to use if it will serve my needs, day-in and day-out. And I think a lot of us that are frustrated feel the same way.

I don't think this will change anything with the issue at hand, but I thought I'd just let you know my thoughts.

jfriend

SamirD wrote: »

I'll need to back up and explain a bit of the workflow for it to make sense.

...

Not that I'm likely to change your workflow Samir, but I have a workflow that sounds like a lot less work. And since it sounds like less work is important to you because it would leave more time for your business, I thought I'd explain.

I keep three copies of every image:

1. One on a working hard disk (where I normally access the images on my computer).
2. One on backup hard disks (backups from one hard disk to another generated nightly with SyncBackSE - an automated backup program).
3. One online with BackBlaze (an agent on my computer backs things up to the BackBlaze service automatically).

I use Smugmug as my viewing platform only so I only put things on Smugmug that I want to share. I use BackBlaze ($50/yr) for my online backups and it's a ton less work than your workflow. The BackBlaze engine works seamlessly in the background putting anything online that's in the sections of my hard disk that I've told BackBlaze to back up. The BackBlaze backup is completely automatic. Both backup systems are versioned so if something corrupts some files locally and that corruption is then backed up, you can retrieve the earlier version.

Things I edit (like my Lightroom catalog) are also backed up along with all my other documents (not just my images). It took me months to get everything up to BackBlaze initially (I just have one home DSL line), but now it just incrementally works in the background whenever I add add new photos to my computer.

I'm sure you'd have to adapt some of the processes you have now, but my workflow is a TON less work than what it sounds like you're doing. I let the BackBlaze and SyncBackSE software do the backup verification for me automatically (software is really good at boring, repetitive things). And, my workflow hasn't changed in numerous years. Every once in a while, I outgrow a hard disk and I have to do some juggling to rotate in a newer larger hard drive, but that's maybe once every 1-2 years.

All I do day-to-day is manage the images on my main working hard drive and the backup programs do all the backup work for me (both locally and online).

mbrady

SamirD wrote: »

But there's one detail that's missing from this workflow. Because the 1gb+ files never make it to me completely via any browser due to intermittent packet loss by my ISP, I have to use wget since it can automatically and correctly resume a download.

Who is your ISP? It sounds like this is the root of many of the problems. If the connection was reliable, then you wouldn't to have to go through all these workarounds with wget just to download the zip file. Is there anything they can do to improve the connection quality? Are there other ISP's available in your area that may provide better service?

MSkaffari

Using wget you can also override the output filename with -o option. wget http://www.example.com/thisisverylongfilename.txt -o a.txt

SamirD

jfriend wrote: »

Not that I'm likely to change your workflow Samir, but I have a workflow that sounds like a lot less work. And since it sounds like less work is important to you because it would leave more time for your business, I thought I'd explain.

I keep three copies of every image:

1. One on a working hard disk (where I normally access the images on my computer).
2. One on backup hard disks (backups from one hard disk to another generated nightly with SyncBackSE - an automated backup program).
3. One online with BackBlaze (an agent on my computer backs things up to the BackBlaze service automatically).

I use Smugmug as my viewing platform only so I only put things on Smugmug that I want to share. I use BackBlaze ($50/yr) for my online backups and it's a ton less work than your workflow. The BackBlaze engine works seamlessly in the background putting anything online that's in the sections of my hard disk that I've told BackBlaze to back up. The BackBlaze backup is completely automatic. Both backup systems are versioned so if something corrupts some files locally and that corruption is then backed up, you can retrieve the earlier version.

Things I edit (like my Lightroom catalog) are also backed up along with all my other documents (not just my images). It took me months to get everything up to BackBlaze initially (I just have one home DSL line), but now it just incrementally works in the background whenever I add add new photos to my computer.

I'm sure you'd have to adapt some of the processes you have now, but my workflow is a TON less work than what it sounds like you're doing. I let the BackBlaze and SyncBackSE software do the backup verification for me automatically (software is really good at boring, repetitive things). And, my workflow hasn't changed in numerous years. Every once in a while, I outgrow a hard disk and I have to do some juggling to rotate in a newer larger hard drive, but that's maybe once every 1-2 years.

All I do day-to-day is manage the images on my main working hard drive and the backup programs do all the backup work for me (both locally and online).

This would work if I use a single computer, but I use a 'distributed computing' model since I have to juggle tasks between all my old systems. I have two file servers in two different cities that are sync'd via vpn. These are my files. Any systems on either side of the network can access the files for work.

As far as images, because I can't do a post process due to my extreme turnaround time and lack of resources, I simply upload everything to SM while I sleep and then hide what doesn't make it and publish. Another gallery is for my daily dump that I download and compare with the original before deletion. Images are also copied to two different external drives that are compared with the original (the SM backup is a third backup). Videos and other non-jpg files are copies to a total of three drives. These drives are compared to each other on a yearly basis due to errors I've encountered with each yearly compare (documented elsewhere here on dgrin).

I've seriously looked into the online backup systems as you use them as well as my brother. But I'm not comfortable with the way do things--compression, just the changed bits, etc. It may work great for everyone else, but I know my luck, and I'd be the person it fails on.

If I had money to throw at it, I'd just get an Amazon s3 account and upload there and be done with it as that's enterprise class robust. But that would cost me hundreds a month.

SamirD

mbrady wrote: »

Who is your ISP? It sounds like this is the root of many of the problems. If the connection was reliable, then you wouldn't to have to go through all these workarounds with wget just to download the zip file. Is there anything they can do to improve the connection quality? Are there other ISP's available in your area that may provide better service?

I'm in Northern Alabama. The ISP is Knology and the ISPs divide the region into monopolies.

This is actually a smaller problem compared to the one I have with their new Arris C4 carrier equipment installed last year. It hammers my Cisco rv016 with 100 packets/sec on each line, making it believe it's under attack. I spent months with Cisco and Knology and neither of them knew why this was happening or how to prevent it. I gave them log files and all sorts of other stuff. I even spent a couple of hundred dollars and bought another rv016 because Cisco initially said my first one was bad.

Finally, I got two used Siemens SpeedStream routers for $3/each and put them on the two lines I don't use for VPN in front of the rv016. They stopped the packets from hitting the rv016. I only have to reboot the rv016 once a day now instead of it being useless. Too bad I lose 40mb of download bandwidth since the SpeedStreams seem limited to 5mb. It doesn't affect my upload and that's all I care about.

So you can see that if I call them and tell them I'm having intermittant packet loss, I have a bigger chance of an airplane landing on my modems than them fixing them.

Just like with SM, it's easier to find a workaround to issues than trying to fix them. Too bad too, this is what I've always seen in third world nations--nothing really fixed, just band-aided--and a whole country running that way. That should be a scary thought.

SamirD

MSkaffari wrote: »

Using wget you can also override the output filename with -o option. wget http://www.example.com/thisisverylongfilename.txt -o a.txt

Why didn't I think of this? :bash I'll try it!

Well, I looked into it, but that switch only outputs the results from wget to that file, like a log. :cry And I researched the wget switches for auto-renaming and it doesn't exist. :cry Good try though!

MSkaffari

Sorry, my bad. The switch I gave you is wrong and will re-direct the output. The switch you want is -O as in Capital O

wget http://www.example.com/thisisverylongfilename.txt -O yourfilename.txt

This works for you?

SamirD

I haven't tried it yet, but here's what I found on the switch in my research:
"Use of ‘-O’ is not intended to mean simply “use the name file instead of the one in the URL;” rather, it is analogous to shell redirection: ‘wget -O file http://foo’ is intended to work like ‘wget -O - http://foo > file’; file will be truncated immediately, and all downloaded content will be written there."

SamirD

Well, now I can't get the two most recent zips to download using wget. The message is "file not found". And it's crazy because I just changed the url on the command in the same window I used to download a 300mb+ zip.

Each one of these is the larger 1gb+ size, so I need to use wget since the browser will time-out.

Any assistance appreciated.

These **&($%$ problems are really testing my patience. I need this done in an hour. Why do I always have issues when I'm under pressure. It's like fate itself is against me.

Andy

SamirD wrote: »

Well, now I can't get the two most recent zips to download using wget. The message is "file not found". And it's crazy because I just changed the url on the command in the same window I used to download a 300mb+ zip.

Each one of these is the larger 1gb+ size, so I need to use wget since the browser will time-out.

Any assistance appreciated.

These **&($%$ problems are really testing my patience. I need this done in an hour. Why do I always have issues when I'm under pressure. It's like fate itself is against me.

Hi Samir, sorry I don't know what wget is. Did you request the zip recently? Can I have the album link you zipped?

SamirD

Here's the two links. They start fine in a browser, but history indicates they will time-out when a single packet doesn't make it to me, and I'll have to keep repeatedly retrying them until no end.
http://newpics.huntsvillecarscene.com/downloads/dfa9d04e611a368293fa/100453
http://newpics.huntsvillecarscene.com/downloads/dfa9d04e611a368293fa/100454

If the only answer will be to use a browser, then I'm just screwed as that's not a solution that will work. And my &*()% ISP isn't going to do a #$(*#$ thing about their problem.

SamirD

No help or suggestions? I've been trying with a browser for two days now and I've not seen past 300mb download without it timing out.

I think this issue with wget is revealing a bug with the new way of doing download all that breaks it when the zip has to be split into pieces. The final link to the file doesn't find the file even though the 302 redirects are followed correctly.

Andy

SamirD wrote: »

No help or suggestions? I've been trying with a browser for two days now and I've not seen past 300mb download without it timing out.

I think this issue with wget is revealing a bug with the new way of doing download all that breaks it when the zip has to be split into pieces. The final link to the file doesn't find the file even though the 302 redirects are followed correctly.

I'm sorry Samir - I just downloaded both ZIPs with no problem - I still don't understand wget, I just used the link and clicked it and saved. I'm kinda stuck here wondering what to say next as far as PD goes. Perhaps your connection is not reliable enough? Is your computer going to sleep? I'm reaching here...

SamirD

I'm mentally exhausted Andy. I really don't have the strength to go through explaining everything again. Google wget. Here's a link to the post where I explain the workflow involving wget and why:
http://www.dgrin.com/showpost.php?p=1586096&postcount=13

This is my last attempt to try this. I had to download the files 7 times before I got two complete ones. It's the ISPs fault, but they won't do anything about it. But SM having a system that will break with a single packet loss isn't really reliable either.

Andy

SamirD wrote: »

I'm mentally exhausted Andy. I really don't have the strength to go through explaining everything again. Google wget. Here's a link to the post where I explain the workflow involving wget and why:
http://www.dgrin.com/showpost.php?p=1586096&postcount=13

This is my last attempt to try this. I had to download the files 7 times before I got two complete ones. It's the ISPs fault, but they won't do anything about it. But SM having a system that will break with a single packet loss isn't really reliable either.

I'm really sorry this is troublesome for you and I wish I had a way to assist with your ISP. Let us know if there's anything we can do.

SamirD

Andy wrote: »

Let us know if there's anything we can do.

If your programmers can work with me to figure out why wget isn't following the redirection links anymore, that would fix the issue for me, and possibly some other rare use case scenarios where the system would break. I've never seen wget fail when a browser works, so something is definitely amiss.

Andy

SamirD wrote: »

If your programmers can work with me to figure out why wget isn't following the redirection links anymore, that would fix the issue for me, and possibly some other rare use case scenarios where the system would break. I've never seen wget fail when a browser works, so something is definitely amiss.

I wish I could, but our guys are heads-down into building new features. Maybe you can get wget hep from the community in the hacks forum here on Dgrin?

SamirD

Andy wrote: »

I wish I could, but our guys are heads-down into building new features. Maybe you can get wget hep from the community in the hacks forum here on Dgrin?

I can understand that, as it's the usual development course at SM, but there's something broken at the back end, and it's going to come out sooner or later. There's absolutely nothing wrong with wget. If you had googled it, you would read what a solid open source product this is.

I give up. Another broken thing at SM I can't use anymore. What a shame.