I'm unclear why there has to be such codependence between www.smugmug.com and customername.SmugMug.com wrt reserved URLs. Is this an insurmountable problem?
Posted from Nexus 7 using Tapatalk
Very good point, this whole issue seems to be an artifact of Smugmug's implementation, not something that has to be. These could very well be two completely separate pages if the implementation chose to support it:
John, unfortunately, there are a few URLs that people still type on the web and best practices always suggest you keep them simple and what customers expect: /help and /search are among the top ones. By changing those to /sm/help and /sm/search, we will confuse a lot of people who type those URLs in today and we break an Internet convention embraced by Google, Facebook, Amazon, Wordpress, Pinterest and the rest.
Part of this release was a big phantom renderer project because Google doesn't index JavaScript. Redirects of URLs is part of what we have to do for good SEO.
In my latest post, I didn't ask you to change the existing URLs. I asked you to put all new URLs in /sm/xxxx form so that you don't continue to make more conflicts with your customers as time passes. With your redirect strategy, every new page you deploy potentially breaks some unsuspecting customer's URL that communicated lowercase to their customers because it worked fine.
Also, it seems worthwhile investigating why customer's sites have to be in the same namespace at all as your smugmug.com page. After all, why can't these be completely separate pages:
Also, it seems worthwhile investigating why customer's sites have to be in the same namespace at all as your smugmug.com page. After all, why can't these be completely separate pages:
Yes, a few people like Paul have mentioned that and I'm with you, that's the perfect solution. I haven't been able to talk to the engineers about that so I don't yet know the gotchas, if there are any, or the scope. The engineers I have to ping for that are working insane hours with Amazon scaling the crazy traffic we're getting right now. When things settle I'll bring this up with them.
But honestly, changing to /sm/help would cause thousands of customers who now type in that URL to post here wondering how we could be so thoughtless as to change something they've been using for 10 years, and why did we break a basic paradigm of the net that all the leading Internet companies adhere to.
Yes, a few people like Paul have mentioned that and I'm with you, that's the perfect solution. I haven't been able to talk to the engineers about that so I don't yet know the gotchas, if there are any, or the scope. The engineers I have to ping for that are working insane hours with Amazon scaling the crazy traffic we're getting right now. When things settle I'll bring this up with them.
But honestly, changing to /sm/help would cause thousands of customers who now type in that URL to post here wondering how we could be so thoughtless as to change something they've been using for 10 years, and why did we break a basic paradigm of the net that all the leading Internet companies adhere to.
For my last several posts, I haven't been asking you to change /help to /sm/help - I get why you're glued to a few existing names. I'm not sure why you're still mentioning that. I did think it would be a good idea to stop making new conflicts everytime you make a new www.smugmug.com page though.
It would be awesome (best solution of everything discussed here) if there's some practical way to completely separate the www.smugmug.com namespace from the user sites. I'm very glad you will inquire about that.
Also, it seems worthwhile investigating why customer's sites have to be in the same namespace at all as your smugmug.com page. After all, why can't these be completely separate pages:
Honestly, it is actually worse than this. I just checked as I have a custom domain with a subdomain for my photos. I'm not posting it publicly (not that it will be hard to find) but subdomain.example.com/pro goes to a smugmug page. Given that my domain is my full name, it is a little irritating that my custom _subdomain_ is hijacked by smugmug for whatever purposes.
I have no pro category or whatever, and it should fail just like when any other random gibberish is thrown on the end. I am not smugmug and my name shouldn't be what people are signing up for on photos.example.com/signup.
0
Options
BaldyRegistered Users, Super ModeratorsPosts: 2,853moderator
I hadn't heard much (actually anything) about this. It's interesting to hear. With this change I started paying more attention to indexing, and that's one of the things making me think to move -- I can't find my own sites on Google, so I know no one else can.
Here's an example. I put up a gallery on 7/27/13 for the "Cape Coral Hurricanes".
In the gallery meta keywords I have "Hurricanes".
In the title I have it, in the filename of every single image I have it.
In the gallery description I have it.
But a very specific search of "site:www.captivephotons.com hurricanes" doesn't find it at all.
How is that possible?
I guarantee if I posted that on a little known site such as my personal site, as plain old HTML and images, it would find it.
So I'm glad you recognize the issue -- but the fix doesn't seem to work.
Hey Ferguson,
I can have the guys who wrote phantom renderer look into this and see what might be going on, but just to help me understand so I can describe the problem to them, is this not the result you were expecting from Google?
For my last several posts, I haven't been asking you to change /help to /sm/help - I get why you're glued to a few existing names. I'm not sure why you're still mentioning that. I did think it would be a good idea to stop making new conflicts everytime you make a new www.smugmug.com page though.
It would be awesome (best solution of everything discussed here) if there's some practical way to completely separate the www.smugmug.com namespace from the user sites. I'm very glad you will inquire about that.
Oh, okay, I understand now. You don't want to see us to create more, like smugmug.com/friendsports or whatever.
But I think we're all on the same page wrt the way this should work if it's possible. This all started, however, as a result of the case sensitive issue with this release. What we all agree would be the best solution doesn't have to do with this release.
One thing I have learned about the case sensitive issue is how much has to do with scaling the site and keeping it fast. So the fix that I mentioned we're working on may not be as easy as we all wish, but I'll have more news about that later. Right now, scaling the site is insane with all the traffic we're doing post announcement.
I can have the guys who wrote phantom renderer look into this and see what might be going on, but just to help me understand so I can describe the problem to them, is this not the result you were expecting from Google?
Thanks,
Baldy
Well, no. At least I think no.
First, I was searching for "Hurricanes" at that site, not the extra 2 words. That finds something else (that does have hte word Hurricane in it), but doesn't get the gallery in question.
With the longer name it finds what used to be a category, now a top level folder. The hierarchy looks like:
Home
--> Events
> Hurricanes (Title "Cape Coral Hurricanes") (Has Meta Keywords "Hurricanes")
> Tampa072713 (Title "Cape Coral Hurricanes vs Tampa") (has Meta Keywords "Hurricanes")
It is this latter where the images lie, and has a title with "Hurricanes" in it. It is that gallery that has the meta keywords, as does the folder above it.
I would expect Google to tag (instead or in addition) the lower level gallery and/or folder. The Events folder has exactly one "Hurricanes" and that in a contained folder.
But back to the original point -- why isn't it finding it just with "Hurricanes".
I wonder what it would have done if the top folder was "Pro Soccer Teams"?
What would you expect it to find in this kind of situation?
PS. I realize I should have put the keyword "Hurricanes" in the images also. I did not by omission, but am not changing it now that it's being discussed.
Oh, okay, I understand now. You don't want to see us to create more, like smugmug.com/friendsports or whatever.
But I think we're all on the same page wrt the way this should work if it's possible. This all started, however, as a result of the case sensitive issue with this release. What we all agree would be the best solution doesn't have to do with this release.
I think it does have to do with this release. /weddings used to work just fine. Once a user converts to new SM, they aren't allowed to have /weddings any more - they are forced to /Weddings and /weddings no longer works. As I understand it, this issue was introduced in the new release as /weddings use to work just fine.
The namespace collision between smugmug.com and user.smugmug.com has existed forever apparently - that I agree and slowly creating more collisions as time goes by and more www.smugmug.com pages are created. But the case sensivity of URLs and the issues that created started with the new SM release. On my legacy site, I can use either:
BaldyRegistered Users, Super ModeratorsPosts: 2,853moderator
edited August 12, 2013
Oh, boy. I have an update that isn't fun to give. I caught up with the guys (like Don) who made the decision to revert to case sensitivity on URLs and it was a decision based on speed and scalability. Everyone involved in performance and scaling has been locked in rooms not sleeping because we're getting a lot of surprises on the amount and type of traffic we're seeing.
The thing is the fix may be easy in terms of code but it looks to be very hard in terms of system speed and scaling. My guess is that's why Google doesn't even bother to redirect /News (which 404s at Google) to /news, because they don't want to make any sacrifices with performance.
We have to meet about this before I have another update for you and right now they're buried in scaling things like search and indexing of photos, because we're seeing a torrent like we never imagined.
Oh, boy. I have an update that isn't fun to give. I caught up with the guys (like Don) who made the decision to revert to case sensitivity on URLs and it was a decision based on speed and scalability. Everyone involved in performance and scaling has been locked in rooms not sleeping because we're getting a lot of surprises on the amount and type of traffic we're seeing.
The thing is the fix may be easy in terms of code but it looks to be very hard in terms of system speed and scaling. My guess is that's why Google doesn't even bother to redirect /News (which 404s at Google) to /news, because they don't want to make any sacrifices with performance.
We have to meet about this before I have another update for you and right now they're buried in scaling things like search and indexing of photos, because we're seeing a torrent like we never imagined.
I hope your team will figure this one out and I wish them best of luck.
right now they're buried in scaling things like search and indexing of photos, because we're seeing a torrent like we never imagined.
Why is this, out of interest? Lots of new signups following the publicity? Lots of existing customers (still) working on their sites? everyone showing off their new shinies to all their family and friends?
My guess is that's why Google doesn't even bother to redirect /News (which 404s at Google) to /news, because they don't want to make any sacrifices with performance.
I hope you all are able to figure something out here when the post-launch dust settles a bit.
As best I can tell Google just doesn't use upper case in URLs (it does appear in some query strings, but I didn't find any examples of upper case in URLs) so I think Google just didn't create this issue for themselves. Their URLs are all lowercase and probably always have been - no consumer confusion.
SM legacy used mixed case by default, but allowed viewers to be case insensitive and it still kept working. Now that is no longer the case, thus the viewer/site owner issues/confusion/broken links. This is a different issue than with Google.
I hope your team will figure this one out and I wish them best of luck.
That. This stuff isn't funny at the best of times but it's a million times worse when you are live with your changes. The updates are hugely appreciated though
But honestly, changing to /sm/help would cause thousands of customers who now type in that URL to post here wondering how we could be so thoughtless as to change something they've been using for 10 years, and why did we break a basic paradigm of the net that all the leading Internet companies adhere to.
Isn't this exactly what you did to us and our clients?
Why is this, out of interest? Lots of new signups following the publicity? Lots of existing customers (still) working on their sites? everyone showing off their new shinies to all their family and friends?
Hi Paul,
The performance guys are so buried I don't have really accurate numbers-based info. When the dust settles, we'll be able to look at more data and do our post-mortems, but I can give you my general feelings from what data I have seen and the areas we're working on.
One is we didn't expect so many people to migrate so quickly. And when they did, they started doing an insane number of operations with the new organizer: moving images, galleries, making folders 4 deep. It's one of those things where we can look back and say the old system was hard, the new organizer is easy, before they launch their sites they're going to do some major reorganization. But still I don't think we would have predicted this many move operations.
You seem to have some computing background so you can probably appreciate the computer science problem of syncing two sites when you haven't gone live yet. You're mapping a 5-level-deep system of folders with a different permission model to a 2-level category system where the user may upload to either one, edit on either side, etc. If you have an Apple laptop and desktop, keeping them in sync is a nightmare and they are the same systems. I think this is why Squarespace didn't try. They just came out with a new system and let their installed base start over if they wanted it.
A much bigger thing to scale has been search. For one thing, we've never seen image inflow like this. A lot of people are coming from Zenfolio, BluDomain and LiveBooks and bring their large collection of images. Many people who knew of SmugMug over the years decided to try us out and are syncing their photos with Lightroom. New customers take very little support on our help desk, but they upload like a firehose and and spend a lot of time with the organizer.
And finally, when tens of thousands of new SmugMug sites got published the first week, the owners showed them off on social media and generated insane image views. The popular style seems to be landscape collage, which really puts the images out there.
It has both a capital F and capital P and has all these years. I've always just copied and pasted the URL, so I never noticed. The problem would come in if I told people to type in /family/photos. In the past we did database lookups when a page not found error happened and we guessed that what I really meant was /Family/Photos, and we guessed right most of the time.
There was a performance penalty for that, which compounded with 5 levels deep.
So what we're seeing is pros putting /weddings on their business cards even though the URL being displayed in their browser is /Weddings. I don't blame them, we let it pass in the past and a lot of people would never notice or think the URLs had capital letters or that it mattered.
But for people who did notice and put /Weddings on their cards, the URL works as it always has.
It has both a capital F and capital P and has all these years. I've always just copied and pasted the URL, so I never noticed. The problem would come in if I told people to type in /family/photos. In the past we did database lookups when a page not found error happened and we guessed that what I really meant was /Family/Photos, and we guessed right most of the time.
There was a performance penalty for that, which compounded with 5 levels deep.
So what we're seeing is pros putting /weddings on their business cards even though the URL being displayed in their browser is /Weddings. I don't blame them, we let it pass in the past and a lot of people would never notice or think the URLs had capital letters or that it mattered.
But for people who did notice and put /Weddings on their cards, the URL works as it always has.
I hope this helps.
Thanks,
Baldy
I think we all get that. But you had a purposeful feature before that some people either purposefully (all lowercase is simpler to remember) or accidentally (just didn't think case mattered) relied upon and now that feature has been removed from new SM. Because of the removal of the feature some of those links are now busted.
Is it really a serious performance issue to do the DB lookup only when it otherwise would have been a 404? Are there that many 404s by regular users relative to all the other page views? It seems you're in a bind here. If there are that many 404s, then are you really happy letting them be 404s rather than taking the viewer to the page they probably wanted to go to? If there aren't that many 404s then can it really be that big a performance hit?
So what we're seeing is pros putting /weddings on their business cards even though the URL being displayed in their browser is /Weddings. I don't blame them, we let it pass in the past and a lot of people would never notice or think the URLs had capital letters or that it mattered.
The fact that SM kept the lowercase version for itself proves the importance.
Add to that all the threads about the many users whose clients are now experiencing the 404 error and it becomes pretty clear that this is an issue. It's been a long time since the internet required users to type case sensitive links, so every time we leave cards for clients not only do we have to remember to use a capital letter, we must put a disclaimer that our links are case sensitive.
The performance guys are so buried I don't have really accurate numbers-based info. When the dust settles, we'll be able to look at more data and do our post-mortems, but I can give you my general feelings from what data I have seen and the areas we're working on.
One is we didn't expect so many people to migrate so quickly. And when they did, they started doing an insane number of operations with the new organizer: moving images, galleries, making folders 4 deep. It's one of those things where we can look back and say the old system was hard, the new organizer is easy, before they launch their sites they're going to do some major reorganization. But still I don't think we would have predicted this many move operations.
You seem to have some computing background so you can probably appreciate the computer science problem of syncing two sites when you haven't gone live yet. You're mapping a 5-level-deep system of folders with a different permission model to a 2-level category system where the user may upload to either one, edit on either side, etc. If you have an Apple laptop and desktop, keeping them in sync is a nightmare and they are the same systems. I think this is why Squarespace didn't try. They just came out with a new system and let their installed base start over if they wanted it.
A much bigger thing to scale has been search. For one thing, we've never seen image inflow like this. A lot of people are coming from Zenfolio, BluDomain and LiveBooks and bring their large collection of images. Many people who knew of SmugMug over the years decided to try us out and are syncing their photos with Lightroom. New customers take very little support on our help desk, but they upload like a firehose and and spend a lot of time with the organizer.
And finally, when tens of thousands of new SmugMug sites got published the first week, the owners showed them off on social media and generated insane image views. The popular style seems to be landscape collage, which really puts the images out there.
Wow, yeah. This is an infrastructure nightmare. Thank goodness you guys planned for an elastic computing and storage services with Amazon. But still I could image many bells and whistles went off when hitting maximum thresholds and then exceeding beyond previous high water marks. And it sounds like it was quite a bit more too. Yet things managed to keep rolling and performance didn't really suffer. In fact in some ways it improved in page open times for my site. Having quick access is really important in this multi platform age. Excessive DB queries can really slow things down.
Any way to just make everything resolve to 'One' case right off the bat like lower (e.g myname.smugmug.com/stuff = MyName.SmugMug.Com/STUFF = MYNAME.SMUGMUG.COM/STUFF) without a DB Query? That would be my preference. I think you can handle this at the Load Balancer level with name resolution rules there like 'make everything lower case' or 'camel case' or 'case insensitive.' It depends on your load balancer. Cisco for example allows you set things as case insensitive. But if those name resolution rules greatly hamper performance then yes, they are hard to justify and enforce site wide. Performance and features are always considerations which at times require tradeoffs that can't please everyone all the time. I Always prefer faster as I know many users do. Maybe only the 404 could be handled with this type of rule - a fork or last ditch effort to get to the intended destination. For example if 404 then ignore case or make camel case. That way the bulk of other users who link to the correct address wouldn't take the performance hit the extra rules might incur. Still, I think case insensitivity right up front would be so much easier in the long run.
Synching problem. Yeah, nightmare times ten!!! Thanks for allowing us legacy folks to migrate in an automated way. But what a pain to maintain both! And for how long?!?
Love, love, love landscape collage!!! Give that graphic designer a BONUS, seriously!
It seems we could offload the cost of execution to the user's browser if we had javascript support, and the maintenance would fall to the smuggers. It's gotta be an edge case anyways in the grand scheme of things (not to minimize the pain, just trying to objectively discuss the millions of users that don't print url's on anything and the billions of users that don't type anything in by hand). Anyhow, I'm just thrilled I can start my gallery names with numbers (dates) now/again, like I had been kinda forced to do due to the 3 levels limitation of the old SM.
It seems we could offload the cost of execution to the user's browser if we had javascript support, and the maintenance would fall to the smuggers. It's gotta be an edge case anyways in the grand scheme of things (not to minimize the pain, just trying to objectively discuss the millions of users that don't print url's on anything and the billions of users that don't type anything in by hand). Anyhow, I'm just thrilled I can start my gallery names with numbers (dates) now/again, like I had been kinda forced to do due to the 3 levels limitation of the old SM.
Andy
Yeah, this occurred to me too. A small piece of JS could attempt to handle this on the client at no real cost to the SM infrastrure - if we were allowed to use JS on new SM. Yet another classic example of how powerful it is to be able to work around issues with a little JS in your customization.
I hadn't heard much (actually anything) about this. It's interesting to hear. With this change I started paying more attention to indexing, and that's one of the things making me think to move -- I can't find my own sites on Google, so I know no one else can.
Here's an example. I put up a gallery on 7/27/13 for the "Cape Coral Hurricanes".
In the gallery meta keywords I have "Hurricanes".
In the title I have it, in the filename of every single image I have it.
In the gallery description I have it.
But a very specific search of "site:www.captivephotons.com hurricanes" doesn't find it at all.
How is that possible?
I guarantee if I posted that on a little known site such as my personal site, as plain old HTML and images, it would find it.
So I'm glad you recognize the issue -- but the fix doesn't seem to work.
Hi Ferguson, I took a deeper look into your site and Google's index of it, here's what I found:
Searching for "site:www.captivephotons.com hurricanes" yields two results for your site: http://grab.by/ppC8 (Not logged in, no personal search results. This is important when comparing search results as Google customizes search results when logged in)
The page contains your images and content, so Google did find it.
I'm not sure when you did your search originally so I can't determine how/why your gallery did not appear. I do know that Google varies how often and how fast it indexes sites. Some it indexes every day if it notices new content frequently. Some it indexes every few weeks or longer.
Wow, yeah. This is an infrastructure nightmare. Thank goodness you guys planned for an elastic computing and storage services with Amazon.
Thanks, Derek. There was a time when we crossed the billion image threshold and we thought that was mind-blowing.
Imagine the sweat here last week when we fell a billion images behind for indexing search, as many of you noticed because so many of our features depend on various search indexes. One of the problems was we bumped against limits at Amazon. They are very responsive, but it still involved rewriting critical infrastructure code on our side to ingest photos faster.
I don't know how Amazon scales like they do, serving sites like Netflix, DropBox, etc. I can't imagine what their data centers look like.
We're really sorry about some of the scaling problems you did see.
It seems we could offload the cost of execution to the user's browser if we had javascript support, and the maintenance would fall to the smuggers. It's gotta be an edge case anyways in the grand scheme of things (not to minimize the pain, just trying to objectively discuss the millions of users that don't print url's on anything and the billions of users that don't type anything in by hand). Anyhow, I'm just thrilled I can start my gallery names with numbers (dates) now/again, like I had been kinda forced to do due to the 3 levels limitation of the old SM.
Andy
Interesting idea. The performance guys have had their hands full with migrations and search, but I will bring this up with them.
Imagine the sweat here last week when we fell a billion images behind for indexing search, as many of you noticed because so many of our features depend on various search indexes. One of the problems was we bumped against limits at Amazon. They are very responsive, but it still involved rewriting critical infrastructure code on our side to ingest photos faster.
...
We're really sorry about some of the scaling problems you did see.
Thanks for the first feedback on what is happening here... that seam to be the reason why the recent photos appear/disappear or why the search results differ from time to time or why smart galleries don't fetch all the photos.
Much appreciated. At least we do now know whats going on here...
1) There is no obvious way to start the migration process. This all happened right as I was having a new child, so I had no time to even think about it. Once the dust settled, I logged in to SmugMug to be greeted with the new look on the SM home page and links there to set up the new SmugMug as a new user - nothing for us legacy users anywhere that I can find on the home page or in our account pages.
2) Finally (after having to go to external search engines and dig around) found the link to start the process, click the button and (for over a week now) just get a message that migrations are on hold. That is fine, but really? No ETA, no 'get in line', no information whatsoever? Not a very customer friendly way to handle this process. Maybe I am just spoiled by the way every other vendor I work with handle these things but the 'normal' process I see these days is for the company to allow you to 'get in line' and then they send you an email (with the needed link) once they have the bandwidth/capacity for for you to join/start the process. I understand that you guys are swamped right now, so putting some type of 'get in line' process after the fact may not be possible, but could you at least put SOME indication of when things will open back up again in the message?
1) There is no obvious way to start the migration process. This all happened right as I was having a new child, so I had no time to even think about it. Once the dust settled, I logged in to SmugMug to be greeted with the new look on the SM home page and links there to set up the new SmugMug as a new user - nothing for us legacy users anywhere that I can find on the home page or in our account pages.
2) Finally (after having to go to external search engines and dig around) found the link to start the process, click the button and (for over a week now) just get a message that migrations are on hold. That is fine, but really? No ETA, no 'get in line', no information whatsoever? Not a very customer friendly way to handle this process. Maybe I am just spoiled by the way every other vendor I work with handle these things but the 'normal' process I see these days is for the company to allow you to 'get in line' and then they send you an email (with the needed link) once they have the bandwidth/capacity for for you to join/start the process. I understand that you guys are swamped right now, so putting some type of 'get in line' process after the fact may not be possible, but could you at least put SOME indication of when things will open back up again in the message?
I think the migration link says to come back on the 19th, doesn't it? We have put the migrations on hold until that date. Please come back to your site on August 19th and you will see the migrate button on your site. That's all there is to it - no need to get in line.
I think the migration link says to come back on the 19th, doesn't it? We have put the migrations on hold until that date. Please come back to your site on August 19th and you will see the migrate button on your site. That's all there is to it - no need to get in line.
It says that now. I didn't see a date yesterday - I too have been checking daily. Glad to have a date. I have migrated my personal (Power) site, but wanted to learn the new software before migrating my business (Business) site. Now that I've seen how amazing my personal site looks, I can barely stand to look at that old legacy format on my Business site.
It's really a huge improvement. I for one can excuse a few bumps in the road.
Comments
http://jfriend.smugmug.com/sports
http://www.smugmug.com/sports
Homepage • Popular
JFriend's javascript customizations • Secrets for getting fast answers on Dgrin
Always include a link to your site when posting a question
Also, it seems worthwhile investigating why customer's sites have to be in the same namespace at all as your smugmug.com page. After all, why can't these be completely separate pages:
http://www.smugmug.com/help
http://jfriend.smugmug.com/help
or these:
http://www.smugmug.com/sports
http://jfriend.smugmug.com/sports
Homepage • Popular
JFriend's javascript customizations • Secrets for getting fast answers on Dgrin
Always include a link to your site when posting a question
But honestly, changing to /sm/help would cause thousands of customers who now type in that URL to post here wondering how we could be so thoughtless as to change something they've been using for 10 years, and why did we break a basic paradigm of the net that all the leading Internet companies adhere to.
It would be awesome (best solution of everything discussed here) if there's some practical way to completely separate the www.smugmug.com namespace from the user sites. I'm very glad you will inquire about that.
Homepage • Popular
JFriend's javascript customizations • Secrets for getting fast answers on Dgrin
Always include a link to your site when posting a question
Honestly, it is actually worse than this. I just checked as I have a custom domain with a subdomain for my photos. I'm not posting it publicly (not that it will be hard to find) but subdomain.example.com/pro goes to a smugmug page. Given that my domain is my full name, it is a little irritating that my custom _subdomain_ is hijacked by smugmug for whatever purposes.
I have no pro category or whatever, and it should fail just like when any other random gibberish is thrown on the end. I am not smugmug and my name shouldn't be what people are signing up for on photos.example.com/signup.
I can have the guys who wrote phantom renderer look into this and see what might be going on, but just to help me understand so I can describe the problem to them, is this not the result you were expecting from Google?
Thanks,
Baldy
But I think we're all on the same page wrt the way this should work if it's possible. This all started, however, as a result of the case sensitive issue with this release. What we all agree would be the best solution doesn't have to do with this release.
One thing I have learned about the case sensitive issue is how much has to do with scaling the site and keeping it fast. So the fix that I mentioned we're working on may not be as easy as we all wish, but I'll have more news about that later. Right now, scaling the site is insane with all the traffic we're doing post announcement.
Well, no. At least I think no.
First, I was searching for "Hurricanes" at that site, not the extra 2 words. That finds something else (that does have hte word Hurricane in it), but doesn't get the gallery in question.
With the longer name it finds what used to be a category, now a top level folder. The hierarchy looks like:
Home
--> Events
> Hurricanes (Title "Cape Coral Hurricanes") (Has Meta Keywords "Hurricanes")
> Tampa072713 (Title "Cape Coral Hurricanes vs Tampa") (has Meta Keywords "Hurricanes")
It is this latter where the images lie, and has a title with "Hurricanes" in it. It is that gallery that has the meta keywords, as does the folder above it.
I would expect Google to tag (instead or in addition) the lower level gallery and/or folder. The Events folder has exactly one "Hurricanes" and that in a contained folder.
But back to the original point -- why isn't it finding it just with "Hurricanes".
I wonder what it would have done if the top folder was "Pro Soccer Teams"?
What would you expect it to find in this kind of situation?
PS. I realize I should have put the keyword "Hurricanes" in the images also. I did not by omission, but am not changing it now that it's being discussed.
The namespace collision between smugmug.com and user.smugmug.com has existed forever apparently - that I agree and slowly creating more collisions as time goes by and more www.smugmug.com pages are created. But the case sensivity of URLs and the issues that created started with the new SM release. On my legacy site, I can use either:
http://friend.smugmug.com/Sports
or
http://friend.smugmug.com/sports
and both go to the same place and both work. Not true with new SM sites.
Homepage • Popular
JFriend's javascript customizations • Secrets for getting fast answers on Dgrin
Always include a link to your site when posting a question
The thing is the fix may be easy in terms of code but it looks to be very hard in terms of system speed and scaling. My guess is that's why Google doesn't even bother to redirect /News (which 404s at Google) to /news, because they don't want to make any sacrifices with performance.
We have to meet about this before I have another update for you and right now they're buried in scaling things like search and indexing of photos, because we're seeing a torrent like we never imagined.
I hope your team will figure this one out and I wish them best of luck.
Why is this, out of interest? Lots of new signups following the publicity? Lots of existing customers (still) working on their sites? everyone showing off their new shinies to all their family and friends?
As best I can tell Google just doesn't use upper case in URLs (it does appear in some query strings, but I didn't find any examples of upper case in URLs) so I think Google just didn't create this issue for themselves. Their URLs are all lowercase and probably always have been - no consumer confusion.
SM legacy used mixed case by default, but allowed viewers to be case insensitive and it still kept working. Now that is no longer the case, thus the viewer/site owner issues/confusion/broken links. This is a different issue than with Google.
Homepage • Popular
JFriend's javascript customizations • Secrets for getting fast answers on Dgrin
Always include a link to your site when posting a question
That. This stuff isn't funny at the best of times but it's a million times worse when you are live with your changes. The updates are hugely appreciated though
Isn't this exactly what you did to us and our clients?
Website
The performance guys are so buried I don't have really accurate numbers-based info. When the dust settles, we'll be able to look at more data and do our post-mortems, but I can give you my general feelings from what data I have seen and the areas we're working on.
One is we didn't expect so many people to migrate so quickly. And when they did, they started doing an insane number of operations with the new organizer: moving images, galleries, making folders 4 deep. It's one of those things where we can look back and say the old system was hard, the new organizer is easy, before they launch their sites they're going to do some major reorganization. But still I don't think we would have predicted this many move operations.
You seem to have some computing background so you can probably appreciate the computer science problem of syncing two sites when you haven't gone live yet. You're mapping a 5-level-deep system of folders with a different permission model to a 2-level category system where the user may upload to either one, edit on either side, etc. If you have an Apple laptop and desktop, keeping them in sync is a nightmare and they are the same systems. I think this is why Squarespace didn't try. They just came out with a new system and let their installed base start over if they wanted it.
A much bigger thing to scale has been search. For one thing, we've never seen image inflow like this. A lot of people are coming from Zenfolio, BluDomain and LiveBooks and bring their large collection of images. Many people who knew of SmugMug over the years decided to try us out and are syncing their photos with Lightroom. New customers take very little support on our help desk, but they upload like a firehose and and spend a lot of time with the organizer.
And finally, when tens of thousands of new SmugMug sites got published the first week, the owners showed them off on social media and generated insane image views. The popular style seems to be landscape collage, which really puts the images out there.
For example, my family galleries have the URL:
http://cmac.smugmug.com/Family/Photos
It has both a capital F and capital P and has all these years. I've always just copied and pasted the URL, so I never noticed. The problem would come in if I told people to type in /family/photos. In the past we did database lookups when a page not found error happened and we guessed that what I really meant was /Family/Photos, and we guessed right most of the time.
There was a performance penalty for that, which compounded with 5 levels deep.
So what we're seeing is pros putting /weddings on their business cards even though the URL being displayed in their browser is /Weddings. I don't blame them, we let it pass in the past and a lot of people would never notice or think the URLs had capital letters or that it mattered.
But for people who did notice and put /Weddings on their cards, the URL works as it always has.
I hope this helps.
Thanks,
Baldy
Is it really a serious performance issue to do the DB lookup only when it otherwise would have been a 404? Are there that many 404s by regular users relative to all the other page views? It seems you're in a bind here. If there are that many 404s, then are you really happy letting them be 404s rather than taking the viewer to the page they probably wanted to go to? If there aren't that many 404s then can it really be that big a performance hit?
Homepage • Popular
JFriend's javascript customizations • Secrets for getting fast answers on Dgrin
Always include a link to your site when posting a question
The fact that SM kept the lowercase version for itself proves the importance.
Add to that all the threads about the many users whose clients are now experiencing the 404 error and it becomes pretty clear that this is an issue. It's been a long time since the internet required users to type case sensitive links, so every time we leave cards for clients not only do we have to remember to use a capital letter, we must put a disclaimer that our links are case sensitive.
Website
Wow, yeah. This is an infrastructure nightmare. Thank goodness you guys planned for an elastic computing and storage services with Amazon. But still I could image many bells and whistles went off when hitting maximum thresholds and then exceeding beyond previous high water marks. And it sounds like it was quite a bit more too. Yet things managed to keep rolling and performance didn't really suffer. In fact in some ways it improved in page open times for my site. Having quick access is really important in this multi platform age. Excessive DB queries can really slow things down.
Any way to just make everything resolve to 'One' case right off the bat like lower (e.g myname.smugmug.com/stuff = MyName.SmugMug.Com/STUFF = MYNAME.SMUGMUG.COM/STUFF) without a DB Query? That would be my preference. I think you can handle this at the Load Balancer level with name resolution rules there like 'make everything lower case' or 'camel case' or 'case insensitive.' It depends on your load balancer. Cisco for example allows you set things as case insensitive. But if those name resolution rules greatly hamper performance then yes, they are hard to justify and enforce site wide. Performance and features are always considerations which at times require tradeoffs that can't please everyone all the time. I Always prefer faster as I know many users do. Maybe only the 404 could be handled with this type of rule - a fork or last ditch effort to get to the intended destination. For example if 404 then ignore case or make camel case. That way the bulk of other users who link to the correct address wouldn't take the performance hit the extra rules might incur. Still, I think case insensitivity right up front would be so much easier in the long run.
Synching problem. Yeah, nightmare times ten!!! Thanks for allowing us legacy folks to migrate in an automated way. But what a pain to maintain both! And for how long?!?
Love, love, love landscape collage!!! Give that graphic designer a BONUS, seriously!
My Smugmug Gallery
Andy
Homepage • Popular
JFriend's javascript customizations • Secrets for getting fast answers on Dgrin
Always include a link to your site when posting a question
Hi Ferguson, I took a deeper look into your site and Google's index of it, here's what I found:
Searching for "site:www.captivephotons.com hurricanes" yields two results for your site: http://grab.by/ppC8 (Not logged in, no personal search results. This is important when comparing search results as Google customizes search results when logged in)
The first is the page you mention (I believe, the url wasn't specified in your original post). Google says it indexed the page on August 10th: http://webcache.googleusercontent.com/search?q=cache:2h64721q-BQJ:www.captivephotons.com/Events/Hurricanes/Tampa072713/30776095_zWDfrK+site:www.captivephotons.com+hurricanes&cd=1&hl=en&ct=clnk&gl=us
The page contains your images and content, so Google did find it.
I'm not sure when you did your search originally so I can't determine how/why your gallery did not appear. I do know that Google varies how often and how fast it indexes sites. Some it indexes every day if it notices new content frequently. Some it indexes every few weeks or longer.
What can give you more insight into google's indexing is setting up Google Webmaster tools for your site: https://www.google.com/webmasters/tools/home?hl=en
Let me know if you have any more questions!
Imagine the sweat here last week when we fell a billion images behind for indexing search, as many of you noticed because so many of our features depend on various search indexes. One of the problems was we bumped against limits at Amazon. They are very responsive, but it still involved rewriting critical infrastructure code on our side to ingest photos faster.
I don't know how Amazon scales like they do, serving sites like Netflix, DropBox, etc. I can't imagine what their data centers look like.
We're really sorry about some of the scaling problems you did see.
Thanks for the first feedback on what is happening here... that seam to be the reason why the recent photos appear/disappear or why the search results differ from time to time or why smart galleries don't fetch all the photos.
Much appreciated. At least we do now know whats going on here...
1) There is no obvious way to start the migration process. This all happened right as I was having a new child, so I had no time to even think about it. Once the dust settled, I logged in to SmugMug to be greeted with the new look on the SM home page and links there to set up the new SmugMug as a new user - nothing for us legacy users anywhere that I can find on the home page or in our account pages.
2) Finally (after having to go to external search engines and dig around) found the link to start the process, click the button and (for over a week now) just get a message that migrations are on hold. That is fine, but really? No ETA, no 'get in line', no information whatsoever? Not a very customer friendly way to handle this process. Maybe I am just spoiled by the way every other vendor I work with handle these things but the 'normal' process I see these days is for the company to allow you to 'get in line' and then they send you an email (with the needed link) once they have the bandwidth/capacity for for you to join/start the process. I understand that you guys are swamped right now, so putting some type of 'get in line' process after the fact may not be possible, but could you at least put SOME indication of when things will open back up again in the message?
I think the migration link says to come back on the 19th, doesn't it? We have put the migrations on hold until that date. Please come back to your site on August 19th and you will see the migrate button on your site. That's all there is to it - no need to get in line.
Support Hero
It says that now. I didn't see a date yesterday - I too have been checking daily. Glad to have a date. I have migrated my personal (Power) site, but wanted to learn the new software before migrating my business (Business) site. Now that I've seen how amazing my personal site looks, I can barely stand to look at that old legacy format on my Business site.
It's really a huge improvement. I for one can excuse a few bumps in the road.