I also just received another email from Google webmaster tools. The errors have not stopped, contrary to the post saying that the problem had been resolved on April 1st. Crawl errors against my site started on March 30th. They increased each day until April 4th and while they decreased slightly in the last two days the errors are still above 4400.
I'm also receiving these error messages. They occurred today again. Attached is the screenshot.
Can SM please keep me posted about what is happening here? I do not think sending back google 503 is right thing to do. Can SM please confirm that these errors will not effect google page ranks? Thanks.
Most of my ~3k errors are based on keyword links. Also a lot of errors on wedding photo pages, which is where I make my money to buy more gear, and pay for this site.
Yup, I've received 3 emails about this so far. The crawl errors continue. Between the lack of updates, the increase in pricing, and now this kind of nonsense, it seems increasingly naive that I rely so heavily on SM as part of my business. Sucks because I really used to love you guys. Amazing that I don't have these kinds of problems with my blog.
Got my first "crawl errors" email from Google today. I won't post a graph as mine looks the same as everyone else's, but I remain concerned. If Google isn't happy with my site, I'M not happy with my site.
Google has since stopped sending us the amount of traffic that would cause us to return an error, however the amount of errors Google has to report to you via the Webmaster tools will continue to happen until they process through all of them. Bottom line is, even though the errors are being displayed as today, yesterday, etc, that information from Google is simply not accurate. I understand you are all getting emails, and may continue too, but he has assured me all will return to normal soon.
I apologize for any inconvenience this is causing.
Why does SmugMug keep "turning away a good amount of this traffic"?
How can traffic to a site be overkill as you state?
Why didn't google issue a report for my blogspot account as well?
This is not the first time I have seen this, can something be done to correct this?
Do other photo hosting sites have this issue?
Any idea when the black slideshow button issue will be fixed?
Shawn,
This is Google Bot traffic. We would never turn away real traffic. Google didn't crawl your blogspot site like they did with SmugMug last week. This isn't an "across the board" thing Google did. For whatever reason, there bots decided to crawl our site a very high and way out of average amount last week. Google identified the issue and corrected it, so their bots are now crawling at normal rates, which will not result in us turning down the bot traffic.
If its just old messages for an issue thats since been resolevd then fair enough. Having said that I'm getting a new message in my webmaster tools interface:
here's something that illustrates the original problem though. For reasons unknown, google decided to index 4 times as many pages towards the end of March.
I still pose the question, why would SmugMug want to limit the amount of crawling the Google Bots are doing?
Is SmugMug not capable of handling it and if so why not?
Hardware?
Software?
Bandwidth?
Essentially, Google is getting no more information with the additional crawls. It is just a repeating crawl.
If its just old messages for an issue thats since been resolevd then fair enough. Having said that I'm getting a new message in my webmaster tools interface:
here's something that illustrates the original problem though. For reasons unknown, google decided to index 4 times as many pages towards the end of March.
Exactly, this was a un needed and rather large bump in crawls.
Lately there have been some DDos attacks on bank and newspaper sites in the Netherlands. It sounds almost the same as this Google problem: A ridiculous amount of data is sent to a server so that people or bots can't access it. I'm wondering if this problem with Google could be something similar?
The wisest men follow their own direction - Euripides
Lately there have been some DDos attacks on bank and newspaper sites in the Netherlands. It sounds almost the same as this Google problem: A ridiculous amount of data is sent to a server so that people or bots can't access it. I'm wondering if this problem with Google could be something similar?
This was just an internal issue with the Google Bots....Not an attack...although it would be fun to investigate an international government cyber attack.:cool
Comments
My Website
Facebook | Twitter | | VSCOgrid | Instagram |
www.facebook.com/manifestphoto
Me, too. Even though "fetch as googlebot" comes up aces. SM, can you let us know what's happening, thanks!
Portfolio • Workshops • Facebook • Twitter
--- Denise
Musings & ramblings at https://denisegoldberg.blogspot.com
Can SM please keep me posted about what is happening here? I do not think sending back google 503 is right thing to do. Can SM please confirm that these errors will not effect google page ranks? Thanks.
-Nitin
www.nitinkansal.com
www.facebook.com/MagicLightPhotography
Same as the other posts "Increase in server errors".
http://photos.frankcourtney.com/
Wedding Photographer Dublin
Screen Shot 2013-04-08 at 9.28.05 AM.jpg
My Photos
My Blog
On Google+
On DrivingLine
I have just had an extensive discussion with our ops engineer regarding this issue. After discussing it with him, I am positive that his analysis is 100% correct. in the last week of March and into April 1st, Google Bots sent a ridiculous amount of traffic to SmugMug sites. Per the graph Zac shared, we turned away a good amount of this traffic, as it was simply overkill. https://www.evernote.com/shard/s74/sh/9e9080e6-9a77-450c-bc93-dd257ea38735/b122d3f72054d2397ebbe29fff376ed5/res/5fd382e9-f1df-41d9-87e6-0b9892570ed9/skitch.png
Google has since stopped sending us the amount of traffic that would cause us to return an error, however the amount of errors Google has to report to you via the Webmaster tools will continue to happen until they process through all of them. Bottom line is, even though the errors are being displayed as today, yesterday, etc, that information from Google is simply not accurate. I understand you are all getting emails, and may continue too, but he has assured me all will return to normal soon.
I apologize for any inconvenience this is causing.
Michael
Facebook
Google+
Twitter
Photo Blog
Shawn,
This is Google Bot traffic. We would never turn away real traffic. Google didn't crawl your blogspot site like they did with SmugMug last week. This isn't an "across the board" thing Google did. For whatever reason, there bots decided to crawl our site a very high and way out of average amount last week. Google identified the issue and corrected it, so their bots are now crawling at normal rates, which will not result in us turning down the bot traffic.
You can follow the black Slideshow button issue on this thread.
[URL="http://dgrin.com/showthread.php?t=231228. [/URL]
Please keep this thread to discussions about the topic at hand only.
Thank you.
Michael
Facebook
Google+
Twitter
Photo Blog
Essentially, Google is getting no more information with the additional crawls. It is just a repeating crawl.
Facebook
Google+
Twitter
Photo Blog
This was caused by an outage on Saturday night, which is described on our Status blog at http://status.smugmug.com/services/entire-site/2013/4/7
You should be seeing this again.
I apologize for any inconvenience this may have caused.
Michael
Facebook
Google+
Twitter
Photo Blog
Exactly, this was a un needed and rather large bump in crawls.
Facebook
Google+
Twitter
Photo Blog
http://www.desiree-fotografie.nl
This was just an internal issue with the Google Bots....Not an attack...although it would be fun to investigate an international government cyber attack.:cool
Facebook
Google+
Twitter
Photo Blog
I just got a (second in 30 days) report, dated April 7th.
Thanks for the notice. Please give it a couple more days and let us know if it continues to happen.
Michael
Facebook
Google+
Twitter
Photo Blog
Just got my 4th or 5th email from Google - wish there was something I could do! :twitch
Screen Shot 2013-04-15 at 8.32.28 PM.jpg
My Website
Facebook | Twitter | | VSCOgrid | Instagram |
Also the same - 3rd April email tonight.
http://photos.frankcourtney.com/
Wedding Photographer Dublin
--- Denise
Musings & ramblings at https://denisegoldberg.blogspot.com
http://www.desiree-fotografie.nl
Michael Bonocore
www.mb-photography-sf.com
www.facebook.com/mbonocore
plus.mb-photography-sf.com
Facebook
Google+
Twitter
Photo Blog
Michael, I got the same email for my site as well. Thanks.
Portfolio • Workshops • Facebook • Twitter