Smugmug SEO after new Google Algorithm Update (around Feb 15th) and slow sitemaps
KQuinlan
Registered Users Posts: 53 Big grins
I wanted to start a dialog about SEO in the new smugmug especially with concern to the new Google algorithm update of the middle of February 2014.
Ever since I switched to the new Smugmug (about 2 weeks after it came out) I had seen an improvement (not alot, but enough to notice) of my placing in google Search Engine Results. I had concluded that something about the new page code was more favorable to google's Search Engine Results algorithm. I even saw someone from smugmug mention that talks with someone from google (or something like that) resulted in some of the SEO changes they had made in the new smugmug.
Here is an example of my site's pageviews from right before my switch to the New Smugmug to a month or two after. The tiny spot where it hit zero was when the switch was made (before smugmug got google analytics working again
)
as you can see there was a small but significant noticible improvement in Pageviews after switching to new smugmug. Part of that might be SEO, and part might be that it looked better, and people stayed on the site longer.
From August through Late December 2003 Traffic stayed pretty steady (no major up or down trend).
Then on around Jan 1st 2014, there was a significant uptick in search engine rankings and website traffic, which stayed pretty steady until about the middle of February
Here is my pageviews from Dec15th 2013 to March 18th 2014
Starting the middle of february, it seems google changed pretty dramatically how it aranged the search engine results, and my traffic went pretty quickly down to almost nothing, as a direct result of being pushed for many of the (40+) keywords I ranked on the first page for, down to the 4th or lower page on many of those keywords.
Here is my Search Engine Impressions from Feb. 1st 2014 to now March 18th 2014
As I point of reference, I looked at both my other (non-smugmug) websites, and they both showed, if anything, a slight increase in search result ranking.
I have a feeling, something about the way the smugmug pages are laid out, or something about the page code is being penalized (not a manual WMT penalty, but a google algorithm penalty). I would love some input from smugmug about this, and if there is anything that can be done to get back to at least where we were last year in the Search Engine Results.
Searching about this issue on the internet resulted in reading a bit about a google algorithm update that people are referring to as "top heavy", because it seems to be penalizing websites with too many adds above the fold. (I do not have any adds on my site at all). I had four links to my other site for the same business, and I removed 2 of them to see if that helps at all (I thought maybe google was treating all external links as adds?)
Sitemap Issue:
One thing I have also recently noticed, that may be somewhat related: I have recently noticed this warning message by a few pages in "sitemap" in Google Web Master Tools
" Some URLs in the Sitemap have a high response time.
Some URLs listed in this Sitemap have a high response time. This may indicate a problem with your server or with the content of the page."
It seems a page or two crawled by google gets flagged with this message every 3 or 4 days. (there are 5 pages marked as slow now). I have my domain purchased directly through smugmug, and have just my smugmug account on the domain.
Thanks,
Kevin
Ever since I switched to the new Smugmug (about 2 weeks after it came out) I had seen an improvement (not alot, but enough to notice) of my placing in google Search Engine Results. I had concluded that something about the new page code was more favorable to google's Search Engine Results algorithm. I even saw someone from smugmug mention that talks with someone from google (or something like that) resulted in some of the SEO changes they had made in the new smugmug.
Here is an example of my site's pageviews from right before my switch to the New Smugmug to a month or two after. The tiny spot where it hit zero was when the switch was made (before smugmug got google analytics working again
)as you can see there was a small but significant noticible improvement in Pageviews after switching to new smugmug. Part of that might be SEO, and part might be that it looked better, and people stayed on the site longer.
From August through Late December 2003 Traffic stayed pretty steady (no major up or down trend).
Then on around Jan 1st 2014, there was a significant uptick in search engine rankings and website traffic, which stayed pretty steady until about the middle of February
Here is my pageviews from Dec15th 2013 to March 18th 2014
Starting the middle of february, it seems google changed pretty dramatically how it aranged the search engine results, and my traffic went pretty quickly down to almost nothing, as a direct result of being pushed for many of the (40+) keywords I ranked on the first page for, down to the 4th or lower page on many of those keywords.
Here is my Search Engine Impressions from Feb. 1st 2014 to now March 18th 2014
As I point of reference, I looked at both my other (non-smugmug) websites, and they both showed, if anything, a slight increase in search result ranking.
I have a feeling, something about the way the smugmug pages are laid out, or something about the page code is being penalized (not a manual WMT penalty, but a google algorithm penalty). I would love some input from smugmug about this, and if there is anything that can be done to get back to at least where we were last year in the Search Engine Results.
Searching about this issue on the internet resulted in reading a bit about a google algorithm update that people are referring to as "top heavy", because it seems to be penalizing websites with too many adds above the fold. (I do not have any adds on my site at all). I had four links to my other site for the same business, and I removed 2 of them to see if that helps at all (I thought maybe google was treating all external links as adds?)
Sitemap Issue:
One thing I have also recently noticed, that may be somewhat related: I have recently noticed this warning message by a few pages in "sitemap" in Google Web Master Tools
" Some URLs in the Sitemap have a high response time.
Some URLs listed in this Sitemap have a high response time. This may indicate a problem with your server or with the content of the page."
It seems a page or two crawled by google gets flagged with this message every 3 or 4 days. (there are 5 pages marked as slow now). I have my domain purchased directly through smugmug, and have just my smugmug account on the domain.
Thanks,
Kevin
0
Comments
Thanks for flagging this. I don't have an immediate answer for this but it sounds critically important, so over the next few hours we'll be investigating. I'll be back to you within a few hours with a status update.
Thanks,
Baldy
-Kevin
It took time for the results to show in Google. 4-5 days ago we fixed the bug we introduced, so hopefully you'll see your numbers return to where they were (or higher, since we think both issues are fixed).
Importantly, we believe the issue affected visibility, not SEO, so your ranking should return to where it was or higher once Google re-sees the pages that temporarily went dark.
This is a really painful bug, and I can't tell you how sorry we are this happened. Please let us know how your ranking tracks over the next few days/weeks.
-Kevin
I have the same issue here than KQuinan. I was ready to create a new post when I saw this one, so I'm more than happy to see your answer! Question: how long it's gonna take to have the crawled pages rank back to normal? As for today 03/20, nothing changed. Do we must be patient?
thanks.
This is the last 6 days... 3-11 to 3-17 could be a sign of an upward trend, but too small and too early to tell yet.
-Kevin
I got a warning on google web. tools about my sitemap today:
Warnings
URLs not accessible
When we tested a sample of the URLs from your Sitemap, we found that some URLs were not accessible to Googlebot due to an HTTP status error. All accessible URLs will still be submitted.
1
HTTP Error: 400
URL: /sitemap-galleryimages.xml.gz
Is it from the bug you discovered earlyer or is it from something I'm doing wrong?
Thanks for any advice!
"Some URLs listed in this Sitemap have a high response time. This may indicate a problem with your server or with the content of the page."
and it was detected on 3/20/14
I still have another Warning on there for:
"URLs not accessible
When we tested a sample of the URLs from your Sitemap, we found that some URLs were not accessible to Googlebot due to an HTTP status error. All accessible URLs will still be submitted."
Issues: 85
HTTP Error: 404
Although the 2nd one is listed as last detected Jan 19th 2014
I'm duplicating my last post in that thread here:
Here is my Google Index Status from the past year:
The last high point just short of the mid-point was from July 14, 2013. Things have been steadily going downhill from there. On 7/14/2013 the total indexed was 28,868. The latest total indexed is 3,578. That's a huge drop.
Note that I unveiled my new smug site on the day on which it was first available.
This is a serious problem, and one that I don't believe I caused.
I did just manually submit the site to index as I haven't done that recently. I'm hoping that improves things, but I shouldn't need to resubmit it. And note that there was a submission in September of last year; the site has been dropping since before then.
I am also seeing a large number of warnings, many that match those reported above.
Update / clarification on 24 March 2014:
I just looked a little deeper into the errors reported on the sitemaps above. As it turns out there are warnings but no errors. Interesting thing is that while the top page shows the date of the submission and the number of errors, they appear to be in the past and not from my most current submission.
--- Denise
Musings & ramblings at https://denisegoldberg.blogspot.com
URL Response Code News Error Detected
http://www.sebastienphotography.com/include/ 404 3/21/2014 Not found
http://www.sebastienphotography.com/contextual-ee 404 3/21/2014 Not found
http://www.sebastienphotography.com/loader/id 404 3/21/2014 Not found
http://www.sebastienphotography.com/wrap-events 404 3/21/2014 Not found
http://www.sebastienphotography.com/wrap-function 404 3/21/2014 Not found
http://www.sebastienphotography.com/wrap-raf 404 3/21/2014 Not found
http://www.sebastienphotography.com/wrap-timer 404 3/21/2014 Not found
http://www.sebastienphotography.com/wrap-xhr 404 3/21/2014 Not found
http://www.sebastienphotography.com/SebastienPhotography/contextual-ee 404 3/22/2014 Not found
http://www.sebastienphotography.com/SebastienPhotography/wrap-function 404 3/22/2014 Not found
I'm still concerned about the drop in the site index though.
Thanks to you and to Erik for your attention to this.
--- Denise
Musings & ramblings at https://denisegoldberg.blogspot.com
Any update by any chance?
thank you.
Sorry for the radio silence. I’ve been working with three other engineers to address the performance issues with Googlebot. While we are not done yet, we wanted to circle back with an update.
What have we done?
We studied our past data, and the data provided by some of you. We learned that the trend started even before we launched the new SmugMug. In order to address it, we’ve made some minor adjustments to the description meta-data tag for pages and folders. You will see the meta data tags updated for all of your folders and pages to properly reflect the values you set. Additionally, we identified some slow calls in the rendering of the page for Googlebot. For these, we’re taking a two phase approach. First we’ve identified a few page assets that were not delivering as optimally as we would like. We have already begun fixing these and have even pushed some live. Second, we have identified some assets that don’t make sense to serve to a robot. We will begin removing these over time.
What’s next?
We have added additional monitoring and alerting to our bot tier to ensure we know immediately if performance degrades. We have also identified a couple of additional resources that are loaded on each page that we will be speeding up. These will go live as we finish them. You may continue to see some slow page loads in your webmaster tools, but they should be decreasing in frequency already.
So for the burning question, how long until the world is set right again?
Unfortunately we just don’t know. Google searches on their own frequency and while we see lots and lots of traffic from Googlebot every day, we can’t be certain when it will crawl your site next. You can always submit the Sitemap file again in the webmaster tools to be sure it has the most up to date information.
Thanks,
Kevin
--- Denise
Musings & ramblings at https://denisegoldberg.blogspot.com
For a point of reference, it was top 10 for 196 keywords on of Feb 1st-2nd
Btw, not by way of complaining but to say our interests are aligned, Googlebot costs us roughly $1 million a year in compute and bandwidth costs, so we want to get the most out of that investment too.
:yikes
"You miss 100% of the shots you don't take" - Wayne Gretzky