Got a sitemap question? Look here.

15791011

Comments

  • richpepprichpepp Registered Users Posts: 360 Major grins
    edited May 30, 2011
    so does right click disabled affect google searching for pics

    I can't answer that question but the simple way to find out is just to switch it off and then track how many of your photos Google has over the next month or so (you really do need to wait that long). Remember that right click isn't any sort of protection. e.g. I got caught out during read-only mode last week when I had to hit a deadline but fortunately even though I had RCP on I still had originals enabled so it took seconds to get the photo downloaded even though I couldn't log in.

    You do need to watermark your photos and reduce the maximum resolution available. You have XL photos and no watermark - that really isn't any protection :(
    despite my hundreds of photos google only finds a couple when searching for my site.

    I suspect that your are talking about image search rather than regular Google web search. Looking at your sitemap-index.xml there are two .gz files which suggests that there are at least 1000 images in the sitemap. This means that Google knows about at least 1000 of your images but has chosen to only keep a few for the purposes of image search.

    Most likely this is because Google doesn't think your site is 'authoritative' enough (in Google speak) which basically means there aren't enough outside links to your site. I can only see a couple on Google which isn't enough. Google needs links from other places to get the idea that your site might be important and there just isn't any way around this (unless you are going to buy AdWords instead to get your site on the front page of course).
    It also only returns links to my site to associated keywords and not the proper galleries?

    Not quite true. If you search for your gallery descriptions they also come up quite a lot. It's just that the keywords come up as well and there are lots of those.

    So, watermark your images, turn down the maximum size available, turn off right click protection and see what happens. I don't think that right click protection is hiding images any more so I wouldn't expect to see any change but it would indeed be good to know.

    Then try getting links to your site and see what happens to the number of images. Try linking out to other relevant sites as well and see what happens e.g. for races link to any official site for the race. The idea is to make your site useful and informative. Try adding the usual blog and keep it going

    Remember, measure, try something, measure again and remember also it can take months to make a difference so persevere. There isn't a single big magic button, just lots of little buttons and you have to press as many as possible :)

    Good luck

    Rich
  • TwoofyTwoofy Registered Users Posts: 171 Major grins
    edited May 30, 2011
    caroline wrote: »
    Attachment not found.

    Thanks for taking a look Greg.

    Caroline

    This is a bit like unwrapping a present with many layers of wrapping paper. Would you be so kind as to capture a screen shot of what you see when you click that one with the yellow triangle? I think whatever is going on is in there. I just verified that file and it looks fine to me, but maybe there is something I missed.

    I'm guessing this is probably just a minor transient error when Google tried to fetch it, and it will try again (probably already has), but has not yet updates Webmaster Tools with the new information.

    - Greg
  • TwoofyTwoofy Registered Users Posts: 171 Major grins
    edited May 30, 2011
    richpepp wrote: »
    so does right click disabled affect google searching for pics, despite my hundreds of photos google only finds a couple when searching for my site.
    I can't answer that question but the simple way to find out is just to switch it off and then track how many of your photos Google has over the next month or so (you really do need to wait that long). Remember that right click isn't any sort of protection. e.g. I got caught out during read-only mode last week when I had to hit a deadline but fortunately even though I had RCP on I still had originals enabled so it took seconds to get the photo downloaded even though I couldn't log in.

    Rich


    If you flip back to page 4-6 of this thread you can see a bit of the discussion around it.

    So, yes, at one time right-click protection would prevent images from getting index by Google Image Search (just the image search - not web search).

    - Greg
  • WinsomeWorksWinsomeWorks Registered Users Posts: 1,935 Major grins
    edited May 30, 2011
    I went from 40 images with RCP enabled to over 500 without it enabled.
    Am I to understand that I can turn RCP back on and 500 images will still show up in a Google Image Search?
    I don't want to try it and find out I lose my 500 images from the image search.
    It has taken me quite awhile to get to this point.
    --Shawn
    I need to hear for 100% sure that this is changed now before I change anything on my site. (although I mostly have RCP on anyway except for family pics). I also would like to know for 100% certain that having keywords only visible to myself does not mean they don't get picked up by Google & the other search engines. I was told it makes no difference, but this RCP thing, which we thought for years wasn't supposed to be affecting our image searches, makes me question whether it's definitely true.
    Anna Lisa Yoder's Images - http://winsomeworks.com ... Handmade Photo Notecards: http://winsomeworks.etsy.com ... Framed/Matted work: http://anna-lisa-yoder.artistwebsites.com/galleries.html ... Scribbles: http://winsomeworks.blogspot.com
    DayBreak, my Folk Music Group (some free mp3s!) http://daybreakfolk.com
  • TwoofyTwoofy Registered Users Posts: 171 Major grins
    edited May 30, 2011
    You can turn RCP on.

    As for the search engines picking up hidden keywords, nothing has changed on our end with this - though the search engines are getting smarter all the time and this *could* be seen as a form of cloaking content. You could ask this question to a group of 10 knowledgable people and probably get at least 6 different answers.

    - Greg
  • chipjchipj Registered Users Posts: 149 Major grins
    edited May 30, 2011
    caroline wrote: »

    Attachment not found.

    Thanks for taking a look Greg.

    Caroline
    Caroline, It looks to me that this error was generated on April 19th. Unless there are errors newer than this date I really wouldn't worry about it all that much.

    What I would do is resubmit your sitemap and then check webmaster tools in a week or so. If the error hasn't cleared up by then, then you may need to check into it further. Google can be quite slow at removing reported errors (that have been taken care of) in Webmaster Tools.

    -anim8tr
  • carolinecaroline Registered Users Posts: 1,302 Major grins
    edited May 30, 2011
    Twoofy wrote: »
    This is a bit like unwrapping a present with many layers of wrapping paper. Would you be so kind as to capture a screen shot of what you see when you click that one with the yellow triangle? I think whatever is going on is in there. I just verified that file and it looks fine to me, but maybe there is something I missed.

    I'm guessing this is probably just a minor transient error when Google tried to fetch it, and it will try again (probably already has), but has not yet updates Webmaster Tools with the new information.

    - Greg
    Hi Greg, here it is:-

    sitemap2.jpg

    Caroline
    Mendip Blog - Blog from The Fog, life on the Mendips
    www.carolineshipsey.co.uk - Follow me on G+

    [/URL]
  • TwoofyTwoofy Registered Users Posts: 171 Major grins
    edited May 31, 2011
    caroline wrote: »
    Hi Greg, here it is:-

    sitemap2.jpg

    Caroline

    Hi Caroline, as anim8tr said this definitely looks like a temporary error - give it a few weeks and everything should be good.

    - Greg
  • richpepprichpepp Registered Users Posts: 360 Major grins
    edited May 31, 2011
    You can resubmit your sitemap to each search engine by logging into each site and then go to their respective sitemap submit tool.

    I might be wrong but I don't think that is actually necessary. The robots.txt file for our sites points to the sitemap for our sites so all search engines will know the sitemaps are there if they want it. Resubmitting won't hurt, it's just that as far as I know you don't need to worry about running round and telling all of the search engines about the sitemap. Our sitemap was read again yesterday by Google without us doing anything and without having submitted it to Google.
  • chipjchipj Registered Users Posts: 149 Major grins
    edited May 31, 2011
    richpepp wrote: »
    I might be wrong but I don't think that is actually necessary. The robots.txt file for our sites points to the sitemap for our sites so all search engines will know the sitemaps are there if they want it. Resubmitting won't hurt, it's just that as far as I know you don't need to worry about running round and telling all of the search engines about the sitemap. Our sitemap was read again yesterday by Google without us doing anything and without having submitted it to Google.

    richpepp, the advantage with re-submitting your sitemap is that it gets the search bots back to your site quicker than just waiting for them to show up. It can sometimes take them weeks to return to your site, unless they know there's new content to index. You let them know of new content by re-submitting your sitemap.

    If you're adding several photo's or have created a new category, sub-category or gallery, it's a good routine to get into, just be sure to not submit your sitemap too often (more than once a day).
  • richpepprichpepp Registered Users Posts: 360 Major grins
    edited May 31, 2011
    Fair enough, thanks for that. It seems to me that our sitemaps are being read every couple of days at the moment so it isn't worth my while resubmitting but I understand why you do it now. My guess though for most folks that they are probably better spending their time on building site links as that is what is most often missing
  • chipjchipj Registered Users Posts: 149 Major grins
    edited May 31, 2011
    richpepp wrote: »
    Fair enough, thanks for that. It seems to me that our sitemaps are being read every couple of days at the moment so it isn't worth my while resubmitting but I understand why you do it now. My guess though for most folks that they are probably better spending their time on building site links as that is what is most often missing

    Well, again it is optional, but I've honestly had better luck doing this than not. Google doesn't visit each and every page on each visit. I prefer they know about my new content to help ensure that they will visit those pages (links).

    According to Google "In most cases, webmasters will benefit from Sitemap submission, and in no case will you be penalized for it." (see page at https://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156184)
  • richpepprichpepp Registered Users Posts: 360 Major grins
    edited May 31, 2011
    anim8tr wrote: »
    "In most cases, webmasters will benefit from Sitemap submission, and in no case will you be penalized for it."

    I believe that quote refers to the initial submission (which SmugMug also do for you via robots.txt) and not to resubmitting the sitemap all the time. However if you are finding better results by resubmitting each time you update content then it is a good thing for you to do thumb.gif
  • chipjchipj Registered Users Posts: 149 Major grins
    edited May 31, 2011
    richpepp wrote: »
    I believe that quote refers to the initial submission (which SmugMug also do for you via robots.txt) and not to resubmitting the sitemap all the time. However if you are finding better results by resubmitting each time you update content then it is a good thing for you to do thumb.gif
    Thanks, again, I don't spend all my time doing this. I only resubmit if I've added a new category, subcategory, gallery or dozens of new images. It helps get my new content into the search index, and then onto the search engine results page more quickly.
  • pgmanpgman Registered Users Posts: 164 Major grins
    edited June 1, 2011
    chipj wrote: »
    richpepp, the advantage with re-submitting your sitemap is that it gets the search bots back to your site quicker than just waiting for them to show up. It can sometimes take them weeks to return to your site, unless they know there's new content to index. You let them know of new content by re-submitting your sitemap.

    If you're adding several photo's or have created a new category, sub-category or gallery, it's a good routine to get into, just be sure to not submit your sitemap too often (more than once a day).
    My experience is that you shouldn't.
    1. SM doesn't update the sitemaps immediately. My understanding is that the sitemaps are a background process. I've had one occurrence when it took 6 days for SM to update the sitemap. Is this general? I don't know.
    2. In the overall scheme of thing you will be penalized for resubmitting identical sitemaps. How badly penalized? Not too bad but it's a negative as part of the 200 signals that Google uses, just like website speed...

    My experience with SM is that it took 2 month from opening my pro account with SM to have Google stabilized with indexing..., but I link from my fairly busy website to my SM site: http://www.foto-biz.com to http://photos.foto-biz.com by link to the image and the gallery.

    ----
    Thanks
    http://foto-biz.com
    The Business of Being a Photographer -- Lightroom Q&A
    Thanks
    http://www.sritch.com
    The Dogs of Vancouver, BC
  • chipjchipj Registered Users Posts: 149 Major grins
    edited June 1, 2011
    Well, you're talking two different processes here. SmugMug updates the sitemap automatically (according to them) when you make a change. There's nothing to be done there. What they don't do is submit the updated sitemap to the search engines after they've been updated.

    Now, again I'm not saying you need to submit your sitemap with each and every change you make. What I am saying is that if you have a change to your site that involves a new category, subcategory or several new images, it is in your best interest to resubmit your sitemap.

    Google heartily endorses this because they want to know about new content. This is also one of the main reasons why blogs do so well in search engine rankings... they automatically ping (or submit a sitemap) each time you make a change to your content.

    I've seen the positive aspects of re-submitting a sitemap time and time again. In fact look at a web development tool called SiteBuildIt. This is a site development tool for beginners. Ironically, their sites tend to rank very well, and one of the main reasons they do is because the SiteBuildIt platform re-submits an updated sitemap on a nightly basis.

    I'm not trying to come off as a know-it all here, but it's an important element that everyone should be aware of. If you can provide proof or documentation that Google penalizes you for doing this, I'd love to see it.
    pgman wrote: »
    My experience is that you shouldn't.
    1. SM doesn't update the sitemaps immediately. My understanding is that the sitemaps are a background process. I've had one occurrence when it took 6 days for SM to update the sitemap. Is this general? I don't know.
    2. In the overall scheme of thing you will be penalized for resubmitting identical sitemaps. How badly penalized? Not too bad but it's a negative as part of the 200 signals that Google uses, just like website speed...

    My experience with SM is that it took 2 month from opening my pro account with SM to have Google stabilized with indexing..., but I link from my fairly busy website to my SM site: http://www.foto-biz.com to http://photos.foto-biz.com by link to the image and the gallery.

    ----
    Thanks
    http://foto-biz.com
    The Business of Being a Photographer -- Lightroom Q&A
  • richpepprichpepp Registered Users Posts: 360 Major grins
    edited June 1, 2011
    You might be correct and there may be no penalty for doing so but I'm with pgman here in that once submitted I just leave it alone and Google comes and reads it itself. In fact while we being chatting about this over the last few days I've noticed that Google has come and read the sitemap on our site every day without us doing a single thing. The last time it was read was today.

    Maybe it is worth saying instead that people should only really consider resubmitting their sitemap if they see that Google hasn't read it for a while rather than feeling it is something they need to keep on top of when in fact they are more likely to see benefit from looking after the other things like blogs and links
  • eyeforimageseyeforimages Registered Users Posts: 29 Big grins
    edited June 7, 2011
    Hi Guys, I've just noticed that the gallery images sitemap has gone from my webmaster tools, sitemap-galleryimages.xml.gz. Is this transient or has the format been changed again not to include a specific sitemap for images?

    Also, the sitemaps submitted mean that this breaks my formatting I've put on the website... perhaps, just a thought on a feature suggestion, is to create the ability to build your own custom sitemap. So I can maintain my own sitemap and the links to the images as I like. So if the custom sitemap field is empty, you guys generate a default one, if the custom sitemap fields have content, you stop building for us and use what we have specified. I'd definitely say that this is a pro feature and would really help to have some control on what URL's are used...

    Thoughts?
    Paul Stoakes
    Eye For Images
    Site: http://www.eyeforimages.com
    Blog: http://blog.eyeforimages.com
  • pgmanpgman Registered Users Posts: 164 Major grins
    edited June 7, 2011
    Important Clarification
    chipj wrote: »
    Well, you're talking two different processes here. SmugMug updates the sitemap automatically (according to them) when you make a change. There's nothing to be done there. What they don't do is submit the updated sitemap to the search engines after they've been updated.
    I just want to clarify that submitting the sitemap has very little to do the the indexing of the website/photo site. It's only the first step.
    1. Usually Google will grab the sitemap within minutes (sometimes a few hours) of being notified.
    2. Then Google decides if it worth coming back to grab the content. It may take days or weeks.
    3. Then Google decides if it should be processed and indexed. It may take days, weeks or even months.
    If the website has a Page Rank of 2+ (it's from Larry Page not from the web page rank), then it will be usually done on the same day.
    Thanks
    http://www.sritch.com
    The Dogs of Vancouver, BC
  • chipjchipj Registered Users Posts: 149 Major grins
    edited June 7, 2011
    pgman, a sitemap has everything to do with site indexing. The reality is that you can't depend on Google finding all your content and links... hence the need for an updated sitemap.

    Now, you claim that Google comes by and captures the sitemap within minutes of being notified. This is more likely true than not, but the issue is that on SmugMug, Google isn't being notified of an updated sitemap. Hence the suggestion that this be done manually after a big update (dozens of photos added, new gallries, etc.). In addition, for the SmugMug site, Google may look at the sitemap-index.xml file once every few days or so, but the true site map info is in the sitemap-base.xml.gz or the /sitemap-galleryimages.xml.gz files. On my site these files haven't been looked at by Google for over a week. I've seen that delay go even longer than this.

    Now, as you've suggested, getting "indexed" is only one step of the process. But, just because your site has been indexed, it doesn't mean that your page links will now show up in the search engine results page. Many times content is indexed and it goes into a testing environment ("sandbox") until Google decides whether or not it should show up on a regular basis in the search engine results page. Only part of their determination is Page Rank, there are many more things they look at. Sometime their testing can take months, and even up to a year.

    As a site owner, anything I can do to to speed up the process is of benefit to me. Submitting a sitemap after a major update reduces the odds that updated content won't be overlooked and thus delayed even further from showing up in the search engines results page. I see lots of issues with the way that the SmugMug site is structured (from a SEO perspective), and any little thing I can do to help Google find my content is going to be of benefit to me (as a site owner) in the long run.
    pgman wrote: »
    I just want to clarify that submitting the sitemap has very little to do the the indexing of the website/photo site. It's only the first step.
    1. Usually Google will grab the sitemap within minutes (sometimes a few hours) of being notified.
    2. Then Google decides if it worth coming back to grab the content. It may take days or weeks.
    3. Then Google decides if it should be processed and indexed. It may take days, weeks or even months.
    If the website has a Page Rank of 2+ (it's from Larry Page not from the web page rank), then it will be usually done on the same day.
  • carolinecaroline Registered Users Posts: 1,302 Major grins
    edited June 9, 2011
    No sitemap??
    I've just checked Google Webmaster tools and it shows 'no sitemap submitted'
    Where do I find my sitemap to submit this myself please?

    Caroline
    Mendip Blog - Blog from The Fog, life on the Mendips
    www.carolineshipsey.co.uk - Follow me on G+

    [/URL]
  • rainforest1155rainforest1155 Registered Users Posts: 4,566 Major grins
    edited June 9, 2011
    caroline wrote: »
    I've just checked Google Webmaster tools and it shows 'no sitemap submitted'
    Where do I find my sitemap to submit this myself please?

    Caroline
    Google should pick it up automatically, but if that doesn't seem to happen, you can manually submit this sitemap file:
    http://www.carolineshipsey.co.uk/sitemap-index.xml

    Do not manually submit any other sitemap files. The index file is sufficient and Google will take care of indexing the included sitemaps.

    Sebastian
    Sebastian
    SmugMug Support Hero
  • carolinecaroline Registered Users Posts: 1,302 Major grins
    edited June 9, 2011
    Google should pick it up automatically, but if that doesn't seem to happen, you can manually submit this sitemap file:
    http://www.carolineshipsey.co.uk/sitemap-index.xml

    Do not manually submit any other sitemap files. The index file is sufficient and Google will take care of indexing the included sitemaps.

    Sebastian

    Thank you Sebastian:) and apologies for not saying thanks for your reply from the help desk a couple of weeks ago re banners etc.

    Caroline
    Mendip Blog - Blog from The Fog, life on the Mendips
    www.carolineshipsey.co.uk - Follow me on G+

    [/URL]
  • richpepprichpepp Registered Users Posts: 360 Major grins
    edited June 9, 2011
    caroline wrote: »
    I've just checked Google Webmaster tools (WMT) and it shows 'no sitemap submitted'

    There is no harm in submitting it yourself but you might want to just check that it isn't actually there as it is very easy to miss. When you click on Sitemaps in WMT you normally first go to the list of sitemaps submitted by you and you haven't submitted any so it says no sitemaps submitted. Look all the way across to the right and you should see a link that says Show Submissions - All(1). If you click on that you should see the one the Smugmug automatically supply

    i-nzWQ4px-O.jpg

    I hope

    Rich

    ps. you've done a fabulous job on your site. I especially love the working horses :)
  • OnFirePhotographyOnFirePhotography Registered Users Posts: 71 Big grins
    edited June 9, 2011
    i know have a yellow warning triangle under /sitemap-galleryimages-1.xml.gz
    which comes under the /sitemap-index.xml what does this mean exactly.

    and can i delete the sitemap-galleries and sitemap-images as they still have red cross's and have never been ticks, do i need them? or do they affect whether my images appear in search results as at the moment, none do.
  • richpepprichpepp Registered Users Posts: 360 Major grins
    edited June 10, 2011
    and can i delete the sitemap-galleries and sitemap-images as they still have red cross's and have never been ticks, do i need them? or do they affect whether my images appear in search results as at the moment, none do.

    If they are the ones that appear under - Show Submissions:By me - then yes you can delete them. The only ones that you care about are the ones that Smugmug submit for you which only appear in the 'All' section.

    Deleting them won't affect whether your images appear in search results as the Smugmug provided sitemaps will tell search engines which photos you have and whether they appear depends on how good a job you do at promoting your site.

    I hope that makes sense of some sort :)

    Rich
    ________________________
    Our stuff:
    Clean water in Karnali
  • OnFirePhotographyOnFirePhotography Registered Users Posts: 71 Big grins
    edited June 11, 2011
    ok, under sitemap-index.xml i have 3 other sitemaps sitemap-base.xml.gz , sitemap-galleryimages-0.xml.gz and sitemap-galleryimages-1.xml.gz .

    galleryimages-1 is now a yellow triangle, the others are ticks, should i delete gallery-images-1 or will it fix itself?

    why are there 2 galleryimages sitemaps?
  • rainforest1155rainforest1155 Registered Users Posts: 4,566 Major grins
    edited June 12, 2011
    ok, under sitemap-index.xml i have 3 other sitemaps sitemap-base.xml.gz , sitemap-galleryimages-0.xml.gz and sitemap-galleryimages-1.xml.gz .

    galleryimages-1 is now a yellow triangle, the others are ticks, should i delete gallery-images-1 or will it fix itself?

    why are there 2 galleryimages sitemaps?

    What details for the error do show when you click the yellow triangle?

    There may be multiple sitemaps due to size restrictions for individual sitemap files. If a file gets too large, it needs to be split up.

    Sebastian
    Sebastian
    SmugMug Support Hero
  • OnFirePhotographyOnFirePhotography Registered Users Posts: 71 Big grins
    edited June 14, 2011
    the yellow triangle gives me this:

    URLs not followed
    When we tested a sample of URLs from your Sitemap, we found that some URLs redirect to other locations. We recommend that your Sitemap contain URLs that point to the final destination (the redirect target) instead of redirecting to another URL.

    HTTP Error: 301
    URL: url_icon.gifhttp://www.onfirephotography.com/Motorsport/Porsche-Carrera-Cup/16977175_wvQtXG/6/1284190540_9tBktNd

    Problem detected on: Jun 2, 2011

    The it links to works fine so not sure what he problem is?
  • TwoofyTwoofy Registered Users Posts: 171 Major grins
    edited June 15, 2011
    the yellow triangle gives me this:

    URLs not followed
    When we tested a sample of URLs from your Sitemap, we found that some URLs redirect to other locations. We recommend that your Sitemap contain URLs that point to the final destination (the redirect target) instead of redirecting to another URL.

    HTTP Error: 301
    URL: url_icon.gifhttp://www.onfirephotography.com/Motorsport/Porsche-Carrera-Cup/16977175_wvQtXG/6/1284190540_9tBktNd

    Problem detected on: Jun 2, 2011

    The it links to works fine so not sure what he problem is?

    I am seeing 200's (success):


    bash$ HEAD -H 'User-Agent: Googlebot-Image' 'http://www.onfirephotography.com/Motorsport/Porsche-Carrera-Cup/16977175_wvQtXG/6/1284190540_9tBktNd'
    200 OK

    bash$ HEAD -H 'User-Agent: Googlebot' 'http://www.onfirephotography.com/Motorsport/Porsche-Carrera-Cup/16977175_wvQtXG/6/1284190540_9tBktNd'
    200 OK


    bash$ HEAD 'http://www.onfirephotography.com/Motorsport/Porsche-Carrera-Cup/16977175_wvQtXG/6/1284190540_9tBktNd'
    200 OK

    This may just be a lag between when you've updated something and when Google fetches and processes the sitemap file. I see this happen fairly frequently and if nothing changes on my site it will eventually clear up. On my sites that are constantly changing warnings like this seem to be more persistent but I've not seen any ill effect from them.

    - Greg
Sign In or Register to comment.