Google Search Console says "blocked" by robots.txt?

jimhjimh MinneapolisPosts: 34Registered Users Big grins

I'm new to SmugMug and not yet very far up the SEO learning curve, so cut me some slack. The first thing that I notice on GSC is that it currently reports 44 pages indexed but 159 "blocked" by robots.txt. And it shows me a robots.txt for my SM site that indeed seems to block many things.

Is this real? Why would SM be using a robots.txt?

Comments

  • Hikin' MikeHikin' Mike Walkin' like a Penguin! Atwater, CaPosts: 4,301Registered Users Major grins

    That's normal to use a robots.txt file to not allow indexing duplicate content (like keywords) and utility files.

  • jimhjimh MinneapolisPosts: 34Registered Users Big grins

    Thanks, maybe if I look deeper GCS will show that the blocked files are in those categories.

  • jimhjimh MinneapolisPosts: 34Registered Users Big grins
    edited October 24, 2018

    Here's my next question.. GCS shows no sitemap. Supposedly SM handles that automatically, but it's been a few weeks. Do I need to submit one manually or is the sitemap just not relevant in this case?

  • Hikin' MikeHikin' Mike Walkin' like a Penguin! Atwater, CaPosts: 4,301Registered Users Major grins

    SmugMug creates your sitemap and you can submit it, but I quit submitting a sitemap to the websites I maintain. Google is pretty good with indexing any new content.

  • jimhjimh MinneapolisPosts: 34Registered Users Big grins

    It's now been a few weeks since my SM site went live, and I'm trying to start understanding what GCS is telling me. I'm looking at 'Coverage" which, as I understand it, should show me the extent to which Google has now indexed my site. It shows 0 'Valid' pages and 24 'Excluded'. Is something that's supposed to be happening - not happening?

  • Hikin' MikeHikin' Mike Walkin' like a Penguin! Atwater, CaPosts: 4,301Registered Users Major grins
    edited October 26, 2018

    @jimh said:
    It's now been a few weeks since my SM site went live, and I'm trying to start understanding what GCS is telling me. I'm looking at 'Coverage" which, as I understand it, should show me the extent to which Google has now indexed my site. It shows 0 'Valid' pages and 24 'Excluded'. Is something that's supposed to be happening - not happening?

    I don't trust GCS. If you want to see what pages are indexed, type in site:http://www.your-website-here.com in your browser. That should show all of the pages that Google (and other search engines) indexed.

  • jimhjimh MinneapolisPosts: 34Registered Users Big grins

    I tried that and it shows.... well.... a bunch of pages on my site. Very far from all. But maybe it's a start.

Sign In or Register to comment.