robots.txt

Robinta · June 27, 2018

Hi all.

I've had my site for about 20 months now and slowly trying to build my online presence up.

I tend to update my portfolio once or twice a month and have a monthly blog too. When I release anything new I always get an upturn in traffic above the average, usually anything from 100-1000% of my usual hits for a few days.

I noticed last month that nothing really happened with my last posting and this month the same has happened (or not happened) again.

Now, perhaps my work isn`t worth viewing to most people - fair enough, but i've looked for potential problems as I have moved a few things around on the site recently.

I'm far from a web builder expert and don`t know any coding, so I tried a few online hints and tips.

Last night I downloaded and ran a program which didn't seem to find any real issues, other than _robots.txt _ being Disallowed across the entire site.

I've tried to read up on this and seen contradictory comments stating that this would stop search engines finding my content (and therefore seeming to show I do have a visibiliity problem) to others stating that this in itself wouldnt cause any issues with Google for example as it is just a protocol.

I do have Site Visibility turned on in the site settings generically and only have one Private folder shown only to myself.

Any clues folks - my website is at www.robintaylorphotography.uk

Thanks

Robin

Hikin' Mike · June 27, 2018

That's normal.

If you want to be found in the search engines, start by reading my SEO for the Photographer link in my signature.

To add, all you need to do is type in 'site:www.robintaylorphotography.uk' and that will display everything the Google has indexed.

leftquark · June 27, 2018

I ran a test on your site and it looks like everything is setup properly. A number of those "tools" websites don't know how to properly handle very specific scenarios, like in our situation, where certain pages on your site do not want to have search engines tracking them (for example your shopping cart pages, your keyword pages, etc). It may just be that it needs some time for google to pick up / re-index your information. We do submit all your images via the sitemap (a tool that tells search engines where your content is).

denisegoldberg · June 27, 2018

Take a look at the help page Make my site findable by search engines. You might also be interested in the discussion on the DGrin thread SEO - Beauty in the eyes of Google.

I don't believe robots.txt is causing problems.

Unfotunately for all of us, just adding photos or posting a new blog entry on your site does not trigger anything to tell folks to go to your site.

Do you post on other sites or forums - showing photos and including links back to your site?

While I don't believe it is causing problems, your keyword page is a bit of a mess - https://www.robintaylorphotography.uk/keyword. It looks like you are allowing SmugMug to create keywords from the file names of your photos. That can be helpful if you use meaningful names, but yours look like the default names the camera created. I would turn that option off. To get to it, go to Settings... Discovery... Search.

Once you have turned that option off I recommend removing the filename keywords from your photos.

Hikin' Mike · June 27, 2018

@Robinta said:
... but i've looked for potential problems as I have moved a few things around on the site recently.

If you've moved some of you content, that could be it. If you changed your website's structure, normally you add re-directs. Unfortunately I don't think SM does that and you (a SmugMug user) can't edit the .htaccess file.

Robinta · June 27, 2018

thanks to everyone for their help and time. A few tings to try out.

Ironically, today my hits suddenly jumped back up again....have you all been linking my page

I do link to my site from facebok, FLickr etc but as @Denisegoldberg mentioned the photo names are default from the image filename via Lightroom..I always set titles for my shots but hadn`t realised I could change this until now, so that could help.

Thanks again.

Robinta · June 27, 2018

one more question....does that mean utilizing tags is important for SEO ? I'd assumed they would only help someone searching within my site itself ?

Hikin' Mike · June 27, 2018

@Robinta said:
one more question....does that mean utilizing tags is important for SEO ? I'd assumed they would only help someone searching within my site itself ?

Correct. IMO, tags are only useful within your own site.

robots.txt

Comments