Feature request: ability to block Google searches

jfriend

After installing statcounter onto my pages, I am now able to see how many people are coming to my site and usually where they are coming from. I was a bit surprised to see people landing directly into certain galleries from a Google search. There are some galleries I'm OK with that, but others that it makes me uncomfortable.

I'd like to be able to shut out Google indexing from some or all of my galleries. I know that I could do that with a site password, but that seriously compromises the usability of my site for the people that I do want to share with and I don't think either you or I want that to be the case. I know I could password protect individual galleries and I have done that in some cases (for pics of other people's kids), but that's also more of a usability compromise than I'd like. I found out today that my gallery description is still indexed even when the gallery itself is password protected. Not a big security issue, but gives me a creepy feeling. I'm also a bit weirded out by Google caching, knowing that even if I respond to a privacy request and remove something from my site, it might live on in Google for a long time.

Is there another answer? Can we have a feature to block Google (and other search engines) from indexing our site or particular galleries without requiring our viewers to use passwords?

--John

flyingpylon

Oh, there's a lot more to worry about than Google's cache. Check out the wayback machine. I can go there and find a site I took offline in 1998!

jfriend

Spooky

flyingpylon wrote:

Oh, there's a lot more to worry about than Google's cache. Check out the wayback machine. I can go there and find a site I took offline in 1998!

That is really spooky! I found some really old stuff of mine too there.

--John

JamesJWeg

HOLY COW! I just found a site I did in 97, I had lost it copy in a drive failure and wanted a little info from it, cool.

James.

jfriend

Could I use meta tag to keep robots out

jfriend wrote:

Can we have a feature to block Google (and other search engines) from indexing our site or particular galleries without requiring our viewers to use passwords?

--John

Another idea. Is it possible to just put something in my page customization that would tell well-behaved robots like search crawlers to skip the indexing of my site?

Would it work if I just put:

<META NAME="ROBOTS" CONTENT="NOINDEX,NOFOLLOW">

in my customization?

jfriend

meta tag for robots already in the page

jfriend wrote:

Would it work if I just put:

<META NAME="ROBOTS" CONTENT="NOINDEX,NOFOLLOW">

in my customization?

Hmmm. I see that Smugmug already has the opposite meta tag in my page with no way for me to override it. This is what is already in my page.

<meta name="robots" content="all, index, follow" />

colourbox

Yes, and on private galleries they seem to reverse the tags so they won't get searched. This matches the behavior of smugmug's own search engine. It sounds like smugmug would have a few competing issues to resolve because there is at least one other thread where some of us want smugmug's search to work inside private/password galleries and also to limit scope to specific galleries (say, those of only one client or project). smugmug's probably going to have to consider all these requests together. Maybe a comprehensive solution would look something like this:

Search options for this gallery:
Outside search engines (allow robots) [Y] [N]
smugmug search engine [Y] [N]
Scope of smugmug search [all of smugmug] [this gallery only]

Just brainstorming...

jfriend

Maybe this would work...

jfriend wrote:

Hmmm. I see that Smugmug already has the opposite meta tag in my page with no way for me to override it. This is what is already in my page.

<meta name="robots" content="all, index, follow" />

Well, if a well-behaved robot obeys the last meta tag that it sees in a page, maybe it will work for me to insert:

<META NAME="ROBOTS" CONTENT="NOINDEX,NOFOLLOW">

in my customization because it appears after the smugmug tag. I guess I'll just have to try it.

Anyone have any idea whether this would work or not?

--John

JamesJWeg

http://www.robotstxt.org/wc/meta-user.html

found that with a quick google search.

James.

jfriend

Thanks.

JamesJWeg wrote:

http://www.robotstxt.org/wc/meta-user.html
found that with a quick google search.

Thanks. That points out that the <META> tag is supposed to be in the <HEAD> section. It looks like smugmug doesn't give me any ability to put any customizations inside the HEAD tag except the text of the title.

--John