As a webmaster in charge of a tiny site that has a forum, I regularly receive complains from users that both the internal search engine and that external searches (like when using Google) are totally polluted by my users' signatures (they're using long signatures and that's part of the forum's experience because signatures makes a lot of sense in my forum).

So basically I'm seeing two options as of now:

  1. Rendering the signature as a picture and when a user click on the "signature picture" it gets taken to a page that contains the real signature (with the links in the signature etc.) and that page is set as being non-crawlable by search engine spiders). This would consume some bandwidth and need some work (because I'd need an HTML renderer producing the picture etc.) but obviously it would solve the issue (there are tiny gotchas in that the signature wouldn't respect the font/color scheme of the users but my users are very creative with their signatures anyway, using custom fonts/colors/size etc. so it's not that much of an issue).

  2. Marking every part of the webpage that contains a signature as being non-crawlable.

However I'm not sure about the later: is this something that can be done? Can you just mark specific parts of a webpage as being non-crawlable?

9 Answers 9


Here is the same answer I provided to noindex tag for google on Stack Overflow:

You can prevent Google from seeing portions of the page by putting those portions in iframes that are blocked by robots.txt.


Disallow: /nocrawl/


This text is crawlable, but the following is
text that search engines can't see:
<iframe src="/nocrawl/content.html" width="100%" height=300 scrolling=no>


Search engines cannot see this text.

Instead of using using iframes, you could load the contents of the hidden file using AJAX. Here is an example that uses jquery ajax to do so:

his text is crawlable, but the following is 
text that search engines can't see:
<div id="hidden"></div>
  • Does adding/injecting control using AJAX will help to disallow and prevent from crawling the same? Commented Jul 27, 2017 at 9:18
  • 1
    As long as the location the AJAX is fetching from is blocked by robots.txt. Commented Jul 27, 2017 at 9:49
  • Will you please check this webmasters.stackexchange.com/questions/108169/… and suggest if any. Commented Jul 27, 2017 at 10:00
  • As long as the location the AJAX is fetching from is blocked by robots.txt - Please elaborate on this. Commented Jul 27, 2017 at 11:05
  • 3
    Google penalises those who hide their javascript from being crawled, in order to prevent abuse. Is the same true of iframes?
    – Jonathan
    Commented Jan 28, 2018 at 16:40

Another solution is to wrap the sig in a span or div with style set to display:none and then use Javascript to take that away so the text displays for browsers with Javascript on. Search engines know it's not going to be displayed so shouldn't index it.

This bit of HTML, CSS and javascript should do it:


<span class="sig">signature goes here</span>


.sig {


<script type="text/javascript"> 

You'll need to include a jquery library.

  • 4
    +1 and I thought about it but wouldn't that be considered a form of "cloaking" by various spiders? Commented Jul 4, 2011 at 13:55
  • 3
    Not by Google: theseonewsblog.com/3383/google-hidden-text Commented Jul 4, 2011 at 14:31
  • 1
    I think it's quite neat :-) Commented Jul 4, 2011 at 14:42
  • This could, in the strictest definition, be considered cloaking. However he could print all of the signature with javascript using a document.write("");. Google does not index anything within javascript. support.google.com/customsearch/bin/… Commented Jan 9, 2013 at 15:33
  • 2
    @Cristol.GdM Correct. This answer is no longer relevant, as search engines like Google now execute JavaScript on the page as part of the indexation process. Commented Apr 8, 2019 at 23:08

I had a similar problem, I solved it with css but it can be done with javascript and jquery too.

1 - I created a class that I will call "disallowed-for-crawlers" and place that class in everything that I did not want the Google bot to see, or place it inside a span with that class.

2 - In the main CSS of the page I will have something like

.disallowed-for-crawlers {

3- Create a CSS file called disallow.css and add that to the robots.txt to be disallowed to be crawled, so crawlers wont access that file, but add it as reference to your page after the main css.

4- In disallow.css I placed the code:

.disallowed-for-crawlers {
    display:block !important;

You can play with javascript or css. I just took advantage of the disallow and the css classes. :) hope it helps someone.

  • I am not sure this works due to crawlers not accessing the .css file (is this a thing? Since when do crawlers access and crawl specific css files?) and not simply due to display:none and crawlers understanding it will not be displayed so they don't index it. Even if this is the case, what do you do to actually display the content to human users?
    – Prinny
    Commented Nov 30, 2018 at 7:55
  • The content is displayed when the step 4 is loaded for the human user since they are allowed to see that file(disallow.css). And about the robots loading CSS that is what respectable search engines do nowadays, that's how they determine when a website is mobile friendly or not, crawlers that do not respect it are not worth to worry about, major search engines read css and javascript to crawl pages, they been doin it for about... almost 6 years now? maybe more. Commented Nov 30, 2018 at 10:36
  • Can you provide sources that back up that claim? Please see webmasters.stackexchange.com/questions/71546/… and yoast.com/dont-block-css-and-js-files and most importantly here webmasters.googleblog.com/2014/10/… where what you describe is portrayed as bad practice.
    – Prinny
    Commented Nov 30, 2018 at 11:13
  • It is a bad practice if I wanted Google to see my website normally and I block all the CSS, and is bad practice because they interpret the CSS, but in this specific case I block one specific file, not all of the css, OP ask about preventing google from reading a section of the page. but I don't want Google to crawl those sections so I block one single CSS ( not all of them, just one). And to back up what claim you said? the one that crawlers read JS and CSS? it is as easy as going to your Google Webmaster Tools and take a look at "Fetch as a robot" you will see there how they read css and js. Commented Dec 1, 2018 at 0:55
  • Also to add, in my specific case is not that I want to do something shady with the Google Crawler, I just don't want google to read a section of information that may seem repetitive in all pages. Like Phone numbers, addresses, related products or information that is not relevant for Google to crawl. Commented Dec 1, 2018 at 1:10

Marking every part of the webpage that contains a signature as being non-crawlable.

You can use the directive nosnippet:

Do not show a text snippet or video preview in the search results for this page. A static image thumbnail (if available) may still be visible, when it results in a better user experience. This applies to all forms of search results (at Google: web search, Google Images, Discover).

If you don't specify this directive, Google may generate a text snippet and video preview based on information found on the page.


<p>This text can be shown in a snippet
<span data-nosnippet>and this part would not be shown</span>.</p>

<div data-nosnippet>not in snippet</div>
<div data-nosnippet="true">also not in snippet</div>
<div data-nosnippet="false">also not in snippet</div>
<!-- all values are ignored -->

<div data-nosnippet>some text</html>
<!-- unclosed "div" will include all content afterwards -->

<mytag data-nosnippet>some text</mytag>
<!-- NOT VALID: not a span, div, or section -->

For an understanding of Google's hidden text policy, check the Google Guide Hidden text and links.

  • 2
    This looks useful. It should be noted that it doesn't prevent the text from getting indexed, it just prevents Google from showing it in the snippet in the search results. Commented Mar 22, 2021 at 10:11

One way to do this is to use an image of text rather than plain text.

It is possible that Google will eventually be smart enough to read the text out of the image, so it might not be completely future-proof, but it should work well for at least a while from now.

There's a bunch of disadvantages to this approach. If a person is visually impaired, it's bad. If you want your content to adapt to mobile devices versus desktop computers, it's bad. (and so on)

But it is a method that currently (somewhat) works.

  • how well does this work if you use alt & title tage appropriately?
    – Jayen
    Commented Dec 26, 2015 at 5:26
  • Haven't tried, but it seems likely that Google would crawl those. It's a major limitation of this approach.
    – James
    Commented Jan 9, 2016 at 4:14

This is easy.

Before you serve your page you need to know whether it is to a bot, a computer or a phone. You then need to set the content accordingly. This is standard practice in this day and age and core functionality of some CMS's.

There are plenty of solutions on SE for doing redirection based on USER AGENT that can be put in your htaccess. If this suits your forum software then you can run different code off the same DB to deliver what Google needs without the chaff and trimmings.

Alternatively you can put a little line in your PHP code that does a 'if USER AGENT == Googlebot then don't show signatures'.

If you really cannot do that then you can get mod_proxy to serve to the bot and use it to strip out anything your php code generates that the bot not need to see.

Technically Google do not approve of their search engine being shown a different page to what the normal site visitor sees, however, to date, they have not taken the BBC and others that provide browser/IP/visitor-specific content off their search engine results. They also have limited means to see if their bot has been 'conned'.

The alternative solution of hiding content with CSS for it to be re-enabled by a script is also a bit of a grey area. According to their own Webmaster Tools guidelines of 20/6/11 this is not a good idea:


That may not be a tablet cast in stone, but it is up to date and by Google.

The hide the content trick will not work with the minority of people that do not have javascript, this may not be a huge concern, however, waiting for the document to load and then showing the signatures will not be a satisfactory viewing experience as you will think the page has loaded, then it will jump about as the hidden signatures show up to then push the content down the page. This type of page load can be irritating if you have a low-end net-top but may not be noticeable if you have a fast developers machine on a fast internet connection.

  • 6
    @ʍǝɥʇɐɯ: serving different content depending on who is accessing the page is kinda frowned upon and may penalize you in search engine as far as I understand it. I much prefer paulmorris' JavaScript solution. Commented Jul 4, 2011 at 15:34
  • @ʍǝɥʇɐɯ: erf, if serving personalized content is the name of the game, so is JavaScript. Last I checked the Web overall didn't really work that well anymore without JavaScript installed (GMail, FaceBook, Google Docs, stack overflow, Google+ --yup I've got it already ;) -- etc.). I don't see no need to criticize paulmorris' solution based on the false premise that JavaScript being not available would be an issue. Commented Jul 4, 2011 at 16:19
  • @ʍǝɥʇɐɯ: You may like this from Matt Cutts (in charge of SEO at Google) on that very subject: theseonewsblog.com/3383/google-hidden-text That was the excellent comment by paulmorris posted in comment to his excellent answer. I'm sorry but calling JavaScript "sillyness" on such a forum is close to trolling. Commented Jul 4, 2011 at 19:34
  • ...and then we get this question: webmasters.stackexchange.com/questions/16398/… - 'keyword stuffing' is silly. Sorry about that. Commented Jul 4, 2011 at 20:17
  • I believe this falls under "cloaking" and therefore it is not a good practice.
    – Prinny
    Commented Nov 30, 2018 at 7:51

No, there is no way to prevent robots crawling parts of pages. It's a whole page or nothing.

The snippets in Google's search results are usually taken from the meta description on the page. So you could make Google show a specific part of the page by putting that in the meta description tag. With user-generated content it's difficult to get good snippets, but taking the first post of the thread would probably work.

The only other way I can think of is to use Javascript. Something like paulmorriss suggested may work, but I think search engines would still index the content if it's in the HTML. You could remove it from the HTML, store it in Javascript string, then add it back on page load. This gets a bit complex, though.

Finally, one thing to keep in mind: if Google is showing user's signatures in their snippets, it has decided that is the part most relevant to the user's query.

  • 1
    the problem is not so much Google showing user's sigs in their snippets as these specific pages getting that highly ranked in Google in the first place. The issue here is precisely that Google may think the sigs are relevant when they're actually not: I mean, that's exactly what my question is all about. Commented Jul 4, 2011 at 15:32
  • @Webby, I don't understand, why don't you want your pages ranking highly? Do you have some example pages and queries so we can see what you're talking about? And if Google is showing a sig in search results, then it is relevant for that search query, even if it's not relevant to the page itself. Commented Jul 4, 2011 at 16:05
  • 1
    I can't give examples but I do want my site/forum to rank highly and it does so very nicely. The problem is that amongst the search results (which are all mostly for my site/forum anyway because it's basically the site on the subject), what should be the real entry pages are flooded amongst signatures. I mean, I do really want to do what I asked in the question. And pictures or JavaScript it is going to be. Commented Jul 4, 2011 at 16:24
  • @Webby, your responses have been a little confusing but you seem to be implying that your user signatures are all separate pages (URLs) and thus appearing as separate results in SERPs. In which case you can block those pages through robots.txt. Otherwise, try the meta description solution I posted above, because that will almost certainly mitigate the problem. Commented Jul 5, 2011 at 9:42

You can put the page in a PHP if with an "else" that lead to a captcha that gives the key for the if part.

I don't really care because if the user credential don't match on my page they do get a blank page or is send to the login page.


if(empty($_SESSION['captcha']) or $_SESSION['captcha'] != $key){
    header("Location: captcha.php");

if(!empty($_SESSION['captcha']) and $_SESSION['captcha'] == $key){

"the page"


$key should be a hash of the current day or something that change so it's not sufficient to add the value to the session.

Write in the comment if you want me to add a example captcha because I don't have one on me now.

  • This answer assumes the websites uses or the developer knows PHP which may not be true. Also, it makes getting to the content difficult for users which is not a good thing.
    – John Conde
    Commented May 27, 2017 at 11:50
  • I can buy that not every one knows PHP but a captcha can be "what is the color of grass", even blind pepole know that. Commented May 27, 2017 at 13:40

Apparently, <!--googleoff: all--> and <!--googleon: all--> do what you want it to.

Read more https://www.google.com/support/enterprise/static/gsa/docs/admin/70/gsa_doc_set/admin_crawl/preparing.html#1076243



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.