I have built a simple age verification jQuery script for a particular brand of alcohol. It asks the user if they are of legal drinking age with a yes or no button. On yes the website stores a cookie and they no longer get that age verification for one month, simple stuff.

The age verification script fills the screen covering all elements behind it, using something like this:

.verify {
    position: fixed;
    left: 0;
    width: 100%;
    height: 100%;

Obviously, this works superbly for users but for search engines, not so much. I'd like to be able to tell Google/Bing to click the yes button, but sadly, I know it's not that easy.

Without knives, daggers and hidden cloaks, how do I tackle this issue?

Please note that I'm aware of a similar question and what makes this question different is the specific setup in mind, not recommendations in general.

  • what about dynamically insert the element using javascript? i.e. holding off on displaying the overlay until the load event has fired on the window object ( or DOMContentLoaded on the document) or using settimeout()? that way when google fetches, they will get the html - without the overlay - then a couple of seconds later the user will see the age confirmation prompt. – the_velour_fog May 18 '17 at 8:50
  • What I did was a $(window).on('load', function () {setTimeout(function(){ $('html').removeClass('crawlFix') },2000); so in the CSS I have .crawlFix .overlay { display: none;} until jQuery removes it after 2 seconds after the page has loaded. – Simon Hayter May 18 '17 at 9:34
  • Is 2seconds enough for Google/Bing through? – Simon Hayter May 18 '17 at 9:34
  • yes I wondered the same thing, I dont think time is the issue, to me the question is whether or not google will execute the javascript as part of parsing and indexing the page. I was wondering if you could wait till the page was next indexed, then take a look at googles cached version - that might show you how google see it? – the_velour_fog May 18 '17 at 9:40
  • Yea, well if you leave what you said as an answer, I'll happily accept. I'm fairly confident that this method works fine. – Simon Hayter May 18 '17 at 9:46

Google will likely see the overlay even if you delay it using a multi-second timer. When Googlebot processes JavaScript it seems to be able to skip over (or speed up) timeouts. In the past, I've tried generating output after a 30 second timer and checking what the "Fetch and Render" tool in GSC "sees". The Fetch and Render tool often returns "immediately" with the "timed" JS generated content already rendered.

But even if Googlebot "clicked yes", it won't store the cookie, so the overlay will be forever present. However, Googlebot can still potentially "see" what's underneath the overlay - but if the "user" can't then Google may choose to ignore it (these days)?

A couple of possibilities:

  1. Block this particular JS file with robots.txt. Googlebot (or any "nice" bot) won't fetch and run this JS file so the overlay never appears.

  2. Only display the overlay to user-agents that support cookies. Googlebot and most search engine bots don't support cookies. Since you are using a cookie as the trigger to hide (or "not show") the overlay, this would be a sensible check anyway.

    Only display the overlay after you have confirmed the presence of a "test" cookie that is set unconditionally on the initial request. (You could perhaps issue an AJAX call to check for the "test" cookie, rather than waiting for the second request - if you want this overlay to appear as soon as? Although the second request might be soon enough.) Googlebot (and most search engine bots) won't set (or strictly speaking "return") the "test" cookie, so your check is never successful and the overlay never appears. This would also stop the overlay from repeatedly appearing for "power users" who have disabled cookies in their browser.

| improve this answer | |
  • nice. any chance you could provide more info on how to do the fetch and render? also what is GSC? is it like google webmaster tools? – the_velour_fog May 19 '17 at 0:17
  • @the_velour_fog Google Search Console (GSC) is the "new" name for Google Webmaster Tools (GWT) (well, I say "new", it actually changed 2 years ago! webmasters.googleblog.com/2015/05/… - I still think the old name was better, but hey!). In GSC > Crawl > Fetch as Google. Enter the URL and hit "Fetch and Render". Google is becoming (has become) frighteningly good at rendering (and indexing) JS generated content. – MrWhite May 19 '17 at 0:27
  • thanks, well theyve been able to run at least some of the modifications javascript made to wepages for some time. the problem was developers couldnt see what googlebot could see. if fetch and render actually does that it would be great – the_velour_fog May 19 '17 at 0:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.