3

Description

I have a static franchise website which has various sub pages each dedicated to an individual franchisee.

For each franchisee the page, the only thing slightly similar between all of them are the page titles, they follow this structure: 

<title>
  Welcome to THE_COMPANY - PRODUCT_DESCRIPTION Services, THE_LOCATION
</title>

THE_COMPANY and PRODUCT_DESCRIPTION are the same across all franchisees, however THE_LOCATION changes depeding on where they are located in the UK.

Each franchisee page has the following <meta /> tags:

<meta name="DC.creator" content="user"/>
<meta name="DC.format" content="text/html"/>
<meta name="DC.language" content="en"/>
<meta name="DC.date.modified" content="2014-01-23T11:22:31+00:00"/>
<meta name="DC.date.created" content="2014-01-23T11:22:09+00:00"/>
<meta name="DC.type" content="Page"/>
<meta name="DC.distribution" content="Global"/>
<meta name="robots" content="ALL"/>
<meta name="distribution" content="Global"/>

The main content on each franchisee page is completely different.

The Problem

There is one particular franchisee page, located in Area A.. Which will not appear in Google Search results at all. However every single other franchisee (if you Google Search for "THE_COMPANY, THE_LOCATION" is number 1). And if I do the same search on Bing, Yahoo or DuckDuckGo, the Area A franchisee is the first result on all of them.

Has Google for some reason black listed one page on the site?

What I Have Tried

  • Ensuring the page is referenced in my sitemap.xml file
  • 'Fetching as Google Bot' the link www.the_company.co.uk/areaa
    • When that came back as OK I would submit to index
  • Resubmitting the sitemap.xml file in Webmaster Tools
  • Linking to the Area A page from another pages content
    • For this I also waited about 3 weeks before checking again to give Google time to re-index
  • Making a change to the page content and waiting another 2 / 3 weeks
  • Removing the page completely and recreating it with an alternative URL

The closest thing I have found to this issue is this StackOverflow question but this particular franchisee has existed for almost a year, it used to appear on Google searches however no longer does. I'm guessing the Panda update wasn't too happy with something on the page, but it hasn't effected anything else on the site and I am at a loss for things to try.

I would greatly appreciate any information or thoughts as to what could have caused this

Thanks.

Update

In line with Daniel Fukudas answer below, I have followed some of his steps but everything seems to check out alright:

HTTP Headers

HTTP/1.1 200 OK =>
Date => Tue, 25 Feb 2014 16:31:29 GMT
Server => Zope/(2.12.16, python 2.6.6, linux2) ZServer/1.1
Content-Length => 40078
Expires => Sat, 01 Jan 2000 00:00:00 GMT
Content-Type => text/html;charset=utf-8
Content-Language => en
Vary => Accept-Encoding
Connection => close

Robots <meta /> tag:

<meta name="robots" content="ALL"/>

I have updated this <meta /> tag to read content="INDEX" instead now.

robots.txt:

User-agent: *
Disallow:

User-Agent: Googlebot
Disallow: /*sendto_form$
Disallow: /*folder_factories$

Using site:THE_COMPANY.co.uk:

Searching for 'AREA A site:THE_COMPANY.co.uk' does not return the page, but regardless of that searching just for site:THE_COMPANY.co.uk will not necessarily return every indexed page, or so I understand...

Update

It appears Google likes to drop pages every now and then from the index, despite my steps above, I left the site alone and the page appeared back in the SERPs by itself.

Improve this question
5
  • Do you get any error when you fetch it as Google? If it fetched successfully, you should be able to 'submit' it.
    – Anon
    Feb 25, 2014 at 15:56
  • @vDog I have submitted it a couple of times after fetching (fetching comes back as 'OK'), apologies I forgot to write that in my original question. But it still will not appear in search results.
    – Dan
    Feb 25, 2014 at 15:58
  • Dan. All you can do is make sure that the page/site is okay and that the search spider can index the page. The rest is up to the search engine. Google can do some funny things from time to time. It may take as much as 60 days under normal conditions before your page shows up. I would reset your site to how it should be, stop pinging Google (I know it's hard...), and wait. It could be that there is an issue that has nothing to do with you at Google and you may be exacerbating the issue. It is not too unusual that Google does not index the full site. Not right away.
    – closetnoc
    Feb 25, 2014 at 16:06
  • @closetnoc I will refrain from doing much else to possibly make the situation worse, however the page originally did appear in results and mysteriously vanished.. It has been up for around 1 year, and appearing in search results up until around the time Panda came out (that might be a coincidence as there's no black hat stuff going on here).
    – Dan
    Feb 25, 2014 at 16:38
  • 1
    Sometimes a page is indexed, but does not show up in the SERPs unless a very specific search query is made using site: and unique keywords. Try that. I have seen something like this even recently but it has always worked out. It may be a recent adjustment on Google's side that makes the page hard to find. When I see these, it is a matter of time before Google makes another adjustment and the page reappears. I am not sure this is the case for you. But like I said Google can do funny things from time to time. My experience is that it will work itself out.
    – closetnoc
    Feb 25, 2014 at 16:51

1 Answer 1

Reset to default
1

Make sure:

  • that page returns code 200 @ HTTP headers. Check X-Robots-Tag. You can check headers here: http://www.webconfs.com/http-header-check.php

  • that meta name="robots" on that particular got content="all" or content="index"

  • to check robots.txt

  • inspect your website @ Google using site:example.com

Improve this answer
1
  • Unfortunately everything seems to check out, I have updated my question to reflect the points you raised, please see the "UPDATE" heading
    – Dan
    Feb 25, 2014 at 16:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.