1

I run a few sites and one client mentioned that their web traffic has gone down and that their home page is no longer listed on Google.

The site has been running for 10 years with no issues.

I added an SSL certificate recently and using web.config file I set it to re-direct.

In search console there was only http listed so I added https and then tried to submit a sitemap.xml file but it keeps saying 'can't be reached' even though you can see it fine in the browser. Also when I try and run Mobile Friendly test it says 'Page cannot be reached - This could be because the page is unavailable or blocked by robots.txt' even though robots.txt lets everything through. I'm baffled by this and it is affecting their ranking.

Can anyone advise what it could be?

You can see in the image that the problems started recently and that the home page is listed as Excluded and 'currently not indexed'. No idea why.

Could it be the re-directing of http / https?

This is the web.config file setup 

<?xml version="1.0" encoding="UTF-8"?> 
<configuration> 
<system.webServer> 
    <httpErrors errorMode="Custom">
    <remove statusCode="404" />
    <error statusCode="404" path="/404.asp" responseMode="ExecuteURL" />
</httpErrors>
<rewrite> 
<rules> 
<clear /> 
<rule name="Redirect to https" stopProcessing="true"> 
<match url=".*" /> 
<conditions> 
<add input="{HTTP_SSLSESSIONID}" pattern="^$" /> 
</conditions> 
<action type="Redirect" url="https://{HTTP_HOST}{REQUEST_URI}" redirectType="Permanent" appendQueryString="false" /> 
</rule> 
</rules> 
</rewrite> 
</system.webServer> 
</configuration>

enter image description here

Improve this question

1 Answer 1

Reset to default
1

Here's what you can do.

  1. Try to ping google to crawl your sitemap once again.
  2. Enter the link to your sitemap in the Search console.
  3. Click on the 'excluded' links and ask Google to re-index them one at a time with the following steps. enter image description here enter image description here enter image description here enter image description here enter image description here
Improve this answer
5
  • Thank you for taking the time to reply. Pinged sitemap then went to the Excluded pages and clicked the magnifying glass icon on the first listing, the home page, it came back with: Availability - URL is unknown to Google and the following screenshot Indexing request rejected' : i.postimg.cc/MG88cDVV/Screenshot-2019-11-04-at-19-32-32.png
    – Webchap
    Nov 4, 2019 at 19:34
  • Can u share a link to the screenshot?
    – Anuvesh
    Nov 4, 2019 at 19:36
  • Sure - postimg.cc/bZL5W16w
    – Webchap
    Nov 4, 2019 at 19:45
  • Also when I try to test Robots.txt I get this: "robots.txt fetch failed - You have a robots.txt file that we are currently unable to fetch. In such cases we stop crawling your site until we get hold of a robots.txt, or fall back to the last known good robots.txt file." even though you can clearly see it at snowscape.co.uk/robots.txt
    – Webchap
    Nov 4, 2019 at 19:47
  • Any ideas on this?
    – Webchap
    Nov 5, 2019 at 11:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.