Google Search Console is telling me:
Googlebot couldn't crawl your URL because your server either requires authentication to access the page, or it is blocking Googlebot from accessing your site.
The URL looks something like this:
I checked the URL in question and my server does indeed reply with a
403 Forbidden response and page.
This is not a URL in our
sitemap.xml and we have never used it for anything. As far as I know, it has never been valid.
I tried the "Mark as Fixed" button in the Google Search Console, but the problem returned.
Any ideas how to properly handle without any detrimental effect to "link juice?" Thanks!
I believe I found the problem. This line in our Apache conf file:
RewriteRule htm$ /$1.html [R=301,L]
There are no regex parentheses, so
$1 will always be empty. So I believe the problem is caused by old backlinks hitting deprecated
I replaced it with this:
RewriteRule (.*)\.htm$ /$1.html [R=301,L]