3

I have a very stranget issue on my site with Google Search Console.

I've added my sitemap here: https://formulastocks.com/sitemap.xml

which links to 2 other site maps:

https://formulastocks.com/sitemap_articles.xml

and

https://formulastocks.com/sitemap_pages.xml

I triple checked the URLs on all of them and they are fine. Google discovers both sitemaps and says it finds the URLs.

However when checking the coverage report I see that all my article URLs are completely messed up.

enter image description here

You can see the pages URLs looks just fine, but the article ones have the entire domain name added twice (obviously) resulting in an error and Google not indexing them correctly.

Obviously Google is ignoring my articles sitemap and instead choosing to crawl my site using the links (I can't think of any other reason it would make up this URL?)

So first thing I did was to check my articles page here: https://formulastocks.com/articles

and my links work correctly in the browser, if I click it. It takes me to the correct article URL. So no problem there.

However I did notice in my tag that the href doesn't include the entire domain name. An example looks like this:

<a href="/articles/mindful-trader-vs-formula-stocks">
   ...
</a>

Instead of:

<a href="https://formulastocks.com/articles/mindful-trader-vs-formula-stocks">
   ...
</a>

But I can't imagine Google's crawler does not work with the / pattern?

By the way I am using next.js' Link component for this following their documentation. So this is the default way it works.

Do I really have to find some strange method of pre-fixing all my <a> tags with the full domain name? (if that's the case why are all my other URLs using the same method normal?

I would really appreciate some help explaining how Google could come up with these nonsensical URLs for my site and refuse to index my pages.

I also double-checked and I don't have links to the articles anywhere else on the website other than the sitemap and the articles page.

Thanks!

Improve this question
3

2 Answers 2

Reset to default
2

These problems are very unlikely to stem from your XML sitemaps. Google would only be crawling URLs like this if you actually included such malformed URLs in the sitemap.

The example relative URL you gave it also fine. Google knows how handle root relative links like <a href="/articles/mindful-trader-vs-formula-stocks"> and is not going to crawl malformed URLs from that.

There are a few places that Google could be finding these URLs:

  • Your site may have malformed links. Just because some of links on your site are fine, it doesn't mean that all of them are. It would be problematic if you linked in any place like any of the following:
    • <a href="https://formulastocks.com/articles/https://formulastocks.com/articles/mindful-trader-vs-formula-stocks">
    • <a href="/articles/https://formulastocks.com/articles/mindful-trader-vs-formula-stocks">
    • <a href="./https://formulastocks.com/articles/mindful-trader-vs-formula-stocks">
  • If your site used to have links like this, Google may continue to crawl them indefinitely and report them as errors in Google Search Console. Fixing the problem on your site doesn't stop Google from remembering the problem.
  • If any external links point to such malformed URLs, Google will crawl them.

These malformed URLs are unlikely to be hurting your site or SEO. Google expects to find 404 errors on every site. 404 errors don't hurt the SEO of the URLs that have good content.

There are a few things you can do:

  • Look in your site's access log and see if any hits on these malfromed URLs have a referrer field. If so, that is an easy way to see where these malformed URLs are being generated.
  • Run a crawler against your own site. If it starts crawling malformed URLs, it should be able to tell you where it found them.
  • Implement redirects from these malformed URLs to real URLs. For example https://formulastocks.com/articles/https://formulastocks.com/articles/mindful-trader-vs-formula-stocks could redirect to https://formulastocks.com/articles/mindful-trader-vs-formula-stocks.
Improve this answer
1
  • Thanks Stephen. I think it was an issue with Google's indexing. I didn't make any changes and after a few days it started indexing it correctly. So likely a bug on Google's end.
    – user100002
    Commented Dec 31, 2021 at 16:41
1

The content articles/mindful-trader-vs-formula-stocks is not canonical and the indication of this page in the sitemap is contrary to Google:

List only canonical URLs in your sitemaps.

The same applies to content articles.

In the source code of the article page, I could not find an indication of the URL to mindful-trader-vs-formula-stocks with the element a href as Google recommends:

Google can follow your links only if they use proper <a> tags with resolvable URLs

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.