3

I've attempted to improve the SEO of my website by submitting a sitemap to Google Search Console.

The status is a success but I have 448 discovered URLs with less than 10 pages on my website: clientsforcounsellors.com/sitemap.xml

Also, when I type in my domain name in the address bar, followed by any slug, e.g. clientsforcounsellors.com/sdlkgr, I'm redirected to my homepage instead of having a 404 page displayed.

What's the problem here? Do soft 404's have anything to do with this?

4

Soft 404s have nothing to with this problem, although for usability and SEO you should show 404 errors instead of redirecting to the home page.

This sitemap appears to be generated by a crawler. You shouldn't do that. Google says it is not a good idea: Google Says Don't Crawl Your Site To Build A Sitemap File. If you have only 10 pages on your site, you should have a sitemap that only lists those 10 pages. At 10 pages, you could build that sitemap by hand. Your content management system may also have plugins that build a sitemap for you from its database.

You should also know that XML sitemaps have almost nothing to do with SEO. They can get Googlebot to crawl all your pages, however they don't help get your content indexed or ranked well. If your site is crawlable (and yours appears to be), Googlebot can find and crawl all your pages anyway. The advantages from an XML sitemap are extra stats in Google Search Console and a signal to Google about which URLs you prefer (once you pare the sitemap down to just your 10 pages.) See The Sitemap Paradox for more details.

It is worthwhile to run an SEO crawler against your website. That can uncover problems that need to be fixed. The crawler that generated your sitemap encountered problems crawling your site. Googlebot and other search engine crawlers are going to have these problems too. Lets take a look at some of the pages that got included in your sitemap:

  • /blog/counsellor-needs-website.html
  • /blog/blog/counsellor-needs-website.html
  • /blog/blog/blog/counsellor-needs-website.html
  • /blog/blog/blog/blog/counsellor-needs-website.html
  • /blog/blog/blog/blog/blog/counsellor-needs-website.html
  • ...

Why is the same page getting include many times with more and more repetitions of /blog in the path? You have broken links on your site. Your links look like

<a href="blog/counsellor-needs-website.html">

which is a relative link. Once you are in the blog/ directory already, it will add another blog directory. It would be better to change this to a "root relative link" which starts with a slash. That type of link is relative to your domain name, rather than to the current page:

<a href="/blog/counsellor-needs-website.html">

You have similar problems with your images, and CSS. They should all use root relative linking

  • <img src="/images/main/money-icon.png" instead of <img src="images/main/money-icon.png"
  • href="/css/style.css" instead of href="css/style.css"

See What is the purpose of leading slash in HTML URLs? and Website crawler returns hundreds of 404 errors for pages that do not exist containing duplicate directories in which somebody has pretty much the same problem but with their "about us" page.


Now that your site has been published with these links, you should redirect all the malformed URLs to their preferred form. If you are using Apache you should be able to do something like this at the top of your .htaccess file:

RewriteEngine On
# Redirect /blog/blog/blog/... URLs
RewriteRule (blog/)+((blog|images|css)/.*) /$2 [R=301,L]

Where

  • RewriteEngine On enables mod_rewrite's rules
  • (blog/)+ matches one or more of the extra /blog directories
  • (blog|images|css) matches the final directory that should be kept
  • /.* matches everything that comes after
  • $2 matches everything in the second set of parenthesis ((blog|images|css)/.*)
  • [R=301] makes it a permanent redirect
  • [L] makes it the last rewrite rule (so no later rewrite rules also try to apply)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.