1

I recently began work on an existing Wordpress website that was extensively revamped 3 months ago. https://thedoctorwithin.com. I'm a bit new to Wordpress, so I thought I should reach out to some of the experts in the community.

Checking ‘Not found’ Crawl Errors in Google Search Console, I notice many irrelevant links that are not present in the website, nor the database, as near as I can tell. When checking the source of these irrelevant links, I notice they’re all generated from various pages in the site, as well as non-existing pages, allegedly in the site, even though these errant pages have never existed. Recently did a thorough search and replace, converting all the old links in the site to the current, mostly flat hierarchy.

Here's an example of the link weirdness I'm seeing:

-https://thedoctorwithin.com/category/seminars/newsletters/page/7/newsletters/page/3/feedback-and-testimonials/ allegedly linked from: - https://thedoctorwithin.com/category/seminars/newsletters/page/7/newsletters/page/3/ (doesn’t exist)

In other cases, these goofy URLs are reported to be linked to from our sitemap. BTW - all the URLs in the sitemap are valid URLs.

Currently, the site has a flat structure. Nearly all the content is merely URL/content/ without further breakdown (or subdirectories). Previous site versions had a more varied page organization, but what I'm seeing doesn't seem to reflect the current page organization, nor the previous site structure.

Had a similar issue come up, in connection to use of Divi's search feature. Ended up with some pretty deep non-existent links branching off of /search/, such as:

I blocked the /search/ branches via robots.txt. No real loss, since neither /search/ nor any of its subdirectories are valid.

There are numerous pre-existing categories and tags on the site. The categories and tags aren't used as pages. I suspect Google, (and other engines,) might be creating arbitrary paths from these, combined with other older links and categories. Looking through the site’s 404 errors, I’m seeing similar behavior from Bing, Moz and other spiders, as well.

I suppose I could use Search Console to remove URL/category/ and URL/tag/. I suppose I could do the same, in regards to other legitimate spiders / search engines. Perhaps it would be better to use Mod Rewrite to lead spiders to pages that actually do exist.

  • Could this odd spidering behavior indicate something needs to be fixed on the site?
  • I welcome suggestions about best practices for dealing with these errant searches. As mentioned above, I've marked them all fixed, but they keep returning.
  • I welcome any further insights about why these strange links are occurring.

BTW - I've seldom posted on Stack Exchange. I welcome any suggestions about how to improve my posting here, in the future.

Thank you.

| improve this question | |
  • Well, isn't this special. The darkweb moved over there. OK, if you are a white hatter, you can make a php file the same name and use php to hack and slash the darkweb server and the connecting client. – drtechno Jul 3 '18 at 19:03
-1

yes i would be concerned about this. If you are getting stats to files, directory structure, and its not on the hosting drive, then someone copied the private key and hosting a version of your site on your url.

This is the first sign of darkweb activity.

Things you need to do:

first, you need to find out if the bandwidth is going out of control, you will see an excessive bandwidth useage from an ip address, trace it back, it would most likely reveal a vpn connection masking the utility pirate's server (because they steal your bandwidth to serve their webserver on your site).

Block this Person!!

check and see if xml-rpc module is disabled. If it is enabled, disable it: https://wordpress.org/plugins/disable-xml-rpc/ Make sure no one can directory surf. (add: Options -Indexes to the .htaccess file)

replace server certificate. Delete the existing one, and never use it again.

| improve this answer | |
  • By "darkweb" do you mean the site is hacked and being used to for other purposes? – Stephen Ostermiller♦ Dec 1 '18 at 1:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.