0

I've just seen in Google Search Console that Google is doing 3 things with spammy internal search links. The link structure is always www.example.com/search?q=[spammy link here]

  1. Around 500 are green and indexed (they don't appear on the sitemap but are indexed in the overall page indexing)
  2. Around 300 are grey and blocked by robots.txt (the instruction here is disallow /search)
  3. 400 more are orange, indexed, though blocked by robots.txt

My robots.txt is set to disallow /search. Do you know why:

  1. these spam links are doing 3 different things
  2. will they impact my site negatively (I've seen some drop in visits since this spam started)
  3. Is there anything I can do about it?

Thank you, I've been Googling for days and still confused about a solution.

Improve this question
6
  • I think the best way is to add a robots meta tag / X-Robots-Tag http header, per developers.google.com/search/docs/crawling-indexing/…. This could be done either by changing the code generating the search result to add a <meta name="robots" content="noindex"> tag to the html file, or the server config files (say .htaccess) to add X-Robots-Tag: noindex to URLs starting with "search?q="
    – Uri Raz
    Apr 13 at 10:27
  • Thank you for your response. Do you know if there are any online guides on how to do either of these steps? My robots.txt file is un-editable due to the website builder I use.
    – Bias_
    Apr 13 at 10:48
  • Where does Google say it is finding links to these pages? I suspect that your site was hacked and that the hacker created links on your own site to them.
    – Stephen Ostermiller ♦
    Apr 13 at 13:22
  • How can any of them be green and indexed if you are disallowing /search in robots.txt? Are some of them using some other URL path?
    – Stephen Ostermiller ♦
    Apr 13 at 13:23
  • @StephenOstermiller "How can any of them be green and indexed if you are disallowing /search in robots.txt?" This is what has confused me - I don't know how they've indexed - every like has the same structure, something like this - www.example.com/search?q=naza8k-สล๊อค รับเครดิตเล่นฟรี 50🌟
    – Bias_
    Apr 13 at 13:59

1 Answer 1

Reset to default
0

If your web server is Apache or LiteSpeed, you can add the X-Robots-Tag by adding the following to .htaccess

<FilesMatch "^search">
 Header always add X-Robots-Tag "noindex"
</FilesMatch>

I don't know what software implements the search functionality for your website, so I can't tell how to change it to add a robots meta tag.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.