2

I have crawlwed my site with a few different SEO tools - Screaming frog and WildShark SEO spider.

Everything seems good apart from some results which are being returned.

I have built an ecommerce store using Woocommerce and use their product filter, filtering using my own custom attributes.

The seo tools return all expected pages but also return the filter results aswell

Ie

https://www.mywebsite.co.uk/brand/brand/?filter_favourite-flavour=blueberry

but in Yoast SEO i have set pa-flavours to noindex.

I was wondering if anyone knows of any way that I can stop this happening as I cannot see any way to change it and am unsure of any code I can add to prevent it of if I add noindex code, where I should add it?

any help would be great

Improve this question

1 Answer 1

Reset to default
1

any way that I can stop this happening

do you mean just a behavior of SEO crawlers? Than set in your Screaming Frog an amount of query strings to zero, like on screesnshot: enter image description here

Improve this answer
5
  • Thank you for your reply. Although I am looking more towards google and other search engines. they fact that Screamingfrom and Webshark are pulling these URLs would suggest to me that they are eligible for indexing. I am keen to noindex these links, however the cms doesnt give me the option and I am unsure where I would need to put the relevant noindex code
    – Paulmcf1987
    Jan 16, 2018 at 12:49
  • if you want to noindex all parametrized urls you could do it with server settings, if your cms doesn't let you do it in backend.
    – Evgeniy
    Jan 16, 2018 at 16:12
  • Thank you for your help thus far. I cannot change the configuration settings of screamingfrog as I am using the free version, that would have helped if i could as it shows noindexed pages which uses up some of the 500 limit I have
    – Paulmcf1987
    Feb 2, 2018 at 19:42
  • @Paulmcf1987 if you have an access to htaccess file (if running Apache webserver) or nginx.conf (running Nginx as webserver) you could deindex your urls using the X-Robots-Tag
    – Evgeniy
    Feb 5, 2018 at 15:21
  • thank you again for your reply. Is there a specific way of doing this to ensure that the wolf while site doesn't very no indexed and only the required urls?
    – Paulmcf1987
    Feb 5, 2018 at 15:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.