2

I have crawlwed my site with a few different SEO tools - Screaming frog and WildShark SEO spider.

Everything seems good apart from some results which are being returned.

I have built an ecommerce store using Woocommerce and use their product filter, filtering using my own custom attributes.

The seo tools return all expected pages but also return the filter results aswell

Ie

https://www.mywebsite.co.uk/brand/brand/?filter_favourite-flavour=blueberry

but in Yoast SEO i have set pa-flavours to noindex.

I was wondering if anyone knows of any way that I can stop this happening as I cannot see any way to change it and am unsure of any code I can add to prevent it of if I add noindex code, where I should add it?

any help would be great

1

any way that I can stop this happening

do you mean just a behavior of SEO crawlers? Than set in your Screaming Frog an amount of query strings to zero, like on screesnshot: enter image description here

  • Thank you for your reply. Although I am looking more towards google and other search engines. they fact that Screamingfrom and Webshark are pulling these URLs would suggest to me that they are eligible for indexing. I am keen to noindex these links, however the cms doesnt give me the option and I am unsure where I would need to put the relevant noindex code – Paulmcf1987 Jan 16 '18 at 12:49
  • if you want to noindex all parametrized urls you could do it with server settings, if your cms doesn't let you do it in backend. – Evgeniy Jan 16 '18 at 16:12
  • Thank you for your help thus far. I cannot change the configuration settings of screamingfrog as I am using the free version, that would have helped if i could as it shows noindexed pages which uses up some of the 500 limit I have – Paulmcf1987 Feb 2 '18 at 19:42
  • @Paulmcf1987 if you have an access to htaccess file (if running Apache webserver) or nginx.conf (running Nginx as webserver) you could deindex your urls using the X-Robots-Tag – Evgeniy Feb 5 '18 at 15:21
  • thank you again for your reply. Is there a specific way of doing this to ensure that the wolf while site doesn't very no indexed and only the required urls? – Paulmcf1987 Feb 5 '18 at 15:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.