I am optimising the crawling experience of the website as a large portion of the site isn't being crawled.
A large portion of the 'crawl budget' is being used on crawling URLs with utm parameters for Google Shopping.
If I block these parameters from being crawled in the robots.txt file, will this have an impact on the Google Shopping ads?
I'm struggling to find confirmation of the Googlebot Google Merchant uses. I'm thinking that if it uses Adsbot - Google then I will block the parameters just for regular Googbot but allow for Adsbot.
My question;
- Does Google Merchant use Adsbot?
- If not, will blocking these pages from Googlebot negatively impact the shopping campaign?
- If it will, are there other alternatives to stop Googlebot wasting time on these pages?