New answers tagged googlebot
1
vote
Bots trying to crawl non-existant movie related pages which appear to be from IMDB on my project management website
I have resolved this using a WAF custom page rule in Cloudflare.
(http.user_agent contains "SemrushBot")
Looks like it is working fine.
Still don't know where SemrushBot was getting those ...
0
votes
Website is not showing any Traffic on Google, SEMrush and also on other Platforms
do you have a cookie consent blocker?
can you confirm the google tag using https://chromewebstore.google.com/detail/tag-manager-assistant/oacpegaegonlmnobkoeiiegdccgcmpnj?pli=1
1
vote
Risk-benefit of cleaning up thousands of "Alternate page with proper canonical tag"?
Since the site ranks, and you don't describe any indexing issues here, it sounds to me like Google has correctly read between the lines and figured it out, despite whatever you have going on the back ...
Top 50 recent answers are included
Related Tags
googlebot × 929seo × 422
google-search-console × 218
web-crawlers × 209
google × 185
google-search × 119
robots.txt × 93
search-engine-indexing × 87
sitemap × 48
google-index × 47
javascript × 45
crawl-errors × 36
404 × 28
crawl-rate × 28
search-engines × 26
url × 23
duplicate-content × 22
301-redirect × 21
redirects × 20
ajax × 20
noindex × 19
html × 18
php × 17
links × 15
googlebot-mobile × 14