Tag Info

New answers tagged


The solution would be to serve the content as html for search engines and js for normal users. But that should be done in a way that is not considered cloaking. So the ideal way for it to happen would be to have exactly two copies of the same page - one using js and another using html and render the page accordingly based on the user agent of the requester. ...


The current GoogleBot Smartphone agent, as tested with the 'Fetch as Google' Tool is essentially a fake iPhone using a headless Webkit Safari 6.0 Engine, running on a Linux x86_64 desktop machine. The default non-responsive viewport width is that of an iPhone at 980px. With a viewport <meta name="viewport" content="width=device-width, ...


Here is Google's document about cloaking. It defines cloaking as "the practice of presenting different content or URLs to human users and search engines." That page also has a Matt Cutts video. In the video at 0:28, Matt defines cloaking as "showing different content to users than Googlebot." Google only cares what you show to real users and what you ...


Google only cares that they see the same thing your users see. If you're only serving up different content to these bad crawlers, and your users are getting your normal content and Google is getting that same content, you're ok.


You should check the site to confirm if the errors are happening or not. Also look at the server logs to see if something happened at the time Google was crawling the site and then resolved itself. It could be an intermitted error that resolved itself but might show up again. So steps to take: Confirm if errors exist on the site for the pages reported in ...


Googlebot will never stop crawling a URL if you once had content there. I have 10 year old sites that still get crawled at old URLs. You have two options: Return 410 Gone status Redirect the page to another URL Google treats the 410 Gone much like 404 and it still shows up in Webmaster Tools as an error, unfortunately.


There's no set time frame for when Google will stop. But they will eventually stop. They need to keep crawling them for a period of time because those pages may only be unavailable temporarily due to an error on your end. So they want to give them a chance to reappear in those cases. It is assumed if those pages were intentionally removed you would serve up ...

Top 50 recent answers are included