New answers tagged


This is a very common problem faced these days. This can be easily solved as follows: If you want to have same page /pets but also drive /pets/cats, /pets/dog to index separately, then ideally they should exist separately i.e. if opened by a http client with a separate url they should respond independently. Also the content populated via ajax or via ...


GET requests could be seen as unique url. So any bot can access get data more quickly as unique url and judge contents. You can also cross verify all GET ajax calls againsts request url and desired format.


Googlebot only performs POST requests under very limited circumstances where it is believed by the Googlebot that it is safe and appropriate. Google takes precautions to avoid performing tasks on a site that could result in executing an unintended user action and Google making POST requests is for crawling purposes only to index what the end user would see. ...


Google is improving it's ability to read javascript rendered pages. However, it's not to a point you can rely on it for inserting critical headers. Here is one such article on researching this possibility. For now, you need to have the important information embedded into the HTML when it's served. There is also the chance that javascript rendered changes ...


This may be an old question but for the purposes of providing it an up to date answer, this is no longer required. Google no longer needs, and in fact encourages users not, to use escape fragments for the URL's as the Googlebot is now able to understand javascript generated page content without needing a URL escape fragment. This can be seen in the ...


Since this has been posted Google has changed the way the index websites and can now even index AJAX pages and the Googlebot sees the page the same as the end user sees it including interactive javascript content and AJAX powered content without needing to make any special changes to your code to suit the Googlebot engine specifically.

Top 50 recent answers are included