Here is the issue. We are using a third party service to index our products and return results based on query parameters that we pass to the third party services server.

I'm using Magento's framework.

Here is how it works: I place a curl request to the third parties server with the various query parameters taken from both the request and pre-defined variables(filter,sort,etc) passed through a particular pages layout update.

The response is turned into an array and a collection is queried from our database based on the SKU's that the third party supplied through the curl request and they are displayed in their own div.

Secondly we get results from their service through their javascript api. At the callback of the results being returned I hide the divs that display the results that are generated server side.

As a result , if javascript is disabled .. and more importantly , if you are a search spider coming to the site, you will still see results.

My question is: Are there any problems with this model? Could we have any seo penalties as a result? Any other suggestions?

  • Is this in reference to an internal search or search result page? Or is this happening on all or many pages? Feb 26, 2013 at 18:56

2 Answers 2


There is one big problem. Google never wants to index a page of search results in its own search results. Your site search pages should be blocked from Googlebot with robots.txt or you risk getting your site penalized from the Google index.

  • We are using the 3rd party two fold. Both to provide us with a quick efficient way to search and also to deliver the products for our category pages. Instead of "moving" products to a category.. Products are qualified to a category based on their attributes. The products included in a "category" must have x , y and z attributes. I am speaking specifically about the category pages. Our search is already blocked by robots and I think in webmaster tools.
    – ajameswolf
    Feb 26, 2013 at 23:27
  • E.g this would not be "search results". Would the fact that were delivering a different experience depending on wither the user has javascript enabled.
    – ajameswolf
    Mar 1, 2013 at 14:19
  • If you are sure that these are not going to be considered search results, then this answer isn't helpful to you. I've provided a second answer that hopefully gets to the meat of the question. Mar 1, 2013 at 15:02

Providing an alternate way to access your site without AJAX is a must for search engine optimization.

Something about your particular implementation confuses me. If you are providing content via this server side method, then why to the JavaScript AJAX call at all? Presumably this means that there is some difference between what users with JavaScript see and what search engine spiders will see.

This could be a problem. Google doesn't like their bot seeing anything different than users. They call it cloaking. If the differences are cosmetic but with the same content, I wouldn't worry about it. It won't hurt Googlebot's feelings not get the right colors or layout. If the content shown is actually different in the two cases, that could be a problem.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.