How to determine if I should hide a page from search robots

Question

We have hundreds of duplicates detected on our site but, these pages are not actually duplication's, they're just really thin content. Our site is an eCommerce site and these are product pages but these products are not the main focus of the site. These pages do not produce much revenue and does not attract much organic traffic.

We have 3 ideas on how to deal with this:

1. Write unique content for every page and optimise them hopefully improving their rankings and traffic/revenue.

2. Automatically populate the pages with content taken from different areas to make it unique.

3. Hide these pages from the search.

It would be economically taxing to create all of the content ourselves so we would like an automatic way of quickly dealing with this duplication issue.

There are hundreds of products that are very similar but not the same so optimising each product and avoiding keyword canibalisation is a great challenge and dose not feel worth it as there are more important pages to deal with. If we optimise more specifically for these products, the volumes of searches are very low and some none at all.

So my questions are:

1. What is the best way to deal with this issue?
2. Is there any point writing hundreds of unique pieces of content for terms that are not often searched.
3. Is it likely hiding these pages from the search will have a negative impact because even though the organic traffic it was receiving will be gone, the duplications will also be gone which should improve the overall quality of the site.
4. Have I looked into this deep enough and is there anything I should be more aware of or find more information about?

Answer 1

For each page you have three options:

• Noindex the page to prevent Google from including it
• Add content to the page and change keywords
• Do nothing and allow Google to index thin duplicate content

Noindex

Any page that wouldn't satisfy a user coming from search should be removed from the search results. This would include

• Pages for products that are discontinued or out of stock
• Pages with insufficient information to allow the user to answer their question (like "should I buy this?" or "does this come in blue?") are likely to hurt your site as a whole.
• Pages with poor quality content such as bad spelling and grammar

When users land on such poor pages, they quickly turn around and leave your site. Google pays attention to this bounce back. Your site will quickly be de-ranked for that query. If enough users indicate they want to leave your site like this, you whole site could get an algorithmic penalty.

Add Content

As you say, adding content is a lot of work. You would want to add content and de-duplicate pages when there is the potential to bring in additional search traffic.

Do Nothing

Internally duplicated content doesn't usually hurt your site. See What is duplicate content and how can I avoid being penalized for it on my site? Most of the time, you can get away with doing nothing to pages. Google will just choose one of the two duplicate pages to index. Generally it will choose the better linked of the two.

Answer 2

It would be economically taxing to create all of the content ourselves so we would like an automatic way of quickly dealing with this duplication issue

correct me if I am wrong but you want to automate the writing process? If my assumption is true don't ever do automated content generation. It will screw you up bigtime.

You will be producing more duplicate and very low quality content. And assuming I am a potential customer browsing your site and reading the item specs and description. I would leave and won't purchase any items since I can't basically understand what I am reading on your site.

In addition to @Wingit86

categorize your items and review them properly. Since their might be a tendency just like wingit's example that I am searching for a Samsung monitor and a coffee cup would show then that's very irrelevant to both search engine and to the customer.

Answer 3

If your site doesn't have a search bar, add one.

Only include your most important products on the search engine. If someone searches for a monitor on a PC equipment site, they should see a monitor. If they search for a cup holder, they shouldn't see anything.

Other than that, I recommend removing the product with the least views/purchases and increasing the content on the other pages.