2

I have an idea to use a (cc-by-sa) data dump from another site, where I will experiment with new ideas. This would still be a site with almost 99% of the data similar.

I can see two "rules" that search engines may apply

  • Don't duplicate content
  • Dont send different data to the bot and the ordinary visitors

Links to the original site will be present as required by the data dump.

If only the titles and other new data is shown to the bot it would solve the first rule but would that violate the second? Or should I simply put these pages in a robots.txt banned area? Would this be going to far.

I don't care about ranking, only not to be banned. It should be found if people were specifically looking for alternate voting schemes, from new content that I generate.

Improve this question
  • Why do you think this interpretation as duplicate would be 'wrongful'? – pkaeding Oct 8 '10 at 23:10
  • Yes, technically it is a duplication, what I was asking for was how to either avoid being interpreted as duplicate or avoid the duplication from triggering a ban. – hultqvist Oct 9 '10 at 8:39
5

Duplicate content is duplicate content. Changing the voting mechanism won't mean a thing. The duplicate content will be filtered out and do you no good. As far as SEO goes, it's a waste of time.

And the Internet doesn't need more of the same. Try creating new content. It's what people are looking for.

Improve this answer
  • I think this might be a legitimate use of duplicate content, but you're right; it's still duplicate content. Some SEO penalties should be expect, but ideally, it should not be to punish the site owner, but rather to make sure SERPs don't contain duplicate entries. So as long as he makes the site useful (e.g. by improving the voting system), then he can still have decent search ranking if people like the site and link to it. If you're going to make a site with legitimate reasons for duplicate content (e.g. syndicated news, press releases, etc.), then you just have to work around it. – Lèse majesté Oct 8 '10 at 14:47
  • 1
    IMHO just changing the voting system and using the original content doesn't really offer the Internet as a whole anything special or useful. It just sounds like piggy backing off of someone else's hard work to me. Plus the core content is the data which will be unchanged. I see no reason for this content to ever show up in the search results. (Removing duplicate content isn't a punishment. It's a smart way to make the search results cleaner and more useful. There's no need to have the same content over and over again especially in cases like this). – John Conde♦ Oct 8 '10 at 15:00
  • I understand what you mean about keeping out duplicate content, and I agree with you for the most part. But there has to be a distinction made (by the search engine) between people who post duplicate content because they're, e.g., posting a press release from another company, and people who just copy-and-paste articles to generate pages for a link wheel or to steal content for use on their spam site. – Lèse majesté Oct 8 '10 at 22:20
  • Also, whether this is a legitimate use or not depends on how bad the current voting system is and how much better he can make it. If it's like all the Wikipedia clones out there, where it's just a superficial layout change with no innovation or added value, then sure, keep it out of SERPs. But if this site implements an ingenious and much more effective collaborative filtering system, then it deserves to be indexed even if the data set isn't unique (think about all the mashups out there using CC data in ingenious new ways). – Lèse majesté Oct 8 '10 at 22:22
  • 1
    +1 John, I agree completely. @eovleg - Even though StackOverflow's content is available under a CC license, I really think you'd be wasting your time cloning it. Is the content valuable? Yes, but what's even more valuable is the community, and that's something you can't easily replicate. What's more, I don't think that StackOverflow is so fundamentally broken that there is a need for a clone with a different voting system. I personally wouldn't see any use for it and I doubt that you'll find enough people who do to make it worth your effort. – Virtuosi Media Oct 9 '10 at 17:16
0

Duplicate content on two or more URLs even on the same domain is bad(and it may bring you penalty from search engines). Can you describe further why exactly do you need to copy the content.

Improve this answer
  • My question is general where the data dump comes from a place I have not control over but I can legally get and reuse its data. More specifically I was now thinking of the StackOverflow data dump. – hultqvist Oct 8 '10 at 9:39
  • AFAIK coping long contents will bring you troubles. If you show only title and short texts(like some kind of aggregation) this should be ok. But if you cope all the content that will be bad. – Ilian Iliev Oct 8 '10 at 9:44
0

You say you don't care about ranking - only not to be banned. If so, you could update your robots.txt file to indicate that the duplicated content should not be indexed.

Improve this answer
  • Does your answer imply that server-side hiding the duplicate data to the web-spiders is not an option? – hultqvist Oct 15 '10 at 10:09
  • No you can certainly hide the duplicate data server side - by the robots.txt is most likely an easier solution. – user978 Oct 15 '10 at 18:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.