Duplicate Content, Copyright, and SEO

Duplicate content and its effect on search rankings

by Bill Marshall
of http://www.oyster-web.co.uk

Last updated: 12 Sep 2007

Duplicate Content is a subject that seems to cause a lot of confusion, yet the basic principles are easy to understand. The search engines want to provide the best results they can to their users; that means that they want each result to be unique and useful. If some of the results contain the same information then that lessens their value. Given that, it makes sense that if a company has two sites containing the same content then even if it is the best content around for a particular Search Term then the search engines don't want to display both copies - they will attempt to establish which is the original and they will list that one and drop the other. And that applies even when everything else about the two sites is perfectly ethical and above board.

Because of this there is really no point in having multiple domains with the same content and it could have unwelcome consequences.

For instance if page A on site 1 is linked to from an external site then it may get listed rather than the same page on site 2.
But if the opposite occurs with page B then it may be the site 2 version that is listed of that. Neither site will have all its pages indexed.

The likely effect is that you'd have two weaker sites instead of one stronger one and with less internal strength the weaker ones will do less well in the search Rankings than the one stronger one would.

A similar situation can occur if you have say, a .com version of a name and a .co.uk or a .uk.com etc. If they all point to the same server and there is no redirection taking place then they are likely to be seen as separate sites with identical content.

In the first scenario the choice is between putting separate content on each domain name that you own, or pointing one site to the other, usually at the domain administration level.

In the second scenario you can either rent more server space for the other domains (and again have separate content) or choose which is to be the main domain. Then apply a Redirect which tells any browser or search spider which requests a file from the subsidiary domains to get it from the main one. Thus only one domain name will be seen by the search engines but the value of any existing links pointing to the others will be maintained.

It is important that the correct type of redirect is employed. Some Hosting companies offers different types of redirect built-in but these are often of an undesirable type. For instance a framed redirect simply loads one address within a frame in another address. Another type of redirect results in a server response called a 302 temporary redirect. Search engines generally don't like these and if they appear on an address for longer than a few weeks then they are likely to be treated with suspicion. The correct response to set up is a 301 permanent redirect which can be done using an htaccess file on an Apache server. It used to be more of a problem setting up a 301 on an IIS server but the more recent versions should provide this via their control panel.

That covers possible duplication of your own content - turning to other people's content you are of course subject to the laws of copyright. If someone has written their own original content then they are entitled to exclusive use of it unless they actively decide otherwise and it cannot be copied without permission other than quoting an extract in order to comment on it. You can of course include an XML feed in a page if the owner makes it available in this manner, but they are still the owners of the piece.

If you find that someone has stolen your content then if they are within your legal area you are fully entitled to pursue them and request that they remove the content or face legal proceedings. (Of course this doesn't help if they are in Taiwan or Russia, but then the effect on your search results in the UK or USA will usually be small at worst.)

Whether the existence of stolen content affects your search ranking depends largely on who the search engines believe was the original creator. Do a search for a section of text (about 15 words is usually sufficient) inside quotation marks. If your copy appears first in the results then there is little to worry about as regards ranking, though of course you may be concerned about other aspects and wish to pursue them.

If on the other hand your content appears below a site using a stolen version then you have a problem. You may wish to try to get the other site to remove the content or you may decide it's not worth the bother and rewrite your own. Much depends on how good the content is and whether you think you can improve it. Whichever you decide it makes sense to record the content in case you find that the same site pinches that too. You are then in a much better position to complain, whether legally or to the search engines themselves requesting that they review the other site.

One other thing to mention. If you syndicate any of your content then make sure your version is being spidered first before you release it, otherwise it may appear that other sites had it first.