Fixing A Common Problem: Duplicate Websites

The Problem:

A very common mistake small businesses are making with their websites is having multiple web addresses by which to reach your home page.

If you go to www.yourdomain.com, the search engines often feel this is a different website to yourdomain.com (without the www)

While it isn’t critical, and most search engines are able to cope with this common problem, you may find you receive significant SEO and traffic improvements by sorting it out.

Potentially your website is being penalized due to duplicate copies of your content, or you have backlinks pointing to four separate pages.

Try each of these on your own websites, obviously replacing “yourdomain.com” with your actual domain name.

www.yourdomain.com

yourdomain.com

Both of the above should redirect to the same location, most appropriately www.yourdomain.com – if you find this isn’t the case, you are presenting at least two copies of your website to the search engines.

Note: If you find one of the above results in a 404 (page not found) error it is best to speak to your web host or website team about resolving this.

In addition – one of the following may also show a duplicate copy of your web site. Three of the below should result in a “page not found” error. The 4th should ideally redirect to www.yourdomain.com


www.yourdomain.com/index.htm

www.yourdomain.com/index.php

www.yourdomain.com/index.asp

www.yourdomain.com/index.html


If you find one of the above shows your website without redirecting to www.yourdomain.com, you are potentially presenting three or more duplicate copies of your website to the search engines.

As per above, one of the four following may be yet another duplicate of your home page.

yourdomain.com/index.htm

yourdomain.com/index.php

yourdomain.com/index.asp

yourdomain.com/index.html

If you find that you can access your homepage via more than one of the above links without being redirected to www.yourdomain.com, you have a little problem which needs to be addressed.

The solution:

Easiest: Contact your webhost and request “a 301 redirect of the non-www to the www, and from index files to the directory url.” – if they ask for further clarification you can provide a link to each of the separate ways you can view your website and say “I need these all to 301 redirect to www.yourdomain.com”

More difficult: DIY… While this depends greatly on your web host, the most common platform is a LAMP server (Linux, Apache, mySQL, PHP) – if this isn’t the case for you, you will need to research “Mod-ReWrite for <Your Platform>”

If, like most websites, yours is hosted on a LAMP server:

You will need to edit (or create) a .htaccess file. This is just a file in the root (“www” or “html”) directory of your web host which is named “.htaccess” – this file needs to contain the following text:

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /index\.html

RewriteRule (.*) http://www.yourdomain.com/ [R=301,L]

Replace “.html” with your index file extension, and “yourdomain.com” with your actual domain.

Warning: If you are using a CMS (Content Management System) you may have conflicting re-write rules. It is best to search for information about 301 permanent redirects that relates specifically to your CMS software. Most will have a plug-in which will take care of it all for you.

6 thoughts on “Fixing A Common Problem: Duplicate Websites

  1. Thanks, sometimes it is the little things like this that can be overlooked, or in my case i didn’t even consider this could be a problem. Cheers

  2. Hi Vince, thanks for another timely post.

    I’m confused however, I have pointed all non www to www(dot) and put the www(dot) version as the preference in webmaster tools.

    My concern I can get my head around is this >

    my website homepage is www(dot)mydomain(dot)com

    www(dot)mydomain(dot)com/index(dot)html is the same page though, is this duplicate content?

    for example in google analytics I have page views for “/” and “/index(dot)html” but they are the same page.

    my navigation links from other pages point to /index(dot)html, should I change this or redirect.

    so confused!
    dont worry if you cant answer i appreciate I’m not paying you!!

  3. You do have a slight problem :-)

    In theory all of your linking should point to the one location – www(dot)mydomain(dot)com – however in this isolated case you may be better off using the canonicalization tag.

    link rel=”canonical” href=”www(dot)mydomain(dot)com”

  4. CLICK Paid Search on

    Terrific detail of this common problem and provided solution. There’s many different ways you can setup a .htaccess file to remove fix this problem, and more. If you’re looking to add more information to this post you could you go into further detail for solutions other than the home page?

  5. Thank you for this Vince. I found that …/index.php doesn’t redirect to my home page, so have put in a request to my host for them to 301 redirect it to my home page.

    I hope you are very much enjoying married life :)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>