Fixing A Common Problem: Duplicate Websites
Make sure to get FREE DIY SEO link packages Vince's Links
The Problem:
A very common mistake small businesses are making with their websites is having multiple web addresses by which to reach your home page.
If you go to www.yourdomain.com, the search engines often feel this is a different website to yourdomain.com (without the www)
While it isn’t critical, and most search engines are able to cope with this common problem, you may find you receive significant SEO and traffic improvements by sorting it out.
Potentially your website is being penalized due to duplicate copies of your content, or you have backlinks pointing to four separate pages.
Try each of these on your own websites, obviously replacing “yourdomain.com” with your actual domain name.
www.yourdomain.com
yourdomain.com
Both of the above should redirect to the same location, most appropriately www.yourdomain.com – if you find this isn’t the case, you are presenting at least two copies of your website to the search engines.
Note: If you find one of the above results in a 404 (page not found) error it is best to speak to your web host or website team about resolving this.
In addition – one of the following may also show a duplicate copy of your web site. Three of the below should result in a “page not found” error. The 4th should ideally redirect to www.yourdomain.com
www.yourdomain.com/index.htm
www.yourdomain.com/index.php
www.yourdomain.com/index.asp
www.yourdomain.com/index.html
If you find one of the above shows your website without redirecting to www.yourdomain.com, you are potentially presenting three or more duplicate copies of your website to the search engines.
As per above, one of the four following may be yet another duplicate of your home page.
yourdomain.com/index.htm
yourdomain.com/index.php
yourdomain.com/index.asp
yourdomain.com/index.html
If you find that you can access your homepage via more than one of the above links without being redirected to www.yourdomain.com, you have a little problem which needs to be addressed.
The solution:
Easiest: Contact your webhost and request “a 301 redirect of the non-www to the www, and from index files to the directory url.” – if they ask for further clarification you can provide a link to each of the separate ways you can view your website and say “I need these all to 301 redirect to www.yourdomain.com”
More difficult: DIY… While this depends greatly on your web host, the most common platform is a LAMP server (Linux, Apache, mySQL, PHP) – if this isn’t the case for you, you will need to research “Mod-ReWrite for <Your Platform>”
If, like most websites, yours is hosted on a LAMP server:
You will need to edit (or create) a .htaccess file. This is just a file in the root (“www” or “html”) directory of your web host which is named “.htaccess” – this file needs to contain the following text:
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /index\.html
RewriteRule (.*) http://www.yourdomain.com/ [R=301,L]
Replace “.html” with your index file extension, and “yourdomain.com” with your actual domain.
Warning: If you are using a CMS (Content Management System) you may have conflicting re-write rules. It is best to search for information about 301 permanent redirects that relates specifically to your CMS software. Most will have a plug-in which will take care of it all for you.
Tags: .htaccess, 301 redirect, duplicate content, Mod-Rewrite, SEO
SEO Guide for 2010
Thanks, sometimes it is the little things like this that can be overlooked, or in my case i didn’t even consider this could be a problem. Cheers
Hi Vince, thanks for another timely post.
I’m confused however, I have pointed all non www to www(dot) and put the www(dot) version as the preference in webmaster tools.
My concern I can get my head around is this >
my website homepage is www(dot)mydomain(dot)com
www(dot)mydomain(dot)com/index(dot)html is the same page though, is this duplicate content?
for example in google analytics I have page views for “/” and “/index(dot)html” but they are the same page.
my navigation links from other pages point to /index(dot)html, should I change this or redirect.
so confused!
dont worry if you cant answer i appreciate I’m not paying you!!
You do have a slight problem
In theory all of your linking should point to the one location - www(dot)mydomain(dot)com - however in this isolated case you may be better off using the canonicalization tag.
link rel=”canonical” href=”www(dot)mydomain(dot)com”
Thanks Vince much appreciated, I’ll look into this
Have a great new year
Danny
Terrific detail of this common problem and provided solution. There’s many different ways you can setup a .htaccess file to remove fix this problem, and more. If you’re looking to add more information to this post you could you go into further detail for solutions other than the home page?
Thank you for this Vince. I found that …/index.php doesn’t redirect to my home page, so have put in a request to my host for them to 301 redirect it to my home page.
I hope you are very much enjoying married life