Avoid wordpress duplicate content

Have you wondered why your wordpress blog doesn’t rank very well, even for long tail terms like “build dynamic header and footer in php“?

One reason is that you might need to fix your permalink structure.

But more likely, you have five or six ways to link to your posts and this creates a massive duplicate content problem on your blog.

The problem

Wordpress makes it easy to find content on your blog by giving you many way to link to the content.

You can find the same post content in several sections of your blog such as: category, page, feed, archive, author, search.

AND if you are putting one blog post into several categories, you are creating even more duplicate content.

Here’s a example from Elliot’s blog. Have a look at the post titled “Affiliate Relationships on Geowebsites”.

post
page
category tropical birds
category lowell
category web development
search

Which URL do you think google or yahoo are going to rank? To a search engine, it’s not clear which link is the most relevant. And if they do rank any of these URL’s it will not be with any authority for the terms anyways, so someone else with a clear link structure using the same terms will outrank you every time.

Fix the duplicate content problem

This problem is simple to fix - just disallow parts of your blog from being spidered by search engines using your robots.txt file. There are lots of sites that either do not use robots.txt or do not use it effectively.

Here’s an example of the robots.txt that I use on this blog:

User-agent: *
Disallow: /wp-admin/
Disallow: /category/
Disallow: /page/
Disallow: /feed/
Disallow: /about/
Disallow: /contact/
Disallow: /disclaimer/
Disallow: /author/
Disallow: /cgi-bin/
Disallow: /download/
Disallow: /?*?/
Disallow: /200?/


What the above lines tell the search engines to do is ignore all the content in the category, page, feed, archive, author sections of the site and focus on one thing — just the blog post link. You are telling the search engines that the most important and most relevant link to your content is the direct link to your blog post, not the link from category or page or feed or search.

So please go fix your permalink structure and duplicate content so that you can increase your rank too!

If you need any help sorting it out, let me know.