I came across this issue earlier today and it has to do with the way robots from search engines like Google (who uses anything else?) work with WordPress. This post is really a mix between search engine optimization as well as the inner workings of WordPress. We’ll dive right in to outline a problem and of course, a solution.

Say you’re like me and you blog about various topics like networking and web development. Chances are, you’ll categorize your content for better organization. So, with WordPress, you’ll now have a two separate URLs for your fresh new categories. These are called Category Pages and will generally list all of your blog posts within that respective category.

yourdomain.com/blog/category/networking
yourdomain.com/blog/category/web-development

You’ve continued blogging and along the way began tagging certain blog posts with keywords. Here’s an example; in one blog post under the web development category, you brushed up a little bit on networking and you decided to add a networking tag to that post. WordPress will in turn create another page for this tag. They’re called Tag Pages and will list the posts that have this tag.

yourdomain.com/blog/tag/networking

Later in the future, you’ve published what might be the highlight of your blogging life and create the ultimate 101 on all things Networking. Of course, you’ll call this blog post simply, “Networking” and you file it under the networking category. Depending on how your Permalinks are set up, WordPress will either create a page with the date of that post along with its title, or if you’re like me, the post’s title will fall under the category of that post.

yourdomain.com/blog/2013/01/17/networking
yourdomain.com/blog/networking/networking

I’m not sure if you noticed, but that second link doesn’t look pretty at all.

After a discussion with SEO and SEM expert Chris Bubinas, he explained where things can get a little confusing. You’ll have three separate URLs that reference networking, fighting for dominance on Google’s search results on the topic of networking. So, what content is important? What should Google’s crawlers index?

Chris brought up another interesting point. What if you’re a company and you want your keywords to funnel traffic back to your domain and not have your corporate site battle it out with your blog on Google? In this scenario, your company’s keyword page should have precedence over the keyword pages on your blog.

yourdomain.com/networking
yourdomain.com/blog/category/networking
yourdomain.com/blog/tag/networking

The quick and dirty here is to block your category and tag pages from being indexed but still followed. Add the following lines of code to your header.php file in your theme folder.

<?php  if (is_category() || is_tag()) { ?>
<meta name="robots" content="noindex,follow" />
<?php } ?>

With this, Google’s robots will not index these types of pages but will follow the links to the content in them.