SEO Showdown: Block Robots from indexing WordPress Category and Tag Pages

I came across this issue earlier today and it has to do with the way robots from search engines like Google (who uses anything else?) work with WordPress. This post is really a mix between search engine optimization as well as the inner workings of WordPress. We’ll dive right in to outline a problem and of course, a solution.

Say you’re like me and you blog about various topics like networking and web development. Chances are, you’ll categorize your content for better organization. So, with WordPress, you’ll now have a two separate URLs for your fresh new categories. These are called Category Pages and will generally list all of your blog posts within that respective category.

You’ve continued blogging and along the way began tagging certain blog posts with keywords. Here’s an example; in one blog post under the web development category, you brushed up a little bit on networking and you decided to add a networking tag to that post. WordPress will in turn create another page for this tag. They’re called Tag Pages and will list the posts that have this tag.

Later in the future, you’ve published what might be the highlight of your blogging life and create the ultimate 101 on all things Networking. Of course, you’ll call this blog post simply, “Networking” and you file it under the networking category. Depending on how your Permalinks are set up, WordPress will either create a page with the date of that post along with its title, or if you’re like me, the post’s title will fall under the category of that post.

I’m not sure if you noticed, but that second link doesn’t look pretty at all.

After a discussion with SEO and SEM expert Chris Bubinas, he explained where things can get a little confusing. You’ll have three separate URLs that reference networking, fighting for dominance on Google’s search results on the topic of networking. So, what content is important? What should Google’s crawlers index?

Chris brought up another interesting point. What if you’re a company and you want your keywords to funnel traffic back to your domain and not have your corporate site battle it out with your blog on Google? In this scenario, your company’s keyword page should have precedence over the keyword pages on your blog.

The quick and dirty here is to block your category and tag pages from being indexed but still followed. Add the following lines of code to your header.php file in your theme folder.

<?php  if (is_category() || is_tag()) { ?>
<meta name="robots" content="noindex,follow" />
<?php } ?>

With this, Google’s robots will not index these types of pages but will follow the links to the content in them.


  1. Eric Hollebone

    Or use the canonical meta tag to tell google which url matters

  2. ahmeddirie

    Thanks for the heads up on that. I was told that canonical links are used on pages with identical sets of content with minor differences. I would assume that category and tag pages aren’t minor.

  3. Eric Hollebone

    This kinda falls into the category of build for humans first (visitors) and machines second (googlebot). Humans need multiple ways to find information; bots do what they are told.

    If there is only one piece of content but multiple ways to navigate/find it then canonical is your friend

  4. Michelle

    Awesome, and exactly what I was looking for. Thanks :)

    I tried All-in-one-seo pack before, but all that did was slow down my website. Your short code worked like a charm. Now all I have to do is see if Google follows what I’m telling them to do :)

    Thanks again!

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>