Thursday, October 27, 2005

[Search Engine] Allowing spiders on archive pages

Just a quick question...

I get a lot of traffic from search engines piping people to my archive pages. In the past, I've let search spiders scan my archives.

The problem is that that's not how I want people browsing my page. I'd prefer they read the individual posts. I'd like to "drive" traffic a little more, but I'm hesitant to lose readers and others discovering my page from search engines.

Your thoughts?

This post was written by William from Never Trust a Skinny Chef...


Kevin said...


"The problem is that that's not how I want people browsing my page. I'd prefer they read the individual posts. I'd like to "drive" traffic a little more"

Why? What is it you hope to accomplish by asserting more control? Are you concerned about losing new readers by restricting their opportunities to find you?

William Conway said...

Here's my take. I've been trying to get my blog to be a little more user-friendly recently. This includes not opening links in new pages and adding title tags for my links.

It's frustrating for me when I follow a search engine link to a blog and have to sift thru dozens of posts to find the relevant content that got me there in the first place. My hope is that by driving traffic toward individual posts that the potential new reader will take more time to read the post, and possibly decide that he or she wants to come back.

I'm trying very hard not to make my blog a personal popularity contest, but I started this blog for two reasons: 1)To have fun and 2)To add meaningful content to the internet (not always easy!). I view "getting the word out" as part of mission #2.

Andrew said...

I have the same thing (I think) Google sends traffic to the monthly page rather than the individual post page - which I would have thought was more relevant.

I could delete the monthly archive I guess but that seems drastic.

Kalyn Denny said...

I do not know much about this topic as far as the technology aspect, so if this is a stupid response, forgive me. However I just wanted to comment that I have gotten at least 15 e-mails in the last month from someone who says something like "I was searching for how to cook spagetti squash and I found your blog". So apparently sometimes people don't mind looking around a little if they like what they see. I should possibly add that I get far, far more search results on MSN and yahoo than I do on Google, which seems to index less often. I don't know why that is, but it seems pretty consistent.

William Conway said...

Yeah, I get a lot more traffic from MSN and yahoo, especially MSN.

You can add a no robots meta tag on all archive pages in blogger, but I guess after posting here and really thinking about it, there's no need to be psycho about how people find my site.