Not like this is news to anyone in the tech world, but I’m constantly amazed at how bad Microsoft’s search engine is. I haven’t done a lot of detailed comparisons, but one ranking I’m always interested in is how well my own sites are doing. Here’s a comparison.
KEYWORD: pocket pc
SITE: Pocket PC Thoughts
Google: 5th result
Live Search: not in the top 120 results
KEYWORD: smartphone
SITE: Smartphone Thoughts
Google: 4th result
Live Search: 53rd result
KEYWORDS: digital media
SITE: Digital Media Thoughts
Google: 4th result
Live Search: not in the top 120 results
KEYWORD: zune
SITE: Zune Thoughts
Google: 17th result
Live Search: 64th result
It’s not that the results returned by Live Search are all that bad – they all seem to be fairly relevant. But the search engine seems to hate my sites and I can’t figure out why. So either Google has it all wrong, and my sites are not valuable resources in their respective categories, or Live Search has such a radically different method of ranking that my sites simply don’t matter. I’d disagree with that of course, so for now Live Search sucks in my book.
UPDATE: Well, turns out the egg is on my face. Although I never explicitly blocked the Live Search bot from scanning my sites, it seems I had an old robots.txt file on there that was stopping it from indexing the content on all my sites save Zune Thoughts. So my apologies to the Live Search team. I’ve now deleted the robots.txt files off my sites, so come back and visit me Mr. Live Search Bot!
Hi Jason,
Thanks for taking the time to write this up and for using our engine — feedback is always appreciated! While I cannot go into detail as to why we are ranking your pages where we are ranking them I can say that we are constantly working to improve our results.
One thing that will definitely help is if we can crawl your sites! While investigating your sites I did notice that the robots.txt on ALL of your sites except the Zune is allowing many crawlers: http://www.digitalmediathoughts.com/robots.txt, but it is NOT allowing our crawler.
We therefore will fall back to the catch all rule which is to block all other crawlers. We would love to have your content in our index so please allow our crawler — the user agent is “MSNBot”. Yes, I know that is confusing that the product name is Live Search and the User-Agent is MSNBot.
Eytan Seidman
Live Search Team
Well that’s certainly interesting – I was using a very old robots.txt file that I honestly hadn’t looked at in a long time…I’ve actually now removed it completely from my three sites, so that should help matters. Thanks for taking the time to stop by, I appreciate it, and I apologize for ranting about something where it was mostly my own damn fault. 😉