Wednesday, February 13th, 2008

Ummm… Microsoft… It’s 2008…

This one boggles the mind of anyone who understands the announcement... Microsoft just announced that it's crawler (msnbot) will support "if-modified-since" and gzip compression. If you understand what that means think about what year this is... It's 2008. A good spider would have been doing that, what - a decade ago (or more)?

If you don't understand those terms, it means that pages will now be compressed between the web site and Microsoft, using less bandwidth in the process (for both parties - webmasters hate spiders that use up their bandwidth), and they now won't download a page unless the web site confirms that the page has changed since the last time it was downloaded. Again, using less bandwidth.

I'm just shocked they didn't do this YEARS ago... If you ever wondered why their MSN/Live's search results sucked compared to say Google, it looks like inefficiency is one of the reasons.

No wonder they want to buy Yahoo!

Categories: MSN/Live, Spiders/Bots

Leave a Reply