MS vs Google - The search engine wars
Log files show the intensity of the power struggle
Remember when getting listed in decent, big name search engines was a combination of luck and skill? Not anymore. With the battle for supremacy, between Microsoft and Google, the rules look to be changing. Once upon a time, if you wanted to get into the big boys' indexes, you needed to spend time strategically choosing and arranging your keywords, description and other undisplayed meta tags, being very careful to tie it in with the actual visible content of your pages. Even then, there was no guarantee of inclusion and one false move could see your hard work undone as your site disappeared overnight from the search engine results. Occasionally, you would see a search engine's spider or bot, signing into your web server log files and wonder when you might see it appear again. Nowadays, things definitely seem a whole lot different.
So what's changed? Well, I know that I haven't become a search engine guru, since the days I described above, that's for sure. Fierce competition between the big operators, namely Microsoft's MSN search and the omnipresent and much admired Google, has seen bot visit frequencies rise and sites being crawled without even visiting an 'Add my site/URL' form anywhere. As a prime example, look at the source of any page here and you will see that I have made no real effort to make my pages appeal to search engines. Over at live.LAMPpost.info, I made quite some effort to make my HTML output as irresistible as possible to the search engine's indexing software and admittedly, I'm satisfied with the results. As my server setup is as home-grown as possible, being self hosted over my residential broadband service, I do not seek big traffic as my 45KB/s upstream just wouldn't cope with too many simultaneous visitors at any one time. Looking at my recent logs and I accept that this may just be coincidence, there is a battle royale going on, with googlebot and msnbot being among my most frequent visitors. I say a battle, as it seems that every few minutes or so one or the other and sometimes, 'A.N.Other-bot' is making an appearance. The interesting thing, is that they roughly alternate between the big two. It's like their scrapping, shoulder to shoulder, jammed in the door frame like an old slap-stick comedy.
Mine's bigger than your's and probably your Dad's too.
It seems like there is a major 'penis comparison'-like contest between them, to see who has the biggest, database of course. In my humble opinion, there has never been a better time to run and provide content to your own website. I modestly accept that there is a possibility that over time, I have weeded out the search engine no-no's from within my mark-up and that my pages have subsequently become more SE-friendly, but I just can't see it. Again, check this page's source, there's nothing special whatsoever, bar a single instruction to the bots and the obligatory robots.txt file. In fact, considering that these pages are essentially flat markup with the odd bit of dynamic html generation, I was pretty surprised to find the SlashBoot.info domain listed in Google at all, especially seeing as I haven't directly asked Google to crawl it. It's obviously found it's way here through following links to it from other places on the net and proceeded to follow my own links around my pages. My plan was to build up some decent content and to bring an updated version of my LAMPpost CMS over to handle the content generation, from a database, before even starting to get the search engines involved. They've beaten me to it, which I have mixed emotions about, but I am not complaining. I'd have rather have gotten it into shape to hit the rankings 'with a bullet' as all crap radio DJs are prone to saying.
If you keep your finger on the pulse of the SE world, as I used to do before my priorities changed with the arrivals of my youngest two in the last couple of years or so, then these developments will be of no surprise to you. I have read a fair few articles about the competition between the now sworn enemies, but didn't expect this added 'bonus'. From time to time, I like to insert unusual wording into my content, to help measure my effectiveness and using the term 'SMTP forwarder' as opposed to the more common and more correct, 'SMTP gateway' I found myself top of an albeit fairly short list of suggested links in Google. Remember, I haven't requested a crawl of slashboot.info or slashboot.org, unless it was in a drunken stupor. I've not overly linked to the domain(s) from my other main domain name, LAMPpost.info either. As I haven't really told anybody about this place, it's a pleasant surprise to see the favourable inclusions. I didn't even add the robots.txt and meta tags, until today, just prior to starting this article, so I think my surprise is fairly justified.
Where will it lead?
It's unlikely to start earning us money, by providing the search engines with content to suggest to searchers, when we remember the AllAdvantage scams and the stink over manipulated click through figures by the more unscrupulous advertising agencies, or banner farms. However, with Google's mentality and innovative style, nothing would surprise me and I think that they would be capable of almost ensuring that any such system wasn't abused or used fraudulently. So you never do know what the future may hold for hobbyist and the more business minded webmaster. At the end of the day, if you are not charged for bandwidth usage/data transfer and your site can cope with any increased traffic, it can only be good for website owners, regardless of any money making opportunities. Personally, I haven't sought to make any money from my websites through adverts and so on, for a good few years now and despite the GoogleAds on here, I still don't. In fact, I haven't made a penny from them as yet, but gain other benefits from having them in place, like the fact that it helps to get pages indexed in the Google database faster. Pages that display GoogleAds get spidered by the Google systems, to enable them to present contextual ads. There's method in my madness!
Post a comment:
| 
| 
