Darrin Chandler

The Search Monoculture by Darrin Chandler

I've been using the Internet since the mid 1990's. The thing I've always loved about the Internet is that you can find almost anything, however obscure. It's an excellent resource, as long as you take the source into account.

I've been using Google for years, and loved it. I practically never used anything else for search. But lately I've found Google search results lacking now and then, and I've been going to other search engines more and more often. I wondered why this should be, but some recent experiences gave me some clues...

In The Beginning

Many years ago at the office, we tried to find web pages for various police departments. It wasn't easy. Eventually we found quite a few, but we had to dig *hard* to uncover them. So we figured that a list of police department web sites would be a nice resource to provide, and we were right. Within months the majority of web traffic was for that page, we had top rankings in all the major search engines, and many sites linked to our list. That was a big lesson for me: when you *can't* easily find something on the Internet it's an opportunity. Spend some time researching, make a web page, and the people will come. I've done this many times over the years, and I've watched the page hits grow.

The Changing Face Of Search

That's all changed now. Lately I made two pages with hard to find information and watched the hits ramp up as they got indexed. Then the hits dropped off to almost nothing! Why? Did a bunch of other sites displace me in the rankings? I searched Google with previously popular, rather specific searches. Google returned only two pages of results, and my pages weren't among them at all. Google had de-indexed my pages completely!

Admittedly, these two pages were not wildly popular, but they got hits. For the search terms that found the pages they were highly relevant. More relevant than the other pages returned, otherwise I wouldn't have bothered to make the pages in the first place.

Presumably, Google de-indexed my pages because no other pages linked to them (yet). They must not be very good if nobody linked to them (yet). But they had a very short life on Google, so how were they supposed to get links? Most visitors merely took the information and ran. Most or all of them don't even have web sites to link to me. It takes time to build up links. Google never gave me that time, even though people were finding my pages.

Market or Die?

What are my options? 1) I can live with things the way they are and take the hits I get from other search engines (few and far between). I'll eventually get some links and Google might put me back in their index. Some day. That's a *really* long, slow process. Or, 2) I can proactively get other sites to link to me.

If these pages were a commercial venture, my personal hobby, or something else I have a big investment in, I would do just that. But I'm only doing this to provide hard to find information, out of the goodness of my heart.

Feedback

When Google was relatively obscure they were only observers. Their PageRank algorithm (based largely on link popularity) gave them better search results. Much better. But only a few savvy people used Google, so it didn't have much effect. As Google grew in popularity, it began to have profound impact. Now that they've gained dominance in the Search Engine category, they've become a feedback system. I.e, they influence the thing they measure. Good PageRank increases your PageRank. Poor PageRank decreases your PageRank. If you don't rank, then you get dropped out of the index altogether. As this feedback continues, Google's results will get worse. They already have. Eventually, Google will only return Britney Spears fan sites and Britney Spears fake nude photos.

If you buy books from brick and mortar stores then you've probably seen a similar feedback system working there. When a new bookstore opens they carry a lot of titles on numerous subjects. As their inventory control system kicks in they devote more and more shelf space to copies of the bestsellers at the expense of everything else. Then I stop going. I can get the latest bestseller *anywhere*.

There are only a few ways to find things on the Internet: search engines, portals, news sites, and web logs. That's about the entire list. Any other resource, such as an enthusiast site with information and links, is usually found first by visiting one of the above. Search engines have the biggest impact because they're the first resort when you want to find something and don't know where it might be, which is most of the time for most people.

Summary

Okay, what have we learned?

• If nobody links to you, Google won't list you.
• If Google doesn't list you, then nobody will link to you.

Which means:

• New content is penalized by default, essentially unfindable: dead.
• You must actively promote your site or it will languish in obscurity.

And that results in:

• Purely informational sites are harder to find than ever before.
• People/Companies who hire SEOs and Link Exchangers, or who do that work themselves get a big ranking bonus, even if their content stinks.

So Google is encouraging SEOs and marketeers while discouraging small informational sites? Doesn't sound like the Google I knew and loved. Like all search engines, Google is always tweaking their algorithms. I hope they manage to get some of the more “interesting” pages back in their index.