Google Worsens Web Experience by Retuning Poor Search Results

Please read the notes at the end of the article for important updates to this Blog post (the last note added February 20, 2013).

After the browser Google has been the best thing the Internet has seen.

I have been a fan and user of Google for more than a decade.  The biggest reason is that Google, more than any entity before it, brought order to the worldwide web. Google’s search engine enabled one to find the highest quality content, quickly, easily and without bias.

Google’s YouTube allowed me to easily publish and share AALBC.com videos — increasing our video viewership by 2 orders of magnitude.  Google Analytics have proven to be an invaluable tool, providing deep insight into my website’s performance.  Google’s new social network Google+ seems to have addressed many of the concerns and issues I have with Facebook and other social platforms. Over the last few years revenues from Google’s AdSense easily covered all of AALBC.com’s overhead. Indeed, AdSense helped make it possible for me to grow AALBC.com from a sideline business into my livelihood.  I could go on, but if should be clear that I love what Google has provided.

Unfortunately my love affair with Google is becoming tainted with skepticism and caution. I no longer believe Google’s near monopolistic dominance of search is in our best interest. In fact, I’m beginning to believe Google’s dominance may be to our detriment over the long term.  The primary reason for this belief is that Google no longer returns the best search results.

Sure, one could easily assert that the “best search results” are a matter of opinion.  At some level, I agree with this sentiment.  However, the example, shown below, reflects an undesirable and recent trend I’ve observed in the relative quality in the ranking of Google’s search results, that I think most reasonable people would agree with.

Google search results tend to skew more toward scandal and away from meaningful content.   

For example, run the following query for the NY Times Bestselling author; “Terry McMillan”  You’ll find the following, or similar, results.

The 3rd ranked result – Two sentences on the Huffington Post site for a page with the title; “Terry McMillan Apologizes To Smiths For Calling Kids ‘Pimped'”

Here is all the text on the page:

Just a few hours after Terry McMillan went on a Twitter rant against Will and Jada Pinkett-Smith for “pimping and exploiting” their kids Jaden and Willow, she’s issued an apology.

It seems she couldn’t take the heat. However, she does make a good point.

That is all the text on the page related to Terry McMillan.  Again, this is the 3rd highest page returned by a search on “Terry McMillan” .  The 4th ranked search result is a USA Today page for an article based upon the same tweeted message entitled “Terry McMillan: Willow and Jaden being ‘pimped and exploited’.”

The fact USA Today used Terry’s tweet as a news story is, in and of itself, astonishing (subject for another rant) .  However, couple this with the fact that Google ranks the 7 month old, USA Today article 4th for a query on “Terry McMillan” and it becomes clear that Google search algorithm is doing something other than retuning sites with the best content.

The next remaining top ten search results include brief articles; including one from Orpah.com entitled, “Author Terry McMillan and ex-husband Jonathan Plummer return to discuss their relationship” (Sep 28, 2010) and the thesmokinggun.com sharing with us, “Terry McMillan Sues Her “Little Fag” Ex | The Smoking Gun” (Mar 22, 2007).

Websites that actually deal with Terry McMillan’s work are buried pages deep in the search results.  The AALBC.com page where I’ve published original video of Terry McMillan, a description of all of her books as well as links to original AALBC.com book reviews is the 11th ranked page.  One saving grace is that McMillan’s own website is ranked first.

Now I’m not saying that Google should not return websites containing scandalous or trivial information about someone — indeed they should.  The scandalous results should be returned to people who are clearly performing a search for that type of information.  The presumption, by Google, that someone querying “Terry McMillan” is looking for websites that provide two sentences about an inappropriate, Twitter message that Terry sent over 7 months ago is, at best, an insult to our intelligence.

Another saving grace is that the 2nd most popular search engine does not exhibit this bias toward scandal.  Compare Yahoo’s (Powered by Bing) search results with the Google search results on “Terry McMillan”.

This is just one example.  I can provide many, many more.  I’ve attempted to reach out to Google regarding this issue and have been unsuccessful.  This problem and related ones are no secret.  A great many websites are complaining about Google questionable search results.   Unfortunately for us, Google is so big and powerful they have no incentive to be responsive; despite the #1 statement of from Google’s philosophy being: “Focus on the user and all else will follow.”

While competition from other search engines provide a glimmer of hope against Google completely dominating search.  Today Google is the most popular search engine.  The majority of new visitors to AALBC.com arrive via a Google search.  I suspect the same is true for most other websites.

I’ve also heard and read additional speculation that Google skews their query results to sites that purchase a great deal of online adverting through Google’s AdWords.  While I can not prove this; the search results shown by the example above do not rule this assertion out.

Of course it could be that web page popularity is a significant factor in Google’s search algorithm.  The sites or pages with the most traffic would tend to rank higher.   Perhaps the very popular, AOL owned, Huffington Post can exert dominance and higher page ranking simply because of it prominence.

Even if the Huffington post brought $10 million dollars in Google ads and was more frequently visited than Facebook; there is no way one can justify that a page containing two sentences (quoted above) belong in the top ten of a search result for the query string “Terry McMillan“.    That Huffington Post  page would be appropriate result for a query entered something like this: Terry McMillan Tweet Willow Jaden Smith

I fully realize I could be making things worse for myself by making these accusations against Google in a public forum.  I have first hand experience with Google shooting first and asking questions later (yet another topic for a future rant).  However, this is a very important issue that needs additional scrutiny and awareness.  We are already losing on-line sources for quality news and information because of a lack of platforms for good journalists and writers.   Now the potential for these platforms are hampered even more by having their content devalued relative to sites that promote more scandalous or otherwise less valuable and relevant information.

As large corporate entities produce, broker, promote and manufacture more scandal, and companies like Google make this information more accessible by elevating it in it’s search results; we are witnessing the Internet becoming less free while corporate interests contort the world wide web in to a entity where profit is the only motive.

Notes: 

1. For the purpose of this article I ran my Google queries using Internet Explorer’s “InPrivate” viewing option.  Presumably, this returns results that are not manipulated based upon browsing history.
2. I also made sure I was not logged into my personal Google account.  I noticed when I’m logged into Google my search results skew toward sites that I frequent.
3. The quotes were included in the query.
4. Following the 3 steps above will improve the likelihood that you queries will return results identical to mine.   However feedback from other indicate the no mater how you run your Google queries the results will not differ so significantly to warrant a change of the opinion expressed in this article.

Troy’s Note (20 Feb 2013)

A GREAT new (to me) search engine http://DuckDuckGo.com

1. Each domain has only one link returned in the results. If you want multiple results from a single domain. You can click the “site search link” next to the results to get more results from that domain. No single domain dominates an entire page of results.
2. You’ll find only one sponsored link and it is very clearly labeled. Sponsors links do not dominate the most prominent area of the search results page. Visitors get answers not commercials.
3. If you want Wikipedia results just type !w at the end of your query and you will be taken directly to Wikipedia. These “bang” queries are available for other popular sites !a for Amazon !g for Google (which is really more of a eCommerce site than a search engine), etc.
4. Official websites are list listed first and denoted as being his “Official Site”.
5. This should have been listed 1st, DuckDuckGo does NOT track you rsearches. Read this: http://donttrack.us/ (it is brilliant).  No lame excuses (lies) explaining that your searches are tracked to bring you better search results or better advertisements.

Returning good search results quickly is very difficult but DuckDuckGo.com looks like it is up for the challenge and is able to do so with any observable bias in it results.  The results do not skew toward big corporations, or scandalous content  — I’m actually excited about this new search engine!

Try a query on Terry McMillan now!
_____________________

Troy’s Note (26 Jan 2012): I just watched 60 Minutes on CNBC which validated my observations below.  60 Minutes’ Leslie Stahl, after running a Google search on the term “60 Minutes”, asked Google’s Executive Chairman Eric Schmidt why the results skewed negative.  Eric said they had no way of distinguishing the popular favorable results from the popular negative results.  That response, not mincing words, is bullshit.  Google had no problem doing it in the past and Microsoft’s Bing search engine accomplishes this today.
_____________________

Troy’s Note (4 Dec 2011): I was really motivated to do something about the issues described in this article.  I decided to create a customized and curated search engine, by taking advantage of Google’s, fee based, Site Search.  The result is Huria Search: http://hura.org   (Huria Search Terry McMillan to see the difference)
_____________________