The WSJ reports that a study will be released tomorrow noting an 8% drop in total “paid search” revenues in 2008.  Google’s Fourth Quarter results will be released Thursday.  While this is clearly bad news for Google, Yahoo!, Microsoft and other companies that sell ads next to the results of their search engines, it’s also terrible news for the Internet users who have come to take for granted not just these free search engines, but the other free services and content cross-subsidized by search ad revenue.  A quick look at the offerings pages of Google,  Yahoo! and Microsoft (downloads and some services) should remind you of a few of these ad-supported offerings.

What’s even worse for users is that search ad spending may be the “canary in the coalmine” for online advertising overall:  A drop in search ad spending may suggest that display ad revenue for 2008 may have fared even worse.  While search ad revenue funds offerings from search engine providers, display ad revenue is the bread & butter of millions of websites, from the “short head” (big websites like ESPN.com) to through the “long tail” (small websites).   As advertisers cut back on buying web ads, there will be less funding available for “Free!” culture—and we’ll all suffer from the resulting decline in creativity and innovation.  

Let’s hope 2009 is a better year for advertising—both search and display—than 2008.

Jerry Yang’s departure as Yahoo! CEO opens the door to a renewed bid by Microsoft to buy Yahoo!’s search business (or Yahoo! itself).  Such a merger could produce a significantly stronger challenger to Google in the search market.  With this possibility in mind, the WSJ just ran a fascinating history of the “paid search” business—the placement of “contextually targeted” ads next to search engine results based on the search terms that produced those results.

In a nutshell, Microsoft failed to see (back in 1998-2003) the enormous potential of paid search—just as small start-ups (such as Google) were starting to develop the technology and business model that today account for a $12+ billion/year industry, which is  twice the size of the display ad market and which supports a great deal of the online content and services we have all come to take for granted online.  Microsoft first put its toe in the water of paid search with a small-scale partnership with Goto.com in 1999-2000.  But this partnership failed because of internal resistance from the managers of Microsoft’s display-ad program.  In 2000, Google launched Adwords and thus began its transformation from start-up into economic colossus.  By 2002, Microsoft realized that it needed to catchup fast, and approached Goto.com (by then renamed Overture) about a takeover.  But Microsoft ultimately chose in 2003 not to buy the startup because  Bill Gates and Steve Ballmer “balked at Overture’s valuation of $1 billion to $2 billion, arguing that Microsoft could create the same service for less.” 

Microsoft, meanwhile, spent the next 18 months deploying hundreds of programmers to build a search engine and a search-ad service, which it code-named Moonshot. The company launched its search engine in late 2004 and its search-ad system in May 2006.

But Microsoft’s ad system came too late:

Advertisers applauded Moonshot for its technical innovation. But Microsoft had trouble coaxing people to migrate to its search engine from Google; advertisers were unwilling to spend large sums on MSN’s search ads. By building a new system instead of buying Overture, Mr. Mehdi says, “we really delayed our time to market.”

What’s most fascinating about the piece is that it seems to suggest that Microsoft missed its opportunities to get into paid search not because it was “dumb,” “uninnovative” or a “bad” company, but for the same sorts of reasons that big, highly successful and even particularly innovative companies fail.  The reasons companies generally succeed in mastering “adaptive” innovation of the technologies behind their established business models are the very reasons why such great companies struggle to encourage or channel the “disruptive” innovation that renders their core technologies and business models obsolete.  This dynamic was described brilliantly in Harvard Business School professor Clayton Christensen’s classic 1997 book The Innovator’s Dilemma:  When New Technologies Cause Great Firms to Fail.  (Read chapter one here and Tim Lee’s recent discussion of the book here.)  

Whatever one thinks about the debate over whether antitrust intervention is necessary to restrain Google’s growth, I’m sure we’d all applaud the evolution of increased competition in the paid search market through market forces.  Let’s hope that Microsoft—as well as Yahoo!—have carefully studied the vast literature produced by business schools in the wake of Christensen’s book about how big companies can avoid the Innovator’s Dilemma by promoting—and capitalizing on—radical innovation from within.  Indeed, this seems to be precisely what has guided Google’s own strategy as it has grown from “disruptive innovator” to become the very sort of behemoth that cannot easily escape the Dilemma, even if corporate managers are fully aware of the problem on a theoretical level.  If Google can do it, Microsoft should be able to, too.  But let’s also not discount the possibility that, no matter how hard Google’s management might try to retain the innovative culture of a start-up, the giant  can’t do that well enough to prevent its own apparent market dominance from being disrupted by new upstart innovators in search and advertising technologies.  

The head of Google Research talked about some of these possibilities in July 2007 and the Google has recently covered other possibilities.  Here are my own bets—for what little they’re worth—as to what such “disruptors” might be:

• Semantic search and social search – whichever search engine masters these tools will likely dominate the market for search, and thus search advertising.
• Micro-payments to search users for using a search engine and discounts for clicking on ads – something Microsoft has pioneered with its Cashback system but which is probably still only in its infancy.
• Behavioral targeting that can make display ads competitive with search ads by making display ads as relevant to consumers as search ads (or even more so), rather than simply trying to target display ads based on the context of a page—which limits the economic value of the ad “display inventory” that websites try to fill with ads, especially for smaller websites in the Internet’s “long tail” whose subject matter might have little relevance to the keywords for products or services that are more highly valued by advertisers.  
• Technologies that allow contextual targeting of ads in or around videos based on the contents of the video (and associated discussion by viewers in comments). Even the imperfect ability to automatically create transcripts of a video, and then search for keywords, could hugely increased the value of advertising associated with video content.

I suspect we’d all be at least a little surprised if we could see what search engines—and online advertising—really looked like in, say, 2019.  But I won’t be terribly surprised if Google—for all its ingenuity—ends up making some of the same mistakes Microsoft made with Search 1.0 ( c. 1998-2005).