My New Empirical Study on Defining and Measuring Search Bias

by on November 3, 2011 · 2 comments

[Cross posted at Truthonthemarket.com]

Tomorrow is the deadline for Eric Schmidt to send his replies to the Senate Judiciary Committee’s follow up questions from his appearance at a hearing on Google antitrust issues last month.  At the hearing, not surprisingly, search neutrality was a hot topic, with representatives from the likes of Yelp and Nextag, as well as Expedia’s lawyer, Tom Barnett (that’s Tom Barnett (2011), not Tom Barnett (2006-08)), weighing in on Google’s purported bias.  One serious problem with the search neutrality/search bias discussions to date has been the dearth of empirical evidence concerning so-called search bias and its likely impact upon consumers.  Hoping to remedy this, I posted a study this morning at the ICLE website both critiquing one of the few, existing pieces of empirical work on the topic (by Ben Edelman, Harvard economist) as well as offering up my own, more expansive empirical analysis.  Chris Sherman at Search Engine Land has a great post covering the study.  The title of his article pretty much says it all:  “Bing More Biased Than Google; Google Not Behaving Anti-competitively.”

One clarification is in order.  The  Search Engine Land piece quotes Geoff responding to the author indicating that the research was undertaken independently.  It’s not clear from the way Sherman presents it in the article, but Geoff did acknowledge (as does the disclosure in the paper itself) that Google supports ICLE.  However, as Sherman notes, although the work was indirectly supported by Google, Google had no hand in the conception or execution of the project.  For that I’m the only one to blame, unfortunately.

Following is a summary of the study from the ICLE website.  I plan to blog about the results and their implications in the coming days in a series of posts.

Google has been the subject of persistent claims that its organic search results are improperly “biased” toward its own content.  Among the most influential is an empirical study released earlier this year by Benjamin Edelman and Benjamin Lockwood, claiming that Google favors its own content “significantly more than others.”  The authors conclude in their study that Google’s search results are problematic and deserving of antitrust scrutiny because of competitive harm.

A new report released by the International Center for Law & Economics and authored by Joshua Wright, Professor of Law and Economics at George Mason University, critiques, replicates, and extends the study, finding Edelman & Lockwood’s claim of Google’s unique bias inaccurate and misleading. Although frequently cited for it, the Edelman & Lockwod study fails to support any claim of consumer harm — or call for antitrust action — arising from Google’s practices.

Prof. Wright’s analysis finds own-content bias is actually an infrequent phenomenon, and Google references its own content more favorably than other search engines far less frequently than does Bing:

  • In the replication of Edelman & Lockwood, Google refers to its own content in its first page of results when its rivals do not for only 7.9% of the queries, whereas Bing does so nearly twice as often (13.2%).
  • Again using Edelman & Lockwood’s own data, neither Bing nor Google demonstrates much bias when considering Microsoft or Google content, respectively, referred to on the first page of search results.
  • In our more robust analysis of a large, random sample of search queries we find that Bing generally favors Microsoft content more frequently—and far more prominently—than Google favors its own content.
  • Google references own content in its first results position when no other engine does in just 6.7% of queries; Bing does so over twice as often (14.3%).

The results suggest that this so-called bias is an efficient business practice, as economists have long understood, and consistent with competition rather than the foreclosure of competition. One necessary condition of the anticompetitive theories of own-content bias raised by Google’s rivals is that the bias must be sufficient in magnitude to exclude rival search engines from achieving efficient scale. A corollary of this condition is that the bias must actually be directed toward Google´s rivals. That Google displays less own-content bias than its closest rival, and that such bias is nonetheless relatively infrequent, demonstrates that this condition is not met, suggesting that intervention aimed at “debiasing” would likely harm, rather than help, consumers.

SHARE: 2 comments

Joshua Wright / Joshua Wright is a Professor of Law at George Mason University School of Law and holds a courtesy appointment in the Department of Economics. Professor Wright was recently appointed as the inaugural Scholar in Residence at the Federal Trade Commission Bureau of Competition, where he served until Fall 2008. Professor Wright was a Visiting Professor at the University of Texas School of Law and was a Visiting Fellow at the Searle Center at the Northwestern University School of Law during the 2008-09 academic year. Professor Wright also regularly lectures on economics, empirical methods, and antitrust economics to state and federal judges through the George Mason University Law and Economics Center Judicial Education Program. Professor Wright received both a J.D. and a Ph.D. in economics from UCLA, where he was managing editor of the UCLA Law Review, and a B.A. in economics with highest departmental honors at the University of California, San Diego. Before coming to George Mason University School of Law, Professor Wright clerked for the Honorable James V. Selna of the Central District of California and taught at the Pepperdine University Graduate School of Public Policy. Professor Wright's areas of expertise include antitrust law and economics, consumer protection, empirical law and economics, intellectual property and the law and economics of contracts. His publications have appeared in leading academic journals, including the Journal of Law and Economics, Antitrust Law Journal, Competition Policy International, Northwestern Law Review, Supreme Court Economic Review, Yale Journal on Regulation, Journal of Competition Law and Economics, Review of Industrial Organization, Review of Law and Economics, and the UCLA Law Review. Professor Wright is also the co-editor of Pioneers of Law and Economics (Elgar Publishing) and Competition Policy and Patent Law under Uncertainty: Regulating Innovation (Cambridge Press). Professor Wright has also testified at the joint Department of Justice/ Federal Trade Commission Hearings on Section 2 of the Sherman Act, the Federal Trade Commission’s FTC at 100 Conference, and the DOJ/ FTC Hearings on the 2010 Horizontal Merger Guidelines.

Previous post:

Next post: