First Amendment Protection of Search Algorithms as Editorial Discretion

by on June 4, 2009 · 34 comments

Cory Doctorow has called for a Wikipedia-style effort to build an open source, non-profit search engine. From his column in The Guardian:

What’s more, the way that search engines determine the ranking and relevance of any given website has become more critical than the editorial berth at the New York Times combined with the chief spots at the major TV networks. Good search engine placement is make-or-break advertising. It’s ideological mindshare. It’s relevance…

It’s a terrible idea to vest this much power with one company, even one as fun, user-centered and technologically excellent as Google. It’s too much power for a handful of companies to wield.

The question of what we can and can’t see when we go hunting for answers demands a transparent, participatory solution. There’s no dictator benevolent enough to entrust with the power to determine our political, commercial, social and ideological agenda. This is one for The People.

Put that way, it’s obvious: if search engines set the public agenda, they should be public.

He goes on to claim that “Google’s algorithms are editorial decisions.”   For Doctorow, this is an outrage: “so much editorial power is better vested in big, transparent, public entities than a few giant private concerns.”

I wish Doctorow well in his effort to crowdsource a Google-killer, but I’m more than a little skeptical that anyone would actually want to use his search engine of The People.  My guess is that, like most things produced in the name of “The People” (Soviet toilet paper comes to mind), it will probably won’t be much fun to use, and will likely chafe noticeably. (For the record, I love and regularly use Wikipedia; I just don’t think that model is unlikely to produce a particularly useful search engine.  As Doctorow himself has noted of Google, “they make incredibly awesome search tools.”)

But I’m glad to see that Doctorow has conceded an important point of constitutional law: The First Amendment protects the editorial discretion of search engines, like all private companies, to decide what to content to communicate.  For a newspaper, that means deciding which articles or editorials to run.  For a library or bookstore, it means which books to carry.  For search engines, it means how to write their search algorithims.

Doctorow’s “We’ll build our own darn rocket ship in the backyard!” response  to his deep concerns about Google’s dominance of search does not, of course, impinge on Google’s editorial discretion—and for that, I commend him.  But others, most notably Frank Pasquale, have indeed proposed government action to address such concerns in ways that most surely would impinge on the First Amendment rights of all search engines.

Pasquale’s comlpaint about Google is essentially the same as Doctorow’s, but rather than proposing an innovative (if unrealistic) alternative (like Doctorow), he  has called (PDF) for the “creation of a Federal Search Commission to parallel the Federal Communications Commission” and declared that ” In order to reduce opportunities for clickfraud and unfair treatment of indexed entities, qualified transparency will be needed in order to open up the ‘black box’ of search engine operations to at least some third parties.”   He focuses on search algorithms because:

The heart of a search engine and the key to its success is its search algorithm. Effective algorithms are protected by a veil of secrecy and by various intellectual property rights. As a result, new entrants cannot easily appropriate existing algorithms. Moreover, many algorithms are trade secrets. Unlike patents, which the patent holder must disclose and which eventually expire, these trade secrets may never enter the public domain. Search algorithms may be analogous to the high-cost infrastructure required for entry into the utility or railroad markets.

He diagnoses the problem as follows:

given the emphasis on secrecy in the search engine business model, no one can verify that such rankings have not been manipulated or that subtler biases in favor of search engines’ partners are not being worked into the search algorithm…

If search engines are to be accountable at all, if their interest is to be balanced against those of the various other claimants involved in search-related disputes, and if social values are to be given any weight, some governmental
agent should be able to peer into the black box of search and determine whether or not illegitimate manipulation has occurred.

But what about editorial discretion?  Why should Google be forced to change its PageRank algorithms any more than The New York Times should be forced to change how it decides which stories to run?  Moreover, why should Google be forced to disclose how this process works?  Assigning a government monitor to sit in on meetings of the Times‘ editorial board “to detect bias” would clearly impinge on their editorial discretion.  Similarly, I don’t see why forcing a Yahoo!, Microsoft or any other search engine to disclose their equivalent processes for ranking search results should pass constitutional muster.

Editorial discretion means getting to make your own decisions, even if they might seem biased to those wise elites who “know better” because, well, it’s your decision and not the government’s!  Saying that speakers can make whatever decisions they want as long as they’re not biased means speakers don’t really have editorial discretion after all.

So, if recognizing that search algorithms are a form of editorial discretion is a problem (as Doctorow implies), it’s only insofar as this might frustrate the desires of those who would regulate search.

  • http://blurringborders.com Kevin D

    “Why should Google be forced to change its PageRank algorithms any more than The New York Times should be forced to change how it decides which stories to run?”

    Though I wouldn't want to put words in his mouth, I imagine it's because Pasquale believes Google to wield monopoly power. Whether or not you agree with that, I think it's a fair progression from monopoly -> government oversight.

  • http://www.pff.org/ Sid Rosenzweig

    I've heard Frank Pasquale present one of his papers on campus, and he does presume that Google wields market power. As an intellectual property attorney myself, I am respectful of the attempts of firms to protect their trade secrets. One critical problem with Frank's argument that the search engines need to be transparent is what happens to website developers. He argues basically that secrecy enables the search engine firms to game the rankings. Were everything forced to be disclosed to the public, we end up with every website trying to game the system from the other direction. While I realize that there's some of that already, it would certainly get a lot worse if people knew exactly what they needed to do — a quick race to the bottom. If the only disclosure is to the government, I'm not sure that I'd trust a government audit. I note, for instance, the skepticism that many feel toward government auditing of electronic voting machines. The voting machines are far less technically complicated than search-engine algorithms. Plus, unlike voting machine firmware, the algorithms are constantly changing, in a way that defies any easy audit.

  • Sneeje

    France in the past few years began an effort to create a government-backed Google-killer. How's that going?

    Also, the assertion that “so much editorial power is better vested in big, transparent, public entities than a few giant private concerns” is unfounded at best. Public and transparent are not a given coupling.

  • http://www.concurringopinions.com/archives/2007/02/are_big_search.html Frank Pasquale

    Hi Sid,

    I think the key here is to disaggregate the many services that Google offers. I don't presume they wield market power in all of them. But I do think that when you look at the numbers in general-purpose search, and search-related advertising, they are quite powerful, having about 70% of market share. Ben Edelman has identified some problems this causes:

    http://www.benedelman.org/news/062708-1.html
    http://www.benedelman.org/publications/ppc-comp

    But the arguments I make don't necessarily depend on Google having monopoly power or even a dominant position. They're essentially consumer protection arguments, similar to the ones made by proponents of net neutrality. I've adumbrated those points in an article entitled “Internet Nondiscrimination Principles,” and I make them more forcefully in a piece called “Beyond Innovation and Competition” (now in draft form, but I'm happy to share it). I think that they need to separate paid and editorial content, and to allow *someone* to verify whether that separation has occurred. Maybe that entity will be governmental, maybe a trusted NGO.

    You're right about the trade secrecy issue being the hardest one, as I noted around p. 34 or 35 of “Internet Nondiscrimination Principles.” But here again disaggregation is key. Are we really concerned that people will game book search to the same extent we're concerned about gaming web search in general? If not, maybe we should get more transparency in how books are arranged.

    (To take just one example–should Google be able to secretly import its rules about the ads it takes into the realm of book search? Note that they've denied ads like “China is Evil” because of ad rules against certain kinds of controversial content. See Verne Kopytoff, “Google’s Ad Rule Complex, Controversial,” The San Francisco Chronicle, August 9, 2004.).

  • http://www.concurringopinions.com/archives/2007/02/are_big_search.html Frank Pasquale

    Thanks for covering this important issue. I differ on one key analogy in your analysis:

    “Assigning a government monitor to sit in on meetings of the Times‘ editorial board 'to detect bias' would clearly impinge on their editorial discretion.”

    That's true–but is Google really like a newspaper? Sometimes–for example, in “Google News,” maybe there is some point of view expressed. But generally, I agree with Robert Post that “First Amendment protections extend to forms of social interactions. The question of First Amendment coverage thus does not turn on the intrinsic nature of source code itself, but instead on the social contexts in which source code is used.” Context is key.

    If people are reading Google News like a newspaper, perhaps we're in “Miami Herald v. Tornillo” world where editorial discretion is near absolute. But if they're using it as a background checker, navigational aid, map of the web, etc., other forms of analysis apply. I've made the case in this paper:
    http://papers.ssrn.com/sol3/papers.cfm?abstract
    and in the URL link on my name, that in some of these contexts Google is more like a credit bureau, or a carrier, or a cartographer. Each of these entities receives less 1A protection than a newspaper.

    Lastly, I think that people deserve rights, not algorithms. The algorithm for general search, or for particular types of searches, is a tool of an existing corporate entity, Google. Should the fact that Google’s results are automated lead them to get less protection than, say, a social search engine that ordered the web?

    I think so, because one might claim that Google is much closer to a data provider than, say, a newspaper. The latter actually expresses a point of view on what the news is; the former merely aggregates information. This difference has consequences for law.

    Data providers like consumer reporting companies can be held more accountable for what they say than a newspaper. If I have a dispute with a newspaper over whether they’ve portrayed me accurately, I’m probably going to have to sue for defamation in order to settle things. But according to an FTC website, “If an investigation doesn’t resolve your dispute with the consumer reporting company, you can ask that a statement of the dispute be included in your file and in future reports. You also can ask the consumer reporting company to provide your statement to anyone who received a copy of your report in the recent past.”

    Moreover, ”only authorized individuals such as potential lenders, employers, insurance underwriters or landlords may access your report, and only if they intend to do business with you.” Finally, in case of disputes, “you’re entitled to add a written statement (100 words or less) explaining your view of the mistake.”

    Why might we want to extend this type of distinction to the search world (and, indeed, strengthen consumer protections vis a vis “black box” data aggregators like ratings agencies and FICO scorers)? I think that there is something deeply troubling about unaccountable power–about a system that can simply spit out some life-changing result without giving a full explanation for it. Suspicion about FICO scores has led some states to prohibit their use in insurance rating, just as Finland has prevented employers from using Google results in evaluating potential applicants. Full First Amendment protection should be reserved for accountable, attributable speech–not the black box data processing systems that are increasingly powerful arbiters of taste, authority, and creditworthiness.

  • http://enigmafoundry.wordpress.com eee_eff

    I wish Doctorow well in his effort to crowdsource a Google-killer, but I’m more than a little skeptical that anyone would actually want to use his search engine of The People. My guess is that, like most things produced in the name of “The People” (Soviet toilet paper comes to mind), it will probably won’t be much fun to use, and will likely chafe noticeably.

    We the People of the United States, in Order to form a more perfect Union, establish Justice, insure domestic Tranquility, provide for the common defence, promote the general Welfare, and secure the Blessings of Liberty to ourselves and our Posterity, do ordain and establish this Constitution for the United States of America….

    hmmmm…. so perhaps so of those things done by or for the people really aren't so bad after all….

  • http://www.techliberation.com Adam Thierer

    You state: “Full First Amendment protection should be reserved for accountable, attributable speech–not the black box data processing systems that are increasingly powerful arbiters of taste, authority, and creditworthiness.”

    I must say, this is quite a novel theory of First Amendment. Last time I checked, our First Amendment freedoms did not depend upon a determination by government officials of what counts as “accountable, attributable speech.” Is anonymous speech automatically less worthy of protection? And you speak of “data processing systems” as if their is no human element behind it/them which have speech rights. But every time we pull out a notepad in public and jot down facts about the world around us, and later share that information online with others, we are engaged in data processing. Where do you draw your lines for what forms of information collection are worthy of protection?

    Moreover, from a strict First Amendment perspective, it is irrelevant that some systems are “arbiters of taste, authority, and creditworthiness.” Professor, you bear the burden of showing why it is that the First Amendment should NOT apply in all these situations since the net result of your approach is government instead acting as the arbiter itself, and that something that IS wholly offensive to America's rich First Amendment tradition.

    In sum, you've turned the First Amendment upside down and made it an affirmative grant of restrictive government power when it is nothing of the sort. This is the distorted view of the “media access” movement that gave us RED LION (which got it wrong), TORNILLO (which got it right), and several decades worth of muddied, misguided First Amendment thinking. “Congress shall make no law..” should have been clear enough, but apparently it wasn't.

    More on this point here: http://techliberation.com/2005/05/10/your-soapb

  • http://techliberation.com/author/berinszoka/ Berin Szoka

    If only that were what “The People” meant in, say, the Democratic People's Republic of (North) Korea!

  • Sneeje

    Ummm…. I think he was talking about things produced FOR COMMERCE in the name of “The People”. None of your examples would apply.

  • Sneeje

    Ummm…. I think he was talking about things produced FOR COMMERCE in the name of “The People”. None of your examples would apply.

  • Sneeje

    Ummm…. I think he was talking about things produced FOR COMMERCE in the name of “The People”. None of your examples would apply.

  • Pingback: P2P Foundation » Blog Archive » Toward a Public Alternative in Digital Archiving and Search

  • Pingback: “Google Bigotry,” Corporate-Bashing & Human Envy — Technology Liberation Front

  • Pingback: The Day Real Internet Freedom Died: Our Forbes Op-Ed on Net Neutrality Regulation — Technology Liberation Front

  • Pingback: The Progress & Freedom Foundation Blog

  • Pingback: Net Neutrality Regulation => Online Product/Service Definitions => Online Taxation — Technology Liberation Front

  • Pingback: The Progress & Freedom Foundation Blog

  • Pingback: The Progress & Freedom Foundation Blog

  • Pingback: The 10 Most Important Info-Tech Policy Books of 2009 — Technology Liberation Front

  • Pingback: Is Google Evil? Part II: Search Neutrality « Blog Entry « Dr. Melissa Clouthier

  • Pingback: We Need “New York Times Neutrality”—NOT!

  • Pingback: NRO Op/Ed: Government v. Google: Why Free Marketeers Should Rally Against Search Neutrality

  • Pingback: Just say no to Ma Bell-era Net neutrality regulation « Internet Freedom Coalition

  • Pingback: Frisco luxury custom walk in closets

  • Pingback: Psychic Readings

  • Pingback: order adipex

  • Pingback: New Jersery Lottery

  • Pingback: free horse racing picks

  • Pingback: seo nom noms bonus

  • Pingback: Unlock Pension

  • Pingback: Day Trading

  • Pingback: premier league philippines

  • Pingback: http://www.youtube.com/watch?v=TIIzKnjOUZ4 ideo

  • Pingback: https://www.nhcps.com/

Previous post:

Next post: