Cory Doctorow has called for a Wikipedia-style effort to build an open source, non-profit search engine. From his column in The Guardian:
What’s more, the way that search engines determine the ranking and relevance of any given website has become more critical than the editorial berth at the New York Times combined with the chief spots at the major TV networks. Good search engine placement is make-or-break advertising. It’s ideological mindshare. It’s relevance…
It’s a terrible idea to vest this much power with one company, even one as fun, user-centered and technologically excellent as Google. It’s too much power for a handful of companies to wield.
The question of what we can and can’t see when we go hunting for answers demands a transparent, participatory solution. There’s no dictator benevolent enough to entrust with the power to determine our political, commercial, social and ideological agenda. This is one for The People.
Put that way, it’s obvious: if search engines set the public agenda, they should be public.
He goes on to claim that “Google’s algorithms are editorial decisions.” For Doctorow, this is an outrage: “so much editorial power is better vested in big, transparent, public entities than a few giant private concerns.”
I wish Doctorow well in his effort to crowdsource a Google-killer, but I’m more than a little skeptical that anyone would actually want to use his search engine of The People. My guess is that, like most things produced in the name of “The People” (Soviet toilet paper comes to mind), it will probably won’t be much fun to use, and will likely chafe noticeably. (For the record, I love and regularly use Wikipedia; I just don’t think that model is unlikely to produce a particularly useful search engine. As Doctorow himself has noted of Google, “they make incredibly awesome search tools.”)
But I’m glad to see that Doctorow has conceded an important point of constitutional law: The First Amendment protects the editorial discretion of search engines, like all private companies, to decide what to content to communicate. For a newspaper, that means deciding which articles or editorials to run. For a library or bookstore, it means which books to carry. For search engines, it means how to write their search algorithims.
Doctorow’s “We’ll build our own darn rocket ship in the backyard!” response to his deep concerns about Google’s dominance of search does not, of course, impinge on Google’s editorial discretion—and for that, I commend him. But others, most notably Frank Pasquale, have indeed proposed government action to address such concerns in ways that most surely would impinge on the First Amendment rights of all search engines.
Pasquale’s comlpaint about Google is essentially the same as Doctorow’s, but rather than proposing an innovative (if unrealistic) alternative (like Doctorow), he has called (PDF) for the “creation of a Federal Search Commission to parallel the Federal Communications Commission” and declared that ” In order to reduce opportunities for clickfraud and unfair treatment of indexed entities, qualified transparency will be needed in order to open up the ‘black box’ of search engine operations to at least some third parties.” He focuses on search algorithms because:
The heart of a search engine and the key to its success is its search algorithm. Effective algorithms are protected by a veil of secrecy and by various intellectual property rights. As a result, new entrants cannot easily appropriate existing algorithms. Moreover, many algorithms are trade secrets. Unlike patents, which the patent holder must disclose and which eventually expire, these trade secrets may never enter the public domain. Search algorithms may be analogous to the high-cost infrastructure required for entry into the utility or railroad markets.
He diagnoses the problem as follows:
given the emphasis on secrecy in the search engine business model, no one can verify that such rankings have not been manipulated or that subtler biases in favor of search engines’ partners are not being worked into the search algorithm…
If search engines are to be accountable at all, if their interest is to be balanced against those of the various other claimants involved in search-related disputes, and if social values are to be given any weight, some governmental
agent should be able to peer into the black box of search and determine whether or not illegitimate manipulation has occurred.
But what about editorial discretion? Why should Google be forced to change its PageRank algorithms any more than The New York Times should be forced to change how it decides which stories to run? Moreover, why should Google be forced to disclose how this process works? Assigning a government monitor to sit in on meetings of the Times‘ editorial board “to detect bias” would clearly impinge on their editorial discretion. Similarly, I don’t see why forcing a Yahoo!, Microsoft or any other search engine to disclose their equivalent processes for ranking search results should pass constitutional muster.
Editorial discretion means getting to make your own decisions, even if they might seem biased to those wise elites who “know better” because, well, it’s your decision and not the government’s! Saying that speakers can make whatever decisions they want as long as they’re not biased means speakers don’t really have editorial discretion after all.
So, if recognizing that search algorithms are a form of editorial discretion is a problem (as Doctorow implies), it’s only insofar as this might frustrate the desires of those who would regulate search.
The Technology Liberation Front is the tech policy blog dedicated to keeping politicians' hands off the 'net and everything else related to technology.