Berin Szoka

“Cry [Censorship] and Let Slip the Dogs of [Regulation]!” - A Lesson in the Dangers of Googlephobia

TLF readers may have heard that Google was craftily censoring my free-market colleagues at the Progress & Freedom Foundation.  Our good friend and invaluable TLF commenter Richard Bennett blogged over  the weekend about how Google seemed to block access to our site when he tried to search for “net neutrality.”

This is one of the most amazing things I’ve ever seen. Google is blocking net neutrality documents from the PFF’s web site, but documents in the same format that deal with other subjects are not flagged “dangerous.”

This is really outrageous, and a clear example of the problem with a monopoly gatekeeper.

This story made the rounds this morning and much of the DC Internet policy community was atwitter with allegations of censorship by Google.  But as I explain in the comment I tried (unsuccessfully) to post on Richard’s blog, this is all an innocent and unfortunate misunderstanding:

Everyone at PFF appreciates your concern, Richard, but what actually happened is quite benign; Google was not certainly censoring anyone!

Here’s what happened…  Unlike the rest of our site, our “Issues & Publications” system relies on an SQL database—which, like any SQL database, is vulnerable to certain kinds of attack.  We recently noticed such an attack and took steps to solve the problem.  This is standard operating procedure for anyone running a site with an SQL database.

The Google search engine relies on the “Badware Website Clearinghouse” kept by StopBadWare.org.  The StopBadware project is a very helpful “neighborhood watch” campaign led by the good folks associated with the Berkman Center at Harvard.  StopBadware works with relies on Google to automatically identify sites that might contain badware, as their FAQ explains: http://www.stopbadware.org/home/faq#partnerwarnings-warning

Once a site is flagged, a warning message will arise when someone attempts to visit the site from the Google search engine or if one is using Google desktop or certain other firewall tools that aim to protect users from visiting dangerous sites.

The reason you encountered that warning page is that our site was quite accurately flagged as potentially dangerous and we had not yet completed the procedure for having our site removed from the Badware Website Clearinghouse, which is explained here:  http://www.stopbadware.org/home/faq#partnerwarnings-remove

We consider the StopBadware a valuable self-help tool for protecting Internet users from potentially harmful software and applaud Google for its leadership in this area.  If this incident demonstrates anything, it’s that an educational campaign would help users understand how the process works, why it’s good for all Internet users and that it is NOT censorship.

This incident demonstrates two more things:  Far too many tech policy observers (including some in the sometimes-factious free market camp) have a hair-trigger reaction to anything involving either (1) Google and (2) censorship.  The combination of the two is downright explosive.

In this particular case, Richard’s concern is understandable, if misplaced:  A number of other people wondered whether Google was somehow blocking access to our site when they tried to access articles on our site over the weekend.

But in general, this should be a lesson to us all:  We should all be a little more careful before making allegations of censorship, monopoly or other abuses.  This kind of tempest-in-a-teapot is exactly what drives bad public policy:  Even if the misunderstanding is corrected quickly, many will remember only the unfounded allegations of abuse, and the political climate will be shifted (if inadvertently) toward regulation based on purely imaged harm.

Even the most tech-savvy among us should be sure to investigate the technical aspects of what we see online before leaping to conclusions–especially given the pace of innovation on the Internet.

July 14, 2008 | Comments |

Viewing 18 Comments

    • ^
    • v
    "Even the most tech-savvy among us should be sure to investigate the technical aspects of what we see online before leaping to conclusions ..."

    Yes, absolutely. The problem is that there's no incentive, no reward, for doing that investigation, rather than crying wolf :-(.

    It's a market failure.
    • ^
    • v
    As you say, this is probably an innocent and unfortunate misunderstanding. Unfortunately we live in a world were we really never know if an excuse was really legitimate or not. Which brings to mind Comcast's defense that it was pursing legitimate "traffic shaping" and not screwing its customers.

    The irony here is that if members of the Progress & Freedom Foundation are complaining about Google's behavior then I detect a double standard. The Progress & Freedom Foundation group continuously asserts that corporations can do whatever they want (Comcast for example) and if you don't like it tough. Now that the finger of blame has been pointed towards Google (a corporation) figuratively undertaking censorship (even it it is misconstrued) we get members of the Progress and Freedom Foundation crying foul!?!?!?!?!?!?!?
    • ^
    • v
    So it's all a technical snafu. As The Church Lady would say, "Well isn't that conveeeeenient???"
    • ^
    • v
    PFF didn't complain about Google censoring them, Steve, I did and it was my mistake and only my mistake. I got a e-mail from a reader alerting me to the situation and I verified that a Google search for "net neutrality" at the PFF web site returned pages flagged as harmful.

    I should have done some research on the system used to identify bad pages, but I was in the middle of another project so I didn't take the time.

    Google is too smart to do something malicious to PFF on purpose, but I was concerned that they might have a system in place that could be gamed by the wily advocate. When I learned the truth, I deleted my post and issued a correction.

    I'm actually surprised that anybody's reading my blog, since my visitor numbers aren't that high, but apparently a few reporters actually do. Wonder of wonders.
    • ^
    • v
    Richard, thanks for the response. Well I guess my PFF haranguing just vaporized. :)
    • ^
    • v
    Steve R.... where have you seen anyone from PFF "complaining about Google's behavior" or "crying foul" in this or any other matter? Please, cite one instance for me. Didn't you even bother reading Berin's post above? He works here at PFF with me and he has PREEMPTIVELY defused this situation by making it clear to others that there is no news here and that Google has done nothing wrong.

    Moreover, I'd like to make it clear that I would defend Google's right to do whatever they want with their system, even though I know they would never take steps to intentionally block alternative viewpoints on public policy issues. But, if they did, I have no problem with that because (a) they would get murdered with bad PR over any such effort and (b) that would incentivize web surfers to just go to another search provider. Moreover, any attempt to block PFF's site or research for any reason would probably be the best thing that ever happened to us because it would make a lot more people aware of us!!

    Some of us aren't scared of free markets, Steve.

    Regardless, I reiterate your challenge to show me a single instance of anyone at PFF complaining about Google's business practices. We have absolutely no problem with Google structuring their business affairs however they see fit. Our only disagreements with them in the past have been over certain policy recommendations where they favor a somewhat more expansive role for government intervention than we have been comfortable with. And those have always been friendly policy disagreements. I have enormous respect for Google and everyone at that company despite our occasional differences on certain policy issues.
    • ^
    • v
    Thank you all for your comments. Let me respond in turn.

    Seth, you've put your finger on the fundamental problem of Internet journalism--or the relative lack thereof. The Internet in general, and blogging in particular, makes it far too easy for even very thoughtful and sophisticated observers (like Richard) to jump to conclusions quickly--and then share those conclusions with the entire world, very few of whom will independently question our conclusions before reposting or reacting to them.

    This is most true where the subject matter is the Internet itself--a subject on which there are plenty of opinion pieces and far too few actual investigative journalism pieces. The same goes for the world of Internet policy, where we all (myself included) must be careful to make sure we really understand the technical details before weighing in on an issue.

    Steve, as noted by Richard, we weren't the ones complaining about Google. The point of my post was to correct the misapprehension of Richard and others who have emailed us at PFF that Google was somehow blocking content they didn't like. As for your point about the lack of transparency as to what Google, ISPs and other so-called "gatekeepers" that might potentially "screw" customers, Google is providing a technical solution to that lack of transparency. For example, as I noted in a recent post, Google is developing a suite of tools that will allow users to monitor how their ISP conducts traffic management. Surely, we can all agree that more transparency is a good thing, but let's recognize how the market is already responding to that need.

    Eric, I appreciate your skepticism. Given how well this little incident suggests the need for a little more caution before anyone alleges some form of abuse by Internet companies, it must indeed seem like the perfect just-so lesson! I can assure you, however, that the SQL attack we experienced was quite real, as was Richard's misunderstanding. Truly, you give us far too much credit by implying that we might have been able to engineer this incident to prove a point--let alone thought to do so in the first place!

    At the risk of repeating myself, let me just add the following. It's difficult for everyone to do in practice, but whenever we read about (especially on a blog) some alleged violation by an evil corporation of the sacred principles of the Internet, we should all try to keep in mind the presumption of innocence that lies at the heart of our Anglo-Saxon legal tradition: the accuser, not the accused, bears the burden of proof. It is the consistent and widespread failure to apply this principle that helps to create the kind of vague and unsubstantiated collective impression of harm or danger where none (or little) actually exists that so often drives bad regulation, legislation and even litigation.
    • ^
    • v
    Adam, you are correct that I do not know of a single post where the PFF has complained as an organization about Google's behavior. I was reacting specifically to the post. Also Richard clarified the origins of his post and he stated that his post was independent of the PFF. Berin wrote "The Internet in general, and blogging in particular, makes it far too easy for even very thoughtful and sophisticated observers (like Richard) to jump to conclusions quickly.

    Also, I do not fear free markets. My concern with the current trend in our economic system is that we don't really have "free" markets and many of those who profess free market principles attempt to "game" the system by hiring lobbyists to "buy" favorable legislation. If a business cannot compete on a level playing field, too bad.

    As further indicator of being free market orientated, I believe that the government has a role in a free market economy. Competition is competition. If you can't compete, too bad.
    • ^
    • v
    Hi Berin,

    Thanks for posting this great clarification of what happened! Please let me know if there's anything we at StopBadware can do to help out.

    I just want to clarify who does the actual flagging of compromised websites. That is in fact Google, specifically their safer searching / anti-malware team, and not StopBadware.org or the Berkman Center. Google's warnings are entirely based on their own internal systems for identification of malware distribution; StopBadware comes in simply to help site owners who want to remove the warnings in learning about badware and getting the warnings removed.

    StopBadware sees our role in the process as an external fail-safe. Our public reviews process is an assurance that Google search users and webmasters alike can request an independent review of any flagged site from a nonprofit organization. We do our best to make the reviews process as transparent as possible. For example, anyone interested in the progress of the PFF review can follow its status here:
    http://www.stopbadware.org/reports/container?re...

    Anyone with questions or concerns about the process is invited to contact me directly, at egeorge AT cyber.law.harvard.edu.

    Thanks!
    Erica
    StopBadware.org staff
    • ^
    • v
    SQL injection is certainly a vulnerability of great concern in database-driven Web sites. However, in this case Google was blocking direct links to PDF documents. There was no opportunity for a reader to be infected by fetching the documents. This is clearly overzealous, even if in this case it was not deliberate censorship. After all, when you're off Google, you're essentially off the Net.
    • ^
    • v
    Thanks for your comment, Brett. The StopBadware clearinghouse blocked every URL beginning with www.pff.org/issues-pubs/ after Google reported a malware attack emanating from that page. (No other URL beginning with www.pff.org was blocked--which is why Richard initially thought there might have been some selective blocking of certain kinds of content.)

    Now, maybe there's a way to "build a better mousetrap," as they say--that is, a more precise way of blocking potentially dangerous sites. If you can think of one, I suggest you share your ideas with the folks a the StopBadware project. I'm sure they would be delighted to talk to you--and I suspect Google has every reason to want to improve the system! But, again, I don't see any even remotely nefarious intent here on Google's part.

    It's also worth noting here that Google wasn't blocking access to content--merely warning people who tried to access content through the Google search engine (or while running Google Desktop, as I do) that sites may contain malware.

    Just try to imagine explaining these technical details to a member of Congress and I think you'll agree that, however imperfect the Google/StopBadware system might be, it's a good deal more flexible and sophisticated than would be any system the government might build. I, for one, am glad that Gooogle took the lead on this, and that we didn't wind up with some system run by government bureaucrats who believe in the "pipes and tubes" model of the Internet.
    • ^
    • v
    Actually, Berin, StopBadware doesn't decide which sites and pages to flag, Google does. StopBadware manages the process of getting sites off the black-list.

    As Brett points out, the list was over-inclusive, but that's probably a matter of erring on the side of caution.

    I find myself correcting a lot of posts lately, and then correcting the corrections. That certainly happened in this case, but it's a very easy thing to do.

    The net neutrality drama of the last year has resulted in more corrected news stories and blog posts than anything I've ever seen, and that in itself is blog-worthy.

    I have agree with Seth that throwing red meat to hungry dogs is the essence of blogging at the A-List sites, but I try to give them good, healthy red meat that doesn't carry too many diseases.
    • ^
    • v
    Berin:

    Good points. It's worth noting, though, that while Google couldn't block access to those documents from the Net, it was blocking access to them via its search engine. Only a network-savvy user would have known that it was possible to manually cut the URL from Google's "warning" page and paste it into the browser's URL bar.

    It seems to me that because the documents themselves were not infected, Google's claims that the documents "may harm your computer" were indeed irresponsible. Their system could have (and should have!) noted that the documents contained nothing malicious, rather than "turning off" access to many documents that were perfectly fine. Otherwise, there would be massive collateral damage if one page near the "top" of a deep Web site -- perhaps one that hosted pages for thousands of people or businesses -- happened to have a link to a single piece of malware.

    As I've said many times, I believe that the only time government should intervene in network management or the operation of the Internet is when there is intentional anticompetitive behavior. If this had indeed turned out to be censorship, the public hue and cry would have been sufficient motivation for Google to change its ways. And I hope that this incident does prompt Google to block only pages or files which actually contain malware, and not perfectly fine, useful documents that happen to share part of a URL with a page that does.
    • ^
    • v
    There's no reasonable way that Google could guarantee that a given link ending in .pdf was truly "safe", especially if it has detected other elements of malware on the site. If the site itself has been compromised, it would be trivial to send "clean" documents and links back to Google's robots, while sending infected files to everyone else.
    • ^
    • v
    If someone was intent upon fooling a malware detection system, he or she would likely not be detected in the first place. Also, it's very easy to scan PDFs for irregularities which would trigger the very few known vulnerabilities that have surfaced in PDF readers to date. I did, and everything on the PFF's site came up clean. If I can do it, a company that's professing to tell you whether Web objects are safe surely can as well. And such a company could easily arrange to scan from addresses that were not easily identifiable as belonging to it.
    • ^
    • v
    So the fact remains that Google flagged PDF documents are "dangerous" when the only danger they posed was free-market ideas. Google probably did this in good faith, but they did make a mistake. Those of us who over-reacted to this have issued corrections and even apologized, but Google has yet to 'fess up to their mistake.

    Double standard, anyone?
    • ^
    • v
    I've updated my blog post on this matter to reflect Brett's findings. If we apply the same standard to Google that Google applies to ISPs, Google is guilty of evil conduct. And we need to be fair and even-handed, don't we?
    • ^
    • v
    Google needs to take responsibility for censoring these blogs by publicly apologizing and pledge to never censor content again...force them to change or else we will blog on non-Google platforms: https://www.thepoint.com/campaigns/stop-google-...

Trackbacks

blog comments powered by Disqus