Is Facebook Now Over-moderating Content?

Reading professor Siva Vaidhyanathan’s recent op-ed in the New York Times, one could reasonably assume that Facebook is now seriously tackling the enormous problem of dangerous information. In detailing his takeaways from a recent hearing with Facebook’s COO Sheryl Sandberg and Twitter CEO Jack Dorsey, Vaidhyanathan explained,

Ms. Sandberg wants us to see this as success. A number so large must mean Facebook is doing something right. Facebook’s machines are determining patterns of origin and content among these pages and quickly quashing them.

Still, we judge exterminators not by the number of roaches they kill, but by the number that survive. If 3 percent of 2.2 billion active users are fake at any time, that’s still 66 million sources of potentially false or dangerous information.

One thing is clear about this arms race: It is an absurd battle of machine against machine. One set of machines create the fake accounts. Another deletes them. This happens millions of times every month. No group of human beings has the time to create millions, let alone billions, of accounts on Facebook by hand. People have been running computer scripts to automate the registration process. That means Facebook’s machines detect the fakes rather easily. (Facebook says that fewer than 1.5 percent of the fakes were identified by users.)

But it could be that, in their zeal to trapple down criticism from all sides, Facebook instead has corrected too far and is now over-moderating. The fundamental problem is that it is nearly impossible to know the true amount of disinformation on a platform. For one, there is little agreement on what kind of content needs to be policed. It is doubtful everyone would agree what constitutes fake news and separates it from disinformation or propaganda and how all of that differs from hate speech. But more fundamentally, even if everyone agreed to what should be taken down, it is still not clear that algorithmic filtering methods would be able to perfectly approximate that.

Detecting content that violates a hate speech code or a disinformation standard leads into a massive operationalization problem. A company like Facebook isn’t going to be perfect. It could produce a detection regime that was either underbroad or overbroad. It is of course only minimal evidence, but I have been seeing a lot of my friends on Facebook post about how their own posts have been taken down and it was clear they were non-political.

Over-moderation could explain why many conservatives have been worried about Twitter and Facebook engaging in soft censorship. Paula Bolyard made a convincing case in the Washington Post,

There have been plenty of credible reports over the past two years claiming anti-conservative bias at the Big Three Internet platforms, including the 2016 revelation that Facebook had routinely suppressed conservative outlets in the network’s “trending” news section. Further, when Alphabet-owned YouTube pulls down and demonetizes mainstream conservative content from sites such as PragerU, it certainly gives the impression that the company has its thumb on the scale.

Bolyard hints at one of the biggest problems in the conversation today. Users cannot peer behind the veil and are thus forced to impute intentions about how the network operates in practice. Here is how Sarah Myers West, a postdoc researcher at the AI Now Institute, described the process,    

Many social network users develop “folk theories” about how platforms work: in the absence of authoritative explanations, they strive to make sense of content moderation processes by drawing connections between related phenomena, developing non-authoritative conceptions of why and how their content was removed

West goes on to cite a study of moderation efforts, which found that users thought Facebook was “powerful, perceptive, and ultimately unknowable.” Both Vaidhyanathan and Bolyard could pushing similar folk theories. They are both astute in their comments and offer a lot to consider, but everyone in this discussion, including the operators at Facebook and Twitter, is hobbled by a fundamental knowledge problem.

Still, each platform has to create its own means of detecting this content, which will need to conform to the specifics of the platform. Evelyn Douek’s report on the Senate Hearing, which you should absolutely go read, helps to fill out some of the details on this point,

[Twitter CEO Jack] Dorsey stated that Twitter does not focus on whether political content originates abroad in determining how to treat it. Because Twitter, unlike Facebook, has no “real name” policy, Twitter cannot prioritize authenticity. Dorsey instead described Twitter as focusing on the use of artificial intelligence and machine learning to detect “behavioural patterns” that suggest coordination between accounts or gaming the system. In a sense, this is also a proxy for a lack of authenticity, but on a systematic rather than an individual scale. Twitter’s focus, according to Dorsey, is on how people game the system in the “shared spaces [on Twitter] where anyone can interject themselves,” rather than the characteristics of profiles that users choose to follow.

Dorsey seems to set up a comparison between the two companies. Facebook’s method of detecting nefarious content deals with the profile, as an authenticated person, in relation to the content that is shared. Twitter, on the other hand, is looking for people to game the system in the “shared spaces [on Twitter] where anyone can interject themselves.” It might be a misread, but Dorsey suggests that Twitter is emphasizing the actions of users, which would lead to a more structural approach.

It goes without saying that Facebook’s social network is different from Twitter’s, leading to different approaches in moderation. Facebook creates dyadic connections. The relationships on Facebook run both ways. Becoming friends means we are in a mutual relationship. Twitter, however, allows for people to follow others without reciprocity. The result are distinct network structures. Pew, for example, was able to distinguish between six different broad structures, including polarized crowds, tight crowds, brand clusters, community clusters, broadcast networks, and support networks. Combined, these features make it difficult for both researchers and operators to understand the scope of the problem and how solutions are working, or not working.

So what are the broad incentives pushing platforms to either over-moderate or under-moderate content? Here is what I could come up with:  

• If content moderation is too broad, it will spark the ire of content creators who might get inadvertently caught up in a filter.
• More content going over the network means more users and more engagement, and thus more advertising dollars, making the platform sensitive to over-moderation.
• Content has both an extensive marginal and an intensive marginal. Facebook will want to expand the overall amount of content to attract people, but they will want to keep the content on the network high quality. Low quality will drive people and advertisers to exit, so they might have an incentive to over moderate.
• Given the current political environment and the California privacy bill, it might make better long term sense to over-moderate or at least engage in the perception of over-moderation to reduce the chance of legal or regulatory pressures in the future.
• The technical filtering solutions could have ambiguous effects on moderation. It could be that a platform simply is not that good at content moderation and has been under providing it.
• Or, the filtering system could be providing an expansive program that has swept up too many people and too much content.
• Given that people think these platforms are “powerful, perceptive, and ultimately unknowable,” the platforms might err on the side of under-moderation simply to reduce the overall experience of content moderation.

Content moderation at scale is difficult. And messy. In creating a technical regime to deal with this problem, we shouldn’t expect platforms to get it perfect. While many have criticized platforms for under-moderation, they might now being over-moderating. Still, there is a massive knowledge problem in trying to understand if the current level of moderation is optimal.    

SHARE: 0 comments