Moderating on Facebook sounds like a nightmare. Casey Newton had an explosive story last year of panic attacks and high turnover. But the problem isn’t just for the moderators themselves, but in aggregate. As a philosophical problem.
Moderation sets the bounds on reality and acceptableness for billions of people. Across cultures and contexts. Across time. Will a private organisation ever be equipped for this? Could anything, really?
I was thinking about this as I finished up Steven Levy’s new book on Facebook. It’s a sweeping account of Facebook’s rise and recent bout with internal territorialism. But there were really interesting digressions on moderating content at scale.
Levy describes intense debates over what is and isn’t kosher. I use the word deliberately as the image of ever expanding notes and commentary strikes as a quasi religious one. A guide that is first quite narrow and absolute but must be argued over, reinterpreted and contextualised to new situations. Interpreted. At scale.
The rules can venture into confounding, Jesuitical flights of logic. Some things are fairly straightforward. There are attempts to define levels of offensiveness in subjects like exposure to human viscera. Some exposure is okay. Other varieties require an “interstitial,” a warning on the screen like the one before a television show that might show a glimpse of buttocks. Outright gore is banned. It takes a judgment call to fit a given bloodbath into the right box. “If you rewind to Facebook’s early, early days, I don’t think many people would have realized that we’d have entire teams debating the nuances of how we define what is nudity, or what exactly is graphic violence,” says Guy Rosen of the Integrity team… Facebook has created a vast number of non-public supplementary documents that drill on specific examples. These are the Talmudic commentaries shedding light on Facebook’s Torah, the official Community Standards. A New York Times reporter said he had collected 1,400 pages of these interpretations…
As an abstract problem, sure, Facebook needs to crackdown on abuse and misinformation etc. etc. But in practice they are dealing with oceans of grey.
And there is no average set of values. Give my grandma and me the same stack of posts and we’d whittle them down to different subsets.
This is why the word interpret is important. Not just of the rules but the posts themselves. And layered on top of all this is that it apparently takes place at such speed as to make the nuances moot.
Facebook expects moderators to make about 400 “jumps” a day, which means an “average handle time” of around 40 seconds for them to determine whether a questionable video or post must remain, be taken down, or in rare cases, escalated to a manager, who might send the most baffling decision to the policy-crats at Menlo Park. Facebook says that there is no set time limit on each decision, but reporting by journalists and several academics who have done deep dives on the process all indicate that pondering existential questions on each piece of content would put one’s low-paying moderation career at risk.
The scale and speed, maybe more than anything else, is what concerns me. At least for now it doesn’t seem like a technological solution is in the offing. The incentive of those tasked with interpreting marginal content is to be restrictive. Better over than underdo it. And recourse is opaque and minimal.
Even given good will among everyone involved, top to bottom, this isn’t a great recipe. This isn’t how we should be determining reality for billions of people.
We must remind ourselves again that history as usually written (peccavimus) is quite different from history as usually lived: the historian records the exceptional because it is interesting-because it is exceptional. If all those individuals who had no Boswell had found their numerically proportionate place in the pages of historians we should have a duller but juster view of the past and of man.
This is from The Lessons of History, a short book that is deeply problematic in some parts and refreshingly frank in others.
This is somewhat understandable given it is more than fifty years old. But the exhortations to not strip history from both historians and ourselves, and so the context within which it has been understood and transmitted, are timeless.
To begin with, do we really know what the past was, what actually happened, or is history ” a fable” not quite “agreed upon”? Our knowledge of any past event is always incomplete, probably inaccurate, beclouded by ambivalent evidence and biased historians, and perhaps distorted by our own patriotic or religious partisanship. “Most history is guessing, and the rest is prejudice”.
I’d argue the same is true for the present. Our view of the world is inevitably shaped by what we find noticeable, what others do, and the context within which this happens.
This could be dictated by the medium – stories related visually are inherently biased by the availability and power of the images. It could also be impacted by time, technology, ideology, culture and many other factors.
But the spectacular reigns supreme. No one sets out to tell a boring anecdote in a bar. The world, the story, reality, as in history, is probably far more mundane.
Have you ever tried to mock up the outcome from a series of coin flips? It turns out that we are horrible at simulating randomness.
But as a great article in Nautilus vividly illustrates, we’re also horrible at accepting randomness.
…when video games truly play by the rules, the player can feel cheated. Sid Meier, the designer of the computer game Civilization, in which players steer a nation through history, politics, and warfare, quickly learned to modify the game’s odds in order to redress this psychological wrinkle. Extensive play-testing revealed that a player who was told that he had a 33 percent chance of success in a battle but then failed to defeat his opponent three times in a row would become irate and incredulous…
…So Meier altered the game to more closely match human cognitive biases; if your odds of winning a battle were 1 in 3, the game guaranteed that you’d win on the third attempt—a misrepresentation of true probability that nevertheless gave the illusion of fairness.
This notion of actual probability somehow being “unfair” is something to ponder. As is how the perversion of genuine probability can feel fair.
Are we more concerned with outcomes than opportunities?
It really is an excellent article and I recommend reading it all. It ranges from loaded dice found in ancient Egyptian tombs, to “pity timers” and faux improvement embedded in modern games, to gambling:
The results of any modern slot machine are based on arcane random-number generators in a computerized network, not on the fortunate conjunction of three wooden wheels. But losing to that sort of luck can be dispiriting. So gambling machines often employ the fiction of physical luck—by, say, making it look as if you just missed out on a king’s ransom as the final matching bar of gold or lemon reels to a stop just shy of a jackpot payout. This entices you to once more bet on odds that remain astronomical.
As always my emphasis.
One of my favourite YouTube channels is called Kurzgesagt. They recently released a video explaining why you should trust their short, animated videos on topics as diverse as string theory, ageing and homeopathy.
The interesting part is not the deep dive into their research, writing and fact checking process; but that they spend almost half the video utterly ripping themselves to shreds.
They call out two videos specifically – one on refugees and another on addiction. They explain why they are problematic and that they have been removed.
I trust them so much more because of this self-flagellation. Because they are willing to admit their mistakes and bias. To explain why these were failures and how they were made. To flesh out the context, what has changed and why.
Going through this so comprehensively makes me believe they’ve learnt from their mistakes. And doing so in a prominent space (rather than, for instance, newspaper corrections being buried on page 15) shows they take it seriously.
Being right is a process, is hard, is often undignified, and it doesn’t get easier. Just look at how many public institutions we’ve built around these principles.
Unfortunately the same can’t be said for most of our media. They tend to prefer a model of trust built on prominence and obscurity rather than transparency. They seek to wish away bias rather than own and deal with it.
That doesn’t work anymore.
Some other great videos from the channel: