How Facebook’s Naive Optimism Built A Toolbox for 21st Century Totalitarianism

Naïveté, hubris, and profound blindness to the lessons of history don’t have a great track record for producing happy outcomes.

“When someone shows you who they are, believe them the first time.” — Maya Angelou

Maybe when we learned that a 19-year old Mark Zuckerberg called 4,000 of his fellow Harvard students , “dumb f$cks”, for trusting him with their personal information, we should have believed him the first time.

But those were the halcyon days of 2010, when Facebook was still a private company and many of us who’d first used it when it was open to college kids only were just nearing the ends of our 20s.

Back then, some of us on the outside hoped that Facebook would evolve into the Internet’s humanizing, unifying force.

We let ourselves believe that if Facebook continued to extend its reach across geographical, national, religious and racial boundaries, maybe it would show us that even though we often disagree about how to get there, all of us want the same things for ourselves and our children: to be safe, to be at peace, to have food and shelter, respect and opportunity and hope.

But let’s face some realities:

• For all of its utopian conceptions of itself, Facebook is a massive, monopolisitic data maw that uses the variable reward mechanisms pioneered by slot machines to feed our brains dopamine while collecting copious amounts of our personal information.
• With this all-consuming hunger for data on the things that make us tick, Facebook has assembled the biggest, most detailed online identity database and the most powerful media-targeting engine ever known.

To any business that has realized that effective content marketing is the single most powerful form of advertising, Facebook’s media targeting capabilites are a boon.

As an entrepreneur who understand his target market and knows how produce authentic stories that resonate with it, I know there is simply no better way than Facebook advertising to get my message in front of exactly the right people.

But as someone who cares deeply about freedom and human rights and wants them to exist in the future instead of collapsing into some form of AI-powered totalitarianism, my feelings about Facebook are much dimmer.

So today, let’s explore one possible logical conclusion of what Facebook has created.

In this scenario, Facebook devolves even further from its status as an vanity-powered slot machine that occasionally moonlights in psychological warfare operations into the perfect toolbox for 21st century totalitarianism.

Like any projection into the future, this remains speculative fiction…for now. If sensibile, freedom-loving humans get our act together, it can remain that way.

The #1 Tool For The 21st Century Totalitarian: Surveillance at Scale

Before the fall of the Berlin Wall and the collapse of the East German communist state, the East German secret police (aka the Stasi) gathered data by hand, relying on stakeouts and a massive network of civilian informants to build detailed files on its targets, whose ranks included pretty much everyone.

Under the watchful eyes of the Stasi, whose file cabinets stretched for miles, case agents were responsible for keeping tabs on individuals and groups.

Though these agents had large and ever-growing dossiers at their disposal, they were essentially confined to what they could capture on paper and keep in their minds.

The 21st century totalitarian has no patience for such an antiquated approach.

And why should he? Compared to Internet-scale data collection, the Stasi’s analog methods are slow, inefficient, and prone to lapses in human judgment.

No, the 21st century totalitarian demands a more sophisticated way to capture and catalog the activities, psychologies, and political sentiments of the population in real time.

Ideally, this would be a place online where people share freely and regularly about themselves. A place just like…Facebook, Instagram, andWhatsApp.

Yeah, that’s the ticket.

#2: Predictive Behavior Modeling (“Digital Precognition”)

To the 21st century totalitarian, ongoing collection of massive amounts of personal data is only part of the equation.

Because once you have access to a large, rich and ever-growing online database of human behavior, you need algorithms that can mine its activities and uncover threats to the status quo.

Along with potential terrorists, these threats include:

• Subversives
• Whistleblowers
• Radicals

If you were a 21st century totalitarian, wouldn’t it be nice if you could build a detailed psychological profile of these types and use your massive identity database and a bit of sophisticated machine learning to detect others in real time?

Sure, you’d surface a substantial number of false positives, but it’s nothing some light investigation and/or interrogation from your secret police couldn’t suss out.

With such a tool at your disposal, it’d be like having the pre-cogs from Minority Report, but instead of weirdly-psychic humans drugged up and living in a bath, you’d have an All-Seeing Machine.

Now, if you’re a grumpy old white, assembling the talent necessary to build such technology would be a challenge.

It would be much better if the naive young people from the same company that built your massive surveillance engine were also thoughtful enough to train an AI to mine their users’ posts and flag “troubling psychological tendencies.”

You know…something exactly like the technology Facebook built into its “suicide warning” bot?

Yep.

#3: Mass Manipulation

The 20th century totalitarian relied on absolute control of the flow of information. Writing official textbooks, burning subversive materials, censoring the media, and publishing a non-stop flow of propaganda was simply the way of things.

But for the 21st century totalitarian, the fragmentation of TV and the free-form frontiers of the Internet complicates this strategy.

Sure, you could just co-opt all the cable news stations, but there are so many other channels! Radio? That’s so 1933.

No, while these blunt forms of manipulation may work for Fox News and Rush Limbaugh, they are too obvious to work across an entire nation.

Today’s totalitarian demands a more subtle way to influence cultural and political sentiment. He wants the kind of thing you’d have if he got his hands on an algorithmically filtered newsfeed that reached over a billion people.

Because see: once you have a way to influence the stories people see every day, you can shape their understanding of reality and their emotional states across geographic, political and economic lines!

With such a tool like that, your totalitarian regime would have the edge it needed to pry open the doors of a divided open society and get itself ready for business.

Naive Optimism + Hubris + a Profound Lack of Historical Awareness = Facebook, 2016–2018

I don’t know Mark Zuckerberg personally, but I’m willing to believe he imagined that he was setting out to eliminate artificial barriers between people and build a better world.

And when I first logged into Facebook in 2004, I thought that’s what it might become too.

Even when Facebook made one boneheaded privacy blunder after another, I was willing to chalk it up to youthful mistakes and give the company the benefit of the doubt.

But then there was the mood-influence study that scandalized us for a couple of weeks back in 2014, when Facebook changed the emotional tone of content showing up in people’s feeds to test the impact it could have on their moods.

The results, not too surprisingly, suggested that Facebook had both the power to manipulate public sentiment and the willingness to run large scale psychological tests without their users’ consent.

And when Facebook allowed itself to be weaponized into a global propaganda distribution machine and Zuckerberg reflexively denied his company’s role in the mass-scale psychological warfare operation that played out in 2016?

Well, that was the last straw.

Or so I thought.

Even when I was hopeful about its potential, I’d always vaguely worried that Facebook’s cultural DNA might eventually evolve it into a perfect toolbox for totalitarians.

The disclosure of Cambridge Analytica’s theft of 50 million profiles and Facebook’s defensive, ham-fisted response to it just really brings the whole thing home.

Because while I am hopeful that the ideals of human freedom and universal human rights will persevere against the paranoia of our modern age, there’s a reasonable case to be made that these ideals could go into hibernation in the relatively near future.

Given how easy it is to scare people about the scary-seeming-but-actually-low-risk Ebola, and how quickly our higher-order intelligence shuts off when we are afraid, it is not crazy to think that under the wrong circumstances — like one or two more mass-scale terrorist attacks on major cities — what’s left of the rule of law, free speech, and the US Constitution gives way to something much closer to 1984.

If Big Brother were to seize the reins of power, sure, he’d use the cable news the way it’s being used today. But Facebook’s data maw, targeting power and sentiment-manipulation capabilities would be far more insidious.

So I’m hitching a ride with the #deletefacebook bandwagon and using this Chrome plugin to delete every one of my historical posts.

Care to join me?

But here’s one weird fact you’ll never believe: Facebook is not the real problem.

In a very real sense, it is is neither accurate nor fair to blame Facebook for the scale of the mess we’re all in today.

The explosion of “tribal reality” that washed over America in 2016 showed up loud, bright, and clear in the 2004 Presidential election, when the majority of Democrats and Republicans believed two completely different versions of reality about 9/11 and the Iraq War.

Back then, Facebook was barely 9 months old.

So while our desire to pin the blame Facebook for dividing Americans into alternative-reality camps is natural and understandable, doing so obscures a much more complicated, painful truth.

We need to face the fact that the gasoline of polarization had been gushing onto the floor for years before Facebook scaled to 1.2 billon users and started handing out matches.

We may not enjoy admitting it, but the root of the problem is not Facebook. Or Twitter. Or YouTube.

The toxic effects they have on our minds, our political culture, and our social norms are merely symptoms of something much more problematic.

They are flowers sprouting from the soil of an economic, political, and psychological environment that rewards the people who build the most extractive, monopolistic business models and architect the most addicting products with huge increases in net worth.

And yet, even that soil is just a symptom of the much larger, more persistent, and long-lived problem at the root of all of this.

The root cause of the majority of our social problems is scarcity…not scarcity of the basic material resources humans need to survive, but scarcity of all the things human beings need to thrive.

Rich and poor. White, brown, black, and yellow. Christian, Jewish, Muslim, Buddhist, Confucian, Taoist, Sikh. Gay, straight, and somewhere in between. Liberal and conservative. Libertarian and authoritarian.

Scarcity touches everyone.

Billions of human beings from every spectrum of identity and every tribe on Earth are starving for love, freedom, dignity, hope, belonging, and self-respect.

These are hardly new developments. We can trace most of them back hundreds, thousands, maybe even hundreds of thousands of years.

Given the chaotic global climate our ancestors faced…which seems to have included massive fluctuations of rainfall that could turn a massive lake into a desert in a single generation and a FREAKIN ICE AGE, some of the most painful forms of scarcity trace back to our species’ earliest days on Earth.

But something miraculous has happened over the last 20 years, and it’s gone into high gear in the last 10.

For the first time in human history, ALL of the philosophical, psychological, and educational frameworks and just about every technology necessary to create and sustain abundant, low-cost access to energy, water, nutritious food, high-quality education, physical and mental health, transportation, and shelter exists at the same time.

Right now, in 2018, we already have everything we need to reduce and possibly eliminate nearly all the waste in our systems. And we can do it all in a way that benefits the planet’s environment instead of harming it and possibly destabilizing it.

The obstacles to the widespread deployment of these ideas and technologies are misaligned incentives and scale.

And thanks to an extraordinary invention by a strange reclusive figure who calls itself Satoshi, humanity may have a path around those obstacles as well

No, that invention is not Bitcoin. It is the blockchain, the much more significant technology that makes Bitcoin possible.

In a much bigger upcoming guide to the blockchain, we’ll explore these things as well.

The first version of this essay appeared on TechCrunch in 2014. Revised and updated: 3/20/2018).