How To Stop Monsters With Truth

E.M. Forster’s 1909 classic “The Machine Stops,” describes a future where humanity lives underground, separated from the natural world, eliminating all impediments to the growth and final hegemony of “The Machine.” Everyone lives in private rooms, and physical interaction is made obsolete by video-calling courtesy of The Machine. All bodily needs — food, clothing, medical assistance — are taken care of by, you guessed it, The Machine.

The Machine is an allegory — a symbol — of a system grown so complex that no single person can understand it, let alone control it. There is a human government, a Central Committee, ostensibly responsible for managing The Machine. But the Central Committee is nothing more than a marionette whose strings are jerked by the all-powerful Machine.

Humanity is obsessed with “ideas,” vague concepts no longer in the service of human flourishing; rather, the intellectuals of the day equate The Machine’s progress with human progress. In short, humanity is “strangled in the garments they have woven.”

The Machine’s repair apparatus breaks, the underground world cracks open, and a flood of air suffocates a population that has evolved to no longer draw strength from the oxygen of natural air.

Forester’s story is full of remarkable prescience, not least of which is a video calling system 90 years before Skype. But beyond technologies in particular, Forester portends a world where human management is replaced by programmed intermediaries — algorithms — who steer the world with millions of invisible commands. This is a world we can all recognize because we live in it.

Tim Maughan’s story in OneZero: “The Modern World Has Finally Become Too Complex for Any of Us to Understand,” describes the complexity that overwhelms us all. “I am here to tell you that the reason so much of the world seems incomprehensible is that it is incomprehensible,” says Maughan.

Maughan details a global supply chain where five to six million algorithmically managed containers cross the oceans at any given time. Behind the supply chain and every other commercial act is an impenetrably complex global financial system.

And we all feel anxious at the prospect of an entire lifetime spent processing a single drop in the vast sea of information. The raw data behemoth is even more monstrous, an average of 1.7 MB is generated per second per person according to Domo.

Our inscrutable system going rogue is a popular theme in fiction, in part because it is such a real threat. Anyone old enough to remember the 2008 financial crisis is aware of how the malfunctioning of a system designed by humans can cause its creators so much suffering.

Make no mistake, we are not claiming the complexity of the modern world is bad for humanity. The supply chain and global financial system are synonymous with a glorious flourishing of wealth and comfort. Rather, we are calling for a commitment to building systems of transparency and accountability into our Machine before it becomes a monster.

The encroachment of wearables onto our bodies and into our social lives foreshadows a future where algorithmic managers guide our every move. Your Google Fitbit telling you to stand up is a helpful health tip. Your Amazon Halo telling you to change your tone is intrusive, but potentially profound communication advice.

You may not worry about the means your devices use to arrive at their recommendations when they are benign, but what about when Google tells you who to marry, or who to vote for? These are no longer questions for science fiction writers, but actual design decisions made by technology companies every day.

Twitter and Square CEO Jack Dorsey, a man who understands the state of technology better than almost anyone, had this to say in an interview with Andrew Yang in March: “Google has a better sense of what we prefer than we do… these algorithms are going to start making more and more of these decisions… Why did my watch just tell me to stand? We are basically giving up our agency completely.”

One solution is regulation: forcing technology companies to tell us how their algorithms come to decisions and what data they have about us. California and Europe give consumers the right to know about, and opt-out of, data held by private companies. Proposed legislation in New York state, called the NYPA, goes a step further — classifying data as personal property and granting users the power to opt-in to sharing their data.

But creating technology that is transparent and human-focused by-design is the best way to lash new innovations to the advancement of human wellbeing. There are at least two major steps we need to take to get there: The first is ensuring data is secure and trustworthy, the second is ensuring the processing itself — the algorithm — is transparent and trustworthy.

IoT devices equipped with so-called “Trusted Execution Environments (TEE),” used today in your phone to safely store your fingerprints, can collect and sign data securely before broadcasting it to a database, or to an algorithm for immediate processing. This signing, which you can think of like a digital watermark, is a root-of-trust that takes us over step-one.

TEEs combined with other technologies like blockchain and confidential computing can form a chain of data provenance that extends all the way to the end-user, who can then verify the provenance of information and inspect algorithmic recommendations if they so choose. Confidential computing ensures that code execution occurs inside a tamper-proof, cordoned off area within the CPU that will always do exactly what it says it’s going to do,

These are brand new technologies that are just starting to see mainstream adoption, but the future is bright with the combination of confidential computing, secure hardware, and blockchain. Together, these technologies offer and end-to-end solution that is both transparent and accountable; ensuring there is never any question about who’s in charge: Man over The Machine.