Have you ever ever puked and had diarrhea on the identical time? I’ve, and when it occurred, I used to be listening to a fan-made audiobook model of Harry Potter and the Methods of Rationality (HPMOR), a fan fiction written by Eliezer Yudkowsky.
No, the dual-ended bodily horror was not incited by the fanfic, however the two experiences are inextricable in my thoughts. I used to be shocked to find years later that the 660,000-word fanfic I marathoned whereas sick has some weird intersections with the extremely rich technorati, together with lots of the figures concerned within the present OpenAI debacle.
Working example: In an Easter egg spotted by 404 Media (which was too minor for anybody else — even me, somebody who’s really learn the thousand-odd web page fanfic — to note), there’s a once-mentioned Quidditch participant within the sprawling story named Emmett Shear. Sure, the identical Emmett Shear who co-founded Twitch and was simply named interim CEO of OpenAI, arguably probably the most influential firm of the 2020s. Shear was a fan of Yudkowsky’s work, following the serialized story because it was revealed on-line. So, as a birthday current, he was gifted a cameo.
Shear is a longtime fan of the writings of Yudkowsky, as are lots of the AI trade’s key gamers. However this Harry Potter fanfic is Yudkowsky’s hottest work.
HPMOR is an alternate universe rewriting of the Harry Potter sequence, which begins with the premise that Harry’s aunt Petunia married an Oxford biochemistry professor as an alternative of the abusive dolt Vernon Dursley. So, Harry grows up as a know-it-all child obsessive about rationalist pondering, an ideology which prizes experimental, scientific pondering to resolve issues, eschewing emotion, faith or different imprecise measures. It’s not three pages into the story earlier than Harry quotes the Feynman Lectures on Physics to attempt to remedy a disagreement between his adoptive mother and father over whether or not or not magic is actual. Should you thought precise Harry Potter may very well be a bit of irritating at occasions (why doesn’t he ever ask Dumbledore the obvious questions?), prepare for this Harry Potter, who might give the eponymous “Younger Sheldon” a run for his cash.
It is smart that Yudkowsky runs in the identical circles as lots of the most influential folks in AI at the moment, since he himself is a longtime AI researcher. In a 2011 New Yorker function on the techno-libertarians of Silicon Valley, George Packer experiences from a cocktail party on the residence of billionaire enterprise capitalist Peter Thiel, who would later co-found and spend money on OpenAI. As “blondes in black attire” pour the boys wine, Packer dines with PayPal co-founders like David Sacks and Luke Nosek. Additionally on the celebration is Patri Friedman, a former Google engineer who received funding from Thiel to start out a nonprofit that goals to construct floating, anarchist sea civilizations impressed by the Burning Man pageant (after 15 years, the group doesn’t appear to have made a lot progress). After which there’s Yudkowsky.
To additional join the events concerned, behold: a 10-month-old selfie of now-ousted OpenAI CEO Sam Altman, Grimes and Yudkowsky.
Yudkowsky isn’t a family title like Altman or Elon Musk. However he tends to crop up repeatedly within the tales behind firms like OpenAI, and even behind the nice romance that introduced us youngsters named X Æ A-Xii, Exa Darkish Sideræl and Techno Mechanicus. No, actually — Musk as soon as needed to tweet a joke about “Roko’s Basilisk,” a thought experiment about synthetic intelligence that originated on LessWrong, Yudkowsky’s weblog and group discussion board. However, because it turned out, Grimes had already made the identical joke a few “Rococo Basilisk” within the music video for her music “Flesh With out Blood.”
HPMOR is sort of actually a recruitment instrument for the rationalist motion, which finds its digital residence on Yudkowsky’s LessWrong. Via an admittedly entertaining story, Yudkowsky makes use of the acquainted world of Harry Potter for example rationalist ideology, displaying how Harry works towards his cognitive biases to develop into a grasp problem-solver. In a ultimate showdown between Harry and Professor Quirrell — his mentor in rationalism who seems to be evil — Yudkowsky broke the fourth wall and gave his readers a “final exam.” As a group, readers needed to submit rationalist theories explaining how Harry might get himself out of a deadly predicament. Fortunately, for the sake of comfortable endings, the group handed.
However the ethical of HPMOR isn’t simply to be a greater rationalist, or as “much less improper” as you could be.
“To me, a lot of HPMOR is about how rationality could make you extremely efficient, however extremely efficient can nonetheless be extremely evil,” my solely different pal who has learn HPMOR advised me. “I really feel like the entire level of HPMOR is that rationality is irrelevant on the finish of the day in case your alignment is to evil.”
However, in fact, we are able to’t all agree on one definition of excellent versus evil. This brings us again to the upheavals at OpenAI, an organization that’s attempting to construct an AI that’s smarter than people. OpenAI desires to align this synthetic basic intelligence (AGI) with human values (such because the human worth of not being killed in an apocalyptic, AI-induced occasion), but it surely simply so occurs that this “alignment research” is Yudkowsky’s specialty.
In March, 1000’s of notable figures in AI signed an open letter arguing for all “AI labs to instantly pause for at the least 6 months.”
Signatories included Meta and Google engineers, founders of Skype, Getty Photos and Pinterest, Stability AI founder Emad Mostaque, Steve Wozniak and even Elon Musk, a co-founder of OpenAI who stepped down in 2018. However Yudkowsky didn’t signal the letter, and as an alternative, penned an op-ed in TIME Magazine to argue {that a} six-month pause isn’t radical sufficient.
“If anyone builds a too-powerful AI, underneath current situations, I count on that each single member of the human species and all organic life on Earth dies shortly thereafter,” Yudkowsky wrote. “There’s no proposed plan for the way we might do any such factor and survive. OpenAI’s overtly declared intention is to make some future AI do our AI alignment homework. Simply listening to that that is the plan must be sufficient to get any smart particular person to panic. The opposite main AI lab, DeepMind, has no plan in any respect.”
Whereas Yudkowsky argues for the doomerist strategy in terms of AI, the OpenAI management kerfuffle has highlighted the wide selection of various beliefs round tips on how to navigate know-how that’s presumably an existential risk.
Appearing because the interim CEO of OpenAI, Shear — now one of the crucial {powerful} folks on this planet, and never a Quidditch seeker in a fanfic — is posting memes in regards to the completely different factions within the AI debate.
There’s the techno-optimists, who assist the expansion of tech in any respect prices, as a result of they assume any issues brought on by this “develop in any respect prices” mentality might be solved by tech itself. Then there’s the efficient accelerationists (e/acc) which appears to be sort of like techno-optimism, however with extra language about how progress in any respect prices is the one approach ahead due to the second regulation of thermodynamics says so. The safetyists (or “decels”) assist the expansion of know-how, however solely in a approach that’s regulated and protected (in the meantime, in his “Techno-Optimist Manifesto,” enterprise capitalist Marc Andreessen decries “belief and security” and “tech ethics” as his enemy). After which there are the doomers, who assume that when AI outsmarts us, it is going to kill us all.
Yudkowsky is a frontrunner among the many doomers, and he’s additionally somebody who has spent the final a long time working in the identical circles as what looks like half of the board of OpenAI. One standard idea about Altman’s ousting is that the board needed to nominate somebody who aligned extra carefully with its “decel” values. So, enter Shear, who we all know is impressed by Yudkowsky and likewise considers himself a doomer-slash-safetyist.
We nonetheless don’t know what’s occurring at OpenAI, and the story appears to vary about as soon as each 10 seconds. For now, techy circles on social media proceed to struggle over decel versus e/acc ideology, utilizing the backdrop of the OpenAI chaos to make their arguments. And within the midst of all of it, I can’t assist however discover it fascinating that, in case you squint at it, all of this traces again to 1 actually tedious Harry Potter fanfic.