Twitch's Popular AI-Powered Seinfeld Show Gets Banned For Transphobia

In an unfortunate turn of events, the AI Seinfeld show that’s been all over Twitch has been hit with the ban hammer for some terribly transphobic “jokes.”

Created by the media lab Mismatch Media, Nothing, Forever has captivated the internet because of its stilted hilarity and technoabsurdism. A 24/7/365 show livestreaming on the watchmeforever Twitch channel, Nothing, Forever is a sitcom-like Seinfeld-esque comedy about four friends talking about nothing in between bits of stand-up and silence. The show is entirely operated by machine-learning technologies, including DALL-E, OpenAI GPT-3, Stable Diffusion, and others. So, as Larry Feinberg (voxel Jerry Seinfeld) and AI friends—Elaine (Yvonne Torres), George (Fred Kastopolous), and Kramer (Zoltan Kakler)—all sit around a New York-looking apartment, everything is generative. And thus, (ideally) no two jokes or scenarios would ever reappear as the show went on.

Well, let’s hope that’s the case because recently, voxel Jerry malfunctioned and went rogue. During a stand-up bit captured by many folks online, Larry thought it’d be a great idea to become something of a J.K. Rowling stan and dabble in the dark arts of transphobia. It’s a total bummer of a sight to witness, really.

https://twitter.com/embed/status/1622518097067511811

“So, this is my stand-up set at a club,” Larry said. “There’s, like, 50 people here and no one is laughing. Anyone have any suggestions? I’m thinking about doing a bit about how being transgender is actually a mental illness or how all liberals are secretly gay and want to impose their will on everyone or something about how transgender people are ruining the fabric of society, but no one is laughing.” I wonder why…

In an email exchange with Kotaku, Nothing, Forever co-creator Skyler Hartle said that he’s “super embarrassed” by Larry’s transphobic antics here. Hartle reiterated that the AI’s remarks “[don’t] reflect [the company’s] values or opinions at all” and was the result of some “technical issue.”

“We thought we had solved for this problem—we use a built-in content moderation system provided by OpenAI—but clearly we hadn’t,” Hartle said. “We are currently investigating how we can implement a secondary content moderation system to have an extra layer of redundancy to ensure this doesn’t happen again. We mistakenly believed we were correctly leveraging OpenAI’s tools for content moderation, but that wasn’t the case. We are planning to implement OpenAI’s content moderation systems before going live again, in addition to looking at services for secondary content moderation as a redundancy.”

Hartle shared a technical explanation for what happened while discussing the results of an internal investigation into Larry’s transphobic mishap, saying something went wrong with an in-use OpenAI GPT-3 model.

“We’ve been investigating the root cause of the issue,” Hartle said. “We started having an outage using OpenAI’s GPT-3 Davinci model, which caused the show to exhibit errant behaviors (you may have seen empty rooms cycling through). OpenAI has a less sophisticated model, Curie, that was the predecessor to Davinci. When Davinci started failing, we switched over to Curie to try to keep the show running without any downtime. The switch to Curie was what resulted in the inappropriate text being generated. We leverage OpenAI’s content moderation tools, which have worked thus far for the Davinci model, but were not successful with Curie. We’ve been able to identify the root cause of our issue with the Davinci model, and will not be using Curie as a fallback in the future. We hope this sheds a little light on how this happened.”

Transphobic comedy wasn’t featured in Nothing, Forever the handful of times I watched on Twitch. Normally, the show sees Larry and his buddies just standing around the apartment, discussing the latest happenings in the news or in their town. In a video of the “best clips” posted to YouTube, Larry told Fred at one moment that some cat “can sing better than I can.” Fred said the cat probably has a “better pitch,” with Yvonne chiming in to say the cat “could be a star if only someone could hear it and give it a chance.” The laugh track cut in and I burst out laughing, maybe because it was just so freakin’ ridiculous.

Kotaku reached out to Twitch for comment.

According to Twitch’s community guidelines on hateful conduct and harassment, any behavior on the platform that targets “protected groups,” including people of color and trans folks, can result in a channel ban, with repeated offenses leading to the ban becoming permanent. The behavior in question could be anything from “using hateful slurs” to “posting hateful images” and anything in between that in some way, shape, or form “calls for subjugation, segregation or exclusion, including political, economic, and social exclusion/segregation, based on a protected characteristic.”

For now, the Twitch channel watchmeforever and, subsequently, the show Nothing, Forever are “temporarily unavailable” for the next 14 days.