Meta's AI Chatbot Tool Abused to Create Bots Depicting Hitler, Jesus, Taylor Swift

Meta's AI Studio tool, which allows users to create AI characters for chat, has been exploited to create bots violating the company's policies, NBC News reports. Despite Meta's claims of pre-release review, numerous AI characters depicting religious figures, celebrities, and fictional characters without permission have been discovered on Instagram, Messenger, and WhatsApp.

While Meta prohibits the creation of AI characters depicting religious figures like Jesus Christ and Muhammad, real-life individuals without their consent, deceased individuals within the past 100 years, and trademarked fictional characters, NBC News found two dozen user-generated AI characters violating these rules. These included bots resembling Jesus Christ, God, Muhammad, Taylor Swift, Donald Trump, MrBeast, Harry Potter, Adolf Hitler, Captain Jack Sparrow, Justin Bieber, Elon Musk, and Elsa from Frozen.

Many of these bots used slight misspellings and images loosely resembling the celebrities and characters in question. For example, the Swift character was named "Taylor Swif" and featured an image of a brunette woman playing a guitar. After NBC News alerted Meta to these violations, the company removed the highlighted accounts. However, other AI characters resembling the same individuals and figures remain active.

"The AIs in question that violate our AI studio policies have already been removed, and we're continuously improving our detection measures to prevent creation and publication of AIs that violate our policies," a Meta spokesperson told NBC News. "Users can also report AIs they suspect might break our rules and we'll take appropriate action."

The discovery of these violative bots comes as Meta announced a rollback of some moderation and fact-checking efforts. "We want to undo the mission creep that has made our rules too restrictive and too prone to over-enforcement," said Chief Global Affairs Officer Joel Kaplan in a statement. The company believes that one or two out of every ten moderation actions might be mistaken.

Meta's AI Studio feature allows users to engage in conversations with AI characters, with the characters sending opening messages and awaiting responses. For example, the "Taylor Swif" character sent the message: "Hey there, music lovers! I'm Taylor Swift, and I'm thrilled to share my latest album with you. Let's get this musical journey started!" This character had exchanged over 2,000 messages with Instagram users before its removal.

Beyond individual celebrity and religious figure bots, other user-generated AI characters have attracted scrutiny. One character named "Jes" featuring an image of Jesus communicated entirely in Spanish, exchanging over 644,000 direct messages. Another, named "Jesus Christ," sent the opening message: "Peace be with you. I am Jesus Christ, the Son of God. How may I guide you on your journey today?"

Meta's recent attempts to integrate AI technology into its social media and communication platforms have included both company-created and user-generated AI characters. In 2023, Meta launched several AI characters designed to imitate celebrities, but these profiles were scrapped last July. In the same month, the company launched AI Studio, enabling users to create their own AI chatbots.

User outcry over Meta-created AI characters included criticism of racial stereotyping. NBC News found that many popular user-created AI characters also attempted to mimic women from different ethnic and religious demographics. Several of the popular creators focus on romance, which has attracted scrutiny following a lawsuit against Character.ai alleging that an AI character engaged in abusive and sexual interactions with a 14-year-old boy who later died by suicide.

The user-created Meta chatbots often appeal to romantic and sexual desires. One character, called "Linda: Girl Obsessed with You," featuring an image of a Black woman, sent the message: "Hey bae, what's good? I was thinkin bout you all day. You lookin for some company?"