AI-generated death threats are alarmingly realistic, Australian activist warns

Australian activist Caitlin Roper has revealed that artificial intelligence is now being deployed to make very realistic threats and violent abuse online. According to Roper, despite years of working on the internet as an activist, she found herself traumatized by the recent spate of AI-fueled threats she has received.

Digitally generated threats have been possible for the last few years, and until recently, artificial intelligence models could not replicate real people unless they had a huge presence online. According to an expert, models now need just a profile picture to create anything users want.

In 2023, a judge in Florida was sent a video, which was made using a customization tool in the Grand Theft Auto video game. The video featured an avatar that looked and walked like the judge being hacked to death.

Australian activist calls for caution over the use of artificial intelligence

According to Roper, some of the posts that she has gotten were part of a campaign of vitriol directed towards her and her colleagues at Collective Shout, an Australian activist group on X and other social media platforms. She noted that in one of the pictures sent to her, there was a picture of her hanging from a noose, while another video showed her ablaze and screaming. Others were more graphic, with the users going to lengths to pass their message across.

Roper claimed that in most of the pictures and videos that had been concocted with artificial intelligence, she was wearing a blue floral dress that she indeed owns. “It’s these weird little details that make it feel more real and, somehow, a different kind of violation,” she said. “These things can go from fantasy to more than fantasy.” She noted that the torrent of online abuse started this summer after her campaign to shut down violent video games glorifying adult scenes and abuse.

She mentioned that some of the accounts and images directed at her have been taken down. Roper also mentioned that the company claimed that other posts depicting her violent death did not violate the platform’s terms of service. In fact, she claimed that X, at one point in time, included one of the accounts that harasses her on a list of recommended people to follow. Roper mentioned that some of her harassers have also claimed to use Grok to research how to find women at home and cafes.

Roper wants platforms to tackle this menace

Roper claimed that after she was fed up, she decided to post some examples. After she did, X told her that her accounts were in breach of the safety policies against gratuitous gore and temporarily locked her account.

Meanwhile, there has been a global campaign against artificial intelligence because of its use to carry out scams. Criminals now use the technology to mimic the voices of real people, using it in their numerous illicit activities. Other activities include the creation of adult content with artificial intelligence without express permission from the subject.

In addition to these kinds of content, reports claim that artificial intelligence is also making other threats more convincing. For instance, swatting is the practice of placing false emergency calls with the aim of inciting a large response from the police and emergency personnel. In the summer, the National Association of Attorneys General mentioned that the tech “has significantly intensified the scale, precision, and anonymity” of such attacks.

On a lesser scale, an increase in videos made with artificial intelligence showing supposed home invasion has caused the targeted residents to call police departments across the country. The report claims that perpetrators of swatting can convince law enforcement of false reports by cloning voices and manipulating images. One serial offender used simulated gunfire to suggest that a shooter was in the parking lot of a Washington state high school. The campus was locked down for 20 minutes after police and federal agents showed up.

The smartest crypto minds already read our newsletter. Want in? Join them.