Reddit has sued Perplexity AI for continuing to use Reddit’s content to train its AI model after prior warnings not to scrape the platform’s content.
As AI systems increasingly rely on publicly available online content to train and generate answers, companies like Reddit are trying to draw firm lines over what is considered “public” and “proprietary” data.
Reddit’s trap exposes alleged data theft
Reddit has filed a lawsuit against Perplexity, a $20 billion AI company, accusing it of illegally collecting data through its platform. According to court documents filed Wednesday in a Manhattan federal court, Reddit said Perplexity ignored instructions not to scrape its content and continued to use Reddit data to generate AI answers.
The complaint says Reddit had explicitly blocked Perplexity from collecting its data, but the AI company’s “answer engine” still produced results containing Reddit content. “The increase was so dramatic that an outside observer hypothesized that the increase was due to Perplexity entering a licensing deal with Reddit,” the lawsuit said. “In truth, there is no license between Perplexity and Reddit.”
To prove its suspicion, Reddit designed a clever digital test. It created a “trap” post that could only be found by Google’s search engine. Google has a legitimate content-licensing deal with Reddit, and so any company without such a deal should have been unable to access the post.
The company described it as the online equivalent of a “marked bill.” If Perplexity’s system reproduced the contents of that hidden post, Reddit would know it had gone around its safeguards possibly by pulling data through Google’s search results, known as SERPs.
Within hours, the supposedly private test post began showing up in responses generated by Perplexity’s AI tool.
“The only way that Perplexity could have obtained that Reddit content and then used it in its ‘answer engine’ is if it and/or its co-defendants scraped Google SERPs,” the lawsuit stated.
Reddit named three data-scraping companies in the suit, Oxylabs UAB, AWM Proxy, and SerpApi. It accused them of helping Perplexity gain unauthorized access to Reddit’s posts, or of selling Reddit’s data to Perplexity.
Reddit’s allegations denied
Perplexity has rejected Reddit’s allegations. The company’s spokesperson Jesse Dwyer stated that Perplexity “will not tolerate threats against openness and the public interest.” The company also said in a Reddit post after the lawsuit was filed that it “does not train AI models on content.”
Representatives of the other companies named in the lawsuit also issued statements. A spokesperson for SerpApi said it plans to “vigorously defend” itself in court. Oxylabs’ chief governance and strategy officer, Denas Grybauskas, said his company was “shocked and disappointed,” adding that Oxylabs “has always been and will continue to be a pioneer and an industry leader in public data collection.”
In August, Cloudflare, an internet infrastructure company, revealed it had conducted a similar test to see if Perplexity was following web-crawling rules. Cloudflare said it created pages marked with code telling Perplexity’s bots not to access them, but it still found the AI company’s crawlers visiting the restricted pages.
Cloudflare’s CEO, Matthew Prince, made headlines by comparing Perplexity’s behavior to that of “North Korean hackers.”
“Some supposedly ‘reputable’ AI companies act more like North Korean hackers,” Prince wrote on X. “Time to name, shame, and hard block them.” Reddit’s lawsuit quoted Prince’s remarks as part of its case.
The smartest crypto minds already read our newsletter. Want in? Join them.
Source: https://www.cryptopolitan.com/perplexity-caught-scraping-data-reddit/
