According to a data analysis company Elon Musk hired as part of his legal dispute with Twitter, spam and fake accounts are more common on Twitter than on comparable social media networks.
The claim made by Cyabra in the company’s first public interview after the study it conducted for Musk, which found that spam and bot accounts make up an estimated 11% of Twitter’s total user base, highlights the potential headache for Musk now that he seems prepared to complete his $44 billion acquisition of the social media platform.
Cyabra CEO Dan Brahmy told CNN last week that in addition to studying Twitter (TWTR), his company has done similar assessments of the company’s rivals although he declined to name any specific platforms and that the fake-account problem appears to be comparatively worse for Twitter (TWTR) than its peers. Using a machine-learning algorithm that analyzes “hundreds” of parameters, he said, Cyabra provides estimates of whether certain online accounts and content are authentic.
“We have a number for all the social media platforms, because that’s our job,” he said, adding that Cyabra’s estimate for Twitter is “definitely not in the low end” relative to its competitors (although the company had more access to data about Twitter when it conducted its analysis for Musk).
Twitter bots have been at the center of the controversy over Musk’s original attempt to back out of the acquisition deal. Despite having previously stated that he wanted to buy the company to address its bot problem, Musk moved to terminate the agreement less than three months after signing the agreement and waiving due diligence in the process. Musk cited claims that Twitter had overstated the number of bots on its platform.
Twitter has for years said that bots make up less than 5% of its monetizable daily active users (mDAU). In a series of tweets in May, Twitter CEO Parag Agrawal acknowledged that “spam harms the experience for real people on Twitter,” and added that, “as such, we are strongly incentivized to detect and remove as much spam as we possibly can, every single day.”
The trial for the deal was scheduled to begin later this month. Twitter sued Musk to close the sale, accusing him of using bots as a cover to back out of it after experiencing buyer’s remorse. But last week, Musk told Twitter he is now ready to move ahead with the deal at the original price, and the judge overseeing the case gave the two sides until Oct. 28 to close the deal or face trial.
The dispute over the true number of fake accounts on Twitter and how to accurately calculate it has also shone a spotlight on the small army of data scientists, artificial intelligence experts and misinformation researchers both within and outside tech companies who spend their days hunting for patterns or anomalies that can uncover whether a given account, or a network of accounts, represents a living, breathing human being or, potentially, automated accounts or bad actors aiming to spread covert influence, false information or spam.
The request was precise: Spam and bot accounts, tell us the number, tell us the methodologies and tell us the confidence level. Cyabra did not work with the other data analysis firms used by Musk’s team, and he declined to discuss whether or how much Cyabra was paid to perform the commissioned analysis.
Cyabra CEO Dan Brahmy
With Musk and Twitter now expected to close the acquisition deal, the public may never get a close look at the evidence each side gathered on the issue or a court ruling on Musk’s claims that bots are more pervasive on the platform than it has let on. But Cyabra is coming forward now, Brahmy said, after the company’s work for Musk’s team was revealed in court filings and hearings in the acquisition dispute. The Israel-based company has roughly 40 employees and counts the US State Department and various consumer brands as customers of its digital authenticity measurement tools, he said. Among its investors is the Peter Thiel-backed Founders Fund.
Many practitioners in the field caution that estimating spam or fake accounts can be an extremely subjective exercise, and that anyone claiming to have a definite number likely doesn’t grasp the complexity of the issue.
Even the creator of Botometer, another service Musk used to estimate of the number of bots on Twitter, has emphasized the challenges behind defining the term “bot” and cautioned that context, intent and even the way an account is managed can complicate matters. There are also good bots designed to share information or entertaining content that are allowed on Twitter and many of which transparently self-identify as automated on the platform.
In the interview last week prior to the revived deal talks, Brahmy acknowledged there are clear limits to his company’s use of machine learning algorithms to determine whether accounts may be authentic, which is why his firm’s report about Twitter expresses a maximum confidence level of about 80%. The company’s algorithms would likely flag Brahmy’s own grandmother as a suspicious account holder, he said, given her propensity to whiplash randomly from topic to topic and to post at unusual hours with numerous grammatical mistakes.
Twitter lawyers in a hearing earlier this month said that neither Cyabra’s analysis, nor an analysis Musk commissioned by another firm that estimated with 90% confidence that bots make up 5.3% of the platform’s user base, support Musk’s claim in his deal termination letter that the amount of spam is “wildly higher than the Twitter estimate” of less than 5% of mDAU.
Cyabra’s analysis
Cyabra first made waves in May, after Musk’s deal was first announced. Using publicly available data, the company claimed at the time that spam and bot accounts represented 13.7% of accounts on Twitter. Soon thereafter, the company was hired by Musk’s team to perform a second analysis, this time based on the “firehose” of data that Twitter had provided directly to the Tesla CEO, according to Brahmy.
That second analysis, which has since been referenced in court filings, concluded with roughly 80% confidence that spam and bot accounts represent 11% of Twitter’s total user base. The finding offers a look at the hurdle Musk may face if he takes over Twitter after previously saying he wanted to “defeat the spam bots or die trying.”
“The request was precise: Spam and bot accounts, tell us the number, tell us the methodologies and tell us the confidence level,” Brahmy said. Brahmy said Cyabra did not work with the other data analysis firms used by Musk’s team, and he declined to discuss whether or how much Cyabra was paid to perform the commissioned analysis.
The finding can’t be compared directly to Twitter’s own estimate that fake and spam accounts comprise less than 5% of mDAU Brahmy said Cyabra was unable to make an estimate of the prevalence of bots as a percentage of mDAU rather than total Twitter users. He declined to elaborate on why, but it may be due to the fact that Twitter’s firehose of data used to conduct the analysis does not show which accounts are or aren’t considered mDAU and only shows those accounts that actively tweet rather than those who, for example, only use the platform to read tweets.
(For its part, Twitter’s disclosures about spam and fake accounts note that its calculation relies on “significant judgment” and the company has not made claims about how the prevalence of such accounts on its platform compares to competitors.)
Factors built into Cyabra’s algorithm used to measure how likely an account is to be fake or spam include the members in that account’s social network and where they’re located; what the account talks about, how frequently and at what hours; whether the account uses one or multiple languages and how fluently it uses those languages; and the type of engagement the account’s content tends to generate, among many others.
