5 hours ago
1
Welcome to Eye on AI! In this edition...xAI releases Grok 4 amid backlash over antisemitic posts…Perplexity launches AI browser to take on Google—starting with its power users…OpenAI is reportedly gearing up to launch its own web browser.
An audio deepfake impersonating Secretary of State Marco Rubio contacted foreign ministers, a U.S. governor, and a member of Congress with AI-generated voicemails mimicking his voice, according to a senior U.S. official and a State Department cable dated July 3.
There’s no public evidence that any of the recipients of the messages, reportedly designed to extract sensitive information or gain account access, were fooled by the scam. But the incident is the latest high-profile example of how easy—and alarmingly convincing—AI voice scams have become.
With just 15 to 30 seconds of someone’s speech uploaded to services like Eleven Labs, Speechify and Respeecher, it’s now possible to type out any message and have it read aloud in their voice. Keep in mind, these are tools used perfectly legitimately for a host of things from accessibility to content creation – but like many AI technologies, can be misused by bad actors.
The threat of deepfakes has escalated
AI-generated deepfakes aren’t new, particularly of C-suite leaders and public officials, but they are becoming a bigger problem. Eight months ago, I reported that more than half of chief information security officers (CISOs) surveyed ranked video and audio deepfakes as a growing concern. That threat has only escalated. A new study by Surfshark found that in the first half of 2025 alone, deepfake-related incidents surged to 580—nearly four times as many as in all of 2024 (150 incidents), and dramatically higher than the 64 incidents reported between 2017 and 2023. Losses from deepfake fraud have also skyrocketed, reaching $897 million cumulatively, with $410 million of that in just the first half of 2025. The most common scheme: impersonating public figures to promote fraudulent investments, which has already resulted in $401 million in losses.
“Deepfakes have evolved into real, active cybersecurity threats,” Aviad Mizrachi, CTO and co-founder of software security company Frontegg, told me by email. “We’re already seeing AI-generated video calls successfully trick employees into authorizing multimillion-dollar payments. These attacks are happening now, and it’s a scam that is becoming alarmingly easy for a hacker to deploy.”
Part of the problem, Mizrachi added, is that traditional authentication methods—usernames, passwords, one-time codes, and authenticator apps—weren’t designed for a world where a scammer can clone your voice or face in seconds. That’s because these scams don’t necessarily involve breaking into an account—they rely on tricking a real person into handing over credentials or authorizing actions themselves.
“Those traditional security measures to check the identity of an individual obviously don’t work anymore,” he said, adding that most cybersecurity teams still overlook deepfakes—and that’s the vulnerability attackers exploit. A convincing fake voice on a voice mail message or video call can persuade someone to bypass normal procedures or approve a wire transfer, even if all the authentication tools are technically in place.
To guard against that kind of deception, Mizrachi said, organizations need to deploy stronger security tools that rely on physical devices—like a smartphone or hardware security key—to prove someone’s identity. These tools, known as FIDO2 or WebAuthn passkeys, are far harder for hackers to fake or phish. And beyond device checks, smart verification systems can also monitor behavioral signals—like typing speed, location, or login habits—to spot anomalies that a cloned voice can’t imitate. Those extra layers make it much harder for a deepfake attack to succeed.
Margaret Cunningham, director of security and AI strategy at security firm Darktrace, said that the impersonation attempt of Rubio demonstrates just how easily generative AI can be used to launch convincing, targeted social engineering attacks.
“This threat didn’t fail because it was poorly crafted—it failed because it missed the right moment of human vulnerability,” she said. “People often don’t make decisions in calm, focused conditions. They respond while multitasking, under pressure, and guided by what feels familiar. In those moments, a trusted voice or official-looking message can easily bypass caution.”
Deepfakes have impacted democracies around the world
Generative AI has also dramatically lowered the barrier to entry when it comes to media manipulation—making it faster, cheaper, and more scalable than ever before. And it is impacting democracies around the world: A recent New York Times report found that AI-powered deepfakes has transformed elections in at least 50 countries around the world, by using them to demean or defame opponents.
“This is the new frontier for influence operations,” Leah Siskind, AI research fellow at the Foundation for Defense of Democracies, told me. “We’ve seen other instances of deepfakes of senior government officials used to gain access to personal accounts, but leveraging AI to influence diplomatic relationships and decision-making is a dangerous escalation. This is an urgent national security issue with serious diplomatic ramifications.”
For now, Siskind recommends that government officials steer clear of popular encrypted platforms like Signal, which while secure in terms of content, lack mechanisms for identity verification. “Given the ease of creating deepfake audio and building out realistic-looking accounts on any consumer-grade messaging app, senior government officials should stick to secure communication channels,” she said.
Note: Check out this new Fortune video about my tour of IBM’s quantum computing test lab. I had a fabulous time hanging out at IBM’s Yorktown Heights campus (a midcentury modern marvel designed by Eero Saarinen, the same guy as the St. Louis Arch and the classic TWA Flight Center at JFK Airport) in New York. The video was part of my coverage for this year’s Fortune 500 issue that included an article that dug deep into IBM’s recent rebound.
As I said in my piece, “walking through the IBM research center is like stepping into two worlds at once. There are the steel and glass curves of Saarinen’s design, punctuated by massive walls made of stones collected from the surrounding fields, with original Eames chairs dotting discussion nooks. But this 20th-century modernism contrasts starkly with the sleek, massive, refrigerator-like quantum computer—among the most advanced in the world—that anchors the collaboration area and working lab, where it whooshes with the steady hum of its cooling system.”
With that, here’s the rest of the AI news.
Sharon Goldman
sharon.goldman@fortune.com
@sharongoldman
This story was originally featured on Fortune.com
English (US) ·