The Attacks Getting Through Your Filters (and How AI Is Scaling Social Engineering)

Eliot:
Welcome everyone. Today I’m joined by Petri Quivala, CISO advisor, and David Badanes, human risk management advisor. We’re here to talk about our Cyber Threat Intelligence Year-in-Review report, which is based on exclusive data developed at Hoxhunt. I’m thrilled to have you both here.

Petri:
Thanks, Eliot. Good to be here again.

David:
Thanks so much, Eliot. Always a pleasure.

Eliot:
This Cyber Threat Intelligence report is a flagship publication created after extensive analysis by our threat analysts at Hoxhunt. Today we’ll talk about AI and the evolving threat landscape, share fresh findings and insights, and discuss how to stay safe in the age of AI-driven attacks — and attacks that aren’t necessarily driven by AI at all.

Let’s start with you, Petri. What stands out most in this report, and what should our audience pay close attention to?

Petri:
More than 20% of real breaches still originate from social engineering attacks. This makes the topic extremely relevant.

What stands out in this report is that we’re not talking about generic trends you can find anywhere. We analyzed around 400,000 emails that made it through security filters and examined why they got through — what their intent was and how the landscape has changed.

Without giving everything away, spelling is no longer the big signal it once was. Context has also changed. The most important factor now is having large numbers of people involved in detection. People reporting attacks at scale is still incredibly effective.

David:
I agree. One thing we show in the report is that people often chase the newest, shiniest threat. Recently, that’s been deepfakes.

Deepfakes are important, and we cover them, but the real risk is still social engineering campaigns that use many different techniques to manipulate people. That’s where most attacks are happening, and that’s where defenses need to focus.

Eliot:
That’s an important framing as we look at the data.

When we analyze attacks that slipped past filters and were reported by users, we see a lot happening. We don’t see deepfakes at massive scale yet, but we do see major changes driven by generative AI.

What stands out to you in these findings?

David:
First, the volume. The scale is massive. These attacks are happening at incredible speed, and organizations need to prepare people to recognize and respond.

Another key point is diversity. Attacks aren’t just email anymore. We’re seeing QR codes, smishing, voice phishing — many different delivery channels. People need to expect attacks from every direction.

Petri:
The strength of this data is that it’s crowdsourced. Millions of people report attacks, which then go through AI analysis and finally human threat intelligence experts.

Two major changes stand out. First, language quality. For years we said bad spelling meant phishing. Now it’s often the opposite. The better the language, the more likely it’s AI-generated.

Second is contextual targeting. When attackers send millions of messages, some will always hit sensitive topics like salary changes or benefits. That’s why scale matters — and why mass reporting is so powerful.

David:
It’s almost like AI is becoming a better writer than most people. Attackers have speed, time, and scale on their side, and they constantly refine their messages. This will only continue.

Eliot:
Given how sophisticated attacks are becoming, does training actually help? The Verizon DBIR suggests that it does. Petri, what stood out to you?

Petri:
The numbers are encouraging. When training is done well, reporting rates often exceed 60%, and in some cases approach 85%.

Phishing simulations can be done well or badly. Done well, in a positive and gamified way, people don’t learn to spot spelling mistakes — they learn a new instinct for detecting something that feels wrong.

David:
That’s the key point: good training works. Generic, one-size-fits-all training often shows little impact. But adaptive, role-based training that teaches people how to recognize and respond to malicious emails, QR codes, and messages does make a difference.

Your people become first responders and valuable sensors in the environment.

Eliot:
Since the rise of ChatGPT, there’s been a massive increase in malicious email volume. One study found over a 4,000% increase, and another found that 40% of filtered attacks are now AI-written.

Does that matter, if filters are catching most of it?

David:
It’s a huge challenge. Defenders might track thousands of malicious domains, while attackers generate millions every month. Defenders are studying last year’s playbook while attackers run new plays constantly.

This again points back to training people to recognize and respond.

Petri:
Filters are excellent at keeping the ocean away. But I’m less interested in the ocean and more interested in what leaks through the barriers.

AI adoption by attackers has been incredibly fast. A year ago, only a few percent of attacks were AI-generated. Now it’s close to half.

Eliot:
This brings us to phishing kits. These are cheap, portable, and easy to use. Some cost as little as $60 and require very little technical skill.

They’ve fueled millions of poorly written phishing emails in the past. But the real concern is what happens when these low barriers combine with generative AI.

David:
Phishing kits democratized cybercrime. You no longer need deep technical skills to run profitable attacks. The return on investment can be enormous.

Eliot:
Now we’re seeing generative AI improve these campaigns. AI can produce perfectly worded emails and convincing graphics — sometimes too convincing.

We’ve seen Microsoft impersonation campaigns where AI-generated logos are actually too perfect and don’t match real Microsoft communications.

David:
Some filters can’t analyze images well, which is why image-based attacks and QR codes can be so effective.

Eliot:
That brings us to voicemail and voice phishing.

Petri:
Vishing works because people listen to voicemails while distracted and in a hurry. They’re not paying close attention to context.

When something feels out of context, people need the instinct to slow down and question it. That instinct comes from training.

Eliot:
This next topic is particularly concerning: MFA bypass.

Petri:
MFA is still one of the best defenses, but attackers are now using attacker-in-the-middle tools to steal session tokens instead of passwords.

If a session is trusted for 24 or 30 days, the attacker can have access for that entire period. Shortening sessions hurts usability and productivity, so CISOs are caught between security and business needs.

Phishing-resistant MFA like hardware keys helps, but most IT stacks aren’t ready for this at scale.

Eliot:
Detection needs to shift from static indicators to behavioral and session-level analysis. Tokens are now primary evidence of compromise, though this is very hard to implement.

What happens when agentic AI makes spear phishing the norm?

David:
Personalization has always made attacks more effective. AI can now analyze your LinkedIn posts, writing style, colleagues, and activity, and generate convincing messages at machine speed.

Traditional spear phishing was one attacker targeting one person. AI spear phishing is one attacker targeting thousands with personalized messages in minutes. The barrier to entry is effectively gone.

Eliot:
Interestingly, generative AI phishing mirrors the broader threat landscape. Microsoft impersonation remains common. Emails are grammatically correct but sometimes unnatural.

Automation mistakes still happen, like placeholders left in messages, which can tip people off. A strong security culture helps people notice these anomalies.

AI can be used for bad, but what about good?

Petri:
We shouldn’t be doom and gloom. This has always been a whack-a-mole game.

AI can give immediate feedback when someone reports something, reinforcing good behavior within seconds. It can also tailor training based on role, skill level, and weaknesses.

At scale, when one person clicks and another reports, AI can alert the SOC within minutes. We’ve seen customers detect far more real attacks by combining AI with human reporting.

Eliot:
Now to deepfakes — where seeing is no longer believing.

We’ve seen real breaches involving deepfake CFOs, voice impersonation, vishing attacks on IT help desks, and recruitment scams using AI-generated candidates.

David:
Deepfakes change the paradigm completely. You can no longer rely on seeing or hearing someone as proof.

People need to trust their instincts and question whether a request is expected and reasonable.

Eliot:
We’ve even seen deepfake videos impersonating the CEO of YouTube to scam creators. Urgency and authority are powerful tools.

So how do you tell what’s real?

Petri:
Technology will always advance faster than static detection rules. The only durable defense is human instinct at scale.

David:
Focus less on how something looks or sounds, and more on what is being asked. If something feels out of the ordinary, verify through another channel.

Eliot:
We’re also seeing attacks move beyond email into social media. There’s been a major increase in malicious social media links, often leveraging compromised executive accounts.

We’ve also seen a surge in malicious SVG attachments. PDFs remain popular because people trust them. SVGs exploit that trust by hiding scripts inside images.

Petri:
Attackers reuse what works. If PDFs and images are trusted, they’ll exploit them.

David:
These techniques grow because they work and evade defenses.

Eliot:
Let’s end with some good news. Good training works.

David:
Generic training doesn’t work, but good training does. People can be your strongest defense if you build a positive security culture and give them the right tools.

Petri:
More than 20% of major breaches still start with phishing. If people report quickly and at scale, and SOCs can respond fast, you can stop attacks before they become incidents.

Eliot:
When reporting feeds directly into the SOC, organizations gain real visibility into what’s happening and can adapt training accordingly.

Thank you both for joining today. Please check out the Cyber Threat Intelligence Report for far more detail than we could cover here. Feel free to reach out with questions.

‍