Can AI stop scam emails? Yes. But not nearly as well as the headlines claim

It’s twenty to four on a Friday. The invoice looks exactly like all the others from the supplier you’ve had for nine years: same logo, same reference, same dry wording. Only the account number is new. There’s something about a change of bank.

Nobody calls to check on a Friday at twenty to four.

That’s how it happens. Scam emails cost European businesses billions every year, and what makes them hard to spot isn’t that they’re clever. It’s that they never look the same twice.

Last summer, a research report made headlines about AI now being our digital bodyguard against fraudsters. We read it. It says something different from the headlines. What it says is also much more useful, so stick around.

First: the number nobody brags about

The researchers had nine different AI services, including ChatGPT, read several thousand real scams. None of them had been trained specifically for the task. They just read and gave their opinion, roughly like if you pasted a suspicious email and asked, "what do you think about this?"

The best ones caught two out of three.

Two out of three. Read that number again before someone tries to sell you AI as the solution to fraud in your organisation. A bodyguard who misses every third threat wouldn’t get the job. They wouldn’t even get an interview.

Then: the number that actually matters

Here’s where it gets interesting, because the researchers did something smarter than just counting hits.

They pitted the regular AI against a specially built program, the kind already used in email filters and security products. Those programs are fed scams that have already happened, and they get really good at spotting exactly those. In the first test, the specialist won, as expected.

Then the researchers showed both of them material neither had seen before. New scams, new approaches.

The specialist lost almost half its ability. The regular AI stayed at the same level as before.

And there’s the whole point. A regular spam filter is like a bouncer with a binder of old mugshots: great at recognising the ones who’ve already been caught, blank-faced when someone new steps up. The AI, on the other hand, stands there listening to how people talk. It reacts to the fake urgency, the gap in the story, the tone that wants you to skip a step you’d never normally skip.

Worse at the old stuff. Much steadier at the new. And it’s the new stuff that hits you, because fraudsters change their approach far more often than filters can be updated.

Two more things from the same report

You don’t need a big project. Even the smaller and cheaper services got noticeably better just by asking the question more clearly. It’s mostly about how you phrase the task, not about building something custom for millions.

The researchers aren’t bragging. They openly write that the material they tested on was smaller and more skewed than they wanted, and that the results varied depending on how the question was asked. That’s unusually honest, and it’s a reason to trust them more, not less.

What two out of three is good for

Not for making decisions. Great for pointing things out.

Build it like a colleague who clears their throat, never like a barrier that silently throws things in the bin. A filter that guesses wrong and deletes a real customer email costs you more than the scam it would have stopped. A small flag that makes a human pause for two seconds costs almost nothing.

And that’s where scams get stopped. Not in the technology. In that little thought: wait, do they usually change banks on a Friday?

How we did it with our own inbox

We tried it on ourselves. It’s the only way we know a claim holds up.

All mail to eurowork.ai goes through the same sorting, whether it’s sent to hej@, hola@, bonjour@ or czesc@. Every email gets an assessment: real email, newsletter, marketing, or scam. The assessment is in the subject line when the email reaches us, with a line about why. You know what you’re looking at before you even open it.

Nothing gets deleted. Nothing gets filtered out. The assessment points, the human decides.

If it’s a real email, two more things happen.

It gets translated into Swedish, and it’s the Swedish version we save. The original is always kept under the translation, so nothing can hide in an interpretation.

And the person who wrote gets a reply in their own language. A short line that says the email arrived, shows with one sentence that we understood what it’s about, and promises a human will read it and get back. The line is marked as written by AI, because it is, and the EU AI Act requires us to say so. The actual answer to the question always comes from a human.

Newsletters, marketing, and scams never get a reply. Neither do those addresses that start with "no-reply" and that no human reads. They get silence, and that’s the right answer. A friendly auto-reply to a mass email is, at best, a waste of time, and at worst, two machines politely thanking each other forever.

A translator at the door, with a bouncer next to them. We’re pretty happy with that.

Three things to take to your next meeting

Don’t believe anyone who says AI solves fraud. Two out of three isn’t a solution, it’s a help.

Take the help anyway, because it holds up when the scam is new. That’s exactly where the old filters fail.

And start small. You can try it on a single inbox, with what you’re already paying for, before anyone needs to make a single big decision. We started with our own.

And call your supplier on a Friday afternoon. They’re actually awake.

The source

The report is called "Measuring and Evaluating the Performance of Generative AI Models for Scam Detection" and was published on 19 July 2026. It’s free to read, and if you want to see the numbers yourself, you’ll find them here: arxiv.org/html/2607.17353v1