Home / Blogs

The Cycle of E-Mail Security

John Levine

Stepping back from the DMARC arguments, it occurs to me that there is a predictable cycle with every new e-mail security technology.

1. Invention and enthusiasm

Someone invents a new way to make e-mail more secure, call it SPF or DKIM or DMARC or (this month's mini-fiasco) PGP in DANE. Each scheme has a model of the way that mail works. For some subset of e-mail, the model works great, for other mail it works less great.

SPF works great for mail sent directly from the sender's server to the recipient, not for mail that is relayed or mail sent from a third party server (the usual example is newspaper send-an-article.) DKIM works great for mail sent or relayed from the sender's server, so-so for mail that is modified as it's relayed (mailing lists) or from third party servers. DMARC works great for much of the mail where SPF or DKIM works, not at all otherwise. In each case, the range of mail that each handles is a large fraction of all mail, but a large fraction is not the same as all.

2. Overselling the benefits and getting FUSSPy

The ball gets rolling, the enthusiasts see how great the scheme is, and how well it works for the mail that fits the model. Sometimes the enthusiasts are different from the inventors, and it is common for the enthusiasts not to understand the model very well. For example, DKIM is deliberately designed so that the domain in a DKIM signature need not match the From: or anything else, which provides a great deal of useful flexibility. But some enthusiasts insisted that "first party" signatures that matched the From:, or maybe the Sender:, were more legitimate than others.

The next step is to observe that if only all legitimate mail fit the model (either the real model or the enthusiasts' model) we could reject the rest and solve the spam problem, or in the case of DMARC, the phishing problem. Never mind that all mail doesn't fit any model, and no mechanical scheme will ever perfectly separate good mail from bad, the lure of a FUSSP is hard to resist.

3. Slandering mail that doesn't fit the model

Since mail that doesn't match the scheme's model is inconvenient, it must therefore also be bad. So the enthusiasts borrow derogatory terms to describe it, with "forged" being the most common. No, mailing lists do not forge mail from subscribers, and courtesy forwarders like computer.org do not forge the origin of the mail they resend.

When the enthusiasts start rejecting mail that doesn't fit the scheme's model, they lose a lot of perfectly good mail, but that of course is the fault of people sending forged mail.

Or they insist that people use clumsy workarounds like SRS which reinvented source routing (which SMTP mail ditched in the 1990s) to try and deal with limitations in SPF.

Or worse, they admit that the model has its limits, but the benefits of the scheme are so overwhelming that the collateral damage is a small cost to pay for progress and only a Luddite would object. Particularly since the cost is invariably paid by someone else.

4. A truce, if you're lucky

SPF and DKIM are pretty well understood now, and it's rare to see mail rejected due to an overstrict SPF policy. DKIM is well integrated into the mail world.

DMARC is still in the "small cost for progress" stage, but current work in the IETF DMARC working group suggests there may be hope yet.

By John Levine, Author, Consultant & Speaker
Follow CircleID on
Related topics: Cybersecurity, Email, Spam

If you are pressed for time ...

... this is for you. More and more professionals are choosing to publish critical posts on CircleID from all corners of the Internet industry. If you find it hard to keep up daily, consider subscribing to our weekly digest. We will provide you a convenient summary report once a week sent directly to your inbox. It's a quick and easy read.

I make a point of reading CircleID. There is no getting around the utility of knowing what thoughtful people are thinking and saying about our industry.

Vinton Cerf, Co-designer of the TCP/IP Protocols & the Architecture of the Internet

Share your comments

The .EMAIL new gTLD Jean Guillon  –  Jun 16, 2015 12:12 PM PDT

Couldn't the applicant have improved email security "at the source" launching the .EMAIL new generic Top-Level Domain?

No. But thanks for asking. John Levine  –  Jun 16, 2015 12:30 PM PDT

No. But thanks for asking.

They're trying to answer the question "Did Todd Knarr  –  Jun 16, 2015 10:48 PM PDT

They're trying to answer the question "Did this email come from the person it claims to have come from?" by looking at the servers used to send it. That's never going to work. It can tell you that the answer is "Yes." with certainty (modulo client compromise), but it can never tell you when the answer is "No.". To completely solve the problem you need a digital signature on the message associated with the sender's identity, not the server's. OTOH if the security enthusiasts accept that in the cases they're interested in (spam) they're going to be dealing with a probability then SPF, DKIM and such are good for winnowing out the presumably-known-good cases and cutting down the risk of false positives from the probability-assessment stage of the spam filters.

What's funny is the technical side of this is solvable, in fact it has been solved in browsers (go look at how StartSSL handles user authentication, and PGP's network of key servers). It's the user side that's the problem when they hit the "OK, now I've got my key, how do I get it into my other email programs?" step. With standalone clients I can answer that with "If the File Save and File Open dialogs are too complex for you, you shouldn't be allowed at the keyboard unsupervised anyway.". Web-based clients like Gmail I have no solution beyond "Give them your private key and pray they're honest and never get compromised." which isn't really a good solution.

Beware the FUSSP John Levine  –  Jun 17, 2015 9:49 AM PDT

You appear to be assuming that if everyone sending mail had a signing certificate, mail would be easier to filter. That is obviously not true, since the vast majority of people in the world haven't sent enough mail to have a reputation one way or the other. The other problem of course is that distributing and managing large numbers of keys is an unsolved and probably insoluble problem.

It would be easier to filter, but Todd Knarr  –  Jun 18, 2015 9:07 AM PDT

It would be easier to filter, but not because of the sender's reputation. Spammers for obvious reasons try to avoid sending mail using their own identity, that would make it easy to send spam. The signatures though would provide an easy way to tell when the mail wasn't from the address it claimed to be from, which would seriously hamstring the spammers. To avoid easy filtering they'd have to generate a very large number of false identities on a constant basis, which complicates the spammers' logistics and leaves a trail that can be identified (and CAs that allow it can be flagged in the same way we flag ISPs that don't do anything about spammers). So without any reputation it's still easy to separate mail into "comes from who it says it does" (may be spam, but at least you won't be chasing a fast-moving source), "sender's forging the source" (if not spam it's probably unwanted for other reasons), "sender vouched for by someone who vouches for known spammers" and "not signed".

The technical problem left is the webmail case, and I think it'd be useful in general for browsers to have a "the site would like the user's signature on this form data" capability just like they have a "the site would like you to generate a key pair and CSR" capability.

Really, it's not a FUSSP John Levine  –  Jun 18, 2015 9:24 AM PDT

Spammers steal user accounts and hijack web servers at stupendous rates. That means that they steal the credentials along with the account, so authentication doesn't help.

There are specific phishing issues against high visibility targets like Paypal, which is what DMARC addresses pretty well. (It also does other things badly.) Paypal already strongly authenticates all of its real mail, so it's easy to recognize mail that purports to be from Paypal but isn't. The hard part is to deal with PayPai.com and other things that resemble famous names enough to fool people.

By the way John Levine  –  Jun 18, 2015 9:25 AM PDT

Spammers already generate a very large number of false identities on a constant basis. You might want to bring your knowledge of the spam world up to date.

To post comments, please login or create an account.



IP Addressing

Sponsored byAvenue4 LLC


Sponsored byVerisign

New TLDs

Sponsored byAfilias

Domain Names

Sponsored byVerisign

DNS Security

Sponsored byAfilias