Home / Blogs

A Few Thoughts on the Future of Email Authentication

Michael Hammer

With the Online Trust Alliance Town Hall Meeting and Email Authentication Roundtable next week as well as the RSA Conference, I decided to pause and think about where we are and where we might be headed with regard to email authentication.

Over the years, many of us have collectively worked to provide a framework for authenticating email. At the transport layer we now have SPF providing a linkage between the RFC5321 "Mail From" address and the originating IP address. At the message layer DKIM allows domains to sign and take responsibility for messages. In combination with ADSP this allows a linkage between the RFC5322 "From" address and the domain it purports to be from.

Unfortunately somewhere on the path to protecting legitimate originators of email and the recipients of email, the larger community has gone astray. Too many domains have chosen a weak implementation (~ or ?) of SPF in their published records or have misunderstood what a strong assertion (-all) means. This has led to some reticence on the part of receiving operators to act in a meaningful and aggressive way, particularly with respect to -all assertions. In addition, many domain owners and administrators are simply unaware of SPF.

In the DKIM space too many signers are signing because they believe that the simple act of signing will help deliverability regardless of other issues that may be associated with their mail streams. This is the tail wagging the dog. DKIM/ADSP simply identifies mail as theirs but says nothing about the qualities of the mail stream. In addition, the ADSP (Author Domain Signing Policy) draft has been bogged down and it is not clear how or whether receiving operators will act on "all" or "discardable" assertions on the part of signers once a stable draft is achieved. It should be remembered that they are the ones most likely to field the complaints if legitimate mail does not reach the intended recipient.

Finally, there is a gaping hole in that what is presented to endusers by the Mail User Agent is the RFC5322 display name, which is not tied to anything and provides an easy mechanism for malicious individuals to bypass both SPF and DKIM/ADSP and engage in phishing abuse or SPAM. While it is still unclear which presentation techniques are successful in assisting endusers to distinguish potential problem emails, steps need to be taken to address the display name problem.

Those of us active in this space need to recognize that if we do not start moving forward in addressing the abuse issues in messaging, government regulators will move us forward and not necessarily in ways that best address the issues in this space. One need only watch the webcast of the recent Congressional Committee on Homeland Security hearings on whether PCI reduces cybercrime. Given that email is a significant vector for cybercrime can a focus on messaging be far behind?

This is not simply about protecting brands, companies or deliverability. It is about protecting endusers, their personal computers, the organizations they work for and society as a whole.

Based on my experience in dealing with phishing abuse, I can state with confidence that a combination of strong SPF assertions and a publicly known practice of DKIM signing ALL mail for a heavily phished domain can reduce the success of phishing attacks by as much as 85% if receiving operators are checking and acting on these assertions.

The gap involving RFC5322 Display Names can be addressed by a change in the way that MUAs display mail to end recipients. If mail is unauthenticated using strong assertions then the MUA should display the RFC5322 email address and NOT the Display Name.

One further step for receiving operators and MUA providers to consider would be to flag unauthenticated mail as potentially risky. Such a step will certainly take time and much discussion.

This would still leave the issue of cousin domains to be addressed. I leave this topic for another day.

There are many who would decry the changes outlined above.

Some would claim that anonymity would be lost. This is simply not true. As long as some organization is willing to take responsibility for the messages I send as an individual, I need not disclose who I am individually. To the extent that the organization taking responsibility for those messages is recognized as not being abusive, other organizations are likely to accept the message I created and injected into the mail stream without my having to disclose my individual identity.

Others argue that the risk of lost mail is too great. I again look to my experience. The increased risk of breakage, other than from implementation mistakes primarily on the part of senders, is actually quite low. To senders who are not paying attention — fix your broken implementations. There are special cases such as mailing lists but various solutions have been offered to address these special cases.

Finally, there are those who insist that differentiation between authenticated and unauthenticated mail penalizes them if they choose not to authenticate. My response to them is that I am simply not sympathetic to their choice. While they may be correct, the internet is no longer a safe small town anymore.

There is a small cadre of senders that have already moved to implement strong assertions and authentication. These are primarily operators of domains subject to phishing abuse. There is a small cadre of large ISPs that have started moving to act on strong assertions and authentication. Some would argue that strong assertions and authentication will only be applicable to a small subset of the mail stream. I would counter that as additional information becomes publicly available there will be ever increasing adoption on the part of both senders and receivers.

So how do we move forward from the status quo?

The first step is for the senders and ISPs referenced above to go public with information on the extent to which strong assertion and authentication is successful in addressing abuse. It should be remembered however that strong assertions and authentication are tools rather than magic bullets.

There also needs to be data on the nature and extent of breakage associated with strong assertions and authentication. For example, Mark Martinec recently posted to IETF-DKIM an interesting list of breakage types (legitimate email) that he has seen in association with DKIM signing. I recently posted a specific example involving message-ids to CircleID. This is the type of information that implementers need in order to minimize the risks and move forward with confidence.

Another step is for vendors of software and appliances to change their default configurations so that strong assertions and authentication by senders are respected out of the box. While I would prefer rejection (SPF) or discard (ADSP assertion) of messages, even quarantine would be a tremendous step forward. Such a step might be facilitated by an organization such as MAAWG or APWG.

Finally, providers of Mail User Agents (MUAs) need to address the RFC5322 From Display Name gap by working on new ways to highlight to the end user the difference between authenticated and unauthenticated email and only displaying the From email address instead of Display Name for unauthenticated mail.

These changes on the part of vendors, coupled with stronger implementations by ISPs and MUA providers can create an incentive for additional senders to investigate and implement strong assertions and authentication. The resulting virtuous cycle in conjunction with the other changes recommended above should have a significant positive impact in reducing mail abuse over the coming years.

Follow CircleID on
SHARE THIS POST

If you are pressed for time ...

... this is for you. More and more professionals are choosing to publish critical posts on CircleID from all corners of the Internet industry. If you find it hard to keep up daily, consider subscribing to our weekly digest. We will provide you a convenient summary report once a week sent directly to your inbox. It's a quick and easy read.

I make a point of reading CircleID. There is no getting around the utility of knowing what thoughtful people are thinking and saying about our industry.

Vinton Cerf, Co-designer of the TCP/IP Protocols & the Architecture of the Internet

Share your comments

The incentives are all wrong David MacQuigg  –  Apr 16, 2009 3:35 PM PST

The reason we don't have universal authentication of legitimate senders is not technical difficulties, but the fact that industry likes the status quo.  That includes the $2B per year anti-spam industry, the network providers whose mailflow is 90% spam, and the ESPs, large and small, who have built private authentication/reputation systems, and don't want to share their "competitive advantage" with the world.  If spam goes away, this industry loses money.

What we need is a system that works in spite of industry opposition.  It cannot depend on various groups doing what they "should" do.  We have been hearing that talk for too many years.  Participants in an email transaction will only do what is in their immediate best interest, and "proselytizing" won't change that.  Regulation might, but governments will probably get it wrong, especially if the regulators are influenced by these same industry groups.  The best role for government is not regulatory, but offering a public service that industry won't provide.

For senders, the incentive to offer authentication is small, so the cost must be even smaller.  The cost of publishing a strong SPF record (worry about lost mail) has proven too high.  DKIM may also prove too costly.

Receivers may tolerate a larger cost, but there are limits on that side also.  Enforcement of SPF policies may result in complaints from their own customers. Running DKIM on every message could be a significant cost if we continue to have 90% of the messages being spam.

If a solution emerges from this mess, it will probably be like the emergence of SMTP in the early 80's.  The large ESPs kept their proprietary systems as long as they could, claiming "technical difficulties" in exchanging messages with other ESPs, but eventually the aggregation of smaller domains using SMTP was larger than any one ESP, and the big ones had to join in.  In the early 80's there was a compelling need for subscribers of different ESPs to exchange messages.  Today, there does not seem to be any compelling need to get rid of spam.

I've got a bridge I'd like to sell you, David .. Suresh Ramasubramanian  –  Apr 18, 2009 8:09 PM PST

You are confusing authentication with reputation, I'm afraid.

Guess who are the single largest users and most enthusiastic adopters of spf, dkim etc?  High volume unsolicited bulk mailers of the sort that spamhaus calls snowshoers. For throwaway domains they setup in a /24 that they can keep for a few weeks at most before they get shut down, and the throwaway domains dumped, a fresh set of domains provisioned.

Authentication doesnt work to stop spam, you know .. it merely tells you email from X is actually from X.  My favorite analogy is a movie theater - the marquee out front says 'Hannah Montana', the ticket stub you get says 'Hannah Montana' .. so quite likely you ARE seeing Hannah Montana, not ''Girls Gone Wild'

Now reputation is whether you think Hannah Montana is a good movie or not.  The authentication that it is Hannah Montana sort of helps, but there are multiple other factors you can authenticate on - starting from the good old IP and domain name ..

In other words authentication is not a cureall for spam.  I remember when SPF was being hyped up that way .. as a cure for everything from spam and phishing all the way to the common cold.  That's as good a way as any to ensure that spf didnt get far at all.

I am a supporter of dkim - it fulfils its authentication role with far less pitfalls than spf had .. but one thing I will tell you now is that its not some sort of silver bullet that'll kill spam dead.

Let's not argue over which authentication method is best. David MacQuigg  –  Apr 21, 2009 6:43 AM PST

I am a supporter of dkim - it fulfils its authentication role with far less pitfalls than spf had

We should support both DKIM and SPF, and any other methods a sender may chose to offer.  If the alleged deficiencies of any method are true, it will lower the reputation of the sender, and motivate that sender to offer a better method.  When google.com tells me in an SPF record to use the PTR method, I don't argue with them.  I do exactly as they request.  The problem is, they still deny responsibility for the spam coming from a transmitter that passes their PTR test.

Based on the amount of spam we have received from Google's transmitters (not our opinion of their methods), we have lowered their rating from A to B.  That means most of their messages go to quarantine, not straight to our recipient's inboxes.

It won't do any good to explain the deficiencies of the PTR method.  What we need is a lot more receivers doing what I do.  Send them your spam reports, but don't accept any excuses.  If they want their A-rating back, they need to do what other large domains do.

Suresh, if you had read the first paragraph of my comment David MacQuigg  –  Apr 19, 2009 4:53 PM PST

you would see I am including reputation as part of the systems I consider successful.  I am not looking for a "silver bullet" that will "kill spam dead".  Spam will continue in one form or another as long as there are recipients who don't mind receiving it.  For those folks, it's probably better to leave things as is.  The obvious scams are a daily reminder that their computers are not secure, a sort of "immunization" for idiots.

More realistic goals are 1) Reduce the number of lost messages for recipients that care about reliable email, and 2) Reduce the number of botnets large enough to pose a DoS threat.  To reach these goals, we need essentially all legitimate senders to offer unambiguous authentication.  Proselytizing hasn't worked.  We need to lower cost of authentication (real or perceived) and provide at least a small immediate benefit.

A Few More Thoughts on Email Authentication… errr… Trust Dave Crocker  –  Apr 29, 2009 10:05 PM PST

Here's the link to my article, somewhat in response to Mike's article.

A Few More Thoughts on Email Authentication… errr… Trust

/d

To post comments, please login or create an account.

Related

Topics

New TLDs

Sponsored byAfilias

DNS Security

Sponsored byAfilias

Cybersecurity

Sponsored byVerisign

Cybercrime

Sponsored byThreat Intelligence Platform

Domain Names

Sponsored byVerisign

Whois

Sponsored byWhoisXML API

IP Addressing

Sponsored byAvenue4 LLC