Home / Blogs

Challenges in Anti-Spam Efforts

Dave Crocker

Without commenting on the particulars as they relate to Goodmail — especially since I am on the advisory board for Habeas, a competitor — let me note that public discussion is largely missing the nature of the current Internet mail realities and the nature of the ways we can deal with them.

There are two articles in the current issue of the Internet Protocol Journal, of which I wrote one, that provide some useful background about this reality.

Simply put, Internet mail needs to sustain spontaneous communications — that is, communications without prior arrangement — and the benefit of such a capability is fundamental. However the scale and diversity of the modern Internet now includes many folk who the security geeks appropriately call Bad Actors. We are stuck with these competing points: Maintaining open contact, but dealing with some very nasty users.

A great deal of very good work has been done, to detect these bad actors and their bad messages. Often, that work is quite helpful. In spite of this the total amount of global spam and email abuse has yet not gone down. We must continue with efforts to detect and deal with Bad Actors, but there is a separate path that is at least as valuable:

We need methods for distinguishing Good Actors. Folks who are deemed "safe". In effect, we need a Trust Overlay for Internet mail, to permit differential handling of mail from these good actors.

In general terms, a trust overlay requires reliable and accurate identification of the actor and a means of assessing their goodness. In other words, authentication and reputation.

We are already pursuing a standard for message transit handling authentication, through Domain Keys Identified Mail (DKIM). There is discussion about various assessment standards for reputation and accreditation. Although DKIM is quite viable in its pre-standards form, there is no candidate for standardized reputation reporting.

With all of this as background, imagine that you are an online service that needs to ensure that a customer order confirmation, or an equivalent critical transaction message, is delivered to the customer. Then imagine that you are offered a means of safely and reliably identifying this specific class of mail, so that it receives differential handling. The incentives for a company to pay to ensure that delivery are substantial.

And that is what the recent announcement is about. It concerns a means of ensuring delivery of "transactional" mail. This is quite different from "marketing" mail and it is not in the least controversial.

I would greatly wish that the mechanisms used open standards, but the basic model of developing a trust-based overlay for Internet mail seems an essential enhancement.

By Dave Crocker, Consultant

Related topics: Email, Internet Protocol, Security, Spam

WEEKLY WRAP — Get CircleID's Weekly Summary Report by Email:

Comments

Re: Challenges in Anti-Spam Efforts Justin Mason  –  Feb 09, 2006 2:26 PM PST

Hm.  If you read the press, such as the ClickZ article, the NY Times, USA Today, and eWeek, there's absolutely no indication that this is intended only for transactional mail; in fact, the USA Today article discusses it in terms of 'advertisers' paying for stamps.


And that is what the recent announcement is about. It concerns a means of ensuring delivery of “transactional” mail. This is quite different from “marketing” mail and it is not in the least controversial.

'Transactional' mail may be one part of what they want to do, but it's certainly not all the announcement was about — at least, not going by what they're saying on the record.

Re: Challenges in Anti-Spam Efforts Dave Crocker  –  Feb 09, 2006 7:05 PM PST

Jason, et al,

Indeed the media reports have been impressively confusing.  And I gather that there is no archival, official press release to consult. So I am going by the follow-up postings from the AOL and Yahoo folks, on various anti-spam lists. 

Independent of the AOL/Yahoo intent, I was trying to underscore the distinctive nature of transactional mail, as well as just how uncontroversial it should be to use a third-party vetting service by the originators of such mail. 

/d

Re: Challenges in Anti-Spam Efforts Eric Thomas  –  Feb 09, 2006 10:45 PM PST

The original AOL announcement that started all this can be found here:

http://www.the-dma.org/cgi/dispnewsstand?article=4405

As you can see, it was not about transactional e-mail but, in practice, about bulk e-mail. Being on the AOL whitelist is a must if you send a lot of mail.

AOL eventually backed off - sort of, they claimed they never made that announcement in the first place. They aren't saying that the journalists lied, the party line seems to be that the announcement was an internal memo that should never have been sent to the press. That did not exactly help anyone sleep tighter at night. You can read a summary a:

http://www.webpronews.com/topnews/...

I think you'll find that most of the people who participated in the outcry don't have a problem with the concept of third-party vetting services per se. As one of the most vocal people, I will tell you up front that I think vetting is a great idea, provided that it is implemented using open standards to promote competition. And it doesn't need to be very complicated - a simple DKIM extension to locate the public key at the vetting company, rather than at the sender, would give you most of what is necessary, plus rapid implementation and deployment.

I would be happy to pay a reasonable fee for vetting. $10,000-40,000 per day is not a reasonable fee for my modest operation. I have a broad blend of customers, including many non-profits that cannot dream of raising that kind of money, let alone spending it on e-mail delivery. The vetting costs obviously do not come close to this figure, and this is why I want to see open standards and competition. In this context, I would applaud vetting. I do not want to sign a blank check, but within reason, I would see to it that LISTSERV became an early adopter.

For the record, I think a reasonable fee level is what we pay for SSL certificates, and I think this can be achieved with technology that either exists or can be developed rapidly.

Eric

Re: Challenges in Anti-Spam Efforts Suresh Ramasubramanian  –  Feb 10, 2006 7:40 AM PST

Eric, there's a minor problem here. A lack of understanding of terminology that AOL people used. Quoted well out of context, In a lobbying group where the people participating in it dont like spam filtering all that much, and see it as a hindrance in (as their former chief bob weintzen put it) emailing a coupon for Tide detergent to every household in America.

And then have an article based on wrong assumptions and misinterpreted facts pushed all around the media.  With mass hysteria about a big corporation stiffing people who want to mail their users.

The EFF hasn't helped at all with Cindy Cohn rambling on about how this is a way to "shake down" non commercial email list operators like Dave Farber, or Aunt Tilly sending out her christmas greetings, into buying a online stamp for whitelisting. I've got a detailed set of thoughts on this for circleid that should be posted shortly.

This post (on spam-l@peach.ease.lsoft.com) by Christine M Borgia, AOL's technical manager for antispam operations, should help explain some of the context.

srs

-------- Original Message --------
Subject: MISC: AOL, the Enhanced Whitelist, & CertifiedEmail
Date: Thu, 9 Feb 2006 18:26:01 -0500
From: Christine M. Borgia
To: SPAM-L@PEACH.EASE.LSOFT.COM

There have been several requests here for clarification from AOL. I hope I can address a lot of the concerns and confusion of the last week around the comments linking the Enhanced Whitelist to the introduction of CertifiedEmail.

1) At least one article made it sound like the AOL whitelist is being
changed. This is not true.  The AOL whitelist will continue as it exists today, free of charge.

2) We are making changes the EWL. Over time the EWL has grown beyond its original intent, which was to reward mailers who could meet stringent requirements over a period of time, with enhancements beyond the regular whitelist. As the EWL grew in it's scope, we began to see it abused. This happened often enough that it became an operational concern to us. Starting last fall we began a process of incremental changes to get the EWL back to what it should be.  This process will continue over the next few months. We anticipate being done with the tightening of the requirements by early summer. There is obviously some confusion that this schedule implied that the EWL was being discontinued.  This is not the intention.  The EWL is like any other operational tool that AOL and others utilize - we are constantly
looking to tune functionality, make them more effective, and ensure that they are serving the purpose for which they were originally intended.

3) CertifiedEmail is a fact at AOL and we believe it will be a success.
Contrary to the reports I've seen, we are not forcing anyone to pay for the inbox. We believe CertifiedEmail provides great benefits to the mailer and we believe it will benefit AOL members, giving them a level of trust in commercial mail they simply can't rely on now. There are many benefits of CertifiedEmail to commercial mailers above and beyond the current limitations of the EWL.

The goal of these changes is to provide the best member experience for
AOL users. We will always evaluate and tweak our processes to achieve
this goal.

Thanks,
Christine

Re: Challenges in Anti-Spam Efforts Dave Crocker  –  Feb 10, 2006 2:34 PM PST

Folks, the major problem here is continuing to focus on what AOL did or did not say.

Instead, the discussion should be about: the need for a trust overlay on email, what it should look like and how it should function.

Really, let's put our energy into something that will move us closer to a goal of distinguishing mail that is known to be acceptable from mail that might or might not be acceptable.

/d

ps. Sorry for all the bolding, but emphasis is exactly the current problem.

Re: Challenges in Anti-Spam Efforts Justin Mason  –  Feb 10, 2006 3:11 PM PST

'let’s put our energy into something that will move us closer to a goal of distinguishing mail that is known to be acceptable from mail that might or might not be acceptable.'

Agreed.  And I also agree that reputation services built on open standards, like DKIM, are a much better way to get there.

Re: Challenges in Anti-Spam Efforts Eric Thomas  –  Feb 10, 2006 5:43 PM PST

Dave,

The reason I made a concrete technical suggestion at such an early stage (extending DKIM to allow the public key to be at the vetting company rather than the sending company) is that I think we need to get moving as soon as possible. Building something on top of DKIM will save that much time.

For while I agree that what happened at AOL is of secondary importance, the existence of this AOL memo suggests that, at a minimum, there have been discussions about a rapid transition to a single-provider vetting service. Even if that was not the case, a lot of people have now been exposed to that idea as a result of the press articles, and it is likely being discussed in boardrooms. Some journalists see e-mail stamps as a done deal and are asking what service I think will be charged for next. I would like to tell them that we have an open standard alternative coming up, but I cannot honestly make this claim. The sooner we have something to show, the better.

Eric

Re: Challenges in Anti-Spam Efforts Suresh Ramasubramanian  –  Feb 10, 2006 8:07 PM PST

My article in circleid on this issue -
http://www.circleid.com/posts/...

Re: Challenges in Anti-Spam Efforts Dave Crocker  –  Feb 11, 2006 11:36 AM PST

Eric, et al,

The reason I made a concrete technical suggestion at such an early stage (extending DKIM to allow the public key to be at the vetting company rather than the sending company) is that I think we need to get moving as soon as possible. Building something on top of DKIM will save that much time.

Well!  Now we can get into something of real substance.  Thank you!

Although DKIM represents relatively mature, basic technology (version 3 of domainkeys), it is relatively new to the larger public forum.  Consequently, most energy has been spent on its basic characteristics and details.  Indeed, that is where the focus should remain, in order to make the quickest possible progress in producing an official standard. 

(However, please forgive me for making a plug about the pre-IETF version that is already deployed for production use.  Take a look at DKIM's site. Existing implementations are using draft-allman-core-01 as a common specification base.  From my experience, I will be quite surprised to see a standardized IETF version appear in as little as a year and not at all surprised if it requires as much as 3. In order to get benefit sooner than that, I am recommending the community use the pre-standards version.  It is the result of an extensive, multi-vendor effort and it is already in field use.)

Now to your own point:

There are different scenarios possible, for using DKIM.  It has some subtle characteristics that provide more flexibility than is obvious.  So in fact, the signature can be with a key from the vetting agency!

Here's how it could work:

The vetting agency assigns an originator a specific DKIM "selector".  A selector serves as a qualifier for the domain name being used for the signature — note that that name need not match the RFC2822.From or RFC2822.Sender fields. The qualifier is added to the domain name, during the DNS query, so that signing attributes can be specific to that selector.

So, a vetting service with a domain name of "vetting-example.com" could give a selector, such as example-orig, specific to a customer Example Originator. The originator would sign under the domain name and selector and recipients would query the DNS for signing information under the domain name:  example-orig.vetting-example.com.

A vetting agency can give different selectors to different customers and can retain real-time control over the DNS query results.

I believe this is a style of use that is a bit similar to the Goodmail model, but note that it is entirely optional and open, rather than proprietary, both in terms of technology and in terms of permitting multiple vetting services.

So, Eric, I believe that your thinking is not only valid, but that DKIM already can support that style of use!

/d

Re: Challenges in Anti-Spam Efforts Miles  –  Feb 12, 2006 1:10 PM PST

Even more options:
- the vetting agencies selector could be part of the RFC2822.From or RFC2822.Sender's domain path, but name served by the vetting agency:  vettingagency._domainkeys.domain.com, which would allow them to serve key.vettingagency._domainkeys.domain.com
- the vetting agency could publish a list of domain/selector pairs in a DNS zone just like they do for IP addresses today — for instance transactional.vettingagency.com

To post comments, please login or create an account.

Related Blogs

Related News

Topics

Industry Updates – Sponsored Posts

Q3 2014 DDoS Trends: Attacks Exceeding 10 Gbps on the Rise

3 Questions to Ask Your DNS Host About DDoS

Afilias Partners With Internet Society to Sponsor Deploy360 ION Conference Series Through 2016

Neustar to Build Multiple Tbps DDoS Mitigation Platform

The Latest Internet Plague: Random Subdomain Attacks

Digging Deep Into DNS Data Discloses Damaging Domains

New gTLDs and Best Practices for Domain Management Policies (Video)

Nominum Announces Future Ready DNS

Non-English "IDN Email" Addresses Are Finally Working!

New from Verisign Labs - Measuring Privacy Disclosures in URL Query Strings

DotConnectAfrica Delegates Attend the Kenya Internet Governance Forum

3 Questions to Ask Your DNS Host about Lowering DDoS Risks

Continuing to Work in the Public Interest

Verisign Named to the OTA's 2014 Online Trust Honor Roll

4 Minutes Vs. 4 Hours: A Responder Explains Emergency DDoS Mitigation

Dyn Acquires Internet Intelligence Company, Renesys

Tips to Address New FFIEC DDoS Requirements

Smokescreening: Data Theft Makes DDoS More Dangerous

dotStrategy Selects Neustar's Registry Threat Mitigation Services for .BUZZ Registry

24 Million Home Routers Expose ISPs to Massive DNS-Based DDoS Attacks

Sponsored Topics

Verisign

Security

Sponsored by
Verisign
dotMobi

Mobile

Sponsored by
dotMobi
Afilias

DNS Security

Sponsored by
Afilias
Minds + Machines

Top-Level Domains

Sponsored by
Minds + Machines