Home / Blogs

An Interview with the Lead Developer of SPF - Part I

Don't miss a thing – sign up for CircleID Weekly Wrap newsletter delivered to your inbox once a week.

CircleID recently interviewed Meng Weng Wong, the lead developer of Sender Policy Framework (SPF) and founder of Pobox.com. As one of the leading anti spam authentication schemes, SPF is used by companies such as AOL, Earthlink, SAP and supported by anti spam companies such as Sophos, Symantec, Brightmail, IronPort, Ciphertrust, MailArmory, MailFrontier, Roaring Penguin Software, and Communigate Pro. Last month, Microsoft announced its agreement to merge Caller ID, its own proposed anti spam authentication scheme, with SPF — the joint standard is called 'Sender ID'.

In this two-part interview, Meng Wong explains how SPF got started, where it is today and what could be expected in the future of email.

CircleID: A lot of attention has been paid to Microsoft's recent announcement about an agreement to combine Caller ID, its anti-spam authentication method, with Sender Policy Framework (SPF), another popular anti-spam authentication scheme. Can you share with us some background about your involvement with SPF?

Meng Wong: In 2002 Paul Vixie, the brains behind BIND, wrote a short paper titled "Repudiating Mail-From”. That inspired two other proposals, "Reverse MX” by Hadmut Danisch and "Designated Mailer Protocol” by Gordon Fecyk. In late 2003 I combined the best of both proposals and called the result "SPF". (I picked the acronym first and decided what it stood for second, which is why old references call it Sender Permitted From. Now it means Sender Policy Framework, which is a touch more accurate. Yes, references to sunblock are intentional.)

I presented the SPF concept at YAPC and (O'Reilly Open Source Convention) OSCON in 2003. Audiences liked the idea, and the mailing list kept growing. In November 2003 at the Hackers Conference I met Mark Lentczner, my co-author and an absolute genius at protocol design. During the conference and in the following weeks we rewrote the spec together. Eric Raymond talked about SPF at the MIT Spam Conference in January and I presented it before the IETF in Seoul in March.

My role in the SPF community is Chief Communicator: I run the website and go to conferences to spread the word. I also wrote Mail::SPF::Query, the reference library.

CircleID: Who is using SPF today and what level of success has it achieved in the fight against spam?

Meng Wong: In the six months since we declared a spec freeze, 20,000 domains have self-reported at the online registry; that's probably a fraction of the true total. Lots of domains have published records, including AOL, Amazon, Google, O'Reilly, SAP, TicketMaster, Mail.com, w3.org, Earthlink and Verizon. And the ones who haven't published are working on it.

We expect adoption to pick up exponentially; according to some estimates, the number of sites checking SPF doubles every three weeks. SPF plugins are available for all the major Mail Transfer Agents (MTAs).

CircleID: SPF, as we understand it, is aimed at addressing a fundamental weakness in Simple Mail Transfer Protocol (SMTP). Could you give us some insight on how this anti-spam authentication works? How does it differ from that of Microsoft's Caller ID?

Meng Wong: Suppose you get a letter in the mail. The return address on the envelope says "Amazon.com". You know that Amazon is based in Seattle, so if the postmark says "Seattle, WA" you could believe it's really from Amazon. But what if the postmark says "Nigeria"? That's suspicious.

The key insight in Vixie's original paper was this: domains publish MX records to tell senders where to send incoming mail. Domains can also publish records to tell receivers where to expect outgoing mail to come from. This kind of thing is generally called a Designated Sender Scheme, because domains designate their outbound mail servers. RMX, DMP, SPF, and CallerID are all Designated Sender Schemes.

Here's how it works:

Suppose example.com receives mail at and When it sends mail, it uses those two servers, as well as Example.com would publish an SPF record that said: "v=spf1 ip4: ip4: ip4: -all".

When a mail server gets mail that claims to be from someone at example.com, that server can fetch example.com's SPF record and see if the connecting SMTP client is designated.

There are a number of shorthands available: for example, "v=spf1 mx" means "I send mail from my MX servers".

Vixie's paper talked about running the authentication check against the domain of the MAIL FROM parameter. Its descendants RMX, DMP, and SPF all use the return-path identity.

Some of the proposals that were presented to the IETF use different identities: instead of the return-path, those proposals attempt to authenticate the HELO argument, or the PTR of the client IP address. The SPF community has recently developed a Unified SPF model that embraces all of the above semantics.

CallerID defines an algorithm to extract from the headers what it calls a Purported Responsible Address. The PRA starts out as the "From:" header, but if there's a "Sender" header present, "Sender" will override. If "Resent-From" is present, that overrides Sender. The algorithm was developed empirically; it is meant to pull out the last logical hop from the headers. If a message was forwarded through a service like acm.org or pobox.com, the PRA should be the alias at the forwarding service that was responsible for causing the final delivery.

SenderID applies SPF checks against the PRA. SPF Classic applies SPF checks against the return-path. In both cases, though, the DNS record that's used in the checks is generally in SPF format. That's why I sometimes talk about "SPF records" and "SPF checks", even when the authenticated identity is different from the return-path.

CircleID: What are the advantages of having the two methods merged? What are the key measures that are required in order to make this collaboration successful?

Meng Wong: In May, Microsoft agreed to adopt SPF's syntax and semantics for their Caller ID proposal. The result, which we're calling SenderID, applies SPF checks to the Purported Responsible Address from the message headers. I believe it represents the best of both worlds. The best thing about the CallerID proposal was the use of XML, which provides a path to future extensibility. SenderID uses a dual-syntax model: because the core semantics are drawn from SPF but are expressed in XML, SenderID clients will be able to parse both XML and regular SPF records. This means that if, in the future, the SPF syntax runs out of steam, and we need to say things that can't be said in the SPF language, we can upgrade to XML with a minimum of pain.

Meanwhile, SPF Classic, which applies SPF checks to the return-path, is still going strong. It is more mature and is better understood; it continues to enjoy a lot of support in the open community. Many people on the MARID mailing list have argued that SPF Classic and SenderID can and should proceed in parallel, especially since they both use the exact same records but can be used for two mutually compatible purposes.

CircleID: You have been quoted in the media, saying:

"In the last six months or so, there's been a fair amount of uncertainty (about sender authentication). These are two very similar proposals and do many of the same things. The big players have been telling us 'When you get your story straight, we can go ahead,' but until that happens, people have been waiting to see what happens"

What sorts of issues have caused the above mentioned uncertainties, particularly among the"big players"?

Meng Wong: The big players wanted CallerID and SPF to converge because they were both designated sender schemes. It didn't make sense to have two different syntaxes and two different semantics for two different identities, because receivers would have to code up two sets of functions that do basically the same thing, and publishers would have to publish two records that say the same thing. Now that Microsoft has committed to remain backward-compatible with the SPF syntax, publishers don't have to worry about publishing both, and receivers don't have to worry about parsing both.

Read Part II of this Interview.

By CircleID Reporter

Related topics: DNS, Email, Security, Spam



Re: An Interview with the Lead Developer of SPF - Part I Tal Golan  –  Jun 29, 2004 10:59 AM PDT

Thank you for sharing an outstanding interview with Meng Wong. I very much look forward to reading Part II and beyond.

Meng's work and ability to garner support within the industry is amazing. While SPF alone does not solve the problem, it is without question the best step forward, from a global infrastructure standpoint, that has been made to date. I applaud Meng for accomplishing a feat that many thought was impossible.

I would like to add that Sendio, Inc. (http://www.sendio.com), provider of the world's only licensed Sender Address Verification (SAV) anti-spam appliance, has incorporated SPF. In the future, please include Sendio in your list of companies in support of SPF.

While the jury is still out on the impact that SPF alone will have on the problem of spam, we are confident that with the addition of the SPF component, Sender Address Verification (SAV) is without question the most effective solution to stopping spam available today.

Re: An Interview with the Lead Developer of SPF - Part I Paul Vixie  –  Jun 30, 2004 7:41 AM PDT

I'm surprised and pleased to learn that the SPF folks were aware of my work (MAIL-FROM) in this area. I would like to correct two misimpressions, and share an observation.

First, MAIL-FROM always used what Meng Weng calls the "return path", see section 2.3 of my proposal where it refers to RFC2821 (which describes the envelope, not the header) and the pseudo-code which refers to "MAIL FROM:<>" which is an envelope ("return path") artifact. In fact, the very name of my proposal ("MAIL-FROM") pertains to the SMTP verb (MAIL FROM) used to express what Meng Weng calls the "return-path identity".

Second, this idea is not original with me. I finally wrote it up and identified the corner cases in 2002, because I was cleaning out my inbox and found mail which Jim Miller had sent me describing the basic idea in 1998. I very much wish that I had jumped on this in 1998 when I first heard about it, and that the community had implemented this before Y2K, so that e-mail could be nearly forgery-free by now. Instead, the IETF ASRG is still trying to merge the merged CallerID/SPF proposal with some other competing work, and the final result is very much an open issue.

Finally, an observation. After co-founding MAPS and turning that crank for a number of years, I know that spam is "like a drug" in that people will go to almost any lengths, no matter how absurd, to send more of it. No "designated sender scheme" will ever be able to cut down the amount of spam that's sent, or received. All it can do is help domain holders avoid the brand dilution of having their domain name forged by spammers. This is a valuable contribution, but we must make it clear that none of these schemes will stop or even slow spam, and that their benefits accrue to domain holders, not to spam recipients.

To post comments, please login or create an account.

Related Blogs

Related News

Explore Topics

Sponsored Topics

Promoted Posts

Now Is the Time for .eco

.eco launches globally at 16:00 UTC on April 25, 2017, when domains will be available on a first-come, first-serve basis. .eco is for businesses, non-profits and people committed to positive change for the planet. See list of registrars offering .eco more»

Boston Ivy Gets Competitive With Its TLDs, Offers Registrars New Wholesale Pricing

With a mission to make its top-level domains available to the broadest market possible, Boston Ivy has permanently reduced its registration, renewal and transfer prices for .Broker, .Forex, .Markets and .Trading. more»

Industry Updates – Sponsored Posts

Leading Internet Associations Strengthen Cooperation

Global Domain Name Registrations Reach 329.3 Million, 2.3 Million Growth in Last Quarter of 2016

Verisign Releases Q4 2016 DDoS Trends Report: 167% Increase in Average Peak Attack from 2015 to 2016

Neustar to be Acquired by Private Investment Group Led by Golden Gate Capital

Verisign Q3 2016 DDoS Trends Report: User Datagram Protocol (UDP) Flood Attacks Continue to Dominate

2016 U.S. Election: An Internet Forecast

Government Guidance for Email Authentication Has Arrived in USA and UK

ValiMail Raises $12M for Its Email Authentication Service

Don't Gamble With Your DNS

Defending Against Layer 7 DDoS Attacks

Understanding the Risks of the Dark Web

New TLD? Make Sure It's Secure

Verisign Releases Q2 2016 DDoS Trends Report - Layer 7 DDoS Attacks a Growing Trend

How Savvy DDoS Attackers Are Using DNSSEC Against Us

Radix Adds Dyn as a DNS Service Provider

Facilitating a Trusted Web Space for Financial Service Professionals

MarkMonitor Partners with CYREN to Deepen Visibility into Global Phishing Attacks

Verisign Named to the Online Trust Alliance's 2016 Honor Roll

Port25 Announces Release of PowerMTA V4.5r5

Dyn Partners with the Internet Systems Consortium to Host Global F-Root Nameservers