Maybe the IETF Won't Publish SPF and Sender-ID as Experimental RFCs After All

By John Levine
John Levine

Yesterday, the IESG, the group that approves RFCs for publication received an appeal from Julian Mehnle to not to publish the Sender-ID spec as an experimental RFC due to technical defects. IESG members' responses were sympathetic to his concerns, so I'd say that a Sender-ID RFC has hit a roadblock.

The problem is simple: Although Sender-ID defines a new record type, called SPF 2.0, it also says that in the absence of a 2.0 record, it uses the older SPF1 record. Since SPF and Sender-ID can use the same records, if you publish an SPF record, you can't tell whether people are using it for SPF or Sender-ID. Ned Freed commented:

There is, after all, an "experiment" in "experimental". A conflict that stands a good chance of making it impossible to get useful results from the experiment is something that clearly needs to be resolved prior to publication.

Mehnle's suggested fix is to one line in the Sender-ID spec, to make it say that the older SPF1 record should only be used for SPF checks, not Sender-ID checks. Technically this is easy to do, probably only a few lines of code in anyone's Sender-ID implementation, but it could present a big political problem for Sender-ID advocates (that is, Microsoft.)

One of the arguments in favor of SPF and Sender-ID has been that vast numbers of people have published SPF records, so there must be a groundswell of support for them. What I think actually happened was that early in SPF's career, during the magic anti-spam silver bullet stage, lots of people published SPF records, and then forgot about them when they found that their spam didn't stop. When Sender-ID came along as a hybrid of SPF and Microsoft's Caller-ID, after lengthy discussion on the MARID list, they decided that since many Sender-ID records would have the same contents as the corresponding SPF records, Sender-ID would use the existing large set of SPF records. As a way to kick-start a package that you want to rush into use, it's not a bad idea. But as part of an experiment, which is what the IETF considers SPF and Sender-ID to be, it's a clear mistake.

The political and practical problem of separating SPF and Sender-ID records is that the number of SPF records that actively maintained is probably a tiny fraction of the total, and only those are likely to be republished as Sender-ID's SPF 2.0 records. Fans of SPF and Sender-ID are not eager to know what that fraction is, because if it's as small as we suspect, it puts the lie to claims of SPF's or Sender-ID's popularity.

Since this question only came up yesterday, nobody has yet had a chance to ask Microsoft if they're willing to change the spec, much less get an answer from them. One possibility, which I consider vanishingly unlikely, is that Microsoft agrees to the change to separate Sender-ID from SPF. A second is that the IESG publishes the Sender-ID spec anyway, since it's been sitting around for months and everyone already knows what it says. A third is that the authors prepare a version of the spec for the IETF different from Microsoft's, which is unlikely since the primary author works for Microsoft. And the last is that the authors say, oh, never mind, and withdraw it.

At this point, I'd say that it's somewhat likely they'll publish it as is, and next most likely that it'll be abandoned. The discussion in the IESG has been quite clear that they don't care what the change is so long as it's possible to experiment with SPF and Sender-ID separately, so it's possible that someone will come up with a fiddle that satisfies the IESG and is sufficiently face-saving for Microsoft, but it's hard to see what that could be, particularly with Microsoft flogging people (via Hotmail) to support the current version of Sender-ID immediately.

By John Levine, Author, Consultant & Speaker. Visit the blog maintained by John Levine here.

Related topics: Spam

Comments

Re: Maybe the IETF Won't Publish SPF and Sender-ID as Experimental RFCs After All wayne  –  Aug 25, 2005 7:35 PM PDT

Geez John, here you go again.  Lots of unsubstantiated claims against SPF.  You are the chair of the IRTF's Anti-Spam Research Group, why don't you do some research?

What I think actually happened was that early in SPF's career, during the magic anti-spam silver bullet stage, lots of people published SPF records, and then forgot about them when they found that their spam didn't stop.

Are you trying to claim that people have stopped publishing new SPF records in significant numbers?  When was this "magic anti-spam silver bullet stage"? 

Both Andy Newton and I have periodically done surveys of all domains in several TLDs (.com, .net, etc.).  We know the answer to the first question, and the answer is that many new SPF records are being published.  The data I have shows that the percentage of email covered by SPF records is increasing. 

The political and practical problem of separating SPF and Sender-ID records is that the number of SPF records that actively maintained is probably a tiny fraction of the total, and only those are likely to be republished as Sender-ID's SPF 2.0 records. Fans of SPF and Sender-ID are not eager to know what that fraction is, because if it's as small as we suspect, it puts the lie to claims of SPF's or Sender-ID's popularity.

I have researched into the number of SPF records that have been added, deleted and changed a couple of times over the last year or so.  That kind of discounts your claim that people in the SPF community aren't eager to know this. 

I know the answers to your speculation, but before I give out the latest data, I want to ask you "what do you consider to be 'small'"?

Also, remember that SPF records were designed so that they wouldn't require that much maintenance.  The many SPF records like "v=spf1 mx -all" are still going to be valid today, even if they have changed the MX records.

Since this question only came up yesterday, nobody has yet had a chance to ask Microsoft if they're willing to change the spec, much less get an answer from them.

Actually, none of what you have said is true.  There have been discussions between the SPF community, the IESG and Microsoft for months about this.  These discussions are easy to find.  I'll let you do the research to find them.  I'll give you a hint though:  If Microsoft had said they would change the spec, we wouldn't have felt the need to file an appeal now, would we?

I realize that you don't like SPF.  That's fine.  But your many claims over the months that SPF isn't being used, or that it is losing mindshare appears to me, based on actual data I have collected, to be just wishful thinking on your part. 

Re: Maybe the IETF Won't Publish SPF and Sender-ID as Experimental RFCs After All John Levine  –  Aug 25, 2005 7:45 PM PDT

My, aren't we touchy.  I expect that if Wayne had numbers that supported his arguments, he would tell us what they are, so I'll let people draw their own conclusions.

Re: Maybe the IETF Won't Publish SPF and Sender-ID as Experimental RFCs After All wayne  –  Aug 25, 2005 8:04 PM PDT

Yeah John, maybe I am being a little touchy.  I guess, as your position as the ASRG chair and such, I expect a lot more from you and for you to back up your claims.

As far as having numbers backing me up, I simply don't want to fall into the trap of saying "X% of the SPF records were updated between <date1> and <date2>", and then you claiming that "see!  X% is small", no matter what the value of X is.

The rest of the stuff I didn't research for you can be found by using google.  It would be pretty silly of me to claim that this stuff hadn't been discussed before unless I knew that it could be easily found.

Re: Maybe the IETF Won't Publish SPF and Sender-ID as Experimental RFCs After All William Leibzon  –  Aug 25, 2005 8:35 PM PDT

Note: this is a repost comments that originally were made on NANOG mail list, see http://www.merit.edu/mail.archives/nanog/2005-08/msg00891.html

Some corrections to what is said in John's article:
1. The appeal is against publication of SID draft (3 SID drafts, although only one is actually mentioned in appeal they go together) as experimental RFC(s) in current form. This does not imply that SPF draft would not be published by itself as article's title suggests (although it maybe delayed if there is another attempt to reconcile the differences). In article text John does not say that SPF draft would not be published because of this appeal, so I'm unsure why such a title…
2. The appeal is made to IETF Chair Brian Carpenter. According to IETF system he can choose to have appeal decided on by IESG or can choose to decide on it himself (in which case his decision can still be appealed to IESG). So far he has not said if IESG as a whole will consider the appeal. In either case, saying that IESG received this appeal is probably not quite correct (they were CCed however).
3. During MARID itself it was decided that new record version would be used (SPF2.0 prefix), which is opposite to what John says in the article about there being decision as part of MARID to reuse existing v=SPF1 records.
4. Nobody knows how many records had been published exclusively for SID because of Microsoft, probably a lot lot less then for SPF. But I don't think maintaining records is such a big deal (rather having to decide what goes into initial record is a lot more of a problem) and this probably would not be the reason why SPF2 SID records would not published if separation happens.
5. As far as last two paragraphs in the article, first of all appeal is being made after people already asked if MS is willing to make a change and they said no. And as far as solution to this that lets "Microsoft save face", it probably can involve using positive results of SID test with v=SPF1 record, but not negative results of such check unless its SPF2 record (which it not say that everyone at SPF "camp" would support this, but it is probably better then now). But I'd not be surprised if instead MS chooses to disregard IESG and IETF and proceed with SID even if its not approved for RFC, nor would I be surprised if IESG is afraid to say no to MS even when it knows this is bad engineering (which would further undermine believe in IETF as being neutral standards body focused on technical excellence and not controlled by large vendors).

Re: Maybe the IETF Won't Publish SPF and Sender-ID as Experimental RFCs After All John Levine  –  Aug 26, 2005 11:05 AM PDT

I thank William for confirming all of my important points.
In MARID they did invent a 2.0 record, but at the same time they decided to fall back to SPF1.

As far as Microsoft saying no, a headline saying "Microsoft blows off IETF" would gather more attention than "Microsoft blows off self-appointed guardians of SPF", so assuming the IESG agrees that there's a problem, MS might possibly want to reconsider.

And although the appeal was only against the Sender-ID draft, the problem is the interaction between the two, so if I were the IESG, I'd wait to see what the outcome was before publishing anything.

Re: Maybe the IETF Won't Publish SPF and Sender-ID as Experimental RFCs After All wayne  –  Aug 26, 2005 11:25 AM PDT

William wrote:

3. During MARID itself it was decided that new record version would be used (SPF2.0 prefix), which is opposite to what John says in the article about there being decision as part of MARID to reuse existing v=SPF1 records.

John replies:

I thank William for confirming all of my important points. In MARID they did invent a 2.0 record, but at the same time they decided to fall back to SPF1.

Sorry John, but again, William is right and you are wrong.  The whole purpose of creating the 2.0 record format in the IETF MARDI workgroup was so that there *wouldn't* be re-use or a fall back to SPFv1 records.