Buying or Selling IPv4 Addresses?

Watch this video to discover how ACCELR/8, a transformative trading platform developed by industry veterans Marc Lindsey and Janine Goodman, enables organizations to buy or sell IPv4 blocks as small as /20s.

Avenue4 LLCRead Message Promoted Post

Home / Blogs

Greylisting Still Works - Part II

John Levine

In my last post I blogged about greylisting, a well-known anti-spam technique for rejecting spam sent by botnets. When a mail server receives a an attempt to deliver mail from an IP address that's never sent mail before, it rejects the message with a "soft fail" error which tells the sender to try again later. Real mail senders always retry, badly written spamware often doesn't. I found that even though everyone knows about greylisting, about 2/3 of IPs don't successfully retry.

Another theory about greylisting is that if you defer mail from a new IP, by the time the sender retries, if it's sending spam it'll have hit spamtraps and been added to blacklists. I recently realized that I have enough log data to check that theory, so I collected some statistics for the past week, which is as long as I keep logs about mail connections from blacklisted hosts. The IPs I greylisted broke down like this:

CountPercent
No retry3,80335.8%
Retry too soon3,34531.5%
One retry1,18311.1%
More than one message1,63515.4%
Blacklisted5615.3%
Retried, blacklisted later890.8%
Total10,616100.0%

No retry and Retry too soon are senders that greylisting kept from sending anything, again, about 2/3 of mail. (My greylister requires that the sender wait at least a minute, since some spamware sends several messages a few seconds apart.)

The next two are senders that retried successfully and sent one message, or more than one message. (If a sender retries too soon, then retries again after more than a minute, it's counted in one of those two categories.) Blacklisted means that when the IP retried, the IP was on one of the a blacklists I use, in nearly all cases Spamhaus Zen. The last line is IPs that retried successfully, but were blacklisted when they tried to send other messages later.

The 5.3% for Blacklisted probably overstates how much mail was caught by waiting to see if an IP was blacklisted. My logs don't say whether the delivery attempt that was blacklisted was trying to deliver a message with the same To and From addresses, in which case it would have been delivered, or a different message, in which case it would just have been greylisted again. Spot checking shows IPs that were greylisted repeatedly, before appearing in a blacklist, which suggests that they were sending different messages.

Also, for the few IPs that were blacklisted later, they were generally blacklisted much later, hours or days later, far longer than any reasonable greylisting strategy would force mail to wait.

So greylisting still works, but it's almost entirely because spamware doesn't retry, not because it gets blacklisted.

By John Levine, Author, Consultant & Speaker
Related topics: Cybersecurity, Email, Malware, Spam
SHARE THIS POST

If you are pressed for time ...

... this is for you. More and more professionals are choosing to publish critical posts on CircleID from all corners of the Internet industry. If you find it hard to keep up daily, consider subscribing to our weekly digest. We will provide you a convenient summary report once a week sent directly to your inbox. It's a quick and easy read.

I make a point of reading CircleID. There is no getting around the utility of knowing what thoughtful people are thinking and saying about our industry.

Vinton Cerf, Co-designer of the TCP/IP Protocols & the Architecture of the Internet

Share your comments

Measure the delay The Famous Brett Watson  –  Dec 09, 2011 6:41 PM PST

This data would be more useful if you included statistics on the timing of sender retries. If the bulk of the successful retries are happening five minutes after the original connection, for example, then it would come as no surprise that the IP has not been added to a blacklist in that interval. The effectiveness of the technique is directly proportional to the overall delivery delay time, and you've provided no data on that front. Maybe you could write a "part three". I'm sure this data would be of general interest.

BCP Alessandro Vesely  –  Dec 16, 2011 9:24 AM PST

Let me just note Murray's attempt at collecting useful hints on greylisting in a BCP draft (appsawg).

To post comments, please login or create an account.

Related

Topics

DNS Security

Sponsored byAfilias

Mobile Internet

Sponsored byAfilias

Cybersecurity

Sponsored byVerisign

IP Addressing

Sponsored byAvenue4 LLC

Promoted Post

Buying or Selling IPv4 Addresses?

Watch this video to discover how ACCELR/8, a transformative trading platform developed by industry veterans Marc Lindsey and Janine Goodman, enables organizations to buy or sell IPv4 blocks as small as /20s.