Home / Blogs

Greylisting Still Works - Part II

John Levine

In my last post I blogged about greylisting, a well-known anti-spam technique for rejecting spam sent by botnets. When a mail server receives a an attempt to deliver mail from an IP address that's never sent mail before, it rejects the message with a "soft fail" error which tells the sender to try again later. Real mail senders always retry, badly written spamware often doesn't. I found that even though everyone knows about greylisting, about 2/3 of IPs don't successfully retry.

Another theory about greylisting is that if you defer mail from a new IP, by the time the sender retries, if it's sending spam it'll have hit spamtraps and been added to blacklists. I recently realized that I have enough log data to check that theory, so I collected some statistics for the past week, which is as long as I keep logs about mail connections from blacklisted hosts. The IPs I greylisted broke down like this:

CountPercent
No retry3,80335.8%
Retry too soon3,34531.5%
One retry1,18311.1%
More than one message1,63515.4%
Blacklisted5615.3%
Retried, blacklisted later890.8%
Total10,616100.0%

No retry and Retry too soon are senders that greylisting kept from sending anything, again, about 2/3 of mail. (My greylister requires that the sender wait at least a minute, since some spamware sends several messages a few seconds apart.)

The next two are senders that retried successfully and sent one message, or more than one message. (If a sender retries too soon, then retries again after more than a minute, it's counted in one of those two categories.) Blacklisted means that when the IP retried, the IP was on one of the a blacklists I use, in nearly all cases Spamhaus Zen. The last line is IPs that retried successfully, but were blacklisted when they tried to send other messages later.

The 5.3% for Blacklisted probably overstates how much mail was caught by waiting to see if an IP was blacklisted. My logs don't say whether the delivery attempt that was blacklisted was trying to deliver a message with the same To and From addresses, in which case it would have been delivered, or a different message, in which case it would just have been greylisted again. Spot checking shows IPs that were greylisted repeatedly, before appearing in a blacklist, which suggests that they were sending different messages.

Also, for the few IPs that were blacklisted later, they were generally blacklisted much later, hours or days later, far longer than any reasonable greylisting strategy would force mail to wait.

So greylisting still works, but it's almost entirely because spamware doesn't retry, not because it gets blacklisted.

By John Levine, Author, Consultant & Speaker. More blog posts from John Levine can also be read here.

Related topics: Email, Malware, Security, Spam

WEEKLY WRAP — Get CircleID's Weekly Summary Report by Email:

Comments

Measure the delay The Famous Brett Watson  –  Dec 09, 2011 7:41 PM PDT

This data would be more useful if you included statistics on the timing of sender retries. If the bulk of the successful retries are happening five minutes after the original connection, for example, then it would come as no surprise that the IP has not been added to a blacklist in that interval. The effectiveness of the technique is directly proportional to the overall delivery delay time, and you've provided no data on that front. Maybe you could write a "part three". I'm sure this data would be of general interest.

BCP Alessandro Vesely  –  Dec 16, 2011 10:24 AM PDT

Let me just note Murray's attempt at collecting useful hints on greylisting in a BCP draft (appsawg).

To post comments, please login or create an account.

Related Blogs

Related News

Topics

Industry Updates – Sponsored Posts

dotStrategy Selects Neustar's Registry Threat Mitigation Services for .BUZZ Registry

24 Million Home Routers Expose ISPs to Massive DNS-Based DDoS Attacks

What Does a DDoS Attack Look Like? (Watch First 3 Minutes of an Actual Attack)

Joining Forces to Advance Protection Against Growing Diversity of DDoS Attacks

Why Managed DNS Means Secure DNS

Rodney Joffe on Why DNS Has Become a Favorite Attack Vector

Motivated to Solve Problems at Verisign

Diversity, Openness and vBSDcon 2013

Neustar's Proposal for New gTLD Collision Risk Mitigation

IT Project Management: Best Practices in Small-Scale Engagements

DDoS Attacks in the United Kingdom: 2012 Annual Trends and Impact Survey

7 Keys to Professional Services Value: A Client-Side Perspective

Neustar Launches Global Partner Program

MarkMonitor Named a Top Trusted Website in OTA's 2013 Online Trust Honor Roll

Neustar Chief Technology Officer Appointed to FCC's Technological Advisory Council

A Look Inside Dyn's 1.2 Billion Monthly Email Delivery Statistics

Hope is Not a Strategy: Neustar Releases 2012 Annual DDoS Attack and Impact Survey

How Neustar Technology Can Help Mitigate DDoS Attacks

Dyn to Host Email Analytics Webinar With Ongage

Reducing the Risks of BYOD with Nominum's Security Solution

Sponsored Topics