Home / Blogs

Greylisting Still Works - Part II

John Levine

In my last post I blogged about greylisting, a well-known anti-spam technique for rejecting spam sent by botnets. When a mail server receives a an attempt to deliver mail from an IP address that's never sent mail before, it rejects the message with a "soft fail" error which tells the sender to try again later. Real mail senders always retry, badly written spamware often doesn't. I found that even though everyone knows about greylisting, about 2/3 of IPs don't successfully retry.

Another theory about greylisting is that if you defer mail from a new IP, by the time the sender retries, if it's sending spam it'll have hit spamtraps and been added to blacklists. I recently realized that I have enough log data to check that theory, so I collected some statistics for the past week, which is as long as I keep logs about mail connections from blacklisted hosts. The IPs I greylisted broke down like this:

CountPercent
No retry3,80335.8%
Retry too soon3,34531.5%
One retry1,18311.1%
More than one message1,63515.4%
Blacklisted5615.3%
Retried, blacklisted later890.8%
Total10,616100.0%

No retry and Retry too soon are senders that greylisting kept from sending anything, again, about 2/3 of mail. (My greylister requires that the sender wait at least a minute, since some spamware sends several messages a few seconds apart.)

The next two are senders that retried successfully and sent one message, or more than one message. (If a sender retries too soon, then retries again after more than a minute, it's counted in one of those two categories.) Blacklisted means that when the IP retried, the IP was on one of the a blacklists I use, in nearly all cases Spamhaus Zen. The last line is IPs that retried successfully, but were blacklisted when they tried to send other messages later.

The 5.3% for Blacklisted probably overstates how much mail was caught by waiting to see if an IP was blacklisted. My logs don't say whether the delivery attempt that was blacklisted was trying to deliver a message with the same To and From addresses, in which case it would have been delivered, or a different message, in which case it would just have been greylisted again. Spot checking shows IPs that were greylisted repeatedly, before appearing in a blacklist, which suggests that they were sending different messages.

Also, for the few IPs that were blacklisted later, they were generally blacklisted much later, hours or days later, far longer than any reasonable greylisting strategy would force mail to wait.

So greylisting still works, but it's almost entirely because spamware doesn't retry, not because it gets blacklisted.

By John Levine, Author, Consultant & Speaker. More blog posts from John Levine can also be read here.

Related topics: Email, Malware, Security, Spam

WEEKLY WRAP — Get CircleID's Weekly Summary Report by Email:

Comments

Measure the delay The Famous Brett Watson  –  Dec 09, 2011 7:41 PM PDT

This data would be more useful if you included statistics on the timing of sender retries. If the bulk of the successful retries are happening five minutes after the original connection, for example, then it would come as no surprise that the IP has not been added to a blacklist in that interval. The effectiveness of the technique is directly proportional to the overall delivery delay time, and you've provided no data on that front. Maybe you could write a "part three". I'm sure this data would be of general interest.

BCP Alessandro Vesely  –  Dec 16, 2011 10:24 AM PDT

Let me just note Murray's attempt at collecting useful hints on greylisting in a BCP draft (appsawg).

To post comments, please login or create an account.

Related Blogs

Related News

Topics

Industry Updates – Sponsored Posts

TLD Security, Spec 11 and Business Implications

Case Study: MailChimp Achieves Efficient Execution and Reliability with PowerMTA

Verisign Named to the Online Trust Alliance's 2015 Honor Roll

Case Study: Emma Swaps Its SMTP Infrastructure for PowerMTA to Handle Growing Mail Volume

3 Key Steps for SMBs to Protect Their Website and Critical Internet Services

Key Considerations for Selecting a Managed DNS Provider

Verisign Mitigates More DDoS Attacks in Q1 2015 than Any Quarter in 2014

Case Study: Email Service Provider GetResponse Scales with PowerMTA

Case Study: How PowerMTA Helped Forfront With Its Growing Message Volume

Hybrid Cloud Proves Clouds Are Worthy of Email Infrastructure

Verisign OpenHybrid for Corero and Amazon Web Services Now Available

Afilias Supports the CrypTech Project - Ambitious Hardware Encryption Effort to Protect User Privacy

Public Sector Experiences Largest Increase in DDoS Attacks (Verisign's Q4 2014 DDoS Trends)

Help Ensure the Availability and Security of Your Enterprise DNS with Verisign Recursive DNS

Verisign iDefense 2015 Cyber-Threats and Trends

What's in Your Attack Surface?

Q3 2014 DDoS Trends: Attacks Exceeding 10 Gbps on the Rise

3 Questions to Ask Your DNS Host About DDoS

Afilias Partners With Internet Society to Sponsor Deploy360 ION Conference Series Through 2016

Neustar to Build Multiple Tbps DDoS Mitigation Platform

Sponsored Topics