Home / Blogs

Greylisting Still Works - Part II

John Levine

In my last post I blogged about greylisting, a well-known anti-spam technique for rejecting spam sent by botnets. When a mail server receives a an attempt to deliver mail from an IP address that's never sent mail before, it rejects the message with a "soft fail" error which tells the sender to try again later. Real mail senders always retry, badly written spamware often doesn't. I found that even though everyone knows about greylisting, about 2/3 of IPs don't successfully retry.

Another theory about greylisting is that if you defer mail from a new IP, by the time the sender retries, if it's sending spam it'll have hit spamtraps and been added to blacklists. I recently realized that I have enough log data to check that theory, so I collected some statistics for the past week, which is as long as I keep logs about mail connections from blacklisted hosts. The IPs I greylisted broke down like this:

CountPercent
No retry3,80335.8%
Retry too soon3,34531.5%
One retry1,18311.1%
More than one message1,63515.4%
Blacklisted5615.3%
Retried, blacklisted later890.8%
Total10,616100.0%

No retry and Retry too soon are senders that greylisting kept from sending anything, again, about 2/3 of mail. (My greylister requires that the sender wait at least a minute, since some spamware sends several messages a few seconds apart.)

The next two are senders that retried successfully and sent one message, or more than one message. (If a sender retries too soon, then retries again after more than a minute, it's counted in one of those two categories.) Blacklisted means that when the IP retried, the IP was on one of the a blacklists I use, in nearly all cases Spamhaus Zen. The last line is IPs that retried successfully, but were blacklisted when they tried to send other messages later.

The 5.3% for Blacklisted probably overstates how much mail was caught by waiting to see if an IP was blacklisted. My logs don't say whether the delivery attempt that was blacklisted was trying to deliver a message with the same To and From addresses, in which case it would have been delivered, or a different message, in which case it would just have been greylisted again. Spot checking shows IPs that were greylisted repeatedly, before appearing in a blacklist, which suggests that they were sending different messages.

Also, for the few IPs that were blacklisted later, they were generally blacklisted much later, hours or days later, far longer than any reasonable greylisting strategy would force mail to wait.

So greylisting still works, but it's almost entirely because spamware doesn't retry, not because it gets blacklisted.

By John Levine, Author, Consultant & Speaker. More blog posts from John Levine can also be read here.

Related topics: Email, Malware, Security, Spam

WEEKLY WRAP — Get CircleID's Weekly Summary Report by Email:

Comments

Measure the delay The Famous Brett Watson  –  Dec 09, 2011 6:41 PM PST

This data would be more useful if you included statistics on the timing of sender retries. If the bulk of the successful retries are happening five minutes after the original connection, for example, then it would come as no surprise that the IP has not been added to a blacklist in that interval. The effectiveness of the technique is directly proportional to the overall delivery delay time, and you've provided no data on that front. Maybe you could write a "part three". I'm sure this data would be of general interest.

BCP Alessandro Vesely  –  Dec 16, 2011 9:24 AM PST

Let me just note Murray's attempt at collecting useful hints on greylisting in a BCP draft (appsawg).

To post comments, please login or create an account.

Related Blogs

Related News

Topics

Industry Updates – Sponsored Posts

Q3 2014 DDoS Trends: Attacks Exceeding 10 Gbps on the Rise

3 Questions to Ask Your DNS Host About DDoS

Afilias Partners With Internet Society to Sponsor Deploy360 ION Conference Series Through 2016

Neustar to Build Multiple Tbps DDoS Mitigation Platform

The Latest Internet Plague: Random Subdomain Attacks

Digging Deep Into DNS Data Discloses Damaging Domains

New gTLDs and Best Practices for Domain Management Policies (Video)

Nominum Announces Future Ready DNS

Non-English "IDN Email" Addresses Are Finally Working!

New from Verisign Labs - Measuring Privacy Disclosures in URL Query Strings

DotConnectAfrica Delegates Attend the Kenya Internet Governance Forum

3 Questions to Ask Your DNS Host about Lowering DDoS Risks

Continuing to Work in the Public Interest

Verisign Named to the OTA's 2014 Online Trust Honor Roll

4 Minutes Vs. 4 Hours: A Responder Explains Emergency DDoS Mitigation

Dyn Acquires Internet Intelligence Company, Renesys

Tips to Address New FFIEC DDoS Requirements

Smokescreening: Data Theft Makes DDoS More Dangerous

dotStrategy Selects Neustar's Registry Threat Mitigation Services for .BUZZ Registry

24 Million Home Routers Expose ISPs to Massive DNS-Based DDoS Attacks

Sponsored Topics

Minds + Machines

Top-Level Domains

Sponsored by
Minds + Machines
dotMobi

Mobile

Sponsored by
dotMobi
Afilias

DNS Security

Sponsored by
Afilias
Verisign

Security

Sponsored by
Verisign