CircleID

The Messaging Anti-Abuse Working Group (MAAWG), of which Return Path (my employer) is a very active participant, met recently in Heidelberg, Germany. Among other exciting projects, they finished two new best practices documents which have been lauded in the press as a big step towards stopping botnet spam.

("Botnets" are networks of computers infected by viruses or other malicious software, invariably without the owner's permission or knowledge, which are used to engage in criminal activities like sending spam or attacking web servers.)

Neither document, however, is actually about botnets — that'll come from the next meeting, which has a botnet theme. Instead, both describe simple ways to improve classification of mail sources, so that reputation scoring may be applied more accurately and effectively. I'll explain this further towards the end.

Email Forwarding Best Practices, edited by two of our friends at Comcast, describes a problem which only affects a small percentage of users — but for those who are affected, it's a big problem. Forwarding, in this context, is when a message is sent to an address which is configured to resend all mail to another address — for example, bob@alumni.stanford.edu may have his mail forwarded to bob@comcast.net.

The problem, as always, is spam. When more than 90% of all email is spam, then more than 90% of mail sent to bob@alumni.stanford.edu is spam — so more than 90% of what Stanford forwards to Comcast is spam. Comcast's spam detection systems will notice that 90% of what they get from alumni.stanford.edu is spam — in other words, a very bad reputation — and will block all mail from that system.

Forwarding has been around pretty much since the beginning of internet email, though not all sites offer it today. The way it works in most places is almost appallingly simple: a message is received, and is immediately sent back out. There's very little processing involved.

As Return Path is constantly advising clients, any legitimate sender needs to avoid looking like a spammer. So do forwarders.

MAAWG, in this document, recommends that forwarders engage in more processing before resending a message. They suggest to forwarders that they catch as much spam as possible, rather than blindly forwarding all of it, and ensure that both the systems they use for forwarding and the forwarded messages themselves are clearly labeled. They further suggest that anti-spam systems should look for these labels, and treat forwarded mail differently from other sources.

The second document is dryly titled Methods for Sharing Dynamic IP Address Space Information with Others. "Dynamic IP Address Space" refers to IP addresses which are dynamically assigned, such as to dial-up, cable, or most DSL connections. These consumer-grade services are how most people access the internet from home, and home computers are statistically extremely likely to be infected — thus, most botnets consist of computers on dynamic addresses.

MAAWG previously published a recommendation that ISPs should take steps to restrict or otherwise control port 25 connections from dynamic addresses, in order to reduce bots' ability to send email. In those cases where this is not possible — and to assist with non-email-related attacks from botnets — MAAWG recommends clearly labeling such dynamic addresses, and keeping them separate from static (non-changing) addresses. The document goes on to list some common labeling methods and styles.

Obviously home users should be able to send email, but their legitimate messages are sent through their ISP's mail servers — often using the submission port, 587 — rather than directly to the recipient's server on port 25.

So, what do the recommendations in these documents have to do with reputation?

In both cases, the clear labeling and transparency make it easier for anti-spam systems to determine which thresholds are appropriate for that type of mail. Dynamic addresses, with few exceptions, shouldn't be sending mail directly at all — thusany volume is suspicious, even before there are complaints or other data to mix in. Forwarding servers, because they're likely to be forwarding some spam no matter how hard they try to catch it, can't be judged solely on IP reputation — but should still have content filters applied.

Reputation systems adjust for other categories, too: the normal behavior of one of Comcast's mail servers will be very different from one of eBay's. Some give greater leeway to ESPs and other commercial senders who subscribe to feedback loops, because the feedback allows them to take action quickly. Others will vary based on the country of origin of the message, knowing that a particular set of users is unlikely to want email written in a language they don't know how to read.

This variety of categories benefits users, because it increases the likelihood of catching unwanted mail while decreasing the likelihood of misplacing something the user actually wanted. However, it frustrates senders who'd prefer to have a single, simple numeric goal that they don't have to think about. (Since the ISPs work for their users, you can guess whose preference wins out.) In any case, one reliable rule is that any behavior outside of the norm — no matter which category's "norm" is used — is considered suspicious.

This article was originally published on the Return Path blog.

By J.D. Falk, Internet Standards and Governance. Visit the blog maintained by J.D. Falk here.

Related topics: Access Providers, Email, Security, Spam, Telecom