Home / Blogs

When DNSBLs Go Bad

John Levine

I have often remarked that any fool can run a DNS-Based Blacklist (DNSBL) and many fools do so. Since approximately nobody uses the incompetently run black lists, they don't matter. Unfortunately, using a DNSBL requires equally little expertise, which becomes a problem when an operator wants to shut down a list.

When someone sets up a mail server (which we'll call an MTA for Mail Transfer Agent), one of the tasks is to configure the anti-spam features, which invariably involves using DNSBLs. Some software (notably Spamassassin) comes with DNSBLs pre-configured, some require you to choose the ones you're going to use.

Since most MTA admins are not mail experts, they ask around, find a set of DNSBLs their friends use, put them in the configuration, and forget about it unless something goes so badly wrong that they have to fix it. Once they're set up, the MTA sends out DNSBL queries for the sending IP address of every incoming message. Depending on the way the MTA is set up, if a DNSBL lists an IP address, the MTA may outright reject the message, or it may use the DNSBL result as part of a scoring system.

The problem arises when one of the DNSBLs wants to shut down. Running one competently is a lot of work, and it's not surprising that people run out of time, retire, or otherwise can't do it any more. The goal is to get all of the people who are querying the DNSBL to stop. How do you do that?

The usual first approach is to notify everyone you've told about the DNSBL, send mail to lists about mail management, and so forth. Then you list nothing, return NXDOMAIN to every request, for a while. This will cause a small drop in query volume, since none of those systems on autopilot will notice or care.

Eventually desperation sets in, as the queries just keep rolling in. I run a small DNSBL (korea.services.net) and although I have no plans to shut it down, I can see from the query logs that some system admins can't type, and are querying kroea.services.net, or in one case http://korea.services.net, with a colon and slashes in the domain name. I would prefer that if they want to query my DNSBL, they query the actual DNSBL, so I've tried various techniques to get their attention. I've introduced very long delays, or used DNS delegation to point to name servers that don't exist on networks that don't exist. These should cause very slow responses on the mail servers, which you'd think that someone might notice and investigate. Nope.

The final step is to put in a wildcard to list the world, answer every query saying that the address is listed. This is somewhat controversial. RFC 6471 on Overview of Best Email DNS-Based List (DNSBL) Operational Practices says not to do it, on the theory that it will cause needless disruption. I used to agree, but now I'm thinking that we need to force those dusty mail systems to fix their software so they'll automatically stop using dead DNSBLs. I put in wildcards for those misspelled korea names several years ago, and also returned deliberately insulting TXT messages that might appear in their logs, e.g. "your mail has been rejected due to sheer incompetence." The abandonware is hammering as hard as ever.

The AHBL list, a widely used list for over a decade, was shut down last year. Predictably, the queries continued to hammer on their web server even though nobody was getting any useful responses, so in exasperation they listed the world last week to try and get the queries to stop. As far as I know, it didn't help much either.

The solution is to automate the client side of shutting down a DNSBL. It is not hard to tell whether a DNSBL is operating. By long tradition, every list contains a test entry for address 127.0.0.2, and does not have an entry for 127.0.0.1. At least since I wrote RFC 5782, on DNS Blacklists and Whitelists, a similar convention for DNSBLs that handle domain names rather than addresses is that the name TEST is listed and INVALID is not.

A client MTA can check the two addresses, and if it gets the desired response and not the undesired one, the list is operating. If it gets neither or it gets both, the list is dead. This catches both DNSBLs that are shut down and don't respond at all, DNSBLs that deliberately list the world, and the common case that a list is abandoned, its domain name registration expires, and is then picked up by a speculator that sets up wildcard DNS pointing to a "for sale" page.

The wildcard looks enough like listing the world to confuse a lot of people. For example, I recently saw complaints about this page which claims that a provider uses defunct DNSBLs called webmail.rhs.mailpolice.com and dynamic.rhs.mailpolice.com. The mailpolice.com domain is owned by a speculator with wildcard DNS and a for-sale page. A few moments' thought should confirm that nobody can be using it to block incoming mail, since if they were, they'd get no incoming mail at all. Nonetheless, someone was sure that they were listed and it needed to be fixed, now.

What needs to be fixed is all those abandonware MTAs. (I realize this is not going to be easy.) If spam filters checked their DNSBLs once a week, and didn't use the ones that failed the check, that would both relieve the load on the nameservers of dead DNSBLs, and it would make the MTAs work better and faster, since they would not be wasting time asking DNSBL questions to which they will never get useful answers. I suppose I can start by sending in patches for Spamassassin.

By John Levine, Author, Consultant & Speaker
Related topics: Email, Spam
SHARE THIS POST

If you are pressed for time ...

... this is for you. More and more professionals are choosing to publish critical posts on CircleID from all corners of the Internet industry. If you find it hard to keep up daily, consider subscribing to our weekly digest. We will provide you a convenient summary report once a week sent directly to your inbox. It's a quick and easy read.

I make a point of reading CircleID. There is no getting around the utility of knowing what thoughtful people are thinking and saying about our industry.

Vinton Cerf, Co-designer of the TCP/IP Protocols & the Architecture of the Internet

Share your comments

Delegating the list name itself to 127.0.0.1 Roland Turner  –  Jan 14, 2015 1:01 AM PDT

In the category of "used DNS delegation to point to name servers that don't exist on networks that don't exist" did you ever try simply delegating listname.service.example to 127.0.0.1 with a suitably lengthy TTL?

This doesn't solve the residual risk of wildcarding by a subsequent registrant, but presumably it eliminates almost all of the DNS traffic.

Didn't work John Levine  –  Jan 14, 2015 7:30 AM PDT

I tried all sorts of stuff. The broken clients seem to have no caches so they just keep coming back regardless of where else you tell them to go.

Many non-broken caches such as unbound will ignore a delegation to 127/0, which is why I tried addresses in reserved or unallocated ranges.

Checking for the test and invalid entries Todd Knarr  –  Jan 14, 2015 10:39 AM PDT

Checking for the test and invalid entries wouldn't help things like Postfix, which put their DNSBL entries in config files that're hard to update automatically, much. All they can really do is drop those entries from their active lists while they're running, but they'll be back the next time the MTA is restarted. Adding sending an e-mail to the administrative address indicating which DNSBL has failed will help, that'd alert admins like me to the problem so we can update configurations. For that matter a standalone tool we can run as a cron job would help too, and we could also use it when we set up a server to test that all our chosen lists are really working and that our provider isn't breaking things with any forced DNS badness.

well, yeah John Levine  –  Jan 14, 2015 10:57 AM PDT

This is something that needs to be fixed in the postfix code, not something you can currently configure around. A possible approach would be an automatically maintained shadow file that tracks the status of the BLs in the config file, so it only consults the BLs not marked dead in the shadow file.

If you want a standalone tool to check BLs, I whipped one up in python for my mailfront/qmail setup in about 15 minutes.

I have not brought you hope, I have not paid your fee... Alessandro Vesely  –  Jan 20, 2015 12:20 PM PDT

That seems to be yet another good reason to have clients contribute a fee to DNSxLs, or at least some kind of active cooperation.  That would keep clients updated as a side effect, and perhaps avoid some unwanted shutdown.

Slack your rope hangman, slack it for a while
I think I see my brother comin' ridin' many a mile
Brother have you brought me hope or have you paid my fee
Or have you come to see me hangin' from the gallows tree?

[listen to it on youtube.]

To post comments, please login or create an account.

Related

Topics

Domain Names

Sponsored byVerisign

IP Addressing

Sponsored byAvenue4 LLC

New TLDs

Sponsored byAfilias

Cybersecurity

Sponsored byVerisign

DNS Security

Sponsored byAfilias