A lot of spam uses fake return addresses. So back around 2000 it occurred to someone that if there were a way to validate the return addresses in mail, they could reject the stuff with bad return addresses.
A straightforward way to do that is a callout, doing a partial mail transaction to see if the putative sender’s mail server accepts mail to that address. This approach was popular for a few years, but due to its combination of ineffectiveness and abusiveness, it’s now used only by small mail systems whose managers don’t know any better.
What’s wrong with it?
- The most obvious problem is that it doesn’t work to verify addresses. There’s lots of reasons why the naively implemented checks that callouts use can succeed for a non-existent address or fail for a good address. Many mail systems (including mine) accept all addresses at the RCPT TO, and reject the bad ones at the DATA command, giving the callout system the mistaken impression that bad addresses are good. Conversely, systems that implement techniques against spam blowback (bounces from spam forging their addresses) can often reject callouts because they look just like blowback, giving the callout system the mistaken impression that good addresses are bad.
- Address verification is not an effective spam filter. These days most spam uses other people’s real addresses taken from the spam target lists, so all you’re verifying is that they stole someone’s address.
- Callouts are abusive. Since upwards of 90% of mail is spam with forged addresses, 90% of your callouts are to people who never sent you mail or otherwise bothered you.
- Callouts are likely to get you blacklisted, since the behavior of callouts is indistinguishable from spammers trying to do bulk listwashing of bad addresses.
- Callouts are discredited. Every large ISP that used to do callouts, most notably Verizon, found it was counterproductive and stopped.
I was dismayed to hear just last week from an acquaintance who was trying to persuade his system manager not to use this brand new anti-spam silver bullet. I hope he succeeded.
John,
Perhaps you might advise. A quick review of recent bounced emails show the following messages:
- 550 5.1.1
- Sorry, I couldn’t find a mail exchanger or IP address. (#5.4.4)
- 550 relay not permitted
- 554 5.7.1
: Relay access denied
- 530 authentication required for relay (#5.7.1)
Are any of these the result of the “callout” practice you identify? If so, perhaps I’ll get more specific as to a remedy the next time I bring attention to the bounce.
Thanks,
Tom Lowenhaupt
In addition to the five reasons that John points out, callouts are not scalable. A server issuing a callout would do so even in response to another callout, unless it is able to detect such circularity. Circularity cannot be detected if routing peculiarities are being deployed, e.g. multihoming or nat. Infinite recursive callouts are thus likely to occur. Dummy envelope senders, such as [email protected] break SMTP semantics even further.
I am the owner of Junk Email Filter and we filter spam for about 4000 domains, and we actually use sender callout verification. I’ll address each of your 5 points.
1) Just because it doesn’t work on some systems doesn’t mean it doesn’t work at all. It is in fact very useful on the systems where it does work, which is most of the world. No one does a callout for every email address presented. My spam filtering system does less than 1/2 of one percent verifications and many of those are cached. You also ignore false failures by ignoring servers that reject all empty senders at the MAIL FROM phase. No one in the real world implements callback the way you are implying.
2) Your second point contradicts your third point that states 90% of sender addresses are forged.
3) Your third point is just a bare allegation of your opinion that callouts are abusive.
4) Your 4th point that callouts are likely to get you blacklisted is dead wrong. I personally am making 50k callouts a day and not getting blacklisted.
5) And finally that callouts are discredited is untrue because every major MTA has built in callout features or third party callout enhancements. My servers receive thousands of callouts a day and I use the data to white list servers. (Spammers don’t do callouts - only spam filters do callouts)
You should do your homework and actually understand the technology before denouncing a technology that you haven’t thought through.
> Your 4th point that callouts are likely to get you blacklisted is dead wrong. > I personally am making 50k callouts a day and not getting blacklisted. 50K a day is peanuts mail volume wise for mailserver farms of the size that'd have to block you so that you'd find out you're being blocked.
Your second point contradicts your third point that states 90% of sender addresses are forged. Wait, what? Point 2: most spam is sent using real end users' addresses Point 3: 90% of spam is sent with forged addresses How are these contradictory? If I send mail using your real, legitimate address, I've sent mail with a forged address, right? If most spam uses other victims' real, legitimate email addresses, that spam is still sent with a forged address, right? And those callouts are then going to go to real, legitimate email addresses that had absolutely nothing to do with the sending of the spam.
There’s a famous quote from Upton Sinclair that applies here.
Yes, apply to BATV too. By the way, SPF would work if used. A "~all" (instead off -all) that stays there FOR YEARS should be blacklisted! Its intended to be TEMPORARY (changing), but its used by some of the bulk mailers that agree with you, forever. I am not seller nor blabher, just a user. I like BATV, domain keys, SPF, or SAV, as long as any of it just works or _at least help_. What I really dont understand and really dont like is: have to pay (!) for beeing out of "backscatter.org", just because you are saying things like these on your note, and many users just "believe" it (because "are not able to understand"). Almost imoral.
Hi! Are there any real reasons for not using SAV (along with other techniques)?
* “...it doesn’t work to verify addresses.”
It seems to me it works. This very post was subject to a valid email sender verification. Accept any RCPT TO and rejecting in DATA changes nothing, right?
* “...systems that implement techniques against [backscatter] can often reject callouts because they look just like blowback”...
Well, it should differentiate it…
* “Address verification is not an effective spam filter. These days most spam uses other people’s real addresses”
So? Well, SAV have worked anyway. SPF would help.
* “Callouts are abusive. Since upwards of 90% of mail is spam with forged addresses, 90% of your callouts are to people who never sent you mail or otherwise bothered you.”
Spam is abusive, not callouts. And callouts are for systems, not people. The systems should just say “ok, it´s valid” (in RCPT TO, in DATA, does not matter). Won´t hurt.
Any valid end-point (IP, port, email, www, whatever) is subject to DoS. Nothing to do with SAV. The spoofing (and its system of origin) is the bad guy, not the one that is trying to verify it.
* “Callouts are likely to get you blacklisted, since…”: ?
It seems to me that blacklisting because of callout is abusive.
* “...the behavior of callouts is indistinguishable from spammers trying to do bulk listwashing of bad addresses.”
This is really a problem. BATV can help. SPF too. The problem again is not SAV (or NDRs).
* “Callouts are discredited. Every large ISP that used to do callouts, most notably Verizon, found it was counterproductive and stopped.”:
Well, i think callouts are in wide use…see this site, for example.
Please don´t take me as a “anti-anything”. I am a user, and I am just trying to understand it. Sure it has cons, but I am just trying to get why anyone should be “blacklisted” just because the blacklister couldn´t see the difference.
And, if it´s well done, won´t hurt. DDoS and dictionary attacks risk are there regardless SAV (it is inherent to endpoints, even if you do it at DATA commands).
Maybe its not SO a bad idea, is it?
Thank you Mr.Watson. I am aware of some drawbacks (as any of the known technics), and it would certainly be lame consider IT to do ALL the job. But still didnt get a good feedback to why it should be SO terribly "BAD BAD dont do IT or go to hell"! My limitations maybe. Sorry about that and thanks for the tolerance. I wont reply to all those terrible reasons (there are already lots of this discussions around). Let me just focus on two points (take as complains): * "More or less good reason 4" - "suffer retaliation" (from the big sources of spam: the big providers): Well...I am a user as any other on Internet. Things should not work this way here. Anyway, you are right: all of this changes nothing, its still a real and serious risk (well, ok, got it: it´s not to understand it technically). * My mistakes: SAV and challenge/responde have a similar concepts but different implementations (as you noted): that´s my point, nothing wrong with the CONCEPT, but with many implementations. Sure, it should not be done ALONE or just with SPF, without caching, etc. Well, ok, I got it: "dont do it or go hell" its ok if you assume everyone is ignorant. I may accept this as a good point (really). The "value for money", if well done, is reasonable. One should consider be a siner, with discretion, and maybe we wont go hell after all. Regards.
Sorry…Sudenly I see…you are talking about mass weapons and me about rocks. If I throw some of it I could get wiped. Its really a risk of the Wild Wild Web.
For a moment I thought it didnt work. :)
Best regards.
Hi folks!
“You’re also confusing a web application with email. The two aren’t comparable. For many reasons, email-based protection on a forum or blog does not (in general) produce the kind of undesirable side-effects that result when the same protection is applied to an email address. “
I have just realized that its possible to do a DDoS to any poor email user, just putting a bot networks to register that poor user email in any (or many) websites that requires a valid email address… :) bingo…
It would be great to know about some of the “many reasons” that would avoid that…
Yes, it's possible to mailbomb someone by forging subscriptions. It's happened to me, many times. But since that mailbomb is mail TO the victim, rather than mail pretending to be FROM the victim, non of this ywo year old message applies.
And for this victim, receiving site confirmations from 1.000.000 web-hosts is different from receiving callouts because… ... ...? ...captchas? Just like the one that CircleID dont have? Because its difficult to do? Harder than send 1.000.000 mails with forged return address? Well, you are right, I am not able to “get” it. My age, maybe.
Getting a large number of COI confirmations for forged subscriptions is indeed very annoying, although unlike non-COI lists at least it's only one confirmation per list. If you have in mind some tweak to the mail system that will prevent bad guys from typing your address into a web form, I suspect I am not the only one who'd be intrigued to hear about it.
COI forms are not standardized and can be abused in a number of ways. A simple tweak: users should tell or confirm to their servers the list they want to subscribe to. The servers connect to each other, exchange info, and end up with two specular COI data sets: a forwarding recipe at the list server, and a forwarding agreement at the user's server.
To abuse a famous quote, this must be a meaning of “simple tweak” with which I am not familiar.