|
||
There are many companies in the spam-fighting business and most, if not all, claim to be hugely successful. Yet spam is exponentially more prevalent today than it was just 2 years ago. How can one conclude that today’s anti spam solutions are working? This year spammers will use machine-generated programs to send trillions of unsolicited email.
Thankfully, a new anti-spam technology has made its way into the market. This approach, known as Sender Address Verification or SAV, is poised to cripple spammer’s ability to deliver machine-generated email. SAV employs a patented methodology that asks first-time senders to verify their email address before a message is forwarded to an individual recipient’s inbox. SAV is easy to set up and provides tremendous value to both IT and business users. Most importantly, SAV eliminates 100% of spam and produces zero false positives. We will discuss more on SAV later in this article.
The Weapons of a Spammer: Anonymity, Automation, and an Arms Race Mentality
There are 3 primary reasons why the problem of spam has exploded:
Anti-Spam Filters: A Flawed Approach
Filtering is a computing technique that employs artificial intelligence and complex algorithms to decipher “wanted” email from “unwanted” email. The two most common filtering techniques are “Bayesian” and “heuristic.” Both are sophisticated and intriguing, and both have failed miserably. Considering the exponential growth of spam and its financial impact in terms of lost employee productivity and wasted IT resources, it is clear that a better solution must be found.
[Flaw] Filters are reactive, a fact that spammers can exploit in 2 ways:
[Flaw] Filters are either “too tight” or “too loose”:
As vendors or their customers tighten filters in hopes of blocking all spam, they begin to filter out important non-spam emails as well, an occurrence known as a “false positive.” When filters are relaxed to limit the number of false positives, more spam emails slip through. This “catch 22” will continue to be an ongoing dilemma for companies that use anti-spam filters.
[Flaw] Filters are being subsumed by SAV techniques:
Acknowledging their innate deficiencies, filtering vendors may be forced to include authentication layers in their anti spam offerings. The current “soup du jour” is Sender Policy Framework (SPF). Unfortunately, with their substantial investment in artificial intelligence (filtering), many vendors are prohibited from abandoning this flawed approach. The results are product offerings that force customers to pay for filtering features that are no longer needed or effective.
Sender Address Verification: Winning the Battle Against Spam
A hallmark of our civilized society is etiquette. As children we are taught to always “knock” before entering a room with a closed door. SAV technology employs this approach to stop spam, requiring first time senders to identify themselves before their message is allowed to enter a recipient’s inbox - a type of digital doorbell.
SAV’s effectiveness is matched only by its simplicity. When persons with whom one has never corresponded send an email, a properly designed SAV system responds to the sender, in the intended recipient’s name, asking the sender to identify themselves by performing a simple task. This one-time verification request comes in the form of an email back to the sender. The simplest task requires the sender to press “REPLY” and “SEND” when they receive the request. Additional tasks, such as puzzle solving (captchas) can be employed if necessary to confirm a human sender. When the sender completes the verification request their email is automatically delivered. A delivery confirmation receipt is sent acknowledging that their email has been delivered and that their address has been automatically placed on the recipient’s approved sender list. Once placed on this “whitelist” all future emails will be delivered directly to the recipient.
SAV removes the veil of anonymity enjoyed by spammers because the verification request is, by definition, an audit trail leading back to the sender. According to the Anti-Phishing Workgroup’s May 2004 report, “...the actual percentage of spoofed phishing emails is likely higher than 95%.” It is highly unlikely that spammers will respond to the verification requests, assuming that the return email address they provide is actually real, because in doing so they would be accepting responsibility for the spam that was sent. SAV turns the tables on the spammers back in favor of email recipients.
Advantages of Sender Address Verification (SAV)
While there are many compelling arguments in favor of SAV, perhaps the most convincing are its 100% effectiveness and ability to help enterprises re-capture billions of dollars in lost productivity and wasted resources.
[Fact] SAV reduces server load and storage costs:
When implemented as a stand-alone network appliance SAV sits at the network edge, in front of the corporate mail infrastructure. Only email that should be delivered is actually allowed into the network. Spam is never allowed to reach the messaging infrastructure, thus limiting the burden on existing resources. In certain industries where government regulations mandate that all email is indexed and archived, a SAV enabled network appliance can free companies from incurring storage costs associated with spam.
[Fact] SAV never deletes legitimate emails:
Spam filters that use mathematical formulas to guess whether an email is legitimate regularly obscure valid email after mistaking it for spam (a false positive). SAV does not employ these techniques nor does it make these costly errors. SAV eliminates the guesswork by merely asking first-time email senders to “knock before entering.”
[Fact] SAV mitigates future spam by exposing the sender:
SMTP (the backbone of Internet messaging) has no mechanism for verifying the validity of the email sender, thus making it an anonymous transport mechanism. There are promising technologies on the horizon, such as Sender Policy Framework (SPF) and Domain Keys. However, at this time the efficacy of either approach cannot be accurately measured due to lack of widespread industry adoption. SAV solves this basic problem not by trying to fix/monitor the actual correspondence that is sent via SMTP, but by closing the “loophole” in SMTP - the anonymity factor. SAV requires that senders of email either be pre-approved by recipients, or be willing to identify themselves.
Dispelling Myths About SAV
While the concept of SAV is quite simple to grasp, many experts have struggled to understand how SAV is implemented. This confusion has allowed a significant amount of misinformation to find its way into both the mainstream and IT trade press. Below are some common myths about SAV:
[Myth] SAV will double the amount of email traffic by flooding the Internet with verification requests:
The misconception is that for every piece of email there will be a corresponding verification request. This is false.
[Myth] SAV systems will flood each other with “ping pong” verification requests:
At the root of any email processing mechanism is an email server. A basic characteristic of any modern email server is the prevention of “bounce” loops. A properly implemented SAV mechanism will, therefore, not be subject to “bounce” loops or message “ping pong.”
[Myth] SAV will block legitimate machine-generated email (e.g., newsletter subscriptions):
Part of every SAV system is a pending queue, or waiting room. This is a place where messages are sent until they are validated as legitimate. If a machine-generated message is sent from a previously unauthorized newsletter subscription or an e-commerce receipt, the message will be waiting for approval in the pending queue. Some SAV systems allow users and administrators to whitelist entire domains and/or parts of domains. For example, a system administrator could globally indicate that any email sent from @Amazon.com, or @VentureWire.com should never be authenticated. Individual end users have the same capability to configure this functionality for their personal accounts.
[Myth] SAV is easily circumvented by “joe-jobing”:
While “joe-jobing” a SAV system is certainly possible, in practice it is cost prohibitive and by actively committing fraud, spammers put themselves at greater risk. In addition, with the creation and proliferation of anti-fraud mechanisms like SPF, Sender ID, etc., “joe-jobing” can be prevented entirely.
Conclusion
Sender Address Verification makes it impossible for the purveyors of spam to realize the financial gains that have been enjoyed in the past. SAV eliminates the unfettered access to email inboxes that was available to crafty spammers, enabling companies to regain productivity and IT resources once lost to fighting spam. As this simple and elegant approach gains widespread acceptance, spam, as we know it, may be entirely eradicated.
—-
Reproduced by CircleID with expressed permission of Sendio, Inc. An Adobe PDF version of this article available for download from Sendio’s website.
Sponsored byVerisign
Sponsored byWhoisXML API
Sponsored byDNIB.com
Sponsored byIPv4.Global
Sponsored byRadix
Sponsored byCSC
Sponsored byVerisign
A World-Renowned Source for Internet Developments. Serving Since 2002.
Lovely. Given how many anti-spam companies there are, all claiming to possess the holy grail of anti-spam, can CircleID readers now expect to be presented with advertorials like this on a regular basis?
Without studying the matter beyond this article, it’s pretty clear that Sendio’s “SAV” technology is “Challenge/Response” by another name. Why not call it C/R? Marketing reasons? Not catchy enough compared to “SAV”? Or is it that Mailblocks claims to have a patent on Challenge/Response (http://about.mailblocks.com/press/press_0507_2003.aspx), and Sendio is trying to make their patent look different? (Or maybe they’re the same company now—who has the time to follow the who’s who of anti-spam products and patents?)
In any case, let me put my case as to why Challenge/Response technology is every bit as rude as spam itself (in the absence of mitigating technology such as SPF, which Golan refers to in rather patronising tones). C/R (or SAV as Golan would have it) is rude because it makes others do your spam filtering for you, as shown by the following example.
Say Sammy the spammer wants to send advertising to Alice. Sammy the spammer knows better than to put his own address on the outgoing mail, so he picks another address—but not just any address—Sammy uses the address of his arch-nemesis, Joe, who once cut off his Internet connection for violating the terms of service. When the anti-spam gateway at Alice’s place picks up the mail, it sends a challenge to Joe, because Joe is listed as the sender. In fact, poor Joe is being inundated by challenges and bounced messages that unthinking and uncaring mail systems everywhere are spewing out, thanks to Sammy’s falsified “from” address.
Without realising it, Alice is making Joe do her spam-filtering for her. If Joe responds to Alice’s challenge, she’ll get the spam, and Joe will get the blame—Sammy the spammer gets off scot-free. Only if Joe ignores the challenge—treating it like a piece of spam and just deleting it—will Alice have her spam-free day. Note that carefully: Alice eliminates one piece of spam by making Joe deal with it for her. This situation is fine if you’re Alice, but not so great if you’re Joe. And if your name is already on a spammer’s list, maybe you’ll be Joe on the next spam run.
Yes, a Joe job like this involves fraud. Well I have some news: shocking as it may seem, many spammers are quite willing to resort to this kind of fraud. More often than not the products they sell don’t live up to the claims in their sales pitch either. It may be hard to accept, but spammers can and will resort to dishonesty as it suits them, and typically settle out of court without admitting any wrongdoing (e.g. Spitzer vs Richter) if they get prosecuted for it.
The argument against Challenge/Response has many more aspects than this, but I’ll leave that to others if they want to take up the slack. If you use a C/R system like this and you’re lucky, the challenges you send out will be undeliverable, wasting as little in the way of network and human resources as possible. If you’re unlucky, you’ll send your challenges to an ill-tempered curmudgeon (like myself), who must then decide whether to give you a spam-free day at his own expense.
What will you do when it’s your turn to be Joe?
I have to say I am just as disappointed as Brett to see this article here at CircleID of all places.
I have not a lot to add to Brett’s comment save to note that most of the attempts to debunk “myths” fall into the “with one bound he was free” category and do not actually say how they will work.
Example: “Spammers do not typically provide valid return email addresses. A properly implemented SAV system can detect this and withhold the request.”
a) Please provide evidence that spammers do not typically provide valid return email addresses. I have a huge archive here full of spams with valid return addresses - it is just that those addresses do not belong to the true sender.
b) How exactly does a properly implemented SAV system detect this, and what is “this” anyway? If “this” is a forged address in a non-existent domain then (absent Sitefinder or near equivalent), this is trivially easy to spot. If “this” is a spoofed but valid return address then, until the entire world has fully implemented SPF, it can’t be done.
The bottom line is that any machine generated response to email is potentially hazardous.
http://chris-linfoot.net/plinks/CWLT-62PG98
As per above comments by Chris Linfoot and The Famous Brett Watson: Sometimes articles are posted because they carry points that are determined to offer good bases for a worthwhile debate. They’re definitely not advertorials (or a paid inclusions).
Thank you for your continued feedback and valuable participation.
Interesting article from Mr. Golan. I recall having seen his posts before and think it’s great that he’s taken the time to contribute. This should always been encouraged as it keeps the forum vibrant. There’s a monicum of respect that’s due in this forum which helps distinguish it from a simple newsgroup flame pit.
You know, I think I get it concerning the SAV and what I read I like. I use Mailblocks for my personal accounts and am aware of some of the strengths/weaknesses from first hand experience. One weakness is the response from some people who object to the Challenge. One strength is that is has been effective. The curious fact is that I haven’t had anyone balk at the Mailblocks challenge. People that I corrospond with, I suppose, are as annoyed with spam as I am and are willing participants in the solution. I’m grateful. I see the SAV approach as being a natural evolution of this and can understand where if I was to use this the positives I’m enjoyed with the Mailblocks service can be maintained while removing the negatives in the presence of this device.
I dropped an email to Mr. Golan and was greeted by his SAV. It was pleased that I was not lead to another mystery website (normally causing me alarm), that the response was from Mr. Golan (not an otherwise mysterious service). In the end, I responded to the SAV without concern or alarm. You know ..if it keeps this guy from getting spam so be it! The inconvenience to me was zero and my intrigue was high.
Challenge/Response has been quite effective for me but I see the evolution to SAV as being well conceived. I surely like the idea of an appliance and have had enough discussions with the network guys to know that they don’t think too highly of running ANYTHING on the Exchange server or on our desktops (high mainteance)
Apologies to the pro filter folks. In my experience ..even the top rated products used at my work(BrightMail) simply fail ..and in circumstances where SAV would have met my needs. Ah .. a future discussion with our network folks.
Cheers to those who hold spammers in the same contempt as I.
The back handed pander towards SPF, while advocating a challenge response approach as the holy grail is flawed.
Let me elaborate from the perspective of a mail list administrator.
If an individual subscribes to a mailing list and verifies her subscription, why is there the added need for the administrator to go through a challenge response process to verify identity? This just adds an administrative headache, unduly punishing those who already follow good mailing practices, while consuming more bandwidth.
Besides, the notice one receives in a challenge response situation typically contains advertising promoting the service.
Sorry, but I don’t see how this is good “etiquette.” As a mailing list administrator, I did not ask to receive a commercial advertisement generated by Servio.
From the sender’s perspective, it is much simpler to publish a sender policy framework record and allow receiving servers to make the required DNS query.
(Yes, I appreciate in doing so, it requires senders to get their DNS house in order. But there can be no denying this is not in the best interest of all.)
Turning to the filter issue, the author somehow suggests the use of sender policy framework for authentication purposes will not allow receivers to carry out further checks.
The sender policy framework presently before MARID allows the receiver to authenticate the sending domain at the transport stage before message content is received.
On the other hand, a challenge response system requires acceptance and storage of the entire message pending the sender going through the response process.
How does this better serve receivers?
Under the proposed framework, if the sending domain is authenticated, the receiver can if she wishes program her mail server to carry out further checks either before accepting delivery of the content, or after delivery of the content.
Prior to content delivery, these checks may involve verifying the sender’s reputation.
After delivery, depending on the results, the receiver can decide to filter the message for content. This can either be done at the gateway level or at the end user’s level.
One other point. As sender authentication evolves, it may also serve as a useful tool in the legal fight against spoofing and phishing.
How does the challenge/response approach help to serve this purpose?
At the same time, there are many unanswered questions surrounding sender authentication.
Fortunately, it does not hold itself out as the magic bullet. And I anticipate as the online community moves forward, wide scale testing of what ever framework is proposed by MARID will reveal flaws, requiring design changes and further work.
Want to put forward challenge/response as a solution, hey all power to you.
But ... let’s not start saying any solution is the holy grail, or the best thing since sliced bread, when we all know better.
John Glube
Toronto, Canada
“The authentication train is about to leave the station, and IT administrators should be buying a ticket to ride.”
Cameron Sturdevant
eWeek (8 March 2004)
My email address is forged by spammers 10 or 50 times a day. So now I will be getting 10 or 50 requests from mail servers asking me to authenticate my email? How do I separate that one legitimate email I send from the 10 or 50?
Sorry, again a flawed technique.
SPAMmers will just change tactics, and when they harvest e-mail addresses from compromised machines, they will make sure they capture that users e-mail address as well, so as to have a legitimate FROM address, that will allready be verified as acceptable by those e-mail addresses.
There is no silver bullet. Until SMTP is replaced with a protocol that verifies the full e-mail address of the sender at the sending end and double checked to at least to the domain name level at the receiving end (SPF, caller ID etc) the scourge of SPAM will continue.
ISP’s also need to take more responsibility.
Some things that ISPs can do in the short term to reduce the ammount of SPAM.
1) Block outgoing port 25 except to their own mail server to force all users to send e-mail through their servers
2) Monitor outgoing e-mails for SPAM (contents/volume) and viruses, and when detected contact the user (either to disinfect their machine, or to tell them to stop spamming depending on what the situation is).
3) By default refuse to receive/send all executable attachments on e-mails unless users request that they be allowed to receive such. This with 1) above would stop most of the common viruses from spreading and make it harder for SPAMmers to have a masive army of zombie machines.
Once responsible ISP have taken the above actions, pressure can be brough to bear on ISP’s and countries that continue to allows SPAMming.
“The biggest thing we can do to reduce spam is sender authentication.”
Brian Sullivan
Senior Director for mail operations, AOL
New York Times (23 June 2004)
Yes, but “Verification” is not the same as “Authentication”. I’m all for Authentication, but Verification will not work, and just cause more issues that it solves.
Tal, you reference the eWeek article of March 8, 2004.
This article references the announcement made earlier that Sendmail was prepared to work with Microsoft in testing Caller ID and Yahoo! in testing DomainKeys.
This article is speaking of Sender authentication.
A different animal than challenge response.
On the issue of sender authentication, the reader may also want to review Is The Writing On The Wall For Spam?
As to the NY Times article of June 23, 2004, this was generated as a result of the publication of the Anti Spam Technology Alliance (ASTA) policy paper earlier that week.
ASTA includes AOL, British Telecom, Comcast, Earthlink, Microsoft and Yahoo!.
The focus of the policy document? To deal with email abuse.
Part of the proposed policy includes implementing sender authentication based initially on SPF/Sender ID and ultimately on DomainKeys.
Of couse this was big news, since the FTC had just issued its report on the feasibility of national Do Not Email List a week prior stating the first step was to develop and implement a standard for sender authentication.
Quoting from the article:
“The Internet providers, however, cautioned that both of these technical approaches [referencing SPF/Sender ID and DomainKeys] are just part of the solution to the problem. Once Internet recipients can verify who is sending them mail, they can start to keep track of who sends legitimate mail and who sends spam.
“I don’t think that users will see a reduction in spam right away,” said Robert Sanders, chief architect at EarthLink. “Identity is just the first step.”
People can access a copy of the ASTA policy document from here.
Colin, I agree with your concern about hackers simply getting in to compromised machines, up loading approved from email addresses and then spoofing these to deliver more spam.
Also, we need to understand the MARID Working Group has yet to finish its work.
All hands (Microsoft, Yahoo!, the people involved in SPF and the folks behind CSV, a proposal which has not received much ink) having agreed to submit their work to the IETF, it is important to take into account the outcome of this process.
My point. Yes, Sender authentication as suggested by the FTC and discussed by ASTA is a step in the right direction.
But we need to clearly understand, a lot of work remains. Sender authentication is simply one part of an overall process in chaging the way we send and receive email.
The effort includes consumer education, technology to help bring email abuse under control which will include authenticating the sender’s domain and reputation, law enforcement to bring those involved with email absue before the Courts and international cooperation.
However, as I said before, don’t let me stop Sendio from marketing Mail Blocks to users who wish to implement it.
However, in my view the approach has its draw backs.
Also, if we are going to talk about sender verification, which is different from sender authentication, it might be helpful to look at other end user products on the market involving sender verification.
These include Titan Key, Vanquish, along with Tmda (a product similar to MailBlocks).
What is difference between SAV and SPF (Sender Policy Framework, http://spf.pobox.com/)?
With SAV, email is accepted and a request for verification is sent back to the purported originator of that email. Given that most spam and virtually all viruses spoof the sender address this can mean that an innocent bystander is left looking at an unsolicited challenge message and wondering what it means.
With SPF, DNS contains a record of those mail servers that should legitimately be sending on behalf of a particular domain. The (SPF aware) recieving system can look up the mail servers permitted to send email on behalf of the domain used by the sender of an email. If a hard fail is encountered, the receiving system may refuse the mail during delivery (so no bounce to a spoofed third party), or accept it and silently delete it instead of delivering it.
Hope that helps.
so here we are a year later - what’s happened since then? any advances that would tend towards or away from SAV or C/R?
any comments appreciated
Tony Toews said:
How about this? My email address is forged by spammers, so that I get 900+ bounces a day! The bounces probably account for 1% of the number of actual emails that have my address forged in them.
So, with SAV being done on some portion of the rest, imagine how many “probes” end up on my mail server!!!
Spammers are forging real addresses to circumvent SAV. Any article that speaks of some “holy grail” for the spam battle is just talking about the latest weapon in a war that will not end any time soon.
I guarantee you this has an effect on people’s mail servers. Check out these links to show proof of the the reality and extent of real-address forging problems:
http://profs.logti.etsmtl.ca/cfuhrman/backscatter/
http://taint.org/2007/03/16/134743a.html
hi Tony
I suggest that you download Carrotmail. this is a reliable spam protector and it guarantees 100% protection.
This is a geat system and i can tell u, u will be very pleased and these problems will stop.
hi tony
sorry, if u are interested in subscribing to this go on carrot.org. if u do have any questions just contact me and ill try and help as much as i can.
thanks alexandra