Back in the days of dial-up modems and transfer speeds measured in hundreds of bits per second, unwanted email messages were actually felt as a significant dent in our personal pocketbooks. As increases in transfer speeds outpaced increases in spam traffic, the hundreds of unwanted emails we received per week became more of a nuisance than a serious financial threat. Today sophisticated spam filters offered by all major email providers keep us from seeing hundreds of unwanted emails on a daily basis, and relatively infrequently allow unwanted messages to reach our coveted Inboxes.
So, to some degree, the spam problem has been mitigated. But this “mitigation” requires multiple layers of protection and enormous amounts of continually-applied effort. First, there’s blacklisting — the “Internet Jail” — which forces spammers to constantly change and taint IP addresses. Then there’s whitelisting, for those who don’t mind never hearing from long-lost pre-Internet friends again. Next, there’s SPF, which helps us be certain that we don’t bounce error messages to the wrong people (though there are loopholes). And finally, as a last resort, we have content filtering, which weeds out all the viruses and other material that the cyber-filter finds objectionable to its artificial conscience. The result — for me, at least — is that I have to manually delete about 30 unwanted emails a day, and Yahoo takes care of the other 300. But I have little idea how well it’s working — I simply don’t have time to wade through 300 emails a day to make sure Yahoo hasn’t censored anything important. Who knows how many people out there are holding little grudges against me for never responding to their messages? Who knows how much pent-up tension false positives are causing in our modern email-centric society?
Since the advent of email decades ago, the paradigm has never changed: A sender composes a message, kisses it goodbye, and hands it off to a delivery service. The delivery service finds the exact addressee, and delivers the message. Back when email users were few and one person’s personal address book may have listed half of all email users in the world, this was fine, because not only could you trust the delivery service, you could trust all of the senders — why would Professor Smith ever want to send Professor Jones an advertisement for magic pills, when they both knew perfectly well that nobody could beat Professor Johnson’s price? The problem is that the system was — and is — tilted in favor of the sender: the sender can be sure that only the specified recipient will receive the message, but there is no guarantee to the recipient that the sender is who the sender says she is. It’s perfectly understandable that the original creators of email saw no problem with this, because this is how regular mail works — we still don’t know who sent the anthrax.
But, whereas regular mail has survived for hundreds — maybe thousands — of years with rare troublemakers, in just several short decades electronic mail has gone from birth to 24-hour life-support, thanks to its most prolific users: Spammers, Scammers, and Phishers. They take advantage of the naivete and good-heartedness of our trusting ancestors to flood our Inboxes with junk, scams, and viruses. They operate in an Inbox-centric universe that was created in an era when only Good existed, when putting something into a person’s Inbox was a matter of sacred trust that everyone respected. But now the innocence is gone, evil runs rampant, and every Inbox has become an open dumping ground for whomever cares to dump.
But the technology already exists to solve this problem, and it’s high time we start using it. We already have a technology that lets the recipient be sure that the sender is who the sender says she is, and it’s almost as popular as email. It’s called “The World Wide Web”, and it’s about time we started using it to solve our email problems. The WWW has been “Outbox"-centric all along: a sender (publisher) sticks a message (file) into her Outbox (web server) and the intended recipients (surfers who know the URL) read it (request it with their browsers). Why not apply this paradigm to email also? And to make things easier on everyone, why not just go ahead and apply the protocol too?
Thus, Hypertext Mail Protocol (HTMP) is born, or Stub Email for short (read “stubby mail"). The implementation path is easy, seamless, and will only take a few years to complete — maybe less; here it is:
Step 1: The early adopting end-user installs an HTMP plugin for her email client. The plugin has only three settings to configure: HTMP Outbox URL, HTTP BASIC AUTH username, HTTP BASIC AUTH password; and performs only one simple two-step operation: whenever the “send” button is pressed, the plugin first takes whatever email message would have been sent — headers and all — and HTTP PUTs it to the HTMP Outbox URL with an un-guessable UUID appended to the end of it, using the HTTP BASIC AUTH username and password to authenticate if challenged by the web server (it is assumed that the PUT operation is performed over a secure channel, such as SSL); second, the plugin replaces the original message with a well-formed email message that has many of the same headers as the original message (To, From, Cc, Subject, etc.), one additional header to specify HTMP compliance (HTMP-Version), and a message body that contains only one thing: the plain-text, fully-qualified URL of the original message that has now been PUT to the WWW. So the message that is actually sent via SMTP to the would-be recipients of the original email message is most likely much shorter than the original message, and contains only a (WWW-accessible) link to the original message; we’ll call this message the “Stub Email” and the link it contains the “Stub” (or “Stub URL"). Recipients of a Stub Email — whether or not they or their email clients have any idea what a Stub Email is — receive a perfectly readable and actionable message, one that contains a URL that they are obviously expected to retrieve. Thus, transition to widespread Stub Email use is seamless.
It is assumed that the Stub’s domain name, perhaps combined with something in its path — when viewed by the recipient — will convey enough information to sufficiently satisfy the recipient that the message is worth retrieving, i.e. we’ll finally have a use for all those .name domain names we bought. During this initial early-adopter phase of HTMP implementation, whether or not a message is worth retrieving is a judgment that each human recipient makes on her own. In later phases, this decision can be made automatically based off of a personal Stub Whitelist (SWL) stored in each person’s email client and maintained individually. A person’s SWL can be shared with friends and/or sync’d with trusted SWL servers, as well as contain listings based off of domain only, partial path, or other matching scheme. The SWL mechanism need not be part of the HTMP specification, but can be left open for networked individuals and email client providers to define independently or in specifications separate from the HTMP specification.
One thing that the HTMP specification should contain — for the purpose of limiting abuse — are certain restrictions on the length of the Subject and syntax and semantics of the Stub URL, such as maximum length, presence of a domain name and not just an ip address (for human accessibility), an encoded-but-human-decipherable sender email address in the Stub URL path, etc. Violators of these rules can be automatically bounced by ISPs using Stub Blacklists (SBL), or automatically deleted or demoted by individual email users using their personal SBL (analogously to SWL).
Step 2: The HTMP plugin is improved to modify the retrieval mechanism of the email client to implement various schemes of SWL and SBL. For whitelisted Stubs, the plugin can automatically retrieve the original message and replace the Stub Email with it, as if there were no such thing as HTMP. For non-whitelisted Stubs, upon viewing — or, depending on user settings, retrieval — of the Stub Email, the plugin can cause the client to display the Stub URL and allow the user to choose to either ignore the email, blacklist the Stub, or retrieve the original message. During this second phase of early adoption, non-Stub Emails are left untouched by the plugin and thus treated normally by HTMP-ified email clients.
Step 3: Popular email servers begin to support server-side SBL.
Step 4: Popular email service providers begin to support HTMP in their web-based email clients, providing PUT-able web directories to their subscribers.
Step 5: Adoption among end-users becomes widespread, and more and more people begin to configure their email clients to automatically filter out and delete email messages that do not contain the HTMP-Version header.
Step 6: The latest versions of all popular email clients natively support HTMP.
Step 7: Popular email service providers begin to reject non-HTMP-conforming email messages.
Step 8: The days of disconnected, free-floating, kiss-and-wave email messages is over. If you want to say something substantial to somebody, you have to tell her exactly where to find you. Non-HTMP-conforming emails fizzle out because nobody’s reading them. All Spammers, Scammers, and Phishers seek shelter in countries you’ve never heard of and continue to find ways to send out trillions of junk emails. But all the unsolicited messages are super-short Stub Emails that link to domain names you’ve never heard of — or never cared to hear of — and you can complain to ICANN if they bother or scam you, and ICANN can cause their domain name to be redirected to a wholesome website displaying helpful Internet safety tips — like not to talk to strangers or click on strange links.
And thus the Spam Problem is solved, spammers can go back to being run-of-the-mill street thugs, and we can hire more cops to protect the streets rather than having to put up with cluttered Inboxes.
Not only does HTMP force bad people to find other things to do, it also allows good people to do things they couldn’t do before. For example, with HTMP, when you want to email your friends and family five photos of your newborn baby, you don’t have to worry about them muttering under their breath as they wait unusually long for their email to download — not knowing whether or not their waiting will be worthwhile. Instead, they’ll get a short Stub Email (read “short stubby mail") that downloads instantaneously, has a Subject of something like “Hot out of the oven: 7 pounds 3 ounces!!!”, and a Stub perhaps like this:
http://nathancheng.com/nathan_w_cheng@yahoo.com/lskjdfLJFLdoifjjpoiuc.eml
Your friends and family can immediately be sure that it is a message from you, and can retrieve it should they choose. Using a HTTP HEAD request, their email client could also tell them how large is the original email that is waiting for them, and based on that they can decide to retrieve it now, later, or not at all.
I can go on and on with many other interesting new possibilities that HTMP brings to email. For example, the HTTP DELETE command can finally be put to good use: how many times have you hit the “send” button only to have a sinking feeling in your stomach that you just did something horribly wrong? Well, now you can correct your mistake by quickly issuing a HTTP DELETE command to delete the message from your HTMP Outbox (hopefully you didn’t say anything stupid in your Subject line, because even HTMP can’t help you with that); or, if you’ve got time to redact, you could simply PUT a new message in its place. Another example of a new possibility is the ability see who retrieved your email and when (ever heard of log files)? But the log file only lists the ip address of the request, you say, there’s no way to tell who actually retrieved the email, you say. Well, it’s already a given that you’ll have BASIC AUTH protecting PUTs and DELETEs, who says you can’t also have recipient-specific BASIC AUTH for your GETs? Then you’ll know. Another new possibility: every HTMP Outbox, by convention, could also function as a blog: when a smart-aleck script kiddy tries to retrieve a directory listing of your HTMP Outbox, he can instead be treated to an index.html file that lists HTMP Outbox entries that you consider to be public; or it could simply be convention that the “/public” subdirectory of every HTMP Outbox is open for all to view, and you can put your blog entries there. So send someone an email, and they automatically know where to find your blog. The possibilities are endless, but I will stop here, for now.
Thanks to Robert Taylor for reviewing my idea, calling it “Hypertext Mail Protocol” instead of the ridiculous name that I had proposed, helping me think through it, encouraging me to publish it, and editing the final draft.
This is store and retrieve. I proposed that scheme under the name of “weemail” two years ago. You do not need the web. You can also use SMTP/FTP. It was the system I first used in 1977 on Xerox machines. I think (and I am not alone) this is the only solution to control spam because it is architectural.
However, we spent time on considering its implementation (I had a site in French for reporting that). Your proposition works, but it can be implemented better in using “weemail servers” and weemail more sophisticated format (I consider your stub mail as a subset of our weemail: same approach but free additions in the mail sent, and more complete architecture). Also considered the OPES additional possibilities (Open Pluggable Edge Services).
An open format permits to give a better subject, multilingual description, dynamic mailing lists, autofilters, etc. etc.
Incidently it may help increasing the spam noise. But it permits to decrease its interest for the spammer.
The problem is to find someone to technically document and develop it.
BTW the traffic reduction is substantial. But one BIG objection consistently received from internuts: sender knows if destinee read the mail and when. This is considered as a chocking idea. It can be is a major privacy violation.
NB. You can propose a very simple other solution to another problem, along the same lines. HTWHOIS.
I am interested if you carry some work on this.