Update 2004-09-02: See the news story "Apache Software Foundation Rejects Microsoft Patent License Agreement for Sender ID."

The IETF has released a revised version of the Internet Draft MTA Authentication Records in DNS from the MARID Working Group, now called the 'Sender ID' specification.

Jointly authored by Jim Lyon (Microsoft) and Meng Weng Wong (Pobox.com), the Sender ID draft represents a merger of the Sender Policy Framework (SPF) specification and Microsoft's Caller ID for E-mail proposal. The authors "hope to simplify industry adoption of effective e-mail authentication technology, thereby helping more swiftly provide greater spam protection to e-mail users worldwide."

Meng Weng Wong has authored a separate informational I-D Behind The Curtain: An Apology for Sender ID. It explains that "Sender ID follows from a set of design decisions; those decisions were motivated by philosophical, engineering, and political considerations. The document reviews some of the important choice that distinguish Sender ID from alternative possibilities in the same space."

Motivation for the Sender ID draft is presented in the abstract: "Internet mail suffers from the fact that much unwanted mail is sent using spoofed addresses — 'spoofed' in this case means the address is used without the permission of the domain owner." The Sender ID document describes mechanisms by which a domain owner can publish its set of outgoing MTAs [Mail Transfer Agents], mechanisms by which SMTP servers can determine what email address is allegedly responsible for most proximately introducing a message into the Internet mail system, and whether that introduction is authorized by the owner of the domain contained in that email address."

One part of the proposal's decision model involves finding a purported responsible address and extracting the domain part of the purported responsible address, called a purported responsible domain. Then an E-mail Policy Document for the purported responsible domain would be located. The E-mail Policy Document is modeled by an XML infoset that contains, among other things, a definition of the four-argument client authorization function. The Sender ID draft "describes those parts of the XML infoset that define the mail acceptance function. The infoset may contain other information relating to e-mail; this other information may be the subject of future IETF consensus processes."

Industry experts are by no means agreed that the Sender ID proposal constitutes a real solution to the spamming problem, or that the net effect will be judged positive by all parties. The authors of the specification assert that the design "is carefully tailored to ensure that the overwhelming majority of legitimate emailers, remailers and mailing list operators are already compliant."

Bibliographic Information

• MTA Authentication Records in DNS. aka Sender ID. IETF MARID Working Group. Internet Draft. By Jim Lyon (Microsoft Corporation) and Meng Weng Wong (Pobox.com, Singapore). IETF Reference: 'draft-ietf-marid-core-01.txt'. June 2004, expires December 2004. 27 pages.

• Behind The Curtain: An Apology for Sender ID. IETF Internet Draft. By Meng Weng Wong (Pobox.com, Singapore). Reference: 'draft-ietf-marid-rationale-00.txt'. June 2004, expires September 2004. 19 pages.

  The architecture of Sender ID follows from a set of design decisions. Those decisions were motivated by philosophical, engineering, and political considerations. This document reviews some of the important choice that distinguish Sender ID from alternative possibilities in the same space.

• Sender Policy Framework (SPF): A Convention to Describe Hosts Authorized to Send SMTP Traffic. By Meng Weng Wong (pobox.com, Singapore) and Mark Lentczner (Mountain View, CA, USA). IETF Internet Draft. Category: Experimental. 40 pages. IETF source: http://www.ietf.org/internet-drafts/draft-mengwong-spf-01.txt.

  Abstract: "Email address forgery is a problem on the Internet today. Domain owners want to control the use of their names in email, but are helpless because they lack the means. This document introduces a language for domains to make email-related declarations in DNS. It defines in detail one possible sender authentication scheme for domains to describe the hosts from which they send mail. SMTP receivers can use this scheme to detect sender forgery..."

• Caller ID for E-mail. By Robert Atkinson (Microsoft Corporation). IETF Internet Draft. Category: Experimental. May 2004, expires November 2004. Reference: 'draft-atkinson-callerid-00.txt'. 38 pages. IETF source: http://www.ietf.org/internet-drafts/draft-atkinson-callerid-00.txt. Note the patent/license declaration.

  Abstract: "When e-mail is handed off today from one organization to another, as a rule no authentication of the sender of the e-mail or the computers delivering it on the sender's behalf takes place. There are some simple steps which can be taken to significantly alleviate this problem, steps which mimic within the e-mail infrastructure the caller ID mechanism found in today's telephone system. This proposal specifies that in addition to today's practice of publishing in DNS the addresses of their incoming mail servers, administrators of domains also publish the addresses of their outgoing mail servers, the addresses of the computers from which mail legitimately originating from that domain will be sent. This information will then be used in enhancements to software that receives incoming mail to verify that computers handing off a message to them in fact are authorized to do so by the legitimate administrator of the domain from which the message is purported to have been sent..."

From the Sender ID Internet Draft (-marid-core-01)

"Today, a huge majority of unwanted email contains headers that lie about the origin of the mail. This is true of most spam and substantially all of the virus email that is sent.

This document describes a mechanism such that receiving MTAs, MDAs and/or MUAs can recognize mail in the above category and take appropriate action. For example, an MTA might refuse to accept a message, an MDA might discard a message rather than placing it into a mailbox, and an MUA might render that message in some distinctive fashion.

In order to avoid further fragmentation of the Internet email system, it is necessary that the Internet community as a whole come to a consensus as to what mail senders should do to make their mail appear non-spoofed, and how mail receivers should determine whether mail is spoofed. On the other hand, it is not necessary to reach a consensus regarding the actions that various parties take once a message has been determined to be spoofed. This can be done unilaterally — one agent might decide to discard a spoofed message while another decides to add a disclaimer...

[The Sender ID design mechanism attempts to determine, when] a message is transferred via SMTP between two unrelated parties, whether the SMTP client host has permission to send mail on behalf of the mailbox that allegedly caused the most recent introduction of the message into the mail delivery system. It also seeks to authenticate the mailbox associated with the most recent introduction of a message into the mail delivery system... Given an email message, and given an IP address from which it has been (or will be) received, is the SMTP client at that host address authorized to send that email message?

The E-mail Policy Document [for the purported responsible domain] contains a description of a 'client authorization function'. This function that takes four arguments and returns a seven-valued result. The arguments are: (1) The local-part of an email address; (2) A domain name, called the 'original domain'; (3) A domain name, called the 'current domain'; (4) An IP address (either IPv4 or IPv6). The result of the function is one of the seven values 'pass', 'fail', 'softFail', 'neutral', 'transientError', 'hardError' or 'none'.

An E-mail Policy Document is modeled by an XML infoset that contains, among other things, a definition of the function described in section 3. This function can be used to determine whether a domain owner is willing to take responsibility for e-mail that is sent by a particular SMTP client.

This document describes those parts of the XML infoset that define the mail acceptance function. The infoset may contain other information relating to e-mail; this other information may be the subject of future IETF consensus processes. Any program that interprets the XML Infoset SHOULD ignore any elements whose tags are not understood by the program..."

Summary from Microsoft's Sender ID Reference Page

The Microsoft description from June 23, 2004 provides a PDF version of the IETF MARID WG marid-core-01 specification under the filename senderid.pdf, with the label "Sender ID Draft Specification: MTA Authentication Records in DNS, a specification to address domain spoofing, incorporating Microsoft Caller ID for E-Mail and the Sender Policy Framework (SPF)."

"The Sender ID draft technical specification has been submitted to the Internet Engineering Task Force (IETF) to help address the widespread problem of domain spoofing, and to provide greater protection against phishing schemes. The Sender ID specification is the convergence of Microsoft's Caller ID for E-Mail proposal and Meng Wong's Sender Policy Framework (SPF).

Domain spoofing refers specifically to the use of someone else's domain name when sending a message and is part of the larger problem of spoofing (the practice of forging the sender's address on e-mail messages). Domain spoofing can also be used by malicious individuals in phisher scams that try to lure consumers into divulging sensitive information by pretending the mail is from a trusted party, such as a consumer's bank.

Sender ID will verify that each e-mail message originates from the Internet domain it claims to come from based on the sender's server IP address. Eliminating domain spoofing will help legitimate senders protect their domain names and reputations, and help recipients more effectively identify and filter junk e-mail..."

From the Microsoft Announcement 2004-06-25

Microsoft Corp., author of the Microsoft Caller ID for E-mail proposal, and Meng Weng Wong, co-founder and CTO of Pobox.com and author of the Sender Policy Framework (SPF), have announced today that they have successfully converged the two proposals into one specification named Sender ID and submitted to the Internet Engineering Task Force (IETF) for consideration as an industrywide standard for e-mail authentication as part of the IETF's efforts to define effective industry Internet e-mail standards to address the problem of spam. Sender ID is designed to help verify the source of e-mail to help eliminate domain spoofing and provide greater protection against phishing schemes. By providing a unified specification, Microsoft and Wong hope to simplify industry adoption of effective e-mail authentication technology, thereby helping more swiftly provide greater spam protection to e-mail users worldwide.

"Spoofing," or sending e-mail purporting to be from someone it's not, is an increasingly common and relatively simple way for spammers to try to trick filters. It can also pose a security risk when used to deliver e-mail viruses or phisher scams, which attempt to trick users into divulging personal information such as credit card numbers or account passwords by pretending to be from a legitimate source, such as a user's bank. Sender ID aims to prevent spoofing by confirming what domain a message came from and thereby increase the effectiveness of spam filters.

Under the merged proposal, organizations will publish information about their outgoing e-mail servers, such as IP addresses, in the Domain Name System (DNS) using the industry-standard XML format. Backward compatibility will be provided for the many domains that have already published information in the SPF TXT format.

The converged specification will enable receiving systems to test for spoofing at the message transport (SMTP) level, or envelope, as originally proposed in SPF, as well as in the message body headers, as originally proposed in Caller ID. Testing for spoofing at the message transport level allows receiving systems to block some spam messages before they are sent. Checking the message body headers is necessary in cases in which a deeper examination of the message contents is required to detect spoofing and phishing.

"Twenty thousand domains have already published SPF records," Wong said. "Sender ID automatically gives those domains additional protection from phishing and spoofing as well."

"Over half of the e-mail targeting our Hotmail customers today come from spoofed domains, and we are committed to taking this trick away from spammers," said Ryan Hamlin, general manager of the Anti-Spam Technology and Strategy group at Microsoft. "We very much look forward to working with the IETF and others in the industry to help swiftly move forward in establishing e-mail authentication standards as a key step toward containing the spam problem for customers worldwide."

Momentum for this kind of standard is building. Earlier this week, the Anti-Spam Technical Alliance (ASTA) formed by key industry stakeholders such as America Online, British Telecom, Comcast, EarthLink, Microsoft and Yahoo! published a host of recommendations for the industry to effectively address the spam problem and recognized the need for the broad adoption of e-mail authentication mechanisms, including the kind of IP-based approaches put forth in Sender ID. Also, in addition to Microsoft, many leading technology companies including America Online, Brightmail, Cloudmark, EarthLink, IronPort, Sendmail, Tumbleweed and VeriSign have already committed to quick adoption and implementation of Sender ID as it moves toward becoming an industry standard.

"AOL is pleased to see the merger between these two proposals, which will help provide enhanced identity in e-mail. We are glad the new standard is fully backwards-compatible with the existing SPF, which is in use by tens of thousands of domains on the Internet already," said Carl Hutzler, director of Antispam Operations at AOL, an ASTA member and key contributor to Wong's SPF proposal. "We look forward to continuing our work with Mr. Wong, Microsoft, our industry partners and the IETF to ensure swift adoption of SPF and the new combined standard."

To be more effective in the fight against junk e-mail, filters need additional information that is not available in e-mail messages today. By making simple but important changes to the e-mail infrastructure, such as those outlined in the Sender ID proposal, greater certainty can be provided about the origin of an e-mail message and enable legitimate senders to more clearly distinguish themselves from spammers.