[Docs] [txt|pdf] [Tracker] [Email] [Diff1] [Diff2] [Nits]

Versions: 00 01 02 03

Internet Draft                                          Paul Hoffman
draft-hoffman-imaa-03.txt                                 IMC & VPNC
October 8, 2003                                     Adam M. Costello
Expires April 8, 2004                                    UC Berkeley



       Internationalizing Mail Addresses in Applications (IMAA)

Status of this Memo

     This document is an Internet-Draft and is in full conformance with
     all provisions of Section 10 of RFC2026.

     Internet-Drafts are working documents of the Internet Engineering
     Task Force (IETF), its areas, and its working groups.  Note
     that other groups may also distribute working documents as
     Internet-Drafts.

     Internet-Drafts are draft documents valid for a maximum of six
     months and may be updated, replaced, or obsoleted by other documents
     at any time.  It is inappropriate to use Internet-Drafts as
     reference material or to cite them other than as "work in progress."

     The list of current Internet-Drafts can be accessed at
     http://www.ietf.org/ietf/1id-abstracts.txt

     The list of Internet-Draft Shadow Directories can be accessed at
     http://www.ietf.org/shadow.html.


Abstract

     The Internationalizing Domain Names in Applications (IDNA)
     specification describes how to process domain names that contain
     characters outside the ASCII repertoire.  A user who has a non-ASCII
     domain name may want to use it in an Internet mail address that
     contains non-ASCII characters not only in the domain part but also
     in the local part (the part to the left of the "@").  This document
     describes how to use non-ASCII characters in local parts.  It
     defines internationalized local parts (ILPs), internationalized mail
     addresses (IMAs), and a mechanism called IMAA for handling them in a
     standard fashion.


1. Introduction

     A mail address consists of local part, an at-sign (@), and a domain
     name.  The IDNA specification [IDNA] describes how to handle domain
     names that have non-ASCII characters.  This document describes how
     to handle non-ASCII characters in the rest of the mail address.

     This document explicitly does not discuss internationalization of
     display names and comments in mail addresses that appear in message
     headers [MSGFMT].  MIME part three [MIME3] describes how use an
     extended set of characters in message headers, and this document
     does not alter that specification.

     This document is being discussed on the ietf-imaa mailing list.  See
     <http://www.imc.org/ietf-imaa/> for information about subscribing
     and the list's archive.

1.1 Relationship to IDNA

     This document relies heavily on IDNA for both its concepts and
     its justification.  This document omits a great deal of the
     justification and design information that might otherwise be found
     here because it is identical to that in IDNA.  Anyone reading this
     document needs to have first read [IDNA], [PUNYCODE], [NAMEPREP],
     and [STRINGPREP].

     There are a few key differences between the way IMAA treats local
     parts of mail addresses and the way IDNA treats domain names.

       - The ACE infix for internationalized local parts is different
         from the ACE prefix for internationalized domain labels.

       - Domain names have an intrinsic segmentation into labels, and
         are already segmented before transformations are performed.
         Local parts, on the other hand, have no intrinsic segmentation.
         The transformations on local parts perform a segmentation
         internally, but it has no external significance.

       - There is no UseSTD3ASCIIRules flag for local parts.

     One apparent difference that is not really a difference is the
     handling of quoting mechanisms.  IDNA did not discuss quoting
     because the phrase "domain label" is presumed to refer to a simple
     literal string.  [DNS] defines domain labels in terms of their
     literal form (which is used in DNS protocol messages), and later
     introduces a quoting syntax for representing domain labels in master
     files, but there is never any doubt that the domain label itself is
     a simple unstructured sequence.  It goes without saying that domain
     labels obtained from contexts that use quoting (like master files)
     need to be reduced to their literal form before any processing is
     done on them.

     Local parts, on the other hand, are defined in [MSGFMT] and [SMTP]
     in terms of their quoted form, as they appear in message headers and
     SMTP commands.  Later it is stated that the quotation characters are
     not really part of the local part.  To avoid any ambiguity, IMAA
     explicitly discusses the process of dequoting and requoting local
     parts.


2. Terminology

     The key words "MUST", "SHALL", "REQUIRED", "SHOULD", "RECOMMENDED",
     and "MAY" in this document are to be interpreted as described in
     RFC 2119 [KEYWORDS].

     Code point, Unicode, and ASCII are defined in [IDNA].

     The "protected code points" are 0..40, 5B..60, 7B..7F (in other
     words, those corresponding to ASCII characters other than letters
     and digits).

     A "mail address" consists of a local part, an at-sign, and a domain
     name, in that order.  The exact details of the syntax depend on the
     context; for example, a "mailbox" in [SMTP] and an "addr-spec" in
     [MSGFMT] are both mail addresses, but they define slightly different
     syntaxes for local parts and domain names.

     A "dequoted local part" is the simple literal text string that
     is the intended "meaning" of a local part after it has undergone
     lexical interpretation.  A dequoted local part excludes optional
     white space, comments, and lexical metacharacters (like backslashes
     and quotation marks used to quote other characters).  Dequoted local
     parts are generally not allowed in protocols (like SMTP commands and
     message headers), but they are needed by IMAA as an intermediate
     form.  The dequoted form of X is sometimes written dequote(X).

     An "internationalized local part" (ILP) is anything that satisfies
     both of the following conditions:  (1) It conforms to the same
     syntax as a non-internationalized local part except that non-ASCII
     Unicode characters are allowed wherever ASCII letters are allowed.
     (2) After it has been dequoted, the ToASCII operation can be applied
     to it without failing (see section 4).  The term "internationalized
     local part" is a generalization, embracing both old ASCII local
     parts and new non-ASCII local parts.  Although most Unicode
     characters can appear in internationalized local parts, ToASCII will
     fail for some inputs.  Anything that fails to satisfy condition 2 is
     not a valid internationalized local part.

     A "traditional local part" is a local part that contains only ASCII
     characters and whose dequoted form would be left unchanged by the
     ToUnicode operation (see section 4).

     An "internationalized mail address" (IMA) consists of an
     internationalized local part, an at-sign, and an internationalized
     domain name [IDNA], in that order.

     Equivalence of local parts is defined in terms of the dequoted form
     (see above) and the ToASCII operation, which constructs an ASCII
     form for a given dequoted local part (whether or not the local part
     was already an ASCII local part).  Two traditional local parts X
     and Y are equivalent if and only if dequote(X) and dequote(Y) are
     exactly identical.  (That is not a new rule, it is inferred from
     [SMTP] and [MSGFMT].)  For internationalized local parts X and Y
     that are not both traditional, they are defined to be equivalent if
     and only if ToASCII(dequote(X)) matches ToASCII(dequote(Y)) using
     a case-insensitive ASCII comparison.  Unlike traditional local
     parts, non-traditional internationalized local parts are always
     case-insensitive.

     Two internationalized mail addresses are equivalent if and only
     if their local parts are equivalent (according to the previous
     definition) and their domain parts are equivalent (according to
     IDNA).

     To allow internationalized labels to be handled by existing
     applications, IDNA uses an "ACE local part" (ACE stands for ASCII
     Compatible Encoding).  An ACE local part is an internationalized
     local part that can be rendered in ASCII and is equivalent to an
     internationalized local part that cannot be rendered in ASCII.
     Given any internationalized local part (in dequoted form) that
     cannot be rendered in ASCII, the ToASCII operation will convert it
     to an equivalent ACE local part (whereas an ASCII local part will
     be left unaltered by ToASCII).  ACE local parts are unsuitable for
     display to users.  The ToUnicode operation will convert any local
     part (in dequoted form) to an equivalent non-ACE local part.  In
     fact, an ACE local part is formally defined to be any local part
     that the ToUnicode operation would alter (whereas non-ACE local
     part are left unaltered by ToUnicode).  The ToASCII and ToUnicode
     operations are specified in section 4.

     The "ACE infix" is defined in this document to be a string of ASCII
     characters that occurs within every encoded segment in a dequoted
     ACE local part.  It is specified in section 5.

     A "mail address slot" is defined in this document to be a protocol
     element or a function argument or a return value (and so on)
     explicitly designated for carrying a mail address (or part of a mail
     address).  Mail address slots exist, for example, in the MAIL and
     RCPT commands of the SMTP protocol, in the To: and Received: fields
     of message headers, and in a mailto: URI in the href attribute of
     an HTML <A> tag.  General text that just happens to contain an mail
     address is not a mail address slot; for example, a mail address
     appearing in the plain text body of a message is not occupying a
     mail address slot.

     An "IMA-aware mail address slot" is defined in this document to
     be a mail address slot explicitly designated for carrying an
     internationalized mail address as defined in this document. The
     designation may be static (for example, in the specification of
     the protocol or interface) or dynamic (for example, as a result of
     negotiation in an interactive session).

     An "IMA-unaware mail address slot" is defined in this document to be
     any mail address slot that is not an IMA-aware mail address slot.
     Obviously, this includes any mail address slot whose specification
     predates this document.


3. Requirements and applicability

3.1 Requirements

     IMAA conformance means adherence to the following four requirements:

      1) In an internationalized mail address, the following characters
         MUST be recognized as at-signs for separating the local part
         from the domain name:  U+0040 (commercial at), U+FF20 (fullwidth
         commercial at).

      2) Whenever a mail address (or part of a mail address) is put
         into an IMA-unaware mail address slot (see section 2), it MUST
         contain only ASCII characters.  Given an internationalized mail
         address, an equivalent mail address satisfying this requirement
         can be obtained by applying ToASCII to the local part as
         specified in section 4, changing the at-sign to U+0040, and
         processing the domain name as specified in [IDNA].

      3) ACE local parts obtained from mail address slots SHOULD be
         hidden from users when it is known that the environment
         can handle the non-ACE form, except when the ACE form is
         explicitly requested.  When it is not known whether or not the
         environment can handle the non-ACE form, the application MAY
         use the non-ACE form (which might fail, such as by not being
         displayed properly), or it MAY use the ACE form (which will
         look unintelligible to the user).  Given an internationalized
         local part, an equivalent non-ACE local part can be obtained
         by applying the ToUnicode operation as specified in section 4.
         When requirements 2 and 3 both apply, requirement 2 takes
         precedence.

      4) If two mail addresses are equivalent and either one refers to a
         mailbox, then both MUST refer to the same mailbox, regardless of
         whether they use the same form of at-sign.

         Discussion:  This implies that non-ASCII local parts cannot be
         deployed in domains whose mail exchangers are case-sensitive.
         IMAA is designed to work without upgrading mail exchangers,
         but it works only for mail exchangers that treat ASCII local
         parts as case-insensitive (which is the common and preferred
         behavior).  All local parts received by an IMA-unaware
         mail exchanger are ASCII, either traditional or ACE, and a
         case-insensitive exchanger will automatically obey requirement 4
         without being aware of it.  Case-sensitive exchangers will not
         correctly handle ACE local parts, but administrators can simply
         refrain from creating ACE local parts in those domains.  This is
         necessary because a round-trip through ToUnicode and ToASCII is
         not case-preserving, and therefore the result might refer to a
         different mailbox (in violation of requirement 4) if interpreted
         by a case-sensitive mail exchanger.

3.2 Applicability

     IMAA is applicable to all mail addresses in all mail address slots
     except where it is explicitly excluded.

     This implies that IMAA is applicable to protocols that predate IMAA.
     Note that mail addresses occupying mail address slots in those
     protocols MUST be in ASCII form (see section 3.1, requirement 2).

3.2.1. Case-sensitive local parts

     IMAA does not apply to local parts that are interpreted
     case-sensitively (see section 3.1 requirement 4).

3.2.2. Local parts versus domain names

     The IMAA ToASCII and ToUnicode operations apply to local parts, not
     to domain labels.  The IDNA ToASCII and ToUnicode operations apply
     to domain labels, not to local parts.  There exist conventions for
     transplanting local parts into domain labels (in DNS SOA records,
     for example), and there may exist conventions for transplanting
     domain names into local parts.  Such conventions that predate
     IMAA are IMA-unaware, and therefore the domain labels receiving
     the transplanted local parts and the local parts receiving the
     transplanted domain names are IMA-unaware slots.  Therefore the
     strings MUST be in ASCII form before they are transplanted.  If they
     were transplanted in non-ASCII form they would risk being passed
     through the wrong ToASCII operation.


4. Conversion operations

     An application converts a local part put into an IMA-unaware mail
     address slot or displayed to a user.  This section specifies the
     steps to perform in the conversion, and the ToASCII and ToUnicode
     operations.

     The input to ToASCII or ToUnicode is a dequoted local part that is a
     sequence of Unicode code points (remember that all ASCII code points
     are also Unicode code points).  If a local part is represented using
     a character set other than Unicode or US-ASCII, it will first need
     to be transcoded to Unicode.

     Starting from a local part, the steps that an application takes to
     do the conversions are:

      1) Decide whether the local part is a "stored string" or a "query
         string" as described in [STRINGPREP] (see section 6 below for a
         discussion).  If this conversion follows the "queries" rule from
         [STRINGPREP], set the flag called "AllowUnassigned".

      2) Save a copy of the local part.

      3) Dequote the local part; that is, perform lexical interpretation
         and remove all nonliteral characters.  For example, for a
         local part that uses the lexical syntax of [SMTP] or [MSGFMT],
         unfold it, remove comments and unquoted white space, and remove
         backslashes and quotation marks used to quote other characters.
         The result is a simple literal text string.

      4) Process the string with either the ToASCII or the ToUnicode
         operation as appropriate.  Typically, you use the ToASCII
         operation if you are about to put the local part into an
         IMA-unaware slot, and you use the ToUnicode operation if you are
         displaying the local part to a user.

      5) If step 4 had no effect on the string, and if the saved local
         part from step 2 is a valid representation of the string in the
         destination context, then the saved local part SHOULD be used,
         otherwise proceed to step 6.

      6) Apply whatever quoting is needed in the destination context
         (if any).  For "mailbox" slots [SMTP] and "addr-spec" slots
         [MSGFMT] the following action suffices:  If the string contains
         any control characters, spaces, or specials [MSGFMT], or if it
         begins or ends with a dot, or contains two consecutive dots,
         then convert it to a quoted-string: insert a backslash before
         every quotation mark and backslash, then enclose the string with
         quotation marks.

     The destination context might also impose a length restriction.
     Depending on whether the restriction applies to the quoted form or
     the dequoted form, the application might want to check the length at
     the very end or just after step 4.

     This process is designed to handle quoting and dequoting when
     necessary; however, local parts that need quoting can be difficult
     for humans to use.  This is already true for ASCII local parts,
     and is even more true for internationalized local parts.  It is
     inadvisable to create such local parts if they are to be used by
     humans.

     The following two subsections define the ToASCII and ToUnicode
     operations that are used in step 4.

     In ToASCII and ToUnicode, the operation of Nameprep is split into
     two halves that are applied at different times.  One half consists
     of Nameprep steps 1 (map) and 2 (normalize); the other half consists
     of Nameprep steps 3 (prohibit) and 4 (check bidi).  The split is
     easy to remember because steps 1 and 2 are string transformations
     that can never fail, while steps 3 and 4 are checks that do
     nothing but succeed or fail.

     This description of the protocol uses specific procedure names,
     names of flags, and so on, in order to facilitate the specification
     of the protocol.  These names, as well as the actual steps of the
     procedures, are not required of an implementation.  In fact, any
     implementation which has the same external behavior as specified in
     this document conforms to this specification.

4.1 ToASCII

     The ToASCII operation takes a sequence of Unicode code points that
     make up a dequoted local part and transforms it into a sequence of
     code points in the ASCII range (0..7F).  If ToASCII succeeds, the
     original sequence and the resulting sequence are equivalent dequoted
     local parts.

     It is important to note that the ToASCII operation can fail.
     ToASCII fails if any step of it fails.  If any step of the
     ToASCII operation fails, that string MUST NOT be used as an
     internationalized local part.  The method for dealing with this
     failure is application-specific.

     The inputs to ToASCII are a sequence of code points, and the
     AllowUnassigned flag.  The output of ToASCII is either a sequence of
     ASCII code points or a failure condition.

     ToASCII never alters a sequence of code points that are all in the
     ASCII range to begin with.  Applying the ToASCII operation multiple
     times has exactly the same effect as applying it just once.

     ToASCII consists of the following steps:

      1. If the sequence contains any code points outside the ASCII range
         (0..7F) then proceed to step 2, otherwise stop, leaving the
         sequence unchanged.

      2. Perform [NAMEPREP] steps 1 (map) and 2 (normalize).

      3. If the sequence is empty then stop, leaving an empty result.

      4. Divide the sequence into segments.  Segment boundaries occur
         wherever a protected code point is adjacent to a non-protected
         code point, and nowhere else.  (Therefore segments are never
         empty, and they alternate between segments containing only
         protected code points and segments containing only non-protected
         code points.)

      5. For each segment perform the following substeps:

         (a) If the segment contains any code points outside the ASCII
             range (0..7F) then proceed to substep b, otherwise leave the
             segment unchanged.

         (b) Perform [NAMEPREP] steps 3 (prohibit) and 4 (check bidi),
             and fail if there is an error.  The AllowUnassigned flag is
             used in [NAMEPREP] step 3.

         (c) Encode the sequence using the encoding algorithm in
             [PUNYCODE] and fail if there is an error.

         (d) Verify that the result contains no more than 59 code points.

         (e) The sequence will contain at most one instance of U+002D
             (hyphen-minus).  If it is absent then prepend the ACE infix;
             otherwise verify that the ACE infix does not already occur
             before the hyphen-minus, and substitute the ACE infix in
             place of it.

      6. Rejoin the segments into a single sequence.


4.2 ToUnicode

     The ToUnicode operation takes a sequence of Unicode code points that
     make up a dequoted local part and returns a sequence of Unicode code
     points.  If the input sequence is a dequoted local part in ACE form,
     then the result is an equivalent dequoted internationalized local
     part that is not in ACE form, otherwise the original sequence is
     returned unaltered.

     ToUnicode never fails.  If any step fails, then the original input
     sequence is returned immediately in that step.

     The Punycode decoder can never output more code points than it
     inputs, but Nameprep can, and therefore ToUnicode can.  Note that
     the number of octets needed to represent a sequence of code points
     depends on the particular character encoding used.

     The inputs to ToUnicode are a sequence of code points, and the
     AllowUnassigned flag.  The output of ToUnicode is a sequence of code
     points.

     ToUnicode consists of the following steps:

      1. If the sequence contains any code points outside the ASCII range
         (0..7F) then proceed to step 2, otherwise skip to step 3.

      2. Perform [NAMEPREP] steps 1 (map) and 2 (normalize).

      3. Verify that the sequence is nonempty.

      4. Divide the sequence into segments (same as step 4 of ToASCII).

      5. For each segment perform the following substeps:

         (a) If the ACE infix does not occur anywhere within the segment
             then leave the segment unchanged, otherwise save a copy of
             the segment and proceed to substep b.

         (b) If the ACE infix occurs at the very beginning of the segment
             then remove it, otherwise substitute U+002D (hyphen-minus)
             in place of the first occurrence of the ACE infix.

         (c) Decode the segment using the decoding algorithm in
             [PUNYCODE] and catch any error.  If there was an error then
             restore the saved copy from substep a.

      6. Verify that at least one segment was altered in step 5.

      7. Rejoin the segments into a single sequence, and save a copy of
         the result.

      8. Apply ToASCII to the current sequence and to a copy of the
         original input.

      9. Verify that the two results of step 8 match using a
         case-insensitive ASCII comparison.

     10. Return the saved copy from step 7.


5. ACE infix

     [[ Note to the IESG and Internet Draft readers: The two uses of the
     string "0iesg1" below are to be changed at time of publication to an
     infix that fulfills the requirements in the first paragraph.  IANA
     will assign this value. ]]

     The ACE infix, used in the conversion operations (section 4), is
     two ASCII letters surrounded by two distinct ASCII digits.  The
     ToASCII and ToUnicode operations MUST recognize the ACE infix in a
     case-insensitive manner.

     The ACE infix for IMAA is "0iesg1" or any capitalization thereof.

     This means that an ACE local part might be
     "foobar!de0iesg1jg4avhby1noc0d!0iesg1d9juau41awczczp", where
     "de-jg4avhby1noc0d" and "d9juau41awczczp" are the results of the
     encoding steps in [PUNYCODE].

     While every encoded segment (segment that would be altered by
     ToUnicode) within an ACE local part contains the ACE infix, not
     every segment containing the ACE infix is an encoded segment.
     Segments that contain the ACE infix but are not encoded segments
     will confuse users, and local parts containing such segments SHOULD
     NOT be used as mailbox names.


6. Stored strings and query strings

     [STRINGPREP] prohibits unassigned code points in "stored strings"
     and allows them in "query strings", but concedes that "different
     Internet protocols use strings very differently, so these terms
     cannot be used exactly in every protocol that needs to use
     stringprep".  In the context of IMAA, the following clarifications
     apply.

     A string that assigns/creates the name of an object is a "stored
     string".  A string that merely refers to an object using a name that
     is presumed to have been assigned/created elsewhere is a "query
     string".

     Examples of stored strings:

       * In a mail server configuration file/database, the strings that
         create the mail addresses associated with the local mailboxes.
         (These mail addresses might be defined in pieces: the domain
         parts might be defined by a set of local domains, and the local
         parts might be defined by a separate set of user names and
         aliases, but the net effect is that these strings create a set
         of mail addresses, and are therefore stored strings.)

       * The msg-id in the Message-ID: field of a message header.

     Examples of query strings:

       * A mail address in the From: or To: or Reply-To: field of a
         message header.

       * A mail address in the MAIL or RCPT command of SMTP.

       * A mail address in a personal address book.

       * A msg-id in the In-Reply-To: or References: field of a message
         header.


7. References

7.1 Normative references

     [IDNA]       Faltstrom, P., Hoffman, P. and A. Costello,
                  "Internationalizing Domain Names in Applications
                  (IDNA)", RFC 3490, March 2003.

     [KEYWORDS]   Bradner, S., "Key words for use in RFCs to Indicate
                  Requirement Levels", BCP 14, RFC 2119, March 1997.

     [MSGFMT]     Resnick, P., "Internet Message Format", RFC 2822,
                  April 2001.

     [NAMEPREP]   Hoffman, P. and M. Blanchet, "Nameprep: A Stringprep
                  Profile for Internationalized Domain Names (IDN)",
                  RFC 3491, March 2003.

     [PUNYCODE]   Costello, A., "Punycode: A Bootstring encoding of
                  Unicode for use with Internationalized Domain Names in
                  Applications (IDNA)", RFC 3492, March 2003.

     [SMTP]       Klensin, J., "Simple Mail Transfer Protocol", RFC 2821,
                  April 2001.

     [STRINGPREP] Hoffman, P. and M. Blanchet, "Preparation of
                  Internationalized Strings ("stringprep")", RFC 3454,
                  December 2002.

7.2 Informative references

     [DNS]        Mockapetris, P., "Domain names - concepts and
                  facilities", STD 13, RFC 1034 and "Domain names -
                  implementation and specification", STD 13, RFC 1035,
                  November 1987.

     [MIME3]      Moore, K., "MIME (Multipurpose Internet Mail
                  Extensions) Part Three: Message Header Extensions for
                  Non-ASCII Text", RFC 2047, November 1996.


8. Security considerations

     Because this document normatively refers to [IDNA], [NAMEPREP],
     [PUNYCODE], and [STRINGPREP], it includes the security
     considerations from those documents as well.

     Internationalized local parts will cause mail addresses to become
     longer, and possibly make it harder to keep lines in a header under
     78 characters.  Lines that are longer than 78 characters (which
     is a SHOULD specification, not a MUST specification, in RFC 2822)
     could possibly cause mail user agents to fail in ways that affect
     security.


9. IANA considerations

     IANA will assign the ACE infix in consultation with the IESG.


10. Authors' addresses

     Paul Hoffman
     Internet Mail Consortium and VPN Consortium
     127 Segre Place
     Santa Cruz, CA 95060  USA
     phoffman@imc.org

     Adam M. Costello
     University of California, Berkeley
     http://www.nicemice.net/amc/


Html markup produced by rfcmarkup 1.129c, available from https://tools.ietf.org/tools/rfcmarkup/