Internet Draft: Qpopper MIME Mangling and Macro Extensions to POP3 R. Gellens Document: draft-gellens-pop-mangle-02.txt Qualcomm Expires: May 17, 2009 November 17, 2008 Intended status: Informational The Qpopper MIME Mangling and Macro Extensions to POP3 Status of this Memo This Internet-Draft is submitted to IETF in full conformance with the provisions of BCP 78 and BCP 79. Internet-Drafts are working documents of the Internet Engineering Task Force (IETF), its areas, and its working groups. Note that other groups may also distribute working documents as Internet- Drafts. Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress." The list of current Internet-Drafts can be accessed at http://www.ietf.org/ietf/1id-abstracts.txt The list of Internet-Draft Shadow Directories can be accessed at http://www.ietf.org/shadow.html. Copyright Notice Copyright (c) 2008 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Abstract This document describes two extensions to the POP protocol that have been supported for several years by some client and servers. One extension, MANGLE, allows clients to request that MIME body parts be removed or converted to a simpler format, that only selected headers Gellens Expires January 2008 [Page 1] Internet Draft Mangle and Macro July 2007 be transmitted, and/or for the message to be transcoded to a different character set. The other extension, MACRO, allows clients to define macros which are expanded when used, saving repetitive transmissions. The MANGLE extension has been useful in at least two situations: it allows clients on constrained devices to avoid downloading body parts which cannot be used on the device, and clients which use the TOP command to "peek" at messages can have the preview transformed into more usable content, as well as avoiding the transmission of undesired headers. The MACRO extension has been especially useful in conjunction with MANGLE, since a MANGLE request can become relatively lengthy. Gellens Expires January 2008 [Page 2] Internet Draft Mangle and Macro July 2007 Table of Contents 1 Conventions Used in this Document . . . . . . . . . . . . . . 3 2. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3 3. MIME Mangling (MANGLE) . . . . . . . . . . . . . . . . . . . 4 3.1. MANGLE Capability . . . . . . . . . . . . . . . . . . . 6 4. Macro Definition (MDEF) . . . . . . . . . . . . . . . . . . . 7 4.1. MACRO Capability . . . . . . . . . . . . . . . . . . . . 8 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 9 6. Security Considerations . . . . . . . . . . . . . . . . . . 9 7. Normative References . . . . . . . . . . . . . . . . . . . . 9 8. Informative References . . . . . . . . . . . . . . . . . . . 10 9. Author's Address . . . . . . . . . . . . . . . . . . . . . . 10 1 Conventions Used in this Document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [KEYWORDS]. 2. Introduction Since May of 2000, the Qpopper open-source POP server has supported the MIME mangling (MANGLE) and macro (MACRO) extensions to the [POP3] protocol. These extensions have been useful in at least two situations: o clients on constrained devices can avoid downloading body parts which cannot be used on the device, can have the content transcoded into a more desirable character set, and can have undesired headers removed prior to transmission; and o clients which use the TOP command to "peek" at messages can have the preview transformed into more usable content, and can make the preview more useful by eliminating extraneous headers. The Eudora for PalmOS client (part of the Eudora Internet Suite for PalmOS) makes use of both extensions. The Eudora clients for both MacOS and Windows make use of the MANGLE extension when using the TOP command. Gellens Expires January 2008 [Page 3] Internet Draft Mangle and Macro July 2007 With the prevalence of attachments and formatted mail, these extensions have been useful in solving the following problems: o clients on constrained devices or links often download attachments that waste bandwidth and must be parsed and discarded, since they cannot be used on the device; o clients on slow links or memory-limited devices waste bandwidth receiving undesired headers; o when the TOP command is used to "peek" at messages, especially when using a bandwidth-constrained connection (such as a non-3G mobile link or a slow dialup with a laptop), often the captured portion of the message is unreadable, being a mess of HTML tags (since many clients show these previews in their raw form, and, being truncated, the HTML may not be syntactically valid and hence difficult for the client to parse). Obviously, IMAP [IMAP] provides a more feature-packed protocol and hence allows for a richer user experience. However, IMAP also imposes significant additional costs on server operators (for example, since with IMAP the server maintains the authoritative mail store, the storage requirements are typically much greater than with POP). In general, the extra costs of IMAP are far outweighed by the additional benefits, and hence IMAP is the recommended mechanism for greater functionality, but there are cases where the extra costs cannot be justified or recovered. In those cases, minor extensions to POP to provide additional value to mobile devices may be a useful interim measure until IMAP can be deployed. 3. MIME Mangling (MANGLE) This extension is announced by including MANGLE in the CAPA [POP-EXT] response. The MANGLE extension can be used as a modifier to the TOP, RETR, and LIST commands. The result is to: o Remove MIME body parts o Optionally remove all headers except those requested o Optionally encode in a specified character set The syntax is: mangle = "mangle(" mangling ")" mangling = attribute "=" value *(";" attribute "=" value ) attribute = "text" / "headers" / "charset" / "lang" / "wrap" value = text-type / header-list / charset Gellens Expires January 2008 [Page 4] Internet Draft Mangle and Macro July 2007 text-type = "plain" / "html" / "enriched" header-list = header *("," header) charset = "us-ascii" / iso-8859-r / "utf-8" / "cp1252" / "iso_2022-jp" iso-8859-r = "iso-8859-" range range = "1"-"15" The following command results in transmitting message 10 as a single part of content-type text/html with only those headers which were requested: retr 10 mangle(text=html;headers=to:,cc:,from:,date:) The following command asks the server for the size of message 2 after mangling as a single part of content-type text/html with only those headers which were requested: list 2 mangle(text=html;headers=to:,cc:,from:,date:) The message can be sent as one of three text types: text/plain, text/html, and text/enriched. The source format can be any of these types or text/plain;format=flowed. If the headers are not specified, the default is to return the following ones: "to:", "cc:", "subject:", "from:", "date:", "reply-to:", "x-Priority:", and "x-UIDL:". Note that "lang" and "wrap" are currently ignored. The value supplied with the "headers" attribute is a comma-separated list of desired headers. Header in the message are matched up to what is entered, so "headers=sub" will cause both the "Subject:" and "Subversive:" headers to be returned, if both exist in the message. In order to be sure to match only the exact header desired, clients may wish to include the ":". The "Mime-Version:" and "Content-Type:" headers are always included, and reflect the mangling that has been applied. Note that there must not be any spaces within the 'mangle' string (since spaces are used to separate parameters to POP commands). Case is not significant with the MANGLE extension. 'mangle(text=plain)' is the same as 'mAnglE(TeXT=PLaiN)', 'headers=to' is the same as 'headers=TO'. Examples: top 1 100 mangle(text=plain;headers=to:,subject:) retr 1 mangle(text=plain;headers=to:,subject:) Gellens Expires January 2008 [Page 5] Internet Draft Mangle and Macro July 2007 retr 1 mangle(text=plain;charset=us-ascii;headers=to:,subject:) list mangle(text=plain;charset=us-ascii;headers=to:,subject:) list 1 mangle(text=plain;charset=us-ascii;headers=to:,subject:) Examples of use: The client retrieves message 1 as it exists in the message store: C: retr 1 S: +OK 665 octets S: Received: from foo.example.net (localhost [127.0.0.1]) S: by hotearth.example.net (8.12.10/8.12.3/1.0) with S: ESMTP id j3L0Lq9B014701 S: for ; S: Wed, 20 Apr 2005 17:21:52 -0700 (PDT) S: Received: (from randy@localhost) S: by hotearth.example.net (8.12.10/8.12.1/Submit) id S: j3L0Lqqj014700 for poptest8; S: Wed, 20 Apr 2005 17:21:52 -0700 (PDT) S: Date: Wed, 20 Apr 2005 17:21:52 -0700 (PDT) S: From: Randall Gellens S: Message-Id: <200504210021.j3L0Lqqj014700@hotearth.example.net> S: To: poptest8@example.net S: Subject: test #1 S: X-UIDL: o=6"!2HZ!!LR+!!7C!"! S: Status: RO S: S: This is a test. S: . The client retrieves message 1 as text/html and with only those headers which start with 'foo' (in this case, none): C: retr 1 mangle(text=html;headers=foo) S: +OK Message follows S: Mime-version: 1.0 S: Content-Type: text/html;charset="us-ascii" S: S:
This is a test.
    S: 
S: . Note that some servers (such as Qpopper) also support the "-no-mime" user name hack. As a way to enable MIME-mangling with clients that do not implement MANGLE, add "-no-mime" to the user name. For example, if the userid is "mary", enter it in the client as "mary-no-mime". Gellens Expires January 2008 [Page 6] Internet Draft Mangle and Macro July 2007 3.1. MANGLE Capability CAPA tag: MANGLE Arguments: none Added commands: none Standard commands affected: TOP, RETR, LIST Announced states / possible differences: both / no Commands valid in states: TRANSACTION Specification reference: This document Discussion: The MANGLE capability indicates the optional MANGLE parameter is available for use with the TOP, RETR, and LIST commands. See Section 3 for the details of the MANGLE parameter. 4. Macro Definition (MDEF) This extension is announced by including MACRO in the CAPA [POP-EXT] response. The client uses the MDEF command to define a macro. The server may impose limits on the length of the macro name and/or value and the number of macros. In the case of Qpopper, the macro (name and value plus parenthesis) must be no greater than 500 octets (100 in older versions of the server), and clients may define up to ten macros. Redefining a macro of the same name is permitted; the new macro replaces the old one. Syntax: macro = "MDEF" macro-name "(" macro-value ")" macro-name = 1*printable macro-value = 1*printable printable = Gellens Expires January 2008 [Page 7] Internet Draft Mangle and Macro July 2007 Note: There must not be any spaces within the mdef command (since spaces are used to separate parameters to POP commands). Once defined, the name of a macro can be used as a parameter to any POP command that takes parameters. When used, the macro value replaces the macro name as the command parameter. Example: mdef mang(mangle(text=plain;headers=to:,subject:)) The above example defines a macro with the name 'mang' whose value is "mangle(text=plain;headers=to:,subject:)". One situation in which macros are helpful is when clients using bandwidth-constrained links wish to use the MANGLE extension multiple times, with identical values in each usage. For example, rather than a client entering the idential MANGLE extension as a parameter to every TOP or RETR command, the client first defines the extension as the value of a macro, then uses the macro name with the TOP or RETR commands. Example of use: C: mdef html(mangle(text=html;headers=foo)) S: +OK Macro "html" accepted C: retr 1 html S: +OK Message follows S: Mime-version: 1.0 S: Content-Type: text/html;charset="us-ascii" S: S:
This is a test.
    S: 
S: . 4.1. MACRO Capability CAPA tag: MACRO Arguments: none Added commands: MDEF Gellens Expires January 2008 [Page 8] Internet Draft Mangle and Macro July 2007 Standard commands affected: All commands valid in TRANSACTION state that take one or more parameters: TOP, RETR, LIST, DELE, UIDL, extension commands Announced states / possible differences: both / no Commands valid in states: TRANSACTION Specification reference: This document Discussion: The MACRO capability indicates the MDEF command is available. See Section 4 for the details of the MDEF command. Note that the CAPA tag (MACRO) differs from the added command (MDEF), which violates the SHOULD in Section 5 of [POP-EXT]. This is unfortunate, but because of an installed base of implementations existing since 2000, fixing it now would be worse than leaving it. 5. IANA Considerations None. 6. Security Considerations Alteration of message text destroys any security wrapper (digital signature) and is useless within encrypted content. However, since the alteration is only for the text sent in direct response to the client, the original unaltered message remains in the message store. Servers need to be careful that macro expansion not exceed buffer sizes. 7. Normative References [KEYWORDS] "Key words for use in RFCs to Indicate Requirement Levels", S. Bradner, RFC 2119, BCP 14, March 1997. [POP3] "Post Office Protocol - Version 3", J. Myers, M. Rose, RFC 1939, STD 53, May 1996. Gellens Expires January 2008 [Page 9] Internet Draft Mangle and Macro July 2007 [POP-EXT] "POP3 Extension Mechanism", R. Gellens, C. Newman, L. Lundblade, RFC 2449, November 1998. 8. Informative References [IMAP] "INTERNET MESSAGE ACCESS PROTOCOL - VERSION 4rev1", M. Crispin, RFC 3501, March 2003. 9. Author's Address Randall Gellens QUALCOMM Incorporated 5775 Morehouse Drive San Diego, CA 92121 rg+ietf@qualcomm.com Gellens Expires January 2008 [Page 10]