S. Bosch July 7, 2016 Sieve Email Filtering: Invoking External Programs spec-bosch-sieve-pipe Abstract The Sieve filtering language (RFC 5228) is explicitly designed to be powerful enough to be useful yet limited in order to allow for a safe filtering system. The base specification of the language makes it impossible for users to do anything more complex (and dangerous) than write simple mail filters. One of the consequences of this security- minded design is that users cannot execute programs external to the Sieve filter. However, this can be a very useful and flexible feature for situations where Sieve cannot provide some uncommon functionality by itself. This document updates the Sieve filtering language with extensions that add support for invoking a predefined set of external programs. Messages can be piped to or filtered through those programs and string data can be input to and retrieved from those programs. Table of Contents 1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 2 2. Conventions Used in This Document . . . . . . . . . . . . . . 2 3. Naming of External Programs . . . . . . . . . . . . . . . . . 2 4. Arguments for External Programs . . . . . . . . . . . . . . . 3 5. Action "pipe" . . . . . . . . . . . . . . . . . . . . . . . . 4 5.1. Interactions with Other Sieve Actions . . . . . . . . . . 4 5.2. Interaction with the Sieve "copy" Extension . . . . . . . 5 6. Action "filter" . . . . . . . . . . . . . . . . . . . . . . . 5 6.1. Interaction with Other Tests and Actions . . . . . . . . 6 7. Action "execute" . . . . . . . . . . . . . . . . . . . . . . 6 8. Actions "filter" and "execute" as Tests . . . . . . . . . . . 7 9. Sieve Capability Strings . . . . . . . . . . . . . . . . . . 8 10. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 8 10.1. Example 1 . . . . . . . . . . . . . . . . . . . . . . . 8 10.2. Example 2 . . . . . . . . . . . . . . . . . . . . . . . 9 10.3. Example 3 . . . . . . . . . . . . . . . . . . . . . . . 9 10.4. Example 4 . . . . . . . . . . . . . . . . . . . . . . . 10 11. Security Considerations . . . . . . . . . . . . . . . . . . . 11 12. References . . . . . . . . . . . . . . . . . . . . . . . . . 11 12.1. Normative References . . . . . . . . . . . . . . . . . . 11 12.2. Informative References . . . . . . . . . . . . . . . . . 12 Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 12 Bosch Expires January 8, 2017 [Page 1] Sieve External Programs July 2016 1. Introduction This is an extension to the Sieve filtering language defined by RFC 5228 [SIEVE]. It adds commands for invoking a predefined set of external programs. Messages can be piped to or filtered through those programs and, alternatively, string data can be passed to and retrieved from those programs. The Sieve language is explicitly designed to be powerful enough to be useful yet limited in order to allow for a safe server-side filtering system. Therefore, the base specification of the language makes it impossible for users to do anything more complex (and dangerous) than write simple mail filters. One of the consequences of this security- minded design is that users cannot execute external programs from their Sieve script. Particularly for server-side filtering setups in which mail accounts have no corresponding system account, allowing the execution of arbitrary programs from the mail filter can be a significant security risk. However, such functionality can also be very useful, for instance to easily implement a custom action or external effect that Sieve normally cannot provide. This document updates the Sieve filtering language with an extension to support invoking a predefined set of external programs using a set of new commands. To mitigate the security concerns, the external programs cannot be chosen arbitrarily; the available programs are restricted through administrator configuration. This extension is specific to the Pigeonhole Sieve implementation for the Dovecot Secure IMAP server. It will therefore most likely not be supported by web interfaces and GUI-based Sieve editors. This extension is primarily meant for use in small setups or global scripts that are managed by the system's administrator. 2. Conventions Used in This Document The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [KEYWORDS]. Conventions for notations are as in [SIEVE] Section 1.1, including use of the "Usage:" label for the definition of action and tagged arguments syntax. 3. Naming of External Programs An external program is identified by a name. This MUST not correspond to a file system path or otherwise have the ability to Bosch Expires January 8, 2017 [Page 2] Sieve External Programs July 2016 point to arbitrary programs on the system. The list of valid program names MUST be limited, subject to administrator configuration. A program name is a sequence of Unicode characters encoded in UTF-8 [UTF-8]. A program name MUST comply with Net-Unicode Definition (Section 2 of [NET-UNICODE]), with the additional restriction of prohibiting the following Unicode characters: o 0000-001F; [CONTROL CHARACTERS] o 002F; SLASH o 007F; DELETE o 0080-009F; [CONTROL CHARACTERS] o 2028; LINE SEPARATOR o 2029; PARAGRAPH SEPARATOR Program names MUST be at least one octet (and hence Unicode character) long. Implementations MUST allow names of up to 128 Unicode characters in length (which can take up to 512 octets when encoded in UTF-8, not counting the terminating NUL), and MAY allow longer names. A server that receives a program name longer than its internal limit MUST reject the corresponding operation, in particular it MUST NOT truncate the program name. Implementations MUST NOT allow variables to be expanded into the program names; in other words, the "program-name" value MUST be a constant string as defined in [VARIABLES], Section 3. 4. Arguments for External Programs Optionally, arguments can be passed to an external program. The arguments are specified as a Sieve string list and are passed to the external program in sequence. Implementations SHOULD NOT impose any structure for these arguments; validity checks are the responsibility of the external program. However, implementations SHOULD limit the maximum number of arguments and the length of each argument. Implementations MUST accept at least 16 arguments with a length of at least 1024 octets each, and MAY allow more and longer arguments. Additionally, implementations MAY restrict the use of certain control characters such as CR and LF, if these can cause unexpected behavior or raise security concerns. Bosch Expires January 8, 2017 [Page 3] Sieve External Programs July 2016 Note that implementations MAY also implicitly pass other data, such as the message envelope, to all executed programs avoiding the need to pass this information explicitly through program arguments. 5. Action "pipe" Usage: "pipe" [":try"] [] The "pipe" action executes the external program identified by the "program-name" argument and pipes the message to it. Much like the "fileinto" and "redirect" actions [SIEVE], this action is a disposition-type action (it is intended to deliver the message) and therefore it cancels Sieve's implicit keep (see Section 2.10.2 of [SIEVE]) by default. The specified "program-name" argument MUST conform to the syntax and restrictions defined in Section 3. A script MUST fail with an appropriate error if it attempts to use the "filter" action with an invalid, restricted or unknown program name. The optional "arguments" argument lists the arguments that are passed to the external program, as explained in Section 4. If the external program invoked by the "pipe" action fails to execute or finishes execution with an error, script execution MUST fail with an appropriate error (causing an implicit "keep" action to be executed), unless the ":try" tag is specified. When the ":try" tag is specified, the "pipe" instruction will attempt execution of the external program, but failure will not cause the whole Sieve script execution to fail with an error. Instead, the Sieve processing continues as if the "pipe" action was never triggered. If the execution of the external program is unsuccessful, the "pipe" action MUST NOT cancel the implicit keep. 5.1. Interactions with Other Sieve Actions By default, the "pipe" action cancels the implicit keep, thereby handing the responsibility for the message over to the external program. This behavior can be overridden using the Sieve "copy" extension [RFC3894] as described in Section 5.2. The "pipe" action can only be executed once per script for a particular external program. A script MUST fail with an appropriate error if it attempts to "pipe" messages to the same program multiple times. Bosch Expires January 8, 2017 [Page 4] Sieve External Programs July 2016 The "pipe" action is incompatible with the Sieve "reject" and "ereject" actions [RFC5429]. 5.2. Interaction with the Sieve "copy" Extension The Sieve "copy" extension [RFC3894] adds an optional ":copy" tagged argument to the "fileinto" and "redirect" action commands. When this tag is specified, these commands do not cancel the implicit "keep". Instead, the requested action is performed in addition to whatever else is happening to the message. When the "vnd.dovecot.pipe" extension is active, the "copy" extension also adds the optional ":copy" tag to the "pipe" action command. This has the familiar effect that when the ":copy" tag is specified, the implicit "keep" will not be canceled by the "pipe" action. When the "copy" extension is active, the syntax of the "pipe" action is represented as follows: Usage: "pipe" [":copy"] [":try"] [] 6. Action "filter" Usage: "filter" [] The "filter" action executes the external program identified by the "program-name" argument and filters the message through it. This means that the message is provided as input to the external program and that the output of the external program is used as the new message. This way, the entire message can be altered using the external program. The "filter" action does not affect Sieve's implicit keep. The specified "program-name" argument MUST conform to the syntax and restrictions defined in Section 3. A script MUST fail with an appropriate error if it attempts to use the "filter" action with an invalid, restricted or unknown program name. The optional "arguments" argument lists the arguments that are passed to the external program, as explained in Section 4. If the external program fails to execute, finishes execution with an error, or fails to provide message output, the "filter" action MUST terminate and leave the message unchanged. Depending on the severity of the error, implementations MAY subsequently fail the entire script execution with an appropriate error (causing an implicit "keep" action to be executed). If no error condition is raised, script processing continues, and prior or subsequent "filter" actions are not affected. Bosch Expires January 8, 2017 [Page 5] Sieve External Programs July 2016 6.1. Interaction with Other Tests and Actions A successful "filter" action effectively changes the message, potentially substituting the message in its entirety with a new version. However, actions such as "reject" and "vacation" that generate [MDN], [DSN], or similar disposition messages MUST do so using the original, unmodified message. Similarly, if an error terminates processing of the script, the original message MUST be used when doing the implicit keep required by Section 2.10.6 of [SIEVE]. All other actions that store, send, or alter the message MUST do so with the current version of the message. This includes the "filter" action itself. When a disposition-type action, such as "fileinto", "redirect" or "pipe", is encountered, the current version of the message is "locked in" for that disposition-type action. Whether the implementation performs the action at that point or batches it for later, it MUST perform the action on the message as it stood at the time, and MUST NOT include subsequent changes encountered later in the script processing. In addition, any tests done on the message and its parts will test the message after all prior "filter" actions have been performed. Because the implicit keep, if it is in effect, acts on the final state of the message, all "filter" actions are performed before any implicit keep. The "filter" action does not affect the applicability of other actions; any action that was applicable before the "filter" invocation is equally applicable to the changed message afterward. 7. Action "execute" Usage: "execute" [":input" / ":pipe"] [":output" ] [] The "execute" action executes the external program identified by the "program-name" argument. Input to the program can be provided using the ":input" or ":pipe" tags. If used in combination with the "variables" extension [VARIABLES], the "execute" action can redirect output from the program to the variable specified using the ":output" tag. This way, string data can be passed to and retrieved from an external program. The "execute" action does not change the message in any way and it never affects Sieve's implicit keep. The specified "program-name" argument MUST conform to the syntax and restrictions defined in Section 3. A script MUST fail with an Bosch Expires January 8, 2017 [Page 6] Sieve External Programs July 2016 appropriate error if it attempts to use the "execute" action with an invalid, restricted or unknown program name. The optional "arguments" argument lists the arguments that are passed to the external program, as explained in Section 4. The ":input" and ":pipe" tags are mutually exclusive, because these both specify input that is passed to the external program. Specifying both for a single "execute" command MUST trigger a compile error. The ":input" tag specifies a string that is passed to the external script as input. This string may also contain variable substitutions when the "variables" extension is active. If instead the ":pipe" tag is specified, the current version of the message itself is passed to the external program. If the ":input" and ":pipe" tags are both omitted, no input is provided to the external program. The ":output" tag specifies the variable to which the output of the external program is to be redirected. If the ":output" tag is omitted, any output from the external program is discarded. The ":output" tag requires the "variables" [VARIABLES] extension to be active. The use of the ":output" tag for the "execute" action without the "variables" extension in the require line MUST trigger a compile error. The "varname" parameter of the ":output" tag specifies the name of the variable. It MUST be a constant string and it MUST conform to the syntax of "variable-name" as defined in [VARIABLES], Section 3. An invalid name MUST be detected as a syntax error. The referenced variable MUST be compatible with the "set" command as described in [VARIABLES], Section 4. This means that match variables cannot be specified and that variable namespaces are only allowed when their specification explicitly indicates compatibility with the "set" command. Use of an incompatible variable MUST trigger a compile error. The data actually stored in the variable MAY be truncated to conform to an implementation-specific limit on variable length. If the external program fails to execute or finishes execution with an error, the "execute" action MUST terminate and leave the contents of the variable referenced with ":output" unchanged. Depending on the severity of the error, implementations MAY subsequently fail the entire script execution with an appropriate error (causing an implicit "keep" action to be executed). 8. Actions "filter" and "execute" as Tests To simplify checking the successful invocation of the external program, the "filter" and "execute" actions can also be used as tests. As such, these will attempt to execute the requested external Bosch Expires January 8, 2017 [Page 7] Sieve External Programs July 2016 program, and will evaluate to "true" if the program executed successfully and, if applicable, output was retrieved from it successfully. The usage as a test is exactly the same as the usage as an action: as a test it doubles as an action and a test of the action's result at the same time. For the "execute" test, a "false" result is not necessarily equal to actual failure: it may just mean that the executed program returned a "false" result, e.g. an exit code higher than zero on Unix systems. Note that any output from the external program is discarded when it yields a "false" result. Similarly, for the "filter" test, programs may return a "false" result to indicate that the message was not changed. In that case the Sieve interpreter will not replace the active message with an identical one, which is beneficial for efficiency. The exact semantics of these tests thus partly depends on the program being executed. To handle missing programs gracefully, implementations MAY let the "filter" and "execute" tests evaluate to "false" if an unknown program name is specified, instead of failing the script with an error as would happen if used as an action. In any other case and irrespective of whether the command is used as an action or a test, passing invalid arguments to the "filter" or "execute" commands, such as a syntactically invalid or restricted program name, MUST always cause the script to fail with an appropriate error. 9. Sieve Capability Strings A Sieve implementation that defines the "pipe" action command will advertise the capability string "vnd.dovecot.pipe". A Sieve implementation that defines the "filter" action command will advertise the capability string "vnd.dovecot.filter". A Sieve implementation that defines the "execute" command will advertise the capability string "vnd.dovecot.execute". 10. Examples The examples outlined in this section all refer to some external program. These programs are imaginary and are only available when the administrator would provide them. 10.1. Example 1 The following example passes messages directed to a "user- request@example.com" address to an external program called "request- handler". The "-request" part of the recipient address is identified Bosch Expires January 8, 2017 [Page 8] Sieve External Programs July 2016 using the "subaddress" extension [SUBADDRESS]. If the program is executed successfully, the message is considered delivered and does not end up in the user's inbox. require [ "vnd.dovecot.pipe", "subaddress", "envelope" ]; if envelope :detail "to" "request" { pipe "request-handler"; } 10.2. Example 2 The following example copies messages addressed to a particular recipient to a program called "printer". This program sends the message to some printer. In this case it is configured for "A4" page format and "draft" quality using the two arguments. Irrespective of whether the message is printed or not, it is also always stored in the user's inbox through Sieve's implicit keep action (which is not canceled due to the specified :copy tag). require [ "vnd.dovecot.pipe", "copy" ]; if address "to" "snailmail@example.com" { pipe :copy "printer" ["A4", "draft"]; } 10.3. Example 3 The following example translates a message from Dutch to English if appropriate. If the message's content language is indicated to be Dutch, the message is filtered through an external program called "translator" with arguments that request Dutch to English translation. Dutch messages are translated and filed into a special folder called "Translated". Other messages are delivered to the user's inbox. require [ "vnd.dovecot.filter", "fileinto" ]; if header "content-language" "nl" { filter "translator" ["nl", "en"]; fileinto "Translated"; stop; } Bosch Expires January 8, 2017 [Page 9] Sieve External Programs July 2016 Note that (formerly) Dutch messages are filed into the "Translated" folder, even when the "translator" program fails. In the following modified example this is prevented by using the filter action as a test: require [ "vnd.dovecot.filter", "fileinto" ]; if header "content-language" "nl" { if filter "translator" ["nl", "en"] { fileinto "Translated"; stop; } } This way, messages only end up in the "Translated" folder when translation was actually successful. 10.4. Example 4 The following example determines whether the user is on vacation by querying an external source. The vacation message is obtained from the external source as well. The program that queries the external source is called "onvacation" and it has one argument: the localpart of the recipient address. The execute action is used as a test, which will evaluate to "true" when the user is determined to be on vacation. This means that the external program "onvacation" exits with a failure when the user is not on vacation. Of course, a vacation response is also not sent when the "onvacation" program truly fails somehow. require [ "vnd.dovecot.execute", "vacation", "variables", "envelope" ]; if envelope :localpart :matches "to" "*" { set "recipient" "${1}"; } if execute :output "vacation_message" "onvacation" "${recipient}" { vacation "${vacation_message}"; } Bosch Expires January 8, 2017 [Page 10] Sieve External Programs July 2016 11. Security Considerations Allowing users to execute programs external to the Sieve filter can be a significant security risk, therefore the extensions presented in this specification must be implemented with great care. The external programs should execute with no more privileges than needed. Particularly the arguments passed to the external programs (see Section 4) need to be handled with scrutiny. The external programs need to check the arguments for validity and SHOULD NOT pass these to system tools directly, as this may introduce the possibility of various kinds of insertion attacks. External programs that work with message content or string input from the Sieve script may have similar security concerns. Unlike the Sieve interpreter itself, an external program can easily consume a large amount of resources if not implemented carefully. This can be triggered by coincidence or intentionally by an attacker. Therefore, the amount of resources available to the external programs SHOULD be limited appropriately. For one, external programs MUST NOT be allowed to execute indefinitely. For improved security, implementations MAY restrict the use of this extension to administrator-controlled global Sieve scripts. In such setups, the external programs are never called directly from the user's personal script. For example, using the "include" extension [INCLUDE], the user's personal script can include global scripts that contain the actual external program invocations. This both abstracts the details of external program invocation from the user's view and it limits access to external programs to whatever the administrator defines. 12. References 12.1. Normative References [KEYWORDS] Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, March 1997. [NET-UNICODE] Klensin, J. and M. Padlipsky, "Unicode Format for Network Interchange", RFC 5198, March 2008. [RFC3894] Degener, J., "Sieve Extension: Copying Without Side Effects", RFC 3894, October 2004. Bosch Expires January 8, 2017 [Page 11] Sieve External Programs July 2016 [SIEVE] Guenther, P. and T. Showalter, "Sieve: An Email Filtering Language", RFC 5228, January 2008. [UTF-8] Yergeau, F., "UTF-8, a transformation format of ISO 10646", STD 63, RFC 3629, November 2003. [VARIABLES] Homme, K., "Sieve Email Filtering: Variables Extension", RFC 5229, January 2008. 12.2. Informative References [DSN] Moore, K. and G. Vaudreuil, "An Extensible Message Format for Delivery Status Notifications", RFC 3464, January 2003. [INCLUDE] Daboo, C. and A. Stone, "Sieve Email Filtering: Include Extension", RFC 6609, May 2012. [MDN] Hansen, T. and G. Vaudreuil, "Message Disposition Notification", RFC 3798, May 2004. [RFC5429] Stone, A., "Sieve Email Filtering: Reject and Extended Reject Extensions", RFC 5429, March 2009. [SUBADDRESS] Murchison, K., "Sieve Email Filtering -- Subaddress Extension", RFC 3598, September 2003. Author's Address Stephan Bosch Enschede NL Email: stephan@rename-it.nl Bosch Expires January 8, 2017 [Page 12]