.. _retrieving-email:
****************
Retrieving Email
****************
Lasso allows messages to be downloaded from an account on a POP email server.
This enables developers to create solutions such as:
- A list archive for a mailing list
- A webmail interface allowing users to check POP accounts
- An auto-responder that can reply to incoming messages with information
Lasso's flexible POP implementation allows messages to be easily retrieved from
a POP server with a minimal amount of coding. Additionally, Lasso allows the
messages available on the POP server to be inspected without downloading or
deleting them. Mail can be downloaded but left on the server so it can be
checked by other clients (and deleted at a later point if necessary).
All messages are downloaded as raw MIME text. The :type:`email_parse` type can
be used to extract the different parts of the downloaded messages, inspect their
headers, or extract attachments from them.
.. note::
Lasso does not support downloading email via the IMAP protocol.
.. _retrieving-email-pop:
Sending POP Commands
====================
.. index:: POP
The :type:`email_pop` type is used to establish a connection to a POP email
server, inspect the available messages, download one or more messages, and mark
messages for deletion.
POP Methods
-----------
The following describes the :type:`email_pop` type and some of its member
methods:
.. type:: email_pop
.. method:: email_pop(\
-server= ?, \
-port= ?, \
-username= ?, \
-password= ?, \
-APOP= ?, \
-timeout= ?, \
-log= ?, \
-debug= ?, \
-get= ?, \
-host= ?, \
-ssl= ?, \
...)
Creates a new POP connection object. Requires a ``-host`` parameter. Takes
optional ``-port`` and ``-timeout`` parameters. The ``-APOP`` parameter
selects authentication method. If ``-username`` and ``-password`` are
specified then a connection is opened to the server with authentication. The
``-get`` parameter specifies which command to perform when calling
`email_pop->get`.
.. member:: email_pop->size()
Returns the number of messages available for download.
.. member: email_pop->get()
.. member:: email_pop->get(command::string= ?)
Performs the command specified when the object was created. "UniqueID" by
default, or can be set to "Retrieve", "Headers", or "Delete".
.. member: email_pop->retrieve()
.. member:: email_pop->retrieve(position::integer= ?)
.. member:: email_pop->retrieve(position::integer, maxLines::integer)
Retrieves the current message from the server. Optionally accepts a position
to retrieve a specific message. Optional second parameter specifies the
maximum number of lines to fetch for each email.
.. member: email_pop->headers()
.. member:: email_pop->headers(position::integer= ?)
Retrieves the headers of the current message from the server. Optionally
accepts a position to get the headers of a specific message.
.. member: email_pop->uniqueID()
.. member:: email_pop->uniqueID(position::integer= ?)
Retrieves the unique ID of the current message from the server. Optionally
accepts a position to get the unique ID of a specific message.
.. member: email_pop->delete()
.. member:: email_pop->delete(position::integer= ?)
Marks the current message for deletion. Optionally accepts a position to mark
a specific message.
.. member:: email_pop->close()
Closes the POP connection, performing any specified deletes.
.. member:: email_pop->cancel()
Closes the POP connection, but does not perform any deletes.
.. member:: email_pop->noOp()
Sends a ping to the server. Allows the connection to be kept open without
timing out.
.. member:: email_pop->authorize(-username::string, -password::string, -APOP::boolean=true)
Requires a ``-username`` and ``-password`` parameter. Optional ``-APOP``
parameter specifies whether APOP authentication should be used or not. Opens
a connection to the server if one is not already established.
Message Retrieval
-----------------
The :type:`email_pop` type is intended to be used with the `iterate` method to
quickly loop through all available messages on the server. The `email_pop->size`
method returns the number of available messages. The `email_pop->get` method
fetches the "UniqueID" of the current message by default or can be set to
"Retrieve" the current message, the "Headers" of the current message, or even to
"Delete" the current message.
The ``-host``, ``-username``, and ``-password`` should be passed to the
:type:`email_pop` object when it is created. The ``-get`` parameter specifies
what command the `email_pop->get` method will perform. In this case it is set to
"UniqueID" (the default). ::
local(myPOP) = email_pop(
-host = 'mail.example.com',
-username = 'POPUSER',
-password = 'MySecretPassword',
-get = 'UniqueID'
)
The `iterate` method can then be used on the "myPOP" variable. For example, this
code will download and delete every message from the target server. The variable
"myID" is set to the unique ID of each message in turn. The
`email_pop->retrieve` method fetches the current message and the
`email_pop->delete` method marks it for deletion. ::
iterate(#myPOP, local(myID)) => {^
#myID
'
'
#myPOP->retrieve
#myPOP->delete
'
`` tags. :: retrieve ?>Message: [#myID]
[email_parse(#myMsg)]
close ?> :: // => //Message: 000000045280dd26
//Date: Mon 11 Nov 2008 9:0:0 -0500 // From: joe@example.com // To: jane@example.com // Subject: Test // Content-Type: text/plain; charset=ISO-8859-1; format=flowed // Content-Transfer-Encoding: 7bit // // Just Testing ////
Inspect Headers of a Downloaded Message ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ There are three ways to inspect the headers of a downloaded message. #. The basic headers of a message can be inspected using the shortcut methods such as `email_parse->from`, `email_parse->to`, `email_parse->subject`, etc. The following example shows how to display the basic headers for a message, where the variable "myMsg" is assumed to be the output from an `email_pop->retrieve` method:: local(myParse) = email_parse(#myMsg) '
To: ' + #myParse->to->encodeHtml + '\n' '
From: ' + #myParse->from->encodeHtml + '\n' '
Subject: ' + #myParse->subject->encodeHtml + '\n' '
Date: ' + #myParse->date->asString->encodeHtml + '\n' // => //
To: Example Recipient //
From: Example Sender //
Subject: Test Message //
Date: Thu 8 Jul 2004 08:07:42 -0700 These headers can be used in conditionals or other code as well. For example, this conditional would perform different tasks based on whether the message is to one address or another:: local(myParse) = email_parse(#myMsg) if(#myParse->to >> 'mailinglist@example.com') => { // ... store the message in the mailing list database ... else(#myParse->to >> 'help@example.com') // ... forward the message to technical support ... else // ... unknown recipient ... } #. The value for any header, including application-specific headers, headers added by mail processing gateways, etc. can be inspected using the `email_parse->header` method. For example, the following code can check whether the message has SpamAssassin headers:: local(myParse) = email_parse(#myMsg) local(spam_version) = string(#myParse->header('X-Spam-Checker-Version')) local(spam_level) = string(#myParse->header('X-Spam-Level')) local(spam_status) = string(#myParse->header('X-Spam-Status')) '
Spam Version: ' + #spam_version->encodeHtml + '\n' '
Spam Level: ' + #spam_level->encodeHtml + '\n' '
Spam Status: ' + #spam_status->encodeHtml + '\n' // => //
Spam Version: SpamAssassin 2.61 //
Spam Level: //
Spam Status: No, hits=-4.6 required=5.0 tests=AWL,BAYES_00 autolearn=ham The spam status can then be checked with a conditional in order to ignore any messages that have been marked as spam (note that the details will depend on what server-side spam checker is being used and which version). :: if(#spam_status >> 'Yes') => { // ... message is spam ... else // ... message is not spam ... } #. The value for all the headers in the message can be displayed using the `email_parse->headers` method, as the following example shows:: local(myParse) = email_parse(#myMsg) iterate(#myParse->headers, local(header)) '
' + #header->first->encodeHtml + ': ' + #header->second->encodeHtml /iterate // => //
Received: From [127.0.0.1] BY example.com ([127.0.0.1]) WITH ESMTP; // Thu, 08 Jul 2004 08:07:42 -0700 //
Mime-Version: 1.0 //
Content-Type: text/plain; charset=US-ASCII; //
Message-Id: <8F6A8289-D0F0-11D8-B21D-0003936AD948@example.com> //
Content-Transfer-Encoding: 7bit //
From: Example Sender//
Subject: Test Message //
Date: Thu, 8 Jul 2004 08:07:42 -0700 //
To: Example RecipientLocate Parts of a Downloaded Message ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The `email_parse->body` method can be used to find the plain text and HTML parts of a message. The following example shows both the plain text and HTML parts of a downloaded message:: local(myParse) = email_parse(#myMsg) ' ' + #myParse->body(-type='text/plain')->encodeHtml + '' '
' + #myParse->body(-type='text/html')->encodeHtml + '
' The `email_parse->size` and `email_parse->get` methods can be used with the `iterate` method to inspect every part of an email message in turn. This will show information about plain text and HTML parts as well as information about attachments. The headers and body of each part is shown:: local(myParse) = email_parse(#myMsg) iterate(#myParse, local(myPart)) iterate(#myPart->header, local(header)) '
' + #header->first->encodeHtml + ': ' + #header->second->encodeHtml + '\n' /iterate '
' + #myPart->body->encodeHtml + '\n' '
\n' /iterate // => //
Content-Type: text/plain; charset=ISO-8859-1 //
Content-Transfer-Encoding: 8bit //
This is the text part of the email message! //
//
Content-Type: text/html; charset=ISO-8859-1 //
Content-Transfer-Encoding: 8bit //
<html> // <body> // <h3>This is the HTML part of the email message!</h3> // </body> // </html> //
Extract Attachments of a Downloaded Message ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Attachments of a multipart message appear as parts with a :mailheader:`Content-Disposition` of "attachment". The name of the attachment can be found by looking at the "name" field of the :mailheader:`Content-Type` header. The data for the attachment is returned as the body of the part. The attachments can be extracted and written out as files that re-create the attached file, or they can be stored in a database, processed by the `image` methods, or served immediately using `web_response->sendFile`. The following example finds all of the attachments for a message using the `iterate` method to cycle through each part in the message and inspect the :mailheader:`Content-Disposition` header using `email_parse->content_disposition`. The name (``email_parse->content_type('name')``) and data (``email_parse->body``) of each part that includes an attachment is used to write out a file using `file->openWrite` and `file->writeBytes` which re-creates the attachment. :: local(myParse) = email_parse(#myMsg) if(#myParse->mode >> 'multipart') => { iterate(#myParse, local(myPart)) => { if(#myPart->content_disposition >> 'attachment') => { local(myFile) = file('/Attachments/' + #myPart->content_type('name')) local(myFileData) = #myPart->body #myFile->doWithClose => { #myFile->openWrite&writeBytes(#myFileData) } } } } .. note:: In order for this code to work, the "Attachments" folder must already exist and have permissions allowing Lasso Server to write to it. Store a Downloaded Message in a Database ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Messages can be stored in a database in several different ways depending on how the messages are going to be used later. - The simple headers and body of a message can be stored by calling `email_parse->asString` directly in an inline:: local(myPOP) = email_pop( -host = 'mail.example.com', -username = 'POPUSER', -password = 'MySecretPassword' ) handle => { #myPOP->close } iterate(#myPOP, local(myID)) => { local(myMsg) = #myPOP->retrieve local(myParse) = email_parse(#myMsg) inline( -add, -database='example', -table='archive', 'email_format'=#myParse->asString ) => {} } - Often it is desirable to store the common headers of the message in individual fields as well as the different body parts. This example shows how to do this:: local(myPOP) = email_pop( -host = 'mail.example.com', -username = 'POPUSER', -password = 'MySecretPassword' ) handle => { #myPOP->close } iterate(#myPOP, local(myID)) => { local(myMsg) = #myPOP->retrieve local(myParse) = email_parse(#myMsg) inline( -add, -database = 'example', -table = 'archive', 'email_format' = #myParse->asString, 'email_to' = #myParse->to, 'email_from' = #myParse->from, 'email_subject' = #myParse->subject, 'email_date' = #myParse->date, 'email_cc' = #myParse->cc, 'email_text' = #myParse->body(-type='text/plain'), 'email_html' = #myParse->body(-type='text/html') ) => {} } - The raw text of messages can be stored using `email_parse->data`. It is generally recommended that the raw text of a message be stored in addition to a more friendly format. This allows additional information to be extracted from the message later if required. :: local(myPOP) = email_pop( -host = 'mail.example.com', -username = 'POPUSER', -password = 'MySecretPassword' ) handle => { #myPOP->close } iterate(#myPOP, local(myID)) => { local(myMsg) = #myPOP->retrieve local(myParse) = email_parse(#myMsg) inline( -add, -database = 'example', -table = 'archive', 'email_text' = #myParse->asString, 'email_raw' = #myParse->data ) => {} } #myPOP->close Ultimately, the choice of which parts of the email message need to be stored in the database will be solution dependent. Email Helper Methods ==================== The email methods use a number of helper methods for their implementation. The following describes a number of these methods and how they can be used independently. .. method:: email_extract() Strips all comments out of a MIME header. If specified with a ``-comment`` parameter returns the comments instead. Used as a utility method by `email_parse->header`. `email_extract` allows the different parts of email headers to be extracted. Email headers containing email addresses are often formatted in one of the three formats below: .. code-block:: none john@example.com "John Doe"john@example.com (John Doe) In all three of these cases the `email_extract` method will return ":ref:`!john@example.com`". The angle brackets in the second example identify the email address as the important part of the header. The parentheses in the third example identify that portion of the header as a comment. If `email_extract` is called with the optional ``-comment`` parameter then it will return ":ref:`!john@example.com`" for the first example and "John Doe" for the two following examples. .. method:: email_findEmails() Returns an array of all email addresses found in the input. Used as a utility method by `email_parse->recipients`. .. method:: email_safeEmail() This method is used as a utility method by `email_parse->header`. It obscures an email address by returning the comment portion or only the username before the "@" character, and can be used to safely display email headers on the web without attracting email address harvesters. This method returns the following output for the example headers above:: // => // john // John Doe // John Doe .. method:: email_translateBreaksToCRLF() Translates all return characters and line feeds in the input into ``"\r\n"`` pairs.